2024-04-23T01:48:50.942 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-23T01:48:50.942 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/teuthology-2024-04-22_22:08:02-orch-reef-distro-default-smithi/7669099 --owner scheduled_teuthology@teuthology --name teuthology-2024-04-22_22:08:02-orch-reef-distro-default-smithi --description orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/teuthology-2024-04-22_22:08:02-orch-reef-distro-default-smithi/7669099/orig.config.yaml 2024-04-23T01:48:50.997 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T01:48:51.084 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-22_22:08:02-orch-reef-distro-default-smithi/7669099 branch: reef description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7669099' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.1 ' name: teuthology-2024-04-22_22:08:02-orch-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 8.stream overrides: admin_socket: branch: reef ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 646b8c7d293d8ec31a40c69f7b3237b19c1be24d ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 646b8c7d293d8ec31a40c69f7b3237b19c1be24d kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: f26c9c3fee57ca330501910de5a07c8769ef5dfc owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 4115 sha1: 646b8c7d293d8ec31a40c69f7b3237b19c1be24d sleep_before_teardown: 0 subset: 43/64 suite: orch suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_f26c9c3fee57ca330501910de5a07c8769ef5dfc/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: f26c9c3fee57ca330501910de5a07c8769ef5dfc targets: smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC/46O7EJx9emMkSwmYqpp1F2tvT/Tqwo/XILMv//tKBiIHFHhqysvkxSnNwn2LIN0NwWzhQD18T3OZ5K2G93DM= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3ZX0Tutjw6eGl1DG6COb2Z2KaK2K8u3uGyFZnA0zyQWVay9mv/guBJ3hZASCoFU0u5Ucav7d8UZsBgU3rznNU= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: exclude_packages: - ceph-volume tag: v18.2.1 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.1 roleless: true - print: '**** done end installing v18.2.1 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-22_22:08:02 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-04-23T01:48:51.084 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_f26c9c3fee57ca330501910de5a07c8769ef5dfc/qa; will attempt to use it 2024-04-23T01:48:51.085 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_f26c9c3fee57ca330501910de5a07c8769ef5dfc/qa/tasks 2024-04-23T01:48:51.085 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-23T01:48:51.087 INFO:teuthology.task.internal:Checking packages... 2024-04-23T01:48:51.111 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '646b8c7d293d8ec31a40c69f7b3237b19c1be24d' 2024-04-23T01:48:51.112 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-23T01:48:51.112 INFO:teuthology.packaging:ref: None 2024-04-23T01:48:51.112 INFO:teuthology.packaging:tag: None 2024-04-23T01:48:51.112 INFO:teuthology.packaging:branch: reef 2024-04-23T01:48:51.112 INFO:teuthology.packaging:sha1: 646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T01:48:51.112 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2024-04-23T01:48:51.416 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-0 2024-04-23T01:48:51.418 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-23T01:48:51.424 INFO:teuthology.task.internal:no buildpackages task found 2024-04-23T01:48:51.424 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-23T01:48:51.448 INFO:teuthology.task.internal:Saving configuration 2024-04-23T01:48:51.468 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-23T01:48:51.474 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T01:48:51.499 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-22_22:08:02-orch-reef-distro-default-smithi/7669099', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 01:43:34.202514', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC/46O7EJx9emMkSwmYqpp1F2tvT/Tqwo/XILMv//tKBiIHFHhqysvkxSnNwn2LIN0NwWzhQD18T3OZ5K2G93DM='} 2024-04-23T01:48:51.520 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-22_22:08:02-orch-reef-distro-default-smithi/7669099', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 01:43:34.200782', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3ZX0Tutjw6eGl1DG6COb2Z2KaK2K8u3uGyFZnA0zyQWVay9mv/guBJ3hZASCoFU0u5Ucav7d8UZsBgU3rznNU='} 2024-04-23T01:48:51.520 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-23T01:48:51.526 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-04-23T01:48:51.526 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-04-23T01:48:51.526 INFO:teuthology.run_tasks:Running task console_log... 2024-04-23T01:48:51.600 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe6e2a670d0>, signals=[15]) 2024-04-23T01:48:51.601 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-23T01:48:51.607 INFO:teuthology.task.internal:Opening connections... 2024-04-23T01:48:51.607 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2024-04-23T01:48:51.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T01:48:51.686 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2024-04-23T01:48:51.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T01:48:51.759 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-23T01:48:51.766 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2024-04-23T01:48:51.788 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2024-04-23T01:48:51.788 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2024-04-23T01:48:51.848 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Stream" 2024-04-23T01:48:51.848 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="8" 2024-04-23T01:48:51.848 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2024-04-23T01:48:51.848 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2024-04-23T01:48:51.848 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="8" 2024-04-23T01:48:51.848 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el8" 2024-04-23T01:48:51.848 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-23T01:48:51.848 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2024-04-23T01:48:51.848 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-23T01:48:51.848 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2024-04-23T01:48:51.848 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T01:48:51.849 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-23T01:48:51.849 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T01:48:51.850 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2024-04-23T01:48:51.870 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2024-04-23T01:48:51.892 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2024-04-23T01:48:51.892 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2024-04-23T01:48:51.953 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2024-04-23T01:48:51.953 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="8" 2024-04-23T01:48:51.953 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2024-04-23T01:48:51.953 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2024-04-23T01:48:51.953 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="8" 2024-04-23T01:48:51.953 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el8" 2024-04-23T01:48:51.953 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-23T01:48:51.953 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2024-04-23T01:48:51.953 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-23T01:48:51.953 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2024-04-23T01:48:51.953 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T01:48:51.954 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-23T01:48:51.954 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T01:48:51.954 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2024-04-23T01:48:51.978 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-23T01:48:51.997 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-23T01:48:52.002 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-23T01:48:52.002 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2024-04-23T01:48:52.005 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2024-04-23T01:48:52.024 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-23T01:48:52.031 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-23T01:48:52.032 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2024-04-23T01:48:52.065 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2024-04-23T01:48:52.099 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-23T01:48:52.153 INFO:teuthology.run_tasks:Running task kernel... 2024-04-23T01:48:52.168 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-23T01:48:52.168 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-23T01:48:52.169 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T01:48:52.169 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T01:48:52.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T01:48:52.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T01:48:52.189 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-04-23T01:48:52.189 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-04-23T01:48:52.252 INFO:teuthology.orchestra.run.smithi192.stdout:4.18.0-489.el8.x86_64 2024-04-23T01:48:52.253 INFO:teuthology.task.kernel:Running kernel on smithi192: 4.18.0-489.el8.x86_64 2024-04-23T01:48:52.253 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum install -y kernel 2024-04-23T01:48:52.255 INFO:teuthology.orchestra.run.smithi071.stdout:4.18.0-489.el8.x86_64 2024-04-23T01:48:52.255 INFO:teuthology.task.kernel:Running kernel on smithi071: 4.18.0-489.el8.x86_64 2024-04-23T01:48:52.255 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum install -y kernel 2024-04-23T01:48:56.942 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 8 - AppStream 13 MB/s | 28 MB 00:02 2024-04-23T01:48:57.000 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 8 - AppStream 14 MB/s | 28 MB 00:02 2024-04-23T01:49:03.359 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 8 - BaseOS 7.9 MB/s | 10 MB 00:01 2024-04-23T01:49:04.117 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 8 - BaseOS 5.0 MB/s | 10 MB 00:02 2024-04-23T01:49:05.561 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 8 - Extras 32 kB/s | 18 kB 00:00 2024-04-23T01:49:05.927 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 8 - Extras common packages 30 kB/s | 7.7 kB 00:00 2024-04-23T01:49:06.170 INFO:teuthology.orchestra.run.smithi192.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-04-23T01:49:06.242 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 8 - Extras 33 kB/s | 18 kB 00:00 2024-04-23T01:49:06.701 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 8 - Extras common packages 22 kB/s | 7.7 kB 00:00 2024-04-23T01:49:07.061 INFO:teuthology.orchestra.run.smithi071.stdout:Copr repo for python3-asyncssh owned by ceph 13 kB/s | 3.5 kB 00:00 2024-04-23T01:49:07.884 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2024-04-23T01:49:09.365 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 7.4 MB/s | 16 MB 00:02 2024-04-23T01:49:11.687 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 403 kB/s | 24 kB 00:00 2024-04-23T01:49:13.145 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 510 kB/s | 24 kB 00:00 2024-04-23T01:49:14.759 INFO:teuthology.orchestra.run.smithi192.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-04-23T01:49:14.759 INFO:teuthology.orchestra.run.smithi192.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-04-23T01:49:14.886 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T01:49:14.887 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T01:49:14.887 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-04-23T01:49:14.887 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T01:49:14.887 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-23T01:49:14.887 INFO:teuthology.orchestra.run.smithi192.stdout: kernel x86_64 4.18.0-552.el8 baseos 10 M 2024-04-23T01:49:14.887 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-04-23T01:49:14.887 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-core x86_64 4.18.0-552.el8 baseos 44 M 2024-04-23T01:49:14.887 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules x86_64 4.18.0-552.el8 baseos 36 M 2024-04-23T01:49:14.887 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:49:14.888 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-23T01:49:14.888 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T01:49:14.888 INFO:teuthology.orchestra.run.smithi192.stdout:Install 3 Packages 2024-04-23T01:49:14.888 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:49:14.888 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 90 M 2024-04-23T01:49:14.888 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 96 M 2024-04-23T01:49:14.889 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-23T01:49:16.167 INFO:teuthology.orchestra.run.smithi071.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-04-23T01:49:16.167 INFO:teuthology.orchestra.run.smithi071.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-04-23T01:49:16.270 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T01:49:16.271 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T01:49:16.271 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-04-23T01:49:16.271 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T01:49:16.272 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-23T01:49:16.272 INFO:teuthology.orchestra.run.smithi071.stdout: kernel x86_64 4.18.0-552.el8 baseos 10 M 2024-04-23T01:49:16.272 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-23T01:49:16.272 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-core x86_64 4.18.0-552.el8 baseos 44 M 2024-04-23T01:49:16.272 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules x86_64 4.18.0-552.el8 baseos 36 M 2024-04-23T01:49:16.272 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:49:16.272 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-23T01:49:16.272 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T01:49:16.272 INFO:teuthology.orchestra.run.smithi071.stdout:Install 3 Packages 2024-04-23T01:49:16.272 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:49:16.272 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 90 M 2024-04-23T01:49:16.272 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 96 M 2024-04-23T01:49:16.272 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-23T01:49:17.672 INFO:teuthology.orchestra.run.smithi192.stdout:(1/3): kernel-4.18.0-552.el8.x86_64.rpm 3.8 MB/s | 10 MB 00:02 2024-04-23T01:49:23.046 INFO:teuthology.orchestra.run.smithi071.stdout:(1/3): kernel-4.18.0-552.el8.x86_64.rpm 1.5 MB/s | 10 MB 00:06 2024-04-23T01:49:25.054 INFO:teuthology.orchestra.run.smithi071.stdout:(2/3): kernel-modules-4.18.0-552.el8.x86_64.rpm 4.2 MB/s | 36 MB 00:08 2024-04-23T01:49:25.348 INFO:teuthology.orchestra.run.smithi192.stdout:(2/3): kernel-modules-4.18.0-552.el8.x86_64.rpm 3.5 MB/s | 36 MB 00:10 2024-04-23T01:49:27.713 INFO:teuthology.orchestra.run.smithi192.stdout:(3/3): kernel-core-4.18.0-552.el8.x86_64.rpm 3.4 MB/s | 44 MB 00:12 2024-04-23T01:49:27.713 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:49:27.713 INFO:teuthology.orchestra.run.smithi192.stdout:Total 7.1 MB/s | 90 MB 00:12 2024-04-23T01:49:28.402 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-23T01:49:28.839 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-23T01:49:28.839 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-23T01:49:30.089 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-23T01:49:30.125 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-23T01:49:32.943 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-23T01:49:33.541 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-23T01:49:37.171 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-23T01:49:37.558 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-23T01:49:40.992 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-23T01:49:41.265 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-4.18.0-552.el8.x86_64 3/3 2024-04-23T01:50:01.257 INFO:teuthology.orchestra.run.smithi071.stdout:(3/3): kernel-core-4.18.0-552.el8.x86_64.rpm 996 kB/s | 44 MB 00:44 2024-04-23T01:50:01.257 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:50:01.257 INFO:teuthology.orchestra.run.smithi071.stdout:Total 2.0 MB/s | 90 MB 00:44 2024-04-23T01:50:01.940 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-23T01:50:02.387 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-23T01:50:02.388 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-23T01:50:03.025 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 3/3 2024-04-23T01:50:03.050 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-23T01:50:03.443 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-4.18.0-552.el8.x86_64 3/3 2024-04-23T01:50:03.443 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-4.18.0-552.el8.x86_64 1/3 2024-04-23T01:50:03.443 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-core-4.18.0-552.el8.x86_64 2/3 2024-04-23T01:50:03.657 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-23T01:50:03.693 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-23T01:50:03.930 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-23T01:50:03.930 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:50:03.930 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-23T01:50:03.930 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-4.18.0-552.el8.x86_64 kernel-core-4.18.0-552.el8.x86_64 2024-04-23T01:50:03.931 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-4.18.0-552.el8.x86_64 2024-04-23T01:50:03.931 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:50:03.931 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T01:50:04.079 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q kernel | sort -rV | head -n 1 2024-04-23T01:50:04.300 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-23T01:50:04.300 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-23T01:50:04.300 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-552.el8.x86_64 2024-04-23T01:50:04.300 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-23T01:50:04.300 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-23T01:50:04.301 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-04-23T01:50:04.301 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi192.front.sepia.ceph.com, path=None, version=distro) 2024-04-23T01:50:04.301 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q kernel | sort -rV | head -n 1 2024-04-23T01:50:04.522 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-23T01:50:04.523 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-23T01:50:04.523 DEBUG:teuthology.orchestra.run.smithi192:> sudo rpm -qi grub2-tools 2024-04-23T01:50:04.556 INFO:teuthology.orchestra.run.smithi192.stdout:Name : grub2-tools 2024-04-23T01:50:04.557 INFO:teuthology.orchestra.run.smithi192.stdout:Epoch : 1 2024-04-23T01:50:04.557 INFO:teuthology.orchestra.run.smithi192.stdout:Version : 2.02 2024-04-23T01:50:04.557 INFO:teuthology.orchestra.run.smithi192.stdout:Release : 129.el8 2024-04-23T01:50:04.557 INFO:teuthology.orchestra.run.smithi192.stdout:Architecture: x86_64 2024-04-23T01:50:04.557 INFO:teuthology.orchestra.run.smithi192.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-04-23T01:50:04.557 INFO:teuthology.orchestra.run.smithi192.stdout:Group : System Environment/Base 2024-04-23T01:50:04.557 INFO:teuthology.orchestra.run.smithi192.stdout:Size : 9265586 2024-04-23T01:50:04.557 INFO:teuthology.orchestra.run.smithi192.stdout:License : GPLv3+ 2024-04-23T01:50:04.557 INFO:teuthology.orchestra.run.smithi192.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-04-23T01:50:04.557 INFO:teuthology.orchestra.run.smithi192.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-04-23T01:50:04.557 INFO:teuthology.orchestra.run.smithi192.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-04-23T01:50:04.557 INFO:teuthology.orchestra.run.smithi192.stdout:Build Host : kbuilder.bsys.centos.org 2024-04-23T01:50:04.557 INFO:teuthology.orchestra.run.smithi192.stdout:Relocations : (not relocatable) 2024-04-23T01:50:04.557 INFO:teuthology.orchestra.run.smithi192.stdout:Packager : CentOS BuildSystem 2024-04-23T01:50:04.558 INFO:teuthology.orchestra.run.smithi192.stdout:Vendor : CentOS 2024-04-23T01:50:04.558 INFO:teuthology.orchestra.run.smithi192.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-23T01:50:04.558 INFO:teuthology.orchestra.run.smithi192.stdout:Summary : Support tools for GRUB. 2024-04-23T01:50:04.558 INFO:teuthology.orchestra.run.smithi192.stdout:Description : 2024-04-23T01:50:04.558 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:50:04.558 INFO:teuthology.orchestra.run.smithi192.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-23T01:50:04.558 INFO:teuthology.orchestra.run.smithi192.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-23T01:50:04.558 INFO:teuthology.orchestra.run.smithi192.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-23T01:50:04.558 INFO:teuthology.orchestra.run.smithi192.stdout:hardware devices. 2024-04-23T01:50:04.558 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:50:04.558 INFO:teuthology.orchestra.run.smithi192.stdout:This subpackage provides tools for support of all platforms. 2024-04-23T01:50:04.559 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-23T01:50:04.559 INFO:teuthology.task.kernel:Updating grub on smithi192 to boot 4.18.0-552.el8.x86_64 2024-04-23T01:50:04.559 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-23T01:50:06.006 INFO:teuthology.orchestra.run.smithi192.stderr:Generating grub configuration file ... 2024-04-23T01:50:06.507 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-23T01:50:07.097 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-23T01:50:08.075 INFO:teuthology.orchestra.run.smithi192.stderr:done 2024-04-23T01:50:08.077 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-04-23T01:50:08.102 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.mQvnWywV8L 2024-04-23T01:50:08.103 DEBUG:teuthology.orchestra.run.smithi192:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.mQvnWywV8L 2024-04-23T01:50:08.198 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.mQvnWywV8L 2024-04-23T01:50:08.307 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.mQvnWywV8L is 6KB 2024-04-23T01:50:08.354 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.mQvnWywV8L 2024-04-23T01:50:08.372 DEBUG:teuthology.orchestra.run.smithi192:> sudo /bin/ls /boot/loader/entries || true 2024-04-23T01:50:08.456 INFO:teuthology.orchestra.run.smithi192.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-04-23T01:50:08.456 INFO:teuthology.orchestra.run.smithi192.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-04-23T01:50:08.457 INFO:teuthology.orchestra.run.smithi192.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-04-23T01:50:08.457 INFO:teuthology.orchestra.run.smithi192.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64.conf 2024-04-23T01:50:08.458 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64 2024-04-23T01:50:08.610 DEBUG:teuthology.orchestra.run.smithi192:> sudo shutdown -r now 2024-04-23T01:50:10.667 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-23T01:50:11.079 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-23T01:50:14.509 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-23T01:50:14.792 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-4.18.0-552.el8.x86_64 3/3 2024-04-23T01:50:36.650 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 3/3 2024-04-23T01:50:36.675 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-23T01:50:37.493 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-4.18.0-552.el8.x86_64 3/3 2024-04-23T01:50:37.493 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-4.18.0-552.el8.x86_64 1/3 2024-04-23T01:50:37.493 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-core-4.18.0-552.el8.x86_64 2/3 2024-04-23T01:50:38.020 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-23T01:50:38.020 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:50:38.021 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-23T01:50:38.021 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-4.18.0-552.el8.x86_64 kernel-core-4.18.0-552.el8.x86_64 2024-04-23T01:50:38.021 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-4.18.0-552.el8.x86_64 2024-04-23T01:50:38.021 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:50:38.021 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T01:50:38.199 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q kernel | sort -rV | head -n 1 2024-04-23T01:50:38.460 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-23T01:50:38.460 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-23T01:50:38.460 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-552.el8.x86_64 2024-04-23T01:50:38.460 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-23T01:50:38.461 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-23T01:50:38.461 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-04-23T01:50:38.461 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi071.front.sepia.ceph.com, path=None, version=distro) 2024-04-23T01:50:38.461 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q kernel | sort -rV | head -n 1 2024-04-23T01:50:38.613 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-04-23T01:50:38.613 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-04-23T01:50:38.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T01:50:38.670 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-23T01:50:38.670 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-23T01:50:38.670 DEBUG:teuthology.orchestra.run.smithi071:> sudo rpm -qi grub2-tools 2024-04-23T01:50:38.703 INFO:teuthology.orchestra.run.smithi071.stdout:Name : grub2-tools 2024-04-23T01:50:38.704 INFO:teuthology.orchestra.run.smithi071.stdout:Epoch : 1 2024-04-23T01:50:38.704 INFO:teuthology.orchestra.run.smithi071.stdout:Version : 2.02 2024-04-23T01:50:38.704 INFO:teuthology.orchestra.run.smithi071.stdout:Release : 129.el8 2024-04-23T01:50:38.704 INFO:teuthology.orchestra.run.smithi071.stdout:Architecture: x86_64 2024-04-23T01:50:38.704 INFO:teuthology.orchestra.run.smithi071.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-04-23T01:50:38.704 INFO:teuthology.orchestra.run.smithi071.stdout:Group : System Environment/Base 2024-04-23T01:50:38.704 INFO:teuthology.orchestra.run.smithi071.stdout:Size : 9265586 2024-04-23T01:50:38.704 INFO:teuthology.orchestra.run.smithi071.stdout:License : GPLv3+ 2024-04-23T01:50:38.704 INFO:teuthology.orchestra.run.smithi071.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-04-23T01:50:38.704 INFO:teuthology.orchestra.run.smithi071.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-04-23T01:50:38.704 INFO:teuthology.orchestra.run.smithi071.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-04-23T01:50:38.704 INFO:teuthology.orchestra.run.smithi071.stdout:Build Host : kbuilder.bsys.centos.org 2024-04-23T01:50:38.704 INFO:teuthology.orchestra.run.smithi071.stdout:Relocations : (not relocatable) 2024-04-23T01:50:38.704 INFO:teuthology.orchestra.run.smithi071.stdout:Packager : CentOS BuildSystem 2024-04-23T01:50:38.705 INFO:teuthology.orchestra.run.smithi071.stdout:Vendor : CentOS 2024-04-23T01:50:38.705 INFO:teuthology.orchestra.run.smithi071.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-23T01:50:38.705 INFO:teuthology.orchestra.run.smithi071.stdout:Summary : Support tools for GRUB. 2024-04-23T01:50:38.705 INFO:teuthology.orchestra.run.smithi071.stdout:Description : 2024-04-23T01:50:38.705 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:50:38.705 INFO:teuthology.orchestra.run.smithi071.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-23T01:50:38.705 INFO:teuthology.orchestra.run.smithi071.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-23T01:50:38.705 INFO:teuthology.orchestra.run.smithi071.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-23T01:50:38.705 INFO:teuthology.orchestra.run.smithi071.stdout:hardware devices. 2024-04-23T01:50:38.705 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:50:38.705 INFO:teuthology.orchestra.run.smithi071.stdout:This subpackage provides tools for support of all platforms. 2024-04-23T01:50:38.706 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-23T01:50:38.706 INFO:teuthology.task.kernel:Updating grub on smithi071 to boot 4.18.0-552.el8.x86_64 2024-04-23T01:50:38.706 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-23T01:50:40.070 INFO:teuthology.orchestra.run.smithi071.stderr:Generating grub configuration file ... 2024-04-23T01:50:42.072 INFO:teuthology.orchestra.run.smithi071.stderr:done 2024-04-23T01:50:42.073 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-04-23T01:50:42.105 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.0DpoCrm2VH 2024-04-23T01:50:42.105 DEBUG:teuthology.orchestra.run.smithi071:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.0DpoCrm2VH 2024-04-23T01:50:42.178 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.0DpoCrm2VH 2024-04-23T01:50:42.322 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.0DpoCrm2VH is 6KB 2024-04-23T01:50:42.370 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.0DpoCrm2VH 2024-04-23T01:50:42.387 DEBUG:teuthology.orchestra.run.smithi071:> sudo /bin/ls /boot/loader/entries || true 2024-04-23T01:50:42.472 INFO:teuthology.orchestra.run.smithi071.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-04-23T01:50:42.473 INFO:teuthology.orchestra.run.smithi071.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-04-23T01:50:42.473 INFO:teuthology.orchestra.run.smithi071.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-04-23T01:50:42.473 INFO:teuthology.orchestra.run.smithi071.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64.conf 2024-04-23T01:50:42.474 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64 2024-04-23T01:50:42.589 DEBUG:teuthology.orchestra.run.smithi071:> sudo shutdown -r now 2024-04-23T01:51:12.620 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-04-23T01:51:12.621 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-23T01:51:12.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T01:51:38.643 DEBUG:teuthology.orchestra.remote:timed out 2024-04-23T01:51:47.651 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-04-23T01:51:47.652 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T01:51:48.107 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-04-23T01:51:48.394 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-04-23T01:51:48.394 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "4.18.0-552.el8.x86_64"... 2024-04-23T01:51:48.394 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-04-23T01:51:48.411 INFO:teuthology.orchestra.run.smithi192.stdout:4.18.0-552.el8.x86_64 2024-04-23T01:51:48.412 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-552.el8.x86_64 vs 4.18.0-552.el8.x86_64 2024-04-23T01:51:48.412 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-23T01:51:48.412 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-23T01:51:49.413 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-04-23T01:51:49.413 DEBUG:teuthology.orchestra.run.smithi192:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-23T01:51:49.635 INFO:teuthology.orchestra.run.smithi192.stdout:ttyS1 2024-04-23T01:51:49.664 DEBUG:teuthology.parallel:result is None 2024-04-23T01:52:12.624 DEBUG:teuthology.orchestra.remote:timed out 2024-04-23T01:52:21.625 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-23T01:52:21.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T01:52:21.993 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-04-23T01:52:22.371 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-23T01:52:22.371 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "4.18.0-552.el8.x86_64"... 2024-04-23T01:52:22.372 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-04-23T01:52:22.389 INFO:teuthology.orchestra.run.smithi071.stdout:4.18.0-552.el8.x86_64 2024-04-23T01:52:22.389 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-552.el8.x86_64 vs 4.18.0-552.el8.x86_64 2024-04-23T01:52:22.389 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-23T01:52:22.389 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-23T01:52:23.390 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-04-23T01:52:23.390 DEBUG:teuthology.orchestra.run.smithi071:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-23T01:52:23.462 INFO:teuthology.orchestra.run.smithi071.stdout:ttyS1 2024-04-23T01:52:23.481 DEBUG:teuthology.parallel:result is None 2024-04-23T01:52:23.482 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-23T01:52:23.490 INFO:teuthology.task.internal:Creating test directory... 2024-04-23T01:52:23.491 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T01:52:23.493 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T01:52:23.516 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-23T01:52:23.523 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-23T01:52:23.533 INFO:teuthology.task.internal:Creating archive directory... 2024-04-23T01:52:23.533 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T01:52:23.551 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T01:52:23.587 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-23T01:52:23.594 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-23T01:52:23.594 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T01:52:23.609 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T01:52:23.650 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T01:52:23.660 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T01:52:23.661 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T01:52:23.672 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T01:52:23.674 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-23T01:52:23.684 INFO:teuthology.task.internal:Configuring sudo... 2024-04-23T01:52:23.684 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T01:52:23.703 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T01:52:23.743 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-23T01:52:23.753 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-23T01:52:23.753 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T01:52:23.771 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T01:52:23.803 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T01:52:23.875 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T01:52:23.932 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T01:52:23.974 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T01:52:24.016 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T01:52:24.016 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T01:52:24.082 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T01:52:24.130 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T01:52:24.189 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T01:52:24.274 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T01:52:24.321 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-23T01:52:24.321 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T01:52:24.389 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2024-04-23T01:52:24.391 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2024-04-23T01:52:24.455 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T01:52:24.498 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T01:52:24.886 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-23T01:52:24.895 INFO:teuthology.task.internal:Starting timer... 2024-04-23T01:52:24.895 INFO:teuthology.run_tasks:Running task pcp... 2024-04-23T01:52:24.904 INFO:teuthology.run_tasks:Running task selinux... 2024-04-23T01:52:24.912 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-04-23T01:52:24.913 DEBUG:teuthology.orchestra.run.smithi071:> sudo service auditd rotate 2024-04-23T01:52:25.053 INFO:teuthology.orchestra.run.smithi071.stdout:Rotating logs: 2024-04-23T01:52:25.055 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2024-04-23T01:52:25.144 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2024-04-23T01:52:25.146 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-23T01:52:25.147 DEBUG:teuthology.orchestra.run.smithi071:> /usr/sbin/getenforce 2024-04-23T01:52:25.172 INFO:teuthology.orchestra.run.smithi071.stdout:Permissive 2024-04-23T01:52:25.173 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2024-04-23T01:52:25.228 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2024-04-23T01:52:25.228 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi071.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2024-04-23T01:52:25.229 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-23T01:52:25.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T01:52:25.256 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-23T01:52:25.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T01:52:25.284 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-23T01:52:25.284 DEBUG:teuthology.orchestra.run.smithi071:> sudo /usr/sbin/setenforce permissive 2024-04-23T01:52:25.321 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2024-04-23T01:52:25.351 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-23T01:52:25.361 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-23T01:52:25.366 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-23T01:52:25.490 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-23T01:52:25.507 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-23T01:52:25.508 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi071.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-23T01:58:29.610 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi071.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2024-04-23T01:58:29.612 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-23T01:58:29.613 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T01:58:29.688 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-04-23T01:58:29.761 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-23T01:58:29.761 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-04-23T01:58:29.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T01:58:29.836 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-04-23T01:58:29.907 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-04-23T01:58:29.908 INFO:teuthology.run_tasks:Running task clock... 2024-04-23T01:58:29.920 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-23T01:58:29.920 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T01:58:29.920 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T01:58:29.923 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T01:58:29.923 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T01:58:29.952 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T01:58:29.966 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T01:58:29.982 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T01:58:29.997 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T01:58:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: ntpd: command not found 2024-04-23T01:58:30.046 INFO:teuthology.orchestra.run.smithi071.stdout:506 Cannot talk to daemon 2024-04-23T01:58:30.059 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T01:58:30.072 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T01:58:30.073 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2024-04-23T01:58:30.084 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2024-04-23T01:58:30.098 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T01:58:30.111 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T01:58:30.132 INFO:teuthology.orchestra.run.smithi071.stderr:bash: ntpq: command not found 2024-04-23T01:58:30.136 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T01:58:30.136 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-04-23T01:58:30.136 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T01:58:30.136 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T01:58:30.136 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T01:58:30.136 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T01:58:30.180 INFO:teuthology.orchestra.run.smithi192.stderr:bash: ntpq: command not found 2024-04-23T01:58:30.184 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T01:58:30.185 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-04-23T01:58:30.185 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T01:58:30.185 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T01:58:30.185 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T01:58:30.185 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T01:58:30.185 INFO:teuthology.run_tasks:Running task pexec... 2024-04-23T01:58:30.195 INFO:teuthology.task.pexec:Executing custom commands... 2024-04-23T01:58:30.195 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi071.front.sepia.ceph.com 2024-04-23T01:58:30.195 DEBUG:teuthology.orchestra.run.smithi071:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-04-23T01:58:30.196 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi192.front.sepia.ceph.com 2024-04-23T01:58:30.196 DEBUG:teuthology.orchestra.run.smithi192:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-04-23T01:58:30.959 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:23 ago on Tue 23 Apr 2024 01:57:07 AM UTC. 2024-04-23T01:58:30.988 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:29 ago on Tue 23 Apr 2024 01:57:01 AM UTC. 2024-04-23T01:58:33.105 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T01:58:33.106 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T01:58:33.106 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-04-23T01:58:33.106 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T01:58:33.106 INFO:teuthology.orchestra.run.smithi071.stdout:Resetting modules: 2024-04-23T01:58:33.106 INFO:teuthology.orchestra.run.smithi071.stdout: container-tools 2024-04-23T01:58:33.106 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:58:33.106 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-23T01:58:33.106 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T01:58:33.106 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:58:33.121 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T01:58:33.122 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T01:58:33.122 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-04-23T01:58:33.122 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T01:58:33.123 INFO:teuthology.orchestra.run.smithi192.stdout:Resetting modules: 2024-04-23T01:58:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: container-tools 2024-04-23T01:58:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:58:33.123 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-23T01:58:33.123 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T01:58:33.123 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:58:33.362 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T01:58:33.383 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T01:58:34.138 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:26 ago on Tue 23 Apr 2024 01:57:07 AM UTC. 2024-04-23T01:58:34.164 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:32 ago on Tue 23 Apr 2024 01:57:01 AM UTC. 2024-04-23T01:58:36.282 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T01:58:36.289 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T01:58:36.319 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T01:58:36.319 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-04-23T01:58:36.320 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T01:58:36.320 INFO:teuthology.orchestra.run.smithi192.stdout:Installing group/module packages: 2024-04-23T01:58:36.320 INFO:teuthology.orchestra.run.smithi192.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-04-23T01:58:36.320 INFO:teuthology.orchestra.run.smithi192.stdout: buildah x86_64 2:1.33.6-2.module_el8+976+11360731 appstream 9.6 M 2024-04-23T01:58:36.320 INFO:teuthology.orchestra.run.smithi192.stdout: cockpit-podman noarch 84.1-1.module_el8+977+8ab54275 appstream 689 k 2024-04-23T01:58:36.320 INFO:teuthology.orchestra.run.smithi192.stdout: crun x86_64 1.14.3-2.module_el8+968+fbb249c7 appstream 258 k 2024-04-23T01:58:36.320 INFO:teuthology.orchestra.run.smithi192.stdout: netavark x86_64 2:1.10.3-1.module_el8+951+32019cde appstream 4.1 M 2024-04-23T01:58:36.320 INFO:teuthology.orchestra.run.smithi192.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-04-23T01:58:36.320 INFO:teuthology.orchestra.run.smithi192.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-04-23T01:58:36.320 INFO:teuthology.orchestra.run.smithi192.stdout: toolbox x86_64 0.0.99.5-2.module_el8+971+3d3df00d appstream 2.5 M 2024-04-23T01:58:36.320 INFO:teuthology.orchestra.run.smithi192.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-04-23T01:58:36.320 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-04-23T01:58:36.321 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-04-23T01:58:36.321 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-04-23T01:58:36.321 INFO:teuthology.orchestra.run.smithi192.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-04-23T01:58:36.321 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2024-04-23T01:58:36.321 INFO:teuthology.orchestra.run.smithi192.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-04-23T01:58:36.321 INFO:teuthology.orchestra.run.smithi192.stdout:Installing module profiles: 2024-04-23T01:58:36.321 INFO:teuthology.orchestra.run.smithi192.stdout: container-tools/common 2024-04-23T01:58:36.321 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:58:36.321 INFO:teuthology.orchestra.run.smithi192.stdout:Enabling module streams: 2024-04-23T01:58:36.321 INFO:teuthology.orchestra.run.smithi192.stdout: container-tools rhel8 2024-04-23T01:58:36.322 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:58:36.322 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-23T01:58:36.322 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T01:58:36.322 INFO:teuthology.orchestra.run.smithi192.stdout:Install 13 Packages 2024-04-23T01:58:36.322 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:58:36.323 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 27 M 2024-04-23T01:58:36.323 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 91 M 2024-04-23T01:58:36.324 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-23T01:58:36.326 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T01:58:36.326 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-23T01:58:36.326 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T01:58:36.326 INFO:teuthology.orchestra.run.smithi071.stdout:Installing group/module packages: 2024-04-23T01:58:36.327 INFO:teuthology.orchestra.run.smithi071.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-04-23T01:58:36.327 INFO:teuthology.orchestra.run.smithi071.stdout: buildah x86_64 2:1.33.6-2.module_el8+976+11360731 appstream 9.6 M 2024-04-23T01:58:36.327 INFO:teuthology.orchestra.run.smithi071.stdout: cockpit-podman noarch 84.1-1.module_el8+977+8ab54275 appstream 689 k 2024-04-23T01:58:36.327 INFO:teuthology.orchestra.run.smithi071.stdout: crun x86_64 1.14.3-2.module_el8+968+fbb249c7 appstream 258 k 2024-04-23T01:58:36.327 INFO:teuthology.orchestra.run.smithi071.stdout: netavark x86_64 2:1.10.3-1.module_el8+951+32019cde appstream 4.1 M 2024-04-23T01:58:36.327 INFO:teuthology.orchestra.run.smithi071.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-04-23T01:58:36.328 INFO:teuthology.orchestra.run.smithi071.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-04-23T01:58:36.328 INFO:teuthology.orchestra.run.smithi071.stdout: toolbox x86_64 0.0.99.5-2.module_el8+971+3d3df00d appstream 2.5 M 2024-04-23T01:58:36.328 INFO:teuthology.orchestra.run.smithi071.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-04-23T01:58:36.328 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-23T01:58:36.328 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-04-23T01:58:36.328 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-04-23T01:58:36.329 INFO:teuthology.orchestra.run.smithi071.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-04-23T01:58:36.329 INFO:teuthology.orchestra.run.smithi071.stdout:Installing weak dependencies: 2024-04-23T01:58:36.329 INFO:teuthology.orchestra.run.smithi071.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-04-23T01:58:36.329 INFO:teuthology.orchestra.run.smithi071.stdout:Installing module profiles: 2024-04-23T01:58:36.329 INFO:teuthology.orchestra.run.smithi071.stdout: container-tools/common 2024-04-23T01:58:36.329 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:58:36.330 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling module streams: 2024-04-23T01:58:36.330 INFO:teuthology.orchestra.run.smithi071.stdout: container-tools rhel8 2024-04-23T01:58:36.330 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:58:36.330 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-23T01:58:36.330 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T01:58:36.330 INFO:teuthology.orchestra.run.smithi071.stdout:Install 13 Packages 2024-04-23T01:58:36.330 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:58:36.331 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 27 M 2024-04-23T01:58:36.331 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 91 M 2024-04-23T01:58:36.331 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-23T01:58:36.576 INFO:teuthology.orchestra.run.smithi071.stdout:(1/13): cockpit-podman-84.1-1.module_el8+977+8a 3.5 MB/s | 689 kB 00:00 2024-04-23T01:58:36.617 INFO:teuthology.orchestra.run.smithi192.stdout:(1/13): cockpit-podman-84.1-1.module_el8+977+8a 2.8 MB/s | 689 kB 00:00 2024-04-23T01:58:36.618 INFO:teuthology.orchestra.run.smithi071.stdout:(2/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 910 kB/s | 38 kB 00:00 2024-04-23T01:58:36.658 INFO:teuthology.orchestra.run.smithi192.stdout:(2/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 910 kB/s | 38 kB 00:00 2024-04-23T01:58:36.677 INFO:teuthology.orchestra.run.smithi071.stdout:(3/13): crun-1.14.3-2.module_el8+968+fbb249c7.x 4.3 MB/s | 258 kB 00:00 2024-04-23T01:58:36.684 INFO:teuthology.orchestra.run.smithi192.stdout:(3/13): aardvark-dns-1.10.0-1.module_el8+871+55 3.3 MB/s | 1.0 MB 00:00 2024-04-23T01:58:36.735 INFO:teuthology.orchestra.run.smithi071.stdout:(4/13): aardvark-dns-1.10.0-1.module_el8+871+55 2.8 MB/s | 1.0 MB 00:00 2024-04-23T01:58:36.767 INFO:teuthology.orchestra.run.smithi192.stdout:(4/13): crun-1.14.3-2.module_el8+968+fbb249c7.x 2.3 MB/s | 258 kB 00:00 2024-04-23T01:58:36.786 INFO:teuthology.orchestra.run.smithi071.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 3.5 MB/s | 180 kB 00:00 2024-04-23T01:58:36.826 INFO:teuthology.orchestra.run.smithi192.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 3.0 MB/s | 180 kB 00:00 2024-04-23T01:58:36.827 INFO:teuthology.orchestra.run.smithi071.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 597 kB/s | 25 kB 00:00 2024-04-23T01:58:36.868 INFO:teuthology.orchestra.run.smithi192.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 597 kB/s | 25 kB 00:00 2024-04-23T01:58:36.869 INFO:teuthology.orchestra.run.smithi071.stdout:(7/13): python3-pyxdg-0.25-16.el8.noarch.rpm 2.2 MB/s | 94 kB 00:00 2024-04-23T01:58:36.910 INFO:teuthology.orchestra.run.smithi192.stdout:(7/13): python3-pyxdg-0.25-16.el8.noarch.rpm 2.2 MB/s | 94 kB 00:00 2024-04-23T01:58:37.295 INFO:teuthology.orchestra.run.smithi071.stdout:(8/13): netavark-1.10.3-1.module_el8+951+32019c 6.6 MB/s | 4.1 MB 00:00 2024-04-23T01:58:37.402 INFO:teuthology.orchestra.run.smithi192.stdout:(8/13): netavark-1.10.3-1.module_el8+951+32019c 5.7 MB/s | 4.1 MB 00:00 2024-04-23T01:58:37.670 INFO:teuthology.orchestra.run.smithi071.stdout:(9/13): toolbox-0.0.99.5-2.module_el8+971+3d3df 6.7 MB/s | 2.5 MB 00:00 2024-04-23T01:58:37.712 INFO:teuthology.orchestra.run.smithi071.stdout:(10/13): udica-0.2.6-20.module_el8+804+f131391c 1.2 MB/s | 53 kB 00:00 2024-04-23T01:58:37.762 INFO:teuthology.orchestra.run.smithi071.stdout:(11/13): yajl-2.1.0-12.el8.x86_64.rpm 816 kB/s | 41 kB 00:00 2024-04-23T01:58:37.819 INFO:teuthology.orchestra.run.smithi192.stdout:(9/13): toolbox-0.0.99.5-2.module_el8+971+3d3df 6.1 MB/s | 2.5 MB 00:00 2024-04-23T01:58:37.861 INFO:teuthology.orchestra.run.smithi192.stdout:(10/13): udica-0.2.6-20.module_el8+804+f131391c 1.2 MB/s | 53 kB 00:00 2024-04-23T01:58:37.902 INFO:teuthology.orchestra.run.smithi192.stdout:(11/13): yajl-2.1.0-12.el8.x86_64.rpm 983 kB/s | 41 kB 00:00 2024-04-23T01:58:38.021 INFO:teuthology.orchestra.run.smithi071.stdout:(12/13): buildah-1.33.6-2.module_el8+976+113607 5.9 MB/s | 9.6 MB 00:01 2024-04-23T01:58:38.078 INFO:teuthology.orchestra.run.smithi192.stdout:(12/13): buildah-1.33.6-2.module_el8+976+113607 5.7 MB/s | 9.6 MB 00:01 2024-04-23T01:58:38.787 INFO:teuthology.orchestra.run.smithi192.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 4.7 MB/s | 8.8 MB 00:01 2024-04-23T01:58:38.787 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:58:38.787 INFO:teuthology.orchestra.run.smithi192.stdout:Total 11 MB/s | 27 MB 00:02 2024-04-23T01:58:38.946 INFO:teuthology.orchestra.run.smithi071.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 4.2 MB/s | 8.8 MB 00:02 2024-04-23T01:58:38.946 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:58:38.947 INFO:teuthology.orchestra.run.smithi071.stdout:Total 10 MB/s | 27 MB 00:02 2024-04-23T01:58:39.057 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-23T01:58:39.078 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-23T01:58:39.079 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-23T01:58:39.218 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-23T01:58:39.239 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-23T01:58:39.239 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-23T01:58:39.422 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-23T01:58:39.425 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-23T01:58:39.499 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-23T01:58:39.503 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-23T01:58:40.153 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-23T01:58:40.343 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-23T01:58:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-04-23T01:58:41.319 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-04-23T01:58:41.358 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-04-23T01:58:41.521 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-04-23T01:58:41.528 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-04-23T01:58:41.717 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-04-23T01:58:41.758 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-04-23T01:58:41.964 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-04-23T01:58:41.980 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-04-23T01:58:42.177 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-04-23T01:58:42.512 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-04-23T01:58:42.701 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-04-23T01:58:43.516 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 7/13 2024-04-23T01:58:43.725 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 7/13 2024-04-23T01:58:43.792 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 8/13 2024-04-23T01:58:44.006 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 8/13 2024-04-23T01:58:44.006 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 9/13 2024-04-23T01:58:44.208 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 9/13 2024-04-23T01:58:44.409 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-04-23T01:58:44.600 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-04-23T01:58:44.665 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-04-23T01:58:44.810 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-04-23T01:58:44.929 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-04-23T01:58:45.035 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-04-23T01:58:45.136 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-04-23T01:58:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-04-23T01:58:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-04-23T01:58:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-04-23T01:58:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2/13 2024-04-23T01:58:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 3/13 2024-04-23T01:58:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-04-23T01:58:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 5/13 2024-04-23T01:58:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 6/13 2024-04-23T01:58:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-04-23T01:58:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-04-23T01:58:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-04-23T01:58:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-04-23T01:58:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-04-23T01:58:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-04-23T01:58:45.787 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-04-23T01:58:45.787 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-04-23T01:58:45.787 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2/13 2024-04-23T01:58:45.788 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 3/13 2024-04-23T01:58:45.788 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-04-23T01:58:45.788 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 5/13 2024-04-23T01:58:45.788 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 6/13 2024-04-23T01:58:45.788 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-04-23T01:58:45.788 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-04-23T01:58:45.788 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-04-23T01:58:45.788 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-04-23T01:58:45.788 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-04-23T01:58:45.788 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-04-23T01:58:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-04-23T01:58:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:58:46.040 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-23T01:58:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-04-23T01:58:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2024-04-23T01:58:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: cockpit-podman-84.1-1.module_el8+977+8ab54275.noarch 2024-04-23T01:58:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-04-23T01:58:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 2024-04-23T01:58:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 2024-04-23T01:58:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-04-23T01:58:46.040 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-04-23T01:58:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-04-23T01:58:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-04-23T01:58:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 2024-04-23T01:58:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-04-23T01:58:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: yajl-2.1.0-12.el8.x86_64 2024-04-23T01:58:46.041 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:58:46.041 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T01:58:46.277 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-04-23T01:58:46.277 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:58:46.277 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-23T01:58:46.277 INFO:teuthology.orchestra.run.smithi192.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-04-23T01:58:46.277 INFO:teuthology.orchestra.run.smithi192.stdout: buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2024-04-23T01:58:46.277 INFO:teuthology.orchestra.run.smithi192.stdout: cockpit-podman-84.1-1.module_el8+977+8ab54275.noarch 2024-04-23T01:58:46.277 INFO:teuthology.orchestra.run.smithi192.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-04-23T01:58:46.277 INFO:teuthology.orchestra.run.smithi192.stdout: crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 2024-04-23T01:58:46.277 INFO:teuthology.orchestra.run.smithi192.stdout: netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 2024-04-23T01:58:46.277 INFO:teuthology.orchestra.run.smithi192.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-04-23T01:58:46.278 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-04-23T01:58:46.278 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-04-23T01:58:46.278 INFO:teuthology.orchestra.run.smithi192.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-04-23T01:58:46.278 INFO:teuthology.orchestra.run.smithi192.stdout: toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 2024-04-23T01:58:46.278 INFO:teuthology.orchestra.run.smithi192.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-04-23T01:58:46.278 INFO:teuthology.orchestra.run.smithi192.stdout: yajl-2.1.0-12.el8.x86_64 2024-04-23T01:58:46.278 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:58:46.278 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T01:58:46.350 DEBUG:teuthology.parallel:result is None 2024-04-23T01:58:46.579 DEBUG:teuthology.parallel:result is None 2024-04-23T01:58:46.580 INFO:teuthology.run_tasks:Running task install... 2024-04-23T01:58:46.589 DEBUG:teuthology.task.install:project ceph 2024-04-23T01:58:46.589 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d'}} 2024-04-23T01:58:46.589 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.1', 'flavor': 'default', 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d'} 2024-04-23T01:58:46.589 INFO:teuthology.task.install:Using flavor: default 2024-04-23T01:58:46.598 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-23T01:58:46.598 INFO:teuthology.task.install:extra packages: [] 2024-04-23T01:58:46.598 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-04-23T01:58:46.599 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-23T01:58:46.599 INFO:teuthology.packaging:ref: None 2024-04-23T01:58:46.599 INFO:teuthology.packaging:tag: v18.2.1 2024-04-23T01:58:46.599 INFO:teuthology.packaging:branch: None 2024-04-23T01:58:46.599 INFO:teuthology.packaging:sha1: 646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T01:58:48.990 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph v18.2.1^{} -> 7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-23T01:58:48.991 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-23T01:58:48.994 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-04-23T01:58:48.994 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-23T01:58:48.994 INFO:teuthology.packaging:ref: None 2024-04-23T01:58:48.994 INFO:teuthology.packaging:tag: v18.2.1 2024-04-23T01:58:48.994 INFO:teuthology.packaging:branch: None 2024-04-23T01:58:48.994 INFO:teuthology.packaging:sha1: 646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T01:58:48.995 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-23T01:58:49.226 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/8/flavors/default/ 2024-04-23T01:58:49.226 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-04-23T01:58:49.239 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/8/flavors/default/ 2024-04-23T01:58:49.240 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-04-23T01:58:49.273 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T01:58:49.273 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T01:58:49.274 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T01:58:49.287 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T01:58:49.287 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-23T01:58:49.287 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T01:58:49.307 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-23T01:58:49.307 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-23T01:58:49.307 INFO:teuthology.packaging:ref: None 2024-04-23T01:58:49.307 INFO:teuthology.packaging:tag: v18.2.1 2024-04-23T01:58:49.307 INFO:teuthology.packaging:branch: None 2024-04-23T01:58:49.307 INFO:teuthology.packaging:sha1: 646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T01:58:49.308 DEBUG:teuthology.orchestra.run.smithi071:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T01:58:49.321 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-23T01:58:49.322 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-23T01:58:49.322 INFO:teuthology.packaging:ref: None 2024-04-23T01:58:49.322 INFO:teuthology.packaging:tag: v18.2.1 2024-04-23T01:58:49.322 INFO:teuthology.packaging:branch: None 2024-04-23T01:58:49.322 INFO:teuthology.packaging:sha1: 646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T01:58:49.322 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T01:58:49.379 DEBUG:teuthology.orchestra.run.smithi071:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T01:58:49.394 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T01:58:49.460 DEBUG:teuthology.orchestra.run.smithi071:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T01:58:49.474 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T01:58:49.528 INFO:teuthology.orchestra.run.smithi071.stdout:check_obsoletes = 1 2024-04-23T01:58:49.530 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum clean all 2024-04-23T01:58:49.542 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2024-04-23T01:58:49.543 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2024-04-23T01:58:49.798 INFO:teuthology.orchestra.run.smithi071.stdout:78 files removed 2024-04-23T01:58:49.816 INFO:teuthology.orchestra.run.smithi192.stdout:78 files removed 2024-04-23T01:58:49.826 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-radosgw 2024-04-23T01:58:49.844 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw 2024-04-23T01:58:50.336 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 959 kB/s | 79 kB 00:00 2024-04-23T01:58:50.337 INFO:teuthology.orchestra.run.smithi071.stdout:ceph packages for x86_64 718 kB/s | 79 kB 00:00 2024-04-23T01:58:50.446 INFO:teuthology.orchestra.run.smithi071.stdout:ceph noarch packages 135 kB/s | 9.4 kB 00:00 2024-04-23T01:58:50.453 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 120 kB/s | 9.4 kB 00:00 2024-04-23T01:58:50.546 INFO:teuthology.orchestra.run.smithi071.stdout:ceph source packages 25 kB/s | 1.8 kB 00:00 2024-04-23T01:58:50.582 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 18 kB/s | 1.8 kB 00:00 2024-04-23T01:58:54.989 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 8 - AppStream 6.3 MB/s | 28 MB 00:04 2024-04-23T01:58:55.145 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 8 - AppStream 6.2 MB/s | 28 MB 00:04 2024-04-23T01:59:02.739 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 8 - BaseOS 3.8 MB/s | 10 MB 00:02 2024-04-23T01:59:02.821 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 8 - BaseOS 3.8 MB/s | 10 MB 00:02 2024-04-23T01:59:05.430 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 8 - Extras 18 kB/s | 18 kB 00:00 2024-04-23T01:59:05.906 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 8 - Extras 12 kB/s | 18 kB 00:01 2024-04-23T01:59:05.914 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 8 - Extras common packages 21 kB/s | 7.7 kB 00:00 2024-04-23T01:59:06.240 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 8 - Extras common packages 34 kB/s | 7.7 kB 00:00 2024-04-23T01:59:07.073 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 8 - PowerTools 4.6 MB/s | 5.0 MB 00:01 2024-04-23T01:59:08.133 INFO:teuthology.orchestra.run.smithi192.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2024-04-23T01:59:08.640 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 8 - PowerTools 2.1 MB/s | 5.0 MB 00:02 2024-04-23T01:59:09.708 INFO:teuthology.orchestra.run.smithi071.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2024-04-23T01:59:10.350 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 7.7 MB/s | 16 MB 00:02 2024-04-23T01:59:11.691 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 8.6 MB/s | 16 MB 00:01 2024-04-23T01:59:14.142 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 437 kB/s | 24 kB 00:00 2024-04-23T01:59:15.493 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 375 kB/s | 24 kB 00:00 2024-04-23T01:59:17.902 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T01:59:17.905 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T01:59:17.905 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-04-23T01:59:17.905 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T01:59:17.905 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-23T01:59:17.905 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:18.2.1-0.el8 ceph 13 M 2024-04-23T01:59:17.905 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2024-04-23T01:59:17.905 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:18.2.1-0.el8 ceph 3.7 M 2024-04-23T01:59:17.905 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:18.2.1-0.el8 ceph 3.6 M 2024-04-23T01:59:17.905 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-04-23T01:59:17.905 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:18.2.1-0.el8 ceph 5.4 M 2024-04-23T01:59:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:18.2.1-0.el8 ceph 22 M 2024-04-23T01:59:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:18.2.1-0.el8 ceph 24 k 2024-04-23T01:59:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-23T01:59:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:18.2.1-0.el8 ceph 774 k 2024-04-23T01:59:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-23T01:59:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-04-23T01:59:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:18.2.1-0.el8 ceph 474 k 2024-04-23T01:59:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-04-23T01:59:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:18.2.1-0.el8 ceph 6.5 M 2024-04-23T01:59:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-23T01:59:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-04-23T01:59:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el8 ceph 45 k 2024-04-23T01:59:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:18.2.1-0.el8 ceph 123 k 2024-04-23T01:59:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:18.2.1-0.el8 ceph 197 k 2024-04-23T01:59:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-04-23T01:59:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:18.2.1-0.el8 ceph 371 k 2024-04-23T01:59:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:18.2.1-0.el8 ceph 365 k 2024-04-23T01:59:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:18.2.1-0.el8 ceph 107 k 2024-04-23T01:59:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-04-23T01:59:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:59:17.907 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-23T01:59:17.907 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T01:59:17.907 INFO:teuthology.orchestra.run.smithi192.stdout:Install 21 Packages 2024-04-23T01:59:17.907 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 2 Packages 2024-04-23T01:59:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:59:17.908 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 59 M 2024-04-23T01:59:17.908 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-23T01:59:18.311 INFO:teuthology.orchestra.run.smithi192.stdout:(1/23): ceph-base-18.2.1-0.el8.x86_64.rpm 43 MB/s | 5.4 MB 00:00 2024-04-23T01:59:18.336 INFO:teuthology.orchestra.run.smithi192.stdout:(2/23): ceph-selinux-18.2.1-0.el8.x86_64.rpm 979 kB/s | 24 kB 00:00 2024-04-23T01:59:18.378 INFO:teuthology.orchestra.run.smithi192.stdout:(3/23): libcephfs2-18.2.1-0.el8.x86_64.rpm 18 MB/s | 774 kB 00:00 2024-04-23T01:59:18.412 INFO:teuthology.orchestra.run.smithi192.stdout:(4/23): libradosstriper1-18.2.1-0.el8.x86_64.rp 14 MB/s | 474 kB 00:00 2024-04-23T01:59:18.545 INFO:teuthology.orchestra.run.smithi192.stdout:(5/23): librgw2-18.2.1-0.el8.x86_64.rpm 49 MB/s | 6.5 MB 00:00 2024-04-23T01:59:18.570 INFO:teuthology.orchestra.run.smithi192.stdout:(6/23): python3-ceph-argparse-18.2.1-0.el8.x86_ 1.8 MB/s | 45 kB 00:00 2024-04-23T01:59:18.596 INFO:teuthology.orchestra.run.smithi192.stdout:(7/23): python3-ceph-common-18.2.1-0.el8.x86_64 4.8 MB/s | 123 kB 00:00 2024-04-23T01:59:18.621 INFO:teuthology.orchestra.run.smithi192.stdout:(8/23): python3-cephfs-18.2.1-0.el8.x86_64.rpm 7.7 MB/s | 197 kB 00:00 2024-04-23T01:59:18.646 INFO:teuthology.orchestra.run.smithi192.stdout:(9/23): python3-rados-18.2.1-0.el8.x86_64.rpm 15 MB/s | 371 kB 00:00 2024-04-23T01:59:18.671 INFO:teuthology.orchestra.run.smithi192.stdout:(10/23): python3-rbd-18.2.1-0.el8.x86_64.rpm 14 MB/s | 365 kB 00:00 2024-04-23T01:59:18.697 INFO:teuthology.orchestra.run.smithi192.stdout:(11/23): python3-rgw-18.2.1-0.el8.x86_64.rpm 4.2 MB/s | 107 kB 00:00 2024-04-23T01:59:18.864 INFO:teuthology.orchestra.run.smithi192.stdout:(12/23): ceph-radosgw-18.2.1-0.el8.x86_64.rpm 19 MB/s | 13 MB 00:00 2024-04-23T01:59:19.114 INFO:teuthology.orchestra.run.smithi192.stdout:(13/23): ceph-common-18.2.1-0.el8.x86_64.rpm 23 MB/s | 22 MB 00:00 2024-04-23T01:59:19.198 INFO:teuthology.orchestra.run.smithi192.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 530 kB/s | 44 kB 00:00 2024-04-23T01:59:19.290 INFO:teuthology.orchestra.run.smithi192.stdout:(15/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 514 kB/s | 47 kB 00:00 2024-04-23T01:59:19.302 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T01:59:19.305 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T01:59:19.305 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-23T01:59:19.306 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T01:59:19.306 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-23T01:59:19.306 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw x86_64 2:18.2.1-0.el8 ceph 13 M 2024-04-23T01:59:19.306 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrading: 2024-04-23T01:59:19.306 INFO:teuthology.orchestra.run.smithi071.stdout: librados2 x86_64 2:18.2.1-0.el8 ceph 3.7 M 2024-04-23T01:59:19.307 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1 x86_64 2:18.2.1-0.el8 ceph 3.6 M 2024-04-23T01:59:19.307 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-23T01:59:19.307 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base x86_64 2:18.2.1-0.el8 ceph 5.4 M 2024-04-23T01:59:19.307 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common x86_64 2:18.2.1-0.el8 ceph 22 M 2024-04-23T01:59:19.307 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux x86_64 2:18.2.1-0.el8 ceph 24 k 2024-04-23T01:59:19.307 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-23T01:59:19.307 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2 x86_64 2:18.2.1-0.el8 ceph 774 k 2024-04-23T01:59:19.308 INFO:teuthology.orchestra.run.smithi071.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-23T01:59:19.308 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-04-23T01:59:19.308 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1 x86_64 2:18.2.1-0.el8 ceph 474 k 2024-04-23T01:59:19.308 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-04-23T01:59:19.308 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2 x86_64 2:18.2.1-0.el8 ceph 6.5 M 2024-04-23T01:59:19.308 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-23T01:59:19.308 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-04-23T01:59:19.309 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el8 ceph 45 k 2024-04-23T01:59:19.309 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common x86_64 2:18.2.1-0.el8 ceph 123 k 2024-04-23T01:59:19.309 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs x86_64 2:18.2.1-0.el8 ceph 197 k 2024-04-23T01:59:19.309 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-04-23T01:59:19.309 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados x86_64 2:18.2.1-0.el8 ceph 371 k 2024-04-23T01:59:19.309 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd x86_64 2:18.2.1-0.el8 ceph 365 k 2024-04-23T01:59:19.309 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw x86_64 2:18.2.1-0.el8 ceph 107 k 2024-04-23T01:59:19.309 INFO:teuthology.orchestra.run.smithi071.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-04-23T01:59:19.310 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:59:19.310 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-23T01:59:19.310 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T01:59:19.310 INFO:teuthology.orchestra.run.smithi071.stdout:Install 21 Packages 2024-04-23T01:59:19.310 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrade 2 Packages 2024-04-23T01:59:19.310 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:59:19.311 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 59 M 2024-04-23T01:59:19.311 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-23T01:59:19.315 INFO:teuthology.orchestra.run.smithi192.stdout:(16/23): librdkafka-1.6.1-1.el8.x86_64.rpm 1.1 MB/s | 677 kB 00:00 2024-04-23T01:59:19.365 INFO:teuthology.orchestra.run.smithi192.stdout:(17/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 518 kB/s | 259 kB 00:00 2024-04-23T01:59:19.607 INFO:teuthology.orchestra.run.smithi192.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 201 kB/s | 59 kB 00:00 2024-04-23T01:59:19.657 INFO:teuthology.orchestra.run.smithi192.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 266 kB/s | 75 kB 00:00 2024-04-23T01:59:19.750 INFO:teuthology.orchestra.run.smithi192.stdout:(20/23): librados2-18.2.1-0.el8.x86_64.rpm 40 MB/s | 3.7 MB 00:00 2024-04-23T01:59:19.775 INFO:teuthology.orchestra.run.smithi192.stdout:(21/23): gperftools-libs-2.7-9.el8.x86_64.rpm 632 kB/s | 306 kB 00:00 2024-04-23T01:59:19.850 INFO:teuthology.orchestra.run.smithi192.stdout:(22/23): librbd1-18.2.1-0.el8.x86_64.rpm 36 MB/s | 3.6 MB 00:00 2024-04-23T01:59:19.894 INFO:teuthology.orchestra.run.smithi071.stdout:(1/23): ceph-base-18.2.1-0.el8.x86_64.rpm 41 MB/s | 5.4 MB 00:00 2024-04-23T01:59:19.912 INFO:teuthology.orchestra.run.smithi071.stdout:(2/23): ceph-selinux-18.2.1-0.el8.x86_64.rpm 1.4 MB/s | 24 kB 00:00 2024-04-23T01:59:19.944 INFO:teuthology.orchestra.run.smithi071.stdout:(3/23): libcephfs2-18.2.1-0.el8.x86_64.rpm 23 MB/s | 774 kB 00:00 2024-04-23T01:59:19.978 INFO:teuthology.orchestra.run.smithi071.stdout:(4/23): libradosstriper1-18.2.1-0.el8.x86_64.rp 14 MB/s | 474 kB 00:00 2024-04-23T01:59:20.067 INFO:teuthology.orchestra.run.smithi192.stdout:(23/23): thrift-0.13.0-2.el8.x86_64.rpm 3.8 MB/s | 1.7 MB 00:00 2024-04-23T01:59:20.071 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:59:20.071 INFO:teuthology.orchestra.run.smithi192.stdout:Total 27 MB/s | 59 MB 00:02 2024-04-23T01:59:20.117 INFO:teuthology.orchestra.run.smithi071.stdout:(5/23): librgw2-18.2.1-0.el8.x86_64.rpm 49 MB/s | 6.5 MB 00:00 2024-04-23T01:59:20.119 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-23T01:59:20.129 INFO:teuthology.orchestra.run.smithi071.stdout:(6/23): python3-ceph-argparse-18.2.1-0.el8.x86_ 2.7 MB/s | 45 kB 00:00 2024-04-23T01:59:20.155 INFO:teuthology.orchestra.run.smithi071.stdout:(7/23): python3-ceph-common-18.2.1-0.el8.x86_64 4.8 MB/s | 123 kB 00:00 2024-04-23T01:59:20.162 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-23T01:59:20.162 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-23T01:59:20.288 INFO:teuthology.orchestra.run.smithi071.stdout:(8/23): ceph-radosgw-18.2.1-0.el8.x86_64.rpm 25 MB/s | 13 MB 00:00 2024-04-23T01:59:20.313 INFO:teuthology.orchestra.run.smithi071.stdout:(9/23): python3-cephfs-18.2.1-0.el8.x86_64.rpm 1.2 MB/s | 197 kB 00:00 2024-04-23T01:59:20.338 INFO:teuthology.orchestra.run.smithi071.stdout:(10/23): python3-rados-18.2.1-0.el8.x86_64.rpm 7.2 MB/s | 371 kB 00:00 2024-04-23T01:59:20.363 INFO:teuthology.orchestra.run.smithi071.stdout:(11/23): python3-rbd-18.2.1-0.el8.x86_64.rpm 7.1 MB/s | 365 kB 00:00 2024-04-23T01:59:20.414 INFO:teuthology.orchestra.run.smithi071.stdout:(12/23): python3-rgw-18.2.1-0.el8.x86_64.rpm 1.4 MB/s | 107 kB 00:00 2024-04-23T01:59:20.518 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-23T01:59:20.521 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-23T01:59:20.647 INFO:teuthology.orchestra.run.smithi071.stdout:(13/23): ceph-common-18.2.1-0.el8.x86_64.rpm 25 MB/s | 22 MB 00:00 2024-04-23T01:59:20.689 INFO:teuthology.orchestra.run.smithi071.stdout:(14/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 943 kB/s | 259 kB 00:00 2024-04-23T01:59:20.765 INFO:teuthology.orchestra.run.smithi071.stdout:(15/23): python3-prettytable-0.7.2-14.el8.noarc 376 kB/s | 44 kB 00:00 2024-04-23T01:59:20.790 INFO:teuthology.orchestra.run.smithi071.stdout:(16/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 469 kB/s | 47 kB 00:00 2024-04-23T01:59:20.815 INFO:teuthology.orchestra.run.smithi071.stdout:(17/23): librdkafka-1.6.1-1.el8.x86_64.rpm 1.5 MB/s | 677 kB 00:00 2024-04-23T01:59:21.099 INFO:teuthology.orchestra.run.smithi071.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 190 kB/s | 59 kB 00:00 2024-04-23T01:59:21.133 INFO:teuthology.orchestra.run.smithi071.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 237 kB/s | 75 kB 00:00 2024-04-23T01:59:21.208 INFO:teuthology.orchestra.run.smithi071.stdout:(20/23): librados2-18.2.1-0.el8.x86_64.rpm 49 MB/s | 3.7 MB 00:00 2024-04-23T01:59:21.233 INFO:teuthology.orchestra.run.smithi071.stdout:(21/23): gperftools-libs-2.7-9.el8.x86_64.rpm 654 kB/s | 306 kB 00:00 2024-04-23T01:59:21.300 INFO:teuthology.orchestra.run.smithi071.stdout:(22/23): librbd1-18.2.1-0.el8.x86_64.rpm 39 MB/s | 3.6 MB 00:00 2024-04-23T01:59:21.575 INFO:teuthology.orchestra.run.smithi071.stdout:(23/23): thrift-0.13.0-2.el8.x86_64.rpm 3.6 MB/s | 1.7 MB 00:00 2024-04-23T01:59:21.579 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:59:21.580 INFO:teuthology.orchestra.run.smithi071.stdout:Total 26 MB/s | 59 MB 00:02 2024-04-23T01:59:21.621 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-23T01:59:21.667 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-23T01:59:21.667 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-23T01:59:21.802 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-23T01:59:22.025 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-23T01:59:22.028 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-23T01:59:22.043 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2024-04-23T01:59:22.298 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2024-04-23T01:59:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2024-04-23T01:59:22.910 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2024-04-23T01:59:23.153 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:18.2.1-0.el8.x86_64 4/25 2024-04-23T01:59:23.218 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-23T01:59:23.234 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:18.2.1-0.el8.x86_64 4/25 2024-04-23T01:59:23.434 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2024-04-23T01:59:23.493 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:18.2.1-0.el8.x86_64 5/25 2024-04-23T01:59:23.689 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2024-04-23T01:59:23.711 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:18.2.1-0.el8.x86_64 6/25 2024-04-23T01:59:23.911 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2024-04-23T01:59:24.106 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el8.x86_64 6/25 2024-04-23T01:59:24.311 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:18.2.1-0.el8.x86_64 7/25 2024-04-23T01:59:24.365 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2024-04-23T01:59:24.372 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:18.2.1-0.el8.x86_64 7/25 2024-04-23T01:59:24.561 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librados2-2:18.2.1-0.el8.x86_64 4/25 2024-04-23T01:59:24.612 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/25 2024-04-23T01:59:24.693 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:18.2.1-0.el8.x86_64 4/25 2024-04-23T01:59:24.793 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/25 2024-04-23T01:59:24.925 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rados-2:18.2.1-0.el8.x86_64 5/25 2024-04-23T01:59:24.981 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el8.x86_64 10/25 2024-04-23T01:59:25.086 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs2-2:18.2.1-0.el8.x86_64 6/25 2024-04-23T01:59:25.167 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:18.2.1-0.el8.x86_64 11/25 2024-04-23T01:59:25.378 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:18.2.1-0.el8.x86_64 12/25 2024-04-23T01:59:25.466 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el8.x86_64 6/25 2024-04-23T01:59:25.544 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:18.2.1-0.el8.x86_64 13/25 2024-04-23T01:59:25.614 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el8.x86_64 13/25 2024-04-23T01:59:25.686 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librbd1-2:18.2.1-0.el8.x86_64 7/25 2024-04-23T01:59:25.748 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:18.2.1-0.el8.x86_64 7/25 2024-04-23T01:59:25.861 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2024-04-23T01:59:25.961 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/25 2024-04-23T01:59:26.133 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/25 2024-04-23T01:59:26.288 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el8.x86_64 10/25 2024-04-23T01:59:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cephfs-2:18.2.1-0.el8.x86_64 11/25 2024-04-23T01:59:26.673 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2024-04-23T01:59:26.744 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rbd-2:18.2.1-0.el8.x86_64 12/25 2024-04-23T01:59:26.836 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:18.2.1-0.el8.x86_64 16/25 2024-04-23T01:59:26.860 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libradosstriper1-2:18.2.1-0.el8.x86_64 13/25 2024-04-23T01:59:26.902 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:18.2.1-0.el8.x86_64 16/25 2024-04-23T01:59:26.931 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el8.x86_64 13/25 2024-04-23T01:59:27.094 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:18.2.1-0.el8.x86_64 17/25 2024-04-23T01:59:27.161 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2024-04-23T01:59:27.278 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-04-23T01:59:27.477 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:18.2.1-0.el8.x86_64 19/25 2024-04-23T01:59:27.944 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2024-04-23T01:59:28.127 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librgw2-2:18.2.1-0.el8.x86_64 16/25 2024-04-23T01:59:28.202 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librgw2-2:18.2.1-0.el8.x86_64 16/25 2024-04-23T01:59:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rgw-2:18.2.1-0.el8.x86_64 17/25 2024-04-23T01:59:28.560 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-04-23T01:59:28.727 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-common-2:18.2.1-0.el8.x86_64 19/25 2024-04-23T01:59:30.087 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el8.x86_64 20/25 2024-04-23T01:59:30.369 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:18.2.1-0.el8.x86_64 20/25 2024-04-23T01:59:30.980 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el8.x86_64 20/25 2024-04-23T01:59:31.285 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:18.2.1-0.el8.x86_64 21/25 2024-04-23T01:59:31.380 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el8.x86_64 21/25 2024-04-23T01:59:31.688 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el8.x86_64 20/25 2024-04-23T01:59:31.844 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:18.2.1-0.el8.x86_64 22/25 2024-04-23T01:59:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-common-2:18.2.1-0.el8.x86_64 20/25 2024-04-23T01:59:32.498 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el8.x86_64 20/25 2024-04-23T01:59:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-base-2:18.2.1-0.el8.x86_64 21/25 2024-04-23T01:59:32.876 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el8.x86_64 21/25 2024-04-23T01:59:33.092 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-selinux-2:18.2.1-0.el8.x86_64 22/25 2024-04-23T01:59:51.155 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el8.x86_64 22/25 2024-04-23T01:59:51.155 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2024-04-23T01:59:51.155 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2024-04-23T01:59:51.155 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /dev 2024-04-23T01:59:51.155 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /run 2024-04-23T01:59:51.156 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2024-04-23T01:59:51.156 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2024-04-23T01:59:51.156 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2024-04-23T01:59:51.156 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2024-04-23T01:59:51.156 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /dev 2024-04-23T01:59:51.156 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:59:52.541 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:18.2.1-0.el8.x86_64 23/25 2024-04-23T01:59:52.561 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el8.x86_64 23/25 2024-04-23T01:59:52.562 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-23T01:59:52.709 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el8.x86_64 22/25 2024-04-23T01:59:52.709 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /sys 2024-04-23T01:59:52.709 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /proc 2024-04-23T01:59:52.709 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /dev 2024-04-23T01:59:52.709 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /run 2024-04-23T01:59:52.709 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /mnt 2024-04-23T01:59:52.709 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /var/tmp 2024-04-23T01:59:52.709 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /home 2024-04-23T01:59:52.709 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /tmp 2024-04-23T01:59:52.709 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /dev 2024-04-23T01:59:52.709 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:59:52.776 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-23T01:59:52.776 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-23T01:59:53.471 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-23T01:59:53.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:18.2.1-0.el8.x86_64 1/25 2024-04-23T01:59:53.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:18.2.1-0.el8.x86_64 2/25 2024-04-23T01:59:53.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el8.x86_64 3/25 2024-04-23T01:59:53.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:18.2.1-0.el8.x86_64 4/25 2024-04-23T01:59:53.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:18.2.1-0.el8.x86_64 5/25 2024-04-23T01:59:53.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:18.2.1-0.el8.x86_64 6/25 2024-04-23T01:59:53.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:18.2.1-0.el8.x86_64 7/25 2024-04-23T01:59:53.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el8.x86_64 8/25 2024-04-23T01:59:53.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el8.x86_64 9/25 2024-04-23T01:59:53.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:18.2.1-0.el8.x86_64 10/25 2024-04-23T01:59:53.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:18.2.1-0.el8.x86_64 11/25 2024-04-23T01:59:53.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:18.2.1-0.el8.x86_64 12/25 2024-04-23T01:59:53.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:18.2.1-0.el8.x86_64 13/25 2024-04-23T01:59:53.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-04-23T01:59:53.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-04-23T01:59:53.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-04-23T01:59:53.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-04-23T01:59:53.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-04-23T01:59:53.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2024-04-23T01:59:53.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2024-04-23T01:59:53.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2024-04-23T01:59:53.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:18.2.1-0.el8.x86_64 22/25 2024-04-23T01:59:53.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-04-23T01:59:53.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:18.2.1-0.el8.x86_64 24/25 2024-04-23T01:59:54.131 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-radosgw-2:18.2.1-0.el8.x86_64 23/25 2024-04-23T01:59:54.152 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el8.x86_64 23/25 2024-04-23T01:59:54.152 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-23T01:59:54.340 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-23T01:59:54.340 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-23T01:59:54.357 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-04-23T01:59:54.357 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:59:54.357 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2024-04-23T01:59:54.357 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:18.2.1-0.el8.x86_64 librbd1-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:54.358 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-23T01:59:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-23T01:59:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-23T01:59:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-04-23T01:59:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-04-23T01:59:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-23T01:59:54.359 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-23T01:59:54.359 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:54.359 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:54.359 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:54.359 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-23T01:59:54.359 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:54.359 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:54.359 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:54.359 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.13.0-2.el8.x86_64 2024-04-23T01:59:54.359 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:59:54.359 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T01:59:54.688 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-test 2024-04-23T01:59:55.043 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-23T01:59:55.043 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-base-2:18.2.1-0.el8.x86_64 1/25 2024-04-23T01:59:55.043 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-common-2:18.2.1-0.el8.x86_64 2/25 2024-04-23T01:59:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el8.x86_64 3/25 2024-04-23T01:59:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-selinux-2:18.2.1-0.el8.x86_64 4/25 2024-04-23T01:59:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs2-2:18.2.1-0.el8.x86_64 5/25 2024-04-23T01:59:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libradosstriper1-2:18.2.1-0.el8.x86_64 6/25 2024-04-23T01:59:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librgw2-2:18.2.1-0.el8.x86_64 7/25 2024-04-23T01:59:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el8.x86_64 8/25 2024-04-23T01:59:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el8.x86_64 9/25 2024-04-23T01:59:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cephfs-2:18.2.1-0.el8.x86_64 10/25 2024-04-23T01:59:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rados-2:18.2.1-0.el8.x86_64 11/25 2024-04-23T01:59:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rbd-2:18.2.1-0.el8.x86_64 12/25 2024-04-23T01:59:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rgw-2:18.2.1-0.el8.x86_64 13/25 2024-04-23T01:59:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-04-23T01:59:55.044 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-04-23T01:59:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-04-23T01:59:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-04-23T01:59:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-04-23T01:59:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2024-04-23T01:59:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2024-04-23T01:59:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2024-04-23T01:59:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:18.2.1-0.el8.x86_64 22/25 2024-04-23T01:59:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-04-23T01:59:55.045 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:18.2.1-0.el8.x86_64 24/25 2024-04-23T01:59:55.420 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:41 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T01:59:56.013 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-04-23T01:59:56.014 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:59:56.014 INFO:teuthology.orchestra.run.smithi071.stdout:Upgraded: 2024-04-23T01:59:56.014 INFO:teuthology.orchestra.run.smithi071.stdout: librados2-2:18.2.1-0.el8.x86_64 librbd1-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:56.014 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-23T01:59:56.014 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:56.014 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:56.014 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:56.014 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:56.014 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-23T01:59:56.014 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:56.014 INFO:teuthology.orchestra.run.smithi071.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-23T01:59:56.014 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-04-23T01:59:56.015 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:56.015 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-04-23T01:59:56.015 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:56.015 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-23T01:59:56.015 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-23T01:59:56.015 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:56.015 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:56.015 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:56.015 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-23T01:59:56.015 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:56.015 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:56.015 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw-2:18.2.1-0.el8.x86_64 2024-04-23T01:59:56.015 INFO:teuthology.orchestra.run.smithi071.stdout: thrift-0.13.0-2.el8.x86_64 2024-04-23T01:59:56.015 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:59:56.016 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T01:59:56.330 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-test 2024-04-23T01:59:57.058 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T01:59:57.059 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T01:59:57.059 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-04-23T01:59:57.060 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T01:59:57.060 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-23T01:59:57.060 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:18.2.1-0.el8 ceph 47 M 2024-04-23T01:59:57.060 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-04-23T01:59:57.060 INFO:teuthology.orchestra.run.smithi192.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-04-23T01:59:57.060 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:18.2.1-0.el8 ceph 141 k 2024-04-23T01:59:57.060 INFO:teuthology.orchestra.run.smithi192.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-04-23T01:59:57.060 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-04-23T01:59:57.060 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-04-23T01:59:57.060 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:59:57.060 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-23T01:59:57.060 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T01:59:57.060 INFO:teuthology.orchestra.run.smithi192.stdout:Install 6 Packages 2024-04-23T01:59:57.061 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T01:59:57.061 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 48 M 2024-04-23T01:59:57.061 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 212 M 2024-04-23T01:59:57.061 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-23T01:59:57.062 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:00:42 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T01:59:57.219 INFO:teuthology.orchestra.run.smithi192.stdout:(1/6): libcephsqlite-18.2.1-0.el8.x86_64.rpm 1.3 MB/s | 141 kB 00:00 2024-04-23T01:59:57.261 INFO:teuthology.orchestra.run.smithi192.stdout:(2/6): jq-1.6-8.el8.x86_64.rpm 1.3 MB/s | 203 kB 00:00 2024-04-23T01:59:57.387 INFO:teuthology.orchestra.run.smithi192.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 2.5 MB/s | 323 kB 00:00 2024-04-23T01:59:57.445 INFO:teuthology.orchestra.run.smithi192.stdout:(4/6): oniguruma-6.8.2-3.el8.x86_64.rpm 835 kB/s | 188 kB 00:00 2024-04-23T01:59:57.929 INFO:teuthology.orchestra.run.smithi192.stdout:(5/6): ceph-test-18.2.1-0.el8.x86_64.rpm 57 MB/s | 47 MB 00:00 2024-04-23T01:59:57.954 INFO:teuthology.orchestra.run.smithi192.stdout:(6/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 125 kB/s | 71 kB 00:00 2024-04-23T01:59:57.956 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:59:57.956 INFO:teuthology.orchestra.run.smithi192.stdout:Total 53 MB/s | 48 MB 00:00 2024-04-23T01:59:57.999 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-23T01:59:58.020 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-23T01:59:58.020 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-23T01:59:58.308 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-23T01:59:58.311 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-23T01:59:58.691 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T01:59:58.693 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T01:59:58.693 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-04-23T01:59:58.693 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T01:59:58.693 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-23T01:59:58.693 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test x86_64 2:18.2.1-0.el8 ceph 47 M 2024-04-23T01:59:58.693 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-23T01:59:58.693 INFO:teuthology.orchestra.run.smithi071.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-04-23T01:59:58.693 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite x86_64 2:18.2.1-0.el8 ceph 141 k 2024-04-23T01:59:58.693 INFO:teuthology.orchestra.run.smithi071.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-04-23T01:59:58.693 INFO:teuthology.orchestra.run.smithi071.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-04-23T01:59:58.693 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-04-23T01:59:58.694 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:59:58.694 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-23T01:59:58.694 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T01:59:58.694 INFO:teuthology.orchestra.run.smithi071.stdout:Install 6 Packages 2024-04-23T01:59:58.694 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T01:59:58.695 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 48 M 2024-04-23T01:59:58.695 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 212 M 2024-04-23T01:59:58.695 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-23T01:59:58.791 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-23T01:59:58.945 INFO:teuthology.orchestra.run.smithi071.stdout:(1/6): libcephsqlite-18.2.1-0.el8.x86_64.rpm 2.4 MB/s | 141 kB 00:00 2024-04-23T01:59:59.022 INFO:teuthology.orchestra.run.smithi071.stdout:(2/6): jq-1.6-8.el8.x86_64.rpm 1.5 MB/s | 203 kB 00:00 2024-04-23T01:59:59.038 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-04-23T01:59:59.071 INFO:teuthology.orchestra.run.smithi071.stdout:(3/6): oniguruma-6.8.2-3.el8.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-23T01:59:59.113 INFO:teuthology.orchestra.run.smithi071.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-04-23T01:59:59.138 INFO:teuthology.orchestra.run.smithi071.stdout:(5/6): socat-1.7.4.1-1.el8.x86_64.rpm 2.7 MB/s | 323 kB 00:00 2024-04-23T01:59:59.225 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-04-23T01:59:59.398 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-04-23T01:59:59.475 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-04-23T01:59:59.599 INFO:teuthology.orchestra.run.smithi071.stdout:(6/6): ceph-test-18.2.1-0.el8.x86_64.rpm 66 MB/s | 47 MB 00:00 2024-04-23T01:59:59.602 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-23T01:59:59.602 INFO:teuthology.orchestra.run.smithi071.stdout:Total 52 MB/s | 48 MB 00:00 2024-04-23T01:59:59.644 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-23T01:59:59.664 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-23T01:59:59.664 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-23T01:59:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-04-23T01:59:59.881 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:18.2.1-0.el8.x86_64 5/6 2024-04-23T01:59:59.955 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-23T01:59:59.958 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-23T02:00:00.427 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-23T02:00:00.626 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-04-23T02:00:00.839 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-04-23T02:00:01.011 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-04-23T02:00:01.110 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-04-23T02:00:01.307 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-04-23T02:00:01.486 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephsqlite-2:18.2.1-0.el8.x86_64 5/6 2024-04-23T02:00:04.398 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el8.x86_64 5/6 2024-04-23T02:00:04.645 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:18.2.1-0.el8.x86_64 6/6 2024-04-23T02:00:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el8.x86_64 6/6 2024-04-23T02:00:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:18.2.1-0.el8.x86_64 1/6 2024-04-23T02:00:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:18.2.1-0.el8.x86_64 2/6 2024-04-23T02:00:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-04-23T02:00:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-04-23T02:00:04.919 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-04-23T02:00:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-04-23T02:00:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:00:05.362 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-23T02:00:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:18.2.1-0.el8.x86_64 jq-1.6-8.el8.x86_64 2024-04-23T02:00:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:18.2.1-0.el8.x86_64 oniguruma-6.8.2-3.el8.x86_64 2024-04-23T02:00:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-1.el8.x86_64 xmlstarlet-1.6.1-20.el8.x86_64 2024-04-23T02:00:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:00:05.362 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:00:05.564 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph 2024-04-23T02:00:05.975 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el8.x86_64 5/6 2024-04-23T02:00:06.217 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-test-2:18.2.1-0.el8.x86_64 6/6 2024-04-23T02:00:06.299 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:52 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:00:06.498 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el8.x86_64 6/6 2024-04-23T02:00:06.498 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-test-2:18.2.1-0.el8.x86_64 1/6 2024-04-23T02:00:06.499 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephsqlite-2:18.2.1-0.el8.x86_64 2/6 2024-04-23T02:00:06.499 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-04-23T02:00:06.499 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-04-23T02:00:06.499 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-04-23T02:00:06.944 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-04-23T02:00:06.944 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:00:06.944 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-23T02:00:06.944 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test-2:18.2.1-0.el8.x86_64 jq-1.6-8.el8.x86_64 2024-04-23T02:00:06.945 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite-2:18.2.1-0.el8.x86_64 oniguruma-6.8.2-3.el8.x86_64 2024-04-23T02:00:06.945 INFO:teuthology.orchestra.run.smithi071.stdout: socat-1.7.4.1-1.el8.x86_64 xmlstarlet-1.6.1-20.el8.x86_64 2024-04-23T02:00:06.945 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:00:06.945 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:00:07.193 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph 2024-04-23T02:00:07.915 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:00:07.918 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:00:07.918 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-04-23T02:00:07.918 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:00:07.918 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-23T02:00:07.918 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:18.2.1-0.el8 ceph 6.3 k 2024-04-23T02:00:07.919 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-04-23T02:00:07.919 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:18.2.1-0.el8 ceph 2.3 M 2024-04-23T02:00:07.919 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:18.2.1-0.el8 ceph 1.6 M 2024-04-23T02:00:07.919 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el8 ceph-noarch 242 k 2024-04-23T02:00:07.919 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:18.2.1-0.el8 ceph 4.6 M 2024-04-23T02:00:07.919 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:18.2.1-0.el8 ceph 18 M 2024-04-23T02:00:07.919 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-23T02:00:07.919 INFO:teuthology.orchestra.run.smithi192.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-23T02:00:07.919 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-04-23T02:00:07.919 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-23T02:00:07.919 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-23T02:00:07.919 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-04-23T02:00:07.919 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-23T02:00:07.920 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-23T02:00:07.920 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-23T02:00:07.920 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-04-23T02:00:07.920 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-04-23T02:00:07.920 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-23T02:00:07.920 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-23T02:00:07.920 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-23T02:00:07.920 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-04-23T02:00:07.920 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-04-23T02:00:07.920 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-04-23T02:00:07.920 INFO:teuthology.orchestra.run.smithi192.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-23T02:00:07.920 INFO:teuthology.orchestra.run.smithi192.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-23T02:00:07.920 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-23T02:00:07.921 INFO:teuthology.orchestra.run.smithi192.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-23T02:00:07.921 INFO:teuthology.orchestra.run.smithi192.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-23T02:00:07.921 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-23T02:00:07.921 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-23T02:00:07.921 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-04-23T02:00:07.921 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-23T02:00:07.921 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:00:07.921 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-23T02:00:07.921 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:00:07.921 INFO:teuthology.orchestra.run.smithi192.stdout:Install 32 Packages 2024-04-23T02:00:07.921 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:00:07.922 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:00:52 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:00:07.922 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 30 M 2024-04-23T02:00:07.923 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 119 M 2024-04-23T02:00:07.923 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-23T02:00:09.559 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:00:09.563 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:00:09.563 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-23T02:00:09.563 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:00:09.563 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-23T02:00:09.563 INFO:teuthology.orchestra.run.smithi071.stdout: ceph x86_64 2:18.2.1-0.el8 ceph 6.3 k 2024-04-23T02:00:09.563 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-23T02:00:09.563 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds x86_64 2:18.2.1-0.el8 ceph 2.3 M 2024-04-23T02:00:09.563 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr x86_64 2:18.2.1-0.el8 ceph 1.6 M 2024-04-23T02:00:09.563 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el8 ceph-noarch 242 k 2024-04-23T02:00:09.563 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon x86_64 2:18.2.1-0.el8 ceph 4.6 M 2024-04-23T02:00:09.563 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd x86_64 2:18.2.1-0.el8 ceph 18 M 2024-04-23T02:00:09.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-23T02:00:09.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-23T02:00:09.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-04-23T02:00:09.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-23T02:00:09.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-23T02:00:09.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-04-23T02:00:09.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-23T02:00:09.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-23T02:00:09.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-23T02:00:09.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-04-23T02:00:09.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-04-23T02:00:09.564 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-23T02:00:09.565 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-23T02:00:09.565 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-23T02:00:09.565 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-04-23T02:00:09.565 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-04-23T02:00:09.565 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-04-23T02:00:09.565 INFO:teuthology.orchestra.run.smithi071.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-23T02:00:09.565 INFO:teuthology.orchestra.run.smithi071.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-23T02:00:09.565 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-23T02:00:09.565 INFO:teuthology.orchestra.run.smithi071.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-23T02:00:09.565 INFO:teuthology.orchestra.run.smithi071.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-23T02:00:09.565 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-23T02:00:09.565 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-23T02:00:09.565 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-04-23T02:00:09.565 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-23T02:00:09.566 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:00:09.566 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-23T02:00:09.566 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:00:09.566 INFO:teuthology.orchestra.run.smithi071.stdout:Install 32 Packages 2024-04-23T02:00:09.566 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:00:09.567 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 30 M 2024-04-23T02:00:09.567 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 119 M 2024-04-23T02:00:09.567 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-23T02:00:42.186 INFO:teuthology.orchestra.run.smithi192.stdout:(1/32): ceph-18.2.1-0.el8.x86_64.rpm 112 kB/s | 6.3 kB 00:00 2024-04-23T02:00:42.268 INFO:teuthology.orchestra.run.smithi192.stdout:(2/32): ceph-mgr-18.2.1-0.el8.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-23T02:00:42.328 INFO:teuthology.orchestra.run.smithi192.stdout:(3/32): ceph-mds-18.2.1-0.el8.x86_64.rpm 12 MB/s | 2.3 MB 00:00 2024-04-23T02:00:42.362 INFO:teuthology.orchestra.run.smithi192.stdout:(4/32): ceph-mgr-modules-core-18.2.1-0.el8.noar 7.1 MB/s | 242 kB 00:00 2024-04-23T02:00:42.446 INFO:teuthology.orchestra.run.smithi192.stdout:(5/32): ceph-mon-18.2.1-0.el8.x86_64.rpm 18 MB/s | 4.6 MB 00:00 2024-04-23T02:00:42.521 INFO:teuthology.orchestra.run.smithi192.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 520 kB/s | 39 kB 00:00 2024-04-23T02:00:42.588 INFO:teuthology.orchestra.run.smithi192.stdout:(7/32): python3-mako-1.0.6-14.el8.noarch.rpm 697 kB/s | 157 kB 00:00 2024-04-23T02:00:42.763 INFO:teuthology.orchestra.run.smithi192.stdout:(8/32): ceph-osd-18.2.1-0.el8.x86_64.rpm 36 MB/s | 18 MB 00:00 2024-04-23T02:00:42.788 INFO:teuthology.orchestra.run.smithi192.stdout:(9/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 388 kB/s | 103 kB 00:00 2024-04-23T02:00:42.813 INFO:teuthology.orchestra.run.smithi192.stdout:(10/32): python3-pytz-2017.2-11.el8.noarch.rpm 253 kB/s | 57 kB 00:00 2024-04-23T02:00:42.889 INFO:teuthology.orchestra.run.smithi192.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 3.6 MB/s | 457 kB 00:00 2024-04-23T02:00:43.031 INFO:teuthology.orchestra.run.smithi192.stdout:(12/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.0 MB/s | 249 kB 00:00 2024-04-23T02:00:43.064 INFO:teuthology.orchestra.run.smithi192.stdout:(13/32): python3-pycparser-2.14-14.el8.noarch.r 622 kB/s | 109 kB 00:00 2024-04-23T02:00:43.131 INFO:teuthology.orchestra.run.smithi192.stdout:(14/32): python3-cryptography-3.2.1-7.el8.x86_6 2.0 MB/s | 641 kB 00:00 2024-04-23T02:00:43.340 INFO:teuthology.orchestra.run.smithi192.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 141 kB/s | 44 kB 00:00 2024-04-23T02:00:43.432 INFO:teuthology.orchestra.run.smithi192.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 504 kB/s | 185 kB 00:00 2024-04-23T02:00:43.482 INFO:teuthology.orchestra.run.smithi192.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 506 kB/s | 173 kB 00:00 2024-04-23T02:00:43.508 INFO:teuthology.orchestra.run.smithi192.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 148 kB/s | 11 kB 00:00 2024-04-23T02:00:43.550 INFO:teuthology.orchestra.run.smithi192.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 263 kB/s | 18 kB 00:00 2024-04-23T02:00:43.575 INFO:teuthology.orchestra.run.smithi192.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 730 kB/s | 49 kB 00:00 2024-04-23T02:00:43.600 INFO:teuthology.orchestra.run.smithi192.stdout:(21/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.4 MB/s | 384 kB 00:00 2024-04-23T02:00:43.625 INFO:teuthology.orchestra.run.smithi192.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 779 kB/s | 59 kB 00:00 2024-04-23T02:00:43.667 INFO:teuthology.orchestra.run.smithi192.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 233 kB/s | 16 kB 00:00 2024-04-23T02:00:43.701 INFO:teuthology.orchestra.run.smithi192.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 249 kB/s | 19 kB 00:00 2024-04-23T02:00:43.734 INFO:teuthology.orchestra.run.smithi192.stdout:(25/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.7 MB/s | 283 kB 00:00 2024-04-23T02:00:43.788 INFO:teuthology.orchestra.run.smithi192.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 198 kB/s | 24 kB 00:00 2024-04-23T02:00:43.800 INFO:teuthology.orchestra.run.smithi071.stdout:(1/32): ceph-18.2.1-0.el8.x86_64.rpm 139 kB/s | 6.3 kB 00:00 2024-04-23T02:00:43.835 INFO:teuthology.orchestra.run.smithi192.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 218 kB/s | 29 kB 00:00 2024-04-23T02:00:43.851 INFO:teuthology.orchestra.run.smithi071.stdout:(2/32): ceph-mgr-18.2.1-0.el8.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-23T02:00:43.860 INFO:teuthology.orchestra.run.smithi192.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 219 kB/s | 27 kB 00:00 2024-04-23T02:00:43.901 INFO:teuthology.orchestra.run.smithi071.stdout:(3/32): ceph-mds-18.2.1-0.el8.x86_64.rpm 16 MB/s | 2.3 MB 00:00 2024-04-23T02:00:43.926 INFO:teuthology.orchestra.run.smithi071.stdout:(4/32): ceph-mgr-modules-core-18.2.1-0.el8.noar 9.5 MB/s | 242 kB 00:00 2024-04-23T02:00:43.927 INFO:teuthology.orchestra.run.smithi192.stdout:(29/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.3 MB/s | 85 kB 00:00 2024-04-23T02:00:43.953 INFO:teuthology.orchestra.run.smithi192.stdout:(30/32): python3-waitress-1.4.3-1.el8.noarch.rp 1.5 MB/s | 255 kB 00:00 2024-04-23T02:00:43.995 INFO:teuthology.orchestra.run.smithi192.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 343 kB/s | 23 kB 00:00 2024-04-23T02:00:44.010 INFO:teuthology.orchestra.run.smithi071.stdout:(5/32): ceph-mon-18.2.1-0.el8.x86_64.rpm 22 MB/s | 4.6 MB 00:00 2024-04-23T02:00:44.068 INFO:teuthology.orchestra.run.smithi071.stdout:(6/32): python3-mako-1.0.6-14.el8.noarch.rpm 1.1 MB/s | 157 kB 00:00 2024-04-23T02:00:44.127 INFO:teuthology.orchestra.run.smithi071.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 1.7 MB/s | 103 kB 00:00 2024-04-23T02:00:44.152 INFO:teuthology.orchestra.run.smithi071.stdout:(8/32): python3-markupsafe-0.23-19.el8.x86_64.r 273 kB/s | 39 kB 00:00 2024-04-23T02:00:44.178 INFO:teuthology.orchestra.run.smithi071.stdout:(9/32): python3-pytz-2017.2-11.el8.noarch.rpm 1.1 MB/s | 57 kB 00:00 2024-04-23T02:00:44.395 INFO:teuthology.orchestra.run.smithi071.stdout:(10/32): ceph-osd-18.2.1-0.el8.x86_64.rpm 33 MB/s | 18 MB 00:00 2024-04-23T02:00:44.461 INFO:teuthology.orchestra.run.smithi192.stdout:(32/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 418 kB/s | 251 kB 00:00 2024-04-23T02:00:44.465 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:00:44.465 INFO:teuthology.orchestra.run.smithi192.stdout:Total 842 kB/s | 30 MB 00:36 2024-04-23T02:00:44.470 INFO:teuthology.orchestra.run.smithi071.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 1.4 MB/s | 457 kB 00:00 2024-04-23T02:00:44.542 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-23T02:00:44.581 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-23T02:00:44.581 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-23T02:00:44.879 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-23T02:00:44.882 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-23T02:00:44.887 INFO:teuthology.orchestra.run.smithi071.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 262 kB/s | 109 kB 00:00 2024-04-23T02:00:44.912 INFO:teuthology.orchestra.run.smithi071.stdout:(13/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 339 kB/s | 249 kB 00:00 2024-04-23T02:00:45.146 INFO:teuthology.orchestra.run.smithi071.stdout:(14/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 169 kB/s | 44 kB 00:00 2024-04-23T02:00:45.179 INFO:teuthology.orchestra.run.smithi071.stdout:(15/32): python3-cryptography-3.2.1-7.el8.x86_6 818 kB/s | 641 kB 00:00 2024-04-23T02:00:45.263 INFO:teuthology.orchestra.run.smithi071.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 528 kB/s | 185 kB 00:00 2024-04-23T02:00:45.314 INFO:teuthology.orchestra.run.smithi071.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.0 MB/s | 173 kB 00:00 2024-04-23T02:00:45.339 INFO:teuthology.orchestra.run.smithi071.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 147 kB/s | 11 kB 00:00 2024-04-23T02:00:45.381 INFO:teuthology.orchestra.run.smithi071.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 263 kB/s | 18 kB 00:00 2024-04-23T02:00:45.444 INFO:teuthology.orchestra.run.smithi071.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 463 kB/s | 49 kB 00:00 2024-04-23T02:00:45.473 INFO:teuthology.orchestra.run.smithi071.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 639 kB/s | 59 kB 00:00 2024-04-23T02:00:45.540 INFO:teuthology.orchestra.run.smithi071.stdout:(22/32): python3-portend-2.6-1.el8.noarch.rpm 234 kB/s | 16 kB 00:00 2024-04-23T02:00:45.565 INFO:teuthology.orchestra.run.smithi071.stdout:(23/32): python3-pecan-1.3.2-9.el8.noarch.rpm 2.3 MB/s | 283 kB 00:00 2024-04-23T02:00:45.590 INFO:teuthology.orchestra.run.smithi071.stdout:(24/32): python3-cherrypy-18.4.0-1.el8.noarch.r 937 kB/s | 384 kB 00:00 2024-04-23T02:00:45.615 INFO:teuthology.orchestra.run.smithi071.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 249 kB/s | 19 kB 00:00 2024-04-23T02:00:45.641 INFO:teuthology.orchestra.run.smithi071.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 316 kB/s | 24 kB 00:00 2024-04-23T02:00:45.666 INFO:teuthology.orchestra.run.smithi071.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 388 kB/s | 29 kB 00:00 2024-04-23T02:00:45.691 INFO:teuthology.orchestra.run.smithi071.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 364 kB/s | 27 kB 00:00 2024-04-23T02:00:45.725 INFO:teuthology.orchestra.run.smithi071.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 3.0 MB/s | 255 kB 00:00 2024-04-23T02:00:45.759 INFO:teuthology.orchestra.run.smithi071.stdout:(30/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.7 MB/s | 251 kB 00:00 2024-04-23T02:00:45.817 INFO:teuthology.orchestra.run.smithi071.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 680 kB/s | 85 kB 00:00 2024-04-23T02:00:45.842 INFO:teuthology.orchestra.run.smithi071.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 196 kB/s | 23 kB 00:00 2024-04-23T02:00:45.847 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:00:45.847 INFO:teuthology.orchestra.run.smithi071.stdout:Total 848 kB/s | 30 MB 00:36 2024-04-23T02:00:45.924 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-23T02:00:45.961 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-23T02:00:45.962 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-23T02:00:46.034 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-23T02:00:46.280 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-04-23T02:00:46.282 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-23T02:00:46.285 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-23T02:00:46.524 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-04-23T02:00:46.775 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-04-23T02:00:47.071 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-04-23T02:00:47.311 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-04-23T02:00:47.452 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-23T02:00:47.534 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-04-23T02:00:47.706 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-04-23T02:00:47.777 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-04-23T02:00:47.968 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-04-23T02:00:47.971 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-04-23T02:00:48.193 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-04-23T02:00:48.215 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-04-23T02:00:48.477 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-04-23T02:00:48.494 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-04-23T02:00:48.713 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-04-23T02:00:48.747 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-04-23T02:00:48.877 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-04-23T02:00:49.022 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-04-23T02:00:49.096 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-04-23T02:00:49.273 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-04-23T02:00:49.456 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-04-23T02:00:49.480 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-04-23T02:00:49.726 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-04-23T02:00:49.743 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-04-23T02:00:49.923 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-04-23T02:00:50.031 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-04-23T02:00:50.224 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-04-23T02:00:50.263 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-04-23T02:00:50.466 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-04-23T02:00:50.687 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-04-23T02:00:50.750 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-04-23T02:00:50.945 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-04-23T02:00:51.004 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-04-23T02:00:51.150 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-04-23T02:00:51.192 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-04-23T02:00:51.411 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-04-23T02:00:51.434 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-04-23T02:00:51.627 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-04-23T02:00:51.671 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-04-23T02:00:51.902 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-04-23T02:00:51.970 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-04-23T02:00:52.151 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-04-23T02:00:52.231 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-04-23T02:00:52.403 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-04-23T02:00:52.490 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-04-23T02:00:52.605 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-04-23T02:00:52.833 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-04-23T02:00:52.838 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-04-23T02:00:53.071 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-04-23T02:00:53.245 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el8.noarch 27/32 2024-04-23T02:00:53.314 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-04-23T02:00:53.623 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:18.2.1-0.el8.x86_64 28/32 2024-04-23T02:00:53.637 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-04-23T02:00:53.988 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-04-23T02:00:54.397 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el8.noarch 27/32 2024-04-23T02:00:54.644 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-2:18.2.1-0.el8.x86_64 28/32 2024-04-23T02:00:55.204 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el8.x86_64 28/32 2024-04-23T02:00:55.423 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:18.2.1-0.el8.x86_64 29/32 2024-04-23T02:00:55.883 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el8.x86_64 29/32 2024-04-23T02:00:56.106 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:18.2.1-0.el8.x86_64 30/32 2024-04-23T02:00:56.213 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el8.x86_64 28/32 2024-04-23T02:00:56.334 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el8.x86_64 30/32 2024-04-23T02:00:56.443 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-osd-2:18.2.1-0.el8.x86_64 29/32 2024-04-23T02:00:56.531 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:18.2.1-0.el8.x86_64 31/32 2024-04-23T02:00:56.548 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el8.x86_64 31/32 2024-04-23T02:00:56.705 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:18.2.1-0.el8.x86_64 32/32 2024-04-23T02:00:56.899 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el8.x86_64 29/32 2024-04-23T02:00:57.116 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-2:18.2.1-0.el8.x86_64 32/32 2024-04-23T02:00:57.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:18.2.1-0.el8.x86_64 1/32 2024-04-23T02:00:57.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:18.2.1-0.el8.x86_64 2/32 2024-04-23T02:00:57.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:18.2.1-0.el8.x86_64 3/32 2024-04-23T02:00:57.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:18.2.1-0.el8.x86_64 4/32 2024-04-23T02:00:57.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:18.2.1-0.el8.x86_64 5/32 2024-04-23T02:00:57.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el8.noarch 6/32 2024-04-23T02:00:57.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-04-23T02:00:57.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-04-23T02:00:57.116 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-04-23T02:00:57.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-04-23T02:00:57.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-04-23T02:00:57.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-04-23T02:00:57.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-04-23T02:00:57.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-04-23T02:00:57.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-04-23T02:00:57.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-04-23T02:00:57.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-04-23T02:00:57.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-04-23T02:00:57.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-04-23T02:00:57.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-04-23T02:00:57.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-04-23T02:00:57.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-04-23T02:00:57.117 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-04-23T02:00:57.118 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-04-23T02:00:57.118 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-04-23T02:00:57.118 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-04-23T02:00:57.118 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-04-23T02:00:57.118 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-04-23T02:00:57.118 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-04-23T02:00:57.118 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-04-23T02:00:57.118 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-04-23T02:00:57.118 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mon-2:18.2.1-0.el8.x86_64 30/32 2024-04-23T02:00:57.330 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el8.x86_64 30/32 2024-04-23T02:00:57.583 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mds-2:18.2.1-0.el8.x86_64 31/32 2024-04-23T02:00:57.600 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el8.x86_64 31/32 2024-04-23T02:00:57.726 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-2:18.2.1-0.el8.x86_64 32/32 2024-04-23T02:00:58.040 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-04-23T02:00:58.040 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:00:58.040 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-23T02:00:58.040 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:18.2.1-0.el8.x86_64 2024-04-23T02:00:58.040 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:18.2.1-0.el8.x86_64 2024-04-23T02:00:58.041 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:18.2.1-0.el8.x86_64 2024-04-23T02:00:58.041 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:18.2.1-0.el8.noarch 2024-04-23T02:00:58.041 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:18.2.1-0.el8.x86_64 2024-04-23T02:00:58.041 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:18.2.1-0.el8.x86_64 2024-04-23T02:00:58.041 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-23T02:00:58.041 INFO:teuthology.orchestra.run.smithi192.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-23T02:00:58.041 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-23T02:00:58.041 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-23T02:00:58.042 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-23T02:00:58.042 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-04-23T02:00:58.042 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-23T02:00:58.042 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-23T02:00:58.042 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-23T02:00:58.042 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-23T02:00:58.042 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-23T02:00:58.042 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-23T02:00:58.042 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-23T02:00:58.042 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-2.6-1.el8.noarch 2024-04-23T02:00:58.042 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-23T02:00:58.043 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-23T02:00:58.043 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-23T02:00:58.043 INFO:teuthology.orchestra.run.smithi192.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-23T02:00:58.043 INFO:teuthology.orchestra.run.smithi192.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-23T02:00:58.043 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-23T02:00:58.043 INFO:teuthology.orchestra.run.smithi192.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-23T02:00:58.043 INFO:teuthology.orchestra.run.smithi192.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-23T02:00:58.043 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-23T02:00:58.043 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-23T02:00:58.043 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-23T02:00:58.043 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-23T02:00:58.044 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:00:58.044 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:00:58.144 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-2:18.2.1-0.el8.x86_64 32/32 2024-04-23T02:00:58.144 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-2:18.2.1-0.el8.x86_64 1/32 2024-04-23T02:00:58.144 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mds-2:18.2.1-0.el8.x86_64 2/32 2024-04-23T02:00:58.144 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-2:18.2.1-0.el8.x86_64 3/32 2024-04-23T02:00:58.144 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mon-2:18.2.1-0.el8.x86_64 4/32 2024-04-23T02:00:58.145 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-osd-2:18.2.1-0.el8.x86_64 5/32 2024-04-23T02:00:58.145 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el8.noarch 6/32 2024-04-23T02:00:58.145 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-04-23T02:00:58.145 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-04-23T02:00:58.145 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-04-23T02:00:58.145 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-04-23T02:00:58.145 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-04-23T02:00:58.145 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-04-23T02:00:58.145 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-04-23T02:00:58.145 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-04-23T02:00:58.145 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-04-23T02:00:58.145 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-04-23T02:00:58.145 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-04-23T02:00:58.145 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-04-23T02:00:58.146 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-04-23T02:00:58.146 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-04-23T02:00:58.146 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-04-23T02:00:58.146 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-04-23T02:00:58.146 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-04-23T02:00:58.146 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-04-23T02:00:58.146 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-04-23T02:00:58.146 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-04-23T02:00:58.146 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-04-23T02:00:58.146 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-04-23T02:00:58.147 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-04-23T02:00:58.147 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-04-23T02:00:58.147 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-04-23T02:00:58.325 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-base 2024-04-23T02:00:59.046 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-04-23T02:00:59.046 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:00:59.046 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-23T02:00:59.046 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-2:18.2.1-0.el8.x86_64 2024-04-23T02:00:59.046 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds-2:18.2.1-0.el8.x86_64 2024-04-23T02:00:59.046 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-2:18.2.1-0.el8.x86_64 2024-04-23T02:00:59.046 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core-2:18.2.1-0.el8.noarch 2024-04-23T02:00:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon-2:18.2.1-0.el8.x86_64 2024-04-23T02:00:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd-2:18.2.1-0.el8.x86_64 2024-04-23T02:00:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-23T02:00:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-23T02:00:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-23T02:00:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-23T02:00:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-23T02:00:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-04-23T02:00:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-23T02:00:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-23T02:00:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-23T02:00:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-23T02:00:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-23T02:00:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-23T02:00:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-23T02:00:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend-2.6-1.el8.noarch 2024-04-23T02:00:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-23T02:00:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-23T02:00:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-23T02:00:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-23T02:00:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-23T02:00:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-23T02:00:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-23T02:00:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-23T02:00:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-23T02:00:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-23T02:00:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-23T02:00:59.049 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-23T02:00:59.049 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:00:59.049 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:00:59.071 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:45 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:00:59.337 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-base 2024-04-23T02:01:00.076 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:45 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:01:00.659 INFO:teuthology.orchestra.run.smithi192.stdout:Package ceph-base-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-23T02:01:00.708 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:01:00.709 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-23T02:01:00.709 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:01:00.816 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install cephadm 2024-04-23T02:01:01.551 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:47 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:01:01.631 INFO:teuthology.orchestra.run.smithi071.stdout:Package ceph-base-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-23T02:01:01.683 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:01:01.684 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-04-23T02:01:01.684 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:01:01.791 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install cephadm 2024-04-23T02:01:02.525 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:47 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:01:03.186 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:01:03.186 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:01:03.187 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-04-23T02:01:03.187 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:01:03.187 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-23T02:01:03.187 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:18.2.1-0.el8 ceph-noarch 171 k 2024-04-23T02:01:03.187 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:03.187 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-23T02:01:03.187 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:01:03.187 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-04-23T02:01:03.187 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:03.187 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 171 k 2024-04-23T02:01:03.187 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 680 k 2024-04-23T02:01:03.187 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-23T02:01:03.236 INFO:teuthology.orchestra.run.smithi192.stdout:cephadm-18.2.1-0.el8.noarch.rpm 3.4 MB/s | 171 kB 00:00 2024-04-23T02:01:03.236 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:01:03.237 INFO:teuthology.orchestra.run.smithi192.stdout:Total 3.4 MB/s | 171 kB 00:00 2024-04-23T02:01:03.237 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-23T02:01:03.245 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-23T02:01:03.246 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-23T02:01:03.247 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-23T02:01:03.251 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-23T02:01:03.293 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-23T02:01:03.825 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:18.2.1-0.el8.noarch 1/1 2024-04-23T02:01:03.993 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:18.2.1-0.el8.noarch 1/1 2024-04-23T02:01:04.165 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:01:04.165 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:01:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-04-23T02:01:04.166 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:01:04.166 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-23T02:01:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm noarch 2:18.2.1-0.el8 ceph-noarch 171 k 2024-04-23T02:01:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:04.166 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-23T02:01:04.166 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:01:04.166 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-04-23T02:01:04.166 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:04.166 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 171 k 2024-04-23T02:01:04.166 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 680 k 2024-04-23T02:01:04.167 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-23T02:01:04.206 INFO:teuthology.orchestra.run.smithi071.stdout:cephadm-18.2.1-0.el8.noarch.rpm 4.3 MB/s | 171 kB 00:00 2024-04-23T02:01:04.207 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:01:04.207 INFO:teuthology.orchestra.run.smithi071.stdout:Total 4.1 MB/s | 171 kB 00:00 2024-04-23T02:01:04.207 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-23T02:01:04.214 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-23T02:01:04.214 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-23T02:01:04.216 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-23T02:01:04.219 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-23T02:01:04.224 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:18.2.1-0.el8.noarch 1/1 2024-04-23T02:01:04.263 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-23T02:01:04.609 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:18.2.1-0.el8.noarch 1/1 2024-04-23T02:01:04.610 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:04.610 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-23T02:01:04.610 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:18.2.1-0.el8.noarch 2024-04-23T02:01:04.610 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:04.610 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:01:04.803 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:18.2.1-0.el8.noarch 1/1 2024-04-23T02:01:04.867 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-immutable-object-cache 2024-04-23T02:01:04.971 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephadm-2:18.2.1-0.el8.noarch 1/1 2024-04-23T02:01:05.200 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:18.2.1-0.el8.noarch 1/1 2024-04-23T02:01:05.569 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephadm-2:18.2.1-0.el8.noarch 1/1 2024-04-23T02:01:05.569 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:05.569 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-23T02:01:05.569 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm-2:18.2.1-0.el8.noarch 2024-04-23T02:01:05.569 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:05.569 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:01:05.608 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:51 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:01:05.763 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-immutable-object-cache 2024-04-23T02:01:06.508 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:51 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:01:07.226 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:01:07.226 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:01:07.226 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-04-23T02:01:07.226 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:01:07.226 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-23T02:01:07.226 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el8 ceph 165 k 2024-04-23T02:01:07.226 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:07.227 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-23T02:01:07.227 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:01:07.227 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-04-23T02:01:07.227 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:07.227 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 165 k 2024-04-23T02:01:07.227 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 503 k 2024-04-23T02:01:07.228 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-23T02:01:07.274 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-immutable-object-cache-18.2.1-0.el8.x86_64 3.5 MB/s | 165 kB 00:00 2024-04-23T02:01:07.275 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:01:07.275 INFO:teuthology.orchestra.run.smithi192.stdout:Total 3.4 MB/s | 165 kB 00:00 2024-04-23T02:01:07.275 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-23T02:01:07.286 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-23T02:01:07.286 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-23T02:01:07.334 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-23T02:01:07.336 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-23T02:01:07.498 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-23T02:01:07.663 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:01:08.074 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:01:08.141 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:01:08.142 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:01:08.142 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repo Size 2024-04-23T02:01:08.142 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:01:08.142 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-23T02:01:08.142 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el8 ceph 165 k 2024-04-23T02:01:08.142 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:08.142 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-23T02:01:08.142 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:01:08.143 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-04-23T02:01:08.143 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:08.143 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 165 k 2024-04-23T02:01:08.143 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 503 k 2024-04-23T02:01:08.143 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-23T02:01:08.215 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-immutable-object-cache-18.2.1-0.el8.x86_64 2.3 MB/s | 165 kB 00:00 2024-04-23T02:01:08.215 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:01:08.215 INFO:teuthology.orchestra.run.smithi071.stdout:Total 2.2 MB/s | 165 kB 00:00 2024-04-23T02:01:08.216 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-23T02:01:08.226 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-23T02:01:08.226 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-23T02:01:08.274 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-23T02:01:08.277 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-23T02:01:08.394 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-23T02:01:08.438 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:01:08.438 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:08.438 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-23T02:01:08.438 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:18.2.1-0.el8.x86_64 2024-04-23T02:01:08.438 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:08.439 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:01:08.605 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:01:08.686 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr 2024-04-23T02:01:09.024 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:01:09.422 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:01:09.422 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:09.422 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-23T02:01:09.422 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache-2:18.2.1-0.el8.x86_64 2024-04-23T02:01:09.422 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:09.423 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:01:09.431 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:55 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:01:09.678 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr 2024-04-23T02:01:10.421 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:55 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:01:11.020 INFO:teuthology.orchestra.run.smithi192.stdout:Package ceph-mgr-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-23T02:01:11.069 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:01:11.070 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-23T02:01:11.070 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:01:11.174 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-dashboard 2024-04-23T02:01:11.905 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:57 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:01:12.012 INFO:teuthology.orchestra.run.smithi071.stdout:Package ceph-mgr-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-23T02:01:12.061 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:01:12.062 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-04-23T02:01:12.062 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:01:12.172 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr-dashboard 2024-04-23T02:01:12.898 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:57 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:01:13.550 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:01:13.551 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:01:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-04-23T02:01:13.551 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:01:13.551 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-23T02:01:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el8 ceph-noarch 1.8 M 2024-04-23T02:01:13.551 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-04-23T02:01:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el8 ceph-noarch 23 k 2024-04-23T02:01:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el8 ceph-noarch 14 k 2024-04-23T02:01:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-04-23T02:01:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-04-23T02:01:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-04-23T02:01:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:13.552 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-23T02:01:13.552 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:01:13.552 INFO:teuthology.orchestra.run.smithi192.stdout:Install 6 Packages 2024-04-23T02:01:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:13.552 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 2.1 M 2024-04-23T02:01:13.553 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 8.5 M 2024-04-23T02:01:13.553 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-23T02:01:13.777 INFO:teuthology.orchestra.run.smithi192.stdout:(1/6): ceph-grafana-dashboards-18.2.1-0.el8.noa 449 kB/s | 23 kB 00:00 2024-04-23T02:01:13.794 INFO:teuthology.orchestra.run.smithi192.stdout:(2/6): ceph-prometheus-alerts-18.2.1-0.el8.noar 213 kB/s | 14 kB 00:00 2024-04-23T02:01:13.836 INFO:teuthology.orchestra.run.smithi192.stdout:(3/6): ceph-mgr-dashboard-18.2.1-0.el8.noarch.r 16 MB/s | 1.8 MB 00:00 2024-04-23T02:01:13.903 INFO:teuthology.orchestra.run.smithi192.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 340 kB/s | 43 kB 00:00 2024-04-23T02:01:14.003 INFO:teuthology.orchestra.run.smithi192.stdout:(5/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 161 kB/s | 34 kB 00:00 2024-04-23T02:01:14.070 INFO:teuthology.orchestra.run.smithi192.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 840 kB/s | 196 kB 00:00 2024-04-23T02:01:14.073 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:01:14.073 INFO:teuthology.orchestra.run.smithi192.stdout:Total 4.0 MB/s | 2.1 MB 00:00 2024-04-23T02:01:14.082 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-23T02:01:14.091 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-23T02:01:14.091 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-23T02:01:14.172 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-23T02:01:14.175 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-23T02:01:14.457 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-23T02:01:14.540 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:01:14.541 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:01:14.542 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-23T02:01:14.542 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:01:14.542 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-23T02:01:14.542 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el8 ceph-noarch 1.8 M 2024-04-23T02:01:14.542 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-23T02:01:14.542 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el8 ceph-noarch 23 k 2024-04-23T02:01:14.542 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el8 ceph-noarch 14 k 2024-04-23T02:01:14.542 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-04-23T02:01:14.542 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-04-23T02:01:14.543 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-04-23T02:01:14.543 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:14.543 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-23T02:01:14.543 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:01:14.543 INFO:teuthology.orchestra.run.smithi071.stdout:Install 6 Packages 2024-04-23T02:01:14.543 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:14.544 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 2.1 M 2024-04-23T02:01:14.544 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 8.5 M 2024-04-23T02:01:14.544 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-23T02:01:14.726 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-04-23T02:01:15.004 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-04-23T02:01:15.205 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-04-23T02:01:15.309 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el8.noarch 4/6 2024-04-23T02:01:15.663 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el8.noarch 5/6 2024-04-23T02:01:15.919 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el8.noarch 6/6 2024-04-23T02:01:16.205 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el8.noarch 6/6 2024-04-23T02:01:16.205 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el8.noarch 1/6 2024-04-23T02:01:16.205 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el8.noarch 2/6 2024-04-23T02:01:16.205 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el8.noarch 3/6 2024-04-23T02:01:16.205 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-04-23T02:01:16.206 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-04-23T02:01:16.609 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-04-23T02:01:16.609 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:16.609 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-23T02:01:16.609 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:18.2.1-0.el8.noarch 2024-04-23T02:01:16.609 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:18.2.1-0.el8.noarch 2024-04-23T02:01:16.609 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:18.2.1-0.el8.noarch 2024-04-23T02:01:16.609 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-04-23T02:01:16.609 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-04-23T02:01:16.609 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.4.1-12.el8.noarch 2024-04-23T02:01:16.610 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:16.610 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:01:16.728 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-23T02:01:17.245 INFO:teuthology.orchestra.run.smithi071.stdout:(1/6): ceph-grafana-dashboards-18.2.1-0.el8.noa 354 kB/s | 23 kB 00:00 2024-04-23T02:01:17.262 INFO:teuthology.orchestra.run.smithi071.stdout:(2/6): ceph-prometheus-alerts-18.2.1-0.el8.noar 178 kB/s | 14 kB 00:00 2024-04-23T02:01:17.312 INFO:teuthology.orchestra.run.smithi071.stdout:(3/6): ceph-mgr-dashboard-18.2.1-0.el8.noarch.r 13 MB/s | 1.8 MB 00:00 2024-04-23T02:01:17.379 INFO:teuthology.orchestra.run.smithi071.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 318 kB/s | 43 kB 00:00 2024-04-23T02:01:17.396 INFO:teuthology.orchestra.run.smithi071.stdout:(5/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 251 kB/s | 34 kB 00:00 2024-04-23T02:01:17.421 INFO:teuthology.orchestra.run.smithi071.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.8 MB/s | 196 kB 00:00 2024-04-23T02:01:17.424 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:01:17.424 INFO:teuthology.orchestra.run.smithi071.stdout:Total 732 kB/s | 2.1 MB 00:02 2024-04-23T02:01:17.435 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-23T02:01:17.445 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-23T02:01:17.445 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-23T02:01:17.455 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:02:03 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:01:17.528 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-23T02:01:17.532 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-23T02:01:17.810 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-23T02:01:18.051 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-04-23T02:01:18.346 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-04-23T02:01:18.556 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-04-23T02:01:18.685 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el8.noarch 4/6 2024-04-23T02:01:19.042 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el8.noarch 5/6 2024-04-23T02:01:19.089 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:01:19.091 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================================ 2024-04-23T02:01:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-04-23T02:01:19.091 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================================ 2024-04-23T02:01:19.091 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-23T02:01:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el8 ceph-noarch 7.4 M 2024-04-23T02:01:19.091 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-04-23T02:01:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-04-23T02:01:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-04-23T02:01:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-04-23T02:01:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-04-23T02:01:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-04-23T02:01:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-04-23T02:01:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-04-23T02:01:19.091 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:19.092 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-23T02:01:19.092 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================================ 2024-04-23T02:01:19.092 INFO:teuthology.orchestra.run.smithi192.stdout:Install 8 Packages 2024-04-23T02:01:19.092 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:19.092 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 36 M 2024-04-23T02:01:19.092 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 210 M 2024-04-23T02:01:19.092 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-23T02:01:19.271 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el8.noarch 6/6 2024-04-23T02:01:19.460 INFO:teuthology.orchestra.run.smithi192.stdout:(1/8): ceph-mgr-diskprediction-local-18.2.1-0.e 56 MB/s | 7.4 MB 00:00 2024-04-23T02:01:19.577 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el8.noarch 6/6 2024-04-23T02:01:19.578 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el8.noarch 1/6 2024-04-23T02:01:19.578 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el8.noarch 2/6 2024-04-23T02:01:19.578 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el8.noarch 3/6 2024-04-23T02:01:19.578 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-04-23T02:01:19.578 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-04-23T02:01:19.778 INFO:teuthology.orchestra.run.smithi192.stdout:(2/8): openblas-0.3.15-4.el8.x86_64.rpm 11 MB/s | 4.7 MB 00:00 2024-04-23T02:01:19.820 INFO:teuthology.orchestra.run.smithi192.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 5.3 MB/s | 225 kB 00:00 2024-04-23T02:01:19.968 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-04-23T02:01:19.968 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:19.968 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-23T02:01:19.968 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards-2:18.2.1-0.el8.noarch 2024-04-23T02:01:19.968 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard-2:18.2.1-0.el8.noarch 2024-04-23T02:01:19.968 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts-2:18.2.1-0.el8.noarch 2024-04-23T02:01:19.968 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-04-23T02:01:19.968 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-04-23T02:01:19.969 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes-2.4.1-12.el8.noarch 2024-04-23T02:01:19.969 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:19.969 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:01:20.118 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-23T02:01:20.187 INFO:teuthology.orchestra.run.smithi192.stdout:(4/8): python3-scipy-1.0.0-21.module_el8+762+77 39 MB/s | 14 MB 00:00 2024-04-23T02:01:20.396 INFO:teuthology.orchestra.run.smithi192.stdout:(5/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 4.6 MB/s | 4.9 MB 00:01 2024-04-23T02:01:20.446 INFO:teuthology.orchestra.run.smithi192.stdout:(6/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 3.8 MB/s | 3.7 MB 00:00 2024-04-23T02:01:20.849 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:05 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:01:20.893 INFO:teuthology.orchestra.run.smithi192.stdout:(7/8): libgfortran-8.5.0-21.el8.x86_64.rpm 915 kB/s | 645 kB 00:00 2024-04-23T02:01:20.929 INFO:teuthology.orchestra.run.smithi192.stdout:(8/8): libquadmath-8.5.0-21.el8.x86_64.rpm 323 kB/s | 172 kB 00:00 2024-04-23T02:01:20.930 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:01:20.930 INFO:teuthology.orchestra.run.smithi192.stdout:Total 20 MB/s | 36 MB 00:01 2024-04-23T02:01:21.164 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-23T02:01:21.183 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-23T02:01:21.183 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-23T02:01:21.589 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-23T02:01:21.592 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-23T02:01:22.259 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-23T02:01:22.435 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-04-23T02:01:22.488 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:01:22.490 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================================ 2024-04-23T02:01:22.490 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-23T02:01:22.490 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================================ 2024-04-23T02:01:22.490 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-23T02:01:22.490 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el8 ceph-noarch 7.4 M 2024-04-23T02:01:22.490 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-23T02:01:22.490 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-04-23T02:01:22.490 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-04-23T02:01:22.490 INFO:teuthology.orchestra.run.smithi071.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-04-23T02:01:22.490 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-04-23T02:01:22.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-04-23T02:01:22.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-04-23T02:01:22.491 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-04-23T02:01:22.491 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:22.491 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-23T02:01:22.491 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================================ 2024-04-23T02:01:22.491 INFO:teuthology.orchestra.run.smithi071.stdout:Install 8 Packages 2024-04-23T02:01:22.491 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:22.491 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 36 M 2024-04-23T02:01:22.492 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 210 M 2024-04-23T02:01:22.492 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-23T02:01:22.569 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-04-23T02:01:22.719 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-04-23T02:01:22.887 INFO:teuthology.orchestra.run.smithi071.stdout:(1/8): ceph-mgr-diskprediction-local-18.2.1-0.e 49 MB/s | 7.4 MB 00:00 2024-04-23T02:01:23.312 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-04-23T02:01:23.485 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-04-23T02:01:23.587 INFO:teuthology.orchestra.run.smithi071.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 5.7 MB/s | 4.9 MB 00:00 2024-04-23T02:01:23.637 INFO:teuthology.orchestra.run.smithi071.stdout:(3/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 5.0 MB/s | 3.7 MB 00:00 2024-04-23T02:01:23.687 INFO:teuthology.orchestra.run.smithi071.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 2.2 MB/s | 225 kB 00:00 2024-04-23T02:01:23.871 INFO:teuthology.orchestra.run.smithi071.stdout:(5/8): openblas-0.3.15-4.el8.x86_64.rpm 4.2 MB/s | 4.7 MB 00:01 2024-04-23T02:01:24.086 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-04-23T02:01:24.260 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-04-23T02:01:24.380 INFO:teuthology.orchestra.run.smithi071.stdout:(6/8): libquadmath-8.5.0-21.el8.x86_64.rpm 339 kB/s | 172 kB 00:00 2024-04-23T02:01:24.430 INFO:teuthology.orchestra.run.smithi071.stdout:(7/8): libgfortran-8.5.0-21.el8.x86_64.rpm 870 kB/s | 645 kB 00:00 2024-04-23T02:01:24.957 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-04-23T02:01:25.713 INFO:teuthology.orchestra.run.smithi071.stdout:(8/8): python3-scipy-1.0.0-21.module_el8+762+77 6.8 MB/s | 14 MB 00:02 2024-04-23T02:01:25.714 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:01:25.715 INFO:teuthology.orchestra.run.smithi071.stdout:Total 11 MB/s | 36 MB 00:03 2024-04-23T02:01:25.952 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-23T02:01:25.970 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-23T02:01:25.970 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-23T02:01:26.101 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-04-23T02:01:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-23T02:01:26.520 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-23T02:01:26.523 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-23T02:01:27.343 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-23T02:01:27.509 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-04-23T02:01:27.635 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-04-23T02:01:27.802 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-04-23T02:01:28.112 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-23T02:01:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-04-23T02:01:28.576 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-04-23T02:01:29.190 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-04-23T02:01:29.309 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-04-23T02:01:29.366 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-04-23T02:01:29.617 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el8.noarch 8/8 2024-04-23T02:01:29.939 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el8.noarch 8/8 2024-04-23T02:01:29.939 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el8.noarch 1/8 2024-04-23T02:01:29.939 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-04-23T02:01:29.939 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-04-23T02:01:29.940 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-04-23T02:01:29.940 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-04-23T02:01:29.940 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-04-23T02:01:29.940 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-04-23T02:01:29.999 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-04-23T02:01:30.363 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-04-23T02:01:30.375 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-04-23T02:01:30.375 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:30.375 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-23T02:01:30.375 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el8.noarch 2024-04-23T02:01:30.375 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-04-23T02:01:30.375 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-04-23T02:01:30.375 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.15-4.el8.x86_64 2024-04-23T02:01:30.375 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-04-23T02:01:30.375 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-04-23T02:01:30.376 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-04-23T02:01:30.376 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-04-23T02:01:30.376 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:30.376 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:01:30.496 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-rook 2024-04-23T02:01:30.601 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-23T02:01:31.224 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:02:17 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:01:32.380 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-23T02:01:32.868 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:01:32.870 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:01:32.870 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-04-23T02:01:32.870 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:01:32.870 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-23T02:01:32.870 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el8 ceph-noarch 50 k 2024-04-23T02:01:32.870 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-04-23T02:01:32.870 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-04-23T02:01:32.870 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-04-23T02:01:32.871 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-04-23T02:01:32.871 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-04-23T02:01:32.871 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-04-23T02:01:32.871 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-04-23T02:01:32.871 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-04-23T02:01:32.871 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-04-23T02:01:32.871 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-04-23T02:01:32.871 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-04-23T02:01:32.871 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-04-23T02:01:32.871 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-04-23T02:01:32.871 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:32.871 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-23T02:01:32.871 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:01:32.872 INFO:teuthology.orchestra.run.smithi192.stdout:Install 13 Packages 2024-04-23T02:01:32.872 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:32.872 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 1.9 M 2024-04-23T02:01:32.872 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 25 M 2024-04-23T02:01:32.872 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-23T02:01:33.603 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-04-23T02:01:33.833 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el8.noarch 8/8 2024-04-23T02:01:34.143 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el8.noarch 8/8 2024-04-23T02:01:34.143 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el8.noarch 1/8 2024-04-23T02:01:34.143 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-04-23T02:01:34.143 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-04-23T02:01:34.143 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-04-23T02:01:34.143 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-04-23T02:01:34.143 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-04-23T02:01:34.143 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-04-23T02:01:34.558 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-04-23T02:01:34.558 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:34.559 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-23T02:01:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el8.noarch 2024-04-23T02:01:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-04-23T02:01:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-04-23T02:01:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-0.3.15-4.el8.x86_64 2024-04-23T02:01:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-04-23T02:01:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-04-23T02:01:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-04-23T02:01:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-04-23T02:01:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:34.559 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:01:34.667 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr-rook 2024-04-23T02:01:35.407 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:20 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:01:37.045 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:01:37.047 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:01:37.047 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-23T02:01:37.047 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:01:37.047 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-23T02:01:37.047 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el8 ceph-noarch 50 k 2024-04-23T02:01:37.047 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-23T02:01:37.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-04-23T02:01:37.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-04-23T02:01:37.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-04-23T02:01:37.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-04-23T02:01:37.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-04-23T02:01:37.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-04-23T02:01:37.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-04-23T02:01:37.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-04-23T02:01:37.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-04-23T02:01:37.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-04-23T02:01:37.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-04-23T02:01:37.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-04-23T02:01:37.048 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:37.048 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-23T02:01:37.048 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:01:37.048 INFO:teuthology.orchestra.run.smithi071.stdout:Install 13 Packages 2024-04-23T02:01:37.048 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:37.049 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 1.9 M 2024-04-23T02:01:37.049 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 25 M 2024-04-23T02:01:37.049 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-23T02:01:39.828 INFO:teuthology.orchestra.run.smithi192.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el8.noarch.rpm 1.2 MB/s | 50 kB 00:00 2024-04-23T02:01:39.836 INFO:teuthology.orchestra.run.smithi071.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el8.noarch.rpm 1.6 MB/s | 50 kB 00:00 2024-04-23T02:01:39.878 INFO:teuthology.orchestra.run.smithi192.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 218 kB/s | 20 kB 00:00 2024-04-23T02:01:39.895 INFO:teuthology.orchestra.run.smithi192.stdout:(3/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 245 kB/s | 27 kB 00:00 2024-04-23T02:01:39.911 INFO:teuthology.orchestra.run.smithi071.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 189 kB/s | 20 kB 00:00 2024-04-23T02:01:39.928 INFO:teuthology.orchestra.run.smithi071.stdout:(3/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 217 kB/s | 27 kB 00:00 2024-04-23T02:01:39.945 INFO:teuthology.orchestra.run.smithi192.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2024-04-23T02:01:39.953 INFO:teuthology.orchestra.run.smithi071.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2024-04-23T02:01:39.970 INFO:teuthology.orchestra.run.smithi192.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 1.2 MB/s | 110 kB 00:00 2024-04-23T02:01:39.987 INFO:teuthology.orchestra.run.smithi071.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 1.4 MB/s | 110 kB 00:00 2024-04-23T02:01:40.104 INFO:teuthology.orchestra.run.smithi192.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 741 kB/s | 155 kB 00:00 2024-04-23T02:01:40.129 INFO:teuthology.orchestra.run.smithi192.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 232 kB/s | 43 kB 00:00 2024-04-23T02:01:40.205 INFO:teuthology.orchestra.run.smithi192.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 141 kB/s | 33 kB 00:00 2024-04-23T02:01:40.271 INFO:teuthology.orchestra.run.smithi071.stdout:(6/13): python3-cachetools-3.1.1-4.el8.noarch.r 116 kB/s | 33 kB 00:00 2024-04-23T02:01:40.305 INFO:teuthology.orchestra.run.smithi071.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 122 kB/s | 43 kB 00:00 2024-04-23T02:01:40.346 INFO:teuthology.orchestra.run.smithi192.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 67 kB/s | 16 kB 00:00 2024-04-23T02:01:40.413 INFO:teuthology.orchestra.run.smithi071.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 115 kB/s | 16 kB 00:00 2024-04-23T02:01:40.439 INFO:teuthology.orchestra.run.smithi192.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 263 kB/s | 81 kB 00:00 2024-04-23T02:01:40.464 INFO:teuthology.orchestra.run.smithi071.stdout:(9/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 289 kB/s | 155 kB 00:00 2024-04-23T02:01:40.506 INFO:teuthology.orchestra.run.smithi192.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 404 kB/s | 64 kB 00:00 2024-04-23T02:01:40.531 INFO:teuthology.orchestra.run.smithi192.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 667 kB/s | 61 kB 00:00 2024-04-23T02:01:40.648 INFO:teuthology.orchestra.run.smithi192.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 2.6 MB/s | 1.2 MB 00:00 2024-04-23T02:01:40.648 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:01:40.648 INFO:teuthology.orchestra.run.smithi192.stdout:Total 255 kB/s | 1.9 MB 00:07 2024-04-23T02:01:40.689 INFO:teuthology.orchestra.run.smithi071.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 212 kB/s | 81 kB 00:00 2024-04-23T02:01:40.704 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-23T02:01:40.720 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-23T02:01:40.720 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-23T02:01:40.810 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-23T02:01:40.812 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-23T02:01:40.814 INFO:teuthology.orchestra.run.smithi071.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 183 kB/s | 64 kB 00:00 2024-04-23T02:01:40.839 INFO:teuthology.orchestra.run.smithi071.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 408 kB/s | 61 kB 00:00 2024-04-23T02:01:40.940 INFO:teuthology.orchestra.run.smithi071.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 2.2 MB/s | 1.2 MB 00:00 2024-04-23T02:01:40.940 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:01:40.940 INFO:teuthology.orchestra.run.smithi071.stdout:Total 509 kB/s | 1.9 MB 00:03 2024-04-23T02:01:40.997 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-23T02:01:41.014 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-23T02:01:41.014 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-23T02:01:41.101 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-23T02:01:41.104 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-23T02:01:41.386 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-23T02:01:41.610 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-23T02:01:41.635 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-04-23T02:01:41.815 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-04-23T02:01:41.850 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-04-23T02:01:42.056 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-04-23T02:01:42.085 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-04-23T02:01:42.323 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-04-23T02:01:42.350 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-04-23T02:01:42.546 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-04-23T02:01:42.548 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-04-23T02:01:42.778 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-04-23T02:01:42.790 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-04-23T02:01:42.993 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-04-23T02:01:43.065 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-04-23T02:01:43.296 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-04-23T02:01:43.300 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-04-23T02:01:43.560 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-04-23T02:01:44.369 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-04-23T02:01:44.616 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-04-23T02:01:44.655 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-04-23T02:01:44.904 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-04-23T02:01:44.923 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-04-23T02:01:45.118 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-04-23T02:01:45.205 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-04-23T02:01:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el8.noarch 13/13 2024-04-23T02:01:45.425 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-04-23T02:01:45.614 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el8.noarch 13/13 2024-04-23T02:01:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el8.noarch 1/13 2024-04-23T02:01:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-04-23T02:01:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-04-23T02:01:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-04-23T02:01:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-04-23T02:01:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-04-23T02:01:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-04-23T02:01:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-04-23T02:01:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-04-23T02:01:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-04-23T02:01:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-04-23T02:01:45.615 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-04-23T02:01:45.622 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el8.noarch 13/13 2024-04-23T02:01:45.902 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el8.noarch 13/13 2024-04-23T02:01:45.903 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el8.noarch 1/13 2024-04-23T02:01:45.903 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-04-23T02:01:45.903 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-04-23T02:01:45.903 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-04-23T02:01:45.903 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-04-23T02:01:45.903 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-04-23T02:01:45.903 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-04-23T02:01:45.903 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-04-23T02:01:45.903 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-04-23T02:01:45.903 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-04-23T02:01:45.903 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-04-23T02:01:45.903 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-04-23T02:01:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-04-23T02:01:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:46.158 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-23T02:01:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:18.2.1-0.el8.noarch 2024-04-23T02:01:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-04-23T02:01:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-04-23T02:01:46.158 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-04-23T02:01:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-04-23T02:01:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-04-23T02:01:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-04-23T02:01:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-04-23T02:01:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-04-23T02:01:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-04-23T02:01:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-04-23T02:01:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el8.noarch 2024-04-23T02:01:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-04-23T02:01:46.159 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:46.159 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:01:46.354 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-04-23T02:01:46.354 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:46.355 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-23T02:01:46.355 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook-2:18.2.1-0.el8.noarch 2024-04-23T02:01:46.355 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-04-23T02:01:46.355 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-04-23T02:01:46.355 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-04-23T02:01:46.355 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-04-23T02:01:46.355 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-04-23T02:01:46.355 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-04-23T02:01:46.355 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-04-23T02:01:46.355 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-04-23T02:01:46.355 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-04-23T02:01:46.355 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-04-23T02:01:46.355 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa-4.9-2.el8.noarch 2024-04-23T02:01:46.355 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-04-23T02:01:46.356 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:46.356 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:01:46.358 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-cephadm 2024-04-23T02:01:46.465 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr-cephadm 2024-04-23T02:01:47.088 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:02:33 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:01:47.192 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:32 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:01:48.731 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:01:48.732 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:01:48.732 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-04-23T02:01:48.732 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:01:48.732 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-23T02:01:48.732 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el8 ceph-noarch 132 k 2024-04-23T02:01:48.732 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-04-23T02:01:48.732 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-04-23T02:01:48.732 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-04-23T02:01:48.732 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-04-23T02:01:48.732 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-04-23T02:01:48.733 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:48.733 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-23T02:01:48.733 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:01:48.733 INFO:teuthology.orchestra.run.smithi192.stdout:Install 5 Packages 2024-04-23T02:01:48.733 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:48.733 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 6.0 M 2024-04-23T02:01:48.733 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 26 M 2024-04-23T02:01:48.733 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-23T02:01:48.831 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:01:48.831 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:01:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-23T02:01:48.832 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:01:48.832 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-23T02:01:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el8 ceph-noarch 132 k 2024-04-23T02:01:48.832 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-23T02:01:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-04-23T02:01:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-04-23T02:01:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-04-23T02:01:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-04-23T02:01:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:48.832 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-23T02:01:48.832 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:01:48.832 INFO:teuthology.orchestra.run.smithi071.stdout:Install 5 Packages 2024-04-23T02:01:48.832 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:48.833 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 6.0 M 2024-04-23T02:01:48.833 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 26 M 2024-04-23T02:01:48.833 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-23T02:01:49.332 INFO:teuthology.orchestra.run.smithi192.stdout:(1/5): ceph-mgr-cephadm-18.2.1-0.el8.noarch.rpm 1.8 MB/s | 132 kB 00:00 2024-04-23T02:01:49.481 INFO:teuthology.orchestra.run.smithi192.stdout:(2/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 3.1 MB/s | 476 kB 00:00 2024-04-23T02:01:49.522 INFO:teuthology.orchestra.run.smithi192.stdout:(3/5): python3-natsort-7.1.1-2.el8.noarch.rpm 1.4 MB/s | 60 kB 00:00 2024-04-23T02:01:49.564 INFO:teuthology.orchestra.run.smithi192.stdout:(4/5): python3-jinja2-2.10.1-4.el8.noarch.rpm 1.9 MB/s | 578 kB 00:00 2024-04-23T02:01:49.873 INFO:teuthology.orchestra.run.smithi192.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 7.8 MB/s | 4.8 MB 00:00 2024-04-23T02:01:49.874 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:01:49.874 INFO:teuthology.orchestra.run.smithi192.stdout:Total 5.3 MB/s | 6.0 MB 00:01 2024-04-23T02:01:49.927 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-23T02:01:49.938 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-23T02:01:49.938 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-23T02:01:50.038 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-23T02:01:50.042 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-23T02:01:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-23T02:01:50.672 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2024-04-23T02:01:51.676 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-04-23T02:01:52.062 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2024-04-23T02:01:52.313 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 4/5 2024-04-23T02:01:52.522 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el8.noarch 5/5 2024-04-23T02:01:52.571 INFO:teuthology.orchestra.run.smithi071.stdout:(1/5): ceph-mgr-cephadm-18.2.1-0.el8.noarch.rpm 3.2 MB/s | 132 kB 00:00 2024-04-23T02:01:52.814 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el8.noarch 5/5 2024-04-23T02:01:52.815 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el8.noarch 1/5 2024-04-23T02:01:52.815 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2024-04-23T02:01:52.815 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/5 2024-04-23T02:01:52.815 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-04-23T02:01:53.005 INFO:teuthology.orchestra.run.smithi071.stdout:(2/5): python3-babel-2.5.1-7.el8.noarch.rpm 10 MB/s | 4.8 MB 00:00 2024-04-23T02:01:53.230 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2024-04-23T02:01:53.231 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:53.231 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-23T02:01:53.231 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:18.2.1-0.el8.noarch python3-asyncssh-2.7.0-2.el8.noarch 2024-04-23T02:01:53.231 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.5.1-7.el8.noarch python3-jinja2-2.10.1-4.el8.noarch 2024-04-23T02:01:53.231 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-04-23T02:01:53.231 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:53.231 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:01:53.322 INFO:teuthology.orchestra.run.smithi071.stdout:(3/5): python3-jinja2-2.10.1-4.el8.noarch.rpm 732 kB/s | 578 kB 00:00 2024-04-23T02:01:53.398 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-fuse 2024-04-23T02:01:53.530 INFO:teuthology.orchestra.run.smithi071.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 115 kB/s | 60 kB 00:00 2024-04-23T02:01:54.130 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:02:40 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:01:55.438 INFO:teuthology.orchestra.run.smithi071.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 166 kB/s | 476 kB 00:02 2024-04-23T02:01:55.441 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:01:55.441 INFO:teuthology.orchestra.run.smithi071.stdout:Total 930 kB/s | 6.0 MB 00:06 2024-04-23T02:01:55.494 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-23T02:01:55.506 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-23T02:01:55.506 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-23T02:01:55.617 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-23T02:01:55.620 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-23T02:01:55.750 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:01:55.750 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:01:55.751 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-04-23T02:01:55.751 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:01:55.751 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-23T02:01:55.751 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:18.2.1-0.el8 ceph 851 k 2024-04-23T02:01:55.751 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-04-23T02:01:55.751 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-04-23T02:01:55.752 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:55.752 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-23T02:01:55.752 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:01:55.752 INFO:teuthology.orchestra.run.smithi192.stdout:Install 2 Packages 2024-04-23T02:01:55.752 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:01:55.752 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 934 k 2024-04-23T02:01:55.753 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 3.0 M 2024-04-23T02:01:55.753 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-23T02:01:55.864 INFO:teuthology.orchestra.run.smithi192.stdout:(1/2): ceph-fuse-18.2.1-0.el8.x86_64.rpm 14 MB/s | 851 kB 00:00 2024-04-23T02:01:55.885 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-23T02:01:56.024 INFO:teuthology.orchestra.run.smithi192.stdout:(2/2): fuse-2.9.7-19.el8.x86_64.rpm 379 kB/s | 83 kB 00:00 2024-04-23T02:01:56.025 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:01:56.025 INFO:teuthology.orchestra.run.smithi192.stdout:Total 3.3 MB/s | 934 kB 00:00 2024-04-23T02:01:56.035 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-23T02:01:56.056 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-23T02:01:56.056 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-23T02:01:56.143 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-23T02:01:56.146 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-23T02:01:56.218 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2024-04-23T02:01:56.336 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-23T02:01:57.171 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-04-23T02:01:57.565 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2024-04-23T02:01:57.670 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.7-19.el8.x86_64 1/2 2024-04-23T02:01:57.885 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 4/5 2024-04-23T02:01:58.094 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el8.noarch 5/5 2024-04-23T02:01:58.374 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el8.noarch 5/5 2024-04-23T02:01:58.374 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el8.noarch 1/5 2024-04-23T02:01:58.374 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2024-04-23T02:01:58.374 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/5 2024-04-23T02:01:58.374 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-04-23T02:01:58.762 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2024-04-23T02:01:58.762 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:58.762 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-23T02:01:58.762 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm-2:18.2.1-0.el8.noarch python3-asyncssh-2.7.0-2.el8.noarch 2024-04-23T02:01:58.763 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel-2.5.1-7.el8.noarch python3-jinja2-2.10.1-4.el8.noarch 2024-04-23T02:01:58.763 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-04-23T02:01:58.763 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:01:58.763 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:01:58.896 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-fuse 2024-04-23T02:01:59.228 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:18.2.1-0.el8.x86_64 2/2 2024-04-23T02:01:59.624 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:44 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:01:59.637 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el8.x86_64 2/2 2024-04-23T02:01:59.637 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:18.2.1-0.el8.x86_64 1/2 2024-04-23T02:02:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 2/2 2024-04-23T02:02:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:00.018 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-23T02:02:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:18.2.1-0.el8.x86_64 fuse-2.9.7-19.el8.x86_64 2024-04-23T02:02:00.018 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:00.018 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:02:00.284 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librados-devel 2024-04-23T02:02:01.036 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:02:47 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:02:01.253 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:02:01.253 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:02:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-04-23T02:02:01.253 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:02:01.253 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-23T02:02:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse x86_64 2:18.2.1-0.el8 ceph 851 k 2024-04-23T02:02:01.254 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-23T02:02:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-04-23T02:02:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:01.254 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-23T02:02:01.254 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:02:01.254 INFO:teuthology.orchestra.run.smithi071.stdout:Install 2 Packages 2024-04-23T02:02:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:01.255 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 934 k 2024-04-23T02:02:01.255 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 3.0 M 2024-04-23T02:02:01.255 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-23T02:02:01.363 INFO:teuthology.orchestra.run.smithi071.stdout:(1/2): ceph-fuse-18.2.1-0.el8.x86_64.rpm 15 MB/s | 851 kB 00:00 2024-04-23T02:02:01.655 INFO:teuthology.orchestra.run.smithi071.stdout:(2/2): fuse-2.9.7-19.el8.x86_64.rpm 239 kB/s | 83 kB 00:00 2024-04-23T02:02:01.656 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:02:01.656 INFO:teuthology.orchestra.run.smithi071.stdout:Total 2.3 MB/s | 934 kB 00:00 2024-04-23T02:02:01.665 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-23T02:02:01.687 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-23T02:02:01.687 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-23T02:02:01.772 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-23T02:02:01.775 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-23T02:02:01.950 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-23T02:02:02.249 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : fuse-2.9.7-19.el8.x86_64 1/2 2024-04-23T02:02:02.477 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-fuse-2:18.2.1-0.el8.x86_64 2/2 2024-04-23T02:02:02.684 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:02:02.685 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:02:02.685 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-04-23T02:02:02.685 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:02:02.685 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-23T02:02:02.685 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:18.2.1-0.el8 ceph 139 k 2024-04-23T02:02:02.685 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:02.685 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-23T02:02:02.685 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:02:02.685 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-04-23T02:02:02.685 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:02.686 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 139 k 2024-04-23T02:02:02.686 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 513 k 2024-04-23T02:02:02.686 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-23T02:02:02.739 INFO:teuthology.orchestra.run.smithi192.stdout:librados-devel-18.2.1-0.el8.x86_64.rpm 2.6 MB/s | 139 kB 00:00 2024-04-23T02:02:02.739 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:02:02.740 INFO:teuthology.orchestra.run.smithi192.stdout:Total 2.5 MB/s | 139 kB 00:00 2024-04-23T02:02:02.740 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-23T02:02:02.745 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-23T02:02:02.745 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-23T02:02:02.794 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-23T02:02:02.797 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-23T02:02:02.851 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el8.x86_64 2/2 2024-04-23T02:02:02.852 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-fuse-2:18.2.1-0.el8.x86_64 1/2 2024-04-23T02:02:02.912 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-23T02:02:03.145 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:03.428 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:03.810 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 2/2 2024-04-23T02:02:03.810 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:03.810 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-23T02:02:03.810 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse-2:18.2.1-0.el8.x86_64 fuse-2.9.7-19.el8.x86_64 2024-04-23T02:02:03.810 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:03.810 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:02:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:03.841 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:03.842 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-23T02:02:03.842 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:18.2.1-0.el8.x86_64 2024-04-23T02:02:03.842 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:03.842 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:02:04.087 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install libcephfs2 2024-04-23T02:02:04.832 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:02:50 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:02:05.248 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install librados-devel 2024-04-23T02:02:06.003 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:50 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:02:06.422 INFO:teuthology.orchestra.run.smithi192.stdout:Package libcephfs2-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-23T02:02:06.470 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:02:06.471 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-23T02:02:06.471 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:02:06.566 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install libcephfs-devel 2024-04-23T02:02:07.300 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:02:53 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:02:07.649 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:02:07.649 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:02:07.649 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-04-23T02:02:07.649 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:02:07.649 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-23T02:02:07.649 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel x86_64 2:18.2.1-0.el8 ceph 139 k 2024-04-23T02:02:07.650 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:07.650 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-23T02:02:07.650 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:02:07.650 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-04-23T02:02:07.650 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:07.650 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 139 k 2024-04-23T02:02:07.650 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 513 k 2024-04-23T02:02:07.651 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-23T02:02:07.704 INFO:teuthology.orchestra.run.smithi071.stdout:librados-devel-18.2.1-0.el8.x86_64.rpm 2.5 MB/s | 139 kB 00:00 2024-04-23T02:02:07.705 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:02:07.705 INFO:teuthology.orchestra.run.smithi071.stdout:Total 2.5 MB/s | 139 kB 00:00 2024-04-23T02:02:07.705 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-23T02:02:07.710 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-23T02:02:07.711 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-23T02:02:07.759 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-23T02:02:07.762 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-23T02:02:07.869 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-23T02:02:08.100 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librados-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:08.373 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:08.807 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:08.807 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:08.807 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-23T02:02:08.807 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel-2:18.2.1-0.el8.x86_64 2024-04-23T02:02:08.807 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:08.807 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:02:08.944 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:02:08.945 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:02:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-04-23T02:02:08.945 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:02:08.945 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-23T02:02:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:18.2.1-0.el8 ceph 31 k 2024-04-23T02:02:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:08.945 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-23T02:02:08.945 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:02:08.945 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-04-23T02:02:08.945 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:08.946 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 31 k 2024-04-23T02:02:08.946 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 139 k 2024-04-23T02:02:08.946 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-23T02:02:08.992 INFO:teuthology.orchestra.run.smithi192.stdout:libcephfs-devel-18.2.1-0.el8.x86_64.rpm 675 kB/s | 31 kB 00:00 2024-04-23T02:02:08.993 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:02:08.993 INFO:teuthology.orchestra.run.smithi192.stdout:Total 657 kB/s | 31 kB 00:00 2024-04-23T02:02:08.993 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-23T02:02:08.996 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-23T02:02:08.996 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-23T02:02:09.022 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-23T02:02:09.025 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-23T02:02:09.036 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install libcephfs2 2024-04-23T02:02:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-23T02:02:09.319 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:09.606 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:09.775 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:54 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:02:10.001 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:10.001 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:10.001 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-23T02:02:10.001 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:18.2.1-0.el8.x86_64 2024-04-23T02:02:10.001 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:10.001 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:02:10.120 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librados2 2024-04-23T02:02:10.850 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:02:56 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:02:11.340 INFO:teuthology.orchestra.run.smithi071.stdout:Package libcephfs2-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-23T02:02:11.388 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:02:11.389 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-04-23T02:02:11.389 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:02:11.483 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install libcephfs-devel 2024-04-23T02:02:12.213 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:57 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:02:12.438 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-23T02:02:12.486 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:02:12.487 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-23T02:02:12.487 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:02:12.594 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librbd1 2024-04-23T02:02:13.328 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:02:59 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:02:13.873 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:02:13.873 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:02:13.874 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-04-23T02:02:13.874 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:02:13.874 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-23T02:02:13.874 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel x86_64 2:18.2.1-0.el8 ceph 31 k 2024-04-23T02:02:13.874 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:13.874 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-23T02:02:13.874 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:02:13.874 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-04-23T02:02:13.874 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:13.875 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 31 k 2024-04-23T02:02:13.875 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 139 k 2024-04-23T02:02:13.875 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-23T02:02:13.923 INFO:teuthology.orchestra.run.smithi071.stdout:libcephfs-devel-18.2.1-0.el8.x86_64.rpm 644 kB/s | 31 kB 00:00 2024-04-23T02:02:13.924 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:02:13.924 INFO:teuthology.orchestra.run.smithi071.stdout:Total 628 kB/s | 31 kB 00:00 2024-04-23T02:02:13.924 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-23T02:02:13.927 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-23T02:02:13.927 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-23T02:02:13.953 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-23T02:02:13.956 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-23T02:02:14.018 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-23T02:02:14.240 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:14.508 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:14.861 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:14.862 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:14.862 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-23T02:02:14.862 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel-2:18.2.1-0.el8.x86_64 2024-04-23T02:02:14.862 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:14.862 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:02:14.934 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-23T02:02:14.982 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:02:14.983 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-23T02:02:14.983 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:02:15.003 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install librados2 2024-04-23T02:02:15.222 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rados 2024-04-23T02:02:15.744 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:03:00 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:02:15.953 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:03:01 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:02:17.329 INFO:teuthology.orchestra.run.smithi071.stdout:Package librados2-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-23T02:02:17.377 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:02:17.378 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-04-23T02:02:17.378 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:02:17.488 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install librbd1 2024-04-23T02:02:17.516 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rados-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-23T02:02:17.566 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:02:17.567 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-23T02:02:17.567 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:02:17.664 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rgw 2024-04-23T02:02:18.211 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:03:03 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:02:18.396 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:03:04 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:02:19.802 INFO:teuthology.orchestra.run.smithi071.stdout:Package librbd1-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-23T02:02:19.850 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:02:19.851 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-04-23T02:02:19.851 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:02:19.949 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install python3-rados 2024-04-23T02:02:20.004 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rgw-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-23T02:02:20.054 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:02:20.055 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-23T02:02:20.055 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:02:20.163 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-cephfs 2024-04-23T02:02:20.672 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:03:05 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:02:20.892 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:03:06 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:02:22.244 INFO:teuthology.orchestra.run.smithi071.stdout:Package python3-rados-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-23T02:02:22.292 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:02:22.293 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-04-23T02:02:22.293 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:02:22.384 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install python3-rgw 2024-04-23T02:02:22.476 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-cephfs-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-23T02:02:22.524 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:02:22.526 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-23T02:02:22.526 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:02:22.654 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rbd 2024-04-23T02:02:23.114 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:03:08 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:02:23.384 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:03:09 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:02:24.692 INFO:teuthology.orchestra.run.smithi071.stdout:Package python3-rgw-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-23T02:02:24.740 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:02:24.741 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-04-23T02:02:24.741 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:02:24.841 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install python3-cephfs 2024-04-23T02:02:24.988 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rbd-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-23T02:02:25.036 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:02:25.037 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-04-23T02:02:25.037 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:02:25.130 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-fuse 2024-04-23T02:02:25.568 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:03:10 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:02:25.865 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:03:11 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:02:27.174 INFO:teuthology.orchestra.run.smithi071.stdout:Package python3-cephfs-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-23T02:02:27.222 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:02:27.224 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-04-23T02:02:27.224 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:02:27.329 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install python3-rbd 2024-04-23T02:02:27.494 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:02:27.495 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:02:27.495 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-04-23T02:02:27.495 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:02:27.495 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-23T02:02:27.495 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:18.2.1-0.el8 ceph 94 k 2024-04-23T02:02:27.495 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:27.495 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-23T02:02:27.495 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:02:27.496 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-04-23T02:02:27.496 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:27.496 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 94 k 2024-04-23T02:02:27.496 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 276 k 2024-04-23T02:02:27.496 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-23T02:02:27.540 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-fuse-18.2.1-0.el8.x86_64.rpm 2.1 MB/s | 94 kB 00:00 2024-04-23T02:02:27.541 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:02:27.541 INFO:teuthology.orchestra.run.smithi192.stdout:Total 2.1 MB/s | 94 kB 00:00 2024-04-23T02:02:27.541 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-23T02:02:27.551 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-23T02:02:27.551 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-23T02:02:27.601 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-23T02:02:27.604 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-23T02:02:27.731 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-23T02:02:27.933 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:28.060 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:03:13 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:02:28.229 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:28.645 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:28.645 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:28.645 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-23T02:02:28.645 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:18.2.1-0.el8.x86_64 2024-04-23T02:02:28.645 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:28.646 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:02:28.846 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-mirror 2024-04-23T02:02:29.589 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:03:15 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:02:29.638 INFO:teuthology.orchestra.run.smithi071.stdout:Package python3-rbd-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-23T02:02:29.688 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:02:29.689 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-04-23T02:02:29.689 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:02:29.787 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install rbd-fuse 2024-04-23T02:02:30.521 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:03:15 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:02:31.218 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:02:31.219 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:02:31.219 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-04-23T02:02:31.219 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:02:31.219 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-23T02:02:31.219 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:18.2.1-0.el8 ceph 3.4 M 2024-04-23T02:02:31.219 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:31.219 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-23T02:02:31.220 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:02:31.220 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-04-23T02:02:31.220 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:31.220 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 3.4 M 2024-04-23T02:02:31.220 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 16 M 2024-04-23T02:02:31.220 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-23T02:02:31.497 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-mirror-18.2.1-0.el8.x86_64.rpm 12 MB/s | 3.4 MB 00:00 2024-04-23T02:02:31.498 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:02:31.498 INFO:teuthology.orchestra.run.smithi192.stdout:Total 12 MB/s | 3.4 MB 00:00 2024-04-23T02:02:31.498 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-23T02:02:31.510 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-23T02:02:31.510 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-23T02:02:31.572 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-23T02:02:31.574 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-23T02:02:32.017 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-23T02:02:32.160 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:02:32.160 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:02:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-04-23T02:02:32.160 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:02:32.160 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-23T02:02:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse x86_64 2:18.2.1-0.el8 ceph 94 k 2024-04-23T02:02:32.161 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:32.161 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-23T02:02:32.161 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:02:32.161 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-04-23T02:02:32.161 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:32.161 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 94 k 2024-04-23T02:02:32.162 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 276 k 2024-04-23T02:02:32.162 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-23T02:02:32.211 INFO:teuthology.orchestra.run.smithi071.stdout:rbd-fuse-18.2.1-0.el8.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-04-23T02:02:32.211 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:02:32.211 INFO:teuthology.orchestra.run.smithi071.stdout:Total 1.8 MB/s | 94 kB 00:00 2024-04-23T02:02:32.212 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-23T02:02:32.222 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-23T02:02:32.222 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-23T02:02:32.234 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:32.270 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-23T02:02:32.273 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-23T02:02:32.392 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-23T02:02:32.602 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-fuse-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:32.905 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:33.031 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:33.032 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-23T02:02:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:18.2.1-0.el8.x86_64 2024-04-23T02:02:33.032 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:33.032 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:02:33.246 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-nbd 2024-04-23T02:02:33.312 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-fuse-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:33.312 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:33.313 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-23T02:02:33.313 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse-2:18.2.1-0.el8.x86_64 2024-04-23T02:02:33.313 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:33.313 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:02:33.531 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install rbd-mirror 2024-04-23T02:02:33.986 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:03:19 ago on Tue 23 Apr 2024 01:59:14 AM UTC. 2024-04-23T02:02:34.268 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:03:19 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:02:35.627 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-04-23T02:02:35.628 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:02:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-04-23T02:02:35.628 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:02:35.628 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-04-23T02:02:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:18.2.1-0.el8 ceph 181 k 2024-04-23T02:02:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:35.629 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-04-23T02:02:35.629 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-04-23T02:02:35.629 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-04-23T02:02:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:35.629 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 181 k 2024-04-23T02:02:35.629 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 568 k 2024-04-23T02:02:35.629 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-04-23T02:02:35.678 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-nbd-18.2.1-0.el8.x86_64.rpm 3.6 MB/s | 181 kB 00:00 2024-04-23T02:02:35.679 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:02:35.679 INFO:teuthology.orchestra.run.smithi192.stdout:Total 3.5 MB/s | 181 kB 00:00 2024-04-23T02:02:35.679 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-04-23T02:02:35.690 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-04-23T02:02:35.690 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-04-23T02:02:35.739 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-04-23T02:02:35.742 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-04-23T02:02:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-04-23T02:02:35.921 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:02:35.921 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:02:35.921 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-04-23T02:02:35.921 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:02:35.922 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-23T02:02:35.922 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror x86_64 2:18.2.1-0.el8 ceph 3.4 M 2024-04-23T02:02:35.922 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:35.922 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-23T02:02:35.922 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:02:35.922 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-04-23T02:02:35.922 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:35.923 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 3.4 M 2024-04-23T02:02:35.923 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 16 M 2024-04-23T02:02:35.923 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-23T02:02:36.030 INFO:teuthology.orchestra.run.smithi071.stdout:rbd-mirror-18.2.1-0.el8.x86_64.rpm 32 MB/s | 3.4 MB 00:00 2024-04-23T02:02:36.031 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:02:36.031 INFO:teuthology.orchestra.run.smithi071.stdout:Total 32 MB/s | 3.4 MB 00:00 2024-04-23T02:02:36.032 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-23T02:02:36.044 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-23T02:02:36.044 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-23T02:02:36.048 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:36.106 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-23T02:02:36.109 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-23T02:02:36.342 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:36.540 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-23T02:02:36.737 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-mirror-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:36.771 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:36.771 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:36.771 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-04-23T02:02:36.771 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:18.2.1-0.el8.x86_64 2024-04-23T02:02:36.771 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:02:36.771 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-04-23T02:02:37.024 DEBUG:teuthology.parallel:result is None 2024-04-23T02:02:37.152 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:37.582 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-mirror-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:37.583 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:37.583 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-23T02:02:37.583 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror-2:18.2.1-0.el8.x86_64 2024-04-23T02:02:37.583 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:37.583 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:02:37.778 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install rbd-nbd 2024-04-23T02:02:38.512 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:03:23 ago on Tue 23 Apr 2024 01:59:15 AM UTC. 2024-04-23T02:02:40.143 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-23T02:02:40.143 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:02:40.143 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-04-23T02:02:40.143 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:02:40.143 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-23T02:02:40.144 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd x86_64 2:18.2.1-0.el8 ceph 181 k 2024-04-23T02:02:40.144 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:40.144 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-23T02:02:40.144 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-23T02:02:40.144 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-04-23T02:02:40.144 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:40.144 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 181 k 2024-04-23T02:02:40.145 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 568 k 2024-04-23T02:02:40.145 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-23T02:02:40.201 INFO:teuthology.orchestra.run.smithi071.stdout:rbd-nbd-18.2.1-0.el8.x86_64.rpm 3.1 MB/s | 181 kB 00:00 2024-04-23T02:02:40.201 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:02:40.202 INFO:teuthology.orchestra.run.smithi071.stdout:Total 3.1 MB/s | 181 kB 00:00 2024-04-23T02:02:40.202 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-23T02:02:40.214 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-23T02:02:40.214 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-23T02:02:40.263 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-23T02:02:40.266 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-23T02:02:40.383 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-23T02:02:40.601 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-nbd-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:40.867 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:41.242 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-nbd-2:18.2.1-0.el8.x86_64 1/1 2024-04-23T02:02:41.242 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:41.242 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-23T02:02:41.242 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd-2:18.2.1-0.el8.x86_64 2024-04-23T02:02:41.242 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:02:41.242 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-23T02:02:41.478 DEBUG:teuthology.parallel:result is None 2024-04-23T02:02:41.479 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-23T02:02:41.479 INFO:teuthology.packaging:ref: None 2024-04-23T02:02:41.479 INFO:teuthology.packaging:tag: v18.2.1 2024-04-23T02:02:41.479 INFO:teuthology.packaging:branch: None 2024-04-23T02:02:41.479 INFO:teuthology.packaging:sha1: 646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T02:02:41.479 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-23T02:02:41.699 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T02:02:41.722 INFO:teuthology.orchestra.run.smithi071.stdout:18.2.1-0.el8 2024-04-23T02:02:41.722 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el8 2024-04-23T02:02:41.722 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-04-23T02:02:41.724 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-23T02:02:41.724 INFO:teuthology.packaging:ref: None 2024-04-23T02:02:41.724 INFO:teuthology.packaging:tag: v18.2.1 2024-04-23T02:02:41.724 INFO:teuthology.packaging:branch: None 2024-04-23T02:02:41.724 INFO:teuthology.packaging:sha1: 646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T02:02:41.724 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-23T02:02:41.864 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T02:02:41.892 INFO:teuthology.orchestra.run.smithi192.stdout:18.2.1-0.el8 2024-04-23T02:02:41.892 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el8 2024-04-23T02:02:41.892 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-04-23T02:02:41.895 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-23T02:02:41.896 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T02:02:41.896 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T02:02:41.925 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-23T02:02:41.925 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T02:02:41.966 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-23T02:02:41.966 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T02:02:41.966 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T02:02:41.992 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T02:02:42.062 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-23T02:02:42.062 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T02:02:42.094 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T02:02:42.165 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-23T02:02:42.165 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T02:02:42.166 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T02:02:42.192 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T02:02:42.258 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-23T02:02:42.258 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T02:02:42.287 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T02:02:42.356 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-23T02:02:42.356 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T02:02:42.356 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T02:02:42.383 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T02:02:42.449 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-23T02:02:42.450 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T02:02:42.478 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T02:02:42.547 INFO:teuthology.run_tasks:Running task print... 2024-04-23T02:02:42.557 INFO:teuthology.task.print:**** done install task... 2024-04-23T02:02:42.558 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-23T02:02:42.648 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.io/ceph/ceph:v18.2.1', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d'} 2024-04-23T02:02:42.648 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.1 2024-04-23T02:02:42.648 INFO:tasks.cephadm:Cluster fsid is 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T02:02:42.648 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-23T02:02:42.648 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-04-23T02:02:42.649 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi071': '172.21.15.71', 'mon.smithi192': '172.21.15.192'} 2024-04-23T02:02:42.649 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-23T02:02:42.649 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname $(hostname -s) 2024-04-23T02:02:42.676 DEBUG:teuthology.orchestra.run.smithi192:> sudo hostname $(hostname -s) 2024-04-23T02:02:42.705 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-23T02:02:42.705 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T02:02:42.928 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '646b8c7d293d8ec31a40c69f7b3237b19c1be24d', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/54930/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1237-g646b8c7d', 'node_name': '172.21.4.64+confusa02', 'job_name': 'ceph-dev-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1237.g646b8c7d'}, 'url': 'https://1.chacra.ceph.com/r/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-04-18 14:10:05.416092', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'reef', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/centos/8/flavors/default/', 'archs': ['source', 'x86_64', 'arm64'], 'distro': 'centos'}] 2024-04-23T02:02:43.066 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref reef, sha1 646b8c7d293d8ec31a40c69f7b3237b19c1be24d from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T02:02:43.068 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/centos/8/x86_64/flavors/default/cephadm 2024-04-23T02:02:43.068 DEBUG:teuthology.orchestra.run.smithi071:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T02:02:43.469 INFO:teuthology.orchestra.run.smithi071.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 Apr 23 02:02 /home/ubuntu/cephtest/cephadm 2024-04-23T02:02:43.470 DEBUG:teuthology.orchestra.run.smithi192:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/646b8c7d293d8ec31a40c69f7b3237b19c1be24d/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T02:02:44.036 INFO:teuthology.orchestra.run.smithi192.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 Apr 23 02:02 /home/ubuntu/cephtest/cephadm 2024-04-23T02:02:44.036 DEBUG:teuthology.orchestra.run.smithi071:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T02:02:44.059 DEBUG:teuthology.orchestra.run.smithi192:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T02:02:44.092 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.1 on all hosts... 2024-04-23T02:02:44.092 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-04-23T02:02:44.103 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-04-23T02:02:45.295 INFO:teuthology.orchestra.run.smithi071.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-04-23T02:02:45.320 INFO:teuthology.orchestra.run.smithi192.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-04-23T02:03:03.786 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:03:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-04-23T02:03:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-04-23T02:03:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: "repo_digests": [ 2024-04-23T02:03:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-04-23T02:03:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-04-23T02:03:03.787 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-04-23T02:03:03.787 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:03:04.092 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-04-23T02:03:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-04-23T02:03:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-04-23T02:03:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: "repo_digests": [ 2024-04-23T02:03:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-04-23T02:03:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-04-23T02:03:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-04-23T02:03:04.092 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-04-23T02:03:04.112 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph 2024-04-23T02:03:04.144 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph 2024-04-23T02:03:04.174 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /etc/ceph 2024-04-23T02:03:04.212 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /etc/ceph 2024-04-23T02:03:04.241 INFO:tasks.cephadm:Writing seed config... 2024-04-23T02:03:04.242 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-04-23T02:03:04.242 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-04-23T02:03:04.242 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-23T02:03:04.242 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-23T02:03:04.242 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-23T02:03:04.242 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-23T02:03:04.242 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-23T02:03:04.243 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-23T02:03:04.243 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-23T02:03:04.243 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-23T02:03:04.243 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-23T02:03:04.243 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-23T02:03:04.243 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-23T02:03:04.243 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-23T02:03:04.243 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-23T02:03:04.243 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-23T02:03:04.243 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-23T02:03:04.243 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-23T02:03:04.243 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-23T02:03:04.243 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-23T02:03:04.244 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-23T02:03:04.244 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-23T02:03:04.244 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-23T02:03:04.244 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-04-23T02:03:04.244 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-23T02:03:04.244 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-23T02:03:04.244 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-23T02:03:04.244 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-23T02:03:04.244 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-23T02:03:04.244 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-23T02:03:04.244 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-23T02:03:04.244 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-23T02:03:04.244 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-23T02:03:04.244 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-23T02:03:04.244 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-23T02:03:04.244 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-23T02:03:04.245 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-23T02:03:04.245 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-23T02:03:04.245 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-23T02:03:04.245 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-23T02:03:04.245 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T02:03:04.246 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-23T02:03:04.270 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 91f5470c-0115-11ef-bc93-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-23T02:03:04.271 DEBUG:teuthology.orchestra.run.smithi071:mon.smithi071> sudo journalctl -f -n 0 -u ceph-91f5470c-0115-11ef-bc93-c7b262605968@mon.smithi071.service 2024-04-23T02:03:04.313 INFO:tasks.cephadm:Bootstrapping... 2024-04-23T02:03:04.313 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 -v bootstrap --fsid 91f5470c-0115-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.71 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-23T02:03:04.339 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:-- Logs begin at Tue 2024-04-23 01:52:06 UTC. -- 2024-04-23T02:03:04.473 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-23T02:03:04.473 INFO:teuthology.orchestra.run.smithi071.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.1', '-v', 'bootstrap', '--fsid', '91f5470c-0115-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.71', '--skip-admin-label'] 2024-04-23T02:03:04.504 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-23T02:03:04.505 INFO:teuthology.orchestra.run.smithi071.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-23T02:03:04.505 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying podman|docker is present... 2024-04-23T02:03:04.535 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-23T02:03:04.535 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying lvm2 is present... 2024-04-23T02:03:04.535 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying time synchronization is in place... 2024-04-23T02:03:04.542 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-23T02:03:04.542 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-23T02:03:04.548 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-23T02:03:04.548 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-04-23T02:03:04.553 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout enabled 2024-04-23T02:03:04.559 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout active 2024-04-23T02:03:04.559 INFO:teuthology.orchestra.run.smithi071.stdout:Unit chronyd.service is enabled and running 2024-04-23T02:03:04.559 INFO:teuthology.orchestra.run.smithi071.stdout:Repeating the final host check... 2024-04-23T02:03:04.590 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-23T02:03:04.590 INFO:teuthology.orchestra.run.smithi071.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-23T02:03:04.590 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl is present 2024-04-23T02:03:04.591 INFO:teuthology.orchestra.run.smithi071.stdout:lvcreate is present 2024-04-23T02:03:04.596 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-23T02:03:04.596 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-23T02:03:04.602 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-23T02:03:04.602 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-04-23T02:03:04.607 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout enabled 2024-04-23T02:03:04.613 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout active 2024-04-23T02:03:04.613 INFO:teuthology.orchestra.run.smithi071.stdout:Unit chronyd.service is enabled and running 2024-04-23T02:03:04.613 INFO:teuthology.orchestra.run.smithi071.stdout:Host looks OK 2024-04-23T02:03:04.613 INFO:teuthology.orchestra.run.smithi071.stdout:Cluster fsid: 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T02:03:04.613 INFO:teuthology.orchestra.run.smithi071.stdout:Acquiring lock 139738791565800 on /run/cephadm/91f5470c-0115-11ef-bc93-c7b262605968.lock 2024-04-23T02:03:04.613 INFO:teuthology.orchestra.run.smithi071.stdout:Lock 139738791565800 acquired on /run/cephadm/91f5470c-0115-11ef-bc93-c7b262605968.lock 2024-04-23T02:03:04.614 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying IP 172.21.15.71 port 3300 ... 2024-04-23T02:03:04.614 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying IP 172.21.15.71 port 6789 ... 2024-04-23T02:03:04.614 INFO:teuthology.orchestra.run.smithi071.stdout:Base mon IP(s) is [172.21.15.71:3300, 172.21.15.71:6789], mon addrv is [v2:172.21.15.71:3300,v1:172.21.15.71:6789] 2024-04-23T02:03:04.618 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.71 metric 100 2024-04-23T02:03:04.618 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.71 metric 100 2024-04-23T02:03:04.621 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-23T02:03:04.621 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-23T02:03:04.621 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1778sec hoplimit 64 pref medium 2024-04-23T02:03:04.623 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-23T02:03:04.623 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-23T02:03:04.623 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-23T02:03:04.624 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-23T02:03:04.624 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2024-04-23T02:03:04.624 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-23T02:03:04.625 INFO:teuthology.orchestra.run.smithi071.stdout:Mon IP `172.21.15.71` is in CIDR network `172.21.0.0/20` 2024-04-23T02:03:04.625 INFO:teuthology.orchestra.run.smithi071.stdout:Mon IP `172.21.15.71` is in CIDR network `172.21.0.0/20` 2024-04-23T02:03:04.625 INFO:teuthology.orchestra.run.smithi071.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-23T02:03:04.625 INFO:teuthology.orchestra.run.smithi071.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-23T02:03:04.626 INFO:teuthology.orchestra.run.smithi071.stdout:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-04-23T02:03:05.498 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-04-23T02:03:05.499 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.1... 2024-04-23T02:03:05.499 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Getting image source signatures 2024-04-23T02:03:05.499 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Copying blob sha256:a733d3c618b71f19c168ebecd1953429dce2c1631835ca182e9551c36dce5989 2024-04-23T02:03:05.499 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Copying blob sha256:7feca07754707458c3945cf0062cf4dabc512f6d90fe1a9a1370b362b6011124 2024-04-23T02:03:05.499 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Copying config sha256:5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-04-23T02:03:05.499 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-23T02:03:06.681 INFO:teuthology.orchestra.run.smithi071.stdout:ceph: stdout ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-04-23T02:03:06.682 INFO:teuthology.orchestra.run.smithi071.stdout:Ceph version: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-04-23T02:03:06.682 INFO:teuthology.orchestra.run.smithi071.stdout:Extracting ceph user uid/gid from container image... 2024-04-23T02:03:07.912 INFO:teuthology.orchestra.run.smithi071.stdout:stat: stdout 167 167 2024-04-23T02:03:07.913 INFO:teuthology.orchestra.run.smithi071.stdout:Creating initial keys... 2024-04-23T02:03:09.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph-authtool: stdout AQDcFidmaMVfFRAAdmS8MFsd7WTPYNaqCSfCrA== 2024-04-23T02:03:10.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph-authtool: stdout AQDdFidmy2BjIRAA1Mb2g2HkF5M25n7/x/CAkA== 2024-04-23T02:03:11.484 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph-authtool: stdout AQDeFidmUvsbLhAAZ8DbxqA3ltuG9COwx2FrGw== 2024-04-23T02:03:11.485 INFO:teuthology.orchestra.run.smithi071.stdout:Creating initial monmap... 2024-04-23T02:03:12.608 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-23T02:03:12.608 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-04-23T02:03:12.608 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T02:03:12.609 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-23T02:03:12.609 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool for smithi071 [v2:172.21.15.71:3300,v1:172.21.15.71:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-23T02:03:12.609 INFO:teuthology.orchestra.run.smithi071.stdout:setting min_mon_release = pacific 2024-04-23T02:03:12.609 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: set fsid to 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T02:03:12.609 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-23T02:03:12.609 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:03:12.609 INFO:teuthology.orchestra.run.smithi071.stdout:Creating mon... 2024-04-23T02:03:13.743 INFO:teuthology.orchestra.run.smithi071.stdout:create mon.smithi071 on 2024-04-23T02:03:13.861 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-04-23T02:03:13.950 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-23T02:03:14.059 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-91f5470c-0115-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-91f5470c-0115-11ef-bc93-c7b262605968.target. 2024-04-23T02:03:14.059 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-91f5470c-0115-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-91f5470c-0115-11ef-bc93-c7b262605968.target. 2024-04-23T02:03:14.234 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-91f5470c-0115-11ef-bc93-c7b262605968@mon.smithi071 2024-04-23T02:03:14.234 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to reset failed state of unit ceph-91f5470c-0115-11ef-bc93-c7b262605968@mon.smithi071.service: Unit ceph-91f5470c-0115-11ef-bc93-c7b262605968@mon.smithi071.service not loaded. 2024-04-23T02:03:14.327 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-91f5470c-0115-11ef-bc93-c7b262605968.target.wants/ceph-91f5470c-0115-11ef-bc93-c7b262605968@mon.smithi071.service → /etc/systemd/system/ceph-91f5470c-0115-11ef-bc93-c7b262605968@.service. 2024-04-23T02:03:14.927 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T02:03:14.928 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-04-23T02:03:14.935 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T02:03:14.935 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-04-23T02:03:14.935 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-04-23T02:03:14.935 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to enable service . firewalld.service is not available 2024-04-23T02:03:14.935 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mon to start... 2024-04-23T02:03:14.935 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mon... 2024-04-23T02:03:15.183 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:14 smithi071 bash[39311]: 9490fa52eaed3881d345288bae1f1dfda28fad1222d6be426f457ee54120600a 2024-04-23T02:03:15.183 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:14 smithi071 systemd[1]: Started Ceph mon.smithi071 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:03:16.202 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout cluster: 2024-04-23T02:03:16.202 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout id: 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T02:03:16.202 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-23T02:03:16.202 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-04-23T02:03:16.202 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout services: 2024-04-23T02:03:16.202 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi071 (age 0.685539s) 2024-04-23T02:03:16.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-23T02:03:16.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-23T02:03:16.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-04-23T02:03:16.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout data: 2024-04-23T02:03:16.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-23T02:03:16.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-23T02:03:16.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-23T02:03:16.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout pgs: 2024-04-23T02:03:16.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-04-23T02:03:16.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.481+0000 7f608b7dd700 1 Processor -- start 2024-04-23T02:03:16.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.481+0000 7f608b7dd700 1 -- start start 2024-04-23T02:03:16.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.482+0000 7f608b7dd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6084106de0 0x7f60841091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:16.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.482+0000 7f608b7dd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60840744f0 con 0x7f6084106de0 2024-04-23T02:03:16.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.482+0000 7f6089579700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6084106de0 0x7f60841091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:16.204 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.482+0000 7f6089579700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6084106de0 0x7f60841091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58646/0 (socket says 172.21.15.71:58646) 2024-04-23T02:03:16.204 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.482+0000 7f6089579700 1 -- 172.21.15.71:0/228707575 learned_addr learned my addr 172.21.15.71:0/228707575 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:16.204 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.482+0000 7f6089579700 1 -- 172.21.15.71:0/228707575 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6084109710 con 0x7f6084106de0 2024-04-23T02:03:16.204 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.482+0000 7f6089579700 1 --2- 172.21.15.71:0/228707575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6084106de0 0x7f60841091d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f60740098a0 tx=0x7f6074009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65314f717c232ca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:16.204 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.483+0000 7f607bfff700 1 -- 172.21.15.71:0/228707575 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6074004180 con 0x7f6084106de0 2024-04-23T02:03:16.204 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.483+0000 7f607bfff700 1 -- 172.21.15.71:0/228707575 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60740042e0 con 0x7f6084106de0 2024-04-23T02:03:16.204 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.483+0000 7f607bfff700 1 -- 172.21.15.71:0/228707575 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60740036a0 con 0x7f6084106de0 2024-04-23T02:03:16.204 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.483+0000 7f608b7dd700 1 -- 172.21.15.71:0/228707575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6084106de0 msgr2=0x7f60841091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:16.204 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.483+0000 7f608b7dd700 1 --2- 172.21.15.71:0/228707575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6084106de0 0x7f60841091d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f60740098a0 tx=0x7f6074009bb0 comp rx=0 tx=0).stop 2024-04-23T02:03:16.204 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.483+0000 7f608b7dd700 1 -- 172.21.15.71:0/228707575 shutdown_connections 2024-04-23T02:03:16.204 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.483+0000 7f608b7dd700 1 --2- 172.21.15.71:0/228707575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6084106de0 0x7f60841091d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:16.204 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.483+0000 7f608b7dd700 1 -- 172.21.15.71:0/228707575 >> 172.21.15.71:0/228707575 conn(0x7f60841007f0 msgr2=0x7f6084102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:16.205 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.483+0000 7f608b7dd700 1 -- 172.21.15.71:0/228707575 shutdown_connections 2024-04-23T02:03:16.205 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.483+0000 7f608b7dd700 1 -- 172.21.15.71:0/228707575 wait complete. 2024-04-23T02:03:16.205 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.484+0000 7f608b7dd700 1 Processor -- start 2024-04-23T02:03:16.205 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.484+0000 7f608b7dd700 1 -- start start 2024-04-23T02:03:16.205 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.484+0000 7f608b7dd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6084106de0 0x7f60841997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:16.205 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.484+0000 7f608b7dd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6084199d30 con 0x7f6084106de0 2024-04-23T02:03:16.205 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.484+0000 7f6089579700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6084106de0 0x7f60841997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:16.205 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.484+0000 7f6089579700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6084106de0 0x7f60841997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58662/0 (socket says 172.21.15.71:58662) 2024-04-23T02:03:16.205 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.484+0000 7f6089579700 1 -- 172.21.15.71:0/302181888 learned_addr learned my addr 172.21.15.71:0/302181888 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:16.205 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.484+0000 7f6089579700 1 -- 172.21.15.71:0/302181888 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6074009550 con 0x7f6084106de0 2024-04-23T02:03:16.205 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:15 smithi071 ceph-mon[39349]: mon.smithi071 is new leader, mons smithi071 in quorum (ranks 0) 2024-04-23T02:03:16.206 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:15 smithi071 ceph-mon[39349]: monmap e1: 1 mons at {smithi071=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-23T02:03:16.206 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:15 smithi071 ceph-mon[39349]: fsmap 2024-04-23T02:03:16.206 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:15 smithi071 ceph-mon[39349]: osdmap e1: 0 total, 0 up, 0 in 2024-04-23T02:03:16.206 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:15 smithi071 ceph-mon[39349]: mgrmap e1: no daemons active 2024-04-23T02:03:16.206 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:15 smithi071 ceph-mon[39349]: from='client.? 172.21.15.71:0/302181888' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-23T02:03:16.208 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.485+0000 7f6089579700 1 --2- 172.21.15.71:0/302181888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6084106de0 0x7f60841997f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6084073930 tx=0x7f6074004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:16.208 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.485+0000 7f607a7fc700 1 -- 172.21.15.71:0/302181888 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6074003ce0 con 0x7f6084106de0 2024-04-23T02:03:16.208 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.485+0000 7f607a7fc700 1 -- 172.21.15.71:0/302181888 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6074020070 con 0x7f6084106de0 2024-04-23T02:03:16.208 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.485+0000 7f607a7fc700 1 -- 172.21.15.71:0/302181888 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6074011630 con 0x7f6084106de0 2024-04-23T02:03:16.208 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.485+0000 7f608b7dd700 1 -- 172.21.15.71:0/302181888 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6084199f30 con 0x7f6084106de0 2024-04-23T02:03:16.209 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.485+0000 7f608b7dd700 1 -- 172.21.15.71:0/302181888 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608419a350 con 0x7f6084106de0 2024-04-23T02:03:16.209 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.485+0000 7f607a7fc700 1 -- 172.21.15.71:0/302181888 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6074022030 con 0x7f6084106de0 2024-04-23T02:03:16.209 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.485+0000 7f607a7fc700 1 -- 172.21.15.71:0/302181888 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6074025d40 con 0x7f6084106de0 2024-04-23T02:03:16.209 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.486+0000 7f608b7dd700 1 -- 172.21.15.71:0/302181888 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6084193630 con 0x7f6084106de0 2024-04-23T02:03:16.209 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.488+0000 7f607a7fc700 1 -- 172.21.15.71:0/302181888 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f6074025480 con 0x7f6084106de0 2024-04-23T02:03:16.209 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.555+0000 7f608b7dd700 1 -- 172.21.15.71:0/302181888 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f608402cc70 con 0x7f6084106de0 2024-04-23T02:03:16.209 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.556+0000 7f607a7fc700 1 -- 172.21.15.71:0/302181888 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f6074011c20 con 0x7f6084106de0 2024-04-23T02:03:16.209 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.556+0000 7f608b7dd700 1 -- 172.21.15.71:0/302181888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6084106de0 msgr2=0x7f60841997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:16.209 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.556+0000 7f608b7dd700 1 --2- 172.21.15.71:0/302181888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6084106de0 0x7f60841997f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6084073930 tx=0x7f6074004750 comp rx=0 tx=0).stop 2024-04-23T02:03:16.209 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.556+0000 7f608b7dd700 1 -- 172.21.15.71:0/302181888 shutdown_connections 2024-04-23T02:03:16.209 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.556+0000 7f608b7dd700 1 --2- 172.21.15.71:0/302181888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6084106de0 0x7f60841997f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:16.209 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.556+0000 7f608b7dd700 1 -- 172.21.15.71:0/302181888 >> 172.21.15.71:0/302181888 conn(0x7f60841007f0 msgr2=0x7f6084102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:16.209 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.556+0000 7f608b7dd700 1 -- 172.21.15.71:0/302181888 shutdown_connections 2024-04-23T02:03:16.210 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:15.557+0000 7f608b7dd700 1 -- 172.21.15.71:0/302181888 wait complete. 2024-04-23T02:03:16.210 INFO:teuthology.orchestra.run.smithi071.stdout:mon is available 2024-04-23T02:03:16.210 INFO:teuthology.orchestra.run.smithi071.stdout:Assimilating anything we can from ceph.conf... 2024-04-23T02:03:17.083 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:16 smithi071 ceph-mon[39349]: from='client.? 172.21.15.71:0/428389674' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-23T02:03:17.084 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:16 smithi071 ceph-mon[39349]: from='client.? 172.21.15.71:0/428389674' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-23T02:03:17.370 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-04-23T02:03:17.376 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [global] 2024-04-23T02:03:17.377 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout fsid = 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T02:03:17.377 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.71:3300,v1:172.21.15.71:6789] 2024-04-23T02:03:17.377 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-23T02:03:17.377 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-23T02:03:17.377 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-23T02:03:17.377 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-23T02:03:17.377 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-04-23T02:03:17.377 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-23T02:03:17.377 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-23T02:03:17.377 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-04-23T02:03:17.377 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [osd] 2024-04-23T02:03:17.377 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-23T02:03:17.377 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-23T02:03:17.378 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-23T02:03:17.378 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-23T02:03:17.378 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-23T02:03:17.378 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.722+0000 7f511c99e700 1 Processor -- start 2024-04-23T02:03:17.378 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.722+0000 7f511c99e700 1 -- start start 2024-04-23T02:03:17.378 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.722+0000 7f511c99e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5118105230 0x7f5118105650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:17.378 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.722+0000 7f511c99e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5118105c20 con 0x7f5118105230 2024-04-23T02:03:17.378 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.722+0000 7f511659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5118105230 0x7f5118105650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:17.378 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.722+0000 7f511659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5118105230 0x7f5118105650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58666/0 (socket says 172.21.15.71:58666) 2024-04-23T02:03:17.378 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.722+0000 7f511659c700 1 -- 172.21.15.71:0/129900725 learned_addr learned my addr 172.21.15.71:0/129900725 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:17.378 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.723+0000 7f511659c700 1 -- 172.21.15.71:0/129900725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5118106430 con 0x7f5118105230 2024-04-23T02:03:17.378 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.723+0000 7f511659c700 1 --2- 172.21.15.71:0/129900725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5118105230 0x7f5118105650 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f51080098a0 tx=0x7f5108009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a01b7e6e5b6ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:17.378 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.723+0000 7f511559a700 1 -- 172.21.15.71:0/129900725 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5108004180 con 0x7f5118105230 2024-04-23T02:03:17.379 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.723+0000 7f511559a700 1 -- 172.21.15.71:0/129900725 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51080042e0 con 0x7f5118105230 2024-04-23T02:03:17.379 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.723+0000 7f511559a700 1 -- 172.21.15.71:0/129900725 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51080036a0 con 0x7f5118105230 2024-04-23T02:03:17.379 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.723+0000 7f511c99e700 1 -- 172.21.15.71:0/129900725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5118105230 msgr2=0x7f5118105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:17.379 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.723+0000 7f511c99e700 1 --2- 172.21.15.71:0/129900725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5118105230 0x7f5118105650 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f51080098a0 tx=0x7f5108009bb0 comp rx=0 tx=0).stop 2024-04-23T02:03:17.379 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.724+0000 7f511c99e700 1 -- 172.21.15.71:0/129900725 shutdown_connections 2024-04-23T02:03:17.379 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.724+0000 7f511c99e700 1 --2- 172.21.15.71:0/129900725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5118105230 0x7f5118105650 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:17.379 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.724+0000 7f511c99e700 1 -- 172.21.15.71:0/129900725 >> 172.21.15.71:0/129900725 conn(0x7f51181007f0 msgr2=0x7f5118102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:17.379 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.724+0000 7f511c99e700 1 -- 172.21.15.71:0/129900725 shutdown_connections 2024-04-23T02:03:17.379 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.724+0000 7f511c99e700 1 -- 172.21.15.71:0/129900725 wait complete. 2024-04-23T02:03:17.379 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.724+0000 7f511c99e700 1 Processor -- start 2024-04-23T02:03:17.379 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.724+0000 7f511c99e700 1 -- start start 2024-04-23T02:03:17.379 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.724+0000 7f511c99e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5118105230 0x7f5118199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:17.379 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.724+0000 7f511c99e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5118199d40 con 0x7f5118105230 2024-04-23T02:03:17.379 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.725+0000 7f511659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5118105230 0x7f5118199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:17.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.725+0000 7f511659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5118105230 0x7f5118199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58672/0 (socket says 172.21.15.71:58672) 2024-04-23T02:03:17.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.725+0000 7f511659c700 1 -- 172.21.15.71:0/428389674 learned_addr learned my addr 172.21.15.71:0/428389674 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:17.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.725+0000 7f511659c700 1 -- 172.21.15.71:0/428389674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5108009550 con 0x7f5118105230 2024-04-23T02:03:17.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.725+0000 7f511659c700 1 --2- 172.21.15.71:0/428389674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5118105230 0x7f5118199800 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5108004000 tx=0x7f5108004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:17.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.725+0000 7f51077fe700 1 -- 172.21.15.71:0/428389674 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5108003ce0 con 0x7f5118105230 2024-04-23T02:03:17.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.725+0000 7f51077fe700 1 -- 172.21.15.71:0/428389674 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5108020070 con 0x7f5118105230 2024-04-23T02:03:17.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.725+0000 7f511c99e700 1 -- 172.21.15.71:0/428389674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5118199f40 con 0x7f5118105230 2024-04-23T02:03:17.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.725+0000 7f51077fe700 1 -- 172.21.15.71:0/428389674 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5108011630 con 0x7f5118105230 2024-04-23T02:03:17.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.725+0000 7f511c99e700 1 -- 172.21.15.71:0/428389674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511819a360 con 0x7f5118105230 2024-04-23T02:03:17.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.725+0000 7f51077fe700 1 -- 172.21.15.71:0/428389674 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5108022030 con 0x7f5118105230 2024-04-23T02:03:17.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.725+0000 7f51077fe700 1 -- 172.21.15.71:0/428389674 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5108025d00 con 0x7f5118105230 2024-04-23T02:03:17.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.726+0000 7f511c99e700 1 -- 172.21.15.71:0/428389674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51181930d0 con 0x7f5118105230 2024-04-23T02:03:17.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.728+0000 7f51077fe700 1 -- 172.21.15.71:0/428389674 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f5108025480 con 0x7f5118105230 2024-04-23T02:03:17.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.794+0000 7f511c99e700 1 -- 172.21.15.71:0/428389674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f511802cef0 con 0x7f5118105230 2024-04-23T02:03:17.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.797+0000 7f51077fe700 1 -- 172.21.15.71:0/428389674 <== mon.0 v2:172.21.15.71:3300/0 7 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f5108011c20 con 0x7f5118105230 2024-04-23T02:03:17.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.797+0000 7f51077fe700 1 -- 172.21.15.71:0/428389674 <== mon.0 v2:172.21.15.71:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f5108039210 con 0x7f5118105230 2024-04-23T02:03:17.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.798+0000 7f511c99e700 1 -- 172.21.15.71:0/428389674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5118105230 msgr2=0x7f5118199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:17.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.798+0000 7f511c99e700 1 --2- 172.21.15.71:0/428389674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5118105230 0x7f5118199800 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5108004000 tx=0x7f5108004750 comp rx=0 tx=0).stop 2024-04-23T02:03:17.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.798+0000 7f511c99e700 1 -- 172.21.15.71:0/428389674 shutdown_connections 2024-04-23T02:03:17.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.798+0000 7f511c99e700 1 --2- 172.21.15.71:0/428389674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5118105230 0x7f5118199800 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:17.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.798+0000 7f511c99e700 1 -- 172.21.15.71:0/428389674 >> 172.21.15.71:0/428389674 conn(0x7f51181007f0 msgr2=0x7f5118073ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:17.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.798+0000 7f511c99e700 1 -- 172.21.15.71:0/428389674 shutdown_connections 2024-04-23T02:03:17.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:16.798+0000 7f511c99e700 1 -- 172.21.15.71:0/428389674 wait complete. 2024-04-23T02:03:17.381 INFO:teuthology.orchestra.run.smithi071.stdout:Generating new minimal ceph.conf... 2024-04-23T02:03:18.415 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:18 smithi071 ceph-mon[39349]: from='client.? 172.21.15.71:0/2870296930' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:03:18.697 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.957+0000 7f4ec2f70700 1 Processor -- start 2024-04-23T02:03:18.697 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.957+0000 7f4ec2f70700 1 -- start start 2024-04-23T02:03:18.697 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.957+0000 7f4ec2f70700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ebc106de0 0x7f4ebc1091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:18.697 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.957+0000 7f4ec2f70700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc079da0 con 0x7f4ebc106de0 2024-04-23T02:03:18.697 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.958+0000 7f4ec0d0c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ebc106de0 0x7f4ebc1091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:18.698 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.958+0000 7f4ec0d0c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ebc106de0 0x7f4ebc1091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58674/0 (socket says 172.21.15.71:58674) 2024-04-23T02:03:18.698 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.958+0000 7f4ec0d0c700 1 -- 172.21.15.71:0/3278766319 learned_addr learned my addr 172.21.15.71:0/3278766319 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:18.698 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.958+0000 7f4ec0d0c700 1 -- 172.21.15.71:0/3278766319 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc109710 con 0x7f4ebc106de0 2024-04-23T02:03:18.698 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.958+0000 7f4ec0d0c700 1 --2- 172.21.15.71:0/3278766319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ebc106de0 0x7f4ebc1091d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00098a0 tx=0x7f4eb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb48ddc86c918959 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:18.698 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.958+0000 7f4ebb7fe700 1 -- 172.21.15.71:0/3278766319 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4eb0004180 con 0x7f4ebc106de0 2024-04-23T02:03:18.699 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.958+0000 7f4ebb7fe700 1 -- 172.21.15.71:0/3278766319 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f4eb00042e0 con 0x7f4ebc106de0 2024-04-23T02:03:18.699 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.958+0000 7f4ebb7fe700 1 -- 172.21.15.71:0/3278766319 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4eb00119d0 con 0x7f4ebc106de0 2024-04-23T02:03:18.699 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.959+0000 7f4ec2f70700 1 -- 172.21.15.71:0/3278766319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ebc106de0 msgr2=0x7f4ebc1091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:18.699 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.959+0000 7f4ec2f70700 1 --2- 172.21.15.71:0/3278766319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ebc106de0 0x7f4ebc1091d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00098a0 tx=0x7f4eb0009bb0 comp rx=0 tx=0).stop 2024-04-23T02:03:18.699 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.959+0000 7f4ec2f70700 1 -- 172.21.15.71:0/3278766319 shutdown_connections 2024-04-23T02:03:18.699 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.959+0000 7f4ec2f70700 1 --2- 172.21.15.71:0/3278766319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ebc106de0 0x7f4ebc1091d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:18.700 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.959+0000 7f4ec2f70700 1 -- 172.21.15.71:0/3278766319 >> 172.21.15.71:0/3278766319 conn(0x7f4ebc1007d0 msgr2=0x7f4ebc102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:18.700 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.959+0000 7f4ec2f70700 1 -- 172.21.15.71:0/3278766319 shutdown_connections 2024-04-23T02:03:18.700 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.959+0000 7f4ec2f70700 1 -- 172.21.15.71:0/3278766319 wait complete. 2024-04-23T02:03:18.700 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.959+0000 7f4ec2f70700 1 Processor -- start 2024-04-23T02:03:18.700 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.959+0000 7f4ec2f70700 1 -- start start 2024-04-23T02:03:18.700 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.959+0000 7f4ec2f70700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ebc106de0 0x7f4ebc19c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:18.700 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.959+0000 7f4ec2f70700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc19c720 con 0x7f4ebc106de0 2024-04-23T02:03:18.700 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.960+0000 7f4ec0d0c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ebc106de0 0x7f4ebc19c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:18.700 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.960+0000 7f4ec0d0c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ebc106de0 0x7f4ebc19c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58690/0 (socket says 172.21.15.71:58690) 2024-04-23T02:03:18.700 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.960+0000 7f4ec0d0c700 1 -- 172.21.15.71:0/2870296930 learned_addr learned my addr 172.21.15.71:0/2870296930 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:18.700 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.960+0000 7f4ec0d0c700 1 -- 172.21.15.71:0/2870296930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb0009550 con 0x7f4ebc106de0 2024-04-23T02:03:18.700 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.960+0000 7f4ec0d0c700 1 --2- 172.21.15.71:0/2870296930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ebc106de0 0x7f4ebc19c1e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0011980 tx=0x7f4eb0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:18.700 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.960+0000 7f4eb9ffb700 1 -- 172.21.15.71:0/2870296930 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4eb0003b40 con 0x7f4ebc106de0 2024-04-23T02:03:18.701 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.960+0000 7f4eb9ffb700 1 -- 172.21.15.71:0/2870296930 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f4eb001a880 con 0x7f4ebc106de0 2024-04-23T02:03:18.701 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.960+0000 7f4ec2f70700 1 -- 172.21.15.71:0/2870296930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ebc19c920 con 0x7f4ebc106de0 2024-04-23T02:03:18.701 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.960+0000 7f4eb9ffb700 1 -- 172.21.15.71:0/2870296930 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4eb0019910 con 0x7f4ebc106de0 2024-04-23T02:03:18.701 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.960+0000 7f4ec2f70700 1 -- 172.21.15.71:0/2870296930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ebc19cd40 con 0x7f4ebc106de0 2024-04-23T02:03:18.701 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.960+0000 7f4eb9ffb700 1 -- 172.21.15.71:0/2870296930 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4eb002d440 con 0x7f4ebc106de0 2024-04-23T02:03:18.701 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.960+0000 7f4eb9ffb700 1 -- 172.21.15.71:0/2870296930 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4eb0024870 con 0x7f4ebc106de0 2024-04-23T02:03:18.701 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.961+0000 7f4ec2f70700 1 -- 172.21.15.71:0/2870296930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ebc0623c0 con 0x7f4ebc106de0 2024-04-23T02:03:18.701 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:17.963+0000 7f4eb9ffb700 1 -- 172.21.15.71:0/2870296930 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f4eb001adf0 con 0x7f4ebc106de0 2024-04-23T02:03:18.701 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:18.030+0000 7f4ec2f70700 1 -- 172.21.15.71:0/2870296930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f4ebc19f780 con 0x7f4ebc106de0 2024-04-23T02:03:18.701 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:18.030+0000 7f4eb9ffb700 1 -- 172.21.15.71:0/2870296930 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f4eb001a9f0 con 0x7f4ebc106de0 2024-04-23T02:03:18.702 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:18.031+0000 7f4ec2f70700 1 -- 172.21.15.71:0/2870296930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ebc106de0 msgr2=0x7f4ebc19c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:18.702 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:18.031+0000 7f4ec2f70700 1 --2- 172.21.15.71:0/2870296930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ebc106de0 0x7f4ebc19c1e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0011980 tx=0x7f4eb0011a60 comp rx=0 tx=0).stop 2024-04-23T02:03:18.702 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:18.031+0000 7f4ec2f70700 1 -- 172.21.15.71:0/2870296930 shutdown_connections 2024-04-23T02:03:18.702 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:18.031+0000 7f4ec2f70700 1 --2- 172.21.15.71:0/2870296930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ebc106de0 0x7f4ebc19c1e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:18.702 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:18.031+0000 7f4ec2f70700 1 -- 172.21.15.71:0/2870296930 >> 172.21.15.71:0/2870296930 conn(0x7f4ebc1007d0 msgr2=0x7f4ebc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:18.702 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:18.031+0000 7f4ec2f70700 1 -- 172.21.15.71:0/2870296930 shutdown_connections 2024-04-23T02:03:18.702 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:18.031+0000 7f4ec2f70700 1 -- 172.21.15.71:0/2870296930 wait complete. 2024-04-23T02:03:18.702 INFO:teuthology.orchestra.run.smithi071.stdout:Restarting the monitor... 2024-04-23T02:03:18.951 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:18 smithi071 systemd[1]: Stopping Ceph mon.smithi071 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T02:03:19.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:18 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[39326]: 2024-04-23T02:03:18.947+0000 7fc1dccc9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi071 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T02:03:19.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:18 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[39326]: 2024-04-23T02:03:18.947+0000 7fc1dccc9700 -1 mon.smithi071@0(leader) e1 *** Got Signal Terminated *** 2024-04-23T02:03:19.828 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:19 smithi071 bash[39676]: ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071 2024-04-23T02:03:20.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 systemd[1]: ceph-91f5470c-0115-11ef-bc93-c7b262605968@mon.smithi071.service: Succeeded. 2024-04-23T02:03:20.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 systemd[1]: Stopped Ceph mon.smithi071 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:03:20.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 systemd[1]: Starting Ceph mon.smithi071 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T02:03:20.586 INFO:teuthology.orchestra.run.smithi071.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-23T02:03:20.800 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: set uid:gid to 167:167 (ceph:ceph) 2024-04-23T02:03:20.800 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable), process ceph-mon, pid 7 2024-04-23T02:03:20.800 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: pidfile_write: ignore empty --pid-file 2024-04-23T02:03:20.800 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: load: jerasure load: lrc 2024-04-23T02:03:20.800 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: RocksDB version: 7.9.2 2024-04-23T02:03:20.800 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Git sha 0 2024-04-23T02:03:20.801 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Compile date 2023-12-11 22:07:34 2024-04-23T02:03:20.801 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: DB SUMMARY 2024-04-23T02:03:20.801 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: DB Session ID: ISGGH52I0AGJMK1XRCEE 2024-04-23T02:03:20.801 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: CURRENT file: CURRENT 2024-04-23T02:03:20.801 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: IDENTITY file: IDENTITY 2024-04-23T02:03:20.801 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-23T02:03:20.801 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi071/store.db dir, Total Num: 1, files: 000008.sst 2024-04-23T02:03:20.801 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi071/store.db: 000009.log size: 96924 ; 2024-04-23T02:03:20.801 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.error_if_exists: 0 2024-04-23T02:03:20.801 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.create_if_missing: 0 2024-04-23T02:03:20.801 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.paranoid_checks: 1 2024-04-23T02:03:20.801 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-23T02:03:20.801 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-23T02:03:20.802 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-23T02:03:20.802 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.env: 0x55d7fb49c720 2024-04-23T02:03:20.802 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.fs: PosixFileSystem 2024-04-23T02:03:20.802 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.info_log: 0x55d7fd56f2e0 2024-04-23T02:03:20.802 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_file_opening_threads: 16 2024-04-23T02:03:20.802 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.statistics: (nil) 2024-04-23T02:03:20.802 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.use_fsync: 0 2024-04-23T02:03:20.802 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_log_file_size: 0 2024-04-23T02:03:20.802 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-23T02:03:20.802 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-23T02:03:20.802 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.keep_log_file_num: 1000 2024-04-23T02:03:20.802 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.recycle_log_file_num: 0 2024-04-23T02:03:20.802 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.allow_fallocate: 1 2024-04-23T02:03:20.802 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.allow_mmap_reads: 0 2024-04-23T02:03:20.803 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.allow_mmap_writes: 0 2024-04-23T02:03:20.803 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.use_direct_reads: 0 2024-04-23T02:03:20.803 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-23T02:03:20.803 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.create_missing_column_families: 0 2024-04-23T02:03:20.803 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.db_log_dir: 2024-04-23T02:03:20.803 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.wal_dir: 2024-04-23T02:03:20.803 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-23T02:03:20.803 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-23T02:03:20.803 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-23T02:03:20.803 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-23T02:03:20.803 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-23T02:03:20.803 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-23T02:03:20.803 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.advise_random_on_open: 1 2024-04-23T02:03:20.803 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.db_write_buffer_size: 0 2024-04-23T02:03:20.803 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.write_buffer_manager: 0x55d7fc7e0140 2024-04-23T02:03:20.804 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-23T02:03:20.804 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-23T02:03:20.804 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-23T02:03:20.804 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.rate_limiter: (nil) 2024-04-23T02:03:20.804 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-23T02:03:20.804 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.wal_recovery_mode: 2 2024-04-23T02:03:20.804 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.enable_thread_tracking: 0 2024-04-23T02:03:20.804 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.enable_pipelined_write: 0 2024-04-23T02:03:20.804 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.unordered_write: 0 2024-04-23T02:03:20.804 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-23T02:03:20.804 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-23T02:03:20.804 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-23T02:03:20.804 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-23T02:03:20.804 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.row_cache: None 2024-04-23T02:03:20.804 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.wal_filter: None 2024-04-23T02:03:20.805 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-23T02:03:20.805 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.allow_ingest_behind: 0 2024-04-23T02:03:20.805 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.two_write_queues: 0 2024-04-23T02:03:20.805 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.manual_wal_flush: 0 2024-04-23T02:03:20.805 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.wal_compression: 0 2024-04-23T02:03:20.805 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.atomic_flush: 0 2024-04-23T02:03:20.805 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-23T02:03:20.805 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-23T02:03:20.805 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-23T02:03:20.805 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.log_readahead_size: 0 2024-04-23T02:03:20.805 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-23T02:03:20.805 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.best_efforts_recovery: 0 2024-04-23T02:03:20.805 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-23T02:03:20.805 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-23T02:03:20.805 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.allow_data_in_errors: 0 2024-04-23T02:03:20.805 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.db_host_id: __hostname__ 2024-04-23T02:03:20.806 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-23T02:03:20.807 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_background_jobs: 2 2024-04-23T02:03:20.807 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_background_compactions: -1 2024-04-23T02:03:20.807 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_subcompactions: 1 2024-04-23T02:03:20.807 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-23T02:03:20.807 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-23T02:03:20.807 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-23T02:03:20.807 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_total_wal_size: 0 2024-04-23T02:03:20.808 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-23T02:03:20.808 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-23T02:03:20.808 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-23T02:03:20.808 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-23T02:03:20.808 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_open_files: -1 2024-04-23T02:03:20.808 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.bytes_per_sync: 0 2024-04-23T02:03:20.808 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-23T02:03:20.808 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-23T02:03:20.808 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compaction_readahead_size: 0 2024-04-23T02:03:20.808 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_background_flushes: -1 2024-04-23T02:03:20.808 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Compression algorithms supported: 2024-04-23T02:03:20.808 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-23T02:03:20.808 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: kZSTD supported: 0 2024-04-23T02:03:20.808 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: kXpressCompression supported: 0 2024-04-23T02:03:20.809 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: kLZ4HCCompression supported: 1 2024-04-23T02:03:20.809 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: kZlibCompression supported: 1 2024-04-23T02:03:20.809 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: kSnappyCompression supported: 1 2024-04-23T02:03:20.809 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: kLZ4Compression supported: 1 2024-04-23T02:03:20.809 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: kBZip2Compression supported: 0 2024-04-23T02:03:20.809 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-23T02:03:20.809 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-23T02:03:20.809 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi071/store.db/MANIFEST-000010 2024-04-23T02:03:20.809 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-23T02:03:20.809 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-23T02:03:20.809 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.merge_operator: 2024-04-23T02:03:20.809 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compaction_filter: None 2024-04-23T02:03:20.809 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compaction_filter_factory: None 2024-04-23T02:03:20.809 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.sst_partitioner_factory: None 2024-04-23T02:03:20.810 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-23T02:03:20.810 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-23T02:03:20.810 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d7fd56f3e0) 2024-04-23T02:03:20.810 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: cache_index_and_filter_blocks: 1 2024-04-23T02:03:20.810 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-23T02:03:20.810 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-23T02:03:20.810 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: pin_top_level_index_and_filter: 1 2024-04-23T02:03:20.810 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: index_type: 0 2024-04-23T02:03:20.810 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: data_block_index_type: 0 2024-04-23T02:03:20.810 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: index_shortening: 1 2024-04-23T02:03:20.810 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-23T02:03:20.810 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: checksum: 4 2024-04-23T02:03:20.810 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: no_block_cache: 0 2024-04-23T02:03:20.810 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: block_cache: 0x55d7fc881350 2024-04-23T02:03:20.811 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: block_cache_name: BinnedLRUCache 2024-04-23T02:03:20.811 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: block_cache_options: 2024-04-23T02:03:20.811 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: capacity : 536870912 2024-04-23T02:03:20.811 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: num_shard_bits : 4 2024-04-23T02:03:20.811 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: strict_capacity_limit : 0 2024-04-23T02:03:20.811 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: high_pri_pool_ratio: 0.000 2024-04-23T02:03:20.811 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: block_cache_compressed: (nil) 2024-04-23T02:03:20.811 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: persistent_cache: (nil) 2024-04-23T02:03:20.811 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: block_size: 4096 2024-04-23T02:03:20.811 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: block_size_deviation: 10 2024-04-23T02:03:20.811 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: block_restart_interval: 16 2024-04-23T02:03:20.811 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: index_block_restart_interval: 1 2024-04-23T02:03:20.811 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: metadata_block_size: 4096 2024-04-23T02:03:20.811 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: partition_filters: 0 2024-04-23T02:03:20.811 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: use_delta_encoding: 1 2024-04-23T02:03:20.812 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: filter_policy: bloomfilter 2024-04-23T02:03:20.812 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: whole_key_filtering: 1 2024-04-23T02:03:20.812 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: verify_compression: 0 2024-04-23T02:03:20.812 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: read_amp_bytes_per_bit: 0 2024-04-23T02:03:20.812 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: format_version: 5 2024-04-23T02:03:20.812 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: enable_index_compression: 1 2024-04-23T02:03:20.812 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: block_align: 0 2024-04-23T02:03:20.812 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: max_auto_readahead_size: 262144 2024-04-23T02:03:20.812 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: prepopulate_block_cache: 0 2024-04-23T02:03:20.812 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: initial_auto_readahead_size: 8192 2024-04-23T02:03:20.812 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: num_file_reads_for_auto_readahead: 2 2024-04-23T02:03:20.812 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.write_buffer_size: 33554432 2024-04-23T02:03:20.812 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_write_buffer_number: 2 2024-04-23T02:03:20.812 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compression: NoCompression 2024-04-23T02:03:20.812 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.bottommost_compression: Disabled 2024-04-23T02:03:20.813 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.prefix_extractor: nullptr 2024-04-23T02:03:20.813 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-23T02:03:20.813 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.num_levels: 7 2024-04-23T02:03:20.813 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-23T02:03:20.813 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-23T02:03:20.813 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-23T02:03:20.813 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-23T02:03:20.813 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-23T02:03:20.813 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-23T02:03:20.813 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-23T02:03:20.813 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-23T02:03:20.813 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-23T02:03:20.813 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-23T02:03:20.814 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-23T02:03:20.814 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-23T02:03:20.814 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-23T02:03:20.814 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compression_opts.level: 32767 2024-04-23T02:03:20.814 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compression_opts.strategy: 0 2024-04-23T02:03:20.814 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-23T02:03:20.814 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-23T02:03:20.814 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-23T02:03:20.814 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-23T02:03:20.814 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compression_opts.enabled: false 2024-04-23T02:03:20.814 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-23T02:03:20.814 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-23T02:03:20.814 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-23T02:03:20.814 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-23T02:03:20.815 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.target_file_size_base: 67108864 2024-04-23T02:03:20.815 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-23T02:03:20.815 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-23T02:03:20.815 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-23T02:03:20.815 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-23T02:03:20.815 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-23T02:03:20.815 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-23T02:03:20.815 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-23T02:03:20.815 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-23T02:03:20.815 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-23T02:03:20.815 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-23T02:03:20.815 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-23T02:03:20.815 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-23T02:03:20.815 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-23T02:03:20.816 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-23T02:03:20.816 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.arena_block_size: 1048576 2024-04-23T02:03:20.816 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-23T02:03:20.816 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-23T02:03:20.816 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.disable_auto_compactions: 0 2024-04-23T02:03:20.816 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-23T02:03:20.816 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-23T02:03:20.816 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-23T02:03:20.816 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-23T02:03:20.816 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-23T02:03:20.816 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-23T02:03:20.816 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-23T02:03:20.816 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-23T02:03:20.816 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-23T02:03:20.817 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-23T02:03:20.817 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.table_properties_collectors: 2024-04-23T02:03:20.817 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.inplace_update_support: 0 2024-04-23T02:03:20.817 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-23T02:03:20.817 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-23T02:03:20.817 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-23T02:03:20.817 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-23T02:03:20.817 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.bloom_locality: 0 2024-04-23T02:03:20.817 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.max_successive_merges: 0 2024-04-23T02:03:20.817 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-23T02:03:20.817 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.paranoid_file_checks: 0 2024-04-23T02:03:20.817 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.force_consistency_checks: 1 2024-04-23T02:03:20.817 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.report_bg_io_stats: 0 2024-04-23T02:03:20.817 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.ttl: 2592000 2024-04-23T02:03:20.817 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-23T02:03:20.818 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-23T02:03:20.818 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-23T02:03:20.818 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.enable_blob_files: false 2024-04-23T02:03:20.818 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.min_blob_size: 0 2024-04-23T02:03:20.818 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.blob_file_size: 268435456 2024-04-23T02:03:20.818 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-23T02:03:20.818 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-23T02:03:20.818 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-23T02:03:20.818 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-23T02:03:20.818 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-23T02:03:20.818 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.blob_file_starting_level: 0 2024-04-23T02:03:20.818 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-23T02:03:20.818 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi071/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-23T02:03:20.818 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-23T02:03:20.819 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 41362561-8ff1-4644-9c70-3e52d3ce932e 2024-04-23T02:03:20.819 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713837800543234, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-23T02:03:20.819 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-23T02:03:20.819 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713837800544250, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 92371, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 308, "table_properties": {"data_size": 90388, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13225, "raw_average_key_size": 49, "raw_value_size": 83127, "raw_average_value_size": 311, "num_data_blocks": 11, "num_entries": 267, "num_filter_entries": 267, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713837800, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "41362561-8ff1-4644-9c70-3e52d3ce932e", "db_session_id": "ISGGH52I0AGJMK1XRCEE", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-23T02:03:20.819 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713837800544333, "job": 1, "event": "recovery_finished"} 2024-04-23T02:03:20.819 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-23T02:03:20.819 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi071/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-23T02:03:20.819 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d7fc91e000 2024-04-23T02:03:20.819 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: DB pointer 0x55d7fc90a000 2024-04-23T02:03:20.819 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-23T02:03:20.819 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-23T02:03:20.819 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: ** DB Stats ** 2024-04-23T02:03:20.819 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T02:03:20.819 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-23T02:03:20.820 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T02:03:20.820 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T02:03:20.820 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-23T02:03:20.820 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T02:03:20.820 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T02:03:20.820 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: 2024-04-23T02:03:20.820 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: ** Compaction Stats [default] ** 2024-04-23T02:03:20.820 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T02:03:20.820 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-23T02:03:20.820 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: L0 2/0 92.08 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T02:03:20.820 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Sum 2/0 92.08 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T02:03:20.820 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T02:03:20.820 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: 2024-04-23T02:03:20.820 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: ** Compaction Stats [default] ** 2024-04-23T02:03:20.820 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T02:03:20.821 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-23T02:03:20.821 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 133.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T02:03:20.821 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: 2024-04-23T02:03:20.821 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-23T02:03:20.821 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: 2024-04-23T02:03:20.821 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T02:03:20.821 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-23T02:03:20.821 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-23T02:03:20.821 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-23T02:03:20.821 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-23T02:03:20.821 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-23T02:03:20.821 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Cumulative compaction: 0.00 GB write, 29.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T02:03:20.821 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Interval compaction: 0.00 GB write, 29.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T02:03:20.821 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-23T02:03:20.822 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Block cache BinnedLRUCache@0x55d7fc881350#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-23T02:03:20.822 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-04-23T02:03:20.822 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: 2024-04-23T02:03:20.822 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-23T02:03:20.822 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: starting mon.smithi071 rank 0 at public addrs [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] at bind addrs [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi071 fsid 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T02:03:20.822 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: mon.smithi071@-1(???) e1 preinit fsid 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T02:03:20.822 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: mon.smithi071@-1(???).mds e1 new map 2024-04-23T02:03:20.822 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: mon.smithi071@-1(???).mds e1 print_map 2024-04-23T02:03:20.822 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: e1 2024-04-23T02:03:20.822 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:03:20.822 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:03:20.822 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: legacy client fscid: -1 2024-04-23T02:03:20.822 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: 2024-04-23T02:03:20.823 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: No filesystems configured 2024-04-23T02:03:20.823 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: mon.smithi071@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-23T02:03:20.823 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: mon.smithi071@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T02:03:20.823 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: mon.smithi071@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T02:03:20.823 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: mon.smithi071@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T02:03:20.823 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: mon.smithi071@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-23T02:03:20.823 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: expand_channel_meta expand map: {default=false} 2024-04-23T02:03:20.823 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: expand_channel_meta from 'false' to 'false' 2024-04-23T02:03:20.823 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: expand_channel_meta expanded map: {default=false} 2024-04-23T02:03:20.823 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: expand_channel_meta expand map: {default=info} 2024-04-23T02:03:20.823 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: expand_channel_meta from 'info' to 'info' 2024-04-23T02:03:20.823 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: expand_channel_meta expanded map: {default=info} 2024-04-23T02:03:20.823 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: expand_channel_meta expand map: {default=daemon} 2024-04-23T02:03:20.823 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: expand_channel_meta from 'daemon' to 'daemon' 2024-04-23T02:03:20.824 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: expand_channel_meta expanded map: {default=daemon} 2024-04-23T02:03:20.824 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: expand_channel_meta expand map: {default=debug} 2024-04-23T02:03:20.824 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: expand_channel_meta from 'debug' to 'debug' 2024-04-23T02:03:20.824 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: expand_channel_meta expanded map: {default=debug} 2024-04-23T02:03:20.824 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 bash[39800]: ff40abe63fd9d8923a652db46e796988ddc9bf2e05d520845a15dcf7ad3fdcb4 2024-04-23T02:03:20.824 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 systemd[1]: Started Ceph mon.smithi071 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:03:20.824 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: mon.smithi071 is new leader, mons smithi071 in quorum (ranks 0) 2024-04-23T02:03:20.824 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: monmap e1: 1 mons at {smithi071=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-23T02:03:20.824 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: fsmap 2024-04-23T02:03:20.824 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: osdmap e1: 0 total, 0 up, 0 in 2024-04-23T02:03:20.824 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:20 smithi071 ceph-mon[39837]: mgrmap e1: no daemons active 2024-04-23T02:03:21.786 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.113+0000 7f7e54fac700 1 Processor -- start 2024-04-23T02:03:21.787 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.113+0000 7f7e54fac700 1 -- start start 2024-04-23T02:03:21.787 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.113+0000 7f7e54fac700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e50106e00 0x7f7e501091f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:21.787 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.113+0000 7f7e54fac700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e50079d80 con 0x7f7e50106e00 2024-04-23T02:03:21.787 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.114+0000 7f7e4e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e50106e00 0x7f7e501091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:21.787 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.114+0000 7f7e4e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e50106e00 0x7f7e501091f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58696/0 (socket says 172.21.15.71:58696) 2024-04-23T02:03:21.787 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.114+0000 7f7e4e59c700 1 -- 172.21.15.71:0/80409039 learned_addr learned my addr 172.21.15.71:0/80409039 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:21.787 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.114+0000 7f7e4e59c700 1 -- 172.21.15.71:0/80409039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e50109730 con 0x7f7e50106e00 2024-04-23T02:03:21.787 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.114+0000 7f7e4e59c700 1 --2- 172.21.15.71:0/80409039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e50106e00 0x7f7e501091f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7e440098a0 tx=0x7f7e44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd68ee63f270d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:21.787 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.114+0000 7f7e4d59a700 1 -- 172.21.15.71:0/80409039 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7e44004180 con 0x7f7e50106e00 2024-04-23T02:03:21.787 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.114+0000 7f7e4d59a700 1 -- 172.21.15.71:0/80409039 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f7e440042e0 con 0x7f7e50106e00 2024-04-23T02:03:21.787 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.114+0000 7f7e4d59a700 1 -- 172.21.15.71:0/80409039 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7e440119d0 con 0x7f7e50106e00 2024-04-23T02:03:21.787 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.115+0000 7f7e54fac700 1 -- 172.21.15.71:0/80409039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e50106e00 msgr2=0x7f7e501091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:21.788 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.115+0000 7f7e54fac700 1 --2- 172.21.15.71:0/80409039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e50106e00 0x7f7e501091f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7e440098a0 tx=0x7f7e44009bb0 comp rx=0 tx=0).stop 2024-04-23T02:03:21.788 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.115+0000 7f7e54fac700 1 -- 172.21.15.71:0/80409039 shutdown_connections 2024-04-23T02:03:21.788 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.115+0000 7f7e54fac700 1 --2- 172.21.15.71:0/80409039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e50106e00 0x7f7e501091f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:21.788 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.115+0000 7f7e54fac700 1 -- 172.21.15.71:0/80409039 >> 172.21.15.71:0/80409039 conn(0x7f7e501007d0 msgr2=0x7f7e50102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:21.788 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.115+0000 7f7e54fac700 1 -- 172.21.15.71:0/80409039 shutdown_connections 2024-04-23T02:03:21.788 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.115+0000 7f7e54fac700 1 -- 172.21.15.71:0/80409039 wait complete. 2024-04-23T02:03:21.788 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.116+0000 7f7e54fac700 1 Processor -- start 2024-04-23T02:03:21.788 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.116+0000 7f7e54fac700 1 -- start start 2024-04-23T02:03:21.788 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.116+0000 7f7e54fac700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e50106e00 0x7f7e50199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:21.788 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.116+0000 7f7e54fac700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5019a510 con 0x7f7e50106e00 2024-04-23T02:03:21.788 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.116+0000 7f7e4e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e50106e00 0x7f7e50199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:21.788 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.116+0000 7f7e4e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e50106e00 0x7f7e50199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58700/0 (socket says 172.21.15.71:58700) 2024-04-23T02:03:21.788 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.116+0000 7f7e4e59c700 1 -- 172.21.15.71:0/92258869 learned_addr learned my addr 172.21.15.71:0/92258869 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:21.788 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.116+0000 7f7e4e59c700 1 -- 172.21.15.71:0/92258869 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e44009550 con 0x7f7e50106e00 2024-04-23T02:03:21.788 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.116+0000 7f7e4e59c700 1 --2- 172.21.15.71:0/92258869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e50106e00 0x7f7e50199fd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7e44006af0 tx=0x7f7e44011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:21.789 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.116+0000 7f7e3f7fe700 1 -- 172.21.15.71:0/92258869 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7e44024430 con 0x7f7e50106e00 2024-04-23T02:03:21.789 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.116+0000 7f7e3f7fe700 1 -- 172.21.15.71:0/92258869 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f7e440249e0 con 0x7f7e50106e00 2024-04-23T02:03:21.789 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.116+0000 7f7e54fac700 1 -- 172.21.15.71:0/92258869 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5019a710 con 0x7f7e50106e00 2024-04-23T02:03:21.789 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.117+0000 7f7e3f7fe700 1 -- 172.21.15.71:0/92258869 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7e4401aa00 con 0x7f7e50106e00 2024-04-23T02:03:21.789 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.117+0000 7f7e54fac700 1 -- 172.21.15.71:0/92258869 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5019ab30 con 0x7f7e50106e00 2024-04-23T02:03:21.789 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.117+0000 7f7e3f7fe700 1 -- 172.21.15.71:0/92258869 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7e4402d3f0 con 0x7f7e50106e00 2024-04-23T02:03:21.789 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.117+0000 7f7e3f7fe700 1 -- 172.21.15.71:0/92258869 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7e44019830 con 0x7f7e50106e00 2024-04-23T02:03:21.789 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.118+0000 7f7e54fac700 1 -- 172.21.15.71:0/92258869 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e500623c0 con 0x7f7e50106e00 2024-04-23T02:03:21.789 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.120+0000 7f7e3f7fe700 1 -- 172.21.15.71:0/92258869 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f7e44024b50 con 0x7f7e50106e00 2024-04-23T02:03:21.789 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.185+0000 7f7e54fac700 1 -- 172.21.15.71:0/92258869 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f7e5019d570 con 0x7f7e50106e00 2024-04-23T02:03:21.789 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.188+0000 7f7e3f7fe700 1 -- 172.21.15.71:0/92258869 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7e44024cf0 con 0x7f7e50106e00 2024-04-23T02:03:21.789 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.188+0000 7f7e54fac700 1 -- 172.21.15.71:0/92258869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e50106e00 msgr2=0x7f7e50199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:21.789 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.188+0000 7f7e54fac700 1 --2- 172.21.15.71:0/92258869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e50106e00 0x7f7e50199fd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7e44006af0 tx=0x7f7e44011a30 comp rx=0 tx=0).stop 2024-04-23T02:03:21.790 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.188+0000 7f7e54fac700 1 -- 172.21.15.71:0/92258869 shutdown_connections 2024-04-23T02:03:21.790 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.188+0000 7f7e54fac700 1 --2- 172.21.15.71:0/92258869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e50106e00 0x7f7e50199fd0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:21.790 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.188+0000 7f7e54fac700 1 -- 172.21.15.71:0/92258869 >> 172.21.15.71:0/92258869 conn(0x7f7e501007d0 msgr2=0x7f7e50101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:21.790 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.188+0000 7f7e54fac700 1 -- 172.21.15.71:0/92258869 shutdown_connections 2024-04-23T02:03:21.790 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:21.188+0000 7f7e54fac700 1 -- 172.21.15.71:0/92258869 wait complete. 2024-04-23T02:03:21.790 INFO:teuthology.orchestra.run.smithi071.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-23T02:03:21.813 INFO:teuthology.orchestra.run.smithi071.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-23T02:03:21.813 INFO:teuthology.orchestra.run.smithi071.stdout:Creating mgr... 2024-04-23T02:03:21.814 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-23T02:03:21.814 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-23T02:03:21.814 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying port 0.0.0.0:8443 ... 2024-04-23T02:03:22.018 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-91f5470c-0115-11ef-bc93-c7b262605968@mgr.smithi071.ukdjvn 2024-04-23T02:03:22.019 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to reset failed state of unit ceph-91f5470c-0115-11ef-bc93-c7b262605968@mgr.smithi071.ukdjvn.service: Unit ceph-91f5470c-0115-11ef-bc93-c7b262605968@mgr.smithi071.ukdjvn.service not loaded. 2024-04-23T02:03:22.106 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-91f5470c-0115-11ef-bc93-c7b262605968.target.wants/ceph-91f5470c-0115-11ef-bc93-c7b262605968@mgr.smithi071.ukdjvn.service → /etc/systemd/system/ceph-91f5470c-0115-11ef-bc93-c7b262605968@.service. 2024-04-23T02:03:22.437 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:22 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/92258869' entity='client.admin' 2024-04-23T02:03:22.698 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T02:03:22.698 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-04-23T02:03:22.706 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T02:03:22.706 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-04-23T02:03:22.706 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-04-23T02:03:22.706 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to enable service . firewalld.service is not available 2024-04-23T02:03:22.714 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T02:03:22.714 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-04-23T02:03:22.722 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T02:03:22.722 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-04-23T02:03:22.722 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-04-23T02:03:22.722 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-04-23T02:03:22.723 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr to start... 2024-04-23T02:03:22.723 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr... 2024-04-23T02:03:23.707 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:23 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1467939346' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T02:03:24.021 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-04-23T02:03:24.021 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-04-23T02:03:24.021 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsid": "91f5470c-0115-11ef-bc93-c7b262605968", 2024-04-23T02:03:24.021 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T02:03:24.021 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T02:03:24.021 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T02:03:24.021 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T02:03:24.021 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:24.021 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T02:03:24.022 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T02:03:24.022 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 0 2024-04-23T02:03:24.022 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-04-23T02:03:24.022 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T02:03:24.022 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "smithi071" 2024-04-23T02:03:24.022 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-04-23T02:03:24.022 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-23T02:03:24.022 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T02:03:24.022 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T02:03:24.022 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-23T02:03:24.022 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T02:03:24.022 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:24.023 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T02:03:24.023 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T02:03:24.023 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T02:03:24.023 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T02:03:24.023 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T02:03:24.023 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T02:03:24.023 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T02:03:24.023 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T02:03:24.023 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:24.023 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T02:03:24.023 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T02:03:24.024 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T02:03:24.024 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T02:03:24.024 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T02:03:24.024 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T02:03:24.024 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T02:03:24.024 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T02:03:24.024 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T02:03:24.024 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:24.024 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T02:03:24.024 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T02:03:24.024 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T02:03:24.024 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T02:03:24.025 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:24.025 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T02:03:24.025 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-23T02:03:24.025 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T02:03:24.025 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T02:03:24.025 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T02:03:24.025 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T02:03:24.025 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T02:03:24.025 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-04-23T02:03:24.025 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T02:03:24.025 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:24.025 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T02:03:24.026 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T02:03:24.026 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T02:03:14.872527+0000", 2024-04-23T02:03:24.026 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T02:03:24.026 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:24.026 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T02:03:24.026 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-04-23T02:03:24.026 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.316+0000 7f99c65a8700 1 Processor -- start 2024-04-23T02:03:24.026 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.316+0000 7f99c65a8700 1 -- start start 2024-04-23T02:03:24.026 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.316+0000 7f99c65a8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c0104600 0x7f99c0104a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:24.026 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.316+0000 7f99c65a8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c0104ff0 con 0x7f99c0104600 2024-04-23T02:03:24.026 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.317+0000 7f99bffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c0104600 0x7f99c0104a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:24.027 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.317+0000 7f99bffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c0104600 0x7f99c0104a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58738/0 (socket says 172.21.15.71:58738) 2024-04-23T02:03:24.027 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.317+0000 7f99bffff700 1 -- 172.21.15.71:0/653705993 learned_addr learned my addr 172.21.15.71:0/653705993 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:24.027 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.317+0000 7f99bffff700 1 -- 172.21.15.71:0/653705993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c0105130 con 0x7f99c0104600 2024-04-23T02:03:24.027 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.317+0000 7f99bffff700 1 --2- 172.21.15.71:0/653705993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c0104600 0x7f99c0104a20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f99b00098a0 tx=0x7f99b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27ac3e5c9becb49 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:24.027 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.317+0000 7f99beffd700 1 -- 172.21.15.71:0/653705993 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99b0004180 con 0x7f99c0104600 2024-04-23T02:03:24.027 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.317+0000 7f99beffd700 1 -- 172.21.15.71:0/653705993 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f99b00042e0 con 0x7f99c0104600 2024-04-23T02:03:24.027 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.317+0000 7f99beffd700 1 -- 172.21.15.71:0/653705993 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99b00119d0 con 0x7f99c0104600 2024-04-23T02:03:24.027 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.318+0000 7f99c65a8700 1 -- 172.21.15.71:0/653705993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c0104600 msgr2=0x7f99c0104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:24.027 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.318+0000 7f99c65a8700 1 --2- 172.21.15.71:0/653705993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c0104600 0x7f99c0104a20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f99b00098a0 tx=0x7f99b0009bb0 comp rx=0 tx=0).stop 2024-04-23T02:03:24.027 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.318+0000 7f99c65a8700 1 -- 172.21.15.71:0/653705993 shutdown_connections 2024-04-23T02:03:24.027 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.318+0000 7f99c65a8700 1 --2- 172.21.15.71:0/653705993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c0104600 0x7f99c0104a20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:24.027 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.318+0000 7f99c65a8700 1 -- 172.21.15.71:0/653705993 >> 172.21.15.71:0/653705993 conn(0x7f99c00fff40 msgr2=0x7f99c0102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:24.028 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.318+0000 7f99c65a8700 1 -- 172.21.15.71:0/653705993 shutdown_connections 2024-04-23T02:03:24.028 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.318+0000 7f99c65a8700 1 -- 172.21.15.71:0/653705993 wait complete. 2024-04-23T02:03:24.028 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.319+0000 7f99c65a8700 1 Processor -- start 2024-04-23T02:03:24.028 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.319+0000 7f99c65a8700 1 -- start start 2024-04-23T02:03:24.028 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.319+0000 7f99c65a8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c0104600 0x7f99c0199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:24.028 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.319+0000 7f99c65a8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c019a4f0 con 0x7f99c0104600 2024-04-23T02:03:24.028 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.319+0000 7f99bffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c0104600 0x7f99c0199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:24.028 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.319+0000 7f99bffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c0104600 0x7f99c0199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58740/0 (socket says 172.21.15.71:58740) 2024-04-23T02:03:24.028 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.320+0000 7f99bffff700 1 -- 172.21.15.71:0/1467939346 learned_addr learned my addr 172.21.15.71:0/1467939346 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:24.028 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.320+0000 7f99bffff700 1 -- 172.21.15.71:0/1467939346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b0009550 con 0x7f99c0104600 2024-04-23T02:03:24.029 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.320+0000 7f99bffff700 1 --2- 172.21.15.71:0/1467939346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c0104600 0x7f99c0199fb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f99b0006af0 tx=0x7f99b0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:24.029 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.320+0000 7f99bd7fa700 1 -- 172.21.15.71:0/1467939346 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99b0024430 con 0x7f99c0104600 2024-04-23T02:03:24.029 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.320+0000 7f99bd7fa700 1 -- 172.21.15.71:0/1467939346 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f99b00249e0 con 0x7f99c0104600 2024-04-23T02:03:24.029 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.320+0000 7f99c65a8700 1 -- 172.21.15.71:0/1467939346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c019a750 con 0x7f99c0104600 2024-04-23T02:03:24.029 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.320+0000 7f99bd7fa700 1 -- 172.21.15.71:0/1467939346 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99b001a970 con 0x7f99c0104600 2024-04-23T02:03:24.029 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.320+0000 7f99c65a8700 1 -- 172.21.15.71:0/1467939346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c019abd0 con 0x7f99c0104600 2024-04-23T02:03:24.029 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.320+0000 7f99bd7fa700 1 -- 172.21.15.71:0/1467939346 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f99b002d3f0 con 0x7f99c0104600 2024-04-23T02:03:24.029 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.320+0000 7f99bd7fa700 1 -- 172.21.15.71:0/1467939346 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f99b00197d0 con 0x7f99c0104600 2024-04-23T02:03:24.029 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.321+0000 7f99c65a8700 1 -- 172.21.15.71:0/1467939346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c01081f0 con 0x7f99c0104600 2024-04-23T02:03:24.029 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.323+0000 7f99bd7fa700 1 -- 172.21.15.71:0/1467939346 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f99b0024b50 con 0x7f99c0104600 2024-04-23T02:03:24.029 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.395+0000 7f99c65a8700 1 -- 172.21.15.71:0/1467939346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f99c019dbb0 con 0x7f99c0104600 2024-04-23T02:03:24.029 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.396+0000 7f99bd7fa700 1 -- 172.21.15.71:0/1467939346 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f99b0024cf0 con 0x7f99c0104600 2024-04-23T02:03:24.030 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.397+0000 7f99c65a8700 1 -- 172.21.15.71:0/1467939346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c0104600 msgr2=0x7f99c0199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:24.030 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.397+0000 7f99c65a8700 1 --2- 172.21.15.71:0/1467939346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c0104600 0x7f99c0199fb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f99b0006af0 tx=0x7f99b0011a30 comp rx=0 tx=0).stop 2024-04-23T02:03:24.030 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.397+0000 7f99c65a8700 1 -- 172.21.15.71:0/1467939346 shutdown_connections 2024-04-23T02:03:24.030 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.397+0000 7f99c65a8700 1 --2- 172.21.15.71:0/1467939346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c0104600 0x7f99c0199fb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:24.030 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.397+0000 7f99c65a8700 1 -- 172.21.15.71:0/1467939346 >> 172.21.15.71:0/1467939346 conn(0x7f99c00fff40 msgr2=0x7f99c0101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:24.030 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.397+0000 7f99c65a8700 1 -- 172.21.15.71:0/1467939346 shutdown_connections 2024-04-23T02:03:24.030 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:23.397+0000 7f99c65a8700 1 -- 172.21.15.71:0/1467939346 wait complete. 2024-04-23T02:03:24.030 INFO:teuthology.orchestra.run.smithi071.stdout:mgr not available, waiting (1/15)... 2024-04-23T02:03:28.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:28 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/71307682' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T02:03:29.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:29 smithi071 ceph-mon[39837]: Activating manager daemon smithi071.ukdjvn 2024-04-23T02:03:29.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:29 smithi071 ceph-mon[39837]: mgrmap e2: smithi071.ukdjvn(active, starting, since 0.00293466s) 2024-04-23T02:03:29.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:29 smithi071 ceph-mon[39837]: from='mgr.14100 172.21.15.71:0/4236212136' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T02:03:29.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:29 smithi071 ceph-mon[39837]: from='mgr.14100 172.21.15.71:0/4236212136' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T02:03:29.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:29 smithi071 ceph-mon[39837]: from='mgr.14100 172.21.15.71:0/4236212136' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T02:03:29.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:29 smithi071 ceph-mon[39837]: from='mgr.14100 172.21.15.71:0/4236212136' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi071"}]: dispatch 2024-04-23T02:03:29.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:29 smithi071 ceph-mon[39837]: from='mgr.14100 172.21.15.71:0/4236212136' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr metadata", "who": "smithi071.ukdjvn", "id": "smithi071.ukdjvn"}]: dispatch 2024-04-23T02:03:29.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:29 smithi071 ceph-mon[39837]: Manager daemon smithi071.ukdjvn is now available 2024-04-23T02:03:29.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:29 smithi071 ceph-mon[39837]: from='mgr.14100 172.21.15.71:0/4236212136' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi071.ukdjvn/mirror_snapshot_schedule"}]: dispatch 2024-04-23T02:03:29.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:29 smithi071 ceph-mon[39837]: from='mgr.14100 172.21.15.71:0/4236212136' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi071.ukdjvn/trash_purge_schedule"}]: dispatch 2024-04-23T02:03:29.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:29 smithi071 ceph-mon[39837]: from='mgr.14100 172.21.15.71:0/4236212136' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:03:29.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:29 smithi071 ceph-mon[39837]: from='mgr.14100 172.21.15.71:0/4236212136' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:03:29.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:29 smithi071 ceph-mon[39837]: from='mgr.14100 172.21.15.71:0/4236212136' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:03:31.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:31 smithi071 ceph-mon[39837]: mgrmap e3: smithi071.ukdjvn(active, since 1.00728s) 2024-04-23T02:03:33.284 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-04-23T02:03:33.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-04-23T02:03:33.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsid": "91f5470c-0115-11ef-bc93-c7b262605968", 2024-04-23T02:03:33.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T02:03:33.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T02:03:33.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T02:03:33.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T02:03:33.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:33.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T02:03:33.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T02:03:33.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 0 2024-04-23T02:03:33.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-04-23T02:03:33.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T02:03:33.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "smithi071" 2024-04-23T02:03:33.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-04-23T02:03:33.287 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-23T02:03:33.288 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T02:03:33.288 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T02:03:33.288 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-23T02:03:33.288 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T02:03:33.288 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:33.288 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T02:03:33.288 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T02:03:33.288 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T02:03:33.288 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T02:03:33.288 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T02:03:33.296 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T02:03:33.296 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T02:03:33.296 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T02:03:33.296 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:33.296 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T02:03:33.296 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T02:03:33.296 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T02:03:33.296 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T02:03:33.296 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T02:03:33.296 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T02:03:33.296 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T02:03:33.297 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T02:03:33.297 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T02:03:33.297 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:33.297 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T02:03:33.297 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T02:03:33.297 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T02:03:33.297 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T02:03:33.297 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:33.297 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T02:03:33.297 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-23T02:03:33.297 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T02:03:33.297 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T02:03:33.297 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T02:03:33.297 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T02:03:33.297 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T02:03:33.297 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-04-23T02:03:33.298 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T02:03:33.298 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:33.298 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T02:03:33.298 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T02:03:33.298 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T02:03:14.872527+0000", 2024-04-23T02:03:33.298 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T02:03:33.298 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:33.298 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T02:03:33.298 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-04-23T02:03:33.298 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.973+0000 7f9540c38700 1 Processor -- start 2024-04-23T02:03:33.298 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.973+0000 7f9540c38700 1 -- start start 2024-04-23T02:03:33.298 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.974+0000 7f9540c38700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c106da0 0x7f953c109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:33.298 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.974+0000 7f9540c38700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c074530 con 0x7f953c106da0 2024-04-23T02:03:33.298 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.974+0000 7f953a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c106da0 0x7f953c109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:33.299 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.974+0000 7f953a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c106da0 0x7f953c109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54852/0 (socket says 172.21.15.71:54852) 2024-04-23T02:03:33.299 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.974+0000 7f953a59c700 1 -- 172.21.15.71:0/2901302103 learned_addr learned my addr 172.21.15.71:0/2901302103 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:33.299 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.974+0000 7f953a59c700 1 -- 172.21.15.71:0/2901302103 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c1096d0 con 0x7f953c106da0 2024-04-23T02:03:33.299 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.975+0000 7f953a59c700 1 --2- 172.21.15.71:0/2901302103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c106da0 0x7f953c109190 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f952c0098a0 tx=0x7f952c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f6941765113dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:33.299 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.975+0000 7f953959a700 1 -- 172.21.15.71:0/2901302103 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f952c004180 con 0x7f953c106da0 2024-04-23T02:03:33.299 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.975+0000 7f953959a700 1 -- 172.21.15.71:0/2901302103 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f952c0042e0 con 0x7f953c106da0 2024-04-23T02:03:33.299 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.975+0000 7f953959a700 1 -- 172.21.15.71:0/2901302103 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f952c0119d0 con 0x7f953c106da0 2024-04-23T02:03:33.299 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.975+0000 7f9540c38700 1 -- 172.21.15.71:0/2901302103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c106da0 msgr2=0x7f953c109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:33.299 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.975+0000 7f9540c38700 1 --2- 172.21.15.71:0/2901302103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c106da0 0x7f953c109190 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f952c0098a0 tx=0x7f952c009bb0 comp rx=0 tx=0).stop 2024-04-23T02:03:33.299 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.975+0000 7f9540c38700 1 -- 172.21.15.71:0/2901302103 shutdown_connections 2024-04-23T02:03:33.299 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.975+0000 7f9540c38700 1 --2- 172.21.15.71:0/2901302103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c106da0 0x7f953c109190 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:33.299 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.975+0000 7f9540c38700 1 -- 172.21.15.71:0/2901302103 >> 172.21.15.71:0/2901302103 conn(0x7f953c1007f0 msgr2=0x7f953c102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:33.299 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.975+0000 7f9540c38700 1 -- 172.21.15.71:0/2901302103 shutdown_connections 2024-04-23T02:03:33.299 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.975+0000 7f9540c38700 1 -- 172.21.15.71:0/2901302103 wait complete. 2024-04-23T02:03:33.300 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.976+0000 7f9540c38700 1 Processor -- start 2024-04-23T02:03:33.300 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.976+0000 7f9540c38700 1 -- start start 2024-04-23T02:03:33.300 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.976+0000 7f9540c38700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c106da0 0x7f953c199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:33.300 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.976+0000 7f9540c38700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c19a530 con 0x7f953c106da0 2024-04-23T02:03:33.300 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.976+0000 7f953a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c106da0 0x7f953c199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:33.300 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.976+0000 7f953a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c106da0 0x7f953c199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54858/0 (socket says 172.21.15.71:54858) 2024-04-23T02:03:33.300 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.976+0000 7f953a59c700 1 -- 172.21.15.71:0/71307682 learned_addr learned my addr 172.21.15.71:0/71307682 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:33.300 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.977+0000 7f953a59c700 1 -- 172.21.15.71:0/71307682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c009550 con 0x7f953c106da0 2024-04-23T02:03:33.300 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.977+0000 7f953a59c700 1 --2- 172.21.15.71:0/71307682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c106da0 0x7f953c199ff0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f952c009130 tx=0x7f952c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:33.300 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.977+0000 7f952b7fe700 1 -- 172.21.15.71:0/71307682 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f952c003900 con 0x7f953c106da0 2024-04-23T02:03:33.300 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.977+0000 7f952b7fe700 1 -- 172.21.15.71:0/71307682 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f952c003a60 con 0x7f953c106da0 2024-04-23T02:03:33.300 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.977+0000 7f952b7fe700 1 -- 172.21.15.71:0/71307682 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f952c0246e0 con 0x7f953c106da0 2024-04-23T02:03:33.300 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.977+0000 7f9540c38700 1 -- 172.21.15.71:0/71307682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f953c19a730 con 0x7f953c106da0 2024-04-23T02:03:33.300 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.977+0000 7f9540c38700 1 -- 172.21.15.71:0/71307682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f953c19ab50 con 0x7f953c106da0 2024-04-23T02:03:33.300 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.977+0000 7f952b7fe700 1 -- 172.21.15.71:0/71307682 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f952c024c50 con 0x7f953c106da0 2024-04-23T02:03:33.301 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.977+0000 7f952b7fe700 1 -- 172.21.15.71:0/71307682 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f952c0193c0 con 0x7f953c106da0 2024-04-23T02:03:33.301 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.978+0000 7f9540c38700 1 -- 172.21.15.71:0/71307682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f953c0623c0 con 0x7f953c106da0 2024-04-23T02:03:33.301 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:27.982+0000 7f952b7fe700 1 -- 172.21.15.71:0/71307682 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f952c01ab80 con 0x7f953c106da0 2024-04-23T02:03:33.301 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:28.053+0000 7f9540c38700 1 -- 172.21.15.71:0/71307682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f953c19d680 con 0x7f953c106da0 2024-04-23T02:03:33.301 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:28.053+0000 7f952b7fe700 1 -- 172.21.15.71:0/71307682 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f952c01a430 con 0x7f953c106da0 2024-04-23T02:03:33.301 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:28.054+0000 7f9540c38700 1 -- 172.21.15.71:0/71307682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c106da0 msgr2=0x7f953c199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:33.301 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:28.054+0000 7f9540c38700 1 --2- 172.21.15.71:0/71307682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c106da0 0x7f953c199ff0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f952c009130 tx=0x7f952c011a30 comp rx=0 tx=0).stop 2024-04-23T02:03:33.301 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:28.054+0000 7f9540c38700 1 -- 172.21.15.71:0/71307682 shutdown_connections 2024-04-23T02:03:33.301 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:28.054+0000 7f9540c38700 1 --2- 172.21.15.71:0/71307682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c106da0 0x7f953c199ff0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:33.301 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:28.054+0000 7f9540c38700 1 -- 172.21.15.71:0/71307682 >> 172.21.15.71:0/71307682 conn(0x7f953c1007f0 msgr2=0x7f953c101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:33.301 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:28.054+0000 7f9540c38700 1 -- 172.21.15.71:0/71307682 shutdown_connections 2024-04-23T02:03:33.301 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:28.054+0000 7f9540c38700 1 -- 172.21.15.71:0/71307682 wait complete. 2024-04-23T02:03:33.301 INFO:teuthology.orchestra.run.smithi071.stdout:mgr not available, waiting (2/15)... 2024-04-23T02:03:33.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:33 smithi071 ceph-mon[39837]: mgrmap e4: smithi071.ukdjvn(active, since 3s) 2024-04-23T02:03:38.932 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:38 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1104647444' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T02:03:39.217 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-04-23T02:03:39.218 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-04-23T02:03:39.218 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsid": "91f5470c-0115-11ef-bc93-c7b262605968", 2024-04-23T02:03:39.218 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T02:03:39.218 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T02:03:39.218 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T02:03:39.218 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T02:03:39.218 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:39.218 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T02:03:39.218 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T02:03:39.218 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 0 2024-04-23T02:03:39.218 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-04-23T02:03:39.218 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T02:03:39.219 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "smithi071" 2024-04-23T02:03:39.219 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-04-23T02:03:39.219 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_age": 18, 2024-04-23T02:03:39.219 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T02:03:39.219 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T02:03:39.219 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-23T02:03:39.219 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T02:03:39.219 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:39.219 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T02:03:39.220 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T02:03:39.220 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T02:03:39.220 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T02:03:39.220 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T02:03:39.228 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T02:03:39.228 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T02:03:39.228 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T02:03:39.228 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:39.228 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T02:03:39.228 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T02:03:39.228 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T02:03:39.228 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T02:03:39.228 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T02:03:39.228 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T02:03:39.229 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T02:03:39.229 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T02:03:39.229 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T02:03:39.229 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:39.229 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T02:03:39.229 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T02:03:39.229 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T02:03:39.229 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T02:03:39.229 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:39.229 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T02:03:39.229 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T02:03:39.229 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T02:03:39.229 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T02:03:39.229 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T02:03:39.229 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T02:03:39.229 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T02:03:39.230 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-04-23T02:03:39.230 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T02:03:39.230 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:39.230 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T02:03:39.230 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T02:03:39.230 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T02:03:14.872527+0000", 2024-04-23T02:03:39.230 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T02:03:39.230 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-04-23T02:03:39.230 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T02:03:39.230 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-04-23T02:03:39.230 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.352+0000 7f203a810700 1 Processor -- start 2024-04-23T02:03:39.230 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.352+0000 7f203a810700 1 -- start start 2024-04-23T02:03:39.230 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.352+0000 7f203a810700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2034104bf0 0x7f2034105010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:39.230 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.352+0000 7f203a810700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20340744d0 con 0x7f2034104bf0 2024-04-23T02:03:39.231 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.352+0000 7f2033fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2034104bf0 0x7f2034105010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:39.231 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.352+0000 7f2033fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2034104bf0 0x7f2034105010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46498/0 (socket says 172.21.15.71:46498) 2024-04-23T02:03:39.231 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.352+0000 7f2033fff700 1 -- 172.21.15.71:0/1658401936 learned_addr learned my addr 172.21.15.71:0/1658401936 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:39.231 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.352+0000 7f2033fff700 1 -- 172.21.15.71:0/1658401936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2034105550 con 0x7f2034104bf0 2024-04-23T02:03:39.231 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.353+0000 7f2033fff700 1 --2- 172.21.15.71:0/1658401936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2034104bf0 0x7f2034105010 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f20240098a0 tx=0x7f2024009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b32e54ac80f8cf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:39.231 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.353+0000 7f2032ffd700 1 -- 172.21.15.71:0/1658401936 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2024004180 con 0x7f2034104bf0 2024-04-23T02:03:39.231 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.353+0000 7f2032ffd700 1 -- 172.21.15.71:0/1658401936 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f20240042e0 con 0x7f2034104bf0 2024-04-23T02:03:39.231 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.353+0000 7f2032ffd700 1 -- 172.21.15.71:0/1658401936 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20240119d0 con 0x7f2034104bf0 2024-04-23T02:03:39.231 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.353+0000 7f203a810700 1 -- 172.21.15.71:0/1658401936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2034104bf0 msgr2=0x7f2034105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:39.231 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.353+0000 7f203a810700 1 --2- 172.21.15.71:0/1658401936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2034104bf0 0x7f2034105010 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f20240098a0 tx=0x7f2024009bb0 comp rx=0 tx=0).stop 2024-04-23T02:03:39.231 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.353+0000 7f203a810700 1 -- 172.21.15.71:0/1658401936 shutdown_connections 2024-04-23T02:03:39.231 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.353+0000 7f203a810700 1 --2- 172.21.15.71:0/1658401936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2034104bf0 0x7f2034105010 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:39.231 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.353+0000 7f203a810700 1 -- 172.21.15.71:0/1658401936 >> 172.21.15.71:0/1658401936 conn(0x7f20341007f0 msgr2=0x7f2034102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:39.231 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.353+0000 7f203a810700 1 -- 172.21.15.71:0/1658401936 shutdown_connections 2024-04-23T02:03:39.231 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.353+0000 7f203a810700 1 -- 172.21.15.71:0/1658401936 wait complete. 2024-04-23T02:03:39.232 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.354+0000 7f203a810700 1 Processor -- start 2024-04-23T02:03:39.232 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.354+0000 7f203a810700 1 -- start start 2024-04-23T02:03:39.232 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.354+0000 7f203a810700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2034104bf0 0x7f203419c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:39.232 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.354+0000 7f203a810700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203419c760 con 0x7f2034104bf0 2024-04-23T02:03:39.232 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.354+0000 7f2033fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2034104bf0 0x7f203419c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:39.232 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.354+0000 7f2033fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2034104bf0 0x7f203419c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46514/0 (socket says 172.21.15.71:46514) 2024-04-23T02:03:39.232 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.354+0000 7f2033fff700 1 -- 172.21.15.71:0/1104647444 learned_addr learned my addr 172.21.15.71:0/1104647444 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:39.232 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.355+0000 7f2033fff700 1 -- 172.21.15.71:0/1104647444 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2024009550 con 0x7f2034104bf0 2024-04-23T02:03:39.232 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.355+0000 7f2033fff700 1 --2- 172.21.15.71:0/1104647444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2034104bf0 0x7f203419c220 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2024009130 tx=0x7f2024011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:39.232 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.355+0000 7f20317fa700 1 -- 172.21.15.71:0/1104647444 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2024003a00 con 0x7f2034104bf0 2024-04-23T02:03:39.232 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.355+0000 7f203a810700 1 -- 172.21.15.71:0/1104647444 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203419c960 con 0x7f2034104bf0 2024-04-23T02:03:39.232 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.355+0000 7f20317fa700 1 -- 172.21.15.71:0/1104647444 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f202401a880 con 0x7f2034104bf0 2024-04-23T02:03:39.232 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.355+0000 7f20317fa700 1 -- 172.21.15.71:0/1104647444 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20240197b0 con 0x7f2034104bf0 2024-04-23T02:03:39.233 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.355+0000 7f203a810700 1 -- 172.21.15.71:0/1104647444 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203419cd80 con 0x7f2034104bf0 2024-04-23T02:03:39.233 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.356+0000 7f20317fa700 1 -- 172.21.15.71:0/1104647444 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f2024019ab0 con 0x7f2034104bf0 2024-04-23T02:03:39.233 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.356+0000 7f20317fa700 1 --2- 172.21.15.71:0/1104647444 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f201c038280 0x7f201c03a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:39.233 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.356+0000 7f20317fa700 1 -- 172.21.15.71:0/1104647444 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f202404c6c0 con 0x7f2034104bf0 2024-04-23T02:03:39.233 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.356+0000 7f20337fe700 1 --2- 172.21.15.71:0/1104647444 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f201c038280 0x7f201c03a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:39.233 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.356+0000 7f203a810700 1 -- 172.21.15.71:0/1104647444 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2034195b00 con 0x7f2034104bf0 2024-04-23T02:03:39.233 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.357+0000 7f20337fe700 1 --2- 172.21.15.71:0/1104647444 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f201c038280 0x7f201c03a740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2028006fd0 tx=0x7f2028006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:39.233 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.361+0000 7f20317fa700 1 -- 172.21.15.71:0/1104647444 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f202402c3a0 con 0x7f2034104bf0 2024-04-23T02:03:39.233 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.620+0000 7f203a810700 1 -- 172.21.15.71:0/1104647444 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f20340623c0 con 0x7f2034104bf0 2024-04-23T02:03:39.233 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.621+0000 7f20317fa700 1 -- 172.21.15.71:0/1104647444 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f2024023390 con 0x7f2034104bf0 2024-04-23T02:03:39.233 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.622+0000 7f203a810700 1 -- 172.21.15.71:0/1104647444 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f201c038280 msgr2=0x7f201c03a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:39.233 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.622+0000 7f203a810700 1 --2- 172.21.15.71:0/1104647444 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f201c038280 0x7f201c03a740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2028006fd0 tx=0x7f2028006e40 comp rx=0 tx=0).stop 2024-04-23T02:03:39.233 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.622+0000 7f203a810700 1 -- 172.21.15.71:0/1104647444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2034104bf0 msgr2=0x7f203419c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:39.233 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.622+0000 7f203a810700 1 --2- 172.21.15.71:0/1104647444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2034104bf0 0x7f203419c220 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2024009130 tx=0x7f2024011a30 comp rx=0 tx=0).stop 2024-04-23T02:03:39.234 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.623+0000 7f203a810700 1 -- 172.21.15.71:0/1104647444 shutdown_connections 2024-04-23T02:03:39.234 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.623+0000 7f203a810700 1 --2- 172.21.15.71:0/1104647444 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f201c038280 0x7f201c03a740 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:39.234 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.623+0000 7f203a810700 1 --2- 172.21.15.71:0/1104647444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2034104bf0 0x7f203419c220 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:39.234 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.623+0000 7f203a810700 1 -- 172.21.15.71:0/1104647444 >> 172.21.15.71:0/1104647444 conn(0x7f20341007f0 msgr2=0x7f20341907b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:39.234 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.623+0000 7f203a810700 1 -- 172.21.15.71:0/1104647444 shutdown_connections 2024-04-23T02:03:39.234 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:38.623+0000 7f203a810700 1 -- 172.21.15.71:0/1104647444 wait complete. 2024-04-23T02:03:39.234 INFO:teuthology.orchestra.run.smithi071.stdout:mgr is available 2024-04-23T02:03:40.271 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:40 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/2076581691' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-23T02:03:40.548 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-04-23T02:03:40.548 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [global] 2024-04-23T02:03:40.548 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout fsid = 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T02:03:40.548 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-23T02:03:40.548 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-23T02:03:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-23T02:03:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-23T02:03:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-04-23T02:03:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-23T02:03:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-23T02:03:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-04-23T02:03:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [osd] 2024-04-23T02:03:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-23T02:03:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-23T02:03:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-23T02:03:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-23T02:03:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-23T02:03:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.761+0000 7f991df95700 1 Processor -- start 2024-04-23T02:03:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.761+0000 7f991df95700 1 -- start start 2024-04-23T02:03:40.550 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.761+0000 7f991df95700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9918106e00 0x7f99181091f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:40.550 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.761+0000 7f991df95700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99180744d0 con 0x7f9918106e00 2024-04-23T02:03:40.550 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.761+0000 7f99177fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9918106e00 0x7f99181091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:40.550 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.761+0000 7f99177fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9918106e00 0x7f99181091f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46518/0 (socket says 172.21.15.71:46518) 2024-04-23T02:03:40.550 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.761+0000 7f99177fe700 1 -- 172.21.15.71:0/968659391 learned_addr learned my addr 172.21.15.71:0/968659391 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:40.550 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.761+0000 7f99177fe700 1 -- 172.21.15.71:0/968659391 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9918109730 con 0x7f9918106e00 2024-04-23T02:03:40.550 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.761+0000 7f99177fe700 1 --2- 172.21.15.71:0/968659391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9918106e00 0x7f99181091f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f99080098a0 tx=0x7f9908009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b33e08188e1d98c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:40.550 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.761+0000 7f99167fc700 1 -- 172.21.15.71:0/968659391 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9908004180 con 0x7f9918106e00 2024-04-23T02:03:40.550 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.762+0000 7f99167fc700 1 -- 172.21.15.71:0/968659391 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f99080042e0 con 0x7f9918106e00 2024-04-23T02:03:40.550 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.762+0000 7f99167fc700 1 -- 172.21.15.71:0/968659391 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99080119d0 con 0x7f9918106e00 2024-04-23T02:03:40.550 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.762+0000 7f991df95700 1 -- 172.21.15.71:0/968659391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9918106e00 msgr2=0x7f99181091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:40.550 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.762+0000 7f991df95700 1 --2- 172.21.15.71:0/968659391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9918106e00 0x7f99181091f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f99080098a0 tx=0x7f9908009bb0 comp rx=0 tx=0).stop 2024-04-23T02:03:40.550 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.762+0000 7f991df95700 1 -- 172.21.15.71:0/968659391 shutdown_connections 2024-04-23T02:03:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.762+0000 7f991df95700 1 --2- 172.21.15.71:0/968659391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9918106e00 0x7f99181091f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.762+0000 7f991df95700 1 -- 172.21.15.71:0/968659391 >> 172.21.15.71:0/968659391 conn(0x7f99181007f0 msgr2=0x7f9918102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.762+0000 7f991df95700 1 -- 172.21.15.71:0/968659391 shutdown_connections 2024-04-23T02:03:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.762+0000 7f991df95700 1 -- 172.21.15.71:0/968659391 wait complete. 2024-04-23T02:03:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.763+0000 7f991df95700 1 Processor -- start 2024-04-23T02:03:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.763+0000 7f991df95700 1 -- start start 2024-04-23T02:03:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.763+0000 7f991df95700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9918106e00 0x7f991819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.763+0000 7f991df95700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991819a560 con 0x7f9918106e00 2024-04-23T02:03:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.763+0000 7f99177fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9918106e00 0x7f991819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.763+0000 7f99177fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9918106e00 0x7f991819a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46524/0 (socket says 172.21.15.71:46524) 2024-04-23T02:03:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.763+0000 7f99177fe700 1 -- 172.21.15.71:0/2076581691 learned_addr learned my addr 172.21.15.71:0/2076581691 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.764+0000 7f99177fe700 1 -- 172.21.15.71:0/2076581691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9908009550 con 0x7f9918106e00 2024-04-23T02:03:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.764+0000 7f99177fe700 1 --2- 172.21.15.71:0/2076581691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9918106e00 0x7f991819a020 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9908009130 tx=0x7f9908011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.764+0000 7f9914ff9700 1 -- 172.21.15.71:0/2076581691 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9908003a00 con 0x7f9918106e00 2024-04-23T02:03:40.552 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.764+0000 7f9914ff9700 1 -- 172.21.15.71:0/2076581691 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f990801a880 con 0x7f9918106e00 2024-04-23T02:03:40.552 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.764+0000 7f991df95700 1 -- 172.21.15.71:0/2076581691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991819a760 con 0x7f9918106e00 2024-04-23T02:03:40.552 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.764+0000 7f9914ff9700 1 -- 172.21.15.71:0/2076581691 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99080197b0 con 0x7f9918106e00 2024-04-23T02:03:40.552 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.764+0000 7f991df95700 1 -- 172.21.15.71:0/2076581691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991819ab80 con 0x7f9918106e00 2024-04-23T02:03:40.552 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.765+0000 7f9914ff9700 1 -- 172.21.15.71:0/2076581691 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f9908019a20 con 0x7f9918106e00 2024-04-23T02:03:40.552 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.765+0000 7f9914ff9700 1 --2- 172.21.15.71:0/2076581691 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f9900038320 0x7f990003a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:40.552 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.765+0000 7f991df95700 1 -- 172.21.15.71:0/2076581691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99180623c0 con 0x7f9918106e00 2024-04-23T02:03:40.552 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.765+0000 7f9914ff9700 1 -- 172.21.15.71:0/2076581691 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f990804c6e0 con 0x7f9918106e00 2024-04-23T02:03:40.552 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.765+0000 7f9916ffd700 1 --2- 172.21.15.71:0/2076581691 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f9900038320 0x7f990003a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:40.552 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.766+0000 7f9916ffd700 1 --2- 172.21.15.71:0/2076581691 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f9900038320 0x7f990003a7e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f990c006fd0 tx=0x7f990c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:40.552 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.770+0000 7f9914ff9700 1 -- 172.21.15.71:0/2076581691 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f990802c3a0 con 0x7f9918106e00 2024-04-23T02:03:40.552 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.955+0000 7f991df95700 1 -- 172.21.15.71:0/2076581691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f991819da00 con 0x7f9918106e00 2024-04-23T02:03:40.552 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.955+0000 7f9914ff9700 1 -- 172.21.15.71:0/2076581691 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f9908046e30 con 0x7f9918106e00 2024-04-23T02:03:40.553 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.958+0000 7f991df95700 1 -- 172.21.15.71:0/2076581691 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f9900038320 msgr2=0x7f990003a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:40.553 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.958+0000 7f991df95700 1 --2- 172.21.15.71:0/2076581691 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f9900038320 0x7f990003a7e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f990c006fd0 tx=0x7f990c006e40 comp rx=0 tx=0).stop 2024-04-23T02:03:40.553 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.958+0000 7f991df95700 1 -- 172.21.15.71:0/2076581691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9918106e00 msgr2=0x7f991819a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:40.553 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.958+0000 7f991df95700 1 --2- 172.21.15.71:0/2076581691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9918106e00 0x7f991819a020 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9908009130 tx=0x7f9908011a30 comp rx=0 tx=0).stop 2024-04-23T02:03:40.553 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.958+0000 7f991df95700 1 -- 172.21.15.71:0/2076581691 shutdown_connections 2024-04-23T02:03:40.553 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.958+0000 7f991df95700 1 --2- 172.21.15.71:0/2076581691 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f9900038320 0x7f990003a7e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:40.553 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.958+0000 7f991df95700 1 --2- 172.21.15.71:0/2076581691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9918106e00 0x7f991819a020 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:40.553 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.958+0000 7f991df95700 1 -- 172.21.15.71:0/2076581691 >> 172.21.15.71:0/2076581691 conn(0x7f99181007f0 msgr2=0x7f9918101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:40.553 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.958+0000 7f991df95700 1 -- 172.21.15.71:0/2076581691 shutdown_connections 2024-04-23T02:03:40.553 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:39.958+0000 7f991df95700 1 -- 172.21.15.71:0/2076581691 wait complete. 2024-04-23T02:03:40.553 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling cephadm module... 2024-04-23T02:03:41.884 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:41 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3808353345' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-23T02:03:42.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.133+0000 7fd903127700 1 Processor -- start 2024-04-23T02:03:42.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.133+0000 7fd903127700 1 -- start start 2024-04-23T02:03:42.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.134+0000 7fd903127700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8fc106de0 0x7fd8fc1091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:42.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.134+0000 7fd903127700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc079da0 con 0x7fd8fc106de0 2024-04-23T02:03:42.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.134+0000 7fd900ec3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8fc106de0 0x7fd8fc1091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:42.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.134+0000 7fd900ec3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8fc106de0 0x7fd8fc1091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46538/0 (socket says 172.21.15.71:46538) 2024-04-23T02:03:42.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.134+0000 7fd900ec3700 1 -- 172.21.15.71:0/2496750114 learned_addr learned my addr 172.21.15.71:0/2496750114 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:42.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.134+0000 7fd900ec3700 1 -- 172.21.15.71:0/2496750114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc109710 con 0x7fd8fc106de0 2024-04-23T02:03:42.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.134+0000 7fd900ec3700 1 --2- 172.21.15.71:0/2496750114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8fc106de0 0x7fd8fc1091d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00098a0 tx=0x7fd8f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48817ce8385a2408 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:42.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.134+0000 7fd8fb7fe700 1 -- 172.21.15.71:0/2496750114 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8f0004180 con 0x7fd8fc106de0 2024-04-23T02:03:42.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.134+0000 7fd8fb7fe700 1 -- 172.21.15.71:0/2496750114 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7fd8f00042e0 con 0x7fd8fc106de0 2024-04-23T02:03:42.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.134+0000 7fd8fb7fe700 1 -- 172.21.15.71:0/2496750114 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8f00119d0 con 0x7fd8fc106de0 2024-04-23T02:03:42.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.135+0000 7fd903127700 1 -- 172.21.15.71:0/2496750114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8fc106de0 msgr2=0x7fd8fc1091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:42.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.135+0000 7fd903127700 1 --2- 172.21.15.71:0/2496750114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8fc106de0 0x7fd8fc1091d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00098a0 tx=0x7fd8f0009bb0 comp rx=0 tx=0).stop 2024-04-23T02:03:42.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.135+0000 7fd903127700 1 -- 172.21.15.71:0/2496750114 shutdown_connections 2024-04-23T02:03:42.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.135+0000 7fd903127700 1 --2- 172.21.15.71:0/2496750114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8fc106de0 0x7fd8fc1091d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:42.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.135+0000 7fd903127700 1 -- 172.21.15.71:0/2496750114 >> 172.21.15.71:0/2496750114 conn(0x7fd8fc1007d0 msgr2=0x7fd8fc102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:42.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.135+0000 7fd903127700 1 -- 172.21.15.71:0/2496750114 shutdown_connections 2024-04-23T02:03:42.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.135+0000 7fd903127700 1 -- 172.21.15.71:0/2496750114 wait complete. 2024-04-23T02:03:42.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.136+0000 7fd903127700 1 Processor -- start 2024-04-23T02:03:42.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.136+0000 7fd903127700 1 -- start start 2024-04-23T02:03:42.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.136+0000 7fd903127700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8fc106de0 0x7fd8fc199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:42.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.136+0000 7fd903127700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc19a520 con 0x7fd8fc106de0 2024-04-23T02:03:42.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.136+0000 7fd900ec3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8fc106de0 0x7fd8fc199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:42.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.136+0000 7fd900ec3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8fc106de0 0x7fd8fc199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46550/0 (socket says 172.21.15.71:46550) 2024-04-23T02:03:42.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.136+0000 7fd900ec3700 1 -- 172.21.15.71:0/3808353345 learned_addr learned my addr 172.21.15.71:0/3808353345 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:42.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.136+0000 7fd900ec3700 1 -- 172.21.15.71:0/3808353345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f0009550 con 0x7fd8fc106de0 2024-04-23T02:03:42.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.137+0000 7fd900ec3700 1 --2- 172.21.15.71:0/3808353345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8fc106de0 0x7fd8fc199fe0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0011980 tx=0x7fd8f0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:42.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.137+0000 7fd8f9ffb700 1 -- 172.21.15.71:0/3808353345 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8f0003b40 con 0x7fd8fc106de0 2024-04-23T02:03:42.283 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.137+0000 7fd8f9ffb700 1 -- 172.21.15.71:0/3808353345 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7fd8f001a880 con 0x7fd8fc106de0 2024-04-23T02:03:42.283 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.137+0000 7fd903127700 1 -- 172.21.15.71:0/3808353345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8fc19a720 con 0x7fd8fc106de0 2024-04-23T02:03:42.283 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.137+0000 7fd8f9ffb700 1 -- 172.21.15.71:0/3808353345 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd8f0019910 con 0x7fd8fc106de0 2024-04-23T02:03:42.283 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.137+0000 7fd903127700 1 -- 172.21.15.71:0/3808353345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8fc19ab40 con 0x7fd8fc106de0 2024-04-23T02:03:42.283 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.138+0000 7fd8f9ffb700 1 -- 172.21.15.71:0/3808353345 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7fd8f0019b80 con 0x7fd8fc106de0 2024-04-23T02:03:42.283 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.138+0000 7fd8f9ffb700 1 --2- 172.21.15.71:0/3808353345 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7fd8ec0382e0 0x7fd8ec03a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:42.283 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.138+0000 7fd8f9ffb700 1 -- 172.21.15.71:0/3808353345 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd8f004d480 con 0x7fd8fc106de0 2024-04-23T02:03:42.283 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.138+0000 7fd8fbfff700 1 --2- 172.21.15.71:0/3808353345 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7fd8ec0382e0 0x7fd8ec03a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:42.283 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.138+0000 7fd903127700 1 -- 172.21.15.71:0/3808353345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8fc0623c0 con 0x7fd8fc106de0 2024-04-23T02:03:42.283 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.138+0000 7fd8fbfff700 1 --2- 172.21.15.71:0/3808353345 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7fd8ec0382e0 0x7fd8ec03a7a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8006fd0 tx=0x7fd8e8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:42.283 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.143+0000 7fd8f9ffb700 1 -- 172.21.15.71:0/3808353345 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd8f002c3a0 con 0x7fd8fc106de0 2024-04-23T02:03:42.283 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.382+0000 7fd903127700 1 -- 172.21.15.71:0/3808353345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fd8fc19d670 con 0x7fd8fc106de0 2024-04-23T02:03:42.283 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.556+0000 7fd8f9ffb700 1 -- 172.21.15.71:0/3808353345 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fd8f0016d60 con 0x7fd8fc106de0 2024-04-23T02:03:42.284 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.557+0000 7fd8f9ffb700 1 -- 172.21.15.71:0/3808353345 <== mon.0 v2:172.21.15.71:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fd8f002dce0 con 0x7fd8fc106de0 2024-04-23T02:03:42.284 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.559+0000 7fd903127700 1 -- 172.21.15.71:0/3808353345 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7fd8ec0382e0 msgr2=0x7fd8ec03a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:42.284 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.559+0000 7fd903127700 1 --2- 172.21.15.71:0/3808353345 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7fd8ec0382e0 0x7fd8ec03a7a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8006fd0 tx=0x7fd8e8006e40 comp rx=0 tx=0).stop 2024-04-23T02:03:42.284 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.559+0000 7fd903127700 1 -- 172.21.15.71:0/3808353345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8fc106de0 msgr2=0x7fd8fc199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:42.284 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.559+0000 7fd903127700 1 --2- 172.21.15.71:0/3808353345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8fc106de0 0x7fd8fc199fe0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0011980 tx=0x7fd8f0011a60 comp rx=0 tx=0).stop 2024-04-23T02:03:42.284 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.559+0000 7fd903127700 1 -- 172.21.15.71:0/3808353345 shutdown_connections 2024-04-23T02:03:42.284 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.559+0000 7fd903127700 1 --2- 172.21.15.71:0/3808353345 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7fd8ec0382e0 0x7fd8ec03a7a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:42.284 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.559+0000 7fd903127700 1 --2- 172.21.15.71:0/3808353345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8fc106de0 0x7fd8fc199fe0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:42.284 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.559+0000 7fd903127700 1 -- 172.21.15.71:0/3808353345 >> 172.21.15.71:0/3808353345 conn(0x7fd8fc1007d0 msgr2=0x7fd8fc101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:42.284 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.560+0000 7fd903127700 1 -- 172.21.15.71:0/3808353345 shutdown_connections 2024-04-23T02:03:42.284 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:41.560+0000 7fd903127700 1 -- 172.21.15.71:0/3808353345 wait complete. 2024-04-23T02:03:42.936 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:42 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3808353345' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-23T02:03:42.937 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:42 smithi071 ceph-mon[39837]: mgrmap e5: smithi071.ukdjvn(active, since 12s) 2024-04-23T02:03:43.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:43 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/2089701083' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-23T02:03:43.760 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-04-23T02:03:43.760 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-23T02:03:43.761 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T02:03:43.761 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "active_name": "smithi071.ukdjvn", 2024-04-23T02:03:43.761 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-23T02:03:43.761 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-04-23T02:03:43.761 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.828+0000 7ff20c04e700 1 Processor -- start 2024-04-23T02:03:43.761 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.828+0000 7ff20c04e700 1 -- start start 2024-04-23T02:03:43.761 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.829+0000 7ff20c04e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff204104610 0x7ff204104a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:43.761 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.829+0000 7ff20c04e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff204105000 con 0x7ff204104610 2024-04-23T02:03:43.761 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.829+0000 7ff209dea700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff204104610 0x7ff204104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:43.761 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.829+0000 7ff209dea700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff204104610 0x7ff204104a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46578/0 (socket says 172.21.15.71:46578) 2024-04-23T02:03:43.761 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.829+0000 7ff209dea700 1 -- 172.21.15.71:0/1133385750 learned_addr learned my addr 172.21.15.71:0/1133385750 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:43.761 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.829+0000 7ff209dea700 1 -- 172.21.15.71:0/1133385750 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff204105140 con 0x7ff204104610 2024-04-23T02:03:43.761 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.829+0000 7ff209dea700 1 --2- 172.21.15.71:0/1133385750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff204104610 0x7ff204104a30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff1f80098a0 tx=0x7ff1f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129ee484b0dfba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:43.761 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.829+0000 7ff208de8700 1 -- 172.21.15.71:0/1133385750 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1f8004180 con 0x7ff204104610 2024-04-23T02:03:43.762 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.829+0000 7ff208de8700 1 -- 172.21.15.71:0/1133385750 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7ff1f80042e0 con 0x7ff204104610 2024-04-23T02:03:43.762 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.829+0000 7ff208de8700 1 -- 172.21.15.71:0/1133385750 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1f80119d0 con 0x7ff204104610 2024-04-23T02:03:43.762 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.830+0000 7ff20c04e700 1 -- 172.21.15.71:0/1133385750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff204104610 msgr2=0x7ff204104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:43.762 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.830+0000 7ff20c04e700 1 --2- 172.21.15.71:0/1133385750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff204104610 0x7ff204104a30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff1f80098a0 tx=0x7ff1f8009bb0 comp rx=0 tx=0).stop 2024-04-23T02:03:43.762 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.830+0000 7ff20c04e700 1 -- 172.21.15.71:0/1133385750 shutdown_connections 2024-04-23T02:03:43.762 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.830+0000 7ff20c04e700 1 --2- 172.21.15.71:0/1133385750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff204104610 0x7ff204104a30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:43.762 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.830+0000 7ff20c04e700 1 -- 172.21.15.71:0/1133385750 >> 172.21.15.71:0/1133385750 conn(0x7ff2040fff40 msgr2=0x7ff204102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:43.762 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.830+0000 7ff20c04e700 1 -- 172.21.15.71:0/1133385750 shutdown_connections 2024-04-23T02:03:43.762 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.830+0000 7ff20c04e700 1 -- 172.21.15.71:0/1133385750 wait complete. 2024-04-23T02:03:43.762 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.831+0000 7ff20c04e700 1 Processor -- start 2024-04-23T02:03:43.762 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.831+0000 7ff20c04e700 1 -- start start 2024-04-23T02:03:43.762 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.831+0000 7ff20c04e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff204104610 0x7ff20419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:43.763 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.831+0000 7ff20c04e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20419a550 con 0x7ff204104610 2024-04-23T02:03:43.763 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.831+0000 7ff209dea700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff204104610 0x7ff20419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:43.763 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.831+0000 7ff209dea700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff204104610 0x7ff20419a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46592/0 (socket says 172.21.15.71:46592) 2024-04-23T02:03:43.763 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.831+0000 7ff209dea700 1 -- 172.21.15.71:0/2089701083 learned_addr learned my addr 172.21.15.71:0/2089701083 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:43.763 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.831+0000 7ff209dea700 1 -- 172.21.15.71:0/2089701083 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f8009550 con 0x7ff204104610 2024-04-23T02:03:43.763 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.831+0000 7ff209dea700 1 --2- 172.21.15.71:0/2089701083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff204104610 0x7ff20419a010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8006af0 tx=0x7ff1f8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:43.763 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.832+0000 7ff1f6ffd700 1 -- 172.21.15.71:0/2089701083 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1f8024430 con 0x7ff204104610 2024-04-23T02:03:43.763 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.832+0000 7ff1f6ffd700 1 -- 172.21.15.71:0/2089701083 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7ff1f80249e0 con 0x7ff204104610 2024-04-23T02:03:43.763 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.832+0000 7ff20c04e700 1 -- 172.21.15.71:0/2089701083 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff20419a750 con 0x7ff204104610 2024-04-23T02:03:43.763 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.832+0000 7ff1f6ffd700 1 -- 172.21.15.71:0/2089701083 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1f801a970 con 0x7ff204104610 2024-04-23T02:03:43.763 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.832+0000 7ff20c04e700 1 -- 172.21.15.71:0/2089701083 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff20419ab70 con 0x7ff204104610 2024-04-23T02:03:43.763 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.832+0000 7ff1f6ffd700 1 -- 172.21.15.71:0/2089701083 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7ff1f801abe0 con 0x7ff204104610 2024-04-23T02:03:43.763 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.833+0000 7ff1f6ffd700 1 --2- 172.21.15.71:0/2089701083 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7ff1f0038370 0x7ff1f003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:43.764 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.833+0000 7ff1f6ffd700 1 -- 172.21.15.71:0/2089701083 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff1f804c2b0 con 0x7ff204104610 2024-04-23T02:03:43.764 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.833+0000 7ff2095e9700 1 -- 172.21.15.71:0/2089701083 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7ff1f0038370 msgr2=0x7ff1f003a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3794652670 2024-04-23T02:03:43.764 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.833+0000 7ff2095e9700 1 --2- 172.21.15.71:0/2089701083 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7ff1f0038370 0x7ff1f003a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T02:03:43.764 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.833+0000 7ff20c04e700 1 -- 172.21.15.71:0/2089701083 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2040623c0 con 0x7ff204104610 2024-04-23T02:03:43.764 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:42.838+0000 7ff1f6ffd700 1 -- 172.21.15.71:0/2089701083 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff1f802d3f0 con 0x7ff204104610 2024-04-23T02:03:43.764 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:43.034+0000 7ff2095e9700 1 -- 172.21.15.71:0/2089701083 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7ff1f0038370 msgr2=0x7ff1f003a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3794652670 2024-04-23T02:03:43.764 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:43.034+0000 7ff2095e9700 1 --2- 172.21.15.71:0/2089701083 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7ff1f0038370 0x7ff1f003a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T02:03:43.764 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:43.089+0000 7ff20c04e700 1 -- 172.21.15.71:0/2089701083 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7ff20419db60 con 0x7ff204104610 2024-04-23T02:03:43.764 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:43.089+0000 7ff1f6ffd700 1 -- 172.21.15.71:0/2089701083 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7ff1f8023390 con 0x7ff204104610 2024-04-23T02:03:43.764 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:43.091+0000 7ff20c04e700 1 -- 172.21.15.71:0/2089701083 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7ff1f0038370 msgr2=0x7ff1f003a830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T02:03:43.764 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:43.091+0000 7ff20c04e700 1 --2- 172.21.15.71:0/2089701083 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7ff1f0038370 0x7ff1f003a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:43.764 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:43.091+0000 7ff20c04e700 1 -- 172.21.15.71:0/2089701083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff204104610 msgr2=0x7ff20419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:43.765 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:43.091+0000 7ff20c04e700 1 --2- 172.21.15.71:0/2089701083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff204104610 0x7ff20419a010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8006af0 tx=0x7ff1f8011a30 comp rx=0 tx=0).stop 2024-04-23T02:03:43.765 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:43.091+0000 7ff20c04e700 1 -- 172.21.15.71:0/2089701083 shutdown_connections 2024-04-23T02:03:43.765 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:43.091+0000 7ff20c04e700 1 --2- 172.21.15.71:0/2089701083 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7ff1f0038370 0x7ff1f003a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:43.765 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:43.091+0000 7ff20c04e700 1 --2- 172.21.15.71:0/2089701083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff204104610 0x7ff20419a010 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:43.765 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:43.091+0000 7ff20c04e700 1 -- 172.21.15.71:0/2089701083 >> 172.21.15.71:0/2089701083 conn(0x7ff2040fff40 msgr2=0x7ff204101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:43.765 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:43.091+0000 7ff20c04e700 1 -- 172.21.15.71:0/2089701083 shutdown_connections 2024-04-23T02:03:43.765 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:43.091+0000 7ff20c04e700 1 -- 172.21.15.71:0/2089701083 wait complete. 2024-04-23T02:03:43.765 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for the mgr to restart... 2024-04-23T02:03:43.765 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr epoch 5... 2024-04-23T02:03:48.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:48 smithi071 ceph-mon[39837]: Active manager daemon smithi071.ukdjvn restarted 2024-04-23T02:03:48.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:48 smithi071 ceph-mon[39837]: Activating manager daemon smithi071.ukdjvn 2024-04-23T02:03:48.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:48 smithi071 ceph-mon[39837]: osdmap e2: 0 total, 0 up, 0 in 2024-04-23T02:03:48.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:48 smithi071 ceph-mon[39837]: mgrmap e6: smithi071.ukdjvn(active, starting, since 0.00319489s) 2024-04-23T02:03:48.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:48 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi071"}]: dispatch 2024-04-23T02:03:48.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:48 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr metadata", "who": "smithi071.ukdjvn", "id": "smithi071.ukdjvn"}]: dispatch 2024-04-23T02:03:48.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:48 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T02:03:48.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:48 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T02:03:48.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:48 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T02:03:48.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:48 smithi071 ceph-mon[39837]: Manager daemon smithi071.ukdjvn is now available 2024-04-23T02:03:48.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:48 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:03:48.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:48 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:03:48.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:48 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:03:48.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:48 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:03:49.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:49 smithi071 ceph-mon[39837]: Found migration_current of "None". Setting to last migration. 2024-04-23T02:03:49.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:49 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:03:49.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:49 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi071.ukdjvn/mirror_snapshot_schedule"}]: dispatch 2024-04-23T02:03:49.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:49 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi071.ukdjvn/trash_purge_schedule"}]: dispatch 2024-04-23T02:03:49.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:49 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:03:49.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:49 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:03:49.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:49 smithi071 ceph-mon[39837]: mgrmap e7: smithi071.ukdjvn(active, since 1.00743s) 2024-04-23T02:03:49.859 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-04-23T02:03:49.859 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-23T02:03:49.859 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-23T02:03:49.859 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-04-23T02:03:49.860 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.289+0000 7f53af805700 1 Processor -- start 2024-04-23T02:03:49.860 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.289+0000 7f53af805700 1 -- start start 2024-04-23T02:03:49.860 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.290+0000 7f53af805700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a8104c10 0x7f53a8105030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:49.860 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.290+0000 7f53af805700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a80744b0 con 0x7f53a8104c10 2024-04-23T02:03:49.860 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.290+0000 7f53ad5a1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a8104c10 0x7f53a8105030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:49.860 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.290+0000 7f53ad5a1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a8104c10 0x7f53a8105030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46598/0 (socket says 172.21.15.71:46598) 2024-04-23T02:03:49.860 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.290+0000 7f53ad5a1700 1 -- 172.21.15.71:0/4030007740 learned_addr learned my addr 172.21.15.71:0/4030007740 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:49.860 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.290+0000 7f53ad5a1700 1 -- 172.21.15.71:0/4030007740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a8105570 con 0x7f53a8104c10 2024-04-23T02:03:49.860 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.290+0000 7f53ad5a1700 1 --2- 172.21.15.71:0/4030007740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a8104c10 0x7f53a8105030 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f53a40098a0 tx=0x7f53a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55bb2d7d9063bd50 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:49.860 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.290+0000 7f539ffff700 1 -- 172.21.15.71:0/4030007740 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53a4004180 con 0x7f53a8104c10 2024-04-23T02:03:49.860 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.290+0000 7f539ffff700 1 -- 172.21.15.71:0/4030007740 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f53a40042e0 con 0x7f53a8104c10 2024-04-23T02:03:49.861 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.290+0000 7f539ffff700 1 -- 172.21.15.71:0/4030007740 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53a40119d0 con 0x7f53a8104c10 2024-04-23T02:03:49.861 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.291+0000 7f53af805700 1 -- 172.21.15.71:0/4030007740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a8104c10 msgr2=0x7f53a8105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:49.861 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.291+0000 7f53af805700 1 --2- 172.21.15.71:0/4030007740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a8104c10 0x7f53a8105030 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f53a40098a0 tx=0x7f53a4009bb0 comp rx=0 tx=0).stop 2024-04-23T02:03:49.861 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.291+0000 7f53af805700 1 -- 172.21.15.71:0/4030007740 shutdown_connections 2024-04-23T02:03:49.861 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.291+0000 7f53af805700 1 --2- 172.21.15.71:0/4030007740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a8104c10 0x7f53a8105030 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:49.861 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.291+0000 7f53af805700 1 -- 172.21.15.71:0/4030007740 >> 172.21.15.71:0/4030007740 conn(0x7f53a81007f0 msgr2=0x7f53a8102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:49.861 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.291+0000 7f53af805700 1 -- 172.21.15.71:0/4030007740 shutdown_connections 2024-04-23T02:03:49.861 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.291+0000 7f53af805700 1 -- 172.21.15.71:0/4030007740 wait complete. 2024-04-23T02:03:49.861 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.292+0000 7f53af805700 1 Processor -- start 2024-04-23T02:03:49.861 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.292+0000 7f53af805700 1 -- start start 2024-04-23T02:03:49.861 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.292+0000 7f53af805700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a8104c10 0x7f53a819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:49.861 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.292+0000 7f53af805700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a819c750 con 0x7f53a8104c10 2024-04-23T02:03:49.862 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.292+0000 7f53ad5a1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a8104c10 0x7f53a819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:49.862 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.292+0000 7f53ad5a1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a8104c10 0x7f53a819c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46612/0 (socket says 172.21.15.71:46612) 2024-04-23T02:03:49.862 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.292+0000 7f53ad5a1700 1 -- 172.21.15.71:0/1736791163 learned_addr learned my addr 172.21.15.71:0/1736791163 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:49.862 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.292+0000 7f53ad5a1700 1 -- 172.21.15.71:0/1736791163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a4009550 con 0x7f53a8104c10 2024-04-23T02:03:49.862 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.293+0000 7f53ad5a1700 1 --2- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a8104c10 0x7f53a819c210 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009130 tx=0x7f53a4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:49.862 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.293+0000 7f539e7fc700 1 -- 172.21.15.71:0/1736791163 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53a4003900 con 0x7f53a8104c10 2024-04-23T02:03:49.862 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.293+0000 7f53af805700 1 -- 172.21.15.71:0/1736791163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a819c950 con 0x7f53a8104c10 2024-04-23T02:03:49.862 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.293+0000 7f539e7fc700 1 -- 172.21.15.71:0/1736791163 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f53a4003a60 con 0x7f53a8104c10 2024-04-23T02:03:49.862 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.293+0000 7f539e7fc700 1 -- 172.21.15.71:0/1736791163 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53a40246e0 con 0x7f53a8104c10 2024-04-23T02:03:49.862 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.293+0000 7f53af805700 1 -- 172.21.15.71:0/1736791163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a819cd70 con 0x7f53a8104c10 2024-04-23T02:03:49.862 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.294+0000 7f539e7fc700 1 -- 172.21.15.71:0/1736791163 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f53a401ab80 con 0x7f53a8104c10 2024-04-23T02:03:49.862 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.294+0000 7f539e7fc700 1 --2- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f5394038320 0x7f539403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:49.863 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.294+0000 7f539e7fc700 1 -- 172.21.15.71:0/1736791163 --> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f539403aef0 con 0x7f5394038320 2024-04-23T02:03:49.863 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.294+0000 7f53acda0700 1 -- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f5394038320 msgr2=0x7f539403a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3794652670 2024-04-23T02:03:49.863 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.294+0000 7f539e7fc700 1 -- 172.21.15.71:0/1736791163 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f53a404c080 con 0x7f53a8104c10 2024-04-23T02:03:49.863 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.294+0000 7f53acda0700 1 --2- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f5394038320 0x7f539403a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T02:03:49.863 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.495+0000 7f53acda0700 1 -- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f5394038320 msgr2=0x7f539403a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3794652670 2024-04-23T02:03:49.863 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.495+0000 7f53acda0700 1 --2- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f5394038320 0x7f539403a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T02:03:49.863 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.896+0000 7f53acda0700 1 -- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f5394038320 msgr2=0x7f539403a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3794652670 2024-04-23T02:03:49.863 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:44.896+0000 7f53acda0700 1 --2- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f5394038320 0x7f539403a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T02:03:49.863 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:45.697+0000 7f53acda0700 1 -- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f5394038320 msgr2=0x7f539403a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3794652670 2024-04-23T02:03:49.863 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:45.697+0000 7f53acda0700 1 --2- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f5394038320 0x7f539403a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-23T02:03:49.863 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:47.299+0000 7f53acda0700 1 -- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f5394038320 msgr2=0x7f539403a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3794652670 2024-04-23T02:03:49.863 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:47.299+0000 7f53acda0700 1 --2- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f5394038320 0x7f539403a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-23T02:03:49.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:48.180+0000 7f539e7fc700 1 -- 172.21.15.71:0/1736791163 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mgrmap(e 6) v1 ==== 45050+0+0 (secure 0 0 0) 0x7f53a402d770 con 0x7f53a8104c10 2024-04-23T02:03:49.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:48.180+0000 7f539e7fc700 1 -- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f5394038320 msgr2=0x7f539403a7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T02:03:49.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:48.180+0000 7f539e7fc700 1 --2- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f5394038320 0x7f539403a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:49.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.183+0000 7f539e7fc700 1 -- 172.21.15.71:0/1736791163 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7f53a404de50 con 0x7f53a8104c10 2024-04-23T02:03:49.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.183+0000 7f539e7fc700 1 --2- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f539403bc00 0x7f539403dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:49.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.183+0000 7f539e7fc700 1 -- 172.21.15.71:0/1736791163 --> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f539403aef0 con 0x7f539403bc00 2024-04-23T02:03:49.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.184+0000 7f53acda0700 1 --2- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f539403bc00 0x7f539403dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:49.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.184+0000 7f53acda0700 1 --2- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f539403bc00 0x7f539403dff0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5398003a10 tx=0x7f5398009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:49.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.185+0000 7f539e7fc700 1 -- 172.21.15.71:0/1736791163 <== mgr.14118 v2:172.21.15.71:6800/3878148961 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f539403aef0 con 0x7f539403bc00 2024-04-23T02:03:49.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.191+0000 7f53af805700 1 -- 172.21.15.71:0/1736791163 --> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f53a819d550 con 0x7f539403bc00 2024-04-23T02:03:49.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.191+0000 7f539e7fc700 1 -- 172.21.15.71:0/1736791163 <== mgr.14118 v2:172.21.15.71:6800/3878148961 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f53a819d550 con 0x7f539403bc00 2024-04-23T02:03:49.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.191+0000 7f53af805700 1 -- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f539403bc00 msgr2=0x7f539403dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:49.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.191+0000 7f53af805700 1 --2- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f539403bc00 0x7f539403dff0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5398003a10 tx=0x7f5398009290 comp rx=0 tx=0).stop 2024-04-23T02:03:49.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.191+0000 7f53af805700 1 -- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a8104c10 msgr2=0x7f53a819c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:49.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.191+0000 7f53af805700 1 --2- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a8104c10 0x7f53a819c210 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009130 tx=0x7f53a4011a30 comp rx=0 tx=0).stop 2024-04-23T02:03:49.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.191+0000 7f53af805700 1 -- 172.21.15.71:0/1736791163 shutdown_connections 2024-04-23T02:03:49.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.191+0000 7f53af805700 1 --2- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f539403bc00 0x7f539403dff0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:49.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.191+0000 7f53af805700 1 --2- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a8104c10 0x7f53a819c210 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:49.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.191+0000 7f53af805700 1 --2- 172.21.15.71:0/1736791163 >> [v2:172.21.15.71:6800/3794652670,v1:172.21.15.71:6801/3794652670] conn(0x7f5394038320 0x7f539403a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:49.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.191+0000 7f53af805700 1 -- 172.21.15.71:0/1736791163 >> 172.21.15.71:0/1736791163 conn(0x7f53a81007f0 msgr2=0x7f53a81907d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:49.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.191+0000 7f53af805700 1 -- 172.21.15.71:0/1736791163 shutdown_connections 2024-04-23T02:03:49.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:49.191+0000 7f53af805700 1 -- 172.21.15.71:0/1736791163 wait complete. 2024-04-23T02:03:49.865 INFO:teuthology.orchestra.run.smithi071.stdout:mgr epoch 5 is available 2024-04-23T02:03:49.865 INFO:teuthology.orchestra.run.smithi071.stdout:Setting orchestrator backend to cephadm... 2024-04-23T02:03:50.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:50 smithi071 ceph-mon[39837]: [23/Apr/2024:02:03:49] ENGINE Bus STARTING 2024-04-23T02:03:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:50 smithi071 ceph-mon[39837]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-23T02:03:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:50 smithi071 ceph-mon[39837]: [23/Apr/2024:02:03:49] ENGINE Serving on https://172.21.15.71:7150 2024-04-23T02:03:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:50 smithi071 ceph-mon[39837]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-23T02:03:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:50 smithi071 ceph-mon[39837]: [23/Apr/2024:02:03:49] ENGINE Serving on http://172.21.15.71:8765 2024-04-23T02:03:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:50 smithi071 ceph-mon[39837]: [23/Apr/2024:02:03:49] ENGINE Bus STARTED 2024-04-23T02:03:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:50 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:03:51.363 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.477+0000 7f167f0b4700 1 Processor -- start 2024-04-23T02:03:51.364 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.477+0000 7f167f0b4700 1 -- start start 2024-04-23T02:03:51.364 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.477+0000 7f167f0b4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1678105230 0x7f1678105650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:51.364 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.477+0000 7f167f0b4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1678105c20 con 0x7f1678105230 2024-04-23T02:03:51.364 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.477+0000 7f167ce50700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1678105230 0x7f1678105650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:51.364 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.477+0000 7f167ce50700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1678105230 0x7f1678105650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43138/0 (socket says 172.21.15.71:43138) 2024-04-23T02:03:51.364 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.477+0000 7f167ce50700 1 -- 172.21.15.71:0/2339233674 learned_addr learned my addr 172.21.15.71:0/2339233674 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:51.364 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.477+0000 7f167ce50700 1 -- 172.21.15.71:0/2339233674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1678106430 con 0x7f1678105230 2024-04-23T02:03:51.364 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.477+0000 7f167ce50700 1 --2- 172.21.15.71:0/2339233674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1678105230 0x7f1678105650 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f166c0098a0 tx=0x7f166c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609beaee364b5e23 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:51.364 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.478+0000 7f16777fe700 1 -- 172.21.15.71:0/2339233674 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f166c004180 con 0x7f1678105230 2024-04-23T02:03:51.364 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.478+0000 7f16777fe700 1 -- 172.21.15.71:0/2339233674 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f166c0042e0 con 0x7f1678105230 2024-04-23T02:03:51.364 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.478+0000 7f16777fe700 1 -- 172.21.15.71:0/2339233674 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f166c0119d0 con 0x7f1678105230 2024-04-23T02:03:51.364 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.478+0000 7f167f0b4700 1 -- 172.21.15.71:0/2339233674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1678105230 msgr2=0x7f1678105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:51.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.478+0000 7f167f0b4700 1 --2- 172.21.15.71:0/2339233674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1678105230 0x7f1678105650 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f166c0098a0 tx=0x7f166c009bb0 comp rx=0 tx=0).stop 2024-04-23T02:03:51.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.478+0000 7f167f0b4700 1 -- 172.21.15.71:0/2339233674 shutdown_connections 2024-04-23T02:03:51.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.478+0000 7f167f0b4700 1 --2- 172.21.15.71:0/2339233674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1678105230 0x7f1678105650 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:51.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.478+0000 7f167f0b4700 1 -- 172.21.15.71:0/2339233674 >> 172.21.15.71:0/2339233674 conn(0x7f16781007f0 msgr2=0x7f1678102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:51.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.478+0000 7f167f0b4700 1 -- 172.21.15.71:0/2339233674 shutdown_connections 2024-04-23T02:03:51.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.478+0000 7f167f0b4700 1 -- 172.21.15.71:0/2339233674 wait complete. 2024-04-23T02:03:51.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.479+0000 7f167f0b4700 1 Processor -- start 2024-04-23T02:03:51.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.479+0000 7f167f0b4700 1 -- start start 2024-04-23T02:03:51.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.479+0000 7f167f0b4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1678105230 0x7f1678198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:51.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.479+0000 7f167f0b4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1678199490 con 0x7f1678105230 2024-04-23T02:03:51.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.479+0000 7f167ce50700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1678105230 0x7f1678198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:51.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.479+0000 7f167ce50700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1678105230 0x7f1678198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43148/0 (socket says 172.21.15.71:43148) 2024-04-23T02:03:51.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.479+0000 7f167ce50700 1 -- 172.21.15.71:0/2787561007 learned_addr learned my addr 172.21.15.71:0/2787561007 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:51.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.479+0000 7f167ce50700 1 -- 172.21.15.71:0/2787561007 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c009550 con 0x7f1678105230 2024-04-23T02:03:51.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.480+0000 7f167ce50700 1 --2- 172.21.15.71:0/2787561007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1678105230 0x7f1678198f50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f166c009130 tx=0x7f166c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:51.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.480+0000 7f1675ffb700 1 -- 172.21.15.71:0/2787561007 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f166c003a00 con 0x7f1678105230 2024-04-23T02:03:51.366 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.480+0000 7f167f0b4700 1 -- 172.21.15.71:0/2787561007 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1678199690 con 0x7f1678105230 2024-04-23T02:03:51.366 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.480+0000 7f1675ffb700 1 -- 172.21.15.71:0/2787561007 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f166c01a880 con 0x7f1678105230 2024-04-23T02:03:51.366 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.480+0000 7f1675ffb700 1 -- 172.21.15.71:0/2787561007 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f166c019760 con 0x7f1678105230 2024-04-23T02:03:51.366 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.480+0000 7f167f0b4700 1 -- 172.21.15.71:0/2787561007 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1678199ab0 con 0x7f1678105230 2024-04-23T02:03:51.366 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.481+0000 7f1675ffb700 1 -- 172.21.15.71:0/2787561007 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f166c019a20 con 0x7f1678105230 2024-04-23T02:03:51.366 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.481+0000 7f167f0b4700 1 -- 172.21.15.71:0/2787561007 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16780623c0 con 0x7f1678105230 2024-04-23T02:03:51.366 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.481+0000 7f1675ffb700 1 --2- 172.21.15.71:0/2787561007 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f16680382e0 0x7f166803a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:51.366 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.481+0000 7f1675ffb700 1 -- 172.21.15.71:0/2787561007 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f166c04c6f0 con 0x7f1678105230 2024-04-23T02:03:51.366 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.481+0000 7f1677fff700 1 --2- 172.21.15.71:0/2787561007 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f16680382e0 0x7f166803a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:51.366 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.481+0000 7f1677fff700 1 --2- 172.21.15.71:0/2787561007 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f16680382e0 0x7f166803a7a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1664006fd0 tx=0x7f1664006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:51.366 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.486+0000 7f1675ffb700 1 -- 172.21.15.71:0/2787561007 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f166c02b750 con 0x7f1678105230 2024-04-23T02:03:51.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.687+0000 7f167f0b4700 1 -- 172.21.15.71:0/2787561007 --> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f1678102ae0 con 0x7f16680382e0 2024-04-23T02:03:51.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.694+0000 7f1675ffb700 1 -- 172.21.15.71:0/2787561007 <== mgr.14118 v2:172.21.15.71:6800/3878148961 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1678102ae0 con 0x7f16680382e0 2024-04-23T02:03:51.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.697+0000 7f167f0b4700 1 -- 172.21.15.71:0/2787561007 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f16680382e0 msgr2=0x7f166803a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:51.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.697+0000 7f167f0b4700 1 --2- 172.21.15.71:0/2787561007 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f16680382e0 0x7f166803a7a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1664006fd0 tx=0x7f1664006e40 comp rx=0 tx=0).stop 2024-04-23T02:03:51.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.697+0000 7f167f0b4700 1 -- 172.21.15.71:0/2787561007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1678105230 msgr2=0x7f1678198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:51.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.697+0000 7f167f0b4700 1 --2- 172.21.15.71:0/2787561007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1678105230 0x7f1678198f50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f166c009130 tx=0x7f166c011a30 comp rx=0 tx=0).stop 2024-04-23T02:03:51.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.697+0000 7f167f0b4700 1 -- 172.21.15.71:0/2787561007 shutdown_connections 2024-04-23T02:03:51.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.697+0000 7f167f0b4700 1 --2- 172.21.15.71:0/2787561007 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f16680382e0 0x7f166803a7a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:51.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.697+0000 7f167f0b4700 1 --2- 172.21.15.71:0/2787561007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1678105230 0x7f1678198f50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:51.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.697+0000 7f167f0b4700 1 -- 172.21.15.71:0/2787561007 >> 172.21.15.71:0/2787561007 conn(0x7f16781007f0 msgr2=0x7f1678101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:51.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.697+0000 7f167f0b4700 1 -- 172.21.15.71:0/2787561007 shutdown_connections 2024-04-23T02:03:51.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:50.697+0000 7f167f0b4700 1 -- 172.21.15.71:0/2787561007 wait complete. 2024-04-23T02:03:51.367 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:51 smithi071 ceph-mon[39837]: mgrmap e8: smithi071.ukdjvn(active, since 2s) 2024-04-23T02:03:51.367 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:51 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:03:51.368 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:51 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:03:52.477 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:52 smithi071 ceph-mon[39837]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:03:52.735 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-23T02:03:52.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.939+0000 7f12af346700 1 Processor -- start 2024-04-23T02:03:52.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.939+0000 7f12af346700 1 -- start start 2024-04-23T02:03:52.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.939+0000 7f12af346700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a8105230 0x7f12a8105650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:52.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.939+0000 7f12af346700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a8105c20 con 0x7f12a8105230 2024-04-23T02:03:52.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.939+0000 7f12ad0e2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a8105230 0x7f12a8105650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:52.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.939+0000 7f12ad0e2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a8105230 0x7f12a8105650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43150/0 (socket says 172.21.15.71:43150) 2024-04-23T02:03:52.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.939+0000 7f12ad0e2700 1 -- 172.21.15.71:0/2632253045 learned_addr learned my addr 172.21.15.71:0/2632253045 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:52.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.939+0000 7f12ad0e2700 1 -- 172.21.15.71:0/2632253045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a8106430 con 0x7f12a8105230 2024-04-23T02:03:52.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.939+0000 7f12ad0e2700 1 --2- 172.21.15.71:0/2632253045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a8105230 0x7f12a8105650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f12980098a0 tx=0x7f1298009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba11361c97cd973 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:52.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.940+0000 7f129ffff700 1 -- 172.21.15.71:0/2632253045 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1298004180 con 0x7f12a8105230 2024-04-23T02:03:52.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.940+0000 7f129ffff700 1 -- 172.21.15.71:0/2632253045 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f12980042e0 con 0x7f12a8105230 2024-04-23T02:03:52.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.940+0000 7f129ffff700 1 -- 172.21.15.71:0/2632253045 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f12980119d0 con 0x7f12a8105230 2024-04-23T02:03:52.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.940+0000 7f12af346700 1 -- 172.21.15.71:0/2632253045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a8105230 msgr2=0x7f12a8105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:52.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.940+0000 7f12af346700 1 --2- 172.21.15.71:0/2632253045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a8105230 0x7f12a8105650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f12980098a0 tx=0x7f1298009bb0 comp rx=0 tx=0).stop 2024-04-23T02:03:52.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.940+0000 7f12af346700 1 -- 172.21.15.71:0/2632253045 shutdown_connections 2024-04-23T02:03:52.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.940+0000 7f12af346700 1 --2- 172.21.15.71:0/2632253045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a8105230 0x7f12a8105650 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:52.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.940+0000 7f12af346700 1 -- 172.21.15.71:0/2632253045 >> 172.21.15.71:0/2632253045 conn(0x7f12a81007d0 msgr2=0x7f12a8102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:52.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.940+0000 7f12af346700 1 -- 172.21.15.71:0/2632253045 shutdown_connections 2024-04-23T02:03:52.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.940+0000 7f12af346700 1 -- 172.21.15.71:0/2632253045 wait complete. 2024-04-23T02:03:52.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.941+0000 7f12af346700 1 Processor -- start 2024-04-23T02:03:52.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.941+0000 7f12af346700 1 -- start start 2024-04-23T02:03:52.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.941+0000 7f12af346700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a8105230 0x7f12a8199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:52.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.941+0000 7f12af346700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a819a510 con 0x7f12a8105230 2024-04-23T02:03:52.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.941+0000 7f12ad0e2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a8105230 0x7f12a8199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:52.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.941+0000 7f12ad0e2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a8105230 0x7f12a8199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43164/0 (socket says 172.21.15.71:43164) 2024-04-23T02:03:52.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.941+0000 7f12ad0e2700 1 -- 172.21.15.71:0/3982344697 learned_addr learned my addr 172.21.15.71:0/3982344697 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:52.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.941+0000 7f12ad0e2700 1 -- 172.21.15.71:0/3982344697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1298009550 con 0x7f12a8105230 2024-04-23T02:03:52.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.941+0000 7f12ad0e2700 1 --2- 172.21.15.71:0/3982344697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a8105230 0x7f12a8199fd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1298006af0 tx=0x7f1298011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:52.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.942+0000 7f129e7fc700 1 -- 172.21.15.71:0/3982344697 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1298003b40 con 0x7f12a8105230 2024-04-23T02:03:52.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.942+0000 7f129e7fc700 1 -- 172.21.15.71:0/3982344697 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f129801a880 con 0x7f12a8105230 2024-04-23T02:03:52.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.942+0000 7f12af346700 1 -- 172.21.15.71:0/3982344697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a819a710 con 0x7f12a8105230 2024-04-23T02:03:52.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.942+0000 7f129e7fc700 1 -- 172.21.15.71:0/3982344697 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1298019890 con 0x7f12a8105230 2024-04-23T02:03:52.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.942+0000 7f12af346700 1 -- 172.21.15.71:0/3982344697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a819ab30 con 0x7f12a8105230 2024-04-23T02:03:52.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.942+0000 7f129e7fc700 1 -- 172.21.15.71:0/3982344697 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f129801a9f0 con 0x7f12a8105230 2024-04-23T02:03:52.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.943+0000 7f12af346700 1 -- 172.21.15.71:0/3982344697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a80623c0 con 0x7f12a8105230 2024-04-23T02:03:52.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.943+0000 7f129e7fc700 1 --2- 172.21.15.71:0/3982344697 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f12940382e0 0x7f129403a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:52.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.943+0000 7f129e7fc700 1 -- 172.21.15.71:0/3982344697 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f129804c720 con 0x7f12a8105230 2024-04-23T02:03:52.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.943+0000 7f12ac8e1700 1 --2- 172.21.15.71:0/3982344697 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f12940382e0 0x7f129403a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:52.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.943+0000 7f12ac8e1700 1 --2- 172.21.15.71:0/3982344697 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f12940382e0 0x7f129403a7a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f12a4006fd0 tx=0x7f12a4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:52.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:51.948+0000 7f129e7fc700 1 -- 172.21.15.71:0/3982344697 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1298018530 con 0x7f12a8105230 2024-04-23T02:03:52.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:52.134+0000 7f12af346700 1 -- 172.21.15.71:0/3982344697 --> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f12a8102960 con 0x7f12940382e0 2024-04-23T02:03:52.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:52.134+0000 7f129e7fc700 1 -- 172.21.15.71:0/3982344697 <== mgr.14118 v2:172.21.15.71:6800/3878148961 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f12a8102960 con 0x7f12940382e0 2024-04-23T02:03:52.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:52.136+0000 7f12af346700 1 -- 172.21.15.71:0/3982344697 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f12940382e0 msgr2=0x7f129403a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:52.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:52.136+0000 7f12af346700 1 --2- 172.21.15.71:0/3982344697 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f12940382e0 0x7f129403a7a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f12a4006fd0 tx=0x7f12a4006e40 comp rx=0 tx=0).stop 2024-04-23T02:03:52.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:52.136+0000 7f12af346700 1 -- 172.21.15.71:0/3982344697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a8105230 msgr2=0x7f12a8199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:52.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:52.136+0000 7f12af346700 1 --2- 172.21.15.71:0/3982344697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a8105230 0x7f12a8199fd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1298006af0 tx=0x7f1298011a30 comp rx=0 tx=0).stop 2024-04-23T02:03:52.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:52.136+0000 7f12af346700 1 -- 172.21.15.71:0/3982344697 shutdown_connections 2024-04-23T02:03:52.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:52.136+0000 7f12af346700 1 --2- 172.21.15.71:0/3982344697 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f12940382e0 0x7f129403a7a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:52.740 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:52.136+0000 7f12af346700 1 --2- 172.21.15.71:0/3982344697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a8105230 0x7f12a8199fd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:52.740 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:52.136+0000 7f12af346700 1 -- 172.21.15.71:0/3982344697 >> 172.21.15.71:0/3982344697 conn(0x7f12a81007d0 msgr2=0x7f12a80797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:52.740 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:52.136+0000 7f12af346700 1 -- 172.21.15.71:0/3982344697 shutdown_connections 2024-04-23T02:03:52.740 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:52.136+0000 7f12af346700 1 -- 172.21.15.71:0/3982344697 wait complete. 2024-04-23T02:03:52.740 INFO:teuthology.orchestra.run.smithi071.stdout:Generating ssh key... 2024-04-23T02:03:53.531 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:53 smithi071 ceph-mon[39837]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:03:54.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.251+0000 7f6792881700 1 Processor -- start 2024-04-23T02:03:54.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.252+0000 7f6792881700 1 -- start start 2024-04-23T02:03:54.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.252+0000 7f6792881700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f678c105720 0x7f678c105b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:54.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.252+0000 7f6792881700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c106110 con 0x7f678c105720 2024-04-23T02:03:54.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.252+0000 7f678bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f678c105720 0x7f678c105b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:54.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.252+0000 7f678bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f678c105720 0x7f678c105b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43170/0 (socket says 172.21.15.71:43170) 2024-04-23T02:03:54.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.252+0000 7f678bfff700 1 -- 172.21.15.71:0/3512944869 learned_addr learned my addr 172.21.15.71:0/3512944869 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:54.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.252+0000 7f678bfff700 1 -- 172.21.15.71:0/3512944869 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c106440 con 0x7f678c105720 2024-04-23T02:03:54.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.252+0000 7f678bfff700 1 --2- 172.21.15.71:0/3512944869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f678c105720 0x7f678c105b40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f677c0098a0 tx=0x7f677c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6fdca75e3fa0311 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:54.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.252+0000 7f678affd700 1 -- 172.21.15.71:0/3512944869 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f677c004180 con 0x7f678c105720 2024-04-23T02:03:54.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.252+0000 7f678affd700 1 -- 172.21.15.71:0/3512944869 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f677c0042e0 con 0x7f678c105720 2024-04-23T02:03:54.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.252+0000 7f678affd700 1 -- 172.21.15.71:0/3512944869 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f677c0119d0 con 0x7f678c105720 2024-04-23T02:03:54.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.253+0000 7f6792881700 1 -- 172.21.15.71:0/3512944869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f678c105720 msgr2=0x7f678c105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:54.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.253+0000 7f6792881700 1 --2- 172.21.15.71:0/3512944869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f678c105720 0x7f678c105b40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f677c0098a0 tx=0x7f677c009bb0 comp rx=0 tx=0).stop 2024-04-23T02:03:54.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.253+0000 7f6792881700 1 -- 172.21.15.71:0/3512944869 shutdown_connections 2024-04-23T02:03:54.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.253+0000 7f6792881700 1 --2- 172.21.15.71:0/3512944869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f678c105720 0x7f678c105b40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:54.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.253+0000 7f6792881700 1 -- 172.21.15.71:0/3512944869 >> 172.21.15.71:0/3512944869 conn(0x7f678c0fff40 msgr2=0x7f678c102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:54.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.253+0000 7f6792881700 1 -- 172.21.15.71:0/3512944869 shutdown_connections 2024-04-23T02:03:54.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.253+0000 7f6792881700 1 -- 172.21.15.71:0/3512944869 wait complete. 2024-04-23T02:03:54.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.254+0000 7f6792881700 1 Processor -- start 2024-04-23T02:03:54.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.254+0000 7f6792881700 1 -- start start 2024-04-23T02:03:54.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.254+0000 7f6792881700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f678c105720 0x7f678c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:54.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.254+0000 7f6792881700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c1993d0 con 0x7f678c105720 2024-04-23T02:03:54.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.254+0000 7f678bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f678c105720 0x7f678c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:54.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.254+0000 7f678bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f678c105720 0x7f678c198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43180/0 (socket says 172.21.15.71:43180) 2024-04-23T02:03:54.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.254+0000 7f678bfff700 1 -- 172.21.15.71:0/1179593904 learned_addr learned my addr 172.21.15.71:0/1179593904 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:54.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.254+0000 7f678bfff700 1 -- 172.21.15.71:0/1179593904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c009550 con 0x7f678c105720 2024-04-23T02:03:54.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.254+0000 7f678bfff700 1 --2- 172.21.15.71:0/1179593904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f678c105720 0x7f678c198e90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f677c006af0 tx=0x7f677c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:54.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.254+0000 7f67897fa700 1 -- 172.21.15.71:0/1179593904 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f677c024430 con 0x7f678c105720 2024-04-23T02:03:54.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.254+0000 7f67897fa700 1 -- 172.21.15.71:0/1179593904 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f677c0249e0 con 0x7f678c105720 2024-04-23T02:03:54.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.255+0000 7f6792881700 1 -- 172.21.15.71:0/1179593904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678c1995d0 con 0x7f678c105720 2024-04-23T02:03:54.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.255+0000 7f67897fa700 1 -- 172.21.15.71:0/1179593904 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f677c01aa00 con 0x7f678c105720 2024-04-23T02:03:54.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.255+0000 7f6792881700 1 -- 172.21.15.71:0/1179593904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c1999f0 con 0x7f678c105720 2024-04-23T02:03:54.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.255+0000 7f67897fa700 1 -- 172.21.15.71:0/1179593904 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f677c01ac70 con 0x7f678c105720 2024-04-23T02:03:54.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.255+0000 7f67897fa700 1 --2- 172.21.15.71:0/1179593904 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f6774038370 0x7f677403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:54.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.255+0000 7f6792881700 1 -- 172.21.15.71:0/1179593904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678c0623c0 con 0x7f678c105720 2024-04-23T02:03:54.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.256+0000 7f67897fa700 1 -- 172.21.15.71:0/1179593904 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f677c04c4f0 con 0x7f678c105720 2024-04-23T02:03:54.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.256+0000 7f678b7fe700 1 --2- 172.21.15.71:0/1179593904 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f6774038370 0x7f677403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:54.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.256+0000 7f678b7fe700 1 --2- 172.21.15.71:0/1179593904 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f6774038370 0x7f677403a830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6780006fd0 tx=0x7f6780006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:54.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.260+0000 7f67897fa700 1 -- 172.21.15.71:0/1179593904 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f677c023750 con 0x7f678c105720 2024-04-23T02:03:54.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.452+0000 7f6792881700 1 -- 172.21.15.71:0/1179593904 --> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f678c102210 con 0x7f6774038370 2024-04-23T02:03:54.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.536+0000 7f67897fa700 1 -- 172.21.15.71:0/1179593904 <== mgr.14118 v2:172.21.15.71:6800/3878148961 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f678c102210 con 0x7f6774038370 2024-04-23T02:03:54.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.538+0000 7f6792881700 1 -- 172.21.15.71:0/1179593904 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f6774038370 msgr2=0x7f677403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:54.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.538+0000 7f6792881700 1 --2- 172.21.15.71:0/1179593904 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f6774038370 0x7f677403a830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6780006fd0 tx=0x7f6780006e40 comp rx=0 tx=0).stop 2024-04-23T02:03:54.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.538+0000 7f6792881700 1 -- 172.21.15.71:0/1179593904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f678c105720 msgr2=0x7f678c198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:54.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.538+0000 7f6792881700 1 --2- 172.21.15.71:0/1179593904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f678c105720 0x7f678c198e90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f677c006af0 tx=0x7f677c011a30 comp rx=0 tx=0).stop 2024-04-23T02:03:54.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.538+0000 7f6792881700 1 -- 172.21.15.71:0/1179593904 shutdown_connections 2024-04-23T02:03:54.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.538+0000 7f6792881700 1 --2- 172.21.15.71:0/1179593904 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f6774038370 0x7f677403a830 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:54.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.538+0000 7f6792881700 1 --2- 172.21.15.71:0/1179593904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f678c105720 0x7f678c198e90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:54.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.538+0000 7f6792881700 1 -- 172.21.15.71:0/1179593904 >> 172.21.15.71:0/1179593904 conn(0x7f678c0fff40 msgr2=0x7f678c1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:54.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.538+0000 7f6792881700 1 -- 172.21.15.71:0/1179593904 shutdown_connections 2024-04-23T02:03:54.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:53.538+0000 7f6792881700 1 -- 172.21.15.71:0/1179593904 wait complete. 2024-04-23T02:03:54.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:54 smithi071 ceph-mon[39837]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:03:54.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:54 smithi071 ceph-mon[39837]: Generating ssh key... 2024-04-23T02:03:54.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:54 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:03:54.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:54 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:03:55.523 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDqtHVXyikLrcynO96rql0xXsL3+bERjrlT6vpKaaGRzuoK/HR+EVcDbQfueqc5mIAv25y6TBCAFkCmNo1Ck+MeMit1N1miKYH2RTpqf4tdI6e1EmbwaMow7Eiqd3tHefwDDAqlho7LmlwhzRRTL3Pc5CvGo2scsyis9Gorn3BMSPrI7ljsPjs1BXEExh6lCNCUD+Y3yPgtK5JhYKlQz0yYWPRwfTlTPzB0SrMj57ex2frtjB4xyJncgH3K30FWD1Xv7/mhtUdRfBof8j+mJi6uo866aZ32IQ0Gbh4qjVzEe8skhFizbOi0XFsoUXMlSipeyt7P+aVF3X+wqWljuAC7uio0Y+oXa20U/B4MyiN9scI7obD3ssj66D9TmDWxgLbGQuhKSdSaO1Rvk88C1Izmwc0K+nEuNi/sJdHJeIu8yfqdoppW5qLeyMdBIZUBlUfzveCt6MJ/FSZw5/2UwmJQI4ZIxEo+0OAwU0I816vYF8NolGZIZ+D00dbgaFJlOs= ceph-91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T02:03:55.523 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.668+0000 7febcf5ef700 1 Processor -- start 2024-04-23T02:03:55.524 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.668+0000 7febcf5ef700 1 -- start start 2024-04-23T02:03:55.524 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.668+0000 7febcf5ef700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc81045e0 0x7febc8104a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:55.524 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.668+0000 7febcf5ef700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc8104fd0 con 0x7febc81045e0 2024-04-23T02:03:55.524 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.668+0000 7febcd38b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc81045e0 0x7febc8104a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:55.524 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.668+0000 7febcd38b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc81045e0 0x7febc8104a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43186/0 (socket says 172.21.15.71:43186) 2024-04-23T02:03:55.524 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.668+0000 7febcd38b700 1 -- 172.21.15.71:0/3155812482 learned_addr learned my addr 172.21.15.71:0/3155812482 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:55.524 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.668+0000 7febcd38b700 1 -- 172.21.15.71:0/3155812482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc8105110 con 0x7febc81045e0 2024-04-23T02:03:55.524 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.668+0000 7febcd38b700 1 --2- 172.21.15.71:0/3155812482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc81045e0 0x7febc8104a00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7febb80098a0 tx=0x7febb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a376d1950e31d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:55.524 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.669+0000 7febbffff700 1 -- 172.21.15.71:0/3155812482 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febb8004180 con 0x7febc81045e0 2024-04-23T02:03:55.524 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.669+0000 7febbffff700 1 -- 172.21.15.71:0/3155812482 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7febb80042e0 con 0x7febc81045e0 2024-04-23T02:03:55.524 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.669+0000 7febbffff700 1 -- 172.21.15.71:0/3155812482 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febb80119d0 con 0x7febc81045e0 2024-04-23T02:03:55.524 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.669+0000 7febcf5ef700 1 -- 172.21.15.71:0/3155812482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc81045e0 msgr2=0x7febc8104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:55.524 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.669+0000 7febcf5ef700 1 --2- 172.21.15.71:0/3155812482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc81045e0 0x7febc8104a00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7febb80098a0 tx=0x7febb8009bb0 comp rx=0 tx=0).stop 2024-04-23T02:03:55.524 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.669+0000 7febcf5ef700 1 -- 172.21.15.71:0/3155812482 shutdown_connections 2024-04-23T02:03:55.525 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.669+0000 7febcf5ef700 1 --2- 172.21.15.71:0/3155812482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc81045e0 0x7febc8104a00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:55.525 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.669+0000 7febcf5ef700 1 -- 172.21.15.71:0/3155812482 >> 172.21.15.71:0/3155812482 conn(0x7febc80fff40 msgr2=0x7febc8102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:55.525 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.669+0000 7febcf5ef700 1 -- 172.21.15.71:0/3155812482 shutdown_connections 2024-04-23T02:03:55.525 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.669+0000 7febcf5ef700 1 -- 172.21.15.71:0/3155812482 wait complete. 2024-04-23T02:03:55.525 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.670+0000 7febcf5ef700 1 Processor -- start 2024-04-23T02:03:55.525 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.670+0000 7febcf5ef700 1 -- start start 2024-04-23T02:03:55.525 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.670+0000 7febcf5ef700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc81045e0 0x7febc8106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:55.525 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.670+0000 7febcf5ef700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc8107500 con 0x7febc81045e0 2024-04-23T02:03:55.525 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.670+0000 7febcd38b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc81045e0 0x7febc8106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:55.525 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.670+0000 7febcd38b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc81045e0 0x7febc8106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43198/0 (socket says 172.21.15.71:43198) 2024-04-23T02:03:55.525 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.670+0000 7febcd38b700 1 -- 172.21.15.71:0/3683118215 learned_addr learned my addr 172.21.15.71:0/3683118215 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:55.525 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.670+0000 7febcd38b700 1 -- 172.21.15.71:0/3683118215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb8009550 con 0x7febc81045e0 2024-04-23T02:03:55.525 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.671+0000 7febcd38b700 1 --2- 172.21.15.71:0/3683118215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc81045e0 0x7febc8106fc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7febb8006af0 tx=0x7febb8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:55.525 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.671+0000 7febbe7fc700 1 -- 172.21.15.71:0/3683118215 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febb8024430 con 0x7febc81045e0 2024-04-23T02:03:55.525 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.671+0000 7febbe7fc700 1 -- 172.21.15.71:0/3683118215 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7febb80249e0 con 0x7febc81045e0 2024-04-23T02:03:55.526 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.671+0000 7febcf5ef700 1 -- 172.21.15.71:0/3683118215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc8105610 con 0x7febc81045e0 2024-04-23T02:03:55.526 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.671+0000 7febbe7fc700 1 -- 172.21.15.71:0/3683118215 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febb801a970 con 0x7febc81045e0 2024-04-23T02:03:55.526 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.671+0000 7febcf5ef700 1 -- 172.21.15.71:0/3683118215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc8105a30 con 0x7febc81045e0 2024-04-23T02:03:55.526 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.671+0000 7febbe7fc700 1 -- 172.21.15.71:0/3683118215 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7febb801abe0 con 0x7febc81045e0 2024-04-23T02:03:55.526 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.672+0000 7febcf5ef700 1 -- 172.21.15.71:0/3683118215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc80623c0 con 0x7febc81045e0 2024-04-23T02:03:55.526 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.672+0000 7febbe7fc700 1 --2- 172.21.15.71:0/3683118215 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7febb4038370 0x7febb403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:55.526 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.672+0000 7febbe7fc700 1 -- 172.21.15.71:0/3683118215 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7febb804c470 con 0x7febc81045e0 2024-04-23T02:03:55.526 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.672+0000 7febccb8a700 1 --2- 172.21.15.71:0/3683118215 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7febb4038370 0x7febb403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:55.527 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.672+0000 7febccb8a700 1 --2- 172.21.15.71:0/3683118215 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7febb4038370 0x7febb403a830 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7febc4006fd0 tx=0x7febc4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:55.527 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.676+0000 7febbe7fc700 1 -- 172.21.15.71:0/3683118215 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7febb8024b50 con 0x7febc81045e0 2024-04-23T02:03:55.527 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.867+0000 7febcf5ef700 1 -- 172.21.15.71:0/3683118215 --> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7febc8102260 con 0x7febb4038370 2024-04-23T02:03:55.527 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.867+0000 7febbe7fc700 1 -- 172.21.15.71:0/3683118215 <== mgr.14118 v2:172.21.15.71:6800/3878148961 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7febc8102260 con 0x7febb4038370 2024-04-23T02:03:55.527 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.869+0000 7febcf5ef700 1 -- 172.21.15.71:0/3683118215 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7febb4038370 msgr2=0x7febb403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:55.527 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.869+0000 7febcf5ef700 1 --2- 172.21.15.71:0/3683118215 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7febb4038370 0x7febb403a830 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7febc4006fd0 tx=0x7febc4006e40 comp rx=0 tx=0).stop 2024-04-23T02:03:55.527 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.869+0000 7febcf5ef700 1 -- 172.21.15.71:0/3683118215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc81045e0 msgr2=0x7febc8106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:55.528 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.869+0000 7febcf5ef700 1 --2- 172.21.15.71:0/3683118215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc81045e0 0x7febc8106fc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7febb8006af0 tx=0x7febb8011a30 comp rx=0 tx=0).stop 2024-04-23T02:03:55.528 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.869+0000 7febcf5ef700 1 -- 172.21.15.71:0/3683118215 shutdown_connections 2024-04-23T02:03:55.528 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.869+0000 7febcf5ef700 1 --2- 172.21.15.71:0/3683118215 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7febb4038370 0x7febb403a830 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:55.528 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.869+0000 7febcf5ef700 1 --2- 172.21.15.71:0/3683118215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc81045e0 0x7febc8106fc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:55.528 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.869+0000 7febcf5ef700 1 -- 172.21.15.71:0/3683118215 >> 172.21.15.71:0/3683118215 conn(0x7febc80fff40 msgr2=0x7febc8101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:55.528 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.869+0000 7febcf5ef700 1 -- 172.21.15.71:0/3683118215 shutdown_connections 2024-04-23T02:03:55.528 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:54.869+0000 7febcf5ef700 1 -- 172.21.15.71:0/3683118215 wait complete. 2024-04-23T02:03:55.528 INFO:teuthology.orchestra.run.smithi071.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-23T02:03:55.528 INFO:teuthology.orchestra.run.smithi071.stdout:Adding key to root@localhost authorized_keys... 2024-04-23T02:03:55.528 INFO:teuthology.orchestra.run.smithi071.stdout:Adding host smithi071... 2024-04-23T02:03:56.657 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:56 smithi071 ceph-mon[39837]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:03:57.941 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:57 smithi071 ceph-mon[39837]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi071", "addr": "172.21.15.71", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:03:58.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:58 smithi071 ceph-mon[39837]: Deploying cephadm binary to smithi071 2024-04-23T02:03:58.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:58 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:03:58.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:58 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:03:59.291 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Added host 'smithi071' with addr '172.21.15.71' 2024-04-23T02:03:59.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.095+0000 7fd147ed3700 1 Processor -- start 2024-04-23T02:03:59.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.095+0000 7fd147ed3700 1 -- start start 2024-04-23T02:03:59.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.096+0000 7fd147ed3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd140104c30 0x7fd140105050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:59.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.096+0000 7fd147ed3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd140074490 con 0x7fd140104c30 2024-04-23T02:03:59.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.096+0000 7fd145c6f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd140104c30 0x7fd140105050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:59.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.096+0000 7fd145c6f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd140104c30 0x7fd140105050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34530/0 (socket says 172.21.15.71:34530) 2024-04-23T02:03:59.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.096+0000 7fd145c6f700 1 -- 172.21.15.71:0/3368612245 learned_addr learned my addr 172.21.15.71:0/3368612245 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:59.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.096+0000 7fd145c6f700 1 -- 172.21.15.71:0/3368612245 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1400745d0 con 0x7fd140104c30 2024-04-23T02:03:59.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.096+0000 7fd145c6f700 1 --2- 172.21.15.71:0/3368612245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd140104c30 0x7fd140105050 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd1300098a0 tx=0x7fd130009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3397cfc99960b1b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:59.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.096+0000 7fd144c6d700 1 -- 172.21.15.71:0/3368612245 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd130004180 con 0x7fd140104c30 2024-04-23T02:03:59.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.096+0000 7fd144c6d700 1 -- 172.21.15.71:0/3368612245 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7fd1300042e0 con 0x7fd140104c30 2024-04-23T02:03:59.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.096+0000 7fd144c6d700 1 -- 172.21.15.71:0/3368612245 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1300119d0 con 0x7fd140104c30 2024-04-23T02:03:59.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.097+0000 7fd147ed3700 1 -- 172.21.15.71:0/3368612245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd140104c30 msgr2=0x7fd140105050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:59.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.097+0000 7fd147ed3700 1 --2- 172.21.15.71:0/3368612245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd140104c30 0x7fd140105050 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd1300098a0 tx=0x7fd130009bb0 comp rx=0 tx=0).stop 2024-04-23T02:03:59.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.097+0000 7fd147ed3700 1 -- 172.21.15.71:0/3368612245 shutdown_connections 2024-04-23T02:03:59.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.097+0000 7fd147ed3700 1 --2- 172.21.15.71:0/3368612245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd140104c30 0x7fd140105050 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:59.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.097+0000 7fd147ed3700 1 -- 172.21.15.71:0/3368612245 >> 172.21.15.71:0/3368612245 conn(0x7fd1401007f0 msgr2=0x7fd140102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:59.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.097+0000 7fd147ed3700 1 -- 172.21.15.71:0/3368612245 shutdown_connections 2024-04-23T02:03:59.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.097+0000 7fd147ed3700 1 -- 172.21.15.71:0/3368612245 wait complete. 2024-04-23T02:03:59.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.098+0000 7fd147ed3700 1 Processor -- start 2024-04-23T02:03:59.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.098+0000 7fd147ed3700 1 -- start start 2024-04-23T02:03:59.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.098+0000 7fd147ed3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd140104c30 0x7fd14019c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:59.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.098+0000 7fd147ed3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14019c740 con 0x7fd140104c30 2024-04-23T02:03:59.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.098+0000 7fd145c6f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd140104c30 0x7fd14019c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:59.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.098+0000 7fd145c6f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd140104c30 0x7fd14019c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34546/0 (socket says 172.21.15.71:34546) 2024-04-23T02:03:59.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.098+0000 7fd145c6f700 1 -- 172.21.15.71:0/905595280 learned_addr learned my addr 172.21.15.71:0/905595280 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:03:59.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.098+0000 7fd145c6f700 1 -- 172.21.15.71:0/905595280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd130009550 con 0x7fd140104c30 2024-04-23T02:03:59.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.098+0000 7fd145c6f700 1 --2- 172.21.15.71:0/905595280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd140104c30 0x7fd14019c200 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd130009130 tx=0x7fd130011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:59.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.098+0000 7fd136ffd700 1 -- 172.21.15.71:0/905595280 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd130003a00 con 0x7fd140104c30 2024-04-23T02:03:59.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.098+0000 7fd136ffd700 1 -- 172.21.15.71:0/905595280 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7fd13001a880 con 0x7fd140104c30 2024-04-23T02:03:59.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.098+0000 7fd147ed3700 1 -- 172.21.15.71:0/905595280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14019c940 con 0x7fd140104c30 2024-04-23T02:03:59.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.098+0000 7fd136ffd700 1 -- 172.21.15.71:0/905595280 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1300197b0 con 0x7fd140104c30 2024-04-23T02:03:59.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.099+0000 7fd147ed3700 1 -- 172.21.15.71:0/905595280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14019cd60 con 0x7fd140104c30 2024-04-23T02:03:59.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.099+0000 7fd136ffd700 1 -- 172.21.15.71:0/905595280 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fd130019a70 con 0x7fd140104c30 2024-04-23T02:03:59.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.100+0000 7fd147ed3700 1 -- 172.21.15.71:0/905595280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd140195ae0 con 0x7fd140104c30 2024-04-23T02:03:59.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.100+0000 7fd136ffd700 1 --2- 172.21.15.71:0/905595280 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fd12c0382d0 0x7fd12c03a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:03:59.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.100+0000 7fd136ffd700 1 -- 172.21.15.71:0/905595280 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd13004c830 con 0x7fd140104c30 2024-04-23T02:03:59.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.100+0000 7fd14546e700 1 --2- 172.21.15.71:0/905595280 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fd12c0382d0 0x7fd12c03a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:03:59.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.100+0000 7fd14546e700 1 --2- 172.21.15.71:0/905595280 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fd12c0382d0 0x7fd12c03a790 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd13c006fd0 tx=0x7fd13c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:03:59.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.104+0000 7fd136ffd700 1 -- 172.21.15.71:0/905595280 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd13002b750 con 0x7fd140104c30 2024-04-23T02:03:59.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:56.297+0000 7fd147ed3700 1 -- 172.21.15.71:0/905595280 --> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi071", "addr": "172.21.15.71", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1400611d0 con 0x7fd12c0382d0 2024-04-23T02:03:59.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:58.355+0000 7fd136ffd700 1 -- 172.21.15.71:0/905595280 <== mgr.14118 v2:172.21.15.71:6800/3878148961 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fd1400611d0 con 0x7fd12c0382d0 2024-04-23T02:03:59.295 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:58.358+0000 7fd147ed3700 1 -- 172.21.15.71:0/905595280 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fd12c0382d0 msgr2=0x7fd12c03a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:59.295 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:58.358+0000 7fd147ed3700 1 --2- 172.21.15.71:0/905595280 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fd12c0382d0 0x7fd12c03a790 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd13c006fd0 tx=0x7fd13c006e40 comp rx=0 tx=0).stop 2024-04-23T02:03:59.295 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:58.358+0000 7fd147ed3700 1 -- 172.21.15.71:0/905595280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd140104c30 msgr2=0x7fd14019c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:03:59.295 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:58.358+0000 7fd147ed3700 1 --2- 172.21.15.71:0/905595280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd140104c30 0x7fd14019c200 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd130009130 tx=0x7fd130011a30 comp rx=0 tx=0).stop 2024-04-23T02:03:59.295 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:58.358+0000 7fd147ed3700 1 -- 172.21.15.71:0/905595280 shutdown_connections 2024-04-23T02:03:59.295 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:58.358+0000 7fd147ed3700 1 --2- 172.21.15.71:0/905595280 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fd12c0382d0 0x7fd12c03a790 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:59.295 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:58.358+0000 7fd147ed3700 1 --2- 172.21.15.71:0/905595280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd140104c30 0x7fd14019c200 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:03:59.295 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:58.358+0000 7fd147ed3700 1 -- 172.21.15.71:0/905595280 >> 172.21.15.71:0/905595280 conn(0x7fd1401007f0 msgr2=0x7fd1401907f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:03:59.295 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:58.358+0000 7fd147ed3700 1 -- 172.21.15.71:0/905595280 shutdown_connections 2024-04-23T02:03:59.295 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:03:58.358+0000 7fd147ed3700 1 -- 172.21.15.71:0/905595280 wait complete. 2024-04-23T02:03:59.295 INFO:teuthology.orchestra.run.smithi071.stdout:Deploying mon service with default placement... 2024-04-23T02:03:59.865 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:03:59 smithi071 ceph-mon[39837]: Added host smithi071 2024-04-23T02:04:01.412 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-23T02:04:01.412 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.299+0000 7f17dac67700 1 Processor -- start 2024-04-23T02:04:01.412 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.299+0000 7f17dac67700 1 -- start start 2024-04-23T02:04:01.412 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.299+0000 7f17dac67700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d4106310 0x7f17d4106730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:01.412 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.299+0000 7f17dac67700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d4106d00 con 0x7f17d4106310 2024-04-23T02:04:01.412 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.299+0000 7f17d8a03700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d4106310 0x7f17d4106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:01.412 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.299+0000 7f17d8a03700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d4106310 0x7f17d4106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34556/0 (socket says 172.21.15.71:34556) 2024-04-23T02:04:01.413 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.299+0000 7f17d8a03700 1 -- 172.21.15.71:0/1984155878 learned_addr learned my addr 172.21.15.71:0/1984155878 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:01.413 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.300+0000 7f17d8a03700 1 -- 172.21.15.71:0/1984155878 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d4107510 con 0x7f17d4106310 2024-04-23T02:04:01.413 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.300+0000 7f17d8a03700 1 --2- 172.21.15.71:0/1984155878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d4106310 0x7f17d4106730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f17c80098a0 tx=0x7f17c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976aea7b7ae7610f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:01.413 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.300+0000 7f17d37fe700 1 -- 172.21.15.71:0/1984155878 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f17c8004180 con 0x7f17d4106310 2024-04-23T02:04:01.413 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.300+0000 7f17d37fe700 1 -- 172.21.15.71:0/1984155878 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f17c80042e0 con 0x7f17d4106310 2024-04-23T02:04:01.413 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.300+0000 7f17d37fe700 1 -- 172.21.15.71:0/1984155878 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f17c80119d0 con 0x7f17d4106310 2024-04-23T02:04:01.413 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.300+0000 7f17dac67700 1 -- 172.21.15.71:0/1984155878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d4106310 msgr2=0x7f17d4106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:01.413 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.300+0000 7f17dac67700 1 --2- 172.21.15.71:0/1984155878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d4106310 0x7f17d4106730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f17c80098a0 tx=0x7f17c8009bb0 comp rx=0 tx=0).stop 2024-04-23T02:04:01.413 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.300+0000 7f17dac67700 1 -- 172.21.15.71:0/1984155878 shutdown_connections 2024-04-23T02:04:01.413 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.301+0000 7f17dac67700 1 --2- 172.21.15.71:0/1984155878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d4106310 0x7f17d4106730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:01.413 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.301+0000 7f17dac67700 1 -- 172.21.15.71:0/1984155878 >> 172.21.15.71:0/1984155878 conn(0x7f17d41018d0 msgr2=0x7f17d4103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:01.413 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.301+0000 7f17dac67700 1 -- 172.21.15.71:0/1984155878 shutdown_connections 2024-04-23T02:04:01.413 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.301+0000 7f17dac67700 1 -- 172.21.15.71:0/1984155878 wait complete. 2024-04-23T02:04:01.414 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.301+0000 7f17dac67700 1 Processor -- start 2024-04-23T02:04:01.414 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.301+0000 7f17dac67700 1 -- start start 2024-04-23T02:04:01.414 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.301+0000 7f17dac67700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d4106310 0x7f17d419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:01.414 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.301+0000 7f17dac67700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d419a550 con 0x7f17d4106310 2024-04-23T02:04:01.414 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.301+0000 7f17d8a03700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d4106310 0x7f17d419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:01.414 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.301+0000 7f17d8a03700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d4106310 0x7f17d419a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34564/0 (socket says 172.21.15.71:34564) 2024-04-23T02:04:01.414 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.301+0000 7f17d8a03700 1 -- 172.21.15.71:0/2035228475 learned_addr learned my addr 172.21.15.71:0/2035228475 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:01.414 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.302+0000 7f17d8a03700 1 -- 172.21.15.71:0/2035228475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c8009550 con 0x7f17d4106310 2024-04-23T02:04:01.415 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.302+0000 7f17d8a03700 1 --2- 172.21.15.71:0/2035228475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d4106310 0x7f17d419a010 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f17c8006af0 tx=0x7f17c8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:01.415 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.302+0000 7f17d1ffb700 1 -- 172.21.15.71:0/2035228475 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f17c8003b40 con 0x7f17d4106310 2024-04-23T02:04:01.415 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.302+0000 7f17d1ffb700 1 -- 172.21.15.71:0/2035228475 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f17c801a880 con 0x7f17d4106310 2024-04-23T02:04:01.415 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.302+0000 7f17dac67700 1 -- 172.21.15.71:0/2035228475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d419a750 con 0x7f17d4106310 2024-04-23T02:04:01.415 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.302+0000 7f17d1ffb700 1 -- 172.21.15.71:0/2035228475 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f17c8019890 con 0x7f17d4106310 2024-04-23T02:04:01.415 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.302+0000 7f17dac67700 1 -- 172.21.15.71:0/2035228475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d419ab70 con 0x7f17d4106310 2024-04-23T02:04:01.415 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.303+0000 7f17d1ffb700 1 -- 172.21.15.71:0/2035228475 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f17c8019b10 con 0x7f17d4106310 2024-04-23T02:04:01.415 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.303+0000 7f17d1ffb700 1 --2- 172.21.15.71:0/2035228475 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f17c40382e0 0x7f17c403a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:01.415 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.303+0000 7f17d1ffb700 1 -- 172.21.15.71:0/2035228475 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f17c804c7c0 con 0x7f17d4106310 2024-04-23T02:04:01.416 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.303+0000 7f17dac67700 1 -- 172.21.15.71:0/2035228475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d40623c0 con 0x7f17d4106310 2024-04-23T02:04:01.416 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.303+0000 7f17d3fff700 1 --2- 172.21.15.71:0/2035228475 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f17c40382e0 0x7f17c403a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:01.416 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.304+0000 7f17d3fff700 1 --2- 172.21.15.71:0/2035228475 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f17c40382e0 0x7f17c403a7a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f17c0006fd0 tx=0x7f17c0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:01.416 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.308+0000 7f17d1ffb700 1 -- 172.21.15.71:0/2035228475 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f17c802b750 con 0x7f17d4106310 2024-04-23T02:04:01.416 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.518+0000 7f17dac67700 1 -- 172.21.15.71:0/2035228475 --> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f17d4103bd0 con 0x7f17c40382e0 2024-04-23T02:04:01.416 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.523+0000 7f17d1ffb700 1 -- 172.21.15.71:0/2035228475 <== mgr.14118 v2:172.21.15.71:6800/3878148961 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f17d4103bd0 con 0x7f17c40382e0 2024-04-23T02:04:01.416 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.525+0000 7f17dac67700 1 -- 172.21.15.71:0/2035228475 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f17c40382e0 msgr2=0x7f17c403a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:01.416 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.525+0000 7f17dac67700 1 --2- 172.21.15.71:0/2035228475 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f17c40382e0 0x7f17c403a7a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f17c0006fd0 tx=0x7f17c0006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:01.416 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.525+0000 7f17dac67700 1 -- 172.21.15.71:0/2035228475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d4106310 msgr2=0x7f17d419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:01.416 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.525+0000 7f17dac67700 1 --2- 172.21.15.71:0/2035228475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d4106310 0x7f17d419a010 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f17c8006af0 tx=0x7f17c8011a30 comp rx=0 tx=0).stop 2024-04-23T02:04:01.417 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.525+0000 7f17dac67700 1 -- 172.21.15.71:0/2035228475 shutdown_connections 2024-04-23T02:04:01.417 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.525+0000 7f17dac67700 1 --2- 172.21.15.71:0/2035228475 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f17c40382e0 0x7f17c403a7a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:01.417 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.525+0000 7f17dac67700 1 --2- 172.21.15.71:0/2035228475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d4106310 0x7f17d419a010 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:01.417 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.525+0000 7f17dac67700 1 -- 172.21.15.71:0/2035228475 >> 172.21.15.71:0/2035228475 conn(0x7f17d41018d0 msgr2=0x7f17d4102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:01.417 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.525+0000 7f17dac67700 1 -- 172.21.15.71:0/2035228475 shutdown_connections 2024-04-23T02:04:01.417 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:00.525+0000 7f17dac67700 1 -- 172.21.15.71:0/2035228475 wait complete. 2024-04-23T02:04:01.417 INFO:teuthology.orchestra.run.smithi071.stdout:Deploying mgr service with default placement... 2024-04-23T02:04:01.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:01 smithi071 ceph-mon[39837]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:04:01.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:01 smithi071 ceph-mon[39837]: Saving service mon spec with placement count:5 2024-04-23T02:04:01.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:01 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:01.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:01 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:01.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:01 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:02.999 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-23T02:04:02.999 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.965+0000 7f9d09d52700 1 Processor -- start 2024-04-23T02:04:02.999 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.965+0000 7f9d09d52700 1 -- start start 2024-04-23T02:04:02.999 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.965+0000 7f9d09d52700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d041045e0 0x7f9d04104a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:02.999 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.965+0000 7f9d09d52700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d04104fd0 con 0x7f9d041045e0 2024-04-23T02:04:02.999 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.965+0000 7f9d037fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d041045e0 0x7f9d04104a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:02.999 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.965+0000 7f9d037fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d041045e0 0x7f9d04104a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34580/0 (socket says 172.21.15.71:34580) 2024-04-23T02:04:02.999 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.965+0000 7f9d037fe700 1 -- 172.21.15.71:0/2811682520 learned_addr learned my addr 172.21.15.71:0/2811682520 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:02.999 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.965+0000 7f9d037fe700 1 -- 172.21.15.71:0/2811682520 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d04105110 con 0x7f9d041045e0 2024-04-23T02:04:02.999 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.965+0000 7f9d037fe700 1 --2- 172.21.15.71:0/2811682520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d041045e0 0x7f9d04104a00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40098a0 tx=0x7f9cf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a1b1949c9d0fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:03.000 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.966+0000 7f9d027fc700 1 -- 172.21.15.71:0/2811682520 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9cf4004180 con 0x7f9d041045e0 2024-04-23T02:04:03.000 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.966+0000 7f9d027fc700 1 -- 172.21.15.71:0/2811682520 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f9cf40042e0 con 0x7f9d041045e0 2024-04-23T02:04:03.000 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.966+0000 7f9d027fc700 1 -- 172.21.15.71:0/2811682520 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9cf4011a30 con 0x7f9d041045e0 2024-04-23T02:04:03.000 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.966+0000 7f9d09d52700 1 -- 172.21.15.71:0/2811682520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d041045e0 msgr2=0x7f9d04104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:03.000 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.966+0000 7f9d09d52700 1 --2- 172.21.15.71:0/2811682520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d041045e0 0x7f9d04104a00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40098a0 tx=0x7f9cf4009bb0 comp rx=0 tx=0).stop 2024-04-23T02:04:03.000 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.966+0000 7f9d09d52700 1 -- 172.21.15.71:0/2811682520 shutdown_connections 2024-04-23T02:04:03.000 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.966+0000 7f9d09d52700 1 --2- 172.21.15.71:0/2811682520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d041045e0 0x7f9d04104a00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:03.000 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.966+0000 7f9d09d52700 1 -- 172.21.15.71:0/2811682520 >> 172.21.15.71:0/2811682520 conn(0x7f9d040fff40 msgr2=0x7f9d04102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:03.000 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.966+0000 7f9d09d52700 1 -- 172.21.15.71:0/2811682520 shutdown_connections 2024-04-23T02:04:03.000 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.966+0000 7f9d09d52700 1 -- 172.21.15.71:0/2811682520 wait complete. 2024-04-23T02:04:03.000 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.967+0000 7f9d09d52700 1 Processor -- start 2024-04-23T02:04:03.000 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.967+0000 7f9d09d52700 1 -- start start 2024-04-23T02:04:03.000 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.967+0000 7f9d09d52700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d041045e0 0x7f9d0419a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:03.001 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.967+0000 7f9d09d52700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0419a580 con 0x7f9d041045e0 2024-04-23T02:04:03.001 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.967+0000 7f9d037fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d041045e0 0x7f9d0419a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:03.001 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.967+0000 7f9d037fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d041045e0 0x7f9d0419a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34582/0 (socket says 172.21.15.71:34582) 2024-04-23T02:04:03.001 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.967+0000 7f9d037fe700 1 -- 172.21.15.71:0/1886971849 learned_addr learned my addr 172.21.15.71:0/1886971849 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:03.001 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.967+0000 7f9d037fe700 1 -- 172.21.15.71:0/1886971849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf4009550 con 0x7f9d041045e0 2024-04-23T02:04:03.001 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.967+0000 7f9d037fe700 1 --2- 172.21.15.71:0/1886971849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d041045e0 0x7f9d0419a040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4000c00 tx=0x7f9cf4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:03.001 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.968+0000 7f9d00ff9700 1 -- 172.21.15.71:0/1886971849 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9cf4003bc0 con 0x7f9d041045e0 2024-04-23T02:04:03.001 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.968+0000 7f9d00ff9700 1 -- 172.21.15.71:0/1886971849 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f9cf40248d0 con 0x7f9d041045e0 2024-04-23T02:04:03.001 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.968+0000 7f9d09d52700 1 -- 172.21.15.71:0/1886971849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0419a780 con 0x7f9d041045e0 2024-04-23T02:04:03.001 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.968+0000 7f9d00ff9700 1 -- 172.21.15.71:0/1886971849 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9cf401a890 con 0x7f9d041045e0 2024-04-23T02:04:03.001 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.968+0000 7f9d09d52700 1 -- 172.21.15.71:0/1886971849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0419aba0 con 0x7f9d041045e0 2024-04-23T02:04:03.001 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.968+0000 7f9d00ff9700 1 -- 172.21.15.71:0/1886971849 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f9cf401ab00 con 0x7f9d041045e0 2024-04-23T02:04:03.001 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.969+0000 7f9d00ff9700 1 --2- 172.21.15.71:0/1886971849 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f9cec038370 0x7f9cec03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:03.001 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.969+0000 7f9d00ff9700 1 -- 172.21.15.71:0/1886971849 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9cf404c630 con 0x7f9d041045e0 2024-04-23T02:04:03.002 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.969+0000 7f9d09d52700 1 -- 172.21.15.71:0/1886971849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d040623c0 con 0x7f9d041045e0 2024-04-23T02:04:03.002 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.969+0000 7f9d02ffd700 1 --2- 172.21.15.71:0/1886971849 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f9cec038370 0x7f9cec03a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:03.002 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.969+0000 7f9d02ffd700 1 --2- 172.21.15.71:0/1886971849 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f9cec038370 0x7f9cec03a830 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8006fd0 tx=0x7f9cf8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:03.002 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:01.973+0000 7f9d00ff9700 1 -- 172.21.15.71:0/1886971849 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9cf401adb0 con 0x7f9d041045e0 2024-04-23T02:04:03.002 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:02.200+0000 7f9d09d52700 1 -- 172.21.15.71:0/1886971849 --> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d04102260 con 0x7f9cec038370 2024-04-23T02:04:03.002 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:02.205+0000 7f9d00ff9700 1 -- 172.21.15.71:0/1886971849 <== mgr.14118 v2:172.21.15.71:6800/3878148961 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9d04102260 con 0x7f9cec038370 2024-04-23T02:04:03.002 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:02.207+0000 7f9d09d52700 1 -- 172.21.15.71:0/1886971849 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f9cec038370 msgr2=0x7f9cec03a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:03.002 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:02.207+0000 7f9d09d52700 1 --2- 172.21.15.71:0/1886971849 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f9cec038370 0x7f9cec03a830 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8006fd0 tx=0x7f9cf8006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:03.002 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:02.207+0000 7f9d09d52700 1 -- 172.21.15.71:0/1886971849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d041045e0 msgr2=0x7f9d0419a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:03.002 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:02.207+0000 7f9d09d52700 1 --2- 172.21.15.71:0/1886971849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d041045e0 0x7f9d0419a040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4000c00 tx=0x7f9cf4011ab0 comp rx=0 tx=0).stop 2024-04-23T02:04:03.002 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:02.207+0000 7f9d09d52700 1 -- 172.21.15.71:0/1886971849 shutdown_connections 2024-04-23T02:04:03.002 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:02.207+0000 7f9d09d52700 1 --2- 172.21.15.71:0/1886971849 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f9cec038370 0x7f9cec03a830 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:03.002 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:02.207+0000 7f9d09d52700 1 --2- 172.21.15.71:0/1886971849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d041045e0 0x7f9d0419a040 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:03.003 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:02.207+0000 7f9d09d52700 1 -- 172.21.15.71:0/1886971849 >> 172.21.15.71:0/1886971849 conn(0x7f9d040fff40 msgr2=0x7f9d04101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:03.003 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:02.207+0000 7f9d09d52700 1 -- 172.21.15.71:0/1886971849 shutdown_connections 2024-04-23T02:04:03.003 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:02.207+0000 7f9d09d52700 1 -- 172.21.15.71:0/1886971849 wait complete. 2024-04-23T02:04:03.003 INFO:teuthology.orchestra.run.smithi071.stdout:Deploying crash service with default placement... 2024-04-23T02:04:03.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:03 smithi071 ceph-mon[39837]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:04:03.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:03 smithi071 ceph-mon[39837]: Saving service mgr spec with placement count:2 2024-04-23T02:04:03.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:03 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:04.443 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-04-23T02:04:04.444 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.565+0000 7f4d74300700 1 Processor -- start 2024-04-23T02:04:04.444 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.565+0000 7f4d74300700 1 -- start start 2024-04-23T02:04:04.444 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.566+0000 7f4d74300700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d6c105230 0x7f4d6c105650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:04.444 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.566+0000 7f4d74300700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c105c20 con 0x7f4d6c105230 2024-04-23T02:04:04.444 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.566+0000 7f4d7209c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d6c105230 0x7f4d6c105650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:04.444 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.566+0000 7f4d7209c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d6c105230 0x7f4d6c105650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34588/0 (socket says 172.21.15.71:34588) 2024-04-23T02:04:04.444 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.566+0000 7f4d7209c700 1 -- 172.21.15.71:0/3533992338 learned_addr learned my addr 172.21.15.71:0/3533992338 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:04.444 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.566+0000 7f4d7209c700 1 -- 172.21.15.71:0/3533992338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c106430 con 0x7f4d6c105230 2024-04-23T02:04:04.444 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.566+0000 7f4d7209c700 1 --2- 172.21.15.71:0/3533992338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d6c105230 0x7f4d6c105650 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0098a0 tx=0x7f4d5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab38488d4b67dc93 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:04.444 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.566+0000 7f4d7109a700 1 -- 172.21.15.71:0/3533992338 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4d5c004180 con 0x7f4d6c105230 2024-04-23T02:04:04.445 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.566+0000 7f4d7109a700 1 -- 172.21.15.71:0/3533992338 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4d5c0042e0 con 0x7f4d6c105230 2024-04-23T02:04:04.445 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.566+0000 7f4d7109a700 1 -- 172.21.15.71:0/3533992338 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4d5c011a30 con 0x7f4d6c105230 2024-04-23T02:04:04.445 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.567+0000 7f4d74300700 1 -- 172.21.15.71:0/3533992338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d6c105230 msgr2=0x7f4d6c105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:04.445 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.567+0000 7f4d74300700 1 --2- 172.21.15.71:0/3533992338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d6c105230 0x7f4d6c105650 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0098a0 tx=0x7f4d5c009bb0 comp rx=0 tx=0).stop 2024-04-23T02:04:04.445 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.567+0000 7f4d74300700 1 -- 172.21.15.71:0/3533992338 shutdown_connections 2024-04-23T02:04:04.445 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.567+0000 7f4d74300700 1 --2- 172.21.15.71:0/3533992338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d6c105230 0x7f4d6c105650 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:04.445 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.567+0000 7f4d74300700 1 -- 172.21.15.71:0/3533992338 >> 172.21.15.71:0/3533992338 conn(0x7f4d6c1007f0 msgr2=0x7f4d6c102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:04.445 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.567+0000 7f4d74300700 1 -- 172.21.15.71:0/3533992338 shutdown_connections 2024-04-23T02:04:04.445 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.567+0000 7f4d74300700 1 -- 172.21.15.71:0/3533992338 wait complete. 2024-04-23T02:04:04.445 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.567+0000 7f4d74300700 1 Processor -- start 2024-04-23T02:04:04.445 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.567+0000 7f4d74300700 1 -- start start 2024-04-23T02:04:04.445 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.568+0000 7f4d74300700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d6c105230 0x7f4d6c1903c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:04.445 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.568+0000 7f4d74300700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c190900 con 0x7f4d6c105230 2024-04-23T02:04:04.446 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.568+0000 7f4d7209c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d6c105230 0x7f4d6c1903c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:04.446 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.568+0000 7f4d7209c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d6c105230 0x7f4d6c1903c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34590/0 (socket says 172.21.15.71:34590) 2024-04-23T02:04:04.446 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.568+0000 7f4d7209c700 1 -- 172.21.15.71:0/3488287840 learned_addr learned my addr 172.21.15.71:0/3488287840 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:04.446 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.568+0000 7f4d7209c700 1 -- 172.21.15.71:0/3488287840 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c009550 con 0x7f4d6c105230 2024-04-23T02:04:04.446 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.568+0000 7f4d7209c700 1 --2- 172.21.15.71:0/3488287840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d6c105230 0x7f4d6c1903c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c000c00 tx=0x7f4d5c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:04.446 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.568+0000 7f4d637fe700 1 -- 172.21.15.71:0/3488287840 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4d5c003b40 con 0x7f4d6c105230 2024-04-23T02:04:04.446 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.568+0000 7f4d637fe700 1 -- 172.21.15.71:0/3488287840 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4d5c0248d0 con 0x7f4d6c105230 2024-04-23T02:04:04.446 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.568+0000 7f4d74300700 1 -- 172.21.15.71:0/3488287840 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6c18ea10 con 0x7f4d6c105230 2024-04-23T02:04:04.446 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.568+0000 7f4d637fe700 1 -- 172.21.15.71:0/3488287840 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4d5c01a820 con 0x7f4d6c105230 2024-04-23T02:04:04.447 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.568+0000 7f4d74300700 1 -- 172.21.15.71:0/3488287840 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6c18ee30 con 0x7f4d6c105230 2024-04-23T02:04:04.447 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.569+0000 7f4d637fe700 1 -- 172.21.15.71:0/3488287840 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f4d5c01aa90 con 0x7f4d6c105230 2024-04-23T02:04:04.447 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.569+0000 7f4d74300700 1 -- 172.21.15.71:0/3488287840 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6c0623c0 con 0x7f4d6c105230 2024-04-23T02:04:04.447 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.569+0000 7f4d637fe700 1 --2- 172.21.15.71:0/3488287840 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f4d58038370 0x7f4d5803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:04.447 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.569+0000 7f4d637fe700 1 -- 172.21.15.71:0/3488287840 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4d5c04c570 con 0x7f4d6c105230 2024-04-23T02:04:04.447 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.569+0000 7f4d7189b700 1 --2- 172.21.15.71:0/3488287840 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f4d58038370 0x7f4d5803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:04.447 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.570+0000 7f4d7189b700 1 --2- 172.21.15.71:0/3488287840 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f4d58038370 0x7f4d5803a830 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4d68006fd0 tx=0x7f4d68006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:04.447 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.574+0000 7f4d637fe700 1 -- 172.21.15.71:0/3488287840 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4d5c0189a0 con 0x7f4d6c105230 2024-04-23T02:04:04.447 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.775+0000 7f4d74300700 1 -- 172.21.15.71:0/3488287840 --> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d6c109630 con 0x7f4d58038370 2024-04-23T02:04:04.447 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.779+0000 7f4d637fe700 1 -- 172.21.15.71:0/3488287840 <== mgr.14118 v2:172.21.15.71:6800/3878148961 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f4d6c109630 con 0x7f4d58038370 2024-04-23T02:04:04.447 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.781+0000 7f4d74300700 1 -- 172.21.15.71:0/3488287840 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f4d58038370 msgr2=0x7f4d5803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:04.447 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.781+0000 7f4d74300700 1 --2- 172.21.15.71:0/3488287840 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f4d58038370 0x7f4d5803a830 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4d68006fd0 tx=0x7f4d68006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:04.447 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.781+0000 7f4d74300700 1 -- 172.21.15.71:0/3488287840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d6c105230 msgr2=0x7f4d6c1903c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:04.448 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.781+0000 7f4d74300700 1 --2- 172.21.15.71:0/3488287840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d6c105230 0x7f4d6c1903c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c000c00 tx=0x7f4d5c011ab0 comp rx=0 tx=0).stop 2024-04-23T02:04:04.448 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.781+0000 7f4d74300700 1 -- 172.21.15.71:0/3488287840 shutdown_connections 2024-04-23T02:04:04.448 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.781+0000 7f4d74300700 1 --2- 172.21.15.71:0/3488287840 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f4d58038370 0x7f4d5803a830 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:04.448 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.781+0000 7f4d74300700 1 --2- 172.21.15.71:0/3488287840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d6c105230 0x7f4d6c1903c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:04.448 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.781+0000 7f4d74300700 1 -- 172.21.15.71:0/3488287840 >> 172.21.15.71:0/3488287840 conn(0x7f4d6c1007f0 msgr2=0x7f4d6c073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:04.448 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.781+0000 7f4d74300700 1 -- 172.21.15.71:0/3488287840 shutdown_connections 2024-04-23T02:04:04.448 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:03.781+0000 7f4d74300700 1 -- 172.21.15.71:0/3488287840 wait complete. 2024-04-23T02:04:04.448 INFO:teuthology.orchestra.run.smithi071.stdout:Deploying ceph-exporter service with default placement... 2024-04-23T02:04:04.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:04 smithi071 ceph-mon[39837]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:04:04.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:04 smithi071 ceph-mon[39837]: Saving service crash spec with placement * 2024-04-23T02:04:04.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:04 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:04.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:04 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:05.874 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-04-23T02:04:05.874 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.002+0000 7fa3841e7700 1 Processor -- start 2024-04-23T02:04:05.874 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.002+0000 7fa3841e7700 1 -- start start 2024-04-23T02:04:05.874 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.002+0000 7fa3841e7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c106de0 0x7fa37c1091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:05.874 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.002+0000 7fa3841e7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c079da0 con 0x7fa37c106de0 2024-04-23T02:04:05.875 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.002+0000 7fa381f83700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c106de0 0x7fa37c1091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:05.875 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.002+0000 7fa381f83700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c106de0 0x7fa37c1091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34594/0 (socket says 172.21.15.71:34594) 2024-04-23T02:04:05.875 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.002+0000 7fa381f83700 1 -- 172.21.15.71:0/471827034 learned_addr learned my addr 172.21.15.71:0/471827034 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:05.875 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.003+0000 7fa381f83700 1 -- 172.21.15.71:0/471827034 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c109710 con 0x7fa37c106de0 2024-04-23T02:04:05.875 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.003+0000 7fa381f83700 1 --2- 172.21.15.71:0/471827034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c106de0 0x7fa37c1091d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa3780098a0 tx=0x7fa378009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db36e42874a98e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:05.875 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.003+0000 7fa380f81700 1 -- 172.21.15.71:0/471827034 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa378004180 con 0x7fa37c106de0 2024-04-23T02:04:05.875 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.003+0000 7fa380f81700 1 -- 172.21.15.71:0/471827034 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fa3780042e0 con 0x7fa37c106de0 2024-04-23T02:04:05.875 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.003+0000 7fa380f81700 1 -- 172.21.15.71:0/471827034 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa378011a30 con 0x7fa37c106de0 2024-04-23T02:04:05.875 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.004+0000 7fa3841e7700 1 -- 172.21.15.71:0/471827034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c106de0 msgr2=0x7fa37c1091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:05.875 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.004+0000 7fa3841e7700 1 --2- 172.21.15.71:0/471827034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c106de0 0x7fa37c1091d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa3780098a0 tx=0x7fa378009bb0 comp rx=0 tx=0).stop 2024-04-23T02:04:05.875 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.004+0000 7fa3841e7700 1 -- 172.21.15.71:0/471827034 shutdown_connections 2024-04-23T02:04:05.875 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.004+0000 7fa3841e7700 1 --2- 172.21.15.71:0/471827034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c106de0 0x7fa37c1091d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:05.875 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.004+0000 7fa3841e7700 1 -- 172.21.15.71:0/471827034 >> 172.21.15.71:0/471827034 conn(0x7fa37c1007d0 msgr2=0x7fa37c102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:05.875 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.004+0000 7fa3841e7700 1 -- 172.21.15.71:0/471827034 shutdown_connections 2024-04-23T02:04:05.875 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.004+0000 7fa3841e7700 1 -- 172.21.15.71:0/471827034 wait complete. 2024-04-23T02:04:05.876 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.005+0000 7fa3841e7700 1 Processor -- start 2024-04-23T02:04:05.876 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.005+0000 7fa3841e7700 1 -- start start 2024-04-23T02:04:05.876 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.005+0000 7fa3841e7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c106de0 0x7fa37c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:05.876 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.005+0000 7fa3841e7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c19a560 con 0x7fa37c106de0 2024-04-23T02:04:05.876 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.005+0000 7fa381f83700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c106de0 0x7fa37c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:05.876 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.005+0000 7fa381f83700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c106de0 0x7fa37c19a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34606/0 (socket says 172.21.15.71:34606) 2024-04-23T02:04:05.876 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.005+0000 7fa381f83700 1 -- 172.21.15.71:0/3295713646 learned_addr learned my addr 172.21.15.71:0/3295713646 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:05.876 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.005+0000 7fa381f83700 1 -- 172.21.15.71:0/3295713646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa378009550 con 0x7fa37c106de0 2024-04-23T02:04:05.876 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.005+0000 7fa381f83700 1 --2- 172.21.15.71:0/3295713646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c106de0 0x7fa37c19a020 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa378006af0 tx=0x7fa378011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:05.876 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.006+0000 7fa372ffd700 1 -- 172.21.15.71:0/3295713646 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa378003bc0 con 0x7fa37c106de0 2024-04-23T02:04:05.876 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.006+0000 7fa372ffd700 1 -- 172.21.15.71:0/3295713646 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fa37801a8c0 con 0x7fa37c106de0 2024-04-23T02:04:05.876 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.006+0000 7fa372ffd700 1 -- 172.21.15.71:0/3295713646 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa378019860 con 0x7fa37c106de0 2024-04-23T02:04:05.876 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.006+0000 7fa3841e7700 1 -- 172.21.15.71:0/3295713646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37c19a760 con 0x7fa37c106de0 2024-04-23T02:04:05.876 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.006+0000 7fa3841e7700 1 -- 172.21.15.71:0/3295713646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37c19ab80 con 0x7fa37c106de0 2024-04-23T02:04:05.876 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.006+0000 7fa372ffd700 1 -- 172.21.15.71:0/3295713646 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fa378019b20 con 0x7fa37c106de0 2024-04-23T02:04:05.876 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.007+0000 7fa3841e7700 1 -- 172.21.15.71:0/3295713646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37c0623c0 con 0x7fa37c106de0 2024-04-23T02:04:05.877 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.007+0000 7fa372ffd700 1 --2- 172.21.15.71:0/3295713646 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fa3680382e0 0x7fa36803a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:05.877 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.007+0000 7fa372ffd700 1 -- 172.21.15.71:0/3295713646 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa37804c6f0 con 0x7fa37c106de0 2024-04-23T02:04:05.877 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.007+0000 7fa381782700 1 --2- 172.21.15.71:0/3295713646 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fa3680382e0 0x7fa36803a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:05.877 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.007+0000 7fa381782700 1 --2- 172.21.15.71:0/3295713646 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fa3680382e0 0x7fa36803a7a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa36c006fd0 tx=0x7fa36c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:05.877 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.011+0000 7fa372ffd700 1 -- 172.21.15.71:0/3295713646 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa37802b750 con 0x7fa37c106de0 2024-04-23T02:04:05.877 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.245+0000 7fa3841e7700 1 -- 172.21.15.71:0/3295713646 --> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fa37c19d680 con 0x7fa3680382e0 2024-04-23T02:04:05.877 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.252+0000 7fa372ffd700 1 -- 172.21.15.71:0/3295713646 <== mgr.14118 v2:172.21.15.71:6800/3878148961 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fa37c19d680 con 0x7fa3680382e0 2024-04-23T02:04:05.877 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.255+0000 7fa3841e7700 1 -- 172.21.15.71:0/3295713646 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fa3680382e0 msgr2=0x7fa36803a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:05.877 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.255+0000 7fa3841e7700 1 --2- 172.21.15.71:0/3295713646 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fa3680382e0 0x7fa36803a7a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa36c006fd0 tx=0x7fa36c006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:05.877 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.255+0000 7fa3841e7700 1 -- 172.21.15.71:0/3295713646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c106de0 msgr2=0x7fa37c19a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:05.877 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.255+0000 7fa3841e7700 1 --2- 172.21.15.71:0/3295713646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c106de0 0x7fa37c19a020 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa378006af0 tx=0x7fa378011ab0 comp rx=0 tx=0).stop 2024-04-23T02:04:05.877 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.255+0000 7fa3841e7700 1 -- 172.21.15.71:0/3295713646 shutdown_connections 2024-04-23T02:04:05.877 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.256+0000 7fa3841e7700 1 --2- 172.21.15.71:0/3295713646 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fa3680382e0 0x7fa36803a7a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:05.877 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.256+0000 7fa3841e7700 1 --2- 172.21.15.71:0/3295713646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c106de0 0x7fa37c19a020 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:05.878 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.256+0000 7fa3841e7700 1 -- 172.21.15.71:0/3295713646 >> 172.21.15.71:0/3295713646 conn(0x7fa37c1007d0 msgr2=0x7fa37c101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:05.878 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.256+0000 7fa3841e7700 1 -- 172.21.15.71:0/3295713646 shutdown_connections 2024-04-23T02:04:05.878 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:05.256+0000 7fa3841e7700 1 -- 172.21.15.71:0/3295713646 wait complete. 2024-04-23T02:04:05.878 INFO:teuthology.orchestra.run.smithi071.stdout:Deploying prometheus service with default placement... 2024-04-23T02:04:06.402 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:06 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:06.403 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:06 smithi071 ceph-mon[39837]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:04:06.403 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:06 smithi071 ceph-mon[39837]: Saving service ceph-exporter spec with placement * 2024-04-23T02:04:06.403 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:06 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:07.476 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-04-23T02:04:07.476 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.612+0000 7f5ff9445700 1 Processor -- start 2024-04-23T02:04:07.477 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.612+0000 7f5ff9445700 1 -- start start 2024-04-23T02:04:07.477 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.612+0000 7f5ff9445700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff4106da0 0x7f5ff4109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:07.477 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.612+0000 7f5ff9445700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff4074530 con 0x7f5ff4106da0 2024-04-23T02:04:07.477 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.613+0000 7f5ff2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff4106da0 0x7f5ff4109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:07.477 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.613+0000 7f5ff2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff4106da0 0x7f5ff4109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44658/0 (socket says 172.21.15.71:44658) 2024-04-23T02:04:07.477 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.613+0000 7f5ff2ffd700 1 -- 172.21.15.71:0/457205113 learned_addr learned my addr 172.21.15.71:0/457205113 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:07.477 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.613+0000 7f5ff2ffd700 1 -- 172.21.15.71:0/457205113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff41096d0 con 0x7f5ff4106da0 2024-04-23T02:04:07.477 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.613+0000 7f5ff2ffd700 1 --2- 172.21.15.71:0/457205113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff4106da0 0x7f5ff4109190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0098a0 tx=0x7f5fdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858510c6300e7fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:07.477 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.613+0000 7f5ff1ffb700 1 -- 172.21.15.71:0/457205113 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fdc004180 con 0x7f5ff4106da0 2024-04-23T02:04:07.477 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.613+0000 7f5ff1ffb700 1 -- 172.21.15.71:0/457205113 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f5fdc0042e0 con 0x7f5ff4106da0 2024-04-23T02:04:07.477 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.613+0000 7f5ff1ffb700 1 -- 172.21.15.71:0/457205113 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fdc011a30 con 0x7f5ff4106da0 2024-04-23T02:04:07.477 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.614+0000 7f5ff9445700 1 -- 172.21.15.71:0/457205113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff4106da0 msgr2=0x7f5ff4109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:07.477 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.614+0000 7f5ff9445700 1 --2- 172.21.15.71:0/457205113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff4106da0 0x7f5ff4109190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0098a0 tx=0x7f5fdc009bb0 comp rx=0 tx=0).stop 2024-04-23T02:04:07.478 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.614+0000 7f5ff9445700 1 -- 172.21.15.71:0/457205113 shutdown_connections 2024-04-23T02:04:07.478 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.614+0000 7f5ff9445700 1 --2- 172.21.15.71:0/457205113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff4106da0 0x7f5ff4109190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:07.478 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.614+0000 7f5ff9445700 1 -- 172.21.15.71:0/457205113 >> 172.21.15.71:0/457205113 conn(0x7f5ff41007f0 msgr2=0x7f5ff4102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:07.478 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.614+0000 7f5ff9445700 1 -- 172.21.15.71:0/457205113 shutdown_connections 2024-04-23T02:04:07.478 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.614+0000 7f5ff9445700 1 -- 172.21.15.71:0/457205113 wait complete. 2024-04-23T02:04:07.478 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.615+0000 7f5ff9445700 1 Processor -- start 2024-04-23T02:04:07.478 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.615+0000 7f5ff9445700 1 -- start start 2024-04-23T02:04:07.478 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.615+0000 7f5ff9445700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff4106da0 0x7f5ff419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:07.478 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.615+0000 7f5ff9445700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff419a570 con 0x7f5ff4106da0 2024-04-23T02:04:07.478 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.615+0000 7f5ff2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff4106da0 0x7f5ff419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:07.478 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.616+0000 7f5ff2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff4106da0 0x7f5ff419a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44666/0 (socket says 172.21.15.71:44666) 2024-04-23T02:04:07.478 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.616+0000 7f5ff2ffd700 1 -- 172.21.15.71:0/346879497 learned_addr learned my addr 172.21.15.71:0/346879497 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:07.478 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.616+0000 7f5ff2ffd700 1 -- 172.21.15.71:0/346879497 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc009550 con 0x7f5ff4106da0 2024-04-23T02:04:07.479 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.616+0000 7f5ff2ffd700 1 --2- 172.21.15.71:0/346879497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff4106da0 0x7f5ff419a030 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc006af0 tx=0x7f5fdc011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:07.479 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.616+0000 7f5febfff700 1 -- 172.21.15.71:0/346879497 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fdc003a80 con 0x7f5ff4106da0 2024-04-23T02:04:07.479 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.616+0000 7f5ff9445700 1 -- 172.21.15.71:0/346879497 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff419a770 con 0x7f5ff4106da0 2024-04-23T02:04:07.479 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.616+0000 7f5febfff700 1 -- 172.21.15.71:0/346879497 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f5fdc003be0 con 0x7f5ff4106da0 2024-04-23T02:04:07.479 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.616+0000 7f5febfff700 1 -- 172.21.15.71:0/346879497 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5fdc019770 con 0x7f5ff4106da0 2024-04-23T02:04:07.479 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.616+0000 7f5ff9445700 1 -- 172.21.15.71:0/346879497 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff419ab90 con 0x7f5ff4106da0 2024-04-23T02:04:07.479 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.617+0000 7f5febfff700 1 -- 172.21.15.71:0/346879497 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f5fdc01a430 con 0x7f5ff4106da0 2024-04-23T02:04:07.479 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.617+0000 7f5febfff700 1 --2- 172.21.15.71:0/346879497 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f5fe00382e0 0x7f5fe003a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:07.479 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.617+0000 7f5febfff700 1 -- 172.21.15.71:0/346879497 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5fdc04c570 con 0x7f5ff4106da0 2024-04-23T02:04:07.479 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.617+0000 7f5ff9445700 1 -- 172.21.15.71:0/346879497 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff40623c0 con 0x7f5ff4106da0 2024-04-23T02:04:07.479 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.617+0000 7f5ff27fc700 1 --2- 172.21.15.71:0/346879497 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f5fe00382e0 0x7f5fe003a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:07.479 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.618+0000 7f5ff27fc700 1 --2- 172.21.15.71:0/346879497 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f5fe00382e0 0x7f5fe003a7a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4006fd0 tx=0x7f5fe4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:07.479 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.622+0000 7f5febfff700 1 -- 172.21.15.71:0/346879497 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5fdc02d440 con 0x7f5ff4106da0 2024-04-23T02:04:07.479 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.827+0000 7f5ff9445700 1 -- 172.21.15.71:0/346879497 --> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ff4102af0 con 0x7f5fe00382e0 2024-04-23T02:04:07.480 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.831+0000 7f5febfff700 1 -- 172.21.15.71:0/346879497 <== mgr.14118 v2:172.21.15.71:6800/3878148961 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f5ff4102af0 con 0x7f5fe00382e0 2024-04-23T02:04:07.480 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.833+0000 7f5ff9445700 1 -- 172.21.15.71:0/346879497 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f5fe00382e0 msgr2=0x7f5fe003a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:07.480 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.833+0000 7f5ff9445700 1 --2- 172.21.15.71:0/346879497 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f5fe00382e0 0x7f5fe003a7a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4006fd0 tx=0x7f5fe4006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:07.480 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.833+0000 7f5ff9445700 1 -- 172.21.15.71:0/346879497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff4106da0 msgr2=0x7f5ff419a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:07.480 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.833+0000 7f5ff9445700 1 --2- 172.21.15.71:0/346879497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff4106da0 0x7f5ff419a030 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc006af0 tx=0x7f5fdc011b20 comp rx=0 tx=0).stop 2024-04-23T02:04:07.480 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.833+0000 7f5ff9445700 1 -- 172.21.15.71:0/346879497 shutdown_connections 2024-04-23T02:04:07.480 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.833+0000 7f5ff9445700 1 --2- 172.21.15.71:0/346879497 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f5fe00382e0 0x7f5fe003a7a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:07.480 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.833+0000 7f5ff9445700 1 --2- 172.21.15.71:0/346879497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff4106da0 0x7f5ff419a030 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:07.480 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.833+0000 7f5ff9445700 1 -- 172.21.15.71:0/346879497 >> 172.21.15.71:0/346879497 conn(0x7f5ff41007f0 msgr2=0x7f5ff4101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:07.480 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.834+0000 7f5ff9445700 1 -- 172.21.15.71:0/346879497 shutdown_connections 2024-04-23T02:04:07.480 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:06.834+0000 7f5ff9445700 1 -- 172.21.15.71:0/346879497 wait complete. 2024-04-23T02:04:07.480 INFO:teuthology.orchestra.run.smithi071.stdout:Deploying grafana service with default placement... 2024-04-23T02:04:07.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:07 smithi071 ceph-mon[39837]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:04:07.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:07 smithi071 ceph-mon[39837]: Saving service prometheus spec with placement count:1 2024-04-23T02:04:07.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:07 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:08.923 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-04-23T02:04:08.923 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.052+0000 7f10df4b0700 1 Processor -- start 2024-04-23T02:04:08.924 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.052+0000 7f10df4b0700 1 -- start start 2024-04-23T02:04:08.924 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.052+0000 7f10df4b0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d8104c10 0x7f10d8105030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:08.924 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.052+0000 7f10df4b0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d80744b0 con 0x7f10d8104c10 2024-04-23T02:04:08.924 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.053+0000 7f10dd24c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d8104c10 0x7f10d8105030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:08.924 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.053+0000 7f10dd24c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d8104c10 0x7f10d8105030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44680/0 (socket says 172.21.15.71:44680) 2024-04-23T02:04:08.924 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.053+0000 7f10dd24c700 1 -- 172.21.15.71:0/3592250083 learned_addr learned my addr 172.21.15.71:0/3592250083 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:08.924 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.053+0000 7f10dd24c700 1 -- 172.21.15.71:0/3592250083 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d8105570 con 0x7f10d8104c10 2024-04-23T02:04:08.924 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.053+0000 7f10dd24c700 1 --2- 172.21.15.71:0/3592250083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d8104c10 0x7f10d8105030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0098a0 tx=0x7f10cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591978bf4a2847d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:08.924 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.053+0000 7f10cbfff700 1 -- 172.21.15.71:0/3592250083 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f10cc004180 con 0x7f10d8104c10 2024-04-23T02:04:08.924 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.053+0000 7f10cbfff700 1 -- 172.21.15.71:0/3592250083 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f10cc0042e0 con 0x7f10d8104c10 2024-04-23T02:04:08.924 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.053+0000 7f10cbfff700 1 -- 172.21.15.71:0/3592250083 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f10cc011a30 con 0x7f10d8104c10 2024-04-23T02:04:08.924 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.054+0000 7f10df4b0700 1 -- 172.21.15.71:0/3592250083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d8104c10 msgr2=0x7f10d8105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:08.924 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.054+0000 7f10df4b0700 1 --2- 172.21.15.71:0/3592250083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d8104c10 0x7f10d8105030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0098a0 tx=0x7f10cc009bb0 comp rx=0 tx=0).stop 2024-04-23T02:04:08.925 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.054+0000 7f10df4b0700 1 -- 172.21.15.71:0/3592250083 shutdown_connections 2024-04-23T02:04:08.925 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.054+0000 7f10df4b0700 1 --2- 172.21.15.71:0/3592250083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d8104c10 0x7f10d8105030 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:08.925 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.054+0000 7f10df4b0700 1 -- 172.21.15.71:0/3592250083 >> 172.21.15.71:0/3592250083 conn(0x7f10d81007f0 msgr2=0x7f10d8102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:08.925 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.054+0000 7f10df4b0700 1 -- 172.21.15.71:0/3592250083 shutdown_connections 2024-04-23T02:04:08.925 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.054+0000 7f10df4b0700 1 -- 172.21.15.71:0/3592250083 wait complete. 2024-04-23T02:04:08.925 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.054+0000 7f10df4b0700 1 Processor -- start 2024-04-23T02:04:08.925 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.054+0000 7f10df4b0700 1 -- start start 2024-04-23T02:04:08.925 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.055+0000 7f10df4b0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d8104c10 0x7f10d819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:08.925 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.055+0000 7f10df4b0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d819c790 con 0x7f10d8104c10 2024-04-23T02:04:08.925 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.055+0000 7f10dd24c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d8104c10 0x7f10d819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:08.925 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.055+0000 7f10dd24c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d8104c10 0x7f10d819c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44684/0 (socket says 172.21.15.71:44684) 2024-04-23T02:04:08.925 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.055+0000 7f10dd24c700 1 -- 172.21.15.71:0/1844987407 learned_addr learned my addr 172.21.15.71:0/1844987407 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:08.925 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.055+0000 7f10dd24c700 1 -- 172.21.15.71:0/1844987407 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10cc009550 con 0x7f10d8104c10 2024-04-23T02:04:08.925 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.055+0000 7f10dd24c700 1 --2- 172.21.15.71:0/1844987407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d8104c10 0x7f10d819c250 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f10cc011a00 tx=0x7f10cc011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:08.925 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.055+0000 7f10ca7fc700 1 -- 172.21.15.71:0/1844987407 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f10cc003bd0 con 0x7f10d8104c10 2024-04-23T02:04:08.926 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.055+0000 7f10ca7fc700 1 -- 172.21.15.71:0/1844987407 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f10cc0248d0 con 0x7f10d8104c10 2024-04-23T02:04:08.926 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.055+0000 7f10df4b0700 1 -- 172.21.15.71:0/1844987407 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d819c990 con 0x7f10d8104c10 2024-04-23T02:04:08.926 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.055+0000 7f10ca7fc700 1 -- 172.21.15.71:0/1844987407 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f10cc01a7e0 con 0x7f10d8104c10 2024-04-23T02:04:08.926 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.055+0000 7f10df4b0700 1 -- 172.21.15.71:0/1844987407 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d819cdb0 con 0x7f10d8104c10 2024-04-23T02:04:08.926 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.056+0000 7f10ca7fc700 1 -- 172.21.15.71:0/1844987407 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f10cc01aa50 con 0x7f10d8104c10 2024-04-23T02:04:08.926 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.056+0000 7f10df4b0700 1 -- 172.21.15.71:0/1844987407 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d8195b20 con 0x7f10d8104c10 2024-04-23T02:04:08.926 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.056+0000 7f10ca7fc700 1 --2- 172.21.15.71:0/1844987407 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f10c4038320 0x7f10c403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:08.926 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.056+0000 7f10ca7fc700 1 -- 172.21.15.71:0/1844987407 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f10cc04c5d0 con 0x7f10d8104c10 2024-04-23T02:04:08.926 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.056+0000 7f10dca4b700 1 --2- 172.21.15.71:0/1844987407 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f10c4038320 0x7f10c403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:08.926 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.056+0000 7f10dca4b700 1 --2- 172.21.15.71:0/1844987407 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f10c4038320 0x7f10c403a7e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f10d4006fd0 tx=0x7f10d4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:08.926 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.061+0000 7f10ca7fc700 1 -- 172.21.15.71:0/1844987407 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f10cc0189a0 con 0x7f10d8104c10 2024-04-23T02:04:08.926 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.263+0000 7f10df4b0700 1 -- 172.21.15.71:0/1844987407 --> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f10d80611d0 con 0x7f10c4038320 2024-04-23T02:04:08.927 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.270+0000 7f10ca7fc700 1 -- 172.21.15.71:0/1844987407 <== mgr.14118 v2:172.21.15.71:6800/3878148961 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f10d80611d0 con 0x7f10c4038320 2024-04-23T02:04:08.927 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.272+0000 7f10df4b0700 1 -- 172.21.15.71:0/1844987407 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f10c4038320 msgr2=0x7f10c403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:08.927 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.272+0000 7f10df4b0700 1 --2- 172.21.15.71:0/1844987407 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f10c4038320 0x7f10c403a7e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f10d4006fd0 tx=0x7f10d4006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:08.927 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.272+0000 7f10df4b0700 1 -- 172.21.15.71:0/1844987407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d8104c10 msgr2=0x7f10d819c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:08.927 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.272+0000 7f10df4b0700 1 --2- 172.21.15.71:0/1844987407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d8104c10 0x7f10d819c250 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f10cc011a00 tx=0x7f10cc011ae0 comp rx=0 tx=0).stop 2024-04-23T02:04:08.927 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.272+0000 7f10df4b0700 1 -- 172.21.15.71:0/1844987407 shutdown_connections 2024-04-23T02:04:08.927 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.272+0000 7f10df4b0700 1 --2- 172.21.15.71:0/1844987407 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f10c4038320 0x7f10c403a7e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:08.927 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.272+0000 7f10df4b0700 1 --2- 172.21.15.71:0/1844987407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10d8104c10 0x7f10d819c250 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:08.927 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.272+0000 7f10df4b0700 1 -- 172.21.15.71:0/1844987407 >> 172.21.15.71:0/1844987407 conn(0x7f10d81007f0 msgr2=0x7f10d81907d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:08.927 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.272+0000 7f10df4b0700 1 -- 172.21.15.71:0/1844987407 shutdown_connections 2024-04-23T02:04:08.927 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:08.272+0000 7f10df4b0700 1 -- 172.21.15.71:0/1844987407 wait complete. 2024-04-23T02:04:08.927 INFO:teuthology.orchestra.run.smithi071.stdout:Deploying node-exporter service with default placement... 2024-04-23T02:04:09.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:09 smithi071 ceph-mon[39837]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:04:09.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:09 smithi071 ceph-mon[39837]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:04:09.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:09 smithi071 ceph-mon[39837]: Saving service grafana spec with placement count:1 2024-04-23T02:04:09.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:09 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:10.660 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-04-23T02:04:10.660 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.716+0000 7f1eb1bad700 1 Processor -- start 2024-04-23T02:04:10.661 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.716+0000 7f1eb1bad700 1 -- start start 2024-04-23T02:04:10.661 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.716+0000 7f1eb1bad700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1eac106de0 0x7f1eac1091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:10.661 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.716+0000 7f1eb1bad700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac0744f0 con 0x7f1eac106de0 2024-04-23T02:04:10.661 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.716+0000 7f1eab7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1eac106de0 0x7f1eac1091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:10.661 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.716+0000 7f1eab7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1eac106de0 0x7f1eac1091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44698/0 (socket says 172.21.15.71:44698) 2024-04-23T02:04:10.661 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.716+0000 7f1eab7fe700 1 -- 172.21.15.71:0/674711702 learned_addr learned my addr 172.21.15.71:0/674711702 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:10.661 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.716+0000 7f1eab7fe700 1 -- 172.21.15.71:0/674711702 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eac109710 con 0x7f1eac106de0 2024-04-23T02:04:10.661 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.716+0000 7f1eab7fe700 1 --2- 172.21.15.71:0/674711702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1eac106de0 0x7f1eac1091d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c0098a0 tx=0x7f1e9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f979f95107da5673 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:10.661 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.717+0000 7f1eaa7fc700 1 -- 172.21.15.71:0/674711702 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e9c004180 con 0x7f1eac106de0 2024-04-23T02:04:10.661 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.717+0000 7f1eaa7fc700 1 -- 172.21.15.71:0/674711702 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1e9c0042e0 con 0x7f1eac106de0 2024-04-23T02:04:10.661 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.717+0000 7f1eaa7fc700 1 -- 172.21.15.71:0/674711702 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e9c011a30 con 0x7f1eac106de0 2024-04-23T02:04:10.662 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.717+0000 7f1eb1bad700 1 -- 172.21.15.71:0/674711702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1eac106de0 msgr2=0x7f1eac1091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:10.662 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.717+0000 7f1eb1bad700 1 --2- 172.21.15.71:0/674711702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1eac106de0 0x7f1eac1091d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c0098a0 tx=0x7f1e9c009bb0 comp rx=0 tx=0).stop 2024-04-23T02:04:10.662 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.717+0000 7f1eb1bad700 1 -- 172.21.15.71:0/674711702 shutdown_connections 2024-04-23T02:04:10.662 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.717+0000 7f1eb1bad700 1 --2- 172.21.15.71:0/674711702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1eac106de0 0x7f1eac1091d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:10.662 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.717+0000 7f1eb1bad700 1 -- 172.21.15.71:0/674711702 >> 172.21.15.71:0/674711702 conn(0x7f1eac1007f0 msgr2=0x7f1eac102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:10.662 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.717+0000 7f1eb1bad700 1 -- 172.21.15.71:0/674711702 shutdown_connections 2024-04-23T02:04:10.662 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.717+0000 7f1eb1bad700 1 -- 172.21.15.71:0/674711702 wait complete. 2024-04-23T02:04:10.662 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.718+0000 7f1eb1bad700 1 Processor -- start 2024-04-23T02:04:10.662 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.718+0000 7f1eb1bad700 1 -- start start 2024-04-23T02:04:10.662 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.718+0000 7f1eb1bad700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1eac106de0 0x7f1eac106740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:10.662 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.718+0000 7f1eb1bad700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac106c80 con 0x7f1eac106de0 2024-04-23T02:04:10.662 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.718+0000 7f1eab7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1eac106de0 0x7f1eac106740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:10.662 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.718+0000 7f1eab7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1eac106de0 0x7f1eac106740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44704/0 (socket says 172.21.15.71:44704) 2024-04-23T02:04:10.662 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.718+0000 7f1eab7fe700 1 -- 172.21.15.71:0/334148585 learned_addr learned my addr 172.21.15.71:0/334148585 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:10.663 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.718+0000 7f1eab7fe700 1 -- 172.21.15.71:0/334148585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c009550 con 0x7f1eac106de0 2024-04-23T02:04:10.663 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.719+0000 7f1eab7fe700 1 --2- 172.21.15.71:0/334148585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1eac106de0 0x7f1eac106740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c006af0 tx=0x7f1e9c011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:10.663 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.719+0000 7f1ea8ff9700 1 -- 172.21.15.71:0/334148585 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e9c003a80 con 0x7f1eac106de0 2024-04-23T02:04:10.663 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.719+0000 7f1ea8ff9700 1 -- 172.21.15.71:0/334148585 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1e9c003be0 con 0x7f1eac106de0 2024-04-23T02:04:10.663 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.719+0000 7f1eb1bad700 1 -- 172.21.15.71:0/334148585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eac104d90 con 0x7f1eac106de0 2024-04-23T02:04:10.663 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.719+0000 7f1ea8ff9700 1 -- 172.21.15.71:0/334148585 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e9c019770 con 0x7f1eac106de0 2024-04-23T02:04:10.663 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.719+0000 7f1eb1bad700 1 -- 172.21.15.71:0/334148585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eac1051b0 con 0x7f1eac106de0 2024-04-23T02:04:10.663 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.720+0000 7f1ea8ff9700 1 -- 172.21.15.71:0/334148585 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f1e9c0199e0 con 0x7f1eac106de0 2024-04-23T02:04:10.663 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.720+0000 7f1ea8ff9700 1 --2- 172.21.15.71:0/334148585 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f1e94038330 0x7f1e9403a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:10.663 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.720+0000 7f1eb1bad700 1 -- 172.21.15.71:0/334148585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eac10a440 con 0x7f1eac106de0 2024-04-23T02:04:10.663 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.720+0000 7f1ea8ff9700 1 -- 172.21.15.71:0/334148585 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1e9c04c560 con 0x7f1eac106de0 2024-04-23T02:04:10.663 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.720+0000 7f1eaaffd700 1 --2- 172.21.15.71:0/334148585 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f1e94038330 0x7f1e9403a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:10.663 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.720+0000 7f1eaaffd700 1 --2- 172.21.15.71:0/334148585 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f1e94038330 0x7f1e9403a7f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0006fd0 tx=0x7f1ea0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:10.663 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.724+0000 7f1ea8ff9700 1 -- 172.21.15.71:0/334148585 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1e9c02c3a0 con 0x7f1eac106de0 2024-04-23T02:04:10.663 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.923+0000 7f1eb1bad700 1 -- 172.21.15.71:0/334148585 --> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f1eac102b00 con 0x7f1e94038330 2024-04-23T02:04:10.663 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.927+0000 7f1ea8ff9700 1 -- 172.21.15.71:0/334148585 <== mgr.14118 v2:172.21.15.71:6800/3878148961 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f1eac102b00 con 0x7f1e94038330 2024-04-23T02:04:10.664 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.929+0000 7f1eb1bad700 1 -- 172.21.15.71:0/334148585 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f1e94038330 msgr2=0x7f1e9403a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:10.664 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.929+0000 7f1eb1bad700 1 --2- 172.21.15.71:0/334148585 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f1e94038330 0x7f1e9403a7f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0006fd0 tx=0x7f1ea0006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:10.664 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.929+0000 7f1eb1bad700 1 -- 172.21.15.71:0/334148585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1eac106de0 msgr2=0x7f1eac106740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:10.664 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.929+0000 7f1eb1bad700 1 --2- 172.21.15.71:0/334148585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1eac106de0 0x7f1eac106740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c006af0 tx=0x7f1e9c011b20 comp rx=0 tx=0).stop 2024-04-23T02:04:10.664 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.929+0000 7f1eb1bad700 1 -- 172.21.15.71:0/334148585 shutdown_connections 2024-04-23T02:04:10.664 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.929+0000 7f1eb1bad700 1 --2- 172.21.15.71:0/334148585 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f1e94038330 0x7f1e9403a7f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:10.664 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.929+0000 7f1eb1bad700 1 --2- 172.21.15.71:0/334148585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1eac106de0 0x7f1eac106740 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:10.664 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.929+0000 7f1eb1bad700 1 -- 172.21.15.71:0/334148585 >> 172.21.15.71:0/334148585 conn(0x7f1eac1007f0 msgr2=0x7f1eac101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:10.664 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.929+0000 7f1eb1bad700 1 -- 172.21.15.71:0/334148585 shutdown_connections 2024-04-23T02:04:10.664 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:09.929+0000 7f1eb1bad700 1 -- 172.21.15.71:0/334148585 wait complete. 2024-04-23T02:04:10.664 INFO:teuthology.orchestra.run.smithi071.stdout:Deploying alertmanager service with default placement... 2024-04-23T02:04:11.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:10 smithi071 ceph-mon[39837]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:04:11.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:10 smithi071 ceph-mon[39837]: Saving service node-exporter spec with placement * 2024-04-23T02:04:11.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:10 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:11.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:10 smithi071 ceph-mon[39837]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:04:12.064 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-04-23T02:04:12.064 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.200+0000 7fd658766700 1 Processor -- start 2024-04-23T02:04:12.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.200+0000 7fd658766700 1 -- start start 2024-04-23T02:04:12.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.200+0000 7fd658766700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd650106e00 0x7fd6501091f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:12.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.200+0000 7fd658766700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6500744d0 con 0x7fd650106e00 2024-04-23T02:04:12.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.200+0000 7fd656502700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd650106e00 0x7fd6501091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:12.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.200+0000 7fd656502700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd650106e00 0x7fd6501091f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44706/0 (socket says 172.21.15.71:44706) 2024-04-23T02:04:12.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.200+0000 7fd656502700 1 -- 172.21.15.71:0/2013708809 learned_addr learned my addr 172.21.15.71:0/2013708809 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:12.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.201+0000 7fd656502700 1 -- 172.21.15.71:0/2013708809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd650109730 con 0x7fd650106e00 2024-04-23T02:04:12.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.201+0000 7fd656502700 1 --2- 172.21.15.71:0/2013708809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd650106e00 0x7fd6501091f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd6400098a0 tx=0x7fd640009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2e6549173f5b5d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:12.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.201+0000 7fd655500700 1 -- 172.21.15.71:0/2013708809 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd640004180 con 0x7fd650106e00 2024-04-23T02:04:12.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.201+0000 7fd655500700 1 -- 172.21.15.71:0/2013708809 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd6400042e0 con 0x7fd650106e00 2024-04-23T02:04:12.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.201+0000 7fd655500700 1 -- 172.21.15.71:0/2013708809 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd640011a30 con 0x7fd650106e00 2024-04-23T02:04:12.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.201+0000 7fd658766700 1 -- 172.21.15.71:0/2013708809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd650106e00 msgr2=0x7fd6501091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:12.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.201+0000 7fd658766700 1 --2- 172.21.15.71:0/2013708809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd650106e00 0x7fd6501091f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd6400098a0 tx=0x7fd640009bb0 comp rx=0 tx=0).stop 2024-04-23T02:04:12.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.201+0000 7fd658766700 1 -- 172.21.15.71:0/2013708809 shutdown_connections 2024-04-23T02:04:12.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.201+0000 7fd658766700 1 --2- 172.21.15.71:0/2013708809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd650106e00 0x7fd6501091f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:12.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.201+0000 7fd658766700 1 -- 172.21.15.71:0/2013708809 >> 172.21.15.71:0/2013708809 conn(0x7fd6501007f0 msgr2=0x7fd650102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:12.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.201+0000 7fd658766700 1 -- 172.21.15.71:0/2013708809 shutdown_connections 2024-04-23T02:04:12.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.201+0000 7fd658766700 1 -- 172.21.15.71:0/2013708809 wait complete. 2024-04-23T02:04:12.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.202+0000 7fd658766700 1 Processor -- start 2024-04-23T02:04:12.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.202+0000 7fd658766700 1 -- start start 2024-04-23T02:04:12.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.202+0000 7fd658766700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd650106e00 0x7fd65019c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:12.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.202+0000 7fd658766700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65019c7b0 con 0x7fd650106e00 2024-04-23T02:04:12.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.202+0000 7fd656502700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd650106e00 0x7fd65019c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:12.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.202+0000 7fd656502700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd650106e00 0x7fd65019c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44710/0 (socket says 172.21.15.71:44710) 2024-04-23T02:04:12.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.202+0000 7fd656502700 1 -- 172.21.15.71:0/564873919 learned_addr learned my addr 172.21.15.71:0/564873919 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:12.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.202+0000 7fd656502700 1 -- 172.21.15.71:0/564873919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd640009550 con 0x7fd650106e00 2024-04-23T02:04:12.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.203+0000 7fd656502700 1 --2- 172.21.15.71:0/564873919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd650106e00 0x7fd65019c270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd640011a00 tx=0x7fd640011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:12.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.203+0000 7fd6477fe700 1 -- 172.21.15.71:0/564873919 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd640003bd0 con 0x7fd650106e00 2024-04-23T02:04:12.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.203+0000 7fd6477fe700 1 -- 172.21.15.71:0/564873919 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd6400248d0 con 0x7fd650106e00 2024-04-23T02:04:12.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.203+0000 7fd658766700 1 -- 172.21.15.71:0/564873919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65019c9b0 con 0x7fd650106e00 2024-04-23T02:04:12.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.203+0000 7fd6477fe700 1 -- 172.21.15.71:0/564873919 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd64001a7e0 con 0x7fd650106e00 2024-04-23T02:04:12.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.203+0000 7fd658766700 1 -- 172.21.15.71:0/564873919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65019cdd0 con 0x7fd650106e00 2024-04-23T02:04:12.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.204+0000 7fd6477fe700 1 -- 172.21.15.71:0/564873919 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fd64001aa70 con 0x7fd650106e00 2024-04-23T02:04:12.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.204+0000 7fd658766700 1 -- 172.21.15.71:0/564873919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6500623c0 con 0x7fd650106e00 2024-04-23T02:04:12.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.204+0000 7fd6477fe700 1 --2- 172.21.15.71:0/564873919 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fd63c038320 0x7fd63c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:12.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.204+0000 7fd6477fe700 1 -- 172.21.15.71:0/564873919 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd64004c5e0 con 0x7fd650106e00 2024-04-23T02:04:12.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.204+0000 7fd655d01700 1 --2- 172.21.15.71:0/564873919 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fd63c038320 0x7fd63c03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:12.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.204+0000 7fd655d01700 1 --2- 172.21.15.71:0/564873919 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fd63c038320 0x7fd63c03a7e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd64c006fd0 tx=0x7fd64c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:12.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.209+0000 7fd6477fe700 1 -- 172.21.15.71:0/564873919 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd64001ad20 con 0x7fd650106e00 2024-04-23T02:04:12.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.410+0000 7fd658766700 1 -- 172.21.15.71:0/564873919 --> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fd650102b30 con 0x7fd63c038320 2024-04-23T02:04:12.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.415+0000 7fd6477fe700 1 -- 172.21.15.71:0/564873919 <== mgr.14118 v2:172.21.15.71:6800/3878148961 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fd650102b30 con 0x7fd63c038320 2024-04-23T02:04:12.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.417+0000 7fd658766700 1 -- 172.21.15.71:0/564873919 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fd63c038320 msgr2=0x7fd63c03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:12.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.417+0000 7fd658766700 1 --2- 172.21.15.71:0/564873919 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fd63c038320 0x7fd63c03a7e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd64c006fd0 tx=0x7fd64c006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:12.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.417+0000 7fd658766700 1 -- 172.21.15.71:0/564873919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd650106e00 msgr2=0x7fd65019c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:12.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.417+0000 7fd658766700 1 --2- 172.21.15.71:0/564873919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd650106e00 0x7fd65019c270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd640011a00 tx=0x7fd640011ae0 comp rx=0 tx=0).stop 2024-04-23T02:04:12.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.417+0000 7fd658766700 1 -- 172.21.15.71:0/564873919 shutdown_connections 2024-04-23T02:04:12.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.417+0000 7fd658766700 1 --2- 172.21.15.71:0/564873919 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fd63c038320 0x7fd63c03a7e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:12.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.417+0000 7fd658766700 1 --2- 172.21.15.71:0/564873919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd650106e00 0x7fd65019c270 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:12.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.417+0000 7fd658766700 1 -- 172.21.15.71:0/564873919 >> 172.21.15.71:0/564873919 conn(0x7fd6501007f0 msgr2=0x7fd650101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:12.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.417+0000 7fd658766700 1 -- 172.21.15.71:0/564873919 shutdown_connections 2024-04-23T02:04:12.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:11.417+0000 7fd658766700 1 -- 172.21.15.71:0/564873919 wait complete. 2024-04-23T02:04:12.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:12 smithi071 ceph-mon[39837]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:04:12.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:12 smithi071 ceph-mon[39837]: Saving service alertmanager spec with placement count:1 2024-04-23T02:04:12.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:12 smithi071 ceph-mon[39837]: from='mgr.14118 172.21.15.71:0/1697862615' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:13.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:13 smithi071 ceph-mon[39837]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:04:13.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:13 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/4126688793' entity='client.admin' 2024-04-23T02:04:13.512 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.640+0000 7f215f0a1700 1 Processor -- start 2024-04-23T02:04:13.512 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.640+0000 7f215f0a1700 1 -- start start 2024-04-23T02:04:13.512 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.640+0000 7f215f0a1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2158105230 0x7f2158105650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:13.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.640+0000 7f215f0a1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2158105c20 con 0x7f2158105230 2024-04-23T02:04:13.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.640+0000 7f215ce3d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2158105230 0x7f2158105650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:13.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.640+0000 7f215ce3d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2158105230 0x7f2158105650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44718/0 (socket says 172.21.15.71:44718) 2024-04-23T02:04:13.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.640+0000 7f215ce3d700 1 -- 172.21.15.71:0/577846488 learned_addr learned my addr 172.21.15.71:0/577846488 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:13.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.640+0000 7f215ce3d700 1 -- 172.21.15.71:0/577846488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2158106430 con 0x7f2158105230 2024-04-23T02:04:13.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.640+0000 7f215ce3d700 1 --2- 172.21.15.71:0/577846488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2158105230 0x7f2158105650 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f214c0098a0 tx=0x7f214c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc6c7b2ffa7135b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:13.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.641+0000 7f21577fe700 1 -- 172.21.15.71:0/577846488 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f214c004180 con 0x7f2158105230 2024-04-23T02:04:13.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.641+0000 7f21577fe700 1 -- 172.21.15.71:0/577846488 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f214c0042e0 con 0x7f2158105230 2024-04-23T02:04:13.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.641+0000 7f21577fe700 1 -- 172.21.15.71:0/577846488 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f214c011a30 con 0x7f2158105230 2024-04-23T02:04:13.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.641+0000 7f215f0a1700 1 -- 172.21.15.71:0/577846488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2158105230 msgr2=0x7f2158105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:13.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.641+0000 7f215f0a1700 1 --2- 172.21.15.71:0/577846488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2158105230 0x7f2158105650 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f214c0098a0 tx=0x7f214c009bb0 comp rx=0 tx=0).stop 2024-04-23T02:04:13.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.641+0000 7f215f0a1700 1 -- 172.21.15.71:0/577846488 shutdown_connections 2024-04-23T02:04:13.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.641+0000 7f215f0a1700 1 --2- 172.21.15.71:0/577846488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2158105230 0x7f2158105650 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:13.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.641+0000 7f215f0a1700 1 -- 172.21.15.71:0/577846488 >> 172.21.15.71:0/577846488 conn(0x7f21581007f0 msgr2=0x7f2158102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:13.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.641+0000 7f215f0a1700 1 -- 172.21.15.71:0/577846488 shutdown_connections 2024-04-23T02:04:13.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.641+0000 7f215f0a1700 1 -- 172.21.15.71:0/577846488 wait complete. 2024-04-23T02:04:13.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.642+0000 7f215f0a1700 1 Processor -- start 2024-04-23T02:04:13.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.642+0000 7f215f0a1700 1 -- start start 2024-04-23T02:04:13.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.642+0000 7f215f0a1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2158105230 0x7f21580741b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:13.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.642+0000 7f215f0a1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215819abc0 con 0x7f2158105230 2024-04-23T02:04:13.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.642+0000 7f215ce3d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2158105230 0x7f21580741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:13.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.642+0000 7f215ce3d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2158105230 0x7f21580741b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44720/0 (socket says 172.21.15.71:44720) 2024-04-23T02:04:13.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.642+0000 7f215ce3d700 1 -- 172.21.15.71:0/4126688793 learned_addr learned my addr 172.21.15.71:0/4126688793 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:13.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.642+0000 7f215ce3d700 1 -- 172.21.15.71:0/4126688793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c009550 con 0x7f2158105230 2024-04-23T02:04:13.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.643+0000 7f215ce3d700 1 --2- 172.21.15.71:0/4126688793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2158105230 0x7f21580741b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f214c006af0 tx=0x7f214c011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:13.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.643+0000 7f2155ffb700 1 -- 172.21.15.71:0/4126688793 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f214c003a80 con 0x7f2158105230 2024-04-23T02:04:13.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.643+0000 7f2155ffb700 1 -- 172.21.15.71:0/4126688793 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f214c003be0 con 0x7f2158105230 2024-04-23T02:04:13.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.643+0000 7f2155ffb700 1 -- 172.21.15.71:0/4126688793 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f214c019770 con 0x7f2158105230 2024-04-23T02:04:13.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.643+0000 7f215f0a1700 1 -- 172.21.15.71:0/4126688793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215819f1d0 con 0x7f2158105230 2024-04-23T02:04:13.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.643+0000 7f215f0a1700 1 -- 172.21.15.71:0/4126688793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f215819f530 con 0x7f2158105230 2024-04-23T02:04:13.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.644+0000 7f2155ffb700 1 -- 172.21.15.71:0/4126688793 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f214c0199e0 con 0x7f2158105230 2024-04-23T02:04:13.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.644+0000 7f2155ffb700 1 --2- 172.21.15.71:0/4126688793 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f2148038330 0x7f214803a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:13.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.644+0000 7f215f0a1700 1 -- 172.21.15.71:0/4126688793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2158193010 con 0x7f2158105230 2024-04-23T02:04:13.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.644+0000 7f2155ffb700 1 -- 172.21.15.71:0/4126688793 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f214c04c590 con 0x7f2158105230 2024-04-23T02:04:13.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.644+0000 7f2157fff700 1 --2- 172.21.15.71:0/4126688793 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f2148038330 0x7f214803a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:13.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.644+0000 7f2157fff700 1 --2- 172.21.15.71:0/4126688793 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f2148038330 0x7f214803a7f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2144006fd0 tx=0x7f2144006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:13.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.649+0000 7f2155ffb700 1 -- 172.21.15.71:0/4126688793 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f214c02d440 con 0x7f2158105230 2024-04-23T02:04:13.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.836+0000 7f215f0a1700 1 -- 172.21.15.71:0/4126688793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f215802d090 con 0x7f2158105230 2024-04-23T02:04:13.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.839+0000 7f2155ffb700 1 -- 172.21.15.71:0/4126688793 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f214c018350 con 0x7f2158105230 2024-04-23T02:04:13.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.841+0000 7f215f0a1700 1 -- 172.21.15.71:0/4126688793 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f2148038330 msgr2=0x7f214803a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:13.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.841+0000 7f215f0a1700 1 --2- 172.21.15.71:0/4126688793 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f2148038330 0x7f214803a7f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2144006fd0 tx=0x7f2144006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:13.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.841+0000 7f215f0a1700 1 -- 172.21.15.71:0/4126688793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2158105230 msgr2=0x7f21580741b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:13.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.841+0000 7f215f0a1700 1 --2- 172.21.15.71:0/4126688793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2158105230 0x7f21580741b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f214c006af0 tx=0x7f214c011b20 comp rx=0 tx=0).stop 2024-04-23T02:04:13.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.841+0000 7f215f0a1700 1 -- 172.21.15.71:0/4126688793 shutdown_connections 2024-04-23T02:04:13.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.841+0000 7f215f0a1700 1 --2- 172.21.15.71:0/4126688793 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f2148038330 0x7f214803a7f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:13.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.841+0000 7f215f0a1700 1 --2- 172.21.15.71:0/4126688793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2158105230 0x7f21580741b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:13.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.841+0000 7f215f0a1700 1 -- 172.21.15.71:0/4126688793 >> 172.21.15.71:0/4126688793 conn(0x7f21581007f0 msgr2=0x7f2158102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:13.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.842+0000 7f215f0a1700 1 -- 172.21.15.71:0/4126688793 shutdown_connections 2024-04-23T02:04:13.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:12.842+0000 7f215f0a1700 1 -- 172.21.15.71:0/4126688793 wait complete. 2024-04-23T02:04:14.834 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.051+0000 7f49dc335700 1 Processor -- start 2024-04-23T02:04:14.834 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.051+0000 7f49dc335700 1 -- start start 2024-04-23T02:04:14.834 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.051+0000 7f49dc335700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4106de0 0x7f49d41091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:14.834 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.051+0000 7f49dc335700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d40744f0 con 0x7f49d4106de0 2024-04-23T02:04:14.834 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.051+0000 7f49da0d1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4106de0 0x7f49d41091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:14.834 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.051+0000 7f49da0d1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4106de0 0x7f49d41091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44728/0 (socket says 172.21.15.71:44728) 2024-04-23T02:04:14.834 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.051+0000 7f49da0d1700 1 -- 172.21.15.71:0/3809890866 learned_addr learned my addr 172.21.15.71:0/3809890866 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:14.834 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.052+0000 7f49da0d1700 1 -- 172.21.15.71:0/3809890866 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d4109710 con 0x7f49d4106de0 2024-04-23T02:04:14.834 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.052+0000 7f49da0d1700 1 --2- 172.21.15.71:0/3809890866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4106de0 0x7f49d41091d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f49d00098a0 tx=0x7f49d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644e5b9f978ac9bd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:14.835 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.052+0000 7f49d90cf700 1 -- 172.21.15.71:0/3809890866 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f49d0004180 con 0x7f49d4106de0 2024-04-23T02:04:14.835 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.052+0000 7f49d90cf700 1 -- 172.21.15.71:0/3809890866 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f49d00042e0 con 0x7f49d4106de0 2024-04-23T02:04:14.835 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.052+0000 7f49d90cf700 1 -- 172.21.15.71:0/3809890866 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f49d0011a30 con 0x7f49d4106de0 2024-04-23T02:04:14.835 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.052+0000 7f49dc335700 1 -- 172.21.15.71:0/3809890866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4106de0 msgr2=0x7f49d41091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:14.835 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.052+0000 7f49dc335700 1 --2- 172.21.15.71:0/3809890866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4106de0 0x7f49d41091d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f49d00098a0 tx=0x7f49d0009bb0 comp rx=0 tx=0).stop 2024-04-23T02:04:14.835 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.052+0000 7f49dc335700 1 -- 172.21.15.71:0/3809890866 shutdown_connections 2024-04-23T02:04:14.835 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.052+0000 7f49dc335700 1 --2- 172.21.15.71:0/3809890866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4106de0 0x7f49d41091d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:14.835 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.052+0000 7f49dc335700 1 -- 172.21.15.71:0/3809890866 >> 172.21.15.71:0/3809890866 conn(0x7f49d41007f0 msgr2=0x7f49d4102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:14.835 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.053+0000 7f49dc335700 1 -- 172.21.15.71:0/3809890866 shutdown_connections 2024-04-23T02:04:14.835 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.053+0000 7f49dc335700 1 -- 172.21.15.71:0/3809890866 wait complete. 2024-04-23T02:04:14.835 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.053+0000 7f49dc335700 1 Processor -- start 2024-04-23T02:04:14.835 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.053+0000 7f49dc335700 1 -- start start 2024-04-23T02:04:14.835 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.053+0000 7f49dc335700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4106de0 0x7f49d419c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:14.836 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.053+0000 7f49dc335700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d419c7b0 con 0x7f49d4106de0 2024-04-23T02:04:14.836 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.053+0000 7f49da0d1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4106de0 0x7f49d419c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:14.836 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.053+0000 7f49da0d1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4106de0 0x7f49d419c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44736/0 (socket says 172.21.15.71:44736) 2024-04-23T02:04:14.836 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.053+0000 7f49da0d1700 1 -- 172.21.15.71:0/664480563 learned_addr learned my addr 172.21.15.71:0/664480563 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:14.836 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.054+0000 7f49da0d1700 1 -- 172.21.15.71:0/664480563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d0009550 con 0x7f49d4106de0 2024-04-23T02:04:14.836 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.054+0000 7f49da0d1700 1 --2- 172.21.15.71:0/664480563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4106de0 0x7f49d419c270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f49d0011a00 tx=0x7f49d0011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:14.836 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.054+0000 7f49cb7fe700 1 -- 172.21.15.71:0/664480563 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f49d0003bd0 con 0x7f49d4106de0 2024-04-23T02:04:14.836 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.054+0000 7f49cb7fe700 1 -- 172.21.15.71:0/664480563 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f49d00248d0 con 0x7f49d4106de0 2024-04-23T02:04:14.836 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.054+0000 7f49dc335700 1 -- 172.21.15.71:0/664480563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d419c9b0 con 0x7f49d4106de0 2024-04-23T02:04:14.836 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.054+0000 7f49cb7fe700 1 -- 172.21.15.71:0/664480563 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f49d001a7e0 con 0x7f49d4106de0 2024-04-23T02:04:14.836 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.054+0000 7f49dc335700 1 -- 172.21.15.71:0/664480563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d419cdd0 con 0x7f49d4106de0 2024-04-23T02:04:14.836 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.055+0000 7f49cb7fe700 1 -- 172.21.15.71:0/664480563 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f49d0024440 con 0x7f49d4106de0 2024-04-23T02:04:14.837 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.055+0000 7f49cb7fe700 1 --2- 172.21.15.71:0/664480563 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f49c0038370 0x7f49c003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:14.837 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.055+0000 7f49cb7fe700 1 -- 172.21.15.71:0/664480563 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f49d004c430 con 0x7f49d4106de0 2024-04-23T02:04:14.837 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.055+0000 7f49d98d0700 1 --2- 172.21.15.71:0/664480563 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f49c0038370 0x7f49c003a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:14.837 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.055+0000 7f49dc335700 1 -- 172.21.15.71:0/664480563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d40623c0 con 0x7f49d4106de0 2024-04-23T02:04:14.837 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.055+0000 7f49d98d0700 1 --2- 172.21.15.71:0/664480563 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f49c0038370 0x7f49c003a830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f49c4006fd0 tx=0x7f49c4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:14.837 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.060+0000 7f49cb7fe700 1 -- 172.21.15.71:0/664480563 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f49d00246f0 con 0x7f49d4106de0 2024-04-23T02:04:14.837 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.245+0000 7f49dc335700 1 -- 172.21.15.71:0/664480563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f49d419f810 con 0x7f49d4106de0 2024-04-23T02:04:14.837 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.248+0000 7f49cb7fe700 1 -- 172.21.15.71:0/664480563 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f49d0018350 con 0x7f49d4106de0 2024-04-23T02:04:14.837 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.250+0000 7f49dc335700 1 -- 172.21.15.71:0/664480563 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f49c0038370 msgr2=0x7f49c003a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:14.837 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.250+0000 7f49dc335700 1 --2- 172.21.15.71:0/664480563 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f49c0038370 0x7f49c003a830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f49c4006fd0 tx=0x7f49c4006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:14.837 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.251+0000 7f49dc335700 1 -- 172.21.15.71:0/664480563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4106de0 msgr2=0x7f49d419c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:14.837 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.251+0000 7f49dc335700 1 --2- 172.21.15.71:0/664480563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4106de0 0x7f49d419c270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f49d0011a00 tx=0x7f49d0011ae0 comp rx=0 tx=0).stop 2024-04-23T02:04:14.837 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.251+0000 7f49dc335700 1 -- 172.21.15.71:0/664480563 shutdown_connections 2024-04-23T02:04:14.837 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.251+0000 7f49dc335700 1 --2- 172.21.15.71:0/664480563 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f49c0038370 0x7f49c003a830 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:14.838 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.251+0000 7f49dc335700 1 --2- 172.21.15.71:0/664480563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d4106de0 0x7f49d419c270 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:14.838 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.251+0000 7f49dc335700 1 -- 172.21.15.71:0/664480563 >> 172.21.15.71:0/664480563 conn(0x7f49d41007f0 msgr2=0x7f49d4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:14.838 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.251+0000 7f49dc335700 1 -- 172.21.15.71:0/664480563 shutdown_connections 2024-04-23T02:04:14.838 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:14.251+0000 7f49dc335700 1 -- 172.21.15.71:0/664480563 wait complete. 2024-04-23T02:04:14.838 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling the dashboard module... 2024-04-23T02:04:15.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:15 smithi071 ceph-mon[39837]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:04:15.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:15 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/664480563' entity='client.admin' 2024-04-23T02:04:16.914 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:16 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/595473847' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-23T02:04:17.291 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.423+0000 7feae6af8700 1 Processor -- start 2024-04-23T02:04:17.291 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.423+0000 7feae6af8700 1 -- start start 2024-04-23T02:04:17.291 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.423+0000 7feae6af8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feae0106da0 0x7feae0109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:17.291 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.423+0000 7feae6af8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae0079de0 con 0x7feae0106da0 2024-04-23T02:04:17.291 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.423+0000 7feae4894700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feae0106da0 0x7feae0109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:17.291 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.423+0000 7feae4894700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feae0106da0 0x7feae0109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34204/0 (socket says 172.21.15.71:34204) 2024-04-23T02:04:17.291 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.423+0000 7feae4894700 1 -- 172.21.15.71:0/3411515823 learned_addr learned my addr 172.21.15.71:0/3411515823 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:17.291 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.424+0000 7feae4894700 1 -- 172.21.15.71:0/3411515823 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae01096d0 con 0x7feae0106da0 2024-04-23T02:04:17.291 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.424+0000 7feae4894700 1 --2- 172.21.15.71:0/3411515823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feae0106da0 0x7feae0109190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fead40098a0 tx=0x7fead4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e528052d383b4333 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:17.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.424+0000 7feadf7fe700 1 -- 172.21.15.71:0/3411515823 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fead4004180 con 0x7feae0106da0 2024-04-23T02:04:17.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.424+0000 7feadf7fe700 1 -- 172.21.15.71:0/3411515823 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fead40042e0 con 0x7feae0106da0 2024-04-23T02:04:17.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.424+0000 7feadf7fe700 1 -- 172.21.15.71:0/3411515823 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fead4011a30 con 0x7feae0106da0 2024-04-23T02:04:17.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.424+0000 7feae6af8700 1 -- 172.21.15.71:0/3411515823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feae0106da0 msgr2=0x7feae0109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:17.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.424+0000 7feae6af8700 1 --2- 172.21.15.71:0/3411515823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feae0106da0 0x7feae0109190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fead40098a0 tx=0x7fead4009bb0 comp rx=0 tx=0).stop 2024-04-23T02:04:17.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.424+0000 7feae6af8700 1 -- 172.21.15.71:0/3411515823 shutdown_connections 2024-04-23T02:04:17.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.424+0000 7feae6af8700 1 --2- 172.21.15.71:0/3411515823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feae0106da0 0x7feae0109190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:17.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.424+0000 7feae6af8700 1 -- 172.21.15.71:0/3411515823 >> 172.21.15.71:0/3411515823 conn(0x7feae01007d0 msgr2=0x7feae0102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:17.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.424+0000 7feae6af8700 1 -- 172.21.15.71:0/3411515823 shutdown_connections 2024-04-23T02:04:17.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.425+0000 7feae6af8700 1 -- 172.21.15.71:0/3411515823 wait complete. 2024-04-23T02:04:17.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.425+0000 7feae6af8700 1 Processor -- start 2024-04-23T02:04:17.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.425+0000 7feae6af8700 1 -- start start 2024-04-23T02:04:17.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.425+0000 7feae6af8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feae0106da0 0x7feae0106700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:17.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.425+0000 7feae6af8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae0106c40 con 0x7feae0106da0 2024-04-23T02:04:17.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.425+0000 7feae4894700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feae0106da0 0x7feae0106700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:17.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.425+0000 7feae4894700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feae0106da0 0x7feae0106700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34218/0 (socket says 172.21.15.71:34218) 2024-04-23T02:04:17.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.425+0000 7feae4894700 1 -- 172.21.15.71:0/595473847 learned_addr learned my addr 172.21.15.71:0/595473847 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:17.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.426+0000 7feae4894700 1 -- 172.21.15.71:0/595473847 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead4009550 con 0x7feae0106da0 2024-04-23T02:04:17.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.426+0000 7feae4894700 1 --2- 172.21.15.71:0/595473847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feae0106da0 0x7feae0106700 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fead4011a00 tx=0x7fead4011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:17.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.426+0000 7feaddffb700 1 -- 172.21.15.71:0/595473847 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fead4003bc0 con 0x7feae0106da0 2024-04-23T02:04:17.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.426+0000 7feaddffb700 1 -- 172.21.15.71:0/595473847 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fead40248c0 con 0x7feae0106da0 2024-04-23T02:04:17.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.426+0000 7feaddffb700 1 -- 172.21.15.71:0/595473847 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fead401a850 con 0x7feae0106da0 2024-04-23T02:04:17.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.426+0000 7feae6af8700 1 -- 172.21.15.71:0/595473847 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae0104d50 con 0x7feae0106da0 2024-04-23T02:04:17.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.426+0000 7feae6af8700 1 -- 172.21.15.71:0/595473847 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae0105170 con 0x7feae0106da0 2024-04-23T02:04:17.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.427+0000 7feaddffb700 1 -- 172.21.15.71:0/595473847 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fead4024a30 con 0x7feae0106da0 2024-04-23T02:04:17.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.427+0000 7feae6af8700 1 -- 172.21.15.71:0/595473847 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae00623c0 con 0x7feae0106da0 2024-04-23T02:04:17.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.427+0000 7feaddffb700 1 --2- 172.21.15.71:0/595473847 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fead0038370 0x7fead003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:17.293 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.427+0000 7feaddffb700 1 -- 172.21.15.71:0/595473847 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fead404c5e0 con 0x7feae0106da0 2024-04-23T02:04:17.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.427+0000 7feadffff700 1 --2- 172.21.15.71:0/595473847 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fead0038370 0x7fead003a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:17.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.428+0000 7feadffff700 1 --2- 172.21.15.71:0/595473847 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fead0038370 0x7fead003a830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7feacc006fd0 tx=0x7feacc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:17.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.432+0000 7feaddffb700 1 -- 172.21.15.71:0/595473847 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fead4024ce0 con 0x7feae0106da0 2024-04-23T02:04:17.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:15.672+0000 7feae6af8700 1 -- 172.21.15.71:0/595473847 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7feae0105470 con 0x7feae0106da0 2024-04-23T02:04:17.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:16.564+0000 7feaddffb700 1 -- 172.21.15.71:0/595473847 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fead402d910 con 0x7feae0106da0 2024-04-23T02:04:17.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:16.566+0000 7feaddffb700 1 -- 172.21.15.71:0/595473847 <== mon.0 v2:172.21.15.71:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fead4018b80 con 0x7feae0106da0 2024-04-23T02:04:17.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:16.569+0000 7feae6af8700 1 -- 172.21.15.71:0/595473847 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fead0038370 msgr2=0x7fead003a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:17.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:16.569+0000 7feae6af8700 1 --2- 172.21.15.71:0/595473847 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fead0038370 0x7fead003a830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7feacc006fd0 tx=0x7feacc006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:17.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:16.569+0000 7feae6af8700 1 -- 172.21.15.71:0/595473847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feae0106da0 msgr2=0x7feae0106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:17.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:16.569+0000 7feae6af8700 1 --2- 172.21.15.71:0/595473847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feae0106da0 0x7feae0106700 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fead4011a00 tx=0x7fead4011ae0 comp rx=0 tx=0).stop 2024-04-23T02:04:17.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:16.569+0000 7feae6af8700 1 -- 172.21.15.71:0/595473847 shutdown_connections 2024-04-23T02:04:17.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:16.569+0000 7feae6af8700 1 --2- 172.21.15.71:0/595473847 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7fead0038370 0x7fead003a830 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:17.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:16.569+0000 7feae6af8700 1 --2- 172.21.15.71:0/595473847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feae0106da0 0x7feae0106700 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:17.294 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:16.569+0000 7feae6af8700 1 -- 172.21.15.71:0/595473847 >> 172.21.15.71:0/595473847 conn(0x7feae01007d0 msgr2=0x7feae0101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:17.295 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:16.569+0000 7feae6af8700 1 -- 172.21.15.71:0/595473847 shutdown_connections 2024-04-23T02:04:17.295 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:16.569+0000 7feae6af8700 1 -- 172.21.15.71:0/595473847 wait complete. 2024-04-23T02:04:17.973 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:17 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/595473847' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-23T02:04:17.973 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:17 smithi071 ceph-mon[39837]: mgrmap e9: smithi071.ukdjvn(active, since 28s) 2024-04-23T02:04:18.900 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-04-23T02:04:18.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-23T02:04:18.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T02:04:18.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "active_name": "smithi071.ukdjvn", 2024-04-23T02:04:18.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-23T02:04:18.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-04-23T02:04:18.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.921+0000 7f3eec428700 1 Processor -- start 2024-04-23T02:04:18.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.921+0000 7f3eec428700 1 -- start start 2024-04-23T02:04:18.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.921+0000 7f3eec428700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee41067f0 0x7f3ee4106c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:18.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.921+0000 7f3eec428700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee41071e0 con 0x7f3ee41067f0 2024-04-23T02:04:18.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.921+0000 7f3eea1c4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee41067f0 0x7f3ee4106c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:18.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.921+0000 7f3eea1c4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee41067f0 0x7f3ee4106c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34246/0 (socket says 172.21.15.71:34246) 2024-04-23T02:04:18.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.921+0000 7f3eea1c4700 1 -- 172.21.15.71:0/879120114 learned_addr learned my addr 172.21.15.71:0/879120114 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:18.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.921+0000 7f3eea1c4700 1 -- 172.21.15.71:0/879120114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee4107320 con 0x7f3ee41067f0 2024-04-23T02:04:18.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.922+0000 7f3eea1c4700 1 --2- 172.21.15.71:0/879120114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee41067f0 0x7f3ee4106c10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00098a0 tx=0x7f3ee0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c36d384d616e419 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:18.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.922+0000 7f3ee91c2700 1 -- 172.21.15.71:0/879120114 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ee0004180 con 0x7f3ee41067f0 2024-04-23T02:04:18.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.922+0000 7f3ee91c2700 1 -- 172.21.15.71:0/879120114 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f3ee00042e0 con 0x7f3ee41067f0 2024-04-23T02:04:18.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.922+0000 7f3ee91c2700 1 -- 172.21.15.71:0/879120114 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ee0011a30 con 0x7f3ee41067f0 2024-04-23T02:04:18.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.922+0000 7f3eec428700 1 -- 172.21.15.71:0/879120114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee41067f0 msgr2=0x7f3ee4106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:18.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.922+0000 7f3eec428700 1 --2- 172.21.15.71:0/879120114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee41067f0 0x7f3ee4106c10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00098a0 tx=0x7f3ee0009bb0 comp rx=0 tx=0).stop 2024-04-23T02:04:18.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.923+0000 7f3eec428700 1 -- 172.21.15.71:0/879120114 shutdown_connections 2024-04-23T02:04:18.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.923+0000 7f3eec428700 1 --2- 172.21.15.71:0/879120114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee41067f0 0x7f3ee4106c10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:18.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.923+0000 7f3eec428700 1 -- 172.21.15.71:0/879120114 >> 172.21.15.71:0/879120114 conn(0x7f3ee40fff40 msgr2=0x7f3ee4102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:18.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.923+0000 7f3eec428700 1 -- 172.21.15.71:0/879120114 shutdown_connections 2024-04-23T02:04:18.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.923+0000 7f3eec428700 1 -- 172.21.15.71:0/879120114 wait complete. 2024-04-23T02:04:18.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.923+0000 7f3eec428700 1 Processor -- start 2024-04-23T02:04:18.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.923+0000 7f3eec428700 1 -- start start 2024-04-23T02:04:18.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.923+0000 7f3eec428700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee41067f0 0x7f3ee419c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:18.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.923+0000 7f3eec428700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee419c740 con 0x7f3ee41067f0 2024-04-23T02:04:18.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.924+0000 7f3eea1c4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee41067f0 0x7f3ee419c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:18.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.924+0000 7f3eea1c4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee41067f0 0x7f3ee419c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34252/0 (socket says 172.21.15.71:34252) 2024-04-23T02:04:18.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.924+0000 7f3eea1c4700 1 -- 172.21.15.71:0/352692309 learned_addr learned my addr 172.21.15.71:0/352692309 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:18.908 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.924+0000 7f3eea1c4700 1 -- 172.21.15.71:0/352692309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee0009550 con 0x7f3ee41067f0 2024-04-23T02:04:18.908 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.924+0000 7f3eea1c4700 1 --2- 172.21.15.71:0/352692309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee41067f0 0x7f3ee419c200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0006af0 tx=0x7f3ee0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:18.908 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.924+0000 7f3edb7fe700 1 -- 172.21.15.71:0/352692309 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ee0003bc0 con 0x7f3ee41067f0 2024-04-23T02:04:18.908 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.924+0000 7f3edb7fe700 1 -- 172.21.15.71:0/352692309 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f3ee001a8c0 con 0x7f3ee41067f0 2024-04-23T02:04:18.908 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.924+0000 7f3eec428700 1 -- 172.21.15.71:0/352692309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee419c940 con 0x7f3ee41067f0 2024-04-23T02:04:18.908 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.924+0000 7f3edb7fe700 1 -- 172.21.15.71:0/352692309 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ee0019860 con 0x7f3ee41067f0 2024-04-23T02:04:18.908 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.924+0000 7f3eec428700 1 -- 172.21.15.71:0/352692309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee419cd60 con 0x7f3ee41067f0 2024-04-23T02:04:18.908 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.925+0000 7f3edb7fe700 1 -- 172.21.15.71:0/352692309 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f3ee0019b20 con 0x7f3ee41067f0 2024-04-23T02:04:18.908 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.925+0000 7f3eec428700 1 -- 172.21.15.71:0/352692309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee4195ee0 con 0x7f3ee41067f0 2024-04-23T02:04:18.908 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:18 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/352692309' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-23T02:04:18.911 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.925+0000 7f3edb7fe700 1 --2- 172.21.15.71:0/352692309 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f3ed0038330 0x7f3ed003a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:18.911 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.925+0000 7f3edb7fe700 1 -- 172.21.15.71:0/352692309 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3ee004c5b0 con 0x7f3ee41067f0 2024-04-23T02:04:18.911 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.925+0000 7f3ee99c3700 1 -- 172.21.15.71:0/352692309 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f3ed0038330 msgr2=0x7f3ed003a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3878148961 2024-04-23T02:04:18.911 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.925+0000 7f3ee99c3700 1 --2- 172.21.15.71:0/352692309 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f3ed0038330 0x7f3ed003a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T02:04:18.911 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:17.936+0000 7f3edb7fe700 1 -- 172.21.15.71:0/352692309 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3ee002d430 con 0x7f3ee41067f0 2024-04-23T02:04:18.911 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:18.126+0000 7f3ee99c3700 1 -- 172.21.15.71:0/352692309 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f3ed0038330 msgr2=0x7f3ed003a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3878148961 2024-04-23T02:04:18.911 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:18.126+0000 7f3ee99c3700 1 --2- 172.21.15.71:0/352692309 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f3ed0038330 0x7f3ed003a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T02:04:18.911 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:18.206+0000 7f3eec428700 1 -- 172.21.15.71:0/352692309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f3ee40623c0 con 0x7f3ee41067f0 2024-04-23T02:04:18.911 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:18.206+0000 7f3edb7fe700 1 -- 172.21.15.71:0/352692309 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f3ee001aa30 con 0x7f3ee41067f0 2024-04-23T02:04:18.911 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:18.208+0000 7f3eec428700 1 -- 172.21.15.71:0/352692309 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f3ed0038330 msgr2=0x7f3ed003a7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T02:04:18.911 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:18.208+0000 7f3eec428700 1 --2- 172.21.15.71:0/352692309 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f3ed0038330 0x7f3ed003a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:18.911 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:18.208+0000 7f3eec428700 1 -- 172.21.15.71:0/352692309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee41067f0 msgr2=0x7f3ee419c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:18.911 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:18.208+0000 7f3eec428700 1 --2- 172.21.15.71:0/352692309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee41067f0 0x7f3ee419c200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0006af0 tx=0x7f3ee0011ab0 comp rx=0 tx=0).stop 2024-04-23T02:04:18.912 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:18.209+0000 7f3eec428700 1 -- 172.21.15.71:0/352692309 shutdown_connections 2024-04-23T02:04:18.912 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:18.209+0000 7f3eec428700 1 --2- 172.21.15.71:0/352692309 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f3ed0038330 0x7f3ed003a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:18.912 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:18.209+0000 7f3eec428700 1 --2- 172.21.15.71:0/352692309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ee41067f0 0x7f3ee419c200 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:18.912 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:18.209+0000 7f3eec428700 1 -- 172.21.15.71:0/352692309 >> 172.21.15.71:0/352692309 conn(0x7f3ee40fff40 msgr2=0x7f3ee4193020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:18.912 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:18.209+0000 7f3eec428700 1 -- 172.21.15.71:0/352692309 shutdown_connections 2024-04-23T02:04:18.912 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:18.209+0000 7f3eec428700 1 -- 172.21.15.71:0/352692309 wait complete. 2024-04-23T02:04:18.912 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for the mgr to restart... 2024-04-23T02:04:18.912 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr epoch 9... 2024-04-23T02:04:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:23 smithi071 ceph-mon[39837]: Active manager daemon smithi071.ukdjvn restarted 2024-04-23T02:04:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:23 smithi071 ceph-mon[39837]: Activating manager daemon smithi071.ukdjvn 2024-04-23T02:04:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:23 smithi071 ceph-mon[39837]: osdmap e3: 0 total, 0 up, 0 in 2024-04-23T02:04:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:23 smithi071 ceph-mon[39837]: mgrmap e10: smithi071.ukdjvn(active, starting, since 0.00564018s) 2024-04-23T02:04:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:23 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi071"}]: dispatch 2024-04-23T02:04:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:23 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr metadata", "who": "smithi071.ukdjvn", "id": "smithi071.ukdjvn"}]: dispatch 2024-04-23T02:04:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:23 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T02:04:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:23 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T02:04:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:23 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T02:04:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:23 smithi071 ceph-mon[39837]: Manager daemon smithi071.ukdjvn is now available 2024-04-23T02:04:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:23 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:04:23.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:23 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:04:24.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:24 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi071.ukdjvn/mirror_snapshot_schedule"}]: dispatch 2024-04-23T02:04:24.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:24 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi071.ukdjvn/trash_purge_schedule"}]: dispatch 2024-04-23T02:04:24.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:24 smithi071 ceph-mon[39837]: [23/Apr/2024:02:04:23] ENGINE Bus STARTING 2024-04-23T02:04:24.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:24 smithi071 ceph-mon[39837]: [23/Apr/2024:02:04:23] ENGINE Serving on https://172.21.15.71:7150 2024-04-23T02:04:24.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:24 smithi071 ceph-mon[39837]: mgrmap e11: smithi071.ukdjvn(active, since 1.0055s) 2024-04-23T02:04:25.125 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-04-23T02:04:25.125 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-23T02:04:25.125 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-23T02:04:25.125 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-04-23T02:04:25.125 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.447+0000 7f7c88177700 1 Processor -- start 2024-04-23T02:04:25.125 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.447+0000 7f7c88177700 1 -- start start 2024-04-23T02:04:25.125 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.447+0000 7f7c88177700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c80104610 0x7f7c80104a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:25.125 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.447+0000 7f7c88177700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c80105000 con 0x7f7c80104610 2024-04-23T02:04:25.125 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.447+0000 7f7c85f13700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c80104610 0x7f7c80104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:25.125 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.447+0000 7f7c85f13700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c80104610 0x7f7c80104a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34260/0 (socket says 172.21.15.71:34260) 2024-04-23T02:04:25.125 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.447+0000 7f7c85f13700 1 -- 172.21.15.71:0/3216782530 learned_addr learned my addr 172.21.15.71:0/3216782530 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:25.126 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.447+0000 7f7c85f13700 1 -- 172.21.15.71:0/3216782530 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c801075a0 con 0x7f7c80104610 2024-04-23T02:04:25.126 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.448+0000 7f7c85f13700 1 --2- 172.21.15.71:0/3216782530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c80104610 0x7f7c80104a30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0098a0 tx=0x7f7c7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fdb025ba7a7b9b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:25.126 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.448+0000 7f7c84f11700 1 -- 172.21.15.71:0/3216782530 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7c7c004180 con 0x7f7c80104610 2024-04-23T02:04:25.126 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.448+0000 7f7c84f11700 1 -- 172.21.15.71:0/3216782530 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7c7c0042e0 con 0x7f7c80104610 2024-04-23T02:04:25.126 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.448+0000 7f7c84f11700 1 -- 172.21.15.71:0/3216782530 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7c7c011a30 con 0x7f7c80104610 2024-04-23T02:04:25.126 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.448+0000 7f7c88177700 1 -- 172.21.15.71:0/3216782530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c80104610 msgr2=0x7f7c80104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:25.126 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.448+0000 7f7c88177700 1 --2- 172.21.15.71:0/3216782530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c80104610 0x7f7c80104a30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0098a0 tx=0x7f7c7c009bb0 comp rx=0 tx=0).stop 2024-04-23T02:04:25.126 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.449+0000 7f7c88177700 1 -- 172.21.15.71:0/3216782530 shutdown_connections 2024-04-23T02:04:25.126 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.449+0000 7f7c88177700 1 --2- 172.21.15.71:0/3216782530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c80104610 0x7f7c80104a30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:25.126 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.449+0000 7f7c88177700 1 -- 172.21.15.71:0/3216782530 >> 172.21.15.71:0/3216782530 conn(0x7f7c800fff40 msgr2=0x7f7c80102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:25.126 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.449+0000 7f7c88177700 1 -- 172.21.15.71:0/3216782530 shutdown_connections 2024-04-23T02:04:25.126 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.449+0000 7f7c88177700 1 -- 172.21.15.71:0/3216782530 wait complete. 2024-04-23T02:04:25.126 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.449+0000 7f7c88177700 1 Processor -- start 2024-04-23T02:04:25.127 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.450+0000 7f7c88177700 1 -- start start 2024-04-23T02:04:25.127 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.450+0000 7f7c88177700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c80104610 0x7f7c80199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:25.127 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.450+0000 7f7c88177700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8019a530 con 0x7f7c80104610 2024-04-23T02:04:25.127 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.450+0000 7f7c85f13700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c80104610 0x7f7c80199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:25.127 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.450+0000 7f7c85f13700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c80104610 0x7f7c80199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34266/0 (socket says 172.21.15.71:34266) 2024-04-23T02:04:25.127 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.450+0000 7f7c85f13700 1 -- 172.21.15.71:0/1600279418 learned_addr learned my addr 172.21.15.71:0/1600279418 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:25.127 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.450+0000 7f7c85f13700 1 -- 172.21.15.71:0/1600279418 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c7c009550 con 0x7f7c80104610 2024-04-23T02:04:25.127 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.450+0000 7f7c85f13700 1 --2- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c80104610 0x7f7c80199ff0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c000c00 tx=0x7f7c7c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:25.127 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.450+0000 7f7c76ffd700 1 -- 172.21.15.71:0/1600279418 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7c7c003bc0 con 0x7f7c80104610 2024-04-23T02:04:25.127 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.450+0000 7f7c76ffd700 1 -- 172.21.15.71:0/1600279418 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7c7c0248d0 con 0x7f7c80104610 2024-04-23T02:04:25.127 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.450+0000 7f7c88177700 1 -- 172.21.15.71:0/1600279418 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c8019a730 con 0x7f7c80104610 2024-04-23T02:04:25.127 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.451+0000 7f7c76ffd700 1 -- 172.21.15.71:0/1600279418 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7c7c01a890 con 0x7f7c80104610 2024-04-23T02:04:25.127 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.451+0000 7f7c88177700 1 -- 172.21.15.71:0/1600279418 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c8019ab50 con 0x7f7c80104610 2024-04-23T02:04:25.127 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.451+0000 7f7c76ffd700 1 -- 172.21.15.71:0/1600279418 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f7c7c01ab50 con 0x7f7c80104610 2024-04-23T02:04:25.128 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.452+0000 7f7c76ffd700 1 --2- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f7c6c038370 0x7f7c6c03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:25.128 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.452+0000 7f7c76ffd700 1 -- 172.21.15.71:0/1600279418 --> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c6c03af40 con 0x7f7c6c038370 2024-04-23T02:04:25.128 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.452+0000 7f7c76ffd700 1 -- 172.21.15.71:0/1600279418 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7c7c04c730 con 0x7f7c80104610 2024-04-23T02:04:25.128 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.452+0000 7f7c85712700 1 -- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f7c6c038370 msgr2=0x7f7c6c03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3878148961 2024-04-23T02:04:25.128 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.452+0000 7f7c85712700 1 --2- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f7c6c038370 0x7f7c6c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T02:04:25.128 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.653+0000 7f7c85712700 1 -- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f7c6c038370 msgr2=0x7f7c6c03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3878148961 2024-04-23T02:04:25.128 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:19.653+0000 7f7c85712700 1 --2- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f7c6c038370 0x7f7c6c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T02:04:25.128 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:20.054+0000 7f7c85712700 1 -- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f7c6c038370 msgr2=0x7f7c6c03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3878148961 2024-04-23T02:04:25.128 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:20.054+0000 7f7c85712700 1 --2- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f7c6c038370 0x7f7c6c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T02:04:25.128 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:20.855+0000 7f7c85712700 1 -- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f7c6c038370 msgr2=0x7f7c6c03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3878148961 2024-04-23T02:04:25.128 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:20.855+0000 7f7c85712700 1 --2- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f7c6c038370 0x7f7c6c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-23T02:04:25.128 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:22.456+0000 7f7c85712700 1 -- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f7c6c038370 msgr2=0x7f7c6c03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3878148961 2024-04-23T02:04:25.128 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:22.456+0000 7f7c85712700 1 --2- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f7c6c038370 0x7f7c6c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-23T02:04:25.128 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:23.251+0000 7f7c76ffd700 1 -- 172.21.15.71:0/1600279418 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mgrmap(e 10) v1 ==== 45063+0+0 (secure 0 0 0) 0x7f7c7c02d3f0 con 0x7f7c80104610 2024-04-23T02:04:25.128 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:23.251+0000 7f7c76ffd700 1 -- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f7c6c038370 msgr2=0x7f7c6c03a830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T02:04:25.129 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:23.251+0000 7f7c76ffd700 1 --2- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f7c6c038370 0x7f7c6c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:25.129 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.250+0000 7f7c76ffd700 1 -- 172.21.15.71:0/1600279418 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f7c7c04e5e0 con 0x7f7c80104610 2024-04-23T02:04:25.129 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.250+0000 7f7c76ffd700 1 --2- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f7c6c03bde0 0x7f7c6c03e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:25.129 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.250+0000 7f7c76ffd700 1 -- 172.21.15.71:0/1600279418 --> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c6c03af40 con 0x7f7c6c03bde0 2024-04-23T02:04:25.129 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.251+0000 7f7c85712700 1 --2- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f7c6c03bde0 0x7f7c6c03e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:25.129 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.251+0000 7f7c85712700 1 --2- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f7c6c03bde0 0x7f7c6c03e1d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7c70003a10 tx=0x7f7c700092b0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:25.129 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.252+0000 7f7c76ffd700 1 -- 172.21.15.71:0/1600279418 <== mgr.14162 v2:172.21.15.71:6800/4165305842 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f7c6c03af40 con 0x7f7c6c03bde0 2024-04-23T02:04:25.129 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.258+0000 7f7c88177700 1 -- 172.21.15.71:0/1600279418 --> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f7c8019b330 con 0x7f7c6c03bde0 2024-04-23T02:04:25.129 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.258+0000 7f7c76ffd700 1 -- 172.21.15.71:0/1600279418 <== mgr.14162 v2:172.21.15.71:6800/4165305842 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f7c8019b330 con 0x7f7c6c03bde0 2024-04-23T02:04:25.129 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.258+0000 7f7c88177700 1 -- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f7c6c03bde0 msgr2=0x7f7c6c03e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:25.129 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.258+0000 7f7c88177700 1 --2- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f7c6c03bde0 0x7f7c6c03e1d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7c70003a10 tx=0x7f7c700092b0 comp rx=0 tx=0).stop 2024-04-23T02:04:25.129 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.258+0000 7f7c88177700 1 -- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c80104610 msgr2=0x7f7c80199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:25.129 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.258+0000 7f7c88177700 1 --2- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c80104610 0x7f7c80199ff0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c000c00 tx=0x7f7c7c011ab0 comp rx=0 tx=0).stop 2024-04-23T02:04:25.129 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.258+0000 7f7c88177700 1 -- 172.21.15.71:0/1600279418 shutdown_connections 2024-04-23T02:04:25.129 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.258+0000 7f7c88177700 1 --2- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f7c6c03bde0 0x7f7c6c03e1d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:25.130 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.258+0000 7f7c88177700 1 --2- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c80104610 0x7f7c80199ff0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:25.130 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.258+0000 7f7c88177700 1 --2- 172.21.15.71:0/1600279418 >> [v2:172.21.15.71:6800/3878148961,v1:172.21.15.71:6801/3878148961] conn(0x7f7c6c038370 0x7f7c6c03a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:25.130 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.258+0000 7f7c88177700 1 -- 172.21.15.71:0/1600279418 >> 172.21.15.71:0/1600279418 conn(0x7f7c800fff40 msgr2=0x7f7c801015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:25.130 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.258+0000 7f7c88177700 1 -- 172.21.15.71:0/1600279418 shutdown_connections 2024-04-23T02:04:25.130 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:24.258+0000 7f7c88177700 1 -- 172.21.15.71:0/1600279418 wait complete. 2024-04-23T02:04:25.130 INFO:teuthology.orchestra.run.smithi071.stdout:mgr epoch 9 is available 2024-04-23T02:04:25.130 INFO:teuthology.orchestra.run.smithi071.stdout:Generating a dashboard self-signed certificate... 2024-04-23T02:04:25.379 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:25 smithi071 ceph-mon[39837]: [23/Apr/2024:02:04:23] ENGINE Serving on http://172.21.15.71:8765 2024-04-23T02:04:25.379 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:25 smithi071 ceph-mon[39837]: [23/Apr/2024:02:04:23] ENGINE Bus STARTED 2024-04-23T02:04:25.379 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:25 smithi071 ceph-mon[39837]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-23T02:04:25.379 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:25 smithi071 ceph-mon[39837]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-23T02:04:25.379 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:25 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:26.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:26 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:26.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:26 smithi071 ceph-mon[39837]: mgrmap e12: smithi071.ukdjvn(active, since 2s) 2024-04-23T02:04:26.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:26 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:26.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:26 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:26.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-23T02:04:26.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.713+0000 7fb2d42ab700 1 Processor -- start 2024-04-23T02:04:26.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.713+0000 7fb2d42ab700 1 -- start start 2024-04-23T02:04:26.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.714+0000 7fb2d42ab700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc07be10 0x7fb2cc07a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:26.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.714+0000 7fb2d42ab700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc07a840 con 0x7fb2cc07be10 2024-04-23T02:04:26.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.714+0000 7fb2d2047700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc07be10 0x7fb2cc07a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:26.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.714+0000 7fb2d2047700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc07be10 0x7fb2cc07a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48706/0 (socket says 172.21.15.71:48706) 2024-04-23T02:04:26.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.714+0000 7fb2d2047700 1 -- 172.21.15.71:0/1341654633 learned_addr learned my addr 172.21.15.71:0/1341654633 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:26.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.714+0000 7fb2d2047700 1 -- 172.21.15.71:0/1341654633 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2cc07a980 con 0x7fb2cc07be10 2024-04-23T02:04:26.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.714+0000 7fb2d2047700 1 --2- 172.21.15.71:0/1341654633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc07be10 0x7fb2cc07a270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0098a0 tx=0x7fb2bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca524746967ec0d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:26.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.714+0000 7fb2d1045700 1 -- 172.21.15.71:0/1341654633 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2bc004180 con 0x7fb2cc07be10 2024-04-23T02:04:26.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.715+0000 7fb2d1045700 1 -- 172.21.15.71:0/1341654633 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fb2bc0042e0 con 0x7fb2cc07be10 2024-04-23T02:04:26.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.715+0000 7fb2d1045700 1 -- 172.21.15.71:0/1341654633 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2bc011a30 con 0x7fb2cc07be10 2024-04-23T02:04:26.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.715+0000 7fb2d42ab700 1 -- 172.21.15.71:0/1341654633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc07be10 msgr2=0x7fb2cc07a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:26.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.715+0000 7fb2d42ab700 1 --2- 172.21.15.71:0/1341654633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc07be10 0x7fb2cc07a270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0098a0 tx=0x7fb2bc009bb0 comp rx=0 tx=0).stop 2024-04-23T02:04:26.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.715+0000 7fb2d42ab700 1 -- 172.21.15.71:0/1341654633 shutdown_connections 2024-04-23T02:04:26.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.715+0000 7fb2d42ab700 1 --2- 172.21.15.71:0/1341654633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc07be10 0x7fb2cc07a270 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:26.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.715+0000 7fb2d42ab700 1 -- 172.21.15.71:0/1341654633 >> 172.21.15.71:0/1341654633 conn(0x7fb2cc1018d0 msgr2=0x7fb2cc103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:26.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.715+0000 7fb2d42ab700 1 -- 172.21.15.71:0/1341654633 shutdown_connections 2024-04-23T02:04:26.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.715+0000 7fb2d42ab700 1 -- 172.21.15.71:0/1341654633 wait complete. 2024-04-23T02:04:26.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.716+0000 7fb2d42ab700 1 Processor -- start 2024-04-23T02:04:26.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.716+0000 7fb2d42ab700 1 -- start start 2024-04-23T02:04:26.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.716+0000 7fb2d42ab700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc07be10 0x7fb2cc19c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:26.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.716+0000 7fb2d42ab700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc19c7b0 con 0x7fb2cc07be10 2024-04-23T02:04:26.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.716+0000 7fb2d2047700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc07be10 0x7fb2cc19c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:26.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.716+0000 7fb2d2047700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc07be10 0x7fb2cc19c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48714/0 (socket says 172.21.15.71:48714) 2024-04-23T02:04:26.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.716+0000 7fb2d2047700 1 -- 172.21.15.71:0/2551305800 learned_addr learned my addr 172.21.15.71:0/2551305800 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:26.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.717+0000 7fb2d2047700 1 -- 172.21.15.71:0/2551305800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc009550 con 0x7fb2cc07be10 2024-04-23T02:04:26.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.717+0000 7fb2d2047700 1 --2- 172.21.15.71:0/2551305800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc07be10 0x7fb2cc19c270 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc006af0 tx=0x7fb2bc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:26.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.717+0000 7fb2c37fe700 1 -- 172.21.15.71:0/2551305800 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2bc003bc0 con 0x7fb2cc07be10 2024-04-23T02:04:26.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.717+0000 7fb2c37fe700 1 -- 172.21.15.71:0/2551305800 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fb2bc01a8c0 con 0x7fb2cc07be10 2024-04-23T02:04:26.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.717+0000 7fb2d42ab700 1 -- 172.21.15.71:0/2551305800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2cc19c9b0 con 0x7fb2cc07be10 2024-04-23T02:04:26.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.717+0000 7fb2c37fe700 1 -- 172.21.15.71:0/2551305800 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2bc019820 con 0x7fb2cc07be10 2024-04-23T02:04:26.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.717+0000 7fb2d42ab700 1 -- 172.21.15.71:0/2551305800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2cc19cdd0 con 0x7fb2cc07be10 2024-04-23T02:04:26.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.718+0000 7fb2c37fe700 1 -- 172.21.15.71:0/2551305800 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fb2bc019aa0 con 0x7fb2cc07be10 2024-04-23T02:04:26.752 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.718+0000 7fb2d42ab700 1 -- 172.21.15.71:0/2551305800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2cc0623c0 con 0x7fb2cc07be10 2024-04-23T02:04:26.752 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.718+0000 7fb2c37fe700 1 --2- 172.21.15.71:0/2551305800 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fb2b8038330 0x7fb2b803a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:26.752 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.718+0000 7fb2c37fe700 1 -- 172.21.15.71:0/2551305800 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb2bc04cbf0 con 0x7fb2cc07be10 2024-04-23T02:04:26.752 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.718+0000 7fb2d1846700 1 --2- 172.21.15.71:0/2551305800 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fb2b8038330 0x7fb2b803a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:26.752 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.718+0000 7fb2d1846700 1 --2- 172.21.15.71:0/2551305800 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fb2b8038330 0x7fb2b803a7f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8006fd0 tx=0x7fb2c8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:26.752 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.723+0000 7fb2c37fe700 1 -- 172.21.15.71:0/2551305800 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb2bc0243f0 con 0x7fb2cc07be10 2024-04-23T02:04:26.752 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:25.933+0000 7fb2d42ab700 1 -- 172.21.15.71:0/2551305800 --> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2cc103c20 con 0x7fb2b8038330 2024-04-23T02:04:26.752 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:26.074+0000 7fb2c37fe700 1 -- 172.21.15.71:0/2551305800 <== mgr.14162 v2:172.21.15.71:6800/4165305842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fb2cc103c20 con 0x7fb2b8038330 2024-04-23T02:04:26.752 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:26.076+0000 7fb2d42ab700 1 -- 172.21.15.71:0/2551305800 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fb2b8038330 msgr2=0x7fb2b803a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:26.752 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:26.076+0000 7fb2d42ab700 1 --2- 172.21.15.71:0/2551305800 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fb2b8038330 0x7fb2b803a7f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8006fd0 tx=0x7fb2c8006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:26.752 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:26.076+0000 7fb2d42ab700 1 -- 172.21.15.71:0/2551305800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc07be10 msgr2=0x7fb2cc19c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:26.752 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:26.076+0000 7fb2d42ab700 1 --2- 172.21.15.71:0/2551305800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc07be10 0x7fb2cc19c270 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc006af0 tx=0x7fb2bc011ab0 comp rx=0 tx=0).stop 2024-04-23T02:04:26.752 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:26.076+0000 7fb2d42ab700 1 -- 172.21.15.71:0/2551305800 shutdown_connections 2024-04-23T02:04:26.752 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:26.076+0000 7fb2d42ab700 1 --2- 172.21.15.71:0/2551305800 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fb2b8038330 0x7fb2b803a7f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:26.752 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:26.076+0000 7fb2d42ab700 1 --2- 172.21.15.71:0/2551305800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc07be10 0x7fb2cc19c270 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:26.753 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:26.076+0000 7fb2d42ab700 1 -- 172.21.15.71:0/2551305800 >> 172.21.15.71:0/2551305800 conn(0x7fb2cc1018d0 msgr2=0x7fb2cc103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:26.753 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:26.076+0000 7fb2d42ab700 1 -- 172.21.15.71:0/2551305800 shutdown_connections 2024-04-23T02:04:26.753 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:26.076+0000 7fb2d42ab700 1 -- 172.21.15.71:0/2551305800 wait complete. 2024-04-23T02:04:26.753 INFO:teuthology.orchestra.run.smithi071.stdout:Creating initial admin user... 2024-04-23T02:04:27.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:27 smithi071 ceph-mon[39837]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:04:28.983 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$8DxTvzuxdcY/21U8Bnowy.Tfy57QhsCcAE1tE2ngVp92/oWlkrYy.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713837868, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-23T02:04:28.983 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.900+0000 7f8c4354d700 1 Processor -- start 2024-04-23T02:04:28.983 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.900+0000 7f8c4354d700 1 -- start start 2024-04-23T02:04:28.983 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.901+0000 7f8c4354d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c3c106de0 0x7f8c3c1091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:28.983 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.901+0000 7f8c4354d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c0744f0 con 0x7f8c3c106de0 2024-04-23T02:04:28.983 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.901+0000 7f8c412e9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c3c106de0 0x7f8c3c1091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:28.983 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.901+0000 7f8c412e9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c3c106de0 0x7f8c3c1091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48722/0 (socket says 172.21.15.71:48722) 2024-04-23T02:04:28.983 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.901+0000 7f8c412e9700 1 -- 172.21.15.71:0/4113746812 learned_addr learned my addr 172.21.15.71:0/4113746812 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:28.983 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.901+0000 7f8c412e9700 1 -- 172.21.15.71:0/4113746812 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c109710 con 0x7f8c3c106de0 2024-04-23T02:04:28.984 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.901+0000 7f8c412e9700 1 --2- 172.21.15.71:0/4113746812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c3c106de0 0x7f8c3c1091d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c0098a0 tx=0x7f8c2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e056f5075b4dac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:28.984 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.901+0000 7f8c33fff700 1 -- 172.21.15.71:0/4113746812 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c2c004180 con 0x7f8c3c106de0 2024-04-23T02:04:28.984 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.901+0000 7f8c33fff700 1 -- 172.21.15.71:0/4113746812 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f8c2c0042e0 con 0x7f8c3c106de0 2024-04-23T02:04:28.984 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.901+0000 7f8c33fff700 1 -- 172.21.15.71:0/4113746812 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c2c011a30 con 0x7f8c3c106de0 2024-04-23T02:04:28.984 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.902+0000 7f8c4354d700 1 -- 172.21.15.71:0/4113746812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c3c106de0 msgr2=0x7f8c3c1091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:28.984 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.902+0000 7f8c4354d700 1 --2- 172.21.15.71:0/4113746812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c3c106de0 0x7f8c3c1091d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c0098a0 tx=0x7f8c2c009bb0 comp rx=0 tx=0).stop 2024-04-23T02:04:28.984 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.902+0000 7f8c4354d700 1 -- 172.21.15.71:0/4113746812 shutdown_connections 2024-04-23T02:04:28.984 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.902+0000 7f8c4354d700 1 --2- 172.21.15.71:0/4113746812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c3c106de0 0x7f8c3c1091d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:28.984 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.902+0000 7f8c4354d700 1 -- 172.21.15.71:0/4113746812 >> 172.21.15.71:0/4113746812 conn(0x7f8c3c1007f0 msgr2=0x7f8c3c102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:28.984 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.902+0000 7f8c4354d700 1 -- 172.21.15.71:0/4113746812 shutdown_connections 2024-04-23T02:04:28.984 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.902+0000 7f8c4354d700 1 -- 172.21.15.71:0/4113746812 wait complete. 2024-04-23T02:04:28.984 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.902+0000 7f8c4354d700 1 Processor -- start 2024-04-23T02:04:28.984 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.903+0000 7f8c4354d700 1 -- start start 2024-04-23T02:04:28.984 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.903+0000 7f8c4354d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c3c106de0 0x7f8c3c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:28.985 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.903+0000 7f8c4354d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c19a5b0 con 0x7f8c3c106de0 2024-04-23T02:04:28.985 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.903+0000 7f8c412e9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c3c106de0 0x7f8c3c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:28.985 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.903+0000 7f8c412e9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c3c106de0 0x7f8c3c19a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48728/0 (socket says 172.21.15.71:48728) 2024-04-23T02:04:28.985 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.903+0000 7f8c412e9700 1 -- 172.21.15.71:0/2620715239 learned_addr learned my addr 172.21.15.71:0/2620715239 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:28.985 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.903+0000 7f8c412e9700 1 -- 172.21.15.71:0/2620715239 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c009550 con 0x7f8c3c106de0 2024-04-23T02:04:28.985 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.903+0000 7f8c412e9700 1 --2- 172.21.15.71:0/2620715239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c3c106de0 0x7f8c3c19a070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c006af0 tx=0x7f8c2c011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:28.985 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.903+0000 7f8c327fc700 1 -- 172.21.15.71:0/2620715239 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c2c003a80 con 0x7f8c3c106de0 2024-04-23T02:04:28.985 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.903+0000 7f8c327fc700 1 -- 172.21.15.71:0/2620715239 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f8c2c003be0 con 0x7f8c3c106de0 2024-04-23T02:04:28.985 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.903+0000 7f8c4354d700 1 -- 172.21.15.71:0/2620715239 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3c19a7b0 con 0x7f8c3c106de0 2024-04-23T02:04:28.985 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.903+0000 7f8c327fc700 1 -- 172.21.15.71:0/2620715239 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c2c019770 con 0x7f8c3c106de0 2024-04-23T02:04:28.985 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.903+0000 7f8c4354d700 1 -- 172.21.15.71:0/2620715239 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3c19abd0 con 0x7f8c3c106de0 2024-04-23T02:04:28.985 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.904+0000 7f8c327fc700 1 -- 172.21.15.71:0/2620715239 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f8c2c0199e0 con 0x7f8c3c106de0 2024-04-23T02:04:28.985 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.904+0000 7f8c327fc700 1 --2- 172.21.15.71:0/2620715239 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f8c28038380 0x7f8c2803a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:28.985 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.904+0000 7f8c327fc700 1 -- 172.21.15.71:0/2620715239 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8c2c04c6c0 con 0x7f8c3c106de0 2024-04-23T02:04:28.986 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.904+0000 7f8c40ae8700 1 --2- 172.21.15.71:0/2620715239 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f8c28038380 0x7f8c2803a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:28.986 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.904+0000 7f8c4354d700 1 -- 172.21.15.71:0/2620715239 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3c0623c0 con 0x7f8c3c106de0 2024-04-23T02:04:28.986 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.905+0000 7f8c40ae8700 1 --2- 172.21.15.71:0/2620715239 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f8c28038380 0x7f8c2803a840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8c38006fd0 tx=0x7f8c38006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:28.986 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:27.909+0000 7f8c327fc700 1 -- 172.21.15.71:0/2620715239 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8c2c02c3a0 con 0x7f8c3c106de0 2024-04-23T02:04:28.986 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:28.111+0000 7f8c4354d700 1 -- 172.21.15.71:0/2620715239 --> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f8c3c19d6d0 con 0x7f8c28038380 2024-04-23T02:04:28.986 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:28.335+0000 7f8c327fc700 1 -- 172.21.15.71:0/2620715239 <== mgr.14162 v2:172.21.15.71:6800/4165305842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f8c3c19d6d0 con 0x7f8c28038380 2024-04-23T02:04:28.986 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:28.337+0000 7f8c4354d700 1 -- 172.21.15.71:0/2620715239 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f8c28038380 msgr2=0x7f8c2803a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:28.986 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:28.337+0000 7f8c4354d700 1 --2- 172.21.15.71:0/2620715239 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f8c28038380 0x7f8c2803a840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8c38006fd0 tx=0x7f8c38006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:28.986 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:28.337+0000 7f8c4354d700 1 -- 172.21.15.71:0/2620715239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c3c106de0 msgr2=0x7f8c3c19a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:28.986 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:28.337+0000 7f8c4354d700 1 --2- 172.21.15.71:0/2620715239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c3c106de0 0x7f8c3c19a070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c006af0 tx=0x7f8c2c011b20 comp rx=0 tx=0).stop 2024-04-23T02:04:28.986 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:28.337+0000 7f8c4354d700 1 -- 172.21.15.71:0/2620715239 shutdown_connections 2024-04-23T02:04:28.986 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:28.337+0000 7f8c4354d700 1 --2- 172.21.15.71:0/2620715239 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f8c28038380 0x7f8c2803a840 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:28.986 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:28.337+0000 7f8c4354d700 1 --2- 172.21.15.71:0/2620715239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c3c106de0 0x7f8c3c19a070 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:28.987 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:28.337+0000 7f8c4354d700 1 -- 172.21.15.71:0/2620715239 >> 172.21.15.71:0/2620715239 conn(0x7f8c3c1007f0 msgr2=0x7f8c3c101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:28.987 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:28.337+0000 7f8c4354d700 1 -- 172.21.15.71:0/2620715239 shutdown_connections 2024-04-23T02:04:28.987 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:28.337+0000 7f8c4354d700 1 -- 172.21.15.71:0/2620715239 wait complete. 2024-04-23T02:04:28.987 INFO:teuthology.orchestra.run.smithi071.stdout:Fetching dashboard port number... 2024-04-23T02:04:29.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:29 smithi071 ceph-mon[39837]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:04:29.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:29 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:30.332 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 8443 2024-04-23T02:04:30.332 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.571+0000 7f8530636700 1 Processor -- start 2024-04-23T02:04:30.332 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.571+0000 7f8530636700 1 -- start start 2024-04-23T02:04:30.332 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.571+0000 7f8530636700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8528106e20 0x7f8528109210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:30.332 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.571+0000 7f8530636700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85280744b0 con 0x7f8528106e20 2024-04-23T02:04:30.332 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.572+0000 7f852e3d2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8528106e20 0x7f8528109210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:30.332 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.572+0000 7f852e3d2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8528106e20 0x7f8528109210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48730/0 (socket says 172.21.15.71:48730) 2024-04-23T02:04:30.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.572+0000 7f852e3d2700 1 -- 172.21.15.71:0/3474492473 learned_addr learned my addr 172.21.15.71:0/3474492473 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:30.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.572+0000 7f852e3d2700 1 -- 172.21.15.71:0/3474492473 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8528109750 con 0x7f8528106e20 2024-04-23T02:04:30.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.572+0000 7f852e3d2700 1 --2- 172.21.15.71:0/3474492473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8528106e20 0x7f8528109210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f85240098a0 tx=0x7f8524009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f48ba94a2814459 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:30.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.572+0000 7f852d3d0700 1 -- 172.21.15.71:0/3474492473 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8524004180 con 0x7f8528106e20 2024-04-23T02:04:30.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.572+0000 7f852d3d0700 1 -- 172.21.15.71:0/3474492473 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f85240042e0 con 0x7f8528106e20 2024-04-23T02:04:30.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.572+0000 7f852d3d0700 1 -- 172.21.15.71:0/3474492473 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8524011a30 con 0x7f8528106e20 2024-04-23T02:04:30.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.573+0000 7f8530636700 1 -- 172.21.15.71:0/3474492473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8528106e20 msgr2=0x7f8528109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:30.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.573+0000 7f8530636700 1 --2- 172.21.15.71:0/3474492473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8528106e20 0x7f8528109210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f85240098a0 tx=0x7f8524009bb0 comp rx=0 tx=0).stop 2024-04-23T02:04:30.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.573+0000 7f8530636700 1 -- 172.21.15.71:0/3474492473 shutdown_connections 2024-04-23T02:04:30.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.573+0000 7f8530636700 1 --2- 172.21.15.71:0/3474492473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8528106e20 0x7f8528109210 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:30.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.573+0000 7f8530636700 1 -- 172.21.15.71:0/3474492473 >> 172.21.15.71:0/3474492473 conn(0x7f85281007f0 msgr2=0x7f8528102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:30.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.573+0000 7f8530636700 1 -- 172.21.15.71:0/3474492473 shutdown_connections 2024-04-23T02:04:30.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.573+0000 7f8530636700 1 -- 172.21.15.71:0/3474492473 wait complete. 2024-04-23T02:04:30.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.574+0000 7f8530636700 1 Processor -- start 2024-04-23T02:04:30.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.574+0000 7f8530636700 1 -- start start 2024-04-23T02:04:30.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.574+0000 7f8530636700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8528106e20 0x7f852819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:30.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.574+0000 7f8530636700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852819a5a0 con 0x7f8528106e20 2024-04-23T02:04:30.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.574+0000 7f852e3d2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8528106e20 0x7f852819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:30.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.574+0000 7f852e3d2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8528106e20 0x7f852819a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48740/0 (socket says 172.21.15.71:48740) 2024-04-23T02:04:30.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.574+0000 7f852e3d2700 1 -- 172.21.15.71:0/2606452964 learned_addr learned my addr 172.21.15.71:0/2606452964 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:30.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.574+0000 7f852e3d2700 1 -- 172.21.15.71:0/2606452964 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8524009550 con 0x7f8528106e20 2024-04-23T02:04:30.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.574+0000 7f852e3d2700 1 --2- 172.21.15.71:0/2606452964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8528106e20 0x7f852819a060 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8524006af0 tx=0x7f8524011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:30.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.574+0000 7f851f7fe700 1 -- 172.21.15.71:0/2606452964 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8524003a80 con 0x7f8528106e20 2024-04-23T02:04:30.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.574+0000 7f851f7fe700 1 -- 172.21.15.71:0/2606452964 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f8524003be0 con 0x7f8528106e20 2024-04-23T02:04:30.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.574+0000 7f8530636700 1 -- 172.21.15.71:0/2606452964 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852819a7a0 con 0x7f8528106e20 2024-04-23T02:04:30.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.574+0000 7f851f7fe700 1 -- 172.21.15.71:0/2606452964 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8524019770 con 0x7f8528106e20 2024-04-23T02:04:30.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.574+0000 7f8530636700 1 -- 172.21.15.71:0/2606452964 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852819abc0 con 0x7f8528106e20 2024-04-23T02:04:30.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.575+0000 7f8530636700 1 -- 172.21.15.71:0/2606452964 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85280623c0 con 0x7f8528106e20 2024-04-23T02:04:30.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.576+0000 7f851f7fe700 1 -- 172.21.15.71:0/2606452964 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f85240199e0 con 0x7f8528106e20 2024-04-23T02:04:30.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.576+0000 7f851f7fe700 1 --2- 172.21.15.71:0/2606452964 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f8514038420 0x7f851403a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:30.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.576+0000 7f851f7fe700 1 -- 172.21.15.71:0/2606452964 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f852404cdd0 con 0x7f8528106e20 2024-04-23T02:04:30.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.576+0000 7f852dbd1700 1 --2- 172.21.15.71:0/2606452964 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f8514038420 0x7f851403a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:30.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.576+0000 7f852dbd1700 1 --2- 172.21.15.71:0/2606452964 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f8514038420 0x7f851403a8e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8518006fd0 tx=0x7f8518006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:30.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.580+0000 7f851f7fe700 1 -- 172.21.15.71:0/2606452964 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f852401acd0 con 0x7f8528106e20 2024-04-23T02:04:30.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.766+0000 7f8530636700 1 -- 172.21.15.71:0/2606452964 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f852819da40 con 0x7f8528106e20 2024-04-23T02:04:30.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.766+0000 7f851f7fe700 1 -- 172.21.15.71:0/2606452964 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f85240172b0 con 0x7f8528106e20 2024-04-23T02:04:30.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.768+0000 7f8530636700 1 -- 172.21.15.71:0/2606452964 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f8514038420 msgr2=0x7f851403a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:30.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.768+0000 7f8530636700 1 --2- 172.21.15.71:0/2606452964 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f8514038420 0x7f851403a8e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8518006fd0 tx=0x7f8518006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:30.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.768+0000 7f8530636700 1 -- 172.21.15.71:0/2606452964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8528106e20 msgr2=0x7f852819a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:30.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.768+0000 7f8530636700 1 --2- 172.21.15.71:0/2606452964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8528106e20 0x7f852819a060 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8524006af0 tx=0x7f8524011b20 comp rx=0 tx=0).stop 2024-04-23T02:04:30.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.768+0000 7f8530636700 1 -- 172.21.15.71:0/2606452964 shutdown_connections 2024-04-23T02:04:30.336 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.768+0000 7f8530636700 1 --2- 172.21.15.71:0/2606452964 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f8514038420 0x7f851403a8e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:30.336 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.768+0000 7f8530636700 1 --2- 172.21.15.71:0/2606452964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8528106e20 0x7f852819a060 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:30.336 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.768+0000 7f8530636700 1 -- 172.21.15.71:0/2606452964 >> 172.21.15.71:0/2606452964 conn(0x7f85281007f0 msgr2=0x7f8528101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:30.336 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.768+0000 7f8530636700 1 -- 172.21.15.71:0/2606452964 shutdown_connections 2024-04-23T02:04:30.336 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:29.769+0000 7f8530636700 1 -- 172.21.15.71:0/2606452964 wait complete. 2024-04-23T02:04:30.340 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T02:04:30.340 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-04-23T02:04:30.348 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T02:04:30.348 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-04-23T02:04:30.348 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-04-23T02:04:30.348 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-23T02:04:30.351 INFO:teuthology.orchestra.run.smithi071.stdout:Ceph Dashboard is now available at: 2024-04-23T02:04:30.351 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:04:30.351 INFO:teuthology.orchestra.run.smithi071.stdout: URL: https://smithi071.front.sepia.ceph.com:8443/ 2024-04-23T02:04:30.351 INFO:teuthology.orchestra.run.smithi071.stdout: User: admin 2024-04-23T02:04:30.351 INFO:teuthology.orchestra.run.smithi071.stdout: Password: m4kvcbg3uw 2024-04-23T02:04:30.351 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:04:30.351 INFO:teuthology.orchestra.run.smithi071.stdout:Saving cluster configuration to /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config directory 2024-04-23T02:04:30.352 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling autotune for osd_memory_target 2024-04-23T02:04:30.591 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:30 smithi071 ceph-mon[39837]: mgrmap e13: smithi071.ukdjvn(active, since 6s) 2024-04-23T02:04:30.591 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:30 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/2606452964' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-23T02:04:31.729 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.884+0000 7efecb645700 1 Processor -- start 2024-04-23T02:04:31.729 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.884+0000 7efecb645700 1 -- start start 2024-04-23T02:04:31.729 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.884+0000 7efecb645700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efec4106da0 0x7efec4109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:31.729 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.884+0000 7efecb645700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec4074530 con 0x7efec4106da0 2024-04-23T02:04:31.729 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.884+0000 7efec93e1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efec4106da0 0x7efec4109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:31.729 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.884+0000 7efec93e1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efec4106da0 0x7efec4109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48754/0 (socket says 172.21.15.71:48754) 2024-04-23T02:04:31.730 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.884+0000 7efec93e1700 1 -- 172.21.15.71:0/3788739031 learned_addr learned my addr 172.21.15.71:0/3788739031 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:31.730 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.885+0000 7efec93e1700 1 -- 172.21.15.71:0/3788739031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec41096d0 con 0x7efec4106da0 2024-04-23T02:04:31.730 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.885+0000 7efec93e1700 1 --2- 172.21.15.71:0/3788739031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efec4106da0 0x7efec4109190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efec00098a0 tx=0x7efec0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9197ed60d537a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:31.730 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.885+0000 7efebbfff700 1 -- 172.21.15.71:0/3788739031 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efec0004180 con 0x7efec4106da0 2024-04-23T02:04:31.730 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.885+0000 7efebbfff700 1 -- 172.21.15.71:0/3788739031 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7efec00042e0 con 0x7efec4106da0 2024-04-23T02:04:31.730 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.885+0000 7efebbfff700 1 -- 172.21.15.71:0/3788739031 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efec0011a30 con 0x7efec4106da0 2024-04-23T02:04:31.730 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.885+0000 7efecb645700 1 -- 172.21.15.71:0/3788739031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efec4106da0 msgr2=0x7efec4109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:31.730 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.885+0000 7efecb645700 1 --2- 172.21.15.71:0/3788739031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efec4106da0 0x7efec4109190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efec00098a0 tx=0x7efec0009bb0 comp rx=0 tx=0).stop 2024-04-23T02:04:31.730 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.886+0000 7efecb645700 1 -- 172.21.15.71:0/3788739031 shutdown_connections 2024-04-23T02:04:31.730 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.886+0000 7efecb645700 1 --2- 172.21.15.71:0/3788739031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efec4106da0 0x7efec4109190 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:31.730 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.886+0000 7efecb645700 1 -- 172.21.15.71:0/3788739031 >> 172.21.15.71:0/3788739031 conn(0x7efec41007f0 msgr2=0x7efec4102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:31.730 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.886+0000 7efecb645700 1 -- 172.21.15.71:0/3788739031 shutdown_connections 2024-04-23T02:04:31.730 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.886+0000 7efecb645700 1 -- 172.21.15.71:0/3788739031 wait complete. 2024-04-23T02:04:31.730 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.886+0000 7efecb645700 1 Processor -- start 2024-04-23T02:04:31.731 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.886+0000 7efecb645700 1 -- start start 2024-04-23T02:04:31.731 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.887+0000 7efecb645700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efec4106da0 0x7efec419a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:31.731 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.887+0000 7efecb645700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec419a5a0 con 0x7efec4106da0 2024-04-23T02:04:31.731 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.887+0000 7efec93e1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efec4106da0 0x7efec419a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:31.731 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.887+0000 7efec93e1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efec4106da0 0x7efec419a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48766/0 (socket says 172.21.15.71:48766) 2024-04-23T02:04:31.731 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.887+0000 7efec93e1700 1 -- 172.21.15.71:0/479376866 learned_addr learned my addr 172.21.15.71:0/479376866 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:31.731 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.887+0000 7efec93e1700 1 -- 172.21.15.71:0/479376866 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec0009550 con 0x7efec4106da0 2024-04-23T02:04:31.731 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.887+0000 7efec93e1700 1 --2- 172.21.15.71:0/479376866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efec4106da0 0x7efec419a060 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efec0006af0 tx=0x7efec0011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:31.731 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.887+0000 7efeba7fc700 1 -- 172.21.15.71:0/479376866 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efec0003a80 con 0x7efec4106da0 2024-04-23T02:04:31.731 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.887+0000 7efeba7fc700 1 -- 172.21.15.71:0/479376866 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7efec0003be0 con 0x7efec4106da0 2024-04-23T02:04:31.731 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.887+0000 7efecb645700 1 -- 172.21.15.71:0/479376866 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec419a7a0 con 0x7efec4106da0 2024-04-23T02:04:31.731 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.887+0000 7efeba7fc700 1 -- 172.21.15.71:0/479376866 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efec0019770 con 0x7efec4106da0 2024-04-23T02:04:31.731 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.887+0000 7efecb645700 1 -- 172.21.15.71:0/479376866 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec419abc0 con 0x7efec4106da0 2024-04-23T02:04:31.731 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.888+0000 7efeba7fc700 1 -- 172.21.15.71:0/479376866 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7efec0019a30 con 0x7efec4106da0 2024-04-23T02:04:31.732 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.888+0000 7efecb645700 1 -- 172.21.15.71:0/479376866 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec40623c0 con 0x7efec4106da0 2024-04-23T02:04:31.732 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.888+0000 7efeba7fc700 1 --2- 172.21.15.71:0/479376866 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7efeb00383d0 0x7efeb003a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:31.732 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.888+0000 7efeba7fc700 1 -- 172.21.15.71:0/479376866 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efec004c770 con 0x7efec4106da0 2024-04-23T02:04:31.732 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.889+0000 7efec8be0700 1 --2- 172.21.15.71:0/479376866 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7efeb00383d0 0x7efeb003a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:31.732 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.889+0000 7efec8be0700 1 --2- 172.21.15.71:0/479376866 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7efeb00383d0 0x7efeb003a890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efeb4006fd0 tx=0x7efeb4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:31.732 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:30.893+0000 7efeba7fc700 1 -- 172.21.15.71:0/479376866 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7efec002b750 con 0x7efec4106da0 2024-04-23T02:04:31.732 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:31.081+0000 7efecb645700 1 -- 172.21.15.71:0/479376866 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7efec419dd30 con 0x7efec4106da0 2024-04-23T02:04:31.732 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:31.082+0000 7efeba7fc700 1 -- 172.21.15.71:0/479376866 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7efec00172b0 con 0x7efec4106da0 2024-04-23T02:04:31.732 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:31.083+0000 7efecb645700 1 -- 172.21.15.71:0/479376866 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7efeb00383d0 msgr2=0x7efeb003a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:31.732 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:31.083+0000 7efecb645700 1 --2- 172.21.15.71:0/479376866 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7efeb00383d0 0x7efeb003a890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efeb4006fd0 tx=0x7efeb4006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:31.732 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:31.083+0000 7efecb645700 1 -- 172.21.15.71:0/479376866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efec4106da0 msgr2=0x7efec419a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:31.732 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:31.083+0000 7efecb645700 1 --2- 172.21.15.71:0/479376866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efec4106da0 0x7efec419a060 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efec0006af0 tx=0x7efec0011b20 comp rx=0 tx=0).stop 2024-04-23T02:04:31.732 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:31.084+0000 7efecb645700 1 -- 172.21.15.71:0/479376866 shutdown_connections 2024-04-23T02:04:31.732 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:31.084+0000 7efecb645700 1 --2- 172.21.15.71:0/479376866 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7efeb00383d0 0x7efeb003a890 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:31.733 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:31.084+0000 7efecb645700 1 --2- 172.21.15.71:0/479376866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efec4106da0 0x7efec419a060 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:31.733 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:31.084+0000 7efecb645700 1 -- 172.21.15.71:0/479376866 >> 172.21.15.71:0/479376866 conn(0x7efec41007f0 msgr2=0x7efec4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:31.733 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:31.084+0000 7efecb645700 1 -- 172.21.15.71:0/479376866 shutdown_connections 2024-04-23T02:04:31.733 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:31.084+0000 7efecb645700 1 -- 172.21.15.71:0/479376866 wait complete. 2024-04-23T02:04:33.196 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.261+0000 7f97f72b2700 1 Processor -- start 2024-04-23T02:04:33.196 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.262+0000 7f97f72b2700 1 -- start start 2024-04-23T02:04:33.196 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.262+0000 7f97f72b2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f0104bd0 0x7f97f0104ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:33.196 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.262+0000 7f97f72b2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f00744f0 con 0x7f97f0104bd0 2024-04-23T02:04:33.196 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.262+0000 7f97f504e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f0104bd0 0x7f97f0104ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:33.196 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.262+0000 7f97f504e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f0104bd0 0x7f97f0104ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48780/0 (socket says 172.21.15.71:48780) 2024-04-23T02:04:33.196 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.262+0000 7f97f504e700 1 -- 172.21.15.71:0/1685497615 learned_addr learned my addr 172.21.15.71:0/1685497615 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:33.196 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.262+0000 7f97f504e700 1 -- 172.21.15.71:0/1685497615 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f0105530 con 0x7f97f0104bd0 2024-04-23T02:04:33.196 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.262+0000 7f97f504e700 1 --2- 172.21.15.71:0/1685497615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f0104bd0 0x7f97f0104ff0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0098a0 tx=0x7f97ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499edfe99484d70 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:33.197 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.262+0000 7f97e7fff700 1 -- 172.21.15.71:0/1685497615 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97ec004180 con 0x7f97f0104bd0 2024-04-23T02:04:33.197 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.262+0000 7f97e7fff700 1 -- 172.21.15.71:0/1685497615 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f97ec0042e0 con 0x7f97f0104bd0 2024-04-23T02:04:33.197 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.262+0000 7f97e7fff700 1 -- 172.21.15.71:0/1685497615 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97ec011a30 con 0x7f97f0104bd0 2024-04-23T02:04:33.197 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.263+0000 7f97f72b2700 1 -- 172.21.15.71:0/1685497615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f0104bd0 msgr2=0x7f97f0104ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:33.197 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.263+0000 7f97f72b2700 1 --2- 172.21.15.71:0/1685497615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f0104bd0 0x7f97f0104ff0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0098a0 tx=0x7f97ec009bb0 comp rx=0 tx=0).stop 2024-04-23T02:04:33.197 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.263+0000 7f97f72b2700 1 -- 172.21.15.71:0/1685497615 shutdown_connections 2024-04-23T02:04:33.197 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.263+0000 7f97f72b2700 1 --2- 172.21.15.71:0/1685497615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f0104bd0 0x7f97f0104ff0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:33.197 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.263+0000 7f97f72b2700 1 -- 172.21.15.71:0/1685497615 >> 172.21.15.71:0/1685497615 conn(0x7f97f01007f0 msgr2=0x7f97f0102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:33.197 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.263+0000 7f97f72b2700 1 -- 172.21.15.71:0/1685497615 shutdown_connections 2024-04-23T02:04:33.197 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.263+0000 7f97f72b2700 1 -- 172.21.15.71:0/1685497615 wait complete. 2024-04-23T02:04:33.197 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.264+0000 7f97f72b2700 1 Processor -- start 2024-04-23T02:04:33.197 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.264+0000 7f97f72b2700 1 -- start start 2024-04-23T02:04:33.197 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.264+0000 7f97f72b2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f0104bd0 0x7f97f019c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:33.197 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.264+0000 7f97f72b2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f019c7a0 con 0x7f97f0104bd0 2024-04-23T02:04:33.197 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.264+0000 7f97f504e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f0104bd0 0x7f97f019c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:33.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.264+0000 7f97f504e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f0104bd0 0x7f97f019c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48792/0 (socket says 172.21.15.71:48792) 2024-04-23T02:04:33.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.264+0000 7f97f504e700 1 -- 172.21.15.71:0/1180275631 learned_addr learned my addr 172.21.15.71:0/1180275631 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:33.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.264+0000 7f97f504e700 1 -- 172.21.15.71:0/1180275631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec009550 con 0x7f97f0104bd0 2024-04-23T02:04:33.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.264+0000 7f97f504e700 1 --2- 172.21.15.71:0/1180275631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f0104bd0 0x7f97f019c260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f97ec011a00 tx=0x7f97ec011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:33.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.264+0000 7f97e67fc700 1 -- 172.21.15.71:0/1180275631 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97ec003bd0 con 0x7f97f0104bd0 2024-04-23T02:04:33.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.265+0000 7f97e67fc700 1 -- 172.21.15.71:0/1180275631 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f97ec0248d0 con 0x7f97f0104bd0 2024-04-23T02:04:33.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.265+0000 7f97f72b2700 1 -- 172.21.15.71:0/1180275631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f019c9a0 con 0x7f97f0104bd0 2024-04-23T02:04:33.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.265+0000 7f97e67fc700 1 -- 172.21.15.71:0/1180275631 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97ec01a7e0 con 0x7f97f0104bd0 2024-04-23T02:04:33.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.265+0000 7f97f72b2700 1 -- 172.21.15.71:0/1180275631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f019cdc0 con 0x7f97f0104bd0 2024-04-23T02:04:33.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.265+0000 7f97e67fc700 1 -- 172.21.15.71:0/1180275631 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f97ec01aa50 con 0x7f97f0104bd0 2024-04-23T02:04:33.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.265+0000 7f97f72b2700 1 -- 172.21.15.71:0/1180275631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f0195b40 con 0x7f97f0104bd0 2024-04-23T02:04:33.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.266+0000 7f97e67fc700 1 --2- 172.21.15.71:0/1180275631 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f97dc038410 0x7f97dc03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:33.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.266+0000 7f97e67fc700 1 -- 172.21.15.71:0/1180275631 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f97ec04c730 con 0x7f97f0104bd0 2024-04-23T02:04:33.199 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.266+0000 7f97f484d700 1 --2- 172.21.15.71:0/1180275631 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f97dc038410 0x7f97dc03a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:33.199 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.266+0000 7f97f484d700 1 --2- 172.21.15.71:0/1180275631 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f97dc038410 0x7f97dc03a8d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f97e0006fd0 tx=0x7f97e0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:33.199 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.270+0000 7f97e67fc700 1 -- 172.21.15.71:0/1180275631 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f97ec0233a0 con 0x7f97f0104bd0 2024-04-23T02:04:33.199 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.565+0000 7f97f72b2700 1 -- 172.21.15.71:0/1180275631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f97f002cf20 con 0x7f97f0104bd0 2024-04-23T02:04:33.199 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.568+0000 7f97e67fc700 1 -- 172.21.15.71:0/1180275631 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f97ec02c350 con 0x7f97f0104bd0 2024-04-23T02:04:33.200 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.570+0000 7f97f72b2700 1 -- 172.21.15.71:0/1180275631 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f97dc038410 msgr2=0x7f97dc03a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:33.200 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.570+0000 7f97f72b2700 1 --2- 172.21.15.71:0/1180275631 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f97dc038410 0x7f97dc03a8d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f97e0006fd0 tx=0x7f97e0006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:33.200 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.570+0000 7f97f72b2700 1 -- 172.21.15.71:0/1180275631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f0104bd0 msgr2=0x7f97f019c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:33.200 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.570+0000 7f97f72b2700 1 --2- 172.21.15.71:0/1180275631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f0104bd0 0x7f97f019c260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f97ec011a00 tx=0x7f97ec011ae0 comp rx=0 tx=0).stop 2024-04-23T02:04:33.200 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.570+0000 7f97f72b2700 1 -- 172.21.15.71:0/1180275631 shutdown_connections 2024-04-23T02:04:33.200 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.570+0000 7f97f72b2700 1 --2- 172.21.15.71:0/1180275631 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f97dc038410 0x7f97dc03a8d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:33.200 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.570+0000 7f97f72b2700 1 --2- 172.21.15.71:0/1180275631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97f0104bd0 0x7f97f019c260 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:33.200 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.570+0000 7f97f72b2700 1 -- 172.21.15.71:0/1180275631 >> 172.21.15.71:0/1180275631 conn(0x7f97f01007f0 msgr2=0x7f97f0190790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:33.200 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.570+0000 7f97f72b2700 1 -- 172.21.15.71:0/1180275631 shutdown_connections 2024-04-23T02:04:33.200 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-04-23T02:04:32.570+0000 7f97f72b2700 1 -- 172.21.15.71:0/1180275631 wait complete. 2024-04-23T02:04:33.200 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-23T02:04:33.200 INFO:teuthology.orchestra.run.smithi071.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-23T02:04:33.200 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:04:33.200 INFO:teuthology.orchestra.run.smithi071.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-23T02:04:33.201 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:04:33.201 INFO:teuthology.orchestra.run.smithi071.stdout:Or, if you are only running a single cluster on this host: 2024-04-23T02:04:33.201 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:04:33.201 INFO:teuthology.orchestra.run.smithi071.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-23T02:04:33.201 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:04:33.201 INFO:teuthology.orchestra.run.smithi071.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-23T02:04:33.201 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:04:33.201 INFO:teuthology.orchestra.run.smithi071.stdout: ceph telemetry on 2024-04-23T02:04:33.201 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:04:33.201 INFO:teuthology.orchestra.run.smithi071.stdout:For more information see: 2024-04-23T02:04:33.201 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:04:33.201 INFO:teuthology.orchestra.run.smithi071.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-23T02:04:33.201 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:04:33.201 INFO:teuthology.orchestra.run.smithi071.stdout:Bootstrap complete. 2024-04-23T02:04:33.230 INFO:tasks.cephadm:Fetching config... 2024-04-23T02:04:33.230 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T02:04:33.230 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-23T02:04:33.289 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-23T02:04:33.289 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T02:04:33.289 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-23T02:04:33.346 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-23T02:04:33.346 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T02:04:33.346 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/keyring of=/dev/stdout 2024-04-23T02:04:33.414 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-23T02:04:33.414 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T02:04:33.414 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-23T02:04:33.471 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-23T02:04:33.472 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDqtHVXyikLrcynO96rql0xXsL3+bERjrlT6vpKaaGRzuoK/HR+EVcDbQfueqc5mIAv25y6TBCAFkCmNo1Ck+MeMit1N1miKYH2RTpqf4tdI6e1EmbwaMow7Eiqd3tHefwDDAqlho7LmlwhzRRTL3Pc5CvGo2scsyis9Gorn3BMSPrI7ljsPjs1BXEExh6lCNCUD+Y3yPgtK5JhYKlQz0yYWPRwfTlTPzB0SrMj57ex2frtjB4xyJncgH3K30FWD1Xv7/mhtUdRfBof8j+mJi6uo866aZ32IQ0Gbh4qjVzEe8skhFizbOi0XFsoUXMlSipeyt7P+aVF3X+wqWljuAC7uio0Y+oXa20U/B4MyiN9scI7obD3ssj66D9TmDWxgLbGQuhKSdSaO1Rvk88C1Izmwc0K+nEuNi/sJdHJeIu8yfqdoppW5qLeyMdBIZUBlUfzveCt6MJ/FSZw5/2UwmJQI4ZIxEo+0OAwU0I816vYF8NolGZIZ+D00dbgaFJlOs= ceph-91f5470c-0115-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T02:04:33.550 INFO:teuthology.orchestra.run.smithi071.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDqtHVXyikLrcynO96rql0xXsL3+bERjrlT6vpKaaGRzuoK/HR+EVcDbQfueqc5mIAv25y6TBCAFkCmNo1Ck+MeMit1N1miKYH2RTpqf4tdI6e1EmbwaMow7Eiqd3tHefwDDAqlho7LmlwhzRRTL3Pc5CvGo2scsyis9Gorn3BMSPrI7ljsPjs1BXEExh6lCNCUD+Y3yPgtK5JhYKlQz0yYWPRwfTlTPzB0SrMj57ex2frtjB4xyJncgH3K30FWD1Xv7/mhtUdRfBof8j+mJi6uo866aZ32IQ0Gbh4qjVzEe8skhFizbOi0XFsoUXMlSipeyt7P+aVF3X+wqWljuAC7uio0Y+oXa20U/B4MyiN9scI7obD3ssj66D9TmDWxgLbGQuhKSdSaO1Rvk88C1Izmwc0K+nEuNi/sJdHJeIu8yfqdoppW5qLeyMdBIZUBlUfzveCt6MJ/FSZw5/2UwmJQI4ZIxEo+0OAwU0I816vYF8NolGZIZ+D00dbgaFJlOs= ceph-91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T02:04:33.561 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDqtHVXyikLrcynO96rql0xXsL3+bERjrlT6vpKaaGRzuoK/HR+EVcDbQfueqc5mIAv25y6TBCAFkCmNo1Ck+MeMit1N1miKYH2RTpqf4tdI6e1EmbwaMow7Eiqd3tHefwDDAqlho7LmlwhzRRTL3Pc5CvGo2scsyis9Gorn3BMSPrI7ljsPjs1BXEExh6lCNCUD+Y3yPgtK5JhYKlQz0yYWPRwfTlTPzB0SrMj57ex2frtjB4xyJncgH3K30FWD1Xv7/mhtUdRfBof8j+mJi6uo866aZ32IQ0Gbh4qjVzEe8skhFizbOi0XFsoUXMlSipeyt7P+aVF3X+wqWljuAC7uio0Y+oXa20U/B4MyiN9scI7obD3ssj66D9TmDWxgLbGQuhKSdSaO1Rvk88C1Izmwc0K+nEuNi/sJdHJeIu8yfqdoppW5qLeyMdBIZUBlUfzveCt6MJ/FSZw5/2UwmJQI4ZIxEo+0OAwU0I816vYF8NolGZIZ+D00dbgaFJlOs= ceph-91f5470c-0115-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T02:04:33.607 INFO:teuthology.orchestra.run.smithi192.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDqtHVXyikLrcynO96rql0xXsL3+bERjrlT6vpKaaGRzuoK/HR+EVcDbQfueqc5mIAv25y6TBCAFkCmNo1Ck+MeMit1N1miKYH2RTpqf4tdI6e1EmbwaMow7Eiqd3tHefwDDAqlho7LmlwhzRRTL3Pc5CvGo2scsyis9Gorn3BMSPrI7ljsPjs1BXEExh6lCNCUD+Y3yPgtK5JhYKlQz0yYWPRwfTlTPzB0SrMj57ex2frtjB4xyJncgH3K30FWD1Xv7/mhtUdRfBof8j+mJi6uo866aZ32IQ0Gbh4qjVzEe8skhFizbOi0XFsoUXMlSipeyt7P+aVF3X+wqWljuAC7uio0Y+oXa20U/B4MyiN9scI7obD3ssj66D9TmDWxgLbGQuhKSdSaO1Rvk88C1Izmwc0K+nEuNi/sJdHJeIu8yfqdoppW5qLeyMdBIZUBlUfzveCt6MJ/FSZw5/2UwmJQI4ZIxEo+0OAwU0I816vYF8NolGZIZ+D00dbgaFJlOs= ceph-91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T02:04:33.618 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-23T02:04:33.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:33 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1180275631' entity='client.admin' 2024-04-23T02:04:33.838 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:04:36.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.222+0000 7fce71825700 1 -- 172.21.15.71:0/1912951987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce6c0feb50 msgr2=0x7fce6c0fef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:36.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.222+0000 7fce71825700 1 --2- 172.21.15.71:0/1912951987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce6c0feb50 0x7fce6c0fef70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fce60009910 tx=0x7fce60009c20 comp rx=0 tx=0).stop 2024-04-23T02:04:36.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.222+0000 7fce71825700 1 -- 172.21.15.71:0/1912951987 shutdown_connections 2024-04-23T02:04:36.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.222+0000 7fce71825700 1 --2- 172.21.15.71:0/1912951987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce6c0feb50 0x7fce6c0fef70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:36.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.222+0000 7fce71825700 1 -- 172.21.15.71:0/1912951987 >> 172.21.15.71:0/1912951987 conn(0x7fce6c0fa4b0 msgr2=0x7fce6c0fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:36.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.222+0000 7fce71825700 1 -- 172.21.15.71:0/1912951987 shutdown_connections 2024-04-23T02:04:36.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.222+0000 7fce71825700 1 -- 172.21.15.71:0/1912951987 wait complete. 2024-04-23T02:04:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.223+0000 7fce71825700 1 Processor -- start 2024-04-23T02:04:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.223+0000 7fce71825700 1 -- start start 2024-04-23T02:04:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.223+0000 7fce71825700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce6c0feb50 0x7fce6c1958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.223+0000 7fce71825700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c195e20 con 0x7fce6c0feb50 2024-04-23T02:04:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.223+0000 7fce6affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce6c0feb50 0x7fce6c1958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.223+0000 7fce6affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce6c0feb50 0x7fce6c1958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33952/0 (socket says 172.21.15.71:33952) 2024-04-23T02:04:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.223+0000 7fce6affd700 1 -- 172.21.15.71:0/2510634883 learned_addr learned my addr 172.21.15.71:0/2510634883 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.224+0000 7fce6affd700 1 -- 172.21.15.71:0/2510634883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce600095f0 con 0x7fce6c0feb50 2024-04-23T02:04:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.224+0000 7fce6affd700 1 --2- 172.21.15.71:0/2510634883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce6c0feb50 0x7fce6c1958e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fce60006010 tx=0x7fce600057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.224+0000 7fce70823700 1 -- 172.21.15.71:0/2510634883 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce6001d070 con 0x7fce6c0feb50 2024-04-23T02:04:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.224+0000 7fce71825700 1 -- 172.21.15.71:0/2510634883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6c196020 con 0x7fce6c0feb50 2024-04-23T02:04:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.224+0000 7fce70823700 1 -- 172.21.15.71:0/2510634883 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fce6000bdf0 con 0x7fce6c0feb50 2024-04-23T02:04:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.224+0000 7fce70823700 1 -- 172.21.15.71:0/2510634883 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce60022e50 con 0x7fce6c0feb50 2024-04-23T02:04:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.224+0000 7fce71825700 1 -- 172.21.15.71:0/2510634883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6c1964c0 con 0x7fce6c0feb50 2024-04-23T02:04:36.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.225+0000 7fce70823700 1 -- 172.21.15.71:0/2510634883 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fce6000f4b0 con 0x7fce6c0feb50 2024-04-23T02:04:36.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.225+0000 7fce71825700 1 -- 172.21.15.71:0/2510634883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6c18f380 con 0x7fce6c0feb50 2024-04-23T02:04:36.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.225+0000 7fce70823700 1 --2- 172.21.15.71:0/2510634883 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fce54038370 0x7fce5403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:36.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.225+0000 7fce70823700 1 -- 172.21.15.71:0/2510634883 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fce6004c300 con 0x7fce6c0feb50 2024-04-23T02:04:36.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.225+0000 7fce6a7fc700 1 --2- 172.21.15.71:0/2510634883 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fce54038370 0x7fce5403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:36.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.225+0000 7fce6a7fc700 1 --2- 172.21.15.71:0/2510634883 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fce54038370 0x7fce5403a830 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fce5c006fd0 tx=0x7fce5c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:36.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.230+0000 7fce70823700 1 -- 172.21.15.71:0/2510634883 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fce6002a770 con 0x7fce6c0feb50 2024-04-23T02:04:36.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.419+0000 7fce71825700 1 -- 172.21.15.71:0/2510634883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fce6c0ff680 con 0x7fce6c0feb50 2024-04-23T02:04:36.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.422+0000 7fce70823700 1 -- 172.21.15.71:0/2510634883 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fce6c0ff680 con 0x7fce6c0feb50 2024-04-23T02:04:36.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.425+0000 7fce71825700 1 -- 172.21.15.71:0/2510634883 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fce54038370 msgr2=0x7fce5403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:36.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.425+0000 7fce71825700 1 --2- 172.21.15.71:0/2510634883 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fce54038370 0x7fce5403a830 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fce5c006fd0 tx=0x7fce5c006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:36.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.425+0000 7fce71825700 1 -- 172.21.15.71:0/2510634883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce6c0feb50 msgr2=0x7fce6c1958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:36.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.425+0000 7fce71825700 1 --2- 172.21.15.71:0/2510634883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce6c0feb50 0x7fce6c1958e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fce60006010 tx=0x7fce600057b0 comp rx=0 tx=0).stop 2024-04-23T02:04:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.427+0000 7fce71825700 1 -- 172.21.15.71:0/2510634883 shutdown_connections 2024-04-23T02:04:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.427+0000 7fce71825700 1 --2- 172.21.15.71:0/2510634883 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fce54038370 0x7fce5403a830 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.427+0000 7fce71825700 1 --2- 172.21.15.71:0/2510634883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce6c0feb50 0x7fce6c1958e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.427+0000 7fce71825700 1 -- 172.21.15.71:0/2510634883 >> 172.21.15.71:0/2510634883 conn(0x7fce6c0fa4b0 msgr2=0x7fce6c0fb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.427+0000 7fce71825700 1 -- 172.21.15.71:0/2510634883 shutdown_connections 2024-04-23T02:04:36.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:36.427+0000 7fce71825700 1 -- 172.21.15.71:0/2510634883 wait complete. 2024-04-23T02:04:37.323 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-23T02:04:37.323 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-23T02:04:37.554 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:04:37.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:37 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/2510634883' entity='client.admin' 2024-04-23T02:04:39.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.865+0000 7f816920c700 1 -- 172.21.15.71:0/4202106753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8164100da0 msgr2=0x7f81641011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:39.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.865+0000 7f816920c700 1 --2- 172.21.15.71:0/4202106753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8164100da0 0x7f81641011c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8154009910 tx=0x7f8154009c20 comp rx=0 tx=0).stop 2024-04-23T02:04:39.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.865+0000 7f816920c700 1 -- 172.21.15.71:0/4202106753 shutdown_connections 2024-04-23T02:04:39.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.865+0000 7f816920c700 1 --2- 172.21.15.71:0/4202106753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8164100da0 0x7f81641011c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:39.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.865+0000 7f816920c700 1 -- 172.21.15.71:0/4202106753 >> 172.21.15.71:0/4202106753 conn(0x7f81640fa490 msgr2=0x7f81640fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:39.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.865+0000 7f816920c700 1 -- 172.21.15.71:0/4202106753 shutdown_connections 2024-04-23T02:04:39.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.865+0000 7f816920c700 1 -- 172.21.15.71:0/4202106753 wait complete. 2024-04-23T02:04:39.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.866+0000 7f816920c700 1 Processor -- start 2024-04-23T02:04:39.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.866+0000 7f816920c700 1 -- start start 2024-04-23T02:04:39.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.866+0000 7f816920c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8164100da0 0x7f81640ff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:39.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.866+0000 7f816920c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81641985e0 con 0x7f8164100da0 2024-04-23T02:04:39.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.866+0000 7f8162d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8164100da0 0x7f81640ff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:39.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.866+0000 7f8162d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8164100da0 0x7f81640ff4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33968/0 (socket says 172.21.15.71:33968) 2024-04-23T02:04:39.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.866+0000 7f8162d9d700 1 -- 172.21.15.71:0/1999792704 learned_addr learned my addr 172.21.15.71:0/1999792704 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:39.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.867+0000 7f8162d9d700 1 -- 172.21.15.71:0/1999792704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81540095f0 con 0x7f8164100da0 2024-04-23T02:04:39.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.867+0000 7f8162d9d700 1 --2- 172.21.15.71:0/1999792704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8164100da0 0x7f81640ff4b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8154006010 tx=0x7f81540057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:39.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.867+0000 7f814bfff700 1 -- 172.21.15.71:0/1999792704 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f815401d070 con 0x7f8164100da0 2024-04-23T02:04:39.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.867+0000 7f816920c700 1 -- 172.21.15.71:0/1999792704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8164198720 con 0x7f8164100da0 2024-04-23T02:04:39.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.867+0000 7f814bfff700 1 -- 172.21.15.71:0/1999792704 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f815400bdf0 con 0x7f8164100da0 2024-04-23T02:04:39.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.867+0000 7f814bfff700 1 -- 172.21.15.71:0/1999792704 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8154022e50 con 0x7f8164100da0 2024-04-23T02:04:39.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.867+0000 7f816920c700 1 -- 172.21.15.71:0/1999792704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8164198bc0 con 0x7f8164100da0 2024-04-23T02:04:39.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.868+0000 7f816920c700 1 -- 172.21.15.71:0/1999792704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f816418e280 con 0x7f8164100da0 2024-04-23T02:04:39.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.868+0000 7f814bfff700 1 -- 172.21.15.71:0/1999792704 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f815400b960 con 0x7f8164100da0 2024-04-23T02:04:39.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.868+0000 7f814bfff700 1 --2- 172.21.15.71:0/1999792704 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f814c038370 0x7f814c03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:39.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.869+0000 7f814bfff700 1 -- 172.21.15.71:0/1999792704 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f815404c3d0 con 0x7f8164100da0 2024-04-23T02:04:39.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.869+0000 7f816259c700 1 --2- 172.21.15.71:0/1999792704 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f814c038370 0x7f814c03a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:39.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.869+0000 7f816259c700 1 --2- 172.21.15.71:0/1999792704 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f814c038370 0x7f814c03a830 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8158006fd0 tx=0x7f8158006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:39.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:39.875+0000 7f814bfff700 1 -- 172.21.15.71:0/1999792704 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8154022700 con 0x7f8164100da0 2024-04-23T02:04:40.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:40.072+0000 7f816920c700 1 -- 172.21.15.71:0/1999792704 --> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f8164061260 con 0x7f814c038370 2024-04-23T02:04:40.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:40.075+0000 7f814bfff700 1 -- 172.21.15.71:0/1999792704 <== mgr.14162 v2:172.21.15.71:6800/4165305842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8164061260 con 0x7f814c038370 2024-04-23T02:04:40.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:40.077+0000 7f816920c700 1 -- 172.21.15.71:0/1999792704 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f814c038370 msgr2=0x7f814c03a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:40.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:40.077+0000 7f816920c700 1 --2- 172.21.15.71:0/1999792704 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f814c038370 0x7f814c03a830 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8158006fd0 tx=0x7f8158006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:40.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:40.077+0000 7f816920c700 1 -- 172.21.15.71:0/1999792704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8164100da0 msgr2=0x7f81640ff4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:40.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:40.077+0000 7f816920c700 1 --2- 172.21.15.71:0/1999792704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8164100da0 0x7f81640ff4b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8154006010 tx=0x7f81540057b0 comp rx=0 tx=0).stop 2024-04-23T02:04:40.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:40.077+0000 7f816920c700 1 -- 172.21.15.71:0/1999792704 shutdown_connections 2024-04-23T02:04:40.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:40.077+0000 7f816920c700 1 --2- 172.21.15.71:0/1999792704 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f814c038370 0x7f814c03a830 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:40.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:40.077+0000 7f816920c700 1 --2- 172.21.15.71:0/1999792704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8164100da0 0x7f81640ff4b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:40.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:40.077+0000 7f816920c700 1 -- 172.21.15.71:0/1999792704 >> 172.21.15.71:0/1999792704 conn(0x7f81640fa490 msgr2=0x7f81640fb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:40.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:40.077+0000 7f816920c700 1 -- 172.21.15.71:0/1999792704 shutdown_connections 2024-04-23T02:04:40.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:40.078+0000 7f816920c700 1 -- 172.21.15.71:0/1999792704 wait complete. 2024-04-23T02:04:40.447 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:40 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:40.447 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:40 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:40.447 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:40 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:04:40.447 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:40 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:40.447 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:40 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi071", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-23T02:04:40.447 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:40 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi071", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-23T02:04:40.447 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:40 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:04:40.447 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:40 smithi071 ceph-mon[39837]: Deploying daemon ceph-exporter.smithi071 on smithi071 2024-04-23T02:04:40.447 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:40 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:41.633 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:41 smithi071 ceph-mon[39837]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:04:41.874 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi192 2024-04-23T02:04:41.874 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-23T02:04:41.874 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.conf 2024-04-23T02:04:41.896 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-23T02:04:41.896 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T02:04:41.957 INFO:tasks.cephadm:Adding host smithi192 to orchestrator... 2024-04-23T02:04:41.957 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph orch host add smithi192 2024-04-23T02:04:42.544 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:04:43.715 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:43 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:43.716 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:43 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:43.716 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:43 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:43.716 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:43 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:43.716 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:43 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi071", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-23T02:04:43.716 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:43 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi071", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-23T02:04:43.716 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:43 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:04:43.716 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:43 smithi071 ceph-mon[39837]: Deploying daemon crash.smithi071 on smithi071 2024-04-23T02:04:43.716 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:43 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:44.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.809+0000 7f8fa7851700 1 -- 172.21.15.71:0/3340488283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fa00ff780 msgr2=0x7f8fa00ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:44.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.809+0000 7f8fa7851700 1 --2- 172.21.15.71:0/3340488283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fa00ff780 0x7f8fa00ffba0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009910 tx=0x7f8f90009c20 comp rx=0 tx=0).stop 2024-04-23T02:04:44.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.809+0000 7f8fa7851700 1 -- 172.21.15.71:0/3340488283 shutdown_connections 2024-04-23T02:04:44.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.809+0000 7f8fa7851700 1 --2- 172.21.15.71:0/3340488283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fa00ff780 0x7f8fa00ffba0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:44.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.809+0000 7f8fa7851700 1 -- 172.21.15.71:0/3340488283 >> 172.21.15.71:0/3340488283 conn(0x7f8fa00fad00 msgr2=0x7f8fa00fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:44.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.809+0000 7f8fa7851700 1 -- 172.21.15.71:0/3340488283 shutdown_connections 2024-04-23T02:04:44.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.809+0000 7f8fa7851700 1 -- 172.21.15.71:0/3340488283 wait complete. 2024-04-23T02:04:44.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.810+0000 7f8fa7851700 1 Processor -- start 2024-04-23T02:04:44.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.810+0000 7f8fa7851700 1 -- start start 2024-04-23T02:04:44.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.810+0000 7f8fa7851700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fa00ff780 0x7f8fa0069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:44.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.810+0000 7f8fa7851700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa0198610 con 0x7f8fa00ff780 2024-04-23T02:04:44.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.810+0000 7f8fa55ed700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fa00ff780 0x7f8fa0069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:44.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.810+0000 7f8fa55ed700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fa00ff780 0x7f8fa0069380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33992/0 (socket says 172.21.15.71:33992) 2024-04-23T02:04:44.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.810+0000 7f8fa55ed700 1 -- 172.21.15.71:0/2264616546 learned_addr learned my addr 172.21.15.71:0/2264616546 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:44.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.810+0000 7f8fa55ed700 1 -- 172.21.15.71:0/2264616546 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f900095f0 con 0x7f8fa00ff780 2024-04-23T02:04:44.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.811+0000 7f8fa55ed700 1 --2- 172.21.15.71:0/2264616546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fa00ff780 0x7f8fa0069380 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8f90006010 tx=0x7f8f900057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:44.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.811+0000 7f8f967fc700 1 -- 172.21.15.71:0/2264616546 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f9001d070 con 0x7f8fa00ff780 2024-04-23T02:04:44.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.811+0000 7f8fa7851700 1 -- 172.21.15.71:0/2264616546 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa0198750 con 0x7f8fa00ff780 2024-04-23T02:04:44.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.811+0000 7f8f967fc700 1 -- 172.21.15.71:0/2264616546 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f8f9000bdf0 con 0x7f8fa00ff780 2024-04-23T02:04:44.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.811+0000 7f8f967fc700 1 -- 172.21.15.71:0/2264616546 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f90022e50 con 0x7f8fa00ff780 2024-04-23T02:04:44.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.811+0000 7f8fa7851700 1 -- 172.21.15.71:0/2264616546 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa0198bf0 con 0x7f8fa00ff780 2024-04-23T02:04:44.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.812+0000 7f8fa7851700 1 -- 172.21.15.71:0/2264616546 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa018e260 con 0x7f8fa00ff780 2024-04-23T02:04:44.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.812+0000 7f8f967fc700 1 -- 172.21.15.71:0/2264616546 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f8f90022700 con 0x7f8fa00ff780 2024-04-23T02:04:44.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.812+0000 7f8f967fc700 1 --2- 172.21.15.71:0/2264616546 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f8f8c038370 0x7f8f8c03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:44.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.812+0000 7f8f967fc700 1 -- 172.21.15.71:0/2264616546 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8f9004c350 con 0x7f8fa00ff780 2024-04-23T02:04:44.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.812+0000 7f8fa4dec700 1 --2- 172.21.15.71:0/2264616546 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f8f8c038370 0x7f8f8c03a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:44.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.812+0000 7f8fa4dec700 1 --2- 172.21.15.71:0/2264616546 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f8f8c038370 0x7f8f8c03a830 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c006fd0 tx=0x7f8f9c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:44.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:44.817+0000 7f8f967fc700 1 -- 172.21.15.71:0/2264616546 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8f9000b960 con 0x7f8fa00ff780 2024-04-23T02:04:44.862 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:44 smithi071 ceph-mon[39837]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:04:45.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:45.021+0000 7f8fa7851700 1 -- 172.21.15.71:0/2264616546 --> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fa00611d0 con 0x7f8f8c038370 2024-04-23T02:04:46.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:46 smithi071 ceph-mon[39837]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:04:46.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:46 smithi071 ceph-mon[39837]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:04:46.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:46 smithi071 ceph-mon[39837]: Deploying cephadm binary to smithi192 2024-04-23T02:04:47.197 INFO:teuthology.orchestra.run.smithi071.stdout:Added host 'smithi192' with addr '172.21.15.192' 2024-04-23T02:04:47.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:47.193+0000 7f8f967fc700 1 -- 172.21.15.71:0/2264616546 <== mgr.14162 v2:172.21.15.71:6800/4165305842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f8fa00611d0 con 0x7f8f8c038370 2024-04-23T02:04:47.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:47.195+0000 7f8fa7851700 1 -- 172.21.15.71:0/2264616546 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f8f8c038370 msgr2=0x7f8f8c03a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:47.195+0000 7f8fa7851700 1 --2- 172.21.15.71:0/2264616546 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f8f8c038370 0x7f8f8c03a830 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c006fd0 tx=0x7f8f9c006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:47.196+0000 7f8fa7851700 1 -- 172.21.15.71:0/2264616546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fa00ff780 msgr2=0x7f8fa0069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:47.196+0000 7f8fa7851700 1 --2- 172.21.15.71:0/2264616546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fa00ff780 0x7f8fa0069380 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8f90006010 tx=0x7f8f900057b0 comp rx=0 tx=0).stop 2024-04-23T02:04:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:47.196+0000 7f8fa7851700 1 -- 172.21.15.71:0/2264616546 shutdown_connections 2024-04-23T02:04:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:47.196+0000 7f8fa7851700 1 --2- 172.21.15.71:0/2264616546 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f8f8c038370 0x7f8f8c03a830 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:47.196+0000 7f8fa7851700 1 --2- 172.21.15.71:0/2264616546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fa00ff780 0x7f8fa0069380 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:47.196+0000 7f8fa7851700 1 -- 172.21.15.71:0/2264616546 >> 172.21.15.71:0/2264616546 conn(0x7f8fa00fad00 msgr2=0x7f8fa00fba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:47.196+0000 7f8fa7851700 1 -- 172.21.15.71:0/2264616546 shutdown_connections 2024-04-23T02:04:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:47.196+0000 7f8fa7851700 1 -- 172.21.15.71:0/2264616546 wait complete. 2024-04-23T02:04:47.904 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-23T02:04:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:47 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:47 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:47 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:47 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:47 smithi071 ceph-mon[39837]: Deploying daemon node-exporter.smithi071 on smithi071 2024-04-23T02:04:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:47 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:47 smithi071 ceph-mon[39837]: Added host smithi192 2024-04-23T02:04:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:47 smithi071 ceph-mon[39837]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:04:48.142 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:04:49.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:49 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:50.560 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:50 smithi071 ceph-mon[39837]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:04:50.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.805+0000 7fc411c60700 1 -- 172.21.15.71:0/4045759214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc40c1008a0 msgr2=0x7fc40c100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:50.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.805+0000 7fc411c60700 1 --2- 172.21.15.71:0/4045759214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc40c1008a0 0x7fc40c100cc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009910 tx=0x7fc3fc009c20 comp rx=0 tx=0).stop 2024-04-23T02:04:50.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.805+0000 7fc411c60700 1 -- 172.21.15.71:0/4045759214 shutdown_connections 2024-04-23T02:04:50.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.805+0000 7fc411c60700 1 --2- 172.21.15.71:0/4045759214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc40c1008a0 0x7fc40c100cc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:50.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.805+0000 7fc411c60700 1 -- 172.21.15.71:0/4045759214 >> 172.21.15.71:0/4045759214 conn(0x7fc40c0fbe20 msgr2=0x7fc40c0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:50.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.805+0000 7fc411c60700 1 -- 172.21.15.71:0/4045759214 shutdown_connections 2024-04-23T02:04:50.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.805+0000 7fc411c60700 1 -- 172.21.15.71:0/4045759214 wait complete. 2024-04-23T02:04:50.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.806+0000 7fc411c60700 1 Processor -- start 2024-04-23T02:04:50.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.806+0000 7fc411c60700 1 -- start start 2024-04-23T02:04:50.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.806+0000 7fc411c60700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc40c1008a0 0x7fc40c195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:50.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.806+0000 7fc411c60700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c195e40 con 0x7fc40c1008a0 2024-04-23T02:04:50.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.806+0000 7fc40b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc40c1008a0 0x7fc40c195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:50.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.806+0000 7fc40b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc40c1008a0 0x7fc40c195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60342/0 (socket says 172.21.15.71:60342) 2024-04-23T02:04:50.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.806+0000 7fc40b7fe700 1 -- 172.21.15.71:0/849486756 learned_addr learned my addr 172.21.15.71:0/849486756 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:50.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.806+0000 7fc40b7fe700 1 -- 172.21.15.71:0/849486756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3fc0095f0 con 0x7fc40c1008a0 2024-04-23T02:04:50.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.807+0000 7fc40b7fe700 1 --2- 172.21.15.71:0/849486756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc40c1008a0 0x7fc40c195900 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc006010 tx=0x7fc3fc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:50.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.807+0000 7fc408ff9700 1 -- 172.21.15.71:0/849486756 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3fc01d070 con 0x7fc40c1008a0 2024-04-23T02:04:50.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.807+0000 7fc408ff9700 1 -- 172.21.15.71:0/849486756 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc3fc00bdf0 con 0x7fc40c1008a0 2024-04-23T02:04:50.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.807+0000 7fc411c60700 1 -- 172.21.15.71:0/849486756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc40c196040 con 0x7fc40c1008a0 2024-04-23T02:04:50.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.807+0000 7fc408ff9700 1 -- 172.21.15.71:0/849486756 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3fc022e50 con 0x7fc40c1008a0 2024-04-23T02:04:50.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.807+0000 7fc411c60700 1 -- 172.21.15.71:0/849486756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc40c1964e0 con 0x7fc40c1008a0 2024-04-23T02:04:50.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.808+0000 7fc408ff9700 1 -- 172.21.15.71:0/849486756 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fc3fc00f460 con 0x7fc40c1008a0 2024-04-23T02:04:50.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.808+0000 7fc408ff9700 1 --2- 172.21.15.71:0/849486756 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc3f40383c0 0x7fc3f403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:50.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.808+0000 7fc408ff9700 1 -- 172.21.15.71:0/849486756 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc3fc00b960 con 0x7fc40c1008a0 2024-04-23T02:04:50.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.808+0000 7fc40affd700 1 --2- 172.21.15.71:0/849486756 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc3f40383c0 0x7fc3f403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:50.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.808+0000 7fc411c60700 1 -- 172.21.15.71:0/849486756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc40c18f380 con 0x7fc40c1008a0 2024-04-23T02:04:50.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.808+0000 7fc40affd700 1 --2- 172.21.15.71:0/849486756 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc3f40383c0 0x7fc3f403a880 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc400006fd0 tx=0x7fc400006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:50.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:50.813+0000 7fc408ff9700 1 -- 172.21.15.71:0/849486756 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc3fc027070 con 0x7fc40c1008a0 2024-04-23T02:04:51.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:51.010+0000 7fc411c60700 1 -- 172.21.15.71:0/849486756 --> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc40c0611d0 con 0x7fc3f40383c0 2024-04-23T02:04:51.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:51.011+0000 7fc408ff9700 1 -- 172.21.15.71:0/849486756 <== mgr.14162 v2:172.21.15.71:6800/4165305842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fc40c0611d0 con 0x7fc3f40383c0 2024-04-23T02:04:51.014 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:04:51.014 INFO:teuthology.orchestra.run.smithi071.stdout:[{"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}, {"addr": "172.21.15.192", "hostname": "smithi192", "labels": [], "status": ""}] 2024-04-23T02:04:51.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:51.013+0000 7fc411c60700 1 -- 172.21.15.71:0/849486756 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc3f40383c0 msgr2=0x7fc3f403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:51.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:51.013+0000 7fc411c60700 1 --2- 172.21.15.71:0/849486756 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc3f40383c0 0x7fc3f403a880 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc400006fd0 tx=0x7fc400006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:51.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:51.013+0000 7fc411c60700 1 -- 172.21.15.71:0/849486756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc40c1008a0 msgr2=0x7fc40c195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:51.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:51.013+0000 7fc411c60700 1 --2- 172.21.15.71:0/849486756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc40c1008a0 0x7fc40c195900 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc006010 tx=0x7fc3fc0057b0 comp rx=0 tx=0).stop 2024-04-23T02:04:51.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:51.013+0000 7fc411c60700 1 -- 172.21.15.71:0/849486756 shutdown_connections 2024-04-23T02:04:51.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:51.013+0000 7fc411c60700 1 --2- 172.21.15.71:0/849486756 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc3f40383c0 0x7fc3f403a880 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:51.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:51.013+0000 7fc411c60700 1 --2- 172.21.15.71:0/849486756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc40c1008a0 0x7fc40c195900 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:51.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:51.013+0000 7fc411c60700 1 -- 172.21.15.71:0/849486756 >> 172.21.15.71:0/849486756 conn(0x7fc40c0fbe20 msgr2=0x7fc40c0fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:51.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:51.013+0000 7fc411c60700 1 -- 172.21.15.71:0/849486756 shutdown_connections 2024-04-23T02:04:51.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:51.013+0000 7fc411c60700 1 -- 172.21.15.71:0/849486756 wait complete. 2024-04-23T02:04:52.049 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-23T02:04:52.049 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-23T02:04:52.278 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:04:52.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:52 smithi071 ceph-mon[39837]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T02:04:52.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:52 smithi071 ceph-mon[39837]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:04:52.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:52 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:52.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:52 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:52.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:52 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:52.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:52 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:53.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:53 smithi071 ceph-mon[39837]: Deploying daemon alertmanager.smithi071 on smithi071 2024-04-23T02:04:53.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:53 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:54.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.306+0000 7fc173318700 1 -- 172.21.15.71:0/216113223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc16c101310 msgr2=0x7fc16c103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:54.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.306+0000 7fc173318700 1 --2- 172.21.15.71:0/216113223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc16c101310 0x7fc16c103700 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc168009910 tx=0x7fc168009c20 comp rx=0 tx=0).stop 2024-04-23T02:04:54.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.306+0000 7fc173318700 1 -- 172.21.15.71:0/216113223 shutdown_connections 2024-04-23T02:04:54.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.306+0000 7fc173318700 1 --2- 172.21.15.71:0/216113223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc16c101310 0x7fc16c103700 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:54.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.306+0000 7fc173318700 1 -- 172.21.15.71:0/216113223 >> 172.21.15.71:0/216113223 conn(0x7fc16c0fad00 msgr2=0x7fc16c0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:54.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.306+0000 7fc173318700 1 -- 172.21.15.71:0/216113223 shutdown_connections 2024-04-23T02:04:54.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.306+0000 7fc173318700 1 -- 172.21.15.71:0/216113223 wait complete. 2024-04-23T02:04:54.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.307+0000 7fc173318700 1 Processor -- start 2024-04-23T02:04:54.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.307+0000 7fc173318700 1 -- start start 2024-04-23T02:04:54.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.307+0000 7fc173318700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc16c101310 0x7fc16c100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.307+0000 7fc173318700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c1011b0 con 0x7fc16c101310 2024-04-23T02:04:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.307+0000 7fc1710b4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc16c101310 0x7fc16c100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.307+0000 7fc1710b4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc16c101310 0x7fc16c100c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60362/0 (socket says 172.21.15.71:60362) 2024-04-23T02:04:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.307+0000 7fc1710b4700 1 -- 172.21.15.71:0/3736442763 learned_addr learned my addr 172.21.15.71:0/3736442763 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:04:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.307+0000 7fc1710b4700 1 -- 172.21.15.71:0/3736442763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1680095f0 con 0x7fc16c101310 2024-04-23T02:04:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.307+0000 7fc1710b4700 1 --2- 172.21.15.71:0/3736442763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc16c101310 0x7fc16c100c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc168004f40 tx=0x7fc1680057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.307+0000 7fc1627fc700 1 -- 172.21.15.71:0/3736442763 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc16801d070 con 0x7fc16c101310 2024-04-23T02:04:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.307+0000 7fc173318700 1 -- 172.21.15.71:0/3736442763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c0ff2c0 con 0x7fc16c101310 2024-04-23T02:04:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.307+0000 7fc1627fc700 1 -- 172.21.15.71:0/3736442763 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc16800bdf0 con 0x7fc16c101310 2024-04-23T02:04:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.307+0000 7fc1627fc700 1 -- 172.21.15.71:0/3736442763 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc168022e00 con 0x7fc16c101310 2024-04-23T02:04:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.308+0000 7fc173318700 1 -- 172.21.15.71:0/3736442763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16c0ff760 con 0x7fc16c101310 2024-04-23T02:04:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.309+0000 7fc1627fc700 1 -- 172.21.15.71:0/3736442763 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fc16800f4b0 con 0x7fc16c101310 2024-04-23T02:04:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.308+0000 7fc173318700 1 -- 172.21.15.71:0/3736442763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16c0623c0 con 0x7fc16c101310 2024-04-23T02:04:54.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.309+0000 7fc1627fc700 1 --2- 172.21.15.71:0/3736442763 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc1580383c0 0x7fc15803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:54.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.309+0000 7fc1627fc700 1 -- 172.21.15.71:0/3736442763 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc16804c510 con 0x7fc16c101310 2024-04-23T02:04:54.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.310+0000 7fc1708b3700 1 --2- 172.21.15.71:0/3736442763 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc1580383c0 0x7fc15803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:54.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.310+0000 7fc1708b3700 1 --2- 172.21.15.71:0/3736442763 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc1580383c0 0x7fc15803a880 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc15c006fd0 tx=0x7fc15c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:54.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.314+0000 7fc1627fc700 1 -- 172.21.15.71:0/3736442763 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc168020bf0 con 0x7fc16c101310 2024-04-23T02:04:54.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.527+0000 7fc173318700 1 -- 172.21.15.71:0/3736442763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fc16c18ed40 con 0x7fc16c101310 2024-04-23T02:04:54.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.573+0000 7fc1627fc700 1 -- 172.21.15.71:0/3736442763 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fc168027070 con 0x7fc16c101310 2024-04-23T02:04:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.576+0000 7fc173318700 1 -- 172.21.15.71:0/3736442763 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc1580383c0 msgr2=0x7fc15803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.576+0000 7fc173318700 1 --2- 172.21.15.71:0/3736442763 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc1580383c0 0x7fc15803a880 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc15c006fd0 tx=0x7fc15c006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.576+0000 7fc173318700 1 -- 172.21.15.71:0/3736442763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc16c101310 msgr2=0x7fc16c100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.576+0000 7fc173318700 1 --2- 172.21.15.71:0/3736442763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc16c101310 0x7fc16c100c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc168004f40 tx=0x7fc1680057b0 comp rx=0 tx=0).stop 2024-04-23T02:04:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.576+0000 7fc173318700 1 -- 172.21.15.71:0/3736442763 shutdown_connections 2024-04-23T02:04:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.576+0000 7fc173318700 1 --2- 172.21.15.71:0/3736442763 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc1580383c0 0x7fc15803a880 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.576+0000 7fc173318700 1 --2- 172.21.15.71:0/3736442763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc16c101310 0x7fc16c100c70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.576+0000 7fc173318700 1 -- 172.21.15.71:0/3736442763 >> 172.21.15.71:0/3736442763 conn(0x7fc16c0fad00 msgr2=0x7fc16c0fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.576+0000 7fc173318700 1 -- 172.21.15.71:0/3736442763 shutdown_connections 2024-04-23T02:04:54.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:04:54.576+0000 7fc173318700 1 -- 172.21.15.71:0/3736442763 wait complete. 2024-04-23T02:04:54.580 INFO:teuthology.orchestra.run.smithi071.stderr:adjusted tunables profile to default 2024-04-23T02:04:54.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:54 smithi071 ceph-mon[39837]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:04:54.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:54 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3736442763' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-23T02:04:55.630 INFO:tasks.cephadm:Adding mon.smithi071 on smithi071 2024-04-23T02:04:55.630 INFO:tasks.cephadm:Adding mon.smithi192 on smithi192 2024-04-23T02:04:55.631 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph orch apply mon '2;smithi071:172.21.15.71=smithi071;smithi192:172.21.15.192=smithi192' 2024-04-23T02:04:55.881 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:04:55.940 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:04:55.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:55 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3736442763' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-23T02:04:55.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:55 smithi071 ceph-mon[39837]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T02:04:56.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:56 smithi071 ceph-mon[39837]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:04:57.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.646+0000 7f3efed5c700 1 -- 172.21.15.192:0/3515339814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef8100880 msgr2=0x7f3ef8100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:57.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.646+0000 7f3efed5c700 1 --2- 172.21.15.192:0/3515339814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef8100880 0x7f3ef8100ca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009910 tx=0x7f3eec009c20 comp rx=0 tx=0).stop 2024-04-23T02:04:57.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.646+0000 7f3efed5c700 1 -- 172.21.15.192:0/3515339814 shutdown_connections 2024-04-23T02:04:57.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.646+0000 7f3efed5c700 1 --2- 172.21.15.192:0/3515339814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef8100880 0x7f3ef8100ca0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:57.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.646+0000 7f3efed5c700 1 -- 172.21.15.192:0/3515339814 >> 172.21.15.192:0/3515339814 conn(0x7f3ef80fbe00 msgr2=0x7f3ef80fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:57.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.646+0000 7f3efed5c700 1 -- 172.21.15.192:0/3515339814 shutdown_connections 2024-04-23T02:04:57.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.646+0000 7f3efed5c700 1 -- 172.21.15.192:0/3515339814 wait complete. 2024-04-23T02:04:57.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.647+0000 7f3efed5c700 1 Processor -- start 2024-04-23T02:04:57.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.647+0000 7f3efed5c700 1 -- start start 2024-04-23T02:04:57.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.647+0000 7f3efed5c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef8100880 0x7f3ef81958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:57.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.647+0000 7f3efed5c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef8195e00 con 0x7f3ef8100880 2024-04-23T02:04:57.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.647+0000 7f3efcaf8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef8100880 0x7f3ef81958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:57.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.647+0000 7f3efcaf8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef8100880 0x7f3ef81958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.192:45928/0 (socket says 172.21.15.192:45928) 2024-04-23T02:04:57.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.647+0000 7f3efcaf8700 1 -- 172.21.15.192:0/3308022749 learned_addr learned my addr 172.21.15.192:0/3308022749 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:04:57.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.647+0000 7f3efcaf8700 1 -- 172.21.15.192:0/3308022749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec0095f0 con 0x7f3ef8100880 2024-04-23T02:04:57.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.647+0000 7f3efcaf8700 1 --2- 172.21.15.192:0/3308022749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef8100880 0x7f3ef81958c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3eec004f70 tx=0x7f3eec005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:57.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.648+0000 7f3ef5ffb700 1 -- 172.21.15.192:0/3308022749 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3eec01d070 con 0x7f3ef8100880 2024-04-23T02:04:57.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.648+0000 7f3ef5ffb700 1 -- 172.21.15.192:0/3308022749 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f3eec00bdf0 con 0x7f3ef8100880 2024-04-23T02:04:57.654 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.648+0000 7f3efed5c700 1 -- 172.21.15.192:0/3308022749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef8196000 con 0x7f3ef8100880 2024-04-23T02:04:57.654 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.648+0000 7f3ef5ffb700 1 -- 172.21.15.192:0/3308022749 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3eec022e40 con 0x7f3ef8100880 2024-04-23T02:04:57.654 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.648+0000 7f3efed5c700 1 -- 172.21.15.192:0/3308022749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef81964a0 con 0x7f3ef8100880 2024-04-23T02:04:57.655 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.649+0000 7f3ef5ffb700 1 -- 172.21.15.192:0/3308022749 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f3eec00b960 con 0x7f3ef8100880 2024-04-23T02:04:57.655 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.649+0000 7f3efed5c700 1 -- 172.21.15.192:0/3308022749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef818f360 con 0x7f3ef8100880 2024-04-23T02:04:57.655 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.649+0000 7f3ef5ffb700 1 --2- 172.21.15.192:0/3308022749 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f3ee80383c0 0x7f3ee803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:04:57.655 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.649+0000 7f3ef5ffb700 1 -- 172.21.15.192:0/3308022749 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3eec04c420 con 0x7f3ef8100880 2024-04-23T02:04:57.655 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.649+0000 7f3ef7fff700 1 --2- 172.21.15.192:0/3308022749 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f3ee80383c0 0x7f3ee803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:04:57.655 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.650+0000 7f3ef7fff700 1 --2- 172.21.15.192:0/3308022749 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f3ee80383c0 0x7f3ee803a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4006fd0 tx=0x7f3ee4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:04:57.658 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.654+0000 7f3ef5ffb700 1 -- 172.21.15.192:0/3308022749 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3eec00d340 con 0x7f3ef8100880 2024-04-23T02:04:57.860 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.856+0000 7f3efed5c700 1 -- 172.21.15.192:0/3308022749 --> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi071:172.21.15.71=smithi071;smithi192:172.21.15.192=smithi192", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ef8061340 con 0x7f3ee80383c0 2024-04-23T02:04:57.867 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.864+0000 7f3ef5ffb700 1 -- 172.21.15.192:0/3308022749 <== mgr.14162 v2:172.21.15.71:6800/4165305842 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3ef8061340 con 0x7f3ee80383c0 2024-04-23T02:04:57.867 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mon update... 2024-04-23T02:04:57.869 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.865+0000 7f3efed5c700 1 -- 172.21.15.192:0/3308022749 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f3ee80383c0 msgr2=0x7f3ee803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:57.869 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.865+0000 7f3efed5c700 1 --2- 172.21.15.192:0/3308022749 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f3ee80383c0 0x7f3ee803a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4006fd0 tx=0x7f3ee4006e40 comp rx=0 tx=0).stop 2024-04-23T02:04:57.869 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.865+0000 7f3efed5c700 1 -- 172.21.15.192:0/3308022749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef8100880 msgr2=0x7f3ef81958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:04:57.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.865+0000 7f3efed5c700 1 --2- 172.21.15.192:0/3308022749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef8100880 0x7f3ef81958c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3eec004f70 tx=0x7f3eec005ee0 comp rx=0 tx=0).stop 2024-04-23T02:04:57.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.865+0000 7f3efed5c700 1 -- 172.21.15.192:0/3308022749 shutdown_connections 2024-04-23T02:04:57.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.865+0000 7f3efed5c700 1 --2- 172.21.15.192:0/3308022749 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f3ee80383c0 0x7f3ee803a880 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:57.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.865+0000 7f3efed5c700 1 --2- 172.21.15.192:0/3308022749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef8100880 0x7f3ef81958c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:04:57.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.865+0000 7f3efed5c700 1 -- 172.21.15.192:0/3308022749 >> 172.21.15.192:0/3308022749 conn(0x7f3ef80fbe00 msgr2=0x7f3ef80fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:04:57.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.865+0000 7f3efed5c700 1 -- 172.21.15.192:0/3308022749 shutdown_connections 2024-04-23T02:04:57.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:04:57.865+0000 7f3efed5c700 1 -- 172.21.15.192:0/3308022749 wait complete. 2024-04-23T02:04:58.586 DEBUG:teuthology.orchestra.run.smithi192:mon.smithi192> sudo journalctl -f -n 0 -u ceph-91f5470c-0115-11ef-bc93-c7b262605968@mon.smithi192.service 2024-04-23T02:04:58.589 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-23T02:04:58.589 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T02:04:58.619 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:-- Logs begin at Tue 2024-04-23 01:51:29 UTC. -- 2024-04-23T02:04:58.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:58 smithi071 ceph-mon[39837]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:04:58.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:58 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:04:58.859 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:04:58.920 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:04:59.947 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:59 smithi071 ceph-mon[39837]: from='client.14195 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi071:172.21.15.71=smithi071;smithi192:172.21.15.192=smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:04:59.948 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:04:59 smithi071 ceph-mon[39837]: Saving service mon spec with placement smithi071:172.21.15.71=smithi071;smithi192:172.21.15.192=smithi192;count:2 2024-04-23T02:05:00.576 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.547+0000 7f5685ad2700 1 -- 172.21.15.192:0/638305771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56800ff780 msgr2=0x7f56800ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:00.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.547+0000 7f5685ad2700 1 --2- 172.21.15.192:0/638305771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56800ff780 0x7f56800ffba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5670009910 tx=0x7f5670009c20 comp rx=0 tx=0).stop 2024-04-23T02:05:00.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.547+0000 7f5685ad2700 1 -- 172.21.15.192:0/638305771 shutdown_connections 2024-04-23T02:05:00.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.547+0000 7f5685ad2700 1 --2- 172.21.15.192:0/638305771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56800ff780 0x7f56800ffba0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:00.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.547+0000 7f5685ad2700 1 -- 172.21.15.192:0/638305771 >> 172.21.15.192:0/638305771 conn(0x7f56800fad20 msgr2=0x7f56800fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:00.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.547+0000 7f5685ad2700 1 -- 172.21.15.192:0/638305771 shutdown_connections 2024-04-23T02:05:00.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.547+0000 7f5685ad2700 1 -- 172.21.15.192:0/638305771 wait complete. 2024-04-23T02:05:00.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.548+0000 7f5685ad2700 1 Processor -- start 2024-04-23T02:05:00.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.548+0000 7f5685ad2700 1 -- start start 2024-04-23T02:05:00.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.548+0000 7f5685ad2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56800ff780 0x7f56801958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:00.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.548+0000 7f5685ad2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5680195e30 con 0x7f56800ff780 2024-04-23T02:05:00.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.548+0000 7f567f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56800ff780 0x7f56801958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:00.577 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.548+0000 7f567f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56800ff780 0x7f56801958f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.192:39384/0 (socket says 172.21.15.192:39384) 2024-04-23T02:05:00.578 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.548+0000 7f567f7fe700 1 -- 172.21.15.192:0/2671885107 learned_addr learned my addr 172.21.15.192:0/2671885107 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:05:00.578 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.549+0000 7f567f7fe700 1 -- 172.21.15.192:0/2671885107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56700095f0 con 0x7f56800ff780 2024-04-23T02:05:00.578 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.549+0000 7f567f7fe700 1 --2- 172.21.15.192:0/2671885107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56800ff780 0x7f56801958f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5670006010 tx=0x7f56700057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:00.578 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.549+0000 7f567cff9700 1 -- 172.21.15.192:0/2671885107 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f567001d070 con 0x7f56800ff780 2024-04-23T02:05:00.578 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.549+0000 7f5685ad2700 1 -- 172.21.15.192:0/2671885107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5680196030 con 0x7f56800ff780 2024-04-23T02:05:00.578 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.549+0000 7f567cff9700 1 -- 172.21.15.192:0/2671885107 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f567000bdf0 con 0x7f56800ff780 2024-04-23T02:05:00.578 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.549+0000 7f567cff9700 1 -- 172.21.15.192:0/2671885107 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5670022e50 con 0x7f56800ff780 2024-04-23T02:05:00.578 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.549+0000 7f5685ad2700 1 -- 172.21.15.192:0/2671885107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56801964d0 con 0x7f56800ff780 2024-04-23T02:05:00.578 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.550+0000 7f567cff9700 1 -- 172.21.15.192:0/2671885107 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f567000f4b0 con 0x7f56800ff780 2024-04-23T02:05:00.578 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.550+0000 7f567cff9700 1 --2- 172.21.15.192:0/2671885107 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f566c038370 0x7f566c03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:00.578 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.550+0000 7f567cff9700 1 -- 172.21.15.192:0/2671885107 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f567004c3e0 con 0x7f56800ff780 2024-04-23T02:05:00.578 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.550+0000 7f5685ad2700 1 -- 172.21.15.192:0/2671885107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568018f270 con 0x7f56800ff780 2024-04-23T02:05:00.578 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.550+0000 7f567effd700 1 --2- 172.21.15.192:0/2671885107 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f566c038370 0x7f566c03a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:00.578 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.551+0000 7f567effd700 1 --2- 172.21.15.192:0/2671885107 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f566c038370 0x7f566c03a830 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5668006fd0 tx=0x7f5668006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:00.579 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.555+0000 7f567cff9700 1 -- 172.21.15.192:0/2671885107 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5670027070 con 0x7f56800ff780 2024-04-23T02:05:00.686 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:00 smithi071 ceph-mon[39837]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:05:00.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.818+0000 7f5685ad2700 1 -- 172.21.15.192:0/2671885107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f56800623c0 con 0x7f56800ff780 2024-04-23T02:05:00.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.818+0000 7f567cff9700 1 -- 172.21.15.192:0/2671885107 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f5670034070 con 0x7f56800ff780 2024-04-23T02:05:00.822 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:05:00.822 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","modified":"2024-04-23T02:03:12.002793Z","created":"2024-04-23T02:03:12.002793Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi071","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T02:05:00.823 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.820+0000 7f5685ad2700 1 -- 172.21.15.192:0/2671885107 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f566c038370 msgr2=0x7f566c03a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:00.823 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.820+0000 7f5685ad2700 1 --2- 172.21.15.192:0/2671885107 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f566c038370 0x7f566c03a830 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5668006fd0 tx=0x7f5668006e40 comp rx=0 tx=0).stop 2024-04-23T02:05:00.824 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.820+0000 7f5685ad2700 1 -- 172.21.15.192:0/2671885107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56800ff780 msgr2=0x7f56801958f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:00.824 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.820+0000 7f5685ad2700 1 --2- 172.21.15.192:0/2671885107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56800ff780 0x7f56801958f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5670006010 tx=0x7f56700057b0 comp rx=0 tx=0).stop 2024-04-23T02:05:00.824 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.820+0000 7f5685ad2700 1 -- 172.21.15.192:0/2671885107 shutdown_connections 2024-04-23T02:05:00.824 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.820+0000 7f5685ad2700 1 --2- 172.21.15.192:0/2671885107 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f566c038370 0x7f566c03a830 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:00.824 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.820+0000 7f5685ad2700 1 --2- 172.21.15.192:0/2671885107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56800ff780 0x7f56801958f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:00.824 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.820+0000 7f5685ad2700 1 -- 172.21.15.192:0/2671885107 >> 172.21.15.192:0/2671885107 conn(0x7f56800fad20 msgr2=0x7f5680068820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:00.824 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.820+0000 7f5685ad2700 1 -- 172.21.15.192:0/2671885107 shutdown_connections 2024-04-23T02:05:00.824 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:00.821+0000 7f5685ad2700 1 -- 172.21.15.192:0/2671885107 wait complete. 2024-04-23T02:05:00.824 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-23T02:05:01.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:01 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/2671885107' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T02:05:02.556 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-23T02:05:02.556 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T02:05:02.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:02 smithi071 ceph-mon[39837]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:05:02.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:02 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:02.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:02 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:02.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:02 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:02.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:02 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:02.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:02 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:02.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:02 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:02.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:02 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:02.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:02 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:02.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:02 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-23T02:05:02.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:02 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:02.805 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:02.872 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:03.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:03 smithi071 ceph-mon[39837]: Regenerating cephadm self-signed grafana TLS certificates 2024-04-23T02:05:03.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:03 smithi071 ceph-mon[39837]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-23T02:05:03.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:03 smithi071 ceph-mon[39837]: Deploying daemon grafana.smithi071 on smithi071 2024-04-23T02:05:03.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:03 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:04.593 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.588+0000 7f52cbaf5700 1 -- 172.21.15.192:0/2375212068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c40ff780 msgr2=0x7f52c40ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:04.593 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.588+0000 7f52cbaf5700 1 --2- 172.21.15.192:0/2375212068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c40ff780 0x7f52c40ffba0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009910 tx=0x7f52b4009c20 comp rx=0 tx=0).stop 2024-04-23T02:05:04.593 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.588+0000 7f52cbaf5700 1 -- 172.21.15.192:0/2375212068 shutdown_connections 2024-04-23T02:05:04.593 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.588+0000 7f52cbaf5700 1 --2- 172.21.15.192:0/2375212068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c40ff780 0x7f52c40ffba0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:04.593 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.588+0000 7f52cbaf5700 1 -- 172.21.15.192:0/2375212068 >> 172.21.15.192:0/2375212068 conn(0x7f52c40fad20 msgr2=0x7f52c40fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:04.593 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.588+0000 7f52cbaf5700 1 -- 172.21.15.192:0/2375212068 shutdown_connections 2024-04-23T02:05:04.593 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.588+0000 7f52cbaf5700 1 -- 172.21.15.192:0/2375212068 wait complete. 2024-04-23T02:05:04.593 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.589+0000 7f52cbaf5700 1 Processor -- start 2024-04-23T02:05:04.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.589+0000 7f52cbaf5700 1 -- start start 2024-04-23T02:05:04.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.589+0000 7f52cbaf5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c40ff780 0x7f52c4194800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:04.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.589+0000 7f52cbaf5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c4194d40 con 0x7f52c40ff780 2024-04-23T02:05:04.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.589+0000 7f52c9891700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c40ff780 0x7f52c4194800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:04.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.589+0000 7f52c9891700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c40ff780 0x7f52c4194800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.192:39404/0 (socket says 172.21.15.192:39404) 2024-04-23T02:05:04.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.589+0000 7f52c9891700 1 -- 172.21.15.192:0/1312139634 learned_addr learned my addr 172.21.15.192:0/1312139634 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:05:04.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.589+0000 7f52c9891700 1 -- 172.21.15.192:0/1312139634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b40095f0 con 0x7f52c40ff780 2024-04-23T02:05:04.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.589+0000 7f52c9891700 1 --2- 172.21.15.192:0/1312139634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c40ff780 0x7f52c4194800 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f52b4004f70 tx=0x7f52b4005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:04.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.590+0000 7f52baffd700 1 -- 172.21.15.192:0/1312139634 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52b401d070 con 0x7f52c40ff780 2024-04-23T02:05:04.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.590+0000 7f52cbaf5700 1 -- 172.21.15.192:0/1312139634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c4194f40 con 0x7f52c40ff780 2024-04-23T02:05:04.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.590+0000 7f52baffd700 1 -- 172.21.15.192:0/1312139634 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f52b400bdf0 con 0x7f52c40ff780 2024-04-23T02:05:04.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.590+0000 7f52cbaf5700 1 -- 172.21.15.192:0/1312139634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c41953e0 con 0x7f52c40ff780 2024-04-23T02:05:04.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.590+0000 7f52baffd700 1 -- 172.21.15.192:0/1312139634 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52b4022e30 con 0x7f52c40ff780 2024-04-23T02:05:04.595 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.590+0000 7f52baffd700 1 -- 172.21.15.192:0/1312139634 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f52b400b960 con 0x7f52c40ff780 2024-04-23T02:05:04.595 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.590+0000 7f52cbaf5700 1 -- 172.21.15.192:0/1312139634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c4105480 con 0x7f52c40ff780 2024-04-23T02:05:04.595 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.591+0000 7f52baffd700 1 --2- 172.21.15.192:0/1312139634 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f52b00383c0 0x7f52b003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:04.595 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.591+0000 7f52baffd700 1 -- 172.21.15.192:0/1312139634 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f52b404c410 con 0x7f52c40ff780 2024-04-23T02:05:04.595 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.591+0000 7f52c9090700 1 --2- 172.21.15.192:0/1312139634 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f52b00383c0 0x7f52b003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:04.596 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.591+0000 7f52c9090700 1 --2- 172.21.15.192:0/1312139634 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f52b00383c0 0x7f52b003a880 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f52c0006fd0 tx=0x7f52c0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:04.603 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.600+0000 7f52baffd700 1 -- 172.21.15.192:0/1312139634 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f52b400f460 con 0x7f52c40ff780 2024-04-23T02:05:04.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.868+0000 7f52cbaf5700 1 -- 172.21.15.192:0/1312139634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f52c4105670 con 0x7f52c40ff780 2024-04-23T02:05:04.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.869+0000 7f52baffd700 1 -- 172.21.15.192:0/1312139634 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f52b4027030 con 0x7f52c40ff780 2024-04-23T02:05:04.872 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:05:04.872 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","modified":"2024-04-23T02:03:12.002793Z","created":"2024-04-23T02:03:12.002793Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi071","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T02:05:04.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.871+0000 7f52cbaf5700 1 -- 172.21.15.192:0/1312139634 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f52b00383c0 msgr2=0x7f52b003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:04.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.871+0000 7f52cbaf5700 1 --2- 172.21.15.192:0/1312139634 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f52b00383c0 0x7f52b003a880 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f52c0006fd0 tx=0x7f52c0006e40 comp rx=0 tx=0).stop 2024-04-23T02:05:04.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.871+0000 7f52cbaf5700 1 -- 172.21.15.192:0/1312139634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c40ff780 msgr2=0x7f52c4194800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:04.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.871+0000 7f52cbaf5700 1 --2- 172.21.15.192:0/1312139634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c40ff780 0x7f52c4194800 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f52b4004f70 tx=0x7f52b4005ee0 comp rx=0 tx=0).stop 2024-04-23T02:05:04.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.871+0000 7f52cbaf5700 1 -- 172.21.15.192:0/1312139634 shutdown_connections 2024-04-23T02:05:04.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.871+0000 7f52cbaf5700 1 --2- 172.21.15.192:0/1312139634 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f52b00383c0 0x7f52b003a880 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:04.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.871+0000 7f52cbaf5700 1 --2- 172.21.15.192:0/1312139634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c40ff780 0x7f52c4194800 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:04.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.871+0000 7f52cbaf5700 1 -- 172.21.15.192:0/1312139634 >> 172.21.15.192:0/1312139634 conn(0x7f52c40fad20 msgr2=0x7f52c40fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:04.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.871+0000 7f52cbaf5700 1 -- 172.21.15.192:0/1312139634 shutdown_connections 2024-04-23T02:05:04.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:04.871+0000 7f52cbaf5700 1 -- 172.21.15.192:0/1312139634 wait complete. 2024-04-23T02:05:04.875 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-23T02:05:04.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:04 smithi071 ceph-mon[39837]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:05:05.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:05 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/1312139634' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T02:05:06.517 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-23T02:05:06.518 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T02:05:06.770 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:06.832 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:06 smithi071 ceph-mon[39837]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:05:08.590 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.587+0000 7f2689a46700 1 -- 172.21.15.192:0/1830181342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26841008a0 msgr2=0x7f2684100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:08.590 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.587+0000 7f2689a46700 1 --2- 172.21.15.192:0/1830181342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26841008a0 0x7f2684100cc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2674009910 tx=0x7f2674009c20 comp rx=0 tx=0).stop 2024-04-23T02:05:08.590 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.587+0000 7f2689a46700 1 -- 172.21.15.192:0/1830181342 shutdown_connections 2024-04-23T02:05:08.590 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.587+0000 7f2689a46700 1 --2- 172.21.15.192:0/1830181342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26841008a0 0x7f2684100cc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:08.590 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.587+0000 7f2689a46700 1 -- 172.21.15.192:0/1830181342 >> 172.21.15.192:0/1830181342 conn(0x7f26840fbe20 msgr2=0x7f26840fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:08.591 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.587+0000 7f2689a46700 1 -- 172.21.15.192:0/1830181342 shutdown_connections 2024-04-23T02:05:08.591 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.587+0000 7f2689a46700 1 -- 172.21.15.192:0/1830181342 wait complete. 2024-04-23T02:05:08.592 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.588+0000 7f2689a46700 1 Processor -- start 2024-04-23T02:05:08.592 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.588+0000 7f2689a46700 1 -- start start 2024-04-23T02:05:08.592 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.588+0000 7f2689a46700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26841008a0 0x7f26841958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:08.592 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.588+0000 7f2689a46700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2684195df0 con 0x7f26841008a0 2024-04-23T02:05:08.592 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.588+0000 7f2682ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26841008a0 0x7f26841958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:08.593 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.588+0000 7f2682ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26841008a0 0x7f26841958b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.192:33068/0 (socket says 172.21.15.192:33068) 2024-04-23T02:05:08.593 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.588+0000 7f2682ffd700 1 -- 172.21.15.192:0/88465760 learned_addr learned my addr 172.21.15.192:0/88465760 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:05:08.593 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.588+0000 7f2682ffd700 1 -- 172.21.15.192:0/88465760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26740095f0 con 0x7f26841008a0 2024-04-23T02:05:08.593 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.589+0000 7f2682ffd700 1 --2- 172.21.15.192:0/88465760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26841008a0 0x7f26841958b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2674006010 tx=0x7f26740057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:08.593 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.589+0000 7f2688a44700 1 -- 172.21.15.192:0/88465760 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f267401d070 con 0x7f26841008a0 2024-04-23T02:05:08.593 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.589+0000 7f2689a46700 1 -- 172.21.15.192:0/88465760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2684195ff0 con 0x7f26841008a0 2024-04-23T02:05:08.593 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.589+0000 7f2688a44700 1 -- 172.21.15.192:0/88465760 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f267400bdf0 con 0x7f26841008a0 2024-04-23T02:05:08.593 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.589+0000 7f2688a44700 1 -- 172.21.15.192:0/88465760 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2674022e50 con 0x7f26841008a0 2024-04-23T02:05:08.593 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.589+0000 7f2689a46700 1 -- 172.21.15.192:0/88465760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2684196490 con 0x7f26841008a0 2024-04-23T02:05:08.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.590+0000 7f2688a44700 1 -- 172.21.15.192:0/88465760 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f2674022700 con 0x7f26841008a0 2024-04-23T02:05:08.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.590+0000 7f2689a46700 1 -- 172.21.15.192:0/88465760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f268418f380 con 0x7f26841008a0 2024-04-23T02:05:08.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.590+0000 7f2688a44700 1 --2- 172.21.15.192:0/88465760 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f266c038370 0x7f266c03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:08.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.590+0000 7f2688a44700 1 -- 172.21.15.192:0/88465760 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f267404c410 con 0x7f26841008a0 2024-04-23T02:05:08.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.590+0000 7f26827fc700 1 --2- 172.21.15.192:0/88465760 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f266c038370 0x7f266c03a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:08.594 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.590+0000 7f26827fc700 1 --2- 172.21.15.192:0/88465760 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f266c038370 0x7f266c03a830 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2678006fd0 tx=0x7f2678006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:08.598 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.596+0000 7f2688a44700 1 -- 172.21.15.192:0/88465760 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f267400b960 con 0x7f26841008a0 2024-04-23T02:05:08.861 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.858+0000 7f2689a46700 1 -- 172.21.15.192:0/88465760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f268402cc70 con 0x7f26841008a0 2024-04-23T02:05:08.862 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.859+0000 7f2688a44700 1 -- 172.21.15.192:0/88465760 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f2674027020 con 0x7f26841008a0 2024-04-23T02:05:08.862 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:05:08.862 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","modified":"2024-04-23T02:03:12.002793Z","created":"2024-04-23T02:03:12.002793Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi071","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T02:05:08.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.861+0000 7f2689a46700 1 -- 172.21.15.192:0/88465760 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f266c038370 msgr2=0x7f266c03a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:08.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.861+0000 7f2689a46700 1 --2- 172.21.15.192:0/88465760 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f266c038370 0x7f266c03a830 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2678006fd0 tx=0x7f2678006e40 comp rx=0 tx=0).stop 2024-04-23T02:05:08.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.861+0000 7f2689a46700 1 -- 172.21.15.192:0/88465760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26841008a0 msgr2=0x7f26841958b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:08.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.861+0000 7f2689a46700 1 --2- 172.21.15.192:0/88465760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26841008a0 0x7f26841958b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2674006010 tx=0x7f26740057b0 comp rx=0 tx=0).stop 2024-04-23T02:05:08.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.861+0000 7f2689a46700 1 -- 172.21.15.192:0/88465760 shutdown_connections 2024-04-23T02:05:08.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.861+0000 7f2689a46700 1 --2- 172.21.15.192:0/88465760 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f266c038370 0x7f266c03a830 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:08.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.861+0000 7f2689a46700 1 --2- 172.21.15.192:0/88465760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26841008a0 0x7f26841958b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:08.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.861+0000 7f2689a46700 1 -- 172.21.15.192:0/88465760 >> 172.21.15.192:0/88465760 conn(0x7f26840fbe20 msgr2=0x7f26840fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:08.865 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.861+0000 7f2689a46700 1 -- 172.21.15.192:0/88465760 shutdown_connections 2024-04-23T02:05:08.865 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:08.861+0000 7f2689a46700 1 -- 172.21.15.192:0/88465760 wait complete. 2024-04-23T02:05:08.865 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-23T02:05:08.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:08 smithi071 ceph-mon[39837]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:05:09.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:09 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/88465760' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T02:05:10.588 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-23T02:05:10.588 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T02:05:10.836 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:10.896 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:10.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:10 smithi071 ceph-mon[39837]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:05:12.491 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.486+0000 7fad5bfee700 1 -- 172.21.15.192:0/208187093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad540ff780 msgr2=0x7fad540ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:12.491 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.486+0000 7fad5bfee700 1 --2- 172.21.15.192:0/208187093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad540ff780 0x7fad540ffba0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fad44009910 tx=0x7fad44009c20 comp rx=0 tx=0).stop 2024-04-23T02:05:12.491 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.486+0000 7fad5bfee700 1 -- 172.21.15.192:0/208187093 shutdown_connections 2024-04-23T02:05:12.491 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.486+0000 7fad5bfee700 1 --2- 172.21.15.192:0/208187093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad540ff780 0x7fad540ffba0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:12.491 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.486+0000 7fad5bfee700 1 -- 172.21.15.192:0/208187093 >> 172.21.15.192:0/208187093 conn(0x7fad540fad20 msgr2=0x7fad540fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:12.491 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.486+0000 7fad5bfee700 1 -- 172.21.15.192:0/208187093 shutdown_connections 2024-04-23T02:05:12.491 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.486+0000 7fad5bfee700 1 -- 172.21.15.192:0/208187093 wait complete. 2024-04-23T02:05:12.491 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.487+0000 7fad5bfee700 1 Processor -- start 2024-04-23T02:05:12.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.487+0000 7fad5bfee700 1 -- start start 2024-04-23T02:05:12.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.487+0000 7fad5bfee700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad540ff780 0x7fad54194800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:12.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.487+0000 7fad5bfee700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad54194d40 con 0x7fad540ff780 2024-04-23T02:05:12.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.487+0000 7fad59d8a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad540ff780 0x7fad54194800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:12.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.487+0000 7fad59d8a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad540ff780 0x7fad54194800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.192:33090/0 (socket says 172.21.15.192:33090) 2024-04-23T02:05:12.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.487+0000 7fad59d8a700 1 -- 172.21.15.192:0/3613494387 learned_addr learned my addr 172.21.15.192:0/3613494387 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:05:12.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.487+0000 7fad59d8a700 1 -- 172.21.15.192:0/3613494387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad440095f0 con 0x7fad540ff780 2024-04-23T02:05:12.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.488+0000 7fad59d8a700 1 --2- 172.21.15.192:0/3613494387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad540ff780 0x7fad54194800 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fad44006010 tx=0x7fad440057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:12.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.488+0000 7fad4affd700 1 -- 172.21.15.192:0/3613494387 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fad4401d070 con 0x7fad540ff780 2024-04-23T02:05:12.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.488+0000 7fad5bfee700 1 -- 172.21.15.192:0/3613494387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad54194f40 con 0x7fad540ff780 2024-04-23T02:05:12.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.488+0000 7fad4affd700 1 -- 172.21.15.192:0/3613494387 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fad4400bdf0 con 0x7fad540ff780 2024-04-23T02:05:12.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.488+0000 7fad4affd700 1 -- 172.21.15.192:0/3613494387 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fad44022e50 con 0x7fad540ff780 2024-04-23T02:05:12.492 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.488+0000 7fad5bfee700 1 -- 172.21.15.192:0/3613494387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad541953e0 con 0x7fad540ff780 2024-04-23T02:05:12.493 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.489+0000 7fad4affd700 1 -- 172.21.15.192:0/3613494387 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fad4400b960 con 0x7fad540ff780 2024-04-23T02:05:12.493 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.489+0000 7fad5bfee700 1 -- 172.21.15.192:0/3613494387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5418e260 con 0x7fad540ff780 2024-04-23T02:05:12.494 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.489+0000 7fad4affd700 1 --2- 172.21.15.192:0/3613494387 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fad400383c0 0x7fad4003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:12.494 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.489+0000 7fad4affd700 1 -- 172.21.15.192:0/3613494387 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fad4404c450 con 0x7fad540ff780 2024-04-23T02:05:12.494 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.489+0000 7fad59589700 1 --2- 172.21.15.192:0/3613494387 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fad400383c0 0x7fad4003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:12.494 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.489+0000 7fad59589700 1 --2- 172.21.15.192:0/3613494387 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fad400383c0 0x7fad4003a880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fad50006fd0 tx=0x7fad50006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:12.497 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.494+0000 7fad4affd700 1 -- 172.21.15.192:0/3613494387 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fad4400f460 con 0x7fad540ff780 2024-04-23T02:05:12.758 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.755+0000 7fad5bfee700 1 -- 172.21.15.192:0/3613494387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fad5402cc70 con 0x7fad540ff780 2024-04-23T02:05:12.759 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.756+0000 7fad4affd700 1 -- 172.21.15.192:0/3613494387 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fad44027020 con 0x7fad540ff780 2024-04-23T02:05:12.759 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:05:12.759 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","modified":"2024-04-23T02:03:12.002793Z","created":"2024-04-23T02:03:12.002793Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi071","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T02:05:12.761 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.758+0000 7fad5bfee700 1 -- 172.21.15.192:0/3613494387 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fad400383c0 msgr2=0x7fad4003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:12.761 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.758+0000 7fad5bfee700 1 --2- 172.21.15.192:0/3613494387 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fad400383c0 0x7fad4003a880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fad50006fd0 tx=0x7fad50006e40 comp rx=0 tx=0).stop 2024-04-23T02:05:12.761 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.758+0000 7fad5bfee700 1 -- 172.21.15.192:0/3613494387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad540ff780 msgr2=0x7fad54194800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:12.761 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.758+0000 7fad5bfee700 1 --2- 172.21.15.192:0/3613494387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad540ff780 0x7fad54194800 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fad44006010 tx=0x7fad440057b0 comp rx=0 tx=0).stop 2024-04-23T02:05:12.761 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.758+0000 7fad5bfee700 1 -- 172.21.15.192:0/3613494387 shutdown_connections 2024-04-23T02:05:12.762 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.758+0000 7fad5bfee700 1 --2- 172.21.15.192:0/3613494387 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fad400383c0 0x7fad4003a880 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:12.762 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.758+0000 7fad5bfee700 1 --2- 172.21.15.192:0/3613494387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad540ff780 0x7fad54194800 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:12.762 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.758+0000 7fad5bfee700 1 -- 172.21.15.192:0/3613494387 >> 172.21.15.192:0/3613494387 conn(0x7fad540fad20 msgr2=0x7fad540fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:12.762 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.758+0000 7fad5bfee700 1 -- 172.21.15.192:0/3613494387 shutdown_connections 2024-04-23T02:05:12.762 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:12.758+0000 7fad5bfee700 1 -- 172.21.15.192:0/3613494387 wait complete. 2024-04-23T02:05:12.762 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-23T02:05:12.937 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:12 smithi071 ceph-mon[39837]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:05:13.725 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:13 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/3613494387' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T02:05:14.438 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-23T02:05:14.438 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T02:05:14.693 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:14.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:14 smithi071 ceph-mon[39837]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:05:14.751 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:16.622 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.612+0000 7fc611852700 1 -- 172.21.15.192:0/2086310652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc60c0ff780 msgr2=0x7fc60c0ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:16.623 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.612+0000 7fc611852700 1 --2- 172.21.15.192:0/2086310652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc60c0ff780 0x7fc60c0ffba0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc600009910 tx=0x7fc600009c20 comp rx=0 tx=0).stop 2024-04-23T02:05:16.623 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.612+0000 7fc611852700 1 -- 172.21.15.192:0/2086310652 shutdown_connections 2024-04-23T02:05:16.623 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.612+0000 7fc611852700 1 --2- 172.21.15.192:0/2086310652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc60c0ff780 0x7fc60c0ffba0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:16.623 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.612+0000 7fc611852700 1 -- 172.21.15.192:0/2086310652 >> 172.21.15.192:0/2086310652 conn(0x7fc60c0fad20 msgr2=0x7fc60c0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:16.623 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.612+0000 7fc611852700 1 -- 172.21.15.192:0/2086310652 shutdown_connections 2024-04-23T02:05:16.623 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.612+0000 7fc611852700 1 -- 172.21.15.192:0/2086310652 wait complete. 2024-04-23T02:05:16.623 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.613+0000 7fc611852700 1 Processor -- start 2024-04-23T02:05:16.623 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.613+0000 7fc611852700 1 -- start start 2024-04-23T02:05:16.623 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.613+0000 7fc611852700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc60c0ff780 0x7fc60c194800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:16.623 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.613+0000 7fc611852700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c194d40 con 0x7fc60c0ff780 2024-04-23T02:05:16.624 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.613+0000 7fc60affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc60c0ff780 0x7fc60c194800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:16.624 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.613+0000 7fc60affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc60c0ff780 0x7fc60c194800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.192:33118/0 (socket says 172.21.15.192:33118) 2024-04-23T02:05:16.624 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.613+0000 7fc60affd700 1 -- 172.21.15.192:0/1771492188 learned_addr learned my addr 172.21.15.192:0/1771492188 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:05:16.624 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.613+0000 7fc60affd700 1 -- 172.21.15.192:0/1771492188 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6000095f0 con 0x7fc60c0ff780 2024-04-23T02:05:16.624 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.614+0000 7fc60affd700 1 --2- 172.21.15.192:0/1771492188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc60c0ff780 0x7fc60c194800 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc600004f70 tx=0x7fc600005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:16.624 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.614+0000 7fc610850700 1 -- 172.21.15.192:0/1771492188 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc60001d070 con 0x7fc60c0ff780 2024-04-23T02:05:16.624 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.614+0000 7fc610850700 1 -- 172.21.15.192:0/1771492188 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc60000bdf0 con 0x7fc60c0ff780 2024-04-23T02:05:16.624 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.614+0000 7fc611852700 1 -- 172.21.15.192:0/1771492188 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60c194f40 con 0x7fc60c0ff780 2024-04-23T02:05:16.624 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.614+0000 7fc610850700 1 -- 172.21.15.192:0/1771492188 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc600022e40 con 0x7fc60c0ff780 2024-04-23T02:05:16.624 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.614+0000 7fc611852700 1 -- 172.21.15.192:0/1771492188 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60c1953e0 con 0x7fc60c0ff780 2024-04-23T02:05:16.624 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.615+0000 7fc610850700 1 -- 172.21.15.192:0/1771492188 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fc60000f4b0 con 0x7fc60c0ff780 2024-04-23T02:05:16.624 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.615+0000 7fc611852700 1 -- 172.21.15.192:0/1771492188 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60c18e260 con 0x7fc60c0ff780 2024-04-23T02:05:16.624 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.615+0000 7fc610850700 1 --2- 172.21.15.192:0/1771492188 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc5f40383c0 0x7fc5f403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:16.624 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.615+0000 7fc610850700 1 -- 172.21.15.192:0/1771492188 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc60004c5d0 con 0x7fc60c0ff780 2024-04-23T02:05:16.625 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.615+0000 7fc60a7fc700 1 --2- 172.21.15.192:0/1771492188 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc5f40383c0 0x7fc5f403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:16.625 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.615+0000 7fc60a7fc700 1 --2- 172.21.15.192:0/1771492188 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc5f40383c0 0x7fc5f403a880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc006fd0 tx=0x7fc5fc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:16.625 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.620+0000 7fc610850700 1 -- 172.21.15.192:0/1771492188 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc600020dd0 con 0x7fc60c0ff780 2024-04-23T02:05:16.884 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.881+0000 7fc611852700 1 -- 172.21.15.192:0/1771492188 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc60c02cc70 con 0x7fc60c0ff780 2024-04-23T02:05:16.885 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.882+0000 7fc610850700 1 -- 172.21.15.192:0/1771492188 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fc600027020 con 0x7fc60c0ff780 2024-04-23T02:05:16.885 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:05:16.885 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","modified":"2024-04-23T02:03:12.002793Z","created":"2024-04-23T02:03:12.002793Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi071","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T02:05:16.887 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.884+0000 7fc611852700 1 -- 172.21.15.192:0/1771492188 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc5f40383c0 msgr2=0x7fc5f403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:16.887 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.884+0000 7fc611852700 1 --2- 172.21.15.192:0/1771492188 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc5f40383c0 0x7fc5f403a880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc006fd0 tx=0x7fc5fc006e40 comp rx=0 tx=0).stop 2024-04-23T02:05:16.887 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.884+0000 7fc611852700 1 -- 172.21.15.192:0/1771492188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc60c0ff780 msgr2=0x7fc60c194800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:16.887 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.884+0000 7fc611852700 1 --2- 172.21.15.192:0/1771492188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc60c0ff780 0x7fc60c194800 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc600004f70 tx=0x7fc600005ee0 comp rx=0 tx=0).stop 2024-04-23T02:05:16.888 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.884+0000 7fc611852700 1 -- 172.21.15.192:0/1771492188 shutdown_connections 2024-04-23T02:05:16.888 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.884+0000 7fc611852700 1 --2- 172.21.15.192:0/1771492188 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc5f40383c0 0x7fc5f403a880 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:16.888 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.884+0000 7fc611852700 1 --2- 172.21.15.192:0/1771492188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc60c0ff780 0x7fc60c194800 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:16.888 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.884+0000 7fc611852700 1 -- 172.21.15.192:0/1771492188 >> 172.21.15.192:0/1771492188 conn(0x7fc60c0fad20 msgr2=0x7fc60c0fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:16.888 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.884+0000 7fc611852700 1 -- 172.21.15.192:0/1771492188 shutdown_connections 2024-04-23T02:05:16.888 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:16.884+0000 7fc611852700 1 -- 172.21.15.192:0/1771492188 wait complete. 2024-04-23T02:05:16.888 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-23T02:05:16.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:16 smithi071 ceph-mon[39837]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:05:16.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:16 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:16.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:16 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:16.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:16 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:16.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:16 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:16.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:16 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:16.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:16 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:16.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:16 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:16.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:16 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:17.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:17 smithi071 ceph-mon[39837]: Deploying daemon prometheus.smithi071 on smithi071 2024-04-23T02:05:17.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:17 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/1771492188' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T02:05:18.557 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-23T02:05:18.557 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T02:05:18.805 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:18.865 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:18.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:18 smithi071 ceph-mon[39837]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:05:18.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:18 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:20.682 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.679+0000 7efd0e159700 1 -- 172.21.15.192:0/635600347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd080ff780 msgr2=0x7efd080ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:20.682 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.679+0000 7efd0e159700 1 --2- 172.21.15.192:0/635600347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd080ff780 0x7efd080ffba0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009910 tx=0x7efcf8009c20 comp rx=0 tx=0).stop 2024-04-23T02:05:20.682 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.679+0000 7efd0e159700 1 -- 172.21.15.192:0/635600347 shutdown_connections 2024-04-23T02:05:20.682 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.679+0000 7efd0e159700 1 --2- 172.21.15.192:0/635600347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd080ff780 0x7efd080ffba0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:20.682 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.679+0000 7efd0e159700 1 -- 172.21.15.192:0/635600347 >> 172.21.15.192:0/635600347 conn(0x7efd080fad00 msgr2=0x7efd080fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:20.683 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.679+0000 7efd0e159700 1 -- 172.21.15.192:0/635600347 shutdown_connections 2024-04-23T02:05:20.683 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.679+0000 7efd0e159700 1 -- 172.21.15.192:0/635600347 wait complete. 2024-04-23T02:05:20.683 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.679+0000 7efd0e159700 1 Processor -- start 2024-04-23T02:05:20.684 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.680+0000 7efd0e159700 1 -- start start 2024-04-23T02:05:20.684 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.680+0000 7efd0e159700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd080ff780 0x7efd081958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:20.685 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.680+0000 7efd0e159700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd08195e00 con 0x7efd080ff780 2024-04-23T02:05:20.685 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.680+0000 7efd077fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd080ff780 0x7efd081958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:20.685 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.680+0000 7efd077fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd080ff780 0x7efd081958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.192:54390/0 (socket says 172.21.15.192:54390) 2024-04-23T02:05:20.685 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.680+0000 7efd077fe700 1 -- 172.21.15.192:0/3244127997 learned_addr learned my addr 172.21.15.192:0/3244127997 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:05:20.685 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.680+0000 7efd077fe700 1 -- 172.21.15.192:0/3244127997 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf80095f0 con 0x7efd080ff780 2024-04-23T02:05:20.685 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.680+0000 7efd077fe700 1 --2- 172.21.15.192:0/3244127997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd080ff780 0x7efd081958c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efcf8004f70 tx=0x7efcf8005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:20.685 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.680+0000 7efd04ff9700 1 -- 172.21.15.192:0/3244127997 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efcf801d070 con 0x7efd080ff780 2024-04-23T02:05:20.685 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.680+0000 7efd04ff9700 1 -- 172.21.15.192:0/3244127997 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7efcf800bdf0 con 0x7efd080ff780 2024-04-23T02:05:20.685 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.680+0000 7efd04ff9700 1 -- 172.21.15.192:0/3244127997 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efcf8022e40 con 0x7efd080ff780 2024-04-23T02:05:20.685 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.680+0000 7efd0e159700 1 -- 172.21.15.192:0/3244127997 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd08196000 con 0x7efd080ff780 2024-04-23T02:05:20.685 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.680+0000 7efd0e159700 1 -- 172.21.15.192:0/3244127997 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd081964a0 con 0x7efd080ff780 2024-04-23T02:05:20.685 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.681+0000 7efd04ff9700 1 -- 172.21.15.192:0/3244127997 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7efcf800b960 con 0x7efd080ff780 2024-04-23T02:05:20.685 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.681+0000 7efd0e159700 1 -- 172.21.15.192:0/3244127997 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd0818f270 con 0x7efd080ff780 2024-04-23T02:05:20.686 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.681+0000 7efd04ff9700 1 --2- 172.21.15.192:0/3244127997 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7efcf00383c0 0x7efcf003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:20.686 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.681+0000 7efd04ff9700 1 -- 172.21.15.192:0/3244127997 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efcf804c4e0 con 0x7efd080ff780 2024-04-23T02:05:20.686 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.682+0000 7efd06ffd700 1 --2- 172.21.15.192:0/3244127997 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7efcf00383c0 0x7efcf003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:20.686 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.682+0000 7efd06ffd700 1 --2- 172.21.15.192:0/3244127997 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7efcf00383c0 0x7efcf003a880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efcfc006fd0 tx=0x7efcfc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:20.689 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.687+0000 7efd04ff9700 1 -- 172.21.15.192:0/3244127997 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7efcf802a360 con 0x7efd080ff780 2024-04-23T02:05:20.958 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.955+0000 7efd0e159700 1 -- 172.21.15.192:0/3244127997 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efd0802cc70 con 0x7efd080ff780 2024-04-23T02:05:20.958 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.955+0000 7efd04ff9700 1 -- 172.21.15.192:0/3244127997 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7efcf8045e60 con 0x7efd080ff780 2024-04-23T02:05:20.959 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:05:20.959 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","modified":"2024-04-23T02:03:12.002793Z","created":"2024-04-23T02:03:12.002793Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi071","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T02:05:20.960 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.957+0000 7efd0e159700 1 -- 172.21.15.192:0/3244127997 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7efcf00383c0 msgr2=0x7efcf003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:20.960 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.957+0000 7efd0e159700 1 --2- 172.21.15.192:0/3244127997 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7efcf00383c0 0x7efcf003a880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efcfc006fd0 tx=0x7efcfc006e40 comp rx=0 tx=0).stop 2024-04-23T02:05:20.960 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.957+0000 7efd0e159700 1 -- 172.21.15.192:0/3244127997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd080ff780 msgr2=0x7efd081958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:20.960 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.957+0000 7efd0e159700 1 --2- 172.21.15.192:0/3244127997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd080ff780 0x7efd081958c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efcf8004f70 tx=0x7efcf8005ee0 comp rx=0 tx=0).stop 2024-04-23T02:05:20.960 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.957+0000 7efd0e159700 1 -- 172.21.15.192:0/3244127997 shutdown_connections 2024-04-23T02:05:20.961 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.957+0000 7efd0e159700 1 --2- 172.21.15.192:0/3244127997 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7efcf00383c0 0x7efcf003a880 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:20.961 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.957+0000 7efd0e159700 1 --2- 172.21.15.192:0/3244127997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd080ff780 0x7efd081958c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:20.961 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.957+0000 7efd0e159700 1 -- 172.21.15.192:0/3244127997 >> 172.21.15.192:0/3244127997 conn(0x7efd080fad00 msgr2=0x7efd08069090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:20.961 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.957+0000 7efd0e159700 1 -- 172.21.15.192:0/3244127997 shutdown_connections 2024-04-23T02:05:20.961 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:20.957+0000 7efd0e159700 1 -- 172.21.15.192:0/3244127997 wait complete. 2024-04-23T02:05:20.961 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-23T02:05:20.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:20 smithi071 ceph-mon[39837]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:05:21.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:21 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/3244127997' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T02:05:22.628 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-23T02:05:22.628 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T02:05:22.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:22 smithi071 ceph-mon[39837]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:05:22.880 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:22.941 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:24.648 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.644+0000 7f1f210a1700 1 -- 172.21.15.192:0/373888989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1c0ff790 msgr2=0x7f1f1c0ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:24.648 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.644+0000 7f1f210a1700 1 --2- 172.21.15.192:0/373888989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1c0ff790 0x7f1f1c0ffbb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009910 tx=0x7f1f0c009c20 comp rx=0 tx=0).stop 2024-04-23T02:05:24.648 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.644+0000 7f1f210a1700 1 -- 172.21.15.192:0/373888989 shutdown_connections 2024-04-23T02:05:24.648 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.644+0000 7f1f210a1700 1 --2- 172.21.15.192:0/373888989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1c0ff790 0x7f1f1c0ffbb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:24.649 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.644+0000 7f1f210a1700 1 -- 172.21.15.192:0/373888989 >> 172.21.15.192:0/373888989 conn(0x7f1f1c0fad30 msgr2=0x7f1f1c0fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:24.649 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.644+0000 7f1f210a1700 1 -- 172.21.15.192:0/373888989 shutdown_connections 2024-04-23T02:05:24.649 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.644+0000 7f1f210a1700 1 -- 172.21.15.192:0/373888989 wait complete. 2024-04-23T02:05:24.649 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.645+0000 7f1f210a1700 1 Processor -- start 2024-04-23T02:05:24.651 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.645+0000 7f1f210a1700 1 -- start start 2024-04-23T02:05:24.651 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.646+0000 7f1f210a1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1c0ff790 0x7f1f1c1947d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:24.651 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.646+0000 7f1f210a1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c194d10 con 0x7f1f1c0ff790 2024-04-23T02:05:24.651 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.646+0000 7f1f1ad9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1c0ff790 0x7f1f1c1947d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:24.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.646+0000 7f1f1ad9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1c0ff790 0x7f1f1c1947d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.192:54402/0 (socket says 172.21.15.192:54402) 2024-04-23T02:05:24.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.646+0000 7f1f1ad9d700 1 -- 172.21.15.192:0/3956491326 learned_addr learned my addr 172.21.15.192:0/3956491326 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:05:24.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.646+0000 7f1f1ad9d700 1 -- 172.21.15.192:0/3956491326 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c0095f0 con 0x7f1f1c0ff790 2024-04-23T02:05:24.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.646+0000 7f1f1ad9d700 1 --2- 172.21.15.192:0/3956491326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1c0ff790 0x7f1f1c1947d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c006010 tx=0x7f1f0c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:24.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.647+0000 7f1f03fff700 1 -- 172.21.15.192:0/3956491326 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f0c01d070 con 0x7f1f1c0ff790 2024-04-23T02:05:24.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.647+0000 7f1f210a1700 1 -- 172.21.15.192:0/3956491326 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1c194f10 con 0x7f1f1c0ff790 2024-04-23T02:05:24.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.647+0000 7f1f03fff700 1 -- 172.21.15.192:0/3956491326 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1f0c00bdf0 con 0x7f1f1c0ff790 2024-04-23T02:05:24.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.647+0000 7f1f03fff700 1 -- 172.21.15.192:0/3956491326 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f0c022e50 con 0x7f1f1c0ff790 2024-04-23T02:05:24.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.647+0000 7f1f210a1700 1 -- 172.21.15.192:0/3956491326 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1c1953b0 con 0x7f1f1c0ff790 2024-04-23T02:05:24.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.647+0000 7f1f210a1700 1 -- 172.21.15.192:0/3956491326 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f1c18e270 con 0x7f1f1c0ff790 2024-04-23T02:05:24.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.648+0000 7f1f03fff700 1 -- 172.21.15.192:0/3956491326 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f1f0c00b960 con 0x7f1f1c0ff790 2024-04-23T02:05:24.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.648+0000 7f1f03fff700 1 --2- 172.21.15.192:0/3956491326 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f1f040383c0 0x7f1f0403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:24.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.648+0000 7f1f03fff700 1 -- 172.21.15.192:0/3956491326 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1f0c04c380 con 0x7f1f1c0ff790 2024-04-23T02:05:24.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.648+0000 7f1f1a59c700 1 --2- 172.21.15.192:0/3956491326 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f1f040383c0 0x7f1f0403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:24.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.648+0000 7f1f1a59c700 1 --2- 172.21.15.192:0/3956491326 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f1f040383c0 0x7f1f0403a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1f10006fd0 tx=0x7f1f10006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:24.656 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.653+0000 7f1f03fff700 1 -- 172.21.15.192:0/3956491326 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1f0c00f460 con 0x7f1f1c0ff790 2024-04-23T02:05:24.916 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.913+0000 7f1f210a1700 1 -- 172.21.15.192:0/3956491326 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1f1c02cc70 con 0x7f1f1c0ff790 2024-04-23T02:05:24.917 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.914+0000 7f1f03fff700 1 -- 172.21.15.192:0/3956491326 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f1f0c027020 con 0x7f1f1c0ff790 2024-04-23T02:05:24.917 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:05:24.917 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","modified":"2024-04-23T02:03:12.002793Z","created":"2024-04-23T02:03:12.002793Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi071","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T02:05:24.919 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.916+0000 7f1f210a1700 1 -- 172.21.15.192:0/3956491326 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f1f040383c0 msgr2=0x7f1f0403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:24.919 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.916+0000 7f1f210a1700 1 --2- 172.21.15.192:0/3956491326 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f1f040383c0 0x7f1f0403a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1f10006fd0 tx=0x7f1f10006e40 comp rx=0 tx=0).stop 2024-04-23T02:05:24.919 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.916+0000 7f1f210a1700 1 -- 172.21.15.192:0/3956491326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1c0ff790 msgr2=0x7f1f1c1947d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:24.919 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.916+0000 7f1f210a1700 1 --2- 172.21.15.192:0/3956491326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1c0ff790 0x7f1f1c1947d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c006010 tx=0x7f1f0c0057b0 comp rx=0 tx=0).stop 2024-04-23T02:05:24.919 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.916+0000 7f1f210a1700 1 -- 172.21.15.192:0/3956491326 shutdown_connections 2024-04-23T02:05:24.919 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.916+0000 7f1f210a1700 1 --2- 172.21.15.192:0/3956491326 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f1f040383c0 0x7f1f0403a880 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:24.920 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.916+0000 7f1f210a1700 1 --2- 172.21.15.192:0/3956491326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1c0ff790 0x7f1f1c1947d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:24.920 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.916+0000 7f1f210a1700 1 -- 172.21.15.192:0/3956491326 >> 172.21.15.192:0/3956491326 conn(0x7f1f1c0fad30 msgr2=0x7f1f1c0fba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:24.920 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.916+0000 7f1f210a1700 1 -- 172.21.15.192:0/3956491326 shutdown_connections 2024-04-23T02:05:24.920 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:24.916+0000 7f1f210a1700 1 -- 172.21.15.192:0/3956491326 wait complete. 2024-04-23T02:05:24.920 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-23T02:05:24.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:24 smithi071 ceph-mon[39837]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:05:25.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:25 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/3956491326' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T02:05:26.579 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-23T02:05:26.580 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T02:05:26.831 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:26.898 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:26.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:26 smithi071 ceph-mon[39837]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:05:28.611 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.607+0000 7f49027a1700 1 -- 172.21.15.192:0/1061016677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48fc100880 msgr2=0x7f48fc100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:28.611 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.607+0000 7f49027a1700 1 --2- 172.21.15.192:0/1061016677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48fc100880 0x7f48fc100ca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009910 tx=0x7f48ec009c20 comp rx=0 tx=0).stop 2024-04-23T02:05:28.611 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.607+0000 7f49027a1700 1 -- 172.21.15.192:0/1061016677 shutdown_connections 2024-04-23T02:05:28.612 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.607+0000 7f49027a1700 1 --2- 172.21.15.192:0/1061016677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48fc100880 0x7f48fc100ca0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:28.612 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.607+0000 7f49027a1700 1 -- 172.21.15.192:0/1061016677 >> 172.21.15.192:0/1061016677 conn(0x7f48fc0fbe00 msgr2=0x7f48fc0fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:28.612 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.607+0000 7f49027a1700 1 -- 172.21.15.192:0/1061016677 shutdown_connections 2024-04-23T02:05:28.612 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.607+0000 7f49027a1700 1 -- 172.21.15.192:0/1061016677 wait complete. 2024-04-23T02:05:28.612 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.608+0000 7f49027a1700 1 Processor -- start 2024-04-23T02:05:28.612 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.608+0000 7f49027a1700 1 -- start start 2024-04-23T02:05:28.612 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.608+0000 7f49027a1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48fc100880 0x7f48fc074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:28.613 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.608+0000 7f49027a1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc075070 con 0x7f48fc100880 2024-04-23T02:05:28.613 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.608+0000 7f48fbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48fc100880 0x7f48fc074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:28.613 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.608+0000 7f48fbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48fc100880 0x7f48fc074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.192:45982/0 (socket says 172.21.15.192:45982) 2024-04-23T02:05:28.613 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.608+0000 7f48fbfff700 1 -- 172.21.15.192:0/3318413212 learned_addr learned my addr 172.21.15.192:0/3318413212 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:05:28.613 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.609+0000 7f48fbfff700 1 -- 172.21.15.192:0/3318413212 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec0095f0 con 0x7f48fc100880 2024-04-23T02:05:28.616 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.609+0000 7f48fbfff700 1 --2- 172.21.15.192:0/3318413212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48fc100880 0x7f48fc074b30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f48ec006010 tx=0x7f48ec0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:28.616 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.609+0000 7f48f97fa700 1 -- 172.21.15.192:0/3318413212 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f48ec01d070 con 0x7f48fc100880 2024-04-23T02:05:28.616 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.609+0000 7f49027a1700 1 -- 172.21.15.192:0/3318413212 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48fc073180 con 0x7f48fc100880 2024-04-23T02:05:28.616 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.609+0000 7f48f97fa700 1 -- 172.21.15.192:0/3318413212 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f48ec00bdf0 con 0x7f48fc100880 2024-04-23T02:05:28.617 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.609+0000 7f49027a1700 1 -- 172.21.15.192:0/3318413212 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48fc073620 con 0x7f48fc100880 2024-04-23T02:05:28.617 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.609+0000 7f48f97fa700 1 -- 172.21.15.192:0/3318413212 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f48ec022e40 con 0x7f48fc100880 2024-04-23T02:05:28.617 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.610+0000 7f48f97fa700 1 -- 172.21.15.192:0/3318413212 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f48ec00f460 con 0x7f48fc100880 2024-04-23T02:05:28.617 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.610+0000 7f49027a1700 1 -- 172.21.15.192:0/3318413212 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48fc0623c0 con 0x7f48fc100880 2024-04-23T02:05:28.617 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.610+0000 7f48f97fa700 1 --2- 172.21.15.192:0/3318413212 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f48e4038410 0x7f48e403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:28.617 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.610+0000 7f48f97fa700 1 -- 172.21.15.192:0/3318413212 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f48ec04c340 con 0x7f48fc100880 2024-04-23T02:05:28.618 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.610+0000 7f48fb7fe700 1 --2- 172.21.15.192:0/3318413212 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f48e4038410 0x7f48e403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:28.618 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.611+0000 7f48fb7fe700 1 --2- 172.21.15.192:0/3318413212 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f48e4038410 0x7f48e403a8d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f48f0006fd0 tx=0x7f48f0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:28.618 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.615+0000 7f48f97fa700 1 -- 172.21.15.192:0/3318413212 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f48ec017440 con 0x7f48fc100880 2024-04-23T02:05:28.880 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.877+0000 7f49027a1700 1 -- 172.21.15.192:0/3318413212 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f48fc18ed40 con 0x7f48fc100880 2024-04-23T02:05:28.880 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.877+0000 7f48f97fa700 1 -- 172.21.15.192:0/3318413212 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f48ec027030 con 0x7f48fc100880 2024-04-23T02:05:28.881 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:05:28.881 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","modified":"2024-04-23T02:03:12.002793Z","created":"2024-04-23T02:03:12.002793Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi071","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T02:05:28.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.879+0000 7f49027a1700 1 -- 172.21.15.192:0/3318413212 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f48e4038410 msgr2=0x7f48e403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:28.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.879+0000 7f49027a1700 1 --2- 172.21.15.192:0/3318413212 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f48e4038410 0x7f48e403a8d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f48f0006fd0 tx=0x7f48f0006e40 comp rx=0 tx=0).stop 2024-04-23T02:05:28.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.879+0000 7f49027a1700 1 -- 172.21.15.192:0/3318413212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48fc100880 msgr2=0x7f48fc074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:28.883 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.879+0000 7f49027a1700 1 --2- 172.21.15.192:0/3318413212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48fc100880 0x7f48fc074b30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f48ec006010 tx=0x7f48ec0057b0 comp rx=0 tx=0).stop 2024-04-23T02:05:28.883 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.879+0000 7f49027a1700 1 -- 172.21.15.192:0/3318413212 shutdown_connections 2024-04-23T02:05:28.883 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.879+0000 7f49027a1700 1 --2- 172.21.15.192:0/3318413212 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f48e4038410 0x7f48e403a8d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:28.883 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.879+0000 7f49027a1700 1 --2- 172.21.15.192:0/3318413212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48fc100880 0x7f48fc074b30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:28.883 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.879+0000 7f49027a1700 1 -- 172.21.15.192:0/3318413212 >> 172.21.15.192:0/3318413212 conn(0x7f48fc0fbe00 msgr2=0x7f48fc0fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:28.883 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.879+0000 7f49027a1700 1 -- 172.21.15.192:0/3318413212 shutdown_connections 2024-04-23T02:05:28.883 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:28.879+0000 7f49027a1700 1 -- 172.21.15.192:0/3318413212 wait complete. 2024-04-23T02:05:28.883 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-23T02:05:28.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:28 smithi071 ceph-mon[39837]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:05:29.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:29 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/3318413212' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T02:05:30.566 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-23T02:05:30.567 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T02:05:30.816 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:30.870 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:30 smithi071 ceph-mon[39837]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:05:32.498 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.495+0000 7fc4490fd700 1 -- 172.21.15.192:0/372343899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4440ff780 msgr2=0x7fc4440ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:32.499 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.495+0000 7fc4490fd700 1 --2- 172.21.15.192:0/372343899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4440ff780 0x7fc4440ffba0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc438009910 tx=0x7fc438009c20 comp rx=0 tx=0).stop 2024-04-23T02:05:32.499 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.495+0000 7fc4490fd700 1 -- 172.21.15.192:0/372343899 shutdown_connections 2024-04-23T02:05:32.499 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.495+0000 7fc4490fd700 1 --2- 172.21.15.192:0/372343899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4440ff780 0x7fc4440ffba0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:32.499 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.495+0000 7fc4490fd700 1 -- 172.21.15.192:0/372343899 >> 172.21.15.192:0/372343899 conn(0x7fc4440fad20 msgr2=0x7fc4440fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:32.499 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.495+0000 7fc4490fd700 1 -- 172.21.15.192:0/372343899 shutdown_connections 2024-04-23T02:05:32.499 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.495+0000 7fc4490fd700 1 -- 172.21.15.192:0/372343899 wait complete. 2024-04-23T02:05:32.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.496+0000 7fc4490fd700 1 Processor -- start 2024-04-23T02:05:32.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.496+0000 7fc4490fd700 1 -- start start 2024-04-23T02:05:32.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.497+0000 7fc4490fd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4440ff780 0x7fc444068af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:32.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.497+0000 7fc4490fd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc444198650 con 0x7fc4440ff780 2024-04-23T02:05:32.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.497+0000 7fc442d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4440ff780 0x7fc444068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:32.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.497+0000 7fc442d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4440ff780 0x7fc444068af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.192:45994/0 (socket says 172.21.15.192:45994) 2024-04-23T02:05:32.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.497+0000 7fc442d9d700 1 -- 172.21.15.192:0/462131371 learned_addr learned my addr 172.21.15.192:0/462131371 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:05:32.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.497+0000 7fc442d9d700 1 -- 172.21.15.192:0/462131371 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4380095f0 con 0x7fc4440ff780 2024-04-23T02:05:32.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.497+0000 7fc442d9d700 1 --2- 172.21.15.192:0/462131371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4440ff780 0x7fc444068af0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc438006010 tx=0x7fc4380057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:32.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.497+0000 7fc42bfff700 1 -- 172.21.15.192:0/462131371 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc43801d070 con 0x7fc4440ff780 2024-04-23T02:05:32.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.497+0000 7fc4490fd700 1 -- 172.21.15.192:0/462131371 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc444198790 con 0x7fc4440ff780 2024-04-23T02:05:32.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.498+0000 7fc42bfff700 1 -- 172.21.15.192:0/462131371 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc43800bdf0 con 0x7fc4440ff780 2024-04-23T02:05:32.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.498+0000 7fc42bfff700 1 -- 172.21.15.192:0/462131371 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc438022e50 con 0x7fc4440ff780 2024-04-23T02:05:32.502 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.498+0000 7fc4490fd700 1 -- 172.21.15.192:0/462131371 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc444198c30 con 0x7fc4440ff780 2024-04-23T02:05:32.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.498+0000 7fc4490fd700 1 -- 172.21.15.192:0/462131371 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44418e260 con 0x7fc4440ff780 2024-04-23T02:05:32.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.498+0000 7fc42bfff700 1 -- 172.21.15.192:0/462131371 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fc43800f4b0 con 0x7fc4440ff780 2024-04-23T02:05:32.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.499+0000 7fc42bfff700 1 --2- 172.21.15.192:0/462131371 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc42c038370 0x7fc42c03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:32.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.499+0000 7fc42bfff700 1 -- 172.21.15.192:0/462131371 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc43804c400 con 0x7fc4440ff780 2024-04-23T02:05:32.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.499+0000 7fc44259c700 1 --2- 172.21.15.192:0/462131371 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc42c038370 0x7fc42c03a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:32.503 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.499+0000 7fc44259c700 1 --2- 172.21.15.192:0/462131371 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc42c038370 0x7fc42c03a830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc434006fd0 tx=0x7fc434006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:32.507 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.504+0000 7fc42bfff700 1 -- 172.21.15.192:0/462131371 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc438017440 con 0x7fc4440ff780 2024-04-23T02:05:32.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:32 smithi071 ceph-mon[39837]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:05:32.770 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.767+0000 7fc4490fd700 1 -- 172.21.15.192:0/462131371 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc44402cc70 con 0x7fc4440ff780 2024-04-23T02:05:32.771 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.768+0000 7fc42bfff700 1 -- 172.21.15.192:0/462131371 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fc438027030 con 0x7fc4440ff780 2024-04-23T02:05:32.771 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:05:32.771 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","modified":"2024-04-23T02:03:12.002793Z","created":"2024-04-23T02:03:12.002793Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi071","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T02:05:32.773 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.770+0000 7fc4490fd700 1 -- 172.21.15.192:0/462131371 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc42c038370 msgr2=0x7fc42c03a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:32.773 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.770+0000 7fc4490fd700 1 --2- 172.21.15.192:0/462131371 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc42c038370 0x7fc42c03a830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc434006fd0 tx=0x7fc434006e40 comp rx=0 tx=0).stop 2024-04-23T02:05:32.773 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.770+0000 7fc4490fd700 1 -- 172.21.15.192:0/462131371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4440ff780 msgr2=0x7fc444068af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:32.773 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.770+0000 7fc4490fd700 1 --2- 172.21.15.192:0/462131371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4440ff780 0x7fc444068af0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc438006010 tx=0x7fc4380057b0 comp rx=0 tx=0).stop 2024-04-23T02:05:32.774 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.770+0000 7fc4490fd700 1 -- 172.21.15.192:0/462131371 shutdown_connections 2024-04-23T02:05:32.774 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.770+0000 7fc4490fd700 1 --2- 172.21.15.192:0/462131371 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fc42c038370 0x7fc42c03a830 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:32.774 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.770+0000 7fc4490fd700 1 --2- 172.21.15.192:0/462131371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4440ff780 0x7fc444068af0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:32.774 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.770+0000 7fc4490fd700 1 -- 172.21.15.192:0/462131371 >> 172.21.15.192:0/462131371 conn(0x7fc4440fad20 msgr2=0x7fc4440fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:32.774 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.770+0000 7fc4490fd700 1 -- 172.21.15.192:0/462131371 shutdown_connections 2024-04-23T02:05:32.774 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:32.770+0000 7fc4490fd700 1 -- 172.21.15.192:0/462131371 wait complete. 2024-04-23T02:05:32.774 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-23T02:05:33.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:33 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/462131371' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T02:05:34.506 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-23T02:05:34.506 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T02:05:34.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:34 smithi071 ceph-mon[39837]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:05:34.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:34 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:34.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:34 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:34.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:34 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:34.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:34 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-23T02:05:34.759 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:34.827 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:36.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.636+0000 7fd39ba5f700 1 -- 172.21.15.192:0/2501672034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3941008a0 msgr2=0x7fd394100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:36.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.636+0000 7fd39ba5f700 1 --2- 172.21.15.192:0/2501672034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3941008a0 0x7fd394100cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd384009910 tx=0x7fd384009c20 comp rx=0 tx=0).stop 2024-04-23T02:05:36.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.636+0000 7fd39ba5f700 1 -- 172.21.15.192:0/2501672034 shutdown_connections 2024-04-23T02:05:36.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.636+0000 7fd39ba5f700 1 --2- 172.21.15.192:0/2501672034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3941008a0 0x7fd394100cc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:36.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.636+0000 7fd39ba5f700 1 -- 172.21.15.192:0/2501672034 >> 172.21.15.192:0/2501672034 conn(0x7fd3940fbe20 msgr2=0x7fd3940fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:36.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.636+0000 7fd39ba5f700 1 -- 172.21.15.192:0/2501672034 shutdown_connections 2024-04-23T02:05:36.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.636+0000 7fd39ba5f700 1 -- 172.21.15.192:0/2501672034 wait complete. 2024-04-23T02:05:36.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.637+0000 7fd39ba5f700 1 Processor -- start 2024-04-23T02:05:36.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.637+0000 7fd39ba5f700 1 -- start start 2024-04-23T02:05:36.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.637+0000 7fd39ba5f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3941008a0 0x7fd394195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:36.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.637+0000 7fd39ba5f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd394195e40 con 0x7fd3941008a0 2024-04-23T02:05:36.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.637+0000 7fd3997fb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3941008a0 0x7fd394195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:36.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.637+0000 7fd3997fb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3941008a0 0x7fd394195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.192:46006/0 (socket says 172.21.15.192:46006) 2024-04-23T02:05:36.675 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.637+0000 7fd3997fb700 1 -- 172.21.15.192:0/2024635317 learned_addr learned my addr 172.21.15.192:0/2024635317 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:05:36.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.637+0000 7fd3997fb700 1 -- 172.21.15.192:0/2024635317 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3840095f0 con 0x7fd3941008a0 2024-04-23T02:05:36.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.637+0000 7fd3997fb700 1 --2- 172.21.15.192:0/2024635317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3941008a0 0x7fd394195900 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd384006010 tx=0x7fd3840057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:36.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.638+0000 7fd38a7fc700 1 -- 172.21.15.192:0/2024635317 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd38401d070 con 0x7fd3941008a0 2024-04-23T02:05:36.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.638+0000 7fd38a7fc700 1 -- 172.21.15.192:0/2024635317 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd38400bdf0 con 0x7fd3941008a0 2024-04-23T02:05:36.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.638+0000 7fd39ba5f700 1 -- 172.21.15.192:0/2024635317 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd394196040 con 0x7fd3941008a0 2024-04-23T02:05:36.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.638+0000 7fd38a7fc700 1 -- 172.21.15.192:0/2024635317 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd384022e50 con 0x7fd3941008a0 2024-04-23T02:05:36.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.638+0000 7fd39ba5f700 1 -- 172.21.15.192:0/2024635317 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3941964e0 con 0x7fd3941008a0 2024-04-23T02:05:36.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.639+0000 7fd38a7fc700 1 -- 172.21.15.192:0/2024635317 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 14) v1 ==== 45353+0+0 (secure 0 0 0) 0x7fd38400b960 con 0x7fd3941008a0 2024-04-23T02:05:36.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.639+0000 7fd39ba5f700 1 -- 172.21.15.192:0/2024635317 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd39418f380 con 0x7fd3941008a0 2024-04-23T02:05:36.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.639+0000 7fd38a7fc700 1 --2- 172.21.15.192:0/2024635317 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fd3800383c0 0x7fd38003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:36.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.639+0000 7fd38a7fc700 1 -- 172.21.15.192:0/2024635317 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd38404c370 con 0x7fd3941008a0 2024-04-23T02:05:36.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.639+0000 7fd398ffa700 1 -- 172.21.15.192:0/2024635317 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fd3800383c0 msgr2=0x7fd38003a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/4165305842 2024-04-23T02:05:36.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.639+0000 7fd398ffa700 1 --2- 172.21.15.192:0/2024635317 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fd3800383c0 0x7fd38003a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T02:05:36.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.644+0000 7fd38a7fc700 1 -- 172.21.15.192:0/2024635317 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd38402acf0 con 0x7fd3941008a0 2024-04-23T02:05:36.843 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.840+0000 7fd398ffa700 1 -- 172.21.15.192:0/2024635317 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fd3800383c0 msgr2=0x7fd38003a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/4165305842 2024-04-23T02:05:36.843 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.840+0000 7fd398ffa700 1 --2- 172.21.15.192:0/2024635317 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fd3800383c0 0x7fd38003a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T02:05:36.857 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:36 smithi071 ceph-mon[39837]: from='mgr.14162 172.21.15.71:0/1299705819' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-23T02:05:36.857 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:36 smithi071 ceph-mon[39837]: mgrmap e14: smithi071.ukdjvn(active, since 72s) 2024-04-23T02:05:36.909 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.906+0000 7fd39ba5f700 1 -- 172.21.15.192:0/2024635317 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd39402cc70 con 0x7fd3941008a0 2024-04-23T02:05:36.910 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.907+0000 7fd38a7fc700 1 -- 172.21.15.192:0/2024635317 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fd384027020 con 0x7fd3941008a0 2024-04-23T02:05:36.910 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:05:36.910 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","modified":"2024-04-23T02:03:12.002793Z","created":"2024-04-23T02:03:12.002793Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi071","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T02:05:36.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.909+0000 7fd39ba5f700 1 -- 172.21.15.192:0/2024635317 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fd3800383c0 msgr2=0x7fd38003a880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T02:05:36.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.909+0000 7fd39ba5f700 1 --2- 172.21.15.192:0/2024635317 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fd3800383c0 0x7fd38003a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:36.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.909+0000 7fd39ba5f700 1 -- 172.21.15.192:0/2024635317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3941008a0 msgr2=0x7fd394195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:36.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.909+0000 7fd39ba5f700 1 --2- 172.21.15.192:0/2024635317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3941008a0 0x7fd394195900 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd384006010 tx=0x7fd3840057b0 comp rx=0 tx=0).stop 2024-04-23T02:05:36.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.909+0000 7fd39ba5f700 1 -- 172.21.15.192:0/2024635317 shutdown_connections 2024-04-23T02:05:36.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.909+0000 7fd39ba5f700 1 --2- 172.21.15.192:0/2024635317 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7fd3800383c0 0x7fd38003a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:36.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.909+0000 7fd39ba5f700 1 --2- 172.21.15.192:0/2024635317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3941008a0 0x7fd394195900 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:36.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.909+0000 7fd39ba5f700 1 -- 172.21.15.192:0/2024635317 >> 172.21.15.192:0/2024635317 conn(0x7fd3940fbe20 msgr2=0x7fd3940fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:36.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.909+0000 7fd39ba5f700 1 -- 172.21.15.192:0/2024635317 shutdown_connections 2024-04-23T02:05:36.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:36.909+0000 7fd39ba5f700 1 -- 172.21.15.192:0/2024635317 wait complete. 2024-04-23T02:05:36.913 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-23T02:05:37.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:37 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/2024635317' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T02:05:38.539 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-23T02:05:38.539 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T02:05:38.793 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:38.853 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:40.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.547+0000 7f7647c47700 1 -- 172.21.15.192:0/2216239029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76400feb50 msgr2=0x7f76400fef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:40.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.547+0000 7f7647c47700 1 --2- 172.21.15.192:0/2216239029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76400feb50 0x7f76400fef70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7630009910 tx=0x7f7630009c20 comp rx=0 tx=0).stop 2024-04-23T02:05:40.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.547+0000 7f7647c47700 1 -- 172.21.15.192:0/2216239029 shutdown_connections 2024-04-23T02:05:40.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.547+0000 7f7647c47700 1 --2- 172.21.15.192:0/2216239029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76400feb50 0x7f76400fef70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:40.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.547+0000 7f7647c47700 1 -- 172.21.15.192:0/2216239029 >> 172.21.15.192:0/2216239029 conn(0x7f76400fa490 msgr2=0x7f76400fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:40.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.547+0000 7f7647c47700 1 -- 172.21.15.192:0/2216239029 shutdown_connections 2024-04-23T02:05:40.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.547+0000 7f7647c47700 1 -- 172.21.15.192:0/2216239029 wait complete. 2024-04-23T02:05:40.554 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.548+0000 7f7647c47700 1 Processor -- start 2024-04-23T02:05:40.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.548+0000 7f7647c47700 1 -- start start 2024-04-23T02:05:40.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.548+0000 7f7647c47700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76400feb50 0x7f76401958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:40.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.548+0000 7f7647c47700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7640195e30 con 0x7f76400feb50 2024-04-23T02:05:40.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.548+0000 7f76459e3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76400feb50 0x7f76401958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:40.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.548+0000 7f76459e3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76400feb50 0x7f76401958f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.192:34496/0 (socket says 172.21.15.192:34496) 2024-04-23T02:05:40.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.548+0000 7f76459e3700 1 -- 172.21.15.192:0/3149536169 learned_addr learned my addr 172.21.15.192:0/3149536169 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:05:40.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.548+0000 7f76459e3700 1 -- 172.21.15.192:0/3149536169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76300095f0 con 0x7f76400feb50 2024-04-23T02:05:40.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.549+0000 7f76459e3700 1 --2- 172.21.15.192:0/3149536169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76400feb50 0x7f76401958f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7630006010 tx=0x7f76300057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:40.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.549+0000 7f7636ffd700 1 -- 172.21.15.192:0/3149536169 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f763001d070 con 0x7f76400feb50 2024-04-23T02:05:40.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.549+0000 7f7647c47700 1 -- 172.21.15.192:0/3149536169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7640196030 con 0x7f76400feb50 2024-04-23T02:05:40.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.549+0000 7f7636ffd700 1 -- 172.21.15.192:0/3149536169 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f763000bdf0 con 0x7f76400feb50 2024-04-23T02:05:40.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.549+0000 7f7636ffd700 1 -- 172.21.15.192:0/3149536169 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7630022e50 con 0x7f76400feb50 2024-04-23T02:05:40.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.549+0000 7f7647c47700 1 -- 172.21.15.192:0/3149536169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76401964d0 con 0x7f76400feb50 2024-04-23T02:05:40.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.549+0000 7f7636ffd700 1 -- 172.21.15.192:0/3149536169 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 14) v1 ==== 45353+0+0 (secure 0 0 0) 0x7f763000f4b0 con 0x7f76400feb50 2024-04-23T02:05:40.555 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.550+0000 7f7647c47700 1 -- 172.21.15.192:0/3149536169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764018f380 con 0x7f76400feb50 2024-04-23T02:05:40.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.550+0000 7f7636ffd700 1 --2- 172.21.15.192:0/3149536169 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f762c0383c0 0x7f762c03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:40.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.550+0000 7f7636ffd700 1 -- 172.21.15.192:0/3149536169 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f763004c3f0 con 0x7f76400feb50 2024-04-23T02:05:40.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.550+0000 7f76451e2700 1 -- 172.21.15.192:0/3149536169 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f762c0383c0 msgr2=0x7f762c03a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/4165305842 2024-04-23T02:05:40.556 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.550+0000 7f76451e2700 1 --2- 172.21.15.192:0/3149536169 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f762c0383c0 0x7f762c03a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T02:05:40.557 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.555+0000 7f7636ffd700 1 -- 172.21.15.192:0/3149536169 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f763002a770 con 0x7f76400feb50 2024-04-23T02:05:40.754 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.751+0000 7f76451e2700 1 -- 172.21.15.192:0/3149536169 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f762c0383c0 msgr2=0x7f762c03a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/4165305842 2024-04-23T02:05:40.754 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.751+0000 7f76451e2700 1 --2- 172.21.15.192:0/3149536169 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f762c0383c0 0x7f762c03a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T02:05:40.818 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.815+0000 7f7647c47700 1 -- 172.21.15.192:0/3149536169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f764002cc70 con 0x7f76400feb50 2024-04-23T02:05:40.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.816+0000 7f7636ffd700 1 -- 172.21.15.192:0/3149536169 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f7630027030 con 0x7f76400feb50 2024-04-23T02:05:40.819 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:05:40.819 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","modified":"2024-04-23T02:03:12.002793Z","created":"2024-04-23T02:03:12.002793Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi071","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T02:05:40.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.818+0000 7f7647c47700 1 -- 172.21.15.192:0/3149536169 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f762c0383c0 msgr2=0x7f762c03a880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T02:05:40.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.818+0000 7f7647c47700 1 --2- 172.21.15.192:0/3149536169 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f762c0383c0 0x7f762c03a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:40.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.818+0000 7f7647c47700 1 -- 172.21.15.192:0/3149536169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76400feb50 msgr2=0x7f76401958f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:40.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.818+0000 7f7647c47700 1 --2- 172.21.15.192:0/3149536169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76400feb50 0x7f76401958f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7630006010 tx=0x7f76300057b0 comp rx=0 tx=0).stop 2024-04-23T02:05:40.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.818+0000 7f7647c47700 1 -- 172.21.15.192:0/3149536169 shutdown_connections 2024-04-23T02:05:40.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.818+0000 7f7647c47700 1 --2- 172.21.15.192:0/3149536169 >> [v2:172.21.15.71:6800/4165305842,v1:172.21.15.71:6801/4165305842] conn(0x7f762c0383c0 0x7f762c03a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:40.822 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.818+0000 7f7647c47700 1 --2- 172.21.15.192:0/3149536169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76400feb50 0x7f76401958f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:40.822 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.818+0000 7f7647c47700 1 -- 172.21.15.192:0/3149536169 >> 172.21.15.192:0/3149536169 conn(0x7f76400fa490 msgr2=0x7f76400fb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:40.822 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.818+0000 7f7647c47700 1 -- 172.21.15.192:0/3149536169 shutdown_connections 2024-04-23T02:05:40.822 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:40.818+0000 7f7647c47700 1 -- 172.21.15.192:0/3149536169 wait complete. 2024-04-23T02:05:40.822 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-23T02:05:41.968 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:41 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/3149536169' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T02:05:42.524 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-23T02:05:42.525 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T02:05:42.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:42 smithi071 ceph-mon[39837]: Active manager daemon smithi071.ukdjvn restarted 2024-04-23T02:05:42.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:42 smithi071 ceph-mon[39837]: Activating manager daemon smithi071.ukdjvn 2024-04-23T02:05:42.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:42 smithi071 ceph-mon[39837]: osdmap e5: 0 total, 0 up, 0 in 2024-04-23T02:05:42.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:42 smithi071 ceph-mon[39837]: mgrmap e15: smithi071.ukdjvn(active, starting, since 0.00320921s) 2024-04-23T02:05:42.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi071"}]: dispatch 2024-04-23T02:05:42.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr metadata", "who": "smithi071.ukdjvn", "id": "smithi071.ukdjvn"}]: dispatch 2024-04-23T02:05:42.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T02:05:42.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T02:05:42.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T02:05:42.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:42 smithi071 ceph-mon[39837]: Manager daemon smithi071.ukdjvn is now available 2024-04-23T02:05:42.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:05:42.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:05:42.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:42.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:05:42.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi071.ukdjvn/mirror_snapshot_schedule"}]: dispatch 2024-04-23T02:05:42.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi071.ukdjvn/trash_purge_schedule"}]: dispatch 2024-04-23T02:05:42.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:42 smithi071 ceph-mon[39837]: [23/Apr/2024:02:05:42] ENGINE Bus STARTING 2024-04-23T02:05:42.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:42 smithi071 ceph-mon[39837]: [23/Apr/2024:02:05:42] ENGINE Serving on https://172.21.15.71:7150 2024-04-23T02:05:42.778 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:42.838 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:43.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:43 smithi071 ceph-mon[39837]: [23/Apr/2024:02:05:42] ENGINE Serving on http://172.21.15.71:8765 2024-04-23T02:05:43.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:43 smithi071 ceph-mon[39837]: [23/Apr/2024:02:05:42] ENGINE Bus STARTED 2024-04-23T02:05:43.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:43 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:43.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:43 smithi071 ceph-mon[39837]: mgrmap e16: smithi071.ukdjvn(active, since 1.00782s) 2024-04-23T02:05:44.638 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.634+0000 7f4ca6efa700 1 -- 172.21.15.192:0/382867878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ca00feb50 msgr2=0x7f4ca00fef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:44.639 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.634+0000 7f4ca6efa700 1 --2- 172.21.15.192:0/382867878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ca00feb50 0x7f4ca00fef70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009910 tx=0x7f4c94009c20 comp rx=0 tx=0).stop 2024-04-23T02:05:44.639 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.634+0000 7f4ca6efa700 1 -- 172.21.15.192:0/382867878 shutdown_connections 2024-04-23T02:05:44.639 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.634+0000 7f4ca6efa700 1 --2- 172.21.15.192:0/382867878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ca00feb50 0x7f4ca00fef70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:44.639 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.634+0000 7f4ca6efa700 1 -- 172.21.15.192:0/382867878 >> 172.21.15.192:0/382867878 conn(0x7f4ca00fa490 msgr2=0x7f4ca00fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:44.639 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.634+0000 7f4ca6efa700 1 -- 172.21.15.192:0/382867878 shutdown_connections 2024-04-23T02:05:44.639 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.634+0000 7f4ca6efa700 1 -- 172.21.15.192:0/382867878 wait complete. 2024-04-23T02:05:44.639 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.635+0000 7f4ca6efa700 1 Processor -- start 2024-04-23T02:05:44.639 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.635+0000 7f4ca6efa700 1 -- start start 2024-04-23T02:05:44.639 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.635+0000 7f4ca6efa700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ca00feb50 0x7f4ca01958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:44.639 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.635+0000 7f4ca6efa700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca0195e30 con 0x7f4ca00feb50 2024-04-23T02:05:44.639 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.635+0000 7f4ca4c96700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ca00feb50 0x7f4ca01958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:44.639 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.635+0000 7f4ca4c96700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ca00feb50 0x7f4ca01958f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.192:34504/0 (socket says 172.21.15.192:34504) 2024-04-23T02:05:44.639 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.635+0000 7f4ca4c96700 1 -- 172.21.15.192:0/53861945 learned_addr learned my addr 172.21.15.192:0/53861945 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:05:44.639 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.635+0000 7f4ca4c96700 1 -- 172.21.15.192:0/53861945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c940095f0 con 0x7f4ca00feb50 2024-04-23T02:05:44.640 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.636+0000 7f4ca4c96700 1 --2- 172.21.15.192:0/53861945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ca00feb50 0x7f4ca01958f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4c94006010 tx=0x7f4c940057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:44.641 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.636+0000 7f4c9dffb700 1 -- 172.21.15.192:0/53861945 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c9401d070 con 0x7f4ca00feb50 2024-04-23T02:05:44.641 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.636+0000 7f4ca6efa700 1 -- 172.21.15.192:0/53861945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca0196030 con 0x7f4ca00feb50 2024-04-23T02:05:44.641 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.636+0000 7f4c9dffb700 1 -- 172.21.15.192:0/53861945 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4c9400bdf0 con 0x7f4ca00feb50 2024-04-23T02:05:44.641 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.636+0000 7f4c9dffb700 1 -- 172.21.15.192:0/53861945 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c94022e50 con 0x7f4ca00feb50 2024-04-23T02:05:44.641 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.636+0000 7f4ca6efa700 1 -- 172.21.15.192:0/53861945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca01964d0 con 0x7f4ca00feb50 2024-04-23T02:05:44.641 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.637+0000 7f4ca6efa700 1 -- 172.21.15.192:0/53861945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca018f380 con 0x7f4ca00feb50 2024-04-23T02:05:44.641 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.637+0000 7f4c9dffb700 1 -- 172.21.15.192:0/53861945 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 45204+0+0 (secure 0 0 0) 0x7f4c9400f4b0 con 0x7f4ca00feb50 2024-04-23T02:05:44.641 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.637+0000 7f4c9dffb700 1 --2- 172.21.15.192:0/53861945 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4c90038200 0x7f4c9003a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:44.641 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.637+0000 7f4c9dffb700 1 -- 172.21.15.192:0/53861945 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f4c9404c070 con 0x7f4ca00feb50 2024-04-23T02:05:44.641 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.637+0000 7f4c9ffff700 1 --2- 172.21.15.192:0/53861945 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4c90038200 0x7f4c9003a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:44.642 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.638+0000 7f4c9ffff700 1 --2- 172.21.15.192:0/53861945 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4c90038200 0x7f4c9003a6c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c006fd0 tx=0x7f4c8c006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:44.645 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.642+0000 7f4c9dffb700 1 -- 172.21.15.192:0/53861945 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4c9402a770 con 0x7f4ca00feb50 2024-04-23T02:05:44.923 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.920+0000 7f4ca6efa700 1 -- 172.21.15.192:0/53861945 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4ca002cc70 con 0x7f4ca00feb50 2024-04-23T02:05:44.924 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.921+0000 7f4c9dffb700 1 -- 172.21.15.192:0/53861945 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f4c94027020 con 0x7f4ca00feb50 2024-04-23T02:05:44.924 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:05:44.924 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","modified":"2024-04-23T02:03:12.002793Z","created":"2024-04-23T02:03:12.002793Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi071","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T02:05:44.927 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.923+0000 7f4ca6efa700 1 -- 172.21.15.192:0/53861945 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4c90038200 msgr2=0x7f4c9003a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:44.927 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.923+0000 7f4ca6efa700 1 --2- 172.21.15.192:0/53861945 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4c90038200 0x7f4c9003a6c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c006fd0 tx=0x7f4c8c006e40 comp rx=0 tx=0).stop 2024-04-23T02:05:44.927 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.923+0000 7f4ca6efa700 1 -- 172.21.15.192:0/53861945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ca00feb50 msgr2=0x7f4ca01958f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:44.927 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.923+0000 7f4ca6efa700 1 --2- 172.21.15.192:0/53861945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ca00feb50 0x7f4ca01958f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4c94006010 tx=0x7f4c940057b0 comp rx=0 tx=0).stop 2024-04-23T02:05:44.927 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.923+0000 7f4ca6efa700 1 -- 172.21.15.192:0/53861945 shutdown_connections 2024-04-23T02:05:44.927 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.923+0000 7f4ca6efa700 1 --2- 172.21.15.192:0/53861945 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4c90038200 0x7f4c9003a6c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:44.927 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.923+0000 7f4ca6efa700 1 --2- 172.21.15.192:0/53861945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ca00feb50 0x7f4ca01958f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:44.927 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.923+0000 7f4ca6efa700 1 -- 172.21.15.192:0/53861945 >> 172.21.15.192:0/53861945 conn(0x7f4ca00fa490 msgr2=0x7f4ca00fb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:44.927 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.924+0000 7f4ca6efa700 1 -- 172.21.15.192:0/53861945 shutdown_connections 2024-04-23T02:05:44.927 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:44.924+0000 7f4ca6efa700 1 -- 172.21.15.192:0/53861945 wait complete. 2024-04-23T02:05:44.928 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-23T02:05:45.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:45 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:45.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:45 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:45.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:45 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/53861945' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T02:05:46.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:46 smithi071 ceph-mon[39837]: mgrmap e17: smithi071.ukdjvn(active, since 3s) 2024-04-23T02:05:46.642 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-23T02:05:46.642 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T02:05:47.533 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:47.584 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-23T02:05:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:47 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:47 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:47 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:47 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:47.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:47 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:05:50.147 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.140+0000 7f1d863b6700 1 -- 172.21.15.192:0/874615874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d80106820 msgr2=0x7f1d80106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:50.147 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.140+0000 7f1d863b6700 1 --2- 172.21.15.192:0/874615874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d80106820 0x7f1d80106c00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009910 tx=0x7f1d70009c20 comp rx=0 tx=0).stop 2024-04-23T02:05:50.147 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.140+0000 7f1d863b6700 1 -- 172.21.15.192:0/874615874 shutdown_connections 2024-04-23T02:05:50.147 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.140+0000 7f1d863b6700 1 --2- 172.21.15.192:0/874615874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d80106820 0x7f1d80106c00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:50.148 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.140+0000 7f1d863b6700 1 -- 172.21.15.192:0/874615874 >> 172.21.15.192:0/874615874 conn(0x7f1d80075290 msgr2=0x7f1d800756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:50.148 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.140+0000 7f1d863b6700 1 -- 172.21.15.192:0/874615874 shutdown_connections 2024-04-23T02:05:50.148 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.140+0000 7f1d863b6700 1 -- 172.21.15.192:0/874615874 wait complete. 2024-04-23T02:05:50.148 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.141+0000 7f1d863b6700 1 Processor -- start 2024-04-23T02:05:50.148 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.141+0000 7f1d863b6700 1 -- start start 2024-04-23T02:05:50.148 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.141+0000 7f1d863b6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d80106820 0x7f1d800ff800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:50.148 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.141+0000 7f1d863b6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d80101780 con 0x7f1d80106820 2024-04-23T02:05:50.148 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.141+0000 7f1d7ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d80106820 0x7f1d800ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:50.148 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.141+0000 7f1d7ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d80106820 0x7f1d800ff800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.192:37906/0 (socket says 172.21.15.192:37906) 2024-04-23T02:05:50.148 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.141+0000 7f1d7ffff700 1 -- 172.21.15.192:0/3157257701 learned_addr learned my addr 172.21.15.192:0/3157257701 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:05:50.148 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.142+0000 7f1d7ffff700 1 -- 172.21.15.192:0/3157257701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d700095f0 con 0x7f1d80106820 2024-04-23T02:05:50.148 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.142+0000 7f1d7ffff700 1 --2- 172.21.15.192:0/3157257701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d80106820 0x7f1d800ff800 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1d70004f40 tx=0x7f1d70005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:50.148 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.142+0000 7f1d7d7fa700 1 -- 172.21.15.192:0/3157257701 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d7001d070 con 0x7f1d80106820 2024-04-23T02:05:50.148 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.142+0000 7f1d7d7fa700 1 -- 172.21.15.192:0/3157257701 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1d70009e60 con 0x7f1d80106820 2024-04-23T02:05:50.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.142+0000 7f1d863b6700 1 -- 172.21.15.192:0/3157257701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d800ffd40 con 0x7f1d80106820 2024-04-23T02:05:50.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.142+0000 7f1d7d7fa700 1 -- 172.21.15.192:0/3157257701 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d70022ce0 con 0x7f1d80106820 2024-04-23T02:05:50.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.142+0000 7f1d863b6700 1 -- 172.21.15.192:0/3157257701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d801001c0 con 0x7f1d80106820 2024-04-23T02:05:50.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.143+0000 7f1d7d7fa700 1 -- 172.21.15.192:0/3157257701 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7f1d7000f460 con 0x7f1d80106820 2024-04-23T02:05:50.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.143+0000 7f1d7d7fa700 1 --2- 172.21.15.192:0/3157257701 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1d68038500 0x7f1d6803a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:50.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.143+0000 7f1d863b6700 1 -- 172.21.15.192:0/3157257701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d80105ad0 con 0x7f1d80106820 2024-04-23T02:05:50.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.143+0000 7f1d7d7fa700 1 -- 172.21.15.192:0/3157257701 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f1d7004c560 con 0x7f1d80106820 2024-04-23T02:05:50.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.143+0000 7f1d7f7fe700 1 --2- 172.21.15.192:0/3157257701 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1d68038500 0x7f1d6803a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:50.150 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.144+0000 7f1d7f7fe700 1 --2- 172.21.15.192:0/3157257701 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1d68038500 0x7f1d6803a9c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1d74006fd0 tx=0x7f1d74006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:50.152 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.148+0000 7f1d7d7fa700 1 -- 172.21.15.192:0/3157257701 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1d70027070 con 0x7f1d80106820 2024-04-23T02:05:50.433 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.429+0000 7f1d863b6700 1 -- 172.21.15.192:0/3157257701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1d801004c0 con 0x7f1d80106820 2024-04-23T02:05:50.433 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.429+0000 7f1d7d7fa700 1 -- 172.21.15.192:0/3157257701 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f1d700209c0 con 0x7f1d80106820 2024-04-23T02:05:50.433 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:05:50.434 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","modified":"2024-04-23T02:03:12.002793Z","created":"2024-04-23T02:03:12.002793Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi071","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T02:05:50.436 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.432+0000 7f1d863b6700 1 -- 172.21.15.192:0/3157257701 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1d68038500 msgr2=0x7f1d6803a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:50.436 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.432+0000 7f1d863b6700 1 --2- 172.21.15.192:0/3157257701 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1d68038500 0x7f1d6803a9c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1d74006fd0 tx=0x7f1d74006e40 comp rx=0 tx=0).stop 2024-04-23T02:05:50.436 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.432+0000 7f1d863b6700 1 -- 172.21.15.192:0/3157257701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d80106820 msgr2=0x7f1d800ff800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:50.436 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.432+0000 7f1d863b6700 1 --2- 172.21.15.192:0/3157257701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d80106820 0x7f1d800ff800 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1d70004f40 tx=0x7f1d70005020 comp rx=0 tx=0).stop 2024-04-23T02:05:50.436 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.432+0000 7f1d863b6700 1 -- 172.21.15.192:0/3157257701 shutdown_connections 2024-04-23T02:05:50.436 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.432+0000 7f1d863b6700 1 --2- 172.21.15.192:0/3157257701 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1d68038500 0x7f1d6803a9c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:50.436 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.432+0000 7f1d863b6700 1 --2- 172.21.15.192:0/3157257701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d80106820 0x7f1d800ff800 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:50.436 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.432+0000 7f1d863b6700 1 -- 172.21.15.192:0/3157257701 >> 172.21.15.192:0/3157257701 conn(0x7f1d80075290 msgr2=0x7f1d800fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:50.436 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.432+0000 7f1d863b6700 1 -- 172.21.15.192:0/3157257701 shutdown_connections 2024-04-23T02:05:50.436 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:50.432+0000 7f1d863b6700 1 -- 172.21.15.192:0/3157257701 wait complete. 2024-04-23T02:05:50.437 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-23T02:05:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:50 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:50 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:50 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:50 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:50 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:05:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:50 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:05:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:50 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:05:50.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:50 smithi071 ceph-mon[39837]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-23T02:05:50.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:50 smithi071 ceph-mon[39837]: Updating smithi192:/etc/ceph/ceph.conf 2024-04-23T02:05:50.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:50 smithi071 ceph-mon[39837]: Updating smithi071:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T02:05:50.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:50 smithi071 ceph-mon[39837]: Updating smithi192:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T02:05:51.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:51 smithi071 ceph-mon[39837]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-23T02:05:51.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:51 smithi071 ceph-mon[39837]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-04-23T02:05:51.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:51 smithi071 ceph-mon[39837]: Updating smithi071:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T02:05:51.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:51 smithi071 ceph-mon[39837]: Updating smithi192:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T02:05:51.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:51 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:51.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:51 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:51.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:51 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:51.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:51 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:51.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:51 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:51.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:51 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi192", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-23T02:05:51.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:51 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi192", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-23T02:05:51.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:51 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:05:51.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:51 smithi071 ceph-mon[39837]: Deploying daemon ceph-exporter.smithi192 on smithi192 2024-04-23T02:05:51.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:51 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/3157257701' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T02:05:52.294 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-23T02:05:52.294 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T02:05:52.565 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T02:05:55.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:54 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:55.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:54 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:55.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:54 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:55.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:54 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:55.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:54 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-23T02:05:55.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:54 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-23T02:05:55.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:54 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:05:55.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:54 smithi071 ceph-mon[39837]: Deploying daemon crash.smithi192 on smithi192 2024-04-23T02:05:55.298 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.163+0000 7fc8f18cf700 1 -- 172.21.15.192:0/2422151494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8ec106820 msgr2=0x7fc8ec106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:55.298 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.163+0000 7fc8f18cf700 1 --2- 172.21.15.192:0/2422151494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8ec106820 0x7fc8ec106c00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009910 tx=0x7fc8d4009c20 comp rx=0 tx=0).stop 2024-04-23T02:05:55.298 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.163+0000 7fc8f18cf700 1 -- 172.21.15.192:0/2422151494 shutdown_connections 2024-04-23T02:05:55.298 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.163+0000 7fc8f18cf700 1 --2- 172.21.15.192:0/2422151494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8ec106820 0x7fc8ec106c00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:55.298 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.163+0000 7fc8f18cf700 1 -- 172.21.15.192:0/2422151494 >> 172.21.15.192:0/2422151494 conn(0x7fc8ec0fc350 msgr2=0x7fc8ec0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:55.299 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.163+0000 7fc8f18cf700 1 -- 172.21.15.192:0/2422151494 shutdown_connections 2024-04-23T02:05:55.299 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.163+0000 7fc8f18cf700 1 -- 172.21.15.192:0/2422151494 wait complete. 2024-04-23T02:05:55.299 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.164+0000 7fc8f18cf700 1 Processor -- start 2024-04-23T02:05:55.299 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.164+0000 7fc8f18cf700 1 -- start start 2024-04-23T02:05:55.299 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.164+0000 7fc8f18cf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8ec106820 0x7fc8ec073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:55.300 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.164+0000 7fc8f18cf700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec075010 con 0x7fc8ec106820 2024-04-23T02:05:55.300 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.164+0000 7fc8eaffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8ec106820 0x7fc8ec073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:55.300 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.164+0000 7fc8eaffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8ec106820 0x7fc8ec073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.192:37928/0 (socket says 172.21.15.192:37928) 2024-04-23T02:05:55.300 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.164+0000 7fc8eaffd700 1 -- 172.21.15.192:0/1540588772 learned_addr learned my addr 172.21.15.192:0/1540588772 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:05:55.300 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.165+0000 7fc8eaffd700 1 -- 172.21.15.192:0/1540588772 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d40095f0 con 0x7fc8ec106820 2024-04-23T02:05:55.300 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.165+0000 7fc8eaffd700 1 --2- 172.21.15.192:0/1540588772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8ec106820 0x7fc8ec073090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4004f40 tx=0x7fc8d4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:55.300 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.165+0000 7fc8f08cd700 1 -- 172.21.15.192:0/1540588772 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc8d401d070 con 0x7fc8ec106820 2024-04-23T02:05:55.300 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.165+0000 7fc8f18cf700 1 -- 172.21.15.192:0/1540588772 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ec0735d0 con 0x7fc8ec106820 2024-04-23T02:05:55.300 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.165+0000 7fc8f18cf700 1 -- 172.21.15.192:0/1540588772 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ec073a50 con 0x7fc8ec106820 2024-04-23T02:05:55.300 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.165+0000 7fc8f08cd700 1 -- 172.21.15.192:0/1540588772 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc8d4005dc0 con 0x7fc8ec106820 2024-04-23T02:05:55.301 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.165+0000 7fc8f08cd700 1 -- 172.21.15.192:0/1540588772 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc8d4022d20 con 0x7fc8ec106820 2024-04-23T02:05:55.301 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.166+0000 7fc8f18cf700 1 -- 172.21.15.192:0/1540588772 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ec104ad0 con 0x7fc8ec106820 2024-04-23T02:05:55.301 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.166+0000 7fc8f08cd700 1 -- 172.21.15.192:0/1540588772 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7fc8d40225e0 con 0x7fc8ec106820 2024-04-23T02:05:55.301 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.166+0000 7fc8f08cd700 1 --2- 172.21.15.192:0/1540588772 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc8d8038550 0x7fc8d803aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:05:55.301 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.166+0000 7fc8f08cd700 1 -- 172.21.15.192:0/1540588772 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fc8d404c540 con 0x7fc8ec106820 2024-04-23T02:05:55.301 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.166+0000 7fc8ea7fc700 1 --2- 172.21.15.192:0/1540588772 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc8d8038550 0x7fc8d803aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:05:55.301 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.167+0000 7fc8ea7fc700 1 --2- 172.21.15.192:0/1540588772 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc8d8038550 0x7fc8d803aa10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc006fd0 tx=0x7fc8dc006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:05:55.301 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.172+0000 7fc8f08cd700 1 -- 172.21.15.192:0/1540588772 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc8d4021840 con 0x7fc8ec106820 2024-04-23T02:05:55.444 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.441+0000 7fc8f18cf700 1 -- 172.21.15.192:0/1540588772 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc8ec0623c0 con 0x7fc8ec106820 2024-04-23T02:05:55.445 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.442+0000 7fc8f08cd700 1 -- 172.21.15.192:0/1540588772 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fc8d4027070 con 0x7fc8ec106820 2024-04-23T02:05:55.445 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:05:55.445 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","modified":"2024-04-23T02:03:12.002793Z","created":"2024-04-23T02:03:12.002793Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi071","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T02:05:55.447 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.444+0000 7fc8f18cf700 1 -- 172.21.15.192:0/1540588772 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc8d8038550 msgr2=0x7fc8d803aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:55.447 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.444+0000 7fc8f18cf700 1 --2- 172.21.15.192:0/1540588772 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc8d8038550 0x7fc8d803aa10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc006fd0 tx=0x7fc8dc006e40 comp rx=0 tx=0).stop 2024-04-23T02:05:55.447 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.444+0000 7fc8f18cf700 1 -- 172.21.15.192:0/1540588772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8ec106820 msgr2=0x7fc8ec073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:05:55.447 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.444+0000 7fc8f18cf700 1 --2- 172.21.15.192:0/1540588772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8ec106820 0x7fc8ec073090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4004f40 tx=0x7fc8d4005020 comp rx=0 tx=0).stop 2024-04-23T02:05:55.447 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.444+0000 7fc8f18cf700 1 -- 172.21.15.192:0/1540588772 shutdown_connections 2024-04-23T02:05:55.447 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.444+0000 7fc8f18cf700 1 --2- 172.21.15.192:0/1540588772 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc8d8038550 0x7fc8d803aa10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:55.447 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.444+0000 7fc8f18cf700 1 --2- 172.21.15.192:0/1540588772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8ec106820 0x7fc8ec073090 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:05:55.447 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.444+0000 7fc8f18cf700 1 -- 172.21.15.192:0/1540588772 >> 172.21.15.192:0/1540588772 conn(0x7fc8ec0fc350 msgr2=0x7fc8ec108c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:05:55.447 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.444+0000 7fc8f18cf700 1 -- 172.21.15.192:0/1540588772 shutdown_connections 2024-04-23T02:05:55.448 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:05:55.444+0000 7fc8f18cf700 1 -- 172.21.15.192:0/1540588772 wait complete. 2024-04-23T02:05:55.448 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-23T02:05:56.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:55 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/1540588772' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T02:05:57.847 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-23T02:05:57.847 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T02:05:58.174 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T02:05:58.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:58 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:58.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:58 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:05:58.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:58 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:58.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:58 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:58.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:58 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:58.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:58 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:05:58.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:05:58 smithi071 ceph-mon[39837]: Deploying daemon node-exporter.smithi192 on smithi192 2024-04-23T02:06:00.971 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.896+0000 7f90af88f700 1 -- 172.21.15.192:0/1760398593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a8106810 msgr2=0x7f90a8106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:06:00.971 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.896+0000 7f90af88f700 1 --2- 172.21.15.192:0/1760398593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a8106810 0x7f90a8106bf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f909c009910 tx=0x7f909c009c20 comp rx=0 tx=0).stop 2024-04-23T02:06:00.971 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.896+0000 7f90af88f700 1 -- 172.21.15.192:0/1760398593 shutdown_connections 2024-04-23T02:06:00.971 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.896+0000 7f90af88f700 1 --2- 172.21.15.192:0/1760398593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a8106810 0x7f90a8106bf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:00.971 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.896+0000 7f90af88f700 1 -- 172.21.15.192:0/1760398593 >> 172.21.15.192:0/1760398593 conn(0x7f90a80fc320 msgr2=0x7f90a80fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:06:00.971 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.897+0000 7f90af88f700 1 -- 172.21.15.192:0/1760398593 shutdown_connections 2024-04-23T02:06:00.971 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.897+0000 7f90af88f700 1 -- 172.21.15.192:0/1760398593 wait complete. 2024-04-23T02:06:00.971 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.897+0000 7f90af88f700 1 Processor -- start 2024-04-23T02:06:00.971 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.897+0000 7f90af88f700 1 -- start start 2024-04-23T02:06:00.971 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.897+0000 7f90af88f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a8106810 0x7f90a819ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:06:00.972 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.897+0000 7f90af88f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a819bfc0 con 0x7f90a8106810 2024-04-23T02:06:00.972 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.897+0000 7f90ad62b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a8106810 0x7f90a819ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:06:00.972 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.897+0000 7f90ad62b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a8106810 0x7f90a819ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.192:46734/0 (socket says 172.21.15.192:46734) 2024-04-23T02:06:00.972 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.898+0000 7f90ad62b700 1 -- 172.21.15.192:0/3696610353 learned_addr learned my addr 172.21.15.192:0/3696610353 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:06:00.972 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.898+0000 7f90ad62b700 1 -- 172.21.15.192:0/3696610353 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c0095f0 con 0x7f90a8106810 2024-04-23T02:06:00.972 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.898+0000 7f90ad62b700 1 --2- 172.21.15.192:0/3696610353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a8106810 0x7f90a819ba80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f909c004f40 tx=0x7f909c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:06:00.972 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.898+0000 7f909a7fc700 1 -- 172.21.15.192:0/3696610353 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f909c01d070 con 0x7f90a8106810 2024-04-23T02:06:00.972 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.898+0000 7f90af88f700 1 -- 172.21.15.192:0/3696610353 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a819c220 con 0x7f90a8106810 2024-04-23T02:06:00.972 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.898+0000 7f909a7fc700 1 -- 172.21.15.192:0/3696610353 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f909c005dc0 con 0x7f90a8106810 2024-04-23T02:06:00.972 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.898+0000 7f909a7fc700 1 -- 172.21.15.192:0/3696610353 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f909c022d70 con 0x7f90a8106810 2024-04-23T02:06:00.972 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.898+0000 7f90af88f700 1 -- 172.21.15.192:0/3696610353 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a8195a70 con 0x7f90a8106810 2024-04-23T02:06:00.972 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.899+0000 7f909a7fc700 1 -- 172.21.15.192:0/3696610353 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7f909c00b960 con 0x7f90a8106810 2024-04-23T02:06:00.972 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.899+0000 7f90af88f700 1 -- 172.21.15.192:0/3696610353 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a80623c0 con 0x7f90a8106810 2024-04-23T02:06:00.972 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.899+0000 7f909a7fc700 1 --2- 172.21.15.192:0/3696610353 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f90940384b0 0x7f909403a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:06:00.972 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.899+0000 7f909a7fc700 1 -- 172.21.15.192:0/3696610353 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f909c04c6f0 con 0x7f90a8106810 2024-04-23T02:06:00.973 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.899+0000 7f90ace2a700 1 --2- 172.21.15.192:0/3696610353 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f90940384b0 0x7f909403a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:06:00.973 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.900+0000 7f90ace2a700 1 --2- 172.21.15.192:0/3696610353 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f90940384b0 0x7f909403a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f90a4006fd0 tx=0x7f90a4006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:06:00.973 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:00.905+0000 7f909a7fc700 1 -- 172.21.15.192:0/3696610353 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f909c022600 con 0x7f90a8106810 2024-04-23T02:06:01.173 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:01.170+0000 7f90af88f700 1 -- 172.21.15.192:0/3696610353 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f90a8196590 con 0x7f90a8106810 2024-04-23T02:06:01.174 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:01.170+0000 7f909a7fc700 1 -- 172.21.15.192:0/3696610353 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f909c027030 con 0x7f90a8106810 2024-04-23T02:06:01.174 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:06:01.174 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","modified":"2024-04-23T02:03:12.002793Z","created":"2024-04-23T02:03:12.002793Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi071","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T02:06:01.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:01.173+0000 7f90af88f700 1 -- 172.21.15.192:0/3696610353 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f90940384b0 msgr2=0x7f909403a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:06:01.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:01.173+0000 7f90af88f700 1 --2- 172.21.15.192:0/3696610353 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f90940384b0 0x7f909403a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f90a4006fd0 tx=0x7f90a4006e40 comp rx=0 tx=0).stop 2024-04-23T02:06:01.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:01.173+0000 7f90af88f700 1 -- 172.21.15.192:0/3696610353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a8106810 msgr2=0x7f90a819ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:06:01.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:01.173+0000 7f90af88f700 1 --2- 172.21.15.192:0/3696610353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a8106810 0x7f90a819ba80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f909c004f40 tx=0x7f909c005020 comp rx=0 tx=0).stop 2024-04-23T02:06:01.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:01.173+0000 7f90af88f700 1 -- 172.21.15.192:0/3696610353 shutdown_connections 2024-04-23T02:06:01.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:01.173+0000 7f90af88f700 1 --2- 172.21.15.192:0/3696610353 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f90940384b0 0x7f909403a970 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:01.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:01.173+0000 7f90af88f700 1 --2- 172.21.15.192:0/3696610353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a8106810 0x7f90a819ba80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:01.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:01.173+0000 7f90af88f700 1 -- 172.21.15.192:0/3696610353 >> 172.21.15.192:0/3696610353 conn(0x7f90a80fc320 msgr2=0x7f90a8108c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:06:01.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:01.174+0000 7f90af88f700 1 -- 172.21.15.192:0/3696610353 shutdown_connections 2024-04-23T02:06:01.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:01.174+0000 7f90af88f700 1 -- 172.21.15.192:0/3696610353 wait complete. 2024-04-23T02:06:01.178 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-23T02:06:01.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:01 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/3696610353' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T02:06:03.153 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-23T02:06:03.153 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T02:06:03.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:02 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:03.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:02 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:03.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:02 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:03.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:02 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:03.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:02 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.qgecnv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T02:06:03.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:02 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.qgecnv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T02:06:03.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:02 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T02:06:03.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:02 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:03.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:02 smithi071 ceph-mon[39837]: Deploying daemon mgr.smithi192.qgecnv on smithi192 2024-04-23T02:06:03.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:02 smithi071 ceph-mon[39837]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:03.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:02 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:03.483 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T02:06:05.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:05 smithi071 ceph-mon[39837]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:05.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:05.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:05.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:05.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:05.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T02:06:05.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:05.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:05 smithi071 ceph-mon[39837]: Deploying daemon mon.smithi192 on smithi192 2024-04-23T02:06:06.011 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.885+0000 7f1ec26c3700 1 -- 172.21.15.192:0/2672745643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ebc106820 msgr2=0x7f1ebc106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:06:06.011 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.885+0000 7f1ec26c3700 1 --2- 172.21.15.192:0/2672745643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ebc106820 0x7f1ebc106c00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009910 tx=0x7f1eac009c20 comp rx=0 tx=0).stop 2024-04-23T02:06:06.011 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.885+0000 7f1ec26c3700 1 -- 172.21.15.192:0/2672745643 shutdown_connections 2024-04-23T02:06:06.011 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.885+0000 7f1ec26c3700 1 --2- 172.21.15.192:0/2672745643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ebc106820 0x7f1ebc106c00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:06.012 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.885+0000 7f1ec26c3700 1 -- 172.21.15.192:0/2672745643 >> 172.21.15.192:0/2672745643 conn(0x7f1ebc0fc350 msgr2=0x7f1ebc0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:06:06.012 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.885+0000 7f1ec26c3700 1 -- 172.21.15.192:0/2672745643 shutdown_connections 2024-04-23T02:06:06.012 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.885+0000 7f1ec26c3700 1 -- 172.21.15.192:0/2672745643 wait complete. 2024-04-23T02:06:06.012 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.885+0000 7f1ec26c3700 1 Processor -- start 2024-04-23T02:06:06.012 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.885+0000 7f1ec26c3700 1 -- start start 2024-04-23T02:06:06.012 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.886+0000 7f1ec26c3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ebc106820 0x7f1ebc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:06:06.012 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.886+0000 7f1ec26c3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc075060 con 0x7f1ebc106820 2024-04-23T02:06:06.012 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.886+0000 7f1ebbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ebc106820 0x7f1ebc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:06:06.012 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.886+0000 7f1ebbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ebc106820 0x7f1ebc073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.192:46776/0 (socket says 172.21.15.192:46776) 2024-04-23T02:06:06.012 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.886+0000 7f1ebbfff700 1 -- 172.21.15.192:0/3033809666 learned_addr learned my addr 172.21.15.192:0/3033809666 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:06:06.012 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.886+0000 7f1ebbfff700 1 -- 172.21.15.192:0/3033809666 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eac0095f0 con 0x7f1ebc106820 2024-04-23T02:06:06.012 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.887+0000 7f1ebbfff700 1 --2- 172.21.15.192:0/3033809666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ebc106820 0x7f1ebc073090 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1eac004f40 tx=0x7f1eac005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:06:06.012 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.887+0000 7f1eb97fa700 1 -- 172.21.15.192:0/3033809666 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1eac01d070 con 0x7f1ebc106820 2024-04-23T02:06:06.012 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.887+0000 7f1ec26c3700 1 -- 172.21.15.192:0/3033809666 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc073690 con 0x7f1ebc106820 2024-04-23T02:06:06.013 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.887+0000 7f1eb97fa700 1 -- 172.21.15.192:0/3033809666 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1eac005dc0 con 0x7f1ebc106820 2024-04-23T02:06:06.013 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.887+0000 7f1eb97fa700 1 -- 172.21.15.192:0/3033809666 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1eac022d70 con 0x7f1ebc106820 2024-04-23T02:06:06.013 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.887+0000 7f1ec26c3700 1 -- 172.21.15.192:0/3033809666 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc073b10 con 0x7f1ebc106820 2024-04-23T02:06:06.013 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.888+0000 7f1ec26c3700 1 -- 172.21.15.192:0/3033809666 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ebc104ad0 con 0x7f1ebc106820 2024-04-23T02:06:06.013 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.888+0000 7f1eb97fa700 1 -- 172.21.15.192:0/3033809666 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7f1eac00f460 con 0x7f1ebc106820 2024-04-23T02:06:06.013 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.888+0000 7f1eb97fa700 1 --2- 172.21.15.192:0/3033809666 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1ea40384b0 0x7f1ea403a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:06:06.013 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.888+0000 7f1eb97fa700 1 -- 172.21.15.192:0/3033809666 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f1eac04c4f0 con 0x7f1ebc106820 2024-04-23T02:06:06.013 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.888+0000 7f1ebb7fe700 1 --2- 172.21.15.192:0/3033809666 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1ea40384b0 0x7f1ea403a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:06:06.013 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.889+0000 7f1ebb7fe700 1 --2- 172.21.15.192:0/3033809666 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1ea40384b0 0x7f1ea403a970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0006fd0 tx=0x7f1eb0006e40 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:06:06.013 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:05.893+0000 7f1eb97fa700 1 -- 172.21.15.192:0/3033809666 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1eac02a950 con 0x7f1ebc106820 2024-04-23T02:06:06.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:06.199+0000 7f1ec26c3700 1 -- 172.21.15.192:0/3033809666 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1ebc073e20 con 0x7f1ebc106820 2024-04-23T02:06:06.203 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:06.200+0000 7f1eb97fa700 1 -- 172.21.15.192:0/3033809666 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f1eac027020 con 0x7f1ebc106820 2024-04-23T02:06:06.204 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:06:06.204 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","modified":"2024-04-23T02:03:12.002793Z","created":"2024-04-23T02:03:12.002793Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi071","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T02:06:06.205 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:06.202+0000 7f1ec26c3700 1 -- 172.21.15.192:0/3033809666 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1ea40384b0 msgr2=0x7f1ea403a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:06:06.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:06.202+0000 7f1ec26c3700 1 --2- 172.21.15.192:0/3033809666 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1ea40384b0 0x7f1ea403a970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0006fd0 tx=0x7f1eb0006e40 comp rx=0 tx=0).stop 2024-04-23T02:06:06.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:06.202+0000 7f1ec26c3700 1 -- 172.21.15.192:0/3033809666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ebc106820 msgr2=0x7f1ebc073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:06:06.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:06.202+0000 7f1ec26c3700 1 --2- 172.21.15.192:0/3033809666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ebc106820 0x7f1ebc073090 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1eac004f40 tx=0x7f1eac005020 comp rx=0 tx=0).stop 2024-04-23T02:06:06.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:06.203+0000 7f1ec26c3700 1 -- 172.21.15.192:0/3033809666 shutdown_connections 2024-04-23T02:06:06.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:06.203+0000 7f1ec26c3700 1 --2- 172.21.15.192:0/3033809666 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1ea40384b0 0x7f1ea403a970 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:06.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:06.203+0000 7f1ec26c3700 1 --2- 172.21.15.192:0/3033809666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ebc106820 0x7f1ebc073090 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:06.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:06.203+0000 7f1ec26c3700 1 -- 172.21.15.192:0/3033809666 >> 172.21.15.192:0/3033809666 conn(0x7f1ebc0fc350 msgr2=0x7f1ebc108c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:06:06.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:06.203+0000 7f1ec26c3700 1 -- 172.21.15.192:0/3033809666 shutdown_connections 2024-04-23T02:06:06.207 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:06.203+0000 7f1ec26c3700 1 -- 172.21.15.192:0/3033809666 wait complete. 2024-04-23T02:06:06.207 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-04-23T02:06:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:06 smithi071 ceph-mon[39837]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:06 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/3033809666' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T02:06:07.939 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-23T02:06:07.940 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T02:06:08.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:08 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:08.834 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:06:09.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:09 smithi071 ceph-mon[39837]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:10.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:10 smithi192 bash[49588]: 9c03e90acc957f84ddd219a7c77ec2c47242db0dcd848f6e2fa5d983d2b33aa2 2024-04-23T02:06:10.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:10 smithi192 systemd[1]: Started Ceph mon.smithi192 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:06:15.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi071"}]: dispatch 2024-04-23T02:06:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:06:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: mon.smithi071 calling monitor election 2024-04-23T02:06:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:06:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.192:0/3631470668' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.qgecnv/crt"}]: dispatch 2024-04-23T02:06:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:06:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:06:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: mon.smithi192 calling monitor election 2024-04-23T02:06:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:06:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:06:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:06:15.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: mon.smithi071 is new leader, mons smithi071,smithi192 in quorum (ranks 0,1) 2024-04-23T02:06:15.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: monmap e2: 2 mons at {smithi071=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],smithi192=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-23T02:06:15.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: fsmap 2024-04-23T02:06:15.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: osdmap e5: 0 total, 0 up, 0 in 2024-04-23T02:06:15.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: mgrmap e17: smithi071.ukdjvn(active, since 33s) 2024-04-23T02:06:15.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: Standby manager daemon smithi192.qgecnv started 2024-04-23T02:06:15.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.192:0/3631470668' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T02:06:15.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: overall HEALTH_OK 2024-04-23T02:06:15.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.192:0/3631470668' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.qgecnv/key"}]: dispatch 2024-04-23T02:06:15.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:15.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.192:0/3631470668' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T02:06:15.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:15.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:15.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:15.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:15.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:15.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi071"}]: dispatch 2024-04-23T02:06:15.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:06:15.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: mon.smithi071 calling monitor election 2024-04-23T02:06:15.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:06:15.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.192:0/3631470668' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.qgecnv/crt"}]: dispatch 2024-04-23T02:06:15.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:15.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:06:15.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:06:15.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: mon.smithi192 calling monitor election 2024-04-23T02:06:15.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:06:15.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:15.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:06:15.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:06:15.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: mon.smithi071 is new leader, mons smithi071,smithi192 in quorum (ranks 0,1) 2024-04-23T02:06:15.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: monmap e2: 2 mons at {smithi071=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],smithi192=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-23T02:06:15.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: fsmap 2024-04-23T02:06:15.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: osdmap e5: 0 total, 0 up, 0 in 2024-04-23T02:06:15.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: mgrmap e17: smithi071.ukdjvn(active, since 33s) 2024-04-23T02:06:15.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: Standby manager daemon smithi192.qgecnv started 2024-04-23T02:06:15.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.192:0/3631470668' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T02:06:15.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: overall HEALTH_OK 2024-04-23T02:06:15.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.192:0/3631470668' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.qgecnv/key"}]: dispatch 2024-04-23T02:06:15.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:15.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.192:0/3631470668' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T02:06:15.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:15.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:15.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:15.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:15.785 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.781+0000 7f1af732a700 1 -- 172.21.15.192:0/3569352878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1adc0035c0 msgr2=0x7f1adc005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:06:15.785 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.781+0000 7f1af732a700 1 --2- 172.21.15.192:0/3569352878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1adc0035c0 0x7f1adc005a50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1af0075e90 tx=0x7f1ae000fe60 comp rx=0 tx=0).stop 2024-04-23T02:06:15.785 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.781+0000 7f1af732a700 1 -- 172.21.15.192:0/3569352878 shutdown_connections 2024-04-23T02:06:15.785 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.781+0000 7f1af732a700 1 --2- 172.21.15.192:0/3569352878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1adc0035c0 0x7f1adc005a50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:15.785 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.781+0000 7f1af732a700 1 -- 172.21.15.192:0/3569352878 >> 172.21.15.192:0/3569352878 conn(0x7f1af0074ae0 msgr2=0x7f1af0074ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:06:15.785 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.781+0000 7f1af732a700 1 -- 172.21.15.192:0/3569352878 shutdown_connections 2024-04-23T02:06:15.785 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.781+0000 7f1af732a700 1 -- 172.21.15.192:0/3569352878 wait complete. 2024-04-23T02:06:15.787 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.782+0000 7f1af732a700 1 Processor -- start 2024-04-23T02:06:15.787 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.782+0000 7f1af732a700 1 -- start start 2024-04-23T02:06:15.787 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.782+0000 7f1af732a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1adc0035c0 0x7f1af0100220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:06:15.787 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1af732a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1af0100760 0x7f1af0100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:06:15.787 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1af732a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af01021e0 con 0x7f1af0100760 2024-04-23T02:06:15.787 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1af732a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af01a2100 con 0x7f1adc0035c0 2024-04-23T02:06:15.787 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1af50c6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1adc0035c0 0x7f1af0100220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:06:15.787 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1af50c6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1adc0035c0 0x7f1af0100220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:48102/0 (socket says 172.21.15.192:48102) 2024-04-23T02:06:15.787 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1af50c6700 1 -- 172.21.15.192:0/208951412 learned_addr learned my addr 172.21.15.192:0/208951412 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:06:15.787 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1af50c6700 1 -- 172.21.15.192:0/208951412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1adc0035c0 msgr2=0x7f1af0100220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-23T02:06:15.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1af48c5700 1 --2- 172.21.15.192:0/208951412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1af0100760 0x7f1af0100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:06:15.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1af50c6700 1 -- 172.21.15.192:0/208951412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1adc0035c0 msgr2=0x7f1af0100220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-23T02:06:15.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1af50c6700 1 --2- 172.21.15.192:0/208951412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1adc0035c0 0x7f1af0100220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T02:06:15.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1af50c6700 1 --2- 172.21.15.192:0/208951412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1adc0035c0 0x7f1af0100220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T02:06:15.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1af48c5700 1 -- 172.21.15.192:0/208951412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1adc0035c0 msgr2=0x7f1af0100220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T02:06:15.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1af48c5700 1 --2- 172.21.15.192:0/208951412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1adc0035c0 0x7f1af0100220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:15.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1af48c5700 1 -- 172.21.15.192:0/208951412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae000f8b0 con 0x7f1af0100760 2024-04-23T02:06:15.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1af48c5700 1 --2- 172.21.15.192:0/208951412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1af0100760 0x7f1af0100be0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00b4d0 tx=0x7f1aec00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:06:15.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1ae67fc700 1 -- 172.21.15.192:0/208951412 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1aec011800 con 0x7f1af0100760 2024-04-23T02:06:15.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1af732a700 1 -- 172.21.15.192:0/208951412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af01a2360 con 0x7f1af0100760 2024-04-23T02:06:15.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1ae67fc700 1 -- 172.21.15.192:0/208951412 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1aec011df0 con 0x7f1af0100760 2024-04-23T02:06:15.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.783+0000 7f1ae67fc700 1 -- 172.21.15.192:0/208951412 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1aec00f2e0 con 0x7f1af0100760 2024-04-23T02:06:15.788 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.784+0000 7f1af732a700 1 -- 172.21.15.192:0/208951412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af01a2880 con 0x7f1af0100760 2024-04-23T02:06:15.789 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.784+0000 7f1af732a700 1 -- 172.21.15.192:0/208951412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af00623c0 con 0x7f1af0100760 2024-04-23T02:06:15.789 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.785+0000 7f1ae67fc700 1 -- 172.21.15.192:0/208951412 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f1aec010380 con 0x7f1af0100760 2024-04-23T02:06:15.789 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.785+0000 7f1ae67fc700 1 --2- 172.21.15.192:0/208951412 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1ad406c5b0 0x7f1ad406ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:06:15.789 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.785+0000 7f1ae67fc700 1 -- 172.21.15.192:0/208951412 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f1aec08a170 con 0x7f1af0100760 2024-04-23T02:06:15.790 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.786+0000 7f1af50c6700 1 --2- 172.21.15.192:0/208951412 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1ad406c5b0 0x7f1ad406ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:06:15.790 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.786+0000 7f1af50c6700 1 --2- 172.21.15.192:0/208951412 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1ad406c5b0 0x7f1ad406ea70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0001eb0 tx=0x7f1ae0005c60 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:06:15.793 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:15.790+0000 7f1ae67fc700 1 -- 172.21.15.192:0/208951412 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1aec055a30 con 0x7f1af0100760 2024-04-23T02:06:16.068 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:16.064+0000 7f1af732a700 1 -- 172.21.15.192:0/208951412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1af0101700 con 0x7f1af0100760 2024-04-23T02:06:16.068 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:16.065+0000 7f1ae67fc700 1 -- 172.21.15.192:0/208951412 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1022 (secure 0 0 0) 0x7f1aec089930 con 0x7f1af0100760 2024-04-23T02:06:16.068 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-04-23T02:06:16.068 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":2,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","modified":"2024-04-23T02:06:10.166790Z","created":"2024-04-23T02:03:12.002793Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi071","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi192","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-23T02:06:16.070 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:16.067+0000 7f1af732a700 1 -- 172.21.15.192:0/208951412 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1ad406c5b0 msgr2=0x7f1ad406ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:06:16.070 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:16.067+0000 7f1af732a700 1 --2- 172.21.15.192:0/208951412 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1ad406c5b0 0x7f1ad406ea70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0001eb0 tx=0x7f1ae0005c60 comp rx=0 tx=0).stop 2024-04-23T02:06:16.070 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:16.067+0000 7f1af732a700 1 -- 172.21.15.192:0/208951412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1af0100760 msgr2=0x7f1af0100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:06:16.071 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:16.067+0000 7f1af732a700 1 --2- 172.21.15.192:0/208951412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1af0100760 0x7f1af0100be0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00b4d0 tx=0x7f1aec00b890 comp rx=0 tx=0).stop 2024-04-23T02:06:16.071 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:16.067+0000 7f1af732a700 1 -- 172.21.15.192:0/208951412 shutdown_connections 2024-04-23T02:06:16.071 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:16.067+0000 7f1af732a700 1 --2- 172.21.15.192:0/208951412 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1ad406c5b0 0x7f1ad406ea70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:16.071 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:16.067+0000 7f1af732a700 1 --2- 172.21.15.192:0/208951412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1adc0035c0 0x7f1af0100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:16.071 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:16.067+0000 7f1af732a700 1 --2- 172.21.15.192:0/208951412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1af0100760 0x7f1af0100be0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:16.071 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:16.067+0000 7f1af732a700 1 -- 172.21.15.192:0/208951412 >> 172.21.15.192:0/208951412 conn(0x7f1af0074ae0 msgr2=0x7f1af00fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:06:16.071 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:16.067+0000 7f1af732a700 1 -- 172.21.15.192:0/208951412 shutdown_connections 2024-04-23T02:06:16.071 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:06:16.067+0000 7f1af732a700 1 -- 172.21.15.192:0/208951412 wait complete. 2024-04-23T02:06:16.071 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 2 2024-04-23T02:06:16.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:16 smithi192 ceph-mon[49662]: mgrmap e18: smithi071.ukdjvn(active, since 33s), standbys: smithi192.qgecnv 2024-04-23T02:06:16.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:16 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr metadata", "who": "smithi192.qgecnv", "id": "smithi192.qgecnv"}]: dispatch 2024-04-23T02:06:16.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:16 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:06:16.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:16 smithi192 ceph-mon[49662]: from='client.? 172.21.15.192:0/208951412' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T02:06:16.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:16 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:06:16.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:16 smithi071 ceph-mon[39837]: mgrmap e18: smithi071.ukdjvn(active, since 33s), standbys: smithi192.qgecnv 2024-04-23T02:06:16.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:16 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr metadata", "who": "smithi192.qgecnv", "id": "smithi192.qgecnv"}]: dispatch 2024-04-23T02:06:16.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:16 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:06:16.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:16 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/208951412' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T02:06:16.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:16 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:06:17.172 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-23T02:06:17.172 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-23T02:06:17.405 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:06:17.455 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:17 smithi192 ceph-mon[49662]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:17.456 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:17 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:17.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:17 smithi071 ceph-mon[39837]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:17.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:17 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:19.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.223+0000 7f8b1794e700 1 -- 172.21.15.71:0/1476168909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b10100090 msgr2=0x7f8b10100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:06:19.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.223+0000 7f8b1794e700 1 --2- 172.21.15.71:0/1476168909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b10100090 0x7f8b10100470 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009910 tx=0x7f8b0c009c20 comp rx=0 tx=0).stop 2024-04-23T02:06:19.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.223+0000 7f8b1794e700 1 -- 172.21.15.71:0/1476168909 shutdown_connections 2024-04-23T02:06:19.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.223+0000 7f8b1794e700 1 --2- 172.21.15.71:0/1476168909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b10100090 0x7f8b10100470 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:19.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.223+0000 7f8b1794e700 1 -- 172.21.15.71:0/1476168909 >> 172.21.15.71:0/1476168909 conn(0x7f8b10075df0 msgr2=0x7f8b10076200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:06:19.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.223+0000 7f8b1794e700 1 -- 172.21.15.71:0/1476168909 shutdown_connections 2024-04-23T02:06:19.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.223+0000 7f8b1794e700 1 -- 172.21.15.71:0/1476168909 wait complete. 2024-04-23T02:06:19.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.223+0000 7f8b1794e700 1 Processor -- start 2024-04-23T02:06:19.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.223+0000 7f8b1794e700 1 -- start start 2024-04-23T02:06:19.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.224+0000 7f8b1794e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b10100090 0x7f8b10196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:06:19.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.224+0000 7f8b1794e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b101970a0 0x7f8b10197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:06:19.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.224+0000 7f8b1794e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1019fbd0 con 0x7f8b10100090 2024-04-23T02:06:19.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.224+0000 7f8b1794e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1019fd10 con 0x7f8b101970a0 2024-04-23T02:06:19.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.224+0000 7f8b156ea700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b10100090 0x7f8b10196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:06:19.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.224+0000 7f8b156ea700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b10100090 0x7f8b10196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44848/0 (socket says 172.21.15.71:44848) 2024-04-23T02:06:19.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.224+0000 7f8b156ea700 1 -- 172.21.15.71:0/1166475466 learned_addr learned my addr 172.21.15.71:0/1166475466 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:06:19.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.224+0000 7f8b14ee9700 1 --2- 172.21.15.71:0/1166475466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b101970a0 0x7f8b10197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:06:19.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.224+0000 7f8b156ea700 1 -- 172.21.15.71:0/1166475466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b101970a0 msgr2=0x7f8b10197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:06:19.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.224+0000 7f8b156ea700 1 --2- 172.21.15.71:0/1166475466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b101970a0 0x7f8b10197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:19.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.224+0000 7f8b156ea700 1 -- 172.21.15.71:0/1166475466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c0095f0 con 0x7f8b10100090 2024-04-23T02:06:19.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.224+0000 7f8b14ee9700 1 --2- 172.21.15.71:0/1166475466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b101970a0 0x7f8b10197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:06:19.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.224+0000 7f8b156ea700 1 --2- 172.21.15.71:0/1166475466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b10100090 0x7f8b10196b60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c005b40 tx=0x7f8b0c004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:06:19.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.224+0000 7f8b067fc700 1 -- 172.21.15.71:0/1166475466 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b0c01d070 con 0x7f8b10100090 2024-04-23T02:06:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.224+0000 7f8b067fc700 1 -- 172.21.15.71:0/1166475466 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f8b0c005dc0 con 0x7f8b10100090 2024-04-23T02:06:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.224+0000 7f8b1794e700 1 -- 172.21.15.71:0/1166475466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1019ff10 con 0x7f8b10100090 2024-04-23T02:06:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.224+0000 7f8b067fc700 1 -- 172.21.15.71:0/1166475466 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b0c00f460 con 0x7f8b10100090 2024-04-23T02:06:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.224+0000 7f8b1794e700 1 -- 172.21.15.71:0/1166475466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b101a03b0 con 0x7f8b10100090 2024-04-23T02:06:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.225+0000 7f8b1794e700 1 -- 172.21.15.71:0/1166475466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b10104120 con 0x7f8b10100090 2024-04-23T02:06:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.226+0000 7f8b067fc700 1 -- 172.21.15.71:0/1166475466 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f8b0c00f5e0 con 0x7f8b10100090 2024-04-23T02:06:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.226+0000 7f8b067fc700 1 --2- 172.21.15.71:0/1166475466 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f8afc06c5c0 0x7f8afc06ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:06:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.226+0000 7f8b067fc700 1 -- 172.21.15.71:0/1166475466 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f8b0c08bfa0 con 0x7f8b10100090 2024-04-23T02:06:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.226+0000 7f8b14ee9700 1 --2- 172.21.15.71:0/1166475466 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f8afc06c5c0 0x7f8afc06ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:06:19.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.226+0000 7f8b14ee9700 1 --2- 172.21.15.71:0/1166475466 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f8afc06c5c0 0x7f8afc06ea80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8b00006fd0 tx=0x7f8b00008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:06:19.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.231+0000 7f8b067fc700 1 -- 172.21.15.71:0/1166475466 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8b0c0bf0e0 con 0x7f8b10100090 2024-04-23T02:06:19.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.430+0000 7f8b1794e700 1 -- 172.21.15.71:0/1166475466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f8b100623c0 con 0x7f8b10100090 2024-04-23T02:06:19.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.430+0000 7f8b067fc700 1 -- 172.21.15.71:0/1166475466 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7f8b0c027090 con 0x7f8b10100090 2024-04-23T02:06:19.434 INFO:teuthology.orchestra.run.smithi071.stdout:# minimal ceph.conf for 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T02:06:19.434 INFO:teuthology.orchestra.run.smithi071.stdout:[global] 2024-04-23T02:06:19.434 INFO:teuthology.orchestra.run.smithi071.stdout: fsid = 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T02:06:19.434 INFO:teuthology.orchestra.run.smithi071.stdout: mon_host = [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2024-04-23T02:06:19.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.432+0000 7f8b1794e700 1 -- 172.21.15.71:0/1166475466 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f8afc06c5c0 msgr2=0x7f8afc06ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:06:19.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.432+0000 7f8b1794e700 1 --2- 172.21.15.71:0/1166475466 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f8afc06c5c0 0x7f8afc06ea80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8b00006fd0 tx=0x7f8b00008040 comp rx=0 tx=0).stop 2024-04-23T02:06:19.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.432+0000 7f8b1794e700 1 -- 172.21.15.71:0/1166475466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b10100090 msgr2=0x7f8b10196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:06:19.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.432+0000 7f8b1794e700 1 --2- 172.21.15.71:0/1166475466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b10100090 0x7f8b10196b60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c005b40 tx=0x7f8b0c004d60 comp rx=0 tx=0).stop 2024-04-23T02:06:19.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.432+0000 7f8b1794e700 1 -- 172.21.15.71:0/1166475466 shutdown_connections 2024-04-23T02:06:19.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.432+0000 7f8b1794e700 1 --2- 172.21.15.71:0/1166475466 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f8afc06c5c0 0x7f8afc06ea80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:19.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.432+0000 7f8b1794e700 1 --2- 172.21.15.71:0/1166475466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b10100090 0x7f8b10196b60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:19.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.432+0000 7f8b1794e700 1 --2- 172.21.15.71:0/1166475466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b101970a0 0x7f8b10197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:19.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.432+0000 7f8b1794e700 1 -- 172.21.15.71:0/1166475466 >> 172.21.15.71:0/1166475466 conn(0x7f8b10075df0 msgr2=0x7f8b10101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:06:19.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.432+0000 7f8b1794e700 1 -- 172.21.15.71:0/1166475466 shutdown_connections 2024-04-23T02:06:19.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:19.432+0000 7f8b1794e700 1 -- 172.21.15.71:0/1166475466 wait complete. 2024-04-23T02:06:19.570 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:19 smithi071 ceph-mon[39837]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:19.570 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:19 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:19.570 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:19 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:19.570 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:19 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:19.570 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:19 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:06:19.570 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:19 smithi071 ceph-mon[39837]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-23T02:06:19.571 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:19 smithi071 ceph-mon[39837]: Updating smithi192:/etc/ceph/ceph.conf 2024-04-23T02:06:19.571 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:19 smithi071 ceph-mon[39837]: Updating smithi071:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T02:06:19.571 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:19 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:19.571 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:19 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:19.571 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:19 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:19.571 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:19 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:19.571 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:19 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:19.571 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:19 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T02:06:19.571 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:19 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T02:06:19.571 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:19 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:19.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:19 smithi192 ceph-mon[49662]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:19.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:19 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:19.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:19 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:19 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:19 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:06:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:19 smithi192 ceph-mon[49662]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-23T02:06:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:19 smithi192 ceph-mon[49662]: Updating smithi192:/etc/ceph/ceph.conf 2024-04-23T02:06:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:19 smithi192 ceph-mon[49662]: Updating smithi071:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T02:06:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:19 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:19 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:19 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:19 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:19 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:19 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T02:06:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:19 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T02:06:19.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:19 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:20.538 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:20 smithi071 ceph-mon[39837]: Updating smithi192:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T02:06:20.538 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:20 smithi071 ceph-mon[39837]: Reconfiguring mon.smithi071 (unknown last config time)... 2024-04-23T02:06:20.538 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:20 smithi071 ceph-mon[39837]: Reconfiguring daemon mon.smithi071 on smithi071 2024-04-23T02:06:20.538 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:20 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1166475466' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:20.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:20 smithi192 ceph-mon[49662]: Updating smithi192:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T02:06:20.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:20 smithi192 ceph-mon[49662]: Reconfiguring mon.smithi071 (unknown last config time)... 2024-04-23T02:06:20.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:20 smithi192 ceph-mon[49662]: Reconfiguring daemon mon.smithi071 on smithi071 2024-04-23T02:06:20.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:20 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1166475466' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:21.093 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-23T02:06:21.093 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T02:06:21.094 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T02:06:21.123 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T02:06:21.123 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T02:06:21.196 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-23T02:06:21.196 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T02:06:21.234 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-23T02:06:21.234 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T02:06:21.307 INFO:tasks.cephadm:Deploying OSDs... 2024-04-23T02:06:21.307 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T02:06:21.307 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T02:06:21.324 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T02:06:21.324 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_1 2024-04-23T02:06:21.383 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T02:06:21.383 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T02:06:21.383 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 165830 Links: 1 2024-04-23T02:06:21.383 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T02:06:21.383 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T02:06:21.383 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-23 02:05:33.752957166 +0000 2024-04-23T02:06:21.383 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-23 01:57:55.816505613 +0000 2024-04-23T02:06:21.383 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-23 01:57:55.816505613 +0000 2024-04-23T02:06:21.383 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-04-23T02:06:21.383 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T02:06:21.430 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:21 smithi071 ceph-mon[39837]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:21.430 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:21 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:21.430 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:21 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:21.430 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:21 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi071.ukdjvn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T02:06:21.430 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:21 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T02:06:21.430 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:21 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:21.449 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-23T02:06:21.449 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-23T02:06:21.449 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000212575 s, 2.4 MB/s 2024-04-23T02:06:21.451 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T02:06:21.509 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_2 2024-04-23T02:06:21.567 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T02:06:21.567 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T02:06:21.567 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 169090 Links: 1 2024-04-23T02:06:21.567 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T02:06:21.567 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T02:06:21.567 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-23 02:05:33.753957148 +0000 2024-04-23T02:06:21.567 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-23 01:57:56.153499734 +0000 2024-04-23T02:06:21.567 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-23 01:57:56.153499734 +0000 2024-04-23T02:06:21.568 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-04-23T02:06:21.568 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T02:06:21.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:21 smithi192 ceph-mon[49662]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:21.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:21 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:21.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:21 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:21.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:21 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi071.ukdjvn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T02:06:21.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:21 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T02:06:21.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:21 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:21.632 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-23T02:06:21.632 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-23T02:06:21.633 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000207035 s, 2.5 MB/s 2024-04-23T02:06:21.633 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T02:06:21.693 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_3 2024-04-23T02:06:21.750 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T02:06:21.750 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T02:06:21.751 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 167335 Links: 1 2024-04-23T02:06:21.751 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T02:06:21.751 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T02:06:21.751 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-23 02:05:33.753957148 +0000 2024-04-23T02:06:21.751 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-23 01:57:56.488493889 +0000 2024-04-23T02:06:21.751 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-23 01:57:56.488493889 +0000 2024-04-23T02:06:21.751 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-04-23T02:06:21.751 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T02:06:21.816 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-23T02:06:21.816 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-23T02:06:21.816 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000177137 s, 2.9 MB/s 2024-04-23T02:06:21.817 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T02:06:21.876 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_4 2024-04-23T02:06:21.934 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T02:06:21.934 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T02:06:21.934 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 173155 Links: 1 2024-04-23T02:06:21.934 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T02:06:21.935 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T02:06:21.935 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-23 02:05:33.753957148 +0000 2024-04-23T02:06:21.935 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-23 01:57:56.835487836 +0000 2024-04-23T02:06:21.935 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-23 01:57:56.835487836 +0000 2024-04-23T02:06:21.935 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-04-23T02:06:21.935 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T02:06:21.999 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-23T02:06:21.999 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-23T02:06:21.999 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000290419 s, 1.8 MB/s 2024-04-23T02:06:22.000 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T02:06:22.059 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-23T02:06:22.059 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T02:06:22.080 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T02:06:22.081 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2024-04-23T02:06:22.142 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T02:06:22.143 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T02:06:22.143 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 167431 Links: 1 2024-04-23T02:06:22.143 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T02:06:22.143 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T02:06:22.143 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-04-23 02:06:09.074546498 +0000 2024-04-23T02:06:22.143 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-04-23 01:57:52.799442824 +0000 2024-04-23T02:06:22.143 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-04-23 01:57:52.799442824 +0000 2024-04-23T02:06:22.143 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: - 2024-04-23T02:06:22.143 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T02:06:22.213 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-04-23T02:06:22.213 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-04-23T02:06:22.213 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000284532 s, 1.8 MB/s 2024-04-23T02:06:22.214 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T02:06:22.276 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2024-04-23T02:06:22.336 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T02:06:22.337 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T02:06:22.337 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 166695 Links: 1 2024-04-23T02:06:22.337 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T02:06:22.337 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T02:06:22.337 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-04-23 02:06:09.074546498 +0000 2024-04-23T02:06:22.337 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-04-23 01:57:53.140435965 +0000 2024-04-23T02:06:22.337 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-04-23 01:57:53.140435965 +0000 2024-04-23T02:06:22.337 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: - 2024-04-23T02:06:22.337 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T02:06:22.404 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-04-23T02:06:22.405 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-04-23T02:06:22.405 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000370787 s, 1.4 MB/s 2024-04-23T02:06:22.406 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T02:06:22.466 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2024-04-23T02:06:22.524 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T02:06:22.524 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T02:06:22.524 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 169214 Links: 1 2024-04-23T02:06:22.524 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T02:06:22.524 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T02:06:22.525 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-04-23 02:06:09.074546498 +0000 2024-04-23T02:06:22.525 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-04-23 01:57:53.513428463 +0000 2024-04-23T02:06:22.525 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-04-23 01:57:53.513428463 +0000 2024-04-23T02:06:22.525 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: - 2024-04-23T02:06:22.525 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T02:06:22.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:22 smithi192 ceph-mon[49662]: Reconfiguring mgr.smithi071.ukdjvn (unknown last config time)... 2024-04-23T02:06:22.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:22 smithi192 ceph-mon[49662]: Reconfiguring daemon mgr.smithi071.ukdjvn on smithi071 2024-04-23T02:06:22.591 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-04-23T02:06:22.592 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-04-23T02:06:22.592 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000161987 s, 3.2 MB/s 2024-04-23T02:06:22.593 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T02:06:22.632 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:22 smithi071 ceph-mon[39837]: Reconfiguring mgr.smithi071.ukdjvn (unknown last config time)... 2024-04-23T02:06:22.632 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:22 smithi071 ceph-mon[39837]: Reconfiguring daemon mgr.smithi071.ukdjvn on smithi071 2024-04-23T02:06:22.652 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2024-04-23T02:06:22.710 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T02:06:22.710 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T02:06:22.710 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 165751 Links: 1 2024-04-23T02:06:22.710 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T02:06:22.710 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T02:06:22.710 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-04-23 02:06:09.074546498 +0000 2024-04-23T02:06:22.710 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-04-23 01:57:53.868421323 +0000 2024-04-23T02:06:22.710 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-04-23 01:57:53.868421323 +0000 2024-04-23T02:06:22.710 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: - 2024-04-23T02:06:22.710 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T02:06:22.775 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-04-23T02:06:22.775 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-04-23T02:06:22.776 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000230659 s, 2.2 MB/s 2024-04-23T02:06:22.776 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T02:06:22.835 INFO:tasks.cephadm:Deploying osd.0 on smithi071 with /dev/vg_nvme/lv_4... 2024-04-23T02:06:22.836 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T02:06:23.644 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:06:23.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:23 smithi071 ceph-mon[39837]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:23.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:23 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:23.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:23 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:23.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:23 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi071", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-23T02:06:23.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:23 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:23.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:23 smithi192 ceph-mon[49662]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:23.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:23 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:23.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:23 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:23.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:23 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi071", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-23T02:06:23.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:23 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:24.848 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:24 smithi071 ceph-mon[39837]: Reconfiguring ceph-exporter.smithi071 (monmap changed)... 2024-04-23T02:06:24.848 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:24 smithi071 ceph-mon[39837]: Reconfiguring daemon ceph-exporter.smithi071 on smithi071 2024-04-23T02:06:24.849 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:24 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:24.849 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:24 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:24.849 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:24 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi071", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-23T02:06:24.849 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:24 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:24.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:24 smithi192 ceph-mon[49662]: Reconfiguring ceph-exporter.smithi071 (monmap changed)... 2024-04-23T02:06:24.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:24 smithi192 ceph-mon[49662]: Reconfiguring daemon ceph-exporter.smithi071 on smithi071 2024-04-23T02:06:24.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:24 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:24.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:24 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:24.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:24 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi071", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-23T02:06:24.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:24 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:25.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:25 smithi192 ceph-mon[49662]: Reconfiguring crash.smithi071 (monmap changed)... 2024-04-23T02:06:25.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:25 smithi192 ceph-mon[49662]: Reconfiguring daemon crash.smithi071 on smithi071 2024-04-23T02:06:25.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:25 smithi192 ceph-mon[49662]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:25.923 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:25 smithi071 ceph-mon[39837]: Reconfiguring crash.smithi071 (monmap changed)... 2024-04-23T02:06:25.923 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:25 smithi071 ceph-mon[39837]: Reconfiguring daemon crash.smithi071 on smithi071 2024-04-23T02:06:25.923 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:25 smithi071 ceph-mon[39837]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:27.551 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:27 smithi071 ceph-mon[39837]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:27.551 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:27 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:27.551 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:27 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:27.551 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:27 smithi071 ceph-mon[39837]: Reconfiguring alertmanager.smithi071 (dependencies changed)... 2024-04-23T02:06:27.551 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:27 smithi071 ceph-mon[39837]: Reconfiguring daemon alertmanager.smithi071 on smithi071 2024-04-23T02:06:27.552 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:27 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:06:27.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:27 smithi192 ceph-mon[49662]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:27.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:27 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:27.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:27 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:27.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:27 smithi192 ceph-mon[49662]: Reconfiguring alertmanager.smithi071 (dependencies changed)... 2024-04-23T02:06:27.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:27 smithi192 ceph-mon[49662]: Reconfiguring daemon alertmanager.smithi071 on smithi071 2024-04-23T02:06:27.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:27 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:06:27.708 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T02:06:27.708 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T02:06:27.708 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-04-23T02:06:27.708 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-04-23T02:06:27.708 INFO:teuthology.orchestra.run.smithi071.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110711 s, 947 MB/s 2024-04-23T02:06:27.708 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-04-23T02:06:29.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:29 smithi071 ceph-mon[39837]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:29.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:29 smithi192 ceph-mon[49662]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:29.767 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_4 2024-04-23T02:06:30.233 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:06:31.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:31 smithi192 ceph-mon[49662]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:31.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:31 smithi071 ceph-mon[39837]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:32.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.659+0000 7f98f2fdd700 1 -- 172.21.15.71:0/2136667805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0fff50 msgr2=0x7f98ec1003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:06:32.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.659+0000 7f98f2fdd700 1 --2- 172.21.15.71:0/2136667805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0fff50 0x7f98ec1003d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009910 tx=0x7f98d8009c20 comp rx=0 tx=0).stop 2024-04-23T02:06:32.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.659+0000 7f98f2fdd700 1 -- 172.21.15.71:0/2136667805 shutdown_connections 2024-04-23T02:06:32.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.659+0000 7f98f2fdd700 1 --2- 172.21.15.71:0/2136667805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0fff50 0x7f98ec1003d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:32.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.659+0000 7f98f2fdd700 1 --2- 172.21.15.71:0/2136667805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98ec0ff5f0 0x7f98ec0ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:32.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.659+0000 7f98f2fdd700 1 -- 172.21.15.71:0/2136667805 >> 172.21.15.71:0/2136667805 conn(0x7f98ec0fb190 msgr2=0x7f98ec0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:06:32.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.659+0000 7f98f2fdd700 1 -- 172.21.15.71:0/2136667805 shutdown_connections 2024-04-23T02:06:32.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.660+0000 7f98f2fdd700 1 -- 172.21.15.71:0/2136667805 wait complete. 2024-04-23T02:06:32.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.660+0000 7f98f2fdd700 1 Processor -- start 2024-04-23T02:06:32.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.660+0000 7f98f2fdd700 1 -- start start 2024-04-23T02:06:32.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.661+0000 7f98f2fdd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98ec0ff5f0 0x7f98ec105170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:06:32.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.661+0000 7f98f2fdd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0fff50 0x7f98ec1056b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:06:32.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.661+0000 7f98f2fdd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec101bd0 con 0x7f98ec0fff50 2024-04-23T02:06:32.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.661+0000 7f98f2fdd700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec101d40 con 0x7f98ec0ff5f0 2024-04-23T02:06:32.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.661+0000 7f98ebfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0fff50 0x7f98ec1056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:06:32.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.661+0000 7f98ebfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0fff50 0x7f98ec1056b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56626/0 (socket says 172.21.15.71:56626) 2024-04-23T02:06:32.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.661+0000 7f98ebfff700 1 -- 172.21.15.71:0/4051656542 learned_addr learned my addr 172.21.15.71:0/4051656542 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:06:32.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.661+0000 7f98f0d79700 1 --2- 172.21.15.71:0/4051656542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98ec0ff5f0 0x7f98ec105170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:06:32.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.661+0000 7f98ebfff700 1 -- 172.21.15.71:0/4051656542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98ec0ff5f0 msgr2=0x7f98ec105170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:06:32.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.661+0000 7f98ebfff700 1 --2- 172.21.15.71:0/4051656542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98ec0ff5f0 0x7f98ec105170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:06:32.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.661+0000 7f98ebfff700 1 -- 172.21.15.71:0/4051656542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d80095f0 con 0x7f98ec0fff50 2024-04-23T02:06:32.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.662+0000 7f98ebfff700 1 --2- 172.21.15.71:0/4051656542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0fff50 0x7f98ec1056b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f98d8005b40 tx=0x7f98d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:06:32.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.662+0000 7f98e9ffb700 1 -- 172.21.15.71:0/4051656542 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98d801d070 con 0x7f98ec0fff50 2024-04-23T02:06:32.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.662+0000 7f98f2fdd700 1 -- 172.21.15.71:0/4051656542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ec101fc0 con 0x7f98ec0fff50 2024-04-23T02:06:32.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.662+0000 7f98e9ffb700 1 -- 172.21.15.71:0/4051656542 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f98d8022470 con 0x7f98ec0fff50 2024-04-23T02:06:32.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.662+0000 7f98e9ffb700 1 -- 172.21.15.71:0/4051656542 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98d800f690 con 0x7f98ec0fff50 2024-04-23T02:06:32.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.662+0000 7f98f2fdd700 1 -- 172.21.15.71:0/4051656542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ec1024b0 con 0x7f98ec0fff50 2024-04-23T02:06:32.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.663+0000 7f98f2fdd700 1 -- 172.21.15.71:0/4051656542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ec1026d0 con 0x7f98ec0fff50 2024-04-23T02:06:32.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.664+0000 7f98e9ffb700 1 -- 172.21.15.71:0/4051656542 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f98d8004af0 con 0x7f98ec0fff50 2024-04-23T02:06:32.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.664+0000 7f98e9ffb700 1 --2- 172.21.15.71:0/4051656542 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f98dc06c470 0x7f98dc06e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:06:32.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.664+0000 7f98e9ffb700 1 -- 172.21.15.71:0/4051656542 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f98d808be40 con 0x7f98ec0fff50 2024-04-23T02:06:32.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.664+0000 7f98f0d79700 1 --2- 172.21.15.71:0/4051656542 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f98dc06c470 0x7f98dc06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:06:32.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.665+0000 7f98f0d79700 1 --2- 172.21.15.71:0/4051656542 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f98dc06c470 0x7f98dc06e930 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f98e0005ea0 tx=0x7f98e0005e30 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:06:32.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.669+0000 7f98e9ffb700 1 -- 172.21.15.71:0/4051656542 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f98d805b250 con 0x7f98ec0fff50 2024-04-23T02:06:32.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:06:32.882+0000 7f98f2fdd700 1 -- 172.21.15.71:0/4051656542 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f98ec109340 con 0x7f98dc06c470 2024-04-23T02:06:33.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:33 smithi071 ceph-mon[39837]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:33.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:33 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T02:06:33.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:33 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T02:06:33.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:33 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:33.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:33 smithi192 ceph-mon[49662]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:33.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:33 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T02:06:33.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:33 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T02:06:33.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:33 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:34.422 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:34 smithi071 ceph-mon[39837]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:06:34.422 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:34 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:34.422 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:34 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:34.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:34 smithi192 ceph-mon[49662]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:06:34.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:34 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:34.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:34 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:35.395 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:35 smithi071 ceph-mon[39837]: Reconfiguring grafana.smithi071 (dependencies changed)... 2024-04-23T02:06:35.395 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:35 smithi071 ceph-mon[39837]: Reconfiguring daemon grafana.smithi071 on smithi071 2024-04-23T02:06:35.395 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:35 smithi071 ceph-mon[39837]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:35.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:35 smithi192 ceph-mon[49662]: Reconfiguring grafana.smithi071 (dependencies changed)... 2024-04-23T02:06:35.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:35 smithi192 ceph-mon[49662]: Reconfiguring daemon grafana.smithi071 on smithi071 2024-04-23T02:06:35.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:35 smithi192 ceph-mon[49662]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:36.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:36 smithi192 ceph-mon[49662]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:36.953 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:36 smithi071 ceph-mon[39837]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:39.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:39 smithi071 ceph-mon[39837]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:39.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:39 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:39.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:39 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:39.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:39 smithi192 ceph-mon[49662]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:39.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:39 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:39.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:39 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:40.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:40 smithi071 ceph-mon[39837]: Reconfiguring prometheus.smithi071 (dependencies changed)... 2024-04-23T02:06:40.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:40 smithi071 ceph-mon[39837]: Reconfiguring daemon prometheus.smithi071 on smithi071 2024-04-23T02:06:40.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:40 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/4080230742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5f48bf1-6e98-4432-885a-f6ed0c42e5d3"}]: dispatch 2024-04-23T02:06:40.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:40 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/4080230742' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5f48bf1-6e98-4432-885a-f6ed0c42e5d3"}]': finished 2024-04-23T02:06:40.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:40 smithi071 ceph-mon[39837]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T02:06:40.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:40 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T02:06:40.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:40 smithi192 ceph-mon[49662]: Reconfiguring prometheus.smithi071 (dependencies changed)... 2024-04-23T02:06:40.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:40 smithi192 ceph-mon[49662]: Reconfiguring daemon prometheus.smithi071 on smithi071 2024-04-23T02:06:40.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:40 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/4080230742' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5f48bf1-6e98-4432-885a-f6ed0c42e5d3"}]: dispatch 2024-04-23T02:06:40.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:40 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/4080230742' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5f48bf1-6e98-4432-885a-f6ed0c42e5d3"}]': finished 2024-04-23T02:06:40.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:40 smithi192 ceph-mon[49662]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T02:06:40.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:40 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T02:06:41.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:41 smithi071 ceph-mon[39837]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:41.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:41 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/2723226045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T02:06:41.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:41 smithi192 ceph-mon[49662]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:41.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:41 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/2723226045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T02:06:42.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:06:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:42 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:06:43.449 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:43 smithi071 ceph-mon[39837]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:43.509 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:43 smithi192 ceph-mon[49662]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:45.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:45 smithi071 ceph-mon[39837]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:45.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:45 smithi192 ceph-mon[49662]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:46.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:46 smithi071 ceph-mon[39837]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:46.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:46 smithi192 ceph-mon[49662]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:49.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:49 smithi071 ceph-mon[39837]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:49.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:49 smithi192 ceph-mon[49662]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:51.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:51 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:51.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:51 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:51.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:51 smithi192 ceph-mon[49662]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:51.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:51 smithi192 ceph-mon[49662]: Reconfiguring ceph-exporter.smithi192 (monmap changed)... 2024-04-23T02:06:51.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:51 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi192", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-23T02:06:51.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:51 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:51.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:51 smithi192 ceph-mon[49662]: Reconfiguring daemon ceph-exporter.smithi192 on smithi192 2024-04-23T02:06:51.421 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:51 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:51.422 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:51 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:51.422 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:51 smithi071 ceph-mon[39837]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:51.422 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:51 smithi071 ceph-mon[39837]: Reconfiguring ceph-exporter.smithi192 (monmap changed)... 2024-04-23T02:06:51.422 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:51 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi192", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-23T02:06:51.422 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:51 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:51.422 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:51 smithi071 ceph-mon[39837]: Reconfiguring daemon ceph-exporter.smithi192 on smithi192 2024-04-23T02:06:52.234 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:52 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T02:06:52.234 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:52 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:52.234 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:52 smithi071 ceph-mon[39837]: Deploying daemon osd.0 on smithi071 2024-04-23T02:06:52.234 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:52 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:52.234 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:52 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:52.234 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:52 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-23T02:06:52.234 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:52 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:52.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:52 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T02:06:52.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:52 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:52.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:52 smithi192 ceph-mon[49662]: Deploying daemon osd.0 on smithi071 2024-04-23T02:06:52.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:52 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:52.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:52 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:52.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:52 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-23T02:06:52.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:52 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:53.233 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:53 smithi071 ceph-mon[39837]: Reconfiguring crash.smithi192 (monmap changed)... 2024-04-23T02:06:53.233 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:53 smithi071 ceph-mon[39837]: Reconfiguring daemon crash.smithi192 on smithi192 2024-04-23T02:06:53.233 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:53 smithi071 ceph-mon[39837]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:53.233 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:53 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:53.233 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:53 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:53.233 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:53 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.qgecnv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T02:06:53.233 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:53 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T02:06:53.233 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:53 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:53.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:53 smithi192 ceph-mon[49662]: Reconfiguring crash.smithi192 (monmap changed)... 2024-04-23T02:06:53.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:53 smithi192 ceph-mon[49662]: Reconfiguring daemon crash.smithi192 on smithi192 2024-04-23T02:06:53.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:53 smithi192 ceph-mon[49662]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:53.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:53 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:53.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:53 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:53.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:53 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.qgecnv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T02:06:53.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:53 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T02:06:53.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:53 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:54.292 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:54 smithi071 ceph-mon[39837]: Reconfiguring mgr.smithi192.qgecnv (monmap changed)... 2024-04-23T02:06:54.292 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:54 smithi071 ceph-mon[39837]: Reconfiguring daemon mgr.smithi192.qgecnv on smithi192 2024-04-23T02:06:54.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:54 smithi192 ceph-mon[49662]: Reconfiguring mgr.smithi192.qgecnv (monmap changed)... 2024-04-23T02:06:54.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:54 smithi192 ceph-mon[49662]: Reconfiguring daemon mgr.smithi192.qgecnv on smithi192 2024-04-23T02:06:55.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:55 smithi071 ceph-mon[39837]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:55.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:55 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:55.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:55 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:55.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:55 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T02:06:55.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:55 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T02:06:55.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:55 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:55.512 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:55 smithi192 ceph-mon[49662]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:55.512 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:55 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:55.512 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:55 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:55.512 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:55 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T02:06:55.513 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:55 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T02:06:55.513 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:55 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:06:56.283 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:56 smithi192 ceph-mon[49662]: Reconfiguring mon.smithi192 (monmap changed)... 2024-04-23T02:06:56.284 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:56 smithi192 ceph-mon[49662]: Reconfiguring daemon mon.smithi192 on smithi192 2024-04-23T02:06:56.393 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:56 smithi071 ceph-mon[39837]: Reconfiguring mon.smithi192 (monmap changed)... 2024-04-23T02:06:56.393 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:56 smithi071 ceph-mon[39837]: Reconfiguring daemon mon.smithi192 on smithi192 2024-04-23T02:06:57.604 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:57.604 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:57.605 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:57.605 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-23T02:06:57.605 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-23T02:06:57.605 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi071.front.sepia.ceph.com:9093"}]: dispatch 2024-04-23T02:06:57.605 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi071.front.sepia.ceph.com:9093"}]: dispatch 2024-04-23T02:06:57.605 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:57.605 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-23T02:06:57.605 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-23T02:06:57.605 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi071.front.sepia.ceph.com:3000"}]: dispatch 2024-04-23T02:06:57.605 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi071.front.sepia.ceph.com:3000"}]: dispatch 2024-04-23T02:06:57.605 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:57.605 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-23T02:06:57.605 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-23T02:06:57.605 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi071.front.sepia.ceph.com:9095"}]: dispatch 2024-04-23T02:06:57.606 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi071.front.sepia.ceph.com:9095"}]: dispatch 2024-04-23T02:06:57.606 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:57.606 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:06:57.606 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:57.606 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:57.606 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:06:57.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:57.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:57.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:57.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-23T02:06:57.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-23T02:06:57.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi071.front.sepia.ceph.com:9093"}]: dispatch 2024-04-23T02:06:57.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi071.front.sepia.ceph.com:9093"}]: dispatch 2024-04-23T02:06:57.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-23T02:06:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-23T02:06:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi071.front.sepia.ceph.com:3000"}]: dispatch 2024-04-23T02:06:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi071.front.sepia.ceph.com:3000"}]: dispatch 2024-04-23T02:06:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-23T02:06:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-23T02:06:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi071.front.sepia.ceph.com:9095"}]: dispatch 2024-04-23T02:06:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi071.front.sepia.ceph.com:9095"}]: dispatch 2024-04-23T02:06:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:06:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:06:59.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:59 smithi071 ceph-mon[39837]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:59.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:59 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:59.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:06:59 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:59.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:59 smithi192 ceph-mon[49662]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:06:59.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:59 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:06:59.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:06:59 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:01.261 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 0 on host 'smithi071' 2024-04-23T02:07:01.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:01.258+0000 7f98e9ffb700 1 -- 172.21.15.71:0/4051656542 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f98ec109340 con 0x7f98dc06c470 2024-04-23T02:07:01.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:01.260+0000 7f98f2fdd700 1 -- 172.21.15.71:0/4051656542 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f98dc06c470 msgr2=0x7f98dc06e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:07:01.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:01.260+0000 7f98f2fdd700 1 --2- 172.21.15.71:0/4051656542 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f98dc06c470 0x7f98dc06e930 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f98e0005ea0 tx=0x7f98e0005e30 comp rx=0 tx=0).stop 2024-04-23T02:07:01.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:01.260+0000 7f98f2fdd700 1 -- 172.21.15.71:0/4051656542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0fff50 msgr2=0x7f98ec1056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:07:01.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:01.260+0000 7f98f2fdd700 1 --2- 172.21.15.71:0/4051656542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0fff50 0x7f98ec1056b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f98d8005b40 tx=0x7f98d800be20 comp rx=0 tx=0).stop 2024-04-23T02:07:01.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:01.260+0000 7f98f2fdd700 1 -- 172.21.15.71:0/4051656542 shutdown_connections 2024-04-23T02:07:01.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:01.260+0000 7f98f2fdd700 1 --2- 172.21.15.71:0/4051656542 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f98dc06c470 0x7f98dc06e930 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:07:01.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:01.260+0000 7f98f2fdd700 1 --2- 172.21.15.71:0/4051656542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98ec0ff5f0 0x7f98ec105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:07:01.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:01.260+0000 7f98f2fdd700 1 --2- 172.21.15.71:0/4051656542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0fff50 0x7f98ec1056b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:07:01.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:01.260+0000 7f98f2fdd700 1 -- 172.21.15.71:0/4051656542 >> 172.21.15.71:0/4051656542 conn(0x7f98ec0fb190 msgr2=0x7f98ec107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:07:01.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:01.260+0000 7f98f2fdd700 1 -- 172.21.15.71:0/4051656542 shutdown_connections 2024-04-23T02:07:01.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:01.260+0000 7f98f2fdd700 1 -- 172.21.15.71:0/4051656542 wait complete. 2024-04-23T02:07:01.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:01 smithi071 ceph-mon[39837]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:07:01.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:01 smithi071 ceph-mon[39837]: from='osd.0 [v2:172.21.15.71:6802/275655379,v1:172.21.15.71:6803/275655379]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T02:07:01.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:01 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:01.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:01 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:01.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:01 smithi192 ceph-mon[49662]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:07:01.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:01 smithi192 ceph-mon[49662]: from='osd.0 [v2:172.21.15.71:6802/275655379,v1:172.21.15.71:6803/275655379]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T02:07:01.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:01 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:01.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:01 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:02.007 DEBUG:teuthology.orchestra.run.smithi071:osd.0> sudo journalctl -f -n 0 -u ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.0.service 2024-04-23T02:07:02.009 INFO:tasks.cephadm:Deploying osd.1 on smithi071 with /dev/vg_nvme/lv_3... 2024-04-23T02:07:02.009 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T02:07:02.040 INFO:journalctl@ceph.osd.0.smithi071.stdout:-- Logs begin at Tue 2024-04-23 01:52:06 UTC. -- 2024-04-23T02:07:02.371 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:07:02.856 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:02 smithi071 ceph-mon[39837]: from='osd.0 [v2:172.21.15.71:6802/275655379,v1:172.21.15.71:6803/275655379]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T02:07:02.857 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:02 smithi071 ceph-mon[39837]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T02:07:02.857 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:02 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T02:07:02.857 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:02 smithi071 ceph-mon[39837]: from='osd.0 [v2:172.21.15.71:6802/275655379,v1:172.21.15.71:6803/275655379]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-23T02:07:02.857 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:02 smithi071 ceph-mon[39837]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:07:02.857 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:07:02 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-0[63099]: 2024-04-23T02:07:02.595+0000 7f24d426f700 -1 osd.0 0 waiting for initial osdmap 2024-04-23T02:07:02.857 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:07:02 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-0[63099]: 2024-04-23T02:07:02.599+0000 7f24ce861700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T02:07:02.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:02 smithi192 ceph-mon[49662]: from='osd.0 [v2:172.21.15.71:6802/275655379,v1:172.21.15.71:6803/275655379]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T02:07:02.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:02 smithi192 ceph-mon[49662]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T02:07:02.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:02 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T02:07:02.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:02 smithi192 ceph-mon[49662]: from='osd.0 [v2:172.21.15.71:6802/275655379,v1:172.21.15.71:6803/275655379]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-23T02:07:02.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:02 smithi192 ceph-mon[49662]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:07:03.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:03 smithi192 ceph-mon[49662]: from='osd.0 [v2:172.21.15.71:6802/275655379,v1:172.21.15.71:6803/275655379]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-23T02:07:03.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:03 smithi192 ceph-mon[49662]: osdmap e8: 1 total, 0 up, 1 in 2024-04-23T02:07:03.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:03 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T02:07:03.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:03 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T02:07:03.952 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:03 smithi071 ceph-mon[39837]: from='osd.0 [v2:172.21.15.71:6802/275655379,v1:172.21.15.71:6803/275655379]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-23T02:07:03.952 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:03 smithi071 ceph-mon[39837]: osdmap e8: 1 total, 0 up, 1 in 2024-04-23T02:07:03.952 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:03 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T02:07:03.952 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:03 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T02:07:04.867 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:04 smithi071 ceph-mon[39837]: purged_snaps scrub starts 2024-04-23T02:07:04.867 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:04 smithi071 ceph-mon[39837]: purged_snaps scrub ok 2024-04-23T02:07:04.867 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:04 smithi071 ceph-mon[39837]: osd.0 [v2:172.21.15.71:6802/275655379,v1:172.21.15.71:6803/275655379] boot 2024-04-23T02:07:04.867 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:04 smithi071 ceph-mon[39837]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T02:07:04.867 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:04 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T02:07:04.867 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:04 smithi071 ceph-mon[39837]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:07:04.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:04 smithi192 ceph-mon[49662]: purged_snaps scrub starts 2024-04-23T02:07:04.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:04 smithi192 ceph-mon[49662]: purged_snaps scrub ok 2024-04-23T02:07:04.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:04 smithi192 ceph-mon[49662]: osd.0 [v2:172.21.15.71:6802/275655379,v1:172.21.15.71:6803/275655379] boot 2024-04-23T02:07:04.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:04 smithi192 ceph-mon[49662]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T02:07:04.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:04 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T02:07:04.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:04 smithi192 ceph-mon[49662]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T02:07:05.850 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T02:07:05.851 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T02:07:05.851 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-04-23T02:07:05.851 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-04-23T02:07:05.851 INFO:teuthology.orchestra.run.smithi071.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0131256 s, 799 MB/s 2024-04-23T02:07:05.851 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-04-23T02:07:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:05 smithi192 ceph-mon[49662]: osdmap e10: 1 total, 1 up, 1 in 2024-04-23T02:07:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:05 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:05 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:05.899 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:05 smithi071 ceph-mon[39837]: osdmap e10: 1 total, 1 up, 1 in 2024-04-23T02:07:05.899 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:05.899 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:06.682 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:06 smithi071 ceph-mon[39837]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:06.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:06 smithi192 ceph-mon[49662]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:06.999 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_3 2024-04-23T02:07:08.139 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:07:09.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:09 smithi071 ceph-mon[39837]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:09.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:09 smithi192 ceph-mon[49662]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:10.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.937+0000 7f703c51f700 1 -- 172.21.15.71:0/689224544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70340ffbc0 msgr2=0x7f70340fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:07:10.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.937+0000 7f703c51f700 1 --2- 172.21.15.71:0/689224544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70340ffbc0 0x7f70340fffe0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7024009910 tx=0x7f7024009c20 comp rx=0 tx=0).stop 2024-04-23T02:07:10.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.937+0000 7f703c51f700 1 -- 172.21.15.71:0/689224544 shutdown_connections 2024-04-23T02:07:10.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.937+0000 7f703c51f700 1 --2- 172.21.15.71:0/689224544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7034100dc0 0x7f7034101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:07:10.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.937+0000 7f703c51f700 1 --2- 172.21.15.71:0/689224544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70340ffbc0 0x7f70340fffe0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:07:10.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.937+0000 7f703c51f700 1 -- 172.21.15.71:0/689224544 >> 172.21.15.71:0/689224544 conn(0x7f70340fb160 msgr2=0x7f70340fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:07:10.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.937+0000 7f703c51f700 1 -- 172.21.15.71:0/689224544 shutdown_connections 2024-04-23T02:07:10.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.937+0000 7f703c51f700 1 -- 172.21.15.71:0/689224544 wait complete. 2024-04-23T02:07:10.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.938+0000 7f703c51f700 1 Processor -- start 2024-04-23T02:07:10.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.938+0000 7f703c51f700 1 -- start start 2024-04-23T02:07:10.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.938+0000 7f703c51f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70340ffbc0 0x7f7034195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:07:10.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.938+0000 7f703c51f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7034100dc0 0x7f7034195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:07:10.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.938+0000 7f703c51f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70341961b0 con 0x7f7034100dc0 2024-04-23T02:07:10.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.938+0000 7f703c51f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70341962f0 con 0x7f70340ffbc0 2024-04-23T02:07:10.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.938+0000 7f7039aba700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7034100dc0 0x7f7034195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:07:10.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.938+0000 7f7039aba700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7034100dc0 0x7f7034195b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32932/0 (socket says 172.21.15.71:32932) 2024-04-23T02:07:10.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.938+0000 7f7039aba700 1 -- 172.21.15.71:0/2791224137 learned_addr learned my addr 172.21.15.71:0/2791224137 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:07:10.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.938+0000 7f703a2bb700 1 --2- 172.21.15.71:0/2791224137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70340ffbc0 0x7f7034195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:07:10.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.938+0000 7f7039aba700 1 -- 172.21.15.71:0/2791224137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70340ffbc0 msgr2=0x7f7034195650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:07:10.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.938+0000 7f7039aba700 1 --2- 172.21.15.71:0/2791224137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70340ffbc0 0x7f7034195650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:07:10.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.938+0000 7f7039aba700 1 -- 172.21.15.71:0/2791224137 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70240095f0 con 0x7f7034100dc0 2024-04-23T02:07:10.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.938+0000 7f703a2bb700 1 --2- 172.21.15.71:0/2791224137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70340ffbc0 0x7f7034195650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:07:10.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.938+0000 7f7039aba700 1 --2- 172.21.15.71:0/2791224137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7034100dc0 0x7f7034195b90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f703000d900 tx=0x7f703000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:07:10.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.938+0000 7f702b7fe700 1 -- 172.21.15.71:0/2791224137 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7030004a00 con 0x7f7034100dc0 2024-04-23T02:07:10.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.938+0000 7f702b7fe700 1 -- 172.21.15.71:0/2791224137 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7030005020 con 0x7f7034100dc0 2024-04-23T02:07:10.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.939+0000 7f703c51f700 1 -- 172.21.15.71:0/2791224137 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f703419ada0 con 0x7f7034100dc0 2024-04-23T02:07:10.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.939+0000 7f702b7fe700 1 -- 172.21.15.71:0/2791224137 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7030009c00 con 0x7f7034100dc0 2024-04-23T02:07:10.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.939+0000 7f703c51f700 1 -- 172.21.15.71:0/2791224137 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703419b2c0 con 0x7f7034100dc0 2024-04-23T02:07:10.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.939+0000 7f703c51f700 1 -- 172.21.15.71:0/2791224137 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f703418f410 con 0x7f7034100dc0 2024-04-23T02:07:10.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.940+0000 7f702b7fe700 1 -- 172.21.15.71:0/2791224137 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f7030005190 con 0x7f7034100dc0 2024-04-23T02:07:10.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.940+0000 7f702b7fe700 1 --2- 172.21.15.71:0/2791224137 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f702006c460 0x7f702006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:07:10.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.940+0000 7f702b7fe700 1 -- 172.21.15.71:0/2791224137 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f703008aa40 con 0x7f7034100dc0 2024-04-23T02:07:10.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.940+0000 7f703a2bb700 1 --2- 172.21.15.71:0/2791224137 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f702006c460 0x7f702006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:07:10.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.941+0000 7f703a2bb700 1 --2- 172.21.15.71:0/2791224137 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f702006c460 0x7f702006e920 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7024006010 tx=0x7f7024005a90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:07:10.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:10.945+0000 7f702b7fe700 1 -- 172.21.15.71:0/2791224137 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f703008ce60 con 0x7f7034100dc0 2024-04-23T02:07:11.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:10 smithi192 ceph-mon[49662]: Detected new or changed devices on smithi071 2024-04-23T02:07:11.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:11.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:11.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:07:11.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:10 smithi192 ceph-mon[49662]: Adjusting osd_memory_target on smithi071 to 11874M 2024-04-23T02:07:11.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:11.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:07:11.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:07:11.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:11.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:10 smithi192 ceph-mon[49662]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:11.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:07:11.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:11.150+0000 7f703c51f700 1 -- 172.21.15.71:0/2791224137 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f703402d050 con 0x7f702006c460 2024-04-23T02:07:11.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:10 smithi071 ceph-mon[39837]: Detected new or changed devices on smithi071 2024-04-23T02:07:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:07:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:10 smithi071 ceph-mon[39837]: Adjusting osd_memory_target on smithi071 to 11874M 2024-04-23T02:07:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:07:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:07:11.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:11.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:10 smithi071 ceph-mon[39837]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:11.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:07:12.263 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:11 smithi071 ceph-mon[39837]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:07:12.263 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:11 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T02:07:12.263 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:11 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T02:07:12.263 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:11 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:07:12.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:11 smithi192 ceph-mon[49662]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:07:12.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:11 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T02:07:12.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:11 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T02:07:12.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:11 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:07:13.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:13 smithi071 ceph-mon[39837]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:13.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:13 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:07:13.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:13 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:13.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:13 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:13.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:13 smithi192 ceph-mon[49662]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:13.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:13 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:07:13.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:13 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:13.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:13 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:15.491 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:15 smithi071 ceph-mon[39837]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:15.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:15 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3906536034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04304fd2-42f9-4525-88e8-c7e20c9c51e6"}]: dispatch 2024-04-23T02:07:15.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:15 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3906536034' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04304fd2-42f9-4525-88e8-c7e20c9c51e6"}]': finished 2024-04-23T02:07:15.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:15 smithi071 ceph-mon[39837]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T02:07:15.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T02:07:15.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:15 smithi192 ceph-mon[49662]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:15.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:15 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/3906536034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04304fd2-42f9-4525-88e8-c7e20c9c51e6"}]: dispatch 2024-04-23T02:07:15.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:15 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/3906536034' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04304fd2-42f9-4525-88e8-c7e20c9c51e6"}]': finished 2024-04-23T02:07:15.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:15 smithi192 ceph-mon[49662]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T02:07:15.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T02:07:16.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:16 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/712667717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T02:07:16.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:16 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/712667717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T02:07:17.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:17 smithi071 ceph-mon[39837]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:17.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:17 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:17.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:17 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:17.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:17 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:07:17.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:17 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:07:17.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:17 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:17.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:17 smithi192 ceph-mon[49662]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:17.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:17 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:17.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:17 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:17.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:17 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:07:17.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:17 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:07:17.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:17 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:19.373 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:19 smithi071 ceph-mon[39837]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:19.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:19 smithi192 ceph-mon[49662]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:21.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:21 smithi071 ceph-mon[39837]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:21.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:21 smithi192 ceph-mon[49662]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:23 smithi071 ceph-mon[39837]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:23 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T02:07:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:23 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:07:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:23 smithi071 ceph-mon[39837]: Deploying daemon osd.1 on smithi071 2024-04-23T02:07:23.564 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:23 smithi192 ceph-mon[49662]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:23.564 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:23 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T02:07:23.564 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:23 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:07:23.564 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:23 smithi192 ceph-mon[49662]: Deploying daemon osd.1 on smithi071 2024-04-23T02:07:25.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:25 smithi071 ceph-mon[39837]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:25.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:25 smithi192 ceph-mon[49662]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:26.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:26 smithi192 ceph-mon[49662]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:26.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:26 smithi071 ceph-mon[39837]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:27.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:27 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:07:27.907 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:27 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:07:29.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:29 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:29.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:29 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:29.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:29 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:07:29.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:29 smithi071 ceph-mon[39837]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:29.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:29 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:29.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:29 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:29.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:29 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:07:29.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:29 smithi192 ceph-mon[49662]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:31.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:31 smithi071 ceph-mon[39837]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:31.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:31.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:31.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:31 smithi192 ceph-mon[49662]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:31.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:31.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:31.813 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 1 on host 'smithi071' 2024-04-23T02:07:31.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:31.809+0000 7f702b7fe700 1 -- 172.21.15.71:0/2791224137 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f703402d050 con 0x7f702006c460 2024-04-23T02:07:31.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:31.810+0000 7f703c51f700 1 -- 172.21.15.71:0/2791224137 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f702006c460 msgr2=0x7f702006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:07:31.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:31.810+0000 7f703c51f700 1 --2- 172.21.15.71:0/2791224137 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f702006c460 0x7f702006e920 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7024006010 tx=0x7f7024005a90 comp rx=0 tx=0).stop 2024-04-23T02:07:31.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:31.810+0000 7f703c51f700 1 -- 172.21.15.71:0/2791224137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7034100dc0 msgr2=0x7f7034195b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:07:31.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:31.810+0000 7f703c51f700 1 --2- 172.21.15.71:0/2791224137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7034100dc0 0x7f7034195b90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f703000d900 tx=0x7f703000dcc0 comp rx=0 tx=0).stop 2024-04-23T02:07:31.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:31.811+0000 7f703c51f700 1 -- 172.21.15.71:0/2791224137 shutdown_connections 2024-04-23T02:07:31.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:31.811+0000 7f703c51f700 1 --2- 172.21.15.71:0/2791224137 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f702006c460 0x7f702006e920 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:07:31.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:31.811+0000 7f703c51f700 1 --2- 172.21.15.71:0/2791224137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70340ffbc0 0x7f7034195650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:07:31.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:31.811+0000 7f703c51f700 1 --2- 172.21.15.71:0/2791224137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7034100dc0 0x7f7034195b90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:07:31.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:31.811+0000 7f703c51f700 1 -- 172.21.15.71:0/2791224137 >> 172.21.15.71:0/2791224137 conn(0x7f70340fb160 msgr2=0x7f7034104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:07:31.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:31.811+0000 7f703c51f700 1 -- 172.21.15.71:0/2791224137 shutdown_connections 2024-04-23T02:07:31.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:31.811+0000 7f703c51f700 1 -- 172.21.15.71:0/2791224137 wait complete. 2024-04-23T02:07:32.772 DEBUG:teuthology.orchestra.run.smithi071:osd.1> sudo journalctl -f -n 0 -u ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.1.service 2024-04-23T02:07:32.774 INFO:tasks.cephadm:Deploying osd.2 on smithi071 with /dev/vg_nvme/lv_2... 2024-04-23T02:07:32.774 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T02:07:32.804 INFO:journalctl@ceph.osd.1.smithi071.stdout:-- Logs begin at Tue 2024-04-23 01:52:06 UTC. -- 2024-04-23T02:07:32.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:32 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:32.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:32 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:32.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:32 smithi071 ceph-mon[39837]: from='osd.1 [v2:172.21.15.71:6810/3686410016,v1:172.21.15.71:6811/3686410016]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T02:07:32.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:32 smithi071 ceph-mon[39837]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:33.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:32 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:33.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:32 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:33.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:32 smithi192 ceph-mon[49662]: from='osd.1 [v2:172.21.15.71:6810/3686410016,v1:172.21.15.71:6811/3686410016]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T02:07:33.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:32 smithi192 ceph-mon[49662]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:33.487 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:07:33.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:33 smithi192 ceph-mon[49662]: from='osd.1 [v2:172.21.15.71:6810/3686410016,v1:172.21.15.71:6811/3686410016]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T02:07:33.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:33 smithi192 ceph-mon[49662]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T02:07:33.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:33 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T02:07:33.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:33 smithi192 ceph-mon[49662]: from='osd.1 [v2:172.21.15.71:6810/3686410016,v1:172.21.15.71:6811/3686410016]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-23T02:07:34.163 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:07:33 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-1[72855]: 2024-04-23T02:07:33.769+0000 7f8d74987700 -1 osd.1 0 waiting for initial osdmap 2024-04-23T02:07:34.164 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:07:33 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-1[72855]: 2024-04-23T02:07:33.777+0000 7f8d70f7d700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T02:07:34.164 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:33 smithi071 ceph-mon[39837]: from='osd.1 [v2:172.21.15.71:6810/3686410016,v1:172.21.15.71:6811/3686410016]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T02:07:34.164 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:33 smithi071 ceph-mon[39837]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T02:07:34.164 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:33 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T02:07:34.164 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:33 smithi071 ceph-mon[39837]: from='osd.1 [v2:172.21.15.71:6810/3686410016,v1:172.21.15.71:6811/3686410016]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-23T02:07:34.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:34 smithi071 ceph-mon[39837]: from='osd.1 [v2:172.21.15.71:6810/3686410016,v1:172.21.15.71:6811/3686410016]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-23T02:07:34.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:34 smithi071 ceph-mon[39837]: osdmap e13: 2 total, 1 up, 2 in 2024-04-23T02:07:34.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:34 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T02:07:34.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:34 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T02:07:34.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:34 smithi071 ceph-mon[39837]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:34.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:34 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T02:07:35.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:34 smithi192 ceph-mon[49662]: from='osd.1 [v2:172.21.15.71:6810/3686410016,v1:172.21.15.71:6811/3686410016]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-23T02:07:35.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:34 smithi192 ceph-mon[49662]: osdmap e13: 2 total, 1 up, 2 in 2024-04-23T02:07:35.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:34 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T02:07:35.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:34 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T02:07:35.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:34 smithi192 ceph-mon[49662]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-23T02:07:35.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:34 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T02:07:36.019 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:35 smithi071 ceph-mon[39837]: purged_snaps scrub starts 2024-04-23T02:07:36.020 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:35 smithi071 ceph-mon[39837]: purged_snaps scrub ok 2024-04-23T02:07:36.020 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:35 smithi071 ceph-mon[39837]: osd.1 [v2:172.21.15.71:6810/3686410016,v1:172.21.15.71:6811/3686410016] boot 2024-04-23T02:07:36.020 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:35 smithi071 ceph-mon[39837]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T02:07:36.020 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:35 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T02:07:36.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:35 smithi192 ceph-mon[49662]: purged_snaps scrub starts 2024-04-23T02:07:36.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:35 smithi192 ceph-mon[49662]: purged_snaps scrub ok 2024-04-23T02:07:36.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:35 smithi192 ceph-mon[49662]: osd.1 [v2:172.21.15.71:6810/3686410016,v1:172.21.15.71:6811/3686410016] boot 2024-04-23T02:07:36.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:35 smithi192 ceph-mon[49662]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T02:07:36.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:35 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T02:07:36.942 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T02:07:36.942 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T02:07:36.942 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-04-23T02:07:36.942 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-04-23T02:07:36.942 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111887 s, 937 MB/s 2024-04-23T02:07:36.942 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-04-23T02:07:37.054 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:36 smithi071 ceph-mon[39837]: osdmap e15: 2 total, 2 up, 2 in 2024-04-23T02:07:37.054 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:36 smithi071 ceph-mon[39837]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:37.054 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:36 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:37.054 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:36 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:37.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:36 smithi192 ceph-mon[49662]: osdmap e15: 2 total, 2 up, 2 in 2024-04-23T02:07:37.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:36 smithi192 ceph-mon[49662]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:37.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:36 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:37.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:36 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:38.524 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_2 2024-04-23T02:07:39.030 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:07:39.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:39 smithi071 ceph-mon[39837]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:39.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:39 smithi192 ceph-mon[49662]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:41.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:41 smithi071 ceph-mon[39837]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:41.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:41 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:41.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:41 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:41.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:41 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:07:41.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:41 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:07:41.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:41 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:41.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:41 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:07:41.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:41 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:07:41.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:41 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:41.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:41 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:07:41.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:41 smithi192 ceph-mon[49662]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:41.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:41 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:41.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:41 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:41.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:41 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:07:41.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:41 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:07:41.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:41 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:41.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:41 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:07:41.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:41 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:07:41.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:41 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:41.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:41 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:07:41.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.809+0000 7f3fc83fb700 1 -- 172.21.15.71:0/2771672636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0101f40 msgr2=0x7f3fc01023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:07:41.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.809+0000 7f3fc83fb700 1 --2- 172.21.15.71:0/2771672636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0101f40 0x7f3fc01023a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009910 tx=0x7f3fb0009c20 comp rx=0 tx=0).stop 2024-04-23T02:07:41.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.809+0000 7f3fc83fb700 1 -- 172.21.15.71:0/2771672636 shutdown_connections 2024-04-23T02:07:41.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.809+0000 7f3fc83fb700 1 --2- 172.21.15.71:0/2771672636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0101f40 0x7f3fc01023a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:07:41.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.809+0000 7f3fc83fb700 1 --2- 172.21.15.71:0/2771672636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3fc00fef60 0x7f3fc00ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:07:41.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.809+0000 7f3fc83fb700 1 -- 172.21.15.71:0/2771672636 >> 172.21.15.71:0/2771672636 conn(0x7f3fc00fa8c0 msgr2=0x7f3fc00fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:07:41.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.809+0000 7f3fc83fb700 1 -- 172.21.15.71:0/2771672636 shutdown_connections 2024-04-23T02:07:41.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.809+0000 7f3fc83fb700 1 -- 172.21.15.71:0/2771672636 wait complete. 2024-04-23T02:07:41.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.810+0000 7f3fc83fb700 1 Processor -- start 2024-04-23T02:07:41.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.810+0000 7f3fc83fb700 1 -- start start 2024-04-23T02:07:41.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.810+0000 7f3fc83fb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3fc00fef60 0x7f3fc01963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:07:41.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.810+0000 7f3fc83fb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0101f40 0x7f3fc0196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:07:41.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.810+0000 7f3fc83fb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc0196f30 con 0x7f3fc0101f40 2024-04-23T02:07:41.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.810+0000 7f3fc83fb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc0197070 con 0x7f3fc00fef60 2024-04-23T02:07:41.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.810+0000 7f3fc5996700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0101f40 0x7f3fc0196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:07:41.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.810+0000 7f3fc5996700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0101f40 0x7f3fc0196910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47336/0 (socket says 172.21.15.71:47336) 2024-04-23T02:07:41.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.810+0000 7f3fc5996700 1 -- 172.21.15.71:0/4031086836 learned_addr learned my addr 172.21.15.71:0/4031086836 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:07:41.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.810+0000 7f3fc6197700 1 --2- 172.21.15.71:0/4031086836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3fc00fef60 0x7f3fc01963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:07:41.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.810+0000 7f3fc5996700 1 -- 172.21.15.71:0/4031086836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3fc00fef60 msgr2=0x7f3fc01963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:07:41.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.810+0000 7f3fc5996700 1 --2- 172.21.15.71:0/4031086836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3fc00fef60 0x7f3fc01963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:07:41.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.810+0000 7f3fc5996700 1 -- 172.21.15.71:0/4031086836 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb00095f0 con 0x7f3fc0101f40 2024-04-23T02:07:41.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.811+0000 7f3fc5996700 1 --2- 172.21.15.71:0/4031086836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0101f40 0x7f3fc0196910 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000bd70 tx=0x7f3fb000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:07:41.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.811+0000 7f3fb77fe700 1 -- 172.21.15.71:0/4031086836 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fb001d070 con 0x7f3fc0101f40 2024-04-23T02:07:41.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.811+0000 7f3fc83fb700 1 -- 172.21.15.71:0/4031086836 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc019bac0 con 0x7f3fc0101f40 2024-04-23T02:07:41.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.811+0000 7f3fb77fe700 1 -- 172.21.15.71:0/4031086836 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f3fb000f460 con 0x7f3fc0101f40 2024-04-23T02:07:41.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.811+0000 7f3fb77fe700 1 -- 172.21.15.71:0/4031086836 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fb0021640 con 0x7f3fc0101f40 2024-04-23T02:07:41.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.811+0000 7f3fc83fb700 1 -- 172.21.15.71:0/4031086836 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc019bfb0 con 0x7f3fc0101f40 2024-04-23T02:07:41.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.812+0000 7f3fc83fb700 1 -- 172.21.15.71:0/4031086836 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc01096c0 con 0x7f3fc0101f40 2024-04-23T02:07:41.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.812+0000 7f3fb77fe700 1 -- 172.21.15.71:0/4031086836 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f3fb000fa20 con 0x7f3fc0101f40 2024-04-23T02:07:41.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.813+0000 7f3fb77fe700 1 --2- 172.21.15.71:0/4031086836 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f3fac06c460 0x7f3fac06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:07:41.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.813+0000 7f3fb77fe700 1 -- 172.21.15.71:0/4031086836 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7f3fb008bd60 con 0x7f3fc0101f40 2024-04-23T02:07:41.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.813+0000 7f3fc6197700 1 --2- 172.21.15.71:0/4031086836 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f3fac06c460 0x7f3fac06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:07:41.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.813+0000 7f3fc6197700 1 --2- 172.21.15.71:0/4031086836 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f3fac06c460 0x7f3fac06e920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0068380 tx=0x7f3fbc007400 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:07:41.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:41.817+0000 7f3fb77fe700 1 -- 172.21.15.71:0/4031086836 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3fb005aca0 con 0x7f3fc0101f40 2024-04-23T02:07:42.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:07:42.015+0000 7f3fc83fb700 1 -- 172.21.15.71:0/4031086836 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fc0106890 con 0x7f3fac06c460 2024-04-23T02:07:42.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:42 smithi071 ceph-mon[39837]: Detected new or changed devices on smithi071 2024-04-23T02:07:42.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:42 smithi071 ceph-mon[39837]: Adjusting osd_memory_target on smithi071 to 5937M 2024-04-23T02:07:42.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T02:07:42.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T02:07:42.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:07:42.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:07:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:42 smithi192 ceph-mon[49662]: Detected new or changed devices on smithi071 2024-04-23T02:07:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:42 smithi192 ceph-mon[49662]: Adjusting osd_memory_target on smithi071 to 5937M 2024-04-23T02:07:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:42 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T02:07:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:42 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T02:07:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:42 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:07:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:42 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:07:43.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:43 smithi071 ceph-mon[39837]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:07:43.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:43 smithi071 ceph-mon[39837]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:43.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:43 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:43.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:43 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:43.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:43 smithi192 ceph-mon[49662]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:07:43.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:43 smithi192 ceph-mon[49662]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:43.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:43 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:43.590 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:43 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:45.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:45 smithi071 ceph-mon[39837]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:45.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:45 smithi192 ceph-mon[49662]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:46.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:46 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1890912497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6797c77c-4ccc-4489-9269-77105d7c45c0"}]: dispatch 2024-04-23T02:07:46.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:46 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1890912497' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6797c77c-4ccc-4489-9269-77105d7c45c0"}]': finished 2024-04-23T02:07:46.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:46 smithi071 ceph-mon[39837]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T02:07:46.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:46 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T02:07:46.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:46 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1890912497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6797c77c-4ccc-4489-9269-77105d7c45c0"}]: dispatch 2024-04-23T02:07:46.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:46 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1890912497' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6797c77c-4ccc-4489-9269-77105d7c45c0"}]': finished 2024-04-23T02:07:46.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:46 smithi192 ceph-mon[49662]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T02:07:46.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:46 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T02:07:47.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:47 smithi071 ceph-mon[39837]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:47.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:47 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/857370253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T02:07:47.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:47 smithi192 ceph-mon[49662]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:47.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:47 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/857370253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T02:07:48.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:48 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:48.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:48 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:48.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:48 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:07:48.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:48 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:07:48.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:48 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:48.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:48 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:48.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:48 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:48.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:48 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:07:48.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:48 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:07:48.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:48 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:07:49.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:49 smithi192 ceph-mon[49662]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:49.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:49 smithi071 ceph-mon[39837]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:51.850 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:51 smithi071 ceph-mon[39837]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:51.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:51 smithi192 ceph-mon[49662]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:53.788 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:53 smithi071 ceph-mon[39837]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:53.788 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:53 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T02:07:53.788 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:53 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:07:53.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:53 smithi192 ceph-mon[49662]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:53.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:53 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T02:07:53.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:53 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:07:54.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:54 smithi192 ceph-mon[49662]: Deploying daemon osd.2 on smithi071 2024-04-23T02:07:54.887 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:54 smithi071 ceph-mon[39837]: Deploying daemon osd.2 on smithi071 2024-04-23T02:07:55.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:55 smithi192 ceph-mon[49662]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:55.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:55 smithi071 ceph-mon[39837]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:56.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:56 smithi071 ceph-mon[39837]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:57.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:56 smithi192 ceph-mon[49662]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:57.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:07:58.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:07:58.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:07:58 smithi192 ceph-mon[49662]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:07:58.964 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:07:58 smithi071 ceph-mon[39837]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:08:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:00 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:00 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:00 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:08:00.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:00 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:00.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:00 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:00.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:00 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:08:01.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:01 smithi071 ceph-mon[39837]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:08:01.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:01 smithi192 ceph-mon[49662]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:08:03.381 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 2 on host 'smithi071' 2024-04-23T02:08:03.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:08:03.377+0000 7f3fb77fe700 1 -- 172.21.15.71:0/4031086836 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3fc0106890 con 0x7f3fac06c460 2024-04-23T02:08:03.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:08:03.379+0000 7f3fc83fb700 1 -- 172.21.15.71:0/4031086836 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f3fac06c460 msgr2=0x7f3fac06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:08:03.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:08:03.379+0000 7f3fc83fb700 1 --2- 172.21.15.71:0/4031086836 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f3fac06c460 0x7f3fac06e920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0068380 tx=0x7f3fbc007400 comp rx=0 tx=0).stop 2024-04-23T02:08:03.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:08:03.380+0000 7f3fc83fb700 1 -- 172.21.15.71:0/4031086836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0101f40 msgr2=0x7f3fc0196910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:08:03.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:08:03.380+0000 7f3fc83fb700 1 --2- 172.21.15.71:0/4031086836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0101f40 0x7f3fc0196910 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000bd70 tx=0x7f3fb000bda0 comp rx=0 tx=0).stop 2024-04-23T02:08:03.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:08:03.380+0000 7f3fc83fb700 1 -- 172.21.15.71:0/4031086836 shutdown_connections 2024-04-23T02:08:03.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:08:03.380+0000 7f3fc83fb700 1 --2- 172.21.15.71:0/4031086836 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f3fac06c460 0x7f3fac06e920 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:08:03.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:08:03.380+0000 7f3fc83fb700 1 --2- 172.21.15.71:0/4031086836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3fc00fef60 0x7f3fc01963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:08:03.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:08:03.380+0000 7f3fc83fb700 1 --2- 172.21.15.71:0/4031086836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0101f40 0x7f3fc0196910 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:08:03.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:08:03.380+0000 7f3fc83fb700 1 -- 172.21.15.71:0/4031086836 >> 172.21.15.71:0/4031086836 conn(0x7f3fc00fa8c0 msgr2=0x7f3fc0105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:08:03.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:08:03.380+0000 7f3fc83fb700 1 -- 172.21.15.71:0/4031086836 shutdown_connections 2024-04-23T02:08:03.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:08:03.380+0000 7f3fc83fb700 1 -- 172.21.15.71:0/4031086836 wait complete. 2024-04-23T02:08:03.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:03 smithi071 ceph-mon[39837]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:08:03.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:03 smithi071 ceph-mon[39837]: from='osd.2 [v2:172.21.15.71:6818/2096213030,v1:172.21.15.71:6819/2096213030]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T02:08:03.615 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:03 smithi192 ceph-mon[49662]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:08:03.615 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:03 smithi192 ceph-mon[49662]: from='osd.2 [v2:172.21.15.71:6818/2096213030,v1:172.21.15.71:6819/2096213030]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T02:08:04.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:04 smithi071 ceph-mon[39837]: from='osd.2 [v2:172.21.15.71:6818/2096213030,v1:172.21.15.71:6819/2096213030]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T02:08:04.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:04 smithi071 ceph-mon[39837]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T02:08:04.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:04 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T02:08:04.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:04 smithi071 ceph-mon[39837]: from='osd.2 [v2:172.21.15.71:6818/2096213030,v1:172.21.15.71:6819/2096213030]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-23T02:08:04.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:04 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:04.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:04 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:04.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:04 smithi192 ceph-mon[49662]: from='osd.2 [v2:172.21.15.71:6818/2096213030,v1:172.21.15.71:6819/2096213030]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T02:08:04.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:04 smithi192 ceph-mon[49662]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T02:08:04.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:04 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T02:08:04.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:04 smithi192 ceph-mon[49662]: from='osd.2 [v2:172.21.15.71:6818/2096213030,v1:172.21.15.71:6819/2096213030]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-23T02:08:04.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:04 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:04.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:04 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:05.026 DEBUG:teuthology.orchestra.run.smithi071:osd.2> sudo journalctl -f -n 0 -u ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.2.service 2024-04-23T02:08:05.028 INFO:tasks.cephadm:Deploying osd.3 on smithi192 with /dev/vg_nvme/lv_4... 2024-04-23T02:08:05.029 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T02:08:05.058 INFO:journalctl@ceph.osd.2.smithi071.stdout:-- Logs begin at Tue 2024-04-23 01:52:06 UTC. -- 2024-04-23T02:08:05.261 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:08:05.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:05 smithi192 ceph-mon[49662]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:08:05.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:05 smithi192 ceph-mon[49662]: from='osd.2 [v2:172.21.15.71:6818/2096213030,v1:172.21.15.71:6819/2096213030]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-23T02:08:05.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:05 smithi192 ceph-mon[49662]: osdmap e18: 3 total, 2 up, 3 in 2024-04-23T02:08:05.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:05 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T02:08:05.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:05 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T02:08:05.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:05 smithi071 ceph-mon[39837]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:08:05.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:05 smithi071 ceph-mon[39837]: from='osd.2 [v2:172.21.15.71:6818/2096213030,v1:172.21.15.71:6819/2096213030]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-23T02:08:05.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:05 smithi071 ceph-mon[39837]: osdmap e18: 3 total, 2 up, 3 in 2024-04-23T02:08:05.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T02:08:05.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T02:08:06.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:06 smithi071 ceph-mon[39837]: purged_snaps scrub starts 2024-04-23T02:08:06.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:06 smithi071 ceph-mon[39837]: purged_snaps scrub ok 2024-04-23T02:08:06.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:06 smithi071 ceph-mon[39837]: osd.2 [v2:172.21.15.71:6818/2096213030,v1:172.21.15.71:6819/2096213030] boot 2024-04-23T02:08:06.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:06 smithi071 ceph-mon[39837]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T02:08:06.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:06 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T02:08:06.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:06 smithi192 ceph-mon[49662]: purged_snaps scrub starts 2024-04-23T02:08:06.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:06 smithi192 ceph-mon[49662]: purged_snaps scrub ok 2024-04-23T02:08:06.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:06 smithi192 ceph-mon[49662]: osd.2 [v2:172.21.15.71:6818/2096213030,v1:172.21.15.71:6819/2096213030] boot 2024-04-23T02:08:06.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:06 smithi192 ceph-mon[49662]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T02:08:06.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:06 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T02:08:07.431 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T02:08:07.431 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T02:08:07.431 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-04-23T02:08:07.431 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-04-23T02:08:07.431 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116795 s, 898 MB/s 2024-04-23T02:08:07.431 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-04-23T02:08:07.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:07 smithi071 ceph-mon[39837]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:08:07.777 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:07 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T02:08:07.777 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:07 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:07.777 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:07 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:07.777 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:07 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T02:08:07.777 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:07 smithi071 ceph-mon[39837]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T02:08:07.777 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:07 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T02:08:07.806 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:07 smithi192 ceph-mon[49662]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-23T02:08:07.806 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:07 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T02:08:07.806 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:07 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:07.806 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:07 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:07.806 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:07 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T02:08:07.806 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:07 smithi192 ceph-mon[49662]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T02:08:07.806 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:07 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T02:08:08.165 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi192:vg_nvme/lv_4 2024-04-23T02:08:08.243 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:08:08 smithi071 sudo[84181]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T02:08:08.243 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:08:08 smithi071 sudo[84181]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T02:08:08.243 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:08:08 smithi071 sudo[84181]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T02:08:08.243 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:08:08 smithi071 sudo[84181]: pam_unix(sudo:session): session closed for user root 2024-04-23T02:08:08.385 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:08:08.637 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:08:08 smithi071 sudo[84185]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T02:08:08.638 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:08:08 smithi071 sudo[84185]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T02:08:08.638 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:08:08 smithi071 sudo[84185]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T02:08:08.638 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:08:08 smithi071 sudo[84185]: pam_unix(sudo:session): session closed for user root 2024-04-23T02:08:08.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:08 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T02:08:08.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:08 smithi192 ceph-mon[49662]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T02:08:08.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:08 smithi192 ceph-mon[49662]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:08.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:08 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:08.921 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:08:08 smithi071 sudo[84212]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T02:08:08.921 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:08:08 smithi071 sudo[84212]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T02:08:08.921 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:08:08 smithi071 sudo[84212]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T02:08:08.921 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:08:08 smithi071 sudo[84212]: pam_unix(sudo:session): session closed for user root 2024-04-23T02:08:08.921 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:08 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T02:08:08.922 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:08 smithi071 ceph-mon[39837]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T02:08:08.922 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:08 smithi071 ceph-mon[39837]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:08.922 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:08 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:09.185 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:08:08 smithi071 sudo[84242]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T02:08:09.186 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:08:08 smithi071 sudo[84242]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T02:08:09.186 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:08:08 smithi071 sudo[84242]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T02:08:09.186 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:08:08 smithi071 sudo[84242]: pam_unix(sudo:session): session closed for user root 2024-04-23T02:08:09.186 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:08:09 smithi071 sudo[84254]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T02:08:09.186 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:08:09 smithi071 sudo[84254]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T02:08:09.187 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:08:09 smithi071 sudo[84254]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T02:08:09.187 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:08:09 smithi071 sudo[84254]: pam_unix(sudo:session): session closed for user root 2024-04-23T02:08:09.493 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:08:09 smithi071 sudo[85629]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T02:08:09.494 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:08:09 smithi071 sudo[85629]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T02:08:09.494 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:08:09 smithi071 sudo[85629]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T02:08:09.494 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:08:09 smithi071 sudo[85629]: pam_unix(sudo:session): session closed for user root 2024-04-23T02:08:09.950 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:09 smithi071 sudo[88176]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T02:08:09.950 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:09 smithi071 sudo[88176]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T02:08:09.950 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:09 smithi071 sudo[88176]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T02:08:09.950 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:09 smithi071 sudo[88176]: pam_unix(sudo:session): session closed for user root 2024-04-23T02:08:10.216 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:09 smithi192 ceph-mon[49662]: osdmap e22: 3 total, 3 up, 3 in 2024-04-23T02:08:10.216 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:09 smithi192 ceph-mon[49662]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T02:08:10.216 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 sudo[53925]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T02:08:10.216 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 sudo[53925]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T02:08:10.216 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 sudo[53925]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T02:08:10.216 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 sudo[53925]: pam_unix(sudo:session): session closed for user root 2024-04-23T02:08:10.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:09 smithi071 sudo[88198]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T02:08:10.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:09 smithi071 sudo[88198]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T02:08:10.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:09 smithi071 sudo[88198]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T02:08:10.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:09 smithi071 sudo[88198]: pam_unix(sudo:session): session closed for user root 2024-04-23T02:08:10.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:09 smithi071 ceph-mon[39837]: osdmap e22: 3 total, 3 up, 3 in 2024-04-23T02:08:10.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:09 smithi071 ceph-mon[39837]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T02:08:10.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.209+0000 7f0acbc37700 1 -- 172.21.15.192:0/2804438878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ac4104120 msgr2=0x7f0ac4106510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:08:10.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.209+0000 7f0acbc37700 1 --2- 172.21.15.192:0/2804438878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ac4104120 0x7f0ac4106510 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009910 tx=0x7f0ab4009c20 comp rx=0 tx=0).stop 2024-04-23T02:08:10.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.209+0000 7f0acbc37700 1 -- 172.21.15.192:0/2804438878 shutdown_connections 2024-04-23T02:08:10.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.209+0000 7f0acbc37700 1 --2- 172.21.15.192:0/2804438878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ac4104120 0x7f0ac4106510 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:08:10.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.209+0000 7f0acbc37700 1 --2- 172.21.15.192:0/2804438878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ac41017f0 0x7f0ac4103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:08:10.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.209+0000 7f0acbc37700 1 -- 172.21.15.192:0/2804438878 >> 172.21.15.192:0/2804438878 conn(0x7f0ac40fb120 msgr2=0x7f0ac40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:08:10.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.209+0000 7f0acbc37700 1 -- 172.21.15.192:0/2804438878 shutdown_connections 2024-04-23T02:08:10.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.209+0000 7f0acbc37700 1 -- 172.21.15.192:0/2804438878 wait complete. 2024-04-23T02:08:10.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.210+0000 7f0acbc37700 1 Processor -- start 2024-04-23T02:08:10.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.210+0000 7f0acbc37700 1 -- start start 2024-04-23T02:08:10.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.210+0000 7f0acbc37700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ac41017f0 0x7f0ac41963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:08:10.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.210+0000 7f0acbc37700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ac4104120 0x7f0ac41968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:08:10.258 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.210+0000 7f0acbc37700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac4196f10 con 0x7f0ac4104120 2024-04-23T02:08:10.258 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.210+0000 7f0acbc37700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac4197050 con 0x7f0ac41017f0 2024-04-23T02:08:10.258 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.211+0000 7f0ac99d3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ac41017f0 0x7f0ac41963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:08:10.258 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.211+0000 7f0ac99d3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ac41017f0 0x7f0ac41963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:37842/0 (socket says 172.21.15.192:37842) 2024-04-23T02:08:10.258 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.211+0000 7f0ac99d3700 1 -- 172.21.15.192:0/560934707 learned_addr learned my addr 172.21.15.192:0/560934707 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:08:10.258 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.211+0000 7f0ac91d2700 1 --2- 172.21.15.192:0/560934707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ac4104120 0x7f0ac41968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:08:10.258 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.211+0000 7f0ac99d3700 1 -- 172.21.15.192:0/560934707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ac4104120 msgr2=0x7f0ac41968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:08:10.258 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.211+0000 7f0ac99d3700 1 --2- 172.21.15.192:0/560934707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ac4104120 0x7f0ac41968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:08:10.258 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.211+0000 7f0ac99d3700 1 -- 172.21.15.192:0/560934707 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab40095f0 con 0x7f0ac41017f0 2024-04-23T02:08:10.258 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.211+0000 7f0ac99d3700 1 --2- 172.21.15.192:0/560934707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ac41017f0 0x7f0ac41963b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000e920 tx=0x7f0ac000ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:08:10.456 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.452+0000 7f0abaffd700 1 -- 172.21.15.192:0/560934707 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ac000c9d0 con 0x7f0ac41017f0 2024-04-23T02:08:10.456 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.452+0000 7f0abaffd700 1 -- 172.21.15.192:0/560934707 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f0ac0010430 con 0x7f0ac41017f0 2024-04-23T02:08:10.456 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.452+0000 7f0acbc37700 1 -- 172.21.15.192:0/560934707 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac419bb00 con 0x7f0ac41017f0 2024-04-23T02:08:10.456 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.452+0000 7f0abaffd700 1 -- 172.21.15.192:0/560934707 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ac0018630 con 0x7f0ac41017f0 2024-04-23T02:08:10.456 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.453+0000 7f0acbc37700 1 -- 172.21.15.192:0/560934707 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac419c050 con 0x7f0ac41017f0 2024-04-23T02:08:10.457 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.454+0000 7f0acbc37700 1 -- 172.21.15.192:0/560934707 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac41096c0 con 0x7f0ac41017f0 2024-04-23T02:08:10.457 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.454+0000 7f0abaffd700 1 -- 172.21.15.192:0/560934707 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f0ac00105a0 con 0x7f0ac41017f0 2024-04-23T02:08:10.458 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.454+0000 7f0abaffd700 1 --2- 172.21.15.192:0/560934707 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0ab006c530 0x7f0ab006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:08:10.458 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.454+0000 7f0abaffd700 1 -- 172.21.15.192:0/560934707 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3232+0+0 (secure 0 0 0) 0x7f0ac0014070 con 0x7f0ac41017f0 2024-04-23T02:08:10.458 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.454+0000 7f0ac91d2700 1 --2- 172.21.15.192:0/560934707 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0ab006c530 0x7f0ab006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:08:10.458 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.455+0000 7f0ac91d2700 1 --2- 172.21.15.192:0/560934707 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0ab006c530 0x7f0ab006e9f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009fd0 tx=0x7f0ab400b5a0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:08:10.462 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.459+0000 7f0abaffd700 1 -- 172.21.15.192:0/560934707 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0ac00520e0 con 0x7f0ac41017f0 2024-04-23T02:08:10.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 sudo[53945]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T02:08:10.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 sudo[53945]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T02:08:10.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 sudo[53945]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T02:08:10.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 sudo[53945]: pam_unix(sudo:session): session closed for user root 2024-04-23T02:08:10.662 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:10.658+0000 7f0acbc37700 1 -- 172.21.15.192:0/560934707 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ac4104d90 con 0x7f0ab006c530 2024-04-23T02:08:11.210 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T02:08:11.210 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi071"}]: dispatch 2024-04-23T02:08:11.210 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:08:11.210 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T02:08:11.210 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi071"}]: dispatch 2024-04-23T02:08:11.210 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:08:11.210 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: mgrmap e19: smithi071.ukdjvn(active, since 2m), standbys: smithi192.qgecnv 2024-04-23T02:08:11.210 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:11.211 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T02:08:11.211 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:11.211 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: Detected new or changed devices on smithi071 2024-04-23T02:08:11.211 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:11.211 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:11.211 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:08:11.211 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:08:11.211 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:08:11.211 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:11.211 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:08:11.211 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:08:11.211 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:11.211 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T02:08:11.212 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T02:08:11.212 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:08:11.212 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:10 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:08:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T02:08:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi071"}]: dispatch 2024-04-23T02:08:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:08:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T02:08:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi071"}]: dispatch 2024-04-23T02:08:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:08:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: mgrmap e19: smithi071.ukdjvn(active, since 2m), standbys: smithi192.qgecnv 2024-04-23T02:08:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T02:08:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: Detected new or changed devices on smithi071 2024-04-23T02:08:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:08:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:08:11.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:08:11.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:11.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:08:11.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:08:11.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:11.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T02:08:11.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T02:08:11.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:08:11.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:10 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:08:12.202 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:11 smithi192 ceph-mon[49662]: Adjusting osd_memory_target on smithi071 to 3958M 2024-04-23T02:08:12.202 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:11 smithi192 ceph-mon[49662]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:08:12.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:11 smithi071 ceph-mon[39837]: Adjusting osd_memory_target on smithi071 to 3958M 2024-04-23T02:08:12.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:11 smithi071 ceph-mon[39837]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:08:13.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:12 smithi071 ceph-mon[39837]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:13.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:12 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:08:13.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:12 smithi192 ceph-mon[49662]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:13.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:12 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:08:14.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:13 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/3662919242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62033906-4d1c-431d-a87d-a12e0a134383"}]: dispatch 2024-04-23T02:08:14.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:13 smithi071 ceph-mon[39837]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62033906-4d1c-431d-a87d-a12e0a134383"}]: dispatch 2024-04-23T02:08:14.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:13 smithi071 ceph-mon[39837]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62033906-4d1c-431d-a87d-a12e0a134383"}]': finished 2024-04-23T02:08:14.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:13 smithi071 ceph-mon[39837]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T02:08:14.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:13 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T02:08:14.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:13 smithi192 ceph-mon[49662]: from='client.? 172.21.15.192:0/3662919242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62033906-4d1c-431d-a87d-a12e0a134383"}]: dispatch 2024-04-23T02:08:14.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:13 smithi192 ceph-mon[49662]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62033906-4d1c-431d-a87d-a12e0a134383"}]: dispatch 2024-04-23T02:08:14.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:13 smithi192 ceph-mon[49662]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62033906-4d1c-431d-a87d-a12e0a134383"}]': finished 2024-04-23T02:08:14.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:13 smithi192 ceph-mon[49662]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T02:08:14.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:13 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T02:08:15.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:14 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/1080433883' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T02:08:15.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:14 smithi071 ceph-mon[39837]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:15.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:14 smithi192 ceph-mon[49662]: from='client.? 172.21.15.192:0/1080433883' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T02:08:15.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:14 smithi192 ceph-mon[49662]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:16.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:16 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:16.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:16 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:16.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:16 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:08:16.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:16 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:08:16.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:16 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:16.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:16 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:16.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:16 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:16.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:16 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:08:16.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:16 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:08:16.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:16 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:17.605 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:17 smithi192 ceph-mon[49662]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:17.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:17 smithi071 ceph-mon[39837]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:19.466 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:19 smithi192 ceph-mon[49662]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:19.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:19 smithi071 ceph-mon[39837]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:21.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:21 smithi192 ceph-mon[49662]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:21.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:21 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T02:08:21.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:21 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:08:21.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:21 smithi071 ceph-mon[39837]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:21.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:21 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T02:08:21.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:21 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:08:22.538 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:22 smithi192 ceph-mon[49662]: Deploying daemon osd.3 on smithi192 2024-04-23T02:08:22.634 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:22 smithi071 ceph-mon[39837]: Deploying daemon osd.3 on smithi192 2024-04-23T02:08:23.613 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:23 smithi192 ceph-mon[49662]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:23.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:23 smithi071 ceph-mon[39837]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:25.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:25 smithi192 ceph-mon[49662]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:25.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:25 smithi071 ceph-mon[39837]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:26.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:26 smithi192 ceph-mon[49662]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:26.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:26 smithi071 ceph-mon[39837]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:27.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:27 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:08:27.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:27 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:27.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:27 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:27.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:27 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:08:28.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:27 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:08:28.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:27 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:28.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:27 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:28.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:27 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:08:28.806 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:28 smithi192 ceph-mon[49662]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:28.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:28 smithi071 ceph-mon[39837]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:30.778 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:30.775+0000 7f0abaffd700 1 -- 172.21.15.192:0/560934707 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0ac4104d90 con 0x7f0ab006c530 2024-04-23T02:08:30.779 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 3 on host 'smithi192' 2024-04-23T02:08:30.781 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:30.777+0000 7f0acbc37700 1 -- 172.21.15.192:0/560934707 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0ab006c530 msgr2=0x7f0ab006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:08:30.781 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:30.778+0000 7f0acbc37700 1 --2- 172.21.15.192:0/560934707 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0ab006c530 0x7f0ab006e9f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009fd0 tx=0x7f0ab400b5a0 comp rx=0 tx=0).stop 2024-04-23T02:08:30.781 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:30.778+0000 7f0acbc37700 1 -- 172.21.15.192:0/560934707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ac41017f0 msgr2=0x7f0ac41963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:08:30.781 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:30.778+0000 7f0acbc37700 1 --2- 172.21.15.192:0/560934707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ac41017f0 0x7f0ac41963b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000e920 tx=0x7f0ac000ec30 comp rx=0 tx=0).stop 2024-04-23T02:08:30.781 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:30.778+0000 7f0acbc37700 1 -- 172.21.15.192:0/560934707 shutdown_connections 2024-04-23T02:08:30.781 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:30.778+0000 7f0acbc37700 1 --2- 172.21.15.192:0/560934707 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0ab006c530 0x7f0ab006e9f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:08:30.781 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:30.778+0000 7f0acbc37700 1 --2- 172.21.15.192:0/560934707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ac41017f0 0x7f0ac41963b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:08:30.782 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:30.778+0000 7f0acbc37700 1 --2- 172.21.15.192:0/560934707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ac4104120 0x7f0ac41968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:08:30.782 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:30.778+0000 7f0acbc37700 1 -- 172.21.15.192:0/560934707 >> 172.21.15.192:0/560934707 conn(0x7f0ac40fb120 msgr2=0x7f0ac40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:08:30.782 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:30.778+0000 7f0acbc37700 1 -- 172.21.15.192:0/560934707 shutdown_connections 2024-04-23T02:08:30.782 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:30.778+0000 7f0acbc37700 1 -- 172.21.15.192:0/560934707 wait complete. 2024-04-23T02:08:31.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:31 smithi071 ceph-mon[39837]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:31.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:31 smithi071 ceph-mon[39837]: from='osd.3 [v2:172.21.15.192:6800/593532086,v1:172.21.15.192:6801/593532086]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T02:08:31.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:31 smithi071 ceph-mon[39837]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T02:08:31.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:31.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:31.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:31.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:31.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:31 smithi192 ceph-mon[49662]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:31.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:31 smithi192 ceph-mon[49662]: from='osd.3 [v2:172.21.15.192:6800/593532086,v1:172.21.15.192:6801/593532086]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T02:08:31.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:31 smithi192 ceph-mon[49662]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T02:08:31.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:31.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:31.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:31.585 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:31.606 DEBUG:teuthology.orchestra.run.smithi192:osd.3> sudo journalctl -f -n 0 -u ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.3.service 2024-04-23T02:08:31.609 INFO:tasks.cephadm:Deploying osd.4 on smithi192 with /dev/vg_nvme/lv_3... 2024-04-23T02:08:31.609 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T02:08:31.640 INFO:journalctl@ceph.osd.3.smithi192.stdout:-- Logs begin at Tue 2024-04-23 01:51:29 UTC. -- 2024-04-23T02:08:32.376 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:08:32 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-3[56562]: 2024-04-23T02:08:32.191+0000 7f780d547700 -1 osd.3 0 waiting for initial osdmap 2024-04-23T02:08:32.376 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:08:32 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-3[56562]: 2024-04-23T02:08:32.197+0000 7f7807b39700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T02:08:32.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:32 smithi192 ceph-mon[49662]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T02:08:32.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:32 smithi192 ceph-mon[49662]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T02:08:32.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:32 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T02:08:32.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:32 smithi192 ceph-mon[49662]: from='osd.3 [v2:172.21.15.192:6800/593532086,v1:172.21.15.192:6801/593532086]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:08:32.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:32 smithi192 ceph-mon[49662]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:08:32.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:32 smithi071 ceph-mon[39837]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T02:08:32.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:32 smithi071 ceph-mon[39837]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T02:08:32.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:32 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T02:08:32.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:32 smithi071 ceph-mon[39837]: from='osd.3 [v2:172.21.15.192:6800/593532086,v1:172.21.15.192:6801/593532086]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:08:32.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:32 smithi071 ceph-mon[39837]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:08:32.570 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:08:33.447 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:33 smithi192 ceph-mon[49662]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:33.447 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:33 smithi192 ceph-mon[49662]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-04-23T02:08:33.447 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:33 smithi192 ceph-mon[49662]: osdmap e25: 4 total, 3 up, 4 in 2024-04-23T02:08:33.447 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:33 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T02:08:33.447 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:33 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T02:08:33.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:33 smithi071 ceph-mon[39837]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:33.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:33 smithi071 ceph-mon[39837]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-04-23T02:08:33.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:33 smithi071 ceph-mon[39837]: osdmap e25: 4 total, 3 up, 4 in 2024-04-23T02:08:33.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:33 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T02:08:33.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:33 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T02:08:34.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:34 smithi192 ceph-mon[49662]: purged_snaps scrub starts 2024-04-23T02:08:34.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:34 smithi192 ceph-mon[49662]: purged_snaps scrub ok 2024-04-23T02:08:34.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:34 smithi192 ceph-mon[49662]: osd.3 [v2:172.21.15.192:6800/593532086,v1:172.21.15.192:6801/593532086] boot 2024-04-23T02:08:34.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:34 smithi192 ceph-mon[49662]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T02:08:34.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:34 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T02:08:34.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:34 smithi071 ceph-mon[39837]: purged_snaps scrub starts 2024-04-23T02:08:34.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:34 smithi071 ceph-mon[39837]: purged_snaps scrub ok 2024-04-23T02:08:34.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:34 smithi071 ceph-mon[39837]: osd.3 [v2:172.21.15.192:6800/593532086,v1:172.21.15.192:6801/593532086] boot 2024-04-23T02:08:34.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:34 smithi071 ceph-mon[39837]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T02:08:34.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:34 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T02:08:35.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:35 smithi071 ceph-mon[39837]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:35.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:35 smithi071 ceph-mon[39837]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T02:08:35.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:35 smithi192 ceph-mon[49662]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-23T02:08:35.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:35 smithi192 ceph-mon[49662]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T02:08:35.977 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T02:08:35.977 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T02:08:35.977 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-04-23T02:08:35.978 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-04-23T02:08:35.978 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109644 s, 956 MB/s 2024-04-23T02:08:35.978 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-04-23T02:08:36.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:36 smithi192 ceph-mon[49662]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T02:08:36.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:36 smithi192 ceph-mon[49662]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T02:08:36.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:36 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:36.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:36 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:36.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:36 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:08:36.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:36 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:36.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:36 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:08:36.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:36 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:08:36.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:36 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:36.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:36 smithi071 ceph-mon[39837]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T02:08:36.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:36 smithi071 ceph-mon[39837]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T02:08:36.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:36 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:36.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:36 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:36.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:36 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:08:36.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:36 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:36.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:36 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:08:36.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:36 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:08:36.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:36 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:36.776 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi192:vg_nvme/lv_3 2024-04-23T02:08:37.000 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:08:37.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:37 smithi071 ceph-mon[39837]: Detected new or changed devices on smithi192 2024-04-23T02:08:37.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:37 smithi071 ceph-mon[39837]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-04-23T02:08:37.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:37 smithi071 ceph-mon[39837]: Adjusting osd_memory_target on smithi192 to 14946M 2024-04-23T02:08:37.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:37 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:08:37.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:37 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:08:37.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:37 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:08:37.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:37 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:37.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:37 smithi071 ceph-mon[39837]: osdmap e30: 4 total, 4 up, 4 in 2024-04-23T02:08:37.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:37 smithi192 ceph-mon[49662]: Detected new or changed devices on smithi192 2024-04-23T02:08:37.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:37 smithi192 ceph-mon[49662]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-04-23T02:08:37.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:37 smithi192 ceph-mon[49662]: Adjusting osd_memory_target on smithi192 to 14946M 2024-04-23T02:08:37.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:37 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:08:37.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:37 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:08:37.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:37 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:08:37.583 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:37 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:37.584 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:37 smithi192 ceph-mon[49662]: osdmap e30: 4 total, 4 up, 4 in 2024-04-23T02:08:38.992 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.986+0000 7f7223f66700 1 -- 172.21.15.192:0/1262816244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f721c100db0 msgr2=0x7f721c101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:08:38.992 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.986+0000 7f7223f66700 1 --2- 172.21.15.192:0/1262816244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f721c100db0 0x7f721c101210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7218009910 tx=0x7f7218009c20 comp rx=0 tx=0).stop 2024-04-23T02:08:38.992 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.986+0000 7f7223f66700 1 -- 172.21.15.192:0/1262816244 shutdown_connections 2024-04-23T02:08:38.992 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.986+0000 7f7223f66700 1 --2- 172.21.15.192:0/1262816244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f721c100db0 0x7f721c101210 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:08:38.992 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.986+0000 7f7223f66700 1 --2- 172.21.15.192:0/1262816244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f721c0ffbb0 0x7f721c0fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:08:38.992 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.986+0000 7f7223f66700 1 -- 172.21.15.192:0/1262816244 >> 172.21.15.192:0/1262816244 conn(0x7f721c0fb150 msgr2=0x7f721c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:08:38.992 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.986+0000 7f7223f66700 1 -- 172.21.15.192:0/1262816244 shutdown_connections 2024-04-23T02:08:38.992 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.986+0000 7f7223f66700 1 -- 172.21.15.192:0/1262816244 wait complete. 2024-04-23T02:08:38.992 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.987+0000 7f7223f66700 1 Processor -- start 2024-04-23T02:08:38.993 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.987+0000 7f7223f66700 1 -- start start 2024-04-23T02:08:38.993 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.987+0000 7f7223f66700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f721c0ffbb0 0x7f721c1952d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:08:38.993 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.987+0000 7f7223f66700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f721c100db0 0x7f721c195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:08:38.993 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.987+0000 7f7223f66700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c195e30 con 0x7f721c100db0 2024-04-23T02:08:38.993 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.987+0000 7f7223f66700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c195f70 con 0x7f721c0ffbb0 2024-04-23T02:08:38.993 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.988+0000 7f7221d02700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f721c0ffbb0 0x7f721c1952d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:08:38.993 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.988+0000 7f7221d02700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f721c0ffbb0 0x7f721c1952d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:56702/0 (socket says 172.21.15.192:56702) 2024-04-23T02:08:38.993 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.988+0000 7f7221d02700 1 -- 172.21.15.192:0/2314885815 learned_addr learned my addr 172.21.15.192:0/2314885815 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:08:38.993 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.988+0000 7f7221501700 1 --2- 172.21.15.192:0/2314885815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f721c100db0 0x7f721c195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:08:38.993 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.988+0000 7f7221d02700 1 -- 172.21.15.192:0/2314885815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f721c100db0 msgr2=0x7f721c195810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:08:38.993 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.988+0000 7f7221d02700 1 --2- 172.21.15.192:0/2314885815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f721c100db0 0x7f721c195810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:08:38.993 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.988+0000 7f7221d02700 1 -- 172.21.15.192:0/2314885815 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72180095f0 con 0x7f721c0ffbb0 2024-04-23T02:08:38.993 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.988+0000 7f7221d02700 1 --2- 172.21.15.192:0/2314885815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f721c0ffbb0 0x7f721c1952d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f721000e8f0 tx=0x7f721000ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:08:38.993 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.988+0000 7f720effd700 1 -- 172.21.15.192:0/2314885815 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7210009790 con 0x7f721c0ffbb0 2024-04-23T02:08:38.994 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.988+0000 7f7223f66700 1 -- 172.21.15.192:0/2314885815 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721c068e70 con 0x7f721c0ffbb0 2024-04-23T02:08:38.994 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.988+0000 7f720effd700 1 -- 172.21.15.192:0/2314885815 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f721000c810 con 0x7f721c0ffbb0 2024-04-23T02:08:38.994 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.988+0000 7f720effd700 1 -- 172.21.15.192:0/2314885815 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7210003c20 con 0x7f721c0ffbb0 2024-04-23T02:08:38.994 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.988+0000 7f7223f66700 1 -- 172.21.15.192:0/2314885815 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721c0693c0 con 0x7f721c0ffbb0 2024-04-23T02:08:38.995 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.989+0000 7f7223f66700 1 -- 172.21.15.192:0/2314885815 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721c1085d0 con 0x7f721c0ffbb0 2024-04-23T02:08:38.995 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.990+0000 7f720effd700 1 -- 172.21.15.192:0/2314885815 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f721000c980 con 0x7f721c0ffbb0 2024-04-23T02:08:38.995 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.990+0000 7f720effd700 1 --2- 172.21.15.192:0/2314885815 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f720806c4e0 0x7f720806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:08:38.995 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.990+0000 7f720effd700 1 -- 172.21.15.192:0/2314885815 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3774+0+0 (secure 0 0 0) 0x7f7210014070 con 0x7f721c0ffbb0 2024-04-23T02:08:38.995 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.990+0000 7f7221501700 1 --2- 172.21.15.192:0/2314885815 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f720806c4e0 0x7f720806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:08:38.995 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.991+0000 7f7221501700 1 --2- 172.21.15.192:0/2314885815 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f720806c4e0 0x7f720806e9a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7218009fd0 tx=0x7f721800b5a0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:08:38.998 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:38.995+0000 7f720effd700 1 -- 172.21.15.192:0/2314885815 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7210059d60 con 0x7f721c0ffbb0 2024-04-23T02:08:39.195 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:39.191+0000 7f7223f66700 1 -- 172.21.15.192:0/2314885815 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f721c105700 con 0x7f720806c4e0 2024-04-23T02:08:39.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:39 smithi071 ceph-mon[39837]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T02:08:39.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:39 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T02:08:39.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:39 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T02:08:39.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:39 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:08:39.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:39 smithi192 ceph-mon[49662]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T02:08:39.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:39 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T02:08:39.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:39 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T02:08:39.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:39 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:08:40.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:40 smithi192 ceph-mon[49662]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:08:40.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:40 smithi071 ceph-mon[39837]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:08:41.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:41 smithi192 ceph-mon[49662]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-23T02:08:41.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:41 smithi071 ceph-mon[39837]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-23T02:08:42.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:42 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/2314690652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a7317f5-dc05-415a-8409-81f80c196a0d"}]: dispatch 2024-04-23T02:08:42.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:42 smithi071 ceph-mon[39837]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a7317f5-dc05-415a-8409-81f80c196a0d"}]: dispatch 2024-04-23T02:08:42.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:42 smithi071 ceph-mon[39837]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a7317f5-dc05-415a-8409-81f80c196a0d"}]': finished 2024-04-23T02:08:42.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:42 smithi071 ceph-mon[39837]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T02:08:42.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T02:08:42.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:08:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:42 smithi192 ceph-mon[49662]: from='client.? 172.21.15.192:0/2314690652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a7317f5-dc05-415a-8409-81f80c196a0d"}]: dispatch 2024-04-23T02:08:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:42 smithi192 ceph-mon[49662]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a7317f5-dc05-415a-8409-81f80c196a0d"}]: dispatch 2024-04-23T02:08:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:42 smithi192 ceph-mon[49662]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a7317f5-dc05-415a-8409-81f80c196a0d"}]': finished 2024-04-23T02:08:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:42 smithi192 ceph-mon[49662]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T02:08:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:42 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T02:08:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:42 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:08:43.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:43 smithi071 ceph-mon[39837]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-23T02:08:43.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:43 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/948253277' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T02:08:43.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:43 smithi192 ceph-mon[49662]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-23T02:08:43.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:43 smithi192 ceph-mon[49662]: from='client.? 172.21.15.192:0/948253277' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T02:08:45.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:45 smithi071 ceph-mon[39837]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-23T02:08:45.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:45 smithi192 ceph-mon[49662]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-23T02:08:46.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:46 smithi192 ceph-mon[49662]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-23T02:08:46.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:46 smithi071 ceph-mon[39837]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-23T02:08:49.362 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:49 smithi192 ceph-mon[49662]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-23T02:08:49.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:49 smithi071 ceph-mon[39837]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-23T02:08:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:50 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T02:08:50.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:50 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:08:50.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:50 smithi071 ceph-mon[39837]: Deploying daemon osd.4 on smithi192 2024-04-23T02:08:50.511 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:50 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T02:08:50.512 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:50 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:08:50.512 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:50 smithi192 ceph-mon[49662]: Deploying daemon osd.4 on smithi192 2024-04-23T02:08:51.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:51 smithi192 ceph-mon[49662]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-23T02:08:51.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:51 smithi071 ceph-mon[39837]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-23T02:08:53.479 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:53 smithi192 ceph-mon[49662]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-23T02:08:53.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:53 smithi071 ceph-mon[39837]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-23T02:08:55.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:55 smithi071 ceph-mon[39837]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-23T02:08:55.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:55 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:55.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:55 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:55.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:55 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:08:55.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:55 smithi192 ceph-mon[49662]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-23T02:08:55.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:55 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:55.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:55 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:55.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:55 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:08:56.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:56 smithi192 ceph-mon[49662]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-23T02:08:56.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:56 smithi071 ceph-mon[39837]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-23T02:08:57.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:08:58.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:08:58.757 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:58 smithi192 ceph-mon[49662]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-23T02:08:58.862 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 4 on host 'smithi192' 2024-04-23T02:08:58.862 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:58.857+0000 7f720effd700 1 -- 172.21.15.192:0/2314885815 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f721c105700 con 0x7f720806c4e0 2024-04-23T02:08:58.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:58.860+0000 7f7223f66700 1 -- 172.21.15.192:0/2314885815 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f720806c4e0 msgr2=0x7f720806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:08:58.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:58.860+0000 7f7223f66700 1 --2- 172.21.15.192:0/2314885815 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f720806c4e0 0x7f720806e9a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7218009fd0 tx=0x7f721800b5a0 comp rx=0 tx=0).stop 2024-04-23T02:08:58.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:58.860+0000 7f7223f66700 1 -- 172.21.15.192:0/2314885815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f721c0ffbb0 msgr2=0x7f721c1952d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:08:58.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:58.860+0000 7f7223f66700 1 --2- 172.21.15.192:0/2314885815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f721c0ffbb0 0x7f721c1952d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f721000e8f0 tx=0x7f721000ec00 comp rx=0 tx=0).stop 2024-04-23T02:08:58.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:58.860+0000 7f7223f66700 1 -- 172.21.15.192:0/2314885815 shutdown_connections 2024-04-23T02:08:58.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:58.860+0000 7f7223f66700 1 --2- 172.21.15.192:0/2314885815 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f720806c4e0 0x7f720806e9a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:08:58.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:58.860+0000 7f7223f66700 1 --2- 172.21.15.192:0/2314885815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f721c0ffbb0 0x7f721c1952d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:08:58.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:58.860+0000 7f7223f66700 1 --2- 172.21.15.192:0/2314885815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f721c100db0 0x7f721c195810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:08:58.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:58.860+0000 7f7223f66700 1 -- 172.21.15.192:0/2314885815 >> 172.21.15.192:0/2314885815 conn(0x7f721c0fb150 msgr2=0x7f721c103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:08:58.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:58.860+0000 7f7223f66700 1 -- 172.21.15.192:0/2314885815 shutdown_connections 2024-04-23T02:08:58.864 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:08:58.860+0000 7f7223f66700 1 -- 172.21.15.192:0/2314885815 wait complete. 2024-04-23T02:08:58.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:58 smithi071 ceph-mon[39837]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-23T02:08:59.651 DEBUG:teuthology.orchestra.run.smithi192:osd.4> sudo journalctl -f -n 0 -u ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.4.service 2024-04-23T02:08:59.654 INFO:tasks.cephadm:Deploying osd.5 on smithi192 with /dev/vg_nvme/lv_2... 2024-04-23T02:08:59.654 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T02:08:59.687 INFO:journalctl@ceph.osd.4.smithi192.stdout:-- Logs begin at Tue 2024-04-23 01:51:29 UTC. -- 2024-04-23T02:08:59.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:59 smithi192 ceph-mon[49662]: from='osd.4 [v2:172.21.15.192:6808/3661697663,v1:172.21.15.192:6809/3661697663]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T02:08:59.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:59 smithi192 ceph-mon[49662]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T02:08:59.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:59 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:59.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:59 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:59.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:59 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:59.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:08:59 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:59.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:59 smithi071 ceph-mon[39837]: from='osd.4 [v2:172.21.15.192:6808/3661697663,v1:172.21.15.192:6809/3661697663]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T02:08:59.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:59 smithi071 ceph-mon[39837]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T02:08:59.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:59 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:59.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:59 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:59.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:59 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:08:59.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:08:59 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:00.484 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:09:00.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:00 smithi071 ceph-mon[39837]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T02:09:00.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:00 smithi071 ceph-mon[39837]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T02:09:00.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:00 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T02:09:00.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:00 smithi071 ceph-mon[39837]: from='osd.4 [v2:172.21.15.192:6808/3661697663,v1:172.21.15.192:6809/3661697663]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:09:00.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:00 smithi071 ceph-mon[39837]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:09:00.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:00 smithi071 ceph-mon[39837]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-23T02:09:01.006 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:09:00 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[64749]: 2024-04-23T02:09:00.663+0000 7f9ec46e9700 -1 osd.4 0 waiting for initial osdmap 2024-04-23T02:09:01.006 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:09:00 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[64749]: 2024-04-23T02:09:00.670+0000 7f9ebecdb700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T02:09:01.006 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:00 smithi192 ceph-mon[49662]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T02:09:01.006 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:00 smithi192 ceph-mon[49662]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T02:09:01.006 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:00 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T02:09:01.007 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:00 smithi192 ceph-mon[49662]: from='osd.4 [v2:172.21.15.192:6808/3661697663,v1:172.21.15.192:6809/3661697663]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:09:01.007 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:00 smithi192 ceph-mon[49662]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:09:01.007 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:00 smithi192 ceph-mon[49662]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-23T02:09:01.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:01 smithi071 ceph-mon[39837]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-04-23T02:09:01.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:01 smithi071 ceph-mon[39837]: osdmap e33: 5 total, 4 up, 5 in 2024-04-23T02:09:01.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:01 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T02:09:01.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:01 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T02:09:02.049 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:01 smithi192 ceph-mon[49662]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-04-23T02:09:02.049 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:01 smithi192 ceph-mon[49662]: osdmap e33: 5 total, 4 up, 5 in 2024-04-23T02:09:02.049 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:01 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T02:09:02.049 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:01 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T02:09:02.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:02 smithi071 ceph-mon[39837]: purged_snaps scrub starts 2024-04-23T02:09:02.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:02 smithi071 ceph-mon[39837]: purged_snaps scrub ok 2024-04-23T02:09:02.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:02 smithi071 ceph-mon[39837]: osd.4 [v2:172.21.15.192:6808/3661697663,v1:172.21.15.192:6809/3661697663] boot 2024-04-23T02:09:02.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:02 smithi071 ceph-mon[39837]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T02:09:02.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:02 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T02:09:02.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:02 smithi071 ceph-mon[39837]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:03.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:02 smithi192 ceph-mon[49662]: purged_snaps scrub starts 2024-04-23T02:09:03.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:02 smithi192 ceph-mon[49662]: purged_snaps scrub ok 2024-04-23T02:09:03.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:02 smithi192 ceph-mon[49662]: osd.4 [v2:172.21.15.192:6808/3661697663,v1:172.21.15.192:6809/3661697663] boot 2024-04-23T02:09:03.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:02 smithi192 ceph-mon[49662]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T02:09:03.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:02 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T02:09:03.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:02 smithi192 ceph-mon[49662]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:03.273 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T02:09:03.273 INFO:teuthology.orchestra.run.smithi192.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T02:09:03.273 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10+0 records in 2024-04-23T02:09:03.274 INFO:teuthology.orchestra.run.smithi192.stderr:10+0 records out 2024-04-23T02:09:03.274 INFO:teuthology.orchestra.run.smithi192.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110265 s, 951 MB/s 2024-04-23T02:09:03.274 INFO:teuthology.orchestra.run.smithi192.stderr:--> Zapping successful for: 2024-04-23T02:09:03.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:03 smithi071 ceph-mon[39837]: osdmap e35: 5 total, 5 up, 5 in 2024-04-23T02:09:04.015 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:03 smithi192 ceph-mon[49662]: osdmap e35: 5 total, 5 up, 5 in 2024-04-23T02:09:04.037 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi192:vg_nvme/lv_2 2024-04-23T02:09:04.672 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:09:04.961 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:04 smithi192 ceph-mon[49662]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:04.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:04 smithi071 ceph-mon[39837]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:05.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:05 smithi071 ceph-mon[39837]: Detected new or changed devices on smithi192 2024-04-23T02:09:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:09:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:09:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:05 smithi071 ceph-mon[39837]: Adjusting osd_memory_target on smithi192 to 7473M 2024-04-23T02:09:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:09:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:09:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:09:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:09:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:09:05.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:06.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:05 smithi192 ceph-mon[49662]: Detected new or changed devices on smithi192 2024-04-23T02:09:06.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:05 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:06.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:05 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:06.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:05 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:09:06.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:05 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:09:06.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:05 smithi192 ceph-mon[49662]: Adjusting osd_memory_target on smithi192 to 7473M 2024-04-23T02:09:06.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:05 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:06.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:05 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:09:06.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:05 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:09:06.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:05 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:06.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:05 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:09:06.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:05 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:09:06.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:05 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:09:06.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:05 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:06.745 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.715+0000 7fb4e6d63700 1 -- 172.21.15.192:0/4172178921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4e01040d0 msgr2=0x7fb4e01064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:06.745 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.715+0000 7fb4e6d63700 1 --2- 172.21.15.192:0/4172178921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4e01040d0 0x7fb4e01064c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009910 tx=0x7fb4cc009c20 comp rx=0 tx=0).stop 2024-04-23T02:09:06.745 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.715+0000 7fb4e6d63700 1 -- 172.21.15.192:0/4172178921 shutdown_connections 2024-04-23T02:09:06.745 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.715+0000 7fb4e6d63700 1 --2- 172.21.15.192:0/4172178921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4e01040d0 0x7fb4e01064c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:06.745 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.715+0000 7fb4e6d63700 1 --2- 172.21.15.192:0/4172178921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4e01017a0 0x7fb4e0103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:06.745 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.715+0000 7fb4e6d63700 1 -- 172.21.15.192:0/4172178921 >> 172.21.15.192:0/4172178921 conn(0x7fb4e00fb190 msgr2=0x7fb4e00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:06.745 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.715+0000 7fb4e6d63700 1 -- 172.21.15.192:0/4172178921 shutdown_connections 2024-04-23T02:09:06.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.715+0000 7fb4e6d63700 1 -- 172.21.15.192:0/4172178921 wait complete. 2024-04-23T02:09:06.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.715+0000 7fb4e6d63700 1 Processor -- start 2024-04-23T02:09:06.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.715+0000 7fb4e6d63700 1 -- start start 2024-04-23T02:09:06.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.715+0000 7fb4e6d63700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4e01017a0 0x7fb4e01963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:06.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.715+0000 7fb4e6d63700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4e01040d0 0x7fb4e01968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:06.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.716+0000 7fb4e6d63700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e0196f00 con 0x7fb4e01017a0 2024-04-23T02:09:06.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.716+0000 7fb4e6d63700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e0197040 con 0x7fb4e01040d0 2024-04-23T02:09:06.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.716+0000 7fb4dffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4e01040d0 0x7fb4e01968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:06.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.716+0000 7fb4dffff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4e01040d0 0x7fb4e01968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:51960/0 (socket says 172.21.15.192:51960) 2024-04-23T02:09:06.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.716+0000 7fb4dffff700 1 -- 172.21.15.192:0/1697085923 learned_addr learned my addr 172.21.15.192:0/1697085923 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:09:06.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.716+0000 7fb4e4aff700 1 --2- 172.21.15.192:0/1697085923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4e01017a0 0x7fb4e01963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:06.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.716+0000 7fb4dffff700 1 -- 172.21.15.192:0/1697085923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4e01017a0 msgr2=0x7fb4e01963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:06.747 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.716+0000 7fb4dffff700 1 --2- 172.21.15.192:0/1697085923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4e01017a0 0x7fb4e01963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:06.747 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.716+0000 7fb4dffff700 1 -- 172.21.15.192:0/1697085923 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc0095f0 con 0x7fb4e01040d0 2024-04-23T02:09:06.747 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.716+0000 7fb4dffff700 1 --2- 172.21.15.192:0/1697085923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4e01040d0 0x7fb4e01968e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0098e0 tx=0x7fb4cc00be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:06.747 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.716+0000 7fb4ddffb700 1 -- 172.21.15.192:0/1697085923 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4cc01d070 con 0x7fb4e01040d0 2024-04-23T02:09:06.747 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.716+0000 7fb4e6d63700 1 -- 172.21.15.192:0/1697085923 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e019ba90 con 0x7fb4e01040d0 2024-04-23T02:09:06.747 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.716+0000 7fb4ddffb700 1 -- 172.21.15.192:0/1697085923 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fb4cc00f460 con 0x7fb4e01040d0 2024-04-23T02:09:06.747 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.716+0000 7fb4ddffb700 1 -- 172.21.15.192:0/1697085923 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4cc021640 con 0x7fb4e01040d0 2024-04-23T02:09:06.747 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.716+0000 7fb4e6d63700 1 -- 172.21.15.192:0/1697085923 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e019bf80 con 0x7fb4e01040d0 2024-04-23T02:09:06.747 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.718+0000 7fb4e6d63700 1 -- 172.21.15.192:0/1697085923 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e01904d0 con 0x7fb4e01040d0 2024-04-23T02:09:06.747 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.718+0000 7fb4ddffb700 1 -- 172.21.15.192:0/1697085923 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fb4cc00fa00 con 0x7fb4e01040d0 2024-04-23T02:09:06.747 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.718+0000 7fb4ddffb700 1 --2- 172.21.15.192:0/1697085923 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fb4d006c4e0 0x7fb4d006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:06.747 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.718+0000 7fb4ddffb700 1 -- 172.21.15.192:0/1697085923 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4206+0+0 (secure 0 0 0) 0x7fb4cc08c6e0 con 0x7fb4e01040d0 2024-04-23T02:09:06.747 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.719+0000 7fb4e4aff700 1 --2- 172.21.15.192:0/1697085923 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fb4d006c4e0 0x7fb4d006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:06.748 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.719+0000 7fb4e4aff700 1 --2- 172.21.15.192:0/1697085923 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fb4d006c4e0 0x7fb4d006e9a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4005fd0 tx=0x7fb4d400a560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:06.748 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.723+0000 7fb4ddffb700 1 -- 172.21.15.192:0/1697085923 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb4cc05ae20 con 0x7fb4e01040d0 2024-04-23T02:09:06.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:06 smithi192 ceph-mon[49662]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:06.921 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:06.917+0000 7fb4e6d63700 1 -- 172.21.15.192:0/1697085923 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4e00611d0 con 0x7fb4d006c4e0 2024-04-23T02:09:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:06 smithi071 ceph-mon[39837]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:07.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:07 smithi071 ceph-mon[39837]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:09:07.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:07 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T02:09:07.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:07 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T02:09:07.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:07 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:09:08.101 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:07 smithi192 ceph-mon[49662]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:09:08.101 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:07 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T02:09:08.102 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:07 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T02:09:08.102 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:07 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:09:08.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:08 smithi071 ceph-mon[39837]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:09.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:08 smithi192 ceph-mon[49662]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:09.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:09 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/3913629873' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68b550d5-d74e-417e-9bcc-98740efaf0ef"}]: dispatch 2024-04-23T02:09:09.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:09 smithi071 ceph-mon[39837]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68b550d5-d74e-417e-9bcc-98740efaf0ef"}]: dispatch 2024-04-23T02:09:09.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:09 smithi071 ceph-mon[39837]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68b550d5-d74e-417e-9bcc-98740efaf0ef"}]': finished 2024-04-23T02:09:09.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:09 smithi071 ceph-mon[39837]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T02:09:09.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:09 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T02:09:10.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:09 smithi192 ceph-mon[49662]: from='client.? 172.21.15.192:0/3913629873' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68b550d5-d74e-417e-9bcc-98740efaf0ef"}]: dispatch 2024-04-23T02:09:10.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:09 smithi192 ceph-mon[49662]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68b550d5-d74e-417e-9bcc-98740efaf0ef"}]: dispatch 2024-04-23T02:09:10.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:09 smithi192 ceph-mon[49662]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68b550d5-d74e-417e-9bcc-98740efaf0ef"}]': finished 2024-04-23T02:09:10.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:09 smithi192 ceph-mon[49662]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T02:09:10.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:09 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T02:09:10.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:10 smithi071 ceph-mon[39837]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:10.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:10 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/48562903' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T02:09:11.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:10 smithi192 ceph-mon[49662]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:11.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:10 smithi192 ceph-mon[49662]: from='client.? 172.21.15.192:0/48562903' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T02:09:12.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:12 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:09:12.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:12 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:09:13.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:13 smithi071 ceph-mon[39837]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:13.540 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:13 smithi192 ceph-mon[49662]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:15.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:15 smithi071 ceph-mon[39837]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:15.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:15 smithi192 ceph-mon[49662]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:16.817 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:16 smithi192 ceph-mon[49662]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:16.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:16 smithi071 ceph-mon[39837]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:17.989 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:17 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T02:09:17.989 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:17 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:09:17.990 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:17 smithi192 ceph-mon[49662]: Deploying daemon osd.5 on smithi192 2024-04-23T02:09:17.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:17 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T02:09:17.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:17 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:09:17.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:17 smithi071 ceph-mon[39837]: Deploying daemon osd.5 on smithi192 2024-04-23T02:09:18.798 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:18 smithi192 ceph-mon[49662]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:18.933 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:18 smithi071 ceph-mon[39837]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:21.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:21 smithi192 ceph-mon[49662]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:21.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:21 smithi071 ceph-mon[39837]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:23.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:23 smithi192 ceph-mon[49662]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:23.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:23 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:23.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:23 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:23.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:23 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:09:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:23 smithi071 ceph-mon[39837]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:23 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:23 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:23.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:23 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:09:25.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:25 smithi192 ceph-mon[49662]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:25.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:25 smithi071 ceph-mon[39837]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:26.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:26 smithi192 ceph-mon[49662]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:26.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:26 smithi071 ceph-mon[39837]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:27.291 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:27.287+0000 7fb4ddffb700 1 -- 172.21.15.192:0/1697085923 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb4e00611d0 con 0x7fb4d006c4e0 2024-04-23T02:09:27.291 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 5 on host 'smithi192' 2024-04-23T02:09:27.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:27.289+0000 7fb4e6d63700 1 -- 172.21.15.192:0/1697085923 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fb4d006c4e0 msgr2=0x7fb4d006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:27.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:27.289+0000 7fb4e6d63700 1 --2- 172.21.15.192:0/1697085923 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fb4d006c4e0 0x7fb4d006e9a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4005fd0 tx=0x7fb4d400a560 comp rx=0 tx=0).stop 2024-04-23T02:09:27.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:27.289+0000 7fb4e6d63700 1 -- 172.21.15.192:0/1697085923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4e01040d0 msgr2=0x7fb4e01968e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:27.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:27.289+0000 7fb4e6d63700 1 --2- 172.21.15.192:0/1697085923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4e01040d0 0x7fb4e01968e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0098e0 tx=0x7fb4cc00be20 comp rx=0 tx=0).stop 2024-04-23T02:09:27.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:27.290+0000 7fb4e6d63700 1 -- 172.21.15.192:0/1697085923 shutdown_connections 2024-04-23T02:09:27.293 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:27.290+0000 7fb4e6d63700 1 --2- 172.21.15.192:0/1697085923 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fb4d006c4e0 0x7fb4d006e9a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:27.294 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:27.290+0000 7fb4e6d63700 1 --2- 172.21.15.192:0/1697085923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4e01017a0 0x7fb4e01963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:27.294 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:27.290+0000 7fb4e6d63700 1 --2- 172.21.15.192:0/1697085923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4e01040d0 0x7fb4e01968e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:27.294 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:27.290+0000 7fb4e6d63700 1 -- 172.21.15.192:0/1697085923 >> 172.21.15.192:0/1697085923 conn(0x7fb4e00fb190 msgr2=0x7fb4e00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:27.294 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:27.290+0000 7fb4e6d63700 1 -- 172.21.15.192:0/1697085923 shutdown_connections 2024-04-23T02:09:27.294 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:27.290+0000 7fb4e6d63700 1 -- 172.21.15.192:0/1697085923 wait complete. 2024-04-23T02:09:27.941 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:27 smithi192 ceph-mon[49662]: from='osd.5 [v2:172.21.15.192:6816/3375258254,v1:172.21.15.192:6817/3375258254]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T02:09:27.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:27 smithi192 ceph-mon[49662]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T02:09:27.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:27 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:09:27.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:27 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:27.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:27 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:27.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:27 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:27.942 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:27 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:27.972 DEBUG:teuthology.orchestra.run.smithi192:osd.5> sudo journalctl -f -n 0 -u ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.5.service 2024-04-23T02:09:27.974 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-04-23T02:09:27.974 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-23T02:09:27.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:27 smithi071 ceph-mon[39837]: from='osd.5 [v2:172.21.15.192:6816/3375258254,v1:172.21.15.192:6817/3375258254]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T02:09:27.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:27 smithi071 ceph-mon[39837]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T02:09:27.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:27 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:09:27.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:27 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:27.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:27 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:27.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:27 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:27.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:27 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:28.006 INFO:journalctl@ceph.osd.5.smithi192.stdout:-- Logs begin at Tue 2024-04-23 01:51:29 UTC. -- 2024-04-23T02:09:28.212 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:09:28.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:28 smithi071 ceph-mon[39837]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T02:09:28.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:28 smithi071 ceph-mon[39837]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T02:09:28.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:28 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T02:09:28.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:28 smithi071 ceph-mon[39837]: from='osd.5 [v2:172.21.15.192:6816/3375258254,v1:172.21.15.192:6817/3375258254]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:09:28.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:28 smithi071 ceph-mon[39837]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:09:28.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:28 smithi071 ceph-mon[39837]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:28.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:28 smithi192 ceph-mon[49662]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T02:09:28.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:28 smithi192 ceph-mon[49662]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T02:09:28.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:28 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T02:09:28.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:28 smithi192 ceph-mon[49662]: from='osd.5 [v2:172.21.15.192:6816/3375258254,v1:172.21.15.192:6817/3375258254]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:09:28.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:28 smithi192 ceph-mon[49662]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:09:28.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:28 smithi192 ceph-mon[49662]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:28.877 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:09:28 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[73690]: 2024-04-23T02:09:28.661+0000 7f0811298700 -1 osd.5 0 waiting for initial osdmap 2024-04-23T02:09:28.877 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:09:28 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[73690]: 2024-04-23T02:09:28.669+0000 7f080b88a700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T02:09:29.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:29 smithi071 ceph-mon[39837]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-04-23T02:09:29.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:29 smithi071 ceph-mon[39837]: osdmap e38: 6 total, 5 up, 6 in 2024-04-23T02:09:29.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:29 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T02:09:29.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:29 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T02:09:29.746 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:29 smithi192 ceph-mon[49662]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-04-23T02:09:29.746 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:29 smithi192 ceph-mon[49662]: osdmap e38: 6 total, 5 up, 6 in 2024-04-23T02:09:29.746 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:29 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T02:09:29.746 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:29 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T02:09:30.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.028+0000 7fc68c9cf700 1 -- 172.21.15.71:0/1947961188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc688100cb0 msgr2=0x7fc6881010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:30.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.028+0000 7fc68c9cf700 1 --2- 172.21.15.71:0/1947961188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc688100cb0 0x7fc6881010d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc670009910 tx=0x7fc670009c20 comp rx=0 tx=0).stop 2024-04-23T02:09:30.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.028+0000 7fc68c9cf700 1 -- 172.21.15.71:0/1947961188 shutdown_connections 2024-04-23T02:09:30.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.028+0000 7fc68c9cf700 1 --2- 172.21.15.71:0/1947961188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc688101eb0 0x7fc688102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:30.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.028+0000 7fc68c9cf700 1 --2- 172.21.15.71:0/1947961188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc688100cb0 0x7fc6881010d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:30.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.028+0000 7fc68c9cf700 1 -- 172.21.15.71:0/1947961188 >> 172.21.15.71:0/1947961188 conn(0x7fc6880fc230 msgr2=0x7fc6880fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:30.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.028+0000 7fc68c9cf700 1 -- 172.21.15.71:0/1947961188 shutdown_connections 2024-04-23T02:09:30.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.028+0000 7fc68c9cf700 1 -- 172.21.15.71:0/1947961188 wait complete. 2024-04-23T02:09:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.029+0000 7fc68c9cf700 1 Processor -- start 2024-04-23T02:09:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.029+0000 7fc68c9cf700 1 -- start start 2024-04-23T02:09:30.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.029+0000 7fc68c9cf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc688100cb0 0x7fc688196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.029+0000 7fc68c9cf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc688101eb0 0x7fc688196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.029+0000 7fc68c9cf700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc688196f70 con 0x7fc688101eb0 2024-04-23T02:09:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.029+0000 7fc68c9cf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6881970b0 con 0x7fc688100cb0 2024-04-23T02:09:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.029+0000 7fc685d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc688101eb0 0x7fc688196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.029+0000 7fc685d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc688101eb0 0x7fc688196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53998/0 (socket says 172.21.15.71:53998) 2024-04-23T02:09:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.029+0000 7fc685d9b700 1 -- 172.21.15.71:0/1398769328 learned_addr learned my addr 172.21.15.71:0/1398769328 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:09:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.029+0000 7fc68659c700 1 --2- 172.21.15.71:0/1398769328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc688100cb0 0x7fc688196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.029+0000 7fc685d9b700 1 -- 172.21.15.71:0/1398769328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc688100cb0 msgr2=0x7fc688196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.029+0000 7fc685d9b700 1 --2- 172.21.15.71:0/1398769328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc688100cb0 0x7fc688196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.029+0000 7fc685d9b700 1 -- 172.21.15.71:0/1398769328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6700095f0 con 0x7fc688101eb0 2024-04-23T02:09:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.029+0000 7fc685d9b700 1 --2- 172.21.15.71:0/1398769328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc688101eb0 0x7fc688196950 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc67800d900 tx=0x7fc67800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:30.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.030+0000 7fc67f7fe700 1 -- 172.21.15.71:0/1398769328 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6780041f0 con 0x7fc688101eb0 2024-04-23T02:09:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.030+0000 7fc67f7fe700 1 -- 172.21.15.71:0/1398769328 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc678004cc0 con 0x7fc688101eb0 2024-04-23T02:09:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.030+0000 7fc68c9cf700 1 -- 172.21.15.71:0/1398769328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68819bb60 con 0x7fc688101eb0 2024-04-23T02:09:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.030+0000 7fc67f7fe700 1 -- 172.21.15.71:0/1398769328 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc678005660 con 0x7fc688101eb0 2024-04-23T02:09:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.030+0000 7fc68c9cf700 1 -- 172.21.15.71:0/1398769328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68819c0b0 con 0x7fc688101eb0 2024-04-23T02:09:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.031+0000 7fc68c9cf700 1 -- 172.21.15.71:0/1398769328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6881096d0 con 0x7fc688101eb0 2024-04-23T02:09:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.031+0000 7fc67f7fe700 1 -- 172.21.15.71:0/1398769328 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fc678004750 con 0x7fc688101eb0 2024-04-23T02:09:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.031+0000 7fc67f7fe700 1 --2- 172.21.15.71:0/1398769328 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc67406c4e0 0x7fc67406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:30.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.032+0000 7fc67f7fe700 1 -- 172.21.15.71:0/1398769328 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc67808b2d0 con 0x7fc688101eb0 2024-04-23T02:09:30.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.032+0000 7fc68659c700 1 --2- 172.21.15.71:0/1398769328 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc67406c4e0 0x7fc67406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:30.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.032+0000 7fc68659c700 1 --2- 172.21.15.71:0/1398769328 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc67406c4e0 0x7fc67406e9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc670006010 tx=0x7fc670005ec0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:30.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.036+0000 7fc67f7fe700 1 -- 172.21.15.71:0/1398769328 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc678059900 con 0x7fc688101eb0 2024-04-23T02:09:30.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.224+0000 7fc68c9cf700 1 -- 172.21.15.71:0/1398769328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fc68819c710 con 0x7fc688101eb0 2024-04-23T02:09:30.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.224+0000 7fc67f7fe700 1 -- 172.21.15.71:0/1398769328 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7fc678016090 con 0x7fc688101eb0 2024-04-23T02:09:30.229 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:09:30.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.226+0000 7fc68c9cf700 1 -- 172.21.15.71:0/1398769328 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc67406c4e0 msgr2=0x7fc67406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:30.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.226+0000 7fc68c9cf700 1 --2- 172.21.15.71:0/1398769328 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc67406c4e0 0x7fc67406e9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc670006010 tx=0x7fc670005ec0 comp rx=0 tx=0).stop 2024-04-23T02:09:30.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.226+0000 7fc68c9cf700 1 -- 172.21.15.71:0/1398769328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc688101eb0 msgr2=0x7fc688196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:30.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.226+0000 7fc68c9cf700 1 --2- 172.21.15.71:0/1398769328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc688101eb0 0x7fc688196950 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc67800d900 tx=0x7fc67800dc10 comp rx=0 tx=0).stop 2024-04-23T02:09:30.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.226+0000 7fc68c9cf700 1 -- 172.21.15.71:0/1398769328 shutdown_connections 2024-04-23T02:09:30.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.226+0000 7fc68c9cf700 1 --2- 172.21.15.71:0/1398769328 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc67406c4e0 0x7fc67406e9a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:30.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.226+0000 7fc68c9cf700 1 --2- 172.21.15.71:0/1398769328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc688100cb0 0x7fc688196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:30.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.226+0000 7fc68c9cf700 1 --2- 172.21.15.71:0/1398769328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc688101eb0 0x7fc688196950 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:30.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.226+0000 7fc68c9cf700 1 -- 172.21.15.71:0/1398769328 >> 172.21.15.71:0/1398769328 conn(0x7fc6880fc230 msgr2=0x7fc6881050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:30.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.226+0000 7fc68c9cf700 1 -- 172.21.15.71:0/1398769328 shutdown_connections 2024-04-23T02:09:30.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:30.226+0000 7fc68c9cf700 1 -- 172.21.15.71:0/1398769328 wait complete. 2024-04-23T02:09:30.952 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:30 smithi071 ceph-mon[39837]: purged_snaps scrub starts 2024-04-23T02:09:30.952 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:30 smithi071 ceph-mon[39837]: purged_snaps scrub ok 2024-04-23T02:09:30.952 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:30 smithi071 ceph-mon[39837]: osd.5 [v2:172.21.15.192:6816/3375258254,v1:172.21.15.192:6817/3375258254] boot 2024-04-23T02:09:30.952 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:30 smithi071 ceph-mon[39837]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T02:09:30.952 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:30 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T02:09:30.952 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:30 smithi071 ceph-mon[39837]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:30.952 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:30 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1398769328' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T02:09:30.972 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":39,"num_osds":6,"num_up_osds":6,"osd_up_since":1713838169,"num_in_osds":6,"osd_in_since":1713838149,"num_remapped_pgs":0} 2024-04-23T02:09:30.973 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T02:09:31.070 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:30 smithi192 ceph-mon[49662]: purged_snaps scrub starts 2024-04-23T02:09:31.070 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:30 smithi192 ceph-mon[49662]: purged_snaps scrub ok 2024-04-23T02:09:31.070 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:30 smithi192 ceph-mon[49662]: osd.5 [v2:172.21.15.192:6816/3375258254,v1:172.21.15.192:6817/3375258254] boot 2024-04-23T02:09:31.070 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:30 smithi192 ceph-mon[49662]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T02:09:31.070 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:30 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T02:09:31.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:30 smithi192 ceph-mon[49662]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-23T02:09:31.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:30 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1398769328' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T02:09:31.236 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:09:31.672 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:31 smithi192 ceph-mon[49662]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T02:09:31.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:31 smithi071 ceph-mon[39837]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T02:09:31.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:31 smithi071 ceph-mon[39837]: Detected new or changed devices on smithi192 2024-04-23T02:09:31.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:31.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:31.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:09:31.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:09:31.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:09:31.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:31 smithi071 ceph-mon[39837]: Adjusting osd_memory_target on smithi192 to 4982M 2024-04-23T02:09:31.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:31.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:09:31.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:09:31.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:31.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:09:31.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:09:31.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:09:31.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:32.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:31 smithi192 ceph-mon[49662]: Detected new or changed devices on smithi192 2024-04-23T02:09:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:09:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:09:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:09:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:31 smithi192 ceph-mon[49662]: Adjusting osd_memory_target on smithi192 to 4982M 2024-04-23T02:09:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:09:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:09:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:09:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:09:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:09:32.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:09:32.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:32 smithi071 ceph-mon[39837]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:33.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.014+0000 7f31ce079700 1 -- 172.21.15.71:0/488703551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c8102760 msgr2=0x7f31c810ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:33.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.014+0000 7f31ce079700 1 --2- 172.21.15.71:0/488703551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c8102760 0x7f31c810ac50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009940 tx=0x7f31b8009c50 comp rx=0 tx=0).stop 2024-04-23T02:09:33.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.014+0000 7f31ce079700 1 -- 172.21.15.71:0/488703551 shutdown_connections 2024-04-23T02:09:33.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.014+0000 7f31ce079700 1 --2- 172.21.15.71:0/488703551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c8102760 0x7f31c810ac50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:33.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.014+0000 7f31ce079700 1 --2- 172.21.15.71:0/488703551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31c8101e40 0x7f31c8102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:33.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.014+0000 7f31ce079700 1 -- 172.21.15.71:0/488703551 >> 172.21.15.71:0/488703551 conn(0x7f31c80fb610 msgr2=0x7f31c80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:33.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.014+0000 7f31ce079700 1 -- 172.21.15.71:0/488703551 shutdown_connections 2024-04-23T02:09:33.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.014+0000 7f31ce079700 1 -- 172.21.15.71:0/488703551 wait complete. 2024-04-23T02:09:33.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.015+0000 7f31ce079700 1 Processor -- start 2024-04-23T02:09:33.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.015+0000 7f31ce079700 1 -- start start 2024-04-23T02:09:33.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.015+0000 7f31ce079700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31c8101e40 0x7f31c819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:33.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.015+0000 7f31ce079700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c8102760 0x7f31c819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:33.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.015+0000 7f31ce079700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c819d170 con 0x7f31c8102760 2024-04-23T02:09:33.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.015+0000 7f31ce079700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c8196620 con 0x7f31c8101e40 2024-04-23T02:09:33.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.015+0000 7f31c6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c8102760 0x7f31c819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:33.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.015+0000 7f31c6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c8102760 0x7f31c819cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54024/0 (socket says 172.21.15.71:54024) 2024-04-23T02:09:33.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.015+0000 7f31c6ffd700 1 -- 172.21.15.71:0/1192194090 learned_addr learned my addr 172.21.15.71:0/1192194090 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:09:33.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.016+0000 7f31c77fe700 1 --2- 172.21.15.71:0/1192194090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31c8101e40 0x7f31c819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:33.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.016+0000 7f31c6ffd700 1 -- 172.21.15.71:0/1192194090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31c8101e40 msgr2=0x7f31c819c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:33.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.016+0000 7f31c6ffd700 1 --2- 172.21.15.71:0/1192194090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31c8101e40 0x7f31c819c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:33.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.016+0000 7f31c6ffd700 1 -- 172.21.15.71:0/1192194090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b80095f0 con 0x7f31c8102760 2024-04-23T02:09:33.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.016+0000 7f31c6ffd700 1 --2- 172.21.15.71:0/1192194090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c8102760 0x7f31c819cae0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f31b80059d0 tx=0x7f31b80055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:33.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.016+0000 7f31c4ff9700 1 -- 172.21.15.71:0/1192194090 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31b801d070 con 0x7f31c8102760 2024-04-23T02:09:33.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.016+0000 7f31c4ff9700 1 -- 172.21.15.71:0/1192194090 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f31b800be80 con 0x7f31c8102760 2024-04-23T02:09:33.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.016+0000 7f31ce079700 1 -- 172.21.15.71:0/1192194090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31c81968a0 con 0x7f31c8102760 2024-04-23T02:09:33.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.016+0000 7f31c4ff9700 1 -- 172.21.15.71:0/1192194090 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31b800fe80 con 0x7f31c8102760 2024-04-23T02:09:33.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.016+0000 7f31ce079700 1 -- 172.21.15.71:0/1192194090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31c8196d90 con 0x7f31c8102760 2024-04-23T02:09:33.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.017+0000 7f31ce079700 1 -- 172.21.15.71:0/1192194090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31c8108350 con 0x7f31c8102760 2024-04-23T02:09:33.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.018+0000 7f31c4ff9700 1 -- 172.21.15.71:0/1192194090 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f31b800f6f0 con 0x7f31c8102760 2024-04-23T02:09:33.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.018+0000 7f31c4ff9700 1 --2- 172.21.15.71:0/1192194090 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f31b006c4e0 0x7f31b006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:33.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.018+0000 7f31c4ff9700 1 -- 172.21.15.71:0/1192194090 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f31b808c580 con 0x7f31c8102760 2024-04-23T02:09:33.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.018+0000 7f31c77fe700 1 --2- 172.21.15.71:0/1192194090 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f31b006c4e0 0x7f31b006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:33.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.018+0000 7f31c77fe700 1 --2- 172.21.15.71:0/1192194090 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f31b006c4e0 0x7f31b006e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f31bc005950 tx=0x7f31bc00b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:33.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.022+0000 7f31c4ff9700 1 -- 172.21.15.71:0/1192194090 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f31b805abb0 con 0x7f31c8102760 2024-04-23T02:09:33.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:32 smithi192 ceph-mon[49662]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:33.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.211+0000 7f31ce079700 1 -- 172.21.15.71:0/1192194090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31c804f2e0 con 0x7f31c8102760 2024-04-23T02:09:33.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.212+0000 7f31c4ff9700 1 -- 172.21.15.71:0/1192194090 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11322 (secure 0 0 0) 0x7f31b805a740 con 0x7f31c8102760 2024-04-23T02:09:33.216 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:09:33.216 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":40,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","created":"2024-04-23T02:03:14.872097+0000","modified":"2024-04-23T02:09:30.664936+0000","last_up_change":"2024-04-23T02:09:29.659762+0000","last_in_change":"2024-04-23T02:09:09.560591+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T02:08:06.416728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5f48bf1-6e98-4432-885a-f6ed0c42e5d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":275655379},{"type":"v1","addr":"172.21.15.71:6803","nonce":275655379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":275655379},{"type":"v1","addr":"172.21.15.71:6805","nonce":275655379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":275655379},{"type":"v1","addr":"172.21.15.71:6809","nonce":275655379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":275655379},{"type":"v1","addr":"172.21.15.71:6807","nonce":275655379}]},"public_addr":"172.21.15.71:6803/275655379","cluster_addr":"172.21.15.71:6805/275655379","heartbeat_back_addr":"172.21.15.71:6809/275655379","heartbeat_front_addr":"172.21.15.71:6807/275655379","state":["exists","up"]},{"osd":1,"uuid":"04304fd2-42f9-4525-88e8-c7e20c9c51e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3686410016},{"type":"v1","addr":"172.21.15.71:6811","nonce":3686410016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3686410016},{"type":"v1","addr":"172.21.15.71:6813","nonce":3686410016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3686410016},{"type":"v1","addr":"172.21.15.71:6817","nonce":3686410016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3686410016},{"type":"v1","addr":"172.21.15.71:6815","nonce":3686410016}]},"public_addr":"172.21.15.71:6811/3686410016","cluster_addr":"172.21.15.71:6813/3686410016","heartbeat_back_addr":"172.21.15.71:6817/3686410016","heartbeat_front_addr":"172.21.15.71:6815/3686410016","state":["exists","up"]},{"osd":2,"uuid":"6797c77c-4ccc-4489-9269-77105d7c45c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2096213030},{"type":"v1","addr":"172.21.15.71:6819","nonce":2096213030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2096213030},{"type":"v1","addr":"172.21.15.71:6821","nonce":2096213030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2096213030},{"type":"v1","addr":"172.21.15.71:6825","nonce":2096213030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2096213030},{"type":"v1","addr":"172.21.15.71:6823","nonce":2096213030}]},"public_addr":"172.21.15.71:6819/2096213030","cluster_addr":"172.21.15.71:6821/2096213030","heartbeat_back_addr":"172.21.15.71:6825/2096213030","heartbeat_front_addr":"172.21.15.71:6823/2096213030","state":["exists","up"]},{"osd":3,"uuid":"62033906-4d1c-431d-a87d-a12e0a134383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":593532086},{"type":"v1","addr":"172.21.15.192:6801","nonce":593532086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":593532086},{"type":"v1","addr":"172.21.15.192:6803","nonce":593532086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":593532086},{"type":"v1","addr":"172.21.15.192:6807","nonce":593532086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":593532086},{"type":"v1","addr":"172.21.15.192:6805","nonce":593532086}]},"public_addr":"172.21.15.192:6801/593532086","cluster_addr":"172.21.15.192:6803/593532086","heartbeat_back_addr":"172.21.15.192:6807/593532086","heartbeat_front_addr":"172.21.15.192:6805/593532086","state":["exists","up"]},{"osd":4,"uuid":"6a7317f5-dc05-415a-8409-81f80c196a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3661697663},{"type":"v1","addr":"172.21.15.192:6809","nonce":3661697663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3661697663},{"type":"v1","addr":"172.21.15.192:6811","nonce":3661697663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3661697663},{"type":"v1","addr":"172.21.15.192:6815","nonce":3661697663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3661697663},{"type":"v1","addr":"172.21.15.192:6813","nonce":3661697663}]},"public_addr":"172.21.15.192:6809/3661697663","cluster_addr":"172.21.15.192:6811/3661697663","heartbeat_back_addr":"172.21.15.192:6815/3661697663","heartbeat_front_addr":"172.21.15.192:6813/3661697663","state":["exists","up"]},{"osd":5,"uuid":"68b550d5-d74e-417e-9bcc-98740efaf0ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3375258254},{"type":"v1","addr":"172.21.15.192:6817","nonce":3375258254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3375258254},{"type":"v1","addr":"172.21.15.192:6819","nonce":3375258254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3375258254},{"type":"v1","addr":"172.21.15.192:6823","nonce":3375258254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3375258254},{"type":"v1","addr":"172.21.15.192:6821","nonce":3375258254}]},"public_addr":"172.21.15.192:6817/3375258254","cluster_addr":"172.21.15.192:6819/3375258254","heartbeat_back_addr":"172.21.15.192:6823/3375258254","heartbeat_front_addr":"172.21.15.192:6821/3375258254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:07:01.863954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:07:32.930839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:04.093747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:31.574485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:59.698428+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:09:28.076223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6801/4165305842":"2024-04-24T02:05:42.095805+0000","172.21.15.71:0/936344738":"2024-04-24T02:05:42.095805+0000","172.21.15.71:0/2206241071":"2024-04-24T02:05:42.095805+0000","172.21.15.71:0/823266478":"2024-04-24T02:04:23.247815+0000","172.21.15.71:6801/3878148961":"2024-04-24T02:04:23.247815+0000","172.21.15.71:6800/3878148961":"2024-04-24T02:04:23.247815+0000","172.21.15.71:0/4205332842":"2024-04-24T02:03:48.178880+0000","172.21.15.71:6800/4165305842":"2024-04-24T02:05:42.095805+0000","172.21.15.71:0/56041968":"2024-04-24T02:05:42.095805+0000","172.21.15.71:6801/3794652670":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/1995129374":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/2225989508":"2024-04-24T02:03:48.178880+0000","172.21.15.71:6800/3794652670":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/294030681":"2024-04-24T02:04:23.247815+0000","172.21.15.71:0/3579793925":"2024-04-24T02:04:23.247815+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T02:09:33.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.214+0000 7f31ce079700 1 -- 172.21.15.71:0/1192194090 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f31b006c4e0 msgr2=0x7f31b006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:33.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.214+0000 7f31ce079700 1 --2- 172.21.15.71:0/1192194090 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f31b006c4e0 0x7f31b006e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f31bc005950 tx=0x7f31bc00b410 comp rx=0 tx=0).stop 2024-04-23T02:09:33.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.214+0000 7f31ce079700 1 -- 172.21.15.71:0/1192194090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c8102760 msgr2=0x7f31c819cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:33.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.214+0000 7f31ce079700 1 --2- 172.21.15.71:0/1192194090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c8102760 0x7f31c819cae0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f31b80059d0 tx=0x7f31b80055f0 comp rx=0 tx=0).stop 2024-04-23T02:09:33.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.214+0000 7f31ce079700 1 -- 172.21.15.71:0/1192194090 shutdown_connections 2024-04-23T02:09:33.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.214+0000 7f31ce079700 1 --2- 172.21.15.71:0/1192194090 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f31b006c4e0 0x7f31b006e9a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:33.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.214+0000 7f31ce079700 1 --2- 172.21.15.71:0/1192194090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31c8101e40 0x7f31c819c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:33.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.214+0000 7f31ce079700 1 --2- 172.21.15.71:0/1192194090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c8102760 0x7f31c819cae0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:33.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.214+0000 7f31ce079700 1 -- 172.21.15.71:0/1192194090 >> 172.21.15.71:0/1192194090 conn(0x7f31c80fb610 msgr2=0x7f31c8105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:33.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.214+0000 7f31ce079700 1 -- 172.21.15.71:0/1192194090 shutdown_connections 2024-04-23T02:09:33.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:33.214+0000 7f31ce079700 1 -- 172.21.15.71:0/1192194090 wait complete. 2024-04-23T02:09:33.908 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:33 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1192194090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T02:09:33.929 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T02:08:06.416728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T02:09:33.930 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-23T02:09:33.966 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:33 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1192194090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T02:09:34.157 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:09:34.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:34 smithi071 ceph-mon[39837]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:35.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:34 smithi192 ceph-mon[49662]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:36.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.032+0000 7f9b8e0bf700 1 -- 172.21.15.71:0/1771567929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88106cd0 msgr2=0x7f9b881070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:36.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.032+0000 7f9b8e0bf700 1 --2- 172.21.15.71:0/1771567929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88106cd0 0x7f9b881070b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009910 tx=0x7f9b7c009c20 comp rx=0 tx=0).stop 2024-04-23T02:09:36.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.032+0000 7f9b8e0bf700 1 -- 172.21.15.71:0/1771567929 shutdown_connections 2024-04-23T02:09:36.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.032+0000 7f9b8e0bf700 1 --2- 172.21.15.71:0/1771567929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b88068580 0x7f9b880689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:36.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.032+0000 7f9b8e0bf700 1 --2- 172.21.15.71:0/1771567929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88106cd0 0x7f9b881070b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:36.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.032+0000 7f9b8e0bf700 1 -- 172.21.15.71:0/1771567929 >> 172.21.15.71:0/1771567929 conn(0x7f9b88075670 msgr2=0x7f9b88075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:36.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.032+0000 7f9b8e0bf700 1 -- 172.21.15.71:0/1771567929 shutdown_connections 2024-04-23T02:09:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.032+0000 7f9b8e0bf700 1 -- 172.21.15.71:0/1771567929 wait complete. 2024-04-23T02:09:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.032+0000 7f9b8e0bf700 1 Processor -- start 2024-04-23T02:09:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.032+0000 7f9b8e0bf700 1 -- start start 2024-04-23T02:09:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.033+0000 7f9b8e0bf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b88068580 0x7f9b8819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.033+0000 7f9b8e0bf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88106cd0 0x7f9b8819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.033+0000 7f9b8e0bf700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8819d250 con 0x7f9b88106cd0 2024-04-23T02:09:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.033+0000 7f9b8e0bf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b88196700 con 0x7f9b88068580 2024-04-23T02:09:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.033+0000 7f9b86ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88106cd0 0x7f9b8819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.033+0000 7f9b86ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88106cd0 0x7f9b8819cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56476/0 (socket says 172.21.15.71:56476) 2024-04-23T02:09:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.033+0000 7f9b86ffd700 1 -- 172.21.15.71:0/1059129262 learned_addr learned my addr 172.21.15.71:0/1059129262 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:09:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.033+0000 7f9b877fe700 1 --2- 172.21.15.71:0/1059129262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b88068580 0x7f9b8819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.033+0000 7f9b86ffd700 1 -- 172.21.15.71:0/1059129262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b88068580 msgr2=0x7f9b8819c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:36.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.033+0000 7f9b86ffd700 1 --2- 172.21.15.71:0/1059129262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b88068580 0x7f9b8819c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:36.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.033+0000 7f9b86ffd700 1 -- 172.21.15.71:0/1059129262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7c0095f0 con 0x7f9b88106cd0 2024-04-23T02:09:36.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.033+0000 7f9b86ffd700 1 --2- 172.21.15.71:0/1059129262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88106cd0 0x7f9b8819cbc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800b4b0 tx=0x7f9b7800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:36.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.033+0000 7f9b84ff9700 1 -- 172.21.15.71:0/1059129262 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b78011800 con 0x7f9b88106cd0 2024-04-23T02:09:36.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.033+0000 7f9b84ff9700 1 -- 172.21.15.71:0/1059129262 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f9b78011df0 con 0x7f9b88106cd0 2024-04-23T02:09:36.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.033+0000 7f9b84ff9700 1 -- 172.21.15.71:0/1059129262 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b78010e50 con 0x7f9b88106cd0 2024-04-23T02:09:36.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.033+0000 7f9b8e0bf700 1 -- 172.21.15.71:0/1059129262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b881969e0 con 0x7f9b88106cd0 2024-04-23T02:09:36.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.033+0000 7f9b8e0bf700 1 -- 172.21.15.71:0/1059129262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b88196f30 con 0x7f9b88106cd0 2024-04-23T02:09:36.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.034+0000 7f9b8e0bf700 1 -- 172.21.15.71:0/1059129262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b881091d0 con 0x7f9b88106cd0 2024-04-23T02:09:36.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.035+0000 7f9b84ff9700 1 -- 172.21.15.71:0/1059129262 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f9b78010380 con 0x7f9b88106cd0 2024-04-23T02:09:36.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.035+0000 7f9b84ff9700 1 --2- 172.21.15.71:0/1059129262 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f9b7006c530 0x7f9b7006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:36.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.035+0000 7f9b84ff9700 1 -- 172.21.15.71:0/1059129262 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f9b7808b080 con 0x7f9b88106cd0 2024-04-23T02:09:36.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.035+0000 7f9b877fe700 1 --2- 172.21.15.71:0/1059129262 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f9b7006c530 0x7f9b7006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:36.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.036+0000 7f9b877fe700 1 --2- 172.21.15.71:0/1059129262 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f9b7006c530 0x7f9b7006e9f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c006010 tx=0x7f9b7c01a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:36.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.040+0000 7f9b84ff9700 1 -- 172.21.15.71:0/1059129262 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9b780be0e0 con 0x7f9b88106cd0 2024-04-23T02:09:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.225+0000 7f9b8e0bf700 1 -- 172.21.15.71:0/1059129262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9b881971b0 con 0x7f9b88106cd0 2024-04-23T02:09:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.225+0000 7f9b84ff9700 1 -- 172.21.15.71:0/1059129262 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9b780150a0 con 0x7f9b88106cd0 2024-04-23T02:09:36.229 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-23T02:09:36.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.227+0000 7f9b8e0bf700 1 -- 172.21.15.71:0/1059129262 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f9b7006c530 msgr2=0x7f9b7006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:36.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.227+0000 7f9b8e0bf700 1 --2- 172.21.15.71:0/1059129262 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f9b7006c530 0x7f9b7006e9f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c006010 tx=0x7f9b7c01a040 comp rx=0 tx=0).stop 2024-04-23T02:09:36.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.227+0000 7f9b8e0bf700 1 -- 172.21.15.71:0/1059129262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88106cd0 msgr2=0x7f9b8819cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:36.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.227+0000 7f9b8e0bf700 1 --2- 172.21.15.71:0/1059129262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88106cd0 0x7f9b8819cbc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800b4b0 tx=0x7f9b7800b7c0 comp rx=0 tx=0).stop 2024-04-23T02:09:36.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.227+0000 7f9b8e0bf700 1 -- 172.21.15.71:0/1059129262 shutdown_connections 2024-04-23T02:09:36.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.227+0000 7f9b8e0bf700 1 --2- 172.21.15.71:0/1059129262 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f9b7006c530 0x7f9b7006e9f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:36.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.227+0000 7f9b8e0bf700 1 --2- 172.21.15.71:0/1059129262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b88068580 0x7f9b8819c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:36.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.227+0000 7f9b8e0bf700 1 --2- 172.21.15.71:0/1059129262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88106cd0 0x7f9b8819cbc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:36.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.227+0000 7f9b8e0bf700 1 -- 172.21.15.71:0/1059129262 >> 172.21.15.71:0/1059129262 conn(0x7f9b88075670 msgr2=0x7f9b880fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:36.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.227+0000 7f9b8e0bf700 1 -- 172.21.15.71:0/1059129262 shutdown_connections 2024-04-23T02:09:36.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:36.227+0000 7f9b8e0bf700 1 -- 172.21.15.71:0/1059129262 wait complete. 2024-04-23T02:09:36.964 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:36 smithi071 ceph-mon[39837]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:36.964 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:36 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1059129262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T02:09:36.986 INFO:tasks.cephadm:Setting up client nodes... 2024-04-23T02:09:36.986 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-23T02:09:37.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:36 smithi192 ceph-mon[49662]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:37.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:36 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1059129262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T02:09:37.214 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:09:39.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.072+0000 7f8a243a4700 1 -- 172.21.15.71:0/3317148244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a1c100cb0 msgr2=0x7f8a1c1010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:39.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.072+0000 7f8a243a4700 1 --2- 172.21.15.71:0/3317148244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a1c100cb0 0x7f8a1c1010d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009910 tx=0x7f8a0c009c20 comp rx=0 tx=0).stop 2024-04-23T02:09:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.072+0000 7f8a243a4700 1 -- 172.21.15.71:0/3317148244 shutdown_connections 2024-04-23T02:09:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.072+0000 7f8a243a4700 1 --2- 172.21.15.71:0/3317148244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a1c101eb0 0x7f8a1c102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.072+0000 7f8a243a4700 1 --2- 172.21.15.71:0/3317148244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a1c100cb0 0x7f8a1c1010d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.072+0000 7f8a243a4700 1 -- 172.21.15.71:0/3317148244 >> 172.21.15.71:0/3317148244 conn(0x7f8a1c0fc230 msgr2=0x7f8a1c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.072+0000 7f8a243a4700 1 -- 172.21.15.71:0/3317148244 shutdown_connections 2024-04-23T02:09:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.072+0000 7f8a243a4700 1 -- 172.21.15.71:0/3317148244 wait complete. 2024-04-23T02:09:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.073+0000 7f8a243a4700 1 Processor -- start 2024-04-23T02:09:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.073+0000 7f8a243a4700 1 -- start start 2024-04-23T02:09:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.073+0000 7f8a243a4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a1c100cb0 0x7f8a1c1964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.073+0000 7f8a243a4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a1c101eb0 0x7f8a1c196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.073+0000 7f8a243a4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c197040 con 0x7f8a1c100cb0 2024-04-23T02:09:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.073+0000 7f8a243a4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c197180 con 0x7f8a1c101eb0 2024-04-23T02:09:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.073+0000 7f8a22140700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a1c100cb0 0x7f8a1c1964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:39.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.073+0000 7f8a22140700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a1c100cb0 0x7f8a1c1964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56496/0 (socket says 172.21.15.71:56496) 2024-04-23T02:09:39.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.073+0000 7f8a22140700 1 -- 172.21.15.71:0/3455397473 learned_addr learned my addr 172.21.15.71:0/3455397473 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:09:39.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.073+0000 7f8a2193f700 1 --2- 172.21.15.71:0/3455397473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a1c101eb0 0x7f8a1c196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:39.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.073+0000 7f8a22140700 1 -- 172.21.15.71:0/3455397473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a1c101eb0 msgr2=0x7f8a1c196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:39.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.073+0000 7f8a22140700 1 --2- 172.21.15.71:0/3455397473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a1c101eb0 0x7f8a1c196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:39.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.073+0000 7f8a22140700 1 -- 172.21.15.71:0/3455397473 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c0095f0 con 0x7f8a1c100cb0 2024-04-23T02:09:39.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.073+0000 7f8a2193f700 1 --2- 172.21.15.71:0/3455397473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a1c101eb0 0x7f8a1c196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:09:39.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.073+0000 7f8a22140700 1 --2- 172.21.15.71:0/3455397473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a1c100cb0 0x7f8a1c1964e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00bda0 tx=0x7f8a0c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:39.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.073+0000 7f8a137fe700 1 -- 172.21.15.71:0/3455397473 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a0c01d070 con 0x7f8a1c100cb0 2024-04-23T02:09:39.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.073+0000 7f8a243a4700 1 -- 172.21.15.71:0/3455397473 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a1c19bbd0 con 0x7f8a1c100cb0 2024-04-23T02:09:39.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.073+0000 7f8a137fe700 1 -- 172.21.15.71:0/3455397473 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f8a0c005940 con 0x7f8a1c100cb0 2024-04-23T02:09:39.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.074+0000 7f8a137fe700 1 -- 172.21.15.71:0/3455397473 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a0c00f460 con 0x7f8a1c100cb0 2024-04-23T02:09:39.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.074+0000 7f8a243a4700 1 -- 172.21.15.71:0/3455397473 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1c19c0c0 con 0x7f8a1c100cb0 2024-04-23T02:09:39.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.074+0000 7f8a243a4700 1 -- 172.21.15.71:0/3455397473 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1c109750 con 0x7f8a1c100cb0 2024-04-23T02:09:39.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.075+0000 7f8a137fe700 1 -- 172.21.15.71:0/3455397473 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f8a0c022870 con 0x7f8a1c100cb0 2024-04-23T02:09:39.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.075+0000 7f8a137fe700 1 --2- 172.21.15.71:0/3455397473 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f8a0806c4e0 0x7f8a0806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:39.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.075+0000 7f8a2193f700 1 --2- 172.21.15.71:0/3455397473 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f8a0806c4e0 0x7f8a0806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:39.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.075+0000 7f8a137fe700 1 -- 172.21.15.71:0/3455397473 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f8a0c08cb80 con 0x7f8a1c100cb0 2024-04-23T02:09:39.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.076+0000 7f8a2193f700 1 --2- 172.21.15.71:0/3455397473 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f8a0806c4e0 0x7f8a0806e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8a18007900 tx=0x7f8a18008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:39.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.081+0000 7f8a137fe700 1 -- 172.21.15.71:0/3455397473 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8a0c05b300 con 0x7f8a1c100cb0 2024-04-23T02:09:39.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.359+0000 7f8a243a4700 1 -- 172.21.15.71:0/3455397473 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f8a1c19c710 con 0x7f8a1c100cb0 2024-04-23T02:09:39.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.362+0000 7f8a137fe700 1 -- 172.21.15.71:0/3455397473 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f8a0c0270f0 con 0x7f8a1c100cb0 2024-04-23T02:09:39.366 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-04-23T02:09:39.366 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBjGCdmUCp8FRAANSUT0Glfm9I/+gHkqVbWnA== 2024-04-23T02:09:39.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.365+0000 7f8a243a4700 1 -- 172.21.15.71:0/3455397473 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f8a0806c4e0 msgr2=0x7f8a0806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:39.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.365+0000 7f8a243a4700 1 --2- 172.21.15.71:0/3455397473 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f8a0806c4e0 0x7f8a0806e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8a18007900 tx=0x7f8a18008040 comp rx=0 tx=0).stop 2024-04-23T02:09:39.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.365+0000 7f8a243a4700 1 -- 172.21.15.71:0/3455397473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a1c100cb0 msgr2=0x7f8a1c1964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:39.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.365+0000 7f8a243a4700 1 --2- 172.21.15.71:0/3455397473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a1c100cb0 0x7f8a1c1964e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00bda0 tx=0x7f8a0c00be80 comp rx=0 tx=0).stop 2024-04-23T02:09:39.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.365+0000 7f8a243a4700 1 -- 172.21.15.71:0/3455397473 shutdown_connections 2024-04-23T02:09:39.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.365+0000 7f8a243a4700 1 --2- 172.21.15.71:0/3455397473 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f8a0806c4e0 0x7f8a0806e9a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:39.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.365+0000 7f8a243a4700 1 --2- 172.21.15.71:0/3455397473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a1c100cb0 0x7f8a1c1964e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:39.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.365+0000 7f8a243a4700 1 --2- 172.21.15.71:0/3455397473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a1c101eb0 0x7f8a1c196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:39.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.365+0000 7f8a243a4700 1 -- 172.21.15.71:0/3455397473 >> 172.21.15.71:0/3455397473 conn(0x7f8a1c0fc230 msgr2=0x7f8a1c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:39.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.365+0000 7f8a243a4700 1 -- 172.21.15.71:0/3455397473 shutdown_connections 2024-04-23T02:09:39.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:39.365+0000 7f8a243a4700 1 -- 172.21.15.71:0/3455397473 wait complete. 2024-04-23T02:09:39.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:39 smithi192 ceph-mon[49662]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:39.399 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:39 smithi071 ceph-mon[39837]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:39.954 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T02:09:39.954 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-23T02:09:39.954 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-23T02:09:39.994 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-23T02:09:40.232 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:09:40.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:40 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3455397473' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T02:09:40.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:40 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3455397473' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T02:09:40.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:40 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/3455397473' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T02:09:40.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:40 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/3455397473' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T02:09:41.507 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:41 smithi192 ceph-mon[49662]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:41.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:41 smithi071 ceph-mon[39837]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:42.071 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.066+0000 7fcc3bcb6700 1 -- 172.21.15.192:0/632665931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc34101ea0 msgr2=0x7fcc34102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:42.071 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.066+0000 7fcc3bcb6700 1 --2- 172.21.15.192:0/632665931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc34101ea0 0x7fcc34102300 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009910 tx=0x7fcc30009c20 comp rx=0 tx=0).stop 2024-04-23T02:09:42.071 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.066+0000 7fcc3bcb6700 1 -- 172.21.15.192:0/632665931 shutdown_connections 2024-04-23T02:09:42.071 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.066+0000 7fcc3bcb6700 1 --2- 172.21.15.192:0/632665931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc34101ea0 0x7fcc34102300 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:42.071 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.066+0000 7fcc3bcb6700 1 --2- 172.21.15.192:0/632665931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc34100ca0 0x7fcc341010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:42.072 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.066+0000 7fcc3bcb6700 1 -- 172.21.15.192:0/632665931 >> 172.21.15.192:0/632665931 conn(0x7fcc340fc260 msgr2=0x7fcc340fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:42.072 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.066+0000 7fcc3bcb6700 1 -- 172.21.15.192:0/632665931 shutdown_connections 2024-04-23T02:09:42.072 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.066+0000 7fcc3bcb6700 1 -- 172.21.15.192:0/632665931 wait complete. 2024-04-23T02:09:42.072 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.067+0000 7fcc3bcb6700 1 Processor -- start 2024-04-23T02:09:42.072 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.067+0000 7fcc3bcb6700 1 -- start start 2024-04-23T02:09:42.072 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.067+0000 7fcc3bcb6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc34100ca0 0x7fcc341964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:42.072 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.067+0000 7fcc3bcb6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc34101ea0 0x7fcc34196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:42.072 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.067+0000 7fcc3bcb6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc34197020 con 0x7fcc34101ea0 2024-04-23T02:09:42.072 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.067+0000 7fcc3bcb6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc34197160 con 0x7fcc34100ca0 2024-04-23T02:09:42.072 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.067+0000 7fcc39a52700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc34100ca0 0x7fcc341964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:42.072 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.067+0000 7fcc39a52700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc34100ca0 0x7fcc341964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:44150/0 (socket says 172.21.15.192:44150) 2024-04-23T02:09:42.072 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.067+0000 7fcc39a52700 1 -- 172.21.15.192:0/3876050684 learned_addr learned my addr 172.21.15.192:0/3876050684 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-04-23T02:09:42.072 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.067+0000 7fcc39a52700 1 -- 172.21.15.192:0/3876050684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc34101ea0 msgr2=0x7fcc34196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:42.073 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.067+0000 7fcc39a52700 1 --2- 172.21.15.192:0/3876050684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc34101ea0 0x7fcc34196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:42.073 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.067+0000 7fcc39a52700 1 -- 172.21.15.192:0/3876050684 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc300095f0 con 0x7fcc34100ca0 2024-04-23T02:09:42.073 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.067+0000 7fcc39a52700 1 --2- 172.21.15.192:0/3876050684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc34100ca0 0x7fcc341964c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400b480 tx=0x7fcc2400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:42.073 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.068+0000 7fcc2affd700 1 -- 172.21.15.192:0/3876050684 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc240118c0 con 0x7fcc34100ca0 2024-04-23T02:09:42.073 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.068+0000 7fcc2affd700 1 -- 172.21.15.192:0/3876050684 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fcc24010380 con 0x7fcc34100ca0 2024-04-23T02:09:42.073 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.068+0000 7fcc3bcb6700 1 -- 172.21.15.192:0/3876050684 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc3419bc10 con 0x7fcc34100ca0 2024-04-23T02:09:42.073 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.068+0000 7fcc2affd700 1 -- 172.21.15.192:0/3876050684 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc2400f510 con 0x7fcc34100ca0 2024-04-23T02:09:42.074 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.068+0000 7fcc3bcb6700 1 -- 172.21.15.192:0/3876050684 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3419c160 con 0x7fcc34100ca0 2024-04-23T02:09:42.074 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.069+0000 7fcc3bcb6700 1 -- 172.21.15.192:0/3876050684 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc34109760 con 0x7fcc34100ca0 2024-04-23T02:09:42.074 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.069+0000 7fcc2affd700 1 -- 172.21.15.192:0/3876050684 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fcc240104f0 con 0x7fcc34100ca0 2024-04-23T02:09:42.074 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.069+0000 7fcc2affd700 1 --2- 172.21.15.192:0/3876050684 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fcc2006c4e0 0x7fcc2006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:42.074 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.070+0000 7fcc2affd700 1 -- 172.21.15.192:0/3876050684 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fcc2408b170 con 0x7fcc34100ca0 2024-04-23T02:09:42.074 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.070+0000 7fcc39251700 1 --2- 172.21.15.192:0/3876050684 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fcc2006c4e0 0x7fcc2006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:42.074 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.070+0000 7fcc39251700 1 --2- 172.21.15.192:0/3876050684 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fcc2006c4e0 0x7fcc2006e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000b5c0 tx=0x7fcc30005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:42.077 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.074+0000 7fcc2affd700 1 -- 172.21.15.192:0/3876050684 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcc24059720 con 0x7fcc34100ca0 2024-04-23T02:09:42.339 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.335+0000 7fcc3bcb6700 1 -- 172.21.15.192:0/3876050684 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fcc3419c960 con 0x7fcc34100ca0 2024-04-23T02:09:42.344 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.340+0000 7fcc2affd700 1 -- 172.21.15.192:0/3876050684 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7fcc240107a0 con 0x7fcc34100ca0 2024-04-23T02:09:42.344 INFO:teuthology.orchestra.run.smithi192.stdout:[client.1] 2024-04-23T02:09:42.344 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBmGCdm8ucbFBAAVGWWVEhF2dYszhfH7Qttuw== 2024-04-23T02:09:42.346 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.343+0000 7fcc3bcb6700 1 -- 172.21.15.192:0/3876050684 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fcc2006c4e0 msgr2=0x7fcc2006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:42.346 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.343+0000 7fcc3bcb6700 1 --2- 172.21.15.192:0/3876050684 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fcc2006c4e0 0x7fcc2006e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000b5c0 tx=0x7fcc30005960 comp rx=0 tx=0).stop 2024-04-23T02:09:42.347 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.343+0000 7fcc3bcb6700 1 -- 172.21.15.192:0/3876050684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc34100ca0 msgr2=0x7fcc341964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:42.347 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.343+0000 7fcc3bcb6700 1 --2- 172.21.15.192:0/3876050684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc34100ca0 0x7fcc341964c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcc2400b480 tx=0x7fcc2400b840 comp rx=0 tx=0).stop 2024-04-23T02:09:42.347 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.343+0000 7fcc3bcb6700 1 -- 172.21.15.192:0/3876050684 shutdown_connections 2024-04-23T02:09:42.347 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.343+0000 7fcc3bcb6700 1 --2- 172.21.15.192:0/3876050684 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fcc2006c4e0 0x7fcc2006e9a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:42.347 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.343+0000 7fcc3bcb6700 1 --2- 172.21.15.192:0/3876050684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc34100ca0 0x7fcc341964c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:42.347 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.343+0000 7fcc3bcb6700 1 --2- 172.21.15.192:0/3876050684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc34101ea0 0x7fcc34196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:42.347 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.343+0000 7fcc3bcb6700 1 -- 172.21.15.192:0/3876050684 >> 172.21.15.192:0/3876050684 conn(0x7fcc340fc260 msgr2=0x7fcc341050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:42.347 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.343+0000 7fcc3bcb6700 1 -- 172.21.15.192:0/3876050684 shutdown_connections 2024-04-23T02:09:42.347 INFO:teuthology.orchestra.run.smithi192.stderr:2024-04-23T02:09:42.343+0000 7fcc3bcb6700 1 -- 172.21.15.192:0/3876050684 wait complete. 2024-04-23T02:09:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:42 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:09:42.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:42 smithi192 ceph-mon[49662]: from='client.? 172.21.15.192:0/3876050684' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T02:09:42.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:42 smithi192 ceph-mon[49662]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T02:09:42.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:42 smithi192 ceph-mon[49662]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T02:09:42.634 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:09:42.634 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:42 smithi071 ceph-mon[39837]: from='client.? 172.21.15.192:0/3876050684' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T02:09:42.634 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:42 smithi071 ceph-mon[39837]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T02:09:42.635 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:42 smithi071 ceph-mon[39837]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T02:09:43.060 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-23T02:09:43.061 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-23T02:09:43.061 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-23T02:09:43.102 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-23T02:09:43.102 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-23T02:09:43.103 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-23T02:09:43.340 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:09:43.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:43 smithi071 ceph-mon[39837]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:43.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:43 smithi192 ceph-mon[49662]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:45.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.235+0000 7f2024ed6700 1 -- 172.21.15.71:0/1330675783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20200ffda0 msgr2=0x7f2020100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:45.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.235+0000 7f2024ed6700 1 --2- 172.21.15.71:0/1330675783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20200ffda0 0x7f2020100180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2014009910 tx=0x7f2014009c20 comp rx=0 tx=0).stop 2024-04-23T02:09:45.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.236+0000 7f2024ed6700 1 -- 172.21.15.71:0/1330675783 shutdown_connections 2024-04-23T02:09:45.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.236+0000 7f2024ed6700 1 --2- 172.21.15.71:0/1330675783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2020100750 0x7f20201047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:45.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.236+0000 7f2024ed6700 1 --2- 172.21.15.71:0/1330675783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20200ffda0 0x7f2020100180 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:45.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.236+0000 7f2024ed6700 1 -- 172.21.15.71:0/1330675783 >> 172.21.15.71:0/1330675783 conn(0x7f20200fb650 msgr2=0x7f20200fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:45.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.236+0000 7f2024ed6700 1 -- 172.21.15.71:0/1330675783 shutdown_connections 2024-04-23T02:09:45.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.236+0000 7f2024ed6700 1 -- 172.21.15.71:0/1330675783 wait complete. 2024-04-23T02:09:45.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.236+0000 7f2024ed6700 1 Processor -- start 2024-04-23T02:09:45.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.236+0000 7f2024ed6700 1 -- start start 2024-04-23T02:09:45.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.236+0000 7f2024ed6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20200ffda0 0x7f202019b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:45.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.237+0000 7f2024ed6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2020100750 0x7f202019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:45.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.237+0000 7f2024ed6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202019c1a0 con 0x7f2020100750 2024-04-23T02:09:45.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.237+0000 7f2024ed6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2020195650 con 0x7f20200ffda0 2024-04-23T02:09:45.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.237+0000 7f201dd9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2020100750 0x7f202019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:45.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.237+0000 7f201dd9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2020100750 0x7f202019bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56508/0 (socket says 172.21.15.71:56508) 2024-04-23T02:09:45.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.237+0000 7f201dd9b700 1 -- 172.21.15.71:0/1877354015 learned_addr learned my addr 172.21.15.71:0/1877354015 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:09:45.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.237+0000 7f201e59c700 1 --2- 172.21.15.71:0/1877354015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20200ffda0 0x7f202019b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:45.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.237+0000 7f201dd9b700 1 -- 172.21.15.71:0/1877354015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20200ffda0 msgr2=0x7f202019b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:45.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.237+0000 7f201dd9b700 1 --2- 172.21.15.71:0/1877354015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20200ffda0 0x7f202019b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:45.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.237+0000 7f201dd9b700 1 -- 172.21.15.71:0/1877354015 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20140095f0 con 0x7f2020100750 2024-04-23T02:09:45.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.237+0000 7f201dd9b700 1 --2- 172.21.15.71:0/1877354015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2020100750 0x7f202019bb10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f201000b480 tx=0x7f201000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:45.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.237+0000 7f200f7fe700 1 -- 172.21.15.71:0/1877354015 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2010011800 con 0x7f2020100750 2024-04-23T02:09:45.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.237+0000 7f2024ed6700 1 -- 172.21.15.71:0/1877354015 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2020195930 con 0x7f2020100750 2024-04-23T02:09:45.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.237+0000 7f200f7fe700 1 -- 172.21.15.71:0/1877354015 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f2010011df0 con 0x7f2020100750 2024-04-23T02:09:45.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.237+0000 7f200f7fe700 1 -- 172.21.15.71:0/1877354015 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2010010e30 con 0x7f2020100750 2024-04-23T02:09:45.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.237+0000 7f2024ed6700 1 -- 172.21.15.71:0/1877354015 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2020195e80 con 0x7f2020100750 2024-04-23T02:09:45.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.238+0000 7f2024ed6700 1 -- 172.21.15.71:0/1877354015 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2020108140 con 0x7f2020100750 2024-04-23T02:09:45.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.239+0000 7f200f7fe700 1 -- 172.21.15.71:0/1877354015 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f2010011960 con 0x7f2020100750 2024-04-23T02:09:45.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.239+0000 7f200f7fe700 1 --2- 172.21.15.71:0/1877354015 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f200806c4e0 0x7f200806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:45.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.239+0000 7f200f7fe700 1 -- 172.21.15.71:0/1877354015 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f201008afc0 con 0x7f2020100750 2024-04-23T02:09:45.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.239+0000 7f201e59c700 1 --2- 172.21.15.71:0/1877354015 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f200806c4e0 0x7f200806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:45.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.240+0000 7f201e59c700 1 --2- 172.21.15.71:0/1877354015 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f200806c4e0 0x7f200806e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2014006010 tx=0x7f201401a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:45.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.244+0000 7f200f7fe700 1 -- 172.21.15.71:0/1877354015 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2010059570 con 0x7f2020100750 2024-04-23T02:09:45.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.488+0000 7f2024ed6700 1 -- 172.21.15.71:0/1877354015 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f202004f2e0 con 0x7f2020100750 2024-04-23T02:09:45.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.491+0000 7f200f7fe700 1 -- 172.21.15.71:0/1877354015 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+173037 (secure 0 0 0) 0x7f2010059100 con 0x7f2020100750 2024-04-23T02:09:45.497 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:09:45.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.494+0000 7f2024ed6700 1 -- 172.21.15.71:0/1877354015 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f200806c4e0 msgr2=0x7f200806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:45.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.494+0000 7f2024ed6700 1 --2- 172.21.15.71:0/1877354015 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f200806c4e0 0x7f200806e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2014006010 tx=0x7f201401a040 comp rx=0 tx=0).stop 2024-04-23T02:09:45.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.494+0000 7f2024ed6700 1 -- 172.21.15.71:0/1877354015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2020100750 msgr2=0x7f202019bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:45.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.494+0000 7f2024ed6700 1 --2- 172.21.15.71:0/1877354015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2020100750 0x7f202019bb10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f201000b480 tx=0x7f201000b790 comp rx=0 tx=0).stop 2024-04-23T02:09:45.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.494+0000 7f2024ed6700 1 -- 172.21.15.71:0/1877354015 shutdown_connections 2024-04-23T02:09:45.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.494+0000 7f2024ed6700 1 --2- 172.21.15.71:0/1877354015 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f200806c4e0 0x7f200806e9a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:45.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.494+0000 7f2024ed6700 1 --2- 172.21.15.71:0/1877354015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20200ffda0 0x7f202019b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:45.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.494+0000 7f2024ed6700 1 --2- 172.21.15.71:0/1877354015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2020100750 0x7f202019bb10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:45.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.494+0000 7f2024ed6700 1 -- 172.21.15.71:0/1877354015 >> 172.21.15.71:0/1877354015 conn(0x7f20200fb650 msgr2=0x7f20200fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:45.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.494+0000 7f2024ed6700 1 -- 172.21.15.71:0/1877354015 shutdown_connections 2024-04-23T02:09:45.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:45.494+0000 7f2024ed6700 1 -- 172.21.15.71:0/1877354015 wait complete. 2024-04-23T02:09:45.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:45 smithi192 ceph-mon[49662]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:45.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:45 smithi071 ceph-mon[39837]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:46.225 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":19,"active_gid":14215,"active_name":"smithi071.ukdjvn","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":4249907286},{"type":"v1","addr":"172.21.15.71:6801","nonce":4249907286}]},"active_addr":"172.21.15.71:6801/4249907286","active_change":"2024-04-23T02:05:42.095930+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14236,"name":"smithi192.qgecnv","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.71:8443/","prometheus":"http://172.21.15.71:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":3473332587}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":3618236204}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":17588705}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":1815873101}]}]} 2024-04-23T02:09:46.229 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-23T02:09:46.230 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-23T02:09:46.230 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T02:09:46.454 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:09:46.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:46 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1877354015' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T02:09:46.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:46 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1877354015' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T02:09:47.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:47 smithi071 ceph-mon[39837]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:47.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:47 smithi192 ceph-mon[49662]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:48.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.503+0000 7fce4b390700 1 -- 172.21.15.71:0/749258403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44100750 msgr2=0x7fce441047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:48.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.503+0000 7fce4b390700 1 --2- 172.21.15.71:0/749258403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44100750 0x7fce441047a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fce40009910 tx=0x7fce40009c20 comp rx=0 tx=0).stop 2024-04-23T02:09:48.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.503+0000 7fce4b390700 1 -- 172.21.15.71:0/749258403 shutdown_connections 2024-04-23T02:09:48.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.503+0000 7fce4b390700 1 --2- 172.21.15.71:0/749258403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44100750 0x7fce441047a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:48.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.503+0000 7fce4b390700 1 --2- 172.21.15.71:0/749258403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce440ffda0 0x7fce44100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:48.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.503+0000 7fce4b390700 1 -- 172.21.15.71:0/749258403 >> 172.21.15.71:0/749258403 conn(0x7fce440fb650 msgr2=0x7fce440fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:48.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.503+0000 7fce4b390700 1 -- 172.21.15.71:0/749258403 shutdown_connections 2024-04-23T02:09:48.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.503+0000 7fce4b390700 1 -- 172.21.15.71:0/749258403 wait complete. 2024-04-23T02:09:48.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.504+0000 7fce4b390700 1 Processor -- start 2024-04-23T02:09:48.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.504+0000 7fce4b390700 1 -- start start 2024-04-23T02:09:48.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.504+0000 7fce4b390700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce440ffda0 0x7fce441977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:48.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.504+0000 7fce4b390700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44100750 0x7fce44197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:48.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.504+0000 7fce4b390700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce441983c0 con 0x7fce44100750 2024-04-23T02:09:48.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.504+0000 7fce4b390700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce44198500 con 0x7fce440ffda0 2024-04-23T02:09:48.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.504+0000 7fce4892b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44100750 0x7fce44197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:48.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.504+0000 7fce4892b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44100750 0x7fce44197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52680/0 (socket says 172.21.15.71:52680) 2024-04-23T02:09:48.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.504+0000 7fce4892b700 1 -- 172.21.15.71:0/489893505 learned_addr learned my addr 172.21.15.71:0/489893505 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:09:48.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.504+0000 7fce4892b700 1 -- 172.21.15.71:0/489893505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce440ffda0 msgr2=0x7fce441977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:48.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.504+0000 7fce4912c700 1 --2- 172.21.15.71:0/489893505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce440ffda0 0x7fce441977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:48.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.505+0000 7fce4892b700 1 --2- 172.21.15.71:0/489893505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce440ffda0 0x7fce441977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:48.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.505+0000 7fce4892b700 1 -- 172.21.15.71:0/489893505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce400095f0 con 0x7fce44100750 2024-04-23T02:09:48.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.505+0000 7fce4912c700 1 --2- 172.21.15.71:0/489893505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce440ffda0 0x7fce441977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:09:48.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.505+0000 7fce4892b700 1 --2- 172.21.15.71:0/489893505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44100750 0x7fce44197ce0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fce400092e0 tx=0x7fce4000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:48.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.505+0000 7fce367fc700 1 -- 172.21.15.71:0/489893505 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce4001d070 con 0x7fce44100750 2024-04-23T02:09:48.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.505+0000 7fce367fc700 1 -- 172.21.15.71:0/489893505 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fce4000f460 con 0x7fce44100750 2024-04-23T02:09:48.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.505+0000 7fce4b390700 1 -- 172.21.15.71:0/489893505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce441a0810 con 0x7fce44100750 2024-04-23T02:09:48.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.505+0000 7fce367fc700 1 -- 172.21.15.71:0/489893505 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce40021600 con 0x7fce44100750 2024-04-23T02:09:48.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.505+0000 7fce4b390700 1 -- 172.21.15.71:0/489893505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce441a0c80 con 0x7fce44100750 2024-04-23T02:09:48.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.506+0000 7fce4b390700 1 -- 172.21.15.71:0/489893505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce441080f0 con 0x7fce44100750 2024-04-23T02:09:48.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.506+0000 7fce367fc700 1 -- 172.21.15.71:0/489893505 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fce4000f5d0 con 0x7fce44100750 2024-04-23T02:09:48.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.507+0000 7fce367fc700 1 --2- 172.21.15.71:0/489893505 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fce3006c4e0 0x7fce3006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:48.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.507+0000 7fce367fc700 1 -- 172.21.15.71:0/489893505 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fce4008c790 con 0x7fce44100750 2024-04-23T02:09:48.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.507+0000 7fce4912c700 1 --2- 172.21.15.71:0/489893505 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fce3006c4e0 0x7fce3006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:48.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.507+0000 7fce4912c700 1 --2- 172.21.15.71:0/489893505 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fce3006c4e0 0x7fce3006e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fce38007900 tx=0x7fce38008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:48.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.511+0000 7fce367fc700 1 -- 172.21.15.71:0/489893505 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fce400572b0 con 0x7fce44100750 2024-04-23T02:09:48.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.701+0000 7fce4b390700 1 -- 172.21.15.71:0/489893505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce44069460 con 0x7fce44100750 2024-04-23T02:09:48.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.702+0000 7fce367fc700 1 -- 172.21.15.71:0/489893505 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11322 (secure 0 0 0) 0x7fce4005a8d0 con 0x7fce44100750 2024-04-23T02:09:48.706 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:09:48.706 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":40,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","created":"2024-04-23T02:03:14.872097+0000","modified":"2024-04-23T02:09:30.664936+0000","last_up_change":"2024-04-23T02:09:29.659762+0000","last_in_change":"2024-04-23T02:09:09.560591+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T02:08:06.416728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5f48bf1-6e98-4432-885a-f6ed0c42e5d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":275655379},{"type":"v1","addr":"172.21.15.71:6803","nonce":275655379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":275655379},{"type":"v1","addr":"172.21.15.71:6805","nonce":275655379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":275655379},{"type":"v1","addr":"172.21.15.71:6809","nonce":275655379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":275655379},{"type":"v1","addr":"172.21.15.71:6807","nonce":275655379}]},"public_addr":"172.21.15.71:6803/275655379","cluster_addr":"172.21.15.71:6805/275655379","heartbeat_back_addr":"172.21.15.71:6809/275655379","heartbeat_front_addr":"172.21.15.71:6807/275655379","state":["exists","up"]},{"osd":1,"uuid":"04304fd2-42f9-4525-88e8-c7e20c9c51e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3686410016},{"type":"v1","addr":"172.21.15.71:6811","nonce":3686410016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3686410016},{"type":"v1","addr":"172.21.15.71:6813","nonce":3686410016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3686410016},{"type":"v1","addr":"172.21.15.71:6817","nonce":3686410016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3686410016},{"type":"v1","addr":"172.21.15.71:6815","nonce":3686410016}]},"public_addr":"172.21.15.71:6811/3686410016","cluster_addr":"172.21.15.71:6813/3686410016","heartbeat_back_addr":"172.21.15.71:6817/3686410016","heartbeat_front_addr":"172.21.15.71:6815/3686410016","state":["exists","up"]},{"osd":2,"uuid":"6797c77c-4ccc-4489-9269-77105d7c45c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2096213030},{"type":"v1","addr":"172.21.15.71:6819","nonce":2096213030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2096213030},{"type":"v1","addr":"172.21.15.71:6821","nonce":2096213030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2096213030},{"type":"v1","addr":"172.21.15.71:6825","nonce":2096213030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2096213030},{"type":"v1","addr":"172.21.15.71:6823","nonce":2096213030}]},"public_addr":"172.21.15.71:6819/2096213030","cluster_addr":"172.21.15.71:6821/2096213030","heartbeat_back_addr":"172.21.15.71:6825/2096213030","heartbeat_front_addr":"172.21.15.71:6823/2096213030","state":["exists","up"]},{"osd":3,"uuid":"62033906-4d1c-431d-a87d-a12e0a134383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":593532086},{"type":"v1","addr":"172.21.15.192:6801","nonce":593532086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":593532086},{"type":"v1","addr":"172.21.15.192:6803","nonce":593532086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":593532086},{"type":"v1","addr":"172.21.15.192:6807","nonce":593532086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":593532086},{"type":"v1","addr":"172.21.15.192:6805","nonce":593532086}]},"public_addr":"172.21.15.192:6801/593532086","cluster_addr":"172.21.15.192:6803/593532086","heartbeat_back_addr":"172.21.15.192:6807/593532086","heartbeat_front_addr":"172.21.15.192:6805/593532086","state":["exists","up"]},{"osd":4,"uuid":"6a7317f5-dc05-415a-8409-81f80c196a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3661697663},{"type":"v1","addr":"172.21.15.192:6809","nonce":3661697663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3661697663},{"type":"v1","addr":"172.21.15.192:6811","nonce":3661697663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3661697663},{"type":"v1","addr":"172.21.15.192:6815","nonce":3661697663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3661697663},{"type":"v1","addr":"172.21.15.192:6813","nonce":3661697663}]},"public_addr":"172.21.15.192:6809/3661697663","cluster_addr":"172.21.15.192:6811/3661697663","heartbeat_back_addr":"172.21.15.192:6815/3661697663","heartbeat_front_addr":"172.21.15.192:6813/3661697663","state":["exists","up"]},{"osd":5,"uuid":"68b550d5-d74e-417e-9bcc-98740efaf0ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3375258254},{"type":"v1","addr":"172.21.15.192:6817","nonce":3375258254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3375258254},{"type":"v1","addr":"172.21.15.192:6819","nonce":3375258254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3375258254},{"type":"v1","addr":"172.21.15.192:6823","nonce":3375258254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3375258254},{"type":"v1","addr":"172.21.15.192:6821","nonce":3375258254}]},"public_addr":"172.21.15.192:6817/3375258254","cluster_addr":"172.21.15.192:6819/3375258254","heartbeat_back_addr":"172.21.15.192:6823/3375258254","heartbeat_front_addr":"172.21.15.192:6821/3375258254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:07:01.863954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:07:32.930839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:04.093747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:31.574485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:59.698428+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:09:28.076223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6801/4165305842":"2024-04-24T02:05:42.095805+0000","172.21.15.71:0/936344738":"2024-04-24T02:05:42.095805+0000","172.21.15.71:0/2206241071":"2024-04-24T02:05:42.095805+0000","172.21.15.71:0/823266478":"2024-04-24T02:04:23.247815+0000","172.21.15.71:6801/3878148961":"2024-04-24T02:04:23.247815+0000","172.21.15.71:6800/3878148961":"2024-04-24T02:04:23.247815+0000","172.21.15.71:0/4205332842":"2024-04-24T02:03:48.178880+0000","172.21.15.71:6800/4165305842":"2024-04-24T02:05:42.095805+0000","172.21.15.71:0/56041968":"2024-04-24T02:05:42.095805+0000","172.21.15.71:6801/3794652670":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/1995129374":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/2225989508":"2024-04-24T02:03:48.178880+0000","172.21.15.71:6800/3794652670":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/294030681":"2024-04-24T02:04:23.247815+0000","172.21.15.71:0/3579793925":"2024-04-24T02:04:23.247815+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T02:09:48.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.704+0000 7fce4b390700 1 -- 172.21.15.71:0/489893505 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fce3006c4e0 msgr2=0x7fce3006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:48.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.704+0000 7fce4b390700 1 --2- 172.21.15.71:0/489893505 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fce3006c4e0 0x7fce3006e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fce38007900 tx=0x7fce38008040 comp rx=0 tx=0).stop 2024-04-23T02:09:48.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.704+0000 7fce4b390700 1 -- 172.21.15.71:0/489893505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44100750 msgr2=0x7fce44197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:48.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.704+0000 7fce4b390700 1 --2- 172.21.15.71:0/489893505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44100750 0x7fce44197ce0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fce400092e0 tx=0x7fce4000be20 comp rx=0 tx=0).stop 2024-04-23T02:09:48.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.704+0000 7fce4b390700 1 -- 172.21.15.71:0/489893505 shutdown_connections 2024-04-23T02:09:48.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.704+0000 7fce4b390700 1 --2- 172.21.15.71:0/489893505 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fce3006c4e0 0x7fce3006e9a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:48.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.704+0000 7fce4b390700 1 --2- 172.21.15.71:0/489893505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce440ffda0 0x7fce441977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:48.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.704+0000 7fce4b390700 1 --2- 172.21.15.71:0/489893505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44100750 0x7fce44197ce0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:48.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.704+0000 7fce4b390700 1 -- 172.21.15.71:0/489893505 >> 172.21.15.71:0/489893505 conn(0x7fce440fb650 msgr2=0x7fce440fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:48.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.704+0000 7fce4b390700 1 -- 172.21.15.71:0/489893505 shutdown_connections 2024-04-23T02:09:48.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:48.704+0000 7fce4b390700 1 -- 172.21.15.71:0/489893505 wait complete. 2024-04-23T02:09:49.489 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-23T02:09:49.489 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T02:09:49.715 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:09:49.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:49 smithi071 ceph-mon[39837]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:49.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:49 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/489893505' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T02:09:49.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:49 smithi192 ceph-mon[49662]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:49.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:49 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/489893505' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T02:09:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.416+0000 7f3762567700 1 -- 172.21.15.71:0/2152437701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f375c0ffa10 msgr2=0x7f375c0ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.416+0000 7f3762567700 1 --2- 172.21.15.71:0/2152437701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f375c0ffa10 0x7f375c0ffdf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f374c009910 tx=0x7f374c009c20 comp rx=0 tx=0).stop 2024-04-23T02:09:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.416+0000 7f3762567700 1 -- 172.21.15.71:0/2152437701 shutdown_connections 2024-04-23T02:09:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.416+0000 7f3762567700 1 --2- 172.21.15.71:0/2152437701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f375c100330 0x7f375c10d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.416+0000 7f3762567700 1 --2- 172.21.15.71:0/2152437701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f375c0ffa10 0x7f375c0ffdf0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.416+0000 7f3762567700 1 -- 172.21.15.71:0/2152437701 >> 172.21.15.71:0/2152437701 conn(0x7f375c0fb650 msgr2=0x7f375c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.416+0000 7f3762567700 1 -- 172.21.15.71:0/2152437701 shutdown_connections 2024-04-23T02:09:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.416+0000 7f3762567700 1 -- 172.21.15.71:0/2152437701 wait complete. 2024-04-23T02:09:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.417+0000 7f3762567700 1 Processor -- start 2024-04-23T02:09:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.417+0000 7f3762567700 1 -- start start 2024-04-23T02:09:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.417+0000 7f3762567700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f375c0ffa10 0x7f375c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.417+0000 7f3762567700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f375c100330 0x7f375c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.417+0000 7f3762567700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c19f430 con 0x7f375c100330 2024-04-23T02:09:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.417+0000 7f3762567700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c1988e0 con 0x7f375c0ffa10 2024-04-23T02:09:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.418+0000 7f375b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f375c100330 0x7f375c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.418+0000 7f375b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f375c100330 0x7f375c19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52686/0 (socket says 172.21.15.71:52686) 2024-04-23T02:09:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.418+0000 7f375b7fe700 1 -- 172.21.15.71:0/2174946467 learned_addr learned my addr 172.21.15.71:0/2174946467 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:09:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.418+0000 7f375bfff700 1 --2- 172.21.15.71:0/2174946467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f375c0ffa10 0x7f375c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.418+0000 7f375b7fe700 1 -- 172.21.15.71:0/2174946467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f375c0ffa10 msgr2=0x7f375c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.418+0000 7f375b7fe700 1 --2- 172.21.15.71:0/2174946467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f375c0ffa10 0x7f375c19e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.418+0000 7f375b7fe700 1 -- 172.21.15.71:0/2174946467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c0095f0 con 0x7f375c100330 2024-04-23T02:09:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.418+0000 7f375b7fe700 1 --2- 172.21.15.71:0/2174946467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f375c100330 0x7f375c19eda0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f375000b4b0 tx=0x7f375000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.418+0000 7f37597fa700 1 -- 172.21.15.71:0/2174946467 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3750011800 con 0x7f375c100330 2024-04-23T02:09:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.418+0000 7f37597fa700 1 -- 172.21.15.71:0/2174946467 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f3750011df0 con 0x7f375c100330 2024-04-23T02:09:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.418+0000 7f3762567700 1 -- 172.21.15.71:0/2174946467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375c198bc0 con 0x7f375c100330 2024-04-23T02:09:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.418+0000 7f37597fa700 1 -- 172.21.15.71:0/2174946467 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3750010e40 con 0x7f375c100330 2024-04-23T02:09:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.418+0000 7f3762567700 1 -- 172.21.15.71:0/2174946467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375c199110 con 0x7f375c100330 2024-04-23T02:09:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.419+0000 7f3762567700 1 -- 172.21.15.71:0/2174946467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f375c10a7f0 con 0x7f375c100330 2024-04-23T02:09:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.420+0000 7f37597fa700 1 -- 172.21.15.71:0/2174946467 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f3750010380 con 0x7f375c100330 2024-04-23T02:09:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.420+0000 7f37597fa700 1 --2- 172.21.15.71:0/2174946467 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f374406c4e0 0x7f374406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.420+0000 7f375bfff700 1 --2- 172.21.15.71:0/2174946467 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f374406c4e0 0x7f374406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.420+0000 7f37597fa700 1 -- 172.21.15.71:0/2174946467 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f375008aff0 con 0x7f375c100330 2024-04-23T02:09:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.420+0000 7f375bfff700 1 --2- 172.21.15.71:0/2174946467 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f374406c4e0 0x7f374406e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f374c000c00 tx=0x7f374c005fb0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:51.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.425+0000 7f37597fa700 1 -- 172.21.15.71:0/2174946467 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3750059620 con 0x7f375c100330 2024-04-23T02:09:51.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:51 smithi071 ceph-mon[39837]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:51.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.615+0000 7f3762567700 1 -- 172.21.15.71:0/2174946467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f375c04f2e0 con 0x7f375c100330 2024-04-23T02:09:51.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.616+0000 7f37597fa700 1 -- 172.21.15.71:0/2174946467 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11322 (secure 0 0 0) 0x7f37500591b0 con 0x7f375c100330 2024-04-23T02:09:51.620 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:09:51.620 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":40,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","created":"2024-04-23T02:03:14.872097+0000","modified":"2024-04-23T02:09:30.664936+0000","last_up_change":"2024-04-23T02:09:29.659762+0000","last_in_change":"2024-04-23T02:09:09.560591+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T02:08:06.416728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5f48bf1-6e98-4432-885a-f6ed0c42e5d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":275655379},{"type":"v1","addr":"172.21.15.71:6803","nonce":275655379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":275655379},{"type":"v1","addr":"172.21.15.71:6805","nonce":275655379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":275655379},{"type":"v1","addr":"172.21.15.71:6809","nonce":275655379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":275655379},{"type":"v1","addr":"172.21.15.71:6807","nonce":275655379}]},"public_addr":"172.21.15.71:6803/275655379","cluster_addr":"172.21.15.71:6805/275655379","heartbeat_back_addr":"172.21.15.71:6809/275655379","heartbeat_front_addr":"172.21.15.71:6807/275655379","state":["exists","up"]},{"osd":1,"uuid":"04304fd2-42f9-4525-88e8-c7e20c9c51e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3686410016},{"type":"v1","addr":"172.21.15.71:6811","nonce":3686410016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3686410016},{"type":"v1","addr":"172.21.15.71:6813","nonce":3686410016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3686410016},{"type":"v1","addr":"172.21.15.71:6817","nonce":3686410016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3686410016},{"type":"v1","addr":"172.21.15.71:6815","nonce":3686410016}]},"public_addr":"172.21.15.71:6811/3686410016","cluster_addr":"172.21.15.71:6813/3686410016","heartbeat_back_addr":"172.21.15.71:6817/3686410016","heartbeat_front_addr":"172.21.15.71:6815/3686410016","state":["exists","up"]},{"osd":2,"uuid":"6797c77c-4ccc-4489-9269-77105d7c45c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2096213030},{"type":"v1","addr":"172.21.15.71:6819","nonce":2096213030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2096213030},{"type":"v1","addr":"172.21.15.71:6821","nonce":2096213030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2096213030},{"type":"v1","addr":"172.21.15.71:6825","nonce":2096213030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2096213030},{"type":"v1","addr":"172.21.15.71:6823","nonce":2096213030}]},"public_addr":"172.21.15.71:6819/2096213030","cluster_addr":"172.21.15.71:6821/2096213030","heartbeat_back_addr":"172.21.15.71:6825/2096213030","heartbeat_front_addr":"172.21.15.71:6823/2096213030","state":["exists","up"]},{"osd":3,"uuid":"62033906-4d1c-431d-a87d-a12e0a134383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":593532086},{"type":"v1","addr":"172.21.15.192:6801","nonce":593532086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":593532086},{"type":"v1","addr":"172.21.15.192:6803","nonce":593532086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":593532086},{"type":"v1","addr":"172.21.15.192:6807","nonce":593532086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":593532086},{"type":"v1","addr":"172.21.15.192:6805","nonce":593532086}]},"public_addr":"172.21.15.192:6801/593532086","cluster_addr":"172.21.15.192:6803/593532086","heartbeat_back_addr":"172.21.15.192:6807/593532086","heartbeat_front_addr":"172.21.15.192:6805/593532086","state":["exists","up"]},{"osd":4,"uuid":"6a7317f5-dc05-415a-8409-81f80c196a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3661697663},{"type":"v1","addr":"172.21.15.192:6809","nonce":3661697663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3661697663},{"type":"v1","addr":"172.21.15.192:6811","nonce":3661697663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3661697663},{"type":"v1","addr":"172.21.15.192:6815","nonce":3661697663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3661697663},{"type":"v1","addr":"172.21.15.192:6813","nonce":3661697663}]},"public_addr":"172.21.15.192:6809/3661697663","cluster_addr":"172.21.15.192:6811/3661697663","heartbeat_back_addr":"172.21.15.192:6815/3661697663","heartbeat_front_addr":"172.21.15.192:6813/3661697663","state":["exists","up"]},{"osd":5,"uuid":"68b550d5-d74e-417e-9bcc-98740efaf0ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3375258254},{"type":"v1","addr":"172.21.15.192:6817","nonce":3375258254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3375258254},{"type":"v1","addr":"172.21.15.192:6819","nonce":3375258254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3375258254},{"type":"v1","addr":"172.21.15.192:6823","nonce":3375258254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3375258254},{"type":"v1","addr":"172.21.15.192:6821","nonce":3375258254}]},"public_addr":"172.21.15.192:6817/3375258254","cluster_addr":"172.21.15.192:6819/3375258254","heartbeat_back_addr":"172.21.15.192:6823/3375258254","heartbeat_front_addr":"172.21.15.192:6821/3375258254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:07:01.863954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:07:32.930839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:04.093747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:31.574485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:59.698428+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:09:28.076223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6801/4165305842":"2024-04-24T02:05:42.095805+0000","172.21.15.71:0/936344738":"2024-04-24T02:05:42.095805+0000","172.21.15.71:0/2206241071":"2024-04-24T02:05:42.095805+0000","172.21.15.71:0/823266478":"2024-04-24T02:04:23.247815+0000","172.21.15.71:6801/3878148961":"2024-04-24T02:04:23.247815+0000","172.21.15.71:6800/3878148961":"2024-04-24T02:04:23.247815+0000","172.21.15.71:0/4205332842":"2024-04-24T02:03:48.178880+0000","172.21.15.71:6800/4165305842":"2024-04-24T02:05:42.095805+0000","172.21.15.71:0/56041968":"2024-04-24T02:05:42.095805+0000","172.21.15.71:6801/3794652670":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/1995129374":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/2225989508":"2024-04-24T02:03:48.178880+0000","172.21.15.71:6800/3794652670":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/294030681":"2024-04-24T02:04:23.247815+0000","172.21.15.71:0/3579793925":"2024-04-24T02:04:23.247815+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T02:09:51.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.618+0000 7f3762567700 1 -- 172.21.15.71:0/2174946467 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f374406c4e0 msgr2=0x7f374406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:51.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.618+0000 7f3762567700 1 --2- 172.21.15.71:0/2174946467 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f374406c4e0 0x7f374406e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f374c000c00 tx=0x7f374c005fb0 comp rx=0 tx=0).stop 2024-04-23T02:09:51.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.618+0000 7f3762567700 1 -- 172.21.15.71:0/2174946467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f375c100330 msgr2=0x7f375c19eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:51.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.618+0000 7f3762567700 1 --2- 172.21.15.71:0/2174946467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f375c100330 0x7f375c19eda0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f375000b4b0 tx=0x7f375000b870 comp rx=0 tx=0).stop 2024-04-23T02:09:51.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.619+0000 7f3762567700 1 -- 172.21.15.71:0/2174946467 shutdown_connections 2024-04-23T02:09:51.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.619+0000 7f3762567700 1 --2- 172.21.15.71:0/2174946467 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f374406c4e0 0x7f374406e9a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:51.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.619+0000 7f3762567700 1 --2- 172.21.15.71:0/2174946467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f375c0ffa10 0x7f375c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:51.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.619+0000 7f3762567700 1 --2- 172.21.15.71:0/2174946467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f375c100330 0x7f375c19eda0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:51.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.619+0000 7f3762567700 1 -- 172.21.15.71:0/2174946467 >> 172.21.15.71:0/2174946467 conn(0x7f375c0fb650 msgr2=0x7f375c0fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:51.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.619+0000 7f3762567700 1 -- 172.21.15.71:0/2174946467 shutdown_connections 2024-04-23T02:09:51.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:51.619+0000 7f3762567700 1 -- 172.21.15.71:0/2174946467 wait complete. 2024-04-23T02:09:51.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:51 smithi192 ceph-mon[49662]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:52.309 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-23T02:09:52.310 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-23T02:09:52.311 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-23T02:09:52.311 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-23T02:09:52.311 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-23T02:09:52.312 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-23T02:09:52.636 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:52 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/2174946467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T02:09:52.664 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:09:52.667 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:09:52.668 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:09:52.690 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:09:52.696 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:09:52.724 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:09:52.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:52 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/2174946467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T02:09:53.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:53 smithi071 ceph-mon[39837]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:53.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:53 smithi192 ceph-mon[49662]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:55.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:55 smithi071 ceph-mon[39837]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:55.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:55 smithi192 ceph-mon[49662]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:56.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:56 smithi071 ceph-mon[39837]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:57.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:56 smithi192 ceph-mon[49662]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:57.941 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:09:58.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:09:58.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.262+0000 7f70d9633700 1 -- 172.21.15.71:0/3412255406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d41006b0 msgr2=0x7f70d4100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:58.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.262+0000 7f70d9633700 1 --2- 172.21.15.71:0/3412255406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d41006b0 0x7f70d4100a90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009910 tx=0x7f70c8009c20 comp rx=0 tx=0).stop 2024-04-23T02:09:58.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.262+0000 7f70d9633700 1 -- 172.21.15.71:0/3412255406 shutdown_connections 2024-04-23T02:09:58.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.262+0000 7f70d9633700 1 --2- 172.21.15.71:0/3412255406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70d4101060 0x7f70d41047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:58.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.262+0000 7f70d9633700 1 --2- 172.21.15.71:0/3412255406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d41006b0 0x7f70d4100a90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:58.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.262+0000 7f70d9633700 1 -- 172.21.15.71:0/3412255406 >> 172.21.15.71:0/3412255406 conn(0x7f70d4074e30 msgr2=0x7f70d4075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:58.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.262+0000 7f70d9633700 1 -- 172.21.15.71:0/3412255406 shutdown_connections 2024-04-23T02:09:58.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.262+0000 7f70d9633700 1 -- 172.21.15.71:0/3412255406 wait complete. 2024-04-23T02:09:58.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.262+0000 7f70d9633700 1 Processor -- start 2024-04-23T02:09:58.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.263+0000 7f70d9633700 1 -- start start 2024-04-23T02:09:58.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.263+0000 7f70d9633700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d41006b0 0x7f70d41976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:58.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.263+0000 7f70d9633700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70d4101060 0x7f70d4197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:58.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.263+0000 7f70d9633700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d4198240 con 0x7f70d41006b0 2024-04-23T02:09:58.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.263+0000 7f70d9633700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d4198380 con 0x7f70d4101060 2024-04-23T02:09:58.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.263+0000 7f70d2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d41006b0 0x7f70d41976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:58.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.263+0000 7f70d2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d41006b0 0x7f70d41976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46062/0 (socket says 172.21.15.71:46062) 2024-04-23T02:09:58.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.263+0000 7f70d2ffd700 1 -- 172.21.15.71:0/2519291798 learned_addr learned my addr 172.21.15.71:0/2519291798 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:09:58.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.263+0000 7f70d2ffd700 1 -- 172.21.15.71:0/2519291798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70d4101060 msgr2=0x7f70d4197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:58.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.263+0000 7f70d2ffd700 1 --2- 172.21.15.71:0/2519291798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70d4101060 0x7f70d4197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:58.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.263+0000 7f70d2ffd700 1 -- 172.21.15.71:0/2519291798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c80095f0 con 0x7f70d41006b0 2024-04-23T02:09:58.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.263+0000 7f70d2ffd700 1 --2- 172.21.15.71:0/2519291798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d41006b0 0x7f70d41976e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f70c8006010 tx=0x7f70c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:58.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.263+0000 7f70bbfff700 1 -- 172.21.15.71:0/2519291798 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70c801d070 con 0x7f70d41006b0 2024-04-23T02:09:58.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.264+0000 7f70bbfff700 1 -- 172.21.15.71:0/2519291798 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f70c800f460 con 0x7f70d41006b0 2024-04-23T02:09:58.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.264+0000 7f70d9633700 1 -- 172.21.15.71:0/2519291798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d41a0750 con 0x7f70d41006b0 2024-04-23T02:09:58.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.264+0000 7f70bbfff700 1 -- 172.21.15.71:0/2519291798 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70c8021640 con 0x7f70d41006b0 2024-04-23T02:09:58.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.264+0000 7f70d9633700 1 -- 172.21.15.71:0/2519291798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d41a0b60 con 0x7f70d41006b0 2024-04-23T02:09:58.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.264+0000 7f70d9633700 1 -- 172.21.15.71:0/2519291798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f70d410cf30 con 0x7f70d41006b0 2024-04-23T02:09:58.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.265+0000 7f70bbfff700 1 -- 172.21.15.71:0/2519291798 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f70c800f9e0 con 0x7f70d41006b0 2024-04-23T02:09:58.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.265+0000 7f70bbfff700 1 --2- 172.21.15.71:0/2519291798 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f70bc06c4e0 0x7f70bc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:58.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.265+0000 7f70bbfff700 1 -- 172.21.15.71:0/2519291798 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f70c808c850 con 0x7f70d41006b0 2024-04-23T02:09:58.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.265+0000 7f70d27fc700 1 --2- 172.21.15.71:0/2519291798 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f70bc06c4e0 0x7f70bc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:58.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.266+0000 7f70bbfff700 1 --2- 172.21.15.71:0/2519291798 >> [v2:172.21.15.71:6810/3686410016,v1:172.21.15.71:6811/3686410016] conn(0x7f70bc0721c0 0x7f70bc0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:09:58.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.266+0000 7f70bbfff700 1 -- 172.21.15.71:0/2519291798 --> [v2:172.21.15.71:6810/3686410016,v1:172.21.15.71:6811/3686410016] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f70bc074c90 con 0x7f70bc0721c0 2024-04-23T02:09:58.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.266+0000 7f70bbfff700 1 -- 172.21.15.71:0/2519291798 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70c80912a0 con 0x7f70d41006b0 2024-04-23T02:09:58.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.266+0000 7f70d37fe700 1 --2- 172.21.15.71:0/2519291798 >> [v2:172.21.15.71:6810/3686410016,v1:172.21.15.71:6811/3686410016] conn(0x7f70bc0721c0 0x7f70bc0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:09:58.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.266+0000 7f70d27fc700 1 --2- 172.21.15.71:0/2519291798 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f70bc06c4e0 0x7f70bc06e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f70d40ffbf0 tx=0x7f70c4006d20 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:58.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.266+0000 7f70d37fe700 1 --2- 172.21.15.71:0/2519291798 >> [v2:172.21.15.71:6810/3686410016,v1:172.21.15.71:6811/3686410016] conn(0x7f70bc0721c0 0x7f70bc0745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:09:58.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.267+0000 7f70bbfff700 1 -- 172.21.15.71:0/2519291798 <== osd.1 v2:172.21.15.71:6810/3686410016 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f70bc074c90 con 0x7f70bc0721c0 2024-04-23T02:09:58.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.282+0000 7f70d9633700 1 -- 172.21.15.71:0/2519291798 --> [v2:172.21.15.71:6810/3686410016,v1:172.21.15.71:6811/3686410016] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f70d404f2e0 con 0x7f70bc0721c0 2024-04-23T02:09:58.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.282+0000 7f70bbfff700 1 -- 172.21.15.71:0/2519291798 <== osd.1 v2:172.21.15.71:6810/3686410016 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f70d404f2e0 con 0x7f70bc0721c0 2024-04-23T02:09:58.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.282+0000 7f70d9633700 1 -- 172.21.15.71:0/2519291798 >> [v2:172.21.15.71:6810/3686410016,v1:172.21.15.71:6811/3686410016] conn(0x7f70bc0721c0 msgr2=0x7f70bc0745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:58.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.282+0000 7f70d9633700 1 --2- 172.21.15.71:0/2519291798 >> [v2:172.21.15.71:6810/3686410016,v1:172.21.15.71:6811/3686410016] conn(0x7f70bc0721c0 0x7f70bc0745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:58.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.283+0000 7f70d9633700 1 -- 172.21.15.71:0/2519291798 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f70bc06c4e0 msgr2=0x7f70bc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:58.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.283+0000 7f70d9633700 1 --2- 172.21.15.71:0/2519291798 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f70bc06c4e0 0x7f70bc06e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f70d40ffbf0 tx=0x7f70c4006d20 comp rx=0 tx=0).stop 2024-04-23T02:09:58.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.283+0000 7f70d9633700 1 -- 172.21.15.71:0/2519291798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d41006b0 msgr2=0x7f70d41976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:09:58.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.283+0000 7f70d9633700 1 --2- 172.21.15.71:0/2519291798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d41006b0 0x7f70d41976e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f70c8006010 tx=0x7f70c800be20 comp rx=0 tx=0).stop 2024-04-23T02:09:58.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.283+0000 7f70d9633700 1 -- 172.21.15.71:0/2519291798 shutdown_connections 2024-04-23T02:09:58.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.283+0000 7f70d9633700 1 --2- 172.21.15.71:0/2519291798 >> [v2:172.21.15.71:6810/3686410016,v1:172.21.15.71:6811/3686410016] conn(0x7f70bc0721c0 0x7f70bc0745e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:58.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.283+0000 7f70d9633700 1 --2- 172.21.15.71:0/2519291798 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f70bc06c4e0 0x7f70bc06e9a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:58.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.283+0000 7f70d9633700 1 --2- 172.21.15.71:0/2519291798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d41006b0 0x7f70d41976e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:58.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.283+0000 7f70d9633700 1 --2- 172.21.15.71:0/2519291798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70d4101060 0x7f70d4197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:09:58.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.283+0000 7f70d9633700 1 -- 172.21.15.71:0/2519291798 >> 172.21.15.71:0/2519291798 conn(0x7f70d4074e30 msgr2=0x7f70d40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:09:58.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.283+0000 7f70d9633700 1 -- 172.21.15.71:0/2519291798 shutdown_connections 2024-04-23T02:09:58.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:09:58.283+0000 7f70d9633700 1 -- 172.21.15.71:0/2519291798 wait complete. 2024-04-23T02:09:58.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:09:58 smithi071 ceph-mon[39837]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:09:59.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:09:58 smithi192 ceph-mon[49662]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:00.133 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:00 smithi071 ceph-mon[39837]: overall HEALTH_OK 2024-04-23T02:10:00.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:00 smithi192 ceph-mon[49662]: overall HEALTH_OK 2024-04-23T02:10:00.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.788+0000 7fc3607f1700 1 -- 172.21.15.71:0/250081560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc358100750 msgr2=0x7fc3581047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:00.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.788+0000 7fc3607f1700 1 --2- 172.21.15.71:0/250081560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc358100750 0x7fc3581047a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc354009910 tx=0x7fc354009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:00.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.788+0000 7fc3607f1700 1 -- 172.21.15.71:0/250081560 shutdown_connections 2024-04-23T02:10:00.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.788+0000 7fc3607f1700 1 --2- 172.21.15.71:0/250081560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc358100750 0x7fc3581047a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:00.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.788+0000 7fc3607f1700 1 --2- 172.21.15.71:0/250081560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3580ffda0 0x7fc358100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:00.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.788+0000 7fc3607f1700 1 -- 172.21.15.71:0/250081560 >> 172.21.15.71:0/250081560 conn(0x7fc3580fb650 msgr2=0x7fc3580fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:00.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.788+0000 7fc3607f1700 1 -- 172.21.15.71:0/250081560 shutdown_connections 2024-04-23T02:10:00.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.788+0000 7fc3607f1700 1 -- 172.21.15.71:0/250081560 wait complete. 2024-04-23T02:10:00.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.789+0000 7fc3607f1700 1 Processor -- start 2024-04-23T02:10:00.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.789+0000 7fc3607f1700 1 -- start start 2024-04-23T02:10:00.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.789+0000 7fc3607f1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3580ffda0 0x7fc3581976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:00.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.789+0000 7fc3607f1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc358100750 0x7fc358197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:00.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.789+0000 7fc3607f1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc358198190 con 0x7fc3580ffda0 2024-04-23T02:10:00.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.789+0000 7fc3607f1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3581982d0 con 0x7fc358100750 2024-04-23T02:10:00.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.789+0000 7fc35e58d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3580ffda0 0x7fc3581976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:00.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.789+0000 7fc35e58d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3580ffda0 0x7fc3581976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46086/0 (socket says 172.21.15.71:46086) 2024-04-23T02:10:00.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.789+0000 7fc35e58d700 1 -- 172.21.15.71:0/2545207513 learned_addr learned my addr 172.21.15.71:0/2545207513 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:00.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.789+0000 7fc35dd8c700 1 --2- 172.21.15.71:0/2545207513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc358100750 0x7fc358197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:00.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.789+0000 7fc35e58d700 1 -- 172.21.15.71:0/2545207513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc358100750 msgr2=0x7fc358197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:00.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.789+0000 7fc35e58d700 1 --2- 172.21.15.71:0/2545207513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc358100750 0x7fc358197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:00.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.789+0000 7fc35e58d700 1 -- 172.21.15.71:0/2545207513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3540095f0 con 0x7fc3580ffda0 2024-04-23T02:10:00.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.789+0000 7fc35e58d700 1 --2- 172.21.15.71:0/2545207513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3580ffda0 0x7fc3581976c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc34800c930 tx=0x7fc34800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:00.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.790+0000 7fc34f7fe700 1 -- 172.21.15.71:0/2545207513 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc348007ad0 con 0x7fc3580ffda0 2024-04-23T02:10:00.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.790+0000 7fc3607f1700 1 -- 172.21.15.71:0/2545207513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3581a0730 con 0x7fc3580ffda0 2024-04-23T02:10:00.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.790+0000 7fc34f7fe700 1 -- 172.21.15.71:0/2545207513 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc348007c30 con 0x7fc3580ffda0 2024-04-23T02:10:00.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.790+0000 7fc34f7fe700 1 -- 172.21.15.71:0/2545207513 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc348018630 con 0x7fc3580ffda0 2024-04-23T02:10:00.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.790+0000 7fc3607f1700 1 -- 172.21.15.71:0/2545207513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3581a0b40 con 0x7fc3580ffda0 2024-04-23T02:10:00.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.790+0000 7fc3607f1700 1 -- 172.21.15.71:0/2545207513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc35810ce80 con 0x7fc3580ffda0 2024-04-23T02:10:00.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.791+0000 7fc34f7fe700 1 -- 172.21.15.71:0/2545207513 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fc34800f450 con 0x7fc3580ffda0 2024-04-23T02:10:00.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.791+0000 7fc34f7fe700 1 --2- 172.21.15.71:0/2545207513 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc34406c4e0 0x7fc34406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:00.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.791+0000 7fc34f7fe700 1 -- 172.21.15.71:0/2545207513 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc34808b680 con 0x7fc3580ffda0 2024-04-23T02:10:00.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.791+0000 7fc35dd8c700 1 --2- 172.21.15.71:0/2545207513 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc34406c4e0 0x7fc34406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:00.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.792+0000 7fc34f7fe700 1 --2- 172.21.15.71:0/2545207513 >> [v2:172.21.15.192:6808/3661697663,v1:172.21.15.192:6809/3661697663] conn(0x7fc3440721c0 0x7fc3440745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:00.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.792+0000 7fc34f7fe700 1 -- 172.21.15.71:0/2545207513 --> [v2:172.21.15.192:6808/3661697663,v1:172.21.15.192:6809/3661697663] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc344074c90 con 0x7fc3440721c0 2024-04-23T02:10:00.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.792+0000 7fc34f7fe700 1 -- 172.21.15.71:0/2545207513 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc34808dd50 con 0x7fc3580ffda0 2024-04-23T02:10:00.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.792+0000 7fc35dd8c700 1 --2- 172.21.15.71:0/2545207513 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc34406c4e0 0x7fc34406e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc354006010 tx=0x7fc35401a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:00.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.792+0000 7fc35ed8e700 1 --2- 172.21.15.71:0/2545207513 >> [v2:172.21.15.192:6808/3661697663,v1:172.21.15.192:6809/3661697663] conn(0x7fc3440721c0 0x7fc3440745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:00.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.792+0000 7fc35ed8e700 1 --2- 172.21.15.71:0/2545207513 >> [v2:172.21.15.192:6808/3661697663,v1:172.21.15.192:6809/3661697663] conn(0x7fc3440721c0 0x7fc3440745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:00.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.793+0000 7fc34f7fe700 1 -- 172.21.15.71:0/2545207513 <== osd.4 v2:172.21.15.192:6808/3661697663 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc344074c90 con 0x7fc3440721c0 2024-04-23T02:10:00.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.808+0000 7fc3607f1700 1 -- 172.21.15.71:0/2545207513 --> [v2:172.21.15.192:6808/3661697663,v1:172.21.15.192:6809/3661697663] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc35804f2e0 con 0x7fc3440721c0 2024-04-23T02:10:00.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.808+0000 7fc34f7fe700 1 -- 172.21.15.71:0/2545207513 <== osd.4 v2:172.21.15.192:6808/3661697663 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc35804f2e0 con 0x7fc3440721c0 2024-04-23T02:10:00.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.809+0000 7fc3607f1700 1 -- 172.21.15.71:0/2545207513 >> [v2:172.21.15.192:6808/3661697663,v1:172.21.15.192:6809/3661697663] conn(0x7fc3440721c0 msgr2=0x7fc3440745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:00.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.809+0000 7fc3607f1700 1 --2- 172.21.15.71:0/2545207513 >> [v2:172.21.15.192:6808/3661697663,v1:172.21.15.192:6809/3661697663] conn(0x7fc3440721c0 0x7fc3440745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:00.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.809+0000 7fc3607f1700 1 -- 172.21.15.71:0/2545207513 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc34406c4e0 msgr2=0x7fc34406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:00.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.809+0000 7fc3607f1700 1 --2- 172.21.15.71:0/2545207513 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc34406c4e0 0x7fc34406e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc354006010 tx=0x7fc35401a040 comp rx=0 tx=0).stop 2024-04-23T02:10:00.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.809+0000 7fc3607f1700 1 -- 172.21.15.71:0/2545207513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3580ffda0 msgr2=0x7fc3581976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:00.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.809+0000 7fc3607f1700 1 --2- 172.21.15.71:0/2545207513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3580ffda0 0x7fc3581976c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc34800c930 tx=0x7fc34800ccf0 comp rx=0 tx=0).stop 2024-04-23T02:10:00.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.809+0000 7fc3607f1700 1 -- 172.21.15.71:0/2545207513 shutdown_connections 2024-04-23T02:10:00.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.809+0000 7fc3607f1700 1 --2- 172.21.15.71:0/2545207513 >> [v2:172.21.15.192:6808/3661697663,v1:172.21.15.192:6809/3661697663] conn(0x7fc3440721c0 0x7fc3440745e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:00.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.809+0000 7fc3607f1700 1 --2- 172.21.15.71:0/2545207513 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc34406c4e0 0x7fc34406e9a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:00.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.809+0000 7fc3607f1700 1 --2- 172.21.15.71:0/2545207513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3580ffda0 0x7fc3581976c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:00.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.809+0000 7fc3607f1700 1 --2- 172.21.15.71:0/2545207513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc358100750 0x7fc358197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:00.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.809+0000 7fc3607f1700 1 -- 172.21.15.71:0/2545207513 >> 172.21.15.71:0/2545207513 conn(0x7fc3580fb650 msgr2=0x7fc3580fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:00.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.809+0000 7fc3607f1700 1 -- 172.21.15.71:0/2545207513 shutdown_connections 2024-04-23T02:10:00.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:00.809+0000 7fc3607f1700 1 -- 172.21.15.71:0/2545207513 wait complete. 2024-04-23T02:10:01.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:01 smithi071 ceph-mon[39837]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:01.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:01 smithi192 ceph-mon[49662]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:02.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.022+0000 7f21b371f700 1 -- 172.21.15.71:0/3763907973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21ac0ffb40 msgr2=0x7f21ac0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:02.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.022+0000 7f21b371f700 1 --2- 172.21.15.71:0/3763907973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21ac0ffb40 0x7f21ac0fffc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f21a8009910 tx=0x7f21a8009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:02.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.022+0000 7f21b371f700 1 -- 172.21.15.71:0/3763907973 shutdown_connections 2024-04-23T02:10:02.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.022+0000 7f21b371f700 1 --2- 172.21.15.71:0/3763907973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21ac0ffb40 0x7f21ac0fffc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:02.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.022+0000 7f21b371f700 1 --2- 172.21.15.71:0/3763907973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21ac105bf0 0x7f21ac105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:02.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.022+0000 7f21b371f700 1 -- 172.21.15.71:0/3763907973 >> 172.21.15.71:0/3763907973 conn(0x7f21ac0fb610 msgr2=0x7f21ac0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:02.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.022+0000 7f21b371f700 1 -- 172.21.15.71:0/3763907973 shutdown_connections 2024-04-23T02:10:02.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.022+0000 7f21b371f700 1 -- 172.21.15.71:0/3763907973 wait complete. 2024-04-23T02:10:02.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.023+0000 7f21b371f700 1 Processor -- start 2024-04-23T02:10:02.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.023+0000 7f21b371f700 1 -- start start 2024-04-23T02:10:02.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.023+0000 7f21b371f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21ac0ffb40 0x7f21ac197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:02.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.023+0000 7f21b371f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21ac105bf0 0x7f21ac197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:02.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.023+0000 7f21b371f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac1980f0 con 0x7f21ac0ffb40 2024-04-23T02:10:02.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.023+0000 7f21b371f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac198230 con 0x7f21ac105bf0 2024-04-23T02:10:02.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.023+0000 7f21b14bb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21ac0ffb40 0x7f21ac197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:02.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.023+0000 7f21b14bb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21ac0ffb40 0x7f21ac197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46104/0 (socket says 172.21.15.71:46104) 2024-04-23T02:10:02.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.023+0000 7f21b14bb700 1 -- 172.21.15.71:0/1205670915 learned_addr learned my addr 172.21.15.71:0/1205670915 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:02.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.023+0000 7f21b14bb700 1 -- 172.21.15.71:0/1205670915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21ac105bf0 msgr2=0x7f21ac197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:02.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.023+0000 7f21b14bb700 1 --2- 172.21.15.71:0/1205670915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21ac105bf0 0x7f21ac197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:02.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.023+0000 7f21b14bb700 1 -- 172.21.15.71:0/1205670915 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a80095f0 con 0x7f21ac0ffb40 2024-04-23T02:10:02.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.023+0000 7f21b14bb700 1 --2- 172.21.15.71:0/1205670915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21ac0ffb40 0x7f21ac197590 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f219c00d8d0 tx=0x7f219c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:02.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.023+0000 7f21a27fc700 1 -- 172.21.15.71:0/1205670915 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f219c009940 con 0x7f21ac0ffb40 2024-04-23T02:10:02.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.023+0000 7f21a27fc700 1 -- 172.21.15.71:0/1205670915 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f219c010460 con 0x7f21ac0ffb40 2024-04-23T02:10:02.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.023+0000 7f21a27fc700 1 -- 172.21.15.71:0/1205670915 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f219c00f5f0 con 0x7f21ac0ffb40 2024-04-23T02:10:02.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.023+0000 7f21b371f700 1 -- 172.21.15.71:0/1205670915 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21ac1a0600 con 0x7f21ac0ffb40 2024-04-23T02:10:02.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.023+0000 7f21b371f700 1 -- 172.21.15.71:0/1205670915 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ac1a0ac0 con 0x7f21ac0ffb40 2024-04-23T02:10:02.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.024+0000 7f21b371f700 1 -- 172.21.15.71:0/1205670915 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f21ac10ceb0 con 0x7f21ac0ffb40 2024-04-23T02:10:02.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.025+0000 7f21a27fc700 1 -- 172.21.15.71:0/1205670915 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f219c0105d0 con 0x7f21ac0ffb40 2024-04-23T02:10:02.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.025+0000 7f21a27fc700 1 --2- 172.21.15.71:0/1205670915 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f219806c4e0 0x7f219806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:02.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.025+0000 7f21b0cba700 1 --2- 172.21.15.71:0/1205670915 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f219806c4e0 0x7f219806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:02.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.025+0000 7f21a27fc700 1 -- 172.21.15.71:0/1205670915 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f219c08b520 con 0x7f21ac0ffb40 2024-04-23T02:10:02.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.025+0000 7f21a27fc700 1 --2- 172.21.15.71:0/1205670915 >> [v2:172.21.15.71:6802/275655379,v1:172.21.15.71:6803/275655379] conn(0x7f21980721c0 0x7f21980745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:02.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.025+0000 7f21a27fc700 1 -- 172.21.15.71:0/1205670915 --> [v2:172.21.15.71:6802/275655379,v1:172.21.15.71:6803/275655379] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2198074c90 con 0x7f21980721c0 2024-04-23T02:10:02.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.025+0000 7f21a27fc700 1 -- 172.21.15.71:0/1205670915 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f219c08dbf0 con 0x7f21ac0ffb40 2024-04-23T02:10:02.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.025+0000 7f21b1cbc700 1 --2- 172.21.15.71:0/1205670915 >> [v2:172.21.15.71:6802/275655379,v1:172.21.15.71:6803/275655379] conn(0x7f21980721c0 0x7f21980745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:02.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.025+0000 7f21b0cba700 1 --2- 172.21.15.71:0/1205670915 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f219806c4e0 0x7f219806e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f21a800b5c0 tx=0x7f21a8005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:02.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.026+0000 7f21b1cbc700 1 --2- 172.21.15.71:0/1205670915 >> [v2:172.21.15.71:6802/275655379,v1:172.21.15.71:6803/275655379] conn(0x7f21980721c0 0x7f21980745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:02.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.026+0000 7f21a27fc700 1 -- 172.21.15.71:0/1205670915 <== osd.0 v2:172.21.15.71:6802/275655379 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2198074c90 con 0x7f21980721c0 2024-04-23T02:10:02.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.041+0000 7f21b371f700 1 -- 172.21.15.71:0/1205670915 --> [v2:172.21.15.71:6802/275655379,v1:172.21.15.71:6803/275655379] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f21ac0679c0 con 0x7f21980721c0 2024-04-23T02:10:02.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.042+0000 7f21a27fc700 1 -- 172.21.15.71:0/1205670915 <== osd.0 v2:172.21.15.71:6802/275655379 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f21ac0679c0 con 0x7f21980721c0 2024-04-23T02:10:02.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.042+0000 7f21b371f700 1 -- 172.21.15.71:0/1205670915 >> [v2:172.21.15.71:6802/275655379,v1:172.21.15.71:6803/275655379] conn(0x7f21980721c0 msgr2=0x7f21980745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:02.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.042+0000 7f21b371f700 1 --2- 172.21.15.71:0/1205670915 >> [v2:172.21.15.71:6802/275655379,v1:172.21.15.71:6803/275655379] conn(0x7f21980721c0 0x7f21980745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:02.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.042+0000 7f21b371f700 1 -- 172.21.15.71:0/1205670915 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f219806c4e0 msgr2=0x7f219806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:02.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.042+0000 7f21b371f700 1 --2- 172.21.15.71:0/1205670915 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f219806c4e0 0x7f219806e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f21a800b5c0 tx=0x7f21a8005960 comp rx=0 tx=0).stop 2024-04-23T02:10:02.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.042+0000 7f21b371f700 1 -- 172.21.15.71:0/1205670915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21ac0ffb40 msgr2=0x7f21ac197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:02.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.042+0000 7f21b371f700 1 --2- 172.21.15.71:0/1205670915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21ac0ffb40 0x7f21ac197590 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f219c00d8d0 tx=0x7f219c00dc90 comp rx=0 tx=0).stop 2024-04-23T02:10:02.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.042+0000 7f21b371f700 1 -- 172.21.15.71:0/1205670915 shutdown_connections 2024-04-23T02:10:02.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.042+0000 7f21b371f700 1 --2- 172.21.15.71:0/1205670915 >> [v2:172.21.15.71:6802/275655379,v1:172.21.15.71:6803/275655379] conn(0x7f21980721c0 0x7f21980745e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:02.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.042+0000 7f21b371f700 1 --2- 172.21.15.71:0/1205670915 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f219806c4e0 0x7f219806e9a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:02.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.042+0000 7f21b371f700 1 --2- 172.21.15.71:0/1205670915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21ac0ffb40 0x7f21ac197590 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:02.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.042+0000 7f21b371f700 1 --2- 172.21.15.71:0/1205670915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21ac105bf0 0x7f21ac197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:02.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.042+0000 7f21b371f700 1 -- 172.21.15.71:0/1205670915 >> 172.21.15.71:0/1205670915 conn(0x7f21ac0fb610 msgr2=0x7f21ac0fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:02.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.042+0000 7f21b371f700 1 -- 172.21.15.71:0/1205670915 shutdown_connections 2024-04-23T02:10:02.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.042+0000 7f21b371f700 1 -- 172.21.15.71:0/1205670915 wait complete. 2024-04-23T02:10:02.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.552+0000 7f0622b39700 1 -- 172.21.15.71:0/2540222019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c0ffb40 msgr2=0x7f061c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:02.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.552+0000 7f0622b39700 1 --2- 172.21.15.71:0/2540222019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c0ffb40 0x7f061c0fffc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0608009910 tx=0x7f0608009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:02.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.552+0000 7f0622b39700 1 -- 172.21.15.71:0/2540222019 shutdown_connections 2024-04-23T02:10:02.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.552+0000 7f0622b39700 1 --2- 172.21.15.71:0/2540222019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c0ffb40 0x7f061c0fffc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:02.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.552+0000 7f0622b39700 1 --2- 172.21.15.71:0/2540222019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f061c105bf0 0x7f061c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:02.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.552+0000 7f0622b39700 1 -- 172.21.15.71:0/2540222019 >> 172.21.15.71:0/2540222019 conn(0x7f061c0fb610 msgr2=0x7f061c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:02.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.552+0000 7f0622b39700 1 -- 172.21.15.71:0/2540222019 shutdown_connections 2024-04-23T02:10:02.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.552+0000 7f0622b39700 1 -- 172.21.15.71:0/2540222019 wait complete. 2024-04-23T02:10:02.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.553+0000 7f0622b39700 1 Processor -- start 2024-04-23T02:10:02.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.553+0000 7f0622b39700 1 -- start start 2024-04-23T02:10:02.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.553+0000 7f0622b39700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c0ffb40 0x7f061c1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:02.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.553+0000 7f0622b39700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f061c105bf0 0x7f061c101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:02.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.553+0000 7f0622b39700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c102230 con 0x7f061c0ffb40 2024-04-23T02:10:02.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.553+0000 7f0622b39700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c102370 con 0x7f061c105bf0 2024-04-23T02:10:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.553+0000 7f06208d5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c0ffb40 0x7f061c1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.553+0000 7f06208d5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c0ffb40 0x7f061c1017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46122/0 (socket says 172.21.15.71:46122) 2024-04-23T02:10:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.553+0000 7f06208d5700 1 -- 172.21.15.71:0/3751272053 learned_addr learned my addr 172.21.15.71:0/3751272053 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.553+0000 7f061bfff700 1 --2- 172.21.15.71:0/3751272053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f061c105bf0 0x7f061c101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.553+0000 7f06208d5700 1 -- 172.21.15.71:0/3751272053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f061c105bf0 msgr2=0x7f061c101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.553+0000 7f06208d5700 1 --2- 172.21.15.71:0/3751272053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f061c105bf0 0x7f061c101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.553+0000 7f06208d5700 1 -- 172.21.15.71:0/3751272053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06080095f0 con 0x7f061c0ffb40 2024-04-23T02:10:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.554+0000 7f06208d5700 1 --2- 172.21.15.71:0/3751272053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c0ffb40 0x7f061c1017b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f061000d8d0 tx=0x7f061000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.554+0000 7f0619ffb700 1 -- 172.21.15.71:0/3751272053 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06100098a0 con 0x7f061c0ffb40 2024-04-23T02:10:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.554+0000 7f0619ffb700 1 -- 172.21.15.71:0/3751272053 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f0610009e90 con 0x7f061c0ffb40 2024-04-23T02:10:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.554+0000 7f0622b39700 1 -- 172.21.15.71:0/3751272053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061c1a28f0 con 0x7f061c0ffb40 2024-04-23T02:10:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.554+0000 7f0619ffb700 1 -- 172.21.15.71:0/3751272053 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f061000f5f0 con 0x7f061c0ffb40 2024-04-23T02:10:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.554+0000 7f0622b39700 1 -- 172.21.15.71:0/3751272053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c1a2d00 con 0x7f061c0ffb40 2024-04-23T02:10:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.555+0000 7f0622b39700 1 -- 172.21.15.71:0/3751272053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f061c10ceb0 con 0x7f061c0ffb40 2024-04-23T02:10:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.555+0000 7f0619ffb700 1 -- 172.21.15.71:0/3751272053 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f0610010460 con 0x7f061c0ffb40 2024-04-23T02:10:02.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.555+0000 7f0619ffb700 1 --2- 172.21.15.71:0/3751272053 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f060c06c4e0 0x7f060c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:02.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.556+0000 7f0619ffb700 1 -- 172.21.15.71:0/3751272053 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f061008b2d0 con 0x7f061c0ffb40 2024-04-23T02:10:02.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.556+0000 7f061bfff700 1 --2- 172.21.15.71:0/3751272053 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f060c06c4e0 0x7f060c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:02.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.556+0000 7f0619ffb700 1 --2- 172.21.15.71:0/3751272053 >> [v2:172.21.15.192:6800/593532086,v1:172.21.15.192:6801/593532086] conn(0x7f060c0721c0 0x7f060c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:02.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.556+0000 7f0619ffb700 1 -- 172.21.15.71:0/3751272053 --> [v2:172.21.15.192:6800/593532086,v1:172.21.15.192:6801/593532086] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f060c074c90 con 0x7f060c0721c0 2024-04-23T02:10:02.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.556+0000 7f0619ffb700 1 -- 172.21.15.71:0/3751272053 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f061008d9a0 con 0x7f061c0ffb40 2024-04-23T02:10:02.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.556+0000 7f061bfff700 1 --2- 172.21.15.71:0/3751272053 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f060c06c4e0 0x7f060c06e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f061c102fa0 tx=0x7f060801a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:02.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.556+0000 7f06210d6700 1 --2- 172.21.15.71:0/3751272053 >> [v2:172.21.15.192:6800/593532086,v1:172.21.15.192:6801/593532086] conn(0x7f060c0721c0 0x7f060c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:02.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.556+0000 7f06210d6700 1 --2- 172.21.15.71:0/3751272053 >> [v2:172.21.15.192:6800/593532086,v1:172.21.15.192:6801/593532086] conn(0x7f060c0721c0 0x7f060c0745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:02.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.557+0000 7f0619ffb700 1 -- 172.21.15.71:0/3751272053 <== osd.3 v2:172.21.15.192:6800/593532086 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f060c074c90 con 0x7f060c0721c0 2024-04-23T02:10:02.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.572+0000 7f0622b39700 1 -- 172.21.15.71:0/3751272053 --> [v2:172.21.15.192:6800/593532086,v1:172.21.15.192:6801/593532086] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f061c0679c0 con 0x7f060c0721c0 2024-04-23T02:10:02.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.573+0000 7f0619ffb700 1 -- 172.21.15.71:0/3751272053 <== osd.3 v2:172.21.15.192:6800/593532086 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f061c0679c0 con 0x7f060c0721c0 2024-04-23T02:10:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.573+0000 7f0622b39700 1 -- 172.21.15.71:0/3751272053 >> [v2:172.21.15.192:6800/593532086,v1:172.21.15.192:6801/593532086] conn(0x7f060c0721c0 msgr2=0x7f060c0745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.573+0000 7f0622b39700 1 --2- 172.21.15.71:0/3751272053 >> [v2:172.21.15.192:6800/593532086,v1:172.21.15.192:6801/593532086] conn(0x7f060c0721c0 0x7f060c0745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.573+0000 7f0622b39700 1 -- 172.21.15.71:0/3751272053 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f060c06c4e0 msgr2=0x7f060c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.573+0000 7f0622b39700 1 --2- 172.21.15.71:0/3751272053 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f060c06c4e0 0x7f060c06e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f061c102fa0 tx=0x7f060801a040 comp rx=0 tx=0).stop 2024-04-23T02:10:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.573+0000 7f0622b39700 1 -- 172.21.15.71:0/3751272053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c0ffb40 msgr2=0x7f061c1017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.573+0000 7f0622b39700 1 --2- 172.21.15.71:0/3751272053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c0ffb40 0x7f061c1017b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f061000d8d0 tx=0x7f061000dbe0 comp rx=0 tx=0).stop 2024-04-23T02:10:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.573+0000 7f0622b39700 1 -- 172.21.15.71:0/3751272053 shutdown_connections 2024-04-23T02:10:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.573+0000 7f0622b39700 1 --2- 172.21.15.71:0/3751272053 >> [v2:172.21.15.192:6800/593532086,v1:172.21.15.192:6801/593532086] conn(0x7f060c0721c0 0x7f060c0745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.573+0000 7f0622b39700 1 --2- 172.21.15.71:0/3751272053 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f060c06c4e0 0x7f060c06e9a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.573+0000 7f0622b39700 1 --2- 172.21.15.71:0/3751272053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c0ffb40 0x7f061c1017b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.573+0000 7f0622b39700 1 --2- 172.21.15.71:0/3751272053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f061c105bf0 0x7f061c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.573+0000 7f0622b39700 1 -- 172.21.15.71:0/3751272053 >> 172.21.15.71:0/3751272053 conn(0x7f061c0fb610 msgr2=0x7f061c068f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.573+0000 7f0622b39700 1 -- 172.21.15.71:0/3751272053 shutdown_connections 2024-04-23T02:10:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:02.573+0000 7f0622b39700 1 -- 172.21.15.71:0/3751272053 wait complete. 2024-04-23T02:10:02.969 INFO:teuthology.orchestra.run.smithi071.stdout:60129542176 2024-04-23T02:10:02.970 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-23T02:10:03.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:03 smithi071 ceph-mon[39837]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:03.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:03 smithi192 ceph-mon[49662]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:03.637 INFO:teuthology.orchestra.run.smithi071.stdout:111669149716 2024-04-23T02:10:03.637 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-23T02:10:04.491 INFO:teuthology.orchestra.run.smithi071.stdout:38654705702 2024-04-23T02:10:04.492 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-23T02:10:05.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.074+0000 7ff4be57b700 1 -- 172.21.15.71:0/1301263352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4b8100750 msgr2=0x7ff4b81047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.074+0000 7ff4be57b700 1 --2- 172.21.15.71:0/1301263352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4b8100750 0x7ff4b81047a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009910 tx=0x7ff4a8009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.074+0000 7ff4be57b700 1 -- 172.21.15.71:0/1301263352 shutdown_connections 2024-04-23T02:10:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.074+0000 7ff4be57b700 1 --2- 172.21.15.71:0/1301263352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4b8100750 0x7ff4b81047a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.074+0000 7ff4be57b700 1 --2- 172.21.15.71:0/1301263352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4b80ffda0 0x7ff4b8100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.074+0000 7ff4be57b700 1 -- 172.21.15.71:0/1301263352 >> 172.21.15.71:0/1301263352 conn(0x7ff4b80fb650 msgr2=0x7ff4b80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.074+0000 7ff4be57b700 1 -- 172.21.15.71:0/1301263352 shutdown_connections 2024-04-23T02:10:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.074+0000 7ff4be57b700 1 -- 172.21.15.71:0/1301263352 wait complete. 2024-04-23T02:10:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.074+0000 7ff4be57b700 1 Processor -- start 2024-04-23T02:10:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.074+0000 7ff4be57b700 1 -- start start 2024-04-23T02:10:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.075+0000 7ff4be57b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4b80ffda0 0x7ff4b81976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.075+0000 7ff4be57b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4b8100750 0x7ff4b8197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.075+0000 7ff4be57b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b8198240 con 0x7ff4b80ffda0 2024-04-23T02:10:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.075+0000 7ff4be57b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b81983b0 con 0x7ff4b8100750 2024-04-23T02:10:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.075+0000 7ff4b7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4b80ffda0 0x7ff4b81976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.075+0000 7ff4b7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4b80ffda0 0x7ff4b81976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46172/0 (socket says 172.21.15.71:46172) 2024-04-23T02:10:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.075+0000 7ff4b7fff700 1 -- 172.21.15.71:0/175520223 learned_addr learned my addr 172.21.15.71:0/175520223 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.075+0000 7ff4b77fe700 1 --2- 172.21.15.71:0/175520223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4b8100750 0x7ff4b8197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.075+0000 7ff4b7fff700 1 -- 172.21.15.71:0/175520223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4b8100750 msgr2=0x7ff4b8197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.075+0000 7ff4b7fff700 1 --2- 172.21.15.71:0/175520223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4b8100750 0x7ff4b8197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.075+0000 7ff4b7fff700 1 -- 172.21.15.71:0/175520223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a80095f0 con 0x7ff4b80ffda0 2024-04-23T02:10:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.075+0000 7ff4b7fff700 1 --2- 172.21.15.71:0/175520223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4b80ffda0 0x7ff4b81976e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00d8d0 tx=0x7ff4ac00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.075+0000 7ff4b57fa700 1 -- 172.21.15.71:0/175520223 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4ac0098a0 con 0x7ff4b80ffda0 2024-04-23T02:10:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.075+0000 7ff4b57fa700 1 -- 172.21.15.71:0/175520223 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ff4ac009e90 con 0x7ff4b80ffda0 2024-04-23T02:10:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.075+0000 7ff4b57fa700 1 -- 172.21.15.71:0/175520223 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4ac00f5f0 con 0x7ff4b80ffda0 2024-04-23T02:10:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.075+0000 7ff4be57b700 1 -- 172.21.15.71:0/175520223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b81a0750 con 0x7ff4b80ffda0 2024-04-23T02:10:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.075+0000 7ff4be57b700 1 -- 172.21.15.71:0/175520223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b81a0c10 con 0x7ff4b80ffda0 2024-04-23T02:10:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.076+0000 7ff4be57b700 1 -- 172.21.15.71:0/175520223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff4b810ce80 con 0x7ff4b80ffda0 2024-04-23T02:10:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.077+0000 7ff4b57fa700 1 -- 172.21.15.71:0/175520223 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7ff4ac010460 con 0x7ff4b80ffda0 2024-04-23T02:10:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.077+0000 7ff4b57fa700 1 --2- 172.21.15.71:0/175520223 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7ff4a006c4e0 0x7ff4a006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.077+0000 7ff4b57fa700 1 -- 172.21.15.71:0/175520223 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff4ac08b3b0 con 0x7ff4b80ffda0 2024-04-23T02:10:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.077+0000 7ff4b77fe700 1 --2- 172.21.15.71:0/175520223 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7ff4a006c4e0 0x7ff4a006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.077+0000 7ff4b57fa700 1 --2- 172.21.15.71:0/175520223 >> [v2:172.21.15.192:6816/3375258254,v1:172.21.15.192:6817/3375258254] conn(0x7ff4a00721c0 0x7ff4a00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.077+0000 7ff4b57fa700 1 -- 172.21.15.71:0/175520223 --> [v2:172.21.15.192:6816/3375258254,v1:172.21.15.192:6817/3375258254] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff4a0074c90 con 0x7ff4a00721c0 2024-04-23T02:10:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.077+0000 7ff4b57fa700 1 -- 172.21.15.71:0/175520223 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4ac08da80 con 0x7ff4b80ffda0 2024-04-23T02:10:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.077+0000 7ff4b77fe700 1 --2- 172.21.15.71:0/175520223 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7ff4a006c4e0 0x7ff4a006e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8006010 tx=0x7ff4a801a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.077+0000 7ff4bcb18700 1 --2- 172.21.15.71:0/175520223 >> [v2:172.21.15.192:6816/3375258254,v1:172.21.15.192:6817/3375258254] conn(0x7ff4a00721c0 0x7ff4a00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.078+0000 7ff4bcb18700 1 --2- 172.21.15.71:0/175520223 >> [v2:172.21.15.192:6816/3375258254,v1:172.21.15.192:6817/3375258254] conn(0x7ff4a00721c0 0x7ff4a00745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.079+0000 7ff4b57fa700 1 -- 172.21.15.71:0/175520223 <== osd.5 v2:172.21.15.192:6816/3375258254 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff4a0074c90 con 0x7ff4a00721c0 2024-04-23T02:10:05.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.094+0000 7ff4be57b700 1 -- 172.21.15.71:0/175520223 --> [v2:172.21.15.192:6816/3375258254,v1:172.21.15.192:6817/3375258254] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff4b804f2e0 con 0x7ff4a00721c0 2024-04-23T02:10:05.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.095+0000 7ff4b57fa700 1 -- 172.21.15.71:0/175520223 <== osd.5 v2:172.21.15.192:6816/3375258254 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff4b804f2e0 con 0x7ff4a00721c0 2024-04-23T02:10:05.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.095+0000 7ff4be57b700 1 -- 172.21.15.71:0/175520223 >> [v2:172.21.15.192:6816/3375258254,v1:172.21.15.192:6817/3375258254] conn(0x7ff4a00721c0 msgr2=0x7ff4a00745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:05.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.095+0000 7ff4be57b700 1 --2- 172.21.15.71:0/175520223 >> [v2:172.21.15.192:6816/3375258254,v1:172.21.15.192:6817/3375258254] conn(0x7ff4a00721c0 0x7ff4a00745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:05.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.095+0000 7ff4be57b700 1 -- 172.21.15.71:0/175520223 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7ff4a006c4e0 msgr2=0x7ff4a006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:05.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.095+0000 7ff4be57b700 1 --2- 172.21.15.71:0/175520223 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7ff4a006c4e0 0x7ff4a006e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8006010 tx=0x7ff4a801a040 comp rx=0 tx=0).stop 2024-04-23T02:10:05.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.095+0000 7ff4be57b700 1 -- 172.21.15.71:0/175520223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4b80ffda0 msgr2=0x7ff4b81976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:05.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.095+0000 7ff4be57b700 1 --2- 172.21.15.71:0/175520223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4b80ffda0 0x7ff4b81976e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00d8d0 tx=0x7ff4ac00dbe0 comp rx=0 tx=0).stop 2024-04-23T02:10:05.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.095+0000 7ff4be57b700 1 -- 172.21.15.71:0/175520223 shutdown_connections 2024-04-23T02:10:05.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.095+0000 7ff4be57b700 1 --2- 172.21.15.71:0/175520223 >> [v2:172.21.15.192:6816/3375258254,v1:172.21.15.192:6817/3375258254] conn(0x7ff4a00721c0 0x7ff4a00745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:05.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.095+0000 7ff4be57b700 1 --2- 172.21.15.71:0/175520223 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7ff4a006c4e0 0x7ff4a006e9a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:05.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.095+0000 7ff4be57b700 1 --2- 172.21.15.71:0/175520223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4b80ffda0 0x7ff4b81976e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:05.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.095+0000 7ff4be57b700 1 --2- 172.21.15.71:0/175520223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4b8100750 0x7ff4b8197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:05.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.095+0000 7ff4be57b700 1 -- 172.21.15.71:0/175520223 >> 172.21.15.71:0/175520223 conn(0x7ff4b80fb650 msgr2=0x7ff4b80fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:05.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.095+0000 7ff4be57b700 1 -- 172.21.15.71:0/175520223 shutdown_connections 2024-04-23T02:10:05.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:05.095+0000 7ff4be57b700 1 -- 172.21.15.71:0/175520223 wait complete. 2024-04-23T02:10:05.447 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:05 smithi071 ceph-mon[39837]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:05.466 INFO:teuthology.orchestra.run.smithi071.stdout:146028888077 2024-04-23T02:10:05.466 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-23T02:10:05.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.213+0000 7f4fe6154700 1 -- 172.21.15.71:0/2059728932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fe0100ea0 msgr2=0x7f4fe0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:05.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.213+0000 7f4fe6154700 1 --2- 172.21.15.71:0/2059728932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fe0100ea0 0x7f4fe0101280 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009910 tx=0x7f4fc8009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:05.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.213+0000 7f4fe6154700 1 -- 172.21.15.71:0/2059728932 shutdown_connections 2024-04-23T02:10:05.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.213+0000 7f4fe6154700 1 --2- 172.21.15.71:0/2059728932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fe0101850 0x7f4fe01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:05.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.213+0000 7f4fe6154700 1 --2- 172.21.15.71:0/2059728932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fe0100ea0 0x7f4fe0101280 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:05.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.213+0000 7f4fe6154700 1 -- 172.21.15.71:0/2059728932 >> 172.21.15.71:0/2059728932 conn(0x7f4fe00fc750 msgr2=0x7f4fe00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:05.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.213+0000 7f4fe6154700 1 -- 172.21.15.71:0/2059728932 shutdown_connections 2024-04-23T02:10:05.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.213+0000 7f4fe6154700 1 -- 172.21.15.71:0/2059728932 wait complete. 2024-04-23T02:10:05.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.214+0000 7f4fe6154700 1 Processor -- start 2024-04-23T02:10:05.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.214+0000 7f4fe6154700 1 -- start start 2024-04-23T02:10:05.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.214+0000 7f4fe6154700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fe0100ea0 0x7f4fe019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:05.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.214+0000 7f4fe6154700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fe0101850 0x7f4fe019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:05.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.214+0000 7f4fe6154700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe019d130 con 0x7f4fe0100ea0 2024-04-23T02:10:05.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.214+0000 7f4fe6154700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe01965e0 con 0x7f4fe0101850 2024-04-23T02:10:05.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.214+0000 7f4fdf7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fe0100ea0 0x7f4fe019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:05.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.214+0000 7f4fdf7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fe0100ea0 0x7f4fe019c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46154/0 (socket says 172.21.15.71:46154) 2024-04-23T02:10:05.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.214+0000 7f4fdf7fe700 1 -- 172.21.15.71:0/2118778538 learned_addr learned my addr 172.21.15.71:0/2118778538 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:05.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.214+0000 7f4fdeffd700 1 --2- 172.21.15.71:0/2118778538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fe0101850 0x7f4fe019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:05.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.215+0000 7f4fdf7fe700 1 -- 172.21.15.71:0/2118778538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fe0101850 msgr2=0x7f4fe019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:05.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.215+0000 7f4fdf7fe700 1 --2- 172.21.15.71:0/2118778538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fe0101850 0x7f4fe019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.215+0000 7f4fdf7fe700 1 -- 172.21.15.71:0/2118778538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc80095f0 con 0x7f4fe0100ea0 2024-04-23T02:10:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.215+0000 7f4fdeffd700 1 --2- 172.21.15.71:0/2118778538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fe0101850 0x7f4fe019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:10:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.215+0000 7f4fdf7fe700 1 --2- 172.21.15.71:0/2118778538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fe0100ea0 0x7f4fe019c560 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8006010 tx=0x7f4fc800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.215+0000 7f4fdcff9700 1 -- 172.21.15.71:0/2118778538 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fc801d070 con 0x7f4fe0100ea0 2024-04-23T02:10:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.215+0000 7f4fdcff9700 1 -- 172.21.15.71:0/2118778538 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4fc800f460 con 0x7f4fe0100ea0 2024-04-23T02:10:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.215+0000 7f4fe6154700 1 -- 172.21.15.71:0/2118778538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe0196860 con 0x7f4fe0100ea0 2024-04-23T02:10:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.215+0000 7f4fdcff9700 1 -- 172.21.15.71:0/2118778538 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fc8021620 con 0x7f4fe0100ea0 2024-04-23T02:10:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.215+0000 7f4fe6154700 1 -- 172.21.15.71:0/2118778538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe0196d50 con 0x7f4fe0100ea0 2024-04-23T02:10:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.216+0000 7f4fe6154700 1 -- 172.21.15.71:0/2118778538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4fe010df70 con 0x7f4fe0100ea0 2024-04-23T02:10:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.216+0000 7f4fdcff9700 1 -- 172.21.15.71:0/2118778538 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f4fc800fa40 con 0x7f4fe0100ea0 2024-04-23T02:10:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.217+0000 7f4fdcff9700 1 --2- 172.21.15.71:0/2118778538 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4fcc06c4e0 0x7f4fcc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.217+0000 7f4fdeffd700 1 --2- 172.21.15.71:0/2118778538 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4fcc06c4e0 0x7f4fcc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.217+0000 7f4fdcff9700 1 -- 172.21.15.71:0/2118778538 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4fc808c6e0 con 0x7f4fe0100ea0 2024-04-23T02:10:05.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.217+0000 7f4fdcff9700 1 --2- 172.21.15.71:0/2118778538 >> [v2:172.21.15.71:6818/2096213030,v1:172.21.15.71:6819/2096213030] conn(0x7f4fcc0721c0 0x7f4fcc0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:05.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.217+0000 7f4fdcff9700 1 -- 172.21.15.71:0/2118778538 --> [v2:172.21.15.71:6818/2096213030,v1:172.21.15.71:6819/2096213030] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fcc074c90 con 0x7f4fcc0721c0 2024-04-23T02:10:05.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.217+0000 7f4fdcff9700 1 -- 172.21.15.71:0/2118778538 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fc808edb0 con 0x7f4fe0100ea0 2024-04-23T02:10:05.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.217+0000 7f4fdffff700 1 --2- 172.21.15.71:0/2118778538 >> [v2:172.21.15.71:6818/2096213030,v1:172.21.15.71:6819/2096213030] conn(0x7f4fcc0721c0 0x7f4fcc0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:05.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.217+0000 7f4fdeffd700 1 --2- 172.21.15.71:0/2118778538 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4fcc06c4e0 0x7f4fcc06e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0005950 tx=0x7f4fd00058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:05.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.217+0000 7f4fdffff700 1 --2- 172.21.15.71:0/2118778538 >> [v2:172.21.15.71:6818/2096213030,v1:172.21.15.71:6819/2096213030] conn(0x7f4fcc0721c0 0x7f4fcc0745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:05.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.218+0000 7f4fdcff9700 1 -- 172.21.15.71:0/2118778538 <== osd.2 v2:172.21.15.71:6818/2096213030 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4fcc074c90 con 0x7f4fcc0721c0 2024-04-23T02:10:05.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.233+0000 7f4fe6154700 1 -- 172.21.15.71:0/2118778538 --> [v2:172.21.15.71:6818/2096213030,v1:172.21.15.71:6819/2096213030] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4fe004f2e0 con 0x7f4fcc0721c0 2024-04-23T02:10:05.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.233+0000 7f4fdcff9700 1 -- 172.21.15.71:0/2118778538 <== osd.2 v2:172.21.15.71:6818/2096213030 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4fe004f2e0 con 0x7f4fcc0721c0 2024-04-23T02:10:05.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.234+0000 7f4fe6154700 1 -- 172.21.15.71:0/2118778538 >> [v2:172.21.15.71:6818/2096213030,v1:172.21.15.71:6819/2096213030] conn(0x7f4fcc0721c0 msgr2=0x7f4fcc0745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:05.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.234+0000 7f4fe6154700 1 --2- 172.21.15.71:0/2118778538 >> [v2:172.21.15.71:6818/2096213030,v1:172.21.15.71:6819/2096213030] conn(0x7f4fcc0721c0 0x7f4fcc0745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:05.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.234+0000 7f4fe6154700 1 -- 172.21.15.71:0/2118778538 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4fcc06c4e0 msgr2=0x7f4fcc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:05.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.234+0000 7f4fe6154700 1 --2- 172.21.15.71:0/2118778538 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4fcc06c4e0 0x7f4fcc06e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0005950 tx=0x7f4fd00058e0 comp rx=0 tx=0).stop 2024-04-23T02:10:05.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.234+0000 7f4fe6154700 1 -- 172.21.15.71:0/2118778538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fe0100ea0 msgr2=0x7f4fe019c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:05.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.234+0000 7f4fe6154700 1 --2- 172.21.15.71:0/2118778538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fe0100ea0 0x7f4fe019c560 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8006010 tx=0x7f4fc800be20 comp rx=0 tx=0).stop 2024-04-23T02:10:05.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.234+0000 7f4fe6154700 1 -- 172.21.15.71:0/2118778538 shutdown_connections 2024-04-23T02:10:05.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.234+0000 7f4fe6154700 1 --2- 172.21.15.71:0/2118778538 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4fcc06c4e0 0x7f4fcc06e9a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:05.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.234+0000 7f4fe6154700 1 --2- 172.21.15.71:0/2118778538 >> [v2:172.21.15.71:6818/2096213030,v1:172.21.15.71:6819/2096213030] conn(0x7f4fcc0721c0 0x7f4fcc0745e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:05.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.234+0000 7f4fe6154700 1 --2- 172.21.15.71:0/2118778538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fe0100ea0 0x7f4fe019c560 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:05.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.234+0000 7f4fe6154700 1 --2- 172.21.15.71:0/2118778538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fe0101850 0x7f4fe019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:05.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.234+0000 7f4fe6154700 1 -- 172.21.15.71:0/2118778538 >> 172.21.15.71:0/2118778538 conn(0x7f4fe00fc750 msgr2=0x7f4fe00fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:05.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.234+0000 7f4fe6154700 1 -- 172.21.15.71:0/2118778538 shutdown_connections 2024-04-23T02:10:05.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:04.234+0000 7f4fe6154700 1 -- 172.21.15.71:0/2118778538 wait complete. 2024-04-23T02:10:05.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:05 smithi192 ceph-mon[49662]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:06.463 INFO:teuthology.orchestra.run.smithi071.stdout:81604378650 2024-04-23T02:10:06.464 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-23T02:10:06.831 INFO:teuthology.orchestra.run.smithi071.stdout:167503724553 2024-04-23T02:10:06.832 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-23T02:10:06.888 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:10:06.891 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:10:06.945 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:10:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:06 smithi071 ceph-mon[39837]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:07.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:06 smithi192 ceph-mon[49662]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:07.152 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:10:08.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.890+0000 7f808b75e700 1 -- 172.21.15.71:0/2345190729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8084105bd0 msgr2=0x7f8084105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:08.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.890+0000 7f808b75e700 1 --2- 172.21.15.71:0/2345190729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8084105bd0 0x7f8084105fb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8074009910 tx=0x7f8074009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:08.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.890+0000 7f808b75e700 1 -- 172.21.15.71:0/2345190729 shutdown_connections 2024-04-23T02:10:08.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.890+0000 7f808b75e700 1 --2- 172.21.15.71:0/2345190729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80840ffc20 0x7f8084100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:08.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.890+0000 7f808b75e700 1 --2- 172.21.15.71:0/2345190729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8084105bd0 0x7f8084105fb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:08.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.890+0000 7f808b75e700 1 -- 172.21.15.71:0/2345190729 >> 172.21.15.71:0/2345190729 conn(0x7f8084075670 msgr2=0x7f8084075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:08.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.890+0000 7f808b75e700 1 -- 172.21.15.71:0/2345190729 shutdown_connections 2024-04-23T02:10:08.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.890+0000 7f808b75e700 1 -- 172.21.15.71:0/2345190729 wait complete. 2024-04-23T02:10:08.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.891+0000 7f808b75e700 1 Processor -- start 2024-04-23T02:10:08.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.891+0000 7f808b75e700 1 -- start start 2024-04-23T02:10:08.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.891+0000 7f808b75e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80840ffc20 0x7f808419b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:08.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.891+0000 7f808b75e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8084105bd0 0x7f808419ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:08.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.891+0000 7f808b75e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808419c110 con 0x7f8084105bd0 2024-04-23T02:10:08.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.891+0000 7f808b75e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80841955c0 con 0x7f80840ffc20 2024-04-23T02:10:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.892+0000 7f8088cf9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8084105bd0 0x7f808419ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.892+0000 7f8088cf9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8084105bd0 0x7f808419ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44932/0 (socket says 172.21.15.71:44932) 2024-04-23T02:10:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.892+0000 7f8088cf9700 1 -- 172.21.15.71:0/274950682 learned_addr learned my addr 172.21.15.71:0/274950682 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.892+0000 7f80894fa700 1 --2- 172.21.15.71:0/274950682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80840ffc20 0x7f808419b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.892+0000 7f8088cf9700 1 -- 172.21.15.71:0/274950682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80840ffc20 msgr2=0x7f808419b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.892+0000 7f8088cf9700 1 --2- 172.21.15.71:0/274950682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80840ffc20 0x7f808419b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.892+0000 7f8088cf9700 1 -- 172.21.15.71:0/274950682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80740095f0 con 0x7f8084105bd0 2024-04-23T02:10:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.892+0000 7f8088cf9700 1 --2- 172.21.15.71:0/274950682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8084105bd0 0x7f808419ba80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f808000d960 tx=0x7f808000dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.892+0000 7f807a7fc700 1 -- 172.21.15.71:0/274950682 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f808000ec20 con 0x7f8084105bd0 2024-04-23T02:10:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.892+0000 7f807a7fc700 1 -- 172.21.15.71:0/274950682 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f808000ed80 con 0x7f8084105bd0 2024-04-23T02:10:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.892+0000 7f808b75e700 1 -- 172.21.15.71:0/274950682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8084195840 con 0x7f8084105bd0 2024-04-23T02:10:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.892+0000 7f807a7fc700 1 -- 172.21.15.71:0/274950682 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8080014700 con 0x7f8084105bd0 2024-04-23T02:10:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.892+0000 7f808b75e700 1 -- 172.21.15.71:0/274950682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8084195d90 con 0x7f8084105bd0 2024-04-23T02:10:08.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.893+0000 7f808b75e700 1 -- 172.21.15.71:0/274950682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808404f2e0 con 0x7f8084105bd0 2024-04-23T02:10:08.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.894+0000 7f807a7fc700 1 -- 172.21.15.71:0/274950682 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f8080016440 con 0x7f8084105bd0 2024-04-23T02:10:08.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.894+0000 7f807a7fc700 1 --2- 172.21.15.71:0/274950682 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f807006c4e0 0x7f807006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:08.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.894+0000 7f807a7fc700 1 -- 172.21.15.71:0/274950682 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f808008ceb0 con 0x7f8084105bd0 2024-04-23T02:10:08.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.894+0000 7f80894fa700 1 --2- 172.21.15.71:0/274950682 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f807006c4e0 0x7f807006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:08.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.894+0000 7f80894fa700 1 --2- 172.21.15.71:0/274950682 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f807006c4e0 0x7f807006e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8074006010 tx=0x7f807401a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:08.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:08.899+0000 7f807a7fc700 1 -- 172.21.15.71:0/274950682 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f808005b510 con 0x7f8084105bd0 2024-04-23T02:10:09.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:09.091+0000 7f808b75e700 1 -- 172.21.15.71:0/274950682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f80841960f0 con 0x7f8084105bd0 2024-04-23T02:10:09.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:09.092+0000 7f807a7fc700 1 -- 172.21.15.71:0/274950682 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f808005b0a0 con 0x7f8084105bd0 2024-04-23T02:10:09.096 INFO:teuthology.orchestra.run.smithi071.stdout:111669149717 2024-04-23T02:10:09.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:09.094+0000 7f808b75e700 1 -- 172.21.15.71:0/274950682 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f807006c4e0 msgr2=0x7f807006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:09.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:09.094+0000 7f808b75e700 1 --2- 172.21.15.71:0/274950682 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f807006c4e0 0x7f807006e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8074006010 tx=0x7f807401a040 comp rx=0 tx=0).stop 2024-04-23T02:10:09.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:09.094+0000 7f808b75e700 1 -- 172.21.15.71:0/274950682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8084105bd0 msgr2=0x7f808419ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:09.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:09.094+0000 7f808b75e700 1 --2- 172.21.15.71:0/274950682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8084105bd0 0x7f808419ba80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f808000d960 tx=0x7f808000dd20 comp rx=0 tx=0).stop 2024-04-23T02:10:09.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:09.094+0000 7f808b75e700 1 -- 172.21.15.71:0/274950682 shutdown_connections 2024-04-23T02:10:09.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:09.094+0000 7f808b75e700 1 --2- 172.21.15.71:0/274950682 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f807006c4e0 0x7f807006e9a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:09.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:09.094+0000 7f808b75e700 1 --2- 172.21.15.71:0/274950682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80840ffc20 0x7f808419b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:09.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:09.094+0000 7f808b75e700 1 --2- 172.21.15.71:0/274950682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8084105bd0 0x7f808419ba80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:09.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:09.094+0000 7f808b75e700 1 -- 172.21.15.71:0/274950682 >> 172.21.15.71:0/274950682 conn(0x7f8084075670 msgr2=0x7f80840fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:09.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:09.094+0000 7f808b75e700 1 -- 172.21.15.71:0/274950682 shutdown_connections 2024-04-23T02:10:09.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:09.094+0000 7f808b75e700 1 -- 172.21.15.71:0/274950682 wait complete. 2024-04-23T02:10:09.347 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:09 smithi071 ceph-mon[39837]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:09.347 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:09 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/274950682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T02:10:09.599 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:10:09.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:09 smithi192 ceph-mon[49662]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:09.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:09 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/274950682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T02:10:11.023 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149716 got 111669149717 for osd.3 2024-04-23T02:10:11.023 DEBUG:teuthology.parallel:result is None 2024-04-23T02:10:11.423 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:11 smithi071 ceph-mon[39837]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:11.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:11 smithi192 ceph-mon[49662]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:12.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:12 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:10:12.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:12 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:10:13.435 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:13 smithi071 ceph-mon[39837]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:13.435 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:13 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3524370465' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T02:10:13.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:13 smithi192 ceph-mon[49662]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:13.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:13 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/3524370465' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T02:10:14.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.083+0000 7fc64073e700 1 -- 172.21.15.71:0/1036491150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc638100ea0 msgr2=0x7fc638101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:14.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.083+0000 7fc64073e700 1 --2- 172.21.15.71:0/1036491150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc638100ea0 0x7fc638101280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc634009910 tx=0x7fc634009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:14.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.083+0000 7fc64073e700 1 -- 172.21.15.71:0/1036491150 shutdown_connections 2024-04-23T02:10:14.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.083+0000 7fc64073e700 1 --2- 172.21.15.71:0/1036491150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc638101850 0x7fc6381058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:14.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.083+0000 7fc64073e700 1 --2- 172.21.15.71:0/1036491150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc638100ea0 0x7fc638101280 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:14.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.083+0000 7fc64073e700 1 -- 172.21.15.71:0/1036491150 >> 172.21.15.71:0/1036491150 conn(0x7fc6380fc750 msgr2=0x7fc6380feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:14.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.083+0000 7fc64073e700 1 -- 172.21.15.71:0/1036491150 shutdown_connections 2024-04-23T02:10:14.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.083+0000 7fc64073e700 1 -- 172.21.15.71:0/1036491150 wait complete. 2024-04-23T02:10:14.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.084+0000 7fc64073e700 1 Processor -- start 2024-04-23T02:10:14.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.084+0000 7fc64073e700 1 -- start start 2024-04-23T02:10:14.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.084+0000 7fc64073e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc638100ea0 0x7fc63819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:14.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.084+0000 7fc64073e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc638101850 0x7fc63819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:14.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.084+0000 7fc64073e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63819d230 con 0x7fc638101850 2024-04-23T02:10:14.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.084+0000 7fc64073e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6381966e0 con 0x7fc638100ea0 2024-04-23T02:10:14.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.084+0000 7fc63dcd9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc638101850 0x7fc63819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:14.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.084+0000 7fc63dcd9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc638101850 0x7fc63819cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44954/0 (socket says 172.21.15.71:44954) 2024-04-23T02:10:14.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.084+0000 7fc63dcd9700 1 -- 172.21.15.71:0/1652651892 learned_addr learned my addr 172.21.15.71:0/1652651892 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:14.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.084+0000 7fc63e4da700 1 --2- 172.21.15.71:0/1652651892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc638100ea0 0x7fc63819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:14.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.085+0000 7fc63dcd9700 1 -- 172.21.15.71:0/1652651892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc638100ea0 msgr2=0x7fc63819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:14.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.085+0000 7fc63dcd9700 1 --2- 172.21.15.71:0/1652651892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc638100ea0 0x7fc63819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:14.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.085+0000 7fc63dcd9700 1 -- 172.21.15.71:0/1652651892 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6340095f0 con 0x7fc638101850 2024-04-23T02:10:14.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.085+0000 7fc63e4da700 1 --2- 172.21.15.71:0/1652651892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc638100ea0 0x7fc63819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:10:14.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.085+0000 7fc63dcd9700 1 --2- 172.21.15.71:0/1652651892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc638101850 0x7fc63819cba0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc62800b480 tx=0x7fc62800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:14.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.085+0000 7fc62f7fe700 1 -- 172.21.15.71:0/1652651892 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6280118c0 con 0x7fc638101850 2024-04-23T02:10:14.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.085+0000 7fc62f7fe700 1 -- 172.21.15.71:0/1652651892 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc628010380 con 0x7fc638101850 2024-04-23T02:10:14.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.085+0000 7fc64073e700 1 -- 172.21.15.71:0/1652651892 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6381969c0 con 0x7fc638101850 2024-04-23T02:10:14.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.085+0000 7fc62f7fe700 1 -- 172.21.15.71:0/1652651892 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc62800f510 con 0x7fc638101850 2024-04-23T02:10:14.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.085+0000 7fc64073e700 1 -- 172.21.15.71:0/1652651892 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc638196f10 con 0x7fc638101850 2024-04-23T02:10:14.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.086+0000 7fc64073e700 1 -- 172.21.15.71:0/1652651892 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc638109240 con 0x7fc638101850 2024-04-23T02:10:14.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.086+0000 7fc62f7fe700 1 -- 172.21.15.71:0/1652651892 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fc6280104f0 con 0x7fc638101850 2024-04-23T02:10:14.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.087+0000 7fc62f7fe700 1 --2- 172.21.15.71:0/1652651892 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc62406c4e0 0x7fc62406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:14.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.087+0000 7fc63e4da700 1 --2- 172.21.15.71:0/1652651892 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc62406c4e0 0x7fc62406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:14.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.087+0000 7fc62f7fe700 1 -- 172.21.15.71:0/1652651892 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc62808b130 con 0x7fc638101850 2024-04-23T02:10:14.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.087+0000 7fc63e4da700 1 --2- 172.21.15.71:0/1652651892 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc62406c4e0 0x7fc62406e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc63400b5c0 tx=0x7fc634005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:14.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.091+0000 7fc62f7fe700 1 -- 172.21.15.71:0/1652651892 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc6280596e0 con 0x7fc638101850 2024-04-23T02:10:14.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.286+0000 7fc64073e700 1 -- 172.21.15.71:0/1652651892 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc63804f2e0 con 0x7fc638101850 2024-04-23T02:10:14.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.287+0000 7fc62f7fe700 1 -- 172.21.15.71:0/1652651892 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc6280107a0 con 0x7fc638101850 2024-04-23T02:10:14.291 INFO:teuthology.orchestra.run.smithi071.stdout:38654705704 2024-04-23T02:10:14.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.289+0000 7fc64073e700 1 -- 172.21.15.71:0/1652651892 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc62406c4e0 msgr2=0x7fc62406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:14.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.289+0000 7fc64073e700 1 --2- 172.21.15.71:0/1652651892 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc62406c4e0 0x7fc62406e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc63400b5c0 tx=0x7fc634005960 comp rx=0 tx=0).stop 2024-04-23T02:10:14.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.289+0000 7fc64073e700 1 -- 172.21.15.71:0/1652651892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc638101850 msgr2=0x7fc63819cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:14.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.289+0000 7fc64073e700 1 --2- 172.21.15.71:0/1652651892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc638101850 0x7fc63819cba0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc62800b480 tx=0x7fc62800b840 comp rx=0 tx=0).stop 2024-04-23T02:10:14.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.289+0000 7fc64073e700 1 -- 172.21.15.71:0/1652651892 shutdown_connections 2024-04-23T02:10:14.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.289+0000 7fc64073e700 1 --2- 172.21.15.71:0/1652651892 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc62406c4e0 0x7fc62406e9a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:14.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.289+0000 7fc64073e700 1 --2- 172.21.15.71:0/1652651892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc638100ea0 0x7fc63819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:14.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.289+0000 7fc64073e700 1 --2- 172.21.15.71:0/1652651892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc638101850 0x7fc63819cba0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:14.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.289+0000 7fc64073e700 1 -- 172.21.15.71:0/1652651892 >> 172.21.15.71:0/1652651892 conn(0x7fc6380fc750 msgr2=0x7fc6380fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:14.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.289+0000 7fc64073e700 1 -- 172.21.15.71:0/1652651892 shutdown_connections 2024-04-23T02:10:14.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.289+0000 7fc64073e700 1 -- 172.21.15.71:0/1652651892 wait complete. 2024-04-23T02:10:14.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.323+0000 7f74398a6700 1 -- 172.21.15.71:0/3077934697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7434100ea0 msgr2=0x7f7434101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:14.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.323+0000 7f74398a6700 1 --2- 172.21.15.71:0/3077934697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7434100ea0 0x7f7434101280 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7428009910 tx=0x7f7428009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:14.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.323+0000 7f74398a6700 1 -- 172.21.15.71:0/3077934697 shutdown_connections 2024-04-23T02:10:14.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.323+0000 7f74398a6700 1 --2- 172.21.15.71:0/3077934697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7434101850 0x7f74341058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:14.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.323+0000 7f74398a6700 1 --2- 172.21.15.71:0/3077934697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7434100ea0 0x7f7434101280 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:14.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.323+0000 7f74398a6700 1 -- 172.21.15.71:0/3077934697 >> 172.21.15.71:0/3077934697 conn(0x7f74340fc750 msgr2=0x7f74340feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:14.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.323+0000 7f74398a6700 1 -- 172.21.15.71:0/3077934697 shutdown_connections 2024-04-23T02:10:14.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.323+0000 7f74398a6700 1 -- 172.21.15.71:0/3077934697 wait complete. 2024-04-23T02:10:14.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.324+0000 7f74398a6700 1 Processor -- start 2024-04-23T02:10:14.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.324+0000 7f74398a6700 1 -- start start 2024-04-23T02:10:14.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.324+0000 7f74398a6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7434100ea0 0x7f743419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:14.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.324+0000 7f74398a6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7434101850 0x7f743419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:14.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.324+0000 7f74398a6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743419d130 con 0x7f7434100ea0 2024-04-23T02:10:14.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.324+0000 7f74398a6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74341965e0 con 0x7f7434101850 2024-04-23T02:10:14.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.324+0000 7f7432ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7434100ea0 0x7f743419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:14.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.324+0000 7f7432ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7434100ea0 0x7f743419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44972/0 (socket says 172.21.15.71:44972) 2024-04-23T02:10:14.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.324+0000 7f7432ffd700 1 -- 172.21.15.71:0/236864070 learned_addr learned my addr 172.21.15.71:0/236864070 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:14.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.324+0000 7f74327fc700 1 --2- 172.21.15.71:0/236864070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7434101850 0x7f743419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:14.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.324+0000 7f7432ffd700 1 -- 172.21.15.71:0/236864070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7434101850 msgr2=0x7f743419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:14.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.324+0000 7f7432ffd700 1 --2- 172.21.15.71:0/236864070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7434101850 0x7f743419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:14.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.324+0000 7f7432ffd700 1 -- 172.21.15.71:0/236864070 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74280095f0 con 0x7f7434100ea0 2024-04-23T02:10:14.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.324+0000 7f7432ffd700 1 --2- 172.21.15.71:0/236864070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7434100ea0 0x7f743419c560 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7428000c00 tx=0x7f742800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:14.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.325+0000 7f74388a4700 1 -- 172.21.15.71:0/236864070 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f742801d070 con 0x7f7434100ea0 2024-04-23T02:10:14.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.325+0000 7f74388a4700 1 -- 172.21.15.71:0/236864070 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f74280044b0 con 0x7f7434100ea0 2024-04-23T02:10:14.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.325+0000 7f74398a6700 1 -- 172.21.15.71:0/236864070 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7434196860 con 0x7f7434100ea0 2024-04-23T02:10:14.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.325+0000 7f74388a4700 1 -- 172.21.15.71:0/236864070 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7428003a70 con 0x7f7434100ea0 2024-04-23T02:10:14.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.325+0000 7f74398a6700 1 -- 172.21.15.71:0/236864070 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7434196cf0 con 0x7f7434100ea0 2024-04-23T02:10:14.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.326+0000 7f74398a6700 1 -- 172.21.15.71:0/236864070 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7434109240 con 0x7f7434100ea0 2024-04-23T02:10:14.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.326+0000 7f74388a4700 1 -- 172.21.15.71:0/236864070 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f74280048e0 con 0x7f7434100ea0 2024-04-23T02:10:14.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.326+0000 7f74388a4700 1 --2- 172.21.15.71:0/236864070 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f741c06c4e0 0x7f741c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:14.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.326+0000 7f74388a4700 1 -- 172.21.15.71:0/236864070 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f742808cb80 con 0x7f7434100ea0 2024-04-23T02:10:14.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.326+0000 7f74327fc700 1 --2- 172.21.15.71:0/236864070 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f741c06c4e0 0x7f741c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:14.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.327+0000 7f74327fc700 1 --2- 172.21.15.71:0/236864070 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f741c06c4e0 0x7f741c06e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7434105b20 tx=0x7f7424009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:14.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.331+0000 7f74388a4700 1 -- 172.21.15.71:0/236864070 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f742805b130 con 0x7f7434100ea0 2024-04-23T02:10:14.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.534+0000 7f74398a6700 1 -- 172.21.15.71:0/236864070 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f743404f2e0 con 0x7f7434100ea0 2024-04-23T02:10:14.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.534+0000 7f74388a4700 1 -- 172.21.15.71:0/236864070 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f742805acc0 con 0x7f7434100ea0 2024-04-23T02:10:14.538 INFO:teuthology.orchestra.run.smithi071.stdout:146028888080 2024-04-23T02:10:14.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.536+0000 7f74398a6700 1 -- 172.21.15.71:0/236864070 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f741c06c4e0 msgr2=0x7f741c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:14.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.536+0000 7f74398a6700 1 --2- 172.21.15.71:0/236864070 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f741c06c4e0 0x7f741c06e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7434105b20 tx=0x7f7424009260 comp rx=0 tx=0).stop 2024-04-23T02:10:14.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.536+0000 7f74398a6700 1 -- 172.21.15.71:0/236864070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7434100ea0 msgr2=0x7f743419c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:14.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.536+0000 7f74398a6700 1 --2- 172.21.15.71:0/236864070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7434100ea0 0x7f743419c560 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7428000c00 tx=0x7f742800be90 comp rx=0 tx=0).stop 2024-04-23T02:10:14.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.536+0000 7f74398a6700 1 -- 172.21.15.71:0/236864070 shutdown_connections 2024-04-23T02:10:14.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.537+0000 7f74398a6700 1 --2- 172.21.15.71:0/236864070 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f741c06c4e0 0x7f741c06e9a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:14.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.537+0000 7f74398a6700 1 --2- 172.21.15.71:0/236864070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7434100ea0 0x7f743419c560 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:14.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.537+0000 7f74398a6700 1 --2- 172.21.15.71:0/236864070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7434101850 0x7f743419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:14.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.537+0000 7f74398a6700 1 -- 172.21.15.71:0/236864070 >> 172.21.15.71:0/236864070 conn(0x7f74340fc750 msgr2=0x7f74340fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:14.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.537+0000 7f74398a6700 1 -- 172.21.15.71:0/236864070 shutdown_connections 2024-04-23T02:10:14.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:14.537+0000 7f74398a6700 1 -- 172.21.15.71:0/236864070 wait complete. 2024-04-23T02:10:15.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.771+0000 7fe5cded4700 1 -- 172.21.15.71:0/3211854792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c8105bd0 msgr2=0x7fe5c8105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:15.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.771+0000 7fe5cded4700 1 --2- 172.21.15.71:0/3211854792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c8105bd0 0x7fe5c8105fb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009910 tx=0x7fe5bc009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:15.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.771+0000 7fe5cded4700 1 -- 172.21.15.71:0/3211854792 shutdown_connections 2024-04-23T02:10:15.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.771+0000 7fe5cded4700 1 --2- 172.21.15.71:0/3211854792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5c80ffc20 0x7fe5c8100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:15.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.771+0000 7fe5cded4700 1 --2- 172.21.15.71:0/3211854792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c8105bd0 0x7fe5c8105fb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:15.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.771+0000 7fe5cded4700 1 -- 172.21.15.71:0/3211854792 >> 172.21.15.71:0/3211854792 conn(0x7fe5c8075670 msgr2=0x7fe5c8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:15.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.771+0000 7fe5cded4700 1 -- 172.21.15.71:0/3211854792 shutdown_connections 2024-04-23T02:10:15.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.771+0000 7fe5cded4700 1 -- 172.21.15.71:0/3211854792 wait complete. 2024-04-23T02:10:15.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.772+0000 7fe5cded4700 1 Processor -- start 2024-04-23T02:10:15.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.772+0000 7fe5cded4700 1 -- start start 2024-04-23T02:10:15.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.772+0000 7fe5cded4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5c80ffc20 0x7fe5c81975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:15.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.772+0000 7fe5cded4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c8105bd0 0x7fe5c8197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:15.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.772+0000 7fe5cded4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c8198140 con 0x7fe5c8105bd0 2024-04-23T02:10:15.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.772+0000 7fe5cded4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c81982b0 con 0x7fe5c80ffc20 2024-04-23T02:10:15.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.772+0000 7fe5c6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c8105bd0 0x7fe5c8197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:15.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.772+0000 7fe5c6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c8105bd0 0x7fe5c8197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44948/0 (socket says 172.21.15.71:44948) 2024-04-23T02:10:15.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.772+0000 7fe5c6ffd700 1 -- 172.21.15.71:0/3524370465 learned_addr learned my addr 172.21.15.71:0/3524370465 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:15.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.772+0000 7fe5c6ffd700 1 -- 172.21.15.71:0/3524370465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5c80ffc20 msgr2=0x7fe5c81975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:15.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.772+0000 7fe5c6ffd700 1 --2- 172.21.15.71:0/3524370465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5c80ffc20 0x7fe5c81975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:15.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.772+0000 7fe5c6ffd700 1 -- 172.21.15.71:0/3524370465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc0095f0 con 0x7fe5c8105bd0 2024-04-23T02:10:15.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.773+0000 7fe5c6ffd700 1 --2- 172.21.15.71:0/3524370465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c8105bd0 0x7fe5c8197b20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800c8a0 tx=0x7fe5b800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:15.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.773+0000 7fe5c4ff9700 1 -- 172.21.15.71:0/3524370465 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5b800cea0 con 0x7fe5c8105bd0 2024-04-23T02:10:15.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.773+0000 7fe5c4ff9700 1 -- 172.21.15.71:0/3524370465 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fe5b8004830 con 0x7fe5c8105bd0 2024-04-23T02:10:15.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.773+0000 7fe5cded4700 1 -- 172.21.15.71:0/3524370465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c81a0650 con 0x7fe5c8105bd0 2024-04-23T02:10:15.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.773+0000 7fe5c4ff9700 1 -- 172.21.15.71:0/3524370465 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5b8005540 con 0x7fe5c8105bd0 2024-04-23T02:10:15.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.773+0000 7fe5cded4700 1 -- 172.21.15.71:0/3524370465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c81a0b40 con 0x7fe5c8105bd0 2024-04-23T02:10:15.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.774+0000 7fe5cded4700 1 -- 172.21.15.71:0/3524370465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c81080d0 con 0x7fe5c8105bd0 2024-04-23T02:10:15.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.774+0000 7fe5c4ff9700 1 -- 172.21.15.71:0/3524370465 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fe5b80049a0 con 0x7fe5c8105bd0 2024-04-23T02:10:15.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.775+0000 7fe5c4ff9700 1 --2- 172.21.15.71:0/3524370465 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fe5b006c4e0 0x7fe5b006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:15.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.775+0000 7fe5c4ff9700 1 -- 172.21.15.71:0/3524370465 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe5b808b0e0 con 0x7fe5c8105bd0 2024-04-23T02:10:15.012 INFO:teuthology.orchestra.run.smithi071.stdout:81604378652 2024-04-23T02:10:15.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.775+0000 7fe5c77fe700 1 --2- 172.21.15.71:0/3524370465 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fe5b006c4e0 0x7fe5b006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:15.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.775+0000 7fe5c77fe700 1 --2- 172.21.15.71:0/3524370465 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fe5b006c4e0 0x7fe5b006e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00b5c0 tx=0x7fe5bc005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:15.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.779+0000 7fe5c4ff9700 1 -- 172.21.15.71:0/3524370465 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe5b8055c80 con 0x7fe5c8105bd0 2024-04-23T02:10:15.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.976+0000 7fe5cded4700 1 -- 172.21.15.71:0/3524370465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe5c804f2e0 con 0x7fe5c8105bd0 2024-04-23T02:10:15.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.976+0000 7fe5c4ff9700 1 -- 172.21.15.71:0/3524370465 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe5b8089e60 con 0x7fe5c8105bd0 2024-04-23T02:10:15.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.978+0000 7fe5cded4700 1 -- 172.21.15.71:0/3524370465 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fe5b006c4e0 msgr2=0x7fe5b006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:15.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.978+0000 7fe5cded4700 1 --2- 172.21.15.71:0/3524370465 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fe5b006c4e0 0x7fe5b006e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00b5c0 tx=0x7fe5bc005960 comp rx=0 tx=0).stop 2024-04-23T02:10:15.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.978+0000 7fe5cded4700 1 -- 172.21.15.71:0/3524370465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c8105bd0 msgr2=0x7fe5c8197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:15.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.978+0000 7fe5cded4700 1 --2- 172.21.15.71:0/3524370465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c8105bd0 0x7fe5c8197b20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800c8a0 tx=0x7fe5b800cc60 comp rx=0 tx=0).stop 2024-04-23T02:10:15.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.978+0000 7fe5cded4700 1 -- 172.21.15.71:0/3524370465 shutdown_connections 2024-04-23T02:10:15.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.978+0000 7fe5cded4700 1 --2- 172.21.15.71:0/3524370465 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fe5b006c4e0 0x7fe5b006e9a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:15.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.978+0000 7fe5cded4700 1 --2- 172.21.15.71:0/3524370465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5c80ffc20 0x7fe5c81975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:15.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.978+0000 7fe5cded4700 1 --2- 172.21.15.71:0/3524370465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c8105bd0 0x7fe5c8197b20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:15.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.978+0000 7fe5cded4700 1 -- 172.21.15.71:0/3524370465 >> 172.21.15.71:0/3524370465 conn(0x7fe5c8075670 msgr2=0x7fe5c80fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:15.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.978+0000 7fe5cded4700 1 -- 172.21.15.71:0/3524370465 shutdown_connections 2024-04-23T02:10:15.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:12.978+0000 7fe5cded4700 1 -- 172.21.15.71:0/3524370465 wait complete. 2024-04-23T02:10:15.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:15 smithi192 ceph-mon[49662]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:15.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:15 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1652651892' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T02:10:15.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:15 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/236864070' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T02:10:15.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:15 smithi071 ceph-mon[39837]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:15.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:15 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1652651892' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T02:10:15.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:15 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/236864070' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T02:10:15.666 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378650 got 81604378652 for osd.2 2024-04-23T02:10:15.667 DEBUG:teuthology.parallel:result is None 2024-04-23T02:10:16.131 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888077 got 146028888080 for osd.4 2024-04-23T02:10:16.131 DEBUG:teuthology.parallel:result is None 2024-04-23T02:10:16.972 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:16 smithi071 ceph-mon[39837]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:16.974 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:10:16.998 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705702 got 38654705704 for osd.0 2024-04-23T02:10:16.998 DEBUG:teuthology.parallel:result is None 2024-04-23T02:10:17.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:16 smithi192 ceph-mon[49662]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:18.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:18 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/4128341356' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T02:10:18.421 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:18 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/4128341356' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T02:10:18.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.781+0000 7fceed0c1700 1 -- 172.21.15.71:0/517321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee80ffda0 msgr2=0x7fcee8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:18.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.781+0000 7fceed0c1700 1 --2- 172.21.15.71:0/517321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee80ffda0 0x7fcee8100180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcedc009910 tx=0x7fcedc009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:18.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.781+0000 7fceed0c1700 1 -- 172.21.15.71:0/517321 shutdown_connections 2024-04-23T02:10:18.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.781+0000 7fceed0c1700 1 --2- 172.21.15.71:0/517321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcee8100750 0x7fcee81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:18.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.781+0000 7fceed0c1700 1 --2- 172.21.15.71:0/517321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee80ffda0 0x7fcee8100180 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:18.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.781+0000 7fceed0c1700 1 -- 172.21.15.71:0/517321 >> 172.21.15.71:0/517321 conn(0x7fcee80fb650 msgr2=0x7fcee80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:18.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.781+0000 7fceed0c1700 1 -- 172.21.15.71:0/517321 shutdown_connections 2024-04-23T02:10:18.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.781+0000 7fceed0c1700 1 -- 172.21.15.71:0/517321 wait complete. 2024-04-23T02:10:18.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.782+0000 7fceed0c1700 1 Processor -- start 2024-04-23T02:10:18.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.782+0000 7fceed0c1700 1 -- start start 2024-04-23T02:10:18.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.782+0000 7fceed0c1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcee80ffda0 0x7fcee8197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:18.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.782+0000 7fceed0c1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee8100750 0x7fcee8197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:18.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.782+0000 7fceed0c1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee81981a0 con 0x7fcee8100750 2024-04-23T02:10:18.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.782+0000 7fceed0c1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee81982e0 con 0x7fcee80ffda0 2024-04-23T02:10:18.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.782+0000 7fcee659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee8100750 0x7fcee8197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:18.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.782+0000 7fcee659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee8100750 0x7fcee8197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44688/0 (socket says 172.21.15.71:44688) 2024-04-23T02:10:18.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.782+0000 7fcee659c700 1 -- 172.21.15.71:0/4128341356 learned_addr learned my addr 172.21.15.71:0/4128341356 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:18.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.783+0000 7fcee6d9d700 1 --2- 172.21.15.71:0/4128341356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcee80ffda0 0x7fcee8197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:18.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.783+0000 7fcee659c700 1 -- 172.21.15.71:0/4128341356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcee80ffda0 msgr2=0x7fcee8197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:18.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.783+0000 7fcee659c700 1 --2- 172.21.15.71:0/4128341356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcee80ffda0 0x7fcee8197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:18.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.783+0000 7fcee659c700 1 -- 172.21.15.71:0/4128341356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc0095f0 con 0x7fcee8100750 2024-04-23T02:10:18.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.783+0000 7fcee659c700 1 --2- 172.21.15.71:0/4128341356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee8100750 0x7fcee8197b80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fced800c8f0 tx=0x7fced800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:18.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.783+0000 7fcecffff700 1 -- 172.21.15.71:0/4128341356 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fced8007a50 con 0x7fcee8100750 2024-04-23T02:10:18.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.783+0000 7fcecffff700 1 -- 172.21.15.71:0/4128341356 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fced8007bb0 con 0x7fcee8100750 2024-04-23T02:10:18.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.783+0000 7fceed0c1700 1 -- 172.21.15.71:0/4128341356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee81a06b0 con 0x7fcee8100750 2024-04-23T02:10:18.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.783+0000 7fcecffff700 1 -- 172.21.15.71:0/4128341356 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fced800fea0 con 0x7fcee8100750 2024-04-23T02:10:18.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.783+0000 7fceed0c1700 1 -- 172.21.15.71:0/4128341356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee81a0b70 con 0x7fcee8100750 2024-04-23T02:10:18.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.784+0000 7fceed0c1700 1 -- 172.21.15.71:0/4128341356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee8108140 con 0x7fcee8100750 2024-04-23T02:10:18.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.784+0000 7fcecffff700 1 -- 172.21.15.71:0/4128341356 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fced8007d20 con 0x7fcee8100750 2024-04-23T02:10:18.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.785+0000 7fcecffff700 1 --2- 172.21.15.71:0/4128341356 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fced006c530 0x7fced006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:18.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.785+0000 7fcecffff700 1 -- 172.21.15.71:0/4128341356 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fced808b520 con 0x7fcee8100750 2024-04-23T02:10:18.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.785+0000 7fcee6d9d700 1 --2- 172.21.15.71:0/4128341356 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fced006c530 0x7fced006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:18.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.785+0000 7fcee6d9d700 1 --2- 172.21.15.71:0/4128341356 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fced006c530 0x7fced006e9f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcedc006010 tx=0x7fcedc01a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:18.538 INFO:teuthology.orchestra.run.smithi071.stdout:60129542180 2024-04-23T02:10:18.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.789+0000 7fcecffff700 1 -- 172.21.15.71:0/4128341356 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fced8055fa0 con 0x7fcee8100750 2024-04-23T02:10:18.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.990+0000 7fceed0c1700 1 -- 172.21.15.71:0/4128341356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcee804f2e0 con 0x7fcee8100750 2024-04-23T02:10:18.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.990+0000 7fcecffff700 1 -- 172.21.15.71:0/4128341356 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fced80595c0 con 0x7fcee8100750 2024-04-23T02:10:18.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.992+0000 7fceed0c1700 1 -- 172.21.15.71:0/4128341356 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fced006c530 msgr2=0x7fced006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:18.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.992+0000 7fceed0c1700 1 --2- 172.21.15.71:0/4128341356 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fced006c530 0x7fced006e9f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcedc006010 tx=0x7fcedc01a040 comp rx=0 tx=0).stop 2024-04-23T02:10:18.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.992+0000 7fceed0c1700 1 -- 172.21.15.71:0/4128341356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee8100750 msgr2=0x7fcee8197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:18.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.992+0000 7fceed0c1700 1 --2- 172.21.15.71:0/4128341356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee8100750 0x7fcee8197b80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fced800c8f0 tx=0x7fced800ccb0 comp rx=0 tx=0).stop 2024-04-23T02:10:18.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.992+0000 7fceed0c1700 1 -- 172.21.15.71:0/4128341356 shutdown_connections 2024-04-23T02:10:18.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.992+0000 7fceed0c1700 1 --2- 172.21.15.71:0/4128341356 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fced006c530 0x7fced006e9f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:18.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.992+0000 7fceed0c1700 1 --2- 172.21.15.71:0/4128341356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcee80ffda0 0x7fcee8197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:18.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.992+0000 7fceed0c1700 1 --2- 172.21.15.71:0/4128341356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee8100750 0x7fcee8197b80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:18.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.992+0000 7fceed0c1700 1 -- 172.21.15.71:0/4128341356 >> 172.21.15.71:0/4128341356 conn(0x7fcee80fb650 msgr2=0x7fcee80fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:18.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.993+0000 7fceed0c1700 1 -- 172.21.15.71:0/4128341356 shutdown_connections 2024-04-23T02:10:18.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:17.993+0000 7fceed0c1700 1 -- 172.21.15.71:0/4128341356 wait complete. 2024-04-23T02:10:19.085 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542176 got 60129542180 for osd.1 2024-04-23T02:10:19.085 DEBUG:teuthology.parallel:result is None 2024-04-23T02:10:19.312 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:19 smithi192 ceph-mon[49662]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:19.373 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:19 smithi071 ceph-mon[39837]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:19.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.622+0000 7f4c59b6a700 1 -- 172.21.15.71:0/2818910848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c54102760 msgr2=0x7f4c5410ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:19.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.622+0000 7f4c59b6a700 1 --2- 172.21.15.71:0/2818910848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c54102760 0x7f4c5410ac50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009940 tx=0x7f4c3c009c50 comp rx=0 tx=0).stop 2024-04-23T02:10:19.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.622+0000 7f4c59b6a700 1 -- 172.21.15.71:0/2818910848 shutdown_connections 2024-04-23T02:10:19.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.622+0000 7f4c59b6a700 1 --2- 172.21.15.71:0/2818910848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c54102760 0x7f4c5410ac50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:19.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.622+0000 7f4c59b6a700 1 --2- 172.21.15.71:0/2818910848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c54101e40 0x7f4c54102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:19.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.622+0000 7f4c59b6a700 1 -- 172.21.15.71:0/2818910848 >> 172.21.15.71:0/2818910848 conn(0x7f4c540fb610 msgr2=0x7f4c540fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:19.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.622+0000 7f4c59b6a700 1 -- 172.21.15.71:0/2818910848 shutdown_connections 2024-04-23T02:10:19.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.622+0000 7f4c59b6a700 1 -- 172.21.15.71:0/2818910848 wait complete. 2024-04-23T02:10:19.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.623+0000 7f4c59b6a700 1 Processor -- start 2024-04-23T02:10:19.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.623+0000 7f4c59b6a700 1 -- start start 2024-04-23T02:10:19.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.623+0000 7f4c59b6a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c54101e40 0x7f4c5419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:19.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.623+0000 7f4c59b6a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c54102760 0x7f4c5419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:19.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.623+0000 7f4c59b6a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5419d290 con 0x7f4c54101e40 2024-04-23T02:10:19.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.623+0000 7f4c59b6a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c54196740 con 0x7f4c54102760 2024-04-23T02:10:19.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.623+0000 7f4c537fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c54101e40 0x7f4c5419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:19.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.623+0000 7f4c537fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c54101e40 0x7f4c5419c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44706/0 (socket says 172.21.15.71:44706) 2024-04-23T02:10:19.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.623+0000 7f4c537fe700 1 -- 172.21.15.71:0/2690533044 learned_addr learned my addr 172.21.15.71:0/2690533044 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:19.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.623+0000 7f4c52ffd700 1 --2- 172.21.15.71:0/2690533044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c54102760 0x7f4c5419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:19.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.623+0000 7f4c537fe700 1 -- 172.21.15.71:0/2690533044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c54102760 msgr2=0x7f4c5419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:19.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.623+0000 7f4c537fe700 1 --2- 172.21.15.71:0/2690533044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c54102760 0x7f4c5419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:19.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.623+0000 7f4c537fe700 1 -- 172.21.15.71:0/2690533044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c0095f0 con 0x7f4c54101e40 2024-04-23T02:10:19.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.623+0000 7f4c537fe700 1 --2- 172.21.15.71:0/2690533044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c54101e40 0x7f4c5419c6c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400b4b0 tx=0x7f4c4400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:19.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.624+0000 7f4c50ff9700 1 -- 172.21.15.71:0/2690533044 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c44011800 con 0x7f4c54101e40 2024-04-23T02:10:19.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.624+0000 7f4c50ff9700 1 -- 172.21.15.71:0/2690533044 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4c44011df0 con 0x7f4c54101e40 2024-04-23T02:10:19.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.624+0000 7f4c59b6a700 1 -- 172.21.15.71:0/2690533044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c54196a20 con 0x7f4c54101e40 2024-04-23T02:10:19.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.624+0000 7f4c50ff9700 1 -- 172.21.15.71:0/2690533044 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c44010e50 con 0x7f4c54101e40 2024-04-23T02:10:19.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.624+0000 7f4c59b6a700 1 -- 172.21.15.71:0/2690533044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c54196f40 con 0x7f4c54101e40 2024-04-23T02:10:19.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.625+0000 7f4c59b6a700 1 -- 172.21.15.71:0/2690533044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c54108350 con 0x7f4c54101e40 2024-04-23T02:10:19.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.625+0000 7f4c50ff9700 1 -- 172.21.15.71:0/2690533044 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f4c44010380 con 0x7f4c54101e40 2024-04-23T02:10:19.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.625+0000 7f4c50ff9700 1 --2- 172.21.15.71:0/2690533044 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4c4006c4e0 0x7f4c4006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:19.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.626+0000 7f4c52ffd700 1 --2- 172.21.15.71:0/2690533044 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4c4006c4e0 0x7f4c4006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:19.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.626+0000 7f4c50ff9700 1 -- 172.21.15.71:0/2690533044 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4c4408b040 con 0x7f4c54101e40 2024-04-23T02:10:19.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.626+0000 7f4c52ffd700 1 --2- 172.21.15.71:0/2690533044 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4c4006c4e0 0x7f4c4006e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c006010 tx=0x7f4c3c005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:19.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.630+0000 7f4c50ff9700 1 -- 172.21.15.71:0/2690533044 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4c440595f0 con 0x7f4c54101e40 2024-04-23T02:10:19.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.818+0000 7f4c59b6a700 1 -- 172.21.15.71:0/2690533044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4c540679c0 con 0x7f4c54101e40 2024-04-23T02:10:19.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.818+0000 7f4c50ff9700 1 -- 172.21.15.71:0/2690533044 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4c44059180 con 0x7f4c54101e40 2024-04-23T02:10:19.822 INFO:teuthology.orchestra.run.smithi071.stdout:167503724556 2024-04-23T02:10:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.820+0000 7f4c59b6a700 1 -- 172.21.15.71:0/2690533044 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4c4006c4e0 msgr2=0x7f4c4006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.820+0000 7f4c59b6a700 1 --2- 172.21.15.71:0/2690533044 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4c4006c4e0 0x7f4c4006e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c006010 tx=0x7f4c3c005960 comp rx=0 tx=0).stop 2024-04-23T02:10:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.820+0000 7f4c59b6a700 1 -- 172.21.15.71:0/2690533044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c54101e40 msgr2=0x7f4c5419c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.820+0000 7f4c59b6a700 1 --2- 172.21.15.71:0/2690533044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c54101e40 0x7f4c5419c6c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400b4b0 tx=0x7f4c4400b7c0 comp rx=0 tx=0).stop 2024-04-23T02:10:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.820+0000 7f4c59b6a700 1 -- 172.21.15.71:0/2690533044 shutdown_connections 2024-04-23T02:10:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.820+0000 7f4c59b6a700 1 --2- 172.21.15.71:0/2690533044 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4c4006c4e0 0x7f4c4006e9a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.820+0000 7f4c59b6a700 1 --2- 172.21.15.71:0/2690533044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c54101e40 0x7f4c5419c6c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:19.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.820+0000 7f4c59b6a700 1 --2- 172.21.15.71:0/2690533044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c54102760 0x7f4c5419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:19.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.820+0000 7f4c59b6a700 1 -- 172.21.15.71:0/2690533044 >> 172.21.15.71:0/2690533044 conn(0x7f4c540fb610 msgr2=0x7f4c54105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:19.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.820+0000 7f4c59b6a700 1 -- 172.21.15.71:0/2690533044 shutdown_connections 2024-04-23T02:10:19.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:19.820+0000 7f4c59b6a700 1 -- 172.21.15.71:0/2690533044 wait complete. 2024-04-23T02:10:20.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:20 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/2690533044' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T02:10:20.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:20 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/2690533044' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T02:10:20.577 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724553 got 167503724556 for osd.5 2024-04-23T02:10:20.577 DEBUG:teuthology.parallel:result is None 2024-04-23T02:10:20.578 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-23T02:10:20.578 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-23T02:10:20.804 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:10:21.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:21 smithi192 ceph-mon[49662]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:21.404 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:21 smithi071 ceph-mon[39837]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:22.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.770+0000 7fca688a7700 1 -- 172.21.15.71:0/4289703626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca64105bf0 msgr2=0x7fca64105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:22.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.770+0000 7fca688a7700 1 --2- 172.21.15.71:0/4289703626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca64105bf0 0x7fca64105fd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009910 tx=0x7fca4c009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:22.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.770+0000 7fca688a7700 1 -- 172.21.15.71:0/4289703626 shutdown_connections 2024-04-23T02:10:22.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.770+0000 7fca688a7700 1 --2- 172.21.15.71:0/4289703626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca640ffb40 0x7fca640fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:22.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.770+0000 7fca688a7700 1 --2- 172.21.15.71:0/4289703626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca64105bf0 0x7fca64105fd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:22.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.770+0000 7fca688a7700 1 -- 172.21.15.71:0/4289703626 >> 172.21.15.71:0/4289703626 conn(0x7fca640fb610 msgr2=0x7fca640fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:22.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.770+0000 7fca688a7700 1 -- 172.21.15.71:0/4289703626 shutdown_connections 2024-04-23T02:10:22.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.770+0000 7fca688a7700 1 -- 172.21.15.71:0/4289703626 wait complete. 2024-04-23T02:10:22.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.771+0000 7fca688a7700 1 Processor -- start 2024-04-23T02:10:22.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.771+0000 7fca688a7700 1 -- start start 2024-04-23T02:10:22.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.771+0000 7fca688a7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca640ffb40 0x7fca641976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:22.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.771+0000 7fca688a7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca64105bf0 0x7fca64197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:22.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.771+0000 7fca688a7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca64198210 con 0x7fca640ffb40 2024-04-23T02:10:22.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.771+0000 7fca688a7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca64198350 con 0x7fca64105bf0 2024-04-23T02:10:22.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.771+0000 7fca6259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca640ffb40 0x7fca641976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:22.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.771+0000 7fca6259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca640ffb40 0x7fca641976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44722/0 (socket says 172.21.15.71:44722) 2024-04-23T02:10:22.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.771+0000 7fca6259c700 1 -- 172.21.15.71:0/1324373278 learned_addr learned my addr 172.21.15.71:0/1324373278 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:22.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.772+0000 7fca6259c700 1 -- 172.21.15.71:0/1324373278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca64105bf0 msgr2=0x7fca64197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:22.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.772+0000 7fca61d9b700 1 --2- 172.21.15.71:0/1324373278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca64105bf0 0x7fca64197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:22.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.772+0000 7fca6259c700 1 --2- 172.21.15.71:0/1324373278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca64105bf0 0x7fca64197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:22.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.772+0000 7fca6259c700 1 -- 172.21.15.71:0/1324373278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c0095f0 con 0x7fca640ffb40 2024-04-23T02:10:22.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.772+0000 7fca61d9b700 1 --2- 172.21.15.71:0/1324373278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca64105bf0 0x7fca64197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:10:22.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.772+0000 7fca6259c700 1 --2- 172.21.15.71:0/1324373278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca640ffb40 0x7fca641976b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fca4c006010 tx=0x7fca4c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:22.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.772+0000 7fca5b7fe700 1 -- 172.21.15.71:0/1324373278 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca4c01d070 con 0x7fca640ffb40 2024-04-23T02:10:22.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.772+0000 7fca5b7fe700 1 -- 172.21.15.71:0/1324373278 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fca4c00f460 con 0x7fca640ffb40 2024-04-23T02:10:22.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.772+0000 7fca688a7700 1 -- 172.21.15.71:0/1324373278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca641a0720 con 0x7fca640ffb40 2024-04-23T02:10:22.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.772+0000 7fca5b7fe700 1 -- 172.21.15.71:0/1324373278 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca4c021640 con 0x7fca640ffb40 2024-04-23T02:10:22.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.772+0000 7fca688a7700 1 -- 172.21.15.71:0/1324373278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca641a0b30 con 0x7fca640ffb40 2024-04-23T02:10:22.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.773+0000 7fca688a7700 1 -- 172.21.15.71:0/1324373278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca64108020 con 0x7fca640ffb40 2024-04-23T02:10:22.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.773+0000 7fca5b7fe700 1 -- 172.21.15.71:0/1324373278 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fca4c00fa60 con 0x7fca640ffb40 2024-04-23T02:10:22.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.774+0000 7fca5b7fe700 1 --2- 172.21.15.71:0/1324373278 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fca5006c4e0 0x7fca5006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:22.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.774+0000 7fca61d9b700 1 --2- 172.21.15.71:0/1324373278 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fca5006c4e0 0x7fca5006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:22.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.774+0000 7fca5b7fe700 1 -- 172.21.15.71:0/1324373278 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fca4c08c820 con 0x7fca640ffb40 2024-04-23T02:10:22.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.774+0000 7fca61d9b700 1 --2- 172.21.15.71:0/1324373278 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fca5006c4e0 0x7fca5006e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fca640696c0 tx=0x7fca54008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:22.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.778+0000 7fca5b7fe700 1 -- 172.21.15.71:0/1324373278 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fca4c057340 con 0x7fca640ffb40 2024-04-23T02:10:22.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.967+0000 7fca688a7700 1 -- 172.21.15.71:0/1324373278 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fca640611d0 con 0x7fca5006c4e0 2024-04-23T02:10:22.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.969+0000 7fca5b7fe700 1 -- 172.21.15.71:0/1324373278 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27974 (secure 0 0 0) 0x7fca640611d0 con 0x7fca5006c4e0 2024-04-23T02:10:22.974 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:10:22.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.971+0000 7fca688a7700 1 -- 172.21.15.71:0/1324373278 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fca5006c4e0 msgr2=0x7fca5006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:22.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.971+0000 7fca688a7700 1 --2- 172.21.15.71:0/1324373278 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fca5006c4e0 0x7fca5006e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fca640696c0 tx=0x7fca54008040 comp rx=0 tx=0).stop 2024-04-23T02:10:22.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.971+0000 7fca688a7700 1 -- 172.21.15.71:0/1324373278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca640ffb40 msgr2=0x7fca641976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:22.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.971+0000 7fca688a7700 1 --2- 172.21.15.71:0/1324373278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca640ffb40 0x7fca641976b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fca4c006010 tx=0x7fca4c00be20 comp rx=0 tx=0).stop 2024-04-23T02:10:22.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.971+0000 7fca688a7700 1 -- 172.21.15.71:0/1324373278 shutdown_connections 2024-04-23T02:10:22.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.971+0000 7fca688a7700 1 --2- 172.21.15.71:0/1324373278 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fca5006c4e0 0x7fca5006e9a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:22.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.971+0000 7fca688a7700 1 --2- 172.21.15.71:0/1324373278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca640ffb40 0x7fca641976b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:22.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.971+0000 7fca688a7700 1 --2- 172.21.15.71:0/1324373278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca64105bf0 0x7fca64197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:22.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.971+0000 7fca688a7700 1 -- 172.21.15.71:0/1324373278 >> 172.21.15.71:0/1324373278 conn(0x7fca640fb610 msgr2=0x7fca640fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:22.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.971+0000 7fca688a7700 1 -- 172.21.15.71:0/1324373278 shutdown_connections 2024-04-23T02:10:22.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:22.971+0000 7fca688a7700 1 -- 172.21.15.71:0/1324373278 wait complete. 2024-04-23T02:10:22.977 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-23T02:10:23.381 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:23 smithi071 ceph-mon[39837]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:23.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:23 smithi192 ceph-mon[49662]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:23.748 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-04-23T02:10:22.146750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165584,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305328,"statfs":{"total":575970213888,"available":575800655872,"internally_reserved":0,"allocated":3964928,"data_stored":2608835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002056"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":187,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-23T02:09:29.662915+0000","last_change":"2024-04-23T02:09:01.663614+0000","last_active":"2024-04-23T02:09:29.662915+0000","last_peered":"2024-04-23T02:09:29.662915+0000","last_clean":"2024-04-23T02:09:29.662915+0000","last_became_active":"2024-04-23T02:09:01.663310+0000","last_became_peered":"2024-04-23T02:09:01.663310+0000","last_unstale":"2024-04-23T02:09:29.662915+0000","last_undegraded":"2024-04-23T02:09:29.662915+0000","last_fullsized":"2024-04-23T02:09:29.662915+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T02:08:06.625838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T02:08:06.625838+0000","last_clean_scrub_stamp":"2024-04-23T02:08:06.625838+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T09:37:41.107099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 23 02:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.434},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.41699999999999998}]},{"osd":1,"last update":"Tue Apr 23 02:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.53900000000000003}]},{"osd":2,"last update":"Tue Apr 23 02:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.495}]},{"osd":3,"last update":"Tue Apr 23 02:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.53100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149720,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":729714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 23 02:09:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.53000000000000003}]},{"osd":1,"last update":"Tue Apr 23 02:09:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.48999999999999999}]},{"osd":2,"last update":"Tue Apr 23 02:09:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.503},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.51200000000000001}]},{"osd":4,"last update":"Tue Apr 23 02:10:02 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.434},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.38500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]}]},{"osd":2,"up_from":19,"seq":81604378654,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 23 02:10:11 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.33800000000000002,"15min":0.33200000000000002},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.192},{"interface":"front","average":{"1min":0.373,"5min":0.42599999999999999,"15min":0.435},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.71699999999999997,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.248}]},{"osd":1,"last update":"Tue Apr 23 02:10:11 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.35699999999999998,"15min":0.35299999999999998},"min":{"1min":0.21299999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.373,"15min":0.373},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.26600000000000001}]},{"osd":3,"last update":"Tue Apr 23 02:09:36 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.53800000000000003}]},{"osd":4,"last update":"Tue Apr 23 02:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]}]},{"osd":0,"up_from":9,"seq":38654705706,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Apr 23 02:09:40 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.309,"15min":0.30499999999999999},"min":{"1min":0.23699999999999999,"5min":0.182,"15min":0.182},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.32000000000000001},"min":{"1min":0.22700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.54600000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.433}]},{"osd":2,"last update":"Tue Apr 23 02:10:12 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.35599999999999998,"15min":0.35999999999999999},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.39600000000000002,"15min":0.39200000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.44500000000000001}]},{"osd":3,"last update":"Tue Apr 23 02:09:36 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.61799999999999999}]},{"osd":4,"last update":"Tue Apr 23 02:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 23 02:09:39 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.29399999999999998,"15min":0.28399999999999997},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.30199999999999999,"15min":0.29399999999999998},"min":{"1min":0.23699999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.44400000000000001}]},{"osd":2,"last update":"Tue Apr 23 02:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.34699999999999998,"15min":0.35099999999999998},"min":{"1min":0.24399999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.311,"5min":0.35799999999999998,"15min":0.36599999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"last":0.46400000000000002}]},{"osd":3,"last update":"Tue Apr 23 02:09:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.80500000000000005}]},{"osd":4,"last update":"Tue Apr 23 02:10:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.76300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T02:10:23.749 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-23T02:10:23.975 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:10:24.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:24 smithi071 ceph-mon[39837]: from='client.14422 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T02:10:24.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:24 smithi192 ceph-mon[49662]: from='client.14422 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T02:10:25.481 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:25 smithi071 ceph-mon[39837]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:25.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:25 smithi192 ceph-mon[49662]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:25.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.707+0000 7f0d89a71700 1 -- 172.21.15.71:0/2177973631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d84101de0 msgr2=0x7f0d841021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:25.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.707+0000 7f0d89a71700 1 --2- 172.21.15.71:0/2177973631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d84101de0 0x7f0d841021c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c009910 tx=0x7f0d6c009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:25.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.707+0000 7f0d89a71700 1 -- 172.21.15.71:0/2177973631 shutdown_connections 2024-04-23T02:10:25.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.707+0000 7f0d89a71700 1 --2- 172.21.15.71:0/2177973631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d84102700 0x7f0d8410abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:25.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.707+0000 7f0d89a71700 1 --2- 172.21.15.71:0/2177973631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d84101de0 0x7f0d841021c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:25.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.707+0000 7f0d89a71700 1 -- 172.21.15.71:0/2177973631 >> 172.21.15.71:0/2177973631 conn(0x7f0d840fb650 msgr2=0x7f0d840fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:25.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.707+0000 7f0d89a71700 1 -- 172.21.15.71:0/2177973631 shutdown_connections 2024-04-23T02:10:25.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.707+0000 7f0d89a71700 1 -- 172.21.15.71:0/2177973631 wait complete. 2024-04-23T02:10:25.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.708+0000 7f0d89a71700 1 Processor -- start 2024-04-23T02:10:25.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.708+0000 7f0d89a71700 1 -- start start 2024-04-23T02:10:25.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.708+0000 7f0d89a71700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d84101de0 0x7f0d840ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:25.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.708+0000 7f0d89a71700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d84102700 0x7f0d840fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:25.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.708+0000 7f0d89a71700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d84100550 con 0x7f0d84102700 2024-04-23T02:10:25.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.708+0000 7f0d89a71700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d841006c0 con 0x7f0d84101de0 2024-04-23T02:10:25.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.708+0000 7f0d82ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d84102700 0x7f0d840fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:25.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.708+0000 7f0d82ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d84102700 0x7f0d840fff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40482/0 (socket says 172.21.15.71:40482) 2024-04-23T02:10:25.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.708+0000 7f0d82ffd700 1 -- 172.21.15.71:0/1969726642 learned_addr learned my addr 172.21.15.71:0/1969726642 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:25.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.709+0000 7f0d837fe700 1 --2- 172.21.15.71:0/1969726642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d84101de0 0x7f0d840ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:25.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.709+0000 7f0d82ffd700 1 -- 172.21.15.71:0/1969726642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d84101de0 msgr2=0x7f0d840ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:25.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.709+0000 7f0d82ffd700 1 --2- 172.21.15.71:0/1969726642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d84101de0 0x7f0d840ffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:25.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.709+0000 7f0d82ffd700 1 -- 172.21.15.71:0/1969726642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c0095f0 con 0x7f0d84102700 2024-04-23T02:10:25.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.709+0000 7f0d82ffd700 1 --2- 172.21.15.71:0/1969726642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d84102700 0x7f0d840fff80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400b3d0 tx=0x7f0d7400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:25.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.709+0000 7f0d80ff9700 1 -- 172.21.15.71:0/1969726642 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d74011800 con 0x7f0d84102700 2024-04-23T02:10:25.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.709+0000 7f0d89a71700 1 -- 172.21.15.71:0/1969726642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d841a2a90 con 0x7f0d84102700 2024-04-23T02:10:25.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.709+0000 7f0d80ff9700 1 -- 172.21.15.71:0/1969726642 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f0d74011df0 con 0x7f0d84102700 2024-04-23T02:10:25.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.709+0000 7f0d80ff9700 1 -- 172.21.15.71:0/1969726642 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d74010e30 con 0x7f0d84102700 2024-04-23T02:10:25.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.709+0000 7f0d89a71700 1 -- 172.21.15.71:0/1969726642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d841a2f00 con 0x7f0d84102700 2024-04-23T02:10:25.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.710+0000 7f0d89a71700 1 -- 172.21.15.71:0/1969726642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d841082f0 con 0x7f0d84102700 2024-04-23T02:10:25.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.710+0000 7f0d80ff9700 1 -- 172.21.15.71:0/1969726642 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f0d74011960 con 0x7f0d84102700 2024-04-23T02:10:25.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.711+0000 7f0d80ff9700 1 --2- 172.21.15.71:0/1969726642 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0d7006c4e0 0x7f0d7006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:25.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.711+0000 7f0d80ff9700 1 -- 172.21.15.71:0/1969726642 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0d7408afc0 con 0x7f0d84102700 2024-04-23T02:10:25.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.711+0000 7f0d837fe700 1 --2- 172.21.15.71:0/1969726642 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0d7006c4e0 0x7f0d7006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:25.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.711+0000 7f0d837fe700 1 --2- 172.21.15.71:0/1969726642 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0d7006c4e0 0x7f0d7006e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c006010 tx=0x7f0d6c01a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:25.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.715+0000 7f0d80ff9700 1 -- 172.21.15.71:0/1969726642 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0d74055ae0 con 0x7f0d84102700 2024-04-23T02:10:25.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.899+0000 7f0d89a71700 1 -- 172.21.15.71:0/1969726642 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d840611d0 con 0x7f0d7006c4e0 2024-04-23T02:10:25.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.900+0000 7f0d80ff9700 1 -- 172.21.15.71:0/1969726642 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27974 (secure 0 0 0) 0x7f0d840611d0 con 0x7f0d7006c4e0 2024-04-23T02:10:25.906 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:10:25.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.903+0000 7f0d89a71700 1 -- 172.21.15.71:0/1969726642 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0d7006c4e0 msgr2=0x7f0d7006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:25.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.903+0000 7f0d89a71700 1 --2- 172.21.15.71:0/1969726642 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0d7006c4e0 0x7f0d7006e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c006010 tx=0x7f0d6c01a040 comp rx=0 tx=0).stop 2024-04-23T02:10:25.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.903+0000 7f0d89a71700 1 -- 172.21.15.71:0/1969726642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d84102700 msgr2=0x7f0d840fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:25.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.903+0000 7f0d89a71700 1 --2- 172.21.15.71:0/1969726642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d84102700 0x7f0d840fff80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400b3d0 tx=0x7f0d7400b790 comp rx=0 tx=0).stop 2024-04-23T02:10:25.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.903+0000 7f0d89a71700 1 -- 172.21.15.71:0/1969726642 shutdown_connections 2024-04-23T02:10:25.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.903+0000 7f0d89a71700 1 --2- 172.21.15.71:0/1969726642 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0d7006c4e0 0x7f0d7006e9a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:25.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.903+0000 7f0d89a71700 1 --2- 172.21.15.71:0/1969726642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d84101de0 0x7f0d840ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:25.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.903+0000 7f0d89a71700 1 --2- 172.21.15.71:0/1969726642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d84102700 0x7f0d840fff80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:25.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.903+0000 7f0d89a71700 1 -- 172.21.15.71:0/1969726642 >> 172.21.15.71:0/1969726642 conn(0x7f0d840fb650 msgr2=0x7f0d84105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:25.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.903+0000 7f0d89a71700 1 -- 172.21.15.71:0/1969726642 shutdown_connections 2024-04-23T02:10:25.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:25.903+0000 7f0d89a71700 1 -- 172.21.15.71:0/1969726642 wait complete. 2024-04-23T02:10:25.908 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-04-23T02:10:26.502 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-04-23T02:10:24.147039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165584,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305328,"statfs":{"total":575970213888,"available":575800655872,"internally_reserved":0,"allocated":3964928,"data_stored":2608835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002030"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":187,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-23T02:09:29.662915+0000","last_change":"2024-04-23T02:09:01.663614+0000","last_active":"2024-04-23T02:09:29.662915+0000","last_peered":"2024-04-23T02:09:29.662915+0000","last_clean":"2024-04-23T02:09:29.662915+0000","last_became_active":"2024-04-23T02:09:01.663310+0000","last_became_peered":"2024-04-23T02:09:01.663310+0000","last_unstale":"2024-04-23T02:09:29.662915+0000","last_undegraded":"2024-04-23T02:09:29.662915+0000","last_fullsized":"2024-04-23T02:09:29.662915+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T02:08:06.625838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T02:08:06.625838+0000","last_clean_scrub_stamp":"2024-04-23T02:08:06.625838+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T09:37:41.107099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 23 02:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.434},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.41699999999999998}]},{"osd":1,"last update":"Tue Apr 23 02:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.53900000000000003}]},{"osd":2,"last update":"Tue Apr 23 02:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.495}]},{"osd":3,"last update":"Tue Apr 23 02:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.53100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149720,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":729714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 23 02:09:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.53000000000000003}]},{"osd":1,"last update":"Tue Apr 23 02:09:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.48999999999999999}]},{"osd":2,"last update":"Tue Apr 23 02:09:33 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.503},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.51200000000000001}]},{"osd":4,"last update":"Tue Apr 23 02:10:02 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.434},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.38500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]}]},{"osd":2,"up_from":19,"seq":81604378654,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 23 02:10:11 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.33800000000000002,"15min":0.33200000000000002},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.192},{"interface":"front","average":{"1min":0.373,"5min":0.42599999999999999,"15min":0.435},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.71699999999999997,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.248}]},{"osd":1,"last update":"Tue Apr 23 02:10:11 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.35699999999999998,"15min":0.35299999999999998},"min":{"1min":0.21299999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.373,"15min":0.373},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.26600000000000001}]},{"osd":3,"last update":"Tue Apr 23 02:09:36 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.53800000000000003}]},{"osd":4,"last update":"Tue Apr 23 02:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]}]},{"osd":0,"up_from":9,"seq":38654705706,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Apr 23 02:09:40 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.309,"15min":0.30499999999999999},"min":{"1min":0.23699999999999999,"5min":0.182,"15min":0.182},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.32000000000000001},"min":{"1min":0.22700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.54600000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.433}]},{"osd":2,"last update":"Tue Apr 23 02:10:12 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.35599999999999998,"15min":0.35999999999999999},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.39600000000000002,"15min":0.39200000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":0.44500000000000001}]},{"osd":3,"last update":"Tue Apr 23 02:09:36 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.61799999999999999}]},{"osd":4,"last update":"Tue Apr 23 02:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Apr 23 02:09:39 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.29399999999999998,"15min":0.28399999999999997},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.30199999999999999,"15min":0.29399999999999998},"min":{"1min":0.23699999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.44400000000000001}]},{"osd":2,"last update":"Tue Apr 23 02:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.34699999999999998,"15min":0.35099999999999998},"min":{"1min":0.24399999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.311,"5min":0.35799999999999998,"15min":0.36599999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"last":0.46400000000000002}]},{"osd":3,"last update":"Tue Apr 23 02:09:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.80500000000000005}]},{"osd":4,"last update":"Tue Apr 23 02:10:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.76300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T02:10:26.504 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-23T02:10:26.504 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-23T02:10:26.504 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-23T02:10:26.505 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-23T02:10:26.734 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:10:26.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:26 smithi071 ceph-mon[39837]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T02:10:26.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:26 smithi071 ceph-mon[39837]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:27.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:26 smithi192 ceph-mon[49662]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T02:10:27.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:26 smithi192 ceph-mon[49662]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:27.985 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:27 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:10:28.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:27 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:10:28.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.482+0000 7f3d0c12e700 1 -- 172.21.15.71:0/2542787481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d040ffc20 msgr2=0x7f3d04100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:28.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.482+0000 7f3d0c12e700 1 --2- 172.21.15.71:0/2542787481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d040ffc20 0x7f3d04100080 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009910 tx=0x7f3d00009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:28.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.482+0000 7f3d0c12e700 1 -- 172.21.15.71:0/2542787481 shutdown_connections 2024-04-23T02:10:28.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.482+0000 7f3d0c12e700 1 --2- 172.21.15.71:0/2542787481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d040ffc20 0x7f3d04100080 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:28.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.482+0000 7f3d0c12e700 1 --2- 172.21.15.71:0/2542787481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d04105bd0 0x7f3d04105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:28.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.482+0000 7f3d0c12e700 1 -- 172.21.15.71:0/2542787481 >> 172.21.15.71:0/2542787481 conn(0x7f3d04075670 msgr2=0x7f3d04075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:28.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.482+0000 7f3d0c12e700 1 -- 172.21.15.71:0/2542787481 shutdown_connections 2024-04-23T02:10:28.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.482+0000 7f3d0c12e700 1 -- 172.21.15.71:0/2542787481 wait complete. 2024-04-23T02:10:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.483+0000 7f3d0c12e700 1 Processor -- start 2024-04-23T02:10:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.483+0000 7f3d0c12e700 1 -- start start 2024-04-23T02:10:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.483+0000 7f3d0c12e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d040ffc20 0x7f3d0419b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.483+0000 7f3d0c12e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d04105bd0 0x7f3d0419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.483+0000 7f3d0c12e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0419c190 con 0x7f3d040ffc20 2024-04-23T02:10:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.483+0000 7f3d0c12e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d04195640 con 0x7f3d04105bd0 2024-04-23T02:10:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.483+0000 7f3d09eca700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d040ffc20 0x7f3d0419b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.483+0000 7f3d09eca700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d040ffc20 0x7f3d0419b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40508/0 (socket says 172.21.15.71:40508) 2024-04-23T02:10:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.483+0000 7f3d09eca700 1 -- 172.21.15.71:0/1213896652 learned_addr learned my addr 172.21.15.71:0/1213896652 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.483+0000 7f3d096c9700 1 --2- 172.21.15.71:0/1213896652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d04105bd0 0x7f3d0419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.483+0000 7f3d09eca700 1 -- 172.21.15.71:0/1213896652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d04105bd0 msgr2=0x7f3d0419bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:28.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.483+0000 7f3d09eca700 1 --2- 172.21.15.71:0/1213896652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d04105bd0 0x7f3d0419bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:28.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.484+0000 7f3d09eca700 1 -- 172.21.15.71:0/1213896652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d000095f0 con 0x7f3d040ffc20 2024-04-23T02:10:28.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.484+0000 7f3d096c9700 1 --2- 172.21.15.71:0/1213896652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d04105bd0 0x7f3d0419bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:10:28.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.484+0000 7f3d09eca700 1 --2- 172.21.15.71:0/1213896652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d040ffc20 0x7f3d0419b5c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800d960 tx=0x7f3cf800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:28.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.484+0000 7f3cf6ffd700 1 -- 172.21.15.71:0/1213896652 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cf800ec20 con 0x7f3d040ffc20 2024-04-23T02:10:28.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.484+0000 7f3d0c12e700 1 -- 172.21.15.71:0/1213896652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d041958c0 con 0x7f3d040ffc20 2024-04-23T02:10:28.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.484+0000 7f3cf6ffd700 1 -- 172.21.15.71:0/1213896652 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f3cf800ed80 con 0x7f3d040ffc20 2024-04-23T02:10:28.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.484+0000 7f3cf6ffd700 1 -- 172.21.15.71:0/1213896652 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cf8014700 con 0x7f3d040ffc20 2024-04-23T02:10:28.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.484+0000 7f3d0c12e700 1 -- 172.21.15.71:0/1213896652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d04195e10 con 0x7f3d040ffc20 2024-04-23T02:10:28.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.485+0000 7f3d0c12e700 1 -- 172.21.15.71:0/1213896652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d0404f2e0 con 0x7f3d040ffc20 2024-04-23T02:10:28.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.485+0000 7f3cf6ffd700 1 -- 172.21.15.71:0/1213896652 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f3cf8016440 con 0x7f3d040ffc20 2024-04-23T02:10:28.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.486+0000 7f3cf6ffd700 1 --2- 172.21.15.71:0/1213896652 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f3cf006c4e0 0x7f3cf006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:28.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.486+0000 7f3cf6ffd700 1 -- 172.21.15.71:0/1213896652 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3cf808ceb0 con 0x7f3d040ffc20 2024-04-23T02:10:28.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.486+0000 7f3d096c9700 1 --2- 172.21.15.71:0/1213896652 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f3cf006c4e0 0x7f3cf006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:28.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.486+0000 7f3d096c9700 1 --2- 172.21.15.71:0/1213896652 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f3cf006c4e0 0x7f3cf006e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000b5c0 tx=0x7f3d00005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:28.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.490+0000 7f3cf6ffd700 1 -- 172.21.15.71:0/1213896652 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3cf805b510 con 0x7f3d040ffc20 2024-04-23T02:10:28.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.721+0000 7f3d0c12e700 1 -- 172.21.15.71:0/1213896652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3d04196b50 con 0x7f3d040ffc20 2024-04-23T02:10:28.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.721+0000 7f3cf6ffd700 1 -- 172.21.15.71:0/1213896652 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f3cf805b0a0 con 0x7f3d040ffc20 2024-04-23T02:10:28.726 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:10:28.726 INFO:teuthology.orchestra.run.smithi071.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-23T02:10:28.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.723+0000 7f3d0c12e700 1 -- 172.21.15.71:0/1213896652 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f3cf006c4e0 msgr2=0x7f3cf006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:28.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.723+0000 7f3d0c12e700 1 --2- 172.21.15.71:0/1213896652 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f3cf006c4e0 0x7f3cf006e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000b5c0 tx=0x7f3d00005960 comp rx=0 tx=0).stop 2024-04-23T02:10:28.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.723+0000 7f3d0c12e700 1 -- 172.21.15.71:0/1213896652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d040ffc20 msgr2=0x7f3d0419b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:28.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.723+0000 7f3d0c12e700 1 --2- 172.21.15.71:0/1213896652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d040ffc20 0x7f3d0419b5c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800d960 tx=0x7f3cf800dd20 comp rx=0 tx=0).stop 2024-04-23T02:10:28.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.723+0000 7f3d0c12e700 1 -- 172.21.15.71:0/1213896652 shutdown_connections 2024-04-23T02:10:28.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.723+0000 7f3d0c12e700 1 --2- 172.21.15.71:0/1213896652 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f3cf006c4e0 0x7f3cf006e9a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:28.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.723+0000 7f3d0c12e700 1 --2- 172.21.15.71:0/1213896652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d040ffc20 0x7f3d0419b5c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:28.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.723+0000 7f3d0c12e700 1 --2- 172.21.15.71:0/1213896652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d04105bd0 0x7f3d0419bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:28.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.723+0000 7f3d0c12e700 1 -- 172.21.15.71:0/1213896652 >> 172.21.15.71:0/1213896652 conn(0x7f3d04075670 msgr2=0x7f3d040fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:28.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.723+0000 7f3d0c12e700 1 -- 172.21.15.71:0/1213896652 shutdown_connections 2024-04-23T02:10:28.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:28.723+0000 7f3d0c12e700 1 -- 172.21.15.71:0/1213896652 wait complete. 2024-04-23T02:10:28.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:28 smithi071 ceph-mon[39837]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:29.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:28 smithi192 ceph-mon[49662]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:29.364 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-23T02:10:29.364 INFO:tasks.cephadm:Setup complete, yielding 2024-04-23T02:10:29.365 INFO:teuthology.run_tasks:Running task print... 2024-04-23T02:10:29.374 INFO:teuthology.task.print:**** done end installing v18.2.1 cephadm ... 2024-04-23T02:10:29.374 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T02:10:29.432 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi071.front.sepia.ceph.com 2024-04-23T02:10:29.433 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-04-23T02:10:29.662 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:10:29.923 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:29 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1213896652' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T02:10:30.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:29 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1213896652' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T02:10:30.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:30 smithi071 ceph-mon[39837]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:31.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:30 smithi192 ceph-mon[49662]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:31.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.812+0000 7f7e8d775700 1 -- 172.21.15.71:0/433326720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e881017c0 msgr2=0x7f7e88103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:31.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.812+0000 7f7e8d775700 1 --2- 172.21.15.71:0/433326720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e881017c0 0x7f7e88103bb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009910 tx=0x7f7e78009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:31.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.812+0000 7f7e8d775700 1 -- 172.21.15.71:0/433326720 shutdown_connections 2024-04-23T02:10:31.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.812+0000 7f7e8d775700 1 --2- 172.21.15.71:0/433326720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e881040f0 0x7f7e881064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:31.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.812+0000 7f7e8d775700 1 --2- 172.21.15.71:0/433326720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e881017c0 0x7f7e88103bb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:31.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.812+0000 7f7e8d775700 1 -- 172.21.15.71:0/433326720 >> 172.21.15.71:0/433326720 conn(0x7f7e880fb150 msgr2=0x7f7e880fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:31.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.812+0000 7f7e8d775700 1 -- 172.21.15.71:0/433326720 shutdown_connections 2024-04-23T02:10:31.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.812+0000 7f7e8d775700 1 -- 172.21.15.71:0/433326720 wait complete. 2024-04-23T02:10:31.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.812+0000 7f7e8d775700 1 Processor -- start 2024-04-23T02:10:31.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.813+0000 7f7e8d775700 1 -- start start 2024-04-23T02:10:31.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.813+0000 7f7e8d775700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e881017c0 0x7f7e88101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:31.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.813+0000 7f7e8d775700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e881040f0 0x7f7e880ff770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:31.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.813+0000 7f7e8d775700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e880ffcb0 con 0x7f7e881017c0 2024-04-23T02:10:31.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.813+0000 7f7e8d775700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e880ffdf0 con 0x7f7e881040f0 2024-04-23T02:10:31.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.813+0000 7f7e86ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e881017c0 0x7f7e88101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:31.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.813+0000 7f7e86ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e881017c0 0x7f7e88101120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40522/0 (socket says 172.21.15.71:40522) 2024-04-23T02:10:31.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.813+0000 7f7e86ffd700 1 -- 172.21.15.71:0/269963663 learned_addr learned my addr 172.21.15.71:0/269963663 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:31.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.813+0000 7f7e867fc700 1 --2- 172.21.15.71:0/269963663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e881040f0 0x7f7e880ff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:31.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.813+0000 7f7e86ffd700 1 -- 172.21.15.71:0/269963663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e881040f0 msgr2=0x7f7e880ff770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:31.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.813+0000 7f7e86ffd700 1 --2- 172.21.15.71:0/269963663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e881040f0 0x7f7e880ff770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:31.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.813+0000 7f7e86ffd700 1 -- 172.21.15.71:0/269963663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e780095f0 con 0x7f7e881017c0 2024-04-23T02:10:31.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.813+0000 7f7e86ffd700 1 --2- 172.21.15.71:0/269963663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e881017c0 0x7f7e88101120 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7e78006010 tx=0x7f7e7800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:31.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.814+0000 7f7e6ffff700 1 -- 172.21.15.71:0/269963663 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e7801d070 con 0x7f7e881017c0 2024-04-23T02:10:31.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.814+0000 7f7e6ffff700 1 -- 172.21.15.71:0/269963663 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7e7800f460 con 0x7f7e881017c0 2024-04-23T02:10:31.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.814+0000 7f7e8d775700 1 -- 172.21.15.71:0/269963663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e88100070 con 0x7f7e881017c0 2024-04-23T02:10:31.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.814+0000 7f7e6ffff700 1 -- 172.21.15.71:0/269963663 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e78021640 con 0x7f7e881017c0 2024-04-23T02:10:31.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.814+0000 7f7e8d775700 1 -- 172.21.15.71:0/269963663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e88100560 con 0x7f7e881017c0 2024-04-23T02:10:31.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.815+0000 7f7e8d775700 1 -- 172.21.15.71:0/269963663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e881904e0 con 0x7f7e881017c0 2024-04-23T02:10:31.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.815+0000 7f7e6ffff700 1 -- 172.21.15.71:0/269963663 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f7e7800f9b0 con 0x7f7e881017c0 2024-04-23T02:10:31.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.815+0000 7f7e6ffff700 1 --2- 172.21.15.71:0/269963663 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f7e7006c4e0 0x7f7e7006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:31.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.816+0000 7f7e6ffff700 1 -- 172.21.15.71:0/269963663 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7e7808c730 con 0x7f7e881017c0 2024-04-23T02:10:31.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.816+0000 7f7e867fc700 1 --2- 172.21.15.71:0/269963663 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f7e7006c4e0 0x7f7e7006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:31.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.816+0000 7f7e867fc700 1 --2- 172.21.15.71:0/269963663 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f7e7006c4e0 0x7f7e7006e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c005950 tx=0x7f7e7c00b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:31.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:31.820+0000 7f7e6ffff700 1 -- 172.21.15.71:0/269963663 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7e7805ace0 con 0x7f7e881017c0 2024-04-23T02:10:32.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:32.023+0000 7f7e8d775700 1 -- 172.21.15.71:0/269963663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f7e8802cc70 con 0x7f7e881017c0 2024-04-23T02:10:32.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:32.027+0000 7f7e6ffff700 1 -- 172.21.15.71:0/269963663 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f7e7805a870 con 0x7f7e881017c0 2024-04-23T02:10:32.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:32.030+0000 7f7e8d775700 1 -- 172.21.15.71:0/269963663 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f7e7006c4e0 msgr2=0x7f7e7006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:32.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:32.030+0000 7f7e8d775700 1 --2- 172.21.15.71:0/269963663 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f7e7006c4e0 0x7f7e7006e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c005950 tx=0x7f7e7c00b410 comp rx=0 tx=0).stop 2024-04-23T02:10:32.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:32.030+0000 7f7e8d775700 1 -- 172.21.15.71:0/269963663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e881017c0 msgr2=0x7f7e88101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:32.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:32.030+0000 7f7e8d775700 1 --2- 172.21.15.71:0/269963663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e881017c0 0x7f7e88101120 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7e78006010 tx=0x7f7e7800bd70 comp rx=0 tx=0).stop 2024-04-23T02:10:32.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:32.030+0000 7f7e8d775700 1 -- 172.21.15.71:0/269963663 shutdown_connections 2024-04-23T02:10:32.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:32.030+0000 7f7e8d775700 1 --2- 172.21.15.71:0/269963663 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f7e7006c4e0 0x7f7e7006e9a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:32.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:32.030+0000 7f7e8d775700 1 --2- 172.21.15.71:0/269963663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e881017c0 0x7f7e88101120 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:32.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:32.030+0000 7f7e8d775700 1 --2- 172.21.15.71:0/269963663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e881040f0 0x7f7e880ff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:32.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:32.030+0000 7f7e8d775700 1 -- 172.21.15.71:0/269963663 >> 172.21.15.71:0/269963663 conn(0x7f7e880fb150 msgr2=0x7f7e880fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:32.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:32.030+0000 7f7e8d775700 1 -- 172.21.15.71:0/269963663 shutdown_connections 2024-04-23T02:10:32.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:32.030+0000 7f7e8d775700 1 -- 172.21.15.71:0/269963663 wait complete. 2024-04-23T02:10:32.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:31 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:10:32.155 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:31 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:10:32.810 INFO:teuthology.run_tasks:Running task print... 2024-04-23T02:10:32.826 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-04-23T02:10:32.826 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T02:10:32.846 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi071.front.sepia.ceph.com 2024-04-23T02:10:32.846 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-23T02:10:33.075 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:10:33.339 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:33 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/269963663' entity='client.admin' 2024-04-23T02:10:33.339 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:33 smithi071 ceph-mon[39837]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:33.339 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:33 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:10:33.339 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:33 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:10:33.339 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:33 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:33.339 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:33 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:10:33.339 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:33 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:10:33.339 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:33 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:10:33.339 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:33 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:33.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:33 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/269963663' entity='client.admin' 2024-04-23T02:10:33.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:33 smithi192 ceph-mon[49662]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:33.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:33 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:10:33.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:33 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:10:33.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:33 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:33.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:33 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:10:33.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:33 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:10:33.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:33 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:10:33.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:33 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:34.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.824+0000 7fa5ab8a1700 1 -- 172.21.15.71:0/3743291744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a41017f0 msgr2=0x7fa5a4103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:34.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.824+0000 7fa5ab8a1700 1 --2- 172.21.15.71:0/3743291744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a41017f0 0x7fa5a4103be0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0009910 tx=0x7fa5a0009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:34.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.824+0000 7fa5ab8a1700 1 -- 172.21.15.71:0/3743291744 shutdown_connections 2024-04-23T02:10:34.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.824+0000 7fa5ab8a1700 1 --2- 172.21.15.71:0/3743291744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5a4104120 0x7fa5a4106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:34.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.824+0000 7fa5ab8a1700 1 --2- 172.21.15.71:0/3743291744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a41017f0 0x7fa5a4103be0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:34.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.824+0000 7fa5ab8a1700 1 -- 172.21.15.71:0/3743291744 >> 172.21.15.71:0/3743291744 conn(0x7fa5a40fb160 msgr2=0x7fa5a40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:34.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.824+0000 7fa5ab8a1700 1 -- 172.21.15.71:0/3743291744 shutdown_connections 2024-04-23T02:10:34.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.824+0000 7fa5ab8a1700 1 -- 172.21.15.71:0/3743291744 wait complete. 2024-04-23T02:10:34.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.825+0000 7fa5ab8a1700 1 Processor -- start 2024-04-23T02:10:34.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.825+0000 7fa5ab8a1700 1 -- start start 2024-04-23T02:10:34.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.825+0000 7fa5ab8a1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a41017f0 0x7fa5a4101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:34.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.825+0000 7fa5ab8a1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5a4104120 0x7fa5a40ff7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:34.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.825+0000 7fa5ab8a1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a40ffce0 con 0x7fa5a41017f0 2024-04-23T02:10:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.825+0000 7fa5ab8a1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a40ffe20 con 0x7fa5a4104120 2024-04-23T02:10:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.825+0000 7fa5a963d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a41017f0 0x7fa5a4101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.825+0000 7fa5a963d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a41017f0 0x7fa5a4101150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40530/0 (socket says 172.21.15.71:40530) 2024-04-23T02:10:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.825+0000 7fa5a963d700 1 -- 172.21.15.71:0/3045607182 learned_addr learned my addr 172.21.15.71:0/3045607182 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.825+0000 7fa5a8e3c700 1 --2- 172.21.15.71:0/3045607182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5a4104120 0x7fa5a40ff7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.825+0000 7fa5a963d700 1 -- 172.21.15.71:0/3045607182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5a4104120 msgr2=0x7fa5a40ff7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.825+0000 7fa5a963d700 1 --2- 172.21.15.71:0/3045607182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5a4104120 0x7fa5a40ff7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.825+0000 7fa5a963d700 1 -- 172.21.15.71:0/3045607182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a00095f0 con 0x7fa5a41017f0 2024-04-23T02:10:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.825+0000 7fa5a8e3c700 1 --2- 172.21.15.71:0/3045607182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5a4104120 0x7fa5a40ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:10:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.825+0000 7fa5a963d700 1 --2- 172.21.15.71:0/3045607182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a41017f0 0x7fa5a4101150 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0006010 tx=0x7fa5a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:34.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.825+0000 7fa5967fc700 1 -- 172.21.15.71:0/3045607182 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5a001d070 con 0x7fa5a41017f0 2024-04-23T02:10:34.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.826+0000 7fa5967fc700 1 -- 172.21.15.71:0/3045607182 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fa5a000f460 con 0x7fa5a41017f0 2024-04-23T02:10:34.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.826+0000 7fa5ab8a1700 1 -- 172.21.15.71:0/3045607182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a41000a0 con 0x7fa5a41017f0 2024-04-23T02:10:34.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.826+0000 7fa5967fc700 1 -- 172.21.15.71:0/3045607182 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5a0021640 con 0x7fa5a41017f0 2024-04-23T02:10:34.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.826+0000 7fa5ab8a1700 1 -- 172.21.15.71:0/3045607182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a4100590 con 0x7fa5a41017f0 2024-04-23T02:10:34.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.826+0000 7fa5ab8a1700 1 -- 172.21.15.71:0/3045607182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a4190500 con 0x7fa5a41017f0 2024-04-23T02:10:34.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.827+0000 7fa5967fc700 1 -- 172.21.15.71:0/3045607182 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fa5a000f940 con 0x7fa5a41017f0 2024-04-23T02:10:34.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.827+0000 7fa5967fc700 1 --2- 172.21.15.71:0/3045607182 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fa59006c4e0 0x7fa59006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:34.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.827+0000 7fa5a8e3c700 1 --2- 172.21.15.71:0/3045607182 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fa59006c4e0 0x7fa59006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:34.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.827+0000 7fa5967fc700 1 -- 172.21.15.71:0/3045607182 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa5a008c7e0 con 0x7fa5a41017f0 2024-04-23T02:10:34.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.828+0000 7fa5a8e3c700 1 --2- 172.21.15.71:0/3045607182 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fa59006c4e0 0x7fa59006e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa598005950 tx=0x7fa5980058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:34.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:34.832+0000 7fa5967fc700 1 -- 172.21.15.71:0/3045607182 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa5a005ad90 con 0x7fa5a41017f0 2024-04-23T02:10:35.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:35.034+0000 7fa5ab8a1700 1 -- 172.21.15.71:0/3045607182 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5a40611d0 con 0x7fa59006c4e0 2024-04-23T02:10:35.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:35.035+0000 7fa5967fc700 1 -- 172.21.15.71:0/3045607182 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fa5a40611d0 con 0x7fa59006c4e0 2024-04-23T02:10:35.039 INFO:teuthology.orchestra.run.smithi071.stdout:Backend: cephadm 2024-04-23T02:10:35.039 INFO:teuthology.orchestra.run.smithi071.stdout:Available: Yes 2024-04-23T02:10:35.039 INFO:teuthology.orchestra.run.smithi071.stdout:Paused: No 2024-04-23T02:10:35.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:35.037+0000 7fa5ab8a1700 1 -- 172.21.15.71:0/3045607182 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fa59006c4e0 msgr2=0x7fa59006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:35.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:35.037+0000 7fa5ab8a1700 1 --2- 172.21.15.71:0/3045607182 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fa59006c4e0 0x7fa59006e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa598005950 tx=0x7fa5980058e0 comp rx=0 tx=0).stop 2024-04-23T02:10:35.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:35.037+0000 7fa5ab8a1700 1 -- 172.21.15.71:0/3045607182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a41017f0 msgr2=0x7fa5a4101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:35.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:35.037+0000 7fa5ab8a1700 1 --2- 172.21.15.71:0/3045607182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a41017f0 0x7fa5a4101150 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0006010 tx=0x7fa5a000be20 comp rx=0 tx=0).stop 2024-04-23T02:10:35.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:35.037+0000 7fa5ab8a1700 1 -- 172.21.15.71:0/3045607182 shutdown_connections 2024-04-23T02:10:35.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:35.037+0000 7fa5ab8a1700 1 --2- 172.21.15.71:0/3045607182 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fa59006c4e0 0x7fa59006e9a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:35.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:35.037+0000 7fa5ab8a1700 1 --2- 172.21.15.71:0/3045607182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a41017f0 0x7fa5a4101150 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:35.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:35.037+0000 7fa5ab8a1700 1 --2- 172.21.15.71:0/3045607182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5a4104120 0x7fa5a40ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:35.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:35.037+0000 7fa5ab8a1700 1 -- 172.21.15.71:0/3045607182 >> 172.21.15.71:0/3045607182 conn(0x7fa5a40fb160 msgr2=0x7fa5a40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:35.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:35.037+0000 7fa5ab8a1700 1 -- 172.21.15.71:0/3045607182 shutdown_connections 2024-04-23T02:10:35.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:35.037+0000 7fa5ab8a1700 1 -- 172.21.15.71:0/3045607182 wait complete. 2024-04-23T02:10:35.440 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:35 smithi071 ceph-mon[39837]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:35.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:35 smithi192 ceph-mon[49662]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:35.742 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-23T02:10:35.971 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:10:36.463 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:36 smithi071 ceph-mon[39837]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:10:36.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:36 smithi192 ceph-mon[49662]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:10:37.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:37 smithi071 ceph-mon[39837]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:37.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:37 smithi192 ceph-mon[49662]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:37.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.904+0000 7fa5b8f87700 1 -- 172.21.15.71:0/996798346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5b4100dc0 msgr2=0x7fa5b4101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:37.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.904+0000 7fa5b8f87700 1 --2- 172.21.15.71:0/996798346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5b4100dc0 0x7fa5b4101220 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009910 tx=0x7fa59c009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:37.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.904+0000 7fa5b8f87700 1 -- 172.21.15.71:0/996798346 shutdown_connections 2024-04-23T02:10:37.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.904+0000 7fa5b8f87700 1 --2- 172.21.15.71:0/996798346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5b4100dc0 0x7fa5b4101220 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:37.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.904+0000 7fa5b8f87700 1 --2- 172.21.15.71:0/996798346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5b40ffbc0 0x7fa5b40fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:37.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.904+0000 7fa5b8f87700 1 -- 172.21.15.71:0/996798346 >> 172.21.15.71:0/996798346 conn(0x7fa5b40fb160 msgr2=0x7fa5b40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:37.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.904+0000 7fa5b8f87700 1 -- 172.21.15.71:0/996798346 shutdown_connections 2024-04-23T02:10:37.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.904+0000 7fa5b8f87700 1 -- 172.21.15.71:0/996798346 wait complete. 2024-04-23T02:10:37.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.905+0000 7fa5b8f87700 1 Processor -- start 2024-04-23T02:10:37.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.905+0000 7fa5b8f87700 1 -- start start 2024-04-23T02:10:37.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.905+0000 7fa5b8f87700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5b40ffbc0 0x7fa5b4103b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:37.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.905+0000 7fa5b8f87700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5b4100dc0 0x7fa5b41021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:37.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.905+0000 7fa5b8f87700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b41026e0 con 0x7fa5b40ffbc0 2024-04-23T02:10:37.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.905+0000 7fa5b8f87700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b4102820 con 0x7fa5b4100dc0 2024-04-23T02:10:37.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.905+0000 7fa5b259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5b40ffbc0 0x7fa5b4103b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:37.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.905+0000 7fa5b259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5b40ffbc0 0x7fa5b4103b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51684/0 (socket says 172.21.15.71:51684) 2024-04-23T02:10:37.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.905+0000 7fa5b259c700 1 -- 172.21.15.71:0/3419563448 learned_addr learned my addr 172.21.15.71:0/3419563448 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:37.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.905+0000 7fa5b259c700 1 -- 172.21.15.71:0/3419563448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5b4100dc0 msgr2=0x7fa5b41021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:37.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.905+0000 7fa5b1d9b700 1 --2- 172.21.15.71:0/3419563448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5b4100dc0 0x7fa5b41021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:37.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.905+0000 7fa5b259c700 1 --2- 172.21.15.71:0/3419563448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5b4100dc0 0x7fa5b41021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:37.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.905+0000 7fa5b259c700 1 -- 172.21.15.71:0/3419563448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59c0095f0 con 0x7fa5b40ffbc0 2024-04-23T02:10:37.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.905+0000 7fa5b1d9b700 1 --2- 172.21.15.71:0/3419563448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5b4100dc0 0x7fa5b41021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:10:37.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.906+0000 7fa5b259c700 1 --2- 172.21.15.71:0/3419563448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5b40ffbc0 0x7fa5b4103b50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400b480 tx=0x7fa5a400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:37.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.906+0000 7fa5ab7fe700 1 -- 172.21.15.71:0/3419563448 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5a40118c0 con 0x7fa5b40ffbc0 2024-04-23T02:10:37.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.906+0000 7fa5ab7fe700 1 -- 172.21.15.71:0/3419563448 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fa5a4010380 con 0x7fa5b40ffbc0 2024-04-23T02:10:37.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.906+0000 7fa5b8f87700 1 -- 172.21.15.71:0/3419563448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b4102b00 con 0x7fa5b40ffbc0 2024-04-23T02:10:37.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.906+0000 7fa5ab7fe700 1 -- 172.21.15.71:0/3419563448 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5a400f510 con 0x7fa5b40ffbc0 2024-04-23T02:10:37.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.906+0000 7fa5b8f87700 1 -- 172.21.15.71:0/3419563448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b419e880 con 0x7fa5b40ffbc0 2024-04-23T02:10:37.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.907+0000 7fa5b8f87700 1 -- 172.21.15.71:0/3419563448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b4068af0 con 0x7fa5b40ffbc0 2024-04-23T02:10:37.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.907+0000 7fa5ab7fe700 1 -- 172.21.15.71:0/3419563448 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fa5a40104f0 con 0x7fa5b40ffbc0 2024-04-23T02:10:37.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.907+0000 7fa5ab7fe700 1 --2- 172.21.15.71:0/3419563448 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fa5a006c4e0 0x7fa5a006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:37.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.908+0000 7fa5b1d9b700 1 --2- 172.21.15.71:0/3419563448 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fa5a006c4e0 0x7fa5a006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:37.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.908+0000 7fa5ab7fe700 1 -- 172.21.15.71:0/3419563448 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa5a408b170 con 0x7fa5b40ffbc0 2024-04-23T02:10:37.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.908+0000 7fa5b1d9b700 1 --2- 172.21.15.71:0/3419563448 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fa5a006c4e0 0x7fa5a006e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00b5c0 tx=0x7fa59c005e20 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:37.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:37.912+0000 7fa5ab7fe700 1 -- 172.21.15.71:0/3419563448 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa5a4059720 con 0x7fa5b40ffbc0 2024-04-23T02:10:38.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:38.107+0000 7fa5b8f87700 1 -- 172.21.15.71:0/3419563448 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5b419ea10 con 0x7fa5a006c4e0 2024-04-23T02:10:38.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:38.116+0000 7fa5ab7fe700 1 -- 172.21.15.71:0/3419563448 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2860 (secure 0 0 0) 0x7fa5b419ea10 con 0x7fa5a006c4e0 2024-04-23T02:10:38.120 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:10:38.120 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (4m) 2m ago 5m 26.2M - 0.25.0 c8568f914cd2 8261e17319fe 2024-04-23T02:10:38.120 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (5m) 2m ago 5m 7054k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:10:38.120 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (4m) 70s ago 4m 6941k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:10:38.120 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (5m) 2m ago 5m 6656k - 18.2.1 5be31c24972a 258512dcbc00 2024-04-23T02:10:38.120 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (4m) 70s ago 4m 6656k - 18.2.1 5be31c24972a feea2c9896a6 2024-04-23T02:10:38.120 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (4m) 2m ago 5m 78.4M - 9.4.7 954c08fa6188 48ea3e02b806 2024-04-23T02:10:38.120 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:9283,8765,8443 running (7m) 2m ago 7m 482M - 18.2.1 5be31c24972a fbe1ad3229f5 2024-04-23T02:10:38.120 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (4m) 70s ago 4m 439M - 18.2.1 5be31c24972a cf73640a7ee0 2024-04-23T02:10:38.120 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (7m) 2m ago 7m 48.4M 2048M 18.2.1 5be31c24972a ff40abe63fd9 2024-04-23T02:10:38.121 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (4m) 70s ago 4m 41.0M 2048M 18.2.1 5be31c24972a 9c03e90acc95 2024-04-23T02:10:38.121 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (5m) 2m ago 5m 17.6M - 1.5.0 0da6a335fe13 3f4d0aaee04e 2024-04-23T02:10:38.121 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (4m) 70s ago 4m 16.7M - 1.5.0 0da6a335fe13 cfd0b49d770c 2024-04-23T02:10:38.121 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (3m) 2m ago 3m 40.9M 3958M 18.2.1 5be31c24972a d3d5f2fa7267 2024-04-23T02:10:38.121 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (3m) 2m ago 3m 41.5M 3958M 18.2.1 5be31c24972a 3992157ff0fd 2024-04-23T02:10:38.121 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (2m) 2m ago 2m 40.5M 3958M 18.2.1 5be31c24972a 12380b6a15a2 2024-04-23T02:10:38.121 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (2m) 70s ago 2m 42.7M 4982M 18.2.1 5be31c24972a d5f587ee3a22 2024-04-23T02:10:38.121 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (103s) 70s ago 103s 38.2M 4982M 18.2.1 5be31c24972a 958d046005f0 2024-04-23T02:10:38.121 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (75s) 70s ago 75s 11.4M 4982M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:10:38.121 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (3m) 2m ago 5m 40.3M - 2.43.0 a07b618ecd1d 8df24da82462 2024-04-23T02:10:38.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:38.118+0000 7fa5b8f87700 1 -- 172.21.15.71:0/3419563448 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fa5a006c4e0 msgr2=0x7fa5a006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:38.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:38.118+0000 7fa5b8f87700 1 --2- 172.21.15.71:0/3419563448 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fa5a006c4e0 0x7fa5a006e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00b5c0 tx=0x7fa59c005e20 comp rx=0 tx=0).stop 2024-04-23T02:10:38.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:38.118+0000 7fa5b8f87700 1 -- 172.21.15.71:0/3419563448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5b40ffbc0 msgr2=0x7fa5b4103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:38.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:38.118+0000 7fa5b8f87700 1 --2- 172.21.15.71:0/3419563448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5b40ffbc0 0x7fa5b4103b50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400b480 tx=0x7fa5a400b840 comp rx=0 tx=0).stop 2024-04-23T02:10:38.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:38.118+0000 7fa5b8f87700 1 -- 172.21.15.71:0/3419563448 shutdown_connections 2024-04-23T02:10:38.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:38.118+0000 7fa5b8f87700 1 --2- 172.21.15.71:0/3419563448 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fa5a006c4e0 0x7fa5a006e9a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:38.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:38.118+0000 7fa5b8f87700 1 --2- 172.21.15.71:0/3419563448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5b40ffbc0 0x7fa5b4103b50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:38.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:38.118+0000 7fa5b8f87700 1 --2- 172.21.15.71:0/3419563448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5b4100dc0 0x7fa5b41021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:38.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:38.118+0000 7fa5b8f87700 1 -- 172.21.15.71:0/3419563448 >> 172.21.15.71:0/3419563448 conn(0x7fa5b40fb160 msgr2=0x7fa5b4068f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:38.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:38.118+0000 7fa5b8f87700 1 -- 172.21.15.71:0/3419563448 shutdown_connections 2024-04-23T02:10:38.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:38.118+0000 7fa5b8f87700 1 -- 172.21.15.71:0/3419563448 wait complete. 2024-04-23T02:10:38.888 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-23T02:10:39.116 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:10:39.465 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:39 smithi192 ceph-mon[49662]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:10:39.466 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:39 smithi192 ceph-mon[49662]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:39.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:39 smithi071 ceph-mon[39837]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:10:39.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:39 smithi071 ceph-mon[39837]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:40.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.922+0000 7fc9e2fb1700 1 -- 172.21.15.71:0/238196801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9dc100ca0 msgr2=0x7fc9dc1010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:40.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.922+0000 7fc9e2fb1700 1 --2- 172.21.15.71:0/238196801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9dc100ca0 0x7fc9dc1010c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009910 tx=0x7fc9d0009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:40.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.922+0000 7fc9e2fb1700 1 -- 172.21.15.71:0/238196801 shutdown_connections 2024-04-23T02:10:40.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.922+0000 7fc9e2fb1700 1 --2- 172.21.15.71:0/238196801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9dc101ea0 0x7fc9dc102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:40.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.922+0000 7fc9e2fb1700 1 --2- 172.21.15.71:0/238196801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9dc100ca0 0x7fc9dc1010c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:40.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.922+0000 7fc9e2fb1700 1 -- 172.21.15.71:0/238196801 >> 172.21.15.71:0/238196801 conn(0x7fc9dc0fc260 msgr2=0x7fc9dc0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:40.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.922+0000 7fc9e2fb1700 1 -- 172.21.15.71:0/238196801 shutdown_connections 2024-04-23T02:10:40.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.922+0000 7fc9e2fb1700 1 -- 172.21.15.71:0/238196801 wait complete. 2024-04-23T02:10:40.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.923+0000 7fc9e2fb1700 1 Processor -- start 2024-04-23T02:10:40.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.923+0000 7fc9e2fb1700 1 -- start start 2024-04-23T02:10:40.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.923+0000 7fc9e2fb1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9dc100ca0 0x7fc9dc196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:40.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.923+0000 7fc9e2fb1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9dc101ea0 0x7fc9dc196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:40.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.923+0000 7fc9e2fb1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc197060 con 0x7fc9dc101ea0 2024-04-23T02:10:40.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.923+0000 7fc9e2fb1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc1971a0 con 0x7fc9dc100ca0 2024-04-23T02:10:40.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.923+0000 7fc9dbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9dc101ea0 0x7fc9dc196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:40.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.923+0000 7fc9dbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9dc101ea0 0x7fc9dc196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51698/0 (socket says 172.21.15.71:51698) 2024-04-23T02:10:40.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.923+0000 7fc9dbfff700 1 -- 172.21.15.71:0/4278805547 learned_addr learned my addr 172.21.15.71:0/4278805547 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:40.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.923+0000 7fc9dbfff700 1 -- 172.21.15.71:0/4278805547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9dc100ca0 msgr2=0x7fc9dc196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:40.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.923+0000 7fc9dbfff700 1 --2- 172.21.15.71:0/4278805547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9dc100ca0 0x7fc9dc196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:40.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.923+0000 7fc9dbfff700 1 -- 172.21.15.71:0/4278805547 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d00095f0 con 0x7fc9dc101ea0 2024-04-23T02:10:40.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.923+0000 7fc9dbfff700 1 --2- 172.21.15.71:0/4278805547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9dc101ea0 0x7fc9dc196a40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800b4b0 tx=0x7fc9c800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:40.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.924+0000 7fc9d9ffb700 1 -- 172.21.15.71:0/4278805547 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9c8011800 con 0x7fc9dc101ea0 2024-04-23T02:10:40.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.924+0000 7fc9d9ffb700 1 -- 172.21.15.71:0/4278805547 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc9c8011df0 con 0x7fc9dc101ea0 2024-04-23T02:10:40.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.924+0000 7fc9e2fb1700 1 -- 172.21.15.71:0/4278805547 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9dc19bc50 con 0x7fc9dc101ea0 2024-04-23T02:10:40.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.924+0000 7fc9d9ffb700 1 -- 172.21.15.71:0/4278805547 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9c8010e40 con 0x7fc9dc101ea0 2024-04-23T02:10:40.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.924+0000 7fc9e2fb1700 1 -- 172.21.15.71:0/4278805547 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9dc19c1a0 con 0x7fc9dc101ea0 2024-04-23T02:10:40.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.925+0000 7fc9e2fb1700 1 -- 172.21.15.71:0/4278805547 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9dc109740 con 0x7fc9dc101ea0 2024-04-23T02:10:40.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.926+0000 7fc9d9ffb700 1 -- 172.21.15.71:0/4278805547 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fc9c8010380 con 0x7fc9dc101ea0 2024-04-23T02:10:40.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.926+0000 7fc9d9ffb700 1 --2- 172.21.15.71:0/4278805547 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc9cc06c490 0x7fc9cc06e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:40.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.926+0000 7fc9d9ffb700 1 -- 172.21.15.71:0/4278805547 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc9c808b6d0 con 0x7fc9dc101ea0 2024-04-23T02:10:40.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.926+0000 7fc9e0d4d700 1 --2- 172.21.15.71:0/4278805547 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc9cc06c490 0x7fc9cc06e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:40.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.927+0000 7fc9e0d4d700 1 --2- 172.21.15.71:0/4278805547 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc9cc06c490 0x7fc9cc06e950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000b5c0 tx=0x7fc9d0005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:40.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:40.931+0000 7fc9d9ffb700 1 -- 172.21.15.71:0/4278805547 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc9c8059c60 con 0x7fc9dc101ea0 2024-04-23T02:10:41.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:41.125+0000 7fc9e2fb1700 1 -- 172.21.15.71:0/4278805547 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9dc1067f0 con 0x7fc9cc06c490 2024-04-23T02:10:41.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:41.130+0000 7fc9d9ffb700 1 -- 172.21.15.71:0/4278805547 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1300 (secure 0 0 0) 0x7fc9dc1067f0 con 0x7fc9cc06c490 2024-04-23T02:10:41.134 INFO:teuthology.orchestra.run.smithi071.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-23T02:10:41.134 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-04-23T02:10:41.134 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter 2/2 2m ago 6m * 2024-04-23T02:10:41.134 INFO:teuthology.orchestra.run.smithi071.stdout:crash 2/2 2m ago 6m * 2024-04-23T02:10:41.134 INFO:teuthology.orchestra.run.smithi071.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-04-23T02:10:41.134 INFO:teuthology.orchestra.run.smithi071.stdout:mgr 2/2 2m ago 6m count:2 2024-04-23T02:10:41.134 INFO:teuthology.orchestra.run.smithi071.stdout:mon 2/2 2m ago 5m smithi071:172.21.15.71=smithi071;smithi192:172.21.15.192=smithi192;count:2 2024-04-23T02:10:41.134 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-04-23T02:10:41.134 INFO:teuthology.orchestra.run.smithi071.stdout:osd 6 2m ago - 2024-04-23T02:10:41.134 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-04-23T02:10:41.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:41.132+0000 7fc9e2fb1700 1 -- 172.21.15.71:0/4278805547 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc9cc06c490 msgr2=0x7fc9cc06e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:41.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:41.132+0000 7fc9e2fb1700 1 --2- 172.21.15.71:0/4278805547 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc9cc06c490 0x7fc9cc06e950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000b5c0 tx=0x7fc9d0005960 comp rx=0 tx=0).stop 2024-04-23T02:10:41.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:41.132+0000 7fc9e2fb1700 1 -- 172.21.15.71:0/4278805547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9dc101ea0 msgr2=0x7fc9dc196a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:41.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:41.132+0000 7fc9e2fb1700 1 --2- 172.21.15.71:0/4278805547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9dc101ea0 0x7fc9dc196a40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800b4b0 tx=0x7fc9c800b870 comp rx=0 tx=0).stop 2024-04-23T02:10:41.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:41.132+0000 7fc9e2fb1700 1 -- 172.21.15.71:0/4278805547 shutdown_connections 2024-04-23T02:10:41.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:41.132+0000 7fc9e2fb1700 1 --2- 172.21.15.71:0/4278805547 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc9cc06c490 0x7fc9cc06e950 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:41.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:41.132+0000 7fc9e2fb1700 1 --2- 172.21.15.71:0/4278805547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9dc100ca0 0x7fc9dc196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:41.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:41.132+0000 7fc9e2fb1700 1 --2- 172.21.15.71:0/4278805547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9dc101ea0 0x7fc9dc196a40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:41.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:41.132+0000 7fc9e2fb1700 1 -- 172.21.15.71:0/4278805547 >> 172.21.15.71:0/4278805547 conn(0x7fc9dc0fc260 msgr2=0x7fc9dc1050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:41.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:41.132+0000 7fc9e2fb1700 1 -- 172.21.15.71:0/4278805547 shutdown_connections 2024-04-23T02:10:41.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:41.132+0000 7fc9e2fb1700 1 -- 172.21.15.71:0/4278805547 wait complete. 2024-04-23T02:10:41.490 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:41 smithi071 ceph-mon[39837]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:41.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:41 smithi192 ceph-mon[49662]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:41.774 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-23T02:10:41.996 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:10:42.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:42 smithi071 ceph-mon[39837]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:10:42.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:10:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:42 smithi192 ceph-mon[49662]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:10:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:42 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:10:43.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:43 smithi071 ceph-mon[39837]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:43.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:43 smithi192 ceph-mon[49662]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.827+0000 7f6c92175700 1 -- 172.21.15.71:0/3053624977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c8c0ffdf0 msgr2=0x7f6c8c100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.827+0000 7f6c92175700 1 --2- 172.21.15.71:0/3053624977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c8c0ffdf0 0x7f6c8c100270 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009910 tx=0x7f6c7c009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.827+0000 7f6c92175700 1 -- 172.21.15.71:0/3053624977 shutdown_connections 2024-04-23T02:10:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.827+0000 7f6c92175700 1 --2- 172.21.15.71:0/3053624977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c8c0ffdf0 0x7f6c8c100270 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.827+0000 7f6c92175700 1 --2- 172.21.15.71:0/3053624977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c8c101220 0x7f6c8c101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.827+0000 7f6c92175700 1 -- 172.21.15.71:0/3053624977 >> 172.21.15.71:0/3053624977 conn(0x7f6c8c0fa8c0 msgr2=0x7f6c8c0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.827+0000 7f6c92175700 1 -- 172.21.15.71:0/3053624977 shutdown_connections 2024-04-23T02:10:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.827+0000 7f6c92175700 1 -- 172.21.15.71:0/3053624977 wait complete. 2024-04-23T02:10:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.827+0000 7f6c92175700 1 Processor -- start 2024-04-23T02:10:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.828+0000 7f6c92175700 1 -- start start 2024-04-23T02:10:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.828+0000 7f6c92175700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c8c0ffdf0 0x7f6c8c198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.828+0000 7f6c92175700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c8c101220 0x7f6c8c198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.828+0000 7f6c92175700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c199160 con 0x7f6c8c101220 2024-04-23T02:10:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.828+0000 7f6c92175700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c1992a0 con 0x7f6c8c0ffdf0 2024-04-23T02:10:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.828+0000 7f6c8affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c8c101220 0x7f6c8c198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.828+0000 7f6c8affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c8c101220 0x7f6c8c198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51716/0 (socket says 172.21.15.71:51716) 2024-04-23T02:10:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.828+0000 7f6c8affd700 1 -- 172.21.15.71:0/2357035568 learned_addr learned my addr 172.21.15.71:0/2357035568 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:43.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.828+0000 7f6c8b7fe700 1 --2- 172.21.15.71:0/2357035568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c8c0ffdf0 0x7f6c8c198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:43.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.828+0000 7f6c8affd700 1 -- 172.21.15.71:0/2357035568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c8c0ffdf0 msgr2=0x7f6c8c198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:43.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.828+0000 7f6c8affd700 1 --2- 172.21.15.71:0/2357035568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c8c0ffdf0 0x7f6c8c198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:43.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.828+0000 7f6c8affd700 1 -- 172.21.15.71:0/2357035568 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c0095f0 con 0x7f6c8c101220 2024-04-23T02:10:43.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.828+0000 7f6c8affd700 1 --2- 172.21.15.71:0/2357035568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c8c101220 0x7f6c8c198b40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00bde0 tx=0x7f6c7c00bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:43.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.828+0000 7f6c88ff9700 1 -- 172.21.15.71:0/2357035568 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c7c01d070 con 0x7f6c8c101220 2024-04-23T02:10:43.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.828+0000 7f6c88ff9700 1 -- 172.21.15.71:0/2357035568 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f6c7c005710 con 0x7f6c8c101220 2024-04-23T02:10:43.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.828+0000 7f6c92175700 1 -- 172.21.15.71:0/2357035568 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8c19dcf0 con 0x7f6c8c101220 2024-04-23T02:10:43.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.828+0000 7f6c88ff9700 1 -- 172.21.15.71:0/2357035568 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c7c003ae0 con 0x7f6c8c101220 2024-04-23T02:10:43.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.829+0000 7f6c92175700 1 -- 172.21.15.71:0/2357035568 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8c19e180 con 0x7f6c8c101220 2024-04-23T02:10:43.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.830+0000 7f6c92175700 1 -- 172.21.15.71:0/2357035568 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8c108680 con 0x7f6c8c101220 2024-04-23T02:10:43.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.830+0000 7f6c88ff9700 1 -- 172.21.15.71:0/2357035568 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f6c7c003d50 con 0x7f6c8c101220 2024-04-23T02:10:43.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.830+0000 7f6c88ff9700 1 --2- 172.21.15.71:0/2357035568 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f6c7406c4e0 0x7f6c7406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:43.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.830+0000 7f6c88ff9700 1 -- 172.21.15.71:0/2357035568 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f6c7c08cc80 con 0x7f6c8c101220 2024-04-23T02:10:43.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.830+0000 7f6c8b7fe700 1 --2- 172.21.15.71:0/2357035568 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f6c7406c4e0 0x7f6c7406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:43.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.831+0000 7f6c8b7fe700 1 --2- 172.21.15.71:0/2357035568 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f6c7406c4e0 0x7f6c7406e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009bf0 tx=0x7f6c80009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:43.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:43.835+0000 7f6c88ff9700 1 -- 172.21.15.71:0/2357035568 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6c7c05b230 con 0x7f6c8c101220 2024-04-23T02:10:44.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:44.029+0000 7f6c92175700 1 -- 172.21.15.71:0/2357035568 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c8c0ff3f0 con 0x7f6c7406c4e0 2024-04-23T02:10:44.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:44.030+0000 7f6c88ff9700 1 -- 172.21.15.71:0/2357035568 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f6c8c0ff3f0 con 0x7f6c7406c4e0 2024-04-23T02:10:44.034 INFO:teuthology.orchestra.run.smithi071.stdout:HOST ADDR LABELS STATUS 2024-04-23T02:10:44.034 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071 172.21.15.71 2024-04-23T02:10:44.034 INFO:teuthology.orchestra.run.smithi071.stdout:smithi192 172.21.15.192 2024-04-23T02:10:44.034 INFO:teuthology.orchestra.run.smithi071.stdout:2 hosts in cluster 2024-04-23T02:10:44.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:44.032+0000 7f6c92175700 1 -- 172.21.15.71:0/2357035568 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f6c7406c4e0 msgr2=0x7f6c7406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:44.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:44.032+0000 7f6c92175700 1 --2- 172.21.15.71:0/2357035568 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f6c7406c4e0 0x7f6c7406e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009bf0 tx=0x7f6c80009260 comp rx=0 tx=0).stop 2024-04-23T02:10:44.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:44.032+0000 7f6c92175700 1 -- 172.21.15.71:0/2357035568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c8c101220 msgr2=0x7f6c8c198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:44.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:44.032+0000 7f6c92175700 1 --2- 172.21.15.71:0/2357035568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c8c101220 0x7f6c8c198b40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00bde0 tx=0x7f6c7c00bec0 comp rx=0 tx=0).stop 2024-04-23T02:10:44.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:44.032+0000 7f6c92175700 1 -- 172.21.15.71:0/2357035568 shutdown_connections 2024-04-23T02:10:44.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:44.032+0000 7f6c92175700 1 --2- 172.21.15.71:0/2357035568 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f6c7406c4e0 0x7f6c7406e9a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:44.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:44.032+0000 7f6c92175700 1 --2- 172.21.15.71:0/2357035568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c8c0ffdf0 0x7f6c8c198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:44.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:44.032+0000 7f6c92175700 1 --2- 172.21.15.71:0/2357035568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c8c101220 0x7f6c8c198b40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:44.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:44.033+0000 7f6c92175700 1 -- 172.21.15.71:0/2357035568 >> 172.21.15.71:0/2357035568 conn(0x7f6c8c0fa8c0 msgr2=0x7f6c8c0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:44.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:44.033+0000 7f6c92175700 1 -- 172.21.15.71:0/2357035568 shutdown_connections 2024-04-23T02:10:44.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:44.033+0000 7f6c92175700 1 -- 172.21.15.71:0/2357035568 wait complete. 2024-04-23T02:10:44.784 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-23T02:10:45.009 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:10:45.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:45 smithi071 ceph-mon[39837]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:10:45.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:45 smithi071 ceph-mon[39837]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:45.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:45 smithi192 ceph-mon[49662]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:10:45.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:45 smithi192 ceph-mon[49662]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:46.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.812+0000 7f11084fb700 1 -- 172.21.15.71:0/2279921821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1100100db0 msgr2=0x7f1100101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:46.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.812+0000 7f11084fb700 1 --2- 172.21.15.71:0/2279921821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1100100db0 0x7f1100101210 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009910 tx=0x7f10f0009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:46.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.812+0000 7f11084fb700 1 -- 172.21.15.71:0/2279921821 shutdown_connections 2024-04-23T02:10:46.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.812+0000 7f11084fb700 1 --2- 172.21.15.71:0/2279921821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1100100db0 0x7f1100101210 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:46.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.812+0000 7f11084fb700 1 --2- 172.21.15.71:0/2279921821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11000ffbb0 0x7f11000fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:46.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.812+0000 7f11084fb700 1 -- 172.21.15.71:0/2279921821 >> 172.21.15.71:0/2279921821 conn(0x7f11000fb150 msgr2=0x7f11000fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:46.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.812+0000 7f11084fb700 1 -- 172.21.15.71:0/2279921821 shutdown_connections 2024-04-23T02:10:46.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.812+0000 7f11084fb700 1 -- 172.21.15.71:0/2279921821 wait complete. 2024-04-23T02:10:46.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.812+0000 7f11084fb700 1 Processor -- start 2024-04-23T02:10:46.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.812+0000 7f11084fb700 1 -- start start 2024-04-23T02:10:46.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.813+0000 7f11084fb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11000ffbb0 0x7f11001953d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:46.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.813+0000 7f11084fb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1100100db0 0x7f1100195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:46.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.813+0000 7f11084fb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1100195f30 con 0x7f11000ffbb0 2024-04-23T02:10:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.813+0000 7f11084fb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1100196070 con 0x7f1100100db0 2024-04-23T02:10:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.813+0000 7f1106297700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11000ffbb0 0x7f11001953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.813+0000 7f1106297700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11000ffbb0 0x7f11001953d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35252/0 (socket says 172.21.15.71:35252) 2024-04-23T02:10:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.813+0000 7f1106297700 1 -- 172.21.15.71:0/1918322998 learned_addr learned my addr 172.21.15.71:0/1918322998 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.813+0000 7f1105a96700 1 --2- 172.21.15.71:0/1918322998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1100100db0 0x7f1100195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.813+0000 7f1106297700 1 -- 172.21.15.71:0/1918322998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1100100db0 msgr2=0x7f1100195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.813+0000 7f1106297700 1 --2- 172.21.15.71:0/1918322998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1100100db0 0x7f1100195910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.813+0000 7f1106297700 1 -- 172.21.15.71:0/1918322998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f00095f0 con 0x7f11000ffbb0 2024-04-23T02:10:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.813+0000 7f1105a96700 1 --2- 172.21.15.71:0/1918322998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1100100db0 0x7f1100195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:10:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.813+0000 7f1106297700 1 --2- 172.21.15.71:0/1918322998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11000ffbb0 0x7f11001953d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00b480 tx=0x7f10fc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.813+0000 7f10f77fe700 1 -- 172.21.15.71:0/1918322998 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10fc0118c0 con 0x7f11000ffbb0 2024-04-23T02:10:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.813+0000 7f10f77fe700 1 -- 172.21.15.71:0/1918322998 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f10fc010380 con 0x7f11000ffbb0 2024-04-23T02:10:46.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.813+0000 7f11084fb700 1 -- 172.21.15.71:0/1918322998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110019ab20 con 0x7f11000ffbb0 2024-04-23T02:10:46.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.813+0000 7f10f77fe700 1 -- 172.21.15.71:0/1918322998 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10fc00f510 con 0x7f11000ffbb0 2024-04-23T02:10:46.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.813+0000 7f11084fb700 1 -- 172.21.15.71:0/1918322998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110019b070 con 0x7f11000ffbb0 2024-04-23T02:10:46.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.814+0000 7f11084fb700 1 -- 172.21.15.71:0/1918322998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1100108670 con 0x7f11000ffbb0 2024-04-23T02:10:46.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.815+0000 7f10f77fe700 1 -- 172.21.15.71:0/1918322998 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f10fc0104f0 con 0x7f11000ffbb0 2024-04-23T02:10:46.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.815+0000 7f10f77fe700 1 --2- 172.21.15.71:0/1918322998 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f10ec06c4e0 0x7f10ec06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:46.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.815+0000 7f10f77fe700 1 -- 172.21.15.71:0/1918322998 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f10fc08b170 con 0x7f11000ffbb0 2024-04-23T02:10:46.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.815+0000 7f1105a96700 1 --2- 172.21.15.71:0/1918322998 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f10ec06c4e0 0x7f10ec06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:46.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.815+0000 7f1105a96700 1 --2- 172.21.15.71:0/1918322998 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f10ec06c4e0 0x7f10ec06e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f10f000b5c0 tx=0x7f10f0005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:46.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:46.819+0000 7f10f77fe700 1 -- 172.21.15.71:0/1918322998 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f10fc059720 con 0x7f11000ffbb0 2024-04-23T02:10:46.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:46 smithi071 ceph-mon[39837]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:47.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:47.014+0000 7f11084fb700 1 -- 172.21.15.71:0/1918322998 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1100105700 con 0x7f10ec06c4e0 2024-04-23T02:10:47.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:47.015+0000 7f10f77fe700 1 -- 172.21.15.71:0/1918322998 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f1100105700 con 0x7f10ec06c4e0 2024-04-23T02:10:47.019 INFO:teuthology.orchestra.run.smithi071.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-23T02:10:47.019 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040172400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-23T02:10:47.019 INFO:teuthology.orchestra.run.smithi071.stdout:smithi192 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500NN400BGN 372G No 75s ago Has a FileSystem, LVM detected 2024-04-23T02:10:47.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:47.017+0000 7f11084fb700 1 -- 172.21.15.71:0/1918322998 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f10ec06c4e0 msgr2=0x7f10ec06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:47.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:47.017+0000 7f11084fb700 1 --2- 172.21.15.71:0/1918322998 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f10ec06c4e0 0x7f10ec06e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f10f000b5c0 tx=0x7f10f0005960 comp rx=0 tx=0).stop 2024-04-23T02:10:47.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:47.017+0000 7f11084fb700 1 -- 172.21.15.71:0/1918322998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11000ffbb0 msgr2=0x7f11001953d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:47.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:47.017+0000 7f11084fb700 1 --2- 172.21.15.71:0/1918322998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11000ffbb0 0x7f11001953d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00b480 tx=0x7f10fc00b840 comp rx=0 tx=0).stop 2024-04-23T02:10:47.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:47.017+0000 7f11084fb700 1 -- 172.21.15.71:0/1918322998 shutdown_connections 2024-04-23T02:10:47.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:47.017+0000 7f11084fb700 1 --2- 172.21.15.71:0/1918322998 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f10ec06c4e0 0x7f10ec06e9a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:47.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:47.017+0000 7f11084fb700 1 --2- 172.21.15.71:0/1918322998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11000ffbb0 0x7f11001953d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:47.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:47.017+0000 7f11084fb700 1 --2- 172.21.15.71:0/1918322998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1100100db0 0x7f1100195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:47.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:47.017+0000 7f11084fb700 1 -- 172.21.15.71:0/1918322998 >> 172.21.15.71:0/1918322998 conn(0x7f11000fb150 msgr2=0x7f1100103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:47.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:47.017+0000 7f11084fb700 1 -- 172.21.15.71:0/1918322998 shutdown_connections 2024-04-23T02:10:47.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:47.017+0000 7f11084fb700 1 -- 172.21.15.71:0/1918322998 wait complete. 2024-04-23T02:10:47.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:46 smithi192 ceph-mon[49662]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:47.728 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T02:10:47.753 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi071.front.sepia.ceph.com 2024-04-23T02:10:47.753 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-04-23T02:10:47.979 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:10:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:47 smithi071 ceph-mon[39837]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:10:48.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:47 smithi192 ceph-mon[49662]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:10:48.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:48 smithi071 ceph-mon[39837]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:49.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:48 smithi192 ceph-mon[49662]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:49.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.829+0000 7fc1d7eb8700 1 -- 172.21.15.71:0/2673564776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1d0101740 msgr2=0x7fc1d0103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:49.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.829+0000 7fc1d7eb8700 1 --2- 172.21.15.71:0/2673564776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1d0101740 0x7fc1d0103b30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009910 tx=0x7fc1cc009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:49.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.829+0000 7fc1d7eb8700 1 -- 172.21.15.71:0/2673564776 shutdown_connections 2024-04-23T02:10:49.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.829+0000 7fc1d7eb8700 1 --2- 172.21.15.71:0/2673564776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1d0104070 0x7fc1d0106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:49.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.829+0000 7fc1d7eb8700 1 --2- 172.21.15.71:0/2673564776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1d0101740 0x7fc1d0103b30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:49.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.829+0000 7fc1d7eb8700 1 -- 172.21.15.71:0/2673564776 >> 172.21.15.71:0/2673564776 conn(0x7fc1d00fb150 msgr2=0x7fc1d00fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:49.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.829+0000 7fc1d7eb8700 1 -- 172.21.15.71:0/2673564776 shutdown_connections 2024-04-23T02:10:49.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.829+0000 7fc1d7eb8700 1 -- 172.21.15.71:0/2673564776 wait complete. 2024-04-23T02:10:49.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.829+0000 7fc1d7eb8700 1 Processor -- start 2024-04-23T02:10:49.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.829+0000 7fc1d7eb8700 1 -- start start 2024-04-23T02:10:49.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.829+0000 7fc1d7eb8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1d0101740 0x7fc1d0196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:49.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.829+0000 7fc1d7eb8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1d0104070 0x7fc1d01969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:49.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.829+0000 7fc1d7eb8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d0196fc0 con 0x7fc1d0101740 2024-04-23T02:10:49.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.829+0000 7fc1d7eb8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d0197100 con 0x7fc1d0104070 2024-04-23T02:10:49.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.830+0000 7fc1d5c54700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1d0101740 0x7fc1d0196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:49.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.830+0000 7fc1d5c54700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1d0101740 0x7fc1d0196460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35260/0 (socket says 172.21.15.71:35260) 2024-04-23T02:10:49.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.830+0000 7fc1d5c54700 1 -- 172.21.15.71:0/3580098548 learned_addr learned my addr 172.21.15.71:0/3580098548 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:49.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.830+0000 7fc1d5453700 1 --2- 172.21.15.71:0/3580098548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1d0104070 0x7fc1d01969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:49.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.830+0000 7fc1d5c54700 1 -- 172.21.15.71:0/3580098548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1d0104070 msgr2=0x7fc1d01969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:49.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.830+0000 7fc1d5c54700 1 --2- 172.21.15.71:0/3580098548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1d0104070 0x7fc1d01969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:49.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.830+0000 7fc1d5c54700 1 -- 172.21.15.71:0/3580098548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc0095f0 con 0x7fc1d0101740 2024-04-23T02:10:49.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.830+0000 7fc1d5c54700 1 --2- 172.21.15.71:0/3580098548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1d0101740 0x7fc1d0196460 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc006010 tx=0x7fc1cc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:49.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.830+0000 7fc1c6ffd700 1 -- 172.21.15.71:0/3580098548 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1cc01d070 con 0x7fc1d0101740 2024-04-23T02:10:49.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.830+0000 7fc1c6ffd700 1 -- 172.21.15.71:0/3580098548 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc1cc022470 con 0x7fc1d0101740 2024-04-23T02:10:49.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.830+0000 7fc1d7eb8700 1 -- 172.21.15.71:0/3580098548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d019bb50 con 0x7fc1d0101740 2024-04-23T02:10:49.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.830+0000 7fc1c6ffd700 1 -- 172.21.15.71:0/3580098548 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1cc00f690 con 0x7fc1d0101740 2024-04-23T02:10:49.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.830+0000 7fc1d7eb8700 1 -- 172.21.15.71:0/3580098548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d019c040 con 0x7fc1d0101740 2024-04-23T02:10:49.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.831+0000 7fc1d7eb8700 1 -- 172.21.15.71:0/3580098548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1d0190560 con 0x7fc1d0101740 2024-04-23T02:10:49.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.832+0000 7fc1c6ffd700 1 -- 172.21.15.71:0/3580098548 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fc1cc022980 con 0x7fc1d0101740 2024-04-23T02:10:49.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.832+0000 7fc1c6ffd700 1 --2- 172.21.15.71:0/3580098548 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc1bc06c4e0 0x7fc1bc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:49.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.833+0000 7fc1d5453700 1 --2- 172.21.15.71:0/3580098548 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc1bc06c4e0 0x7fc1bc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:49.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.833+0000 7fc1c6ffd700 1 -- 172.21.15.71:0/3580098548 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc1cc08d460 con 0x7fc1d0101740 2024-04-23T02:10:49.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.833+0000 7fc1d5453700 1 --2- 172.21.15.71:0/3580098548 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc1bc06c4e0 0x7fc1bc06e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0006fd0 tx=0x7fc1c0008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:49.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:49.836+0000 7fc1c6ffd700 1 -- 172.21.15.71:0/3580098548 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc1cc05ba30 con 0x7fc1d0101740 2024-04-23T02:10:50.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:50.054+0000 7fc1d7eb8700 1 -- 172.21.15.71:0/3580098548 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1d00611d0 con 0x7fc1bc06c4e0 2024-04-23T02:10:50.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:50 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-23T02:10:50.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:50 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-23T02:10:51.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:51.370+0000 7fc1c6ffd700 1 -- 172.21.15.71:0/3580098548 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc1d00611d0 con 0x7fc1bc06c4e0 2024-04-23T02:10:51.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:51.372+0000 7fc1d7eb8700 1 -- 172.21.15.71:0/3580098548 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc1bc06c4e0 msgr2=0x7fc1bc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:51.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:51.372+0000 7fc1d7eb8700 1 --2- 172.21.15.71:0/3580098548 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc1bc06c4e0 0x7fc1bc06e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0006fd0 tx=0x7fc1c0008040 comp rx=0 tx=0).stop 2024-04-23T02:10:51.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:51.372+0000 7fc1d7eb8700 1 -- 172.21.15.71:0/3580098548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1d0101740 msgr2=0x7fc1d0196460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:51.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:51.372+0000 7fc1d7eb8700 1 --2- 172.21.15.71:0/3580098548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1d0101740 0x7fc1d0196460 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc006010 tx=0x7fc1cc00be20 comp rx=0 tx=0).stop 2024-04-23T02:10:51.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:51.372+0000 7fc1d7eb8700 1 -- 172.21.15.71:0/3580098548 shutdown_connections 2024-04-23T02:10:51.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:51.372+0000 7fc1d7eb8700 1 --2- 172.21.15.71:0/3580098548 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fc1bc06c4e0 0x7fc1bc06e9a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:51.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:51.372+0000 7fc1d7eb8700 1 --2- 172.21.15.71:0/3580098548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1d0101740 0x7fc1d0196460 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:51.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:51.372+0000 7fc1d7eb8700 1 --2- 172.21.15.71:0/3580098548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1d0104070 0x7fc1d01969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:51.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:51.372+0000 7fc1d7eb8700 1 -- 172.21.15.71:0/3580098548 >> 172.21.15.71:0/3580098548 conn(0x7fc1d00fb150 msgr2=0x7fc1d00fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:51.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:51.372+0000 7fc1d7eb8700 1 -- 172.21.15.71:0/3580098548 shutdown_connections 2024-04-23T02:10:51.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:51.372+0000 7fc1d7eb8700 1 -- 172.21.15.71:0/3580098548 wait complete. 2024-04-23T02:10:51.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:51 smithi192 ceph-mon[49662]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:10:51.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:51 smithi192 ceph-mon[49662]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:51.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:51 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-23T02:10:51.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:51 smithi192 ceph-mon[49662]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T02:10:51.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:51 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-23T02:10:51.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:51 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[39814]: 2024-04-23T02:10:51.345+0000 7fe2218aa700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T02:10:51.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:51 smithi071 ceph-mon[39837]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:10:51.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:51 smithi071 ceph-mon[39837]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:51.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:51 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-23T02:10:51.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:51 smithi071 ceph-mon[39837]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T02:10:51.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:51 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-23T02:10:51.980 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-23T02:10:52.552 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:52 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-23T02:10:52.552 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:52 smithi071 ceph-mon[39837]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T02:10:52.552 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:52 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-23T02:10:52.552 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:52 smithi071 ceph-mon[39837]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T02:10:52.552 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:52 smithi071 ceph-mon[39837]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T02:10:52.552 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:52 smithi071 ceph-mon[39837]: osdmap e43: 6 total, 6 up, 6 in 2024-04-23T02:10:52.552 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:52 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-23T02:10:52.552 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:52 smithi071 ceph-mon[39837]: fsmap cephfs:0 2024-04-23T02:10:52.552 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:52 smithi071 ceph-mon[39837]: Saving service mds.cephfs spec with placement count:4 2024-04-23T02:10:52.552 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:52 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:52.553 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:52 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:10:52.553 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:52 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:10:52.553 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:52 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:10:52.553 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:52 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:52.553 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:52 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi071.qusrrb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T02:10:52.553 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:52 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi071.qusrrb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T02:10:52.553 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:52 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:10:52.553 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:52 smithi071 ceph-mon[39837]: Deploying daemon mds.cephfs.smithi071.qusrrb on smithi071 2024-04-23T02:10:52.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:52 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-23T02:10:52.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:52 smithi192 ceph-mon[49662]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T02:10:52.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:52 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-23T02:10:52.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:52 smithi192 ceph-mon[49662]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T02:10:52.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:52 smithi192 ceph-mon[49662]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T02:10:52.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:52 smithi192 ceph-mon[49662]: osdmap e43: 6 total, 6 up, 6 in 2024-04-23T02:10:52.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:52 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-23T02:10:52.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:52 smithi192 ceph-mon[49662]: fsmap cephfs:0 2024-04-23T02:10:52.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:52 smithi192 ceph-mon[49662]: Saving service mds.cephfs spec with placement count:4 2024-04-23T02:10:52.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:52 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:52.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:52 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:10:52.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:52 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:10:52.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:52 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:10:52.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:52 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:52.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:52 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi071.qusrrb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T02:10:52.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:52 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi071.qusrrb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T02:10:52.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:52 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:10:52.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:52 smithi192 ceph-mon[49662]: Deploying daemon mds.cephfs.smithi071.qusrrb on smithi071 2024-04-23T02:10:53.308 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:10:53.585 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:53 smithi071 ceph-mon[39837]: pgmap v186: 65 pgs: 20 creating+peering, 4 active+clean, 41 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:53.585 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:53 smithi071 ceph-mon[39837]: osdmap e44: 6 total, 6 up, 6 in 2024-04-23T02:10:53.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:53 smithi192 ceph-mon[49662]: pgmap v186: 65 pgs: 20 creating+peering, 4 active+clean, 41 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:53.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:53 smithi192 ceph-mon[49662]: osdmap e44: 6 total, 6 up, 6 in 2024-04-23T02:10:54.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:54 smithi192 ceph-mon[49662]: osdmap e45: 6 total, 6 up, 6 in 2024-04-23T02:10:54.651 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:54 smithi071 ceph-mon[39837]: osdmap e45: 6 total, 6 up, 6 in 2024-04-23T02:10:55.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.556+0000 7f25e0d71700 1 -- 172.21.15.71:0/38838383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25dc100cb0 msgr2=0x7f25dc1010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:55.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.556+0000 7f25e0d71700 1 --2- 172.21.15.71:0/38838383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25dc100cb0 0x7f25dc1010d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009910 tx=0x7f25cc009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:55.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.556+0000 7f25e0d71700 1 -- 172.21.15.71:0/38838383 shutdown_connections 2024-04-23T02:10:55.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.556+0000 7f25e0d71700 1 --2- 172.21.15.71:0/38838383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25dc101eb0 0x7f25dc102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:55.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.556+0000 7f25e0d71700 1 --2- 172.21.15.71:0/38838383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25dc100cb0 0x7f25dc1010d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:55.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.556+0000 7f25e0d71700 1 -- 172.21.15.71:0/38838383 >> 172.21.15.71:0/38838383 conn(0x7f25dc0fc230 msgr2=0x7f25dc0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:55.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.556+0000 7f25e0d71700 1 -- 172.21.15.71:0/38838383 shutdown_connections 2024-04-23T02:10:55.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.556+0000 7f25e0d71700 1 -- 172.21.15.71:0/38838383 wait complete. 2024-04-23T02:10:55.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.556+0000 7f25e0d71700 1 Processor -- start 2024-04-23T02:10:55.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25e0d71700 1 -- start start 2024-04-23T02:10:55.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25e0d71700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25dc100cb0 0x7f25dc1964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:55.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25e0d71700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25dc101eb0 0x7f25dc196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:55.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25e0d71700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc197040 con 0x7f25dc100cb0 2024-04-23T02:10:55.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25e0d71700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc197180 con 0x7f25dc101eb0 2024-04-23T02:10:55.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25da59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25dc100cb0 0x7f25dc1964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:55.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25da59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25dc100cb0 0x7f25dc1964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59612/0 (socket says 172.21.15.71:59612) 2024-04-23T02:10:55.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25da59c700 1 -- 172.21.15.71:0/2313655750 learned_addr learned my addr 172.21.15.71:0/2313655750 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:55.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25d9d9b700 1 --2- 172.21.15.71:0/2313655750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25dc101eb0 0x7f25dc196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:55.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25da59c700 1 -- 172.21.15.71:0/2313655750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25dc101eb0 msgr2=0x7f25dc196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:55.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25da59c700 1 --2- 172.21.15.71:0/2313655750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25dc101eb0 0x7f25dc196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:55.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25da59c700 1 -- 172.21.15.71:0/2313655750 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc0095f0 con 0x7f25dc100cb0 2024-04-23T02:10:55.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25d9d9b700 1 --2- 172.21.15.71:0/2313655750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25dc101eb0 0x7f25dc196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:10:55.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25da59c700 1 --2- 172.21.15.71:0/2313655750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25dc100cb0 0x7f25dc1964e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00bda0 tx=0x7f25cc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:55.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25cb7fe700 1 -- 172.21.15.71:0/2313655750 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25cc01d070 con 0x7f25dc100cb0 2024-04-23T02:10:55.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25cb7fe700 1 -- 172.21.15.71:0/2313655750 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f25cc022470 con 0x7f25dc100cb0 2024-04-23T02:10:55.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25cb7fe700 1 -- 172.21.15.71:0/2313655750 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25cc00f690 con 0x7f25dc100cb0 2024-04-23T02:10:55.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25e0d71700 1 -- 172.21.15.71:0/2313655750 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25dc19bbd0 con 0x7f25dc100cb0 2024-04-23T02:10:55.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.557+0000 7f25e0d71700 1 -- 172.21.15.71:0/2313655750 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25dc19c0c0 con 0x7f25dc100cb0 2024-04-23T02:10:55.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.558+0000 7f25e0d71700 1 -- 172.21.15.71:0/2313655750 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25dc109750 con 0x7f25dc100cb0 2024-04-23T02:10:55.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.559+0000 7f25cb7fe700 1 -- 172.21.15.71:0/2313655750 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f25cc0229c0 con 0x7f25dc100cb0 2024-04-23T02:10:55.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.559+0000 7f25cb7fe700 1 --2- 172.21.15.71:0/2313655750 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f25c406c530 0x7f25c406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:55.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.559+0000 7f25cb7fe700 1 -- 172.21.15.71:0/2313655750 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f25cc08cf60 con 0x7f25dc100cb0 2024-04-23T02:10:55.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.559+0000 7f25d9d9b700 1 --2- 172.21.15.71:0/2313655750 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f25c406c530 0x7f25c406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:55.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.560+0000 7f25d9d9b700 1 --2- 172.21.15.71:0/2313655750 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f25c406c530 0x7f25c406e9f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f25d0007900 tx=0x7f25d0008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:55.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.564+0000 7f25cb7fe700 1 -- 172.21.15.71:0/2313655750 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f25cc05b1e0 con 0x7f25dc100cb0 2024-04-23T02:10:55.640 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:55 smithi192 ceph-mon[49662]: pgmap v189: 65 pgs: 20 creating+peering, 4 active+clean, 41 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:55.640 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:55 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:55.640 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:55 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:55.640 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:55 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:55.640 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:55 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.gtmyst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T02:10:55.640 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:55 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.gtmyst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T02:10:55.640 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:55 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:10:55.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:55 smithi071 ceph-mon[39837]: pgmap v189: 65 pgs: 20 creating+peering, 4 active+clean, 41 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:10:55.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:55 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:55.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:55 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:55.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:55 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:55.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:55 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.gtmyst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T02:10:55.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:55 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.gtmyst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T02:10:55.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:55 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:10:55.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.794+0000 7f25e0d71700 1 -- 172.21.15.71:0/2313655750 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f25dc109940 con 0x7f25dc100cb0 2024-04-23T02:10:55.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.795+0000 7f25cb7fe700 1 -- 172.21.15.71:0/2313655750 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 4 v4) v1 ==== 75+0+1264 (secure 0 0 0) 0x7f25cc027500 con 0x7f25dc100cb0 2024-04-23T02:10:55.799 INFO:teuthology.orchestra.run.smithi071.stdout:e4 2024-04-23T02:10:55.799 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:10:55.799 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:10:55.799 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:10:55.799 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:10:55.799 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:10:55.799 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:10:55.799 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 4 2024-04-23T02:10:55.799 INFO:teuthology.orchestra.run.smithi071.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-23T02:10:55.799 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:10:55.800 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:10:55.374682+0000 2024-04-23T02:10:55.800 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:10:55.800 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:10:55.800 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:10:55.800 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:10:55.800 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:10:55.800 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:10:55.800 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:10:55.800 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 0 2024-04-23T02:10:55.800 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:10:55.800 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 1 2024-04-23T02:10:55.800 INFO:teuthology.orchestra.run.smithi071.stdout:in 0 2024-04-23T02:10:55.800 INFO:teuthology.orchestra.run.smithi071.stdout:up {0=14462} 2024-04-23T02:10:55.800 INFO:teuthology.orchestra.run.smithi071.stdout:failed 2024-04-23T02:10:55.800 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:10:55.801 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:10:55.801 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:10:55.801 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:10:55.801 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data disabled 2024-04-23T02:10:55.801 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:10:55.801 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:10:55.801 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 0 2024-04-23T02:10:55.801 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{0:14462} state up:creating seq 1 addr [v2:172.21.15.71:6826/3504670324,v1:172.21.15.71:6827/3504670324] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:10:55.801 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:10:55.801 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:10:55.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.797+0000 7f25e0d71700 1 -- 172.21.15.71:0/2313655750 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f25c406c530 msgr2=0x7f25c406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:55.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.797+0000 7f25e0d71700 1 --2- 172.21.15.71:0/2313655750 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f25c406c530 0x7f25c406e9f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f25d0007900 tx=0x7f25d0008040 comp rx=0 tx=0).stop 2024-04-23T02:10:55.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.797+0000 7f25e0d71700 1 -- 172.21.15.71:0/2313655750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25dc100cb0 msgr2=0x7f25dc1964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:55.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.797+0000 7f25e0d71700 1 --2- 172.21.15.71:0/2313655750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25dc100cb0 0x7f25dc1964e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00bda0 tx=0x7f25cc00be80 comp rx=0 tx=0).stop 2024-04-23T02:10:55.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.797+0000 7f25e0d71700 1 -- 172.21.15.71:0/2313655750 shutdown_connections 2024-04-23T02:10:55.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.797+0000 7f25e0d71700 1 --2- 172.21.15.71:0/2313655750 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f25c406c530 0x7f25c406e9f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:55.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.797+0000 7f25e0d71700 1 --2- 172.21.15.71:0/2313655750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25dc100cb0 0x7f25dc1964e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:55.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.797+0000 7f25e0d71700 1 --2- 172.21.15.71:0/2313655750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25dc101eb0 0x7f25dc196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:55.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.797+0000 7f25e0d71700 1 -- 172.21.15.71:0/2313655750 >> 172.21.15.71:0/2313655750 conn(0x7f25dc0fc230 msgr2=0x7f25dc1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:55.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.797+0000 7f25e0d71700 1 -- 172.21.15.71:0/2313655750 shutdown_connections 2024-04-23T02:10:55.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:55.797+0000 7f25e0d71700 1 -- 172.21.15.71:0/2313655750 wait complete. 2024-04-23T02:10:55.803 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 4 2024-04-23T02:10:56.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:56 smithi071 ceph-mon[39837]: Deploying daemon mds.cephfs.smithi192.gtmyst on smithi192 2024-04-23T02:10:56.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:56 smithi071 ceph-mon[39837]: mds.? [v2:172.21.15.71:6826/3504670324,v1:172.21.15.71:6827/3504670324] up:boot 2024-04-23T02:10:56.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:56 smithi071 ceph-mon[39837]: daemon mds.cephfs.smithi071.qusrrb assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-23T02:10:56.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:56 smithi071 ceph-mon[39837]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T02:10:56.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:56 smithi071 ceph-mon[39837]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T02:10:56.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:56 smithi071 ceph-mon[39837]: Cluster is now healthy 2024-04-23T02:10:56.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:56 smithi071 ceph-mon[39837]: fsmap cephfs:0 1 up:standby 2024-04-23T02:10:56.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:56 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.qusrrb"}]: dispatch 2024-04-23T02:10:56.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:56 smithi071 ceph-mon[39837]: fsmap cephfs:1 {0=cephfs.smithi071.qusrrb=up:creating} 2024-04-23T02:10:56.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:56 smithi071 ceph-mon[39837]: daemon mds.cephfs.smithi071.qusrrb is now active in filesystem cephfs as rank 0 2024-04-23T02:10:56.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:56 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/2313655750' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:10:56.548 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T02:10:56.557 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi071.front.sepia.ceph.com 2024-04-23T02:10:56.557 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-04-23T02:10:56.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:56 smithi192 ceph-mon[49662]: Deploying daemon mds.cephfs.smithi192.gtmyst on smithi192 2024-04-23T02:10:56.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:56 smithi192 ceph-mon[49662]: mds.? [v2:172.21.15.71:6826/3504670324,v1:172.21.15.71:6827/3504670324] up:boot 2024-04-23T02:10:56.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:56 smithi192 ceph-mon[49662]: daemon mds.cephfs.smithi071.qusrrb assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-23T02:10:56.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:56 smithi192 ceph-mon[49662]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T02:10:56.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:56 smithi192 ceph-mon[49662]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T02:10:56.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:56 smithi192 ceph-mon[49662]: Cluster is now healthy 2024-04-23T02:10:56.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:56 smithi192 ceph-mon[49662]: fsmap cephfs:0 1 up:standby 2024-04-23T02:10:56.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:56 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.qusrrb"}]: dispatch 2024-04-23T02:10:56.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:56 smithi192 ceph-mon[49662]: fsmap cephfs:1 {0=cephfs.smithi071.qusrrb=up:creating} 2024-04-23T02:10:56.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:56 smithi192 ceph-mon[49662]: daemon mds.cephfs.smithi071.qusrrb is now active in filesystem cephfs as rank 0 2024-04-23T02:10:56.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:56 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/2313655750' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:10:56.785 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:10:57.683 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:57 smithi192 ceph-mon[49662]: pgmap v190: 65 pgs: 8 creating+peering, 53 active+clean, 4 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 425 B/s wr, 1 op/s 2024-04-23T02:10:57.683 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:57 smithi192 ceph-mon[49662]: mds.? [v2:172.21.15.71:6826/3504670324,v1:172.21.15.71:6827/3504670324] up:active 2024-04-23T02:10:57.683 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:57 smithi192 ceph-mon[49662]: fsmap cephfs:1 {0=cephfs.smithi071.qusrrb=up:active} 2024-04-23T02:10:57.683 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:10:57.737 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:57 smithi071 ceph-mon[39837]: pgmap v190: 65 pgs: 8 creating+peering, 53 active+clean, 4 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 425 B/s wr, 1 op/s 2024-04-23T02:10:57.737 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:57 smithi071 ceph-mon[39837]: mds.? [v2:172.21.15.71:6826/3504670324,v1:172.21.15.71:6827/3504670324] up:active 2024-04-23T02:10:57.737 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:57 smithi071 ceph-mon[39837]: fsmap cephfs:1 {0=cephfs.smithi071.qusrrb=up:active} 2024-04-23T02:10:57.737 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:10:58.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:58 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:58.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:58 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:58.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:58 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:58.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:58 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi071.vxejye", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T02:10:58.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:58 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi071.vxejye", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T02:10:58.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:58 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:10:58.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:58 smithi071 ceph-mon[39837]: Deploying daemon mds.cephfs.smithi071.vxejye on smithi071 2024-04-23T02:10:58.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:58 smithi071 ceph-mon[39837]: pgmap v191: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 452 B/s wr, 3 op/s 2024-04-23T02:10:58.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:58 smithi071 ceph-mon[39837]: mds.? [v2:172.21.15.192:6824/3780549896,v1:172.21.15.192:6825/3780549896] up:boot 2024-04-23T02:10:58.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:58 smithi071 ceph-mon[39837]: fsmap cephfs:1 {0=cephfs.smithi071.qusrrb=up:active} 1 up:standby 2024-04-23T02:10:58.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:58 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.gtmyst"}]: dispatch 2024-04-23T02:10:58.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:58 smithi071 ceph-mon[39837]: fsmap cephfs:1 {0=cephfs.smithi071.qusrrb=up:active} 1 up:standby 2024-04-23T02:10:59.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:58 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:59.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:58 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:59.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:58 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:10:59.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:58 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi071.vxejye", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T02:10:59.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:58 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi071.vxejye", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T02:10:59.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:58 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:10:59.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:58 smithi192 ceph-mon[49662]: Deploying daemon mds.cephfs.smithi071.vxejye on smithi071 2024-04-23T02:10:59.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:58 smithi192 ceph-mon[49662]: pgmap v191: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 452 B/s wr, 3 op/s 2024-04-23T02:10:59.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:58 smithi192 ceph-mon[49662]: mds.? [v2:172.21.15.192:6824/3780549896,v1:172.21.15.192:6825/3780549896] up:boot 2024-04-23T02:10:59.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:58 smithi192 ceph-mon[49662]: fsmap cephfs:1 {0=cephfs.smithi071.qusrrb=up:active} 1 up:standby 2024-04-23T02:10:59.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:58 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.gtmyst"}]: dispatch 2024-04-23T02:10:59.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:58 smithi192 ceph-mon[49662]: fsmap cephfs:1 {0=cephfs.smithi071.qusrrb=up:active} 1 up:standby 2024-04-23T02:10:59.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.819+0000 7f287dd22700 1 -- 172.21.15.71:0/2428766282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2878100da0 msgr2=0x7f2878101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:59.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.819+0000 7f287dd22700 1 --2- 172.21.15.71:0/2428766282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2878100da0 0x7f2878101200 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2868009910 tx=0x7f2868009c20 comp rx=0 tx=0).stop 2024-04-23T02:10:59.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.819+0000 7f287dd22700 1 -- 172.21.15.71:0/2428766282 shutdown_connections 2024-04-23T02:10:59.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.819+0000 7f287dd22700 1 --2- 172.21.15.71:0/2428766282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2878100da0 0x7f2878101200 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.819+0000 7f287dd22700 1 --2- 172.21.15.71:0/2428766282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28780ffba0 0x7f28780fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.819+0000 7f287dd22700 1 -- 172.21.15.71:0/2428766282 >> 172.21.15.71:0/2428766282 conn(0x7f28780fb120 msgr2=0x7f28780fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.819+0000 7f287dd22700 1 -- 172.21.15.71:0/2428766282 shutdown_connections 2024-04-23T02:10:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.819+0000 7f287dd22700 1 -- 172.21.15.71:0/2428766282 wait complete. 2024-04-23T02:10:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.819+0000 7f287dd22700 1 Processor -- start 2024-04-23T02:10:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.819+0000 7f287dd22700 1 -- start start 2024-04-23T02:10:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.820+0000 7f287dd22700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28780ffba0 0x7f28781964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.820+0000 7f287dd22700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2878100da0 0x7f2878196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.820+0000 7f287dd22700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2878197050 con 0x7f28780ffba0 2024-04-23T02:10:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.820+0000 7f287dd22700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2878197190 con 0x7f2878100da0 2024-04-23T02:10:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.820+0000 7f28777fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28780ffba0 0x7f28781964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.820+0000 7f28777fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28780ffba0 0x7f28781964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59630/0 (socket says 172.21.15.71:59630) 2024-04-23T02:10:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.820+0000 7f28777fe700 1 -- 172.21.15.71:0/3201876019 learned_addr learned my addr 172.21.15.71:0/3201876019 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:10:59.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.820+0000 7f2876ffd700 1 --2- 172.21.15.71:0/3201876019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2878100da0 0x7f2878196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.820+0000 7f28777fe700 1 -- 172.21.15.71:0/3201876019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2878100da0 msgr2=0x7f2878196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.820+0000 7f28777fe700 1 --2- 172.21.15.71:0/3201876019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2878100da0 0x7f2878196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.820+0000 7f28777fe700 1 -- 172.21.15.71:0/3201876019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28680095f0 con 0x7f28780ffba0 2024-04-23T02:10:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.820+0000 7f2876ffd700 1 --2- 172.21.15.71:0/3201876019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2878100da0 0x7f2878196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:10:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.820+0000 7f28777fe700 1 --2- 172.21.15.71:0/3201876019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28780ffba0 0x7f28781964f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f286c00b480 tx=0x7f286c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.820+0000 7f2874ff9700 1 -- 172.21.15.71:0/3201876019 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f286c0118c0 con 0x7f28780ffba0 2024-04-23T02:10:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.820+0000 7f2874ff9700 1 -- 172.21.15.71:0/3201876019 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f286c010380 con 0x7f28780ffba0 2024-04-23T02:10:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.820+0000 7f287dd22700 1 -- 172.21.15.71:0/3201876019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287819bc40 con 0x7f28780ffba0 2024-04-23T02:10:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.820+0000 7f2874ff9700 1 -- 172.21.15.71:0/3201876019 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f286c00f510 con 0x7f28780ffba0 2024-04-23T02:10:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.820+0000 7f287dd22700 1 -- 172.21.15.71:0/3201876019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287819c190 con 0x7f28780ffba0 2024-04-23T02:10:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.821+0000 7f287dd22700 1 -- 172.21.15.71:0/3201876019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2878108660 con 0x7f28780ffba0 2024-04-23T02:10:59.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.822+0000 7f2874ff9700 1 -- 172.21.15.71:0/3201876019 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f286c0104f0 con 0x7f28780ffba0 2024-04-23T02:10:59.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.823+0000 7f2874ff9700 1 --2- 172.21.15.71:0/3201876019 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f286006c4e0 0x7f286006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:10:59.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.823+0000 7f2874ff9700 1 -- 172.21.15.71:0/3201876019 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f286c08b4b0 con 0x7f28780ffba0 2024-04-23T02:10:59.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.823+0000 7f2876ffd700 1 --2- 172.21.15.71:0/3201876019 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f286006c4e0 0x7f286006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:10:59.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.823+0000 7f2876ffd700 1 --2- 172.21.15.71:0/3201876019 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f286006c4e0 0x7f286006e9a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f28680059d0 tx=0x7f2868005920 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:10:59.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:58.827+0000 7f2874ff9700 1 -- 172.21.15.71:0/3201876019 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f286c059720 con 0x7f28780ffba0 2024-04-23T02:10:59.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:59.061+0000 7f287dd22700 1 -- 172.21.15.71:0/3201876019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f287819c7a0 con 0x7f28780ffba0 2024-04-23T02:10:59.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:59.782+0000 7f2874ff9700 1 -- 172.21.15.71:0/3201876019 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v8) v1 ==== 105+0+0 (secure 0 0 0) 0x7f286c0107a0 con 0x7f28780ffba0 2024-04-23T02:10:59.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:59.784+0000 7f287dd22700 1 -- 172.21.15.71:0/3201876019 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f286006c4e0 msgr2=0x7f286006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:59.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:59.784+0000 7f287dd22700 1 --2- 172.21.15.71:0/3201876019 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f286006c4e0 0x7f286006e9a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f28680059d0 tx=0x7f2868005920 comp rx=0 tx=0).stop 2024-04-23T02:10:59.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:59.784+0000 7f287dd22700 1 -- 172.21.15.71:0/3201876019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28780ffba0 msgr2=0x7f28781964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:10:59.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:59.784+0000 7f287dd22700 1 --2- 172.21.15.71:0/3201876019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28780ffba0 0x7f28781964f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f286c00b480 tx=0x7f286c00b840 comp rx=0 tx=0).stop 2024-04-23T02:10:59.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:59.784+0000 7f287dd22700 1 -- 172.21.15.71:0/3201876019 shutdown_connections 2024-04-23T02:10:59.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:59.784+0000 7f287dd22700 1 --2- 172.21.15.71:0/3201876019 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f286006c4e0 0x7f286006e9a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:59.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:59.784+0000 7f287dd22700 1 --2- 172.21.15.71:0/3201876019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28780ffba0 0x7f28781964f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:59.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:59.784+0000 7f287dd22700 1 --2- 172.21.15.71:0/3201876019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2878100da0 0x7f2878196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:10:59.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:59.784+0000 7f287dd22700 1 -- 172.21.15.71:0/3201876019 >> 172.21.15.71:0/3201876019 conn(0x7f28780fb120 msgr2=0x7f2878068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:10:59.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:59.784+0000 7f287dd22700 1 -- 172.21.15.71:0/3201876019 shutdown_connections 2024-04-23T02:10:59.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:10:59.784+0000 7f287dd22700 1 -- 172.21.15.71:0/3201876019 wait complete. 2024-04-23T02:10:59.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:10:59 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/3201876019' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-04-23T02:10:59.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:10:59 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3201876019' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-04-23T02:11:00.560 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T02:11:00.570 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi071.front.sepia.ceph.com 2024-04-23T02:11:00.571 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-04-23T02:11:01.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:00 smithi192 ceph-mon[49662]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T02:11:01.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:00 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/3201876019' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-04-23T02:11:01.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:00 smithi192 ceph-mon[49662]: daemon mds.cephfs.smithi192.gtmyst assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T02:11:01.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:00 smithi192 ceph-mon[49662]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-23T02:11:01.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:00 smithi192 ceph-mon[49662]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T02:11:01.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:00 smithi192 ceph-mon[49662]: fsmap cephfs:1 {0=cephfs.smithi071.qusrrb=up:active} 1 up:standby 2024-04-23T02:11:01.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:00 smithi192 ceph-mon[49662]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi192.gtmyst=up:creating} 2024-04-23T02:11:01.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:00 smithi192 ceph-mon[49662]: daemon mds.cephfs.smithi192.gtmyst is now active in filesystem cephfs as rank 1 2024-04-23T02:11:01.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:00 smithi192 ceph-mon[49662]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-23T02:11:01.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:00 smithi071 ceph-mon[39837]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T02:11:01.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:00 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3201876019' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-04-23T02:11:01.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:00 smithi071 ceph-mon[39837]: daemon mds.cephfs.smithi192.gtmyst assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T02:11:01.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:00 smithi071 ceph-mon[39837]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-23T02:11:01.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:00 smithi071 ceph-mon[39837]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T02:11:01.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:00 smithi071 ceph-mon[39837]: fsmap cephfs:1 {0=cephfs.smithi071.qusrrb=up:active} 1 up:standby 2024-04-23T02:11:01.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:00 smithi071 ceph-mon[39837]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi192.gtmyst=up:creating} 2024-04-23T02:11:01.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:00 smithi071 ceph-mon[39837]: daemon mds.cephfs.smithi192.gtmyst is now active in filesystem cephfs as rank 1 2024-04-23T02:11:01.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:00 smithi071 ceph-mon[39837]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-23T02:11:01.752 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:11:01.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:01 smithi071 ceph-mon[39837]: mds.? [v2:172.21.15.192:6824/3780549896,v1:172.21.15.192:6825/3780549896] up:active 2024-04-23T02:11:01.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:01 smithi071 ceph-mon[39837]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi192.gtmyst=up:active} 2024-04-23T02:11:01.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:01 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:01.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:01 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:02.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:01 smithi192 ceph-mon[49662]: mds.? [v2:172.21.15.192:6824/3780549896,v1:172.21.15.192:6825/3780549896] up:active 2024-04-23T02:11:02.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:01 smithi192 ceph-mon[49662]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi192.gtmyst=up:active} 2024-04-23T02:11:02.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:01 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:02.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:01 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:02.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:02 smithi071 ceph-mon[39837]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-23T02:11:02.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:02 smithi071 ceph-mon[39837]: Cluster is now healthy 2024-04-23T02:11:02.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:02 smithi071 ceph-mon[39837]: mds.? [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] up:boot 2024-04-23T02:11:02.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:02 smithi071 ceph-mon[39837]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi192.gtmyst=up:active} 1 up:standby 2024-04-23T02:11:02.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:02 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.vxejye"}]: dispatch 2024-04-23T02:11:02.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:02 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:02.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:02 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.azdyze", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T02:11:02.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:02 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.azdyze", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T02:11:02.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:02 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:11:02.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:02 smithi071 ceph-mon[39837]: Deploying daemon mds.cephfs.smithi192.azdyze on smithi192 2024-04-23T02:11:02.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:02 smithi071 ceph-mon[39837]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s wr, 6 op/s 2024-04-23T02:11:03.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:02 smithi192 ceph-mon[49662]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-23T02:11:03.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:02 smithi192 ceph-mon[49662]: Cluster is now healthy 2024-04-23T02:11:03.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:02 smithi192 ceph-mon[49662]: mds.? [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] up:boot 2024-04-23T02:11:03.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:02 smithi192 ceph-mon[49662]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi192.gtmyst=up:active} 1 up:standby 2024-04-23T02:11:03.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:02 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.vxejye"}]: dispatch 2024-04-23T02:11:03.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:02 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:03.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:02 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.azdyze", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T02:11:03.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:02 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.azdyze", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T02:11:03.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:02 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:11:03.039 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:02 smithi192 ceph-mon[49662]: Deploying daemon mds.cephfs.smithi192.azdyze on smithi192 2024-04-23T02:11:03.040 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:02 smithi192 ceph-mon[49662]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s wr, 6 op/s 2024-04-23T02:11:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.585+0000 7fb02eb15700 1 -- 172.21.15.71:0/968657588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb028100e10 msgr2=0x7fb028101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.585+0000 7fb02eb15700 1 --2- 172.21.15.71:0/968657588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb028100e10 0x7fb028101270 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb014009910 tx=0x7fb014009c20 comp rx=0 tx=0).stop 2024-04-23T02:11:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.585+0000 7fb02eb15700 1 -- 172.21.15.71:0/968657588 shutdown_connections 2024-04-23T02:11:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.585+0000 7fb02eb15700 1 --2- 172.21.15.71:0/968657588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb028100e10 0x7fb028101270 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.585+0000 7fb02eb15700 1 --2- 172.21.15.71:0/968657588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0280ffc10 0x7fb028100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.585+0000 7fb02eb15700 1 -- 172.21.15.71:0/968657588 >> 172.21.15.71:0/968657588 conn(0x7fb0280fb190 msgr2=0x7fb0280fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.585+0000 7fb02eb15700 1 -- 172.21.15.71:0/968657588 shutdown_connections 2024-04-23T02:11:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.585+0000 7fb02eb15700 1 -- 172.21.15.71:0/968657588 wait complete. 2024-04-23T02:11:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.585+0000 7fb02eb15700 1 Processor -- start 2024-04-23T02:11:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.585+0000 7fb02eb15700 1 -- start start 2024-04-23T02:11:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.585+0000 7fb02eb15700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0280ffc10 0x7fb0281963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.586+0000 7fb02eb15700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb028100e10 0x7fb028196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.586+0000 7fb02eb15700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb028196f20 con 0x7fb028100e10 2024-04-23T02:11:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.586+0000 7fb02eb15700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb028197060 con 0x7fb0280ffc10 2024-04-23T02:11:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.586+0000 7fb027fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb028100e10 0x7fb028196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.586+0000 7fb027fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb028100e10 0x7fb028196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59660/0 (socket says 172.21.15.71:59660) 2024-04-23T02:11:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.586+0000 7fb027fff700 1 -- 172.21.15.71:0/3206749324 learned_addr learned my addr 172.21.15.71:0/3206749324 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:11:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.586+0000 7fb02c8b1700 1 --2- 172.21.15.71:0/3206749324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0280ffc10 0x7fb0281963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.586+0000 7fb027fff700 1 -- 172.21.15.71:0/3206749324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0280ffc10 msgr2=0x7fb0281963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.586+0000 7fb027fff700 1 --2- 172.21.15.71:0/3206749324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0280ffc10 0x7fb0281963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.586+0000 7fb027fff700 1 -- 172.21.15.71:0/3206749324 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0140095f0 con 0x7fb028100e10 2024-04-23T02:11:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.586+0000 7fb02c8b1700 1 --2- 172.21.15.71:0/3206749324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0280ffc10 0x7fb0281963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T02:11:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.586+0000 7fb027fff700 1 --2- 172.21.15.71:0/3206749324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb028100e10 0x7fb028196900 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb01400bde0 tx=0x7fb01400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.586+0000 7fb025ffb700 1 -- 172.21.15.71:0/3206749324 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb01401d070 con 0x7fb028100e10 2024-04-23T02:11:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.586+0000 7fb025ffb700 1 -- 172.21.15.71:0/3206749324 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fb014005dc0 con 0x7fb028100e10 2024-04-23T02:11:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.586+0000 7fb02eb15700 1 -- 172.21.15.71:0/3206749324 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02819bab0 con 0x7fb028100e10 2024-04-23T02:11:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.586+0000 7fb025ffb700 1 -- 172.21.15.71:0/3206749324 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb014003b10 con 0x7fb028100e10 2024-04-23T02:11:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.587+0000 7fb02eb15700 1 -- 172.21.15.71:0/3206749324 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02819bf40 con 0x7fb028100e10 2024-04-23T02:11:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.587+0000 7fb02eb15700 1 -- 172.21.15.71:0/3206749324 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0281085c0 con 0x7fb028100e10 2024-04-23T02:11:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.588+0000 7fb025ffb700 1 -- 172.21.15.71:0/3206749324 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fb0140048e0 con 0x7fb028100e10 2024-04-23T02:11:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.588+0000 7fb025ffb700 1 --2- 172.21.15.71:0/3206749324 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fb01806c4e0 0x7fb01806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.588+0000 7fb025ffb700 1 -- 172.21.15.71:0/3206749324 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fb01408d020 con 0x7fb028100e10 2024-04-23T02:11:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.588+0000 7fb02c8b1700 1 --2- 172.21.15.71:0/3206749324 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fb01806c4e0 0x7fb01806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:03.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.589+0000 7fb02c8b1700 1 --2- 172.21.15.71:0/3206749324 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fb01806c4e0 0x7fb01806e9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb028100c70 tx=0x7fb01c009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:03.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.593+0000 7fb025ffb700 1 -- 172.21.15.71:0/3206749324 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb0140c00e0 con 0x7fb028100e10 2024-04-23T02:11:03.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.822+0000 7fb02eb15700 1 -- 172.21.15.71:0/3206749324 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fb02819c5e0 con 0x7fb028100e10 2024-04-23T02:11:03.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.876+0000 7fb025ffb700 1 -- 172.21.15.71:0/3206749324 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v12) v1 ==== 121+0+0 (secure 0 0 0) 0x7fb01405b1f0 con 0x7fb028100e10 2024-04-23T02:11:03.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.878+0000 7fb02eb15700 1 -- 172.21.15.71:0/3206749324 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fb01806c4e0 msgr2=0x7fb01806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:03.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.878+0000 7fb02eb15700 1 --2- 172.21.15.71:0/3206749324 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fb01806c4e0 0x7fb01806e9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb028100c70 tx=0x7fb01c009260 comp rx=0 tx=0).stop 2024-04-23T02:11:03.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.878+0000 7fb02eb15700 1 -- 172.21.15.71:0/3206749324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb028100e10 msgr2=0x7fb028196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:03.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.878+0000 7fb02eb15700 1 --2- 172.21.15.71:0/3206749324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb028100e10 0x7fb028196900 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb01400bde0 tx=0x7fb01400be10 comp rx=0 tx=0).stop 2024-04-23T02:11:03.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.879+0000 7fb02eb15700 1 -- 172.21.15.71:0/3206749324 shutdown_connections 2024-04-23T02:11:03.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.879+0000 7fb02eb15700 1 --2- 172.21.15.71:0/3206749324 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fb01806c4e0 0x7fb01806e9a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:03.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.879+0000 7fb02eb15700 1 --2- 172.21.15.71:0/3206749324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0280ffc10 0x7fb0281963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:03.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.879+0000 7fb02eb15700 1 --2- 172.21.15.71:0/3206749324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb028100e10 0x7fb028196900 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:03.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.879+0000 7fb02eb15700 1 -- 172.21.15.71:0/3206749324 >> 172.21.15.71:0/3206749324 conn(0x7fb0280fb190 msgr2=0x7fb028068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:03.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.879+0000 7fb02eb15700 1 -- 172.21.15.71:0/3206749324 shutdown_connections 2024-04-23T02:11:03.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:03.879+0000 7fb02eb15700 1 -- 172.21.15.71:0/3206749324 wait complete. 2024-04-23T02:11:04.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:03 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/3206749324' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T02:11:04.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:03 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3206749324' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T02:11:04.705 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T02:11:04.715 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi071.front.sepia.ceph.com 2024-04-23T02:11:04.715 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-04-23T02:11:04.977 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:11:04.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:04 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3206749324' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T02:11:04.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:04 smithi071 ceph-mon[39837]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi192.gtmyst=up:active} 1 up:standby 2024-04-23T02:11:04.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:04 smithi071 ceph-mon[39837]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi192.gtmyst=up:active} 1 up:standby-replay 2024-04-23T02:11:04.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:04 smithi071 ceph-mon[39837]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s wr, 6 op/s 2024-04-23T02:11:04.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:04 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:04.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:04 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:04.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:04 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:04.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:04 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:04.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:04 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:04.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:04 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:11:05.354 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:04 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/3206749324' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T02:11:05.354 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:04 smithi192 ceph-mon[49662]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi192.gtmyst=up:active} 1 up:standby 2024-04-23T02:11:05.354 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:04 smithi192 ceph-mon[49662]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi192.gtmyst=up:active} 1 up:standby-replay 2024-04-23T02:11:05.354 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:04 smithi192 ceph-mon[49662]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s wr, 6 op/s 2024-04-23T02:11:05.354 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:04 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:05.354 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:04 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:05.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:04 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:05.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:04 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:05.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:04 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:05.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:04 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:11:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:05 smithi071 ceph-mon[39837]: mds.? [v2:172.21.15.192:6826/2100078284,v1:172.21.15.192:6827/2100078284] up:boot 2024-04-23T02:11:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:05 smithi071 ceph-mon[39837]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi192.gtmyst=up:active} 1 up:standby-replay 1 up:standby 2024-04-23T02:11:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:05 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.azdyze"}]: dispatch 2024-04-23T02:11:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:05 smithi071 ceph-mon[39837]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi192.gtmyst=up:active} 2 up:standby-replay 2024-04-23T02:11:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:05 smithi071 ceph-mon[39837]: Dropping low affinity active daemon mds.cephfs.smithi071.qusrrb in favor of higher affinity standby. 2024-04-23T02:11:05.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:05 smithi071 ceph-mon[39837]: Replacing daemon mds.cephfs.smithi071.qusrrb as rank 0 with standby daemon mds.cephfs.smithi071.vxejye 2024-04-23T02:11:05.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:05 smithi071 ceph-mon[39837]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T02:11:05.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:05 smithi071 ceph-mon[39837]: osdmap e46: 6 total, 6 up, 6 in 2024-04-23T02:11:05.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:05 smithi071 ceph-mon[39837]: mds.? [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] up:standby-replay 2024-04-23T02:11:05.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:05 smithi071 ceph-mon[39837]: fsmap cephfs:2/2 {0=cephfs.smithi071.vxejye=up:replay,1=cephfs.smithi192.gtmyst=up:active} 1 up:standby-replay 2024-04-23T02:11:06.260 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:05 smithi192 ceph-mon[49662]: mds.? [v2:172.21.15.192:6826/2100078284,v1:172.21.15.192:6827/2100078284] up:boot 2024-04-23T02:11:06.260 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:05 smithi192 ceph-mon[49662]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi192.gtmyst=up:active} 1 up:standby-replay 1 up:standby 2024-04-23T02:11:06.260 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:05 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.azdyze"}]: dispatch 2024-04-23T02:11:06.261 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:05 smithi192 ceph-mon[49662]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi192.gtmyst=up:active} 2 up:standby-replay 2024-04-23T02:11:06.261 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:05 smithi192 ceph-mon[49662]: Dropping low affinity active daemon mds.cephfs.smithi071.qusrrb in favor of higher affinity standby. 2024-04-23T02:11:06.261 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:05 smithi192 ceph-mon[49662]: Replacing daemon mds.cephfs.smithi071.qusrrb as rank 0 with standby daemon mds.cephfs.smithi071.vxejye 2024-04-23T02:11:06.261 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:05 smithi192 ceph-mon[49662]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T02:11:06.261 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:05 smithi192 ceph-mon[49662]: osdmap e46: 6 total, 6 up, 6 in 2024-04-23T02:11:06.261 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:05 smithi192 ceph-mon[49662]: mds.? [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] up:standby-replay 2024-04-23T02:11:06.261 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:05 smithi192 ceph-mon[49662]: fsmap cephfs:2/2 {0=cephfs.smithi071.vxejye=up:replay,1=cephfs.smithi192.gtmyst=up:active} 1 up:standby-replay 2024-04-23T02:11:07.133 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:06 smithi071 ceph-mon[39837]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-04-23T02:11:07.133 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:06 smithi071 ceph-mon[39837]: mds.? [v2:172.21.15.71:6826/3442634960,v1:172.21.15.71:6827/3442634960] up:boot 2024-04-23T02:11:07.133 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:06 smithi071 ceph-mon[39837]: mds.? [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] up:resolve 2024-04-23T02:11:07.133 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:06 smithi071 ceph-mon[39837]: fsmap cephfs:2/2 {0=cephfs.smithi071.vxejye=up:resolve,1=cephfs.smithi192.gtmyst=up:active} 1 up:standby-replay 1 up:standby 2024-04-23T02:11:07.134 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:06 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.qusrrb"}]: dispatch 2024-04-23T02:11:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:06 smithi192 ceph-mon[49662]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-04-23T02:11:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:06 smithi192 ceph-mon[49662]: mds.? [v2:172.21.15.71:6826/3442634960,v1:172.21.15.71:6827/3442634960] up:boot 2024-04-23T02:11:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:06 smithi192 ceph-mon[49662]: mds.? [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] up:resolve 2024-04-23T02:11:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:06 smithi192 ceph-mon[49662]: fsmap cephfs:2/2 {0=cephfs.smithi071.vxejye=up:resolve,1=cephfs.smithi192.gtmyst=up:active} 1 up:standby-replay 1 up:standby 2024-04-23T02:11:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:06 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.qusrrb"}]: dispatch 2024-04-23T02:11:07.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.406+0000 7f9d791e9700 1 -- 172.21.15.71:0/108797995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d74101790 msgr2=0x7f9d74103b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:07.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.406+0000 7f9d791e9700 1 --2- 172.21.15.71:0/108797995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d74101790 0x7f9d74103b80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009910 tx=0x7f9d64009c20 comp rx=0 tx=0).stop 2024-04-23T02:11:07.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.406+0000 7f9d791e9700 1 -- 172.21.15.71:0/108797995 shutdown_connections 2024-04-23T02:11:07.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.406+0000 7f9d791e9700 1 --2- 172.21.15.71:0/108797995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d741040c0 0x7f9d741064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:07.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.406+0000 7f9d791e9700 1 --2- 172.21.15.71:0/108797995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d74101790 0x7f9d74103b80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:07.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.406+0000 7f9d791e9700 1 -- 172.21.15.71:0/108797995 >> 172.21.15.71:0/108797995 conn(0x7f9d740fb120 msgr2=0x7f9d740fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:07.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.406+0000 7f9d791e9700 1 -- 172.21.15.71:0/108797995 shutdown_connections 2024-04-23T02:11:07.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.406+0000 7f9d791e9700 1 -- 172.21.15.71:0/108797995 wait complete. 2024-04-23T02:11:07.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.407+0000 7f9d791e9700 1 Processor -- start 2024-04-23T02:11:07.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.407+0000 7f9d791e9700 1 -- start start 2024-04-23T02:11:07.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.407+0000 7f9d791e9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d74101790 0x7f9d741963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:07.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.407+0000 7f9d791e9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d741040c0 0x7f9d741968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:07.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.407+0000 7f9d791e9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d74196f10 con 0x7f9d741040c0 2024-04-23T02:11:07.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.407+0000 7f9d791e9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d74197050 con 0x7f9d74101790 2024-04-23T02:11:07.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.408+0000 7f9d7259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d741040c0 0x7f9d741968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:07.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.408+0000 7f9d7259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d741040c0 0x7f9d741968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45688/0 (socket says 172.21.15.71:45688) 2024-04-23T02:11:07.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.408+0000 7f9d7259c700 1 -- 172.21.15.71:0/1095421523 learned_addr learned my addr 172.21.15.71:0/1095421523 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:11:07.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.408+0000 7f9d72d9d700 1 --2- 172.21.15.71:0/1095421523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d74101790 0x7f9d741963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:07.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.408+0000 7f9d7259c700 1 -- 172.21.15.71:0/1095421523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d74101790 msgr2=0x7f9d741963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:07.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.408+0000 7f9d7259c700 1 --2- 172.21.15.71:0/1095421523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d74101790 0x7f9d741963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:07.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.408+0000 7f9d7259c700 1 -- 172.21.15.71:0/1095421523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d640095f0 con 0x7f9d741040c0 2024-04-23T02:11:07.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.408+0000 7f9d7259c700 1 --2- 172.21.15.71:0/1095421523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d741040c0 0x7f9d741968f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00d8d0 tx=0x7f9d5c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:07.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.408+0000 7f9d6bfff700 1 -- 172.21.15.71:0/1095421523 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d5c0098a0 con 0x7f9d741040c0 2024-04-23T02:11:07.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.408+0000 7f9d791e9700 1 -- 172.21.15.71:0/1095421523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7419bb00 con 0x7f9d741040c0 2024-04-23T02:11:07.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.408+0000 7f9d6bfff700 1 -- 172.21.15.71:0/1095421523 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f9d5c009e90 con 0x7f9d741040c0 2024-04-23T02:11:07.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.408+0000 7f9d6bfff700 1 -- 172.21.15.71:0/1095421523 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d5c00f5f0 con 0x7f9d741040c0 2024-04-23T02:11:07.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.408+0000 7f9d791e9700 1 -- 172.21.15.71:0/1095421523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7419c050 con 0x7f9d741040c0 2024-04-23T02:11:07.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.409+0000 7f9d791e9700 1 -- 172.21.15.71:0/1095421523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d741096c0 con 0x7f9d741040c0 2024-04-23T02:11:07.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.410+0000 7f9d6bfff700 1 -- 172.21.15.71:0/1095421523 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f9d5c00f820 con 0x7f9d741040c0 2024-04-23T02:11:07.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.410+0000 7f9d6bfff700 1 --2- 172.21.15.71:0/1095421523 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f9d6006c4e0 0x7f9d6006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:07.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.410+0000 7f9d72d9d700 1 --2- 172.21.15.71:0/1095421523 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f9d6006c4e0 0x7f9d6006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:07.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.410+0000 7f9d6bfff700 1 -- 172.21.15.71:0/1095421523 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9d5c08b770 con 0x7f9d741040c0 2024-04-23T02:11:07.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.410+0000 7f9d72d9d700 1 --2- 172.21.15.71:0/1095421523 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f9d6006c4e0 0x7f9d6006e9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9d64006010 tx=0x7f9d6400b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:07.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.415+0000 7f9d6bfff700 1 -- 172.21.15.71:0/1095421523 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9d5c059980 con 0x7f9d741040c0 2024-04-23T02:11:07.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.651+0000 7f9d791e9700 1 -- 172.21.15.71:0/1095421523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9d741098e0 con 0x7f9d741040c0 2024-04-23T02:11:07.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.672+0000 7f9d6bfff700 1 -- 172.21.15.71:0/1095421523 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v18) v1 ==== 168+0+0 (secure 0 0 0) 0x7f9d5c059510 con 0x7f9d741040c0 2024-04-23T02:11:07.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.674+0000 7f9d791e9700 1 -- 172.21.15.71:0/1095421523 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f9d6006c4e0 msgr2=0x7f9d6006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:07.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.674+0000 7f9d791e9700 1 --2- 172.21.15.71:0/1095421523 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f9d6006c4e0 0x7f9d6006e9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9d64006010 tx=0x7f9d6400b540 comp rx=0 tx=0).stop 2024-04-23T02:11:07.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.674+0000 7f9d791e9700 1 -- 172.21.15.71:0/1095421523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d741040c0 msgr2=0x7f9d741968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:07.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.674+0000 7f9d791e9700 1 --2- 172.21.15.71:0/1095421523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d741040c0 0x7f9d741968f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00d8d0 tx=0x7f9d5c00dbe0 comp rx=0 tx=0).stop 2024-04-23T02:11:07.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.674+0000 7f9d791e9700 1 -- 172.21.15.71:0/1095421523 shutdown_connections 2024-04-23T02:11:07.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.674+0000 7f9d791e9700 1 --2- 172.21.15.71:0/1095421523 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f9d6006c4e0 0x7f9d6006e9a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:07.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.674+0000 7f9d791e9700 1 --2- 172.21.15.71:0/1095421523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d74101790 0x7f9d741963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:07.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.674+0000 7f9d791e9700 1 --2- 172.21.15.71:0/1095421523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d741040c0 0x7f9d741968f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:07.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.674+0000 7f9d791e9700 1 -- 172.21.15.71:0/1095421523 >> 172.21.15.71:0/1095421523 conn(0x7f9d740fb120 msgr2=0x7f9d740fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:07.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.674+0000 7f9d791e9700 1 -- 172.21.15.71:0/1095421523 shutdown_connections 2024-04-23T02:11:07.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:07.674+0000 7f9d791e9700 1 -- 172.21.15.71:0/1095421523 wait complete. 2024-04-23T02:11:07.680 INFO:teuthology.orchestra.run.smithi071.stderr:inline data enabled 2024-04-23T02:11:08.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:08 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:08.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:08 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:08.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:08 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:08.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:08 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1095421523' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T02:11:08.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:08 smithi192 ceph-mon[49662]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-04-23T02:11:08.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:08 smithi192 ceph-mon[49662]: mds.? [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] up:reconnect 2024-04-23T02:11:08.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:08 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1095421523' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T02:11:08.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:08 smithi192 ceph-mon[49662]: fsmap cephfs:2/2 {0=cephfs.smithi071.vxejye=up:reconnect,1=cephfs.smithi192.gtmyst=up:active} 1 up:standby-replay 1 up:standby 2024-04-23T02:11:08.468 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:08 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:08.469 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:08 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:08.469 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:08 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:08.469 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:08 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1095421523' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T02:11:08.469 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:08 smithi071 ceph-mon[39837]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-04-23T02:11:08.469 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:08 smithi071 ceph-mon[39837]: mds.? [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] up:reconnect 2024-04-23T02:11:08.469 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:08 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1095421523' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T02:11:08.469 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:08 smithi071 ceph-mon[39837]: fsmap cephfs:2/2 {0=cephfs.smithi071.vxejye=up:reconnect,1=cephfs.smithi192.gtmyst=up:active} 1 up:standby-replay 1 up:standby 2024-04-23T02:11:08.488 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T02:11:08.498 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi071.front.sepia.ceph.com 2024-04-23T02:11:08.499 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-23T02:11:08.865 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:11:09.343 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:09 smithi071 ceph-mon[39837]: pgmap v197: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-04-23T02:11:09.343 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:09 smithi071 ceph-mon[39837]: mds.? [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] up:rejoin 2024-04-23T02:11:09.343 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:09 smithi071 ceph-mon[39837]: mds.? [v2:172.21.15.192:6824/3780549896,v1:172.21.15.192:6825/3780549896] up:active 2024-04-23T02:11:09.343 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:09 smithi071 ceph-mon[39837]: mds.? [v2:172.21.15.192:6826/2100078284,v1:172.21.15.192:6827/2100078284] up:standby-replay 2024-04-23T02:11:09.344 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:09 smithi071 ceph-mon[39837]: fsmap cephfs:2/2 {0=cephfs.smithi071.vxejye=up:rejoin,1=cephfs.smithi192.gtmyst=up:active} 1 up:standby-replay 1 up:standby 2024-04-23T02:11:09.344 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:09 smithi071 ceph-mon[39837]: daemon mds.cephfs.smithi071.vxejye is now active in filesystem cephfs as rank 0 2024-04-23T02:11:09.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:09 smithi192 ceph-mon[49662]: pgmap v197: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-04-23T02:11:09.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:09 smithi192 ceph-mon[49662]: mds.? [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] up:rejoin 2024-04-23T02:11:09.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:09 smithi192 ceph-mon[49662]: mds.? [v2:172.21.15.192:6824/3780549896,v1:172.21.15.192:6825/3780549896] up:active 2024-04-23T02:11:09.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:09 smithi192 ceph-mon[49662]: mds.? [v2:172.21.15.192:6826/2100078284,v1:172.21.15.192:6827/2100078284] up:standby-replay 2024-04-23T02:11:09.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:09 smithi192 ceph-mon[49662]: fsmap cephfs:2/2 {0=cephfs.smithi071.vxejye=up:rejoin,1=cephfs.smithi192.gtmyst=up:active} 1 up:standby-replay 1 up:standby 2024-04-23T02:11:09.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:09 smithi192 ceph-mon[49662]: daemon mds.cephfs.smithi071.vxejye is now active in filesystem cephfs as rank 0 2024-04-23T02:11:10.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:10 smithi071 ceph-mon[39837]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T02:11:10.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:10 smithi071 ceph-mon[39837]: mds.? [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] up:active 2024-04-23T02:11:10.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:10 smithi071 ceph-mon[39837]: fsmap cephfs:2 {0=cephfs.smithi071.vxejye=up:active,1=cephfs.smithi192.gtmyst=up:active} 1 up:standby-replay 1 up:standby 2024-04-23T02:11:10.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:10 smithi071 ceph-mon[39837]: fsmap cephfs:2 {0=cephfs.smithi071.vxejye=up:active,1=cephfs.smithi192.gtmyst=up:active} 2 up:standby-replay 2024-04-23T02:11:10.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:10 smithi192 ceph-mon[49662]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T02:11:10.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:10 smithi192 ceph-mon[49662]: mds.? [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] up:active 2024-04-23T02:11:10.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:10 smithi192 ceph-mon[49662]: fsmap cephfs:2 {0=cephfs.smithi071.vxejye=up:active,1=cephfs.smithi192.gtmyst=up:active} 1 up:standby-replay 1 up:standby 2024-04-23T02:11:10.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:10 smithi192 ceph-mon[49662]: fsmap cephfs:2 {0=cephfs.smithi071.vxejye=up:active,1=cephfs.smithi192.gtmyst=up:active} 2 up:standby-replay 2024-04-23T02:11:11.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:11 smithi192 ceph-mon[49662]: pgmap v198: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-23T02:11:11.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:11 smithi071 ceph-mon[39837]: pgmap v198: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-23T02:11:11.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.506+0000 7fd1e017a700 1 -- 172.21.15.71:0/92534021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d8100ca0 msgr2=0x7fd1d81010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:11.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.506+0000 7fd1e017a700 1 --2- 172.21.15.71:0/92534021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d8100ca0 0x7fd1d81010c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009910 tx=0x7fd1c8009c20 comp rx=0 tx=0).stop 2024-04-23T02:11:11.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.506+0000 7fd1e017a700 1 -- 172.21.15.71:0/92534021 shutdown_connections 2024-04-23T02:11:11.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.506+0000 7fd1e017a700 1 --2- 172.21.15.71:0/92534021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1d8101ea0 0x7fd1d8102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:11.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.506+0000 7fd1e017a700 1 --2- 172.21.15.71:0/92534021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d8100ca0 0x7fd1d81010c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:11.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.506+0000 7fd1e017a700 1 -- 172.21.15.71:0/92534021 >> 172.21.15.71:0/92534021 conn(0x7fd1d80fc260 msgr2=0x7fd1d80fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:11.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.506+0000 7fd1e017a700 1 -- 172.21.15.71:0/92534021 shutdown_connections 2024-04-23T02:11:11.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.506+0000 7fd1e017a700 1 -- 172.21.15.71:0/92534021 wait complete. 2024-04-23T02:11:11.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.507+0000 7fd1e017a700 1 Processor -- start 2024-04-23T02:11:11.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.507+0000 7fd1e017a700 1 -- start start 2024-04-23T02:11:11.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.507+0000 7fd1e017a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d8100ca0 0x7fd1d8196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:11.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.507+0000 7fd1e017a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1d8101ea0 0x7fd1d8196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:11.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.507+0000 7fd1e017a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d8197060 con 0x7fd1d8100ca0 2024-04-23T02:11:11.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.507+0000 7fd1e017a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d81971a0 con 0x7fd1d8101ea0 2024-04-23T02:11:11.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.507+0000 7fd1ddf16700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d8100ca0 0x7fd1d8196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:11.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.507+0000 7fd1ddf16700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d8100ca0 0x7fd1d8196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45718/0 (socket says 172.21.15.71:45718) 2024-04-23T02:11:11.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.507+0000 7fd1ddf16700 1 -- 172.21.15.71:0/1736961152 learned_addr learned my addr 172.21.15.71:0/1736961152 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:11:11.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.507+0000 7fd1ddf16700 1 -- 172.21.15.71:0/1736961152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1d8101ea0 msgr2=0x7fd1d8196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:11.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.507+0000 7fd1dd715700 1 --2- 172.21.15.71:0/1736961152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1d8101ea0 0x7fd1d8196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:11.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.507+0000 7fd1ddf16700 1 --2- 172.21.15.71:0/1736961152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1d8101ea0 0x7fd1d8196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:11.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.507+0000 7fd1ddf16700 1 -- 172.21.15.71:0/1736961152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c80095f0 con 0x7fd1d8100ca0 2024-04-23T02:11:11.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.508+0000 7fd1ddf16700 1 --2- 172.21.15.71:0/1736961152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d8100ca0 0x7fd1d8196500 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800bda0 tx=0x7fd1c800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:11.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.508+0000 7fd1ceffd700 1 -- 172.21.15.71:0/1736961152 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1c801d070 con 0x7fd1d8100ca0 2024-04-23T02:11:11.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.508+0000 7fd1ceffd700 1 -- 172.21.15.71:0/1736961152 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd1c8022470 con 0x7fd1d8100ca0 2024-04-23T02:11:11.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.508+0000 7fd1e017a700 1 -- 172.21.15.71:0/1736961152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d819bbf0 con 0x7fd1d8100ca0 2024-04-23T02:11:11.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.508+0000 7fd1ceffd700 1 -- 172.21.15.71:0/1736961152 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1c800f690 con 0x7fd1d8100ca0 2024-04-23T02:11:11.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.508+0000 7fd1e017a700 1 -- 172.21.15.71:0/1736961152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d819c0e0 con 0x7fd1d8100ca0 2024-04-23T02:11:11.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.509+0000 7fd1e017a700 1 -- 172.21.15.71:0/1736961152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d8109740 con 0x7fd1d8100ca0 2024-04-23T02:11:11.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.509+0000 7fd1ceffd700 1 -- 172.21.15.71:0/1736961152 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fd1c80229c0 con 0x7fd1d8100ca0 2024-04-23T02:11:11.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.510+0000 7fd1ceffd700 1 --2- 172.21.15.71:0/1736961152 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fd1c406c4e0 0x7fd1c406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:11.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.510+0000 7fd1ceffd700 1 -- 172.21.15.71:0/1736961152 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd1c808cf20 con 0x7fd1d8100ca0 2024-04-23T02:11:11.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.510+0000 7fd1dd715700 1 --2- 172.21.15.71:0/1736961152 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fd1c406c4e0 0x7fd1c406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:11.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.510+0000 7fd1dd715700 1 --2- 172.21.15.71:0/1736961152 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fd1c406c4e0 0x7fd1c406e9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4006fd0 tx=0x7fd1d4008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:11.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.514+0000 7fd1ceffd700 1 -- 172.21.15.71:0/1736961152 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd1c80c00e0 con 0x7fd1d8100ca0 2024-04-23T02:11:11.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.752+0000 7fd1e017a700 1 -- 172.21.15.71:0/1736961152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd1d8109930 con 0x7fd1d8100ca0 2024-04-23T02:11:11.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.753+0000 7fd1ceffd700 1 -- 172.21.15.71:0/1736961152 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1848 (secure 0 0 0) 0x7fd1c8027750 con 0x7fd1d8100ca0 2024-04-23T02:11:11.757 INFO:teuthology.orchestra.run.smithi071.stdout:e21 2024-04-23T02:11:11.757 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:11:11.757 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:11:11.757 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:11:11.757 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:11:11.757 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:11:11.757 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:11:11.757 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 21 2024-04-23T02:11:11.757 INFO:teuthology.orchestra.run.smithi071.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T02:11:11.757 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:11:11.758 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:11:09.675662+0000 2024-04-23T02:11:11.758 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:11:11.758 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:11:11.758 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:11:11.758 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:11:11.758 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:11:11.758 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:11:11.758 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:11:11.758 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 46 2024-04-23T02:11:11.758 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:11:11.758 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:11:11.758 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:11:11.758 INFO:teuthology.orchestra.run.smithi071.stdout:up {0=14478,1=24243} 2024-04-23T02:11:11.759 INFO:teuthology.orchestra.run.smithi071.stdout:failed 2024-04-23T02:11:11.759 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:11:11.759 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:11:11.759 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:11:11.759 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:11:11.759 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:11:11.759 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:11:11.759 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:11:11.759 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:11:11.759 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:11:11.759 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/3442634960,v1:172.21.15.71:6827/3442634960] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:11:11.759 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{1:24243} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.192:6824/3780549896,v1:172.21.15.192:6825/3780549896] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:11:11.759 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{1:24255} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2100078284,v1:172.21.15.192:6827/2100078284] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:11:11.759 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:11:11.759 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:11:11.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.755+0000 7fd1e017a700 1 -- 172.21.15.71:0/1736961152 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fd1c406c4e0 msgr2=0x7fd1c406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:11.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.755+0000 7fd1e017a700 1 --2- 172.21.15.71:0/1736961152 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fd1c406c4e0 0x7fd1c406e9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4006fd0 tx=0x7fd1d4008040 comp rx=0 tx=0).stop 2024-04-23T02:11:11.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.755+0000 7fd1e017a700 1 -- 172.21.15.71:0/1736961152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d8100ca0 msgr2=0x7fd1d8196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:11.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.755+0000 7fd1e017a700 1 --2- 172.21.15.71:0/1736961152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d8100ca0 0x7fd1d8196500 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800bda0 tx=0x7fd1c800be80 comp rx=0 tx=0).stop 2024-04-23T02:11:11.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.755+0000 7fd1e017a700 1 -- 172.21.15.71:0/1736961152 shutdown_connections 2024-04-23T02:11:11.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.755+0000 7fd1e017a700 1 --2- 172.21.15.71:0/1736961152 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fd1c406c4e0 0x7fd1c406e9a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:11.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.755+0000 7fd1e017a700 1 --2- 172.21.15.71:0/1736961152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d8100ca0 0x7fd1d8196500 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:11.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.755+0000 7fd1e017a700 1 --2- 172.21.15.71:0/1736961152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1d8101ea0 0x7fd1d8196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:11.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.755+0000 7fd1e017a700 1 -- 172.21.15.71:0/1736961152 >> 172.21.15.71:0/1736961152 conn(0x7fd1d80fc260 msgr2=0x7fd1d81050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:11.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.755+0000 7fd1e017a700 1 -- 172.21.15.71:0/1736961152 shutdown_connections 2024-04-23T02:11:11.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:11.755+0000 7fd1e017a700 1 -- 172.21.15.71:0/1736961152 wait complete. 2024-04-23T02:11:11.761 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 21 2024-04-23T02:11:12.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:12 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1736961152' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:11:12.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:12 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1736961152' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:11:12.625 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-04-23T02:11:13.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:13 smithi071 ceph-mon[39837]: pgmap v199: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T02:11:13.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:13 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:13.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:13 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:11:13.370 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:11:13.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:13 smithi192 ceph-mon[49662]: pgmap v199: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T02:11:13.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:13 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:13.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:13 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:11:15.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.284+0000 7f5c4c5b0700 1 -- 172.21.15.71:0/2820009575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c44100e10 msgr2=0x7f5c44101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:15.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.284+0000 7f5c4c5b0700 1 --2- 172.21.15.71:0/2820009575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c44100e10 0x7f5c44101270 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5c38009910 tx=0x7f5c38009c20 comp rx=0 tx=0).stop 2024-04-23T02:11:15.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.284+0000 7f5c4c5b0700 1 -- 172.21.15.71:0/2820009575 shutdown_connections 2024-04-23T02:11:15.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.284+0000 7f5c4c5b0700 1 --2- 172.21.15.71:0/2820009575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c44100e10 0x7f5c44101270 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:15.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.284+0000 7f5c4c5b0700 1 --2- 172.21.15.71:0/2820009575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c440ffc10 0x7f5c44100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:15.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.284+0000 7f5c4c5b0700 1 -- 172.21.15.71:0/2820009575 >> 172.21.15.71:0/2820009575 conn(0x7f5c440fb190 msgr2=0x7f5c440fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:15.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.284+0000 7f5c4c5b0700 1 -- 172.21.15.71:0/2820009575 shutdown_connections 2024-04-23T02:11:15.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.284+0000 7f5c4c5b0700 1 -- 172.21.15.71:0/2820009575 wait complete. 2024-04-23T02:11:15.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c4c5b0700 1 Processor -- start 2024-04-23T02:11:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c4c5b0700 1 -- start start 2024-04-23T02:11:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c4c5b0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c440ffc10 0x7f5c44069100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c4c5b0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c44100e10 0x7f5c4419b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c4c5b0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4419b6d0 con 0x7f5c440ffc10 2024-04-23T02:11:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c4c5b0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4419b840 con 0x7f5c44100e10 2024-04-23T02:11:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c4a34c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c440ffc10 0x7f5c44069100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c4a34c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c440ffc10 0x7f5c44069100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45748/0 (socket says 172.21.15.71:45748) 2024-04-23T02:11:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c4a34c700 1 -- 172.21.15.71:0/3291847907 learned_addr learned my addr 172.21.15.71:0/3291847907 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:11:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c49b4b700 1 --2- 172.21.15.71:0/3291847907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c44100e10 0x7f5c4419b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c4a34c700 1 -- 172.21.15.71:0/3291847907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c44100e10 msgr2=0x7f5c4419b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c4a34c700 1 --2- 172.21.15.71:0/3291847907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c44100e10 0x7f5c4419b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c4a34c700 1 -- 172.21.15.71:0/3291847907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c380095f0 con 0x7f5c440ffc10 2024-04-23T02:11:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c4a34c700 1 --2- 172.21.15.71:0/3291847907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c440ffc10 0x7f5c44069100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5c34007e90 tx=0x7f5c3400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c437fe700 1 -- 172.21.15.71:0/3291847907 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c34017070 con 0x7f5c440ffc10 2024-04-23T02:11:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c437fe700 1 -- 172.21.15.71:0/3291847907 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f5c3400f040 con 0x7f5c440ffc10 2024-04-23T02:11:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c4c5b0700 1 -- 172.21.15.71:0/3291847907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4419bb20 con 0x7f5c440ffc10 2024-04-23T02:11:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c437fe700 1 -- 172.21.15.71:0/3291847907 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c34004710 con 0x7f5c440ffc10 2024-04-23T02:11:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.285+0000 7f5c4c5b0700 1 -- 172.21.15.71:0/3291847907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4419c070 con 0x7f5c440ffc10 2024-04-23T02:11:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.286+0000 7f5c4c5b0700 1 -- 172.21.15.71:0/3291847907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c441085c0 con 0x7f5c440ffc10 2024-04-23T02:11:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.287+0000 7f5c437fe700 1 -- 172.21.15.71:0/3291847907 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f5c34003a00 con 0x7f5c440ffc10 2024-04-23T02:11:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.287+0000 7f5c437fe700 1 --2- 172.21.15.71:0/3291847907 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f5c2c06c4e0 0x7f5c2c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.287+0000 7f5c437fe700 1 -- 172.21.15.71:0/3291847907 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5c3408b1f0 con 0x7f5c440ffc10 2024-04-23T02:11:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.287+0000 7f5c49b4b700 1 --2- 172.21.15.71:0/3291847907 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f5c2c06c4e0 0x7f5c2c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:15.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.288+0000 7f5c49b4b700 1 --2- 172.21.15.71:0/3291847907 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f5c2c06c4e0 0x7f5c2c06e9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5c44069660 tx=0x7f5c3800b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:15.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.292+0000 7f5c437fe700 1 -- 172.21.15.71:0/3291847907 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5c34059400 con 0x7f5c440ffc10 2024-04-23T02:11:15.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:15 smithi071 ceph-mon[39837]: pgmap v200: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T02:11:15.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:15.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:15.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:11:15.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:11:15.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:15.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:15 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:11:15.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.546+0000 7f5c4c5b0700 1 -- 172.21.15.71:0/3291847907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c441087b0 con 0x7f5c440ffc10 2024-04-23T02:11:15.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.546+0000 7f5c437fe700 1 -- 172.21.15.71:0/3291847907 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4773 (secure 0 0 0) 0x7f5c34058f90 con 0x7f5c440ffc10 2024-04-23T02:11:15.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.548+0000 7f5c4c5b0700 1 -- 172.21.15.71:0/3291847907 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f5c2c06c4e0 msgr2=0x7f5c2c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:15.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.548+0000 7f5c4c5b0700 1 --2- 172.21.15.71:0/3291847907 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f5c2c06c4e0 0x7f5c2c06e9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5c44069660 tx=0x7f5c3800b540 comp rx=0 tx=0).stop 2024-04-23T02:11:15.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.548+0000 7f5c4c5b0700 1 -- 172.21.15.71:0/3291847907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c440ffc10 msgr2=0x7f5c44069100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:15.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.548+0000 7f5c4c5b0700 1 --2- 172.21.15.71:0/3291847907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c440ffc10 0x7f5c44069100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5c34007e90 tx=0x7f5c3400d3b0 comp rx=0 tx=0).stop 2024-04-23T02:11:15.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.548+0000 7f5c4c5b0700 1 -- 172.21.15.71:0/3291847907 shutdown_connections 2024-04-23T02:11:15.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.548+0000 7f5c4c5b0700 1 --2- 172.21.15.71:0/3291847907 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f5c2c06c4e0 0x7f5c2c06e9a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:15.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.548+0000 7f5c4c5b0700 1 --2- 172.21.15.71:0/3291847907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c440ffc10 0x7f5c44069100 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:15.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.548+0000 7f5c4c5b0700 1 --2- 172.21.15.71:0/3291847907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c44100e10 0x7f5c4419b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:15.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.548+0000 7f5c4c5b0700 1 -- 172.21.15.71:0/3291847907 >> 172.21.15.71:0/3291847907 conn(0x7f5c440fb190 msgr2=0x7f5c44104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:15.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.548+0000 7f5c4c5b0700 1 -- 172.21.15.71:0/3291847907 shutdown_connections 2024-04-23T02:11:15.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:15.548+0000 7f5c4c5b0700 1 -- 172.21.15.71:0/3291847907 wait complete. 2024-04-23T02:11:15.553 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 21 2024-04-23T02:11:15.562 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:11:15.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:15 smithi192 ceph-mon[49662]: pgmap v200: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T02:11:15.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:15.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:11:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:11:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:15.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:15 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:11:16.425 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:16 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3291847907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T02:11:16.426 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-04-23T02:11:16.500 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:16 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/3291847907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T02:11:17.001 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:11:17.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:17 smithi071 ceph-mon[39837]: pgmap v201: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 10 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-23T02:11:17.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:17 smithi192 ceph-mon[49662]: pgmap v201: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 10 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-23T02:11:18.557 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:18 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:18.557 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:18 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:18.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:18 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:18.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:18 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:19.550 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:19 smithi071 ceph-mon[39837]: pgmap v202: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-23T02:11:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:19 smithi192 ceph-mon[49662]: pgmap v202: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-23T02:11:19.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.854+0000 7f01509ed700 1 -- 172.21.15.71:0/2088690259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c104130 msgr2=0x7f014c106520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:19.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.854+0000 7f01509ed700 1 --2- 172.21.15.71:0/2088690259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c104130 0x7f014c106520 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0140009910 tx=0x7f0140009c20 comp rx=0 tx=0).stop 2024-04-23T02:11:19.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.855+0000 7f01509ed700 1 -- 172.21.15.71:0/2088690259 shutdown_connections 2024-04-23T02:11:19.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.855+0000 7f01509ed700 1 --2- 172.21.15.71:0/2088690259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c104130 0x7f014c106520 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:19.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.855+0000 7f01509ed700 1 --2- 172.21.15.71:0/2088690259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f014c101800 0x7f014c103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:19.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.855+0000 7f01509ed700 1 -- 172.21.15.71:0/2088690259 >> 172.21.15.71:0/2088690259 conn(0x7f014c0fb190 msgr2=0x7f014c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:19.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.855+0000 7f01509ed700 1 -- 172.21.15.71:0/2088690259 shutdown_connections 2024-04-23T02:11:19.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.855+0000 7f01509ed700 1 -- 172.21.15.71:0/2088690259 wait complete. 2024-04-23T02:11:19.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.855+0000 7f01509ed700 1 Processor -- start 2024-04-23T02:11:19.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.855+0000 7f01509ed700 1 -- start start 2024-04-23T02:11:19.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.855+0000 7f01509ed700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c101800 0x7f014c1963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:19.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.855+0000 7f01509ed700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f014c104130 0x7f014c196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:19.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.855+0000 7f01509ed700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c196f20 con 0x7f014c101800 2024-04-23T02:11:19.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.855+0000 7f01509ed700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c197060 con 0x7f014c104130 2024-04-23T02:11:19.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.856+0000 7f014a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c101800 0x7f014c1963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:19.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.856+0000 7f014a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c101800 0x7f014c1963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55494/0 (socket says 172.21.15.71:55494) 2024-04-23T02:11:19.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.856+0000 7f014a59c700 1 -- 172.21.15.71:0/3829484255 learned_addr learned my addr 172.21.15.71:0/3829484255 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:11:19.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.856+0000 7f0149d9b700 1 --2- 172.21.15.71:0/3829484255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f014c104130 0x7f014c196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:19.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.856+0000 7f014a59c700 1 -- 172.21.15.71:0/3829484255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f014c104130 msgr2=0x7f014c196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:19.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.856+0000 7f014a59c700 1 --2- 172.21.15.71:0/3829484255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f014c104130 0x7f014c196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:19.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.856+0000 7f014a59c700 1 -- 172.21.15.71:0/3829484255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01400095f0 con 0x7f014c101800 2024-04-23T02:11:19.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.856+0000 7f014a59c700 1 --2- 172.21.15.71:0/3829484255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c101800 0x7f014c1963c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f013c00d8d0 tx=0x7f013c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:19.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.856+0000 7f013b7fe700 1 -- 172.21.15.71:0/3829484255 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f013c0098a0 con 0x7f014c101800 2024-04-23T02:11:19.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.856+0000 7f013b7fe700 1 -- 172.21.15.71:0/3829484255 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f013c009e90 con 0x7f014c101800 2024-04-23T02:11:19.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.856+0000 7f01509ed700 1 -- 172.21.15.71:0/3829484255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014c19bb10 con 0x7f014c101800 2024-04-23T02:11:19.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.856+0000 7f013b7fe700 1 -- 172.21.15.71:0/3829484255 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f013c00f5f0 con 0x7f014c101800 2024-04-23T02:11:19.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.856+0000 7f01509ed700 1 -- 172.21.15.71:0/3829484255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014c19c060 con 0x7f014c101800 2024-04-23T02:11:19.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.857+0000 7f01509ed700 1 -- 172.21.15.71:0/3829484255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014c1904d0 con 0x7f014c101800 2024-04-23T02:11:19.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.858+0000 7f013b7fe700 1 -- 172.21.15.71:0/3829484255 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f013c010460 con 0x7f014c101800 2024-04-23T02:11:19.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.858+0000 7f013b7fe700 1 --2- 172.21.15.71:0/3829484255 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f013406c4e0 0x7f013406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:19.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.858+0000 7f013b7fe700 1 -- 172.21.15.71:0/3829484255 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f013c08b740 con 0x7f014c101800 2024-04-23T02:11:19.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.858+0000 7f0149d9b700 1 --2- 172.21.15.71:0/3829484255 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f013406c4e0 0x7f013406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:19.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.859+0000 7f0149d9b700 1 --2- 172.21.15.71:0/3829484255 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f013406c4e0 0x7f013406e9a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0140006010 tx=0x7f014000b540 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:19.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:19.863+0000 7f013b7fe700 1 -- 172.21.15.71:0/3829484255 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f013c059950 con 0x7f014c101800 2024-04-23T02:11:20.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:20.144+0000 7f01509ed700 1 -- 172.21.15.71:0/3829484255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f014c02cc70 con 0x7f014c101800 2024-04-23T02:11:20.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:20.145+0000 7f013b7fe700 1 -- 172.21.15.71:0/3829484255 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v21) v1 ==== 78+0+83 (secure 0 0 0) 0x7f013c010710 con 0x7f014c101800 2024-04-23T02:11:20.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:20.147+0000 7f01509ed700 1 -- 172.21.15.71:0/3829484255 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f013406c4e0 msgr2=0x7f013406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:20.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:20.147+0000 7f01509ed700 1 --2- 172.21.15.71:0/3829484255 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f013406c4e0 0x7f013406e9a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0140006010 tx=0x7f014000b540 comp rx=0 tx=0).stop 2024-04-23T02:11:20.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:20.147+0000 7f01509ed700 1 -- 172.21.15.71:0/3829484255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c101800 msgr2=0x7f014c1963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:20.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:20.147+0000 7f01509ed700 1 --2- 172.21.15.71:0/3829484255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c101800 0x7f014c1963c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f013c00d8d0 tx=0x7f013c00dbe0 comp rx=0 tx=0).stop 2024-04-23T02:11:20.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:20.147+0000 7f01509ed700 1 -- 172.21.15.71:0/3829484255 shutdown_connections 2024-04-23T02:11:20.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:20.147+0000 7f01509ed700 1 --2- 172.21.15.71:0/3829484255 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f013406c4e0 0x7f013406e9a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:20.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:20.147+0000 7f01509ed700 1 --2- 172.21.15.71:0/3829484255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c101800 0x7f014c1963c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:20.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:20.147+0000 7f01509ed700 1 --2- 172.21.15.71:0/3829484255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f014c104130 0x7f014c196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:20.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:20.147+0000 7f01509ed700 1 -- 172.21.15.71:0/3829484255 >> 172.21.15.71:0/3829484255 conn(0x7f014c0fb190 msgr2=0x7f014c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:20.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:20.147+0000 7f01509ed700 1 -- 172.21.15.71:0/3829484255 shutdown_connections 2024-04-23T02:11:20.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:20.147+0000 7f01509ed700 1 -- 172.21.15.71:0/3829484255 wait complete. 2024-04-23T02:11:20.163 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:11:20.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:20 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3829484255' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-23T02:11:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:20 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/3829484255' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-23T02:11:21.238 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-04-23T02:11:21.259 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T02:11:21.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.411+0000 7f411509e700 1 -- 172.21.15.71:0/1662860583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4110101d00 msgr2=0x7f4110102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.411+0000 7f411509e700 1 --2- 172.21.15.71:0/1662860583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4110101d00 0x7f4110102160 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4100009910 tx=0x7f4100009c20 comp rx=0 tx=0).stop 2024-04-23T02:11:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.411+0000 7f411509e700 1 -- 172.21.15.71:0/1662860583 shutdown_connections 2024-04-23T02:11:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.411+0000 7f411509e700 1 --2- 172.21.15.71:0/1662860583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4110101d00 0x7f4110102160 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.411+0000 7f411509e700 1 --2- 172.21.15.71:0/1662860583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4110100b10 0x7f4110100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.411+0000 7f411509e700 1 -- 172.21.15.71:0/1662860583 >> 172.21.15.71:0/1662860583 conn(0x7f41100fc060 msgr2=0x7f41100fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.411+0000 7f411509e700 1 -- 172.21.15.71:0/1662860583 shutdown_connections 2024-04-23T02:11:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.412+0000 7f411509e700 1 -- 172.21.15.71:0/1662860583 wait complete. 2024-04-23T02:11:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.412+0000 7f411509e700 1 Processor -- start 2024-04-23T02:11:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.412+0000 7f411509e700 1 -- start start 2024-04-23T02:11:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.413+0000 7f411509e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4110100b10 0x7f4110196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.413+0000 7f411509e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4110101d00 0x7f41101966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.413+0000 7f411509e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4110196cc0 con 0x7f4110101d00 2024-04-23T02:11:21.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.413+0000 7f411509e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4110196e00 con 0x7f4110100b10 2024-04-23T02:11:21.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.413+0000 7f410e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4110101d00 0x7f41101966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:21.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.413+0000 7f410e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4110101d00 0x7f41101966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55506/0 (socket says 172.21.15.71:55506) 2024-04-23T02:11:21.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.413+0000 7f410e59c700 1 -- 172.21.15.71:0/2157493687 learned_addr learned my addr 172.21.15.71:0/2157493687 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:11:21.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.413+0000 7f410ed9d700 1 --2- 172.21.15.71:0/2157493687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4110100b10 0x7f4110196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:21.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.413+0000 7f410e59c700 1 -- 172.21.15.71:0/2157493687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4110100b10 msgr2=0x7f4110196160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:21.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.413+0000 7f410e59c700 1 --2- 172.21.15.71:0/2157493687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4110100b10 0x7f4110196160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:21.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.413+0000 7f410e59c700 1 -- 172.21.15.71:0/2157493687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41000095f0 con 0x7f4110101d00 2024-04-23T02:11:21.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.413+0000 7f410e59c700 1 --2- 172.21.15.71:0/2157493687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4110101d00 0x7f41101966a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f41000098e0 tx=0x7f410000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:21.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.413+0000 7f4107fff700 1 -- 172.21.15.71:0/2157493687 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f410001d070 con 0x7f4110101d00 2024-04-23T02:11:21.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.413+0000 7f411509e700 1 -- 172.21.15.71:0/2157493687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411019b840 con 0x7f4110101d00 2024-04-23T02:11:21.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.413+0000 7f4107fff700 1 -- 172.21.15.71:0/2157493687 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f410000f460 con 0x7f4110101d00 2024-04-23T02:11:21.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.413+0000 7f4107fff700 1 -- 172.21.15.71:0/2157493687 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4100021640 con 0x7f4110101d00 2024-04-23T02:11:21.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.413+0000 7f411509e700 1 -- 172.21.15.71:0/2157493687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411019bdc0 con 0x7f4110101d00 2024-04-23T02:11:21.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.414+0000 7f411509e700 1 -- 172.21.15.71:0/2157493687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4110109540 con 0x7f4110101d00 2024-04-23T02:11:21.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.415+0000 7f4107fff700 1 -- 172.21.15.71:0/2157493687 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f410000f5d0 con 0x7f4110101d00 2024-04-23T02:11:21.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.415+0000 7f4107fff700 1 --2- 172.21.15.71:0/2157493687 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f40fc06c5a0 0x7f40fc06ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:21.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.415+0000 7f410ed9d700 1 --2- 172.21.15.71:0/2157493687 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f40fc06c5a0 0x7f40fc06ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:21.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.415+0000 7f4107fff700 1 -- 172.21.15.71:0/2157493687 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f410008cad0 con 0x7f4110101d00 2024-04-23T02:11:21.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.416+0000 7f410ed9d700 1 --2- 172.21.15.71:0/2157493687 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f40fc06c5a0 0x7f40fc06ea60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f40f8005fd0 tx=0x7f40f800b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:21.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.420+0000 7f4107fff700 1 -- 172.21.15.71:0/2157493687 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f410005ace0 con 0x7f4110101d00 2024-04-23T02:11:21.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:21 smithi071 ceph-mon[39837]: pgmap v203: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-23T02:11:21.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:21 smithi192 ceph-mon[49662]: pgmap v203: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-23T02:11:21.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.629+0000 7f411509e700 1 -- 172.21.15.71:0/2157493687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f411019c400 con 0x7f4110101d00 2024-04-23T02:11:21.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.631+0000 7f4107fff700 1 -- 172.21.15.71:0/2157493687 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+15125 (secure 0 0 0) 0x7f410005a870 con 0x7f4110101d00 2024-04-23T02:11:21.635 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:11:21.636 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":46,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","created":"2024-04-23T02:03:14.872097+0000","modified":"2024-04-23T02:11:05.663477+0000","last_up_change":"2024-04-23T02:09:29.659762+0000","last_in_change":"2024-04-23T02:09:09.560591+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T02:08:06.416728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-23T02:10:50.283526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-23T02:10:50.576356+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5f48bf1-6e98-4432-885a-f6ed0c42e5d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":275655379},{"type":"v1","addr":"172.21.15.71:6803","nonce":275655379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":275655379},{"type":"v1","addr":"172.21.15.71:6805","nonce":275655379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":275655379},{"type":"v1","addr":"172.21.15.71:6809","nonce":275655379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":275655379},{"type":"v1","addr":"172.21.15.71:6807","nonce":275655379}]},"public_addr":"172.21.15.71:6803/275655379","cluster_addr":"172.21.15.71:6805/275655379","heartbeat_back_addr":"172.21.15.71:6809/275655379","heartbeat_front_addr":"172.21.15.71:6807/275655379","state":["exists","up"]},{"osd":1,"uuid":"04304fd2-42f9-4525-88e8-c7e20c9c51e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3686410016},{"type":"v1","addr":"172.21.15.71:6811","nonce":3686410016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3686410016},{"type":"v1","addr":"172.21.15.71:6813","nonce":3686410016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3686410016},{"type":"v1","addr":"172.21.15.71:6817","nonce":3686410016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3686410016},{"type":"v1","addr":"172.21.15.71:6815","nonce":3686410016}]},"public_addr":"172.21.15.71:6811/3686410016","cluster_addr":"172.21.15.71:6813/3686410016","heartbeat_back_addr":"172.21.15.71:6817/3686410016","heartbeat_front_addr":"172.21.15.71:6815/3686410016","state":["exists","up"]},{"osd":2,"uuid":"6797c77c-4ccc-4489-9269-77105d7c45c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":2096213030},{"type":"v1","addr":"172.21.15.71:6819","nonce":2096213030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":2096213030},{"type":"v1","addr":"172.21.15.71:6821","nonce":2096213030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2096213030},{"type":"v1","addr":"172.21.15.71:6825","nonce":2096213030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":2096213030},{"type":"v1","addr":"172.21.15.71:6823","nonce":2096213030}]},"public_addr":"172.21.15.71:6819/2096213030","cluster_addr":"172.21.15.71:6821/2096213030","heartbeat_back_addr":"172.21.15.71:6825/2096213030","heartbeat_front_addr":"172.21.15.71:6823/2096213030","state":["exists","up"]},{"osd":3,"uuid":"62033906-4d1c-431d-a87d-a12e0a134383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":593532086},{"type":"v1","addr":"172.21.15.192:6801","nonce":593532086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":593532086},{"type":"v1","addr":"172.21.15.192:6803","nonce":593532086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":593532086},{"type":"v1","addr":"172.21.15.192:6807","nonce":593532086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":593532086},{"type":"v1","addr":"172.21.15.192:6805","nonce":593532086}]},"public_addr":"172.21.15.192:6801/593532086","cluster_addr":"172.21.15.192:6803/593532086","heartbeat_back_addr":"172.21.15.192:6807/593532086","heartbeat_front_addr":"172.21.15.192:6805/593532086","state":["exists","up"]},{"osd":4,"uuid":"6a7317f5-dc05-415a-8409-81f80c196a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3661697663},{"type":"v1","addr":"172.21.15.192:6809","nonce":3661697663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3661697663},{"type":"v1","addr":"172.21.15.192:6811","nonce":3661697663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3661697663},{"type":"v1","addr":"172.21.15.192:6815","nonce":3661697663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3661697663},{"type":"v1","addr":"172.21.15.192:6813","nonce":3661697663}]},"public_addr":"172.21.15.192:6809/3661697663","cluster_addr":"172.21.15.192:6811/3661697663","heartbeat_back_addr":"172.21.15.192:6815/3661697663","heartbeat_front_addr":"172.21.15.192:6813/3661697663","state":["exists","up"]},{"osd":5,"uuid":"68b550d5-d74e-417e-9bcc-98740efaf0ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3375258254},{"type":"v1","addr":"172.21.15.192:6817","nonce":3375258254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3375258254},{"type":"v1","addr":"172.21.15.192:6819","nonce":3375258254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3375258254},{"type":"v1","addr":"172.21.15.192:6823","nonce":3375258254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3375258254},{"type":"v1","addr":"172.21.15.192:6821","nonce":3375258254}]},"public_addr":"172.21.15.192:6817/3375258254","cluster_addr":"172.21.15.192:6819/3375258254","heartbeat_back_addr":"172.21.15.192:6823/3375258254","heartbeat_front_addr":"172.21.15.192:6821/3375258254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:07:01.863954+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:07:32.930839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:04.093747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:31.574485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:59.698428+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:09:28.076223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6827/3504670324":"2024-04-24T02:11:05.663451+0000","172.21.15.71:0/3579793925":"2024-04-24T02:04:23.247815+0000","172.21.15.71:0/2225989508":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/294030681":"2024-04-24T02:04:23.247815+0000","172.21.15.71:0/1995129374":"2024-04-24T02:03:48.178880+0000","172.21.15.71:6801/3794652670":"2024-04-24T02:03:48.178880+0000","172.21.15.71:6800/4165305842":"2024-04-24T02:05:42.095805+0000","172.21.15.71:6800/3794652670":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/4205332842":"2024-04-24T02:03:48.178880+0000","172.21.15.71:6800/3878148961":"2024-04-24T02:04:23.247815+0000","172.21.15.71:6801/3878148961":"2024-04-24T02:04:23.247815+0000","172.21.15.71:0/823266478":"2024-04-24T02:04:23.247815+0000","172.21.15.71:0/936344738":"2024-04-24T02:05:42.095805+0000","172.21.15.71:0/56041968":"2024-04-24T02:05:42.095805+0000","172.21.15.71:6801/4165305842":"2024-04-24T02:05:42.095805+0000","172.21.15.71:0/2206241071":"2024-04-24T02:05:42.095805+0000","172.21.15.71:6826/3504670324":"2024-04-24T02:11:05.663451+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T02:11:21.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.633+0000 7f411509e700 1 -- 172.21.15.71:0/2157493687 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f40fc06c5a0 msgr2=0x7f40fc06ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:21.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.633+0000 7f411509e700 1 --2- 172.21.15.71:0/2157493687 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f40fc06c5a0 0x7f40fc06ea60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f40f8005fd0 tx=0x7f40f800b410 comp rx=0 tx=0).stop 2024-04-23T02:11:21.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.633+0000 7f411509e700 1 -- 172.21.15.71:0/2157493687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4110101d00 msgr2=0x7f41101966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:21.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.633+0000 7f411509e700 1 --2- 172.21.15.71:0/2157493687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4110101d00 0x7f41101966a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f41000098e0 tx=0x7f410000bd70 comp rx=0 tx=0).stop 2024-04-23T02:11:21.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.633+0000 7f411509e700 1 -- 172.21.15.71:0/2157493687 shutdown_connections 2024-04-23T02:11:21.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.633+0000 7f411509e700 1 --2- 172.21.15.71:0/2157493687 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f40fc06c5a0 0x7f40fc06ea60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:21.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.633+0000 7f411509e700 1 --2- 172.21.15.71:0/2157493687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4110100b10 0x7f4110196160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:21.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.633+0000 7f411509e700 1 --2- 172.21.15.71:0/2157493687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4110101d00 0x7f41101966a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:21.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.633+0000 7f411509e700 1 -- 172.21.15.71:0/2157493687 >> 172.21.15.71:0/2157493687 conn(0x7f41100fc060 msgr2=0x7f4110104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:21.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.633+0000 7f411509e700 1 -- 172.21.15.71:0/2157493687 shutdown_connections 2024-04-23T02:11:21.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.633+0000 7f411509e700 1 -- 172.21.15.71:0/2157493687 wait complete. 2024-04-23T02:11:21.651 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T02:08:06.416728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-23T02:10:50.283526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-23T02:10:50.576356+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T02:11:21.651 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T02:11:21.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.772+0000 7f0faf165700 1 -- 172.21.15.71:0/343688080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fa80ff9e0 msgr2=0x7f0fa80ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:21.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.772+0000 7f0faf165700 1 --2- 172.21.15.71:0/343688080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fa80ff9e0 0x7f0fa80ffe00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009910 tx=0x7f0f9c009c20 comp rx=0 tx=0).stop 2024-04-23T02:11:21.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.772+0000 7f0faf165700 1 -- 172.21.15.71:0/343688080 shutdown_connections 2024-04-23T02:11:21.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.772+0000 7f0faf165700 1 --2- 172.21.15.71:0/343688080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fa8100bd0 0x7f0fa8101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:21.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.772+0000 7f0faf165700 1 --2- 172.21.15.71:0/343688080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fa80ff9e0 0x7f0fa80ffe00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:21.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.772+0000 7f0faf165700 1 -- 172.21.15.71:0/343688080 >> 172.21.15.71:0/343688080 conn(0x7f0fa80faf70 msgr2=0x7f0fa80fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:21.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.772+0000 7f0faf165700 1 -- 172.21.15.71:0/343688080 shutdown_connections 2024-04-23T02:11:21.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.772+0000 7f0faf165700 1 -- 172.21.15.71:0/343688080 wait complete. 2024-04-23T02:11:21.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.772+0000 7f0faf165700 1 Processor -- start 2024-04-23T02:11:21.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.772+0000 7f0faf165700 1 -- start start 2024-04-23T02:11:21.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.773+0000 7f0faf165700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fa80ff9e0 0x7f0fa8195160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:21.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.773+0000 7f0faf165700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fa8100bd0 0x7f0fa81956a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:21.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.773+0000 7f0faf165700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa8195cc0 con 0x7f0fa8100bd0 2024-04-23T02:11:21.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.773+0000 7f0faf165700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa8195e00 con 0x7f0fa80ff9e0 2024-04-23T02:11:21.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.773+0000 7f0fa7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fa8100bd0 0x7f0fa81956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:21.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.773+0000 7f0fa7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fa8100bd0 0x7f0fa81956a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55510/0 (socket says 172.21.15.71:55510) 2024-04-23T02:11:21.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.773+0000 7f0fa7fff700 1 -- 172.21.15.71:0/1958036324 learned_addr learned my addr 172.21.15.71:0/1958036324 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:11:21.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.773+0000 7f0facf01700 1 --2- 172.21.15.71:0/1958036324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fa80ff9e0 0x7f0fa8195160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:21.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.773+0000 7f0fa7fff700 1 -- 172.21.15.71:0/1958036324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fa80ff9e0 msgr2=0x7f0fa8195160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:21.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.773+0000 7f0fa7fff700 1 --2- 172.21.15.71:0/1958036324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fa80ff9e0 0x7f0fa8195160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:21.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.773+0000 7f0fa7fff700 1 -- 172.21.15.71:0/1958036324 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9c0095f0 con 0x7f0fa8100bd0 2024-04-23T02:11:21.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.773+0000 7f0fa7fff700 1 --2- 172.21.15.71:0/1958036324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fa8100bd0 0x7f0fa81956a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0f9400d8d0 tx=0x7f0f9400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:21.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.773+0000 7f0fa5ffb700 1 -- 172.21.15.71:0/1958036324 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f94009940 con 0x7f0fa8100bd0 2024-04-23T02:11:21.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.773+0000 7f0fa5ffb700 1 -- 172.21.15.71:0/1958036324 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f0f94010460 con 0x7f0fa8100bd0 2024-04-23T02:11:21.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.773+0000 7f0faf165700 1 -- 172.21.15.71:0/1958036324 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa819a8a0 con 0x7f0fa8100bd0 2024-04-23T02:11:21.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.773+0000 7f0fa5ffb700 1 -- 172.21.15.71:0/1958036324 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f9400f5f0 con 0x7f0fa8100bd0 2024-04-23T02:11:21.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.773+0000 7f0faf165700 1 -- 172.21.15.71:0/1958036324 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa819ae80 con 0x7f0fa8100bd0 2024-04-23T02:11:21.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.774+0000 7f0faf165700 1 -- 172.21.15.71:0/1958036324 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa8108410 con 0x7f0fa8100bd0 2024-04-23T02:11:21.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.775+0000 7f0fa5ffb700 1 -- 172.21.15.71:0/1958036324 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f0f940105d0 con 0x7f0fa8100bd0 2024-04-23T02:11:21.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.775+0000 7f0fa5ffb700 1 --2- 172.21.15.71:0/1958036324 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0f9806c4e0 0x7f0f9806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:21.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.775+0000 7f0fa5ffb700 1 -- 172.21.15.71:0/1958036324 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0f9408b8c0 con 0x7f0fa8100bd0 2024-04-23T02:11:21.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.775+0000 7f0facf01700 1 --2- 172.21.15.71:0/1958036324 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0f9806c4e0 0x7f0f9806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:21.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.775+0000 7f0facf01700 1 --2- 172.21.15.71:0/1958036324 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0f9806c4e0 0x7f0f9806e9a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c006010 tx=0x7f0f9c005a90 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:21.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.779+0000 7f0fa5ffb700 1 -- 172.21.15.71:0/1958036324 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0f94059ad0 con 0x7f0fa8100bd0 2024-04-23T02:11:21.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.987+0000 7f0faf165700 1 -- 172.21.15.71:0/1958036324 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0fa819b4f0 con 0x7f0fa8100bd0 2024-04-23T02:11:21.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.987+0000 7f0fa5ffb700 1 -- 172.21.15.71:0/1958036324 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0f94010880 con 0x7f0fa8100bd0 2024-04-23T02:11:21.992 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-23T02:11:21.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.989+0000 7f0faf165700 1 -- 172.21.15.71:0/1958036324 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0f9806c4e0 msgr2=0x7f0f9806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:21.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.990+0000 7f0faf165700 1 --2- 172.21.15.71:0/1958036324 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0f9806c4e0 0x7f0f9806e9a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c006010 tx=0x7f0f9c005a90 comp rx=0 tx=0).stop 2024-04-23T02:11:21.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.990+0000 7f0faf165700 1 -- 172.21.15.71:0/1958036324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fa8100bd0 msgr2=0x7f0fa81956a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:21.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.990+0000 7f0faf165700 1 --2- 172.21.15.71:0/1958036324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fa8100bd0 0x7f0fa81956a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0f9400d8d0 tx=0x7f0f9400dc90 comp rx=0 tx=0).stop 2024-04-23T02:11:21.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.990+0000 7f0faf165700 1 -- 172.21.15.71:0/1958036324 shutdown_connections 2024-04-23T02:11:21.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.990+0000 7f0faf165700 1 --2- 172.21.15.71:0/1958036324 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0f9806c4e0 0x7f0f9806e9a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:21.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.990+0000 7f0faf165700 1 --2- 172.21.15.71:0/1958036324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fa80ff9e0 0x7f0fa8195160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:21.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.990+0000 7f0faf165700 1 --2- 172.21.15.71:0/1958036324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fa8100bd0 0x7f0fa81956a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:21.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.990+0000 7f0faf165700 1 -- 172.21.15.71:0/1958036324 >> 172.21.15.71:0/1958036324 conn(0x7f0fa80faf70 msgr2=0x7f0fa8103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:21.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.990+0000 7f0faf165700 1 -- 172.21.15.71:0/1958036324 shutdown_connections 2024-04-23T02:11:21.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:21.990+0000 7f0faf165700 1 -- 172.21.15.71:0/1958036324 wait complete. 2024-04-23T02:11:22.007 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-23T02:11:22.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.128+0000 7f0e49ef5700 1 -- 172.21.15.71:0/1343571601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e44101610 msgr2=0x7f0e44103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:22.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.128+0000 7f0e49ef5700 1 --2- 172.21.15.71:0/1343571601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e44101610 0x7f0e44103a00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009910 tx=0x7f0e34009c20 comp rx=0 tx=0).stop 2024-04-23T02:11:22.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.128+0000 7f0e49ef5700 1 -- 172.21.15.71:0/1343571601 shutdown_connections 2024-04-23T02:11:22.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.128+0000 7f0e49ef5700 1 --2- 172.21.15.71:0/1343571601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e44103f40 0x7f0e44106330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:22.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.128+0000 7f0e49ef5700 1 --2- 172.21.15.71:0/1343571601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e44101610 0x7f0e44103a00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:22.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.128+0000 7f0e49ef5700 1 -- 172.21.15.71:0/1343571601 >> 172.21.15.71:0/1343571601 conn(0x7f0e440faf40 msgr2=0x7f0e440fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:22.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.128+0000 7f0e49ef5700 1 -- 172.21.15.71:0/1343571601 shutdown_connections 2024-04-23T02:11:22.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.128+0000 7f0e49ef5700 1 -- 172.21.15.71:0/1343571601 wait complete. 2024-04-23T02:11:22.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.128+0000 7f0e49ef5700 1 Processor -- start 2024-04-23T02:11:22.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.129+0000 7f0e49ef5700 1 -- start start 2024-04-23T02:11:22.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.129+0000 7f0e49ef5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e44101610 0x7f0e44196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:22.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.129+0000 7f0e49ef5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e44103f40 0x7f0e441967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:22.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.129+0000 7f0e49ef5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e44196dd0 con 0x7f0e44103f40 2024-04-23T02:11:22.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.129+0000 7f0e49ef5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e44196f10 con 0x7f0e44101610 2024-04-23T02:11:22.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.129+0000 7f0e42ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e44103f40 0x7f0e441967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:22.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.129+0000 7f0e42ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e44103f40 0x7f0e441967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55530/0 (socket says 172.21.15.71:55530) 2024-04-23T02:11:22.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.129+0000 7f0e42ffd700 1 -- 172.21.15.71:0/1476770858 learned_addr learned my addr 172.21.15.71:0/1476770858 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:11:22.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.129+0000 7f0e437fe700 1 --2- 172.21.15.71:0/1476770858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e44101610 0x7f0e44196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:22.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.129+0000 7f0e42ffd700 1 -- 172.21.15.71:0/1476770858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e44101610 msgr2=0x7f0e44196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:22.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.129+0000 7f0e42ffd700 1 --2- 172.21.15.71:0/1476770858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e44101610 0x7f0e44196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:22.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.129+0000 7f0e42ffd700 1 -- 172.21.15.71:0/1476770858 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e340095f0 con 0x7f0e44103f40 2024-04-23T02:11:22.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.129+0000 7f0e437fe700 1 --2- 172.21.15.71:0/1476770858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e44101610 0x7f0e44196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T02:11:22.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.130+0000 7f0e42ffd700 1 --2- 172.21.15.71:0/1476770858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e44103f40 0x7f0e441967b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800e920 tx=0x7f0e3800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:22.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.130+0000 7f0e40ff9700 1 -- 172.21.15.71:0/1476770858 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e3800c9d0 con 0x7f0e44103f40 2024-04-23T02:11:22.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.130+0000 7f0e40ff9700 1 -- 172.21.15.71:0/1476770858 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f0e38010430 con 0x7f0e44103f40 2024-04-23T02:11:22.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.130+0000 7f0e40ff9700 1 -- 172.21.15.71:0/1476770858 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e38018630 con 0x7f0e44103f40 2024-04-23T02:11:22.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.130+0000 7f0e49ef5700 1 -- 172.21.15.71:0/1476770858 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4419b9b0 con 0x7f0e44103f40 2024-04-23T02:11:22.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.130+0000 7f0e49ef5700 1 -- 172.21.15.71:0/1476770858 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4419bf90 con 0x7f0e44103f40 2024-04-23T02:11:22.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.131+0000 7f0e49ef5700 1 -- 172.21.15.71:0/1476770858 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e44109510 con 0x7f0e44103f40 2024-04-23T02:11:22.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.131+0000 7f0e40ff9700 1 -- 172.21.15.71:0/1476770858 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f0e380188f0 con 0x7f0e44103f40 2024-04-23T02:11:22.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.132+0000 7f0e40ff9700 1 --2- 172.21.15.71:0/1476770858 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0e2c06c4e0 0x7f0e2c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:22.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.132+0000 7f0e437fe700 1 --2- 172.21.15.71:0/1476770858 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0e2c06c4e0 0x7f0e2c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:22.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.132+0000 7f0e40ff9700 1 -- 172.21.15.71:0/1476770858 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0e38014070 con 0x7f0e44103f40 2024-04-23T02:11:22.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.132+0000 7f0e437fe700 1 --2- 172.21.15.71:0/1476770858 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0e2c06c4e0 0x7f0e2c06e9a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009fd0 tx=0x7f0e34005940 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:22.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.136+0000 7f0e40ff9700 1 -- 172.21.15.71:0/1476770858 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0e380520e0 con 0x7f0e44103f40 2024-04-23T02:11:22.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.328+0000 7f0e49ef5700 1 -- 172.21.15.71:0/1476770858 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f0e4419c720 con 0x7f0e44103f40 2024-04-23T02:11:22.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.328+0000 7f0e40ff9700 1 -- 172.21.15.71:0/1476770858 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f0e38059dc0 con 0x7f0e44103f40 2024-04-23T02:11:22.333 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-23T02:11:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.330+0000 7f0e49ef5700 1 -- 172.21.15.71:0/1476770858 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0e2c06c4e0 msgr2=0x7f0e2c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.330+0000 7f0e49ef5700 1 --2- 172.21.15.71:0/1476770858 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0e2c06c4e0 0x7f0e2c06e9a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009fd0 tx=0x7f0e34005940 comp rx=0 tx=0).stop 2024-04-23T02:11:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.330+0000 7f0e49ef5700 1 -- 172.21.15.71:0/1476770858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e44103f40 msgr2=0x7f0e441967b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.330+0000 7f0e49ef5700 1 --2- 172.21.15.71:0/1476770858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e44103f40 0x7f0e441967b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800e920 tx=0x7f0e3800ec30 comp rx=0 tx=0).stop 2024-04-23T02:11:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.330+0000 7f0e49ef5700 1 -- 172.21.15.71:0/1476770858 shutdown_connections 2024-04-23T02:11:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.330+0000 7f0e49ef5700 1 --2- 172.21.15.71:0/1476770858 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f0e2c06c4e0 0x7f0e2c06e9a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.330+0000 7f0e49ef5700 1 --2- 172.21.15.71:0/1476770858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e44101610 0x7f0e44196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.330+0000 7f0e49ef5700 1 --2- 172.21.15.71:0/1476770858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e44103f40 0x7f0e441967b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.330+0000 7f0e49ef5700 1 -- 172.21.15.71:0/1476770858 >> 172.21.15.71:0/1476770858 conn(0x7f0e440faf40 msgr2=0x7f0e440fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.330+0000 7f0e49ef5700 1 -- 172.21.15.71:0/1476770858 shutdown_connections 2024-04-23T02:11:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.330+0000 7f0e49ef5700 1 -- 172.21.15.71:0/1476770858 wait complete. 2024-04-23T02:11:22.347 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-23T02:11:22.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.487+0000 7fce22c30700 1 -- 172.21.15.71:0/1305992056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0ff190 msgr2=0x7fce1c0ff5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:22.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.487+0000 7fce22c30700 1 --2- 172.21.15.71:0/1305992056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0ff190 0x7fce1c0ff5b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fce1400b0a0 tx=0x7fce1400b3b0 comp rx=0 tx=0).stop 2024-04-23T02:11:22.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.487+0000 7fce22c30700 1 -- 172.21.15.71:0/1305992056 shutdown_connections 2024-04-23T02:11:22.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.487+0000 7fce22c30700 1 --2- 172.21.15.71:0/1305992056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce1c100380 0x7fce1c1007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:22.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.487+0000 7fce22c30700 1 --2- 172.21.15.71:0/1305992056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0ff190 0x7fce1c0ff5b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:22.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.487+0000 7fce22c30700 1 -- 172.21.15.71:0/1305992056 >> 172.21.15.71:0/1305992056 conn(0x7fce1c0fa740 msgr2=0x7fce1c0fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:22.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.487+0000 7fce22c30700 1 -- 172.21.15.71:0/1305992056 shutdown_connections 2024-04-23T02:11:22.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.487+0000 7fce22c30700 1 -- 172.21.15.71:0/1305992056 wait complete. 2024-04-23T02:11:22.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.488+0000 7fce22c30700 1 Processor -- start 2024-04-23T02:11:22.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.488+0000 7fce22c30700 1 -- start start 2024-04-23T02:11:22.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.488+0000 7fce22c30700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0ff190 0x7fce1c104230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:22.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.488+0000 7fce22c30700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce1c100380 0x7fce1c102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:22.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.488+0000 7fce22c30700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c102db0 con 0x7fce1c0ff190 2024-04-23T02:11:22.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.488+0000 7fce22c30700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c102ef0 con 0x7fce1c100380 2024-04-23T02:11:22.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.488+0000 7fce209cc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0ff190 0x7fce1c104230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:22.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.488+0000 7fce209cc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0ff190 0x7fce1c104230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55548/0 (socket says 172.21.15.71:55548) 2024-04-23T02:11:22.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.488+0000 7fce209cc700 1 -- 172.21.15.71:0/4179809207 learned_addr learned my addr 172.21.15.71:0/4179809207 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:11:22.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.488+0000 7fce209cc700 1 -- 172.21.15.71:0/4179809207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce1c100380 msgr2=0x7fce1c102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:22.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.488+0000 7fce209cc700 1 --2- 172.21.15.71:0/4179809207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce1c100380 0x7fce1c102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:22.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.488+0000 7fce209cc700 1 -- 172.21.15.71:0/4179809207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce14009d00 con 0x7fce1c0ff190 2024-04-23T02:11:22.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.488+0000 7fce209cc700 1 --2- 172.21.15.71:0/4179809207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0ff190 0x7fce1c104230 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fce14015040 tx=0x7fce1400bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:22.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.489+0000 7fce19ffb700 1 -- 172.21.15.71:0/4179809207 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce1400e040 con 0x7fce1c0ff190 2024-04-23T02:11:22.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.489+0000 7fce19ffb700 1 -- 172.21.15.71:0/4179809207 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fce14003de0 con 0x7fce1c0ff190 2024-04-23T02:11:22.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.489+0000 7fce22c30700 1 -- 172.21.15.71:0/4179809207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1c103170 con 0x7fce1c0ff190 2024-04-23T02:11:22.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.489+0000 7fce19ffb700 1 -- 172.21.15.71:0/4179809207 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce1401d7c0 con 0x7fce1c0ff190 2024-04-23T02:11:22.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.489+0000 7fce22c30700 1 -- 172.21.15.71:0/4179809207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1c1036f0 con 0x7fce1c0ff190 2024-04-23T02:11:22.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.490+0000 7fce22c30700 1 -- 172.21.15.71:0/4179809207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1c107bc0 con 0x7fce1c0ff190 2024-04-23T02:11:22.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.490+0000 7fce19ffb700 1 -- 172.21.15.71:0/4179809207 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fce14019070 con 0x7fce1c0ff190 2024-04-23T02:11:22.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.491+0000 7fce19ffb700 1 --2- 172.21.15.71:0/4179809207 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fce0406c4e0 0x7fce0406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:22.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.491+0000 7fce19ffb700 1 -- 172.21.15.71:0/4179809207 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fce1408cfb0 con 0x7fce1c0ff190 2024-04-23T02:11:22.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.491+0000 7fce1bfff700 1 --2- 172.21.15.71:0/4179809207 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fce0406c4e0 0x7fce0406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:22.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.491+0000 7fce1bfff700 1 --2- 172.21.15.71:0/4179809207 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fce0406c4e0 0x7fce0406e9a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fce0c005950 tx=0x7fce0c0058e0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:22.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.496+0000 7fce19ffb700 1 -- 172.21.15.71:0/4179809207 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fce1405b1c0 con 0x7fce1c0ff190 2024-04-23T02:11:22.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:22 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/2157493687' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T02:11:22.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:22 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1958036324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T02:11:22.635 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:22 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/2157493687' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T02:11:22.635 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:22 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1958036324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T02:11:22.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.704+0000 7fce22c30700 1 -- 172.21.15.71:0/4179809207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fce1c107d50 con 0x7fce1c0ff190 2024-04-23T02:11:22.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.705+0000 7fce19ffb700 1 -- 172.21.15.71:0/4179809207 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7fce1405ad50 con 0x7fce1c0ff190 2024-04-23T02:11:22.710 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-23T02:11:22.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.707+0000 7fce22c30700 1 -- 172.21.15.71:0/4179809207 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fce0406c4e0 msgr2=0x7fce0406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:22.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.707+0000 7fce22c30700 1 --2- 172.21.15.71:0/4179809207 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fce0406c4e0 0x7fce0406e9a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fce0c005950 tx=0x7fce0c0058e0 comp rx=0 tx=0).stop 2024-04-23T02:11:22.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.707+0000 7fce22c30700 1 -- 172.21.15.71:0/4179809207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0ff190 msgr2=0x7fce1c104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:22.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.707+0000 7fce22c30700 1 --2- 172.21.15.71:0/4179809207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0ff190 0x7fce1c104230 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fce14015040 tx=0x7fce1400bdc0 comp rx=0 tx=0).stop 2024-04-23T02:11:22.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.707+0000 7fce22c30700 1 -- 172.21.15.71:0/4179809207 shutdown_connections 2024-04-23T02:11:22.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.707+0000 7fce22c30700 1 --2- 172.21.15.71:0/4179809207 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fce0406c4e0 0x7fce0406e9a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:22.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.707+0000 7fce22c30700 1 --2- 172.21.15.71:0/4179809207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0ff190 0x7fce1c104230 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:22.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.707+0000 7fce22c30700 1 --2- 172.21.15.71:0/4179809207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce1c100380 0x7fce1c102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:22.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.707+0000 7fce22c30700 1 -- 172.21.15.71:0/4179809207 >> 172.21.15.71:0/4179809207 conn(0x7fce1c0fa740 msgr2=0x7fce1c0fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:22.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.707+0000 7fce22c30700 1 -- 172.21.15.71:0/4179809207 shutdown_connections 2024-04-23T02:11:22.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.707+0000 7fce22c30700 1 -- 172.21.15.71:0/4179809207 wait complete. 2024-04-23T02:11:22.724 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T02:11:22.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.842+0000 7f8196823700 1 -- 172.21.15.71:0/2992132046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8190100bf0 msgr2=0x7f8190101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:22.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.842+0000 7f8196823700 1 --2- 172.21.15.71:0/2992132046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8190100bf0 0x7f8190101050 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8184009910 tx=0x7f8184009c20 comp rx=0 tx=0).stop 2024-04-23T02:11:22.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.842+0000 7f8196823700 1 -- 172.21.15.71:0/2992132046 shutdown_connections 2024-04-23T02:11:22.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.842+0000 7f8196823700 1 --2- 172.21.15.71:0/2992132046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8190100bf0 0x7f8190101050 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:22.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.842+0000 7f8196823700 1 --2- 172.21.15.71:0/2992132046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81900ffa00 0x7f81900ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:22.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.842+0000 7f8196823700 1 -- 172.21.15.71:0/2992132046 >> 172.21.15.71:0/2992132046 conn(0x7f81900faf70 msgr2=0x7f81900fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:22.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.842+0000 7f8196823700 1 -- 172.21.15.71:0/2992132046 shutdown_connections 2024-04-23T02:11:22.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.842+0000 7f8196823700 1 -- 172.21.15.71:0/2992132046 wait complete. 2024-04-23T02:11:22.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.842+0000 7f8196823700 1 Processor -- start 2024-04-23T02:11:22.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.842+0000 7f8196823700 1 -- start start 2024-04-23T02:11:22.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.843+0000 7f8196823700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81900ffa00 0x7f8190196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:22.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.843+0000 7f8196823700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8190100bf0 0x7f8190196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:22.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.843+0000 7f8196823700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8190196cb0 con 0x7f8190100bf0 2024-04-23T02:11:22.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.843+0000 7f8196823700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8190196df0 con 0x7f81900ffa00 2024-04-23T02:11:22.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.843+0000 7f818f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8190100bf0 0x7f8190196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:22.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.843+0000 7f818f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8190100bf0 0x7f8190196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55574/0 (socket says 172.21.15.71:55574) 2024-04-23T02:11:22.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.843+0000 7f818f7fe700 1 -- 172.21.15.71:0/813202086 learned_addr learned my addr 172.21.15.71:0/813202086 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:11:22.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.843+0000 7f818ffff700 1 --2- 172.21.15.71:0/813202086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81900ffa00 0x7f8190196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:22.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.843+0000 7f818f7fe700 1 -- 172.21.15.71:0/813202086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81900ffa00 msgr2=0x7f8190196150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:22.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.843+0000 7f818f7fe700 1 --2- 172.21.15.71:0/813202086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81900ffa00 0x7f8190196150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:22.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.843+0000 7f818f7fe700 1 -- 172.21.15.71:0/813202086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81840095f0 con 0x7f8190100bf0 2024-04-23T02:11:22.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.843+0000 7f818f7fe700 1 --2- 172.21.15.71:0/813202086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8190100bf0 0x7f8190196690 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f818400b5c0 tx=0x7f818400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:22.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.843+0000 7f818d7fa700 1 -- 172.21.15.71:0/813202086 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f818401d070 con 0x7f8190100bf0 2024-04-23T02:11:22.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.843+0000 7f8196823700 1 -- 172.21.15.71:0/813202086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819019b830 con 0x7f8190100bf0 2024-04-23T02:11:22.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.843+0000 7f818d7fa700 1 -- 172.21.15.71:0/813202086 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f818400f460 con 0x7f8190100bf0 2024-04-23T02:11:22.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.843+0000 7f818d7fa700 1 -- 172.21.15.71:0/813202086 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8184021640 con 0x7f8190100bf0 2024-04-23T02:11:22.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.843+0000 7f8196823700 1 -- 172.21.15.71:0/813202086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819019bdb0 con 0x7f8190100bf0 2024-04-23T02:11:22.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.844+0000 7f8196823700 1 -- 172.21.15.71:0/813202086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8190108430 con 0x7f8190100bf0 2024-04-23T02:11:22.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.845+0000 7f818d7fa700 1 -- 172.21.15.71:0/813202086 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f818400fa20 con 0x7f8190100bf0 2024-04-23T02:11:22.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.845+0000 7f818d7fa700 1 --2- 172.21.15.71:0/813202086 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f817806c5a0 0x7f817806ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:22.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.845+0000 7f818d7fa700 1 -- 172.21.15.71:0/813202086 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f818408cb20 con 0x7f8190100bf0 2024-04-23T02:11:22.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.845+0000 7f818ffff700 1 --2- 172.21.15.71:0/813202086 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f817806c5a0 0x7f817806ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:22.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.846+0000 7f818ffff700 1 --2- 172.21.15.71:0/813202086 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f817806c5a0 0x7f817806ea60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8190100a50 tx=0x7f818000b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:22.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:22.850+0000 7f818d7fa700 1 -- 172.21.15.71:0/813202086 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f818405ad30 con 0x7f8190100bf0 2024-04-23T02:11:23.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:23.086+0000 7f8196823700 1 -- 172.21.15.71:0/813202086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f819019c3f0 con 0x7f8190100bf0 2024-04-23T02:11:23.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:23.087+0000 7f818d7fa700 1 -- 172.21.15.71:0/813202086 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4773 (secure 0 0 0) 0x7f818405a8c0 con 0x7f8190100bf0 2024-04-23T02:11:23.092 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:11:23.092 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:11:09.675662+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14478,"mds_1":24243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi071.vxejye","rank":0,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6829/467545394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":467545394},{"type":"v1","addr":"172.21.15.71:6829","nonce":467545394}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14490":{"gid":14490,"name":"cephfs.smithi071.qusrrb","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6827/3442634960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3442634960},{"type":"v1","addr":"172.21.15.71:6827","nonce":3442634960}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24243":{"gid":24243,"name":"cephfs.smithi192.gtmyst","rank":1,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.192:6825/3780549896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3780549896},{"type":"v1","addr":"172.21.15.192:6825","nonce":3780549896}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24255":{"gid":24255,"name":"cephfs.smithi192.azdyze","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.192:6827/2100078284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2100078284},{"type":"v1","addr":"172.21.15.192:6827","nonce":2100078284}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:11:23.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:23.089+0000 7f8196823700 1 -- 172.21.15.71:0/813202086 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f817806c5a0 msgr2=0x7f817806ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:23.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:23.089+0000 7f8196823700 1 --2- 172.21.15.71:0/813202086 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f817806c5a0 0x7f817806ea60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8190100a50 tx=0x7f818000b410 comp rx=0 tx=0).stop 2024-04-23T02:11:23.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:23.089+0000 7f8196823700 1 -- 172.21.15.71:0/813202086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8190100bf0 msgr2=0x7f8190196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:23.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:23.089+0000 7f8196823700 1 --2- 172.21.15.71:0/813202086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8190100bf0 0x7f8190196690 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f818400b5c0 tx=0x7f818400bd70 comp rx=0 tx=0).stop 2024-04-23T02:11:23.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:23.089+0000 7f8196823700 1 -- 172.21.15.71:0/813202086 shutdown_connections 2024-04-23T02:11:23.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:23.089+0000 7f8196823700 1 --2- 172.21.15.71:0/813202086 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f817806c5a0 0x7f817806ea60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:23.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:23.089+0000 7f8196823700 1 --2- 172.21.15.71:0/813202086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81900ffa00 0x7f8190196150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:23.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:23.089+0000 7f8196823700 1 --2- 172.21.15.71:0/813202086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8190100bf0 0x7f8190196690 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:23.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:23.089+0000 7f8196823700 1 -- 172.21.15.71:0/813202086 >> 172.21.15.71:0/813202086 conn(0x7f81900faf70 msgr2=0x7f8190067da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:23.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:23.089+0000 7f8196823700 1 -- 172.21.15.71:0/813202086 shutdown_connections 2024-04-23T02:11:23.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:23.089+0000 7f8196823700 1 -- 172.21.15.71:0/813202086 wait complete. 2024-04-23T02:11:23.095 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 21 2024-04-23T02:11:23.106 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 21, 'max_mds': 2, 'flags': 50} 2024-04-23T02:11:23.107 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-23T02:11:23.131 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-23T02:11:23.131 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-04-23T02:11:23.132 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-23T02:11:23.132 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-23T02:11:23.132 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T02:11:23.132 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-23T02:11:23.132 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-23T02:11:23.132 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-23T02:11:23.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:23.132 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-04-23T02:11:23.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:23.157 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link delete ceph-brx 2024-04-23T02:11:23.235 INFO:teuthology.orchestra.run.smithi192.stderr:Cannot find device "ceph-brx" 2024-04-23T02:11:23.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T02:11:23.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:23.237 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-04-23T02:11:23.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:23.256 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link delete ceph-brx 2024-04-23T02:11:23.325 INFO:teuthology.orchestra.run.smithi071.stderr:Cannot find device "ceph-brx" 2024-04-23T02:11:23.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T02:11:23.326 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-23T02:11:23.326 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-23T02:11:23.391 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:23 smithi071 ceph-mon[39837]: pgmap v204: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-23T02:11:23.391 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:23 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1476770858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-23T02:11:23.391 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:23 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:23.391 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:23 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:23.391 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:23 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:11:23.391 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:23 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:11:23.391 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:23 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:23.391 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:23 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/4179809207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-23T02:11:23.391 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:23 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/813202086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T02:11:23.589 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:11:23.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:23 smithi192 ceph-mon[49662]: pgmap v204: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-23T02:11:23.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:23 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1476770858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-23T02:11:23.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:23 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:23 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:23 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:11:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:23 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:11:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:23 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:11:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:23 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/4179809207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-23T02:11:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:23 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/813202086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T02:11:25.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.440+0000 7f5f63f6e700 1 -- 172.21.15.71:0/3693042554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f5c105bd0 msgr2=0x7f5f5c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:25.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.440+0000 7f5f63f6e700 1 --2- 172.21.15.71:0/3693042554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f5c105bd0 0x7f5f5c105fb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009910 tx=0x7f5f4c009c20 comp rx=0 tx=0).stop 2024-04-23T02:11:25.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.440+0000 7f5f63f6e700 1 -- 172.21.15.71:0/3693042554 shutdown_connections 2024-04-23T02:11:25.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.440+0000 7f5f63f6e700 1 --2- 172.21.15.71:0/3693042554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5c0ffc20 0x7f5f5c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:25.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.440+0000 7f5f63f6e700 1 --2- 172.21.15.71:0/3693042554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f5c105bd0 0x7f5f5c105fb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:25.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.440+0000 7f5f63f6e700 1 -- 172.21.15.71:0/3693042554 >> 172.21.15.71:0/3693042554 conn(0x7f5f5c075670 msgr2=0x7f5f5c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:25.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.441+0000 7f5f63f6e700 1 -- 172.21.15.71:0/3693042554 shutdown_connections 2024-04-23T02:11:25.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.441+0000 7f5f63f6e700 1 -- 172.21.15.71:0/3693042554 wait complete. 2024-04-23T02:11:25.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.441+0000 7f5f63f6e700 1 Processor -- start 2024-04-23T02:11:25.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.441+0000 7f5f63f6e700 1 -- start start 2024-04-23T02:11:25.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.442+0000 7f5f63f6e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5c0ffc20 0x7f5f5c195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:25.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.442+0000 7f5f63f6e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f5c105bd0 0x7f5f5c195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:25.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.442+0000 7f5f63f6e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c196260 con 0x7f5f5c105bd0 2024-04-23T02:11:25.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.442+0000 7f5f63f6e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c198f90 con 0x7f5f5c0ffc20 2024-04-23T02:11:25.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.442+0000 7f5f61509700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f5c105bd0 0x7f5f5c195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:25.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.442+0000 7f5f61509700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f5c105bd0 0x7f5f5c195bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46864/0 (socket says 172.21.15.71:46864) 2024-04-23T02:11:25.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.442+0000 7f5f61509700 1 -- 172.21.15.71:0/1349312147 learned_addr learned my addr 172.21.15.71:0/1349312147 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:11:25.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.442+0000 7f5f61d0a700 1 --2- 172.21.15.71:0/1349312147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5c0ffc20 0x7f5f5c195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:25.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.442+0000 7f5f61509700 1 -- 172.21.15.71:0/1349312147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5c0ffc20 msgr2=0x7f5f5c195690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:25.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.442+0000 7f5f61509700 1 --2- 172.21.15.71:0/1349312147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5c0ffc20 0x7f5f5c195690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:25.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.442+0000 7f5f61509700 1 -- 172.21.15.71:0/1349312147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c0095f0 con 0x7f5f5c105bd0 2024-04-23T02:11:25.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.442+0000 7f5f61509700 1 --2- 172.21.15.71:0/1349312147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f5c105bd0 0x7f5f5c195bd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800d8d0 tx=0x7f5f5800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:25.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.442+0000 7f5f52ffd700 1 -- 172.21.15.71:0/1349312147 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f580098a0 con 0x7f5f5c105bd0 2024-04-23T02:11:25.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.442+0000 7f5f52ffd700 1 -- 172.21.15.71:0/1349312147 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f5f58009e90 con 0x7f5f5c105bd0 2024-04-23T02:11:25.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.442+0000 7f5f52ffd700 1 -- 172.21.15.71:0/1349312147 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f5800f5f0 con 0x7f5f5c105bd0 2024-04-23T02:11:25.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.442+0000 7f5f63f6e700 1 -- 172.21.15.71:0/1349312147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5c199270 con 0x7f5f5c105bd0 2024-04-23T02:11:25.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.442+0000 7f5f63f6e700 1 -- 172.21.15.71:0/1349312147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5c1997c0 con 0x7f5f5c105bd0 2024-04-23T02:11:25.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.443+0000 7f5f63f6e700 1 -- 172.21.15.71:0/1349312147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f5c1080d0 con 0x7f5f5c105bd0 2024-04-23T02:11:25.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.444+0000 7f5f52ffd700 1 -- 172.21.15.71:0/1349312147 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f5f58010460 con 0x7f5f5c105bd0 2024-04-23T02:11:25.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.444+0000 7f5f52ffd700 1 --2- 172.21.15.71:0/1349312147 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f5f4806c4e0 0x7f5f4806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:25.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.444+0000 7f5f52ffd700 1 -- 172.21.15.71:0/1349312147 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5f5808b750 con 0x7f5f5c105bd0 2024-04-23T02:11:25.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.444+0000 7f5f61d0a700 1 --2- 172.21.15.71:0/1349312147 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f5f4806c4e0 0x7f5f4806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:25.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.445+0000 7f5f61d0a700 1 --2- 172.21.15.71:0/1349312147 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f5f4806c4e0 0x7f5f4806e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c006010 tx=0x7f5f4c01a040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:25.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.448+0000 7f5f52ffd700 1 -- 172.21.15.71:0/1349312147 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5f58055ed0 con 0x7f5f5c105bd0 2024-04-23T02:11:25.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:25 smithi071 ceph-mon[39837]: pgmap v205: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-23T02:11:25.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:25 smithi192 ceph-mon[49662]: pgmap v205: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-23T02:11:25.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.683+0000 7f5f63f6e700 1 -- 172.21.15.71:0/1349312147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5f5c04f2e0 con 0x7f5f5c105bd0 2024-04-23T02:11:25.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.684+0000 7f5f52ffd700 1 -- 172.21.15.71:0/1349312147 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+83 (secure 0 0 0) 0x7f5f58010710 con 0x7f5f5c105bd0 2024-04-23T02:11:25.689 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-23T02:11:25.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.686+0000 7f5f63f6e700 1 -- 172.21.15.71:0/1349312147 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f5f4806c4e0 msgr2=0x7f5f4806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:25.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.686+0000 7f5f63f6e700 1 --2- 172.21.15.71:0/1349312147 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f5f4806c4e0 0x7f5f4806e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c006010 tx=0x7f5f4c01a040 comp rx=0 tx=0).stop 2024-04-23T02:11:25.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.686+0000 7f5f63f6e700 1 -- 172.21.15.71:0/1349312147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f5c105bd0 msgr2=0x7f5f5c195bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:25.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.686+0000 7f5f63f6e700 1 --2- 172.21.15.71:0/1349312147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f5c105bd0 0x7f5f5c195bd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800d8d0 tx=0x7f5f5800dbe0 comp rx=0 tx=0).stop 2024-04-23T02:11:25.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.686+0000 7f5f63f6e700 1 -- 172.21.15.71:0/1349312147 shutdown_connections 2024-04-23T02:11:25.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.686+0000 7f5f63f6e700 1 --2- 172.21.15.71:0/1349312147 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f5f4806c4e0 0x7f5f4806e9a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:25.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.686+0000 7f5f63f6e700 1 --2- 172.21.15.71:0/1349312147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5c0ffc20 0x7f5f5c195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:25.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.686+0000 7f5f63f6e700 1 --2- 172.21.15.71:0/1349312147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f5c105bd0 0x7f5f5c195bd0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:25.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.686+0000 7f5f63f6e700 1 -- 172.21.15.71:0/1349312147 >> 172.21.15.71:0/1349312147 conn(0x7f5f5c075670 msgr2=0x7f5f5c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:25.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.686+0000 7f5f63f6e700 1 -- 172.21.15.71:0/1349312147 shutdown_connections 2024-04-23T02:11:25.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:25.686+0000 7f5f63f6e700 1 -- 172.21.15.71:0/1349312147 wait complete. 2024-04-23T02:11:26.517 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T02:11:26.517 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T02:11:26.517 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2024-04-23T02:11:26.517 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T02:11:26.517 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T02:11:26.517 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T02:11:26.518 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T02:11:26.518 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T02:11:26.518 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T02:11:26.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:26.518 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2024-04-23T02:11:26.580 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T02:11:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T02:11:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T02:11:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-04-23T02:11:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-04-23T02:11:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-04-23T02:11:26.580 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T02:11:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2024-04-23T02:11:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2024-04-23T02:11:26.580 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T02:11:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2024-04-23T02:11:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2024-04-23T02:11:26.581 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T02:11:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2024-04-23T02:11:26.581 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T02:11:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2024-04-23T02:11:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-23T02:11:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42054sec preferred_lft 42054sec 2024-04-23T02:11:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2024-04-23T02:11:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-04-23T02:11:26.582 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-23T02:11:26.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:26.582 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2024-04-23T02:11:26.582 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-04-23T02:11:26.582 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add name ceph-brx type bridge 2024-04-23T02:11:26.582 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip addr flush dev ceph-brx 2024-04-23T02:11:26.582 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set ceph-brx up 2024-04-23T02:11:26.582 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-23T02:11:26.582 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-04-23T02:11:26.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:26 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1349312147' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T02:11:26.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:26 smithi071 ceph-mon[39837]: pgmap v206: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-23T02:11:26.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:26.746 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T02:11:26.774 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-04-23T02:11:26.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:26.775 DEBUG:teuthology.orchestra.run.smithi071:> route 2024-04-23T02:11:26.865 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2024-04-23T02:11:26.865 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T02:11:26.866 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-23T02:11:26.866 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-23T02:11:26.866 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T02:11:26.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:26.866 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2024-04-23T02:11:26.866 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-04-23T02:11:26.866 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-23T02:11:26.866 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-23T02:11:26.867 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-23T02:11:26.867 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-04-23T02:11:27.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:27.102 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-04-23T02:11:27.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:27.120 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list-id 2024-04-23T02:11:27.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:26 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1349312147' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T02:11:27.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:26 smithi192 ceph-mon[49662]: pgmap v206: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-23T02:11:27.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:27.178 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2024-04-23T02:11:27.178 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-04-23T02:11:27.178 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T02:11:27.178 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-23T02:11:27.178 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-04-23T02:11:27.272 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-23T02:11:27.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:27.273 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2024-04-23T02:11:27.273 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-04-23T02:11:27.273 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-23T02:11:27.273 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-23T02:11:27.273 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-23T02:11:27.273 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-23T02:11:27.273 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-23T02:11:27.273 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-04-23T02:11:27.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:27.473 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2024-04-23T02:11:27.473 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-04-23T02:11:27.473 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2024-04-23T02:11:27.473 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set dev brx.0 master ceph-brx 2024-04-23T02:11:27.473 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-04-23T02:11:27.570 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-23T02:11:27.570 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T02:11:27.570 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T02:11:27.591 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T02:11:27.592 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T02:11:27.592 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T02:11:27.649 DEBUG:teuthology.orchestra.run.smithi071:> sudo modprobe fuse 2024-04-23T02:11:27.717 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-23T02:11:27.718 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-04-23T02:11:27.784 INFO:teuthology.orchestra.run.smithi071.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-04-23T02:11:27.785 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-04-23T02:11:27.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:27.786 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2024-04-23T02:11:27.843 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T02:11:27.844 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-23T02:11:27.886 DEBUG:teuthology.orchestra.run.smithi071:> sudo modprobe fuse 2024-04-23T02:11:27.908 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:27 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:11:27.912 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-23T02:11:27.912 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-04-23T02:11:27.979 INFO:teuthology.orchestra.run.smithi071.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-04-23T02:11:27.979 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-04-23T02:11:27.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:27.980 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2024-04-23T02:11:27.996 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2024-04-23T02:11:27.991+0000 7fbbf94615c0 -1 init, newargv = 0x56335d04a360 newargc=15 2024-04-23T02:11:27.996 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[100693]: starting ceph client 2024-04-23T02:11:28.003 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[100693]: starting fuse 2024-04-23T02:11:28.014 INFO:teuthology.orchestra.run.smithi071.stdout:173 2024-04-23T02:11:28.015 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [173] 2024-04-23T02:11:28.015 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-04-23T02:11:28.015 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2024-04-23T02:11:28.015 DEBUG:teuthology.orchestra.run.smithi071:> import re 2024-04-23T02:11:28.015 DEBUG:teuthology.orchestra.run.smithi071:> import os 2024-04-23T02:11:28.015 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2024-04-23T02:11:28.015 DEBUG:teuthology.orchestra.run.smithi071:> 2024-04-23T02:11:28.015 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2024-04-23T02:11:28.015 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-23T02:11:28.016 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2024-04-23T02:11:28.016 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-23T02:11:28.016 DEBUG:teuthology.orchestra.run.smithi071:> 2024-04-23T02:11:28.016 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2024-04-23T02:11:28.016 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2024-04-23T02:11:28.016 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2024-04-23T02:11:28.016 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2024-04-23T02:11:28.016 DEBUG:teuthology.orchestra.run.smithi071:> 2024-04-23T02:11:28.016 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2024-04-23T02:11:28.016 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T02:11:28.016 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T02:11:28.016 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T02:11:28.017 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2024-04-23T02:11:28.017 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2024-04-23T02:11:28.017 DEBUG:teuthology.orchestra.run.smithi071:> return f 2024-04-23T02:11:28.017 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T02:11:28.017 DEBUG:teuthology.orchestra.run.smithi071:> 2024-04-23T02:11:28.017 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2024-04-23T02:11:28.017 DEBUG:teuthology.orchestra.run.smithi071:> ' 2024-04-23T02:11:28.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:27 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:11:28.138 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.100693.asok 2024-04-23T02:11:28.320 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.100693.asok 2024-04-23T02:11:28.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:28.321 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.100693.asok status 2024-04-23T02:11:28.478 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:11:28.478 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2024-04-23T02:11:28.478 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-04-23T02:11:28.478 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-04-23T02:11:28.479 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2024-04-23T02:11:28.479 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2024-04-23T02:11:28.479 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-23T02:11:28.479 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "100693", 2024-04-23T02:11:28.479 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2024-04-23T02:11:28.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:11:28.479 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 0, 2024-04-23T02:11:28.479 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2024-04-23T02:11:28.479 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 14534, 2024-04-23T02:11:28.479 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2024-04-23T02:11:28.479 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2024-04-23T02:11:28.479 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2024-04-23T02:11:28.480 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 14534 2024-04-23T02:11:28.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:11:28.480 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2024-04-23T02:11:28.480 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-04-23T02:11:28.480 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2024-04-23T02:11:28.480 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1982449790 2024-04-23T02:11:28.480 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:11:28.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:11:28.480 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2024-04-23T02:11:28.480 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-04-23T02:11:28.480 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2024-04-23T02:11:28.480 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1982449790 2024-04-23T02:11:28.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:11:28.481 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.14534 192.168.0.1:0/1982449790", 2024-04-23T02:11:28.481 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/1982449790", 2024-04-23T02:11:28.481 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 1, 2024-04-23T02:11:28.481 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 21, 2024-04-23T02:11:28.481 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 46, 2024-04-23T02:11:28.481 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 0, 2024-04-23T02:11:28.481 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2024-04-23T02:11:28.481 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2024-04-23T02:11:28.481 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:11:28.489 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-23T02:11:28.717 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:11:28.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:28 smithi071 ceph-mon[39837]: pgmap v207: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:11:29.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:28 smithi192 ceph-mon[49662]: pgmap v207: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:11:30.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.702+0000 7f527ff06700 1 -- 172.21.15.71:0/650624609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52780fff50 msgr2=0x7f5278105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:30.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.702+0000 7f527ff06700 1 --2- 172.21.15.71:0/650624609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52780fff50 0x7f5278105840 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5274009910 tx=0x7f5274009c20 comp rx=0 tx=0).stop 2024-04-23T02:11:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.702+0000 7f527ff06700 1 -- 172.21.15.71:0/650624609 shutdown_connections 2024-04-23T02:11:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.702+0000 7f527ff06700 1 --2- 172.21.15.71:0/650624609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52780fff50 0x7f5278105840 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.702+0000 7f527ff06700 1 --2- 172.21.15.71:0/650624609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52780ff5a0 0x7f52780ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.702+0000 7f527ff06700 1 -- 172.21.15.71:0/650624609 >> 172.21.15.71:0/650624609 conn(0x7f5278074e30 msgr2=0x7f5278075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.702+0000 7f527ff06700 1 -- 172.21.15.71:0/650624609 shutdown_connections 2024-04-23T02:11:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.702+0000 7f527ff06700 1 -- 172.21.15.71:0/650624609 wait complete. 2024-04-23T02:11:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.703+0000 7f527ff06700 1 Processor -- start 2024-04-23T02:11:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.703+0000 7f527ff06700 1 -- start start 2024-04-23T02:11:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.703+0000 7f527ff06700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52780ff5a0 0x7f527819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.703+0000 7f527ff06700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52780fff50 0x7f527819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.703+0000 7f527ff06700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527819d250 con 0x7f52780fff50 2024-04-23T02:11:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.703+0000 7f527ff06700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5278196700 con 0x7f52780ff5a0 2024-04-23T02:11:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.703+0000 7f527d4a1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52780fff50 0x7f527819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.703+0000 7f527d4a1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52780fff50 0x7f527819cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46888/0 (socket says 172.21.15.71:46888) 2024-04-23T02:11:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.703+0000 7f527d4a1700 1 -- 172.21.15.71:0/3769712766 learned_addr learned my addr 172.21.15.71:0/3769712766 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:11:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.703+0000 7f527dca2700 1 --2- 172.21.15.71:0/3769712766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52780ff5a0 0x7f527819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.703+0000 7f527d4a1700 1 -- 172.21.15.71:0/3769712766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52780ff5a0 msgr2=0x7f527819c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.703+0000 7f527d4a1700 1 --2- 172.21.15.71:0/3769712766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52780ff5a0 0x7f527819c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.703+0000 7f527d4a1700 1 -- 172.21.15.71:0/3769712766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52740095f0 con 0x7f52780fff50 2024-04-23T02:11:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.703+0000 7f527d4a1700 1 --2- 172.21.15.71:0/3769712766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52780fff50 0x7f527819cbc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f52740092e0 tx=0x7f527400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.704+0000 7f526effd700 1 -- 172.21.15.71:0/3769712766 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f527401d070 con 0x7f52780fff50 2024-04-23T02:11:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.704+0000 7f527ff06700 1 -- 172.21.15.71:0/3769712766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5278196980 con 0x7f52780fff50 2024-04-23T02:11:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.704+0000 7f526effd700 1 -- 172.21.15.71:0/3769712766 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f527400f460 con 0x7f52780fff50 2024-04-23T02:11:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.704+0000 7f526effd700 1 -- 172.21.15.71:0/3769712766 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5274021600 con 0x7f52780fff50 2024-04-23T02:11:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.704+0000 7f527ff06700 1 -- 172.21.15.71:0/3769712766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5278196e70 con 0x7f52780fff50 2024-04-23T02:11:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.704+0000 7f527ff06700 1 -- 172.21.15.71:0/3769712766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5278109220 con 0x7f52780fff50 2024-04-23T02:11:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.705+0000 7f526effd700 1 -- 172.21.15.71:0/3769712766 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f527400f5d0 con 0x7f52780fff50 2024-04-23T02:11:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.705+0000 7f526effd700 1 --2- 172.21.15.71:0/3769712766 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f526406c4e0 0x7f526406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.705+0000 7f526effd700 1 -- 172.21.15.71:0/3769712766 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f527408cb30 con 0x7f52780fff50 2024-04-23T02:11:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.705+0000 7f527dca2700 1 --2- 172.21.15.71:0/3769712766 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f526406c4e0 0x7f526406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.706+0000 7f527dca2700 1 --2- 172.21.15.71:0/3769712766 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f526406c4e0 0x7f526406e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5268006fd0 tx=0x7f5268008040 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:30.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.710+0000 7f526effd700 1 -- 172.21.15.71:0/3769712766 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f527405ad40 con 0x7f52780fff50 2024-04-23T02:11:30.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.942+0000 7f527ff06700 1 -- 172.21.15.71:0/3769712766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f527804f2e0 con 0x7f52780fff50 2024-04-23T02:11:30.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.943+0000 7f526effd700 1 -- 172.21.15.71:0/3769712766 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+83 (secure 0 0 0) 0x7f527405a8d0 con 0x7f52780fff50 2024-04-23T02:11:30.947 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-23T02:11:30.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.945+0000 7f527ff06700 1 -- 172.21.15.71:0/3769712766 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f526406c4e0 msgr2=0x7f526406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:30.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.945+0000 7f527ff06700 1 --2- 172.21.15.71:0/3769712766 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f526406c4e0 0x7f526406e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5268006fd0 tx=0x7f5268008040 comp rx=0 tx=0).stop 2024-04-23T02:11:30.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.945+0000 7f527ff06700 1 -- 172.21.15.71:0/3769712766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52780fff50 msgr2=0x7f527819cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:30.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.945+0000 7f527ff06700 1 --2- 172.21.15.71:0/3769712766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52780fff50 0x7f527819cbc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f52740092e0 tx=0x7f527400be20 comp rx=0 tx=0).stop 2024-04-23T02:11:30.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.945+0000 7f527ff06700 1 -- 172.21.15.71:0/3769712766 shutdown_connections 2024-04-23T02:11:30.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.945+0000 7f527ff06700 1 --2- 172.21.15.71:0/3769712766 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f526406c4e0 0x7f526406e9a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:30.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.945+0000 7f527ff06700 1 --2- 172.21.15.71:0/3769712766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52780ff5a0 0x7f527819c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:30.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.945+0000 7f527ff06700 1 --2- 172.21.15.71:0/3769712766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52780fff50 0x7f527819cbc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:30.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.945+0000 7f527ff06700 1 -- 172.21.15.71:0/3769712766 >> 172.21.15.71:0/3769712766 conn(0x7f5278074e30 msgr2=0x7f52780fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:30.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.945+0000 7f527ff06700 1 -- 172.21.15.71:0/3769712766 shutdown_connections 2024-04-23T02:11:30.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:30.945+0000 7f527ff06700 1 -- 172.21.15.71:0/3769712766 wait complete. 2024-04-23T02:11:31.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:31 smithi071 ceph-mon[39837]: pgmap v208: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 170 B/s wr, 4 op/s 2024-04-23T02:11:31.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:31 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3769712766' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T02:11:31.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:31 smithi192 ceph-mon[49662]: pgmap v208: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 170 B/s wr, 4 op/s 2024-04-23T02:11:31.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:31 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/3769712766' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T02:11:31.737 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T02:11:31.737 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T02:11:31.737 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2024-04-23T02:11:31.738 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-23T02:11:31.738 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-23T02:11:31.738 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T02:11:31.738 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T02:11:31.738 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T02:11:31.738 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-23T02:11:31.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:31.738 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2024-04-23T02:11:31.761 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T02:11:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T02:11:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T02:11:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-04-23T02:11:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2024-04-23T02:11:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-04-23T02:11:31.762 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T02:11:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2024-04-23T02:11:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2024-04-23T02:11:31.762 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T02:11:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2024-04-23T02:11:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2024-04-23T02:11:31.762 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T02:11:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2024-04-23T02:11:31.763 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T02:11:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2024-04-23T02:11:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-23T02:11:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 42011sec preferred_lft 42011sec 2024-04-23T02:11:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2024-04-23T02:11:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-04-23T02:11:31.763 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-23T02:11:31.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:31.763 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2024-04-23T02:11:31.763 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-04-23T02:11:31.764 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add name ceph-brx type bridge 2024-04-23T02:11:31.764 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip addr flush dev ceph-brx 2024-04-23T02:11:31.764 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set ceph-brx up 2024-04-23T02:11:31.764 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-23T02:11:31.764 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-04-23T02:11:31.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:31.934 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T02:11:31.964 INFO:teuthology.orchestra.run.smithi192.stdout:1 2024-04-23T02:11:31.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:31.965 DEBUG:teuthology.orchestra.run.smithi192:> route 2024-04-23T02:11:32.038 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2024-04-23T02:11:32.038 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T02:11:32.038 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-23T02:11:32.038 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-23T02:11:32.038 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T02:11:32.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:32.039 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2024-04-23T02:11:32.039 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-04-23T02:11:32.039 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-23T02:11:32.039 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-23T02:11:32.039 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-23T02:11:32.039 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-04-23T02:11:32.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:32.322 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-04-23T02:11:32.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:32.340 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list-id 2024-04-23T02:11:32.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:32.397 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2024-04-23T02:11:32.397 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-04-23T02:11:32.397 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-23T02:11:32.398 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-23T02:11:32.398 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-04-23T02:11:32.492 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-23T02:11:32.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:32.492 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2024-04-23T02:11:32.492 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-04-23T02:11:32.492 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-23T02:11:32.492 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-23T02:11:32.492 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-23T02:11:32.493 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-23T02:11:32.493 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-23T02:11:32.493 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-04-23T02:11:32.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:32.710 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2024-04-23T02:11:32.710 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-04-23T02:11:32.710 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2024-04-23T02:11:32.710 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set dev brx.0 master ceph-brx 2024-04-23T02:11:32.711 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-04-23T02:11:32.808 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-23T02:11:32.808 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T02:11:32.808 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-23T02:11:32.826 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-23T02:11:32.826 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T02:11:32.826 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-23T02:11:32.884 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2024-04-23T02:11:32.952 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-23T02:11:32.952 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-04-23T02:11:33.019 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-04-23T02:11:33.020 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-04-23T02:11:33.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:33.020 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2024-04-23T02:11:33.078 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-23T02:11:33.078 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-23T02:11:33.121 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2024-04-23T02:11:33.148 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-23T02:11:33.148 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-04-23T02:11:33.215 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-04-23T02:11:33.215 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-04-23T02:11:33.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:33.216 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2024-04-23T02:11:33.286 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2024-04-23T02:11:33.282+0000 7fbd31f9e5c0 -1 init, newargv = 0x560f61679490 newargc=15 2024-04-23T02:11:33.286 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[82127]: starting ceph client 2024-04-23T02:11:33.292 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[82127]: starting fuse 2024-04-23T02:11:33.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:33 smithi192 ceph-mon[49662]: pgmap v209: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-04-23T02:11:33.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:33 smithi071 ceph-mon[39837]: pgmap v209: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-04-23T02:11:34.376 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2024-04-23T02:11:34.411 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-23T02:11:34.411 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-04-23T02:11:34.484 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-04-23T02:11:34.485 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-04-23T02:11:34.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:34.486 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2024-04-23T02:11:34.546 INFO:teuthology.orchestra.run.smithi192.stdout:137 2024-04-23T02:11:34.546 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [137] 2024-04-23T02:11:34.546 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-04-23T02:11:34.547 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2024-04-23T02:11:34.547 DEBUG:teuthology.orchestra.run.smithi192:> import re 2024-04-23T02:11:34.547 DEBUG:teuthology.orchestra.run.smithi192:> import os 2024-04-23T02:11:34.547 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2024-04-23T02:11:34.547 DEBUG:teuthology.orchestra.run.smithi192:> 2024-04-23T02:11:34.547 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2024-04-23T02:11:34.547 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-23T02:11:34.547 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2024-04-23T02:11:34.547 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-23T02:11:34.547 DEBUG:teuthology.orchestra.run.smithi192:> 2024-04-23T02:11:34.547 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2024-04-23T02:11:34.547 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2024-04-23T02:11:34.547 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2024-04-23T02:11:34.547 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2024-04-23T02:11:34.548 DEBUG:teuthology.orchestra.run.smithi192:> 2024-04-23T02:11:34.548 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2024-04-23T02:11:34.548 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-23T02:11:34.548 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-23T02:11:34.548 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-23T02:11:34.548 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2024-04-23T02:11:34.548 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2024-04-23T02:11:34.548 DEBUG:teuthology.orchestra.run.smithi192:> return f 2024-04-23T02:11:34.548 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-23T02:11:34.548 DEBUG:teuthology.orchestra.run.smithi192:> 2024-04-23T02:11:34.548 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2024-04-23T02:11:34.548 DEBUG:teuthology.orchestra.run.smithi192:> ' 2024-04-23T02:11:34.676 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.82127.asok 2024-04-23T02:11:34.858 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.82127.asok 2024-04-23T02:11:34.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:34.858 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.82127.asok status 2024-04-23T02:11:35.021 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-04-23T02:11:35.022 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2024-04-23T02:11:35.022 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-04-23T02:11:35.022 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-04-23T02:11:35.022 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2024-04-23T02:11:35.022 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2024-04-23T02:11:35.022 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-23T02:11:35.022 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "82127", 2024-04-23T02:11:35.022 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2024-04-23T02:11:35.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-04-23T02:11:35.022 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2024-04-23T02:11:35.022 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2024-04-23T02:11:35.022 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 24303, 2024-04-23T02:11:35.023 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2024-04-23T02:11:35.023 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2024-04-23T02:11:35.023 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2024-04-23T02:11:35.023 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 24303 2024-04-23T02:11:35.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-04-23T02:11:35.023 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2024-04-23T02:11:35.023 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2024-04-23T02:11:35.023 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2024-04-23T02:11:35.023 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 490649041 2024-04-23T02:11:35.023 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-04-23T02:11:35.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-04-23T02:11:35.023 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2024-04-23T02:11:35.023 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2024-04-23T02:11:35.023 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2024-04-23T02:11:35.024 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 490649041 2024-04-23T02:11:35.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-04-23T02:11:35.024 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.24303 192.168.0.1:0/490649041", 2024-04-23T02:11:35.024 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/490649041", 2024-04-23T02:11:35.024 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2024-04-23T02:11:35.024 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 21, 2024-04-23T02:11:35.024 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 46, 2024-04-23T02:11:35.024 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2024-04-23T02:11:35.024 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2024-04-23T02:11:35.024 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2024-04-23T02:11:35.024 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-04-23T02:11:35.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:35.033 DEBUG:teuthology.orchestra.run.smithi071:> stat --file-system '--printf=%T 2024-04-23T02:11:35.033 DEBUG:teuthology.orchestra.run.smithi071:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-23T02:11:35.058 INFO:teuthology.orchestra.run.smithi071.stdout:fuseblk 2024-04-23T02:11:35.058 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-23T02:11:35.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:35.059 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T02:11:35.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:35.135 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2024-04-23T02:11:35.135 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-23T02:11:35.154 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2024-04-23T02:11:35.154 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-23T02:11:35.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:11:35.154 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-23T02:11:35.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:35 smithi071 ceph-mon[39837]: pgmap v210: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-04-23T02:11:35.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:35 smithi192 ceph-mon[49662]: pgmap v210: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-04-23T02:11:36.636 INFO:teuthology.run_tasks:Running task print... 2024-04-23T02:11:36.648 INFO:teuthology.task.print:**** done client 2024-04-23T02:11:36.648 INFO:teuthology.run_tasks:Running task parallel... 2024-04-23T02:11:36.659 INFO:teuthology.task.parallel:starting parallel... 2024-04-23T02:11:36.659 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-23T02:11:36.660 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-23T02:11:36.660 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi071.front.sepia.ceph.com 2024-04-23T02:11:36.660 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-23T02:11:36.661 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-23T02:11:36.661 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-23T02:11:36.665 INFO:tasks.workunit:Pulling workunits from ref f26c9c3fee57ca330501910de5a07c8769ef5dfc 2024-04-23T02:11:36.665 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-23T02:11:36.665 INFO:tasks.workunit:timeout=3h 2024-04-23T02:11:36.665 INFO:tasks.workunit:cleanup=True 2024-04-23T02:11:36.665 DEBUG:teuthology.orchestra.run.smithi071:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-23T02:11:36.691 INFO:teuthology.orchestra.run.smithi071.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-23T02:11:36.691 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-23T02:11:36.691 INFO:teuthology.orchestra.run.smithi071.stdout:Device: adh/173d Inode: 1 Links: 2 2024-04-23T02:11:36.691 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T02:11:36.691 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-23T02:11:36.691 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-23T02:11:36.691 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-23 02:10:55.378814637 +0000 2024-04-23T02:11:36.691 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-23 02:11:35.127725836 +0000 2024-04-23T02:11:36.692 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-04-23T02:11:36.692 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-23T02:11:36.692 DEBUG:teuthology.orchestra.run.smithi071:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-23T02:11:36.768 DEBUG:teuthology.orchestra.run.smithi192:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-23T02:11:36.792 INFO:teuthology.orchestra.run.smithi192.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-23T02:11:36.793 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-23T02:11:36.793 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 89h/137d Inode: 1 Links: 3 2024-04-23T02:11:36.793 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T02:11:36.793 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-23T02:11:36.793 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-23T02:11:36.793 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-04-23 02:11:36.758152608 +0000 2024-04-23T02:11:36.793 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-04-23 02:11:36.758152608 +0000 2024-04-23T02:11:36.793 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: - 2024-04-23T02:11:36.793 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-23T02:11:36.794 DEBUG:teuthology.orchestra.run.smithi192:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-23T02:11:36.870 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f26c9c3fee57ca330501910de5a07c8769ef5dfc 2024-04-23T02:11:36.871 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout f26c9c3fee57ca330501910de5a07c8769ef5dfc 2024-04-23T02:11:36.889 INFO:tasks.workunit.client.0.smithi071.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-23T02:11:36.911 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:11:36.931 INFO:tasks.workunit.client.1.smithi192.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-23T02:11:36.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:36 smithi071 ceph-mon[39837]: pgmap v211: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-23T02:11:37.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:36 smithi192 ceph-mon[49662]: pgmap v211: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-23T02:11:38.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.835+0000 7f9721cea700 1 -- 172.21.15.71:0/2141565859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f971c0fef80 msgr2=0x7f971c0ff3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:38.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.835+0000 7f9721cea700 1 --2- 172.21.15.71:0/2141565859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f971c0fef80 0x7f971c0ff3a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9704009910 tx=0x7f9704009c20 comp rx=0 tx=0).stop 2024-04-23T02:11:38.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.835+0000 7f9721cea700 1 -- 172.21.15.71:0/2141565859 shutdown_connections 2024-04-23T02:11:38.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.835+0000 7f9721cea700 1 --2- 172.21.15.71:0/2141565859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f971c102000 0x7f971c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:38.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.835+0000 7f9721cea700 1 --2- 172.21.15.71:0/2141565859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f971c0fef80 0x7f971c0ff3a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:38.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.835+0000 7f9721cea700 1 -- 172.21.15.71:0/2141565859 >> 172.21.15.71:0/2141565859 conn(0x7f971c0fa8c0 msgr2=0x7f971c0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:38.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.835+0000 7f9721cea700 1 -- 172.21.15.71:0/2141565859 shutdown_connections 2024-04-23T02:11:38.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.835+0000 7f9721cea700 1 -- 172.21.15.71:0/2141565859 wait complete. 2024-04-23T02:11:38.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.836+0000 7f9721cea700 1 Processor -- start 2024-04-23T02:11:38.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.836+0000 7f9721cea700 1 -- start start 2024-04-23T02:11:38.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.836+0000 7f9721cea700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f971c0fef80 0x7f971c196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:38.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.836+0000 7f9721cea700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f971c102000 0x7f971c196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:38.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.836+0000 7f9721cea700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c196ed0 con 0x7f971c102000 2024-04-23T02:11:38.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.836+0000 7f9721cea700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c197010 con 0x7f971c0fef80 2024-04-23T02:11:38.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.836+0000 7f971affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f971c102000 0x7f971c196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:38.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.836+0000 7f971affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f971c102000 0x7f971c196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33646/0 (socket says 172.21.15.71:33646) 2024-04-23T02:11:38.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.836+0000 7f971affd700 1 -- 172.21.15.71:0/2639326276 learned_addr learned my addr 172.21.15.71:0/2639326276 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:11:38.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.836+0000 7f971b7fe700 1 --2- 172.21.15.71:0/2639326276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f971c0fef80 0x7f971c196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:38.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.836+0000 7f971affd700 1 -- 172.21.15.71:0/2639326276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f971c0fef80 msgr2=0x7f971c196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:38.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.836+0000 7f971affd700 1 --2- 172.21.15.71:0/2639326276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f971c0fef80 0x7f971c196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:38.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.836+0000 7f971affd700 1 -- 172.21.15.71:0/2639326276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97040095f0 con 0x7f971c102000 2024-04-23T02:11:38.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.836+0000 7f971b7fe700 1 --2- 172.21.15.71:0/2639326276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f971c0fef80 0x7f971c196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:11:38.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.837+0000 7f971affd700 1 --2- 172.21.15.71:0/2639326276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f971c102000 0x7f971c196940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f970c00c930 tx=0x7f970c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:38.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.837+0000 7f9718ff9700 1 -- 172.21.15.71:0/2639326276 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f970c007ad0 con 0x7f971c102000 2024-04-23T02:11:38.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.837+0000 7f9721cea700 1 -- 172.21.15.71:0/2639326276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f971c19bad0 con 0x7f971c102000 2024-04-23T02:11:38.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.837+0000 7f9718ff9700 1 -- 172.21.15.71:0/2639326276 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f970c007c30 con 0x7f971c102000 2024-04-23T02:11:38.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.837+0000 7f9718ff9700 1 -- 172.21.15.71:0/2639326276 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f970c018630 con 0x7f971c102000 2024-04-23T02:11:38.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.837+0000 7f9721cea700 1 -- 172.21.15.71:0/2639326276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971c19bff0 con 0x7f971c102000 2024-04-23T02:11:38.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.838+0000 7f9721cea700 1 -- 172.21.15.71:0/2639326276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f971c1096c0 con 0x7f971c102000 2024-04-23T02:11:38.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.838+0000 7f9718ff9700 1 -- 172.21.15.71:0/2639326276 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f970c018790 con 0x7f971c102000 2024-04-23T02:11:38.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.839+0000 7f9718ff9700 1 --2- 172.21.15.71:0/2639326276 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f970806c4e0 0x7f970806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:38.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.839+0000 7f9718ff9700 1 -- 172.21.15.71:0/2639326276 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f970c08b980 con 0x7f971c102000 2024-04-23T02:11:38.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.839+0000 7f971b7fe700 1 --2- 172.21.15.71:0/2639326276 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f970806c4e0 0x7f970806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:38.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.839+0000 7f971b7fe700 1 --2- 172.21.15.71:0/2639326276 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f970806c4e0 0x7f970806e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9704006010 tx=0x7f970400b560 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:38.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:38.843+0000 7f9718ff9700 1 -- 172.21.15.71:0/2639326276 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f970c059c10 con 0x7f971c102000 2024-04-23T02:11:39.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:39.038+0000 7f9721cea700 1 -- 172.21.15.71:0/2639326276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f971c109880 con 0x7f971c102000 2024-04-23T02:11:39.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:39.038+0000 7f9718ff9700 1 -- 172.21.15.71:0/2639326276 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f970c0597a0 con 0x7f971c102000 2024-04-23T02:11:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:39.040+0000 7f9721cea700 1 -- 172.21.15.71:0/2639326276 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f970806c4e0 msgr2=0x7f970806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:39.040+0000 7f9721cea700 1 --2- 172.21.15.71:0/2639326276 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f970806c4e0 0x7f970806e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9704006010 tx=0x7f970400b560 comp rx=0 tx=0).stop 2024-04-23T02:11:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:39.040+0000 7f9721cea700 1 -- 172.21.15.71:0/2639326276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f971c102000 msgr2=0x7f971c196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:39.040+0000 7f9721cea700 1 --2- 172.21.15.71:0/2639326276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f971c102000 0x7f971c196940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f970c00c930 tx=0x7f970c00ccf0 comp rx=0 tx=0).stop 2024-04-23T02:11:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:39.040+0000 7f9721cea700 1 -- 172.21.15.71:0/2639326276 shutdown_connections 2024-04-23T02:11:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:39.040+0000 7f9721cea700 1 --2- 172.21.15.71:0/2639326276 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f970806c4e0 0x7f970806e9a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:39.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:39.040+0000 7f9721cea700 1 --2- 172.21.15.71:0/2639326276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f971c0fef80 0x7f971c196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:39.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:39.040+0000 7f9721cea700 1 --2- 172.21.15.71:0/2639326276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f971c102000 0x7f971c196940 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:39.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:39.040+0000 7f9721cea700 1 -- 172.21.15.71:0/2639326276 >> 172.21.15.71:0/2639326276 conn(0x7f971c0fa8c0 msgr2=0x7f971c105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:39.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:39.040+0000 7f9721cea700 1 -- 172.21.15.71:0/2639326276 shutdown_connections 2024-04-23T02:11:39.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:39.040+0000 7f9721cea700 1 -- 172.21.15.71:0/2639326276 wait complete. 2024-04-23T02:11:39.442 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:39 smithi071 ceph-mon[39837]: pgmap v212: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-23T02:11:39.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:39 smithi192 ceph-mon[49662]: pgmap v212: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-23T02:11:39.833 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-23T02:11:40.103 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:11:41.445 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:41 smithi071 ceph-mon[39837]: pgmap v213: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-23T02:11:41.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:41 smithi192 ceph-mon[49662]: pgmap v213: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-23T02:11:42.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.001+0000 7f1a55531700 1 -- 172.21.15.71:0/949131775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a50100cb0 msgr2=0x7f1a501010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:42.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.001+0000 7f1a55531700 1 --2- 172.21.15.71:0/949131775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a50100cb0 0x7f1a501010d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009910 tx=0x7f1a40009c20 comp rx=0 tx=0).stop 2024-04-23T02:11:42.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.001+0000 7f1a55531700 1 -- 172.21.15.71:0/949131775 shutdown_connections 2024-04-23T02:11:42.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.001+0000 7f1a55531700 1 --2- 172.21.15.71:0/949131775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a50101eb0 0x7f1a50102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:42.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.001+0000 7f1a55531700 1 --2- 172.21.15.71:0/949131775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a50100cb0 0x7f1a501010d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:42.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.001+0000 7f1a55531700 1 -- 172.21.15.71:0/949131775 >> 172.21.15.71:0/949131775 conn(0x7f1a500fc230 msgr2=0x7f1a500fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:42.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.001+0000 7f1a55531700 1 -- 172.21.15.71:0/949131775 shutdown_connections 2024-04-23T02:11:42.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.001+0000 7f1a55531700 1 -- 172.21.15.71:0/949131775 wait complete. 2024-04-23T02:11:42.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.002+0000 7f1a55531700 1 Processor -- start 2024-04-23T02:11:42.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.002+0000 7f1a55531700 1 -- start start 2024-04-23T02:11:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.002+0000 7f1a55531700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a50100cb0 0x7f1a50196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.002+0000 7f1a55531700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a50101eb0 0x7f1a50196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.002+0000 7f1a55531700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a50197070 con 0x7f1a50100cb0 2024-04-23T02:11:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.002+0000 7f1a55531700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a501971b0 con 0x7f1a50101eb0 2024-04-23T02:11:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.003+0000 7f1a4effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a50100cb0 0x7f1a50196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.003+0000 7f1a4effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a50100cb0 0x7f1a50196510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33662/0 (socket says 172.21.15.71:33662) 2024-04-23T02:11:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.003+0000 7f1a4effd700 1 -- 172.21.15.71:0/211238440 learned_addr learned my addr 172.21.15.71:0/211238440 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:11:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.003+0000 7f1a4effd700 1 -- 172.21.15.71:0/211238440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a50101eb0 msgr2=0x7f1a50196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:42.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.003+0000 7f1a4effd700 1 --2- 172.21.15.71:0/211238440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a50101eb0 0x7f1a50196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:42.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.003+0000 7f1a4effd700 1 -- 172.21.15.71:0/211238440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a400095f0 con 0x7f1a50100cb0 2024-04-23T02:11:42.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.003+0000 7f1a4effd700 1 --2- 172.21.15.71:0/211238440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a50100cb0 0x7f1a50196510 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000bda0 tx=0x7f1a4000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:42.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.003+0000 7f1a37fff700 1 -- 172.21.15.71:0/211238440 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a4001d070 con 0x7f1a50100cb0 2024-04-23T02:11:42.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.003+0000 7f1a55531700 1 -- 172.21.15.71:0/211238440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5019bc00 con 0x7f1a50100cb0 2024-04-23T02:11:42.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.003+0000 7f1a37fff700 1 -- 172.21.15.71:0/211238440 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1a40022470 con 0x7f1a50100cb0 2024-04-23T02:11:42.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.003+0000 7f1a37fff700 1 -- 172.21.15.71:0/211238440 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a4000f690 con 0x7f1a50100cb0 2024-04-23T02:11:42.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.003+0000 7f1a55531700 1 -- 172.21.15.71:0/211238440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5019c0f0 con 0x7f1a50100cb0 2024-04-23T02:11:42.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.004+0000 7f1a55531700 1 -- 172.21.15.71:0/211238440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a50109750 con 0x7f1a50100cb0 2024-04-23T02:11:42.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.005+0000 7f1a37fff700 1 -- 172.21.15.71:0/211238440 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f1a400229c0 con 0x7f1a50100cb0 2024-04-23T02:11:42.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.005+0000 7f1a37fff700 1 --2- 172.21.15.71:0/211238440 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1a3806c4e0 0x7f1a3806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:42.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.005+0000 7f1a37fff700 1 -- 172.21.15.71:0/211238440 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f1a4008cf80 con 0x7f1a50100cb0 2024-04-23T02:11:42.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.005+0000 7f1a4e7fc700 1 --2- 172.21.15.71:0/211238440 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1a3806c4e0 0x7f1a3806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:42.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.005+0000 7f1a4e7fc700 1 --2- 172.21.15.71:0/211238440 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1a3806c4e0 0x7f1a3806e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1a440097b0 tx=0x7f1a44006d20 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:42.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.010+0000 7f1a37fff700 1 -- 172.21.15.71:0/211238440 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1a4005b360 con 0x7f1a50100cb0 2024-04-23T02:11:42.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.200+0000 7f1a55531700 1 -- 172.21.15.71:0/211238440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f1a50109940 con 0x7f1a50100cb0 2024-04-23T02:11:42.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.200+0000 7f1a37fff700 1 -- 172.21.15.71:0/211238440 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f1a4005aef0 con 0x7f1a50100cb0 2024-04-23T02:11:42.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.202+0000 7f1a55531700 1 -- 172.21.15.71:0/211238440 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1a3806c4e0 msgr2=0x7f1a3806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:42.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.202+0000 7f1a55531700 1 --2- 172.21.15.71:0/211238440 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1a3806c4e0 0x7f1a3806e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1a440097b0 tx=0x7f1a44006d20 comp rx=0 tx=0).stop 2024-04-23T02:11:42.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.202+0000 7f1a55531700 1 -- 172.21.15.71:0/211238440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a50100cb0 msgr2=0x7f1a50196510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:42.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.202+0000 7f1a55531700 1 --2- 172.21.15.71:0/211238440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a50100cb0 0x7f1a50196510 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000bda0 tx=0x7f1a4000be80 comp rx=0 tx=0).stop 2024-04-23T02:11:42.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.202+0000 7f1a55531700 1 -- 172.21.15.71:0/211238440 shutdown_connections 2024-04-23T02:11:42.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.202+0000 7f1a55531700 1 --2- 172.21.15.71:0/211238440 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f1a3806c4e0 0x7f1a3806e9a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:42.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.202+0000 7f1a55531700 1 --2- 172.21.15.71:0/211238440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a50100cb0 0x7f1a50196510 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:42.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.202+0000 7f1a55531700 1 --2- 172.21.15.71:0/211238440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a50101eb0 0x7f1a50196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:42.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.202+0000 7f1a55531700 1 -- 172.21.15.71:0/211238440 >> 172.21.15.71:0/211238440 conn(0x7f1a500fc230 msgr2=0x7f1a501050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:42.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.203+0000 7f1a55531700 1 -- 172.21.15.71:0/211238440 shutdown_connections 2024-04-23T02:11:42.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:42.203+0000 7f1a55531700 1 -- 172.21.15.71:0/211238440 wait complete. 2024-04-23T02:11:42.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:11:42.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:42 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:11:42.942 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-23T02:11:43.171 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:11:43.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:43 smithi071 ceph-mon[39837]: pgmap v214: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-23T02:11:43.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:43 smithi192 ceph-mon[49662]: pgmap v214: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-23T02:11:45.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.056+0000 7ff7a8081700 1 -- 172.21.15.71:0/1600337871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7a0101f40 msgr2=0x7ff7a01023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:45.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.056+0000 7ff7a8081700 1 --2- 172.21.15.71:0/1600337871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7a0101f40 0x7ff7a01023a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009910 tx=0x7ff79c009c20 comp rx=0 tx=0).stop 2024-04-23T02:11:45.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.056+0000 7ff7a8081700 1 -- 172.21.15.71:0/1600337871 shutdown_connections 2024-04-23T02:11:45.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.056+0000 7ff7a8081700 1 --2- 172.21.15.71:0/1600337871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7a0101f40 0x7ff7a01023a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:45.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.056+0000 7ff7a8081700 1 --2- 172.21.15.71:0/1600337871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7a00fef60 0x7ff7a00ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:45.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.056+0000 7ff7a8081700 1 -- 172.21.15.71:0/1600337871 >> 172.21.15.71:0/1600337871 conn(0x7ff7a00fa8c0 msgr2=0x7ff7a00fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:45.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.056+0000 7ff7a8081700 1 -- 172.21.15.71:0/1600337871 shutdown_connections 2024-04-23T02:11:45.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.056+0000 7ff7a8081700 1 -- 172.21.15.71:0/1600337871 wait complete. 2024-04-23T02:11:45.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.057+0000 7ff7a8081700 1 Processor -- start 2024-04-23T02:11:45.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.057+0000 7ff7a8081700 1 -- start start 2024-04-23T02:11:45.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.057+0000 7ff7a8081700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7a00fef60 0x7ff7a0196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:45.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.057+0000 7ff7a8081700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7a0101f40 0x7ff7a0196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:45.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.057+0000 7ff7a8081700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a0197060 con 0x7ff7a00fef60 2024-04-23T02:11:45.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.057+0000 7ff7a8081700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a01971a0 con 0x7ff7a0101f40 2024-04-23T02:11:45.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.057+0000 7ff7a5e1d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7a00fef60 0x7ff7a0196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:45.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.057+0000 7ff7a5e1d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7a00fef60 0x7ff7a0196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33688/0 (socket says 172.21.15.71:33688) 2024-04-23T02:11:45.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.057+0000 7ff7a5e1d700 1 -- 172.21.15.71:0/2371330018 learned_addr learned my addr 172.21.15.71:0/2371330018 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:11:45.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.058+0000 7ff7a5e1d700 1 -- 172.21.15.71:0/2371330018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7a0101f40 msgr2=0x7ff7a0196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:45.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.058+0000 7ff7a561c700 1 --2- 172.21.15.71:0/2371330018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7a0101f40 0x7ff7a0196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:45.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.058+0000 7ff7a5e1d700 1 --2- 172.21.15.71:0/2371330018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7a0101f40 0x7ff7a0196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:45.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.058+0000 7ff7a5e1d700 1 -- 172.21.15.71:0/2371330018 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79c0095f0 con 0x7ff7a00fef60 2024-04-23T02:11:45.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.058+0000 7ff7a5e1d700 1 --2- 172.21.15.71:0/2371330018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7a00fef60 0x7ff7a0196500 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff79000b480 tx=0x7ff79000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:45.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.058+0000 7ff796ffd700 1 -- 172.21.15.71:0/2371330018 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7900118c0 con 0x7ff7a00fef60 2024-04-23T02:11:45.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.058+0000 7ff796ffd700 1 -- 172.21.15.71:0/2371330018 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ff790010380 con 0x7ff7a00fef60 2024-04-23T02:11:45.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.058+0000 7ff796ffd700 1 -- 172.21.15.71:0/2371330018 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff79000f510 con 0x7ff7a00fef60 2024-04-23T02:11:45.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.058+0000 7ff7a8081700 1 -- 172.21.15.71:0/2371330018 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a019bc50 con 0x7ff7a00fef60 2024-04-23T02:11:45.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.058+0000 7ff7a8081700 1 -- 172.21.15.71:0/2371330018 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a019c1a0 con 0x7ff7a00fef60 2024-04-23T02:11:45.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.059+0000 7ff7a8081700 1 -- 172.21.15.71:0/2371330018 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a0109760 con 0x7ff7a00fef60 2024-04-23T02:11:45.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.059+0000 7ff796ffd700 1 -- 172.21.15.71:0/2371330018 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7ff7900104f0 con 0x7ff7a00fef60 2024-04-23T02:11:45.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.060+0000 7ff796ffd700 1 --2- 172.21.15.71:0/2371330018 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7ff78c06c4e0 0x7ff78c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:45.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.060+0000 7ff796ffd700 1 -- 172.21.15.71:0/2371330018 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff79008b5b0 con 0x7ff7a00fef60 2024-04-23T02:11:45.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.060+0000 7ff7a561c700 1 --2- 172.21.15.71:0/2371330018 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7ff78c06c4e0 0x7ff78c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:45.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.060+0000 7ff7a561c700 1 --2- 172.21.15.71:0/2371330018 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7ff78c06c4e0 0x7ff78c06e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00b5c0 tx=0x7ff79c005960 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:45.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.064+0000 7ff796ffd700 1 -- 172.21.15.71:0/2371330018 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff790059720 con 0x7ff7a00fef60 2024-04-23T02:11:45.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.255+0000 7ff7a8081700 1 -- 172.21.15.71:0/2371330018 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7ff7a0109920 con 0x7ff7a00fef60 2024-04-23T02:11:45.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.256+0000 7ff796ffd700 1 -- 172.21.15.71:0/2371330018 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7ff7900107a0 con 0x7ff7a00fef60 2024-04-23T02:11:45.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.258+0000 7ff7a8081700 1 -- 172.21.15.71:0/2371330018 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7ff78c06c4e0 msgr2=0x7ff78c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:45.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.258+0000 7ff7a8081700 1 --2- 172.21.15.71:0/2371330018 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7ff78c06c4e0 0x7ff78c06e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00b5c0 tx=0x7ff79c005960 comp rx=0 tx=0).stop 2024-04-23T02:11:45.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.258+0000 7ff7a8081700 1 -- 172.21.15.71:0/2371330018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7a00fef60 msgr2=0x7ff7a0196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:45.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.258+0000 7ff7a8081700 1 --2- 172.21.15.71:0/2371330018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7a00fef60 0x7ff7a0196500 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff79000b480 tx=0x7ff79000b840 comp rx=0 tx=0).stop 2024-04-23T02:11:45.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.258+0000 7ff7a8081700 1 -- 172.21.15.71:0/2371330018 shutdown_connections 2024-04-23T02:11:45.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.258+0000 7ff7a8081700 1 --2- 172.21.15.71:0/2371330018 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7ff78c06c4e0 0x7ff78c06e9a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:45.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.258+0000 7ff7a8081700 1 --2- 172.21.15.71:0/2371330018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7a00fef60 0x7ff7a0196500 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:45.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.258+0000 7ff7a8081700 1 --2- 172.21.15.71:0/2371330018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7a0101f40 0x7ff7a0196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:45.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.258+0000 7ff7a8081700 1 -- 172.21.15.71:0/2371330018 >> 172.21.15.71:0/2371330018 conn(0x7ff7a00fa8c0 msgr2=0x7ff7a0105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:45.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.258+0000 7ff7a8081700 1 -- 172.21.15.71:0/2371330018 shutdown_connections 2024-04-23T02:11:45.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:45.258+0000 7ff7a8081700 1 -- 172.21.15.71:0/2371330018 wait complete. 2024-04-23T02:11:45.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:45 smithi192 ceph-mon[49662]: pgmap v215: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 682 B/s wr, 4 op/s 2024-04-23T02:11:45.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:45 smithi071 ceph-mon[39837]: pgmap v215: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 682 B/s wr, 4 op/s 2024-04-23T02:11:46.046 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-04-23T02:11:46.273 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:11:46.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:46 smithi071 ceph-mon[39837]: pgmap v216: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-23T02:11:47.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:46 smithi192 ceph-mon[49662]: pgmap v216: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-23T02:11:48.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.256+0000 7f842c705700 1 -- 172.21.15.71:0/1422302290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8424100cb0 msgr2=0x7f84241010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:48.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.256+0000 7f842c705700 1 --2- 172.21.15.71:0/1422302290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8424100cb0 0x7f84241010d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8420009910 tx=0x7f8420009c20 comp rx=0 tx=0).stop 2024-04-23T02:11:48.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.256+0000 7f842c705700 1 -- 172.21.15.71:0/1422302290 shutdown_connections 2024-04-23T02:11:48.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.256+0000 7f842c705700 1 --2- 172.21.15.71:0/1422302290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8424101eb0 0x7f8424102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:48.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.256+0000 7f842c705700 1 --2- 172.21.15.71:0/1422302290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8424100cb0 0x7f84241010d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:48.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.256+0000 7f842c705700 1 -- 172.21.15.71:0/1422302290 >> 172.21.15.71:0/1422302290 conn(0x7f84240fc230 msgr2=0x7f84240fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:11:48.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.256+0000 7f842c705700 1 -- 172.21.15.71:0/1422302290 shutdown_connections 2024-04-23T02:11:48.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.256+0000 7f842c705700 1 -- 172.21.15.71:0/1422302290 wait complete. 2024-04-23T02:11:48.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.257+0000 7f842c705700 1 Processor -- start 2024-04-23T02:11:48.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.257+0000 7f842c705700 1 -- start start 2024-04-23T02:11:48.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.257+0000 7f842c705700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8424100cb0 0x7f84241964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:48.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.257+0000 7f842c705700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8424101eb0 0x7f8424196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:48.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.257+0000 7f842c705700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8424197040 con 0x7f8424101eb0 2024-04-23T02:11:48.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.257+0000 7f842c705700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8424197180 con 0x7f8424100cb0 2024-04-23T02:11:48.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.257+0000 7f8429ca0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8424101eb0 0x7f8424196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.257+0000 7f8429ca0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8424101eb0 0x7f8424196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50306/0 (socket says 172.21.15.71:50306) 2024-04-23T02:11:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.257+0000 7f8429ca0700 1 -- 172.21.15.71:0/3779794037 learned_addr learned my addr 172.21.15.71:0/3779794037 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:11:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.257+0000 7f842a4a1700 1 --2- 172.21.15.71:0/3779794037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8424100cb0 0x7f84241964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.257+0000 7f8429ca0700 1 -- 172.21.15.71:0/3779794037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8424100cb0 msgr2=0x7f84241964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:11:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.257+0000 7f8429ca0700 1 --2- 172.21.15.71:0/3779794037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8424100cb0 0x7f84241964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:11:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.257+0000 7f8429ca0700 1 -- 172.21.15.71:0/3779794037 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84200095f0 con 0x7f8424101eb0 2024-04-23T02:11:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.257+0000 7f8429ca0700 1 --2- 172.21.15.71:0/3779794037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8424101eb0 0x7f8424196a20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f841400b4b0 tx=0x7f841400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.258+0000 7f841b7fe700 1 -- 172.21.15.71:0/3779794037 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8414011800 con 0x7f8424101eb0 2024-04-23T02:11:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.258+0000 7f842c705700 1 -- 172.21.15.71:0/3779794037 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842419bc30 con 0x7f8424101eb0 2024-04-23T02:11:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.258+0000 7f841b7fe700 1 -- 172.21.15.71:0/3779794037 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f8414011df0 con 0x7f8424101eb0 2024-04-23T02:11:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.258+0000 7f841b7fe700 1 -- 172.21.15.71:0/3779794037 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8414010e40 con 0x7f8424101eb0 2024-04-23T02:11:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.258+0000 7f842c705700 1 -- 172.21.15.71:0/3779794037 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842419c180 con 0x7f8424101eb0 2024-04-23T02:11:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.258+0000 7f842c705700 1 -- 172.21.15.71:0/3779794037 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8424109750 con 0x7f8424101eb0 2024-04-23T02:11:48.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.259+0000 7f841b7fe700 1 -- 172.21.15.71:0/3779794037 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f8414010380 con 0x7f8424101eb0 2024-04-23T02:11:48.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.259+0000 7f841b7fe700 1 --2- 172.21.15.71:0/3779794037 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f841006c490 0x7f841006e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:11:48.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.260+0000 7f841b7fe700 1 -- 172.21.15.71:0/3779794037 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f841408b490 con 0x7f8424101eb0 2024-04-23T02:11:48.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.260+0000 7f842a4a1700 1 --2- 172.21.15.71:0/3779794037 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f841006c490 0x7f841006e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:11:48.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.260+0000 7f842a4a1700 1 --2- 172.21.15.71:0/3779794037 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f841006c490 0x7f841006e950 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8420006010 tx=0x7f842000b580 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:11:48.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.264+0000 7f841b7fe700 1 -- 172.21.15.71:0/3779794037 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f84140596a0 con 0x7f8424101eb0 2024-04-23T02:11:48.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:11:48.472+0000 7f842c705700 1 -- 172.21.15.71:0/3779794037 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f8424106800 con 0x7f841006c490 2024-04-23T02:11:49.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:49 smithi071 ceph-mon[39837]: pgmap v217: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 597 B/s wr, 4 op/s 2024-04-23T02:11:49.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:49 smithi071 ceph-mon[39837]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:11:49.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:49 smithi192 ceph-mon[49662]: pgmap v217: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 597 B/s wr, 4 op/s 2024-04-23T02:11:49.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:49 smithi192 ceph-mon[49662]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:11:51.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:51 smithi071 ceph-mon[39837]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-23T02:11:51.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:51 smithi192 ceph-mon[49662]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-23T02:11:53.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:53 smithi071 ceph-mon[39837]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 682 B/s wr, 4 op/s 2024-04-23T02:11:53.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:53 smithi192 ceph-mon[49662]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 682 B/s wr, 4 op/s 2024-04-23T02:11:55.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:55 smithi071 ceph-mon[39837]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 3 op/s 2024-04-23T02:11:55.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:55 smithi192 ceph-mon[49662]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 3 op/s 2024-04-23T02:11:56.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:56 smithi071 ceph-mon[39837]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-23T02:11:57.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:56 smithi192 ceph-mon[49662]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-23T02:11:57.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:11:58.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:11:58.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:11:58 smithi071 ceph-mon[39837]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-23T02:11:59.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:11:58 smithi192 ceph-mon[49662]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-23T02:12:01.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:01 smithi071 ceph-mon[39837]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-23T02:12:01.630 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:01 smithi192 ceph-mon[49662]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-23T02:12:03.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:03 smithi071 ceph-mon[39837]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 4 op/s 2024-04-23T02:12:03.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:03 smithi192 ceph-mon[49662]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 4 op/s 2024-04-23T02:12:05.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:05 smithi071 ceph-mon[39837]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:05.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:05 smithi192 ceph-mon[49662]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:06 smithi071 ceph-mon[39837]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:12:07.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:06 smithi192 ceph-mon[49662]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:12:09.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:09 smithi071 ceph-mon[39837]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:09.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:09 smithi192 ceph-mon[49662]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:11.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:11 smithi071 ceph-mon[39837]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:11.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:11 smithi192 ceph-mon[49662]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:12.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:12 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:12:12.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:12 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:12:13.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:13 smithi071 ceph-mon[39837]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:13.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:13 smithi192 ceph-mon[49662]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:15.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:15 smithi071 ceph-mon[39837]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:15.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:15 smithi192 ceph-mon[49662]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:16.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:16 smithi071 ceph-mon[39837]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:12:16.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:16.980+0000 7f841b7fe700 1 -- 172.21.15.71:0/3779794037 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f8424106800 con 0x7f841006c490 2024-04-23T02:12:16.985 INFO:teuthology.orchestra.run.smithi071.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T02:12:16.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:16.982+0000 7f842c705700 1 -- 172.21.15.71:0/3779794037 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f841006c490 msgr2=0x7f841006e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:16.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:16.982+0000 7f842c705700 1 --2- 172.21.15.71:0/3779794037 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f841006c490 0x7f841006e950 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8420006010 tx=0x7f842000b580 comp rx=0 tx=0).stop 2024-04-23T02:12:16.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:16.982+0000 7f842c705700 1 -- 172.21.15.71:0/3779794037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8424101eb0 msgr2=0x7f8424196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:16.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:16.982+0000 7f842c705700 1 --2- 172.21.15.71:0/3779794037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8424101eb0 0x7f8424196a20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f841400b4b0 tx=0x7f841400b870 comp rx=0 tx=0).stop 2024-04-23T02:12:16.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:16.982+0000 7f842c705700 1 -- 172.21.15.71:0/3779794037 shutdown_connections 2024-04-23T02:12:16.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:16.982+0000 7f842c705700 1 --2- 172.21.15.71:0/3779794037 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f841006c490 0x7f841006e950 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:16.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:16.982+0000 7f842c705700 1 --2- 172.21.15.71:0/3779794037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8424100cb0 0x7f84241964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:16.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:16.982+0000 7f842c705700 1 --2- 172.21.15.71:0/3779794037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8424101eb0 0x7f8424196a20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:16.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:16.982+0000 7f842c705700 1 -- 172.21.15.71:0/3779794037 >> 172.21.15.71:0/3779794037 conn(0x7f84240fc230 msgr2=0x7f84241050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:16.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:16.982+0000 7f842c705700 1 -- 172.21.15.71:0/3779794037 shutdown_connections 2024-04-23T02:12:16.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:16.982+0000 7f842c705700 1 -- 172.21.15.71:0/3779794037 wait complete. 2024-04-23T02:12:17.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:16 smithi192 ceph-mon[49662]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:12:17.818 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-04-23T02:12:18.152 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:17 smithi071 ceph-mon[39837]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T02:12:18.152 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:17 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:12:18.152 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:17 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:12:18.153 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:17 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:12:18.153 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:17 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:12:18.153 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:17 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:12:18.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:17 smithi192 ceph-mon[49662]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T02:12:18.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:17 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:12:18.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:17 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:12:18.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:17 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:12:18.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:17 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:12:18.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:17 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:12:18.524 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:12:19.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:18 smithi071 ceph-mon[39837]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T02:12:19.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:18 smithi071 ceph-mon[39837]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:19.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:18 smithi192 ceph-mon[49662]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T02:12:19.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:18 smithi192 ceph-mon[49662]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:20 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:12:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:20 smithi192 ceph-mon[49662]: Upgrade: Target is version 18.2.2-1237-g646b8c7d (reef) 2024-04-23T02:12:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:20 smithi192 ceph-mon[49662]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, digests ['quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210'] 2024-04-23T02:12:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:20 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:12:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:20 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:12:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:20 smithi192 ceph-mon[49662]: Upgrade: Need to upgrade myself (mgr.smithi071.ukdjvn) 2024-04-23T02:12:20.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:20 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:12:20.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:20 smithi071 ceph-mon[39837]: Upgrade: Target is version 18.2.2-1237-g646b8c7d (reef) 2024-04-23T02:12:20.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:20 smithi071 ceph-mon[39837]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, digests ['quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210'] 2024-04-23T02:12:20.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:20 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:12:20.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:20 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:12:20.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:20 smithi071 ceph-mon[39837]: Upgrade: Need to upgrade myself (mgr.smithi071.ukdjvn) 2024-04-23T02:12:21.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.448+0000 7f4ceea6a700 1 -- 172.21.15.71:0/2090453299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ce8101170 msgr2=0x7f4ce8101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:21.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.448+0000 7f4ceea6a700 1 --2- 172.21.15.71:0/2090453299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ce8101170 0x7f4ce8101590 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009910 tx=0x7f4cdc009c20 comp rx=0 tx=0).stop 2024-04-23T02:12:21.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.448+0000 7f4ceea6a700 1 -- 172.21.15.71:0/2090453299 shutdown_connections 2024-04-23T02:12:21.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.448+0000 7f4ceea6a700 1 --2- 172.21.15.71:0/2090453299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ce80fed00 0x7f4ce80ff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:21.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.448+0000 7f4ceea6a700 1 --2- 172.21.15.71:0/2090453299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ce8101170 0x7f4ce8101590 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:21.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.448+0000 7f4ceea6a700 1 -- 172.21.15.71:0/2090453299 >> 172.21.15.71:0/2090453299 conn(0x7f4ce80fa8c0 msgr2=0x7f4ce80fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:21.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.448+0000 7f4ceea6a700 1 -- 172.21.15.71:0/2090453299 shutdown_connections 2024-04-23T02:12:21.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.448+0000 7f4ceea6a700 1 -- 172.21.15.71:0/2090453299 wait complete. 2024-04-23T02:12:21.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.449+0000 7f4ceea6a700 1 Processor -- start 2024-04-23T02:12:21.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.449+0000 7f4ceea6a700 1 -- start start 2024-04-23T02:12:21.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.449+0000 7f4ceea6a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ce80fed00 0x7f4ce8198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:21.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.449+0000 7f4ceea6a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ce8101170 0x7f4ce8198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:21.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.449+0000 7f4ceea6a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce8199190 con 0x7f4ce8101170 2024-04-23T02:12:21.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.449+0000 7f4ceea6a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce81992d0 con 0x7f4ce80fed00 2024-04-23T02:12:21.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.449+0000 7f4ce7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ce8101170 0x7f4ce8198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:21.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.449+0000 7f4ce7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ce8101170 0x7f4ce8198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43370/0 (socket says 172.21.15.71:43370) 2024-04-23T02:12:21.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.449+0000 7f4ce7fff700 1 -- 172.21.15.71:0/409064077 learned_addr learned my addr 172.21.15.71:0/409064077 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:12:21.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.449+0000 7f4cec806700 1 --2- 172.21.15.71:0/409064077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ce80fed00 0x7f4ce8198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:21.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.449+0000 7f4ce7fff700 1 -- 172.21.15.71:0/409064077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ce80fed00 msgr2=0x7f4ce8198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:21.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.449+0000 7f4ce7fff700 1 --2- 172.21.15.71:0/409064077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ce80fed00 0x7f4ce8198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:21.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.449+0000 7f4ce7fff700 1 -- 172.21.15.71:0/409064077 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd4009520 con 0x7f4ce8101170 2024-04-23T02:12:21.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.450+0000 7f4ce7fff700 1 --2- 172.21.15.71:0/409064077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ce8101170 0x7f4ce8198b70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400eab0 tx=0x7f4cd400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:21.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.450+0000 7f4ce5ffb700 1 -- 172.21.15.71:0/409064077 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cd400cba0 con 0x7f4ce8101170 2024-04-23T02:12:21.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.450+0000 7f4ceea6a700 1 -- 172.21.15.71:0/409064077 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cdc0095f0 con 0x7f4ce8101170 2024-04-23T02:12:21.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.450+0000 7f4ce5ffb700 1 -- 172.21.15.71:0/409064077 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4cd40044b0 con 0x7f4ce8101170 2024-04-23T02:12:21.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.450+0000 7f4ce5ffb700 1 -- 172.21.15.71:0/409064077 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cd4005020 con 0x7f4ce8101170 2024-04-23T02:12:21.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.450+0000 7f4ceea6a700 1 -- 172.21.15.71:0/409064077 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce819e140 con 0x7f4ce8101170 2024-04-23T02:12:21.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.451+0000 7f4ceea6a700 1 -- 172.21.15.71:0/409064077 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce810b8b0 con 0x7f4ce8101170 2024-04-23T02:12:21.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.452+0000 7f4ce5ffb700 1 -- 172.21.15.71:0/409064077 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f4cd4004750 con 0x7f4ce8101170 2024-04-23T02:12:21.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.452+0000 7f4ce5ffb700 1 --2- 172.21.15.71:0/409064077 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4cd806c4e0 0x7f4cd806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:21.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.452+0000 7f4ce5ffb700 1 -- 172.21.15.71:0/409064077 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4cd4014070 con 0x7f4ce8101170 2024-04-23T02:12:21.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.452+0000 7f4cec806700 1 --2- 172.21.15.71:0/409064077 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4cd806c4e0 0x7f4cd806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:21.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.453+0000 7f4cec806700 1 --2- 172.21.15.71:0/409064077 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4cd806c4e0 0x7f4cd806e9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009fd0 tx=0x7f4cdc00b5a0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:21.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.460+0000 7f4ce5ffb700 1 -- 172.21.15.71:0/409064077 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4cd4059a40 con 0x7f4ce8101170 2024-04-23T02:12:21.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:21 smithi071 ceph-mon[39837]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 on smithi192 2024-04-23T02:12:21.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:21 smithi071 ceph-mon[39837]: pgmap v233: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:21.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:21 smithi192 ceph-mon[49662]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 on smithi192 2024-04-23T02:12:21.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:21 smithi192 ceph-mon[49662]: pgmap v233: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:21.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.783+0000 7f4ceea6a700 1 -- 172.21.15.71:0/409064077 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ce819e810 con 0x7f4cd806c4e0 2024-04-23T02:12:21.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.785+0000 7f4ce5ffb700 1 -- 172.21.15.71:0/409064077 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f4ce819e810 con 0x7f4cd806c4e0 2024-04-23T02:12:21.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.788+0000 7f4ceea6a700 1 -- 172.21.15.71:0/409064077 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4cd806c4e0 msgr2=0x7f4cd806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:21.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.788+0000 7f4ceea6a700 1 --2- 172.21.15.71:0/409064077 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4cd806c4e0 0x7f4cd806e9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009fd0 tx=0x7f4cdc00b5a0 comp rx=0 tx=0).stop 2024-04-23T02:12:21.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.788+0000 7f4ceea6a700 1 -- 172.21.15.71:0/409064077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ce8101170 msgr2=0x7f4ce8198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:21.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.788+0000 7f4ceea6a700 1 --2- 172.21.15.71:0/409064077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ce8101170 0x7f4ce8198b70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400eab0 tx=0x7f4cd400edc0 comp rx=0 tx=0).stop 2024-04-23T02:12:21.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.788+0000 7f4ceea6a700 1 -- 172.21.15.71:0/409064077 shutdown_connections 2024-04-23T02:12:21.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.788+0000 7f4ceea6a700 1 --2- 172.21.15.71:0/409064077 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f4cd806c4e0 0x7f4cd806e9a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:21.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.788+0000 7f4ceea6a700 1 --2- 172.21.15.71:0/409064077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ce80fed00 0x7f4ce8198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:21.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.788+0000 7f4ceea6a700 1 --2- 172.21.15.71:0/409064077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ce8101170 0x7f4ce8198b70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:21.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.788+0000 7f4ceea6a700 1 -- 172.21.15.71:0/409064077 >> 172.21.15.71:0/409064077 conn(0x7f4ce80fa8c0 msgr2=0x7f4ce8108400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:21.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.788+0000 7f4ceea6a700 1 -- 172.21.15.71:0/409064077 shutdown_connections 2024-04-23T02:12:21.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.788+0000 7f4ceea6a700 1 -- 172.21.15.71:0/409064077 wait complete. 2024-04-23T02:12:21.806 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:12:21.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.925+0000 7f9bc86ad700 1 -- 172.21.15.71:0/3189564545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bc0100e10 msgr2=0x7f9bc0101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:21.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.925+0000 7f9bc86ad700 1 --2- 172.21.15.71:0/3189564545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bc0100e10 0x7f9bc0101270 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009910 tx=0x7f9bbc009c20 comp rx=0 tx=0).stop 2024-04-23T02:12:21.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.925+0000 7f9bc86ad700 1 -- 172.21.15.71:0/3189564545 shutdown_connections 2024-04-23T02:12:21.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.925+0000 7f9bc86ad700 1 --2- 172.21.15.71:0/3189564545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bc0100e10 0x7f9bc0101270 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:21.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.925+0000 7f9bc86ad700 1 --2- 172.21.15.71:0/3189564545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bc00ffc10 0x7f9bc0100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:21.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.925+0000 7f9bc86ad700 1 -- 172.21.15.71:0/3189564545 >> 172.21.15.71:0/3189564545 conn(0x7f9bc00fb190 msgr2=0x7f9bc00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:21.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.925+0000 7f9bc86ad700 1 -- 172.21.15.71:0/3189564545 shutdown_connections 2024-04-23T02:12:21.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.925+0000 7f9bc86ad700 1 -- 172.21.15.71:0/3189564545 wait complete. 2024-04-23T02:12:21.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.926+0000 7f9bc86ad700 1 Processor -- start 2024-04-23T02:12:21.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.926+0000 7f9bc86ad700 1 -- start start 2024-04-23T02:12:21.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.926+0000 7f9bc86ad700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bc00ffc10 0x7f9bc01963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:21.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.926+0000 7f9bc86ad700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bc0100e10 0x7f9bc0196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:21.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.926+0000 7f9bc86ad700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc0196f20 con 0x7f9bc0100e10 2024-04-23T02:12:21.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.926+0000 7f9bc86ad700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc0197060 con 0x7f9bc00ffc10 2024-04-23T02:12:21.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.926+0000 7f9bc6449700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bc00ffc10 0x7f9bc01963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:21.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.926+0000 7f9bc6449700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bc00ffc10 0x7f9bc01963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.71:53888/0 (socket says 172.21.15.71:53888) 2024-04-23T02:12:21.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.926+0000 7f9bc6449700 1 -- 172.21.15.71:0/2952620402 learned_addr learned my addr 172.21.15.71:0/2952620402 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:12:21.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.926+0000 7f9bc5c48700 1 --2- 172.21.15.71:0/2952620402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bc0100e10 0x7f9bc0196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:21.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.926+0000 7f9bc6449700 1 -- 172.21.15.71:0/2952620402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bc0100e10 msgr2=0x7f9bc0196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:21.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.926+0000 7f9bc6449700 1 --2- 172.21.15.71:0/2952620402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bc0100e10 0x7f9bc0196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:21.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.926+0000 7f9bc6449700 1 -- 172.21.15.71:0/2952620402 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc0095f0 con 0x7f9bc00ffc10 2024-04-23T02:12:21.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.927+0000 7f9bc6449700 1 --2- 172.21.15.71:0/2952620402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bc00ffc10 0x7f9bc01963c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400e8f0 tx=0x7f9bb400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:21.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.927+0000 7f9bb37fe700 1 -- 172.21.15.71:0/2952620402 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bb4009790 con 0x7f9bc00ffc10 2024-04-23T02:12:21.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.927+0000 7f9bb37fe700 1 -- 172.21.15.71:0/2952620402 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f9bb40044b0 con 0x7f9bc00ffc10 2024-04-23T02:12:21.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.927+0000 7f9bc86ad700 1 -- 172.21.15.71:0/2952620402 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc006a950 con 0x7f9bc00ffc10 2024-04-23T02:12:21.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.927+0000 7f9bc86ad700 1 -- 172.21.15.71:0/2952620402 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc006aea0 con 0x7f9bc00ffc10 2024-04-23T02:12:21.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.928+0000 7f9bb37fe700 1 -- 172.21.15.71:0/2952620402 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bb4003d30 con 0x7f9bc00ffc10 2024-04-23T02:12:21.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.929+0000 7f9bb37fe700 1 -- 172.21.15.71:0/2952620402 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f9bb4018450 con 0x7f9bc00ffc10 2024-04-23T02:12:21.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.929+0000 7f9bb37fe700 1 --2- 172.21.15.71:0/2952620402 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f9bac06c4e0 0x7f9bac06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:21.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.929+0000 7f9bb37fe700 1 -- 172.21.15.71:0/2952620402 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9bb4014070 con 0x7f9bc00ffc10 2024-04-23T02:12:21.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.929+0000 7f9bc5c48700 1 --2- 172.21.15.71:0/2952620402 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f9bac06c4e0 0x7f9bac06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:21.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.930+0000 7f9bc86ad700 1 -- 172.21.15.71:0/2952620402 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba4005320 con 0x7f9bc00ffc10 2024-04-23T02:12:21.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.930+0000 7f9bc5c48700 1 --2- 172.21.15.71:0/2952620402 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f9bac06c4e0 0x7f9bac06e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00b600 tx=0x7f9bbc009f70 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:21.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:21.938+0000 7f9bb37fe700 1 -- 172.21.15.71:0/2952620402 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9bb405a790 con 0x7f9bc00ffc10 2024-04-23T02:12:22.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.200+0000 7f9bc86ad700 1 -- 172.21.15.71:0/2952620402 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ba4000bf0 con 0x7f9bac06c4e0 2024-04-23T02:12:22.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.201+0000 7f9bb37fe700 1 -- 172.21.15.71:0/2952620402 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f9ba4000bf0 con 0x7f9bac06c4e0 2024-04-23T02:12:22.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.204+0000 7f9bc86ad700 1 -- 172.21.15.71:0/2952620402 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f9bac06c4e0 msgr2=0x7f9bac06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:22.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.204+0000 7f9bc86ad700 1 --2- 172.21.15.71:0/2952620402 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f9bac06c4e0 0x7f9bac06e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00b600 tx=0x7f9bbc009f70 comp rx=0 tx=0).stop 2024-04-23T02:12:22.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.204+0000 7f9bc86ad700 1 -- 172.21.15.71:0/2952620402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bc00ffc10 msgr2=0x7f9bc01963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:22.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.204+0000 7f9bc86ad700 1 --2- 172.21.15.71:0/2952620402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bc00ffc10 0x7f9bc01963c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400e8f0 tx=0x7f9bb400ecb0 comp rx=0 tx=0).stop 2024-04-23T02:12:22.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.204+0000 7f9bc86ad700 1 -- 172.21.15.71:0/2952620402 shutdown_connections 2024-04-23T02:12:22.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.204+0000 7f9bc86ad700 1 --2- 172.21.15.71:0/2952620402 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f9bac06c4e0 0x7f9bac06e9a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:22.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.204+0000 7f9bc86ad700 1 --2- 172.21.15.71:0/2952620402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bc00ffc10 0x7f9bc01963c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:22.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.204+0000 7f9bc86ad700 1 --2- 172.21.15.71:0/2952620402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bc0100e10 0x7f9bc0196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:22.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.204+0000 7f9bc86ad700 1 -- 172.21.15.71:0/2952620402 >> 172.21.15.71:0/2952620402 conn(0x7f9bc00fb190 msgr2=0x7f9bc0104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:22.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.204+0000 7f9bc86ad700 1 -- 172.21.15.71:0/2952620402 shutdown_connections 2024-04-23T02:12:22.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.204+0000 7f9bc86ad700 1 -- 172.21.15.71:0/2952620402 wait complete. 2024-04-23T02:12:22.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.328+0000 7f3729b10700 1 -- 172.21.15.71:0/3348806048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3724100dc0 msgr2=0x7f3724101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:22.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.328+0000 7f3729b10700 1 --2- 172.21.15.71:0/3348806048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3724100dc0 0x7f3724101220 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3718009910 tx=0x7f3718009c20 comp rx=0 tx=0).stop 2024-04-23T02:12:22.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.328+0000 7f3729b10700 1 -- 172.21.15.71:0/3348806048 shutdown_connections 2024-04-23T02:12:22.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.328+0000 7f3729b10700 1 --2- 172.21.15.71:0/3348806048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3724100dc0 0x7f3724101220 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:22.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.328+0000 7f3729b10700 1 --2- 172.21.15.71:0/3348806048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37240ffbc0 0x7f37240fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:22.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.328+0000 7f3729b10700 1 -- 172.21.15.71:0/3348806048 >> 172.21.15.71:0/3348806048 conn(0x7f37240fb160 msgr2=0x7f37240fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:22.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.328+0000 7f3729b10700 1 -- 172.21.15.71:0/3348806048 shutdown_connections 2024-04-23T02:12:22.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.328+0000 7f3729b10700 1 -- 172.21.15.71:0/3348806048 wait complete. 2024-04-23T02:12:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.328+0000 7f3729b10700 1 Processor -- start 2024-04-23T02:12:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.329+0000 7f3729b10700 1 -- start start 2024-04-23T02:12:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.329+0000 7f3729b10700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37240ffbc0 0x7f3724195630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.329+0000 7f3729b10700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3724100dc0 0x7f3724195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.329+0000 7f3729b10700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3724196190 con 0x7f37240ffbc0 2024-04-23T02:12:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.329+0000 7f3729b10700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37241962d0 con 0x7f3724100dc0 2024-04-23T02:12:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.329+0000 7f37237fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37240ffbc0 0x7f3724195630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.329+0000 7f37237fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37240ffbc0 0x7f3724195630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43410/0 (socket says 172.21.15.71:43410) 2024-04-23T02:12:22.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.329+0000 7f37237fe700 1 -- 172.21.15.71:0/2768624822 learned_addr learned my addr 172.21.15.71:0/2768624822 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:12:22.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.329+0000 7f3722ffd700 1 --2- 172.21.15.71:0/2768624822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3724100dc0 0x7f3724195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:22.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.329+0000 7f37237fe700 1 -- 172.21.15.71:0/2768624822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3724100dc0 msgr2=0x7f3724195b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:22.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.329+0000 7f37237fe700 1 --2- 172.21.15.71:0/2768624822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3724100dc0 0x7f3724195b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:22.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.329+0000 7f37237fe700 1 -- 172.21.15.71:0/2768624822 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37180095f0 con 0x7f37240ffbc0 2024-04-23T02:12:22.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.330+0000 7f37237fe700 1 --2- 172.21.15.71:0/2768624822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37240ffbc0 0x7f3724195630 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f371400e8f0 tx=0x7f371400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:22.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.330+0000 7f3720ff9700 1 -- 172.21.15.71:0/2768624822 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3714009790 con 0x7f37240ffbc0 2024-04-23T02:12:22.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.330+0000 7f3729b10700 1 -- 172.21.15.71:0/2768624822 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3724068fb0 con 0x7f37240ffbc0 2024-04-23T02:12:22.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.330+0000 7f3720ff9700 1 -- 172.21.15.71:0/2768624822 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f371400c810 con 0x7f37240ffbc0 2024-04-23T02:12:22.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.330+0000 7f3720ff9700 1 -- 172.21.15.71:0/2768624822 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3714003c20 con 0x7f37240ffbc0 2024-04-23T02:12:22.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.330+0000 7f3729b10700 1 -- 172.21.15.71:0/2768624822 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3724069500 con 0x7f37240ffbc0 2024-04-23T02:12:22.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.331+0000 7f3729b10700 1 -- 172.21.15.71:0/2768624822 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f372418f3f0 con 0x7f37240ffbc0 2024-04-23T02:12:22.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.331+0000 7f3720ff9700 1 -- 172.21.15.71:0/2768624822 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f371400c980 con 0x7f37240ffbc0 2024-04-23T02:12:22.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.332+0000 7f3720ff9700 1 --2- 172.21.15.71:0/2768624822 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f370c06c4e0 0x7f370c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:22.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.332+0000 7f3720ff9700 1 -- 172.21.15.71:0/2768624822 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f3714014070 con 0x7f37240ffbc0 2024-04-23T02:12:22.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.332+0000 7f3722ffd700 1 --2- 172.21.15.71:0/2768624822 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f370c06c4e0 0x7f370c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:22.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.332+0000 7f3722ffd700 1 --2- 172.21.15.71:0/2768624822 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f370c06c4e0 0x7f370c06e9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3718009fd0 tx=0x7f371800b5a0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:22.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.339+0000 7f3720ff9700 1 -- 172.21.15.71:0/2768624822 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3714059d60 con 0x7f37240ffbc0 2024-04-23T02:12:22.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.617+0000 7f3729b10700 1 -- 172.21.15.71:0/2768624822 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3724069780 con 0x7f370c06c4e0 2024-04-23T02:12:22.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.630+0000 7f3720ff9700 1 -- 172.21.15.71:0/2768624822 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7f3724069780 con 0x7f370c06c4e0 2024-04-23T02:12:22.636 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:12:22.636 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (5m) 59s ago 7m 26.2M - 0.25.0 c8568f914cd2 8261e17319fe 2024-04-23T02:12:22.636 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (7m) 59s ago 7m 7537k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:12:22.636 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (6m) 65s ago 6m 7461k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:12:22.636 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (7m) 59s ago 7m 6656k - 18.2.1 5be31c24972a 258512dcbc00 2024-04-23T02:12:22.636 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (6m) 65s ago 6m 6656k - 18.2.1 5be31c24972a feea2c9896a6 2024-04-23T02:12:22.636 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (5m) 59s ago 7m 88.2M - 9.4.7 954c08fa6188 48ea3e02b806 2024-04-23T02:12:22.637 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (88s) 59s ago 87s 13.2M - 18.2.1 5be31c24972a a515de2a841c 2024-04-23T02:12:22.637 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (81s) 59s ago 80s 15.5M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:12:22.637 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (78s) 65s ago 78s 14.8M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:12:22.637 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (85s) 65s ago 84s 17.8M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:12:22.637 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:9283,8765,8443 running (9m) 59s ago 8m 489M - 18.2.1 5be31c24972a fbe1ad3229f5 2024-04-23T02:12:22.637 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (6m) 65s ago 6m 440M - 18.2.1 5be31c24972a cf73640a7ee0 2024-04-23T02:12:22.637 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (9m) 59s ago 9m 50.1M 2048M 18.2.1 5be31c24972a ff40abe63fd9 2024-04-23T02:12:22.637 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (6m) 65s ago 6m 36.5M 2048M 18.2.1 5be31c24972a 9c03e90acc95 2024-04-23T02:12:22.637 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (7m) 59s ago 7m 18.5M - 1.5.0 0da6a335fe13 3f4d0aaee04e 2024-04-23T02:12:22.637 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (6m) 65s ago 6m 16.7M - 1.5.0 0da6a335fe13 cfd0b49d770c 2024-04-23T02:12:22.637 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (5m) 59s ago 5m 47.3M 3958M 18.2.1 5be31c24972a d3d5f2fa7267 2024-04-23T02:12:22.637 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (4m) 59s ago 4m 48.7M 3958M 18.2.1 5be31c24972a 3992157ff0fd 2024-04-23T02:12:22.637 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (4m) 59s ago 4m 46.6M 3958M 18.2.1 5be31c24972a 12380b6a15a2 2024-04-23T02:12:22.637 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (3m) 65s ago 3m 46.8M 4982M 18.2.1 5be31c24972a d5f587ee3a22 2024-04-23T02:12:22.638 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (3m) 65s ago 3m 42.9M 4982M 18.2.1 5be31c24972a 958d046005f0 2024-04-23T02:12:22.638 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (3m) 65s ago 2m 45.0M 4982M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:12:22.638 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (5m) 59s ago 6m 56.4M - 2.43.0 a07b618ecd1d 8df24da82462 2024-04-23T02:12:22.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.633+0000 7f3729b10700 1 -- 172.21.15.71:0/2768624822 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f370c06c4e0 msgr2=0x7f370c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:22.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.633+0000 7f3729b10700 1 --2- 172.21.15.71:0/2768624822 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f370c06c4e0 0x7f370c06e9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3718009fd0 tx=0x7f371800b5a0 comp rx=0 tx=0).stop 2024-04-23T02:12:22.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.633+0000 7f3729b10700 1 -- 172.21.15.71:0/2768624822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37240ffbc0 msgr2=0x7f3724195630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:22.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.633+0000 7f3729b10700 1 --2- 172.21.15.71:0/2768624822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37240ffbc0 0x7f3724195630 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f371400e8f0 tx=0x7f371400ec00 comp rx=0 tx=0).stop 2024-04-23T02:12:22.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.633+0000 7f3729b10700 1 -- 172.21.15.71:0/2768624822 shutdown_connections 2024-04-23T02:12:22.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.633+0000 7f3729b10700 1 --2- 172.21.15.71:0/2768624822 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f370c06c4e0 0x7f370c06e9a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:22.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.633+0000 7f3729b10700 1 --2- 172.21.15.71:0/2768624822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37240ffbc0 0x7f3724195630 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:22.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.633+0000 7f3729b10700 1 --2- 172.21.15.71:0/2768624822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3724100dc0 0x7f3724195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:22.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.633+0000 7f3729b10700 1 -- 172.21.15.71:0/2768624822 >> 172.21.15.71:0/2768624822 conn(0x7f37240fb160 msgr2=0x7f37241041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:22.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.634+0000 7f3729b10700 1 -- 172.21.15.71:0/2768624822 shutdown_connections 2024-04-23T02:12:22.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.634+0000 7f3729b10700 1 -- 172.21.15.71:0/2768624822 wait complete. 2024-04-23T02:12:22.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.758+0000 7fb991fa3700 1 -- 172.21.15.71:0/185758132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb98c0ffc10 msgr2=0x7fb98c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:22.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.758+0000 7fb991fa3700 1 --2- 172.21.15.71:0/185758132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb98c0ffc10 0x7fb98c100030 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009910 tx=0x7fb97c009c20 comp rx=0 tx=0).stop 2024-04-23T02:12:22.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.758+0000 7fb991fa3700 1 -- 172.21.15.71:0/185758132 shutdown_connections 2024-04-23T02:12:22.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.758+0000 7fb991fa3700 1 --2- 172.21.15.71:0/185758132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb98c100e10 0x7fb98c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:22.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.758+0000 7fb991fa3700 1 --2- 172.21.15.71:0/185758132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb98c0ffc10 0x7fb98c100030 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:22.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.758+0000 7fb991fa3700 1 -- 172.21.15.71:0/185758132 >> 172.21.15.71:0/185758132 conn(0x7fb98c0fb190 msgr2=0x7fb98c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:22.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.758+0000 7fb991fa3700 1 -- 172.21.15.71:0/185758132 shutdown_connections 2024-04-23T02:12:22.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.758+0000 7fb991fa3700 1 -- 172.21.15.71:0/185758132 wait complete. 2024-04-23T02:12:22.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.758+0000 7fb991fa3700 1 Processor -- start 2024-04-23T02:12:22.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.759+0000 7fb991fa3700 1 -- start start 2024-04-23T02:12:22.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.759+0000 7fb991fa3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb98c0ffc10 0x7fb98c1941f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:22.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.759+0000 7fb991fa3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb98c100e10 0x7fb98c194730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:22.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.759+0000 7fb991fa3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c194d50 con 0x7fb98c0ffc10 2024-04-23T02:12:22.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.759+0000 7fb991fa3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c194e90 con 0x7fb98c100e10 2024-04-23T02:12:22.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.759+0000 7fb98b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb98c0ffc10 0x7fb98c1941f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:22.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.759+0000 7fb98b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb98c0ffc10 0x7fb98c1941f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43432/0 (socket says 172.21.15.71:43432) 2024-04-23T02:12:22.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.759+0000 7fb98b7fe700 1 -- 172.21.15.71:0/3439695442 learned_addr learned my addr 172.21.15.71:0/3439695442 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:12:22.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.759+0000 7fb98affd700 1 --2- 172.21.15.71:0/3439695442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb98c100e10 0x7fb98c194730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:22.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.759+0000 7fb98b7fe700 1 -- 172.21.15.71:0/3439695442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb98c100e10 msgr2=0x7fb98c194730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:22.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.759+0000 7fb98b7fe700 1 --2- 172.21.15.71:0/3439695442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb98c100e10 0x7fb98c194730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:22.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.759+0000 7fb98b7fe700 1 -- 172.21.15.71:0/3439695442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c0095f0 con 0x7fb98c0ffc10 2024-04-23T02:12:22.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.759+0000 7fb98b7fe700 1 --2- 172.21.15.71:0/3439695442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb98c0ffc10 0x7fb98c1941f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00bde0 tx=0x7fb97c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:22.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.760+0000 7fb988ff9700 1 -- 172.21.15.71:0/3439695442 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb97c01d070 con 0x7fb98c0ffc10 2024-04-23T02:12:22.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.760+0000 7fb988ff9700 1 -- 172.21.15.71:0/3439695442 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fb97c005dc0 con 0x7fb98c0ffc10 2024-04-23T02:12:22.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.760+0000 7fb988ff9700 1 -- 172.21.15.71:0/3439695442 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb97c003ad0 con 0x7fb98c0ffc10 2024-04-23T02:12:22.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.760+0000 7fb991fa3700 1 -- 172.21.15.71:0/3439695442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98c1998e0 con 0x7fb98c0ffc10 2024-04-23T02:12:22.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.760+0000 7fb991fa3700 1 -- 172.21.15.71:0/3439695442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98c199d70 con 0x7fb98c0ffc10 2024-04-23T02:12:22.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.761+0000 7fb991fa3700 1 -- 172.21.15.71:0/3439695442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98c1074b0 con 0x7fb98c0ffc10 2024-04-23T02:12:22.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.762+0000 7fb988ff9700 1 -- 172.21.15.71:0/3439695442 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fb97c0048e0 con 0x7fb98c0ffc10 2024-04-23T02:12:22.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.762+0000 7fb988ff9700 1 --2- 172.21.15.71:0/3439695442 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fb974070900 0x7fb974072dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:22.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.762+0000 7fb988ff9700 1 -- 172.21.15.71:0/3439695442 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb97c08cf30 con 0x7fb98c0ffc10 2024-04-23T02:12:22.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.762+0000 7fb98affd700 1 --2- 172.21.15.71:0/3439695442 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fb974070900 0x7fb974072dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:22.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.763+0000 7fb98affd700 1 --2- 172.21.15.71:0/3439695442 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fb974070900 0x7fb974072dc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb980009c60 tx=0x7fb980009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:22.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:22.770+0000 7fb988ff9700 1 -- 172.21.15.71:0/3439695442 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb97c0c00e0 con 0x7fb98c0ffc10 2024-04-23T02:12:23.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.172+0000 7fb991fa3700 1 -- 172.21.15.71:0/3439695442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb98c19a460 con 0x7fb98c0ffc10 2024-04-23T02:12:23.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.173+0000 7fb988ff9700 1 -- 172.21.15.71:0/3439695442 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7fb97c026070 con 0x7fb98c0ffc10 2024-04-23T02:12:23.178 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:12:23.178 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:12:23.178 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-23T02:12:23.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:12:23.178 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:12:23.178 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-23T02:12:23.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:12:23.178 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:12:23.178 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-23T02:12:23.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:12:23.179 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:12:23.179 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-23T02:12:23.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:12:23.179 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:12:23.179 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-04-23T02:12:23.179 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:12:23.179 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:12:23.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.175+0000 7fb991fa3700 1 -- 172.21.15.71:0/3439695442 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fb974070900 msgr2=0x7fb974072dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:23.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.175+0000 7fb991fa3700 1 --2- 172.21.15.71:0/3439695442 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fb974070900 0x7fb974072dc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb980009c60 tx=0x7fb980009260 comp rx=0 tx=0).stop 2024-04-23T02:12:23.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.175+0000 7fb991fa3700 1 -- 172.21.15.71:0/3439695442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb98c0ffc10 msgr2=0x7fb98c1941f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:23.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.175+0000 7fb991fa3700 1 --2- 172.21.15.71:0/3439695442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb98c0ffc10 0x7fb98c1941f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00bde0 tx=0x7fb97c00be10 comp rx=0 tx=0).stop 2024-04-23T02:12:23.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.176+0000 7fb991fa3700 1 -- 172.21.15.71:0/3439695442 shutdown_connections 2024-04-23T02:12:23.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.176+0000 7fb991fa3700 1 --2- 172.21.15.71:0/3439695442 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fb974070900 0x7fb974072dc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:23.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.176+0000 7fb991fa3700 1 --2- 172.21.15.71:0/3439695442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb98c0ffc10 0x7fb98c1941f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:23.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.176+0000 7fb991fa3700 1 --2- 172.21.15.71:0/3439695442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb98c100e10 0x7fb98c194730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:23.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.176+0000 7fb991fa3700 1 -- 172.21.15.71:0/3439695442 >> 172.21.15.71:0/3439695442 conn(0x7fb98c0fb190 msgr2=0x7fb98c104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:23.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.176+0000 7fb991fa3700 1 -- 172.21.15.71:0/3439695442 shutdown_connections 2024-04-23T02:12:23.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.176+0000 7fb991fa3700 1 -- 172.21.15.71:0/3439695442 wait complete. 2024-04-23T02:12:23.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.294+0000 7fd248a47700 1 -- 172.21.15.71:0/2248538323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd244100db0 msgr2=0x7fd244101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:23.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.294+0000 7fd248a47700 1 --2- 172.21.15.71:0/2248538323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd244100db0 0x7fd244101210 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd238009910 tx=0x7fd238009c20 comp rx=0 tx=0).stop 2024-04-23T02:12:23.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.294+0000 7fd248a47700 1 -- 172.21.15.71:0/2248538323 shutdown_connections 2024-04-23T02:12:23.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.294+0000 7fd248a47700 1 --2- 172.21.15.71:0/2248538323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd244100db0 0x7fd244101210 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:23.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.294+0000 7fd248a47700 1 --2- 172.21.15.71:0/2248538323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2440ffbb0 0x7fd2440fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:23.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.294+0000 7fd248a47700 1 -- 172.21.15.71:0/2248538323 >> 172.21.15.71:0/2248538323 conn(0x7fd2440fb150 msgr2=0x7fd2440fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:23.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.294+0000 7fd248a47700 1 -- 172.21.15.71:0/2248538323 shutdown_connections 2024-04-23T02:12:23.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.294+0000 7fd248a47700 1 -- 172.21.15.71:0/2248538323 wait complete. 2024-04-23T02:12:23.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.295+0000 7fd248a47700 1 Processor -- start 2024-04-23T02:12:23.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.295+0000 7fd248a47700 1 -- start start 2024-04-23T02:12:23.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.295+0000 7fd248a47700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2440ffbb0 0x7fd2440699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:23.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.295+0000 7fd248a47700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd244100db0 0x7fd244198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:23.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.295+0000 7fd248a47700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2441994f0 con 0x7fd2440ffbb0 2024-04-23T02:12:23.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.295+0000 7fd248a47700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd244199630 con 0x7fd244100db0 2024-04-23T02:12:23.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.295+0000 7fd24259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2440ffbb0 0x7fd2440699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:23.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.295+0000 7fd24259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2440ffbb0 0x7fd2440699d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43450/0 (socket says 172.21.15.71:43450) 2024-04-23T02:12:23.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.295+0000 7fd24259c700 1 -- 172.21.15.71:0/1088954701 learned_addr learned my addr 172.21.15.71:0/1088954701 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:12:23.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.295+0000 7fd241d9b700 1 --2- 172.21.15.71:0/1088954701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd244100db0 0x7fd244198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:23.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.296+0000 7fd24259c700 1 -- 172.21.15.71:0/1088954701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd244100db0 msgr2=0x7fd244198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:23.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.296+0000 7fd24259c700 1 --2- 172.21.15.71:0/1088954701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd244100db0 0x7fd244198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:23.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.296+0000 7fd24259c700 1 -- 172.21.15.71:0/1088954701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2380095f0 con 0x7fd2440ffbb0 2024-04-23T02:12:23.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.296+0000 7fd24259c700 1 --2- 172.21.15.71:0/1088954701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2440ffbb0 0x7fd2440699d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd23400e8f0 tx=0x7fd23400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:23.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.296+0000 7fd2337fe700 1 -- 172.21.15.71:0/1088954701 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd234009790 con 0x7fd2440ffbb0 2024-04-23T02:12:23.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.296+0000 7fd248a47700 1 -- 172.21.15.71:0/1088954701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd244199910 con 0x7fd2440ffbb0 2024-04-23T02:12:23.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.296+0000 7fd2337fe700 1 -- 172.21.15.71:0/1088954701 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd23400c810 con 0x7fd2440ffbb0 2024-04-23T02:12:23.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.296+0000 7fd2337fe700 1 -- 172.21.15.71:0/1088954701 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd234003c20 con 0x7fd2440ffbb0 2024-04-23T02:12:23.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.296+0000 7fd248a47700 1 -- 172.21.15.71:0/1088954701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd244199e60 con 0x7fd2440ffbb0 2024-04-23T02:12:23.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.297+0000 7fd248a47700 1 -- 172.21.15.71:0/1088954701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2441085d0 con 0x7fd2440ffbb0 2024-04-23T02:12:23.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.299+0000 7fd2337fe700 1 -- 172.21.15.71:0/1088954701 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fd23400c980 con 0x7fd2440ffbb0 2024-04-23T02:12:23.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.299+0000 7fd2337fe700 1 --2- 172.21.15.71:0/1088954701 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fd22c06c530 0x7fd22c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:23.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.299+0000 7fd2337fe700 1 -- 172.21.15.71:0/1088954701 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd234014070 con 0x7fd2440ffbb0 2024-04-23T02:12:23.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.300+0000 7fd241d9b700 1 --2- 172.21.15.71:0/1088954701 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fd22c06c530 0x7fd22c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:23.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.300+0000 7fd241d9b700 1 --2- 172.21.15.71:0/1088954701 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fd22c06c530 0x7fd22c06e9f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd238009fd0 tx=0x7fd23800b5a0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:23.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.307+0000 7fd2337fe700 1 -- 172.21.15.71:0/1088954701 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd2340510e0 con 0x7fd2440ffbb0 2024-04-23T02:12:23.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.539+0000 7fd248a47700 1 -- 172.21.15.71:0/1088954701 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd244105700 con 0x7fd22c06c530 2024-04-23T02:12:23.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.540+0000 7fd2337fe700 1 -- 172.21.15.71:0/1088954701 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fd244105700 con 0x7fd22c06c530 2024-04-23T02:12:23.545 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:12:23.545 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:12:23.545 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:12:23.545 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-23T02:12:23.545 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [], 2024-04-23T02:12:23.545 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "0/2 daemons upgraded", 2024-04-23T02:12:23.545 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 image on host smithi192", 2024-04-23T02:12:23.545 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:12:23.545 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:12:23.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.542+0000 7fd248a47700 1 -- 172.21.15.71:0/1088954701 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fd22c06c530 msgr2=0x7fd22c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:23.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.542+0000 7fd248a47700 1 --2- 172.21.15.71:0/1088954701 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fd22c06c530 0x7fd22c06e9f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd238009fd0 tx=0x7fd23800b5a0 comp rx=0 tx=0).stop 2024-04-23T02:12:23.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.542+0000 7fd248a47700 1 -- 172.21.15.71:0/1088954701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2440ffbb0 msgr2=0x7fd2440699d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:23.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.542+0000 7fd248a47700 1 --2- 172.21.15.71:0/1088954701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2440ffbb0 0x7fd2440699d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd23400e8f0 tx=0x7fd23400ec00 comp rx=0 tx=0).stop 2024-04-23T02:12:23.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.542+0000 7fd248a47700 1 -- 172.21.15.71:0/1088954701 shutdown_connections 2024-04-23T02:12:23.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.542+0000 7fd248a47700 1 --2- 172.21.15.71:0/1088954701 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7fd22c06c530 0x7fd22c06e9f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:23.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.542+0000 7fd248a47700 1 --2- 172.21.15.71:0/1088954701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2440ffbb0 0x7fd2440699d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:23.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.542+0000 7fd248a47700 1 --2- 172.21.15.71:0/1088954701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd244100db0 0x7fd244198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:23.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.542+0000 7fd248a47700 1 -- 172.21.15.71:0/1088954701 >> 172.21.15.71:0/1088954701 conn(0x7fd2440fb150 msgr2=0x7fd244103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:23.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.542+0000 7fd248a47700 1 -- 172.21.15.71:0/1088954701 shutdown_connections 2024-04-23T02:12:23.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:23.542+0000 7fd248a47700 1 -- 172.21.15.71:0/1088954701 wait complete. 2024-04-23T02:12:23.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:23 smithi192 ceph-mon[49662]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:12:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:23 smithi192 ceph-mon[49662]: pgmap v234: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:23 smithi192 ceph-mon[49662]: from='client.24315 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:12:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:23 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/3439695442' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:12:23.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:23 smithi071 ceph-mon[39837]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:12:23.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:23 smithi071 ceph-mon[39837]: pgmap v234: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:23.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:23 smithi071 ceph-mon[39837]: from='client.24315 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:12:23.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:23 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3439695442' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:12:24.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:24 smithi192 ceph-mon[49662]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:12:24.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:24 smithi192 ceph-mon[49662]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:12:24.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:24 smithi071 ceph-mon[39837]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:12:24.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:24 smithi071 ceph-mon[39837]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:12:25.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:25 smithi192 ceph-mon[49662]: pgmap v235: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:25.641 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:25 smithi071 ceph-mon[39837]: pgmap v235: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:26.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:26 smithi071 ceph-mon[39837]: pgmap v236: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:12:27.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:26 smithi192 ceph-mon[49662]: pgmap v236: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:12:27.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:27 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:12:28.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:27 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:12:28.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:28 smithi071 ceph-mon[39837]: pgmap v237: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:29.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:28 smithi192 ceph-mon[49662]: pgmap v237: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:31.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:31 smithi071 ceph-mon[39837]: pgmap v238: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:31.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:31 smithi192 ceph-mon[49662]: pgmap v238: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:33.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:33 smithi071 ceph-mon[39837]: pgmap v239: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:33.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:33 smithi192 ceph-mon[49662]: pgmap v239: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:35.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:35 smithi071 ceph-mon[39837]: pgmap v240: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:35.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:35 smithi192 ceph-mon[49662]: pgmap v240: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:36.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:36 smithi071 ceph-mon[39837]: pgmap v241: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:12:37.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:36 smithi192 ceph-mon[49662]: pgmap v241: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:12:39.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:39 smithi071 ceph-mon[39837]: pgmap v242: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:39.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:39 smithi192 ceph-mon[49662]: pgmap v242: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:41.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:41 smithi071 ceph-mon[39837]: pgmap v243: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:41.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:41 smithi192 ceph-mon[49662]: pgmap v243: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:42 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:12:42.634 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:42 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:12:43.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:43 smithi192 ceph-mon[49662]: pgmap v244: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:43.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:43 smithi071 ceph-mon[39837]: pgmap v244: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:45.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:45 smithi192 ceph-mon[49662]: pgmap v245: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:45.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:45 smithi071 ceph-mon[39837]: pgmap v245: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:46.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:46 smithi071 ceph-mon[39837]: pgmap v246: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:12:47.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:46 smithi192 ceph-mon[49662]: pgmap v246: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:12:49.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:49 smithi071 ceph-mon[39837]: pgmap v247: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:49.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:49 smithi192 ceph-mon[49662]: pgmap v247: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:49.715 INFO:tasks.workunit.client.1.smithi192.stderr:Updating files: 96% (12035/12484) Updating files: 97% (12110/12484) Updating files: 98% (12235/12484) Updating files: 99% (12360/12484) Updating files: 100% (12484/12484) Updating files: 100% (12484/12484), done. 2024-04-23T02:12:50.715 INFO:tasks.workunit.client.1.smithi192.stderr:Note: switching to 'f26c9c3fee57ca330501910de5a07c8769ef5dfc'. 2024-04-23T02:12:50.715 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-04-23T02:12:50.715 INFO:tasks.workunit.client.1.smithi192.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-23T02:12:50.715 INFO:tasks.workunit.client.1.smithi192.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-23T02:12:50.715 INFO:tasks.workunit.client.1.smithi192.stderr:state without impacting any branches by switching back to a branch. 2024-04-23T02:12:50.715 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-04-23T02:12:50.715 INFO:tasks.workunit.client.1.smithi192.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-23T02:12:50.715 INFO:tasks.workunit.client.1.smithi192.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-23T02:12:50.715 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-04-23T02:12:50.715 INFO:tasks.workunit.client.1.smithi192.stderr: git switch -c 2024-04-23T02:12:50.715 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-04-23T02:12:50.715 INFO:tasks.workunit.client.1.smithi192.stderr:Or undo this operation with: 2024-04-23T02:12:50.715 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-04-23T02:12:50.716 INFO:tasks.workunit.client.1.smithi192.stderr: git switch - 2024-04-23T02:12:50.716 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-04-23T02:12:50.716 INFO:tasks.workunit.client.1.smithi192.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-23T02:12:50.716 INFO:tasks.workunit.client.1.smithi192.stderr: 2024-04-23T02:12:50.716 INFO:tasks.workunit.client.1.smithi192.stderr:HEAD is now at f26c9c3fee5 Merge pull request #57040 from zdover23/wip-doc-2024-04-23-backport-57022-to-reef 2024-04-23T02:12:50.725 DEBUG:teuthology.orchestra.run.smithi192:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-23T02:12:50.797 INFO:tasks.workunit.client.1.smithi192.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-23T02:12:50.799 INFO:tasks.workunit.client.1.smithi192.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-23T02:12:50.800 INFO:tasks.workunit.client.1.smithi192.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-23T02:12:51.336 INFO:tasks.workunit.client.1.smithi192.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-23T02:12:51.397 INFO:tasks.workunit.client.1.smithi192.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-23T02:12:51.436 INFO:tasks.workunit.client.1.smithi192.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-23T02:12:51.437 INFO:tasks.workunit.client.1.smithi192.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-23T02:12:51.437 INFO:tasks.workunit.client.1.smithi192.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-23T02:12:51.475 INFO:tasks.workunit.client.1.smithi192.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-23T02:12:51.482 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-04-23T02:12:51.482 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-23T02:12:51.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:51 smithi071 ceph-mon[39837]: pgmap v248: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:51.540 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-04-23T02:12:51.541 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-23T02:12:51.541 DEBUG:teuthology.orchestra.run.smithi192:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f26c9c3fee57ca330501910de5a07c8769ef5dfc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-04-23T02:12:51.609 INFO:tasks.workunit.client.1.smithi192.stderr:+ mkdir -p fsstress 2024-04-23T02:12:51.612 INFO:tasks.workunit.client.1.smithi192.stderr:+ pushd fsstress 2024-04-23T02:12:51.613 INFO:tasks.workunit.client.1.smithi192.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-23T02:12:51.613 INFO:tasks.workunit.client.1.smithi192.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-23T02:12:51.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:51 smithi192 ceph-mon[49662]: pgmap v248: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:52.177 INFO:tasks.workunit.client.1.smithi192.stderr:+ tar xzf ltp-full.tgz 2024-04-23T02:12:53.289 INFO:tasks.workunit.client.0.smithi071.stderr:Updating files: 88% (10988/12484) Updating files: 89% (11111/12484) Updating files: 90% (11236/12484) Updating files: 91% (11361/12484) Updating files: 92% (11486/12484) Updating files: 93% (11611/12484) Updating files: 94% (11735/12484) Updating files: 95% (11860/12484) Updating files: 96% (11985/12484) Updating files: 97% (12110/12484) Updating files: 98% (12235/12484) Updating files: 99% (12360/12484) Updating files: 100% (12484/12484) Updating files: 100% (12484/12484), done. 2024-04-23T02:12:53.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:53 smithi071 ceph-mon[39837]: pgmap v249: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:53.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:53 smithi192 ceph-mon[49662]: pgmap v249: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:12:53.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.669+0000 7f29fb7df700 1 -- 172.21.15.71:0/2269891979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f4100ca0 msgr2=0x7f29f41010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:53.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.669+0000 7f29fb7df700 1 --2- 172.21.15.71:0/2269891979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f4100ca0 0x7f29f41010c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f29f0009910 tx=0x7f29f0009c20 comp rx=0 tx=0).stop 2024-04-23T02:12:53.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.669+0000 7f29fb7df700 1 -- 172.21.15.71:0/2269891979 shutdown_connections 2024-04-23T02:12:53.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.669+0000 7f29fb7df700 1 --2- 172.21.15.71:0/2269891979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29f4101ea0 0x7f29f4102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:53.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.669+0000 7f29fb7df700 1 --2- 172.21.15.71:0/2269891979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f4100ca0 0x7f29f41010c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:53.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.669+0000 7f29fb7df700 1 -- 172.21.15.71:0/2269891979 >> 172.21.15.71:0/2269891979 conn(0x7f29f40fc260 msgr2=0x7f29f40fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:53.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.670+0000 7f29fb7df700 1 -- 172.21.15.71:0/2269891979 shutdown_connections 2024-04-23T02:12:53.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.670+0000 7f29fb7df700 1 -- 172.21.15.71:0/2269891979 wait complete. 2024-04-23T02:12:53.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.671+0000 7f29fb7df700 1 Processor -- start 2024-04-23T02:12:53.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.671+0000 7f29fb7df700 1 -- start start 2024-04-23T02:12:53.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.671+0000 7f29fb7df700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29f4100ca0 0x7f29f4196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:53.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.671+0000 7f29fb7df700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f4101ea0 0x7f29f4196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:53.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.671+0000 7f29fb7df700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f4196f60 con 0x7f29f4101ea0 2024-04-23T02:12:53.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.671+0000 7f29fb7df700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f41970a0 con 0x7f29f4100ca0 2024-04-23T02:12:53.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.672+0000 7f29f8d7a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f4101ea0 0x7f29f4196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:53.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.672+0000 7f29f8d7a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f4101ea0 0x7f29f4196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50968/0 (socket says 172.21.15.71:50968) 2024-04-23T02:12:53.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.672+0000 7f29f8d7a700 1 -- 172.21.15.71:0/3974947868 learned_addr learned my addr 172.21.15.71:0/3974947868 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:12:53.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.672+0000 7f29f957b700 1 --2- 172.21.15.71:0/3974947868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29f4100ca0 0x7f29f4196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:53.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.672+0000 7f29f8d7a700 1 -- 172.21.15.71:0/3974947868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29f4100ca0 msgr2=0x7f29f4196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:53.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.672+0000 7f29f8d7a700 1 --2- 172.21.15.71:0/3974947868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29f4100ca0 0x7f29f4196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:53.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.672+0000 7f29f8d7a700 1 -- 172.21.15.71:0/3974947868 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f00095f0 con 0x7f29f4101ea0 2024-04-23T02:12:53.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.672+0000 7f29f8d7a700 1 --2- 172.21.15.71:0/3974947868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f4101ea0 0x7f29f4196940 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f29e400e920 tx=0x7f29e400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:53.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.672+0000 7f29ea7fc700 1 -- 172.21.15.71:0/3974947868 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29e400c9d0 con 0x7f29f4101ea0 2024-04-23T02:12:53.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.672+0000 7f29fb7df700 1 -- 172.21.15.71:0/3974947868 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f419bb50 con 0x7f29f4101ea0 2024-04-23T02:12:53.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.672+0000 7f29ea7fc700 1 -- 172.21.15.71:0/3974947868 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f29e4010430 con 0x7f29f4101ea0 2024-04-23T02:12:53.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.672+0000 7f29ea7fc700 1 -- 172.21.15.71:0/3974947868 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29e4018630 con 0x7f29f4101ea0 2024-04-23T02:12:53.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.672+0000 7f29fb7df700 1 -- 172.21.15.71:0/3974947868 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f419c0a0 con 0x7f29f4101ea0 2024-04-23T02:12:53.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.674+0000 7f29fb7df700 1 -- 172.21.15.71:0/3974947868 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29f41096c0 con 0x7f29f4101ea0 2024-04-23T02:12:53.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.674+0000 7f29ea7fc700 1 -- 172.21.15.71:0/3974947868 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f29e40105a0 con 0x7f29f4101ea0 2024-04-23T02:12:53.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.675+0000 7f29ea7fc700 1 --2- 172.21.15.71:0/3974947868 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f29e006c4e0 0x7f29e006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:53.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.675+0000 7f29f957b700 1 --2- 172.21.15.71:0/3974947868 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f29e006c4e0 0x7f29e006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:53.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.675+0000 7f29ea7fc700 1 -- 172.21.15.71:0/3974947868 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f29e4014070 con 0x7f29f4101ea0 2024-04-23T02:12:53.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.675+0000 7f29f957b700 1 --2- 172.21.15.71:0/3974947868 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f29e006c4e0 0x7f29e006e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f29f0009fd0 tx=0x7f29f000b5a0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:53.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.680+0000 7f29ea7fc700 1 -- 172.21.15.71:0/3974947868 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f29e4059be0 con 0x7f29f4101ea0 2024-04-23T02:12:53.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.932+0000 7f29fb7df700 1 -- 172.21.15.71:0/3974947868 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f29f41067f0 con 0x7f29e006c4e0 2024-04-23T02:12:53.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.933+0000 7f29ea7fc700 1 -- 172.21.15.71:0/3974947868 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f29f41067f0 con 0x7f29e006c4e0 2024-04-23T02:12:53.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.935+0000 7f29fb7df700 1 -- 172.21.15.71:0/3974947868 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f29e006c4e0 msgr2=0x7f29e006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:53.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.935+0000 7f29fb7df700 1 --2- 172.21.15.71:0/3974947868 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f29e006c4e0 0x7f29e006e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f29f0009fd0 tx=0x7f29f000b5a0 comp rx=0 tx=0).stop 2024-04-23T02:12:53.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.935+0000 7f29fb7df700 1 -- 172.21.15.71:0/3974947868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f4101ea0 msgr2=0x7f29f4196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:53.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.935+0000 7f29fb7df700 1 --2- 172.21.15.71:0/3974947868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f4101ea0 0x7f29f4196940 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f29e400e920 tx=0x7f29e400ec30 comp rx=0 tx=0).stop 2024-04-23T02:12:53.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.935+0000 7f29fb7df700 1 -- 172.21.15.71:0/3974947868 shutdown_connections 2024-04-23T02:12:53.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.935+0000 7f29fb7df700 1 --2- 172.21.15.71:0/3974947868 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f29e006c4e0 0x7f29e006e9a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:53.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.935+0000 7f29fb7df700 1 --2- 172.21.15.71:0/3974947868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29f4100ca0 0x7f29f4196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:53.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.935+0000 7f29fb7df700 1 --2- 172.21.15.71:0/3974947868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f4101ea0 0x7f29f4196940 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:53.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.935+0000 7f29fb7df700 1 -- 172.21.15.71:0/3974947868 >> 172.21.15.71:0/3974947868 conn(0x7f29f40fc260 msgr2=0x7f29f41050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:53.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.936+0000 7f29fb7df700 1 -- 172.21.15.71:0/3974947868 shutdown_connections 2024-04-23T02:12:53.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:53.936+0000 7f29fb7df700 1 -- 172.21.15.71:0/3974947868 wait complete. 2024-04-23T02:12:53.952 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:12:53.963 INFO:tasks.workunit.client.0.smithi071.stderr:Note: switching to 'f26c9c3fee57ca330501910de5a07c8769ef5dfc'. 2024-04-23T02:12:53.964 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-04-23T02:12:53.964 INFO:tasks.workunit.client.0.smithi071.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-23T02:12:53.964 INFO:tasks.workunit.client.0.smithi071.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-23T02:12:53.964 INFO:tasks.workunit.client.0.smithi071.stderr:state without impacting any branches by switching back to a branch. 2024-04-23T02:12:53.964 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-04-23T02:12:53.964 INFO:tasks.workunit.client.0.smithi071.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-23T02:12:53.964 INFO:tasks.workunit.client.0.smithi071.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-23T02:12:53.964 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-04-23T02:12:53.964 INFO:tasks.workunit.client.0.smithi071.stderr: git switch -c 2024-04-23T02:12:53.964 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-04-23T02:12:53.964 INFO:tasks.workunit.client.0.smithi071.stderr:Or undo this operation with: 2024-04-23T02:12:53.964 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-04-23T02:12:53.964 INFO:tasks.workunit.client.0.smithi071.stderr: git switch - 2024-04-23T02:12:53.964 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-04-23T02:12:53.965 INFO:tasks.workunit.client.0.smithi071.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-23T02:12:53.965 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-04-23T02:12:53.965 INFO:tasks.workunit.client.0.smithi071.stderr:HEAD is now at f26c9c3fee5 Merge pull request #57040 from zdover23/wip-doc-2024-04-23-backport-57022-to-reef 2024-04-23T02:12:53.970 DEBUG:teuthology.orchestra.run.smithi071:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-23T02:12:54.009 INFO:tasks.workunit.client.0.smithi071.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-23T02:12:54.011 INFO:tasks.workunit.client.0.smithi071.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-23T02:12:54.011 INFO:tasks.workunit.client.0.smithi071.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-23T02:12:54.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.049+0000 7f24de3a2700 1 -- 172.21.15.71:0/717864684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d80ffdf0 msgr2=0x7f24d8100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:54.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.049+0000 7f24de3a2700 1 --2- 172.21.15.71:0/717864684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d80ffdf0 0x7f24d8100270 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009910 tx=0x7f24cc009c20 comp rx=0 tx=0).stop 2024-04-23T02:12:54.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.049+0000 7f24de3a2700 1 -- 172.21.15.71:0/717864684 shutdown_connections 2024-04-23T02:12:54.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.049+0000 7f24de3a2700 1 --2- 172.21.15.71:0/717864684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d80ffdf0 0x7f24d8100270 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:54.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.049+0000 7f24de3a2700 1 --2- 172.21.15.71:0/717864684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24d8101220 0x7f24d8101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:54.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.049+0000 7f24de3a2700 1 -- 172.21.15.71:0/717864684 >> 172.21.15.71:0/717864684 conn(0x7f24d80fa8c0 msgr2=0x7f24d80fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:54.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.049+0000 7f24de3a2700 1 -- 172.21.15.71:0/717864684 shutdown_connections 2024-04-23T02:12:54.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.049+0000 7f24de3a2700 1 -- 172.21.15.71:0/717864684 wait complete. 2024-04-23T02:12:54.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.050+0000 7f24de3a2700 1 Processor -- start 2024-04-23T02:12:54.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.050+0000 7f24de3a2700 1 -- start start 2024-04-23T02:12:54.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.050+0000 7f24de3a2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24d80ffdf0 0x7f24d8103b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:54.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.050+0000 7f24de3a2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d8101220 0x7f24d81021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:54.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.050+0000 7f24de3a2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d8102720 con 0x7f24d8101220 2024-04-23T02:12:54.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.050+0000 7f24de3a2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d8102860 con 0x7f24d80ffdf0 2024-04-23T02:12:54.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.050+0000 7f24d77fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d8101220 0x7f24d81021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:54.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.050+0000 7f24d77fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d8101220 0x7f24d81021e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50996/0 (socket says 172.21.15.71:50996) 2024-04-23T02:12:54.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.050+0000 7f24d77fe700 1 -- 172.21.15.71:0/194670248 learned_addr learned my addr 172.21.15.71:0/194670248 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:12:54.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.050+0000 7f24d7fff700 1 --2- 172.21.15.71:0/194670248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24d80ffdf0 0x7f24d8103b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:54.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.051+0000 7f24d77fe700 1 -- 172.21.15.71:0/194670248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24d80ffdf0 msgr2=0x7f24d8103b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:54.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.051+0000 7f24d77fe700 1 --2- 172.21.15.71:0/194670248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24d80ffdf0 0x7f24d8103b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:54.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.051+0000 7f24d77fe700 1 -- 172.21.15.71:0/194670248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc0095f0 con 0x7f24d8101220 2024-04-23T02:12:54.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.051+0000 7f24d77fe700 1 --2- 172.21.15.71:0/194670248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d8101220 0x7f24d81021e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00bd70 tx=0x7f24cc00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:54.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.051+0000 7f24d57fa700 1 -- 172.21.15.71:0/194670248 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24cc01d070 con 0x7f24d8101220 2024-04-23T02:12:54.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.051+0000 7f24de3a2700 1 -- 172.21.15.71:0/194670248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d8102ae0 con 0x7f24d8101220 2024-04-23T02:12:54.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.051+0000 7f24d57fa700 1 -- 172.21.15.71:0/194670248 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f24cc00f460 con 0x7f24d8101220 2024-04-23T02:12:54.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.051+0000 7f24d57fa700 1 -- 172.21.15.71:0/194670248 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24cc021640 con 0x7f24d8101220 2024-04-23T02:12:54.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.051+0000 7f24de3a2700 1 -- 172.21.15.71:0/194670248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d8102fd0 con 0x7f24d8101220 2024-04-23T02:12:54.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.052+0000 7f24de3a2700 1 -- 172.21.15.71:0/194670248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d8108680 con 0x7f24d8101220 2024-04-23T02:12:54.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.052+0000 7f24d57fa700 1 -- 172.21.15.71:0/194670248 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f24cc00fa20 con 0x7f24d8101220 2024-04-23T02:12:54.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.053+0000 7f24d57fa700 1 --2- 172.21.15.71:0/194670248 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f24c006c4e0 0x7f24c006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:54.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.053+0000 7f24d57fa700 1 -- 172.21.15.71:0/194670248 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f24cc08cad0 con 0x7f24d8101220 2024-04-23T02:12:54.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.053+0000 7f24d7fff700 1 --2- 172.21.15.71:0/194670248 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f24c006c4e0 0x7f24c006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:54.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.053+0000 7f24d7fff700 1 --2- 172.21.15.71:0/194670248 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f24c006c4e0 0x7f24c006e9a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f24c8007be0 tx=0x7f24c80073d0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:54.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.058+0000 7f24d57fa700 1 -- 172.21.15.71:0/194670248 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f24cc05ace0 con 0x7f24d8101220 2024-04-23T02:12:54.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.310+0000 7f24de3a2700 1 -- 172.21.15.71:0/194670248 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f24d819e8e0 con 0x7f24c006c4e0 2024-04-23T02:12:54.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.311+0000 7f24d57fa700 1 -- 172.21.15.71:0/194670248 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f24d819e8e0 con 0x7f24c006c4e0 2024-04-23T02:12:54.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.313+0000 7f24de3a2700 1 -- 172.21.15.71:0/194670248 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f24c006c4e0 msgr2=0x7f24c006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:54.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.313+0000 7f24de3a2700 1 --2- 172.21.15.71:0/194670248 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f24c006c4e0 0x7f24c006e9a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f24c8007be0 tx=0x7f24c80073d0 comp rx=0 tx=0).stop 2024-04-23T02:12:54.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.313+0000 7f24de3a2700 1 -- 172.21.15.71:0/194670248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d8101220 msgr2=0x7f24d81021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:54.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.313+0000 7f24de3a2700 1 --2- 172.21.15.71:0/194670248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d8101220 0x7f24d81021e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00bd70 tx=0x7f24cc00bda0 comp rx=0 tx=0).stop 2024-04-23T02:12:54.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.313+0000 7f24de3a2700 1 -- 172.21.15.71:0/194670248 shutdown_connections 2024-04-23T02:12:54.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.313+0000 7f24de3a2700 1 --2- 172.21.15.71:0/194670248 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f24c006c4e0 0x7f24c006e9a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:54.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.313+0000 7f24de3a2700 1 --2- 172.21.15.71:0/194670248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24d80ffdf0 0x7f24d8103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:54.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.313+0000 7f24de3a2700 1 --2- 172.21.15.71:0/194670248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d8101220 0x7f24d81021e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:54.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.313+0000 7f24de3a2700 1 -- 172.21.15.71:0/194670248 >> 172.21.15.71:0/194670248 conn(0x7f24d80fa8c0 msgr2=0x7f24d80fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:54.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.313+0000 7f24de3a2700 1 -- 172.21.15.71:0/194670248 shutdown_connections 2024-04-23T02:12:54.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.313+0000 7f24de3a2700 1 -- 172.21.15.71:0/194670248 wait complete. 2024-04-23T02:12:54.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.433+0000 7ff56598e700 1 -- 172.21.15.71:0/565079090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5600ff320 msgr2=0x7ff5600ff740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:54.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.433+0000 7ff56598e700 1 --2- 172.21.15.71:0/565079090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5600ff320 0x7ff5600ff740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff550009910 tx=0x7ff550009c20 comp rx=0 tx=0).stop 2024-04-23T02:12:54.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.433+0000 7ff56598e700 1 -- 172.21.15.71:0/565079090 shutdown_connections 2024-04-23T02:12:54.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.433+0000 7ff56598e700 1 --2- 172.21.15.71:0/565079090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff560100520 0x7ff560100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:54.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.433+0000 7ff56598e700 1 --2- 172.21.15.71:0/565079090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5600ff320 0x7ff5600ff740 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:54.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.433+0000 7ff56598e700 1 -- 172.21.15.71:0/565079090 >> 172.21.15.71:0/565079090 conn(0x7ff5600fa8c0 msgr2=0x7ff5600fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:54.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.433+0000 7ff56598e700 1 -- 172.21.15.71:0/565079090 shutdown_connections 2024-04-23T02:12:54.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.433+0000 7ff56598e700 1 -- 172.21.15.71:0/565079090 wait complete. 2024-04-23T02:12:54.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.434+0000 7ff56598e700 1 Processor -- start 2024-04-23T02:12:54.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.434+0000 7ff56598e700 1 -- start start 2024-04-23T02:12:54.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.434+0000 7ff56598e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5600ff320 0x7ff5601963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:54.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.434+0000 7ff56598e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff560100520 0x7ff560196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:54.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.434+0000 7ff56598e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff560196f50 con 0x7ff5600ff320 2024-04-23T02:12:54.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.434+0000 7ff56598e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff560197090 con 0x7ff560100520 2024-04-23T02:12:54.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.434+0000 7ff55effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5600ff320 0x7ff5601963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:54.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.434+0000 7ff55effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5600ff320 0x7ff5601963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51014/0 (socket says 172.21.15.71:51014) 2024-04-23T02:12:54.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.434+0000 7ff55effd700 1 -- 172.21.15.71:0/2907511479 learned_addr learned my addr 172.21.15.71:0/2907511479 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:12:54.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.434+0000 7ff55e7fc700 1 --2- 172.21.15.71:0/2907511479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff560100520 0x7ff560196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:54.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.434+0000 7ff55effd700 1 -- 172.21.15.71:0/2907511479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff560100520 msgr2=0x7ff560196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:54.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.434+0000 7ff55effd700 1 --2- 172.21.15.71:0/2907511479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff560100520 0x7ff560196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:54.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.434+0000 7ff55effd700 1 -- 172.21.15.71:0/2907511479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5500095f0 con 0x7ff5600ff320 2024-04-23T02:12:54.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.434+0000 7ff55effd700 1 --2- 172.21.15.71:0/2907511479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5600ff320 0x7ff5601963f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff55000bda0 tx=0x7ff55000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:54.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.435+0000 7ff56498c700 1 -- 172.21.15.71:0/2907511479 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff55001d070 con 0x7ff5600ff320 2024-04-23T02:12:54.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.435+0000 7ff56498c700 1 -- 172.21.15.71:0/2907511479 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ff55000f460 con 0x7ff5600ff320 2024-04-23T02:12:54.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.435+0000 7ff56598e700 1 -- 172.21.15.71:0/2907511479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56019bae0 con 0x7ff5600ff320 2024-04-23T02:12:54.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.435+0000 7ff56498c700 1 -- 172.21.15.71:0/2907511479 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff550022650 con 0x7ff5600ff320 2024-04-23T02:12:54.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.435+0000 7ff56598e700 1 -- 172.21.15.71:0/2907511479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56019bfd0 con 0x7ff5600ff320 2024-04-23T02:12:54.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.436+0000 7ff56598e700 1 -- 172.21.15.71:0/2907511479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5601085c0 con 0x7ff5600ff320 2024-04-23T02:12:54.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.436+0000 7ff56498c700 1 -- 172.21.15.71:0/2907511479 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7ff55000fa20 con 0x7ff5600ff320 2024-04-23T02:12:54.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.437+0000 7ff56498c700 1 --2- 172.21.15.71:0/2907511479 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7ff54806c4e0 0x7ff54806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:54.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.437+0000 7ff56498c700 1 -- 172.21.15.71:0/2907511479 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff55008cbe0 con 0x7ff5600ff320 2024-04-23T02:12:54.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.437+0000 7ff55e7fc700 1 --2- 172.21.15.71:0/2907511479 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7ff54806c4e0 0x7ff54806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:54.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.437+0000 7ff55e7fc700 1 --2- 172.21.15.71:0/2907511479 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7ff54806c4e0 0x7ff54806e9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff554005950 tx=0x7ff55400b410 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:54.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.442+0000 7ff56498c700 1 -- 172.21.15.71:0/2907511479 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff55005adf0 con 0x7ff5600ff320 2024-04-23T02:12:54.595 INFO:tasks.workunit.client.0.smithi071.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-23T02:12:54.660 INFO:tasks.workunit.client.0.smithi071.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-23T02:12:54.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.670+0000 7ff56598e700 1 -- 172.21.15.71:0/2907511479 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff560106070 con 0x7ff54806c4e0 2024-04-23T02:12:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.680+0000 7ff56498c700 1 -- 172.21.15.71:0/2907511479 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3528 (secure 0 0 0) 0x7ff560106070 con 0x7ff54806c4e0 2024-04-23T02:12:54.685 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:12:54.685 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (6m) 92s ago 7m 26.2M - 0.25.0 c8568f914cd2 8261e17319fe 2024-04-23T02:12:54.685 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (8m) 92s ago 8m 7537k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:12:54.685 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (7m) 97s ago 7m 7461k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:12:54.685 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (8m) 92s ago 8m 6656k - 18.2.1 5be31c24972a 258512dcbc00 2024-04-23T02:12:54.685 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (6m) 97s ago 6m 6656k - 18.2.1 5be31c24972a feea2c9896a6 2024-04-23T02:12:54.686 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (6m) 92s ago 7m 88.2M - 9.4.7 954c08fa6188 48ea3e02b806 2024-04-23T02:12:54.686 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (2m) 92s ago 119s 13.2M - 18.2.1 5be31c24972a a515de2a841c 2024-04-23T02:12:54.686 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (113s) 92s ago 112s 15.5M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:12:54.686 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (110s) 97s ago 110s 14.8M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:12:54.686 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (117s) 97s ago 116s 17.8M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:12:54.687 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:9283,8765,8443 running (9m) 92s ago 9m 489M - 18.2.1 5be31c24972a fbe1ad3229f5 2024-04-23T02:12:54.687 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (6m) 97s ago 6m 440M - 18.2.1 5be31c24972a cf73640a7ee0 2024-04-23T02:12:54.687 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (9m) 92s ago 9m 50.1M 2048M 18.2.1 5be31c24972a ff40abe63fd9 2024-04-23T02:12:54.687 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (6m) 97s ago 6m 36.5M 2048M 18.2.1 5be31c24972a 9c03e90acc95 2024-04-23T02:12:54.687 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (8m) 92s ago 8m 18.5M - 1.5.0 0da6a335fe13 3f4d0aaee04e 2024-04-23T02:12:54.687 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (6m) 97s ago 6m 16.7M - 1.5.0 0da6a335fe13 cfd0b49d770c 2024-04-23T02:12:54.687 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (5m) 92s ago 5m 47.3M 3958M 18.2.1 5be31c24972a d3d5f2fa7267 2024-04-23T02:12:54.687 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (5m) 92s ago 5m 48.7M 3958M 18.2.1 5be31c24972a 3992157ff0fd 2024-04-23T02:12:54.688 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (4m) 92s ago 4m 46.6M 3958M 18.2.1 5be31c24972a 12380b6a15a2 2024-04-23T02:12:54.688 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (4m) 97s ago 4m 46.8M 4982M 18.2.1 5be31c24972a d5f587ee3a22 2024-04-23T02:12:54.688 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (4m) 97s ago 3m 42.9M 4982M 18.2.1 5be31c24972a 958d046005f0 2024-04-23T02:12:54.688 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (3m) 97s ago 3m 45.0M 4982M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:12:54.688 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (6m) 92s ago 7m 56.4M - 2.43.0 a07b618ecd1d 8df24da82462 2024-04-23T02:12:54.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.683+0000 7ff56598e700 1 -- 172.21.15.71:0/2907511479 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7ff54806c4e0 msgr2=0x7ff54806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:54.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.683+0000 7ff56598e700 1 --2- 172.21.15.71:0/2907511479 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7ff54806c4e0 0x7ff54806e9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff554005950 tx=0x7ff55400b410 comp rx=0 tx=0).stop 2024-04-23T02:12:54.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.683+0000 7ff56598e700 1 -- 172.21.15.71:0/2907511479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5600ff320 msgr2=0x7ff5601963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:54.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.683+0000 7ff56598e700 1 --2- 172.21.15.71:0/2907511479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5600ff320 0x7ff5601963f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff55000bda0 tx=0x7ff55000bdd0 comp rx=0 tx=0).stop 2024-04-23T02:12:54.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.683+0000 7ff56598e700 1 -- 172.21.15.71:0/2907511479 shutdown_connections 2024-04-23T02:12:54.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.683+0000 7ff56598e700 1 --2- 172.21.15.71:0/2907511479 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7ff54806c4e0 0x7ff54806e9a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:54.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.683+0000 7ff56598e700 1 --2- 172.21.15.71:0/2907511479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5600ff320 0x7ff5601963f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:54.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.683+0000 7ff56598e700 1 --2- 172.21.15.71:0/2907511479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff560100520 0x7ff560196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:54.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.683+0000 7ff56598e700 1 -- 172.21.15.71:0/2907511479 >> 172.21.15.71:0/2907511479 conn(0x7ff5600fa8c0 msgr2=0x7ff560105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:54.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.684+0000 7ff56598e700 1 -- 172.21.15.71:0/2907511479 shutdown_connections 2024-04-23T02:12:54.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.684+0000 7ff56598e700 1 -- 172.21.15.71:0/2907511479 wait complete. 2024-04-23T02:12:54.700 INFO:tasks.workunit.client.0.smithi071.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-23T02:12:54.702 INFO:tasks.workunit.client.0.smithi071.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-23T02:12:54.702 INFO:tasks.workunit.client.0.smithi071.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-23T02:12:54.741 INFO:tasks.workunit.client.0.smithi071.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-23T02:12:54.744 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T02:12:54.744 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-23T02:12:54.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.794+0000 7f43a0137700 1 -- 172.21.15.71:0/1284578440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43980fef60 msgr2=0x7f43980ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:54.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.794+0000 7f43a0137700 1 --2- 172.21.15.71:0/1284578440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43980fef60 0x7f43980ff380 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4394009910 tx=0x7f4394009c20 comp rx=0 tx=0).stop 2024-04-23T02:12:54.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.794+0000 7f43a0137700 1 -- 172.21.15.71:0/1284578440 shutdown_connections 2024-04-23T02:12:54.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.794+0000 7f43a0137700 1 --2- 172.21.15.71:0/1284578440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4398101f40 0x7f43981023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:54.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.794+0000 7f43a0137700 1 --2- 172.21.15.71:0/1284578440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43980fef60 0x7f43980ff380 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:54.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.795+0000 7f43a0137700 1 -- 172.21.15.71:0/1284578440 >> 172.21.15.71:0/1284578440 conn(0x7f43980fa8c0 msgr2=0x7f43980fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:54.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.795+0000 7f43a0137700 1 -- 172.21.15.71:0/1284578440 shutdown_connections 2024-04-23T02:12:54.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.795+0000 7f43a0137700 1 -- 172.21.15.71:0/1284578440 wait complete. 2024-04-23T02:12:54.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.795+0000 7f43a0137700 1 Processor -- start 2024-04-23T02:12:54.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.795+0000 7f43a0137700 1 -- start start 2024-04-23T02:12:54.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.796+0000 7f43a0137700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43980fef60 0x7f4398196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:54.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.796+0000 7f43a0137700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4398101f40 0x7f4398196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:54.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.796+0000 7f43a0137700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4398196f60 con 0x7f43980fef60 2024-04-23T02:12:54.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.796+0000 7f43a0137700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43981970a0 con 0x7f4398101f40 2024-04-23T02:12:54.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.796+0000 7f439ded3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43980fef60 0x7f4398196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:54.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.796+0000 7f439ded3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43980fef60 0x7f4398196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51030/0 (socket says 172.21.15.71:51030) 2024-04-23T02:12:54.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.796+0000 7f439ded3700 1 -- 172.21.15.71:0/468417668 learned_addr learned my addr 172.21.15.71:0/468417668 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:12:54.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.796+0000 7f439d6d2700 1 --2- 172.21.15.71:0/468417668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4398101f40 0x7f4398196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:54.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.796+0000 7f439ded3700 1 -- 172.21.15.71:0/468417668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4398101f40 msgr2=0x7f4398196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:54.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.796+0000 7f439ded3700 1 --2- 172.21.15.71:0/468417668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4398101f40 0x7f4398196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:54.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.796+0000 7f439ded3700 1 -- 172.21.15.71:0/468417668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43940095f0 con 0x7f43980fef60 2024-04-23T02:12:54.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.796+0000 7f439d6d2700 1 --2- 172.21.15.71:0/468417668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4398101f40 0x7f4398196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T02:12:54.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.796+0000 7f439ded3700 1 --2- 172.21.15.71:0/468417668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43980fef60 0x7f4398196400 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4394000c00 tx=0x7f439400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:54.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.797+0000 7f438effd700 1 -- 172.21.15.71:0/468417668 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f439401d070 con 0x7f43980fef60 2024-04-23T02:12:54.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.797+0000 7f438effd700 1 -- 172.21.15.71:0/468417668 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4394005710 con 0x7f43980fef60 2024-04-23T02:12:54.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.797+0000 7f43a0137700 1 -- 172.21.15.71:0/468417668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f439819baf0 con 0x7f43980fef60 2024-04-23T02:12:54.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.797+0000 7f438effd700 1 -- 172.21.15.71:0/468417668 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43940039c0 con 0x7f43980fef60 2024-04-23T02:12:54.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.797+0000 7f43a0137700 1 -- 172.21.15.71:0/468417668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f439819bf80 con 0x7f43980fef60 2024-04-23T02:12:54.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.798+0000 7f43a0137700 1 -- 172.21.15.71:0/468417668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43981096c0 con 0x7f43980fef60 2024-04-23T02:12:54.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.798+0000 7f438effd700 1 -- 172.21.15.71:0/468417668 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f43940041f0 con 0x7f43980fef60 2024-04-23T02:12:54.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.799+0000 7f438effd700 1 --2- 172.21.15.71:0/468417668 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f438406c4e0 0x7f438406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:54.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.799+0000 7f438effd700 1 -- 172.21.15.71:0/468417668 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f439408cf70 con 0x7f43980fef60 2024-04-23T02:12:54.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.799+0000 7f439d6d2700 1 --2- 172.21.15.71:0/468417668 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f438406c4e0 0x7f438406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:54.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.799+0000 7f439d6d2700 1 --2- 172.21.15.71:0/468417668 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f438406c4e0 0x7f438406e9a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4388011490 tx=0x7f4388009260 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:54.806 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-23T02:12:54.807 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-23T02:12:54.808 DEBUG:teuthology.orchestra.run.smithi071:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f26c9c3fee57ca330501910de5a07c8769ef5dfc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-23T02:12:54.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:54.803+0000 7f438effd700 1 -- 172.21.15.71:0/468417668 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f43940c00e0 con 0x7f43980fef60 2024-04-23T02:12:54.837 INFO:tasks.workunit.client.0.smithi071.stderr:+ mkdir -p fsstress 2024-04-23T02:12:54.841 INFO:tasks.workunit.client.0.smithi071.stderr:+ pushd fsstress 2024-04-23T02:12:54.842 INFO:tasks.workunit.client.0.smithi071.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-23T02:12:54.842 INFO:tasks.workunit.client.0.smithi071.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-23T02:12:55.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.117+0000 7f43a0137700 1 -- 172.21.15.71:0/468417668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f439819c650 con 0x7f43980fef60 2024-04-23T02:12:55.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.117+0000 7f438effd700 1 -- 172.21.15.71:0/468417668 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f439405b180 con 0x7f43980fef60 2024-04-23T02:12:55.121 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:12:55.121 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:12:55.121 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-23T02:12:55.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:12:55.122 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:12:55.122 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-23T02:12:55.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:12:55.122 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:12:55.122 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-23T02:12:55.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:12:55.122 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:12:55.122 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-23T02:12:55.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:12:55.122 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:12:55.122 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-04-23T02:12:55.123 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:12:55.123 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:12:55.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.119+0000 7f43a0137700 1 -- 172.21.15.71:0/468417668 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f438406c4e0 msgr2=0x7f438406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:55.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.119+0000 7f43a0137700 1 --2- 172.21.15.71:0/468417668 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f438406c4e0 0x7f438406e9a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4388011490 tx=0x7f4388009260 comp rx=0 tx=0).stop 2024-04-23T02:12:55.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.119+0000 7f43a0137700 1 -- 172.21.15.71:0/468417668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43980fef60 msgr2=0x7f4398196400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:55.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.119+0000 7f43a0137700 1 --2- 172.21.15.71:0/468417668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43980fef60 0x7f4398196400 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4394000c00 tx=0x7f439400bde0 comp rx=0 tx=0).stop 2024-04-23T02:12:55.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.120+0000 7f43a0137700 1 -- 172.21.15.71:0/468417668 shutdown_connections 2024-04-23T02:12:55.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.120+0000 7f43a0137700 1 --2- 172.21.15.71:0/468417668 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f438406c4e0 0x7f438406e9a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:55.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.120+0000 7f43a0137700 1 --2- 172.21.15.71:0/468417668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43980fef60 0x7f4398196400 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:55.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.120+0000 7f43a0137700 1 --2- 172.21.15.71:0/468417668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4398101f40 0x7f4398196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:55.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.120+0000 7f43a0137700 1 -- 172.21.15.71:0/468417668 >> 172.21.15.71:0/468417668 conn(0x7f43980fa8c0 msgr2=0x7f4398105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:55.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.120+0000 7f43a0137700 1 -- 172.21.15.71:0/468417668 shutdown_connections 2024-04-23T02:12:55.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.120+0000 7f43a0137700 1 -- 172.21.15.71:0/468417668 wait complete. 2024-04-23T02:12:55.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.235+0000 7f54ab55f700 1 -- 172.21.15.71:0/2461751044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a4101eb0 msgr2=0x7f54a4102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:55.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.235+0000 7f54ab55f700 1 --2- 172.21.15.71:0/2461751044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a4101eb0 0x7f54a4102310 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5494009910 tx=0x7f5494009c20 comp rx=0 tx=0).stop 2024-04-23T02:12:55.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.235+0000 7f54ab55f700 1 -- 172.21.15.71:0/2461751044 shutdown_connections 2024-04-23T02:12:55.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.235+0000 7f54ab55f700 1 --2- 172.21.15.71:0/2461751044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a4101eb0 0x7f54a4102310 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:55.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.235+0000 7f54ab55f700 1 --2- 172.21.15.71:0/2461751044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54a4100cb0 0x7f54a41010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:55.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.235+0000 7f54ab55f700 1 -- 172.21.15.71:0/2461751044 >> 172.21.15.71:0/2461751044 conn(0x7f54a40fc230 msgr2=0x7f54a40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:55.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.235+0000 7f54ab55f700 1 -- 172.21.15.71:0/2461751044 shutdown_connections 2024-04-23T02:12:55.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.235+0000 7f54ab55f700 1 -- 172.21.15.71:0/2461751044 wait complete. 2024-04-23T02:12:55.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.236+0000 7f54ab55f700 1 Processor -- start 2024-04-23T02:12:55.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.236+0000 7f54ab55f700 1 -- start start 2024-04-23T02:12:55.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.236+0000 7f54ab55f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a4100cb0 0x7f54a41963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:55.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.236+0000 7f54ab55f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54a4101eb0 0x7f54a41968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:55.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.236+0000 7f54ab55f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a4196e20 con 0x7f54a4100cb0 2024-04-23T02:12:55.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.236+0000 7f54ab55f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a4196f90 con 0x7f54a4101eb0 2024-04-23T02:12:55.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.236+0000 7f54a92fb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a4100cb0 0x7f54a41963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:55.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.236+0000 7f54a92fb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a4100cb0 0x7f54a41963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51046/0 (socket says 172.21.15.71:51046) 2024-04-23T02:12:55.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.236+0000 7f54a92fb700 1 -- 172.21.15.71:0/3122271306 learned_addr learned my addr 172.21.15.71:0/3122271306 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:12:55.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.236+0000 7f54a8afa700 1 --2- 172.21.15.71:0/3122271306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54a4101eb0 0x7f54a41968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:55.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.236+0000 7f54a92fb700 1 -- 172.21.15.71:0/3122271306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54a4101eb0 msgr2=0x7f54a41968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:55.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.236+0000 7f54a92fb700 1 --2- 172.21.15.71:0/3122271306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54a4101eb0 0x7f54a41968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:55.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.236+0000 7f54a92fb700 1 -- 172.21.15.71:0/3122271306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54940095f0 con 0x7f54a4100cb0 2024-04-23T02:12:55.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.237+0000 7f54a92fb700 1 --2- 172.21.15.71:0/3122271306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a4100cb0 0x7f54a41963a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f54a000e8f0 tx=0x7f54a000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:55.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.238+0000 7f549a7fc700 1 -- 172.21.15.71:0/3122271306 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54a0009790 con 0x7f54a4100cb0 2024-04-23T02:12:55.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.238+0000 7f54ab55f700 1 -- 172.21.15.71:0/3122271306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a4197970 con 0x7f54a4100cb0 2024-04-23T02:12:55.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.239+0000 7f54ab55f700 1 -- 172.21.15.71:0/3122271306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a4197e70 con 0x7f54a4100cb0 2024-04-23T02:12:55.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.239+0000 7f549a7fc700 1 -- 172.21.15.71:0/3122271306 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f54a000c810 con 0x7f54a4100cb0 2024-04-23T02:12:55.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.239+0000 7f549a7fc700 1 -- 172.21.15.71:0/3122271306 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54a0003c20 con 0x7f54a4100cb0 2024-04-23T02:12:55.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.240+0000 7f54ab55f700 1 -- 172.21.15.71:0/3122271306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5488005320 con 0x7f54a4100cb0 2024-04-23T02:12:55.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.240+0000 7f549a7fc700 1 -- 172.21.15.71:0/3122271306 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f54a0018430 con 0x7f54a4100cb0 2024-04-23T02:12:55.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.240+0000 7f549a7fc700 1 --2- 172.21.15.71:0/3122271306 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f549006c530 0x7f549006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:12:55.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.240+0000 7f549a7fc700 1 -- 172.21.15.71:0/3122271306 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f54a0014070 con 0x7f54a4100cb0 2024-04-23T02:12:55.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.241+0000 7f54a8afa700 1 --2- 172.21.15.71:0/3122271306 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f549006c530 0x7f549006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:12:55.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.241+0000 7f54a8afa700 1 --2- 172.21.15.71:0/3122271306 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f549006c530 0x7f549006e9f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f54a41974d0 tx=0x7f549400b5a0 comp rx=0 tx=0).ready entity=mgr.14215 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:12:55.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.245+0000 7f549a7fc700 1 -- 172.21.15.71:0/3122271306 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f54a0056340 con 0x7f54a4100cb0 2024-04-23T02:12:55.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:55 smithi071 ceph-mon[39837]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:12:55.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:55 smithi071 ceph-mon[39837]: pgmap v250: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:55.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:55 smithi071 ceph-mon[39837]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:12:55.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:55 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/468417668' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:12:55.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.504+0000 7f54ab55f700 1 -- 172.21.15.71:0/3122271306 --> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5488000bf0 con 0x7f549006c530 2024-04-23T02:12:55.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.505+0000 7f549a7fc700 1 -- 172.21.15.71:0/3122271306 <== mgr.14215 v2:172.21.15.71:6800/4249907286 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f5488000bf0 con 0x7f549006c530 2024-04-23T02:12:55.509 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:12:55.509 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:12:55.510 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:12:55.510 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-23T02:12:55.510 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [], 2024-04-23T02:12:55.510 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "0/2 daemons upgraded", 2024-04-23T02:12:55.510 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 image on host smithi192", 2024-04-23T02:12:55.510 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:12:55.510 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:12:55.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.508+0000 7f54ab55f700 1 -- 172.21.15.71:0/3122271306 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f549006c530 msgr2=0x7f549006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:55.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.508+0000 7f54ab55f700 1 --2- 172.21.15.71:0/3122271306 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f549006c530 0x7f549006e9f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f54a41974d0 tx=0x7f549400b5a0 comp rx=0 tx=0).stop 2024-04-23T02:12:55.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.508+0000 7f54ab55f700 1 -- 172.21.15.71:0/3122271306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a4100cb0 msgr2=0x7f54a41963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:12:55.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.508+0000 7f54ab55f700 1 --2- 172.21.15.71:0/3122271306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a4100cb0 0x7f54a41963a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f54a000e8f0 tx=0x7f54a000ec00 comp rx=0 tx=0).stop 2024-04-23T02:12:55.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.508+0000 7f54ab55f700 1 -- 172.21.15.71:0/3122271306 shutdown_connections 2024-04-23T02:12:55.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.508+0000 7f54ab55f700 1 --2- 172.21.15.71:0/3122271306 >> [v2:172.21.15.71:6800/4249907286,v1:172.21.15.71:6801/4249907286] conn(0x7f549006c530 0x7f549006e9f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:55.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.508+0000 7f54ab55f700 1 --2- 172.21.15.71:0/3122271306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a4100cb0 0x7f54a41963a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:55.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.508+0000 7f54ab55f700 1 --2- 172.21.15.71:0/3122271306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54a4101eb0 0x7f54a41968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:12:55.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.508+0000 7f54ab55f700 1 -- 172.21.15.71:0/3122271306 >> 172.21.15.71:0/3122271306 conn(0x7f54a40fc230 msgr2=0x7f54a41050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:12:55.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.508+0000 7f54ab55f700 1 -- 172.21.15.71:0/3122271306 shutdown_connections 2024-04-23T02:12:55.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:12:55.508+0000 7f54ab55f700 1 -- 172.21.15.71:0/3122271306 wait complete. 2024-04-23T02:12:55.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:55 smithi192 ceph-mon[49662]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:12:55.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:55 smithi192 ceph-mon[49662]: pgmap v250: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:12:55.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:55 smithi192 ceph-mon[49662]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:12:55.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:55 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/468417668' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:12:55.861 INFO:tasks.workunit.client.0.smithi071.stderr:+ tar xzf ltp-full.tgz 2024-04-23T02:12:56.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:56 smithi192 ceph-mon[49662]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:12:56.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:56 smithi192 ceph-mon[49662]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:12:56.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:56 smithi071 ceph-mon[39837]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:12:56.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:56 smithi071 ceph-mon[39837]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:12:57.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:57 smithi192 ceph-mon[49662]: pgmap v251: 65 pgs: 65 active+clean; 13 MiB data, 193 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s rd, 1.1 MiB/s wr, 32 op/s 2024-04-23T02:12:57.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:57 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:12:57.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:57 smithi071 ceph-mon[39837]: pgmap v251: 65 pgs: 65 active+clean; 13 MiB data, 193 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s rd, 1.1 MiB/s wr, 32 op/s 2024-04-23T02:12:57.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:57 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:12:59.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:59 smithi192 ceph-mon[49662]: pgmap v252: 65 pgs: 65 active+clean; 14 MiB data, 211 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s rd, 1.1 MiB/s wr, 40 op/s 2024-04-23T02:12:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:59 smithi192 ceph-mon[49662]: Upgrade: Updating mgr.smithi192.qgecnv 2024-04-23T02:12:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:59 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:12:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:59 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.qgecnv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T02:12:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:59 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T02:12:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:59 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:12:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:12:59 smithi192 ceph-mon[49662]: Deploying daemon mgr.smithi192.qgecnv on smithi192 2024-04-23T02:12:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:59 smithi071 ceph-mon[39837]: pgmap v252: 65 pgs: 65 active+clean; 14 MiB data, 211 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s rd, 1.1 MiB/s wr, 40 op/s 2024-04-23T02:12:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:59 smithi071 ceph-mon[39837]: Upgrade: Updating mgr.smithi192.qgecnv 2024-04-23T02:12:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:59 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:12:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:59 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.qgecnv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T02:12:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:59 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T02:12:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:59 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:12:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:12:59 smithi071 ceph-mon[39837]: Deploying daemon mgr.smithi192.qgecnv on smithi192 2024-04-23T02:13:01.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:01 smithi192 ceph-mon[49662]: pgmap v253: 65 pgs: 65 active+clean; 25 MiB data, 237 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s rd, 2.1 MiB/s wr, 76 op/s 2024-04-23T02:13:01.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:01 smithi071 ceph-mon[39837]: pgmap v253: 65 pgs: 65 active+clean; 25 MiB data, 237 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s rd, 2.1 MiB/s wr, 76 op/s 2024-04-23T02:13:03.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:03 smithi071 ceph-mon[39837]: pgmap v254: 65 pgs: 65 active+clean; 37 MiB data, 291 MiB used, 536 GiB / 536 GiB avail; 684 KiB/s rd, 3.1 MiB/s wr, 126 op/s 2024-04-23T02:13:03.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:03 smithi071 ceph-mon[39837]: fsmap cephfs:2 {0=cephfs.smithi071.vxejye=up:active,1=cephfs.smithi192.gtmyst=up:active} 2 up:standby-replay 2024-04-23T02:13:03.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:03 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:13:03.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:03 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:13:03.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:03 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:13:03.745 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:03 smithi192 ceph-mon[49662]: pgmap v254: 65 pgs: 65 active+clean; 37 MiB data, 291 MiB used, 536 GiB / 536 GiB avail; 684 KiB/s rd, 3.1 MiB/s wr, 126 op/s 2024-04-23T02:13:03.745 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:03 smithi192 ceph-mon[49662]: fsmap cephfs:2 {0=cephfs.smithi071.vxejye=up:active,1=cephfs.smithi192.gtmyst=up:active} 2 up:standby-replay 2024-04-23T02:13:03.746 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:03 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:13:03.746 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:03 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:13:03.746 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:03 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:13:05.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:05 smithi192 ceph-mon[49662]: pgmap v255: 65 pgs: 65 active+clean; 37 MiB data, 291 MiB used, 536 GiB / 536 GiB avail; 684 KiB/s rd, 3.1 MiB/s wr, 125 op/s 2024-04-23T02:13:05.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:05 smithi071 ceph-mon[39837]: pgmap v255: 65 pgs: 65 active+clean; 37 MiB data, 291 MiB used, 536 GiB / 536 GiB avail; 684 KiB/s rd, 3.1 MiB/s wr, 125 op/s 2024-04-23T02:13:07.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:06 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:13:07.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:06 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:13:07.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:06 smithi071 ceph-mon[39837]: pgmap v256: 65 pgs: 65 active+clean; 59 MiB data, 355 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 187 op/s 2024-04-23T02:13:07.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:06 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:13:07.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:06 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:13:07.253 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:06 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:13:07.253 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:06 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:13:07.253 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:06 smithi192 ceph-mon[49662]: pgmap v256: 65 pgs: 65 active+clean; 59 MiB data, 355 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 187 op/s 2024-04-23T02:13:07.253 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:06 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:13:07.253 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:06 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:13:09.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:09 smithi071 ceph-mon[39837]: pgmap v257: 65 pgs: 65 active+clean; 65 MiB data, 372 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 173 op/s 2024-04-23T02:13:09.523 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:09 smithi192 ceph-mon[49662]: pgmap v257: 65 pgs: 65 active+clean; 65 MiB data, 372 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 173 op/s 2024-04-23T02:13:11.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:11 smithi192 ceph-mon[49662]: pgmap v258: 65 pgs: 65 active+clean; 81 MiB data, 418 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 201 op/s 2024-04-23T02:13:11.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:11 smithi192 ceph-mon[49662]: Standby manager daemon smithi192.qgecnv restarted 2024-04-23T02:13:11.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:11 smithi192 ceph-mon[49662]: Standby manager daemon smithi192.qgecnv started 2024-04-23T02:13:11.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:11 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.qgecnv/crt"}]: dispatch 2024-04-23T02:13:11.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:11 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T02:13:11.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:11 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.qgecnv/key"}]: dispatch 2024-04-23T02:13:11.355 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:11 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T02:13:11.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:11 smithi071 ceph-mon[39837]: pgmap v258: 65 pgs: 65 active+clean; 81 MiB data, 418 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 201 op/s 2024-04-23T02:13:11.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:11 smithi071 ceph-mon[39837]: Standby manager daemon smithi192.qgecnv restarted 2024-04-23T02:13:11.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:11 smithi071 ceph-mon[39837]: Standby manager daemon smithi192.qgecnv started 2024-04-23T02:13:11.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:11 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.qgecnv/crt"}]: dispatch 2024-04-23T02:13:11.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:11 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T02:13:11.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:11 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.qgecnv/key"}]: dispatch 2024-04-23T02:13:11.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:11 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T02:13:12.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:13:12.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:13:12.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:13:12.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:13:12.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:13:12.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:13:12.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr fail", "who": "smithi071.ukdjvn"}]: dispatch 2024-04-23T02:13:12.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: osdmap e47: 6 total, 6 up, 6 in 2024-04-23T02:13:12.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "mgr fail", "who": "smithi071.ukdjvn"}]': finished 2024-04-23T02:13:12.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: mgrmap e20: smithi192.qgecnv(active, starting, since 0.00608923s) 2024-04-23T02:13:12.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mon metadata", "id": "smithi071"}]: dispatch 2024-04-23T02:13:12.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:13:12.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.vxejye"}]: dispatch 2024-04-23T02:13:12.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.qusrrb"}]: dispatch 2024-04-23T02:13:12.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.gtmyst"}]: dispatch 2024-04-23T02:13:12.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.azdyze"}]: dispatch 2024-04-23T02:13:12.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mgr metadata", "who": "smithi192.qgecnv", "id": "smithi192.qgecnv"}]: dispatch 2024-04-23T02:13:12.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T02:13:12.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T02:13:12.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T02:13:12.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T02:13:12.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T02:13:12.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T02:13:12.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T02:13:12.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T02:13:12.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T02:13:12.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: Manager daemon smithi192.qgecnv is now available 2024-04-23T02:13:12.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:13:12.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:13:12.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.qgecnv/mirror_snapshot_schedule"}]: dispatch 2024-04-23T02:13:12.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.qgecnv/mirror_snapshot_schedule"}]: dispatch 2024-04-23T02:13:12.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.qgecnv/trash_purge_schedule"}]: dispatch 2024-04-23T02:13:12.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.qgecnv/trash_purge_schedule"}]: dispatch 2024-04-23T02:13:12.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: [23/Apr/2024:02:13:12] ENGINE Bus STARTING 2024-04-23T02:13:12.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: Deploying cephadm binary to smithi192 2024-04-23T02:13:12.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: Deploying cephadm binary to smithi071 2024-04-23T02:13:12.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: [23/Apr/2024:02:13:12] ENGINE Serving on https://172.21.15.192:7150 2024-04-23T02:13:12.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: [23/Apr/2024:02:13:12] ENGINE Serving on http://172.21.15.192:8765 2024-04-23T02:13:12.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:12 smithi071 ceph-mon[39837]: [23/Apr/2024:02:13:12] ENGINE Bus STARTED 2024-04-23T02:13:12.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:13:12.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:13:12.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:13:12.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:13:12.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:13:12.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:13:12.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr fail", "who": "smithi071.ukdjvn"}]: dispatch 2024-04-23T02:13:12.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: osdmap e47: 6 total, 6 up, 6 in 2024-04-23T02:13:12.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.14215 172.21.15.71:0/1084198518' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "mgr fail", "who": "smithi071.ukdjvn"}]': finished 2024-04-23T02:13:12.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: mgrmap e20: smithi192.qgecnv(active, starting, since 0.00608923s) 2024-04-23T02:13:12.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mon metadata", "id": "smithi071"}]: dispatch 2024-04-23T02:13:12.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:13:12.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.vxejye"}]: dispatch 2024-04-23T02:13:12.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.qusrrb"}]: dispatch 2024-04-23T02:13:12.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.gtmyst"}]: dispatch 2024-04-23T02:13:12.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.azdyze"}]: dispatch 2024-04-23T02:13:12.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mgr metadata", "who": "smithi192.qgecnv", "id": "smithi192.qgecnv"}]: dispatch 2024-04-23T02:13:12.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T02:13:12.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T02:13:12.879 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T02:13:12.879 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T02:13:12.879 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T02:13:12.879 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T02:13:12.879 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T02:13:12.879 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T02:13:12.879 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T02:13:12.879 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: Manager daemon smithi192.qgecnv is now available 2024-04-23T02:13:12.879 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:13:12.879 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:13:12.879 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.qgecnv/mirror_snapshot_schedule"}]: dispatch 2024-04-23T02:13:12.879 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.qgecnv/mirror_snapshot_schedule"}]: dispatch 2024-04-23T02:13:12.879 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.qgecnv/trash_purge_schedule"}]: dispatch 2024-04-23T02:13:12.879 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi192.qgecnv/trash_purge_schedule"}]: dispatch 2024-04-23T02:13:12.879 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: [23/Apr/2024:02:13:12] ENGINE Bus STARTING 2024-04-23T02:13:12.880 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: Deploying cephadm binary to smithi192 2024-04-23T02:13:12.880 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: Deploying cephadm binary to smithi071 2024-04-23T02:13:12.880 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: [23/Apr/2024:02:13:12] ENGINE Serving on https://172.21.15.192:7150 2024-04-23T02:13:12.880 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: [23/Apr/2024:02:13:12] ENGINE Serving on http://172.21.15.192:8765 2024-04-23T02:13:12.880 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:12 smithi192 ceph-mon[49662]: [23/Apr/2024:02:13:12] ENGINE Bus STARTED 2024-04-23T02:13:13.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:13 smithi071 ceph-mon[39837]: mgrmap e21: smithi192.qgecnv(active, since 1.00776s) 2024-04-23T02:13:13.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:13 smithi071 ceph-mon[39837]: pgmap v3: 65 pgs: 65 active+clean; 106 MiB data, 524 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:13:13.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:13 smithi192 ceph-mon[49662]: mgrmap e21: smithi192.qgecnv(active, since 1.00776s) 2024-04-23T02:13:13.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:13 smithi192 ceph-mon[49662]: pgmap v3: 65 pgs: 65 active+clean; 106 MiB data, 524 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:13:14.866 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:14 smithi071 ceph-mon[39837]: pgmap v4: 65 pgs: 65 active+clean; 106 MiB data, 524 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:13:14.867 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:14 smithi071 ceph-mon[39837]: mgrmap e22: smithi192.qgecnv(active, since 2s) 2024-04-23T02:13:14.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:14 smithi192 ceph-mon[49662]: pgmap v4: 65 pgs: 65 active+clean; 106 MiB data, 524 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:13:14.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:14 smithi192 ceph-mon[49662]: mgrmap e22: smithi192.qgecnv(active, since 2s) 2024-04-23T02:13:16.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:16 smithi071 ceph-mon[39837]: pgmap v5: 65 pgs: 65 active+clean; 106 MiB data, 524 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:13:16.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:16 smithi071 ceph-mon[39837]: mgrmap e23: smithi192.qgecnv(active, since 4s) 2024-04-23T02:13:16.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:16 smithi192 ceph-mon[49662]: pgmap v5: 65 pgs: 65 active+clean; 106 MiB data, 524 MiB used, 536 GiB / 536 GiB avail 2024-04-23T02:13:16.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:16 smithi192 ceph-mon[49662]: mgrmap e23: smithi192.qgecnv(active, since 4s) 2024-04-23T02:13:18.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:18 smithi071 ceph-mon[39837]: pgmap v6: 65 pgs: 65 active+clean; 111 MiB data, 537 MiB used, 536 GiB / 536 GiB avail; 821 KiB/s rd, 928 KiB/s wr, 37 op/s 2024-04-23T02:13:18.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:18 smithi192 ceph-mon[49662]: pgmap v6: 65 pgs: 65 active+clean; 111 MiB data, 537 MiB used, 536 GiB / 536 GiB avail; 821 KiB/s rd, 928 KiB/s wr, 37 op/s 2024-04-23T02:13:21.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: pgmap v7: 65 pgs: 65 active+clean; 120 MiB data, 573 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-04-23T02:13:21.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:21.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:21.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: Standby manager daemon smithi071.ukdjvn started 2024-04-23T02:13:21.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.71:0/2826339542' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi071.ukdjvn/crt"}]: dispatch 2024-04-23T02:13:21.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.71:0/2826339542' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T02:13:21.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.71:0/2826339542' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi071.ukdjvn/key"}]: dispatch 2024-04-23T02:13:21.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.71:0/2826339542' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T02:13:21.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:21.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:21.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:21.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:21.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:21.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:21.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:21.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:21.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:21.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:21.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:20 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:21.306 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: pgmap v7: 65 pgs: 65 active+clean; 120 MiB data, 573 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-04-23T02:13:21.306 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:21.306 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:21.306 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: Standby manager daemon smithi071.ukdjvn started 2024-04-23T02:13:21.306 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.71:0/2826339542' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi071.ukdjvn/crt"}]: dispatch 2024-04-23T02:13:21.306 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.71:0/2826339542' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T02:13:21.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.71:0/2826339542' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi071.ukdjvn/key"}]: dispatch 2024-04-23T02:13:21.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.71:0/2826339542' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T02:13:21.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:21.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:21.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:21.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:21.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:21.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:21.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:21.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:21.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:21.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:21.307 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:20 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:22.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:21 smithi192 ceph-mon[49662]: Adjusting osd_memory_target on smithi071 to 1227M 2024-04-23T02:13:22.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:21 smithi192 ceph-mon[49662]: mgrmap e24: smithi192.qgecnv(active, since 9s), standbys: smithi071.ukdjvn 2024-04-23T02:13:22.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:21 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mgr metadata", "who": "smithi071.ukdjvn", "id": "smithi071.ukdjvn"}]: dispatch 2024-04-23T02:13:22.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:21 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:22.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:21 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:22.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:21 smithi071 ceph-mon[39837]: Adjusting osd_memory_target on smithi071 to 1227M 2024-04-23T02:13:22.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:21 smithi071 ceph-mon[39837]: mgrmap e24: smithi192.qgecnv(active, since 9s), standbys: smithi071.ukdjvn 2024-04-23T02:13:22.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:21 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mgr metadata", "who": "smithi071.ukdjvn", "id": "smithi071.ukdjvn"}]: dispatch 2024-04-23T02:13:22.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:21 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:22.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:21 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:23.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:23 smithi071 ceph-mon[39837]: pgmap v8: 65 pgs: 65 active+clean; 120 MiB data, 573 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-04-23T02:13:23.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:23 smithi192 ceph-mon[49662]: pgmap v8: 65 pgs: 65 active+clean; 120 MiB data, 573 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-04-23T02:13:25.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:25 smithi071 ceph-mon[39837]: pgmap v9: 65 pgs: 65 active+clean; 148 MiB data, 677 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 264 op/s 2024-04-23T02:13:25.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:25 smithi192 ceph-mon[49662]: pgmap v9: 65 pgs: 65 active+clean; 148 MiB data, 677 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 264 op/s 2024-04-23T02:13:25.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.636+0000 7f41f89fd700 1 -- 172.21.15.71:0/2686258831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41f4100e10 msgr2=0x7f41f4101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:25.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.636+0000 7f41f89fd700 1 --2- 172.21.15.71:0/2686258831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41f4100e10 0x7f41f4101270 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009910 tx=0x7f41dc009c20 comp rx=0 tx=0).stop 2024-04-23T02:13:25.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.637+0000 7f41f89fd700 1 -- 172.21.15.71:0/2686258831 shutdown_connections 2024-04-23T02:13:25.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.637+0000 7f41f89fd700 1 --2- 172.21.15.71:0/2686258831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41f4100e10 0x7f41f4101270 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:25.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.637+0000 7f41f89fd700 1 --2- 172.21.15.71:0/2686258831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41f40ffc10 0x7f41f4100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:25.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.637+0000 7f41f89fd700 1 -- 172.21.15.71:0/2686258831 >> 172.21.15.71:0/2686258831 conn(0x7f41f40fb190 msgr2=0x7f41f40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:13:25.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.637+0000 7f41f89fd700 1 -- 172.21.15.71:0/2686258831 shutdown_connections 2024-04-23T02:13:25.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.637+0000 7f41f89fd700 1 -- 172.21.15.71:0/2686258831 wait complete. 2024-04-23T02:13:25.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.637+0000 7f41f89fd700 1 Processor -- start 2024-04-23T02:13:25.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.637+0000 7f41f89fd700 1 -- start start 2024-04-23T02:13:25.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.638+0000 7f41f89fd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41f40ffc10 0x7f41f4107bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:25.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.638+0000 7f41f89fd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41f4100e10 0x7f41f4106210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:25.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.638+0000 7f41f89fd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f419a010 con 0x7f41f40ffc10 2024-04-23T02:13:25.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.638+0000 7f41f89fd700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f419a150 con 0x7f41f4100e10 2024-04-23T02:13:25.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.638+0000 7f41f1d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41f4100e10 0x7f41f4106210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:25.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.638+0000 7f41f1d9b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41f4100e10 0x7f41f4106210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.71:47012/0 (socket says 172.21.15.71:47012) 2024-04-23T02:13:25.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.638+0000 7f41f1d9b700 1 -- 172.21.15.71:0/2593668593 learned_addr learned my addr 172.21.15.71:0/2593668593 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:13:25.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.638+0000 7f41f259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41f40ffc10 0x7f41f4107bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:25.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.638+0000 7f41f1d9b700 1 -- 172.21.15.71:0/2593668593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41f40ffc10 msgr2=0x7f41f4107bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:25.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.638+0000 7f41f1d9b700 1 --2- 172.21.15.71:0/2593668593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41f40ffc10 0x7f41f4107bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:25.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.638+0000 7f41f1d9b700 1 -- 172.21.15.71:0/2593668593 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc0095f0 con 0x7f41f4100e10 2024-04-23T02:13:25.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.638+0000 7f41f259c700 1 --2- 172.21.15.71:0/2593668593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41f40ffc10 0x7f41f4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T02:13:25.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.638+0000 7f41f1d9b700 1 --2- 172.21.15.71:0/2593668593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41f4100e10 0x7f41f4106210 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00bde0 tx=0x7f41dc00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:13:25.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.639+0000 7f41eb7fe700 1 -- 172.21.15.71:0/2593668593 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41dc01d070 con 0x7f41f4100e10 2024-04-23T02:13:25.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.639+0000 7f41f89fd700 1 -- 172.21.15.71:0/2593668593 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f419a3d0 con 0x7f41f4100e10 2024-04-23T02:13:25.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.639+0000 7f41f89fd700 1 -- 172.21.15.71:0/2593668593 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f419a860 con 0x7f41f4100e10 2024-04-23T02:13:25.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.639+0000 7f41eb7fe700 1 -- 172.21.15.71:0/2593668593 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f41dc005dc0 con 0x7f41f4100e10 2024-04-23T02:13:25.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.639+0000 7f41eb7fe700 1 -- 172.21.15.71:0/2593668593 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41dc003a90 con 0x7f41f4100e10 2024-04-23T02:13:25.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.640+0000 7f41f89fd700 1 -- 172.21.15.71:0/2593668593 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f41096c0 con 0x7f41f4100e10 2024-04-23T02:13:25.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.641+0000 7f41eb7fe700 1 -- 172.21.15.71:0/2593668593 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 90681+0+0 (secure 0 0 0) 0x7f41dc003bf0 con 0x7f41f4100e10 2024-04-23T02:13:25.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.641+0000 7f41eb7fe700 1 --2- 172.21.15.71:0/2593668593 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f41e006ca20 0x7f41e006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:25.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.641+0000 7f41eb7fe700 1 -- 172.21.15.71:0/2593668593 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f41dc08d850 con 0x7f41f4100e10 2024-04-23T02:13:25.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.641+0000 7f41f259c700 1 --2- 172.21.15.71:0/2593668593 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f41e006ca20 0x7f41e006eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:25.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.642+0000 7f41f259c700 1 --2- 172.21.15.71:0/2593668593 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f41e006ca20 0x7f41e006eee0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f41f4100c70 tx=0x7f41ec00b040 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:13:25.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.647+0000 7f41eb7fe700 1 -- 172.21.15.71:0/2593668593 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f41dc05b330 con 0x7f41f4100e10 2024-04-23T02:13:25.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.940+0000 7f41f89fd700 1 -- 172.21.15.71:0/2593668593 --> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f41f4107090 con 0x7f41e006ca20 2024-04-23T02:13:25.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.941+0000 7f41eb7fe700 1 -- 172.21.15.71:0/2593668593 <== mgr.24351 v2:172.21.15.192:6828/966231696 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f41f4107090 con 0x7f41e006ca20 2024-04-23T02:13:25.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.945+0000 7f41f89fd700 1 -- 172.21.15.71:0/2593668593 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f41e006ca20 msgr2=0x7f41e006eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:25.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.945+0000 7f41f89fd700 1 --2- 172.21.15.71:0/2593668593 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f41e006ca20 0x7f41e006eee0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f41f4100c70 tx=0x7f41ec00b040 comp rx=0 tx=0).stop 2024-04-23T02:13:25.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.945+0000 7f41f89fd700 1 -- 172.21.15.71:0/2593668593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41f4100e10 msgr2=0x7f41f4106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:25.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.945+0000 7f41f89fd700 1 --2- 172.21.15.71:0/2593668593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41f4100e10 0x7f41f4106210 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00bde0 tx=0x7f41dc00be10 comp rx=0 tx=0).stop 2024-04-23T02:13:25.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.945+0000 7f41f89fd700 1 -- 172.21.15.71:0/2593668593 shutdown_connections 2024-04-23T02:13:25.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.945+0000 7f41f89fd700 1 --2- 172.21.15.71:0/2593668593 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f41e006ca20 0x7f41e006eee0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:25.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.945+0000 7f41f89fd700 1 --2- 172.21.15.71:0/2593668593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41f40ffc10 0x7f41f4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:25.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.945+0000 7f41f89fd700 1 --2- 172.21.15.71:0/2593668593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41f4100e10 0x7f41f4106210 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:25.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.945+0000 7f41f89fd700 1 -- 172.21.15.71:0/2593668593 >> 172.21.15.71:0/2593668593 conn(0x7f41f40fb190 msgr2=0x7f41f4104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:13:25.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.945+0000 7f41f89fd700 1 -- 172.21.15.71:0/2593668593 shutdown_connections 2024-04-23T02:13:25.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:25.945+0000 7f41f89fd700 1 -- 172.21.15.71:0/2593668593 wait complete. 2024-04-23T02:13:25.962 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:13:26.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.071+0000 7f24e9fcc700 1 -- 172.21.15.71:0/2530430878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24e40fef60 msgr2=0x7f24e40ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:26.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.071+0000 7f24e9fcc700 1 --2- 172.21.15.71:0/2530430878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24e40fef60 0x7f24e40ff380 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009910 tx=0x7f24d4009c20 comp rx=0 tx=0).stop 2024-04-23T02:13:26.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.071+0000 7f24e9fcc700 1 -- 172.21.15.71:0/2530430878 shutdown_connections 2024-04-23T02:13:26.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.071+0000 7f24e9fcc700 1 --2- 172.21.15.71:0/2530430878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24e4101f40 0x7f24e41023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:26.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.071+0000 7f24e9fcc700 1 --2- 172.21.15.71:0/2530430878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24e40fef60 0x7f24e40ff380 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:26.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.071+0000 7f24e9fcc700 1 -- 172.21.15.71:0/2530430878 >> 172.21.15.71:0/2530430878 conn(0x7f24e40fa8c0 msgr2=0x7f24e40fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:13:26.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.072+0000 7f24e9fcc700 1 -- 172.21.15.71:0/2530430878 shutdown_connections 2024-04-23T02:13:26.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.072+0000 7f24e9fcc700 1 -- 172.21.15.71:0/2530430878 wait complete. 2024-04-23T02:13:26.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.072+0000 7f24e9fcc700 1 Processor -- start 2024-04-23T02:13:26.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.073+0000 7f24e9fcc700 1 -- start start 2024-04-23T02:13:26.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.073+0000 7f24e9fcc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24e40fef60 0x7f24e4196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:26.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.073+0000 7f24e9fcc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24e4101f40 0x7f24e4196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:26.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.073+0000 7f24e9fcc700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e4196f60 con 0x7f24e40fef60 2024-04-23T02:13:26.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.073+0000 7f24e9fcc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e41970a0 con 0x7f24e4101f40 2024-04-23T02:13:26.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.073+0000 7f24e2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24e4101f40 0x7f24e4196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:26.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.073+0000 7f24e2ffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24e4101f40 0x7f24e4196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.71:47030/0 (socket says 172.21.15.71:47030) 2024-04-23T02:13:26.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.073+0000 7f24e2ffd700 1 -- 172.21.15.71:0/736589584 learned_addr learned my addr 172.21.15.71:0/736589584 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:13:26.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.073+0000 7f24e37fe700 1 --2- 172.21.15.71:0/736589584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24e40fef60 0x7f24e4196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:26.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.074+0000 7f24e2ffd700 1 -- 172.21.15.71:0/736589584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24e40fef60 msgr2=0x7f24e4196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:26.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.074+0000 7f24e2ffd700 1 --2- 172.21.15.71:0/736589584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24e40fef60 0x7f24e4196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:26.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.074+0000 7f24e2ffd700 1 -- 172.21.15.71:0/736589584 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d40095f0 con 0x7f24e4101f40 2024-04-23T02:13:26.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.074+0000 7f24e37fe700 1 --2- 172.21.15.71:0/736589584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24e40fef60 0x7f24e4196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T02:13:26.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.074+0000 7f24e2ffd700 1 --2- 172.21.15.71:0/736589584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24e4101f40 0x7f24e4196940 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f24d800d8d0 tx=0x7f24d800dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:13:26.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.074+0000 7f24e0ff9700 1 -- 172.21.15.71:0/736589584 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24d80098a0 con 0x7f24e4101f40 2024-04-23T02:13:26.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.074+0000 7f24e0ff9700 1 -- 172.21.15.71:0/736589584 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f24d8009e90 con 0x7f24e4101f40 2024-04-23T02:13:26.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.074+0000 7f24e0ff9700 1 -- 172.21.15.71:0/736589584 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24d800f5f0 con 0x7f24e4101f40 2024-04-23T02:13:26.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.074+0000 7f24e9fcc700 1 -- 172.21.15.71:0/736589584 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e419bb50 con 0x7f24e4101f40 2024-04-23T02:13:26.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.074+0000 7f24e9fcc700 1 -- 172.21.15.71:0/736589584 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e419c0a0 con 0x7f24e4101f40 2024-04-23T02:13:26.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.076+0000 7f24e9fcc700 1 -- 172.21.15.71:0/736589584 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e41096c0 con 0x7f24e4101f40 2024-04-23T02:13:26.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.077+0000 7f24e0ff9700 1 -- 172.21.15.71:0/736589584 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 90681+0+0 (secure 0 0 0) 0x7f24d8010460 con 0x7f24e4101f40 2024-04-23T02:13:26.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.077+0000 7f24e0ff9700 1 --2- 172.21.15.71:0/736589584 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f24cc06ca20 0x7f24cc06eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:26.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.077+0000 7f24e0ff9700 1 -- 172.21.15.71:0/736589584 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f24d808bee0 con 0x7f24e4101f40 2024-04-23T02:13:26.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.078+0000 7f24e37fe700 1 --2- 172.21.15.71:0/736589584 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f24cc06ca20 0x7f24cc06eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:26.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.079+0000 7f24e37fe700 1 --2- 172.21.15.71:0/736589584 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f24cc06ca20 0x7f24cc06eee0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f24d4006010 tx=0x7f24d400b540 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:13:26.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.085+0000 7f24e0ff9700 1 -- 172.21.15.71:0/736589584 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f24d8059a40 con 0x7f24e4101f40 2024-04-23T02:13:26.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.374+0000 7f24e9fcc700 1 -- 172.21.15.71:0/736589584 --> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f24e419bce0 con 0x7f24cc06ca20 2024-04-23T02:13:26.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.376+0000 7f24e0ff9700 1 -- 172.21.15.71:0/736589584 <== mgr.24351 v2:172.21.15.192:6828/966231696 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f24e419bce0 con 0x7f24cc06ca20 2024-04-23T02:13:26.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.378+0000 7f24e9fcc700 1 -- 172.21.15.71:0/736589584 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f24cc06ca20 msgr2=0x7f24cc06eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:26.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.378+0000 7f24e9fcc700 1 --2- 172.21.15.71:0/736589584 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f24cc06ca20 0x7f24cc06eee0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f24d4006010 tx=0x7f24d400b540 comp rx=0 tx=0).stop 2024-04-23T02:13:26.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.378+0000 7f24e9fcc700 1 -- 172.21.15.71:0/736589584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24e4101f40 msgr2=0x7f24e4196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:26.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.378+0000 7f24e9fcc700 1 --2- 172.21.15.71:0/736589584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24e4101f40 0x7f24e4196940 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f24d800d8d0 tx=0x7f24d800dbe0 comp rx=0 tx=0).stop 2024-04-23T02:13:26.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.378+0000 7f24e9fcc700 1 -- 172.21.15.71:0/736589584 shutdown_connections 2024-04-23T02:13:26.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.378+0000 7f24e9fcc700 1 --2- 172.21.15.71:0/736589584 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f24cc06ca20 0x7f24cc06eee0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:26.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.378+0000 7f24e9fcc700 1 --2- 172.21.15.71:0/736589584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24e40fef60 0x7f24e4196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:26.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.378+0000 7f24e9fcc700 1 --2- 172.21.15.71:0/736589584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24e4101f40 0x7f24e4196940 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:26.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.378+0000 7f24e9fcc700 1 -- 172.21.15.71:0/736589584 >> 172.21.15.71:0/736589584 conn(0x7f24e40fa8c0 msgr2=0x7f24e4105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:13:26.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.378+0000 7f24e9fcc700 1 -- 172.21.15.71:0/736589584 shutdown_connections 2024-04-23T02:13:26.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.378+0000 7f24e9fcc700 1 -- 172.21.15.71:0/736589584 wait complete. 2024-04-23T02:13:26.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.520+0000 7f42df7d9700 1 -- 172.21.15.71:0/4114341096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d8100da0 msgr2=0x7f42d8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:26.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.520+0000 7f42df7d9700 1 --2- 172.21.15.71:0/4114341096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d8100da0 0x7f42d8101200 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009910 tx=0x7f42c8009c20 comp rx=0 tx=0).stop 2024-04-23T02:13:26.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.520+0000 7f42df7d9700 1 -- 172.21.15.71:0/4114341096 shutdown_connections 2024-04-23T02:13:26.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.520+0000 7f42df7d9700 1 --2- 172.21.15.71:0/4114341096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d8100da0 0x7f42d8101200 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:26.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.520+0000 7f42df7d9700 1 --2- 172.21.15.71:0/4114341096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42d80ffba0 0x7f42d80fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:26.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.520+0000 7f42df7d9700 1 -- 172.21.15.71:0/4114341096 >> 172.21.15.71:0/4114341096 conn(0x7f42d80fb120 msgr2=0x7f42d80fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:13:26.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.520+0000 7f42df7d9700 1 -- 172.21.15.71:0/4114341096 shutdown_connections 2024-04-23T02:13:26.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.520+0000 7f42df7d9700 1 -- 172.21.15.71:0/4114341096 wait complete. 2024-04-23T02:13:26.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.521+0000 7f42df7d9700 1 Processor -- start 2024-04-23T02:13:26.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.521+0000 7f42df7d9700 1 -- start start 2024-04-23T02:13:26.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.521+0000 7f42df7d9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d80ffba0 0x7f42d81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:26.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.521+0000 7f42df7d9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42d8100da0 0x7f42d8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:26.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.521+0000 7f42df7d9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8199100 con 0x7f42d8100da0 2024-04-23T02:13:26.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.521+0000 7f42df7d9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8199240 con 0x7f42d80ffba0 2024-04-23T02:13:26.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.522+0000 7f42dd575700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d80ffba0 0x7f42d81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:26.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.522+0000 7f42dd575700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d80ffba0 0x7f42d81985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.71:47058/0 (socket says 172.21.15.71:47058) 2024-04-23T02:13:26.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.522+0000 7f42dd575700 1 -- 172.21.15.71:0/2349496720 learned_addr learned my addr 172.21.15.71:0/2349496720 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:13:26.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.522+0000 7f42dcd74700 1 --2- 172.21.15.71:0/2349496720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42d8100da0 0x7f42d8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:26.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.522+0000 7f42dd575700 1 -- 172.21.15.71:0/2349496720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42d8100da0 msgr2=0x7f42d8198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:26.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.522+0000 7f42dd575700 1 --2- 172.21.15.71:0/2349496720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42d8100da0 0x7f42d8198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:26.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.522+0000 7f42dd575700 1 -- 172.21.15.71:0/2349496720 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c80095f0 con 0x7f42d80ffba0 2024-04-23T02:13:26.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.522+0000 7f42dd575700 1 --2- 172.21.15.71:0/2349496720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d80ffba0 0x7f42d81985a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f42d400d900 tx=0x7f42d400dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:13:26.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.522+0000 7f42ce7fc700 1 -- 172.21.15.71:0/2349496720 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42d40041f0 con 0x7f42d80ffba0 2024-04-23T02:13:26.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.522+0000 7f42ce7fc700 1 -- 172.21.15.71:0/2349496720 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f42d4004350 con 0x7f42d80ffba0 2024-04-23T02:13:26.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.522+0000 7f42df7d9700 1 -- 172.21.15.71:0/2349496720 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d819dcf0 con 0x7f42d80ffba0 2024-04-23T02:13:26.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.523+0000 7f42ce7fc700 1 -- 172.21.15.71:0/2349496720 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42d4003c10 con 0x7f42d80ffba0 2024-04-23T02:13:26.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.523+0000 7f42df7d9700 1 -- 172.21.15.71:0/2349496720 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d819e1e0 con 0x7f42d80ffba0 2024-04-23T02:13:26.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.524+0000 7f42df7d9700 1 -- 172.21.15.71:0/2349496720 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d81074d0 con 0x7f42d80ffba0 2024-04-23T02:13:26.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.525+0000 7f42ce7fc700 1 -- 172.21.15.71:0/2349496720 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) v1 ==== 90681+0+0 (secure 0 0 0) 0x7f42d4009730 con 0x7f42d80ffba0 2024-04-23T02:13:26.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.525+0000 7f42ce7fc700 1 --2- 172.21.15.71:0/2349496720 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f42c406ca10 0x7f42c406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:26.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.525+0000 7f42ce7fc700 1 -- 172.21.15.71:0/2349496720 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f42d408bff0 con 0x7f42d80ffba0 2024-04-23T02:13:26.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.525+0000 7f42dcd74700 1 --2- 172.21.15.71:0/2349496720 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f42c406ca10 0x7f42c406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:26.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.526+0000 7f42dcd74700 1 --2- 172.21.15.71:0/2349496720 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f42c406ca10 0x7f42c406eed0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f42c8006010 tx=0x7f42c800b560 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:13:26.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.530+0000 7f42ce7fc700 1 -- 172.21.15.71:0/2349496720 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f42d4059b50 con 0x7f42d80ffba0 2024-04-23T02:13:26.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.821+0000 7f42df7d9700 1 -- 172.21.15.71:0/2349496720 --> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f42d81056f0 con 0x7f42c406ca10 2024-04-23T02:13:26.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.834+0000 7f42ce7fc700 1 -- 172.21.15.71:0/2349496720 <== mgr.24351 v2:172.21.15.192:6828/966231696 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f42d81056f0 con 0x7f42c406ca10 2024-04-23T02:13:26.838 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:13:26.839 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (6m) 6s ago 8m 26.3M - 0.25.0 c8568f914cd2 8261e17319fe 2024-04-23T02:13:26.839 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (8m) 6s ago 8m 7700k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:13:26.839 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (7m) 6s ago 7m 7616k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:13:26.839 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (8m) 6s ago 8m 6656k - 18.2.1 5be31c24972a 258512dcbc00 2024-04-23T02:13:26.839 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (7m) 6s ago 7m 6656k - 18.2.1 5be31c24972a feea2c9896a6 2024-04-23T02:13:26.839 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (6m) 6s ago 8m 89.3M - 9.4.7 954c08fa6188 48ea3e02b806 2024-04-23T02:13:26.839 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (2m) 6s ago 2m 74.4M - 18.2.1 5be31c24972a a515de2a841c 2024-04-23T02:13:26.839 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (2m) 6s ago 2m 101M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:13:26.839 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (2m) 6s ago 2m 63.8M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:13:26.839 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (2m) 6s ago 2m 76.6M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:13:26.839 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:9283,8765,8443 running (10m) 6s ago 10m 200M - 18.2.1 5be31c24972a fbe1ad3229f5 2024-04-23T02:13:26.839 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (25s) 6s ago 7m 483M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 151cb2226032 2024-04-23T02:13:26.839 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (10m) 6s ago 10m 53.0M 2048M 18.2.1 5be31c24972a ff40abe63fd9 2024-04-23T02:13:26.839 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (7m) 6s ago 7m 40.3M 2048M 18.2.1 5be31c24972a 9c03e90acc95 2024-04-23T02:13:26.839 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (8m) 6s ago 8m 19.2M - 1.5.0 0da6a335fe13 3f4d0aaee04e 2024-04-23T02:13:26.840 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (7m) 6s ago 7m 16.7M - 1.5.0 0da6a335fe13 cfd0b49d770c 2024-04-23T02:13:26.840 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (6m) 6s ago 6m 72.0M 1227M 18.2.1 5be31c24972a d3d5f2fa7267 2024-04-23T02:13:26.840 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (5m) 6s ago 5m 77.9M 1227M 18.2.1 5be31c24972a 3992157ff0fd 2024-04-23T02:13:26.840 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (5m) 6s ago 5m 65.2M 1227M 18.2.1 5be31c24972a 12380b6a15a2 2024-04-23T02:13:26.840 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (5m) 6s ago 5m 79.5M 4982M 18.2.1 5be31c24972a d5f587ee3a22 2024-04-23T02:13:26.840 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (4m) 6s ago 4m 72.5M 4982M 18.2.1 5be31c24972a 958d046005f0 2024-04-23T02:13:26.840 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (4m) 6s ago 4m 69.2M 4982M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:13:26.840 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (6m) 6s ago 7m 60.7M - 2.43.0 a07b618ecd1d 8df24da82462 2024-04-23T02:13:26.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.838+0000 7f42df7d9700 1 -- 172.21.15.71:0/2349496720 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f42c406ca10 msgr2=0x7f42c406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:26.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.838+0000 7f42df7d9700 1 --2- 172.21.15.71:0/2349496720 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f42c406ca10 0x7f42c406eed0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f42c8006010 tx=0x7f42c800b560 comp rx=0 tx=0).stop 2024-04-23T02:13:26.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.838+0000 7f42df7d9700 1 -- 172.21.15.71:0/2349496720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d80ffba0 msgr2=0x7f42d81985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:26.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.838+0000 7f42df7d9700 1 --2- 172.21.15.71:0/2349496720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d80ffba0 0x7f42d81985a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f42d400d900 tx=0x7f42d400dc10 comp rx=0 tx=0).stop 2024-04-23T02:13:26.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.838+0000 7f42df7d9700 1 -- 172.21.15.71:0/2349496720 shutdown_connections 2024-04-23T02:13:26.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.838+0000 7f42df7d9700 1 --2- 172.21.15.71:0/2349496720 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f42c406ca10 0x7f42c406eed0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:26.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.838+0000 7f42df7d9700 1 --2- 172.21.15.71:0/2349496720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d80ffba0 0x7f42d81985a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:26.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.838+0000 7f42df7d9700 1 --2- 172.21.15.71:0/2349496720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42d8100da0 0x7f42d8198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:26.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.838+0000 7f42df7d9700 1 -- 172.21.15.71:0/2349496720 >> 172.21.15.71:0/2349496720 conn(0x7f42d80fb120 msgr2=0x7f42d8103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:13:26.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.838+0000 7f42df7d9700 1 -- 172.21.15.71:0/2349496720 shutdown_connections 2024-04-23T02:13:26.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.838+0000 7f42df7d9700 1 -- 172.21.15.71:0/2349496720 wait complete. 2024-04-23T02:13:26.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.971+0000 7ff7c6d4b700 1 -- 172.21.15.71:0/1895906265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c01000c0 msgr2=0x7ff7c01004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:26.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.971+0000 7ff7c6d4b700 1 --2- 172.21.15.71:0/1895906265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c01000c0 0x7ff7c01004e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009910 tx=0x7ff7b4009c20 comp rx=0 tx=0).stop 2024-04-23T02:13:26.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.971+0000 7ff7c6d4b700 1 -- 172.21.15.71:0/1895906265 shutdown_connections 2024-04-23T02:13:26.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.971+0000 7ff7c6d4b700 1 --2- 172.21.15.71:0/1895906265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7c0100de0 0x7ff7c0101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:26.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.972+0000 7ff7c6d4b700 1 --2- 172.21.15.71:0/1895906265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c01000c0 0x7ff7c01004e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:26.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.972+0000 7ff7c6d4b700 1 -- 172.21.15.71:0/1895906265 >> 172.21.15.71:0/1895906265 conn(0x7ff7c00fa8c0 msgr2=0x7ff7c00fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:13:26.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.972+0000 7ff7c6d4b700 1 -- 172.21.15.71:0/1895906265 shutdown_connections 2024-04-23T02:13:26.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.972+0000 7ff7c6d4b700 1 -- 172.21.15.71:0/1895906265 wait complete. 2024-04-23T02:13:26.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.972+0000 7ff7c6d4b700 1 Processor -- start 2024-04-23T02:13:26.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.972+0000 7ff7c6d4b700 1 -- start start 2024-04-23T02:13:26.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.973+0000 7ff7c6d4b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c01000c0 0x7ff7c0196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:26.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.973+0000 7ff7c6d4b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7c0100de0 0x7ff7c0196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:26.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.973+0000 7ff7c6d4b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c0196f70 con 0x7ff7c01000c0 2024-04-23T02:13:26.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.973+0000 7ff7c6d4b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c01970b0 con 0x7ff7c0100de0 2024-04-23T02:13:26.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.973+0000 7ff7c4ae7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c01000c0 0x7ff7c0196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:26.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.973+0000 7ff7c4ae7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c01000c0 0x7ff7c0196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60666/0 (socket says 172.21.15.71:60666) 2024-04-23T02:13:26.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.973+0000 7ff7c4ae7700 1 -- 172.21.15.71:0/1992155714 learned_addr learned my addr 172.21.15.71:0/1992155714 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:13:26.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.973+0000 7ff7c4ae7700 1 -- 172.21.15.71:0/1992155714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7c0100de0 msgr2=0x7ff7c0196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:26.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.973+0000 7ff7bffff700 1 --2- 172.21.15.71:0/1992155714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7c0100de0 0x7ff7c0196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:26.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.973+0000 7ff7c4ae7700 1 --2- 172.21.15.71:0/1992155714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7c0100de0 0x7ff7c0196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:26.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.973+0000 7ff7c4ae7700 1 -- 172.21.15.71:0/1992155714 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b40095f0 con 0x7ff7c01000c0 2024-04-23T02:13:26.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.973+0000 7ff7bffff700 1 --2- 172.21.15.71:0/1992155714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7c0100de0 0x7ff7c0196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:13:26.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.973+0000 7ff7c4ae7700 1 --2- 172.21.15.71:0/1992155714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c01000c0 0x7ff7c0196410 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4000c00 tx=0x7ff7b400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:13:26.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.973+0000 7ff7bdffb700 1 -- 172.21.15.71:0/1992155714 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7b401d070 con 0x7ff7c01000c0 2024-04-23T02:13:26.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.974+0000 7ff7bdffb700 1 -- 172.21.15.71:0/1992155714 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ff7b4005710 con 0x7ff7c01000c0 2024-04-23T02:13:26.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.974+0000 7ff7bdffb700 1 -- 172.21.15.71:0/1992155714 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7b4003c20 con 0x7ff7c01000c0 2024-04-23T02:13:26.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.974+0000 7ff7c6d4b700 1 -- 172.21.15.71:0/1992155714 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c0102000 con 0x7ff7c01000c0 2024-04-23T02:13:26.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.974+0000 7ff7c6d4b700 1 -- 172.21.15.71:0/1992155714 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c01023c0 con 0x7ff7c01000c0 2024-04-23T02:13:26.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.976+0000 7ff7ab7fe700 1 -- 172.21.15.71:0/1992155714 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c0103e90 con 0x7ff7c01000c0 2024-04-23T02:13:26.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.976+0000 7ff7bdffb700 1 -- 172.21.15.71:0/1992155714 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) v1 ==== 90681+0+0 (secure 0 0 0) 0x7ff7b40048e0 con 0x7ff7c01000c0 2024-04-23T02:13:26.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.976+0000 7ff7bdffb700 1 --2- 172.21.15.71:0/1992155714 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7ff7b006ca10 0x7ff7b006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:26.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.976+0000 7ff7bdffb700 1 -- 172.21.15.71:0/1992155714 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7ff7b408d7d0 con 0x7ff7c01000c0 2024-04-23T02:13:26.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.977+0000 7ff7bffff700 1 --2- 172.21.15.71:0/1992155714 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7ff7b006ca10 0x7ff7b006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:26.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.977+0000 7ff7bffff700 1 --2- 172.21.15.71:0/1992155714 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7ff7b006ca10 0x7ff7b006eed0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00ffdf0 tx=0x7ff7ac009260 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:13:26.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:26.982+0000 7ff7bdffb700 1 -- 172.21.15.71:0/1992155714 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7ff7b405b330 con 0x7ff7c01000c0 2024-04-23T02:13:27.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.352+0000 7ff7ab7fe700 1 -- 172.21.15.71:0/1992155714 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff7c00619a0 con 0x7ff7c01000c0 2024-04-23T02:13:27.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.352+0000 7ff7bdffb700 1 -- 172.21.15.71:0/1992155714 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+770 (secure 0 0 0) 0x7ff7b4057030 con 0x7ff7c01000c0 2024-04-23T02:13:27.356 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:13:27.356 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:13:27.356 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-23T02:13:27.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:13:27.357 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:13:27.357 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:13:27.357 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1 2024-04-23T02:13:27.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:13:27.357 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:13:27.357 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-23T02:13:27.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:13:27.357 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:13:27.357 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-23T02:13:27.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:13:27.357 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:13:27.357 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-04-23T02:13:27.357 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1 2024-04-23T02:13:27.358 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:13:27.358 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:13:27.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.355+0000 7ff7ab7fe700 1 -- 172.21.15.71:0/1992155714 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7ff7b006ca10 msgr2=0x7ff7b006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:27.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.355+0000 7ff7ab7fe700 1 --2- 172.21.15.71:0/1992155714 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7ff7b006ca10 0x7ff7b006eed0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00ffdf0 tx=0x7ff7ac009260 comp rx=0 tx=0).stop 2024-04-23T02:13:27.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.355+0000 7ff7ab7fe700 1 -- 172.21.15.71:0/1992155714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c01000c0 msgr2=0x7ff7c0196410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:27.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.355+0000 7ff7ab7fe700 1 --2- 172.21.15.71:0/1992155714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c01000c0 0x7ff7c0196410 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4000c00 tx=0x7ff7b400be90 comp rx=0 tx=0).stop 2024-04-23T02:13:27.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.355+0000 7ff7ab7fe700 1 -- 172.21.15.71:0/1992155714 shutdown_connections 2024-04-23T02:13:27.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.355+0000 7ff7ab7fe700 1 --2- 172.21.15.71:0/1992155714 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7ff7b006ca10 0x7ff7b006eed0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:27.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.355+0000 7ff7ab7fe700 1 --2- 172.21.15.71:0/1992155714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c01000c0 0x7ff7c0196410 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:27.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.355+0000 7ff7ab7fe700 1 --2- 172.21.15.71:0/1992155714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7c0100de0 0x7ff7c0196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:27.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.356+0000 7ff7ab7fe700 1 -- 172.21.15.71:0/1992155714 >> 172.21.15.71:0/1992155714 conn(0x7ff7c00fa8c0 msgr2=0x7ff7c00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:13:27.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.356+0000 7ff7ab7fe700 1 -- 172.21.15.71:0/1992155714 shutdown_connections 2024-04-23T02:13:27.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.356+0000 7ff7ab7fe700 1 -- 172.21.15.71:0/1992155714 wait complete. 2024-04-23T02:13:27.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.494+0000 7f141078f700 1 -- 172.21.15.71:0/3350880184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1408100ca0 msgr2=0x7f14081010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:27.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.494+0000 7f141078f700 1 --2- 172.21.15.71:0/3350880184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1408100ca0 0x7f14081010c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1404009910 tx=0x7f1404009c20 comp rx=0 tx=0).stop 2024-04-23T02:13:27.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.495+0000 7f141078f700 1 -- 172.21.15.71:0/3350880184 shutdown_connections 2024-04-23T02:13:27.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.495+0000 7f141078f700 1 --2- 172.21.15.71:0/3350880184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1408101ea0 0x7f1408102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:27.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.495+0000 7f141078f700 1 --2- 172.21.15.71:0/3350880184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1408100ca0 0x7f14081010c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:27.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.495+0000 7f141078f700 1 -- 172.21.15.71:0/3350880184 >> 172.21.15.71:0/3350880184 conn(0x7f14080fc260 msgr2=0x7f14080fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:13:27.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.495+0000 7f141078f700 1 -- 172.21.15.71:0/3350880184 shutdown_connections 2024-04-23T02:13:27.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.495+0000 7f141078f700 1 -- 172.21.15.71:0/3350880184 wait complete. 2024-04-23T02:13:27.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.495+0000 7f141078f700 1 Processor -- start 2024-04-23T02:13:27.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.495+0000 7f141078f700 1 -- start start 2024-04-23T02:13:27.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.495+0000 7f141078f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1408100ca0 0x7f1408198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:27.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.495+0000 7f141078f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1408101ea0 0x7f1408198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:27.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.495+0000 7f141078f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1408199180 con 0x7f1408101ea0 2024-04-23T02:13:27.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.495+0000 7f141078f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14081992c0 con 0x7f1408100ca0 2024-04-23T02:13:27.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.496+0000 7f140e52b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1408100ca0 0x7f1408198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:27.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.496+0000 7f140dd2a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1408101ea0 0x7f1408198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:27.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.496+0000 7f140e52b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1408100ca0 0x7f1408198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.71:47104/0 (socket says 172.21.15.71:47104) 2024-04-23T02:13:27.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.496+0000 7f140dd2a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1408101ea0 0x7f1408198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60678/0 (socket says 172.21.15.71:60678) 2024-04-23T02:13:27.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.496+0000 7f140e52b700 1 -- 172.21.15.71:0/3582908269 learned_addr learned my addr 172.21.15.71:0/3582908269 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:13:27.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.496+0000 7f140dd2a700 1 -- 172.21.15.71:0/3582908269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1408100ca0 msgr2=0x7f1408198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:27.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.496+0000 7f140dd2a700 1 --2- 172.21.15.71:0/3582908269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1408100ca0 0x7f1408198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:27.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.496+0000 7f140dd2a700 1 -- 172.21.15.71:0/3582908269 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14040095f0 con 0x7f1408101ea0 2024-04-23T02:13:27.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.496+0000 7f140e52b700 1 --2- 172.21.15.71:0/3582908269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1408100ca0 0x7f1408198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T02:13:27.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.496+0000 7f140dd2a700 1 --2- 172.21.15.71:0/3582908269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1408101ea0 0x7f1408198b60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f13f800e970 tx=0x7f13f800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:13:27.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.496+0000 7f13ff7fe700 1 -- 172.21.15.71:0/3582908269 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13f800c9d0 con 0x7f1408101ea0 2024-04-23T02:13:27.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.496+0000 7f141078f700 1 -- 172.21.15.71:0/3582908269 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140819dd70 con 0x7f1408101ea0 2024-04-23T02:13:27.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.496+0000 7f13ff7fe700 1 -- 172.21.15.71:0/3582908269 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f13f8010430 con 0x7f1408101ea0 2024-04-23T02:13:27.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.496+0000 7f13ff7fe700 1 -- 172.21.15.71:0/3582908269 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13f8018630 con 0x7f1408101ea0 2024-04-23T02:13:27.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.497+0000 7f141078f700 1 -- 172.21.15.71:0/3582908269 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140819e2c0 con 0x7f1408101ea0 2024-04-23T02:13:27.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.498+0000 7f141078f700 1 -- 172.21.15.71:0/3582908269 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1408105f80 con 0x7f1408101ea0 2024-04-23T02:13:27.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.498+0000 7f13ff7fe700 1 -- 172.21.15.71:0/3582908269 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) v1 ==== 90681+0+0 (secure 0 0 0) 0x7f13f80105a0 con 0x7f1408101ea0 2024-04-23T02:13:27.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.499+0000 7f13ff7fe700 1 --2- 172.21.15.71:0/3582908269 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f13f406ca10 0x7f13f406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:27.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.499+0000 7f13ff7fe700 1 -- 172.21.15.71:0/3582908269 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f13f8014070 con 0x7f1408101ea0 2024-04-23T02:13:27.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.499+0000 7f140e52b700 1 --2- 172.21.15.71:0/3582908269 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f13f406ca10 0x7f13f406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:27.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.500+0000 7f140e52b700 1 --2- 172.21.15.71:0/3582908269 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f13f406ca10 0x7f13f406eed0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f140400b600 tx=0x7f1404005fb0 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:13:27.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.506+0000 7f13ff7fe700 1 -- 172.21.15.71:0/3582908269 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f13f80529b0 con 0x7f1408101ea0 2024-04-23T02:13:27.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:27 smithi192 ceph-mon[49662]: pgmap v10: 65 pgs: 65 active+clean; 148 MiB data, 677 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2024-04-23T02:13:27.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:27 smithi192 ceph-mon[49662]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:13:27.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:27 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:13:27.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:27 smithi071 ceph-mon[39837]: pgmap v10: 65 pgs: 65 active+clean; 148 MiB data, 677 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2024-04-23T02:13:27.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:27 smithi071 ceph-mon[39837]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:13:27.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:27 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:13:27.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.820+0000 7f141078f700 1 -- 172.21.15.71:0/3582908269 --> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f14080611d0 con 0x7f13f406ca10 2024-04-23T02:13:27.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.821+0000 7f13ff7fe700 1 -- 172.21.15.71:0/3582908269 <== mgr.24351 v2:172.21.15.192:6828/966231696 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f14080611d0 con 0x7f13f406ca10 2024-04-23T02:13:27.825 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:13:27.825 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:13:27.825 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:13:27.825 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-23T02:13:27.825 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [], 2024-04-23T02:13:27.825 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "1/2 daemons upgraded", 2024-04-23T02:13:27.826 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "", 2024-04-23T02:13:27.826 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:13:27.826 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:13:27.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.824+0000 7f141078f700 1 -- 172.21.15.71:0/3582908269 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f13f406ca10 msgr2=0x7f13f406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:27.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.824+0000 7f141078f700 1 --2- 172.21.15.71:0/3582908269 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f13f406ca10 0x7f13f406eed0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f140400b600 tx=0x7f1404005fb0 comp rx=0 tx=0).stop 2024-04-23T02:13:27.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.824+0000 7f141078f700 1 -- 172.21.15.71:0/3582908269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1408101ea0 msgr2=0x7f1408198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:27.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.824+0000 7f141078f700 1 --2- 172.21.15.71:0/3582908269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1408101ea0 0x7f1408198b60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f13f800e970 tx=0x7f13f800ec80 comp rx=0 tx=0).stop 2024-04-23T02:13:27.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.824+0000 7f141078f700 1 -- 172.21.15.71:0/3582908269 shutdown_connections 2024-04-23T02:13:27.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.824+0000 7f141078f700 1 --2- 172.21.15.71:0/3582908269 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f13f406ca10 0x7f13f406eed0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:27.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.824+0000 7f141078f700 1 --2- 172.21.15.71:0/3582908269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1408100ca0 0x7f1408198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:27.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.824+0000 7f141078f700 1 --2- 172.21.15.71:0/3582908269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1408101ea0 0x7f1408198b60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:27.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.824+0000 7f141078f700 1 -- 172.21.15.71:0/3582908269 >> 172.21.15.71:0/3582908269 conn(0x7f14080fc260 msgr2=0x7f14081030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:13:27.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.824+0000 7f141078f700 1 -- 172.21.15.71:0/3582908269 shutdown_connections 2024-04-23T02:13:27.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:27.824+0000 7f141078f700 1 -- 172.21.15.71:0/3582908269 wait complete. 2024-04-23T02:13:28.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:28 smithi192 ceph-mon[49662]: from='client.24381 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:13:28.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:28 smithi192 ceph-mon[49662]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:13:28.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:28 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1992155714' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:13:28.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:28 smithi071 ceph-mon[39837]: from='client.24381 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:13:28.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:28 smithi071 ceph-mon[39837]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:13:28.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:28 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1992155714' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:13:29.632 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:29 smithi192 ceph-mon[49662]: pgmap v11: 65 pgs: 65 active+clean; 153 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 265 op/s 2024-04-23T02:13:29.632 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:29 smithi192 ceph-mon[49662]: from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:13:29.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:29 smithi071 ceph-mon[39837]: pgmap v11: 65 pgs: 65 active+clean; 153 MiB data, 694 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 265 op/s 2024-04-23T02:13:29.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:29 smithi071 ceph-mon[39837]: from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:13:31.529 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:31 smithi071 ceph-mon[39837]: pgmap v12: 65 pgs: 65 active+clean; 172 MiB data, 765 MiB used, 536 GiB / 536 GiB avail; 3.1 MiB/s rd, 5.2 MiB/s wr, 402 op/s 2024-04-23T02:13:31.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:31 smithi192 ceph-mon[49662]: pgmap v12: 65 pgs: 65 active+clean; 172 MiB data, 765 MiB used, 536 GiB / 536 GiB avail; 3.1 MiB/s rd, 5.2 MiB/s wr, 402 op/s 2024-04-23T02:13:33.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:33 smithi192 ceph-mon[49662]: pgmap v13: 65 pgs: 65 active+clean; 172 MiB data, 765 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 4.4 MiB/s wr, 319 op/s 2024-04-23T02:13:33.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:33 smithi071 ceph-mon[39837]: pgmap v13: 65 pgs: 65 active+clean; 172 MiB data, 765 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 4.4 MiB/s wr, 319 op/s 2024-04-23T02:13:34.909 INFO:tasks.workunit.client.1.smithi192.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-23T02:13:34.927 INFO:tasks.workunit.client.1.smithi192.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-23T02:13:34.927 INFO:tasks.workunit.client.1.smithi192.stderr:+ make 2024-04-23T02:13:35.486 INFO:tasks.workunit.client.1.smithi192.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-23T02:13:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:35 smithi192 ceph-mon[49662]: pgmap v14: 65 pgs: 65 active+clean; 201 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 3.7 MiB/s rd, 7.0 MiB/s wr, 554 op/s 2024-04-23T02:13:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:35 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:35 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:35 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:35 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:35 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:35 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:35 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:35 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:35 smithi192 ceph-mon[49662]: Adjusting osd_memory_target on smithi192 to 2251M 2024-04-23T02:13:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:35 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:35 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:13:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:35 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:13:35.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:35 smithi192 ceph-mon[49662]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-23T02:13:35.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:35 smithi192 ceph-mon[49662]: Updating smithi192:/etc/ceph/ceph.conf 2024-04-23T02:13:35.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:35 smithi071 ceph-mon[39837]: pgmap v14: 65 pgs: 65 active+clean; 201 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 3.7 MiB/s rd, 7.0 MiB/s wr, 554 op/s 2024-04-23T02:13:35.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:35 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:35.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:35 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:35.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:35 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:35.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:35 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:35.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:35 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:35.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:35 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:35.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:35 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:35.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:35 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T02:13:35.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:35 smithi071 ceph-mon[39837]: Adjusting osd_memory_target on smithi192 to 2251M 2024-04-23T02:13:35.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:35 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:35.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:35 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:13:35.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:35 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:13:35.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:35 smithi071 ceph-mon[39837]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-23T02:13:35.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:35 smithi071 ceph-mon[39837]: Updating smithi192:/etc/ceph/ceph.conf 2024-04-23T02:13:36.591 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:36 smithi071 ceph-mon[39837]: Updating smithi071:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T02:13:36.592 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:36 smithi071 ceph-mon[39837]: Updating smithi192:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T02:13:36.592 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:36 smithi071 ceph-mon[39837]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-04-23T02:13:36.592 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:36 smithi071 ceph-mon[39837]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-23T02:13:36.592 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:36 smithi071 ceph-mon[39837]: Updating smithi071:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T02:13:36.592 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:36 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:36.592 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:36 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:36.592 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:36 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:36.592 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:36 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:36.592 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:36 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:36.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:36 smithi192 ceph-mon[49662]: Updating smithi071:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T02:13:36.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:36 smithi192 ceph-mon[49662]: Updating smithi192:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T02:13:36.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:36 smithi192 ceph-mon[49662]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-04-23T02:13:36.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:36 smithi192 ceph-mon[49662]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-23T02:13:36.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:36 smithi192 ceph-mon[49662]: Updating smithi071:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T02:13:36.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:36 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:36.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:36 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:36.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:36 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:36.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:36 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:36.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:36 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:13:36.691 INFO:tasks.workunit.client.0.smithi071.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-23T02:13:36.696 INFO:tasks.workunit.client.0.smithi071.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-23T02:13:36.697 INFO:tasks.workunit.client.0.smithi071.stderr:+ make 2024-04-23T02:13:36.912 INFO:tasks.workunit.client.1.smithi192.stderr:++ readlink -f fsstress 2024-04-23T02:13:36.913 INFO:tasks.workunit.client.1.smithi192.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-23T02:13:36.913 INFO:tasks.workunit.client.1.smithi192.stderr:+ popd 2024-04-23T02:13:36.914 INFO:tasks.workunit.client.1.smithi192.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-23T02:13:36.915 INFO:tasks.workunit.client.1.smithi192.stderr:+ popd 2024-04-23T02:13:36.915 INFO:tasks.workunit.client.1.smithi192.stdout:~/cephtest/mnt.1/client.1/tmp 2024-04-23T02:13:36.915 INFO:tasks.workunit.client.1.smithi192.stderr:++ mktemp -d -p . 2024-04-23T02:13:36.920 INFO:tasks.workunit.client.1.smithi192.stderr:+ T=./tmp.W1wvVtvA74 2024-04-23T02:13:36.920 INFO:tasks.workunit.client.1.smithi192.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.W1wvVtvA74 -l 1 -n 1000 -p 10 -v 2024-04-23T02:13:36.922 INFO:tasks.workunit.client.1.smithi192.stdout:seed = 1713967922 2024-04-23T02:13:36.926 INFO:tasks.workunit.client.1.smithi192.stdout:0/0: dwrite - no filename 2024-04-23T02:13:36.926 INFO:tasks.workunit.client.1.smithi192.stdout:0/1: chown . 110879 1 2024-04-23T02:13:36.926 INFO:tasks.workunit.client.1.smithi192.stdout:0/2: write - no filename 2024-04-23T02:13:36.926 INFO:tasks.workunit.client.1.smithi192.stdout:0/3: rename - no filename 2024-04-23T02:13:36.928 INFO:tasks.workunit.client.1.smithi192.stdout:0/4: creat f0 x:0 0 0 2024-04-23T02:13:36.928 INFO:tasks.workunit.client.1.smithi192.stdout:0/5: truncate f0 606969 0 2024-04-23T02:13:36.929 INFO:tasks.workunit.client.1.smithi192.stdout:1/0: creat f0 x:0 0 0 2024-04-23T02:13:36.931 INFO:tasks.workunit.client.1.smithi192.stdout:3/0: dwrite - no filename 2024-04-23T02:13:36.931 INFO:tasks.workunit.client.1.smithi192.stdout:3/1: write - no filename 2024-04-23T02:13:36.931 INFO:tasks.workunit.client.1.smithi192.stdout:3/2: dread - no filename 2024-04-23T02:13:36.931 INFO:tasks.workunit.client.1.smithi192.stdout:3/3: dread - no filename 2024-04-23T02:13:36.933 INFO:tasks.workunit.client.1.smithi192.stdout:4/0: dread - no filename 2024-04-23T02:13:36.933 INFO:tasks.workunit.client.1.smithi192.stdout:4/1: dwrite - no filename 2024-04-23T02:13:36.933 INFO:tasks.workunit.client.1.smithi192.stdout:4/2: fdatasync - no filename 2024-04-23T02:13:36.934 INFO:tasks.workunit.client.1.smithi192.stdout:4/3: dread - no filename 2024-04-23T02:13:36.934 INFO:tasks.workunit.client.1.smithi192.stdout:4/4: write - no filename 2024-04-23T02:13:36.934 INFO:tasks.workunit.client.1.smithi192.stdout:4/5: readlink - no filename 2024-04-23T02:13:36.934 INFO:tasks.workunit.client.1.smithi192.stdout:4/6: rename - no filename 2024-04-23T02:13:36.934 INFO:tasks.workunit.client.1.smithi192.stdout:4/7: rmdir - no directory 2024-04-23T02:13:36.934 INFO:tasks.workunit.client.1.smithi192.stdout:4/8: link - no file 2024-04-23T02:13:36.934 INFO:tasks.workunit.client.1.smithi192.stdout:4/9: truncate - no filename 2024-04-23T02:13:36.934 INFO:tasks.workunit.client.1.smithi192.stdout:3/4: symlink l0 0 2024-04-23T02:13:36.935 INFO:tasks.workunit.client.1.smithi192.stdout:5/0: rename - no filename 2024-04-23T02:13:36.935 INFO:tasks.workunit.client.1.smithi192.stdout:4/10: mknod c0 0 2024-04-23T02:13:36.937 INFO:tasks.workunit.client.1.smithi192.stdout:6/0: dread - no filename 2024-04-23T02:13:36.938 INFO:tasks.workunit.client.1.smithi192.stdout:6/1: chown . 11 1 2024-04-23T02:13:36.938 INFO:tasks.workunit.client.1.smithi192.stdout:6/2: dread - no filename 2024-04-23T02:13:36.938 INFO:tasks.workunit.client.1.smithi192.stdout:0/6: dwrite f0 [0,4194304] 0 2024-04-23T02:13:36.938 INFO:tasks.workunit.client.1.smithi192.stdout:6/3: chown . 0 1 2024-04-23T02:13:36.939 INFO:tasks.workunit.client.1.smithi192.stdout:0/7: write f0 [25603,97971] 0 2024-04-23T02:13:36.939 INFO:tasks.workunit.client.1.smithi192.stdout:4/11: unlink c0 0 2024-04-23T02:13:36.940 INFO:tasks.workunit.client.1.smithi192.stdout:7/0: truncate - no filename 2024-04-23T02:13:36.940 INFO:tasks.workunit.client.1.smithi192.stdout:7/1: write - no filename 2024-04-23T02:13:36.940 INFO:tasks.workunit.client.1.smithi192.stdout:7/2: unlink - no file 2024-04-23T02:13:36.940 INFO:tasks.workunit.client.1.smithi192.stdout:7/3: dwrite - no filename 2024-04-23T02:13:36.940 INFO:tasks.workunit.client.1.smithi192.stdout:7/4: write - no filename 2024-04-23T02:13:36.941 INFO:tasks.workunit.client.1.smithi192.stdout:5/1: creat f0 x:0 0 0 2024-04-23T02:13:36.942 INFO:tasks.workunit.client.1.smithi192.stdout:0/8: symlink l1 0 2024-04-23T02:13:36.943 INFO:tasks.workunit.client.1.smithi192.stdout:4/12: creat f1 x:0 0 0 2024-04-23T02:13:36.944 INFO:tasks.workunit.client.1.smithi192.stdout:6/4: creat f0 x:0 0 0 2024-04-23T02:13:36.944 INFO:tasks.workunit.client.1.smithi192.stdout:4/13: dread - f1 zero size 2024-04-23T02:13:36.945 INFO:tasks.workunit.client.1.smithi192.stdout:6/5: read - f0 zero size 2024-04-23T02:13:36.945 INFO:tasks.workunit.client.1.smithi192.stdout:6/6: rmdir - no directory 2024-04-23T02:13:36.945 INFO:tasks.workunit.client.1.smithi192.stdout:5/2: symlink l1 0 2024-04-23T02:13:36.946 INFO:tasks.workunit.client.1.smithi192.stdout:6/7: read - f0 zero size 2024-04-23T02:13:36.947 INFO:tasks.workunit.client.1.smithi192.stdout:0/9: rename f0 to f2 0 2024-04-23T02:13:36.948 INFO:tasks.workunit.client.1.smithi192.stdout:8/0: write - no filename 2024-04-23T02:13:36.948 INFO:tasks.workunit.client.1.smithi192.stdout:0/10: readlink l1 0 2024-04-23T02:13:36.948 INFO:tasks.workunit.client.1.smithi192.stdout:7/5: creat f0 x:0 0 0 2024-04-23T02:13:36.949 INFO:tasks.workunit.client.1.smithi192.stdout:4/14: symlink l2 0 2024-04-23T02:13:36.950 INFO:tasks.workunit.client.1.smithi192.stdout:4/15: chown f1 0 1 2024-04-23T02:13:36.951 INFO:tasks.workunit.client.1.smithi192.stdout:4/16: chown f1 143899963 1 2024-04-23T02:13:36.951 INFO:tasks.workunit.client.1.smithi192.stdout:4/17: rmdir - no directory 2024-04-23T02:13:36.951 INFO:tasks.workunit.client.1.smithi192.stdout:5/3: mkdir d2 0 2024-04-23T02:13:36.952 INFO:tasks.workunit.client.1.smithi192.stdout:9/0: dread - no filename 2024-04-23T02:13:36.954 INFO:tasks.workunit.client.1.smithi192.stdout:6/8: rename f0 to f1 0 2024-04-23T02:13:36.955 INFO:tasks.workunit.client.1.smithi192.stdout:4/18: mkdir d3 0 2024-04-23T02:13:36.956 INFO:tasks.workunit.client.1.smithi192.stdout:4/19: truncate f1 407602 0 2024-04-23T02:13:36.958 INFO:tasks.workunit.client.1.smithi192.stdout:8/1: creat f0 x:0 0 0 2024-04-23T02:13:36.959 INFO:tasks.workunit.client.1.smithi192.stdout:6/9: symlink l2 0 2024-04-23T02:13:36.960 INFO:tasks.workunit.client.1.smithi192.stdout:6/10: readlink l2 0 2024-04-23T02:13:36.960 INFO:tasks.workunit.client.1.smithi192.stdout:5/4: link f0 d2/f3 0 2024-04-23T02:13:36.961 INFO:tasks.workunit.client.1.smithi192.stdout:5/5: read - d2/f3 zero size 2024-04-23T02:13:36.962 INFO:tasks.workunit.client.1.smithi192.stdout:9/1: mkdir d0 0 2024-04-23T02:13:36.963 INFO:tasks.workunit.client.1.smithi192.stdout:8/2: mkdir d1 0 2024-04-23T02:13:36.964 INFO:tasks.workunit.client.1.smithi192.stdout:8/3: write f0 [909084,64635] 0 2024-04-23T02:13:36.964 INFO:tasks.workunit.client.1.smithi192.stdout:4/20: getdents d3 0 2024-04-23T02:13:36.965 INFO:tasks.workunit.client.1.smithi192.stdout:6/11: write f1 [590819,119890] 0 2024-04-23T02:13:36.965 INFO:tasks.workunit.client.1.smithi192.stdout:4/21: chown l2 34776 1 2024-04-23T02:13:36.969 INFO:tasks.workunit.client.1.smithi192.stdout:4/22: readlink l2 0 2024-04-23T02:13:36.969 INFO:tasks.workunit.client.1.smithi192.stdout:8/4: read f0 [497507,31443] 0 2024-04-23T02:13:36.969 INFO:tasks.workunit.client.1.smithi192.stdout:8/5: write f0 [662617,113827] 0 2024-04-23T02:13:36.970 INFO:tasks.workunit.client.1.smithi192.stdout:4/23: dread f1 [0,4194304] 0 2024-04-23T02:13:36.972 INFO:tasks.workunit.client.1.smithi192.stdout:8/6: read f0 [827698,48494] 0 2024-04-23T02:13:36.976 INFO:tasks.workunit.client.1.smithi192.stdout:4/24: rmdir d3 0 2024-04-23T02:13:36.977 INFO:tasks.workunit.client.1.smithi192.stdout:6/12: dwrite f1 [0,4194304] 0 2024-04-23T02:13:36.977 INFO:tasks.workunit.client.1.smithi192.stdout:8/7: dread f0 [0,4194304] 0 2024-04-23T02:13:36.978 INFO:tasks.workunit.client.1.smithi192.stdout:4/25: creat f4 x:0 0 0 2024-04-23T02:13:36.979 INFO:tasks.workunit.client.1.smithi192.stdout:8/8: fdatasync f0 0 2024-04-23T02:13:36.980 INFO:tasks.workunit.client.1.smithi192.stdout:8/9: fdatasync f0 0 2024-04-23T02:13:36.982 INFO:tasks.workunit.client.1.smithi192.stdout:6/13: mkdir d3 0 2024-04-23T02:13:36.986 INFO:tasks.workunit.client.1.smithi192.stdout:6/14: mknod d3/c4 0 2024-04-23T02:13:36.988 INFO:tasks.workunit.client.1.smithi192.stdout:4/26: dwrite f1 [0,4194304] 0 2024-04-23T02:13:36.993 INFO:tasks.workunit.client.1.smithi192.stdout:6/15: mknod d3/c5 0 2024-04-23T02:13:36.999 INFO:tasks.workunit.client.1.smithi192.stdout:6/16: creat d3/f6 x:0 0 0 2024-04-23T02:13:37.001 INFO:tasks.workunit.client.1.smithi192.stdout:6/17: mknod d3/c7 0 2024-04-23T02:13:37.002 INFO:tasks.workunit.client.1.smithi192.stdout:1/1: fsync f0 0 2024-04-23T02:13:37.003 INFO:tasks.workunit.client.1.smithi192.stdout:6/18: truncate d3/f6 597833 0 2024-04-23T02:13:37.003 INFO:tasks.workunit.client.1.smithi192.stdout:4/27: dwrite f1 [0,4194304] 0 2024-04-23T02:13:37.004 INFO:tasks.workunit.client.1.smithi192.stdout:4/28: dread - f4 zero size 2024-04-23T02:13:37.004 INFO:tasks.workunit.client.1.smithi192.stdout:1/2: write f0 [916765,97236] 0 2024-04-23T02:13:37.007 INFO:tasks.workunit.client.1.smithi192.stdout:1/3: write f0 [1706500,54118] 0 2024-04-23T02:13:37.007 INFO:tasks.workunit.client.1.smithi192.stdout:4/29: write f4 [406479,62463] 0 2024-04-23T02:13:37.012 INFO:tasks.workunit.client.1.smithi192.stdout:4/30: symlink l5 0 2024-04-23T02:13:37.013 INFO:tasks.workunit.client.1.smithi192.stdout:1/4: creat f1 x:0 0 0 2024-04-23T02:13:37.014 INFO:tasks.workunit.client.1.smithi192.stdout:4/31: creat f6 x:0 0 0 2024-04-23T02:13:37.016 INFO:tasks.workunit.client.1.smithi192.stdout:1/5: write f1 [494896,119427] 0 2024-04-23T02:13:37.018 INFO:tasks.workunit.client.1.smithi192.stdout:6/19: dwrite d3/f6 [0,4194304] 0 2024-04-23T02:13:37.018 INFO:tasks.workunit.client.1.smithi192.stdout:7/6: fsync f0 0 2024-04-23T02:13:37.019 INFO:tasks.workunit.client.1.smithi192.stdout:6/20: readlink l2 0 2024-04-23T02:13:37.020 INFO:tasks.workunit.client.1.smithi192.stdout:1/6: symlink l2 0 2024-04-23T02:13:37.021 INFO:tasks.workunit.client.1.smithi192.stdout:4/32: link l5 l7 0 2024-04-23T02:13:37.022 INFO:tasks.workunit.client.1.smithi192.stdout:4/33: truncate f6 19509 0 2024-04-23T02:13:37.026 INFO:tasks.workunit.client.1.smithi192.stdout:4/34: mknod c8 0 2024-04-23T02:13:37.029 INFO:tasks.workunit.client.1.smithi192.stdout:4/35: readlink l5 0 2024-04-23T02:13:37.030 INFO:tasks.workunit.client.1.smithi192.stdout:4/36: fsync f1 0 2024-04-23T02:13:37.030 INFO:tasks.workunit.client.1.smithi192.stdout:7/7: getdents . 0 2024-04-23T02:13:37.031 INFO:tasks.workunit.client.1.smithi192.stdout:4/37: write f1 [3168834,48784] 0 2024-04-23T02:13:37.032 INFO:tasks.workunit.client.1.smithi192.stdout:5/6: truncate f0 410720 0 2024-04-23T02:13:37.033 INFO:tasks.workunit.client.1.smithi192.stdout:4/38: unlink f6 0 2024-04-23T02:13:37.037 INFO:tasks.workunit.client.1.smithi192.stdout:4/39: readlink l5 0 2024-04-23T02:13:37.039 INFO:tasks.workunit.client.1.smithi192.stdout:4/40: fsync f1 0 2024-04-23T02:13:37.040 INFO:tasks.workunit.client.1.smithi192.stdout:7/8: dwrite f0 [0,4194304] 0 2024-04-23T02:13:37.040 INFO:tasks.workunit.client.1.smithi192.stdout:5/7: creat d2/f4 x:0 0 0 2024-04-23T02:13:37.042 INFO:tasks.workunit.client.1.smithi192.stdout:4/41: readlink l5 0 2024-04-23T02:13:37.043 INFO:tasks.workunit.client.1.smithi192.stdout:7/9: mknod c1 0 2024-04-23T02:13:37.044 INFO:tasks.workunit.client.1.smithi192.stdout:5/8: creat d2/f5 x:0 0 0 2024-04-23T02:13:37.045 INFO:tasks.workunit.client.1.smithi192.stdout:5/9: chown d2/f5 6865246 1 2024-04-23T02:13:37.045 INFO:tasks.workunit.client.1.smithi192.stdout:4/42: creat f9 x:0 0 0 2024-04-23T02:13:37.048 INFO:tasks.workunit.client.1.smithi192.stdout:7/10: unlink f0 0 2024-04-23T02:13:37.049 INFO:tasks.workunit.client.1.smithi192.stdout:5/10: mkdir d2/d6 0 2024-04-23T02:13:37.051 INFO:tasks.workunit.client.1.smithi192.stdout:4/43: dread f1 [0,4194304] 0 2024-04-23T02:13:37.052 INFO:tasks.workunit.client.1.smithi192.stdout:7/11: symlink l2 0 2024-04-23T02:13:37.054 INFO:tasks.workunit.client.1.smithi192.stdout:4/44: mknod ca 0 2024-04-23T02:13:37.057 INFO:tasks.workunit.client.1.smithi192.stdout:7/12: rename c1 to c3 0 2024-04-23T02:13:37.057 INFO:tasks.workunit.client.1.smithi192.stdout:5/11: stat f0 0 2024-04-23T02:13:37.059 INFO:tasks.workunit.client.1.smithi192.stdout:4/45: symlink lb 0 2024-04-23T02:13:37.061 INFO:tasks.workunit.client.1.smithi192.stdout:4/46: chown f1 63840 1 2024-04-23T02:13:37.063 INFO:tasks.workunit.client.1.smithi192.stdout:7/13: link l2 l4 0 2024-04-23T02:13:37.065 INFO:tasks.workunit.client.1.smithi192.stdout:1/7: truncate f0 531369 0 2024-04-23T02:13:37.069 INFO:tasks.workunit.client.1.smithi192.stdout:7/14: chown l2 187 1 2024-04-23T02:13:37.069 INFO:tasks.workunit.client.1.smithi192.stdout:7/15: dread - no filename 2024-04-23T02:13:37.070 INFO:tasks.workunit.client.1.smithi192.stdout:4/47: link f9 fc 0 2024-04-23T02:13:37.074 INFO:tasks.workunit.client.1.smithi192.stdout:7/16: readlink l4 0 2024-04-23T02:13:37.074 INFO:tasks.workunit.client.1.smithi192.stdout:7/17: write - no filename 2024-04-23T02:13:37.074 INFO:tasks.workunit.client.1.smithi192.stdout:5/12: dwrite d2/f5 [0,4194304] 0 2024-04-23T02:13:37.075 INFO:tasks.workunit.client.1.smithi192.stdout:4/48: write f4 [1275956,17694] 0 2024-04-23T02:13:37.077 INFO:tasks.workunit.client.1.smithi192.stdout:1/8: dread f1 [0,4194304] 0 2024-04-23T02:13:37.079 INFO:tasks.workunit.client.1.smithi192.stdout:7/18: mkdir d5 0 2024-04-23T02:13:37.079 INFO:tasks.workunit.client.1.smithi192.stdout:7/19: dwrite - no filename 2024-04-23T02:13:37.079 INFO:tasks.workunit.client.1.smithi192.stdout:7/20: fsync - no filename 2024-04-23T02:13:37.079 INFO:tasks.workunit.client.1.smithi192.stdout:6/21: dwrite f1 [4194304,4194304] 0 2024-04-23T02:13:37.080 INFO:tasks.workunit.client.1.smithi192.stdout:4/49: write f1 [1053864,128120] 0 2024-04-23T02:13:37.082 INFO:tasks.workunit.client.0.smithi071.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-23T02:13:37.082 INFO:tasks.workunit.client.1.smithi192.stdout:1/9: chown f1 5482353 1 2024-04-23T02:13:37.083 INFO:tasks.workunit.client.1.smithi192.stdout:4/50: write f4 [351974,101804] 0 2024-04-23T02:13:37.083 INFO:tasks.workunit.client.1.smithi192.stdout:4/51: rmdir - no directory 2024-04-23T02:13:37.088 INFO:tasks.workunit.client.1.smithi192.stdout:6/22: dread f1 [4194304,4194304] 0 2024-04-23T02:13:37.097 INFO:tasks.workunit.client.1.smithi192.stdout:6/23: dwrite d3/f6 [0,4194304] 0 2024-04-23T02:13:37.098 INFO:tasks.workunit.client.1.smithi192.stdout:7/21: mknod d5/c6 0 2024-04-23T02:13:37.101 INFO:tasks.workunit.client.1.smithi192.stdout:7/22: creat d5/f7 x:0 0 0 2024-04-23T02:13:37.102 INFO:tasks.workunit.client.1.smithi192.stdout:6/24: symlink d3/l8 0 2024-04-23T02:13:37.103 INFO:tasks.workunit.client.1.smithi192.stdout:6/25: chown d3/c7 15 1 2024-04-23T02:13:37.105 INFO:tasks.workunit.client.1.smithi192.stdout:7/23: rename d5/f7 to d5/f8 0 2024-04-23T02:13:37.108 INFO:tasks.workunit.client.1.smithi192.stdout:7/24: symlink d5/l9 0 2024-04-23T02:13:37.109 INFO:tasks.workunit.client.1.smithi192.stdout:4/52: write f9 [845552,89042] 0 2024-04-23T02:13:37.111 INFO:tasks.workunit.client.1.smithi192.stdout:7/25: read - d5/f8 zero size 2024-04-23T02:13:37.113 INFO:tasks.workunit.client.1.smithi192.stdout:4/53: mknod cd 0 2024-04-23T02:13:37.116 INFO:tasks.workunit.client.1.smithi192.stdout:4/54: mkdir de 0 2024-04-23T02:13:37.117 INFO:tasks.workunit.client.1.smithi192.stdout:7/26: mknod d5/ca 0 2024-04-23T02:13:37.118 INFO:tasks.workunit.client.1.smithi192.stdout:7/27: dread - d5/f8 zero size 2024-04-23T02:13:37.119 INFO:tasks.workunit.client.1.smithi192.stdout:4/55: symlink de/lf 0 2024-04-23T02:13:37.122 INFO:tasks.workunit.client.1.smithi192.stdout:6/26: getdents d3 0 2024-04-23T02:13:37.122 INFO:tasks.workunit.client.1.smithi192.stdout:7/28: creat d5/fb x:0 0 0 2024-04-23T02:13:37.123 INFO:tasks.workunit.client.1.smithi192.stdout:4/56: creat de/f10 x:0 0 0 2024-04-23T02:13:37.124 INFO:tasks.workunit.client.1.smithi192.stdout:7/29: read - d5/fb zero size 2024-04-23T02:13:37.125 INFO:tasks.workunit.client.1.smithi192.stdout:5/13: dwrite d2/f3 [0,4194304] 0 2024-04-23T02:13:37.126 INFO:tasks.workunit.client.1.smithi192.stdout:4/57: write f9 [180007,33681] 0 2024-04-23T02:13:37.126 INFO:tasks.workunit.client.1.smithi192.stdout:6/27: write d3/f6 [117902,29154] 0 2024-04-23T02:13:37.127 INFO:tasks.workunit.client.1.smithi192.stdout:6/28: stat l2 0 2024-04-23T02:13:37.128 INFO:tasks.workunit.client.1.smithi192.stdout:4/58: fdatasync de/f10 0 2024-04-23T02:13:37.129 INFO:tasks.workunit.client.1.smithi192.stdout:6/29: write d3/f6 [3211859,64494] 0 2024-04-23T02:13:37.129 INFO:tasks.workunit.client.1.smithi192.stdout:4/59: rename de to de/d11 22 2024-04-23T02:13:37.131 INFO:tasks.workunit.client.1.smithi192.stdout:5/14: write d2/f5 [2878436,51875] 0 2024-04-23T02:13:37.132 INFO:tasks.workunit.client.1.smithi192.stdout:7/30: truncate d5/fb 791017 0 2024-04-23T02:13:37.134 INFO:tasks.workunit.client.1.smithi192.stdout:1/10: write f0 [1148057,41838] 0 2024-04-23T02:13:37.134 INFO:tasks.workunit.client.1.smithi192.stdout:4/60: mknod de/c12 0 2024-04-23T02:13:37.136 INFO:tasks.workunit.client.1.smithi192.stdout:4/61: chown cd 16246428 1 2024-04-23T02:13:37.136 INFO:tasks.workunit.client.1.smithi192.stdout:5/15: mknod d2/d6/c7 0 2024-04-23T02:13:37.136 INFO:tasks.workunit.client.1.smithi192.stdout:7/31: write d5/f8 [269405,112610] 0 2024-04-23T02:13:37.137 INFO:tasks.workunit.client.1.smithi192.stdout:4/62: rename de to de/d13 22 2024-04-23T02:13:37.139 INFO:tasks.workunit.client.1.smithi192.stdout:5/16: dread - d2/f4 zero size 2024-04-23T02:13:37.141 INFO:tasks.workunit.client.1.smithi192.stdout:4/63: mknod de/c14 0 2024-04-23T02:13:37.141 INFO:tasks.workunit.client.1.smithi192.stdout:6/30: dwrite d3/f6 [0,4194304] 0 2024-04-23T02:13:37.143 INFO:tasks.workunit.client.1.smithi192.stdout:5/17: creat d2/d6/f8 x:0 0 0 2024-04-23T02:13:37.143 INFO:tasks.workunit.client.1.smithi192.stdout:4/64: dread - de/f10 zero size 2024-04-23T02:13:37.144 INFO:tasks.workunit.client.1.smithi192.stdout:6/31: write d3/f6 [3144423,50593] 0 2024-04-23T02:13:37.146 INFO:tasks.workunit.client.1.smithi192.stdout:4/65: mkdir de/d15 0 2024-04-23T02:13:37.149 INFO:tasks.workunit.client.1.smithi192.stdout:5/18: link l1 d2/l9 0 2024-04-23T02:13:37.149 INFO:tasks.workunit.client.1.smithi192.stdout:7/32: dwrite d5/fb [0,4194304] 0 2024-04-23T02:13:37.152 INFO:tasks.workunit.client.1.smithi192.stdout:4/66: creat de/d15/f16 x:0 0 0 2024-04-23T02:13:37.156 INFO:tasks.workunit.client.1.smithi192.stdout:7/33: rmdir d5 39 2024-04-23T02:13:37.157 INFO:tasks.workunit.client.1.smithi192.stdout:4/67: symlink de/l17 0 2024-04-23T02:13:37.159 INFO:tasks.workunit.client.1.smithi192.stdout:4/68: fdatasync f9 0 2024-04-23T02:13:37.159 INFO:tasks.workunit.client.1.smithi192.stdout:7/34: stat d5 0 2024-04-23T02:13:37.160 INFO:tasks.workunit.client.1.smithi192.stdout:4/69: readlink de/lf 0 2024-04-23T02:13:37.160 INFO:tasks.workunit.client.1.smithi192.stdout:5/19: dwrite d2/f3 [0,4194304] 0 2024-04-23T02:13:37.161 INFO:tasks.workunit.client.1.smithi192.stdout:5/20: chown d2/d6 221306796 1 2024-04-23T02:13:37.164 INFO:tasks.workunit.client.1.smithi192.stdout:4/70: unlink f1 0 2024-04-23T02:13:37.164 INFO:tasks.workunit.client.1.smithi192.stdout:7/35: dread d5/f8 [0,4194304] 0 2024-04-23T02:13:37.169 INFO:tasks.workunit.client.1.smithi192.stdout:7/36: symlink d5/lc 0 2024-04-23T02:13:37.173 INFO:tasks.workunit.client.1.smithi192.stdout:5/21: dwrite d2/f3 [0,4194304] 0 2024-04-23T02:13:37.177 INFO:tasks.workunit.client.1.smithi192.stdout:5/22: write d2/f4 [433895,2302] 0 2024-04-23T02:13:37.179 INFO:tasks.workunit.client.1.smithi192.stdout:5/23: stat d2 0 2024-04-23T02:13:37.179 INFO:tasks.workunit.client.1.smithi192.stdout:4/71: dwrite fc [0,4194304] 0 2024-04-23T02:13:37.182 INFO:tasks.workunit.client.1.smithi192.stdout:5/24: symlink d2/la 0 2024-04-23T02:13:37.183 INFO:tasks.workunit.client.1.smithi192.stdout:5/25: chown d2/d6/f8 8303741 1 2024-04-23T02:13:37.184 INFO:tasks.workunit.client.1.smithi192.stdout:4/72: unlink cd 0 2024-04-23T02:13:37.185 INFO:tasks.workunit.client.1.smithi192.stdout:7/37: dwrite d5/fb [0,4194304] 0 2024-04-23T02:13:37.188 INFO:tasks.workunit.client.1.smithi192.stdout:5/26: mkdir d2/db 0 2024-04-23T02:13:37.189 INFO:tasks.workunit.client.1.smithi192.stdout:4/73: creat de/f18 x:0 0 0 2024-04-23T02:13:37.189 INFO:tasks.workunit.client.1.smithi192.stdout:7/38: mknod d5/cd 0 2024-04-23T02:13:37.191 INFO:tasks.workunit.client.1.smithi192.stdout:4/74: dread - de/d15/f16 zero size 2024-04-23T02:13:37.194 INFO:tasks.workunit.client.1.smithi192.stdout:4/75: mkdir de/d15/d19 0 2024-04-23T02:13:37.219 INFO:tasks.workunit.client.1.smithi192.stdout:5/27: truncate f0 4136092 0 2024-04-23T02:13:37.219 INFO:tasks.workunit.client.1.smithi192.stdout:7/39: truncate d5/f8 79678 0 2024-04-23T02:13:37.219 INFO:tasks.workunit.client.1.smithi192.stdout:4/76: dwrite f4 [0,4194304] 0 2024-04-23T02:13:37.220 INFO:tasks.workunit.client.1.smithi192.stdout:4/77: fdatasync f9 0 2024-04-23T02:13:37.227 INFO:tasks.workunit.client.1.smithi192.stdout:7/40: dwrite d5/fb [4194304,4194304] 0 2024-04-23T02:13:37.228 INFO:tasks.workunit.client.1.smithi192.stdout:7/41: chown d5/lc 27208 1 2024-04-23T02:13:37.325 INFO:tasks.workunit.client.1.smithi192.stdout:7/42: symlink d5/le 0 2024-04-23T02:13:37.326 INFO:tasks.workunit.client.1.smithi192.stdout:4/78: mkdir de/d15/d19/d1a 0 2024-04-23T02:13:37.327 INFO:tasks.workunit.client.1.smithi192.stdout:7/43: symlink d5/lf 0 2024-04-23T02:13:37.328 INFO:tasks.workunit.client.1.smithi192.stdout:4/79: creat de/f1b x:0 0 0 2024-04-23T02:13:37.331 INFO:tasks.workunit.client.1.smithi192.stdout:4/80: getdents de/d15/d19/d1a 0 2024-04-23T02:13:37.333 INFO:tasks.workunit.client.1.smithi192.stdout:4/81: rename de/c12 to de/d15/d19/d1a/c1c 0 2024-04-23T02:13:37.334 INFO:tasks.workunit.client.1.smithi192.stdout:4/82: dread - de/f1b zero size 2024-04-23T02:13:37.335 INFO:tasks.workunit.client.1.smithi192.stdout:5/28: read f0 [560979,4392] 0 2024-04-23T02:13:37.340 INFO:tasks.workunit.client.1.smithi192.stdout:5/29: symlink d2/db/lc 0 2024-04-23T02:13:37.341 INFO:tasks.workunit.client.1.smithi192.stdout:4/83: dwrite f9 [0,4194304] 0 2024-04-23T02:13:37.342 INFO:tasks.workunit.client.1.smithi192.stdout:4/84: chown de/d15/f16 425 1 2024-04-23T02:13:37.345 INFO:tasks.workunit.client.1.smithi192.stdout:4/85: creat de/d15/f1d x:0 0 0 2024-04-23T02:13:37.346 INFO:tasks.workunit.client.1.smithi192.stdout:5/30: dread d2/f5 [0,4194304] 0 2024-04-23T02:13:37.347 INFO:tasks.workunit.client.1.smithi192.stdout:5/31: dread - d2/d6/f8 zero size 2024-04-23T02:13:37.348 INFO:tasks.workunit.client.1.smithi192.stdout:5/32: write d2/d6/f8 [1007499,88953] 0 2024-04-23T02:13:37.349 INFO:tasks.workunit.client.1.smithi192.stdout:4/86: creat de/d15/d19/d1a/f1e x:0 0 0 2024-04-23T02:13:37.350 INFO:tasks.workunit.client.1.smithi192.stdout:4/87: chown c8 10053 1 2024-04-23T02:13:37.351 INFO:tasks.workunit.client.1.smithi192.stdout:5/33: rmdir d2/d6 39 2024-04-23T02:13:37.358 INFO:tasks.workunit.client.1.smithi192.stdout:4/88: dwrite de/d15/f1d [0,4194304] 0 2024-04-23T02:13:37.359 INFO:tasks.workunit.client.1.smithi192.stdout:5/34: dread f0 [0,4194304] 0 2024-04-23T02:13:37.364 INFO:tasks.workunit.client.1.smithi192.stdout:5/35: write d2/d6/f8 [964466,7051] 0 2024-04-23T02:13:37.368 INFO:tasks.workunit.client.1.smithi192.stdout:5/36: dread f0 [0,4194304] 0 2024-04-23T02:13:37.371 INFO:tasks.workunit.client.1.smithi192.stdout:5/37: creat d2/fd x:0 0 0 2024-04-23T02:13:37.375 INFO:tasks.workunit.client.1.smithi192.stdout:5/38: rename d2/d6/f8 to d2/fe 0 2024-04-23T02:13:37.377 INFO:tasks.workunit.client.1.smithi192.stdout:5/39: symlink d2/lf 0 2024-04-23T02:13:37.380 INFO:tasks.workunit.client.1.smithi192.stdout:5/40: link d2/f5 d2/f10 0 2024-04-23T02:13:37.381 INFO:tasks.workunit.client.1.smithi192.stdout:5/41: chown d2/f5 1 1 2024-04-23T02:13:37.383 INFO:tasks.workunit.client.1.smithi192.stdout:5/42: write d2/f10 [2387233,32598] 0 2024-04-23T02:13:37.385 INFO:tasks.workunit.client.1.smithi192.stdout:5/43: creat d2/db/f11 x:0 0 0 2024-04-23T02:13:37.386 INFO:tasks.workunit.client.1.smithi192.stdout:5/44: dread - d2/db/f11 zero size 2024-04-23T02:13:37.427 INFO:tasks.workunit.client.1.smithi192.stdout:5/45: rmdir d2/db 39 2024-04-23T02:13:37.429 INFO:tasks.workunit.client.1.smithi192.stdout:7/44: dwrite d5/f8 [0,4194304] 0 2024-04-23T02:13:37.432 INFO:tasks.workunit.client.1.smithi192.stdout:4/89: dwrite de/d15/f1d [4194304,4194304] 0 2024-04-23T02:13:37.435 INFO:tasks.workunit.client.1.smithi192.stdout:4/90: readlink l7 0 2024-04-23T02:13:37.437 INFO:tasks.workunit.client.1.smithi192.stdout:5/46: truncate d2/f10 3399313 0 2024-04-23T02:13:37.438 INFO:tasks.workunit.client.1.smithi192.stdout:5/47: truncate d2/fd 695094 0 2024-04-23T02:13:37.441 INFO:tasks.workunit.client.1.smithi192.stdout:5/48: rmdir d2/db 39 2024-04-23T02:13:37.442 INFO:tasks.workunit.client.1.smithi192.stdout:5/49: write d2/fe [1797848,33721] 0 2024-04-23T02:13:37.447 INFO:tasks.workunit.client.1.smithi192.stdout:7/45: truncate d5/f8 3595364 0 2024-04-23T02:13:37.448 INFO:tasks.workunit.client.1.smithi192.stdout:4/91: truncate f4 3423492 0 2024-04-23T02:13:37.449 INFO:tasks.workunit.client.1.smithi192.stdout:7/46: mknod d5/c10 0 2024-04-23T02:13:37.449 INFO:tasks.workunit.client.1.smithi192.stdout:4/92: dread - de/d15/d19/d1a/f1e zero size 2024-04-23T02:13:37.450 INFO:tasks.workunit.client.1.smithi192.stdout:4/93: chown de/d15 3 1 2024-04-23T02:13:37.452 INFO:tasks.workunit.client.1.smithi192.stdout:4/94: mknod de/d15/c1f 0 2024-04-23T02:13:37.455 INFO:tasks.workunit.client.1.smithi192.stdout:4/95: unlink de/f10 0 2024-04-23T02:13:37.455 INFO:tasks.workunit.client.1.smithi192.stdout:7/47: dread d5/fb [0,4194304] 0 2024-04-23T02:13:37.460 INFO:tasks.workunit.client.1.smithi192.stdout:5/50: dread d2/f5 [0,4194304] 0 2024-04-23T02:13:37.464 INFO:tasks.workunit.client.1.smithi192.stdout:5/51: write d2/f4 [1122628,76104] 0 2024-04-23T02:13:37.467 INFO:tasks.workunit.client.1.smithi192.stdout:4/96: dwrite de/d15/d19/d1a/f1e [0,4194304] 0 2024-04-23T02:13:37.470 INFO:tasks.workunit.client.1.smithi192.stdout:7/48: dwrite d5/f8 [0,4194304] 0 2024-04-23T02:13:37.484 INFO:tasks.workunit.client.1.smithi192.stdout:5/52: dwrite d2/f4 [0,4194304] 0 2024-04-23T02:13:37.487 INFO:tasks.workunit.client.1.smithi192.stdout:5/53: creat d2/d6/f12 x:0 0 0 2024-04-23T02:13:37.488 INFO:tasks.workunit.client.1.smithi192.stdout:7/49: dwrite d5/f8 [0,4194304] 0 2024-04-23T02:13:37.489 INFO:tasks.workunit.client.1.smithi192.stdout:7/50: write d5/f8 [1288322,127356] 0 2024-04-23T02:13:37.490 INFO:tasks.workunit.client.1.smithi192.stdout:7/51: chown d5/f8 17139 1 2024-04-23T02:13:37.494 INFO:tasks.workunit.client.1.smithi192.stdout:7/52: fdatasync d5/fb 0 2024-04-23T02:13:37.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:37 smithi071 ceph-mon[39837]: Updating smithi192:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T02:13:37.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:37 smithi071 ceph-mon[39837]: pgmap v15: 65 pgs: 65 active+clean; 201 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 410 op/s 2024-04-23T02:13:37.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:37 smithi071 ceph-mon[39837]: Reconfiguring prometheus.smithi071 (dependencies changed)... 2024-04-23T02:13:37.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:37 smithi071 ceph-mon[39837]: Reconfiguring daemon prometheus.smithi071 on smithi071 2024-04-23T02:13:37.495 INFO:tasks.workunit.client.1.smithi192.stdout:4/97: write f4 [4331573,118719] 0 2024-04-23T02:13:37.496 INFO:tasks.workunit.client.1.smithi192.stdout:4/98: chown c8 1125887221 1 2024-04-23T02:13:37.497 INFO:tasks.workunit.client.1.smithi192.stdout:4/99: fdatasync de/d15/f16 0 2024-04-23T02:13:37.498 INFO:tasks.workunit.client.1.smithi192.stdout:4/100: chown fc 11 1 2024-04-23T02:13:37.499 INFO:tasks.workunit.client.1.smithi192.stdout:5/54: rmdir d2/d6 39 2024-04-23T02:13:37.500 INFO:tasks.workunit.client.1.smithi192.stdout:4/101: write de/f1b [57513,94467] 0 2024-04-23T02:13:37.501 INFO:tasks.workunit.client.1.smithi192.stdout:7/53: link l4 d5/l11 0 2024-04-23T02:13:37.504 INFO:tasks.workunit.client.1.smithi192.stdout:5/55: rename d2/f4 to d2/db/f13 0 2024-04-23T02:13:37.504 INFO:tasks.workunit.client.1.smithi192.stdout:4/102: rename de/d15/d19 to de/d15/d19/d1a/d20 22 2024-04-23T02:13:37.506 INFO:tasks.workunit.client.1.smithi192.stdout:7/54: symlink d5/l12 0 2024-04-23T02:13:37.507 INFO:tasks.workunit.client.1.smithi192.stdout:4/103: unlink fc 0 2024-04-23T02:13:37.508 INFO:tasks.workunit.client.1.smithi192.stdout:7/55: unlink d5/l12 0 2024-04-23T02:13:37.510 INFO:tasks.workunit.client.1.smithi192.stdout:7/56: write d5/fb [3374550,115877] 0 2024-04-23T02:13:37.510 INFO:tasks.workunit.client.1.smithi192.stdout:5/56: mkdir d2/d6/d14 0 2024-04-23T02:13:37.512 INFO:tasks.workunit.client.1.smithi192.stdout:5/57: dread - d2/db/f11 zero size 2024-04-23T02:13:37.516 INFO:tasks.workunit.client.1.smithi192.stdout:4/104: dwrite de/f18 [0,4194304] 0 2024-04-23T02:13:37.518 INFO:tasks.workunit.client.1.smithi192.stdout:5/58: rename d2/f3 to d2/f15 0 2024-04-23T02:13:37.521 INFO:tasks.workunit.client.1.smithi192.stdout:7/57: rmdir d5 39 2024-04-23T02:13:37.523 INFO:tasks.workunit.client.1.smithi192.stdout:4/105: truncate de/d15/d19/d1a/f1e 2700311 0 2024-04-23T02:13:37.524 INFO:tasks.workunit.client.1.smithi192.stdout:4/106: truncate de/d15/f16 517923 0 2024-04-23T02:13:37.526 INFO:tasks.workunit.client.1.smithi192.stdout:4/107: unlink de/d15/f1d 0 2024-04-23T02:13:37.527 INFO:tasks.workunit.client.1.smithi192.stdout:4/108: stat de/f1b 0 2024-04-23T02:13:37.531 INFO:tasks.workunit.client.1.smithi192.stdout:4/109: dread de/f18 [0,4194304] 0 2024-04-23T02:13:37.532 INFO:tasks.workunit.client.1.smithi192.stdout:4/110: creat de/f21 x:0 0 0 2024-04-23T02:13:37.533 INFO:tasks.workunit.client.1.smithi192.stdout:4/111: chown l7 77 1 2024-04-23T02:13:37.536 INFO:tasks.workunit.client.1.smithi192.stdout:4/112: mknod de/d15/d19/d1a/c22 0 2024-04-23T02:13:37.542 INFO:tasks.workunit.client.1.smithi192.stdout:4/113: dwrite de/f1b [0,4194304] 0 2024-04-23T02:13:37.545 INFO:tasks.workunit.client.1.smithi192.stdout:4/114: creat de/d15/d19/d1a/f23 x:0 0 0 2024-04-23T02:13:37.545 INFO:tasks.workunit.client.1.smithi192.stdout:4/115: chown de 3 1 2024-04-23T02:13:37.549 INFO:tasks.workunit.client.1.smithi192.stdout:4/116: unlink de/d15/d19/d1a/c1c 0 2024-04-23T02:13:37.550 INFO:tasks.workunit.client.1.smithi192.stdout:4/117: dread - de/d15/d19/d1a/f23 zero size 2024-04-23T02:13:37.550 INFO:tasks.workunit.client.1.smithi192.stdout:4/118: chown de/f1b 21 1 2024-04-23T02:13:37.551 INFO:tasks.workunit.client.1.smithi192.stdout:4/119: chown de/l17 22476 1 2024-04-23T02:13:37.557 INFO:tasks.workunit.client.1.smithi192.stdout:4/120: write de/d15/d19/d1a/f1e [1166410,35194] 0 2024-04-23T02:13:37.559 INFO:tasks.workunit.client.1.smithi192.stdout:4/121: mkdir de/d24 0 2024-04-23T02:13:37.562 INFO:tasks.workunit.client.1.smithi192.stdout:4/122: link de/l17 de/d24/l25 0 2024-04-23T02:13:37.563 INFO:tasks.workunit.client.1.smithi192.stdout:4/123: mkdir de/d24/d26 0 2024-04-23T02:13:37.567 INFO:tasks.workunit.client.1.smithi192.stdout:4/124: dread f4 [0,4194304] 0 2024-04-23T02:13:37.567 INFO:tasks.workunit.client.1.smithi192.stdout:4/125: readlink l5 0 2024-04-23T02:13:37.570 INFO:tasks.workunit.client.1.smithi192.stdout:4/126: rename de/lf to de/d15/d19/d1a/l27 0 2024-04-23T02:13:37.571 INFO:tasks.workunit.client.1.smithi192.stdout:4/127: creat de/d24/f28 x:0 0 0 2024-04-23T02:13:37.573 INFO:tasks.workunit.client.1.smithi192.stdout:4/128: link ca de/c29 0 2024-04-23T02:13:37.575 INFO:tasks.workunit.client.1.smithi192.stdout:4/129: mkdir de/d15/d2a 0 2024-04-23T02:13:37.575 INFO:tasks.workunit.client.1.smithi192.stdout:4/130: fdatasync de/d24/f28 0 2024-04-23T02:13:37.582 INFO:tasks.workunit.client.1.smithi192.stdout:4/131: dwrite de/d24/f28 [0,4194304] 0 2024-04-23T02:13:37.585 INFO:tasks.workunit.client.1.smithi192.stdout:4/132: symlink de/d24/d26/l2b 0 2024-04-23T02:13:37.592 INFO:tasks.workunit.client.1.smithi192.stdout:4/133: dwrite de/f21 [0,4194304] 0 2024-04-23T02:13:37.594 INFO:tasks.workunit.client.1.smithi192.stdout:4/134: write de/f21 [2739862,40359] 0 2024-04-23T02:13:37.597 INFO:tasks.workunit.client.1.smithi192.stdout:4/135: creat de/d15/d2a/f2c x:0 0 0 2024-04-23T02:13:37.599 INFO:tasks.workunit.client.1.smithi192.stdout:4/136: creat de/d15/f2d x:0 0 0 2024-04-23T02:13:37.602 INFO:tasks.workunit.client.1.smithi192.stdout:4/137: unlink de/d24/l25 0 2024-04-23T02:13:37.614 INFO:tasks.workunit.client.1.smithi192.stdout:4/138: dwrite de/f1b [4194304,4194304] 0 2024-04-23T02:13:37.616 INFO:tasks.workunit.client.1.smithi192.stdout:4/139: readlink de/l17 0 2024-04-23T02:13:37.617 INFO:tasks.workunit.client.1.smithi192.stdout:4/140: write de/f18 [3721205,117992] 0 2024-04-23T02:13:37.626 INFO:tasks.workunit.client.1.smithi192.stdout:4/141: dwrite f9 [0,4194304] 0 2024-04-23T02:13:37.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:37 smithi192 ceph-mon[49662]: Updating smithi192:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T02:13:37.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:37 smithi192 ceph-mon[49662]: pgmap v15: 65 pgs: 65 active+clean; 201 MiB data, 817 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 410 op/s 2024-04-23T02:13:37.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:37 smithi192 ceph-mon[49662]: Reconfiguring prometheus.smithi071 (dependencies changed)... 2024-04-23T02:13:37.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:37 smithi192 ceph-mon[49662]: Reconfiguring daemon prometheus.smithi071 on smithi071 2024-04-23T02:13:37.636 INFO:tasks.workunit.client.1.smithi192.stdout:4/142: dwrite de/f18 [4194304,4194304] 0 2024-04-23T02:13:37.650 INFO:tasks.workunit.client.1.smithi192.stdout:4/143: dwrite de/d15/f2d [0,4194304] 0 2024-04-23T02:13:37.658 INFO:tasks.workunit.client.1.smithi192.stdout:4/144: creat de/d15/f2e x:0 0 0 2024-04-23T02:13:37.667 INFO:tasks.workunit.client.1.smithi192.stdout:4/145: dwrite de/d15/d2a/f2c [0,4194304] 0 2024-04-23T02:13:37.673 INFO:tasks.workunit.client.1.smithi192.stdout:4/146: getdents de/d15/d19/d1a 0 2024-04-23T02:13:37.677 INFO:tasks.workunit.client.1.smithi192.stdout:4/147: getdents de/d24/d26 0 2024-04-23T02:13:37.679 INFO:tasks.workunit.client.1.smithi192.stdout:4/148: mkdir de/d15/d2f 0 2024-04-23T02:13:37.679 INFO:tasks.workunit.client.1.smithi192.stdout:4/149: chown l5 853996820 1 2024-04-23T02:13:37.682 INFO:tasks.workunit.client.1.smithi192.stdout:4/150: link de/c14 de/d15/d19/c30 0 2024-04-23T02:13:37.683 INFO:tasks.workunit.client.1.smithi192.stdout:4/151: stat de/d15/d19/d1a/f1e 0 2024-04-23T02:13:37.686 INFO:tasks.workunit.client.1.smithi192.stdout:4/152: unlink f9 0 2024-04-23T02:13:37.687 INFO:tasks.workunit.client.1.smithi192.stdout:4/153: fdatasync de/d15/d19/d1a/f1e 0 2024-04-23T02:13:37.687 INFO:tasks.workunit.client.1.smithi192.stdout:4/154: stat de/d15/d19/d1a/c22 0 2024-04-23T02:13:37.693 INFO:tasks.workunit.client.1.smithi192.stdout:4/155: dwrite de/d15/d19/d1a/f1e [0,4194304] 0 2024-04-23T02:13:37.696 INFO:tasks.workunit.client.1.smithi192.stdout:4/156: creat de/d15/f31 x:0 0 0 2024-04-23T02:13:37.702 INFO:tasks.workunit.client.1.smithi192.stdout:4/157: creat de/d15/d2f/f32 x:0 0 0 2024-04-23T02:13:37.705 INFO:tasks.workunit.client.1.smithi192.stdout:4/158: dread de/d15/f2d [0,4194304] 0 2024-04-23T02:13:37.714 INFO:tasks.workunit.client.1.smithi192.stdout:4/159: rename de/d15/d2f to de/d24/d33 0 2024-04-23T02:13:37.717 INFO:tasks.workunit.client.1.smithi192.stdout:4/160: getdents de/d15/d19 0 2024-04-23T02:13:37.718 INFO:tasks.workunit.client.1.smithi192.stdout:4/161: truncate de/d15/f31 507646 0 2024-04-23T02:13:37.718 INFO:tasks.workunit.client.1.smithi192.stdout:4/162: chown de/d15/f31 309 1 2024-04-23T02:13:37.721 INFO:tasks.workunit.client.1.smithi192.stdout:4/163: rmdir de/d24 39 2024-04-23T02:13:37.724 INFO:tasks.workunit.client.1.smithi192.stdout:4/164: dread de/f21 [0,4194304] 0 2024-04-23T02:13:37.727 INFO:tasks.workunit.client.1.smithi192.stdout:4/165: dread de/f21 [0,4194304] 0 2024-04-23T02:13:37.730 INFO:tasks.workunit.client.1.smithi192.stdout:4/166: mkdir de/d24/d33/d34 0 2024-04-23T02:13:37.730 INFO:tasks.workunit.client.1.smithi192.stdout:4/167: dread - de/d15/f2e zero size 2024-04-23T02:13:37.732 INFO:tasks.workunit.client.1.smithi192.stdout:4/168: mknod de/c35 0 2024-04-23T02:13:37.732 INFO:tasks.workunit.client.1.smithi192.stdout:4/169: chown de/d24/f28 290994 1 2024-04-23T02:13:37.733 INFO:tasks.workunit.client.1.smithi192.stdout:4/170: dread de/d15/f31 [0,4194304] 0 2024-04-23T02:13:37.734 INFO:tasks.workunit.client.1.smithi192.stdout:4/171: read de/f1b [7651383,91194] 0 2024-04-23T02:13:37.735 INFO:tasks.workunit.client.1.smithi192.stdout:4/172: creat de/d15/f36 x:0 0 0 2024-04-23T02:13:37.736 INFO:tasks.workunit.client.1.smithi192.stdout:4/173: write de/d24/d33/f32 [382197,92137] 0 2024-04-23T02:13:37.743 INFO:tasks.workunit.client.1.smithi192.stdout:4/174: dwrite f4 [4194304,4194304] 0 2024-04-23T02:13:37.745 INFO:tasks.workunit.client.1.smithi192.stdout:4/175: chown de/c14 439 1 2024-04-23T02:13:37.747 INFO:tasks.workunit.client.0.smithi071.stderr:++ readlink -f fsstress 2024-04-23T02:13:37.749 INFO:tasks.workunit.client.0.smithi071.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-23T02:13:37.749 INFO:tasks.workunit.client.0.smithi071.stderr:+ popd 2024-04-23T02:13:37.751 INFO:tasks.workunit.client.0.smithi071.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-23T02:13:37.751 INFO:tasks.workunit.client.0.smithi071.stderr:+ popd 2024-04-23T02:13:37.752 INFO:tasks.workunit.client.0.smithi071.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-23T02:13:37.752 INFO:tasks.workunit.client.0.smithi071.stderr:++ mktemp -d -p . 2024-04-23T02:13:37.753 INFO:tasks.workunit.client.1.smithi192.stdout:4/176: dwrite de/d15/d19/d1a/f23 [0,4194304] 0 2024-04-23T02:13:37.754 INFO:tasks.workunit.client.1.smithi192.stdout:4/177: stat de/c14 0 2024-04-23T02:13:37.756 INFO:tasks.workunit.client.1.smithi192.stdout:4/178: chown de/d15/d19/d1a/l27 1568771169 1 2024-04-23T02:13:37.756 INFO:tasks.workunit.client.0.smithi071.stderr:+ T=./tmp.oALTRHA6YO 2024-04-23T02:13:37.756 INFO:tasks.workunit.client.0.smithi071.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.oALTRHA6YO -l 1 -n 1000 -p 10 -v 2024-04-23T02:13:37.756 INFO:tasks.workunit.client.1.smithi192.stdout:4/179: dread - de/d15/f2e zero size 2024-04-23T02:13:37.757 INFO:tasks.workunit.client.1.smithi192.stdout:4/180: getdents de/d24/d33/d34 0 2024-04-23T02:13:37.759 INFO:tasks.workunit.client.1.smithi192.stdout:4/181: mkdir de/d24/d37 0 2024-04-23T02:13:37.759 INFO:tasks.workunit.client.0.smithi071.stdout:seed = 1714197145 2024-04-23T02:13:37.761 INFO:tasks.workunit.client.1.smithi192.stdout:4/182: symlink de/d24/d33/d34/l38 0 2024-04-23T02:13:37.763 INFO:tasks.workunit.client.1.smithi192.stdout:4/183: symlink de/d15/d2a/l39 0 2024-04-23T02:13:37.764 INFO:tasks.workunit.client.0.smithi071.stdout:0/0: write - no filename 2024-04-23T02:13:37.764 INFO:tasks.workunit.client.0.smithi071.stdout:0/1: chown . 460451 1 2024-04-23T02:13:37.764 INFO:tasks.workunit.client.0.smithi071.stdout:0/2: write - no filename 2024-04-23T02:13:37.764 INFO:tasks.workunit.client.0.smithi071.stdout:0/3: chown . 26108371 1 2024-04-23T02:13:37.764 INFO:tasks.workunit.client.0.smithi071.stdout:0/4: read - no filename 2024-04-23T02:13:37.765 INFO:tasks.workunit.client.0.smithi071.stdout:1/0: dread - no filename 2024-04-23T02:13:37.765 INFO:tasks.workunit.client.0.smithi071.stdout:1/1: dread - no filename 2024-04-23T02:13:37.766 INFO:tasks.workunit.client.1.smithi192.stdout:4/184: rename de/d15/f16 to de/d15/d19/d1a/f3a 0 2024-04-23T02:13:37.766 INFO:tasks.workunit.client.0.smithi071.stdout:0/5: mkdir d0 0 2024-04-23T02:13:37.767 INFO:tasks.workunit.client.0.smithi071.stdout:2/0: write - no filename 2024-04-23T02:13:37.767 INFO:tasks.workunit.client.0.smithi071.stdout:2/1: rename - no filename 2024-04-23T02:13:37.767 INFO:tasks.workunit.client.1.smithi192.stdout:4/185: symlink de/d24/d26/l3b 0 2024-04-23T02:13:37.768 INFO:tasks.workunit.client.0.smithi071.stdout:1/2: creat f0 x:0 0 0 2024-04-23T02:13:37.768 INFO:tasks.workunit.client.1.smithi192.stdout:4/186: read de/d15/d19/d1a/f3a [1017,5190] 0 2024-04-23T02:13:37.769 INFO:tasks.workunit.client.0.smithi071.stdout:0/6: creat d0/f1 x:0 0 0 2024-04-23T02:13:37.770 INFO:tasks.workunit.client.1.smithi192.stdout:4/187: rmdir de/d24 39 2024-04-23T02:13:37.770 INFO:tasks.workunit.client.0.smithi071.stdout:3/0: write - no filename 2024-04-23T02:13:37.770 INFO:tasks.workunit.client.0.smithi071.stdout:1/3: creat f1 x:0 0 0 2024-04-23T02:13:37.770 INFO:tasks.workunit.client.0.smithi071.stdout:0/7: write d0/f1 [80630,95718] 0 2024-04-23T02:13:37.771 INFO:tasks.workunit.client.1.smithi192.stdout:4/188: write de/d24/f28 [5159266,70163] 0 2024-04-23T02:13:37.772 INFO:tasks.workunit.client.0.smithi071.stdout:2/2: mkdir d0 0 2024-04-23T02:13:37.772 INFO:tasks.workunit.client.0.smithi071.stdout:2/3: readlink - no filename 2024-04-23T02:13:37.772 INFO:tasks.workunit.client.0.smithi071.stdout:2/4: write - no filename 2024-04-23T02:13:37.773 INFO:tasks.workunit.client.1.smithi192.stdout:4/189: truncate de/f1b 3495503 0 2024-04-23T02:13:37.774 INFO:tasks.workunit.client.0.smithi071.stdout:1/4: creat f2 x:0 0 0 2024-04-23T02:13:37.775 INFO:tasks.workunit.client.0.smithi071.stdout:0/8: creat d0/f2 x:0 0 0 2024-04-23T02:13:37.775 INFO:tasks.workunit.client.0.smithi071.stdout:3/1: getdents . 0 2024-04-23T02:13:37.775 INFO:tasks.workunit.client.0.smithi071.stdout:3/2: readlink - no filename 2024-04-23T02:13:37.775 INFO:tasks.workunit.client.0.smithi071.stdout:3/3: chown . 282 1 2024-04-23T02:13:37.775 INFO:tasks.workunit.client.0.smithi071.stdout:3/4: dread - no filename 2024-04-23T02:13:37.775 INFO:tasks.workunit.client.0.smithi071.stdout:3/5: dread - no filename 2024-04-23T02:13:37.775 INFO:tasks.workunit.client.0.smithi071.stdout:3/6: write - no filename 2024-04-23T02:13:37.776 INFO:tasks.workunit.client.0.smithi071.stdout:5/0: stat - no entries 2024-04-23T02:13:37.776 INFO:tasks.workunit.client.0.smithi071.stdout:4/0: mknod c0 0 2024-04-23T02:13:37.777 INFO:tasks.workunit.client.0.smithi071.stdout:2/5: creat d0/f1 x:0 0 0 2024-04-23T02:13:37.777 INFO:tasks.workunit.client.0.smithi071.stdout:0/9: mkdir d0/d3 0 2024-04-23T02:13:37.778 INFO:tasks.workunit.client.0.smithi071.stdout:2/6: fdatasync d0/f1 0 2024-04-23T02:13:37.779 INFO:tasks.workunit.client.0.smithi071.stdout:6/0: link - no file 2024-04-23T02:13:37.779 INFO:tasks.workunit.client.1.smithi192.stdout:4/190: link de/d15/d19/d1a/l27 de/d24/d33/d34/l3c 0 2024-04-23T02:13:37.780 INFO:tasks.workunit.client.0.smithi071.stdout:4/1: rename c0 to c1 0 2024-04-23T02:13:37.780 INFO:tasks.workunit.client.0.smithi071.stdout:4/2: truncate - no filename 2024-04-23T02:13:37.780 INFO:tasks.workunit.client.1.smithi192.stdout:4/191: write de/d15/d2a/f2c [1630194,2250] 0 2024-04-23T02:13:37.781 INFO:tasks.workunit.client.0.smithi071.stdout:2/7: mkdir d0/d2 0 2024-04-23T02:13:37.782 INFO:tasks.workunit.client.0.smithi071.stdout:5/1: mknod c0 0 2024-04-23T02:13:37.782 INFO:tasks.workunit.client.0.smithi071.stdout:5/2: fsync - no filename 2024-04-23T02:13:37.783 INFO:tasks.workunit.client.0.smithi071.stdout:5/3: read - no filename 2024-04-23T02:13:37.783 INFO:tasks.workunit.client.0.smithi071.stdout:5/4: truncate - no filename 2024-04-23T02:13:37.785 INFO:tasks.workunit.client.0.smithi071.stdout:7/0: dread - no filename 2024-04-23T02:13:37.785 INFO:tasks.workunit.client.0.smithi071.stdout:7/1: write - no filename 2024-04-23T02:13:37.785 INFO:tasks.workunit.client.0.smithi071.stdout:7/2: chown . 141732 1 2024-04-23T02:13:37.785 INFO:tasks.workunit.client.0.smithi071.stdout:7/3: fdatasync - no filename 2024-04-23T02:13:37.786 INFO:tasks.workunit.client.0.smithi071.stdout:4/3: mkdir d2 0 2024-04-23T02:13:37.786 INFO:tasks.workunit.client.0.smithi071.stdout:4/4: chown d2 942 1 2024-04-23T02:13:37.786 INFO:tasks.workunit.client.0.smithi071.stdout:4/5: truncate - no filename 2024-04-23T02:13:37.787 INFO:tasks.workunit.client.0.smithi071.stdout:0/10: mknod d0/d3/c4 0 2024-04-23T02:13:37.789 INFO:tasks.workunit.client.1.smithi192.stdout:4/192: dwrite de/d15/f2e [0,4194304] 0 2024-04-23T02:13:37.790 INFO:tasks.workunit.client.0.smithi071.stdout:2/8: unlink d0/f1 0 2024-04-23T02:13:37.790 INFO:tasks.workunit.client.0.smithi071.stdout:1/5: fsync f0 0 2024-04-23T02:13:37.790 INFO:tasks.workunit.client.0.smithi071.stdout:1/6: dread - f0 zero size 2024-04-23T02:13:37.790 INFO:tasks.workunit.client.0.smithi071.stdout:5/5: creat f1 x:0 0 0 2024-04-23T02:13:37.791 INFO:tasks.workunit.client.0.smithi071.stdout:1/7: chown f2 114222809 1 2024-04-23T02:13:37.791 INFO:tasks.workunit.client.0.smithi071.stdout:6/1: creat f0 x:0 0 0 2024-04-23T02:13:37.792 INFO:tasks.workunit.client.1.smithi192.stdout:4/193: rename de/d24/d33 to de/d15/d19/d3d 0 2024-04-23T02:13:37.792 INFO:tasks.workunit.client.0.smithi071.stdout:1/8: write f1 [539151,125975] 0 2024-04-23T02:13:37.795 INFO:tasks.workunit.client.0.smithi071.stdout:8/0: dwrite - no filename 2024-04-23T02:13:37.795 INFO:tasks.workunit.client.0.smithi071.stdout:8/1: read - no filename 2024-04-23T02:13:37.795 INFO:tasks.workunit.client.0.smithi071.stdout:8/2: dwrite - no filename 2024-04-23T02:13:37.795 INFO:tasks.workunit.client.0.smithi071.stdout:8/3: write - no filename 2024-04-23T02:13:37.795 INFO:tasks.workunit.client.0.smithi071.stdout:8/4: read - no filename 2024-04-23T02:13:37.795 INFO:tasks.workunit.client.0.smithi071.stdout:8/5: dwrite - no filename 2024-04-23T02:13:37.795 INFO:tasks.workunit.client.1.smithi192.stdout:4/194: rmdir de/d24/d37 0 2024-04-23T02:13:37.797 INFO:tasks.workunit.client.1.smithi192.stdout:4/195: symlink de/d15/d19/l3e 0 2024-04-23T02:13:37.798 INFO:tasks.workunit.client.0.smithi071.stdout:2/9: mknod d0/c3 0 2024-04-23T02:13:37.801 INFO:tasks.workunit.client.0.smithi071.stdout:9/0: dread - no filename 2024-04-23T02:13:37.801 INFO:tasks.workunit.client.0.smithi071.stdout:9/1: truncate - no filename 2024-04-23T02:13:37.801 INFO:tasks.workunit.client.0.smithi071.stdout:9/2: write - no filename 2024-04-23T02:13:37.801 INFO:tasks.workunit.client.1.smithi192.stdout:4/196: dread f4 [4194304,4194304] 0 2024-04-23T02:13:37.801 INFO:tasks.workunit.client.0.smithi071.stdout:6/2: rename f0 to f1 0 2024-04-23T02:13:37.803 INFO:tasks.workunit.client.0.smithi071.stdout:5/6: dwrite f1 [0,4194304] 0 2024-04-23T02:13:37.803 INFO:tasks.workunit.client.0.smithi071.stdout:7/4: mknod c0 0 2024-04-23T02:13:37.803 INFO:tasks.workunit.client.0.smithi071.stdout:1/9: creat f3 x:0 0 0 2024-04-23T02:13:37.803 INFO:tasks.workunit.client.1.smithi192.stdout:4/197: creat de/d15/d19/d1a/f3f x:0 0 0 2024-04-23T02:13:37.804 INFO:tasks.workunit.client.0.smithi071.stdout:4/6: symlink d2/l3 0 2024-04-23T02:13:37.805 INFO:tasks.workunit.client.0.smithi071.stdout:2/10: mknod d0/c4 0 2024-04-23T02:13:37.805 INFO:tasks.workunit.client.0.smithi071.stdout:2/11: dwrite - no filename 2024-04-23T02:13:37.805 INFO:tasks.workunit.client.0.smithi071.stdout:2/12: read - no filename 2024-04-23T02:13:37.805 INFO:tasks.workunit.client.0.smithi071.stdout:2/13: read - no filename 2024-04-23T02:13:37.806 INFO:tasks.workunit.client.0.smithi071.stdout:2/14: stat d0 0 2024-04-23T02:13:37.806 INFO:tasks.workunit.client.0.smithi071.stdout:2/15: truncate - no filename 2024-04-23T02:13:37.807 INFO:tasks.workunit.client.0.smithi071.stdout:2/16: chown d0/c3 766838583 1 2024-04-23T02:13:37.807 INFO:tasks.workunit.client.0.smithi071.stdout:2/17: fdatasync - no filename 2024-04-23T02:13:37.807 INFO:tasks.workunit.client.0.smithi071.stdout:2/18: write - no filename 2024-04-23T02:13:37.807 INFO:tasks.workunit.client.0.smithi071.stdout:6/3: creat f2 x:0 0 0 2024-04-23T02:13:37.807 INFO:tasks.workunit.client.1.smithi192.stdout:4/198: symlink de/d15/d19/d1a/l40 0 2024-04-23T02:13:37.807 INFO:tasks.workunit.client.1.smithi192.stdout:4/199: write de/f21 [843336,74027] 0 2024-04-23T02:13:37.808 INFO:tasks.workunit.client.1.smithi192.stdout:4/200: fdatasync de/d15/d19/d1a/f3f 0 2024-04-23T02:13:37.809 INFO:tasks.workunit.client.0.smithi071.stdout:1/10: mknod c4 0 2024-04-23T02:13:37.809 INFO:tasks.workunit.client.0.smithi071.stdout:1/11: rmdir - no directory 2024-04-23T02:13:37.809 INFO:tasks.workunit.client.0.smithi071.stdout:1/12: fsync f2 0 2024-04-23T02:13:37.810 INFO:tasks.workunit.client.0.smithi071.stdout:1/13: rmdir - no directory 2024-04-23T02:13:37.810 INFO:tasks.workunit.client.1.smithi192.stdout:4/201: mknod de/d15/d19/c41 0 2024-04-23T02:13:37.812 INFO:tasks.workunit.client.0.smithi071.stdout:8/6: mknod c0 0 2024-04-23T02:13:37.812 INFO:tasks.workunit.client.0.smithi071.stdout:8/7: write - no filename 2024-04-23T02:13:37.812 INFO:tasks.workunit.client.0.smithi071.stdout:8/8: fsync - no filename 2024-04-23T02:13:37.812 INFO:tasks.workunit.client.0.smithi071.stdout:8/9: fdatasync - no filename 2024-04-23T02:13:37.812 INFO:tasks.workunit.client.0.smithi071.stdout:8/10: fsync - no filename 2024-04-23T02:13:37.812 INFO:tasks.workunit.client.0.smithi071.stdout:8/11: dread - no filename 2024-04-23T02:13:37.813 INFO:tasks.workunit.client.1.smithi192.stdout:4/202: symlink de/d24/d26/l42 0 2024-04-23T02:13:37.813 INFO:tasks.workunit.client.0.smithi071.stdout:8/12: chown c0 320821661 1 2024-04-23T02:13:37.813 INFO:tasks.workunit.client.0.smithi071.stdout:8/13: write - no filename 2024-04-23T02:13:37.814 INFO:tasks.workunit.client.0.smithi071.stdout:9/3: creat f0 x:0 0 0 2024-04-23T02:13:37.816 INFO:tasks.workunit.client.0.smithi071.stdout:1/14: dread f1 [0,4194304] 0 2024-04-23T02:13:37.817 INFO:tasks.workunit.client.0.smithi071.stdout:6/4: rename f2 to f3 0 2024-04-23T02:13:37.820 INFO:tasks.workunit.client.1.smithi192.stdout:4/203: dwrite de/d15/d19/d3d/f32 [0,4194304] 0 2024-04-23T02:13:37.821 INFO:tasks.workunit.client.0.smithi071.stdout:0/11: getdents d0 0 2024-04-23T02:13:37.822 INFO:tasks.workunit.client.0.smithi071.stdout:0/12: chown d0/f2 197038 1 2024-04-23T02:13:37.823 INFO:tasks.workunit.client.0.smithi071.stdout:0/13: write d0/f1 [80817,103656] 0 2024-04-23T02:13:37.823 INFO:tasks.workunit.client.0.smithi071.stdout:8/14: symlink l1 0 2024-04-23T02:13:37.824 INFO:tasks.workunit.client.0.smithi071.stdout:8/15: chown l1 0 1 2024-04-23T02:13:37.824 INFO:tasks.workunit.client.0.smithi071.stdout:8/16: dread - no filename 2024-04-23T02:13:37.824 INFO:tasks.workunit.client.0.smithi071.stdout:8/17: write - no filename 2024-04-23T02:13:37.824 INFO:tasks.workunit.client.0.smithi071.stdout:8/18: read - no filename 2024-04-23T02:13:37.825 INFO:tasks.workunit.client.0.smithi071.stdout:0/14: chown d0/f1 3 1 2024-04-23T02:13:37.825 INFO:tasks.workunit.client.0.smithi071.stdout:9/4: mkdir d1 0 2024-04-23T02:13:37.826 INFO:tasks.workunit.client.0.smithi071.stdout:2/19: getdents d0/d2 0 2024-04-23T02:13:37.826 INFO:tasks.workunit.client.0.smithi071.stdout:2/20: dwrite - no filename 2024-04-23T02:13:37.827 INFO:tasks.workunit.client.0.smithi071.stdout:2/21: dread - no filename 2024-04-23T02:13:37.827 INFO:tasks.workunit.client.0.smithi071.stdout:2/22: write - no filename 2024-04-23T02:13:37.830 INFO:tasks.workunit.client.0.smithi071.stdout:7/5: link c0 c1 0 2024-04-23T02:13:37.830 INFO:tasks.workunit.client.0.smithi071.stdout:7/6: dwrite - no filename 2024-04-23T02:13:37.830 INFO:tasks.workunit.client.0.smithi071.stdout:7/7: fdatasync - no filename 2024-04-23T02:13:37.831 INFO:tasks.workunit.client.0.smithi071.stdout:7/8: truncate - no filename 2024-04-23T02:13:37.833 INFO:tasks.workunit.client.1.smithi192.stdout:4/204: truncate de/d15/f2e 2871836 0 2024-04-23T02:13:37.833 INFO:tasks.workunit.client.0.smithi071.stdout:8/19: creat f2 x:0 0 0 2024-04-23T02:13:37.834 INFO:tasks.workunit.client.0.smithi071.stdout:8/20: dread - f2 zero size 2024-04-23T02:13:37.834 INFO:tasks.workunit.client.1.smithi192.stdout:4/205: truncate de/d15/d19/d1a/f3f 353278 0 2024-04-23T02:13:37.834 INFO:tasks.workunit.client.0.smithi071.stdout:5/7: fdatasync f1 0 2024-04-23T02:13:37.835 INFO:tasks.workunit.client.0.smithi071.stdout:0/15: creat d0/f5 x:0 0 0 2024-04-23T02:13:37.835 INFO:tasks.workunit.client.0.smithi071.stdout:0/16: dread - d0/f2 zero size 2024-04-23T02:13:37.835 INFO:tasks.workunit.client.0.smithi071.stdout:1/15: unlink f1 0 2024-04-23T02:13:37.836 INFO:tasks.workunit.client.0.smithi071.stdout:8/21: write f2 [394478,46382] 0 2024-04-23T02:13:37.836 INFO:tasks.workunit.client.1.smithi192.stdout:4/206: mknod de/d15/d2a/c43 0 2024-04-23T02:13:37.837 INFO:tasks.workunit.client.0.smithi071.stdout:1/16: fdatasync f0 0 2024-04-23T02:13:37.837 INFO:tasks.workunit.client.0.smithi071.stdout:4/7: getdents d2 0 2024-04-23T02:13:37.838 INFO:tasks.workunit.client.0.smithi071.stdout:9/5: mknod d1/c2 0 2024-04-23T02:13:37.838 INFO:tasks.workunit.client.0.smithi071.stdout:1/17: fdatasync f3 0 2024-04-23T02:13:37.839 INFO:tasks.workunit.client.0.smithi071.stdout:9/6: fdatasync f0 0 2024-04-23T02:13:37.839 INFO:tasks.workunit.client.0.smithi071.stdout:5/8: rename f1 to f2 0 2024-04-23T02:13:37.841 INFO:tasks.workunit.client.0.smithi071.stdout:9/7: chown d1 270533 1 2024-04-23T02:13:37.841 INFO:tasks.workunit.client.0.smithi071.stdout:7/9: chown c1 1083969 1 2024-04-23T02:13:37.843 INFO:tasks.workunit.client.1.smithi192.stdout:4/207: truncate de/d15/d19/d1a/f1e 742188 0 2024-04-23T02:13:37.844 INFO:tasks.workunit.client.0.smithi071.stdout:4/8: creat d2/f4 x:0 0 0 2024-04-23T02:13:37.844 INFO:tasks.workunit.client.1.smithi192.stdout:4/208: write de/f18 [3201106,107355] 0 2024-04-23T02:13:37.845 INFO:tasks.workunit.client.0.smithi071.stdout:1/18: creat f5 x:0 0 0 2024-04-23T02:13:37.846 INFO:tasks.workunit.client.0.smithi071.stdout:6/5: getdents . 0 2024-04-23T02:13:37.847 INFO:tasks.workunit.client.0.smithi071.stdout:1/19: dread - f3 zero size 2024-04-23T02:13:37.847 INFO:tasks.workunit.client.0.smithi071.stdout:6/6: dread - f1 zero size 2024-04-23T02:13:37.847 INFO:tasks.workunit.client.0.smithi071.stdout:5/9: mkdir d3 0 2024-04-23T02:13:37.848 INFO:tasks.workunit.client.0.smithi071.stdout:4/9: write d2/f4 [97879,86546] 0 2024-04-23T02:13:37.849 INFO:tasks.workunit.client.1.smithi192.stdout:4/209: chown de/d15/d19/d1a/f1e 333 1 2024-04-23T02:13:37.850 INFO:tasks.workunit.client.1.smithi192.stdout:4/210: chown de/d15/d19/c30 6802 1 2024-04-23T02:13:37.850 INFO:tasks.workunit.client.0.smithi071.stdout:7/10: stat c1 0 2024-04-23T02:13:37.850 INFO:tasks.workunit.client.0.smithi071.stdout:7/11: dwrite - no filename 2024-04-23T02:13:37.851 INFO:tasks.workunit.client.1.smithi192.stdout:4/211: fdatasync de/d15/f2d 0 2024-04-23T02:13:37.851 INFO:tasks.workunit.client.0.smithi071.stdout:8/22: dwrite f2 [0,4194304] 0 2024-04-23T02:13:37.852 INFO:tasks.workunit.client.1.smithi192.stdout:4/212: fsync f4 0 2024-04-23T02:13:37.852 INFO:tasks.workunit.client.0.smithi071.stdout:4/10: write d2/f4 [500329,19313] 0 2024-04-23T02:13:37.853 INFO:tasks.workunit.client.0.smithi071.stdout:8/23: read f2 [2345841,56010] 0 2024-04-23T02:13:37.854 INFO:tasks.workunit.client.0.smithi071.stdout:1/20: creat f6 x:0 0 0 2024-04-23T02:13:37.855 INFO:tasks.workunit.client.0.smithi071.stdout:1/21: write f0 [516279,108670] 0 2024-04-23T02:13:37.856 INFO:tasks.workunit.client.1.smithi192.stdout:4/213: dread de/d15/f2e [0,4194304] 0 2024-04-23T02:13:37.857 INFO:tasks.workunit.client.0.smithi071.stdout:8/24: dread f2 [0,4194304] 0 2024-04-23T02:13:37.859 INFO:tasks.workunit.client.0.smithi071.stdout:9/8: rename f0 to d1/f3 0 2024-04-23T02:13:37.860 INFO:tasks.workunit.client.1.smithi192.stdout:4/214: dread de/d24/f28 [0,4194304] 0 2024-04-23T02:13:37.861 INFO:tasks.workunit.client.0.smithi071.stdout:5/10: symlink d3/l4 0 2024-04-23T02:13:37.864 INFO:tasks.workunit.client.0.smithi071.stdout:7/12: rename c1 to c2 0 2024-04-23T02:13:37.865 INFO:tasks.workunit.client.0.smithi071.stdout:6/7: link f1 f4 0 2024-04-23T02:13:37.866 INFO:tasks.workunit.client.0.smithi071.stdout:4/11: symlink d2/l5 0 2024-04-23T02:13:37.866 INFO:tasks.workunit.client.0.smithi071.stdout:6/8: truncate f3 617401 0 2024-04-23T02:13:37.867 INFO:tasks.workunit.client.0.smithi071.stdout:0/17: getdents d0 0 2024-04-23T02:13:37.868 INFO:tasks.workunit.client.0.smithi071.stdout:4/12: fdatasync d2/f4 0 2024-04-23T02:13:37.868 INFO:tasks.workunit.client.0.smithi071.stdout:1/22: creat f7 x:0 0 0 2024-04-23T02:13:37.868 INFO:tasks.workunit.client.0.smithi071.stdout:8/25: dwrite f2 [0,4194304] 0 2024-04-23T02:13:37.873 INFO:tasks.workunit.client.1.smithi192.stdout:4/215: dwrite de/d15/f2e [0,4194304] 0 2024-04-23T02:13:37.874 INFO:tasks.workunit.client.1.smithi192.stdout:4/216: chown ca 0 1 2024-04-23T02:13:37.877 INFO:tasks.workunit.client.1.smithi192.stdout:4/217: mknod de/d24/c44 0 2024-04-23T02:13:37.881 INFO:tasks.workunit.client.1.smithi192.stdout:4/218: dread de/d15/d2a/f2c [0,4194304] 0 2024-04-23T02:13:37.882 INFO:tasks.workunit.client.1.smithi192.stdout:4/219: stat de/d15/d19/d1a/f23 0 2024-04-23T02:13:37.883 INFO:tasks.workunit.client.1.smithi192.stdout:4/220: write de/d15/f36 [118114,12168] 0 2024-04-23T02:13:37.886 INFO:tasks.workunit.client.1.smithi192.stdout:4/221: getdents de/d15/d19 0 2024-04-23T02:13:37.887 INFO:tasks.workunit.client.0.smithi071.stdout:1/23: write f5 [238109,29462] 0 2024-04-23T02:13:37.887 INFO:tasks.workunit.client.0.smithi071.stdout:5/11: creat d3/f5 x:0 0 0 2024-04-23T02:13:37.888 INFO:tasks.workunit.client.0.smithi071.stdout:7/13: mkdir d3 0 2024-04-23T02:13:37.888 INFO:tasks.workunit.client.0.smithi071.stdout:4/13: read d2/f4 [412421,78707] 0 2024-04-23T02:13:37.889 INFO:tasks.workunit.client.1.smithi192.stdout:4/222: dread de/d24/f28 [0,4194304] 0 2024-04-23T02:13:37.891 INFO:tasks.workunit.client.1.smithi192.stdout:4/223: rmdir de/d15/d19/d3d 39 2024-04-23T02:13:37.891 INFO:tasks.workunit.client.0.smithi071.stdout:6/9: write f1 [767151,88625] 0 2024-04-23T02:13:37.892 INFO:tasks.workunit.client.0.smithi071.stdout:6/10: stat f1 0 2024-04-23T02:13:37.893 INFO:tasks.workunit.client.0.smithi071.stdout:6/11: stat f4 0 2024-04-23T02:13:37.897 INFO:tasks.workunit.client.0.smithi071.stdout:1/24: dread f0 [0,4194304] 0 2024-04-23T02:13:37.899 INFO:tasks.workunit.client.1.smithi192.stdout:4/224: dwrite de/d15/d19/d3d/f32 [0,4194304] 0 2024-04-23T02:13:37.900 INFO:tasks.workunit.client.0.smithi071.stdout:1/25: dread - f7 zero size 2024-04-23T02:13:37.903 INFO:tasks.workunit.client.1.smithi192.stdout:4/225: dread de/d15/d19/d1a/f23 [0,4194304] 0 2024-04-23T02:13:37.905 INFO:tasks.workunit.client.0.smithi071.stdout:9/9: dwrite d1/f3 [0,4194304] 0 2024-04-23T02:13:37.906 INFO:tasks.workunit.client.1.smithi192.stdout:4/226: getdents de/d15/d19 0 2024-04-23T02:13:37.907 INFO:tasks.workunit.client.0.smithi071.stdout:4/14: dwrite d2/f4 [0,4194304] 0 2024-04-23T02:13:37.910 INFO:tasks.workunit.client.1.smithi192.stdout:4/227: dread de/d15/d2a/f2c [0,4194304] 0 2024-04-23T02:13:37.913 INFO:tasks.workunit.client.1.smithi192.stdout:4/228: dread de/d15/f2d [0,4194304] 0 2024-04-23T02:13:37.915 INFO:tasks.workunit.client.0.smithi071.stdout:6/12: dwrite f4 [0,4194304] 0 2024-04-23T02:13:37.915 INFO:tasks.workunit.client.0.smithi071.stdout:6/13: readlink - no filename 2024-04-23T02:13:37.921 INFO:tasks.workunit.client.1.smithi192.stdout:4/229: dwrite de/d15/f2e [0,4194304] 0 2024-04-23T02:13:37.927 INFO:tasks.workunit.client.1.smithi192.stdout:4/230: symlink de/l45 0 2024-04-23T02:13:37.928 INFO:tasks.workunit.client.1.smithi192.stdout:4/231: read de/d15/d2a/f2c [3306296,27791] 0 2024-04-23T02:13:37.952 INFO:tasks.workunit.client.1.smithi192.stdout:4/232: truncate de/d15/d2a/f2c 1418374 0 2024-04-23T02:13:37.953 INFO:tasks.workunit.client.1.smithi192.stdout:4/233: creat de/d15/d19/d3d/f46 x:0 0 0 2024-04-23T02:13:37.957 INFO:tasks.workunit.client.1.smithi192.stdout:4/234: dread de/f1b [0,4194304] 0 2024-04-23T02:13:37.961 INFO:tasks.workunit.client.1.smithi192.stdout:4/235: rmdir de/d15/d2a 39 2024-04-23T02:13:37.963 INFO:tasks.workunit.client.1.smithi192.stdout:4/236: dread de/d15/d19/d1a/f3f [0,4194304] 0 2024-04-23T02:13:37.963 INFO:tasks.workunit.client.1.smithi192.stdout:4/237: chown de/d15/f2d 4071 1 2024-04-23T02:13:37.964 INFO:tasks.workunit.client.1.smithi192.stdout:4/238: truncate de/d15/d19/d3d/f46 292095 0 2024-04-23T02:13:37.966 INFO:tasks.workunit.client.1.smithi192.stdout:4/239: mknod de/d24/c47 0 2024-04-23T02:13:37.973 INFO:tasks.workunit.client.1.smithi192.stdout:4/240: dwrite de/f1b [0,4194304] 0 2024-04-23T02:13:37.974 INFO:tasks.workunit.client.1.smithi192.stdout:4/241: rename de to de/d24/d26/d48 22 2024-04-23T02:13:37.980 INFO:tasks.workunit.client.1.smithi192.stdout:4/242: truncate de/f21 1852362 0 2024-04-23T02:13:37.984 INFO:tasks.workunit.client.1.smithi192.stdout:4/243: dread de/d15/d19/d1a/f23 [0,4194304] 0 2024-04-23T02:13:37.987 INFO:tasks.workunit.client.1.smithi192.stdout:4/244: dread de/d15/d19/d3d/f32 [0,4194304] 0 2024-04-23T02:13:37.990 INFO:tasks.workunit.client.1.smithi192.stdout:4/245: rename de/d24 to de/d15/d19/d3d/d49 0 2024-04-23T02:13:37.992 INFO:tasks.workunit.client.1.smithi192.stdout:4/246: mkdir de/d15/d19/d3d/d34/d4a 0 2024-04-23T02:13:38.019 INFO:tasks.workunit.client.0.smithi071.stdout:0/18: mknod d0/c6 0 2024-04-23T02:13:38.020 INFO:tasks.workunit.client.0.smithi071.stdout:8/26: symlink l3 0 2024-04-23T02:13:38.020 INFO:tasks.workunit.client.0.smithi071.stdout:0/19: truncate d0/f5 320151 0 2024-04-23T02:13:38.021 INFO:tasks.workunit.client.0.smithi071.stdout:0/20: stat d0/d3/c4 0 2024-04-23T02:13:38.021 INFO:tasks.workunit.client.0.smithi071.stdout:0/21: read - d0/f2 zero size 2024-04-23T02:13:38.022 INFO:tasks.workunit.client.0.smithi071.stdout:0/22: fsync d0/f2 0 2024-04-23T02:13:38.023 INFO:tasks.workunit.client.0.smithi071.stdout:0/23: read d0/f5 [281413,64976] 0 2024-04-23T02:13:38.024 INFO:tasks.workunit.client.0.smithi071.stdout:0/24: write d0/f5 [418187,30136] 0 2024-04-23T02:13:38.024 INFO:tasks.workunit.client.0.smithi071.stdout:0/25: chown d0/f5 571 1 2024-04-23T02:13:38.025 INFO:tasks.workunit.client.0.smithi071.stdout:0/26: chown d0/f5 5155 1 2024-04-23T02:13:38.026 INFO:tasks.workunit.client.1.smithi192.stdout:4/247: dwrite de/d15/d2a/f2c [0,4194304] 0 2024-04-23T02:13:38.026 INFO:tasks.workunit.client.0.smithi071.stdout:9/10: creat d1/f4 x:0 0 0 2024-04-23T02:13:38.027 INFO:tasks.workunit.client.0.smithi071.stdout:8/27: mknod c4 0 2024-04-23T02:13:38.028 INFO:tasks.workunit.client.1.smithi192.stdout:4/248: fsync de/d15/d19/d1a/f3a 0 2024-04-23T02:13:38.028 INFO:tasks.workunit.client.1.smithi192.stdout:4/249: fsync f4 0 2024-04-23T02:13:38.028 INFO:tasks.workunit.client.0.smithi071.stdout:0/27: dread d0/f5 [0,4194304] 0 2024-04-23T02:13:38.029 INFO:tasks.workunit.client.0.smithi071.stdout:0/28: write d0/f5 [581110,15609] 0 2024-04-23T02:13:38.030 INFO:tasks.workunit.client.1.smithi192.stdout:4/250: unlink lb 0 2024-04-23T02:13:38.030 INFO:tasks.workunit.client.0.smithi071.stdout:7/14: rename c0 to d3/c4 0 2024-04-23T02:13:38.032 INFO:tasks.workunit.client.0.smithi071.stdout:0/29: write d0/f2 [55929,68011] 0 2024-04-23T02:13:38.034 INFO:tasks.workunit.client.0.smithi071.stdout:0/30: write d0/f2 [148296,121751] 0 2024-04-23T02:13:38.037 INFO:tasks.workunit.client.1.smithi192.stdout:4/251: write de/f21 [232127,43969] 0 2024-04-23T02:13:38.038 INFO:tasks.workunit.client.0.smithi071.stdout:6/14: link f3 f5 0 2024-04-23T02:13:38.040 INFO:tasks.workunit.client.0.smithi071.stdout:9/11: symlink d1/l5 0 2024-04-23T02:13:38.041 INFO:tasks.workunit.client.0.smithi071.stdout:8/28: symlink l5 0 2024-04-23T02:13:38.042 INFO:tasks.workunit.client.0.smithi071.stdout:9/12: write d1/f3 [120458,56763] 0 2024-04-23T02:13:38.042 INFO:tasks.workunit.client.0.smithi071.stdout:8/29: stat l1 0 2024-04-23T02:13:38.043 INFO:tasks.workunit.client.1.smithi192.stdout:4/252: rename de/d15/d19/d3d/d49/d26/l42 to de/l4b 0 2024-04-23T02:13:38.044 INFO:tasks.workunit.client.0.smithi071.stdout:5/12: link c0 d3/c6 0 2024-04-23T02:13:38.044 INFO:tasks.workunit.client.0.smithi071.stdout:9/13: dread - d1/f4 zero size 2024-04-23T02:13:38.045 INFO:tasks.workunit.client.0.smithi071.stdout:0/31: dwrite d0/f2 [0,4194304] 0 2024-04-23T02:13:38.045 INFO:tasks.workunit.client.0.smithi071.stdout:8/30: write f2 [3529188,115700] 0 2024-04-23T02:13:38.047 INFO:tasks.workunit.client.0.smithi071.stdout:5/13: truncate d3/f5 36877 0 2024-04-23T02:13:38.047 INFO:tasks.workunit.client.0.smithi071.stdout:9/14: write d1/f4 [670746,90434] 0 2024-04-23T02:13:38.047 INFO:tasks.workunit.client.0.smithi071.stdout:8/31: write f2 [2827433,55652] 0 2024-04-23T02:13:38.048 INFO:tasks.workunit.client.0.smithi071.stdout:8/32: readlink l3 0 2024-04-23T02:13:38.049 INFO:tasks.workunit.client.0.smithi071.stdout:9/15: write d1/f3 [608723,32704] 0 2024-04-23T02:13:38.050 INFO:tasks.workunit.client.0.smithi071.stdout:6/15: mkdir d6 0 2024-04-23T02:13:38.052 INFO:tasks.workunit.client.1.smithi192.stdout:4/253: dwrite de/d15/d19/d3d/f46 [0,4194304] 0 2024-04-23T02:13:38.053 INFO:tasks.workunit.client.0.smithi071.stdout:7/15: unlink c2 0 2024-04-23T02:13:38.053 INFO:tasks.workunit.client.0.smithi071.stdout:7/16: chown d3 329223857 1 2024-04-23T02:13:38.053 INFO:tasks.workunit.client.0.smithi071.stdout:7/17: readlink - no filename 2024-04-23T02:13:38.053 INFO:tasks.workunit.client.0.smithi071.stdout:7/18: dread - no filename 2024-04-23T02:13:38.053 INFO:tasks.workunit.client.0.smithi071.stdout:7/19: rename d3 to d3/d5 22 2024-04-23T02:13:38.053 INFO:tasks.workunit.client.0.smithi071.stdout:7/20: dread - no filename 2024-04-23T02:13:38.056 INFO:tasks.workunit.client.0.smithi071.stdout:0/32: write d0/f5 [219718,72964] 0 2024-04-23T02:13:38.057 INFO:tasks.workunit.client.0.smithi071.stdout:1/26: getdents . 0 2024-04-23T02:13:38.058 INFO:tasks.workunit.client.0.smithi071.stdout:1/27: truncate f3 528971 0 2024-04-23T02:13:38.058 INFO:tasks.workunit.client.1.smithi192.stdout:4/254: dread de/d15/f36 [0,4194304] 0 2024-04-23T02:13:38.058 INFO:tasks.workunit.client.0.smithi071.stdout:5/14: symlink d3/l7 0 2024-04-23T02:13:38.061 INFO:tasks.workunit.client.0.smithi071.stdout:1/28: write f3 [1172321,33758] 0 2024-04-23T02:13:38.061 INFO:tasks.workunit.client.0.smithi071.stdout:1/29: readlink - no filename 2024-04-23T02:13:38.061 INFO:tasks.workunit.client.0.smithi071.stdout:5/15: read d3/f5 [7894,76043] 0 2024-04-23T02:13:38.062 INFO:tasks.workunit.client.1.smithi192.stdout:4/255: creat de/d15/d2a/f4c x:0 0 0 2024-04-23T02:13:38.063 INFO:tasks.workunit.client.0.smithi071.stdout:9/16: rename d1/l5 to d1/l6 0 2024-04-23T02:13:38.064 INFO:tasks.workunit.client.1.smithi192.stdout:4/256: write de/d15/d2a/f4c [438625,98651] 0 2024-04-23T02:13:38.066 INFO:tasks.workunit.client.1.smithi192.stdout:4/257: creat de/f4d x:0 0 0 2024-04-23T02:13:38.066 INFO:tasks.workunit.client.0.smithi071.stdout:1/30: write f5 [346763,98194] 0 2024-04-23T02:13:38.068 INFO:tasks.workunit.client.1.smithi192.stdout:4/258: symlink de/d15/l4e 0 2024-04-23T02:13:38.070 INFO:tasks.workunit.client.1.smithi192.stdout:4/259: mknod de/d15/d19/d3d/d34/d4a/c4f 0 2024-04-23T02:13:38.070 INFO:tasks.workunit.client.0.smithi071.stdout:0/33: dwrite d0/f5 [0,4194304] 0 2024-04-23T02:13:38.071 INFO:tasks.workunit.client.0.smithi071.stdout:5/16: dread f2 [0,4194304] 0 2024-04-23T02:13:38.072 INFO:tasks.workunit.client.1.smithi192.stdout:4/260: creat de/d15/d19/d3d/d49/f50 x:0 0 0 2024-04-23T02:13:38.074 INFO:tasks.workunit.client.0.smithi071.stdout:6/16: mknod d6/c7 0 2024-04-23T02:13:38.075 INFO:tasks.workunit.client.0.smithi071.stdout:0/34: dread d0/f1 [0,4194304] 0 2024-04-23T02:13:38.076 INFO:tasks.workunit.client.1.smithi192.stdout:4/261: dread de/d15/d19/d3d/f32 [0,4194304] 0 2024-04-23T02:13:38.077 INFO:tasks.workunit.client.0.smithi071.stdout:5/17: mkdir d3/d8 0 2024-04-23T02:13:38.078 INFO:tasks.workunit.client.1.smithi192.stdout:4/262: write de/d15/d19/d1a/f3a [1049690,96974] 0 2024-04-23T02:13:38.079 INFO:tasks.workunit.client.0.smithi071.stdout:6/17: write f5 [407214,36125] 0 2024-04-23T02:13:38.081 INFO:tasks.workunit.client.1.smithi192.stdout:4/263: mknod de/d15/d19/d3d/d34/d4a/c51 0 2024-04-23T02:13:38.081 INFO:tasks.workunit.client.0.smithi071.stdout:0/35: dread d0/f2 [0,4194304] 0 2024-04-23T02:13:38.082 INFO:tasks.workunit.client.0.smithi071.stdout:5/18: creat d3/f9 x:0 0 0 2024-04-23T02:13:38.084 INFO:tasks.workunit.client.0.smithi071.stdout:6/18: dread f1 [0,4194304] 0 2024-04-23T02:13:38.084 INFO:tasks.workunit.client.0.smithi071.stdout:6/19: stat f1 0 2024-04-23T02:13:38.085 INFO:tasks.workunit.client.0.smithi071.stdout:0/36: symlink d0/l7 0 2024-04-23T02:13:38.087 INFO:tasks.workunit.client.1.smithi192.stdout:4/264: truncate de/f18 2377450 0 2024-04-23T02:13:38.089 INFO:tasks.workunit.client.1.smithi192.stdout:4/265: symlink de/d15/d19/d1a/l52 0 2024-04-23T02:13:38.089 INFO:tasks.workunit.client.0.smithi071.stdout:5/19: unlink c0 0 2024-04-23T02:13:38.091 INFO:tasks.workunit.client.1.smithi192.stdout:4/266: mkdir de/d15/d53 0 2024-04-23T02:13:38.091 INFO:tasks.workunit.client.0.smithi071.stdout:5/20: chown d3/l4 1 1 2024-04-23T02:13:38.091 INFO:tasks.workunit.client.0.smithi071.stdout:6/20: link f1 d6/f8 0 2024-04-23T02:13:38.092 INFO:tasks.workunit.client.0.smithi071.stdout:0/37: dwrite d0/f5 [0,4194304] 0 2024-04-23T02:13:38.092 INFO:tasks.workunit.client.1.smithi192.stdout:4/267: creat de/d15/d19/d3d/f54 x:0 0 0 2024-04-23T02:13:38.093 INFO:tasks.workunit.client.0.smithi071.stdout:5/21: write f2 [2130901,14447] 0 2024-04-23T02:13:38.093 INFO:tasks.workunit.client.0.smithi071.stdout:5/22: chown d3 501926011 1 2024-04-23T02:13:38.094 INFO:tasks.workunit.client.1.smithi192.stdout:4/268: mknod de/d15/c55 0 2024-04-23T02:13:38.095 INFO:tasks.workunit.client.0.smithi071.stdout:0/38: write d0/f5 [4103496,45003] 0 2024-04-23T02:13:38.095 INFO:tasks.workunit.client.0.smithi071.stdout:0/39: readlink d0/l7 0 2024-04-23T02:13:38.096 INFO:tasks.workunit.client.0.smithi071.stdout:0/40: read d0/f1 [17732,93431] 0 2024-04-23T02:13:38.100 INFO:tasks.workunit.client.0.smithi071.stdout:5/23: rename d3/d8 to d3/da 0 2024-04-23T02:13:38.101 INFO:tasks.workunit.client.1.smithi192.stdout:4/269: dwrite de/d15/d19/d1a/f3a [0,4194304] 0 2024-04-23T02:13:38.103 INFO:tasks.workunit.client.0.smithi071.stdout:5/24: fdatasync d3/f5 0 2024-04-23T02:13:38.104 INFO:tasks.workunit.client.0.smithi071.stdout:5/25: write f2 [4147844,126194] 0 2024-04-23T02:13:38.108 INFO:tasks.workunit.client.1.smithi192.stdout:4/270: dread de/d15/d19/d1a/f23 [0,4194304] 0 2024-04-23T02:13:38.110 INFO:tasks.workunit.client.1.smithi192.stdout:4/271: creat de/d15/d19/d3d/d34/d4a/f56 x:0 0 0 2024-04-23T02:13:38.111 INFO:tasks.workunit.client.1.smithi192.stdout:4/272: fdatasync de/f21 0 2024-04-23T02:13:38.114 INFO:tasks.workunit.client.1.smithi192.stdout:4/273: rename de/d15/f2e to de/d15/d19/d3d/f57 0 2024-04-23T02:13:38.140 INFO:tasks.workunit.client.0.smithi071.stdout:7/21: rename d3/c4 to d3/c6 0 2024-04-23T02:13:38.141 INFO:tasks.workunit.client.0.smithi071.stdout:4/15: truncate d2/f4 1936523 0 2024-04-23T02:13:38.142 INFO:tasks.workunit.client.0.smithi071.stdout:7/22: mknod d3/c7 0 2024-04-23T02:13:38.142 INFO:tasks.workunit.client.0.smithi071.stdout:7/23: truncate - no filename 2024-04-23T02:13:38.143 INFO:tasks.workunit.client.0.smithi071.stdout:7/24: rename d3 to d3/d8 22 2024-04-23T02:13:38.143 INFO:tasks.workunit.client.0.smithi071.stdout:7/25: read - no filename 2024-04-23T02:13:38.143 INFO:tasks.workunit.client.0.smithi071.stdout:7/26: stat d3/c7 0 2024-04-23T02:13:38.150 INFO:tasks.workunit.client.0.smithi071.stdout:9/17: rename d1/l6 to d1/l7 0 2024-04-23T02:13:38.150 INFO:tasks.workunit.client.0.smithi071.stdout:8/33: truncate f2 722650 0 2024-04-23T02:13:38.151 INFO:tasks.workunit.client.0.smithi071.stdout:9/18: creat d1/f8 x:0 0 0 2024-04-23T02:13:38.153 INFO:tasks.workunit.client.0.smithi071.stdout:9/19: creat d1/f9 x:0 0 0 2024-04-23T02:13:38.154 INFO:tasks.workunit.client.0.smithi071.stdout:9/20: stat d1/f8 0 2024-04-23T02:13:38.157 INFO:tasks.workunit.client.0.smithi071.stdout:5/26: fsync d3/f9 0 2024-04-23T02:13:38.157 INFO:tasks.workunit.client.0.smithi071.stdout:6/21: write d6/f8 [4304985,5282] 0 2024-04-23T02:13:38.158 INFO:tasks.workunit.client.0.smithi071.stdout:5/27: stat d3/da 0 2024-04-23T02:13:38.158 INFO:tasks.workunit.client.0.smithi071.stdout:9/21: dread d1/f3 [0,4194304] 0 2024-04-23T02:13:38.160 INFO:tasks.workunit.client.0.smithi071.stdout:6/22: creat d6/f9 x:0 0 0 2024-04-23T02:13:38.163 INFO:tasks.workunit.client.0.smithi071.stdout:5/28: creat d3/da/fb x:0 0 0 2024-04-23T02:13:38.167 INFO:tasks.workunit.client.0.smithi071.stdout:6/23: dwrite f1 [0,4194304] 0 2024-04-23T02:13:38.167 INFO:tasks.workunit.client.0.smithi071.stdout:6/24: write f5 [1238179,17174] 0 2024-04-23T02:13:38.168 INFO:tasks.workunit.client.0.smithi071.stdout:0/41: truncate d0/f2 2937254 0 2024-04-23T02:13:38.174 INFO:tasks.workunit.client.0.smithi071.stdout:6/25: creat d6/fa x:0 0 0 2024-04-23T02:13:38.175 INFO:tasks.workunit.client.0.smithi071.stdout:0/42: mknod d0/c8 0 2024-04-23T02:13:38.177 INFO:tasks.workunit.client.0.smithi071.stdout:9/22: readlink d1/l7 0 2024-04-23T02:13:38.178 INFO:tasks.workunit.client.0.smithi071.stdout:7/27: stat d3/c6 0 2024-04-23T02:13:38.178 INFO:tasks.workunit.client.0.smithi071.stdout:7/28: truncate - no filename 2024-04-23T02:13:38.178 INFO:tasks.workunit.client.0.smithi071.stdout:7/29: dwrite - no filename 2024-04-23T02:13:38.178 INFO:tasks.workunit.client.0.smithi071.stdout:7/30: write - no filename 2024-04-23T02:13:38.179 INFO:tasks.workunit.client.0.smithi071.stdout:9/23: dread - d1/f8 zero size 2024-04-23T02:13:38.184 INFO:tasks.workunit.client.0.smithi071.stdout:6/26: dwrite d6/f8 [0,4194304] 0 2024-04-23T02:13:38.185 INFO:tasks.workunit.client.0.smithi071.stdout:7/31: creat d3/f9 x:0 0 0 2024-04-23T02:13:38.192 INFO:tasks.workunit.client.0.smithi071.stdout:7/32: creat d3/fa x:0 0 0 2024-04-23T02:13:38.195 INFO:tasks.workunit.client.0.smithi071.stdout:6/27: dwrite d6/f9 [0,4194304] 0 2024-04-23T02:13:38.199 INFO:tasks.workunit.client.0.smithi071.stdout:5/29: fdatasync d3/da/fb 0 2024-04-23T02:13:38.200 INFO:tasks.workunit.client.0.smithi071.stdout:7/33: symlink d3/lb 0 2024-04-23T02:13:38.200 INFO:tasks.workunit.client.0.smithi071.stdout:9/24: dwrite d1/f3 [0,4194304] 0 2024-04-23T02:13:38.208 INFO:tasks.workunit.client.0.smithi071.stdout:0/43: getdents d0 0 2024-04-23T02:13:38.209 INFO:tasks.workunit.client.0.smithi071.stdout:9/25: stat d1/f3 0 2024-04-23T02:13:38.210 INFO:tasks.workunit.client.0.smithi071.stdout:7/34: chown d3/c6 31193864 1 2024-04-23T02:13:38.213 INFO:tasks.workunit.client.0.smithi071.stdout:9/26: write d1/f8 [213568,61398] 0 2024-04-23T02:13:38.213 INFO:tasks.workunit.client.0.smithi071.stdout:0/44: symlink d0/d3/l9 0 2024-04-23T02:13:38.215 INFO:tasks.workunit.client.0.smithi071.stdout:5/30: symlink d3/lc 0 2024-04-23T02:13:38.217 INFO:tasks.workunit.client.0.smithi071.stdout:7/35: creat d3/fc x:0 0 0 2024-04-23T02:13:38.219 INFO:tasks.workunit.client.0.smithi071.stdout:4/16: dwrite d2/f4 [0,4194304] 0 2024-04-23T02:13:38.219 INFO:tasks.workunit.client.0.smithi071.stdout:0/45: mkdir d0/da 0 2024-04-23T02:13:38.220 INFO:tasks.workunit.client.0.smithi071.stdout:6/28: dread f1 [0,4194304] 0 2024-04-23T02:13:38.222 INFO:tasks.workunit.client.0.smithi071.stdout:0/46: chown d0/d3 55 1 2024-04-23T02:13:38.222 INFO:tasks.workunit.client.0.smithi071.stdout:9/27: rename d1/f8 to d1/fa 0 2024-04-23T02:13:38.222 INFO:tasks.workunit.client.0.smithi071.stdout:8/34: dwrite f2 [0,4194304] 0 2024-04-23T02:13:38.224 INFO:tasks.workunit.client.0.smithi071.stdout:8/35: write f2 [3491665,125106] 0 2024-04-23T02:13:38.224 INFO:tasks.workunit.client.0.smithi071.stdout:8/36: rmdir - no directory 2024-04-23T02:13:38.225 INFO:tasks.workunit.client.0.smithi071.stdout:7/36: dread - d3/fc zero size 2024-04-23T02:13:38.225 INFO:tasks.workunit.client.0.smithi071.stdout:5/31: write d3/f9 [971265,64388] 0 2024-04-23T02:13:38.225 INFO:tasks.workunit.client.0.smithi071.stdout:0/47: rename d0/da to d0/da/db 22 2024-04-23T02:13:38.226 INFO:tasks.workunit.client.0.smithi071.stdout:6/29: read f1 [4275946,103318] 0 2024-04-23T02:13:38.227 INFO:tasks.workunit.client.0.smithi071.stdout:9/28: creat d1/fb x:0 0 0 2024-04-23T02:13:38.232 INFO:tasks.workunit.client.0.smithi071.stdout:7/37: rename d3/fa to d3/fd 0 2024-04-23T02:13:38.233 INFO:tasks.workunit.client.0.smithi071.stdout:6/30: creat d6/fb x:0 0 0 2024-04-23T02:13:38.234 INFO:tasks.workunit.client.0.smithi071.stdout:4/17: dread d2/f4 [0,4194304] 0 2024-04-23T02:13:38.235 INFO:tasks.workunit.client.0.smithi071.stdout:9/29: creat d1/fc x:0 0 0 2024-04-23T02:13:38.237 INFO:tasks.workunit.client.0.smithi071.stdout:9/30: write d1/f4 [304303,22415] 0 2024-04-23T02:13:38.237 INFO:tasks.workunit.client.0.smithi071.stdout:5/32: rename d3/f5 to d3/da/fd 0 2024-04-23T02:13:38.239 INFO:tasks.workunit.client.0.smithi071.stdout:7/38: mkdir d3/de 0 2024-04-23T02:13:38.241 INFO:tasks.workunit.client.0.smithi071.stdout:4/18: mknod d2/c6 0 2024-04-23T02:13:38.243 INFO:tasks.workunit.client.0.smithi071.stdout:9/31: symlink d1/ld 0 2024-04-23T02:13:38.244 INFO:tasks.workunit.client.0.smithi071.stdout:5/33: mknod d3/da/ce 0 2024-04-23T02:13:38.246 INFO:tasks.workunit.client.0.smithi071.stdout:8/37: getdents . 0 2024-04-23T02:13:38.247 INFO:tasks.workunit.client.0.smithi071.stdout:9/32: creat d1/fe x:0 0 0 2024-04-23T02:13:38.249 INFO:tasks.workunit.client.0.smithi071.stdout:8/38: write f2 [1904570,68688] 0 2024-04-23T02:13:38.250 INFO:tasks.workunit.client.0.smithi071.stdout:7/39: dwrite d3/fc [0,4194304] 0 2024-04-23T02:13:38.251 INFO:tasks.workunit.client.0.smithi071.stdout:7/40: stat d3/fd 0 2024-04-23T02:13:38.252 INFO:tasks.workunit.client.0.smithi071.stdout:4/19: rmdir d2 39 2024-04-23T02:13:38.253 INFO:tasks.workunit.client.0.smithi071.stdout:5/34: mkdir d3/df 0 2024-04-23T02:13:38.255 INFO:tasks.workunit.client.0.smithi071.stdout:5/35: dread - d3/da/fb zero size 2024-04-23T02:13:38.256 INFO:tasks.workunit.client.0.smithi071.stdout:8/39: rename f2 to f6 0 2024-04-23T02:13:38.257 INFO:tasks.workunit.client.0.smithi071.stdout:7/41: creat d3/ff x:0 0 0 2024-04-23T02:13:38.258 INFO:tasks.workunit.client.0.smithi071.stdout:9/33: dwrite d1/f3 [0,4194304] 0 2024-04-23T02:13:38.259 INFO:tasks.workunit.client.0.smithi071.stdout:7/42: dread - d3/ff zero size 2024-04-23T02:13:38.262 INFO:tasks.workunit.client.0.smithi071.stdout:5/36: write d3/da/fd [552735,119337] 0 2024-04-23T02:13:38.263 INFO:tasks.workunit.client.0.smithi071.stdout:9/34: mknod d1/cf 0 2024-04-23T02:13:38.265 INFO:tasks.workunit.client.0.smithi071.stdout:5/37: chown d3/df 177589 1 2024-04-23T02:13:38.268 INFO:tasks.workunit.client.0.smithi071.stdout:4/20: dread d2/f4 [0,4194304] 0 2024-04-23T02:13:38.268 INFO:tasks.workunit.client.0.smithi071.stdout:5/38: dread - d3/da/fb zero size 2024-04-23T02:13:38.269 INFO:tasks.workunit.client.0.smithi071.stdout:7/43: dread d3/fc [0,4194304] 0 2024-04-23T02:13:38.274 INFO:tasks.workunit.client.0.smithi071.stdout:9/35: creat d1/f10 x:0 0 0 2024-04-23T02:13:38.274 INFO:tasks.workunit.client.0.smithi071.stdout:7/44: chown d3/c7 0 1 2024-04-23T02:13:38.274 INFO:tasks.workunit.client.0.smithi071.stdout:8/40: dwrite f6 [4194304,4194304] 0 2024-04-23T02:13:38.274 INFO:tasks.workunit.client.0.smithi071.stdout:4/21: rmdir d2 39 2024-04-23T02:13:38.274 INFO:tasks.workunit.client.0.smithi071.stdout:9/36: symlink d1/l11 0 2024-04-23T02:13:38.276 INFO:tasks.workunit.client.0.smithi071.stdout:8/41: mknod c7 0 2024-04-23T02:13:38.277 INFO:tasks.workunit.client.0.smithi071.stdout:5/39: mkdir d3/df/d10 0 2024-04-23T02:13:38.277 INFO:tasks.workunit.client.0.smithi071.stdout:7/45: symlink d3/l10 0 2024-04-23T02:13:38.278 INFO:tasks.workunit.client.0.smithi071.stdout:7/46: read d3/fc [1895474,61257] 0 2024-04-23T02:13:38.278 INFO:tasks.workunit.client.0.smithi071.stdout:4/22: mkdir d2/d7 0 2024-04-23T02:13:38.279 INFO:tasks.workunit.client.0.smithi071.stdout:4/23: read d2/f4 [2437192,18351] 0 2024-04-23T02:13:38.280 INFO:tasks.workunit.client.0.smithi071.stdout:7/47: creat d3/f11 x:0 0 0 2024-04-23T02:13:38.282 INFO:tasks.workunit.client.0.smithi071.stdout:4/24: creat d2/f8 x:0 0 0 2024-04-23T02:13:38.282 INFO:tasks.workunit.client.0.smithi071.stdout:5/40: creat d3/f11 x:0 0 0 2024-04-23T02:13:38.283 INFO:tasks.workunit.client.0.smithi071.stdout:7/48: creat d3/f12 x:0 0 0 2024-04-23T02:13:38.285 INFO:tasks.workunit.client.0.smithi071.stdout:7/49: getdents d3/de 0 2024-04-23T02:13:38.286 INFO:tasks.workunit.client.0.smithi071.stdout:7/50: write d3/f11 [159980,83927] 0 2024-04-23T02:13:38.286 INFO:tasks.workunit.client.0.smithi071.stdout:5/41: dread f2 [0,4194304] 0 2024-04-23T02:13:38.288 INFO:tasks.workunit.client.0.smithi071.stdout:7/51: write d3/f12 [882655,103238] 0 2024-04-23T02:13:38.288 INFO:tasks.workunit.client.0.smithi071.stdout:8/42: fdatasync f6 0 2024-04-23T02:13:38.289 INFO:tasks.workunit.client.0.smithi071.stdout:8/43: chown c4 3374080 1 2024-04-23T02:13:38.289 INFO:tasks.workunit.client.0.smithi071.stdout:7/52: write d3/f12 [631238,109492] 0 2024-04-23T02:13:38.293 INFO:tasks.workunit.client.0.smithi071.stdout:5/42: creat d3/f12 x:0 0 0 2024-04-23T02:13:38.295 INFO:tasks.workunit.client.0.smithi071.stdout:8/44: unlink c4 0 2024-04-23T02:13:38.295 INFO:tasks.workunit.client.0.smithi071.stdout:8/45: write f6 [3221493,126966] 0 2024-04-23T02:13:38.296 INFO:tasks.workunit.client.0.smithi071.stdout:7/53: mknod d3/c13 0 2024-04-23T02:13:38.297 INFO:tasks.workunit.client.0.smithi071.stdout:5/43: creat d3/da/f13 x:0 0 0 2024-04-23T02:13:38.298 INFO:tasks.workunit.client.0.smithi071.stdout:5/44: dread - d3/da/fb zero size 2024-04-23T02:13:38.299 INFO:tasks.workunit.client.0.smithi071.stdout:8/46: creat f8 x:0 0 0 2024-04-23T02:13:38.300 INFO:tasks.workunit.client.0.smithi071.stdout:4/25: rmdir d2 39 2024-04-23T02:13:38.301 INFO:tasks.workunit.client.0.smithi071.stdout:5/45: chown d3/da/ce 1549 1 2024-04-23T02:13:38.306 INFO:tasks.workunit.client.0.smithi071.stdout:7/54: dwrite d3/f12 [0,4194304] 0 2024-04-23T02:13:38.306 INFO:tasks.workunit.client.0.smithi071.stdout:5/46: creat d3/df/d10/f14 x:0 0 0 2024-04-23T02:13:38.307 INFO:tasks.workunit.client.0.smithi071.stdout:7/55: readlink d3/l10 0 2024-04-23T02:13:38.310 INFO:tasks.workunit.client.0.smithi071.stdout:5/47: creat d3/da/f15 x:0 0 0 2024-04-23T02:13:38.312 INFO:tasks.workunit.client.0.smithi071.stdout:5/48: fdatasync d3/df/d10/f14 0 2024-04-23T02:13:38.313 INFO:tasks.workunit.client.0.smithi071.stdout:8/47: dwrite f8 [0,4194304] 0 2024-04-23T02:13:38.313 INFO:tasks.workunit.client.0.smithi071.stdout:8/48: rmdir - no directory 2024-04-23T02:13:38.313 INFO:tasks.workunit.client.0.smithi071.stdout:4/26: dread d2/f4 [0,4194304] 0 2024-04-23T02:13:38.316 INFO:tasks.workunit.client.0.smithi071.stdout:4/27: truncate d2/f4 4411049 0 2024-04-23T02:13:38.316 INFO:tasks.workunit.client.0.smithi071.stdout:8/49: creat f9 x:0 0 0 2024-04-23T02:13:38.316 INFO:tasks.workunit.client.0.smithi071.stdout:7/56: truncate d3/fc 3898182 0 2024-04-23T02:13:38.318 INFO:tasks.workunit.client.0.smithi071.stdout:5/49: rmdir d3/df/d10 39 2024-04-23T02:13:38.319 INFO:tasks.workunit.client.0.smithi071.stdout:4/28: write d2/f4 [2692745,127892] 0 2024-04-23T02:13:38.320 INFO:tasks.workunit.client.0.smithi071.stdout:8/50: rename c0 to ca 0 2024-04-23T02:13:38.320 INFO:tasks.workunit.client.0.smithi071.stdout:4/29: chown d2/f4 596533 1 2024-04-23T02:13:38.321 INFO:tasks.workunit.client.0.smithi071.stdout:7/57: mknod d3/c14 0 2024-04-23T02:13:38.322 INFO:tasks.workunit.client.0.smithi071.stdout:5/50: mknod d3/da/c16 0 2024-04-23T02:13:38.325 INFO:tasks.workunit.client.0.smithi071.stdout:4/30: creat d2/d7/f9 x:0 0 0 2024-04-23T02:13:38.326 INFO:tasks.workunit.client.0.smithi071.stdout:5/51: truncate d3/da/fb 997408 0 2024-04-23T02:13:38.327 INFO:tasks.workunit.client.0.smithi071.stdout:4/31: rename d2 to d2/d7/da 22 2024-04-23T02:13:38.327 INFO:tasks.workunit.client.0.smithi071.stdout:7/58: rmdir d3/de 0 2024-04-23T02:13:38.330 INFO:tasks.workunit.client.0.smithi071.stdout:4/32: truncate d2/f4 4505102 0 2024-04-23T02:13:38.330 INFO:tasks.workunit.client.0.smithi071.stdout:7/59: chown d3/f11 173171106 1 2024-04-23T02:13:38.331 INFO:tasks.workunit.client.0.smithi071.stdout:8/51: dwrite f9 [0,4194304] 0 2024-04-23T02:13:38.332 INFO:tasks.workunit.client.0.smithi071.stdout:4/33: dread - d2/d7/f9 zero size 2024-04-23T02:13:38.332 INFO:tasks.workunit.client.0.smithi071.stdout:5/52: creat d3/df/f17 x:0 0 0 2024-04-23T02:13:38.333 INFO:tasks.workunit.client.0.smithi071.stdout:4/34: chown d2/f8 49 1 2024-04-23T02:13:38.334 INFO:tasks.workunit.client.0.smithi071.stdout:5/53: fsync d3/da/f13 0 2024-04-23T02:13:38.334 INFO:tasks.workunit.client.0.smithi071.stdout:7/60: symlink d3/l15 0 2024-04-23T02:13:38.336 INFO:tasks.workunit.client.0.smithi071.stdout:7/61: readlink d3/lb 0 2024-04-23T02:13:38.336 INFO:tasks.workunit.client.0.smithi071.stdout:5/54: fdatasync d3/da/fd 0 2024-04-23T02:13:38.339 INFO:tasks.workunit.client.0.smithi071.stdout:4/35: rename c1 to d2/cb 0 2024-04-23T02:13:38.342 INFO:tasks.workunit.client.0.smithi071.stdout:8/52: dwrite f9 [0,4194304] 0 2024-04-23T02:13:38.342 INFO:tasks.workunit.client.0.smithi071.stdout:4/36: dread - d2/d7/f9 zero size 2024-04-23T02:13:38.343 INFO:tasks.workunit.client.0.smithi071.stdout:7/62: write d3/fd [417317,127784] 0 2024-04-23T02:13:38.347 INFO:tasks.workunit.client.0.smithi071.stdout:4/37: creat d2/d7/fc x:0 0 0 2024-04-23T02:13:38.349 INFO:tasks.workunit.client.0.smithi071.stdout:4/38: read d2/f4 [3730463,128476] 0 2024-04-23T02:13:38.351 INFO:tasks.workunit.client.0.smithi071.stdout:5/55: dwrite d3/f11 [0,4194304] 0 2024-04-23T02:13:38.355 INFO:tasks.workunit.client.0.smithi071.stdout:8/53: dwrite f6 [4194304,4194304] 0 2024-04-23T02:13:38.356 INFO:tasks.workunit.client.0.smithi071.stdout:8/54: read f6 [1289600,6421] 0 2024-04-23T02:13:38.358 INFO:tasks.workunit.client.0.smithi071.stdout:4/39: rename d2/l3 to d2/d7/ld 0 2024-04-23T02:13:38.358 INFO:tasks.workunit.client.0.smithi071.stdout:7/63: write d3/fc [2310760,101277] 0 2024-04-23T02:13:38.359 INFO:tasks.workunit.client.0.smithi071.stdout:8/55: mknod cb 0 2024-04-23T02:13:38.359 INFO:tasks.workunit.client.0.smithi071.stdout:8/56: rmdir - no directory 2024-04-23T02:13:38.361 INFO:tasks.workunit.client.0.smithi071.stdout:4/40: symlink d2/d7/le 0 2024-04-23T02:13:38.363 INFO:tasks.workunit.client.0.smithi071.stdout:8/57: mkdir dc 0 2024-04-23T02:13:38.365 INFO:tasks.workunit.client.0.smithi071.stdout:7/64: dread d3/f12 [0,4194304] 0 2024-04-23T02:13:38.366 INFO:tasks.workunit.client.0.smithi071.stdout:4/41: fsync d2/f4 0 2024-04-23T02:13:38.368 INFO:tasks.workunit.client.0.smithi071.stdout:4/42: truncate d2/d7/fc 808812 0 2024-04-23T02:13:38.368 INFO:tasks.workunit.client.0.smithi071.stdout:5/56: link d3/lc d3/l18 0 2024-04-23T02:13:38.369 INFO:tasks.workunit.client.0.smithi071.stdout:4/43: dread - d2/d7/f9 zero size 2024-04-23T02:13:38.369 INFO:tasks.workunit.client.0.smithi071.stdout:7/65: mknod d3/c16 0 2024-04-23T02:13:38.370 INFO:tasks.workunit.client.0.smithi071.stdout:5/57: write d3/da/fb [1232269,16053] 0 2024-04-23T02:13:38.370 INFO:tasks.workunit.client.0.smithi071.stdout:8/58: rename ca to dc/cd 0 2024-04-23T02:13:38.371 INFO:tasks.workunit.client.0.smithi071.stdout:5/58: write d3/da/fd [1670682,111863] 0 2024-04-23T02:13:38.372 INFO:tasks.workunit.client.0.smithi071.stdout:7/66: mkdir d3/d17 0 2024-04-23T02:13:38.372 INFO:tasks.workunit.client.0.smithi071.stdout:5/59: write d3/f11 [4489339,26192] 0 2024-04-23T02:13:38.373 INFO:tasks.workunit.client.0.smithi071.stdout:8/59: write f6 [2589901,55036] 0 2024-04-23T02:13:38.375 INFO:tasks.workunit.client.0.smithi071.stdout:4/44: rename d2/f8 to d2/d7/ff 0 2024-04-23T02:13:38.376 INFO:tasks.workunit.client.0.smithi071.stdout:7/67: creat d3/f18 x:0 0 0 2024-04-23T02:13:38.378 INFO:tasks.workunit.client.0.smithi071.stdout:4/45: mkdir d2/d7/d10 0 2024-04-23T02:13:38.379 INFO:tasks.workunit.client.0.smithi071.stdout:5/60: symlink d3/df/d10/l19 0 2024-04-23T02:13:38.379 INFO:tasks.workunit.client.0.smithi071.stdout:5/61: dread - d3/df/f17 zero size 2024-04-23T02:13:38.380 INFO:tasks.workunit.client.0.smithi071.stdout:5/62: chown d3/da/f13 7140 1 2024-04-23T02:13:38.381 INFO:tasks.workunit.client.0.smithi071.stdout:7/68: rename d3/c16 to d3/d17/c19 0 2024-04-23T02:13:38.381 INFO:tasks.workunit.client.0.smithi071.stdout:7/69: dread - d3/f18 zero size 2024-04-23T02:13:38.382 INFO:tasks.workunit.client.0.smithi071.stdout:4/46: symlink d2/l11 0 2024-04-23T02:13:38.382 INFO:tasks.workunit.client.0.smithi071.stdout:4/47: stat d2/d7/le 0 2024-04-23T02:13:38.384 INFO:tasks.workunit.client.0.smithi071.stdout:5/63: mkdir d3/da/d1a 0 2024-04-23T02:13:38.384 INFO:tasks.workunit.client.0.smithi071.stdout:5/64: chown d3/f12 2231 1 2024-04-23T02:13:38.385 INFO:tasks.workunit.client.0.smithi071.stdout:4/48: mknod d2/d7/d10/c12 0 2024-04-23T02:13:38.387 INFO:tasks.workunit.client.0.smithi071.stdout:5/65: creat d3/da/d1a/f1b x:0 0 0 2024-04-23T02:13:38.388 INFO:tasks.workunit.client.0.smithi071.stdout:4/49: creat d2/d7/d10/f13 x:0 0 0 2024-04-23T02:13:38.389 INFO:tasks.workunit.client.0.smithi071.stdout:5/66: mkdir d3/da/d1a/d1c 0 2024-04-23T02:13:38.390 INFO:tasks.workunit.client.0.smithi071.stdout:4/50: creat d2/d7/f14 x:0 0 0 2024-04-23T02:13:38.391 INFO:tasks.workunit.client.0.smithi071.stdout:5/67: chown d3/lc 0 1 2024-04-23T02:13:38.392 INFO:tasks.workunit.client.0.smithi071.stdout:4/51: mkdir d2/d15 0 2024-04-23T02:13:38.394 INFO:tasks.workunit.client.0.smithi071.stdout:5/68: mkdir d3/d1d 0 2024-04-23T02:13:38.396 INFO:tasks.workunit.client.0.smithi071.stdout:4/52: dread d2/f4 [0,4194304] 0 2024-04-23T02:13:38.396 INFO:tasks.workunit.client.0.smithi071.stdout:5/69: unlink d3/df/d10/l19 0 2024-04-23T02:13:38.396 INFO:tasks.workunit.client.0.smithi071.stdout:4/53: fdatasync d2/f4 0 2024-04-23T02:13:38.399 INFO:tasks.workunit.client.0.smithi071.stdout:4/54: creat d2/d7/f16 x:0 0 0 2024-04-23T02:13:38.401 INFO:tasks.workunit.client.0.smithi071.stdout:4/55: truncate d2/f4 4335887 0 2024-04-23T02:13:38.402 INFO:tasks.workunit.client.0.smithi071.stdout:4/56: dread - d2/d7/d10/f13 zero size 2024-04-23T02:13:38.402 INFO:tasks.workunit.client.0.smithi071.stdout:4/57: readlink d2/l11 0 2024-04-23T02:13:38.403 INFO:tasks.workunit.client.0.smithi071.stdout:5/70: dread d3/da/fd [0,4194304] 0 2024-04-23T02:13:38.403 INFO:tasks.workunit.client.0.smithi071.stdout:5/71: read f2 [1151233,53719] 0 2024-04-23T02:13:38.404 INFO:tasks.workunit.client.0.smithi071.stdout:4/58: rmdir d2 39 2024-04-23T02:13:38.405 INFO:tasks.workunit.client.0.smithi071.stdout:4/59: chown d2 171727 1 2024-04-23T02:13:38.407 INFO:tasks.workunit.client.0.smithi071.stdout:5/72: creat d3/da/d1a/d1c/f1e x:0 0 0 2024-04-23T02:13:38.408 INFO:tasks.workunit.client.0.smithi071.stdout:4/60: fsync d2/d7/d10/f13 0 2024-04-23T02:13:38.409 INFO:tasks.workunit.client.0.smithi071.stdout:4/61: mkdir d2/d7/d17 0 2024-04-23T02:13:38.410 INFO:tasks.workunit.client.0.smithi071.stdout:4/62: truncate d2/d7/fc 935457 0 2024-04-23T02:13:38.412 INFO:tasks.workunit.client.0.smithi071.stdout:4/63: mkdir d2/d7/d18 0 2024-04-23T02:13:38.414 INFO:tasks.workunit.client.0.smithi071.stdout:4/64: mknod d2/d7/d17/c19 0 2024-04-23T02:13:38.415 INFO:tasks.workunit.client.0.smithi071.stdout:4/65: dread - d2/d7/f14 zero size 2024-04-23T02:13:38.421 INFO:tasks.workunit.client.0.smithi071.stdout:4/66: dwrite d2/d7/ff [0,4194304] 0 2024-04-23T02:13:38.422 INFO:tasks.workunit.client.0.smithi071.stdout:4/67: fdatasync d2/d7/f14 0 2024-04-23T02:13:38.423 INFO:tasks.workunit.client.0.smithi071.stdout:4/68: truncate d2/d7/d10/f13 187211 0 2024-04-23T02:13:38.425 INFO:tasks.workunit.client.0.smithi071.stdout:4/69: getdents d2/d7/d17 0 2024-04-23T02:13:39.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:39 smithi192 ceph-mon[49662]: pgmap v16: 65 pgs: 65 active+clean; 219 MiB data, 854 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 434 op/s 2024-04-23T02:13:39.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:39 smithi071 ceph-mon[39837]: pgmap v16: 65 pgs: 65 active+clean; 219 MiB data, 854 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 434 op/s 2024-04-23T02:13:41.565 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:41 smithi192 ceph-mon[49662]: pgmap v17: 65 pgs: 65 active+clean; 306 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 16 MiB/s wr, 529 op/s 2024-04-23T02:13:41.653 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:41 smithi071 ceph-mon[39837]: pgmap v17: 65 pgs: 65 active+clean; 306 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 16 MiB/s wr, 529 op/s 2024-04-23T02:13:42.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:42 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:13:42.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:42 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:13:43.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:43 smithi192 ceph-mon[49662]: pgmap v18: 65 pgs: 65 active+clean; 306 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 14 MiB/s wr, 376 op/s 2024-04-23T02:13:43.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:43 smithi071 ceph-mon[39837]: pgmap v18: 65 pgs: 65 active+clean; 306 MiB data, 1002 MiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 14 MiB/s wr, 376 op/s 2024-04-23T02:13:45.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:45 smithi192 ceph-mon[49662]: pgmap v19: 65 pgs: 65 active+clean; 397 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 30 MiB/s wr, 412 op/s 2024-04-23T02:13:45.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:45 smithi071 ceph-mon[39837]: pgmap v19: 65 pgs: 65 active+clean; 397 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 30 MiB/s wr, 412 op/s 2024-04-23T02:13:47.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:47 smithi192 ceph-mon[49662]: pgmap v20: 65 pgs: 65 active+clean; 397 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 27 MiB/s wr, 177 op/s 2024-04-23T02:13:47.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:47 smithi071 ceph-mon[39837]: pgmap v20: 65 pgs: 65 active+clean; 397 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 27 MiB/s wr, 177 op/s 2024-04-23T02:13:48.188 INFO:tasks.workunit.client.1.smithi192.stdout:2/0: sync 2024-04-23T02:13:48.188 INFO:tasks.workunit.client.1.smithi192.stdout:2/1: dread - no filename 2024-04-23T02:13:48.188 INFO:tasks.workunit.client.1.smithi192.stdout:2/2: dread - no filename 2024-04-23T02:13:48.190 INFO:tasks.workunit.client.1.smithi192.stdout:2/3: getdents . 0 2024-04-23T02:13:48.190 INFO:tasks.workunit.client.1.smithi192.stdout:2/4: link - no file 2024-04-23T02:13:48.190 INFO:tasks.workunit.client.1.smithi192.stdout:2/5: truncate - no filename 2024-04-23T02:13:48.190 INFO:tasks.workunit.client.1.smithi192.stdout:2/6: unlink - no file 2024-04-23T02:13:48.190 INFO:tasks.workunit.client.1.smithi192.stdout:2/7: dwrite - no filename 2024-04-23T02:13:48.190 INFO:tasks.workunit.client.1.smithi192.stdout:2/8: dread - no filename 2024-04-23T02:13:48.190 INFO:tasks.workunit.client.1.smithi192.stdout:2/9: link - no file 2024-04-23T02:13:48.191 INFO:tasks.workunit.client.1.smithi192.stdout:2/10: creat f0 x:0 0 0 2024-04-23T02:13:48.192 INFO:tasks.workunit.client.1.smithi192.stdout:2/11: stat f0 0 2024-04-23T02:13:48.193 INFO:tasks.workunit.client.1.smithi192.stdout:2/12: mknod c1 0 2024-04-23T02:13:48.193 INFO:tasks.workunit.client.1.smithi192.stdout:2/13: chown f0 1491753 1 2024-04-23T02:13:48.194 INFO:tasks.workunit.client.1.smithi192.stdout:2/14: dread - f0 zero size 2024-04-23T02:13:48.194 INFO:tasks.workunit.client.1.smithi192.stdout:2/15: write f0 [470660,1364] 0 2024-04-23T02:13:48.196 INFO:tasks.workunit.client.1.smithi192.stdout:2/16: write f0 [1282437,19476] 0 2024-04-23T02:13:48.204 INFO:tasks.workunit.client.1.smithi192.stdout:2/17: dwrite f0 [0,4194304] 0 2024-04-23T02:13:48.209 INFO:tasks.workunit.client.1.smithi192.stdout:2/18: dread f0 [0,4194304] 0 2024-04-23T02:13:48.220 INFO:tasks.workunit.client.1.smithi192.stdout:2/19: fdatasync f0 0 2024-04-23T02:13:48.220 INFO:tasks.workunit.client.1.smithi192.stdout:2/20: rmdir - no directory 2024-04-23T02:13:48.223 INFO:tasks.workunit.client.1.smithi192.stdout:2/21: dread f0 [0,4194304] 0 2024-04-23T02:13:48.225 INFO:tasks.workunit.client.1.smithi192.stdout:2/22: mknod c2 0 2024-04-23T02:13:48.229 INFO:tasks.workunit.client.1.smithi192.stdout:2/23: dwrite f0 [0,4194304] 0 2024-04-23T02:13:48.232 INFO:tasks.workunit.client.1.smithi192.stdout:2/24: unlink c2 0 2024-04-23T02:13:48.233 INFO:tasks.workunit.client.1.smithi192.stdout:2/25: write f0 [5060938,16178] 0 2024-04-23T02:13:48.745 INFO:tasks.workunit.client.1.smithi192.stdout:3/5: sync 2024-04-23T02:13:48.745 INFO:tasks.workunit.client.1.smithi192.stdout:3/6: dwrite - no filename 2024-04-23T02:13:48.745 INFO:tasks.workunit.client.1.smithi192.stdout:3/7: readlink l0 0 2024-04-23T02:13:48.820 INFO:tasks.workunit.client.1.smithi192.stdout:9/2: sync 2024-04-23T02:13:48.820 INFO:tasks.workunit.client.1.smithi192.stdout:9/3: fdatasync - no filename 2024-04-23T02:13:48.820 INFO:tasks.workunit.client.1.smithi192.stdout:8/10: sync 2024-04-23T02:13:48.820 INFO:tasks.workunit.client.1.smithi192.stdout:0/11: sync 2024-04-23T02:13:48.820 INFO:tasks.workunit.client.1.smithi192.stdout:1/11: sync 2024-04-23T02:13:48.821 INFO:tasks.workunit.client.1.smithi192.stdout:1/12: creat f3 x:0 0 0 2024-04-23T02:13:48.822 INFO:tasks.workunit.client.1.smithi192.stdout:0/12: creat f3 x:0 0 0 2024-04-23T02:13:48.824 INFO:tasks.workunit.client.1.smithi192.stdout:1/13: creat f4 x:0 0 0 2024-04-23T02:13:48.824 INFO:tasks.workunit.client.1.smithi192.stdout:8/11: dread f0 [0,4194304] 0 2024-04-23T02:13:48.825 INFO:tasks.workunit.client.1.smithi192.stdout:9/4: creat d0/f1 x:0 0 0 2024-04-23T02:13:48.826 INFO:tasks.workunit.client.1.smithi192.stdout:1/14: chown f0 0 1 2024-04-23T02:13:48.826 INFO:tasks.workunit.client.1.smithi192.stdout:1/15: stat f0 0 2024-04-23T02:13:48.827 INFO:tasks.workunit.client.1.smithi192.stdout:9/5: dread - d0/f1 zero size 2024-04-23T02:13:48.828 INFO:tasks.workunit.client.1.smithi192.stdout:8/12: creat d1/f2 x:0 0 0 2024-04-23T02:13:48.829 INFO:tasks.workunit.client.1.smithi192.stdout:1/16: mknod c5 0 2024-04-23T02:13:48.830 INFO:tasks.workunit.client.1.smithi192.stdout:9/6: creat d0/f2 x:0 0 0 2024-04-23T02:13:48.830 INFO:tasks.workunit.client.1.smithi192.stdout:9/7: chown d0 1150 1 2024-04-23T02:13:48.832 INFO:tasks.workunit.client.1.smithi192.stdout:9/8: dread - d0/f2 zero size 2024-04-23T02:13:48.832 INFO:tasks.workunit.client.1.smithi192.stdout:8/13: creat d1/f3 x:0 0 0 2024-04-23T02:13:48.832 INFO:tasks.workunit.client.1.smithi192.stdout:0/13: dwrite f3 [0,4194304] 0 2024-04-23T02:13:48.833 INFO:tasks.workunit.client.1.smithi192.stdout:9/9: stat d0 0 2024-04-23T02:13:48.833 INFO:tasks.workunit.client.1.smithi192.stdout:0/14: chown f2 51074134 1 2024-04-23T02:13:48.834 INFO:tasks.workunit.client.1.smithi192.stdout:9/10: dread - d0/f1 zero size 2024-04-23T02:13:48.835 INFO:tasks.workunit.client.1.smithi192.stdout:1/17: symlink l6 0 2024-04-23T02:13:48.835 INFO:tasks.workunit.client.1.smithi192.stdout:0/15: write f3 [1364776,80837] 0 2024-04-23T02:13:48.835 INFO:tasks.workunit.client.1.smithi192.stdout:8/14: fsync f0 0 2024-04-23T02:13:48.836 INFO:tasks.workunit.client.1.smithi192.stdout:1/18: chown f1 577718588 1 2024-04-23T02:13:48.836 INFO:tasks.workunit.client.1.smithi192.stdout:0/16: read f3 [3086497,95874] 0 2024-04-23T02:13:48.837 INFO:tasks.workunit.client.1.smithi192.stdout:0/17: readlink l1 0 2024-04-23T02:13:48.840 INFO:tasks.workunit.client.1.smithi192.stdout:8/15: dread f0 [0,4194304] 0 2024-04-23T02:13:48.840 INFO:tasks.workunit.client.1.smithi192.stdout:8/16: stat d1/f2 0 2024-04-23T02:13:48.841 INFO:tasks.workunit.client.1.smithi192.stdout:8/17: read f0 [792187,11697] 0 2024-04-23T02:13:48.845 INFO:tasks.workunit.client.1.smithi192.stdout:1/19: truncate f0 464144 0 2024-04-23T02:13:48.851 INFO:tasks.workunit.client.1.smithi192.stdout:8/18: unlink f0 0 2024-04-23T02:13:48.851 INFO:tasks.workunit.client.1.smithi192.stdout:8/19: rmdir d1 39 2024-04-23T02:13:48.852 INFO:tasks.workunit.client.1.smithi192.stdout:1/20: link f4 f7 0 2024-04-23T02:13:48.852 INFO:tasks.workunit.client.1.smithi192.stdout:8/20: chown d1 757 1 2024-04-23T02:13:48.853 INFO:tasks.workunit.client.1.smithi192.stdout:0/18: getdents . 0 2024-04-23T02:13:48.858 INFO:tasks.workunit.client.1.smithi192.stdout:0/19: rename l1 to l4 0 2024-04-23T02:13:48.861 INFO:tasks.workunit.client.1.smithi192.stdout:0/20: creat f5 x:0 0 0 2024-04-23T02:13:48.861 INFO:tasks.workunit.client.1.smithi192.stdout:8/21: dwrite d1/f2 [0,4194304] 0 2024-04-23T02:13:48.861 INFO:tasks.workunit.client.1.smithi192.stdout:0/21: read - f5 zero size 2024-04-23T02:13:48.862 INFO:tasks.workunit.client.1.smithi192.stdout:8/22: write d1/f3 [833856,91343] 0 2024-04-23T02:13:48.862 INFO:tasks.workunit.client.1.smithi192.stdout:0/22: chown f3 40703 1 2024-04-23T02:13:48.873 INFO:tasks.workunit.client.1.smithi192.stdout:8/23: dread d1/f2 [0,4194304] 0 2024-04-23T02:13:48.874 INFO:tasks.workunit.client.1.smithi192.stdout:0/23: dwrite f5 [0,4194304] 0 2024-04-23T02:13:48.877 INFO:tasks.workunit.client.1.smithi192.stdout:0/24: readlink l4 0 2024-04-23T02:13:48.877 INFO:tasks.workunit.client.1.smithi192.stdout:0/25: rmdir - no directory 2024-04-23T02:13:48.877 INFO:tasks.workunit.client.1.smithi192.stdout:1/21: dwrite f0 [0,4194304] 0 2024-04-23T02:13:48.878 INFO:tasks.workunit.client.1.smithi192.stdout:0/26: truncate f5 5107412 0 2024-04-23T02:13:48.886 INFO:tasks.workunit.client.1.smithi192.stdout:8/24: link d1/f2 d1/f4 0 2024-04-23T02:13:48.887 INFO:tasks.workunit.client.1.smithi192.stdout:8/25: symlink d1/l5 0 2024-04-23T02:13:48.887 INFO:tasks.workunit.client.1.smithi192.stdout:8/26: creat d1/f6 x:0 0 0 2024-04-23T02:13:48.890 INFO:tasks.workunit.client.1.smithi192.stdout:8/27: symlink d1/l7 0 2024-04-23T02:13:48.891 INFO:tasks.workunit.client.1.smithi192.stdout:0/27: dwrite f3 [0,4194304] 0 2024-04-23T02:13:48.892 INFO:tasks.workunit.client.1.smithi192.stdout:8/28: write d1/f2 [897361,123921] 0 2024-04-23T02:13:48.896 INFO:tasks.workunit.client.1.smithi192.stdout:0/28: mkdir d6 0 2024-04-23T02:13:48.899 INFO:tasks.workunit.client.1.smithi192.stdout:3/8: sync 2024-04-23T02:13:48.899 INFO:tasks.workunit.client.1.smithi192.stdout:5/59: sync 2024-04-23T02:13:48.899 INFO:tasks.workunit.client.1.smithi192.stdout:6/32: sync 2024-04-23T02:13:48.899 INFO:tasks.workunit.client.1.smithi192.stdout:4/274: sync 2024-04-23T02:13:48.899 INFO:tasks.workunit.client.1.smithi192.stdout:2/26: sync 2024-04-23T02:13:48.900 INFO:tasks.workunit.client.1.smithi192.stdout:7/58: sync 2024-04-23T02:13:48.901 INFO:tasks.workunit.client.1.smithi192.stdout:3/9: mkdir d1 0 2024-04-23T02:13:48.905 INFO:tasks.workunit.client.1.smithi192.stdout:5/60: mkdir d2/db/d16 0 2024-04-23T02:13:48.907 INFO:tasks.workunit.client.1.smithi192.stdout:8/29: dwrite d1/f2 [0,4194304] 0 2024-04-23T02:13:48.907 INFO:tasks.workunit.client.1.smithi192.stdout:4/275: chown de/d15/d19/d3d/f32 1853 1 2024-04-23T02:13:48.908 INFO:tasks.workunit.client.1.smithi192.stdout:7/59: read d5/fb [6008587,80315] 0 2024-04-23T02:13:48.910 INFO:tasks.workunit.client.1.smithi192.stdout:3/10: creat d1/f2 x:0 0 0 2024-04-23T02:13:48.911 INFO:tasks.workunit.client.1.smithi192.stdout:1/22: write f4 [551668,113232] 0 2024-04-23T02:13:48.914 INFO:tasks.workunit.client.1.smithi192.stdout:0/29: dwrite f3 [0,4194304] 0 2024-04-23T02:13:48.918 INFO:tasks.workunit.client.1.smithi192.stdout:4/276: fsync de/d15/d19/d1a/f23 0 2024-04-23T02:13:48.919 INFO:tasks.workunit.client.1.smithi192.stdout:6/33: write d3/f6 [4695775,23176] 0 2024-04-23T02:13:48.919 INFO:tasks.workunit.client.1.smithi192.stdout:2/27: dread f0 [0,4194304] 0 2024-04-23T02:13:48.923 INFO:tasks.workunit.client.1.smithi192.stdout:4/277: chown de/d15/d19/d1a 8131 1 2024-04-23T02:13:48.928 INFO:tasks.workunit.client.1.smithi192.stdout:7/60: dwrite d5/f8 [0,4194304] 0 2024-04-23T02:13:48.929 INFO:tasks.workunit.client.1.smithi192.stdout:5/61: dwrite d2/fe [0,4194304] 0 2024-04-23T02:13:48.935 INFO:tasks.workunit.client.1.smithi192.stdout:8/30: fdatasync d1/f4 0 2024-04-23T02:13:48.942 INFO:tasks.workunit.client.1.smithi192.stdout:8/31: dread d1/f3 [0,4194304] 0 2024-04-23T02:13:48.943 INFO:tasks.workunit.client.1.smithi192.stdout:8/32: chown d1/f2 433084699 1 2024-04-23T02:13:48.943 INFO:tasks.workunit.client.1.smithi192.stdout:8/33: rename d1 to d1/d8 22 2024-04-23T02:13:48.943 INFO:tasks.workunit.client.1.smithi192.stdout:8/34: chown d1/f6 1040 1 2024-04-23T02:13:48.944 INFO:tasks.workunit.client.1.smithi192.stdout:8/35: chown d1/l7 13 1 2024-04-23T02:13:48.944 INFO:tasks.workunit.client.1.smithi192.stdout:8/36: truncate d1/f3 1259946 0 2024-04-23T02:13:48.945 INFO:tasks.workunit.client.1.smithi192.stdout:8/37: dread - d1/f6 zero size 2024-04-23T02:13:48.945 INFO:tasks.workunit.client.1.smithi192.stdout:8/38: chown d1/l5 793376460 1 2024-04-23T02:13:48.985 INFO:tasks.workunit.client.1.smithi192.stdout:6/34: creat d3/f9 x:0 0 0 2024-04-23T02:13:48.986 INFO:tasks.workunit.client.1.smithi192.stdout:2/28: write f0 [5004716,67066] 0 2024-04-23T02:13:48.987 INFO:tasks.workunit.client.1.smithi192.stdout:2/29: write f0 [1497498,105886] 0 2024-04-23T02:13:48.988 INFO:tasks.workunit.client.1.smithi192.stdout:1/23: truncate f4 583615 0 2024-04-23T02:13:48.991 INFO:tasks.workunit.client.1.smithi192.stdout:8/39: mkdir d1/d9 0 2024-04-23T02:13:48.992 INFO:tasks.workunit.client.1.smithi192.stdout:8/40: write d1/f6 [437882,88942] 0 2024-04-23T02:13:48.993 INFO:tasks.workunit.client.1.smithi192.stdout:3/11: getdents d1 0 2024-04-23T02:13:48.994 INFO:tasks.workunit.client.1.smithi192.stdout:3/12: write d1/f2 [788605,128612] 0 2024-04-23T02:13:48.995 INFO:tasks.workunit.client.1.smithi192.stdout:6/35: creat d3/fa x:0 0 0 2024-04-23T02:13:48.997 INFO:tasks.workunit.client.1.smithi192.stdout:4/278: creat de/d15/d53/f58 x:0 0 0 2024-04-23T02:13:48.999 INFO:tasks.workunit.client.1.smithi192.stdout:2/30: rename f0 to f3 0 2024-04-23T02:13:48.999 INFO:tasks.workunit.client.1.smithi192.stdout:4/279: stat de/d15/d19/d1a/f3a 0 2024-04-23T02:13:49.001 INFO:tasks.workunit.client.1.smithi192.stdout:1/24: rename l2 to l8 0 2024-04-23T02:13:49.003 INFO:tasks.workunit.client.1.smithi192.stdout:4/280: chown de/d15/d19/d3d/d49/f50 25 1 2024-04-23T02:13:49.003 INFO:tasks.workunit.client.1.smithi192.stdout:3/13: mkdir d1/d3 0 2024-04-23T02:13:49.003 INFO:tasks.workunit.client.1.smithi192.stdout:5/62: dwrite d2/db/f11 [0,4194304] 0 2024-04-23T02:13:49.005 INFO:tasks.workunit.client.1.smithi192.stdout:6/36: mknod d3/cb 0 2024-04-23T02:13:49.007 INFO:tasks.workunit.client.1.smithi192.stdout:6/37: chown d3/c4 41963496 1 2024-04-23T02:13:49.011 INFO:tasks.workunit.client.1.smithi192.stdout:2/31: creat f4 x:0 0 0 2024-04-23T02:13:49.014 INFO:tasks.workunit.client.1.smithi192.stdout:2/32: truncate f4 723947 0 2024-04-23T02:13:49.015 INFO:tasks.workunit.client.1.smithi192.stdout:8/41: truncate d1/f2 2119969 0 2024-04-23T02:13:49.015 INFO:tasks.workunit.client.1.smithi192.stdout:3/14: dwrite d1/f2 [0,4194304] 0 2024-04-23T02:13:49.015 INFO:tasks.workunit.client.1.smithi192.stdout:4/281: rmdir de/d15/d19/d3d/d49 39 2024-04-23T02:13:49.017 INFO:tasks.workunit.client.1.smithi192.stdout:6/38: unlink d3/c5 0 2024-04-23T02:13:49.018 INFO:tasks.workunit.client.1.smithi192.stdout:6/39: dread - d3/fa zero size 2024-04-23T02:13:49.018 INFO:tasks.workunit.client.1.smithi192.stdout:7/61: getdents d5 0 2024-04-23T02:13:49.019 INFO:tasks.workunit.client.1.smithi192.stdout:5/63: mknod d2/c17 0 2024-04-23T02:13:49.020 INFO:tasks.workunit.client.1.smithi192.stdout:7/62: write d5/f8 [187937,39793] 0 2024-04-23T02:13:49.020 INFO:tasks.workunit.client.1.smithi192.stdout:2/33: symlink l5 0 2024-04-23T02:13:49.021 INFO:tasks.workunit.client.1.smithi192.stdout:5/64: write d2/db/f11 [1288583,100572] 0 2024-04-23T02:13:49.021 INFO:tasks.workunit.client.1.smithi192.stdout:7/63: read d5/f8 [2000043,3632] 0 2024-04-23T02:13:49.025 INFO:tasks.workunit.client.1.smithi192.stdout:6/40: symlink d3/lc 0 2024-04-23T02:13:49.026 INFO:tasks.workunit.client.1.smithi192.stdout:6/41: fdatasync d3/fa 0 2024-04-23T02:13:49.027 INFO:tasks.workunit.client.1.smithi192.stdout:2/34: creat f6 x:0 0 0 2024-04-23T02:13:49.028 INFO:tasks.workunit.client.1.smithi192.stdout:6/42: truncate d3/fa 407056 0 2024-04-23T02:13:49.029 INFO:tasks.workunit.client.1.smithi192.stdout:8/42: creat d1/d9/fa x:0 0 0 2024-04-23T02:13:49.029 INFO:tasks.workunit.client.1.smithi192.stdout:2/35: chown f3 12299743 1 2024-04-23T02:13:49.029 INFO:tasks.workunit.client.1.smithi192.stdout:3/15: mkdir d1/d3/d4 0 2024-04-23T02:13:49.030 INFO:tasks.workunit.client.1.smithi192.stdout:5/65: creat d2/db/f18 x:0 0 0 2024-04-23T02:13:49.031 INFO:tasks.workunit.client.1.smithi192.stdout:7/64: mknod d5/c13 0 2024-04-23T02:13:49.033 INFO:tasks.workunit.client.1.smithi192.stdout:1/25: fsync f7 0 2024-04-23T02:13:49.035 INFO:tasks.workunit.client.1.smithi192.stdout:3/16: dread d1/f2 [0,4194304] 0 2024-04-23T02:13:49.036 INFO:tasks.workunit.client.1.smithi192.stdout:2/36: creat f7 x:0 0 0 2024-04-23T02:13:49.037 INFO:tasks.workunit.client.1.smithi192.stdout:9/11: sync 2024-04-23T02:13:49.038 INFO:tasks.workunit.client.1.smithi192.stdout:9/12: truncate d0/f2 421324 0 2024-04-23T02:13:49.039 INFO:tasks.workunit.client.1.smithi192.stdout:8/43: mknod d1/d9/cb 0 2024-04-23T02:13:49.040 INFO:tasks.workunit.client.1.smithi192.stdout:3/17: dread d1/f2 [0,4194304] 0 2024-04-23T02:13:49.042 INFO:tasks.workunit.client.1.smithi192.stdout:7/65: write d5/f8 [850460,39272] 0 2024-04-23T02:13:49.045 INFO:tasks.workunit.client.1.smithi192.stdout:2/37: mkdir d8 0 2024-04-23T02:13:49.048 INFO:tasks.workunit.client.1.smithi192.stdout:8/44: dwrite d1/d9/fa [0,4194304] 0 2024-04-23T02:13:49.048 INFO:tasks.workunit.client.1.smithi192.stdout:9/13: mkdir d0/d3 0 2024-04-23T02:13:49.049 INFO:tasks.workunit.client.1.smithi192.stdout:9/14: write d0/f1 [639605,67105] 0 2024-04-23T02:13:49.056 INFO:tasks.workunit.client.1.smithi192.stdout:5/66: creat d2/d6/d14/f19 x:0 0 0 2024-04-23T02:13:49.057 INFO:tasks.workunit.client.1.smithi192.stdout:3/18: creat d1/f5 x:0 0 0 2024-04-23T02:13:49.058 INFO:tasks.workunit.client.1.smithi192.stdout:3/19: stat d1 0 2024-04-23T02:13:49.058 INFO:tasks.workunit.client.1.smithi192.stdout:4/282: getdents de/d15/d19/d3d/d34/d4a 0 2024-04-23T02:13:49.058 INFO:tasks.workunit.client.1.smithi192.stdout:9/15: fsync d0/f1 0 2024-04-23T02:13:49.058 INFO:tasks.workunit.client.1.smithi192.stdout:9/16: stat d0 0 2024-04-23T02:13:49.058 INFO:tasks.workunit.client.1.smithi192.stdout:4/283: read de/d15/d19/d3d/f57 [2089094,58572] 0 2024-04-23T02:13:49.058 INFO:tasks.workunit.client.1.smithi192.stdout:6/43: link d3/lc d3/ld 0 2024-04-23T02:13:49.060 INFO:tasks.workunit.client.1.smithi192.stdout:6/44: fsync d3/fa 0 2024-04-23T02:13:49.060 INFO:tasks.workunit.client.1.smithi192.stdout:9/17: dread d0/f2 [0,4194304] 0 2024-04-23T02:13:49.060 INFO:tasks.workunit.client.1.smithi192.stdout:2/38: mknod d8/c9 0 2024-04-23T02:13:49.060 INFO:tasks.workunit.client.1.smithi192.stdout:3/20: write d1/f2 [26637,103813] 0 2024-04-23T02:13:49.062 INFO:tasks.workunit.client.1.smithi192.stdout:8/45: dread d1/f3 [0,4194304] 0 2024-04-23T02:13:49.063 INFO:tasks.workunit.client.1.smithi192.stdout:2/39: rmdir d8 39 2024-04-23T02:13:49.065 INFO:tasks.workunit.client.1.smithi192.stdout:9/18: rename d0/f2 to d0/f4 0 2024-04-23T02:13:49.070 INFO:tasks.workunit.client.1.smithi192.stdout:4/284: mkdir de/d15/d19/d3d/d49/d26/d59 0 2024-04-23T02:13:49.074 INFO:tasks.workunit.client.1.smithi192.stdout:8/46: fsync d1/f4 0 2024-04-23T02:13:49.077 INFO:tasks.workunit.client.1.smithi192.stdout:2/40: rename d8/c9 to d8/ca 0 2024-04-23T02:13:49.078 INFO:tasks.workunit.client.1.smithi192.stdout:6/45: dwrite d3/f9 [0,4194304] 0 2024-04-23T02:13:49.078 INFO:tasks.workunit.client.1.smithi192.stdout:5/67: dwrite d2/db/f13 [0,4194304] 0 2024-04-23T02:13:49.081 INFO:tasks.workunit.client.1.smithi192.stdout:4/285: truncate f4 8125587 0 2024-04-23T02:13:49.083 INFO:tasks.workunit.client.1.smithi192.stdout:9/19: truncate d0/f1 159181 0 2024-04-23T02:13:49.089 INFO:tasks.workunit.client.1.smithi192.stdout:2/41: dread f4 [0,4194304] 0 2024-04-23T02:13:49.090 INFO:tasks.workunit.client.1.smithi192.stdout:8/47: unlink d1/l5 0 2024-04-23T02:13:49.093 INFO:tasks.workunit.client.1.smithi192.stdout:2/42: write f6 [524339,19056] 0 2024-04-23T02:13:49.094 INFO:tasks.workunit.client.1.smithi192.stdout:7/66: dwrite d5/fb [4194304,4194304] 0 2024-04-23T02:13:49.094 INFO:tasks.workunit.client.1.smithi192.stdout:3/21: dwrite d1/f2 [4194304,4194304] 0 2024-04-23T02:13:49.094 INFO:tasks.workunit.client.1.smithi192.stdout:4/286: unlink de/d15/d19/d3d/d49/d26/l2b 0 2024-04-23T02:13:49.096 INFO:tasks.workunit.client.1.smithi192.stdout:3/22: dread - d1/f5 zero size 2024-04-23T02:13:49.097 INFO:tasks.workunit.client.1.smithi192.stdout:2/43: chown f6 1674 1 2024-04-23T02:13:49.098 INFO:tasks.workunit.client.1.smithi192.stdout:5/68: mknod d2/c1a 0 2024-04-23T02:13:49.098 INFO:tasks.workunit.client.1.smithi192.stdout:8/48: mknod d1/d9/cc 0 2024-04-23T02:13:49.102 INFO:tasks.workunit.client.1.smithi192.stdout:4/287: read de/d15/f36 [12410,42108] 0 2024-04-23T02:13:49.103 INFO:tasks.workunit.client.1.smithi192.stdout:3/23: mkdir d1/d3/d6 0 2024-04-23T02:13:49.106 INFO:tasks.workunit.client.1.smithi192.stdout:6/46: getdents d3 0 2024-04-23T02:13:49.106 INFO:tasks.workunit.client.1.smithi192.stdout:8/49: creat d1/fd x:0 0 0 2024-04-23T02:13:49.109 INFO:tasks.workunit.client.1.smithi192.stdout:6/47: rmdir d3 39 2024-04-23T02:13:49.112 INFO:tasks.workunit.client.1.smithi192.stdout:7/67: dread d5/fb [4194304,4194304] 0 2024-04-23T02:13:49.115 INFO:tasks.workunit.client.1.smithi192.stdout:2/44: dwrite f6 [0,4194304] 0 2024-04-23T02:13:49.115 INFO:tasks.workunit.client.1.smithi192.stdout:8/50: write d1/d9/fa [4870227,5622] 0 2024-04-23T02:13:49.118 INFO:tasks.workunit.client.1.smithi192.stdout:5/69: dwrite d2/db/f13 [0,4194304] 0 2024-04-23T02:13:49.119 INFO:tasks.workunit.client.1.smithi192.stdout:7/68: mknod d5/c14 0 2024-04-23T02:13:49.119 INFO:tasks.workunit.client.1.smithi192.stdout:6/48: chown d3/f6 42611305 1 2024-04-23T02:13:49.120 INFO:tasks.workunit.client.1.smithi192.stdout:5/70: fdatasync d2/d6/d14/f19 0 2024-04-23T02:13:49.121 INFO:tasks.workunit.client.1.smithi192.stdout:6/49: read f1 [5366347,112577] 0 2024-04-23T02:13:49.121 INFO:tasks.workunit.client.1.smithi192.stdout:4/288: rename de/d15/d19/d1a/l40 to de/d15/d19/d3d/d49/d26/l5a 0 2024-04-23T02:13:49.122 INFO:tasks.workunit.client.1.smithi192.stdout:8/51: creat d1/fe x:0 0 0 2024-04-23T02:13:49.123 INFO:tasks.workunit.client.1.smithi192.stdout:3/24: mkdir d1/d3/d6/d7 0 2024-04-23T02:13:49.127 INFO:tasks.workunit.client.1.smithi192.stdout:3/25: dread - d1/f5 zero size 2024-04-23T02:13:49.127 INFO:tasks.workunit.client.1.smithi192.stdout:2/45: creat d8/fb x:0 0 0 2024-04-23T02:13:49.129 INFO:tasks.workunit.client.1.smithi192.stdout:7/69: unlink d5/l9 0 2024-04-23T02:13:49.132 INFO:tasks.workunit.client.1.smithi192.stdout:3/26: fsync d1/f2 0 2024-04-23T02:13:49.136 INFO:tasks.workunit.client.1.smithi192.stdout:4/289: fsync de/d15/d19/d3d/f32 0 2024-04-23T02:13:49.137 INFO:tasks.workunit.client.1.smithi192.stdout:2/46: write d8/fb [238006,73442] 0 2024-04-23T02:13:49.137 INFO:tasks.workunit.client.1.smithi192.stdout:9/20: write d0/f1 [193794,30473] 0 2024-04-23T02:13:49.139 INFO:tasks.workunit.client.1.smithi192.stdout:8/52: mkdir d1/df 0 2024-04-23T02:13:49.141 INFO:tasks.workunit.client.1.smithi192.stdout:2/47: chown c1 4181041 1 2024-04-23T02:13:49.141 INFO:tasks.workunit.client.1.smithi192.stdout:8/53: fsync d1/fe 0 2024-04-23T02:13:49.142 INFO:tasks.workunit.client.1.smithi192.stdout:6/50: dwrite d3/f9 [4194304,4194304] 0 2024-04-23T02:13:49.143 INFO:tasks.workunit.client.1.smithi192.stdout:7/70: rename d5/fb to d5/f15 0 2024-04-23T02:13:49.145 INFO:tasks.workunit.client.1.smithi192.stdout:9/21: mknod d0/c5 0 2024-04-23T02:13:49.146 INFO:tasks.workunit.client.1.smithi192.stdout:5/71: dwrite d2/d6/d14/f19 [0,4194304] 0 2024-04-23T02:13:49.152 INFO:tasks.workunit.client.1.smithi192.stdout:4/290: rmdir de/d15/d19/d3d/d49/d26 39 2024-04-23T02:13:49.154 INFO:tasks.workunit.client.1.smithi192.stdout:2/48: symlink d8/lc 0 2024-04-23T02:13:49.154 INFO:tasks.workunit.client.1.smithi192.stdout:2/49: readlink l5 0 2024-04-23T02:13:49.155 INFO:tasks.workunit.client.1.smithi192.stdout:5/72: unlink d2/d6/d14/f19 0 2024-04-23T02:13:49.157 INFO:tasks.workunit.client.1.smithi192.stdout:6/51: write d3/f9 [9230822,119665] 0 2024-04-23T02:13:49.158 INFO:tasks.workunit.client.1.smithi192.stdout:3/27: rmdir d1/d3/d4 0 2024-04-23T02:13:49.162 INFO:tasks.workunit.client.1.smithi192.stdout:7/71: dwrite d5/f8 [0,4194304] 0 2024-04-23T02:13:49.162 INFO:tasks.workunit.client.1.smithi192.stdout:5/73: rmdir d2/db 39 2024-04-23T02:13:49.163 INFO:tasks.workunit.client.1.smithi192.stdout:2/50: dread d8/fb [0,4194304] 0 2024-04-23T02:13:49.163 INFO:tasks.workunit.client.1.smithi192.stdout:2/51: readlink l5 0 2024-04-23T02:13:49.164 INFO:tasks.workunit.client.1.smithi192.stdout:4/291: truncate de/f21 953038 0 2024-04-23T02:13:49.164 INFO:tasks.workunit.client.1.smithi192.stdout:6/52: symlink d3/le 0 2024-04-23T02:13:49.167 INFO:tasks.workunit.client.1.smithi192.stdout:3/28: creat d1/d3/f8 x:0 0 0 2024-04-23T02:13:49.173 INFO:tasks.workunit.client.1.smithi192.stdout:8/54: write d1/f2 [2621416,65398] 0 2024-04-23T02:13:49.173 INFO:tasks.workunit.client.1.smithi192.stdout:7/72: dwrite d5/f8 [0,4194304] 0 2024-04-23T02:13:49.173 INFO:tasks.workunit.client.1.smithi192.stdout:5/74: mkdir d2/db/d16/d1b 0 2024-04-23T02:13:49.176 INFO:tasks.workunit.client.1.smithi192.stdout:7/73: write d5/f8 [340924,88783] 0 2024-04-23T02:13:49.177 INFO:tasks.workunit.client.1.smithi192.stdout:8/55: fdatasync d1/f4 0 2024-04-23T02:13:49.184 INFO:tasks.workunit.client.1.smithi192.stdout:9/22: truncate d0/f1 172985 0 2024-04-23T02:13:49.184 INFO:tasks.workunit.client.1.smithi192.stdout:3/29: symlink d1/d3/d6/d7/l9 0 2024-04-23T02:13:49.188 INFO:tasks.workunit.client.1.smithi192.stdout:3/30: unlink l0 0 2024-04-23T02:13:49.190 INFO:tasks.workunit.client.1.smithi192.stdout:8/56: dwrite d1/d9/fa [0,4194304] 0 2024-04-23T02:13:49.192 INFO:tasks.workunit.client.1.smithi192.stdout:9/23: symlink d0/d3/l6 0 2024-04-23T02:13:49.192 INFO:tasks.workunit.client.1.smithi192.stdout:3/31: write d1/f5 [606022,70414] 0 2024-04-23T02:13:49.200 INFO:tasks.workunit.client.1.smithi192.stdout:2/52: dwrite f4 [0,4194304] 0 2024-04-23T02:13:49.200 INFO:tasks.workunit.client.1.smithi192.stdout:5/75: write d2/d6/f12 [884859,74685] 0 2024-04-23T02:13:49.200 INFO:tasks.workunit.client.1.smithi192.stdout:3/32: dread - d1/d3/f8 zero size 2024-04-23T02:13:49.200 INFO:tasks.workunit.client.1.smithi192.stdout:2/53: dread - f7 zero size 2024-04-23T02:13:49.200 INFO:tasks.workunit.client.1.smithi192.stdout:3/33: readlink d1/d3/d6/d7/l9 0 2024-04-23T02:13:49.200 INFO:tasks.workunit.client.1.smithi192.stdout:9/24: rename d0/c5 to d0/c7 0 2024-04-23T02:13:49.203 INFO:tasks.workunit.client.1.smithi192.stdout:8/57: rename d1/f4 to d1/f10 0 2024-04-23T02:13:49.204 INFO:tasks.workunit.client.1.smithi192.stdout:7/74: truncate d5/f15 1187473 0 2024-04-23T02:13:49.204 INFO:tasks.workunit.client.1.smithi192.stdout:8/58: dread - d1/fd zero size 2024-04-23T02:13:49.205 INFO:tasks.workunit.client.1.smithi192.stdout:7/75: chown d5/le 77 1 2024-04-23T02:13:49.205 INFO:tasks.workunit.client.1.smithi192.stdout:2/54: mknod d8/cd 0 2024-04-23T02:13:49.207 INFO:tasks.workunit.client.1.smithi192.stdout:3/34: unlink d1/d3/d6/d7/l9 0 2024-04-23T02:13:49.207 INFO:tasks.workunit.client.1.smithi192.stdout:5/76: dread f0 [0,4194304] 0 2024-04-23T02:13:49.213 INFO:tasks.workunit.client.1.smithi192.stdout:3/35: dread d1/f5 [0,4194304] 0 2024-04-23T02:13:49.216 INFO:tasks.workunit.client.1.smithi192.stdout:3/36: mknod d1/d3/ca 0 2024-04-23T02:13:49.218 INFO:tasks.workunit.client.1.smithi192.stdout:9/25: dwrite d0/f4 [0,4194304] 0 2024-04-23T02:13:49.220 INFO:tasks.workunit.client.1.smithi192.stdout:3/37: fdatasync d1/f5 0 2024-04-23T02:13:49.220 INFO:tasks.workunit.client.1.smithi192.stdout:3/38: readlink - no filename 2024-04-23T02:13:49.220 INFO:tasks.workunit.client.1.smithi192.stdout:2/55: write d8/fb [303996,69891] 0 2024-04-23T02:13:49.222 INFO:tasks.workunit.client.1.smithi192.stdout:3/39: rename d1/d3/d6 to d1/d3/d6/db 22 2024-04-23T02:13:49.222 INFO:tasks.workunit.client.1.smithi192.stdout:9/26: chown d0/f1 0 1 2024-04-23T02:13:49.222 INFO:tasks.workunit.client.1.smithi192.stdout:3/40: fdatasync d1/f5 0 2024-04-23T02:13:49.222 INFO:tasks.workunit.client.1.smithi192.stdout:2/56: unlink d8/cd 0 2024-04-23T02:13:49.223 INFO:tasks.workunit.client.1.smithi192.stdout:3/41: stat d1/d3 0 2024-04-23T02:13:49.224 INFO:tasks.workunit.client.1.smithi192.stdout:5/77: write d2/f15 [4317197,31248] 0 2024-04-23T02:13:49.225 INFO:tasks.workunit.client.1.smithi192.stdout:3/42: write d1/d3/f8 [97577,128324] 0 2024-04-23T02:13:49.227 INFO:tasks.workunit.client.1.smithi192.stdout:9/27: rename d0/f4 to d0/d3/f8 0 2024-04-23T02:13:49.228 INFO:tasks.workunit.client.1.smithi192.stdout:7/76: write d5/f15 [1770386,44082] 0 2024-04-23T02:13:49.228 INFO:tasks.workunit.client.1.smithi192.stdout:2/57: symlink d8/le 0 2024-04-23T02:13:49.231 INFO:tasks.workunit.client.1.smithi192.stdout:9/28: mkdir d0/d9 0 2024-04-23T02:13:49.232 INFO:tasks.workunit.client.1.smithi192.stdout:2/58: mknod d8/cf 0 2024-04-23T02:13:49.232 INFO:tasks.workunit.client.1.smithi192.stdout:2/59: read - f7 zero size 2024-04-23T02:13:49.236 INFO:tasks.workunit.client.1.smithi192.stdout:3/43: truncate d1/f5 236829 0 2024-04-23T02:13:49.237 INFO:tasks.workunit.client.1.smithi192.stdout:9/29: creat d0/d3/fa x:0 0 0 2024-04-23T02:13:49.239 INFO:tasks.workunit.client.1.smithi192.stdout:2/60: creat d8/f10 x:0 0 0 2024-04-23T02:13:49.239 INFO:tasks.workunit.client.1.smithi192.stdout:2/61: write f7 [954102,113370] 0 2024-04-23T02:13:49.240 INFO:tasks.workunit.client.1.smithi192.stdout:2/62: readlink d8/lc 0 2024-04-23T02:13:49.241 INFO:tasks.workunit.client.1.smithi192.stdout:5/78: rename d2/lf to d2/l1c 0 2024-04-23T02:13:49.241 INFO:tasks.workunit.client.1.smithi192.stdout:2/63: chown d8/le 36 1 2024-04-23T02:13:49.242 INFO:tasks.workunit.client.1.smithi192.stdout:3/44: rename d1/d3/ca to d1/d3/d6/cc 0 2024-04-23T02:13:49.242 INFO:tasks.workunit.client.1.smithi192.stdout:7/77: link d5/le d5/l16 0 2024-04-23T02:13:49.243 INFO:tasks.workunit.client.1.smithi192.stdout:5/79: symlink d2/d6/l1d 0 2024-04-23T02:13:49.244 INFO:tasks.workunit.client.1.smithi192.stdout:2/64: mknod d8/c11 0 2024-04-23T02:13:49.245 INFO:tasks.workunit.client.1.smithi192.stdout:9/30: creat d0/d9/fb x:0 0 0 2024-04-23T02:13:49.247 INFO:tasks.workunit.client.1.smithi192.stdout:9/31: truncate d0/d9/fb 897028 0 2024-04-23T02:13:49.247 INFO:tasks.workunit.client.1.smithi192.stdout:3/45: unlink d1/d3/f8 0 2024-04-23T02:13:49.248 INFO:tasks.workunit.client.1.smithi192.stdout:5/80: dread d2/db/f13 [0,4194304] 0 2024-04-23T02:13:49.248 INFO:tasks.workunit.client.1.smithi192.stdout:7/78: mkdir d5/d17 0 2024-04-23T02:13:49.248 INFO:tasks.workunit.client.1.smithi192.stdout:3/46: read d1/f2 [1018482,11425] 0 2024-04-23T02:13:49.249 INFO:tasks.workunit.client.1.smithi192.stdout:7/79: write d5/f15 [2586929,116858] 0 2024-04-23T02:13:49.250 INFO:tasks.workunit.client.1.smithi192.stdout:2/65: unlink f6 0 2024-04-23T02:13:49.253 INFO:tasks.workunit.client.1.smithi192.stdout:9/32: creat d0/fc x:0 0 0 2024-04-23T02:13:49.257 INFO:tasks.workunit.client.1.smithi192.stdout:2/66: dread f7 [0,4194304] 0 2024-04-23T02:13:49.257 INFO:tasks.workunit.client.1.smithi192.stdout:9/33: unlink d0/c7 0 2024-04-23T02:13:49.258 INFO:tasks.workunit.client.1.smithi192.stdout:9/34: readlink d0/d3/l6 0 2024-04-23T02:13:49.327 INFO:tasks.workunit.client.1.smithi192.stdout:0/30: sync 2024-04-23T02:13:49.329 INFO:tasks.workunit.client.1.smithi192.stdout:3/47: dwrite d1/f2 [0,4194304] 0 2024-04-23T02:13:49.329 INFO:tasks.workunit.client.1.smithi192.stdout:5/81: dread d2/f5 [0,4194304] 0 2024-04-23T02:13:49.330 INFO:tasks.workunit.client.1.smithi192.stdout:7/80: dwrite d5/f8 [0,4194304] 0 2024-04-23T02:13:49.331 INFO:tasks.workunit.client.1.smithi192.stdout:0/31: read f3 [557569,59047] 0 2024-04-23T02:13:49.337 INFO:tasks.workunit.client.1.smithi192.stdout:9/35: mknod d0/d9/cd 0 2024-04-23T02:13:49.339 INFO:tasks.workunit.client.1.smithi192.stdout:7/81: mkdir d5/d18 0 2024-04-23T02:13:49.340 INFO:tasks.workunit.client.1.smithi192.stdout:2/67: getdents d8 0 2024-04-23T02:13:49.343 INFO:tasks.workunit.client.1.smithi192.stdout:9/36: dread d0/d3/f8 [0,4194304] 0 2024-04-23T02:13:49.346 INFO:tasks.workunit.client.1.smithi192.stdout:2/68: creat d8/f12 x:0 0 0 2024-04-23T02:13:49.347 INFO:tasks.workunit.client.1.smithi192.stdout:0/32: creat d6/f7 x:0 0 0 2024-04-23T02:13:49.348 INFO:tasks.workunit.client.1.smithi192.stdout:0/33: chown f5 224 1 2024-04-23T02:13:49.349 INFO:tasks.workunit.client.1.smithi192.stdout:7/82: rename d5/lc to d5/d18/l19 0 2024-04-23T02:13:49.350 INFO:tasks.workunit.client.1.smithi192.stdout:9/37: mknod d0/ce 0 2024-04-23T02:13:49.350 INFO:tasks.workunit.client.1.smithi192.stdout:9/38: chown d0/d9 106 1 2024-04-23T02:13:49.352 INFO:tasks.workunit.client.1.smithi192.stdout:3/48: fdatasync d1/f2 0 2024-04-23T02:13:49.352 INFO:tasks.workunit.client.1.smithi192.stdout:2/69: symlink d8/l13 0 2024-04-23T02:13:49.353 INFO:tasks.workunit.client.1.smithi192.stdout:2/70: chown f7 4261875 1 2024-04-23T02:13:49.354 INFO:tasks.workunit.client.1.smithi192.stdout:0/34: creat d6/f8 x:0 0 0 2024-04-23T02:13:49.354 INFO:tasks.workunit.client.1.smithi192.stdout:7/83: dread d5/f15 [0,4194304] 0 2024-04-23T02:13:49.357 INFO:tasks.workunit.client.1.smithi192.stdout:3/49: mkdir d1/d3/dd 0 2024-04-23T02:13:49.358 INFO:tasks.workunit.client.1.smithi192.stdout:2/71: mkdir d8/d14 0 2024-04-23T02:13:49.359 INFO:tasks.workunit.client.1.smithi192.stdout:0/35: dread f5 [0,4194304] 0 2024-04-23T02:13:49.359 INFO:tasks.workunit.client.1.smithi192.stdout:2/72: write d8/f10 [87875,5358] 0 2024-04-23T02:13:49.361 INFO:tasks.workunit.client.1.smithi192.stdout:7/84: creat d5/d18/f1a x:0 0 0 2024-04-23T02:13:49.362 INFO:tasks.workunit.client.1.smithi192.stdout:7/85: chown d5/d17 71 1 2024-04-23T02:13:49.362 INFO:tasks.workunit.client.1.smithi192.stdout:5/82: dwrite d2/db/f13 [0,4194304] 0 2024-04-23T02:13:49.364 INFO:tasks.workunit.client.1.smithi192.stdout:0/36: symlink d6/l9 0 2024-04-23T02:13:49.375 INFO:tasks.workunit.client.1.smithi192.stdout:2/73: rmdir d8 39 2024-04-23T02:13:49.376 INFO:tasks.workunit.client.1.smithi192.stdout:5/83: link d2/la d2/db/l1e 0 2024-04-23T02:13:49.377 INFO:tasks.workunit.client.1.smithi192.stdout:7/86: dwrite d5/f15 [0,4194304] 0 2024-04-23T02:13:49.382 INFO:tasks.workunit.client.1.smithi192.stdout:5/84: rename d2/f10 to d2/db/d16/d1b/f1f 0 2024-04-23T02:13:49.384 INFO:tasks.workunit.client.1.smithi192.stdout:7/87: unlink d5/d18/f1a 0 2024-04-23T02:13:49.388 INFO:tasks.workunit.client.1.smithi192.stdout:7/88: write d5/f8 [2603430,13187] 0 2024-04-23T02:13:49.388 INFO:tasks.workunit.client.1.smithi192.stdout:0/37: getdents d6 0 2024-04-23T02:13:49.388 INFO:tasks.workunit.client.1.smithi192.stdout:2/74: dwrite f7 [0,4194304] 0 2024-04-23T02:13:49.396 INFO:tasks.workunit.client.1.smithi192.stdout:5/85: creat d2/db/f20 x:0 0 0 2024-04-23T02:13:49.396 INFO:tasks.workunit.client.1.smithi192.stdout:3/50: write d1/f5 [685230,10493] 0 2024-04-23T02:13:49.397 INFO:tasks.workunit.client.1.smithi192.stdout:5/86: write d2/db/f18 [285049,20550] 0 2024-04-23T02:13:49.397 INFO:tasks.workunit.client.1.smithi192.stdout:0/38: truncate f5 317322 0 2024-04-23T02:13:49.398 INFO:tasks.workunit.client.1.smithi192.stdout:7/89: dread d5/f8 [0,4194304] 0 2024-04-23T02:13:49.401 INFO:tasks.workunit.client.1.smithi192.stdout:3/51: creat d1/d3/d6/d7/fe x:0 0 0 2024-04-23T02:13:49.401 INFO:tasks.workunit.client.1.smithi192.stdout:5/87: read d2/db/f18 [8906,42660] 0 2024-04-23T02:13:49.402 INFO:tasks.workunit.client.1.smithi192.stdout:3/52: chown d1/d3/d6/d7 125416354 1 2024-04-23T02:13:49.405 INFO:tasks.workunit.client.1.smithi192.stdout:7/90: rename d5/c10 to d5/c1b 0 2024-04-23T02:13:49.405 INFO:tasks.workunit.client.1.smithi192.stdout:7/91: truncate d5/f8 4334603 0 2024-04-23T02:13:49.408 INFO:tasks.workunit.client.1.smithi192.stdout:2/75: truncate f3 2668645 0 2024-04-23T02:13:49.409 INFO:tasks.workunit.client.1.smithi192.stdout:3/53: mknod d1/d3/d6/d7/cf 0 2024-04-23T02:13:49.411 INFO:tasks.workunit.client.1.smithi192.stdout:7/92: unlink d5/lf 0 2024-04-23T02:13:49.411 INFO:tasks.workunit.client.1.smithi192.stdout:0/39: write f2 [2916035,46068] 0 2024-04-23T02:13:49.413 INFO:tasks.workunit.client.1.smithi192.stdout:0/40: chown d6/l9 79348 1 2024-04-23T02:13:49.415 INFO:tasks.workunit.client.1.smithi192.stdout:0/41: symlink d6/la 0 2024-04-23T02:13:49.418 INFO:tasks.workunit.client.1.smithi192.stdout:2/76: dwrite f7 [0,4194304] 0 2024-04-23T02:13:49.421 INFO:tasks.workunit.client.1.smithi192.stdout:3/54: link d1/f2 d1/d3/d6/f10 0 2024-04-23T02:13:49.424 INFO:tasks.workunit.client.1.smithi192.stdout:7/93: mkdir d5/d17/d1c 0 2024-04-23T02:13:49.425 INFO:tasks.workunit.client.1.smithi192.stdout:5/88: dwrite d2/fd [0,4194304] 0 2024-04-23T02:13:49.427 INFO:tasks.workunit.client.1.smithi192.stdout:5/89: fsync d2/db/f11 0 2024-04-23T02:13:49.428 INFO:tasks.workunit.client.1.smithi192.stdout:0/42: dread f2 [0,4194304] 0 2024-04-23T02:13:49.431 INFO:tasks.workunit.client.1.smithi192.stdout:3/55: creat d1/d3/d6/d7/f11 x:0 0 0 2024-04-23T02:13:49.431 INFO:tasks.workunit.client.1.smithi192.stdout:7/94: stat d5/le 0 2024-04-23T02:13:49.432 INFO:tasks.workunit.client.1.smithi192.stdout:2/77: rmdir d8 39 2024-04-23T02:13:49.432 INFO:tasks.workunit.client.1.smithi192.stdout:7/95: chown d5/c13 545196596 1 2024-04-23T02:13:49.433 INFO:tasks.workunit.client.1.smithi192.stdout:0/43: rmdir d6 39 2024-04-23T02:13:49.435 INFO:tasks.workunit.client.1.smithi192.stdout:3/56: symlink d1/d3/d6/l12 0 2024-04-23T02:13:49.437 INFO:tasks.workunit.client.1.smithi192.stdout:2/78: dread f4 [0,4194304] 0 2024-04-23T02:13:49.440 INFO:tasks.workunit.client.1.smithi192.stdout:5/90: fdatasync d2/f5 0 2024-04-23T02:13:49.441 INFO:tasks.workunit.client.1.smithi192.stdout:0/44: fsync f3 0 2024-04-23T02:13:49.442 INFO:tasks.workunit.client.1.smithi192.stdout:5/91: chown d2/db/f13 58 1 2024-04-23T02:13:49.443 INFO:tasks.workunit.client.1.smithi192.stdout:3/57: dread d1/d3/d6/f10 [4194304,4194304] 0 2024-04-23T02:13:49.447 INFO:tasks.workunit.client.1.smithi192.stdout:7/96: dwrite d5/f8 [0,4194304] 0 2024-04-23T02:13:49.449 INFO:tasks.workunit.client.1.smithi192.stdout:2/79: rmdir d8 39 2024-04-23T02:13:49.450 INFO:tasks.workunit.client.1.smithi192.stdout:3/58: dread d1/f2 [4194304,4194304] 0 2024-04-23T02:13:49.452 INFO:tasks.workunit.client.1.smithi192.stdout:0/45: symlink d6/lb 0 2024-04-23T02:13:49.452 INFO:tasks.workunit.client.1.smithi192.stdout:0/46: chown f2 699896922 1 2024-04-23T02:13:49.454 INFO:tasks.workunit.client.1.smithi192.stdout:2/80: chown d8/ca 9790391 1 2024-04-23T02:13:49.455 INFO:tasks.workunit.client.1.smithi192.stdout:3/59: rmdir d1 39 2024-04-23T02:13:49.458 INFO:tasks.workunit.client.1.smithi192.stdout:2/81: rename d8/f10 to d8/d14/f15 0 2024-04-23T02:13:49.462 INFO:tasks.workunit.client.1.smithi192.stdout:2/82: dread - d8/f12 zero size 2024-04-23T02:13:49.462 INFO:tasks.workunit.client.1.smithi192.stdout:2/83: fdatasync d8/fb 0 2024-04-23T02:13:49.462 INFO:tasks.workunit.client.1.smithi192.stdout:0/47: truncate f5 273206 0 2024-04-23T02:13:49.463 INFO:tasks.workunit.client.1.smithi192.stdout:2/84: write f7 [1907508,129205] 0 2024-04-23T02:13:49.463 INFO:tasks.workunit.client.1.smithi192.stdout:7/97: fdatasync d5/f8 0 2024-04-23T02:13:49.464 INFO:tasks.workunit.client.1.smithi192.stdout:3/60: creat d1/d3/d6/d7/f13 x:0 0 0 2024-04-23T02:13:49.467 INFO:tasks.workunit.client.1.smithi192.stdout:3/61: unlink d1/d3/d6/d7/fe 0 2024-04-23T02:13:49.467 INFO:tasks.workunit.client.1.smithi192.stdout:7/98: dread d5/f15 [0,4194304] 0 2024-04-23T02:13:49.467 INFO:tasks.workunit.client.1.smithi192.stdout:3/62: read d1/f2 [7018438,62185] 0 2024-04-23T02:13:49.468 INFO:tasks.workunit.client.1.smithi192.stdout:7/99: chown d5/f8 33 1 2024-04-23T02:13:49.469 INFO:tasks.workunit.client.1.smithi192.stdout:0/48: link f2 d6/fc 0 2024-04-23T02:13:49.469 INFO:tasks.workunit.client.1.smithi192.stdout:3/63: dread - d1/d3/d6/d7/f11 zero size 2024-04-23T02:13:49.469 INFO:tasks.workunit.client.1.smithi192.stdout:5/92: truncate d2/fe 3868334 0 2024-04-23T02:13:49.470 INFO:tasks.workunit.client.1.smithi192.stdout:5/93: stat f0 0 2024-04-23T02:13:49.470 INFO:tasks.workunit.client.1.smithi192.stdout:3/64: readlink d1/d3/d6/l12 0 2024-04-23T02:13:49.472 INFO:tasks.workunit.client.1.smithi192.stdout:7/100: write d5/f15 [692870,39765] 0 2024-04-23T02:13:49.474 INFO:tasks.workunit.client.1.smithi192.stdout:0/49: rename f2 to d6/fd 0 2024-04-23T02:13:49.475 INFO:tasks.workunit.client.1.smithi192.stdout:5/94: mknod d2/d6/d14/c21 0 2024-04-23T02:13:49.477 INFO:tasks.workunit.client.1.smithi192.stdout:7/101: creat d5/d18/f1d x:0 0 0 2024-04-23T02:13:49.477 INFO:tasks.workunit.client.1.smithi192.stdout:7/102: read d5/f15 [301988,87476] 0 2024-04-23T02:13:49.477 INFO:tasks.workunit.client.1.smithi192.stdout:0/50: fsync f3 0 2024-04-23T02:13:49.478 INFO:tasks.workunit.client.1.smithi192.stdout:7/103: fdatasync d5/d18/f1d 0 2024-04-23T02:13:49.479 INFO:tasks.workunit.client.1.smithi192.stdout:5/95: symlink d2/db/d16/l22 0 2024-04-23T02:13:49.483 INFO:tasks.workunit.client.1.smithi192.stdout:0/51: rename d6/f8 to d6/fe 0 2024-04-23T02:13:49.483 INFO:tasks.workunit.client.1.smithi192.stdout:7/104: dread d5/f8 [0,4194304] 0 2024-04-23T02:13:49.484 INFO:tasks.workunit.client.1.smithi192.stdout:5/96: dread d2/fd [0,4194304] 0 2024-04-23T02:13:49.485 INFO:tasks.workunit.client.1.smithi192.stdout:5/97: fsync d2/db/f13 0 2024-04-23T02:13:49.487 INFO:tasks.workunit.client.1.smithi192.stdout:5/98: fdatasync d2/db/f13 0 2024-04-23T02:13:49.488 INFO:tasks.workunit.client.1.smithi192.stdout:7/105: readlink l4 0 2024-04-23T02:13:49.491 INFO:tasks.workunit.client.1.smithi192.stdout:5/99: dread d2/db/d16/d1b/f1f [0,4194304] 0 2024-04-23T02:13:49.492 INFO:tasks.workunit.client.1.smithi192.stdout:5/100: readlink d2/d6/l1d 0 2024-04-23T02:13:49.492 INFO:tasks.workunit.client.1.smithi192.stdout:0/52: write f3 [2002735,117315] 0 2024-04-23T02:13:49.493 INFO:tasks.workunit.client.1.smithi192.stdout:7/106: read d5/f15 [3071250,78363] 0 2024-04-23T02:13:49.494 INFO:tasks.workunit.client.1.smithi192.stdout:7/107: fsync d5/d18/f1d 0 2024-04-23T02:13:49.495 INFO:tasks.workunit.client.1.smithi192.stdout:5/101: write d2/db/f18 [1058471,50936] 0 2024-04-23T02:13:49.498 INFO:tasks.workunit.client.1.smithi192.stdout:7/108: rename d5/cd to d5/d17/c1e 0 2024-04-23T02:13:49.499 INFO:tasks.workunit.client.1.smithi192.stdout:1/26: sync 2024-04-23T02:13:49.499 INFO:tasks.workunit.client.1.smithi192.stdout:4/292: sync 2024-04-23T02:13:49.499 INFO:tasks.workunit.client.1.smithi192.stdout:1/27: stat l6 0 2024-04-23T02:13:49.502 INFO:tasks.workunit.client.1.smithi192.stdout:0/53: dread f5 [0,4194304] 0 2024-04-23T02:13:49.504 INFO:tasks.workunit.client.1.smithi192.stdout:1/28: dread f1 [0,4194304] 0 2024-04-23T02:13:49.505 INFO:tasks.workunit.client.1.smithi192.stdout:0/54: mkdir d6/df 0 2024-04-23T02:13:49.505 INFO:tasks.workunit.client.1.smithi192.stdout:5/102: dwrite d2/db/f20 [0,4194304] 0 2024-04-23T02:13:49.506 INFO:tasks.workunit.client.1.smithi192.stdout:1/29: chown f7 4079 1 2024-04-23T02:13:49.508 INFO:tasks.workunit.client.1.smithi192.stdout:4/293: rename de/d15/d19/c30 to de/c5b 0 2024-04-23T02:13:49.508 INFO:tasks.workunit.client.1.smithi192.stdout:7/109: dread d5/f8 [0,4194304] 0 2024-04-23T02:13:49.510 INFO:tasks.workunit.client.1.smithi192.stdout:7/110: dread - d5/d18/f1d zero size 2024-04-23T02:13:49.511 INFO:tasks.workunit.client.1.smithi192.stdout:0/55: rename d6/fd to d6/f10 0 2024-04-23T02:13:49.512 INFO:tasks.workunit.client.1.smithi192.stdout:5/103: mknod d2/db/d16/d1b/c23 0 2024-04-23T02:13:49.513 INFO:tasks.workunit.client.1.smithi192.stdout:0/56: dread - d6/fe zero size 2024-04-23T02:13:49.514 INFO:tasks.workunit.client.1.smithi192.stdout:4/294: creat de/d15/d2a/f5c x:0 0 0 2024-04-23T02:13:49.515 INFO:tasks.workunit.client.1.smithi192.stdout:1/30: dwrite f0 [0,4194304] 0 2024-04-23T02:13:49.518 INFO:tasks.workunit.client.1.smithi192.stdout:7/111: mknod d5/d17/d1c/c1f 0 2024-04-23T02:13:49.520 INFO:tasks.workunit.client.1.smithi192.stdout:1/31: creat f9 x:0 0 0 2024-04-23T02:13:49.521 INFO:tasks.workunit.client.1.smithi192.stdout:4/295: mknod de/c5d 0 2024-04-23T02:13:49.521 INFO:tasks.workunit.client.1.smithi192.stdout:0/57: mknod d6/df/c11 0 2024-04-23T02:13:49.523 INFO:tasks.workunit.client.1.smithi192.stdout:5/104: dread d2/fe [0,4194304] 0 2024-04-23T02:13:49.525 INFO:tasks.workunit.client.1.smithi192.stdout:7/112: rename d5/f8 to d5/d18/f20 0 2024-04-23T02:13:49.526 INFO:tasks.workunit.client.1.smithi192.stdout:5/105: mknod d2/db/d16/c24 0 2024-04-23T02:13:49.528 INFO:tasks.workunit.client.1.smithi192.stdout:4/296: symlink de/d15/d19/d1a/l5e 0 2024-04-23T02:13:49.529 INFO:tasks.workunit.client.1.smithi192.stdout:5/106: rmdir d2/d6/d14 39 2024-04-23T02:13:49.531 INFO:tasks.workunit.client.1.smithi192.stdout:0/58: dwrite d6/f7 [0,4194304] 0 2024-04-23T02:13:49.533 INFO:tasks.workunit.client.1.smithi192.stdout:0/59: write d6/f7 [1658970,10987] 0 2024-04-23T02:13:49.534 INFO:tasks.workunit.client.1.smithi192.stdout:4/297: write de/d15/d19/d3d/f57 [3607212,58432] 0 2024-04-23T02:13:49.535 INFO:tasks.workunit.client.1.smithi192.stdout:0/60: mknod d6/c12 0 2024-04-23T02:13:49.537 INFO:tasks.workunit.client.1.smithi192.stdout:4/298: unlink de/d15/f36 0 2024-04-23T02:13:49.538 INFO:tasks.workunit.client.1.smithi192.stdout:4/299: chown de/d15/d19/d1a/f3a 101 1 2024-04-23T02:13:49.558 INFO:tasks.workunit.client.1.smithi192.stdout:0/61: getdents d6/df 0 2024-04-23T02:13:49.559 INFO:tasks.workunit.client.1.smithi192.stdout:1/32: write f7 [889911,74773] 0 2024-04-23T02:13:49.560 INFO:tasks.workunit.client.1.smithi192.stdout:1/33: chown f7 1401777 1 2024-04-23T02:13:49.560 INFO:tasks.workunit.client.1.smithi192.stdout:1/34: rmdir - no directory 2024-04-23T02:13:49.563 INFO:tasks.workunit.client.1.smithi192.stdout:1/35: unlink f7 0 2024-04-23T02:13:49.563 INFO:tasks.workunit.client.1.smithi192.stdout:0/62: dread f3 [0,4194304] 0 2024-04-23T02:13:49.563 INFO:tasks.workunit.client.1.smithi192.stdout:1/36: fsync f9 0 2024-04-23T02:13:49.565 INFO:tasks.workunit.client.1.smithi192.stdout:0/63: unlink d6/c12 0 2024-04-23T02:13:49.569 INFO:tasks.workunit.client.1.smithi192.stdout:0/64: mkdir d6/df/d13 0 2024-04-23T02:13:49.570 INFO:tasks.workunit.client.1.smithi192.stdout:4/300: write de/d15/d19/d1a/f3a [3251788,7148] 0 2024-04-23T02:13:49.572 INFO:tasks.workunit.client.1.smithi192.stdout:5/107: dwrite d2/fe [4194304,4194304] 0 2024-04-23T02:13:49.572 INFO:tasks.workunit.client.1.smithi192.stdout:0/65: creat d6/df/f14 x:0 0 0 2024-04-23T02:13:49.576 INFO:tasks.workunit.client.1.smithi192.stdout:0/66: rename f3 to d6/df/d13/f15 0 2024-04-23T02:13:49.578 INFO:tasks.workunit.client.1.smithi192.stdout:5/108: rename d2/d6/c7 to d2/d6/c25 0 2024-04-23T02:13:49.579 INFO:tasks.workunit.client.1.smithi192.stdout:0/67: mkdir d6/df/d16 0 2024-04-23T02:13:49.580 INFO:tasks.workunit.client.1.smithi192.stdout:4/301: getdents de/d15/d19/d3d/d49/d26 0 2024-04-23T02:13:49.580 INFO:tasks.workunit.client.1.smithi192.stdout:0/68: stat d6/df/d13/f15 0 2024-04-23T02:13:49.582 INFO:tasks.workunit.client.1.smithi192.stdout:0/69: chown d6/df 1 1 2024-04-23T02:13:49.583 INFO:tasks.workunit.client.1.smithi192.stdout:4/302: truncate de/f1b 5109336 0 2024-04-23T02:13:49.587 INFO:tasks.workunit.client.1.smithi192.stdout:5/109: dwrite d2/db/f11 [0,4194304] 0 2024-04-23T02:13:49.587 INFO:tasks.workunit.client.1.smithi192.stdout:4/303: creat de/d15/d19/d3d/f5f x:0 0 0 2024-04-23T02:13:49.591 INFO:tasks.workunit.client.1.smithi192.stdout:5/110: write d2/db/f18 [322853,70842] 0 2024-04-23T02:13:49.593 INFO:tasks.workunit.client.1.smithi192.stdout:5/111: fsync d2/db/f18 0 2024-04-23T02:13:49.594 INFO:tasks.workunit.client.1.smithi192.stdout:0/70: dwrite d6/df/d13/f15 [0,4194304] 0 2024-04-23T02:13:49.599 INFO:tasks.workunit.client.1.smithi192.stdout:5/112: unlink d2/c1a 0 2024-04-23T02:13:49.600 INFO:tasks.workunit.client.1.smithi192.stdout:0/71: symlink d6/df/d13/l17 0 2024-04-23T02:13:49.601 INFO:tasks.workunit.client.1.smithi192.stdout:0/72: dread f5 [0,4194304] 0 2024-04-23T02:13:49.603 INFO:tasks.workunit.client.1.smithi192.stdout:5/113: dread d2/f5 [0,4194304] 0 2024-04-23T02:13:49.604 INFO:tasks.workunit.client.1.smithi192.stdout:0/73: unlink d6/fe 0 2024-04-23T02:13:49.606 INFO:tasks.workunit.client.1.smithi192.stdout:0/74: write d6/df/d13/f15 [2175445,101821] 0 2024-04-23T02:13:49.611 INFO:tasks.workunit.client.1.smithi192.stdout:0/75: unlink d6/df/d13/l17 0 2024-04-23T02:13:49.612 INFO:tasks.workunit.client.1.smithi192.stdout:5/114: dwrite d2/db/f20 [0,4194304] 0 2024-04-23T02:13:49.615 INFO:tasks.workunit.client.1.smithi192.stdout:0/76: creat d6/f18 x:0 0 0 2024-04-23T02:13:49.617 INFO:tasks.workunit.client.1.smithi192.stdout:4/304: dwrite de/d15/d19/d3d/d49/f28 [4194304,4194304] 0 2024-04-23T02:13:49.618 INFO:tasks.workunit.client.1.smithi192.stdout:5/115: rename d2/f15 to d2/db/d16/f26 0 2024-04-23T02:13:49.619 INFO:tasks.workunit.client.1.smithi192.stdout:0/77: fsync d6/fc 0 2024-04-23T02:13:49.624 INFO:tasks.workunit.client.1.smithi192.stdout:0/78: dread d6/f10 [0,4194304] 0 2024-04-23T02:13:49.625 INFO:tasks.workunit.client.1.smithi192.stdout:4/305: creat de/d15/d19/d3d/d49/f60 x:0 0 0 2024-04-23T02:13:49.625 INFO:tasks.workunit.client.1.smithi192.stdout:0/79: read - d6/f18 zero size 2024-04-23T02:13:49.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:49 smithi192 ceph-mon[49662]: pgmap v21: 65 pgs: 65 active+clean; 398 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 27 MiB/s wr, 185 op/s 2024-04-23T02:13:49.630 INFO:tasks.workunit.client.1.smithi192.stdout:0/80: creat d6/df/d16/f19 x:0 0 0 2024-04-23T02:13:49.631 INFO:tasks.workunit.client.1.smithi192.stdout:0/81: chown d6 862858285 1 2024-04-23T02:13:49.635 INFO:tasks.workunit.client.1.smithi192.stdout:0/82: mknod d6/df/c1a 0 2024-04-23T02:13:49.635 INFO:tasks.workunit.client.1.smithi192.stdout:4/306: dwrite de/d15/d19/d3d/f5f [0,4194304] 0 2024-04-23T02:13:49.636 INFO:tasks.workunit.client.1.smithi192.stdout:0/83: fsync d6/df/d13/f15 0 2024-04-23T02:13:49.637 INFO:tasks.workunit.client.1.smithi192.stdout:4/307: fsync de/d15/f2d 0 2024-04-23T02:13:49.639 INFO:tasks.workunit.client.1.smithi192.stdout:0/84: unlink d6/df/d13/f15 0 2024-04-23T02:13:49.641 INFO:tasks.workunit.client.1.smithi192.stdout:5/116: write d2/f5 [994288,8570] 0 2024-04-23T02:13:49.642 INFO:tasks.workunit.client.1.smithi192.stdout:4/308: mkdir de/d15/d2a/d61 0 2024-04-23T02:13:49.644 INFO:tasks.workunit.client.1.smithi192.stdout:0/85: truncate d6/f10 5201532 0 2024-04-23T02:13:49.646 INFO:tasks.workunit.client.1.smithi192.stdout:0/86: creat d6/f1b x:0 0 0 2024-04-23T02:13:49.648 INFO:tasks.workunit.client.1.smithi192.stdout:0/87: write d6/f18 [787643,151] 0 2024-04-23T02:13:49.649 INFO:tasks.workunit.client.1.smithi192.stdout:0/88: truncate d6/df/f14 888017 0 2024-04-23T02:13:49.651 INFO:tasks.workunit.client.1.smithi192.stdout:0/89: mknod d6/c1c 0 2024-04-23T02:13:49.657 INFO:tasks.workunit.client.1.smithi192.stdout:0/90: getdents d6 0 2024-04-23T02:13:49.659 INFO:tasks.workunit.client.1.smithi192.stdout:0/91: symlink d6/df/l1d 0 2024-04-23T02:13:49.659 INFO:tasks.workunit.client.1.smithi192.stdout:0/92: chown d6/c1c 3 1 2024-04-23T02:13:49.660 INFO:tasks.workunit.client.1.smithi192.stdout:0/93: readlink d6/df/l1d 0 2024-04-23T02:13:49.660 INFO:tasks.workunit.client.1.smithi192.stdout:0/94: chown d6/df/d16 267 1 2024-04-23T02:13:49.661 INFO:tasks.workunit.client.1.smithi192.stdout:0/95: symlink d6/df/d16/l1e 0 2024-04-23T02:13:49.665 INFO:tasks.workunit.client.1.smithi192.stdout:0/96: unlink l4 0 2024-04-23T02:13:49.666 INFO:tasks.workunit.client.1.smithi192.stdout:0/97: fdatasync d6/f1b 0 2024-04-23T02:13:49.666 INFO:tasks.workunit.client.1.smithi192.stdout:0/98: truncate d6/fc 1704528 0 2024-04-23T02:13:49.666 INFO:tasks.workunit.client.1.smithi192.stdout:0/99: fdatasync d6/f7 0 2024-04-23T02:13:49.668 INFO:tasks.workunit.client.1.smithi192.stdout:0/100: mknod d6/c1f 0 2024-04-23T02:13:49.670 INFO:tasks.workunit.client.1.smithi192.stdout:0/101: mknod d6/c20 0 2024-04-23T02:13:49.672 INFO:tasks.workunit.client.1.smithi192.stdout:0/102: unlink d6/c1c 0 2024-04-23T02:13:49.674 INFO:tasks.workunit.client.1.smithi192.stdout:0/103: symlink d6/df/l21 0 2024-04-23T02:13:49.674 INFO:tasks.workunit.client.1.smithi192.stdout:0/104: rename d6/df to d6/df/d13/d22 22 2024-04-23T02:13:49.674 INFO:tasks.workunit.client.1.smithi192.stdout:0/105: chown d6/df/d13 133463279 1 2024-04-23T02:13:49.676 INFO:tasks.workunit.client.1.smithi192.stdout:0/106: creat d6/df/f23 x:0 0 0 2024-04-23T02:13:49.678 INFO:tasks.workunit.client.1.smithi192.stdout:0/107: symlink d6/l24 0 2024-04-23T02:13:49.692 INFO:tasks.workunit.client.1.smithi192.stdout:0/108: dwrite d6/fc [0,4194304] 0 2024-04-23T02:13:49.693 INFO:tasks.workunit.client.1.smithi192.stdout:0/109: stat d6/c20 0 2024-04-23T02:13:49.728 INFO:tasks.workunit.client.1.smithi192.stdout:8/59: sync 2024-04-23T02:13:49.728 INFO:tasks.workunit.client.1.smithi192.stdout:6/53: sync 2024-04-23T02:13:49.728 INFO:tasks.workunit.client.1.smithi192.stdout:9/39: sync 2024-04-23T02:13:49.729 INFO:tasks.workunit.client.1.smithi192.stdout:9/40: write d0/fc [213276,1166] 0 2024-04-23T02:13:49.730 INFO:tasks.workunit.client.1.smithi192.stdout:0/110: truncate d6/df/f14 544602 0 2024-04-23T02:13:49.733 INFO:tasks.workunit.client.1.smithi192.stdout:8/60: link d1/fd d1/d9/f11 0 2024-04-23T02:13:49.736 INFO:tasks.workunit.client.1.smithi192.stdout:0/111: mkdir d6/df/d13/d25 0 2024-04-23T02:13:49.736 INFO:tasks.workunit.client.1.smithi192.stdout:9/41: write d0/f1 [376257,113745] 0 2024-04-23T02:13:49.737 INFO:tasks.workunit.client.1.smithi192.stdout:9/42: fsync d0/d9/fb 0 2024-04-23T02:13:49.738 INFO:tasks.workunit.client.1.smithi192.stdout:9/43: write d0/fc [250584,104447] 0 2024-04-23T02:13:49.739 INFO:tasks.workunit.client.1.smithi192.stdout:8/61: mkdir d1/d12 0 2024-04-23T02:13:49.740 INFO:tasks.workunit.client.1.smithi192.stdout:8/62: rename d1/d9 to d1/d9/d13 22 2024-04-23T02:13:49.741 INFO:tasks.workunit.client.1.smithi192.stdout:6/54: truncate d3/f6 366796 0 2024-04-23T02:13:49.741 INFO:tasks.workunit.client.1.smithi192.stdout:0/112: dread d6/f18 [0,4194304] 0 2024-04-23T02:13:49.743 INFO:tasks.workunit.client.1.smithi192.stdout:9/44: creat d0/d9/ff x:0 0 0 2024-04-23T02:13:49.744 INFO:tasks.workunit.client.1.smithi192.stdout:9/45: chown d0/f1 20 1 2024-04-23T02:13:49.744 INFO:tasks.workunit.client.1.smithi192.stdout:8/63: creat d1/d9/f14 x:0 0 0 2024-04-23T02:13:49.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:49 smithi071 ceph-mon[39837]: pgmap v21: 65 pgs: 65 active+clean; 398 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 27 MiB/s wr, 185 op/s 2024-04-23T02:13:49.745 INFO:tasks.workunit.client.1.smithi192.stdout:0/113: mkdir d6/df/d13/d25/d26 0 2024-04-23T02:13:49.745 INFO:tasks.workunit.client.1.smithi192.stdout:9/46: write d0/f1 [514984,64969] 0 2024-04-23T02:13:49.748 INFO:tasks.workunit.client.1.smithi192.stdout:0/114: write d6/df/d16/f19 [667524,85576] 0 2024-04-23T02:13:49.751 INFO:tasks.workunit.client.1.smithi192.stdout:9/47: rename d0/fc to d0/d9/f10 0 2024-04-23T02:13:49.753 INFO:tasks.workunit.client.1.smithi192.stdout:8/64: dwrite d1/f6 [0,4194304] 0 2024-04-23T02:13:49.754 INFO:tasks.workunit.client.1.smithi192.stdout:9/48: mknod d0/c11 0 2024-04-23T02:13:49.754 INFO:tasks.workunit.client.1.smithi192.stdout:8/65: chown d1/d9/f14 7903 1 2024-04-23T02:13:49.755 INFO:tasks.workunit.client.1.smithi192.stdout:9/49: write d0/d3/fa [712490,85840] 0 2024-04-23T02:13:49.757 INFO:tasks.workunit.client.1.smithi192.stdout:0/115: creat d6/df/d13/d25/d26/f27 x:0 0 0 2024-04-23T02:13:49.759 INFO:tasks.workunit.client.1.smithi192.stdout:0/116: symlink d6/l28 0 2024-04-23T02:13:49.760 INFO:tasks.workunit.client.1.smithi192.stdout:8/66: chown d1/d9/f14 5200 1 2024-04-23T02:13:49.760 INFO:tasks.workunit.client.1.smithi192.stdout:0/117: write d6/f7 [2773062,18769] 0 2024-04-23T02:13:49.761 INFO:tasks.workunit.client.1.smithi192.stdout:9/50: symlink d0/l12 0 2024-04-23T02:13:49.763 INFO:tasks.workunit.client.1.smithi192.stdout:9/51: write d0/d9/ff [901875,80508] 0 2024-04-23T02:13:49.763 INFO:tasks.workunit.client.1.smithi192.stdout:8/67: rmdir d1 39 2024-04-23T02:13:49.764 INFO:tasks.workunit.client.1.smithi192.stdout:0/118: stat d6/df/f14 0 2024-04-23T02:13:49.766 INFO:tasks.workunit.client.1.smithi192.stdout:8/68: write d1/f6 [1359605,88542] 0 2024-04-23T02:13:49.766 INFO:tasks.workunit.client.1.smithi192.stdout:9/52: rename d0/d9/cd to d0/d9/c13 0 2024-04-23T02:13:49.769 INFO:tasks.workunit.client.1.smithi192.stdout:0/119: rename d6/l9 to d6/df/l29 0 2024-04-23T02:13:49.773 INFO:tasks.workunit.client.1.smithi192.stdout:9/53: rename d0/f1 to d0/d9/f14 0 2024-04-23T02:13:49.774 INFO:tasks.workunit.client.1.smithi192.stdout:0/120: mknod d6/df/c2a 0 2024-04-23T02:13:49.774 INFO:tasks.workunit.client.1.smithi192.stdout:6/55: write d3/f6 [95493,104921] 0 2024-04-23T02:13:49.777 INFO:tasks.workunit.client.1.smithi192.stdout:9/54: write d0/d9/f14 [582253,22307] 0 2024-04-23T02:13:49.778 INFO:tasks.workunit.client.1.smithi192.stdout:9/55: chown d0/d9/f14 93205 1 2024-04-23T02:13:49.779 INFO:tasks.workunit.client.1.smithi192.stdout:9/56: chown d0/d9/f10 14 1 2024-04-23T02:13:49.780 INFO:tasks.workunit.client.1.smithi192.stdout:0/121: chown d6/df/l29 107093705 1 2024-04-23T02:13:49.782 INFO:tasks.workunit.client.1.smithi192.stdout:0/122: unlink d6/l28 0 2024-04-23T02:13:49.787 INFO:tasks.workunit.client.1.smithi192.stdout:6/56: link d3/ld d3/lf 0 2024-04-23T02:13:49.788 INFO:tasks.workunit.client.1.smithi192.stdout:8/69: dwrite d1/f3 [0,4194304] 0 2024-04-23T02:13:49.789 INFO:tasks.workunit.client.1.smithi192.stdout:6/57: chown d3/f9 1 1 2024-04-23T02:13:49.792 INFO:tasks.workunit.client.1.smithi192.stdout:8/70: fsync d1/d9/f11 0 2024-04-23T02:13:49.794 INFO:tasks.workunit.client.1.smithi192.stdout:6/58: mknod d3/c10 0 2024-04-23T02:13:49.794 INFO:tasks.workunit.client.1.smithi192.stdout:0/123: dwrite d6/df/d16/f19 [0,4194304] 0 2024-04-23T02:13:49.794 INFO:tasks.workunit.client.1.smithi192.stdout:8/71: chown d1/f2 23 1 2024-04-23T02:13:49.797 INFO:tasks.workunit.client.1.smithi192.stdout:6/59: write f1 [2036821,81395] 0 2024-04-23T02:13:49.798 INFO:tasks.workunit.client.1.smithi192.stdout:8/72: rename d1/f2 to d1/f15 0 2024-04-23T02:13:49.804 INFO:tasks.workunit.client.1.smithi192.stdout:0/124: dread d6/f10 [0,4194304] 0 2024-04-23T02:13:49.805 INFO:tasks.workunit.client.1.smithi192.stdout:8/73: dread d1/f10 [0,4194304] 0 2024-04-23T02:13:49.809 INFO:tasks.workunit.client.1.smithi192.stdout:8/74: mknod d1/d12/c16 0 2024-04-23T02:13:49.809 INFO:tasks.workunit.client.1.smithi192.stdout:6/60: dwrite d3/fa [0,4194304] 0 2024-04-23T02:13:49.812 INFO:tasks.workunit.client.1.smithi192.stdout:6/61: mkdir d3/d11 0 2024-04-23T02:13:49.813 INFO:tasks.workunit.client.1.smithi192.stdout:8/75: rename d1/d9/cb to d1/d9/c17 0 2024-04-23T02:13:49.814 INFO:tasks.workunit.client.1.smithi192.stdout:0/125: dread d6/f7 [0,4194304] 0 2024-04-23T02:13:49.815 INFO:tasks.workunit.client.1.smithi192.stdout:6/62: creat d3/f12 x:0 0 0 2024-04-23T02:13:49.815 INFO:tasks.workunit.client.1.smithi192.stdout:8/76: chown d1/fe 987029 1 2024-04-23T02:13:49.820 INFO:tasks.workunit.client.1.smithi192.stdout:6/63: dread d3/fa [0,4194304] 0 2024-04-23T02:13:49.820 INFO:tasks.workunit.client.1.smithi192.stdout:6/64: stat d3 0 2024-04-23T02:13:49.820 INFO:tasks.workunit.client.1.smithi192.stdout:0/126: creat d6/df/d13/d25/f2b x:0 0 0 2024-04-23T02:13:49.821 INFO:tasks.workunit.client.1.smithi192.stdout:8/77: mknod d1/df/c18 0 2024-04-23T02:13:49.821 INFO:tasks.workunit.client.1.smithi192.stdout:8/78: dread - d1/fe zero size 2024-04-23T02:13:49.822 INFO:tasks.workunit.client.1.smithi192.stdout:0/127: unlink d6/c1f 0 2024-04-23T02:13:49.824 INFO:tasks.workunit.client.1.smithi192.stdout:8/79: truncate d1/d9/f11 621431 0 2024-04-23T02:13:49.825 INFO:tasks.workunit.client.1.smithi192.stdout:6/65: creat d3/d11/f13 x:0 0 0 2024-04-23T02:13:49.827 INFO:tasks.workunit.client.1.smithi192.stdout:0/128: symlink d6/l2c 0 2024-04-23T02:13:49.831 INFO:tasks.workunit.client.1.smithi192.stdout:6/66: rename d3/ld to d3/d11/l14 0 2024-04-23T02:13:49.833 INFO:tasks.workunit.client.1.smithi192.stdout:6/67: write d3/f12 [144070,64112] 0 2024-04-23T02:13:49.837 INFO:tasks.workunit.client.1.smithi192.stdout:8/80: truncate d1/d9/f11 1050546 0 2024-04-23T02:13:49.838 INFO:tasks.workunit.client.1.smithi192.stdout:0/129: dwrite d6/f10 [0,4194304] 0 2024-04-23T02:13:49.843 INFO:tasks.workunit.client.1.smithi192.stdout:0/130: mknod d6/df/d13/d25/c2d 0 2024-04-23T02:13:49.843 INFO:tasks.workunit.client.1.smithi192.stdout:6/68: dwrite f1 [4194304,4194304] 0 2024-04-23T02:13:49.846 INFO:tasks.workunit.client.1.smithi192.stdout:8/81: read d1/fd [412568,64771] 0 2024-04-23T02:13:49.850 INFO:tasks.workunit.client.1.smithi192.stdout:6/69: mkdir d3/d15 0 2024-04-23T02:13:49.850 INFO:tasks.workunit.client.1.smithi192.stdout:0/131: creat d6/df/d16/f2e x:0 0 0 2024-04-23T02:13:49.852 INFO:tasks.workunit.client.1.smithi192.stdout:8/82: mkdir d1/d19 0 2024-04-23T02:13:49.852 INFO:tasks.workunit.client.1.smithi192.stdout:6/70: dread d3/f12 [0,4194304] 0 2024-04-23T02:13:49.853 INFO:tasks.workunit.client.1.smithi192.stdout:8/83: rename d1 to d1/d9/d1a 22 2024-04-23T02:13:49.853 INFO:tasks.workunit.client.1.smithi192.stdout:0/132: symlink d6/df/d13/d25/l2f 0 2024-04-23T02:13:49.854 INFO:tasks.workunit.client.1.smithi192.stdout:8/84: write d1/fe [349468,92648] 0 2024-04-23T02:13:49.856 INFO:tasks.workunit.client.1.smithi192.stdout:8/85: mknod d1/c1b 0 2024-04-23T02:13:49.858 INFO:tasks.workunit.client.1.smithi192.stdout:0/133: getdents d6/df 0 2024-04-23T02:13:49.858 INFO:tasks.workunit.client.1.smithi192.stdout:8/86: dread d1/d9/f11 [0,4194304] 0 2024-04-23T02:13:49.860 INFO:tasks.workunit.client.1.smithi192.stdout:0/134: creat d6/df/f30 x:0 0 0 2024-04-23T02:13:49.861 INFO:tasks.workunit.client.1.smithi192.stdout:8/87: creat d1/df/f1c x:0 0 0 2024-04-23T02:13:49.864 INFO:tasks.workunit.client.1.smithi192.stdout:0/135: rename d6/fc to d6/df/d13/f31 0 2024-04-23T02:13:49.865 INFO:tasks.workunit.client.1.smithi192.stdout:8/88: dread d1/f10 [0,4194304] 0 2024-04-23T02:13:49.869 INFO:tasks.workunit.client.1.smithi192.stdout:0/136: truncate d6/df/f14 1416960 0 2024-04-23T02:13:49.870 INFO:tasks.workunit.client.1.smithi192.stdout:0/137: read - d6/df/f30 zero size 2024-04-23T02:13:49.870 INFO:tasks.workunit.client.1.smithi192.stdout:0/138: truncate d6/f1b 928174 0 2024-04-23T02:13:49.870 INFO:tasks.workunit.client.1.smithi192.stdout:6/71: truncate f1 816857 0 2024-04-23T02:13:49.873 INFO:tasks.workunit.client.1.smithi192.stdout:0/139: getdents d6/df 0 2024-04-23T02:13:49.875 INFO:tasks.workunit.client.1.smithi192.stdout:8/89: truncate d1/f15 1813138 0 2024-04-23T02:13:49.878 INFO:tasks.workunit.client.1.smithi192.stdout:0/140: rename d6/f7 to d6/df/f32 0 2024-04-23T02:13:49.879 INFO:tasks.workunit.client.1.smithi192.stdout:6/72: dwrite d3/f6 [0,4194304] 0 2024-04-23T02:13:49.884 INFO:tasks.workunit.client.1.smithi192.stdout:6/73: mknod d3/d11/c16 0 2024-04-23T02:13:49.885 INFO:tasks.workunit.client.1.smithi192.stdout:6/74: chown d3/d15 6 1 2024-04-23T02:13:49.886 INFO:tasks.workunit.client.1.smithi192.stdout:0/141: rename d6/df/d13/d25/c2d to d6/df/d13/d25/d26/c33 0 2024-04-23T02:13:49.888 INFO:tasks.workunit.client.1.smithi192.stdout:6/75: symlink d3/d15/l17 0 2024-04-23T02:13:49.889 INFO:tasks.workunit.client.1.smithi192.stdout:8/90: truncate d1/d9/fa 832397 0 2024-04-23T02:13:49.890 INFO:tasks.workunit.client.1.smithi192.stdout:0/142: symlink d6/df/d13/d25/d26/l34 0 2024-04-23T02:13:49.891 INFO:tasks.workunit.client.1.smithi192.stdout:6/76: mkdir d3/d18 0 2024-04-23T02:13:49.892 INFO:tasks.workunit.client.1.smithi192.stdout:8/91: rename d1/f6 to d1/f1d 0 2024-04-23T02:13:49.895 INFO:tasks.workunit.client.1.smithi192.stdout:0/143: dread d6/df/f14 [0,4194304] 0 2024-04-23T02:13:49.896 INFO:tasks.workunit.client.1.smithi192.stdout:0/144: readlink d6/df/d13/d25/l2f 0 2024-04-23T02:13:49.897 INFO:tasks.workunit.client.1.smithi192.stdout:8/92: link d1/d9/f11 d1/d12/f1e 0 2024-04-23T02:13:49.899 INFO:tasks.workunit.client.1.smithi192.stdout:8/93: write d1/d9/f14 [271709,44588] 0 2024-04-23T02:13:49.901 INFO:tasks.workunit.client.1.smithi192.stdout:0/145: dread d6/df/d16/f19 [0,4194304] 0 2024-04-23T02:13:49.908 INFO:tasks.workunit.client.1.smithi192.stdout:6/77: truncate f1 600381 0 2024-04-23T02:13:49.911 INFO:tasks.workunit.client.1.smithi192.stdout:6/78: write d3/f12 [748843,8503] 0 2024-04-23T02:13:49.913 INFO:tasks.workunit.client.1.smithi192.stdout:0/146: dwrite d6/df/d13/d25/d26/f27 [0,4194304] 0 2024-04-23T02:13:49.915 INFO:tasks.workunit.client.1.smithi192.stdout:6/79: creat d3/f19 x:0 0 0 2024-04-23T02:13:49.917 INFO:tasks.workunit.client.1.smithi192.stdout:0/147: rmdir d6/df 39 2024-04-23T02:13:49.918 INFO:tasks.workunit.client.1.smithi192.stdout:8/94: dwrite d1/d9/f11 [0,4194304] 0 2024-04-23T02:13:49.921 INFO:tasks.workunit.client.1.smithi192.stdout:6/80: chown d3/d15/l17 347005 1 2024-04-23T02:13:49.923 INFO:tasks.workunit.client.1.smithi192.stdout:8/95: unlink d1/fd 0 2024-04-23T02:13:49.926 INFO:tasks.workunit.client.1.smithi192.stdout:8/96: rename d1/l7 to d1/l1f 0 2024-04-23T02:13:49.928 INFO:tasks.workunit.client.1.smithi192.stdout:8/97: creat d1/f20 x:0 0 0 2024-04-23T02:13:49.928 INFO:tasks.workunit.client.1.smithi192.stdout:6/81: fsync d3/f19 0 2024-04-23T02:13:49.936 INFO:tasks.workunit.client.1.smithi192.stdout:8/98: fdatasync d1/d12/f1e 0 2024-04-23T02:13:49.940 INFO:tasks.workunit.client.1.smithi192.stdout:8/99: unlink d1/d9/f14 0 2024-04-23T02:13:49.941 INFO:tasks.workunit.client.1.smithi192.stdout:8/100: write d1/f20 [249327,15534] 0 2024-04-23T02:13:49.941 INFO:tasks.workunit.client.1.smithi192.stdout:2/85: sync 2024-04-23T02:13:49.942 INFO:tasks.workunit.client.1.smithi192.stdout:2/86: write f7 [607000,82814] 0 2024-04-23T02:13:49.945 INFO:tasks.workunit.client.1.smithi192.stdout:8/101: rename d1/l1f to d1/l21 0 2024-04-23T02:13:49.945 INFO:tasks.workunit.client.1.smithi192.stdout:8/102: stat d1/f3 0 2024-04-23T02:13:49.951 INFO:tasks.workunit.client.1.smithi192.stdout:2/87: write f3 [1307103,113267] 0 2024-04-23T02:13:49.954 INFO:tasks.workunit.client.1.smithi192.stdout:2/88: rename d8/le to d8/d14/l16 0 2024-04-23T02:13:49.954 INFO:tasks.workunit.client.1.smithi192.stdout:8/103: dwrite d1/d12/f1e [4194304,4194304] 0 2024-04-23T02:13:49.956 INFO:tasks.workunit.client.1.smithi192.stdout:2/89: rmdir d8/d14 39 2024-04-23T02:13:49.958 INFO:tasks.workunit.client.1.smithi192.stdout:8/104: creat d1/d9/f22 x:0 0 0 2024-04-23T02:13:49.959 INFO:tasks.workunit.client.1.smithi192.stdout:8/105: truncate d1/f20 917941 0 2024-04-23T02:13:49.960 INFO:tasks.workunit.client.1.smithi192.stdout:2/90: dread f7 [0,4194304] 0 2024-04-23T02:13:49.966 INFO:tasks.workunit.client.1.smithi192.stdout:2/91: dwrite f4 [0,4194304] 0 2024-04-23T02:13:49.975 INFO:tasks.workunit.client.1.smithi192.stdout:2/92: dwrite f7 [0,4194304] 0 2024-04-23T02:13:49.981 INFO:tasks.workunit.client.1.smithi192.stdout:2/93: rename d8/l13 to d8/d14/l17 0 2024-04-23T02:13:49.981 INFO:tasks.workunit.client.1.smithi192.stdout:2/94: chown f4 67570 1 2024-04-23T02:13:49.987 INFO:tasks.workunit.client.1.smithi192.stdout:2/95: dwrite f4 [0,4194304] 0 2024-04-23T02:13:49.991 INFO:tasks.workunit.client.1.smithi192.stdout:2/96: creat d8/f18 x:0 0 0 2024-04-23T02:13:49.993 INFO:tasks.workunit.client.1.smithi192.stdout:2/97: mknod d8/d14/c19 0 2024-04-23T02:13:49.994 INFO:tasks.workunit.client.1.smithi192.stdout:2/98: chown c1 114887948 1 2024-04-23T02:13:49.996 INFO:tasks.workunit.client.1.smithi192.stdout:2/99: creat d8/d14/f1a x:0 0 0 2024-04-23T02:13:49.996 INFO:tasks.workunit.client.1.smithi192.stdout:2/100: truncate d8/f12 112519 0 2024-04-23T02:13:49.997 INFO:tasks.workunit.client.1.smithi192.stdout:2/101: chown d8/d14/f15 393582651 1 2024-04-23T02:13:49.997 INFO:tasks.workunit.client.1.smithi192.stdout:2/102: truncate f3 3630732 0 2024-04-23T02:13:50.004 INFO:tasks.workunit.client.1.smithi192.stdout:2/103: rmdir d8/d14 39 2024-04-23T02:13:50.005 INFO:tasks.workunit.client.1.smithi192.stdout:2/104: write d8/fb [602903,32671] 0 2024-04-23T02:13:50.006 INFO:tasks.workunit.client.1.smithi192.stdout:2/105: fsync d8/fb 0 2024-04-23T02:13:50.009 INFO:tasks.workunit.client.1.smithi192.stdout:2/106: creat d8/f1b x:0 0 0 2024-04-23T02:13:50.145 INFO:tasks.workunit.client.1.smithi192.stdout:3/65: sync 2024-04-23T02:13:50.146 INFO:tasks.workunit.client.1.smithi192.stdout:3/66: dread - d1/d3/d6/d7/f13 zero size 2024-04-23T02:13:50.147 INFO:tasks.workunit.client.1.smithi192.stdout:3/67: unlink d1/f2 0 2024-04-23T02:13:50.148 INFO:tasks.workunit.client.1.smithi192.stdout:3/68: dread - d1/d3/d6/d7/f13 zero size 2024-04-23T02:13:50.189 INFO:tasks.workunit.client.1.smithi192.stdout:1/37: sync 2024-04-23T02:13:50.189 INFO:tasks.workunit.client.1.smithi192.stdout:2/107: sync 2024-04-23T02:13:50.189 INFO:tasks.workunit.client.1.smithi192.stdout:8/106: sync 2024-04-23T02:13:50.189 INFO:tasks.workunit.client.1.smithi192.stdout:7/113: sync 2024-04-23T02:13:50.189 INFO:tasks.workunit.client.1.smithi192.stdout:9/57: sync 2024-04-23T02:13:50.189 INFO:tasks.workunit.client.1.smithi192.stdout:5/117: sync 2024-04-23T02:13:50.189 INFO:tasks.workunit.client.1.smithi192.stdout:6/82: sync 2024-04-23T02:13:50.189 INFO:tasks.workunit.client.1.smithi192.stdout:0/148: sync 2024-04-23T02:13:50.189 INFO:tasks.workunit.client.1.smithi192.stdout:4/309: sync 2024-04-23T02:13:50.191 INFO:tasks.workunit.client.1.smithi192.stdout:1/38: chown l8 43571 1 2024-04-23T02:13:50.191 INFO:tasks.workunit.client.1.smithi192.stdout:1/39: rmdir - no directory 2024-04-23T02:13:50.191 INFO:tasks.workunit.client.1.smithi192.stdout:8/107: unlink d1/f20 0 2024-04-23T02:13:50.195 INFO:tasks.workunit.client.1.smithi192.stdout:0/149: write d6/f18 [824413,70503] 0 2024-04-23T02:13:50.197 INFO:tasks.workunit.client.1.smithi192.stdout:1/40: creat fa x:0 0 0 2024-04-23T02:13:50.198 INFO:tasks.workunit.client.1.smithi192.stdout:2/108: symlink d8/d14/l1c 0 2024-04-23T02:13:50.199 INFO:tasks.workunit.client.1.smithi192.stdout:9/58: dwrite d0/d9/ff [0,4194304] 0 2024-04-23T02:13:50.200 INFO:tasks.workunit.client.1.smithi192.stdout:7/114: write d5/f15 [5123879,113458] 0 2024-04-23T02:13:50.201 INFO:tasks.workunit.client.1.smithi192.stdout:2/109: chown d8/d14/l1c 1 1 2024-04-23T02:13:50.201 INFO:tasks.workunit.client.1.smithi192.stdout:7/115: stat d5/l11 0 2024-04-23T02:13:50.202 INFO:tasks.workunit.client.1.smithi192.stdout:4/310: fdatasync de/f1b 0 2024-04-23T02:13:50.203 INFO:tasks.workunit.client.1.smithi192.stdout:6/83: symlink d3/d18/l1a 0 2024-04-23T02:13:50.204 INFO:tasks.workunit.client.1.smithi192.stdout:8/108: dread d1/f15 [0,4194304] 0 2024-04-23T02:13:50.207 INFO:tasks.workunit.client.1.smithi192.stdout:9/59: symlink d0/l15 0 2024-04-23T02:13:50.208 INFO:tasks.workunit.client.1.smithi192.stdout:9/60: stat d0/d9/f14 0 2024-04-23T02:13:50.209 INFO:tasks.workunit.client.1.smithi192.stdout:5/118: truncate f0 1715044 0 2024-04-23T02:13:50.213 INFO:tasks.workunit.client.1.smithi192.stdout:6/84: mkdir d3/d15/d1b 0 2024-04-23T02:13:50.214 INFO:tasks.workunit.client.1.smithi192.stdout:0/150: mkdir d6/df/d35 0 2024-04-23T02:13:50.215 INFO:tasks.workunit.client.1.smithi192.stdout:4/311: rmdir de/d15/d19/d3d/d34/d4a 39 2024-04-23T02:13:50.216 INFO:tasks.workunit.client.1.smithi192.stdout:8/109: truncate d1/f10 1372404 0 2024-04-23T02:13:50.217 INFO:tasks.workunit.client.1.smithi192.stdout:9/61: unlink d0/c11 0 2024-04-23T02:13:50.218 INFO:tasks.workunit.client.1.smithi192.stdout:7/116: dwrite d5/d18/f20 [0,4194304] 0 2024-04-23T02:13:50.220 INFO:tasks.workunit.client.1.smithi192.stdout:6/85: symlink d3/l1c 0 2024-04-23T02:13:50.223 INFO:tasks.workunit.client.1.smithi192.stdout:7/117: fsync d5/f15 0 2024-04-23T02:13:50.226 INFO:tasks.workunit.client.1.smithi192.stdout:9/62: dread d0/d9/fb [0,4194304] 0 2024-04-23T02:13:50.227 INFO:tasks.workunit.client.1.smithi192.stdout:4/312: mknod de/d15/d19/d3d/d49/d26/c62 0 2024-04-23T02:13:50.228 INFO:tasks.workunit.client.1.smithi192.stdout:6/86: dread d3/f12 [0,4194304] 0 2024-04-23T02:13:50.230 INFO:tasks.workunit.client.1.smithi192.stdout:2/110: creat d8/f1d x:0 0 0 2024-04-23T02:13:50.230 INFO:tasks.workunit.client.1.smithi192.stdout:7/118: chown d5/c6 344621 1 2024-04-23T02:13:50.232 INFO:tasks.workunit.client.1.smithi192.stdout:8/110: dwrite d1/d9/f11 [8388608,4194304] 0 2024-04-23T02:13:50.234 INFO:tasks.workunit.client.1.smithi192.stdout:5/119: creat d2/f27 x:0 0 0 2024-04-23T02:13:50.235 INFO:tasks.workunit.client.1.smithi192.stdout:4/313: fdatasync de/d15/d19/d1a/f3a 0 2024-04-23T02:13:50.235 INFO:tasks.workunit.client.1.smithi192.stdout:3/69: sync 2024-04-23T02:13:50.238 INFO:tasks.workunit.client.1.smithi192.stdout:2/111: write d8/f1b [894116,123384] 0 2024-04-23T02:13:50.241 INFO:tasks.workunit.client.1.smithi192.stdout:7/119: write d5/f15 [2015078,105540] 0 2024-04-23T02:13:50.244 INFO:tasks.workunit.client.1.smithi192.stdout:1/41: truncate f4 372394 0 2024-04-23T02:13:50.247 INFO:tasks.workunit.client.1.smithi192.stdout:9/63: link d0/d9/fb d0/f16 0 2024-04-23T02:13:50.248 INFO:tasks.workunit.client.1.smithi192.stdout:2/112: mknod d8/d14/c1e 0 2024-04-23T02:13:50.249 INFO:tasks.workunit.client.1.smithi192.stdout:8/111: mkdir d1/d12/d23 0 2024-04-23T02:13:50.250 INFO:tasks.workunit.client.1.smithi192.stdout:1/42: write f9 [735994,34510] 0 2024-04-23T02:13:50.252 INFO:tasks.workunit.client.1.smithi192.stdout:0/151: dwrite d6/df/d13/f31 [0,4194304] 0 2024-04-23T02:13:50.253 INFO:tasks.workunit.client.1.smithi192.stdout:7/120: rename d5/ca to d5/d18/c21 0 2024-04-23T02:13:50.259 INFO:tasks.workunit.client.1.smithi192.stdout:4/314: creat de/d15/d19/d3d/d49/d26/d59/f63 x:0 0 0 2024-04-23T02:13:50.261 INFO:tasks.workunit.client.1.smithi192.stdout:1/43: symlink lb 0 2024-04-23T02:13:50.261 INFO:tasks.workunit.client.1.smithi192.stdout:3/70: write d1/d3/d6/d7/f11 [951777,107485] 0 2024-04-23T02:13:50.263 INFO:tasks.workunit.client.1.smithi192.stdout:2/113: dread f4 [0,4194304] 0 2024-04-23T02:13:50.263 INFO:tasks.workunit.client.1.smithi192.stdout:5/120: dwrite d2/fe [0,4194304] 0 2024-04-23T02:13:50.264 INFO:tasks.workunit.client.1.smithi192.stdout:6/87: dwrite d3/fa [0,4194304] 0 2024-04-23T02:13:50.264 INFO:tasks.workunit.client.1.smithi192.stdout:9/64: unlink d0/d9/c13 0 2024-04-23T02:13:50.265 INFO:tasks.workunit.client.1.smithi192.stdout:7/121: creat d5/d18/f22 x:0 0 0 2024-04-23T02:13:50.268 INFO:tasks.workunit.client.1.smithi192.stdout:1/44: chown f9 890 1 2024-04-23T02:13:50.268 INFO:tasks.workunit.client.1.smithi192.stdout:8/112: chown d1/d9/fa 49980824 1 2024-04-23T02:13:50.271 INFO:tasks.workunit.client.1.smithi192.stdout:2/114: dread d8/fb [0,4194304] 0 2024-04-23T02:13:50.271 INFO:tasks.workunit.client.1.smithi192.stdout:3/71: symlink d1/l14 0 2024-04-23T02:13:50.273 INFO:tasks.workunit.client.1.smithi192.stdout:8/113: write d1/f3 [3451516,115543] 0 2024-04-23T02:13:50.277 INFO:tasks.workunit.client.1.smithi192.stdout:9/65: symlink d0/d3/l17 0 2024-04-23T02:13:50.277 INFO:tasks.workunit.client.1.smithi192.stdout:5/121: truncate d2/f27 80904 0 2024-04-23T02:13:50.277 INFO:tasks.workunit.client.1.smithi192.stdout:3/72: unlink d1/l14 0 2024-04-23T02:13:50.278 INFO:tasks.workunit.client.1.smithi192.stdout:0/152: link f5 d6/df/d35/f36 0 2024-04-23T02:13:50.283 INFO:tasks.workunit.client.1.smithi192.stdout:5/122: chown d2/db/f18 81 1 2024-04-23T02:13:50.283 INFO:tasks.workunit.client.1.smithi192.stdout:9/66: mknod d0/c18 0 2024-04-23T02:13:50.284 INFO:tasks.workunit.client.1.smithi192.stdout:3/73: read - d1/d3/d6/d7/f13 zero size 2024-04-23T02:13:50.284 INFO:tasks.workunit.client.1.smithi192.stdout:4/315: rename de/d15/c55 to de/d15/d19/d3d/d49/c64 0 2024-04-23T02:13:50.288 INFO:tasks.workunit.client.1.smithi192.stdout:1/45: dwrite f3 [0,4194304] 0 2024-04-23T02:13:50.291 INFO:tasks.workunit.client.1.smithi192.stdout:7/122: dread d5/d18/f20 [0,4194304] 0 2024-04-23T02:13:50.291 INFO:tasks.workunit.client.1.smithi192.stdout:6/88: dread d3/fa [0,4194304] 0 2024-04-23T02:13:50.292 INFO:tasks.workunit.client.1.smithi192.stdout:9/67: read d0/d9/f14 [493790,108496] 0 2024-04-23T02:13:50.293 INFO:tasks.workunit.client.1.smithi192.stdout:3/74: write d1/d3/d6/d7/f13 [908535,59407] 0 2024-04-23T02:13:50.294 INFO:tasks.workunit.client.1.smithi192.stdout:2/115: getdents d8/d14 0 2024-04-23T02:13:50.295 INFO:tasks.workunit.client.1.smithi192.stdout:8/114: getdents d1/d9 0 2024-04-23T02:13:50.298 INFO:tasks.workunit.client.1.smithi192.stdout:1/46: mkdir dc 0 2024-04-23T02:13:50.298 INFO:tasks.workunit.client.1.smithi192.stdout:8/115: write d1/fe [127750,16051] 0 2024-04-23T02:13:50.300 INFO:tasks.workunit.client.1.smithi192.stdout:5/123: mknod d2/d6/c28 0 2024-04-23T02:13:50.304 INFO:tasks.workunit.client.1.smithi192.stdout:6/89: dwrite d3/f6 [0,4194304] 0 2024-04-23T02:13:50.305 INFO:tasks.workunit.client.1.smithi192.stdout:7/123: symlink d5/d18/l23 0 2024-04-23T02:13:50.306 INFO:tasks.workunit.client.1.smithi192.stdout:2/116: mknod d8/d14/c1f 0 2024-04-23T02:13:50.309 INFO:tasks.workunit.client.1.smithi192.stdout:2/117: write f3 [2842520,39349] 0 2024-04-23T02:13:50.316 INFO:tasks.workunit.client.1.smithi192.stdout:6/90: write d3/fa [3687080,48610] 0 2024-04-23T02:13:50.316 INFO:tasks.workunit.client.1.smithi192.stdout:7/124: dread d5/d18/f20 [0,4194304] 0 2024-04-23T02:13:50.318 INFO:tasks.workunit.client.1.smithi192.stdout:5/124: dwrite d2/db/f11 [0,4194304] 0 2024-04-23T02:13:50.320 INFO:tasks.workunit.client.1.smithi192.stdout:3/75: mkdir d1/d3/dd/d15 0 2024-04-23T02:13:50.322 INFO:tasks.workunit.client.1.smithi192.stdout:6/91: write d3/f9 [5011187,47811] 0 2024-04-23T02:13:50.324 INFO:tasks.workunit.client.1.smithi192.stdout:7/125: chown l2 87893 1 2024-04-23T02:13:50.325 INFO:tasks.workunit.client.1.smithi192.stdout:4/316: rmdir de/d15/d19/d3d 39 2024-04-23T02:13:50.325 INFO:tasks.workunit.client.1.smithi192.stdout:4/317: stat de/d15/d19/d1a 0 2024-04-23T02:13:50.325 INFO:tasks.workunit.client.1.smithi192.stdout:3/76: chown d1/d3/dd 290141 1 2024-04-23T02:13:50.326 INFO:tasks.workunit.client.1.smithi192.stdout:7/126: write d5/f15 [5125716,90652] 0 2024-04-23T02:13:50.327 INFO:tasks.workunit.client.1.smithi192.stdout:9/68: link d0/d3/l17 d0/l19 0 2024-04-23T02:13:50.333 INFO:tasks.workunit.client.1.smithi192.stdout:3/77: fsync d1/f5 0 2024-04-23T02:13:50.334 INFO:tasks.workunit.client.1.smithi192.stdout:6/92: rmdir d3 39 2024-04-23T02:13:50.334 INFO:tasks.workunit.client.1.smithi192.stdout:8/116: creat d1/d12/f24 x:0 0 0 2024-04-23T02:13:50.336 INFO:tasks.workunit.client.1.smithi192.stdout:9/69: symlink d0/l1a 0 2024-04-23T02:13:50.336 INFO:tasks.workunit.client.1.smithi192.stdout:8/117: chown d1/d12/d23 13 1 2024-04-23T02:13:50.338 INFO:tasks.workunit.client.1.smithi192.stdout:7/127: fsync d5/d18/f20 0 2024-04-23T02:13:50.339 INFO:tasks.workunit.client.1.smithi192.stdout:1/47: dread f4 [0,4194304] 0 2024-04-23T02:13:50.340 INFO:tasks.workunit.client.1.smithi192.stdout:3/78: chown d1/d3/d6/f10 1 1 2024-04-23T02:13:50.342 INFO:tasks.workunit.client.1.smithi192.stdout:2/118: getdents d8/d14 0 2024-04-23T02:13:50.343 INFO:tasks.workunit.client.1.smithi192.stdout:2/119: read - d8/f18 zero size 2024-04-23T02:13:50.344 INFO:tasks.workunit.client.1.smithi192.stdout:9/70: write d0/d3/f8 [2699080,127423] 0 2024-04-23T02:13:50.344 INFO:tasks.workunit.client.1.smithi192.stdout:8/118: symlink d1/df/l25 0 2024-04-23T02:13:50.345 INFO:tasks.workunit.client.1.smithi192.stdout:2/120: write d8/f1b [318201,40868] 0 2024-04-23T02:13:50.347 INFO:tasks.workunit.client.1.smithi192.stdout:2/121: truncate d8/f1d 61845 0 2024-04-23T02:13:50.347 INFO:tasks.workunit.client.1.smithi192.stdout:7/128: creat d5/d17/d1c/f24 x:0 0 0 2024-04-23T02:13:50.348 INFO:tasks.workunit.client.1.smithi192.stdout:1/48: unlink l6 0 2024-04-23T02:13:50.350 INFO:tasks.workunit.client.1.smithi192.stdout:5/125: truncate d2/db/d16/d1b/f1f 2765182 0 2024-04-23T02:13:50.351 INFO:tasks.workunit.client.1.smithi192.stdout:7/129: dread d5/d18/f20 [0,4194304] 0 2024-04-23T02:13:50.430 INFO:tasks.workunit.client.1.smithi192.stdout:4/318: truncate de/d15/d19/d1a/f3a 930137 0 2024-04-23T02:13:50.430 INFO:tasks.workunit.client.1.smithi192.stdout:9/71: mknod d0/d3/c1b 0 2024-04-23T02:13:50.432 INFO:tasks.workunit.client.1.smithi192.stdout:2/122: write d8/d14/f15 [452080,120592] 0 2024-04-23T02:13:50.441 INFO:tasks.workunit.client.1.smithi192.stdout:3/79: rmdir d1/d3/dd/d15 0 2024-04-23T02:13:50.442 INFO:tasks.workunit.client.1.smithi192.stdout:5/126: rename f0 to d2/db/d16/d1b/f29 0 2024-04-23T02:13:50.444 INFO:tasks.workunit.client.1.smithi192.stdout:6/93: getdents d3 0 2024-04-23T02:13:50.446 INFO:tasks.workunit.client.1.smithi192.stdout:9/72: link d0/d9/ff d0/d3/f1c 0 2024-04-23T02:13:50.446 INFO:tasks.workunit.client.1.smithi192.stdout:2/123: mknod d8/c20 0 2024-04-23T02:13:50.448 INFO:tasks.workunit.client.1.smithi192.stdout:3/80: symlink d1/l16 0 2024-04-23T02:13:50.449 INFO:tasks.workunit.client.1.smithi192.stdout:3/81: fsync d1/f5 0 2024-04-23T02:13:50.449 INFO:tasks.workunit.client.1.smithi192.stdout:5/127: rmdir d2/db/d16 39 2024-04-23T02:13:50.450 INFO:tasks.workunit.client.1.smithi192.stdout:7/130: truncate d5/f15 4142159 0 2024-04-23T02:13:50.451 INFO:tasks.workunit.client.1.smithi192.stdout:5/128: dread d2/f27 [0,4194304] 0 2024-04-23T02:13:50.453 INFO:tasks.workunit.client.1.smithi192.stdout:2/124: rename d8/d14/l16 to d8/d14/l21 0 2024-04-23T02:13:50.454 INFO:tasks.workunit.client.1.smithi192.stdout:2/125: stat d8/f1b 0 2024-04-23T02:13:50.454 INFO:tasks.workunit.client.1.smithi192.stdout:3/82: mkdir d1/d3/d17 0 2024-04-23T02:13:50.456 INFO:tasks.workunit.client.1.smithi192.stdout:7/131: symlink d5/l25 0 2024-04-23T02:13:50.461 INFO:tasks.workunit.client.1.smithi192.stdout:9/73: rename d0/l19 to d0/d3/l1d 0 2024-04-23T02:13:50.461 INFO:tasks.workunit.client.1.smithi192.stdout:4/319: truncate de/d15/d19/d3d/f57 1663647 0 2024-04-23T02:13:50.462 INFO:tasks.workunit.client.1.smithi192.stdout:2/126: symlink d8/d14/l22 0 2024-04-23T02:13:50.464 INFO:tasks.workunit.client.1.smithi192.stdout:3/83: creat d1/f18 x:0 0 0 2024-04-23T02:13:50.465 INFO:tasks.workunit.client.1.smithi192.stdout:7/132: dwrite d5/d18/f1d [0,4194304] 0 2024-04-23T02:13:50.465 INFO:tasks.workunit.client.1.smithi192.stdout:7/133: fsync d5/d17/d1c/f24 0 2024-04-23T02:13:50.467 INFO:tasks.workunit.client.1.smithi192.stdout:5/129: chown d2/db/d16/d1b/f29 287351 1 2024-04-23T02:13:50.470 INFO:tasks.workunit.client.1.smithi192.stdout:9/74: mkdir d0/d1e 0 2024-04-23T02:13:50.472 INFO:tasks.workunit.client.1.smithi192.stdout:9/75: chown d0/d3 3848159 1 2024-04-23T02:13:50.473 INFO:tasks.workunit.client.1.smithi192.stdout:7/134: mkdir d5/d17/d1c/d26 0 2024-04-23T02:13:50.474 INFO:tasks.workunit.client.1.smithi192.stdout:4/320: dread de/d15/d2a/f4c [0,4194304] 0 2024-04-23T02:13:50.474 INFO:tasks.workunit.client.1.smithi192.stdout:6/94: getdents d3/d15 0 2024-04-23T02:13:50.475 INFO:tasks.workunit.client.1.smithi192.stdout:4/321: dread - de/d15/d2a/f5c zero size 2024-04-23T02:13:50.475 INFO:tasks.workunit.client.1.smithi192.stdout:6/95: chown d3/d15/d1b 0 1 2024-04-23T02:13:50.476 INFO:tasks.workunit.client.1.smithi192.stdout:6/96: dread - d3/f19 zero size 2024-04-23T02:13:50.476 INFO:tasks.workunit.client.1.smithi192.stdout:0/153: sync 2024-04-23T02:13:50.483 INFO:tasks.workunit.client.1.smithi192.stdout:5/130: truncate d2/f5 3383860 0 2024-04-23T02:13:50.484 INFO:tasks.workunit.client.1.smithi192.stdout:6/97: dwrite d3/f9 [4194304,4194304] 0 2024-04-23T02:13:50.484 INFO:tasks.workunit.client.1.smithi192.stdout:7/135: read d5/f15 [3221503,39037] 0 2024-04-23T02:13:50.485 INFO:tasks.workunit.client.1.smithi192.stdout:5/131: chown d2/db/d16/d1b/c23 15745 1 2024-04-23T02:13:50.485 INFO:tasks.workunit.client.1.smithi192.stdout:7/136: fdatasync d5/d17/d1c/f24 0 2024-04-23T02:13:50.495 INFO:tasks.workunit.client.1.smithi192.stdout:2/127: truncate f7 137796 0 2024-04-23T02:13:50.495 INFO:tasks.workunit.client.1.smithi192.stdout:0/154: symlink d6/df/d13/l37 0 2024-04-23T02:13:50.495 INFO:tasks.workunit.client.1.smithi192.stdout:9/76: mkdir d0/d1e/d1f 0 2024-04-23T02:13:50.495 INFO:tasks.workunit.client.1.smithi192.stdout:6/98: creat d3/d11/f1d x:0 0 0 2024-04-23T02:13:50.496 INFO:tasks.workunit.client.1.smithi192.stdout:0/155: read d6/f18 [463618,102310] 0 2024-04-23T02:13:50.496 INFO:tasks.workunit.client.1.smithi192.stdout:7/137: rename d5/c14 to d5/d17/d1c/d26/c27 0 2024-04-23T02:13:50.496 INFO:tasks.workunit.client.1.smithi192.stdout:7/138: readlink d5/le 0 2024-04-23T02:13:50.496 INFO:tasks.workunit.client.1.smithi192.stdout:4/322: creat de/d15/d19/d3d/d49/f65 x:0 0 0 2024-04-23T02:13:50.497 INFO:tasks.workunit.client.1.smithi192.stdout:2/128: unlink d8/d14/c19 0 2024-04-23T02:13:50.497 INFO:tasks.workunit.client.1.smithi192.stdout:5/132: dwrite d2/db/f20 [0,4194304] 0 2024-04-23T02:13:50.499 INFO:tasks.workunit.client.1.smithi192.stdout:2/129: write d8/fb [810998,30939] 0 2024-04-23T02:13:50.501 INFO:tasks.workunit.client.1.smithi192.stdout:0/156: truncate d6/df/d13/d25/d26/f27 4474334 0 2024-04-23T02:13:50.502 INFO:tasks.workunit.client.1.smithi192.stdout:6/99: dread d3/f9 [0,4194304] 0 2024-04-23T02:13:50.503 INFO:tasks.workunit.client.1.smithi192.stdout:0/157: truncate d6/df/d16/f2e 397848 0 2024-04-23T02:13:50.504 INFO:tasks.workunit.client.1.smithi192.stdout:7/139: creat d5/d17/d1c/d26/f28 x:0 0 0 2024-04-23T02:13:50.504 INFO:tasks.workunit.client.1.smithi192.stdout:6/100: read d3/fa [3358051,118444] 0 2024-04-23T02:13:50.509 INFO:tasks.workunit.client.1.smithi192.stdout:2/130: mkdir d8/d23 0 2024-04-23T02:13:50.511 INFO:tasks.workunit.client.1.smithi192.stdout:0/158: creat d6/df/d16/f38 x:0 0 0 2024-04-23T02:13:50.513 INFO:tasks.workunit.client.1.smithi192.stdout:7/140: dwrite d5/d18/f1d [0,4194304] 0 2024-04-23T02:13:50.514 INFO:tasks.workunit.client.1.smithi192.stdout:5/133: creat d2/f2a x:0 0 0 2024-04-23T02:13:50.516 INFO:tasks.workunit.client.1.smithi192.stdout:6/101: mknod d3/d11/c1e 0 2024-04-23T02:13:50.519 INFO:tasks.workunit.client.1.smithi192.stdout:2/131: unlink d8/d14/c1e 0 2024-04-23T02:13:50.521 INFO:tasks.workunit.client.1.smithi192.stdout:0/159: dread d6/df/f32 [0,4194304] 0 2024-04-23T02:13:50.523 INFO:tasks.workunit.client.1.smithi192.stdout:7/141: dwrite d5/d17/d1c/d26/f28 [0,4194304] 0 2024-04-23T02:13:50.524 INFO:tasks.workunit.client.1.smithi192.stdout:7/142: chown d5/l25 569036 1 2024-04-23T02:13:50.525 INFO:tasks.workunit.client.1.smithi192.stdout:9/77: write d0/d3/f1c [5017479,6043] 0 2024-04-23T02:13:50.525 INFO:tasks.workunit.client.1.smithi192.stdout:7/143: chown d5/d17/d1c/c1f 12652857 1 2024-04-23T02:13:50.526 INFO:tasks.workunit.client.1.smithi192.stdout:6/102: rename d3/f6 to d3/d18/f1f 0 2024-04-23T02:13:50.526 INFO:tasks.workunit.client.1.smithi192.stdout:9/78: write d0/d9/ff [3622866,69020] 0 2024-04-23T02:13:50.531 INFO:tasks.workunit.client.1.smithi192.stdout:4/323: write de/d15/d19/d3d/f46 [834331,16254] 0 2024-04-23T02:13:50.531 INFO:tasks.workunit.client.1.smithi192.stdout:5/134: mkdir d2/db/d2b 0 2024-04-23T02:13:50.534 INFO:tasks.workunit.client.1.smithi192.stdout:7/144: mknod d5/d18/c29 0 2024-04-23T02:13:50.535 INFO:tasks.workunit.client.1.smithi192.stdout:2/132: creat d8/d23/f24 x:0 0 0 2024-04-23T02:13:50.536 INFO:tasks.workunit.client.1.smithi192.stdout:2/133: fdatasync d8/d14/f1a 0 2024-04-23T02:13:50.537 INFO:tasks.workunit.client.1.smithi192.stdout:2/134: chown c1 16170 1 2024-04-23T02:13:50.541 INFO:tasks.workunit.client.1.smithi192.stdout:5/135: read d2/f5 [1990722,29540] 0 2024-04-23T02:13:50.543 INFO:tasks.workunit.client.1.smithi192.stdout:7/145: dread d5/d18/f20 [0,4194304] 0 2024-04-23T02:13:50.543 INFO:tasks.workunit.client.1.smithi192.stdout:4/324: mknod de/d15/d2a/d61/c66 0 2024-04-23T02:13:50.544 INFO:tasks.workunit.client.1.smithi192.stdout:6/103: dwrite d3/f9 [4194304,4194304] 0 2024-04-23T02:13:50.546 INFO:tasks.workunit.client.1.smithi192.stdout:7/146: dread - d5/d17/d1c/f24 zero size 2024-04-23T02:13:50.546 INFO:tasks.workunit.client.1.smithi192.stdout:4/325: dread - de/d15/d19/d3d/d49/f60 zero size 2024-04-23T02:13:50.546 INFO:tasks.workunit.client.1.smithi192.stdout:9/79: rmdir d0/d1e/d1f 0 2024-04-23T02:13:50.550 INFO:tasks.workunit.client.1.smithi192.stdout:4/326: dread - de/d15/d2a/f5c zero size 2024-04-23T02:13:50.550 INFO:tasks.workunit.client.1.smithi192.stdout:2/135: mkdir d8/d25 0 2024-04-23T02:13:50.551 INFO:tasks.workunit.client.1.smithi192.stdout:7/147: rename d5/d17/d1c to d5/d17/d1c/d2a 22 2024-04-23T02:13:50.552 INFO:tasks.workunit.client.1.smithi192.stdout:9/80: rename d0/l1a to d0/l20 0 2024-04-23T02:13:50.554 INFO:tasks.workunit.client.1.smithi192.stdout:5/136: dwrite d2/db/f13 [0,4194304] 0 2024-04-23T02:13:50.556 INFO:tasks.workunit.client.1.smithi192.stdout:6/104: truncate d3/fa 142255 0 2024-04-23T02:13:50.556 INFO:tasks.workunit.client.1.smithi192.stdout:6/105: dread - d3/d11/f1d zero size 2024-04-23T02:13:50.557 INFO:tasks.workunit.client.1.smithi192.stdout:2/136: mkdir d8/d26 0 2024-04-23T02:13:50.558 INFO:tasks.workunit.client.1.smithi192.stdout:4/327: symlink de/d15/d2a/d61/l67 0 2024-04-23T02:13:50.559 INFO:tasks.workunit.client.1.smithi192.stdout:6/106: chown d3/d15/l17 63 1 2024-04-23T02:13:50.559 INFO:tasks.workunit.client.1.smithi192.stdout:9/81: dread d0/d9/f10 [0,4194304] 0 2024-04-23T02:13:50.562 INFO:tasks.workunit.client.1.smithi192.stdout:0/160: write f5 [416098,36843] 0 2024-04-23T02:13:50.562 INFO:tasks.workunit.client.1.smithi192.stdout:4/328: read - de/d15/d53/f58 zero size 2024-04-23T02:13:50.564 INFO:tasks.workunit.client.1.smithi192.stdout:9/82: mknod d0/c21 0 2024-04-23T02:13:50.565 INFO:tasks.workunit.client.1.smithi192.stdout:6/107: mknod d3/d15/d1b/c20 0 2024-04-23T02:13:50.568 INFO:tasks.workunit.client.1.smithi192.stdout:7/148: dwrite d5/d17/d1c/d26/f28 [0,4194304] 0 2024-04-23T02:13:50.568 INFO:tasks.workunit.client.1.smithi192.stdout:0/161: chown d6/df/d16/f2e 21370797 1 2024-04-23T02:13:50.569 INFO:tasks.workunit.client.1.smithi192.stdout:6/108: readlink d3/d18/l1a 0 2024-04-23T02:13:50.570 INFO:tasks.workunit.client.1.smithi192.stdout:6/109: chown d3/l8 1214007 1 2024-04-23T02:13:50.572 INFO:tasks.workunit.client.1.smithi192.stdout:7/149: chown d5/d18/c29 4 1 2024-04-23T02:13:50.572 INFO:tasks.workunit.client.1.smithi192.stdout:0/162: mknod d6/df/d13/d25/d26/c39 0 2024-04-23T02:13:50.574 INFO:tasks.workunit.client.1.smithi192.stdout:6/110: mkdir d3/d15/d1b/d21 0 2024-04-23T02:13:50.574 INFO:tasks.workunit.client.1.smithi192.stdout:2/137: dwrite d8/d14/f15 [0,4194304] 0 2024-04-23T02:13:50.576 INFO:tasks.workunit.client.1.smithi192.stdout:0/163: chown d6/df/d13/d25/l2f 3204 1 2024-04-23T02:13:50.576 INFO:tasks.workunit.client.1.smithi192.stdout:6/111: stat d3/le 0 2024-04-23T02:13:50.577 INFO:tasks.workunit.client.1.smithi192.stdout:7/150: creat d5/d17/f2b x:0 0 0 2024-04-23T02:13:50.579 INFO:tasks.workunit.client.1.smithi192.stdout:0/164: mkdir d6/df/d13/d25/d3a 0 2024-04-23T02:13:50.581 INFO:tasks.workunit.client.1.smithi192.stdout:7/151: mknod d5/d17/c2c 0 2024-04-23T02:13:50.585 INFO:tasks.workunit.client.1.smithi192.stdout:2/138: dwrite d8/d23/f24 [0,4194304] 0 2024-04-23T02:13:50.588 INFO:tasks.workunit.client.1.smithi192.stdout:0/165: dread d6/df/d13/f31 [0,4194304] 0 2024-04-23T02:13:50.590 INFO:tasks.workunit.client.1.smithi192.stdout:9/83: readlink d0/l20 0 2024-04-23T02:13:50.591 INFO:tasks.workunit.client.1.smithi192.stdout:2/139: dread d8/d14/f15 [0,4194304] 0 2024-04-23T02:13:50.595 INFO:tasks.workunit.client.1.smithi192.stdout:7/152: creat d5/d17/d1c/f2d x:0 0 0 2024-04-23T02:13:50.595 INFO:tasks.workunit.client.1.smithi192.stdout:6/112: rename d3/fa to d3/f22 0 2024-04-23T02:13:50.599 INFO:tasks.workunit.client.1.smithi192.stdout:9/84: rename d0/d3/c1b to d0/d1e/c22 0 2024-04-23T02:13:50.599 INFO:tasks.workunit.client.1.smithi192.stdout:5/137: write d2/db/d16/d1b/f1f [1079907,33145] 0 2024-04-23T02:13:50.601 INFO:tasks.workunit.client.1.smithi192.stdout:7/153: creat d5/f2e x:0 0 0 2024-04-23T02:13:50.601 INFO:tasks.workunit.client.1.smithi192.stdout:6/113: dread d3/f9 [0,4194304] 0 2024-04-23T02:13:50.602 INFO:tasks.workunit.client.1.smithi192.stdout:6/114: chown d3/d11/f1d 407 1 2024-04-23T02:13:50.604 INFO:tasks.workunit.client.1.smithi192.stdout:7/154: write d5/d17/d1c/f24 [921745,123867] 0 2024-04-23T02:13:50.604 INFO:tasks.workunit.client.1.smithi192.stdout:4/329: write de/f18 [741844,75387] 0 2024-04-23T02:13:50.605 INFO:tasks.workunit.client.1.smithi192.stdout:2/140: mkdir d8/d26/d27 0 2024-04-23T02:13:50.607 INFO:tasks.workunit.client.1.smithi192.stdout:9/85: rmdir d0/d9 39 2024-04-23T02:13:50.607 INFO:tasks.workunit.client.1.smithi192.stdout:5/138: dread d2/d6/f12 [0,4194304] 0 2024-04-23T02:13:50.608 INFO:tasks.workunit.client.1.smithi192.stdout:7/155: truncate d5/d17/d1c/d26/f28 4909751 0 2024-04-23T02:13:50.608 INFO:tasks.workunit.client.1.smithi192.stdout:2/141: chown d8/d23/f24 1881401 1 2024-04-23T02:13:50.610 INFO:tasks.workunit.client.1.smithi192.stdout:7/156: truncate d5/d17/d1c/f24 1960255 0 2024-04-23T02:13:50.611 INFO:tasks.workunit.client.1.smithi192.stdout:6/115: write d3/d18/f1f [1653551,2495] 0 2024-04-23T02:13:50.611 INFO:tasks.workunit.client.1.smithi192.stdout:6/116: fdatasync d3/d11/f13 0 2024-04-23T02:13:50.612 INFO:tasks.workunit.client.1.smithi192.stdout:0/166: creat d6/f3b x:0 0 0 2024-04-23T02:13:50.613 INFO:tasks.workunit.client.1.smithi192.stdout:4/330: unlink de/d15/d19/d1a/f3f 0 2024-04-23T02:13:50.617 INFO:tasks.workunit.client.1.smithi192.stdout:2/142: mkdir d8/d14/d28 0 2024-04-23T02:13:50.617 INFO:tasks.workunit.client.1.smithi192.stdout:0/167: dread d6/df/d13/d25/d26/f27 [0,4194304] 0 2024-04-23T02:13:50.618 INFO:tasks.workunit.client.1.smithi192.stdout:9/86: dread d0/d9/f14 [0,4194304] 0 2024-04-23T02:13:50.620 INFO:tasks.workunit.client.1.smithi192.stdout:9/87: chown d0 708815924 1 2024-04-23T02:13:50.620 INFO:tasks.workunit.client.1.smithi192.stdout:7/157: readlink d5/d18/l19 0 2024-04-23T02:13:50.624 INFO:tasks.workunit.client.1.smithi192.stdout:6/117: write d3/f9 [1278723,49725] 0 2024-04-23T02:13:50.625 INFO:tasks.workunit.client.1.smithi192.stdout:4/331: fsync de/d15/d2a/f2c 0 2024-04-23T02:13:50.627 INFO:tasks.workunit.client.1.smithi192.stdout:0/168: dwrite d6/f3b [0,4194304] 0 2024-04-23T02:13:50.630 INFO:tasks.workunit.client.1.smithi192.stdout:2/143: rename d8/d23 to d8/d26/d29 0 2024-04-23T02:13:50.630 INFO:tasks.workunit.client.1.smithi192.stdout:0/169: dread d6/df/d16/f2e [0,4194304] 0 2024-04-23T02:13:50.630 INFO:tasks.workunit.client.1.smithi192.stdout:2/144: chown d8/d26 82454010 1 2024-04-23T02:13:50.631 INFO:tasks.workunit.client.1.smithi192.stdout:0/170: fdatasync d6/df/f30 0 2024-04-23T02:13:50.634 INFO:tasks.workunit.client.1.smithi192.stdout:9/88: write d0/d9/f14 [109222,69664] 0 2024-04-23T02:13:50.636 INFO:tasks.workunit.client.1.smithi192.stdout:6/118: mknod d3/d15/c23 0 2024-04-23T02:13:50.637 INFO:tasks.workunit.client.1.smithi192.stdout:5/139: write d2/db/d16/d1b/f29 [1203807,91653] 0 2024-04-23T02:13:50.637 INFO:tasks.workunit.client.1.smithi192.stdout:4/332: mknod de/d15/d19/d3d/d49/d26/c68 0 2024-04-23T02:13:50.641 INFO:tasks.workunit.client.1.smithi192.stdout:0/171: truncate d6/f18 632771 0 2024-04-23T02:13:50.641 INFO:tasks.workunit.client.1.smithi192.stdout:0/172: fsync d6/df/d16/f38 0 2024-04-23T02:13:50.642 INFO:tasks.workunit.client.1.smithi192.stdout:6/119: rmdir d3/d18 39 2024-04-23T02:13:50.642 INFO:tasks.workunit.client.1.smithi192.stdout:0/173: chown d6/df/d13/d25/d26/l34 12 1 2024-04-23T02:13:50.646 INFO:tasks.workunit.client.1.smithi192.stdout:6/120: readlink d3/d18/l1a 0 2024-04-23T02:13:50.648 INFO:tasks.workunit.client.1.smithi192.stdout:0/174: symlink d6/df/d13/l3c 0 2024-04-23T02:13:50.649 INFO:tasks.workunit.client.1.smithi192.stdout:0/175: read d6/df/d16/f19 [1475727,25246] 0 2024-04-23T02:13:50.649 INFO:tasks.workunit.client.1.smithi192.stdout:4/333: symlink de/d15/d19/d3d/d49/l69 0 2024-04-23T02:13:50.651 INFO:tasks.workunit.client.1.smithi192.stdout:6/121: read d3/f22 [27001,131034] 0 2024-04-23T02:13:50.652 INFO:tasks.workunit.client.1.smithi192.stdout:6/122: dread d3/f22 [0,4194304] 0 2024-04-23T02:13:50.652 INFO:tasks.workunit.client.1.smithi192.stdout:5/140: link d2/db/d16/d1b/f1f d2/f2c 0 2024-04-23T02:13:50.655 INFO:tasks.workunit.client.1.smithi192.stdout:9/89: getdents d0/d9 0 2024-04-23T02:13:50.655 INFO:tasks.workunit.client.1.smithi192.stdout:5/141: fsync d2/db/f18 0 2024-04-23T02:13:50.656 INFO:tasks.workunit.client.1.smithi192.stdout:7/158: truncate d5/d18/f1d 550947 0 2024-04-23T02:13:50.657 INFO:tasks.workunit.client.1.smithi192.stdout:2/145: truncate d8/f1b 132326 0 2024-04-23T02:13:50.657 INFO:tasks.workunit.client.1.smithi192.stdout:6/123: mknod d3/d15/d1b/c24 0 2024-04-23T02:13:50.658 INFO:tasks.workunit.client.1.smithi192.stdout:7/159: dread - d5/d18/f22 zero size 2024-04-23T02:13:50.660 INFO:tasks.workunit.client.1.smithi192.stdout:6/124: write d3/f19 [907373,93742] 0 2024-04-23T02:13:50.661 INFO:tasks.workunit.client.1.smithi192.stdout:4/334: dwrite de/d15/d2a/f5c [0,4194304] 0 2024-04-23T02:13:50.663 INFO:tasks.workunit.client.1.smithi192.stdout:6/125: chown d3 18352098 1 2024-04-23T02:13:50.663 INFO:tasks.workunit.client.1.smithi192.stdout:2/146: dread d8/fb [0,4194304] 0 2024-04-23T02:13:50.667 INFO:tasks.workunit.client.1.smithi192.stdout:7/160: stat d5/d17/c1e 0 2024-04-23T02:13:50.669 INFO:tasks.workunit.client.1.smithi192.stdout:0/176: truncate d6/f10 3893371 0 2024-04-23T02:13:50.670 INFO:tasks.workunit.client.1.smithi192.stdout:0/177: dread - d6/df/d13/d25/f2b zero size 2024-04-23T02:13:50.673 INFO:tasks.workunit.client.1.smithi192.stdout:6/126: truncate d3/f22 627208 0 2024-04-23T02:13:50.674 INFO:tasks.workunit.client.1.smithi192.stdout:9/90: write d0/d9/fb [861636,94221] 0 2024-04-23T02:13:50.675 INFO:tasks.workunit.client.1.smithi192.stdout:5/142: write d2/f5 [3357345,118992] 0 2024-04-23T02:13:50.678 INFO:tasks.workunit.client.1.smithi192.stdout:2/147: creat d8/d14/d28/f2a x:0 0 0 2024-04-23T02:13:50.682 INFO:tasks.workunit.client.1.smithi192.stdout:9/91: creat d0/d1e/f23 x:0 0 0 2024-04-23T02:13:50.684 INFO:tasks.workunit.client.1.smithi192.stdout:6/127: dwrite d3/f9 [8388608,4194304] 0 2024-04-23T02:13:50.685 INFO:tasks.workunit.client.1.smithi192.stdout:0/178: write d6/df/f14 [90329,124026] 0 2024-04-23T02:13:50.695 INFO:tasks.workunit.client.1.smithi192.stdout:5/143: mkdir d2/d6/d2d 0 2024-04-23T02:13:50.695 INFO:tasks.workunit.client.1.smithi192.stdout:2/148: dread d8/d26/d29/f24 [0,4194304] 0 2024-04-23T02:13:50.695 INFO:tasks.workunit.client.1.smithi192.stdout:4/335: dwrite de/d15/d19/d3d/f32 [4194304,4194304] 0 2024-04-23T02:13:50.695 INFO:tasks.workunit.client.1.smithi192.stdout:7/161: getdents d5/d17/d1c 0 2024-04-23T02:13:50.696 INFO:tasks.workunit.client.1.smithi192.stdout:6/128: unlink d3/d18/f1f 0 2024-04-23T02:13:50.696 INFO:tasks.workunit.client.1.smithi192.stdout:2/149: chown d8/fb 1928293678 1 2024-04-23T02:13:50.696 INFO:tasks.workunit.client.1.smithi192.stdout:7/162: chown d5/d18/l19 136561108 1 2024-04-23T02:13:50.698 INFO:tasks.workunit.client.1.smithi192.stdout:5/144: stat l1 0 2024-04-23T02:13:50.699 INFO:tasks.workunit.client.1.smithi192.stdout:9/92: symlink d0/l24 0 2024-04-23T02:13:50.701 INFO:tasks.workunit.client.1.smithi192.stdout:6/129: fdatasync d3/f9 0 2024-04-23T02:13:50.703 INFO:tasks.workunit.client.1.smithi192.stdout:4/336: mkdir de/d15/d19/d3d/d49/d26/d59/d6a 0 2024-04-23T02:13:50.705 INFO:tasks.workunit.client.1.smithi192.stdout:7/163: chown d5/d17/d1c/d26/c27 78306 1 2024-04-23T02:13:50.706 INFO:tasks.workunit.client.1.smithi192.stdout:7/164: chown d5/d18/l19 15 1 2024-04-23T02:13:50.708 INFO:tasks.workunit.client.1.smithi192.stdout:9/93: creat d0/d9/f25 x:0 0 0 2024-04-23T02:13:50.708 INFO:tasks.workunit.client.1.smithi192.stdout:0/179: write d6/df/d13/d25/d26/f27 [2895251,14911] 0 2024-04-23T02:13:50.709 INFO:tasks.workunit.client.1.smithi192.stdout:2/150: symlink d8/l2b 0 2024-04-23T02:13:50.710 INFO:tasks.workunit.client.1.smithi192.stdout:9/94: fsync d0/d3/f8 0 2024-04-23T02:13:50.711 INFO:tasks.workunit.client.1.smithi192.stdout:0/180: readlink d6/df/l29 0 2024-04-23T02:13:50.715 INFO:tasks.workunit.client.1.smithi192.stdout:6/130: link d3/d15/d1b/c20 d3/d15/d1b/c25 0 2024-04-23T02:13:50.716 INFO:tasks.workunit.client.1.smithi192.stdout:6/131: readlink l2 0 2024-04-23T02:13:50.717 INFO:tasks.workunit.client.1.smithi192.stdout:9/95: creat d0/d9/f26 x:0 0 0 2024-04-23T02:13:50.720 INFO:tasks.workunit.client.1.smithi192.stdout:7/165: dwrite d5/d18/f22 [0,4194304] 0 2024-04-23T02:13:50.720 INFO:tasks.workunit.client.1.smithi192.stdout:5/145: getdents d2/db/d16 0 2024-04-23T02:13:50.720 INFO:tasks.workunit.client.1.smithi192.stdout:8/119: sync 2024-04-23T02:13:50.720 INFO:tasks.workunit.client.1.smithi192.stdout:4/337: getdents de/d15/d2a 0 2024-04-23T02:13:50.721 INFO:tasks.workunit.client.1.smithi192.stdout:2/151: truncate f7 834759 0 2024-04-23T02:13:50.724 INFO:tasks.workunit.client.1.smithi192.stdout:9/96: mknod d0/d1e/c27 0 2024-04-23T02:13:50.727 INFO:tasks.workunit.client.1.smithi192.stdout:5/146: dread d2/db/f11 [0,4194304] 0 2024-04-23T02:13:50.727 INFO:tasks.workunit.client.1.smithi192.stdout:7/166: dread d5/d18/f20 [0,4194304] 0 2024-04-23T02:13:50.727 INFO:tasks.workunit.client.1.smithi192.stdout:8/120: creat d1/d9/f26 x:0 0 0 2024-04-23T02:13:50.728 INFO:tasks.workunit.client.1.smithi192.stdout:0/181: write d6/df/d16/f2e [945969,81760] 0 2024-04-23T02:13:50.730 INFO:tasks.workunit.client.1.smithi192.stdout:6/132: mknod d3/d15/d1b/d21/c26 0 2024-04-23T02:13:50.731 INFO:tasks.workunit.client.1.smithi192.stdout:9/97: rename d0/d3/fa to d0/d3/f28 0 2024-04-23T02:13:50.731 INFO:tasks.workunit.client.1.smithi192.stdout:8/121: chown d1/f3 79859 1 2024-04-23T02:13:50.734 INFO:tasks.workunit.client.1.smithi192.stdout:0/182: write d6/df/d16/f2e [2027404,94784] 0 2024-04-23T02:13:50.737 INFO:tasks.workunit.client.1.smithi192.stdout:4/338: rename de/d15/d19/d1a/l5e to de/l6b 0 2024-04-23T02:13:50.737 INFO:tasks.workunit.client.1.smithi192.stdout:9/98: truncate d0/f16 1399435 0 2024-04-23T02:13:50.738 INFO:tasks.workunit.client.1.smithi192.stdout:8/122: symlink d1/d19/l27 0 2024-04-23T02:13:50.738 INFO:tasks.workunit.client.1.smithi192.stdout:6/133: mknod d3/d15/c27 0 2024-04-23T02:13:50.739 INFO:tasks.workunit.client.1.smithi192.stdout:0/183: write d6/df/f30 [831925,35526] 0 2024-04-23T02:13:50.741 INFO:tasks.workunit.client.1.smithi192.stdout:4/339: read de/d15/d2a/f5c [3985140,7339] 0 2024-04-23T02:13:50.742 INFO:tasks.workunit.client.1.smithi192.stdout:9/99: mknod d0/c29 0 2024-04-23T02:13:50.743 INFO:tasks.workunit.client.1.smithi192.stdout:2/152: dwrite d8/fb [0,4194304] 0 2024-04-23T02:13:50.745 INFO:tasks.workunit.client.1.smithi192.stdout:5/147: dread d2/db/f20 [0,4194304] 0 2024-04-23T02:13:50.746 INFO:tasks.workunit.client.1.smithi192.stdout:0/184: truncate d6/df/f30 978680 0 2024-04-23T02:13:50.746 INFO:tasks.workunit.client.1.smithi192.stdout:8/123: creat d1/d9/f28 x:0 0 0 2024-04-23T02:13:50.748 INFO:tasks.workunit.client.1.smithi192.stdout:5/148: write d2/f2a [957114,33372] 0 2024-04-23T02:13:50.751 INFO:tasks.workunit.client.1.smithi192.stdout:6/134: mkdir d3/d28 0 2024-04-23T02:13:50.752 INFO:tasks.workunit.client.1.smithi192.stdout:4/340: mknod de/d15/d19/d1a/c6c 0 2024-04-23T02:13:50.753 INFO:tasks.workunit.client.1.smithi192.stdout:8/124: symlink d1/d19/l29 0 2024-04-23T02:13:50.754 INFO:tasks.workunit.client.1.smithi192.stdout:0/185: creat d6/df/d13/d25/f3d x:0 0 0 2024-04-23T02:13:50.755 INFO:tasks.workunit.client.1.smithi192.stdout:7/167: write d5/d18/f1d [456312,54543] 0 2024-04-23T02:13:50.757 INFO:tasks.workunit.client.1.smithi192.stdout:9/100: dread d0/d3/f28 [0,4194304] 0 2024-04-23T02:13:50.757 INFO:tasks.workunit.client.1.smithi192.stdout:5/149: fdatasync d2/d6/f12 0 2024-04-23T02:13:50.757 INFO:tasks.workunit.client.1.smithi192.stdout:7/168: readlink l4 0 2024-04-23T02:13:50.757 INFO:tasks.workunit.client.1.smithi192.stdout:0/186: chown d6/lb 737911029 1 2024-04-23T02:13:50.825 INFO:tasks.workunit.client.1.smithi192.stdout:8/125: rename d1/d9/f28 to d1/d9/f2a 0 2024-04-23T02:13:50.825 INFO:tasks.workunit.client.1.smithi192.stdout:8/126: fsync d1/df/f1c 0 2024-04-23T02:13:50.826 INFO:tasks.workunit.client.1.smithi192.stdout:2/153: creat d8/f2c x:0 0 0 2024-04-23T02:13:50.827 INFO:tasks.workunit.client.1.smithi192.stdout:2/154: fdatasync d8/f2c 0 2024-04-23T02:13:50.829 INFO:tasks.workunit.client.1.smithi192.stdout:1/49: sync 2024-04-23T02:13:50.830 INFO:tasks.workunit.client.1.smithi192.stdout:3/84: sync 2024-04-23T02:13:50.830 INFO:tasks.workunit.client.1.smithi192.stdout:9/101: symlink d0/d1e/l2a 0 2024-04-23T02:13:50.830 INFO:tasks.workunit.client.1.smithi192.stdout:9/102: fdatasync d0/d9/f25 0 2024-04-23T02:13:50.831 INFO:tasks.workunit.client.1.smithi192.stdout:0/187: symlink d6/df/d13/d25/l3e 0 2024-04-23T02:13:50.831 INFO:tasks.workunit.client.1.smithi192.stdout:3/85: write d1/d3/d6/d7/f11 [1674589,128520] 0 2024-04-23T02:13:50.831 INFO:tasks.workunit.client.1.smithi192.stdout:9/103: read d0/d3/f8 [922522,35836] 0 2024-04-23T02:13:50.832 INFO:tasks.workunit.client.1.smithi192.stdout:7/169: rename d5/d18 to d5/d17/d2f 0 2024-04-23T02:13:50.833 INFO:tasks.workunit.client.1.smithi192.stdout:7/170: dread - d5/d17/f2b zero size 2024-04-23T02:13:50.835 INFO:tasks.workunit.client.1.smithi192.stdout:8/127: mknod d1/df/c2b 0 2024-04-23T02:13:50.836 INFO:tasks.workunit.client.1.smithi192.stdout:5/150: mkdir d2/d2e 0 2024-04-23T02:13:50.836 INFO:tasks.workunit.client.1.smithi192.stdout:8/128: truncate d1/d9/f2a 393893 0 2024-04-23T02:13:50.837 INFO:tasks.workunit.client.1.smithi192.stdout:2/155: mknod d8/d26/d29/c2d 0 2024-04-23T02:13:50.839 INFO:tasks.workunit.client.1.smithi192.stdout:1/50: write f4 [990061,88212] 0 2024-04-23T02:13:50.840 INFO:tasks.workunit.client.1.smithi192.stdout:0/188: unlink d6/la 0 2024-04-23T02:13:50.843 INFO:tasks.workunit.client.1.smithi192.stdout:4/341: dwrite de/d15/f31 [0,4194304] 0 2024-04-23T02:13:50.843 INFO:tasks.workunit.client.1.smithi192.stdout:3/86: creat d1/d3/f19 x:0 0 0 2024-04-23T02:13:50.844 INFO:tasks.workunit.client.1.smithi192.stdout:7/171: mkdir d5/d17/d1c/d30 0 2024-04-23T02:13:50.846 INFO:tasks.workunit.client.1.smithi192.stdout:3/87: truncate d1/d3/d6/d7/f13 1571355 0 2024-04-23T02:13:50.846 INFO:tasks.workunit.client.1.smithi192.stdout:8/129: rename d1/d19 to d1/d12/d23/d2c 0 2024-04-23T02:13:50.849 INFO:tasks.workunit.client.1.smithi192.stdout:0/189: mknod d6/df/d13/d25/d26/c3f 0 2024-04-23T02:13:50.849 INFO:tasks.workunit.client.1.smithi192.stdout:5/151: mknod d2/db/d2b/c2f 0 2024-04-23T02:13:50.851 INFO:tasks.workunit.client.1.smithi192.stdout:4/342: creat de/d15/d19/d1a/f6d x:0 0 0 2024-04-23T02:13:50.854 INFO:tasks.workunit.client.1.smithi192.stdout:7/172: mkdir d5/d17/d2f/d31 0 2024-04-23T02:13:50.854 INFO:tasks.workunit.client.1.smithi192.stdout:3/88: dread d1/d3/d6/d7/f11 [0,4194304] 0 2024-04-23T02:13:50.863 INFO:tasks.workunit.client.1.smithi192.stdout:4/343: mknod de/d15/d19/d3d/d49/d26/c6e 0 2024-04-23T02:13:50.865 INFO:tasks.workunit.client.1.smithi192.stdout:5/152: dread d2/fd [0,4194304] 0 2024-04-23T02:13:50.866 INFO:tasks.workunit.client.1.smithi192.stdout:3/89: truncate d1/d3/d6/f10 1145227 0 2024-04-23T02:13:50.867 INFO:tasks.workunit.client.1.smithi192.stdout:8/130: write d1/d9/fa [102485,13673] 0 2024-04-23T02:13:50.868 INFO:tasks.workunit.client.1.smithi192.stdout:1/51: link f1 dc/fd 0 2024-04-23T02:13:50.870 INFO:tasks.workunit.client.1.smithi192.stdout:9/104: getdents d0/d1e 0 2024-04-23T02:13:50.871 INFO:tasks.workunit.client.1.smithi192.stdout:0/190: mkdir d6/d40 0 2024-04-23T02:13:50.871 INFO:tasks.workunit.client.1.smithi192.stdout:8/131: write d1/df/f1c [364993,46971] 0 2024-04-23T02:13:50.873 INFO:tasks.workunit.client.1.smithi192.stdout:7/173: creat d5/d17/d2f/d31/f32 x:0 0 0 2024-04-23T02:13:50.873 INFO:tasks.workunit.client.1.smithi192.stdout:9/105: dread - d0/d9/f25 zero size 2024-04-23T02:13:50.874 INFO:tasks.workunit.client.1.smithi192.stdout:2/156: dwrite d8/f1b [0,4194304] 0 2024-04-23T02:13:50.876 INFO:tasks.workunit.client.1.smithi192.stdout:4/344: dread de/d15/d19/d1a/f3a [0,4194304] 0 2024-04-23T02:13:50.877 INFO:tasks.workunit.client.1.smithi192.stdout:5/153: mkdir d2/d6/d2d/d30 0 2024-04-23T02:13:50.878 INFO:tasks.workunit.client.1.smithi192.stdout:8/132: readlink d1/d12/d23/d2c/l29 0 2024-04-23T02:13:50.880 INFO:tasks.workunit.client.1.smithi192.stdout:7/174: creat d5/d17/d2f/f33 x:0 0 0 2024-04-23T02:13:50.881 INFO:tasks.workunit.client.1.smithi192.stdout:8/133: write d1/d12/f1e [8379260,107938] 0 2024-04-23T02:13:50.881 INFO:tasks.workunit.client.1.smithi192.stdout:1/52: dread f3 [0,4194304] 0 2024-04-23T02:13:50.883 INFO:tasks.workunit.client.1.smithi192.stdout:7/175: dread - d5/f2e zero size 2024-04-23T02:13:50.884 INFO:tasks.workunit.client.1.smithi192.stdout:8/134: write d1/d12/f24 [395405,126588] 0 2024-04-23T02:13:50.888 INFO:tasks.workunit.client.1.smithi192.stdout:5/154: rmdir d2/db/d16 39 2024-04-23T02:13:50.889 INFO:tasks.workunit.client.1.smithi192.stdout:7/176: fdatasync d5/d17/d2f/d31/f32 0 2024-04-23T02:13:50.893 INFO:tasks.workunit.client.1.smithi192.stdout:7/177: write d5/d17/d2f/f20 [1168621,114348] 0 2024-04-23T02:13:50.897 INFO:tasks.workunit.client.1.smithi192.stdout:0/191: dwrite d6/df/d16/f19 [0,4194304] 0 2024-04-23T02:13:50.898 INFO:tasks.workunit.client.1.smithi192.stdout:2/157: rename d8/d14/d28 to d8/d2e 0 2024-04-23T02:13:50.898 INFO:tasks.workunit.client.1.smithi192.stdout:3/90: write d1/d3/d6/f10 [1628891,120633] 0 2024-04-23T02:13:50.900 INFO:tasks.workunit.client.1.smithi192.stdout:1/53: truncate dc/fd 910047 0 2024-04-23T02:13:50.902 INFO:tasks.workunit.client.0.smithi071.stdout:1/31: sync 2024-04-23T02:13:50.903 INFO:tasks.workunit.client.1.smithi192.stdout:2/158: chown d8/fb 382656 1 2024-04-23T02:13:50.903 INFO:tasks.workunit.client.1.smithi192.stdout:4/345: write de/f21 [94533,4476] 0 2024-04-23T02:13:50.903 INFO:tasks.workunit.client.0.smithi071.stdout:0/48: sync 2024-04-23T02:13:50.903 INFO:tasks.workunit.client.0.smithi071.stdout:2/23: sync 2024-04-23T02:13:50.904 INFO:tasks.workunit.client.0.smithi071.stdout:9/37: sync 2024-04-23T02:13:50.904 INFO:tasks.workunit.client.0.smithi071.stdout:2/24: dread - no filename 2024-04-23T02:13:50.904 INFO:tasks.workunit.client.0.smithi071.stdout:4/70: sync 2024-04-23T02:13:50.904 INFO:tasks.workunit.client.0.smithi071.stdout:7/70: sync 2024-04-23T02:13:50.904 INFO:tasks.workunit.client.0.smithi071.stdout:6/31: sync 2024-04-23T02:13:50.904 INFO:tasks.workunit.client.0.smithi071.stdout:3/7: sync 2024-04-23T02:13:50.904 INFO:tasks.workunit.client.0.smithi071.stdout:3/8: dread - no filename 2024-04-23T02:13:50.905 INFO:tasks.workunit.client.0.smithi071.stdout:8/60: sync 2024-04-23T02:13:50.905 INFO:tasks.workunit.client.0.smithi071.stdout:5/73: sync 2024-04-23T02:13:50.905 INFO:tasks.workunit.client.0.smithi071.stdout:3/9: chown . 845 1 2024-04-23T02:13:50.905 INFO:tasks.workunit.client.0.smithi071.stdout:3/10: fdatasync - no filename 2024-04-23T02:13:50.905 INFO:tasks.workunit.client.0.smithi071.stdout:3/11: dwrite - no filename 2024-04-23T02:13:50.905 INFO:tasks.workunit.client.0.smithi071.stdout:3/12: link - no file 2024-04-23T02:13:50.906 INFO:tasks.workunit.client.0.smithi071.stdout:1/32: symlink l8 0 2024-04-23T02:13:50.907 INFO:tasks.workunit.client.1.smithi192.stdout:8/135: dwrite d1/d12/f24 [0,4194304] 0 2024-04-23T02:13:50.907 INFO:tasks.workunit.client.1.smithi192.stdout:2/159: write f3 [2833860,2759] 0 2024-04-23T02:13:50.907 INFO:tasks.workunit.client.1.smithi192.stdout:0/192: symlink d6/df/d13/d25/d26/l41 0 2024-04-23T02:13:50.910 INFO:tasks.workunit.client.0.smithi071.stdout:0/49: read d0/f1 [113221,36673] 0 2024-04-23T02:13:50.911 INFO:tasks.workunit.client.0.smithi071.stdout:2/25: creat d0/f5 x:0 0 0 2024-04-23T02:13:50.912 INFO:tasks.workunit.client.1.smithi192.stdout:8/136: rename d1 to d1/df/d2d 22 2024-04-23T02:13:50.912 INFO:tasks.workunit.client.0.smithi071.stdout:7/71: truncate d3/fc 277990 0 2024-04-23T02:13:50.913 INFO:tasks.workunit.client.1.smithi192.stdout:5/155: dread d2/db/f11 [0,4194304] 0 2024-04-23T02:13:50.913 INFO:tasks.workunit.client.0.smithi071.stdout:7/72: rename d3/d17 to d3/d17/d1a 22 2024-04-23T02:13:50.914 INFO:tasks.workunit.client.1.smithi192.stdout:8/137: fdatasync d1/d9/f22 0 2024-04-23T02:13:50.915 INFO:tasks.workunit.client.0.smithi071.stdout:3/13: creat f0 x:0 0 0 2024-04-23T02:13:50.918 INFO:tasks.workunit.client.0.smithi071.stdout:8/61: rename dc/cd to dc/ce 0 2024-04-23T02:13:50.918 INFO:tasks.workunit.client.0.smithi071.stdout:3/14: write f0 [592738,104388] 0 2024-04-23T02:13:50.920 INFO:tasks.workunit.client.0.smithi071.stdout:4/71: creat d2/d15/f1a x:0 0 0 2024-04-23T02:13:50.920 INFO:tasks.workunit.client.1.smithi192.stdout:4/346: unlink de/d15/d19/d3d/d49/l69 0 2024-04-23T02:13:50.921 INFO:tasks.workunit.client.0.smithi071.stdout:4/72: chown d2/d7/fc 71 1 2024-04-23T02:13:50.921 INFO:tasks.workunit.client.1.smithi192.stdout:3/91: dwrite d1/d3/f19 [0,4194304] 0 2024-04-23T02:13:50.922 INFO:tasks.workunit.client.0.smithi071.stdout:0/50: mkdir d0/d3/dc 0 2024-04-23T02:13:50.922 INFO:tasks.workunit.client.1.smithi192.stdout:7/178: dread d5/d17/d2f/f22 [0,4194304] 0 2024-04-23T02:13:50.923 INFO:tasks.workunit.client.0.smithi071.stdout:2/26: creat d0/f6 x:0 0 0 2024-04-23T02:13:50.923 INFO:tasks.workunit.client.0.smithi071.stdout:7/73: rmdir d3 39 2024-04-23T02:13:50.924 INFO:tasks.workunit.client.1.smithi192.stdout:0/193: creat d6/df/f42 x:0 0 0 2024-04-23T02:13:50.924 INFO:tasks.workunit.client.1.smithi192.stdout:2/160: rename d8/f12 to d8/d26/d27/f2f 0 2024-04-23T02:13:50.925 INFO:tasks.workunit.client.0.smithi071.stdout:5/74: creat d3/f1f x:0 0 0 2024-04-23T02:13:50.925 INFO:tasks.workunit.client.0.smithi071.stdout:2/27: read - d0/f6 zero size 2024-04-23T02:13:50.926 INFO:tasks.workunit.client.1.smithi192.stdout:3/92: read d1/d3/f19 [2839558,125159] 0 2024-04-23T02:13:50.927 INFO:tasks.workunit.client.1.smithi192.stdout:7/179: read d5/d17/d2f/f20 [2254802,9631] 0 2024-04-23T02:13:50.929 INFO:tasks.workunit.client.0.smithi071.stdout:8/62: fsync f8 0 2024-04-23T02:13:50.930 INFO:tasks.workunit.client.1.smithi192.stdout:8/138: dwrite d1/f3 [0,4194304] 0 2024-04-23T02:13:50.930 INFO:tasks.workunit.client.0.smithi071.stdout:9/38: dwrite d1/fc [0,4194304] 0 2024-04-23T02:13:50.931 INFO:tasks.workunit.client.0.smithi071.stdout:0/51: symlink d0/d3/ld 0 2024-04-23T02:13:50.933 INFO:tasks.workunit.client.0.smithi071.stdout:3/15: mknod c1 0 2024-04-23T02:13:50.935 INFO:tasks.workunit.client.1.smithi192.stdout:2/161: symlink d8/d14/l30 0 2024-04-23T02:13:50.935 INFO:tasks.workunit.client.1.smithi192.stdout:2/162: fdatasync d8/f1d 0 2024-04-23T02:13:50.935 INFO:tasks.workunit.client.1.smithi192.stdout:0/194: symlink d6/df/d13/l43 0 2024-04-23T02:13:50.935 INFO:tasks.workunit.client.1.smithi192.stdout:5/156: link d2/c17 d2/d6/d2d/d30/c31 0 2024-04-23T02:13:50.936 INFO:tasks.workunit.client.0.smithi071.stdout:2/28: chown d0/c3 361180 1 2024-04-23T02:13:50.936 INFO:tasks.workunit.client.0.smithi071.stdout:7/74: chown d3/f12 0 1 2024-04-23T02:13:50.937 INFO:tasks.workunit.client.1.smithi192.stdout:5/157: fdatasync d2/db/f13 0 2024-04-23T02:13:50.937 INFO:tasks.workunit.client.0.smithi071.stdout:4/73: rename d2/d7/f9 to d2/d7/d18/f1b 0 2024-04-23T02:13:50.937 INFO:tasks.workunit.client.0.smithi071.stdout:3/16: write f0 [31435,52926] 0 2024-04-23T02:13:50.939 INFO:tasks.workunit.client.0.smithi071.stdout:6/32: getdents d6 0 2024-04-23T02:13:50.939 INFO:tasks.workunit.client.1.smithi192.stdout:8/139: rename d1/d9/c17 to d1/d12/d23/d2c/c2e 0 2024-04-23T02:13:50.940 INFO:tasks.workunit.client.0.smithi071.stdout:2/29: chown d0 167 1 2024-04-23T02:13:50.941 INFO:tasks.workunit.client.1.smithi192.stdout:3/93: mkdir d1/d3/d17/d1a 0 2024-04-23T02:13:50.942 INFO:tasks.workunit.client.0.smithi071.stdout:1/33: dwrite f2 [0,4194304] 0 2024-04-23T02:13:50.942 INFO:tasks.workunit.client.0.smithi071.stdout:4/74: write d2/d15/f1a [758939,80485] 0 2024-04-23T02:13:50.943 INFO:tasks.workunit.client.1.smithi192.stdout:1/54: dwrite dc/fd [0,4194304] 0 2024-04-23T02:13:50.945 INFO:tasks.workunit.client.0.smithi071.stdout:8/63: creat dc/ff x:0 0 0 2024-04-23T02:13:50.946 INFO:tasks.workunit.client.1.smithi192.stdout:2/163: rename d8/d2e to d8/d26/d27/d31 0 2024-04-23T02:13:50.946 INFO:tasks.workunit.client.1.smithi192.stdout:8/140: mknod d1/d12/d23/d2c/c2f 0 2024-04-23T02:13:50.948 INFO:tasks.workunit.client.1.smithi192.stdout:2/164: write d8/d14/f1a [784426,23910] 0 2024-04-23T02:13:50.948 INFO:tasks.workunit.client.1.smithi192.stdout:8/141: chown d1/d9/f11 443 1 2024-04-23T02:13:50.949 INFO:tasks.workunit.client.1.smithi192.stdout:5/158: mkdir d2/d32 0 2024-04-23T02:13:50.950 INFO:tasks.workunit.client.1.smithi192.stdout:3/94: mkdir d1/d3/d6/d7/d1b 0 2024-04-23T02:13:50.952 INFO:tasks.workunit.client.1.smithi192.stdout:0/195: rmdir d6/d40 0 2024-04-23T02:13:50.953 INFO:tasks.workunit.client.0.smithi071.stdout:5/75: creat d3/d1d/f20 x:0 0 0 2024-04-23T02:13:50.955 INFO:tasks.workunit.client.1.smithi192.stdout:7/180: getdents d5/d17/d1c/d26 0 2024-04-23T02:13:50.959 INFO:tasks.workunit.client.0.smithi071.stdout:1/34: mknod c9 0 2024-04-23T02:13:50.959 INFO:tasks.workunit.client.0.smithi071.stdout:6/33: creat d6/fc x:0 0 0 2024-04-23T02:13:50.960 INFO:tasks.workunit.client.1.smithi192.stdout:3/95: unlink d1/f18 0 2024-04-23T02:13:50.960 INFO:tasks.workunit.client.1.smithi192.stdout:1/55: symlink dc/le 0 2024-04-23T02:13:50.960 INFO:tasks.workunit.client.1.smithi192.stdout:2/165: rename d8/fb to d8/d25/f32 0 2024-04-23T02:13:50.961 INFO:tasks.workunit.client.0.smithi071.stdout:8/64: mkdir dc/d10 0 2024-04-23T02:13:50.961 INFO:tasks.workunit.client.1.smithi192.stdout:8/142: write d1/d12/f1e [1415565,127763] 0 2024-04-23T02:13:50.961 INFO:tasks.workunit.client.1.smithi192.stdout:7/181: truncate d5/d17/d1c/f2d 566792 0 2024-04-23T02:13:50.962 INFO:tasks.workunit.client.0.smithi071.stdout:3/17: dwrite f0 [0,4194304] 0 2024-04-23T02:13:50.964 INFO:tasks.workunit.client.0.smithi071.stdout:2/30: mkdir d0/d2/d7 0 2024-04-23T02:13:50.966 INFO:tasks.workunit.client.0.smithi071.stdout:5/76: truncate d3/f12 465804 0 2024-04-23T02:13:50.966 INFO:tasks.workunit.client.0.smithi071.stdout:4/75: link d2/d7/le d2/l1c 0 2024-04-23T02:13:50.967 INFO:tasks.workunit.client.1.smithi192.stdout:5/159: creat d2/d32/f33 x:0 0 0 2024-04-23T02:13:50.967 INFO:tasks.workunit.client.1.smithi192.stdout:7/182: read d5/d17/d1c/d26/f28 [1479833,90002] 0 2024-04-23T02:13:50.969 INFO:tasks.workunit.client.0.smithi071.stdout:3/18: dread f0 [0,4194304] 0 2024-04-23T02:13:50.970 INFO:tasks.workunit.client.0.smithi071.stdout:2/31: rename d0/c3 to d0/d2/c8 0 2024-04-23T02:13:50.972 INFO:tasks.workunit.client.0.smithi071.stdout:8/65: truncate f8 3294978 0 2024-04-23T02:13:50.973 INFO:tasks.workunit.client.0.smithi071.stdout:9/39: write d1/fe [629188,89466] 0 2024-04-23T02:13:50.974 INFO:tasks.workunit.client.0.smithi071.stdout:4/76: mkdir d2/d15/d1d 0 2024-04-23T02:13:50.974 INFO:tasks.workunit.client.0.smithi071.stdout:1/35: fdatasync f2 0 2024-04-23T02:13:50.975 INFO:tasks.workunit.client.0.smithi071.stdout:5/77: truncate d3/da/fd 833414 0 2024-04-23T02:13:50.976 INFO:tasks.workunit.client.1.smithi192.stdout:0/196: dwrite d6/df/f14 [0,4194304] 0 2024-04-23T02:13:50.977 INFO:tasks.workunit.client.1.smithi192.stdout:7/183: creat d5/d17/d1c/d26/f34 x:0 0 0 2024-04-23T02:13:50.977 INFO:tasks.workunit.client.0.smithi071.stdout:2/32: creat d0/d2/f9 x:0 0 0 2024-04-23T02:13:50.977 INFO:tasks.workunit.client.0.smithi071.stdout:8/66: read f6 [43797,103010] 0 2024-04-23T02:13:50.977 INFO:tasks.workunit.client.0.smithi071.stdout:3/19: dread f0 [0,4194304] 0 2024-04-23T02:13:50.978 INFO:tasks.workunit.client.0.smithi071.stdout:2/33: dread - d0/f6 zero size 2024-04-23T02:13:50.978 INFO:tasks.workunit.client.0.smithi071.stdout:9/40: unlink d1/cf 0 2024-04-23T02:13:50.978 INFO:tasks.workunit.client.1.smithi192.stdout:3/96: dread d1/d3/d6/d7/f13 [0,4194304] 0 2024-04-23T02:13:50.979 INFO:tasks.workunit.client.0.smithi071.stdout:2/34: truncate d0/f6 977064 0 2024-04-23T02:13:50.980 INFO:tasks.workunit.client.0.smithi071.stdout:2/35: dread - d0/d2/f9 zero size 2024-04-23T02:13:50.980 INFO:tasks.workunit.client.1.smithi192.stdout:5/160: rename d2/d6/d2d/d30/c31 to d2/d2e/c34 0 2024-04-23T02:13:50.981 INFO:tasks.workunit.client.0.smithi071.stdout:1/36: creat fa x:0 0 0 2024-04-23T02:13:50.982 INFO:tasks.workunit.client.0.smithi071.stdout:2/36: truncate d0/f5 62882 0 2024-04-23T02:13:50.983 INFO:tasks.workunit.client.0.smithi071.stdout:4/77: creat d2/d15/f1e x:0 0 0 2024-04-23T02:13:50.983 INFO:tasks.workunit.client.0.smithi071.stdout:4/78: chown d2/d7/ff 0 1 2024-04-23T02:13:50.983 INFO:tasks.workunit.client.0.smithi071.stdout:2/37: fdatasync d0/d2/f9 0 2024-04-23T02:13:50.984 INFO:tasks.workunit.client.1.smithi192.stdout:3/97: fdatasync d1/d3/d6/f10 0 2024-04-23T02:13:50.985 INFO:tasks.workunit.client.0.smithi071.stdout:2/38: stat d0/c4 0 2024-04-23T02:13:50.985 INFO:tasks.workunit.client.0.smithi071.stdout:4/79: write d2/d15/f1a [1745258,1830] 0 2024-04-23T02:13:50.986 INFO:tasks.workunit.client.0.smithi071.stdout:2/39: dread - d0/d2/f9 zero size 2024-04-23T02:13:50.986 INFO:tasks.workunit.client.1.smithi192.stdout:2/166: dwrite d8/f1d [0,4194304] 0 2024-04-23T02:13:50.988 INFO:tasks.workunit.client.0.smithi071.stdout:2/40: rename d0/d2 to d0/d2/da 22 2024-04-23T02:13:50.988 INFO:tasks.workunit.client.1.smithi192.stdout:5/161: creat d2/d32/f35 x:0 0 0 2024-04-23T02:13:50.989 INFO:tasks.workunit.client.1.smithi192.stdout:0/197: write d6/df/d13/d25/f2b [313815,129599] 0 2024-04-23T02:13:50.989 INFO:tasks.workunit.client.1.smithi192.stdout:7/184: creat d5/d17/d1c/d30/f35 x:0 0 0 2024-04-23T02:13:50.989 INFO:tasks.workunit.client.1.smithi192.stdout:2/167: readlink l5 0 2024-04-23T02:13:50.990 INFO:tasks.workunit.client.0.smithi071.stdout:5/78: symlink d3/df/d10/l21 0 2024-04-23T02:13:50.990 INFO:tasks.workunit.client.0.smithi071.stdout:8/67: fdatasync f6 0 2024-04-23T02:13:50.990 INFO:tasks.workunit.client.0.smithi071.stdout:2/41: read d0/f5 [45505,64190] 0 2024-04-23T02:13:50.991 INFO:tasks.workunit.client.1.smithi192.stdout:8/143: dwrite d1/d9/f2a [0,4194304] 0 2024-04-23T02:13:50.995 INFO:tasks.workunit.client.0.smithi071.stdout:3/20: symlink l2 0 2024-04-23T02:13:50.997 INFO:tasks.workunit.client.0.smithi071.stdout:9/41: fdatasync d1/fa 0 2024-04-23T02:13:50.997 INFO:tasks.workunit.client.1.smithi192.stdout:0/198: chown d6/df/d13/d25/d26/l34 8509 1 2024-04-23T02:13:50.997 INFO:tasks.workunit.client.1.smithi192.stdout:8/144: write d1/d9/fa [1329188,60843] 0 2024-04-23T02:13:50.997 INFO:tasks.workunit.client.1.smithi192.stdout:5/162: chown d2/d6/d14 228698885 1 2024-04-23T02:13:50.999 INFO:tasks.workunit.client.1.smithi192.stdout:8/145: chown d1/c1b 25 1 2024-04-23T02:13:50.999 INFO:tasks.workunit.client.1.smithi192.stdout:2/168: mknod d8/d26/d29/c33 0 2024-04-23T02:13:51.000 INFO:tasks.workunit.client.1.smithi192.stdout:0/199: write d6/df/d16/f19 [3993561,87640] 0 2024-04-23T02:13:51.001 INFO:tasks.workunit.client.0.smithi071.stdout:0/52: write d0/f2 [2827879,109298] 0 2024-04-23T02:13:51.001 INFO:tasks.workunit.client.0.smithi071.stdout:4/80: truncate d2/d7/f16 410366 0 2024-04-23T02:13:51.004 INFO:tasks.workunit.client.0.smithi071.stdout:3/21: write f0 [1929370,25568] 0 2024-04-23T02:13:51.004 INFO:tasks.workunit.client.0.smithi071.stdout:8/68: dwrite dc/ff [0,4194304] 0 2024-04-23T02:13:51.006 INFO:tasks.workunit.client.0.smithi071.stdout:3/22: write f0 [2809198,34535] 0 2024-04-23T02:13:51.006 INFO:tasks.workunit.client.1.smithi192.stdout:7/185: dread d5/d17/d1c/d26/f28 [0,4194304] 0 2024-04-23T02:13:51.013 INFO:tasks.workunit.client.1.smithi192.stdout:8/146: dwrite d1/d12/f24 [0,4194304] 0 2024-04-23T02:13:51.013 INFO:tasks.workunit.client.0.smithi071.stdout:4/81: creat d2/d7/f1f x:0 0 0 2024-04-23T02:13:51.018 INFO:tasks.workunit.client.0.smithi071.stdout:3/23: symlink l3 0 2024-04-23T02:13:51.018 INFO:tasks.workunit.client.0.smithi071.stdout:6/34: dwrite f5 [0,4194304] 0 2024-04-23T02:13:51.020 INFO:tasks.workunit.client.0.smithi071.stdout:2/42: rmdir d0 39 2024-04-23T02:13:51.022 INFO:tasks.workunit.client.0.smithi071.stdout:5/79: rename d3/da/f15 to d3/f22 0 2024-04-23T02:13:51.023 INFO:tasks.workunit.client.1.smithi192.stdout:5/163: mknod d2/d6/c36 0 2024-04-23T02:13:51.023 INFO:tasks.workunit.client.1.smithi192.stdout:2/169: mknod d8/c34 0 2024-04-23T02:13:51.028 INFO:tasks.workunit.client.1.smithi192.stdout:8/147: readlink d1/l21 0 2024-04-23T02:13:51.028 INFO:tasks.workunit.client.1.smithi192.stdout:0/200: rename d6/df/d16 to d6/d44 0 2024-04-23T02:13:51.030 INFO:tasks.workunit.client.1.smithi192.stdout:7/186: link d5/le d5/d17/d2f/l36 0 2024-04-23T02:13:51.032 INFO:tasks.workunit.client.1.smithi192.stdout:8/148: mknod d1/d12/d23/d2c/c30 0 2024-04-23T02:13:51.033 INFO:tasks.workunit.client.0.smithi071.stdout:8/69: dwrite f6 [4194304,4194304] 0 2024-04-23T02:13:51.033 INFO:tasks.workunit.client.1.smithi192.stdout:2/170: rename d8/d26/d29/c33 to d8/d26/d27/c35 0 2024-04-23T02:13:51.033 INFO:tasks.workunit.client.0.smithi071.stdout:8/70: chown dc 21931 1 2024-04-23T02:13:51.034 INFO:tasks.workunit.client.0.smithi071.stdout:7/75: dwrite d3/fc [0,4194304] 0 2024-04-23T02:13:51.035 INFO:tasks.workunit.client.1.smithi192.stdout:0/201: mkdir d6/d44/d45 0 2024-04-23T02:13:51.037 INFO:tasks.workunit.client.1.smithi192.stdout:7/187: dread - d5/d17/d2f/f33 zero size 2024-04-23T02:13:51.037 INFO:tasks.workunit.client.1.smithi192.stdout:2/171: dread - d8/f2c zero size 2024-04-23T02:13:51.038 INFO:tasks.workunit.client.1.smithi192.stdout:3/98: dwrite d1/d3/d6/d7/f11 [0,4194304] 0 2024-04-23T02:13:51.038 INFO:tasks.workunit.client.1.smithi192.stdout:8/149: mknod d1/d12/d23/d2c/c31 0 2024-04-23T02:13:51.041 INFO:tasks.workunit.client.1.smithi192.stdout:7/188: readlink d5/d17/d2f/l23 0 2024-04-23T02:13:51.042 INFO:tasks.workunit.client.1.smithi192.stdout:5/164: dwrite d2/d32/f33 [0,4194304] 0 2024-04-23T02:13:51.042 INFO:tasks.workunit.client.1.smithi192.stdout:8/150: stat d1/df 0 2024-04-23T02:13:51.044 INFO:tasks.workunit.client.1.smithi192.stdout:7/189: read - d5/f2e zero size 2024-04-23T02:13:51.046 INFO:tasks.workunit.client.1.smithi192.stdout:5/165: read d2/f5 [2272346,16222] 0 2024-04-23T02:13:51.046 INFO:tasks.workunit.client.1.smithi192.stdout:8/151: symlink d1/df/l32 0 2024-04-23T02:13:51.047 INFO:tasks.workunit.client.1.smithi192.stdout:7/190: fsync d5/d17/d1c/d30/f35 0 2024-04-23T02:13:51.048 INFO:tasks.workunit.client.1.smithi192.stdout:8/152: chown d1/d12/d23 9548408 1 2024-04-23T02:13:51.049 INFO:tasks.workunit.client.1.smithi192.stdout:8/153: write d1/fe [21410,89383] 0 2024-04-23T02:13:51.053 INFO:tasks.workunit.client.1.smithi192.stdout:0/202: rename d6/df/d35/f36 to d6/d44/f46 0 2024-04-23T02:13:51.056 INFO:tasks.workunit.client.1.smithi192.stdout:7/191: rmdir d5/d17/d1c 39 2024-04-23T02:13:51.059 INFO:tasks.workunit.client.1.smithi192.stdout:0/203: mkdir d6/df/d47 0 2024-04-23T02:13:51.060 INFO:tasks.workunit.client.1.smithi192.stdout:0/204: write d6/d44/f19 [2170456,51184] 0 2024-04-23T02:13:51.062 INFO:tasks.workunit.client.1.smithi192.stdout:2/172: rename d8/f1b to d8/f36 0 2024-04-23T02:13:51.067 INFO:tasks.workunit.client.1.smithi192.stdout:0/205: rename d6/l2c to d6/df/d35/l48 0 2024-04-23T02:13:51.068 INFO:tasks.workunit.client.1.smithi192.stdout:2/173: write d8/d26/d27/f2f [893726,84100] 0 2024-04-23T02:13:51.068 INFO:tasks.workunit.client.1.smithi192.stdout:7/192: mknod d5/d17/c37 0 2024-04-23T02:13:51.070 INFO:tasks.workunit.client.1.smithi192.stdout:5/166: getdents d2/d6 0 2024-04-23T02:13:51.071 INFO:tasks.workunit.client.1.smithi192.stdout:3/99: write d1/d3/d6/d7/f13 [239102,4200] 0 2024-04-23T02:13:51.072 INFO:tasks.workunit.client.1.smithi192.stdout:2/174: mknod d8/d26/c37 0 2024-04-23T02:13:51.072 INFO:tasks.workunit.client.1.smithi192.stdout:7/193: chown d5/d17/d2f/f22 503 1 2024-04-23T02:13:51.074 INFO:tasks.workunit.client.1.smithi192.stdout:7/194: fdatasync d5/d17/d2f/f1d 0 2024-04-23T02:13:51.076 INFO:tasks.workunit.client.1.smithi192.stdout:2/175: creat d8/d25/f38 x:0 0 0 2024-04-23T02:13:51.082 INFO:tasks.workunit.client.1.smithi192.stdout:0/206: dwrite d6/df/d13/d25/f2b [0,4194304] 0 2024-04-23T02:13:51.082 INFO:tasks.workunit.client.1.smithi192.stdout:3/100: creat d1/d3/d6/f1c x:0 0 0 2024-04-23T02:13:51.082 INFO:tasks.workunit.client.1.smithi192.stdout:7/195: chown d5/d17/d2f/l36 1 1 2024-04-23T02:13:51.082 INFO:tasks.workunit.client.1.smithi192.stdout:2/176: unlink d8/ca 0 2024-04-23T02:13:51.085 INFO:tasks.workunit.client.1.smithi192.stdout:7/196: stat d5/d17/d2f/f22 0 2024-04-23T02:13:51.086 INFO:tasks.workunit.client.1.smithi192.stdout:3/101: readlink d1/d3/d6/l12 0 2024-04-23T02:13:51.087 INFO:tasks.workunit.client.1.smithi192.stdout:0/207: mkdir d6/df/d35/d49 0 2024-04-23T02:13:51.090 INFO:tasks.workunit.client.1.smithi192.stdout:8/154: dwrite d1/f10 [0,4194304] 0 2024-04-23T02:13:51.092 INFO:tasks.workunit.client.1.smithi192.stdout:5/167: dwrite d2/db/d16/d1b/f29 [0,4194304] 0 2024-04-23T02:13:51.095 INFO:tasks.workunit.client.1.smithi192.stdout:0/208: creat d6/df/f4a x:0 0 0 2024-04-23T02:13:51.097 INFO:tasks.workunit.client.1.smithi192.stdout:7/197: symlink d5/d17/d1c/d30/l38 0 2024-04-23T02:13:51.098 INFO:tasks.workunit.client.1.smithi192.stdout:3/102: mknod d1/d3/d6/d7/c1d 0 2024-04-23T02:13:51.100 INFO:tasks.workunit.client.1.smithi192.stdout:5/168: dread d2/d6/f12 [0,4194304] 0 2024-04-23T02:13:51.101 INFO:tasks.workunit.client.1.smithi192.stdout:2/177: dwrite d8/d25/f38 [0,4194304] 0 2024-04-23T02:13:51.101 INFO:tasks.workunit.client.1.smithi192.stdout:8/155: mknod d1/c33 0 2024-04-23T02:13:51.102 INFO:tasks.workunit.client.1.smithi192.stdout:5/169: chown d2/db/d16 3764867 1 2024-04-23T02:13:51.111 INFO:tasks.workunit.client.1.smithi192.stdout:8/156: symlink d1/d12/l34 0 2024-04-23T02:13:51.111 INFO:tasks.workunit.client.1.smithi192.stdout:0/209: dread d6/df/d13/f31 [0,4194304] 0 2024-04-23T02:13:51.113 INFO:tasks.workunit.client.1.smithi192.stdout:3/103: getdents d1 0 2024-04-23T02:13:51.114 INFO:tasks.workunit.client.1.smithi192.stdout:5/170: getdents d2/d6/d2d/d30 0 2024-04-23T02:13:51.114 INFO:tasks.workunit.client.1.smithi192.stdout:0/210: read - d6/df/f4a zero size 2024-04-23T02:13:51.115 INFO:tasks.workunit.client.1.smithi192.stdout:2/178: dwrite d8/f36 [0,4194304] 0 2024-04-23T02:13:51.116 INFO:tasks.workunit.client.1.smithi192.stdout:5/171: write d2/fe [4451460,44324] 0 2024-04-23T02:13:51.118 INFO:tasks.workunit.client.1.smithi192.stdout:0/211: readlink d6/df/d35/l48 0 2024-04-23T02:13:51.118 INFO:tasks.workunit.client.1.smithi192.stdout:2/179: symlink d8/d26/d27/l39 0 2024-04-23T02:13:51.120 INFO:tasks.workunit.client.1.smithi192.stdout:5/172: chown d2/db/d16/d1b/f1f 93 1 2024-04-23T02:13:51.122 INFO:tasks.workunit.client.1.smithi192.stdout:0/212: fsync d6/f18 0 2024-04-23T02:13:51.122 INFO:tasks.workunit.client.1.smithi192.stdout:2/180: rmdir d8/d25 39 2024-04-23T02:13:51.123 INFO:tasks.workunit.client.1.smithi192.stdout:8/157: dread d1/f1d [0,4194304] 0 2024-04-23T02:13:51.125 INFO:tasks.workunit.client.1.smithi192.stdout:3/104: dwrite d1/f5 [0,4194304] 0 2024-04-23T02:13:51.126 INFO:tasks.workunit.client.1.smithi192.stdout:5/173: rmdir d2/d6/d2d 39 2024-04-23T02:13:51.126 INFO:tasks.workunit.client.1.smithi192.stdout:2/181: write d8/f18 [92025,39039] 0 2024-04-23T02:13:51.127 INFO:tasks.workunit.client.1.smithi192.stdout:6/135: sync 2024-04-23T02:13:51.127 INFO:tasks.workunit.client.0.smithi071.stdout:1/37: write f0 [1205655,77350] 0 2024-04-23T02:13:51.127 INFO:tasks.workunit.client.0.smithi071.stdout:1/38: readlink l8 0 2024-04-23T02:13:51.129 INFO:tasks.workunit.client.0.smithi071.stdout:5/80: mknod d3/da/d1a/c23 0 2024-04-23T02:13:51.130 INFO:tasks.workunit.client.0.smithi071.stdout:4/82: mkdir d2/d15/d1d/d20 0 2024-04-23T02:13:51.133 INFO:tasks.workunit.client.1.smithi192.stdout:0/213: write d6/df/d13/d25/d26/f27 [5103626,71291] 0 2024-04-23T02:13:51.133 INFO:tasks.workunit.client.1.smithi192.stdout:8/158: write d1/d9/fa [2219052,109199] 0 2024-04-23T02:13:51.135 INFO:tasks.workunit.client.0.smithi071.stdout:7/76: unlink d3/f9 0 2024-04-23T02:13:51.137 INFO:tasks.workunit.client.1.smithi192.stdout:6/136: rmdir d3/d15 39 2024-04-23T02:13:51.137 INFO:tasks.workunit.client.0.smithi071.stdout:7/77: write d3/fc [678226,54157] 0 2024-04-23T02:13:51.140 INFO:tasks.workunit.client.0.smithi071.stdout:0/53: write d0/f1 [1158852,61393] 0 2024-04-23T02:13:51.140 INFO:tasks.workunit.client.0.smithi071.stdout:4/83: symlink d2/d15/d1d/l21 0 2024-04-23T02:13:51.143 INFO:tasks.workunit.client.1.smithi192.stdout:3/105: unlink d1/d3/d6/d7/f11 0 2024-04-23T02:13:51.143 INFO:tasks.workunit.client.0.smithi071.stdout:1/39: dwrite f2 [0,4194304] 0 2024-04-23T02:13:51.143 INFO:tasks.workunit.client.0.smithi071.stdout:7/78: read d3/fc [1983849,28607] 0 2024-04-23T02:13:51.144 INFO:tasks.workunit.client.1.smithi192.stdout:2/182: dread d8/d14/f15 [0,4194304] 0 2024-04-23T02:13:51.144 INFO:tasks.workunit.client.1.smithi192.stdout:0/214: mkdir d6/d4b 0 2024-04-23T02:13:51.146 INFO:tasks.workunit.client.1.smithi192.stdout:5/174: dwrite d2/f2a [0,4194304] 0 2024-04-23T02:13:51.146 INFO:tasks.workunit.client.0.smithi071.stdout:1/40: dread - fa zero size 2024-04-23T02:13:51.149 INFO:tasks.workunit.client.1.smithi192.stdout:3/106: write d1/d3/f19 [5014454,11823] 0 2024-04-23T02:13:51.150 INFO:tasks.workunit.client.1.smithi192.stdout:8/159: dwrite d1/d12/f24 [0,4194304] 0 2024-04-23T02:13:51.150 INFO:tasks.workunit.client.0.smithi071.stdout:0/54: mkdir d0/d3/de 0 2024-04-23T02:13:51.152 INFO:tasks.workunit.client.1.smithi192.stdout:2/183: write d8/d25/f32 [2012376,120115] 0 2024-04-23T02:13:51.152 INFO:tasks.workunit.client.1.smithi192.stdout:8/160: write d1/df/f1c [988548,67217] 0 2024-04-23T02:13:51.154 INFO:tasks.workunit.client.0.smithi071.stdout:2/43: write d0/d2/f9 [1788,59264] 0 2024-04-23T02:13:51.155 INFO:tasks.workunit.client.0.smithi071.stdout:4/84: creat d2/d15/f22 x:0 0 0 2024-04-23T02:13:51.156 INFO:tasks.workunit.client.0.smithi071.stdout:9/42: dwrite d1/f10 [0,4194304] 0 2024-04-23T02:13:51.162 INFO:tasks.workunit.client.0.smithi071.stdout:2/44: dread d0/f6 [0,4194304] 0 2024-04-23T02:13:51.163 INFO:tasks.workunit.client.1.smithi192.stdout:0/215: creat d6/df/d47/f4c x:0 0 0 2024-04-23T02:13:51.163 INFO:tasks.workunit.client.1.smithi192.stdout:3/107: dread d1/d3/d6/d7/f13 [0,4194304] 0 2024-04-23T02:13:51.163 INFO:tasks.workunit.client.1.smithi192.stdout:8/161: read d1/d9/f2a [496835,128260] 0 2024-04-23T02:13:51.164 INFO:tasks.workunit.client.1.smithi192.stdout:8/162: readlink d1/df/l32 0 2024-04-23T02:13:51.164 INFO:tasks.workunit.client.1.smithi192.stdout:8/163: chown d1/d12/d23/d2c/c30 0 1 2024-04-23T02:13:51.164 INFO:tasks.workunit.client.1.smithi192.stdout:8/164: readlink d1/d12/d23/d2c/l29 0 2024-04-23T02:13:51.164 INFO:tasks.workunit.client.1.smithi192.stdout:2/184: read d8/f1d [1197096,47246] 0 2024-04-23T02:13:51.165 INFO:tasks.workunit.client.0.smithi071.stdout:7/79: dread d3/f11 [0,4194304] 0 2024-04-23T02:13:51.166 INFO:tasks.workunit.client.1.smithi192.stdout:0/216: symlink d6/d44/d45/l4d 0 2024-04-23T02:13:51.171 INFO:tasks.workunit.client.0.smithi071.stdout:2/45: creat d0/d2/fb x:0 0 0 2024-04-23T02:13:51.173 INFO:tasks.workunit.client.0.smithi071.stdout:4/85: rmdir d2/d15/d1d/d20 0 2024-04-23T02:13:51.173 INFO:tasks.workunit.client.1.smithi192.stdout:6/137: write d3/f22 [1463074,100528] 0 2024-04-23T02:13:51.175 INFO:tasks.workunit.client.0.smithi071.stdout:9/43: getdents d1 0 2024-04-23T02:13:51.175 INFO:tasks.workunit.client.0.smithi071.stdout:6/35: dwrite d6/fa [0,4194304] 0 2024-04-23T02:13:51.176 INFO:tasks.workunit.client.1.smithi192.stdout:6/138: rmdir d3 39 2024-04-23T02:13:51.177 INFO:tasks.workunit.client.0.smithi071.stdout:2/46: unlink d0/f5 0 2024-04-23T02:13:51.179 INFO:tasks.workunit.client.1.smithi192.stdout:8/165: rename d1/d12/d23/d2c/c2e to d1/d12/c35 0 2024-04-23T02:13:51.180 INFO:tasks.workunit.client.1.smithi192.stdout:3/108: dwrite d1/d3/d6/d7/f13 [0,4194304] 0 2024-04-23T02:13:51.182 INFO:tasks.workunit.client.1.smithi192.stdout:0/217: mknod d6/df/d13/d25/d26/c4e 0 2024-04-23T02:13:51.182 INFO:tasks.workunit.client.0.smithi071.stdout:9/44: creat d1/f12 x:0 0 0 2024-04-23T02:13:51.183 INFO:tasks.workunit.client.0.smithi071.stdout:5/81: dwrite d3/da/fd [0,4194304] 0 2024-04-23T02:13:51.184 INFO:tasks.workunit.client.0.smithi071.stdout:8/71: dread f8 [0,4194304] 0 2024-04-23T02:13:51.185 INFO:tasks.workunit.client.1.smithi192.stdout:6/139: truncate d3/d11/f13 74298 0 2024-04-23T02:13:51.187 INFO:tasks.workunit.client.1.smithi192.stdout:5/175: dwrite d2/f2c [0,4194304] 0 2024-04-23T02:13:51.188 INFO:tasks.workunit.client.1.smithi192.stdout:0/218: write d6/f3b [4907550,102484] 0 2024-04-23T02:13:51.189 INFO:tasks.workunit.client.0.smithi071.stdout:4/86: rename d2/d7 to d2/d15/d1d/d23 0 2024-04-23T02:13:51.189 INFO:tasks.workunit.client.1.smithi192.stdout:2/185: dwrite d8/d25/f32 [0,4194304] 0 2024-04-23T02:13:51.196 INFO:tasks.workunit.client.1.smithi192.stdout:5/176: symlink d2/db/d2b/l37 0 2024-04-23T02:13:51.196 INFO:tasks.workunit.client.0.smithi071.stdout:0/55: write d0/f5 [3580526,22061] 0 2024-04-23T02:13:51.197 INFO:tasks.workunit.client.1.smithi192.stdout:8/166: mknod d1/d12/c36 0 2024-04-23T02:13:51.198 INFO:tasks.workunit.client.0.smithi071.stdout:5/82: unlink d3/l7 0 2024-04-23T02:13:51.198 INFO:tasks.workunit.client.0.smithi071.stdout:6/36: dread d6/fa [0,4194304] 0 2024-04-23T02:13:51.199 INFO:tasks.workunit.client.0.smithi071.stdout:1/41: dwrite f7 [0,4194304] 0 2024-04-23T02:13:51.199 INFO:tasks.workunit.client.1.smithi192.stdout:0/219: truncate d6/df/f23 97051 0 2024-04-23T02:13:51.199 INFO:tasks.workunit.client.1.smithi192.stdout:6/140: truncate f1 670687 0 2024-04-23T02:13:51.203 INFO:tasks.workunit.client.0.smithi071.stdout:7/80: write d3/f12 [3959635,124026] 0 2024-04-23T02:13:51.203 INFO:tasks.workunit.client.1.smithi192.stdout:2/186: creat d8/d25/f3a x:0 0 0 2024-04-23T02:13:51.205 INFO:tasks.workunit.client.1.smithi192.stdout:3/109: dwrite d1/d3/d6/f1c [0,4194304] 0 2024-04-23T02:13:51.205 INFO:tasks.workunit.client.1.smithi192.stdout:6/141: chown d3/d15/c27 515240418 1 2024-04-23T02:13:51.207 INFO:tasks.workunit.client.0.smithi071.stdout:1/42: mkdir db 0 2024-04-23T02:13:51.207 INFO:tasks.workunit.client.0.smithi071.stdout:9/45: dwrite d1/f12 [0,4194304] 0 2024-04-23T02:13:51.207 INFO:tasks.workunit.client.1.smithi192.stdout:8/167: creat d1/d12/d23/f37 x:0 0 0 2024-04-23T02:13:51.209 INFO:tasks.workunit.client.1.smithi192.stdout:8/168: write d1/d9/f22 [227113,4251] 0 2024-04-23T02:13:51.210 INFO:tasks.workunit.client.1.smithi192.stdout:0/220: write d6/df/f32 [3495936,102276] 0 2024-04-23T02:13:51.210 INFO:tasks.workunit.client.0.smithi071.stdout:8/72: link c7 dc/d10/c11 0 2024-04-23T02:13:51.211 INFO:tasks.workunit.client.1.smithi192.stdout:3/110: mknod d1/d3/d17/c1e 0 2024-04-23T02:13:51.212 INFO:tasks.workunit.client.0.smithi071.stdout:1/43: truncate f0 1961303 0 2024-04-23T02:13:51.214 INFO:tasks.workunit.client.0.smithi071.stdout:6/37: dread d6/fa [0,4194304] 0 2024-04-23T02:13:51.214 INFO:tasks.workunit.client.0.smithi071.stdout:4/87: truncate d2/d15/d1d/d23/d10/f13 640704 0 2024-04-23T02:13:51.215 INFO:tasks.workunit.client.1.smithi192.stdout:0/221: read - d6/df/d13/d25/f3d zero size 2024-04-23T02:13:51.215 INFO:tasks.workunit.client.0.smithi071.stdout:0/56: creat d0/d3/de/ff x:0 0 0 2024-04-23T02:13:51.217 INFO:tasks.workunit.client.1.smithi192.stdout:5/177: creat d2/d6/d2d/d30/f38 x:0 0 0 2024-04-23T02:13:51.218 INFO:tasks.workunit.client.0.smithi071.stdout:7/81: rename d3/lb to d3/l1b 0 2024-04-23T02:13:51.219 INFO:tasks.workunit.client.1.smithi192.stdout:6/142: dread d3/f22 [0,4194304] 0 2024-04-23T02:13:51.220 INFO:tasks.workunit.client.0.smithi071.stdout:0/57: chown d0/f5 67 1 2024-04-23T02:13:51.221 INFO:tasks.workunit.client.0.smithi071.stdout:9/46: dread d1/fc [0,4194304] 0 2024-04-23T02:13:51.223 INFO:tasks.workunit.client.1.smithi192.stdout:8/169: link d1/d9/f11 d1/d12/d23/d2c/f38 0 2024-04-23T02:13:51.223 INFO:tasks.workunit.client.1.smithi192.stdout:5/178: mknod d2/d6/d2d/c39 0 2024-04-23T02:13:51.224 INFO:tasks.workunit.client.1.smithi192.stdout:0/222: read d6/d44/f46 [8896,70534] 0 2024-04-23T02:13:51.224 INFO:tasks.workunit.client.1.smithi192.stdout:8/170: fdatasync d1/d9/f11 0 2024-04-23T02:13:51.224 INFO:tasks.workunit.client.1.smithi192.stdout:5/179: write d2/d6/d2d/d30/f38 [318020,78526] 0 2024-04-23T02:13:51.225 INFO:tasks.workunit.client.0.smithi071.stdout:7/82: mkdir d3/d17/d1c 0 2024-04-23T02:13:51.226 INFO:tasks.workunit.client.0.smithi071.stdout:8/73: dwrite dc/ff [0,4194304] 0 2024-04-23T02:13:51.228 INFO:tasks.workunit.client.1.smithi192.stdout:5/180: creat d2/db/d2b/f3a x:0 0 0 2024-04-23T02:13:51.229 INFO:tasks.workunit.client.1.smithi192.stdout:0/223: creat d6/df/d13/d25/d3a/f4f x:0 0 0 2024-04-23T02:13:51.230 INFO:tasks.workunit.client.0.smithi071.stdout:4/88: dread d2/f4 [0,4194304] 0 2024-04-23T02:13:51.232 INFO:tasks.workunit.client.1.smithi192.stdout:0/224: creat d6/df/f50 x:0 0 0 2024-04-23T02:13:51.233 INFO:tasks.workunit.client.1.smithi192.stdout:8/171: dwrite d1/d12/f1e [4194304,4194304] 0 2024-04-23T02:13:51.234 INFO:tasks.workunit.client.1.smithi192.stdout:5/181: creat d2/d6/f3b x:0 0 0 2024-04-23T02:13:51.234 INFO:tasks.workunit.client.0.smithi071.stdout:0/58: dwrite d0/f2 [0,4194304] 0 2024-04-23T02:13:51.236 INFO:tasks.workunit.client.0.smithi071.stdout:0/59: truncate d0/f2 4868608 0 2024-04-23T02:13:51.236 INFO:tasks.workunit.client.1.smithi192.stdout:8/172: write d1/f3 [4085231,89289] 0 2024-04-23T02:13:51.236 INFO:tasks.workunit.client.1.smithi192.stdout:5/182: symlink d2/d32/l3c 0 2024-04-23T02:13:51.237 INFO:tasks.workunit.client.0.smithi071.stdout:0/60: stat d0/d3/de/ff 0 2024-04-23T02:13:51.237 INFO:tasks.workunit.client.0.smithi071.stdout:0/61: chown d0/d3/de 452 1 2024-04-23T02:13:51.240 INFO:tasks.workunit.client.1.smithi192.stdout:5/183: getdents d2/db/d16 0 2024-04-23T02:13:51.242 INFO:tasks.workunit.client.1.smithi192.stdout:5/184: mkdir d2/db/d16/d1b/d3d 0 2024-04-23T02:13:51.245 INFO:tasks.workunit.client.1.smithi192.stdout:0/225: dread f5 [0,4194304] 0 2024-04-23T02:13:51.249 INFO:tasks.workunit.client.1.smithi192.stdout:5/185: dread d2/fe [0,4194304] 0 2024-04-23T02:13:51.250 INFO:tasks.workunit.client.0.smithi071.stdout:0/62: dread d0/f1 [0,4194304] 0 2024-04-23T02:13:51.251 INFO:tasks.workunit.client.1.smithi192.stdout:0/226: dread - d6/df/d13/d25/d3a/f4f zero size 2024-04-23T02:13:51.253 INFO:tasks.workunit.client.1.smithi192.stdout:3/111: fdatasync d1/d3/d6/f1c 0 2024-04-23T02:13:51.256 INFO:tasks.workunit.client.1.smithi192.stdout:0/227: creat d6/df/d13/f51 x:0 0 0 2024-04-23T02:13:51.263 INFO:tasks.workunit.client.1.smithi192.stdout:2/187: truncate d8/d25/f38 2282362 0 2024-04-23T02:13:51.263 INFO:tasks.workunit.client.1.smithi192.stdout:2/188: stat f4 0 2024-04-23T02:13:51.267 INFO:tasks.workunit.client.1.smithi192.stdout:0/228: getdents d6/df/d13/d25/d3a 0 2024-04-23T02:13:51.270 INFO:tasks.workunit.client.1.smithi192.stdout:0/229: creat d6/d4b/f52 x:0 0 0 2024-04-23T02:13:51.271 INFO:tasks.workunit.client.1.smithi192.stdout:8/173: dread d1/d12/f1e [0,4194304] 0 2024-04-23T02:13:51.272 INFO:tasks.workunit.client.1.smithi192.stdout:0/230: truncate d6/df/f23 505950 0 2024-04-23T02:13:51.273 INFO:tasks.workunit.client.1.smithi192.stdout:3/112: truncate d1/d3/f19 1430203 0 2024-04-23T02:13:51.276 INFO:tasks.workunit.client.1.smithi192.stdout:2/189: rmdir d8/d25 39 2024-04-23T02:13:51.278 INFO:tasks.workunit.client.0.smithi071.stdout:9/47: creat d1/f13 x:0 0 0 2024-04-23T02:13:51.279 INFO:tasks.workunit.client.0.smithi071.stdout:7/83: creat d3/f1d x:0 0 0 2024-04-23T02:13:51.281 INFO:tasks.workunit.client.1.smithi192.stdout:8/174: creat d1/d12/f39 x:0 0 0 2024-04-23T02:13:51.282 INFO:tasks.workunit.client.1.smithi192.stdout:5/186: dwrite d2/d6/f12 [0,4194304] 0 2024-04-23T02:13:51.285 INFO:tasks.workunit.client.0.smithi071.stdout:2/47: truncate d0/f6 532696 0 2024-04-23T02:13:51.286 INFO:tasks.workunit.client.0.smithi071.stdout:2/48: chown d0/d2/fb 113925 1 2024-04-23T02:13:51.286 INFO:tasks.workunit.client.0.smithi071.stdout:7/84: rename d3/c13 to d3/d17/d1c/c1e 0 2024-04-23T02:13:51.287 INFO:tasks.workunit.client.1.smithi192.stdout:3/113: rename d1/d3/dd to d1/d3/d17/d1f 0 2024-04-23T02:13:51.288 INFO:tasks.workunit.client.1.smithi192.stdout:8/175: write d1/d12/d23/f37 [13538,78397] 0 2024-04-23T02:13:51.290 INFO:tasks.workunit.client.0.smithi071.stdout:5/83: write d3/f12 [1374060,46265] 0 2024-04-23T02:13:51.292 INFO:tasks.workunit.client.1.smithi192.stdout:6/143: dwrite f1 [0,4194304] 0 2024-04-23T02:13:51.293 INFO:tasks.workunit.client.0.smithi071.stdout:7/85: fdatasync d3/ff 0 2024-04-23T02:13:51.295 INFO:tasks.workunit.client.1.smithi192.stdout:8/176: unlink d1/df/l32 0 2024-04-23T02:13:51.300 INFO:tasks.workunit.client.0.smithi071.stdout:0/63: getdents d0/d3/de 0 2024-04-23T02:13:51.300 INFO:tasks.workunit.client.0.smithi071.stdout:8/74: write f9 [2779395,21095] 0 2024-04-23T02:13:51.300 INFO:tasks.workunit.client.0.smithi071.stdout:7/86: truncate d3/f11 161089 0 2024-04-23T02:13:51.301 INFO:tasks.workunit.client.1.smithi192.stdout:0/231: dwrite d6/d44/f46 [0,4194304] 0 2024-04-23T02:13:51.302 INFO:tasks.workunit.client.1.smithi192.stdout:6/144: rmdir d3/d18 39 2024-04-23T02:13:51.302 INFO:tasks.workunit.client.0.smithi071.stdout:8/75: chown f9 0 1 2024-04-23T02:13:51.304 INFO:tasks.workunit.client.0.smithi071.stdout:6/38: dwrite d6/fb [0,4194304] 0 2024-04-23T02:13:51.310 INFO:tasks.workunit.client.0.smithi071.stdout:5/84: rename d3/da/fb to d3/df/f24 0 2024-04-23T02:13:51.310 INFO:tasks.workunit.client.1.smithi192.stdout:0/232: chown d6/d4b/f52 45641417 1 2024-04-23T02:13:51.311 INFO:tasks.workunit.client.1.smithi192.stdout:5/187: dread d2/d32/f33 [0,4194304] 0 2024-04-23T02:13:51.311 INFO:tasks.workunit.client.1.smithi192.stdout:2/190: dwrite d8/d25/f3a [0,4194304] 0 2024-04-23T02:13:51.311 INFO:tasks.workunit.client.0.smithi071.stdout:8/76: symlink dc/l12 0 2024-04-23T02:13:51.312 INFO:tasks.workunit.client.1.smithi192.stdout:6/145: chown d3/d11/f1d 726767467 1 2024-04-23T02:13:51.312 INFO:tasks.workunit.client.1.smithi192.stdout:0/233: write d6/df/f32 [1507529,117610] 0 2024-04-23T02:13:51.312 INFO:tasks.workunit.client.0.smithi071.stdout:2/49: truncate d0/d2/f9 23204 0 2024-04-23T02:13:51.313 INFO:tasks.workunit.client.1.smithi192.stdout:5/188: unlink d2/db/f20 0 2024-04-23T02:13:51.314 INFO:tasks.workunit.client.0.smithi071.stdout:5/85: mknod d3/df/d10/c25 0 2024-04-23T02:13:51.314 INFO:tasks.workunit.client.0.smithi071.stdout:0/64: dread d0/f5 [0,4194304] 0 2024-04-23T02:13:51.314 INFO:tasks.workunit.client.0.smithi071.stdout:9/48: dwrite d1/f4 [0,4194304] 0 2024-04-23T02:13:51.319 INFO:tasks.workunit.client.0.smithi071.stdout:2/50: unlink d0/c4 0 2024-04-23T02:13:51.320 INFO:tasks.workunit.client.1.smithi192.stdout:5/189: write d2/db/f18 [1752392,118701] 0 2024-04-23T02:13:51.321 INFO:tasks.workunit.client.0.smithi071.stdout:9/49: mknod d1/c14 0 2024-04-23T02:13:51.321 INFO:tasks.workunit.client.1.smithi192.stdout:2/191: unlink f3 0 2024-04-23T02:13:51.321 INFO:tasks.workunit.client.1.smithi192.stdout:3/114: dwrite d1/d3/d6/d7/f13 [0,4194304] 0 2024-04-23T02:13:51.323 INFO:tasks.workunit.client.1.smithi192.stdout:6/146: creat d3/d11/f29 x:0 0 0 2024-04-23T02:13:51.323 INFO:tasks.workunit.client.0.smithi071.stdout:5/86: creat d3/da/d1a/f26 x:0 0 0 2024-04-23T02:13:51.323 INFO:tasks.workunit.client.0.smithi071.stdout:6/39: link f1 d6/fd 0 2024-04-23T02:13:51.324 INFO:tasks.workunit.client.0.smithi071.stdout:6/40: chown f3 11 1 2024-04-23T02:13:51.324 INFO:tasks.workunit.client.0.smithi071.stdout:0/65: mkdir d0/d3/de/d10 0 2024-04-23T02:13:51.326 INFO:tasks.workunit.client.0.smithi071.stdout:0/66: read d0/f5 [2290718,130763] 0 2024-04-23T02:13:51.326 INFO:tasks.workunit.client.1.smithi192.stdout:8/177: dread d1/d12/f24 [0,4194304] 0 2024-04-23T02:13:51.327 INFO:tasks.workunit.client.1.smithi192.stdout:6/147: chown d3/l1c 214051520 1 2024-04-23T02:13:51.328 INFO:tasks.workunit.client.0.smithi071.stdout:0/67: write d0/f2 [1366750,254] 0 2024-04-23T02:13:51.328 INFO:tasks.workunit.client.1.smithi192.stdout:0/234: symlink d6/df/d13/d25/d26/l53 0 2024-04-23T02:13:51.330 INFO:tasks.workunit.client.1.smithi192.stdout:6/148: creat d3/d15/d1b/f2a x:0 0 0 2024-04-23T02:13:51.331 INFO:tasks.workunit.client.1.smithi192.stdout:8/178: write d1/d9/f11 [11783149,15793] 0 2024-04-23T02:13:51.331 INFO:tasks.workunit.client.0.smithi071.stdout:5/87: rmdir d3/da/d1a/d1c 39 2024-04-23T02:13:51.332 INFO:tasks.workunit.client.1.smithi192.stdout:2/192: dread d8/f18 [0,4194304] 0 2024-04-23T02:13:51.332 INFO:tasks.workunit.client.1.smithi192.stdout:5/190: mkdir d2/d6/d14/d3e 0 2024-04-23T02:13:51.332 INFO:tasks.workunit.client.0.smithi071.stdout:5/88: write d3/f12 [2418171,107733] 0 2024-04-23T02:13:51.332 INFO:tasks.workunit.client.0.smithi071.stdout:2/51: creat d0/d2/d7/fc x:0 0 0 2024-04-23T02:13:51.335 INFO:tasks.workunit.client.0.smithi071.stdout:8/77: link c7 dc/c13 0 2024-04-23T02:13:51.340 INFO:tasks.workunit.client.0.smithi071.stdout:8/78: dread f6 [4194304,4194304] 0 2024-04-23T02:13:51.341 INFO:tasks.workunit.client.1.smithi192.stdout:2/193: chown d8/f1d 67 1 2024-04-23T02:13:51.341 INFO:tasks.workunit.client.1.smithi192.stdout:3/115: dread d1/d3/d6/d7/f13 [0,4194304] 0 2024-04-23T02:13:51.342 INFO:tasks.workunit.client.1.smithi192.stdout:8/179: symlink d1/d9/l3a 0 2024-04-23T02:13:51.342 INFO:tasks.workunit.client.0.smithi071.stdout:0/68: rename d0/d3/l9 to d0/d3/dc/l11 0 2024-04-23T02:13:51.343 INFO:tasks.workunit.client.1.smithi192.stdout:5/191: unlink d2/d2e/c34 0 2024-04-23T02:13:51.344 INFO:tasks.workunit.client.0.smithi071.stdout:5/89: truncate f2 4879944 0 2024-04-23T02:13:51.344 INFO:tasks.workunit.client.1.smithi192.stdout:6/149: creat d3/f2b x:0 0 0 2024-04-23T02:13:51.345 INFO:tasks.workunit.client.1.smithi192.stdout:3/116: unlink d1/d3/d6/d7/f13 0 2024-04-23T02:13:51.348 INFO:tasks.workunit.client.0.smithi071.stdout:0/69: symlink d0/d3/dc/l12 0 2024-04-23T02:13:51.349 INFO:tasks.workunit.client.1.smithi192.stdout:2/194: getdents d8 0 2024-04-23T02:13:51.349 INFO:tasks.workunit.client.1.smithi192.stdout:3/117: chown d1/d3/d17/c1e 0 1 2024-04-23T02:13:51.351 INFO:tasks.workunit.client.0.smithi071.stdout:5/90: truncate d3/da/d1a/f1b 504332 0 2024-04-23T02:13:51.351 INFO:tasks.workunit.client.1.smithi192.stdout:3/118: stat d1/l16 0 2024-04-23T02:13:51.352 INFO:tasks.workunit.client.1.smithi192.stdout:9/106: sync 2024-04-23T02:13:51.353 INFO:tasks.workunit.client.0.smithi071.stdout:6/41: truncate f1 4119106 0 2024-04-23T02:13:51.354 INFO:tasks.workunit.client.1.smithi192.stdout:8/180: dwrite d1/d12/d23/d2c/f38 [0,4194304] 0 2024-04-23T02:13:51.354 INFO:tasks.workunit.client.1.smithi192.stdout:2/195: unlink d8/d14/l1c 0 2024-04-23T02:13:51.354 INFO:tasks.workunit.client.1.smithi192.stdout:3/119: stat d1/d3/d6/l12 0 2024-04-23T02:13:51.356 INFO:tasks.workunit.client.0.smithi071.stdout:8/79: creat dc/d10/f14 x:0 0 0 2024-04-23T02:13:51.357 INFO:tasks.workunit.client.0.smithi071.stdout:0/70: creat d0/d3/de/f13 x:0 0 0 2024-04-23T02:13:51.357 INFO:tasks.workunit.client.1.smithi192.stdout:0/235: truncate f5 1698714 0 2024-04-23T02:13:51.360 INFO:tasks.workunit.client.1.smithi192.stdout:3/120: rename d1/d3/d6/d7 to d1/d3/d6/d7/d20 22 2024-04-23T02:13:51.361 INFO:tasks.workunit.client.1.smithi192.stdout:9/107: dread d0/d9/f14 [0,4194304] 0 2024-04-23T02:13:51.362 INFO:tasks.workunit.client.1.smithi192.stdout:9/108: stat d0/d9/fb 0 2024-04-23T02:13:51.363 INFO:tasks.workunit.client.1.smithi192.stdout:6/150: dwrite d3/d15/d1b/f2a [0,4194304] 0 2024-04-23T02:13:51.363 INFO:tasks.workunit.client.0.smithi071.stdout:8/80: mkdir dc/d15 0 2024-04-23T02:13:51.363 INFO:tasks.workunit.client.0.smithi071.stdout:7/87: link d3/f11 d3/d17/f1f 0 2024-04-23T02:13:51.364 INFO:tasks.workunit.client.0.smithi071.stdout:5/91: dwrite d3/da/fd [0,4194304] 0 2024-04-23T02:13:51.365 INFO:tasks.workunit.client.1.smithi192.stdout:3/121: mknod d1/d3/d17/c21 0 2024-04-23T02:13:51.366 INFO:tasks.workunit.client.1.smithi192.stdout:2/196: dread d8/d26/d27/f2f [0,4194304] 0 2024-04-23T02:13:51.366 INFO:tasks.workunit.client.1.smithi192.stdout:6/151: write d3/d11/f29 [1009375,9170] 0 2024-04-23T02:13:51.366 INFO:tasks.workunit.client.1.smithi192.stdout:3/122: fdatasync d1/d3/d6/f1c 0 2024-04-23T02:13:51.370 INFO:tasks.workunit.client.0.smithi071.stdout:7/88: read d3/f12 [3507166,123755] 0 2024-04-23T02:13:51.370 INFO:tasks.workunit.client.0.smithi071.stdout:8/81: mknod dc/d10/c16 0 2024-04-23T02:13:51.373 INFO:tasks.workunit.client.1.smithi192.stdout:8/181: mknod d1/d12/d23/d2c/c3b 0 2024-04-23T02:13:51.374 INFO:tasks.workunit.client.0.smithi071.stdout:6/42: dwrite d6/fb [0,4194304] 0 2024-04-23T02:13:51.374 INFO:tasks.workunit.client.0.smithi071.stdout:5/92: fsync d3/df/f17 0 2024-04-23T02:13:51.376 INFO:tasks.workunit.client.1.smithi192.stdout:9/109: mknod d0/d1e/c2b 0 2024-04-23T02:13:51.379 INFO:tasks.workunit.client.1.smithi192.stdout:0/236: creat d6/df/d35/d49/f54 x:0 0 0 2024-04-23T02:13:51.380 INFO:tasks.workunit.client.1.smithi192.stdout:8/182: write d1/f15 [1784587,70007] 0 2024-04-23T02:13:51.381 INFO:tasks.workunit.client.0.smithi071.stdout:7/89: creat d3/d17/d1c/f20 x:0 0 0 2024-04-23T02:13:51.381 INFO:tasks.workunit.client.0.smithi071.stdout:6/43: mkdir d6/de 0 2024-04-23T02:13:51.383 INFO:tasks.workunit.client.0.smithi071.stdout:9/50: dwrite d1/fb [0,4194304] 0 2024-04-23T02:13:51.385 INFO:tasks.workunit.client.1.smithi192.stdout:6/152: link d3/d15/c23 d3/d18/c2c 0 2024-04-23T02:13:51.385 INFO:tasks.workunit.client.1.smithi192.stdout:8/183: creat d1/d12/d23/f3c x:0 0 0 2024-04-23T02:13:51.386 INFO:tasks.workunit.client.0.smithi071.stdout:5/93: read - d3/da/d1a/d1c/f1e zero size 2024-04-23T02:13:51.386 INFO:tasks.workunit.client.0.smithi071.stdout:0/71: dwrite d0/d3/de/ff [0,4194304] 0 2024-04-23T02:13:51.386 INFO:tasks.workunit.client.0.smithi071.stdout:7/90: symlink d3/d17/l21 0 2024-04-23T02:13:51.387 INFO:tasks.workunit.client.0.smithi071.stdout:9/51: symlink d1/l15 0 2024-04-23T02:13:51.388 INFO:tasks.workunit.client.0.smithi071.stdout:5/94: mknod d3/da/c27 0 2024-04-23T02:13:51.388 INFO:tasks.workunit.client.0.smithi071.stdout:0/72: truncate d0/d3/de/f13 561723 0 2024-04-23T02:13:51.389 INFO:tasks.workunit.client.1.smithi192.stdout:2/197: dwrite d8/f36 [4194304,4194304] 0 2024-04-23T02:13:51.391 INFO:tasks.workunit.client.1.smithi192.stdout:8/184: dread - d1/d12/f39 zero size 2024-04-23T02:13:51.395 INFO:tasks.workunit.client.1.smithi192.stdout:9/110: link d0/d9/f26 d0/f2c 0 2024-04-23T02:13:51.396 INFO:tasks.workunit.client.1.smithi192.stdout:3/123: dwrite d1/d3/d6/f10 [0,4194304] 0 2024-04-23T02:13:51.399 INFO:tasks.workunit.client.1.smithi192.stdout:8/185: fdatasync d1/d9/fa 0 2024-04-23T02:13:51.400 INFO:tasks.workunit.client.0.smithi071.stdout:9/52: write d1/f4 [1150527,29952] 0 2024-04-23T02:13:51.401 INFO:tasks.workunit.client.0.smithi071.stdout:6/44: rmdir d6/de 0 2024-04-23T02:13:51.401 INFO:tasks.workunit.client.1.smithi192.stdout:2/198: symlink d8/d14/l3b 0 2024-04-23T02:13:51.403 INFO:tasks.workunit.client.0.smithi071.stdout:7/91: creat d3/d17/f22 x:0 0 0 2024-04-23T02:13:51.404 INFO:tasks.workunit.client.1.smithi192.stdout:6/153: dwrite d3/f2b [0,4194304] 0 2024-04-23T02:13:51.405 INFO:tasks.workunit.client.0.smithi071.stdout:5/95: rmdir d3/df 39 2024-04-23T02:13:51.406 INFO:tasks.workunit.client.0.smithi071.stdout:6/45: creat d6/ff x:0 0 0 2024-04-23T02:13:51.407 INFO:tasks.workunit.client.0.smithi071.stdout:5/96: write d3/f12 [510396,38520] 0 2024-04-23T02:13:51.408 INFO:tasks.workunit.client.1.smithi192.stdout:9/111: mknod d0/d1e/c2d 0 2024-04-23T02:13:51.408 INFO:tasks.workunit.client.1.smithi192.stdout:8/186: symlink d1/d9/l3d 0 2024-04-23T02:13:51.408 INFO:tasks.workunit.client.0.smithi071.stdout:7/92: write d3/fc [804399,18659] 0 2024-04-23T02:13:51.408 INFO:tasks.workunit.client.0.smithi071.stdout:0/73: mkdir d0/d3/de/d10/d14 0 2024-04-23T02:13:51.409 INFO:tasks.workunit.client.1.smithi192.stdout:6/154: readlink l2 0 2024-04-23T02:13:51.409 INFO:tasks.workunit.client.1.smithi192.stdout:2/199: rename d8/f2c to d8/d26/f3c 0 2024-04-23T02:13:51.409 INFO:tasks.workunit.client.0.smithi071.stdout:6/46: mknod d6/c10 0 2024-04-23T02:13:51.411 INFO:tasks.workunit.client.0.smithi071.stdout:6/47: write d6/ff [1002077,14075] 0 2024-04-23T02:13:51.412 INFO:tasks.workunit.client.0.smithi071.stdout:0/74: unlink d0/c6 0 2024-04-23T02:13:51.412 INFO:tasks.workunit.client.0.smithi071.stdout:7/93: mknod d3/d17/c23 0 2024-04-23T02:13:51.414 INFO:tasks.workunit.client.1.smithi192.stdout:0/237: dwrite d6/df/f30 [0,4194304] 0 2024-04-23T02:13:51.415 INFO:tasks.workunit.client.0.smithi071.stdout:5/97: symlink d3/df/d10/l28 0 2024-04-23T02:13:51.415 INFO:tasks.workunit.client.0.smithi071.stdout:0/75: symlink d0/d3/l15 0 2024-04-23T02:13:51.416 INFO:tasks.workunit.client.1.smithi192.stdout:0/238: chown d6/df/f14 0 1 2024-04-23T02:13:51.417 INFO:tasks.workunit.client.1.smithi192.stdout:8/187: mkdir d1/d9/d3e 0 2024-04-23T02:13:51.418 INFO:tasks.workunit.client.1.smithi192.stdout:9/112: readlink d0/l12 0 2024-04-23T02:13:51.418 INFO:tasks.workunit.client.0.smithi071.stdout:2/52: write d0/f6 [870105,2586] 0 2024-04-23T02:13:51.419 INFO:tasks.workunit.client.1.smithi192.stdout:6/155: creat d3/d18/f2d x:0 0 0 2024-04-23T02:13:51.420 INFO:tasks.workunit.client.0.smithi071.stdout:5/98: creat d3/f29 x:0 0 0 2024-04-23T02:13:51.420 INFO:tasks.workunit.client.1.smithi192.stdout:8/188: creat d1/d9/f3f x:0 0 0 2024-04-23T02:13:51.421 INFO:tasks.workunit.client.1.smithi192.stdout:0/239: mkdir d6/df/d47/d55 0 2024-04-23T02:13:51.422 INFO:tasks.workunit.client.0.smithi071.stdout:5/99: write d3/f12 [3447236,129150] 0 2024-04-23T02:13:51.422 INFO:tasks.workunit.client.1.smithi192.stdout:9/113: creat d0/d3/f2e x:0 0 0 2024-04-23T02:13:51.424 INFO:tasks.workunit.client.0.smithi071.stdout:2/53: rename d0/d2/d7 to d0/d2/dd 0 2024-04-23T02:13:51.425 INFO:tasks.workunit.client.1.smithi192.stdout:0/240: chown d6/d44/f38 362690 1 2024-04-23T02:13:51.425 INFO:tasks.workunit.client.1.smithi192.stdout:9/114: fsync d0/d9/ff 0 2024-04-23T02:13:51.428 INFO:tasks.workunit.client.1.smithi192.stdout:6/156: rename d3/f9 to d3/d28/f2e 0 2024-04-23T02:13:51.437 INFO:tasks.workunit.client.1.smithi192.stdout:6/157: dread d3/d15/d1b/f2a [0,4194304] 0 2024-04-23T02:13:51.441 INFO:tasks.workunit.client.1.smithi192.stdout:9/115: dwrite d0/d3/f28 [0,4194304] 0 2024-04-23T02:13:51.445 INFO:tasks.workunit.client.1.smithi192.stdout:9/116: write d0/d9/f14 [1177628,108306] 0 2024-04-23T02:13:51.448 INFO:tasks.workunit.client.1.smithi192.stdout:9/117: symlink d0/l2f 0 2024-04-23T02:13:51.449 INFO:tasks.workunit.client.1.smithi192.stdout:9/118: dread - d0/d9/f25 zero size 2024-04-23T02:13:51.455 INFO:tasks.workunit.client.0.smithi071.stdout:8/82: write f8 [1283230,128188] 0 2024-04-23T02:13:51.455 INFO:tasks.workunit.client.1.smithi192.stdout:9/119: link d0/d3/l1d d0/d3/l30 0 2024-04-23T02:13:51.456 INFO:tasks.workunit.client.1.smithi192.stdout:9/120: chown d0/d1e/c2d 221 1 2024-04-23T02:13:51.458 INFO:tasks.workunit.client.0.smithi071.stdout:9/53: rmdir d1 39 2024-04-23T02:13:51.459 INFO:tasks.workunit.client.1.smithi192.stdout:8/189: getdents d1/d12/d23 0 2024-04-23T02:13:51.460 INFO:tasks.workunit.client.1.smithi192.stdout:8/190: readlink d1/d12/d23/d2c/l27 0 2024-04-23T02:13:51.465 INFO:tasks.workunit.client.1.smithi192.stdout:9/121: link d0/d3/f28 d0/f31 0 2024-04-23T02:13:51.466 INFO:tasks.workunit.client.0.smithi071.stdout:9/54: dwrite d1/f10 [0,4194304] 0 2024-04-23T02:13:51.466 INFO:tasks.workunit.client.0.smithi071.stdout:0/76: truncate d0/d3/de/f13 446960 0 2024-04-23T02:13:51.467 INFO:tasks.workunit.client.1.smithi192.stdout:8/191: creat d1/d12/d23/f40 x:0 0 0 2024-04-23T02:13:51.467 INFO:tasks.workunit.client.0.smithi071.stdout:2/54: write d0/d2/f9 [12237,85094] 0 2024-04-23T02:13:51.467 INFO:tasks.workunit.client.0.smithi071.stdout:0/77: chown d0/d3/dc/l12 18 1 2024-04-23T02:13:51.468 INFO:tasks.workunit.client.1.smithi192.stdout:3/124: write d1/d3/f19 [1523623,7095] 0 2024-04-23T02:13:51.469 INFO:tasks.workunit.client.1.smithi192.stdout:2/200: dread d8/f18 [0,4194304] 0 2024-04-23T02:13:51.470 INFO:tasks.workunit.client.0.smithi071.stdout:8/83: write f6 [1726454,990] 0 2024-04-23T02:13:51.472 INFO:tasks.workunit.client.0.smithi071.stdout:8/84: rename dc/d10 to dc/d10/d17 22 2024-04-23T02:13:51.472 INFO:tasks.workunit.client.0.smithi071.stdout:0/78: rmdir d0/d3/de/d10 39 2024-04-23T02:13:51.472 INFO:tasks.workunit.client.1.smithi192.stdout:9/122: truncate d0/d3/f8 294914 0 2024-04-23T02:13:51.474 INFO:tasks.workunit.client.0.smithi071.stdout:9/55: link d1/f9 d1/f16 0 2024-04-23T02:13:51.475 INFO:tasks.workunit.client.0.smithi071.stdout:8/85: write dc/d10/f14 [633623,76234] 0 2024-04-23T02:13:51.475 INFO:tasks.workunit.client.1.smithi192.stdout:9/123: write d0/d1e/f23 [683626,118435] 0 2024-04-23T02:13:51.478 INFO:tasks.workunit.client.0.smithi071.stdout:9/56: write d1/f4 [261170,35332] 0 2024-04-23T02:13:51.480 INFO:tasks.workunit.client.0.smithi071.stdout:8/86: creat dc/f18 x:0 0 0 2024-04-23T02:13:51.480 INFO:tasks.workunit.client.0.smithi071.stdout:9/57: readlink d1/l11 0 2024-04-23T02:13:51.482 INFO:tasks.workunit.client.0.smithi071.stdout:8/87: dread - dc/f18 zero size 2024-04-23T02:13:51.482 INFO:tasks.workunit.client.1.smithi192.stdout:6/158: write d3/f12 [85941,64430] 0 2024-04-23T02:13:51.482 INFO:tasks.workunit.client.0.smithi071.stdout:2/55: dwrite d0/d2/dd/fc [0,4194304] 0 2024-04-23T02:13:51.483 INFO:tasks.workunit.client.0.smithi071.stdout:9/58: chown d1/l15 0 1 2024-04-23T02:13:51.483 INFO:tasks.workunit.client.1.smithi192.stdout:0/241: write d6/d44/f46 [1892625,66097] 0 2024-04-23T02:13:51.484 INFO:tasks.workunit.client.0.smithi071.stdout:8/88: truncate dc/f18 200177 0 2024-04-23T02:13:51.487 INFO:tasks.workunit.client.0.smithi071.stdout:9/59: creat d1/f17 x:0 0 0 2024-04-23T02:13:51.488 INFO:tasks.workunit.client.1.smithi192.stdout:2/201: dread d8/d25/f38 [0,4194304] 0 2024-04-23T02:13:51.489 INFO:tasks.workunit.client.1.smithi192.stdout:3/125: dwrite d1/f5 [4194304,4194304] 0 2024-04-23T02:13:51.489 INFO:tasks.workunit.client.1.smithi192.stdout:6/159: rmdir d3/d15 39 2024-04-23T02:13:51.490 INFO:tasks.workunit.client.0.smithi071.stdout:9/60: write d1/f13 [328703,57207] 0 2024-04-23T02:13:51.490 INFO:tasks.workunit.client.0.smithi071.stdout:6/48: dwrite f4 [0,4194304] 0 2024-04-23T02:13:51.491 INFO:tasks.workunit.client.0.smithi071.stdout:6/49: stat d6/c7 0 2024-04-23T02:13:51.493 INFO:tasks.workunit.client.0.smithi071.stdout:8/89: symlink dc/d15/l19 0 2024-04-23T02:13:51.495 INFO:tasks.workunit.client.0.smithi071.stdout:9/61: dread - d1/f16 zero size 2024-04-23T02:13:51.495 INFO:tasks.workunit.client.1.smithi192.stdout:9/124: dwrite d0/d3/f2e [0,4194304] 0 2024-04-23T02:13:51.496 INFO:tasks.workunit.client.0.smithi071.stdout:2/56: link d0/d2/c8 d0/d2/ce 0 2024-04-23T02:13:51.496 INFO:tasks.workunit.client.1.smithi192.stdout:8/192: dread d1/d9/fa [0,4194304] 0 2024-04-23T02:13:51.498 INFO:tasks.workunit.client.0.smithi071.stdout:8/90: chown dc/c13 705241874 1 2024-04-23T02:13:51.499 INFO:tasks.workunit.client.1.smithi192.stdout:3/126: read d1/d3/d6/f1c [3254702,107637] 0 2024-04-23T02:13:51.499 INFO:tasks.workunit.client.0.smithi071.stdout:9/62: truncate d1/fa 1061587 0 2024-04-23T02:13:51.501 INFO:tasks.workunit.client.0.smithi071.stdout:6/50: dwrite d6/fb [0,4194304] 0 2024-04-23T02:13:51.502 INFO:tasks.workunit.client.0.smithi071.stdout:2/57: mkdir d0/d2/dd/df 0 2024-04-23T02:13:51.502 INFO:tasks.workunit.client.0.smithi071.stdout:2/58: readlink - no filename 2024-04-23T02:13:51.502 INFO:tasks.workunit.client.0.smithi071.stdout:6/51: stat d6 0 2024-04-23T02:13:51.502 INFO:tasks.workunit.client.0.smithi071.stdout:6/52: readlink - no filename 2024-04-23T02:13:51.502 INFO:tasks.workunit.client.0.smithi071.stdout:6/53: readlink - no filename 2024-04-23T02:13:51.504 INFO:tasks.workunit.client.0.smithi071.stdout:9/63: creat d1/f18 x:0 0 0 2024-04-23T02:13:51.504 INFO:tasks.workunit.client.0.smithi071.stdout:2/59: read d0/d2/dd/fc [1848201,95570] 0 2024-04-23T02:13:51.505 INFO:tasks.workunit.client.1.smithi192.stdout:8/193: write d1/d12/d23/d2c/f38 [2538845,16783] 0 2024-04-23T02:13:51.509 INFO:tasks.workunit.client.0.smithi071.stdout:8/91: dread f6 [4194304,4194304] 0 2024-04-23T02:13:51.512 INFO:tasks.workunit.client.1.smithi192.stdout:6/160: dwrite d3/f2b [0,4194304] 0 2024-04-23T02:13:51.512 INFO:tasks.workunit.client.1.smithi192.stdout:8/194: mkdir d1/df/d41 0 2024-04-23T02:13:51.513 INFO:tasks.workunit.client.1.smithi192.stdout:0/242: dwrite d6/df/d13/d25/d3a/f4f [0,4194304] 0 2024-04-23T02:13:51.513 INFO:tasks.workunit.client.1.smithi192.stdout:9/125: dread d0/d9/fb [0,4194304] 0 2024-04-23T02:13:51.516 INFO:tasks.workunit.client.0.smithi071.stdout:8/92: truncate dc/d10/f14 729613 0 2024-04-23T02:13:51.519 INFO:tasks.workunit.client.0.smithi071.stdout:0/79: dread d0/d3/de/f13 [0,4194304] 0 2024-04-23T02:13:51.519 INFO:tasks.workunit.client.0.smithi071.stdout:6/54: dwrite d6/fd [4194304,4194304] 0 2024-04-23T02:13:51.521 INFO:tasks.workunit.client.0.smithi071.stdout:2/60: dread d0/d2/dd/fc [0,4194304] 0 2024-04-23T02:13:51.522 INFO:tasks.workunit.client.1.smithi192.stdout:2/202: link d8/d14/l22 d8/l3d 0 2024-04-23T02:13:51.522 INFO:tasks.workunit.client.0.smithi071.stdout:0/80: read d0/f5 [728835,102752] 0 2024-04-23T02:13:51.523 INFO:tasks.workunit.client.0.smithi071.stdout:9/64: dwrite d1/f10 [0,4194304] 0 2024-04-23T02:13:51.524 INFO:tasks.workunit.client.1.smithi192.stdout:0/243: mkdir d6/df/d56 0 2024-04-23T02:13:51.526 INFO:tasks.workunit.client.0.smithi071.stdout:0/81: read d0/f2 [2519128,13428] 0 2024-04-23T02:13:51.527 INFO:tasks.workunit.client.1.smithi192.stdout:8/195: creat d1/d12/d23/d2c/f42 x:0 0 0 2024-04-23T02:13:51.530 INFO:tasks.workunit.client.1.smithi192.stdout:2/203: chown d8/d26/d27/f2f 5179150 1 2024-04-23T02:13:51.532 INFO:tasks.workunit.client.1.smithi192.stdout:8/196: write d1/d12/f39 [351406,85546] 0 2024-04-23T02:13:51.532 INFO:tasks.workunit.client.1.smithi192.stdout:6/161: dwrite d3/f12 [0,4194304] 0 2024-04-23T02:13:51.533 INFO:tasks.workunit.client.0.smithi071.stdout:9/65: mknod d1/c19 0 2024-04-23T02:13:51.534 INFO:tasks.workunit.client.0.smithi071.stdout:8/93: getdents dc 0 2024-04-23T02:13:51.534 INFO:tasks.workunit.client.1.smithi192.stdout:6/162: chown d3/le 1192411 1 2024-04-23T02:13:51.537 INFO:tasks.workunit.client.0.smithi071.stdout:0/82: mkdir d0/d3/de/d10/d16 0 2024-04-23T02:13:51.537 INFO:tasks.workunit.client.1.smithi192.stdout:2/204: creat d8/d26/d27/f3e x:0 0 0 2024-04-23T02:13:51.538 INFO:tasks.workunit.client.0.smithi071.stdout:6/55: dwrite d6/fc [0,4194304] 0 2024-04-23T02:13:51.538 INFO:tasks.workunit.client.0.smithi071.stdout:8/94: creat dc/d10/f1a x:0 0 0 2024-04-23T02:13:51.542 INFO:tasks.workunit.client.1.smithi192.stdout:8/197: creat d1/df/d41/f43 x:0 0 0 2024-04-23T02:13:51.543 INFO:tasks.workunit.client.0.smithi071.stdout:8/95: mknod dc/c1b 0 2024-04-23T02:13:51.546 INFO:tasks.workunit.client.0.smithi071.stdout:0/83: symlink d0/d3/de/l17 0 2024-04-23T02:13:51.549 INFO:tasks.workunit.client.1.smithi192.stdout:2/205: rename d8/d26/d27/f3e to d8/d25/f3f 0 2024-04-23T02:13:51.549 INFO:tasks.workunit.client.1.smithi192.stdout:8/198: read d1/df/f1c [51171,26327] 0 2024-04-23T02:13:51.550 INFO:tasks.workunit.client.0.smithi071.stdout:9/66: dwrite d1/f10 [4194304,4194304] 0 2024-04-23T02:13:51.550 INFO:tasks.workunit.client.0.smithi071.stdout:6/56: dread f4 [4194304,4194304] 0 2024-04-23T02:13:51.551 INFO:tasks.workunit.client.0.smithi071.stdout:0/84: write d0/f5 [1410216,22222] 0 2024-04-23T02:13:51.551 INFO:tasks.workunit.client.1.smithi192.stdout:6/163: dwrite d3/d15/d1b/f2a [0,4194304] 0 2024-04-23T02:13:51.551 INFO:tasks.workunit.client.1.smithi192.stdout:2/206: read d8/d25/f38 [64491,62000] 0 2024-04-23T02:13:51.552 INFO:tasks.workunit.client.0.smithi071.stdout:9/67: stat d1/f12 0 2024-04-23T02:13:51.553 INFO:tasks.workunit.client.0.smithi071.stdout:9/68: chown d1/fa 3640 1 2024-04-23T02:13:51.554 INFO:tasks.workunit.client.0.smithi071.stdout:8/96: dread f6 [0,4194304] 0 2024-04-23T02:13:51.554 INFO:tasks.workunit.client.1.smithi192.stdout:6/164: read - d3/d11/f1d zero size 2024-04-23T02:13:51.557 INFO:tasks.workunit.client.0.smithi071.stdout:8/97: write dc/f18 [184109,94255] 0 2024-04-23T02:13:51.559 INFO:tasks.workunit.client.0.smithi071.stdout:9/69: write d1/fc [1845042,92104] 0 2024-04-23T02:13:51.560 INFO:tasks.workunit.client.0.smithi071.stdout:0/85: truncate d0/d3/de/f13 222917 0 2024-04-23T02:13:51.562 INFO:tasks.workunit.client.0.smithi071.stdout:9/70: mkdir d1/d1a 0 2024-04-23T02:13:51.562 INFO:tasks.workunit.client.1.smithi192.stdout:6/165: symlink d3/d15/d1b/d21/l2f 0 2024-04-23T02:13:51.563 INFO:tasks.workunit.client.0.smithi071.stdout:0/86: creat d0/f18 x:0 0 0 2024-04-23T02:13:51.565 INFO:tasks.workunit.client.1.smithi192.stdout:2/207: mknod d8/d14/c40 0 2024-04-23T02:13:51.566 INFO:tasks.workunit.client.1.smithi192.stdout:8/199: dwrite d1/d9/f11 [4194304,4194304] 0 2024-04-23T02:13:51.568 INFO:tasks.workunit.client.1.smithi192.stdout:9/126: dwrite d0/d3/f28 [0,4194304] 0 2024-04-23T02:13:51.570 INFO:tasks.workunit.client.1.smithi192.stdout:6/166: rename d3/d15/c27 to d3/d18/c30 0 2024-04-23T02:13:51.570 INFO:tasks.workunit.client.1.smithi192.stdout:2/208: fdatasync f4 0 2024-04-23T02:13:51.572 INFO:tasks.workunit.client.0.smithi071.stdout:0/87: dread d0/f5 [0,4194304] 0 2024-04-23T02:13:51.572 INFO:tasks.workunit.client.0.smithi071.stdout:9/71: dwrite d1/fe [0,4194304] 0 2024-04-23T02:13:51.572 INFO:tasks.workunit.client.1.smithi192.stdout:9/127: creat d0/d9/f32 x:0 0 0 2024-04-23T02:13:51.573 INFO:tasks.workunit.client.1.smithi192.stdout:9/128: write d0/f31 [2393076,39496] 0 2024-04-23T02:13:51.575 INFO:tasks.workunit.client.0.smithi071.stdout:2/61: truncate d0/d2/dd/fc 686941 0 2024-04-23T02:13:51.575 INFO:tasks.workunit.client.1.smithi192.stdout:9/129: chown d0/d1e/l2a 2 1 2024-04-23T02:13:51.576 INFO:tasks.workunit.client.1.smithi192.stdout:8/200: rename d1/d12/d23/d2c/c31 to d1/d12/d23/c44 0 2024-04-23T02:13:51.577 INFO:tasks.workunit.client.0.smithi071.stdout:9/72: symlink d1/l1b 0 2024-04-23T02:13:51.577 INFO:tasks.workunit.client.1.smithi192.stdout:6/167: dread d3/f2b [0,4194304] 0 2024-04-23T02:13:51.578 INFO:tasks.workunit.client.1.smithi192.stdout:2/209: write d8/d25/f3f [99574,24923] 0 2024-04-23T02:13:51.579 INFO:tasks.workunit.client.1.smithi192.stdout:6/168: chown d3/d15/d1b/c24 2015733 1 2024-04-23T02:13:51.579 INFO:tasks.workunit.client.0.smithi071.stdout:0/88: symlink d0/da/l19 0 2024-04-23T02:13:51.582 INFO:tasks.workunit.client.0.smithi071.stdout:2/62: chown d0/d2/ce 163664 1 2024-04-23T02:13:51.582 INFO:tasks.workunit.client.1.smithi192.stdout:8/201: mkdir d1/df/d41/d45 0 2024-04-23T02:13:51.583 INFO:tasks.workunit.client.0.smithi071.stdout:9/73: rename d1/f12 to d1/d1a/f1c 0 2024-04-23T02:13:51.584 INFO:tasks.workunit.client.1.smithi192.stdout:9/130: creat d0/f33 x:0 0 0 2024-04-23T02:13:51.585 INFO:tasks.workunit.client.0.smithi071.stdout:9/74: write d1/fa [97355,104896] 0 2024-04-23T02:13:51.586 INFO:tasks.workunit.client.1.smithi192.stdout:8/202: write d1/d9/f3f [324005,99104] 0 2024-04-23T02:13:51.587 INFO:tasks.workunit.client.0.smithi071.stdout:0/89: rename d0/f5 to d0/da/f1a 0 2024-04-23T02:13:51.589 INFO:tasks.workunit.client.0.smithi071.stdout:2/63: getdents d0/d2/dd 0 2024-04-23T02:13:51.589 INFO:tasks.workunit.client.1.smithi192.stdout:8/203: write d1/d9/f22 [123499,120230] 0 2024-04-23T02:13:51.590 INFO:tasks.workunit.client.1.smithi192.stdout:6/169: rename d3/d18/c2c to d3/d15/d1b/c31 0 2024-04-23T02:13:51.591 INFO:tasks.workunit.client.0.smithi071.stdout:2/64: write d0/d2/f9 [139464,95534] 0 2024-04-23T02:13:51.591 INFO:tasks.workunit.client.1.smithi192.stdout:6/170: read - d3/d11/f1d zero size 2024-04-23T02:13:51.596 INFO:tasks.workunit.client.0.smithi071.stdout:8/98: dread dc/f18 [0,4194304] 0 2024-04-23T02:13:51.596 INFO:tasks.workunit.client.1.smithi192.stdout:2/210: dread d8/f36 [4194304,4194304] 0 2024-04-23T02:13:51.597 INFO:tasks.workunit.client.0.smithi071.stdout:6/57: truncate d6/ff 792781 0 2024-04-23T02:13:51.597 INFO:tasks.workunit.client.1.smithi192.stdout:4/347: sync 2024-04-23T02:13:51.598 INFO:tasks.workunit.client.1.smithi192.stdout:9/131: dwrite d0/f16 [0,4194304] 0 2024-04-23T02:13:51.599 INFO:tasks.workunit.client.0.smithi071.stdout:8/99: creat dc/d10/f1c x:0 0 0 2024-04-23T02:13:51.600 INFO:tasks.workunit.client.0.smithi071.stdout:8/100: chown dc/f18 4 1 2024-04-23T02:13:51.603 INFO:tasks.workunit.client.1.smithi192.stdout:8/204: dread d1/d12/d23/d2c/f38 [0,4194304] 0 2024-04-23T02:13:51.603 INFO:tasks.workunit.client.1.smithi192.stdout:8/205: fdatasync d1/f15 0 2024-04-23T02:13:51.603 INFO:tasks.workunit.client.0.smithi071.stdout:6/58: dread f5 [0,4194304] 0 2024-04-23T02:13:51.604 INFO:tasks.workunit.client.0.smithi071.stdout:8/101: rename l1 to dc/l1d 0 2024-04-23T02:13:51.605 INFO:tasks.workunit.client.1.smithi192.stdout:2/211: creat d8/d26/d29/f41 x:0 0 0 2024-04-23T02:13:51.605 INFO:tasks.workunit.client.1.smithi192.stdout:4/348: mknod de/d15/d19/d1a/c6f 0 2024-04-23T02:13:51.607 INFO:tasks.workunit.client.0.smithi071.stdout:8/102: unlink cb 0 2024-04-23T02:13:51.607 INFO:tasks.workunit.client.1.smithi192.stdout:4/349: write de/f21 [1209155,105020] 0 2024-04-23T02:13:51.607 INFO:tasks.workunit.client.1.smithi192.stdout:9/132: write d0/d9/f10 [862378,29669] 0 2024-04-23T02:13:51.607 INFO:tasks.workunit.client.1.smithi192.stdout:9/133: chown d0/c29 4389 1 2024-04-23T02:13:51.609 INFO:tasks.workunit.client.0.smithi071.stdout:2/65: rmdir d0 39 2024-04-23T02:13:51.609 INFO:tasks.workunit.client.0.smithi071.stdout:2/66: readlink - no filename 2024-04-23T02:13:51.609 INFO:tasks.workunit.client.0.smithi071.stdout:6/59: fdatasync d6/ff 0 2024-04-23T02:13:51.610 INFO:tasks.workunit.client.1.smithi192.stdout:2/212: mknod d8/d14/c42 0 2024-04-23T02:13:51.610 INFO:tasks.workunit.client.0.smithi071.stdout:8/103: write dc/d10/f1c [941568,58516] 0 2024-04-23T02:13:51.611 INFO:tasks.workunit.client.0.smithi071.stdout:8/104: stat dc/ff 0 2024-04-23T02:13:51.612 INFO:tasks.workunit.client.0.smithi071.stdout:6/60: creat d6/f11 x:0 0 0 2024-04-23T02:13:51.612 INFO:tasks.workunit.client.0.smithi071.stdout:9/75: getdents d1/d1a 0 2024-04-23T02:13:51.615 INFO:tasks.workunit.client.1.smithi192.stdout:8/206: dwrite d1/d12/d23/f3c [0,4194304] 0 2024-04-23T02:13:51.616 INFO:tasks.workunit.client.1.smithi192.stdout:2/213: mkdir d8/d26/d29/d43 0 2024-04-23T02:13:51.617 INFO:tasks.workunit.client.0.smithi071.stdout:0/90: write d0/f2 [5150868,118837] 0 2024-04-23T02:13:51.618 INFO:tasks.workunit.client.0.smithi071.stdout:9/76: mknod d1/c1d 0 2024-04-23T02:13:51.620 INFO:tasks.workunit.client.1.smithi192.stdout:6/171: rmdir d3 39 2024-04-23T02:13:51.621 INFO:tasks.workunit.client.1.smithi192.stdout:4/350: getdents de/d15/d19/d3d/d34 0 2024-04-23T02:13:51.621 INFO:tasks.workunit.client.0.smithi071.stdout:8/105: dread f9 [0,4194304] 0 2024-04-23T02:13:51.622 INFO:tasks.workunit.client.0.smithi071.stdout:0/91: symlink d0/d3/de/d10/l1b 0 2024-04-23T02:13:51.623 INFO:tasks.workunit.client.1.smithi192.stdout:2/214: rename d8/d14/l30 to d8/d26/d29/d43/l44 0 2024-04-23T02:13:51.624 INFO:tasks.workunit.client.0.smithi071.stdout:6/61: dwrite d6/fb [0,4194304] 0 2024-04-23T02:13:51.625 INFO:tasks.workunit.client.1.smithi192.stdout:4/351: dread de/d15/d19/d1a/f3a [0,4194304] 0 2024-04-23T02:13:51.625 INFO:tasks.workunit.client.1.smithi192.stdout:6/172: truncate d3/d18/f2d 939427 0 2024-04-23T02:13:51.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:51 smithi192 ceph-mon[49662]: pgmap v22: 65 pgs: 65 active+clean; 398 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 26 MiB/s wr, 164 op/s 2024-04-23T02:13:51.627 INFO:tasks.workunit.client.1.smithi192.stdout:4/352: fsync de/d15/d19/d3d/d49/d26/d59/f63 0 2024-04-23T02:13:51.628 INFO:tasks.workunit.client.0.smithi071.stdout:6/62: truncate d6/fb 4714077 0 2024-04-23T02:13:51.629 INFO:tasks.workunit.client.1.smithi192.stdout:8/207: mkdir d1/df/d41/d45/d46 0 2024-04-23T02:13:51.629 INFO:tasks.workunit.client.0.smithi071.stdout:0/92: rename d0/d3/dc to d0/d3/d1c 0 2024-04-23T02:13:51.630 INFO:tasks.workunit.client.0.smithi071.stdout:3/24: sync 2024-04-23T02:13:51.630 INFO:tasks.workunit.client.1.smithi192.stdout:6/173: creat d3/d15/d1b/d21/f32 x:0 0 0 2024-04-23T02:13:51.630 INFO:tasks.workunit.client.0.smithi071.stdout:6/63: read f1 [1498366,86347] 0 2024-04-23T02:13:51.633 INFO:tasks.workunit.client.0.smithi071.stdout:9/77: dwrite d1/f10 [8388608,4194304] 0 2024-04-23T02:13:51.633 INFO:tasks.workunit.client.1.smithi192.stdout:6/174: truncate d3/d11/f1d 672141 0 2024-04-23T02:13:51.634 INFO:tasks.workunit.client.1.smithi192.stdout:4/353: mkdir de/d15/d19/d3d/d49/d70 0 2024-04-23T02:13:51.634 INFO:tasks.workunit.client.0.smithi071.stdout:3/25: symlink l4 0 2024-04-23T02:13:51.635 INFO:tasks.workunit.client.0.smithi071.stdout:8/106: dread f9 [0,4194304] 0 2024-04-23T02:13:51.636 INFO:tasks.workunit.client.0.smithi071.stdout:3/26: chown l3 4 1 2024-04-23T02:13:51.637 INFO:tasks.workunit.client.1.smithi192.stdout:8/208: creat d1/df/d41/d45/d46/f47 x:0 0 0 2024-04-23T02:13:51.637 INFO:tasks.workunit.client.0.smithi071.stdout:3/27: stat c1 0 2024-04-23T02:13:51.637 INFO:tasks.workunit.client.1.smithi192.stdout:6/175: creat d3/d11/f33 x:0 0 0 2024-04-23T02:13:51.642 INFO:tasks.workunit.client.1.smithi192.stdout:6/176: write d3/f12 [4152253,30476] 0 2024-04-23T02:13:51.642 INFO:tasks.workunit.client.1.smithi192.stdout:2/215: dwrite d8/d26/d29/f24 [0,4194304] 0 2024-04-23T02:13:51.644 INFO:tasks.workunit.client.1.smithi192.stdout:6/177: fsync d3/d11/f1d 0 2024-04-23T02:13:51.646 INFO:tasks.workunit.client.0.smithi071.stdout:0/93: getdents d0/d3/de/d10/d16 0 2024-04-23T02:13:51.647 INFO:tasks.workunit.client.1.smithi192.stdout:9/134: dwrite d0/f2c [0,4194304] 0 2024-04-23T02:13:51.648 INFO:tasks.workunit.client.1.smithi192.stdout:8/209: write d1/df/d41/d45/d46/f47 [24117,94865] 0 2024-04-23T02:13:51.650 INFO:tasks.workunit.client.0.smithi071.stdout:3/28: creat f5 x:0 0 0 2024-04-23T02:13:51.651 INFO:tasks.workunit.client.0.smithi071.stdout:3/29: symlink l6 0 2024-04-23T02:13:51.655 INFO:tasks.workunit.client.1.smithi192.stdout:2/216: truncate d8/f18 608322 0 2024-04-23T02:13:51.656 INFO:tasks.workunit.client.0.smithi071.stdout:3/30: mkdir d7 0 2024-04-23T02:13:51.657 INFO:tasks.workunit.client.0.smithi071.stdout:8/107: getdents dc/d15 0 2024-04-23T02:13:51.658 INFO:tasks.workunit.client.0.smithi071.stdout:8/108: fdatasync f8 0 2024-04-23T02:13:51.660 INFO:tasks.workunit.client.0.smithi071.stdout:0/94: link d0/da/f1a d0/d3/de/f1d 0 2024-04-23T02:13:51.660 INFO:tasks.workunit.client.1.smithi192.stdout:6/178: dwrite d3/d11/f13 [0,4194304] 0 2024-04-23T02:13:51.661 INFO:tasks.workunit.client.1.smithi192.stdout:8/210: unlink d1/c1b 0 2024-04-23T02:13:51.661 INFO:tasks.workunit.client.0.smithi071.stdout:0/95: chown d0/d3/d1c/l12 40152 1 2024-04-23T02:13:51.663 INFO:tasks.workunit.client.1.smithi192.stdout:2/217: creat d8/d26/d27/f45 x:0 0 0 2024-04-23T02:13:51.664 INFO:tasks.workunit.client.0.smithi071.stdout:8/109: creat dc/d10/f1e x:0 0 0 2024-04-23T02:13:51.664 INFO:tasks.workunit.client.0.smithi071.stdout:0/96: write d0/f2 [1758724,29158] 0 2024-04-23T02:13:51.665 INFO:tasks.workunit.client.1.smithi192.stdout:9/135: getdents d0/d3 0 2024-04-23T02:13:51.666 INFO:tasks.workunit.client.1.smithi192.stdout:8/211: creat d1/df/d41/d45/d46/f48 x:0 0 0 2024-04-23T02:13:51.667 INFO:tasks.workunit.client.1.smithi192.stdout:9/136: stat d0/d9/f26 0 2024-04-23T02:13:51.668 INFO:tasks.workunit.client.1.smithi192.stdout:4/354: dwrite de/d15/d19/d1a/f23 [0,4194304] 0 2024-04-23T02:13:51.668 INFO:tasks.workunit.client.0.smithi071.stdout:1/44: sync 2024-04-23T02:13:51.668 INFO:tasks.workunit.client.0.smithi071.stdout:4/89: sync 2024-04-23T02:13:51.669 INFO:tasks.workunit.client.0.smithi071.stdout:5/100: sync 2024-04-23T02:13:51.669 INFO:tasks.workunit.client.0.smithi071.stdout:7/94: sync 2024-04-23T02:13:51.669 INFO:tasks.workunit.client.0.smithi071.stdout:6/64: truncate d6/fd 7707619 0 2024-04-23T02:13:51.671 INFO:tasks.workunit.client.0.smithi071.stdout:3/31: fsync f5 0 2024-04-23T02:13:51.671 INFO:tasks.workunit.client.1.smithi192.stdout:9/137: write d0/f33 [212260,65331] 0 2024-04-23T02:13:51.672 INFO:tasks.workunit.client.1.smithi192.stdout:8/212: mkdir d1/d12/d23/d2c/d49 0 2024-04-23T02:13:51.672 INFO:tasks.workunit.client.1.smithi192.stdout:4/355: chown de/d15/d19/d3d/d34/d4a/c4f 336339 1 2024-04-23T02:13:51.675 INFO:tasks.workunit.client.1.smithi192.stdout:6/179: dwrite f1 [0,4194304] 0 2024-04-23T02:13:51.675 INFO:tasks.workunit.client.0.smithi071.stdout:9/78: truncate d1/f10 1873440 0 2024-04-23T02:13:51.676 INFO:tasks.workunit.client.0.smithi071.stdout:5/101: chown d3/lc 215 1 2024-04-23T02:13:51.677 INFO:tasks.workunit.client.0.smithi071.stdout:4/90: write d2/d15/d1d/d23/f1f [104019,69307] 0 2024-04-23T02:13:51.677 INFO:tasks.workunit.client.1.smithi192.stdout:8/213: link d1/d9/f11 d1/f4a 0 2024-04-23T02:13:51.679 INFO:tasks.workunit.client.1.smithi192.stdout:9/138: creat d0/f34 x:0 0 0 2024-04-23T02:13:51.680 INFO:tasks.workunit.client.0.smithi071.stdout:1/45: write f2 [2517071,127778] 0 2024-04-23T02:13:51.681 INFO:tasks.workunit.client.0.smithi071.stdout:5/102: read - d3/f29 zero size 2024-04-23T02:13:51.683 INFO:tasks.workunit.client.1.smithi192.stdout:8/214: mkdir d1/d12/d23/d4b 0 2024-04-23T02:13:51.684 INFO:tasks.workunit.client.0.smithi071.stdout:7/95: mknod d3/d17/d1c/c24 0 2024-04-23T02:13:51.684 INFO:tasks.workunit.client.1.smithi192.stdout:9/139: readlink d0/d3/l17 0 2024-04-23T02:13:51.687 INFO:tasks.workunit.client.1.smithi192.stdout:4/356: creat de/d15/d19/d3d/d34/d4a/f71 x:0 0 0 2024-04-23T02:13:51.687 INFO:tasks.workunit.client.0.smithi071.stdout:9/79: symlink d1/d1a/l1e 0 2024-04-23T02:13:51.687 INFO:tasks.workunit.client.1.smithi192.stdout:8/215: readlink d1/d12/d23/d2c/l29 0 2024-04-23T02:13:51.689 INFO:tasks.workunit.client.0.smithi071.stdout:0/97: dwrite d0/f2 [4194304,4194304] 0 2024-04-23T02:13:51.693 INFO:tasks.workunit.client.0.smithi071.stdout:5/103: chown d3/d1d 22823 1 2024-04-23T02:13:51.694 INFO:tasks.workunit.client.0.smithi071.stdout:8/110: dwrite dc/d10/f1e [0,4194304] 0 2024-04-23T02:13:51.694 INFO:tasks.workunit.client.0.smithi071.stdout:3/32: mkdir d7/d8 0 2024-04-23T02:13:51.696 INFO:tasks.workunit.client.0.smithi071.stdout:4/91: fdatasync d2/d15/d1d/d23/f16 0 2024-04-23T02:13:51.696 INFO:tasks.workunit.client.0.smithi071.stdout:2/67: dwrite d0/d2/dd/fc [0,4194304] 0 2024-04-23T02:13:51.698 INFO:tasks.workunit.client.0.smithi071.stdout:5/104: fsync d3/f1f 0 2024-04-23T02:13:51.698 INFO:tasks.workunit.client.0.smithi071.stdout:9/80: write d1/fa [1212047,11813] 0 2024-04-23T02:13:51.702 INFO:tasks.workunit.client.0.smithi071.stdout:1/46: mknod db/cc 0 2024-04-23T02:13:51.702 INFO:tasks.workunit.client.0.smithi071.stdout:4/92: chown d2/l5 486622 1 2024-04-23T02:13:51.703 INFO:tasks.workunit.client.1.smithi192.stdout:4/357: chown de/d15/d19/d3d/d49/d70 122609712 1 2024-04-23T02:13:51.703 INFO:tasks.workunit.client.1.smithi192.stdout:6/180: dwrite d3/d11/f13 [4194304,4194304] 0 2024-04-23T02:13:51.703 INFO:tasks.workunit.client.1.smithi192.stdout:9/140: dwrite d0/d9/f26 [0,4194304] 0 2024-04-23T02:13:51.703 INFO:tasks.workunit.client.0.smithi071.stdout:0/98: dread d0/f1 [0,4194304] 0 2024-04-23T02:13:51.707 INFO:tasks.workunit.client.0.smithi071.stdout:8/111: write f9 [4245951,93514] 0 2024-04-23T02:13:51.707 INFO:tasks.workunit.client.0.smithi071.stdout:3/33: creat d7/d8/f9 x:0 0 0 2024-04-23T02:13:51.709 INFO:tasks.workunit.client.0.smithi071.stdout:4/93: truncate d2/d15/f1e 869088 0 2024-04-23T02:13:51.712 INFO:tasks.workunit.client.0.smithi071.stdout:0/99: creat d0/d3/de/d10/d14/f1e x:0 0 0 2024-04-23T02:13:51.717 INFO:tasks.workunit.client.0.smithi071.stdout:0/100: link d0/d3/de/d10/d14/f1e d0/d3/de/d10/f1f 0 2024-04-23T02:13:51.718 INFO:tasks.workunit.client.0.smithi071.stdout:9/81: getdents d1/d1a 0 2024-04-23T02:13:51.719 INFO:tasks.workunit.client.0.smithi071.stdout:9/82: dread - d1/f17 zero size 2024-04-23T02:13:51.720 INFO:tasks.workunit.client.0.smithi071.stdout:0/101: creat d0/f20 x:0 0 0 2024-04-23T02:13:51.720 INFO:tasks.workunit.client.1.smithi192.stdout:8/216: dwrite d1/d12/d23/d2c/f42 [0,4194304] 0 2024-04-23T02:13:51.722 INFO:tasks.workunit.client.0.smithi071.stdout:0/102: creat d0/d3/d1c/f21 x:0 0 0 2024-04-23T02:13:51.725 INFO:tasks.workunit.client.0.smithi071.stdout:7/96: write d3/f18 [224983,74301] 0 2024-04-23T02:13:51.727 INFO:tasks.workunit.client.0.smithi071.stdout:7/97: write d3/f18 [362211,60100] 0 2024-04-23T02:13:51.728 INFO:tasks.workunit.client.0.smithi071.stdout:9/83: dwrite d1/fc [0,4194304] 0 2024-04-23T02:13:51.728 INFO:tasks.workunit.client.0.smithi071.stdout:7/98: fdatasync d3/fc 0 2024-04-23T02:13:51.729 INFO:tasks.workunit.client.1.smithi192.stdout:6/181: dwrite d3/d11/f33 [0,4194304] 0 2024-04-23T02:13:51.733 INFO:tasks.workunit.client.0.smithi071.stdout:0/103: dread d0/f2 [4194304,4194304] 0 2024-04-23T02:13:51.734 INFO:tasks.workunit.client.0.smithi071.stdout:2/68: fsync d0/d2/dd/fc 0 2024-04-23T02:13:51.741 INFO:tasks.workunit.client.0.smithi071.stdout:1/47: truncate f7 739168 0 2024-04-23T02:13:51.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:51 smithi071 ceph-mon[39837]: pgmap v22: 65 pgs: 65 active+clean; 398 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 26 MiB/s wr, 164 op/s 2024-04-23T02:13:51.745 INFO:tasks.workunit.client.0.smithi071.stdout:1/48: dread - f6 zero size 2024-04-23T02:13:51.746 INFO:tasks.workunit.client.1.smithi192.stdout:4/358: link de/f21 de/d15/d19/d1a/f72 0 2024-04-23T02:13:51.747 INFO:tasks.workunit.client.0.smithi071.stdout:7/99: dwrite d3/d17/f22 [0,4194304] 0 2024-04-23T02:13:51.748 INFO:tasks.workunit.client.0.smithi071.stdout:5/105: dwrite d3/da/f13 [0,4194304] 0 2024-04-23T02:13:51.749 INFO:tasks.workunit.client.1.smithi192.stdout:6/182: dread d3/d11/f29 [0,4194304] 0 2024-04-23T02:13:51.751 INFO:tasks.workunit.client.1.smithi192.stdout:6/183: chown d3/l1c 304843 1 2024-04-23T02:13:51.751 INFO:tasks.workunit.client.0.smithi071.stdout:1/49: creat db/fd x:0 0 0 2024-04-23T02:13:51.751 INFO:tasks.workunit.client.0.smithi071.stdout:2/69: dwrite d0/d2/fb [0,4194304] 0 2024-04-23T02:13:51.752 INFO:tasks.workunit.client.0.smithi071.stdout:7/100: mkdir d3/d17/d1c/d25 0 2024-04-23T02:13:51.754 INFO:tasks.workunit.client.0.smithi071.stdout:5/106: creat d3/d1d/f2a x:0 0 0 2024-04-23T02:13:51.754 INFO:tasks.workunit.client.0.smithi071.stdout:2/70: chown d0/d2/fb 11316 1 2024-04-23T02:13:51.755 INFO:tasks.workunit.client.0.smithi071.stdout:1/50: creat db/fe x:0 0 0 2024-04-23T02:13:51.758 INFO:tasks.workunit.client.1.smithi192.stdout:6/184: creat d3/f34 x:0 0 0 2024-04-23T02:13:51.759 INFO:tasks.workunit.client.0.smithi071.stdout:2/71: write d0/f6 [1643525,12749] 0 2024-04-23T02:13:51.762 INFO:tasks.workunit.client.0.smithi071.stdout:1/51: mkdir db/df 0 2024-04-23T02:13:51.763 INFO:tasks.workunit.client.1.smithi192.stdout:6/185: mknod d3/d15/c35 0 2024-04-23T02:13:51.763 INFO:tasks.workunit.client.0.smithi071.stdout:2/72: symlink d0/l10 0 2024-04-23T02:13:51.764 INFO:tasks.workunit.client.0.smithi071.stdout:2/73: chown d0/d2/fb 0 1 2024-04-23T02:13:51.765 INFO:tasks.workunit.client.1.smithi192.stdout:6/186: write d3/f2b [3082306,83453] 0 2024-04-23T02:13:51.766 INFO:tasks.workunit.client.0.smithi071.stdout:2/74: mknod d0/d2/dd/c11 0 2024-04-23T02:13:51.769 INFO:tasks.workunit.client.0.smithi071.stdout:7/101: truncate d3/fc 3068991 0 2024-04-23T02:13:51.769 INFO:tasks.workunit.client.1.smithi192.stdout:9/141: truncate d0/d9/f10 645873 0 2024-04-23T02:13:51.769 INFO:tasks.workunit.client.0.smithi071.stdout:2/75: rmdir d0 39 2024-04-23T02:13:51.770 INFO:tasks.workunit.client.0.smithi071.stdout:9/84: write d1/f16 [251013,62741] 0 2024-04-23T02:13:51.771 INFO:tasks.workunit.client.1.smithi192.stdout:8/217: write d1/d12/f1e [7182856,98092] 0 2024-04-23T02:13:51.772 INFO:tasks.workunit.client.1.smithi192.stdout:9/142: symlink d0/d3/l35 0 2024-04-23T02:13:51.777 INFO:tasks.workunit.client.1.smithi192.stdout:8/218: dread d1/d12/f1e [0,4194304] 0 2024-04-23T02:13:51.779 INFO:tasks.workunit.client.1.smithi192.stdout:8/219: mkdir d1/df/d41/d4c 0 2024-04-23T02:13:51.779 INFO:tasks.workunit.client.0.smithi071.stdout:9/85: rename d1/c2 to d1/d1a/c1f 0 2024-04-23T02:13:51.780 INFO:tasks.workunit.client.1.smithi192.stdout:8/220: read d1/fe [189478,75772] 0 2024-04-23T02:13:51.780 INFO:tasks.workunit.client.0.smithi071.stdout:2/76: rename d0/d2 to d0/d2/dd/df/d12 22 2024-04-23T02:13:51.782 INFO:tasks.workunit.client.0.smithi071.stdout:0/104: truncate d0/d3/de/ff 3227747 0 2024-04-23T02:13:51.782 INFO:tasks.workunit.client.0.smithi071.stdout:9/86: mknod d1/d1a/c20 0 2024-04-23T02:13:51.783 INFO:tasks.workunit.client.1.smithi192.stdout:8/221: mkdir d1/d12/d23/d2c/d4d 0 2024-04-23T02:13:51.784 INFO:tasks.workunit.client.0.smithi071.stdout:2/77: symlink d0/d2/l13 0 2024-04-23T02:13:51.785 INFO:tasks.workunit.client.0.smithi071.stdout:7/102: dwrite d3/d17/d1c/f20 [0,4194304] 0 2024-04-23T02:13:51.786 INFO:tasks.workunit.client.0.smithi071.stdout:2/78: chown d0/d2/dd 100165804 1 2024-04-23T02:13:51.788 INFO:tasks.workunit.client.1.smithi192.stdout:4/359: dwrite de/d15/d2a/f4c [0,4194304] 0 2024-04-23T02:13:51.790 INFO:tasks.workunit.client.1.smithi192.stdout:8/222: rmdir d1/d12/d23/d2c/d49 0 2024-04-23T02:13:51.794 INFO:tasks.workunit.client.0.smithi071.stdout:5/107: truncate d3/da/f13 2448724 0 2024-04-23T02:13:51.795 INFO:tasks.workunit.client.0.smithi071.stdout:7/103: getdents d3 0 2024-04-23T02:13:51.799 INFO:tasks.workunit.client.1.smithi192.stdout:6/187: truncate f1 2107032 0 2024-04-23T02:13:51.799 INFO:tasks.workunit.client.1.smithi192.stdout:9/143: truncate d0/f2c 1509884 0 2024-04-23T02:13:51.799 INFO:tasks.workunit.client.1.smithi192.stdout:8/223: unlink d1/d12/d23/d2c/f42 0 2024-04-23T02:13:51.800 INFO:tasks.workunit.client.1.smithi192.stdout:4/360: mkdir de/d15/d19/d3d/d49/d70/d73 0 2024-04-23T02:13:51.800 INFO:tasks.workunit.client.1.smithi192.stdout:8/224: readlink d1/d12/d23/d2c/l27 0 2024-04-23T02:13:51.800 INFO:tasks.workunit.client.1.smithi192.stdout:6/188: creat d3/d15/d1b/d21/f36 x:0 0 0 2024-04-23T02:13:51.801 INFO:tasks.workunit.client.1.smithi192.stdout:6/189: chown d3/le 161 1 2024-04-23T02:13:51.802 INFO:tasks.workunit.client.1.smithi192.stdout:6/190: chown d3/d11/f33 32 1 2024-04-23T02:13:51.802 INFO:tasks.workunit.client.0.smithi071.stdout:9/87: rmdir d1 39 2024-04-23T02:13:51.804 INFO:tasks.workunit.client.0.smithi071.stdout:7/104: creat d3/d17/d1c/f26 x:0 0 0 2024-04-23T02:13:51.807 INFO:tasks.workunit.client.1.smithi192.stdout:6/191: symlink d3/l37 0 2024-04-23T02:13:51.808 INFO:tasks.workunit.client.0.smithi071.stdout:2/79: dwrite d0/d2/fb [0,4194304] 0 2024-04-23T02:13:51.809 INFO:tasks.workunit.client.1.smithi192.stdout:8/225: dwrite d1/d12/d23/f40 [0,4194304] 0 2024-04-23T02:13:51.809 INFO:tasks.workunit.client.1.smithi192.stdout:6/192: write d3/d11/f33 [5120192,83138] 0 2024-04-23T02:13:51.811 INFO:tasks.workunit.client.1.smithi192.stdout:6/193: write d3/d15/d1b/f2a [2124456,90913] 0 2024-04-23T02:13:51.812 INFO:tasks.workunit.client.0.smithi071.stdout:9/88: fdatasync d1/f4 0 2024-04-23T02:13:51.814 INFO:tasks.workunit.client.0.smithi071.stdout:5/108: link d3/f11 d3/da/d1a/f2b 0 2024-04-23T02:13:51.815 INFO:tasks.workunit.client.1.smithi192.stdout:8/226: mknod d1/d12/d23/d2c/d4d/c4e 0 2024-04-23T02:13:51.815 INFO:tasks.workunit.client.0.smithi071.stdout:7/105: creat d3/d17/d1c/d25/f27 x:0 0 0 2024-04-23T02:13:51.816 INFO:tasks.workunit.client.0.smithi071.stdout:9/89: unlink d1/f13 0 2024-04-23T02:13:51.817 INFO:tasks.workunit.client.0.smithi071.stdout:9/90: fsync d1/f16 0 2024-04-23T02:13:51.817 INFO:tasks.workunit.client.0.smithi071.stdout:1/52: dwrite f7 [0,4194304] 0 2024-04-23T02:13:51.818 INFO:tasks.workunit.client.0.smithi071.stdout:2/80: dread d0/d2/fb [0,4194304] 0 2024-04-23T02:13:51.822 INFO:tasks.workunit.client.1.smithi192.stdout:8/227: dread d1/d12/f24 [0,4194304] 0 2024-04-23T02:13:51.828 INFO:tasks.workunit.client.1.smithi192.stdout:8/228: dwrite d1/f3 [0,4194304] 0 2024-04-23T02:13:51.830 INFO:tasks.workunit.client.0.smithi071.stdout:1/53: truncate f5 623477 0 2024-04-23T02:13:51.831 INFO:tasks.workunit.client.0.smithi071.stdout:1/54: chown f7 340 1 2024-04-23T02:13:51.833 INFO:tasks.workunit.client.1.smithi192.stdout:8/229: mknod d1/d9/d3e/c4f 0 2024-04-23T02:13:51.834 INFO:tasks.workunit.client.1.smithi192.stdout:8/230: chown d1/df/c2b 0 1 2024-04-23T02:13:51.835 INFO:tasks.workunit.client.0.smithi071.stdout:5/109: write d3/f11 [311564,10278] 0 2024-04-23T02:13:51.835 INFO:tasks.workunit.client.0.smithi071.stdout:7/106: write d3/fc [2485296,22928] 0 2024-04-23T02:13:51.836 INFO:tasks.workunit.client.0.smithi071.stdout:1/55: mknod db/df/c10 0 2024-04-23T02:13:51.837 INFO:tasks.workunit.client.0.smithi071.stdout:7/107: dread - d3/ff zero size 2024-04-23T02:13:51.837 INFO:tasks.workunit.client.1.smithi192.stdout:8/231: creat d1/df/f50 x:0 0 0 2024-04-23T02:13:51.838 INFO:tasks.workunit.client.1.smithi192.stdout:9/144: truncate d0/d3/f28 2154861 0 2024-04-23T02:13:51.838 INFO:tasks.workunit.client.0.smithi071.stdout:9/91: dread d1/f16 [0,4194304] 0 2024-04-23T02:13:51.838 INFO:tasks.workunit.client.1.smithi192.stdout:8/232: write d1/f15 [1641741,118896] 0 2024-04-23T02:13:51.841 INFO:tasks.workunit.client.1.smithi192.stdout:9/145: creat d0/d9/f36 x:0 0 0 2024-04-23T02:13:51.842 INFO:tasks.workunit.client.0.smithi071.stdout:2/81: dwrite d0/d2/fb [0,4194304] 0 2024-04-23T02:13:51.844 INFO:tasks.workunit.client.0.smithi071.stdout:1/56: mkdir db/d11 0 2024-04-23T02:13:51.845 INFO:tasks.workunit.client.0.smithi071.stdout:2/82: write d0/f6 [1975321,74863] 0 2024-04-23T02:13:51.846 INFO:tasks.workunit.client.0.smithi071.stdout:7/108: mknod d3/d17/d1c/c28 0 2024-04-23T02:13:51.850 INFO:tasks.workunit.client.0.smithi071.stdout:2/83: fdatasync d0/d2/dd/fc 0 2024-04-23T02:13:51.852 INFO:tasks.workunit.client.1.smithi192.stdout:9/146: dread d0/d3/f8 [0,4194304] 0 2024-04-23T02:13:51.855 INFO:tasks.workunit.client.0.smithi071.stdout:7/109: rename d3/f18 to d3/d17/d1c/f29 0 2024-04-23T02:13:51.856 INFO:tasks.workunit.client.1.smithi192.stdout:6/194: dread f1 [0,4194304] 0 2024-04-23T02:13:51.861 INFO:tasks.workunit.client.0.smithi071.stdout:7/110: mkdir d3/d17/d1c/d2a 0 2024-04-23T02:13:51.861 INFO:tasks.workunit.client.0.smithi071.stdout:9/92: dread d1/fe [0,4194304] 0 2024-04-23T02:13:51.862 INFO:tasks.workunit.client.0.smithi071.stdout:2/84: dread d0/d2/f9 [0,4194304] 0 2024-04-23T02:13:51.862 INFO:tasks.workunit.client.0.smithi071.stdout:5/110: dwrite d3/da/d1a/f26 [0,4194304] 0 2024-04-23T02:13:51.864 INFO:tasks.workunit.client.1.smithi192.stdout:6/195: truncate d3/d28/f2e 37767 0 2024-04-23T02:13:51.864 INFO:tasks.workunit.client.1.smithi192.stdout:6/196: chown d3/d11/f29 353584200 1 2024-04-23T02:13:51.865 INFO:tasks.workunit.client.0.smithi071.stdout:9/93: rmdir d1 39 2024-04-23T02:13:51.866 INFO:tasks.workunit.client.0.smithi071.stdout:2/85: mkdir d0/d2/dd/d14 0 2024-04-23T02:13:51.866 INFO:tasks.workunit.client.1.smithi192.stdout:9/147: rename d0/f31 to d0/d9/f37 0 2024-04-23T02:13:51.866 INFO:tasks.workunit.client.1.smithi192.stdout:6/197: rename d3/d11 to d3/d11/d38 22 2024-04-23T02:13:51.868 INFO:tasks.workunit.client.0.smithi071.stdout:7/111: creat d3/d17/d1c/f2b x:0 0 0 2024-04-23T02:13:51.868 INFO:tasks.workunit.client.1.smithi192.stdout:9/148: creat d0/d9/f38 x:0 0 0 2024-04-23T02:13:51.868 INFO:tasks.workunit.client.0.smithi071.stdout:2/86: stat d0/d2/fb 0 2024-04-23T02:13:51.869 INFO:tasks.workunit.client.1.smithi192.stdout:6/198: mknod d3/d15/d1b/d21/c39 0 2024-04-23T02:13:51.869 INFO:tasks.workunit.client.0.smithi071.stdout:1/57: dwrite db/fe [0,4194304] 0 2024-04-23T02:13:51.871 INFO:tasks.workunit.client.1.smithi192.stdout:9/149: unlink d0/d3/l1d 0 2024-04-23T02:13:51.873 INFO:tasks.workunit.client.1.smithi192.stdout:6/199: fdatasync d3/d28/f2e 0 2024-04-23T02:13:51.875 INFO:tasks.workunit.client.0.smithi071.stdout:9/94: mknod d1/c21 0 2024-04-23T02:13:51.875 INFO:tasks.workunit.client.0.smithi071.stdout:1/58: rename db/df/c10 to db/c12 0 2024-04-23T02:13:51.877 INFO:tasks.workunit.client.1.smithi192.stdout:6/200: stat d3/d15/d1b/c20 0 2024-04-23T02:13:51.879 INFO:tasks.workunit.client.0.smithi071.stdout:7/112: link d3/fc d3/d17/d1c/f2c 0 2024-04-23T02:13:51.880 INFO:tasks.workunit.client.0.smithi071.stdout:5/111: truncate d3/da/fd 3972034 0 2024-04-23T02:13:51.880 INFO:tasks.workunit.client.1.smithi192.stdout:6/201: write d3/d15/d1b/d21/f32 [342014,53814] 0 2024-04-23T02:13:51.880 INFO:tasks.workunit.client.1.smithi192.stdout:9/150: dwrite d0/f34 [0,4194304] 0 2024-04-23T02:13:51.881 INFO:tasks.workunit.client.0.smithi071.stdout:7/113: stat d3/d17/d1c/f26 0 2024-04-23T02:13:51.881 INFO:tasks.workunit.client.0.smithi071.stdout:9/95: dwrite d1/fc [0,4194304] 0 2024-04-23T02:13:51.882 INFO:tasks.workunit.client.1.smithi192.stdout:9/151: write d0/f16 [4781384,19772] 0 2024-04-23T02:13:51.884 INFO:tasks.workunit.client.1.smithi192.stdout:6/202: mkdir d3/d15/d1b/d21/d3a 0 2024-04-23T02:13:51.885 INFO:tasks.workunit.client.0.smithi071.stdout:9/96: write d1/f16 [815497,117099] 0 2024-04-23T02:13:51.886 INFO:tasks.workunit.client.1.smithi192.stdout:9/152: creat d0/d3/f39 x:0 0 0 2024-04-23T02:13:51.888 INFO:tasks.workunit.client.0.smithi071.stdout:7/114: mknod d3/d17/c2d 0 2024-04-23T02:13:51.889 INFO:tasks.workunit.client.1.smithi192.stdout:6/203: link d3/d11/c1e d3/d15/c3b 0 2024-04-23T02:13:51.890 INFO:tasks.workunit.client.0.smithi071.stdout:9/97: truncate d1/f17 15698 0 2024-04-23T02:13:51.894 INFO:tasks.workunit.client.0.smithi071.stdout:1/59: getdents db 0 2024-04-23T02:13:51.896 INFO:tasks.workunit.client.0.smithi071.stdout:1/60: write f2 [1551568,31309] 0 2024-04-23T02:13:51.898 INFO:tasks.workunit.client.0.smithi071.stdout:9/98: creat d1/d1a/f22 x:0 0 0 2024-04-23T02:13:51.899 INFO:tasks.workunit.client.1.smithi192.stdout:9/153: chown d0/d3/l17 10495 1 2024-04-23T02:13:51.902 INFO:tasks.workunit.client.0.smithi071.stdout:9/99: write d1/f17 [444583,99034] 0 2024-04-23T02:13:51.902 INFO:tasks.workunit.client.0.smithi071.stdout:7/115: write d3/fc [103484,77450] 0 2024-04-23T02:13:51.904 INFO:tasks.workunit.client.0.smithi071.stdout:7/116: dread - d3/f1d zero size 2024-04-23T02:13:51.905 INFO:tasks.workunit.client.0.smithi071.stdout:5/112: dwrite d3/da/d1a/f2b [4194304,4194304] 0 2024-04-23T02:13:51.908 INFO:tasks.workunit.client.1.smithi192.stdout:9/154: dwrite d0/d9/f14 [0,4194304] 0 2024-04-23T02:13:51.908 INFO:tasks.workunit.client.0.smithi071.stdout:5/113: readlink d3/df/d10/l28 0 2024-04-23T02:13:51.909 INFO:tasks.workunit.client.0.smithi071.stdout:7/117: dread - d3/d17/d1c/f2b zero size 2024-04-23T02:13:51.909 INFO:tasks.workunit.client.0.smithi071.stdout:9/100: unlink d1/c14 0 2024-04-23T02:13:51.911 INFO:tasks.workunit.client.1.smithi192.stdout:6/204: dwrite d3/d28/f2e [0,4194304] 0 2024-04-23T02:13:51.911 INFO:tasks.workunit.client.0.smithi071.stdout:5/114: chown d3/f29 22 1 2024-04-23T02:13:51.913 INFO:tasks.workunit.client.0.smithi071.stdout:9/101: creat d1/d1a/f23 x:0 0 0 2024-04-23T02:13:51.913 INFO:tasks.workunit.client.1.smithi192.stdout:6/205: write d3/f19 [1322822,76625] 0 2024-04-23T02:13:51.914 INFO:tasks.workunit.client.0.smithi071.stdout:5/115: unlink d3/da/ce 0 2024-04-23T02:13:51.914 INFO:tasks.workunit.client.1.smithi192.stdout:9/155: unlink d0/l20 0 2024-04-23T02:13:51.916 INFO:tasks.workunit.client.0.smithi071.stdout:9/102: creat d1/d1a/f24 x:0 0 0 2024-04-23T02:13:51.917 INFO:tasks.workunit.client.1.smithi192.stdout:9/156: write d0/d9/f25 [72004,102850] 0 2024-04-23T02:13:51.917 INFO:tasks.workunit.client.0.smithi071.stdout:5/116: dread - d3/d1d/f20 zero size 2024-04-23T02:13:51.919 INFO:tasks.workunit.client.0.smithi071.stdout:9/103: unlink d1/l11 0 2024-04-23T02:13:51.919 INFO:tasks.workunit.client.1.smithi192.stdout:9/157: write d0/d9/fb [240514,111232] 0 2024-04-23T02:13:51.920 INFO:tasks.workunit.client.1.smithi192.stdout:9/158: chown d0/l24 30 1 2024-04-23T02:13:51.921 INFO:tasks.workunit.client.1.smithi192.stdout:6/206: rename d3/d11/c16 to d3/d15/c3c 0 2024-04-23T02:13:51.922 INFO:tasks.workunit.client.1.smithi192.stdout:6/207: read - d3/f34 zero size 2024-04-23T02:13:51.922 INFO:tasks.workunit.client.0.smithi071.stdout:5/117: mkdir d3/d1d/d2c 0 2024-04-23T02:13:51.922 INFO:tasks.workunit.client.0.smithi071.stdout:1/61: dwrite db/fe [0,4194304] 0 2024-04-23T02:13:51.926 INFO:tasks.workunit.client.1.smithi192.stdout:9/159: creat d0/d9/f3a x:0 0 0 2024-04-23T02:13:51.926 INFO:tasks.workunit.client.1.smithi192.stdout:9/160: dread - d0/d9/f32 zero size 2024-04-23T02:13:51.927 INFO:tasks.workunit.client.0.smithi071.stdout:5/118: stat d3/da/f13 0 2024-04-23T02:13:51.927 INFO:tasks.workunit.client.0.smithi071.stdout:1/62: chown fa 502275013 1 2024-04-23T02:13:51.927 INFO:tasks.workunit.client.0.smithi071.stdout:1/63: truncate f7 4575669 0 2024-04-23T02:13:51.927 INFO:tasks.workunit.client.0.smithi071.stdout:9/104: dread d1/f17 [0,4194304] 0 2024-04-23T02:13:51.928 INFO:tasks.workunit.client.1.smithi192.stdout:6/208: dread d3/d11/f13 [4194304,4194304] 0 2024-04-23T02:13:51.930 INFO:tasks.workunit.client.0.smithi071.stdout:9/105: creat d1/d1a/f25 x:0 0 0 2024-04-23T02:13:51.933 INFO:tasks.workunit.client.0.smithi071.stdout:1/64: dwrite f2 [0,4194304] 0 2024-04-23T02:13:51.933 INFO:tasks.workunit.client.0.smithi071.stdout:9/106: symlink d1/d1a/l26 0 2024-04-23T02:13:51.943 INFO:tasks.workunit.client.0.smithi071.stdout:9/107: dwrite d1/d1a/f23 [0,4194304] 0 2024-04-23T02:13:51.947 INFO:tasks.workunit.client.1.smithi192.stdout:7/198: sync 2024-04-23T02:13:51.947 INFO:tasks.workunit.client.1.smithi192.stdout:1/56: sync 2024-04-23T02:13:51.948 INFO:tasks.workunit.client.1.smithi192.stdout:7/199: truncate d5/f2e 592183 0 2024-04-23T02:13:51.954 INFO:tasks.workunit.client.1.smithi192.stdout:9/161: rename d0/d3/l30 to d0/d3/l3b 0 2024-04-23T02:13:51.955 INFO:tasks.workunit.client.1.smithi192.stdout:1/57: creat dc/ff x:0 0 0 2024-04-23T02:13:51.957 INFO:tasks.workunit.client.1.smithi192.stdout:7/200: unlink d5/d17/f2b 0 2024-04-23T02:13:51.957 INFO:tasks.workunit.client.1.smithi192.stdout:6/209: write d3/d11/f13 [9170042,81103] 0 2024-04-23T02:13:51.964 INFO:tasks.workunit.client.1.smithi192.stdout:7/201: symlink d5/d17/d1c/d30/l39 0 2024-04-23T02:13:51.964 INFO:tasks.workunit.client.0.smithi071.stdout:5/119: rename d3/da/fd to d3/d1d/d2c/f2d 0 2024-04-23T02:13:51.967 INFO:tasks.workunit.client.1.smithi192.stdout:9/162: dwrite d0/d9/ff [0,4194304] 0 2024-04-23T02:13:51.967 INFO:tasks.workunit.client.1.smithi192.stdout:6/210: rename d3/d15/d1b/d21/c39 to d3/d15/c3d 0 2024-04-23T02:13:51.967 INFO:tasks.workunit.client.1.smithi192.stdout:7/202: stat d5/d17/d2f 0 2024-04-23T02:13:51.969 INFO:tasks.workunit.client.1.smithi192.stdout:9/163: dread - d0/d9/f3a zero size 2024-04-23T02:13:51.970 INFO:tasks.workunit.client.0.smithi071.stdout:5/120: write d3/d1d/f20 [799668,89853] 0 2024-04-23T02:13:51.970 INFO:tasks.workunit.client.0.smithi071.stdout:9/108: rmdir d1 39 2024-04-23T02:13:51.972 INFO:tasks.workunit.client.1.smithi192.stdout:1/58: dwrite f4 [0,4194304] 0 2024-04-23T02:13:51.973 INFO:tasks.workunit.client.0.smithi071.stdout:5/121: creat d3/da/d1a/d1c/f2e x:0 0 0 2024-04-23T02:13:51.974 INFO:tasks.workunit.client.1.smithi192.stdout:6/211: unlink d3/d15/d1b/d21/c26 0 2024-04-23T02:13:51.974 INFO:tasks.workunit.client.0.smithi071.stdout:9/109: truncate d1/d1a/f22 979739 0 2024-04-23T02:13:51.975 INFO:tasks.workunit.client.1.smithi192.stdout:9/164: mkdir d0/d1e/d3c 0 2024-04-23T02:13:51.975 INFO:tasks.workunit.client.0.smithi071.stdout:9/110: fdatasync d1/f9 0 2024-04-23T02:13:51.979 INFO:tasks.workunit.client.0.smithi071.stdout:5/122: creat d3/df/f2f x:0 0 0 2024-04-23T02:13:51.982 INFO:tasks.workunit.client.1.smithi192.stdout:9/165: dread d0/d3/f28 [0,4194304] 0 2024-04-23T02:13:51.982 INFO:tasks.workunit.client.0.smithi071.stdout:5/123: read d3/f12 [3067026,71760] 0 2024-04-23T02:13:51.984 INFO:tasks.workunit.client.0.smithi071.stdout:5/124: dread - d3/f29 zero size 2024-04-23T02:13:51.985 INFO:tasks.workunit.client.0.smithi071.stdout:9/111: dwrite d1/f4 [0,4194304] 0 2024-04-23T02:13:51.986 INFO:tasks.workunit.client.1.smithi192.stdout:7/203: write d5/d17/d2f/f20 [4936392,3945] 0 2024-04-23T02:13:51.987 INFO:tasks.workunit.client.1.smithi192.stdout:7/204: fdatasync d5/d17/d1c/d26/f34 0 2024-04-23T02:13:51.989 INFO:tasks.workunit.client.0.smithi071.stdout:9/112: rename d1/d1a/l26 to d1/d1a/l27 0 2024-04-23T02:13:51.989 INFO:tasks.workunit.client.1.smithi192.stdout:9/166: link d0/d9/f10 d0/d3/f3d 0 2024-04-23T02:13:51.991 INFO:tasks.workunit.client.0.smithi071.stdout:9/113: unlink d1/l1b 0 2024-04-23T02:13:51.991 INFO:tasks.workunit.client.0.smithi071.stdout:5/125: link d3/da/c27 d3/da/d1a/d1c/c30 0 2024-04-23T02:13:51.991 INFO:tasks.workunit.client.0.smithi071.stdout:5/126: chown d3/da 42579 1 2024-04-23T02:13:51.993 INFO:tasks.workunit.client.1.smithi192.stdout:9/167: dread d0/d3/f28 [0,4194304] 0 2024-04-23T02:13:51.993 INFO:tasks.workunit.client.1.smithi192.stdout:6/212: dread d3/f19 [0,4194304] 0 2024-04-23T02:13:51.993 INFO:tasks.workunit.client.1.smithi192.stdout:7/205: getdents d5/d17 0 2024-04-23T02:13:51.994 INFO:tasks.workunit.client.0.smithi071.stdout:5/127: creat d3/da/d1a/d1c/f31 x:0 0 0 2024-04-23T02:13:51.995 INFO:tasks.workunit.client.0.smithi071.stdout:5/128: fsync d3/f29 0 2024-04-23T02:13:51.995 INFO:tasks.workunit.client.1.smithi192.stdout:7/206: fdatasync d5/d17/d1c/f24 0 2024-04-23T02:13:51.995 INFO:tasks.workunit.client.1.smithi192.stdout:9/168: dread d0/d3/f8 [0,4194304] 0 2024-04-23T02:13:51.996 INFO:tasks.workunit.client.1.smithi192.stdout:1/59: truncate f9 494030 0 2024-04-23T02:13:51.997 INFO:tasks.workunit.client.0.smithi071.stdout:5/129: dread - d3/d1d/f2a zero size 2024-04-23T02:13:51.997 INFO:tasks.workunit.client.0.smithi071.stdout:9/114: link d1/d1a/f22 d1/f28 0 2024-04-23T02:13:51.998 INFO:tasks.workunit.client.0.smithi071.stdout:5/130: symlink d3/d1d/d2c/l32 0 2024-04-23T02:13:51.998 INFO:tasks.workunit.client.1.smithi192.stdout:9/169: rmdir d0/d9 39 2024-04-23T02:13:51.998 INFO:tasks.workunit.client.1.smithi192.stdout:6/213: fdatasync d3/d11/f1d 0 2024-04-23T02:13:51.999 INFO:tasks.workunit.client.1.smithi192.stdout:1/60: creat dc/f10 x:0 0 0 2024-04-23T02:13:52.000 INFO:tasks.workunit.client.1.smithi192.stdout:1/61: stat dc/f10 0 2024-04-23T02:13:52.001 INFO:tasks.workunit.client.1.smithi192.stdout:7/207: rename d5/d17/d2f/c29 to d5/d17/d1c/c3a 0 2024-04-23T02:13:52.003 INFO:tasks.workunit.client.0.smithi071.stdout:9/115: dwrite d1/fe [0,4194304] 0 2024-04-23T02:13:52.011 INFO:tasks.workunit.client.1.smithi192.stdout:7/208: link d5/c1b d5/d17/c3b 0 2024-04-23T02:13:52.011 INFO:tasks.workunit.client.1.smithi192.stdout:6/214: getdents d3 0 2024-04-23T02:13:52.013 INFO:tasks.workunit.client.0.smithi071.stdout:9/116: dwrite d1/d1a/f25 [0,4194304] 0 2024-04-23T02:13:52.013 INFO:tasks.workunit.client.1.smithi192.stdout:6/215: symlink d3/d28/l3e 0 2024-04-23T02:13:52.016 INFO:tasks.workunit.client.1.smithi192.stdout:6/216: truncate d3/d11/f29 1608786 0 2024-04-23T02:13:52.020 INFO:tasks.workunit.client.0.smithi071.stdout:5/131: dwrite d3/df/d10/f14 [0,4194304] 0 2024-04-23T02:13:52.021 INFO:tasks.workunit.client.1.smithi192.stdout:9/170: dwrite d0/f2c [0,4194304] 0 2024-04-23T02:13:52.022 INFO:tasks.workunit.client.0.smithi071.stdout:9/117: truncate d1/f10 620495 0 2024-04-23T02:13:52.023 INFO:tasks.workunit.client.0.smithi071.stdout:9/118: write d1/d1a/f23 [1848050,43917] 0 2024-04-23T02:13:52.028 INFO:tasks.workunit.client.1.smithi192.stdout:1/62: dread f0 [0,4194304] 0 2024-04-23T02:13:52.028 INFO:tasks.workunit.client.1.smithi192.stdout:6/217: link d3/d15/d1b/c24 d3/d15/d1b/d21/d3a/c3f 0 2024-04-23T02:13:52.033 INFO:tasks.workunit.client.0.smithi071.stdout:9/119: dwrite d1/fa [0,4194304] 0 2024-04-23T02:13:52.034 INFO:tasks.workunit.client.1.smithi192.stdout:9/171: chown d0/d3/l17 13325431 1 2024-04-23T02:13:52.035 INFO:tasks.workunit.client.0.smithi071.stdout:9/120: fsync d1/d1a/f1c 0 2024-04-23T02:13:52.037 INFO:tasks.workunit.client.0.smithi071.stdout:5/132: dwrite d3/d1d/d2c/f2d [0,4194304] 0 2024-04-23T02:13:52.038 INFO:tasks.workunit.client.1.smithi192.stdout:6/218: link d3/d15/d1b/c24 d3/d11/c40 0 2024-04-23T02:13:52.039 INFO:tasks.workunit.client.0.smithi071.stdout:5/133: write d3/da/d1a/d1c/f2e [774984,121231] 0 2024-04-23T02:13:52.040 INFO:tasks.workunit.client.0.smithi071.stdout:9/121: link d1/c1d d1/c29 0 2024-04-23T02:13:52.040 INFO:tasks.workunit.client.0.smithi071.stdout:5/134: dread - d3/df/f17 zero size 2024-04-23T02:13:52.042 INFO:tasks.workunit.client.1.smithi192.stdout:1/63: dwrite f0 [4194304,4194304] 0 2024-04-23T02:13:52.043 INFO:tasks.workunit.client.0.smithi071.stdout:9/122: symlink d1/l2a 0 2024-04-23T02:13:52.043 INFO:tasks.workunit.client.0.smithi071.stdout:9/123: chown d1/d1a/f22 5082837 1 2024-04-23T02:13:52.044 INFO:tasks.workunit.client.0.smithi071.stdout:5/135: rmdir d3/d1d 39 2024-04-23T02:13:52.048 INFO:tasks.workunit.client.0.smithi071.stdout:9/124: dread d1/fc [0,4194304] 0 2024-04-23T02:13:52.051 INFO:tasks.workunit.client.0.smithi071.stdout:9/125: read d1/f9 [796640,49533] 0 2024-04-23T02:13:52.052 INFO:tasks.workunit.client.1.smithi192.stdout:1/64: creat dc/f11 x:0 0 0 2024-04-23T02:13:52.052 INFO:tasks.workunit.client.1.smithi192.stdout:6/219: link d3/c7 d3/d18/c41 0 2024-04-23T02:13:52.053 INFO:tasks.workunit.client.1.smithi192.stdout:9/172: dread d0/d9/f26 [0,4194304] 0 2024-04-23T02:13:52.053 INFO:tasks.workunit.client.1.smithi192.stdout:1/65: write f0 [8694649,76150] 0 2024-04-23T02:13:52.058 INFO:tasks.workunit.client.0.smithi071.stdout:9/126: getdents d1 0 2024-04-23T02:13:52.060 INFO:tasks.workunit.client.1.smithi192.stdout:9/173: dread d0/d9/f25 [0,4194304] 0 2024-04-23T02:13:52.061 INFO:tasks.workunit.client.1.smithi192.stdout:9/174: chown d0/f34 42276 1 2024-04-23T02:13:52.061 INFO:tasks.workunit.client.0.smithi071.stdout:9/127: write d1/fc [3357558,111728] 0 2024-04-23T02:13:52.061 INFO:tasks.workunit.client.1.smithi192.stdout:6/220: dwrite d3/d15/d1b/f2a [0,4194304] 0 2024-04-23T02:13:52.062 INFO:tasks.workunit.client.0.smithi071.stdout:5/136: dwrite d3/f9 [0,4194304] 0 2024-04-23T02:13:52.063 INFO:tasks.workunit.client.1.smithi192.stdout:6/221: truncate d3/f2b 4367867 0 2024-04-23T02:13:52.064 INFO:tasks.workunit.client.0.smithi071.stdout:5/137: chown d3/da/d1a/f26 5453 1 2024-04-23T02:13:52.064 INFO:tasks.workunit.client.0.smithi071.stdout:9/128: rmdir d1 39 2024-04-23T02:13:52.065 INFO:tasks.workunit.client.1.smithi192.stdout:9/175: truncate d0/d3/f28 2091997 0 2024-04-23T02:13:52.067 INFO:tasks.workunit.client.1.smithi192.stdout:6/222: chown d3/d15/d1b/d21/d3a/c3f 1164 1 2024-04-23T02:13:52.068 INFO:tasks.workunit.client.0.smithi071.stdout:5/138: link d3/f1f d3/da/f33 0 2024-04-23T02:13:52.069 INFO:tasks.workunit.client.1.smithi192.stdout:9/176: creat d0/d3/f3e x:0 0 0 2024-04-23T02:13:52.070 INFO:tasks.workunit.client.1.smithi192.stdout:6/223: write d3/f19 [2045840,64846] 0 2024-04-23T02:13:52.072 INFO:tasks.workunit.client.0.smithi071.stdout:5/139: mkdir d3/d34 0 2024-04-23T02:13:52.073 INFO:tasks.workunit.client.1.smithi192.stdout:9/177: mknod d0/c3f 0 2024-04-23T02:13:52.074 INFO:tasks.workunit.client.0.smithi071.stdout:5/140: fdatasync d3/da/f13 0 2024-04-23T02:13:52.075 INFO:tasks.workunit.client.1.smithi192.stdout:9/178: symlink d0/d3/l40 0 2024-04-23T02:13:52.075 INFO:tasks.workunit.client.0.smithi071.stdout:9/129: dwrite d1/fb [4194304,4194304] 0 2024-04-23T02:13:52.076 INFO:tasks.workunit.client.0.smithi071.stdout:5/141: write d3/f29 [86968,119755] 0 2024-04-23T02:13:52.078 INFO:tasks.workunit.client.1.smithi192.stdout:9/179: creat d0/d3/f41 x:0 0 0 2024-04-23T02:13:52.080 INFO:tasks.workunit.client.1.smithi192.stdout:9/180: rmdir d0/d1e 39 2024-04-23T02:13:52.080 INFO:tasks.workunit.client.1.smithi192.stdout:6/224: dwrite d3/d11/f29 [0,4194304] 0 2024-04-23T02:13:52.083 INFO:tasks.workunit.client.0.smithi071.stdout:9/130: rename d1/d1a/f1c to d1/f2b 0 2024-04-23T02:13:52.083 INFO:tasks.workunit.client.0.smithi071.stdout:5/142: mknod d3/d1d/d2c/c35 0 2024-04-23T02:13:52.084 INFO:tasks.workunit.client.0.smithi071.stdout:5/143: symlink d3/d1d/l36 0 2024-04-23T02:13:52.086 INFO:tasks.workunit.client.1.smithi192.stdout:9/181: creat d0/d9/f42 x:0 0 0 2024-04-23T02:13:52.090 INFO:tasks.workunit.client.1.smithi192.stdout:9/182: chown d0/d3/f28 659331 1 2024-04-23T02:13:52.090 INFO:tasks.workunit.client.1.smithi192.stdout:9/183: stat d0/f33 0 2024-04-23T02:13:52.090 INFO:tasks.workunit.client.1.smithi192.stdout:6/225: dread d3/d28/f2e [0,4194304] 0 2024-04-23T02:13:52.091 INFO:tasks.workunit.client.1.smithi192.stdout:9/184: fdatasync d0/f16 0 2024-04-23T02:13:52.092 INFO:tasks.workunit.client.1.smithi192.stdout:9/185: fsync d0/d3/f2e 0 2024-04-23T02:13:52.093 INFO:tasks.workunit.client.1.smithi192.stdout:6/226: symlink d3/d11/l42 0 2024-04-23T02:13:52.093 INFO:tasks.workunit.client.1.smithi192.stdout:9/186: read - d0/d3/f39 zero size 2024-04-23T02:13:52.096 INFO:tasks.workunit.client.1.smithi192.stdout:6/227: unlink d3/d15/c23 0 2024-04-23T02:13:52.098 INFO:tasks.workunit.client.1.smithi192.stdout:9/187: rename d0/d9/f10 to d0/d9/f43 0 2024-04-23T02:13:52.099 INFO:tasks.workunit.client.1.smithi192.stdout:9/188: chown d0/c3f 193289399 1 2024-04-23T02:13:52.100 INFO:tasks.workunit.client.0.smithi071.stdout:9/131: fdatasync d1/fb 0 2024-04-23T02:13:52.101 INFO:tasks.workunit.client.1.smithi192.stdout:9/189: mkdir d0/d9/d44 0 2024-04-23T02:13:52.101 INFO:tasks.workunit.client.1.smithi192.stdout:6/228: dread d3/f19 [0,4194304] 0 2024-04-23T02:13:52.102 INFO:tasks.workunit.client.0.smithi071.stdout:9/132: truncate d1/f16 1159595 0 2024-04-23T02:13:52.103 INFO:tasks.workunit.client.1.smithi192.stdout:9/190: chown d0/d1e/c2d 2504 1 2024-04-23T02:13:52.103 INFO:tasks.workunit.client.1.smithi192.stdout:6/229: creat d3/d11/f43 x:0 0 0 2024-04-23T02:13:52.104 INFO:tasks.workunit.client.0.smithi071.stdout:5/144: write d3/f22 [1042315,84670] 0 2024-04-23T02:13:52.106 INFO:tasks.workunit.client.0.smithi071.stdout:5/145: mknod d3/d1d/d2c/c37 0 2024-04-23T02:13:52.106 INFO:tasks.workunit.client.0.smithi071.stdout:5/146: chown d3/da/f33 98592 1 2024-04-23T02:13:52.107 INFO:tasks.workunit.client.1.smithi192.stdout:6/230: mkdir d3/d11/d44 0 2024-04-23T02:13:52.108 INFO:tasks.workunit.client.1.smithi192.stdout:9/191: creat d0/d9/d44/f45 x:0 0 0 2024-04-23T02:13:52.108 INFO:tasks.workunit.client.0.smithi071.stdout:5/147: creat d3/da/f38 x:0 0 0 2024-04-23T02:13:52.109 INFO:tasks.workunit.client.0.smithi071.stdout:5/148: write d3/da/d1a/f2b [1874792,75196] 0 2024-04-23T02:13:52.110 INFO:tasks.workunit.client.0.smithi071.stdout:5/149: read - d3/df/f17 zero size 2024-04-23T02:13:52.110 INFO:tasks.workunit.client.1.smithi192.stdout:9/192: fsync d0/f2c 0 2024-04-23T02:13:52.111 INFO:tasks.workunit.client.0.smithi071.stdout:5/150: stat d3/da/d1a/f26 0 2024-04-23T02:13:52.112 INFO:tasks.workunit.client.0.smithi071.stdout:5/151: unlink d3/df/f2f 0 2024-04-23T02:13:52.114 INFO:tasks.workunit.client.1.smithi192.stdout:9/193: creat d0/d3/f46 x:0 0 0 2024-04-23T02:13:52.114 INFO:tasks.workunit.client.1.smithi192.stdout:6/231: dwrite d3/d15/d1b/d21/f32 [0,4194304] 0 2024-04-23T02:13:52.115 INFO:tasks.workunit.client.1.smithi192.stdout:9/194: stat d0/l15 0 2024-04-23T02:13:52.116 INFO:tasks.workunit.client.0.smithi071.stdout:5/152: dread d3/f9 [0,4194304] 0 2024-04-23T02:13:52.118 INFO:tasks.workunit.client.1.smithi192.stdout:9/195: creat d0/d9/d44/f47 x:0 0 0 2024-04-23T02:13:52.124 INFO:tasks.workunit.client.1.smithi192.stdout:6/232: dwrite d3/d15/d1b/d21/f36 [0,4194304] 0 2024-04-23T02:13:52.125 INFO:tasks.workunit.client.0.smithi071.stdout:5/153: dwrite d3/f12 [0,4194304] 0 2024-04-23T02:13:52.127 INFO:tasks.workunit.client.1.smithi192.stdout:6/233: dread d3/d18/f2d [0,4194304] 0 2024-04-23T02:13:52.127 INFO:tasks.workunit.client.1.smithi192.stdout:9/196: rename d0/d9/f14 to d0/f48 0 2024-04-23T02:13:52.128 INFO:tasks.workunit.client.0.smithi071.stdout:6/65: sync 2024-04-23T02:13:52.128 INFO:tasks.workunit.client.0.smithi071.stdout:5/154: creat d3/da/d1a/d1c/f39 x:0 0 0 2024-04-23T02:13:52.132 INFO:tasks.workunit.client.0.smithi071.stdout:6/66: getdents d6 0 2024-04-23T02:13:52.133 INFO:tasks.workunit.client.1.smithi192.stdout:9/197: dread d0/f48 [0,4194304] 0 2024-04-23T02:13:52.134 INFO:tasks.workunit.client.0.smithi071.stdout:6/67: creat d6/f12 x:0 0 0 2024-04-23T02:13:52.136 INFO:tasks.workunit.client.0.smithi071.stdout:6/68: mknod d6/c13 0 2024-04-23T02:13:52.140 INFO:tasks.workunit.client.1.smithi192.stdout:9/198: rename d0/f48 to d0/d9/d44/f49 0 2024-04-23T02:13:52.144 INFO:tasks.workunit.client.1.smithi192.stdout:6/234: link d3/le d3/d15/d1b/d21/d3a/l45 0 2024-04-23T02:13:52.144 INFO:tasks.workunit.client.1.smithi192.stdout:6/235: stat d3/d18/l1a 0 2024-04-23T02:13:52.148 INFO:tasks.workunit.client.0.smithi071.stdout:5/155: getdents d3/da/d1a/d1c 0 2024-04-23T02:13:52.150 INFO:tasks.workunit.client.0.smithi071.stdout:6/69: rmdir d6 39 2024-04-23T02:13:52.150 INFO:tasks.workunit.client.1.smithi192.stdout:9/199: getdents d0 0 2024-04-23T02:13:52.151 INFO:tasks.workunit.client.0.smithi071.stdout:5/156: symlink d3/da/d1a/d1c/l3a 0 2024-04-23T02:13:52.151 INFO:tasks.workunit.client.1.smithi192.stdout:9/200: read d0/f16 [215874,88680] 0 2024-04-23T02:13:52.152 INFO:tasks.workunit.client.1.smithi192.stdout:6/236: dwrite d3/f2b [4194304,4194304] 0 2024-04-23T02:13:52.153 INFO:tasks.workunit.client.0.smithi071.stdout:6/70: creat d6/f14 x:0 0 0 2024-04-23T02:13:52.154 INFO:tasks.workunit.client.1.smithi192.stdout:9/201: mknod d0/d9/c4a 0 2024-04-23T02:13:52.155 INFO:tasks.workunit.client.0.smithi071.stdout:6/71: fsync d6/f11 0 2024-04-23T02:13:52.155 INFO:tasks.workunit.client.0.smithi071.stdout:5/157: truncate d3/da/d1a/d1c/f1e 452030 0 2024-04-23T02:13:52.155 INFO:tasks.workunit.client.1.smithi192.stdout:6/237: unlink d3/d15/d1b/d21/f36 0 2024-04-23T02:13:52.156 INFO:tasks.workunit.client.0.smithi071.stdout:5/158: creat d3/d1d/f3b x:0 0 0 2024-04-23T02:13:52.157 INFO:tasks.workunit.client.1.smithi192.stdout:9/202: symlink d0/d1e/l4b 0 2024-04-23T02:13:52.158 INFO:tasks.workunit.client.0.smithi071.stdout:5/159: mknod d3/d34/c3c 0 2024-04-23T02:13:52.158 INFO:tasks.workunit.client.1.smithi192.stdout:9/203: dread - d0/d3/f3e zero size 2024-04-23T02:13:52.158 INFO:tasks.workunit.client.0.smithi071.stdout:6/72: dread d6/f9 [0,4194304] 0 2024-04-23T02:13:52.159 INFO:tasks.workunit.client.1.smithi192.stdout:3/127: sync 2024-04-23T02:13:52.159 INFO:tasks.workunit.client.1.smithi192.stdout:0/244: sync 2024-04-23T02:13:52.159 INFO:tasks.workunit.client.1.smithi192.stdout:5/192: sync 2024-04-23T02:13:52.161 INFO:tasks.workunit.client.1.smithi192.stdout:9/204: write d0/d3/f41 [323028,118183] 0 2024-04-23T02:13:52.161 INFO:tasks.workunit.client.0.smithi071.stdout:6/73: symlink d6/l15 0 2024-04-23T02:13:52.163 INFO:tasks.workunit.client.1.smithi192.stdout:0/245: symlink d6/df/d13/l57 0 2024-04-23T02:13:52.163 INFO:tasks.workunit.client.0.smithi071.stdout:5/160: link d3/lc d3/l3d 0 2024-04-23T02:13:52.166 INFO:tasks.workunit.client.1.smithi192.stdout:9/205: creat d0/d1e/f4c x:0 0 0 2024-04-23T02:13:52.166 INFO:tasks.workunit.client.0.smithi071.stdout:5/161: symlink d3/da/d1a/d1c/l3e 0 2024-04-23T02:13:52.166 INFO:tasks.workunit.client.1.smithi192.stdout:5/193: creat d2/db/d16/d1b/d3d/f3f x:0 0 0 2024-04-23T02:13:52.167 INFO:tasks.workunit.client.1.smithi192.stdout:6/238: dwrite d3/d11/f33 [4194304,4194304] 0 2024-04-23T02:13:52.168 INFO:tasks.workunit.client.0.smithi071.stdout:6/74: dwrite d6/fb [4194304,4194304] 0 2024-04-23T02:13:52.168 INFO:tasks.workunit.client.1.smithi192.stdout:9/206: chown d0/d3/f8 4030042 1 2024-04-23T02:13:52.170 INFO:tasks.workunit.client.0.smithi071.stdout:6/75: creat d6/f16 x:0 0 0 2024-04-23T02:13:52.171 INFO:tasks.workunit.client.1.smithi192.stdout:3/128: dwrite d1/f5 [4194304,4194304] 0 2024-04-23T02:13:52.173 INFO:tasks.workunit.client.0.smithi071.stdout:5/162: rename d3/d1d/d2c/l32 to d3/da/d1a/l3f 0 2024-04-23T02:13:52.174 INFO:tasks.workunit.client.1.smithi192.stdout:0/246: rename d6/df/f42 to d6/df/d13/d25/d3a/f58 0 2024-04-23T02:13:52.174 INFO:tasks.workunit.client.1.smithi192.stdout:5/194: mkdir d2/db/d16/d1b/d3d/d40 0 2024-04-23T02:13:52.175 INFO:tasks.workunit.client.0.smithi071.stdout:6/76: mkdir d6/d17 0 2024-04-23T02:13:52.176 INFO:tasks.workunit.client.1.smithi192.stdout:0/247: fdatasync d6/df/d13/d25/d26/f27 0 2024-04-23T02:13:52.177 INFO:tasks.workunit.client.0.smithi071.stdout:6/77: creat d6/f18 x:0 0 0 2024-04-23T02:13:52.177 INFO:tasks.workunit.client.1.smithi192.stdout:0/248: read f5 [559154,45562] 0 2024-04-23T02:13:52.179 INFO:tasks.workunit.client.1.smithi192.stdout:0/249: write d6/f1b [4328,25972] 0 2024-04-23T02:13:52.180 INFO:tasks.workunit.client.1.smithi192.stdout:9/207: mkdir d0/d9/d4d 0 2024-04-23T02:13:52.183 INFO:tasks.workunit.client.1.smithi192.stdout:5/195: rename d2/d6/d2d/d30 to d2/d6/d2d/d41 0 2024-04-23T02:13:52.185 INFO:tasks.workunit.client.1.smithi192.stdout:0/250: symlink d6/d44/d45/l59 0 2024-04-23T02:13:52.186 INFO:tasks.workunit.client.1.smithi192.stdout:3/129: mkdir d1/d3/d6/d7/d1b/d22 0 2024-04-23T02:13:52.187 INFO:tasks.workunit.client.1.smithi192.stdout:0/251: write d6/d4b/f52 [451969,76527] 0 2024-04-23T02:13:52.188 INFO:tasks.workunit.client.1.smithi192.stdout:0/252: read - d6/df/d35/d49/f54 zero size 2024-04-23T02:13:52.188 INFO:tasks.workunit.client.1.smithi192.stdout:9/208: creat d0/f4e x:0 0 0 2024-04-23T02:13:52.189 INFO:tasks.workunit.client.1.smithi192.stdout:0/253: chown d6/df/d13/d25 55659159 1 2024-04-23T02:13:52.189 INFO:tasks.workunit.client.0.smithi071.stdout:5/163: write f2 [976170,105634] 0 2024-04-23T02:13:52.193 INFO:tasks.workunit.client.1.smithi192.stdout:5/196: rename d2/l1c to d2/d6/d14/l42 0 2024-04-23T02:13:52.194 INFO:tasks.workunit.client.1.smithi192.stdout:3/130: truncate d1/d3/d6/f1c 768105 0 2024-04-23T02:13:52.195 INFO:tasks.workunit.client.1.smithi192.stdout:5/197: read d2/db/f11 [1894008,10794] 0 2024-04-23T02:13:52.196 INFO:tasks.workunit.client.1.smithi192.stdout:0/254: rename d6/d44/d45/l59 to d6/df/d13/l5a 0 2024-04-23T02:13:52.198 INFO:tasks.workunit.client.1.smithi192.stdout:5/198: mkdir d2/d6/d2d/d41/d43 0 2024-04-23T02:13:52.201 INFO:tasks.workunit.client.1.smithi192.stdout:0/255: mkdir d6/df/d13/d25/d3a/d5b 0 2024-04-23T02:13:52.202 INFO:tasks.workunit.client.1.smithi192.stdout:5/199: dread d2/d6/f12 [0,4194304] 0 2024-04-23T02:13:52.205 INFO:tasks.workunit.client.1.smithi192.stdout:9/209: rename d0/d3 to d0/d9/d44/d4f 0 2024-04-23T02:13:52.206 INFO:tasks.workunit.client.1.smithi192.stdout:9/210: fdatasync d0/d9/f42 0 2024-04-23T02:13:52.206 INFO:tasks.workunit.client.1.smithi192.stdout:0/256: mknod d6/df/d35/d49/c5c 0 2024-04-23T02:13:52.211 INFO:tasks.workunit.client.1.smithi192.stdout:0/257: symlink d6/df/d13/d25/d26/l5d 0 2024-04-23T02:13:52.212 INFO:tasks.workunit.client.1.smithi192.stdout:9/211: dwrite d0/d9/d44/d4f/f39 [0,4194304] 0 2024-04-23T02:13:52.215 INFO:tasks.workunit.client.1.smithi192.stdout:0/258: mknod d6/df/c5e 0 2024-04-23T02:13:52.215 INFO:tasks.workunit.client.1.smithi192.stdout:9/212: chown d0/d9/d44/d4f/f3d 1726439 1 2024-04-23T02:13:52.218 INFO:tasks.workunit.client.1.smithi192.stdout:9/213: creat d0/d9/f50 x:0 0 0 2024-04-23T02:13:52.220 INFO:tasks.workunit.client.1.smithi192.stdout:9/214: mkdir d0/d9/d44/d51 0 2024-04-23T02:13:52.220 INFO:tasks.workunit.client.1.smithi192.stdout:6/239: truncate d3/f12 1679535 0 2024-04-23T02:13:52.222 INFO:tasks.workunit.client.1.smithi192.stdout:6/240: write d3/d11/f13 [5199842,105608] 0 2024-04-23T02:13:52.222 INFO:tasks.workunit.client.1.smithi192.stdout:9/215: truncate d0/d9/d44/d4f/f3e 91387 0 2024-04-23T02:13:52.224 INFO:tasks.workunit.client.1.smithi192.stdout:6/241: chown d3/le 5459 1 2024-04-23T02:13:52.226 INFO:tasks.workunit.client.1.smithi192.stdout:6/242: write d3/d15/d1b/d21/f32 [3598867,30007] 0 2024-04-23T02:13:52.230 INFO:tasks.workunit.client.1.smithi192.stdout:9/216: link d0/d9/ff d0/d9/f52 0 2024-04-23T02:13:52.231 INFO:tasks.workunit.client.1.smithi192.stdout:9/217: stat d0/d9/d44/d4f/f1c 0 2024-04-23T02:13:52.234 INFO:tasks.workunit.client.1.smithi192.stdout:5/200: truncate d2/db/d16/d1b/f1f 2804963 0 2024-04-23T02:13:52.235 INFO:tasks.workunit.client.1.smithi192.stdout:9/218: fdatasync d0/d9/f37 0 2024-04-23T02:13:52.237 INFO:tasks.workunit.client.1.smithi192.stdout:9/219: write d0/d9/f50 [685413,84514] 0 2024-04-23T02:13:52.237 INFO:tasks.workunit.client.1.smithi192.stdout:5/201: write d2/d6/f12 [3274660,53403] 0 2024-04-23T02:13:52.240 INFO:tasks.workunit.client.1.smithi192.stdout:0/259: write d6/f10 [880794,126398] 0 2024-04-23T02:13:52.240 INFO:tasks.workunit.client.1.smithi192.stdout:9/220: write d0/f33 [311144,1536] 0 2024-04-23T02:13:52.245 INFO:tasks.workunit.client.1.smithi192.stdout:9/221: unlink d0/d9/ff 0 2024-04-23T02:13:52.247 INFO:tasks.workunit.client.1.smithi192.stdout:9/222: read - d0/d9/f36 zero size 2024-04-23T02:13:52.252 INFO:tasks.workunit.client.1.smithi192.stdout:0/260: getdents d6/d44 0 2024-04-23T02:13:52.253 INFO:tasks.workunit.client.1.smithi192.stdout:9/223: mkdir d0/d9/d44/d53 0 2024-04-23T02:13:52.255 INFO:tasks.workunit.client.1.smithi192.stdout:3/131: dwrite d1/d3/d6/f1c [0,4194304] 0 2024-04-23T02:13:52.260 INFO:tasks.workunit.client.1.smithi192.stdout:6/243: dwrite f1 [0,4194304] 0 2024-04-23T02:13:52.261 INFO:tasks.workunit.client.1.smithi192.stdout:5/202: dread d2/f2a [0,4194304] 0 2024-04-23T02:13:52.262 INFO:tasks.workunit.client.1.smithi192.stdout:0/261: truncate d6/df/d47/f4c 990173 0 2024-04-23T02:13:52.267 INFO:tasks.workunit.client.1.smithi192.stdout:5/203: chown d2/db/lc 9 1 2024-04-23T02:13:52.267 INFO:tasks.workunit.client.1.smithi192.stdout:6/244: write d3/f2b [3739237,34018] 0 2024-04-23T02:13:52.267 INFO:tasks.workunit.client.1.smithi192.stdout:3/132: write d1/d3/f19 [447451,119005] 0 2024-04-23T02:13:52.273 INFO:tasks.workunit.client.1.smithi192.stdout:5/204: read d2/f27 [31142,115163] 0 2024-04-23T02:13:52.274 INFO:tasks.workunit.client.1.smithi192.stdout:3/133: dwrite d1/d3/d6/f10 [0,4194304] 0 2024-04-23T02:13:52.277 INFO:tasks.workunit.client.1.smithi192.stdout:9/224: rename d0/d9/d44/d4f/f46 to d0/f54 0 2024-04-23T02:13:52.283 INFO:tasks.workunit.client.1.smithi192.stdout:6/245: link d3/l37 d3/d15/d1b/d21/d3a/l46 0 2024-04-23T02:13:52.284 INFO:tasks.workunit.client.1.smithi192.stdout:0/262: link d6/df/d13/d25/d26/l34 d6/df/d47/d55/l5f 0 2024-04-23T02:13:52.285 INFO:tasks.workunit.client.1.smithi192.stdout:0/263: chown d6/df/f23 15101558 1 2024-04-23T02:13:52.286 INFO:tasks.workunit.client.1.smithi192.stdout:3/134: creat d1/d3/d17/d1a/f23 x:0 0 0 2024-04-23T02:13:52.289 INFO:tasks.workunit.client.1.smithi192.stdout:0/264: truncate d6/df/d13/f51 153545 0 2024-04-23T02:13:52.294 INFO:tasks.workunit.client.1.smithi192.stdout:6/246: dwrite d3/d11/f13 [0,4194304] 0 2024-04-23T02:13:52.294 INFO:tasks.workunit.client.1.smithi192.stdout:9/225: dread d0/d9/d44/d4f/f28 [0,4194304] 0 2024-04-23T02:13:52.295 INFO:tasks.workunit.client.1.smithi192.stdout:6/247: stat l2 0 2024-04-23T02:13:52.296 INFO:tasks.workunit.client.1.smithi192.stdout:5/205: fdatasync d2/f5 0 2024-04-23T02:13:52.299 INFO:tasks.workunit.client.1.smithi192.stdout:9/226: chown d0/d1e/d3c 0 1 2024-04-23T02:13:52.300 INFO:tasks.workunit.client.1.smithi192.stdout:3/135: dwrite d1/d3/d6/f10 [0,4194304] 0 2024-04-23T02:13:52.302 INFO:tasks.workunit.client.1.smithi192.stdout:0/265: dwrite d6/df/f23 [0,4194304] 0 2024-04-23T02:13:52.302 INFO:tasks.workunit.client.1.smithi192.stdout:6/248: read d3/d18/f2d [136412,124912] 0 2024-04-23T02:13:52.305 INFO:tasks.workunit.client.1.smithi192.stdout:5/206: creat d2/d32/f44 x:0 0 0 2024-04-23T02:13:52.311 INFO:tasks.workunit.client.1.smithi192.stdout:3/136: mkdir d1/d3/d6/d7/d1b/d24 0 2024-04-23T02:13:52.311 INFO:tasks.workunit.client.1.smithi192.stdout:0/266: fdatasync d6/d44/f46 0 2024-04-23T02:13:52.311 INFO:tasks.workunit.client.1.smithi192.stdout:9/227: mknod d0/d9/d44/d53/c55 0 2024-04-23T02:13:52.312 INFO:tasks.workunit.client.1.smithi192.stdout:5/207: symlink d2/d6/d2d/d41/l45 0 2024-04-23T02:13:52.312 INFO:tasks.workunit.client.0.smithi071.stdout:3/34: sync 2024-04-23T02:13:52.312 INFO:tasks.workunit.client.0.smithi071.stdout:8/112: sync 2024-04-23T02:13:52.314 INFO:tasks.workunit.client.0.smithi071.stdout:3/35: truncate d7/d8/f9 848715 0 2024-04-23T02:13:52.314 INFO:tasks.workunit.client.1.smithi192.stdout:3/137: chown d1/d3/d17/d1a/f23 9316310 1 2024-04-23T02:13:52.314 INFO:tasks.workunit.client.0.smithi071.stdout:8/113: symlink dc/l1f 0 2024-04-23T02:13:52.315 INFO:tasks.workunit.client.1.smithi192.stdout:0/267: dread d6/df/d13/f51 [0,4194304] 0 2024-04-23T02:13:52.316 INFO:tasks.workunit.client.0.smithi071.stdout:3/36: rename d7/d8 to d7/d8/da 22 2024-04-23T02:13:52.316 INFO:tasks.workunit.client.0.smithi071.stdout:8/114: rmdir dc/d15 39 2024-04-23T02:13:52.316 INFO:tasks.workunit.client.0.smithi071.stdout:3/37: fsync d7/d8/f9 0 2024-04-23T02:13:52.316 INFO:tasks.workunit.client.1.smithi192.stdout:3/138: symlink d1/l25 0 2024-04-23T02:13:52.316 INFO:tasks.workunit.client.0.smithi071.stdout:8/115: stat dc/f18 0 2024-04-23T02:13:52.318 INFO:tasks.workunit.client.0.smithi071.stdout:3/38: write f0 [2938678,99082] 0 2024-04-23T02:13:52.319 INFO:tasks.workunit.client.0.smithi071.stdout:3/39: readlink l6 0 2024-04-23T02:13:52.319 INFO:tasks.workunit.client.1.smithi192.stdout:6/249: dwrite d3/d11/f33 [4194304,4194304] 0 2024-04-23T02:13:52.320 INFO:tasks.workunit.client.0.smithi071.stdout:3/40: truncate d7/d8/f9 1672477 0 2024-04-23T02:13:52.321 INFO:tasks.workunit.client.1.smithi192.stdout:9/228: rename d0/d9/d44/d4f/f39 to d0/f56 0 2024-04-23T02:13:52.321 INFO:tasks.workunit.client.0.smithi071.stdout:3/41: read f0 [1108513,91028] 0 2024-04-23T02:13:52.325 INFO:tasks.workunit.client.0.smithi071.stdout:3/42: creat d7/fb x:0 0 0 2024-04-23T02:13:52.325 INFO:tasks.workunit.client.1.smithi192.stdout:9/229: dread - d0/d9/d44/f45 zero size 2024-04-23T02:13:52.326 INFO:tasks.workunit.client.0.smithi071.stdout:8/116: dwrite f8 [0,4194304] 0 2024-04-23T02:13:52.326 INFO:tasks.workunit.client.0.smithi071.stdout:3/43: readlink l6 0 2024-04-23T02:13:52.326 INFO:tasks.workunit.client.0.smithi071.stdout:8/117: read f8 [2141945,12330] 0 2024-04-23T02:13:52.327 INFO:tasks.workunit.client.1.smithi192.stdout:5/208: dwrite d2/db/f18 [0,4194304] 0 2024-04-23T02:13:52.328 INFO:tasks.workunit.client.1.smithi192.stdout:3/139: symlink d1/d3/d17/d1a/l26 0 2024-04-23T02:13:52.329 INFO:tasks.workunit.client.0.smithi071.stdout:3/44: creat d7/d8/fc x:0 0 0 2024-04-23T02:13:52.332 INFO:tasks.workunit.client.0.smithi071.stdout:3/45: creat d7/fd x:0 0 0 2024-04-23T02:13:52.338 INFO:tasks.workunit.client.1.smithi192.stdout:3/140: rename d1/d3/d6/f1c to d1/d3/d17/f27 0 2024-04-23T02:13:52.338 INFO:tasks.workunit.client.0.smithi071.stdout:3/46: creat d7/d8/fe x:0 0 0 2024-04-23T02:13:52.339 INFO:tasks.workunit.client.0.smithi071.stdout:8/118: dwrite dc/d10/f1c [0,4194304] 0 2024-04-23T02:13:52.341 INFO:tasks.workunit.client.0.smithi071.stdout:8/119: write dc/ff [3857748,18111] 0 2024-04-23T02:13:52.342 INFO:tasks.workunit.client.1.smithi192.stdout:9/230: symlink d0/d9/d44/d51/l57 0 2024-04-23T02:13:52.342 INFO:tasks.workunit.client.1.smithi192.stdout:5/209: mkdir d2/db/d46 0 2024-04-23T02:13:52.343 INFO:tasks.workunit.client.1.smithi192.stdout:3/141: chown d1/d3/d6 12762 1 2024-04-23T02:13:52.346 INFO:tasks.workunit.client.1.smithi192.stdout:9/231: dread - d0/d9/f38 zero size 2024-04-23T02:13:52.346 INFO:tasks.workunit.client.1.smithi192.stdout:9/232: chown d0/d1e 1 1 2024-04-23T02:13:52.346 INFO:tasks.workunit.client.1.smithi192.stdout:5/210: symlink d2/db/d16/d1b/l47 0 2024-04-23T02:13:52.347 INFO:tasks.workunit.client.1.smithi192.stdout:0/268: dread d6/df/f32 [0,4194304] 0 2024-04-23T02:13:52.349 INFO:tasks.workunit.client.1.smithi192.stdout:9/233: readlink d0/d9/d44/d51/l57 0 2024-04-23T02:13:52.350 INFO:tasks.workunit.client.1.smithi192.stdout:5/211: rmdir d2/db/d16/d1b 39 2024-04-23T02:13:52.351 INFO:tasks.workunit.client.0.smithi071.stdout:3/47: dwrite d7/d8/fe [0,4194304] 0 2024-04-23T02:13:52.352 INFO:tasks.workunit.client.1.smithi192.stdout:0/269: rmdir d6/df/d13 39 2024-04-23T02:13:52.354 INFO:tasks.workunit.client.0.smithi071.stdout:8/120: dwrite dc/d10/f1e [4194304,4194304] 0 2024-04-23T02:13:52.354 INFO:tasks.workunit.client.1.smithi192.stdout:0/270: truncate d6/df/f50 192737 0 2024-04-23T02:13:52.355 INFO:tasks.workunit.client.1.smithi192.stdout:9/234: mkdir d0/d9/d44/d58 0 2024-04-23T02:13:52.355 INFO:tasks.workunit.client.1.smithi192.stdout:3/142: dwrite d1/d3/d17/f27 [0,4194304] 0 2024-04-23T02:13:52.358 INFO:tasks.workunit.client.1.smithi192.stdout:6/250: truncate d3/f19 559216 0 2024-04-23T02:13:52.359 INFO:tasks.workunit.client.0.smithi071.stdout:8/121: creat dc/f20 x:0 0 0 2024-04-23T02:13:52.360 INFO:tasks.workunit.client.1.smithi192.stdout:3/143: write d1/d3/d17/d1a/f23 [132655,44712] 0 2024-04-23T02:13:52.362 INFO:tasks.workunit.client.0.smithi071.stdout:3/48: getdents d7 0 2024-04-23T02:13:52.363 INFO:tasks.workunit.client.1.smithi192.stdout:9/235: mknod d0/d9/d44/c59 0 2024-04-23T02:13:52.366 INFO:tasks.workunit.client.0.smithi071.stdout:3/49: write f0 [1320388,36329] 0 2024-04-23T02:13:52.366 INFO:tasks.workunit.client.0.smithi071.stdout:3/50: write d7/fd [897715,121823] 0 2024-04-23T02:13:52.367 INFO:tasks.workunit.client.1.smithi192.stdout:9/236: dread - d0/d9/d44/f45 zero size 2024-04-23T02:13:52.371 INFO:tasks.workunit.client.1.smithi192.stdout:5/212: write d2/f5 [3612516,117342] 0 2024-04-23T02:13:52.371 INFO:tasks.workunit.client.1.smithi192.stdout:6/251: dwrite d3/d11/f33 [4194304,4194304] 0 2024-04-23T02:13:52.375 INFO:tasks.workunit.client.1.smithi192.stdout:9/237: creat d0/d9/f5a x:0 0 0 2024-04-23T02:13:52.377 INFO:tasks.workunit.client.1.smithi192.stdout:6/252: mknod d3/d15/d1b/c47 0 2024-04-23T02:13:52.378 INFO:tasks.workunit.client.1.smithi192.stdout:9/238: mkdir d0/d9/d44/d51/d5b 0 2024-04-23T02:13:52.381 INFO:tasks.workunit.client.1.smithi192.stdout:6/253: truncate d3/d11/f43 241384 0 2024-04-23T02:13:52.381 INFO:tasks.workunit.client.0.smithi071.stdout:3/51: fdatasync d7/fd 0 2024-04-23T02:13:52.382 INFO:tasks.workunit.client.1.smithi192.stdout:0/271: write d6/df/d13/f51 [170765,63796] 0 2024-04-23T02:13:52.383 INFO:tasks.workunit.client.0.smithi071.stdout:3/52: getdents d7 0 2024-04-23T02:13:52.384 INFO:tasks.workunit.client.1.smithi192.stdout:0/272: stat d6/f3b 0 2024-04-23T02:13:52.385 INFO:tasks.workunit.client.1.smithi192.stdout:5/213: dwrite d2/db/f11 [0,4194304] 0 2024-04-23T02:13:52.385 INFO:tasks.workunit.client.1.smithi192.stdout:0/273: stat d6/df/f14 0 2024-04-23T02:13:52.386 INFO:tasks.workunit.client.1.smithi192.stdout:0/274: write d6/f3b [4829826,121751] 0 2024-04-23T02:13:52.386 INFO:tasks.workunit.client.1.smithi192.stdout:5/214: chown d2/d6/f12 277616 1 2024-04-23T02:13:52.387 INFO:tasks.workunit.client.1.smithi192.stdout:9/239: link d0/d9/d44/d4f/f2e d0/f5c 0 2024-04-23T02:13:52.389 INFO:tasks.workunit.client.1.smithi192.stdout:9/240: chown d0/d9/f5a 3426408 1 2024-04-23T02:13:52.390 INFO:tasks.workunit.client.0.smithi071.stdout:3/53: dwrite d7/d8/fe [0,4194304] 0 2024-04-23T02:13:52.390 INFO:tasks.workunit.client.0.smithi071.stdout:8/122: fsync dc/d10/f1e 0 2024-04-23T02:13:52.392 INFO:tasks.workunit.client.0.smithi071.stdout:3/54: truncate d7/fb 489455 0 2024-04-23T02:13:52.392 INFO:tasks.workunit.client.0.smithi071.stdout:3/55: fdatasync f5 0 2024-04-23T02:13:52.392 INFO:tasks.workunit.client.1.smithi192.stdout:6/254: getdents d3/d11 0 2024-04-23T02:13:52.393 INFO:tasks.workunit.client.0.smithi071.stdout:8/123: fsync dc/f18 0 2024-04-23T02:13:52.394 INFO:tasks.workunit.client.1.smithi192.stdout:5/215: mkdir d2/db/d16/d48 0 2024-04-23T02:13:52.396 INFO:tasks.workunit.client.1.smithi192.stdout:9/241: rename d0/d1e/l2a to d0/d9/d44/l5d 0 2024-04-23T02:13:52.396 INFO:tasks.workunit.client.1.smithi192.stdout:0/275: getdents d6/df/d47 0 2024-04-23T02:13:52.398 INFO:tasks.workunit.client.0.smithi071.stdout:3/56: symlink d7/d8/lf 0 2024-04-23T02:13:52.399 INFO:tasks.workunit.client.1.smithi192.stdout:6/255: mkdir d3/d11/d44/d48 0 2024-04-23T02:13:52.399 INFO:tasks.workunit.client.1.smithi192.stdout:9/242: fdatasync d0/d9/d44/d4f/f8 0 2024-04-23T02:13:52.401 INFO:tasks.workunit.client.0.smithi071.stdout:3/57: mkdir d7/d10 0 2024-04-23T02:13:52.402 INFO:tasks.workunit.client.0.smithi071.stdout:8/124: dwrite dc/d10/f1a [0,4194304] 0 2024-04-23T02:13:52.404 INFO:tasks.workunit.client.0.smithi071.stdout:8/125: write dc/d10/f1e [77221,85662] 0 2024-04-23T02:13:52.405 INFO:tasks.workunit.client.1.smithi192.stdout:6/256: creat d3/f49 x:0 0 0 2024-04-23T02:13:52.405 INFO:tasks.workunit.client.0.smithi071.stdout:8/126: dread dc/f18 [0,4194304] 0 2024-04-23T02:13:52.408 INFO:tasks.workunit.client.0.smithi071.stdout:8/127: mknod dc/d10/c21 0 2024-04-23T02:13:52.409 INFO:tasks.workunit.client.1.smithi192.stdout:5/216: mkdir d2/db/d16/d1b/d3d/d40/d49 0 2024-04-23T02:13:52.409 INFO:tasks.workunit.client.1.smithi192.stdout:0/276: link d6/lb d6/df/d35/l60 0 2024-04-23T02:13:52.409 INFO:tasks.workunit.client.0.smithi071.stdout:8/128: creat dc/d10/f22 x:0 0 0 2024-04-23T02:13:52.411 INFO:tasks.workunit.client.1.smithi192.stdout:3/144: dread d1/f5 [0,4194304] 0 2024-04-23T02:13:52.412 INFO:tasks.workunit.client.1.smithi192.stdout:3/145: write d1/d3/f19 [721656,61905] 0 2024-04-23T02:13:52.412 INFO:tasks.workunit.client.0.smithi071.stdout:8/129: creat dc/d15/f23 x:0 0 0 2024-04-23T02:13:52.413 INFO:tasks.workunit.client.0.smithi071.stdout:8/130: chown dc/c13 174837 1 2024-04-23T02:13:52.413 INFO:tasks.workunit.client.1.smithi192.stdout:6/257: rename d3/d15/d1b/d21/f32 to d3/d11/f4a 0 2024-04-23T02:13:52.414 INFO:tasks.workunit.client.1.smithi192.stdout:6/258: chown d3/d11/f1d 3008 1 2024-04-23T02:13:52.415 INFO:tasks.workunit.client.1.smithi192.stdout:0/277: creat d6/df/d47/f61 x:0 0 0 2024-04-23T02:13:52.416 INFO:tasks.workunit.client.0.smithi071.stdout:3/58: dread d7/fd [0,4194304] 0 2024-04-23T02:13:52.418 INFO:tasks.workunit.client.1.smithi192.stdout:6/259: write d3/f34 [361829,101928] 0 2024-04-23T02:13:52.418 INFO:tasks.workunit.client.1.smithi192.stdout:0/278: dread - d6/d44/f38 zero size 2024-04-23T02:13:52.418 INFO:tasks.workunit.client.0.smithi071.stdout:3/59: creat d7/d8/f11 x:0 0 0 2024-04-23T02:13:52.418 INFO:tasks.workunit.client.0.smithi071.stdout:8/131: dread dc/d10/f1a [0,4194304] 0 2024-04-23T02:13:52.419 INFO:tasks.workunit.client.1.smithi192.stdout:9/243: truncate d0/d9/f26 61675 0 2024-04-23T02:13:52.420 INFO:tasks.workunit.client.0.smithi071.stdout:3/60: mknod d7/d8/c12 0 2024-04-23T02:13:52.421 INFO:tasks.workunit.client.0.smithi071.stdout:8/132: mknod dc/d15/c24 0 2024-04-23T02:13:52.422 INFO:tasks.workunit.client.0.smithi071.stdout:3/61: symlink d7/d8/l13 0 2024-04-23T02:13:52.423 INFO:tasks.workunit.client.0.smithi071.stdout:3/62: write d7/d8/f11 [650345,45502] 0 2024-04-23T02:13:52.424 INFO:tasks.workunit.client.0.smithi071.stdout:3/63: write d7/d8/fe [3003220,102329] 0 2024-04-23T02:13:52.424 INFO:tasks.workunit.client.1.smithi192.stdout:6/260: creat d3/d15/f4b x:0 0 0 2024-04-23T02:13:52.426 INFO:tasks.workunit.client.1.smithi192.stdout:3/146: dwrite d1/d3/d17/d1a/f23 [0,4194304] 0 2024-04-23T02:13:52.426 INFO:tasks.workunit.client.1.smithi192.stdout:9/244: fsync d0/d9/f32 0 2024-04-23T02:13:52.427 INFO:tasks.workunit.client.0.smithi071.stdout:8/133: fsync dc/d15/f23 0 2024-04-23T02:13:52.429 INFO:tasks.workunit.client.1.smithi192.stdout:6/261: symlink d3/d15/l4c 0 2024-04-23T02:13:52.429 INFO:tasks.workunit.client.0.smithi071.stdout:8/134: creat dc/d15/f25 x:0 0 0 2024-04-23T02:13:52.432 INFO:tasks.workunit.client.1.smithi192.stdout:0/279: dwrite d6/df/d47/f61 [0,4194304] 0 2024-04-23T02:13:52.432 INFO:tasks.workunit.client.0.smithi071.stdout:8/135: mknod dc/d10/c26 0 2024-04-23T02:13:52.433 INFO:tasks.workunit.client.1.smithi192.stdout:0/280: readlink d6/df/d13/l43 0 2024-04-23T02:13:52.433 INFO:tasks.workunit.client.1.smithi192.stdout:3/147: rmdir d1/d3/d6/d7 39 2024-04-23T02:13:52.433 INFO:tasks.workunit.client.0.smithi071.stdout:3/64: truncate d7/fb 139692 0 2024-04-23T02:13:52.436 INFO:tasks.workunit.client.0.smithi071.stdout:3/65: mknod d7/c14 0 2024-04-23T02:13:52.437 INFO:tasks.workunit.client.0.smithi071.stdout:8/136: dread dc/d10/f1c [0,4194304] 0 2024-04-23T02:13:52.437 INFO:tasks.workunit.client.0.smithi071.stdout:3/66: write d7/d8/f11 [1204238,57556] 0 2024-04-23T02:13:52.438 INFO:tasks.workunit.client.0.smithi071.stdout:3/67: readlink d7/d8/lf 0 2024-04-23T02:13:52.442 INFO:tasks.workunit.client.0.smithi071.stdout:3/68: creat d7/d10/f15 x:0 0 0 2024-04-23T02:13:52.443 INFO:tasks.workunit.client.1.smithi192.stdout:3/148: chown d1/d3/d6/d7/c1d 468204 1 2024-04-23T02:13:52.443 INFO:tasks.workunit.client.1.smithi192.stdout:9/245: dread d0/d9/d44/d4f/f1c [4194304,4194304] 0 2024-04-23T02:13:52.444 INFO:tasks.workunit.client.0.smithi071.stdout:3/69: creat d7/d8/f16 x:0 0 0 2024-04-23T02:13:52.445 INFO:tasks.workunit.client.1.smithi192.stdout:3/149: symlink d1/d3/d6/d7/d1b/l28 0 2024-04-23T02:13:52.446 INFO:tasks.workunit.client.1.smithi192.stdout:0/281: dwrite d6/df/d13/d25/d26/f27 [0,4194304] 0 2024-04-23T02:13:52.446 INFO:tasks.workunit.client.0.smithi071.stdout:8/137: dwrite f6 [0,4194304] 0 2024-04-23T02:13:52.448 INFO:tasks.workunit.client.0.smithi071.stdout:8/138: creat dc/d10/f27 x:0 0 0 2024-04-23T02:13:52.449 INFO:tasks.workunit.client.0.smithi071.stdout:8/139: chown dc/c13 13502334 1 2024-04-23T02:13:52.449 INFO:tasks.workunit.client.0.smithi071.stdout:8/140: truncate f8 4508755 0 2024-04-23T02:13:52.449 INFO:tasks.workunit.client.1.smithi192.stdout:9/246: creat d0/d9/d44/f5e x:0 0 0 2024-04-23T02:13:52.450 INFO:tasks.workunit.client.1.smithi192.stdout:0/282: write d6/df/d13/f31 [3731923,58333] 0 2024-04-23T02:13:52.451 INFO:tasks.workunit.client.1.smithi192.stdout:0/283: write d6/f3b [1738668,123738] 0 2024-04-23T02:13:52.451 INFO:tasks.workunit.client.1.smithi192.stdout:4/361: sync 2024-04-23T02:13:52.451 INFO:tasks.workunit.client.1.smithi192.stdout:2/218: sync 2024-04-23T02:13:52.452 INFO:tasks.workunit.client.0.smithi071.stdout:4/94: sync 2024-04-23T02:13:52.452 INFO:tasks.workunit.client.0.smithi071.stdout:4/95: fdatasync d2/d15/d1d/d23/f1f 0 2024-04-23T02:13:52.454 INFO:tasks.workunit.client.0.smithi071.stdout:4/96: chown d2/d15/f22 3484071 1 2024-04-23T02:13:52.455 INFO:tasks.workunit.client.1.smithi192.stdout:2/219: rename d8/d26/d29 to d8/d26/d29/d43/d46 22 2024-04-23T02:13:52.455 INFO:tasks.workunit.client.1.smithi192.stdout:3/150: mkdir d1/d3/d6/d7/d29 0 2024-04-23T02:13:52.458 INFO:tasks.workunit.client.1.smithi192.stdout:4/362: mkdir de/d15/d2a/d74 0 2024-04-23T02:13:52.462 INFO:tasks.workunit.client.1.smithi192.stdout:3/151: read d1/d3/d17/d1a/f23 [2064414,56574] 0 2024-04-23T02:13:52.462 INFO:tasks.workunit.client.1.smithi192.stdout:0/284: dread d6/df/d13/f51 [0,4194304] 0 2024-04-23T02:13:52.462 INFO:tasks.workunit.client.0.smithi071.stdout:3/70: dread d7/d8/f11 [0,4194304] 0 2024-04-23T02:13:52.463 INFO:tasks.workunit.client.0.smithi071.stdout:3/71: chown l4 41723 1 2024-04-23T02:13:52.464 INFO:tasks.workunit.client.1.smithi192.stdout:6/262: truncate d3/f2b 5026038 0 2024-04-23T02:13:52.464 INFO:tasks.workunit.client.0.smithi071.stdout:3/72: fsync d7/d8/f11 0 2024-04-23T02:13:52.466 INFO:tasks.workunit.client.1.smithi192.stdout:9/247: dwrite d0/d9/d44/f49 [0,4194304] 0 2024-04-23T02:13:52.467 INFO:tasks.workunit.client.1.smithi192.stdout:2/220: readlink d8/d26/d29/d43/l44 0 2024-04-23T02:13:52.471 INFO:tasks.workunit.client.1.smithi192.stdout:0/285: creat d6/df/d13/d25/d3a/f62 x:0 0 0 2024-04-23T02:13:52.472 INFO:tasks.workunit.client.0.smithi071.stdout:4/97: write d2/d15/d1d/d23/f16 [1411518,84541] 0 2024-04-23T02:13:52.473 INFO:tasks.workunit.client.1.smithi192.stdout:2/221: rmdir d8/d26/d29/d43 39 2024-04-23T02:13:52.477 INFO:tasks.workunit.client.1.smithi192.stdout:4/363: read de/d15/d19/d3d/d49/f28 [982491,87183] 0 2024-04-23T02:13:52.477 INFO:tasks.workunit.client.1.smithi192.stdout:2/222: symlink d8/d25/l47 0 2024-04-23T02:13:52.478 INFO:tasks.workunit.client.0.smithi071.stdout:8/141: dwrite dc/f18 [0,4194304] 0 2024-04-23T02:13:52.479 INFO:tasks.workunit.client.1.smithi192.stdout:4/364: write de/f18 [855612,121961] 0 2024-04-23T02:13:52.479 INFO:tasks.workunit.client.1.smithi192.stdout:0/286: creat d6/df/d47/f63 x:0 0 0 2024-04-23T02:13:52.480 INFO:tasks.workunit.client.0.smithi071.stdout:4/98: truncate d2/d15/d1d/d23/d18/f1b 912205 0 2024-04-23T02:13:52.481 INFO:tasks.workunit.client.1.smithi192.stdout:3/152: dwrite d1/d3/d6/f10 [0,4194304] 0 2024-04-23T02:13:52.485 INFO:tasks.workunit.client.0.smithi071.stdout:4/99: creat d2/d15/d1d/d23/d17/f24 x:0 0 0 2024-04-23T02:13:52.485 INFO:tasks.workunit.client.0.smithi071.stdout:4/100: stat d2/c6 0 2024-04-23T02:13:52.486 INFO:tasks.workunit.client.1.smithi192.stdout:6/263: link d3/d11/l42 d3/d15/l4d 0 2024-04-23T02:13:52.487 INFO:tasks.workunit.client.0.smithi071.stdout:8/142: creat dc/f28 x:0 0 0 2024-04-23T02:13:52.488 INFO:tasks.workunit.client.0.smithi071.stdout:8/143: chown dc/ff 303 1 2024-04-23T02:13:52.493 INFO:tasks.workunit.client.0.smithi071.stdout:4/101: dwrite d2/d15/f1a [0,4194304] 0 2024-04-23T02:13:52.495 INFO:tasks.workunit.client.0.smithi071.stdout:4/102: chown d2/d15/d1d/l21 3418 1 2024-04-23T02:13:52.496 INFO:tasks.workunit.client.1.smithi192.stdout:0/287: write d6/f10 [1817548,43166] 0 2024-04-23T02:13:52.497 INFO:tasks.workunit.client.1.smithi192.stdout:3/153: chown d1/d3/d6/f10 5102 1 2024-04-23T02:13:52.501 INFO:tasks.workunit.client.1.smithi192.stdout:0/288: dread - d6/df/d47/f63 zero size 2024-04-23T02:13:52.501 INFO:tasks.workunit.client.1.smithi192.stdout:6/264: dwrite d3/d15/f4b [0,4194304] 0 2024-04-23T02:13:52.509 INFO:tasks.workunit.client.0.smithi071.stdout:8/144: fsync dc/f18 0 2024-04-23T02:13:52.510 INFO:tasks.workunit.client.1.smithi192.stdout:9/248: truncate d0/f56 1495782 0 2024-04-23T02:13:52.510 INFO:tasks.workunit.client.1.smithi192.stdout:3/154: dread d1/d3/d17/f27 [0,4194304] 0 2024-04-23T02:13:52.511 INFO:tasks.workunit.client.0.smithi071.stdout:8/145: creat dc/d15/f29 x:0 0 0 2024-04-23T02:13:52.511 INFO:tasks.workunit.client.0.smithi071.stdout:4/103: read d2/d15/d1d/d23/d18/f1b [18666,56515] 0 2024-04-23T02:13:52.511 INFO:tasks.workunit.client.0.smithi071.stdout:8/146: write dc/ff [1924790,69823] 0 2024-04-23T02:13:52.512 INFO:tasks.workunit.client.0.smithi071.stdout:4/104: chown d2/d15/d1d/d23/d10/c12 9 1 2024-04-23T02:13:52.513 INFO:tasks.workunit.client.0.smithi071.stdout:8/147: truncate dc/d15/f25 984529 0 2024-04-23T02:13:52.515 INFO:tasks.workunit.client.0.smithi071.stdout:4/105: write d2/d15/f1a [1146224,114727] 0 2024-04-23T02:13:52.515 INFO:tasks.workunit.client.1.smithi192.stdout:0/289: rename d6/df/d35/d49/c5c to d6/df/d13/d25/d26/c64 0 2024-04-23T02:13:52.516 INFO:tasks.workunit.client.1.smithi192.stdout:6/265: creat d3/d15/d1b/d21/d3a/f4e x:0 0 0 2024-04-23T02:13:52.516 INFO:tasks.workunit.client.0.smithi071.stdout:4/106: fsync d2/d15/f1e 0 2024-04-23T02:13:52.517 INFO:tasks.workunit.client.1.smithi192.stdout:3/155: creat d1/d3/d6/d7/f2a x:0 0 0 2024-04-23T02:13:52.519 INFO:tasks.workunit.client.0.smithi071.stdout:8/148: dread f8 [0,4194304] 0 2024-04-23T02:13:52.519 INFO:tasks.workunit.client.1.smithi192.stdout:9/249: fsync d0/d9/f43 0 2024-04-23T02:13:52.519 INFO:tasks.workunit.client.0.smithi071.stdout:4/107: symlink d2/d15/d1d/d23/l25 0 2024-04-23T02:13:52.522 INFO:tasks.workunit.client.1.smithi192.stdout:3/156: mkdir d1/d3/d17/d2b 0 2024-04-23T02:13:52.522 INFO:tasks.workunit.client.0.smithi071.stdout:4/108: symlink d2/d15/d1d/d23/d10/l26 0 2024-04-23T02:13:52.523 INFO:tasks.workunit.client.0.smithi071.stdout:8/149: rename dc/l1d to dc/d15/l2a 0 2024-04-23T02:13:52.524 INFO:tasks.workunit.client.0.smithi071.stdout:4/109: truncate d2/d15/d1d/d23/d17/f24 783035 0 2024-04-23T02:13:52.524 INFO:tasks.workunit.client.1.smithi192.stdout:9/250: unlink d0/d9/d44/d4f/l17 0 2024-04-23T02:13:52.525 INFO:tasks.workunit.client.0.smithi071.stdout:8/150: mknod dc/d15/c2b 0 2024-04-23T02:13:52.525 INFO:tasks.workunit.client.1.smithi192.stdout:4/365: dwrite de/d15/d19/d3d/f57 [0,4194304] 0 2024-04-23T02:13:52.528 INFO:tasks.workunit.client.1.smithi192.stdout:4/366: readlink l5 0 2024-04-23T02:13:52.528 INFO:tasks.workunit.client.1.smithi192.stdout:3/157: rmdir d1 39 2024-04-23T02:13:52.531 INFO:tasks.workunit.client.1.smithi192.stdout:0/290: dwrite d6/df/d35/d49/f54 [0,4194304] 0 2024-04-23T02:13:52.532 INFO:tasks.workunit.client.0.smithi071.stdout:8/151: dwrite dc/d10/f14 [0,4194304] 0 2024-04-23T02:13:52.532 INFO:tasks.workunit.client.0.smithi071.stdout:8/152: stat dc/ce 0 2024-04-23T02:13:52.533 INFO:tasks.workunit.client.1.smithi192.stdout:6/266: rename d3/d15/d1b/c20 to d3/c4f 0 2024-04-23T02:13:52.533 INFO:tasks.workunit.client.0.smithi071.stdout:8/153: write dc/d15/f25 [1110337,58200] 0 2024-04-23T02:13:52.535 INFO:tasks.workunit.client.1.smithi192.stdout:4/367: creat de/d15/d19/d3d/d34/d4a/f75 x:0 0 0 2024-04-23T02:13:52.536 INFO:tasks.workunit.client.0.smithi071.stdout:8/154: write dc/ff [1962291,71746] 0 2024-04-23T02:13:52.541 INFO:tasks.workunit.client.0.smithi071.stdout:8/155: truncate dc/d15/f29 493878 0 2024-04-23T02:13:52.543 INFO:tasks.workunit.client.0.smithi071.stdout:4/110: write d2/d15/d1d/d23/ff [717522,47176] 0 2024-04-23T02:13:52.543 INFO:tasks.workunit.client.0.smithi071.stdout:4/111: chown d2/d15/d1d 22 1 2024-04-23T02:13:52.543 INFO:tasks.workunit.client.0.smithi071.stdout:8/156: mknod dc/c2c 0 2024-04-23T02:13:52.544 INFO:tasks.workunit.client.0.smithi071.stdout:4/112: write d2/d15/f1e [1467313,49386] 0 2024-04-23T02:13:52.544 INFO:tasks.workunit.client.0.smithi071.stdout:8/157: fdatasync f6 0 2024-04-23T02:13:52.545 INFO:tasks.workunit.client.1.smithi192.stdout:6/267: rename d3/f19 to d3/d11/d44/f50 0 2024-04-23T02:13:52.545 INFO:tasks.workunit.client.1.smithi192.stdout:3/158: dread d1/d3/d17/f27 [0,4194304] 0 2024-04-23T02:13:52.545 INFO:tasks.workunit.client.0.smithi071.stdout:4/113: truncate d2/d15/d1d/d23/d17/f24 1182972 0 2024-04-23T02:13:52.548 INFO:tasks.workunit.client.0.smithi071.stdout:4/114: dread d2/d15/d1d/d23/d18/f1b [0,4194304] 0 2024-04-23T02:13:52.548 INFO:tasks.workunit.client.1.smithi192.stdout:6/268: read d3/f34 [271926,111265] 0 2024-04-23T02:13:52.551 INFO:tasks.workunit.client.1.smithi192.stdout:6/269: fdatasync d3/f2b 0 2024-04-23T02:13:52.552 INFO:tasks.workunit.client.1.smithi192.stdout:3/159: rmdir d1/d3/d6/d7/d1b/d22 0 2024-04-23T02:13:52.552 INFO:tasks.workunit.client.0.smithi071.stdout:4/115: getdents d2/d15/d1d/d23 0 2024-04-23T02:13:52.552 INFO:tasks.workunit.client.0.smithi071.stdout:8/158: rmdir dc 39 2024-04-23T02:13:52.554 INFO:tasks.workunit.client.0.smithi071.stdout:4/116: stat d2/d15/d1d/d23/d17 0 2024-04-23T02:13:52.554 INFO:tasks.workunit.client.1.smithi192.stdout:6/270: mkdir d3/d15/d1b/d21/d3a/d51 0 2024-04-23T02:13:52.554 INFO:tasks.workunit.client.0.smithi071.stdout:8/159: fsync dc/f28 0 2024-04-23T02:13:52.555 INFO:tasks.workunit.client.0.smithi071.stdout:4/117: chown d2/d15/d1d/d23/d17/f24 1135 1 2024-04-23T02:13:52.555 INFO:tasks.workunit.client.1.smithi192.stdout:3/160: write d1/f5 [1499688,75169] 0 2024-04-23T02:13:52.556 INFO:tasks.workunit.client.1.smithi192.stdout:6/271: unlink d3/d15/l17 0 2024-04-23T02:13:52.557 INFO:tasks.workunit.client.1.smithi192.stdout:3/161: mknod d1/d3/d17/c2c 0 2024-04-23T02:13:52.557 INFO:tasks.workunit.client.0.smithi071.stdout:4/118: mkdir d2/d15/d1d/d23/d18/d27 0 2024-04-23T02:13:52.558 INFO:tasks.workunit.client.0.smithi071.stdout:4/119: chown d2/c6 1997 1 2024-04-23T02:13:52.560 INFO:tasks.workunit.client.1.smithi192.stdout:3/162: creat d1/d3/d17/f2d x:0 0 0 2024-04-23T02:13:52.561 INFO:tasks.workunit.client.0.smithi071.stdout:8/160: dwrite dc/f18 [0,4194304] 0 2024-04-23T02:13:52.561 INFO:tasks.workunit.client.0.smithi071.stdout:4/120: dread d2/d15/d1d/d23/d18/f1b [0,4194304] 0 2024-04-23T02:13:52.561 INFO:tasks.workunit.client.0.smithi071.stdout:8/161: write f6 [2021683,10961] 0 2024-04-23T02:13:52.564 INFO:tasks.workunit.client.0.smithi071.stdout:8/162: mkdir dc/d15/d2d 0 2024-04-23T02:13:52.565 INFO:tasks.workunit.client.1.smithi192.stdout:3/163: mknod d1/d3/d17/d1a/c2e 0 2024-04-23T02:13:52.566 INFO:tasks.workunit.client.0.smithi071.stdout:4/121: rmdir d2/d15 39 2024-04-23T02:13:52.566 INFO:tasks.workunit.client.1.smithi192.stdout:6/272: rmdir d3/d11/d44/d48 0 2024-04-23T02:13:52.567 INFO:tasks.workunit.client.0.smithi071.stdout:8/163: mknod dc/d15/d2d/c2e 0 2024-04-23T02:13:52.568 INFO:tasks.workunit.client.1.smithi192.stdout:4/368: dwrite de/d15/d19/d3d/d49/f50 [0,4194304] 0 2024-04-23T02:13:52.570 INFO:tasks.workunit.client.1.smithi192.stdout:6/273: chown d3/d15/c3b 29019 1 2024-04-23T02:13:52.571 INFO:tasks.workunit.client.1.smithi192.stdout:3/164: creat d1/d3/d6/f2f x:0 0 0 2024-04-23T02:13:52.573 INFO:tasks.workunit.client.1.smithi192.stdout:4/369: unlink de/d15/d2a/c43 0 2024-04-23T02:13:52.574 INFO:tasks.workunit.client.0.smithi071.stdout:4/122: dwrite d2/d15/d1d/d23/d17/f24 [0,4194304] 0 2024-04-23T02:13:52.575 INFO:tasks.workunit.client.0.smithi071.stdout:8/164: dwrite f9 [0,4194304] 0 2024-04-23T02:13:52.575 INFO:tasks.workunit.client.0.smithi071.stdout:4/123: stat d2/l11 0 2024-04-23T02:13:52.576 INFO:tasks.workunit.client.0.smithi071.stdout:4/124: fdatasync d2/d15/d1d/d23/f1f 0 2024-04-23T02:13:52.577 INFO:tasks.workunit.client.0.smithi071.stdout:8/165: mknod dc/c2f 0 2024-04-23T02:13:52.578 INFO:tasks.workunit.client.1.smithi192.stdout:3/165: symlink d1/d3/d17/l30 0 2024-04-23T02:13:52.579 INFO:tasks.workunit.client.0.smithi071.stdout:8/166: mkdir dc/d10/d30 0 2024-04-23T02:13:52.580 INFO:tasks.workunit.client.0.smithi071.stdout:8/167: dread - dc/f28 zero size 2024-04-23T02:13:52.580 INFO:tasks.workunit.client.1.smithi192.stdout:3/166: chown d1/l16 187507242 1 2024-04-23T02:13:52.582 INFO:tasks.workunit.client.0.smithi071.stdout:8/168: creat dc/d15/f31 x:0 0 0 2024-04-23T02:13:52.583 INFO:tasks.workunit.client.0.smithi071.stdout:8/169: readlink l3 0 2024-04-23T02:13:52.584 INFO:tasks.workunit.client.0.smithi071.stdout:8/170: write dc/d10/f1a [4636400,22389] 0 2024-04-23T02:13:52.586 INFO:tasks.workunit.client.0.smithi071.stdout:8/171: mkdir dc/d10/d32 0 2024-04-23T02:13:52.588 INFO:tasks.workunit.client.0.smithi071.stdout:8/172: dread dc/d15/f29 [0,4194304] 0 2024-04-23T02:13:52.590 INFO:tasks.workunit.client.1.smithi192.stdout:3/167: creat d1/d3/d6/d7/f31 x:0 0 0 2024-04-23T02:13:52.592 INFO:tasks.workunit.client.1.smithi192.stdout:6/274: dwrite d3/f22 [0,4194304] 0 2024-04-23T02:13:52.592 INFO:tasks.workunit.client.1.smithi192.stdout:4/370: dwrite de/d15/d19/d1a/f6d [0,4194304] 0 2024-04-23T02:13:52.593 INFO:tasks.workunit.client.1.smithi192.stdout:6/275: stat d3/d28/f2e 0 2024-04-23T02:13:52.594 INFO:tasks.workunit.client.1.smithi192.stdout:4/371: readlink de/d15/l4e 0 2024-04-23T02:13:52.594 INFO:tasks.workunit.client.1.smithi192.stdout:6/276: chown d3/d15/d1b 4755 1 2024-04-23T02:13:52.595 INFO:tasks.workunit.client.1.smithi192.stdout:6/277: stat d3/d15/c3b 0 2024-04-23T02:13:52.596 INFO:tasks.workunit.client.0.smithi071.stdout:4/125: dwrite d2/f4 [0,4194304] 0 2024-04-23T02:13:52.597 INFO:tasks.workunit.client.1.smithi192.stdout:4/372: fsync de/f1b 0 2024-04-23T02:13:52.598 INFO:tasks.workunit.client.1.smithi192.stdout:3/168: creat d1/d3/d17/d2b/f32 x:0 0 0 2024-04-23T02:13:52.599 INFO:tasks.workunit.client.1.smithi192.stdout:6/278: mkdir d3/d15/d1b/d21/d52 0 2024-04-23T02:13:52.601 INFO:tasks.workunit.client.0.smithi071.stdout:4/126: unlink d2/d15/d1d/d23/d17/c19 0 2024-04-23T02:13:52.605 INFO:tasks.workunit.client.1.smithi192.stdout:3/169: rename d1/d3/d6/d7/d1b to d1/d3/d17/d1f/d33 0 2024-04-23T02:13:52.607 INFO:tasks.workunit.client.0.smithi071.stdout:4/127: rename d2/d15/d1d/d23/d10/c12 to d2/c28 0 2024-04-23T02:13:52.607 INFO:tasks.workunit.client.0.smithi071.stdout:4/128: readlink d2/l1c 0 2024-04-23T02:13:52.608 INFO:tasks.workunit.client.1.smithi192.stdout:3/170: mkdir d1/d3/d17/d1f/d33/d24/d34 0 2024-04-23T02:13:52.609 INFO:tasks.workunit.client.0.smithi071.stdout:4/129: mkdir d2/d15/d1d/d23/d29 0 2024-04-23T02:13:52.609 INFO:tasks.workunit.client.1.smithi192.stdout:3/171: symlink d1/d3/d17/l35 0 2024-04-23T02:13:52.610 INFO:tasks.workunit.client.0.smithi071.stdout:2/87: sync 2024-04-23T02:13:52.610 INFO:tasks.workunit.client.0.smithi071.stdout:0/105: sync 2024-04-23T02:13:52.610 INFO:tasks.workunit.client.1.smithi192.stdout:3/172: chown d1/d3/d17/d1f/d33/d24/d34 0 1 2024-04-23T02:13:52.611 INFO:tasks.workunit.client.1.smithi192.stdout:3/173: chown d1/d3/d17 4489690 1 2024-04-23T02:13:52.611 INFO:tasks.workunit.client.0.smithi071.stdout:2/88: read d0/d2/dd/fc [3100982,85312] 0 2024-04-23T02:13:52.612 INFO:tasks.workunit.client.0.smithi071.stdout:2/89: rename d0 to d0/d15 22 2024-04-23T02:13:52.612 INFO:tasks.workunit.client.0.smithi071.stdout:0/106: creat d0/d3/d1c/f22 x:0 0 0 2024-04-23T02:13:52.613 INFO:tasks.workunit.client.0.smithi071.stdout:4/130: mkdir d2/d2a 0 2024-04-23T02:13:52.615 INFO:tasks.workunit.client.0.smithi071.stdout:2/90: rename d0/d2/ce to d0/d2/c16 0 2024-04-23T02:13:52.616 INFO:tasks.workunit.client.0.smithi071.stdout:4/131: fdatasync d2/d15/d1d/d23/d18/f1b 0 2024-04-23T02:13:52.617 INFO:tasks.workunit.client.0.smithi071.stdout:0/107: mkdir d0/d3/de/d10/d16/d23 0 2024-04-23T02:13:52.617 INFO:tasks.workunit.client.0.smithi071.stdout:2/91: symlink d0/d2/dd/l17 0 2024-04-23T02:13:52.619 INFO:tasks.workunit.client.0.smithi071.stdout:0/108: mkdir d0/d24 0 2024-04-23T02:13:52.619 INFO:tasks.workunit.client.0.smithi071.stdout:0/109: chown d0/f20 6 1 2024-04-23T02:13:52.620 INFO:tasks.workunit.client.0.smithi071.stdout:4/132: mknod d2/d15/d1d/d23/d29/c2b 0 2024-04-23T02:13:52.622 INFO:tasks.workunit.client.0.smithi071.stdout:0/110: symlink d0/d3/de/l25 0 2024-04-23T02:13:52.626 INFO:tasks.workunit.client.0.smithi071.stdout:4/133: rename d2/d15/d1d/d23/d18 to d2/d15/d1d/d2c 0 2024-04-23T02:13:52.627 INFO:tasks.workunit.client.0.smithi071.stdout:2/92: dwrite d0/d2/f9 [0,4194304] 0 2024-04-23T02:13:52.630 INFO:tasks.workunit.client.1.smithi192.stdout:4/373: dread de/d15/d19/d3d/f46 [0,4194304] 0 2024-04-23T02:13:52.631 INFO:tasks.workunit.client.0.smithi071.stdout:2/93: stat d0/d2/c8 0 2024-04-23T02:13:52.636 INFO:tasks.workunit.client.0.smithi071.stdout:2/94: creat d0/d2/dd/f18 x:0 0 0 2024-04-23T02:13:52.636 INFO:tasks.workunit.client.0.smithi071.stdout:8/173: truncate f9 4213499 0 2024-04-23T02:13:52.637 INFO:tasks.workunit.client.0.smithi071.stdout:8/174: readlink l3 0 2024-04-23T02:13:52.641 INFO:tasks.workunit.client.0.smithi071.stdout:2/95: creat d0/d2/dd/d14/f19 x:0 0 0 2024-04-23T02:13:52.644 INFO:tasks.workunit.client.0.smithi071.stdout:2/96: chown d0/d2/dd/d14/f19 198 1 2024-04-23T02:13:52.646 INFO:tasks.workunit.client.0.smithi071.stdout:8/175: dwrite dc/d10/f27 [0,4194304] 0 2024-04-23T02:13:52.646 INFO:tasks.workunit.client.1.smithi192.stdout:3/174: write d1/d3/d17/d1a/f23 [3248021,124185] 0 2024-04-23T02:13:52.647 INFO:tasks.workunit.client.0.smithi071.stdout:8/176: write dc/f20 [441225,127528] 0 2024-04-23T02:13:52.648 INFO:tasks.workunit.client.1.smithi192.stdout:6/279: dwrite d3/f2b [4194304,4194304] 0 2024-04-23T02:13:52.649 INFO:tasks.workunit.client.0.smithi071.stdout:4/134: write d2/d15/d1d/d23/fc [1479351,119762] 0 2024-04-23T02:13:52.650 INFO:tasks.workunit.client.1.smithi192.stdout:3/175: mknod d1/d3/d17/d1a/c36 0 2024-04-23T02:13:52.652 INFO:tasks.workunit.client.0.smithi071.stdout:0/111: write d0/d3/de/f13 [1062788,104581] 0 2024-04-23T02:13:52.653 INFO:tasks.workunit.client.0.smithi071.stdout:8/177: mknod dc/d15/d2d/c33 0 2024-04-23T02:13:52.654 INFO:tasks.workunit.client.0.smithi071.stdout:8/178: readlink l5 0 2024-04-23T02:13:52.654 INFO:tasks.workunit.client.0.smithi071.stdout:4/135: mkdir d2/d2a/d2d 0 2024-04-23T02:13:52.657 INFO:tasks.workunit.client.0.smithi071.stdout:0/112: dread - d0/d3/de/d10/f1f zero size 2024-04-23T02:13:52.659 INFO:tasks.workunit.client.0.smithi071.stdout:8/179: write dc/d10/f1c [1521480,44434] 0 2024-04-23T02:13:52.661 INFO:tasks.workunit.client.0.smithi071.stdout:0/113: mkdir d0/da/d26 0 2024-04-23T02:13:52.661 INFO:tasks.workunit.client.1.smithi192.stdout:6/280: link d3/l37 d3/d15/l53 0 2024-04-23T02:13:52.662 INFO:tasks.workunit.client.0.smithi071.stdout:2/97: dwrite d0/d2/fb [4194304,4194304] 0 2024-04-23T02:13:52.663 INFO:tasks.workunit.client.0.smithi071.stdout:0/114: dread - d0/f18 zero size 2024-04-23T02:13:52.667 INFO:tasks.workunit.client.0.smithi071.stdout:2/98: creat d0/d2/dd/d14/f1a x:0 0 0 2024-04-23T02:13:52.669 INFO:tasks.workunit.client.1.smithi192.stdout:6/281: dwrite d3/d11/f29 [0,4194304] 0 2024-04-23T02:13:52.669 INFO:tasks.workunit.client.0.smithi071.stdout:4/136: dwrite d2/d15/d1d/d23/f16 [0,4194304] 0 2024-04-23T02:13:52.670 INFO:tasks.workunit.client.0.smithi071.stdout:0/115: rename d0/d3/de/l17 to d0/da/l27 0 2024-04-23T02:13:52.670 INFO:tasks.workunit.client.0.smithi071.stdout:2/99: write d0/d2/f9 [3178308,120162] 0 2024-04-23T02:13:52.670 INFO:tasks.workunit.client.0.smithi071.stdout:8/180: dread dc/d10/f27 [0,4194304] 0 2024-04-23T02:13:52.673 INFO:tasks.workunit.client.0.smithi071.stdout:4/137: write d2/d15/d1d/d23/f16 [279454,94985] 0 2024-04-23T02:13:52.673 INFO:tasks.workunit.client.0.smithi071.stdout:2/100: dread - d0/d2/dd/d14/f1a zero size 2024-04-23T02:13:52.674 INFO:tasks.workunit.client.1.smithi192.stdout:6/282: stat d3/d11/d44/f50 0 2024-04-23T02:13:52.674 INFO:tasks.workunit.client.0.smithi071.stdout:8/181: mknod dc/d15/c34 0 2024-04-23T02:13:52.676 INFO:tasks.workunit.client.1.smithi192.stdout:6/283: chown d3/d15 1 1 2024-04-23T02:13:52.679 INFO:tasks.workunit.client.0.smithi071.stdout:0/116: creat d0/d3/de/f28 x:0 0 0 2024-04-23T02:13:52.679 INFO:tasks.workunit.client.1.smithi192.stdout:6/284: stat d3/d15/d1b/c25 0 2024-04-23T02:13:52.680 INFO:tasks.workunit.client.0.smithi071.stdout:8/182: mknod dc/d10/d32/c35 0 2024-04-23T02:13:52.681 INFO:tasks.workunit.client.0.smithi071.stdout:0/117: dread - d0/d3/d1c/f21 zero size 2024-04-23T02:13:52.683 INFO:tasks.workunit.client.0.smithi071.stdout:8/183: symlink dc/d15/l36 0 2024-04-23T02:13:52.684 INFO:tasks.workunit.client.1.smithi192.stdout:6/285: mknod d3/d11/d44/c54 0 2024-04-23T02:13:52.684 INFO:tasks.workunit.client.1.smithi192.stdout:3/176: rmdir d1/d3 39 2024-04-23T02:13:52.684 INFO:tasks.workunit.client.0.smithi071.stdout:2/101: dwrite d0/f6 [0,4194304] 0 2024-04-23T02:13:52.688 INFO:tasks.workunit.client.1.smithi192.stdout:4/374: rmdir de/d15/d53 39 2024-04-23T02:13:52.689 INFO:tasks.workunit.client.0.smithi071.stdout:8/184: creat dc/f37 x:0 0 0 2024-04-23T02:13:52.690 INFO:tasks.workunit.client.0.smithi071.stdout:8/185: write dc/ff [11975,94067] 0 2024-04-23T02:13:52.690 INFO:tasks.workunit.client.0.smithi071.stdout:2/102: dread d0/f6 [0,4194304] 0 2024-04-23T02:13:52.691 INFO:tasks.workunit.client.0.smithi071.stdout:2/103: dread - d0/d2/dd/d14/f1a zero size 2024-04-23T02:13:52.693 INFO:tasks.workunit.client.1.smithi192.stdout:3/177: symlink d1/d3/d17/d2b/l37 0 2024-04-23T02:13:52.693 INFO:tasks.workunit.client.1.smithi192.stdout:4/375: creat de/d15/d19/d3d/d34/f76 x:0 0 0 2024-04-23T02:13:52.694 INFO:tasks.workunit.client.1.smithi192.stdout:6/286: link d3/c7 d3/d28/c55 0 2024-04-23T02:13:52.696 INFO:tasks.workunit.client.1.smithi192.stdout:4/376: read de/d15/d19/d3d/d49/f50 [2912436,121774] 0 2024-04-23T02:13:52.697 INFO:tasks.workunit.client.1.smithi192.stdout:6/287: creat d3/d28/f56 x:0 0 0 2024-04-23T02:13:52.700 INFO:tasks.workunit.client.1.smithi192.stdout:6/288: creat d3/f57 x:0 0 0 2024-04-23T02:13:52.700 INFO:tasks.workunit.client.1.smithi192.stdout:6/289: fsync f1 0 2024-04-23T02:13:52.701 INFO:tasks.workunit.client.1.smithi192.stdout:3/178: mkdir d1/d3/d17/d1f/d33/d24/d34/d38 0 2024-04-23T02:13:52.701 INFO:tasks.workunit.client.1.smithi192.stdout:6/290: read - d3/d28/f56 zero size 2024-04-23T02:13:52.702 INFO:tasks.workunit.client.1.smithi192.stdout:3/179: truncate d1/d3/d6/f2f 255818 0 2024-04-23T02:13:52.703 INFO:tasks.workunit.client.1.smithi192.stdout:3/180: fsync d1/f5 0 2024-04-23T02:13:52.703 INFO:tasks.workunit.client.1.smithi192.stdout:6/291: chown d3/d15/l4d 28049679 1 2024-04-23T02:13:52.704 INFO:tasks.workunit.client.1.smithi192.stdout:3/181: fsync d1/d3/d6/d7/f2a 0 2024-04-23T02:13:52.706 INFO:tasks.workunit.client.1.smithi192.stdout:6/292: rmdir d3/d15 39 2024-04-23T02:13:52.708 INFO:tasks.workunit.client.1.smithi192.stdout:3/182: symlink d1/d3/d17/l39 0 2024-04-23T02:13:52.710 INFO:tasks.workunit.client.1.smithi192.stdout:3/183: read d1/d3/d17/d1a/f23 [869294,66449] 0 2024-04-23T02:13:52.712 INFO:tasks.workunit.client.0.smithi071.stdout:0/118: write d0/da/f1a [4899742,32545] 0 2024-04-23T02:13:52.712 INFO:tasks.workunit.client.1.smithi192.stdout:3/184: dread - d1/d3/d6/d7/f31 zero size 2024-04-23T02:13:52.713 INFO:tasks.workunit.client.0.smithi071.stdout:2/104: truncate d0/d2/f9 3959454 0 2024-04-23T02:13:52.714 INFO:tasks.workunit.client.1.smithi192.stdout:4/377: dread de/d15/d19/d1a/f1e [0,4194304] 0 2024-04-23T02:13:52.714 INFO:tasks.workunit.client.0.smithi071.stdout:0/119: symlink d0/d3/de/d10/l29 0 2024-04-23T02:13:52.715 INFO:tasks.workunit.client.0.smithi071.stdout:2/105: write d0/d2/dd/d14/f19 [617066,117137] 0 2024-04-23T02:13:52.715 INFO:tasks.workunit.client.1.smithi192.stdout:6/293: dwrite d3/d11/f33 [4194304,4194304] 0 2024-04-23T02:13:52.716 INFO:tasks.workunit.client.1.smithi192.stdout:3/185: mknod d1/d3/d17/d2b/c3a 0 2024-04-23T02:13:52.717 INFO:tasks.workunit.client.0.smithi071.stdout:2/106: symlink d0/l1b 0 2024-04-23T02:13:52.717 INFO:tasks.workunit.client.0.smithi071.stdout:0/120: rmdir d0/d3/de/d10/d16 39 2024-04-23T02:13:52.717 INFO:tasks.workunit.client.0.smithi071.stdout:2/107: fdatasync d0/d2/dd/f18 0 2024-04-23T02:13:52.718 INFO:tasks.workunit.client.1.smithi192.stdout:4/378: rename de to de/d15/d53/d77 22 2024-04-23T02:13:52.718 INFO:tasks.workunit.client.1.smithi192.stdout:6/294: truncate d3/d11/f4a 4829718 0 2024-04-23T02:13:52.718 INFO:tasks.workunit.client.0.smithi071.stdout:0/121: fsync d0/d3/de/f1d 0 2024-04-23T02:13:52.719 INFO:tasks.workunit.client.0.smithi071.stdout:4/138: dread d2/d15/d1d/d23/ff [0,4194304] 0 2024-04-23T02:13:52.720 INFO:tasks.workunit.client.0.smithi071.stdout:2/108: creat d0/d2/dd/d14/f1c x:0 0 0 2024-04-23T02:13:52.720 INFO:tasks.workunit.client.1.smithi192.stdout:6/295: mkdir d3/d18/d58 0 2024-04-23T02:13:52.721 INFO:tasks.workunit.client.1.smithi192.stdout:4/379: chown de/c5d 51132 1 2024-04-23T02:13:52.722 INFO:tasks.workunit.client.0.smithi071.stdout:0/122: unlink d0/da/f1a 0 2024-04-23T02:13:52.724 INFO:tasks.workunit.client.0.smithi071.stdout:2/109: creat d0/d2/f1d x:0 0 0 2024-04-23T02:13:52.724 INFO:tasks.workunit.client.1.smithi192.stdout:3/186: dread d1/d3/d17/d1a/f23 [0,4194304] 0 2024-04-23T02:13:52.725 INFO:tasks.workunit.client.0.smithi071.stdout:2/110: fsync d0/d2/dd/d14/f1a 0 2024-04-23T02:13:52.725 INFO:tasks.workunit.client.1.smithi192.stdout:3/187: write d1/f5 [5342321,118847] 0 2024-04-23T02:13:52.726 INFO:tasks.workunit.client.1.smithi192.stdout:4/380: truncate de/d15/d19/d1a/f3a 560700 0 2024-04-23T02:13:52.726 INFO:tasks.workunit.client.0.smithi071.stdout:4/139: getdents d2/d2a 0 2024-04-23T02:13:52.727 INFO:tasks.workunit.client.1.smithi192.stdout:3/188: chown d1/d3/f19 382375 1 2024-04-23T02:13:52.727 INFO:tasks.workunit.client.1.smithi192.stdout:6/296: dread d3/d11/f13 [4194304,4194304] 0 2024-04-23T02:13:52.728 INFO:tasks.workunit.client.0.smithi071.stdout:2/111: rename d0/f6 to d0/f1e 0 2024-04-23T02:13:52.728 INFO:tasks.workunit.client.1.smithi192.stdout:3/189: chown d1/d3/d6/d7 1 1 2024-04-23T02:13:52.729 INFO:tasks.workunit.client.0.smithi071.stdout:4/140: creat d2/d15/d1d/d2c/f2e x:0 0 0 2024-04-23T02:13:52.731 INFO:tasks.workunit.client.1.smithi192.stdout:6/297: creat d3/d11/d44/f59 x:0 0 0 2024-04-23T02:13:52.732 INFO:tasks.workunit.client.1.smithi192.stdout:3/190: mkdir d1/d3/d6/d7/d29/d3b 0 2024-04-23T02:13:52.733 INFO:tasks.workunit.client.0.smithi071.stdout:4/141: getdents d2/d15 0 2024-04-23T02:13:52.733 INFO:tasks.workunit.client.1.smithi192.stdout:6/298: mknod d3/d28/c5a 0 2024-04-23T02:13:52.735 INFO:tasks.workunit.client.0.smithi071.stdout:4/142: creat d2/d15/d1d/d23/d29/f2f x:0 0 0 2024-04-23T02:13:52.736 INFO:tasks.workunit.client.0.smithi071.stdout:2/112: rmdir d0/d2/dd/d14 39 2024-04-23T02:13:52.737 INFO:tasks.workunit.client.0.smithi071.stdout:4/143: mknod d2/d15/c30 0 2024-04-23T02:13:52.738 INFO:tasks.workunit.client.1.smithi192.stdout:7/209: sync 2024-04-23T02:13:52.738 INFO:tasks.workunit.client.1.smithi192.stdout:8/233: sync 2024-04-23T02:13:52.738 INFO:tasks.workunit.client.0.smithi071.stdout:4/144: stat d2/d15/d1d/d2c/f1b 0 2024-04-23T02:13:52.740 INFO:tasks.workunit.client.1.smithi192.stdout:4/381: dread de/f21 [0,4194304] 0 2024-04-23T02:13:52.741 INFO:tasks.workunit.client.0.smithi071.stdout:2/113: mkdir d0/d2/dd/d14/d1f 0 2024-04-23T02:13:52.741 INFO:tasks.workunit.client.1.smithi192.stdout:6/299: dread d3/d11/f33 [4194304,4194304] 0 2024-04-23T02:13:52.744 INFO:tasks.workunit.client.1.smithi192.stdout:3/191: dwrite d1/d3/d6/d7/f31 [0,4194304] 0 2024-04-23T02:13:52.744 INFO:tasks.workunit.client.1.smithi192.stdout:4/382: symlink de/d15/d2a/d61/l78 0 2024-04-23T02:13:52.744 INFO:tasks.workunit.client.0.smithi071.stdout:0/123: dwrite d0/d3/de/d10/f1f [0,4194304] 0 2024-04-23T02:13:52.746 INFO:tasks.workunit.client.0.smithi071.stdout:2/114: creat d0/f20 x:0 0 0 2024-04-23T02:13:52.746 INFO:tasks.workunit.client.0.smithi071.stdout:2/115: stat d0/d2/f1d 0 2024-04-23T02:13:52.750 INFO:tasks.workunit.client.0.smithi071.stdout:2/116: write d0/d2/dd/fc [930490,66771] 0 2024-04-23T02:13:52.750 INFO:tasks.workunit.client.1.smithi192.stdout:8/234: link d1/df/d41/f43 d1/d12/d23/d4b/f51 0 2024-04-23T02:13:52.750 INFO:tasks.workunit.client.1.smithi192.stdout:3/192: rmdir d1/d3 39 2024-04-23T02:13:52.751 INFO:tasks.workunit.client.1.smithi192.stdout:4/383: mknod de/d15/d2a/d74/c79 0 2024-04-23T02:13:52.755 INFO:tasks.workunit.client.0.smithi071.stdout:2/117: dread d0/d2/fb [0,4194304] 0 2024-04-23T02:13:52.756 INFO:tasks.workunit.client.1.smithi192.stdout:8/235: creat d1/d12/d23/d2c/d4d/f52 x:0 0 0 2024-04-23T02:13:52.758 INFO:tasks.workunit.client.1.smithi192.stdout:4/384: write de/d15/d19/d1a/f6d [4400390,36019] 0 2024-04-23T02:13:52.759 INFO:tasks.workunit.client.0.smithi071.stdout:2/118: dread d0/f1e [0,4194304] 0 2024-04-23T02:13:52.760 INFO:tasks.workunit.client.1.smithi192.stdout:6/300: dwrite d3/f2b [4194304,4194304] 0 2024-04-23T02:13:52.760 INFO:tasks.workunit.client.0.smithi071.stdout:0/124: link d0/da/l19 d0/d3/de/d10/d16/d23/l2a 0 2024-04-23T02:13:52.761 INFO:tasks.workunit.client.0.smithi071.stdout:2/119: creat d0/d2/f21 x:0 0 0 2024-04-23T02:13:52.762 INFO:tasks.workunit.client.0.smithi071.stdout:0/125: fsync d0/f2 0 2024-04-23T02:13:52.762 INFO:tasks.workunit.client.1.smithi192.stdout:8/236: write d1/d12/f39 [294986,12311] 0 2024-04-23T02:13:52.763 INFO:tasks.workunit.client.1.smithi192.stdout:4/385: chown de/d15/d19/d3d/d49/d26/c6e 5645663 1 2024-04-23T02:13:52.764 INFO:tasks.workunit.client.1.smithi192.stdout:3/193: rename d1/d3/d17/d2b/l37 to d1/d3/d6/d7/d29/d3b/l3c 0 2024-04-23T02:13:52.765 INFO:tasks.workunit.client.0.smithi071.stdout:0/126: mkdir d0/d3/de/d10/d14/d2b 0 2024-04-23T02:13:52.766 INFO:tasks.workunit.client.0.smithi071.stdout:2/120: dread d0/d2/f9 [0,4194304] 0 2024-04-23T02:13:52.766 INFO:tasks.workunit.client.0.smithi071.stdout:0/127: chown d0/d3/de/d10/d14/d2b 6 1 2024-04-23T02:13:52.767 INFO:tasks.workunit.client.1.smithi192.stdout:3/194: dread - d1/d3/d6/d7/f2a zero size 2024-04-23T02:13:52.767 INFO:tasks.workunit.client.1.smithi192.stdout:6/301: mkdir d3/d15/d5b 0 2024-04-23T02:13:52.768 INFO:tasks.workunit.client.0.smithi071.stdout:2/121: chown d0/d2/c16 9515 1 2024-04-23T02:13:52.768 INFO:tasks.workunit.client.1.smithi192.stdout:4/386: mknod de/c7a 0 2024-04-23T02:13:52.769 INFO:tasks.workunit.client.1.smithi192.stdout:6/302: chown d3/d28/l3e 323 1 2024-04-23T02:13:52.771 INFO:tasks.workunit.client.1.smithi192.stdout:8/237: dread d1/d12/f24 [0,4194304] 0 2024-04-23T02:13:52.772 INFO:tasks.workunit.client.1.smithi192.stdout:4/387: dread - de/d15/d19/d3d/f54 zero size 2024-04-23T02:13:52.772 INFO:tasks.workunit.client.1.smithi192.stdout:6/303: write f1 [1895880,81365] 0 2024-04-23T02:13:52.779 INFO:tasks.workunit.client.1.smithi192.stdout:6/304: creat d3/f5c x:0 0 0 2024-04-23T02:13:52.779 INFO:tasks.workunit.client.0.smithi071.stdout:0/128: rename d0/da to d0/d3/de/d2c 0 2024-04-23T02:13:52.780 INFO:tasks.workunit.client.1.smithi192.stdout:4/388: link de/c35 de/d15/d19/d3d/d49/d26/c7b 0 2024-04-23T02:13:52.780 INFO:tasks.workunit.client.0.smithi071.stdout:2/122: truncate d0/d2/fb 4167887 0 2024-04-23T02:13:52.781 INFO:tasks.workunit.client.1.smithi192.stdout:8/238: dread d1/df/f1c [0,4194304] 0 2024-04-23T02:13:52.781 INFO:tasks.workunit.client.1.smithi192.stdout:6/305: symlink d3/d15/d1b/d21/d3a/l5d 0 2024-04-23T02:13:52.782 INFO:tasks.workunit.client.0.smithi071.stdout:2/123: creat d0/d2/dd/d14/f22 x:0 0 0 2024-04-23T02:13:52.783 INFO:tasks.workunit.client.1.smithi192.stdout:6/306: write d3/d11/d44/f59 [940825,106011] 0 2024-04-23T02:13:52.783 INFO:tasks.workunit.client.0.smithi071.stdout:2/124: dread - d0/d2/dd/f18 zero size 2024-04-23T02:13:52.783 INFO:tasks.workunit.client.1.smithi192.stdout:3/195: truncate d1/d3/d17/f27 1620466 0 2024-04-23T02:13:52.786 INFO:tasks.workunit.client.1.smithi192.stdout:8/239: rmdir d1/df/d41 39 2024-04-23T02:13:52.786 INFO:tasks.workunit.client.1.smithi192.stdout:6/307: dread d3/d11/d44/f50 [0,4194304] 0 2024-04-23T02:13:52.787 INFO:tasks.workunit.client.0.smithi071.stdout:2/125: creat d0/d2/dd/d14/f23 x:0 0 0 2024-04-23T02:13:52.789 INFO:tasks.workunit.client.0.smithi071.stdout:0/129: getdents d0/d3/de/d2c 0 2024-04-23T02:13:52.790 INFO:tasks.workunit.client.0.smithi071.stdout:0/130: truncate d0/f20 252708 0 2024-04-23T02:13:52.791 INFO:tasks.workunit.client.1.smithi192.stdout:6/308: dread d3/d11/f13 [0,4194304] 0 2024-04-23T02:13:52.793 INFO:tasks.workunit.client.1.smithi192.stdout:3/196: write d1/d3/d17/d1a/f23 [3647531,62947] 0 2024-04-23T02:13:52.794 INFO:tasks.workunit.client.1.smithi192.stdout:6/309: dread - d3/f57 zero size 2024-04-23T02:13:52.796 INFO:tasks.workunit.client.1.smithi192.stdout:3/197: write d1/d3/d17/f2d [51959,59576] 0 2024-04-23T02:13:52.797 INFO:tasks.workunit.client.0.smithi071.stdout:2/126: truncate d0/d2/f9 1259695 0 2024-04-23T02:13:52.798 INFO:tasks.workunit.client.0.smithi071.stdout:0/131: dwrite d0/d3/d1c/f22 [0,4194304] 0 2024-04-23T02:13:52.801 INFO:tasks.workunit.client.1.smithi192.stdout:6/310: dread d3/f22 [0,4194304] 0 2024-04-23T02:13:52.802 INFO:tasks.workunit.client.0.smithi071.stdout:0/132: rename d0/d3/de/f13 to d0/d3/f2d 0 2024-04-23T02:13:52.803 INFO:tasks.workunit.client.0.smithi071.stdout:2/127: getdents d0/d2/dd 0 2024-04-23T02:13:52.805 INFO:tasks.workunit.client.1.smithi192.stdout:3/198: dwrite d1/d3/d17/d1a/f23 [0,4194304] 0 2024-04-23T02:13:52.806 INFO:tasks.workunit.client.1.smithi192.stdout:6/311: creat d3/d15/d5b/f5e x:0 0 0 2024-04-23T02:13:52.811 INFO:tasks.workunit.client.1.smithi192.stdout:3/199: creat d1/d3/d17/f3d x:0 0 0 2024-04-23T02:13:52.812 INFO:tasks.workunit.client.1.smithi192.stdout:6/312: getdents d3/d15/d1b 0 2024-04-23T02:13:52.812 INFO:tasks.workunit.client.0.smithi071.stdout:2/128: dwrite d0/f1e [0,4194304] 0 2024-04-23T02:13:52.813 INFO:tasks.workunit.client.1.smithi192.stdout:3/200: write d1/d3/d17/d2b/f32 [217426,73631] 0 2024-04-23T02:13:52.816 INFO:tasks.workunit.client.0.smithi071.stdout:2/129: write d0/d2/dd/d14/f1c [449890,129459] 0 2024-04-23T02:13:52.816 INFO:tasks.workunit.client.1.smithi192.stdout:6/313: mkdir d3/d15/d1b/d21/d3a/d5f 0 2024-04-23T02:13:52.816 INFO:tasks.workunit.client.1.smithi192.stdout:3/201: write d1/d3/d17/d1a/f23 [4644256,97931] 0 2024-04-23T02:13:52.817 INFO:tasks.workunit.client.1.smithi192.stdout:6/314: truncate d3/f57 386942 0 2024-04-23T02:13:52.818 INFO:tasks.workunit.client.1.smithi192.stdout:8/240: dread d1/d12/d23/f3c [0,4194304] 0 2024-04-23T02:13:52.819 INFO:tasks.workunit.client.1.smithi192.stdout:6/315: fdatasync d3/d15/d1b/d21/d3a/f4e 0 2024-04-23T02:13:52.819 INFO:tasks.workunit.client.1.smithi192.stdout:8/241: chown d1/f15 190622 1 2024-04-23T02:13:52.819 INFO:tasks.workunit.client.1.smithi192.stdout:3/202: mknod d1/d3/d6/d7/d29/c3e 0 2024-04-23T02:13:52.821 INFO:tasks.workunit.client.0.smithi071.stdout:0/133: truncate d0/d3/de/f28 709858 0 2024-04-23T02:13:52.821 INFO:tasks.workunit.client.1.smithi192.stdout:8/242: chown d1/d9/cc 1123504966 1 2024-04-23T02:13:52.822 INFO:tasks.workunit.client.1.smithi192.stdout:8/243: dread - d1/d9/f26 zero size 2024-04-23T02:13:52.822 INFO:tasks.workunit.client.1.smithi192.stdout:6/316: rmdir d3/d11/d44 39 2024-04-23T02:13:52.824 INFO:tasks.workunit.client.0.smithi071.stdout:0/134: creat d0/d3/de/d2c/d26/f2e x:0 0 0 2024-04-23T02:13:52.825 INFO:tasks.workunit.client.0.smithi071.stdout:0/135: fsync d0/d3/d1c/f22 0 2024-04-23T02:13:52.826 INFO:tasks.workunit.client.1.smithi192.stdout:3/203: mknod d1/d3/d17/d1f/d33/d24/d34/d38/c3f 0 2024-04-23T02:13:52.828 INFO:tasks.workunit.client.1.smithi192.stdout:6/317: creat d3/d15/d1b/d21/f60 x:0 0 0 2024-04-23T02:13:52.829 INFO:tasks.workunit.client.1.smithi192.stdout:6/318: fsync d3/f5c 0 2024-04-23T02:13:52.830 INFO:tasks.workunit.client.1.smithi192.stdout:6/319: truncate d3/d28/f56 366331 0 2024-04-23T02:13:52.831 INFO:tasks.workunit.client.1.smithi192.stdout:3/204: mkdir d1/d3/d17/d1f/d33/d24/d34/d40 0 2024-04-23T02:13:52.833 INFO:tasks.workunit.client.1.smithi192.stdout:6/320: dread d3/d11/d44/f50 [0,4194304] 0 2024-04-23T02:13:52.834 INFO:tasks.workunit.client.1.smithi192.stdout:6/321: chown d3/d15/d1b/c24 3571621 1 2024-04-23T02:13:52.834 INFO:tasks.workunit.client.1.smithi192.stdout:6/322: chown d3/d11/f43 3 1 2024-04-23T02:13:52.835 INFO:tasks.workunit.client.1.smithi192.stdout:3/205: rename d1/d3/d6/f2f to d1/d3/d6/d7/d29/d3b/f41 0 2024-04-23T02:13:52.839 INFO:tasks.workunit.client.1.smithi192.stdout:6/323: rmdir d3/d15/d1b/d21/d3a/d5f 0 2024-04-23T02:13:52.840 INFO:tasks.workunit.client.1.smithi192.stdout:8/244: write d1/df/f1c [815433,66249] 0 2024-04-23T02:13:52.841 INFO:tasks.workunit.client.1.smithi192.stdout:6/324: stat d3/d28/f2e 0 2024-04-23T02:13:52.843 INFO:tasks.workunit.client.1.smithi192.stdout:3/206: dwrite d1/d3/d17/d1a/f23 [0,4194304] 0 2024-04-23T02:13:52.843 INFO:tasks.workunit.client.1.smithi192.stdout:8/245: chown d1/d12/d23/d2c/c3b 129032 1 2024-04-23T02:13:52.844 INFO:tasks.workunit.client.1.smithi192.stdout:3/207: readlink d1/d3/d17/d1f/d33/l28 0 2024-04-23T02:13:52.847 INFO:tasks.workunit.client.1.smithi192.stdout:6/325: link d3/f2b d3/d15/d1b/d21/d3a/f61 0 2024-04-23T02:13:52.848 INFO:tasks.workunit.client.1.smithi192.stdout:8/246: creat d1/df/d41/d45/d46/f53 x:0 0 0 2024-04-23T02:13:52.849 INFO:tasks.workunit.client.0.smithi071.stdout:1/65: sync 2024-04-23T02:13:52.849 INFO:tasks.workunit.client.0.smithi071.stdout:7/118: sync 2024-04-23T02:13:52.850 INFO:tasks.workunit.client.1.smithi192.stdout:8/247: write d1/f3 [3441453,109549] 0 2024-04-23T02:13:52.850 INFO:tasks.workunit.client.1.smithi192.stdout:3/208: rename d1/d3/d17/d2b/c3a to d1/d3/d17/d1f/c42 0 2024-04-23T02:13:52.852 INFO:tasks.workunit.client.0.smithi071.stdout:7/119: unlink d3/d17/d1c/f2b 0 2024-04-23T02:13:52.853 INFO:tasks.workunit.client.1.smithi192.stdout:8/248: unlink d1/d12/d23/d2c/c2f 0 2024-04-23T02:13:52.853 INFO:tasks.workunit.client.1.smithi192.stdout:6/326: getdents d3/d15/d5b 0 2024-04-23T02:13:52.855 INFO:tasks.workunit.client.0.smithi071.stdout:7/120: symlink d3/d17/d1c/l2e 0 2024-04-23T02:13:52.856 INFO:tasks.workunit.client.1.smithi192.stdout:6/327: symlink d3/d15/d5b/l62 0 2024-04-23T02:13:52.858 INFO:tasks.workunit.client.1.smithi192.stdout:8/249: symlink d1/df/d41/d45/l54 0 2024-04-23T02:13:52.860 INFO:tasks.workunit.client.1.smithi192.stdout:3/209: link d1/d3/d17/d1f/d33/l28 d1/l43 0 2024-04-23T02:13:52.861 INFO:tasks.workunit.client.1.smithi192.stdout:6/328: dread d3/d11/f13 [4194304,4194304] 0 2024-04-23T02:13:52.863 INFO:tasks.workunit.client.1.smithi192.stdout:6/329: symlink d3/d15/d1b/d21/l63 0 2024-04-23T02:13:52.864 INFO:tasks.workunit.client.1.smithi192.stdout:3/210: dread d1/f5 [0,4194304] 0 2024-04-23T02:13:52.865 INFO:tasks.workunit.client.1.smithi192.stdout:6/330: mkdir d3/d15/d1b/d21/d52/d64 0 2024-04-23T02:13:52.866 INFO:tasks.workunit.client.1.smithi192.stdout:3/211: mkdir d1/d3/d17/d2b/d44 0 2024-04-23T02:13:52.867 INFO:tasks.workunit.client.1.smithi192.stdout:6/331: creat d3/d15/d1b/d21/d52/d64/f65 x:0 0 0 2024-04-23T02:13:52.868 INFO:tasks.workunit.client.1.smithi192.stdout:3/212: creat d1/d3/d17/d1f/d33/d24/f45 x:0 0 0 2024-04-23T02:13:52.869 INFO:tasks.workunit.client.1.smithi192.stdout:6/332: mkdir d3/d15/d1b/d66 0 2024-04-23T02:13:52.869 INFO:tasks.workunit.client.1.smithi192.stdout:6/333: fdatasync d3/f5c 0 2024-04-23T02:13:52.871 INFO:tasks.workunit.client.1.smithi192.stdout:6/334: write d3/d11/f43 [977653,68258] 0 2024-04-23T02:13:52.875 INFO:tasks.workunit.client.1.smithi192.stdout:6/335: unlink d3/d28/c55 0 2024-04-23T02:13:52.876 INFO:tasks.workunit.client.1.smithi192.stdout:6/336: chown d3/d11/d44/f59 30367560 1 2024-04-23T02:13:52.877 INFO:tasks.workunit.client.1.smithi192.stdout:6/337: dread d3/d11/d44/f50 [0,4194304] 0 2024-04-23T02:13:52.880 INFO:tasks.workunit.client.1.smithi192.stdout:3/213: write d1/d3/d17/f27 [87272,71093] 0 2024-04-23T02:13:52.880 INFO:tasks.workunit.client.1.smithi192.stdout:6/338: chown d3/d15/d1b/d21/d52 7592370 1 2024-04-23T02:13:52.970 INFO:tasks.workunit.client.1.smithi192.stdout:6/339: creat d3/d15/d1b/d21/d52/f67 x:0 0 0 2024-04-23T02:13:52.971 INFO:tasks.workunit.client.1.smithi192.stdout:3/214: mkdir d1/d3/d17/d1f/d33/d24/d34/d40/d46 0 2024-04-23T02:13:52.976 INFO:tasks.workunit.client.1.smithi192.stdout:3/215: write d1/d3/d17/f27 [1412209,95706] 0 2024-04-23T02:13:52.977 INFO:tasks.workunit.client.1.smithi192.stdout:3/216: stat d1/d3/d6/d7/d29/d3b 0 2024-04-23T02:13:52.984 INFO:tasks.workunit.client.1.smithi192.stdout:6/340: dwrite d3/d18/f2d [0,4194304] 0 2024-04-23T02:13:52.984 INFO:tasks.workunit.client.1.smithi192.stdout:3/217: dread d1/d3/d6/f10 [0,4194304] 0 2024-04-23T02:13:52.985 INFO:tasks.workunit.client.1.smithi192.stdout:6/341: dread - d3/f49 zero size 2024-04-23T02:13:52.989 INFO:tasks.workunit.client.1.smithi192.stdout:6/342: symlink d3/d15/d1b/d21/d3a/l68 0 2024-04-23T02:13:52.992 INFO:tasks.workunit.client.1.smithi192.stdout:6/343: creat d3/d15/d5b/f69 x:0 0 0 2024-04-23T02:13:52.992 INFO:tasks.workunit.client.1.smithi192.stdout:3/218: link d1/d3/d17/l35 d1/d3/d17/d2b/l47 0 2024-04-23T02:13:52.992 INFO:tasks.workunit.client.1.smithi192.stdout:6/344: chown d3/d11/f13 63849 1 2024-04-23T02:13:52.993 INFO:tasks.workunit.client.1.smithi192.stdout:3/219: rmdir d1/d3 39 2024-04-23T02:13:52.994 INFO:tasks.workunit.client.1.smithi192.stdout:6/345: rmdir d3/d15/d1b 39 2024-04-23T02:13:52.995 INFO:tasks.workunit.client.1.smithi192.stdout:6/346: truncate d3/d28/f56 1042867 0 2024-04-23T02:13:52.996 INFO:tasks.workunit.client.1.smithi192.stdout:3/220: fdatasync d1/d3/d6/d7/f2a 0 2024-04-23T02:13:52.999 INFO:tasks.workunit.client.1.smithi192.stdout:3/221: truncate d1/d3/d17/d1f/d33/d24/f45 178574 0 2024-04-23T02:13:53.002 INFO:tasks.workunit.client.1.smithi192.stdout:6/347: dwrite d3/f57 [0,4194304] 0 2024-04-23T02:13:53.003 INFO:tasks.workunit.client.1.smithi192.stdout:3/222: creat d1/d3/d17/d1f/d33/d24/d34/d38/f48 x:0 0 0 2024-04-23T02:13:53.004 INFO:tasks.workunit.client.1.smithi192.stdout:3/223: stat d1/d3/d17/c1e 0 2024-04-23T02:13:53.004 INFO:tasks.workunit.client.1.smithi192.stdout:6/348: write d3/d11/f13 [2181097,123908] 0 2024-04-23T02:13:53.008 INFO:tasks.workunit.client.1.smithi192.stdout:3/224: creat d1/d3/f49 x:0 0 0 2024-04-23T02:13:53.010 INFO:tasks.workunit.client.0.smithi071.stdout:9/133: sync 2024-04-23T02:13:53.010 INFO:tasks.workunit.client.0.smithi071.stdout:3/73: sync 2024-04-23T02:13:53.010 INFO:tasks.workunit.client.0.smithi071.stdout:6/78: sync 2024-04-23T02:13:53.010 INFO:tasks.workunit.client.0.smithi071.stdout:5/164: sync 2024-04-23T02:13:53.015 INFO:tasks.workunit.client.1.smithi192.stdout:3/225: rename d1/d3/d17/c21 to d1/d3/d17/d2b/d44/c4a 0 2024-04-23T02:13:53.015 INFO:tasks.workunit.client.1.smithi192.stdout:6/349: dwrite f1 [0,4194304] 0 2024-04-23T02:13:53.015 INFO:tasks.workunit.client.1.smithi192.stdout:3/226: write d1/d3/d17/f2d [86491,18274] 0 2024-04-23T02:13:53.016 INFO:tasks.workunit.client.0.smithi071.stdout:3/74: chown d7/d8 4 1 2024-04-23T02:13:53.016 INFO:tasks.workunit.client.0.smithi071.stdout:5/165: creat d3/d34/f40 x:0 0 0 2024-04-23T02:13:53.017 INFO:tasks.workunit.client.0.smithi071.stdout:6/79: fsync f1 0 2024-04-23T02:13:53.017 INFO:tasks.workunit.client.0.smithi071.stdout:9/134: dread d1/fa [0,4194304] 0 2024-04-23T02:13:53.017 INFO:tasks.workunit.client.0.smithi071.stdout:5/166: write d3/d1d/f3b [451741,47901] 0 2024-04-23T02:13:53.017 INFO:tasks.workunit.client.0.smithi071.stdout:3/75: mknod d7/c17 0 2024-04-23T02:13:53.018 INFO:tasks.workunit.client.1.smithi192.stdout:6/350: readlink d3/d15/d1b/d21/l2f 0 2024-04-23T02:13:53.018 INFO:tasks.workunit.client.0.smithi071.stdout:6/80: mknod d6/c19 0 2024-04-23T02:13:53.018 INFO:tasks.workunit.client.0.smithi071.stdout:3/76: symlink d7/d8/l18 0 2024-04-23T02:13:53.019 INFO:tasks.workunit.client.0.smithi071.stdout:3/77: stat l4 0 2024-04-23T02:13:53.020 INFO:tasks.workunit.client.0.smithi071.stdout:5/167: creat d3/da/d1a/d1c/f41 x:0 0 0 2024-04-23T02:13:53.023 INFO:tasks.workunit.client.1.smithi192.stdout:6/351: dread d3/f34 [0,4194304] 0 2024-04-23T02:13:53.023 INFO:tasks.workunit.client.1.smithi192.stdout:3/227: dwrite d1/d3/d6/f10 [0,4194304] 0 2024-04-23T02:13:53.024 INFO:tasks.workunit.client.0.smithi071.stdout:9/135: dwrite d1/f4 [0,4194304] 0 2024-04-23T02:13:53.024 INFO:tasks.workunit.client.1.smithi192.stdout:3/228: dread - d1/d3/d6/d7/f2a zero size 2024-04-23T02:13:53.027 INFO:tasks.workunit.client.0.smithi071.stdout:9/136: mknod d1/d1a/c2c 0 2024-04-23T02:13:53.028 INFO:tasks.workunit.client.0.smithi071.stdout:9/137: dread - d1/f18 zero size 2024-04-23T02:13:53.031 INFO:tasks.workunit.client.1.smithi192.stdout:6/352: rename d3/d15/l4c to d3/d11/l6a 0 2024-04-23T02:13:53.032 INFO:tasks.workunit.client.0.smithi071.stdout:9/138: dread d1/fe [0,4194304] 0 2024-04-23T02:13:53.032 INFO:tasks.workunit.client.1.smithi192.stdout:6/353: fdatasync d3/d28/f56 0 2024-04-23T02:13:53.033 INFO:tasks.workunit.client.0.smithi071.stdout:9/139: dread - d1/d1a/f24 zero size 2024-04-23T02:13:53.033 INFO:tasks.workunit.client.0.smithi071.stdout:3/78: write d7/fb [76627,68979] 0 2024-04-23T02:13:53.036 INFO:tasks.workunit.client.1.smithi192.stdout:3/229: dread d1/d3/d17/d1a/f23 [0,4194304] 0 2024-04-23T02:13:53.037 INFO:tasks.workunit.client.1.smithi192.stdout:6/354: creat d3/d11/f6b x:0 0 0 2024-04-23T02:13:53.038 INFO:tasks.workunit.client.1.smithi192.stdout:3/230: dread d1/d3/d17/d1f/d33/d24/f45 [0,4194304] 0 2024-04-23T02:13:53.039 INFO:tasks.workunit.client.0.smithi071.stdout:6/81: dwrite d6/ff [0,4194304] 0 2024-04-23T02:13:53.040 INFO:tasks.workunit.client.1.smithi192.stdout:3/231: dread - d1/d3/d6/d7/f2a zero size 2024-04-23T02:13:53.040 INFO:tasks.workunit.client.0.smithi071.stdout:5/168: dread d3/df/f24 [0,4194304] 0 2024-04-23T02:13:53.040 INFO:tasks.workunit.client.1.smithi192.stdout:6/355: mkdir d3/d6c 0 2024-04-23T02:13:53.041 INFO:tasks.workunit.client.0.smithi071.stdout:3/79: link l4 d7/l19 0 2024-04-23T02:13:53.042 INFO:tasks.workunit.client.0.smithi071.stdout:6/82: fsync d6/f8 0 2024-04-23T02:13:53.043 INFO:tasks.workunit.client.0.smithi071.stdout:3/80: chown d7/fb 2 1 2024-04-23T02:13:53.046 INFO:tasks.workunit.client.1.smithi192.stdout:6/356: rename d3/d11/d44 to d3/d11/d6d 0 2024-04-23T02:13:53.047 INFO:tasks.workunit.client.0.smithi071.stdout:3/81: dread d7/fb [0,4194304] 0 2024-04-23T02:13:53.047 INFO:tasks.workunit.client.0.smithi071.stdout:6/83: getdents d6/d17 0 2024-04-23T02:13:53.049 INFO:tasks.workunit.client.0.smithi071.stdout:5/169: dwrite d3/d34/f40 [0,4194304] 0 2024-04-23T02:13:53.050 INFO:tasks.workunit.client.1.smithi192.stdout:3/232: dwrite d1/d3/f49 [0,4194304] 0 2024-04-23T02:13:53.050 INFO:tasks.workunit.client.1.smithi192.stdout:6/357: symlink d3/d15/d1b/d21/l6e 0 2024-04-23T02:13:53.052 INFO:tasks.workunit.client.1.smithi192.stdout:3/233: stat d1/d3/d17/d1a/c36 0 2024-04-23T02:13:53.052 INFO:tasks.workunit.client.1.smithi192.stdout:1/66: sync 2024-04-23T02:13:53.052 INFO:tasks.workunit.client.1.smithi192.stdout:5/217: sync 2024-04-23T02:13:53.052 INFO:tasks.workunit.client.1.smithi192.stdout:2/223: sync 2024-04-23T02:13:53.052 INFO:tasks.workunit.client.0.smithi071.stdout:9/140: write d1/f10 [910524,116387] 0 2024-04-23T02:13:53.053 INFO:tasks.workunit.client.1.smithi192.stdout:6/358: creat d3/d18/f6f x:0 0 0 2024-04-23T02:13:53.055 INFO:tasks.workunit.client.0.smithi071.stdout:5/170: readlink d3/lc 0 2024-04-23T02:13:53.058 INFO:tasks.workunit.client.0.smithi071.stdout:9/141: write d1/f10 [1548313,49503] 0 2024-04-23T02:13:53.058 INFO:tasks.workunit.client.1.smithi192.stdout:2/224: read d8/d14/f15 [2255878,102383] 0 2024-04-23T02:13:53.060 INFO:tasks.workunit.client.0.smithi071.stdout:9/142: readlink d1/ld 0 2024-04-23T02:13:53.063 INFO:tasks.workunit.client.0.smithi071.stdout:3/82: dwrite d7/fd [0,4194304] 0 2024-04-23T02:13:53.063 INFO:tasks.workunit.client.1.smithi192.stdout:3/234: rename d1/d3/d6/l12 to d1/d3/d6/d7/d29/l4b 0 2024-04-23T02:13:53.063 INFO:tasks.workunit.client.1.smithi192.stdout:2/225: truncate d8/d14/f1a 1851437 0 2024-04-23T02:13:53.065 INFO:tasks.workunit.client.0.smithi071.stdout:6/84: dwrite d6/f14 [0,4194304] 0 2024-04-23T02:13:53.065 INFO:tasks.workunit.client.0.smithi071.stdout:6/85: stat d6/f18 0 2024-04-23T02:13:53.065 INFO:tasks.workunit.client.0.smithi071.stdout:3/83: chown d7/d8/fc 0 1 2024-04-23T02:13:53.066 INFO:tasks.workunit.client.0.smithi071.stdout:9/143: write d1/fa [639380,42925] 0 2024-04-23T02:13:53.067 INFO:tasks.workunit.client.0.smithi071.stdout:3/84: dread - f5 zero size 2024-04-23T02:13:53.067 INFO:tasks.workunit.client.0.smithi071.stdout:6/86: write d6/f14 [1243292,21710] 0 2024-04-23T02:13:53.068 INFO:tasks.workunit.client.1.smithi192.stdout:5/218: creat d2/db/d16/f4a x:0 0 0 2024-04-23T02:13:53.068 INFO:tasks.workunit.client.1.smithi192.stdout:1/67: link c5 dc/c12 0 2024-04-23T02:13:53.068 INFO:tasks.workunit.client.1.smithi192.stdout:6/359: dwrite d3/d18/f6f [0,4194304] 0 2024-04-23T02:13:53.069 INFO:tasks.workunit.client.0.smithi071.stdout:8/186: sync 2024-04-23T02:13:53.069 INFO:tasks.workunit.client.0.smithi071.stdout:4/145: sync 2024-04-23T02:13:53.069 INFO:tasks.workunit.client.0.smithi071.stdout:2/130: sync 2024-04-23T02:13:53.070 INFO:tasks.workunit.client.0.smithi071.stdout:1/66: sync 2024-04-23T02:13:53.070 INFO:tasks.workunit.client.0.smithi071.stdout:0/136: sync 2024-04-23T02:13:53.073 INFO:tasks.workunit.client.0.smithi071.stdout:7/121: sync 2024-04-23T02:13:53.073 INFO:tasks.workunit.client.0.smithi071.stdout:5/171: getdents d3/d1d 0 2024-04-23T02:13:53.074 INFO:tasks.workunit.client.1.smithi192.stdout:3/235: dread d1/d3/d17/d2b/f32 [0,4194304] 0 2024-04-23T02:13:53.074 INFO:tasks.workunit.client.0.smithi071.stdout:8/187: truncate dc/d15/f23 17164 0 2024-04-23T02:13:53.075 INFO:tasks.workunit.client.0.smithi071.stdout:4/146: chown d2/d15/d1d/d23/f14 8667219 1 2024-04-23T02:13:53.075 INFO:tasks.workunit.client.0.smithi071.stdout:1/67: unlink f6 0 2024-04-23T02:13:53.075 INFO:tasks.workunit.client.0.smithi071.stdout:6/87: getdents d6/d17 0 2024-04-23T02:13:53.076 INFO:tasks.workunit.client.1.smithi192.stdout:2/226: rename d8/d26/d27/c35 to d8/d14/c48 0 2024-04-23T02:13:53.078 INFO:tasks.workunit.client.1.smithi192.stdout:2/227: chown d8 46831579 1 2024-04-23T02:13:53.078 INFO:tasks.workunit.client.1.smithi192.stdout:6/360: symlink d3/d11/d6d/l70 0 2024-04-23T02:13:53.078 INFO:tasks.workunit.client.1.smithi192.stdout:5/219: mkdir d2/d6/d2d/d41/d43/d4b 0 2024-04-23T02:13:53.079 INFO:tasks.workunit.client.0.smithi071.stdout:1/68: readlink l8 0 2024-04-23T02:13:53.080 INFO:tasks.workunit.client.1.smithi192.stdout:3/236: mkdir d1/d3/d17/d1f/d33/d24/d34/d40/d4c 0 2024-04-23T02:13:53.080 INFO:tasks.workunit.client.0.smithi071.stdout:6/88: chown d6/f9 2370 1 2024-04-23T02:13:53.081 INFO:tasks.workunit.client.1.smithi192.stdout:6/361: write d3/d28/f56 [724273,55232] 0 2024-04-23T02:13:53.083 INFO:tasks.workunit.client.0.smithi071.stdout:5/172: dread d3/df/f24 [0,4194304] 0 2024-04-23T02:13:53.083 INFO:tasks.workunit.client.1.smithi192.stdout:2/228: symlink d8/d14/l49 0 2024-04-23T02:13:53.084 INFO:tasks.workunit.client.0.smithi071.stdout:3/85: link d7/d8/fc d7/d8/f1a 0 2024-04-23T02:13:53.085 INFO:tasks.workunit.client.1.smithi192.stdout:5/220: truncate d2/f27 265151 0 2024-04-23T02:13:53.085 INFO:tasks.workunit.client.0.smithi071.stdout:2/131: mknod d0/d2/dd/d14/d1f/c24 0 2024-04-23T02:13:53.085 INFO:tasks.workunit.client.1.smithi192.stdout:1/68: dwrite f0 [4194304,4194304] 0 2024-04-23T02:13:53.087 INFO:tasks.workunit.client.0.smithi071.stdout:1/69: mkdir db/df/d13 0 2024-04-23T02:13:53.089 INFO:tasks.workunit.client.0.smithi071.stdout:6/89: write f5 [1676469,64112] 0 2024-04-23T02:13:53.089 INFO:tasks.workunit.client.1.smithi192.stdout:3/237: symlink d1/d3/d17/d1f/l4d 0 2024-04-23T02:13:53.090 INFO:tasks.workunit.client.0.smithi071.stdout:3/86: creat d7/d8/f1b x:0 0 0 2024-04-23T02:13:53.090 INFO:tasks.workunit.client.1.smithi192.stdout:1/69: creat dc/f13 x:0 0 0 2024-04-23T02:13:53.091 INFO:tasks.workunit.client.0.smithi071.stdout:0/137: getdents d0/d3/de/d2c/d26 0 2024-04-23T02:13:53.092 INFO:tasks.workunit.client.0.smithi071.stdout:9/144: dwrite d1/d1a/f25 [0,4194304] 0 2024-04-23T02:13:53.093 INFO:tasks.workunit.client.0.smithi071.stdout:1/70: write fa [126337,35607] 0 2024-04-23T02:13:53.093 INFO:tasks.workunit.client.1.smithi192.stdout:6/362: truncate d3/d11/f1d 275795 0 2024-04-23T02:13:53.094 INFO:tasks.workunit.client.1.smithi192.stdout:6/363: stat d3/le 0 2024-04-23T02:13:53.095 INFO:tasks.workunit.client.1.smithi192.stdout:6/364: dread - d3/d15/d5b/f5e zero size 2024-04-23T02:13:53.096 INFO:tasks.workunit.client.1.smithi192.stdout:5/221: mknod d2/d32/c4c 0 2024-04-23T02:13:53.096 INFO:tasks.workunit.client.1.smithi192.stdout:1/70: mkdir dc/d14 0 2024-04-23T02:13:53.097 INFO:tasks.workunit.client.1.smithi192.stdout:6/365: readlink d3/d15/d1b/d21/l2f 0 2024-04-23T02:13:53.098 INFO:tasks.workunit.client.1.smithi192.stdout:2/229: link d8/d26/d27/l39 d8/d26/d29/l4a 0 2024-04-23T02:13:53.098 INFO:tasks.workunit.client.0.smithi071.stdout:9/145: stat d1/ld 0 2024-04-23T02:13:53.100 INFO:tasks.workunit.client.0.smithi071.stdout:8/188: dwrite dc/ff [0,4194304] 0 2024-04-23T02:13:53.101 INFO:tasks.workunit.client.1.smithi192.stdout:5/222: readlink d2/db/l1e 0 2024-04-23T02:13:53.101 INFO:tasks.workunit.client.1.smithi192.stdout:6/366: creat d3/d6c/f71 x:0 0 0 2024-04-23T02:13:53.102 INFO:tasks.workunit.client.1.smithi192.stdout:6/367: readlink d3/d11/l42 0 2024-04-23T02:13:53.103 INFO:tasks.workunit.client.1.smithi192.stdout:5/223: write d2/db/d16/d1b/f1f [4470783,50921] 0 2024-04-23T02:13:53.103 INFO:tasks.workunit.client.1.smithi192.stdout:3/238: dread d1/d3/d6/d7/f31 [0,4194304] 0 2024-04-23T02:13:53.104 INFO:tasks.workunit.client.1.smithi192.stdout:5/224: chown d2/db/d46 96 1 2024-04-23T02:13:53.105 INFO:tasks.workunit.client.0.smithi071.stdout:9/146: chown d1/f28 62761 1 2024-04-23T02:13:53.105 INFO:tasks.workunit.client.0.smithi071.stdout:7/122: write d3/ff [621523,73235] 0 2024-04-23T02:13:53.105 INFO:tasks.workunit.client.1.smithi192.stdout:6/368: write d3/d11/f29 [4167233,44527] 0 2024-04-23T02:13:53.105 INFO:tasks.workunit.client.1.smithi192.stdout:3/239: read d1/f5 [7908108,61721] 0 2024-04-23T02:13:53.110 INFO:tasks.workunit.client.0.smithi071.stdout:1/71: getdents db/df/d13 0 2024-04-23T02:13:53.111 INFO:tasks.workunit.client.1.smithi192.stdout:5/225: rmdir d2/d6/d2d/d41/d43 39 2024-04-23T02:13:53.112 INFO:tasks.workunit.client.0.smithi071.stdout:9/147: creat d1/d1a/f2d x:0 0 0 2024-04-23T02:13:53.114 INFO:tasks.workunit.client.0.smithi071.stdout:7/123: read d3/fd [177471,65724] 0 2024-04-23T02:13:53.114 INFO:tasks.workunit.client.0.smithi071.stdout:2/132: dwrite d0/d2/dd/fc [0,4194304] 0 2024-04-23T02:13:53.115 INFO:tasks.workunit.client.1.smithi192.stdout:6/369: dwrite d3/d15/d1b/d21/d52/d64/f65 [0,4194304] 0 2024-04-23T02:13:53.116 INFO:tasks.workunit.client.0.smithi071.stdout:6/90: dread d6/f9 [0,4194304] 0 2024-04-23T02:13:53.118 INFO:tasks.workunit.client.0.smithi071.stdout:9/148: rmdir d1 39 2024-04-23T02:13:53.118 INFO:tasks.workunit.client.1.smithi192.stdout:5/226: dwrite d2/d32/f44 [0,4194304] 0 2024-04-23T02:13:53.129 INFO:tasks.workunit.client.0.smithi071.stdout:9/149: unlink d1/d1a/c2c 0 2024-04-23T02:13:53.129 INFO:tasks.workunit.client.0.smithi071.stdout:3/87: dwrite d7/fd [0,4194304] 0 2024-04-23T02:13:53.130 INFO:tasks.workunit.client.0.smithi071.stdout:6/91: creat d6/d17/f1a x:0 0 0 2024-04-23T02:13:53.132 INFO:tasks.workunit.client.0.smithi071.stdout:4/147: dwrite d2/d15/d1d/d2c/f1b [0,4194304] 0 2024-04-23T02:13:53.133 INFO:tasks.workunit.client.0.smithi071.stdout:7/124: chown d3/d17/d1c/c1e 58485214 1 2024-04-23T02:13:53.133 INFO:tasks.workunit.client.1.smithi192.stdout:1/71: dread f3 [0,4194304] 0 2024-04-23T02:13:53.134 INFO:tasks.workunit.client.0.smithi071.stdout:8/189: dwrite dc/d10/f22 [0,4194304] 0 2024-04-23T02:13:53.136 INFO:tasks.workunit.client.0.smithi071.stdout:3/88: mknod d7/c1c 0 2024-04-23T02:13:53.137 INFO:tasks.workunit.client.0.smithi071.stdout:9/150: unlink d1/d1a/f24 0 2024-04-23T02:13:53.138 INFO:tasks.workunit.client.0.smithi071.stdout:6/92: write d6/f11 [615395,2525] 0 2024-04-23T02:13:53.140 INFO:tasks.workunit.client.0.smithi071.stdout:3/89: mkdir d7/d1d 0 2024-04-23T02:13:53.141 INFO:tasks.workunit.client.0.smithi071.stdout:2/133: dread d0/d2/dd/d14/f19 [0,4194304] 0 2024-04-23T02:13:53.141 INFO:tasks.workunit.client.0.smithi071.stdout:8/190: creat dc/d15/d2d/f38 x:0 0 0 2024-04-23T02:13:53.142 INFO:tasks.workunit.client.0.smithi071.stdout:1/72: dread f2 [0,4194304] 0 2024-04-23T02:13:53.143 INFO:tasks.workunit.client.0.smithi071.stdout:2/134: dread - d0/d2/f1d zero size 2024-04-23T02:13:53.143 INFO:tasks.workunit.client.0.smithi071.stdout:8/191: dread - dc/f28 zero size 2024-04-23T02:13:53.144 INFO:tasks.workunit.client.0.smithi071.stdout:3/90: write f5 [407299,14446] 0 2024-04-23T02:13:53.146 INFO:tasks.workunit.client.0.smithi071.stdout:8/192: stat dc/d10/c26 0 2024-04-23T02:13:53.146 INFO:tasks.workunit.client.0.smithi071.stdout:7/125: creat d3/d17/d1c/d25/f2f x:0 0 0 2024-04-23T02:13:53.147 INFO:tasks.workunit.client.0.smithi071.stdout:1/73: write f0 [2452505,56281] 0 2024-04-23T02:13:53.148 INFO:tasks.workunit.client.0.smithi071.stdout:9/151: mknod d1/d1a/c2e 0 2024-04-23T02:13:53.149 INFO:tasks.workunit.client.0.smithi071.stdout:6/93: mknod d6/d17/c1b 0 2024-04-23T02:13:53.149 INFO:tasks.workunit.client.1.smithi192.stdout:3/240: symlink d1/d3/d17/d1f/d33/l4e 0 2024-04-23T02:13:53.150 INFO:tasks.workunit.client.0.smithi071.stdout:9/152: readlink d1/l2a 0 2024-04-23T02:13:53.150 INFO:tasks.workunit.client.0.smithi071.stdout:7/126: chown d3/f12 834591353 1 2024-04-23T02:13:53.153 INFO:tasks.workunit.client.0.smithi071.stdout:4/148: dread d2/d15/d1d/d23/f16 [0,4194304] 0 2024-04-23T02:13:53.154 INFO:tasks.workunit.client.1.smithi192.stdout:5/227: write d2/d32/f33 [1879117,109944] 0 2024-04-23T02:13:53.155 INFO:tasks.workunit.client.1.smithi192.stdout:3/241: stat d1/d3/d17/d2b/d44/c4a 0 2024-04-23T02:13:53.157 INFO:tasks.workunit.client.0.smithi071.stdout:0/138: dread d0/d3/de/f1d [0,4194304] 0 2024-04-23T02:13:53.157 INFO:tasks.workunit.client.0.smithi071.stdout:9/153: dread d1/f9 [0,4194304] 0 2024-04-23T02:13:53.158 INFO:tasks.workunit.client.1.smithi192.stdout:1/72: mkdir dc/d14/d15 0 2024-04-23T02:13:53.158 INFO:tasks.workunit.client.0.smithi071.stdout:2/135: creat d0/d2/dd/d14/d1f/f25 x:0 0 0 2024-04-23T02:13:53.159 INFO:tasks.workunit.client.1.smithi192.stdout:6/370: link d3/f57 d3/d15/d1b/f72 0 2024-04-23T02:13:53.162 INFO:tasks.workunit.client.1.smithi192.stdout:1/73: mkdir dc/d16 0 2024-04-23T02:13:53.162 INFO:tasks.workunit.client.0.smithi071.stdout:7/127: link d3/d17/d1c/c24 d3/c30 0 2024-04-23T02:13:53.162 INFO:tasks.workunit.client.0.smithi071.stdout:0/139: write d0/f20 [366433,72481] 0 2024-04-23T02:13:53.162 INFO:tasks.workunit.client.0.smithi071.stdout:9/154: symlink d1/d1a/l2f 0 2024-04-23T02:13:53.163 INFO:tasks.workunit.client.1.smithi192.stdout:1/74: stat dc/le 0 2024-04-23T02:13:53.164 INFO:tasks.workunit.client.1.smithi192.stdout:1/75: readlink l8 0 2024-04-23T02:13:53.165 INFO:tasks.workunit.client.0.smithi071.stdout:4/149: truncate d2/d15/d1d/d23/f14 187600 0 2024-04-23T02:13:53.165 INFO:tasks.workunit.client.1.smithi192.stdout:1/76: stat dc/d16 0 2024-04-23T02:13:53.166 INFO:tasks.workunit.client.0.smithi071.stdout:6/94: dwrite f3 [0,4194304] 0 2024-04-23T02:13:53.166 INFO:tasks.workunit.client.0.smithi071.stdout:7/128: chown d3/d17/c23 219790231 1 2024-04-23T02:13:53.166 INFO:tasks.workunit.client.1.smithi192.stdout:5/228: dwrite d2/fe [4194304,4194304] 0 2024-04-23T02:13:53.168 INFO:tasks.workunit.client.0.smithi071.stdout:3/91: dwrite d7/d8/fe [0,4194304] 0 2024-04-23T02:13:53.169 INFO:tasks.workunit.client.0.smithi071.stdout:6/95: stat d6/fb 0 2024-04-23T02:13:53.169 INFO:tasks.workunit.client.0.smithi071.stdout:7/129: write d3/fc [2059147,45039] 0 2024-04-23T02:13:53.170 INFO:tasks.workunit.client.1.smithi192.stdout:1/77: write f1 [2675056,73393] 0 2024-04-23T02:13:53.170 INFO:tasks.workunit.client.0.smithi071.stdout:9/155: creat d1/f30 x:0 0 0 2024-04-23T02:13:53.176 INFO:tasks.workunit.client.0.smithi071.stdout:3/92: creat d7/d10/f1e x:0 0 0 2024-04-23T02:13:53.177 INFO:tasks.workunit.client.0.smithi071.stdout:7/130: rmdir d3 39 2024-04-23T02:13:53.177 INFO:tasks.workunit.client.1.smithi192.stdout:3/242: dwrite d1/d3/d6/d7/f31 [4194304,4194304] 0 2024-04-23T02:13:53.178 INFO:tasks.workunit.client.0.smithi071.stdout:6/96: dread d6/f14 [0,4194304] 0 2024-04-23T02:13:53.179 INFO:tasks.workunit.client.1.smithi192.stdout:5/229: rename d2/d32/f33 to d2/d6/f4d 0 2024-04-23T02:13:53.181 INFO:tasks.workunit.client.0.smithi071.stdout:9/156: creat d1/d1a/f31 x:0 0 0 2024-04-23T02:13:53.183 INFO:tasks.workunit.client.0.smithi071.stdout:4/150: truncate d2/d15/d1d/d23/d10/f13 793070 0 2024-04-23T02:13:53.183 INFO:tasks.workunit.client.1.smithi192.stdout:5/230: chown d2/db/f18 6021097 1 2024-04-23T02:13:53.186 INFO:tasks.workunit.client.1.smithi192.stdout:5/231: mkdir d2/d2e/d4e 0 2024-04-23T02:13:53.186 INFO:tasks.workunit.client.0.smithi071.stdout:4/151: chown d2/d2a 976344 1 2024-04-23T02:13:53.187 INFO:tasks.workunit.client.0.smithi071.stdout:0/140: rmdir d0/d24 0 2024-04-23T02:13:53.188 INFO:tasks.workunit.client.0.smithi071.stdout:1/74: write f5 [1017510,111541] 0 2024-04-23T02:13:53.189 INFO:tasks.workunit.client.1.smithi192.stdout:3/243: creat d1/d3/d17/d1a/f4f x:0 0 0 2024-04-23T02:13:53.190 INFO:tasks.workunit.client.1.smithi192.stdout:1/78: dwrite dc/fd [0,4194304] 0 2024-04-23T02:13:53.190 INFO:tasks.workunit.client.1.smithi192.stdout:5/232: chown d2/c17 793248921 1 2024-04-23T02:13:53.191 INFO:tasks.workunit.client.0.smithi071.stdout:9/157: mkdir d1/d32 0 2024-04-23T02:13:53.192 INFO:tasks.workunit.client.0.smithi071.stdout:7/131: chown d3/d17/d1c/d25 349 1 2024-04-23T02:13:53.192 INFO:tasks.workunit.client.0.smithi071.stdout:0/141: fdatasync d0/d3/f2d 0 2024-04-23T02:13:53.193 INFO:tasks.workunit.client.1.smithi192.stdout:1/79: write f1 [5016279,38094] 0 2024-04-23T02:13:53.196 INFO:tasks.workunit.client.0.smithi071.stdout:4/152: write d2/d15/d1d/d23/d17/f24 [670367,47601] 0 2024-04-23T02:13:53.196 INFO:tasks.workunit.client.0.smithi071.stdout:3/93: dwrite f5 [0,4194304] 0 2024-04-23T02:13:53.197 INFO:tasks.workunit.client.0.smithi071.stdout:9/158: dread - d1/f18 zero size 2024-04-23T02:13:53.198 INFO:tasks.workunit.client.0.smithi071.stdout:6/97: dwrite d6/fb [4194304,4194304] 0 2024-04-23T02:13:53.199 INFO:tasks.workunit.client.1.smithi192.stdout:3/244: creat d1/d3/d17/d1f/d33/d24/f50 x:0 0 0 2024-04-23T02:13:53.201 INFO:tasks.workunit.client.1.smithi192.stdout:5/233: mkdir d2/db/d16/d48/d4f 0 2024-04-23T02:13:53.202 INFO:tasks.workunit.client.0.smithi071.stdout:3/94: write d7/fd [3132011,58737] 0 2024-04-23T02:13:53.202 INFO:tasks.workunit.client.1.smithi192.stdout:3/245: fdatasync d1/f5 0 2024-04-23T02:13:53.202 INFO:tasks.workunit.client.1.smithi192.stdout:6/371: dread d3/d15/d1b/d21/d52/d64/f65 [0,4194304] 0 2024-04-23T02:13:53.204 INFO:tasks.workunit.client.1.smithi192.stdout:1/80: link dc/c12 dc/d16/c17 0 2024-04-23T02:13:53.205 INFO:tasks.workunit.client.0.smithi071.stdout:6/98: mknod d6/d17/c1c 0 2024-04-23T02:13:53.205 INFO:tasks.workunit.client.1.smithi192.stdout:6/372: mknod d3/d15/d1b/d66/c73 0 2024-04-23T02:13:53.206 INFO:tasks.workunit.client.1.smithi192.stdout:1/81: write dc/f13 [448860,83108] 0 2024-04-23T02:13:53.209 INFO:tasks.workunit.client.1.smithi192.stdout:6/373: fsync d3/d11/f33 0 2024-04-23T02:13:53.210 INFO:tasks.workunit.client.0.smithi071.stdout:8/193: dread dc/d10/f1e [0,4194304] 0 2024-04-23T02:13:53.210 INFO:tasks.workunit.client.1.smithi192.stdout:5/234: read d2/db/f13 [923550,33304] 0 2024-04-23T02:13:53.211 INFO:tasks.workunit.client.0.smithi071.stdout:9/159: truncate d1/d1a/f2d 1013888 0 2024-04-23T02:13:53.211 INFO:tasks.workunit.client.0.smithi071.stdout:2/136: rename d0/d2/dd to d0/d2/d26 0 2024-04-23T02:13:53.212 INFO:tasks.workunit.client.1.smithi192.stdout:6/374: chown d3/le 58 1 2024-04-23T02:13:53.212 INFO:tasks.workunit.client.0.smithi071.stdout:6/99: chown d6/l15 975 1 2024-04-23T02:13:53.213 INFO:tasks.workunit.client.0.smithi071.stdout:0/142: mknod d0/d3/de/d10/d16/d23/c2f 0 2024-04-23T02:13:53.214 INFO:tasks.workunit.client.0.smithi071.stdout:1/75: dwrite fa [0,4194304] 0 2024-04-23T02:13:53.214 INFO:tasks.workunit.client.0.smithi071.stdout:8/194: dread - dc/f37 zero size 2024-04-23T02:13:53.215 INFO:tasks.workunit.client.1.smithi192.stdout:5/235: symlink d2/db/d16/d48/l50 0 2024-04-23T02:13:53.216 INFO:tasks.workunit.client.0.smithi071.stdout:7/132: getdents d3/d17 0 2024-04-23T02:13:53.216 INFO:tasks.workunit.client.0.smithi071.stdout:9/160: write d1/f4 [1308628,114301] 0 2024-04-23T02:13:53.216 INFO:tasks.workunit.client.0.smithi071.stdout:8/195: chown c7 198785 1 2024-04-23T02:13:53.217 INFO:tasks.workunit.client.0.smithi071.stdout:2/137: chown d0/d2/d26/d14/f19 92 1 2024-04-23T02:13:53.217 INFO:tasks.workunit.client.1.smithi192.stdout:6/375: symlink d3/d15/d1b/l74 0 2024-04-23T02:13:53.218 INFO:tasks.workunit.client.1.smithi192.stdout:1/82: dwrite f3 [4194304,4194304] 0 2024-04-23T02:13:53.218 INFO:tasks.workunit.client.0.smithi071.stdout:0/143: write d0/d3/d1c/f22 [4928892,113200] 0 2024-04-23T02:13:53.220 INFO:tasks.workunit.client.0.smithi071.stdout:3/95: rename d7/fb to d7/d8/f1f 0 2024-04-23T02:13:53.222 INFO:tasks.workunit.client.0.smithi071.stdout:4/153: dread d2/d15/d1d/d23/ff [0,4194304] 0 2024-04-23T02:13:53.224 INFO:tasks.workunit.client.0.smithi071.stdout:6/100: write d6/fa [2725163,101952] 0 2024-04-23T02:13:53.225 INFO:tasks.workunit.client.0.smithi071.stdout:2/138: mknod d0/d2/c27 0 2024-04-23T02:13:53.226 INFO:tasks.workunit.client.0.smithi071.stdout:7/133: chown d3/d17/d1c/c24 7 1 2024-04-23T02:13:53.228 INFO:tasks.workunit.client.0.smithi071.stdout:3/96: write d7/d8/f16 [310942,8895] 0 2024-04-23T02:13:53.228 INFO:tasks.workunit.client.1.smithi192.stdout:9/251: sync 2024-04-23T02:13:53.228 INFO:tasks.workunit.client.1.smithi192.stdout:0/291: sync 2024-04-23T02:13:53.228 INFO:tasks.workunit.client.1.smithi192.stdout:7/210: sync 2024-04-23T02:13:53.228 INFO:tasks.workunit.client.1.smithi192.stdout:6/376: creat d3/d18/d58/f75 x:0 0 0 2024-04-23T02:13:53.229 INFO:tasks.workunit.client.1.smithi192.stdout:5/236: dwrite d2/d32/f44 [0,4194304] 0 2024-04-23T02:13:53.230 INFO:tasks.workunit.client.0.smithi071.stdout:0/144: read d0/d3/de/f28 [350031,16838] 0 2024-04-23T02:13:53.230 INFO:tasks.workunit.client.1.smithi192.stdout:9/252: chown d0/c21 2927233 1 2024-04-23T02:13:53.230 INFO:tasks.workunit.client.1.smithi192.stdout:5/237: chown l1 35251 1 2024-04-23T02:13:53.231 INFO:tasks.workunit.client.0.smithi071.stdout:6/101: readlink d6/l15 0 2024-04-23T02:13:53.232 INFO:tasks.workunit.client.1.smithi192.stdout:0/292: mknod d6/df/d56/c65 0 2024-04-23T02:13:53.232 INFO:tasks.workunit.client.1.smithi192.stdout:1/83: dread dc/f13 [0,4194304] 0 2024-04-23T02:13:53.234 INFO:tasks.workunit.client.1.smithi192.stdout:1/84: stat f4 0 2024-04-23T02:13:53.235 INFO:tasks.workunit.client.1.smithi192.stdout:5/238: symlink d2/d6/d14/l51 0 2024-04-23T02:13:53.235 INFO:tasks.workunit.client.1.smithi192.stdout:1/85: dread - fa zero size 2024-04-23T02:13:53.236 INFO:tasks.workunit.client.1.smithi192.stdout:0/293: rename d6/df to d6/df/d13/d25/d3a/d66 22 2024-04-23T02:13:53.236 INFO:tasks.workunit.client.1.smithi192.stdout:1/86: dread - dc/f10 zero size 2024-04-23T02:13:53.238 INFO:tasks.workunit.client.1.smithi192.stdout:9/253: mkdir d0/d1e/d3c/d5f 0 2024-04-23T02:13:53.239 INFO:tasks.workunit.client.0.smithi071.stdout:8/196: unlink dc/d15/l2a 0 2024-04-23T02:13:53.239 INFO:tasks.workunit.client.0.smithi071.stdout:0/145: symlink d0/d3/l30 0 2024-04-23T02:13:53.240 INFO:tasks.workunit.client.1.smithi192.stdout:7/211: creat d5/d17/d1c/f3c x:0 0 0 2024-04-23T02:13:53.241 INFO:tasks.workunit.client.0.smithi071.stdout:4/154: link d2/d15/f1e d2/d15/d1d/d23/d29/f31 0 2024-04-23T02:13:53.241 INFO:tasks.workunit.client.0.smithi071.stdout:9/161: dwrite d1/d1a/f31 [0,4194304] 0 2024-04-23T02:13:53.241 INFO:tasks.workunit.client.0.smithi071.stdout:6/102: rename d6/fd to d6/f1d 0 2024-04-23T02:13:53.242 INFO:tasks.workunit.client.1.smithi192.stdout:9/254: mknod d0/d1e/d3c/c60 0 2024-04-23T02:13:53.243 INFO:tasks.workunit.client.1.smithi192.stdout:1/87: truncate dc/f13 1365283 0 2024-04-23T02:13:53.244 INFO:tasks.workunit.client.0.smithi071.stdout:2/139: mknod d0/d2/d26/df/c28 0 2024-04-23T02:13:53.244 INFO:tasks.workunit.client.0.smithi071.stdout:1/76: dwrite f0 [0,4194304] 0 2024-04-23T02:13:53.245 INFO:tasks.workunit.client.1.smithi192.stdout:1/88: fsync f1 0 2024-04-23T02:13:53.245 INFO:tasks.workunit.client.0.smithi071.stdout:0/146: write d0/d3/d1c/f21 [1030752,101391] 0 2024-04-23T02:13:53.245 INFO:tasks.workunit.client.1.smithi192.stdout:7/212: mkdir d5/d17/d1c/d30/d3d 0 2024-04-23T02:13:53.246 INFO:tasks.workunit.client.1.smithi192.stdout:1/89: stat f1 0 2024-04-23T02:13:53.246 INFO:tasks.workunit.client.0.smithi071.stdout:3/97: truncate d7/d8/f11 69285 0 2024-04-23T02:13:53.249 INFO:tasks.workunit.client.1.smithi192.stdout:7/213: chown d5/d17/d1c/f2d 123020 1 2024-04-23T02:13:53.249 INFO:tasks.workunit.client.1.smithi192.stdout:6/377: dread d3/d15/f4b [0,4194304] 0 2024-04-23T02:13:53.249 INFO:tasks.workunit.client.1.smithi192.stdout:5/239: dwrite d2/db/d16/d1b/f1f [0,4194304] 0 2024-04-23T02:13:53.249 INFO:tasks.workunit.client.0.smithi071.stdout:9/162: chown d1/d1a/f25 67576 1 2024-04-23T02:13:53.250 INFO:tasks.workunit.client.0.smithi071.stdout:1/77: write db/fd [620966,103004] 0 2024-04-23T02:13:53.250 INFO:tasks.workunit.client.1.smithi192.stdout:7/214: stat d5/f15 0 2024-04-23T02:13:53.251 INFO:tasks.workunit.client.1.smithi192.stdout:5/240: chown d2/l9 2148 1 2024-04-23T02:13:53.251 INFO:tasks.workunit.client.1.smithi192.stdout:9/255: mkdir d0/d9/d44/d51/d5b/d61 0 2024-04-23T02:13:53.252 INFO:tasks.workunit.client.1.smithi192.stdout:5/241: readlink d2/db/l1e 0 2024-04-23T02:13:53.255 INFO:tasks.workunit.client.0.smithi071.stdout:7/134: write d3/f12 [5057827,23142] 0 2024-04-23T02:13:53.255 INFO:tasks.workunit.client.0.smithi071.stdout:9/163: fdatasync d1/f4 0 2024-04-23T02:13:53.255 INFO:tasks.workunit.client.1.smithi192.stdout:5/242: write d2/d6/f4d [3112860,99942] 0 2024-04-23T02:13:53.255 INFO:tasks.workunit.client.0.smithi071.stdout:8/197: dwrite dc/d10/f1c [0,4194304] 0 2024-04-23T02:13:53.259 INFO:tasks.workunit.client.1.smithi192.stdout:6/378: creat d3/f76 x:0 0 0 2024-04-23T02:13:53.260 INFO:tasks.workunit.client.0.smithi071.stdout:9/164: fdatasync d1/fa 0 2024-04-23T02:13:53.261 INFO:tasks.workunit.client.1.smithi192.stdout:1/90: dread f9 [0,4194304] 0 2024-04-23T02:13:53.262 INFO:tasks.workunit.client.0.smithi071.stdout:8/198: readlink dc/d15/l19 0 2024-04-23T02:13:53.263 INFO:tasks.workunit.client.1.smithi192.stdout:5/243: mkdir d2/db/d2b/d52 0 2024-04-23T02:13:53.264 INFO:tasks.workunit.client.1.smithi192.stdout:5/244: fdatasync d2/d6/f4d 0 2024-04-23T02:13:53.266 INFO:tasks.workunit.client.0.smithi071.stdout:6/103: dread f5 [0,4194304] 0 2024-04-23T02:13:53.268 INFO:tasks.workunit.client.0.smithi071.stdout:4/155: fdatasync d2/d15/d1d/d23/d10/f13 0 2024-04-23T02:13:53.269 INFO:tasks.workunit.client.1.smithi192.stdout:0/294: truncate d6/df/f23 830387 0 2024-04-23T02:13:53.270 INFO:tasks.workunit.client.1.smithi192.stdout:0/295: stat d6/df/d13/f31 0 2024-04-23T02:13:53.274 INFO:tasks.workunit.client.0.smithi071.stdout:0/147: dread d0/d3/de/f1d [0,4194304] 0 2024-04-23T02:13:53.274 INFO:tasks.workunit.client.0.smithi071.stdout:3/98: link d7/c1c d7/d8/c20 0 2024-04-23T02:13:53.277 INFO:tasks.workunit.client.1.smithi192.stdout:0/296: mkdir d6/df/d13/d25/d26/d67 0 2024-04-23T02:13:53.278 INFO:tasks.workunit.client.0.smithi071.stdout:2/140: dwrite d0/d2/d26/d14/f23 [0,4194304] 0 2024-04-23T02:13:53.278 INFO:tasks.workunit.client.1.smithi192.stdout:5/245: symlink d2/d2e/d4e/l53 0 2024-04-23T02:13:53.278 INFO:tasks.workunit.client.1.smithi192.stdout:9/256: rmdir d0/d9/d4d 0 2024-04-23T02:13:53.279 INFO:tasks.workunit.client.1.smithi192.stdout:5/246: stat d2/d6 0 2024-04-23T02:13:53.280 INFO:tasks.workunit.client.0.smithi071.stdout:1/78: dwrite f0 [0,4194304] 0 2024-04-23T02:13:53.283 INFO:tasks.workunit.client.0.smithi071.stdout:9/165: mknod d1/d1a/c33 0 2024-04-23T02:13:53.283 INFO:tasks.workunit.client.0.smithi071.stdout:7/135: dwrite d3/ff [0,4194304] 0 2024-04-23T02:13:53.284 INFO:tasks.workunit.client.1.smithi192.stdout:6/379: creat d3/d15/d1b/f77 x:0 0 0 2024-04-23T02:13:53.284 INFO:tasks.workunit.client.0.smithi071.stdout:2/141: fsync d0/d2/d26/d14/f1a 0 2024-04-23T02:13:53.285 INFO:tasks.workunit.client.1.smithi192.stdout:0/297: rmdir d6/d4b 39 2024-04-23T02:13:53.287 INFO:tasks.workunit.client.0.smithi071.stdout:8/199: dread dc/d10/f14 [0,4194304] 0 2024-04-23T02:13:53.289 INFO:tasks.workunit.client.0.smithi071.stdout:6/104: dwrite d6/f16 [0,4194304] 0 2024-04-23T02:13:53.291 INFO:tasks.workunit.client.0.smithi071.stdout:4/156: rmdir d2/d15/d1d/d2c/d27 0 2024-04-23T02:13:53.294 INFO:tasks.workunit.client.0.smithi071.stdout:0/148: mkdir d0/d3/de/d10/d14/d2b/d31 0 2024-04-23T02:13:53.296 INFO:tasks.workunit.client.0.smithi071.stdout:3/99: dread d7/fd [0,4194304] 0 2024-04-23T02:13:53.298 INFO:tasks.workunit.client.0.smithi071.stdout:9/166: dwrite d1/f30 [0,4194304] 0 2024-04-23T02:13:53.298 INFO:tasks.workunit.client.0.smithi071.stdout:3/100: chown d7/d10/f1e 6 1 2024-04-23T02:13:53.302 INFO:tasks.workunit.client.0.smithi071.stdout:5/173: sync 2024-04-23T02:13:53.304 INFO:tasks.workunit.client.1.smithi192.stdout:9/257: dwrite d0/d9/d44/f47 [0,4194304] 0 2024-04-23T02:13:53.307 INFO:tasks.workunit.client.0.smithi071.stdout:6/105: creat d6/d17/f1e x:0 0 0 2024-04-23T02:13:53.307 INFO:tasks.workunit.client.0.smithi071.stdout:8/200: rename dc/d15/f31 to dc/d15/f39 0 2024-04-23T02:13:53.308 INFO:tasks.workunit.client.0.smithi071.stdout:4/157: mknod d2/d15/d1d/d2c/c32 0 2024-04-23T02:13:53.309 INFO:tasks.workunit.client.1.smithi192.stdout:5/247: dwrite d2/db/f11 [0,4194304] 0 2024-04-23T02:13:53.310 INFO:tasks.workunit.client.1.smithi192.stdout:1/91: dwrite f9 [0,4194304] 0 2024-04-23T02:13:53.310 INFO:tasks.workunit.client.0.smithi071.stdout:0/149: rename d0/d3/de/d10/d16/d23 to d0/d3/de/d10/d14/d32 0 2024-04-23T02:13:53.312 INFO:tasks.workunit.client.0.smithi071.stdout:5/174: creat d3/d1d/f42 x:0 0 0 2024-04-23T02:13:53.313 INFO:tasks.workunit.client.0.smithi071.stdout:9/167: symlink d1/d32/l34 0 2024-04-23T02:13:53.315 INFO:tasks.workunit.client.1.smithi192.stdout:9/258: chown d0/d9/d44/d4f 36 1 2024-04-23T02:13:53.315 INFO:tasks.workunit.client.0.smithi071.stdout:6/106: mkdir d6/d1f 0 2024-04-23T02:13:53.319 INFO:tasks.workunit.client.0.smithi071.stdout:8/201: dwrite dc/d15/f23 [0,4194304] 0 2024-04-23T02:13:53.319 INFO:tasks.workunit.client.0.smithi071.stdout:4/158: unlink d2/d15/d1d/d23/d29/f2f 0 2024-04-23T02:13:53.320 INFO:tasks.workunit.client.0.smithi071.stdout:3/101: mknod d7/d1d/c21 0 2024-04-23T02:13:53.320 INFO:tasks.workunit.client.0.smithi071.stdout:8/202: stat dc/d10/f22 0 2024-04-23T02:13:53.322 INFO:tasks.workunit.client.1.smithi192.stdout:9/259: dread - d0/d9/f3a zero size 2024-04-23T02:13:53.323 INFO:tasks.workunit.client.1.smithi192.stdout:5/248: creat d2/db/d16/d1b/f54 x:0 0 0 2024-04-23T02:13:53.323 INFO:tasks.workunit.client.1.smithi192.stdout:6/380: dwrite d3/d15/d1b/d21/d52/d64/f65 [0,4194304] 0 2024-04-23T02:13:53.323 INFO:tasks.workunit.client.0.smithi071.stdout:4/159: dread - d2/d15/d1d/d2c/f2e zero size 2024-04-23T02:13:53.325 INFO:tasks.workunit.client.0.smithi071.stdout:5/175: mkdir d3/d1d/d43 0 2024-04-23T02:13:53.325 INFO:tasks.workunit.client.0.smithi071.stdout:9/168: rename d1/c19 to d1/c35 0 2024-04-23T02:13:53.329 INFO:tasks.workunit.client.1.smithi192.stdout:1/92: dwrite f4 [0,4194304] 0 2024-04-23T02:13:53.329 INFO:tasks.workunit.client.0.smithi071.stdout:6/107: dwrite d6/ff [0,4194304] 0 2024-04-23T02:13:53.331 INFO:tasks.workunit.client.0.smithi071.stdout:3/102: mkdir d7/d22 0 2024-04-23T02:13:53.331 INFO:tasks.workunit.client.1.smithi192.stdout:9/260: fsync d0/d9/f25 0 2024-04-23T02:13:53.331 INFO:tasks.workunit.client.1.smithi192.stdout:1/93: chown dc/ff 43295 1 2024-04-23T02:13:53.333 INFO:tasks.workunit.client.1.smithi192.stdout:5/249: creat d2/d6/f55 x:0 0 0 2024-04-23T02:13:53.335 INFO:tasks.workunit.client.1.smithi192.stdout:9/261: dread - d0/d9/f3a zero size 2024-04-23T02:13:53.335 INFO:tasks.workunit.client.0.smithi071.stdout:8/203: mkdir dc/d3a 0 2024-04-23T02:13:53.336 INFO:tasks.workunit.client.0.smithi071.stdout:4/160: creat d2/d15/d1d/d2c/f33 x:0 0 0 2024-04-23T02:13:53.336 INFO:tasks.workunit.client.0.smithi071.stdout:5/176: symlink d3/l44 0 2024-04-23T02:13:53.337 INFO:tasks.workunit.client.1.smithi192.stdout:9/262: chown d0/d1e/f23 52 1 2024-04-23T02:13:53.337 INFO:tasks.workunit.client.1.smithi192.stdout:5/250: rmdir d2/db/d16/d48 39 2024-04-23T02:13:53.338 INFO:tasks.workunit.client.1.smithi192.stdout:1/94: mkdir dc/d14/d15/d18 0 2024-04-23T02:13:53.338 INFO:tasks.workunit.client.1.smithi192.stdout:6/381: dwrite d3/d11/f4a [0,4194304] 0 2024-04-23T02:13:53.340 INFO:tasks.workunit.client.0.smithi071.stdout:3/103: mkdir d7/d22/d23 0 2024-04-23T02:13:53.342 INFO:tasks.workunit.client.0.smithi071.stdout:5/177: dread - d3/d1d/f2a zero size 2024-04-23T02:13:53.342 INFO:tasks.workunit.client.0.smithi071.stdout:8/204: rename dc/f18 to dc/d10/d32/f3b 0 2024-04-23T02:13:53.344 INFO:tasks.workunit.client.0.smithi071.stdout:3/104: chown d7/d8/l13 1 1 2024-04-23T02:13:53.345 INFO:tasks.workunit.client.0.smithi071.stdout:6/108: dread d6/f11 [0,4194304] 0 2024-04-23T02:13:53.348 INFO:tasks.workunit.client.1.smithi192.stdout:5/251: chown d2/db/d16/d48/l50 3 1 2024-04-23T02:13:53.348 INFO:tasks.workunit.client.1.smithi192.stdout:0/298: write d6/f18 [719336,103501] 0 2024-04-23T02:13:53.348 INFO:tasks.workunit.client.1.smithi192.stdout:5/252: fsync d2/d6/f4d 0 2024-04-23T02:13:53.349 INFO:tasks.workunit.client.1.smithi192.stdout:9/263: rename d0/l15 to d0/l62 0 2024-04-23T02:13:53.351 INFO:tasks.workunit.client.0.smithi071.stdout:5/178: write d3/d1d/f20 [1550229,75494] 0 2024-04-23T02:13:53.351 INFO:tasks.workunit.client.1.smithi192.stdout:5/253: fdatasync d2/db/d2b/f3a 0 2024-04-23T02:13:53.352 INFO:tasks.workunit.client.1.smithi192.stdout:0/299: unlink d6/df/c2a 0 2024-04-23T02:13:53.352 INFO:tasks.workunit.client.1.smithi192.stdout:5/254: write d2/fe [2406112,44343] 0 2024-04-23T02:13:53.353 INFO:tasks.workunit.client.0.smithi071.stdout:4/161: rename d2/d15/d1d/d23/f14 to d2/d2a/d2d/f34 0 2024-04-23T02:13:53.353 INFO:tasks.workunit.client.0.smithi071.stdout:9/169: dwrite d1/d1a/f2d [0,4194304] 0 2024-04-23T02:13:53.354 INFO:tasks.workunit.client.1.smithi192.stdout:6/382: link d3/d15/d1b/d21/d52/d64/f65 d3/d18/f78 0 2024-04-23T02:13:53.358 INFO:tasks.workunit.client.0.smithi071.stdout:6/109: mknod d6/c20 0 2024-04-23T02:13:53.360 INFO:tasks.workunit.client.0.smithi071.stdout:5/179: write d3/d1d/f20 [1068552,2119] 0 2024-04-23T02:13:53.361 INFO:tasks.workunit.client.1.smithi192.stdout:0/300: creat d6/f68 x:0 0 0 2024-04-23T02:13:53.361 INFO:tasks.workunit.client.1.smithi192.stdout:1/95: dread dc/fd [4194304,4194304] 0 2024-04-23T02:13:53.361 INFO:tasks.workunit.client.1.smithi192.stdout:6/383: mkdir d3/d28/d79 0 2024-04-23T02:13:53.361 INFO:tasks.workunit.client.1.smithi192.stdout:0/301: write d6/f68 [137994,30178] 0 2024-04-23T02:13:53.363 INFO:tasks.workunit.client.0.smithi071.stdout:9/170: unlink d1/ld 0 2024-04-23T02:13:53.364 INFO:tasks.workunit.client.0.smithi071.stdout:8/205: dwrite dc/f37 [0,4194304] 0 2024-04-23T02:13:53.365 INFO:tasks.workunit.client.0.smithi071.stdout:7/136: write d3/d17/f1f [283046,120566] 0 2024-04-23T02:13:53.365 INFO:tasks.workunit.client.0.smithi071.stdout:5/180: chown d3/df/d10/l28 6006681 1 2024-04-23T02:13:53.369 INFO:tasks.workunit.client.1.smithi192.stdout:1/96: mkdir dc/d16/d19 0 2024-04-23T02:13:53.371 INFO:tasks.workunit.client.1.smithi192.stdout:1/97: truncate f1 5188944 0 2024-04-23T02:13:53.374 INFO:tasks.workunit.client.0.smithi071.stdout:0/150: rmdir d0 39 2024-04-23T02:13:53.376 INFO:tasks.workunit.client.0.smithi071.stdout:9/171: fdatasync d1/f9 0 2024-04-23T02:13:53.377 INFO:tasks.workunit.client.1.smithi192.stdout:9/264: dwrite d0/d9/d44/f45 [0,4194304] 0 2024-04-23T02:13:53.378 INFO:tasks.workunit.client.0.smithi071.stdout:7/137: dwrite d3/d17/d1c/d25/f27 [0,4194304] 0 2024-04-23T02:13:53.378 INFO:tasks.workunit.client.1.smithi192.stdout:1/98: write dc/fd [4222947,110090] 0 2024-04-23T02:13:53.380 INFO:tasks.workunit.client.1.smithi192.stdout:6/384: dwrite d3/d18/f2d [0,4194304] 0 2024-04-23T02:13:53.381 INFO:tasks.workunit.client.0.smithi071.stdout:5/181: mknod d3/d1d/d2c/c45 0 2024-04-23T02:13:53.382 INFO:tasks.workunit.client.1.smithi192.stdout:0/302: creat d6/df/d13/d25/d26/d67/f69 x:0 0 0 2024-04-23T02:13:53.382 INFO:tasks.workunit.client.1.smithi192.stdout:1/99: write dc/f10 [986261,86619] 0 2024-04-23T02:13:53.384 INFO:tasks.workunit.client.0.smithi071.stdout:4/162: link d2/d15/f1a d2/d15/d1d/f35 0 2024-04-23T02:13:53.385 INFO:tasks.workunit.client.1.smithi192.stdout:9/265: symlink d0/d9/d44/d51/d5b/d61/l63 0 2024-04-23T02:13:53.385 INFO:tasks.workunit.client.1.smithi192.stdout:0/303: chown d6/df/d13/d25/f2b 127789738 1 2024-04-23T02:13:53.386 INFO:tasks.workunit.client.1.smithi192.stdout:4/389: sync 2024-04-23T02:13:53.386 INFO:tasks.workunit.client.1.smithi192.stdout:5/255: dwrite d2/db/d16/d1b/f29 [0,4194304] 0 2024-04-23T02:13:53.387 INFO:tasks.workunit.client.1.smithi192.stdout:0/304: stat d6/df/f32 0 2024-04-23T02:13:53.391 INFO:tasks.workunit.client.0.smithi071.stdout:7/138: dwrite d3/fc [0,4194304] 0 2024-04-23T02:13:53.391 INFO:tasks.workunit.client.0.smithi071.stdout:8/206: creat dc/d3a/f3c x:0 0 0 2024-04-23T02:13:53.392 INFO:tasks.workunit.client.1.smithi192.stdout:1/100: write dc/f13 [1289816,47338] 0 2024-04-23T02:13:53.396 INFO:tasks.workunit.client.1.smithi192.stdout:9/266: mkdir d0/d9/d44/d51/d64 0 2024-04-23T02:13:53.398 INFO:tasks.workunit.client.1.smithi192.stdout:6/385: getdents d3 0 2024-04-23T02:13:53.398 INFO:tasks.workunit.client.1.smithi192.stdout:5/256: mkdir d2/d2e/d56 0 2024-04-23T02:13:53.402 INFO:tasks.workunit.client.1.smithi192.stdout:0/305: symlink d6/df/d13/d25/d26/d67/l6a 0 2024-04-23T02:13:53.403 INFO:tasks.workunit.client.1.smithi192.stdout:1/101: dread f9 [0,4194304] 0 2024-04-23T02:13:53.403 INFO:tasks.workunit.client.0.smithi071.stdout:4/163: creat d2/d2a/f36 x:0 0 0 2024-04-23T02:13:53.403 INFO:tasks.workunit.client.1.smithi192.stdout:4/390: mkdir de/d15/d19/d3d/d34/d7c 0 2024-04-23T02:13:53.404 INFO:tasks.workunit.client.0.smithi071.stdout:0/151: truncate d0/f1 1875800 0 2024-04-23T02:13:53.405 INFO:tasks.workunit.client.0.smithi071.stdout:7/139: mkdir d3/d17/d1c/d31 0 2024-04-23T02:13:53.405 INFO:tasks.workunit.client.1.smithi192.stdout:6/386: symlink d3/d15/d1b/d21/d3a/l7a 0 2024-04-23T02:13:53.406 INFO:tasks.workunit.client.0.smithi071.stdout:0/152: dread - d0/f18 zero size 2024-04-23T02:13:53.406 INFO:tasks.workunit.client.0.smithi071.stdout:8/207: dread dc/d15/f29 [0,4194304] 0 2024-04-23T02:13:53.406 INFO:tasks.workunit.client.0.smithi071.stdout:8/208: chown f9 61928 1 2024-04-23T02:13:53.406 INFO:tasks.workunit.client.1.smithi192.stdout:9/267: creat d0/d9/f65 x:0 0 0 2024-04-23T02:13:53.407 INFO:tasks.workunit.client.0.smithi071.stdout:0/153: mknod d0/d3/de/d10/d16/c33 0 2024-04-23T02:13:53.410 INFO:tasks.workunit.client.1.smithi192.stdout:4/391: chown de/d15/d19/c41 4141 1 2024-04-23T02:13:53.410 INFO:tasks.workunit.client.1.smithi192.stdout:6/387: symlink d3/d15/d1b/d21/d52/d64/l7b 0 2024-04-23T02:13:53.411 INFO:tasks.workunit.client.0.smithi071.stdout:0/154: link d0/d3/de/d10/l29 d0/l34 0 2024-04-23T02:13:53.412 INFO:tasks.workunit.client.1.smithi192.stdout:6/388: fsync d3/d6c/f71 0 2024-04-23T02:13:53.413 INFO:tasks.workunit.client.0.smithi071.stdout:0/155: write d0/d3/de/d10/f1f [2874139,120557] 0 2024-04-23T02:13:53.414 INFO:tasks.workunit.client.0.smithi071.stdout:8/209: fsync dc/d10/d32/f3b 0 2024-04-23T02:13:53.415 INFO:tasks.workunit.client.1.smithi192.stdout:9/268: getdents d0/d9/d44/d58 0 2024-04-23T02:13:53.415 INFO:tasks.workunit.client.0.smithi071.stdout:0/156: creat d0/d3/de/d10/d14/d32/f35 x:0 0 0 2024-04-23T02:13:53.416 INFO:tasks.workunit.client.0.smithi071.stdout:8/210: fdatasync dc/d15/f39 0 2024-04-23T02:13:53.416 INFO:tasks.workunit.client.0.smithi071.stdout:0/157: readlink d0/l7 0 2024-04-23T02:13:53.417 INFO:tasks.workunit.client.0.smithi071.stdout:8/211: chown dc/c2c 505 1 2024-04-23T02:13:53.418 INFO:tasks.workunit.client.1.smithi192.stdout:6/389: write d3/d18/f2d [1837752,65768] 0 2024-04-23T02:13:53.418 INFO:tasks.workunit.client.1.smithi192.stdout:4/392: symlink de/d15/d19/d3d/l7d 0 2024-04-23T02:13:53.418 INFO:tasks.workunit.client.1.smithi192.stdout:1/102: dwrite f4 [0,4194304] 0 2024-04-23T02:13:53.419 INFO:tasks.workunit.client.0.smithi071.stdout:8/212: chown dc/d10/c11 0 1 2024-04-23T02:13:53.419 INFO:tasks.workunit.client.0.smithi071.stdout:6/110: rmdir d6 39 2024-04-23T02:13:53.419 INFO:tasks.workunit.client.0.smithi071.stdout:8/213: read - dc/d3a/f3c zero size 2024-04-23T02:13:53.420 INFO:tasks.workunit.client.1.smithi192.stdout:1/103: readlink dc/le 0 2024-04-23T02:13:53.420 INFO:tasks.workunit.client.1.smithi192.stdout:0/306: dread d6/df/d47/f61 [0,4194304] 0 2024-04-23T02:13:53.421 INFO:tasks.workunit.client.0.smithi071.stdout:3/105: truncate f0 4164414 0 2024-04-23T02:13:53.421 INFO:tasks.workunit.client.1.smithi192.stdout:9/269: creat d0/d9/d44/d51/f66 x:0 0 0 2024-04-23T02:13:53.421 INFO:tasks.workunit.client.1.smithi192.stdout:6/390: mkdir d3/d15/d1b/d21/d7c 0 2024-04-23T02:13:53.422 INFO:tasks.workunit.client.1.smithi192.stdout:1/104: write fa [789432,28281] 0 2024-04-23T02:13:53.422 INFO:tasks.workunit.client.1.smithi192.stdout:4/393: dread - de/f4d zero size 2024-04-23T02:13:53.423 INFO:tasks.workunit.client.1.smithi192.stdout:0/307: chown d6/df/f32 3140 1 2024-04-23T02:13:53.423 INFO:tasks.workunit.client.0.smithi071.stdout:0/158: rename d0/d3/de/d2c/l27 to d0/d3/de/d10/d14/d2b/d31/l36 0 2024-04-23T02:13:53.425 INFO:tasks.workunit.client.1.smithi192.stdout:9/270: write d0/d9/f25 [195483,58004] 0 2024-04-23T02:13:53.426 INFO:tasks.workunit.client.1.smithi192.stdout:6/391: mknod d3/d15/d1b/d66/c7d 0 2024-04-23T02:13:53.426 INFO:tasks.workunit.client.1.smithi192.stdout:0/308: write d6/f18 [880807,16392] 0 2024-04-23T02:13:53.427 INFO:tasks.workunit.client.0.smithi071.stdout:8/214: dread dc/d10/f14 [0,4194304] 0 2024-04-23T02:13:53.428 INFO:tasks.workunit.client.0.smithi071.stdout:6/111: rename d6/d17/f1e to d6/d1f/f21 0 2024-04-23T02:13:53.429 INFO:tasks.workunit.client.1.smithi192.stdout:5/257: truncate d2/db/d16/d1b/f1f 3908798 0 2024-04-23T02:13:53.429 INFO:tasks.workunit.client.0.smithi071.stdout:8/215: truncate dc/d3a/f3c 442009 0 2024-04-23T02:13:53.430 INFO:tasks.workunit.client.0.smithi071.stdout:3/106: getdents d7/d22 0 2024-04-23T02:13:53.432 INFO:tasks.workunit.client.0.smithi071.stdout:6/112: unlink d6/c19 0 2024-04-23T02:13:53.433 INFO:tasks.workunit.client.1.smithi192.stdout:4/394: creat de/d15/d2a/d74/f7e x:0 0 0 2024-04-23T02:13:53.434 INFO:tasks.workunit.client.0.smithi071.stdout:6/113: write d6/d17/f1a [121582,39967] 0 2024-04-23T02:13:53.435 INFO:tasks.workunit.client.1.smithi192.stdout:9/271: unlink d0/d9/f43 0 2024-04-23T02:13:53.435 INFO:tasks.workunit.client.1.smithi192.stdout:1/105: creat dc/d16/d19/f1a x:0 0 0 2024-04-23T02:13:53.436 INFO:tasks.workunit.client.1.smithi192.stdout:5/258: rmdir d2/d2e/d4e 39 2024-04-23T02:13:53.436 INFO:tasks.workunit.client.0.smithi071.stdout:8/216: dread dc/d10/f22 [0,4194304] 0 2024-04-23T02:13:53.437 INFO:tasks.workunit.client.1.smithi192.stdout:6/392: mknod d3/d11/c7e 0 2024-04-23T02:13:53.438 INFO:tasks.workunit.client.0.smithi071.stdout:5/182: dread d3/da/d1a/f26 [0,4194304] 0 2024-04-23T02:13:53.439 INFO:tasks.workunit.client.1.smithi192.stdout:0/309: dwrite d6/d44/f19 [0,4194304] 0 2024-04-23T02:13:53.439 INFO:tasks.workunit.client.1.smithi192.stdout:1/106: fdatasync dc/d16/d19/f1a 0 2024-04-23T02:13:53.439 INFO:tasks.workunit.client.1.smithi192.stdout:4/395: chown de/d15/d19/d3d/f57 722 1 2024-04-23T02:13:53.440 INFO:tasks.workunit.client.0.smithi071.stdout:5/183: fsync d3/d34/f40 0 2024-04-23T02:13:53.440 INFO:tasks.workunit.client.1.smithi192.stdout:9/272: creat d0/d9/d44/d51/d5b/f67 x:0 0 0 2024-04-23T02:13:53.442 INFO:tasks.workunit.client.0.smithi071.stdout:5/184: mkdir d3/da/d1a/d1c/d46 0 2024-04-23T02:13:53.443 INFO:tasks.workunit.client.0.smithi071.stdout:9/172: truncate d1/f30 3024015 0 2024-04-23T02:13:53.444 INFO:tasks.workunit.client.1.smithi192.stdout:0/310: mkdir d6/df/d13/d6b 0 2024-04-23T02:13:53.445 INFO:tasks.workunit.client.1.smithi192.stdout:6/393: mkdir d3/d15/d1b/d21/d52/d7f 0 2024-04-23T02:13:53.446 INFO:tasks.workunit.client.0.smithi071.stdout:9/173: chown d1/d1a/l27 31803632 1 2024-04-23T02:13:53.447 INFO:tasks.workunit.client.1.smithi192.stdout:5/259: link d2/db/f11 d2/db/d16/f57 0 2024-04-23T02:13:53.448 INFO:tasks.workunit.client.0.smithi071.stdout:5/185: symlink d3/l47 0 2024-04-23T02:13:53.449 INFO:tasks.workunit.client.0.smithi071.stdout:9/174: write d1/d1a/f25 [2331122,23326] 0 2024-04-23T02:13:53.451 INFO:tasks.workunit.client.1.smithi192.stdout:1/107: dread f0 [4194304,4194304] 0 2024-04-23T02:13:53.451 INFO:tasks.workunit.client.1.smithi192.stdout:6/394: mkdir d3/d15/d1b/d66/d80 0 2024-04-23T02:13:53.452 INFO:tasks.workunit.client.0.smithi071.stdout:4/164: dwrite d2/d15/d1d/d23/d29/f31 [0,4194304] 0 2024-04-23T02:13:53.452 INFO:tasks.workunit.client.1.smithi192.stdout:1/108: read dc/fd [638937,118917] 0 2024-04-23T02:13:53.453 INFO:tasks.workunit.client.1.smithi192.stdout:4/396: creat de/d15/d19/d3d/d49/d26/d59/d6a/f7f x:0 0 0 2024-04-23T02:13:53.453 INFO:tasks.workunit.client.0.smithi071.stdout:0/159: rename d0/d3/de/d2c to d0/d3/de/d10/d14/d32/d37 0 2024-04-23T02:13:53.454 INFO:tasks.workunit.client.1.smithi192.stdout:0/311: rename d6/d44/f38 to d6/d4b/f6c 0 2024-04-23T02:13:53.455 INFO:tasks.workunit.client.0.smithi071.stdout:5/186: creat d3/df/d10/f48 x:0 0 0 2024-04-23T02:13:53.456 INFO:tasks.workunit.client.1.smithi192.stdout:9/273: truncate d0/d9/d44/d4f/f8 836058 0 2024-04-23T02:13:53.456 INFO:tasks.workunit.client.1.smithi192.stdout:9/274: dread - d0/f54 zero size 2024-04-23T02:13:53.458 INFO:tasks.workunit.client.1.smithi192.stdout:4/397: mkdir de/d15/d2a/d80 0 2024-04-23T02:13:53.459 INFO:tasks.workunit.client.1.smithi192.stdout:6/395: dread d3/d11/f29 [0,4194304] 0 2024-04-23T02:13:53.460 INFO:tasks.workunit.client.1.smithi192.stdout:5/260: creat d2/db/d16/d1b/d3d/d40/d49/f58 x:0 0 0 2024-04-23T02:13:53.461 INFO:tasks.workunit.client.0.smithi071.stdout:3/107: rename d7/d8/f1f to d7/d22/d23/f24 0 2024-04-23T02:13:53.462 INFO:tasks.workunit.client.1.smithi192.stdout:5/261: write d2/d6/f4d [313520,34150] 0 2024-04-23T02:13:53.463 INFO:tasks.workunit.client.1.smithi192.stdout:6/396: write d3/d11/f43 [1125601,76344] 0 2024-04-23T02:13:53.466 INFO:tasks.workunit.client.1.smithi192.stdout:9/275: symlink d0/d9/d44/d51/l68 0 2024-04-23T02:13:53.467 INFO:tasks.workunit.client.0.smithi071.stdout:8/217: rename dc/d15/f29 to dc/d10/f3d 0 2024-04-23T02:13:53.467 INFO:tasks.workunit.client.1.smithi192.stdout:6/397: mknod d3/d6c/c81 0 2024-04-23T02:13:53.467 INFO:tasks.workunit.client.1.smithi192.stdout:0/312: dread d6/df/d13/d25/d26/f27 [4194304,4194304] 0 2024-04-23T02:13:53.468 INFO:tasks.workunit.client.0.smithi071.stdout:8/218: chown l5 109921 1 2024-04-23T02:13:53.468 INFO:tasks.workunit.client.0.smithi071.stdout:3/108: creat d7/d22/f25 x:0 0 0 2024-04-23T02:13:53.471 INFO:tasks.workunit.client.1.smithi192.stdout:4/398: dwrite de/d15/d19/d3d/f5f [0,4194304] 0 2024-04-23T02:13:53.472 INFO:tasks.workunit.client.0.smithi071.stdout:6/114: dread d6/fa [0,4194304] 0 2024-04-23T02:13:53.472 INFO:tasks.workunit.client.0.smithi071.stdout:8/219: symlink dc/d10/l3e 0 2024-04-23T02:13:53.472 INFO:tasks.workunit.client.0.smithi071.stdout:8/220: readlink l3 0 2024-04-23T02:13:53.473 INFO:tasks.workunit.client.0.smithi071.stdout:5/187: creat d3/da/d1a/f49 x:0 0 0 2024-04-23T02:13:53.473 INFO:tasks.workunit.client.1.smithi192.stdout:0/313: dread d6/df/f23 [0,4194304] 0 2024-04-23T02:13:53.474 INFO:tasks.workunit.client.0.smithi071.stdout:9/175: write d1/f9 [1131077,114203] 0 2024-04-23T02:13:53.474 INFO:tasks.workunit.client.0.smithi071.stdout:3/109: rmdir d7/d1d 39 2024-04-23T02:13:53.474 INFO:tasks.workunit.client.1.smithi192.stdout:4/399: chown de/d15/d19/d3d/d49/f60 961 1 2024-04-23T02:13:53.475 INFO:tasks.workunit.client.0.smithi071.stdout:9/176: chown d1/l7 165718 1 2024-04-23T02:13:53.475 INFO:tasks.workunit.client.0.smithi071.stdout:5/188: chown d3/df/d10/f48 1518167 1 2024-04-23T02:13:53.475 INFO:tasks.workunit.client.1.smithi192.stdout:0/314: stat d6/df/f32 0 2024-04-23T02:13:53.476 INFO:tasks.workunit.client.0.smithi071.stdout:0/160: chown d0/d3/de/f1d 2189 1 2024-04-23T02:13:53.477 INFO:tasks.workunit.client.1.smithi192.stdout:0/315: truncate d6/d4b/f52 1280018 0 2024-04-23T02:13:53.477 INFO:tasks.workunit.client.1.smithi192.stdout:4/400: mkdir de/d15/d53/d81 0 2024-04-23T02:13:53.483 INFO:tasks.workunit.client.0.smithi071.stdout:6/115: write d6/fa [5070373,36665] 0 2024-04-23T02:13:53.483 INFO:tasks.workunit.client.0.smithi071.stdout:0/161: rename d0/d3/d1c to d0/d3/d1c/d38 22 2024-04-23T02:13:53.483 INFO:tasks.workunit.client.0.smithi071.stdout:3/110: mkdir d7/d22/d23/d26 0 2024-04-23T02:13:53.484 INFO:tasks.workunit.client.1.smithi192.stdout:0/316: creat d6/f6d x:0 0 0 2024-04-23T02:13:53.485 INFO:tasks.workunit.client.0.smithi071.stdout:5/189: mkdir d3/da/d1a/d1c/d4a 0 2024-04-23T02:13:53.485 INFO:tasks.workunit.client.0.smithi071.stdout:9/177: creat d1/d1a/f36 x:0 0 0 2024-04-23T02:13:53.485 INFO:tasks.workunit.client.1.smithi192.stdout:0/317: chown d6/df/d13/f31 87 1 2024-04-23T02:13:53.486 INFO:tasks.workunit.client.0.smithi071.stdout:6/116: mkdir d6/d22 0 2024-04-23T02:13:53.487 INFO:tasks.workunit.client.0.smithi071.stdout:8/221: dread dc/d15/f25 [0,4194304] 0 2024-04-23T02:13:53.488 INFO:tasks.workunit.client.0.smithi071.stdout:3/111: mkdir d7/d27 0 2024-04-23T02:13:53.488 INFO:tasks.workunit.client.0.smithi071.stdout:6/117: stat d6/c10 0 2024-04-23T02:13:53.490 INFO:tasks.workunit.client.0.smithi071.stdout:9/178: creat d1/f37 x:0 0 0 2024-04-23T02:13:53.491 INFO:tasks.workunit.client.0.smithi071.stdout:8/222: creat dc/d15/d2d/f3f x:0 0 0 2024-04-23T02:13:53.493 INFO:tasks.workunit.client.0.smithi071.stdout:0/162: mknod d0/d3/de/d10/d14/d32/d37/d26/c39 0 2024-04-23T02:13:53.493 INFO:tasks.workunit.client.0.smithi071.stdout:3/112: chown d7/fd 633184 1 2024-04-23T02:13:53.493 INFO:tasks.workunit.client.0.smithi071.stdout:6/118: stat d6/c10 0 2024-04-23T02:13:53.493 INFO:tasks.workunit.client.0.smithi071.stdout:5/190: write d3/df/d10/f14 [1817624,26039] 0 2024-04-23T02:13:53.494 INFO:tasks.workunit.client.0.smithi071.stdout:4/165: dwrite d2/d15/d1d/d23/d10/f13 [0,4194304] 0 2024-04-23T02:13:53.494 INFO:tasks.workunit.client.0.smithi071.stdout:0/163: chown d0/d3/d1c 121382401 1 2024-04-23T02:13:53.495 INFO:tasks.workunit.client.0.smithi071.stdout:8/223: symlink dc/d3a/l40 0 2024-04-23T02:13:53.496 INFO:tasks.workunit.client.1.smithi192.stdout:1/109: truncate fa 527657 0 2024-04-23T02:13:53.496 INFO:tasks.workunit.client.0.smithi071.stdout:8/224: read dc/ff [3571980,30160] 0 2024-04-23T02:13:53.498 INFO:tasks.workunit.client.1.smithi192.stdout:1/110: symlink dc/l1b 0 2024-04-23T02:13:53.498 INFO:tasks.workunit.client.0.smithi071.stdout:6/119: mknod d6/d17/c23 0 2024-04-23T02:13:53.499 INFO:tasks.workunit.client.1.smithi192.stdout:5/262: link d2/f5 d2/d6/f59 0 2024-04-23T02:13:53.501 INFO:tasks.workunit.client.1.smithi192.stdout:1/111: unlink dc/le 0 2024-04-23T02:13:53.501 INFO:tasks.workunit.client.0.smithi071.stdout:0/164: truncate d0/d3/de/d10/f1f 4794286 0 2024-04-23T02:13:53.502 INFO:tasks.workunit.client.0.smithi071.stdout:8/225: truncate dc/d15/f39 839565 0 2024-04-23T02:13:53.502 INFO:tasks.workunit.client.0.smithi071.stdout:3/113: truncate d7/d8/f11 578118 0 2024-04-23T02:13:53.504 INFO:tasks.workunit.client.0.smithi071.stdout:5/191: mkdir d3/d1d/d43/d4b 0 2024-04-23T02:13:53.504 INFO:tasks.workunit.client.1.smithi192.stdout:9/276: dwrite d0/d9/f26 [0,4194304] 0 2024-04-23T02:13:53.504 INFO:tasks.workunit.client.0.smithi071.stdout:5/192: stat d3/f1f 0 2024-04-23T02:13:53.507 INFO:tasks.workunit.client.1.smithi192.stdout:5/263: creat d2/db/d16/d1b/d3d/d40/f5a x:0 0 0 2024-04-23T02:13:53.507 INFO:tasks.workunit.client.1.smithi192.stdout:0/318: truncate d6/f1b 390266 0 2024-04-23T02:13:53.509 INFO:tasks.workunit.client.0.smithi071.stdout:0/165: symlink d0/d3/l3a 0 2024-04-23T02:13:53.509 INFO:tasks.workunit.client.1.smithi192.stdout:1/112: rmdir dc/d16 39 2024-04-23T02:13:53.509 INFO:tasks.workunit.client.1.smithi192.stdout:5/264: fsync d2/db/d16/d1b/d3d/f3f 0 2024-04-23T02:13:53.513 INFO:tasks.workunit.client.0.smithi071.stdout:0/166: write d0/d3/de/d10/d14/f1e [69368,32203] 0 2024-04-23T02:13:53.513 INFO:tasks.workunit.client.1.smithi192.stdout:9/277: rename d0/f33 to d0/d9/d44/d51/d5b/f69 0 2024-04-23T02:13:53.514 INFO:tasks.workunit.client.0.smithi071.stdout:3/114: mkdir d7/d28 0 2024-04-23T02:13:53.515 INFO:tasks.workunit.client.0.smithi071.stdout:0/167: write d0/d3/f2d [1473902,79334] 0 2024-04-23T02:13:53.516 INFO:tasks.workunit.client.0.smithi071.stdout:3/115: write d7/d10/f1e [693586,83170] 0 2024-04-23T02:13:53.517 INFO:tasks.workunit.client.1.smithi192.stdout:5/265: unlink d2/c17 0 2024-04-23T02:13:53.517 INFO:tasks.workunit.client.1.smithi192.stdout:1/113: dread dc/fd [4194304,4194304] 0 2024-04-23T02:13:53.519 INFO:tasks.workunit.client.1.smithi192.stdout:5/266: write d2/db/d16/f26 [2613803,34239] 0 2024-04-23T02:13:53.520 INFO:tasks.workunit.client.0.smithi071.stdout:5/193: creat d3/da/d1a/f4c x:0 0 0 2024-04-23T02:13:53.521 INFO:tasks.workunit.client.0.smithi071.stdout:3/116: mknod d7/d22/c29 0 2024-04-23T02:13:53.522 INFO:tasks.workunit.client.1.smithi192.stdout:0/319: creat d6/df/d13/d6b/f6e x:0 0 0 2024-04-23T02:13:53.523 INFO:tasks.workunit.client.0.smithi071.stdout:5/194: creat d3/da/f4d x:0 0 0 2024-04-23T02:13:53.524 INFO:tasks.workunit.client.1.smithi192.stdout:0/320: chown d6/df/d13/d25/d26/d67/l6a 168655419 1 2024-04-23T02:13:53.525 INFO:tasks.workunit.client.0.smithi071.stdout:3/117: symlink d7/d22/l2a 0 2024-04-23T02:13:53.528 INFO:tasks.workunit.client.1.smithi192.stdout:9/278: rmdir d0/d9/d44/d51 39 2024-04-23T02:13:53.528 INFO:tasks.workunit.client.1.smithi192.stdout:1/114: symlink dc/l1c 0 2024-04-23T02:13:53.529 INFO:tasks.workunit.client.0.smithi071.stdout:9/179: write d1/f30 [2104197,58796] 0 2024-04-23T02:13:53.530 INFO:tasks.workunit.client.0.smithi071.stdout:3/118: creat d7/d22/d23/d26/f2b x:0 0 0 2024-04-23T02:13:53.531 INFO:tasks.workunit.client.0.smithi071.stdout:8/226: truncate dc/d10/f1c 2520483 0 2024-04-23T02:13:53.531 INFO:tasks.workunit.client.1.smithi192.stdout:5/267: link d2/f2a d2/db/d2b/f5b 0 2024-04-23T02:13:53.532 INFO:tasks.workunit.client.0.smithi071.stdout:4/166: dread d2/d15/d1d/f35 [0,4194304] 0 2024-04-23T02:13:53.533 INFO:tasks.workunit.client.1.smithi192.stdout:0/321: rename d6/f3b to d6/df/d13/d25/d3a/f6f 0 2024-04-23T02:13:53.534 INFO:tasks.workunit.client.0.smithi071.stdout:9/180: creat d1/d1a/f38 x:0 0 0 2024-04-23T02:13:53.535 INFO:tasks.workunit.client.0.smithi071.stdout:6/120: dwrite d6/f14 [4194304,4194304] 0 2024-04-23T02:13:53.538 INFO:tasks.workunit.client.0.smithi071.stdout:8/227: mknod dc/d10/d32/c41 0 2024-04-23T02:13:53.539 INFO:tasks.workunit.client.0.smithi071.stdout:3/119: truncate d7/d8/f16 797285 0 2024-04-23T02:13:53.540 INFO:tasks.workunit.client.0.smithi071.stdout:8/228: fsync dc/f28 0 2024-04-23T02:13:53.541 INFO:tasks.workunit.client.1.smithi192.stdout:1/115: unlink dc/d16/d19/f1a 0 2024-04-23T02:13:53.543 INFO:tasks.workunit.client.1.smithi192.stdout:5/268: creat d2/db/d16/d48/d4f/f5c x:0 0 0 2024-04-23T02:13:53.544 INFO:tasks.workunit.client.0.smithi071.stdout:5/195: dwrite d3/d1d/f20 [0,4194304] 0 2024-04-23T02:13:53.544 INFO:tasks.workunit.client.1.smithi192.stdout:0/322: creat d6/df/d47/d55/f70 x:0 0 0 2024-04-23T02:13:53.544 INFO:tasks.workunit.client.1.smithi192.stdout:5/269: dread - d2/d32/f35 zero size 2024-04-23T02:13:53.546 INFO:tasks.workunit.client.1.smithi192.stdout:9/279: unlink d0/d9/d44/d4f/l3b 0 2024-04-23T02:13:53.548 INFO:tasks.workunit.client.0.smithi071.stdout:6/121: mknod d6/d1f/c24 0 2024-04-23T02:13:53.549 INFO:tasks.workunit.client.1.smithi192.stdout:1/116: dwrite f4 [0,4194304] 0 2024-04-23T02:13:53.550 INFO:tasks.workunit.client.1.smithi192.stdout:0/323: truncate d6/df/d47/f61 4287916 0 2024-04-23T02:13:53.550 INFO:tasks.workunit.client.0.smithi071.stdout:0/168: dwrite d0/f1 [0,4194304] 0 2024-04-23T02:13:53.552 INFO:tasks.workunit.client.1.smithi192.stdout:5/270: symlink d2/db/d16/d1b/d3d/d40/d49/l5d 0 2024-04-23T02:13:53.552 INFO:tasks.workunit.client.1.smithi192.stdout:9/280: rmdir d0/d9/d44/d51 39 2024-04-23T02:13:53.553 INFO:tasks.workunit.client.0.smithi071.stdout:6/122: write d6/fb [6946922,106374] 0 2024-04-23T02:13:53.554 INFO:tasks.workunit.client.1.smithi192.stdout:0/324: mkdir d6/d44/d71 0 2024-04-23T02:13:53.554 INFO:tasks.workunit.client.0.smithi071.stdout:6/123: readlink d6/l15 0 2024-04-23T02:13:53.558 INFO:tasks.workunit.client.0.smithi071.stdout:0/169: write d0/d3/f2d [2363291,76792] 0 2024-04-23T02:13:53.559 INFO:tasks.workunit.client.1.smithi192.stdout:1/117: mknod dc/d14/d15/c1d 0 2024-04-23T02:13:53.559 INFO:tasks.workunit.client.1.smithi192.stdout:9/281: unlink d0/f56 0 2024-04-23T02:13:53.560 INFO:tasks.workunit.client.0.smithi071.stdout:9/181: write d1/f28 [627079,5434] 0 2024-04-23T02:13:53.561 INFO:tasks.workunit.client.0.smithi071.stdout:4/167: dwrite d2/d15/d1d/f35 [0,4194304] 0 2024-04-23T02:13:53.562 INFO:tasks.workunit.client.0.smithi071.stdout:3/120: write d7/d8/fc [807070,22252] 0 2024-04-23T02:13:53.562 INFO:tasks.workunit.client.1.smithi192.stdout:5/271: rename d2/db/d16/d48/l50 to d2/db/l5e 0 2024-04-23T02:13:53.563 INFO:tasks.workunit.client.0.smithi071.stdout:9/182: fsync d1/fa 0 2024-04-23T02:13:53.570 INFO:tasks.workunit.client.0.smithi071.stdout:4/168: fsync d2/d15/d1d/d23/fc 0 2024-04-23T02:13:53.570 INFO:tasks.workunit.client.0.smithi071.stdout:6/124: dread d6/ff [0,4194304] 0 2024-04-23T02:13:53.570 INFO:tasks.workunit.client.0.smithi071.stdout:4/169: stat d2/d15/d1d/d23/d17 0 2024-04-23T02:13:53.571 INFO:tasks.workunit.client.1.smithi192.stdout:1/118: rename l8 to dc/l1e 0 2024-04-23T02:13:53.573 INFO:tasks.workunit.client.0.smithi071.stdout:6/125: chown f3 7376823 1 2024-04-23T02:13:53.574 INFO:tasks.workunit.client.0.smithi071.stdout:6/126: readlink d6/l15 0 2024-04-23T02:13:53.574 INFO:tasks.workunit.client.0.smithi071.stdout:6/127: write d6/fc [4565050,62520] 0 2024-04-23T02:13:53.575 INFO:tasks.workunit.client.1.smithi192.stdout:9/282: read d0/d9/d44/d4f/f2e [3982038,25262] 0 2024-04-23T02:13:53.577 INFO:tasks.workunit.client.1.smithi192.stdout:1/119: link dc/f11 dc/d16/f1f 0 2024-04-23T02:13:53.577 INFO:tasks.workunit.client.0.smithi071.stdout:0/170: fdatasync d0/d3/de/f1d 0 2024-04-23T02:13:53.577 INFO:tasks.workunit.client.1.smithi192.stdout:1/120: stat f9 0 2024-04-23T02:13:53.578 INFO:tasks.workunit.client.0.smithi071.stdout:9/183: mkdir d1/d1a/d39 0 2024-04-23T02:13:53.578 INFO:tasks.workunit.client.0.smithi071.stdout:9/184: readlink d1/l15 0 2024-04-23T02:13:53.578 INFO:tasks.workunit.client.0.smithi071.stdout:5/196: write d3/da/d1a/d1c/f1e [1449687,103970] 0 2024-04-23T02:13:53.580 INFO:tasks.workunit.client.0.smithi071.stdout:5/197: stat f2 0 2024-04-23T02:13:53.581 INFO:tasks.workunit.client.0.smithi071.stdout:9/185: fsync d1/d1a/f2d 0 2024-04-23T02:13:53.581 INFO:tasks.workunit.client.1.smithi192.stdout:9/283: creat d0/d9/d44/d51/d5b/d61/f6a x:0 0 0 2024-04-23T02:13:53.582 INFO:tasks.workunit.client.0.smithi071.stdout:6/128: dread d6/fa [0,4194304] 0 2024-04-23T02:13:53.582 INFO:tasks.workunit.client.0.smithi071.stdout:0/171: write d0/d3/de/d10/f1f [2771993,36501] 0 2024-04-23T02:13:53.584 INFO:tasks.workunit.client.0.smithi071.stdout:0/172: chown d0/f18 1044326297 1 2024-04-23T02:13:53.584 INFO:tasks.workunit.client.0.smithi071.stdout:4/170: link d2/d15/d1d/d23/d10/l26 d2/d2a/d2d/l37 0 2024-04-23T02:13:53.585 INFO:tasks.workunit.client.0.smithi071.stdout:9/186: mkdir d1/d32/d3a 0 2024-04-23T02:13:53.586 INFO:tasks.workunit.client.1.smithi192.stdout:1/121: dwrite f9 [0,4194304] 0 2024-04-23T02:13:53.586 INFO:tasks.workunit.client.0.smithi071.stdout:5/198: mkdir d3/da/d1a/d1c/d46/d4e 0 2024-04-23T02:13:53.588 INFO:tasks.workunit.client.0.smithi071.stdout:5/199: chown d3/da/f33 163 1 2024-04-23T02:13:53.589 INFO:tasks.workunit.client.0.smithi071.stdout:8/229: dread dc/d10/d32/f3b [0,4194304] 0 2024-04-23T02:13:53.590 INFO:tasks.workunit.client.0.smithi071.stdout:6/129: rename d6/fa to d6/d1f/f25 0 2024-04-23T02:13:53.591 INFO:tasks.workunit.client.0.smithi071.stdout:8/230: readlink dc/l1f 0 2024-04-23T02:13:53.592 INFO:tasks.workunit.client.0.smithi071.stdout:4/171: fsync d2/d15/d1d/d23/f16 0 2024-04-23T02:13:53.593 INFO:tasks.workunit.client.1.smithi192.stdout:9/284: dwrite d0/f16 [4194304,4194304] 0 2024-04-23T02:13:53.593 INFO:tasks.workunit.client.0.smithi071.stdout:6/130: read d6/fc [1951950,98631] 0 2024-04-23T02:13:53.595 INFO:tasks.workunit.client.0.smithi071.stdout:0/173: dread d0/d3/de/d10/d14/f1e [0,4194304] 0 2024-04-23T02:13:53.595 INFO:tasks.workunit.client.0.smithi071.stdout:5/200: unlink d3/l44 0 2024-04-23T02:13:53.595 INFO:tasks.workunit.client.0.smithi071.stdout:4/172: chown d2/d15/d1d/d23/ff 1260431534 1 2024-04-23T02:13:53.596 INFO:tasks.workunit.client.0.smithi071.stdout:6/131: write d6/f16 [3984227,14275] 0 2024-04-23T02:13:53.599 INFO:tasks.workunit.client.0.smithi071.stdout:9/187: creat d1/d32/d3a/f3b x:0 0 0 2024-04-23T02:13:53.604 INFO:tasks.workunit.client.0.smithi071.stdout:5/201: chown d3/da/d1a/f26 1594 1 2024-04-23T02:13:53.604 INFO:tasks.workunit.client.0.smithi071.stdout:4/173: unlink d2/f4 0 2024-04-23T02:13:53.607 INFO:tasks.workunit.client.0.smithi071.stdout:5/202: creat d3/da/f4f x:0 0 0 2024-04-23T02:13:53.607 INFO:tasks.workunit.client.1.smithi192.stdout:9/285: dwrite d0/d9/d44/d4f/f2e [0,4194304] 0 2024-04-23T02:13:53.608 INFO:tasks.workunit.client.0.smithi071.stdout:4/174: creat d2/d15/d1d/d23/f38 x:0 0 0 2024-04-23T02:13:53.609 INFO:tasks.workunit.client.0.smithi071.stdout:3/121: dread f0 [0,4194304] 0 2024-04-23T02:13:53.610 INFO:tasks.workunit.client.0.smithi071.stdout:9/188: dwrite d1/fb [4194304,4194304] 0 2024-04-23T02:13:53.610 INFO:tasks.workunit.client.1.smithi192.stdout:5/272: dwrite d2/db/d16/d1b/f1f [0,4194304] 0 2024-04-23T02:13:53.612 INFO:tasks.workunit.client.1.smithi192.stdout:5/273: dread - d2/db/d16/d48/d4f/f5c zero size 2024-04-23T02:13:53.619 INFO:tasks.workunit.client.0.smithi071.stdout:5/203: dread d3/da/d1a/f1b [0,4194304] 0 2024-04-23T02:13:53.620 INFO:tasks.workunit.client.0.smithi071.stdout:3/122: rmdir d7/d22/d23 39 2024-04-23T02:13:53.620 INFO:tasks.workunit.client.0.smithi071.stdout:4/175: unlink d2/d15/d1d/d2c/f33 0 2024-04-23T02:13:53.622 INFO:tasks.workunit.client.0.smithi071.stdout:9/189: mknod d1/d32/c3c 0 2024-04-23T02:13:53.623 INFO:tasks.workunit.client.0.smithi071.stdout:5/204: chown d3/da/d1a/f2b 89 1 2024-04-23T02:13:53.623 INFO:tasks.workunit.client.0.smithi071.stdout:9/190: read - d1/d1a/f36 zero size 2024-04-23T02:13:53.625 INFO:tasks.workunit.client.0.smithi071.stdout:5/205: chown d3/df/f17 14944 1 2024-04-23T02:13:53.625 INFO:tasks.workunit.client.0.smithi071.stdout:6/132: dwrite d6/d1f/f25 [4194304,4194304] 0 2024-04-23T02:13:53.626 INFO:tasks.workunit.client.0.smithi071.stdout:9/191: truncate d1/f9 1414840 0 2024-04-23T02:13:53.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:53 smithi192 ceph-mon[49662]: pgmap v23: 65 pgs: 65 active+clean; 398 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 16 MiB/s wr, 46 op/s 2024-04-23T02:13:53.627 INFO:tasks.workunit.client.0.smithi071.stdout:5/206: stat d3/d1d/l36 0 2024-04-23T02:13:53.628 INFO:tasks.workunit.client.0.smithi071.stdout:4/176: rmdir d2/d15/d1d 39 2024-04-23T02:13:53.629 INFO:tasks.workunit.client.0.smithi071.stdout:0/174: write d0/d3/de/ff [4137275,107561] 0 2024-04-23T02:13:53.629 INFO:tasks.workunit.client.1.smithi192.stdout:8/250: sync 2024-04-23T02:13:53.630 INFO:tasks.workunit.client.0.smithi071.stdout:9/192: creat d1/f3d x:0 0 0 2024-04-23T02:13:53.630 INFO:tasks.workunit.client.0.smithi071.stdout:3/123: mkdir d7/d27/d2c 0 2024-04-23T02:13:53.632 INFO:tasks.workunit.client.1.smithi192.stdout:8/251: creat d1/d9/d3e/f55 x:0 0 0 2024-04-23T02:13:53.632 INFO:tasks.workunit.client.0.smithi071.stdout:6/133: creat d6/d22/f26 x:0 0 0 2024-04-23T02:13:53.633 INFO:tasks.workunit.client.0.smithi071.stdout:9/193: write d1/fa [513083,22893] 0 2024-04-23T02:13:53.635 INFO:tasks.workunit.client.1.smithi192.stdout:8/252: symlink d1/df/l56 0 2024-04-23T02:13:53.636 INFO:tasks.workunit.client.0.smithi071.stdout:5/207: creat d3/da/d1a/d1c/d46/d4e/f50 x:0 0 0 2024-04-23T02:13:53.636 INFO:tasks.workunit.client.0.smithi071.stdout:4/177: write d2/d15/d1d/f35 [2305233,106143] 0 2024-04-23T02:13:53.636 INFO:tasks.workunit.client.0.smithi071.stdout:6/134: dread - d6/f12 zero size 2024-04-23T02:13:53.636 INFO:tasks.workunit.client.0.smithi071.stdout:5/208: readlink d3/lc 0 2024-04-23T02:13:53.638 INFO:tasks.workunit.client.1.smithi192.stdout:8/253: rename d1/d12/d23/d2c/l29 to d1/d12/d23/d2c/l57 0 2024-04-23T02:13:53.639 INFO:tasks.workunit.client.0.smithi071.stdout:9/194: rename d1/l15 to d1/d1a/l3e 0 2024-04-23T02:13:53.641 INFO:tasks.workunit.client.1.smithi192.stdout:8/254: read d1/d12/d23/f37 [18564,84406] 0 2024-04-23T02:13:53.641 INFO:tasks.workunit.client.0.smithi071.stdout:9/195: write d1/d32/d3a/f3b [44728,76678] 0 2024-04-23T02:13:53.642 INFO:tasks.workunit.client.0.smithi071.stdout:4/178: mkdir d2/d2a/d2d/d39 0 2024-04-23T02:13:53.644 INFO:tasks.workunit.client.0.smithi071.stdout:5/209: unlink d3/d1d/d2c/c35 0 2024-04-23T02:13:53.645 INFO:tasks.workunit.client.0.smithi071.stdout:9/196: symlink d1/d32/d3a/l3f 0 2024-04-23T02:13:53.645 INFO:tasks.workunit.client.0.smithi071.stdout:4/179: stat d2/d15/d1d/f35 0 2024-04-23T02:13:53.647 INFO:tasks.workunit.client.0.smithi071.stdout:3/124: dwrite d7/d22/f25 [0,4194304] 0 2024-04-23T02:13:53.649 INFO:tasks.workunit.client.0.smithi071.stdout:9/197: creat d1/d32/f40 x:0 0 0 2024-04-23T02:13:53.650 INFO:tasks.workunit.client.0.smithi071.stdout:3/125: symlink d7/d10/l2d 0 2024-04-23T02:13:53.652 INFO:tasks.workunit.client.0.smithi071.stdout:9/198: write d1/d32/d3a/f3b [468136,49996] 0 2024-04-23T02:13:53.653 INFO:tasks.workunit.client.0.smithi071.stdout:6/135: dwrite d6/d1f/f21 [0,4194304] 0 2024-04-23T02:13:53.656 INFO:tasks.workunit.client.0.smithi071.stdout:9/199: mkdir d1/d32/d41 0 2024-04-23T02:13:53.657 INFO:tasks.workunit.client.0.smithi071.stdout:3/126: dread f0 [0,4194304] 0 2024-04-23T02:13:53.657 INFO:tasks.workunit.client.0.smithi071.stdout:6/136: mknod d6/d1f/c27 0 2024-04-23T02:13:53.659 INFO:tasks.workunit.client.0.smithi071.stdout:9/200: mkdir d1/d1a/d39/d42 0 2024-04-23T02:13:53.660 INFO:tasks.workunit.client.0.smithi071.stdout:9/201: readlink d1/d32/l34 0 2024-04-23T02:13:53.660 INFO:tasks.workunit.client.0.smithi071.stdout:6/137: creat d6/d22/f28 x:0 0 0 2024-04-23T02:13:53.660 INFO:tasks.workunit.client.0.smithi071.stdout:9/202: chown d1/d1a/c2e 3 1 2024-04-23T02:13:53.670 INFO:tasks.workunit.client.1.smithi192.stdout:8/255: dwrite d1/df/f50 [0,4194304] 0 2024-04-23T02:13:53.672 INFO:tasks.workunit.client.1.smithi192.stdout:5/274: dread d2/db/f13 [0,4194304] 0 2024-04-23T02:13:53.672 INFO:tasks.workunit.client.0.smithi071.stdout:3/127: dread d7/d8/fc [0,4194304] 0 2024-04-23T02:13:53.675 INFO:tasks.workunit.client.1.smithi192.stdout:8/256: chown d1/d12/d23/c44 1273 1 2024-04-23T02:13:53.675 INFO:tasks.workunit.client.1.smithi192.stdout:5/275: dread - d2/db/d16/d48/d4f/f5c zero size 2024-04-23T02:13:53.676 INFO:tasks.workunit.client.0.smithi071.stdout:0/175: truncate d0/d3/d1c/f21 455793 0 2024-04-23T02:13:53.679 INFO:tasks.workunit.client.0.smithi071.stdout:5/210: write d3/da/f33 [234060,72978] 0 2024-04-23T02:13:53.681 INFO:tasks.workunit.client.0.smithi071.stdout:3/128: dwrite d7/d8/fe [4194304,4194304] 0 2024-04-23T02:13:53.682 INFO:tasks.workunit.client.1.smithi192.stdout:9/286: dread d0/f34 [0,4194304] 0 2024-04-23T02:13:53.682 INFO:tasks.workunit.client.0.smithi071.stdout:0/176: write d0/d3/de/d10/f1f [2704417,112694] 0 2024-04-23T02:13:53.683 INFO:tasks.workunit.client.1.smithi192.stdout:9/287: truncate d0/d1e/f4c 354276 0 2024-04-23T02:13:53.684 INFO:tasks.workunit.client.0.smithi071.stdout:6/138: getdents d6/d22 0 2024-04-23T02:13:53.686 INFO:tasks.workunit.client.1.smithi192.stdout:8/257: dwrite d1/f10 [4194304,4194304] 0 2024-04-23T02:13:53.686 INFO:tasks.workunit.client.0.smithi071.stdout:5/211: rename d3/l47 to d3/da/d1a/d1c/d4a/l51 0 2024-04-23T02:13:53.687 INFO:tasks.workunit.client.0.smithi071.stdout:0/177: symlink d0/d3/de/d10/l3b 0 2024-04-23T02:13:53.688 INFO:tasks.workunit.client.1.smithi192.stdout:8/258: symlink d1/d9/l58 0 2024-04-23T02:13:53.689 INFO:tasks.workunit.client.0.smithi071.stdout:6/139: creat d6/d22/f29 x:0 0 0 2024-04-23T02:13:53.689 INFO:tasks.workunit.client.0.smithi071.stdout:0/178: rename d0/d3 to d0/d3/de/d10/d14/d32/d37/d3c 22 2024-04-23T02:13:53.689 INFO:tasks.workunit.client.0.smithi071.stdout:5/212: mknod d3/da/d1a/d1c/c52 0 2024-04-23T02:13:53.691 INFO:tasks.workunit.client.0.smithi071.stdout:6/140: mknod d6/d1f/c2a 0 2024-04-23T02:13:53.691 INFO:tasks.workunit.client.1.smithi192.stdout:8/259: mkdir d1/d9/d3e/d59 0 2024-04-23T02:13:53.692 INFO:tasks.workunit.client.0.smithi071.stdout:0/179: creat d0/d3/f3d x:0 0 0 2024-04-23T02:13:53.693 INFO:tasks.workunit.client.0.smithi071.stdout:5/213: creat d3/d1d/d43/d4b/f53 x:0 0 0 2024-04-23T02:13:53.696 INFO:tasks.workunit.client.0.smithi071.stdout:6/141: getdents d6/d17 0 2024-04-23T02:13:53.697 INFO:tasks.workunit.client.0.smithi071.stdout:5/214: mkdir d3/d1d/d43/d4b/d54 0 2024-04-23T02:13:53.697 INFO:tasks.workunit.client.0.smithi071.stdout:6/142: chown d6/f8 822003 1 2024-04-23T02:13:53.698 INFO:tasks.workunit.client.0.smithi071.stdout:6/143: chown f3 99077194 1 2024-04-23T02:13:53.699 INFO:tasks.workunit.client.0.smithi071.stdout:5/215: mknod d3/d1d/d43/d4b/d54/c55 0 2024-04-23T02:13:53.699 INFO:tasks.workunit.client.1.smithi192.stdout:8/260: dwrite d1/d9/f22 [0,4194304] 0 2024-04-23T02:13:53.700 INFO:tasks.workunit.client.1.smithi192.stdout:8/261: write d1/df/f50 [3450394,5672] 0 2024-04-23T02:13:53.702 INFO:tasks.workunit.client.0.smithi071.stdout:5/216: symlink d3/da/d1a/d1c/d46/d4e/l56 0 2024-04-23T02:13:53.702 INFO:tasks.workunit.client.0.smithi071.stdout:6/144: dread d6/f9 [0,4194304] 0 2024-04-23T02:13:53.704 INFO:tasks.workunit.client.0.smithi071.stdout:5/217: mknod d3/da/d1a/d1c/c57 0 2024-04-23T02:13:53.708 INFO:tasks.workunit.client.1.smithi192.stdout:8/262: dwrite d1/d12/f39 [0,4194304] 0 2024-04-23T02:13:53.715 INFO:tasks.workunit.client.1.smithi192.stdout:8/263: link d1/d9/f3f d1/d12/d23/d4b/f5a 0 2024-04-23T02:13:53.716 INFO:tasks.workunit.client.1.smithi192.stdout:5/276: dread d2/d6/f12 [0,4194304] 0 2024-04-23T02:13:53.717 INFO:tasks.workunit.client.1.smithi192.stdout:8/264: fsync d1/d12/d23/f3c 0 2024-04-23T02:13:53.718 INFO:tasks.workunit.client.1.smithi192.stdout:8/265: stat d1/d12/d23/f3c 0 2024-04-23T02:13:53.719 INFO:tasks.workunit.client.0.smithi071.stdout:1/79: sync 2024-04-23T02:13:53.720 INFO:tasks.workunit.client.1.smithi192.stdout:5/277: creat d2/d2e/f5f x:0 0 0 2024-04-23T02:13:53.721 INFO:tasks.workunit.client.0.smithi071.stdout:1/80: mknod db/df/d13/c14 0 2024-04-23T02:13:53.722 INFO:tasks.workunit.client.1.smithi192.stdout:5/278: dread - d2/db/d16/d1b/d3d/d40/f5a zero size 2024-04-23T02:13:53.722 INFO:tasks.workunit.client.1.smithi192.stdout:8/266: dread - d1/df/d41/d45/d46/f53 zero size 2024-04-23T02:13:53.723 INFO:tasks.workunit.client.0.smithi071.stdout:1/81: truncate db/fd 1762789 0 2024-04-23T02:13:53.724 INFO:tasks.workunit.client.1.smithi192.stdout:5/279: write d2/db/f13 [4112478,18014] 0 2024-04-23T02:13:53.724 INFO:tasks.workunit.client.0.smithi071.stdout:0/180: truncate d0/d3/de/ff 1639142 0 2024-04-23T02:13:53.725 INFO:tasks.workunit.client.1.smithi192.stdout:5/280: dread - d2/d32/f35 zero size 2024-04-23T02:13:53.726 INFO:tasks.workunit.client.0.smithi071.stdout:1/82: creat db/df/d13/f15 x:0 0 0 2024-04-23T02:13:53.727 INFO:tasks.workunit.client.1.smithi192.stdout:5/281: write d2/db/d16/d48/d4f/f5c [339563,123397] 0 2024-04-23T02:13:53.727 INFO:tasks.workunit.client.1.smithi192.stdout:8/267: dread d1/d12/f39 [0,4194304] 0 2024-04-23T02:13:53.728 INFO:tasks.workunit.client.1.smithi192.stdout:5/282: truncate d2/f2c 4314306 0 2024-04-23T02:13:53.728 INFO:tasks.workunit.client.1.smithi192.stdout:5/283: chown d2/d6/d2d/d41 38954571 1 2024-04-23T02:13:53.730 INFO:tasks.workunit.client.0.smithi071.stdout:6/145: write f4 [674287,9273] 0 2024-04-23T02:13:53.731 INFO:tasks.workunit.client.1.smithi192.stdout:5/284: mkdir d2/d6/d14/d60 0 2024-04-23T02:13:53.732 INFO:tasks.workunit.client.0.smithi071.stdout:1/83: dread fa [0,4194304] 0 2024-04-23T02:13:53.733 INFO:tasks.workunit.client.0.smithi071.stdout:6/146: stat d6/d22/f29 0 2024-04-23T02:13:53.733 INFO:tasks.workunit.client.0.smithi071.stdout:0/181: getdents d0/d3/de/d10/d14/d32/d37/d26 0 2024-04-23T02:13:53.733 INFO:tasks.workunit.client.0.smithi071.stdout:6/147: stat f3 0 2024-04-23T02:13:53.733 INFO:tasks.workunit.client.0.smithi071.stdout:5/218: write d3/da/d1a/f1b [1096976,90362] 0 2024-04-23T02:13:53.734 INFO:tasks.workunit.client.1.smithi192.stdout:5/285: read d2/db/f13 [2009590,39039] 0 2024-04-23T02:13:53.735 INFO:tasks.workunit.client.0.smithi071.stdout:6/148: chown d6/c10 1615052 1 2024-04-23T02:13:53.735 INFO:tasks.workunit.client.0.smithi071.stdout:1/84: rename c4 to db/d11/c16 0 2024-04-23T02:13:53.735 INFO:tasks.workunit.client.0.smithi071.stdout:0/182: stat d0/d3/de/d10/d14/d2b/d31 0 2024-04-23T02:13:53.736 INFO:tasks.workunit.client.0.smithi071.stdout:6/149: write d6/d1f/f25 [5987793,83946] 0 2024-04-23T02:13:53.736 INFO:tasks.workunit.client.1.smithi192.stdout:8/268: dwrite d1/f1d [0,4194304] 0 2024-04-23T02:13:53.737 INFO:tasks.workunit.client.0.smithi071.stdout:6/150: chown d6/l15 0 1 2024-04-23T02:13:53.737 INFO:tasks.workunit.client.1.smithi192.stdout:8/269: chown d1/d12 0 1 2024-04-23T02:13:53.738 INFO:tasks.workunit.client.0.smithi071.stdout:6/151: chown d6/d17 1 1 2024-04-23T02:13:53.738 INFO:tasks.workunit.client.1.smithi192.stdout:5/286: unlink d2/db/d16/d1b/d3d/d40/f5a 0 2024-04-23T02:13:53.740 INFO:tasks.workunit.client.0.smithi071.stdout:5/219: rename d3/df/f17 to d3/da/d1a/d1c/d4a/f58 0 2024-04-23T02:13:53.740 INFO:tasks.workunit.client.0.smithi071.stdout:0/183: rmdir d0/d3/de/d10/d14/d2b/d31 39 2024-04-23T02:13:53.741 INFO:tasks.workunit.client.0.smithi071.stdout:5/220: dread - d3/da/d1a/f49 zero size 2024-04-23T02:13:53.741 INFO:tasks.workunit.client.0.smithi071.stdout:6/152: symlink d6/d22/l2b 0 2024-04-23T02:13:53.743 INFO:tasks.workunit.client.1.smithi192.stdout:8/270: symlink d1/l5b 0 2024-04-23T02:13:53.743 INFO:tasks.workunit.client.1.smithi192.stdout:5/287: write d2/d32/f35 [144624,6660] 0 2024-04-23T02:13:53.743 INFO:tasks.workunit.client.1.smithi192.stdout:8/271: chown d1/d12/d23/d2c/l27 7 1 2024-04-23T02:13:53.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:53 smithi071 ceph-mon[39837]: pgmap v23: 65 pgs: 65 active+clean; 398 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 16 MiB/s wr, 46 op/s 2024-04-23T02:13:53.747 INFO:tasks.workunit.client.1.smithi192.stdout:8/272: symlink d1/d12/d23/d4b/l5c 0 2024-04-23T02:13:53.748 INFO:tasks.workunit.client.0.smithi071.stdout:1/85: write f3 [1534242,88302] 0 2024-04-23T02:13:53.749 INFO:tasks.workunit.client.1.smithi192.stdout:8/273: mkdir d1/df/d41/d5d 0 2024-04-23T02:13:53.755 INFO:tasks.workunit.client.0.smithi071.stdout:1/86: dwrite f7 [0,4194304] 0 2024-04-23T02:13:53.755 INFO:tasks.workunit.client.0.smithi071.stdout:1/87: stat f0 0 2024-04-23T02:13:53.755 INFO:tasks.workunit.client.1.smithi192.stdout:5/288: truncate d2/f2c 140283 0 2024-04-23T02:13:53.756 INFO:tasks.workunit.client.1.smithi192.stdout:8/274: write d1/d9/f3f [445295,5784] 0 2024-04-23T02:13:53.757 INFO:tasks.workunit.client.1.smithi192.stdout:5/289: stat d2/d6/d14/l42 0 2024-04-23T02:13:53.760 INFO:tasks.workunit.client.1.smithi192.stdout:5/290: rename d2/db/d16/d1b/d3d/f3f to d2/db/d16/d1b/f61 0 2024-04-23T02:13:53.762 INFO:tasks.workunit.client.0.smithi071.stdout:1/88: dread f0 [0,4194304] 0 2024-04-23T02:13:53.763 INFO:tasks.workunit.client.1.smithi192.stdout:8/275: link d1/df/c2b d1/df/c5e 0 2024-04-23T02:13:53.764 INFO:tasks.workunit.client.0.smithi071.stdout:1/89: mknod db/df/d13/c17 0 2024-04-23T02:13:53.766 INFO:tasks.workunit.client.1.smithi192.stdout:5/291: dread d2/db/f11 [0,4194304] 0 2024-04-23T02:13:53.767 INFO:tasks.workunit.client.0.smithi071.stdout:0/184: dwrite d0/d3/de/ff [0,4194304] 0 2024-04-23T02:13:53.769 INFO:tasks.workunit.client.1.smithi192.stdout:5/292: chown d2/d32/f44 1 1 2024-04-23T02:13:53.770 INFO:tasks.workunit.client.1.smithi192.stdout:8/276: creat d1/df/d41/d5d/f5f x:0 0 0 2024-04-23T02:13:53.770 INFO:tasks.workunit.client.0.smithi071.stdout:6/153: dread d6/f8 [4194304,4194304] 0 2024-04-23T02:13:53.770 INFO:tasks.workunit.client.0.smithi071.stdout:1/90: rename db/df/d13/c14 to db/d11/c18 0 2024-04-23T02:13:53.770 INFO:tasks.workunit.client.0.smithi071.stdout:1/91: chown db/d11 1 1 2024-04-23T02:13:53.770 INFO:tasks.workunit.client.0.smithi071.stdout:0/185: creat d0/d3/de/d10/d16/f3e x:0 0 0 2024-04-23T02:13:53.771 INFO:tasks.workunit.client.0.smithi071.stdout:6/154: rmdir d6/d22 39 2024-04-23T02:13:53.772 INFO:tasks.workunit.client.0.smithi071.stdout:1/92: truncate db/df/d13/f15 373621 0 2024-04-23T02:13:53.772 INFO:tasks.workunit.client.1.smithi192.stdout:5/293: symlink d2/db/d2b/d52/l62 0 2024-04-23T02:13:53.773 INFO:tasks.workunit.client.0.smithi071.stdout:0/186: readlink d0/d3/d1c/l11 0 2024-04-23T02:13:53.773 INFO:tasks.workunit.client.1.smithi192.stdout:8/277: creat d1/d12/d23/f60 x:0 0 0 2024-04-23T02:13:53.776 INFO:tasks.workunit.client.1.smithi192.stdout:5/294: mknod d2/d2e/d4e/c63 0 2024-04-23T02:13:53.777 INFO:tasks.workunit.client.1.smithi192.stdout:8/278: stat d1/df/c2b 0 2024-04-23T02:13:53.777 INFO:tasks.workunit.client.0.smithi071.stdout:6/155: dread d6/f1d [4194304,4194304] 0 2024-04-23T02:13:53.777 INFO:tasks.workunit.client.1.smithi192.stdout:8/279: chown d1/c33 234 1 2024-04-23T02:13:53.778 INFO:tasks.workunit.client.1.smithi192.stdout:8/280: chown d1/f1d 1938079 1 2024-04-23T02:13:53.779 INFO:tasks.workunit.client.1.smithi192.stdout:5/295: creat d2/d2e/f64 x:0 0 0 2024-04-23T02:13:53.780 INFO:tasks.workunit.client.0.smithi071.stdout:1/93: dwrite f3 [0,4194304] 0 2024-04-23T02:13:53.780 INFO:tasks.workunit.client.0.smithi071.stdout:6/156: creat d6/d17/f2c x:0 0 0 2024-04-23T02:13:53.780 INFO:tasks.workunit.client.0.smithi071.stdout:0/187: dread d0/d3/d1c/f21 [0,4194304] 0 2024-04-23T02:13:53.782 INFO:tasks.workunit.client.0.smithi071.stdout:1/94: write db/df/d13/f15 [1175276,2161] 0 2024-04-23T02:13:53.782 INFO:tasks.workunit.client.0.smithi071.stdout:6/157: fdatasync f1 0 2024-04-23T02:13:53.783 INFO:tasks.workunit.client.1.smithi192.stdout:8/281: rename d1/d12/d23/d4b/l5c to d1/df/d41/d45/d46/l61 0 2024-04-23T02:13:53.783 INFO:tasks.workunit.client.0.smithi071.stdout:0/188: dread - d0/d3/de/d10/d16/f3e zero size 2024-04-23T02:13:53.784 INFO:tasks.workunit.client.0.smithi071.stdout:6/158: symlink d6/d1f/l2d 0 2024-04-23T02:13:53.785 INFO:tasks.workunit.client.0.smithi071.stdout:1/95: creat db/df/f19 x:0 0 0 2024-04-23T02:13:53.785 INFO:tasks.workunit.client.1.smithi192.stdout:8/282: mknod d1/c62 0 2024-04-23T02:13:53.786 INFO:tasks.workunit.client.0.smithi071.stdout:0/189: dread - d0/d3/de/d10/d14/d32/d37/d26/f2e zero size 2024-04-23T02:13:53.787 INFO:tasks.workunit.client.1.smithi192.stdout:8/283: stat d1/df/c2b 0 2024-04-23T02:13:53.787 INFO:tasks.workunit.client.0.smithi071.stdout:0/190: fsync d0/f1 0 2024-04-23T02:13:53.788 INFO:tasks.workunit.client.1.smithi192.stdout:5/296: dwrite d2/d2e/f5f [0,4194304] 0 2024-04-23T02:13:53.790 INFO:tasks.workunit.client.1.smithi192.stdout:8/284: creat d1/f63 x:0 0 0 2024-04-23T02:13:53.792 INFO:tasks.workunit.client.1.smithi192.stdout:8/285: truncate d1/d12/d23/d4b/f5a 875171 0 2024-04-23T02:13:53.793 INFO:tasks.workunit.client.0.smithi071.stdout:6/159: dwrite d6/ff [0,4194304] 0 2024-04-23T02:13:53.793 INFO:tasks.workunit.client.0.smithi071.stdout:1/96: dwrite db/fe [0,4194304] 0 2024-04-23T02:13:53.794 INFO:tasks.workunit.client.0.smithi071.stdout:1/97: stat db/cc 0 2024-04-23T02:13:53.796 INFO:tasks.workunit.client.1.smithi192.stdout:8/286: creat d1/df/d41/d4c/f64 x:0 0 0 2024-04-23T02:13:53.799 INFO:tasks.workunit.client.1.smithi192.stdout:8/287: creat d1/df/d41/f65 x:0 0 0 2024-04-23T02:13:53.801 INFO:tasks.workunit.client.1.smithi192.stdout:8/288: mknod d1/d12/d23/d4b/c66 0 2024-04-23T02:13:53.806 INFO:tasks.workunit.client.1.smithi192.stdout:8/289: write d1/df/d41/d5d/f5f [454901,82306] 0 2024-04-23T02:13:53.806 INFO:tasks.workunit.client.1.smithi192.stdout:5/297: dread d2/db/d16/d1b/f1f [0,4194304] 0 2024-04-23T02:13:53.807 INFO:tasks.workunit.client.1.smithi192.stdout:8/290: read - d1/d9/f26 zero size 2024-04-23T02:13:53.807 INFO:tasks.workunit.client.1.smithi192.stdout:5/298: chown d2/f27 130269 1 2024-04-23T02:13:53.808 INFO:tasks.workunit.client.1.smithi192.stdout:8/291: fsync d1/d12/d23/d2c/f38 0 2024-04-23T02:13:53.809 INFO:tasks.workunit.client.1.smithi192.stdout:8/292: read - d1/df/d41/f43 zero size 2024-04-23T02:13:53.810 INFO:tasks.workunit.client.1.smithi192.stdout:5/299: creat d2/db/d16/d1b/d3d/f65 x:0 0 0 2024-04-23T02:13:53.811 INFO:tasks.workunit.client.1.smithi192.stdout:8/293: write d1/d12/d23/f3c [4972777,44424] 0 2024-04-23T02:13:53.812 INFO:tasks.workunit.client.1.smithi192.stdout:5/300: dread - d2/db/d16/d1b/d3d/f65 zero size 2024-04-23T02:13:53.814 INFO:tasks.workunit.client.1.smithi192.stdout:5/301: symlink d2/db/d16/d1b/d3d/d40/l66 0 2024-04-23T02:13:53.819 INFO:tasks.workunit.client.1.smithi192.stdout:5/302: write d2/db/d16/d1b/f1f [910133,116715] 0 2024-04-23T02:13:53.825 INFO:tasks.workunit.client.1.smithi192.stdout:5/303: dread d2/f2c [0,4194304] 0 2024-04-23T02:13:53.829 INFO:tasks.workunit.client.1.smithi192.stdout:5/304: rename d2/d2e/f5f to d2/db/d16/d1b/d3d/f67 0 2024-04-23T02:13:53.832 INFO:tasks.workunit.client.1.smithi192.stdout:5/305: unlink d2/d6/f55 0 2024-04-23T02:13:53.836 INFO:tasks.workunit.client.1.smithi192.stdout:5/306: dread d2/db/f11 [0,4194304] 0 2024-04-23T02:13:53.836 INFO:tasks.workunit.client.1.smithi192.stdout:8/294: dread d1/f3 [0,4194304] 0 2024-04-23T02:13:53.838 INFO:tasks.workunit.client.1.smithi192.stdout:5/307: creat d2/d6/f68 x:0 0 0 2024-04-23T02:13:53.840 INFO:tasks.workunit.client.1.smithi192.stdout:5/308: mknod d2/d2e/d4e/c69 0 2024-04-23T02:13:53.847 INFO:tasks.workunit.client.1.smithi192.stdout:3/246: sync 2024-04-23T02:13:53.847 INFO:tasks.workunit.client.1.smithi192.stdout:7/215: sync 2024-04-23T02:13:53.848 INFO:tasks.workunit.client.1.smithi192.stdout:6/398: sync 2024-04-23T02:13:53.848 INFO:tasks.workunit.client.1.smithi192.stdout:4/401: sync 2024-04-23T02:13:53.848 INFO:tasks.workunit.client.1.smithi192.stdout:2/230: sync 2024-04-23T02:13:53.850 INFO:tasks.workunit.client.1.smithi192.stdout:2/231: write d8/d14/f1a [1900792,21805] 0 2024-04-23T02:13:53.852 INFO:tasks.workunit.client.1.smithi192.stdout:8/295: dread d1/d12/f1e [8388608,4194304] 0 2024-04-23T02:13:53.853 INFO:tasks.workunit.client.1.smithi192.stdout:7/216: write d5/d17/d1c/d30/f35 [302008,43636] 0 2024-04-23T02:13:53.853 INFO:tasks.workunit.client.1.smithi192.stdout:6/399: dread d3/d11/f1d [0,4194304] 0 2024-04-23T02:13:53.853 INFO:tasks.workunit.client.1.smithi192.stdout:4/402: write de/d15/d19/d1a/f6d [3016727,81519] 0 2024-04-23T02:13:53.854 INFO:tasks.workunit.client.1.smithi192.stdout:3/247: creat d1/d3/d17/d2b/f51 x:0 0 0 2024-04-23T02:13:53.857 INFO:tasks.workunit.client.1.smithi192.stdout:5/309: dread d2/fd [0,4194304] 0 2024-04-23T02:13:53.857 INFO:tasks.workunit.client.1.smithi192.stdout:7/217: write d5/d17/d1c/d30/f35 [265330,27654] 0 2024-04-23T02:13:53.857 INFO:tasks.workunit.client.1.smithi192.stdout:8/296: creat d1/df/d41/f67 x:0 0 0 2024-04-23T02:13:53.857 INFO:tasks.workunit.client.1.smithi192.stdout:2/232: creat d8/d26/d29/f4b x:0 0 0 2024-04-23T02:13:53.858 INFO:tasks.workunit.client.1.smithi192.stdout:6/400: truncate d3/d15/d1b/f77 768044 0 2024-04-23T02:13:53.859 INFO:tasks.workunit.client.1.smithi192.stdout:4/403: mknod de/d15/d19/d3d/d49/d70/c82 0 2024-04-23T02:13:53.859 INFO:tasks.workunit.client.1.smithi192.stdout:8/297: dread - d1/df/d41/f67 zero size 2024-04-23T02:13:53.860 INFO:tasks.workunit.client.1.smithi192.stdout:6/401: write d3/d11/d6d/f59 [209177,113712] 0 2024-04-23T02:13:53.861 INFO:tasks.workunit.client.1.smithi192.stdout:7/218: truncate d5/f15 479662 0 2024-04-23T02:13:53.862 INFO:tasks.workunit.client.1.smithi192.stdout:4/404: creat de/d15/d53/f83 x:0 0 0 2024-04-23T02:13:53.864 INFO:tasks.workunit.client.1.smithi192.stdout:6/402: fdatasync d3/d15/d1b/f72 0 2024-04-23T02:13:53.865 INFO:tasks.workunit.client.1.smithi192.stdout:4/405: write de/d15/d19/d1a/f23 [756088,40749] 0 2024-04-23T02:13:53.867 INFO:tasks.workunit.client.1.smithi192.stdout:6/403: creat d3/d15/d5b/f82 x:0 0 0 2024-04-23T02:13:53.869 INFO:tasks.workunit.client.1.smithi192.stdout:4/406: fdatasync de/f4d 0 2024-04-23T02:13:53.874 INFO:tasks.workunit.client.1.smithi192.stdout:4/407: fsync de/d15/d19/d3d/f54 0 2024-04-23T02:13:53.878 INFO:tasks.workunit.client.1.smithi192.stdout:8/298: write d1/d9/f22 [4875213,128743] 0 2024-04-23T02:13:53.879 INFO:tasks.workunit.client.1.smithi192.stdout:6/404: dwrite d3/d15/d1b/d21/f60 [0,4194304] 0 2024-04-23T02:13:53.881 INFO:tasks.workunit.client.1.smithi192.stdout:4/408: symlink de/d15/d19/d1a/l84 0 2024-04-23T02:13:53.884 INFO:tasks.workunit.client.1.smithi192.stdout:8/299: dread d1/df/d41/d45/d46/f47 [0,4194304] 0 2024-04-23T02:13:53.884 INFO:tasks.workunit.client.1.smithi192.stdout:7/219: dread d5/d17/d1c/d26/f28 [0,4194304] 0 2024-04-23T02:13:53.887 INFO:tasks.workunit.client.1.smithi192.stdout:8/300: mkdir d1/d12/d68 0 2024-04-23T02:13:53.891 INFO:tasks.workunit.client.1.smithi192.stdout:4/409: dwrite de/d15/d19/d1a/f23 [0,4194304] 0 2024-04-23T02:13:53.894 INFO:tasks.workunit.client.1.smithi192.stdout:6/405: dread d3/d15/d1b/d21/d52/d64/f65 [0,4194304] 0 2024-04-23T02:13:53.894 INFO:tasks.workunit.client.1.smithi192.stdout:7/220: dread d5/d17/d1c/d26/f28 [0,4194304] 0 2024-04-23T02:13:53.896 INFO:tasks.workunit.client.1.smithi192.stdout:7/221: rmdir d5/d17 39 2024-04-23T02:13:53.898 INFO:tasks.workunit.client.1.smithi192.stdout:8/301: dwrite d1/df/f50 [0,4194304] 0 2024-04-23T02:13:53.899 INFO:tasks.workunit.client.1.smithi192.stdout:4/410: truncate de/d15/d19/d3d/d34/d4a/f75 493623 0 2024-04-23T02:13:53.900 INFO:tasks.workunit.client.1.smithi192.stdout:4/411: chown de/d15/l4e 262530512 1 2024-04-23T02:13:53.901 INFO:tasks.workunit.client.1.smithi192.stdout:8/302: fsync d1/d12/d23/f37 0 2024-04-23T02:13:53.903 INFO:tasks.workunit.client.1.smithi192.stdout:6/406: dread d3/d18/f78 [0,4194304] 0 2024-04-23T02:13:53.906 INFO:tasks.workunit.client.1.smithi192.stdout:4/412: symlink de/d15/d19/d3d/d49/d70/l85 0 2024-04-23T02:13:53.906 INFO:tasks.workunit.client.1.smithi192.stdout:6/407: symlink d3/d6c/l83 0 2024-04-23T02:13:53.911 INFO:tasks.workunit.client.1.smithi192.stdout:7/222: dwrite d5/d17/d1c/d30/f35 [0,4194304] 0 2024-04-23T02:13:53.913 INFO:tasks.workunit.client.1.smithi192.stdout:4/413: creat de/d15/d19/d3d/d49/d26/d59/d6a/f86 x:0 0 0 2024-04-23T02:13:53.916 INFO:tasks.workunit.client.1.smithi192.stdout:4/414: stat de 0 2024-04-23T02:13:53.916 INFO:tasks.workunit.client.1.smithi192.stdout:8/303: dread d1/df/f1c [0,4194304] 0 2024-04-23T02:13:53.918 INFO:tasks.workunit.client.1.smithi192.stdout:4/415: readlink de/l6b 0 2024-04-23T02:13:53.918 INFO:tasks.workunit.client.0.smithi071.stdout:6/160: mknod d6/d22/c2e 0 2024-04-23T02:13:53.919 INFO:tasks.workunit.client.0.smithi071.stdout:1/98: getdents db/df/d13 0 2024-04-23T02:13:53.920 INFO:tasks.workunit.client.1.smithi192.stdout:6/408: dwrite d3/d11/d6d/f59 [0,4194304] 0 2024-04-23T02:13:53.920 INFO:tasks.workunit.client.0.smithi071.stdout:6/161: unlink f5 0 2024-04-23T02:13:53.920 INFO:tasks.workunit.client.1.smithi192.stdout:8/304: unlink d1/d9/f2a 0 2024-04-23T02:13:53.921 INFO:tasks.workunit.client.0.smithi071.stdout:6/162: chown f4 238677377 1 2024-04-23T02:13:53.924 INFO:tasks.workunit.client.1.smithi192.stdout:6/409: symlink d3/d15/d1b/d21/d52/l84 0 2024-04-23T02:13:53.930 INFO:tasks.workunit.client.0.smithi071.stdout:1/99: dread f5 [0,4194304] 0 2024-04-23T02:13:53.932 INFO:tasks.workunit.client.1.smithi192.stdout:7/223: dread d5/d17/d2f/f1d [0,4194304] 0 2024-04-23T02:13:53.933 INFO:tasks.workunit.client.1.smithi192.stdout:7/224: chown d5/f2e 50740439 1 2024-04-23T02:13:53.938 INFO:tasks.workunit.client.1.smithi192.stdout:8/305: rmdir d1/d9 39 2024-04-23T02:13:53.939 INFO:tasks.workunit.client.1.smithi192.stdout:6/410: truncate d3/d15/d1b/d21/d52/d64/f65 2504460 0 2024-04-23T02:13:53.940 INFO:tasks.workunit.client.1.smithi192.stdout:7/225: write d5/f15 [765630,121032] 0 2024-04-23T02:13:53.942 INFO:tasks.workunit.client.1.smithi192.stdout:7/226: rmdir d5/d17 39 2024-04-23T02:13:53.943 INFO:tasks.workunit.client.1.smithi192.stdout:6/411: creat d3/d15/d1b/d21/d52/f85 x:0 0 0 2024-04-23T02:13:53.947 INFO:tasks.workunit.client.1.smithi192.stdout:6/412: mknod d3/c86 0 2024-04-23T02:13:53.947 INFO:tasks.workunit.client.1.smithi192.stdout:7/227: creat d5/d17/d1c/f3e x:0 0 0 2024-04-23T02:13:53.947 INFO:tasks.workunit.client.1.smithi192.stdout:4/416: dread de/d15/d19/d3d/d49/f28 [0,4194304] 0 2024-04-23T02:13:53.949 INFO:tasks.workunit.client.1.smithi192.stdout:8/306: dwrite d1/df/d41/d5d/f5f [0,4194304] 0 2024-04-23T02:13:53.956 INFO:tasks.workunit.client.1.smithi192.stdout:8/307: truncate d1/fe 708428 0 2024-04-23T02:13:53.959 INFO:tasks.workunit.client.1.smithi192.stdout:6/413: rmdir d3/d15/d1b/d21/d52/d64 39 2024-04-23T02:13:53.961 INFO:tasks.workunit.client.1.smithi192.stdout:8/308: write d1/df/d41/d4c/f64 [373933,24141] 0 2024-04-23T02:13:53.963 INFO:tasks.workunit.client.1.smithi192.stdout:7/228: dwrite d5/d17/d2f/f1d [0,4194304] 0 2024-04-23T02:13:53.965 INFO:tasks.workunit.client.1.smithi192.stdout:8/309: read d1/df/f50 [281257,86214] 0 2024-04-23T02:13:53.968 INFO:tasks.workunit.client.1.smithi192.stdout:8/310: chown d1/f3 22 1 2024-04-23T02:13:53.970 INFO:tasks.workunit.client.1.smithi192.stdout:4/417: dwrite de/d15/d19/d3d/d49/d26/d59/d6a/f86 [0,4194304] 0 2024-04-23T02:13:53.973 INFO:tasks.workunit.client.1.smithi192.stdout:6/414: mknod d3/d18/c87 0 2024-04-23T02:13:53.973 INFO:tasks.workunit.client.1.smithi192.stdout:8/311: creat d1/d12/d23/d4b/f69 x:0 0 0 2024-04-23T02:13:53.973 INFO:tasks.workunit.client.1.smithi192.stdout:4/418: write de/f21 [1470846,18079] 0 2024-04-23T02:13:53.974 INFO:tasks.workunit.client.1.smithi192.stdout:7/229: mknod d5/d17/d1c/d30/d3d/c3f 0 2024-04-23T02:13:53.975 INFO:tasks.workunit.client.0.smithi071.stdout:2/142: sync 2024-04-23T02:13:53.975 INFO:tasks.workunit.client.0.smithi071.stdout:7/140: sync 2024-04-23T02:13:53.977 INFO:tasks.workunit.client.0.smithi071.stdout:6/163: mknod d6/d17/c2f 0 2024-04-23T02:13:53.977 INFO:tasks.workunit.client.0.smithi071.stdout:7/141: chown d3/d17/d1c/c28 112825 1 2024-04-23T02:13:53.977 INFO:tasks.workunit.client.0.smithi071.stdout:6/164: fsync f1 0 2024-04-23T02:13:53.978 INFO:tasks.workunit.client.1.smithi192.stdout:7/230: read d5/d17/d1c/f24 [109474,10487] 0 2024-04-23T02:13:53.981 INFO:tasks.workunit.client.0.smithi071.stdout:7/142: symlink d3/d17/d1c/l32 0 2024-04-23T02:13:53.983 INFO:tasks.workunit.client.1.smithi192.stdout:7/231: rename d5/l25 to d5/d17/d2f/l40 0 2024-04-23T02:13:53.984 INFO:tasks.workunit.client.0.smithi071.stdout:7/143: chown d3/d17/d1c/l32 1446003 1 2024-04-23T02:13:53.986 INFO:tasks.workunit.client.0.smithi071.stdout:2/143: dwrite d0/d2/d26/f18 [0,4194304] 0 2024-04-23T02:13:53.986 INFO:tasks.workunit.client.0.smithi071.stdout:6/165: dread d6/d1f/f25 [0,4194304] 0 2024-04-23T02:13:53.986 INFO:tasks.workunit.client.1.smithi192.stdout:7/232: chown d5/f2e 189966118 1 2024-04-23T02:13:53.988 INFO:tasks.workunit.client.0.smithi071.stdout:7/144: rmdir d3 39 2024-04-23T02:13:53.988 INFO:tasks.workunit.client.1.smithi192.stdout:7/233: stat d5/d17/d2f/l19 0 2024-04-23T02:13:53.989 INFO:tasks.workunit.client.1.smithi192.stdout:4/419: dwrite de/d15/d19/d3d/f46 [0,4194304] 0 2024-04-23T02:13:53.989 INFO:tasks.workunit.client.1.smithi192.stdout:8/312: dwrite d1/d9/d3e/f55 [0,4194304] 0 2024-04-23T02:13:53.991 INFO:tasks.workunit.client.0.smithi071.stdout:1/100: truncate f5 433263 0 2024-04-23T02:13:53.991 INFO:tasks.workunit.client.0.smithi071.stdout:6/166: write d6/ff [2497132,26046] 0 2024-04-23T02:13:53.995 INFO:tasks.workunit.client.0.smithi071.stdout:1/101: mkdir db/d1a 0 2024-04-23T02:13:53.996 INFO:tasks.workunit.client.1.smithi192.stdout:7/234: mkdir d5/d17/d2f/d31/d41 0 2024-04-23T02:13:53.996 INFO:tasks.workunit.client.0.smithi071.stdout:7/145: chown d3/l15 6 1 2024-04-23T02:13:53.997 INFO:tasks.workunit.client.1.smithi192.stdout:4/420: symlink de/d15/d19/d3d/d49/d70/d73/l87 0 2024-04-23T02:13:53.999 INFO:tasks.workunit.client.1.smithi192.stdout:8/313: mkdir d1/d12/d68/d6a 0 2024-04-23T02:13:54.003 INFO:tasks.workunit.client.1.smithi192.stdout:7/235: unlink d5/d17/d2f/f20 0 2024-04-23T02:13:54.004 INFO:tasks.workunit.client.0.smithi071.stdout:2/144: dwrite d0/d2/d26/d14/f1a [0,4194304] 0 2024-04-23T02:13:54.004 INFO:tasks.workunit.client.0.smithi071.stdout:1/102: creat db/df/d13/f1b x:0 0 0 2024-04-23T02:13:54.005 INFO:tasks.workunit.client.0.smithi071.stdout:6/167: dwrite d6/d1f/f21 [0,4194304] 0 2024-04-23T02:13:54.006 INFO:tasks.workunit.client.1.smithi192.stdout:4/421: creat de/d15/d2a/d61/f88 x:0 0 0 2024-04-23T02:13:54.007 INFO:tasks.workunit.client.0.smithi071.stdout:1/103: stat f3 0 2024-04-23T02:13:54.008 INFO:tasks.workunit.client.0.smithi071.stdout:7/146: creat d3/d17/d1c/d25/f33 x:0 0 0 2024-04-23T02:13:54.010 INFO:tasks.workunit.client.1.smithi192.stdout:4/422: mkdir de/d15/d2a/d61/d89 0 2024-04-23T02:13:54.012 INFO:tasks.workunit.client.0.smithi071.stdout:6/168: mknod d6/d17/c30 0 2024-04-23T02:13:54.014 INFO:tasks.workunit.client.1.smithi192.stdout:8/314: dwrite d1/d9/d3e/f55 [0,4194304] 0 2024-04-23T02:13:54.017 INFO:tasks.workunit.client.0.smithi071.stdout:6/169: readlink d6/l15 0 2024-04-23T02:13:54.017 INFO:tasks.workunit.client.0.smithi071.stdout:1/104: fdatasync f5 0 2024-04-23T02:13:54.018 INFO:tasks.workunit.client.1.smithi192.stdout:4/423: truncate de/d15/d19/d1a/f3a 1154755 0 2024-04-23T02:13:54.018 INFO:tasks.workunit.client.1.smithi192.stdout:8/315: write d1/df/f1c [1569713,76898] 0 2024-04-23T02:13:54.020 INFO:tasks.workunit.client.1.smithi192.stdout:7/236: dwrite d5/d17/d1c/d30/f35 [0,4194304] 0 2024-04-23T02:13:54.021 INFO:tasks.workunit.client.0.smithi071.stdout:7/147: getdents d3 0 2024-04-23T02:13:54.022 INFO:tasks.workunit.client.0.smithi071.stdout:7/148: chown d3/d17 8469337 1 2024-04-23T02:13:54.024 INFO:tasks.workunit.client.0.smithi071.stdout:7/149: fdatasync d3/d17/f22 0 2024-04-23T02:13:54.024 INFO:tasks.workunit.client.0.smithi071.stdout:6/170: dread d6/f16 [0,4194304] 0 2024-04-23T02:13:54.025 INFO:tasks.workunit.client.0.smithi071.stdout:7/150: dread - d3/f1d zero size 2024-04-23T02:13:54.026 INFO:tasks.workunit.client.0.smithi071.stdout:6/171: chown d6/c7 0 1 2024-04-23T02:13:54.027 INFO:tasks.workunit.client.0.smithi071.stdout:1/105: dwrite f0 [0,4194304] 0 2024-04-23T02:13:54.028 INFO:tasks.workunit.client.1.smithi192.stdout:7/237: unlink d5/d17/d2f/f1d 0 2024-04-23T02:13:54.031 INFO:tasks.workunit.client.1.smithi192.stdout:4/424: mkdir de/d15/d19/d3d/d49/d26/d59/d8a 0 2024-04-23T02:13:54.032 INFO:tasks.workunit.client.0.smithi071.stdout:7/151: mkdir d3/d17/d1c/d25/d34 0 2024-04-23T02:13:54.034 INFO:tasks.workunit.client.1.smithi192.stdout:8/316: dwrite d1/f63 [0,4194304] 0 2024-04-23T02:13:54.037 INFO:tasks.workunit.client.0.smithi071.stdout:7/152: creat d3/d17/d1c/d25/d34/f35 x:0 0 0 2024-04-23T02:13:54.038 INFO:tasks.workunit.client.0.smithi071.stdout:6/172: dwrite d6/f11 [0,4194304] 0 2024-04-23T02:13:54.041 INFO:tasks.workunit.client.0.smithi071.stdout:7/153: creat d3/d17/f36 x:0 0 0 2024-04-23T02:13:54.042 INFO:tasks.workunit.client.0.smithi071.stdout:6/173: unlink d6/d22/f29 0 2024-04-23T02:13:54.042 INFO:tasks.workunit.client.0.smithi071.stdout:6/174: stat d6/c13 0 2024-04-23T02:13:54.043 INFO:tasks.workunit.client.0.smithi071.stdout:7/154: write d3/d17/d1c/d25/d34/f35 [464151,63372] 0 2024-04-23T02:13:54.044 INFO:tasks.workunit.client.0.smithi071.stdout:6/175: read d6/d1f/f21 [3238797,53935] 0 2024-04-23T02:13:54.045 INFO:tasks.workunit.client.1.smithi192.stdout:4/425: write de/d15/d19/d3d/d49/f60 [614570,4644] 0 2024-04-23T02:13:54.045 INFO:tasks.workunit.client.1.smithi192.stdout:8/317: symlink d1/d12/d23/l6b 0 2024-04-23T02:13:54.046 INFO:tasks.workunit.client.0.smithi071.stdout:7/155: mkdir d3/d17/d1c/d25/d37 0 2024-04-23T02:13:54.048 INFO:tasks.workunit.client.0.smithi071.stdout:6/176: mknod d6/c31 0 2024-04-23T02:13:54.048 INFO:tasks.workunit.client.0.smithi071.stdout:6/177: chown d6/d22/f26 0 1 2024-04-23T02:13:54.049 INFO:tasks.workunit.client.0.smithi071.stdout:7/156: creat d3/d17/d1c/d25/d37/f38 x:0 0 0 2024-04-23T02:13:54.050 INFO:tasks.workunit.client.1.smithi192.stdout:8/318: symlink d1/d12/d68/l6c 0 2024-04-23T02:13:54.050 INFO:tasks.workunit.client.1.smithi192.stdout:7/238: dwrite d5/f2e [0,4194304] 0 2024-04-23T02:13:54.050 INFO:tasks.workunit.client.0.smithi071.stdout:7/157: read - d3/d17/d1c/d25/f2f zero size 2024-04-23T02:13:54.050 INFO:tasks.workunit.client.0.smithi071.stdout:6/178: creat d6/d1f/f32 x:0 0 0 2024-04-23T02:13:54.051 INFO:tasks.workunit.client.0.smithi071.stdout:7/158: chown d3/d17/d1c/d25/f27 938 1 2024-04-23T02:13:54.052 INFO:tasks.workunit.client.1.smithi192.stdout:4/426: dread de/d15/d19/d3d/d49/d26/d59/d6a/f86 [0,4194304] 0 2024-04-23T02:13:54.053 INFO:tasks.workunit.client.0.smithi071.stdout:7/159: mkdir d3/d17/d1c/d25/d34/d39 0 2024-04-23T02:13:54.054 INFO:tasks.workunit.client.0.smithi071.stdout:2/145: truncate d0/d2/d26/d14/f1a 3124876 0 2024-04-23T02:13:54.055 INFO:tasks.workunit.client.0.smithi071.stdout:7/160: chown d3/l15 4407 1 2024-04-23T02:13:54.056 INFO:tasks.workunit.client.0.smithi071.stdout:7/161: fdatasync d3/fc 0 2024-04-23T02:13:54.057 INFO:tasks.workunit.client.0.smithi071.stdout:2/146: creat d0/d2/f29 x:0 0 0 2024-04-23T02:13:54.059 INFO:tasks.workunit.client.0.smithi071.stdout:6/179: dwrite d6/f18 [0,4194304] 0 2024-04-23T02:13:54.060 INFO:tasks.workunit.client.1.smithi192.stdout:7/239: mknod d5/d17/c42 0 2024-04-23T02:13:54.060 INFO:tasks.workunit.client.1.smithi192.stdout:8/319: rename d1/df/d41/d5d/f5f to d1/df/d41/d45/d46/f6d 0 2024-04-23T02:13:54.061 INFO:tasks.workunit.client.1.smithi192.stdout:8/320: stat d1/f1d 0 2024-04-23T02:13:54.061 INFO:tasks.workunit.client.0.smithi071.stdout:2/147: mknod d0/d2/d26/df/c2a 0 2024-04-23T02:13:54.061 INFO:tasks.workunit.client.1.smithi192.stdout:4/427: creat de/d15/d19/d3d/f8b x:0 0 0 2024-04-23T02:13:54.063 INFO:tasks.workunit.client.0.smithi071.stdout:2/148: creat d0/f2b x:0 0 0 2024-04-23T02:13:54.067 INFO:tasks.workunit.client.1.smithi192.stdout:8/321: dread d1/f3 [0,4194304] 0 2024-04-23T02:13:54.068 INFO:tasks.workunit.client.1.smithi192.stdout:4/428: creat de/f8c x:0 0 0 2024-04-23T02:13:54.068 INFO:tasks.workunit.client.0.smithi071.stdout:6/180: dwrite d6/d17/f1a [0,4194304] 0 2024-04-23T02:13:54.069 INFO:tasks.workunit.client.1.smithi192.stdout:7/240: dwrite d5/d17/d2f/f22 [0,4194304] 0 2024-04-23T02:13:54.073 INFO:tasks.workunit.client.1.smithi192.stdout:8/322: creat d1/df/d41/d4c/f6e x:0 0 0 2024-04-23T02:13:54.074 INFO:tasks.workunit.client.1.smithi192.stdout:8/323: stat d1/d9/l58 0 2024-04-23T02:13:54.074 INFO:tasks.workunit.client.1.smithi192.stdout:7/241: mkdir d5/d43 0 2024-04-23T02:13:54.075 INFO:tasks.workunit.client.1.smithi192.stdout:8/324: fsync d1/d12/d23/f60 0 2024-04-23T02:13:54.077 INFO:tasks.workunit.client.1.smithi192.stdout:7/242: creat d5/d17/d1c/d26/f44 x:0 0 0 2024-04-23T02:13:54.078 INFO:tasks.workunit.client.1.smithi192.stdout:8/325: symlink d1/d9/d3e/l6f 0 2024-04-23T02:13:54.078 INFO:tasks.workunit.client.1.smithi192.stdout:4/429: read de/d15/d19/d3d/d49/f50 [3030376,121351] 0 2024-04-23T02:13:54.079 INFO:tasks.workunit.client.1.smithi192.stdout:7/243: chown d5/d17/d1c/d26/f34 1279 1 2024-04-23T02:13:54.079 INFO:tasks.workunit.client.1.smithi192.stdout:8/326: chown d1/d9/cc 71011 1 2024-04-23T02:13:54.081 INFO:tasks.workunit.client.1.smithi192.stdout:4/430: chown de/d15/d19/d3d/d49/d26/c7b 233 1 2024-04-23T02:13:54.083 INFO:tasks.workunit.client.1.smithi192.stdout:8/327: creat d1/d9/d3e/f70 x:0 0 0 2024-04-23T02:13:54.084 INFO:tasks.workunit.client.1.smithi192.stdout:4/431: write de/f21 [1996750,100842] 0 2024-04-23T02:13:54.084 INFO:tasks.workunit.client.1.smithi192.stdout:7/244: creat d5/d17/d1c/f45 x:0 0 0 2024-04-23T02:13:54.085 INFO:tasks.workunit.client.1.smithi192.stdout:4/432: read de/f1b [4467384,25875] 0 2024-04-23T02:13:54.085 INFO:tasks.workunit.client.1.smithi192.stdout:8/328: chown d1/d12/d23/c44 0 1 2024-04-23T02:13:54.086 INFO:tasks.workunit.client.1.smithi192.stdout:8/329: read - d1/df/d41/d45/d46/f48 zero size 2024-04-23T02:13:54.087 INFO:tasks.workunit.client.1.smithi192.stdout:7/245: creat d5/d17/d2f/f46 x:0 0 0 2024-04-23T02:13:54.088 INFO:tasks.workunit.client.1.smithi192.stdout:8/330: write d1/d9/f11 [7068965,89871] 0 2024-04-23T02:13:54.089 INFO:tasks.workunit.client.1.smithi192.stdout:7/246: fdatasync d5/d17/d2f/d31/f32 0 2024-04-23T02:13:54.091 INFO:tasks.workunit.client.1.smithi192.stdout:8/331: mkdir d1/df/d41/d4c/d71 0 2024-04-23T02:13:54.092 INFO:tasks.workunit.client.1.smithi192.stdout:8/332: write d1/df/d41/d4c/f6e [718409,1219] 0 2024-04-23T02:13:54.092 INFO:tasks.workunit.client.1.smithi192.stdout:8/333: chown d1/df/d41/d4c 1007418 1 2024-04-23T02:13:54.093 INFO:tasks.workunit.client.1.smithi192.stdout:8/334: readlink d1/d9/l58 0 2024-04-23T02:13:54.093 INFO:tasks.workunit.client.1.smithi192.stdout:7/247: link d5/d17/d2f/l19 d5/d17/d2f/d31/d41/l47 0 2024-04-23T02:13:54.095 INFO:tasks.workunit.client.1.smithi192.stdout:8/335: creat d1/f72 x:0 0 0 2024-04-23T02:13:54.097 INFO:tasks.workunit.client.1.smithi192.stdout:8/336: write d1/df/d41/f67 [431566,84324] 0 2024-04-23T02:13:54.101 INFO:tasks.workunit.client.1.smithi192.stdout:7/248: dwrite d5/d17/d1c/f24 [0,4194304] 0 2024-04-23T02:13:54.101 INFO:tasks.workunit.client.1.smithi192.stdout:0/325: sync 2024-04-23T02:13:54.104 INFO:tasks.workunit.client.1.smithi192.stdout:0/326: chown d6/df/d13/l5a 40328651 1 2024-04-23T02:13:54.105 INFO:tasks.workunit.client.1.smithi192.stdout:0/327: fdatasync d6/f6d 0 2024-04-23T02:13:54.108 INFO:tasks.workunit.client.1.smithi192.stdout:0/328: dread - d6/df/d47/d55/f70 zero size 2024-04-23T02:13:54.109 INFO:tasks.workunit.client.0.smithi071.stdout:6/181: fsync d6/d1f/f32 0 2024-04-23T02:13:54.112 INFO:tasks.workunit.client.1.smithi192.stdout:7/249: dwrite d5/f15 [0,4194304] 0 2024-04-23T02:13:54.112 INFO:tasks.workunit.client.0.smithi071.stdout:2/149: getdents d0/d2/d26/df 0 2024-04-23T02:13:54.114 INFO:tasks.workunit.client.0.smithi071.stdout:2/150: chown d0/d2/d26/d14/f22 11053784 1 2024-04-23T02:13:54.114 INFO:tasks.workunit.client.0.smithi071.stdout:2/151: readlink d0/l10 0 2024-04-23T02:13:54.115 INFO:tasks.workunit.client.1.smithi192.stdout:7/250: write d5/d17/d1c/d26/f44 [470749,3184] 0 2024-04-23T02:13:54.116 INFO:tasks.workunit.client.0.smithi071.stdout:8/231: sync 2024-04-23T02:13:54.117 INFO:tasks.workunit.client.0.smithi071.stdout:6/182: dwrite d6/d17/f2c [0,4194304] 0 2024-04-23T02:13:54.120 INFO:tasks.workunit.client.1.smithi192.stdout:4/433: write de/d15/d19/d1a/f1e [1107177,86896] 0 2024-04-23T02:13:54.121 INFO:tasks.workunit.client.0.smithi071.stdout:6/183: creat d6/d17/f33 x:0 0 0 2024-04-23T02:13:54.122 INFO:tasks.workunit.client.1.smithi192.stdout:8/337: rename d1/df/d41 to d1/d12/d23/d4b/d73 0 2024-04-23T02:13:54.126 INFO:tasks.workunit.client.0.smithi071.stdout:6/184: creat d6/f34 x:0 0 0 2024-04-23T02:13:54.126 INFO:tasks.workunit.client.0.smithi071.stdout:8/232: dwrite dc/f28 [0,4194304] 0 2024-04-23T02:13:54.127 INFO:tasks.workunit.client.1.smithi192.stdout:4/434: read de/d15/d19/d1a/f72 [591758,77539] 0 2024-04-23T02:13:54.127 INFO:tasks.workunit.client.1.smithi192.stdout:7/251: fdatasync d5/d17/d1c/d26/f44 0 2024-04-23T02:13:54.128 INFO:tasks.workunit.client.0.smithi071.stdout:6/185: write d6/fc [2663405,83624] 0 2024-04-23T02:13:54.128 INFO:tasks.workunit.client.1.smithi192.stdout:7/252: write d5/f2e [1375440,16980] 0 2024-04-23T02:13:54.132 INFO:tasks.workunit.client.0.smithi071.stdout:6/186: dread d6/f11 [0,4194304] 0 2024-04-23T02:13:54.133 INFO:tasks.workunit.client.0.smithi071.stdout:6/187: write d6/f8 [1684333,94364] 0 2024-04-23T02:13:54.133 INFO:tasks.workunit.client.1.smithi192.stdout:4/435: creat de/d15/d19/d3d/d34/d4a/f8d x:0 0 0 2024-04-23T02:13:54.137 INFO:tasks.workunit.client.1.smithi192.stdout:7/253: unlink d5/c1b 0 2024-04-23T02:13:54.137 INFO:tasks.workunit.client.0.smithi071.stdout:6/188: dread d6/f18 [0,4194304] 0 2024-04-23T02:13:54.138 INFO:tasks.workunit.client.1.smithi192.stdout:0/329: truncate d6/df/d47/f61 2520621 0 2024-04-23T02:13:54.138 INFO:tasks.workunit.client.1.smithi192.stdout:7/254: dread - d5/d17/d2f/d31/f32 zero size 2024-04-23T02:13:54.140 INFO:tasks.workunit.client.0.smithi071.stdout:6/189: creat d6/d22/f35 x:0 0 0 2024-04-23T02:13:54.140 INFO:tasks.workunit.client.1.smithi192.stdout:0/330: chown d6/d44/d45/l4d 158766210 1 2024-04-23T02:13:54.143 INFO:tasks.workunit.client.0.smithi071.stdout:6/190: link f4 d6/d17/f36 0 2024-04-23T02:13:54.144 INFO:tasks.workunit.client.1.smithi192.stdout:7/255: rename d5/d17/d1c/d26/f44 to d5/d17/d2f/f48 0 2024-04-23T02:13:54.144 INFO:tasks.workunit.client.1.smithi192.stdout:0/331: creat d6/f72 x:0 0 0 2024-04-23T02:13:54.146 INFO:tasks.workunit.client.0.smithi071.stdout:6/191: dread d6/f16 [0,4194304] 0 2024-04-23T02:13:54.147 INFO:tasks.workunit.client.1.smithi192.stdout:7/256: mknod d5/c49 0 2024-04-23T02:13:54.147 INFO:tasks.workunit.client.1.smithi192.stdout:4/436: dwrite de/f18 [0,4194304] 0 2024-04-23T02:13:54.147 INFO:tasks.workunit.client.1.smithi192.stdout:0/332: mkdir d6/df/d13/d6b/d73 0 2024-04-23T02:13:54.148 INFO:tasks.workunit.client.0.smithi071.stdout:6/192: creat d6/d22/f37 x:0 0 0 2024-04-23T02:13:54.149 INFO:tasks.workunit.client.0.smithi071.stdout:6/193: mknod d6/d17/c38 0 2024-04-23T02:13:54.151 INFO:tasks.workunit.client.0.smithi071.stdout:6/194: symlink d6/d22/l39 0 2024-04-23T02:13:54.152 INFO:tasks.workunit.client.1.smithi192.stdout:0/333: creat d6/df/d35/f74 x:0 0 0 2024-04-23T02:13:54.153 INFO:tasks.workunit.client.0.smithi071.stdout:6/195: write d6/f12 [431283,52673] 0 2024-04-23T02:13:54.156 INFO:tasks.workunit.client.1.smithi192.stdout:4/437: rename de/d15/d19/d3d/f46 to de/d15/d19/d3d/d49/d70/d73/f8e 0 2024-04-23T02:13:54.156 INFO:tasks.workunit.client.1.smithi192.stdout:7/257: dread d5/d17/d2f/f22 [0,4194304] 0 2024-04-23T02:13:54.156 INFO:tasks.workunit.client.1.smithi192.stdout:8/338: dread d1/d12/d23/f3c [0,4194304] 0 2024-04-23T02:13:54.157 INFO:tasks.workunit.client.0.smithi071.stdout:2/152: dread d0/f1e [0,4194304] 0 2024-04-23T02:13:54.157 INFO:tasks.workunit.client.0.smithi071.stdout:2/153: readlink d0/l10 0 2024-04-23T02:13:54.159 INFO:tasks.workunit.client.0.smithi071.stdout:2/154: mknod d0/d2/c2c 0 2024-04-23T02:13:54.161 INFO:tasks.workunit.client.0.smithi071.stdout:2/155: symlink d0/l2d 0 2024-04-23T02:13:54.162 INFO:tasks.workunit.client.0.smithi071.stdout:2/156: truncate d0/f20 877479 0 2024-04-23T02:13:54.162 INFO:tasks.workunit.client.0.smithi071.stdout:6/196: rmdir d6 39 2024-04-23T02:13:54.164 INFO:tasks.workunit.client.1.smithi192.stdout:7/258: dwrite d5/d17/d1c/d30/f35 [0,4194304] 0 2024-04-23T02:13:54.164 INFO:tasks.workunit.client.1.smithi192.stdout:8/339: rename d1/d12/d23/d2c/l27 to d1/d12/d68/d6a/l74 0 2024-04-23T02:13:54.164 INFO:tasks.workunit.client.1.smithi192.stdout:0/334: getdents d6/df/d13/d25/d26 0 2024-04-23T02:13:54.167 INFO:tasks.workunit.client.1.smithi192.stdout:8/340: dread - d1/d9/d3e/f70 zero size 2024-04-23T02:13:54.167 INFO:tasks.workunit.client.1.smithi192.stdout:7/259: fdatasync d5/d17/d1c/f2d 0 2024-04-23T02:13:54.167 INFO:tasks.workunit.client.0.smithi071.stdout:6/197: getdents d6/d22 0 2024-04-23T02:13:54.169 INFO:tasks.workunit.client.1.smithi192.stdout:8/341: unlink d1/d9/l3d 0 2024-04-23T02:13:54.170 INFO:tasks.workunit.client.0.smithi071.stdout:6/198: getdents d6/d1f 0 2024-04-23T02:13:54.171 INFO:tasks.workunit.client.1.smithi192.stdout:8/342: write d1/d9/f22 [3729823,29926] 0 2024-04-23T02:13:54.171 INFO:tasks.workunit.client.0.smithi071.stdout:2/157: getdents d0 0 2024-04-23T02:13:54.171 INFO:tasks.workunit.client.0.smithi071.stdout:6/199: fsync d6/f16 0 2024-04-23T02:13:54.171 INFO:tasks.workunit.client.0.smithi071.stdout:2/158: fdatasync d0/d2/f29 0 2024-04-23T02:13:54.172 INFO:tasks.workunit.client.1.smithi192.stdout:7/260: mkdir d5/d17/d1c/d4a 0 2024-04-23T02:13:54.173 INFO:tasks.workunit.client.0.smithi071.stdout:6/200: mknod d6/d22/c3a 0 2024-04-23T02:13:54.174 INFO:tasks.workunit.client.0.smithi071.stdout:6/201: chown d6/d1f/f25 26505677 1 2024-04-23T02:13:54.174 INFO:tasks.workunit.client.0.smithi071.stdout:2/159: creat d0/f2e x:0 0 0 2024-04-23T02:13:54.175 INFO:tasks.workunit.client.0.smithi071.stdout:6/202: write d6/d22/f35 [781229,92275] 0 2024-04-23T02:13:54.175 INFO:tasks.workunit.client.1.smithi192.stdout:0/335: link d6/d44/d45/l4d d6/df/d13/d25/l75 0 2024-04-23T02:13:54.176 INFO:tasks.workunit.client.0.smithi071.stdout:2/160: fsync d0/f1e 0 2024-04-23T02:13:54.177 INFO:tasks.workunit.client.0.smithi071.stdout:2/161: mkdir d0/d2/d26/d14/d1f/d2f 0 2024-04-23T02:13:54.178 INFO:tasks.workunit.client.1.smithi192.stdout:7/261: creat d5/d17/f4b x:0 0 0 2024-04-23T02:13:54.178 INFO:tasks.workunit.client.1.smithi192.stdout:0/336: rmdir d6/df/d13/d25/d3a 39 2024-04-23T02:13:54.181 INFO:tasks.workunit.client.1.smithi192.stdout:7/262: mkdir d5/d43/d4c 0 2024-04-23T02:13:54.182 INFO:tasks.workunit.client.0.smithi071.stdout:2/162: mkdir d0/d2/d26/df/d30 0 2024-04-23T02:13:54.182 INFO:tasks.workunit.client.0.smithi071.stdout:2/163: rename d0/l2d to d0/d2/d26/df/d30/l31 0 2024-04-23T02:13:54.182 INFO:tasks.workunit.client.0.smithi071.stdout:2/164: read - d0/d2/f21 zero size 2024-04-23T02:13:54.183 INFO:tasks.workunit.client.1.smithi192.stdout:7/263: symlink d5/d43/l4d 0 2024-04-23T02:13:54.184 INFO:tasks.workunit.client.0.smithi071.stdout:2/165: fsync d0/d2/fb 0 2024-04-23T02:13:54.186 INFO:tasks.workunit.client.1.smithi192.stdout:7/264: truncate d5/d17/d2f/f48 641549 0 2024-04-23T02:13:54.190 INFO:tasks.workunit.client.1.smithi192.stdout:0/337: link d6/c20 d6/df/d13/d25/c76 0 2024-04-23T02:13:54.191 INFO:tasks.workunit.client.1.smithi192.stdout:7/265: mkdir d5/d43/d4c/d4e 0 2024-04-23T02:13:54.192 INFO:tasks.workunit.client.1.smithi192.stdout:8/343: dread d1/d12/d23/f40 [0,4194304] 0 2024-04-23T02:13:54.193 INFO:tasks.workunit.client.1.smithi192.stdout:0/338: truncate d6/df/d13/d25/d3a/f62 986864 0 2024-04-23T02:13:54.195 INFO:tasks.workunit.client.1.smithi192.stdout:0/339: write d6/df/d13/d25/d26/d67/f69 [752709,89563] 0 2024-04-23T02:13:54.196 INFO:tasks.workunit.client.1.smithi192.stdout:0/340: write d6/f18 [655067,89845] 0 2024-04-23T02:13:54.201 INFO:tasks.workunit.client.1.smithi192.stdout:8/344: getdents d1/d9 0 2024-04-23T02:13:54.202 INFO:tasks.workunit.client.1.smithi192.stdout:0/341: mknod d6/df/d13/d25/d3a/d5b/c77 0 2024-04-23T02:13:54.203 INFO:tasks.workunit.client.1.smithi192.stdout:0/342: write d6/df/d13/d25/d3a/f4f [3083738,11147] 0 2024-04-23T02:13:54.205 INFO:tasks.workunit.client.1.smithi192.stdout:8/345: creat d1/d9/d3e/d59/f75 x:0 0 0 2024-04-23T02:13:54.205 INFO:tasks.workunit.client.1.smithi192.stdout:0/343: write d6/df/d47/d55/f70 [26273,114417] 0 2024-04-23T02:13:54.213 INFO:tasks.workunit.client.1.smithi192.stdout:8/346: creat d1/d12/d23/d4b/d73/d45/f76 x:0 0 0 2024-04-23T02:13:54.215 INFO:tasks.workunit.client.1.smithi192.stdout:0/344: dwrite d6/d44/f19 [4194304,4194304] 0 2024-04-23T02:13:54.220 INFO:tasks.workunit.client.1.smithi192.stdout:0/345: getdents d6/df/d13/d6b 0 2024-04-23T02:13:54.225 INFO:tasks.workunit.client.1.smithi192.stdout:8/347: rmdir d1 39 2024-04-23T02:13:54.225 INFO:tasks.workunit.client.1.smithi192.stdout:7/266: write d5/d17/d2f/f48 [1404963,76782] 0 2024-04-23T02:13:54.232 INFO:tasks.workunit.client.1.smithi192.stdout:0/346: rename d6/df/d13/d25/d26/l41 to d6/df/d13/d25/d3a/d5b/l78 0 2024-04-23T02:13:54.234 INFO:tasks.workunit.client.1.smithi192.stdout:8/348: truncate d1/d12/d23/d4b/d73/d45/d46/f47 1134993 0 2024-04-23T02:13:54.234 INFO:tasks.workunit.client.1.smithi192.stdout:0/347: link f5 d6/d4b/f79 0 2024-04-23T02:13:54.235 INFO:tasks.workunit.client.1.smithi192.stdout:7/267: mknod d5/d17/d1c/d4a/c4f 0 2024-04-23T02:13:54.236 INFO:tasks.workunit.client.1.smithi192.stdout:8/349: fsync d1/d12/d23/d4b/d73/d45/f76 0 2024-04-23T02:13:54.237 INFO:tasks.workunit.client.1.smithi192.stdout:7/268: unlink l4 0 2024-04-23T02:13:54.245 INFO:tasks.workunit.client.1.smithi192.stdout:0/348: write f5 [349192,57865] 0 2024-04-23T02:13:54.246 INFO:tasks.workunit.client.1.smithi192.stdout:8/350: dwrite d1/d12/d23/d4b/d73/d4c/f6e [0,4194304] 0 2024-04-23T02:13:54.247 INFO:tasks.workunit.client.1.smithi192.stdout:7/269: dwrite d5/d17/d1c/f24 [0,4194304] 0 2024-04-23T02:13:54.248 INFO:tasks.workunit.client.1.smithi192.stdout:0/349: chown d6/df/d35/d49/f54 0 1 2024-04-23T02:13:54.251 INFO:tasks.workunit.client.1.smithi192.stdout:0/350: rmdir d6/df/d35 39 2024-04-23T02:13:54.254 INFO:tasks.workunit.client.1.smithi192.stdout:7/270: dread d5/f15 [0,4194304] 0 2024-04-23T02:13:54.256 INFO:tasks.workunit.client.1.smithi192.stdout:7/271: write d5/d17/d2f/f46 [1009852,104242] 0 2024-04-23T02:13:54.257 INFO:tasks.workunit.client.1.smithi192.stdout:0/351: getdents d6/df/d13/d25/d3a/d5b 0 2024-04-23T02:13:54.260 INFO:tasks.workunit.client.1.smithi192.stdout:7/272: rename d5/d17/d1c/d30 to d5/d17/d2f/d31/d41/d50 0 2024-04-23T02:13:54.261 INFO:tasks.workunit.client.1.smithi192.stdout:1/122: sync 2024-04-23T02:13:54.261 INFO:tasks.workunit.client.1.smithi192.stdout:0/352: creat d6/df/d35/d49/f7a x:0 0 0 2024-04-23T02:13:54.262 INFO:tasks.workunit.client.1.smithi192.stdout:7/273: mknod d5/d17/d2f/d31/d41/d50/c51 0 2024-04-23T02:13:54.264 INFO:tasks.workunit.client.1.smithi192.stdout:0/353: mkdir d6/df/d13/d25/d3a/d7b 0 2024-04-23T02:13:54.266 INFO:tasks.workunit.client.1.smithi192.stdout:0/354: mknod d6/df/d13/c7c 0 2024-04-23T02:13:54.267 INFO:tasks.workunit.client.1.smithi192.stdout:1/123: dwrite f9 [0,4194304] 0 2024-04-23T02:13:54.268 INFO:tasks.workunit.client.1.smithi192.stdout:1/124: dread - dc/ff zero size 2024-04-23T02:13:54.268 INFO:tasks.workunit.client.1.smithi192.stdout:1/125: stat dc/f13 0 2024-04-23T02:13:54.269 INFO:tasks.workunit.client.1.smithi192.stdout:0/355: creat d6/df/d13/d6b/f7d x:0 0 0 2024-04-23T02:13:54.270 INFO:tasks.workunit.client.1.smithi192.stdout:1/126: fdatasync dc/f11 0 2024-04-23T02:13:54.271 INFO:tasks.workunit.client.1.smithi192.stdout:0/356: chown d6/df/d13/d25/d26/c4e 161693519 1 2024-04-23T02:13:54.274 INFO:tasks.workunit.client.0.smithi071.stdout:4/180: sync 2024-04-23T02:13:54.274 INFO:tasks.workunit.client.0.smithi071.stdout:9/203: sync 2024-04-23T02:13:54.276 INFO:tasks.workunit.client.1.smithi192.stdout:1/127: rename dc/c12 to dc/d14/d15/c20 0 2024-04-23T02:13:54.277 INFO:tasks.workunit.client.0.smithi071.stdout:9/204: rename d1/d32/l34 to d1/d1a/l43 0 2024-04-23T02:13:54.277 INFO:tasks.workunit.client.0.smithi071.stdout:9/205: fsync d1/f37 0 2024-04-23T02:13:54.278 INFO:tasks.workunit.client.0.smithi071.stdout:4/181: creat d2/d15/f3a x:0 0 0 2024-04-23T02:13:54.279 INFO:tasks.workunit.client.1.smithi192.stdout:0/357: dwrite d6/df/d13/d25/f3d [0,4194304] 0 2024-04-23T02:13:54.279 INFO:tasks.workunit.client.0.smithi071.stdout:9/206: write d1/d1a/f31 [431923,47404] 0 2024-04-23T02:13:54.283 INFO:tasks.workunit.client.0.smithi071.stdout:4/182: rename d2/d2a/d2d/f34 to d2/d2a/f3b 0 2024-04-23T02:13:54.284 INFO:tasks.workunit.client.0.smithi071.stdout:4/183: readlink d2/d15/d1d/d23/le 0 2024-04-23T02:13:54.285 INFO:tasks.workunit.client.0.smithi071.stdout:4/184: truncate d2/d2a/f36 713051 0 2024-04-23T02:13:54.286 INFO:tasks.workunit.client.0.smithi071.stdout:4/185: write d2/d15/d1d/d23/d10/f13 [3653708,72711] 0 2024-04-23T02:13:54.288 INFO:tasks.workunit.client.1.smithi192.stdout:0/358: creat d6/df/d13/d6b/d73/f7e x:0 0 0 2024-04-23T02:13:54.288 INFO:tasks.workunit.client.1.smithi192.stdout:1/128: dread fa [0,4194304] 0 2024-04-23T02:13:54.288 INFO:tasks.workunit.client.1.smithi192.stdout:1/129: chown f9 15842 1 2024-04-23T02:13:54.289 INFO:tasks.workunit.client.1.smithi192.stdout:0/359: rmdir d6/df/d13/d25/d3a 39 2024-04-23T02:13:54.289 INFO:tasks.workunit.client.1.smithi192.stdout:0/360: fsync d6/df/f14 0 2024-04-23T02:13:54.293 INFO:tasks.workunit.client.1.smithi192.stdout:0/361: unlink d6/df/d13/d25/d3a/d5b/c77 0 2024-04-23T02:13:54.294 INFO:tasks.workunit.client.1.smithi192.stdout:0/362: symlink d6/df/d56/l7f 0 2024-04-23T02:13:54.296 INFO:tasks.workunit.client.1.smithi192.stdout:1/130: dread dc/f13 [0,4194304] 0 2024-04-23T02:13:54.297 INFO:tasks.workunit.client.1.smithi192.stdout:0/363: rmdir d6/df/d13/d6b 39 2024-04-23T02:13:54.298 INFO:tasks.workunit.client.1.smithi192.stdout:0/364: chown f5 3 1 2024-04-23T02:13:54.298 INFO:tasks.workunit.client.1.smithi192.stdout:1/131: chown c5 362905025 1 2024-04-23T02:13:54.299 INFO:tasks.workunit.client.0.smithi071.stdout:9/207: dwrite d1/f3 [0,4194304] 0 2024-04-23T02:13:54.300 INFO:tasks.workunit.client.1.smithi192.stdout:1/132: write dc/f10 [980886,122144] 0 2024-04-23T02:13:54.300 INFO:tasks.workunit.client.1.smithi192.stdout:1/133: chown dc/l1b 5450461 1 2024-04-23T02:13:54.302 INFO:tasks.workunit.client.1.smithi192.stdout:0/365: creat d6/df/d13/d25/d3a/d5b/f80 x:0 0 0 2024-04-23T02:13:54.302 INFO:tasks.workunit.client.0.smithi071.stdout:9/208: symlink d1/d1a/d39/d42/l44 0 2024-04-23T02:13:54.303 INFO:tasks.workunit.client.1.smithi192.stdout:0/366: fdatasync d6/df/f50 0 2024-04-23T02:13:54.304 INFO:tasks.workunit.client.1.smithi192.stdout:0/367: chown d6/df/d13/d25/d26 551005 1 2024-04-23T02:13:54.305 INFO:tasks.workunit.client.0.smithi071.stdout:9/209: getdents d1/d32/d41 0 2024-04-23T02:13:54.306 INFO:tasks.workunit.client.1.smithi192.stdout:0/368: write d6/d44/f19 [7165667,20938] 0 2024-04-23T02:13:54.306 INFO:tasks.workunit.client.0.smithi071.stdout:9/210: write d1/f16 [529442,87044] 0 2024-04-23T02:13:54.308 INFO:tasks.workunit.client.0.smithi071.stdout:9/211: mknod d1/c45 0 2024-04-23T02:13:54.308 INFO:tasks.workunit.client.1.smithi192.stdout:1/134: write fa [1489711,90086] 0 2024-04-23T02:13:54.309 INFO:tasks.workunit.client.1.smithi192.stdout:0/369: rmdir d6/df/d13 39 2024-04-23T02:13:54.312 INFO:tasks.workunit.client.0.smithi071.stdout:4/186: dread d2/d15/d1d/d23/f16 [0,4194304] 0 2024-04-23T02:13:54.312 INFO:tasks.workunit.client.0.smithi071.stdout:4/187: stat d2/d15/d1d/d2c/c32 0 2024-04-23T02:13:54.313 INFO:tasks.workunit.client.0.smithi071.stdout:4/188: read d2/d15/d1d/d23/d29/f31 [838453,111310] 0 2024-04-23T02:13:54.314 INFO:tasks.workunit.client.1.smithi192.stdout:1/135: getdents dc 0 2024-04-23T02:13:54.314 INFO:tasks.workunit.client.0.smithi071.stdout:4/189: fsync d2/d15/d1d/d23/f1f 0 2024-04-23T02:13:54.315 INFO:tasks.workunit.client.1.smithi192.stdout:1/136: truncate dc/f13 1566501 0 2024-04-23T02:13:54.316 INFO:tasks.workunit.client.0.smithi071.stdout:4/190: write d2/d15/d1d/d23/fc [1825855,53302] 0 2024-04-23T02:13:54.317 INFO:tasks.workunit.client.0.smithi071.stdout:4/191: read d2/d15/d1d/d23/f1f [164082,3617] 0 2024-04-23T02:13:54.317 INFO:tasks.workunit.client.1.smithi192.stdout:0/370: link d6/df/d35/l60 d6/df/d13/d25/d26/l81 0 2024-04-23T02:13:54.317 INFO:tasks.workunit.client.0.smithi071.stdout:9/212: truncate d1/d1a/f22 278288 0 2024-04-23T02:13:54.320 INFO:tasks.workunit.client.0.smithi071.stdout:4/192: getdents d2/d15/d1d/d23/d10 0 2024-04-23T02:13:54.320 INFO:tasks.workunit.client.1.smithi192.stdout:0/371: read d6/df/f30 [422514,104032] 0 2024-04-23T02:13:54.321 INFO:tasks.workunit.client.1.smithi192.stdout:0/372: fdatasync d6/df/d47/d55/f70 0 2024-04-23T02:13:54.325 INFO:tasks.workunit.client.0.smithi071.stdout:4/193: dwrite d2/d15/d1d/d23/f38 [0,4194304] 0 2024-04-23T02:13:54.327 INFO:tasks.workunit.client.1.smithi192.stdout:0/373: dwrite d6/d44/f19 [0,4194304] 0 2024-04-23T02:13:54.328 INFO:tasks.workunit.client.1.smithi192.stdout:0/374: chown d6/df/d13/d25/f3d 1869 1 2024-04-23T02:13:54.330 INFO:tasks.workunit.client.1.smithi192.stdout:0/375: rmdir d6/d4b 39 2024-04-23T02:13:54.330 INFO:tasks.workunit.client.1.smithi192.stdout:0/376: dread - d6/f72 zero size 2024-04-23T02:13:54.336 INFO:tasks.workunit.client.1.smithi192.stdout:0/377: dwrite d6/df/d13/f31 [0,4194304] 0 2024-04-23T02:13:54.337 INFO:tasks.workunit.client.0.smithi071.stdout:4/194: dwrite d2/d15/f1e [0,4194304] 0 2024-04-23T02:13:54.339 INFO:tasks.workunit.client.0.smithi071.stdout:4/195: symlink d2/d15/d1d/d23/l3c 0 2024-04-23T02:13:54.343 INFO:tasks.workunit.client.0.smithi071.stdout:4/196: getdents d2/d2a 0 2024-04-23T02:13:54.344 INFO:tasks.workunit.client.1.smithi192.stdout:0/378: dwrite d6/d44/f19 [0,4194304] 0 2024-04-23T02:13:54.345 INFO:tasks.workunit.client.1.smithi192.stdout:0/379: write d6/f72 [88481,121461] 0 2024-04-23T02:13:54.351 INFO:tasks.workunit.client.0.smithi071.stdout:4/197: dwrite d2/d15/d1d/d2c/f2e [0,4194304] 0 2024-04-23T02:13:54.353 INFO:tasks.workunit.client.0.smithi071.stdout:4/198: write d2/d15/d1d/d23/d10/f13 [970102,17016] 0 2024-04-23T02:13:54.354 INFO:tasks.workunit.client.1.smithi192.stdout:0/380: creat d6/df/d13/d25/d26/f82 x:0 0 0 2024-04-23T02:13:54.356 INFO:tasks.workunit.client.1.smithi192.stdout:0/381: truncate d6/df/d47/f63 858913 0 2024-04-23T02:13:54.356 INFO:tasks.workunit.client.0.smithi071.stdout:4/199: rmdir d2/d15/d1d/d23/d29 39 2024-04-23T02:13:54.358 INFO:tasks.workunit.client.0.smithi071.stdout:4/200: mknod d2/d15/d1d/d23/d10/c3d 0 2024-04-23T02:13:54.359 INFO:tasks.workunit.client.0.smithi071.stdout:4/201: stat d2/d15/d1d/d23/d10/l26 0 2024-04-23T02:13:54.360 INFO:tasks.workunit.client.0.smithi071.stdout:4/202: chown d2 56 1 2024-04-23T02:13:54.368 INFO:tasks.workunit.client.0.smithi071.stdout:4/203: dwrite d2/d15/f3a [0,4194304] 0 2024-04-23T02:13:54.369 INFO:tasks.workunit.client.0.smithi071.stdout:4/204: write d2/d15/f3a [1303194,33822] 0 2024-04-23T02:13:54.373 INFO:tasks.workunit.client.0.smithi071.stdout:4/205: mkdir d2/d15/d3e 0 2024-04-23T02:13:54.375 INFO:tasks.workunit.client.0.smithi071.stdout:4/206: mknod d2/c3f 0 2024-04-23T02:13:54.376 INFO:tasks.workunit.client.0.smithi071.stdout:4/207: fdatasync d2/d15/d1d/d23/d10/f13 0 2024-04-23T02:13:54.376 INFO:tasks.workunit.client.0.smithi071.stdout:4/208: fdatasync d2/d15/f1a 0 2024-04-23T02:13:54.384 INFO:tasks.workunit.client.0.smithi071.stdout:4/209: truncate d2/d2a/f36 682173 0 2024-04-23T02:13:54.387 INFO:tasks.workunit.client.0.smithi071.stdout:4/210: getdents d2/d15/d1d/d2c 0 2024-04-23T02:13:54.392 INFO:tasks.workunit.client.0.smithi071.stdout:4/211: dread d2/d15/d1d/d23/d29/f31 [0,4194304] 0 2024-04-23T02:13:54.394 INFO:tasks.workunit.client.0.smithi071.stdout:4/212: mknod d2/d15/d1d/c40 0 2024-04-23T02:13:54.395 INFO:tasks.workunit.client.0.smithi071.stdout:4/213: creat d2/d15/d1d/d23/d17/f41 x:0 0 0 2024-04-23T02:13:54.398 INFO:tasks.workunit.client.0.smithi071.stdout:4/214: getdents d2/d2a/d2d 0 2024-04-23T02:13:54.400 INFO:tasks.workunit.client.0.smithi071.stdout:4/215: mkdir d2/d2a/d2d/d39/d42 0 2024-04-23T02:13:54.400 INFO:tasks.workunit.client.0.smithi071.stdout:4/216: chown d2/l1c 9225816 1 2024-04-23T02:13:54.401 INFO:tasks.workunit.client.0.smithi071.stdout:4/217: dread - d2/d15/d1d/d23/d17/f41 zero size 2024-04-23T02:13:54.402 INFO:tasks.workunit.client.0.smithi071.stdout:3/129: sync 2024-04-23T02:13:54.402 INFO:tasks.workunit.client.0.smithi071.stdout:5/221: sync 2024-04-23T02:13:54.404 INFO:tasks.workunit.client.0.smithi071.stdout:4/218: rename d2/d15/d1d/d23/d10/f13 to d2/f43 0 2024-04-23T02:13:54.406 INFO:tasks.workunit.client.0.smithi071.stdout:4/219: fsync d2/d15/d1d/d23/f1f 0 2024-04-23T02:13:54.408 INFO:tasks.workunit.client.0.smithi071.stdout:3/130: link d7/d22/f25 d7/d27/d2c/f2e 0 2024-04-23T02:13:54.408 INFO:tasks.workunit.client.0.smithi071.stdout:4/220: write d2/d15/d1d/f35 [168535,33471] 0 2024-04-23T02:13:54.411 INFO:tasks.workunit.client.0.smithi071.stdout:5/222: dwrite d3/f12 [0,4194304] 0 2024-04-23T02:13:54.414 INFO:tasks.workunit.client.0.smithi071.stdout:5/223: rmdir d3/d1d 39 2024-04-23T02:13:54.416 INFO:tasks.workunit.client.0.smithi071.stdout:3/131: getdents d7/d22/d23 0 2024-04-23T02:13:54.418 INFO:tasks.workunit.client.0.smithi071.stdout:5/224: rename d3/l3d to d3/d34/l59 0 2024-04-23T02:13:54.419 INFO:tasks.workunit.client.0.smithi071.stdout:4/221: dwrite d2/d15/d1d/d23/d17/f41 [0,4194304] 0 2024-04-23T02:13:54.419 INFO:tasks.workunit.client.0.smithi071.stdout:3/132: creat d7/d10/f2f x:0 0 0 2024-04-23T02:13:54.420 INFO:tasks.workunit.client.0.smithi071.stdout:3/133: chown l6 9201 1 2024-04-23T02:13:54.420 INFO:tasks.workunit.client.0.smithi071.stdout:4/222: chown d2/d15/d1d 38 1 2024-04-23T02:13:54.420 INFO:tasks.workunit.client.0.smithi071.stdout:3/134: chown l3 631 1 2024-04-23T02:13:54.423 INFO:tasks.workunit.client.0.smithi071.stdout:4/223: symlink d2/d15/d3e/l44 0 2024-04-23T02:13:54.425 INFO:tasks.workunit.client.0.smithi071.stdout:3/135: rename d7/d22/c29 to d7/d10/c30 0 2024-04-23T02:13:54.425 INFO:tasks.workunit.client.0.smithi071.stdout:4/224: write d2/d15/d1d/d23/d17/f24 [3402045,24674] 0 2024-04-23T02:13:54.426 INFO:tasks.workunit.client.0.smithi071.stdout:3/136: chown l3 29 1 2024-04-23T02:13:54.426 INFO:tasks.workunit.client.0.smithi071.stdout:3/137: chown f0 514780 1 2024-04-23T02:13:54.428 INFO:tasks.workunit.client.0.smithi071.stdout:5/225: write d3/df/f24 [1291450,1568] 0 2024-04-23T02:13:54.430 INFO:tasks.workunit.client.0.smithi071.stdout:5/226: chown d3/da/d1a/d1c/d4a/f58 3079 1 2024-04-23T02:13:54.430 INFO:tasks.workunit.client.0.smithi071.stdout:3/138: read d7/fd [1849296,107112] 0 2024-04-23T02:13:54.432 INFO:tasks.workunit.client.0.smithi071.stdout:5/227: mkdir d3/da/d1a/d1c/d4a/d5a 0 2024-04-23T02:13:54.433 INFO:tasks.workunit.client.0.smithi071.stdout:5/228: chown d3/c6 6 1 2024-04-23T02:13:54.435 INFO:tasks.workunit.client.0.smithi071.stdout:5/229: unlink d3/da/c16 0 2024-04-23T02:13:54.436 INFO:tasks.workunit.client.0.smithi071.stdout:5/230: readlink d3/l18 0 2024-04-23T02:13:54.438 INFO:tasks.workunit.client.0.smithi071.stdout:5/231: mknod d3/c5b 0 2024-04-23T02:13:54.439 INFO:tasks.workunit.client.0.smithi071.stdout:5/232: chown d3/da/d1a/d1c/d4a/f58 117943120 1 2024-04-23T02:13:54.440 INFO:tasks.workunit.client.0.smithi071.stdout:3/139: truncate d7/d8/fc 87095 0 2024-04-23T02:13:54.440 INFO:tasks.workunit.client.0.smithi071.stdout:3/140: chown d7/d22/d23/d26 369569364 1 2024-04-23T02:13:54.442 INFO:tasks.workunit.client.0.smithi071.stdout:3/141: read d7/d8/f16 [690897,25065] 0 2024-04-23T02:13:54.442 INFO:tasks.workunit.client.0.smithi071.stdout:5/233: creat d3/f5c x:0 0 0 2024-04-23T02:13:54.443 INFO:tasks.workunit.client.0.smithi071.stdout:5/234: fdatasync d3/da/d1a/f49 0 2024-04-23T02:13:54.443 INFO:tasks.workunit.client.0.smithi071.stdout:3/142: chown d7/d27/d2c/f2e 6534597 1 2024-04-23T02:13:54.444 INFO:tasks.workunit.client.0.smithi071.stdout:5/235: write d3/f5c [315776,96632] 0 2024-04-23T02:13:54.446 INFO:tasks.workunit.client.0.smithi071.stdout:3/143: truncate d7/d22/f25 147979 0 2024-04-23T02:13:54.452 INFO:tasks.workunit.client.0.smithi071.stdout:5/236: dwrite d3/da/d1a/d1c/f31 [0,4194304] 0 2024-04-23T02:13:54.456 INFO:tasks.workunit.client.0.smithi071.stdout:5/237: mknod d3/da/d1a/d1c/d4a/c5d 0 2024-04-23T02:13:54.456 INFO:tasks.workunit.client.0.smithi071.stdout:3/144: dwrite d7/d10/f2f [0,4194304] 0 2024-04-23T02:13:54.461 INFO:tasks.workunit.client.0.smithi071.stdout:3/145: mkdir d7/d1d/d31 0 2024-04-23T02:13:54.461 INFO:tasks.workunit.client.0.smithi071.stdout:5/238: creat d3/da/d1a/d1c/d4a/f5e x:0 0 0 2024-04-23T02:13:54.462 INFO:tasks.workunit.client.0.smithi071.stdout:5/239: stat d3/da/d1a/d1c/d46/d4e 0 2024-04-23T02:13:54.464 INFO:tasks.workunit.client.0.smithi071.stdout:5/240: write d3/da/d1a/d1c/f2e [56848,5230] 0 2024-04-23T02:13:54.471 INFO:tasks.workunit.client.0.smithi071.stdout:5/241: mkdir d3/d1d/d43/d4b/d54/d5f 0 2024-04-23T02:13:54.472 INFO:tasks.workunit.client.0.smithi071.stdout:5/242: creat d3/df/d10/f60 x:0 0 0 2024-04-23T02:13:54.473 INFO:tasks.workunit.client.0.smithi071.stdout:5/243: dread - d3/da/d1a/f4c zero size 2024-04-23T02:13:54.474 INFO:tasks.workunit.client.0.smithi071.stdout:5/244: readlink d3/d34/l59 0 2024-04-23T02:13:54.475 INFO:tasks.workunit.client.0.smithi071.stdout:5/245: rmdir d3/d34 39 2024-04-23T02:13:54.483 INFO:tasks.workunit.client.0.smithi071.stdout:5/246: dwrite d3/d1d/d43/d4b/f53 [0,4194304] 0 2024-04-23T02:13:54.484 INFO:tasks.workunit.client.0.smithi071.stdout:5/247: fdatasync d3/df/d10/f14 0 2024-04-23T02:13:54.487 INFO:tasks.workunit.client.1.smithi192.stdout:5/310: sync 2024-04-23T02:13:54.487 INFO:tasks.workunit.client.1.smithi192.stdout:9/288: sync 2024-04-23T02:13:54.487 INFO:tasks.workunit.client.1.smithi192.stdout:3/248: sync 2024-04-23T02:13:54.488 INFO:tasks.workunit.client.0.smithi071.stdout:5/248: unlink d3/da/f4f 0 2024-04-23T02:13:54.488 INFO:tasks.workunit.client.1.smithi192.stdout:5/311: chown d2/db/f18 2198 1 2024-04-23T02:13:54.489 INFO:tasks.workunit.client.0.smithi071.stdout:3/146: dwrite d7/d8/fc [0,4194304] 0 2024-04-23T02:13:54.490 INFO:tasks.workunit.client.0.smithi071.stdout:5/249: write d3/da/d1a/d1c/d4a/f5e [408324,104627] 0 2024-04-23T02:13:54.490 INFO:tasks.workunit.client.1.smithi192.stdout:3/249: mknod d1/d3/d17/d1f/c52 0 2024-04-23T02:13:54.491 INFO:tasks.workunit.client.1.smithi192.stdout:5/312: dread d2/db/d16/d1b/f1f [0,4194304] 0 2024-04-23T02:13:54.492 INFO:tasks.workunit.client.0.smithi071.stdout:5/250: write d3/df/d10/f14 [3871401,87248] 0 2024-04-23T02:13:54.492 INFO:tasks.workunit.client.0.smithi071.stdout:3/147: mknod d7/c32 0 2024-04-23T02:13:54.494 INFO:tasks.workunit.client.0.smithi071.stdout:3/148: symlink d7/d27/d2c/l33 0 2024-04-23T02:13:54.494 INFO:tasks.workunit.client.1.smithi192.stdout:9/289: dread d0/d9/d44/d4f/f3d [0,4194304] 0 2024-04-23T02:13:54.495 INFO:tasks.workunit.client.1.smithi192.stdout:5/313: unlink d2/f2a 0 2024-04-23T02:13:54.495 INFO:tasks.workunit.client.1.smithi192.stdout:3/250: creat d1/d3/d17/d2b/f53 x:0 0 0 2024-04-23T02:13:54.495 INFO:tasks.workunit.client.1.smithi192.stdout:9/290: chown d0/f34 804554631 1 2024-04-23T02:13:54.496 INFO:tasks.workunit.client.0.smithi071.stdout:5/251: rename d3/df/d10/l21 to d3/df/l61 0 2024-04-23T02:13:54.497 INFO:tasks.workunit.client.1.smithi192.stdout:5/314: read d2/d6/d2d/d41/f38 [169653,85905] 0 2024-04-23T02:13:54.497 INFO:tasks.workunit.client.0.smithi071.stdout:3/149: mknod d7/d28/c34 0 2024-04-23T02:13:54.498 INFO:tasks.workunit.client.0.smithi071.stdout:5/252: creat d3/da/d1a/d1c/f62 x:0 0 0 2024-04-23T02:13:54.499 INFO:tasks.workunit.client.0.smithi071.stdout:5/253: stat d3/d1d/d43/d4b 0 2024-04-23T02:13:54.499 INFO:tasks.workunit.client.1.smithi192.stdout:9/291: symlink d0/d1e/d3c/d5f/l6b 0 2024-04-23T02:13:54.500 INFO:tasks.workunit.client.0.smithi071.stdout:3/150: symlink d7/d28/l35 0 2024-04-23T02:13:54.500 INFO:tasks.workunit.client.1.smithi192.stdout:9/292: read - d0/d9/d44/f5e zero size 2024-04-23T02:13:54.500 INFO:tasks.workunit.client.1.smithi192.stdout:3/251: unlink d1/d3/d17/c2c 0 2024-04-23T02:13:54.501 INFO:tasks.workunit.client.0.smithi071.stdout:5/254: mknod d3/d1d/d2c/c63 0 2024-04-23T02:13:54.503 INFO:tasks.workunit.client.1.smithi192.stdout:5/315: rmdir d2/d2e/d56 0 2024-04-23T02:13:54.503 INFO:tasks.workunit.client.0.smithi071.stdout:5/255: symlink d3/d34/l64 0 2024-04-23T02:13:54.504 INFO:tasks.workunit.client.1.smithi192.stdout:5/316: write d2/db/d2b/f3a [105959,94280] 0 2024-04-23T02:13:54.507 INFO:tasks.workunit.client.0.smithi071.stdout:3/151: dread d7/d8/f16 [0,4194304] 0 2024-04-23T02:13:54.508 INFO:tasks.workunit.client.0.smithi071.stdout:3/152: write d7/d10/f15 [487692,33417] 0 2024-04-23T02:13:54.516 INFO:tasks.workunit.client.0.smithi071.stdout:5/256: truncate d3/d1d/d2c/f2d 2277866 0 2024-04-23T02:13:54.519 INFO:tasks.workunit.client.0.smithi071.stdout:5/257: creat d3/d1d/d43/f65 x:0 0 0 2024-04-23T02:13:54.521 INFO:tasks.workunit.client.1.smithi192.stdout:5/317: dread d2/db/d16/f26 [0,4194304] 0 2024-04-23T02:13:54.521 INFO:tasks.workunit.client.0.smithi071.stdout:3/153: dwrite d7/d22/f25 [0,4194304] 0 2024-04-23T02:13:54.523 INFO:tasks.workunit.client.0.smithi071.stdout:5/258: rename d3/f12 to d3/d1d/d43/d4b/d54/f66 0 2024-04-23T02:13:54.523 INFO:tasks.workunit.client.0.smithi071.stdout:5/259: readlink d3/d1d/l36 0 2024-04-23T02:13:54.526 INFO:tasks.workunit.client.0.smithi071.stdout:2/166: sync 2024-04-23T02:13:54.526 INFO:tasks.workunit.client.0.smithi071.stdout:1/106: sync 2024-04-23T02:13:54.526 INFO:tasks.workunit.client.0.smithi071.stdout:7/162: sync 2024-04-23T02:13:54.526 INFO:tasks.workunit.client.0.smithi071.stdout:0/191: sync 2024-04-23T02:13:54.527 INFO:tasks.workunit.client.0.smithi071.stdout:8/233: sync 2024-04-23T02:13:54.527 INFO:tasks.workunit.client.0.smithi071.stdout:6/203: sync 2024-04-23T02:13:54.527 INFO:tasks.workunit.client.0.smithi071.stdout:3/154: mknod d7/d27/d2c/c36 0 2024-04-23T02:13:54.528 INFO:tasks.workunit.client.0.smithi071.stdout:5/260: fdatasync d3/da/d1a/d1c/d4a/f58 0 2024-04-23T02:13:54.528 INFO:tasks.workunit.client.0.smithi071.stdout:7/163: chown d3/l10 25887 1 2024-04-23T02:13:54.530 INFO:tasks.workunit.client.0.smithi071.stdout:8/234: symlink dc/d10/l42 0 2024-04-23T02:13:54.533 INFO:tasks.workunit.client.0.smithi071.stdout:5/261: dread - d3/da/d1a/d1c/f62 zero size 2024-04-23T02:13:54.533 INFO:tasks.workunit.client.0.smithi071.stdout:2/167: creat d0/d2/d26/f32 x:0 0 0 2024-04-23T02:13:54.536 INFO:tasks.workunit.client.0.smithi071.stdout:8/235: read dc/f28 [1133167,109553] 0 2024-04-23T02:13:54.540 INFO:tasks.workunit.client.0.smithi071.stdout:2/168: fdatasync d0/d2/fb 0 2024-04-23T02:13:54.542 INFO:tasks.workunit.client.0.smithi071.stdout:3/155: truncate d7/d8/f11 1579110 0 2024-04-23T02:13:54.543 INFO:tasks.workunit.client.0.smithi071.stdout:8/236: fdatasync dc/d10/d32/f3b 0 2024-04-23T02:13:54.548 INFO:tasks.workunit.client.0.smithi071.stdout:1/107: dwrite db/fd [0,4194304] 0 2024-04-23T02:13:54.548 INFO:tasks.workunit.client.0.smithi071.stdout:6/204: dwrite d6/d22/f37 [0,4194304] 0 2024-04-23T02:13:54.554 INFO:tasks.workunit.client.0.smithi071.stdout:2/169: mknod d0/d2/d26/d14/d1f/c33 0 2024-04-23T02:13:54.556 INFO:tasks.workunit.client.0.smithi071.stdout:1/108: dread - db/df/f19 zero size 2024-04-23T02:13:54.558 INFO:tasks.workunit.client.0.smithi071.stdout:7/164: dwrite d3/d17/d1c/d25/f33 [0,4194304] 0 2024-04-23T02:13:54.558 INFO:tasks.workunit.client.0.smithi071.stdout:3/156: write f5 [4948551,121954] 0 2024-04-23T02:13:54.559 INFO:tasks.workunit.client.0.smithi071.stdout:0/192: dwrite d0/d3/de/d10/f1f [4194304,4194304] 0 2024-04-23T02:13:54.559 INFO:tasks.workunit.client.0.smithi071.stdout:2/170: mkdir d0/d2/d34 0 2024-04-23T02:13:54.567 INFO:tasks.workunit.client.0.smithi071.stdout:8/237: truncate dc/d10/d32/f3b 1194443 0 2024-04-23T02:13:54.567 INFO:tasks.workunit.client.0.smithi071.stdout:1/109: creat db/df/f1c x:0 0 0 2024-04-23T02:13:54.568 INFO:tasks.workunit.client.0.smithi071.stdout:1/110: chown f7 134491651 1 2024-04-23T02:13:54.568 INFO:tasks.workunit.client.0.smithi071.stdout:7/165: fsync d3/f1d 0 2024-04-23T02:13:54.568 INFO:tasks.workunit.client.0.smithi071.stdout:3/157: rename d7/c17 to d7/c37 0 2024-04-23T02:13:54.569 INFO:tasks.workunit.client.0.smithi071.stdout:2/171: mknod d0/c35 0 2024-04-23T02:13:54.569 INFO:tasks.workunit.client.0.smithi071.stdout:8/238: write dc/f37 [4713361,103924] 0 2024-04-23T02:13:54.571 INFO:tasks.workunit.client.0.smithi071.stdout:0/193: creat d0/d3/f3f x:0 0 0 2024-04-23T02:13:54.573 INFO:tasks.workunit.client.0.smithi071.stdout:3/158: mkdir d7/d10/d38 0 2024-04-23T02:13:54.575 INFO:tasks.workunit.client.0.smithi071.stdout:7/166: mkdir d3/d17/d1c/d25/d3a 0 2024-04-23T02:13:54.576 INFO:tasks.workunit.client.0.smithi071.stdout:6/205: dwrite d6/d1f/f25 [0,4194304] 0 2024-04-23T02:13:54.576 INFO:tasks.workunit.client.0.smithi071.stdout:5/262: truncate d3/df/d10/f14 3202163 0 2024-04-23T02:13:54.576 INFO:tasks.workunit.client.0.smithi071.stdout:3/159: chown d7/d10/f2f 8203 1 2024-04-23T02:13:54.576 INFO:tasks.workunit.client.0.smithi071.stdout:2/172: rmdir d0/d2/d26/d14 39 2024-04-23T02:13:54.578 INFO:tasks.workunit.client.0.smithi071.stdout:1/111: getdents db 0 2024-04-23T02:13:54.580 INFO:tasks.workunit.client.0.smithi071.stdout:0/194: unlink d0/d3/ld 0 2024-04-23T02:13:54.582 INFO:tasks.workunit.client.0.smithi071.stdout:3/160: truncate d7/d8/f1b 15293 0 2024-04-23T02:13:54.583 INFO:tasks.workunit.client.1.smithi192.stdout:9/293: symlink d0/d9/d44/d58/l6c 0 2024-04-23T02:13:54.585 INFO:tasks.workunit.client.0.smithi071.stdout:6/206: rmdir d6/d1f 39 2024-04-23T02:13:54.586 INFO:tasks.workunit.client.0.smithi071.stdout:5/263: chown d3/d1d/d2c/f2d 1175019 1 2024-04-23T02:13:54.586 INFO:tasks.workunit.client.1.smithi192.stdout:5/318: write d2/db/d16/d1b/d3d/f67 [258848,72191] 0 2024-04-23T02:13:54.587 INFO:tasks.workunit.client.1.smithi192.stdout:9/294: mkdir d0/d9/d44/d51/d5b/d6d 0 2024-04-23T02:13:54.588 INFO:tasks.workunit.client.0.smithi071.stdout:9/213: sync 2024-04-23T02:13:54.589 INFO:tasks.workunit.client.0.smithi071.stdout:7/167: mkdir d3/d17/d1c/d25/d34/d39/d3b 0 2024-04-23T02:13:54.589 INFO:tasks.workunit.client.0.smithi071.stdout:1/112: chown db/d11/c16 4 1 2024-04-23T02:13:54.589 INFO:tasks.workunit.client.1.smithi192.stdout:3/252: creat d1/d3/d17/d1f/d33/d24/d34/f54 x:0 0 0 2024-04-23T02:13:54.589 INFO:tasks.workunit.client.0.smithi071.stdout:2/173: mkdir d0/d2/d26/d14/d1f/d36 0 2024-04-23T02:13:54.590 INFO:tasks.workunit.client.1.smithi192.stdout:9/295: mknod d0/d1e/d3c/c6e 0 2024-04-23T02:13:54.590 INFO:tasks.workunit.client.0.smithi071.stdout:6/207: unlink d6/f14 0 2024-04-23T02:13:54.591 INFO:tasks.workunit.client.1.smithi192.stdout:5/319: mknod d2/db/d16/d1b/d3d/c6a 0 2024-04-23T02:13:54.591 INFO:tasks.workunit.client.0.smithi071.stdout:8/239: dwrite dc/f28 [4194304,4194304] 0 2024-04-23T02:13:54.593 INFO:tasks.workunit.client.1.smithi192.stdout:5/320: chown d2/db/d16/d1b/d3d/f67 442864 1 2024-04-23T02:13:54.594 INFO:tasks.workunit.client.0.smithi071.stdout:5/264: symlink d3/df/l67 0 2024-04-23T02:13:54.594 INFO:tasks.workunit.client.0.smithi071.stdout:3/161: truncate d7/d22/d23/d26/f2b 900990 0 2024-04-23T02:13:54.594 INFO:tasks.workunit.client.0.smithi071.stdout:7/168: readlink d3/l15 0 2024-04-23T02:13:54.595 INFO:tasks.workunit.client.0.smithi071.stdout:0/195: dread - d0/d3/de/d10/d14/d32/d37/d26/f2e zero size 2024-04-23T02:13:54.597 INFO:tasks.workunit.client.0.smithi071.stdout:6/208: dread - d6/f34 zero size 2024-04-23T02:13:54.598 INFO:tasks.workunit.client.0.smithi071.stdout:5/265: chown d3/d1d/f2a 15470 1 2024-04-23T02:13:54.598 INFO:tasks.workunit.client.1.smithi192.stdout:3/253: dwrite d1/f5 [4194304,4194304] 0 2024-04-23T02:13:54.604 INFO:tasks.workunit.client.1.smithi192.stdout:9/296: dwrite d0/d9/d44/d51/d5b/f67 [0,4194304] 0 2024-04-23T02:13:54.604 INFO:tasks.workunit.client.0.smithi071.stdout:3/162: symlink d7/d27/d2c/l39 0 2024-04-23T02:13:54.606 INFO:tasks.workunit.client.1.smithi192.stdout:5/321: link d2/d6/f68 d2/d6/d2d/f6b 0 2024-04-23T02:13:54.607 INFO:tasks.workunit.client.0.smithi071.stdout:1/113: rename db/df/d13/f15 to db/d11/f1d 0 2024-04-23T02:13:54.608 INFO:tasks.workunit.client.1.smithi192.stdout:2/233: sync 2024-04-23T02:13:54.610 INFO:tasks.workunit.client.0.smithi071.stdout:3/163: chown d7/fd 125260920 1 2024-04-23T02:13:54.611 INFO:tasks.workunit.client.1.smithi192.stdout:5/322: chown d2/d2e/d4e/c69 9077 1 2024-04-23T02:13:54.611 INFO:tasks.workunit.client.1.smithi192.stdout:3/254: dread d1/d3/d17/d2b/f32 [0,4194304] 0 2024-04-23T02:13:54.612 INFO:tasks.workunit.client.1.smithi192.stdout:9/297: dread d0/f34 [0,4194304] 0 2024-04-23T02:13:54.613 INFO:tasks.workunit.client.1.smithi192.stdout:9/298: stat d0/d9/d44/d51/d5b/f69 0 2024-04-23T02:13:54.614 INFO:tasks.workunit.client.0.smithi071.stdout:1/114: symlink db/d11/l1e 0 2024-04-23T02:13:54.617 INFO:tasks.workunit.client.0.smithi071.stdout:8/240: write dc/ff [3054291,111914] 0 2024-04-23T02:13:54.618 INFO:tasks.workunit.client.1.smithi192.stdout:2/234: creat d8/d14/f4c x:0 0 0 2024-04-23T02:13:54.618 INFO:tasks.workunit.client.0.smithi071.stdout:8/241: chown dc/d10 210 1 2024-04-23T02:13:54.622 INFO:tasks.workunit.client.0.smithi071.stdout:2/174: dwrite d0/d2/f21 [0,4194304] 0 2024-04-23T02:13:54.624 INFO:tasks.workunit.client.0.smithi071.stdout:9/214: dwrite d1/f10 [0,4194304] 0 2024-04-23T02:13:54.625 INFO:tasks.workunit.client.1.smithi192.stdout:5/323: dwrite d2/db/d16/d1b/f29 [4194304,4194304] 0 2024-04-23T02:13:54.625 INFO:tasks.workunit.client.0.smithi071.stdout:3/164: creat d7/d22/d23/f3a x:0 0 0 2024-04-23T02:13:54.626 INFO:tasks.workunit.client.1.smithi192.stdout:3/255: stat d1/d3/d17/l35 0 2024-04-23T02:13:54.628 INFO:tasks.workunit.client.0.smithi071.stdout:7/169: dwrite d3/d17/d1c/d25/d34/f35 [0,4194304] 0 2024-04-23T02:13:54.632 INFO:tasks.workunit.client.0.smithi071.stdout:1/115: truncate db/df/f1c 989753 0 2024-04-23T02:13:54.632 INFO:tasks.workunit.client.0.smithi071.stdout:8/242: creat dc/d15/d2d/f43 x:0 0 0 2024-04-23T02:13:54.632 INFO:tasks.workunit.client.0.smithi071.stdout:3/165: chown d7/d22/d23/d26 473093 1 2024-04-23T02:13:54.632 INFO:tasks.workunit.client.0.smithi071.stdout:0/196: dread d0/d3/de/ff [0,4194304] 0 2024-04-23T02:13:54.633 INFO:tasks.workunit.client.0.smithi071.stdout:2/175: chown d0/d2/d26/d14/d1f/c24 4185283 1 2024-04-23T02:13:54.633 INFO:tasks.workunit.client.1.smithi192.stdout:2/235: symlink d8/d26/d29/d43/l4d 0 2024-04-23T02:13:54.635 INFO:tasks.workunit.client.0.smithi071.stdout:5/266: dwrite d3/f22 [0,4194304] 0 2024-04-23T02:13:54.635 INFO:tasks.workunit.client.0.smithi071.stdout:9/215: mknod d1/d32/d3a/c46 0 2024-04-23T02:13:54.637 INFO:tasks.workunit.client.1.smithi192.stdout:2/236: chown d8/d25/f32 3578 1 2024-04-23T02:13:54.638 INFO:tasks.workunit.client.0.smithi071.stdout:1/116: chown l8 1430 1 2024-04-23T02:13:54.641 INFO:tasks.workunit.client.1.smithi192.stdout:2/237: chown d8/d14/c1f 324 1 2024-04-23T02:13:54.641 INFO:tasks.workunit.client.0.smithi071.stdout:2/176: write d0/f2e [400927,14273] 0 2024-04-23T02:13:54.641 INFO:tasks.workunit.client.0.smithi071.stdout:8/243: write dc/d10/f1a [455418,128383] 0 2024-04-23T02:13:54.642 INFO:tasks.workunit.client.1.smithi192.stdout:9/299: link d0/c29 d0/d9/c6f 0 2024-04-23T02:13:54.642 INFO:tasks.workunit.client.1.smithi192.stdout:2/238: chown d8/d14/c42 20636274 1 2024-04-23T02:13:54.642 INFO:tasks.workunit.client.1.smithi192.stdout:9/300: dread - d0/d9/f38 zero size 2024-04-23T02:13:54.644 INFO:tasks.workunit.client.1.smithi192.stdout:2/239: symlink d8/d26/d27/l4e 0 2024-04-23T02:13:54.645 INFO:tasks.workunit.client.0.smithi071.stdout:0/197: unlink d0/f1 0 2024-04-23T02:13:54.645 INFO:tasks.workunit.client.1.smithi192.stdout:9/301: mknod d0/d1e/d3c/d5f/c70 0 2024-04-23T02:13:54.646 INFO:tasks.workunit.client.0.smithi071.stdout:4/225: sync 2024-04-23T02:13:54.646 INFO:tasks.workunit.client.0.smithi071.stdout:9/216: write d1/d32/d3a/f3b [329840,30944] 0 2024-04-23T02:13:54.646 INFO:tasks.workunit.client.0.smithi071.stdout:7/170: creat d3/d17/d1c/d25/d34/f3c x:0 0 0 2024-04-23T02:13:54.647 INFO:tasks.workunit.client.0.smithi071.stdout:0/198: readlink d0/l7 0 2024-04-23T02:13:54.648 INFO:tasks.workunit.client.1.smithi192.stdout:2/240: mknod d8/d26/d29/c4f 0 2024-04-23T02:13:54.649 INFO:tasks.workunit.client.1.smithi192.stdout:6/415: sync 2024-04-23T02:13:54.649 INFO:tasks.workunit.client.1.smithi192.stdout:8/351: sync 2024-04-23T02:13:54.649 INFO:tasks.workunit.client.1.smithi192.stdout:4/438: sync 2024-04-23T02:13:54.649 INFO:tasks.workunit.client.1.smithi192.stdout:7/274: sync 2024-04-23T02:13:54.650 INFO:tasks.workunit.client.1.smithi192.stdout:8/352: chown d1/d9/d3e 7 1 2024-04-23T02:13:54.650 INFO:tasks.workunit.client.0.smithi071.stdout:2/177: fdatasync d0/d2/d26/d14/f23 0 2024-04-23T02:13:54.651 INFO:tasks.workunit.client.1.smithi192.stdout:2/241: readlink d8/d14/l21 0 2024-04-23T02:13:54.652 INFO:tasks.workunit.client.0.smithi071.stdout:9/217: write d1/f9 [1873940,45190] 0 2024-04-23T02:13:54.652 INFO:tasks.workunit.client.1.smithi192.stdout:8/353: chown d1/d12/l34 21303 1 2024-04-23T02:13:54.652 INFO:tasks.workunit.client.0.smithi071.stdout:1/117: mknod db/df/d13/c1f 0 2024-04-23T02:13:54.653 INFO:tasks.workunit.client.1.smithi192.stdout:2/242: chown d8/d25/l47 454060894 1 2024-04-23T02:13:54.654 INFO:tasks.workunit.client.0.smithi071.stdout:4/226: chown d2/d15/d1d/d23/ff 110195707 1 2024-04-23T02:13:54.654 INFO:tasks.workunit.client.1.smithi192.stdout:6/416: mknod d3/c88 0 2024-04-23T02:13:54.654 INFO:tasks.workunit.client.1.smithi192.stdout:9/302: link d0/c3f d0/d9/d44/d51/d5b/d6d/c71 0 2024-04-23T02:13:54.655 INFO:tasks.workunit.client.1.smithi192.stdout:8/354: mknod d1/d12/d23/d2c/d4d/c77 0 2024-04-23T02:13:54.657 INFO:tasks.workunit.client.1.smithi192.stdout:2/243: write d8/d26/d29/f24 [3623871,13404] 0 2024-04-23T02:13:54.657 INFO:tasks.workunit.client.0.smithi071.stdout:9/218: chown d1/d1a/l1e 902 1 2024-04-23T02:13:54.659 INFO:tasks.workunit.client.1.smithi192.stdout:4/439: read f4 [7563950,91838] 0 2024-04-23T02:13:54.659 INFO:tasks.workunit.client.1.smithi192.stdout:8/355: chown d1/d12/d23/d4b/d73 0 1 2024-04-23T02:13:54.659 INFO:tasks.workunit.client.0.smithi071.stdout:5/267: getdents d3/d1d/d43 0 2024-04-23T02:13:54.662 INFO:tasks.workunit.client.0.smithi071.stdout:2/178: mknod d0/d2/d26/df/c37 0 2024-04-23T02:13:54.662 INFO:tasks.workunit.client.1.smithi192.stdout:8/356: stat d1/l21 0 2024-04-23T02:13:54.663 INFO:tasks.workunit.client.0.smithi071.stdout:3/166: dwrite d7/fd [0,4194304] 0 2024-04-23T02:13:54.667 INFO:tasks.workunit.client.1.smithi192.stdout:2/244: truncate d8/d26/d27/f2f 1402174 0 2024-04-23T02:13:54.667 INFO:tasks.workunit.client.0.smithi071.stdout:6/209: dread d6/f1d [0,4194304] 0 2024-04-23T02:13:54.668 INFO:tasks.workunit.client.0.smithi071.stdout:5/268: symlink d3/d1d/d2c/l68 0 2024-04-23T02:13:54.668 INFO:tasks.workunit.client.0.smithi071.stdout:1/118: dread db/d11/f1d [0,4194304] 0 2024-04-23T02:13:54.668 INFO:tasks.workunit.client.1.smithi192.stdout:4/440: chown de/d15/d19/d3d/d34/d4a/c4f 26088806 1 2024-04-23T02:13:54.669 INFO:tasks.workunit.client.1.smithi192.stdout:9/303: symlink d0/d9/l72 0 2024-04-23T02:13:54.669 INFO:tasks.workunit.client.1.smithi192.stdout:7/275: dwrite d5/d17/d2f/d31/d41/d50/f35 [4194304,4194304] 0 2024-04-23T02:13:54.670 INFO:tasks.workunit.client.0.smithi071.stdout:2/179: dread - d0/d2/d26/d14/f22 zero size 2024-04-23T02:13:54.670 INFO:tasks.workunit.client.1.smithi192.stdout:7/276: chown d5/c6 253692323 1 2024-04-23T02:13:54.671 INFO:tasks.workunit.client.0.smithi071.stdout:7/171: dread d3/d17/d1c/d25/f33 [0,4194304] 0 2024-04-23T02:13:54.671 INFO:tasks.workunit.client.1.smithi192.stdout:8/357: symlink d1/d12/d23/d4b/d73/l78 0 2024-04-23T02:13:54.673 INFO:tasks.workunit.client.0.smithi071.stdout:5/269: creat d3/d1d/d43/f69 x:0 0 0 2024-04-23T02:13:54.674 INFO:tasks.workunit.client.0.smithi071.stdout:9/219: truncate d1/d1a/f2d 3974827 0 2024-04-23T02:13:54.674 INFO:tasks.workunit.client.1.smithi192.stdout:7/277: dread d5/d17/d1c/f2d [0,4194304] 0 2024-04-23T02:13:54.676 INFO:tasks.workunit.client.0.smithi071.stdout:7/172: chown d3/d17/d1c/d25/d34/f35 187 1 2024-04-23T02:13:54.676 INFO:tasks.workunit.client.0.smithi071.stdout:9/220: chown d1/d32 351478 1 2024-04-23T02:13:54.677 INFO:tasks.workunit.client.0.smithi071.stdout:4/227: dread d2/d15/d1d/d2c/f2e [0,4194304] 0 2024-04-23T02:13:54.677 INFO:tasks.workunit.client.0.smithi071.stdout:6/210: rename d6/fb to d6/d1f/f3b 0 2024-04-23T02:13:54.680 INFO:tasks.workunit.client.1.smithi192.stdout:9/304: fdatasync d0/d9/fb 0 2024-04-23T02:13:54.681 INFO:tasks.workunit.client.1.smithi192.stdout:4/441: creat de/d15/f8f x:0 0 0 2024-04-23T02:13:54.682 INFO:tasks.workunit.client.0.smithi071.stdout:3/167: dread d7/d10/f15 [0,4194304] 0 2024-04-23T02:13:54.684 INFO:tasks.workunit.client.0.smithi071.stdout:4/228: write d2/d15/d1d/d23/ff [106969,7896] 0 2024-04-23T02:13:54.687 INFO:tasks.workunit.client.0.smithi071.stdout:2/180: dread d0/f20 [0,4194304] 0 2024-04-23T02:13:54.687 INFO:tasks.workunit.client.0.smithi071.stdout:1/119: truncate f0 2515261 0 2024-04-23T02:13:54.688 INFO:tasks.workunit.client.0.smithi071.stdout:8/244: dread dc/f37 [0,4194304] 0 2024-04-23T02:13:54.689 INFO:tasks.workunit.client.1.smithi192.stdout:8/358: dwrite d1/d12/d23/d4b/d73/d45/d46/f53 [0,4194304] 0 2024-04-23T02:13:54.689 INFO:tasks.workunit.client.1.smithi192.stdout:3/256: dread d1/d3/f49 [0,4194304] 0 2024-04-23T02:13:54.689 INFO:tasks.workunit.client.1.smithi192.stdout:4/442: mknod de/d15/d19/d3d/d49/d26/d59/c90 0 2024-04-23T02:13:54.689 INFO:tasks.workunit.client.1.smithi192.stdout:5/324: dread d2/db/d2b/f3a [0,4194304] 0 2024-04-23T02:13:54.690 INFO:tasks.workunit.client.1.smithi192.stdout:9/305: getdents d0/d1e/d3c/d5f 0 2024-04-23T02:13:54.691 INFO:tasks.workunit.client.0.smithi071.stdout:3/168: mkdir d7/d28/d3b 0 2024-04-23T02:13:54.691 INFO:tasks.workunit.client.0.smithi071.stdout:2/181: chown d0/d2/d26/d14/d1f/c24 18 1 2024-04-23T02:13:54.692 INFO:tasks.workunit.client.0.smithi071.stdout:9/221: unlink d1/d1a/l2f 0 2024-04-23T02:13:54.692 INFO:tasks.workunit.client.1.smithi192.stdout:5/325: fsync d2/db/d16/d1b/d3d/d40/d49/f58 0 2024-04-23T02:13:54.692 INFO:tasks.workunit.client.1.smithi192.stdout:4/443: truncate de/f21 2911045 0 2024-04-23T02:13:54.694 INFO:tasks.workunit.client.0.smithi071.stdout:5/270: dread d3/d1d/d43/d4b/f53 [0,4194304] 0 2024-04-23T02:13:54.694 INFO:tasks.workunit.client.0.smithi071.stdout:1/120: mkdir db/df/d13/d20 0 2024-04-23T02:13:54.695 INFO:tasks.workunit.client.1.smithi192.stdout:4/444: readlink de/d15/d19/d3d/d49/d26/l3b 0 2024-04-23T02:13:54.695 INFO:tasks.workunit.client.0.smithi071.stdout:2/182: write d0/d2/f29 [262887,60700] 0 2024-04-23T02:13:54.696 INFO:tasks.workunit.client.1.smithi192.stdout:7/278: fdatasync d5/d17/d2f/d31/d41/d50/f35 0 2024-04-23T02:13:54.698 INFO:tasks.workunit.client.0.smithi071.stdout:4/229: mknod d2/d15/d1d/d2c/c45 0 2024-04-23T02:13:54.698 INFO:tasks.workunit.client.1.smithi192.stdout:3/257: link d1/f5 d1/f55 0 2024-04-23T02:13:54.698 INFO:tasks.workunit.client.1.smithi192.stdout:8/359: dwrite d1/d12/d23/d2c/f38 [8388608,4194304] 0 2024-04-23T02:13:54.699 INFO:tasks.workunit.client.1.smithi192.stdout:9/306: symlink d0/l73 0 2024-04-23T02:13:54.703 INFO:tasks.workunit.client.0.smithi071.stdout:0/199: dwrite d0/d3/de/d10/d16/f3e [0,4194304] 0 2024-04-23T02:13:54.703 INFO:tasks.workunit.client.0.smithi071.stdout:6/211: rename f3 to d6/f3c 0 2024-04-23T02:13:54.703 INFO:tasks.workunit.client.0.smithi071.stdout:1/121: stat db/d11/c16 0 2024-04-23T02:13:54.706 INFO:tasks.workunit.client.0.smithi071.stdout:4/230: creat d2/d2a/d2d/d39/f46 x:0 0 0 2024-04-23T02:13:54.707 INFO:tasks.workunit.client.0.smithi071.stdout:1/122: write f3 [3078699,12030] 0 2024-04-23T02:13:54.708 INFO:tasks.workunit.client.0.smithi071.stdout:2/183: dread - d0/d2/d26/d14/f22 zero size 2024-04-23T02:13:54.708 INFO:tasks.workunit.client.0.smithi071.stdout:1/123: readlink db/d11/l1e 0 2024-04-23T02:13:54.708 INFO:tasks.workunit.client.0.smithi071.stdout:7/173: getdents d3/d17 0 2024-04-23T02:13:54.709 INFO:tasks.workunit.client.0.smithi071.stdout:1/124: readlink l8 0 2024-04-23T02:13:54.710 INFO:tasks.workunit.client.0.smithi071.stdout:5/271: rename d3/f9 to d3/d34/f6a 0 2024-04-23T02:13:54.711 INFO:tasks.workunit.client.0.smithi071.stdout:7/174: readlink d3/l10 0 2024-04-23T02:13:54.712 INFO:tasks.workunit.client.1.smithi192.stdout:8/360: dwrite d1/d12/d23/d4b/d73/d45/d46/f53 [0,4194304] 0 2024-04-23T02:13:54.713 INFO:tasks.workunit.client.0.smithi071.stdout:6/212: mknod d6/d1f/c3d 0 2024-04-23T02:13:54.713 INFO:tasks.workunit.client.0.smithi071.stdout:5/272: fdatasync d3/da/d1a/d1c/f31 0 2024-04-23T02:13:54.714 INFO:tasks.workunit.client.0.smithi071.stdout:4/231: mknod d2/d15/d1d/c47 0 2024-04-23T02:13:54.717 INFO:tasks.workunit.client.1.smithi192.stdout:7/279: stat l2 0 2024-04-23T02:13:54.717 INFO:tasks.workunit.client.0.smithi071.stdout:7/175: creat d3/f3d x:0 0 0 2024-04-23T02:13:54.720 INFO:tasks.workunit.client.0.smithi071.stdout:7/176: write d3/d17/d1c/d25/f27 [5056097,72343] 0 2024-04-23T02:13:54.721 INFO:tasks.workunit.client.1.smithi192.stdout:6/417: dread d3/d15/d1b/d21/d3a/f61 [0,4194304] 0 2024-04-23T02:13:54.721 INFO:tasks.workunit.client.1.smithi192.stdout:8/361: read d1/d12/f24 [1754049,22387] 0 2024-04-23T02:13:54.721 INFO:tasks.workunit.client.0.smithi071.stdout:0/200: rename d0/d3/de/d10/d14/d32/d37/l19 to d0/l40 0 2024-04-23T02:13:54.723 INFO:tasks.workunit.client.1.smithi192.stdout:8/362: truncate d1/f72 910062 0 2024-04-23T02:13:54.723 INFO:tasks.workunit.client.0.smithi071.stdout:7/177: truncate d3/d17/d1c/d25/f2f 671284 0 2024-04-23T02:13:54.724 INFO:tasks.workunit.client.1.smithi192.stdout:7/280: fsync d5/d17/f4b 0 2024-04-23T02:13:54.725 INFO:tasks.workunit.client.0.smithi071.stdout:6/213: mkdir d6/d3e 0 2024-04-23T02:13:54.725 INFO:tasks.workunit.client.0.smithi071.stdout:1/125: mkdir db/df/d13/d20/d21 0 2024-04-23T02:13:54.726 INFO:tasks.workunit.client.1.smithi192.stdout:6/418: fsync d3/d11/f6b 0 2024-04-23T02:13:54.726 INFO:tasks.workunit.client.1.smithi192.stdout:8/363: write d1/d12/d23/f60 [589610,77115] 0 2024-04-23T02:13:54.726 INFO:tasks.workunit.client.0.smithi071.stdout:1/126: chown db/df/d13/d20/d21 971175677 1 2024-04-23T02:13:54.727 INFO:tasks.workunit.client.1.smithi192.stdout:8/364: stat d1/d9/f11 0 2024-04-23T02:13:54.728 INFO:tasks.workunit.client.0.smithi071.stdout:7/178: symlink d3/d17/d1c/l3e 0 2024-04-23T02:13:54.728 INFO:tasks.workunit.client.1.smithi192.stdout:7/281: symlink d5/d17/d1c/l52 0 2024-04-23T02:13:54.729 INFO:tasks.workunit.client.0.smithi071.stdout:4/232: link d2/d15/d1d/d23/d29/c2b d2/d2a/d2d/d39/d42/c48 0 2024-04-23T02:13:54.729 INFO:tasks.workunit.client.0.smithi071.stdout:6/214: write d6/d1f/f3b [1352930,99908] 0 2024-04-23T02:13:54.731 INFO:tasks.workunit.client.0.smithi071.stdout:1/127: unlink db/df/d13/c1f 0 2024-04-23T02:13:54.731 INFO:tasks.workunit.client.0.smithi071.stdout:6/215: chown d6/d17/c1c 512904268 1 2024-04-23T02:13:54.731 INFO:tasks.workunit.client.0.smithi071.stdout:1/128: chown db/c12 757426 1 2024-04-23T02:13:54.732 INFO:tasks.workunit.client.1.smithi192.stdout:8/365: mknod d1/d9/d3e/d59/c79 0 2024-04-23T02:13:54.732 INFO:tasks.workunit.client.0.smithi071.stdout:7/179: creat d3/d17/d1c/f3f x:0 0 0 2024-04-23T02:13:54.733 INFO:tasks.workunit.client.0.smithi071.stdout:4/233: mkdir d2/d2a/d2d/d39/d42/d49 0 2024-04-23T02:13:54.733 INFO:tasks.workunit.client.0.smithi071.stdout:6/216: write d6/d17/f33 [630703,68921] 0 2024-04-23T02:13:54.734 INFO:tasks.workunit.client.1.smithi192.stdout:8/366: truncate d1/f63 4656162 0 2024-04-23T02:13:54.734 INFO:tasks.workunit.client.0.smithi071.stdout:1/129: creat db/f22 x:0 0 0 2024-04-23T02:13:54.734 INFO:tasks.workunit.client.0.smithi071.stdout:7/180: read d3/d17/d1c/d25/f33 [2709003,70858] 0 2024-04-23T02:13:54.735 INFO:tasks.workunit.client.1.smithi192.stdout:6/419: symlink d3/d15/d1b/d21/d7c/l89 0 2024-04-23T02:13:54.736 INFO:tasks.workunit.client.1.smithi192.stdout:3/258: dread d1/d3/d17/f27 [0,4194304] 0 2024-04-23T02:13:54.736 INFO:tasks.workunit.client.0.smithi071.stdout:7/181: chown d3/fc 542323 1 2024-04-23T02:13:54.738 INFO:tasks.workunit.client.1.smithi192.stdout:8/367: symlink d1/d12/d23/d4b/d73/d4c/l7a 0 2024-04-23T02:13:54.739 INFO:tasks.workunit.client.1.smithi192.stdout:3/259: dread - d1/d3/d17/d1f/d33/d24/d34/f54 zero size 2024-04-23T02:13:54.741 INFO:tasks.workunit.client.1.smithi192.stdout:4/445: write de/d15/d2a/f5c [2258107,113082] 0 2024-04-23T02:13:54.741 INFO:tasks.workunit.client.0.smithi071.stdout:1/130: getdents db/d1a 0 2024-04-23T02:13:54.742 INFO:tasks.workunit.client.1.smithi192.stdout:7/282: dread d5/d17/d2f/f22 [0,4194304] 0 2024-04-23T02:13:54.744 INFO:tasks.workunit.client.1.smithi192.stdout:7/283: read d5/f15 [2809679,121721] 0 2024-04-23T02:13:54.745 INFO:tasks.workunit.client.1.smithi192.stdout:4/446: dread - de/d15/d19/d3d/d34/f76 zero size 2024-04-23T02:13:54.746 INFO:tasks.workunit.client.0.smithi071.stdout:7/182: mknod d3/d17/d1c/d2a/c40 0 2024-04-23T02:13:54.747 INFO:tasks.workunit.client.0.smithi071.stdout:8/245: dwrite dc/d10/d32/f3b [0,4194304] 0 2024-04-23T02:13:54.748 INFO:tasks.workunit.client.1.smithi192.stdout:3/260: truncate d1/d3/d17/f2d 384607 0 2024-04-23T02:13:54.748 INFO:tasks.workunit.client.1.smithi192.stdout:4/447: dread - de/f8c zero size 2024-04-23T02:13:54.749 INFO:tasks.workunit.client.1.smithi192.stdout:7/284: fsync d5/d17/d1c/d26/f28 0 2024-04-23T02:13:54.749 INFO:tasks.workunit.client.0.smithi071.stdout:6/217: dread d6/d17/f1a [0,4194304] 0 2024-04-23T02:13:54.751 INFO:tasks.workunit.client.0.smithi071.stdout:9/222: dwrite d1/f28 [0,4194304] 0 2024-04-23T02:13:54.751 INFO:tasks.workunit.client.0.smithi071.stdout:5/273: truncate d3/da/d1a/f1b 1107960 0 2024-04-23T02:13:54.752 INFO:tasks.workunit.client.1.smithi192.stdout:6/420: truncate d3/d15/d1b/d21/f60 3835164 0 2024-04-23T02:13:54.753 INFO:tasks.workunit.client.0.smithi071.stdout:8/246: fsync dc/d10/f1e 0 2024-04-23T02:13:54.753 INFO:tasks.workunit.client.1.smithi192.stdout:5/326: dwrite d2/db/f13 [0,4194304] 0 2024-04-23T02:13:54.754 INFO:tasks.workunit.client.0.smithi071.stdout:6/218: write d6/d22/f28 [468591,4382] 0 2024-04-23T02:13:54.754 INFO:tasks.workunit.client.0.smithi071.stdout:1/131: getdents db/d1a 0 2024-04-23T02:13:54.755 INFO:tasks.workunit.client.0.smithi071.stdout:5/274: write d3/d1d/d43/f69 [315696,69568] 0 2024-04-23T02:13:54.756 INFO:tasks.workunit.client.1.smithi192.stdout:9/307: dread d0/d9/f37 [0,4194304] 0 2024-04-23T02:13:54.757 INFO:tasks.workunit.client.1.smithi192.stdout:5/327: dread d2/d6/f59 [0,4194304] 0 2024-04-23T02:13:54.757 INFO:tasks.workunit.client.0.smithi071.stdout:1/132: stat db/df/d13/d20 0 2024-04-23T02:13:54.757 INFO:tasks.workunit.client.0.smithi071.stdout:9/223: symlink d1/d32/d41/l47 0 2024-04-23T02:13:54.759 INFO:tasks.workunit.client.0.smithi071.stdout:1/133: readlink db/d11/l1e 0 2024-04-23T02:13:54.759 INFO:tasks.workunit.client.0.smithi071.stdout:4/234: truncate d2/d15/d1d/f35 2602436 0 2024-04-23T02:13:54.761 INFO:tasks.workunit.client.0.smithi071.stdout:6/219: mkdir d6/d3e/d3f 0 2024-04-23T02:13:54.763 INFO:tasks.workunit.client.0.smithi071.stdout:9/224: fdatasync d1/fe 0 2024-04-23T02:13:54.763 INFO:tasks.workunit.client.0.smithi071.stdout:5/275: mkdir d3/d34/d6b 0 2024-04-23T02:13:54.764 INFO:tasks.workunit.client.0.smithi071.stdout:2/184: dread d0/d2/d26/d14/f23 [0,4194304] 0 2024-04-23T02:13:54.765 INFO:tasks.workunit.client.1.smithi192.stdout:9/308: read d0/d9/d44/d4f/f28 [11140,66397] 0 2024-04-23T02:13:54.765 INFO:tasks.workunit.client.1.smithi192.stdout:6/421: creat d3/d18/f8a x:0 0 0 2024-04-23T02:13:54.766 INFO:tasks.workunit.client.1.smithi192.stdout:7/285: mknod d5/d43/d4c/d4e/c53 0 2024-04-23T02:13:54.766 INFO:tasks.workunit.client.0.smithi071.stdout:8/247: rename dc/d15/f25 to dc/d10/d32/f44 0 2024-04-23T02:13:54.766 INFO:tasks.workunit.client.0.smithi071.stdout:5/276: dread - d3/da/f38 zero size 2024-04-23T02:13:54.768 INFO:tasks.workunit.client.0.smithi071.stdout:1/134: dread db/fd [0,4194304] 0 2024-04-23T02:13:54.770 INFO:tasks.workunit.client.0.smithi071.stdout:2/185: symlink d0/d2/d26/df/d30/l38 0 2024-04-23T02:13:54.770 INFO:tasks.workunit.client.0.smithi071.stdout:8/248: readlink dc/d10/l3e 0 2024-04-23T02:13:54.771 INFO:tasks.workunit.client.0.smithi071.stdout:4/235: symlink d2/d15/d1d/d23/l4a 0 2024-04-23T02:13:54.772 INFO:tasks.workunit.client.1.smithi192.stdout:8/368: dread d1/d9/fa [0,4194304] 0 2024-04-23T02:13:54.772 INFO:tasks.workunit.client.0.smithi071.stdout:1/135: fdatasync db/fd 0 2024-04-23T02:13:54.772 INFO:tasks.workunit.client.1.smithi192.stdout:9/309: rename d0/d1e to d0/d9/d44/d74 0 2024-04-23T02:13:54.773 INFO:tasks.workunit.client.0.smithi071.stdout:5/277: symlink d3/d1d/d43/l6c 0 2024-04-23T02:13:54.775 INFO:tasks.workunit.client.0.smithi071.stdout:1/136: readlink l8 0 2024-04-23T02:13:54.775 INFO:tasks.workunit.client.1.smithi192.stdout:9/310: chown d0/d9/f5a 5 1 2024-04-23T02:13:54.776 INFO:tasks.workunit.client.1.smithi192.stdout:7/286: link d5/d17/d2f/d31/f32 d5/d17/d2f/d31/d41/f54 0 2024-04-23T02:13:54.779 INFO:tasks.workunit.client.1.smithi192.stdout:4/448: write de/d15/d19/d3d/d34/d4a/f56 [299181,127473] 0 2024-04-23T02:13:54.779 INFO:tasks.workunit.client.1.smithi192.stdout:9/311: fsync d0/d9/d44/f45 0 2024-04-23T02:13:54.780 INFO:tasks.workunit.client.0.smithi071.stdout:6/220: getdents d6 0 2024-04-23T02:13:54.782 INFO:tasks.workunit.client.1.smithi192.stdout:9/312: read - d0/f4e zero size 2024-04-23T02:13:54.783 INFO:tasks.workunit.client.0.smithi071.stdout:1/137: chown f0 52 1 2024-04-23T02:13:54.783 INFO:tasks.workunit.client.1.smithi192.stdout:3/261: dwrite d1/d3/d17/d1a/f23 [4194304,4194304] 0 2024-04-23T02:13:54.784 INFO:tasks.workunit.client.1.smithi192.stdout:9/313: stat d0/d9/d44/d53 0 2024-04-23T02:13:54.784 INFO:tasks.workunit.client.1.smithi192.stdout:8/369: mkdir d1/d12/d23/d4b/d73/d4c/d71/d7b 0 2024-04-23T02:13:54.786 INFO:tasks.workunit.client.1.smithi192.stdout:4/449: write de/d15/d19/d3d/d49/d26/d59/d6a/f7f [673511,58015] 0 2024-04-23T02:13:54.787 INFO:tasks.workunit.client.0.smithi071.stdout:6/221: mknod d6/d17/c40 0 2024-04-23T02:13:54.788 INFO:tasks.workunit.client.1.smithi192.stdout:0/382: sync 2024-04-23T02:13:54.788 INFO:tasks.workunit.client.1.smithi192.stdout:1/137: sync 2024-04-23T02:13:54.788 INFO:tasks.workunit.client.1.smithi192.stdout:8/370: fdatasync d1/d9/f22 0 2024-04-23T02:13:54.789 INFO:tasks.workunit.client.0.smithi071.stdout:9/225: dwrite d1/d1a/f23 [0,4194304] 0 2024-04-23T02:13:54.791 INFO:tasks.workunit.client.1.smithi192.stdout:9/314: symlink d0/d9/d44/d58/l75 0 2024-04-23T02:13:54.791 INFO:tasks.workunit.client.1.smithi192.stdout:3/262: creat d1/f56 x:0 0 0 2024-04-23T02:13:54.793 INFO:tasks.workunit.client.0.smithi071.stdout:5/278: dread d3/da/d1a/d1c/f2e [0,4194304] 0 2024-04-23T02:13:54.796 INFO:tasks.workunit.client.0.smithi071.stdout:2/186: dwrite d0/d2/f21 [4194304,4194304] 0 2024-04-23T02:13:54.797 INFO:tasks.workunit.client.1.smithi192.stdout:9/315: dread - d0/d9/f65 zero size 2024-04-23T02:13:54.797 INFO:tasks.workunit.client.1.smithi192.stdout:7/287: dwrite d5/f15 [4194304,4194304] 0 2024-04-23T02:13:54.797 INFO:tasks.workunit.client.1.smithi192.stdout:6/422: truncate d3/f22 667474 0 2024-04-23T02:13:54.798 INFO:tasks.workunit.client.0.smithi071.stdout:1/138: rmdir db/df/d13/d20 39 2024-04-23T02:13:54.798 INFO:tasks.workunit.client.0.smithi071.stdout:5/279: readlink d3/d34/l64 0 2024-04-23T02:13:54.798 INFO:tasks.workunit.client.0.smithi071.stdout:8/249: dwrite dc/d15/d2d/f43 [0,4194304] 0 2024-04-23T02:13:54.799 INFO:tasks.workunit.client.0.smithi071.stdout:8/250: stat dc 0 2024-04-23T02:13:54.799 INFO:tasks.workunit.client.0.smithi071.stdout:4/236: dwrite d2/d2a/d2d/d39/f46 [0,4194304] 0 2024-04-23T02:13:54.800 INFO:tasks.workunit.client.0.smithi071.stdout:6/222: creat d6/d22/f41 x:0 0 0 2024-04-23T02:13:54.806 INFO:tasks.workunit.client.1.smithi192.stdout:0/383: chown d6/df/d13/d6b/d73/f7e 387990398 1 2024-04-23T02:13:54.810 INFO:tasks.workunit.client.1.smithi192.stdout:3/263: rmdir d1/d3/d17/d2b 39 2024-04-23T02:13:54.810 INFO:tasks.workunit.client.0.smithi071.stdout:9/226: symlink d1/d32/l48 0 2024-04-23T02:13:54.811 INFO:tasks.workunit.client.0.smithi071.stdout:8/251: chown dc/l1f 301782 1 2024-04-23T02:13:54.813 INFO:tasks.workunit.client.0.smithi071.stdout:7/183: dread d3/d17/f22 [0,4194304] 0 2024-04-23T02:13:54.814 INFO:tasks.workunit.client.0.smithi071.stdout:4/237: fdatasync d2/d15/d1d/d23/d17/f41 0 2024-04-23T02:13:54.815 INFO:tasks.workunit.client.0.smithi071.stdout:8/252: write dc/d15/f23 [5085790,119997] 0 2024-04-23T02:13:54.816 INFO:tasks.workunit.client.0.smithi071.stdout:6/223: creat d6/d3e/f42 x:0 0 0 2024-04-23T02:13:54.816 INFO:tasks.workunit.client.0.smithi071.stdout:5/280: mkdir d3/da/d1a/d6d 0 2024-04-23T02:13:54.817 INFO:tasks.workunit.client.1.smithi192.stdout:1/138: dwrite dc/ff [0,4194304] 0 2024-04-23T02:13:54.820 INFO:tasks.workunit.client.0.smithi071.stdout:4/238: fdatasync d2/d15/f3a 0 2024-04-23T02:13:54.821 INFO:tasks.workunit.client.0.smithi071.stdout:2/187: getdents d0/d2/d34 0 2024-04-23T02:13:54.822 INFO:tasks.workunit.client.1.smithi192.stdout:3/264: chown d1/d3/d17/d1f/l4d 729255569 1 2024-04-23T02:13:54.822 INFO:tasks.workunit.client.1.smithi192.stdout:6/423: creat d3/d15/d1b/d66/d80/f8b x:0 0 0 2024-04-23T02:13:54.823 INFO:tasks.workunit.client.1.smithi192.stdout:8/371: mkdir d1/d12/d23/d4b/d73/d4c/d71/d7b/d7c 0 2024-04-23T02:13:54.824 INFO:tasks.workunit.client.0.smithi071.stdout:8/253: mknod dc/d10/c45 0 2024-04-23T02:13:54.825 INFO:tasks.workunit.client.0.smithi071.stdout:9/227: dwrite d1/fc [0,4194304] 0 2024-04-23T02:13:54.826 INFO:tasks.workunit.client.0.smithi071.stdout:9/228: fsync d1/f30 0 2024-04-23T02:13:54.827 INFO:tasks.workunit.client.1.smithi192.stdout:8/372: read d1/fe [505892,91106] 0 2024-04-23T02:13:54.828 INFO:tasks.workunit.client.1.smithi192.stdout:0/384: write d6/d44/f46 [1496401,68839] 0 2024-04-23T02:13:54.828 INFO:tasks.workunit.client.1.smithi192.stdout:5/328: dwrite d2/d6/d2d/f6b [0,4194304] 0 2024-04-23T02:13:54.828 INFO:tasks.workunit.client.0.smithi071.stdout:4/239: truncate d2/d15/d1d/d23/f1f 1211699 0 2024-04-23T02:13:54.828 INFO:tasks.workunit.client.0.smithi071.stdout:2/188: symlink d0/l39 0 2024-04-23T02:13:54.829 INFO:tasks.workunit.client.1.smithi192.stdout:9/316: getdents d0 0 2024-04-23T02:13:54.830 INFO:tasks.workunit.client.1.smithi192.stdout:6/424: chown d3/c4f 1988 1 2024-04-23T02:13:54.830 INFO:tasks.workunit.client.1.smithi192.stdout:7/288: link d5/d17/d2f/l40 d5/d17/d1c/d26/l55 0 2024-04-23T02:13:54.831 INFO:tasks.workunit.client.0.smithi071.stdout:4/240: stat d2/d15/d1d/c40 0 2024-04-23T02:13:54.831 INFO:tasks.workunit.client.0.smithi071.stdout:9/229: unlink d1/f17 0 2024-04-23T02:13:54.831 INFO:tasks.workunit.client.0.smithi071.stdout:9/230: fdatasync d1/f30 0 2024-04-23T02:13:54.832 INFO:tasks.workunit.client.0.smithi071.stdout:7/184: dwrite d3/d17/f22 [0,4194304] 0 2024-04-23T02:13:54.834 INFO:tasks.workunit.client.1.smithi192.stdout:3/265: write d1/d3/d6/d7/d29/d3b/f41 [471195,35414] 0 2024-04-23T02:13:54.836 INFO:tasks.workunit.client.1.smithi192.stdout:6/425: fdatasync d3/d18/f8a 0 2024-04-23T02:13:54.839 INFO:tasks.workunit.client.0.smithi071.stdout:8/254: rename dc/d10/d32/f3b to dc/f46 0 2024-04-23T02:13:54.839 INFO:tasks.workunit.client.0.smithi071.stdout:5/281: getdents d3/da/d1a/d1c/d46/d4e 0 2024-04-23T02:13:54.839 INFO:tasks.workunit.client.1.smithi192.stdout:8/373: read d1/d12/d23/d4b/f5a [622088,111425] 0 2024-04-23T02:13:54.840 INFO:tasks.workunit.client.0.smithi071.stdout:8/255: stat dc/d15/f23 0 2024-04-23T02:13:54.841 INFO:tasks.workunit.client.1.smithi192.stdout:1/139: getdents dc/d14/d15 0 2024-04-23T02:13:54.842 INFO:tasks.workunit.client.0.smithi071.stdout:5/282: write d3/da/d1a/d1c/f31 [264943,96583] 0 2024-04-23T02:13:54.842 INFO:tasks.workunit.client.0.smithi071.stdout:8/256: write dc/f20 [849356,15893] 0 2024-04-23T02:13:54.842 INFO:tasks.workunit.client.0.smithi071.stdout:2/189: unlink d0/d2/d26/fc 0 2024-04-23T02:13:54.843 INFO:tasks.workunit.client.1.smithi192.stdout:1/140: readlink dc/l1b 0 2024-04-23T02:13:54.844 INFO:tasks.workunit.client.0.smithi071.stdout:9/231: creat d1/d1a/d39/d42/f49 x:0 0 0 2024-04-23T02:13:54.845 INFO:tasks.workunit.client.0.smithi071.stdout:2/190: truncate d0/d2/f29 892453 0 2024-04-23T02:13:54.846 INFO:tasks.workunit.client.0.smithi071.stdout:9/232: chown d1/d32/d3a/c46 0 1 2024-04-23T02:13:54.846 INFO:tasks.workunit.client.0.smithi071.stdout:2/191: chown d0/f20 2145242688 1 2024-04-23T02:13:54.847 INFO:tasks.workunit.client.0.smithi071.stdout:9/233: write d1/f18 [245502,131021] 0 2024-04-23T02:13:54.847 INFO:tasks.workunit.client.0.smithi071.stdout:4/241: mknod d2/d15/d1d/d23/c4b 0 2024-04-23T02:13:54.850 INFO:tasks.workunit.client.0.smithi071.stdout:9/234: write d1/d1a/f31 [3094913,99700] 0 2024-04-23T02:13:54.852 INFO:tasks.workunit.client.0.smithi071.stdout:8/257: rename dc/d10/d32/f44 to dc/d3a/f47 0 2024-04-23T02:13:54.852 INFO:tasks.workunit.client.0.smithi071.stdout:2/192: chown d0/d2/d26/f18 326151583 1 2024-04-23T02:13:54.853 INFO:tasks.workunit.client.1.smithi192.stdout:7/289: link d5/c6 d5/d17/d1c/c56 0 2024-04-23T02:13:54.853 INFO:tasks.workunit.client.0.smithi071.stdout:8/258: chown dc/d10/f1e 204 1 2024-04-23T02:13:54.854 INFO:tasks.workunit.client.1.smithi192.stdout:3/266: mkdir d1/d3/d17/d2b/d44/d57 0 2024-04-23T02:13:54.854 INFO:tasks.workunit.client.0.smithi071.stdout:9/235: creat d1/d32/f4a x:0 0 0 2024-04-23T02:13:54.856 INFO:tasks.workunit.client.0.smithi071.stdout:2/193: unlink d0/f20 0 2024-04-23T02:13:54.857 INFO:tasks.workunit.client.1.smithi192.stdout:9/317: link d0/l12 d0/d9/d44/d4f/l76 0 2024-04-23T02:13:54.857 INFO:tasks.workunit.client.0.smithi071.stdout:2/194: stat d0/f2e 0 2024-04-23T02:13:54.858 INFO:tasks.workunit.client.0.smithi071.stdout:9/236: mknod d1/d32/d3a/c4b 0 2024-04-23T02:13:54.859 INFO:tasks.workunit.client.1.smithi192.stdout:3/267: read - d1/d3/d17/d2b/f51 zero size 2024-04-23T02:13:54.860 INFO:tasks.workunit.client.0.smithi071.stdout:9/237: write d1/f10 [4288373,16636] 0 2024-04-23T02:13:54.860 INFO:tasks.workunit.client.1.smithi192.stdout:0/385: dwrite d6/f18 [0,4194304] 0 2024-04-23T02:13:54.860 INFO:tasks.workunit.client.1.smithi192.stdout:5/329: link d2/d6/d2d/c39 d2/c6c 0 2024-04-23T02:13:54.861 INFO:tasks.workunit.client.0.smithi071.stdout:1/139: write f0 [3131629,67899] 0 2024-04-23T02:13:54.861 INFO:tasks.workunit.client.1.smithi192.stdout:7/290: write d5/f2e [2920152,24217] 0 2024-04-23T02:13:54.863 INFO:tasks.workunit.client.1.smithi192.stdout:8/374: mkdir d1/d12/d7d 0 2024-04-23T02:13:54.864 INFO:tasks.workunit.client.0.smithi071.stdout:9/238: creat d1/d32/d41/f4c x:0 0 0 2024-04-23T02:13:54.868 INFO:tasks.workunit.client.1.smithi192.stdout:6/426: dwrite d3/d15/d1b/d21/d52/f85 [0,4194304] 0 2024-04-23T02:13:54.869 INFO:tasks.workunit.client.1.smithi192.stdout:5/330: write d2/d6/f4d [1108911,6132] 0 2024-04-23T02:13:54.870 INFO:tasks.workunit.client.1.smithi192.stdout:7/291: fdatasync d5/d17/d1c/f2d 0 2024-04-23T02:13:54.872 INFO:tasks.workunit.client.0.smithi071.stdout:6/224: truncate d6/d17/f36 2646066 0 2024-04-23T02:13:54.875 INFO:tasks.workunit.client.0.smithi071.stdout:4/242: getdents d2/d15/d1d/d23 0 2024-04-23T02:13:54.875 INFO:tasks.workunit.client.1.smithi192.stdout:6/427: fsync d3/d11/d6d/f59 0 2024-04-23T02:13:54.875 INFO:tasks.workunit.client.1.smithi192.stdout:7/292: fdatasync d5/d17/d1c/d26/f34 0 2024-04-23T02:13:54.880 INFO:tasks.workunit.client.1.smithi192.stdout:9/318: rename d0/d9/d44/d4f/f2e to d0/d9/d44/d51/d64/f77 0 2024-04-23T02:13:54.881 INFO:tasks.workunit.client.0.smithi071.stdout:9/239: dwrite d1/f37 [0,4194304] 0 2024-04-23T02:13:54.882 INFO:tasks.workunit.client.0.smithi071.stdout:9/240: fsync d1/f16 0 2024-04-23T02:13:54.883 INFO:tasks.workunit.client.0.smithi071.stdout:5/283: dread d3/d1d/f20 [0,4194304] 0 2024-04-23T02:13:54.883 INFO:tasks.workunit.client.0.smithi071.stdout:6/225: dread d6/d17/f1a [0,4194304] 0 2024-04-23T02:13:54.884 INFO:tasks.workunit.client.0.smithi071.stdout:7/185: truncate d3/ff 1720598 0 2024-04-23T02:13:54.884 INFO:tasks.workunit.client.1.smithi192.stdout:1/141: dwrite dc/f13 [0,4194304] 0 2024-04-23T02:13:54.884 INFO:tasks.workunit.client.0.smithi071.stdout:5/284: chown d3/da/d1a/d6d 896362 1 2024-04-23T02:13:54.885 INFO:tasks.workunit.client.1.smithi192.stdout:5/331: rmdir d2/db/d16 39 2024-04-23T02:13:54.887 INFO:tasks.workunit.client.0.smithi071.stdout:9/241: symlink d1/l4d 0 2024-04-23T02:13:54.887 INFO:tasks.workunit.client.0.smithi071.stdout:5/285: read - d3/df/d10/f48 zero size 2024-04-23T02:13:54.888 INFO:tasks.workunit.client.0.smithi071.stdout:5/286: chown d3/f1f 1 1 2024-04-23T02:13:54.890 INFO:tasks.workunit.client.0.smithi071.stdout:9/242: mkdir d1/d4e 0 2024-04-23T02:13:54.891 INFO:tasks.workunit.client.1.smithi192.stdout:5/332: write d2/d6/f4d [3342894,65890] 0 2024-04-23T02:13:54.891 INFO:tasks.workunit.client.1.smithi192.stdout:6/428: creat d3/d11/d6d/f8c x:0 0 0 2024-04-23T02:13:54.893 INFO:tasks.workunit.client.1.smithi192.stdout:7/293: rmdir d5/d17 39 2024-04-23T02:13:54.895 INFO:tasks.workunit.client.0.smithi071.stdout:9/243: creat d1/d4e/f4f x:0 0 0 2024-04-23T02:13:54.896 INFO:tasks.workunit.client.1.smithi192.stdout:8/375: dwrite d1/d12/d23/d4b/d73/d45/d46/f48 [0,4194304] 0 2024-04-23T02:13:54.897 INFO:tasks.workunit.client.1.smithi192.stdout:1/142: creat dc/d14/f21 x:0 0 0 2024-04-23T02:13:54.898 INFO:tasks.workunit.client.0.smithi071.stdout:8/259: dread dc/d10/f1a [0,4194304] 0 2024-04-23T02:13:54.899 INFO:tasks.workunit.client.0.smithi071.stdout:5/287: dwrite d3/da/d1a/d1c/f31 [0,4194304] 0 2024-04-23T02:13:54.902 INFO:tasks.workunit.client.1.smithi192.stdout:0/386: dwrite d6/d4b/f52 [0,4194304] 0 2024-04-23T02:13:54.902 INFO:tasks.workunit.client.0.smithi071.stdout:8/260: mknod dc/d15/d2d/c48 0 2024-04-23T02:13:54.903 INFO:tasks.workunit.client.0.smithi071.stdout:6/226: truncate d6/f18 2541521 0 2024-04-23T02:13:54.906 INFO:tasks.workunit.client.1.smithi192.stdout:5/333: dread d2/f2c [0,4194304] 0 2024-04-23T02:13:54.906 INFO:tasks.workunit.client.0.smithi071.stdout:8/261: creat dc/d15/d2d/f49 x:0 0 0 2024-04-23T02:13:54.906 INFO:tasks.workunit.client.0.smithi071.stdout:5/288: rename d3/d1d/d43/d4b/d54 to d3/d1d/d43/d4b/d54/d5f/d6e 22 2024-04-23T02:13:54.908 INFO:tasks.workunit.client.0.smithi071.stdout:9/244: dwrite d1/f10 [0,4194304] 0 2024-04-23T02:13:54.913 INFO:tasks.workunit.client.1.smithi192.stdout:6/429: stat d3/lf 0 2024-04-23T02:13:54.913 INFO:tasks.workunit.client.1.smithi192.stdout:1/143: symlink dc/d16/d19/l22 0 2024-04-23T02:13:54.914 INFO:tasks.workunit.client.0.smithi071.stdout:9/245: write d1/fc [2762461,73996] 0 2024-04-23T02:13:54.916 INFO:tasks.workunit.client.1.smithi192.stdout:6/430: dread - d3/d15/d5b/f82 zero size 2024-04-23T02:13:54.916 INFO:tasks.workunit.client.1.smithi192.stdout:0/387: fsync d6/f1b 0 2024-04-23T02:13:54.916 INFO:tasks.workunit.client.1.smithi192.stdout:7/294: chown d5/d17/d2f/d31/d41/d50/l38 12478 1 2024-04-23T02:13:54.918 INFO:tasks.workunit.client.0.smithi071.stdout:7/186: truncate d3/d17/f22 2999334 0 2024-04-23T02:13:54.918 INFO:tasks.workunit.client.0.smithi071.stdout:4/243: dread d2/f43 [0,4194304] 0 2024-04-23T02:13:54.921 INFO:tasks.workunit.client.0.smithi071.stdout:9/246: truncate d1/d32/d3a/f3b 1205896 0 2024-04-23T02:13:54.923 INFO:tasks.workunit.client.0.smithi071.stdout:6/227: dread d6/d22/f28 [0,4194304] 0 2024-04-23T02:13:54.923 INFO:tasks.workunit.client.0.smithi071.stdout:8/262: dwrite dc/ff [4194304,4194304] 0 2024-04-23T02:13:54.924 INFO:tasks.workunit.client.0.smithi071.stdout:9/247: creat d1/f50 x:0 0 0 2024-04-23T02:13:54.924 INFO:tasks.workunit.client.0.smithi071.stdout:6/228: stat d6/d1f/f25 0 2024-04-23T02:13:54.925 INFO:tasks.workunit.client.0.smithi071.stdout:9/248: write d1/fb [2310703,117434] 0 2024-04-23T02:13:54.926 INFO:tasks.workunit.client.1.smithi192.stdout:0/388: write d6/df/d35/d49/f54 [455753,43741] 0 2024-04-23T02:13:54.926 INFO:tasks.workunit.client.1.smithi192.stdout:7/295: dread d5/d17/d1c/f2d [0,4194304] 0 2024-04-23T02:13:54.926 INFO:tasks.workunit.client.0.smithi071.stdout:5/289: read d3/df/d10/f14 [1688372,73274] 0 2024-04-23T02:13:54.927 INFO:tasks.workunit.client.0.smithi071.stdout:9/249: write d1/fc [4515433,26465] 0 2024-04-23T02:13:54.928 INFO:tasks.workunit.client.1.smithi192.stdout:0/389: read d6/f18 [1463748,69613] 0 2024-04-23T02:13:54.929 INFO:tasks.workunit.client.0.smithi071.stdout:7/187: rename d3/d17/d1c/d25 to d3/d17/d41 0 2024-04-23T02:13:54.933 INFO:tasks.workunit.client.1.smithi192.stdout:6/431: mkdir d3/d15/d1b/d21/d52/d64/d8d 0 2024-04-23T02:13:54.934 INFO:tasks.workunit.client.1.smithi192.stdout:5/334: dread d2/db/d16/d48/d4f/f5c [0,4194304] 0 2024-04-23T02:13:54.935 INFO:tasks.workunit.client.0.smithi071.stdout:4/244: link d2/l11 d2/d15/d1d/d23/l4c 0 2024-04-23T02:13:54.937 INFO:tasks.workunit.client.0.smithi071.stdout:7/188: dread d3/d17/d41/d34/f35 [0,4194304] 0 2024-04-23T02:13:54.937 INFO:tasks.workunit.client.0.smithi071.stdout:5/290: symlink d3/d34/l6f 0 2024-04-23T02:13:54.938 INFO:tasks.workunit.client.0.smithi071.stdout:7/189: chown d3/f3d 8136 1 2024-04-23T02:13:54.938 INFO:tasks.workunit.client.0.smithi071.stdout:5/291: chown d3/df/l67 292428885 1 2024-04-23T02:13:54.939 INFO:tasks.workunit.client.1.smithi192.stdout:3/268: dread d1/f55 [0,4194304] 0 2024-04-23T02:13:54.939 INFO:tasks.workunit.client.1.smithi192.stdout:7/296: unlink d5/d17/d2f/c21 0 2024-04-23T02:13:54.939 INFO:tasks.workunit.client.1.smithi192.stdout:7/297: readlink d5/le 0 2024-04-23T02:13:54.940 INFO:tasks.workunit.client.0.smithi071.stdout:4/245: creat d2/d15/d3e/f4d x:0 0 0 2024-04-23T02:13:54.940 INFO:tasks.workunit.client.1.smithi192.stdout:3/269: chown d1/d3/d17/d1f/d33/d24/d34/d38/f48 3 1 2024-04-23T02:13:54.940 INFO:tasks.workunit.client.1.smithi192.stdout:5/335: mknod d2/d2e/d4e/c6d 0 2024-04-23T02:13:54.941 INFO:tasks.workunit.client.0.smithi071.stdout:5/292: creat d3/da/d1a/d1c/d46/d4e/f70 x:0 0 0 2024-04-23T02:13:54.941 INFO:tasks.workunit.client.0.smithi071.stdout:4/246: rmdir d2/d15/d1d/d2c 39 2024-04-23T02:13:54.942 INFO:tasks.workunit.client.1.smithi192.stdout:7/298: dread d5/d17/d1c/f2d [0,4194304] 0 2024-04-23T02:13:54.942 INFO:tasks.workunit.client.1.smithi192.stdout:6/432: chown d3/d11/l6a 0 1 2024-04-23T02:13:54.942 INFO:tasks.workunit.client.1.smithi192.stdout:0/390: getdents d6/d44/d45 0 2024-04-23T02:13:54.943 INFO:tasks.workunit.client.1.smithi192.stdout:6/433: chown d3/d15/d1b/f72 223 1 2024-04-23T02:13:54.943 INFO:tasks.workunit.client.0.smithi071.stdout:5/293: creat d3/d1d/f71 x:0 0 0 2024-04-23T02:13:54.944 INFO:tasks.workunit.client.1.smithi192.stdout:3/270: mknod d1/d3/d17/c58 0 2024-04-23T02:13:54.945 INFO:tasks.workunit.client.0.smithi071.stdout:5/294: read - d3/da/f4d zero size 2024-04-23T02:13:54.945 INFO:tasks.workunit.client.1.smithi192.stdout:7/299: stat d5/d17/d1c/d4a/c4f 0 2024-04-23T02:13:54.945 INFO:tasks.workunit.client.0.smithi071.stdout:4/247: mknod d2/d15/d1d/d23/d10/c4e 0 2024-04-23T02:13:54.946 INFO:tasks.workunit.client.1.smithi192.stdout:0/391: fsync d6/df/d13/d25/d3a/f6f 0 2024-04-23T02:13:54.947 INFO:tasks.workunit.client.0.smithi071.stdout:5/295: creat d3/d1d/d43/d4b/f72 x:0 0 0 2024-04-23T02:13:54.947 INFO:tasks.workunit.client.1.smithi192.stdout:7/300: write d5/d17/d2f/f33 [738546,115282] 0 2024-04-23T02:13:54.947 INFO:tasks.workunit.client.1.smithi192.stdout:5/336: rmdir d2/d6/d14/d3e 0 2024-04-23T02:13:54.948 INFO:tasks.workunit.client.0.smithi071.stdout:4/248: mkdir d2/d15/d1d/d23/d4f 0 2024-04-23T02:13:54.948 INFO:tasks.workunit.client.1.smithi192.stdout:3/271: rmdir d1/d3/d17/d1f/d33 39 2024-04-23T02:13:54.949 INFO:tasks.workunit.client.0.smithi071.stdout:5/296: mknod d3/da/d1a/d1c/d4a/d5a/c73 0 2024-04-23T02:13:54.951 INFO:tasks.workunit.client.1.smithi192.stdout:5/337: write d2/db/f18 [3697628,14652] 0 2024-04-23T02:13:54.954 INFO:tasks.workunit.client.0.smithi071.stdout:4/249: rename d2/d15/d1d/d23/f1f to d2/d15/d1d/d23/d4f/f50 0 2024-04-23T02:13:54.954 INFO:tasks.workunit.client.0.smithi071.stdout:5/297: dread d3/d1d/d43/d4b/d54/f66 [0,4194304] 0 2024-04-23T02:13:54.956 INFO:tasks.workunit.client.0.smithi071.stdout:4/250: symlink d2/d2a/l51 0 2024-04-23T02:13:54.956 INFO:tasks.workunit.client.1.smithi192.stdout:9/319: dread d0/d9/f26 [0,4194304] 0 2024-04-23T02:13:54.957 INFO:tasks.workunit.client.0.smithi071.stdout:5/298: mkdir d3/d34/d74 0 2024-04-23T02:13:54.957 INFO:tasks.workunit.client.0.smithi071.stdout:5/299: stat d3/da/d1a/d1c/f31 0 2024-04-23T02:13:54.958 INFO:tasks.workunit.client.1.smithi192.stdout:5/338: truncate d2/db/f11 2066244 0 2024-04-23T02:13:54.961 INFO:tasks.workunit.client.0.smithi071.stdout:4/251: rename d2/d15/d1d/d23/le to d2/d15/d3e/l52 0 2024-04-23T02:13:54.963 INFO:tasks.workunit.client.1.smithi192.stdout:5/339: creat d2/d32/f6e x:0 0 0 2024-04-23T02:13:54.963 INFO:tasks.workunit.client.1.smithi192.stdout:6/434: dread d3/d28/f56 [0,4194304] 0 2024-04-23T02:13:54.964 INFO:tasks.workunit.client.0.smithi071.stdout:4/252: fdatasync d2/d15/d1d/d23/f16 0 2024-04-23T02:13:54.965 INFO:tasks.workunit.client.0.smithi071.stdout:5/300: dwrite d3/da/f33 [0,4194304] 0 2024-04-23T02:13:54.965 INFO:tasks.workunit.client.1.smithi192.stdout:1/144: dread f4 [0,4194304] 0 2024-04-23T02:13:54.966 INFO:tasks.workunit.client.0.smithi071.stdout:5/301: chown d3/da/d1a/d1c/d46 0 1 2024-04-23T02:13:54.967 INFO:tasks.workunit.client.1.smithi192.stdout:8/376: truncate d1/df/f1c 516178 0 2024-04-23T02:13:54.969 INFO:tasks.workunit.client.1.smithi192.stdout:1/145: read f9 [543784,48268] 0 2024-04-23T02:13:54.970 INFO:tasks.workunit.client.0.smithi071.stdout:5/302: readlink d3/da/d1a/d1c/d4a/l51 0 2024-04-23T02:13:54.970 INFO:tasks.workunit.client.0.smithi071.stdout:4/253: mkdir d2/d53 0 2024-04-23T02:13:54.970 INFO:tasks.workunit.client.1.smithi192.stdout:2/245: sync 2024-04-23T02:13:54.970 INFO:tasks.workunit.client.0.smithi071.stdout:5/303: fdatasync d3/da/f13 0 2024-04-23T02:13:54.971 INFO:tasks.workunit.client.1.smithi192.stdout:5/340: chown d2/db/d16/d1b/d3d/c6a 188 1 2024-04-23T02:13:54.972 INFO:tasks.workunit.client.1.smithi192.stdout:1/146: read f9 [2188955,119177] 0 2024-04-23T02:13:54.975 INFO:tasks.workunit.client.1.smithi192.stdout:3/272: dwrite d1/d3/d17/d1f/d33/d24/d34/f54 [0,4194304] 0 2024-04-23T02:13:54.975 INFO:tasks.workunit.client.0.smithi071.stdout:4/254: creat d2/d2a/d2d/f54 x:0 0 0 2024-04-23T02:13:54.977 INFO:tasks.workunit.client.0.smithi071.stdout:5/304: stat d3/da/d1a/d1c/d4a/d5a/c73 0 2024-04-23T02:13:54.977 INFO:tasks.workunit.client.0.smithi071.stdout:9/250: truncate d1/d1a/f31 3643805 0 2024-04-23T02:13:54.978 INFO:tasks.workunit.client.1.smithi192.stdout:7/301: write d5/d17/d2f/d31/f32 [426963,91064] 0 2024-04-23T02:13:54.978 INFO:tasks.workunit.client.1.smithi192.stdout:5/341: write d2/d6/d2d/f6b [990138,107524] 0 2024-04-23T02:13:54.979 INFO:tasks.workunit.client.0.smithi071.stdout:9/251: fsync d1/f4 0 2024-04-23T02:13:54.980 INFO:tasks.workunit.client.1.smithi192.stdout:2/246: write d8/f36 [6659358,36323] 0 2024-04-23T02:13:54.981 INFO:tasks.workunit.client.0.smithi071.stdout:4/255: fsync d2/d15/d1d/d23/f16 0 2024-04-23T02:13:54.983 INFO:tasks.workunit.client.1.smithi192.stdout:8/377: rename d1/d12/d23/d4b/d73/d4c/d71 to d1/d12/d23/d4b/d73/d7e 0 2024-04-23T02:13:54.984 INFO:tasks.workunit.client.0.smithi071.stdout:8/263: dwrite dc/d10/f3d [0,4194304] 0 2024-04-23T02:13:54.985 INFO:tasks.workunit.client.0.smithi071.stdout:5/305: creat d3/d1d/d43/f75 x:0 0 0 2024-04-23T02:13:54.985 INFO:tasks.workunit.client.1.smithi192.stdout:7/302: write d5/d17/d1c/f3c [511596,109067] 0 2024-04-23T02:13:54.985 INFO:tasks.workunit.client.1.smithi192.stdout:3/273: chown d1/d3/d17/d1f/d33/d24/d34/d38/c3f 7322 1 2024-04-23T02:13:54.986 INFO:tasks.workunit.client.1.smithi192.stdout:1/147: mkdir dc/d14/d23 0 2024-04-23T02:13:54.986 INFO:tasks.workunit.client.1.smithi192.stdout:0/392: dread d6/df/d47/f61 [0,4194304] 0 2024-04-23T02:13:54.990 INFO:tasks.workunit.client.0.smithi071.stdout:5/306: symlink d3/da/l76 0 2024-04-23T02:13:54.991 INFO:tasks.workunit.client.1.smithi192.stdout:2/247: mkdir d8/d26/d29/d43/d50 0 2024-04-23T02:13:54.992 INFO:tasks.workunit.client.0.smithi071.stdout:8/264: rename dc/f37 to dc/d10/d32/f4a 0 2024-04-23T02:13:54.992 INFO:tasks.workunit.client.1.smithi192.stdout:6/435: getdents d3/d15/d1b/d21/d7c 0 2024-04-23T02:13:54.993 INFO:tasks.workunit.client.1.smithi192.stdout:0/393: write d6/f72 [248492,91853] 0 2024-04-23T02:13:54.994 INFO:tasks.workunit.client.1.smithi192.stdout:1/148: rename dc/d14/d15 to dc/d14/d15/d18/d24 22 2024-04-23T02:13:54.994 INFO:tasks.workunit.client.1.smithi192.stdout:7/303: truncate d5/d17/d1c/f3c 1105566 0 2024-04-23T02:13:54.997 INFO:tasks.workunit.client.0.smithi071.stdout:5/307: dwrite d3/f11 [0,4194304] 0 2024-04-23T02:13:54.998 INFO:tasks.workunit.client.0.smithi071.stdout:8/265: write f9 [4374094,110421] 0 2024-04-23T02:13:55.002 INFO:tasks.workunit.client.1.smithi192.stdout:9/320: dwrite d0/d9/d44/d4f/f3d [0,4194304] 0 2024-04-23T02:13:55.002 INFO:tasks.workunit.client.0.smithi071.stdout:8/266: stat dc/f28 0 2024-04-23T02:13:55.003 INFO:tasks.workunit.client.0.smithi071.stdout:8/267: creat dc/d10/d32/f4b x:0 0 0 2024-04-23T02:13:55.004 INFO:tasks.workunit.client.1.smithi192.stdout:1/149: write dc/ff [2069653,50127] 0 2024-04-23T02:13:55.007 INFO:tasks.workunit.client.1.smithi192.stdout:3/274: mknod d1/d3/d6/c59 0 2024-04-23T02:13:55.008 INFO:tasks.workunit.client.1.smithi192.stdout:2/248: chown d8/d26/d29/d43 108649 1 2024-04-23T02:13:55.008 INFO:tasks.workunit.client.1.smithi192.stdout:5/342: link d2/d6/d2d/c39 d2/d6/d2d/c6f 0 2024-04-23T02:13:55.009 INFO:tasks.workunit.client.1.smithi192.stdout:8/378: creat d1/d12/d23/d4b/d73/d7e/d7b/d7c/f7f x:0 0 0 2024-04-23T02:13:55.010 INFO:tasks.workunit.client.0.smithi071.stdout:5/308: rename d3/da/d1a/d1c/c57 to d3/d1d/c77 0 2024-04-23T02:13:55.010 INFO:tasks.workunit.client.1.smithi192.stdout:7/304: rename d5/d17/d2f to d5/d17/d1c/d57 0 2024-04-23T02:13:55.011 INFO:tasks.workunit.client.0.smithi071.stdout:8/268: mkdir dc/d4c 0 2024-04-23T02:13:55.012 INFO:tasks.workunit.client.1.smithi192.stdout:2/249: write d8/d14/f4c [68791,73555] 0 2024-04-23T02:13:55.014 INFO:tasks.workunit.client.1.smithi192.stdout:1/150: symlink dc/d14/d15/l25 0 2024-04-23T02:13:55.015 INFO:tasks.workunit.client.1.smithi192.stdout:2/250: truncate d8/d25/f3f 199378 0 2024-04-23T02:13:55.016 INFO:tasks.workunit.client.0.smithi071.stdout:4/256: truncate d2/d15/d1d/d23/d29/f31 1550878 0 2024-04-23T02:13:55.016 INFO:tasks.workunit.client.1.smithi192.stdout:0/394: creat d6/df/f83 x:0 0 0 2024-04-23T02:13:55.017 INFO:tasks.workunit.client.0.smithi071.stdout:4/257: readlink d2/d2a/l51 0 2024-04-23T02:13:55.018 INFO:tasks.workunit.client.1.smithi192.stdout:5/343: creat d2/db/d16/d1b/d3d/d40/d49/f70 x:0 0 0 2024-04-23T02:13:55.019 INFO:tasks.workunit.client.1.smithi192.stdout:0/395: chown d6/df/d13/d25/d3a/d7b 494289 1 2024-04-23T02:13:55.020 INFO:tasks.workunit.client.0.smithi071.stdout:5/309: dwrite d3/d1d/f2a [0,4194304] 0 2024-04-23T02:13:55.021 INFO:tasks.workunit.client.1.smithi192.stdout:6/436: rename d3/d11/d6d to d3/d15/d1b/d66/d80/d8e 0 2024-04-23T02:13:55.021 INFO:tasks.workunit.client.0.smithi071.stdout:8/269: dwrite dc/d15/f23 [0,4194304] 0 2024-04-23T02:13:55.022 INFO:tasks.workunit.client.1.smithi192.stdout:3/275: rename d1/d3/d17/d2b/d44 to d1/d3/d17/d2b/d44/d57/d5a 22 2024-04-23T02:13:55.022 INFO:tasks.workunit.client.1.smithi192.stdout:5/344: chown d2/d6/f12 210997178 1 2024-04-23T02:13:55.022 INFO:tasks.workunit.client.0.smithi071.stdout:4/258: creat d2/d2a/d2d/d39/d42/f55 x:0 0 0 2024-04-23T02:13:55.023 INFO:tasks.workunit.client.0.smithi071.stdout:8/270: chown dc/ce 960 1 2024-04-23T02:13:55.023 INFO:tasks.workunit.client.1.smithi192.stdout:4/450: sync 2024-04-23T02:13:55.024 INFO:tasks.workunit.client.0.smithi071.stdout:4/259: chown d2/d15/d1d/d23 41 1 2024-04-23T02:13:55.024 INFO:tasks.workunit.client.0.smithi071.stdout:8/271: chown dc/d15/d2d/f43 518052 1 2024-04-23T02:13:55.030 INFO:tasks.workunit.client.1.smithi192.stdout:7/305: dwrite d5/d17/d1c/f45 [0,4194304] 0 2024-04-23T02:13:55.033 INFO:tasks.workunit.client.0.smithi071.stdout:5/310: link d3/f1f d3/da/d1a/d1c/d4a/f78 0 2024-04-23T02:13:55.035 INFO:tasks.workunit.client.0.smithi071.stdout:5/311: dread - d3/da/d1a/d1c/f39 zero size 2024-04-23T02:13:55.035 INFO:tasks.workunit.client.0.smithi071.stdout:8/272: getdents dc/d15/d2d 0 2024-04-23T02:13:55.036 INFO:tasks.workunit.client.1.smithi192.stdout:0/396: creat d6/d44/d71/f84 x:0 0 0 2024-04-23T02:13:55.036 INFO:tasks.workunit.client.1.smithi192.stdout:4/451: creat de/d15/d19/d3d/d49/d70/f91 x:0 0 0 2024-04-23T02:13:55.036 INFO:tasks.workunit.client.0.smithi071.stdout:5/312: fsync d3/df/d10/f48 0 2024-04-23T02:13:55.037 INFO:tasks.workunit.client.1.smithi192.stdout:3/276: unlink d1/d3/d17/d2b/l47 0 2024-04-23T02:13:55.038 INFO:tasks.workunit.client.0.smithi071.stdout:8/273: symlink dc/d3a/l4d 0 2024-04-23T02:13:55.038 INFO:tasks.workunit.client.1.smithi192.stdout:0/397: chown d6/d44/d45 10169567 1 2024-04-23T02:13:55.039 INFO:tasks.workunit.client.0.smithi071.stdout:8/274: fdatasync dc/d15/d2d/f38 0 2024-04-23T02:13:55.039 INFO:tasks.workunit.client.0.smithi071.stdout:5/313: symlink d3/da/d1a/d1c/d46/l79 0 2024-04-23T02:13:55.039 INFO:tasks.workunit.client.1.smithi192.stdout:7/306: stat d5/d17/d1c/c3a 0 2024-04-23T02:13:55.041 INFO:tasks.workunit.client.0.smithi071.stdout:5/314: dread - d3/da/d1a/d1c/d46/d4e/f50 zero size 2024-04-23T02:13:55.041 INFO:tasks.workunit.client.1.smithi192.stdout:6/437: dread d3/d15/d1b/d21/d52/f85 [0,4194304] 0 2024-04-23T02:13:55.042 INFO:tasks.workunit.client.0.smithi071.stdout:5/315: readlink d3/d34/l59 0 2024-04-23T02:13:55.043 INFO:tasks.workunit.client.1.smithi192.stdout:5/345: write d2/db/d2b/f3a [1011435,118105] 0 2024-04-23T02:13:55.043 INFO:tasks.workunit.client.1.smithi192.stdout:2/251: getdents d8/d26 0 2024-04-23T02:13:55.045 INFO:tasks.workunit.client.0.smithi071.stdout:5/316: truncate d3/da/f13 2595120 0 2024-04-23T02:13:55.046 INFO:tasks.workunit.client.0.smithi071.stdout:8/275: dwrite dc/f20 [0,4194304] 0 2024-04-23T02:13:55.047 INFO:tasks.workunit.client.1.smithi192.stdout:3/277: symlink d1/l5b 0 2024-04-23T02:13:55.050 INFO:tasks.workunit.client.0.smithi071.stdout:8/276: symlink dc/d10/l4e 0 2024-04-23T02:13:55.051 INFO:tasks.workunit.client.1.smithi192.stdout:2/252: mkdir d8/d14/d51 0 2024-04-23T02:13:55.052 INFO:tasks.workunit.client.1.smithi192.stdout:5/346: write d2/db/d16/d1b/f29 [1382141,48038] 0 2024-04-23T02:13:55.054 INFO:tasks.workunit.client.0.smithi071.stdout:8/277: rename dc/d10/c16 to dc/d10/d30/c4f 0 2024-04-23T02:13:55.055 INFO:tasks.workunit.client.1.smithi192.stdout:6/438: creat d3/d15/d1b/f8f x:0 0 0 2024-04-23T02:13:55.056 INFO:tasks.workunit.client.0.smithi071.stdout:8/278: write f8 [5227769,108010] 0 2024-04-23T02:13:55.057 INFO:tasks.workunit.client.0.smithi071.stdout:4/260: dread d2/d15/d1d/f35 [0,4194304] 0 2024-04-23T02:13:55.058 INFO:tasks.workunit.client.1.smithi192.stdout:2/253: readlink d8/d14/l22 0 2024-04-23T02:13:55.059 INFO:tasks.workunit.client.1.smithi192.stdout:1/151: dread f3 [4194304,4194304] 0 2024-04-23T02:13:55.060 INFO:tasks.workunit.client.1.smithi192.stdout:2/254: dread - d8/d26/d29/f4b zero size 2024-04-23T02:13:55.061 INFO:tasks.workunit.client.0.smithi071.stdout:4/261: symlink d2/d15/d1d/d23/l56 0 2024-04-23T02:13:55.061 INFO:tasks.workunit.client.0.smithi071.stdout:8/279: dread dc/ff [4194304,4194304] 0 2024-04-23T02:13:55.061 INFO:tasks.workunit.client.0.smithi071.stdout:5/317: dwrite d3/da/d1a/d1c/d4a/f78 [0,4194304] 0 2024-04-23T02:13:55.062 INFO:tasks.workunit.client.0.smithi071.stdout:8/280: write dc/d3a/f3c [663785,39187] 0 2024-04-23T02:13:55.063 INFO:tasks.workunit.client.1.smithi192.stdout:4/452: dwrite de/d15/f8f [0,4194304] 0 2024-04-23T02:13:55.064 INFO:tasks.workunit.client.1.smithi192.stdout:2/255: readlink d8/d25/l47 0 2024-04-23T02:13:55.064 INFO:tasks.workunit.client.0.smithi071.stdout:8/281: mkdir dc/d15/d50 0 2024-04-23T02:13:55.066 INFO:tasks.workunit.client.1.smithi192.stdout:1/152: symlink dc/d16/d19/l26 0 2024-04-23T02:13:55.067 INFO:tasks.workunit.client.0.smithi071.stdout:8/282: mknod dc/d15/c51 0 2024-04-23T02:13:55.067 INFO:tasks.workunit.client.0.smithi071.stdout:5/318: rename d3/df/d10/c25 to d3/da/d1a/d1c/d4a/c7a 0 2024-04-23T02:13:55.068 INFO:tasks.workunit.client.1.smithi192.stdout:0/398: truncate d6/df/d13/d25/f3d 652052 0 2024-04-23T02:13:55.068 INFO:tasks.workunit.client.0.smithi071.stdout:5/319: stat d3/d1d/d43/l6c 0 2024-04-23T02:13:55.073 INFO:tasks.workunit.client.1.smithi192.stdout:5/347: dwrite d2/d32/f6e [0,4194304] 0 2024-04-23T02:13:55.073 INFO:tasks.workunit.client.1.smithi192.stdout:7/307: truncate d5/d17/d1c/d57/d31/d41/f54 488596 0 2024-04-23T02:13:55.074 INFO:tasks.workunit.client.1.smithi192.stdout:4/453: read de/d15/d19/d1a/f3a [337137,101616] 0 2024-04-23T02:13:55.074 INFO:tasks.workunit.client.1.smithi192.stdout:5/348: stat d2/d6/d2d/d41/l45 0 2024-04-23T02:13:55.076 INFO:tasks.workunit.client.1.smithi192.stdout:1/153: creat dc/d14/f27 x:0 0 0 2024-04-23T02:13:55.077 INFO:tasks.workunit.client.1.smithi192.stdout:6/439: link d3/d15/l4d d3/d6c/l90 0 2024-04-23T02:13:55.078 INFO:tasks.workunit.client.1.smithi192.stdout:7/308: creat d5/d17/d1c/d4a/f58 x:0 0 0 2024-04-23T02:13:55.079 INFO:tasks.workunit.client.1.smithi192.stdout:4/454: mkdir de/d15/d2a/d74/d92 0 2024-04-23T02:13:55.081 INFO:tasks.workunit.client.1.smithi192.stdout:4/455: write de/d15/d2a/d61/f88 [464274,90686] 0 2024-04-23T02:13:55.086 INFO:tasks.workunit.client.1.smithi192.stdout:3/278: truncate d1/d3/d17/d2b/f32 183917 0 2024-04-23T02:13:55.086 INFO:tasks.workunit.client.1.smithi192.stdout:0/399: dread d6/df/d13/d25/f3d [0,4194304] 0 2024-04-23T02:13:55.087 INFO:tasks.workunit.client.1.smithi192.stdout:5/349: mknod d2/db/d16/d48/c71 0 2024-04-23T02:13:55.094 INFO:tasks.workunit.client.1.smithi192.stdout:4/456: mkdir de/d15/d19/d3d/d49/d93 0 2024-04-23T02:13:55.098 INFO:tasks.workunit.client.1.smithi192.stdout:5/350: dwrite d2/d32/f35 [0,4194304] 0 2024-04-23T02:13:55.098 INFO:tasks.workunit.client.1.smithi192.stdout:6/440: getdents d3/d15/d1b/d66/d80 0 2024-04-23T02:13:55.102 INFO:tasks.workunit.client.1.smithi192.stdout:2/256: truncate d8/d25/f3a 779071 0 2024-04-23T02:13:55.102 INFO:tasks.workunit.client.1.smithi192.stdout:4/457: fsync de/d15/d19/d3d/d34/d4a/f8d 0 2024-04-23T02:13:55.102 INFO:tasks.workunit.client.1.smithi192.stdout:6/441: write d3/d11/f4a [4515486,19909] 0 2024-04-23T02:13:55.105 INFO:tasks.workunit.client.1.smithi192.stdout:2/257: write d8/d26/d29/f4b [1006587,51482] 0 2024-04-23T02:13:55.107 INFO:tasks.workunit.client.1.smithi192.stdout:3/279: link d1/d3/d6/c59 d1/d3/d17/c5c 0 2024-04-23T02:13:55.107 INFO:tasks.workunit.client.1.smithi192.stdout:6/442: mkdir d3/d18/d91 0 2024-04-23T02:13:55.116 INFO:tasks.workunit.client.1.smithi192.stdout:2/258: truncate f7 1828146 0 2024-04-23T02:13:55.116 INFO:tasks.workunit.client.1.smithi192.stdout:0/400: dwrite d6/df/d13/d25/d3a/f58 [0,4194304] 0 2024-04-23T02:13:55.117 INFO:tasks.workunit.client.0.smithi071.stdout:8/283: creat dc/d10/d30/f52 x:0 0 0 2024-04-23T02:13:55.118 INFO:tasks.workunit.client.1.smithi192.stdout:5/351: rmdir d2/d6/d2d/d41/d43/d4b 0 2024-04-23T02:13:55.120 INFO:tasks.workunit.client.1.smithi192.stdout:0/401: write d6/df/d13/d6b/f6e [735169,8435] 0 2024-04-23T02:13:55.120 INFO:tasks.workunit.client.1.smithi192.stdout:3/280: mkdir d1/d3/d17/d1f/d33/d24/d34/d40/d5d 0 2024-04-23T02:13:55.120 INFO:tasks.workunit.client.1.smithi192.stdout:1/154: read dc/fd [4299221,47905] 0 2024-04-23T02:13:55.121 INFO:tasks.workunit.client.0.smithi071.stdout:5/320: mknod d3/df/c7b 0 2024-04-23T02:13:55.123 INFO:tasks.workunit.client.0.smithi071.stdout:4/262: fsync d2/d15/f1e 0 2024-04-23T02:13:55.124 INFO:tasks.workunit.client.0.smithi071.stdout:8/284: symlink dc/d15/d50/l53 0 2024-04-23T02:13:55.124 INFO:tasks.workunit.client.0.smithi071.stdout:4/263: dread - d2/d15/f22 zero size 2024-04-23T02:13:55.126 INFO:tasks.workunit.client.0.smithi071.stdout:8/285: write f8 [2967273,130998] 0 2024-04-23T02:13:55.127 INFO:tasks.workunit.client.0.smithi071.stdout:5/321: symlink d3/da/d1a/d1c/d46/d4e/l7c 0 2024-04-23T02:13:55.128 INFO:tasks.workunit.client.0.smithi071.stdout:5/322: read - d3/df/d10/f48 zero size 2024-04-23T02:13:55.130 INFO:tasks.workunit.client.1.smithi192.stdout:3/281: unlink d1/l43 0 2024-04-23T02:13:55.132 INFO:tasks.workunit.client.0.smithi071.stdout:8/286: creat dc/f54 x:0 0 0 2024-04-23T02:13:55.133 INFO:tasks.workunit.client.1.smithi192.stdout:4/458: dwrite de/d15/d2a/f2c [0,4194304] 0 2024-04-23T02:13:55.133 INFO:tasks.workunit.client.1.smithi192.stdout:2/259: dwrite d8/d14/f1a [0,4194304] 0 2024-04-23T02:13:55.133 INFO:tasks.workunit.client.1.smithi192.stdout:0/402: rename d6/d44 to d6/df/d56/d85 0 2024-04-23T02:13:55.134 INFO:tasks.workunit.client.0.smithi071.stdout:8/287: write dc/d15/f23 [2963263,69128] 0 2024-04-23T02:13:55.135 INFO:tasks.workunit.client.0.smithi071.stdout:8/288: fsync dc/d10/f27 0 2024-04-23T02:13:55.137 INFO:tasks.workunit.client.1.smithi192.stdout:7/309: dread d5/d17/d1c/d57/d31/f32 [0,4194304] 0 2024-04-23T02:13:55.137 INFO:tasks.workunit.client.0.smithi071.stdout:8/289: mkdir dc/d10/d55 0 2024-04-23T02:13:55.137 INFO:tasks.workunit.client.1.smithi192.stdout:6/443: truncate d3/d28/f56 220091 0 2024-04-23T02:13:55.138 INFO:tasks.workunit.client.0.smithi071.stdout:8/290: chown dc/d15/d2d/f3f 8 1 2024-04-23T02:13:55.139 INFO:tasks.workunit.client.1.smithi192.stdout:5/352: dread d2/db/f13 [0,4194304] 0 2024-04-23T02:13:55.144 INFO:tasks.workunit.client.1.smithi192.stdout:1/155: dread f4 [0,4194304] 0 2024-04-23T02:13:55.147 INFO:tasks.workunit.client.1.smithi192.stdout:5/353: write d2/db/d16/d1b/d3d/f65 [74030,69964] 0 2024-04-23T02:13:55.147 INFO:tasks.workunit.client.1.smithi192.stdout:7/310: mkdir d5/d17/d1c/d59 0 2024-04-23T02:13:55.151 INFO:tasks.workunit.client.0.smithi071.stdout:5/323: truncate f2 2964539 0 2024-04-23T02:13:55.152 INFO:tasks.workunit.client.0.smithi071.stdout:4/264: truncate d2/d15/d1d/d23/ff 1089781 0 2024-04-23T02:13:55.153 INFO:tasks.workunit.client.1.smithi192.stdout:1/156: mknod dc/c28 0 2024-04-23T02:13:55.153 INFO:tasks.workunit.client.0.smithi071.stdout:5/324: symlink d3/da/d1a/d1c/d4a/l7d 0 2024-04-23T02:13:55.154 INFO:tasks.workunit.client.1.smithi192.stdout:0/403: mkdir d6/df/d13/d25/d86 0 2024-04-23T02:13:55.154 INFO:tasks.workunit.client.0.smithi071.stdout:4/265: write d2/d15/d1d/d23/d4f/f50 [1291362,71253] 0 2024-04-23T02:13:55.157 INFO:tasks.workunit.client.1.smithi192.stdout:3/282: creat d1/d3/d17/d1f/d33/d24/d34/d40/d46/f5e x:0 0 0 2024-04-23T02:13:55.158 INFO:tasks.workunit.client.0.smithi071.stdout:4/266: chown d2/d15/d1d/d23/d17/f24 31 1 2024-04-23T02:13:55.159 INFO:tasks.workunit.client.1.smithi192.stdout:2/260: dwrite d8/d26/d27/d31/f2a [0,4194304] 0 2024-04-23T02:13:55.159 INFO:tasks.workunit.client.1.smithi192.stdout:7/311: write d5/d17/d1c/d26/f28 [1270554,74362] 0 2024-04-23T02:13:55.160 INFO:tasks.workunit.client.0.smithi071.stdout:4/267: write d2/d15/d1d/d23/d17/f24 [993350,5477] 0 2024-04-23T02:13:55.161 INFO:tasks.workunit.client.0.smithi071.stdout:5/325: dwrite d3/f5c [0,4194304] 0 2024-04-23T02:13:55.164 INFO:tasks.workunit.client.0.smithi071.stdout:8/291: dread dc/ff [0,4194304] 0 2024-04-23T02:13:55.164 INFO:tasks.workunit.client.1.smithi192.stdout:4/459: dwrite de/d15/d19/d3d/f5f [0,4194304] 0 2024-04-23T02:13:55.164 INFO:tasks.workunit.client.1.smithi192.stdout:0/404: rmdir d6/df/d13/d25/d3a/d5b 39 2024-04-23T02:13:55.168 INFO:tasks.workunit.client.0.smithi071.stdout:5/326: creat d3/d1d/d43/d4b/f7e x:0 0 0 2024-04-23T02:13:55.170 INFO:tasks.workunit.client.0.smithi071.stdout:5/327: truncate d3/d1d/d43/f75 885371 0 2024-04-23T02:13:55.171 INFO:tasks.workunit.client.0.smithi071.stdout:5/328: fdatasync d3/d1d/d43/f75 0 2024-04-23T02:13:55.172 INFO:tasks.workunit.client.0.smithi071.stdout:8/292: rename dc/d4c to dc/d56 0 2024-04-23T02:13:55.173 INFO:tasks.workunit.client.1.smithi192.stdout:4/460: stat de/d15/d19/d3d/d49/d26 0 2024-04-23T02:13:55.174 INFO:tasks.workunit.client.1.smithi192.stdout:0/405: readlink d6/df/d13/d25/d26/d67/l6a 0 2024-04-23T02:13:55.174 INFO:tasks.workunit.client.1.smithi192.stdout:5/354: rename d2/db/l5e to d2/d2e/l72 0 2024-04-23T02:13:55.177 INFO:tasks.workunit.client.0.smithi071.stdout:8/293: creat dc/f57 x:0 0 0 2024-04-23T02:13:55.177 INFO:tasks.workunit.client.0.smithi071.stdout:5/329: link d3/d34/l6f d3/d34/d74/l7f 0 2024-04-23T02:13:55.178 INFO:tasks.workunit.client.1.smithi192.stdout:0/406: readlink d6/df/d56/l7f 0 2024-04-23T02:13:55.178 INFO:tasks.workunit.client.0.smithi071.stdout:8/294: chown dc/d10/c26 0 1 2024-04-23T02:13:55.179 INFO:tasks.workunit.client.0.smithi071.stdout:8/295: fdatasync dc/f28 0 2024-04-23T02:13:55.180 INFO:tasks.workunit.client.1.smithi192.stdout:6/444: dwrite d3/f22 [0,4194304] 0 2024-04-23T02:13:55.181 INFO:tasks.workunit.client.0.smithi071.stdout:5/330: creat d3/da/d1a/f80 x:0 0 0 2024-04-23T02:13:55.182 INFO:tasks.workunit.client.1.smithi192.stdout:1/157: dread - dc/f11 zero size 2024-04-23T02:13:55.182 INFO:tasks.workunit.client.0.smithi071.stdout:8/296: creat dc/d10/f58 x:0 0 0 2024-04-23T02:13:55.184 INFO:tasks.workunit.client.0.smithi071.stdout:8/297: truncate dc/d15/d2d/f38 662914 0 2024-04-23T02:13:55.184 INFO:tasks.workunit.client.0.smithi071.stdout:8/298: dread - dc/f57 zero size 2024-04-23T02:13:55.187 INFO:tasks.workunit.client.1.smithi192.stdout:7/312: dread d5/d17/d1c/d57/f48 [0,4194304] 0 2024-04-23T02:13:55.187 INFO:tasks.workunit.client.1.smithi192.stdout:6/445: dread - d3/d15/d1b/d21/d3a/f4e zero size 2024-04-23T02:13:55.190 INFO:tasks.workunit.client.1.smithi192.stdout:4/461: dread de/d15/d2a/f2c [0,4194304] 0 2024-04-23T02:13:55.191 INFO:tasks.workunit.client.0.smithi071.stdout:8/299: dwrite dc/d3a/f3c [0,4194304] 0 2024-04-23T02:13:55.191 INFO:tasks.workunit.client.1.smithi192.stdout:0/407: creat d6/df/d13/d25/d3a/f87 x:0 0 0 2024-04-23T02:13:55.192 INFO:tasks.workunit.client.1.smithi192.stdout:6/446: dread - d3/f5c zero size 2024-04-23T02:13:55.194 INFO:tasks.workunit.client.0.smithi071.stdout:8/300: mkdir dc/d10/d30/d59 0 2024-04-23T02:13:55.195 INFO:tasks.workunit.client.1.smithi192.stdout:5/355: dwrite d2/d32/f44 [0,4194304] 0 2024-04-23T02:13:55.195 INFO:tasks.workunit.client.1.smithi192.stdout:7/313: rename d5/d17/d1c/d57/d31 to d5/d17/d1c/d59/d5a 0 2024-04-23T02:13:55.195 INFO:tasks.workunit.client.1.smithi192.stdout:0/408: write d6/df/d47/f4c [981297,27319] 0 2024-04-23T02:13:55.197 INFO:tasks.workunit.client.0.smithi071.stdout:8/301: truncate dc/d10/f1c 2445125 0 2024-04-23T02:13:55.200 INFO:tasks.workunit.client.0.smithi071.stdout:5/331: rename d3/da/d1a/d1c/d4a to d3/d34/d6b/d81 0 2024-04-23T02:13:55.205 INFO:tasks.workunit.client.1.smithi192.stdout:7/314: fsync d5/d17/d1c/f3e 0 2024-04-23T02:13:55.207 INFO:tasks.workunit.client.1.smithi192.stdout:0/409: dread d6/f72 [0,4194304] 0 2024-04-23T02:13:55.207 INFO:tasks.workunit.client.1.smithi192.stdout:5/356: fdatasync d2/d6/d2d/d41/f38 0 2024-04-23T02:13:55.209 INFO:tasks.workunit.client.1.smithi192.stdout:4/462: truncate de/d15/d19/d3d/d34/d4a/f56 827548 0 2024-04-23T02:13:55.209 INFO:tasks.workunit.client.0.smithi071.stdout:5/332: dwrite d3/d34/d6b/d81/f78 [4194304,4194304] 0 2024-04-23T02:13:55.210 INFO:tasks.workunit.client.1.smithi192.stdout:6/447: creat d3/d15/d1b/d21/d52/f92 x:0 0 0 2024-04-23T02:13:55.212 INFO:tasks.workunit.client.0.smithi071.stdout:5/333: chown d3/d34/d74/l7f 87466452 1 2024-04-23T02:13:55.214 INFO:tasks.workunit.client.1.smithi192.stdout:7/315: write d5/d17/d1c/d59/d5a/d41/d50/f35 [4031642,81633] 0 2024-04-23T02:13:55.216 INFO:tasks.workunit.client.0.smithi071.stdout:5/334: creat d3/d34/d6b/d81/d5a/f82 x:0 0 0 2024-04-23T02:13:55.216 INFO:tasks.workunit.client.1.smithi192.stdout:2/261: dwrite d8/d25/f3a [0,4194304] 0 2024-04-23T02:13:55.217 INFO:tasks.workunit.client.1.smithi192.stdout:7/316: readlink d5/d17/d1c/d57/l36 0 2024-04-23T02:13:55.221 INFO:tasks.workunit.client.1.smithi192.stdout:2/262: stat d8/d26/d27/l39 0 2024-04-23T02:13:55.221 INFO:tasks.workunit.client.0.smithi071.stdout:5/335: mkdir d3/df/d83 0 2024-04-23T02:13:55.222 INFO:tasks.workunit.client.0.smithi071.stdout:5/336: chown d3/f1f 0 1 2024-04-23T02:13:55.223 INFO:tasks.workunit.client.1.smithi192.stdout:7/317: unlink d5/d17/c1e 0 2024-04-23T02:13:55.225 INFO:tasks.workunit.client.1.smithi192.stdout:2/263: rename d8/d26/d29/f4b to d8/d26/d29/d43/f52 0 2024-04-23T02:13:55.225 INFO:tasks.workunit.client.0.smithi071.stdout:5/337: creat d3/da/d1a/d1c/d46/d4e/f84 x:0 0 0 2024-04-23T02:13:55.226 INFO:tasks.workunit.client.1.smithi192.stdout:2/264: write d8/d14/f4c [1051632,109466] 0 2024-04-23T02:13:55.226 INFO:tasks.workunit.client.0.smithi071.stdout:4/268: dread d2/d15/d1d/d23/ff [0,4194304] 0 2024-04-23T02:13:55.229 INFO:tasks.workunit.client.0.smithi071.stdout:5/338: rename d3/da/d1a/d1c to d3/da/d1a/d1c/d46/d4e/d85 22 2024-04-23T02:13:55.229 INFO:tasks.workunit.client.0.smithi071.stdout:8/302: dwrite dc/d10/f1c [0,4194304] 0 2024-04-23T02:13:55.230 INFO:tasks.workunit.client.1.smithi192.stdout:2/265: write d8/d14/f15 [5072975,60571] 0 2024-04-23T02:13:55.231 INFO:tasks.workunit.client.0.smithi071.stdout:8/303: rmdir dc/d3a 39 2024-04-23T02:13:55.233 INFO:tasks.workunit.client.0.smithi071.stdout:5/339: mkdir d3/d86 0 2024-04-23T02:13:55.234 INFO:tasks.workunit.client.0.smithi071.stdout:4/269: dread d2/d15/d1d/d2c/f2e [0,4194304] 0 2024-04-23T02:13:55.234 INFO:tasks.workunit.client.0.smithi071.stdout:5/340: chown d3/da/d1a 1 1 2024-04-23T02:13:55.234 INFO:tasks.workunit.client.1.smithi192.stdout:7/318: dwrite d5/d17/d1c/d57/f48 [0,4194304] 0 2024-04-23T02:13:55.236 INFO:tasks.workunit.client.0.smithi071.stdout:5/341: mkdir d3/da/d1a/d1c/d46/d87 0 2024-04-23T02:13:55.237 INFO:tasks.workunit.client.1.smithi192.stdout:7/319: rmdir d5/d17/d1c/d59/d5a/d41 39 2024-04-23T02:13:55.239 INFO:tasks.workunit.client.0.smithi071.stdout:8/304: truncate f9 3772075 0 2024-04-23T02:13:55.239 INFO:tasks.workunit.client.0.smithi071.stdout:5/342: write d3/da/d1a/f4c [693368,45122] 0 2024-04-23T02:13:55.240 INFO:tasks.workunit.client.1.smithi192.stdout:7/320: getdents d5/d43/d4c 0 2024-04-23T02:13:55.240 INFO:tasks.workunit.client.0.smithi071.stdout:8/305: read dc/d10/f1c [2205304,101616] 0 2024-04-23T02:13:55.242 INFO:tasks.workunit.client.1.smithi192.stdout:7/321: chown d5/d17/c37 9882 1 2024-04-23T02:13:55.242 INFO:tasks.workunit.client.0.smithi071.stdout:4/270: dwrite d2/d15/d1d/d23/f16 [0,4194304] 0 2024-04-23T02:13:55.242 INFO:tasks.workunit.client.0.smithi071.stdout:5/343: mknod d3/df/d10/c88 0 2024-04-23T02:13:55.244 INFO:tasks.workunit.client.1.smithi192.stdout:7/322: symlink d5/d43/d4c/d4e/l5b 0 2024-04-23T02:13:55.246 INFO:tasks.workunit.client.1.smithi192.stdout:7/323: mkdir d5/d43/d4c/d5c 0 2024-04-23T02:13:55.247 INFO:tasks.workunit.client.0.smithi071.stdout:4/271: fdatasync d2/d15/d1d/d23/f38 0 2024-04-23T02:13:55.248 INFO:tasks.workunit.client.0.smithi071.stdout:5/344: creat d3/df/f89 x:0 0 0 2024-04-23T02:13:55.249 INFO:tasks.workunit.client.1.smithi192.stdout:7/324: mknod d5/d43/d4c/d4e/c5d 0 2024-04-23T02:13:55.250 INFO:tasks.workunit.client.0.smithi071.stdout:4/272: mkdir d2/d15/d1d/d57 0 2024-04-23T02:13:55.250 INFO:tasks.workunit.client.0.smithi071.stdout:8/306: dwrite dc/f57 [0,4194304] 0 2024-04-23T02:13:55.256 INFO:tasks.workunit.client.0.smithi071.stdout:8/307: rename dc/d15/c51 to dc/d15/d2d/c5a 0 2024-04-23T02:13:55.257 INFO:tasks.workunit.client.0.smithi071.stdout:4/273: mkdir d2/d15/d1d/d23/d58 0 2024-04-23T02:13:55.258 INFO:tasks.workunit.client.0.smithi071.stdout:8/308: mknod dc/d56/c5b 0 2024-04-23T02:13:55.261 INFO:tasks.workunit.client.0.smithi071.stdout:5/345: write d3/d1d/d2c/f2d [1537930,117649] 0 2024-04-23T02:13:55.265 INFO:tasks.workunit.client.0.smithi071.stdout:5/346: write d3/da/d1a/f26 [5061264,113249] 0 2024-04-23T02:13:55.266 INFO:tasks.workunit.client.0.smithi071.stdout:2/195: sync 2024-04-23T02:13:55.266 INFO:tasks.workunit.client.0.smithi071.stdout:0/201: sync 2024-04-23T02:13:55.266 INFO:tasks.workunit.client.0.smithi071.stdout:3/169: sync 2024-04-23T02:13:55.267 INFO:tasks.workunit.client.0.smithi071.stdout:4/274: dwrite d2/d15/d1d/d23/f38 [0,4194304] 0 2024-04-23T02:13:55.269 INFO:tasks.workunit.client.0.smithi071.stdout:8/309: dwrite dc/d10/f14 [4194304,4194304] 0 2024-04-23T02:13:55.275 INFO:tasks.workunit.client.0.smithi071.stdout:4/275: write d2/d15/d1d/d23/d17/f24 [3700406,87684] 0 2024-04-23T02:13:55.276 INFO:tasks.workunit.client.0.smithi071.stdout:5/347: rename d3/da/d1a/f49 to d3/d1d/d2c/f8a 0 2024-04-23T02:13:55.280 INFO:tasks.workunit.client.0.smithi071.stdout:8/310: symlink dc/d15/d2d/l5c 0 2024-04-23T02:13:55.283 INFO:tasks.workunit.client.0.smithi071.stdout:0/202: dwrite d0/d3/d1c/f22 [0,4194304] 0 2024-04-23T02:13:55.284 INFO:tasks.workunit.client.0.smithi071.stdout:2/196: rename d0/d2/d26/d14/d1f/d2f to d0/d2/d26/d3a 0 2024-04-23T02:13:55.285 INFO:tasks.workunit.client.0.smithi071.stdout:4/276: chown d2/d15/d1d/d23/d29/c2b 101 1 2024-04-23T02:13:55.286 INFO:tasks.workunit.client.0.smithi071.stdout:5/348: write d3/d1d/d43/d4b/d54/f66 [3230204,67729] 0 2024-04-23T02:13:55.287 INFO:tasks.workunit.client.0.smithi071.stdout:3/170: dwrite d7/fd [0,4194304] 0 2024-04-23T02:13:55.288 INFO:tasks.workunit.client.0.smithi071.stdout:3/171: chown d7/d22/d23 6 1 2024-04-23T02:13:55.289 INFO:tasks.workunit.client.0.smithi071.stdout:5/349: dread - d3/da/d1a/d1c/f41 zero size 2024-04-23T02:13:55.292 INFO:tasks.workunit.client.0.smithi071.stdout:3/172: mknod d7/d28/d3b/c3c 0 2024-04-23T02:13:55.294 INFO:tasks.workunit.client.0.smithi071.stdout:0/203: truncate d0/d3/de/f28 1455648 0 2024-04-23T02:13:55.297 INFO:tasks.workunit.client.0.smithi071.stdout:4/277: dwrite d2/d2a/d2d/d39/f46 [0,4194304] 0 2024-04-23T02:13:55.302 INFO:tasks.workunit.client.0.smithi071.stdout:4/278: fdatasync d2/d15/d1d/d23/ff 0 2024-04-23T02:13:55.306 INFO:tasks.workunit.client.0.smithi071.stdout:8/311: truncate dc/f20 225556 0 2024-04-23T02:13:55.307 INFO:tasks.workunit.client.1.smithi192.stdout:4/463: write de/f21 [3123477,41746] 0 2024-04-23T02:13:55.308 INFO:tasks.workunit.client.0.smithi071.stdout:0/204: dread d0/d3/d1c/f22 [0,4194304] 0 2024-04-23T02:13:55.309 INFO:tasks.workunit.client.0.smithi071.stdout:3/173: write f0 [5196552,11825] 0 2024-04-23T02:13:55.310 INFO:tasks.workunit.client.0.smithi071.stdout:8/312: mkdir dc/d15/d5d 0 2024-04-23T02:13:55.311 INFO:tasks.workunit.client.0.smithi071.stdout:5/350: dwrite d3/d1d/d2c/f8a [0,4194304] 0 2024-04-23T02:13:55.312 INFO:tasks.workunit.client.0.smithi071.stdout:4/279: truncate d2/d2a/d2d/d39/d42/f55 766412 0 2024-04-23T02:13:55.313 INFO:tasks.workunit.client.1.smithi192.stdout:0/410: dwrite d6/df/d47/f61 [0,4194304] 0 2024-04-23T02:13:55.314 INFO:tasks.workunit.client.0.smithi071.stdout:0/205: creat d0/d3/de/d10/d14/d32/f41 x:0 0 0 2024-04-23T02:13:55.316 INFO:tasks.workunit.client.1.smithi192.stdout:0/411: dread - d6/df/d13/d6b/d73/f7e zero size 2024-04-23T02:13:55.316 INFO:tasks.workunit.client.1.smithi192.stdout:4/464: mknod de/d15/d53/c94 0 2024-04-23T02:13:55.317 INFO:tasks.workunit.client.0.smithi071.stdout:8/313: truncate dc/f28 4648295 0 2024-04-23T02:13:55.320 INFO:tasks.workunit.client.0.smithi071.stdout:4/280: creat d2/d15/d1d/d23/f59 x:0 0 0 2024-04-23T02:13:55.321 INFO:tasks.workunit.client.1.smithi192.stdout:0/412: rmdir d6/df/d35 39 2024-04-23T02:13:55.321 INFO:tasks.workunit.client.1.smithi192.stdout:4/465: mkdir de/d15/d19/d1a/d95 0 2024-04-23T02:13:55.321 INFO:tasks.workunit.client.0.smithi071.stdout:5/351: dwrite d3/da/f38 [0,4194304] 0 2024-04-23T02:13:55.322 INFO:tasks.workunit.client.1.smithi192.stdout:0/413: fsync d6/df/d13/d25/d3a/f62 0 2024-04-23T02:13:55.324 INFO:tasks.workunit.client.0.smithi071.stdout:8/314: rename dc/d10/d32/c35 to dc/d10/d30/d59/c5e 0 2024-04-23T02:13:55.325 INFO:tasks.workunit.client.1.smithi192.stdout:5/357: truncate d2/d6/f68 295784 0 2024-04-23T02:13:55.325 INFO:tasks.workunit.client.1.smithi192.stdout:7/325: rmdir d5/d17 39 2024-04-23T02:13:55.325 INFO:tasks.workunit.client.0.smithi071.stdout:5/352: creat d3/d34/f8b x:0 0 0 2024-04-23T02:13:55.326 INFO:tasks.workunit.client.1.smithi192.stdout:0/414: truncate d6/df/d13/d6b/f7d 301681 0 2024-04-23T02:13:55.329 INFO:tasks.workunit.client.0.smithi071.stdout:5/353: write d3/d34/f6a [1967510,44817] 0 2024-04-23T02:13:55.331 INFO:tasks.workunit.client.0.smithi071.stdout:5/354: write d3/df/d10/f48 [698602,116664] 0 2024-04-23T02:13:55.331 INFO:tasks.workunit.client.0.smithi071.stdout:4/281: rename d2/d15/d1d/d23/f38 to d2/f5a 0 2024-04-23T02:13:55.331 INFO:tasks.workunit.client.1.smithi192.stdout:0/415: stat d6/df/d13/d25/f2b 0 2024-04-23T02:13:55.332 INFO:tasks.workunit.client.0.smithi071.stdout:5/355: chown d3/d1d/d43/d4b/d54/d5f 0 1 2024-04-23T02:13:55.333 INFO:tasks.workunit.client.0.smithi071.stdout:4/282: write d2/d2a/d2d/f54 [336552,31916] 0 2024-04-23T02:13:55.333 INFO:tasks.workunit.client.0.smithi071.stdout:2/197: read d0/d2/d26/f18 [3455645,122949] 0 2024-04-23T02:13:55.336 INFO:tasks.workunit.client.1.smithi192.stdout:2/266: write d8/d25/f38 [3053321,41641] 0 2024-04-23T02:13:55.338 INFO:tasks.workunit.client.0.smithi071.stdout:2/198: creat d0/d2/d26/df/f3b x:0 0 0 2024-04-23T02:13:55.339 INFO:tasks.workunit.client.0.smithi071.stdout:5/356: mknod d3/df/c8c 0 2024-04-23T02:13:55.339 INFO:tasks.workunit.client.1.smithi192.stdout:4/466: link l2 de/d15/d53/l96 0 2024-04-23T02:13:55.339 INFO:tasks.workunit.client.0.smithi071.stdout:4/283: dread d2/d2a/d2d/f54 [0,4194304] 0 2024-04-23T02:13:55.340 INFO:tasks.workunit.client.1.smithi192.stdout:5/358: write d2/f5 [388639,223] 0 2024-04-23T02:13:55.341 INFO:tasks.workunit.client.0.smithi071.stdout:2/199: write d0/d2/d26/d14/f1c [922146,103141] 0 2024-04-23T02:13:55.342 INFO:tasks.workunit.client.0.smithi071.stdout:2/200: chown d0/d2/d26/f18 4957 1 2024-04-23T02:13:55.342 INFO:tasks.workunit.client.1.smithi192.stdout:7/326: symlink d5/d17/d1c/d59/d5a/l5e 0 2024-04-23T02:13:55.342 INFO:tasks.workunit.client.1.smithi192.stdout:2/267: unlink d8/d14/c48 0 2024-04-23T02:13:55.343 INFO:tasks.workunit.client.0.smithi071.stdout:2/201: truncate d0/d2/f1d 289900 0 2024-04-23T02:13:55.345 INFO:tasks.workunit.client.0.smithi071.stdout:4/284: creat d2/d15/d1d/d23/f5b x:0 0 0 2024-04-23T02:13:55.346 INFO:tasks.workunit.client.1.smithi192.stdout:4/467: mkdir de/d15/d2a/d97 0 2024-04-23T02:13:55.346 INFO:tasks.workunit.client.0.smithi071.stdout:2/202: unlink d0/d2/c27 0 2024-04-23T02:13:55.348 INFO:tasks.workunit.client.0.smithi071.stdout:2/203: mkdir d0/d2/d3c 0 2024-04-23T02:13:55.348 INFO:tasks.workunit.client.1.smithi192.stdout:2/268: mkdir d8/d26/d27/d53 0 2024-04-23T02:13:55.349 INFO:tasks.workunit.client.0.smithi071.stdout:2/204: dread - d0/d2/d26/df/f3b zero size 2024-04-23T02:13:55.350 INFO:tasks.workunit.client.1.smithi192.stdout:7/327: dread d5/d17/d1c/f45 [0,4194304] 0 2024-04-23T02:13:55.350 INFO:tasks.workunit.client.0.smithi071.stdout:2/205: fsync d0/d2/f29 0 2024-04-23T02:13:55.351 INFO:tasks.workunit.client.0.smithi071.stdout:2/206: truncate d0/d2/d26/f32 1040862 0 2024-04-23T02:13:55.351 INFO:tasks.workunit.client.1.smithi192.stdout:4/468: rename de/d15/d2a/d74/c79 to de/d15/d53/d81/c98 0 2024-04-23T02:13:55.353 INFO:tasks.workunit.client.0.smithi071.stdout:2/207: creat d0/d2/d26/d3a/f3d x:0 0 0 2024-04-23T02:13:55.356 INFO:tasks.workunit.client.1.smithi192.stdout:2/269: rename d8/d26/d27/f2f to d8/d26/d27/d53/f54 0 2024-04-23T02:13:55.357 INFO:tasks.workunit.client.0.smithi071.stdout:2/208: unlink d0/d2/d26/df/d30/l31 0 2024-04-23T02:13:55.357 INFO:tasks.workunit.client.1.smithi192.stdout:4/469: chown de/d15/d2a/d74/f7e 829073142 1 2024-04-23T02:13:55.357 INFO:tasks.workunit.client.1.smithi192.stdout:2/270: fdatasync d8/d25/f32 0 2024-04-23T02:13:55.358 INFO:tasks.workunit.client.1.smithi192.stdout:2/271: fdatasync d8/d26/d29/f24 0 2024-04-23T02:13:55.359 INFO:tasks.workunit.client.0.smithi071.stdout:2/209: getdents d0/d2/d26/d14/d1f 0 2024-04-23T02:13:55.360 INFO:tasks.workunit.client.1.smithi192.stdout:4/470: mknod de/d15/d2a/d97/c99 0 2024-04-23T02:13:55.361 INFO:tasks.workunit.client.0.smithi071.stdout:2/210: write d0/d2/f1d [1043445,56516] 0 2024-04-23T02:13:55.362 INFO:tasks.workunit.client.1.smithi192.stdout:2/272: getdents d8/d26/d29/d43/d50 0 2024-04-23T02:13:55.362 INFO:tasks.workunit.client.0.smithi071.stdout:2/211: truncate d0/d2/d26/d14/f1c 1253009 0 2024-04-23T02:13:55.363 INFO:tasks.workunit.client.1.smithi192.stdout:2/273: write d8/d26/f3c [625713,17975] 0 2024-04-23T02:13:55.365 INFO:tasks.workunit.client.1.smithi192.stdout:3/283: sync 2024-04-23T02:13:55.365 INFO:tasks.workunit.client.1.smithi192.stdout:8/379: sync 2024-04-23T02:13:55.365 INFO:tasks.workunit.client.1.smithi192.stdout:1/158: sync 2024-04-23T02:13:55.365 INFO:tasks.workunit.client.1.smithi192.stdout:9/321: sync 2024-04-23T02:13:55.365 INFO:tasks.workunit.client.1.smithi192.stdout:2/274: chown d8/d26/d29/d43 258512 1 2024-04-23T02:13:55.366 INFO:tasks.workunit.client.0.smithi071.stdout:0/206: dread d0/d3/de/d10/f1f [0,4194304] 0 2024-04-23T02:13:55.366 INFO:tasks.workunit.client.1.smithi192.stdout:4/471: creat de/d15/d19/f9a x:0 0 0 2024-04-23T02:13:55.370 INFO:tasks.workunit.client.0.smithi071.stdout:0/207: write d0/d3/de/d10/d14/d32/f35 [745353,68171] 0 2024-04-23T02:13:55.371 INFO:tasks.workunit.client.1.smithi192.stdout:4/472: dread - de/f4d zero size 2024-04-23T02:13:55.371 INFO:tasks.workunit.client.0.smithi071.stdout:8/315: rename dc/d10/d30/d59 to dc/d15/d50/d5f 0 2024-04-23T02:13:55.374 INFO:tasks.workunit.client.0.smithi071.stdout:8/316: dread dc/d15/f39 [0,4194304] 0 2024-04-23T02:13:55.375 INFO:tasks.workunit.client.0.smithi071.stdout:8/317: read - dc/f54 zero size 2024-04-23T02:13:55.375 INFO:tasks.workunit.client.0.smithi071.stdout:3/174: dread d7/d8/f1a [0,4194304] 0 2024-04-23T02:13:55.376 INFO:tasks.workunit.client.0.smithi071.stdout:2/212: rename d0/d2/c2c to d0/d2/d34/c3e 0 2024-04-23T02:13:55.376 INFO:tasks.workunit.client.0.smithi071.stdout:8/318: read - dc/f54 zero size 2024-04-23T02:13:55.377 INFO:tasks.workunit.client.1.smithi192.stdout:8/380: dread d1/d9/fa [0,4194304] 0 2024-04-23T02:13:55.377 INFO:tasks.workunit.client.0.smithi071.stdout:0/208: mknod d0/d3/de/d10/d14/d32/c42 0 2024-04-23T02:13:55.379 INFO:tasks.workunit.client.1.smithi192.stdout:8/381: chown d1/d12/f39 6 1 2024-04-23T02:13:55.379 INFO:tasks.workunit.client.1.smithi192.stdout:4/473: mknod de/d15/d19/d3d/d49/d70/c9b 0 2024-04-23T02:13:55.380 INFO:tasks.workunit.client.0.smithi071.stdout:8/319: truncate dc/d10/f1c 5079624 0 2024-04-23T02:13:55.380 INFO:tasks.workunit.client.1.smithi192.stdout:2/275: dread d8/d26/d27/d31/f2a [0,4194304] 0 2024-04-23T02:13:55.380 INFO:tasks.workunit.client.0.smithi071.stdout:5/357: truncate d3/d1d/f3b 238951 0 2024-04-23T02:13:55.381 INFO:tasks.workunit.client.1.smithi192.stdout:0/416: truncate d6/df/d47/f4c 544805 0 2024-04-23T02:13:55.382 INFO:tasks.workunit.client.1.smithi192.stdout:4/474: chown de/d15/d19/d1a/c6c 1 1 2024-04-23T02:13:55.382 INFO:tasks.workunit.client.1.smithi192.stdout:8/382: dread - d1/d12/d23/d4b/d73/f43 zero size 2024-04-23T02:13:55.384 INFO:tasks.workunit.client.0.smithi071.stdout:5/358: stat d3/da/c27 0 2024-04-23T02:13:55.384 INFO:tasks.workunit.client.1.smithi192.stdout:1/159: creat dc/f29 x:0 0 0 2024-04-23T02:13:55.385 INFO:tasks.workunit.client.1.smithi192.stdout:5/359: dread d2/fd [0,4194304] 0 2024-04-23T02:13:55.385 INFO:tasks.workunit.client.0.smithi071.stdout:3/175: dwrite d7/d10/f1e [0,4194304] 0 2024-04-23T02:13:55.387 INFO:tasks.workunit.client.1.smithi192.stdout:7/328: dread d5/f2e [0,4194304] 0 2024-04-23T02:13:55.388 INFO:tasks.workunit.client.1.smithi192.stdout:2/276: unlink d8/d26/d29/f24 0 2024-04-23T02:13:55.388 INFO:tasks.workunit.client.1.smithi192.stdout:3/284: getdents d1/d3/d17/d1f/d33/d24/d34/d40 0 2024-04-23T02:13:55.390 INFO:tasks.workunit.client.1.smithi192.stdout:5/360: dread d2/db/d16/d1b/d3d/f65 [0,4194304] 0 2024-04-23T02:13:55.390 INFO:tasks.workunit.client.1.smithi192.stdout:8/383: creat d1/d12/d23/d4b/d73/d45/f80 x:0 0 0 2024-04-23T02:13:55.391 INFO:tasks.workunit.client.0.smithi071.stdout:8/320: creat dc/d10/d55/f60 x:0 0 0 2024-04-23T02:13:55.394 INFO:tasks.workunit.client.0.smithi071.stdout:3/176: symlink d7/d22/d23/l3d 0 2024-04-23T02:13:55.394 INFO:tasks.workunit.client.1.smithi192.stdout:4/475: fsync de/d15/d19/d3d/d49/f28 0 2024-04-23T02:13:55.394 INFO:tasks.workunit.client.1.smithi192.stdout:4/476: chown de/d15/d19/d3d/f8b 1219826 1 2024-04-23T02:13:55.395 INFO:tasks.workunit.client.1.smithi192.stdout:4/477: stat de/d15/c1f 0 2024-04-23T02:13:55.395 INFO:tasks.workunit.client.0.smithi071.stdout:8/321: creat dc/d56/f61 x:0 0 0 2024-04-23T02:13:55.397 INFO:tasks.workunit.client.1.smithi192.stdout:9/322: write d0/d9/f52 [507444,62986] 0 2024-04-23T02:13:55.398 INFO:tasks.workunit.client.1.smithi192.stdout:0/417: dread d6/df/f32 [0,4194304] 0 2024-04-23T02:13:55.398 INFO:tasks.workunit.client.0.smithi071.stdout:3/177: dread d7/d22/d23/d26/f2b [0,4194304] 0 2024-04-23T02:13:55.398 INFO:tasks.workunit.client.0.smithi071.stdout:5/359: dwrite d3/d1d/d43/f69 [0,4194304] 0 2024-04-23T02:13:55.399 INFO:tasks.workunit.client.1.smithi192.stdout:5/361: unlink d2/d2e/d4e/c63 0 2024-04-23T02:13:55.400 INFO:tasks.workunit.client.0.smithi071.stdout:5/360: truncate d3/da/f4d 588762 0 2024-04-23T02:13:55.401 INFO:tasks.workunit.client.1.smithi192.stdout:4/478: symlink de/d15/d53/l9c 0 2024-04-23T02:13:55.401 INFO:tasks.workunit.client.0.smithi071.stdout:2/213: dread d0/d2/f21 [0,4194304] 0 2024-04-23T02:13:55.402 INFO:tasks.workunit.client.0.smithi071.stdout:3/178: mkdir d7/d27/d3e 0 2024-04-23T02:13:55.402 INFO:tasks.workunit.client.1.smithi192.stdout:2/277: creat d8/d26/d29/d43/d50/f55 x:0 0 0 2024-04-23T02:13:55.403 INFO:tasks.workunit.client.0.smithi071.stdout:2/214: fsync d0/d2/d26/d3a/f3d 0 2024-04-23T02:13:55.404 INFO:tasks.workunit.client.0.smithi071.stdout:8/322: mknod dc/d3a/c62 0 2024-04-23T02:13:55.405 INFO:tasks.workunit.client.1.smithi192.stdout:9/323: creat d0/d9/d44/d51/d64/f78 x:0 0 0 2024-04-23T02:13:55.405 INFO:tasks.workunit.client.0.smithi071.stdout:3/179: dread d7/d10/f15 [0,4194304] 0 2024-04-23T02:13:55.405 INFO:tasks.workunit.client.0.smithi071.stdout:5/361: creat d3/d34/d6b/f8d x:0 0 0 2024-04-23T02:13:55.410 INFO:tasks.workunit.client.0.smithi071.stdout:5/362: write d3/d34/f40 [3370666,120514] 0 2024-04-23T02:13:55.410 INFO:tasks.workunit.client.0.smithi071.stdout:2/215: creat d0/f3f x:0 0 0 2024-04-23T02:13:55.410 INFO:tasks.workunit.client.0.smithi071.stdout:3/180: symlink d7/d10/d38/l3f 0 2024-04-23T02:13:55.411 INFO:tasks.workunit.client.0.smithi071.stdout:0/209: truncate d0/d3/de/d10/f1f 5391349 0 2024-04-23T02:13:55.415 INFO:tasks.workunit.client.1.smithi192.stdout:0/418: creat d6/df/d13/d25/d3a/d7b/f88 x:0 0 0 2024-04-23T02:13:55.415 INFO:tasks.workunit.client.0.smithi071.stdout:8/323: dwrite dc/d10/f27 [4194304,4194304] 0 2024-04-23T02:13:55.416 INFO:tasks.workunit.client.1.smithi192.stdout:9/324: chown d0/d9/d44/d4f/f1c 128478 1 2024-04-23T02:13:55.416 INFO:tasks.workunit.client.0.smithi071.stdout:2/216: link d0/f2e d0/d2/d26/df/f40 0 2024-04-23T02:13:55.419 INFO:tasks.workunit.client.0.smithi071.stdout:2/217: read d0/d2/d26/df/f40 [366148,46910] 0 2024-04-23T02:13:55.419 INFO:tasks.workunit.client.1.smithi192.stdout:4/479: symlink de/d15/d2a/d74/d92/l9d 0 2024-04-23T02:13:55.420 INFO:tasks.workunit.client.0.smithi071.stdout:3/181: creat d7/d27/d3e/f40 x:0 0 0 2024-04-23T02:13:55.421 INFO:tasks.workunit.client.1.smithi192.stdout:1/160: getdents dc/d14 0 2024-04-23T02:13:55.422 INFO:tasks.workunit.client.1.smithi192.stdout:2/278: creat d8/d14/d51/f56 x:0 0 0 2024-04-23T02:13:55.423 INFO:tasks.workunit.client.0.smithi071.stdout:8/324: rmdir dc/d3a 39 2024-04-23T02:13:55.424 INFO:tasks.workunit.client.1.smithi192.stdout:0/419: unlink d6/df/d13/d6b/f7d 0 2024-04-23T02:13:55.426 INFO:tasks.workunit.client.0.smithi071.stdout:2/218: mknod d0/d2/d26/d3a/c41 0 2024-04-23T02:13:55.428 INFO:tasks.workunit.client.0.smithi071.stdout:8/325: mkdir dc/d10/d32/d63 0 2024-04-23T02:13:55.428 INFO:tasks.workunit.client.1.smithi192.stdout:9/325: write d0/f54 [356684,69190] 0 2024-04-23T02:13:55.429 INFO:tasks.workunit.client.0.smithi071.stdout:3/182: dwrite d7/d27/d3e/f40 [0,4194304] 0 2024-04-23T02:13:55.430 INFO:tasks.workunit.client.1.smithi192.stdout:8/384: truncate d1/f1d 1126268 0 2024-04-23T02:13:55.430 INFO:tasks.workunit.client.0.smithi071.stdout:2/219: creat d0/d2/d26/d14/f42 x:0 0 0 2024-04-23T02:13:55.431 INFO:tasks.workunit.client.1.smithi192.stdout:9/326: fsync d0/d9/d44/f5e 0 2024-04-23T02:13:55.434 INFO:tasks.workunit.client.1.smithi192.stdout:7/329: dwrite d5/d17/d1c/d59/d5a/d41/f54 [0,4194304] 0 2024-04-23T02:13:55.435 INFO:tasks.workunit.client.0.smithi071.stdout:8/326: mkdir dc/d56/d64 0 2024-04-23T02:13:55.435 INFO:tasks.workunit.client.0.smithi071.stdout:3/183: rmdir d7 39 2024-04-23T02:13:55.437 INFO:tasks.workunit.client.1.smithi192.stdout:5/362: truncate d2/f5 413579 0 2024-04-23T02:13:55.439 INFO:tasks.workunit.client.0.smithi071.stdout:5/363: truncate d3/da/d1a/f2b 757405 0 2024-04-23T02:13:55.439 INFO:tasks.workunit.client.1.smithi192.stdout:7/330: read d5/f15 [371283,118584] 0 2024-04-23T02:13:55.439 INFO:tasks.workunit.client.0.smithi071.stdout:8/327: read dc/d10/f22 [2643407,31867] 0 2024-04-23T02:13:55.441 INFO:tasks.workunit.client.0.smithi071.stdout:2/220: dwrite d0/d2/d26/d3a/f3d [0,4194304] 0 2024-04-23T02:13:55.441 INFO:tasks.workunit.client.0.smithi071.stdout:3/184: creat d7/d10/d38/f41 x:0 0 0 2024-04-23T02:13:55.442 INFO:tasks.workunit.client.0.smithi071.stdout:0/210: write d0/d3/de/f28 [743320,46932] 0 2024-04-23T02:13:55.443 INFO:tasks.workunit.client.1.smithi192.stdout:9/327: dread d0/d9/d44/d51/d5b/f67 [0,4194304] 0 2024-04-23T02:13:55.444 INFO:tasks.workunit.client.1.smithi192.stdout:7/331: dread d5/f2e [0,4194304] 0 2024-04-23T02:13:55.444 INFO:tasks.workunit.client.0.smithi071.stdout:5/364: chown d3/d34/d6b/d81/c7a 131796894 1 2024-04-23T02:13:55.446 INFO:tasks.workunit.client.1.smithi192.stdout:0/420: rmdir d6 39 2024-04-23T02:13:55.447 INFO:tasks.workunit.client.1.smithi192.stdout:4/480: write de/f4d [893146,122870] 0 2024-04-23T02:13:55.448 INFO:tasks.workunit.client.1.smithi192.stdout:8/385: mkdir d1/d12/d23/d4b/d73/d7e/d7b/d7c/d81 0 2024-04-23T02:13:55.448 INFO:tasks.workunit.client.0.smithi071.stdout:5/365: chown d3/d1d/d2c/l68 14 1 2024-04-23T02:13:55.449 INFO:tasks.workunit.client.0.smithi071.stdout:5/366: chown d3/d1d/d2c/f8a 6 1 2024-04-23T02:13:55.452 INFO:tasks.workunit.client.0.smithi071.stdout:8/328: rename dc/d10/f1a to dc/d10/d55/f65 0 2024-04-23T02:13:55.456 INFO:tasks.workunit.client.0.smithi071.stdout:8/329: write dc/d10/f22 [3677782,111382] 0 2024-04-23T02:13:55.456 INFO:tasks.workunit.client.0.smithi071.stdout:5/367: mknod d3/d1d/d43/d4b/d54/d5f/c8e 0 2024-04-23T02:13:55.459 INFO:tasks.workunit.client.1.smithi192.stdout:2/279: dread f4 [0,4194304] 0 2024-04-23T02:13:55.460 INFO:tasks.workunit.client.0.smithi071.stdout:5/368: mkdir d3/d8f 0 2024-04-23T02:13:55.461 INFO:tasks.workunit.client.1.smithi192.stdout:4/481: symlink de/d15/d19/d3d/d49/d26/d59/l9e 0 2024-04-23T02:13:55.461 INFO:tasks.workunit.client.0.smithi071.stdout:8/330: unlink dc/d3a/c62 0 2024-04-23T02:13:55.461 INFO:tasks.workunit.client.1.smithi192.stdout:1/161: truncate fa 104893 0 2024-04-23T02:13:55.462 INFO:tasks.workunit.client.1.smithi192.stdout:8/386: mkdir d1/d12/d23/d4b/d73/d4c/d82 0 2024-04-23T02:13:55.463 INFO:tasks.workunit.client.1.smithi192.stdout:8/387: fdatasync d1/d9/d3e/f70 0 2024-04-23T02:13:55.464 INFO:tasks.workunit.client.0.smithi071.stdout:8/331: creat dc/d10/d30/f66 x:0 0 0 2024-04-23T02:13:55.466 INFO:tasks.workunit.client.1.smithi192.stdout:2/280: mkdir d8/d26/d27/d57 0 2024-04-23T02:13:55.467 INFO:tasks.workunit.client.0.smithi071.stdout:8/332: fsync dc/d3a/f47 0 2024-04-23T02:13:55.468 INFO:tasks.workunit.client.0.smithi071.stdout:5/369: dwrite d3/da/d1a/d1c/f62 [0,4194304] 0 2024-04-23T02:13:55.468 INFO:tasks.workunit.client.1.smithi192.stdout:1/162: truncate dc/f10 1579391 0 2024-04-23T02:13:55.469 INFO:tasks.workunit.client.1.smithi192.stdout:8/388: creat d1/d12/d68/f83 x:0 0 0 2024-04-23T02:13:55.470 INFO:tasks.workunit.client.1.smithi192.stdout:7/332: link d5/d17/d1c/d26/c27 d5/d17/d1c/d57/c5f 0 2024-04-23T02:13:55.470 INFO:tasks.workunit.client.0.smithi071.stdout:0/211: dread d0/d3/de/f1d [0,4194304] 0 2024-04-23T02:13:55.471 INFO:tasks.workunit.client.1.smithi192.stdout:4/482: mkdir de/d15/d19/d3d/d9f 0 2024-04-23T02:13:55.472 INFO:tasks.workunit.client.0.smithi071.stdout:0/212: read d0/d3/d1c/f22 [4402003,41828] 0 2024-04-23T02:13:55.473 INFO:tasks.workunit.client.0.smithi071.stdout:5/370: mknod d3/d1d/d43/c90 0 2024-04-23T02:13:55.473 INFO:tasks.workunit.client.1.smithi192.stdout:4/483: readlink de/l45 0 2024-04-23T02:13:55.473 INFO:tasks.workunit.client.1.smithi192.stdout:0/421: truncate d6/df/d47/f4c 973950 0 2024-04-23T02:13:55.476 INFO:tasks.workunit.client.0.smithi071.stdout:2/221: write d0/d2/f9 [213202,47377] 0 2024-04-23T02:13:55.477 INFO:tasks.workunit.client.0.smithi071.stdout:8/333: dwrite dc/d10/f27 [0,4194304] 0 2024-04-23T02:13:55.479 INFO:tasks.workunit.client.1.smithi192.stdout:8/389: creat d1/d12/d23/d4b/d73/d4c/f84 x:0 0 0 2024-04-23T02:13:55.482 INFO:tasks.workunit.client.1.smithi192.stdout:1/163: dwrite dc/f29 [0,4194304] 0 2024-04-23T02:13:55.486 INFO:tasks.workunit.client.1.smithi192.stdout:2/281: rename d8/d26/d29/d43/f52 to d8/d26/d29/f58 0 2024-04-23T02:13:55.487 INFO:tasks.workunit.client.1.smithi192.stdout:4/484: write de/d15/d19/d3d/d34/d4a/f8d [323253,29168] 0 2024-04-23T02:13:55.488 INFO:tasks.workunit.client.0.smithi071.stdout:2/222: rename d0/d2/f21 to d0/d2/d26/d14/f43 0 2024-04-23T02:13:55.488 INFO:tasks.workunit.client.1.smithi192.stdout:0/422: link d6/df/d13/d25/d26/f82 d6/df/d47/f89 0 2024-04-23T02:13:55.491 INFO:tasks.workunit.client.1.smithi192.stdout:8/390: mknod d1/d12/d23/d4b/d73/d7e/d7b/d7c/d81/c85 0 2024-04-23T02:13:55.492 INFO:tasks.workunit.client.0.smithi071.stdout:3/185: dwrite d7/d10/f1e [4194304,4194304] 0 2024-04-23T02:13:55.494 INFO:tasks.workunit.client.1.smithi192.stdout:9/328: dwrite d0/d9/d44/d4f/f28 [0,4194304] 0 2024-04-23T02:13:55.494 INFO:tasks.workunit.client.1.smithi192.stdout:7/333: dread d5/d17/d1c/d59/d5a/d41/d50/f35 [4194304,4194304] 0 2024-04-23T02:13:55.495 INFO:tasks.workunit.client.0.smithi071.stdout:5/371: dwrite d3/d1d/d2c/f8a [4194304,4194304] 0 2024-04-23T02:13:55.496 INFO:tasks.workunit.client.0.smithi071.stdout:2/223: creat d0/d2/d26/d3a/f44 x:0 0 0 2024-04-23T02:13:55.498 INFO:tasks.workunit.client.0.smithi071.stdout:0/213: dwrite d0/d3/de/f1d [4194304,4194304] 0 2024-04-23T02:13:55.498 INFO:tasks.workunit.client.1.smithi192.stdout:4/485: truncate de/d15/d2a/d74/f7e 90718 0 2024-04-23T02:13:55.499 INFO:tasks.workunit.client.0.smithi071.stdout:5/372: fsync d3/d1d/d2c/f2d 0 2024-04-23T02:13:55.499 INFO:tasks.workunit.client.0.smithi071.stdout:0/214: fsync d0/f2 0 2024-04-23T02:13:55.499 INFO:tasks.workunit.client.0.smithi071.stdout:8/334: dread dc/d10/d55/f65 [0,4194304] 0 2024-04-23T02:13:55.500 INFO:tasks.workunit.client.1.smithi192.stdout:7/334: truncate d5/d17/f4b 347492 0 2024-04-23T02:13:55.500 INFO:tasks.workunit.client.0.smithi071.stdout:2/224: mkdir d0/d2/d45 0 2024-04-23T02:13:55.502 INFO:tasks.workunit.client.1.smithi192.stdout:9/329: write d0/d9/d44/d51/d5b/d61/f6a [462586,11012] 0 2024-04-23T02:13:55.503 INFO:tasks.workunit.client.0.smithi071.stdout:8/335: stat dc/d15/d5d 0 2024-04-23T02:13:55.503 INFO:tasks.workunit.client.0.smithi071.stdout:6/229: sync 2024-04-23T02:13:55.503 INFO:tasks.workunit.client.0.smithi071.stdout:1/140: sync 2024-04-23T02:13:55.503 INFO:tasks.workunit.client.0.smithi071.stdout:7/190: sync 2024-04-23T02:13:55.505 INFO:tasks.workunit.client.1.smithi192.stdout:4/486: chown de/d15/d19/d3d/d49/d26/c7b 129340 1 2024-04-23T02:13:55.509 INFO:tasks.workunit.client.0.smithi071.stdout:5/373: dread - d3/da/d1a/d1c/d46/d4e/f84 zero size 2024-04-23T02:13:55.509 INFO:tasks.workunit.client.0.smithi071.stdout:1/141: symlink db/l23 0 2024-04-23T02:13:55.510 INFO:tasks.workunit.client.0.smithi071.stdout:6/230: symlink d6/d17/l43 0 2024-04-23T02:13:55.511 INFO:tasks.workunit.client.0.smithi071.stdout:5/374: read - d3/d1d/d43/d4b/f72 zero size 2024-04-23T02:13:55.512 INFO:tasks.workunit.client.1.smithi192.stdout:7/335: dwrite d5/d17/d1c/d57/f48 [0,4194304] 0 2024-04-23T02:13:55.512 INFO:tasks.workunit.client.0.smithi071.stdout:1/142: write db/fe [3525872,13170] 0 2024-04-23T02:13:55.515 INFO:tasks.workunit.client.1.smithi192.stdout:2/282: link d8/d26/d29/d43/l44 d8/d26/d27/d53/l59 0 2024-04-23T02:13:55.522 INFO:tasks.workunit.client.0.smithi071.stdout:3/186: dwrite d7/d22/f25 [4194304,4194304] 0 2024-04-23T02:13:55.522 INFO:tasks.workunit.client.1.smithi192.stdout:4/487: mkdir de/d15/d2a/d74/d92/da0 0 2024-04-23T02:13:55.525 INFO:tasks.workunit.client.1.smithi192.stdout:7/336: mknod d5/d43/c60 0 2024-04-23T02:13:55.525 INFO:tasks.workunit.client.1.smithi192.stdout:8/391: link d1/d12/d23/d2c/c30 d1/d9/c86 0 2024-04-23T02:13:55.526 INFO:tasks.workunit.client.1.smithi192.stdout:2/283: mknod d8/d26/c5a 0 2024-04-23T02:13:55.527 INFO:tasks.workunit.client.0.smithi071.stdout:1/143: creat db/df/d13/d20/f24 x:0 0 0 2024-04-23T02:13:55.527 INFO:tasks.workunit.client.0.smithi071.stdout:2/225: dwrite d0/d2/d26/d3a/f3d [0,4194304] 0 2024-04-23T02:13:55.529 INFO:tasks.workunit.client.1.smithi192.stdout:1/164: link c5 dc/d16/d19/c2a 0 2024-04-23T02:13:55.529 INFO:tasks.workunit.client.1.smithi192.stdout:7/337: dread d5/d17/f4b [0,4194304] 0 2024-04-23T02:13:55.531 INFO:tasks.workunit.client.0.smithi071.stdout:5/375: dread d3/d34/d6b/d81/f78 [0,4194304] 0 2024-04-23T02:13:55.531 INFO:tasks.workunit.client.1.smithi192.stdout:1/165: stat dc/d14/f27 0 2024-04-23T02:13:55.532 INFO:tasks.workunit.client.1.smithi192.stdout:9/330: creat d0/d9/d44/d4f/f79 x:0 0 0 2024-04-23T02:13:55.534 INFO:tasks.workunit.client.0.smithi071.stdout:3/187: unlink l2 0 2024-04-23T02:13:55.535 INFO:tasks.workunit.client.1.smithi192.stdout:4/488: dwrite de/d15/d19/d3d/d49/d26/d59/f63 [0,4194304] 0 2024-04-23T02:13:55.535 INFO:tasks.workunit.client.1.smithi192.stdout:8/392: mkdir d1/d12/d68/d87 0 2024-04-23T02:13:55.536 INFO:tasks.workunit.client.1.smithi192.stdout:9/331: chown d0/d9/d44/d74/d3c/d5f/l6b 6693 1 2024-04-23T02:13:55.537 INFO:tasks.workunit.client.0.smithi071.stdout:7/191: dwrite d3/d17/d1c/f26 [0,4194304] 0 2024-04-23T02:13:55.538 INFO:tasks.workunit.client.0.smithi071.stdout:6/231: dwrite d6/d3e/f42 [0,4194304] 0 2024-04-23T02:13:55.538 INFO:tasks.workunit.client.0.smithi071.stdout:8/336: write dc/f28 [4322579,21596] 0 2024-04-23T02:13:55.538 INFO:tasks.workunit.client.0.smithi071.stdout:2/226: mknod d0/d2/d26/d14/d1f/d36/c46 0 2024-04-23T02:13:55.539 INFO:tasks.workunit.client.0.smithi071.stdout:5/376: write d3/d34/d6b/f8d [900770,106318] 0 2024-04-23T02:13:55.543 INFO:tasks.workunit.client.1.smithi192.stdout:1/166: readlink dc/l1c 0 2024-04-23T02:13:55.543 INFO:tasks.workunit.client.0.smithi071.stdout:3/188: unlink d7/d8/f9 0 2024-04-23T02:13:55.543 INFO:tasks.workunit.client.0.smithi071.stdout:6/232: symlink d6/d1f/l44 0 2024-04-23T02:13:55.544 INFO:tasks.workunit.client.0.smithi071.stdout:2/227: stat d0/d2/d26/df/c28 0 2024-04-23T02:13:55.544 INFO:tasks.workunit.client.0.smithi071.stdout:7/192: mknod d3/d17/d41/c42 0 2024-04-23T02:13:55.544 INFO:tasks.workunit.client.1.smithi192.stdout:8/393: rmdir d1/d12/d23/d4b/d73 39 2024-04-23T02:13:55.544 INFO:tasks.workunit.client.0.smithi071.stdout:5/377: chown d3/d1d/d43 364193 1 2024-04-23T02:13:55.544 INFO:tasks.workunit.client.0.smithi071.stdout:8/337: rmdir dc/d56 39 2024-04-23T02:13:55.545 INFO:tasks.workunit.client.0.smithi071.stdout:8/338: stat dc 0 2024-04-23T02:13:55.545 INFO:tasks.workunit.client.1.smithi192.stdout:2/284: dwrite d8/d14/d51/f56 [0,4194304] 0 2024-04-23T02:13:55.546 INFO:tasks.workunit.client.0.smithi071.stdout:7/193: write d3/d17/d41/d34/f3c [611775,112599] 0 2024-04-23T02:13:55.548 INFO:tasks.workunit.client.0.smithi071.stdout:3/189: fsync d7/d27/d2c/f2e 0 2024-04-23T02:13:55.550 INFO:tasks.workunit.client.0.smithi071.stdout:7/194: fsync d3/d17/d1c/f2c 0 2024-04-23T02:13:55.550 INFO:tasks.workunit.client.0.smithi071.stdout:2/228: mknod d0/d2/c47 0 2024-04-23T02:13:55.552 INFO:tasks.workunit.client.0.smithi071.stdout:0/215: truncate d0/d3/f2d 2097951 0 2024-04-23T02:13:55.554 INFO:tasks.workunit.client.1.smithi192.stdout:4/489: mkdir de/d15/d19/d3d/d34/d7c/da1 0 2024-04-23T02:13:55.555 INFO:tasks.workunit.client.0.smithi071.stdout:8/339: mknod dc/d15/d50/d5f/c67 0 2024-04-23T02:13:55.556 INFO:tasks.workunit.client.0.smithi071.stdout:3/190: mkdir d7/d8/d42 0 2024-04-23T02:13:55.557 INFO:tasks.workunit.client.1.smithi192.stdout:9/332: rename d0/d9/d44/d51/d5b/d6d/c71 to d0/d9/d44/d53/c7a 0 2024-04-23T02:13:55.558 INFO:tasks.workunit.client.0.smithi071.stdout:5/378: dwrite d3/d34/f6a [0,4194304] 0 2024-04-23T02:13:55.560 INFO:tasks.workunit.client.1.smithi192.stdout:2/285: creat d8/d26/d27/d53/f5b x:0 0 0 2024-04-23T02:13:55.562 INFO:tasks.workunit.client.0.smithi071.stdout:6/233: rename d6/f1d to d6/d22/f45 0 2024-04-23T02:13:55.564 INFO:tasks.workunit.client.1.smithi192.stdout:4/490: mknod de/d15/d2a/ca2 0 2024-04-23T02:13:55.565 INFO:tasks.workunit.client.1.smithi192.stdout:1/167: rename lb to dc/d14/d23/l2b 0 2024-04-23T02:13:55.565 INFO:tasks.workunit.client.0.smithi071.stdout:3/191: write d7/d8/fc [4227126,42769] 0 2024-04-23T02:13:55.566 INFO:tasks.workunit.client.0.smithi071.stdout:5/379: creat d3/d34/d6b/d81/f91 x:0 0 0 2024-04-23T02:13:55.567 INFO:tasks.workunit.client.0.smithi071.stdout:7/195: mkdir d3/d17/d1c/d2a/d43 0 2024-04-23T02:13:55.568 INFO:tasks.workunit.client.0.smithi071.stdout:5/380: fsync d3/da/f4d 0 2024-04-23T02:13:55.568 INFO:tasks.workunit.client.0.smithi071.stdout:6/234: mkdir d6/d1f/d46 0 2024-04-23T02:13:55.569 INFO:tasks.workunit.client.0.smithi071.stdout:8/340: mkdir dc/d15/d5d/d68 0 2024-04-23T02:13:55.570 INFO:tasks.workunit.client.1.smithi192.stdout:9/333: readlink d0/d9/d44/d4f/l35 0 2024-04-23T02:13:55.570 INFO:tasks.workunit.client.1.smithi192.stdout:1/168: read - dc/d14/f27 zero size 2024-04-23T02:13:55.570 INFO:tasks.workunit.client.1.smithi192.stdout:6/448: sync 2024-04-23T02:13:55.570 INFO:tasks.workunit.client.0.smithi071.stdout:3/192: unlink d7/d22/d23/f3a 0 2024-04-23T02:13:55.571 INFO:tasks.workunit.client.1.smithi192.stdout:8/394: dwrite d1/d12/d23/f3c [4194304,4194304] 0 2024-04-23T02:13:55.571 INFO:tasks.workunit.client.0.smithi071.stdout:7/196: symlink d3/l44 0 2024-04-23T02:13:55.573 INFO:tasks.workunit.client.1.smithi192.stdout:4/491: readlink de/d15/d19/d3d/d49/d70/d73/l87 0 2024-04-23T02:13:55.575 INFO:tasks.workunit.client.0.smithi071.stdout:8/341: unlink dc/d10/f58 0 2024-04-23T02:13:55.575 INFO:tasks.workunit.client.1.smithi192.stdout:9/334: creat d0/d9/d44/d51/d5b/d61/f7b x:0 0 0 2024-04-23T02:13:55.575 INFO:tasks.workunit.client.1.smithi192.stdout:8/395: chown d1/d12/c36 20813 1 2024-04-23T02:13:55.578 INFO:tasks.workunit.client.0.smithi071.stdout:5/381: rename d3/da/d1a/d1c/f1e to d3/da/f92 0 2024-04-23T02:13:55.580 INFO:tasks.workunit.client.0.smithi071.stdout:5/382: write d3/da/d1a/d1c/d46/d4e/f70 [640252,3986] 0 2024-04-23T02:13:55.582 INFO:tasks.workunit.client.0.smithi071.stdout:8/342: creat dc/d56/d64/f69 x:0 0 0 2024-04-23T02:13:55.583 INFO:tasks.workunit.client.0.smithi071.stdout:1/144: dread fa [0,4194304] 0 2024-04-23T02:13:55.584 INFO:tasks.workunit.client.0.smithi071.stdout:5/383: symlink d3/df/l93 0 2024-04-23T02:13:55.584 INFO:tasks.workunit.client.0.smithi071.stdout:8/343: write dc/d10/f14 [1746310,54522] 0 2024-04-23T02:13:55.586 INFO:tasks.workunit.client.0.smithi071.stdout:5/384: readlink d3/d34/d6b/d81/l51 0 2024-04-23T02:13:55.586 INFO:tasks.workunit.client.1.smithi192.stdout:2/286: dwrite f4 [0,4194304] 0 2024-04-23T02:13:55.587 INFO:tasks.workunit.client.1.smithi192.stdout:4/492: mkdir de/d15/d2a/da3 0 2024-04-23T02:13:55.587 INFO:tasks.workunit.client.0.smithi071.stdout:2/229: rmdir d0/d2/d26 39 2024-04-23T02:13:55.591 INFO:tasks.workunit.client.0.smithi071.stdout:1/145: rename db/fe to db/f25 0 2024-04-23T02:13:55.592 INFO:tasks.workunit.client.1.smithi192.stdout:7/338: dread d5/d17/d1c/d26/f28 [0,4194304] 0 2024-04-23T02:13:55.592 INFO:tasks.workunit.client.1.smithi192.stdout:6/449: rename d3/d15/d1b/d21/d52/f85 to d3/d15/d1b/d21/d3a/d51/f93 0 2024-04-23T02:13:55.595 INFO:tasks.workunit.client.0.smithi071.stdout:3/193: fdatasync d7/d8/fc 0 2024-04-23T02:13:55.595 INFO:tasks.workunit.client.1.smithi192.stdout:1/169: dwrite dc/f29 [0,4194304] 0 2024-04-23T02:13:55.596 INFO:tasks.workunit.client.0.smithi071.stdout:1/146: fdatasync f2 0 2024-04-23T02:13:55.596 INFO:tasks.workunit.client.0.smithi071.stdout:2/230: rmdir d0/d2/d26/d3a 39 2024-04-23T02:13:55.596 INFO:tasks.workunit.client.1.smithi192.stdout:7/339: write d5/d17/d1c/f24 [4811135,100816] 0 2024-04-23T02:13:55.597 INFO:tasks.workunit.client.0.smithi071.stdout:8/344: link f9 dc/d56/f6a 0 2024-04-23T02:13:55.597 INFO:tasks.workunit.client.1.smithi192.stdout:7/340: chown d5/d17/d1c/d59/d5a/d41/f54 58452568 1 2024-04-23T02:13:55.597 INFO:tasks.workunit.client.1.smithi192.stdout:2/287: creat d8/d25/f5c x:0 0 0 2024-04-23T02:13:55.600 INFO:tasks.workunit.client.0.smithi071.stdout:3/194: write d7/d22/f25 [672695,101831] 0 2024-04-23T02:13:55.603 INFO:tasks.workunit.client.0.smithi071.stdout:1/147: unlink db/df/d13/d20/f24 0 2024-04-23T02:13:55.603 INFO:tasks.workunit.client.0.smithi071.stdout:5/385: dwrite d3/da/d1a/d1c/f2e [0,4194304] 0 2024-04-23T02:13:55.604 INFO:tasks.workunit.client.0.smithi071.stdout:0/216: dread d0/d3/de/ff [0,4194304] 0 2024-04-23T02:13:55.607 INFO:tasks.workunit.client.0.smithi071.stdout:8/345: rename dc/d10/d32/f4a to dc/d15/f6b 0 2024-04-23T02:13:55.609 INFO:tasks.workunit.client.0.smithi071.stdout:3/195: creat d7/d10/d38/f43 x:0 0 0 2024-04-23T02:13:55.609 INFO:tasks.workunit.client.0.smithi071.stdout:8/346: chown dc/d15/d50/d5f/c67 106 1 2024-04-23T02:13:55.610 INFO:tasks.workunit.client.1.smithi192.stdout:2/288: chown d8/l2b 103 1 2024-04-23T02:13:55.611 INFO:tasks.workunit.client.0.smithi071.stdout:1/148: mknod db/df/d13/d20/c26 0 2024-04-23T02:13:55.612 INFO:tasks.workunit.client.0.smithi071.stdout:3/196: mkdir d7/d10/d44 0 2024-04-23T02:13:55.613 INFO:tasks.workunit.client.0.smithi071.stdout:3/197: chown d7/d22/d23 0 1 2024-04-23T02:13:55.614 INFO:tasks.workunit.client.0.smithi071.stdout:7/197: getdents d3/d17/d1c/d2a 0 2024-04-23T02:13:55.614 INFO:tasks.workunit.client.1.smithi192.stdout:8/396: rename d1/d12/d68/d87 to d1/d12/d23/d4b/d73/d7e/d88 0 2024-04-23T02:13:55.615 INFO:tasks.workunit.client.1.smithi192.stdout:4/493: symlink de/d15/d19/la4 0 2024-04-23T02:13:55.615 INFO:tasks.workunit.client.0.smithi071.stdout:7/198: fsync d3/f11 0 2024-04-23T02:13:55.616 INFO:tasks.workunit.client.0.smithi071.stdout:8/347: unlink dc/d10/d55/f60 0 2024-04-23T02:13:55.616 INFO:tasks.workunit.client.1.smithi192.stdout:6/450: getdents d3/d15/d1b/d21/d52/d64/d8d 0 2024-04-23T02:13:55.616 INFO:tasks.workunit.client.1.smithi192.stdout:2/289: dread d8/f18 [0,4194304] 0 2024-04-23T02:13:55.617 INFO:tasks.workunit.client.0.smithi071.stdout:8/348: fdatasync dc/d3a/f3c 0 2024-04-23T02:13:55.618 INFO:tasks.workunit.client.1.smithi192.stdout:1/170: mknod dc/c2c 0 2024-04-23T02:13:55.618 INFO:tasks.workunit.client.1.smithi192.stdout:8/397: dread - d1/d12/d23/d4b/d73/d45/f80 zero size 2024-04-23T02:13:55.619 INFO:tasks.workunit.client.0.smithi071.stdout:0/217: creat d0/d3/de/d10/d14/d2b/f43 x:0 0 0 2024-04-23T02:13:55.624 INFO:tasks.workunit.client.0.smithi071.stdout:3/198: rename d7/d8/c12 to d7/d27/c45 0 2024-04-23T02:13:55.625 INFO:tasks.workunit.client.0.smithi071.stdout:7/199: rmdir d3/d17/d41/d34/d39 39 2024-04-23T02:13:55.626 INFO:tasks.workunit.client.1.smithi192.stdout:7/341: link d5/d17/d1c/d4a/c4f d5/d17/d1c/d59/d5a/d41/d50/d3d/c61 0 2024-04-23T02:13:55.626 INFO:tasks.workunit.client.1.smithi192.stdout:1/171: truncate fa 145917 0 2024-04-23T02:13:55.627 INFO:tasks.workunit.client.0.smithi071.stdout:1/149: dwrite db/d11/f1d [0,4194304] 0 2024-04-23T02:13:55.627 INFO:tasks.workunit.client.1.smithi192.stdout:6/451: dread d3/f34 [0,4194304] 0 2024-04-23T02:13:55.627 INFO:tasks.workunit.client.1.smithi192.stdout:7/342: write d5/d17/d1c/d57/f46 [348537,113405] 0 2024-04-23T02:13:55.629 INFO:tasks.workunit.client.1.smithi192.stdout:4/494: rename de/d15/d19/d3d/d34/l38 to de/d15/d2a/d61/la5 0 2024-04-23T02:13:55.630 INFO:tasks.workunit.client.0.smithi071.stdout:0/218: rmdir d0/d3/de/d10/d14/d32 39 2024-04-23T02:13:55.631 INFO:tasks.workunit.client.0.smithi071.stdout:1/150: chown db/d1a 383 1 2024-04-23T02:13:55.631 INFO:tasks.workunit.client.0.smithi071.stdout:6/235: dread d6/d1f/f3b [0,4194304] 0 2024-04-23T02:13:55.633 INFO:tasks.workunit.client.1.smithi192.stdout:4/495: write de/d15/d19/d1a/f23 [1270379,34879] 0 2024-04-23T02:13:55.635 INFO:tasks.workunit.client.1.smithi192.stdout:2/290: getdents d8/d26/d29/d43/d50 0 2024-04-23T02:13:55.635 INFO:tasks.workunit.client.0.smithi071.stdout:1/151: fsync db/df/f19 0 2024-04-23T02:13:55.637 INFO:tasks.workunit.client.1.smithi192.stdout:7/343: creat d5/d17/d1c/d59/d5a/d41/d50/f62 x:0 0 0 2024-04-23T02:13:55.638 INFO:tasks.workunit.client.0.smithi071.stdout:3/199: mknod d7/d1d/d31/c46 0 2024-04-23T02:13:55.639 INFO:tasks.workunit.client.0.smithi071.stdout:7/200: dwrite d3/f12 [4194304,4194304] 0 2024-04-23T02:13:55.640 INFO:tasks.workunit.client.1.smithi192.stdout:9/335: truncate d0/d9/d44/d4f/f28 2029395 0 2024-04-23T02:13:55.640 INFO:tasks.workunit.client.1.smithi192.stdout:8/398: rename d1/d9/d3e/f70 to d1/d12/d7d/f89 0 2024-04-23T02:13:55.642 INFO:tasks.workunit.client.0.smithi071.stdout:1/152: symlink db/d11/l27 0 2024-04-23T02:13:55.643 INFO:tasks.workunit.client.1.smithi192.stdout:4/496: dread de/d15/d19/d3d/d49/f60 [0,4194304] 0 2024-04-23T02:13:55.644 INFO:tasks.workunit.client.1.smithi192.stdout:8/399: write d1/d12/d23/d4b/f69 [341235,105129] 0 2024-04-23T02:13:55.644 INFO:tasks.workunit.client.0.smithi071.stdout:6/236: creat d6/d3e/d3f/f47 x:0 0 0 2024-04-23T02:13:55.645 INFO:tasks.workunit.client.0.smithi071.stdout:3/200: unlink c1 0 2024-04-23T02:13:55.645 INFO:tasks.workunit.client.0.smithi071.stdout:3/201: stat f0 0 2024-04-23T02:13:55.645 INFO:tasks.workunit.client.0.smithi071.stdout:1/153: truncate db/f25 4357447 0 2024-04-23T02:13:55.646 INFO:tasks.workunit.client.1.smithi192.stdout:1/172: fsync fa 0 2024-04-23T02:13:55.646 INFO:tasks.workunit.client.1.smithi192.stdout:9/336: readlink d0/d9/d44/d4f/l40 0 2024-04-23T02:13:55.647 INFO:tasks.workunit.client.0.smithi071.stdout:6/237: chown d6/f3c 929458 1 2024-04-23T02:13:55.647 INFO:tasks.workunit.client.0.smithi071.stdout:1/154: fdatasync db/df/f19 0 2024-04-23T02:13:55.647 INFO:tasks.workunit.client.1.smithi192.stdout:4/497: truncate de/d15/d53/f83 686260 0 2024-04-23T02:13:55.648 INFO:tasks.workunit.client.1.smithi192.stdout:8/400: mkdir d1/d9/d3e/d59/d8a 0 2024-04-23T02:13:55.649 INFO:tasks.workunit.client.0.smithi071.stdout:7/201: creat d3/d17/d1c/f45 x:0 0 0 2024-04-23T02:13:55.649 INFO:tasks.workunit.client.1.smithi192.stdout:1/173: rmdir dc/d16/d19 39 2024-04-23T02:13:55.651 INFO:tasks.workunit.client.1.smithi192.stdout:4/498: creat de/d15/d53/d81/fa6 x:0 0 0 2024-04-23T02:13:55.651 INFO:tasks.workunit.client.0.smithi071.stdout:6/238: rename d6/c20 to d6/d1f/d46/c48 0 2024-04-23T02:13:55.652 INFO:tasks.workunit.client.1.smithi192.stdout:1/174: dread - dc/f11 zero size 2024-04-23T02:13:55.652 INFO:tasks.workunit.client.0.smithi071.stdout:6/239: readlink d6/d1f/l2d 0 2024-04-23T02:13:55.653 INFO:tasks.workunit.client.0.smithi071.stdout:3/202: mkdir d7/d8/d42/d47 0 2024-04-23T02:13:55.655 INFO:tasks.workunit.client.0.smithi071.stdout:1/155: symlink db/d1a/l28 0 2024-04-23T02:13:55.657 INFO:tasks.workunit.client.1.smithi192.stdout:8/401: mkdir d1/d12/d23/d4b/d73/d7e/d7b/d7c/d8b 0 2024-04-23T02:13:55.657 INFO:tasks.workunit.client.1.smithi192.stdout:8/402: truncate d1/d12/d23/f40 4679569 0 2024-04-23T02:13:55.657 INFO:tasks.workunit.client.1.smithi192.stdout:9/337: rename d0/f34 to d0/d9/d44/f7c 0 2024-04-23T02:13:55.658 INFO:tasks.workunit.client.1.smithi192.stdout:4/499: write de/d15/d19/d3d/d49/d26/d59/d6a/f7f [937163,74695] 0 2024-04-23T02:13:55.659 INFO:tasks.workunit.client.1.smithi192.stdout:8/403: mkdir d1/d12/d23/d2c/d8c 0 2024-04-23T02:13:55.662 INFO:tasks.workunit.client.1.smithi192.stdout:4/500: write de/d15/d2a/d74/f7e [1033357,65685] 0 2024-04-23T02:13:55.663 INFO:tasks.workunit.client.0.smithi071.stdout:3/203: dwrite d7/d8/fe [0,4194304] 0 2024-04-23T02:13:55.664 INFO:tasks.workunit.client.0.smithi071.stdout:3/204: fsync d7/d10/f2f 0 2024-04-23T02:13:55.664 INFO:tasks.workunit.client.0.smithi071.stdout:2/231: dread d0/d2/d26/f18 [0,4194304] 0 2024-04-23T02:13:55.665 INFO:tasks.workunit.client.1.smithi192.stdout:8/404: dread - d1/d12/d23/d4b/d73/d4c/f84 zero size 2024-04-23T02:13:55.670 INFO:tasks.workunit.client.0.smithi071.stdout:3/205: dread d7/d27/d3e/f40 [0,4194304] 0 2024-04-23T02:13:55.671 INFO:tasks.workunit.client.1.smithi192.stdout:1/175: dwrite dc/ff [0,4194304] 0 2024-04-23T02:13:55.671 INFO:tasks.workunit.client.0.smithi071.stdout:3/206: readlink d7/d27/d2c/l33 0 2024-04-23T02:13:55.673 INFO:tasks.workunit.client.1.smithi192.stdout:1/176: write dc/f29 [1349052,43718] 0 2024-04-23T02:13:55.673 INFO:tasks.workunit.client.0.smithi071.stdout:1/156: dwrite db/f25 [0,4194304] 0 2024-04-23T02:13:55.675 INFO:tasks.workunit.client.1.smithi192.stdout:4/501: rename de/d15/d19/d3d/d49/d26/d59/d6a/f7f to de/d15/d19/d3d/d49/d26/d59/d6a/fa7 0 2024-04-23T02:13:55.676 INFO:tasks.workunit.client.0.smithi071.stdout:7/202: creat d3/d17/d41/d34/d39/d3b/f46 x:0 0 0 2024-04-23T02:13:55.680 INFO:tasks.workunit.client.1.smithi192.stdout:1/177: mknod dc/d16/c2d 0 2024-04-23T02:13:55.680 INFO:tasks.workunit.client.1.smithi192.stdout:4/502: truncate de/d15/d53/d81/fa6 47015 0 2024-04-23T02:13:55.684 INFO:tasks.workunit.client.0.smithi071.stdout:2/232: link d0/d2/d26/d14/d1f/c33 d0/d2/c48 0 2024-04-23T02:13:55.687 INFO:tasks.workunit.client.0.smithi071.stdout:1/157: creat db/df/d13/d20/d21/f29 x:0 0 0 2024-04-23T02:13:55.688 INFO:tasks.workunit.client.0.smithi071.stdout:0/219: dwrite d0/d3/de/d10/d14/f1e [0,4194304] 0 2024-04-23T02:13:55.689 INFO:tasks.workunit.client.0.smithi071.stdout:8/349: dread dc/f28 [0,4194304] 0 2024-04-23T02:13:55.690 INFO:tasks.workunit.client.1.smithi192.stdout:1/178: write dc/f13 [2424203,118302] 0 2024-04-23T02:13:55.690 INFO:tasks.workunit.client.1.smithi192.stdout:9/338: mkdir d0/d9/d44/d74/d3c/d5f/d7d 0 2024-04-23T02:13:55.691 INFO:tasks.workunit.client.0.smithi071.stdout:2/233: rmdir d0/d2/d26/d14/d1f 39 2024-04-23T02:13:55.691 INFO:tasks.workunit.client.0.smithi071.stdout:6/240: truncate d6/f11 2717586 0 2024-04-23T02:13:55.693 INFO:tasks.workunit.client.0.smithi071.stdout:1/158: creat db/df/f2a x:0 0 0 2024-04-23T02:13:55.695 INFO:tasks.workunit.client.1.smithi192.stdout:9/339: creat d0/d9/d44/d74/d3c/d5f/f7e x:0 0 0 2024-04-23T02:13:55.697 INFO:tasks.workunit.client.1.smithi192.stdout:1/179: rmdir dc/d14/d15/d18 0 2024-04-23T02:13:55.698 INFO:tasks.workunit.client.0.smithi071.stdout:2/234: mkdir d0/d2/d49 0 2024-04-23T02:13:55.699 INFO:tasks.workunit.client.1.smithi192.stdout:1/180: read dc/fd [698825,4963] 0 2024-04-23T02:13:55.700 INFO:tasks.workunit.client.1.smithi192.stdout:9/340: symlink d0/d9/d44/l7f 0 2024-04-23T02:13:55.702 INFO:tasks.workunit.client.1.smithi192.stdout:1/181: mknod dc/d14/d23/c2e 0 2024-04-23T02:13:55.703 INFO:tasks.workunit.client.0.smithi071.stdout:2/235: write d0/f3f [580577,45969] 0 2024-04-23T02:13:55.703 INFO:tasks.workunit.client.1.smithi192.stdout:4/503: truncate de/d15/d19/d3d/f5f 744988 0 2024-04-23T02:13:55.704 INFO:tasks.workunit.client.0.smithi071.stdout:8/350: dwrite dc/f28 [4194304,4194304] 0 2024-04-23T02:13:55.704 INFO:tasks.workunit.client.1.smithi192.stdout:9/341: chown d0/f4e 17624 1 2024-04-23T02:13:55.704 INFO:tasks.workunit.client.0.smithi071.stdout:0/220: rename d0/d3/de/d10/d14/d2b to d0/d3/de/d10/d14/d32/d44 0 2024-04-23T02:13:55.705 INFO:tasks.workunit.client.0.smithi071.stdout:3/207: truncate d7/fd 1752013 0 2024-04-23T02:13:55.705 INFO:tasks.workunit.client.0.smithi071.stdout:2/236: read d0/d2/d26/d14/f23 [2720060,51716] 0 2024-04-23T02:13:55.708 INFO:tasks.workunit.client.1.smithi192.stdout:1/182: symlink dc/d16/d19/l2f 0 2024-04-23T02:13:55.709 INFO:tasks.workunit.client.1.smithi192.stdout:1/183: chown dc/f10 712689 1 2024-04-23T02:13:55.711 INFO:tasks.workunit.client.1.smithi192.stdout:4/504: dwrite de/f8c [0,4194304] 0 2024-04-23T02:13:55.713 INFO:tasks.workunit.client.0.smithi071.stdout:1/159: getdents db/d1a 0 2024-04-23T02:13:55.713 INFO:tasks.workunit.client.0.smithi071.stdout:2/237: read - d0/d2/d26/d3a/f44 zero size 2024-04-23T02:13:55.714 INFO:tasks.workunit.client.1.smithi192.stdout:1/184: mkdir dc/d16/d19/d30 0 2024-04-23T02:13:55.714 INFO:tasks.workunit.client.1.smithi192.stdout:4/505: write de/d15/d19/d3d/d34/d4a/f71 [741481,125212] 0 2024-04-23T02:13:55.715 INFO:tasks.workunit.client.0.smithi071.stdout:1/160: chown db/df/f2a 8 1 2024-04-23T02:13:55.715 INFO:tasks.workunit.client.1.smithi192.stdout:8/405: dread d1/d12/f24 [0,4194304] 0 2024-04-23T02:13:55.715 INFO:tasks.workunit.client.0.smithi071.stdout:8/351: fdatasync dc/d56/f6a 0 2024-04-23T02:13:55.718 INFO:tasks.workunit.client.1.smithi192.stdout:1/185: write f4 [2321703,93537] 0 2024-04-23T02:13:55.719 INFO:tasks.workunit.client.1.smithi192.stdout:8/406: chown d1/d12/d68/d6a/l74 42 1 2024-04-23T02:13:55.722 INFO:tasks.workunit.client.1.smithi192.stdout:8/407: write d1/d12/d7d/f89 [376925,92447] 0 2024-04-23T02:13:55.722 INFO:tasks.workunit.client.0.smithi071.stdout:1/161: truncate db/df/d13/d20/d21/f29 240305 0 2024-04-23T02:13:55.723 INFO:tasks.workunit.client.1.smithi192.stdout:8/408: stat d1/f15 0 2024-04-23T02:13:55.724 INFO:tasks.workunit.client.0.smithi071.stdout:1/162: dread - db/df/d13/f1b zero size 2024-04-23T02:13:55.724 INFO:tasks.workunit.client.0.smithi071.stdout:3/208: link d7/d8/c20 d7/d28/d3b/c48 0 2024-04-23T02:13:55.726 INFO:tasks.workunit.client.0.smithi071.stdout:2/238: creat d0/d2/d26/d14/d1f/d36/f4a x:0 0 0 2024-04-23T02:13:55.726 INFO:tasks.workunit.client.0.smithi071.stdout:0/221: link d0/d3/f2d d0/d3/de/d10/d14/d32/f45 0 2024-04-23T02:13:55.726 INFO:tasks.workunit.client.0.smithi071.stdout:1/163: write f5 [1159630,109712] 0 2024-04-23T02:13:55.728 INFO:tasks.workunit.client.0.smithi071.stdout:8/352: rename dc/ff to dc/d10/d55/f6c 0 2024-04-23T02:13:55.728 INFO:tasks.workunit.client.0.smithi071.stdout:1/164: chown db/df/f19 980451016 1 2024-04-23T02:13:55.730 INFO:tasks.workunit.client.0.smithi071.stdout:9/252: sync 2024-04-23T02:13:55.731 INFO:tasks.workunit.client.1.smithi192.stdout:8/409: dwrite d1/d9/f22 [0,4194304] 0 2024-04-23T02:13:55.732 INFO:tasks.workunit.client.0.smithi071.stdout:3/209: rename d7/d28/l35 to d7/d8/d42/d47/l49 0 2024-04-23T02:13:55.733 INFO:tasks.workunit.client.0.smithi071.stdout:3/210: readlink d7/d10/l2d 0 2024-04-23T02:13:55.734 INFO:tasks.workunit.client.1.smithi192.stdout:8/410: dread - d1/d12/d23/d4b/d73/f65 zero size 2024-04-23T02:13:55.734 INFO:tasks.workunit.client.1.smithi192.stdout:4/506: dread de/f21 [0,4194304] 0 2024-04-23T02:13:55.735 INFO:tasks.workunit.client.1.smithi192.stdout:8/411: chown d1/d9/d3e 12224 1 2024-04-23T02:13:55.735 INFO:tasks.workunit.client.0.smithi071.stdout:8/353: symlink dc/d10/d55/l6d 0 2024-04-23T02:13:55.736 INFO:tasks.workunit.client.0.smithi071.stdout:6/241: dread d6/d22/f45 [0,4194304] 0 2024-04-23T02:13:55.736 INFO:tasks.workunit.client.1.smithi192.stdout:4/507: creat de/d15/d19/d3d/d49/d26/fa8 x:0 0 0 2024-04-23T02:13:55.738 INFO:tasks.workunit.client.1.smithi192.stdout:8/412: creat d1/d9/d3e/d59/d8a/f8d x:0 0 0 2024-04-23T02:13:55.743 INFO:tasks.workunit.client.0.smithi071.stdout:8/354: mknod dc/d56/c6e 0 2024-04-23T02:13:55.744 INFO:tasks.workunit.client.1.smithi192.stdout:8/413: dread d1/d12/d7d/f89 [0,4194304] 0 2024-04-23T02:13:55.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:55 smithi071 ceph-mon[39837]: pgmap v24: 65 pgs: 65 active+clean; 895 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 5.2 MiB/s rd, 100 MiB/s wr, 171 op/s 2024-04-23T02:13:55.745 INFO:tasks.workunit.client.1.smithi192.stdout:8/414: chown d1/d12/d23/d4b/d73/d7e/d88 21768544 1 2024-04-23T02:13:55.746 INFO:tasks.workunit.client.0.smithi071.stdout:0/222: symlink d0/d3/de/d10/d14/d32/d44/d31/l46 0 2024-04-23T02:13:55.746 INFO:tasks.workunit.client.0.smithi071.stdout:1/165: dwrite f5 [0,4194304] 0 2024-04-23T02:13:55.747 INFO:tasks.workunit.client.0.smithi071.stdout:2/239: dwrite d0/d2/d26/d14/f42 [0,4194304] 0 2024-04-23T02:13:55.748 INFO:tasks.workunit.client.0.smithi071.stdout:1/166: readlink l8 0 2024-04-23T02:13:55.748 INFO:tasks.workunit.client.0.smithi071.stdout:6/242: symlink d6/d3e/d3f/l49 0 2024-04-23T02:13:55.749 INFO:tasks.workunit.client.1.smithi192.stdout:8/415: symlink d1/d12/d23/d4b/d73/d7e/l8e 0 2024-04-23T02:13:55.749 INFO:tasks.workunit.client.0.smithi071.stdout:0/223: fdatasync d0/d3/f3f 0 2024-04-23T02:13:55.750 INFO:tasks.workunit.client.0.smithi071.stdout:9/253: dwrite d1/f28 [0,4194304] 0 2024-04-23T02:13:55.752 INFO:tasks.workunit.client.0.smithi071.stdout:9/254: fdatasync d1/fa 0 2024-04-23T02:13:55.753 INFO:tasks.workunit.client.1.smithi192.stdout:8/416: write d1/d12/d23/d4b/d73/d45/d46/f48 [4120932,75469] 0 2024-04-23T02:13:55.753 INFO:tasks.workunit.client.1.smithi192.stdout:4/508: dwrite de/f1b [0,4194304] 0 2024-04-23T02:13:55.758 INFO:tasks.workunit.client.0.smithi071.stdout:6/243: creat d6/d3e/f4a x:0 0 0 2024-04-23T02:13:55.760 INFO:tasks.workunit.client.0.smithi071.stdout:0/224: mknod d0/d3/de/d10/d14/d32/d37/c47 0 2024-04-23T02:13:55.761 INFO:tasks.workunit.client.1.smithi192.stdout:8/417: chown d1/d12/d23/d4b/d73/d45/f76 7406 1 2024-04-23T02:13:55.765 INFO:tasks.workunit.client.0.smithi071.stdout:8/355: rmdir dc/d15/d5d/d68 0 2024-04-23T02:13:55.765 INFO:tasks.workunit.client.1.smithi192.stdout:4/509: rename de/d15/d19/d1a/c6f to de/d15/d19/d3d/d49/d26/d59/d8a/ca9 0 2024-04-23T02:13:55.765 INFO:tasks.workunit.client.1.smithi192.stdout:8/418: chown d1/d12/c16 3 1 2024-04-23T02:13:55.766 INFO:tasks.workunit.client.0.smithi071.stdout:2/240: symlink d0/d2/d3c/l4b 0 2024-04-23T02:13:55.767 INFO:tasks.workunit.client.1.smithi192.stdout:8/419: chown d1/d9/fa 98 1 2024-04-23T02:13:55.768 INFO:tasks.workunit.client.1.smithi192.stdout:4/510: readlink de/d15/d19/d1a/l27 0 2024-04-23T02:13:55.768 INFO:tasks.workunit.client.0.smithi071.stdout:3/211: dwrite d7/d8/f16 [0,4194304] 0 2024-04-23T02:13:55.769 INFO:tasks.workunit.client.0.smithi071.stdout:2/241: stat d0/d2/d26/d14/d1f/d36 0 2024-04-23T02:13:55.770 INFO:tasks.workunit.client.1.smithi192.stdout:8/420: mkdir d1/d9/d8f 0 2024-04-23T02:13:55.771 INFO:tasks.workunit.client.1.smithi192.stdout:4/511: creat de/d15/d2a/d74/d92/faa x:0 0 0 2024-04-23T02:13:55.771 INFO:tasks.workunit.client.0.smithi071.stdout:9/255: truncate d1/fe 244468 0 2024-04-23T02:13:55.771 INFO:tasks.workunit.client.1.smithi192.stdout:8/421: mkdir d1/d9/d8f/d90 0 2024-04-23T02:13:55.772 INFO:tasks.workunit.client.1.smithi192.stdout:8/422: fdatasync d1/d12/d68/f83 0 2024-04-23T02:13:55.772 INFO:tasks.workunit.client.0.smithi071.stdout:9/256: chown d1/d1a/d39/d42/f49 56300660 1 2024-04-23T02:13:55.773 INFO:tasks.workunit.client.0.smithi071.stdout:8/356: symlink dc/d15/d2d/l6f 0 2024-04-23T02:13:55.774 INFO:tasks.workunit.client.0.smithi071.stdout:0/225: mkdir d0/d3/de/d48 0 2024-04-23T02:13:55.775 INFO:tasks.workunit.client.0.smithi071.stdout:3/212: rmdir d7/d27 39 2024-04-23T02:13:55.775 INFO:tasks.workunit.client.0.smithi071.stdout:0/226: write d0/f18 [67438,11887] 0 2024-04-23T02:13:55.777 INFO:tasks.workunit.client.0.smithi071.stdout:6/244: getdents d6/d1f/d46 0 2024-04-23T02:13:55.781 INFO:tasks.workunit.client.0.smithi071.stdout:0/227: mknod d0/d3/de/d48/c49 0 2024-04-23T02:13:55.783 INFO:tasks.workunit.client.0.smithi071.stdout:3/213: rename d7/d28/d3b/c48 to d7/d22/d23/c4a 0 2024-04-23T02:13:55.783 INFO:tasks.workunit.client.0.smithi071.stdout:0/228: rmdir d0/d3/de/d10 39 2024-04-23T02:13:55.785 INFO:tasks.workunit.client.0.smithi071.stdout:2/242: dwrite d0/d2/d26/d14/f43 [0,4194304] 0 2024-04-23T02:13:55.785 INFO:tasks.workunit.client.0.smithi071.stdout:0/229: fdatasync d0/f2 0 2024-04-23T02:13:55.785 INFO:tasks.workunit.client.0.smithi071.stdout:3/214: chown d7/d10/l2d 13382 1 2024-04-23T02:13:55.786 INFO:tasks.workunit.client.0.smithi071.stdout:3/215: write d7/d10/f2f [1292903,109180] 0 2024-04-23T02:13:55.787 INFO:tasks.workunit.client.0.smithi071.stdout:2/243: chown d0/d2/d3c/l4b 7262 1 2024-04-23T02:13:55.793 INFO:tasks.workunit.client.0.smithi071.stdout:2/244: dread d0/f3f [0,4194304] 0 2024-04-23T02:13:55.794 INFO:tasks.workunit.client.0.smithi071.stdout:0/230: link d0/l7 d0/d3/de/d10/d14/d32/d37/d26/l4a 0 2024-04-23T02:13:55.795 INFO:tasks.workunit.client.0.smithi071.stdout:2/245: write d0/d2/f29 [157838,38217] 0 2024-04-23T02:13:55.797 INFO:tasks.workunit.client.0.smithi071.stdout:3/216: dwrite d7/d10/f1e [0,4194304] 0 2024-04-23T02:13:55.799 INFO:tasks.workunit.client.0.smithi071.stdout:8/357: dread dc/d15/f6b [0,4194304] 0 2024-04-23T02:13:55.800 INFO:tasks.workunit.client.0.smithi071.stdout:3/217: chown l4 27 1 2024-04-23T02:13:55.800 INFO:tasks.workunit.client.0.smithi071.stdout:8/358: fsync dc/d10/f27 0 2024-04-23T02:13:55.803 INFO:tasks.workunit.client.0.smithi071.stdout:3/218: read d7/d22/f25 [4536900,5745] 0 2024-04-23T02:13:55.803 INFO:tasks.workunit.client.0.smithi071.stdout:2/246: creat d0/d2/d26/f4c x:0 0 0 2024-04-23T02:13:55.804 INFO:tasks.workunit.client.0.smithi071.stdout:8/359: read dc/d10/f27 [7961286,49850] 0 2024-04-23T02:13:55.808 INFO:tasks.workunit.client.0.smithi071.stdout:3/219: dread d7/d10/f1e [4194304,4194304] 0 2024-04-23T02:13:55.809 INFO:tasks.workunit.client.0.smithi071.stdout:8/360: unlink f8 0 2024-04-23T02:13:55.809 INFO:tasks.workunit.client.0.smithi071.stdout:8/361: write dc/d10/f22 [2913442,4955] 0 2024-04-23T02:13:55.813 INFO:tasks.workunit.client.0.smithi071.stdout:2/247: rename d0/f2b to d0/d2/f4d 0 2024-04-23T02:13:55.813 INFO:tasks.workunit.client.1.smithi192.stdout:3/285: sync 2024-04-23T02:13:55.814 INFO:tasks.workunit.client.0.smithi071.stdout:8/362: mknod dc/d15/d5d/c70 0 2024-04-23T02:13:55.814 INFO:tasks.workunit.client.1.smithi192.stdout:3/286: truncate d1/f56 584052 0 2024-04-23T02:13:55.814 INFO:tasks.workunit.client.0.smithi071.stdout:8/363: chown c7 452498113 1 2024-04-23T02:13:55.816 INFO:tasks.workunit.client.0.smithi071.stdout:2/248: creat d0/d2/d3c/f4e x:0 0 0 2024-04-23T02:13:55.816 INFO:tasks.workunit.client.0.smithi071.stdout:8/364: mknod dc/d10/d32/c71 0 2024-04-23T02:13:55.823 INFO:tasks.workunit.client.1.smithi192.stdout:4/512: unlink c8 0 2024-04-23T02:13:55.823 INFO:tasks.workunit.client.1.smithi192.stdout:3/287: mkdir d1/d3/d6/d7/d5f 0 2024-04-23T02:13:55.823 INFO:tasks.workunit.client.1.smithi192.stdout:8/423: write d1/d12/d7d/f89 [871413,61006] 0 2024-04-23T02:13:55.824 INFO:tasks.workunit.client.1.smithi192.stdout:4/513: stat de/d15/d19/d3d/d34/f76 0 2024-04-23T02:13:55.824 INFO:tasks.workunit.client.0.smithi071.stdout:8/365: mkdir dc/d15/d72 0 2024-04-23T02:13:55.824 INFO:tasks.workunit.client.0.smithi071.stdout:2/249: mkdir d0/d2/d26/d14/d1f/d4f 0 2024-04-23T02:13:55.824 INFO:tasks.workunit.client.0.smithi071.stdout:8/366: creat dc/d15/f73 x:0 0 0 2024-04-23T02:13:55.824 INFO:tasks.workunit.client.0.smithi071.stdout:2/250: creat d0/d2/d34/f50 x:0 0 0 2024-04-23T02:13:55.825 INFO:tasks.workunit.client.1.smithi192.stdout:4/514: write de/d15/d19/d3d/d34/d4a/f71 [1131443,85783] 0 2024-04-23T02:13:55.825 INFO:tasks.workunit.client.0.smithi071.stdout:6/245: dread d6/d22/f28 [0,4194304] 0 2024-04-23T02:13:55.827 INFO:tasks.workunit.client.1.smithi192.stdout:8/424: rename d1/d12/d23/f40 to d1/d12/d23/f91 0 2024-04-23T02:13:55.828 INFO:tasks.workunit.client.0.smithi071.stdout:1/167: truncate fa 444793 0 2024-04-23T02:13:55.828 INFO:tasks.workunit.client.0.smithi071.stdout:2/251: truncate d0/d2/d26/df/f40 1307915 0 2024-04-23T02:13:55.828 INFO:tasks.workunit.client.0.smithi071.stdout:6/246: fsync d6/d22/f28 0 2024-04-23T02:13:55.829 INFO:tasks.workunit.client.1.smithi192.stdout:8/425: read d1/fe [380226,99919] 0 2024-04-23T02:13:55.829 INFO:tasks.workunit.client.0.smithi071.stdout:0/231: dread d0/d3/de/d10/d14/d32/f45 [0,4194304] 0 2024-04-23T02:13:55.829 INFO:tasks.workunit.client.0.smithi071.stdout:1/168: chown l8 1880 1 2024-04-23T02:13:55.830 INFO:tasks.workunit.client.1.smithi192.stdout:4/515: mknod de/d15/d53/d81/cab 0 2024-04-23T02:13:55.831 INFO:tasks.workunit.client.1.smithi192.stdout:4/516: chown f4 41869841 1 2024-04-23T02:13:55.832 INFO:tasks.workunit.client.0.smithi071.stdout:8/367: dwrite dc/d56/f61 [0,4194304] 0 2024-04-23T02:13:55.835 INFO:tasks.workunit.client.1.smithi192.stdout:8/426: getdents d1/d12/d23/d4b/d73/d7e/d88 0 2024-04-23T02:13:55.835 INFO:tasks.workunit.client.1.smithi192.stdout:3/288: truncate d1/d3/f19 724436 0 2024-04-23T02:13:55.836 INFO:tasks.workunit.client.0.smithi071.stdout:2/252: write d0/d2/d26/d14/f23 [1383104,60109] 0 2024-04-23T02:13:55.838 INFO:tasks.workunit.client.0.smithi071.stdout:8/368: rename l5 to dc/d56/d64/l74 0 2024-04-23T02:13:55.839 INFO:tasks.workunit.client.1.smithi192.stdout:4/517: dwrite de/d15/d2a/d74/f7e [0,4194304] 0 2024-04-23T02:13:55.839 INFO:tasks.workunit.client.1.smithi192.stdout:3/289: unlink d1/l5b 0 2024-04-23T02:13:55.840 INFO:tasks.workunit.client.0.smithi071.stdout:0/232: creat d0/d3/de/d10/d14/d32/d37/d26/f4b x:0 0 0 2024-04-23T02:13:55.841 INFO:tasks.workunit.client.0.smithi071.stdout:1/169: mkdir db/df/d13/d2b 0 2024-04-23T02:13:55.841 INFO:tasks.workunit.client.0.smithi071.stdout:0/233: write d0/f2 [1421018,51030] 0 2024-04-23T02:13:55.843 INFO:tasks.workunit.client.0.smithi071.stdout:1/170: truncate db/df/f2a 748850 0 2024-04-23T02:13:55.844 INFO:tasks.workunit.client.0.smithi071.stdout:2/253: mkdir d0/d2/d51 0 2024-04-23T02:13:55.844 INFO:tasks.workunit.client.0.smithi071.stdout:1/171: fsync db/df/d13/d20/d21/f29 0 2024-04-23T02:13:55.846 INFO:tasks.workunit.client.1.smithi192.stdout:8/427: creat d1/d12/f92 x:0 0 0 2024-04-23T02:13:55.847 INFO:tasks.workunit.client.0.smithi071.stdout:0/234: write d0/d3/de/f1d [8102841,4007] 0 2024-04-23T02:13:55.849 INFO:tasks.workunit.client.1.smithi192.stdout:3/290: creat d1/d3/d17/d1f/d33/d24/d34/d40/d4c/f60 x:0 0 0 2024-04-23T02:13:55.851 INFO:tasks.workunit.client.1.smithi192.stdout:8/428: dread d1/d12/d7d/f89 [0,4194304] 0 2024-04-23T02:13:55.851 INFO:tasks.workunit.client.0.smithi071.stdout:1/172: symlink db/df/l2c 0 2024-04-23T02:13:55.854 INFO:tasks.workunit.client.1.smithi192.stdout:4/518: dwrite de/f21 [0,4194304] 0 2024-04-23T02:13:55.854 INFO:tasks.workunit.client.0.smithi071.stdout:1/173: chown db/d1a 174940 1 2024-04-23T02:13:55.855 INFO:tasks.workunit.client.1.smithi192.stdout:4/519: write de/f8c [3399938,15987] 0 2024-04-23T02:13:55.856 INFO:tasks.workunit.client.1.smithi192.stdout:8/429: creat d1/d9/d8f/d90/f93 x:0 0 0 2024-04-23T02:13:55.856 INFO:tasks.workunit.client.0.smithi071.stdout:8/369: creat dc/f75 x:0 0 0 2024-04-23T02:13:55.856 INFO:tasks.workunit.client.0.smithi071.stdout:1/174: dread db/df/f2a [0,4194304] 0 2024-04-23T02:13:55.857 INFO:tasks.workunit.client.0.smithi071.stdout:2/254: creat d0/d2/d45/f52 x:0 0 0 2024-04-23T02:13:55.857 INFO:tasks.workunit.client.1.smithi192.stdout:3/291: rename d1/d3/d6/d7/d29/l4b to d1/d3/d17/d2b/d44/d57/l61 0 2024-04-23T02:13:55.859 INFO:tasks.workunit.client.1.smithi192.stdout:4/520: mkdir de/d15/d53/d81/dac 0 2024-04-23T02:13:55.859 INFO:tasks.workunit.client.1.smithi192.stdout:3/292: chown d1/d3/d17/d2b/d44/c4a 5181 1 2024-04-23T02:13:55.860 INFO:tasks.workunit.client.1.smithi192.stdout:4/521: write de/d15/f8f [3569138,90910] 0 2024-04-23T02:13:55.862 INFO:tasks.workunit.client.1.smithi192.stdout:4/522: chown de/d15/d2a/d74 3 1 2024-04-23T02:13:55.865 INFO:tasks.workunit.client.0.smithi071.stdout:8/370: symlink dc/d3a/l76 0 2024-04-23T02:13:55.865 INFO:tasks.workunit.client.0.smithi071.stdout:0/235: dwrite d0/d3/de/d10/d14/d32/d37/d26/f4b [0,4194304] 0 2024-04-23T02:13:55.866 INFO:tasks.workunit.client.1.smithi192.stdout:3/293: rename d1/d3/d17/c58 to d1/d3/c62 0 2024-04-23T02:13:55.867 INFO:tasks.workunit.client.1.smithi192.stdout:3/294: read d1/d3/d17/f2d [317724,56816] 0 2024-04-23T02:13:55.868 INFO:tasks.workunit.client.0.smithi071.stdout:3/220: dwrite d7/d27/d3e/f40 [4194304,4194304] 0 2024-04-23T02:13:55.868 INFO:tasks.workunit.client.1.smithi192.stdout:8/430: link d1/d12/d23/c44 d1/d12/d23/d4b/d73/d5d/c94 0 2024-04-23T02:13:55.869 INFO:tasks.workunit.client.0.smithi071.stdout:1/175: symlink db/df/l2d 0 2024-04-23T02:13:55.870 INFO:tasks.workunit.client.1.smithi192.stdout:3/295: read d1/d3/d17/f2d [230762,19508] 0 2024-04-23T02:13:55.871 INFO:tasks.workunit.client.0.smithi071.stdout:3/221: chown d7/d1d/c21 218142280 1 2024-04-23T02:13:55.872 INFO:tasks.workunit.client.0.smithi071.stdout:2/255: chown d0/d2/d34/c3e 94243 1 2024-04-23T02:13:55.872 INFO:tasks.workunit.client.0.smithi071.stdout:6/247: dread d6/d17/f1a [0,4194304] 0 2024-04-23T02:13:55.872 INFO:tasks.workunit.client.1.smithi192.stdout:8/431: rmdir d1/d12/d23/d4b/d73/d7e/d7b 39 2024-04-23T02:13:55.873 INFO:tasks.workunit.client.0.smithi071.stdout:8/371: fdatasync dc/d15/f39 0 2024-04-23T02:13:55.876 INFO:tasks.workunit.client.1.smithi192.stdout:3/296: creat d1/d3/d17/d1f/d33/d24/d34/d40/f63 x:0 0 0 2024-04-23T02:13:55.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:55 smithi192 ceph-mon[49662]: pgmap v24: 65 pgs: 65 active+clean; 895 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 5.2 MiB/s rd, 100 MiB/s wr, 171 op/s 2024-04-23T02:13:55.877 INFO:tasks.workunit.client.0.smithi071.stdout:0/236: readlink d0/d3/de/d10/d14/d32/d44/d31/l46 0 2024-04-23T02:13:55.878 INFO:tasks.workunit.client.1.smithi192.stdout:4/523: dwrite de/d15/d19/d3d/d49/d26/d59/d6a/f86 [0,4194304] 0 2024-04-23T02:13:55.878 INFO:tasks.workunit.client.0.smithi071.stdout:6/248: fdatasync d6/d1f/f3b 0 2024-04-23T02:13:55.879 INFO:tasks.workunit.client.1.smithi192.stdout:8/432: rename d1/d12/d68/d6a to d1/d12/d23/d2c/d8c/d95 0 2024-04-23T02:13:55.879 INFO:tasks.workunit.client.0.smithi071.stdout:0/237: write d0/f18 [643294,62347] 0 2024-04-23T02:13:55.881 INFO:tasks.workunit.client.0.smithi071.stdout:1/176: read db/fd [529786,46203] 0 2024-04-23T02:13:55.884 INFO:tasks.workunit.client.0.smithi071.stdout:8/372: truncate dc/d3a/f3c 647101 0 2024-04-23T02:13:55.886 INFO:tasks.workunit.client.1.smithi192.stdout:4/524: truncate de/d15/f2d 2907338 0 2024-04-23T02:13:55.887 INFO:tasks.workunit.client.0.smithi071.stdout:3/222: dwrite f0 [0,4194304] 0 2024-04-23T02:13:55.888 INFO:tasks.workunit.client.0.smithi071.stdout:8/373: write dc/d15/d2d/f3f [336631,128534] 0 2024-04-23T02:13:55.889 INFO:tasks.workunit.client.0.smithi071.stdout:3/223: read d7/d8/f16 [3397617,94915] 0 2024-04-23T02:13:55.890 INFO:tasks.workunit.client.0.smithi071.stdout:6/249: link d6/d1f/l2d d6/d3e/d3f/l4b 0 2024-04-23T02:13:55.890 INFO:tasks.workunit.client.1.smithi192.stdout:4/525: write de/d15/d19/d3d/d34/d4a/f8d [868229,9817] 0 2024-04-23T02:13:55.892 INFO:tasks.workunit.client.1.smithi192.stdout:4/526: rmdir de/d15/d19/d3d/d34 39 2024-04-23T02:13:55.893 INFO:tasks.workunit.client.1.smithi192.stdout:8/433: symlink d1/d12/d23/d4b/d73/d7e/d7b/d7c/d81/l96 0 2024-04-23T02:13:55.895 INFO:tasks.workunit.client.0.smithi071.stdout:0/238: dwrite d0/d3/d1c/f22 [0,4194304] 0 2024-04-23T02:13:55.895 INFO:tasks.workunit.client.1.smithi192.stdout:8/434: write d1/d12/d23/d4b/d73/d4c/f84 [822725,36001] 0 2024-04-23T02:13:55.896 INFO:tasks.workunit.client.0.smithi071.stdout:1/177: link db/cc db/df/d13/d20/d21/c2e 0 2024-04-23T02:13:55.899 INFO:tasks.workunit.client.1.smithi192.stdout:8/435: write d1/d12/d23/d4b/d73/d45/d46/f53 [2335631,57212] 0 2024-04-23T02:13:55.900 INFO:tasks.workunit.client.0.smithi071.stdout:0/239: stat d0/d3/de/d10/d14/d32/d37/d26/l4a 0 2024-04-23T02:13:55.902 INFO:tasks.workunit.client.0.smithi071.stdout:0/240: write d0/d3/de/d10/d14/d32/d37/d26/f4b [4015375,104012] 0 2024-04-23T02:13:55.903 INFO:tasks.workunit.client.1.smithi192.stdout:8/436: dread d1/d12/d23/f37 [0,4194304] 0 2024-04-23T02:13:55.904 INFO:tasks.workunit.client.0.smithi071.stdout:0/241: chown d0/d3/de/d10/d14/d32/f45 145670795 1 2024-04-23T02:13:55.907 INFO:tasks.workunit.client.0.smithi071.stdout:2/256: fsync d0/d2/d26/df/f40 0 2024-04-23T02:13:55.908 INFO:tasks.workunit.client.1.smithi192.stdout:8/437: mknod d1/d12/d23/d2c/d8c/d95/c97 0 2024-04-23T02:13:55.908 INFO:tasks.workunit.client.1.smithi192.stdout:4/527: dwrite de/d15/d19/d3d/f57 [0,4194304] 0 2024-04-23T02:13:55.910 INFO:tasks.workunit.client.1.smithi192.stdout:8/438: readlink d1/d9/l3a 0 2024-04-23T02:13:55.912 INFO:tasks.workunit.client.0.smithi071.stdout:3/224: dread d7/d8/f11 [0,4194304] 0 2024-04-23T02:13:55.912 INFO:tasks.workunit.client.0.smithi071.stdout:2/257: truncate d0/d2/d26/f4c 1011790 0 2024-04-23T02:13:55.913 INFO:tasks.workunit.client.0.smithi071.stdout:0/242: dwrite d0/d3/de/d10/d14/d32/f41 [0,4194304] 0 2024-04-23T02:13:55.914 INFO:tasks.workunit.client.1.smithi192.stdout:4/528: link de/d15/c1f de/d15/d19/d3d/d49/d70/d73/cad 0 2024-04-23T02:13:55.915 INFO:tasks.workunit.client.0.smithi071.stdout:2/258: dread d0/d2/d26/f4c [0,4194304] 0 2024-04-23T02:13:55.917 INFO:tasks.workunit.client.1.smithi192.stdout:8/439: dread d1/d9/fa [0,4194304] 0 2024-04-23T02:13:55.919 INFO:tasks.workunit.client.0.smithi071.stdout:0/243: symlink d0/d3/de/d10/d14/d32/d44/l4c 0 2024-04-23T02:13:55.919 INFO:tasks.workunit.client.0.smithi071.stdout:2/259: creat d0/d2/d51/f53 x:0 0 0 2024-04-23T02:13:55.921 INFO:tasks.workunit.client.1.smithi192.stdout:4/529: dread de/f1b [0,4194304] 0 2024-04-23T02:13:55.921 INFO:tasks.workunit.client.0.smithi071.stdout:2/260: write d0/d2/d51/f53 [257707,37047] 0 2024-04-23T02:13:55.926 INFO:tasks.workunit.client.0.smithi071.stdout:1/178: rename db/df/d13/d20/d21 to db/df/d13/d2f 0 2024-04-23T02:13:55.927 INFO:tasks.workunit.client.1.smithi192.stdout:4/530: creat de/d15/d19/d3d/d49/d70/d73/fae x:0 0 0 2024-04-23T02:13:55.927 INFO:tasks.workunit.client.0.smithi071.stdout:1/179: write db/f22 [107085,32081] 0 2024-04-23T02:13:55.928 INFO:tasks.workunit.client.0.smithi071.stdout:2/261: creat d0/d2/d26/d14/d1f/d4f/f54 x:0 0 0 2024-04-23T02:13:55.929 INFO:tasks.workunit.client.0.smithi071.stdout:0/244: creat d0/d3/de/d10/d14/f4d x:0 0 0 2024-04-23T02:13:55.930 INFO:tasks.workunit.client.0.smithi071.stdout:0/245: dread - d0/d3/f3f zero size 2024-04-23T02:13:55.930 INFO:tasks.workunit.client.0.smithi071.stdout:1/180: mkdir db/df/d13/d2b/d30 0 2024-04-23T02:13:55.931 INFO:tasks.workunit.client.1.smithi192.stdout:8/440: dread d1/fe [0,4194304] 0 2024-04-23T02:13:55.932 INFO:tasks.workunit.client.0.smithi071.stdout:1/181: symlink db/df/d13/d2b/l31 0 2024-04-23T02:13:55.933 INFO:tasks.workunit.client.1.smithi192.stdout:4/531: creat de/d15/d19/d3d/d34/d7c/faf x:0 0 0 2024-04-23T02:13:55.933 INFO:tasks.workunit.client.0.smithi071.stdout:0/246: creat d0/d3/de/d10/d14/d32/d37/f4e x:0 0 0 2024-04-23T02:13:55.934 INFO:tasks.workunit.client.1.smithi192.stdout:4/532: creat de/d15/d19/d1a/fb0 x:0 0 0 2024-04-23T02:13:55.935 INFO:tasks.workunit.client.1.smithi192.stdout:3/297: dread d1/d3/d6/d7/f31 [4194304,4194304] 0 2024-04-23T02:13:55.935 INFO:tasks.workunit.client.0.smithi071.stdout:0/247: chown d0/d3/de/d10/d14/d32 23529 1 2024-04-23T02:13:55.937 INFO:tasks.workunit.client.0.smithi071.stdout:0/248: chown d0/d3/de/d10/d14/d32/f45 616 1 2024-04-23T02:13:55.937 INFO:tasks.workunit.client.1.smithi192.stdout:8/441: rename d1/d12/d23/d2c/f38 to d1/d12/d23/d4b/d73/d7e/d7b/d7c/f98 0 2024-04-23T02:13:55.938 INFO:tasks.workunit.client.0.smithi071.stdout:1/182: dwrite f3 [4194304,4194304] 0 2024-04-23T02:13:55.939 INFO:tasks.workunit.client.0.smithi071.stdout:0/249: chown d0/l34 175218066 1 2024-04-23T02:13:55.941 INFO:tasks.workunit.client.1.smithi192.stdout:8/442: dread d1/d9/fa [0,4194304] 0 2024-04-23T02:13:55.941 INFO:tasks.workunit.client.1.smithi192.stdout:3/298: rmdir d1/d3/d17/d1f/d33/d24 39 2024-04-23T02:13:55.941 INFO:tasks.workunit.client.0.smithi071.stdout:0/250: stat d0/d3/de/d10/d14/d32/d37/d26/f4b 0 2024-04-23T02:13:55.943 INFO:tasks.workunit.client.0.smithi071.stdout:0/251: chown d0/d3/de/d10/d14/d32/f41 0 1 2024-04-23T02:13:55.943 INFO:tasks.workunit.client.1.smithi192.stdout:4/533: creat de/d15/d19/d3d/d49/d70/d73/fb1 x:0 0 0 2024-04-23T02:13:55.944 INFO:tasks.workunit.client.0.smithi071.stdout:3/225: write d7/d22/f25 [4498558,111177] 0 2024-04-23T02:13:55.945 INFO:tasks.workunit.client.0.smithi071.stdout:1/183: dread f5 [0,4194304] 0 2024-04-23T02:13:55.947 INFO:tasks.workunit.client.1.smithi192.stdout:8/443: rename d1/d9/f26 to d1/d12/d23/d4b/d73/d7e/d7b/d7c/d8b/f99 0 2024-04-23T02:13:55.947 INFO:tasks.workunit.client.0.smithi071.stdout:6/250: dread d6/f18 [0,4194304] 0 2024-04-23T02:13:55.948 INFO:tasks.workunit.client.0.smithi071.stdout:1/184: chown db/d1a/l28 854083469 1 2024-04-23T02:13:55.948 INFO:tasks.workunit.client.0.smithi071.stdout:0/252: symlink d0/d3/de/d10/d14/d32/l4f 0 2024-04-23T02:13:55.950 INFO:tasks.workunit.client.0.smithi071.stdout:1/185: dread db/df/f2a [0,4194304] 0 2024-04-23T02:13:55.951 INFO:tasks.workunit.client.0.smithi071.stdout:6/251: mknod d6/d3e/c4c 0 2024-04-23T02:13:55.951 INFO:tasks.workunit.client.1.smithi192.stdout:4/534: rename de/d15/d19/d3d/d49/d26/d59/d6a to de/d15/d2a/db2 0 2024-04-23T02:13:55.952 INFO:tasks.workunit.client.0.smithi071.stdout:3/226: symlink d7/d10/d44/l4b 0 2024-04-23T02:13:55.953 INFO:tasks.workunit.client.0.smithi071.stdout:6/252: read d6/d22/f28 [86507,56477] 0 2024-04-23T02:13:55.953 INFO:tasks.workunit.client.0.smithi071.stdout:3/227: chown d7 3942306 1 2024-04-23T02:13:55.954 INFO:tasks.workunit.client.0.smithi071.stdout:1/186: creat db/df/d13/d20/f32 x:0 0 0 2024-04-23T02:13:55.955 INFO:tasks.workunit.client.0.smithi071.stdout:6/253: chown d6/f18 1862 1 2024-04-23T02:13:55.956 INFO:tasks.workunit.client.0.smithi071.stdout:6/254: write d6/f16 [1224034,17867] 0 2024-04-23T02:13:55.958 INFO:tasks.workunit.client.0.smithi071.stdout:0/253: dwrite d0/d3/de/f1d [4194304,4194304] 0 2024-04-23T02:13:55.959 INFO:tasks.workunit.client.0.smithi071.stdout:0/254: chown d0/f2 853333840 1 2024-04-23T02:13:55.960 INFO:tasks.workunit.client.1.smithi192.stdout:4/535: dwrite de/d15/d19/d3d/d49/d70/d73/f8e [0,4194304] 0 2024-04-23T02:13:55.961 INFO:tasks.workunit.client.0.smithi071.stdout:0/255: mknod d0/d3/de/d10/d14/d32/d44/d31/c50 0 2024-04-23T02:13:55.964 INFO:tasks.workunit.client.1.smithi192.stdout:4/536: creat de/d15/d19/d3d/d49/d70/d73/fb3 x:0 0 0 2024-04-23T02:13:55.964 INFO:tasks.workunit.client.0.smithi071.stdout:1/187: write db/df/f2a [989127,95135] 0 2024-04-23T02:13:55.970 INFO:tasks.workunit.client.0.smithi071.stdout:1/188: mknod db/df/d13/d20/c33 0 2024-04-23T02:13:55.973 INFO:tasks.workunit.client.0.smithi071.stdout:6/255: dwrite d6/f18 [0,4194304] 0 2024-04-23T02:13:55.974 INFO:tasks.workunit.client.1.smithi192.stdout:4/537: dwrite de/d15/d19/d3d/d34/d4a/f75 [0,4194304] 0 2024-04-23T02:13:55.977 INFO:tasks.workunit.client.1.smithi192.stdout:4/538: symlink de/d15/d19/d1a/d95/lb4 0 2024-04-23T02:13:55.978 INFO:tasks.workunit.client.1.smithi192.stdout:8/444: dread d1/d12/d23/d4b/d73/d4c/f6e [0,4194304] 0 2024-04-23T02:13:55.982 INFO:tasks.workunit.client.1.smithi192.stdout:8/445: creat d1/d12/d7d/f9a x:0 0 0 2024-04-23T02:13:55.983 INFO:tasks.workunit.client.0.smithi071.stdout:0/256: dwrite d0/d3/d1c/f22 [4194304,4194304] 0 2024-04-23T02:13:55.985 INFO:tasks.workunit.client.0.smithi071.stdout:1/189: mkdir db/d34 0 2024-04-23T02:13:55.987 INFO:tasks.workunit.client.1.smithi192.stdout:4/539: write f4 [5089578,93569] 0 2024-04-23T02:13:55.988 INFO:tasks.workunit.client.0.smithi071.stdout:3/228: dwrite d7/d8/f11 [0,4194304] 0 2024-04-23T02:13:55.989 INFO:tasks.workunit.client.1.smithi192.stdout:4/540: chown de/d15/d2a/d61/l78 5154 1 2024-04-23T02:13:55.991 INFO:tasks.workunit.client.0.smithi071.stdout:6/256: mkdir d6/d17/d4d 0 2024-04-23T02:13:55.992 INFO:tasks.workunit.client.1.smithi192.stdout:8/446: dwrite d1/f3 [0,4194304] 0 2024-04-23T02:13:55.994 INFO:tasks.workunit.client.0.smithi071.stdout:3/229: rename d7/d1d/d31 to d7/d28/d3b/d4c 0 2024-04-23T02:13:55.994 INFO:tasks.workunit.client.0.smithi071.stdout:1/190: dread fa [0,4194304] 0 2024-04-23T02:13:55.995 INFO:tasks.workunit.client.0.smithi071.stdout:6/257: mkdir d6/d1f/d4e 0 2024-04-23T02:13:55.995 INFO:tasks.workunit.client.0.smithi071.stdout:3/230: readlink d7/d27/d2c/l39 0 2024-04-23T02:13:55.997 INFO:tasks.workunit.client.0.smithi071.stdout:0/257: creat d0/d3/de/d10/d14/f51 x:0 0 0 2024-04-23T02:13:55.997 INFO:tasks.workunit.client.1.smithi192.stdout:8/447: creat d1/df/f9b x:0 0 0 2024-04-23T02:13:55.998 INFO:tasks.workunit.client.0.smithi071.stdout:1/191: creat db/df/d13/d2b/f35 x:0 0 0 2024-04-23T02:13:55.999 INFO:tasks.workunit.client.0.smithi071.stdout:6/258: mknod d6/d22/c4f 0 2024-04-23T02:13:56.000 INFO:tasks.workunit.client.0.smithi071.stdout:3/231: fdatasync d7/d10/f15 0 2024-04-23T02:13:56.002 INFO:tasks.workunit.client.0.smithi071.stdout:6/259: write d6/d17/f2c [2841169,108696] 0 2024-04-23T02:13:56.003 INFO:tasks.workunit.client.0.smithi071.stdout:3/232: chown d7/d8/c20 4944 1 2024-04-23T02:13:56.004 INFO:tasks.workunit.client.0.smithi071.stdout:1/192: write db/fd [1104345,21073] 0 2024-04-23T02:13:56.005 INFO:tasks.workunit.client.0.smithi071.stdout:0/258: dread d0/d3/f2d [0,4194304] 0 2024-04-23T02:13:56.006 INFO:tasks.workunit.client.1.smithi192.stdout:8/448: dwrite d1/d12/f24 [0,4194304] 0 2024-04-23T02:13:56.008 INFO:tasks.workunit.client.1.smithi192.stdout:8/449: read - d1/d9/d3e/d59/f75 zero size 2024-04-23T02:13:56.009 INFO:tasks.workunit.client.0.smithi071.stdout:0/259: write d0/d3/de/d10/d14/f1e [1127759,119979] 0 2024-04-23T02:13:56.009 INFO:tasks.workunit.client.0.smithi071.stdout:6/260: symlink d6/d1f/l50 0 2024-04-23T02:13:56.009 INFO:tasks.workunit.client.0.smithi071.stdout:3/233: mknod d7/d8/d42/d47/c4d 0 2024-04-23T02:13:56.010 INFO:tasks.workunit.client.0.smithi071.stdout:0/260: chown d0/d3/l3a 0 1 2024-04-23T02:13:56.010 INFO:tasks.workunit.client.0.smithi071.stdout:3/234: chown d7/fd 27222378 1 2024-04-23T02:13:56.012 INFO:tasks.workunit.client.0.smithi071.stdout:0/261: stat d0/d3/de/d10/d14/d32/d37/d26 0 2024-04-23T02:13:56.013 INFO:tasks.workunit.client.1.smithi192.stdout:4/541: dwrite de/d15/d19/d3d/d49/f50 [0,4194304] 0 2024-04-23T02:13:56.013 INFO:tasks.workunit.client.0.smithi071.stdout:1/193: rename f0 to db/df/d13/f36 0 2024-04-23T02:13:56.014 INFO:tasks.workunit.client.1.smithi192.stdout:8/450: mkdir d1/d12/d23/d4b/d73/d7e/d7b/d7c/d81/d9c 0 2024-04-23T02:13:56.015 INFO:tasks.workunit.client.0.smithi071.stdout:0/262: readlink d0/d3/de/d10/d14/d32/l4f 0 2024-04-23T02:13:56.018 INFO:tasks.workunit.client.1.smithi192.stdout:8/451: symlink d1/d12/d23/d4b/d73/l9d 0 2024-04-23T02:13:56.022 INFO:tasks.workunit.client.0.smithi071.stdout:3/235: dwrite d7/d10/d38/f41 [0,4194304] 0 2024-04-23T02:13:56.022 INFO:tasks.workunit.client.0.smithi071.stdout:3/236: chown d7/d10 64 1 2024-04-23T02:13:56.023 INFO:tasks.workunit.client.1.smithi192.stdout:8/452: write d1/d12/d23/d4b/d73/d45/f80 [1031166,75137] 0 2024-04-23T02:13:56.023 INFO:tasks.workunit.client.1.smithi192.stdout:8/453: stat d1/d12/d23/d2c/d4d/c4e 0 2024-04-23T02:13:56.024 INFO:tasks.workunit.client.1.smithi192.stdout:0/423: sync 2024-04-23T02:13:56.024 INFO:tasks.workunit.client.1.smithi192.stdout:5/363: sync 2024-04-23T02:13:56.024 INFO:tasks.workunit.client.1.smithi192.stdout:8/454: link d1/c33 d1/d12/d23/d2c/d4d/c9e 0 2024-04-23T02:13:56.025 INFO:tasks.workunit.client.0.smithi071.stdout:3/237: link d7/d10/f15 d7/d8/f4e 0 2024-04-23T02:13:56.027 INFO:tasks.workunit.client.1.smithi192.stdout:5/364: write d2/d32/f44 [2427710,130979] 0 2024-04-23T02:13:56.028 INFO:tasks.workunit.client.1.smithi192.stdout:0/424: rmdir d6/d4b 39 2024-04-23T02:13:56.029 INFO:tasks.workunit.client.1.smithi192.stdout:8/455: mknod d1/d12/d23/d4b/d73/d7e/d88/c9f 0 2024-04-23T02:13:56.032 INFO:tasks.workunit.client.1.smithi192.stdout:5/365: getdents d2/d6/d14 0 2024-04-23T02:13:56.033 INFO:tasks.workunit.client.1.smithi192.stdout:0/425: symlink d6/df/d35/l8a 0 2024-04-23T02:13:56.037 INFO:tasks.workunit.client.1.smithi192.stdout:0/426: mkdir d6/df/d35/d8b 0 2024-04-23T02:13:56.037 INFO:tasks.workunit.client.1.smithi192.stdout:0/427: stat d6/f18 0 2024-04-23T02:13:56.038 INFO:tasks.workunit.client.0.smithi071.stdout:1/194: write fa [881094,23922] 0 2024-04-23T02:13:56.040 INFO:tasks.workunit.client.1.smithi192.stdout:0/428: creat d6/df/d13/d6b/f8c x:0 0 0 2024-04-23T02:13:56.041 INFO:tasks.workunit.client.0.smithi071.stdout:3/238: dwrite d7/d27/d2c/f2e [0,4194304] 0 2024-04-23T02:13:56.042 INFO:tasks.workunit.client.1.smithi192.stdout:0/429: chown d6/df/d56/d85 4 1 2024-04-23T02:13:56.042 INFO:tasks.workunit.client.0.smithi071.stdout:1/195: chown db/df/f1c 261 1 2024-04-23T02:13:56.047 INFO:tasks.workunit.client.0.smithi071.stdout:6/261: dwrite d6/f8 [0,4194304] 0 2024-04-23T02:13:56.047 INFO:tasks.workunit.client.0.smithi071.stdout:1/196: mknod db/df/d13/d2b/c37 0 2024-04-23T02:13:56.048 INFO:tasks.workunit.client.0.smithi071.stdout:3/239: write d7/d10/f15 [347360,62790] 0 2024-04-23T02:13:56.049 INFO:tasks.workunit.client.0.smithi071.stdout:3/240: chown l4 1795716405 1 2024-04-23T02:13:56.051 INFO:tasks.workunit.client.0.smithi071.stdout:1/197: read f7 [2460216,104647] 0 2024-04-23T02:13:56.052 INFO:tasks.workunit.client.0.smithi071.stdout:6/262: creat d6/d1f/d4e/f51 x:0 0 0 2024-04-23T02:13:56.053 INFO:tasks.workunit.client.0.smithi071.stdout:3/241: rename d7/d10/d44 to d7/d8/d4f 0 2024-04-23T02:13:56.055 INFO:tasks.workunit.client.0.smithi071.stdout:1/198: link db/d11/l1e db/d34/l38 0 2024-04-23T02:13:56.057 INFO:tasks.workunit.client.0.smithi071.stdout:1/199: creat db/df/f39 x:0 0 0 2024-04-23T02:13:56.058 INFO:tasks.workunit.client.0.smithi071.stdout:1/200: chown db/df/d13/f1b 21486 1 2024-04-23T02:13:56.061 INFO:tasks.workunit.client.1.smithi192.stdout:5/366: dwrite d2/f5 [0,4194304] 0 2024-04-23T02:13:56.062 INFO:tasks.workunit.client.0.smithi071.stdout:1/201: getdents db 0 2024-04-23T02:13:56.062 INFO:tasks.workunit.client.0.smithi071.stdout:6/263: dread d6/f12 [0,4194304] 0 2024-04-23T02:13:56.064 INFO:tasks.workunit.client.0.smithi071.stdout:1/202: rmdir db/d1a 39 2024-04-23T02:13:56.065 INFO:tasks.workunit.client.0.smithi071.stdout:6/264: symlink d6/d1f/l52 0 2024-04-23T02:13:56.066 INFO:tasks.workunit.client.1.smithi192.stdout:0/430: dwrite d6/df/d47/f4c [0,4194304] 0 2024-04-23T02:13:56.066 INFO:tasks.workunit.client.0.smithi071.stdout:1/203: mkdir db/df/d3a 0 2024-04-23T02:13:56.067 INFO:tasks.workunit.client.0.smithi071.stdout:1/204: stat db/df/d13/d20 0 2024-04-23T02:13:56.067 INFO:tasks.workunit.client.0.smithi071.stdout:1/205: chown l8 934 1 2024-04-23T02:13:56.068 INFO:tasks.workunit.client.1.smithi192.stdout:8/456: dread d1/d12/d23/d4b/d73/d45/d46/f48 [0,4194304] 0 2024-04-23T02:13:56.068 INFO:tasks.workunit.client.1.smithi192.stdout:5/367: rename d2/db/d16/d1b/d3d/c6a to d2/db/d16/d1b/c73 0 2024-04-23T02:13:56.069 INFO:tasks.workunit.client.0.smithi071.stdout:1/206: dread - db/df/d13/d20/f32 zero size 2024-04-23T02:13:56.070 INFO:tasks.workunit.client.0.smithi071.stdout:6/265: getdents d6/d17 0 2024-04-23T02:13:56.071 INFO:tasks.workunit.client.0.smithi071.stdout:6/266: dread - d6/d1f/f32 zero size 2024-04-23T02:13:56.071 INFO:tasks.workunit.client.0.smithi071.stdout:1/207: fdatasync db/f22 0 2024-04-23T02:13:56.071 INFO:tasks.workunit.client.1.smithi192.stdout:0/431: unlink d6/df/d13/d25/f2b 0 2024-04-23T02:13:56.074 INFO:tasks.workunit.client.1.smithi192.stdout:8/457: creat d1/d12/d23/d4b/d73/d7e/d7b/d7c/d81/d9c/fa0 x:0 0 0 2024-04-23T02:13:56.075 INFO:tasks.workunit.client.0.smithi071.stdout:1/208: creat db/f3b x:0 0 0 2024-04-23T02:13:56.076 INFO:tasks.workunit.client.1.smithi192.stdout:0/432: write d6/df/f30 [2190402,115872] 0 2024-04-23T02:13:56.078 INFO:tasks.workunit.client.1.smithi192.stdout:8/458: mknod d1/d12/d23/d4b/d73/d7e/d7b/d7c/d81/ca1 0 2024-04-23T02:13:56.080 INFO:tasks.workunit.client.1.smithi192.stdout:5/368: link d2/db/d2b/l37 d2/d6/d2d/l74 0 2024-04-23T02:13:56.080 INFO:tasks.workunit.client.1.smithi192.stdout:8/459: write d1/d9/d3e/f55 [5086068,97990] 0 2024-04-23T02:13:56.083 INFO:tasks.workunit.client.0.smithi071.stdout:4/285: sync 2024-04-23T02:13:56.084 INFO:tasks.workunit.client.1.smithi192.stdout:8/460: creat d1/d9/d8f/d90/fa2 x:0 0 0 2024-04-23T02:13:56.084 INFO:tasks.workunit.client.0.smithi071.stdout:1/209: dwrite db/df/d13/f1b [0,4194304] 0 2024-04-23T02:13:56.085 INFO:tasks.workunit.client.0.smithi071.stdout:1/210: chown f2 5626700 1 2024-04-23T02:13:56.086 INFO:tasks.workunit.client.1.smithi192.stdout:5/369: link d2/d6/l1d d2/d6/d14/d60/l75 0 2024-04-23T02:13:56.088 INFO:tasks.workunit.client.0.smithi071.stdout:4/286: unlink d2/d15/d1d/d23/d4f/f50 0 2024-04-23T02:13:56.089 INFO:tasks.workunit.client.0.smithi071.stdout:4/287: chown d2/d15/d1d/f35 7118 1 2024-04-23T02:13:56.091 INFO:tasks.workunit.client.0.smithi071.stdout:1/211: rename f2 to db/df/d13/d2f/f3c 0 2024-04-23T02:13:56.091 INFO:tasks.workunit.client.1.smithi192.stdout:5/370: creat d2/db/d2b/d52/f76 x:0 0 0 2024-04-23T02:13:56.092 INFO:tasks.workunit.client.0.smithi071.stdout:1/212: write db/f22 [786583,122155] 0 2024-04-23T02:13:56.094 INFO:tasks.workunit.client.0.smithi071.stdout:1/213: symlink db/df/d13/d2b/l3d 0 2024-04-23T02:13:56.095 INFO:tasks.workunit.client.1.smithi192.stdout:8/461: dwrite d1/f10 [8388608,4194304] 0 2024-04-23T02:13:56.097 INFO:tasks.workunit.client.1.smithi192.stdout:5/371: write d2/db/d2b/d52/f76 [1006474,29780] 0 2024-04-23T02:13:56.097 INFO:tasks.workunit.client.0.smithi071.stdout:1/214: creat db/df/d13/d2f/f3e x:0 0 0 2024-04-23T02:13:56.098 INFO:tasks.workunit.client.0.smithi071.stdout:1/215: write db/fd [4299915,98531] 0 2024-04-23T02:13:56.099 INFO:tasks.workunit.client.1.smithi192.stdout:0/433: truncate d6/df/d56/d85/f19 789825 0 2024-04-23T02:13:56.100 INFO:tasks.workunit.client.0.smithi071.stdout:1/216: symlink db/df/d3a/l3f 0 2024-04-23T02:13:56.104 INFO:tasks.workunit.client.0.smithi071.stdout:6/267: dread d6/fc [0,4194304] 0 2024-04-23T02:13:56.105 INFO:tasks.workunit.client.1.smithi192.stdout:5/372: mknod d2/d2e/c77 0 2024-04-23T02:13:56.106 INFO:tasks.workunit.client.0.smithi071.stdout:4/288: dread d2/f43 [0,4194304] 0 2024-04-23T02:13:56.107 INFO:tasks.workunit.client.1.smithi192.stdout:8/462: write d1/d12/d23/d4b/d73/d4c/f6e [2051367,96251] 0 2024-04-23T02:13:56.107 INFO:tasks.workunit.client.0.smithi071.stdout:6/268: creat d6/d22/f53 x:0 0 0 2024-04-23T02:13:56.109 INFO:tasks.workunit.client.0.smithi071.stdout:6/269: creat d6/d1f/d4e/f54 x:0 0 0 2024-04-23T02:13:56.110 INFO:tasks.workunit.client.1.smithi192.stdout:8/463: mkdir d1/d12/d23/d4b/d73/d7e/d7b/d7c/da3 0 2024-04-23T02:13:56.111 INFO:tasks.workunit.client.0.smithi071.stdout:6/270: creat d6/f55 x:0 0 0 2024-04-23T02:13:56.112 INFO:tasks.workunit.client.0.smithi071.stdout:6/271: fsync d6/fc 0 2024-04-23T02:13:56.112 INFO:tasks.workunit.client.0.smithi071.stdout:6/272: chown d6/d1f/c3d 107995 1 2024-04-23T02:13:56.113 INFO:tasks.workunit.client.1.smithi192.stdout:5/373: dwrite d2/db/d16/d1b/d3d/d40/d49/f70 [0,4194304] 0 2024-04-23T02:13:56.113 INFO:tasks.workunit.client.1.smithi192.stdout:8/464: write d1/d9/d8f/d90/f93 [195157,34054] 0 2024-04-23T02:13:56.116 INFO:tasks.workunit.client.1.smithi192.stdout:5/374: dread - d2/d6/f3b zero size 2024-04-23T02:13:56.116 INFO:tasks.workunit.client.0.smithi071.stdout:6/273: dread f1 [0,4194304] 0 2024-04-23T02:13:56.119 INFO:tasks.workunit.client.1.smithi192.stdout:8/465: creat d1/d12/d23/d4b/d73/d7e/d88/fa4 x:0 0 0 2024-04-23T02:13:56.120 INFO:tasks.workunit.client.1.smithi192.stdout:8/466: dread - d1/d12/d23/d2c/d4d/f52 zero size 2024-04-23T02:13:56.122 INFO:tasks.workunit.client.1.smithi192.stdout:5/375: symlink d2/db/d16/l78 0 2024-04-23T02:13:56.123 INFO:tasks.workunit.client.1.smithi192.stdout:8/467: fsync d1/d12/f39 0 2024-04-23T02:13:56.125 INFO:tasks.workunit.client.1.smithi192.stdout:5/376: creat d2/db/d16/d1b/f79 x:0 0 0 2024-04-23T02:13:56.125 INFO:tasks.workunit.client.0.smithi071.stdout:4/289: dread d2/d15/f3a [0,4194304] 0 2024-04-23T02:13:56.126 INFO:tasks.workunit.client.1.smithi192.stdout:0/434: dread d6/df/d35/d49/f54 [0,4194304] 0 2024-04-23T02:13:56.128 INFO:tasks.workunit.client.1.smithi192.stdout:0/435: dread d6/f72 [0,4194304] 0 2024-04-23T02:13:56.130 INFO:tasks.workunit.client.0.smithi071.stdout:1/217: getdents db/df/d13/d2f 0 2024-04-23T02:13:56.130 INFO:tasks.workunit.client.1.smithi192.stdout:5/377: write d2/db/d16/d1b/f29 [6327660,70160] 0 2024-04-23T02:13:56.130 INFO:tasks.workunit.client.1.smithi192.stdout:8/468: write d1/fe [957986,126327] 0 2024-04-23T02:13:56.130 INFO:tasks.workunit.client.0.smithi071.stdout:1/218: dread - db/df/f39 zero size 2024-04-23T02:13:56.131 INFO:tasks.workunit.client.1.smithi192.stdout:5/378: fsync d2/d6/f3b 0 2024-04-23T02:13:56.132 INFO:tasks.workunit.client.0.smithi071.stdout:1/219: write db/df/f39 [607824,87106] 0 2024-04-23T02:13:56.134 INFO:tasks.workunit.client.1.smithi192.stdout:8/469: rmdir d1/d12/d23/d2c/d8c 39 2024-04-23T02:13:56.135 INFO:tasks.workunit.client.1.smithi192.stdout:0/436: mkdir d6/df/d35/d8b/d8d 0 2024-04-23T02:13:56.135 INFO:tasks.workunit.client.1.smithi192.stdout:5/379: creat d2/db/d16/d1b/f7a x:0 0 0 2024-04-23T02:13:56.136 INFO:tasks.workunit.client.1.smithi192.stdout:0/437: write d6/df/f14 [4229828,8918] 0 2024-04-23T02:13:56.137 INFO:tasks.workunit.client.0.smithi071.stdout:1/220: rename db/d11/l1e to db/df/d13/l40 0 2024-04-23T02:13:56.137 INFO:tasks.workunit.client.1.smithi192.stdout:0/438: chown d6/df/d13 1234 1 2024-04-23T02:13:56.138 INFO:tasks.workunit.client.1.smithi192.stdout:8/470: creat d1/d12/fa5 x:0 0 0 2024-04-23T02:13:56.139 INFO:tasks.workunit.client.1.smithi192.stdout:0/439: chown d6/df/d13/d6b/d73/f7e 258412 1 2024-04-23T02:13:56.140 INFO:tasks.workunit.client.1.smithi192.stdout:5/380: rename d2/d2e/c77 to d2/c7b 0 2024-04-23T02:13:56.140 INFO:tasks.workunit.client.0.smithi071.stdout:1/221: rmdir db/df/d13/d2b/d30 0 2024-04-23T02:13:56.140 INFO:tasks.workunit.client.0.smithi071.stdout:4/290: dread d2/d2a/d2d/d39/f46 [0,4194304] 0 2024-04-23T02:13:56.141 INFO:tasks.workunit.client.1.smithi192.stdout:8/471: creat d1/d12/d23/d4b/d73/d5d/fa6 x:0 0 0 2024-04-23T02:13:56.142 INFO:tasks.workunit.client.1.smithi192.stdout:0/440: mknod d6/d4b/c8e 0 2024-04-23T02:13:56.143 INFO:tasks.workunit.client.0.smithi071.stdout:4/291: creat d2/d2a/d2d/f5c x:0 0 0 2024-04-23T02:13:56.144 INFO:tasks.workunit.client.1.smithi192.stdout:5/381: mknod d2/db/d16/c7c 0 2024-04-23T02:13:56.145 INFO:tasks.workunit.client.1.smithi192.stdout:5/382: chown d2/d6/f3b 1 1 2024-04-23T02:13:56.146 INFO:tasks.workunit.client.0.smithi071.stdout:4/292: truncate d2/d15/f1e 452296 0 2024-04-23T02:13:56.148 INFO:tasks.workunit.client.0.smithi071.stdout:4/293: unlink d2/l1c 0 2024-04-23T02:13:56.149 INFO:tasks.workunit.client.1.smithi192.stdout:8/472: dwrite d1/f3 [0,4194304] 0 2024-04-23T02:13:56.150 INFO:tasks.workunit.client.0.smithi071.stdout:1/222: truncate f3 6798009 0 2024-04-23T02:13:56.150 INFO:tasks.workunit.client.0.smithi071.stdout:1/223: stat db/df/d13/d2f 0 2024-04-23T02:13:56.151 INFO:tasks.workunit.client.1.smithi192.stdout:0/441: link d6/df/d13/d25/d26/c3f d6/df/d56/d85/d45/c8f 0 2024-04-23T02:13:56.151 INFO:tasks.workunit.client.0.smithi071.stdout:4/294: write d2/f43 [2881961,36293] 0 2024-04-23T02:13:56.152 INFO:tasks.workunit.client.1.smithi192.stdout:5/383: dread d2/d6/f59 [0,4194304] 0 2024-04-23T02:13:56.152 INFO:tasks.workunit.client.0.smithi071.stdout:1/224: fdatasync db/df/d13/d2f/f3c 0 2024-04-23T02:13:56.156 INFO:tasks.workunit.client.0.smithi071.stdout:4/295: rename d2/d15/d1d/d23/d17/f24 to d2/d15/d1d/d23/f5d 0 2024-04-23T02:13:56.158 INFO:tasks.workunit.client.1.smithi192.stdout:0/442: link d6/df/d13/d25/d26/d67/f69 d6/df/d13/d25/d26/f90 0 2024-04-23T02:13:56.159 INFO:tasks.workunit.client.1.smithi192.stdout:0/443: dread - d6/df/f4a zero size 2024-04-23T02:13:56.159 INFO:tasks.workunit.client.1.smithi192.stdout:5/384: dwrite d2/d6/f4d [0,4194304] 0 2024-04-23T02:13:56.159 INFO:tasks.workunit.client.1.smithi192.stdout:8/473: rmdir d1/d12 39 2024-04-23T02:13:56.160 INFO:tasks.workunit.client.0.smithi071.stdout:4/296: unlink d2/d15/d1d/c40 0 2024-04-23T02:13:56.160 INFO:tasks.workunit.client.0.smithi071.stdout:1/225: dwrite db/df/d13/d2f/f3c [0,4194304] 0 2024-04-23T02:13:56.162 INFO:tasks.workunit.client.1.smithi192.stdout:5/385: truncate d2/f5 4984833 0 2024-04-23T02:13:56.162 INFO:tasks.workunit.client.0.smithi071.stdout:1/226: write f7 [3951871,108227] 0 2024-04-23T02:13:56.163 INFO:tasks.workunit.client.1.smithi192.stdout:5/386: fdatasync d2/db/d16/d1b/d3d/d40/d49/f70 0 2024-04-23T02:13:56.165 INFO:tasks.workunit.client.0.smithi071.stdout:4/297: mkdir d2/d15/d1d/d23/d58/d5e 0 2024-04-23T02:13:56.167 INFO:tasks.workunit.client.1.smithi192.stdout:8/474: read - d1/d12/d23/d4b/d73/d7e/d7b/d7c/d8b/f99 zero size 2024-04-23T02:13:56.168 INFO:tasks.workunit.client.1.smithi192.stdout:0/444: creat d6/df/d13/d25/f91 x:0 0 0 2024-04-23T02:13:56.168 INFO:tasks.workunit.client.0.smithi071.stdout:1/227: symlink db/df/d13/d2b/l41 0 2024-04-23T02:13:56.169 INFO:tasks.workunit.client.0.smithi071.stdout:4/298: symlink d2/d53/l5f 0 2024-04-23T02:13:56.169 INFO:tasks.workunit.client.1.smithi192.stdout:0/445: chown d6/df/d35/l8a 607 1 2024-04-23T02:13:56.169 INFO:tasks.workunit.client.0.smithi071.stdout:1/228: stat db/d11/c18 0 2024-04-23T02:13:56.170 INFO:tasks.workunit.client.0.smithi071.stdout:4/299: chown d2/d2a/d2d/f5c 0 1 2024-04-23T02:13:56.172 INFO:tasks.workunit.client.0.smithi071.stdout:4/300: creat d2/d15/d1d/d23/d10/f60 x:0 0 0 2024-04-23T02:13:56.172 INFO:tasks.workunit.client.1.smithi192.stdout:0/446: fdatasync d6/df/d13/d25/d26/f27 0 2024-04-23T02:13:56.173 INFO:tasks.workunit.client.0.smithi071.stdout:1/229: chown f3 0 1 2024-04-23T02:13:56.175 INFO:tasks.workunit.client.1.smithi192.stdout:8/475: dwrite d1/d12/d23/d4b/d73/d5d/fa6 [0,4194304] 0 2024-04-23T02:13:56.176 INFO:tasks.workunit.client.0.smithi071.stdout:1/230: creat db/df/d13/d2b/f42 x:0 0 0 2024-04-23T02:13:56.180 INFO:tasks.workunit.client.1.smithi192.stdout:8/476: symlink d1/d9/la7 0 2024-04-23T02:13:56.181 INFO:tasks.workunit.client.1.smithi192.stdout:8/477: stat d1/d12/d23/d4b/d73/d4c 0 2024-04-23T02:13:56.182 INFO:tasks.workunit.client.0.smithi071.stdout:1/231: dwrite fa [0,4194304] 0 2024-04-23T02:13:56.183 INFO:tasks.workunit.client.0.smithi071.stdout:1/232: chown db/df/d13/d2b/c37 125033 1 2024-04-23T02:13:56.185 INFO:tasks.workunit.client.1.smithi192.stdout:8/478: creat d1/df/fa8 x:0 0 0 2024-04-23T02:13:56.186 INFO:tasks.workunit.client.1.smithi192.stdout:8/479: chown d1/d12/d23/d4b/d73/d7e/d7b/d7c/d81 359609 1 2024-04-23T02:13:56.189 INFO:tasks.workunit.client.1.smithi192.stdout:8/480: rename d1/d12/d23/d4b/d73/d7e/d7b to d1/d12/d23/d4b/d73/d7e/d88/da9 0 2024-04-23T02:13:56.191 INFO:tasks.workunit.client.0.smithi071.stdout:1/233: dwrite db/df/d13/d2f/f3c [0,4194304] 0 2024-04-23T02:13:56.191 INFO:tasks.workunit.client.1.smithi192.stdout:8/481: stat d1/d12/d23/d2c/d8c 0 2024-04-23T02:13:56.193 INFO:tasks.workunit.client.1.smithi192.stdout:0/447: write d6/df/d56/d85/f19 [1576747,52588] 0 2024-04-23T02:13:56.193 INFO:tasks.workunit.client.0.smithi071.stdout:1/234: creat db/df/d13/d2b/f43 x:0 0 0 2024-04-23T02:13:56.197 INFO:tasks.workunit.client.1.smithi192.stdout:0/448: creat d6/df/d47/f92 x:0 0 0 2024-04-23T02:13:56.198 INFO:tasks.workunit.client.1.smithi192.stdout:5/387: dread d2/db/d16/d1b/f29 [0,4194304] 0 2024-04-23T02:13:56.199 INFO:tasks.workunit.client.1.smithi192.stdout:8/482: truncate d1/d12/d23/f91 2368149 0 2024-04-23T02:13:56.200 INFO:tasks.workunit.client.0.smithi071.stdout:1/235: creat db/df/d3a/f44 x:0 0 0 2024-04-23T02:13:56.201 INFO:tasks.workunit.client.0.smithi071.stdout:4/301: rmdir d2/d15/d1d/d23/d17 39 2024-04-23T02:13:56.202 INFO:tasks.workunit.client.0.smithi071.stdout:1/236: write db/f25 [4032129,20760] 0 2024-04-23T02:13:56.202 INFO:tasks.workunit.client.1.smithi192.stdout:8/483: chown d1/d12/d23/d4b/d73/d4c/d82 1345003 1 2024-04-23T02:13:56.202 INFO:tasks.workunit.client.0.smithi071.stdout:1/237: stat db/df/f2a 0 2024-04-23T02:13:56.204 INFO:tasks.workunit.client.0.smithi071.stdout:1/238: creat db/df/f45 x:0 0 0 2024-04-23T02:13:56.205 INFO:tasks.workunit.client.1.smithi192.stdout:5/388: dread d2/db/f13 [0,4194304] 0 2024-04-23T02:13:56.206 INFO:tasks.workunit.client.1.smithi192.stdout:8/484: mkdir d1/d9/d3e/d59/d8a/daa 0 2024-04-23T02:13:56.206 INFO:tasks.workunit.client.1.smithi192.stdout:0/449: dwrite d6/df/d13/d25/f91 [0,4194304] 0 2024-04-23T02:13:56.206 INFO:tasks.workunit.client.0.smithi071.stdout:4/302: rename d2/d15/d1d/d2c/f1b to d2/d15/d1d/f61 0 2024-04-23T02:13:56.208 INFO:tasks.workunit.client.1.smithi192.stdout:8/485: fsync d1/d12/d7d/f9a 0 2024-04-23T02:13:56.208 INFO:tasks.workunit.client.1.smithi192.stdout:5/389: chown d2/d6/d2d/d41 184244675 1 2024-04-23T02:13:56.209 INFO:tasks.workunit.client.0.smithi071.stdout:4/303: fdatasync d2/d15/d1d/d23/d17/f41 0 2024-04-23T02:13:56.210 INFO:tasks.workunit.client.1.smithi192.stdout:5/390: read - d2/d2e/f64 zero size 2024-04-23T02:13:56.212 INFO:tasks.workunit.client.0.smithi071.stdout:1/239: dwrite db/fd [0,4194304] 0 2024-04-23T02:13:56.212 INFO:tasks.workunit.client.1.smithi192.stdout:0/450: mkdir d6/df/d13/d25/d3a/d93 0 2024-04-23T02:13:56.213 INFO:tasks.workunit.client.1.smithi192.stdout:5/391: write d2/d6/d2d/d41/f38 [698154,71135] 0 2024-04-23T02:13:56.214 INFO:tasks.workunit.client.1.smithi192.stdout:5/392: chown d2/d6/d2d/d41/d43 324135355 1 2024-04-23T02:13:56.216 INFO:tasks.workunit.client.1.smithi192.stdout:8/486: rename d1/d12/d23/d4b/d73/f67 to d1/d12/d23/d4b/d73/fab 0 2024-04-23T02:13:56.219 INFO:tasks.workunit.client.1.smithi192.stdout:0/451: mkdir d6/df/d13/d25/d94 0 2024-04-23T02:13:56.222 INFO:tasks.workunit.client.1.smithi192.stdout:5/393: read d2/db/f18 [1981093,25727] 0 2024-04-23T02:13:56.226 INFO:tasks.workunit.client.1.smithi192.stdout:0/452: mkdir d6/df/d13/d25/d95 0 2024-04-23T02:13:56.226 INFO:tasks.workunit.client.1.smithi192.stdout:5/394: chown d2/d2e/d4e/c69 217 1 2024-04-23T02:13:56.227 INFO:tasks.workunit.client.1.smithi192.stdout:5/395: mknod d2/db/d16/d1b/d3d/c7d 0 2024-04-23T02:13:56.227 INFO:tasks.workunit.client.0.smithi071.stdout:1/240: dread fa [0,4194304] 0 2024-04-23T02:13:56.227 INFO:tasks.workunit.client.0.smithi071.stdout:1/241: stat db/df/d13/d2b/f35 0 2024-04-23T02:13:56.227 INFO:tasks.workunit.client.0.smithi071.stdout:1/242: creat db/df/d13/d20/f46 x:0 0 0 2024-04-23T02:13:56.227 INFO:tasks.workunit.client.1.smithi192.stdout:5/396: chown d2/db/d2b/d52/f76 198569194 1 2024-04-23T02:13:56.228 INFO:tasks.workunit.client.0.smithi071.stdout:4/304: dwrite d2/d2a/f36 [0,4194304] 0 2024-04-23T02:13:56.230 INFO:tasks.workunit.client.0.smithi071.stdout:1/243: creat db/df/d3a/f47 x:0 0 0 2024-04-23T02:13:56.231 INFO:tasks.workunit.client.1.smithi192.stdout:5/397: creat d2/f7e x:0 0 0 2024-04-23T02:13:56.231 INFO:tasks.workunit.client.0.smithi071.stdout:4/305: symlink d2/d15/d1d/d23/d17/l62 0 2024-04-23T02:13:56.232 INFO:tasks.workunit.client.0.smithi071.stdout:1/244: mkdir db/df/d13/d2f/d48 0 2024-04-23T02:13:56.233 INFO:tasks.workunit.client.0.smithi071.stdout:4/306: creat d2/d2a/d2d/d39/f63 x:0 0 0 2024-04-23T02:13:56.235 INFO:tasks.workunit.client.0.smithi071.stdout:1/245: creat db/df/d13/d2f/d48/f49 x:0 0 0 2024-04-23T02:13:56.235 INFO:tasks.workunit.client.1.smithi192.stdout:8/487: fsync d1/d12/d23/f91 0 2024-04-23T02:13:56.236 INFO:tasks.workunit.client.0.smithi071.stdout:1/246: write db/df/f39 [473796,127398] 0 2024-04-23T02:13:56.237 INFO:tasks.workunit.client.1.smithi192.stdout:8/488: fsync d1/df/f9b 0 2024-04-23T02:13:56.238 INFO:tasks.workunit.client.1.smithi192.stdout:5/398: dread d2/f2c [0,4194304] 0 2024-04-23T02:13:56.238 INFO:tasks.workunit.client.1.smithi192.stdout:2/291: sync 2024-04-23T02:13:56.238 INFO:tasks.workunit.client.1.smithi192.stdout:6/452: sync 2024-04-23T02:13:56.241 INFO:tasks.workunit.client.1.smithi192.stdout:0/453: write d6/f18 [1681236,1801] 0 2024-04-23T02:13:56.243 INFO:tasks.workunit.client.1.smithi192.stdout:0/454: dread - d6/df/d13/d6b/d73/f7e zero size 2024-04-23T02:13:56.243 INFO:tasks.workunit.client.1.smithi192.stdout:2/292: mkdir d8/d26/d29/d43/d5d 0 2024-04-23T02:13:56.243 INFO:tasks.workunit.client.0.smithi071.stdout:4/307: dwrite d2/d15/d1d/d23/fc [0,4194304] 0 2024-04-23T02:13:56.243 INFO:tasks.workunit.client.0.smithi071.stdout:1/247: creat db/df/d13/d20/f4a x:0 0 0 2024-04-23T02:13:56.244 INFO:tasks.workunit.client.1.smithi192.stdout:6/453: getdents d3/d15/d1b/d21/d52/d7f 0 2024-04-23T02:13:56.244 INFO:tasks.workunit.client.1.smithi192.stdout:2/293: dread - d8/d26/d29/f41 zero size 2024-04-23T02:13:56.247 INFO:tasks.workunit.client.0.smithi071.stdout:4/308: truncate d2/d15/f3a 1219079 0 2024-04-23T02:13:56.252 INFO:tasks.workunit.client.1.smithi192.stdout:6/454: creat d3/d18/f94 x:0 0 0 2024-04-23T02:13:56.252 INFO:tasks.workunit.client.0.smithi071.stdout:4/309: rename d2/d15/d1d/d23/fc to d2/f64 0 2024-04-23T02:13:56.253 INFO:tasks.workunit.client.1.smithi192.stdout:2/294: mkdir d8/d26/d29/d43/d50/d5e 0 2024-04-23T02:13:56.254 INFO:tasks.workunit.client.1.smithi192.stdout:8/489: write d1/df/f50 [4479511,74633] 0 2024-04-23T02:13:56.254 INFO:tasks.workunit.client.1.smithi192.stdout:5/399: dwrite d2/db/d16/d1b/f29 [4194304,4194304] 0 2024-04-23T02:13:56.260 INFO:tasks.workunit.client.1.smithi192.stdout:0/455: dwrite d6/df/d47/d55/f70 [0,4194304] 0 2024-04-23T02:13:56.263 INFO:tasks.workunit.client.1.smithi192.stdout:6/455: mknod d3/d15/c95 0 2024-04-23T02:13:56.263 INFO:tasks.workunit.client.1.smithi192.stdout:8/490: write d1/d12/d23/d4b/d73/d7e/d88/fa4 [750336,106858] 0 2024-04-23T02:13:56.267 INFO:tasks.workunit.client.1.smithi192.stdout:5/400: getdents d2/d2e 0 2024-04-23T02:13:56.270 INFO:tasks.workunit.client.1.smithi192.stdout:0/456: truncate d6/df/d13/d6b/f8c 490915 0 2024-04-23T02:13:56.271 INFO:tasks.workunit.client.1.smithi192.stdout:2/295: dwrite d8/d14/f4c [0,4194304] 0 2024-04-23T02:13:56.271 INFO:tasks.workunit.client.1.smithi192.stdout:8/491: mknod d1/d12/d23/d4b/d73/d4c/cac 0 2024-04-23T02:13:56.273 INFO:tasks.workunit.client.1.smithi192.stdout:5/401: rmdir d2/db/d16/d1b/d3d/d40 39 2024-04-23T02:13:56.276 INFO:tasks.workunit.client.1.smithi192.stdout:2/296: write d8/d25/f3f [914317,3451] 0 2024-04-23T02:13:56.277 INFO:tasks.workunit.client.1.smithi192.stdout:2/297: readlink d8/l2b 0 2024-04-23T02:13:56.277 INFO:tasks.workunit.client.1.smithi192.stdout:2/298: write d8/d25/f5c [438867,120032] 0 2024-04-23T02:13:56.280 INFO:tasks.workunit.client.1.smithi192.stdout:6/456: getdents d3/d15 0 2024-04-23T02:13:56.282 INFO:tasks.workunit.client.1.smithi192.stdout:0/457: symlink d6/df/d13/d25/d3a/d93/l96 0 2024-04-23T02:13:56.283 INFO:tasks.workunit.client.1.smithi192.stdout:2/299: creat d8/d26/d29/d43/d50/d5e/f5f x:0 0 0 2024-04-23T02:13:56.284 INFO:tasks.workunit.client.1.smithi192.stdout:8/492: link d1/d9/f3f d1/d12/d68/fad 0 2024-04-23T02:13:56.284 INFO:tasks.workunit.client.1.smithi192.stdout:0/458: write d6/df/f83 [301796,75607] 0 2024-04-23T02:13:56.285 INFO:tasks.workunit.client.1.smithi192.stdout:5/402: creat d2/db/d2b/f7f x:0 0 0 2024-04-23T02:13:56.285 INFO:tasks.workunit.client.1.smithi192.stdout:2/300: dread - d8/d26/d27/f45 zero size 2024-04-23T02:13:56.287 INFO:tasks.workunit.client.1.smithi192.stdout:2/301: fsync d8/f18 0 2024-04-23T02:13:56.288 INFO:tasks.workunit.client.1.smithi192.stdout:0/459: truncate d6/df/d56/d85/f2e 2799235 0 2024-04-23T02:13:56.289 INFO:tasks.workunit.client.1.smithi192.stdout:2/302: stat d8/d26/c37 0 2024-04-23T02:13:56.291 INFO:tasks.workunit.client.1.smithi192.stdout:0/460: mknod d6/df/d35/c97 0 2024-04-23T02:13:56.293 INFO:tasks.workunit.client.1.smithi192.stdout:2/303: symlink d8/d25/l60 0 2024-04-23T02:13:56.295 INFO:tasks.workunit.client.1.smithi192.stdout:8/493: dwrite d1/df/fa8 [0,4194304] 0 2024-04-23T02:13:56.295 INFO:tasks.workunit.client.1.smithi192.stdout:0/461: readlink d6/df/d13/d25/d26/l34 0 2024-04-23T02:13:56.303 INFO:tasks.workunit.client.1.smithi192.stdout:8/494: rmdir d1/d9/d3e/d59 39 2024-04-23T02:13:56.304 INFO:tasks.workunit.client.1.smithi192.stdout:5/403: truncate d2/db/d16/d1b/f29 5503907 0 2024-04-23T02:13:56.305 INFO:tasks.workunit.client.1.smithi192.stdout:2/304: dread d8/d26/d29/f58 [0,4194304] 0 2024-04-23T02:13:56.307 INFO:tasks.workunit.client.1.smithi192.stdout:0/462: dwrite d6/df/d13/d25/d3a/f58 [4194304,4194304] 0 2024-04-23T02:13:56.307 INFO:tasks.workunit.client.1.smithi192.stdout:8/495: unlink d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/fa0 0 2024-04-23T02:13:56.309 INFO:tasks.workunit.client.1.smithi192.stdout:5/404: mknod d2/d6/c80 0 2024-04-23T02:13:56.309 INFO:tasks.workunit.client.1.smithi192.stdout:5/405: stat d2/d32/f35 0 2024-04-23T02:13:56.311 INFO:tasks.workunit.client.1.smithi192.stdout:2/305: creat d8/d26/f61 x:0 0 0 2024-04-23T02:13:56.311 INFO:tasks.workunit.client.1.smithi192.stdout:5/406: chown d2/db/d16/d1b/f61 1579275922 1 2024-04-23T02:13:56.311 INFO:tasks.workunit.client.1.smithi192.stdout:0/463: chown d6/df/d13/d25/d26/c3f 543 1 2024-04-23T02:13:56.312 INFO:tasks.workunit.client.1.smithi192.stdout:8/496: mkdir d1/d12/d23/d4b/d73/d45/dae 0 2024-04-23T02:13:56.313 INFO:tasks.workunit.client.0.smithi071.stdout:5/386: sync 2024-04-23T02:13:56.314 INFO:tasks.workunit.client.1.smithi192.stdout:0/464: symlink d6/df/d56/d85/l98 0 2024-04-23T02:13:56.314 INFO:tasks.workunit.client.1.smithi192.stdout:8/497: rmdir d1/d12 39 2024-04-23T02:13:56.315 INFO:tasks.workunit.client.1.smithi192.stdout:2/306: getdents d8/d26/d27/d57 0 2024-04-23T02:13:56.317 INFO:tasks.workunit.client.1.smithi192.stdout:5/407: creat d2/db/d16/d1b/d3d/d40/f81 x:0 0 0 2024-04-23T02:13:56.318 INFO:tasks.workunit.client.1.smithi192.stdout:0/465: mknod d6/df/d56/c99 0 2024-04-23T02:13:56.318 INFO:tasks.workunit.client.1.smithi192.stdout:5/408: dread d2/f27 [0,4194304] 0 2024-04-23T02:13:56.321 INFO:tasks.workunit.client.0.smithi071.stdout:4/310: mkdir d2/d2a/d2d/d39/d42/d49/d65 0 2024-04-23T02:13:56.323 INFO:tasks.workunit.client.1.smithi192.stdout:2/307: link d8/d14/c1f d8/c62 0 2024-04-23T02:13:56.325 INFO:tasks.workunit.client.0.smithi071.stdout:5/387: creat d3/d1d/d43/d4b/d54/f94 x:0 0 0 2024-04-23T02:13:56.325 INFO:tasks.workunit.client.0.smithi071.stdout:4/311: dread d2/d2a/d2d/f54 [0,4194304] 0 2024-04-23T02:13:56.326 INFO:tasks.workunit.client.1.smithi192.stdout:5/409: creat d2/db/d2b/f82 x:0 0 0 2024-04-23T02:13:56.327 INFO:tasks.workunit.client.1.smithi192.stdout:2/308: fsync d8/f18 0 2024-04-23T02:13:56.327 INFO:tasks.workunit.client.0.smithi071.stdout:1/248: dwrite f3 [0,4194304] 0 2024-04-23T02:13:56.329 INFO:tasks.workunit.client.1.smithi192.stdout:2/309: chown d8/d14/d51 7 1 2024-04-23T02:13:56.330 INFO:tasks.workunit.client.0.smithi071.stdout:4/312: fdatasync d2/d15/d1d/f35 0 2024-04-23T02:13:56.332 INFO:tasks.workunit.client.1.smithi192.stdout:0/466: dwrite d6/df/d47/f61 [0,4194304] 0 2024-04-23T02:13:56.332 INFO:tasks.workunit.client.0.smithi071.stdout:1/249: unlink l8 0 2024-04-23T02:13:56.333 INFO:tasks.workunit.client.1.smithi192.stdout:8/498: dwrite d1/d12/d23/d4b/d73/f65 [0,4194304] 0 2024-04-23T02:13:56.334 INFO:tasks.workunit.client.0.smithi071.stdout:4/313: creat d2/d2a/d2d/f66 x:0 0 0 2024-04-23T02:13:56.334 INFO:tasks.workunit.client.0.smithi071.stdout:5/388: dwrite d3/d34/f6a [0,4194304] 0 2024-04-23T02:13:56.335 INFO:tasks.workunit.client.0.smithi071.stdout:4/314: chown d2/d15/d3e/f4d 11916 1 2024-04-23T02:13:56.337 INFO:tasks.workunit.client.0.smithi071.stdout:5/389: creat d3/d1d/d43/f95 x:0 0 0 2024-04-23T02:13:56.337 INFO:tasks.workunit.client.1.smithi192.stdout:5/410: creat d2/db/d16/d1b/d3d/d40/d49/f83 x:0 0 0 2024-04-23T02:13:56.338 INFO:tasks.workunit.client.0.smithi071.stdout:1/250: creat db/f4b x:0 0 0 2024-04-23T02:13:56.339 INFO:tasks.workunit.client.0.smithi071.stdout:4/315: readlink d2/d15/d1d/d23/ld 0 2024-04-23T02:13:56.340 INFO:tasks.workunit.client.0.smithi071.stdout:5/390: fsync d3/d34/d6b/d81/f58 0 2024-04-23T02:13:56.344 INFO:tasks.workunit.client.1.smithi192.stdout:2/310: dread d8/f18 [0,4194304] 0 2024-04-23T02:13:56.347 INFO:tasks.workunit.client.0.smithi071.stdout:4/316: rename d2/d15 to d2/d2a/d2d/d39/d42/d49/d67 0 2024-04-23T02:13:56.347 INFO:tasks.workunit.client.1.smithi192.stdout:2/311: chown d8/d26/f3c 53002976 1 2024-04-23T02:13:56.350 INFO:tasks.workunit.client.1.smithi192.stdout:5/411: link d2/f27 d2/db/d16/d48/d4f/f84 0 2024-04-23T02:13:56.351 INFO:tasks.workunit.client.0.smithi071.stdout:1/251: link db/l23 db/l4c 0 2024-04-23T02:13:56.351 INFO:tasks.workunit.client.1.smithi192.stdout:8/499: getdents d1/d12/d23/d4b/d73/d5d 0 2024-04-23T02:13:56.352 INFO:tasks.workunit.client.0.smithi071.stdout:5/391: rename d3/f5c to d3/da/d1a/d1c/d46/d87/f96 0 2024-04-23T02:13:56.352 INFO:tasks.workunit.client.1.smithi192.stdout:5/412: chown d2/db/d2b/f3a 1429683 1 2024-04-23T02:13:56.355 INFO:tasks.workunit.client.1.smithi192.stdout:0/467: dwrite d6/d4b/f79 [0,4194304] 0 2024-04-23T02:13:56.356 INFO:tasks.workunit.client.0.smithi071.stdout:1/252: rmdir db/df/d13/d20 39 2024-04-23T02:13:56.358 INFO:tasks.workunit.client.0.smithi071.stdout:5/392: mknod d3/d1d/d43/d4b/d54/c97 0 2024-04-23T02:13:56.360 INFO:tasks.workunit.client.0.smithi071.stdout:4/317: getdents d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c 0 2024-04-23T02:13:56.361 INFO:tasks.workunit.client.1.smithi192.stdout:8/500: creat d1/d12/d23/d2c/d8c/faf x:0 0 0 2024-04-23T02:13:56.362 INFO:tasks.workunit.client.1.smithi192.stdout:2/312: getdents d8/d26/d27/d53 0 2024-04-23T02:13:56.362 INFO:tasks.workunit.client.1.smithi192.stdout:5/413: mknod d2/db/d16/d1b/c85 0 2024-04-23T02:13:56.362 INFO:tasks.workunit.client.0.smithi071.stdout:1/253: getdents db/d34 0 2024-04-23T02:13:56.363 INFO:tasks.workunit.client.0.smithi071.stdout:5/393: creat d3/df/f98 x:0 0 0 2024-04-23T02:13:56.364 INFO:tasks.workunit.client.1.smithi192.stdout:0/468: fsync d6/df/f23 0 2024-04-23T02:13:56.364 INFO:tasks.workunit.client.1.smithi192.stdout:8/501: fsync d1/d9/f11 0 2024-04-23T02:13:56.364 INFO:tasks.workunit.client.1.smithi192.stdout:2/313: read d8/d25/f5c [71049,57057] 0 2024-04-23T02:13:56.366 INFO:tasks.workunit.client.1.smithi192.stdout:0/469: symlink d6/df/d56/l9a 0 2024-04-23T02:13:56.367 INFO:tasks.workunit.client.0.smithi071.stdout:4/318: symlink d2/d2a/d2d/d39/d42/d49/d67/l68 0 2024-04-23T02:13:56.368 INFO:tasks.workunit.client.1.smithi192.stdout:5/414: link d2/db/f18 d2/d6/d14/f86 0 2024-04-23T02:13:56.370 INFO:tasks.workunit.client.1.smithi192.stdout:2/314: rename d8/d25/l60 to d8/d26/d27/d53/l63 0 2024-04-23T02:13:56.370 INFO:tasks.workunit.client.0.smithi071.stdout:1/254: dwrite db/df/f1c [0,4194304] 0 2024-04-23T02:13:56.371 INFO:tasks.workunit.client.1.smithi192.stdout:2/315: chown d8/d14/c42 3742 1 2024-04-23T02:13:56.371 INFO:tasks.workunit.client.1.smithi192.stdout:0/470: creat d6/df/d35/d8b/f9b x:0 0 0 2024-04-23T02:13:56.372 INFO:tasks.workunit.client.1.smithi192.stdout:5/415: symlink d2/d6/d2d/l87 0 2024-04-23T02:13:56.373 INFO:tasks.workunit.client.1.smithi192.stdout:0/471: readlink d6/df/d13/d25/d26/l5d 0 2024-04-23T02:13:56.373 INFO:tasks.workunit.client.0.smithi071.stdout:1/255: rename db/df/d3a/l3f to db/d11/l4d 0 2024-04-23T02:13:56.374 INFO:tasks.workunit.client.0.smithi071.stdout:1/256: mknod db/d34/c4e 0 2024-04-23T02:13:56.375 INFO:tasks.workunit.client.1.smithi192.stdout:2/316: write d8/f18 [638462,123046] 0 2024-04-23T02:13:56.375 INFO:tasks.workunit.client.0.smithi071.stdout:1/257: fdatasync db/df/d13/d2f/f3c 0 2024-04-23T02:13:56.375 INFO:tasks.workunit.client.1.smithi192.stdout:5/416: readlink d2/la 0 2024-04-23T02:13:56.382 INFO:tasks.workunit.client.0.smithi071.stdout:5/394: write d3/da/d1a/f1b [1556211,33425] 0 2024-04-23T02:13:56.383 INFO:tasks.workunit.client.1.smithi192.stdout:0/472: mknod d6/df/d35/d8b/d8d/c9c 0 2024-04-23T02:13:56.383 INFO:tasks.workunit.client.1.smithi192.stdout:5/417: mkdir d2/d2e/d88 0 2024-04-23T02:13:56.384 INFO:tasks.workunit.client.0.smithi071.stdout:1/258: dwrite db/df/d13/d2b/f35 [0,4194304] 0 2024-04-23T02:13:56.384 INFO:tasks.workunit.client.1.smithi192.stdout:0/473: fsync d6/df/f83 0 2024-04-23T02:13:56.386 INFO:tasks.workunit.client.1.smithi192.stdout:5/418: read - d2/d6/f3b zero size 2024-04-23T02:13:56.386 INFO:tasks.workunit.client.1.smithi192.stdout:2/317: dwrite d8/d14/f4c [0,4194304] 0 2024-04-23T02:13:56.387 INFO:tasks.workunit.client.0.smithi071.stdout:1/259: dread - db/df/d13/d20/f32 zero size 2024-04-23T02:13:56.388 INFO:tasks.workunit.client.1.smithi192.stdout:5/419: write d2/db/d2b/f7f [968863,58907] 0 2024-04-23T02:13:56.394 INFO:tasks.workunit.client.0.smithi071.stdout:1/260: dwrite db/df/d13/d2b/f42 [0,4194304] 0 2024-04-23T02:13:56.396 INFO:tasks.workunit.client.0.smithi071.stdout:1/261: symlink db/df/d13/d2b/l4f 0 2024-04-23T02:13:56.397 INFO:tasks.workunit.client.1.smithi192.stdout:2/318: read d8/d26/f3c [620295,82309] 0 2024-04-23T02:13:56.399 INFO:tasks.workunit.client.0.smithi071.stdout:5/395: read d3/da/d1a/d1c/f31 [112204,100681] 0 2024-04-23T02:13:56.400 INFO:tasks.workunit.client.1.smithi192.stdout:2/319: fdatasync d8/f1d 0 2024-04-23T02:13:56.402 INFO:tasks.workunit.client.1.smithi192.stdout:2/320: chown d8/d26/d27/l4e 540 1 2024-04-23T02:13:56.403 INFO:tasks.workunit.client.0.smithi071.stdout:5/396: mknod d3/da/d1a/d1c/c99 0 2024-04-23T02:13:56.405 INFO:tasks.workunit.client.1.smithi192.stdout:2/321: creat d8/d26/d27/d53/f64 x:0 0 0 2024-04-23T02:13:56.405 INFO:tasks.workunit.client.1.smithi192.stdout:2/322: chown d8/d14/l22 2 1 2024-04-23T02:13:56.409 INFO:tasks.workunit.client.0.smithi071.stdout:5/397: dwrite d3/d1d/d43/d4b/f53 [0,4194304] 0 2024-04-23T02:13:56.409 INFO:tasks.workunit.client.0.smithi071.stdout:5/398: fdatasync d3/d34/f8b 0 2024-04-23T02:13:56.413 INFO:tasks.workunit.client.1.smithi192.stdout:0/474: truncate d6/d4b/f79 409137 0 2024-04-23T02:13:56.414 INFO:tasks.workunit.client.1.smithi192.stdout:0/475: readlink d6/df/l21 0 2024-04-23T02:13:56.416 INFO:tasks.workunit.client.0.smithi071.stdout:5/399: creat d3/da/d1a/d6d/f9a x:0 0 0 2024-04-23T02:13:56.416 INFO:tasks.workunit.client.1.smithi192.stdout:0/476: unlink d6/df/d35/d49/f7a 0 2024-04-23T02:13:56.417 INFO:tasks.workunit.client.0.smithi071.stdout:1/262: getdents db/d34 0 2024-04-23T02:13:56.417 INFO:tasks.workunit.client.0.smithi071.stdout:1/263: dread - db/df/f45 zero size 2024-04-23T02:13:56.418 INFO:tasks.workunit.client.0.smithi071.stdout:5/400: write d3/da/d1a/d6d/f9a [993789,71335] 0 2024-04-23T02:13:56.421 INFO:tasks.workunit.client.1.smithi192.stdout:0/477: mkdir d6/d4b/d9d 0 2024-04-23T02:13:56.422 INFO:tasks.workunit.client.0.smithi071.stdout:1/264: mknod db/df/d13/c50 0 2024-04-23T02:13:56.422 INFO:tasks.workunit.client.0.smithi071.stdout:1/265: symlink db/df/l51 0 2024-04-23T02:13:56.423 INFO:tasks.workunit.client.0.smithi071.stdout:5/401: link d3/da/d1a/d1c/d46/d4e/f50 d3/df/d83/f9b 0 2024-04-23T02:13:56.425 INFO:tasks.workunit.client.0.smithi071.stdout:1/266: getdents db/df/d13/d2b 0 2024-04-23T02:13:56.426 INFO:tasks.workunit.client.0.smithi071.stdout:5/402: mknod d3/df/d83/c9c 0 2024-04-23T02:13:56.426 INFO:tasks.workunit.client.1.smithi192.stdout:0/478: dwrite d6/df/d13/d25/d3a/f87 [0,4194304] 0 2024-04-23T02:13:56.432 INFO:tasks.workunit.client.0.smithi071.stdout:5/403: dwrite d3/d34/f40 [0,4194304] 0 2024-04-23T02:13:56.434 INFO:tasks.workunit.client.0.smithi071.stdout:5/404: unlink d3/d1d/l36 0 2024-04-23T02:13:56.436 INFO:tasks.workunit.client.1.smithi192.stdout:7/344: sync 2024-04-23T02:13:56.437 INFO:tasks.workunit.client.0.smithi071.stdout:5/405: symlink d3/d1d/d43/d4b/l9d 0 2024-04-23T02:13:56.437 INFO:tasks.workunit.client.1.smithi192.stdout:7/345: chown d5/d43 481975 1 2024-04-23T02:13:56.437 INFO:tasks.workunit.client.0.smithi071.stdout:5/406: chown d3/da/d1a/d1c/l3e 1359673 1 2024-04-23T02:13:56.439 INFO:tasks.workunit.client.0.smithi071.stdout:5/407: creat d3/d34/d6b/f9e x:0 0 0 2024-04-23T02:13:56.439 INFO:tasks.workunit.client.0.smithi071.stdout:5/408: readlink d3/da/d1a/d1c/d46/l79 0 2024-04-23T02:13:56.440 INFO:tasks.workunit.client.0.smithi071.stdout:5/409: fsync d3/d1d/f2a 0 2024-04-23T02:13:56.445 INFO:tasks.workunit.client.1.smithi192.stdout:7/346: dwrite d5/d17/d1c/d57/f46 [0,4194304] 0 2024-04-23T02:13:56.450 INFO:tasks.workunit.client.1.smithi192.stdout:7/347: dread d5/d17/d1c/d57/f48 [0,4194304] 0 2024-04-23T02:13:56.451 INFO:tasks.workunit.client.1.smithi192.stdout:7/348: chown d5/d43/d4c/d4e/c53 523142 1 2024-04-23T02:13:56.452 INFO:tasks.workunit.client.1.smithi192.stdout:0/479: dread d6/df/d13/d25/d26/f27 [0,4194304] 0 2024-04-23T02:13:56.452 INFO:tasks.workunit.client.1.smithi192.stdout:0/480: chown d6/df/l1d 115319675 1 2024-04-23T02:13:56.455 INFO:tasks.workunit.client.1.smithi192.stdout:7/349: unlink d5/d17/d1c/d57/c5f 0 2024-04-23T02:13:56.463 INFO:tasks.workunit.client.1.smithi192.stdout:0/481: dwrite d6/df/d13/d6b/f8c [0,4194304] 0 2024-04-23T02:13:56.463 INFO:tasks.workunit.client.1.smithi192.stdout:7/350: dread d5/d17/d1c/d59/d5a/f32 [0,4194304] 0 2024-04-23T02:13:56.465 INFO:tasks.workunit.client.1.smithi192.stdout:7/351: mkdir d5/d43/d4c/d63 0 2024-04-23T02:13:56.468 INFO:tasks.workunit.client.1.smithi192.stdout:7/352: fsync d5/d17/d1c/d26/f34 0 2024-04-23T02:13:56.470 INFO:tasks.workunit.client.1.smithi192.stdout:7/353: symlink d5/d17/d1c/d57/l64 0 2024-04-23T02:13:56.479 INFO:tasks.workunit.client.1.smithi192.stdout:7/354: dwrite d5/f15 [0,4194304] 0 2024-04-23T02:13:56.486 INFO:tasks.workunit.client.1.smithi192.stdout:7/355: dwrite d5/d17/d1c/f3e [0,4194304] 0 2024-04-23T02:13:56.489 INFO:tasks.workunit.client.1.smithi192.stdout:7/356: chown d5/d17/d1c/d26/c27 29 1 2024-04-23T02:13:56.490 INFO:tasks.workunit.client.1.smithi192.stdout:7/357: read d5/d17/d1c/d59/d5a/f32 [1464740,23459] 0 2024-04-23T02:13:56.495 INFO:tasks.workunit.client.1.smithi192.stdout:7/358: dread d5/d17/d1c/f3c [0,4194304] 0 2024-04-23T02:13:56.499 INFO:tasks.workunit.client.1.smithi192.stdout:7/359: link d5/d17/d1c/d59/d5a/d41/l47 d5/d43/l65 0 2024-04-23T02:13:56.501 INFO:tasks.workunit.client.1.smithi192.stdout:7/360: getdents d5/d43/d4c/d5c 0 2024-04-23T02:13:56.503 INFO:tasks.workunit.client.1.smithi192.stdout:7/361: rename d5/d43/d4c/d4e/c53 to d5/d17/d1c/d57/c66 0 2024-04-23T02:13:56.504 INFO:tasks.workunit.client.1.smithi192.stdout:7/362: chown d5/d43/d4c/d4e 3586475 1 2024-04-23T02:13:56.504 INFO:tasks.workunit.client.1.smithi192.stdout:7/363: fsync d5/d17/d1c/f24 0 2024-04-23T02:13:56.505 INFO:tasks.workunit.client.1.smithi192.stdout:7/364: chown d5/d17/d1c/d59/d5a/d41/d50/l39 0 1 2024-04-23T02:13:56.507 INFO:tasks.workunit.client.1.smithi192.stdout:7/365: mkdir d5/d43/d4c/d5c/d67 0 2024-04-23T02:13:56.508 INFO:tasks.workunit.client.1.smithi192.stdout:7/366: write d5/d17/d1c/d4a/f58 [542398,106348] 0 2024-04-23T02:13:56.510 INFO:tasks.workunit.client.1.smithi192.stdout:7/367: creat d5/d17/d1c/d57/f68 x:0 0 0 2024-04-23T02:13:56.516 INFO:tasks.workunit.client.1.smithi192.stdout:7/368: dwrite d5/d17/d1c/f2d [0,4194304] 0 2024-04-23T02:13:56.526 INFO:tasks.workunit.client.1.smithi192.stdout:7/369: dread d5/d17/d1c/d4a/f58 [0,4194304] 0 2024-04-23T02:13:56.528 INFO:tasks.workunit.client.1.smithi192.stdout:7/370: getdents d5/d43/d4c/d63 0 2024-04-23T02:13:56.530 INFO:tasks.workunit.client.1.smithi192.stdout:7/371: write d5/d17/d1c/f45 [5001381,18696] 0 2024-04-23T02:13:56.530 INFO:tasks.workunit.client.1.smithi192.stdout:7/372: readlink d5/l16 0 2024-04-23T02:13:56.531 INFO:tasks.workunit.client.0.smithi071.stdout:7/203: sync 2024-04-23T02:13:56.531 INFO:tasks.workunit.client.0.smithi071.stdout:9/257: sync 2024-04-23T02:13:56.533 INFO:tasks.workunit.client.0.smithi071.stdout:9/258: read d1/f30 [1850556,15989] 0 2024-04-23T02:13:56.534 INFO:tasks.workunit.client.0.smithi071.stdout:7/204: symlink d3/d17/d41/l47 0 2024-04-23T02:13:56.535 INFO:tasks.workunit.client.0.smithi071.stdout:9/259: rmdir d1/d4e 39 2024-04-23T02:13:56.537 INFO:tasks.workunit.client.1.smithi192.stdout:7/373: dwrite d5/d17/d1c/d59/d5a/d41/d50/f35 [0,4194304] 0 2024-04-23T02:13:56.537 INFO:tasks.workunit.client.0.smithi071.stdout:7/205: rename d3/f11 to d3/d17/d1c/d31/f48 0 2024-04-23T02:13:56.539 INFO:tasks.workunit.client.1.smithi192.stdout:7/374: write d5/d17/d1c/d57/f22 [1919678,124083] 0 2024-04-23T02:13:56.539 INFO:tasks.workunit.client.0.smithi071.stdout:9/260: rename d1/d32/c3c to d1/d32/d3a/c51 0 2024-04-23T02:13:56.543 INFO:tasks.workunit.client.0.smithi071.stdout:9/261: symlink d1/d4e/l52 0 2024-04-23T02:13:56.543 INFO:tasks.workunit.client.1.smithi192.stdout:7/375: creat d5/d17/d1c/d59/d5a/d41/d50/f69 x:0 0 0 2024-04-23T02:13:56.545 INFO:tasks.workunit.client.0.smithi071.stdout:7/206: dwrite d3/d17/d1c/f26 [0,4194304] 0 2024-04-23T02:13:56.549 INFO:tasks.workunit.client.1.smithi192.stdout:7/376: dwrite d5/d17/d1c/f2d [0,4194304] 0 2024-04-23T02:13:56.550 INFO:tasks.workunit.client.0.smithi071.stdout:7/207: chown d3/d17/f22 525 1 2024-04-23T02:13:56.552 INFO:tasks.workunit.client.0.smithi071.stdout:9/262: rmdir d1 39 2024-04-23T02:13:56.554 INFO:tasks.workunit.client.0.smithi071.stdout:9/263: write d1/f30 [2821993,124638] 0 2024-04-23T02:13:56.555 INFO:tasks.workunit.client.0.smithi071.stdout:7/208: getdents d3/d17/d1c/d31 0 2024-04-23T02:13:56.556 INFO:tasks.workunit.client.0.smithi071.stdout:7/209: dread - d3/f3d zero size 2024-04-23T02:13:56.557 INFO:tasks.workunit.client.1.smithi192.stdout:7/377: dwrite d5/f15 [4194304,4194304] 0 2024-04-23T02:13:56.559 INFO:tasks.workunit.client.0.smithi071.stdout:9/264: rename d1/d1a/f36 to d1/d1a/d39/f53 0 2024-04-23T02:13:56.560 INFO:tasks.workunit.client.0.smithi071.stdout:7/210: creat d3/d17/d41/d34/d39/d3b/f49 x:0 0 0 2024-04-23T02:13:56.560 INFO:tasks.workunit.client.0.smithi071.stdout:9/265: write d1/f18 [279880,68733] 0 2024-04-23T02:13:56.561 INFO:tasks.workunit.client.1.smithi192.stdout:7/378: rename d5/d17/d1c/d57/l64 to d5/d17/d1c/d59/d5a/d41/d50/d3d/l6a 0 2024-04-23T02:13:56.562 INFO:tasks.workunit.client.0.smithi071.stdout:7/211: dread - d3/d17/d41/d37/f38 zero size 2024-04-23T02:13:56.562 INFO:tasks.workunit.client.0.smithi071.stdout:9/266: mkdir d1/d4e/d54 0 2024-04-23T02:13:56.563 INFO:tasks.workunit.client.0.smithi071.stdout:7/212: chown d3/d17/d1c/f45 482 1 2024-04-23T02:13:56.563 INFO:tasks.workunit.client.1.smithi192.stdout:7/379: write d5/f2e [744099,33480] 0 2024-04-23T02:13:56.565 INFO:tasks.workunit.client.1.smithi192.stdout:7/380: symlink d5/d43/d4c/d5c/d67/l6b 0 2024-04-23T02:13:56.567 INFO:tasks.workunit.client.0.smithi071.stdout:9/267: unlink d1/f18 0 2024-04-23T02:13:56.567 INFO:tasks.workunit.client.0.smithi071.stdout:9/268: write d1/f4 [4665063,10771] 0 2024-04-23T02:13:56.567 INFO:tasks.workunit.client.1.smithi192.stdout:7/381: truncate d5/d17/d1c/f3c 1512946 0 2024-04-23T02:13:56.567 INFO:tasks.workunit.client.0.smithi071.stdout:7/213: symlink d3/d17/d41/d3a/l4a 0 2024-04-23T02:13:56.568 INFO:tasks.workunit.client.1.smithi192.stdout:7/382: stat d5/d17/d1c/d57/l23 0 2024-04-23T02:13:56.569 INFO:tasks.workunit.client.1.smithi192.stdout:7/383: dread - d5/d17/d1c/d26/f34 zero size 2024-04-23T02:13:56.570 INFO:tasks.workunit.client.0.smithi071.stdout:9/269: read d1/d1a/f31 [3284429,47631] 0 2024-04-23T02:13:56.571 INFO:tasks.workunit.client.1.smithi192.stdout:7/384: creat d5/d43/d4c/f6c x:0 0 0 2024-04-23T02:13:56.572 INFO:tasks.workunit.client.1.smithi192.stdout:7/385: truncate d5/d17/d1c/d26/f34 717723 0 2024-04-23T02:13:56.572 INFO:tasks.workunit.client.1.smithi192.stdout:7/386: stat d5/d17/d1c/d59 0 2024-04-23T02:13:56.573 INFO:tasks.workunit.client.0.smithi071.stdout:9/270: creat d1/d1a/f55 x:0 0 0 2024-04-23T02:13:56.575 INFO:tasks.workunit.client.1.smithi192.stdout:7/387: unlink d5/d17/d1c/d57/l40 0 2024-04-23T02:13:56.578 INFO:tasks.workunit.client.1.smithi192.stdout:7/388: rename d5/d17/d1c/d57/c66 to d5/d17/d1c/d59/c6d 0 2024-04-23T02:13:56.580 INFO:tasks.workunit.client.1.smithi192.stdout:7/389: creat d5/d43/f6e x:0 0 0 2024-04-23T02:13:56.581 INFO:tasks.workunit.client.0.smithi071.stdout:9/271: getdents d1/d1a/d39 0 2024-04-23T02:13:56.583 INFO:tasks.workunit.client.0.smithi071.stdout:9/272: mknod d1/d1a/c56 0 2024-04-23T02:13:56.583 INFO:tasks.workunit.client.1.smithi192.stdout:7/390: dread d5/d17/d1c/d57/f48 [0,4194304] 0 2024-04-23T02:13:56.585 INFO:tasks.workunit.client.0.smithi071.stdout:9/273: creat d1/d32/d41/f57 x:0 0 0 2024-04-23T02:13:56.586 INFO:tasks.workunit.client.0.smithi071.stdout:7/214: truncate d3/f12 7079493 0 2024-04-23T02:13:56.587 INFO:tasks.workunit.client.0.smithi071.stdout:7/215: symlink d3/l4b 0 2024-04-23T02:13:56.589 INFO:tasks.workunit.client.0.smithi071.stdout:7/216: creat d3/d17/d1c/d2a/f4c x:0 0 0 2024-04-23T02:13:56.590 INFO:tasks.workunit.client.0.smithi071.stdout:7/217: readlink d3/d17/d1c/l3e 0 2024-04-23T02:13:56.591 INFO:tasks.workunit.client.0.smithi071.stdout:9/274: getdents d1/d32/d41 0 2024-04-23T02:13:56.592 INFO:tasks.workunit.client.0.smithi071.stdout:7/218: read d3/d17/d41/d34/f3c [522378,104929] 0 2024-04-23T02:13:56.598 INFO:tasks.workunit.client.0.smithi071.stdout:7/219: getdents d3/d17/d41/d34/d39/d3b 0 2024-04-23T02:13:56.611 INFO:tasks.workunit.client.0.smithi071.stdout:9/275: dread d1/f4 [0,4194304] 0 2024-04-23T02:13:56.613 INFO:tasks.workunit.client.0.smithi071.stdout:9/276: mknod d1/d4e/d54/c58 0 2024-04-23T02:13:56.625 INFO:tasks.workunit.client.0.smithi071.stdout:9/277: dread d1/d1a/f31 [0,4194304] 0 2024-04-23T02:13:56.626 INFO:tasks.workunit.client.0.smithi071.stdout:9/278: symlink d1/d32/l59 0 2024-04-23T02:13:56.627 INFO:tasks.workunit.client.0.smithi071.stdout:9/279: write d1/f37 [952354,87751] 0 2024-04-23T02:13:56.628 INFO:tasks.workunit.client.0.smithi071.stdout:9/280: chown d1/d4e/f4f 1418782852 1 2024-04-23T02:13:56.628 INFO:tasks.workunit.client.0.smithi071.stdout:9/281: stat d1/d4e/l52 0 2024-04-23T02:13:56.635 INFO:tasks.workunit.client.1.smithi192.stdout:1/186: sync 2024-04-23T02:13:56.635 INFO:tasks.workunit.client.1.smithi192.stdout:9/342: sync 2024-04-23T02:13:56.636 INFO:tasks.workunit.client.1.smithi192.stdout:9/343: dread - d0/d9/f3a zero size 2024-04-23T02:13:56.637 INFO:tasks.workunit.client.1.smithi192.stdout:1/187: creat dc/d16/f31 x:0 0 0 2024-04-23T02:13:56.638 INFO:tasks.workunit.client.1.smithi192.stdout:9/344: symlink d0/d9/d44/d74/d3c/d5f/l80 0 2024-04-23T02:13:56.641 INFO:tasks.workunit.client.1.smithi192.stdout:9/345: creat d0/f81 x:0 0 0 2024-04-23T02:13:56.642 INFO:tasks.workunit.client.1.smithi192.stdout:1/188: getdents dc/d14/d23 0 2024-04-23T02:13:56.643 INFO:tasks.workunit.client.0.smithi071.stdout:9/282: dread d1/f2b [0,4194304] 0 2024-04-23T02:13:56.644 INFO:tasks.workunit.client.0.smithi071.stdout:9/283: write d1/f30 [1754795,59600] 0 2024-04-23T02:13:56.649 INFO:tasks.workunit.client.0.smithi071.stdout:9/284: mkdir d1/d1a/d5a 0 2024-04-23T02:13:56.651 INFO:tasks.workunit.client.0.smithi071.stdout:2/262: sync 2024-04-23T02:13:56.651 INFO:tasks.workunit.client.0.smithi071.stdout:8/374: sync 2024-04-23T02:13:56.655 INFO:tasks.workunit.client.0.smithi071.stdout:2/263: dread d0/d2/d26/f32 [0,4194304] 0 2024-04-23T02:13:56.656 INFO:tasks.workunit.client.0.smithi071.stdout:8/375: truncate dc/d3a/f3c 824243 0 2024-04-23T02:13:56.656 INFO:tasks.workunit.client.0.smithi071.stdout:9/285: dwrite d1/d4e/f4f [0,4194304] 0 2024-04-23T02:13:56.658 INFO:tasks.workunit.client.0.smithi071.stdout:9/286: write d1/d32/f40 [579090,15682] 0 2024-04-23T02:13:56.661 INFO:tasks.workunit.client.0.smithi071.stdout:9/287: creat d1/d4e/f5b x:0 0 0 2024-04-23T02:13:56.661 INFO:tasks.workunit.client.0.smithi071.stdout:9/288: read - d1/d1a/f55 zero size 2024-04-23T02:13:56.679 INFO:tasks.workunit.client.1.smithi192.stdout:9/346: creat d0/d9/d44/d74/f82 x:0 0 0 2024-04-23T02:13:56.690 INFO:tasks.workunit.client.1.smithi192.stdout:9/347: link d0/d9/d44/d74/c2b d0/d9/d44/d74/d3c/c83 0 2024-04-23T02:13:56.697 INFO:tasks.workunit.client.0.smithi071.stdout:9/289: getdents d1/d1a 0 2024-04-23T02:13:56.700 INFO:tasks.workunit.client.0.smithi071.stdout:9/290: mknod d1/d1a/d39/d42/c5c 0 2024-04-23T02:13:56.702 INFO:tasks.workunit.client.0.smithi071.stdout:9/291: mknod d1/d1a/d39/c5d 0 2024-04-23T02:13:56.702 INFO:tasks.workunit.client.1.smithi192.stdout:1/189: dwrite dc/d16/f1f [0,4194304] 0 2024-04-23T02:13:56.703 INFO:tasks.workunit.client.1.smithi192.stdout:1/190: dread - dc/d16/f31 zero size 2024-04-23T02:13:56.704 INFO:tasks.workunit.client.0.smithi071.stdout:9/292: link d1/fb d1/d1a/d5a/f5e 0 2024-04-23T02:13:56.705 INFO:tasks.workunit.client.0.smithi071.stdout:9/293: dread - d1/d4e/f5b zero size 2024-04-23T02:13:56.716 INFO:tasks.workunit.client.0.smithi071.stdout:8/376: link dc/d10/l42 dc/l77 0 2024-04-23T02:13:56.717 INFO:tasks.workunit.client.0.smithi071.stdout:8/377: stat dc/f57 0 2024-04-23T02:13:56.717 INFO:tasks.workunit.client.0.smithi071.stdout:2/264: rmdir d0/d2/d26 39 2024-04-23T02:13:56.720 INFO:tasks.workunit.client.0.smithi071.stdout:2/265: rmdir d0/d2/d26/d3a 39 2024-04-23T02:13:56.721 INFO:tasks.workunit.client.0.smithi071.stdout:8/378: dread dc/d10/f27 [4194304,4194304] 0 2024-04-23T02:13:56.722 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:56 smithi071 ceph-mon[39837]: pgmap v25: 65 pgs: 65 active+clean; 895 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 4.1 MiB/s rd, 84 MiB/s wr, 135 op/s 2024-04-23T02:13:56.722 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:56 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:13:56.722 INFO:tasks.workunit.client.0.smithi071.stdout:2/266: write d0/d2/d26/d14/f23 [1049772,114178] 0 2024-04-23T02:13:56.723 INFO:tasks.workunit.client.1.smithi192.stdout:9/348: dread d0/f16 [4194304,4194304] 0 2024-04-23T02:13:56.725 INFO:tasks.workunit.client.1.smithi192.stdout:9/349: stat d0/d9/d44/d51/d5b/d61 0 2024-04-23T02:13:56.725 INFO:tasks.workunit.client.0.smithi071.stdout:8/379: mkdir dc/d78 0 2024-04-23T02:13:56.726 INFO:tasks.workunit.client.0.smithi071.stdout:2/267: rmdir d0/d2/d26 39 2024-04-23T02:13:56.727 INFO:tasks.workunit.client.1.smithi192.stdout:1/191: dwrite dc/fd [4194304,4194304] 0 2024-04-23T02:13:56.729 INFO:tasks.workunit.client.1.smithi192.stdout:9/350: write d0/d9/d44/f47 [3009319,55876] 0 2024-04-23T02:13:56.730 INFO:tasks.workunit.client.0.smithi071.stdout:8/380: dread dc/d15/f6b [0,4194304] 0 2024-04-23T02:13:56.731 INFO:tasks.workunit.client.0.smithi071.stdout:8/381: fdatasync dc/d10/d32/f4b 0 2024-04-23T02:13:56.736 INFO:tasks.workunit.client.1.smithi192.stdout:9/351: rename d0/d9/f38 to d0/d9/d44/d51/d5b/d61/f84 0 2024-04-23T02:13:56.736 INFO:tasks.workunit.client.1.smithi192.stdout:1/192: mkdir dc/d16/d19/d30/d32 0 2024-04-23T02:13:56.738 INFO:tasks.workunit.client.0.smithi071.stdout:2/268: dwrite d0/d2/d26/d14/f19 [0,4194304] 0 2024-04-23T02:13:56.739 INFO:tasks.workunit.client.0.smithi071.stdout:9/294: dread d1/d1a/f25 [0,4194304] 0 2024-04-23T02:13:56.741 INFO:tasks.workunit.client.1.smithi192.stdout:1/193: creat dc/d16/d19/d30/d32/f33 x:0 0 0 2024-04-23T02:13:56.742 INFO:tasks.workunit.client.0.smithi071.stdout:2/269: mkdir d0/d2/d34/d55 0 2024-04-23T02:13:56.743 INFO:tasks.workunit.client.0.smithi071.stdout:9/295: mkdir d1/d32/d3a/d5f 0 2024-04-23T02:13:56.744 INFO:tasks.workunit.client.1.smithi192.stdout:1/194: mkdir dc/d16/d19/d30/d32/d34 0 2024-04-23T02:13:56.744 INFO:tasks.workunit.client.0.smithi071.stdout:9/296: stat d1/fc 0 2024-04-23T02:13:56.745 INFO:tasks.workunit.client.0.smithi071.stdout:2/270: dread - d0/d2/d26/d3a/f44 zero size 2024-04-23T02:13:56.747 INFO:tasks.workunit.client.0.smithi071.stdout:2/271: fsync d0/d2/d26/d14/f42 0 2024-04-23T02:13:56.747 INFO:tasks.workunit.client.1.smithi192.stdout:9/352: dwrite d0/d9/d44/d4f/f3e [0,4194304] 0 2024-04-23T02:13:56.747 INFO:tasks.workunit.client.0.smithi071.stdout:9/297: creat d1/d32/d3a/d5f/f60 x:0 0 0 2024-04-23T02:13:56.749 INFO:tasks.workunit.client.1.smithi192.stdout:1/195: mkdir dc/d14/d15/d35 0 2024-04-23T02:13:56.750 INFO:tasks.workunit.client.0.smithi071.stdout:9/298: mknod d1/d32/d3a/d5f/c61 0 2024-04-23T02:13:56.750 INFO:tasks.workunit.client.1.smithi192.stdout:9/353: chown d0/l62 1248 1 2024-04-23T02:13:56.753 INFO:tasks.workunit.client.1.smithi192.stdout:9/354: stat d0/l12 0 2024-04-23T02:13:56.754 INFO:tasks.workunit.client.0.smithi071.stdout:8/382: dread dc/d10/f3d [0,4194304] 0 2024-04-23T02:13:56.757 INFO:tasks.workunit.client.1.smithi192.stdout:9/355: symlink d0/d9/d44/l85 0 2024-04-23T02:13:56.757 INFO:tasks.workunit.client.0.smithi071.stdout:8/383: creat dc/d15/f79 x:0 0 0 2024-04-23T02:13:56.759 INFO:tasks.workunit.client.1.smithi192.stdout:1/196: dwrite dc/f10 [0,4194304] 0 2024-04-23T02:13:56.761 INFO:tasks.workunit.client.0.smithi071.stdout:8/384: rename dc/d15/l36 to dc/d10/l7a 0 2024-04-23T02:13:56.762 INFO:tasks.workunit.client.0.smithi071.stdout:8/385: write dc/f57 [1923122,22416] 0 2024-04-23T02:13:56.763 INFO:tasks.workunit.client.0.smithi071.stdout:8/386: dread - dc/d10/d32/f4b zero size 2024-04-23T02:13:56.764 INFO:tasks.workunit.client.0.smithi071.stdout:8/387: dread - dc/d15/f73 zero size 2024-04-23T02:13:56.765 INFO:tasks.workunit.client.1.smithi192.stdout:1/197: unlink dc/f11 0 2024-04-23T02:13:56.766 INFO:tasks.workunit.client.0.smithi071.stdout:2/272: truncate d0/d2/d51/f53 262913 0 2024-04-23T02:13:56.768 INFO:tasks.workunit.client.0.smithi071.stdout:8/388: unlink dc/d10/c26 0 2024-04-23T02:13:56.770 INFO:tasks.workunit.client.0.smithi071.stdout:2/273: mkdir d0/d2/d51/d56 0 2024-04-23T02:13:56.770 INFO:tasks.workunit.client.0.smithi071.stdout:9/299: dread d1/d1a/d5a/f5e [0,4194304] 0 2024-04-23T02:13:56.771 INFO:tasks.workunit.client.0.smithi071.stdout:8/389: mknod dc/d15/d50/c7b 0 2024-04-23T02:13:56.772 INFO:tasks.workunit.client.0.smithi071.stdout:9/300: stat d1/d32/d3a/c46 0 2024-04-23T02:13:56.775 INFO:tasks.workunit.client.0.smithi071.stdout:8/390: chown dc/d3a/f3c 116 1 2024-04-23T02:13:56.776 INFO:tasks.workunit.client.1.smithi192.stdout:3/299: sync 2024-04-23T02:13:56.776 INFO:tasks.workunit.client.1.smithi192.stdout:4/542: sync 2024-04-23T02:13:56.776 INFO:tasks.workunit.client.0.smithi071.stdout:2/274: mknod d0/d2/d26/d14/d1f/d4f/c57 0 2024-04-23T02:13:56.776 INFO:tasks.workunit.client.0.smithi071.stdout:8/391: dread - dc/d10/d30/f52 zero size 2024-04-23T02:13:56.778 INFO:tasks.workunit.client.0.smithi071.stdout:8/392: dread - dc/d10/d30/f66 zero size 2024-04-23T02:13:56.778 INFO:tasks.workunit.client.0.smithi071.stdout:9/301: unlink d1/d1a/f38 0 2024-04-23T02:13:56.778 INFO:tasks.workunit.client.1.smithi192.stdout:4/543: mknod de/d15/d2a/d74/cb5 0 2024-04-23T02:13:56.778 INFO:tasks.workunit.client.1.smithi192.stdout:3/300: truncate d1/d3/d17/d1f/d33/d24/d34/d40/d4c/f60 859854 0 2024-04-23T02:13:56.780 INFO:tasks.workunit.client.0.smithi071.stdout:2/275: truncate d0/d2/d26/d14/d1f/d36/f4a 10225 0 2024-04-23T02:13:56.780 INFO:tasks.workunit.client.0.smithi071.stdout:8/393: write dc/d15/d2d/f43 [4627626,98793] 0 2024-04-23T02:13:56.781 INFO:tasks.workunit.client.0.smithi071.stdout:9/302: symlink d1/d4e/d54/l62 0 2024-04-23T02:13:56.781 INFO:tasks.workunit.client.1.smithi192.stdout:4/544: mkdir de/d15/d19/d3d/db6 0 2024-04-23T02:13:56.781 INFO:tasks.workunit.client.1.smithi192.stdout:4/545: chown de/c5d 0 1 2024-04-23T02:13:56.782 INFO:tasks.workunit.client.0.smithi071.stdout:9/303: dread - d1/d4e/f5b zero size 2024-04-23T02:13:56.782 INFO:tasks.workunit.client.1.smithi192.stdout:3/301: unlink d1/d3/d17/d1f/d33/d24/d34/d40/d4c/f60 0 2024-04-23T02:13:56.783 INFO:tasks.workunit.client.0.smithi071.stdout:2/276: write d0/d2/d26/d14/f22 [891876,34754] 0 2024-04-23T02:13:56.785 INFO:tasks.workunit.client.0.smithi071.stdout:9/304: creat d1/d32/d3a/d5f/f63 x:0 0 0 2024-04-23T02:13:56.785 INFO:tasks.workunit.client.0.smithi071.stdout:2/277: chown d0/d2/d26/d14/d1f/f25 6 1 2024-04-23T02:13:56.786 INFO:tasks.workunit.client.0.smithi071.stdout:2/278: read - d0/d2/d3c/f4e zero size 2024-04-23T02:13:56.790 INFO:tasks.workunit.client.0.smithi071.stdout:2/279: fsync d0/d2/d26/f32 0 2024-04-23T02:13:56.791 INFO:tasks.workunit.client.1.smithi192.stdout:3/302: creat d1/d3/d6/d7/f64 x:0 0 0 2024-04-23T02:13:56.792 INFO:tasks.workunit.client.0.smithi071.stdout:2/280: chown d0/d2/d26/d14/f23 105 1 2024-04-23T02:13:56.794 INFO:tasks.workunit.client.0.smithi071.stdout:6/274: sync 2024-04-23T02:13:56.794 INFO:tasks.workunit.client.0.smithi071.stdout:0/263: sync 2024-04-23T02:13:56.794 INFO:tasks.workunit.client.0.smithi071.stdout:3/242: sync 2024-04-23T02:13:56.794 INFO:tasks.workunit.client.0.smithi071.stdout:2/281: mkdir d0/d2/d45/d58 0 2024-04-23T02:13:56.796 INFO:tasks.workunit.client.1.smithi192.stdout:3/303: mkdir d1/d3/d17/d65 0 2024-04-23T02:13:56.797 INFO:tasks.workunit.client.0.smithi071.stdout:3/243: mknod d7/d28/c50 0 2024-04-23T02:13:56.798 INFO:tasks.workunit.client.0.smithi071.stdout:6/275: creat d6/d1f/d46/f56 x:0 0 0 2024-04-23T02:13:56.799 INFO:tasks.workunit.client.0.smithi071.stdout:2/282: mknod d0/d2/d26/d14/d1f/d36/c59 0 2024-04-23T02:13:56.801 INFO:tasks.workunit.client.0.smithi071.stdout:9/305: dread d1/f37 [0,4194304] 0 2024-04-23T02:13:56.802 INFO:tasks.workunit.client.0.smithi071.stdout:8/394: dwrite dc/d3a/f3c [0,4194304] 0 2024-04-23T02:13:56.802 INFO:tasks.workunit.client.0.smithi071.stdout:9/306: write d1/f16 [787584,18670] 0 2024-04-23T02:13:56.804 INFO:tasks.workunit.client.0.smithi071.stdout:9/307: chown d1/d4e/l52 440855 1 2024-04-23T02:13:56.805 INFO:tasks.workunit.client.0.smithi071.stdout:6/276: symlink d6/d1f/l57 0 2024-04-23T02:13:56.806 INFO:tasks.workunit.client.1.smithi192.stdout:3/304: truncate d1/f55 2103276 0 2024-04-23T02:13:56.806 INFO:tasks.workunit.client.1.smithi192.stdout:4/546: dwrite de/d15/d19/d3d/f54 [0,4194304] 0 2024-04-23T02:13:56.808 INFO:tasks.workunit.client.0.smithi071.stdout:9/308: write d1/f4 [455404,117304] 0 2024-04-23T02:13:56.810 INFO:tasks.workunit.client.0.smithi071.stdout:6/277: rename d6/d22/c3a to d6/d1f/d46/c58 0 2024-04-23T02:13:56.811 INFO:tasks.workunit.client.0.smithi071.stdout:6/278: chown d6/d1f/l50 19 1 2024-04-23T02:13:56.812 INFO:tasks.workunit.client.0.smithi071.stdout:3/244: link d7/d22/l2a d7/d8/l51 0 2024-04-23T02:13:56.812 INFO:tasks.workunit.client.1.smithi192.stdout:4/547: write de/d15/d2a/d74/d92/faa [134128,57955] 0 2024-04-23T02:13:56.815 INFO:tasks.workunit.client.1.smithi192.stdout:3/305: mknod d1/d3/d17/d1f/d33/d24/d34/d40/d46/c66 0 2024-04-23T02:13:56.815 INFO:tasks.workunit.client.0.smithi071.stdout:3/245: mkdir d7/d22/d23/d26/d52 0 2024-04-23T02:13:56.816 INFO:tasks.workunit.client.1.smithi192.stdout:3/306: fsync d1/d3/d6/d7/d29/d3b/f41 0 2024-04-23T02:13:56.817 INFO:tasks.workunit.client.0.smithi071.stdout:9/309: truncate d1/fe 1117011 0 2024-04-23T02:13:56.818 INFO:tasks.workunit.client.0.smithi071.stdout:0/264: truncate d0/d3/d1c/f22 6950349 0 2024-04-23T02:13:56.819 INFO:tasks.workunit.client.0.smithi071.stdout:8/395: dread dc/d10/f27 [0,4194304] 0 2024-04-23T02:13:56.820 INFO:tasks.workunit.client.0.smithi071.stdout:3/246: rename d7/d22/d23/c4a to d7/d27/c53 0 2024-04-23T02:13:56.821 INFO:tasks.workunit.client.1.smithi192.stdout:4/548: link l5 de/d15/lb7 0 2024-04-23T02:13:56.822 INFO:tasks.workunit.client.1.smithi192.stdout:4/549: stat de/d15/d19/d3d/d49/d26/c7b 0 2024-04-23T02:13:56.822 INFO:tasks.workunit.client.1.smithi192.stdout:4/550: chown de/d15/d19/f9a 296 1 2024-04-23T02:13:56.822 INFO:tasks.workunit.client.1.smithi192.stdout:3/307: dread d1/d3/d6/d7/d29/d3b/f41 [0,4194304] 0 2024-04-23T02:13:56.823 INFO:tasks.workunit.client.0.smithi071.stdout:8/396: rename dc/d15/c24 to dc/d10/d32/c7c 0 2024-04-23T02:13:56.824 INFO:tasks.workunit.client.1.smithi192.stdout:3/308: mknod d1/c67 0 2024-04-23T02:13:56.825 INFO:tasks.workunit.client.1.smithi192.stdout:3/309: readlink d1/d3/d17/d1f/l4d 0 2024-04-23T02:13:56.825 INFO:tasks.workunit.client.0.smithi071.stdout:8/397: rmdir dc/d3a 39 2024-04-23T02:13:56.826 INFO:tasks.workunit.client.1.smithi192.stdout:4/551: rmdir de/d15/d19/d3d/d49/d70 39 2024-04-23T02:13:56.827 INFO:tasks.workunit.client.0.smithi071.stdout:2/283: dwrite d0/d2/fb [0,4194304] 0 2024-04-23T02:13:56.828 INFO:tasks.workunit.client.0.smithi071.stdout:3/247: dread d7/d8/f4e [0,4194304] 0 2024-04-23T02:13:56.828 INFO:tasks.workunit.client.0.smithi071.stdout:8/398: mknod dc/d10/c7d 0 2024-04-23T02:13:56.830 INFO:tasks.workunit.client.1.smithi192.stdout:4/552: dread - de/d15/d53/f58 zero size 2024-04-23T02:13:56.834 INFO:tasks.workunit.client.0.smithi071.stdout:8/399: write dc/d10/d55/f65 [2427906,72044] 0 2024-04-23T02:13:56.838 INFO:tasks.workunit.client.1.smithi192.stdout:3/310: rename d1/d3/c62 to d1/d3/d6/d7/d5f/c68 0 2024-04-23T02:13:56.841 INFO:tasks.workunit.client.0.smithi071.stdout:2/284: dwrite d0/d2/d26/df/f40 [0,4194304] 0 2024-04-23T02:13:56.842 INFO:tasks.workunit.client.0.smithi071.stdout:8/400: rmdir dc/d10/d55 39 2024-04-23T02:13:56.844 INFO:tasks.workunit.client.0.smithi071.stdout:2/285: creat d0/d2/d45/f5a x:0 0 0 2024-04-23T02:13:56.846 INFO:tasks.workunit.client.0.smithi071.stdout:3/248: dwrite d7/d10/f1e [4194304,4194304] 0 2024-04-23T02:13:56.847 INFO:tasks.workunit.client.1.smithi192.stdout:3/311: dwrite d1/d3/d6/d7/d29/d3b/f41 [0,4194304] 0 2024-04-23T02:13:56.850 INFO:tasks.workunit.client.1.smithi192.stdout:3/312: rmdir d1/d3/d6/d7/d5f 39 2024-04-23T02:13:56.852 INFO:tasks.workunit.client.0.smithi071.stdout:0/265: write d0/d3/d1c/f22 [338669,116637] 0 2024-04-23T02:13:56.854 INFO:tasks.workunit.client.0.smithi071.stdout:2/286: link d0/d2/d26/d3a/f3d d0/d2/d49/f5b 0 2024-04-23T02:13:56.855 INFO:tasks.workunit.client.0.smithi071.stdout:0/266: mkdir d0/d3/de/d10/d14/d32/d44/d31/d52 0 2024-04-23T02:13:56.856 INFO:tasks.workunit.client.1.smithi192.stdout:3/313: symlink d1/d3/d17/d1f/d33/d24/d34/d38/l69 0 2024-04-23T02:13:56.857 INFO:tasks.workunit.client.0.smithi071.stdout:3/249: dwrite d7/d8/fc [0,4194304] 0 2024-04-23T02:13:56.860 INFO:tasks.workunit.client.0.smithi071.stdout:0/267: truncate d0/d3/de/d10/d14/d32/d44/f43 148715 0 2024-04-23T02:13:56.863 INFO:tasks.workunit.client.0.smithi071.stdout:2/287: mkdir d0/d2/d26/d14/d1f/d36/d5c 0 2024-04-23T02:13:56.863 INFO:tasks.workunit.client.0.smithi071.stdout:8/401: truncate dc/d10/f14 4195989 0 2024-04-23T02:13:56.864 INFO:tasks.workunit.client.1.smithi192.stdout:3/314: dwrite d1/d3/d6/d7/f31 [0,4194304] 0 2024-04-23T02:13:56.865 INFO:tasks.workunit.client.0.smithi071.stdout:0/268: rename d0/d3/de/d10/d14/d32/d37/f4e to d0/d3/de/d10/d14/d32/d37/f53 0 2024-04-23T02:13:56.867 INFO:tasks.workunit.client.0.smithi071.stdout:8/402: creat dc/d15/f7e x:0 0 0 2024-04-23T02:13:56.867 INFO:tasks.workunit.client.0.smithi071.stdout:0/269: chown d0/d3/de/d10/d16/f3e 8284 1 2024-04-23T02:13:56.868 INFO:tasks.workunit.client.0.smithi071.stdout:2/288: write d0/d2/d26/d14/f1a [1337265,55896] 0 2024-04-23T02:13:56.869 INFO:tasks.workunit.client.0.smithi071.stdout:8/403: mkdir dc/d10/d32/d7f 0 2024-04-23T02:13:56.870 INFO:tasks.workunit.client.1.smithi192.stdout:3/315: mknod d1/d3/d17/d1f/d33/d24/d34/d40/d5d/c6a 0 2024-04-23T02:13:56.870 INFO:tasks.workunit.client.1.smithi192.stdout:4/553: dread de/d15/f2d [0,4194304] 0 2024-04-23T02:13:56.872 INFO:tasks.workunit.client.1.smithi192.stdout:3/316: mknod d1/d3/d17/d1f/d33/d24/d34/d38/c6b 0 2024-04-23T02:13:56.873 INFO:tasks.workunit.client.0.smithi071.stdout:8/404: unlink dc/d15/f73 0 2024-04-23T02:13:56.874 INFO:tasks.workunit.client.1.smithi192.stdout:3/317: fdatasync d1/d3/d6/d7/f2a 0 2024-04-23T02:13:56.876 INFO:tasks.workunit.client.0.smithi071.stdout:3/250: fsync d7/d10/f1e 0 2024-04-23T02:13:56.878 INFO:tasks.workunit.client.1.smithi192.stdout:3/318: dread - d1/d3/d17/d1f/d33/d24/d34/d40/f63 zero size 2024-04-23T02:13:56.879 INFO:tasks.workunit.client.0.smithi071.stdout:3/251: chown d7/d22/l2a 2719 1 2024-04-23T02:13:56.880 INFO:tasks.workunit.client.0.smithi071.stdout:2/289: dwrite d0/d2/d26/d3a/f3d [0,4194304] 0 2024-04-23T02:13:56.880 INFO:tasks.workunit.client.1.smithi192.stdout:4/554: dwrite de/d15/d19/d1a/fb0 [0,4194304] 0 2024-04-23T02:13:56.881 INFO:tasks.workunit.client.1.smithi192.stdout:4/555: chown de/l17 3728549 1 2024-04-23T02:13:56.882 INFO:tasks.workunit.client.0.smithi071.stdout:2/290: fdatasync d0/d2/d26/df/f3b 0 2024-04-23T02:13:56.883 INFO:tasks.workunit.client.1.smithi192.stdout:4/556: stat de/d15/d19/d3d/f5f 0 2024-04-23T02:13:56.885 INFO:tasks.workunit.client.1.smithi192.stdout:4/557: mknod de/d15/d53/cb8 0 2024-04-23T02:13:56.885 INFO:tasks.workunit.client.1.smithi192.stdout:4/558: readlink de/l17 0 2024-04-23T02:13:56.886 INFO:tasks.workunit.client.0.smithi071.stdout:2/291: mknod d0/d2/d45/d58/c5d 0 2024-04-23T02:13:56.889 INFO:tasks.workunit.client.0.smithi071.stdout:2/292: rename d0/f3f to d0/d2/d26/df/f5e 0 2024-04-23T02:13:56.899 INFO:tasks.workunit.client.0.smithi071.stdout:8/405: dread dc/d10/d55/f65 [0,4194304] 0 2024-04-23T02:13:56.900 INFO:tasks.workunit.client.0.smithi071.stdout:8/406: write f6 [725392,90875] 0 2024-04-23T02:13:56.902 INFO:tasks.workunit.client.0.smithi071.stdout:8/407: creat dc/d10/d55/f80 x:0 0 0 2024-04-23T02:13:56.905 INFO:tasks.workunit.client.1.smithi192.stdout:4/559: fsync de/d15/d19/d1a/fb0 0 2024-04-23T02:13:56.914 INFO:tasks.workunit.client.0.smithi071.stdout:3/252: dread d7/d27/d3e/f40 [4194304,4194304] 0 2024-04-23T02:13:56.914 INFO:tasks.workunit.client.0.smithi071.stdout:8/408: truncate dc/d10/d55/f6c 3459004 0 2024-04-23T02:13:56.915 INFO:tasks.workunit.client.1.smithi192.stdout:3/319: dread d1/d3/f49 [0,4194304] 0 2024-04-23T02:13:56.916 INFO:tasks.workunit.client.0.smithi071.stdout:3/253: unlink f5 0 2024-04-23T02:13:56.918 INFO:tasks.workunit.client.0.smithi071.stdout:8/409: mknod dc/d10/c81 0 2024-04-23T02:13:56.919 INFO:tasks.workunit.client.1.smithi192.stdout:8/502: sync 2024-04-23T02:13:56.919 INFO:tasks.workunit.client.1.smithi192.stdout:0/482: sync 2024-04-23T02:13:56.919 INFO:tasks.workunit.client.1.smithi192.stdout:7/391: sync 2024-04-23T02:13:56.919 INFO:tasks.workunit.client.1.smithi192.stdout:5/420: sync 2024-04-23T02:13:56.919 INFO:tasks.workunit.client.1.smithi192.stdout:6/457: sync 2024-04-23T02:13:56.919 INFO:tasks.workunit.client.1.smithi192.stdout:2/323: sync 2024-04-23T02:13:56.920 INFO:tasks.workunit.client.0.smithi071.stdout:3/254: rename d7/d8/l13 to d7/d28/l54 0 2024-04-23T02:13:56.920 INFO:tasks.workunit.client.1.smithi192.stdout:0/483: chown d6/f6d 1496824158 1 2024-04-23T02:13:56.924 INFO:tasks.workunit.client.1.smithi192.stdout:5/421: truncate d2/d2e/f64 102747 0 2024-04-23T02:13:56.925 INFO:tasks.workunit.client.1.smithi192.stdout:7/392: unlink d5/d17/d1c/f24 0 2024-04-23T02:13:56.926 INFO:tasks.workunit.client.1.smithi192.stdout:6/458: fsync d3/d15/d1b/d21/d3a/d51/f93 0 2024-04-23T02:13:56.926 INFO:tasks.workunit.client.1.smithi192.stdout:4/560: dread de/d15/d19/d3d/f32 [0,4194304] 0 2024-04-23T02:13:56.928 INFO:tasks.workunit.client.0.smithi071.stdout:3/255: truncate d7/d10/d38/f41 3143451 0 2024-04-23T02:13:56.930 INFO:tasks.workunit.client.1.smithi192.stdout:2/324: mknod d8/d26/d27/d57/c65 0 2024-04-23T02:13:56.932 INFO:tasks.workunit.client.1.smithi192.stdout:5/422: creat d2/db/d16/d1b/f89 x:0 0 0 2024-04-23T02:13:56.933 INFO:tasks.workunit.client.1.smithi192.stdout:7/393: chown d5/d17/d1c/f3c 51504 1 2024-04-23T02:13:56.933 INFO:tasks.workunit.client.0.smithi071.stdout:3/256: getdents d7/d22 0 2024-04-23T02:13:56.936 INFO:tasks.workunit.client.1.smithi192.stdout:4/561: mkdir de/d15/d19/d3d/d34/db9 0 2024-04-23T02:13:56.937 INFO:tasks.workunit.client.1.smithi192.stdout:3/320: dwrite d1/d3/f19 [0,4194304] 0 2024-04-23T02:13:56.937 INFO:tasks.workunit.client.0.smithi071.stdout:3/257: dread d7/d8/f11 [0,4194304] 0 2024-04-23T02:13:56.937 INFO:tasks.workunit.client.1.smithi192.stdout:0/484: dread d6/df/d13/d25/f91 [0,4194304] 0 2024-04-23T02:13:56.939 INFO:tasks.workunit.client.1.smithi192.stdout:5/423: chown d2/db/d2b/f82 47787660 1 2024-04-23T02:13:56.940 INFO:tasks.workunit.client.0.smithi071.stdout:3/258: creat d7/d22/f55 x:0 0 0 2024-04-23T02:13:56.940 INFO:tasks.workunit.client.1.smithi192.stdout:8/503: link d1/d12/f39 d1/d9/d3e/d59/fb0 0 2024-04-23T02:13:56.941 INFO:tasks.workunit.client.0.smithi071.stdout:3/259: read d7/d8/f11 [684404,100072] 0 2024-04-23T02:13:56.944 INFO:tasks.workunit.client.0.smithi071.stdout:7/220: sync 2024-04-23T02:13:56.944 INFO:tasks.workunit.client.0.smithi071.stdout:4/319: sync 2024-04-23T02:13:56.944 INFO:tasks.workunit.client.0.smithi071.stdout:1/267: sync 2024-04-23T02:13:56.944 INFO:tasks.workunit.client.0.smithi071.stdout:5/410: sync 2024-04-23T02:13:56.946 INFO:tasks.workunit.client.1.smithi192.stdout:0/485: read d6/df/d56/d85/f46 [35552,56167] 0 2024-04-23T02:13:56.946 INFO:tasks.workunit.client.0.smithi071.stdout:7/221: write d3/f1d [344585,45573] 0 2024-04-23T02:13:56.947 INFO:tasks.workunit.client.1.smithi192.stdout:0/486: dread - d6/f6d zero size 2024-04-23T02:13:56.947 INFO:tasks.workunit.client.0.smithi071.stdout:1/268: symlink db/d34/l52 0 2024-04-23T02:13:56.948 INFO:tasks.workunit.client.0.smithi071.stdout:3/260: truncate d7/d22/f55 707713 0 2024-04-23T02:13:56.948 INFO:tasks.workunit.client.0.smithi071.stdout:5/411: symlink d3/d34/d6b/d81/d5a/l9f 0 2024-04-23T02:13:56.950 INFO:tasks.workunit.client.0.smithi071.stdout:1/269: chown db/f25 260 1 2024-04-23T02:13:56.950 INFO:tasks.workunit.client.0.smithi071.stdout:7/222: write d3/d17/d1c/f29 [224684,99038] 0 2024-04-23T02:13:56.952 INFO:tasks.workunit.client.1.smithi192.stdout:3/321: dread d1/d3/d17/d2b/f32 [0,4194304] 0 2024-04-23T02:13:56.953 INFO:tasks.workunit.client.1.smithi192.stdout:8/504: mkdir d1/d12/d23/d4b/d73/d7e/d88/da9/db1 0 2024-04-23T02:13:56.954 INFO:tasks.workunit.client.1.smithi192.stdout:0/487: chown d6/df/d56/d85/f19 2 1 2024-04-23T02:13:56.954 INFO:tasks.workunit.client.0.smithi071.stdout:1/270: creat db/df/d13/d20/f53 x:0 0 0 2024-04-23T02:13:56.955 INFO:tasks.workunit.client.1.smithi192.stdout:4/562: getdents de/d15/d19/d3d/d49 0 2024-04-23T02:13:56.957 INFO:tasks.workunit.client.0.smithi071.stdout:7/223: symlink d3/d17/d41/d34/l4d 0 2024-04-23T02:13:56.957 INFO:tasks.workunit.client.0.smithi071.stdout:1/271: rmdir db/d11 39 2024-04-23T02:13:56.957 INFO:tasks.workunit.client.0.smithi071.stdout:4/320: dwrite d2/d2a/f36 [0,4194304] 0 2024-04-23T02:13:56.957 INFO:tasks.workunit.client.0.smithi071.stdout:5/412: dread d3/f29 [0,4194304] 0 2024-04-23T02:13:56.959 INFO:tasks.workunit.client.1.smithi192.stdout:7/394: truncate d5/d17/d1c/d59/d5a/f32 3450290 0 2024-04-23T02:13:56.959 INFO:tasks.workunit.client.1.smithi192.stdout:4/563: write f4 [1637869,33162] 0 2024-04-23T02:13:56.960 INFO:tasks.workunit.client.0.smithi071.stdout:3/261: dwrite d7/d27/d2c/f2e [4194304,4194304] 0 2024-04-23T02:13:56.961 INFO:tasks.workunit.client.1.smithi192.stdout:6/459: dread d3/f57 [0,4194304] 0 2024-04-23T02:13:56.961 INFO:tasks.workunit.client.1.smithi192.stdout:5/424: dwrite d2/db/d16/d1b/f54 [0,4194304] 0 2024-04-23T02:13:56.962 INFO:tasks.workunit.client.0.smithi071.stdout:7/224: rename d3/d17/f22 to d3/d17/d1c/d2a/d43/f4e 0 2024-04-23T02:13:56.963 INFO:tasks.workunit.client.0.smithi071.stdout:7/225: read - d3/d17/d1c/f3f zero size 2024-04-23T02:13:56.964 INFO:tasks.workunit.client.0.smithi071.stdout:4/321: write d2/d2a/d2d/d39/d42/d49/d67/d1d/f61 [866574,19184] 0 2024-04-23T02:13:56.965 INFO:tasks.workunit.client.0.smithi071.stdout:1/272: creat db/f54 x:0 0 0 2024-04-23T02:13:56.966 INFO:tasks.workunit.client.0.smithi071.stdout:4/322: chown d2/d2a/d2d/d39/d42/d49/d67/d1d/f35 8 1 2024-04-23T02:13:56.967 INFO:tasks.workunit.client.0.smithi071.stdout:7/226: mkdir d3/d17/d41/d37/d4f 0 2024-04-23T02:13:56.967 INFO:tasks.workunit.client.1.smithi192.stdout:6/460: chown d3/c86 62728 1 2024-04-23T02:13:56.968 INFO:tasks.workunit.client.0.smithi071.stdout:3/262: mkdir d7/d27/d2c/d56 0 2024-04-23T02:13:56.968 INFO:tasks.workunit.client.0.smithi071.stdout:3/263: chown d7/d27/d3e 118749793 1 2024-04-23T02:13:56.968 INFO:tasks.workunit.client.1.smithi192.stdout:2/325: dwrite f7 [0,4194304] 0 2024-04-23T02:13:56.970 INFO:tasks.workunit.client.0.smithi071.stdout:1/273: mknod db/c55 0 2024-04-23T02:13:56.970 INFO:tasks.workunit.client.1.smithi192.stdout:7/395: mknod d5/d17/d1c/d4a/c6f 0 2024-04-23T02:13:56.971 INFO:tasks.workunit.client.1.smithi192.stdout:2/326: write d8/d25/f3f [920678,44818] 0 2024-04-23T02:13:56.971 INFO:tasks.workunit.client.1.smithi192.stdout:4/564: chown de/d15/d19/d3d/f32 12719514 1 2024-04-23T02:13:56.972 INFO:tasks.workunit.client.0.smithi071.stdout:7/227: creat d3/d17/d41/d37/f50 x:0 0 0 2024-04-23T02:13:56.973 INFO:tasks.workunit.client.1.smithi192.stdout:8/505: creat d1/d9/d3e/fb2 x:0 0 0 2024-04-23T02:13:56.973 INFO:tasks.workunit.client.1.smithi192.stdout:9/356: sync 2024-04-23T02:13:56.973 INFO:tasks.workunit.client.1.smithi192.stdout:1/198: sync 2024-04-23T02:13:56.974 INFO:tasks.workunit.client.1.smithi192.stdout:9/357: read - d0/d9/d44/d4f/f79 zero size 2024-04-23T02:13:56.976 INFO:tasks.workunit.client.1.smithi192.stdout:7/396: unlink d5/d17/f4b 0 2024-04-23T02:13:56.979 INFO:tasks.workunit.client.1.smithi192.stdout:2/327: truncate d8/d26/d29/f58 1356028 0 2024-04-23T02:13:56.980 INFO:tasks.workunit.client.0.smithi071.stdout:3/264: dwrite f0 [0,4194304] 0 2024-04-23T02:13:56.981 INFO:tasks.workunit.client.1.smithi192.stdout:9/358: write d0/f4e [585366,34759] 0 2024-04-23T02:13:56.984 INFO:tasks.workunit.client.1.smithi192.stdout:7/397: write d5/d17/d1c/d59/d5a/d41/d50/f69 [539691,9094] 0 2024-04-23T02:13:56.986 INFO:tasks.workunit.client.0.smithi071.stdout:1/274: getdents db/d1a 0 2024-04-23T02:13:56.987 INFO:tasks.workunit.client.0.smithi071.stdout:4/323: getdents d2/d2a/d2d/d39/d42/d49/d67/d3e 0 2024-04-23T02:13:56.990 INFO:tasks.workunit.client.1.smithi192.stdout:9/359: chown d0/d9/d44/d74/l4b 273440 1 2024-04-23T02:13:56.990 INFO:tasks.workunit.client.1.smithi192.stdout:5/425: creat d2/d6/f8a x:0 0 0 2024-04-23T02:13:56.990 INFO:tasks.workunit.client.1.smithi192.stdout:8/506: symlink d1/d9/d3e/lb3 0 2024-04-23T02:13:56.990 INFO:tasks.workunit.client.0.smithi071.stdout:4/324: write d2/d2a/f36 [23766,85851] 0 2024-04-23T02:13:56.991 INFO:tasks.workunit.client.1.smithi192.stdout:0/488: write d6/df/d13/d25/d26/f82 [1013413,94233] 0 2024-04-23T02:13:56.992 INFO:tasks.workunit.client.0.smithi071.stdout:7/228: dwrite d3/d17/d41/d37/f38 [0,4194304] 0 2024-04-23T02:13:56.994 INFO:tasks.workunit.client.1.smithi192.stdout:1/199: fsync dc/d16/f31 0 2024-04-23T02:13:56.996 INFO:tasks.workunit.client.0.smithi071.stdout:4/325: write d2/f43 [4636879,96611] 0 2024-04-23T02:13:56.997 INFO:tasks.workunit.client.1.smithi192.stdout:5/426: truncate d2/d6/f3b 334629 0 2024-04-23T02:13:56.997 INFO:tasks.workunit.client.0.smithi071.stdout:1/275: getdents db/df/d13/d2f 0 2024-04-23T02:13:56.997 INFO:tasks.workunit.client.0.smithi071.stdout:7/229: chown d3/d17/d41/f2f 608612286 1 2024-04-23T02:13:56.999 INFO:tasks.workunit.client.0.smithi071.stdout:7/230: stat d3/d17/c2d 0 2024-04-23T02:13:56.999 INFO:tasks.workunit.client.0.smithi071.stdout:3/265: dwrite d7/d8/f1b [0,4194304] 0 2024-04-23T02:13:57.000 INFO:tasks.workunit.client.1.smithi192.stdout:4/565: link de/d15/d19/la4 de/d15/d19/d3d/d34/d7c/da1/lba 0 2024-04-23T02:13:57.003 INFO:tasks.workunit.client.0.smithi071.stdout:1/276: symlink db/df/l56 0 2024-04-23T02:13:57.005 INFO:tasks.workunit.client.1.smithi192.stdout:5/427: dread d2/d2e/f64 [0,4194304] 0 2024-04-23T02:13:57.006 INFO:tasks.workunit.client.0.smithi071.stdout:3/266: rename d7/d10/f1e to d7/d27/d2c/f57 0 2024-04-23T02:13:57.006 INFO:tasks.workunit.client.1.smithi192.stdout:7/398: mkdir d5/d17/d1c/d59/d5a/d70 0 2024-04-23T02:13:57.007 INFO:tasks.workunit.client.0.smithi071.stdout:1/277: chown db/df/d13/d20/c26 946403026 1 2024-04-23T02:13:57.007 INFO:tasks.workunit.client.1.smithi192.stdout:9/360: dwrite d0/d9/d44/d74/f82 [0,4194304] 0 2024-04-23T02:13:57.008 INFO:tasks.workunit.client.1.smithi192.stdout:6/461: link d3/d15/d1b/d21/d52/d64/f65 d3/d28/f96 0 2024-04-23T02:13:57.010 INFO:tasks.workunit.client.0.smithi071.stdout:3/267: unlink d7/d27/d2c/f2e 0 2024-04-23T02:13:57.011 INFO:tasks.workunit.client.0.smithi071.stdout:4/326: mknod d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d58/d5e/c69 0 2024-04-23T02:13:57.011 INFO:tasks.workunit.client.0.smithi071.stdout:7/231: dwrite d3/d17/d1c/f45 [0,4194304] 0 2024-04-23T02:13:57.012 INFO:tasks.workunit.client.0.smithi071.stdout:4/327: fdatasync d2/d2a/f36 0 2024-04-23T02:13:57.013 INFO:tasks.workunit.client.1.smithi192.stdout:3/322: truncate d1/d3/d6/d7/d29/d3b/f41 2343223 0 2024-04-23T02:13:57.014 INFO:tasks.workunit.client.1.smithi192.stdout:0/489: symlink d6/df/d13/d25/d95/l9e 0 2024-04-23T02:13:57.014 INFO:tasks.workunit.client.0.smithi071.stdout:1/278: mkdir db/d1a/d57 0 2024-04-23T02:13:57.015 INFO:tasks.workunit.client.0.smithi071.stdout:1/279: fsync db/df/d3a/f44 0 2024-04-23T02:13:57.018 INFO:tasks.workunit.client.1.smithi192.stdout:8/507: creat d1/d12/d23/d4b/d73/d4c/d82/fb4 x:0 0 0 2024-04-23T02:13:57.026 INFO:tasks.workunit.client.1.smithi192.stdout:2/328: dwrite d8/d26/d29/f58 [0,4194304] 0 2024-04-23T02:13:57.027 INFO:tasks.workunit.client.1.smithi192.stdout:0/490: write d6/df/f4a [67710,129169] 0 2024-04-23T02:13:57.029 INFO:tasks.workunit.client.1.smithi192.stdout:7/399: symlink d5/l71 0 2024-04-23T02:13:57.032 INFO:tasks.workunit.client.1.smithi192.stdout:3/323: unlink d1/d3/d17/d1a/c2e 0 2024-04-23T02:13:57.032 INFO:tasks.workunit.client.1.smithi192.stdout:4/566: dwrite de/f8c [4194304,4194304] 0 2024-04-23T02:13:57.034 INFO:tasks.workunit.client.1.smithi192.stdout:9/361: mknod d0/d9/d44/d51/d5b/d61/c86 0 2024-04-23T02:13:57.038 INFO:tasks.workunit.client.1.smithi192.stdout:1/200: dwrite f4 [0,4194304] 0 2024-04-23T02:13:57.044 INFO:tasks.workunit.client.1.smithi192.stdout:6/462: fsync d3/d15/d1b/d21/d3a/f4e 0 2024-04-23T02:13:57.045 INFO:tasks.workunit.client.1.smithi192.stdout:3/324: write d1/d3/d17/d1a/f4f [381395,96962] 0 2024-04-23T02:13:57.045 INFO:tasks.workunit.client.1.smithi192.stdout:9/362: write d0/d9/d44/f5e [36960,68081] 0 2024-04-23T02:13:57.049 INFO:tasks.workunit.client.1.smithi192.stdout:8/508: rmdir d1/d12/d23/d4b/d73/d4c 39 2024-04-23T02:13:57.049 INFO:tasks.workunit.client.1.smithi192.stdout:9/363: readlink d0/d9/d44/l7f 0 2024-04-23T02:13:57.049 INFO:tasks.workunit.client.1.smithi192.stdout:1/201: read - dc/d14/f21 zero size 2024-04-23T02:13:57.050 INFO:tasks.workunit.client.1.smithi192.stdout:0/491: creat d6/df/d35/d49/f9f x:0 0 0 2024-04-23T02:13:57.050 INFO:tasks.workunit.client.1.smithi192.stdout:7/400: write d5/d17/d1c/d57/f48 [652176,55353] 0 2024-04-23T02:13:57.051 INFO:tasks.workunit.client.1.smithi192.stdout:1/202: stat f0 0 2024-04-23T02:13:57.051 INFO:tasks.workunit.client.1.smithi192.stdout:5/428: rename d2/d6/d14/f86 to d2/db/d16/f8b 0 2024-04-23T02:13:57.055 INFO:tasks.workunit.client.1.smithi192.stdout:7/401: fdatasync d5/f2e 0 2024-04-23T02:13:57.055 INFO:tasks.workunit.client.1.smithi192.stdout:2/329: dwrite d8/d26/d27/d31/f2a [4194304,4194304] 0 2024-04-23T02:13:57.064 INFO:tasks.workunit.client.1.smithi192.stdout:6/463: truncate d3/d18/f78 1170547 0 2024-04-23T02:13:57.067 INFO:tasks.workunit.client.1.smithi192.stdout:2/330: symlink d8/d26/d29/d43/l66 0 2024-04-23T02:13:57.070 INFO:tasks.workunit.client.1.smithi192.stdout:2/331: dread - d8/d26/d27/d53/f5b zero size 2024-04-23T02:13:57.071 INFO:tasks.workunit.client.1.smithi192.stdout:6/464: mknod d3/d28/c97 0 2024-04-23T02:13:57.074 INFO:tasks.workunit.client.1.smithi192.stdout:2/332: dread - d8/d26/d27/f45 zero size 2024-04-23T02:13:57.075 INFO:tasks.workunit.client.1.smithi192.stdout:5/429: link d2/db/d16/d1b/d3d/d40/d49/f58 d2/db/d16/d1b/d3d/d40/f8c 0 2024-04-23T02:13:57.075 INFO:tasks.workunit.client.1.smithi192.stdout:8/509: dwrite d1/d12/d7d/f9a [0,4194304] 0 2024-04-23T02:13:57.084 INFO:tasks.workunit.client.1.smithi192.stdout:8/510: creat d1/d12/d23/d2c/d8c/d95/fb5 x:0 0 0 2024-04-23T02:13:57.084 INFO:tasks.workunit.client.1.smithi192.stdout:3/325: dread d1/f5 [0,4194304] 0 2024-04-23T02:13:57.087 INFO:tasks.workunit.client.1.smithi192.stdout:2/333: chown d8/d26/d29/d43/d50/d5e/f5f 14 1 2024-04-23T02:13:57.088 INFO:tasks.workunit.client.1.smithi192.stdout:5/430: creat d2/d6/f8d x:0 0 0 2024-04-23T02:13:57.091 INFO:tasks.workunit.client.1.smithi192.stdout:7/402: dwrite d5/d17/d1c/d26/f28 [0,4194304] 0 2024-04-23T02:13:57.091 INFO:tasks.workunit.client.1.smithi192.stdout:0/492: dread d6/f10 [0,4194304] 0 2024-04-23T02:13:57.091 INFO:tasks.workunit.client.1.smithi192.stdout:6/465: dwrite f1 [0,4194304] 0 2024-04-23T02:13:57.093 INFO:tasks.workunit.client.1.smithi192.stdout:3/326: dread - d1/d3/d17/d2b/f53 zero size 2024-04-23T02:13:57.093 INFO:tasks.workunit.client.1.smithi192.stdout:0/493: read f5 [341060,36057] 0 2024-04-23T02:13:57.095 INFO:tasks.workunit.client.1.smithi192.stdout:5/431: write d2/d32/f44 [3650725,57877] 0 2024-04-23T02:13:57.101 INFO:tasks.workunit.client.1.smithi192.stdout:6/466: chown d3/f49 3 1 2024-04-23T02:13:57.102 INFO:tasks.workunit.client.1.smithi192.stdout:5/432: write d2/db/d16/d1b/f7a [322168,95364] 0 2024-04-23T02:13:57.103 INFO:tasks.workunit.client.1.smithi192.stdout:8/511: creat d1/d12/d23/d4b/d73/d4c/d82/fb6 x:0 0 0 2024-04-23T02:13:57.104 INFO:tasks.workunit.client.1.smithi192.stdout:6/467: stat d3/d15/d1b/f8f 0 2024-04-23T02:13:57.104 INFO:tasks.workunit.client.1.smithi192.stdout:7/403: creat d5/d43/d4c/d5c/f72 x:0 0 0 2024-04-23T02:13:57.106 INFO:tasks.workunit.client.1.smithi192.stdout:3/327: mkdir d1/d3/d17/d1f/d33/d24/d34/d6c 0 2024-04-23T02:13:57.106 INFO:tasks.workunit.client.1.smithi192.stdout:4/567: dread de/d15/f8f [0,4194304] 0 2024-04-23T02:13:57.108 INFO:tasks.workunit.client.1.smithi192.stdout:0/494: symlink d6/df/d13/d25/la0 0 2024-04-23T02:13:57.110 INFO:tasks.workunit.client.1.smithi192.stdout:3/328: fdatasync d1/d3/d6/f10 0 2024-04-23T02:13:57.110 INFO:tasks.workunit.client.1.smithi192.stdout:1/203: dread dc/f13 [0,4194304] 0 2024-04-23T02:13:57.112 INFO:tasks.workunit.client.1.smithi192.stdout:5/433: mknod d2/d6/d14/d60/c8e 0 2024-04-23T02:13:57.112 INFO:tasks.workunit.client.1.smithi192.stdout:6/468: dread d3/d15/d1b/f77 [0,4194304] 0 2024-04-23T02:13:57.112 INFO:tasks.workunit.client.1.smithi192.stdout:4/568: write de/d15/d19/d1a/fb0 [4494526,68368] 0 2024-04-23T02:13:57.114 INFO:tasks.workunit.client.1.smithi192.stdout:2/334: dwrite d8/d25/f5c [0,4194304] 0 2024-04-23T02:13:57.115 INFO:tasks.workunit.client.1.smithi192.stdout:0/495: chown d6/df/d56/d85/d45/c8f 2 1 2024-04-23T02:13:57.115 INFO:tasks.workunit.client.1.smithi192.stdout:3/329: dread - d1/d3/d6/d7/f64 zero size 2024-04-23T02:13:57.117 INFO:tasks.workunit.client.1.smithi192.stdout:7/404: truncate d5/d17/d1c/d26/f34 764719 0 2024-04-23T02:13:57.119 INFO:tasks.workunit.client.1.smithi192.stdout:6/469: mknod d3/d15/d1b/d66/d80/d8e/c98 0 2024-04-23T02:13:57.120 INFO:tasks.workunit.client.1.smithi192.stdout:2/335: creat d8/d14/f67 x:0 0 0 2024-04-23T02:13:57.121 INFO:tasks.workunit.client.1.smithi192.stdout:2/336: stat d8/d26/f61 0 2024-04-23T02:13:57.122 INFO:tasks.workunit.client.1.smithi192.stdout:1/204: rename f4 to dc/d16/d19/d30/d32/f36 0 2024-04-23T02:13:57.125 INFO:tasks.workunit.client.1.smithi192.stdout:4/569: mkdir de/d15/dbb 0 2024-04-23T02:13:57.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:56 smithi192 ceph-mon[49662]: pgmap v25: 65 pgs: 65 active+clean; 895 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 4.1 MiB/s rd, 84 MiB/s wr, 135 op/s 2024-04-23T02:13:57.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:56 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:13:57.127 INFO:tasks.workunit.client.0.smithi071.stdout:7/232: mkdir d3/d51 0 2024-04-23T02:13:57.128 INFO:tasks.workunit.client.0.smithi071.stdout:3/268: rename d7/d8/l18 to d7/d27/d3e/l58 0 2024-04-23T02:13:57.128 INFO:tasks.workunit.client.0.smithi071.stdout:4/328: rmdir d2/d2a 39 2024-04-23T02:13:57.128 INFO:tasks.workunit.client.0.smithi071.stdout:1/280: creat db/df/f58 x:0 0 0 2024-04-23T02:13:57.130 INFO:tasks.workunit.client.0.smithi071.stdout:4/329: write d2/f43 [3419639,51467] 0 2024-04-23T02:13:57.130 INFO:tasks.workunit.client.0.smithi071.stdout:7/233: write d3/d17/d1c/f29 [798694,17041] 0 2024-04-23T02:13:57.131 INFO:tasks.workunit.client.0.smithi071.stdout:1/281: readlink db/d1a/l28 0 2024-04-23T02:13:57.132 INFO:tasks.workunit.client.0.smithi071.stdout:7/234: chown d3/d17/d1c 2249460 1 2024-04-23T02:13:57.132 INFO:tasks.workunit.client.1.smithi192.stdout:8/512: rename d1/d12/d23/d4b/d73/l78 to d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d8b/lb7 0 2024-04-23T02:13:57.133 INFO:tasks.workunit.client.1.smithi192.stdout:5/434: creat d2/db/d2b/d52/f8f x:0 0 0 2024-04-23T02:13:57.133 INFO:tasks.workunit.client.1.smithi192.stdout:4/570: dread de/d15/d19/d3d/f5f [0,4194304] 0 2024-04-23T02:13:57.134 INFO:tasks.workunit.client.1.smithi192.stdout:0/496: symlink d6/df/d13/d25/d3a/d5b/la1 0 2024-04-23T02:13:57.135 INFO:tasks.workunit.client.1.smithi192.stdout:2/337: symlink d8/d26/d29/d43/d5d/l68 0 2024-04-23T02:13:57.135 INFO:tasks.workunit.client.1.smithi192.stdout:8/513: fsync d1/d12/d23/d4b/d73/d4c/f6e 0 2024-04-23T02:13:57.135 INFO:tasks.workunit.client.0.smithi071.stdout:1/282: rename f7 to db/d34/f59 0 2024-04-23T02:13:57.136 INFO:tasks.workunit.client.0.smithi071.stdout:1/283: chown fa 900552438 1 2024-04-23T02:13:57.136 INFO:tasks.workunit.client.1.smithi192.stdout:2/338: stat d8/d14/l22 0 2024-04-23T02:13:57.136 INFO:tasks.workunit.client.0.smithi071.stdout:7/235: fdatasync d3/d17/d41/d34/f35 0 2024-04-23T02:13:57.138 INFO:tasks.workunit.client.0.smithi071.stdout:4/330: rmdir d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d58/d5e 39 2024-04-23T02:13:57.140 INFO:tasks.workunit.client.0.smithi071.stdout:1/284: symlink db/df/d3a/l5a 0 2024-04-23T02:13:57.141 INFO:tasks.workunit.client.0.smithi071.stdout:1/285: chown db/df/d13/d2b/f35 3496037 1 2024-04-23T02:13:57.141 INFO:tasks.workunit.client.0.smithi071.stdout:7/236: dread d3/d17/d1c/f29 [0,4194304] 0 2024-04-23T02:13:57.142 INFO:tasks.workunit.client.0.smithi071.stdout:4/331: symlink d2/d2a/d2d/l6a 0 2024-04-23T02:13:57.143 INFO:tasks.workunit.client.1.smithi192.stdout:0/497: mknod d6/df/d13/d6b/d73/ca2 0 2024-04-23T02:13:57.146 INFO:tasks.workunit.client.0.smithi071.stdout:4/332: creat d2/d53/f6b x:0 0 0 2024-04-23T02:13:57.146 INFO:tasks.workunit.client.1.smithi192.stdout:2/339: dread d8/d14/f4c [0,4194304] 0 2024-04-23T02:13:57.146 INFO:tasks.workunit.client.1.smithi192.stdout:5/435: rmdir d2/db/d2b 39 2024-04-23T02:13:57.148 INFO:tasks.workunit.client.1.smithi192.stdout:5/436: readlink l1 0 2024-04-23T02:13:57.148 INFO:tasks.workunit.client.0.smithi071.stdout:3/269: dwrite d7/fd [0,4194304] 0 2024-04-23T02:13:57.149 INFO:tasks.workunit.client.1.smithi192.stdout:6/470: rename d3/d18/c87 to d3/c99 0 2024-04-23T02:13:57.152 INFO:tasks.workunit.client.1.smithi192.stdout:3/330: dwrite d1/d3/d17/d1f/d33/d24/f45 [0,4194304] 0 2024-04-23T02:13:57.153 INFO:tasks.workunit.client.0.smithi071.stdout:4/333: rmdir d2/d2a/d2d/d39/d42/d49/d67/d3e 39 2024-04-23T02:13:57.154 INFO:tasks.workunit.client.1.smithi192.stdout:6/471: fsync d3/d11/f13 0 2024-04-23T02:13:57.154 INFO:tasks.workunit.client.1.smithi192.stdout:2/340: fsync d8/f1d 0 2024-04-23T02:13:57.160 INFO:tasks.workunit.client.1.smithi192.stdout:2/341: write d8/d26/d29/d43/d50/f55 [689417,91628] 0 2024-04-23T02:13:57.160 INFO:tasks.workunit.client.0.smithi071.stdout:4/334: getdents d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d10 0 2024-04-23T02:13:57.162 INFO:tasks.workunit.client.1.smithi192.stdout:7/405: rename d5/d17/d1c/d4a/c6f to d5/d17/d1c/c73 0 2024-04-23T02:13:57.163 INFO:tasks.workunit.client.1.smithi192.stdout:1/205: dwrite dc/d16/f1f [0,4194304] 0 2024-04-23T02:13:57.165 INFO:tasks.workunit.client.0.smithi071.stdout:1/286: dread db/f25 [0,4194304] 0 2024-04-23T02:13:57.166 INFO:tasks.workunit.client.1.smithi192.stdout:4/571: truncate de/d15/d19/d3d/d34/d4a/f71 1014671 0 2024-04-23T02:13:57.167 INFO:tasks.workunit.client.0.smithi071.stdout:3/270: dwrite d7/fd [4194304,4194304] 0 2024-04-23T02:13:57.167 INFO:tasks.workunit.client.0.smithi071.stdout:7/237: dwrite d3/f12 [0,4194304] 0 2024-04-23T02:13:57.168 INFO:tasks.workunit.client.0.smithi071.stdout:4/335: dread - d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f5b zero size 2024-04-23T02:13:57.171 INFO:tasks.workunit.client.0.smithi071.stdout:1/287: truncate db/df/d13/d20/f32 142634 0 2024-04-23T02:13:57.172 INFO:tasks.workunit.client.1.smithi192.stdout:3/331: mkdir d1/d3/d17/d1a/d6d 0 2024-04-23T02:13:57.173 INFO:tasks.workunit.client.0.smithi071.stdout:7/238: write d3/d17/f1f [83559,100917] 0 2024-04-23T02:13:57.174 INFO:tasks.workunit.client.0.smithi071.stdout:4/336: rename d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c to d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c/d6c 22 2024-04-23T02:13:57.174 INFO:tasks.workunit.client.1.smithi192.stdout:0/498: rename d6/df/d13/d25/d95/l9e to d6/d4b/d9d/la3 0 2024-04-23T02:13:57.174 INFO:tasks.workunit.client.1.smithi192.stdout:2/342: mkdir d8/d26/d27/d57/d69 0 2024-04-23T02:13:57.177 INFO:tasks.workunit.client.0.smithi071.stdout:1/288: unlink db/df/l2d 0 2024-04-23T02:13:57.177 INFO:tasks.workunit.client.1.smithi192.stdout:8/514: dread d1/d12/f1e [0,4194304] 0 2024-04-23T02:13:57.179 INFO:tasks.workunit.client.1.smithi192.stdout:5/437: getdents d2/d6/d2d 0 2024-04-23T02:13:57.179 INFO:tasks.workunit.client.1.smithi192.stdout:4/572: write de/d15/d2a/d61/f88 [338084,115135] 0 2024-04-23T02:13:57.181 INFO:tasks.workunit.client.1.smithi192.stdout:6/472: dwrite d3/d15/d5b/f5e [0,4194304] 0 2024-04-23T02:13:57.183 INFO:tasks.workunit.client.0.smithi071.stdout:1/289: rename db/df/d13/d20/f46 to db/d1a/d57/f5b 0 2024-04-23T02:13:57.184 INFO:tasks.workunit.client.0.smithi071.stdout:7/239: dwrite d3/d17/d1c/d31/f48 [0,4194304] 0 2024-04-23T02:13:57.185 INFO:tasks.workunit.client.0.smithi071.stdout:1/290: dread - db/df/d13/d2f/d48/f49 zero size 2024-04-23T02:13:57.186 INFO:tasks.workunit.client.0.smithi071.stdout:7/240: chown d3/d17/d41/d34/f3c 473138 1 2024-04-23T02:13:57.186 INFO:tasks.workunit.client.0.smithi071.stdout:1/291: fsync db/df/d13/d20/f32 0 2024-04-23T02:13:57.187 INFO:tasks.workunit.client.0.smithi071.stdout:1/292: stat db/d1a 0 2024-04-23T02:13:57.188 INFO:tasks.workunit.client.0.smithi071.stdout:7/241: dread - d3/d17/d41/d37/f50 zero size 2024-04-23T02:13:57.190 INFO:tasks.workunit.client.0.smithi071.stdout:1/293: readlink db/d34/l38 0 2024-04-23T02:13:57.191 INFO:tasks.workunit.client.0.smithi071.stdout:1/294: dread - db/df/d13/d20/f4a zero size 2024-04-23T02:13:57.192 INFO:tasks.workunit.client.1.smithi192.stdout:0/499: rmdir d6/df/d13/d25/d3a/d93 39 2024-04-23T02:13:57.194 INFO:tasks.workunit.client.1.smithi192.stdout:0/500: fdatasync d6/f6d 0 2024-04-23T02:13:57.194 INFO:tasks.workunit.client.1.smithi192.stdout:6/473: chown d3/d15/d1b/d21/d7c/l89 180 1 2024-04-23T02:13:57.196 INFO:tasks.workunit.client.1.smithi192.stdout:7/406: dwrite d5/d17/d1c/d57/f48 [0,4194304] 0 2024-04-23T02:13:57.196 INFO:tasks.workunit.client.0.smithi071.stdout:4/337: write d2/d2a/d2d/d39/d42/d49/d67/f1e [944272,35741] 0 2024-04-23T02:13:57.197 INFO:tasks.workunit.client.0.smithi071.stdout:7/242: dwrite d3/d17/d1c/f26 [0,4194304] 0 2024-04-23T02:13:57.201 INFO:tasks.workunit.client.0.smithi071.stdout:7/243: rename d3/l1b to d3/d17/d1c/l52 0 2024-04-23T02:13:57.203 INFO:tasks.workunit.client.1.smithi192.stdout:1/206: dwrite dc/d16/f1f [0,4194304] 0 2024-04-23T02:13:57.206 INFO:tasks.workunit.client.0.smithi071.stdout:7/244: creat d3/d17/d41/d34/d39/d3b/f53 x:0 0 0 2024-04-23T02:13:57.208 INFO:tasks.workunit.client.1.smithi192.stdout:5/438: creat d2/db/d16/d1b/f90 x:0 0 0 2024-04-23T02:13:57.209 INFO:tasks.workunit.client.1.smithi192.stdout:8/515: creat d1/d12/d23/d4b/d73/d4c/d82/fb8 x:0 0 0 2024-04-23T02:13:57.211 INFO:tasks.workunit.client.0.smithi071.stdout:7/245: link d3/d17/d41/f27 d3/d17/d41/d34/d39/f54 0 2024-04-23T02:13:57.211 INFO:tasks.workunit.client.1.smithi192.stdout:2/343: dwrite d8/d14/f1a [0,4194304] 0 2024-04-23T02:13:57.213 INFO:tasks.workunit.client.0.smithi071.stdout:7/246: mknod d3/d17/d41/c55 0 2024-04-23T02:13:57.215 INFO:tasks.workunit.client.0.smithi071.stdout:7/247: fsync d3/d17/d1c/f20 0 2024-04-23T02:13:57.216 INFO:tasks.workunit.client.1.smithi192.stdout:1/207: chown dc/l1e 44 1 2024-04-23T02:13:57.217 INFO:tasks.workunit.client.0.smithi071.stdout:7/248: creat d3/d17/d1c/d31/f56 x:0 0 0 2024-04-23T02:13:57.221 INFO:tasks.workunit.client.0.smithi071.stdout:7/249: link d3/d17/d1c/c24 d3/d51/c57 0 2024-04-23T02:13:57.221 INFO:tasks.workunit.client.0.smithi071.stdout:7/250: dread - d3/d17/f36 zero size 2024-04-23T02:13:57.223 INFO:tasks.workunit.client.0.smithi071.stdout:7/251: write d3/d17/d41/d34/d39/d3b/f46 [465851,114532] 0 2024-04-23T02:13:57.223 INFO:tasks.workunit.client.0.smithi071.stdout:7/252: fdatasync d3/f3d 0 2024-04-23T02:13:57.224 INFO:tasks.workunit.client.0.smithi071.stdout:7/253: fsync d3/d17/d1c/d2a/f4c 0 2024-04-23T02:13:57.225 INFO:tasks.workunit.client.1.smithi192.stdout:5/439: dwrite d2/d6/f3b [0,4194304] 0 2024-04-23T02:13:57.226 INFO:tasks.workunit.client.1.smithi192.stdout:6/474: link d3/d28/f2e d3/d28/f9a 0 2024-04-23T02:13:57.227 INFO:tasks.workunit.client.1.smithi192.stdout:1/208: dread dc/d16/d19/d30/d32/f36 [0,4194304] 0 2024-04-23T02:13:57.228 INFO:tasks.workunit.client.1.smithi192.stdout:1/209: dread - dc/d16/f31 zero size 2024-04-23T02:13:57.229 INFO:tasks.workunit.client.1.smithi192.stdout:3/332: rename d1/d3/d6/d7/d29/d3b/f41 to d1/d3/d17/f6e 0 2024-04-23T02:13:57.231 INFO:tasks.workunit.client.0.smithi071.stdout:7/254: dwrite d3/d17/d1c/d31/f56 [0,4194304] 0 2024-04-23T02:13:57.231 INFO:tasks.workunit.client.1.smithi192.stdout:2/344: creat d8/d26/d27/d57/d69/f6a x:0 0 0 2024-04-23T02:13:57.232 INFO:tasks.workunit.client.1.smithi192.stdout:7/407: link d5/d17/d1c/d57/f46 d5/d17/d1c/f74 0 2024-04-23T02:13:57.232 INFO:tasks.workunit.client.1.smithi192.stdout:4/573: link de/d15/d19/d1a/d95/lb4 de/d15/d19/d3d/db6/lbc 0 2024-04-23T02:13:57.233 INFO:tasks.workunit.client.1.smithi192.stdout:5/440: creat d2/db/d16/d1b/d3d/d40/d49/f91 x:0 0 0 2024-04-23T02:13:57.234 INFO:tasks.workunit.client.1.smithi192.stdout:6/475: symlink d3/d28/l9b 0 2024-04-23T02:13:57.234 INFO:tasks.workunit.client.0.smithi071.stdout:7/255: creat d3/d17/d41/d3a/f58 x:0 0 0 2024-04-23T02:13:57.234 INFO:tasks.workunit.client.1.smithi192.stdout:4/574: write de/d15/d53/d81/fa6 [41046,94537] 0 2024-04-23T02:13:57.236 INFO:tasks.workunit.client.1.smithi192.stdout:3/333: truncate d1/f5 2590294 0 2024-04-23T02:13:57.238 INFO:tasks.workunit.client.0.smithi071.stdout:7/256: mkdir d3/d17/d41/d34/d59 0 2024-04-23T02:13:57.240 INFO:tasks.workunit.client.0.smithi071.stdout:7/257: getdents d3/d17/d41/d37/d4f 0 2024-04-23T02:13:57.241 INFO:tasks.workunit.client.1.smithi192.stdout:7/408: mkdir d5/d17/d1c/d57/d75 0 2024-04-23T02:13:57.241 INFO:tasks.workunit.client.1.smithi192.stdout:2/345: creat d8/d26/f6b x:0 0 0 2024-04-23T02:13:57.243 INFO:tasks.workunit.client.1.smithi192.stdout:1/210: creat dc/d14/d15/d35/f37 x:0 0 0 2024-04-23T02:13:57.244 INFO:tasks.workunit.client.1.smithi192.stdout:6/476: mknod d3/d28/c9c 0 2024-04-23T02:13:57.244 INFO:tasks.workunit.client.1.smithi192.stdout:8/516: link d1/d9/d3e/d59/c79 d1/df/cb9 0 2024-04-23T02:13:57.246 INFO:tasks.workunit.client.1.smithi192.stdout:7/409: creat d5/d17/d1c/d4a/f76 x:0 0 0 2024-04-23T02:13:57.247 INFO:tasks.workunit.client.0.smithi071.stdout:4/338: truncate d2/d2a/d2d/d39/f46 380645 0 2024-04-23T02:13:57.247 INFO:tasks.workunit.client.1.smithi192.stdout:8/517: write d1/f72 [392740,62282] 0 2024-04-23T02:13:57.248 INFO:tasks.workunit.client.1.smithi192.stdout:6/477: chown d3/d15/c3d 165847870 1 2024-04-23T02:13:57.248 INFO:tasks.workunit.client.1.smithi192.stdout:1/211: chown dc/d16/c17 141721810 1 2024-04-23T02:13:57.249 INFO:tasks.workunit.client.1.smithi192.stdout:4/575: symlink de/d15/d19/d3d/db6/lbd 0 2024-04-23T02:13:57.249 INFO:tasks.workunit.client.1.smithi192.stdout:8/518: chown d1/d12/d23/f3c 3544854 1 2024-04-23T02:13:57.250 INFO:tasks.workunit.client.1.smithi192.stdout:1/212: fdatasync dc/d16/f1f 0 2024-04-23T02:13:57.251 INFO:tasks.workunit.client.0.smithi071.stdout:7/258: write d3/d17/d1c/f20 [5041512,751] 0 2024-04-23T02:13:57.251 INFO:tasks.workunit.client.1.smithi192.stdout:5/441: creat d2/db/d2b/d52/f92 x:0 0 0 2024-04-23T02:13:57.252 INFO:tasks.workunit.client.1.smithi192.stdout:6/478: write d3/d15/d5b/f5e [4159904,36251] 0 2024-04-23T02:13:57.253 INFO:tasks.workunit.client.0.smithi071.stdout:7/259: creat d3/d17/f5a x:0 0 0 2024-04-23T02:13:57.253 INFO:tasks.workunit.client.1.smithi192.stdout:2/346: link d8/d25/f32 d8/d25/f6c 0 2024-04-23T02:13:57.254 INFO:tasks.workunit.client.1.smithi192.stdout:5/442: chown d2/d6/d2d 191 1 2024-04-23T02:13:57.254 INFO:tasks.workunit.client.1.smithi192.stdout:4/576: chown de/d15/d19/d3d/d49/d70/d73/fb3 3535 1 2024-04-23T02:13:57.255 INFO:tasks.workunit.client.0.smithi071.stdout:7/260: mknod d3/d17/d41/d37/d4f/c5b 0 2024-04-23T02:13:57.257 INFO:tasks.workunit.client.0.smithi071.stdout:7/261: creat d3/d17/d41/d37/d4f/f5c x:0 0 0 2024-04-23T02:13:57.259 INFO:tasks.workunit.client.0.smithi071.stdout:7/262: creat d3/d17/d1c/d31/f5d x:0 0 0 2024-04-23T02:13:57.259 INFO:tasks.workunit.client.1.smithi192.stdout:1/213: readlink dc/l1b 0 2024-04-23T02:13:57.260 INFO:tasks.workunit.client.1.smithi192.stdout:7/410: rename d5/d17/d1c/c3a to d5/d43/d4c/c77 0 2024-04-23T02:13:57.261 INFO:tasks.workunit.client.0.smithi071.stdout:4/339: dread d2/d2a/d2d/d39/f46 [0,4194304] 0 2024-04-23T02:13:57.262 INFO:tasks.workunit.client.1.smithi192.stdout:2/347: dread d8/d14/f15 [4194304,4194304] 0 2024-04-23T02:13:57.264 INFO:tasks.workunit.client.1.smithi192.stdout:3/334: getdents d1/d3/d6 0 2024-04-23T02:13:57.264 INFO:tasks.workunit.client.0.smithi071.stdout:4/340: creat d2/d2a/d2d/d39/d42/d49/d67/d1d/f6d x:0 0 0 2024-04-23T02:13:57.265 INFO:tasks.workunit.client.0.smithi071.stdout:4/341: dread - d2/d2a/d2d/f5c zero size 2024-04-23T02:13:57.266 INFO:tasks.workunit.client.1.smithi192.stdout:8/519: unlink d1/d12/d23/d4b/f5a 0 2024-04-23T02:13:57.267 INFO:tasks.workunit.client.1.smithi192.stdout:8/520: write d1/d12/d23/f60 [300927,992] 0 2024-04-23T02:13:57.268 INFO:tasks.workunit.client.1.smithi192.stdout:8/521: chown d1/f72 27 1 2024-04-23T02:13:57.269 INFO:tasks.workunit.client.0.smithi071.stdout:4/342: link d2/cb d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d29/c6e 0 2024-04-23T02:13:57.270 INFO:tasks.workunit.client.1.smithi192.stdout:7/411: symlink d5/d17/d1c/d59/d5a/d41/l78 0 2024-04-23T02:13:57.271 INFO:tasks.workunit.client.1.smithi192.stdout:7/412: dread - d5/d17/d1c/d4a/f76 zero size 2024-04-23T02:13:57.272 INFO:tasks.workunit.client.1.smithi192.stdout:2/348: chown d8/c62 83521 1 2024-04-23T02:13:57.272 INFO:tasks.workunit.client.1.smithi192.stdout:7/413: read d5/d17/d1c/f2d [1354792,100883] 0 2024-04-23T02:13:57.274 INFO:tasks.workunit.client.0.smithi071.stdout:4/343: rename d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d17/f41 to d2/d2a/d2d/d39/d42/d49/d67/d3e/f6f 0 2024-04-23T02:13:57.276 INFO:tasks.workunit.client.0.smithi071.stdout:4/344: creat d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d4f/f70 x:0 0 0 2024-04-23T02:13:57.276 INFO:tasks.workunit.client.1.smithi192.stdout:1/214: symlink dc/d16/l38 0 2024-04-23T02:13:57.277 INFO:tasks.workunit.client.0.smithi071.stdout:4/345: dread - d2/d53/f6b zero size 2024-04-23T02:13:57.277 INFO:tasks.workunit.client.1.smithi192.stdout:8/522: creat d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/fba x:0 0 0 2024-04-23T02:13:57.277 INFO:tasks.workunit.client.0.smithi071.stdout:4/346: chown d2/d2a/d2d/d39/d42/d49/d67/d1d/f6d 333106 1 2024-04-23T02:13:57.280 INFO:tasks.workunit.client.1.smithi192.stdout:7/414: unlink d5/d17/d1c/d57/f68 0 2024-04-23T02:13:57.280 INFO:tasks.workunit.client.0.smithi071.stdout:4/347: creat d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f71 x:0 0 0 2024-04-23T02:13:57.281 INFO:tasks.workunit.client.1.smithi192.stdout:2/349: symlink d8/d26/d29/d43/d50/d5e/l6d 0 2024-04-23T02:13:57.281 INFO:tasks.workunit.client.0.smithi071.stdout:4/348: fsync d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d10/f60 0 2024-04-23T02:13:57.281 INFO:tasks.workunit.client.1.smithi192.stdout:4/577: link de/d15/d19/d3d/d49/f65 de/d15/d19/d3d/d34/d7c/da1/fbe 0 2024-04-23T02:13:57.282 INFO:tasks.workunit.client.0.smithi071.stdout:4/349: write d2/d2a/d2d/d39/d42/f55 [335370,88908] 0 2024-04-23T02:13:57.283 INFO:tasks.workunit.client.1.smithi192.stdout:5/443: rename d2/db/d16/d1b/f61 to d2/db/d2b/f93 0 2024-04-23T02:13:57.285 INFO:tasks.workunit.client.0.smithi071.stdout:4/350: creat d2/d2a/d2d/d39/d42/d49/d67/d1d/d57/f72 x:0 0 0 2024-04-23T02:13:57.286 INFO:tasks.workunit.client.1.smithi192.stdout:8/523: fdatasync d1/d12/d23/d4b/d73/d45/d46/f48 0 2024-04-23T02:13:57.288 INFO:tasks.workunit.client.1.smithi192.stdout:8/524: dread - d1/d12/d23/d4b/d73/d4c/d82/fb6 zero size 2024-04-23T02:13:57.288 INFO:tasks.workunit.client.1.smithi192.stdout:7/415: mknod d5/d17/d1c/d59/d5a/d41/d50/d3d/c79 0 2024-04-23T02:13:57.289 INFO:tasks.workunit.client.1.smithi192.stdout:2/350: symlink d8/d25/l6e 0 2024-04-23T02:13:57.291 INFO:tasks.workunit.client.1.smithi192.stdout:5/444: mknod d2/d6/d2d/d41/d43/c94 0 2024-04-23T02:13:57.291 INFO:tasks.workunit.client.0.smithi071.stdout:4/351: dwrite d2/d2a/d2d/d39/d42/d49/d67/d1d/d57/f72 [0,4194304] 0 2024-04-23T02:13:57.292 INFO:tasks.workunit.client.0.smithi071.stdout:4/352: dread - d2/d2a/d2d/d39/d42/d49/d67/d3e/f4d zero size 2024-04-23T02:13:57.293 INFO:tasks.workunit.client.1.smithi192.stdout:6/479: dwrite d3/d15/d1b/f77 [0,4194304] 0 2024-04-23T02:13:57.296 INFO:tasks.workunit.client.0.smithi071.stdout:4/353: rename d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d58 to d2/d2a/d73 0 2024-04-23T02:13:57.297 INFO:tasks.workunit.client.1.smithi192.stdout:3/335: rename d1/d3/d17/l35 to d1/d3/l6f 0 2024-04-23T02:13:57.298 INFO:tasks.workunit.client.0.smithi071.stdout:4/354: readlink d2/d2a/d2d/d39/d42/d49/d67/d3e/l52 0 2024-04-23T02:13:57.301 INFO:tasks.workunit.client.0.smithi071.stdout:4/355: symlink d2/d2a/d2d/l74 0 2024-04-23T02:13:57.301 INFO:tasks.workunit.client.1.smithi192.stdout:2/351: dread d8/d25/f5c [0,4194304] 0 2024-04-23T02:13:57.301 INFO:tasks.workunit.client.1.smithi192.stdout:3/336: write d1/d3/d17/d1a/f23 [4364746,101468] 0 2024-04-23T02:13:57.301 INFO:tasks.workunit.client.1.smithi192.stdout:8/525: mknod d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/cbb 0 2024-04-23T02:13:57.304 INFO:tasks.workunit.client.1.smithi192.stdout:1/215: creat dc/f39 x:0 0 0 2024-04-23T02:13:57.305 INFO:tasks.workunit.client.1.smithi192.stdout:6/480: rmdir d3/d15/d1b 39 2024-04-23T02:13:57.305 INFO:tasks.workunit.client.0.smithi071.stdout:4/356: mknod d2/d2a/d73/d5e/c75 0 2024-04-23T02:13:57.305 INFO:tasks.workunit.client.1.smithi192.stdout:5/445: dread d2/db/d2b/f3a [0,4194304] 0 2024-04-23T02:13:57.306 INFO:tasks.workunit.client.1.smithi192.stdout:4/578: rename de/d15/d2a/db2/fa7 to de/d15/d2a/d97/fbf 0 2024-04-23T02:13:57.308 INFO:tasks.workunit.client.1.smithi192.stdout:2/352: symlink d8/d14/d51/l6f 0 2024-04-23T02:13:57.309 INFO:tasks.workunit.client.1.smithi192.stdout:6/481: chown f1 57194 1 2024-04-23T02:13:57.310 INFO:tasks.workunit.client.1.smithi192.stdout:3/337: creat d1/d3/d17/d1f/d33/d24/d34/d38/f70 x:0 0 0 2024-04-23T02:13:57.310 INFO:tasks.workunit.client.1.smithi192.stdout:1/216: creat dc/d16/d19/f3a x:0 0 0 2024-04-23T02:13:57.312 INFO:tasks.workunit.client.1.smithi192.stdout:6/482: chown d3/d11/f33 190606494 1 2024-04-23T02:13:57.312 INFO:tasks.workunit.client.1.smithi192.stdout:1/217: readlink dc/d14/d15/l25 0 2024-04-23T02:13:57.314 INFO:tasks.workunit.client.1.smithi192.stdout:4/579: truncate de/d15/f2d 2381609 0 2024-04-23T02:13:57.316 INFO:tasks.workunit.client.1.smithi192.stdout:1/218: read f3 [5723743,108860] 0 2024-04-23T02:13:57.317 INFO:tasks.workunit.client.1.smithi192.stdout:8/526: symlink d1/d12/d23/d4b/d73/d7e/d88/da9/lbc 0 2024-04-23T02:13:57.318 INFO:tasks.workunit.client.1.smithi192.stdout:7/416: getdents d5/d17/d1c/d59/d5a/d41/d50 0 2024-04-23T02:13:57.320 INFO:tasks.workunit.client.1.smithi192.stdout:8/527: fdatasync d1/d12/d23/f37 0 2024-04-23T02:13:57.320 INFO:tasks.workunit.client.1.smithi192.stdout:3/338: dread d1/d3/d17/d1f/d33/d24/f45 [0,4194304] 0 2024-04-23T02:13:57.322 INFO:tasks.workunit.client.1.smithi192.stdout:1/219: truncate dc/d16/f31 131108 0 2024-04-23T02:13:57.325 INFO:tasks.workunit.client.1.smithi192.stdout:4/580: symlink de/d15/d2a/db2/lc0 0 2024-04-23T02:13:57.326 INFO:tasks.workunit.client.1.smithi192.stdout:3/339: read - d1/d3/d17/d1f/d33/d24/f50 zero size 2024-04-23T02:13:57.326 INFO:tasks.workunit.client.1.smithi192.stdout:8/528: chown d1/d9/d3e/d59/d8a/daa 6825 1 2024-04-23T02:13:57.327 INFO:tasks.workunit.client.1.smithi192.stdout:7/417: dread - d5/d17/d1c/d59/d5a/d41/d50/f62 zero size 2024-04-23T02:13:57.328 INFO:tasks.workunit.client.1.smithi192.stdout:5/446: creat d2/f95 x:0 0 0 2024-04-23T02:13:57.329 INFO:tasks.workunit.client.1.smithi192.stdout:2/353: dwrite d8/d26/f3c [0,4194304] 0 2024-04-23T02:13:57.331 INFO:tasks.workunit.client.1.smithi192.stdout:4/581: unlink de/c29 0 2024-04-23T02:13:57.334 INFO:tasks.workunit.client.1.smithi192.stdout:5/447: dread - d2/db/d16/d1b/f79 zero size 2024-04-23T02:13:57.338 INFO:tasks.workunit.client.1.smithi192.stdout:7/418: write d5/d17/d1c/d57/f22 [173997,2539] 0 2024-04-23T02:13:57.338 INFO:tasks.workunit.client.1.smithi192.stdout:5/448: chown d2/d6/f4d 1318 1 2024-04-23T02:13:57.340 INFO:tasks.workunit.client.1.smithi192.stdout:3/340: read d1/d3/d17/f2d [177282,31474] 0 2024-04-23T02:13:57.340 INFO:tasks.workunit.client.1.smithi192.stdout:5/449: readlink d2/db/d16/l78 0 2024-04-23T02:13:57.343 INFO:tasks.workunit.client.1.smithi192.stdout:1/220: dwrite dc/d14/f21 [0,4194304] 0 2024-04-23T02:13:57.344 INFO:tasks.workunit.client.1.smithi192.stdout:1/221: stat dc/f39 0 2024-04-23T02:13:57.345 INFO:tasks.workunit.client.1.smithi192.stdout:4/582: truncate de/d15/d19/d3d/d49/d70/f91 102746 0 2024-04-23T02:13:57.346 INFO:tasks.workunit.client.1.smithi192.stdout:4/583: chown de/d15/d19/d1a/l84 495996 1 2024-04-23T02:13:57.349 INFO:tasks.workunit.client.1.smithi192.stdout:1/222: symlink dc/d14/d15/d35/l3b 0 2024-04-23T02:13:57.351 INFO:tasks.workunit.client.1.smithi192.stdout:8/529: truncate d1/d9/f22 4288516 0 2024-04-23T02:13:57.352 INFO:tasks.workunit.client.1.smithi192.stdout:3/341: mkdir d1/d3/d6/d7/d71 0 2024-04-23T02:13:57.352 INFO:tasks.workunit.client.1.smithi192.stdout:1/223: dread - dc/d16/d19/f3a zero size 2024-04-23T02:13:57.353 INFO:tasks.workunit.client.1.smithi192.stdout:4/584: symlink de/d15/d2a/d74/d92/da0/lc1 0 2024-04-23T02:13:57.355 INFO:tasks.workunit.client.1.smithi192.stdout:8/530: fdatasync d1/d9/d8f/d90/f93 0 2024-04-23T02:13:57.355 INFO:tasks.workunit.client.1.smithi192.stdout:5/450: creat d2/db/d16/f96 x:0 0 0 2024-04-23T02:13:57.361 INFO:tasks.workunit.client.1.smithi192.stdout:6/483: dwrite d3/d28/f96 [0,4194304] 0 2024-04-23T02:13:57.366 INFO:tasks.workunit.client.1.smithi192.stdout:4/585: write de/f1b [998697,69147] 0 2024-04-23T02:13:57.366 INFO:tasks.workunit.client.1.smithi192.stdout:8/531: fdatasync d1/d12/f39 0 2024-04-23T02:13:57.368 INFO:tasks.workunit.client.1.smithi192.stdout:9/364: sync 2024-04-23T02:13:57.369 INFO:tasks.workunit.client.1.smithi192.stdout:3/342: creat d1/d3/d6/d7/f72 x:0 0 0 2024-04-23T02:13:57.370 INFO:tasks.workunit.client.1.smithi192.stdout:4/586: read f4 [2893991,115056] 0 2024-04-23T02:13:57.374 INFO:tasks.workunit.client.1.smithi192.stdout:1/224: dwrite dc/ff [4194304,4194304] 0 2024-04-23T02:13:57.393 INFO:tasks.workunit.client.0.smithi071.stdout:6/279: sync 2024-04-23T02:13:57.395 INFO:tasks.workunit.client.0.smithi071.stdout:6/280: symlink d6/d1f/d46/l59 0 2024-04-23T02:13:57.396 INFO:tasks.workunit.client.0.smithi071.stdout:6/281: readlink d6/d22/l2b 0 2024-04-23T02:13:57.397 INFO:tasks.workunit.client.0.smithi071.stdout:6/282: fdatasync d6/f9 0 2024-04-23T02:13:57.399 INFO:tasks.workunit.client.0.smithi071.stdout:6/283: unlink d6/d1f/l50 0 2024-04-23T02:13:57.400 INFO:tasks.workunit.client.0.smithi071.stdout:6/284: stat d6/d1f/l44 0 2024-04-23T02:13:57.400 INFO:tasks.workunit.client.0.smithi071.stdout:6/285: write d6/fc [1845298,6731] 0 2024-04-23T02:13:57.427 INFO:tasks.workunit.client.1.smithi192.stdout:2/354: dread d8/d25/f32 [0,4194304] 0 2024-04-23T02:13:57.428 INFO:tasks.workunit.client.1.smithi192.stdout:1/225: write dc/ff [8010726,54415] 0 2024-04-23T02:13:57.431 INFO:tasks.workunit.client.1.smithi192.stdout:3/343: fdatasync d1/d3/d17/d2b/f32 0 2024-04-23T02:13:57.433 INFO:tasks.workunit.client.1.smithi192.stdout:5/451: creat d2/d6/d2d/d41/f97 x:0 0 0 2024-04-23T02:13:57.434 INFO:tasks.workunit.client.1.smithi192.stdout:8/532: symlink d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/lbd 0 2024-04-23T02:13:57.437 INFO:tasks.workunit.client.1.smithi192.stdout:6/484: dread - d3/d15/d1b/f8f zero size 2024-04-23T02:13:57.438 INFO:tasks.workunit.client.1.smithi192.stdout:9/365: mknod d0/d9/c87 0 2024-04-23T02:13:57.438 INFO:tasks.workunit.client.1.smithi192.stdout:4/587: dwrite de/d15/d19/d3d/d49/d26/fa8 [0,4194304] 0 2024-04-23T02:13:57.440 INFO:tasks.workunit.client.1.smithi192.stdout:2/355: dread d8/d25/f6c [0,4194304] 0 2024-04-23T02:13:57.440 INFO:tasks.workunit.client.1.smithi192.stdout:4/588: fsync de/f21 0 2024-04-23T02:13:57.441 INFO:tasks.workunit.client.1.smithi192.stdout:6/485: truncate d3/d18/f6f 4832426 0 2024-04-23T02:13:57.444 INFO:tasks.workunit.client.1.smithi192.stdout:1/226: mknod dc/d16/c3c 0 2024-04-23T02:13:57.445 INFO:tasks.workunit.client.1.smithi192.stdout:8/533: rename d1/l21 to d1/d9/d8f/lbe 0 2024-04-23T02:13:57.446 INFO:tasks.workunit.client.1.smithi192.stdout:3/344: mknod d1/d3/d6/d7/d71/c73 0 2024-04-23T02:13:57.449 INFO:tasks.workunit.client.1.smithi192.stdout:9/366: unlink d0/d9/d44/d74/f4c 0 2024-04-23T02:13:57.450 INFO:tasks.workunit.client.1.smithi192.stdout:1/227: write dc/f13 [1262,1642] 0 2024-04-23T02:13:57.453 INFO:tasks.workunit.client.1.smithi192.stdout:2/356: rename d8/d26/d29/d43/d50/f55 to d8/d26/d27/f70 0 2024-04-23T02:13:57.455 INFO:tasks.workunit.client.1.smithi192.stdout:8/534: mknod d1/d12/d23/d4b/d73/d7e/d88/cbf 0 2024-04-23T02:13:57.456 INFO:tasks.workunit.client.1.smithi192.stdout:2/357: fdatasync d8/d26/d27/f45 0 2024-04-23T02:13:57.458 INFO:tasks.workunit.client.1.smithi192.stdout:4/589: creat de/d15/d2a/d80/fc2 x:0 0 0 2024-04-23T02:13:57.459 INFO:tasks.workunit.client.1.smithi192.stdout:6/486: dwrite d3/d11/f29 [0,4194304] 0 2024-04-23T02:13:57.459 INFO:tasks.workunit.client.1.smithi192.stdout:6/487: stat d3/d11/l14 0 2024-04-23T02:13:57.459 INFO:tasks.workunit.client.1.smithi192.stdout:4/590: chown de/c7a 13 1 2024-04-23T02:13:57.461 INFO:tasks.workunit.client.1.smithi192.stdout:1/228: creat dc/d14/d23/f3d x:0 0 0 2024-04-23T02:13:57.463 INFO:tasks.workunit.client.1.smithi192.stdout:3/345: rename d1/d3/d17/d1a/l26 to d1/d3/l74 0 2024-04-23T02:13:57.466 INFO:tasks.workunit.client.1.smithi192.stdout:3/346: write d1/d3/d17/d1f/d33/d24/d34/d40/d46/f5e [62014,97820] 0 2024-04-23T02:13:57.467 INFO:tasks.workunit.client.1.smithi192.stdout:8/535: creat d1/d9/d8f/fc0 x:0 0 0 2024-04-23T02:13:57.468 INFO:tasks.workunit.client.1.smithi192.stdout:2/358: creat d8/d25/f71 x:0 0 0 2024-04-23T02:13:57.469 INFO:tasks.workunit.client.1.smithi192.stdout:2/359: stat d8/d26/d27/l39 0 2024-04-23T02:13:57.469 INFO:tasks.workunit.client.1.smithi192.stdout:6/488: stat d3/d15/d1b/f2a 0 2024-04-23T02:13:57.473 INFO:tasks.workunit.client.1.smithi192.stdout:3/347: symlink d1/d3/d17/d1f/d33/d24/d34/l75 0 2024-04-23T02:13:57.474 INFO:tasks.workunit.client.1.smithi192.stdout:2/360: dread d8/d26/d27/d53/f54 [0,4194304] 0 2024-04-23T02:13:57.475 INFO:tasks.workunit.client.1.smithi192.stdout:3/348: chown d1/d3/d6/d7/f72 168 1 2024-04-23T02:13:57.476 INFO:tasks.workunit.client.1.smithi192.stdout:2/361: write d8/d25/f3a [3769207,35801] 0 2024-04-23T02:13:57.476 INFO:tasks.workunit.client.1.smithi192.stdout:9/367: getdents d0/d9/d44/d58 0 2024-04-23T02:13:57.478 INFO:tasks.workunit.client.1.smithi192.stdout:1/229: link dc/f13 dc/d16/d19/d30/d32/d34/f3e 0 2024-04-23T02:13:57.480 INFO:tasks.workunit.client.1.smithi192.stdout:1/230: readlink dc/d14/d15/l25 0 2024-04-23T02:13:57.480 INFO:tasks.workunit.client.1.smithi192.stdout:6/489: write d3/d15/f4b [1225605,122429] 0 2024-04-23T02:13:57.482 INFO:tasks.workunit.client.1.smithi192.stdout:2/362: symlink d8/d26/l72 0 2024-04-23T02:13:57.483 INFO:tasks.workunit.client.1.smithi192.stdout:1/231: dread dc/d16/f31 [0,4194304] 0 2024-04-23T02:13:57.483 INFO:tasks.workunit.client.1.smithi192.stdout:3/349: dread d1/d3/f19 [0,4194304] 0 2024-04-23T02:13:57.485 INFO:tasks.workunit.client.1.smithi192.stdout:8/536: creat d1/d12/fc1 x:0 0 0 2024-04-23T02:13:57.486 INFO:tasks.workunit.client.1.smithi192.stdout:9/368: mknod d0/d9/d44/d74/c88 0 2024-04-23T02:13:57.488 INFO:tasks.workunit.client.1.smithi192.stdout:6/490: mkdir d3/d15/d1b/d66/d80/d8e/d9d 0 2024-04-23T02:13:57.488 INFO:tasks.workunit.client.1.smithi192.stdout:4/591: truncate de/d15/d2a/f5c 346070 0 2024-04-23T02:13:57.489 INFO:tasks.workunit.client.1.smithi192.stdout:3/350: creat d1/d3/d17/d2b/f76 x:0 0 0 2024-04-23T02:13:57.490 INFO:tasks.workunit.client.1.smithi192.stdout:3/351: chown d1/d3/d17/d1f/d33 56227 1 2024-04-23T02:13:57.491 INFO:tasks.workunit.client.1.smithi192.stdout:2/363: rename d8/d25/f5c to d8/d26/d29/d43/d5d/f73 0 2024-04-23T02:13:57.493 INFO:tasks.workunit.client.1.smithi192.stdout:9/369: mkdir d0/d9/d44/d58/d89 0 2024-04-23T02:13:57.494 INFO:tasks.workunit.client.1.smithi192.stdout:9/370: stat d0/d9/d44/d51/d5b/d61/f6a 0 2024-04-23T02:13:57.496 INFO:tasks.workunit.client.1.smithi192.stdout:3/352: readlink d1/d3/d17/d1f/d33/l28 0 2024-04-23T02:13:57.498 INFO:tasks.workunit.client.1.smithi192.stdout:1/232: rename f0 to dc/d16/d19/d30/f3f 0 2024-04-23T02:13:57.499 INFO:tasks.workunit.client.1.smithi192.stdout:2/364: mkdir d8/d26/d74 0 2024-04-23T02:13:57.500 INFO:tasks.workunit.client.1.smithi192.stdout:8/537: mknod d1/d9/d3e/d59/d8a/daa/cc2 0 2024-04-23T02:13:57.501 INFO:tasks.workunit.client.1.smithi192.stdout:6/491: creat d3/d15/d1b/d21/d52/d64/d8d/f9e x:0 0 0 2024-04-23T02:13:57.504 INFO:tasks.workunit.client.1.smithi192.stdout:8/538: write d1/d12/d23/d4b/d73/d4c/f84 [1249193,31621] 0 2024-04-23T02:13:57.507 INFO:tasks.workunit.client.1.smithi192.stdout:3/353: symlink d1/d3/d6/d7/d29/d3b/l77 0 2024-04-23T02:13:57.512 INFO:tasks.workunit.client.1.smithi192.stdout:4/592: write de/d15/d53/f58 [634347,1787] 0 2024-04-23T02:13:57.512 INFO:tasks.workunit.client.1.smithi192.stdout:2/365: dwrite d8/d26/f3c [0,4194304] 0 2024-04-23T02:13:57.513 INFO:tasks.workunit.client.1.smithi192.stdout:3/354: fsync d1/d3/d17/d1f/d33/d24/d34/f54 0 2024-04-23T02:13:57.513 INFO:tasks.workunit.client.1.smithi192.stdout:9/371: dread d0/d9/d44/f47 [0,4194304] 0 2024-04-23T02:13:57.515 INFO:tasks.workunit.client.1.smithi192.stdout:8/539: rename d1/f3 to d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/fc3 0 2024-04-23T02:13:57.518 INFO:tasks.workunit.client.1.smithi192.stdout:2/366: read - d8/d26/f6b zero size 2024-04-23T02:13:57.520 INFO:tasks.workunit.client.1.smithi192.stdout:2/367: write d8/d25/f3a [3312321,95433] 0 2024-04-23T02:13:57.524 INFO:tasks.workunit.client.1.smithi192.stdout:9/372: creat d0/d9/d44/d74/f8a x:0 0 0 2024-04-23T02:13:57.527 INFO:tasks.workunit.client.1.smithi192.stdout:4/593: mkdir de/d15/dc3 0 2024-04-23T02:13:57.528 INFO:tasks.workunit.client.1.smithi192.stdout:6/492: dread d3/f22 [0,4194304] 0 2024-04-23T02:13:57.529 INFO:tasks.workunit.client.0.smithi071.stdout:9/310: sync 2024-04-23T02:13:57.529 INFO:tasks.workunit.client.0.smithi071.stdout:0/270: sync 2024-04-23T02:13:57.530 INFO:tasks.workunit.client.0.smithi071.stdout:9/311: dread - d1/d1a/d39/d42/f49 zero size 2024-04-23T02:13:57.530 INFO:tasks.workunit.client.0.smithi071.stdout:9/312: readlink d1/l2a 0 2024-04-23T02:13:57.531 INFO:tasks.workunit.client.0.smithi071.stdout:0/271: truncate d0/d3/de/d10/d14/d32/f35 1188325 0 2024-04-23T02:13:57.531 INFO:tasks.workunit.client.1.smithi192.stdout:2/368: link d8/l2b d8/d26/d27/d53/l75 0 2024-04-23T02:13:57.532 INFO:tasks.workunit.client.0.smithi071.stdout:9/313: symlink d1/d4e/d54/l64 0 2024-04-23T02:13:57.533 INFO:tasks.workunit.client.1.smithi192.stdout:1/233: dwrite dc/d16/d19/d30/d32/d34/f3e [0,4194304] 0 2024-04-23T02:13:57.533 INFO:tasks.workunit.client.1.smithi192.stdout:1/234: readlink dc/d16/l38 0 2024-04-23T02:13:57.534 INFO:tasks.workunit.client.0.smithi071.stdout:9/314: write d1/d1a/f25 [918383,13359] 0 2024-04-23T02:13:57.535 INFO:tasks.workunit.client.1.smithi192.stdout:4/594: creat de/d15/d19/d3d/d34/d4a/fc4 x:0 0 0 2024-04-23T02:13:57.536 INFO:tasks.workunit.client.0.smithi071.stdout:0/272: dread d0/d3/d1c/f21 [0,4194304] 0 2024-04-23T02:13:57.537 INFO:tasks.workunit.client.0.smithi071.stdout:9/315: mknod d1/d1a/d39/d42/c65 0 2024-04-23T02:13:57.538 INFO:tasks.workunit.client.1.smithi192.stdout:3/355: write d1/d3/d17/d2b/f32 [424536,5019] 0 2024-04-23T02:13:57.538 INFO:tasks.workunit.client.0.smithi071.stdout:9/316: fdatasync d1/f50 0 2024-04-23T02:13:57.539 INFO:tasks.workunit.client.1.smithi192.stdout:2/369: chown d8/lc 100143801 1 2024-04-23T02:13:57.539 INFO:tasks.workunit.client.1.smithi192.stdout:3/356: chown d1/d3/d17/d2b/f51 32 1 2024-04-23T02:13:57.539 INFO:tasks.workunit.client.1.smithi192.stdout:9/373: rmdir d0/d9/d44/d51/d5b/d6d 0 2024-04-23T02:13:57.540 INFO:tasks.workunit.client.0.smithi071.stdout:0/273: mknod d0/d3/de/d10/d14/d32/d37/d26/c54 0 2024-04-23T02:13:57.540 INFO:tasks.workunit.client.0.smithi071.stdout:9/317: truncate d1/f50 452009 0 2024-04-23T02:13:57.541 INFO:tasks.workunit.client.1.smithi192.stdout:9/374: dread - d0/d9/f32 zero size 2024-04-23T02:13:57.541 INFO:tasks.workunit.client.1.smithi192.stdout:4/595: mkdir de/d15/d53/dc5 0 2024-04-23T02:13:57.542 INFO:tasks.workunit.client.1.smithi192.stdout:9/375: stat d0/d9/d44/f7c 0 2024-04-23T02:13:57.543 INFO:tasks.workunit.client.0.smithi071.stdout:0/274: creat d0/d3/de/d10/d16/f55 x:0 0 0 2024-04-23T02:13:57.544 INFO:tasks.workunit.client.1.smithi192.stdout:1/235: link dc/d16/f1f dc/d14/d15/d35/f40 0 2024-04-23T02:13:57.544 INFO:tasks.workunit.client.0.smithi071.stdout:9/318: unlink d1/d1a/f23 0 2024-04-23T02:13:57.544 INFO:tasks.workunit.client.0.smithi071.stdout:0/275: rename d0/d3/de to d0/d3/de/d10/d16/d56 22 2024-04-23T02:13:57.545 INFO:tasks.workunit.client.1.smithi192.stdout:2/370: creat d8/d26/d29/d43/d50/f76 x:0 0 0 2024-04-23T02:13:57.546 INFO:tasks.workunit.client.0.smithi071.stdout:0/276: dread - d0/d3/de/d10/d14/f4d zero size 2024-04-23T02:13:57.546 INFO:tasks.workunit.client.1.smithi192.stdout:3/357: creat d1/d3/d17/d2b/d44/d57/f78 x:0 0 0 2024-04-23T02:13:57.547 INFO:tasks.workunit.client.1.smithi192.stdout:6/493: write d3/d15/d1b/d66/d80/d8e/f50 [76318,58873] 0 2024-04-23T02:13:57.548 INFO:tasks.workunit.client.0.smithi071.stdout:0/277: truncate d0/d3/de/d10/d14/f51 965730 0 2024-04-23T02:13:57.548 INFO:tasks.workunit.client.0.smithi071.stdout:9/319: dread d1/d1a/d5a/f5e [0,4194304] 0 2024-04-23T02:13:57.550 INFO:tasks.workunit.client.0.smithi071.stdout:9/320: mkdir d1/d4e/d66 0 2024-04-23T02:13:57.551 INFO:tasks.workunit.client.1.smithi192.stdout:1/236: symlink dc/d16/d19/d30/l41 0 2024-04-23T02:13:57.552 INFO:tasks.workunit.client.0.smithi071.stdout:9/321: symlink d1/d1a/d39/l67 0 2024-04-23T02:13:57.553 INFO:tasks.workunit.client.1.smithi192.stdout:8/540: dread d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/fc3 [0,4194304] 0 2024-04-23T02:13:57.553 INFO:tasks.workunit.client.1.smithi192.stdout:4/596: creat de/d15/d19/d3d/d49/d93/fc6 x:0 0 0 2024-04-23T02:13:57.554 INFO:tasks.workunit.client.0.smithi071.stdout:9/322: fsync d1/d1a/d5a/f5e 0 2024-04-23T02:13:57.555 INFO:tasks.workunit.client.1.smithi192.stdout:3/358: chown d1/d3/d17/d1f/d33/d24/d34/d40/d46/f5e 0 1 2024-04-23T02:13:57.555 INFO:tasks.workunit.client.1.smithi192.stdout:6/494: readlink d3/d15/d5b/l62 0 2024-04-23T02:13:57.555 INFO:tasks.workunit.client.1.smithi192.stdout:8/541: readlink d1/d9/l3a 0 2024-04-23T02:13:57.555 INFO:tasks.workunit.client.1.smithi192.stdout:1/237: write dc/d16/d19/d30/f3f [7990238,18154] 0 2024-04-23T02:13:57.556 INFO:tasks.workunit.client.0.smithi071.stdout:9/323: mkdir d1/d1a/d68 0 2024-04-23T02:13:57.556 INFO:tasks.workunit.client.1.smithi192.stdout:2/371: rename d8/c11 to d8/d26/d29/c77 0 2024-04-23T02:13:57.557 INFO:tasks.workunit.client.1.smithi192.stdout:1/238: write dc/d16/d19/d30/d32/f33 [775762,74847] 0 2024-04-23T02:13:57.557 INFO:tasks.workunit.client.0.smithi071.stdout:9/324: creat d1/d4e/d54/f69 x:0 0 0 2024-04-23T02:13:57.558 INFO:tasks.workunit.client.1.smithi192.stdout:3/359: creat d1/d3/d6/f79 x:0 0 0 2024-04-23T02:13:57.560 INFO:tasks.workunit.client.1.smithi192.stdout:6/495: creat d3/d18/f9f x:0 0 0 2024-04-23T02:13:57.560 INFO:tasks.workunit.client.0.smithi071.stdout:9/325: creat d1/d1a/d5a/f6a x:0 0 0 2024-04-23T02:13:57.561 INFO:tasks.workunit.client.1.smithi192.stdout:3/360: fdatasync d1/d3/d17/d2b/f32 0 2024-04-23T02:13:57.561 INFO:tasks.workunit.client.1.smithi192.stdout:1/239: dread dc/d16/f31 [0,4194304] 0 2024-04-23T02:13:57.563 INFO:tasks.workunit.client.0.smithi071.stdout:0/278: dwrite d0/d3/d1c/f21 [0,4194304] 0 2024-04-23T02:13:57.563 INFO:tasks.workunit.client.1.smithi192.stdout:2/372: rmdir d8/d26/d29/d43/d50/d5e 39 2024-04-23T02:13:57.566 INFO:tasks.workunit.client.1.smithi192.stdout:2/373: dread - d8/d26/d29/f41 zero size 2024-04-23T02:13:57.567 INFO:tasks.workunit.client.0.smithi071.stdout:9/326: link d1/d1a/d39/f53 d1/d1a/d5a/f6b 0 2024-04-23T02:13:57.568 INFO:tasks.workunit.client.0.smithi071.stdout:9/327: chown d1/d1a/l3e 235 1 2024-04-23T02:13:57.568 INFO:tasks.workunit.client.1.smithi192.stdout:3/361: chown d1/d3/d17/d1f/d33/d24/d34/d40/d46/f5e 45 1 2024-04-23T02:13:57.568 INFO:tasks.workunit.client.1.smithi192.stdout:4/597: link de/d15/d19/d3d/d49/f60 de/d15/d19/d1a/fc7 0 2024-04-23T02:13:57.569 INFO:tasks.workunit.client.0.smithi071.stdout:0/279: creat d0/d3/de/d10/f57 x:0 0 0 2024-04-23T02:13:57.570 INFO:tasks.workunit.client.0.smithi071.stdout:9/328: mknod d1/d4e/d54/c6c 0 2024-04-23T02:13:57.571 INFO:tasks.workunit.client.0.smithi071.stdout:0/280: write d0/d3/de/d10/d14/d32/d37/d26/f4b [851654,22840] 0 2024-04-23T02:13:57.571 INFO:tasks.workunit.client.1.smithi192.stdout:6/496: dwrite d3/d11/f13 [8388608,4194304] 0 2024-04-23T02:13:57.575 INFO:tasks.workunit.client.1.smithi192.stdout:9/376: write d0/d9/f50 [12503,79257] 0 2024-04-23T02:13:57.577 INFO:tasks.workunit.client.0.smithi071.stdout:1/295: sync 2024-04-23T02:13:57.577 INFO:tasks.workunit.client.0.smithi071.stdout:7/263: sync 2024-04-23T02:13:57.577 INFO:tasks.workunit.client.0.smithi071.stdout:3/271: sync 2024-04-23T02:13:57.577 INFO:tasks.workunit.client.0.smithi071.stdout:2/293: sync 2024-04-23T02:13:57.577 INFO:tasks.workunit.client.0.smithi071.stdout:5/413: sync 2024-04-23T02:13:57.577 INFO:tasks.workunit.client.0.smithi071.stdout:8/410: sync 2024-04-23T02:13:57.578 INFO:tasks.workunit.client.0.smithi071.stdout:4/357: sync 2024-04-23T02:13:57.578 INFO:tasks.workunit.client.0.smithi071.stdout:6/286: sync 2024-04-23T02:13:57.578 INFO:tasks.workunit.client.0.smithi071.stdout:5/414: chown d3 12 1 2024-04-23T02:13:57.579 INFO:tasks.workunit.client.1.smithi192.stdout:4/598: creat de/d15/dbb/fc8 x:0 0 0 2024-04-23T02:13:57.582 INFO:tasks.workunit.client.0.smithi071.stdout:8/411: readlink dc/d3a/l76 0 2024-04-23T02:13:57.582 INFO:tasks.workunit.client.0.smithi071.stdout:7/264: write d3/d17/f1f [3538684,129886] 0 2024-04-23T02:13:57.583 INFO:tasks.workunit.client.1.smithi192.stdout:6/497: mkdir d3/d28/d79/da0 0 2024-04-23T02:13:57.583 INFO:tasks.workunit.client.1.smithi192.stdout:1/240: link dc/d16/l38 dc/d14/d23/l42 0 2024-04-23T02:13:57.584 INFO:tasks.workunit.client.0.smithi071.stdout:5/415: rmdir d3/d1d/d43 39 2024-04-23T02:13:57.584 INFO:tasks.workunit.client.0.smithi071.stdout:0/281: dwrite d0/d3/de/d10/d16/f55 [0,4194304] 0 2024-04-23T02:13:57.585 INFO:tasks.workunit.client.1.smithi192.stdout:9/377: read d0/f2c [2944167,108404] 0 2024-04-23T02:13:57.586 INFO:tasks.workunit.client.0.smithi071.stdout:4/358: stat d2/d2a/d2d/d39/d42/d49/d67/d1d/c47 0 2024-04-23T02:13:57.586 INFO:tasks.workunit.client.0.smithi071.stdout:2/294: write d0/d2/f9 [2242664,74793] 0 2024-04-23T02:13:57.587 INFO:tasks.workunit.client.1.smithi192.stdout:2/374: getdents d8/d26/d29/d43/d50 0 2024-04-23T02:13:57.587 INFO:tasks.workunit.client.0.smithi071.stdout:1/296: write db/d34/f59 [2876750,116816] 0 2024-04-23T02:13:57.588 INFO:tasks.workunit.client.1.smithi192.stdout:1/241: mknod dc/d14/c43 0 2024-04-23T02:13:57.588 INFO:tasks.workunit.client.0.smithi071.stdout:6/287: write d6/d1f/f21 [4333334,79954] 0 2024-04-23T02:13:57.590 INFO:tasks.workunit.client.0.smithi071.stdout:3/272: creat d7/d28/d3b/d4c/f59 x:0 0 0 2024-04-23T02:13:57.591 INFO:tasks.workunit.client.1.smithi192.stdout:6/498: creat d3/d15/d1b/d21/fa1 x:0 0 0 2024-04-23T02:13:57.591 INFO:tasks.workunit.client.0.smithi071.stdout:4/359: chown d2/c3f 1911736944 1 2024-04-23T02:13:57.592 INFO:tasks.workunit.client.0.smithi071.stdout:2/295: chown d0/d2/d26/df/f40 1768319 1 2024-04-23T02:13:57.592 INFO:tasks.workunit.client.1.smithi192.stdout:3/362: dwrite d1/d3/f19 [0,4194304] 0 2024-04-23T02:13:57.592 INFO:tasks.workunit.client.1.smithi192.stdout:4/599: mknod de/d15/d2a/d61/cc9 0 2024-04-23T02:13:57.593 INFO:tasks.workunit.client.0.smithi071.stdout:1/297: rmdir db/d1a/d57 39 2024-04-23T02:13:57.596 INFO:tasks.workunit.client.1.smithi192.stdout:2/375: write d8/d14/f1a [4081431,52266] 0 2024-04-23T02:13:57.597 INFO:tasks.workunit.client.1.smithi192.stdout:2/376: write d8/d26/d27/f45 [291495,20502] 0 2024-04-23T02:13:57.598 INFO:tasks.workunit.client.0.smithi071.stdout:6/288: creat d6/d1f/d46/f5a x:0 0 0 2024-04-23T02:13:57.599 INFO:tasks.workunit.client.0.smithi071.stdout:0/282: symlink d0/d3/de/d10/d14/l58 0 2024-04-23T02:13:57.599 INFO:tasks.workunit.client.0.smithi071.stdout:2/296: dread - d0/d2/d34/f50 zero size 2024-04-23T02:13:57.600 INFO:tasks.workunit.client.1.smithi192.stdout:8/542: truncate d1/d12/d23/d4b/d73/d45/d46/f53 3381546 0 2024-04-23T02:13:57.600 INFO:tasks.workunit.client.0.smithi071.stdout:3/273: chown d7/d28/l54 34 1 2024-04-23T02:13:57.601 INFO:tasks.workunit.client.1.smithi192.stdout:2/377: write d8/d25/f3f [644209,18856] 0 2024-04-23T02:13:57.602 INFO:tasks.workunit.client.0.smithi071.stdout:1/298: write db/df/d3a/f47 [532597,87081] 0 2024-04-23T02:13:57.602 INFO:tasks.workunit.client.0.smithi071.stdout:7/265: dwrite d3/d17/f5a [0,4194304] 0 2024-04-23T02:13:57.602 INFO:tasks.workunit.client.1.smithi192.stdout:8/543: write d1/d9/d3e/fb2 [550189,11395] 0 2024-04-23T02:13:57.606 INFO:tasks.workunit.client.0.smithi071.stdout:5/416: dread d3/df/f24 [0,4194304] 0 2024-04-23T02:13:57.606 INFO:tasks.workunit.client.0.smithi071.stdout:4/360: creat d2/d2a/d2d/d39/d42/d49/d67/d3e/f76 x:0 0 0 2024-04-23T02:13:57.609 INFO:tasks.workunit.client.0.smithi071.stdout:7/266: creat d3/d17/d41/f5e x:0 0 0 2024-04-23T02:13:57.611 INFO:tasks.workunit.client.1.smithi192.stdout:4/600: rename de/d15/d19/d3d to de/d15/d2a/d74/d92/da0/dca 0 2024-04-23T02:13:57.612 INFO:tasks.workunit.client.0.smithi071.stdout:8/412: write dc/d10/f14 [4476299,28489] 0 2024-04-23T02:13:57.612 INFO:tasks.workunit.client.0.smithi071.stdout:6/289: rmdir d6/d1f/d46 39 2024-04-23T02:13:57.613 INFO:tasks.workunit.client.1.smithi192.stdout:2/378: creat d8/d26/f78 x:0 0 0 2024-04-23T02:13:57.614 INFO:tasks.workunit.client.0.smithi071.stdout:0/283: read d0/d3/de/d10/d14/d32/f45 [1454916,7785] 0 2024-04-23T02:13:57.616 INFO:tasks.workunit.client.0.smithi071.stdout:8/413: fsync dc/f54 0 2024-04-23T02:13:57.616 INFO:tasks.workunit.client.0.smithi071.stdout:6/290: readlink d6/d1f/l57 0 2024-04-23T02:13:57.617 INFO:tasks.workunit.client.1.smithi192.stdout:6/499: mkdir d3/d15/d1b/da2 0 2024-04-23T02:13:57.618 INFO:tasks.workunit.client.0.smithi071.stdout:5/417: chown d3/df/d10/f60 7623815 1 2024-04-23T02:13:57.619 INFO:tasks.workunit.client.1.smithi192.stdout:4/601: rmdir de/d15/d2a/d74/d92/da0/dca/db6 39 2024-04-23T02:13:57.620 INFO:tasks.workunit.client.1.smithi192.stdout:8/544: unlink d1/d12/d23/d2c/d4d/c9e 0 2024-04-23T02:13:57.622 INFO:tasks.workunit.client.0.smithi071.stdout:7/267: mkdir d3/d17/d41/d34/d59/d5f 0 2024-04-23T02:13:57.622 INFO:tasks.workunit.client.1.smithi192.stdout:9/378: getdents d0/d9/d44/d74/d3c/d5f 0 2024-04-23T02:13:57.623 INFO:tasks.workunit.client.0.smithi071.stdout:8/414: creat dc/d15/d50/f82 x:0 0 0 2024-04-23T02:13:57.623 INFO:tasks.workunit.client.1.smithi192.stdout:6/500: symlink d3/d15/d1b/d66/d80/la3 0 2024-04-23T02:13:57.624 INFO:tasks.workunit.client.1.smithi192.stdout:4/602: read de/d15/d19/d1a/f1e [205138,27142] 0 2024-04-23T02:13:57.625 INFO:tasks.workunit.client.0.smithi071.stdout:3/274: dwrite d7/d8/fe [4194304,4194304] 0 2024-04-23T02:13:57.626 INFO:tasks.workunit.client.1.smithi192.stdout:9/379: chown d0/d9/l72 75092649 1 2024-04-23T02:13:57.626 INFO:tasks.workunit.client.1.smithi192.stdout:6/501: dread - d3/d11/f6b zero size 2024-04-23T02:13:57.626 INFO:tasks.workunit.client.0.smithi071.stdout:4/361: write d2/d2a/d2d/d39/d42/d49/d67/d1d/d57/f72 [3239285,116965] 0 2024-04-23T02:13:57.627 INFO:tasks.workunit.client.1.smithi192.stdout:8/545: read - d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d8b/f99 zero size 2024-04-23T02:13:57.627 INFO:tasks.workunit.client.0.smithi071.stdout:6/291: read d6/d1f/f3b [990869,21323] 0 2024-04-23T02:13:57.627 INFO:tasks.workunit.client.1.smithi192.stdout:1/242: truncate dc/d14/f21 828335 0 2024-04-23T02:13:57.628 INFO:tasks.workunit.client.0.smithi071.stdout:7/268: stat d3/d17/d1c/d31 0 2024-04-23T02:13:57.630 INFO:tasks.workunit.client.0.smithi071.stdout:9/329: dread d1/d1a/f22 [0,4194304] 0 2024-04-23T02:13:57.631 INFO:tasks.workunit.client.0.smithi071.stdout:2/297: dwrite d0/d2/d34/f50 [0,4194304] 0 2024-04-23T02:13:57.632 INFO:tasks.workunit.client.0.smithi071.stdout:5/418: creat d3/d1d/d43/d4b/fa0 x:0 0 0 2024-04-23T02:13:57.633 INFO:tasks.workunit.client.1.smithi192.stdout:9/380: dread d0/d9/d44/f5e [0,4194304] 0 2024-04-23T02:13:57.634 INFO:tasks.workunit.client.1.smithi192.stdout:1/243: chown dc/d16/d19/f3a 38 1 2024-04-23T02:13:57.635 INFO:tasks.workunit.client.1.smithi192.stdout:0/501: sync 2024-04-23T02:13:57.640 INFO:tasks.workunit.client.0.smithi071.stdout:5/419: mknod d3/df/d10/ca1 0 2024-04-23T02:13:57.640 INFO:tasks.workunit.client.0.smithi071.stdout:1/299: dwrite db/d1a/d57/f5b [0,4194304] 0 2024-04-23T02:13:57.640 INFO:tasks.workunit.client.0.smithi071.stdout:9/330: truncate d1/d1a/d39/d42/f49 799993 0 2024-04-23T02:13:57.643 INFO:tasks.workunit.client.0.smithi071.stdout:2/298: creat d0/d2/d3c/f5f x:0 0 0 2024-04-23T02:13:57.644 INFO:tasks.workunit.client.0.smithi071.stdout:3/275: truncate d7/d8/f16 5191658 0 2024-04-23T02:13:57.647 INFO:tasks.workunit.client.0.smithi071.stdout:6/292: write d6/d1f/d46/f5a [440964,127156] 0 2024-04-23T02:13:57.649 INFO:tasks.workunit.client.1.smithi192.stdout:6/502: dread d3/f57 [0,4194304] 0 2024-04-23T02:13:57.649 INFO:tasks.workunit.client.1.smithi192.stdout:6/503: fdatasync d3/f5c 0 2024-04-23T02:13:57.649 INFO:tasks.workunit.client.1.smithi192.stdout:9/381: dread - d0/d9/f32 zero size 2024-04-23T02:13:57.649 INFO:tasks.workunit.client.1.smithi192.stdout:0/502: write d6/df/d47/f89 [1601201,36184] 0 2024-04-23T02:13:57.650 INFO:tasks.workunit.client.0.smithi071.stdout:9/331: mknod d1/d4e/d66/c6d 0 2024-04-23T02:13:57.650 INFO:tasks.workunit.client.0.smithi071.stdout:3/276: symlink d7/d27/d3e/l5a 0 2024-04-23T02:13:57.650 INFO:tasks.workunit.client.1.smithi192.stdout:4/603: dwrite de/d15/d2a/d74/d92/da0/dca/d49/d70/f91 [0,4194304] 0 2024-04-23T02:13:57.652 INFO:tasks.workunit.client.0.smithi071.stdout:2/299: creat d0/d2/d45/f60 x:0 0 0 2024-04-23T02:13:57.654 INFO:tasks.workunit.client.0.smithi071.stdout:7/269: rename d3/d17/d41/f27 to d3/d17/d41/d34/d59/d5f/f60 0 2024-04-23T02:13:57.658 INFO:tasks.workunit.client.1.smithi192.stdout:2/379: truncate f4 1314095 0 2024-04-23T02:13:57.658 INFO:tasks.workunit.client.1.smithi192.stdout:3/363: dread d1/d3/d17/d1f/d33/d24/d34/f54 [0,4194304] 0 2024-04-23T02:13:57.658 INFO:tasks.workunit.client.0.smithi071.stdout:9/332: read d1/f2b [3340419,41837] 0 2024-04-23T02:13:57.658 INFO:tasks.workunit.client.0.smithi071.stdout:0/284: dwrite d0/d3/de/d10/d14/f4d [0,4194304] 0 2024-04-23T02:13:57.666 INFO:tasks.workunit.client.1.smithi192.stdout:0/503: dwrite d6/df/f50 [0,4194304] 0 2024-04-23T02:13:57.666 INFO:tasks.workunit.client.1.smithi192.stdout:7/419: sync 2024-04-23T02:13:57.667 INFO:tasks.workunit.client.0.smithi071.stdout:5/420: dwrite d3/d34/f6a [0,4194304] 0 2024-04-23T02:13:57.667 INFO:tasks.workunit.client.1.smithi192.stdout:5/452: sync 2024-04-23T02:13:57.671 INFO:tasks.workunit.client.0.smithi071.stdout:0/285: truncate d0/d3/de/ff 4381269 0 2024-04-23T02:13:57.671 INFO:tasks.workunit.client.0.smithi071.stdout:5/421: chown d3/df/d10/f48 240035557 1 2024-04-23T02:13:57.671 INFO:tasks.workunit.client.1.smithi192.stdout:5/453: read - d2/db/d16/d1b/d3d/d40/d49/f83 zero size 2024-04-23T02:13:57.676 INFO:tasks.workunit.client.0.smithi071.stdout:6/293: dwrite d6/d22/f28 [0,4194304] 0 2024-04-23T02:13:57.678 INFO:tasks.workunit.client.0.smithi071.stdout:7/270: dwrite d3/d17/d1c/f3f [0,4194304] 0 2024-04-23T02:13:57.678 INFO:tasks.workunit.client.0.smithi071.stdout:0/286: chown d0/d3/de/l25 709 1 2024-04-23T02:13:57.680 INFO:tasks.workunit.client.0.smithi071.stdout:5/422: creat d3/d8f/fa2 x:0 0 0 2024-04-23T02:13:57.683 INFO:tasks.workunit.client.0.smithi071.stdout:4/362: dread d2/d2a/d2d/d39/d42/d49/d67/f1a [0,4194304] 0 2024-04-23T02:13:57.684 INFO:tasks.workunit.client.0.smithi071.stdout:7/271: chown d3/d17/d1c/c28 17 1 2024-04-23T02:13:57.687 INFO:tasks.workunit.client.0.smithi071.stdout:0/287: mkdir d0/d3/de/d10/d14/d32/d44/d59 0 2024-04-23T02:13:57.687 INFO:tasks.workunit.client.1.smithi192.stdout:8/546: mknod d1/d12/d23/d4b/d73/d4c/cc4 0 2024-04-23T02:13:57.687 INFO:tasks.workunit.client.0.smithi071.stdout:8/415: dread dc/d15/f23 [0,4194304] 0 2024-04-23T02:13:57.690 INFO:tasks.workunit.client.0.smithi071.stdout:0/288: fsync d0/f18 0 2024-04-23T02:13:57.694 INFO:tasks.workunit.client.0.smithi071.stdout:0/289: dread - d0/d3/de/d10/d14/d32/d37/d26/f2e zero size 2024-04-23T02:13:57.694 INFO:tasks.workunit.client.0.smithi071.stdout:6/294: dwrite d6/d1f/d46/f5a [0,4194304] 0 2024-04-23T02:13:57.695 INFO:tasks.workunit.client.0.smithi071.stdout:8/416: getdents dc/d3a 0 2024-04-23T02:13:57.697 INFO:tasks.workunit.client.0.smithi071.stdout:6/295: write d6/d22/f53 [1015595,107058] 0 2024-04-23T02:13:57.697 INFO:tasks.workunit.client.0.smithi071.stdout:4/363: link d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f5d d2/d2a/d2d/d39/d42/d49/f77 0 2024-04-23T02:13:57.698 INFO:tasks.workunit.client.0.smithi071.stdout:8/417: stat dc/d10/l42 0 2024-04-23T02:13:57.698 INFO:tasks.workunit.client.0.smithi071.stdout:6/296: stat d6/d22/f45 0 2024-04-23T02:13:57.700 INFO:tasks.workunit.client.1.smithi192.stdout:8/547: write d1/d9/d3e/d59/f75 [200748,37885] 0 2024-04-23T02:13:57.701 INFO:tasks.workunit.client.0.smithi071.stdout:8/418: fdatasync dc/d15/f23 0 2024-04-23T02:13:57.702 INFO:tasks.workunit.client.0.smithi071.stdout:6/297: write d6/d22/f45 [4947194,50938] 0 2024-04-23T02:13:57.705 INFO:tasks.workunit.client.0.smithi071.stdout:5/423: read d3/da/d1a/d1c/f62 [725120,44690] 0 2024-04-23T02:13:57.705 INFO:tasks.workunit.client.1.smithi192.stdout:9/382: unlink d0/d9/d44/d51/d5b/d61/l63 0 2024-04-23T02:13:57.706 INFO:tasks.workunit.client.0.smithi071.stdout:4/364: getdents d2/d53 0 2024-04-23T02:13:57.709 INFO:tasks.workunit.client.0.smithi071.stdout:4/365: write d2/d2a/f36 [2395882,75484] 0 2024-04-23T02:13:57.711 INFO:tasks.workunit.client.1.smithi192.stdout:8/548: fdatasync d1/d12/d23/d2c/d8c/d95/fb5 0 2024-04-23T02:13:57.711 INFO:tasks.workunit.client.1.smithi192.stdout:2/380: creat d8/d25/f79 x:0 0 0 2024-04-23T02:13:57.711 INFO:tasks.workunit.client.0.smithi071.stdout:8/419: rmdir dc/d10/d32/d7f 0 2024-04-23T02:13:57.712 INFO:tasks.workunit.client.1.smithi192.stdout:8/549: write d1/f15 [2763823,19893] 0 2024-04-23T02:13:57.712 INFO:tasks.workunit.client.0.smithi071.stdout:0/290: link d0/d3/de/l25 d0/d3/de/d10/d14/d32/d44/l5a 0 2024-04-23T02:13:57.712 INFO:tasks.workunit.client.0.smithi071.stdout:6/298: dwrite d6/f8 [0,4194304] 0 2024-04-23T02:13:57.714 INFO:tasks.workunit.client.1.smithi192.stdout:6/504: link d3/d18/f6f d3/d28/d79/fa4 0 2024-04-23T02:13:57.718 INFO:tasks.workunit.client.0.smithi071.stdout:1/300: truncate db/df/d13/f1b 3159071 0 2024-04-23T02:13:57.718 INFO:tasks.workunit.client.0.smithi071.stdout:4/366: write d2/d2a/d2d/d39/d42/d49/f77 [1595151,30] 0 2024-04-23T02:13:57.721 INFO:tasks.workunit.client.0.smithi071.stdout:9/333: truncate d1/d4e/f4f 1570773 0 2024-04-23T02:13:57.722 INFO:tasks.workunit.client.1.smithi192.stdout:2/381: rmdir d8/d26/d29/d43/d5d 39 2024-04-23T02:13:57.722 INFO:tasks.workunit.client.1.smithi192.stdout:0/504: mkdir d6/df/d56/d85/da4 0 2024-04-23T02:13:57.723 INFO:tasks.workunit.client.0.smithi071.stdout:3/277: truncate d7/d8/f1b 3564844 0 2024-04-23T02:13:57.725 INFO:tasks.workunit.client.0.smithi071.stdout:8/420: getdents dc/d3a 0 2024-04-23T02:13:57.726 INFO:tasks.workunit.client.0.smithi071.stdout:0/291: chown d0/d3/de/f28 4 1 2024-04-23T02:13:57.727 INFO:tasks.workunit.client.0.smithi071.stdout:1/301: creat db/df/d13/d20/f5c x:0 0 0 2024-04-23T02:13:57.728 INFO:tasks.workunit.client.1.smithi192.stdout:2/382: write d8/d25/f79 [987220,90726] 0 2024-04-23T02:13:57.729 INFO:tasks.workunit.client.0.smithi071.stdout:9/334: symlink d1/d1a/d39/l6e 0 2024-04-23T02:13:57.729 INFO:tasks.workunit.client.0.smithi071.stdout:6/299: dwrite d6/f8 [0,4194304] 0 2024-04-23T02:13:57.729 INFO:tasks.workunit.client.0.smithi071.stdout:7/272: dread d3/d17/d41/f33 [0,4194304] 0 2024-04-23T02:13:57.730 INFO:tasks.workunit.client.1.smithi192.stdout:6/505: write d3/d15/d1b/d66/d80/d8e/f8c [84512,42047] 0 2024-04-23T02:13:57.731 INFO:tasks.workunit.client.1.smithi192.stdout:7/420: rmdir d5/d43/d4c/d63 0 2024-04-23T02:13:57.731 INFO:tasks.workunit.client.0.smithi071.stdout:4/367: write d2/d2a/d2d/f66 [907816,66561] 0 2024-04-23T02:13:57.732 INFO:tasks.workunit.client.0.smithi071.stdout:0/292: read d0/d3/de/d10/d14/f51 [71259,125301] 0 2024-04-23T02:13:57.733 INFO:tasks.workunit.client.1.smithi192.stdout:1/244: getdents dc/d14/d15 0 2024-04-23T02:13:57.733 INFO:tasks.workunit.client.1.smithi192.stdout:6/506: dread - d3/d15/d5b/f69 zero size 2024-04-23T02:13:57.735 INFO:tasks.workunit.client.0.smithi071.stdout:9/335: fdatasync d1/d32/f4a 0 2024-04-23T02:13:57.738 INFO:tasks.workunit.client.0.smithi071.stdout:0/293: chown d0/d3/de/d10/d14/d32/c42 163667 1 2024-04-23T02:13:57.739 INFO:tasks.workunit.client.0.smithi071.stdout:1/302: dread - db/df/f19 zero size 2024-04-23T02:13:57.739 INFO:tasks.workunit.client.0.smithi071.stdout:2/300: dread d0/d2/d26/d14/f23 [0,4194304] 0 2024-04-23T02:13:57.740 INFO:tasks.workunit.client.1.smithi192.stdout:7/421: truncate d5/d17/d1c/f2d 4516812 0 2024-04-23T02:13:57.741 INFO:tasks.workunit.client.0.smithi071.stdout:6/300: getdents d6/d1f 0 2024-04-23T02:13:57.741 INFO:tasks.workunit.client.0.smithi071.stdout:5/424: dread d3/f22 [0,4194304] 0 2024-04-23T02:13:57.742 INFO:tasks.workunit.client.1.smithi192.stdout:7/422: read d5/d17/d1c/d26/f28 [2184332,74512] 0 2024-04-23T02:13:57.742 INFO:tasks.workunit.client.0.smithi071.stdout:0/294: rmdir d0/d3 39 2024-04-23T02:13:57.744 INFO:tasks.workunit.client.1.smithi192.stdout:1/245: mkdir dc/d16/d19/d30/d32/d44 0 2024-04-23T02:13:57.744 INFO:tasks.workunit.client.0.smithi071.stdout:2/301: dread - d0/d2/d26/df/f3b zero size 2024-04-23T02:13:57.746 INFO:tasks.workunit.client.0.smithi071.stdout:2/302: chown d0 50 1 2024-04-23T02:13:57.747 INFO:tasks.workunit.client.0.smithi071.stdout:5/425: write d3/da/f4d [471422,95510] 0 2024-04-23T02:13:57.747 INFO:tasks.workunit.client.0.smithi071.stdout:1/303: mkdir db/d5d 0 2024-04-23T02:13:57.748 INFO:tasks.workunit.client.0.smithi071.stdout:3/278: dwrite d7/fd [4194304,4194304] 0 2024-04-23T02:13:57.748 INFO:tasks.workunit.client.0.smithi071.stdout:6/301: rename d6/d17/f33 to d6/d3e/f5b 0 2024-04-23T02:13:57.748 INFO:tasks.workunit.client.0.smithi071.stdout:2/303: stat d0 0 2024-04-23T02:13:57.749 INFO:tasks.workunit.client.0.smithi071.stdout:0/295: chown d0/d3/d1c/f22 9 1 2024-04-23T02:13:57.749 INFO:tasks.workunit.client.1.smithi192.stdout:6/507: dwrite d3/d15/d5b/f69 [0,4194304] 0 2024-04-23T02:13:57.751 INFO:tasks.workunit.client.0.smithi071.stdout:1/304: write fa [4784344,4969] 0 2024-04-23T02:13:57.752 INFO:tasks.workunit.client.0.smithi071.stdout:7/273: dread d3/d17/d1c/d31/f48 [0,4194304] 0 2024-04-23T02:13:57.753 INFO:tasks.workunit.client.1.smithi192.stdout:1/246: symlink dc/d14/d23/l45 0 2024-04-23T02:13:57.755 INFO:tasks.workunit.client.1.smithi192.stdout:9/383: link d0/d9/d44/d74/d3c/d5f/l6b d0/d9/d44/d4f/l8b 0 2024-04-23T02:13:57.755 INFO:tasks.workunit.client.1.smithi192.stdout:0/505: dread d6/df/d13/d25/d26/f90 [0,4194304] 0 2024-04-23T02:13:57.755 INFO:tasks.workunit.client.0.smithi071.stdout:6/302: rename d6/ff to d6/f5c 0 2024-04-23T02:13:57.756 INFO:tasks.workunit.client.0.smithi071.stdout:1/305: fdatasync db/df/d13/d2b/f35 0 2024-04-23T02:13:57.758 INFO:tasks.workunit.client.0.smithi071.stdout:5/426: rename d3/da/d1a/f4c to d3/d34/d6b/fa3 0 2024-04-23T02:13:57.760 INFO:tasks.workunit.client.1.smithi192.stdout:7/423: mknod d5/d17/d1c/d59/d5a/d41/c7a 0 2024-04-23T02:13:57.761 INFO:tasks.workunit.client.0.smithi071.stdout:2/304: creat d0/d2/d51/d56/f61 x:0 0 0 2024-04-23T02:13:57.762 INFO:tasks.workunit.client.0.smithi071.stdout:6/303: mkdir d6/d1f/d5d 0 2024-04-23T02:13:57.765 INFO:tasks.workunit.client.0.smithi071.stdout:1/306: creat db/d1a/d57/f5e x:0 0 0 2024-04-23T02:13:57.771 INFO:tasks.workunit.client.1.smithi192.stdout:1/247: dread dc/d16/d19/d30/d32/d34/f3e [0,4194304] 0 2024-04-23T02:13:57.771 INFO:tasks.workunit.client.1.smithi192.stdout:2/383: getdents d8/d26/d27/d53 0 2024-04-23T02:13:57.772 INFO:tasks.workunit.client.1.smithi192.stdout:0/506: mkdir d6/df/d13/d25/d86/da5 0 2024-04-23T02:13:57.773 INFO:tasks.workunit.client.0.smithi071.stdout:1/307: stat db/d11/c16 0 2024-04-23T02:13:57.773 INFO:tasks.workunit.client.0.smithi071.stdout:0/296: dwrite d0/d3/de/d10/f1f [0,4194304] 0 2024-04-23T02:13:57.775 INFO:tasks.workunit.client.1.smithi192.stdout:9/384: mknod d0/d9/d44/d58/d89/c8c 0 2024-04-23T02:13:57.777 INFO:tasks.workunit.client.1.smithi192.stdout:1/248: symlink dc/d14/d23/l46 0 2024-04-23T02:13:57.780 INFO:tasks.workunit.client.0.smithi071.stdout:1/308: write db/df/f1c [3864847,2323] 0 2024-04-23T02:13:57.781 INFO:tasks.workunit.client.0.smithi071.stdout:7/274: dwrite d3/f3d [0,4194304] 0 2024-04-23T02:13:57.782 INFO:tasks.workunit.client.1.smithi192.stdout:8/550: symlink d1/lc5 0 2024-04-23T02:13:57.785 INFO:tasks.workunit.client.0.smithi071.stdout:3/279: dwrite d7/d10/f15 [0,4194304] 0 2024-04-23T02:13:57.789 INFO:tasks.workunit.client.1.smithi192.stdout:1/249: truncate dc/d14/d15/d35/f40 2666270 0 2024-04-23T02:13:57.791 INFO:tasks.workunit.client.1.smithi192.stdout:0/507: dwrite d6/df/d35/f74 [0,4194304] 0 2024-04-23T02:13:57.795 INFO:tasks.workunit.client.0.smithi071.stdout:2/305: dwrite d0/d2/d26/d14/d1f/f25 [0,4194304] 0 2024-04-23T02:13:57.799 INFO:tasks.workunit.client.0.smithi071.stdout:9/336: dwrite d1/d1a/d5a/f6b [0,4194304] 0 2024-04-23T02:13:57.801 INFO:tasks.workunit.client.1.smithi192.stdout:3/364: truncate d1/d3/f49 2443212 0 2024-04-23T02:13:57.801 INFO:tasks.workunit.client.1.smithi192.stdout:1/250: rmdir dc/d14 39 2024-04-23T02:13:57.802 INFO:tasks.workunit.client.0.smithi071.stdout:5/427: write d3/d34/d6b/d81/f58 [105954,91711] 0 2024-04-23T02:13:57.802 INFO:tasks.workunit.client.0.smithi071.stdout:7/275: symlink d3/d17/l61 0 2024-04-23T02:13:57.803 INFO:tasks.workunit.client.0.smithi071.stdout:1/309: dread db/df/f39 [0,4194304] 0 2024-04-23T02:13:57.804 INFO:tasks.workunit.client.1.smithi192.stdout:0/508: dread d6/f1b [0,4194304] 0 2024-04-23T02:13:57.805 INFO:tasks.workunit.client.1.smithi192.stdout:0/509: fdatasync d6/df/d47/f4c 0 2024-04-23T02:13:57.807 INFO:tasks.workunit.client.0.smithi071.stdout:2/306: write d0/d2/d26/d14/f23 [3559130,48460] 0 2024-04-23T02:13:57.807 INFO:tasks.workunit.client.0.smithi071.stdout:9/337: read d1/f28 [214372,62421] 0 2024-04-23T02:13:57.808 INFO:tasks.workunit.client.1.smithi192.stdout:0/510: truncate f5 140819 0 2024-04-23T02:13:57.810 INFO:tasks.workunit.client.0.smithi071.stdout:4/368: dwrite d2/d2a/d2d/d39/d42/d49/d67/d1d/f35 [0,4194304] 0 2024-04-23T02:13:57.810 INFO:tasks.workunit.client.0.smithi071.stdout:5/428: mknod d3/da/d1a/d6d/ca4 0 2024-04-23T02:13:57.812 INFO:tasks.workunit.client.1.smithi192.stdout:0/511: getdents d6 0 2024-04-23T02:13:57.812 INFO:tasks.workunit.client.1.smithi192.stdout:6/508: write d3/d11/f1d [190912,72822] 0 2024-04-23T02:13:57.814 INFO:tasks.workunit.client.0.smithi071.stdout:3/280: dwrite d7/d10/d38/f43 [0,4194304] 0 2024-04-23T02:13:57.815 INFO:tasks.workunit.client.0.smithi071.stdout:7/276: write d3/d17/d41/d37/f50 [795091,109215] 0 2024-04-23T02:13:57.815 INFO:tasks.workunit.client.1.smithi192.stdout:0/512: mkdir d6/df/d13/d25/d3a/da6 0 2024-04-23T02:13:57.816 INFO:tasks.workunit.client.0.smithi071.stdout:5/429: fsync d3/da/d1a/f1b 0 2024-04-23T02:13:57.817 INFO:tasks.workunit.client.0.smithi071.stdout:2/307: dread - d0/d2/d45/f5a zero size 2024-04-23T02:13:57.817 INFO:tasks.workunit.client.1.smithi192.stdout:6/509: rename d3/l37 to d3/la5 0 2024-04-23T02:13:57.818 INFO:tasks.workunit.client.1.smithi192.stdout:6/510: chown d3/d15/d5b 70 1 2024-04-23T02:13:57.818 INFO:tasks.workunit.client.0.smithi071.stdout:5/430: chown d3/d1d/d2c 6877405 1 2024-04-23T02:13:57.819 INFO:tasks.workunit.client.1.smithi192.stdout:6/511: creat d3/d15/d1b/d21/d3a/fa6 x:0 0 0 2024-04-23T02:13:57.820 INFO:tasks.workunit.client.1.smithi192.stdout:0/513: link d6/df/l21 d6/df/d13/d25/d86/da5/la7 0 2024-04-23T02:13:57.820 INFO:tasks.workunit.client.0.smithi071.stdout:9/338: mknod d1/d32/d3a/d5f/c6f 0 2024-04-23T02:13:57.821 INFO:tasks.workunit.client.1.smithi192.stdout:0/514: truncate d6/df/f4a 510380 0 2024-04-23T02:13:57.821 INFO:tasks.workunit.client.1.smithi192.stdout:6/512: unlink d3/d11/l14 0 2024-04-23T02:13:57.821 INFO:tasks.workunit.client.0.smithi071.stdout:3/281: stat d7/d8/d4f/l4b 0 2024-04-23T02:13:57.822 INFO:tasks.workunit.client.1.smithi192.stdout:6/513: readlink d3/d18/l1a 0 2024-04-23T02:13:57.822 INFO:tasks.workunit.client.0.smithi071.stdout:4/369: dread - d2/d2a/d2d/d39/d42/d49/d67/d3e/f76 zero size 2024-04-23T02:13:57.822 INFO:tasks.workunit.client.0.smithi071.stdout:7/277: fdatasync d3/d17/d1c/f29 0 2024-04-23T02:13:57.823 INFO:tasks.workunit.client.0.smithi071.stdout:6/304: dwrite d6/d1f/f21 [0,4194304] 0 2024-04-23T02:13:57.823 INFO:tasks.workunit.client.0.smithi071.stdout:5/431: readlink d3/da/d1a/l3f 0 2024-04-23T02:13:57.824 INFO:tasks.workunit.client.0.smithi071.stdout:6/305: stat d6/d17/c1b 0 2024-04-23T02:13:57.825 INFO:tasks.workunit.client.0.smithi071.stdout:2/308: mknod d0/d2/d45/c62 0 2024-04-23T02:13:57.827 INFO:tasks.workunit.client.0.smithi071.stdout:9/339: creat d1/d1a/f70 x:0 0 0 2024-04-23T02:13:57.828 INFO:tasks.workunit.client.0.smithi071.stdout:9/340: fsync d1/fa 0 2024-04-23T02:13:57.829 INFO:tasks.workunit.client.0.smithi071.stdout:3/282: chown d7/d10/d38/f41 25032 1 2024-04-23T02:13:57.829 INFO:tasks.workunit.client.0.smithi071.stdout:2/309: write d0/d2/d3c/f4e [120632,98537] 0 2024-04-23T02:13:57.833 INFO:tasks.workunit.client.0.smithi071.stdout:7/278: mknod d3/d17/d41/d3a/c62 0 2024-04-23T02:13:57.833 INFO:tasks.workunit.client.0.smithi071.stdout:6/306: creat d6/d3e/f5e x:0 0 0 2024-04-23T02:13:57.833 INFO:tasks.workunit.client.0.smithi071.stdout:4/370: unlink d2/d2a/d2d/d39/d42/d49/d67/d1d/l21 0 2024-04-23T02:13:57.834 INFO:tasks.workunit.client.0.smithi071.stdout:9/341: rmdir d1 39 2024-04-23T02:13:57.835 INFO:tasks.workunit.client.1.smithi192.stdout:1/251: read fa [57226,53054] 0 2024-04-23T02:13:57.835 INFO:tasks.workunit.client.0.smithi071.stdout:5/432: rename d3/d1d/d43/d4b/d54 to d3/df/d10/da5 0 2024-04-23T02:13:57.836 INFO:tasks.workunit.client.0.smithi071.stdout:1/310: getdents db/df/d13/d2f/d48 0 2024-04-23T02:13:57.837 INFO:tasks.workunit.client.0.smithi071.stdout:2/310: dread d0/d2/d26/d14/f23 [0,4194304] 0 2024-04-23T02:13:57.838 INFO:tasks.workunit.client.0.smithi071.stdout:4/371: unlink d2/c3f 0 2024-04-23T02:13:57.840 INFO:tasks.workunit.client.0.smithi071.stdout:5/433: mknod d3/da/ca6 0 2024-04-23T02:13:57.842 INFO:tasks.workunit.client.0.smithi071.stdout:5/434: chown d3/da/d1a/c23 1910145 1 2024-04-23T02:13:57.842 INFO:tasks.workunit.client.0.smithi071.stdout:9/342: mkdir d1/d1a/d71 0 2024-04-23T02:13:57.843 INFO:tasks.workunit.client.1.smithi192.stdout:4/604: rename de/d15/d2a/d74/d92/da0/dca/d49/f28 to de/d15/d2a/fcb 0 2024-04-23T02:13:57.846 INFO:tasks.workunit.client.0.smithi071.stdout:6/307: creat d6/d1f/d5d/f5f x:0 0 0 2024-04-23T02:13:57.846 INFO:tasks.workunit.client.0.smithi071.stdout:4/372: symlink d2/d2a/l78 0 2024-04-23T02:13:57.849 INFO:tasks.workunit.client.1.smithi192.stdout:3/365: rename d1/d3/d17/f2d to d1/d3/f7a 0 2024-04-23T02:13:57.849 INFO:tasks.workunit.client.0.smithi071.stdout:2/311: rename d0/d2/d26/d14/d1f/d36/c59 to d0/c63 0 2024-04-23T02:13:57.850 INFO:tasks.workunit.client.0.smithi071.stdout:7/279: link d3/d17/d41/l47 d3/d17/d41/l63 0 2024-04-23T02:13:57.852 INFO:tasks.workunit.client.1.smithi192.stdout:4/605: link de/c7a de/d15/d53/ccc 0 2024-04-23T02:13:57.852 INFO:tasks.workunit.client.0.smithi071.stdout:9/343: symlink d1/d32/d41/l72 0 2024-04-23T02:13:57.852 INFO:tasks.workunit.client.0.smithi071.stdout:7/280: readlink d3/d17/l21 0 2024-04-23T02:13:57.852 INFO:tasks.workunit.client.0.smithi071.stdout:1/311: rmdir db/d5d 0 2024-04-23T02:13:57.853 INFO:tasks.workunit.client.1.smithi192.stdout:3/366: mkdir d1/d3/d17/d1f/d33/d24/d34/d38/d7b 0 2024-04-23T02:13:57.853 INFO:tasks.workunit.client.0.smithi071.stdout:9/344: fdatasync d1/fa 0 2024-04-23T02:13:57.854 INFO:tasks.workunit.client.1.smithi192.stdout:3/367: dread - d1/d3/d17/f3d zero size 2024-04-23T02:13:57.856 INFO:tasks.workunit.client.0.smithi071.stdout:2/312: creat d0/d2/d34/d55/f64 x:0 0 0 2024-04-23T02:13:57.856 INFO:tasks.workunit.client.0.smithi071.stdout:7/281: unlink d3/d17/c2d 0 2024-04-23T02:13:57.856 INFO:tasks.workunit.client.1.smithi192.stdout:4/606: getdents de/d15/d19/d1a 0 2024-04-23T02:13:57.857 INFO:tasks.workunit.client.0.smithi071.stdout:5/435: dwrite d3/da/d1a/d1c/d46/d4e/f84 [0,4194304] 0 2024-04-23T02:13:57.862 INFO:tasks.workunit.client.1.smithi192.stdout:4/607: dread de/d15/d53/d81/fa6 [0,4194304] 0 2024-04-23T02:13:57.863 INFO:tasks.workunit.client.0.smithi071.stdout:7/282: stat d3/d17/d41/d34 0 2024-04-23T02:13:57.868 INFO:tasks.workunit.client.0.smithi071.stdout:2/313: read - d0/d2/d26/d3a/f44 zero size 2024-04-23T02:13:57.869 INFO:tasks.workunit.client.0.smithi071.stdout:5/436: mkdir d3/d34/d6b/d81/d5a/da7 0 2024-04-23T02:13:57.873 INFO:tasks.workunit.client.1.smithi192.stdout:9/385: write d0/d9/d44/d4f/f8 [539068,54420] 0 2024-04-23T02:13:57.873 INFO:tasks.workunit.client.1.smithi192.stdout:3/368: dwrite d1/d3/d6/f79 [0,4194304] 0 2024-04-23T02:13:57.874 INFO:tasks.workunit.client.1.smithi192.stdout:8/551: write d1/d12/d68/fad [733836,67676] 0 2024-04-23T02:13:57.876 INFO:tasks.workunit.client.1.smithi192.stdout:3/369: stat d1/d3/d6/f79 0 2024-04-23T02:13:57.877 INFO:tasks.workunit.client.0.smithi071.stdout:9/345: dwrite d1/f2b [4194304,4194304] 0 2024-04-23T02:13:57.878 INFO:tasks.workunit.client.0.smithi071.stdout:3/283: dread d7/d10/d38/f41 [0,4194304] 0 2024-04-23T02:13:57.879 INFO:tasks.workunit.client.1.smithi192.stdout:6/514: write d3/d28/f9a [1778089,74597] 0 2024-04-23T02:13:57.879 INFO:tasks.workunit.client.1.smithi192.stdout:2/384: truncate f4 30589 0 2024-04-23T02:13:57.884 INFO:tasks.workunit.client.0.smithi071.stdout:4/373: dread d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/ff [0,4194304] 0 2024-04-23T02:13:57.884 INFO:tasks.workunit.client.1.smithi192.stdout:1/252: truncate dc/ff 3507611 0 2024-04-23T02:13:57.886 INFO:tasks.workunit.client.0.smithi071.stdout:0/297: dread d0/d3/de/d10/d16/f3e [0,4194304] 0 2024-04-23T02:13:57.887 INFO:tasks.workunit.client.1.smithi192.stdout:9/386: dwrite d0/d9/d44/d51/f66 [0,4194304] 0 2024-04-23T02:13:57.888 INFO:tasks.workunit.client.0.smithi071.stdout:1/312: dwrite db/df/d13/d20/f32 [0,4194304] 0 2024-04-23T02:13:57.888 INFO:tasks.workunit.client.1.smithi192.stdout:3/370: getdents d1/d3/d17/d1a/d6d 0 2024-04-23T02:13:57.890 INFO:tasks.workunit.client.1.smithi192.stdout:9/387: read - d0/d9/d44/d74/d3c/d5f/f7e zero size 2024-04-23T02:13:57.890 INFO:tasks.workunit.client.1.smithi192.stdout:2/385: rmdir d8/d26/d29/d43/d5d 39 2024-04-23T02:13:57.891 INFO:tasks.workunit.client.1.smithi192.stdout:3/371: truncate d1/d3/f19 4812156 0 2024-04-23T02:13:57.892 INFO:tasks.workunit.client.0.smithi071.stdout:9/346: creat d1/d4e/d66/f73 x:0 0 0 2024-04-23T02:13:57.895 INFO:tasks.workunit.client.1.smithi192.stdout:1/253: chown dc/d14/f27 5643433 1 2024-04-23T02:13:57.895 INFO:tasks.workunit.client.1.smithi192.stdout:2/386: truncate d8/d26/f6b 941406 0 2024-04-23T02:13:57.896 INFO:tasks.workunit.client.1.smithi192.stdout:4/608: creat de/d15/d2a/d74/d92/da0/dca/d49/d26/d59/fcd x:0 0 0 2024-04-23T02:13:57.897 INFO:tasks.workunit.client.1.smithi192.stdout:2/387: fdatasync d8/d26/d29/d43/d50/d5e/f5f 0 2024-04-23T02:13:57.898 INFO:tasks.workunit.client.0.smithi071.stdout:7/283: truncate d3/d17/d1c/f2c 1453673 0 2024-04-23T02:13:57.900 INFO:tasks.workunit.client.1.smithi192.stdout:8/552: dread d1/d12/d23/d4b/d73/d45/d46/f47 [0,4194304] 0 2024-04-23T02:13:57.901 INFO:tasks.workunit.client.0.smithi071.stdout:5/437: mkdir d3/d34/d6b/d81/d5a/da7/da8 0 2024-04-23T02:13:57.901 INFO:tasks.workunit.client.0.smithi071.stdout:7/284: chown d3/d17/d1c/d2a/c40 9546 1 2024-04-23T02:13:57.903 INFO:tasks.workunit.client.0.smithi071.stdout:2/314: dread d0/d2/f1d [0,4194304] 0 2024-04-23T02:13:57.903 INFO:tasks.workunit.client.0.smithi071.stdout:1/313: dwrite fa [0,4194304] 0 2024-04-23T02:13:57.904 INFO:tasks.workunit.client.1.smithi192.stdout:3/372: mkdir d1/d3/d17/d1a/d7c 0 2024-04-23T02:13:57.904 INFO:tasks.workunit.client.1.smithi192.stdout:2/388: rename d8 to d8/d26/d29/d43/d7a 22 2024-04-23T02:13:57.904 INFO:tasks.workunit.client.1.smithi192.stdout:4/609: rmdir de/d15/d2a/d74/d92/da0/dca/d49 39 2024-04-23T02:13:57.905 INFO:tasks.workunit.client.0.smithi071.stdout:3/284: fdatasync d7/d8/f1b 0 2024-04-23T02:13:57.905 INFO:tasks.workunit.client.0.smithi071.stdout:5/438: fsync d3/da/d1a/d1c/f2e 0 2024-04-23T02:13:57.909 INFO:tasks.workunit.client.1.smithi192.stdout:9/388: creat d0/d9/d44/d74/d3c/d5f/d7d/f8d x:0 0 0 2024-04-23T02:13:57.918 INFO:tasks.workunit.client.1.smithi192.stdout:4/610: rmdir de/d15/d53/dc5 0 2024-04-23T02:13:57.918 INFO:tasks.workunit.client.1.smithi192.stdout:8/553: unlink d1/df/f1c 0 2024-04-23T02:13:57.920 INFO:tasks.workunit.client.1.smithi192.stdout:3/373: dread d1/f55 [0,4194304] 0 2024-04-23T02:13:57.920 INFO:tasks.workunit.client.0.smithi071.stdout:0/298: creat d0/d3/de/d10/d14/f5b x:0 0 0 2024-04-23T02:13:57.922 INFO:tasks.workunit.client.1.smithi192.stdout:0/515: dwrite d6/df/d56/d85/f46 [0,4194304] 0 2024-04-23T02:13:57.923 INFO:tasks.workunit.client.0.smithi071.stdout:7/285: creat d3/d17/d1c/f64 x:0 0 0 2024-04-23T02:13:57.925 INFO:tasks.workunit.client.0.smithi071.stdout:3/285: stat d7/d10/c30 0 2024-04-23T02:13:57.926 INFO:tasks.workunit.client.0.smithi071.stdout:4/374: getdents d2/d2a/d2d/d39/d42/d49/d67 0 2024-04-23T02:13:57.927 INFO:tasks.workunit.client.1.smithi192.stdout:2/389: read d8/d14/f15 [3748001,22564] 0 2024-04-23T02:13:57.927 INFO:tasks.workunit.client.1.smithi192.stdout:8/554: truncate d1/d12/d23/d2c/d4d/f52 885380 0 2024-04-23T02:13:57.928 INFO:tasks.workunit.client.0.smithi071.stdout:1/314: truncate db/df/d13/d2b/f35 1508405 0 2024-04-23T02:13:57.929 INFO:tasks.workunit.client.1.smithi192.stdout:9/389: creat d0/d9/d44/d4f/f8e x:0 0 0 2024-04-23T02:13:57.929 INFO:tasks.workunit.client.1.smithi192.stdout:0/516: mknod d6/df/d56/ca8 0 2024-04-23T02:13:57.932 INFO:tasks.workunit.client.1.smithi192.stdout:3/374: creat d1/d3/d6/f7d x:0 0 0 2024-04-23T02:13:57.934 INFO:tasks.workunit.client.0.smithi071.stdout:7/286: symlink d3/d17/d1c/d31/l65 0 2024-04-23T02:13:57.934 INFO:tasks.workunit.client.1.smithi192.stdout:3/375: read d1/d3/d17/d2b/f32 [142533,16284] 0 2024-04-23T02:13:57.935 INFO:tasks.workunit.client.0.smithi071.stdout:5/439: truncate d3/da/d1a/d1c/f2e 409891 0 2024-04-23T02:13:57.935 INFO:tasks.workunit.client.0.smithi071.stdout:3/286: dread d7/d22/d23/d26/f2b [0,4194304] 0 2024-04-23T02:13:57.936 INFO:tasks.workunit.client.0.smithi071.stdout:7/287: stat d3/d17/d41/d34/l4d 0 2024-04-23T02:13:57.936 INFO:tasks.workunit.client.1.smithi192.stdout:4/611: getdents de/d15/d2a/d97 0 2024-04-23T02:13:57.938 INFO:tasks.workunit.client.0.smithi071.stdout:3/287: write d7/d8/f1a [865411,112422] 0 2024-04-23T02:13:57.939 INFO:tasks.workunit.client.0.smithi071.stdout:5/440: chown d3/df/d10/da5/d5f/c8e 3480303 1 2024-04-23T02:13:57.939 INFO:tasks.workunit.client.1.smithi192.stdout:1/254: dread f3 [0,4194304] 0 2024-04-23T02:13:57.941 INFO:tasks.workunit.client.0.smithi071.stdout:7/288: creat d3/d17/d41/f66 x:0 0 0 2024-04-23T02:13:57.942 INFO:tasks.workunit.client.1.smithi192.stdout:3/376: symlink d1/d3/d17/d65/l7e 0 2024-04-23T02:13:57.943 INFO:tasks.workunit.client.0.smithi071.stdout:7/289: write d3/d17/d1c/f26 [4094426,19244] 0 2024-04-23T02:13:57.944 INFO:tasks.workunit.client.0.smithi071.stdout:1/315: write f3 [5395846,63280] 0 2024-04-23T02:13:57.944 INFO:tasks.workunit.client.1.smithi192.stdout:4/612: mknod de/d15/d2a/cce 0 2024-04-23T02:13:57.946 INFO:tasks.workunit.client.1.smithi192.stdout:9/390: dwrite d0/d9/d44/f7c [0,4194304] 0 2024-04-23T02:13:57.948 INFO:tasks.workunit.client.0.smithi071.stdout:3/288: creat d7/d27/d3e/f5b x:0 0 0 2024-04-23T02:13:57.950 INFO:tasks.workunit.client.1.smithi192.stdout:3/377: creat d1/d3/d17/d1f/d33/d24/d34/f7f x:0 0 0 2024-04-23T02:13:57.951 INFO:tasks.workunit.client.0.smithi071.stdout:5/441: rename d3/da/d1a/d1c/d46/d87 to d3/d34/d6b/d81/da9 0 2024-04-23T02:13:57.951 INFO:tasks.workunit.client.0.smithi071.stdout:3/289: stat d7/d8/d42/d47/c4d 0 2024-04-23T02:13:57.951 INFO:tasks.workunit.client.0.smithi071.stdout:4/375: getdents d2/d2a/d73/d5e 0 2024-04-23T02:13:57.952 INFO:tasks.workunit.client.1.smithi192.stdout:4/613: creat de/d15/d2a/d61/d89/fcf x:0 0 0 2024-04-23T02:13:57.953 INFO:tasks.workunit.client.1.smithi192.stdout:8/555: dread d1/d12/d23/f91 [0,4194304] 0 2024-04-23T02:13:57.955 INFO:tasks.workunit.client.1.smithi192.stdout:8/556: write d1/d12/d23/d4b/d73/d4c/f6e [3566590,30375] 0 2024-04-23T02:13:57.958 INFO:tasks.workunit.client.0.smithi071.stdout:0/299: truncate d0/d3/de/d10/d14/d32/d37/d26/f4b 1144118 0 2024-04-23T02:13:57.959 INFO:tasks.workunit.client.0.smithi071.stdout:1/316: rename db/d1a to db/df/d5f 0 2024-04-23T02:13:57.960 INFO:tasks.workunit.client.1.smithi192.stdout:3/378: getdents d1/d3 0 2024-04-23T02:13:57.961 INFO:tasks.workunit.client.0.smithi071.stdout:3/290: symlink d7/d10/d38/l5c 0 2024-04-23T02:13:57.961 INFO:tasks.workunit.client.1.smithi192.stdout:4/614: creat de/d15/d2a/d74/d92/da0/dca/d49/d26/fd0 x:0 0 0 2024-04-23T02:13:57.962 INFO:tasks.workunit.client.1.smithi192.stdout:1/255: write dc/d16/f31 [798692,66701] 0 2024-04-23T02:13:57.962 INFO:tasks.workunit.client.0.smithi071.stdout:2/315: dread d0/d2/fb [0,4194304] 0 2024-04-23T02:13:57.963 INFO:tasks.workunit.client.0.smithi071.stdout:7/290: dwrite d3/d17/d1c/f45 [4194304,4194304] 0 2024-04-23T02:13:57.967 INFO:tasks.workunit.client.1.smithi192.stdout:2/390: dread d8/d14/d51/f56 [0,4194304] 0 2024-04-23T02:13:57.967 INFO:tasks.workunit.client.1.smithi192.stdout:4/615: read - de/d15/dbb/fc8 zero size 2024-04-23T02:13:57.969 INFO:tasks.workunit.client.0.smithi071.stdout:1/317: unlink db/df/d13/d2b/f42 0 2024-04-23T02:13:57.969 INFO:tasks.workunit.client.0.smithi071.stdout:3/291: fsync d7/d22/d23/d26/f2b 0 2024-04-23T02:13:57.970 INFO:tasks.workunit.client.1.smithi192.stdout:3/379: write d1/d3/d6/d7/f72 [500286,66851] 0 2024-04-23T02:13:57.971 INFO:tasks.workunit.client.0.smithi071.stdout:2/316: creat d0/d2/d26/df/f65 x:0 0 0 2024-04-23T02:13:57.972 INFO:tasks.workunit.client.0.smithi071.stdout:4/376: mknod d2/d2a/d2d/d39/d42/d49/d67/d3e/c79 0 2024-04-23T02:13:57.973 INFO:tasks.workunit.client.1.smithi192.stdout:8/557: dwrite d1/d12/fa5 [0,4194304] 0 2024-04-23T02:13:57.974 INFO:tasks.workunit.client.1.smithi192.stdout:2/391: unlink c1 0 2024-04-23T02:13:57.974 INFO:tasks.workunit.client.0.smithi071.stdout:5/442: dwrite d3/d34/d6b/d81/d5a/f82 [0,4194304] 0 2024-04-23T02:13:57.977 INFO:tasks.workunit.client.1.smithi192.stdout:1/256: creat dc/d16/d19/d30/d32/f47 x:0 0 0 2024-04-23T02:13:57.977 INFO:tasks.workunit.client.1.smithi192.stdout:2/392: rmdir d8/d26/d27/d31 39 2024-04-23T02:13:57.979 INFO:tasks.workunit.client.1.smithi192.stdout:8/558: creat d1/d12/d23/d4b/d73/d7e/d88/fc6 x:0 0 0 2024-04-23T02:13:57.981 INFO:tasks.workunit.client.1.smithi192.stdout:8/559: read - d1/d12/d23/d4b/d73/f43 zero size 2024-04-23T02:13:57.981 INFO:tasks.workunit.client.1.smithi192.stdout:3/380: rename d1/d3/d6/d7/f31 to d1/d3/d17/d1f/d33/d24/d34/d38/f80 0 2024-04-23T02:13:57.984 INFO:tasks.workunit.client.1.smithi192.stdout:8/560: write d1/d12/d68/fad [796130,84786] 0 2024-04-23T02:13:57.988 INFO:tasks.workunit.client.1.smithi192.stdout:1/257: dwrite dc/d16/d19/d30/d32/f33 [0,4194304] 0 2024-04-23T02:13:57.991 INFO:tasks.workunit.client.1.smithi192.stdout:8/561: creat d1/d12/d23/d4b/fc7 x:0 0 0 2024-04-23T02:13:57.992 INFO:tasks.workunit.client.0.smithi071.stdout:2/317: creat d0/d2/d26/d14/d1f/d4f/f66 x:0 0 0 2024-04-23T02:13:57.993 INFO:tasks.workunit.client.1.smithi192.stdout:2/393: dwrite d8/d14/d51/f56 [0,4194304] 0 2024-04-23T02:13:57.994 INFO:tasks.workunit.client.1.smithi192.stdout:8/562: truncate d1/d12/fc1 162471 0 2024-04-23T02:13:57.996 INFO:tasks.workunit.client.1.smithi192.stdout:2/394: symlink d8/d26/l7b 0 2024-04-23T02:13:57.997 INFO:tasks.workunit.client.0.smithi071.stdout:3/292: creat d7/d8/d42/f5d x:0 0 0 2024-04-23T02:13:57.997 INFO:tasks.workunit.client.0.smithi071.stdout:0/300: link d0/d3/f3d d0/d3/de/d10/d14/d32/d44/d31/d52/f5c 0 2024-04-23T02:13:57.997 INFO:tasks.workunit.client.1.smithi192.stdout:2/395: read - d8/d26/d29/f41 zero size 2024-04-23T02:13:57.998 INFO:tasks.workunit.client.1.smithi192.stdout:1/258: link dc/d16/d19/l2f dc/d16/d19/d30/d32/l48 0 2024-04-23T02:13:58.001 INFO:tasks.workunit.client.0.smithi071.stdout:7/291: link d3/d17/f36 d3/d17/d41/d34/d59/f67 0 2024-04-23T02:13:58.002 INFO:tasks.workunit.client.0.smithi071.stdout:1/318: dwrite f5 [4194304,4194304] 0 2024-04-23T02:13:58.003 INFO:tasks.workunit.client.0.smithi071.stdout:4/377: symlink d2/d2a/d2d/d39/d42/d49/d67/d1d/l7a 0 2024-04-23T02:13:58.010 INFO:tasks.workunit.client.0.smithi071.stdout:5/443: dwrite d3/da/d1a/d6d/f9a [0,4194304] 0 2024-04-23T02:13:58.011 INFO:tasks.workunit.client.0.smithi071.stdout:3/293: creat d7/d27/d3e/f5e x:0 0 0 2024-04-23T02:13:58.011 INFO:tasks.workunit.client.0.smithi071.stdout:4/378: readlink d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/ld 0 2024-04-23T02:13:58.011 INFO:tasks.workunit.client.0.smithi071.stdout:7/292: write d3/d17/d1c/f29 [1782270,55950] 0 2024-04-23T02:13:58.013 INFO:tasks.workunit.client.0.smithi071.stdout:1/319: rmdir db/df/d3a 39 2024-04-23T02:13:58.014 INFO:tasks.workunit.client.0.smithi071.stdout:2/318: creat d0/d2/f67 x:0 0 0 2024-04-23T02:13:58.015 INFO:tasks.workunit.client.0.smithi071.stdout:7/293: read d3/d17/d1c/f26 [2285823,89102] 0 2024-04-23T02:13:58.020 INFO:tasks.workunit.client.0.smithi071.stdout:5/444: creat d3/d34/d74/faa x:0 0 0 2024-04-23T02:13:58.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.017+0000 7f12d8f8f700 1 -- 172.21.15.71:0/3171010709 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12c40053e0 con 0x7f12d40ffc10 2024-04-23T02:13:58.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.018+0000 7f12dc1f5700 1 -- 172.21.15.71:0/3171010709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d40ffc10 msgr2=0x7f12d4100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:58.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.018+0000 7f12dc1f5700 1 --2- 172.21.15.71:0/3171010709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d40ffc10 0x7f12d4100030 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009910 tx=0x7f12c4009c20 comp rx=0 tx=0).stop 2024-04-23T02:13:58.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.019+0000 7f12dc1f5700 1 -- 172.21.15.71:0/3171010709 shutdown_connections 2024-04-23T02:13:58.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.019+0000 7f12dc1f5700 1 --2- 172.21.15.71:0/3171010709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12d4100e10 0x7f12d4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:58.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.019+0000 7f12dc1f5700 1 --2- 172.21.15.71:0/3171010709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d40ffc10 0x7f12d4100030 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:58.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.019+0000 7f12dc1f5700 1 -- 172.21.15.71:0/3171010709 >> 172.21.15.71:0/3171010709 conn(0x7f12d40fb190 msgr2=0x7f12d40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:13:58.026 INFO:tasks.workunit.client.0.smithi071.stdout:1/320: mkdir db/d60 0 2024-04-23T02:13:58.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.023+0000 7f12dc1f5700 1 -- 172.21.15.71:0/3171010709 shutdown_connections 2024-04-23T02:13:58.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.023+0000 7f12dc1f5700 1 -- 172.21.15.71:0/3171010709 wait complete. 2024-04-23T02:13:58.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.024+0000 7f12dc1f5700 1 Processor -- start 2024-04-23T02:13:58.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.024+0000 7f12dc1f5700 1 -- start start 2024-04-23T02:13:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.024+0000 7f12dc1f5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d40ffc10 0x7f12d4071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.024+0000 7f12dc1f5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12d4100e10 0x7f12d40723a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.024+0000 7f12dc1f5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d4072950 con 0x7f12d40ffc10 2024-04-23T02:13:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.025+0000 7f12dc1f5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d4072ac0 con 0x7f12d4100e10 2024-04-23T02:13:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.025+0000 7f12d9f91700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d40ffc10 0x7f12d4071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.025+0000 7f12d9f91700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d40ffc10 0x7f12d4071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50154/0 (socket says 172.21.15.71:50154) 2024-04-23T02:13:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.025+0000 7f12d9f91700 1 -- 172.21.15.71:0/1984734065 learned_addr learned my addr 172.21.15.71:0/1984734065 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:13:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.025+0000 7f12d9790700 1 --2- 172.21.15.71:0/1984734065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12d4100e10 0x7f12d40723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.025+0000 7f12d9f91700 1 -- 172.21.15.71:0/1984734065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12d4100e10 msgr2=0x7f12d40723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.025+0000 7f12d9f91700 1 --2- 172.21.15.71:0/1984734065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12d4100e10 0x7f12d40723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.025+0000 7f12d9f91700 1 -- 172.21.15.71:0/1984734065 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c40095f0 con 0x7f12d40ffc10 2024-04-23T02:13:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.025+0000 7f12d9f91700 1 --2- 172.21.15.71:0/1984734065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d40ffc10 0x7f12d4071e40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f12c400bda0 tx=0x7f12c400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:13:58.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.025+0000 7f12caffd700 1 -- 172.21.15.71:0/1984734065 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12c401d070 con 0x7f12d40ffc10 2024-04-23T02:13:58.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.025+0000 7f12dc1f5700 1 -- 172.21.15.71:0/1984734065 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d4190c00 con 0x7f12d40ffc10 2024-04-23T02:13:58.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.025+0000 7f12caffd700 1 -- 172.21.15.71:0/1984734065 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f12c400f460 con 0x7f12d40ffc10 2024-04-23T02:13:58.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.026+0000 7f12caffd700 1 -- 172.21.15.71:0/1984734065 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12c4017630 con 0x7f12d40ffc10 2024-04-23T02:13:58.031 INFO:tasks.workunit.client.0.smithi071.stdout:5/445: mkdir d3/d34/d6b/d81/d5a/da7/dab 0 2024-04-23T02:13:58.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.026+0000 7f12dc1f5700 1 -- 172.21.15.71:0/1984734065 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d41910f0 con 0x7f12d40ffc10 2024-04-23T02:13:58.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.027+0000 7f12dc1f5700 1 -- 172.21.15.71:0/1984734065 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d41074b0 con 0x7f12d40ffc10 2024-04-23T02:13:58.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.028+0000 7f12caffd700 1 -- 172.21.15.71:0/1984734065 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) v1 ==== 90681+0+0 (secure 0 0 0) 0x7f12c400fa40 con 0x7f12d40ffc10 2024-04-23T02:13:58.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.029+0000 7f12caffd700 1 --2- 172.21.15.71:0/1984734065 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f12c0070e30 0x7f12c00732f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:58.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.029+0000 7f12caffd700 1 -- 172.21.15.71:0/1984734065 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f12c408dc10 con 0x7f12d40ffc10 2024-04-23T02:13:58.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.029+0000 7f12d9790700 1 --2- 172.21.15.71:0/1984734065 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f12c0070e30 0x7f12c00732f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:58.033 INFO:tasks.workunit.client.0.smithi071.stdout:0/301: rename d0/d3/de/d10/d14/d32/d37/d26/c39 to d0/d3/de/d10/d14/d32/d44/c5d 0 2024-04-23T02:13:58.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.029+0000 7f12d9790700 1 --2- 172.21.15.71:0/1984734065 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f12c0070e30 0x7f12c00732f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f12d4072f70 tx=0x7f12d000a380 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:13:58.035 INFO:tasks.workunit.client.0.smithi071.stdout:0/302: chown d0/d3/de/d10/d14/d32/d44/d31/d52 997 1 2024-04-23T02:13:58.035 INFO:tasks.workunit.client.0.smithi071.stdout:1/321: link db/df/f58 db/d34/f61 0 2024-04-23T02:13:58.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.034+0000 7f12caffd700 1 -- 172.21.15.71:0/1984734065 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f12c4057d80 con 0x7f12d40ffc10 2024-04-23T02:13:58.041 INFO:tasks.workunit.client.0.smithi071.stdout:5/446: creat d3/df/d83/fac x:0 0 0 2024-04-23T02:13:58.041 INFO:tasks.workunit.client.0.smithi071.stdout:3/294: rename d7/d27/d3e to d7/d8/d42/d5f 0 2024-04-23T02:13:58.041 INFO:tasks.workunit.client.0.smithi071.stdout:0/303: creat d0/d3/de/d10/d16/f5e x:0 0 0 2024-04-23T02:13:58.043 INFO:tasks.workunit.client.0.smithi071.stdout:2/319: link d0/d2/d26/d14/d1f/d4f/c57 d0/d2/d51/d56/c68 0 2024-04-23T02:13:58.043 INFO:tasks.workunit.client.0.smithi071.stdout:3/295: read d7/d22/f55 [645257,116657] 0 2024-04-23T02:13:58.044 INFO:tasks.workunit.client.0.smithi071.stdout:5/447: truncate d3/df/d10/f60 501544 0 2024-04-23T02:13:58.045 INFO:tasks.workunit.client.0.smithi071.stdout:0/304: unlink d0/f18 0 2024-04-23T02:13:58.047 INFO:tasks.workunit.client.0.smithi071.stdout:3/296: dread d7/d8/f1a [4194304,4194304] 0 2024-04-23T02:13:58.048 INFO:tasks.workunit.client.0.smithi071.stdout:7/294: write d3/fc [1990307,24731] 0 2024-04-23T02:13:58.051 INFO:tasks.workunit.client.0.smithi071.stdout:1/322: link db/df/d13/d2b/l4f db/df/d13/d2b/l62 0 2024-04-23T02:13:58.052 INFO:tasks.workunit.client.0.smithi071.stdout:5/448: truncate d3/da/d1a/d1c/f41 409459 0 2024-04-23T02:13:58.055 INFO:tasks.workunit.client.0.smithi071.stdout:5/449: fdatasync d3/da/f38 0 2024-04-23T02:13:58.056 INFO:tasks.workunit.client.0.smithi071.stdout:1/323: chown db/d34/l52 16312 1 2024-04-23T02:13:58.056 INFO:tasks.workunit.client.0.smithi071.stdout:3/297: truncate d7/d8/d42/f5d 428388 0 2024-04-23T02:13:58.058 INFO:tasks.workunit.client.0.smithi071.stdout:7/295: symlink d3/d17/d41/d37/d4f/l68 0 2024-04-23T02:13:58.058 INFO:tasks.workunit.client.0.smithi071.stdout:4/379: dread d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f5d [0,4194304] 0 2024-04-23T02:13:58.060 INFO:tasks.workunit.client.0.smithi071.stdout:2/320: dwrite d0/d2/d26/d14/f1c [0,4194304] 0 2024-04-23T02:13:58.061 INFO:tasks.workunit.client.0.smithi071.stdout:0/305: dread d0/d3/de/d10/d14/d32/d44/f43 [0,4194304] 0 2024-04-23T02:13:58.062 INFO:tasks.workunit.client.0.smithi071.stdout:1/324: chown db 106447331 1 2024-04-23T02:13:58.067 INFO:tasks.workunit.client.0.smithi071.stdout:5/450: truncate d3/d1d/d43/f69 4678577 0 2024-04-23T02:13:58.067 INFO:tasks.workunit.client.0.smithi071.stdout:4/380: mkdir d2/d2a/d2d/d7b 0 2024-04-23T02:13:58.067 INFO:tasks.workunit.client.0.smithi071.stdout:7/296: stat d3/d17/d41/d34/d39/d3b/f49 0 2024-04-23T02:13:58.067 INFO:tasks.workunit.client.0.smithi071.stdout:3/298: creat d7/d28/d3b/d4c/f60 x:0 0 0 2024-04-23T02:13:58.067 INFO:tasks.workunit.client.0.smithi071.stdout:5/451: fsync d3/d34/d6b/f8d 0 2024-04-23T02:13:58.070 INFO:tasks.workunit.client.0.smithi071.stdout:0/306: symlink d0/d3/de/d10/d14/d32/d37/l5f 0 2024-04-23T02:13:58.072 INFO:tasks.workunit.client.0.smithi071.stdout:2/321: truncate d0/d2/d26/df/f40 4515307 0 2024-04-23T02:13:58.072 INFO:tasks.workunit.client.0.smithi071.stdout:5/452: unlink d3/d1d/f2a 0 2024-04-23T02:13:58.073 INFO:tasks.workunit.client.0.smithi071.stdout:1/325: rename db/df/d13/d20/c33 to db/c63 0 2024-04-23T02:13:58.074 INFO:tasks.workunit.client.0.smithi071.stdout:7/297: chown d3/d17/d1c/c1e 383699978 1 2024-04-23T02:13:58.074 INFO:tasks.workunit.client.0.smithi071.stdout:4/381: chown d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c/c32 3456682 1 2024-04-23T02:13:58.074 INFO:tasks.workunit.client.0.smithi071.stdout:3/299: creat d7/d8/f61 x:0 0 0 2024-04-23T02:13:58.079 INFO:tasks.workunit.client.0.smithi071.stdout:7/298: creat d3/f69 x:0 0 0 2024-04-23T02:13:58.079 INFO:tasks.workunit.client.0.smithi071.stdout:2/322: creat d0/d2/d26/df/f69 x:0 0 0 2024-04-23T02:13:58.080 INFO:tasks.workunit.client.0.smithi071.stdout:5/453: rename d3/da/c27 to d3/d1d/d43/d4b/cad 0 2024-04-23T02:13:58.084 INFO:tasks.workunit.client.0.smithi071.stdout:0/307: dwrite d0/f20 [0,4194304] 0 2024-04-23T02:13:58.085 INFO:tasks.workunit.client.0.smithi071.stdout:7/299: creat d3/d17/d1c/d31/f6a x:0 0 0 2024-04-23T02:13:58.087 INFO:tasks.workunit.client.0.smithi071.stdout:3/300: getdents d7/d28/d3b 0 2024-04-23T02:13:58.091 INFO:tasks.workunit.client.0.smithi071.stdout:2/323: dwrite d0/d2/fb [0,4194304] 0 2024-04-23T02:13:58.094 INFO:tasks.workunit.client.0.smithi071.stdout:0/308: creat d0/d3/de/d10/d14/d32/d37/d26/f60 x:0 0 0 2024-04-23T02:13:58.096 INFO:tasks.workunit.client.0.smithi071.stdout:2/324: mkdir d0/d2/d26/d14/d1f/d36/d6a 0 2024-04-23T02:13:58.097 INFO:tasks.workunit.client.0.smithi071.stdout:3/301: getdents d7/d10/d38 0 2024-04-23T02:13:58.100 INFO:tasks.workunit.client.1.smithi192.stdout:5/454: sync 2024-04-23T02:13:58.100 INFO:tasks.workunit.client.0.smithi071.stdout:7/300: dwrite d3/d17/d41/f33 [0,4194304] 0 2024-04-23T02:13:58.101 INFO:tasks.workunit.client.0.smithi071.stdout:3/302: chown d7/d27/d2c/l39 3558 1 2024-04-23T02:13:58.101 INFO:tasks.workunit.client.0.smithi071.stdout:2/325: write d0/d2/d26/d14/d1f/f25 [4672296,116565] 0 2024-04-23T02:13:58.102 INFO:tasks.workunit.client.1.smithi192.stdout:8/563: dwrite d1/d12/d23/f3c [4194304,4194304] 0 2024-04-23T02:13:58.102 INFO:tasks.workunit.client.1.smithi192.stdout:1/259: creat dc/d14/d15/d35/f49 x:0 0 0 2024-04-23T02:13:58.104 INFO:tasks.workunit.client.1.smithi192.stdout:2/396: dread d8/d26/d27/f45 [0,4194304] 0 2024-04-23T02:13:58.106 INFO:tasks.workunit.client.0.smithi071.stdout:7/301: write d3/d17/d41/f66 [431049,96070] 0 2024-04-23T02:13:58.106 INFO:tasks.workunit.client.1.smithi192.stdout:9/391: dread d0/d9/d44/d51/d5b/f67 [0,4194304] 0 2024-04-23T02:13:58.106 INFO:tasks.workunit.client.0.smithi071.stdout:3/303: unlink d7/d22/l2a 0 2024-04-23T02:13:58.109 INFO:tasks.workunit.client.0.smithi071.stdout:5/454: read d3/da/f13 [623921,130570] 0 2024-04-23T02:13:58.109 INFO:tasks.workunit.client.1.smithi192.stdout:5/455: mkdir d2/d6/d2d/d41/d43/d98 0 2024-04-23T02:13:58.113 INFO:tasks.workunit.client.1.smithi192.stdout:8/564: creat d1/d12/d23/d2c/d4d/fc8 x:0 0 0 2024-04-23T02:13:58.113 INFO:tasks.workunit.client.1.smithi192.stdout:9/392: unlink d0/d9/d44/d58/l75 0 2024-04-23T02:13:58.113 INFO:tasks.workunit.client.1.smithi192.stdout:8/565: write d1/f72 [574222,68859] 0 2024-04-23T02:13:58.115 INFO:tasks.workunit.client.1.smithi192.stdout:2/397: dread d8/d26/f3c [0,4194304] 0 2024-04-23T02:13:58.118 INFO:tasks.workunit.client.1.smithi192.stdout:8/566: creat d1/fc9 x:0 0 0 2024-04-23T02:13:58.119 INFO:tasks.workunit.client.1.smithi192.stdout:2/398: dread d8/d25/f6c [0,4194304] 0 2024-04-23T02:13:58.120 INFO:tasks.workunit.client.1.smithi192.stdout:8/567: mknod d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/cca 0 2024-04-23T02:13:58.121 INFO:tasks.workunit.client.1.smithi192.stdout:2/399: mkdir d8/d26/d74/d7c 0 2024-04-23T02:13:58.123 INFO:tasks.workunit.client.1.smithi192.stdout:8/568: mkdir d1/dcb 0 2024-04-23T02:13:58.124 INFO:tasks.workunit.client.1.smithi192.stdout:2/400: mknod d8/d26/c7d 0 2024-04-23T02:13:58.125 INFO:tasks.workunit.client.1.smithi192.stdout:2/401: truncate d8/d26/d29/f41 724865 0 2024-04-23T02:13:58.126 INFO:tasks.workunit.client.1.smithi192.stdout:8/569: truncate d1/d9/f11 6054727 0 2024-04-23T02:13:58.127 INFO:tasks.workunit.client.1.smithi192.stdout:2/402: creat d8/d14/d51/f7e x:0 0 0 2024-04-23T02:13:58.128 INFO:tasks.workunit.client.1.smithi192.stdout:8/570: truncate d1/d12/d23/d4b/fc7 143689 0 2024-04-23T02:13:58.131 INFO:tasks.workunit.client.1.smithi192.stdout:2/403: creat d8/d26/d29/d43/f7f x:0 0 0 2024-04-23T02:13:58.132 INFO:tasks.workunit.client.1.smithi192.stdout:2/404: write d8/f36 [2304920,30667] 0 2024-04-23T02:13:58.133 INFO:tasks.workunit.client.1.smithi192.stdout:2/405: write d8/d26/f6b [260382,38914] 0 2024-04-23T02:13:58.133 INFO:tasks.workunit.client.0.smithi071.stdout:7/302: read d3/d17/d41/d34/d59/d5f/f60 [4729013,119548] 0 2024-04-23T02:13:58.134 INFO:tasks.workunit.client.0.smithi071.stdout:3/304: creat d7/d28/f62 x:0 0 0 2024-04-23T02:13:58.135 INFO:tasks.workunit.client.1.smithi192.stdout:9/393: read d0/d9/d44/d74/f23 [6475,92190] 0 2024-04-23T02:13:58.136 INFO:tasks.workunit.client.0.smithi071.stdout:2/326: dwrite d0/d2/f9 [0,4194304] 0 2024-04-23T02:13:58.136 INFO:tasks.workunit.client.1.smithi192.stdout:2/406: mkdir d8/d26/d29/d80 0 2024-04-23T02:13:58.138 INFO:tasks.workunit.client.1.smithi192.stdout:2/407: mkdir d8/d26/d81 0 2024-04-23T02:13:58.140 INFO:tasks.workunit.client.1.smithi192.stdout:9/394: rename d0/c3f to d0/d9/d44/d53/c8f 0 2024-04-23T02:13:58.141 INFO:tasks.workunit.client.1.smithi192.stdout:9/395: stat d0/d9/f37 0 2024-04-23T02:13:58.141 INFO:tasks.workunit.client.0.smithi071.stdout:0/309: dread d0/d3/de/d10/d14/d32/f45 [0,4194304] 0 2024-04-23T02:13:58.144 INFO:tasks.workunit.client.0.smithi071.stdout:3/305: creat d7/d28/d3b/d4c/f63 x:0 0 0 2024-04-23T02:13:58.144 INFO:tasks.workunit.client.0.smithi071.stdout:5/455: symlink d3/d34/d6b/d81/d5a/da7/da8/lae 0 2024-04-23T02:13:58.144 INFO:tasks.workunit.client.0.smithi071.stdout:7/303: creat d3/d17/d41/d34/d39/d3b/f6b x:0 0 0 2024-04-23T02:13:58.146 INFO:tasks.workunit.client.0.smithi071.stdout:5/456: read - d3/d1d/f42 zero size 2024-04-23T02:13:58.147 INFO:tasks.workunit.client.1.smithi192.stdout:9/396: dwrite d0/d9/f50 [0,4194304] 0 2024-04-23T02:13:58.147 INFO:tasks.workunit.client.0.smithi071.stdout:3/306: chown d7/d28/f62 284805 1 2024-04-23T02:13:58.149 INFO:tasks.workunit.client.1.smithi192.stdout:9/397: rmdir d0/d9/d44/d74/d3c/d5f 39 2024-04-23T02:13:58.150 INFO:tasks.workunit.client.1.smithi192.stdout:9/398: fsync d0/d9/d44/f45 0 2024-04-23T02:13:58.150 INFO:tasks.workunit.client.0.smithi071.stdout:5/457: stat d3/d1d/d43/f75 0 2024-04-23T02:13:58.152 INFO:tasks.workunit.client.0.smithi071.stdout:0/310: write d0/d3/de/d10/d14/d32/d44/f43 [944379,109447] 0 2024-04-23T02:13:58.154 INFO:tasks.workunit.client.0.smithi071.stdout:5/458: read - d3/df/d83/fac zero size 2024-04-23T02:13:58.159 INFO:tasks.workunit.client.1.smithi192.stdout:9/399: dwrite d0/d9/d44/d51/f66 [0,4194304] 0 2024-04-23T02:13:58.160 INFO:tasks.workunit.client.0.smithi071.stdout:1/326: dwrite db/df/d13/d2b/f35 [0,4194304] 0 2024-04-23T02:13:58.160 INFO:tasks.workunit.client.1.smithi192.stdout:9/400: fsync d0/d9/d44/d4f/f3e 0 2024-04-23T02:13:58.162 INFO:tasks.workunit.client.0.smithi071.stdout:1/327: dread - db/df/d13/d20/f4a zero size 2024-04-23T02:13:58.163 INFO:tasks.workunit.client.0.smithi071.stdout:7/304: rename d3/d17/d41/d34/d39/d3b/f6b to d3/d17/d41/d34/d59/d5f/f6c 0 2024-04-23T02:13:58.163 INFO:tasks.workunit.client.0.smithi071.stdout:2/327: truncate d0/d2/d26/df/f40 2623028 0 2024-04-23T02:13:58.167 INFO:tasks.workunit.client.1.smithi192.stdout:8/571: dread d1/d9/f11 [0,4194304] 0 2024-04-23T02:13:58.170 INFO:tasks.workunit.client.0.smithi071.stdout:2/328: truncate d0/d2/d26/d14/d1f/d4f/f66 167281 0 2024-04-23T02:13:58.172 INFO:tasks.workunit.client.1.smithi192.stdout:9/401: creat d0/d9/d44/d74/d3c/f90 x:0 0 0 2024-04-23T02:13:58.174 INFO:tasks.workunit.client.0.smithi071.stdout:3/307: dwrite d7/d8/f1a [0,4194304] 0 2024-04-23T02:13:58.175 INFO:tasks.workunit.client.1.smithi192.stdout:9/402: mknod d0/d9/d44/d53/c91 0 2024-04-23T02:13:58.177 INFO:tasks.workunit.client.1.smithi192.stdout:8/572: rmdir d1/d12/d23/d4b/d73/d45/dae 0 2024-04-23T02:13:58.177 INFO:tasks.workunit.client.0.smithi071.stdout:5/459: rename d3/df to d3/d1d/daf 0 2024-04-23T02:13:58.179 INFO:tasks.workunit.client.1.smithi192.stdout:9/403: symlink d0/l92 0 2024-04-23T02:13:58.180 INFO:tasks.workunit.client.0.smithi071.stdout:2/329: symlink d0/d2/d26/d14/d1f/d36/l6b 0 2024-04-23T02:13:58.180 INFO:tasks.workunit.client.1.smithi192.stdout:8/573: fdatasync d1/d12/f1e 0 2024-04-23T02:13:58.181 INFO:tasks.workunit.client.0.smithi071.stdout:3/308: mkdir d7/d10/d64 0 2024-04-23T02:13:58.182 INFO:tasks.workunit.client.1.smithi192.stdout:8/574: write d1/d12/f1e [3258376,29067] 0 2024-04-23T02:13:58.183 INFO:tasks.workunit.client.0.smithi071.stdout:0/311: rename d0/d3/de/d10/d14/d32/d44/d31/c50 to d0/d3/de/d10/d14/d32/d37/c61 0 2024-04-23T02:13:58.185 INFO:tasks.workunit.client.1.smithi192.stdout:8/575: chown d1/df/c18 11632260 1 2024-04-23T02:13:58.187 INFO:tasks.workunit.client.1.smithi192.stdout:8/576: dread d1/d12/d23/f37 [0,4194304] 0 2024-04-23T02:13:58.187 INFO:tasks.workunit.client.1.smithi192.stdout:9/404: rename d0/d9/f52 to d0/d9/d44/d74/d3c/d5f/d7d/f93 0 2024-04-23T02:13:58.187 INFO:tasks.workunit.client.0.smithi071.stdout:7/305: truncate d3/d17/d41/f66 183241 0 2024-04-23T02:13:58.188 INFO:tasks.workunit.client.1.smithi192.stdout:9/405: chown d0/d9/d44/d51/l68 139275 1 2024-04-23T02:13:58.190 INFO:tasks.workunit.client.0.smithi071.stdout:2/330: symlink d0/d2/l6c 0 2024-04-23T02:13:58.190 INFO:tasks.workunit.client.1.smithi192.stdout:9/406: dread - d0/d9/d44/d51/d5b/d61/f7b zero size 2024-04-23T02:13:58.191 INFO:tasks.workunit.client.0.smithi071.stdout:3/309: fsync d7/d22/d23/f24 0 2024-04-23T02:13:58.191 INFO:tasks.workunit.client.0.smithi071.stdout:1/328: dwrite db/df/f58 [0,4194304] 0 2024-04-23T02:13:58.193 INFO:tasks.workunit.client.0.smithi071.stdout:3/310: readlink l6 0 2024-04-23T02:13:58.193 INFO:tasks.workunit.client.0.smithi071.stdout:1/329: stat db/df/d13/d2b/l31 0 2024-04-23T02:13:58.193 INFO:tasks.workunit.client.0.smithi071.stdout:1/330: dread - db/f3b zero size 2024-04-23T02:13:58.193 INFO:tasks.workunit.client.0.smithi071.stdout:5/460: link d3/l18 d3/d34/d6b/lb0 0 2024-04-23T02:13:58.195 INFO:tasks.workunit.client.1.smithi192.stdout:8/577: dread d1/d12/d23/d4b/d73/d7e/d88/fa4 [0,4194304] 0 2024-04-23T02:13:58.195 INFO:tasks.workunit.client.0.smithi071.stdout:1/331: chown db/df/l56 127 1 2024-04-23T02:13:58.196 INFO:tasks.workunit.client.0.smithi071.stdout:1/332: fdatasync db/df/d13/d2f/f3e 0 2024-04-23T02:13:58.196 INFO:tasks.workunit.client.0.smithi071.stdout:1/333: dread - db/f3b zero size 2024-04-23T02:13:58.198 INFO:tasks.workunit.client.0.smithi071.stdout:0/312: link d0/d3/f3d d0/d3/f62 0 2024-04-23T02:13:58.201 INFO:tasks.workunit.client.0.smithi071.stdout:5/461: chown d3/lc 8756 1 2024-04-23T02:13:58.201 INFO:tasks.workunit.client.0.smithi071.stdout:1/334: write db/df/d13/d2f/f3c [367864,97885] 0 2024-04-23T02:13:58.208 INFO:tasks.workunit.client.0.smithi071.stdout:3/311: dwrite d7/d28/d3b/d4c/f59 [0,4194304] 0 2024-04-23T02:13:58.209 INFO:tasks.workunit.client.0.smithi071.stdout:1/335: fsync db/df/d3a/f44 0 2024-04-23T02:13:58.210 INFO:tasks.workunit.client.0.smithi071.stdout:5/462: symlink d3/d86/lb1 0 2024-04-23T02:13:58.211 INFO:tasks.workunit.client.0.smithi071.stdout:1/336: readlink db/df/d3a/l5a 0 2024-04-23T02:13:58.212 INFO:tasks.workunit.client.0.smithi071.stdout:0/313: getdents d0/d3/de 0 2024-04-23T02:13:58.214 INFO:tasks.workunit.client.0.smithi071.stdout:5/463: creat d3/da/d1a/d1c/fb2 x:0 0 0 2024-04-23T02:13:58.215 INFO:tasks.workunit.client.0.smithi071.stdout:1/337: rmdir db/df/d13/d2f 39 2024-04-23T02:13:58.218 INFO:tasks.workunit.client.0.smithi071.stdout:1/338: read db/df/f2a [334778,88590] 0 2024-04-23T02:13:58.219 INFO:tasks.workunit.client.0.smithi071.stdout:5/464: symlink d3/d34/d6b/d81/d5a/da7/da8/lb3 0 2024-04-23T02:13:58.220 INFO:tasks.workunit.client.1.smithi192.stdout:8/578: dread d1/f10 [8388608,4194304] 0 2024-04-23T02:13:58.222 INFO:tasks.workunit.client.0.smithi071.stdout:1/339: creat db/d11/f64 x:0 0 0 2024-04-23T02:13:58.226 INFO:tasks.workunit.client.0.smithi071.stdout:1/340: dread - db/df/d13/d2b/f43 zero size 2024-04-23T02:13:58.227 INFO:tasks.workunit.client.0.smithi071.stdout:5/465: write d3/d34/d6b/fa3 [804784,121768] 0 2024-04-23T02:13:58.227 INFO:tasks.workunit.client.0.smithi071.stdout:7/306: dread d3/d17/d41/d37/f38 [0,4194304] 0 2024-04-23T02:13:58.227 INFO:tasks.workunit.client.1.smithi192.stdout:8/579: dwrite d1/d12/d23/d4b/d73/d4c/d82/fb6 [0,4194304] 0 2024-04-23T02:13:58.228 INFO:tasks.workunit.client.0.smithi071.stdout:0/314: truncate d0/d3/de/d10/d14/d32/f41 2408128 0 2024-04-23T02:13:58.229 INFO:tasks.workunit.client.0.smithi071.stdout:1/341: unlink db/df/d13/f36 0 2024-04-23T02:13:58.229 INFO:tasks.workunit.client.0.smithi071.stdout:7/307: truncate d3/d17/d41/d34/d39/d3b/f46 851837 0 2024-04-23T02:13:58.229 INFO:tasks.workunit.client.0.smithi071.stdout:1/342: readlink db/d34/l38 0 2024-04-23T02:13:58.230 INFO:tasks.workunit.client.0.smithi071.stdout:5/466: mkdir d3/d34/d6b/d81/da9/db4 0 2024-04-23T02:13:58.232 INFO:tasks.workunit.client.1.smithi192.stdout:8/580: unlink d1/d12/d23/d4b/d73/d45/d46/l61 0 2024-04-23T02:13:58.234 INFO:tasks.workunit.client.0.smithi071.stdout:0/315: mkdir d0/d63 0 2024-04-23T02:13:58.235 INFO:tasks.workunit.client.1.smithi192.stdout:8/581: symlink d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/lcc 0 2024-04-23T02:13:58.237 INFO:tasks.workunit.client.0.smithi071.stdout:0/316: readlink d0/d3/d1c/l12 0 2024-04-23T02:13:58.238 INFO:tasks.workunit.client.0.smithi071.stdout:1/343: symlink db/df/d13/d2f/d48/l65 0 2024-04-23T02:13:58.239 INFO:tasks.workunit.client.0.smithi071.stdout:7/308: getdents d3/d17/d41/d37/d4f 0 2024-04-23T02:13:58.240 INFO:tasks.workunit.client.0.smithi071.stdout:0/317: fdatasync d0/d3/de/d10/d14/d32/f45 0 2024-04-23T02:13:58.242 INFO:tasks.workunit.client.0.smithi071.stdout:1/344: creat db/df/d3a/f66 x:0 0 0 2024-04-23T02:13:58.243 INFO:tasks.workunit.client.1.smithi192.stdout:8/582: dwrite d1/d12/d7d/f89 [0,4194304] 0 2024-04-23T02:13:58.243 INFO:tasks.workunit.client.1.smithi192.stdout:8/583: readlink d1/d12/d68/l6c 0 2024-04-23T02:13:58.244 INFO:tasks.workunit.client.0.smithi071.stdout:5/467: dwrite d3/da/d1a/d1c/f31 [0,4194304] 0 2024-04-23T02:13:58.245 INFO:tasks.workunit.client.0.smithi071.stdout:7/309: rmdir d3/d17/d41/d34/d39 39 2024-04-23T02:13:58.247 INFO:tasks.workunit.client.0.smithi071.stdout:3/312: dread d7/d8/f16 [0,4194304] 0 2024-04-23T02:13:58.247 INFO:tasks.workunit.client.1.smithi192.stdout:8/584: link d1/d9/d3e/lb3 d1/d12/d23/d4b/d73/d4c/lcd 0 2024-04-23T02:13:58.252 INFO:tasks.workunit.client.0.smithi071.stdout:1/345: symlink db/df/d3a/l67 0 2024-04-23T02:13:58.254 INFO:tasks.workunit.client.0.smithi071.stdout:1/346: fdatasync db/df/d3a/f47 0 2024-04-23T02:13:58.260 INFO:tasks.workunit.client.0.smithi071.stdout:5/468: creat d3/d34/d6b/d81/da9/db4/fb5 x:0 0 0 2024-04-23T02:13:58.269 INFO:tasks.workunit.client.0.smithi071.stdout:3/313: dwrite d7/d8/d42/d5f/f5b [0,4194304] 0 2024-04-23T02:13:58.269 INFO:tasks.workunit.client.0.smithi071.stdout:5/469: unlink d3/da/d1a/d1c/d46/d4e/l7c 0 2024-04-23T02:13:58.270 INFO:tasks.workunit.client.0.smithi071.stdout:1/347: dwrite db/df/d13/d2b/f35 [0,4194304] 0 2024-04-23T02:13:58.270 INFO:tasks.workunit.client.0.smithi071.stdout:5/470: mkdir d3/da/d1a/d1c/db6 0 2024-04-23T02:13:58.271 INFO:tasks.workunit.client.0.smithi071.stdout:3/314: dread d7/d22/d23/d26/f2b [0,4194304] 0 2024-04-23T02:13:58.272 INFO:tasks.workunit.client.0.smithi071.stdout:1/348: mkdir db/d34/d68 0 2024-04-23T02:13:58.274 INFO:tasks.workunit.client.0.smithi071.stdout:1/349: truncate db/df/d13/d2f/f3e 446539 0 2024-04-23T02:13:58.276 INFO:tasks.workunit.client.0.smithi071.stdout:3/315: truncate d7/d8/f1a 4429795 0 2024-04-23T02:13:58.279 INFO:tasks.workunit.client.0.smithi071.stdout:1/350: symlink db/df/d13/d2b/l69 0 2024-04-23T02:13:58.280 INFO:tasks.workunit.client.0.smithi071.stdout:3/316: unlink l4 0 2024-04-23T02:13:58.282 INFO:tasks.workunit.client.0.smithi071.stdout:7/310: write d3/d17/d41/f66 [371204,85192] 0 2024-04-23T02:13:58.284 INFO:tasks.workunit.client.1.smithi192.stdout:8/585: dread d1/d9/f22 [0,4194304] 0 2024-04-23T02:13:58.284 INFO:tasks.workunit.client.0.smithi071.stdout:3/317: creat d7/d28/d3b/d4c/f65 x:0 0 0 2024-04-23T02:13:58.285 INFO:tasks.workunit.client.0.smithi071.stdout:1/351: creat db/f6a x:0 0 0 2024-04-23T02:13:58.285 INFO:tasks.workunit.client.0.smithi071.stdout:3/318: readlink d7/d10/l2d 0 2024-04-23T02:13:58.286 INFO:tasks.workunit.client.1.smithi192.stdout:8/586: creat d1/d12/d7d/fce x:0 0 0 2024-04-23T02:13:58.288 INFO:tasks.workunit.client.1.smithi192.stdout:8/587: creat d1/d12/d23/d2c/d4d/fcf x:0 0 0 2024-04-23T02:13:58.288 INFO:tasks.workunit.client.0.smithi071.stdout:3/319: fdatasync d7/d10/d38/f41 0 2024-04-23T02:13:58.290 INFO:tasks.workunit.client.0.smithi071.stdout:7/311: link d3/d17/d1c/c28 d3/d17/d41/d37/d4f/c6d 0 2024-04-23T02:13:58.291 INFO:tasks.workunit.client.0.smithi071.stdout:3/320: write d7/d8/d42/d5f/f40 [8677237,92228] 0 2024-04-23T02:13:58.294 INFO:tasks.workunit.client.1.smithi192.stdout:8/588: dwrite d1/f15 [0,4194304] 0 2024-04-23T02:13:58.296 INFO:tasks.workunit.client.0.smithi071.stdout:3/321: unlink d7/d27/c45 0 2024-04-23T02:13:58.298 INFO:tasks.workunit.client.0.smithi071.stdout:5/471: dread d3/d1d/d2c/f2d [0,4194304] 0 2024-04-23T02:13:58.298 INFO:tasks.workunit.client.1.smithi192.stdout:8/589: mkdir d1/df/dd0 0 2024-04-23T02:13:58.298 INFO:tasks.workunit.client.0.smithi071.stdout:3/322: write d7/d8/d42/d5f/f5e [750355,56189] 0 2024-04-23T02:13:58.299 INFO:tasks.workunit.client.0.smithi071.stdout:7/312: dwrite d3/d17/d1c/d31/f56 [0,4194304] 0 2024-04-23T02:13:58.302 INFO:tasks.workunit.client.1.smithi192.stdout:8/590: link d1/d12/d23/c44 d1/d12/d68/cd1 0 2024-04-23T02:13:58.304 INFO:tasks.workunit.client.0.smithi071.stdout:0/318: dread d0/d3/de/d10/d14/d32/f41 [0,4194304] 0 2024-04-23T02:13:58.307 INFO:tasks.workunit.client.0.smithi071.stdout:7/313: read d3/d17/d1c/f2c [1769375,20189] 0 2024-04-23T02:13:58.308 INFO:tasks.workunit.client.0.smithi071.stdout:7/314: chown d3/d17/d1c/d31/l65 1531041 1 2024-04-23T02:13:58.309 INFO:tasks.workunit.client.0.smithi071.stdout:7/315: readlink d3/l10 0 2024-04-23T02:13:58.309 INFO:tasks.workunit.client.0.smithi071.stdout:3/323: chown d7/d8/f61 179752 1 2024-04-23T02:13:58.312 INFO:tasks.workunit.client.1.smithi192.stdout:8/591: dwrite d1/d12/d23/d4b/d73/d4c/d82/fb6 [4194304,4194304] 0 2024-04-23T02:13:58.315 INFO:tasks.workunit.client.1.smithi192.stdout:8/592: stat d1/d12/d23/d4b/d73/d4c/l7a 0 2024-04-23T02:13:58.316 INFO:tasks.workunit.client.1.smithi192.stdout:8/593: mkdir d1/d12/d7d/dd2 0 2024-04-23T02:13:58.317 INFO:tasks.workunit.client.0.smithi071.stdout:5/472: link d3/d1d/d2c/c45 d3/d34/d6b/d81/d5a/cb7 0 2024-04-23T02:13:58.318 INFO:tasks.workunit.client.1.smithi192.stdout:8/594: fdatasync d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/f98 0 2024-04-23T02:13:58.318 INFO:tasks.workunit.client.0.smithi071.stdout:0/319: creat d0/d3/de/d10/d14/f64 x:0 0 0 2024-04-23T02:13:58.319 INFO:tasks.workunit.client.0.smithi071.stdout:7/316: mkdir d3/d17/d41/d34/d6e 0 2024-04-23T02:13:58.321 INFO:tasks.workunit.client.1.smithi192.stdout:8/595: dread d1/d9/d8f/d90/f93 [0,4194304] 0 2024-04-23T02:13:58.322 INFO:tasks.workunit.client.0.smithi071.stdout:7/317: chown d3/d17/d41/d37/d4f/f5c 3860274 1 2024-04-23T02:13:58.323 INFO:tasks.workunit.client.0.smithi071.stdout:7/318: chown d3/d17/d41/d34/f3c 1863 1 2024-04-23T02:13:58.325 INFO:tasks.workunit.client.0.smithi071.stdout:0/320: dread d0/d3/de/d10/d14/f4d [0,4194304] 0 2024-04-23T02:13:58.326 INFO:tasks.workunit.client.0.smithi071.stdout:7/319: rmdir d3/d17/d1c/d31 39 2024-04-23T02:13:58.328 INFO:tasks.workunit.client.1.smithi192.stdout:8/596: dwrite d1/d9/d3e/fb2 [0,4194304] 0 2024-04-23T02:13:58.330 INFO:tasks.workunit.client.0.smithi071.stdout:0/321: chown d0/d3/de/d10/d14/d32/d44/l5a 380082650 1 2024-04-23T02:13:58.331 INFO:tasks.workunit.client.0.smithi071.stdout:5/473: rename d3/d1d/d2c to d3/da/d1a/db8 0 2024-04-23T02:13:58.333 INFO:tasks.workunit.client.0.smithi071.stdout:7/320: unlink d3/d17/d41/f5e 0 2024-04-23T02:13:58.333 INFO:tasks.workunit.client.1.smithi192.stdout:8/597: rename d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/f98 to d1/d12/d23/fd3 0 2024-04-23T02:13:58.334 INFO:tasks.workunit.client.0.smithi071.stdout:7/321: write d3/f69 [932842,45124] 0 2024-04-23T02:13:58.335 INFO:tasks.workunit.client.0.smithi071.stdout:7/322: chown d3/l10 1113656792 1 2024-04-23T02:13:58.335 INFO:tasks.workunit.client.1.smithi192.stdout:8/598: mknod d1/d12/d23/d4b/d73/d5d/cd4 0 2024-04-23T02:13:58.336 INFO:tasks.workunit.client.1.smithi192.stdout:8/599: dread - d1/d9/d8f/d90/fa2 zero size 2024-04-23T02:13:58.338 INFO:tasks.workunit.client.0.smithi071.stdout:0/322: rename d0/d3/d1c/l11 to d0/d63/l65 0 2024-04-23T02:13:58.339 INFO:tasks.workunit.client.1.smithi192.stdout:8/600: rename d1/d12/d23/d2c/d8c/faf to d1/d12/d7d/fd5 0 2024-04-23T02:13:58.341 INFO:tasks.workunit.client.0.smithi071.stdout:5/474: mkdir d3/d1d/db9 0 2024-04-23T02:13:58.343 INFO:tasks.workunit.client.0.smithi071.stdout:0/323: creat d0/d63/f66 x:0 0 0 2024-04-23T02:13:58.345 INFO:tasks.workunit.client.1.smithi192.stdout:8/601: write d1/d12/d23/d4b/f51 [579944,74233] 0 2024-04-23T02:13:58.345 INFO:tasks.workunit.client.0.smithi071.stdout:3/324: rename d7/d8/d4f to d7/d8/d42/d5f/d66 0 2024-04-23T02:13:58.346 INFO:tasks.workunit.client.1.smithi192.stdout:8/602: chown d1/d12/d23/d4b/d73/d4c/f84 238277876 1 2024-04-23T02:13:58.347 INFO:tasks.workunit.client.1.smithi192.stdout:8/603: fdatasync d1/d12/d23/d4b/d73/d45/f76 0 2024-04-23T02:13:58.347 INFO:tasks.workunit.client.1.smithi192.stdout:7/424: sync 2024-04-23T02:13:58.347 INFO:tasks.workunit.client.0.smithi071.stdout:5/475: getdents d3/d34/d6b/d81/d5a/da7/dab 0 2024-04-23T02:13:58.348 INFO:tasks.workunit.client.1.smithi192.stdout:7/425: chown d5/d17/d1c/d59/d5a/d70 533472 1 2024-04-23T02:13:58.348 INFO:tasks.workunit.client.0.smithi071.stdout:0/324: dread d0/d3/de/d10/d14/d32/f45 [0,4194304] 0 2024-04-23T02:13:58.348 INFO:tasks.workunit.client.0.smithi071.stdout:5/476: chown d3/d1d/daf/d10/f14 86097639 1 2024-04-23T02:13:58.350 INFO:tasks.workunit.client.1.smithi192.stdout:7/426: write d5/d17/d1c/d4a/f58 [129767,85768] 0 2024-04-23T02:13:58.350 INFO:tasks.workunit.client.0.smithi071.stdout:5/477: creat d3/da/d1a/d6d/fba x:0 0 0 2024-04-23T02:13:58.352 INFO:tasks.workunit.client.0.smithi071.stdout:0/325: write d0/d3/de/d10/d14/d32/d37/f53 [331209,28202] 0 2024-04-23T02:13:58.353 INFO:tasks.workunit.client.1.smithi192.stdout:7/427: creat d5/d17/d1c/d59/d5a/d41/d50/f7b x:0 0 0 2024-04-23T02:13:58.354 INFO:tasks.workunit.client.0.smithi071.stdout:5/478: creat d3/d34/d6b/d81/d5a/da7/da8/fbb x:0 0 0 2024-04-23T02:13:58.356 INFO:tasks.workunit.client.0.smithi071.stdout:7/323: dread d3/d17/d1c/d2a/d43/f4e [0,4194304] 0 2024-04-23T02:13:58.357 INFO:tasks.workunit.client.0.smithi071.stdout:5/479: chown d3/d34/d6b/d81/d5a/l9f 1085 1 2024-04-23T02:13:58.358 INFO:tasks.workunit.client.1.smithi192.stdout:8/604: dread d1/d12/d23/d4b/d73/d4c/f6e [0,4194304] 0 2024-04-23T02:13:58.359 INFO:tasks.workunit.client.1.smithi192.stdout:8/605: chown d1/f15 901474986 1 2024-04-23T02:13:58.359 INFO:tasks.workunit.client.1.smithi192.stdout:8/606: chown d1/l5b 608478 1 2024-04-23T02:13:58.360 INFO:tasks.workunit.client.1.smithi192.stdout:8/607: fsync d1/df/f50 0 2024-04-23T02:13:58.360 INFO:tasks.workunit.client.0.smithi071.stdout:7/324: rmdir d3/d17/d41/d37 39 2024-04-23T02:13:58.361 INFO:tasks.workunit.client.0.smithi071.stdout:0/326: dwrite d0/d3/de/f28 [0,4194304] 0 2024-04-23T02:13:58.361 INFO:tasks.workunit.client.1.smithi192.stdout:7/428: write d5/d17/d1c/d59/d5a/d41/f54 [669660,34705] 0 2024-04-23T02:13:58.362 INFO:tasks.workunit.client.1.smithi192.stdout:8/608: write d1/d12/d23/d2c/d4d/fc8 [963581,10892] 0 2024-04-23T02:13:58.363 INFO:tasks.workunit.client.0.smithi071.stdout:5/480: unlink d3/d34/d6b/d81/da9/db4/fb5 0 2024-04-23T02:13:58.364 INFO:tasks.workunit.client.1.smithi192.stdout:7/429: symlink d5/d43/d4c/d5c/l7c 0 2024-04-23T02:13:58.364 INFO:tasks.workunit.client.0.smithi071.stdout:0/327: read d0/d3/de/f1d [6980083,124535] 0 2024-04-23T02:13:58.365 INFO:tasks.workunit.client.1.smithi192.stdout:8/609: creat d1/d12/d23/d4b/d73/d45/d46/fd6 x:0 0 0 2024-04-23T02:13:58.369 INFO:tasks.workunit.client.1.smithi192.stdout:7/430: read d5/d17/d1c/d26/f28 [2452003,104550] 0 2024-04-23T02:13:58.371 INFO:tasks.workunit.client.0.smithi071.stdout:3/325: dread d7/d8/f11 [0,4194304] 0 2024-04-23T02:13:58.372 INFO:tasks.workunit.client.0.smithi071.stdout:7/325: creat d3/d17/d41/d37/f6f x:0 0 0 2024-04-23T02:13:58.373 INFO:tasks.workunit.client.0.smithi071.stdout:5/481: mknod d3/d1d/cbc 0 2024-04-23T02:13:58.376 INFO:tasks.workunit.client.0.smithi071.stdout:8/421: sync 2024-04-23T02:13:58.379 INFO:tasks.workunit.client.0.smithi071.stdout:3/326: mkdir d7/d10/d38/d67 0 2024-04-23T02:13:58.381 INFO:tasks.workunit.client.0.smithi071.stdout:0/328: dwrite d0/d3/de/d10/d14/d32/d37/f53 [0,4194304] 0 2024-04-23T02:13:58.381 INFO:tasks.workunit.client.0.smithi071.stdout:7/326: symlink d3/d17/d41/d34/d39/d3b/l70 0 2024-04-23T02:13:58.384 INFO:tasks.workunit.client.0.smithi071.stdout:7/327: write d3/f12 [7783124,123693] 0 2024-04-23T02:13:58.386 INFO:tasks.workunit.client.0.smithi071.stdout:8/422: dwrite f6 [4194304,4194304] 0 2024-04-23T02:13:58.393 INFO:tasks.workunit.client.0.smithi071.stdout:3/327: rename d7/d28 to d7/d8/d42/d5f/d68 0 2024-04-23T02:13:58.394 INFO:tasks.workunit.client.0.smithi071.stdout:5/482: write d3/da/d1a/f2b [1492236,129645] 0 2024-04-23T02:13:58.397 INFO:tasks.workunit.client.0.smithi071.stdout:3/328: symlink d7/d10/d38/l69 0 2024-04-23T02:13:58.398 INFO:tasks.workunit.client.0.smithi071.stdout:0/329: write d0/d3/de/d10/d14/d32/f45 [347073,85047] 0 2024-04-23T02:13:58.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.395+0000 7f12dc1f5700 1 -- 172.21.15.71:0/1984734065 --> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f12d4105760 con 0x7f12c0070e30 2024-04-23T02:13:58.400 INFO:tasks.workunit.client.0.smithi071.stdout:7/328: rename d3/d17/l61 to d3/d17/d41/l71 0 2024-04-23T02:13:58.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.397+0000 7f12caffd700 1 -- 172.21.15.71:0/1984734065 <== mgr.24351 v2:172.21.15.192:6828/966231696 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f12d4105760 con 0x7f12c0070e30 2024-04-23T02:13:58.401 INFO:tasks.workunit.client.0.smithi071.stdout:5/483: symlink d3/d1d/db9/lbd 0 2024-04-23T02:13:58.402 INFO:tasks.workunit.client.0.smithi071.stdout:0/330: fdatasync d0/d3/de/ff 0 2024-04-23T02:13:58.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.399+0000 7f12dc1f5700 1 -- 172.21.15.71:0/1984734065 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f12c0070e30 msgr2=0x7f12c00732f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:58.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.399+0000 7f12dc1f5700 1 --2- 172.21.15.71:0/1984734065 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f12c0070e30 0x7f12c00732f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f12d4072f70 tx=0x7f12d000a380 comp rx=0 tx=0).stop 2024-04-23T02:13:58.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.400+0000 7f12dc1f5700 1 -- 172.21.15.71:0/1984734065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d40ffc10 msgr2=0x7f12d4071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:58.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.400+0000 7f12dc1f5700 1 --2- 172.21.15.71:0/1984734065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d40ffc10 0x7f12d4071e40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f12c400bda0 tx=0x7f12c400bdd0 comp rx=0 tx=0).stop 2024-04-23T02:13:58.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.400+0000 7f12dc1f5700 1 -- 172.21.15.71:0/1984734065 shutdown_connections 2024-04-23T02:13:58.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.400+0000 7f12dc1f5700 1 --2- 172.21.15.71:0/1984734065 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f12c0070e30 0x7f12c00732f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:58.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.400+0000 7f12dc1f5700 1 --2- 172.21.15.71:0/1984734065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d40ffc10 0x7f12d4071e40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:58.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.400+0000 7f12dc1f5700 1 --2- 172.21.15.71:0/1984734065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12d4100e10 0x7f12d40723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:58.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.400+0000 7f12dc1f5700 1 -- 172.21.15.71:0/1984734065 >> 172.21.15.71:0/1984734065 conn(0x7f12d40fb190 msgr2=0x7f12d4104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:13:58.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.400+0000 7f12dc1f5700 1 -- 172.21.15.71:0/1984734065 shutdown_connections 2024-04-23T02:13:58.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.400+0000 7f12dc1f5700 1 -- 172.21.15.71:0/1984734065 wait complete. 2024-04-23T02:13:58.405 INFO:tasks.workunit.client.0.smithi071.stdout:5/484: creat d3/da/d1a/d6d/fbe x:0 0 0 2024-04-23T02:13:58.405 INFO:tasks.workunit.client.0.smithi071.stdout:8/423: rename dc/d10/l7a to dc/d15/d5d/l83 0 2024-04-23T02:13:58.407 INFO:tasks.workunit.client.0.smithi071.stdout:0/331: symlink d0/d3/de/d10/d14/d32/d44/l67 0 2024-04-23T02:13:58.408 INFO:tasks.workunit.client.0.smithi071.stdout:3/329: rename l6 to d7/d27/d2c/l6a 0 2024-04-23T02:13:58.409 INFO:tasks.workunit.client.0.smithi071.stdout:8/424: symlink dc/d15/d50/d5f/l84 0 2024-04-23T02:13:58.412 INFO:tasks.workunit.client.0.smithi071.stdout:8/425: chown dc/d15/d2d/c33 370338406 1 2024-04-23T02:13:58.413 INFO:tasks.workunit.client.0.smithi071.stdout:3/330: creat d7/d22/d23/f6b x:0 0 0 2024-04-23T02:13:58.415 INFO:tasks.workunit.client.0.smithi071.stdout:8/426: write dc/d10/f14 [2743739,81473] 0 2024-04-23T02:13:58.415 INFO:tasks.workunit.client.0.smithi071.stdout:7/329: dread d3/fc [0,4194304] 0 2024-04-23T02:13:58.416 INFO:tasks.workunit.client.0.smithi071.stdout:7/330: stat d3/d17/c19 0 2024-04-23T02:13:58.417 INFO:tasks.workunit.client.0.smithi071.stdout:0/332: dwrite d0/d3/de/d10/d16/f5e [0,4194304] 0 2024-04-23T02:13:58.423 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:13:58.434 INFO:tasks.workunit.client.0.smithi071.stdout:3/331: dwrite d7/d22/d23/f24 [0,4194304] 0 2024-04-23T02:13:58.434 INFO:tasks.workunit.client.0.smithi071.stdout:0/333: dwrite d0/d3/f3f [0,4194304] 0 2024-04-23T02:13:58.434 INFO:tasks.workunit.client.0.smithi071.stdout:7/331: write d3/d17/f36 [681636,115455] 0 2024-04-23T02:13:58.439 INFO:tasks.workunit.client.0.smithi071.stdout:3/332: chown d7/d8/fc 4 1 2024-04-23T02:13:58.443 INFO:tasks.workunit.client.0.smithi071.stdout:0/334: symlink d0/d3/de/d10/d14/d32/d44/d31/d52/l68 0 2024-04-23T02:13:58.445 INFO:tasks.workunit.client.0.smithi071.stdout:0/335: readlink d0/d3/l3a 0 2024-04-23T02:13:58.445 INFO:tasks.workunit.client.0.smithi071.stdout:3/333: creat d7/d1d/f6c x:0 0 0 2024-04-23T02:13:58.448 INFO:tasks.workunit.client.0.smithi071.stdout:7/332: dwrite d3/d17/d1c/f64 [0,4194304] 0 2024-04-23T02:13:58.448 INFO:tasks.workunit.client.0.smithi071.stdout:8/427: dread dc/d15/d2d/f43 [0,4194304] 0 2024-04-23T02:13:58.451 INFO:tasks.workunit.client.0.smithi071.stdout:0/336: truncate d0/d3/f3d 988659 0 2024-04-23T02:13:58.454 INFO:tasks.workunit.client.0.smithi071.stdout:8/428: rename dc/d15/f79 to dc/d10/f85 0 2024-04-23T02:13:58.458 INFO:tasks.workunit.client.0.smithi071.stdout:3/334: rename d7/d8/d42/d5f/d68/d3b/c3c to d7/d10/d64/c6d 0 2024-04-23T02:13:58.459 INFO:tasks.workunit.client.0.smithi071.stdout:8/429: creat dc/d15/f86 x:0 0 0 2024-04-23T02:13:58.459 INFO:tasks.workunit.client.0.smithi071.stdout:7/333: link d3/d17/d41/d34/d39/f54 d3/d17/d41/d34/d59/f72 0 2024-04-23T02:13:58.464 INFO:tasks.workunit.client.0.smithi071.stdout:8/430: creat dc/d15/f87 x:0 0 0 2024-04-23T02:13:58.464 INFO:tasks.workunit.client.0.smithi071.stdout:0/337: dwrite d0/d3/de/d10/d16/f5e [0,4194304] 0 2024-04-23T02:13:58.466 INFO:tasks.workunit.client.0.smithi071.stdout:0/338: chown d0/f2 3034450 1 2024-04-23T02:13:58.467 INFO:tasks.workunit.client.0.smithi071.stdout:7/334: dread d3/d17/d41/f33 [0,4194304] 0 2024-04-23T02:13:58.468 INFO:tasks.workunit.client.0.smithi071.stdout:3/335: rename d7/c32 to d7/d8/d42/d5f/d68/d3b/c6e 0 2024-04-23T02:13:58.469 INFO:tasks.workunit.client.1.smithi192.stdout:8/610: mknod d1/d9/d8f/cd7 0 2024-04-23T02:13:58.469 INFO:tasks.workunit.client.0.smithi071.stdout:8/431: creat dc/d15/d2d/f88 x:0 0 0 2024-04-23T02:13:58.470 INFO:tasks.workunit.client.1.smithi192.stdout:8/611: dread - d1/d12/d7d/fd5 zero size 2024-04-23T02:13:58.471 INFO:tasks.workunit.client.0.smithi071.stdout:8/432: chown dc/d3a 89602 1 2024-04-23T02:13:58.472 INFO:tasks.workunit.client.1.smithi192.stdout:7/431: dwrite d5/d17/d1c/d59/d5a/d41/d50/f62 [0,4194304] 0 2024-04-23T02:13:58.473 INFO:tasks.workunit.client.1.smithi192.stdout:8/612: write d1/d12/d68/fad [1630967,123440] 0 2024-04-23T02:13:58.475 INFO:tasks.workunit.client.1.smithi192.stdout:7/432: fdatasync d5/d17/d1c/d59/d5a/d41/d50/f69 0 2024-04-23T02:13:58.477 INFO:tasks.workunit.client.0.smithi071.stdout:8/433: unlink dc/d15/d2d/f43 0 2024-04-23T02:13:58.477 INFO:tasks.workunit.client.0.smithi071.stdout:0/339: getdents d0/d3/de/d10/d14/d32/d44/d31/d52 0 2024-04-23T02:13:58.478 INFO:tasks.workunit.client.1.smithi192.stdout:8/613: creat d1/d12/d23/d4b/d73/d5d/fd8 x:0 0 0 2024-04-23T02:13:58.479 INFO:tasks.workunit.client.1.smithi192.stdout:7/433: chown c3 100180 1 2024-04-23T02:13:58.480 INFO:tasks.workunit.client.1.smithi192.stdout:7/434: stat d5/d17/d1c/l52 0 2024-04-23T02:13:58.481 INFO:tasks.workunit.client.0.smithi071.stdout:7/335: dwrite d3/f1d [0,4194304] 0 2024-04-23T02:13:58.482 INFO:tasks.workunit.client.1.smithi192.stdout:7/435: mknod d5/d43/c7d 0 2024-04-23T02:13:58.483 INFO:tasks.workunit.client.1.smithi192.stdout:8/614: creat d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/fd9 x:0 0 0 2024-04-23T02:13:58.484 INFO:tasks.workunit.client.0.smithi071.stdout:0/340: truncate d0/d3/de/d10/d14/d32/f41 806977 0 2024-04-23T02:13:58.486 INFO:tasks.workunit.client.0.smithi071.stdout:8/434: dread dc/d15/d2d/f38 [0,4194304] 0 2024-04-23T02:13:58.486 INFO:tasks.workunit.client.1.smithi192.stdout:7/436: unlink d5/d17/d1c/d59/d5a/d41/c7a 0 2024-04-23T02:13:58.487 INFO:tasks.workunit.client.0.smithi071.stdout:7/336: truncate d3/d17/f36 1514447 0 2024-04-23T02:13:58.488 INFO:tasks.workunit.client.0.smithi071.stdout:8/435: chown dc/d10/f1e 199186 1 2024-04-23T02:13:58.489 INFO:tasks.workunit.client.0.smithi071.stdout:3/336: dwrite d7/d22/f55 [0,4194304] 0 2024-04-23T02:13:58.492 INFO:tasks.workunit.client.0.smithi071.stdout:0/341: rename d0/d3/l3a to d0/l69 0 2024-04-23T02:13:58.493 INFO:tasks.workunit.client.1.smithi192.stdout:7/437: creat d5/d17/d1c/d59/d5a/d70/f7e x:0 0 0 2024-04-23T02:13:58.493 INFO:tasks.workunit.client.0.smithi071.stdout:7/337: truncate d3/fd 1549780 0 2024-04-23T02:13:58.494 INFO:tasks.workunit.client.1.smithi192.stdout:8/615: dwrite d1/d12/d23/d2c/d4d/fc8 [0,4194304] 0 2024-04-23T02:13:58.496 INFO:tasks.workunit.client.0.smithi071.stdout:3/337: mkdir d7/d10/d38/d67/d6f 0 2024-04-23T02:13:58.496 INFO:tasks.workunit.client.1.smithi192.stdout:7/438: fdatasync d5/d17/d1c/d59/d5a/d41/d50/f62 0 2024-04-23T02:13:58.497 INFO:tasks.workunit.client.0.smithi071.stdout:3/338: chown d7/d8/d42/d5f 5871 1 2024-04-23T02:13:58.497 INFO:tasks.workunit.client.0.smithi071.stdout:0/342: creat d0/d3/de/f6a x:0 0 0 2024-04-23T02:13:58.498 INFO:tasks.workunit.client.0.smithi071.stdout:7/338: mkdir d3/d17/d1c/d2a/d73 0 2024-04-23T02:13:58.502 INFO:tasks.workunit.client.1.smithi192.stdout:8/616: mkdir d1/d12/d23/d4b/d73/dda 0 2024-04-23T02:13:58.502 INFO:tasks.workunit.client.1.smithi192.stdout:7/439: creat d5/d43/d4c/d4e/f7f x:0 0 0 2024-04-23T02:13:58.503 INFO:tasks.workunit.client.0.smithi071.stdout:8/436: link l3 dc/d10/d32/l89 0 2024-04-23T02:13:58.503 INFO:tasks.workunit.client.0.smithi071.stdout:3/339: write d7/fd [7797476,6507] 0 2024-04-23T02:13:58.503 INFO:tasks.workunit.client.1.smithi192.stdout:7/440: readlink d5/le 0 2024-04-23T02:13:58.504 INFO:tasks.workunit.client.0.smithi071.stdout:0/343: creat d0/f6b x:0 0 0 2024-04-23T02:13:58.506 INFO:tasks.workunit.client.1.smithi192.stdout:7/441: symlink d5/d43/d4c/d4e/l80 0 2024-04-23T02:13:58.508 INFO:tasks.workunit.client.0.smithi071.stdout:7/339: dread d3/d17/d41/d34/d39/d3b/f46 [0,4194304] 0 2024-04-23T02:13:58.510 INFO:tasks.workunit.client.1.smithi192.stdout:7/442: creat d5/d43/d4c/d5c/d67/f81 x:0 0 0 2024-04-23T02:13:58.510 INFO:tasks.workunit.client.0.smithi071.stdout:7/340: mkdir d3/d74 0 2024-04-23T02:13:58.511 INFO:tasks.workunit.client.1.smithi192.stdout:8/617: dwrite d1/d12/d23/d4b/d73/d4c/f84 [0,4194304] 0 2024-04-23T02:13:58.512 INFO:tasks.workunit.client.1.smithi192.stdout:8/618: stat d1/d12/d23/d4b/d73/d7e/d88/da9/lbc 0 2024-04-23T02:13:58.513 INFO:tasks.workunit.client.1.smithi192.stdout:8/619: dread - d1/d9/d8f/d90/fa2 zero size 2024-04-23T02:13:58.514 INFO:tasks.workunit.client.0.smithi071.stdout:7/341: link d3/d17/d1c/f26 d3/d17/d41/d37/f75 0 2024-04-23T02:13:58.514 INFO:tasks.workunit.client.1.smithi192.stdout:7/443: fsync d5/d17/d1c/d57/f46 0 2024-04-23T02:13:58.515 INFO:tasks.workunit.client.1.smithi192.stdout:7/444: dread - d5/d43/d4c/d5c/f72 zero size 2024-04-23T02:13:58.517 INFO:tasks.workunit.client.0.smithi071.stdout:7/342: rename d3/d17/d41/d37/d4f/l68 to d3/d17/d41/d34/d59/l76 0 2024-04-23T02:13:58.518 INFO:tasks.workunit.client.1.smithi192.stdout:7/445: creat d5/d43/d4c/d5c/d67/f82 x:0 0 0 2024-04-23T02:13:58.519 INFO:tasks.workunit.client.0.smithi071.stdout:7/343: chown d3/d17/d41/d37/d4f/c5b 2560 1 2024-04-23T02:13:58.521 INFO:tasks.workunit.client.1.smithi192.stdout:7/446: creat d5/d17/d1c/d59/d5a/d41/d50/d3d/f83 x:0 0 0 2024-04-23T02:13:58.521 INFO:tasks.workunit.client.0.smithi071.stdout:7/344: mknod d3/d17/d41/d34/d39/c77 0 2024-04-23T02:13:58.522 INFO:tasks.workunit.client.1.smithi192.stdout:7/447: dread - d5/d43/d4c/d5c/d67/f81 zero size 2024-04-23T02:13:58.524 INFO:tasks.workunit.client.0.smithi071.stdout:7/345: creat d3/d17/d1c/d2a/f78 x:0 0 0 2024-04-23T02:13:58.525 INFO:tasks.workunit.client.1.smithi192.stdout:7/448: unlink d5/d43/d4c/d4e/l80 0 2024-04-23T02:13:58.525 INFO:tasks.workunit.client.0.smithi071.stdout:0/344: chown d0/d3/de/d10/d14/d32/d44 25812264 1 2024-04-23T02:13:58.526 INFO:tasks.workunit.client.1.smithi192.stdout:7/449: readlink d5/d17/d1c/d59/d5a/l5e 0 2024-04-23T02:13:58.526 INFO:tasks.workunit.client.0.smithi071.stdout:7/346: readlink d3/d17/d41/l71 0 2024-04-23T02:13:58.528 INFO:tasks.workunit.client.1.smithi192.stdout:7/450: creat d5/d17/d1c/d4a/f84 x:0 0 0 2024-04-23T02:13:58.529 INFO:tasks.workunit.client.1.smithi192.stdout:7/451: chown d5/d43/d4c/d4e/l5b 0 1 2024-04-23T02:13:58.529 INFO:tasks.workunit.client.1.smithi192.stdout:8/620: dwrite d1/d12/f39 [0,4194304] 0 2024-04-23T02:13:58.530 INFO:tasks.workunit.client.0.smithi071.stdout:0/345: unlink d0/d3/de/d10/d14/d32/d37/d26/f60 0 2024-04-23T02:13:58.530 INFO:tasks.workunit.client.0.smithi071.stdout:0/346: chown d0/d3/de/d10/d14/f1e 10 1 2024-04-23T02:13:58.530 INFO:tasks.workunit.client.1.smithi192.stdout:8/621: dread - d1/d9/d8f/d90/fa2 zero size 2024-04-23T02:13:58.531 INFO:tasks.workunit.client.0.smithi071.stdout:0/347: truncate d0/d3/de/f6a 468659 0 2024-04-23T02:13:58.532 INFO:tasks.workunit.client.1.smithi192.stdout:6/515: sync 2024-04-23T02:13:58.533 INFO:tasks.workunit.client.0.smithi071.stdout:7/347: creat d3/d17/d41/d34/d6e/f79 x:0 0 0 2024-04-23T02:13:58.533 INFO:tasks.workunit.client.1.smithi192.stdout:7/452: rename l2 to d5/d43/l85 0 2024-04-23T02:13:58.537 INFO:tasks.workunit.client.1.smithi192.stdout:6/516: creat d3/d15/d1b/d21/d52/d64/fa7 x:0 0 0 2024-04-23T02:13:58.540 INFO:tasks.workunit.client.1.smithi192.stdout:6/517: read d3/f2b [3029667,38612] 0 2024-04-23T02:13:58.542 INFO:tasks.workunit.client.1.smithi192.stdout:7/453: dwrite d5/d17/d1c/d59/d5a/d70/f7e [0,4194304] 0 2024-04-23T02:13:58.544 INFO:tasks.workunit.client.1.smithi192.stdout:6/518: write d3/d15/d1b/d21/d52/f92 [240166,11749] 0 2024-04-23T02:13:58.544 INFO:tasks.workunit.client.0.smithi071.stdout:7/348: write d3/fc [1297054,117603] 0 2024-04-23T02:13:58.546 INFO:tasks.workunit.client.0.smithi071.stdout:0/348: write d0/d3/de/d10/d14/d32/f41 [648588,118715] 0 2024-04-23T02:13:58.550 INFO:tasks.workunit.client.0.smithi071.stdout:0/349: symlink d0/d3/de/d10/d14/d32/d44/d31/l6c 0 2024-04-23T02:13:58.552 INFO:tasks.workunit.client.1.smithi192.stdout:7/454: getdents d5/d43/d4c/d4e 0 2024-04-23T02:13:58.555 INFO:tasks.workunit.client.0.smithi071.stdout:7/349: dwrite d3/d17/d1c/d31/f5d [0,4194304] 0 2024-04-23T02:13:58.555 INFO:tasks.workunit.client.1.smithi192.stdout:7/455: unlink d5/f2e 0 2024-04-23T02:13:58.558 INFO:tasks.workunit.client.1.smithi192.stdout:6/519: truncate d3/d28/f96 152868 0 2024-04-23T02:13:58.559 INFO:tasks.workunit.client.0.smithi071.stdout:0/350: rename d0/d3/l30 to d0/d3/de/d48/l6d 0 2024-04-23T02:13:58.560 INFO:tasks.workunit.client.0.smithi071.stdout:7/350: fsync d3/ff 0 2024-04-23T02:13:58.562 INFO:tasks.workunit.client.0.smithi071.stdout:0/351: symlink d0/d3/de/d10/d14/d32/l6e 0 2024-04-23T02:13:58.563 INFO:tasks.workunit.client.1.smithi192.stdout:6/520: symlink d3/d15/d1b/la8 0 2024-04-23T02:13:58.564 INFO:tasks.workunit.client.0.smithi071.stdout:7/351: unlink d3/c7 0 2024-04-23T02:13:58.565 INFO:tasks.workunit.client.0.smithi071.stdout:0/352: write d0/d3/de/d10/d14/f5b [1025119,117767] 0 2024-04-23T02:13:58.566 INFO:tasks.workunit.client.1.smithi192.stdout:6/521: unlink d3/cb 0 2024-04-23T02:13:58.567 INFO:tasks.workunit.client.1.smithi192.stdout:6/522: dread - d3/d18/f9f zero size 2024-04-23T02:13:58.567 INFO:tasks.workunit.client.0.smithi071.stdout:0/353: dread - d0/d3/de/d10/f57 zero size 2024-04-23T02:13:58.570 INFO:tasks.workunit.client.1.smithi192.stdout:6/523: creat d3/d11/fa9 x:0 0 0 2024-04-23T02:13:58.570 INFO:tasks.workunit.client.0.smithi071.stdout:7/352: rename d3/d17/l21 to d3/d17/d41/l7a 0 2024-04-23T02:13:58.573 INFO:tasks.workunit.client.0.smithi071.stdout:0/354: rename d0/d3/de/d10/l29 to d0/d3/de/d10/d14/d32/d44/d31/l6f 0 2024-04-23T02:13:58.574 INFO:tasks.workunit.client.1.smithi192.stdout:6/524: link d3/d15/d1b/d21/f60 d3/d15/d1b/da2/faa 0 2024-04-23T02:13:58.575 INFO:tasks.workunit.client.1.smithi192.stdout:6/525: mkdir d3/d15/d1b/d21/d52/d64/d8d/dab 0 2024-04-23T02:13:58.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.572+0000 7f1d002c1700 1 -- 172.21.15.71:0/3673263290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cf8100e10 msgr2=0x7f1cf8101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:58.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.572+0000 7f1d002c1700 1 --2- 172.21.15.71:0/3673263290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cf8100e10 0x7f1cf8101270 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009910 tx=0x7f1cf4009c20 comp rx=0 tx=0).stop 2024-04-23T02:13:58.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.572+0000 7f1d002c1700 1 -- 172.21.15.71:0/3673263290 shutdown_connections 2024-04-23T02:13:58.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.572+0000 7f1d002c1700 1 --2- 172.21.15.71:0/3673263290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cf8100e10 0x7f1cf8101270 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:58.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.572+0000 7f1d002c1700 1 --2- 172.21.15.71:0/3673263290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cf80ffc10 0x7f1cf8100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:58.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.572+0000 7f1d002c1700 1 -- 172.21.15.71:0/3673263290 >> 172.21.15.71:0/3673263290 conn(0x7f1cf80fb190 msgr2=0x7f1cf80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:13:58.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.572+0000 7f1d002c1700 1 -- 172.21.15.71:0/3673263290 shutdown_connections 2024-04-23T02:13:58.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.572+0000 7f1d002c1700 1 -- 172.21.15.71:0/3673263290 wait complete. 2024-04-23T02:13:58.577 INFO:tasks.workunit.client.1.smithi192.stdout:6/526: write d3/d15/d1b/d66/d80/d8e/f8c [1027003,7884] 0 2024-04-23T02:13:58.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.573+0000 7f1d002c1700 1 Processor -- start 2024-04-23T02:13:58.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.573+0000 7f1d002c1700 1 -- start start 2024-04-23T02:13:58.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.573+0000 7f1d002c1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cf80ffc10 0x7f1cf819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:58.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.573+0000 7f1d002c1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cf8100e10 0x7f1cf819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:58.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.573+0000 7f1d002c1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf819d520 con 0x7f1cf8100e10 2024-04-23T02:13:58.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.573+0000 7f1d002c1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf819d660 con 0x7f1cf80ffc10 2024-04-23T02:13:58.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.573+0000 7f1cfd85c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cf8100e10 0x7f1cf819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:58.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.573+0000 7f1cfd85c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cf8100e10 0x7f1cf819cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50170/0 (socket says 172.21.15.71:50170) 2024-04-23T02:13:58.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.574+0000 7f1cfd85c700 1 -- 172.21.15.71:0/888108925 learned_addr learned my addr 172.21.15.71:0/888108925 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:13:58.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.574+0000 7f1cfe05d700 1 --2- 172.21.15.71:0/888108925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cf80ffc10 0x7f1cf819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:58.579 INFO:tasks.workunit.client.0.smithi071.stdout:7/353: dwrite d3/fc [0,4194304] 0 2024-04-23T02:13:58.579 INFO:tasks.workunit.client.1.smithi192.stdout:7/456: dread d5/d17/d1c/f2d [0,4194304] 0 2024-04-23T02:13:58.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.574+0000 7f1cfd85c700 1 -- 172.21.15.71:0/888108925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cf80ffc10 msgr2=0x7f1cf819c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:58.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.574+0000 7f1cfd85c700 1 --2- 172.21.15.71:0/888108925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cf80ffc10 0x7f1cf819c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:58.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.574+0000 7f1cfd85c700 1 -- 172.21.15.71:0/888108925 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf40095f0 con 0x7f1cf8100e10 2024-04-23T02:13:58.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.574+0000 7f1cfd85c700 1 --2- 172.21.15.71:0/888108925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cf8100e10 0x7f1cf819cf00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400bde0 tx=0x7f1cf400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:13:58.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.574+0000 7f1cef7fe700 1 -- 172.21.15.71:0/888108925 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cf401d070 con 0x7f1cf8100e10 2024-04-23T02:13:58.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.574+0000 7f1d002c1700 1 -- 172.21.15.71:0/888108925 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf81a20b0 con 0x7f1cf8100e10 2024-04-23T02:13:58.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.574+0000 7f1cef7fe700 1 -- 172.21.15.71:0/888108925 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1cf4005710 con 0x7f1cf8100e10 2024-04-23T02:13:58.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.574+0000 7f1d002c1700 1 -- 172.21.15.71:0/888108925 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf81a2540 con 0x7f1cf8100e10 2024-04-23T02:13:58.580 INFO:tasks.workunit.client.0.smithi071.stdout:0/355: truncate d0/d3/de/d10/d14/d32/f35 1237184 0 2024-04-23T02:13:58.581 INFO:tasks.workunit.client.1.smithi192.stdout:7/457: fsync d5/d17/d1c/d59/d5a/d70/f7e 0 2024-04-23T02:13:58.582 INFO:tasks.workunit.client.1.smithi192.stdout:7/458: dread - d5/d17/d1c/d59/d5a/d41/d50/f7b zero size 2024-04-23T02:13:58.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.578+0000 7f1cef7fe700 1 -- 172.21.15.71:0/888108925 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cf4003a90 con 0x7f1cf8100e10 2024-04-23T02:13:58.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.578+0000 7f1cef7fe700 1 -- 172.21.15.71:0/888108925 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) v1 ==== 90681+0+0 (secure 0 0 0) 0x7f1cf4003cb0 con 0x7f1cf8100e10 2024-04-23T02:13:58.586 INFO:tasks.workunit.client.0.smithi071.stdout:7/354: link d3/d17/d41/d34/d39/d3b/l70 d3/d17/d1c/d31/l7b 0 2024-04-23T02:13:58.590 INFO:tasks.workunit.client.0.smithi071.stdout:0/356: dwrite d0/d3/f2d [0,4194304] 0 2024-04-23T02:13:58.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.579+0000 7f1cef7fe700 1 --2- 172.21.15.71:0/888108925 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f1ce406ca60 0x7f1ce406ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:58.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.579+0000 7f1cef7fe700 1 -- 172.21.15.71:0/888108925 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f1cf405c5d0 con 0x7f1cf8100e10 2024-04-23T02:13:58.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.580+0000 7f1cfe05d700 1 --2- 172.21.15.71:0/888108925 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f1ce406ca60 0x7f1ce406ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:58.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.580+0000 7f1d002c1700 1 -- 172.21.15.71:0/888108925 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cdc005320 con 0x7f1cf8100e10 2024-04-23T02:13:58.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.580+0000 7f1cfe05d700 1 --2- 172.21.15.71:0/888108925 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f1ce406ca60 0x7f1ce406ef20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8100c70 tx=0x7f1ce8009260 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:13:58.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.587+0000 7f1cef7fe700 1 -- 172.21.15.71:0/888108925 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f1cf4093060 con 0x7f1cf8100e10 2024-04-23T02:13:58.604 INFO:tasks.workunit.client.0.smithi071.stdout:0/357: symlink d0/d3/de/d10/d14/d32/d44/d59/l70 0 2024-04-23T02:13:58.610 INFO:tasks.workunit.client.0.smithi071.stdout:7/355: dwrite d3/d17/d41/d34/d39/d3b/f46 [0,4194304] 0 2024-04-23T02:13:58.612 INFO:tasks.workunit.client.0.smithi071.stdout:0/358: dread d0/d3/de/d10/d14/d32/d37/f53 [0,4194304] 0 2024-04-23T02:13:58.615 INFO:tasks.workunit.client.0.smithi071.stdout:7/356: rmdir d3/d17/d41/d34/d59 39 2024-04-23T02:13:58.617 INFO:tasks.workunit.client.0.smithi071.stdout:0/359: write d0/d3/de/d10/d14/f64 [27818,98327] 0 2024-04-23T02:13:58.618 INFO:tasks.workunit.client.1.smithi192.stdout:6/527: dread d3/f2b [0,4194304] 0 2024-04-23T02:13:58.622 INFO:tasks.workunit.client.0.smithi071.stdout:7/357: dwrite d3/d17/d1c/f3f [0,4194304] 0 2024-04-23T02:13:58.623 INFO:tasks.workunit.client.0.smithi071.stdout:0/360: unlink d0/d3/de/d10/d14/d32/d44/c5d 0 2024-04-23T02:13:58.624 INFO:tasks.workunit.client.0.smithi071.stdout:7/358: write d3/d17/d41/d34/d39/d3b/f53 [97479,6751] 0 2024-04-23T02:13:58.625 INFO:tasks.workunit.client.1.smithi192.stdout:6/528: link d3/d15/f4b d3/d15/d1b/d21/d3a/fac 0 2024-04-23T02:13:58.626 INFO:tasks.workunit.client.1.smithi192.stdout:6/529: chown d3/d15/d1b/d21/d3a/f4e 716215 1 2024-04-23T02:13:58.626 INFO:tasks.workunit.client.0.smithi071.stdout:0/361: symlink d0/d3/de/d10/d14/d32/d44/d31/l71 0 2024-04-23T02:13:58.634 INFO:tasks.workunit.client.0.smithi071.stdout:7/359: link d3/l10 d3/d17/d41/d34/l7c 0 2024-04-23T02:13:58.642 INFO:tasks.workunit.client.0.smithi071.stdout:0/362: truncate d0/d3/f2d 1390790 0 2024-04-23T02:13:58.643 INFO:tasks.workunit.client.0.smithi071.stdout:7/360: truncate d3/d17/d1c/f29 825834 0 2024-04-23T02:13:58.645 INFO:tasks.workunit.client.0.smithi071.stdout:0/363: unlink d0/d3/de/d10/d16/f3e 0 2024-04-23T02:13:58.646 INFO:tasks.workunit.client.1.smithi192.stdout:6/530: dread d3/d15/d1b/d66/d80/d8e/f59 [0,4194304] 0 2024-04-23T02:13:58.647 INFO:tasks.workunit.client.0.smithi071.stdout:7/361: dread d3/d17/d41/d37/f38 [0,4194304] 0 2024-04-23T02:13:58.647 INFO:tasks.workunit.client.1.smithi192.stdout:6/531: dread - d3/d15/d1b/d66/d80/f8b zero size 2024-04-23T02:13:58.648 INFO:tasks.workunit.client.0.smithi071.stdout:7/362: fdatasync d3/d17/d1c/d2a/f78 0 2024-04-23T02:13:58.649 INFO:tasks.workunit.client.0.smithi071.stdout:0/364: rename d0/d3/de/d10/d14/f51 to d0/d3/de/d10/d14/d32/d44/d31/f72 0 2024-04-23T02:13:58.653 INFO:tasks.workunit.client.1.smithi192.stdout:6/532: dwrite d3/d15/d1b/d21/d3a/f4e [0,4194304] 0 2024-04-23T02:13:58.653 INFO:tasks.workunit.client.0.smithi071.stdout:7/363: link d3/d17/d1c/l3e d3/d17/d41/l7d 0 2024-04-23T02:13:58.655 INFO:tasks.workunit.client.0.smithi071.stdout:7/364: read - d3/d17/d41/d37/f6f zero size 2024-04-23T02:13:58.655 INFO:tasks.workunit.client.0.smithi071.stdout:0/365: dread d0/d3/de/f28 [0,4194304] 0 2024-04-23T02:13:58.657 INFO:tasks.workunit.client.0.smithi071.stdout:0/366: chown d0/d3/de/d10/d14/l58 512791567 1 2024-04-23T02:13:58.659 INFO:tasks.workunit.client.1.smithi192.stdout:6/533: symlink d3/d15/d5b/lad 0 2024-04-23T02:13:58.661 INFO:tasks.workunit.client.1.smithi192.stdout:6/534: creat d3/d28/d79/da0/fae x:0 0 0 2024-04-23T02:13:58.666 INFO:tasks.workunit.client.0.smithi071.stdout:7/365: dwrite d3/d17/d1c/d31/f6a [0,4194304] 0 2024-04-23T02:13:58.668 INFO:tasks.workunit.client.1.smithi192.stdout:6/535: dwrite d3/d15/d1b/f8f [0,4194304] 0 2024-04-23T02:13:58.670 INFO:tasks.workunit.client.0.smithi071.stdout:7/366: read d3/d17/d41/d34/d59/f67 [1270677,566] 0 2024-04-23T02:13:58.675 INFO:tasks.workunit.client.1.smithi192.stdout:6/536: dwrite d3/d15/d1b/f8f [0,4194304] 0 2024-04-23T02:13:58.676 INFO:tasks.workunit.client.0.smithi071.stdout:0/367: write d0/d3/de/d10/d14/d32/d44/d31/f72 [575930,49605] 0 2024-04-23T02:13:58.679 INFO:tasks.workunit.client.0.smithi071.stdout:7/367: creat d3/d17/d41/d34/d59/d5f/f7e x:0 0 0 2024-04-23T02:13:58.681 INFO:tasks.workunit.client.0.smithi071.stdout:0/368: unlink d0/d3/de/d10/d14/d32/d44/d59/l70 0 2024-04-23T02:13:58.682 INFO:tasks.workunit.client.0.smithi071.stdout:7/368: creat d3/d51/f7f x:0 0 0 2024-04-23T02:13:58.684 INFO:tasks.workunit.client.0.smithi071.stdout:7/369: readlink d3/d17/d41/l71 0 2024-04-23T02:13:58.686 INFO:tasks.workunit.client.0.smithi071.stdout:7/370: mkdir d3/d51/d80 0 2024-04-23T02:13:58.688 INFO:tasks.workunit.client.0.smithi071.stdout:7/371: creat d3/d17/d1c/d31/f81 x:0 0 0 2024-04-23T02:13:58.690 INFO:tasks.workunit.client.1.smithi192.stdout:3/381: sync 2024-04-23T02:13:58.690 INFO:tasks.workunit.client.1.smithi192.stdout:0/517: sync 2024-04-23T02:13:58.690 INFO:tasks.workunit.client.1.smithi192.stdout:4/616: sync 2024-04-23T02:13:58.691 INFO:tasks.workunit.client.0.smithi071.stdout:0/369: write d0/d3/f3d [634711,25314] 0 2024-04-23T02:13:58.691 INFO:tasks.workunit.client.1.smithi192.stdout:3/382: readlink d1/d3/d17/d65/l7e 0 2024-04-23T02:13:58.693 INFO:tasks.workunit.client.0.smithi071.stdout:0/370: dread d0/d3/de/f6a [0,4194304] 0 2024-04-23T02:13:58.693 INFO:tasks.workunit.client.0.smithi071.stdout:7/372: dread d3/d17/d41/f33 [0,4194304] 0 2024-04-23T02:13:58.694 INFO:tasks.workunit.client.1.smithi192.stdout:4/617: stat de/d15/d53/d81/c98 0 2024-04-23T02:13:58.694 INFO:tasks.workunit.client.0.smithi071.stdout:0/371: write d0/d3/d1c/f21 [3859150,72597] 0 2024-04-23T02:13:58.695 INFO:tasks.workunit.client.1.smithi192.stdout:0/518: mkdir d6/df/d13/d6b/d73/da9 0 2024-04-23T02:13:58.696 INFO:tasks.workunit.client.0.smithi071.stdout:7/373: symlink d3/d17/d41/d34/d59/l82 0 2024-04-23T02:13:58.698 INFO:tasks.workunit.client.1.smithi192.stdout:4/618: symlink de/d15/d2a/d74/d92/da0/dca/d9f/ld1 0 2024-04-23T02:13:58.699 INFO:tasks.workunit.client.1.smithi192.stdout:3/383: dread d1/d3/f19 [0,4194304] 0 2024-04-23T02:13:58.699 INFO:tasks.workunit.client.0.smithi071.stdout:0/372: getdents d0/d63 0 2024-04-23T02:13:58.699 INFO:tasks.workunit.client.1.smithi192.stdout:0/519: symlink d6/df/d13/d6b/d73/da9/laa 0 2024-04-23T02:13:58.700 INFO:tasks.workunit.client.0.smithi071.stdout:7/374: getdents d3/d17/d1c/d2a/d43 0 2024-04-23T02:13:58.701 INFO:tasks.workunit.client.1.smithi192.stdout:4/619: mknod de/d15/d53/d81/dac/cd2 0 2024-04-23T02:13:58.701 INFO:tasks.workunit.client.0.smithi071.stdout:7/375: fsync d3/d17/d1c/f45 0 2024-04-23T02:13:58.702 INFO:tasks.workunit.client.1.smithi192.stdout:0/520: creat d6/df/d13/d6b/fab x:0 0 0 2024-04-23T02:13:58.703 INFO:tasks.workunit.client.1.smithi192.stdout:0/521: write d6/df/d13/d6b/f6e [722577,59590] 0 2024-04-23T02:13:58.703 INFO:tasks.workunit.client.0.smithi071.stdout:7/376: read d3/d17/d41/d34/f3c [597390,103198] 0 2024-04-23T02:13:58.705 INFO:tasks.workunit.client.0.smithi071.stdout:0/373: dread d0/d3/de/d10/d14/d32/d44/d31/d52/f5c [0,4194304] 0 2024-04-23T02:13:58.705 INFO:tasks.workunit.client.1.smithi192.stdout:4/620: rename de/d15/d19/d1a/f3a to de/d15/d2a/d74/d92/da0/dca/d49/d26/d59/fd3 0 2024-04-23T02:13:58.706 INFO:tasks.workunit.client.0.smithi071.stdout:7/377: mkdir d3/d17/d1c/d2a/d73/d83 0 2024-04-23T02:13:58.709 INFO:tasks.workunit.client.0.smithi071.stdout:7/378: chown d3/fd 34795582 1 2024-04-23T02:13:58.712 INFO:tasks.workunit.client.0.smithi071.stdout:7/379: dread d3/d17/d41/d34/d39/d3b/f53 [0,4194304] 0 2024-04-23T02:13:58.713 INFO:tasks.workunit.client.0.smithi071.stdout:0/374: dwrite d0/d3/de/ff [4194304,4194304] 0 2024-04-23T02:13:58.719 INFO:tasks.workunit.client.0.smithi071.stdout:0/375: rename d0/d3/de/l25 to d0/d3/de/d10/d14/d32/l73 0 2024-04-23T02:13:58.721 INFO:tasks.workunit.client.0.smithi071.stdout:7/380: dwrite d3/d17/d41/f2f [0,4194304] 0 2024-04-23T02:13:58.723 INFO:tasks.workunit.client.0.smithi071.stdout:7/381: write d3/d17/d41/d34/d59/d5f/f7e [65122,24098] 0 2024-04-23T02:13:58.724 INFO:tasks.workunit.client.0.smithi071.stdout:7/382: chown d3/ff 0 1 2024-04-23T02:13:58.728 INFO:tasks.workunit.client.0.smithi071.stdout:7/383: creat d3/d17/d41/d34/d39/d3b/f84 x:0 0 0 2024-04-23T02:13:58.729 INFO:tasks.workunit.client.0.smithi071.stdout:7/384: readlink d3/d17/d41/d34/l4d 0 2024-04-23T02:13:58.733 INFO:tasks.workunit.client.0.smithi071.stdout:7/385: fdatasync d3/d17/d1c/d2a/f4c 0 2024-04-23T02:13:58.733 INFO:tasks.workunit.client.0.smithi071.stdout:0/376: fsync d0/d3/de/d10/d14/d32/d44/f43 0 2024-04-23T02:13:58.733 INFO:tasks.workunit.client.0.smithi071.stdout:7/386: write d3/d17/d41/d34/d39/d3b/f46 [4312939,105075] 0 2024-04-23T02:13:58.735 INFO:tasks.workunit.client.0.smithi071.stdout:0/377: chown d0/l69 515086 1 2024-04-23T02:13:58.736 INFO:tasks.workunit.client.0.smithi071.stdout:7/387: symlink d3/d17/d41/d3a/l85 0 2024-04-23T02:13:58.738 INFO:tasks.workunit.client.0.smithi071.stdout:0/378: rename d0/d3/de/f6a to d0/d63/f74 0 2024-04-23T02:13:58.746 INFO:tasks.workunit.client.0.smithi071.stdout:0/379: unlink d0/d3/de/d10/d14/d32/d37/c47 0 2024-04-23T02:13:58.747 INFO:tasks.workunit.client.0.smithi071.stdout:0/380: write d0/d3/de/d10/d14/d32/f41 [264382,1853] 0 2024-04-23T02:13:58.750 INFO:tasks.workunit.client.0.smithi071.stdout:0/381: mknod d0/d3/de/d48/c75 0 2024-04-23T02:13:58.756 INFO:tasks.workunit.client.0.smithi071.stdout:0/382: dwrite d0/d3/de/d10/d14/d32/d37/f53 [0,4194304] 0 2024-04-23T02:13:58.757 INFO:tasks.workunit.client.0.smithi071.stdout:0/383: read d0/f20 [2225686,29894] 0 2024-04-23T02:13:58.758 INFO:tasks.workunit.client.0.smithi071.stdout:0/384: chown d0/d3/de/d10/d16/c33 21870 1 2024-04-23T02:13:58.761 INFO:tasks.workunit.client.0.smithi071.stdout:0/385: dread d0/d3/de/d10/d14/f64 [0,4194304] 0 2024-04-23T02:13:58.763 INFO:tasks.workunit.client.0.smithi071.stdout:0/386: dread d0/d3/de/d10/d14/f64 [0,4194304] 0 2024-04-23T02:13:58.763 INFO:tasks.workunit.client.0.smithi071.stdout:0/387: truncate d0/f6b 665885 0 2024-04-23T02:13:58.767 INFO:tasks.workunit.client.0.smithi071.stdout:0/388: rmdir d0/d3/de/d10/d14/d32/d37 39 2024-04-23T02:13:58.770 INFO:tasks.workunit.client.0.smithi071.stdout:0/389: mkdir d0/d76 0 2024-04-23T02:13:58.771 INFO:tasks.workunit.client.0.smithi071.stdout:0/390: creat d0/d3/de/d10/d16/f77 x:0 0 0 2024-04-23T02:13:58.776 INFO:tasks.workunit.client.0.smithi071.stdout:0/391: rename d0/d3/de/d10/d14/d32/l73 to d0/d3/de/l78 0 2024-04-23T02:13:58.778 INFO:tasks.workunit.client.0.smithi071.stdout:0/392: write d0/d3/de/d10/f1f [6269187,105543] 0 2024-04-23T02:13:58.779 INFO:tasks.workunit.client.1.smithi192.stdout:0/522: link d6/df/d35/d49/f9f d6/df/d13/d25/d26/fac 0 2024-04-23T02:13:58.782 INFO:tasks.workunit.client.1.smithi192.stdout:0/523: rename d6/f72 to d6/df/d56/d85/d45/fad 0 2024-04-23T02:13:58.783 INFO:tasks.workunit.client.1.smithi192.stdout:4/621: symlink de/d15/d2a/d74/d92/da0/dca/d49/d70/d73/ld4 0 2024-04-23T02:13:58.785 INFO:tasks.workunit.client.0.smithi071.stdout:0/393: dwrite d0/d3/de/d10/d16/f55 [4194304,4194304] 0 2024-04-23T02:13:58.787 INFO:tasks.workunit.client.1.smithi192.stdout:4/622: truncate de/f21 5645 0 2024-04-23T02:13:58.790 INFO:tasks.workunit.client.0.smithi071.stdout:0/394: dread d0/d3/de/d10/d14/d32/d44/d31/d52/f5c [0,4194304] 0 2024-04-23T02:13:58.791 INFO:tasks.workunit.client.0.smithi071.stdout:0/395: chown d0/d3/de/d10/d14/f4d 13999 1 2024-04-23T02:13:58.793 INFO:tasks.workunit.client.0.smithi071.stdout:0/396: read d0/d3/de/d10/d14/d32/d44/d31/f72 [600244,36802] 0 2024-04-23T02:13:58.796 INFO:tasks.workunit.client.0.smithi071.stdout:0/397: write d0/d3/de/f1d [1059234,108119] 0 2024-04-23T02:13:58.798 INFO:tasks.workunit.client.0.smithi071.stdout:0/398: write d0/d3/de/d10/d16/f55 [9373438,8482] 0 2024-04-23T02:13:58.804 INFO:tasks.workunit.client.0.smithi071.stdout:0/399: truncate d0/d3/de/f28 2801210 0 2024-04-23T02:13:58.804 INFO:tasks.workunit.client.0.smithi071.stdout:0/400: fsync d0/f6b 0 2024-04-23T02:13:58.807 INFO:tasks.workunit.client.0.smithi071.stdout:0/401: rmdir d0/d3/de/d10/d14/d32/d37/d26 39 2024-04-23T02:13:58.809 INFO:tasks.workunit.client.0.smithi071.stdout:0/402: dread d0/d63/f74 [0,4194304] 0 2024-04-23T02:13:58.811 INFO:tasks.workunit.client.0.smithi071.stdout:0/403: symlink d0/d3/d1c/l79 0 2024-04-23T02:13:58.817 INFO:tasks.workunit.client.0.smithi071.stdout:0/404: dwrite d0/d3/de/d10/d16/f77 [0,4194304] 0 2024-04-23T02:13:58.821 INFO:tasks.workunit.client.0.smithi071.stdout:0/405: stat d0/d3/de/l78 0 2024-04-23T02:13:58.822 INFO:tasks.workunit.client.0.smithi071.stdout:0/406: read d0/d3/de/d10/d16/f5e [1047770,61242] 0 2024-04-23T02:13:58.824 INFO:tasks.workunit.client.0.smithi071.stdout:0/407: truncate d0/d3/de/d10/f57 387027 0 2024-04-23T02:13:58.826 INFO:tasks.workunit.client.0.smithi071.stdout:0/408: creat d0/d3/f7a x:0 0 0 2024-04-23T02:13:58.831 INFO:tasks.workunit.client.0.smithi071.stdout:0/409: dread d0/d3/de/d10/d14/f5b [0,4194304] 0 2024-04-23T02:13:58.836 INFO:tasks.workunit.client.0.smithi071.stdout:6/308: sync 2024-04-23T02:13:58.839 INFO:tasks.workunit.client.0.smithi071.stdout:6/309: mknod d6/d17/c60 0 2024-04-23T02:13:58.841 INFO:tasks.workunit.client.0.smithi071.stdout:0/410: dwrite d0/d63/f74 [0,4194304] 0 2024-04-23T02:13:58.843 INFO:tasks.workunit.client.0.smithi071.stdout:6/310: mknod d6/d1f/d46/c61 0 2024-04-23T02:13:58.844 INFO:tasks.workunit.client.0.smithi071.stdout:6/311: stat d6/d22/c2e 0 2024-04-23T02:13:58.848 INFO:tasks.workunit.client.0.smithi071.stdout:0/411: creat d0/d3/de/d10/d14/d32/d37/f7b x:0 0 0 2024-04-23T02:13:58.854 INFO:tasks.workunit.client.0.smithi071.stdout:0/412: rename d0/l34 to d0/l7c 0 2024-04-23T02:13:58.856 INFO:tasks.workunit.client.0.smithi071.stdout:0/413: symlink d0/d3/de/d10/d14/l7d 0 2024-04-23T02:13:58.859 INFO:tasks.workunit.client.0.smithi071.stdout:0/414: mkdir d0/d3/de/d48/d7e 0 2024-04-23T02:13:58.859 INFO:tasks.workunit.client.1.smithi192.stdout:0/524: mkdir d6/df/d47/d55/dae 0 2024-04-23T02:13:58.859 INFO:tasks.workunit.client.1.smithi192.stdout:1/260: sync 2024-04-23T02:13:58.859 INFO:tasks.workunit.client.1.smithi192.stdout:5/456: sync 2024-04-23T02:13:58.860 INFO:tasks.workunit.client.0.smithi071.stdout:0/415: write d0/f2 [8969310,8795] 0 2024-04-23T02:13:58.860 INFO:tasks.workunit.client.1.smithi192.stdout:5/457: stat d2/db/d16/d48/d4f/f5c 0 2024-04-23T02:13:58.862 INFO:tasks.workunit.client.0.smithi071.stdout:0/416: mknod d0/d3/de/d48/c7f 0 2024-04-23T02:13:58.863 INFO:tasks.workunit.client.1.smithi192.stdout:5/458: creat d2/db/d2b/d52/f99 x:0 0 0 2024-04-23T02:13:58.865 INFO:tasks.workunit.client.1.smithi192.stdout:0/525: link d6/df/d47/f89 d6/df/d13/d25/d86/da5/faf 0 2024-04-23T02:13:58.867 INFO:tasks.workunit.client.1.smithi192.stdout:5/459: fdatasync d2/db/d2b/f93 0 2024-04-23T02:13:58.867 INFO:tasks.workunit.client.0.smithi071.stdout:6/312: dread d6/f8 [0,4194304] 0 2024-04-23T02:13:58.867 INFO:tasks.workunit.client.1.smithi192.stdout:1/261: dwrite dc/d14/d23/f3d [0,4194304] 0 2024-04-23T02:13:58.869 INFO:tasks.workunit.client.0.smithi071.stdout:6/313: creat d6/d1f/d46/f62 x:0 0 0 2024-04-23T02:13:58.870 INFO:tasks.workunit.client.0.smithi071.stdout:6/314: rmdir d6/d1f 39 2024-04-23T02:13:58.872 INFO:tasks.workunit.client.0.smithi071.stdout:6/315: symlink d6/d17/d4d/l63 0 2024-04-23T02:13:58.873 INFO:tasks.workunit.client.0.smithi071.stdout:6/316: truncate d6/d3e/f5b 1117398 0 2024-04-23T02:13:58.877 INFO:tasks.workunit.client.1.smithi192.stdout:5/460: creat d2/db/f9a x:0 0 0 2024-04-23T02:13:58.877 INFO:tasks.workunit.client.0.smithi071.stdout:6/317: rename d6/d1f/d46/c61 to d6/d3e/c64 0 2024-04-23T02:13:58.878 INFO:tasks.workunit.client.1.smithi192.stdout:5/461: stat d2/db/d2b/d52/f99 0 2024-04-23T02:13:58.878 INFO:tasks.workunit.client.1.smithi192.stdout:1/262: getdents dc/d16/d19/d30/d32 0 2024-04-23T02:13:58.879 INFO:tasks.workunit.client.1.smithi192.stdout:1/263: stat dc/d14/f27 0 2024-04-23T02:13:58.880 INFO:tasks.workunit.client.1.smithi192.stdout:1/264: write dc/fd [5604133,33754] 0 2024-04-23T02:13:58.880 INFO:tasks.workunit.client.0.smithi071.stdout:6/318: symlink d6/d1f/d4e/l65 0 2024-04-23T02:13:58.881 INFO:tasks.workunit.client.0.smithi071.stdout:6/319: write d6/fc [2419245,130514] 0 2024-04-23T02:13:58.881 INFO:tasks.workunit.client.1.smithi192.stdout:1/265: chown dc/d16/d19/d30/d32/d34 0 1 2024-04-23T02:13:58.881 INFO:tasks.workunit.client.1.smithi192.stdout:5/462: symlink d2/db/d16/d1b/d3d/d40/d49/l9b 0 2024-04-23T02:13:58.885 INFO:tasks.workunit.client.1.smithi192.stdout:1/266: mknod dc/d16/d19/d30/c4a 0 2024-04-23T02:13:58.886 INFO:tasks.workunit.client.0.smithi071.stdout:6/320: getdents d6/d3e/d3f 0 2024-04-23T02:13:58.886 INFO:tasks.workunit.client.0.smithi071.stdout:6/321: chown d6/d17/c40 37798564 1 2024-04-23T02:13:58.886 INFO:tasks.workunit.client.1.smithi192.stdout:5/463: creat d2/d32/f9c x:0 0 0 2024-04-23T02:13:58.886 INFO:tasks.workunit.client.1.smithi192.stdout:1/267: chown dc/d16/d19/d30/l41 22 1 2024-04-23T02:13:58.887 INFO:tasks.workunit.client.1.smithi192.stdout:1/268: readlink dc/d14/d23/l2b 0 2024-04-23T02:13:58.887 INFO:tasks.workunit.client.1.smithi192.stdout:0/526: dread d6/df/d13/d25/d26/f27 [0,4194304] 0 2024-04-23T02:13:58.888 INFO:tasks.workunit.client.1.smithi192.stdout:1/269: write dc/d16/f31 [1416968,8075] 0 2024-04-23T02:13:58.892 INFO:tasks.workunit.client.1.smithi192.stdout:1/270: mknod dc/d16/d19/d30/d32/d44/c4b 0 2024-04-23T02:13:58.893 INFO:tasks.workunit.client.1.smithi192.stdout:5/464: write d2/db/d16/f57 [1646443,16730] 0 2024-04-23T02:13:58.893 INFO:tasks.workunit.client.1.smithi192.stdout:0/527: symlink d6/df/d13/d25/d86/da5/lb0 0 2024-04-23T02:13:58.895 INFO:tasks.workunit.client.1.smithi192.stdout:5/465: fsync d2/db/d16/d1b/d3d/d40/f81 0 2024-04-23T02:13:58.895 INFO:tasks.workunit.client.1.smithi192.stdout:0/528: dread - d6/d4b/f6c zero size 2024-04-23T02:13:58.896 INFO:tasks.workunit.client.1.smithi192.stdout:1/271: fsync dc/d14/f21 0 2024-04-23T02:13:58.897 INFO:tasks.workunit.client.1.smithi192.stdout:0/529: chown d6/df/d13/d25/la0 251292290 1 2024-04-23T02:13:58.898 INFO:tasks.workunit.client.1.smithi192.stdout:5/466: creat d2/d6/d2d/d41/d43/f9d x:0 0 0 2024-04-23T02:13:58.901 INFO:tasks.workunit.client.1.smithi192.stdout:0/530: mkdir d6/df/d56/db1 0 2024-04-23T02:13:58.902 INFO:tasks.workunit.client.1.smithi192.stdout:0/531: write d6/df/d56/d85/d71/f84 [1010752,112740] 0 2024-04-23T02:13:58.902 INFO:tasks.workunit.client.1.smithi192.stdout:5/467: chown d2/d6/d2d/c39 5747 1 2024-04-23T02:13:58.903 INFO:tasks.workunit.client.1.smithi192.stdout:1/272: creat dc/d16/d19/d30/d32/d34/f4c x:0 0 0 2024-04-23T02:13:58.904 INFO:tasks.workunit.client.1.smithi192.stdout:0/532: symlink d6/df/d47/d55/dae/lb2 0 2024-04-23T02:13:58.905 INFO:tasks.workunit.client.1.smithi192.stdout:1/273: creat dc/d16/d19/d30/d32/f4d x:0 0 0 2024-04-23T02:13:58.906 INFO:tasks.workunit.client.1.smithi192.stdout:0/533: fsync d6/df/d13/f31 0 2024-04-23T02:13:58.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.905+0000 7f1d002c1700 1 -- 172.21.15.71:0/888108925 --> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cdc000bf0 con 0x7f1ce406ca60 2024-04-23T02:13:58.910 INFO:tasks.workunit.client.1.smithi192.stdout:0/534: truncate d6/df/d47/f4c 1180432 0 2024-04-23T02:13:58.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.907+0000 7f1cef7fe700 1 -- 172.21.15.71:0/888108925 <== mgr.24351 v2:172.21.15.192:6828/966231696 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f1cdc000bf0 con 0x7f1ce406ca60 2024-04-23T02:13:58.911 INFO:tasks.workunit.client.1.smithi192.stdout:1/274: link dc/d16/d19/d30/c4a dc/d14/d15/d35/c4e 0 2024-04-23T02:13:58.913 INFO:tasks.workunit.client.1.smithi192.stdout:0/535: unlink d6/df/f23 0 2024-04-23T02:13:58.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.909+0000 7f1d002c1700 1 -- 172.21.15.71:0/888108925 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f1ce406ca60 msgr2=0x7f1ce406ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:58.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.909+0000 7f1d002c1700 1 --2- 172.21.15.71:0/888108925 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f1ce406ca60 0x7f1ce406ef20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8100c70 tx=0x7f1ce8009260 comp rx=0 tx=0).stop 2024-04-23T02:13:58.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.909+0000 7f1d002c1700 1 -- 172.21.15.71:0/888108925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cf8100e10 msgr2=0x7f1cf819cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:58.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.909+0000 7f1d002c1700 1 --2- 172.21.15.71:0/888108925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cf8100e10 0x7f1cf819cf00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400bde0 tx=0x7f1cf400be10 comp rx=0 tx=0).stop 2024-04-23T02:13:58.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.909+0000 7f1d002c1700 1 -- 172.21.15.71:0/888108925 shutdown_connections 2024-04-23T02:13:58.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.909+0000 7f1d002c1700 1 --2- 172.21.15.71:0/888108925 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f1ce406ca60 0x7f1ce406ef20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:58.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.909+0000 7f1d002c1700 1 --2- 172.21.15.71:0/888108925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cf80ffc10 0x7f1cf819c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:58.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.909+0000 7f1d002c1700 1 --2- 172.21.15.71:0/888108925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cf8100e10 0x7f1cf819cf00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:58.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.909+0000 7f1d002c1700 1 -- 172.21.15.71:0/888108925 >> 172.21.15.71:0/888108925 conn(0x7f1cf80fb190 msgr2=0x7f1cf8068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:13:58.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.909+0000 7f1d002c1700 1 -- 172.21.15.71:0/888108925 shutdown_connections 2024-04-23T02:13:58.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:58.909+0000 7f1d002c1700 1 -- 172.21.15.71:0/888108925 wait complete. 2024-04-23T02:13:58.920 INFO:tasks.workunit.client.1.smithi192.stdout:0/536: write d6/df/d13/f51 [1199372,112170] 0 2024-04-23T02:13:58.922 INFO:tasks.workunit.client.1.smithi192.stdout:1/275: dwrite dc/d16/d19/d30/d32/d34/f3e [0,4194304] 0 2024-04-23T02:13:58.928 INFO:tasks.workunit.client.1.smithi192.stdout:1/276: write dc/d14/f21 [369395,41631] 0 2024-04-23T02:13:58.932 INFO:tasks.workunit.client.1.smithi192.stdout:0/537: dwrite d6/f6d [0,4194304] 0 2024-04-23T02:13:58.942 INFO:tasks.workunit.client.1.smithi192.stdout:1/277: dwrite dc/d16/d19/d30/f3f [4194304,4194304] 0 2024-04-23T02:13:58.945 INFO:tasks.workunit.client.1.smithi192.stdout:0/538: dwrite d6/df/f4a [0,4194304] 0 2024-04-23T02:13:58.949 INFO:tasks.workunit.client.1.smithi192.stdout:0/539: creat d6/df/d13/d25/d94/fb3 x:0 0 0 2024-04-23T02:13:58.951 INFO:tasks.workunit.client.1.smithi192.stdout:0/540: fdatasync d6/df/f50 0 2024-04-23T02:13:58.966 INFO:tasks.workunit.client.1.smithi192.stdout:0/541: write d6/df/d13/d25/d86/da5/faf [514506,63082] 0 2024-04-23T02:13:58.967 INFO:tasks.workunit.client.1.smithi192.stdout:0/542: mknod d6/df/d13/d6b/cb4 0 2024-04-23T02:13:58.969 INFO:tasks.workunit.client.1.smithi192.stdout:0/543: creat d6/fb5 x:0 0 0 2024-04-23T02:13:58.971 INFO:tasks.workunit.client.1.smithi192.stdout:0/544: dread - d6/df/d13/d25/d26/fac zero size 2024-04-23T02:13:58.975 INFO:tasks.workunit.client.1.smithi192.stdout:0/545: rename d6/df/d13/d25/d26/l34 to d6/df/d56/d85/d71/lb6 0 2024-04-23T02:13:58.985 INFO:tasks.workunit.client.1.smithi192.stdout:2/408: sync 2024-04-23T02:13:58.985 INFO:tasks.workunit.client.1.smithi192.stdout:9/407: sync 2024-04-23T02:13:58.985 INFO:tasks.workunit.client.1.smithi192.stdout:7/459: sync 2024-04-23T02:13:58.985 INFO:tasks.workunit.client.1.smithi192.stdout:6/537: sync 2024-04-23T02:13:58.985 INFO:tasks.workunit.client.1.smithi192.stdout:8/622: sync 2024-04-23T02:13:58.990 INFO:tasks.workunit.client.1.smithi192.stdout:7/460: write d5/d17/d1c/d59/d5a/f32 [3420537,32657] 0 2024-04-23T02:13:58.990 INFO:tasks.workunit.client.1.smithi192.stdout:8/623: stat d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/cca 0 2024-04-23T02:13:58.992 INFO:tasks.workunit.client.1.smithi192.stdout:6/538: creat d3/d15/d1b/d66/d80/d8e/faf x:0 0 0 2024-04-23T02:13:58.992 INFO:tasks.workunit.client.0.smithi071.stdout:9/347: sync 2024-04-23T02:13:58.992 INFO:tasks.workunit.client.0.smithi071.stdout:4/382: sync 2024-04-23T02:13:58.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:58 smithi071 ceph-mon[39837]: pgmap v26: 65 pgs: 65 active+clean; 1.2 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 21 MiB/s rd, 144 MiB/s wr, 211 op/s 2024-04-23T02:13:58.995 INFO:tasks.workunit.client.1.smithi192.stdout:0/546: dread d6/df/d47/d55/f70 [0,4194304] 0 2024-04-23T02:13:58.996 INFO:tasks.workunit.client.1.smithi192.stdout:6/539: truncate d3/d18/f9f 742045 0 2024-04-23T02:13:58.997 INFO:tasks.workunit.client.1.smithi192.stdout:0/547: stat d6/df/d13/l43 0 2024-04-23T02:13:58.998 INFO:tasks.workunit.client.1.smithi192.stdout:8/624: rename d1/df/l25 to d1/d12/d23/d2c/d8c/d95/ldb 0 2024-04-23T02:13:58.999 INFO:tasks.workunit.client.0.smithi071.stdout:4/383: creat d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c/f7c x:0 0 0 2024-04-23T02:13:59.000 INFO:tasks.workunit.client.1.smithi192.stdout:2/409: dwrite d8/d26/d29/d43/d50/d5e/f5f [0,4194304] 0 2024-04-23T02:13:59.003 INFO:tasks.workunit.client.1.smithi192.stdout:0/548: fdatasync d6/df/d13/d25/d26/f27 0 2024-04-23T02:13:59.006 INFO:tasks.workunit.client.0.smithi071.stdout:9/348: dwrite d1/d1a/f70 [0,4194304] 0 2024-04-23T02:13:59.007 INFO:tasks.workunit.client.1.smithi192.stdout:8/625: creat d1/d12/d23/d4b/fdc x:0 0 0 2024-04-23T02:13:59.008 INFO:tasks.workunit.client.1.smithi192.stdout:6/540: write d3/d15/d1b/d21/d52/f67 [28670,18854] 0 2024-04-23T02:13:59.009 INFO:tasks.workunit.client.1.smithi192.stdout:0/549: truncate d6/df/d35/d49/f54 5150004 0 2024-04-23T02:13:59.010 INFO:tasks.workunit.client.0.smithi071.stdout:4/384: creat d2/d2a/d2d/d39/d42/d49/d67/d3e/f7d x:0 0 0 2024-04-23T02:13:59.013 INFO:tasks.workunit.client.1.smithi192.stdout:9/408: dread d0/d9/d44/d4f/f3d [0,4194304] 0 2024-04-23T02:13:59.014 INFO:tasks.workunit.client.1.smithi192.stdout:8/626: mknod d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/da3/cdd 0 2024-04-23T02:13:59.015 INFO:tasks.workunit.client.1.smithi192.stdout:9/409: write d0/d9/d44/d4f/f8 [923105,116198] 0 2024-04-23T02:13:59.016 INFO:tasks.workunit.client.0.smithi071.stdout:4/385: mkdir d2/d2a/d2d/d39/d42/d49/d67/d7e 0 2024-04-23T02:13:59.018 INFO:tasks.workunit.client.1.smithi192.stdout:9/410: write d0/d9/f3a [537263,6634] 0 2024-04-23T02:13:59.021 INFO:tasks.workunit.client.1.smithi192.stdout:6/541: write d3/d15/d1b/f72 [571196,1078] 0 2024-04-23T02:13:59.021 INFO:tasks.workunit.client.0.smithi071.stdout:9/349: dwrite d1/d1a/f25 [4194304,4194304] 0 2024-04-23T02:13:59.023 INFO:tasks.workunit.client.1.smithi192.stdout:6/542: chown d3 149789112 1 2024-04-23T02:13:59.023 INFO:tasks.workunit.client.0.smithi071.stdout:4/386: creat d2/d2a/d73/f7f x:0 0 0 2024-04-23T02:13:59.024 INFO:tasks.workunit.client.0.smithi071.stdout:9/350: truncate d1/f3d 409466 0 2024-04-23T02:13:59.026 INFO:tasks.workunit.client.0.smithi071.stdout:9/351: symlink d1/d4e/d54/l74 0 2024-04-23T02:13:59.027 INFO:tasks.workunit.client.1.smithi192.stdout:0/550: dwrite d6/df/d47/d55/f70 [4194304,4194304] 0 2024-04-23T02:13:59.030 INFO:tasks.workunit.client.1.smithi192.stdout:8/627: link d1/d12/d23/d4b/d73/d7e/d88/da9/lbc d1/d12/d23/d4b/d73/d4c/lde 0 2024-04-23T02:13:59.031 INFO:tasks.workunit.client.1.smithi192.stdout:0/551: write d6/df/d13/d25/d26/f82 [1319052,2893] 0 2024-04-23T02:13:59.037 INFO:tasks.workunit.client.1.smithi192.stdout:0/552: mknod d6/df/cb7 0 2024-04-23T02:13:59.040 INFO:tasks.workunit.client.1.smithi192.stdout:9/411: dwrite d0/d9/d44/d74/f82 [0,4194304] 0 2024-04-23T02:13:59.042 INFO:tasks.workunit.client.1.smithi192.stdout:0/553: rmdir d6/df/d13/d25/d86 39 2024-04-23T02:13:59.042 INFO:tasks.workunit.client.1.smithi192.stdout:9/412: read - d0/d9/d44/d4f/f8e zero size 2024-04-23T02:13:59.045 INFO:tasks.workunit.client.1.smithi192.stdout:8/628: dwrite d1/d12/d23/d2c/d4d/fc8 [0,4194304] 0 2024-04-23T02:13:59.046 INFO:tasks.workunit.client.0.smithi071.stdout:9/352: dread d1/f3 [0,4194304] 0 2024-04-23T02:13:59.048 INFO:tasks.workunit.client.0.smithi071.stdout:4/387: dwrite d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f5b [0,4194304] 0 2024-04-23T02:13:59.049 INFO:tasks.workunit.client.0.smithi071.stdout:9/353: dread d1/f50 [0,4194304] 0 2024-04-23T02:13:59.052 INFO:tasks.workunit.client.0.smithi071.stdout:4/388: creat d2/d2a/f80 x:0 0 0 2024-04-23T02:13:59.055 INFO:tasks.workunit.client.0.smithi071.stdout:4/389: fsync d2/d2a/d2d/d39/f46 0 2024-04-23T02:13:59.055 INFO:tasks.workunit.client.1.smithi192.stdout:0/554: mknod d6/df/d13/d25/d26/cb8 0 2024-04-23T02:13:59.055 INFO:tasks.workunit.client.1.smithi192.stdout:9/413: mknod d0/d9/d44/d53/c94 0 2024-04-23T02:13:59.056 INFO:tasks.workunit.client.1.smithi192.stdout:0/555: stat d6/df/f30 0 2024-04-23T02:13:59.058 INFO:tasks.workunit.client.0.smithi071.stdout:4/390: stat d2/d2a/d2d/d39/d42/d49/d67/d3e/l52 0 2024-04-23T02:13:59.058 INFO:tasks.workunit.client.1.smithi192.stdout:6/543: dread d3/d15/d1b/f2a [0,4194304] 0 2024-04-23T02:13:59.060 INFO:tasks.workunit.client.1.smithi192.stdout:0/556: mknod d6/df/d47/d55/dae/cb9 0 2024-04-23T02:13:59.061 INFO:tasks.workunit.client.0.smithi071.stdout:4/391: creat d2/d2a/d2d/d39/d42/d49/d67/d7e/f81 x:0 0 0 2024-04-23T02:13:59.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.058+0000 7f0d8efc3700 1 -- 172.21.15.71:0/1788068129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d88074dc0 msgr2=0x7f0d88073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:59.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.058+0000 7f0d8efc3700 1 --2- 172.21.15.71:0/1788068129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d88074dc0 0x7f0d88073220 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009910 tx=0x7f0d7c009c20 comp rx=0 tx=0).stop 2024-04-23T02:13:59.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.058+0000 7f0d8efc3700 1 -- 172.21.15.71:0/1788068129 shutdown_connections 2024-04-23T02:13:59.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.058+0000 7f0d8efc3700 1 --2- 172.21.15.71:0/1788068129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d880737f0 0x7f0d88073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:59.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.058+0000 7f0d8efc3700 1 --2- 172.21.15.71:0/1788068129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d88074dc0 0x7f0d88073220 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:59.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.058+0000 7f0d8efc3700 1 -- 172.21.15.71:0/1788068129 >> 172.21.15.71:0/1788068129 conn(0x7f0d880fc230 msgr2=0x7f0d880fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:13:59.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.058+0000 7f0d8efc3700 1 -- 172.21.15.71:0/1788068129 shutdown_connections 2024-04-23T02:13:59.063 INFO:tasks.workunit.client.1.smithi192.stdout:6/544: mkdir d3/d11/db0 0 2024-04-23T02:13:59.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.058+0000 7f0d8efc3700 1 -- 172.21.15.71:0/1788068129 wait complete. 2024-04-23T02:13:59.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.058+0000 7f0d8efc3700 1 Processor -- start 2024-04-23T02:13:59.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.059+0000 7f0d8efc3700 1 -- start start 2024-04-23T02:13:59.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.059+0000 7f0d8efc3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d880737f0 0x7f0d8810c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:59.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.059+0000 7f0d8efc3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d88074dc0 0x7f0d8810c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:59.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.059+0000 7f0d8efc3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8810cc80 con 0x7f0d88074dc0 2024-04-23T02:13:59.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.059+0000 7f0d8efc3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8810cdc0 con 0x7f0d880737f0 2024-04-23T02:13:59.064 INFO:tasks.workunit.client.1.smithi192.stdout:8/629: dwrite d1/d12/d23/d4b/d73/d5d/fd8 [0,4194304] 0 2024-04-23T02:13:59.064 INFO:tasks.workunit.client.0.smithi071.stdout:4/392: mkdir d2/d2a/d2d/d39/d42/d49/d67/d7e/d82 0 2024-04-23T02:13:59.065 INFO:tasks.workunit.client.1.smithi192.stdout:6/545: chown d3/d15/d1b/d21 44962010 1 2024-04-23T02:13:59.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.059+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d88074dc0 0x7f0d8810c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:59.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.059+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d88074dc0 0x7f0d8810c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50190/0 (socket says 172.21.15.71:50190) 2024-04-23T02:13:59.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.059+0000 7f0d87fff700 1 -- 172.21.15.71:0/3023778726 learned_addr learned my addr 172.21.15.71:0/3023778726 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:13:59.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.059+0000 7f0d8cd5f700 1 --2- 172.21.15.71:0/3023778726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d880737f0 0x7f0d8810c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:59.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.059+0000 7f0d87fff700 1 -- 172.21.15.71:0/3023778726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d880737f0 msgr2=0x7f0d8810c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:59.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.059+0000 7f0d87fff700 1 --2- 172.21.15.71:0/3023778726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d880737f0 0x7f0d8810c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:59.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.059+0000 7f0d87fff700 1 -- 172.21.15.71:0/3023778726 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c0095f0 con 0x7f0d88074dc0 2024-04-23T02:13:59.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.060+0000 7f0d87fff700 1 --2- 172.21.15.71:0/3023778726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d88074dc0 0x7f0d8810c660 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400d900 tx=0x7f0d7400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:13:59.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.060+0000 7f0d85ffb700 1 -- 172.21.15.71:0/3023778726 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d7400b920 con 0x7f0d88074dc0 2024-04-23T02:13:59.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.060+0000 7f0d85ffb700 1 -- 172.21.15.71:0/3023778726 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f0d74010460 con 0x7f0d88074dc0 2024-04-23T02:13:59.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.060+0000 7f0d85ffb700 1 -- 172.21.15.71:0/3023778726 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d7400f5d0 con 0x7f0d88074dc0 2024-04-23T02:13:59.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.061+0000 7f0d8efc3700 1 -- 172.21.15.71:0/3023778726 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8806a950 con 0x7f0d88074dc0 2024-04-23T02:13:59.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.061+0000 7f0d8efc3700 1 -- 172.21.15.71:0/3023778726 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8806ae20 con 0x7f0d88074dc0 2024-04-23T02:13:59.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.062+0000 7f0d8efc3700 1 -- 172.21.15.71:0/3023778726 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8810fce0 con 0x7f0d88074dc0 2024-04-23T02:13:59.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.063+0000 7f0d85ffb700 1 -- 172.21.15.71:0/3023778726 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) v1 ==== 90681+0+0 (secure 0 0 0) 0x7f0d740105d0 con 0x7f0d88074dc0 2024-04-23T02:13:59.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.063+0000 7f0d85ffb700 1 --2- 172.21.15.71:0/3023778726 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f0d7806ca20 0x7f0d7806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:59.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.063+0000 7f0d85ffb700 1 -- 172.21.15.71:0/3023778726 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f0d7408c160 con 0x7f0d88074dc0 2024-04-23T02:13:59.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.064+0000 7f0d8cd5f700 1 --2- 172.21.15.71:0/3023778726 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f0d7806ca20 0x7f0d7806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:59.068 INFO:tasks.workunit.client.0.smithi071.stdout:4/393: creat d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d29/f83 x:0 0 0 2024-04-23T02:13:59.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.068+0000 7f0d8cd5f700 1 --2- 172.21.15.71:0/3023778726 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f0d7806ca20 0x7f0d7806eee0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c006010 tx=0x7f0d7c005ec0 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:13:59.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.069+0000 7f0d85ffb700 1 -- 172.21.15.71:0/3023778726 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f0d74059ba0 con 0x7f0d88074dc0 2024-04-23T02:13:59.073 INFO:tasks.workunit.client.1.smithi192.stdout:8/630: truncate d1/d12/d23/d4b/d73/f43 1183007 0 2024-04-23T02:13:59.073 INFO:tasks.workunit.client.1.smithi192.stdout:0/557: dread d6/df/d13/d25/d26/f90 [0,4194304] 0 2024-04-23T02:13:59.077 INFO:tasks.workunit.client.1.smithi192.stdout:8/631: truncate d1/d9/fa 1803641 0 2024-04-23T02:13:59.079 INFO:tasks.workunit.client.1.smithi192.stdout:8/632: mkdir d1/d12/d68/ddf 0 2024-04-23T02:13:59.081 INFO:tasks.workunit.client.1.smithi192.stdout:9/414: dread d0/d9/d44/d74/d3c/d5f/d7d/f93 [0,4194304] 0 2024-04-23T02:13:59.081 INFO:tasks.workunit.client.1.smithi192.stdout:8/633: chown d1/d9/d3e/l6f 821752 1 2024-04-23T02:13:59.082 INFO:tasks.workunit.client.1.smithi192.stdout:0/558: dwrite d6/df/d35/d8b/f9b [0,4194304] 0 2024-04-23T02:13:59.083 INFO:tasks.workunit.client.1.smithi192.stdout:8/634: write d1/d12/fa5 [5025412,104333] 0 2024-04-23T02:13:59.089 INFO:tasks.workunit.client.1.smithi192.stdout:9/415: truncate d0/d9/d44/d51/d5b/f69 730673 0 2024-04-23T02:13:59.090 INFO:tasks.workunit.client.0.smithi071.stdout:9/354: write d1/d4e/f4f [1726620,7829] 0 2024-04-23T02:13:59.092 INFO:tasks.workunit.client.1.smithi192.stdout:8/635: creat d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/fe0 x:0 0 0 2024-04-23T02:13:59.092 INFO:tasks.workunit.client.1.smithi192.stdout:6/546: truncate d3/d11/f43 429908 0 2024-04-23T02:13:59.093 INFO:tasks.workunit.client.1.smithi192.stdout:6/547: dread - d3/d15/d1b/d66/d80/f8b zero size 2024-04-23T02:13:59.094 INFO:tasks.workunit.client.0.smithi071.stdout:9/355: read d1/d32/f40 [503945,102628] 0 2024-04-23T02:13:59.096 INFO:tasks.workunit.client.1.smithi192.stdout:9/416: mknod d0/d9/d44/d4f/c95 0 2024-04-23T02:13:59.097 INFO:tasks.workunit.client.0.smithi071.stdout:9/356: mkdir d1/d4e/d54/d75 0 2024-04-23T02:13:59.097 INFO:tasks.workunit.client.1.smithi192.stdout:9/417: chown d0/d9/d44/d51/d5b/d61/f84 20584 1 2024-04-23T02:13:59.098 INFO:tasks.workunit.client.1.smithi192.stdout:8/636: mkdir d1/de1 0 2024-04-23T02:13:59.101 INFO:tasks.workunit.client.1.smithi192.stdout:6/548: mknod d3/cb1 0 2024-04-23T02:13:59.103 INFO:tasks.workunit.client.1.smithi192.stdout:8/637: mkdir d1/d12/d23/d4b/d73/d7e/d88/da9/db1/de2 0 2024-04-23T02:13:59.104 INFO:tasks.workunit.client.0.smithi071.stdout:9/357: dwrite d1/d1a/d5a/f6a [0,4194304] 0 2024-04-23T02:13:59.105 INFO:tasks.workunit.client.1.smithi192.stdout:6/549: mkdir d3/d15/d1b/d21/d52/db2 0 2024-04-23T02:13:59.106 INFO:tasks.workunit.client.1.smithi192.stdout:0/559: dread d6/df/f50 [0,4194304] 0 2024-04-23T02:13:59.106 INFO:tasks.workunit.client.1.smithi192.stdout:8/638: write d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/fba [975435,80243] 0 2024-04-23T02:13:59.107 INFO:tasks.workunit.client.0.smithi071.stdout:9/358: mkdir d1/d4e/d66/d76 0 2024-04-23T02:13:59.107 INFO:tasks.workunit.client.1.smithi192.stdout:8/639: chown d1/d12/d23/d4b/fdc 710012 1 2024-04-23T02:13:59.108 INFO:tasks.workunit.client.0.smithi071.stdout:4/394: dread d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c/f2e [0,4194304] 0 2024-04-23T02:13:59.108 INFO:tasks.workunit.client.1.smithi192.stdout:0/560: fsync d6/df/d13/d25/d94/fb3 0 2024-04-23T02:13:59.108 INFO:tasks.workunit.client.1.smithi192.stdout:6/550: mknod d3/d15/d1b/d21/d7c/cb3 0 2024-04-23T02:13:59.110 INFO:tasks.workunit.client.1.smithi192.stdout:0/561: stat d6/df/d13/d6b/cb4 0 2024-04-23T02:13:59.110 INFO:tasks.workunit.client.0.smithi071.stdout:4/395: chown d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d10/l26 0 1 2024-04-23T02:13:59.111 INFO:tasks.workunit.client.1.smithi192.stdout:8/640: mkdir d1/d12/d7d/dd2/de3 0 2024-04-23T02:13:59.112 INFO:tasks.workunit.client.0.smithi071.stdout:9/359: mkdir d1/d32/d3a/d77 0 2024-04-23T02:13:59.113 INFO:tasks.workunit.client.0.smithi071.stdout:4/396: fsync d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d29/f31 0 2024-04-23T02:13:59.113 INFO:tasks.workunit.client.1.smithi192.stdout:0/562: chown d6/l24 8111029 1 2024-04-23T02:13:59.114 INFO:tasks.workunit.client.0.smithi071.stdout:9/360: getdents d1/d32/d3a/d77 0 2024-04-23T02:13:59.115 INFO:tasks.workunit.client.0.smithi071.stdout:4/397: mkdir d2/d2a/d73/d5e/d84 0 2024-04-23T02:13:59.116 INFO:tasks.workunit.client.0.smithi071.stdout:9/361: chown d1/fe 7 1 2024-04-23T02:13:59.117 INFO:tasks.workunit.client.1.smithi192.stdout:0/563: dread d6/df/f4a [0,4194304] 0 2024-04-23T02:13:59.117 INFO:tasks.workunit.client.1.smithi192.stdout:8/641: rename d1/d9/la7 to d1/d12/d23/d4b/le4 0 2024-04-23T02:13:59.117 INFO:tasks.workunit.client.0.smithi071.stdout:4/398: write d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f5b [1932074,72086] 0 2024-04-23T02:13:59.117 INFO:tasks.workunit.client.0.smithi071.stdout:9/362: write d1/d1a/d5a/f5e [1323255,120781] 0 2024-04-23T02:13:59.119 INFO:tasks.workunit.client.0.smithi071.stdout:9/363: write d1/d32/d41/f57 [120437,67031] 0 2024-04-23T02:13:59.120 INFO:tasks.workunit.client.1.smithi192.stdout:0/564: fdatasync d6/df/d35/d49/f54 0 2024-04-23T02:13:59.121 INFO:tasks.workunit.client.0.smithi071.stdout:4/399: creat d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d29/f85 x:0 0 0 2024-04-23T02:13:59.123 INFO:tasks.workunit.client.0.smithi071.stdout:4/400: chown d2/d2a/d2d/d39/f63 840 1 2024-04-23T02:13:59.124 INFO:tasks.workunit.client.0.smithi071.stdout:4/401: write d2/d2a/d2d/f5c [1044384,61115] 0 2024-04-23T02:13:59.125 INFO:tasks.workunit.client.1.smithi192.stdout:8/642: creat d1/d12/d23/d2c/fe5 x:0 0 0 2024-04-23T02:13:59.126 INFO:tasks.workunit.client.0.smithi071.stdout:2/331: sync 2024-04-23T02:13:59.126 INFO:tasks.workunit.client.0.smithi071.stdout:4/402: chown d2/l11 293 1 2024-04-23T02:13:59.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:58 smithi192 ceph-mon[49662]: pgmap v26: 65 pgs: 65 active+clean; 1.2 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 21 MiB/s rd, 144 MiB/s wr, 211 op/s 2024-04-23T02:13:59.127 INFO:tasks.workunit.client.1.smithi192.stdout:0/565: symlink d6/df/d13/d25/d3a/d5b/lba 0 2024-04-23T02:13:59.128 INFO:tasks.workunit.client.0.smithi071.stdout:2/332: read d0/d2/d26/d14/f19 [2932021,37680] 0 2024-04-23T02:13:59.128 INFO:tasks.workunit.client.1.smithi192.stdout:8/643: read d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/fc3 [1819241,125335] 0 2024-04-23T02:13:59.130 INFO:tasks.workunit.client.1.smithi192.stdout:9/418: dread d0/d9/d44/f7c [0,4194304] 0 2024-04-23T02:13:59.130 INFO:tasks.workunit.client.0.smithi071.stdout:2/333: mkdir d0/d2/d26/d14/d1f/d36/d6d 0 2024-04-23T02:13:59.133 INFO:tasks.workunit.client.1.smithi192.stdout:9/419: dread - d0/d9/f32 zero size 2024-04-23T02:13:59.133 INFO:tasks.workunit.client.1.smithi192.stdout:0/566: symlink d6/df/d13/d25/d3a/d93/lbb 0 2024-04-23T02:13:59.133 INFO:tasks.workunit.client.0.smithi071.stdout:2/334: creat d0/d2/d26/d14/d1f/d36/d6a/f6e x:0 0 0 2024-04-23T02:13:59.134 INFO:tasks.workunit.client.0.smithi071.stdout:4/403: dwrite d2/d2a/d2d/d39/d42/d49/d67/d1d/d57/f72 [0,4194304] 0 2024-04-23T02:13:59.134 INFO:tasks.workunit.client.0.smithi071.stdout:2/335: readlink d0/d2/d26/df/d30/l38 0 2024-04-23T02:13:59.135 INFO:tasks.workunit.client.0.smithi071.stdout:4/404: fdatasync d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f59 0 2024-04-23T02:13:59.136 INFO:tasks.workunit.client.1.smithi192.stdout:6/551: dwrite d3/d28/d79/fa4 [0,4194304] 0 2024-04-23T02:13:59.137 INFO:tasks.workunit.client.1.smithi192.stdout:9/420: mkdir d0/d9/d44/d51/d5b/d96 0 2024-04-23T02:13:59.139 INFO:tasks.workunit.client.1.smithi192.stdout:8/644: dwrite d1/d9/d3e/f55 [0,4194304] 0 2024-04-23T02:13:59.139 INFO:tasks.workunit.client.1.smithi192.stdout:9/421: readlink d0/l2f 0 2024-04-23T02:13:59.140 INFO:tasks.workunit.client.1.smithi192.stdout:0/567: truncate d6/df/d47/f4c 1094778 0 2024-04-23T02:13:59.142 INFO:tasks.workunit.client.0.smithi071.stdout:4/405: dwrite d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f16 [0,4194304] 0 2024-04-23T02:13:59.143 INFO:tasks.workunit.client.1.smithi192.stdout:8/645: write d1/d12/d23/d4b/d73/d4c/d82/fb4 [146140,125110] 0 2024-04-23T02:13:59.144 INFO:tasks.workunit.client.1.smithi192.stdout:8/646: chown d1/fe 18770983 1 2024-04-23T02:13:59.147 INFO:tasks.workunit.client.0.smithi071.stdout:9/364: dread d1/f30 [0,4194304] 0 2024-04-23T02:13:59.147 INFO:tasks.workunit.client.1.smithi192.stdout:3/384: sync 2024-04-23T02:13:59.148 INFO:tasks.workunit.client.1.smithi192.stdout:4/623: sync 2024-04-23T02:13:59.148 INFO:tasks.workunit.client.1.smithi192.stdout:8/647: write d1/f63 [166696,119552] 0 2024-04-23T02:13:59.151 INFO:tasks.workunit.client.1.smithi192.stdout:6/552: read d3/d15/d1b/d21/f60 [298443,42994] 0 2024-04-23T02:13:59.153 INFO:tasks.workunit.client.1.smithi192.stdout:6/553: write d3/d11/f4a [1354946,100258] 0 2024-04-23T02:13:59.155 INFO:tasks.workunit.client.1.smithi192.stdout:6/554: chown d3/d15/d1b/d21/d3a/fa6 124 1 2024-04-23T02:13:59.156 INFO:tasks.workunit.client.1.smithi192.stdout:3/385: unlink d1/d3/d17/d2b/f53 0 2024-04-23T02:13:59.157 INFO:tasks.workunit.client.1.smithi192.stdout:6/555: write d3/d18/f94 [606129,12458] 0 2024-04-23T02:13:59.158 INFO:tasks.workunit.client.0.smithi071.stdout:4/406: dwrite d2/d2a/d2d/d39/d42/d49/d67/f1e [0,4194304] 0 2024-04-23T02:13:59.159 INFO:tasks.workunit.client.1.smithi192.stdout:0/568: link d6/df/d56/d85/l98 d6/d4b/d9d/lbc 0 2024-04-23T02:13:59.159 INFO:tasks.workunit.client.1.smithi192.stdout:8/648: read - d1/d9/d8f/d90/fa2 zero size 2024-04-23T02:13:59.159 INFO:tasks.workunit.client.1.smithi192.stdout:4/624: mkdir de/d15/d19/d1a/d95/dd5 0 2024-04-23T02:13:59.159 INFO:tasks.workunit.client.1.smithi192.stdout:0/569: dread - d6/fb5 zero size 2024-04-23T02:13:59.161 INFO:tasks.workunit.client.1.smithi192.stdout:9/422: dread d0/d9/d44/d4f/f41 [0,4194304] 0 2024-04-23T02:13:59.161 INFO:tasks.workunit.client.0.smithi071.stdout:9/365: dwrite d1/d32/d41/f57 [0,4194304] 0 2024-04-23T02:13:59.162 INFO:tasks.workunit.client.1.smithi192.stdout:0/570: fdatasync d6/f18 0 2024-04-23T02:13:59.163 INFO:tasks.workunit.client.1.smithi192.stdout:3/386: rename d1/d3/d6/d7/f64 to d1/d3/d6/f81 0 2024-04-23T02:13:59.164 INFO:tasks.workunit.client.1.smithi192.stdout:8/649: write d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/fd9 [440539,1984] 0 2024-04-23T02:13:59.168 INFO:tasks.workunit.client.1.smithi192.stdout:4/625: readlink de/d15/d2a/d74/d92/da0/dca/d34/d7c/da1/lba 0 2024-04-23T02:13:59.169 INFO:tasks.workunit.client.1.smithi192.stdout:0/571: unlink d6/df/d13/d25/d26/d67/l6a 0 2024-04-23T02:13:59.171 INFO:tasks.workunit.client.0.smithi071.stdout:9/366: mkdir d1/d32/d3a/d78 0 2024-04-23T02:13:59.171 INFO:tasks.workunit.client.0.smithi071.stdout:4/407: chown d2/d2a/d2d/d39/d42/d49/d67/f1a 4 1 2024-04-23T02:13:59.171 INFO:tasks.workunit.client.1.smithi192.stdout:4/626: mknod de/d15/d2a/d74/d92/da0/dca/d49/d70/d73/cd6 0 2024-04-23T02:13:59.173 INFO:tasks.workunit.client.1.smithi192.stdout:0/572: symlink d6/df/d13/d6b/d73/lbd 0 2024-04-23T02:13:59.174 INFO:tasks.workunit.client.1.smithi192.stdout:8/650: mkdir d1/d12/d23/d4b/de6 0 2024-04-23T02:13:59.174 INFO:tasks.workunit.client.1.smithi192.stdout:4/627: write de/d15/d2a/d74/d92/da0/dca/f8b [837284,71295] 0 2024-04-23T02:13:59.175 INFO:tasks.workunit.client.1.smithi192.stdout:0/573: dread - d6/df/d13/d6b/d73/f7e zero size 2024-04-23T02:13:59.175 INFO:tasks.workunit.client.0.smithi071.stdout:9/367: mknod d1/d1a/d68/c79 0 2024-04-23T02:13:59.177 INFO:tasks.workunit.client.1.smithi192.stdout:3/387: creat d1/d3/d6/d7/f82 x:0 0 0 2024-04-23T02:13:59.178 INFO:tasks.workunit.client.0.smithi071.stdout:4/408: link d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d29/c6e d2/d2a/d73/c86 0 2024-04-23T02:13:59.178 INFO:tasks.workunit.client.0.smithi071.stdout:9/368: write d1/d32/f40 [500152,23125] 0 2024-04-23T02:13:59.181 INFO:tasks.workunit.client.0.smithi071.stdout:9/369: rename d1/f9 to d1/d32/d3a/d78/f7a 0 2024-04-23T02:13:59.186 INFO:tasks.workunit.client.1.smithi192.stdout:8/651: dwrite d1/d9/d3e/d59/f75 [0,4194304] 0 2024-04-23T02:13:59.188 INFO:tasks.workunit.client.1.smithi192.stdout:3/388: link d1/d3/d6/d7/f72 d1/d3/d17/d1f/d33/d24/d34/d40/f83 0 2024-04-23T02:13:59.189 INFO:tasks.workunit.client.0.smithi071.stdout:4/409: dwrite d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f5d [0,4194304] 0 2024-04-23T02:13:59.191 INFO:tasks.workunit.client.0.smithi071.stdout:4/410: write d2/d2a/d2d/d39/d42/d49/d67/d7e/f81 [594178,6669] 0 2024-04-23T02:13:59.192 INFO:tasks.workunit.client.1.smithi192.stdout:4/628: dwrite de/d15/d2a/d74/d92/da0/dca/d34/d4a/f56 [0,4194304] 0 2024-04-23T02:13:59.195 INFO:tasks.workunit.client.0.smithi071.stdout:4/411: creat d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d10/f87 x:0 0 0 2024-04-23T02:13:59.197 INFO:tasks.workunit.client.0.smithi071.stdout:4/412: rmdir d2/d53 39 2024-04-23T02:13:59.199 INFO:tasks.workunit.client.1.smithi192.stdout:4/629: truncate de/d15/d2a/d74/d92/da0/dca/d49/d26/d59/f63 4610978 0 2024-04-23T02:13:59.207 INFO:tasks.workunit.client.0.smithi071.stdout:9/370: dwrite d1/f3d [0,4194304] 0 2024-04-23T02:13:59.209 INFO:tasks.workunit.client.1.smithi192.stdout:8/652: dread d1/f1d [0,4194304] 0 2024-04-23T02:13:59.211 INFO:tasks.workunit.client.0.smithi071.stdout:9/371: creat d1/d4e/d66/d76/f7b x:0 0 0 2024-04-23T02:13:59.211 INFO:tasks.workunit.client.1.smithi192.stdout:8/653: dread - d1/d12/f92 zero size 2024-04-23T02:13:59.212 INFO:tasks.workunit.client.1.smithi192.stdout:9/423: dwrite d0/d9/f37 [0,4194304] 0 2024-04-23T02:13:59.221 INFO:tasks.workunit.client.1.smithi192.stdout:3/389: rename d1/d3/d17/d1f/d33/d24/d34/d40 to d1/d3/d17/d1f/d33/d24/d34/d6c/d84 0 2024-04-23T02:13:59.222 INFO:tasks.workunit.client.1.smithi192.stdout:4/630: truncate de/d15/d19/d1a/fb0 3655124 0 2024-04-23T02:13:59.224 INFO:tasks.workunit.client.1.smithi192.stdout:0/574: dwrite d6/df/d13/d25/d26/f90 [0,4194304] 0 2024-04-23T02:13:59.227 INFO:tasks.workunit.client.1.smithi192.stdout:0/575: chown d6/df/d13/d25/d26/f27 160621091 1 2024-04-23T02:13:59.228 INFO:tasks.workunit.client.1.smithi192.stdout:4/631: fsync de/d15/d2a/d74/d92/da0/dca/d34/d4a/f56 0 2024-04-23T02:13:59.228 INFO:tasks.workunit.client.1.smithi192.stdout:0/576: readlink d6/df/l29 0 2024-04-23T02:13:59.231 INFO:tasks.workunit.client.1.smithi192.stdout:8/654: link d1/df/fa8 d1/d12/d23/d4b/d73/d7e/fe7 0 2024-04-23T02:13:59.232 INFO:tasks.workunit.client.1.smithi192.stdout:9/424: rename d0/d9/d44/d4f/l76 to d0/d9/d44/d74/d3c/d5f/l97 0 2024-04-23T02:13:59.232 INFO:tasks.workunit.client.1.smithi192.stdout:3/390: creat d1/d3/d17/d1f/d33/d24/d34/d6c/d84/d46/f85 x:0 0 0 2024-04-23T02:13:59.232 INFO:tasks.workunit.client.1.smithi192.stdout:4/632: write de/d15/d2a/d74/d92/da0/dca/d34/d4a/f56 [1091587,4250] 0 2024-04-23T02:13:59.234 INFO:tasks.workunit.client.1.smithi192.stdout:0/577: creat d6/df/d56/db1/fbe x:0 0 0 2024-04-23T02:13:59.236 INFO:tasks.workunit.client.1.smithi192.stdout:3/391: chown d1/d3/d17/c5c 62 1 2024-04-23T02:13:59.239 INFO:tasks.workunit.client.1.smithi192.stdout:4/633: creat de/d15/d53/d81/dac/fd7 x:0 0 0 2024-04-23T02:13:59.239 INFO:tasks.workunit.client.1.smithi192.stdout:8/655: rename d1/d12/d23/l6b to d1/de1/le8 0 2024-04-23T02:13:59.240 INFO:tasks.workunit.client.1.smithi192.stdout:3/392: mknod d1/d3/c86 0 2024-04-23T02:13:59.241 INFO:tasks.workunit.client.1.smithi192.stdout:0/578: mknod d6/df/d13/d25/d3a/da6/cbf 0 2024-04-23T02:13:59.242 INFO:tasks.workunit.client.1.smithi192.stdout:9/425: dwrite d0/d9/d44/d51/f66 [0,4194304] 0 2024-04-23T02:13:59.244 INFO:tasks.workunit.client.1.smithi192.stdout:8/656: unlink d1/d12/d23/f60 0 2024-04-23T02:13:59.245 INFO:tasks.workunit.client.1.smithi192.stdout:3/393: chown d1/d3/d6/d7/d29/d3b/l77 500 1 2024-04-23T02:13:59.248 INFO:tasks.workunit.client.1.smithi192.stdout:4/634: write de/d15/d2a/d74/d92/da0/dca/d49/d26/d59/f63 [3672407,35594] 0 2024-04-23T02:13:59.251 INFO:tasks.workunit.client.0.smithi071.stdout:4/413: dread d2/d2a/d2d/d39/d42/d49/d67/d3e/f6f [0,4194304] 0 2024-04-23T02:13:59.255 INFO:tasks.workunit.client.1.smithi192.stdout:9/426: mkdir d0/d98 0 2024-04-23T02:13:59.256 INFO:tasks.workunit.client.1.smithi192.stdout:8/657: mknod d1/d9/d3e/ce9 0 2024-04-23T02:13:59.257 INFO:tasks.workunit.client.0.smithi071.stdout:4/414: dread d2/d2a/d2d/d39/d42/f55 [0,4194304] 0 2024-04-23T02:13:59.258 INFO:tasks.workunit.client.1.smithi192.stdout:0/579: rename d6/df/d56/d85/d45/fad to d6/df/fc0 0 2024-04-23T02:13:59.259 INFO:tasks.workunit.client.0.smithi071.stdout:4/415: stat d2/d2a/d2d/d39/d42/d49/d67/d3e/f4d 0 2024-04-23T02:13:59.259 INFO:tasks.workunit.client.1.smithi192.stdout:9/427: symlink d0/d9/d44/d74/l99 0 2024-04-23T02:13:59.261 INFO:tasks.workunit.client.1.smithi192.stdout:8/658: creat d1/d9/fea x:0 0 0 2024-04-23T02:13:59.262 INFO:tasks.workunit.client.0.smithi071.stdout:4/416: rename d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d4f/f70 to d2/d53/f88 0 2024-04-23T02:13:59.263 INFO:tasks.workunit.client.1.smithi192.stdout:4/635: rename de/d15/d2a/d61 to de/d15/d2a/d74/d92/da0/dca/d49/d70/dd8 0 2024-04-23T02:13:59.264 INFO:tasks.workunit.client.0.smithi071.stdout:4/417: write d2/d2a/d2d/f54 [491856,50888] 0 2024-04-23T02:13:59.265 INFO:tasks.workunit.client.1.smithi192.stdout:0/580: mkdir d6/df/d13/d6b/dc1 0 2024-04-23T02:13:59.265 INFO:tasks.workunit.client.0.smithi071.stdout:4/418: dread - d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f59 zero size 2024-04-23T02:13:59.266 INFO:tasks.workunit.client.1.smithi192.stdout:9/428: write d0/f2c [240445,108741] 0 2024-04-23T02:13:59.271 INFO:tasks.workunit.client.1.smithi192.stdout:4/636: truncate de/d15/d19/d1a/fc7 260665 0 2024-04-23T02:13:59.273 INFO:tasks.workunit.client.1.smithi192.stdout:3/394: write d1/f5 [1548694,96398] 0 2024-04-23T02:13:59.274 INFO:tasks.workunit.client.1.smithi192.stdout:0/581: dwrite d6/df/d56/d85/f46 [0,4194304] 0 2024-04-23T02:13:59.274 INFO:tasks.workunit.client.1.smithi192.stdout:9/429: dread d0/d9/d44/d4f/f8 [0,4194304] 0 2024-04-23T02:13:59.282 INFO:tasks.workunit.client.1.smithi192.stdout:8/659: getdents d1/d12 0 2024-04-23T02:13:59.283 INFO:tasks.workunit.client.1.smithi192.stdout:4/637: symlink de/d15/ld9 0 2024-04-23T02:13:59.284 INFO:tasks.workunit.client.1.smithi192.stdout:3/395: getdents d1/d3/d17/d1f/d33/d24/d34/d38/d7b 0 2024-04-23T02:13:59.287 INFO:tasks.workunit.client.1.smithi192.stdout:0/582: dwrite d6/df/d13/d25/d26/f27 [4194304,4194304] 0 2024-04-23T02:13:59.290 INFO:tasks.workunit.client.1.smithi192.stdout:9/430: dwrite d0/d9/d44/d74/d3c/f90 [0,4194304] 0 2024-04-23T02:13:59.291 INFO:tasks.workunit.client.1.smithi192.stdout:9/431: stat d0/d9/d44/c59 0 2024-04-23T02:13:59.294 INFO:tasks.workunit.client.1.smithi192.stdout:0/583: mkdir d6/df/d56/db1/dc2 0 2024-04-23T02:13:59.294 INFO:tasks.workunit.client.1.smithi192.stdout:3/396: truncate d1/d3/d17/f6e 1188430 0 2024-04-23T02:13:59.297 INFO:tasks.workunit.client.1.smithi192.stdout:0/584: unlink d6/df/c11 0 2024-04-23T02:13:59.299 INFO:tasks.workunit.client.1.smithi192.stdout:3/397: rename d1/d3/d17/d1a/c36 to d1/d3/d17/d65/c87 0 2024-04-23T02:13:59.305 INFO:tasks.workunit.client.1.smithi192.stdout:0/585: link d6/df/d13/d6b/cb4 d6/df/d56/d85/cc3 0 2024-04-23T02:13:59.305 INFO:tasks.workunit.client.1.smithi192.stdout:9/432: dwrite d0/d9/d44/d74/d3c/d5f/d7d/f8d [0,4194304] 0 2024-04-23T02:13:59.307 INFO:tasks.workunit.client.1.smithi192.stdout:3/398: rename d1/d3/d17/d1f/d33/d24/d34/d38/f80 to d1/d3/d17/d1f/d33/d24/d34/d6c/d84/d4c/f88 0 2024-04-23T02:13:59.308 INFO:tasks.workunit.client.1.smithi192.stdout:0/586: symlink d6/df/d56/lc4 0 2024-04-23T02:13:59.309 INFO:tasks.workunit.client.1.smithi192.stdout:3/399: chown d1/d3/d6/f10 1 1 2024-04-23T02:13:59.312 INFO:tasks.workunit.client.1.smithi192.stdout:0/587: creat d6/df/d35/d8b/fc5 x:0 0 0 2024-04-23T02:13:59.312 INFO:tasks.workunit.client.1.smithi192.stdout:3/400: mknod d1/d3/d6/d7/d71/c89 0 2024-04-23T02:13:59.318 INFO:tasks.workunit.client.1.smithi192.stdout:3/401: mknod d1/d3/d6/d7/c8a 0 2024-04-23T02:13:59.319 INFO:tasks.workunit.client.1.smithi192.stdout:9/433: dwrite d0/f5c [0,4194304] 0 2024-04-23T02:13:59.321 INFO:tasks.workunit.client.1.smithi192.stdout:9/434: read - d0/d9/f32 zero size 2024-04-23T02:13:59.322 INFO:tasks.workunit.client.1.smithi192.stdout:0/588: dwrite d6/df/d35/d8b/f9b [0,4194304] 0 2024-04-23T02:13:59.323 INFO:tasks.workunit.client.1.smithi192.stdout:9/435: write d0/d9/f26 [4021216,10496] 0 2024-04-23T02:13:59.324 INFO:tasks.workunit.client.1.smithi192.stdout:3/402: truncate d1/d3/d17/d1f/d33/d24/d34/d38/f70 642891 0 2024-04-23T02:13:59.336 INFO:tasks.workunit.client.0.smithi071.stdout:1/352: sync 2024-04-23T02:13:59.336 INFO:tasks.workunit.client.0.smithi071.stdout:5/485: sync 2024-04-23T02:13:59.339 INFO:tasks.workunit.client.0.smithi071.stdout:1/353: creat db/d60/f6b x:0 0 0 2024-04-23T02:13:59.341 INFO:tasks.workunit.client.0.smithi071.stdout:1/354: rename db/d34/l52 to db/d34/d68/l6c 0 2024-04-23T02:13:59.342 INFO:tasks.workunit.client.0.smithi071.stdout:1/355: mknod db/df/d3a/c6d 0 2024-04-23T02:13:59.357 INFO:tasks.workunit.client.0.smithi071.stdout:1/356: dread db/f25 [0,4194304] 0 2024-04-23T02:13:59.358 INFO:tasks.workunit.client.0.smithi071.stdout:1/357: dread - db/df/d13/d20/f5c zero size 2024-04-23T02:13:59.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.355+0000 7f0d8efc3700 1 -- 172.21.15.71:0/3023778726 --> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d88103d60 con 0x7f0d7806ca20 2024-04-23T02:13:59.360 INFO:tasks.workunit.client.0.smithi071.stdout:1/358: creat db/df/d13/d2f/d48/f6e x:0 0 0 2024-04-23T02:13:59.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.366+0000 7f0d85ffb700 1 -- 172.21.15.71:0/3023778726 <== mgr.24351 v2:172.21.15.192:6828/966231696 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0d88103d60 con 0x7f0d7806ca20 2024-04-23T02:13:59.370 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:13:59.370 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (7m) 39s ago 8m 26.3M - 0.25.0 c8568f914cd2 8261e17319fe 2024-04-23T02:13:59.370 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (9m) 39s ago 9m 7700k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:13:59.371 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (8m) 38s ago 8m 7616k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:13:59.371 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (9m) 39s ago 9m 6656k - 18.2.1 5be31c24972a 258512dcbc00 2024-04-23T02:13:59.371 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (8m) 38s ago 8m 6656k - 18.2.1 5be31c24972a feea2c9896a6 2024-04-23T02:13:59.371 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (7m) 39s ago 8m 89.3M - 9.4.7 954c08fa6188 48ea3e02b806 2024-04-23T02:13:59.371 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (3m) 39s ago 3m 74.4M - 18.2.1 5be31c24972a a515de2a841c 2024-04-23T02:13:59.371 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (2m) 39s ago 2m 101M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:13:59.371 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (2m) 38s ago 2m 63.8M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:13:59.371 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (3m) 38s ago 3m 76.6M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:13:59.371 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:9283,8765,8443 running (10m) 39s ago 10m 200M - 18.2.1 5be31c24972a fbe1ad3229f5 2024-04-23T02:13:59.371 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (57s) 38s ago 7m 483M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 151cb2226032 2024-04-23T02:13:59.371 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (10m) 39s ago 10m 53.0M 2048M 18.2.1 5be31c24972a ff40abe63fd9 2024-04-23T02:13:59.371 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (7m) 38s ago 7m 40.3M 2048M 18.2.1 5be31c24972a 9c03e90acc95 2024-04-23T02:13:59.371 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (9m) 39s ago 9m 19.2M - 1.5.0 0da6a335fe13 3f4d0aaee04e 2024-04-23T02:13:59.371 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (7m) 38s ago 7m 16.7M - 1.5.0 0da6a335fe13 cfd0b49d770c 2024-04-23T02:13:59.371 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (7m) 39s ago 7m 72.0M 1227M 18.2.1 5be31c24972a d3d5f2fa7267 2024-04-23T02:13:59.372 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (6m) 39s ago 6m 77.9M 1227M 18.2.1 5be31c24972a 3992157ff0fd 2024-04-23T02:13:59.372 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (6m) 39s ago 6m 65.2M 1227M 18.2.1 5be31c24972a 12380b6a15a2 2024-04-23T02:13:59.372 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (5m) 38s ago 5m 79.5M 2251M 18.2.1 5be31c24972a d5f587ee3a22 2024-04-23T02:13:59.372 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (5m) 38s ago 5m 72.5M 2251M 18.2.1 5be31c24972a 958d046005f0 2024-04-23T02:13:59.372 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (4m) 38s ago 4m 69.2M 2251M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:13:59.372 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (7m) 39s ago 8m 60.7M - 2.43.0 a07b618ecd1d 8df24da82462 2024-04-23T02:13:59.372 INFO:tasks.workunit.client.1.smithi192.stdout:5/468: sync 2024-04-23T02:13:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.368+0000 7f0d8efc3700 1 -- 172.21.15.71:0/3023778726 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f0d7806ca20 msgr2=0x7f0d7806eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.368+0000 7f0d8efc3700 1 --2- 172.21.15.71:0/3023778726 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f0d7806ca20 0x7f0d7806eee0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c006010 tx=0x7f0d7c005ec0 comp rx=0 tx=0).stop 2024-04-23T02:13:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.368+0000 7f0d8efc3700 1 -- 172.21.15.71:0/3023778726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d88074dc0 msgr2=0x7f0d8810c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.368+0000 7f0d8efc3700 1 --2- 172.21.15.71:0/3023778726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d88074dc0 0x7f0d8810c660 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400d900 tx=0x7f0d7400dc10 comp rx=0 tx=0).stop 2024-04-23T02:13:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.368+0000 7f0d8efc3700 1 -- 172.21.15.71:0/3023778726 shutdown_connections 2024-04-23T02:13:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.368+0000 7f0d8efc3700 1 --2- 172.21.15.71:0/3023778726 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f0d7806ca20 0x7f0d7806eee0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:59.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.368+0000 7f0d8efc3700 1 --2- 172.21.15.71:0/3023778726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d880737f0 0x7f0d8810c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:59.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.368+0000 7f0d8efc3700 1 --2- 172.21.15.71:0/3023778726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d88074dc0 0x7f0d8810c660 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:59.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.368+0000 7f0d8efc3700 1 -- 172.21.15.71:0/3023778726 >> 172.21.15.71:0/3023778726 conn(0x7f0d880fc230 msgr2=0x7f0d88102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:13:59.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.368+0000 7f0d8efc3700 1 -- 172.21.15.71:0/3023778726 shutdown_connections 2024-04-23T02:13:59.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.368+0000 7f0d8efc3700 1 -- 172.21.15.71:0/3023778726 wait complete. 2024-04-23T02:13:59.377 INFO:tasks.workunit.client.1.smithi192.stdout:0/589: rmdir d6/df/d13/d25/d95 0 2024-04-23T02:13:59.378 INFO:tasks.workunit.client.1.smithi192.stdout:5/469: chown d2/d6/f8a 1600415 1 2024-04-23T02:13:59.379 INFO:tasks.workunit.client.1.smithi192.stdout:9/436: dwrite d0/d9/d44/d4f/f1c [0,4194304] 0 2024-04-23T02:13:59.380 INFO:tasks.workunit.client.1.smithi192.stdout:0/590: symlink d6/df/d35/d49/lc6 0 2024-04-23T02:13:59.382 INFO:tasks.workunit.client.0.smithi071.stdout:1/359: rename db/d60 to db/df/d5f/d6f 0 2024-04-23T02:13:59.383 INFO:tasks.workunit.client.0.smithi071.stdout:1/360: write f5 [2465135,106291] 0 2024-04-23T02:13:59.386 INFO:tasks.workunit.client.1.smithi192.stdout:3/403: dwrite d1/d3/f7a [0,4194304] 0 2024-04-23T02:13:59.389 INFO:tasks.workunit.client.1.smithi192.stdout:9/437: link d0/d9/d44/d74/d3c/d5f/l97 d0/d9/d44/d74/d3c/d5f/l9a 0 2024-04-23T02:13:59.392 INFO:tasks.workunit.client.0.smithi071.stdout:1/361: dwrite db/df/d13/d20/f32 [0,4194304] 0 2024-04-23T02:13:59.393 INFO:tasks.workunit.client.0.smithi071.stdout:1/362: readlink db/df/d3a/l5a 0 2024-04-23T02:13:59.396 INFO:tasks.workunit.client.0.smithi071.stdout:1/363: creat db/df/d13/d2b/f70 x:0 0 0 2024-04-23T02:13:59.397 INFO:tasks.workunit.client.1.smithi192.stdout:3/404: dwrite d1/d3/d6/f7d [0,4194304] 0 2024-04-23T02:13:59.398 INFO:tasks.workunit.client.0.smithi071.stdout:1/364: write db/df/f39 [1359423,89690] 0 2024-04-23T02:13:59.399 INFO:tasks.workunit.client.1.smithi192.stdout:0/591: truncate d6/df/f30 1532063 0 2024-04-23T02:13:59.399 INFO:tasks.workunit.client.1.smithi192.stdout:9/438: symlink d0/d9/d44/d4f/l9b 0 2024-04-23T02:13:59.402 INFO:tasks.workunit.client.1.smithi192.stdout:9/439: creat d0/d9/d44/d51/d64/f9c x:0 0 0 2024-04-23T02:13:59.403 INFO:tasks.workunit.client.0.smithi071.stdout:1/365: link db/df/d13/d20/f32 db/df/d13/f71 0 2024-04-23T02:13:59.407 INFO:tasks.workunit.client.0.smithi071.stdout:1/366: write db/df/d13/d20/f32 [3749627,86289] 0 2024-04-23T02:13:59.410 INFO:tasks.workunit.client.0.smithi071.stdout:1/367: mkdir db/df/d13/d72 0 2024-04-23T02:13:59.414 INFO:tasks.workunit.client.0.smithi071.stdout:1/368: rename db/f25 to db/df/d13/d2b/f73 0 2024-04-23T02:13:59.421 INFO:tasks.workunit.client.0.smithi071.stdout:1/369: dwrite db/f4b [0,4194304] 0 2024-04-23T02:13:59.424 INFO:tasks.workunit.client.0.smithi071.stdout:1/370: symlink db/df/d13/d2f/l74 0 2024-04-23T02:13:59.425 INFO:tasks.workunit.client.1.smithi192.stdout:0/592: dread d6/f18 [0,4194304] 0 2024-04-23T02:13:59.426 INFO:tasks.workunit.client.1.smithi192.stdout:0/593: chown d6/df/d13/d25/d3a/da6 76 1 2024-04-23T02:13:59.427 INFO:tasks.workunit.client.0.smithi071.stdout:1/371: mkdir db/df/d13/d20/d75 0 2024-04-23T02:13:59.427 INFO:tasks.workunit.client.0.smithi071.stdout:1/372: dread - db/df/d5f/d57/f5e zero size 2024-04-23T02:13:59.429 INFO:tasks.workunit.client.1.smithi192.stdout:0/594: rename d6/df/d13/d25/d3a/d5b to d6/d4b/d9d/dc7 0 2024-04-23T02:13:59.430 INFO:tasks.workunit.client.0.smithi071.stdout:1/373: creat db/d34/d68/f76 x:0 0 0 2024-04-23T02:13:59.432 INFO:tasks.workunit.client.1.smithi192.stdout:0/595: fsync d6/df/d35/d49/f9f 0 2024-04-23T02:13:59.435 INFO:tasks.workunit.client.1.smithi192.stdout:0/596: creat d6/df/d13/d25/d86/da5/fc8 x:0 0 0 2024-04-23T02:13:59.436 INFO:tasks.workunit.client.0.smithi071.stdout:8/437: sync 2024-04-23T02:13:59.437 INFO:tasks.workunit.client.0.smithi071.stdout:3/340: sync 2024-04-23T02:13:59.438 INFO:tasks.workunit.client.0.smithi071.stdout:1/374: truncate db/f22 357909 0 2024-04-23T02:13:59.439 INFO:tasks.workunit.client.0.smithi071.stdout:1/375: truncate db/f54 779946 0 2024-04-23T02:13:59.440 INFO:tasks.workunit.client.0.smithi071.stdout:3/341: getdents d7/d22/d23/d26/d52 0 2024-04-23T02:13:59.440 INFO:tasks.workunit.client.1.smithi192.stdout:0/597: dread d6/df/f50 [0,4194304] 0 2024-04-23T02:13:59.441 INFO:tasks.workunit.client.0.smithi071.stdout:1/376: dread - db/df/d13/d20/f4a zero size 2024-04-23T02:13:59.441 INFO:tasks.workunit.client.0.smithi071.stdout:1/377: dread - db/f6a zero size 2024-04-23T02:13:59.442 INFO:tasks.workunit.client.1.smithi192.stdout:0/598: rmdir d6/d4b/d9d 39 2024-04-23T02:13:59.442 INFO:tasks.workunit.client.0.smithi071.stdout:8/438: link dc/c1b dc/d10/d32/d63/c8a 0 2024-04-23T02:13:59.443 INFO:tasks.workunit.client.0.smithi071.stdout:1/378: creat db/d34/d68/f77 x:0 0 0 2024-04-23T02:13:59.446 INFO:tasks.workunit.client.1.smithi192.stdout:0/599: link d6/f1b d6/df/d35/d49/fc9 0 2024-04-23T02:13:59.447 INFO:tasks.workunit.client.1.smithi192.stdout:0/600: dread - d6/df/d56/db1/fbe zero size 2024-04-23T02:13:59.448 INFO:tasks.workunit.client.0.smithi071.stdout:1/379: creat db/df/d13/d2b/f78 x:0 0 0 2024-04-23T02:13:59.448 INFO:tasks.workunit.client.0.smithi071.stdout:3/342: truncate f0 4390271 0 2024-04-23T02:13:59.449 INFO:tasks.workunit.client.0.smithi071.stdout:1/380: dread - db/df/d13/d20/f4a zero size 2024-04-23T02:13:59.452 INFO:tasks.workunit.client.0.smithi071.stdout:8/439: link dc/d10/d30/c4f dc/d10/c8b 0 2024-04-23T02:13:59.452 INFO:tasks.workunit.client.1.smithi192.stdout:0/601: dwrite d6/df/f4a [0,4194304] 0 2024-04-23T02:13:59.453 INFO:tasks.workunit.client.1.smithi192.stdout:0/602: fsync d6/df/d47/f61 0 2024-04-23T02:13:59.454 INFO:tasks.workunit.client.0.smithi071.stdout:3/343: link d7/d8/fc d7/d10/d64/f70 0 2024-04-23T02:13:59.454 INFO:tasks.workunit.client.0.smithi071.stdout:8/440: write dc/d15/d2d/f49 [976413,11375] 0 2024-04-23T02:13:59.455 INFO:tasks.workunit.client.1.smithi192.stdout:0/603: write d6/df/d13/d25/d86/da5/faf [1518182,15246] 0 2024-04-23T02:13:59.458 INFO:tasks.workunit.client.1.smithi192.stdout:0/604: mknod d6/df/d13/d25/d26/d67/cca 0 2024-04-23T02:13:59.459 INFO:tasks.workunit.client.0.smithi071.stdout:8/441: fdatasync dc/d3a/f3c 0 2024-04-23T02:13:59.460 INFO:tasks.workunit.client.0.smithi071.stdout:1/381: dwrite db/df/f45 [0,4194304] 0 2024-04-23T02:13:59.460 INFO:tasks.workunit.client.1.smithi192.stdout:0/605: unlink d6/df/d47/f63 0 2024-04-23T02:13:59.461 INFO:tasks.workunit.client.0.smithi071.stdout:1/382: stat db/df/d5f 0 2024-04-23T02:13:59.462 INFO:tasks.workunit.client.1.smithi192.stdout:0/606: write d6/df/d13/d25/f91 [4800019,46188] 0 2024-04-23T02:13:59.463 INFO:tasks.workunit.client.0.smithi071.stdout:1/383: readlink db/d34/l38 0 2024-04-23T02:13:59.463 INFO:tasks.workunit.client.0.smithi071.stdout:3/344: write d7/d8/d42/d5f/f5b [139097,125741] 0 2024-04-23T02:13:59.466 INFO:tasks.workunit.client.0.smithi071.stdout:1/384: truncate db/df/d13/d20/f4a 1010688 0 2024-04-23T02:13:59.469 INFO:tasks.workunit.client.0.smithi071.stdout:8/442: dwrite f6 [0,4194304] 0 2024-04-23T02:13:59.470 INFO:tasks.workunit.client.0.smithi071.stdout:8/443: readlink dc/l1f 0 2024-04-23T02:13:59.471 INFO:tasks.workunit.client.0.smithi071.stdout:3/345: rename d7/d8/lf to d7/d8/d42/l71 0 2024-04-23T02:13:59.471 INFO:tasks.workunit.client.0.smithi071.stdout:1/385: truncate db/df/d13/d2b/f78 817282 0 2024-04-23T02:13:59.472 INFO:tasks.workunit.client.0.smithi071.stdout:3/346: write d7/d1d/f6c [759278,80691] 0 2024-04-23T02:13:59.478 INFO:tasks.workunit.client.0.smithi071.stdout:3/347: chown d7/d8/fe 31 1 2024-04-23T02:13:59.480 INFO:tasks.workunit.client.0.smithi071.stdout:8/444: dwrite dc/d10/f27 [4194304,4194304] 0 2024-04-23T02:13:59.481 INFO:tasks.workunit.client.0.smithi071.stdout:1/386: link db/df/d5f/d6f/f6b db/df/d13/d72/f79 0 2024-04-23T02:13:59.483 INFO:tasks.workunit.client.0.smithi071.stdout:1/387: write db/fd [2572471,38136] 0 2024-04-23T02:13:59.485 INFO:tasks.workunit.client.0.smithi071.stdout:3/348: rename d7/d22/f25 to d7/d1d/f72 0 2024-04-23T02:13:59.485 INFO:tasks.workunit.client.0.smithi071.stdout:1/388: write db/df/d13/d20/f32 [1565500,46706] 0 2024-04-23T02:13:59.493 INFO:tasks.workunit.client.0.smithi071.stdout:8/445: getdents dc/d10 0 2024-04-23T02:13:59.497 INFO:tasks.workunit.client.0.smithi071.stdout:8/446: mkdir dc/d15/d5d/d8c 0 2024-04-23T02:13:59.498 INFO:tasks.workunit.client.0.smithi071.stdout:8/447: rename dc/d15/d5d/d8c to dc/d10/d8d 0 2024-04-23T02:13:59.502 INFO:tasks.workunit.client.0.smithi071.stdout:1/389: write db/df/d13/f1b [3927318,112243] 0 2024-04-23T02:13:59.504 INFO:tasks.workunit.client.0.smithi071.stdout:3/349: dread d7/d8/f1b [0,4194304] 0 2024-04-23T02:13:59.508 INFO:tasks.workunit.client.0.smithi071.stdout:8/448: fsync dc/d10/d30/f66 0 2024-04-23T02:13:59.508 INFO:tasks.workunit.client.0.smithi071.stdout:3/350: rename d7/d1d/f6c to d7/d1d/f73 0 2024-04-23T02:13:59.511 INFO:tasks.workunit.client.1.smithi192.stdout:7/461: sync 2024-04-23T02:13:59.511 INFO:tasks.workunit.client.1.smithi192.stdout:1/278: sync 2024-04-23T02:13:59.512 INFO:tasks.workunit.client.0.smithi071.stdout:1/390: dwrite db/df/d13/d2f/d48/f6e [0,4194304] 0 2024-04-23T02:13:59.512 INFO:tasks.workunit.client.1.smithi192.stdout:7/462: dread - d5/d43/d4c/d5c/d67/f82 zero size 2024-04-23T02:13:59.512 INFO:tasks.workunit.client.0.smithi071.stdout:8/449: write dc/d10/f14 [4571298,57394] 0 2024-04-23T02:13:59.513 INFO:tasks.workunit.client.0.smithi071.stdout:3/351: truncate d7/d8/d42/d5f/d68/f62 823409 0 2024-04-23T02:13:59.513 INFO:tasks.workunit.client.0.smithi071.stdout:1/391: chown db/d34/d68 47 1 2024-04-23T02:13:59.514 INFO:tasks.workunit.client.1.smithi192.stdout:1/279: rename c5 to dc/c4f 0 2024-04-23T02:13:59.515 INFO:tasks.workunit.client.0.smithi071.stdout:8/450: dread - dc/d10/f85 zero size 2024-04-23T02:13:59.515 INFO:tasks.workunit.client.0.smithi071.stdout:1/392: dread - db/df/d3a/f44 zero size 2024-04-23T02:13:59.515 INFO:tasks.workunit.client.1.smithi192.stdout:7/463: read d5/d17/d1c/d4a/f58 [112029,31686] 0 2024-04-23T02:13:59.518 INFO:tasks.workunit.client.0.smithi071.stdout:3/352: mkdir d7/d8/d42/d5f/d68/d3b/d4c/d74 0 2024-04-23T02:13:59.518 INFO:tasks.workunit.client.0.smithi071.stdout:7/388: sync 2024-04-23T02:13:59.518 INFO:tasks.workunit.client.0.smithi071.stdout:2/336: sync 2024-04-23T02:13:59.519 INFO:tasks.workunit.client.0.smithi071.stdout:0/417: sync 2024-04-23T02:13:59.519 INFO:tasks.workunit.client.0.smithi071.stdout:6/322: sync 2024-04-23T02:13:59.519 INFO:tasks.workunit.client.0.smithi071.stdout:5/486: sync 2024-04-23T02:13:59.519 INFO:tasks.workunit.client.0.smithi071.stdout:9/372: sync 2024-04-23T02:13:59.519 INFO:tasks.workunit.client.0.smithi071.stdout:4/419: sync 2024-04-23T02:13:59.519 INFO:tasks.workunit.client.0.smithi071.stdout:8/451: mknod dc/d3a/c8e 0 2024-04-23T02:13:59.520 INFO:tasks.workunit.client.0.smithi071.stdout:1/393: creat db/d11/f7a x:0 0 0 2024-04-23T02:13:59.522 INFO:tasks.workunit.client.0.smithi071.stdout:7/389: dread - d3/d51/f7f zero size 2024-04-23T02:13:59.523 INFO:tasks.workunit.client.0.smithi071.stdout:2/337: chown d0/l1b 843254394 1 2024-04-23T02:13:59.525 INFO:tasks.workunit.client.0.smithi071.stdout:9/373: write d1/d32/f40 [1553478,12522] 0 2024-04-23T02:13:59.525 INFO:tasks.workunit.client.0.smithi071.stdout:6/323: symlink d6/d3e/d3f/l66 0 2024-04-23T02:13:59.525 INFO:tasks.workunit.client.0.smithi071.stdout:1/394: write db/d11/f64 [789395,60553] 0 2024-04-23T02:13:59.526 INFO:tasks.workunit.client.1.smithi192.stdout:7/464: dwrite d5/d17/d1c/d59/d5a/d41/f54 [0,4194304] 0 2024-04-23T02:13:59.528 INFO:tasks.workunit.client.0.smithi071.stdout:8/452: creat dc/d78/f8f x:0 0 0 2024-04-23T02:13:59.529 INFO:tasks.workunit.client.0.smithi071.stdout:0/418: symlink d0/d3/de/d10/d14/d32/d44/l80 0 2024-04-23T02:13:59.529 INFO:tasks.workunit.client.0.smithi071.stdout:7/390: fdatasync d3/d17/d1c/d31/f6a 0 2024-04-23T02:13:59.530 INFO:tasks.workunit.client.1.smithi192.stdout:7/465: creat d5/d17/d1c/d26/f86 x:0 0 0 2024-04-23T02:13:59.531 INFO:tasks.workunit.client.0.smithi071.stdout:2/338: chown d0/d2/c47 731 1 2024-04-23T02:13:59.531 INFO:tasks.workunit.client.0.smithi071.stdout:6/324: creat d6/d1f/d5d/f67 x:0 0 0 2024-04-23T02:13:59.531 INFO:tasks.workunit.client.0.smithi071.stdout:9/374: symlink d1/d32/d3a/l7c 0 2024-04-23T02:13:59.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.528+0000 7f03fbeca700 1 -- 172.21.15.71:0/3849343548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03f40ff590 msgr2=0x7f03f41019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:59.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.528+0000 7f03fbeca700 1 --2- 172.21.15.71:0/3849343548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03f40ff590 0x7f03f41019c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009910 tx=0x7f03f0009c20 comp rx=0 tx=0).stop 2024-04-23T02:13:59.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.528+0000 7f03fbeca700 1 -- 172.21.15.71:0/3849343548 shutdown_connections 2024-04-23T02:13:59.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.528+0000 7f03fbeca700 1 --2- 172.21.15.71:0/3849343548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03f4101f00 0x7f03f4104390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:59.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.528+0000 7f03fbeca700 1 --2- 172.21.15.71:0/3849343548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03f40ff590 0x7f03f41019c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:59.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.528+0000 7f03fbeca700 1 -- 172.21.15.71:0/3849343548 >> 172.21.15.71:0/3849343548 conn(0x7f03f40fb190 msgr2=0x7f03f40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:13:59.533 INFO:tasks.workunit.client.0.smithi071.stdout:1/395: write db/df/d13/d2b/f35 [4829311,16590] 0 2024-04-23T02:13:59.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.528+0000 7f03fbeca700 1 -- 172.21.15.71:0/3849343548 shutdown_connections 2024-04-23T02:13:59.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.528+0000 7f03fbeca700 1 -- 172.21.15.71:0/3849343548 wait complete. 2024-04-23T02:13:59.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.529+0000 7f03fbeca700 1 Processor -- start 2024-04-23T02:13:59.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.529+0000 7f03fbeca700 1 -- start start 2024-04-23T02:13:59.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.529+0000 7f03fbeca700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03f40ff590 0x7f03f4071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:59.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.529+0000 7f03fbeca700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03f4101f00 0x7f03f40723f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:59.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.529+0000 7f03fbeca700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f4072a40 con 0x7f03f4101f00 2024-04-23T02:13:59.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.529+0000 7f03fbeca700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f41aa820 con 0x7f03f40ff590 2024-04-23T02:13:59.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.530+0000 7f03f9c66700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03f40ff590 0x7f03f4071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:59.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.530+0000 7f03f9c66700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03f40ff590 0x7f03f4071eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.71:37738/0 (socket says 172.21.15.71:37738) 2024-04-23T02:13:59.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.530+0000 7f03f9c66700 1 -- 172.21.15.71:0/1409135634 learned_addr learned my addr 172.21.15.71:0/1409135634 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:13:59.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.530+0000 7f03f9465700 1 --2- 172.21.15.71:0/1409135634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03f4101f00 0x7f03f40723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:59.535 INFO:tasks.workunit.client.1.smithi192.stdout:1/280: dread dc/f10 [0,4194304] 0 2024-04-23T02:13:59.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.531+0000 7f03f9465700 1 -- 172.21.15.71:0/1409135634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03f40ff590 msgr2=0x7f03f4071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:13:59.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.531+0000 7f03f9465700 1 --2- 172.21.15.71:0/1409135634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03f40ff590 0x7f03f4071eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:13:59.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.531+0000 7f03f9465700 1 -- 172.21.15.71:0/1409135634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f00095f0 con 0x7f03f4101f00 2024-04-23T02:13:59.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.531+0000 7f03f9465700 1 --2- 172.21.15.71:0/1409135634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03f4101f00 0x7f03f40723f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f03e400b3d0 tx=0x7f03e400b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:13:59.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.532+0000 7f03eaffd700 1 -- 172.21.15.71:0/1409135634 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03e4011800 con 0x7f03f4101f00 2024-04-23T02:13:59.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.532+0000 7f03eaffd700 1 -- 172.21.15.71:0/1409135634 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f03e400be90 con 0x7f03f4101f00 2024-04-23T02:13:59.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.532+0000 7f03eaffd700 1 -- 172.21.15.71:0/1409135634 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03e4010db0 con 0x7f03f4101f00 2024-04-23T02:13:59.537 INFO:tasks.workunit.client.1.smithi192.stdout:1/281: chown dc/d16/d19/d30/d32/d34/f3e 495018 1 2024-04-23T02:13:59.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.532+0000 7f03fbeca700 1 -- 172.21.15.71:0/1409135634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f4105c70 con 0x7f03f4101f00 2024-04-23T02:13:59.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.532+0000 7f03fbeca700 1 -- 172.21.15.71:0/1409135634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f41061c0 con 0x7f03f4101f00 2024-04-23T02:13:59.539 INFO:tasks.workunit.client.1.smithi192.stdout:1/282: link f3 dc/d14/d23/f50 0 2024-04-23T02:13:59.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.533+0000 7f03fbeca700 1 -- 172.21.15.71:0/1409135634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f410fd30 con 0x7f03f4101f00 2024-04-23T02:13:59.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.535+0000 7f03eaffd700 1 -- 172.21.15.71:0/1409135634 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) v1 ==== 90681+0+0 (secure 0 0 0) 0x7f03e4011960 con 0x7f03f4101f00 2024-04-23T02:13:59.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.535+0000 7f03eaffd700 1 --2- 172.21.15.71:0/1409135634 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f03e006cb20 0x7f03e006efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:13:59.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.535+0000 7f03eaffd700 1 -- 172.21.15.71:0/1409135634 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f03e408b9a0 con 0x7f03f4101f00 2024-04-23T02:13:59.540 INFO:tasks.workunit.client.0.smithi071.stdout:4/420: link d2/d2a/f80 d2/f89 0 2024-04-23T02:13:59.541 INFO:tasks.workunit.client.1.smithi192.stdout:1/283: mkdir dc/d51 0 2024-04-23T02:13:59.541 INFO:tasks.workunit.client.0.smithi071.stdout:8/453: symlink dc/d15/d50/d5f/l90 0 2024-04-23T02:13:59.541 INFO:tasks.workunit.client.0.smithi071.stdout:0/419: write d0/d3/de/d10/d14/d32/d37/f7b [1010511,93374] 0 2024-04-23T02:13:59.542 INFO:tasks.workunit.client.0.smithi071.stdout:1/396: write db/d11/f1d [3659161,55226] 0 2024-04-23T02:13:59.542 INFO:tasks.workunit.client.1.smithi192.stdout:1/284: write dc/d14/d23/f3d [807382,50275] 0 2024-04-23T02:13:59.543 INFO:tasks.workunit.client.0.smithi071.stdout:5/487: truncate d3/da/d1a/d1c/f2e 1346689 0 2024-04-23T02:13:59.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.536+0000 7f03f9c66700 1 --2- 172.21.15.71:0/1409135634 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f03e006cb20 0x7f03e006efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:13:59.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.536+0000 7f03f9c66700 1 --2- 172.21.15.71:0/1409135634 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f03e006cb20 0x7f03e006efe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f03f000bfa0 tx=0x7f03f000b560 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:13:59.547 INFO:tasks.workunit.client.0.smithi071.stdout:8/454: creat dc/d78/f91 x:0 0 0 2024-04-23T02:13:59.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:13:59.544+0000 7f03eaffd700 1 -- 172.21.15.71:0/1409135634 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f03e40595f0 con 0x7f03f4101f00 2024-04-23T02:13:59.551 INFO:tasks.workunit.client.0.smithi071.stdout:9/375: chown d1/d1a/d39/d42/c65 1 1 2024-04-23T02:13:59.551 INFO:tasks.workunit.client.0.smithi071.stdout:5/488: chown d3/da/d1a/d1c/l3a 105 1 2024-04-23T02:13:59.551 INFO:tasks.workunit.client.0.smithi071.stdout:1/397: read f3 [3568756,129233] 0 2024-04-23T02:13:59.551 INFO:tasks.workunit.client.0.smithi071.stdout:1/398: stat db/df/d13/d2b/l69 0 2024-04-23T02:13:59.551 INFO:tasks.workunit.client.0.smithi071.stdout:2/339: dread d0/d2/d26/d14/d1f/d4f/f66 [0,4194304] 0 2024-04-23T02:13:59.552 INFO:tasks.workunit.client.1.smithi192.stdout:1/285: dwrite dc/d16/d19/d30/d32/f4d [0,4194304] 0 2024-04-23T02:13:59.555 INFO:tasks.workunit.client.1.smithi192.stdout:1/286: unlink dc/d14/d23/l46 0 2024-04-23T02:13:59.556 INFO:tasks.workunit.client.1.smithi192.stdout:7/466: dwrite d5/d17/d1c/f2d [4194304,4194304] 0 2024-04-23T02:13:59.557 INFO:tasks.workunit.client.0.smithi071.stdout:3/353: dread d7/d10/f15 [0,4194304] 0 2024-04-23T02:13:59.558 INFO:tasks.workunit.client.0.smithi071.stdout:6/325: chown d6/d22/f35 214089 1 2024-04-23T02:13:59.558 INFO:tasks.workunit.client.0.smithi071.stdout:5/489: read d3/d1d/daf/d10/f48 [152423,31102] 0 2024-04-23T02:13:59.559 INFO:tasks.workunit.client.1.smithi192.stdout:7/467: write d5/d17/d1c/d57/f22 [5210685,99175] 0 2024-04-23T02:13:59.559 INFO:tasks.workunit.client.0.smithi071.stdout:1/399: mkdir db/df/d13/d2f/d7b 0 2024-04-23T02:13:59.559 INFO:tasks.workunit.client.0.smithi071.stdout:4/421: dwrite d2/d2a/f36 [0,4194304] 0 2024-04-23T02:13:59.561 INFO:tasks.workunit.client.1.smithi192.stdout:7/468: mknod d5/d17/c87 0 2024-04-23T02:13:59.565 INFO:tasks.workunit.client.0.smithi071.stdout:2/340: write d0/d2/d26/f18 [2047757,71475] 0 2024-04-23T02:13:59.565 INFO:tasks.workunit.client.0.smithi071.stdout:9/376: mknod d1/d1a/d71/c7d 0 2024-04-23T02:13:59.565 INFO:tasks.workunit.client.0.smithi071.stdout:3/354: write d7/d8/d42/d5f/d68/f62 [1281385,75348] 0 2024-04-23T02:13:59.566 INFO:tasks.workunit.client.0.smithi071.stdout:4/422: readlink d2/d2a/d2d/l74 0 2024-04-23T02:13:59.567 INFO:tasks.workunit.client.1.smithi192.stdout:7/469: creat d5/d17/d1c/d59/d5a/d70/f88 x:0 0 0 2024-04-23T02:13:59.567 INFO:tasks.workunit.client.1.smithi192.stdout:1/287: dwrite dc/d16/d19/d30/d32/d34/f4c [0,4194304] 0 2024-04-23T02:13:59.568 INFO:tasks.workunit.client.0.smithi071.stdout:1/400: write db/df/d13/d20/f5c [513170,81298] 0 2024-04-23T02:13:59.568 INFO:tasks.workunit.client.0.smithi071.stdout:0/420: getdents d0/d3/de/d10/d14/d32/d44/d31/d52 0 2024-04-23T02:13:59.569 INFO:tasks.workunit.client.0.smithi071.stdout:7/391: dwrite d3/d17/d1c/d2a/d43/f4e [0,4194304] 0 2024-04-23T02:13:59.572 INFO:tasks.workunit.client.0.smithi071.stdout:8/455: link dc/d15/f86 dc/d10/d8d/f92 0 2024-04-23T02:13:59.574 INFO:tasks.workunit.client.1.smithi192.stdout:1/288: chown dc/d14/d15/d35/f40 4764 1 2024-04-23T02:13:59.575 INFO:tasks.workunit.client.1.smithi192.stdout:7/470: mknod d5/d43/d4c/d4e/c89 0 2024-04-23T02:13:59.576 INFO:tasks.workunit.client.0.smithi071.stdout:9/377: mkdir d1/d32/d7e 0 2024-04-23T02:13:59.577 INFO:tasks.workunit.client.0.smithi071.stdout:5/490: symlink d3/d1d/lbf 0 2024-04-23T02:13:59.578 INFO:tasks.workunit.client.1.smithi192.stdout:7/471: write d5/d17/d1c/d59/d5a/d41/d50/f35 [4049404,31773] 0 2024-04-23T02:13:59.580 INFO:tasks.workunit.client.1.smithi192.stdout:7/472: write d5/d17/d1c/d59/d5a/d41/d50/f62 [645188,32813] 0 2024-04-23T02:13:59.581 INFO:tasks.workunit.client.1.smithi192.stdout:1/289: rename dc/d16/d19/d30/d32/d34/f4c to dc/d16/d19/d30/d32/d44/f52 0 2024-04-23T02:13:59.586 INFO:tasks.workunit.client.1.smithi192.stdout:1/290: creat dc/d16/d19/d30/d32/d44/f53 x:0 0 0 2024-04-23T02:13:59.586 INFO:tasks.workunit.client.0.smithi071.stdout:6/326: creat d6/d1f/f68 x:0 0 0 2024-04-23T02:13:59.586 INFO:tasks.workunit.client.0.smithi071.stdout:2/341: dwrite d0/d2/d45/f5a [0,4194304] 0 2024-04-23T02:13:59.586 INFO:tasks.workunit.client.0.smithi071.stdout:5/491: chown d3/da/d1a/d1c/f62 6441962 1 2024-04-23T02:13:59.588 INFO:tasks.workunit.client.0.smithi071.stdout:3/355: write d7/d10/f15 [3956616,116510] 0 2024-04-23T02:13:59.589 INFO:tasks.workunit.client.1.smithi192.stdout:1/291: fdatasync dc/d14/d23/f50 0 2024-04-23T02:13:59.591 INFO:tasks.workunit.client.1.smithi192.stdout:1/292: write dc/d16/d19/d30/d32/f36 [4733973,67019] 0 2024-04-23T02:13:59.592 INFO:tasks.workunit.client.0.smithi071.stdout:8/456: chown dc/d56/f6a 0 1 2024-04-23T02:13:59.593 INFO:tasks.workunit.client.0.smithi071.stdout:1/401: dwrite db/df/d5f/d57/f5e [0,4194304] 0 2024-04-23T02:13:59.596 INFO:tasks.workunit.client.0.smithi071.stdout:0/421: rmdir d0/d3/de/d10/d14/d32/d37 39 2024-04-23T02:13:59.604 INFO:tasks.workunit.client.0.smithi071.stdout:2/342: symlink d0/d2/d51/l6f 0 2024-04-23T02:13:59.604 INFO:tasks.workunit.client.0.smithi071.stdout:6/327: dread - d6/d1f/f32 zero size 2024-04-23T02:13:59.604 INFO:tasks.workunit.client.0.smithi071.stdout:3/356: mkdir d7/d22/d23/d26/d75 0 2024-04-23T02:13:59.605 INFO:tasks.workunit.client.0.smithi071.stdout:7/392: truncate d3/d17/d1c/f45 6764278 0 2024-04-23T02:13:59.605 INFO:tasks.workunit.client.0.smithi071.stdout:6/328: chown d6/d1f/d4e 76370 1 2024-04-23T02:13:59.607 INFO:tasks.workunit.client.0.smithi071.stdout:4/423: creat d2/d2a/d73/f8a x:0 0 0 2024-04-23T02:13:59.611 INFO:tasks.workunit.client.0.smithi071.stdout:2/343: mkdir d0/d70 0 2024-04-23T02:13:59.611 INFO:tasks.workunit.client.0.smithi071.stdout:3/357: chown d7/d22/d23/l3d 740683 1 2024-04-23T02:13:59.613 INFO:tasks.workunit.client.1.smithi192.stdout:7/473: truncate d5/d17/d1c/f3e 1161724 0 2024-04-23T02:13:59.614 INFO:tasks.workunit.client.0.smithi071.stdout:7/393: dread - d3/d17/d41/d34/d39/d3b/f84 zero size 2024-04-23T02:13:59.616 INFO:tasks.workunit.client.0.smithi071.stdout:5/492: getdents d3 0 2024-04-23T02:13:59.617 INFO:tasks.workunit.client.1.smithi192.stdout:7/474: dread d5/d17/d1c/d59/d5a/d70/f7e [0,4194304] 0 2024-04-23T02:13:59.620 INFO:tasks.workunit.client.0.smithi071.stdout:2/344: rmdir d0/d2/d3c 39 2024-04-23T02:13:59.621 INFO:tasks.workunit.client.1.smithi192.stdout:7/475: dread d5/d17/d1c/d4a/f58 [0,4194304] 0 2024-04-23T02:13:59.622 INFO:tasks.workunit.client.1.smithi192.stdout:7/476: write d5/d17/d1c/d59/d5a/d41/d50/f69 [1269085,124672] 0 2024-04-23T02:13:59.623 INFO:tasks.workunit.client.0.smithi071.stdout:8/457: link dc/d10/d32/c71 dc/d78/c93 0 2024-04-23T02:13:59.624 INFO:tasks.workunit.client.0.smithi071.stdout:7/394: mkdir d3/d17/d1c/d2a/d43/d86 0 2024-04-23T02:13:59.628 INFO:tasks.workunit.client.0.smithi071.stdout:5/493: mkdir d3/d1d/daf/d10/da5/d5f/dc0 0 2024-04-23T02:13:59.633 INFO:tasks.workunit.client.0.smithi071.stdout:3/358: dwrite d7/d10/d38/f43 [0,4194304] 0 2024-04-23T02:13:59.638 INFO:tasks.workunit.client.0.smithi071.stdout:0/422: write d0/d3/f2d [1974768,46288] 0 2024-04-23T02:13:59.638 INFO:tasks.workunit.client.0.smithi071.stdout:4/424: dwrite d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d10/f87 [0,4194304] 0 2024-04-23T02:13:59.640 INFO:tasks.workunit.client.0.smithi071.stdout:8/458: creat dc/d10/d55/f94 x:0 0 0 2024-04-23T02:13:59.642 INFO:tasks.workunit.client.0.smithi071.stdout:7/395: chown d3/d17/d1c/d31/f56 703743 1 2024-04-23T02:13:59.645 INFO:tasks.workunit.client.0.smithi071.stdout:3/359: creat d7/d8/d42/d5f/d68/d3b/f76 x:0 0 0 2024-04-23T02:13:59.648 INFO:tasks.workunit.client.0.smithi071.stdout:9/378: dread d1/d1a/d5a/f5e [4194304,4194304] 0 2024-04-23T02:13:59.651 INFO:tasks.workunit.client.0.smithi071.stdout:3/360: read - d7/d8/f61 zero size 2024-04-23T02:13:59.651 INFO:tasks.workunit.client.0.smithi071.stdout:8/459: mkdir dc/d15/d72/d95 0 2024-04-23T02:13:59.651 INFO:tasks.workunit.client.0.smithi071.stdout:7/396: read d3/d17/d41/f66 [384618,65958] 0 2024-04-23T02:13:59.655 INFO:tasks.workunit.client.0.smithi071.stdout:9/379: readlink d1/d4e/d54/l64 0 2024-04-23T02:13:59.655 INFO:tasks.workunit.client.0.smithi071.stdout:0/423: read - d0/d3/de/d10/d14/d32/d37/d26/f2e zero size 2024-04-23T02:13:59.655 INFO:tasks.workunit.client.0.smithi071.stdout:4/425: read - d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f71 zero size 2024-04-23T02:13:59.657 INFO:tasks.workunit.client.0.smithi071.stdout:6/329: dread d6/f16 [0,4194304] 0 2024-04-23T02:13:59.663 INFO:tasks.workunit.client.0.smithi071.stdout:3/361: readlink d7/d8/d42/d5f/l58 0 2024-04-23T02:13:59.663 INFO:tasks.workunit.client.0.smithi071.stdout:8/460: read f9 [2280721,45880] 0 2024-04-23T02:13:59.663 INFO:tasks.workunit.client.0.smithi071.stdout:2/345: dwrite d0/d2/d3c/f5f [0,4194304] 0 2024-04-23T02:13:59.663 INFO:tasks.workunit.client.0.smithi071.stdout:6/330: mknod d6/d1f/d4e/c69 0 2024-04-23T02:13:59.666 INFO:tasks.workunit.client.0.smithi071.stdout:5/494: dwrite d3/d34/d6b/d81/d5a/da7/da8/fbb [0,4194304] 0 2024-04-23T02:13:59.670 INFO:tasks.workunit.client.0.smithi071.stdout:5/495: symlink d3/d8f/lc1 0 2024-04-23T02:13:59.674 INFO:tasks.workunit.client.0.smithi071.stdout:0/424: mknod d0/d3/de/d48/d7e/c81 0 2024-04-23T02:13:59.677 INFO:tasks.workunit.client.0.smithi071.stdout:4/426: mkdir d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d4f/d8b 0 2024-04-23T02:13:59.680 INFO:tasks.workunit.client.0.smithi071.stdout:3/362: symlink d7/l77 0 2024-04-23T02:13:59.684 INFO:tasks.workunit.client.0.smithi071.stdout:6/331: unlink d6/d22/f37 0 2024-04-23T02:13:59.686 INFO:tasks.workunit.client.0.smithi071.stdout:5/496: mknod d3/d34/d6b/d81/d5a/da7/da8/cc2 0 2024-04-23T02:13:59.686 INFO:tasks.workunit.client.0.smithi071.stdout:7/397: write d3/d17/d1c/f45 [6717930,94985] 0 2024-04-23T02:13:59.687 INFO:tasks.workunit.client.0.smithi071.stdout:9/380: dwrite d1/d1a/d39/d42/f49 [0,4194304] 0 2024-04-23T02:13:59.690 INFO:tasks.workunit.client.0.smithi071.stdout:6/332: symlink d6/d3e/d3f/l6a 0 2024-04-23T02:13:59.692 INFO:tasks.workunit.client.0.smithi071.stdout:4/427: chown d2/d2a/d2d/d39/d42/d49/f77 105481560 1 2024-04-23T02:13:59.694 INFO:tasks.workunit.client.1.smithi192.stdout:6/556: sync 2024-04-23T02:13:59.694 INFO:tasks.workunit.client.1.smithi192.stdout:2/410: sync 2024-04-23T02:13:59.695 INFO:tasks.workunit.client.0.smithi071.stdout:5/497: fdatasync d3/da/d1a/d1c/f41 0 2024-04-23T02:13:59.696 INFO:tasks.workunit.client.0.smithi071.stdout:0/425: creat d0/d3/de/d10/d14/d32/f82 x:0 0 0 2024-04-23T02:13:59.697 INFO:tasks.workunit.client.0.smithi071.stdout:8/461: dwrite dc/d15/d50/f82 [0,4194304] 0 2024-04-23T02:13:59.697 INFO:tasks.workunit.client.0.smithi071.stdout:7/398: mkdir d3/d51/d87 0 2024-04-23T02:13:59.699 INFO:tasks.workunit.client.1.smithi192.stdout:2/411: link f7 d8/d26/d27/d53/f82 0 2024-04-23T02:13:59.700 INFO:tasks.workunit.client.1.smithi192.stdout:6/557: creat d3/d15/d1b/fb4 x:0 0 0 2024-04-23T02:13:59.700 INFO:tasks.workunit.client.1.smithi192.stdout:2/412: dread - d8/d26/d29/d43/f7f zero size 2024-04-23T02:13:59.700 INFO:tasks.workunit.client.0.smithi071.stdout:2/346: dwrite d0/d2/f4d [0,4194304] 0 2024-04-23T02:13:59.700 INFO:tasks.workunit.client.0.smithi071.stdout:7/399: fsync d3/d17/d1c/d31/f81 0 2024-04-23T02:13:59.704 INFO:tasks.workunit.client.1.smithi192.stdout:2/413: unlink d8/d26/d29/d43/d50/d5e/l6d 0 2024-04-23T02:13:59.706 INFO:tasks.workunit.client.0.smithi071.stdout:0/426: write d0/d63/f66 [1019902,18310] 0 2024-04-23T02:13:59.708 INFO:tasks.workunit.client.0.smithi071.stdout:4/428: dread - d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d29/f83 zero size 2024-04-23T02:13:59.708 INFO:tasks.workunit.client.1.smithi192.stdout:6/558: dwrite d3/d18/f94 [0,4194304] 0 2024-04-23T02:13:59.710 INFO:tasks.workunit.client.0.smithi071.stdout:6/333: mknod d6/d1f/d5d/c6b 0 2024-04-23T02:13:59.716 INFO:tasks.workunit.client.0.smithi071.stdout:5/498: creat d3/da/d1a/d1c/d46/d4e/fc3 x:0 0 0 2024-04-23T02:13:59.716 INFO:tasks.workunit.client.0.smithi071.stdout:0/427: symlink d0/d63/l83 0 2024-04-23T02:13:59.716 INFO:tasks.workunit.client.1.smithi192.stdout:2/414: dwrite d8/d14/f4c [0,4194304] 0 2024-04-23T02:13:59.720 INFO:tasks.workunit.client.1.smithi192.stdout:6/559: dwrite d3/d18/f6f [0,4194304] 0 2024-04-23T02:13:59.722 INFO:tasks.workunit.client.1.smithi192.stdout:6/560: stat d3/d15/d1b/la8 0 2024-04-23T02:13:59.723 INFO:tasks.workunit.client.0.smithi071.stdout:4/429: creat d2/d2a/d2d/d7b/f8c x:0 0 0 2024-04-23T02:13:59.723 INFO:tasks.workunit.client.0.smithi071.stdout:5/499: dread - d3/d1d/d43/f95 zero size 2024-04-23T02:13:59.723 INFO:tasks.workunit.client.0.smithi071.stdout:7/400: unlink d3/d17/d41/d34/d39/f54 0 2024-04-23T02:13:59.724 INFO:tasks.workunit.client.0.smithi071.stdout:9/381: rename d1/d1a/c1f to d1/c7f 0 2024-04-23T02:13:59.724 INFO:tasks.workunit.client.1.smithi192.stdout:6/561: mknod d3/d18/d58/cb5 0 2024-04-23T02:13:59.727 INFO:tasks.workunit.client.1.smithi192.stdout:6/562: rmdir d3/d15/d1b/d21 39 2024-04-23T02:13:59.728 INFO:tasks.workunit.client.1.smithi192.stdout:6/563: truncate d3/d18/f8a 23127 0 2024-04-23T02:13:59.731 INFO:tasks.workunit.client.1.smithi192.stdout:6/564: creat d3/d15/fb6 x:0 0 0 2024-04-23T02:13:59.732 INFO:tasks.workunit.client.0.smithi071.stdout:9/382: readlink d1/d32/l59 0 2024-04-23T02:13:59.732 INFO:tasks.workunit.client.0.smithi071.stdout:3/363: rename d7/d10/d64/f70 to d7/d22/d23/d26/f78 0 2024-04-23T02:13:59.737 INFO:tasks.workunit.client.0.smithi071.stdout:6/334: link d6/c10 d6/d1f/d5d/c6c 0 2024-04-23T02:13:59.737 INFO:tasks.workunit.client.0.smithi071.stdout:0/428: mkdir d0/d3/de/d10/d14/d84 0 2024-04-23T02:13:59.740 INFO:tasks.workunit.client.0.smithi071.stdout:0/429: write d0/d3/f3f [114633,106456] 0 2024-04-23T02:13:59.747 INFO:tasks.workunit.client.0.smithi071.stdout:4/430: symlink d2/d2a/d2d/d39/d42/d49/d65/l8d 0 2024-04-23T02:13:59.747 INFO:tasks.workunit.client.0.smithi071.stdout:9/383: link d1/d32/d3a/d5f/f60 d1/d1a/d68/f80 0 2024-04-23T02:13:59.747 INFO:tasks.workunit.client.0.smithi071.stdout:2/347: dwrite d0/d2/d26/f32 [0,4194304] 0 2024-04-23T02:13:59.749 INFO:tasks.workunit.client.1.smithi192.stdout:2/415: dread f7 [0,4194304] 0 2024-04-23T02:13:59.753 INFO:tasks.workunit.client.1.smithi192.stdout:2/416: creat d8/d26/d29/d43/f83 x:0 0 0 2024-04-23T02:13:59.754 INFO:tasks.workunit.client.1.smithi192.stdout:6/565: rmdir d3/d15/d1b 39 2024-04-23T02:13:59.756 INFO:tasks.workunit.client.0.smithi071.stdout:0/430: readlink d0/d3/de/d10/d14/d32/d44/d31/l6c 0 2024-04-23T02:13:59.756 INFO:tasks.workunit.client.0.smithi071.stdout:3/364: mknod d7/c79 0 2024-04-23T02:13:59.756 INFO:tasks.workunit.client.1.smithi192.stdout:2/417: write d8/d26/d29/d43/f7f [776391,88112] 0 2024-04-23T02:13:59.757 INFO:tasks.workunit.client.0.smithi071.stdout:7/401: dread d3/fd [0,4194304] 0 2024-04-23T02:13:59.757 INFO:tasks.workunit.client.1.smithi192.stdout:6/566: truncate d3/d11/f1d 1074666 0 2024-04-23T02:13:59.759 INFO:tasks.workunit.client.1.smithi192.stdout:2/418: truncate d8/d26/d27/f45 1222083 0 2024-04-23T02:13:59.761 INFO:tasks.workunit.client.0.smithi071.stdout:2/348: read d0/d2/d26/d14/f22 [257856,45804] 0 2024-04-23T02:13:59.761 INFO:tasks.workunit.client.1.smithi192.stdout:2/419: mknod d8/d14/d51/c84 0 2024-04-23T02:13:59.762 INFO:tasks.workunit.client.1.smithi192.stdout:6/567: mknod d3/d15/d1b/d66/d80/d8e/d9d/cb7 0 2024-04-23T02:13:59.762 INFO:tasks.workunit.client.0.smithi071.stdout:7/402: readlink d3/l44 0 2024-04-23T02:13:59.763 INFO:tasks.workunit.client.1.smithi192.stdout:2/420: rmdir d8/d26 39 2024-04-23T02:13:59.764 INFO:tasks.workunit.client.1.smithi192.stdout:6/568: fdatasync d3/d15/d1b/d21/d52/f92 0 2024-04-23T02:13:59.767 INFO:tasks.workunit.client.0.smithi071.stdout:5/500: link d3/da/d1a/d1c/c30 d3/d1d/daf/d10/da5/d5f/dc0/cc4 0 2024-04-23T02:13:59.767 INFO:tasks.workunit.client.0.smithi071.stdout:8/462: rename dc/d15/d2d/f88 to dc/f96 0 2024-04-23T02:13:59.769 INFO:tasks.workunit.client.1.smithi192.stdout:6/569: dread d3/d18/f78 [0,4194304] 0 2024-04-23T02:13:59.770 INFO:tasks.workunit.client.1.smithi192.stdout:2/421: rename d8/d26/f3c to d8/d26/d29/f85 0 2024-04-23T02:13:59.772 INFO:tasks.workunit.client.1.smithi192.stdout:6/570: rename d3/d18/f8a to d3/d15/d5b/fb8 0 2024-04-23T02:13:59.774 INFO:tasks.workunit.client.0.smithi071.stdout:6/335: dwrite f4 [0,4194304] 0 2024-04-23T02:13:59.774 INFO:tasks.workunit.client.1.smithi192.stdout:2/422: creat d8/f86 x:0 0 0 2024-04-23T02:13:59.777 INFO:tasks.workunit.client.0.smithi071.stdout:7/403: creat d3/d17/d1c/d2a/d43/d86/f88 x:0 0 0 2024-04-23T02:13:59.777 INFO:tasks.workunit.client.1.smithi192.stdout:2/423: chown d8/l2b 742828 1 2024-04-23T02:13:59.777 INFO:tasks.workunit.client.1.smithi192.stdout:2/424: dread - d8/d14/d51/f7e zero size 2024-04-23T02:13:59.779 INFO:tasks.workunit.client.0.smithi071.stdout:2/349: getdents d0/d2/d45 0 2024-04-23T02:13:59.779 INFO:tasks.workunit.client.1.smithi192.stdout:2/425: write d8/d26/d27/f45 [1299841,46748] 0 2024-04-23T02:13:59.780 INFO:tasks.workunit.client.0.smithi071.stdout:8/463: symlink dc/d10/d30/l97 0 2024-04-23T02:13:59.783 INFO:tasks.workunit.client.0.smithi071.stdout:1/402: sync 2024-04-23T02:13:59.783 INFO:tasks.workunit.client.0.smithi071.stdout:6/336: write d6/d22/f35 [1327353,75431] 0 2024-04-23T02:13:59.783 INFO:tasks.workunit.client.0.smithi071.stdout:5/501: dread d3/da/f4d [0,4194304] 0 2024-04-23T02:13:59.784 INFO:tasks.workunit.client.0.smithi071.stdout:0/431: truncate d0/d3/d1c/f21 498746 0 2024-04-23T02:13:59.785 INFO:tasks.workunit.client.0.smithi071.stdout:9/384: dread d1/f37 [0,4194304] 0 2024-04-23T02:13:59.789 INFO:tasks.workunit.client.1.smithi192.stdout:2/426: dwrite d8/d25/f71 [0,4194304] 0 2024-04-23T02:13:59.792 INFO:tasks.workunit.client.1.smithi192.stdout:6/571: dwrite d3/d28/f96 [0,4194304] 0 2024-04-23T02:13:59.793 INFO:tasks.workunit.client.1.smithi192.stdout:2/427: creat d8/d26/d27/f87 x:0 0 0 2024-04-23T02:13:59.794 INFO:tasks.workunit.client.1.smithi192.stdout:2/428: chown d8/d25/l6e 35248854 1 2024-04-23T02:13:59.795 INFO:tasks.workunit.client.0.smithi071.stdout:4/431: dwrite d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c/f2e [4194304,4194304] 0 2024-04-23T02:13:59.796 INFO:tasks.workunit.client.1.smithi192.stdout:2/429: read d8/d26/d29/f41 [316035,61288] 0 2024-04-23T02:13:59.797 INFO:tasks.workunit.client.1.smithi192.stdout:2/430: write d8/d26/d29/d43/f83 [647204,46554] 0 2024-04-23T02:13:59.798 INFO:tasks.workunit.client.0.smithi071.stdout:2/350: dwrite d0/d2/d26/d14/f42 [0,4194304] 0 2024-04-23T02:13:59.799 INFO:tasks.workunit.client.1.smithi192.stdout:2/431: write d8/d26/d27/d53/f5b [21553,7338] 0 2024-04-23T02:13:59.800 INFO:tasks.workunit.client.0.smithi071.stdout:9/385: mkdir d1/d32/d3a/d78/d81 0 2024-04-23T02:13:59.801 INFO:tasks.workunit.client.0.smithi071.stdout:8/464: creat dc/d10/d55/f98 x:0 0 0 2024-04-23T02:13:59.801 INFO:tasks.workunit.client.1.smithi192.stdout:6/572: dwrite d3/f22 [0,4194304] 0 2024-04-23T02:13:59.802 INFO:tasks.workunit.client.0.smithi071.stdout:5/502: read d3/da/d1a/db8/f8a [7426403,47053] 0 2024-04-23T02:13:59.802 INFO:tasks.workunit.client.1.smithi192.stdout:2/432: rmdir d8/d26 39 2024-04-23T02:13:59.806 INFO:tasks.workunit.client.0.smithi071.stdout:8/465: fdatasync dc/d10/d30/f52 0 2024-04-23T02:13:59.806 INFO:tasks.workunit.client.0.smithi071.stdout:9/386: fdatasync d1/f28 0 2024-04-23T02:13:59.806 INFO:tasks.workunit.client.0.smithi071.stdout:6/337: dwrite d6/d3e/f42 [0,4194304] 0 2024-04-23T02:13:59.810 INFO:tasks.workunit.client.0.smithi071.stdout:6/338: fsync d6/d1f/d5d/f5f 0 2024-04-23T02:13:59.812 INFO:tasks.workunit.client.0.smithi071.stdout:5/503: readlink d3/l18 0 2024-04-23T02:13:59.812 INFO:tasks.workunit.client.1.smithi192.stdout:6/573: rename d3/d15/d1b/d21/d3a/f61 to d3/d15/d1b/d66/d80/fb9 0 2024-04-23T02:13:59.813 INFO:tasks.workunit.client.0.smithi071.stdout:7/404: creat d3/d17/d41/d34/f89 x:0 0 0 2024-04-23T02:13:59.816 INFO:tasks.workunit.client.0.smithi071.stdout:0/432: creat d0/d3/de/d10/d14/f85 x:0 0 0 2024-04-23T02:13:59.817 INFO:tasks.workunit.client.0.smithi071.stdout:4/432: creat d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f8e x:0 0 0 2024-04-23T02:13:59.818 INFO:tasks.workunit.client.1.smithi192.stdout:6/574: creat d3/d15/d1b/d21/d52/d64/d8d/fba x:0 0 0 2024-04-23T02:13:59.818 INFO:tasks.workunit.client.0.smithi071.stdout:2/351: dwrite d0/d2/d26/d14/f43 [0,4194304] 0 2024-04-23T02:13:59.818 INFO:tasks.workunit.client.0.smithi071.stdout:1/403: link db/l23 db/df/d13/d2f/d7b/l7c 0 2024-04-23T02:13:59.819 INFO:tasks.workunit.client.1.smithi192.stdout:6/575: readlink d3/lf 0 2024-04-23T02:13:59.821 INFO:tasks.workunit.client.0.smithi071.stdout:6/339: chown d6/d1f/d5d/c6c 46 1 2024-04-23T02:13:59.826 INFO:tasks.workunit.client.0.smithi071.stdout:5/504: mknod d3/d34/d6b/d81/d5a/da7/cc5 0 2024-04-23T02:13:59.826 INFO:tasks.workunit.client.0.smithi071.stdout:9/387: dwrite d1/d4e/f4f [0,4194304] 0 2024-04-23T02:13:59.829 INFO:tasks.workunit.client.0.smithi071.stdout:8/466: symlink dc/l99 0 2024-04-23T02:13:59.829 INFO:tasks.workunit.client.0.smithi071.stdout:3/365: dread d7/d10/d38/f41 [0,4194304] 0 2024-04-23T02:13:59.834 INFO:tasks.workunit.client.0.smithi071.stdout:6/340: unlink d6/d17/f1a 0 2024-04-23T02:13:59.838 INFO:tasks.workunit.client.1.smithi192.stdout:2/433: dread d8/d26/d29/f85 [0,4194304] 0 2024-04-23T02:13:59.840 INFO:tasks.workunit.client.0.smithi071.stdout:1/404: creat db/d34/f7d x:0 0 0 2024-04-23T02:13:59.842 INFO:tasks.workunit.client.1.smithi192.stdout:2/434: dread d8/d26/d29/d43/f7f [0,4194304] 0 2024-04-23T02:13:59.843 INFO:tasks.workunit.client.1.smithi192.stdout:2/435: write d8/d26/d27/f87 [549996,64953] 0 2024-04-23T02:13:59.843 INFO:tasks.workunit.client.0.smithi071.stdout:9/388: mknod d1/d32/d3a/c82 0 2024-04-23T02:13:59.844 INFO:tasks.workunit.client.1.smithi192.stdout:8/660: sync 2024-04-23T02:13:59.845 INFO:tasks.workunit.client.0.smithi071.stdout:3/366: creat d7/d10/d64/f7a x:0 0 0 2024-04-23T02:13:59.845 INFO:tasks.workunit.client.0.smithi071.stdout:6/341: creat d6/d1f/d5d/f6d x:0 0 0 2024-04-23T02:13:59.845 INFO:tasks.workunit.client.0.smithi071.stdout:5/505: write d3/da/f33 [1635175,65141] 0 2024-04-23T02:13:59.846 INFO:tasks.workunit.client.1.smithi192.stdout:2/436: write d8/d26/d27/d31/f2a [4662156,93160] 0 2024-04-23T02:13:59.847 INFO:tasks.workunit.client.0.smithi071.stdout:6/342: dread - d6/d1f/f68 zero size 2024-04-23T02:13:59.848 INFO:tasks.workunit.client.1.smithi192.stdout:2/437: rmdir d8/d14/d51 39 2024-04-23T02:13:59.849 INFO:tasks.workunit.client.0.smithi071.stdout:1/405: creat db/df/d13/d2b/f7e x:0 0 0 2024-04-23T02:13:59.849 INFO:tasks.workunit.client.1.smithi192.stdout:2/438: write d8/d26/d29/f58 [3834332,127659] 0 2024-04-23T02:13:59.850 INFO:tasks.workunit.client.0.smithi071.stdout:3/367: symlink d7/d8/d42/d47/l7b 0 2024-04-23T02:13:59.851 INFO:tasks.workunit.client.0.smithi071.stdout:2/352: dwrite d0/d2/f4d [0,4194304] 0 2024-04-23T02:13:59.852 INFO:tasks.workunit.client.1.smithi192.stdout:2/439: write d8/d14/f4c [3141696,111672] 0 2024-04-23T02:13:59.852 INFO:tasks.workunit.client.0.smithi071.stdout:8/467: dread dc/d15/d50/f82 [0,4194304] 0 2024-04-23T02:13:59.855 INFO:tasks.workunit.client.0.smithi071.stdout:5/506: rename d3/d34/d6b/f9e to d3/d8f/fc6 0 2024-04-23T02:13:59.857 INFO:tasks.workunit.client.0.smithi071.stdout:6/343: creat d6/d1f/d46/f6e x:0 0 0 2024-04-23T02:13:59.860 INFO:tasks.workunit.client.0.smithi071.stdout:8/468: fsync dc/d10/d32/f4b 0 2024-04-23T02:13:59.861 INFO:tasks.workunit.client.0.smithi071.stdout:1/406: write db/df/d13/f71 [826251,122556] 0 2024-04-23T02:13:59.861 INFO:tasks.workunit.client.1.smithi192.stdout:2/440: dwrite d8/d25/f32 [0,4194304] 0 2024-04-23T02:13:59.866 INFO:tasks.workunit.client.0.smithi071.stdout:4/433: dwrite d2/d2a/d2d/d39/d42/d49/d67/f22 [0,4194304] 0 2024-04-23T02:13:59.866 INFO:tasks.workunit.client.0.smithi071.stdout:8/469: readlink dc/d15/d50/d5f/l90 0 2024-04-23T02:13:59.866 INFO:tasks.workunit.client.0.smithi071.stdout:5/507: symlink d3/d1d/lc7 0 2024-04-23T02:13:59.867 INFO:tasks.workunit.client.1.smithi192.stdout:2/441: symlink d8/d26/d27/l88 0 2024-04-23T02:13:59.870 INFO:tasks.workunit.client.1.smithi192.stdout:8/661: dread d1/fe [0,4194304] 0 2024-04-23T02:13:59.874 INFO:tasks.workunit.client.1.smithi192.stdout:8/662: symlink d1/d9/d3e/d59/d8a/leb 0 2024-04-23T02:13:59.874 INFO:tasks.workunit.client.1.smithi192.stdout:2/442: dwrite d8/d26/d27/d31/f2a [4194304,4194304] 0 2024-04-23T02:13:59.875 INFO:tasks.workunit.client.0.smithi071.stdout:5/508: mknod d3/cc8 0 2024-04-23T02:13:59.875 INFO:tasks.workunit.client.0.smithi071.stdout:1/407: mkdir db/d11/d7f 0 2024-04-23T02:13:59.875 INFO:tasks.workunit.client.0.smithi071.stdout:6/344: link d6/d3e/d3f/f47 d6/d1f/d5d/f6f 0 2024-04-23T02:13:59.880 INFO:tasks.workunit.client.0.smithi071.stdout:3/368: dwrite d7/d8/d42/d5f/d68/f62 [0,4194304] 0 2024-04-23T02:13:59.884 INFO:tasks.workunit.client.0.smithi071.stdout:6/345: mknod d6/d1f/d46/c70 0 2024-04-23T02:13:59.884 INFO:tasks.workunit.client.0.smithi071.stdout:4/434: chown d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d10/f87 1 1 2024-04-23T02:13:59.884 INFO:tasks.workunit.client.1.smithi192.stdout:2/443: dwrite d8/d26/d29/d43/d50/d5e/f5f [0,4194304] 0 2024-04-23T02:13:59.891 INFO:tasks.workunit.client.0.smithi071.stdout:1/408: rename db/f4b to db/d34/d68/f80 0 2024-04-23T02:13:59.892 INFO:tasks.workunit.client.0.smithi071.stdout:6/346: creat d6/d1f/d5d/f71 x:0 0 0 2024-04-23T02:13:59.892 INFO:tasks.workunit.client.0.smithi071.stdout:5/509: creat d3/d1d/daf/d83/fc9 x:0 0 0 2024-04-23T02:13:59.894 INFO:tasks.workunit.client.1.smithi192.stdout:8/663: rmdir d1/d9 39 2024-04-23T02:13:59.894 INFO:tasks.workunit.client.0.smithi071.stdout:5/510: chown d3/d1d/cbc 78314783 1 2024-04-23T02:13:59.900 INFO:tasks.workunit.client.0.smithi071.stdout:6/347: read - d6/d1f/d4e/f51 zero size 2024-04-23T02:13:59.902 INFO:tasks.workunit.client.1.smithi192.stdout:8/664: dwrite d1/d9/d3e/fb2 [0,4194304] 0 2024-04-23T02:13:59.906 INFO:tasks.workunit.client.0.smithi071.stdout:5/511: dread - d3/d1d/d43/d4b/f72 zero size 2024-04-23T02:13:59.908 INFO:tasks.workunit.client.0.smithi071.stdout:1/409: creat db/d34/d68/f81 x:0 0 0 2024-04-23T02:13:59.909 INFO:tasks.workunit.client.0.smithi071.stdout:1/410: fdatasync db/df/d13/f1b 0 2024-04-23T02:13:59.911 INFO:tasks.workunit.client.1.smithi192.stdout:8/665: dwrite d1/f72 [0,4194304] 0 2024-04-23T02:13:59.912 INFO:tasks.workunit.client.0.smithi071.stdout:6/348: truncate d6/d1f/d4e/f54 37907 0 2024-04-23T02:13:59.915 INFO:tasks.workunit.client.0.smithi071.stdout:4/435: creat d2/d2a/d2d/d39/d42/d49/d67/f8f x:0 0 0 2024-04-23T02:13:59.917 INFO:tasks.workunit.client.1.smithi192.stdout:8/666: chown d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d8b/lb7 15779612 1 2024-04-23T02:13:59.917 INFO:tasks.workunit.client.1.smithi192.stdout:2/444: dwrite d8/d14/f15 [0,4194304] 0 2024-04-23T02:13:59.918 INFO:tasks.workunit.client.1.smithi192.stdout:2/445: readlink d8/d26/d27/l39 0 2024-04-23T02:13:59.919 INFO:tasks.workunit.client.1.smithi192.stdout:2/446: readlink d8/l3d 0 2024-04-23T02:13:59.921 INFO:tasks.workunit.client.0.smithi071.stdout:1/411: unlink db/d34/l38 0 2024-04-23T02:13:59.923 INFO:tasks.workunit.client.1.smithi192.stdout:2/447: link d8/d26/d27/d53/f82 d8/d26/d29/d43/d50/d5e/f89 0 2024-04-23T02:13:59.925 INFO:tasks.workunit.client.1.smithi192.stdout:2/448: creat d8/d26/d27/d57/d69/f8a x:0 0 0 2024-04-23T02:13:59.927 INFO:tasks.workunit.client.1.smithi192.stdout:2/449: fsync d8/d26/d29/d43/f83 0 2024-04-23T02:13:59.927 INFO:tasks.workunit.client.1.smithi192.stdout:2/450: dread - d8/d26/f78 zero size 2024-04-23T02:13:59.930 INFO:tasks.workunit.client.0.smithi071.stdout:7/405: dwrite d3/d17/d41/d37/f75 [0,4194304] 0 2024-04-23T02:13:59.930 INFO:tasks.workunit.client.1.smithi192.stdout:2/451: getdents d8/d14 0 2024-04-23T02:13:59.931 INFO:tasks.workunit.client.1.smithi192.stdout:2/452: read d8/d26/d29/d43/f83 [177048,20788] 0 2024-04-23T02:13:59.933 INFO:tasks.workunit.client.0.smithi071.stdout:2/353: dwrite d0/d2/d26/f4c [0,4194304] 0 2024-04-23T02:13:59.935 INFO:tasks.workunit.client.1.smithi192.stdout:2/453: unlink d8/d26/d27/f45 0 2024-04-23T02:13:59.937 INFO:tasks.workunit.client.1.smithi192.stdout:2/454: unlink d8/d25/l47 0 2024-04-23T02:13:59.937 INFO:tasks.workunit.client.0.smithi071.stdout:6/349: rename d6/l15 to d6/d1f/d5d/l72 0 2024-04-23T02:13:59.939 INFO:tasks.workunit.client.1.smithi192.stdout:2/455: creat d8/d26/d27/d31/f8b x:0 0 0 2024-04-23T02:13:59.947 INFO:tasks.workunit.client.0.smithi071.stdout:5/512: getdents d3/da/d1a/d1c/d46/d4e 0 2024-04-23T02:13:59.948 INFO:tasks.workunit.client.0.smithi071.stdout:2/354: mkdir d0/d2/d45/d58/d71 0 2024-04-23T02:13:59.953 INFO:tasks.workunit.client.0.smithi071.stdout:3/369: dwrite d7/d8/f1b [0,4194304] 0 2024-04-23T02:13:59.960 INFO:tasks.workunit.client.0.smithi071.stdout:8/470: dwrite dc/d15/f39 [0,4194304] 0 2024-04-23T02:13:59.962 INFO:tasks.workunit.client.0.smithi071.stdout:4/436: chown d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/l4c 0 1 2024-04-23T02:13:59.962 INFO:tasks.workunit.client.0.smithi071.stdout:5/513: fdatasync d3/d1d/f20 0 2024-04-23T02:13:59.965 INFO:tasks.workunit.client.1.smithi192.stdout:2/456: dread d8/f1d [0,4194304] 0 2024-04-23T02:13:59.968 INFO:tasks.workunit.client.0.smithi071.stdout:6/350: dwrite d6/d1f/d5d/f6d [0,4194304] 0 2024-04-23T02:13:59.970 INFO:tasks.workunit.client.0.smithi071.stdout:5/514: read d3/d34/f40 [233565,29844] 0 2024-04-23T02:13:59.971 INFO:tasks.workunit.client.1.smithi192.stdout:2/457: unlink d8/d14/c40 0 2024-04-23T02:13:59.971 INFO:tasks.workunit.client.0.smithi071.stdout:2/355: rename d0/d2/d26/df/f5e to d0/d2/d34/d55/f72 0 2024-04-23T02:13:59.974 INFO:tasks.workunit.client.1.smithi192.stdout:2/458: dread d8/d25/f71 [0,4194304] 0 2024-04-23T02:13:59.979 INFO:tasks.workunit.client.0.smithi071.stdout:1/412: dwrite db/df/d13/d20/f32 [0,4194304] 0 2024-04-23T02:13:59.979 INFO:tasks.workunit.client.0.smithi071.stdout:6/351: write d6/d3e/f42 [4941009,44249] 0 2024-04-23T02:13:59.979 INFO:tasks.workunit.client.0.smithi071.stdout:4/437: symlink d2/d2a/d2d/d39/d42/d49/d65/l90 0 2024-04-23T02:13:59.980 INFO:tasks.workunit.client.0.smithi071.stdout:1/413: chown f5 24 1 2024-04-23T02:13:59.981 INFO:tasks.workunit.client.1.smithi192.stdout:2/459: link d8/cf d8/d26/d74/d7c/c8c 0 2024-04-23T02:13:59.982 INFO:tasks.workunit.client.0.smithi071.stdout:5/515: truncate d3/d1d/f71 492879 0 2024-04-23T02:13:59.982 INFO:tasks.workunit.client.0.smithi071.stdout:1/414: fdatasync db/df/f1c 0 2024-04-23T02:13:59.983 INFO:tasks.workunit.client.0.smithi071.stdout:4/438: write d2/d2a/d2d/d39/d42/d49/f77 [2878666,125712] 0 2024-04-23T02:13:59.984 INFO:tasks.workunit.client.0.smithi071.stdout:3/370: getdents d7 0 2024-04-23T02:13:59.985 INFO:tasks.workunit.client.0.smithi071.stdout:2/356: mkdir d0/d2/d26/d73 0 2024-04-23T02:13:59.987 INFO:tasks.workunit.client.0.smithi071.stdout:6/352: rmdir d6/d3e 39 2024-04-23T02:13:59.989 INFO:tasks.workunit.client.1.smithi192.stdout:2/460: dread d8/d25/f71 [0,4194304] 0 2024-04-23T02:13:59.990 INFO:tasks.workunit.client.0.smithi071.stdout:3/371: creat d7/d8/d42/f7c x:0 0 0 2024-04-23T02:13:59.994 INFO:tasks.workunit.client.1.smithi192.stdout:2/461: mknod d8/d26/d27/d57/c8d 0 2024-04-23T02:13:59.994 INFO:tasks.workunit.client.1.smithi192.stdout:2/462: chown d8 9542126 1 2024-04-23T02:13:59.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:59 smithi071 ceph-mon[39837]: from='client.14642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:13:59.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:59 smithi071 ceph-mon[39837]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:13:59.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:13:59 smithi071 ceph-mon[39837]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:13:59.998 INFO:tasks.workunit.client.1.smithi192.stdout:2/463: rename d8/d14/d51/l6f to d8/d26/l8e 0 2024-04-23T02:13:59.998 INFO:tasks.workunit.client.1.smithi192.stdout:3/405: sync 2024-04-23T02:13:59.998 INFO:tasks.workunit.client.1.smithi192.stdout:4/638: sync 2024-04-23T02:13:59.999 INFO:tasks.workunit.client.1.smithi192.stdout:9/440: sync 2024-04-23T02:13:59.999 INFO:tasks.workunit.client.1.smithi192.stdout:5/470: sync 2024-04-23T02:13:59.999 INFO:tasks.workunit.client.1.smithi192.stdout:1/293: sync 2024-04-23T02:13:59.999 INFO:tasks.workunit.client.1.smithi192.stdout:0/607: sync 2024-04-23T02:14:00.000 INFO:tasks.workunit.client.0.smithi071.stdout:6/353: creat d6/d1f/d5d/f73 x:0 0 0 2024-04-23T02:14:00.001 INFO:tasks.workunit.client.1.smithi192.stdout:4/639: stat de/d15/d53/l9c 0 2024-04-23T02:14:00.001 INFO:tasks.workunit.client.1.smithi192.stdout:1/294: fsync f3 0 2024-04-23T02:14:00.002 INFO:tasks.workunit.client.1.smithi192.stdout:0/608: write d6/df/d35/d8b/f9b [4767984,122812] 0 2024-04-23T02:14:00.003 INFO:tasks.workunit.client.1.smithi192.stdout:5/471: chown d2/db/f13 92 1 2024-04-23T02:14:00.005 INFO:tasks.workunit.client.1.smithi192.stdout:3/406: mkdir d1/d3/d17/d1f/d33/d24/d34/d38/d8b 0 2024-04-23T02:14:00.006 INFO:tasks.workunit.client.1.smithi192.stdout:1/295: stat dc/d14/d15/d35/f37 0 2024-04-23T02:14:00.006 INFO:tasks.workunit.client.1.smithi192.stdout:5/472: chown d2/d2e/f64 2891 1 2024-04-23T02:14:00.008 INFO:tasks.workunit.client.1.smithi192.stdout:0/609: creat d6/df/d56/d85/d45/fcb x:0 0 0 2024-04-23T02:14:00.008 INFO:tasks.workunit.client.1.smithi192.stdout:3/407: fdatasync d1/d3/d17/d1f/d33/d24/f50 0 2024-04-23T02:14:00.008 INFO:tasks.workunit.client.1.smithi192.stdout:4/640: dread - de/d15/d2a/d74/d92/da0/dca/d49/d70/d73/fb1 zero size 2024-04-23T02:14:00.009 INFO:tasks.workunit.client.1.smithi192.stdout:5/473: chown d2/d6/d2d/c39 13922 1 2024-04-23T02:14:00.011 INFO:tasks.workunit.client.1.smithi192.stdout:5/474: readlink d2/db/lc 0 2024-04-23T02:14:00.011 INFO:tasks.workunit.client.1.smithi192.stdout:9/441: link d0/f4e d0/d9/d44/f9d 0 2024-04-23T02:14:00.011 INFO:tasks.workunit.client.0.smithi071.stdout:5/516: rename d3/da/f13 to d3/da/d1a/db8/fca 0 2024-04-23T02:14:00.013 INFO:tasks.workunit.client.1.smithi192.stdout:3/408: unlink d1/d3/d17/c5c 0 2024-04-23T02:14:00.013 INFO:tasks.workunit.client.0.smithi071.stdout:2/357: read d0/d2/d34/f50 [3299066,130145] 0 2024-04-23T02:14:00.014 INFO:tasks.workunit.client.1.smithi192.stdout:9/442: write d0/f2c [1812958,1102] 0 2024-04-23T02:14:00.016 INFO:tasks.workunit.client.1.smithi192.stdout:0/610: rename d6/f1b to d6/df/d56/d85/d71/fcc 0 2024-04-23T02:14:00.016 INFO:tasks.workunit.client.0.smithi071.stdout:7/406: dwrite d3/d17/d1c/f29 [0,4194304] 0 2024-04-23T02:14:00.023 INFO:tasks.workunit.client.1.smithi192.stdout:1/296: dwrite dc/d14/d15/d35/f37 [0,4194304] 0 2024-04-23T02:14:00.024 INFO:tasks.workunit.client.1.smithi192.stdout:3/409: write d1/d3/d17/d1f/d33/d24/d34/d6c/d84/d46/f85 [994376,71376] 0 2024-04-23T02:14:00.024 INFO:tasks.workunit.client.1.smithi192.stdout:2/464: truncate d8/d26/d27/d53/f54 315052 0 2024-04-23T02:14:00.025 INFO:tasks.workunit.client.0.smithi071.stdout:6/354: chown d6/d3e/f4a 1883710 1 2024-04-23T02:14:00.025 INFO:tasks.workunit.client.1.smithi192.stdout:9/443: getdents d0/d9/d44/d53 0 2024-04-23T02:14:00.026 INFO:tasks.workunit.client.0.smithi071.stdout:7/407: creat d3/d51/d80/f8a x:0 0 0 2024-04-23T02:14:00.027 INFO:tasks.workunit.client.0.smithi071.stdout:2/358: mkdir d0/d2/d51/d74 0 2024-04-23T02:14:00.028 INFO:tasks.workunit.client.1.smithi192.stdout:0/611: truncate d6/df/d56/d85/d71/f84 1307247 0 2024-04-23T02:14:00.030 INFO:tasks.workunit.client.1.smithi192.stdout:9/444: readlink d0/d9/d44/d4f/l40 0 2024-04-23T02:14:00.031 INFO:tasks.workunit.client.1.smithi192.stdout:0/612: write d6/df/d13/d25/d3a/f58 [9319473,127190] 0 2024-04-23T02:14:00.032 INFO:tasks.workunit.client.0.smithi071.stdout:6/355: unlink d6/f55 0 2024-04-23T02:14:00.033 INFO:tasks.workunit.client.0.smithi071.stdout:2/359: write d0/d2/d45/f52 [370011,11411] 0 2024-04-23T02:14:00.035 INFO:tasks.workunit.client.1.smithi192.stdout:3/410: read d1/d3/d17/f6e [1184672,49534] 0 2024-04-23T02:14:00.035 INFO:tasks.workunit.client.1.smithi192.stdout:4/641: dread de/d15/d2a/d74/d92/da0/dca/d34/d4a/f71 [0,4194304] 0 2024-04-23T02:14:00.035 INFO:tasks.workunit.client.1.smithi192.stdout:2/465: unlink d8/d26/d27/d53/f82 0 2024-04-23T02:14:00.037 INFO:tasks.workunit.client.0.smithi071.stdout:5/517: rename d3/da/d1a/d1c/d46/d4e/fc3 to d3/d1d/d43/fcb 0 2024-04-23T02:14:00.038 INFO:tasks.workunit.client.1.smithi192.stdout:0/613: symlink d6/df/d13/d25/lcd 0 2024-04-23T02:14:00.040 INFO:tasks.workunit.client.0.smithi071.stdout:7/408: symlink d3/d17/d41/d34/d39/l8b 0 2024-04-23T02:14:00.040 INFO:tasks.workunit.client.0.smithi071.stdout:4/439: dwrite d2/d2a/d2d/d39/d42/d49/d67/d3e/f4d [0,4194304] 0 2024-04-23T02:14:00.042 INFO:tasks.workunit.client.1.smithi192.stdout:3/411: creat d1/d3/f8c x:0 0 0 2024-04-23T02:14:00.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.037+0000 7f03fbeca700 1 -- 172.21.15.71:0/1409135634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f03f410fef0 con 0x7f03f4101f00 2024-04-23T02:14:00.043 INFO:tasks.workunit.client.0.smithi071.stdout:1/415: dwrite db/df/d5f/d57/f5b [4194304,4194304] 0 2024-04-23T02:14:00.044 INFO:tasks.workunit.client.1.smithi192.stdout:9/445: dwrite d0/d9/d44/d74/f8a [0,4194304] 0 2024-04-23T02:14:00.044 INFO:tasks.workunit.client.1.smithi192.stdout:2/466: symlink d8/d26/d74/l8f 0 2024-04-23T02:14:00.044 INFO:tasks.workunit.client.0.smithi071.stdout:2/360: chown d0/d2/d26 493562 1 2024-04-23T02:14:00.045 INFO:tasks.workunit.client.0.smithi071.stdout:7/409: chown d3/d17/d41/f2f 5800303 1 2024-04-23T02:14:00.047 INFO:tasks.workunit.client.0.smithi071.stdout:7/410: write d3/d17/d1c/d31/f56 [964610,100752] 0 2024-04-23T02:14:00.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.043+0000 7f03eaffd700 1 -- 172.21.15.71:0/1409135634 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+770 (secure 0 0 0) 0x7f03e4059180 con 0x7f03f4101f00 2024-04-23T02:14:00.049 INFO:tasks.workunit.client.0.smithi071.stdout:6/356: readlink d6/d1f/l57 0 2024-04-23T02:14:00.051 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:14:00.051 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:14:00.051 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-23T02:14:00.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:14:00.051 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:14:00.051 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:14:00.051 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1 2024-04-23T02:14:00.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:14:00.052 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:14:00.052 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-23T02:14:00.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:14:00.052 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:14:00.052 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-23T02:14:00.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:14:00.052 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:14:00.052 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-04-23T02:14:00.052 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1 2024-04-23T02:14:00.052 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:14:00.052 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:14:00.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.047+0000 7f03fbeca700 1 -- 172.21.15.71:0/1409135634 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f03e006cb20 msgr2=0x7f03e006efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:00.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.047+0000 7f03fbeca700 1 --2- 172.21.15.71:0/1409135634 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f03e006cb20 0x7f03e006efe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f03f000bfa0 tx=0x7f03f000b560 comp rx=0 tx=0).stop 2024-04-23T02:14:00.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.047+0000 7f03fbeca700 1 -- 172.21.15.71:0/1409135634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03f4101f00 msgr2=0x7f03f40723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:00.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.047+0000 7f03fbeca700 1 --2- 172.21.15.71:0/1409135634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03f4101f00 0x7f03f40723f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f03e400b3d0 tx=0x7f03e400b6e0 comp rx=0 tx=0).stop 2024-04-23T02:14:00.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.047+0000 7f03fbeca700 1 -- 172.21.15.71:0/1409135634 shutdown_connections 2024-04-23T02:14:00.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.047+0000 7f03fbeca700 1 --2- 172.21.15.71:0/1409135634 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f03e006cb20 0x7f03e006efe0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:00.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.047+0000 7f03fbeca700 1 --2- 172.21.15.71:0/1409135634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03f40ff590 0x7f03f4071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:00.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.047+0000 7f03fbeca700 1 --2- 172.21.15.71:0/1409135634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03f4101f00 0x7f03f40723f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:00.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.047+0000 7f03fbeca700 1 -- 172.21.15.71:0/1409135634 >> 172.21.15.71:0/1409135634 conn(0x7f03f40fb190 msgr2=0x7f03f40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:14:00.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.048+0000 7f03fbeca700 1 -- 172.21.15.71:0/1409135634 shutdown_connections 2024-04-23T02:14:00.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.048+0000 7f03fbeca700 1 -- 172.21.15.71:0/1409135634 wait complete. 2024-04-23T02:14:00.054 INFO:tasks.workunit.client.1.smithi192.stdout:1/297: write f3 [6259413,63702] 0 2024-04-23T02:14:00.054 INFO:tasks.workunit.client.1.smithi192.stdout:3/412: write d1/d3/d6/d7/f2a [465008,52619] 0 2024-04-23T02:14:00.054 INFO:tasks.workunit.client.1.smithi192.stdout:4/642: truncate de/d15/d2a/d74/d92/da0/dca/f32 934750 0 2024-04-23T02:14:00.056 INFO:tasks.workunit.client.0.smithi071.stdout:4/440: mkdir d2/d2a/d2d/d39/d42/d49/d91 0 2024-04-23T02:14:00.057 INFO:tasks.workunit.client.0.smithi071.stdout:2/361: mkdir d0/d2/d26/d3a/d75 0 2024-04-23T02:14:00.057 INFO:tasks.workunit.client.1.smithi192.stdout:9/446: fdatasync d0/d9/d44/d51/d5b/d61/f6a 0 2024-04-23T02:14:00.058 INFO:tasks.workunit.client.0.smithi071.stdout:2/362: stat d0/d2/d34/d55/f72 0 2024-04-23T02:14:00.059 INFO:tasks.workunit.client.0.smithi071.stdout:7/411: mknod d3/d17/d41/d34/c8c 0 2024-04-23T02:14:00.060 INFO:tasks.workunit.client.1.smithi192.stdout:1/298: creat dc/d14/d15/f54 x:0 0 0 2024-04-23T02:14:00.063 INFO:tasks.workunit.client.1.smithi192.stdout:5/475: dread d2/f2c [0,4194304] 0 2024-04-23T02:14:00.063 INFO:tasks.workunit.client.0.smithi071.stdout:2/363: symlink d0/d2/d26/df/l76 0 2024-04-23T02:14:00.063 INFO:tasks.workunit.client.0.smithi071.stdout:1/416: creat db/df/d5f/f82 x:0 0 0 2024-04-23T02:14:00.063 INFO:tasks.workunit.client.1.smithi192.stdout:0/614: dwrite d6/df/d56/d85/f46 [0,4194304] 0 2024-04-23T02:14:00.064 INFO:tasks.workunit.client.1.smithi192.stdout:6/576: sync 2024-04-23T02:14:00.064 INFO:tasks.workunit.client.1.smithi192.stdout:7/477: sync 2024-04-23T02:14:00.065 INFO:tasks.workunit.client.0.smithi071.stdout:4/441: rename d2/d53/f6b to d2/d2a/d2d/d39/d42/d49/d91/f92 0 2024-04-23T02:14:00.065 INFO:tasks.workunit.client.0.smithi071.stdout:1/417: dread - db/df/f19 zero size 2024-04-23T02:14:00.065 INFO:tasks.workunit.client.0.smithi071.stdout:3/372: dread d7/d8/f16 [0,4194304] 0 2024-04-23T02:14:00.065 INFO:tasks.workunit.client.1.smithi192.stdout:2/467: rmdir d8/d26/d29 39 2024-04-23T02:14:00.066 INFO:tasks.workunit.client.1.smithi192.stdout:6/577: chown d3/d11/f4a 5 1 2024-04-23T02:14:00.066 INFO:tasks.workunit.client.0.smithi071.stdout:7/412: symlink d3/d17/l8d 0 2024-04-23T02:14:00.067 INFO:tasks.workunit.client.0.smithi071.stdout:1/418: write db/fd [4393237,24605] 0 2024-04-23T02:14:00.070 INFO:tasks.workunit.client.0.smithi071.stdout:4/442: creat d2/d2a/d2d/d39/d42/d49/d67/d3e/f93 x:0 0 0 2024-04-23T02:14:00.070 INFO:tasks.workunit.client.1.smithi192.stdout:5/476: truncate d2/db/d16/d1b/d3d/d40/d49/f58 622356 0 2024-04-23T02:14:00.073 INFO:tasks.workunit.client.0.smithi071.stdout:1/419: rename db/df/f45 to db/df/d3a/f83 0 2024-04-23T02:14:00.074 INFO:tasks.workunit.client.1.smithi192.stdout:6/578: mkdir d3/d15/d1b/d66/d80/dbb 0 2024-04-23T02:14:00.075 INFO:tasks.workunit.client.1.smithi192.stdout:9/447: mkdir d0/d9/d44/d74/d3c/d9e 0 2024-04-23T02:14:00.076 INFO:tasks.workunit.client.0.smithi071.stdout:2/364: truncate d0/d2/d51/f53 107625 0 2024-04-23T02:14:00.076 INFO:tasks.workunit.client.0.smithi071.stdout:7/413: truncate d3/d17/d41/d34/d59/f67 587200 0 2024-04-23T02:14:00.077 INFO:tasks.workunit.client.1.smithi192.stdout:3/413: creat d1/d3/d17/d1f/d33/f8d x:0 0 0 2024-04-23T02:14:00.078 INFO:tasks.workunit.client.0.smithi071.stdout:1/420: unlink db/df/l51 0 2024-04-23T02:14:00.079 INFO:tasks.workunit.client.1.smithi192.stdout:7/478: dread d5/d17/d1c/f3c [0,4194304] 0 2024-04-23T02:14:00.079 INFO:tasks.workunit.client.1.smithi192.stdout:6/579: creat d3/d15/d1b/d66/d80/d8e/d9d/fbc x:0 0 0 2024-04-23T02:14:00.080 INFO:tasks.workunit.client.1.smithi192.stdout:5/477: getdents d2/d6/d2d/d41/d43/d98 0 2024-04-23T02:14:00.082 INFO:tasks.workunit.client.1.smithi192.stdout:6/580: write d3/d15/d1b/d21/d3a/fa6 [287052,2608] 0 2024-04-23T02:14:00.082 INFO:tasks.workunit.client.1.smithi192.stdout:7/479: write d5/d43/d4c/d5c/d67/f81 [268534,94667] 0 2024-04-23T02:14:00.082 INFO:tasks.workunit.client.1.smithi192.stdout:9/448: rename d0/d9/d44/d74/d3c/d5f/l80 to d0/d9/d44/d74/l9f 0 2024-04-23T02:14:00.084 INFO:tasks.workunit.client.0.smithi071.stdout:1/421: rename db/d11/l27 to db/d34/l84 0 2024-04-23T02:14:00.086 INFO:tasks.workunit.client.0.smithi071.stdout:4/443: dwrite d2/d2a/d2d/d39/d42/f55 [0,4194304] 0 2024-04-23T02:14:00.087 INFO:tasks.workunit.client.1.smithi192.stdout:6/581: mkdir d3/d15/d1b/d21/d3a/dbd 0 2024-04-23T02:14:00.089 INFO:tasks.workunit.client.1.smithi192.stdout:7/480: symlink d5/d17/d1c/d59/d5a/l8a 0 2024-04-23T02:14:00.091 INFO:tasks.workunit.client.0.smithi071.stdout:1/422: mkdir db/df/d13/d2f/d48/d85 0 2024-04-23T02:14:00.091 INFO:tasks.workunit.client.0.smithi071.stdout:5/518: write d3/d1d/d43/f69 [5688401,7594] 0 2024-04-23T02:14:00.091 INFO:tasks.workunit.client.1.smithi192.stdout:9/449: getdents d0/d9/d44/d51 0 2024-04-23T02:14:00.092 INFO:tasks.workunit.client.0.smithi071.stdout:4/444: creat d2/d2a/d2d/d39/d42/d49/d67/d1d/d57/f94 x:0 0 0 2024-04-23T02:14:00.093 INFO:tasks.workunit.client.1.smithi192.stdout:4/643: rmdir de/d15/d2a/d74 39 2024-04-23T02:14:00.096 INFO:tasks.workunit.client.1.smithi192.stdout:7/481: mkdir d5/d17/d1c/d57/d8b 0 2024-04-23T02:14:00.097 INFO:tasks.workunit.client.0.smithi071.stdout:6/357: write d6/d22/f26 [292210,116811] 0 2024-04-23T02:14:00.098 INFO:tasks.workunit.client.0.smithi071.stdout:4/445: symlink d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d29/l95 0 2024-04-23T02:14:00.100 INFO:tasks.workunit.client.0.smithi071.stdout:4/446: readlink d2/d53/l5f 0 2024-04-23T02:14:00.101 INFO:tasks.workunit.client.1.smithi192.stdout:7/482: mknod d5/d17/d1c/d59/d5a/d41/c8c 0 2024-04-23T02:14:00.101 INFO:tasks.workunit.client.0.smithi071.stdout:6/358: truncate d6/f18 4977088 0 2024-04-23T02:14:00.103 INFO:tasks.workunit.client.0.smithi071.stdout:3/373: truncate d7/d8/fe 4920303 0 2024-04-23T02:14:00.103 INFO:tasks.workunit.client.1.smithi192.stdout:6/582: dwrite d3/d11/fa9 [0,4194304] 0 2024-04-23T02:14:00.105 INFO:tasks.workunit.client.1.smithi192.stdout:6/583: stat d3/d15/d1b/d21/d52/d64/d8d 0 2024-04-23T02:14:00.106 INFO:tasks.workunit.client.1.smithi192.stdout:2/468: dread d8/f36 [0,4194304] 0 2024-04-23T02:14:00.109 INFO:tasks.workunit.client.1.smithi192.stdout:7/483: chown d5/d17/d1c/d59/d5a/d70/f88 190468 1 2024-04-23T02:14:00.110 INFO:tasks.workunit.client.0.smithi071.stdout:1/423: dwrite db/d34/d68/f80 [4194304,4194304] 0 2024-04-23T02:14:00.110 INFO:tasks.workunit.client.0.smithi071.stdout:0/433: sync 2024-04-23T02:14:00.110 INFO:tasks.workunit.client.0.smithi071.stdout:6/359: creat d6/d1f/d4e/f74 x:0 0 0 2024-04-23T02:14:00.111 INFO:tasks.workunit.client.0.smithi071.stdout:4/447: rename d2/f64 to d2/d2a/d2d/d39/d42/f96 0 2024-04-23T02:14:00.112 INFO:tasks.workunit.client.1.smithi192.stdout:4/644: mknod de/d15/d53/d81/dac/cda 0 2024-04-23T02:14:00.112 INFO:tasks.workunit.client.0.smithi071.stdout:2/365: dread d0/d2/d26/d3a/f3d [0,4194304] 0 2024-04-23T02:14:00.115 INFO:tasks.workunit.client.1.smithi192.stdout:5/478: truncate d2/db/f11 228145 0 2024-04-23T02:14:00.117 INFO:tasks.workunit.client.0.smithi071.stdout:6/360: creat d6/d3e/f75 x:0 0 0 2024-04-23T02:14:00.118 INFO:tasks.workunit.client.0.smithi071.stdout:5/519: dread d3/da/d1a/d1c/f2e [0,4194304] 0 2024-04-23T02:14:00.120 INFO:tasks.workunit.client.0.smithi071.stdout:0/434: mknod d0/c86 0 2024-04-23T02:14:00.122 INFO:tasks.workunit.client.0.smithi071.stdout:3/374: rmdir d7/d27/d2c/d56 0 2024-04-23T02:14:00.123 INFO:tasks.workunit.client.1.smithi192.stdout:0/615: dwrite d6/df/f30 [0,4194304] 0 2024-04-23T02:14:00.123 INFO:tasks.workunit.client.1.smithi192.stdout:4/645: dread de/d15/d53/f83 [0,4194304] 0 2024-04-23T02:14:00.124 INFO:tasks.workunit.client.1.smithi192.stdout:0/616: stat d6/df/d13/d25/d94 0 2024-04-23T02:14:00.124 INFO:tasks.workunit.client.1.smithi192.stdout:6/584: dread d3/d18/f94 [0,4194304] 0 2024-04-23T02:14:00.124 INFO:tasks.workunit.client.1.smithi192.stdout:0/617: chown d6/df/d13/d25/d86/da5 3449 1 2024-04-23T02:14:00.126 INFO:tasks.workunit.client.1.smithi192.stdout:7/484: symlink d5/d43/l8d 0 2024-04-23T02:14:00.126 INFO:tasks.workunit.client.0.smithi071.stdout:6/361: symlink d6/d22/l76 0 2024-04-23T02:14:00.127 INFO:tasks.workunit.client.0.smithi071.stdout:4/448: chown d2/d2a/d2d/d39/d42/d49/d67/d1d/f35 870 1 2024-04-23T02:14:00.128 INFO:tasks.workunit.client.1.smithi192.stdout:2/469: rename d8/d26/d74/l8f to d8/d26/d27/d57/d69/l90 0 2024-04-23T02:14:00.130 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:59 smithi192 ceph-mon[49662]: from='client.14642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:14:00.131 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:59 smithi192 ceph-mon[49662]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:14:00.131 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:13:59 smithi192 ceph-mon[49662]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:14:00.131 INFO:tasks.workunit.client.1.smithi192.stdout:2/470: read - d8/f86 zero size 2024-04-23T02:14:00.132 INFO:tasks.workunit.client.0.smithi071.stdout:5/520: chown d3/da/d1a/db8/c63 12176263 1 2024-04-23T02:14:00.133 INFO:tasks.workunit.client.1.smithi192.stdout:0/618: symlink d6/df/d47/d55/lce 0 2024-04-23T02:14:00.134 INFO:tasks.workunit.client.0.smithi071.stdout:1/424: dwrite db/d34/d68/f76 [0,4194304] 0 2024-04-23T02:14:00.134 INFO:tasks.workunit.client.0.smithi071.stdout:6/362: dread - d6/d1f/f32 zero size 2024-04-23T02:14:00.136 INFO:tasks.workunit.client.0.smithi071.stdout:7/414: dwrite d3/d17/d41/d34/d59/f72 [0,4194304] 0 2024-04-23T02:14:00.139 INFO:tasks.workunit.client.1.smithi192.stdout:5/479: link d2/d6/d2d/d41/f97 d2/db/d16/d48/f9e 0 2024-04-23T02:14:00.140 INFO:tasks.workunit.client.1.smithi192.stdout:4/646: creat de/d15/d2a/d74/d92/da0/dca/d49/d70/d73/fdb x:0 0 0 2024-04-23T02:14:00.141 INFO:tasks.workunit.client.0.smithi071.stdout:2/366: dwrite d0/d2/d26/df/f65 [0,4194304] 0 2024-04-23T02:14:00.141 INFO:tasks.workunit.client.0.smithi071.stdout:6/363: mkdir d6/d17/d77 0 2024-04-23T02:14:00.142 INFO:tasks.workunit.client.1.smithi192.stdout:5/480: fsync d2/d6/d2d/d41/f38 0 2024-04-23T02:14:00.142 INFO:tasks.workunit.client.1.smithi192.stdout:9/450: dread d0/d9/f26 [0,4194304] 0 2024-04-23T02:14:00.144 INFO:tasks.workunit.client.1.smithi192.stdout:7/485: link d5/d43/d4c/d5c/l7c d5/d17/d1c/d59/l8e 0 2024-04-23T02:14:00.144 INFO:tasks.workunit.client.0.smithi071.stdout:5/521: symlink d3/d34/d6b/d81/d5a/da7/lcc 0 2024-04-23T02:14:00.146 INFO:tasks.workunit.client.1.smithi192.stdout:0/619: dwrite d6/df/d35/d49/f54 [4194304,4194304] 0 2024-04-23T02:14:00.146 INFO:tasks.workunit.client.1.smithi192.stdout:9/451: fdatasync d0/d9/d44/d4f/f8e 0 2024-04-23T02:14:00.147 INFO:tasks.workunit.client.0.smithi071.stdout:4/449: truncate d2/d2a/d2d/d39/d42/d49/d91/f92 73750 0 2024-04-23T02:14:00.147 INFO:tasks.workunit.client.0.smithi071.stdout:5/522: fsync d3/da/f33 0 2024-04-23T02:14:00.148 INFO:tasks.workunit.client.1.smithi192.stdout:0/620: dread - d6/d4b/f6c zero size 2024-04-23T02:14:00.150 INFO:tasks.workunit.client.1.smithi192.stdout:5/481: creat d2/db/d16/d1b/f9f x:0 0 0 2024-04-23T02:14:00.151 INFO:tasks.workunit.client.0.smithi071.stdout:1/425: mkdir db/df/d13/d72/d86 0 2024-04-23T02:14:00.151 INFO:tasks.workunit.client.0.smithi071.stdout:2/367: read - d0/d2/d26/df/f69 zero size 2024-04-23T02:14:00.152 INFO:tasks.workunit.client.0.smithi071.stdout:5/523: write d3/da/d1a/d1c/f39 [505705,29288] 0 2024-04-23T02:14:00.158 INFO:tasks.workunit.client.1.smithi192.stdout:7/486: dwrite d5/d17/d1c/d26/f86 [0,4194304] 0 2024-04-23T02:14:00.159 INFO:tasks.workunit.client.0.smithi071.stdout:6/364: rename d6/d17/f36 to d6/d17/f78 0 2024-04-23T02:14:00.159 INFO:tasks.workunit.client.1.smithi192.stdout:6/585: write d3/d15/d1b/f2a [3680741,111905] 0 2024-04-23T02:14:00.160 INFO:tasks.workunit.client.0.smithi071.stdout:2/368: creat d0/d2/d26/d3a/f77 x:0 0 0 2024-04-23T02:14:00.161 INFO:tasks.workunit.client.1.smithi192.stdout:9/452: mkdir d0/d9/d44/d51/d5b/d96/da0 0 2024-04-23T02:14:00.163 INFO:tasks.workunit.client.0.smithi071.stdout:6/365: fsync d6/d3e/f4a 0 2024-04-23T02:14:00.164 INFO:tasks.workunit.client.1.smithi192.stdout:9/453: stat d0/d9/d44/d74/f8a 0 2024-04-23T02:14:00.164 INFO:tasks.workunit.client.0.smithi071.stdout:7/415: creat d3/d17/d1c/f8e x:0 0 0 2024-04-23T02:14:00.165 INFO:tasks.workunit.client.1.smithi192.stdout:0/621: dwrite d6/df/d13/d25/d26/f82 [0,4194304] 0 2024-04-23T02:14:00.165 INFO:tasks.workunit.client.0.smithi071.stdout:4/450: write d2/d2a/d2d/d39/d42/d49/d67/f1a [3854893,15725] 0 2024-04-23T02:14:00.170 INFO:tasks.workunit.client.0.smithi071.stdout:4/451: readlink d2/d2a/d2d/l37 0 2024-04-23T02:14:00.170 INFO:tasks.workunit.client.0.smithi071.stdout:1/426: dwrite db/df/d13/d2f/f3c [4194304,4194304] 0 2024-04-23T02:14:00.171 INFO:tasks.workunit.client.1.smithi192.stdout:6/586: mkdir d3/d28/d79/dbe 0 2024-04-23T02:14:00.171 INFO:tasks.workunit.client.1.smithi192.stdout:5/482: rmdir d2/d2e/d88 0 2024-04-23T02:14:00.174 INFO:tasks.workunit.client.0.smithi071.stdout:5/524: dwrite d3/da/d1a/f1b [0,4194304] 0 2024-04-23T02:14:00.175 INFO:tasks.workunit.client.1.smithi192.stdout:7/487: chown d5/d17/d1c/f3e 1180187 1 2024-04-23T02:14:00.175 INFO:tasks.workunit.client.1.smithi192.stdout:9/454: symlink d0/d9/d44/d58/d89/la1 0 2024-04-23T02:14:00.177 INFO:tasks.workunit.client.1.smithi192.stdout:7/488: chown d5/d43/l4d 419 1 2024-04-23T02:14:00.178 INFO:tasks.workunit.client.1.smithi192.stdout:2/471: truncate d8/d25/f32 1423496 0 2024-04-23T02:14:00.179 INFO:tasks.workunit.client.1.smithi192.stdout:4/647: write de/d15/d2a/d74/d92/da0/dca/d34/d7c/da1/fbe [81899,123297] 0 2024-04-23T02:14:00.179 INFO:tasks.workunit.client.1.smithi192.stdout:9/455: chown d0/d9/d44/d74/d3c/d5f/f7e 53065244 1 2024-04-23T02:14:00.180 INFO:tasks.workunit.client.1.smithi192.stdout:9/456: stat d0/d9/d44/f49 0 2024-04-23T02:14:00.182 INFO:tasks.workunit.client.0.smithi071.stdout:5/525: mknod d3/da/ccd 0 2024-04-23T02:14:00.182 INFO:tasks.workunit.client.0.smithi071.stdout:3/375: dread d7/d27/d2c/f57 [4194304,4194304] 0 2024-04-23T02:14:00.182 INFO:tasks.workunit.client.0.smithi071.stdout:1/427: creat db/d34/d68/f87 x:0 0 0 2024-04-23T02:14:00.185 INFO:tasks.workunit.client.1.smithi192.stdout:9/457: dread - d0/d9/d44/d74/d3c/d5f/f7e zero size 2024-04-23T02:14:00.185 INFO:tasks.workunit.client.1.smithi192.stdout:2/472: creat d8/f91 x:0 0 0 2024-04-23T02:14:00.187 INFO:tasks.workunit.client.0.smithi071.stdout:3/376: readlink d7/d8/d42/d5f/l58 0 2024-04-23T02:14:00.187 INFO:tasks.workunit.client.0.smithi071.stdout:2/369: dread d0/d2/d45/f5a [0,4194304] 0 2024-04-23T02:14:00.188 INFO:tasks.workunit.client.0.smithi071.stdout:1/428: mknod db/df/d13/d2f/d48/c88 0 2024-04-23T02:14:00.189 INFO:tasks.workunit.client.0.smithi071.stdout:5/526: unlink d3/da/d1a/d6d/ca4 0 2024-04-23T02:14:00.190 INFO:tasks.workunit.client.0.smithi071.stdout:4/452: dread d2/d2a/d2d/d39/f46 [0,4194304] 0 2024-04-23T02:14:00.192 INFO:tasks.workunit.client.1.smithi192.stdout:2/473: mknod d8/d14/c92 0 2024-04-23T02:14:00.192 INFO:tasks.workunit.client.0.smithi071.stdout:2/370: rmdir d0/d2/d34/d55 39 2024-04-23T02:14:00.192 INFO:tasks.workunit.client.0.smithi071.stdout:7/416: dwrite d3/d17/f1f [0,4194304] 0 2024-04-23T02:14:00.193 INFO:tasks.workunit.client.1.smithi192.stdout:7/489: dwrite d5/d43/f6e [0,4194304] 0 2024-04-23T02:14:00.196 INFO:tasks.workunit.client.1.smithi192.stdout:2/474: creat d8/d26/d27/d57/f93 x:0 0 0 2024-04-23T02:14:00.200 INFO:tasks.workunit.client.1.smithi192.stdout:2/475: mknod d8/d26/d27/c94 0 2024-04-23T02:14:00.200 INFO:tasks.workunit.client.1.smithi192.stdout:2/476: stat d8/f1d 0 2024-04-23T02:14:00.201 INFO:tasks.workunit.client.0.smithi071.stdout:3/377: read d7/d22/d23/f24 [1662660,67776] 0 2024-04-23T02:14:00.202 INFO:tasks.workunit.client.1.smithi192.stdout:2/477: readlink d8/d14/l21 0 2024-04-23T02:14:00.205 INFO:tasks.workunit.client.1.smithi192.stdout:0/622: write d6/df/d13/d25/d3a/f4f [4102075,105561] 0 2024-04-23T02:14:00.206 INFO:tasks.workunit.client.0.smithi071.stdout:7/417: rmdir d3/d17/d41/d34/d39 39 2024-04-23T02:14:00.213 INFO:tasks.workunit.client.1.smithi192.stdout:2/478: creat d8/d26/d29/d80/f95 x:0 0 0 2024-04-23T02:14:00.213 INFO:tasks.workunit.client.0.smithi071.stdout:4/453: dread d2/d2a/d2d/d39/d42/d49/d67/d7e/f81 [0,4194304] 0 2024-04-23T02:14:00.213 INFO:tasks.workunit.client.0.smithi071.stdout:5/527: dwrite d3/da/d1a/d6d/fba [0,4194304] 0 2024-04-23T02:14:00.215 INFO:tasks.workunit.client.1.smithi192.stdout:0/623: rmdir d6/df/d56/db1/dc2 0 2024-04-23T02:14:00.215 INFO:tasks.workunit.client.0.smithi071.stdout:7/418: truncate d3/d17/d41/d34/d39/d3b/f84 837407 0 2024-04-23T02:14:00.216 INFO:tasks.workunit.client.0.smithi071.stdout:5/528: write d3/da/d1a/d6d/fbe [950458,88979] 0 2024-04-23T02:14:00.217 INFO:tasks.workunit.client.0.smithi071.stdout:1/429: link db/d34/f59 db/df/d5f/d57/f89 0 2024-04-23T02:14:00.221 INFO:tasks.workunit.client.0.smithi071.stdout:3/378: getdents d7/d8/d42/d5f/d68/d3b/d4c/d74 0 2024-04-23T02:14:00.222 INFO:tasks.workunit.client.0.smithi071.stdout:4/454: write d2/d2a/d2d/f66 [1948924,62302] 0 2024-04-23T02:14:00.222 INFO:tasks.workunit.client.1.smithi192.stdout:6/587: dwrite d3/d28/f56 [0,4194304] 0 2024-04-23T02:14:00.222 INFO:tasks.workunit.client.1.smithi192.stdout:0/624: write d6/df/f30 [75650,710] 0 2024-04-23T02:14:00.223 INFO:tasks.workunit.client.0.smithi071.stdout:5/529: chown d3/da/d1a/d6d/f9a 493571298 1 2024-04-23T02:14:00.225 INFO:tasks.workunit.client.1.smithi192.stdout:6/588: chown d3/d15/d1b/d66/c7d 101217 1 2024-04-23T02:14:00.225 INFO:tasks.workunit.client.0.smithi071.stdout:6/366: dread d6/d1f/d46/f5a [0,4194304] 0 2024-04-23T02:14:00.226 INFO:tasks.workunit.client.1.smithi192.stdout:2/479: rename d8/d14/f67 to d8/f96 0 2024-04-23T02:14:00.227 INFO:tasks.workunit.client.1.smithi192.stdout:9/458: dwrite d0/d9/d44/d74/f23 [0,4194304] 0 2024-04-23T02:14:00.227 INFO:tasks.workunit.client.0.smithi071.stdout:7/419: rename d3/f3d to d3/d17/d1c/f8f 0 2024-04-23T02:14:00.228 INFO:tasks.workunit.client.1.smithi192.stdout:5/483: dread d2/db/d16/d1b/f54 [0,4194304] 0 2024-04-23T02:14:00.229 INFO:tasks.workunit.client.1.smithi192.stdout:0/625: stat d6/df/d13/d25/d3a/f58 0 2024-04-23T02:14:00.235 INFO:tasks.workunit.client.1.smithi192.stdout:7/490: dwrite d5/d17/d1c/d57/f46 [4194304,4194304] 0 2024-04-23T02:14:00.236 INFO:tasks.workunit.client.0.smithi071.stdout:7/420: read d3/d17/d1c/f3f [2050682,64706] 0 2024-04-23T02:14:00.236 INFO:tasks.workunit.client.0.smithi071.stdout:3/379: mknod d7/d10/d64/c7d 0 2024-04-23T02:14:00.238 INFO:tasks.workunit.client.0.smithi071.stdout:2/371: truncate d0/d2/d3c/f5f 2321219 0 2024-04-23T02:14:00.239 INFO:tasks.workunit.client.1.smithi192.stdout:7/491: write d5/d17/d1c/f74 [510151,66499] 0 2024-04-23T02:14:00.240 INFO:tasks.workunit.client.0.smithi071.stdout:6/367: mkdir d6/d22/d79 0 2024-04-23T02:14:00.241 INFO:tasks.workunit.client.0.smithi071.stdout:1/430: dwrite db/d34/f61 [4194304,4194304] 0 2024-04-23T02:14:00.242 INFO:tasks.workunit.client.0.smithi071.stdout:3/380: fsync d7/d8/f11 0 2024-04-23T02:14:00.242 INFO:tasks.workunit.client.1.smithi192.stdout:7/492: chown d5/d17/d1c/d57/l23 1696151767 1 2024-04-23T02:14:00.244 INFO:tasks.workunit.client.1.smithi192.stdout:2/480: unlink d8/d14/d51/f56 0 2024-04-23T02:14:00.246 INFO:tasks.workunit.client.1.smithi192.stdout:0/626: truncate d6/df/d47/f4c 1205085 0 2024-04-23T02:14:00.246 INFO:tasks.workunit.client.0.smithi071.stdout:6/368: chown d6/d1f/d5d/f5f 639 1 2024-04-23T02:14:00.248 INFO:tasks.workunit.client.0.smithi071.stdout:3/381: dread - d7/d8/d42/d5f/d68/d3b/d4c/f60 zero size 2024-04-23T02:14:00.249 INFO:tasks.workunit.client.0.smithi071.stdout:6/369: chown d6/f5c 0 1 2024-04-23T02:14:00.249 INFO:tasks.workunit.client.0.smithi071.stdout:5/530: rename d3/d34/d6b/d81/d5a/da7/dab to d3/d1d/d43/d4b/dce 0 2024-04-23T02:14:00.249 INFO:tasks.workunit.client.1.smithi192.stdout:2/481: unlink d8/d26/d27/l88 0 2024-04-23T02:14:00.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.244+0000 7f3fecfe9700 1 -- 172.21.15.71:0/2550296862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8109130 msgr2=0x7f3fe8109550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:00.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.244+0000 7f3fecfe9700 1 --2- 172.21.15.71:0/2550296862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8109130 0x7f3fe8109550 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800aa20 tx=0x7f3fd800ad30 comp rx=0 tx=0).stop 2024-04-23T02:14:00.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.245+0000 7f3fecfe9700 1 -- 172.21.15.71:0/2550296862 shutdown_connections 2024-04-23T02:14:00.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.246+0000 7f3fecfe9700 1 --2- 172.21.15.71:0/2550296862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3fe8107d90 0x7f3fe8108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:00.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.246+0000 7f3fecfe9700 1 --2- 172.21.15.71:0/2550296862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8109130 0x7f3fe8109550 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:00.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.246+0000 7f3fecfe9700 1 -- 172.21.15.71:0/2550296862 >> 172.21.15.71:0/2550296862 conn(0x7f3fe806da40 msgr2=0x7f3fe806fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:14:00.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.246+0000 7f3fecfe9700 1 -- 172.21.15.71:0/2550296862 shutdown_connections 2024-04-23T02:14:00.251 INFO:tasks.workunit.client.0.smithi071.stdout:1/431: unlink f3 0 2024-04-23T02:14:00.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.246+0000 7f3fecfe9700 1 -- 172.21.15.71:0/2550296862 wait complete. 2024-04-23T02:14:00.251 INFO:tasks.workunit.client.0.smithi071.stdout:4/455: rename d2/d2a/d2d/d39 to d2/d2a/d2d/d39/d42/d49/d67/d3e/d97 22 2024-04-23T02:14:00.252 INFO:tasks.workunit.client.0.smithi071.stdout:2/372: write d0/d2/d26/d14/f42 [3231276,34139] 0 2024-04-23T02:14:00.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.247+0000 7f3fecfe9700 1 Processor -- start 2024-04-23T02:14:00.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.247+0000 7f3fecfe9700 1 -- start start 2024-04-23T02:14:00.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.247+0000 7f3fecfe9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3fe8107d90 0x7f3fe81add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:14:00.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.247+0000 7f3fecfe9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8109130 0x7f3fe81ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:14:00.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.247+0000 7f3fecfe9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe81ae9b0 con 0x7f3fe8109130 2024-04-23T02:14:00.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.247+0000 7f3fecfe9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe81b3070 con 0x7f3fe8107d90 2024-04-23T02:14:00.253 INFO:tasks.workunit.client.1.smithi192.stdout:7/493: link d5/d43/l8d d5/d17/d1c/d4a/l8f 0 2024-04-23T02:14:00.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.249+0000 7f3fe659c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3fe8107d90 0x7f3fe81add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:14:00.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.249+0000 7f3fe5d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8109130 0x7f3fe81ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:14:00.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.250+0000 7f3fe5d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8109130 0x7f3fe81ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50220/0 (socket says 172.21.15.71:50220) 2024-04-23T02:14:00.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.250+0000 7f3fe5d9b700 1 -- 172.21.15.71:0/3801931604 learned_addr learned my addr 172.21.15.71:0/3801931604 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:14:00.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.250+0000 7f3fe5d9b700 1 -- 172.21.15.71:0/3801931604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3fe8107d90 msgr2=0x7f3fe81add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:00.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.250+0000 7f3fe5d9b700 1 --2- 172.21.15.71:0/3801931604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3fe8107d90 0x7f3fe81add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:00.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.250+0000 7f3fe5d9b700 1 -- 172.21.15.71:0/3801931604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd800a700 con 0x7f3fe8109130 2024-04-23T02:14:00.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.250+0000 7f3fe5d9b700 1 --2- 172.21.15.71:0/3801931604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8109130 0x7f3fe81ae340 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00b4d0 tx=0x7f3fdc00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:14:00.255 INFO:tasks.workunit.client.1.smithi192.stdout:2/482: unlink f7 0 2024-04-23T02:14:00.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.250+0000 7f3fd77fe700 1 -- 172.21.15.71:0/3801931604 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fdc011800 con 0x7f3fe8109130 2024-04-23T02:14:00.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.250+0000 7f3fd77fe700 1 -- 172.21.15.71:0/3801931604 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f3fdc011df0 con 0x7f3fe8109130 2024-04-23T02:14:00.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.250+0000 7f3fecfe9700 1 -- 172.21.15.71:0/3801931604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe81b3270 con 0x7f3fe8109130 2024-04-23T02:14:00.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.250+0000 7f3fecfe9700 1 -- 172.21.15.71:0/3801931604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe81b3770 con 0x7f3fe8109130 2024-04-23T02:14:00.256 INFO:tasks.workunit.client.0.smithi071.stdout:6/370: dread - d6/d1f/d46/f56 zero size 2024-04-23T02:14:00.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.251+0000 7f3fd77fe700 1 -- 172.21.15.71:0/3801931604 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fdc010e70 con 0x7f3fe8109130 2024-04-23T02:14:00.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.252+0000 7f3fecfe9700 1 -- 172.21.15.71:0/3801931604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc8005320 con 0x7f3fe8109130 2024-04-23T02:14:00.257 INFO:tasks.workunit.client.0.smithi071.stdout:2/373: fdatasync d0/d2/d26/d3a/f77 0 2024-04-23T02:14:00.257 INFO:tasks.workunit.client.1.smithi192.stdout:7/494: creat d5/d17/d1c/d57/f90 x:0 0 0 2024-04-23T02:14:00.258 INFO:tasks.workunit.client.1.smithi192.stdout:2/483: mknod d8/d26/d27/d31/c97 0 2024-04-23T02:14:00.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.254+0000 7f3fd77fe700 1 -- 172.21.15.71:0/3801931604 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) v1 ==== 90681+0+0 (secure 0 0 0) 0x7f3fdc011960 con 0x7f3fe8109130 2024-04-23T02:14:00.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.254+0000 7f3fd77fe700 1 --2- 172.21.15.71:0/3801931604 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f3fd006cab0 0x7f3fd006ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:14:00.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.254+0000 7f3fd77fe700 1 -- 172.21.15.71:0/3801931604 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f3fdc08bee0 con 0x7f3fe8109130 2024-04-23T02:14:00.259 INFO:tasks.workunit.client.0.smithi071.stdout:7/421: link d3/d17/d1c/f2c d3/d17/d41/d34/d59/d5f/f90 0 2024-04-23T02:14:00.259 INFO:tasks.workunit.client.0.smithi071.stdout:2/374: chown d0/d2/d3c 5 1 2024-04-23T02:14:00.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.255+0000 7f3fe659c700 1 --2- 172.21.15.71:0/3801931604 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f3fd006cab0 0x7f3fd006ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:14:00.260 INFO:tasks.workunit.client.0.smithi071.stdout:4/456: chown d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/l56 14 1 2024-04-23T02:14:00.260 INFO:tasks.workunit.client.1.smithi192.stdout:6/589: dwrite d3/d18/d58/f75 [0,4194304] 0 2024-04-23T02:14:00.260 INFO:tasks.workunit.client.1.smithi192.stdout:7/495: read - d5/d17/d1c/d57/f90 zero size 2024-04-23T02:14:00.261 INFO:tasks.workunit.client.1.smithi192.stdout:2/484: mknod d8/d26/d81/c98 0 2024-04-23T02:14:00.262 INFO:tasks.workunit.client.1.smithi192.stdout:7/496: chown d5/d17/d1c/d59/d5a/d41/d50/d3d/c79 7275832 1 2024-04-23T02:14:00.263 INFO:tasks.workunit.client.0.smithi071.stdout:7/422: mknod d3/d17/d41/d34/d6e/c91 0 2024-04-23T02:14:00.263 INFO:tasks.workunit.client.0.smithi071.stdout:3/382: dread d7/d8/f1b [0,4194304] 0 2024-04-23T02:14:00.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.259+0000 7f3fe659c700 1 --2- 172.21.15.71:0/3801931604 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f3fd006cab0 0x7f3fd006ef70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8000c00 tx=0x7f3fd800af70 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:14:00.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.260+0000 7f3fd77fe700 1 -- 172.21.15.71:0/3801931604 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f3fdc059b30 con 0x7f3fe8109130 2024-04-23T02:14:00.264 INFO:tasks.workunit.client.1.smithi192.stdout:7/497: write d5/d43/d4c/d5c/d67/f82 [551392,111434] 0 2024-04-23T02:14:00.265 INFO:tasks.workunit.client.0.smithi071.stdout:4/457: readlink d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/l4c 0 2024-04-23T02:14:00.265 INFO:tasks.workunit.client.1.smithi192.stdout:6/590: fdatasync d3/d15/f4b 0 2024-04-23T02:14:00.267 INFO:tasks.workunit.client.0.smithi071.stdout:5/531: dwrite d3/da/d1a/d6d/fba [0,4194304] 0 2024-04-23T02:14:00.267 INFO:tasks.workunit.client.0.smithi071.stdout:3/383: write d7/d1d/f73 [514747,51162] 0 2024-04-23T02:14:00.270 INFO:tasks.workunit.client.0.smithi071.stdout:2/375: dread d0/d2/d45/f52 [0,4194304] 0 2024-04-23T02:14:00.271 INFO:tasks.workunit.client.1.smithi192.stdout:7/498: unlink d5/d17/d1c/d59/d5a/f32 0 2024-04-23T02:14:00.273 INFO:tasks.workunit.client.0.smithi071.stdout:4/458: mknod d2/d2a/d2d/d39/d42/d49/d67/d1d/c98 0 2024-04-23T02:14:00.275 INFO:tasks.workunit.client.0.smithi071.stdout:3/384: mkdir d7/d22/d23/d26/d7e 0 2024-04-23T02:14:00.276 INFO:tasks.workunit.client.1.smithi192.stdout:2/485: dwrite d8/d26/f61 [0,4194304] 0 2024-04-23T02:14:00.276 INFO:tasks.workunit.client.1.smithi192.stdout:6/591: creat d3/d15/d1b/d21/d52/db2/fbf x:0 0 0 2024-04-23T02:14:00.277 INFO:tasks.workunit.client.0.smithi071.stdout:7/423: mkdir d3/d17/d1c/d2a/d73/d83/d92 0 2024-04-23T02:14:00.280 INFO:tasks.workunit.client.0.smithi071.stdout:6/371: dwrite d6/f18 [0,4194304] 0 2024-04-23T02:14:00.280 INFO:tasks.workunit.client.1.smithi192.stdout:7/499: mknod d5/d17/d1c/d59/c91 0 2024-04-23T02:14:00.281 INFO:tasks.workunit.client.1.smithi192.stdout:9/459: dwrite d0/d9/d44/d51/d5b/f67 [0,4194304] 0 2024-04-23T02:14:00.282 INFO:tasks.workunit.client.1.smithi192.stdout:6/592: write d3/d15/d5b/f69 [1611673,116218] 0 2024-04-23T02:14:00.283 INFO:tasks.workunit.client.0.smithi071.stdout:7/424: unlink d3/l15 0 2024-04-23T02:14:00.284 INFO:tasks.workunit.client.0.smithi071.stdout:4/459: write d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c/f7c [83884,97122] 0 2024-04-23T02:14:00.284 INFO:tasks.workunit.client.0.smithi071.stdout:5/532: dwrite d3/d1d/daf/d83/fc9 [0,4194304] 0 2024-04-23T02:14:00.286 INFO:tasks.workunit.client.0.smithi071.stdout:6/372: write d6/d1f/d5d/f73 [375787,24756] 0 2024-04-23T02:14:00.286 INFO:tasks.workunit.client.1.smithi192.stdout:5/484: write d2/db/d16/f26 [5083192,4823] 0 2024-04-23T02:14:00.288 INFO:tasks.workunit.client.0.smithi071.stdout:2/376: mknod d0/d2/d34/c78 0 2024-04-23T02:14:00.288 INFO:tasks.workunit.client.0.smithi071.stdout:4/460: dread - d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d29/f83 zero size 2024-04-23T02:14:00.289 INFO:tasks.workunit.client.1.smithi192.stdout:8/667: sync 2024-04-23T02:14:00.290 INFO:tasks.workunit.client.0.smithi071.stdout:2/377: write d0/d2/d26/f4c [2457435,127663] 0 2024-04-23T02:14:00.292 INFO:tasks.workunit.client.0.smithi071.stdout:2/378: chown d0/d2/d26/df/c37 3719 1 2024-04-23T02:14:00.293 INFO:tasks.workunit.client.1.smithi192.stdout:8/668: dread - d1/d12/d23/d4b/d73/d45/d46/fd6 zero size 2024-04-23T02:14:00.293 INFO:tasks.workunit.client.0.smithi071.stdout:3/385: symlink d7/d8/l7f 0 2024-04-23T02:14:00.296 INFO:tasks.workunit.client.1.smithi192.stdout:7/500: link d5/d17/d1c/d4a/f84 d5/d17/f92 0 2024-04-23T02:14:00.296 INFO:tasks.workunit.client.1.smithi192.stdout:5/485: write d2/db/d16/d1b/d3d/f67 [2315705,106880] 0 2024-04-23T02:14:00.296 INFO:tasks.workunit.client.0.smithi071.stdout:5/533: rmdir d3/da/d1a 39 2024-04-23T02:14:00.298 INFO:tasks.workunit.client.0.smithi071.stdout:3/386: dread d7/d27/d2c/f57 [4194304,4194304] 0 2024-04-23T02:14:00.299 INFO:tasks.workunit.client.1.smithi192.stdout:2/486: dwrite d8/d26/d29/d80/f95 [0,4194304] 0 2024-04-23T02:14:00.300 INFO:tasks.workunit.client.1.smithi192.stdout:7/501: fdatasync d5/d43/d4c/d4e/f7f 0 2024-04-23T02:14:00.300 INFO:tasks.workunit.client.1.smithi192.stdout:9/460: dread d0/d9/d44/d51/d5b/d61/f6a [0,4194304] 0 2024-04-23T02:14:00.303 INFO:tasks.workunit.client.1.smithi192.stdout:9/461: chown d0/d9/d44/d74/d3c/d5f 3471774 1 2024-04-23T02:14:00.305 INFO:tasks.workunit.client.1.smithi192.stdout:2/487: mkdir d8/d26/d27/d57/d69/d99 0 2024-04-23T02:14:00.306 INFO:tasks.workunit.client.1.smithi192.stdout:9/462: write d0/d9/f3a [778552,35447] 0 2024-04-23T02:14:00.310 INFO:tasks.workunit.client.1.smithi192.stdout:8/669: chown d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d8b 177 1 2024-04-23T02:14:00.312 INFO:tasks.workunit.client.0.smithi071.stdout:3/387: mkdir d7/d8/d42/d80 0 2024-04-23T02:14:00.315 INFO:tasks.workunit.client.0.smithi071.stdout:5/534: creat d3/da/fcf x:0 0 0 2024-04-23T02:14:00.321 INFO:tasks.workunit.client.0.smithi071.stdout:6/373: getdents d6 0 2024-04-23T02:14:00.321 INFO:tasks.workunit.client.0.smithi071.stdout:6/374: stat d6/d17/f78 0 2024-04-23T02:14:00.322 INFO:tasks.workunit.client.1.smithi192.stdout:7/502: creat d5/d17/d1c/d59/f93 x:0 0 0 2024-04-23T02:14:00.322 INFO:tasks.workunit.client.1.smithi192.stdout:2/488: unlink d8/f91 0 2024-04-23T02:14:00.323 INFO:tasks.workunit.client.0.smithi071.stdout:5/535: rename d3/d34/d6b/d81 to d3/dd0 0 2024-04-23T02:14:00.324 INFO:tasks.workunit.client.0.smithi071.stdout:6/375: write d6/d1f/d46/f62 [597109,98212] 0 2024-04-23T02:14:00.325 INFO:tasks.workunit.client.1.smithi192.stdout:9/463: rename d0/d9/l72 to d0/d9/d44/d58/la2 0 2024-04-23T02:14:00.327 INFO:tasks.workunit.client.0.smithi071.stdout:1/432: dread f5 [4194304,4194304] 0 2024-04-23T02:14:00.327 INFO:tasks.workunit.client.1.smithi192.stdout:0/627: dread d6/df/d47/d55/f70 [4194304,4194304] 0 2024-04-23T02:14:00.327 INFO:tasks.workunit.client.0.smithi071.stdout:3/388: dwrite d7/d8/d42/d5f/d68/d3b/d4c/f63 [0,4194304] 0 2024-04-23T02:14:00.328 INFO:tasks.workunit.client.1.smithi192.stdout:8/670: creat d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/fec x:0 0 0 2024-04-23T02:14:00.328 INFO:tasks.workunit.client.1.smithi192.stdout:2/489: dread d8/d26/d29/d43/f83 [0,4194304] 0 2024-04-23T02:14:00.329 INFO:tasks.workunit.client.0.smithi071.stdout:5/536: write d3/da/d1a/f2b [2109698,84234] 0 2024-04-23T02:14:00.330 INFO:tasks.workunit.client.1.smithi192.stdout:7/503: symlink d5/d17/d1c/l94 0 2024-04-23T02:14:00.330 INFO:tasks.workunit.client.1.smithi192.stdout:9/464: creat d0/d9/d44/d74/d3c/d5f/d7d/fa3 x:0 0 0 2024-04-23T02:14:00.332 INFO:tasks.workunit.client.0.smithi071.stdout:1/433: write db/df/d13/d2b/f70 [694675,119712] 0 2024-04-23T02:14:00.333 INFO:tasks.workunit.client.0.smithi071.stdout:6/376: rename d6/d17/f78 to d6/d17/d77/f7a 0 2024-04-23T02:14:00.333 INFO:tasks.workunit.client.0.smithi071.stdout:3/389: symlink d7/d8/d42/d47/l81 0 2024-04-23T02:14:00.335 INFO:tasks.workunit.client.0.smithi071.stdout:6/377: read d6/d22/f45 [3001061,50386] 0 2024-04-23T02:14:00.336 INFO:tasks.workunit.client.0.smithi071.stdout:5/537: creat d3/d1d/daf/d10/da5/d5f/fd1 x:0 0 0 2024-04-23T02:14:00.337 INFO:tasks.workunit.client.1.smithi192.stdout:5/486: write d2/db/f11 [822274,101815] 0 2024-04-23T02:14:00.338 INFO:tasks.workunit.client.1.smithi192.stdout:2/490: mknod d8/d26/d27/c9a 0 2024-04-23T02:14:00.340 INFO:tasks.workunit.client.0.smithi071.stdout:1/434: mkdir db/d34/d8a 0 2024-04-23T02:14:00.340 INFO:tasks.workunit.client.0.smithi071.stdout:3/390: creat d7/d10/d38/d67/f82 x:0 0 0 2024-04-23T02:14:00.340 INFO:tasks.workunit.client.1.smithi192.stdout:6/593: dread f1 [0,4194304] 0 2024-04-23T02:14:00.346 INFO:tasks.workunit.client.0.smithi071.stdout:3/391: symlink d7/d8/d42/d5f/d68/d3b/d4c/d74/l83 0 2024-04-23T02:14:00.346 INFO:tasks.workunit.client.1.smithi192.stdout:0/628: dwrite d6/df/d13/d25/d86/da5/faf [0,4194304] 0 2024-04-23T02:14:00.347 INFO:tasks.workunit.client.0.smithi071.stdout:2/379: dread d0/f1e [0,4194304] 0 2024-04-23T02:14:00.348 INFO:tasks.workunit.client.0.smithi071.stdout:5/538: rename d3/dd0 to d3/d1d/d43/dd2 0 2024-04-23T02:14:00.348 INFO:tasks.workunit.client.1.smithi192.stdout:8/671: dwrite d1/d12/d23/d4b/f51 [0,4194304] 0 2024-04-23T02:14:00.348 INFO:tasks.workunit.client.1.smithi192.stdout:5/487: mknod d2/db/d16/d48/d4f/ca0 0 2024-04-23T02:14:00.351 INFO:tasks.workunit.client.0.smithi071.stdout:6/378: getdents d6/d17 0 2024-04-23T02:14:00.353 INFO:tasks.workunit.client.0.smithi071.stdout:3/392: creat d7/d8/d42/d5f/d68/f84 x:0 0 0 2024-04-23T02:14:00.354 INFO:tasks.workunit.client.0.smithi071.stdout:5/539: creat d3/da/d1a/d6d/fd3 x:0 0 0 2024-04-23T02:14:00.356 INFO:tasks.workunit.client.0.smithi071.stdout:2/380: rename d0/d2/d51/d74 to d0/d2/d49/d79 0 2024-04-23T02:14:00.357 INFO:tasks.workunit.client.0.smithi071.stdout:6/379: symlink d6/d1f/d4e/l7b 0 2024-04-23T02:14:00.358 INFO:tasks.workunit.client.1.smithi192.stdout:7/504: truncate d5/d17/d1c/d59/d5a/d41/d50/f69 1678215 0 2024-04-23T02:14:00.359 INFO:tasks.workunit.client.0.smithi071.stdout:6/380: dread - d6/d1f/d4e/f51 zero size 2024-04-23T02:14:00.359 INFO:tasks.workunit.client.0.smithi071.stdout:5/540: fsync d3/d1d/d43/dd2/da9/f96 0 2024-04-23T02:14:00.359 INFO:tasks.workunit.client.1.smithi192.stdout:9/465: write d0/d9/d44/f9d [1019456,110915] 0 2024-04-23T02:14:00.360 INFO:tasks.workunit.client.1.smithi192.stdout:0/629: creat d6/df/d13/d25/d26/fcf x:0 0 0 2024-04-23T02:14:00.360 INFO:tasks.workunit.client.0.smithi071.stdout:2/381: chown d0/d2/d26/l17 38000 1 2024-04-23T02:14:00.360 INFO:tasks.workunit.client.1.smithi192.stdout:6/594: rename d3/d11/l42 to d3/d28/d79/da0/lc0 0 2024-04-23T02:14:00.363 INFO:tasks.workunit.client.0.smithi071.stdout:6/381: mknod d6/c7c 0 2024-04-23T02:14:00.363 INFO:tasks.workunit.client.1.smithi192.stdout:5/488: unlink d2/db/d16/d1b/f7a 0 2024-04-23T02:14:00.365 INFO:tasks.workunit.client.1.smithi192.stdout:6/595: rename d3/d28/d79 to d3/d28/d79/dc1 22 2024-04-23T02:14:00.365 INFO:tasks.workunit.client.0.smithi071.stdout:2/382: rename d0/d2/d26/d14/d1f/c24 to d0/d2/d26/d14/d1f/d36/d6d/c7a 0 2024-04-23T02:14:00.366 INFO:tasks.workunit.client.1.smithi192.stdout:9/466: mknod d0/d98/ca4 0 2024-04-23T02:14:00.366 INFO:tasks.workunit.client.1.smithi192.stdout:2/491: dwrite d8/d14/f15 [0,4194304] 0 2024-04-23T02:14:00.367 INFO:tasks.workunit.client.0.smithi071.stdout:6/382: rmdir d6/d1f/d5d 39 2024-04-23T02:14:00.370 INFO:tasks.workunit.client.0.smithi071.stdout:5/541: unlink d3/da/d1a/db8/c45 0 2024-04-23T02:14:00.371 INFO:tasks.workunit.client.1.smithi192.stdout:8/672: dwrite d1/d12/f39 [0,4194304] 0 2024-04-23T02:14:00.371 INFO:tasks.workunit.client.1.smithi192.stdout:5/489: chown d2/db/d16/d1b/d3d/d40/f8c 402726 1 2024-04-23T02:14:00.371 INFO:tasks.workunit.client.1.smithi192.stdout:7/505: symlink d5/d17/d1c/d26/l95 0 2024-04-23T02:14:00.371 INFO:tasks.workunit.client.1.smithi192.stdout:2/492: readlink d8/lc 0 2024-04-23T02:14:00.376 INFO:tasks.workunit.client.1.smithi192.stdout:8/673: chown d1/df/f9b 227451 1 2024-04-23T02:14:00.376 INFO:tasks.workunit.client.1.smithi192.stdout:9/467: read d0/d9/f3a [441482,60219] 0 2024-04-23T02:14:00.376 INFO:tasks.workunit.client.1.smithi192.stdout:7/506: write d5/d17/d1c/d26/f86 [2967532,29528] 0 2024-04-23T02:14:00.377 INFO:tasks.workunit.client.1.smithi192.stdout:0/630: dread f5 [0,4194304] 0 2024-04-23T02:14:00.377 INFO:tasks.workunit.client.0.smithi071.stdout:6/383: dread d6/d17/d77/f7a [4194304,4194304] 0 2024-04-23T02:14:00.380 INFO:tasks.workunit.client.0.smithi071.stdout:5/542: mknod d3/d1d/daf/cd4 0 2024-04-23T02:14:00.381 INFO:tasks.workunit.client.0.smithi071.stdout:5/543: fsync d3/d1d/daf/d83/fac 0 2024-04-23T02:14:00.382 INFO:tasks.workunit.client.0.smithi071.stdout:6/384: dread f4 [0,4194304] 0 2024-04-23T02:14:00.383 INFO:tasks.workunit.client.1.smithi192.stdout:2/493: rename d8/d14/f4c to d8/d26/d74/d7c/f9b 0 2024-04-23T02:14:00.384 INFO:tasks.workunit.client.1.smithi192.stdout:8/674: rename d1/d9/d8f to d1/d9/d8f/ded 22 2024-04-23T02:14:00.385 INFO:tasks.workunit.client.0.smithi071.stdout:5/544: creat d3/d1d/db9/fd5 x:0 0 0 2024-04-23T02:14:00.386 INFO:tasks.workunit.client.0.smithi071.stdout:6/385: fdatasync d6/d1f/f3b 0 2024-04-23T02:14:00.388 INFO:tasks.workunit.client.1.smithi192.stdout:9/468: dwrite d0/f16 [0,4194304] 0 2024-04-23T02:14:00.389 INFO:tasks.workunit.client.0.smithi071.stdout:2/383: truncate d0/d2/d26/f18 3769569 0 2024-04-23T02:14:00.389 INFO:tasks.workunit.client.1.smithi192.stdout:9/469: readlink d0/d9/d44/d74/l4b 0 2024-04-23T02:14:00.390 INFO:tasks.workunit.client.1.smithi192.stdout:9/470: write d0/d9/d44/f45 [1786191,72239] 0 2024-04-23T02:14:00.390 INFO:tasks.workunit.client.0.smithi071.stdout:5/545: creat d3/d1d/daf/d10/da5/d5f/fd6 x:0 0 0 2024-04-23T02:14:00.391 INFO:tasks.workunit.client.0.smithi071.stdout:3/393: dwrite d7/d8/fc [0,4194304] 0 2024-04-23T02:14:00.392 INFO:tasks.workunit.client.0.smithi071.stdout:6/386: mknod d6/d1f/d46/c7d 0 2024-04-23T02:14:00.395 INFO:tasks.workunit.client.0.smithi071.stdout:3/394: write d7/d8/d42/d5f/d68/f84 [170884,118395] 0 2024-04-23T02:14:00.395 INFO:tasks.workunit.client.0.smithi071.stdout:2/384: fdatasync d0/d2/d34/f50 0 2024-04-23T02:14:00.395 INFO:tasks.workunit.client.0.smithi071.stdout:6/387: read - d6/d1f/f32 zero size 2024-04-23T02:14:00.401 INFO:tasks.workunit.client.0.smithi071.stdout:6/388: symlink d6/d1f/d46/l7e 0 2024-04-23T02:14:00.404 INFO:tasks.workunit.client.0.smithi071.stdout:5/546: dwrite d3/da/d1a/db8/f2d [0,4194304] 0 2024-04-23T02:14:00.410 INFO:tasks.workunit.client.0.smithi071.stdout:3/395: getdents d7/d8/d42/d5f 0 2024-04-23T02:14:00.413 INFO:tasks.workunit.client.0.smithi071.stdout:5/547: symlink d3/d1d/d43/d4b/ld7 0 2024-04-23T02:14:00.413 INFO:tasks.workunit.client.0.smithi071.stdout:2/385: dwrite d0/d2/d26/d14/d1f/f25 [4194304,4194304] 0 2024-04-23T02:14:00.417 INFO:tasks.workunit.client.0.smithi071.stdout:5/548: fdatasync d3/d1d/daf/d83/fac 0 2024-04-23T02:14:00.419 INFO:tasks.workunit.client.0.smithi071.stdout:3/396: truncate d7/d8/d42/d5f/d68/d3b/d4c/f65 570674 0 2024-04-23T02:14:00.419 INFO:tasks.workunit.client.0.smithi071.stdout:9/389: sync 2024-04-23T02:14:00.420 INFO:tasks.workunit.client.0.smithi071.stdout:6/389: dwrite d6/d1f/d46/f56 [0,4194304] 0 2024-04-23T02:14:00.422 INFO:tasks.workunit.client.0.smithi071.stdout:9/390: chown d1/d32/d3a/c46 231 1 2024-04-23T02:14:00.422 INFO:tasks.workunit.client.0.smithi071.stdout:3/397: fdatasync d7/d10/f2f 0 2024-04-23T02:14:00.422 INFO:tasks.workunit.client.0.smithi071.stdout:2/386: rename d0/d2/d34/c78 to d0/d2/d51/d56/c7b 0 2024-04-23T02:14:00.423 INFO:tasks.workunit.client.0.smithi071.stdout:5/549: mkdir d3/da/d1a/d1c/d46/d4e/dd8 0 2024-04-23T02:14:00.425 INFO:tasks.workunit.client.0.smithi071.stdout:2/387: dread - d0/d2/d26/d3a/f44 zero size 2024-04-23T02:14:00.425 INFO:tasks.workunit.client.0.smithi071.stdout:3/398: write d7/d8/fc [3916133,43949] 0 2024-04-23T02:14:00.431 INFO:tasks.workunit.client.1.smithi192.stdout:5/490: creat d2/d6/d2d/d41/fa1 x:0 0 0 2024-04-23T02:14:00.443 INFO:tasks.workunit.client.1.smithi192.stdout:0/631: rmdir d6/df/d56/d85/d45 39 2024-04-23T02:14:00.444 INFO:tasks.workunit.client.1.smithi192.stdout:7/507: rename d5/d17/d1c/d57/f46 to d5/d17/d1c/d57/d8b/f96 0 2024-04-23T02:14:00.446 INFO:tasks.workunit.client.1.smithi192.stdout:2/494: symlink d8/d26/d29/d80/l9c 0 2024-04-23T02:14:00.450 INFO:tasks.workunit.client.1.smithi192.stdout:9/471: symlink d0/d9/d44/d51/d5b/la5 0 2024-04-23T02:14:00.450 INFO:tasks.workunit.client.1.smithi192.stdout:5/491: dread d2/f27 [0,4194304] 0 2024-04-23T02:14:00.452 INFO:tasks.workunit.client.1.smithi192.stdout:7/508: truncate d5/d17/d1c/d26/f28 1646290 0 2024-04-23T02:14:00.455 INFO:tasks.workunit.client.1.smithi192.stdout:8/675: link d1/d9/f3f d1/d9/fee 0 2024-04-23T02:14:00.456 INFO:tasks.workunit.client.1.smithi192.stdout:9/472: write d0/d9/d44/f7c [1888693,85241] 0 2024-04-23T02:14:00.457 INFO:tasks.workunit.client.1.smithi192.stdout:7/509: write d5/d17/d1c/d59/d5a/d41/d50/d3d/f83 [915243,62178] 0 2024-04-23T02:14:00.459 INFO:tasks.workunit.client.1.smithi192.stdout:0/632: dwrite d6/df/fc0 [0,4194304] 0 2024-04-23T02:14:00.461 INFO:tasks.workunit.client.1.smithi192.stdout:2/495: link d8/d14/f1a d8/d25/f9d 0 2024-04-23T02:14:00.464 INFO:tasks.workunit.client.1.smithi192.stdout:0/633: mkdir d6/df/d13/d25/d94/dd0 0 2024-04-23T02:14:00.468 INFO:tasks.workunit.client.1.smithi192.stdout:2/496: symlink d8/d26/l9e 0 2024-04-23T02:14:00.469 INFO:tasks.workunit.client.0.smithi071.stdout:3/399: symlink d7/d10/l85 0 2024-04-23T02:14:00.469 INFO:tasks.workunit.client.0.smithi071.stdout:6/390: link d6/d17/c40 d6/d1f/c7f 0 2024-04-23T02:14:00.470 INFO:tasks.workunit.client.0.smithi071.stdout:9/391: link d1/d32/d3a/d5f/f60 d1/d32/d3a/d5f/f83 0 2024-04-23T02:14:00.471 INFO:tasks.workunit.client.1.smithi192.stdout:2/497: readlink d8/d26/l8e 0 2024-04-23T02:14:00.471 INFO:tasks.workunit.client.1.smithi192.stdout:1/299: sync 2024-04-23T02:14:00.472 INFO:tasks.workunit.client.0.smithi071.stdout:3/400: mkdir d7/d86 0 2024-04-23T02:14:00.474 INFO:tasks.workunit.client.0.smithi071.stdout:6/391: chown d6/d1f/d5d/f71 40450 1 2024-04-23T02:14:00.474 INFO:tasks.workunit.client.1.smithi192.stdout:1/300: chown dc/d16/d19/f3a 5099437 1 2024-04-23T02:14:00.475 INFO:tasks.workunit.client.0.smithi071.stdout:6/392: write d6/d1f/f3b [343705,116253] 0 2024-04-23T02:14:00.476 INFO:tasks.workunit.client.1.smithi192.stdout:5/492: write d2/db/d16/d1b/d3d/d40/d49/f58 [503391,100926] 0 2024-04-23T02:14:00.477 INFO:tasks.workunit.client.0.smithi071.stdout:9/392: dread d1/d1a/d39/d42/f49 [0,4194304] 0 2024-04-23T02:14:00.479 INFO:tasks.workunit.client.1.smithi192.stdout:1/301: creat dc/d16/d19/d30/f55 x:0 0 0 2024-04-23T02:14:00.479 INFO:tasks.workunit.client.0.smithi071.stdout:6/393: creat d6/d17/d4d/f80 x:0 0 0 2024-04-23T02:14:00.480 INFO:tasks.workunit.client.1.smithi192.stdout:1/302: fdatasync dc/d16/d19/d30/d32/f47 0 2024-04-23T02:14:00.481 INFO:tasks.workunit.client.0.smithi071.stdout:9/393: creat d1/d32/d3a/f84 x:0 0 0 2024-04-23T02:14:00.481 INFO:tasks.workunit.client.0.smithi071.stdout:6/394: read d6/d22/f45 [2993696,6087] 0 2024-04-23T02:14:00.481 INFO:tasks.workunit.client.1.smithi192.stdout:2/498: creat d8/d26/d29/d43/f9f x:0 0 0 2024-04-23T02:14:00.482 INFO:tasks.workunit.client.1.smithi192.stdout:2/499: dread - d8/d26/d27/d53/f64 zero size 2024-04-23T02:14:00.483 INFO:tasks.workunit.client.1.smithi192.stdout:5/493: rename d2/db/d16/l78 to d2/db/d16/d1b/d3d/d40/d49/la2 0 2024-04-23T02:14:00.484 INFO:tasks.workunit.client.1.smithi192.stdout:2/500: dread d8/d26/d29/d43/f7f [0,4194304] 0 2024-04-23T02:14:00.485 INFO:tasks.workunit.client.0.smithi071.stdout:3/401: creat d7/d8/d42/f87 x:0 0 0 2024-04-23T02:14:00.485 INFO:tasks.workunit.client.1.smithi192.stdout:1/303: unlink dc/f13 0 2024-04-23T02:14:00.485 INFO:tasks.workunit.client.1.smithi192.stdout:2/501: dread - d8/d26/d27/d53/f64 zero size 2024-04-23T02:14:00.486 INFO:tasks.workunit.client.0.smithi071.stdout:2/388: dwrite d0/d2/d26/df/f40 [0,4194304] 0 2024-04-23T02:14:00.486 INFO:tasks.workunit.client.0.smithi071.stdout:9/394: mknod d1/d32/d41/c85 0 2024-04-23T02:14:00.486 INFO:tasks.workunit.client.1.smithi192.stdout:0/634: link d6/df/d47/d55/l5f d6/df/d56/d85/d45/ld1 0 2024-04-23T02:14:00.487 INFO:tasks.workunit.client.1.smithi192.stdout:0/635: stat d6/df/d47/d55/dae 0 2024-04-23T02:14:00.487 INFO:tasks.workunit.client.1.smithi192.stdout:7/510: truncate d5/d17/d1c/d59/d5a/d41/d50/f35 7364131 0 2024-04-23T02:14:00.489 INFO:tasks.workunit.client.1.smithi192.stdout:0/636: read - d6/df/d13/d25/d86/da5/fc8 zero size 2024-04-23T02:14:00.489 INFO:tasks.workunit.client.1.smithi192.stdout:1/304: dread dc/d14/d15/d35/f37 [0,4194304] 0 2024-04-23T02:14:00.490 INFO:tasks.workunit.client.0.smithi071.stdout:9/395: write d1/d1a/f22 [3599471,49284] 0 2024-04-23T02:14:00.490 INFO:tasks.workunit.client.1.smithi192.stdout:5/494: read d2/d32/f35 [1851975,7303] 0 2024-04-23T02:14:00.491 INFO:tasks.workunit.client.0.smithi071.stdout:6/395: dread d6/f18 [0,4194304] 0 2024-04-23T02:14:00.492 INFO:tasks.workunit.client.1.smithi192.stdout:2/502: mkdir d8/d26/d27/d57/da0 0 2024-04-23T02:14:00.493 INFO:tasks.workunit.client.1.smithi192.stdout:7/511: mkdir d5/d43/d4c/d5c/d97 0 2024-04-23T02:14:00.493 INFO:tasks.workunit.client.1.smithi192.stdout:2/503: readlink d8/lc 0 2024-04-23T02:14:00.494 INFO:tasks.workunit.client.1.smithi192.stdout:7/512: fdatasync d5/f15 0 2024-04-23T02:14:00.495 INFO:tasks.workunit.client.1.smithi192.stdout:2/504: dread - d8/d26/d27/d57/d69/f6a zero size 2024-04-23T02:14:00.496 INFO:tasks.workunit.client.0.smithi071.stdout:3/402: rename d7/d27/d2c to d7/d8/d42/d80/d88 0 2024-04-23T02:14:00.496 INFO:tasks.workunit.client.1.smithi192.stdout:1/305: creat dc/d16/d19/d30/d32/d34/f56 x:0 0 0 2024-04-23T02:14:00.497 INFO:tasks.workunit.client.1.smithi192.stdout:7/513: write d5/d43/d4c/d5c/d67/f81 [703988,82514] 0 2024-04-23T02:14:00.497 INFO:tasks.workunit.client.1.smithi192.stdout:5/495: mknod d2/d2e/ca3 0 2024-04-23T02:14:00.497 INFO:tasks.workunit.client.0.smithi071.stdout:6/396: dread d6/d1f/d5d/f6d [0,4194304] 0 2024-04-23T02:14:00.498 INFO:tasks.workunit.client.0.smithi071.stdout:3/403: fsync d7/d8/d42/d5f/d68/f84 0 2024-04-23T02:14:00.498 INFO:tasks.workunit.client.0.smithi071.stdout:9/396: creat d1/f86 x:0 0 0 2024-04-23T02:14:00.499 INFO:tasks.workunit.client.1.smithi192.stdout:1/306: read fa [118050,129123] 0 2024-04-23T02:14:00.500 INFO:tasks.workunit.client.0.smithi071.stdout:9/397: write d1/d1a/d5a/f6b [4103344,60691] 0 2024-04-23T02:14:00.500 INFO:tasks.workunit.client.0.smithi071.stdout:3/404: chown d7/c37 38125 1 2024-04-23T02:14:00.501 INFO:tasks.workunit.client.1.smithi192.stdout:2/505: symlink d8/d26/la1 0 2024-04-23T02:14:00.502 INFO:tasks.workunit.client.1.smithi192.stdout:1/307: write dc/d16/f31 [1170130,31373] 0 2024-04-23T02:14:00.502 INFO:tasks.workunit.client.0.smithi071.stdout:6/397: mknod d6/d1f/d5d/c81 0 2024-04-23T02:14:00.503 INFO:tasks.workunit.client.1.smithi192.stdout:5/496: creat d2/d6/d14/d60/fa4 x:0 0 0 2024-04-23T02:14:00.506 INFO:tasks.workunit.client.1.smithi192.stdout:5/497: fsync d2/db/d16/d1b/d3d/d40/d49/f70 0 2024-04-23T02:14:00.506 INFO:tasks.workunit.client.0.smithi071.stdout:6/398: rename d6/d1f/d4e/f51 to d6/d3e/f82 0 2024-04-23T02:14:00.507 INFO:tasks.workunit.client.1.smithi192.stdout:2/506: mkdir d8/d26/d27/d53/da2 0 2024-04-23T02:14:00.508 INFO:tasks.workunit.client.0.smithi071.stdout:3/405: fdatasync d7/d8/f16 0 2024-04-23T02:14:00.513 INFO:tasks.workunit.client.0.smithi071.stdout:2/389: dwrite d0/d2/d3c/f5f [0,4194304] 0 2024-04-23T02:14:00.513 INFO:tasks.workunit.client.0.smithi071.stdout:3/406: write d7/d8/d42/d5f/d68/f84 [242678,111328] 0 2024-04-23T02:14:00.515 INFO:tasks.workunit.client.0.smithi071.stdout:3/407: chown d7/d8/d42/d5f/d68 59 1 2024-04-23T02:14:00.515 INFO:tasks.workunit.client.0.smithi071.stdout:6/399: rename d6/d22/f26 to d6/d1f/f83 0 2024-04-23T02:14:00.516 INFO:tasks.workunit.client.1.smithi192.stdout:0/637: dread d6/f10 [0,4194304] 0 2024-04-23T02:14:00.517 INFO:tasks.workunit.client.1.smithi192.stdout:0/638: fdatasync d6/df/f30 0 2024-04-23T02:14:00.518 INFO:tasks.workunit.client.0.smithi071.stdout:3/408: creat d7/d8/d42/d80/f89 x:0 0 0 2024-04-23T02:14:00.519 INFO:tasks.workunit.client.0.smithi071.stdout:6/400: truncate d6/f3c 4226243 0 2024-04-23T02:14:00.521 INFO:tasks.workunit.client.1.smithi192.stdout:0/639: creat d6/df/d35/d49/fd2 x:0 0 0 2024-04-23T02:14:00.523 INFO:tasks.workunit.client.0.smithi071.stdout:6/401: rename d6/d22/f45 to d6/d17/d77/f84 0 2024-04-23T02:14:00.525 INFO:tasks.workunit.client.0.smithi071.stdout:9/398: dread d1/d1a/f2d [0,4194304] 0 2024-04-23T02:14:00.525 INFO:tasks.workunit.client.1.smithi192.stdout:2/507: dwrite d8/d26/d27/d31/f2a [0,4194304] 0 2024-04-23T02:14:00.525 INFO:tasks.workunit.client.1.smithi192.stdout:5/498: creat d2/db/d2b/fa5 x:0 0 0 2024-04-23T02:14:00.526 INFO:tasks.workunit.client.1.smithi192.stdout:5/499: chown d2/db/d16/d1b/f90 63 1 2024-04-23T02:14:00.529 INFO:tasks.workunit.client.0.smithi071.stdout:3/409: dwrite d7/d22/d23/f24 [0,4194304] 0 2024-04-23T02:14:00.531 INFO:tasks.workunit.client.0.smithi071.stdout:6/402: rename d6/d17/d4d to d6/d17/d77/d85 0 2024-04-23T02:14:00.535 INFO:tasks.workunit.client.1.smithi192.stdout:2/508: write d8/f36 [3258927,93381] 0 2024-04-23T02:14:00.535 INFO:tasks.workunit.client.1.smithi192.stdout:7/514: dread d5/d17/d1c/d57/f22 [0,4194304] 0 2024-04-23T02:14:00.535 INFO:tasks.workunit.client.1.smithi192.stdout:1/308: dread dc/d16/d19/d30/d32/f33 [0,4194304] 0 2024-04-23T02:14:00.535 INFO:tasks.workunit.client.1.smithi192.stdout:7/515: read - d5/d17/d1c/d59/d5a/d70/f88 zero size 2024-04-23T02:14:00.536 INFO:tasks.workunit.client.0.smithi071.stdout:9/399: rename d1/c7f to d1/d32/d3a/d78/c87 0 2024-04-23T02:14:00.543 INFO:tasks.workunit.client.1.smithi192.stdout:1/309: dread dc/d16/d19/d30/d32/f33 [0,4194304] 0 2024-04-23T02:14:00.543 INFO:tasks.workunit.client.0.smithi071.stdout:6/403: dwrite d6/d1f/d5d/f71 [0,4194304] 0 2024-04-23T02:14:00.545 INFO:tasks.workunit.client.0.smithi071.stdout:3/410: rename d7/d8/d42/d47/l49 to d7/d8/l8a 0 2024-04-23T02:14:00.546 INFO:tasks.workunit.client.1.smithi192.stdout:5/500: write d2/d6/d2d/d41/f97 [299371,8131] 0 2024-04-23T02:14:00.547 INFO:tasks.workunit.client.1.smithi192.stdout:7/516: creat d5/d17/d1c/d59/d5a/d41/f98 x:0 0 0 2024-04-23T02:14:00.547 INFO:tasks.workunit.client.0.smithi071.stdout:3/411: chown d7/d22/d23/l3d 2683720 1 2024-04-23T02:14:00.548 INFO:tasks.workunit.client.0.smithi071.stdout:9/400: creat d1/d32/d3a/d77/f88 x:0 0 0 2024-04-23T02:14:00.548 INFO:tasks.workunit.client.0.smithi071.stdout:6/404: write d6/d3e/f5b [1990417,41255] 0 2024-04-23T02:14:00.549 INFO:tasks.workunit.client.1.smithi192.stdout:1/310: rename dc/d16/d19/c2a to dc/d51/c57 0 2024-04-23T02:14:00.550 INFO:tasks.workunit.client.1.smithi192.stdout:5/501: rename d2/d6/d14/d60 to d2/d6/d14/d60/da6 22 2024-04-23T02:14:00.550 INFO:tasks.workunit.client.1.smithi192.stdout:7/517: creat d5/d43/f99 x:0 0 0 2024-04-23T02:14:00.551 INFO:tasks.workunit.client.0.smithi071.stdout:6/405: write d6/d3e/f42 [2226276,6548] 0 2024-04-23T02:14:00.551 INFO:tasks.workunit.client.0.smithi071.stdout:6/406: write d6/d22/f53 [1113621,21653] 0 2024-04-23T02:14:00.552 INFO:tasks.workunit.client.1.smithi192.stdout:7/518: chown d5/d17/d1c/d57/l36 9555354 1 2024-04-23T02:14:00.553 INFO:tasks.workunit.client.0.smithi071.stdout:9/401: creat d1/d32/d3a/d77/f89 x:0 0 0 2024-04-23T02:14:00.553 INFO:tasks.workunit.client.0.smithi071.stdout:9/402: write d1/d1a/d39/f53 [3890889,68978] 0 2024-04-23T02:14:00.554 INFO:tasks.workunit.client.0.smithi071.stdout:9/403: chown d1/l4d 2518 1 2024-04-23T02:14:00.556 INFO:tasks.workunit.client.1.smithi192.stdout:5/502: write d2/db/f13 [4646809,117125] 0 2024-04-23T02:14:00.557 INFO:tasks.workunit.client.1.smithi192.stdout:7/519: creat d5/d17/d1c/d59/f9a x:0 0 0 2024-04-23T02:14:00.558 INFO:tasks.workunit.client.1.smithi192.stdout:5/503: chown d2/d6/d2d/d41/d43/c94 0 1 2024-04-23T02:14:00.558 INFO:tasks.workunit.client.1.smithi192.stdout:5/504: dread - d2/d6/f8a zero size 2024-04-23T02:14:00.560 INFO:tasks.workunit.client.0.smithi071.stdout:3/412: dread d7/d1d/f73 [0,4194304] 0 2024-04-23T02:14:00.560 INFO:tasks.workunit.client.1.smithi192.stdout:5/505: chown d2/db/d16/f26 103 1 2024-04-23T02:14:00.560 INFO:tasks.workunit.client.1.smithi192.stdout:1/311: rmdir dc 39 2024-04-23T02:14:00.562 INFO:tasks.workunit.client.1.smithi192.stdout:7/520: mkdir d5/d17/d1c/d59/d5a/d9b 0 2024-04-23T02:14:00.562 INFO:tasks.workunit.client.1.smithi192.stdout:1/312: write dc/d16/d19/d30/f3f [5079343,18157] 0 2024-04-23T02:14:00.562 INFO:tasks.workunit.client.0.smithi071.stdout:3/413: mkdir d7/d8/d42/d5f/d68/d3b/d4c/d8b 0 2024-04-23T02:14:00.563 INFO:tasks.workunit.client.0.smithi071.stdout:3/414: fdatasync d7/d8/f1a 0 2024-04-23T02:14:00.565 INFO:tasks.workunit.client.0.smithi071.stdout:3/415: write d7/d8/d42/d5f/f5b [559786,19179] 0 2024-04-23T02:14:00.566 INFO:tasks.workunit.client.1.smithi192.stdout:5/506: truncate d2/db/d2b/f5b 4068798 0 2024-04-23T02:14:00.568 INFO:tasks.workunit.client.0.smithi071.stdout:3/416: creat d7/d8/d42/d5f/f8c x:0 0 0 2024-04-23T02:14:00.568 INFO:tasks.workunit.client.1.smithi192.stdout:7/521: dread d5/d43/d4c/d5c/d67/f81 [0,4194304] 0 2024-04-23T02:14:00.572 INFO:tasks.workunit.client.0.smithi071.stdout:3/417: truncate f0 4567332 0 2024-04-23T02:14:00.572 INFO:tasks.workunit.client.1.smithi192.stdout:2/509: dread d8/d14/f1a [0,4194304] 0 2024-04-23T02:14:00.572 INFO:tasks.workunit.client.1.smithi192.stdout:7/522: dread - d5/d17/d1c/d57/f90 zero size 2024-04-23T02:14:00.572 INFO:tasks.workunit.client.0.smithi071.stdout:6/407: write d6/f9 [355896,8108] 0 2024-04-23T02:14:00.574 INFO:tasks.workunit.client.0.smithi071.stdout:9/404: dread d1/fc [0,4194304] 0 2024-04-23T02:14:00.574 INFO:tasks.workunit.client.0.smithi071.stdout:3/418: symlink d7/d22/d23/d26/l8d 0 2024-04-23T02:14:00.575 INFO:tasks.workunit.client.1.smithi192.stdout:1/313: dwrite dc/f39 [0,4194304] 0 2024-04-23T02:14:00.576 INFO:tasks.workunit.client.1.smithi192.stdout:5/507: dread d2/db/d16/d1b/d3d/d40/d49/f58 [0,4194304] 0 2024-04-23T02:14:00.576 INFO:tasks.workunit.client.0.smithi071.stdout:6/408: rename d6/d17/c38 to d6/d17/c86 0 2024-04-23T02:14:00.577 INFO:tasks.workunit.client.0.smithi071.stdout:3/419: fdatasync d7/d10/f2f 0 2024-04-23T02:14:00.578 INFO:tasks.workunit.client.0.smithi071.stdout:6/409: chown d6/d1f/d4e/c69 732 1 2024-04-23T02:14:00.580 INFO:tasks.workunit.client.0.smithi071.stdout:3/420: mknod d7/d22/c8e 0 2024-04-23T02:14:00.581 INFO:tasks.workunit.client.1.smithi192.stdout:2/510: read f4 [6979,19412] 0 2024-04-23T02:14:00.581 INFO:tasks.workunit.client.0.smithi071.stdout:6/410: unlink d6/d1f/d5d/f5f 0 2024-04-23T02:14:00.583 INFO:tasks.workunit.client.0.smithi071.stdout:6/411: read d6/d3e/f42 [2580607,87280] 0 2024-04-23T02:14:00.583 INFO:tasks.workunit.client.0.smithi071.stdout:3/421: getdents d7/d86 0 2024-04-23T02:14:00.584 INFO:tasks.workunit.client.1.smithi192.stdout:2/511: fdatasync d8/d26/d27/d53/f5b 0 2024-04-23T02:14:00.584 INFO:tasks.workunit.client.0.smithi071.stdout:6/412: fdatasync d6/d1f/f68 0 2024-04-23T02:14:00.585 INFO:tasks.workunit.client.0.smithi071.stdout:6/413: chown d6/d1f/d4e/f74 2548370 1 2024-04-23T02:14:00.587 INFO:tasks.workunit.client.0.smithi071.stdout:3/422: rename d7/d8/d42/d80/d88/c36 to d7/d1d/c8f 0 2024-04-23T02:14:00.588 INFO:tasks.workunit.client.1.smithi192.stdout:7/523: dwrite d5/d17/d1c/d59/d5a/d70/f7e [4194304,4194304] 0 2024-04-23T02:14:00.589 INFO:tasks.workunit.client.0.smithi071.stdout:6/414: creat d6/d1f/d5d/f87 x:0 0 0 2024-04-23T02:14:00.590 INFO:tasks.workunit.client.1.smithi192.stdout:5/508: truncate d2/db/d16/d1b/d3d/d40/d49/f70 865322 0 2024-04-23T02:14:00.593 INFO:tasks.workunit.client.0.smithi071.stdout:6/415: rename d6/f5c to d6/d17/d77/d85/f88 0 2024-04-23T02:14:00.593 INFO:tasks.workunit.client.0.smithi071.stdout:6/416: chown d6/d3e/f82 16091 1 2024-04-23T02:14:00.594 INFO:tasks.workunit.client.1.smithi192.stdout:5/509: chown d2/d2e/d4e/c6d 1985392 1 2024-04-23T02:14:00.595 INFO:tasks.workunit.client.0.smithi071.stdout:3/423: rmdir d7/d22/d23 39 2024-04-23T02:14:00.600 INFO:tasks.workunit.client.1.smithi192.stdout:2/512: dread d8/d25/f3f [0,4194304] 0 2024-04-23T02:14:00.602 INFO:tasks.workunit.client.0.smithi071.stdout:6/417: creat d6/d17/d77/d85/f89 x:0 0 0 2024-04-23T02:14:00.602 INFO:tasks.workunit.client.0.smithi071.stdout:3/424: rename d7/d8/d42/d80/d88/l33 to d7/d22/d23/d26/d75/l90 0 2024-04-23T02:14:00.604 INFO:tasks.workunit.client.1.smithi192.stdout:1/314: dwrite f9 [4194304,4194304] 0 2024-04-23T02:14:00.605 INFO:tasks.workunit.client.0.smithi071.stdout:3/425: creat d7/d8/d42/d5f/d68/f91 x:0 0 0 2024-04-23T02:14:00.607 INFO:tasks.workunit.client.1.smithi192.stdout:1/315: rmdir dc/d14/d15 39 2024-04-23T02:14:00.607 INFO:tasks.workunit.client.1.smithi192.stdout:5/510: dwrite d2/db/d16/d1b/f90 [0,4194304] 0 2024-04-23T02:14:00.608 INFO:tasks.workunit.client.0.smithi071.stdout:3/426: creat d7/d8/d42/d5f/d68/d3b/d4c/d74/f92 x:0 0 0 2024-04-23T02:14:00.610 INFO:tasks.workunit.client.0.smithi071.stdout:3/427: mknod d7/d22/d23/d26/c93 0 2024-04-23T02:14:00.612 INFO:tasks.workunit.client.1.smithi192.stdout:7/524: truncate d5/d17/d1c/f74 8214465 0 2024-04-23T02:14:00.612 INFO:tasks.workunit.client.0.smithi071.stdout:3/428: mknod d7/d10/d38/d67/d6f/c94 0 2024-04-23T02:14:00.614 INFO:tasks.workunit.client.0.smithi071.stdout:3/429: truncate d7/d8/d42/d80/f89 776430 0 2024-04-23T02:14:00.615 INFO:tasks.workunit.client.1.smithi192.stdout:1/316: mknod dc/d14/c58 0 2024-04-23T02:14:00.615 INFO:tasks.workunit.client.0.smithi071.stdout:3/430: read d7/d22/d23/d26/f2b [292078,42866] 0 2024-04-23T02:14:00.616 INFO:tasks.workunit.client.1.smithi192.stdout:2/513: dread d8/d26/d27/f70 [0,4194304] 0 2024-04-23T02:14:00.616 INFO:tasks.workunit.client.1.smithi192.stdout:5/511: mknod d2/db/d16/ca7 0 2024-04-23T02:14:00.617 INFO:tasks.workunit.client.1.smithi192.stdout:7/525: symlink d5/d17/d1c/d26/l9c 0 2024-04-23T02:14:00.617 INFO:tasks.workunit.client.1.smithi192.stdout:7/526: readlink d5/l71 0 2024-04-23T02:14:00.618 INFO:tasks.workunit.client.0.smithi071.stdout:9/405: dread d1/f3d [0,4194304] 0 2024-04-23T02:14:00.623 INFO:tasks.workunit.client.0.smithi071.stdout:3/431: symlink d7/d8/d42/d5f/d68/l95 0 2024-04-23T02:14:00.625 INFO:tasks.workunit.client.1.smithi192.stdout:2/514: dwrite d8/d26/d27/d57/d69/f8a [0,4194304] 0 2024-04-23T02:14:00.626 INFO:tasks.workunit.client.0.smithi071.stdout:6/418: dwrite d6/f34 [0,4194304] 0 2024-04-23T02:14:00.626 INFO:tasks.workunit.client.0.smithi071.stdout:3/432: mknod d7/d10/c96 0 2024-04-23T02:14:00.632 INFO:tasks.workunit.client.0.smithi071.stdout:9/406: dwrite d1/f3 [0,4194304] 0 2024-04-23T02:14:00.634 INFO:tasks.workunit.client.1.smithi192.stdout:5/512: truncate d2/db/d16/d1b/f54 1027528 0 2024-04-23T02:14:00.634 INFO:tasks.workunit.client.1.smithi192.stdout:7/527: write d5/d17/d1c/f3c [214508,89847] 0 2024-04-23T02:14:00.634 INFO:tasks.workunit.client.0.smithi071.stdout:6/419: write d6/d1f/d4e/f54 [977970,79551] 0 2024-04-23T02:14:00.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.634+0000 7f3fecfe9700 1 -- 172.21.15.71:0/3801931604 --> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fc8000bf0 con 0x7f3fd006cab0 2024-04-23T02:14:00.639 INFO:tasks.workunit.client.0.smithi071.stdout:6/420: fsync d6/d1f/d5d/f67 0 2024-04-23T02:14:00.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.636+0000 7f3fd77fe700 1 -- 172.21.15.71:0/3801931604 <== mgr.24351 v2:172.21.15.192:6828/966231696 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f3fc8000bf0 con 0x7f3fd006cab0 2024-04-23T02:14:00.641 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:14:00.641 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:14:00.641 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:14:00.641 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-23T02:14:00.641 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [], 2024-04-23T02:14:00.641 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "1/2 daemons upgraded", 2024-04-23T02:14:00.641 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "", 2024-04-23T02:14:00.641 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:14:00.641 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:14:00.642 INFO:tasks.workunit.client.0.smithi071.stdout:3/433: dwrite d7/d22/d23/f24 [0,4194304] 0 2024-04-23T02:14:00.642 INFO:tasks.workunit.client.1.smithi192.stdout:5/513: rename d2/db/d16/d1b/l47 to d2/db/d16/d1b/d3d/la8 0 2024-04-23T02:14:00.642 INFO:tasks.workunit.client.1.smithi192.stdout:5/514: dread - d2/d32/f9c zero size 2024-04-23T02:14:00.643 INFO:tasks.workunit.client.0.smithi071.stdout:3/434: readlink d7/d8/d42/d5f/l5a 0 2024-04-23T02:14:00.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.639+0000 7f3fecfe9700 1 -- 172.21.15.71:0/3801931604 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f3fd006cab0 msgr2=0x7f3fd006ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:00.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.639+0000 7f3fecfe9700 1 --2- 172.21.15.71:0/3801931604 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f3fd006cab0 0x7f3fd006ef70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8000c00 tx=0x7f3fd800af70 comp rx=0 tx=0).stop 2024-04-23T02:14:00.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.639+0000 7f3fecfe9700 1 -- 172.21.15.71:0/3801931604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8109130 msgr2=0x7f3fe81ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:00.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.639+0000 7f3fecfe9700 1 --2- 172.21.15.71:0/3801931604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8109130 0x7f3fe81ae340 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00b4d0 tx=0x7f3fdc00b7e0 comp rx=0 tx=0).stop 2024-04-23T02:14:00.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.639+0000 7f3fecfe9700 1 -- 172.21.15.71:0/3801931604 shutdown_connections 2024-04-23T02:14:00.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.639+0000 7f3fecfe9700 1 --2- 172.21.15.71:0/3801931604 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f3fd006cab0 0x7f3fd006ef70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:00.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.639+0000 7f3fecfe9700 1 --2- 172.21.15.71:0/3801931604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3fe8107d90 0x7f3fe81add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:00.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.639+0000 7f3fecfe9700 1 --2- 172.21.15.71:0/3801931604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8109130 0x7f3fe81ae340 secure :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00b4d0 tx=0x7f3fdc00b7e0 comp rx=0 tx=0).stop 2024-04-23T02:14:00.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.639+0000 7f3fecfe9700 1 -- 172.21.15.71:0/3801931604 >> 172.21.15.71:0/3801931604 conn(0x7f3fe806da40 msgr2=0x7f3fe810bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:14:00.644 INFO:tasks.workunit.client.1.smithi192.stdout:7/528: truncate d5/d17/d1c/d59/d5a/d41/f54 360546 0 2024-04-23T02:14:00.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.640+0000 7f3fecfe9700 1 -- 172.21.15.71:0/3801931604 shutdown_connections 2024-04-23T02:14:00.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:00.640+0000 7f3fecfe9700 1 -- 172.21.15.71:0/3801931604 wait complete. 2024-04-23T02:14:00.645 INFO:tasks.workunit.client.0.smithi071.stdout:9/407: creat d1/d32/d3a/f8a x:0 0 0 2024-04-23T02:14:00.645 INFO:tasks.workunit.client.1.smithi192.stdout:7/529: truncate d5/d17/d1c/d26/f34 1182281 0 2024-04-23T02:14:00.648 INFO:tasks.workunit.client.0.smithi071.stdout:6/421: getdents d6/d22/d79 0 2024-04-23T02:14:00.649 INFO:tasks.workunit.client.0.smithi071.stdout:9/408: unlink d1/d32/d3a/d5f/f83 0 2024-04-23T02:14:00.651 INFO:tasks.workunit.client.1.smithi192.stdout:5/515: link d2/db/d2b/d52/l62 d2/db/d16/d1b/d3d/d40/d49/la9 0 2024-04-23T02:14:00.651 INFO:tasks.workunit.client.1.smithi192.stdout:2/515: write d8/d26/d29/f41 [16148,16764] 0 2024-04-23T02:14:00.652 INFO:tasks.workunit.client.1.smithi192.stdout:7/530: mknod d5/d43/d4c/d5c/d97/c9d 0 2024-04-23T02:14:00.652 INFO:tasks.workunit.client.0.smithi071.stdout:3/435: dwrite d7/d22/f55 [0,4194304] 0 2024-04-23T02:14:00.654 INFO:tasks.workunit.client.1.smithi192.stdout:7/531: truncate d5/d17/d1c/d4a/f76 59655 0 2024-04-23T02:14:00.655 INFO:tasks.workunit.client.0.smithi071.stdout:9/409: truncate d1/d1a/f2d 3439454 0 2024-04-23T02:14:00.655 INFO:tasks.workunit.client.1.smithi192.stdout:2/516: chown d8/d26/d27/d57/d69/l90 100 1 2024-04-23T02:14:00.655 INFO:tasks.workunit.client.0.smithi071.stdout:3/436: read - d7/d8/d42/d5f/d68/f91 zero size 2024-04-23T02:14:00.656 INFO:tasks.workunit.client.1.smithi192.stdout:2/517: stat d8/d26/d29/d43/l66 0 2024-04-23T02:14:00.657 INFO:tasks.workunit.client.1.smithi192.stdout:7/532: write d5/d17/d1c/d26/f34 [768966,69330] 0 2024-04-23T02:14:00.660 INFO:tasks.workunit.client.0.smithi071.stdout:3/437: dread d7/d8/d42/f5d [0,4194304] 0 2024-04-23T02:14:00.660 INFO:tasks.workunit.client.0.smithi071.stdout:6/422: dwrite d6/d1f/d4e/f54 [0,4194304] 0 2024-04-23T02:14:00.660 INFO:tasks.workunit.client.1.smithi192.stdout:7/533: chown d5/d43/d4c/d5c/d67/l6b 480656764 1 2024-04-23T02:14:00.661 INFO:tasks.workunit.client.0.smithi071.stdout:9/410: read d1/d32/d41/f57 [3505792,109546] 0 2024-04-23T02:14:00.662 INFO:tasks.workunit.client.0.smithi071.stdout:3/438: chown d7/d8/d42/d5f/d68/l95 2098769193 1 2024-04-23T02:14:00.662 INFO:tasks.workunit.client.0.smithi071.stdout:6/423: dread - d6/d17/d77/d85/f80 zero size 2024-04-23T02:14:00.662 INFO:tasks.workunit.client.1.smithi192.stdout:2/518: creat d8/d26/d27/d53/fa3 x:0 0 0 2024-04-23T02:14:00.664 INFO:tasks.workunit.client.0.smithi071.stdout:3/439: symlink d7/d27/l97 0 2024-04-23T02:14:00.665 INFO:tasks.workunit.client.1.smithi192.stdout:5/516: dwrite d2/db/d16/d1b/d3d/d40/f8c [0,4194304] 0 2024-04-23T02:14:00.667 INFO:tasks.workunit.client.0.smithi071.stdout:6/424: rename d6/f9 to d6/d3e/f8a 0 2024-04-23T02:14:00.672 INFO:tasks.workunit.client.0.smithi071.stdout:9/411: dwrite d1/d32/d3a/d77/f88 [0,4194304] 0 2024-04-23T02:14:00.673 INFO:tasks.workunit.client.1.smithi192.stdout:2/519: rename d8/d14/d51 to d8/d26/d29/da4 0 2024-04-23T02:14:00.677 INFO:tasks.workunit.client.1.smithi192.stdout:2/520: readlink d8/d14/l21 0 2024-04-23T02:14:00.678 INFO:tasks.workunit.client.0.smithi071.stdout:9/412: mkdir d1/d4e/d66/d8b 0 2024-04-23T02:14:00.680 INFO:tasks.workunit.client.1.smithi192.stdout:2/521: symlink d8/d26/la5 0 2024-04-23T02:14:00.681 INFO:tasks.workunit.client.1.smithi192.stdout:2/522: chown d8/d26/d29/d43/d50 1228203 1 2024-04-23T02:14:00.681 INFO:tasks.workunit.client.1.smithi192.stdout:5/517: dread d2/db/d2b/f3a [0,4194304] 0 2024-04-23T02:14:00.683 INFO:tasks.workunit.client.1.smithi192.stdout:5/518: write d2/db/d2b/d52/f99 [76461,111251] 0 2024-04-23T02:14:00.684 INFO:tasks.workunit.client.1.smithi192.stdout:5/519: chown d2/f5 273 1 2024-04-23T02:14:00.685 INFO:tasks.workunit.client.0.smithi071.stdout:9/413: dwrite d1/f3d [0,4194304] 0 2024-04-23T02:14:00.688 INFO:tasks.workunit.client.1.smithi192.stdout:5/520: write d2/f95 [564451,46706] 0 2024-04-23T02:14:00.693 INFO:tasks.workunit.client.1.smithi192.stdout:2/523: dread d8/d26/d29/d43/d50/d5e/f5f [0,4194304] 0 2024-04-23T02:14:00.693 INFO:tasks.workunit.client.0.smithi071.stdout:9/414: dwrite d1/d1a/f25 [8388608,4194304] 0 2024-04-23T02:14:00.694 INFO:tasks.workunit.client.0.smithi071.stdout:9/415: readlink d1/d1a/d39/l6e 0 2024-04-23T02:14:00.696 INFO:tasks.workunit.client.0.smithi071.stdout:9/416: mkdir d1/d1a/d8c 0 2024-04-23T02:14:00.698 INFO:tasks.workunit.client.0.smithi071.stdout:6/425: truncate d6/d1f/f3b 4377781 0 2024-04-23T02:14:00.698 INFO:tasks.workunit.client.0.smithi071.stdout:9/417: stat d1/c35 0 2024-04-23T02:14:00.698 INFO:tasks.workunit.client.1.smithi192.stdout:2/524: creat d8/fa6 x:0 0 0 2024-04-23T02:14:00.705 INFO:tasks.workunit.client.1.smithi192.stdout:2/525: dwrite d8/d25/f3a [0,4194304] 0 2024-04-23T02:14:00.707 INFO:tasks.workunit.client.0.smithi071.stdout:9/418: mkdir d1/d8d 0 2024-04-23T02:14:00.707 INFO:tasks.workunit.client.0.smithi071.stdout:6/426: dwrite d6/d17/d77/f7a [4194304,4194304] 0 2024-04-23T02:14:00.708 INFO:tasks.workunit.client.0.smithi071.stdout:9/419: dread - d1/d4e/d66/f73 zero size 2024-04-23T02:14:00.711 INFO:tasks.workunit.client.0.smithi071.stdout:9/420: symlink d1/l8e 0 2024-04-23T02:14:00.711 INFO:tasks.workunit.client.0.smithi071.stdout:8/471: sync 2024-04-23T02:14:00.714 INFO:tasks.workunit.client.0.smithi071.stdout:9/421: unlink d1/d32/d3a/f84 0 2024-04-23T02:14:00.717 INFO:tasks.workunit.client.0.smithi071.stdout:8/472: mkdir dc/d10/d8d/d9a 0 2024-04-23T02:14:00.718 INFO:tasks.workunit.client.0.smithi071.stdout:8/473: readlink dc/d3a/l4d 0 2024-04-23T02:14:00.718 INFO:tasks.workunit.client.0.smithi071.stdout:8/474: chown dc/d10/d55 1 1 2024-04-23T02:14:00.718 INFO:tasks.workunit.client.0.smithi071.stdout:9/422: creat d1/d1a/d39/f8f x:0 0 0 2024-04-23T02:14:00.722 INFO:tasks.workunit.client.1.smithi192.stdout:5/521: dread d2/d6/f4d [0,4194304] 0 2024-04-23T02:14:00.722 INFO:tasks.workunit.client.0.smithi071.stdout:9/423: mkdir d1/d32/d3a/d90 0 2024-04-23T02:14:00.722 INFO:tasks.workunit.client.0.smithi071.stdout:3/440: dread d7/d8/d42/d5f/f5b [0,4194304] 0 2024-04-23T02:14:00.726 INFO:tasks.workunit.client.1.smithi192.stdout:5/522: creat d2/d6/d2d/faa x:0 0 0 2024-04-23T02:14:00.727 INFO:tasks.workunit.client.0.smithi071.stdout:3/441: mkdir d7/d8/d42/d5f/d68/d3b/d98 0 2024-04-23T02:14:00.727 INFO:tasks.workunit.client.1.smithi192.stdout:5/523: creat d2/d2e/fab x:0 0 0 2024-04-23T02:14:00.727 INFO:tasks.workunit.client.1.smithi192.stdout:5/524: dread d2/db/d2b/f3a [0,4194304] 0 2024-04-23T02:14:00.729 INFO:tasks.workunit.client.0.smithi071.stdout:9/424: dwrite d1/f3 [0,4194304] 0 2024-04-23T02:14:00.731 INFO:tasks.workunit.client.1.smithi192.stdout:5/525: creat d2/d6/d2d/fac x:0 0 0 2024-04-23T02:14:00.733 INFO:tasks.workunit.client.0.smithi071.stdout:9/425: mknod d1/d4e/d66/c91 0 2024-04-23T02:14:00.734 INFO:tasks.workunit.client.1.smithi192.stdout:2/526: truncate d8/d25/f3a 2987736 0 2024-04-23T02:14:00.735 INFO:tasks.workunit.client.1.smithi192.stdout:5/526: creat d2/db/fad x:0 0 0 2024-04-23T02:14:00.736 INFO:tasks.workunit.client.1.smithi192.stdout:3/414: sync 2024-04-23T02:14:00.736 INFO:tasks.workunit.client.1.smithi192.stdout:4/648: sync 2024-04-23T02:14:00.737 INFO:tasks.workunit.client.0.smithi071.stdout:9/426: unlink d1/f3 0 2024-04-23T02:14:00.738 INFO:tasks.workunit.client.1.smithi192.stdout:3/415: truncate d1/d3/d6/f81 526166 0 2024-04-23T02:14:00.739 INFO:tasks.workunit.client.1.smithi192.stdout:5/527: mknod d2/db/d16/d1b/d3d/d40/cae 0 2024-04-23T02:14:00.739 INFO:tasks.workunit.client.0.smithi071.stdout:3/442: link d7/d8/d42/d5f/d68/c50 d7/d8/d42/d5f/d68/d3b/d98/c99 0 2024-04-23T02:14:00.741 INFO:tasks.workunit.client.0.smithi071.stdout:8/475: write dc/d56/f6a [1621754,26195] 0 2024-04-23T02:14:00.742 INFO:tasks.workunit.client.1.smithi192.stdout:3/416: creat d1/d3/d6/d7/d29/d3b/f8e x:0 0 0 2024-04-23T02:14:00.742 INFO:tasks.workunit.client.1.smithi192.stdout:2/527: link d8/d14/l3b d8/d26/d29/d43/d50/d5e/la7 0 2024-04-23T02:14:00.743 INFO:tasks.workunit.client.0.smithi071.stdout:8/476: symlink dc/d56/l9b 0 2024-04-23T02:14:00.743 INFO:tasks.workunit.client.1.smithi192.stdout:2/528: truncate d8/f86 368808 0 2024-04-23T02:14:00.744 INFO:tasks.workunit.client.0.smithi071.stdout:8/477: write dc/d15/f87 [315550,102778] 0 2024-04-23T02:14:00.745 INFO:tasks.workunit.client.1.smithi192.stdout:2/529: write d8/d26/d27/d53/fa3 [812292,66476] 0 2024-04-23T02:14:00.745 INFO:tasks.workunit.client.0.smithi071.stdout:8/478: write dc/d56/f61 [985251,114260] 0 2024-04-23T02:14:00.747 INFO:tasks.workunit.client.1.smithi192.stdout:3/417: getdents d1/d3/d17/d1f/d33 0 2024-04-23T02:14:00.747 INFO:tasks.workunit.client.1.smithi192.stdout:2/530: write d8/d26/d27/d31/f2a [4639862,90980] 0 2024-04-23T02:14:00.753 INFO:tasks.workunit.client.1.smithi192.stdout:2/531: dread d8/d26/f6b [0,4194304] 0 2024-04-23T02:14:00.753 INFO:tasks.workunit.client.0.smithi071.stdout:8/479: dwrite dc/d10/d55/f80 [0,4194304] 0 2024-04-23T02:14:00.754 INFO:tasks.workunit.client.0.smithi071.stdout:8/480: readlink dc/l1f 0 2024-04-23T02:14:00.755 INFO:tasks.workunit.client.0.smithi071.stdout:9/427: dwrite d1/f10 [0,4194304] 0 2024-04-23T02:14:00.758 INFO:tasks.workunit.client.1.smithi192.stdout:2/532: chown d8/d14/l22 78051080 1 2024-04-23T02:14:00.758 INFO:tasks.workunit.client.1.smithi192.stdout:3/418: dwrite d1/d3/d17/d1f/d33/d24/d34/f7f [0,4194304] 0 2024-04-23T02:14:00.759 INFO:tasks.workunit.client.0.smithi071.stdout:8/481: unlink dc/d56/f6a 0 2024-04-23T02:14:00.760 INFO:tasks.workunit.client.1.smithi192.stdout:4/649: dwrite de/f21 [0,4194304] 0 2024-04-23T02:14:00.760 INFO:tasks.workunit.client.0.smithi071.stdout:8/482: read dc/f28 [7033774,26567] 0 2024-04-23T02:14:00.761 INFO:tasks.workunit.client.0.smithi071.stdout:9/428: rename d1/d1a/c20 to d1/d32/d7e/c92 0 2024-04-23T02:14:00.761 INFO:tasks.workunit.client.1.smithi192.stdout:3/419: chown d1/d3/d17/d1f/d33/d24/d34/d6c/d84/d46/c66 1259966147 1 2024-04-23T02:14:00.763 INFO:tasks.workunit.client.1.smithi192.stdout:2/533: mkdir d8/d26/d29/da4/da8 0 2024-04-23T02:14:00.764 INFO:tasks.workunit.client.1.smithi192.stdout:4/650: unlink de/d15/d2a/l39 0 2024-04-23T02:14:00.765 INFO:tasks.workunit.client.0.smithi071.stdout:9/429: getdents d1/d32/d3a/d78 0 2024-04-23T02:14:00.765 INFO:tasks.workunit.client.0.smithi071.stdout:8/483: getdents dc/d10/d32/d63 0 2024-04-23T02:14:00.766 INFO:tasks.workunit.client.0.smithi071.stdout:9/430: rmdir d1/d32/d3a/d78 39 2024-04-23T02:14:00.767 INFO:tasks.workunit.client.1.smithi192.stdout:3/420: symlink d1/d3/d17/d1f/d33/d24/d34/d38/d8b/l8f 0 2024-04-23T02:14:00.767 INFO:tasks.workunit.client.0.smithi071.stdout:8/484: write dc/d15/d2d/f49 [382977,60520] 0 2024-04-23T02:14:00.768 INFO:tasks.workunit.client.0.smithi071.stdout:9/431: symlink d1/d4e/l93 0 2024-04-23T02:14:00.769 INFO:tasks.workunit.client.1.smithi192.stdout:4/651: creat de/d15/d2a/d74/d92/da0/dca/d49/d70/dd8/d89/fdc x:0 0 0 2024-04-23T02:14:00.769 INFO:tasks.workunit.client.0.smithi071.stdout:9/432: fsync d1/d1a/f55 0 2024-04-23T02:14:00.771 INFO:tasks.workunit.client.0.smithi071.stdout:9/433: write d1/d1a/f22 [2933251,123905] 0 2024-04-23T02:14:00.772 INFO:tasks.workunit.client.1.smithi192.stdout:2/534: dwrite d8/d25/f9d [0,4194304] 0 2024-04-23T02:14:00.774 INFO:tasks.workunit.client.1.smithi192.stdout:3/421: rename d1/d3/d17/d1f/d33/d24/d34/d6c/d84/d4c/f88 to d1/d3/d17/d1f/d33/d24/d34/d6c/f90 0 2024-04-23T02:14:00.775 INFO:tasks.workunit.client.0.smithi071.stdout:9/434: unlink d1/d4e/d66/d76/f7b 0 2024-04-23T02:14:00.776 INFO:tasks.workunit.client.0.smithi071.stdout:9/435: rmdir d1/d1a/d39 39 2024-04-23T02:14:00.777 INFO:tasks.workunit.client.1.smithi192.stdout:3/422: unlink d1/d3/d6/d7/f72 0 2024-04-23T02:14:00.778 INFO:tasks.workunit.client.1.smithi192.stdout:3/423: chown d1/d3/d17/d2b/d44/d57/f78 213359350 1 2024-04-23T02:14:00.778 INFO:tasks.workunit.client.1.smithi192.stdout:2/535: rename d8/d26/d29/d43/d5d/f73 to d8/d26/d29/d43/d50/fa9 0 2024-04-23T02:14:00.779 INFO:tasks.workunit.client.0.smithi071.stdout:9/436: mkdir d1/d32/d3a/d94 0 2024-04-23T02:14:00.780 INFO:tasks.workunit.client.0.smithi071.stdout:9/437: creat d1/d4e/d66/d8b/f95 x:0 0 0 2024-04-23T02:14:00.783 INFO:tasks.workunit.client.0.smithi071.stdout:9/438: rename d1/d1a/l3e to d1/d4e/d66/d8b/l96 0 2024-04-23T02:14:00.785 INFO:tasks.workunit.client.1.smithi192.stdout:3/424: link d1/d3/d17/f27 d1/d3/d17/d1a/d7c/f91 0 2024-04-23T02:14:00.786 INFO:tasks.workunit.client.0.smithi071.stdout:9/439: symlink d1/d8d/l97 0 2024-04-23T02:14:00.787 INFO:tasks.workunit.client.0.smithi071.stdout:9/440: creat d1/d1a/d8c/f98 x:0 0 0 2024-04-23T02:14:00.790 INFO:tasks.workunit.client.0.smithi071.stdout:9/441: getdents d1/d32/d41 0 2024-04-23T02:14:00.790 INFO:tasks.workunit.client.0.smithi071.stdout:9/442: dread - d1/d32/d3a/f8a zero size 2024-04-23T02:14:00.791 INFO:tasks.workunit.client.0.smithi071.stdout:9/443: chown d1/d32/d3a/d94 265889698 1 2024-04-23T02:14:00.791 INFO:tasks.workunit.client.0.smithi071.stdout:9/444: dread - d1/f86 zero size 2024-04-23T02:14:00.793 INFO:tasks.workunit.client.0.smithi071.stdout:9/445: chown d1/d1a/f2d 24481060 1 2024-04-23T02:14:00.797 INFO:tasks.workunit.client.1.smithi192.stdout:2/536: dwrite d8/d26/d27/f70 [0,4194304] 0 2024-04-23T02:14:00.798 INFO:tasks.workunit.client.1.smithi192.stdout:2/537: readlink d8/d26/d27/d57/d69/l90 0 2024-04-23T02:14:00.803 INFO:tasks.workunit.client.1.smithi192.stdout:4/652: dread de/d15/d2a/fcb [0,4194304] 0 2024-04-23T02:14:00.809 INFO:tasks.workunit.client.1.smithi192.stdout:4/653: fdatasync de/d15/d2a/d74/d92/da0/dca/f57 0 2024-04-23T02:14:00.809 INFO:tasks.workunit.client.1.smithi192.stdout:4/654: readlink de/d15/d2a/d74/d92/da0/dca/d49/d26/l3b 0 2024-04-23T02:14:00.813 INFO:tasks.workunit.client.1.smithi192.stdout:2/538: link d8/d26/d27/d57/d69/f6a d8/d14/faa 0 2024-04-23T02:14:00.814 INFO:tasks.workunit.client.1.smithi192.stdout:4/655: mknod de/d15/d2a/d74/d92/da0/dca/cdd 0 2024-04-23T02:14:00.816 INFO:tasks.workunit.client.1.smithi192.stdout:4/656: dread - de/d15/d2a/d74/d92/da0/dca/d49/d70/d73/fae zero size 2024-04-23T02:14:00.819 INFO:tasks.workunit.client.1.smithi192.stdout:2/539: dwrite d8/d26/d27/d57/d69/f8a [0,4194304] 0 2024-04-23T02:14:00.821 INFO:tasks.workunit.client.1.smithi192.stdout:2/540: write d8/f96 [281778,90244] 0 2024-04-23T02:14:00.841 INFO:tasks.workunit.client.1.smithi192.stdout:2/541: fdatasync d8/d26/d27/d57/d69/f8a 0 2024-04-23T02:14:00.847 INFO:tasks.workunit.client.1.smithi192.stdout:2/542: dwrite d8/f96 [0,4194304] 0 2024-04-23T02:14:00.849 INFO:tasks.workunit.client.0.smithi071.stdout:9/446: write d1/fe [1638937,32990] 0 2024-04-23T02:14:00.854 INFO:tasks.workunit.client.0.smithi071.stdout:9/447: rmdir d1/d4e/d66/d76 0 2024-04-23T02:14:00.854 INFO:tasks.workunit.client.1.smithi192.stdout:2/543: write d8/d26/d29/d43/f7f [770111,16733] 0 2024-04-23T02:14:00.855 INFO:tasks.workunit.client.0.smithi071.stdout:9/448: mkdir d1/d32/d3a/d90/d99 0 2024-04-23T02:14:00.856 INFO:tasks.workunit.client.0.smithi071.stdout:9/449: chown d1/d32/d3a/d5f 462025327 1 2024-04-23T02:14:00.857 INFO:tasks.workunit.client.1.smithi192.stdout:2/544: mkdir d8/d26/d29/da4/dab 0 2024-04-23T02:14:00.858 INFO:tasks.workunit.client.0.smithi071.stdout:9/450: creat d1/f9a x:0 0 0 2024-04-23T02:14:00.859 INFO:tasks.workunit.client.0.smithi071.stdout:9/451: write d1/f10 [4713584,33039] 0 2024-04-23T02:14:00.860 INFO:tasks.workunit.client.1.smithi192.stdout:3/425: dwrite d1/d3/d17/d1f/d33/d24/d34/d6c/d84/f83 [0,4194304] 0 2024-04-23T02:14:00.867 INFO:tasks.workunit.client.1.smithi192.stdout:3/426: creat d1/d3/d17/d1f/d33/d24/d34/f92 x:0 0 0 2024-04-23T02:14:00.867 INFO:tasks.workunit.client.1.smithi192.stdout:2/545: symlink d8/d26/d29/da4/da8/lac 0 2024-04-23T02:14:00.869 INFO:tasks.workunit.client.1.smithi192.stdout:3/427: creat d1/d3/d17/d2b/d44/f93 x:0 0 0 2024-04-23T02:14:00.873 INFO:tasks.workunit.client.1.smithi192.stdout:3/428: dread - d1/d3/d17/d1f/d33/d24/d34/d6c/d84/f63 zero size 2024-04-23T02:14:00.875 INFO:tasks.workunit.client.1.smithi192.stdout:2/546: dwrite d8/d26/f6b [0,4194304] 0 2024-04-23T02:14:00.876 INFO:tasks.workunit.client.1.smithi192.stdout:2/547: chown d8/d26/d29/d80/l9c 976 1 2024-04-23T02:14:00.878 INFO:tasks.workunit.client.1.smithi192.stdout:3/429: mkdir d1/d3/d17/d1f/d33/d24/d34/d6c/d84/d46/d94 0 2024-04-23T02:14:00.880 INFO:tasks.workunit.client.1.smithi192.stdout:2/548: creat d8/d25/fad x:0 0 0 2024-04-23T02:14:00.881 INFO:tasks.workunit.client.1.smithi192.stdout:3/430: truncate d1/d3/f49 2876000 0 2024-04-23T02:14:00.882 INFO:tasks.workunit.client.1.smithi192.stdout:2/549: mkdir d8/d26/d27/d31/dae 0 2024-04-23T02:14:00.882 INFO:tasks.workunit.client.0.smithi071.stdout:9/452: dread d1/d1a/f2d [0,4194304] 0 2024-04-23T02:14:00.884 INFO:tasks.workunit.client.1.smithi192.stdout:2/550: mkdir d8/daf 0 2024-04-23T02:14:00.885 INFO:tasks.workunit.client.1.smithi192.stdout:3/431: rmdir d1/d3/d17/d1a/d6d 0 2024-04-23T02:14:00.885 INFO:tasks.workunit.client.0.smithi071.stdout:9/453: fdatasync d1/d32/d3a/d78/f7a 0 2024-04-23T02:14:00.886 INFO:tasks.workunit.client.1.smithi192.stdout:3/432: chown d1/d3/d17/d1f/d33/d24/d34/d6c/d84/d46/d94 26374 1 2024-04-23T02:14:00.886 INFO:tasks.workunit.client.0.smithi071.stdout:9/454: fsync d1/f50 0 2024-04-23T02:14:00.888 INFO:tasks.workunit.client.0.smithi071.stdout:9/455: readlink d1/d4e/d66/d8b/l96 0 2024-04-23T02:14:00.895 INFO:tasks.workunit.client.0.smithi071.stdout:9/456: truncate d1/d1a/d39/f53 3364725 0 2024-04-23T02:14:00.896 INFO:tasks.workunit.client.1.smithi192.stdout:3/433: write d1/d3/d17/f6e [1555674,112890] 0 2024-04-23T02:14:00.896 INFO:tasks.workunit.client.0.smithi071.stdout:9/457: write d1/d32/d3a/d5f/f63 [561472,71942] 0 2024-04-23T02:14:00.897 INFO:tasks.workunit.client.1.smithi192.stdout:3/434: chown d1/d3/d6/d7/d71/c73 203 1 2024-04-23T02:14:00.899 INFO:tasks.workunit.client.0.smithi071.stdout:9/458: rename d1/f50 to d1/d32/d3a/d94/f9b 0 2024-04-23T02:14:00.899 INFO:tasks.workunit.client.0.smithi071.stdout:9/459: dread - d1/f9a zero size 2024-04-23T02:14:00.900 INFO:tasks.workunit.client.0.smithi071.stdout:9/460: chown d1/d32/f4a 1018565943 1 2024-04-23T02:14:00.901 INFO:tasks.workunit.client.1.smithi192.stdout:3/435: truncate d1/f5 1296055 0 2024-04-23T02:14:00.902 INFO:tasks.workunit.client.0.smithi071.stdout:9/461: creat d1/d32/d3a/d94/f9c x:0 0 0 2024-04-23T02:14:00.902 INFO:tasks.workunit.client.0.smithi071.stdout:9/462: fsync d1/f2b 0 2024-04-23T02:14:00.903 INFO:tasks.workunit.client.1.smithi192.stdout:3/436: creat d1/d3/d6/d7/d71/f95 x:0 0 0 2024-04-23T02:14:00.910 INFO:tasks.workunit.client.1.smithi192.stdout:3/437: dwrite d1/d3/d17/d1f/d33/f8d [0,4194304] 0 2024-04-23T02:14:00.913 INFO:tasks.workunit.client.1.smithi192.stdout:3/438: write d1/d3/d17/d2b/d44/d57/f78 [777245,80333] 0 2024-04-23T02:14:00.914 INFO:tasks.workunit.client.0.smithi071.stdout:9/463: dwrite d1/d32/d3a/d5f/f60 [0,4194304] 0 2024-04-23T02:14:00.915 INFO:tasks.workunit.client.0.smithi071.stdout:9/464: read - d1/d1a/d8c/f98 zero size 2024-04-23T02:14:00.922 INFO:tasks.workunit.client.1.smithi192.stdout:3/439: write d1/d3/d17/d1a/d7c/f91 [1767178,30437] 0 2024-04-23T02:14:00.923 INFO:tasks.workunit.client.0.smithi071.stdout:0/435: sync 2024-04-23T02:14:00.924 INFO:tasks.workunit.client.0.smithi071.stdout:9/465: dwrite d1/d32/d3a/f8a [0,4194304] 0 2024-04-23T02:14:00.926 INFO:tasks.workunit.client.1.smithi192.stdout:3/440: rename d1/d3/d6/f7d to d1/d3/d17/d1f/d33/d24/d34/d38/d7b/f96 0 2024-04-23T02:14:00.927 INFO:tasks.workunit.client.1.smithi192.stdout:3/441: dread - d1/d3/d17/d1f/d33/d24/f50 zero size 2024-04-23T02:14:00.928 INFO:tasks.workunit.client.0.smithi071.stdout:0/436: mkdir d0/d3/de/d10/d14/d32/d44/d87 0 2024-04-23T02:14:00.929 INFO:tasks.workunit.client.1.smithi192.stdout:3/442: creat d1/d3/d17/d2b/d44/d57/f97 x:0 0 0 2024-04-23T02:14:00.931 INFO:tasks.workunit.client.1.smithi192.stdout:3/443: dread - d1/d3/d17/d1f/d33/d24/d34/d6c/d84/f63 zero size 2024-04-23T02:14:00.934 INFO:tasks.workunit.client.0.smithi071.stdout:9/466: link d1/d1a/d68/f80 d1/d1a/d68/f9d 0 2024-04-23T02:14:00.937 INFO:tasks.workunit.client.1.smithi192.stdout:3/444: dwrite d1/d3/d6/d7/f2a [0,4194304] 0 2024-04-23T02:14:00.938 INFO:tasks.workunit.client.1.smithi192.stdout:3/445: write d1/d3/d17/f3d [782098,114670] 0 2024-04-23T02:14:00.942 INFO:tasks.workunit.client.0.smithi071.stdout:0/437: chown d0/d3/de/d10/d14/d32/d44/d31/l46 43325295 1 2024-04-23T02:14:00.945 INFO:tasks.workunit.client.0.smithi071.stdout:0/438: unlink d0/d3/f3d 0 2024-04-23T02:14:00.946 INFO:tasks.workunit.client.0.smithi071.stdout:0/439: chown d0/d3/de/d10/d16/c33 1078866 1 2024-04-23T02:14:00.946 INFO:tasks.workunit.client.1.smithi192.stdout:3/446: dwrite d1/d3/d17/d1f/d33/d24/d34/f7f [0,4194304] 0 2024-04-23T02:14:00.948 INFO:tasks.workunit.client.1.smithi192.stdout:3/447: stat d1/d3/d17/d2b/d44/d57/f97 0 2024-04-23T02:14:00.948 INFO:tasks.workunit.client.0.smithi071.stdout:0/440: mknod d0/d3/de/d10/d14/d32/d44/d59/c88 0 2024-04-23T02:14:00.949 INFO:tasks.workunit.client.0.smithi071.stdout:0/441: readlink d0/d3/de/d10/d14/d32/d44/d31/d52/l68 0 2024-04-23T02:14:00.951 INFO:tasks.workunit.client.0.smithi071.stdout:0/442: truncate d0/f20 5240807 0 2024-04-23T02:14:00.952 INFO:tasks.workunit.client.0.smithi071.stdout:0/443: write d0/d3/d1c/f22 [1341037,100051] 0 2024-04-23T02:14:00.955 INFO:tasks.workunit.client.0.smithi071.stdout:0/444: write d0/d3/de/d10/d16/f55 [516565,119563] 0 2024-04-23T02:14:00.957 INFO:tasks.workunit.client.0.smithi071.stdout:9/467: truncate d1/f3d 3115963 0 2024-04-23T02:14:00.958 INFO:tasks.workunit.client.1.smithi192.stdout:3/448: dread d1/f55 [0,4194304] 0 2024-04-23T02:14:00.960 INFO:tasks.workunit.client.0.smithi071.stdout:9/468: getdents d1/d32/d41 0 2024-04-23T02:14:00.961 INFO:tasks.workunit.client.0.smithi071.stdout:0/445: mknod d0/d3/de/d10/d14/c89 0 2024-04-23T02:14:00.963 INFO:tasks.workunit.client.1.smithi192.stdout:3/449: mkdir d1/d3/d17/d1f/d33/d24/d34/d6c/d84/d5d/d98 0 2024-04-23T02:14:00.963 INFO:tasks.workunit.client.0.smithi071.stdout:0/446: read d0/d3/de/d10/d14/f1e [2416087,26619] 0 2024-04-23T02:14:00.963 INFO:tasks.workunit.client.0.smithi071.stdout:9/469: creat d1/d1a/d71/f9e x:0 0 0 2024-04-23T02:14:00.968 INFO:tasks.workunit.client.0.smithi071.stdout:9/470: dread d1/d32/d3a/d94/f9b [0,4194304] 0 2024-04-23T02:14:00.969 INFO:tasks.workunit.client.1.smithi192.stdout:3/450: dwrite d1/d3/d17/d1f/d33/d24/f45 [0,4194304] 0 2024-04-23T02:14:00.971 INFO:tasks.workunit.client.0.smithi071.stdout:0/447: mkdir d0/d3/de/d10/d14/d32/d8a 0 2024-04-23T02:14:00.972 INFO:tasks.workunit.client.1.smithi192.stdout:3/451: read d1/d3/f19 [2537703,4261] 0 2024-04-23T02:14:00.972 INFO:tasks.workunit.client.1.smithi192.stdout:6/596: sync 2024-04-23T02:14:00.973 INFO:tasks.workunit.client.0.smithi071.stdout:9/471: truncate d1/f16 390604 0 2024-04-23T02:14:00.974 INFO:tasks.workunit.client.1.smithi192.stdout:3/452: mkdir d1/d3/d17/d65/d99 0 2024-04-23T02:14:00.975 INFO:tasks.workunit.client.0.smithi071.stdout:0/448: unlink d0/c86 0 2024-04-23T02:14:00.975 INFO:tasks.workunit.client.1.smithi192.stdout:6/597: truncate d3/d15/d1b/da2/faa 3968212 0 2024-04-23T02:14:00.976 INFO:tasks.workunit.client.1.smithi192.stdout:3/453: mkdir d1/d3/d17/d2b/d9a 0 2024-04-23T02:14:00.977 INFO:tasks.workunit.client.1.smithi192.stdout:3/454: readlink d1/d3/d17/l39 0 2024-04-23T02:14:00.978 INFO:tasks.workunit.client.1.smithi192.stdout:6/598: dread d3/f34 [0,4194304] 0 2024-04-23T02:14:00.979 INFO:tasks.workunit.client.0.smithi071.stdout:9/472: rename d1/d1a/d8c to d1/d32/d3a/d78/d9f 0 2024-04-23T02:14:00.980 INFO:tasks.workunit.client.1.smithi192.stdout:6/599: fdatasync d3/d15/d5b/fb8 0 2024-04-23T02:14:00.981 INFO:tasks.workunit.client.0.smithi071.stdout:0/449: unlink d0/d3/de/d10/d14/d32/l4f 0 2024-04-23T02:14:00.982 INFO:tasks.workunit.client.0.smithi071.stdout:0/450: fsync d0/d63/f74 0 2024-04-23T02:14:00.982 INFO:tasks.workunit.client.1.smithi192.stdout:6/600: mkdir d3/d28/d79/da0/dc2 0 2024-04-23T02:14:00.984 INFO:tasks.workunit.client.1.smithi192.stdout:6/601: creat d3/d15/d1b/da2/fc3 x:0 0 0 2024-04-23T02:14:00.986 INFO:tasks.workunit.client.1.smithi192.stdout:6/602: write d3/d28/f2e [1750172,64064] 0 2024-04-23T02:14:00.988 INFO:tasks.workunit.client.0.smithi071.stdout:9/473: dwrite d1/d32/d3a/d5f/f63 [0,4194304] 0 2024-04-23T02:14:00.989 INFO:tasks.workunit.client.1.smithi192.stdout:6/603: mknod d3/d15/d1b/d21/d3a/cc4 0 2024-04-23T02:14:00.992 INFO:tasks.workunit.client.0.smithi071.stdout:9/474: mknod d1/d32/d3a/d5f/ca0 0 2024-04-23T02:14:00.995 INFO:tasks.workunit.client.0.smithi071.stdout:0/451: write d0/f20 [1161158,80028] 0 2024-04-23T02:14:00.995 INFO:tasks.workunit.client.1.smithi192.stdout:6/604: symlink d3/d15/d1b/d21/d52/d64/d8d/dab/lc5 0 2024-04-23T02:14:00.996 INFO:tasks.workunit.client.0.smithi071.stdout:0/452: readlink d0/d3/de/d10/d14/d32/d44/l67 0 2024-04-23T02:14:00.996 INFO:tasks.workunit.client.0.smithi071.stdout:9/475: rename d1/d4e/d66/c6d to d1/d32/d7e/ca1 0 2024-04-23T02:14:00.999 INFO:tasks.workunit.client.0.smithi071.stdout:9/476: write d1/d1a/f70 [3593393,21744] 0 2024-04-23T02:14:00.999 INFO:tasks.workunit.client.0.smithi071.stdout:2/390: sync 2024-04-23T02:14:00.999 INFO:tasks.workunit.client.0.smithi071.stdout:8/485: sync 2024-04-23T02:14:00.999 INFO:tasks.workunit.client.0.smithi071.stdout:3/443: sync 2024-04-23T02:14:00.999 INFO:tasks.workunit.client.0.smithi071.stdout:6/427: sync 2024-04-23T02:14:01.000 INFO:tasks.workunit.client.0.smithi071.stdout:1/435: sync 2024-04-23T02:14:01.000 INFO:tasks.workunit.client.0.smithi071.stdout:4/461: sync 2024-04-23T02:14:01.000 INFO:tasks.workunit.client.0.smithi071.stdout:5/550: sync 2024-04-23T02:14:01.000 INFO:tasks.workunit.client.0.smithi071.stdout:7/425: sync 2024-04-23T02:14:01.004 INFO:tasks.workunit.client.0.smithi071.stdout:0/453: rename d0/d3/d1c to d0/d3/de/d48/d7e/d8b 0 2024-04-23T02:14:01.004 INFO:tasks.workunit.client.0.smithi071.stdout:3/444: rmdir d7/d10/d64 39 2024-04-23T02:14:01.004 INFO:tasks.workunit.client.0.smithi071.stdout:9/477: chown d1/d32/d3a/d77/f89 0 1 2024-04-23T02:14:01.014 INFO:tasks.workunit.client.1.smithi192.stdout:6/605: rmdir d3 39 2024-04-23T02:14:01.015 INFO:tasks.workunit.client.0.smithi071.stdout:6/428: truncate d6/d3e/f75 115606 0 2024-04-23T02:14:01.018 INFO:tasks.workunit.client.1.smithi192.stdout:6/606: symlink d3/d15/d1b/d66/d80/d8e/d9d/lc6 0 2024-04-23T02:14:01.020 INFO:tasks.workunit.client.1.smithi192.stdout:6/607: symlink d3/d15/d1b/d21/d3a/dbd/lc7 0 2024-04-23T02:14:01.021 INFO:tasks.workunit.client.1.smithi192.stdout:6/608: fdatasync d3/f57 0 2024-04-23T02:14:01.022 INFO:tasks.workunit.client.0.smithi071.stdout:2/391: dread d0/d2/d34/d55/f72 [0,4194304] 0 2024-04-23T02:14:01.023 INFO:tasks.workunit.client.1.smithi192.stdout:6/609: creat d3/d15/d1b/d21/d52/d7f/fc8 x:0 0 0 2024-04-23T02:14:01.024 INFO:tasks.workunit.client.1.smithi192.stdout:6/610: write d3/d11/f4a [4742324,130652] 0 2024-04-23T02:14:01.027 INFO:tasks.workunit.client.1.smithi192.stdout:6/611: symlink d3/d28/d79/da0/dc2/lc9 0 2024-04-23T02:14:01.027 INFO:tasks.workunit.client.1.smithi192.stdout:6/612: fsync f1 0 2024-04-23T02:14:01.028 INFO:tasks.workunit.client.0.smithi071.stdout:1/436: creat db/d34/d8a/f8b x:0 0 0 2024-04-23T02:14:01.029 INFO:tasks.workunit.client.1.smithi192.stdout:6/613: mkdir d3/d18/dca 0 2024-04-23T02:14:01.030 INFO:tasks.workunit.client.0.smithi071.stdout:9/478: mknod d1/d32/d41/ca2 0 2024-04-23T02:14:01.030 INFO:tasks.workunit.client.0.smithi071.stdout:6/429: symlink d6/d17/d77/d85/l8b 0 2024-04-23T02:14:01.030 INFO:tasks.workunit.client.0.smithi071.stdout:2/392: dread - d0/d2/d45/f60 zero size 2024-04-23T02:14:01.032 INFO:tasks.workunit.client.1.smithi192.stdout:6/614: unlink d3/d15/d1b/d21/d3a/l68 0 2024-04-23T02:14:01.035 INFO:tasks.workunit.client.1.smithi192.stdout:6/615: mkdir d3/d28/d79/dbe/dcb 0 2024-04-23T02:14:01.035 INFO:tasks.workunit.client.1.smithi192.stdout:6/616: chown d3/d15/d5b/f82 531448372 1 2024-04-23T02:14:01.036 INFO:tasks.workunit.client.0.smithi071.stdout:7/426: dread d3/ff [0,4194304] 0 2024-04-23T02:14:01.036 INFO:tasks.workunit.client.0.smithi071.stdout:5/551: dwrite d3/d1d/d43/d4b/f72 [0,4194304] 0 2024-04-23T02:14:01.036 INFO:tasks.workunit.client.0.smithi071.stdout:4/462: rename d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d17 to d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c/d99 0 2024-04-23T02:14:01.036 INFO:tasks.workunit.client.0.smithi071.stdout:8/486: dwrite dc/d56/d64/f69 [0,4194304] 0 2024-04-23T02:14:01.037 INFO:tasks.workunit.client.0.smithi071.stdout:6/430: stat d6/d17/c23 0 2024-04-23T02:14:01.037 INFO:tasks.workunit.client.0.smithi071.stdout:3/445: dwrite d7/d8/d42/d5f/d68/f84 [0,4194304] 0 2024-04-23T02:14:01.041 INFO:tasks.workunit.client.0.smithi071.stdout:3/446: fdatasync d7/d8/d42/d5f/d68/f62 0 2024-04-23T02:14:01.041 INFO:tasks.workunit.client.0.smithi071.stdout:5/552: dread - d3/da/d1a/d1c/fb2 zero size 2024-04-23T02:14:01.042 INFO:tasks.workunit.client.0.smithi071.stdout:6/431: fsync d6/d3e/f5e 0 2024-04-23T02:14:01.042 INFO:tasks.workunit.client.0.smithi071.stdout:1/437: truncate db/df/d13/d2b/f78 1167090 0 2024-04-23T02:14:01.043 INFO:tasks.workunit.client.0.smithi071.stdout:8/487: dread - dc/d10/f85 zero size 2024-04-23T02:14:01.045 INFO:tasks.workunit.client.0.smithi071.stdout:8/488: dread - dc/d15/f7e zero size 2024-04-23T02:14:01.046 INFO:tasks.workunit.client.0.smithi071.stdout:4/463: unlink d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/ld 0 2024-04-23T02:14:01.047 INFO:tasks.workunit.client.0.smithi071.stdout:3/447: chown d7/d10/d64 4256 1 2024-04-23T02:14:01.047 INFO:tasks.workunit.client.0.smithi071.stdout:7/427: dread - d3/d17/d41/d34/d39/d3b/f49 zero size 2024-04-23T02:14:01.047 INFO:tasks.workunit.client.0.smithi071.stdout:6/432: readlink d6/d1f/d46/l59 0 2024-04-23T02:14:01.049 INFO:tasks.workunit.client.0.smithi071.stdout:2/393: dread d0/d2/d51/f53 [0,4194304] 0 2024-04-23T02:14:01.052 INFO:tasks.workunit.client.0.smithi071.stdout:4/464: mknod d2/d2a/d2d/d7b/c9a 0 2024-04-23T02:14:01.053 INFO:tasks.workunit.client.0.smithi071.stdout:3/448: chown d7/d8/d42/d5f/f40 57451916 1 2024-04-23T02:14:01.054 INFO:tasks.workunit.client.0.smithi071.stdout:1/438: dread db/df/d13/d2b/f78 [0,4194304] 0 2024-04-23T02:14:01.055 INFO:tasks.workunit.client.0.smithi071.stdout:5/553: mknod d3/d1d/d43/d4b/cd9 0 2024-04-23T02:14:01.057 INFO:tasks.workunit.client.0.smithi071.stdout:6/433: write d6/d17/d77/d85/f88 [4908450,84633] 0 2024-04-23T02:14:01.058 INFO:tasks.workunit.client.0.smithi071.stdout:8/489: dread dc/d15/d2d/f49 [0,4194304] 0 2024-04-23T02:14:01.059 INFO:tasks.workunit.client.0.smithi071.stdout:7/428: rmdir d3/d17/d41/d34/d6e 39 2024-04-23T02:14:01.061 INFO:tasks.workunit.client.0.smithi071.stdout:4/465: unlink d2/d2a/d73/f8a 0 2024-04-23T02:14:01.063 INFO:tasks.workunit.client.0.smithi071.stdout:4/466: truncate d2/f89 777324 0 2024-04-23T02:14:01.063 INFO:tasks.workunit.client.0.smithi071.stdout:1/439: creat db/d34/f8c x:0 0 0 2024-04-23T02:14:01.064 INFO:tasks.workunit.client.0.smithi071.stdout:3/449: symlink d7/d8/d42/d5f/d68/l9a 0 2024-04-23T02:14:01.064 INFO:tasks.workunit.client.0.smithi071.stdout:4/467: readlink d2/d53/l5f 0 2024-04-23T02:14:01.066 INFO:tasks.workunit.client.0.smithi071.stdout:4/468: write d2/d2a/d2d/d39/d42/f55 [127147,70843] 0 2024-04-23T02:14:01.067 INFO:tasks.workunit.client.0.smithi071.stdout:5/554: rename d3/da/d1a/d1c/f39 to d3/d1d/d43/dd2/fda 0 2024-04-23T02:14:01.071 INFO:tasks.workunit.client.0.smithi071.stdout:6/434: creat d6/d17/d77/f8c x:0 0 0 2024-04-23T02:14:01.071 INFO:tasks.workunit.client.0.smithi071.stdout:8/490: unlink dc/f75 0 2024-04-23T02:14:01.072 INFO:tasks.workunit.client.0.smithi071.stdout:5/555: dread d3/d34/d6b/fa3 [0,4194304] 0 2024-04-23T02:14:01.072 INFO:tasks.workunit.client.0.smithi071.stdout:7/429: mknod d3/c93 0 2024-04-23T02:14:01.074 INFO:tasks.workunit.client.0.smithi071.stdout:5/556: write d3/d34/f6a [151148,105063] 0 2024-04-23T02:14:01.077 INFO:tasks.workunit.client.0.smithi071.stdout:9/479: write d1/d32/d3a/d78/f7a [1397193,100863] 0 2024-04-23T02:14:01.078 INFO:tasks.workunit.client.0.smithi071.stdout:9/480: fdatasync d1/d32/d3a/f8a 0 2024-04-23T02:14:01.080 INFO:tasks.workunit.client.0.smithi071.stdout:4/469: unlink d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d29/c2b 0 2024-04-23T02:14:01.081 INFO:tasks.workunit.client.0.smithi071.stdout:4/470: readlink d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/l56 0 2024-04-23T02:14:01.082 INFO:tasks.workunit.client.0.smithi071.stdout:2/394: rmdir d0/d2/d49/d79 0 2024-04-23T02:14:01.082 INFO:tasks.workunit.client.0.smithi071.stdout:4/471: write d2/d2a/d2d/d39/d42/d49/d67/d1d/f61 [1293364,264] 0 2024-04-23T02:14:01.083 INFO:tasks.workunit.client.0.smithi071.stdout:6/435: creat d6/d3e/f8d x:0 0 0 2024-04-23T02:14:01.086 INFO:tasks.workunit.client.0.smithi071.stdout:7/430: mknod d3/d17/d41/d3a/c94 0 2024-04-23T02:14:01.087 INFO:tasks.workunit.client.0.smithi071.stdout:1/440: getdents db/df/d13/d72/d86 0 2024-04-23T02:14:01.088 INFO:tasks.workunit.client.0.smithi071.stdout:5/557: truncate d3/d1d/daf/f89 33877 0 2024-04-23T02:14:01.092 INFO:tasks.workunit.client.0.smithi071.stdout:9/481: symlink d1/d32/d41/la3 0 2024-04-23T02:14:01.093 INFO:tasks.workunit.client.0.smithi071.stdout:2/395: fsync d0/d2/d45/f5a 0 2024-04-23T02:14:01.094 INFO:tasks.workunit.client.0.smithi071.stdout:2/396: chown d0/d2/d26/d14/d1f/d36 44 1 2024-04-23T02:14:01.095 INFO:tasks.workunit.client.0.smithi071.stdout:8/491: rename dc/d10/c11 to dc/d10/d55/c9c 0 2024-04-23T02:14:01.099 INFO:tasks.workunit.client.0.smithi071.stdout:1/441: mkdir db/d11/d8d 0 2024-04-23T02:14:01.100 INFO:tasks.workunit.client.0.smithi071.stdout:5/558: rmdir d3/d1d/d43/dd2 39 2024-04-23T02:14:01.107 INFO:tasks.workunit.client.0.smithi071.stdout:1/442: readlink db/d11/l4d 0 2024-04-23T02:14:01.108 INFO:tasks.workunit.client.0.smithi071.stdout:1/443: dread - db/d34/d68/f87 zero size 2024-04-23T02:14:01.111 INFO:tasks.workunit.client.0.smithi071.stdout:6/436: write d6/d1f/d5d/f6f [258214,118885] 0 2024-04-23T02:14:01.118 INFO:tasks.workunit.client.0.smithi071.stdout:2/397: unlink d0/d2/d51/d56/c68 0 2024-04-23T02:14:01.122 INFO:tasks.workunit.client.0.smithi071.stdout:4/472: link d2/d2a/d2d/d39/d42/c48 d2/d2a/d73/d5e/c9b 0 2024-04-23T02:14:01.123 INFO:tasks.workunit.client.0.smithi071.stdout:7/431: truncate d3/d17/d41/d34/d59/d5f/f60 4546055 0 2024-04-23T02:14:01.123 INFO:tasks.workunit.client.0.smithi071.stdout:4/473: dread - d2/d2a/d2d/d7b/f8c zero size 2024-04-23T02:14:01.126 INFO:tasks.workunit.client.0.smithi071.stdout:6/437: symlink d6/d17/d77/l8e 0 2024-04-23T02:14:01.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:00 smithi192 ceph-mon[49662]: pgmap v27: 65 pgs: 65 active+clean; 1.7 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 43 MiB/s rd, 219 MiB/s wr, 367 op/s 2024-04-23T02:14:01.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:00 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1409135634' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:14:01.130 INFO:tasks.workunit.client.0.smithi071.stdout:2/398: rmdir d0/d2 39 2024-04-23T02:14:01.134 INFO:tasks.workunit.client.0.smithi071.stdout:7/432: read d3/d17/d41/d34/d59/f67 [87120,124153] 0 2024-04-23T02:14:01.138 INFO:tasks.workunit.client.0.smithi071.stdout:5/559: rename d3/da/d1a/db8/c37 to d3/d1d/cdb 0 2024-04-23T02:14:01.138 INFO:tasks.workunit.client.0.smithi071.stdout:5/560: chown d3/da/fcf 19827 1 2024-04-23T02:14:01.139 INFO:tasks.workunit.client.0.smithi071.stdout:6/438: mkdir d6/d3e/d8f 0 2024-04-23T02:14:01.139 INFO:tasks.workunit.client.0.smithi071.stdout:1/444: truncate db/df/d5f/d57/f5b 4106762 0 2024-04-23T02:14:01.140 INFO:tasks.workunit.client.0.smithi071.stdout:6/439: chown d6/d3e/d3f/l49 28862 1 2024-04-23T02:14:01.140 INFO:tasks.workunit.client.0.smithi071.stdout:1/445: chown db/df/d3a 61778832 1 2024-04-23T02:14:01.142 INFO:tasks.workunit.client.0.smithi071.stdout:8/492: fdatasync dc/d10/d55/f6c 0 2024-04-23T02:14:01.142 INFO:tasks.workunit.client.0.smithi071.stdout:1/446: dread - db/df/f19 zero size 2024-04-23T02:14:01.143 INFO:tasks.workunit.client.0.smithi071.stdout:7/433: fsync d3/d17/d41/f33 0 2024-04-23T02:14:01.144 INFO:tasks.workunit.client.0.smithi071.stdout:6/440: creat d6/d22/f90 x:0 0 0 2024-04-23T02:14:01.145 INFO:tasks.workunit.client.0.smithi071.stdout:4/474: chown d2/d2a/d2d/d39/d42/d49/d67/d3e/l52 1913 1 2024-04-23T02:14:01.147 INFO:tasks.workunit.client.0.smithi071.stdout:2/399: dread - d0/d2/d26/d14/d1f/d4f/f54 zero size 2024-04-23T02:14:01.148 INFO:tasks.workunit.client.0.smithi071.stdout:5/561: truncate d3/da/d1a/d1c/d46/d4e/f70 1011541 0 2024-04-23T02:14:01.150 INFO:tasks.workunit.client.0.smithi071.stdout:7/434: symlink d3/d17/l95 0 2024-04-23T02:14:01.151 INFO:tasks.workunit.client.0.smithi071.stdout:5/562: write d3/d1d/d43/d4b/f72 [4203601,104794] 0 2024-04-23T02:14:01.152 INFO:tasks.workunit.client.0.smithi071.stdout:6/441: dread d6/f18 [0,4194304] 0 2024-04-23T02:14:01.158 INFO:tasks.workunit.client.0.smithi071.stdout:4/475: rename d2/d2a/d2d/d39/d42/d49/d65 to d2/d2a/d2d/d39/d42/d49/d67/d1d/d57/d9c 0 2024-04-23T02:14:01.159 INFO:tasks.workunit.client.0.smithi071.stdout:7/435: dwrite d3/d17/d1c/d2a/d43/f4e [0,4194304] 0 2024-04-23T02:14:01.159 INFO:tasks.workunit.client.0.smithi071.stdout:4/476: chown d2/d2a/d2d/l6a 0 1 2024-04-23T02:14:01.160 INFO:tasks.workunit.client.0.smithi071.stdout:6/442: mkdir d6/d91 0 2024-04-23T02:14:01.162 INFO:tasks.workunit.client.0.smithi071.stdout:8/493: link dc/l77 dc/d10/d55/l9d 0 2024-04-23T02:14:01.164 INFO:tasks.workunit.client.0.smithi071.stdout:1/447: truncate db/d11/f1d 3358191 0 2024-04-23T02:14:01.165 INFO:tasks.workunit.client.0.smithi071.stdout:1/448: truncate db/d34/d68/f87 328212 0 2024-04-23T02:14:01.165 INFO:tasks.workunit.client.0.smithi071.stdout:2/400: link d0/d2/d26/d3a/f3d d0/d2/d45/d58/f7c 0 2024-04-23T02:14:01.167 INFO:tasks.workunit.client.0.smithi071.stdout:1/449: write db/d34/d68/f76 [90648,11167] 0 2024-04-23T02:14:01.168 INFO:tasks.workunit.client.0.smithi071.stdout:1/450: stat db/df/d13/d2b/f73 0 2024-04-23T02:14:01.169 INFO:tasks.workunit.client.0.smithi071.stdout:6/443: mkdir d6/d22/d92 0 2024-04-23T02:14:01.171 INFO:tasks.workunit.client.0.smithi071.stdout:4/477: rmdir d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d29 39 2024-04-23T02:14:01.174 INFO:tasks.workunit.client.0.smithi071.stdout:7/436: mkdir d3/d17/d1c/d2a/d73/d83/d92/d96 0 2024-04-23T02:14:01.176 INFO:tasks.workunit.client.0.smithi071.stdout:6/444: unlink d6/d17/d77/f7a 0 2024-04-23T02:14:01.177 INFO:tasks.workunit.client.0.smithi071.stdout:6/445: write d6/fc [2474600,44333] 0 2024-04-23T02:14:01.177 INFO:tasks.workunit.client.1.smithi192.stdout:8/676: sync 2024-04-23T02:14:01.179 INFO:tasks.workunit.client.0.smithi071.stdout:4/478: creat d2/d2a/d2d/d39/d42/d49/d67/d1d/d57/d9c/f9d x:0 0 0 2024-04-23T02:14:01.180 INFO:tasks.workunit.client.1.smithi192.stdout:8/677: creat d1/d12/d23/d4b/d73/d4c/d82/fef x:0 0 0 2024-04-23T02:14:01.181 INFO:tasks.workunit.client.0.smithi071.stdout:7/437: creat d3/d17/d41/d34/d39/f97 x:0 0 0 2024-04-23T02:14:01.182 INFO:tasks.workunit.client.0.smithi071.stdout:8/494: link dc/d78/f8f dc/d10/d8d/f9e 0 2024-04-23T02:14:01.183 INFO:tasks.workunit.client.1.smithi192.stdout:8/678: mkdir d1/d12/d23/d4b/d73/d7e/df0 0 2024-04-23T02:14:01.185 INFO:tasks.workunit.client.0.smithi071.stdout:5/563: rename d3/da/d1a/c23 to d3/d1d/d43/dd2/d5a/da7/da8/cdc 0 2024-04-23T02:14:01.188 INFO:tasks.workunit.client.0.smithi071.stdout:2/401: dread d0/d2/d26/d3a/f3d [0,4194304] 0 2024-04-23T02:14:01.190 INFO:tasks.workunit.client.0.smithi071.stdout:7/438: dwrite d3/d17/d1c/d31/f56 [0,4194304] 0 2024-04-23T02:14:01.190 INFO:tasks.workunit.client.0.smithi071.stdout:6/446: symlink d6/d22/d92/l93 0 2024-04-23T02:14:01.190 INFO:tasks.workunit.client.0.smithi071.stdout:8/495: dread - dc/d10/d30/f66 zero size 2024-04-23T02:14:01.193 INFO:tasks.workunit.client.0.smithi071.stdout:4/479: creat d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d29/f9e x:0 0 0 2024-04-23T02:14:01.197 INFO:tasks.workunit.client.0.smithi071.stdout:7/439: creat d3/d17/d1c/d31/f98 x:0 0 0 2024-04-23T02:14:01.202 INFO:tasks.workunit.client.0.smithi071.stdout:1/451: truncate f5 6406549 0 2024-04-23T02:14:01.203 INFO:tasks.workunit.client.0.smithi071.stdout:8/496: dwrite dc/d15/d50/f82 [0,4194304] 0 2024-04-23T02:14:01.206 INFO:tasks.workunit.client.0.smithi071.stdout:5/564: dread d3/da/f33 [0,4194304] 0 2024-04-23T02:14:01.211 INFO:tasks.workunit.client.0.smithi071.stdout:7/440: rename d3/d17/d41/d34/d39/c77 to d3/d17/d1c/d2a/d73/c99 0 2024-04-23T02:14:01.213 INFO:tasks.workunit.client.0.smithi071.stdout:2/402: getdents d0 0 2024-04-23T02:14:01.214 INFO:tasks.workunit.client.0.smithi071.stdout:8/497: truncate dc/d10/d55/f6c 4349650 0 2024-04-23T02:14:01.215 INFO:tasks.workunit.client.0.smithi071.stdout:5/565: creat d3/d34/d74/fdd x:0 0 0 2024-04-23T02:14:01.216 INFO:tasks.workunit.client.0.smithi071.stdout:7/441: write d3/d17/d1c/f20 [4015732,8195] 0 2024-04-23T02:14:01.217 INFO:tasks.workunit.client.0.smithi071.stdout:1/452: dwrite db/df/d13/d20/f4a [0,4194304] 0 2024-04-23T02:14:01.220 INFO:tasks.workunit.client.0.smithi071.stdout:8/498: creat dc/f9f x:0 0 0 2024-04-23T02:14:01.220 INFO:tasks.workunit.client.0.smithi071.stdout:7/442: write d3/d17/d41/f33 [4714754,61223] 0 2024-04-23T02:14:01.225 INFO:tasks.workunit.client.0.smithi071.stdout:1/453: mknod db/df/d13/d20/c8e 0 2024-04-23T02:14:01.226 INFO:tasks.workunit.client.0.smithi071.stdout:5/566: dread d3/f1f [0,4194304] 0 2024-04-23T02:14:01.227 INFO:tasks.workunit.client.0.smithi071.stdout:2/403: dwrite d0/d2/d51/d56/f61 [0,4194304] 0 2024-04-23T02:14:01.228 INFO:tasks.workunit.client.0.smithi071.stdout:1/454: fdatasync db/df/d13/d2b/f35 0 2024-04-23T02:14:01.230 INFO:tasks.workunit.client.0.smithi071.stdout:1/455: write db/d34/d8a/f8b [820165,33318] 0 2024-04-23T02:14:01.230 INFO:tasks.workunit.client.0.smithi071.stdout:5/567: creat d3/d1d/daf/d10/fde x:0 0 0 2024-04-23T02:14:01.232 INFO:tasks.workunit.client.0.smithi071.stdout:2/404: rename d0/d2/d51 to d0/d2/d45/d7d 0 2024-04-23T02:14:01.232 INFO:tasks.workunit.client.0.smithi071.stdout:5/568: write d3/d34/d74/fdd [1007627,63259] 0 2024-04-23T02:14:01.232 INFO:tasks.workunit.client.0.smithi071.stdout:1/456: creat db/d34/d8a/f8f x:0 0 0 2024-04-23T02:14:01.236 INFO:tasks.workunit.client.0.smithi071.stdout:5/569: truncate d3/d1d/daf/d10/da5/d5f/fd6 455151 0 2024-04-23T02:14:01.237 INFO:tasks.workunit.client.0.smithi071.stdout:5/570: fdatasync d3/d1d/d43/f69 0 2024-04-23T02:14:01.237 INFO:tasks.workunit.client.0.smithi071.stdout:2/405: rename d0/d2/c48 to d0/d2/d45/c7e 0 2024-04-23T02:14:01.238 INFO:tasks.workunit.client.0.smithi071.stdout:2/406: fsync d0/d2/d34/f50 0 2024-04-23T02:14:01.241 INFO:tasks.workunit.client.0.smithi071.stdout:1/457: link db/df/d13/d2b/f73 db/df/d13/d2f/d48/d85/f90 0 2024-04-23T02:14:01.241 INFO:tasks.workunit.client.0.smithi071.stdout:2/407: symlink d0/d2/d26/d14/d1f/d36/l7f 0 2024-04-23T02:14:01.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:00 smithi071 ceph-mon[39837]: pgmap v27: 65 pgs: 65 active+clean; 1.7 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 43 MiB/s rd, 219 MiB/s wr, 367 op/s 2024-04-23T02:14:01.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:00 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1409135634' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:14:01.245 INFO:tasks.workunit.client.0.smithi071.stdout:2/408: rename d0/d2/d3c to d0/d2/d26/df/d30/d80 0 2024-04-23T02:14:01.248 INFO:tasks.workunit.client.0.smithi071.stdout:1/458: dwrite db/d34/f7d [0,4194304] 0 2024-04-23T02:14:01.316 INFO:tasks.workunit.client.1.smithi192.stdout:0/640: sync 2024-04-23T02:14:01.316 INFO:tasks.workunit.client.1.smithi192.stdout:9/473: sync 2024-04-23T02:14:01.317 INFO:tasks.workunit.client.0.smithi071.stdout:2/409: read d0/d2/d26/d14/f43 [7541516,91030] 0 2024-04-23T02:14:01.317 INFO:tasks.workunit.client.0.smithi071.stdout:2/410: chown d0/d2/d26/d14/f42 28688044 1 2024-04-23T02:14:01.317 INFO:tasks.workunit.client.1.smithi192.stdout:9/474: fsync d0/d9/d44/d51/d5b/d61/f7b 0 2024-04-23T02:14:01.318 INFO:tasks.workunit.client.0.smithi071.stdout:2/411: chown d0/d2/d26/df/c2a 1 1 2024-04-23T02:14:01.319 INFO:tasks.workunit.client.0.smithi071.stdout:2/412: write d0/d2/d26/d14/f43 [7476584,110528] 0 2024-04-23T02:14:01.320 INFO:tasks.workunit.client.1.smithi192.stdout:0/641: dread d6/df/d13/d25/d3a/f62 [0,4194304] 0 2024-04-23T02:14:01.320 INFO:tasks.workunit.client.1.smithi192.stdout:9/475: truncate d0/d9/f25 503262 0 2024-04-23T02:14:01.324 INFO:tasks.workunit.client.0.smithi071.stdout:6/447: truncate d6/d17/d77/d85/f88 496709 0 2024-04-23T02:14:01.325 INFO:tasks.workunit.client.0.smithi071.stdout:4/480: truncate d2/d2a/d2d/d39/d42/d49/d67/d3e/f6f 428706 0 2024-04-23T02:14:01.326 INFO:tasks.workunit.client.1.smithi192.stdout:9/476: link d0/d9/d44/f45 d0/fa6 0 2024-04-23T02:14:01.326 INFO:tasks.workunit.client.0.smithi071.stdout:2/413: getdents d0/d2/d45/d7d/d56 0 2024-04-23T02:14:01.327 INFO:tasks.workunit.client.0.smithi071.stdout:4/481: chown d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f71 72636 1 2024-04-23T02:14:01.327 INFO:tasks.workunit.client.0.smithi071.stdout:6/448: stat d6/d1f/c2a 0 2024-04-23T02:14:01.328 INFO:tasks.workunit.client.1.smithi192.stdout:9/477: mknod d0/d9/d44/d51/d5b/ca7 0 2024-04-23T02:14:01.329 INFO:tasks.workunit.client.1.smithi192.stdout:9/478: write d0/f16 [1499454,92441] 0 2024-04-23T02:14:01.330 INFO:tasks.workunit.client.0.smithi071.stdout:6/449: creat d6/f94 x:0 0 0 2024-04-23T02:14:01.331 INFO:tasks.workunit.client.0.smithi071.stdout:4/482: write d2/d2a/d2d/d39/d42/f55 [4846792,37276] 0 2024-04-23T02:14:01.333 INFO:tasks.workunit.client.1.smithi192.stdout:9/479: rename d0/l62 to d0/la8 0 2024-04-23T02:14:01.334 INFO:tasks.workunit.client.0.smithi071.stdout:7/443: write d3/ff [2426424,21287] 0 2024-04-23T02:14:01.334 INFO:tasks.workunit.client.1.smithi192.stdout:0/642: write d6/df/d47/f4c [744329,11277] 0 2024-04-23T02:14:01.336 INFO:tasks.workunit.client.1.smithi192.stdout:9/480: symlink d0/d9/d44/d51/la9 0 2024-04-23T02:14:01.336 INFO:tasks.workunit.client.0.smithi071.stdout:2/414: rmdir d0/d2/d45/d58/d71 0 2024-04-23T02:14:01.337 INFO:tasks.workunit.client.0.smithi071.stdout:8/499: write dc/d10/d8d/f92 [427421,29484] 0 2024-04-23T02:14:01.337 INFO:tasks.workunit.client.1.smithi192.stdout:0/643: truncate d6/df/d35/d49/f54 8634149 0 2024-04-23T02:14:01.338 INFO:tasks.workunit.client.0.smithi071.stdout:6/450: getdents d6/d3e/d8f 0 2024-04-23T02:14:01.338 INFO:tasks.workunit.client.1.smithi192.stdout:9/481: symlink d0/d9/d44/d58/laa 0 2024-04-23T02:14:01.338 INFO:tasks.workunit.client.0.smithi071.stdout:2/415: dread - d0/d2/d26/df/f3b zero size 2024-04-23T02:14:01.342 INFO:tasks.workunit.client.0.smithi071.stdout:8/500: chown dc/d15/d2d/f38 4 1 2024-04-23T02:14:01.342 INFO:tasks.workunit.client.0.smithi071.stdout:2/416: readlink d0/d2/d26/d14/d1f/d36/l6b 0 2024-04-23T02:14:01.342 INFO:tasks.workunit.client.0.smithi071.stdout:4/483: mknod d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/c9f 0 2024-04-23T02:14:01.343 INFO:tasks.workunit.client.1.smithi192.stdout:0/644: symlink d6/df/d13/d25/d3a/d93/ld3 0 2024-04-23T02:14:01.343 INFO:tasks.workunit.client.1.smithi192.stdout:9/482: symlink d0/d9/d44/d58/d89/lab 0 2024-04-23T02:14:01.344 INFO:tasks.workunit.client.1.smithi192.stdout:0/645: creat d6/df/d13/d25/d94/dd0/fd4 x:0 0 0 2024-04-23T02:14:01.344 INFO:tasks.workunit.client.1.smithi192.stdout:0/646: chown d6/df/d13/c7c 4127 1 2024-04-23T02:14:01.344 INFO:tasks.workunit.client.0.smithi071.stdout:8/501: chown dc/d10/c8b 1 1 2024-04-23T02:14:01.346 INFO:tasks.workunit.client.0.smithi071.stdout:7/444: rename d3/d17/d1c/d31/l7b to d3/d17/d1c/d2a/d73/d83/d92/d96/l9a 0 2024-04-23T02:14:01.347 INFO:tasks.workunit.client.0.smithi071.stdout:2/417: dwrite d0/d2/d45/f5a [0,4194304] 0 2024-04-23T02:14:01.347 INFO:tasks.workunit.client.1.smithi192.stdout:9/483: dwrite d0/d9/d44/d4f/f1c [4194304,4194304] 0 2024-04-23T02:14:01.347 INFO:tasks.workunit.client.0.smithi071.stdout:4/484: symlink d2/d2a/d2d/d39/d42/d49/d67/d7e/la0 0 2024-04-23T02:14:01.351 INFO:tasks.workunit.client.1.smithi192.stdout:0/647: dwrite d6/df/d13/d25/d26/d67/f69 [0,4194304] 0 2024-04-23T02:14:01.353 INFO:tasks.workunit.client.1.smithi192.stdout:9/484: dread d0/d9/d44/d4f/f41 [0,4194304] 0 2024-04-23T02:14:01.353 INFO:tasks.workunit.client.1.smithi192.stdout:0/648: chown d6/df/d13/d6b/d73/da9/laa 4166145 1 2024-04-23T02:14:01.353 INFO:tasks.workunit.client.0.smithi071.stdout:6/451: rename d6/d22/d79 to d6/d1f/d5d/d95 0 2024-04-23T02:14:01.354 INFO:tasks.workunit.client.0.smithi071.stdout:6/452: readlink d6/d22/l39 0 2024-04-23T02:14:01.356 INFO:tasks.workunit.client.1.smithi192.stdout:9/485: read d0/d9/d44/d51/d5b/f67 [2348566,116442] 0 2024-04-23T02:14:01.356 INFO:tasks.workunit.client.0.smithi071.stdout:5/571: dwrite d3/da/d1a/f26 [4194304,4194304] 0 2024-04-23T02:14:01.357 INFO:tasks.workunit.client.1.smithi192.stdout:0/649: rmdir d6/d4b 39 2024-04-23T02:14:01.359 INFO:tasks.workunit.client.0.smithi071.stdout:8/502: write dc/d10/f85 [133109,127450] 0 2024-04-23T02:14:01.361 INFO:tasks.workunit.client.1.smithi192.stdout:0/650: symlink d6/df/d13/d6b/dc1/ld5 0 2024-04-23T02:14:01.362 INFO:tasks.workunit.client.0.smithi071.stdout:2/418: rename d0/d2/f67 to d0/d2/d26/d3a/f81 0 2024-04-23T02:14:01.367 INFO:tasks.workunit.client.1.smithi192.stdout:0/651: mkdir d6/df/d13/d25/d3a/d93/dd6 0 2024-04-23T02:14:01.368 INFO:tasks.workunit.client.1.smithi192.stdout:9/486: dwrite d0/d9/d44/d51/d5b/f69 [0,4194304] 0 2024-04-23T02:14:01.369 INFO:tasks.workunit.client.0.smithi071.stdout:1/459: write db/d11/f1d [4348889,87769] 0 2024-04-23T02:14:01.371 INFO:tasks.workunit.client.0.smithi071.stdout:5/572: read d3/da/d1a/d1c/d46/d4e/f84 [1532067,127798] 0 2024-04-23T02:14:01.372 INFO:tasks.workunit.client.0.smithi071.stdout:8/503: rename dc/d10 to dc/d15/da0 0 2024-04-23T02:14:01.373 INFO:tasks.workunit.client.0.smithi071.stdout:7/445: creat d3/d17/d41/d34/f9b x:0 0 0 2024-04-23T02:14:01.374 INFO:tasks.workunit.client.0.smithi071.stdout:7/446: chown d3/d17/d1c/f8e 157393 1 2024-04-23T02:14:01.375 INFO:tasks.workunit.client.1.smithi192.stdout:0/652: mkdir d6/df/d13/d6b/dc1/dd7 0 2024-04-23T02:14:01.376 INFO:tasks.workunit.client.0.smithi071.stdout:5/573: write d3/da/f92 [264974,1362] 0 2024-04-23T02:14:01.377 INFO:tasks.workunit.client.0.smithi071.stdout:6/453: dread d6/d22/f35 [0,4194304] 0 2024-04-23T02:14:01.377 INFO:tasks.workunit.client.0.smithi071.stdout:4/485: getdents d2/d2a/d2d/d39/d42/d49/d67/d1d/d57 0 2024-04-23T02:14:01.378 INFO:tasks.workunit.client.0.smithi071.stdout:5/574: chown d3/da/d1a 34 1 2024-04-23T02:14:01.379 INFO:tasks.workunit.client.0.smithi071.stdout:1/460: symlink db/d11/d7f/l91 0 2024-04-23T02:14:01.380 INFO:tasks.workunit.client.1.smithi192.stdout:1/317: sync 2024-04-23T02:14:01.380 INFO:tasks.workunit.client.1.smithi192.stdout:7/534: sync 2024-04-23T02:14:01.380 INFO:tasks.workunit.client.1.smithi192.stdout:5/528: sync 2024-04-23T02:14:01.381 INFO:tasks.workunit.client.1.smithi192.stdout:9/487: dwrite d0/d9/d44/d51/d5b/d61/f7b [0,4194304] 0 2024-04-23T02:14:01.381 INFO:tasks.workunit.client.0.smithi071.stdout:6/454: creat d6/d17/d77/f96 x:0 0 0 2024-04-23T02:14:01.383 INFO:tasks.workunit.client.0.smithi071.stdout:6/455: chown d6/d1f/d46/f6e 789 1 2024-04-23T02:14:01.383 INFO:tasks.workunit.client.0.smithi071.stdout:4/486: truncate d2/d2a/d2d/d39/d42/d49/d67/d3e/f93 372607 0 2024-04-23T02:14:01.383 INFO:tasks.workunit.client.1.smithi192.stdout:1/318: write dc/d16/d19/d30/f3f [8905045,76290] 0 2024-04-23T02:14:01.384 INFO:tasks.workunit.client.0.smithi071.stdout:8/504: rename dc/d15/da0/d32/l89 to dc/d15/d50/la1 0 2024-04-23T02:14:01.384 INFO:tasks.workunit.client.0.smithi071.stdout:6/456: chown d6/d22/f28 0 1 2024-04-23T02:14:01.384 INFO:tasks.workunit.client.0.smithi071.stdout:6/457: stat d6/c10 0 2024-04-23T02:14:01.385 INFO:tasks.workunit.client.0.smithi071.stdout:1/461: mkdir db/d92 0 2024-04-23T02:14:01.388 INFO:tasks.workunit.client.0.smithi071.stdout:4/487: symlink d2/d2a/d2d/d7b/la1 0 2024-04-23T02:14:01.390 INFO:tasks.workunit.client.0.smithi071.stdout:6/458: rmdir d6/d3e 39 2024-04-23T02:14:01.391 INFO:tasks.workunit.client.1.smithi192.stdout:5/529: mknod d2/db/d16/d1b/d3d/d40/d49/caf 0 2024-04-23T02:14:01.392 INFO:tasks.workunit.client.0.smithi071.stdout:4/488: read - d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d10/f60 zero size 2024-04-23T02:14:01.392 INFO:tasks.workunit.client.0.smithi071.stdout:6/459: dread - d6/d17/d77/f8c zero size 2024-04-23T02:14:01.392 INFO:tasks.workunit.client.1.smithi192.stdout:9/488: stat d0/d9/d44/d58/d89 0 2024-04-23T02:14:01.392 INFO:tasks.workunit.client.0.smithi071.stdout:5/575: mknod d3/d1d/daf/cdf 0 2024-04-23T02:14:01.394 INFO:tasks.workunit.client.1.smithi192.stdout:5/530: chown d2/db/d16/d1b/d3d/f67 18444 1 2024-04-23T02:14:01.395 INFO:tasks.workunit.client.0.smithi071.stdout:1/462: rename db/d11/l4d to db/df/d13/d72/l93 0 2024-04-23T02:14:01.395 INFO:tasks.workunit.client.0.smithi071.stdout:1/463: read - db/df/d5f/f82 zero size 2024-04-23T02:14:01.396 INFO:tasks.workunit.client.0.smithi071.stdout:4/489: mkdir d2/d2a/da2 0 2024-04-23T02:14:01.397 INFO:tasks.workunit.client.1.smithi192.stdout:0/653: dwrite d6/df/d56/db1/fbe [0,4194304] 0 2024-04-23T02:14:01.398 INFO:tasks.workunit.client.0.smithi071.stdout:1/464: chown db/df/d13/d2b/c37 228059 1 2024-04-23T02:14:01.398 INFO:tasks.workunit.client.0.smithi071.stdout:4/490: chown d2/d2a/l51 53271 1 2024-04-23T02:14:01.398 INFO:tasks.workunit.client.0.smithi071.stdout:6/460: unlink d6/d1f/d5d/f6d 0 2024-04-23T02:14:01.399 INFO:tasks.workunit.client.1.smithi192.stdout:0/654: write d6/df/d13/d6b/fab [112430,17430] 0 2024-04-23T02:14:01.400 INFO:tasks.workunit.client.0.smithi071.stdout:6/461: stat d6/d1f/d46/c7d 0 2024-04-23T02:14:01.408 INFO:tasks.workunit.client.0.smithi071.stdout:6/462: write d6/d1f/f68 [417873,12116] 0 2024-04-23T02:14:01.408 INFO:tasks.workunit.client.0.smithi071.stdout:7/447: dread d3/d17/d41/d34/d59/d5f/f60 [0,4194304] 0 2024-04-23T02:14:01.409 INFO:tasks.workunit.client.0.smithi071.stdout:1/465: dwrite db/df/f39 [0,4194304] 0 2024-04-23T02:14:01.409 INFO:tasks.workunit.client.1.smithi192.stdout:1/319: mknod dc/d16/d19/d30/c59 0 2024-04-23T02:14:01.411 INFO:tasks.workunit.client.1.smithi192.stdout:5/531: mknod d2/db/d16/d1b/d3d/d40/d49/cb0 0 2024-04-23T02:14:01.413 INFO:tasks.workunit.client.0.smithi071.stdout:4/491: creat d2/d2a/d2d/d39/d42/d49/d67/d7e/d82/fa3 x:0 0 0 2024-04-23T02:14:01.413 INFO:tasks.workunit.client.0.smithi071.stdout:1/466: write db/df/f19 [766499,86526] 0 2024-04-23T02:14:01.413 INFO:tasks.workunit.client.0.smithi071.stdout:2/419: dread d0/d2/d26/d14/f23 [0,4194304] 0 2024-04-23T02:14:01.413 INFO:tasks.workunit.client.1.smithi192.stdout:1/320: symlink dc/d51/l5a 0 2024-04-23T02:14:01.414 INFO:tasks.workunit.client.0.smithi071.stdout:6/463: symlink d6/d1f/d5d/l97 0 2024-04-23T02:14:01.415 INFO:tasks.workunit.client.0.smithi071.stdout:2/420: mkdir d0/d82 0 2024-04-23T02:14:01.417 INFO:tasks.workunit.client.0.smithi071.stdout:1/467: symlink db/d11/l94 0 2024-04-23T02:14:01.417 INFO:tasks.workunit.client.1.smithi192.stdout:0/655: mknod d6/d4b/d9d/dc7/cd8 0 2024-04-23T02:14:01.418 INFO:tasks.workunit.client.1.smithi192.stdout:3/455: sync 2024-04-23T02:14:01.418 INFO:tasks.workunit.client.1.smithi192.stdout:4/657: sync 2024-04-23T02:14:01.418 INFO:tasks.workunit.client.1.smithi192.stdout:6/617: sync 2024-04-23T02:14:01.418 INFO:tasks.workunit.client.1.smithi192.stdout:2/551: sync 2024-04-23T02:14:01.418 INFO:tasks.workunit.client.1.smithi192.stdout:8/679: sync 2024-04-23T02:14:01.418 INFO:tasks.workunit.client.1.smithi192.stdout:7/535: link d5/d17/d1c/d59/d5a/d41/f54 d5/d17/f9e 0 2024-04-23T02:14:01.419 INFO:tasks.workunit.client.1.smithi192.stdout:6/618: readlink d3/d15/d1b/d21/d3a/l7a 0 2024-04-23T02:14:01.420 INFO:tasks.workunit.client.0.smithi071.stdout:6/464: creat d6/d1f/d5d/d95/f98 x:0 0 0 2024-04-23T02:14:01.420 INFO:tasks.workunit.client.1.smithi192.stdout:1/321: rename dc/d16/f31 to dc/d16/d19/d30/f5b 0 2024-04-23T02:14:01.421 INFO:tasks.workunit.client.0.smithi071.stdout:6/465: fsync d6/d17/d77/f96 0 2024-04-23T02:14:01.421 INFO:tasks.workunit.client.1.smithi192.stdout:2/552: chown d8/d25/f38 12981 1 2024-04-23T02:14:01.421 INFO:tasks.workunit.client.0.smithi071.stdout:4/492: truncate d2/d2a/d2d/d39/d42/d49/d67/f3a 2023439 0 2024-04-23T02:14:01.422 INFO:tasks.workunit.client.0.smithi071.stdout:6/466: fsync d6/d17/d77/f96 0 2024-04-23T02:14:01.422 INFO:tasks.workunit.client.1.smithi192.stdout:8/680: creat d1/de1/ff1 x:0 0 0 2024-04-23T02:14:01.423 INFO:tasks.workunit.client.0.smithi071.stdout:1/468: creat db/df/d13/d20/d75/f95 x:0 0 0 2024-04-23T02:14:01.423 INFO:tasks.workunit.client.1.smithi192.stdout:2/553: stat d8/d26/d27/d31/f8b 0 2024-04-23T02:14:01.423 INFO:tasks.workunit.client.1.smithi192.stdout:8/681: readlink d1/d9/l3a 0 2024-04-23T02:14:01.424 INFO:tasks.workunit.client.1.smithi192.stdout:7/536: chown d5/d17/d1c/c73 8936153 1 2024-04-23T02:14:01.424 INFO:tasks.workunit.client.1.smithi192.stdout:8/682: chown d1/df/l56 934 1 2024-04-23T02:14:01.426 INFO:tasks.workunit.client.0.smithi071.stdout:6/467: truncate d6/d3e/f75 526092 0 2024-04-23T02:14:01.427 INFO:tasks.workunit.client.1.smithi192.stdout:6/619: mknod d3/d28/d79/da0/ccc 0 2024-04-23T02:14:01.427 INFO:tasks.workunit.client.1.smithi192.stdout:0/656: mknod d6/df/d56/d85/cd9 0 2024-04-23T02:14:01.429 INFO:tasks.workunit.client.1.smithi192.stdout:6/620: readlink d3/d28/l3e 0 2024-04-23T02:14:01.430 INFO:tasks.workunit.client.0.smithi071.stdout:7/448: dwrite d3/d17/d41/d34/d59/d5f/f6c [0,4194304] 0 2024-04-23T02:14:01.433 INFO:tasks.workunit.client.1.smithi192.stdout:2/554: rename d8/d26/c5a to d8/d26/d29/d43/d5d/cb0 0 2024-04-23T02:14:01.434 INFO:tasks.workunit.client.1.smithi192.stdout:8/683: mknod d1/d12/d23/d4b/d73/d4c/d82/cf2 0 2024-04-23T02:14:01.434 INFO:tasks.workunit.client.1.smithi192.stdout:2/555: truncate d8/f86 742034 0 2024-04-23T02:14:01.435 INFO:tasks.workunit.client.0.smithi071.stdout:7/449: mknod d3/d17/d41/d37/c9c 0 2024-04-23T02:14:01.435 INFO:tasks.workunit.client.0.smithi071.stdout:6/468: getdents d6/d17/d77/d85 0 2024-04-23T02:14:01.436 INFO:tasks.workunit.client.1.smithi192.stdout:2/556: dread - d8/d26/d27/d31/f8b zero size 2024-04-23T02:14:01.438 INFO:tasks.workunit.client.1.smithi192.stdout:3/456: symlink d1/d3/d17/d1f/d33/d24/d34/d6c/d84/l9b 0 2024-04-23T02:14:01.439 INFO:tasks.workunit.client.1.smithi192.stdout:6/621: rename d3/d6c/f71 to d3/d18/d58/fcd 0 2024-04-23T02:14:01.441 INFO:tasks.workunit.client.0.smithi071.stdout:6/469: symlink d6/d17/d77/l99 0 2024-04-23T02:14:01.443 INFO:tasks.workunit.client.0.smithi071.stdout:6/470: dread - d6/d1f/d5d/f87 zero size 2024-04-23T02:14:01.444 INFO:tasks.workunit.client.1.smithi192.stdout:0/657: creat d6/fda x:0 0 0 2024-04-23T02:14:01.445 INFO:tasks.workunit.client.0.smithi071.stdout:7/450: dwrite d3/d17/d1c/d31/f48 [0,4194304] 0 2024-04-23T02:14:01.446 INFO:tasks.workunit.client.0.smithi071.stdout:1/469: rmdir db/df/d13 39 2024-04-23T02:14:01.447 INFO:tasks.workunit.client.1.smithi192.stdout:8/684: dread d1/d9/fee [0,4194304] 0 2024-04-23T02:14:01.448 INFO:tasks.workunit.client.1.smithi192.stdout:0/658: write d6/df/d35/f74 [3060511,23157] 0 2024-04-23T02:14:01.448 INFO:tasks.workunit.client.1.smithi192.stdout:2/557: dwrite d8/d26/d29/f41 [0,4194304] 0 2024-04-23T02:14:01.449 INFO:tasks.workunit.client.0.smithi071.stdout:2/421: write d0/d2/d26/df/d30/d80/f4e [1240648,63897] 0 2024-04-23T02:14:01.452 INFO:tasks.workunit.client.1.smithi192.stdout:0/659: fdatasync d6/df/d47/f89 0 2024-04-23T02:14:01.456 INFO:tasks.workunit.client.1.smithi192.stdout:2/558: write d8/d26/d27/d57/d69/f8a [1547933,996] 0 2024-04-23T02:14:01.457 INFO:tasks.workunit.client.0.smithi071.stdout:1/470: dread - db/df/d13/d20/f53 zero size 2024-04-23T02:14:01.458 INFO:tasks.workunit.client.1.smithi192.stdout:7/537: rmdir d5/d17/d1c/d59/d5a 39 2024-04-23T02:14:01.459 INFO:tasks.workunit.client.0.smithi071.stdout:6/471: creat d6/d1f/f9a x:0 0 0 2024-04-23T02:14:01.459 INFO:tasks.workunit.client.1.smithi192.stdout:6/622: dwrite d3/d18/f78 [0,4194304] 0 2024-04-23T02:14:01.462 INFO:tasks.workunit.client.1.smithi192.stdout:5/532: truncate d2/db/d16/d1b/f1f 3413803 0 2024-04-23T02:14:01.464 INFO:tasks.workunit.client.1.smithi192.stdout:8/685: mknod d1/d12/d23/d4b/d73/d4c/cf3 0 2024-04-23T02:14:01.464 INFO:tasks.workunit.client.0.smithi071.stdout:6/472: fsync f1 0 2024-04-23T02:14:01.466 INFO:tasks.workunit.client.0.smithi071.stdout:4/493: dwrite d2/d2a/f80 [0,4194304] 0 2024-04-23T02:14:01.467 INFO:tasks.workunit.client.1.smithi192.stdout:7/538: rmdir d5/d17/d1c/d57 39 2024-04-23T02:14:01.468 INFO:tasks.workunit.client.1.smithi192.stdout:9/489: dwrite d0/d9/f65 [0,4194304] 0 2024-04-23T02:14:01.469 INFO:tasks.workunit.client.0.smithi071.stdout:4/494: mknod d2/d2a/d2d/d7b/ca4 0 2024-04-23T02:14:01.471 INFO:tasks.workunit.client.1.smithi192.stdout:7/539: stat d5/d43/d4c/d5c/d97/c9d 0 2024-04-23T02:14:01.473 INFO:tasks.workunit.client.1.smithi192.stdout:2/559: mknod d8/d26/d29/da4/dab/cb1 0 2024-04-23T02:14:01.475 INFO:tasks.workunit.client.0.smithi071.stdout:4/495: creat d2/d53/fa5 x:0 0 0 2024-04-23T02:14:01.475 INFO:tasks.workunit.client.1.smithi192.stdout:1/322: truncate f9 5902041 0 2024-04-23T02:14:01.475 INFO:tasks.workunit.client.0.smithi071.stdout:4/496: readlink d2/d53/l5f 0 2024-04-23T02:14:01.476 INFO:tasks.workunit.client.1.smithi192.stdout:2/560: write d8/d26/f78 [156079,40760] 0 2024-04-23T02:14:01.476 INFO:tasks.workunit.client.0.smithi071.stdout:2/422: dwrite d0/f2e [0,4194304] 0 2024-04-23T02:14:01.477 INFO:tasks.workunit.client.1.smithi192.stdout:5/533: creat d2/db/d16/d1b/d3d/d40/d49/fb1 x:0 0 0 2024-04-23T02:14:01.480 INFO:tasks.workunit.client.0.smithi071.stdout:2/423: write d0/d2/fb [832863,18746] 0 2024-04-23T02:14:01.481 INFO:tasks.workunit.client.1.smithi192.stdout:3/457: rmdir d1/d3/d17/d1f/d33/d24/d34 39 2024-04-23T02:14:01.481 INFO:tasks.workunit.client.1.smithi192.stdout:9/490: write d0/d9/d44/f47 [477115,117206] 0 2024-04-23T02:14:01.485 INFO:tasks.workunit.client.0.smithi071.stdout:7/451: dread d3/d17/d1c/f45 [4194304,4194304] 0 2024-04-23T02:14:01.486 INFO:tasks.workunit.client.1.smithi192.stdout:1/323: stat dc/d14/d15/l25 0 2024-04-23T02:14:01.487 INFO:tasks.workunit.client.1.smithi192.stdout:5/534: mknod d2/d6/d2d/cb2 0 2024-04-23T02:14:01.489 INFO:tasks.workunit.client.1.smithi192.stdout:6/623: dwrite d3/d15/d1b/d66/d80/fb9 [4194304,4194304] 0 2024-04-23T02:14:01.490 INFO:tasks.workunit.client.0.smithi071.stdout:2/424: unlink d0/d2/f4d 0 2024-04-23T02:14:01.491 INFO:tasks.workunit.client.0.smithi071.stdout:7/452: creat d3/d17/d41/d34/d59/f9d x:0 0 0 2024-04-23T02:14:01.491 INFO:tasks.workunit.client.1.smithi192.stdout:2/561: truncate d8/d26/d27/f70 5127281 0 2024-04-23T02:14:01.493 INFO:tasks.workunit.client.0.smithi071.stdout:7/453: creat d3/d51/d80/f9e x:0 0 0 2024-04-23T02:14:01.494 INFO:tasks.workunit.client.1.smithi192.stdout:8/686: creat d1/df/ff4 x:0 0 0 2024-04-23T02:14:01.495 INFO:tasks.workunit.client.0.smithi071.stdout:7/454: chown d3/d17/d1c/d31/f81 2033864861 1 2024-04-23T02:14:01.495 INFO:tasks.workunit.client.0.smithi071.stdout:2/425: fdatasync d0/d2/d26/f32 0 2024-04-23T02:14:01.495 INFO:tasks.workunit.client.0.smithi071.stdout:4/497: truncate d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f8e 483177 0 2024-04-23T02:14:01.496 INFO:tasks.workunit.client.1.smithi192.stdout:4/658: dwrite de/d15/d2a/d74/d92/da0/dca/f5f [0,4194304] 0 2024-04-23T02:14:01.496 INFO:tasks.workunit.client.1.smithi192.stdout:3/458: fdatasync d1/d3/d17/d2b/f51 0 2024-04-23T02:14:01.497 INFO:tasks.workunit.client.0.smithi071.stdout:4/498: chown d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/c9f 8152932 1 2024-04-23T02:14:01.498 INFO:tasks.workunit.client.0.smithi071.stdout:2/426: mkdir d0/d2/d34/d83 0 2024-04-23T02:14:01.501 INFO:tasks.workunit.client.0.smithi071.stdout:4/499: creat d2/d2a/d2d/d39/fa6 x:0 0 0 2024-04-23T02:14:01.502 INFO:tasks.workunit.client.1.smithi192.stdout:0/660: truncate d6/f6d 509015 0 2024-04-23T02:14:01.505 INFO:tasks.workunit.client.0.smithi071.stdout:4/500: write d2/d2a/d2d/f54 [1285348,87142] 0 2024-04-23T02:14:01.505 INFO:tasks.workunit.client.0.smithi071.stdout:7/455: dwrite d3/d17/d41/d34/d59/d5f/f6c [0,4194304] 0 2024-04-23T02:14:01.506 INFO:tasks.workunit.client.1.smithi192.stdout:5/535: mkdir d2/d6/d2d/db3 0 2024-04-23T02:14:01.506 INFO:tasks.workunit.client.0.smithi071.stdout:2/427: rename d0/d2/d34/d55/f64 to d0/d2/d34/f84 0 2024-04-23T02:14:01.507 INFO:tasks.workunit.client.0.smithi071.stdout:4/501: chown d2/d53/l5f 40872964 1 2024-04-23T02:14:01.508 INFO:tasks.workunit.client.1.smithi192.stdout:1/324: truncate dc/d16/f1f 1454406 0 2024-04-23T02:14:01.510 INFO:tasks.workunit.client.1.smithi192.stdout:8/687: read - d1/d12/d68/f83 zero size 2024-04-23T02:14:01.510 INFO:tasks.workunit.client.0.smithi071.stdout:7/456: mknod d3/d17/d41/d34/d59/d5f/c9f 0 2024-04-23T02:14:01.511 INFO:tasks.workunit.client.0.smithi071.stdout:2/428: unlink d0/d2/d26/d14/f43 0 2024-04-23T02:14:01.511 INFO:tasks.workunit.client.1.smithi192.stdout:2/562: dread d8/d25/f3a [0,4194304] 0 2024-04-23T02:14:01.512 INFO:tasks.workunit.client.0.smithi071.stdout:4/502: mknod d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d29/ca7 0 2024-04-23T02:14:01.513 INFO:tasks.workunit.client.1.smithi192.stdout:5/536: mkdir d2/d6/db4 0 2024-04-23T02:14:01.515 INFO:tasks.workunit.client.0.smithi071.stdout:2/429: mkdir d0/d2/d26/d14/d1f/d36/d5c/d85 0 2024-04-23T02:14:01.515 INFO:tasks.workunit.client.1.smithi192.stdout:3/459: rmdir d1/d3/d17/d1f/d33/d24/d34/d38/d7b 39 2024-04-23T02:14:01.515 INFO:tasks.workunit.client.1.smithi192.stdout:4/659: dwrite de/d15/d2a/d74/d92/da0/dca/d49/d70/f91 [0,4194304] 0 2024-04-23T02:14:01.516 INFO:tasks.workunit.client.0.smithi071.stdout:4/503: rmdir d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d4f 39 2024-04-23T02:14:01.521 INFO:tasks.workunit.client.1.smithi192.stdout:8/688: stat d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/fd9 0 2024-04-23T02:14:01.522 INFO:tasks.workunit.client.0.smithi071.stdout:4/504: mknod d2/d2a/d2d/d39/d42/d49/ca8 0 2024-04-23T02:14:01.522 INFO:tasks.workunit.client.0.smithi071.stdout:2/430: dwrite d0/d2/d26/df/f69 [0,4194304] 0 2024-04-23T02:14:01.524 INFO:tasks.workunit.client.1.smithi192.stdout:1/325: link dc/l1e dc/d51/l5c 0 2024-04-23T02:14:01.526 INFO:tasks.workunit.client.1.smithi192.stdout:5/537: mknod d2/d6/db4/cb5 0 2024-04-23T02:14:01.526 INFO:tasks.workunit.client.0.smithi071.stdout:4/505: readlink d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/l4a 0 2024-04-23T02:14:01.527 INFO:tasks.workunit.client.1.smithi192.stdout:9/491: dwrite d0/d9/d44/d4f/f41 [0,4194304] 0 2024-04-23T02:14:01.529 INFO:tasks.workunit.client.1.smithi192.stdout:5/538: dread - d2/d2e/fab zero size 2024-04-23T02:14:01.529 INFO:tasks.workunit.client.0.smithi071.stdout:4/506: write d2/d2a/d2d/d39/d42/d49/d67/f1e [2441200,51729] 0 2024-04-23T02:14:01.531 INFO:tasks.workunit.client.0.smithi071.stdout:4/507: write d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c/f7c [750661,37708] 0 2024-04-23T02:14:01.535 INFO:tasks.workunit.client.1.smithi192.stdout:8/689: creat d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/da3/ff5 x:0 0 0 2024-04-23T02:14:01.536 INFO:tasks.workunit.client.1.smithi192.stdout:3/460: creat d1/d3/d17/d1f/d33/d24/d34/d6c/d84/d5d/f9c x:0 0 0 2024-04-23T02:14:01.538 INFO:tasks.workunit.client.1.smithi192.stdout:8/690: fsync d1/d12/d23/d4b/d73/d7e/d88/fc6 0 2024-04-23T02:14:01.539 INFO:tasks.workunit.client.1.smithi192.stdout:5/539: mkdir d2/d6/db4/db6 0 2024-04-23T02:14:01.541 INFO:tasks.workunit.client.1.smithi192.stdout:1/326: mknod dc/d16/c5d 0 2024-04-23T02:14:01.541 INFO:tasks.workunit.client.1.smithi192.stdout:2/563: getdents d8/d26/d27/d31/dae 0 2024-04-23T02:14:01.541 INFO:tasks.workunit.client.1.smithi192.stdout:5/540: dread - d2/db/d16/d1b/d3d/d40/d49/f83 zero size 2024-04-23T02:14:01.542 INFO:tasks.workunit.client.1.smithi192.stdout:2/564: fdatasync d8/d26/d27/d31/f8b 0 2024-04-23T02:14:01.542 INFO:tasks.workunit.client.1.smithi192.stdout:9/492: getdents d0/d9/d44/d51/d5b/d96/da0 0 2024-04-23T02:14:01.543 INFO:tasks.workunit.client.1.smithi192.stdout:1/327: write dc/f39 [3953518,7886] 0 2024-04-23T02:14:01.544 INFO:tasks.workunit.client.1.smithi192.stdout:8/691: symlink d1/d12/d23/d2c/d4d/lf6 0 2024-04-23T02:14:01.546 INFO:tasks.workunit.client.1.smithi192.stdout:3/461: symlink d1/l9d 0 2024-04-23T02:14:01.550 INFO:tasks.workunit.client.1.smithi192.stdout:5/541: unlink d2/d6/d14/l42 0 2024-04-23T02:14:01.551 INFO:tasks.workunit.client.1.smithi192.stdout:9/493: rmdir d0/d9/d44/d4f 39 2024-04-23T02:14:01.552 INFO:tasks.workunit.client.0.smithi071.stdout:2/431: write d0/d2/f29 [327949,114540] 0 2024-04-23T02:14:01.553 INFO:tasks.workunit.client.1.smithi192.stdout:0/661: dwrite d6/f6d [0,4194304] 0 2024-04-23T02:14:01.554 INFO:tasks.workunit.client.0.smithi071.stdout:4/508: dread d2/d2a/d2d/d39/d42/d49/d67/f1a [0,4194304] 0 2024-04-23T02:14:01.554 INFO:tasks.workunit.client.0.smithi071.stdout:4/509: fsync d2/f89 0 2024-04-23T02:14:01.555 INFO:tasks.workunit.client.0.smithi071.stdout:2/432: creat d0/d70/f86 x:0 0 0 2024-04-23T02:14:01.556 INFO:tasks.workunit.client.1.smithi192.stdout:1/328: fsync dc/d14/f27 0 2024-04-23T02:14:01.562 INFO:tasks.workunit.client.1.smithi192.stdout:2/565: fsync d8/d26/d29/d43/d50/fa9 0 2024-04-23T02:14:01.562 INFO:tasks.workunit.client.1.smithi192.stdout:3/462: dread d1/d3/d17/d2b/d44/d57/f78 [0,4194304] 0 2024-04-23T02:14:01.564 INFO:tasks.workunit.client.1.smithi192.stdout:4/660: getdents de/d15/d2a/d74/d92/da0/dca/d49/d70/dd8/d89 0 2024-04-23T02:14:01.565 INFO:tasks.workunit.client.1.smithi192.stdout:9/494: creat d0/d9/d44/d51/d64/fac x:0 0 0 2024-04-23T02:14:01.566 INFO:tasks.workunit.client.1.smithi192.stdout:0/662: mknod d6/df/d47/d55/cdb 0 2024-04-23T02:14:01.567 INFO:tasks.workunit.client.0.smithi071.stdout:2/433: dread d0/d2/d45/f52 [0,4194304] 0 2024-04-23T02:14:01.567 INFO:tasks.workunit.client.1.smithi192.stdout:5/542: dwrite d2/d6/d2d/d41/f97 [0,4194304] 0 2024-04-23T02:14:01.569 INFO:tasks.workunit.client.0.smithi071.stdout:4/510: dwrite d2/d2a/d2d/d39/f46 [0,4194304] 0 2024-04-23T02:14:01.569 INFO:tasks.workunit.client.0.smithi071.stdout:4/511: stat d2/l11 0 2024-04-23T02:14:01.570 INFO:tasks.workunit.client.0.smithi071.stdout:4/512: truncate d2/d53/f88 781471 0 2024-04-23T02:14:01.571 INFO:tasks.workunit.client.1.smithi192.stdout:1/329: mkdir dc/d14/d23/d5e 0 2024-04-23T02:14:01.572 INFO:tasks.workunit.client.1.smithi192.stdout:4/661: read de/d15/d2a/d74/d92/da0/dca/d49/d26/fa8 [3283537,6876] 0 2024-04-23T02:14:01.574 INFO:tasks.workunit.client.1.smithi192.stdout:4/662: fsync de/d15/d53/f58 0 2024-04-23T02:14:01.574 INFO:tasks.workunit.client.1.smithi192.stdout:2/566: rmdir d8 39 2024-04-23T02:14:01.576 INFO:tasks.workunit.client.1.smithi192.stdout:8/692: write d1/d12/f24 [1187447,71377] 0 2024-04-23T02:14:01.577 INFO:tasks.workunit.client.1.smithi192.stdout:3/463: symlink d1/d3/d17/d65/l9e 0 2024-04-23T02:14:01.581 INFO:tasks.workunit.client.1.smithi192.stdout:1/330: symlink dc/d16/d19/d30/d32/d44/l5f 0 2024-04-23T02:14:01.584 INFO:tasks.workunit.client.1.smithi192.stdout:4/663: mknod de/d15/d19/d1a/d95/cde 0 2024-04-23T02:14:01.584 INFO:tasks.workunit.client.1.smithi192.stdout:3/464: dread d1/d3/d17/d1a/f4f [0,4194304] 0 2024-04-23T02:14:01.587 INFO:tasks.workunit.client.1.smithi192.stdout:5/543: dread d2/db/d2b/f7f [0,4194304] 0 2024-04-23T02:14:01.589 INFO:tasks.workunit.client.0.smithi071.stdout:2/434: dread d0/f1e [0,4194304] 0 2024-04-23T02:14:01.590 INFO:tasks.workunit.client.1.smithi192.stdout:9/495: symlink d0/d9/d44/d74/d3c/d9e/lad 0 2024-04-23T02:14:01.593 INFO:tasks.workunit.client.1.smithi192.stdout:5/544: mkdir d2/db/d16/d48/db7 0 2024-04-23T02:14:01.594 INFO:tasks.workunit.client.1.smithi192.stdout:1/331: dwrite f1 [4194304,4194304] 0 2024-04-23T02:14:01.594 INFO:tasks.workunit.client.1.smithi192.stdout:0/663: creat d6/df/d13/fdc x:0 0 0 2024-04-23T02:14:01.594 INFO:tasks.workunit.client.1.smithi192.stdout:3/465: write d1/d3/d17/d1f/d33/d24/d34/f7f [3680148,14464] 0 2024-04-23T02:14:01.596 INFO:tasks.workunit.client.0.smithi071.stdout:2/435: dwrite d0/d2/d34/f84 [0,4194304] 0 2024-04-23T02:14:01.598 INFO:tasks.workunit.client.1.smithi192.stdout:3/466: readlink d1/d3/l6f 0 2024-04-23T02:14:01.599 INFO:tasks.workunit.client.1.smithi192.stdout:5/545: mkdir d2/db8 0 2024-04-23T02:14:01.600 INFO:tasks.workunit.client.0.smithi071.stdout:2/436: creat d0/d82/f87 x:0 0 0 2024-04-23T02:14:01.601 INFO:tasks.workunit.client.1.smithi192.stdout:4/664: rename de/d15/d2a/d74/d92/da0/dca/d34/l3c to de/d15/d19/d1a/ldf 0 2024-04-23T02:14:01.602 INFO:tasks.workunit.client.1.smithi192.stdout:5/546: write d2/db/d16/d1b/d3d/d40/f8c [2843045,40358] 0 2024-04-23T02:14:01.605 INFO:tasks.workunit.client.0.smithi071.stdout:4/513: dread d2/d2a/f36 [0,4194304] 0 2024-04-23T02:14:01.608 INFO:tasks.workunit.client.0.smithi071.stdout:4/514: write d2/d2a/d2d/d39/d42/d49/d67/d1d/f61 [4865781,107693] 0 2024-04-23T02:14:01.609 INFO:tasks.workunit.client.0.smithi071.stdout:2/437: link d0/d2/d26/d14/d1f/c33 d0/d2/d26/d3a/c88 0 2024-04-23T02:14:01.610 INFO:tasks.workunit.client.0.smithi071.stdout:4/515: rmdir d2/d2a/d73 39 2024-04-23T02:14:01.612 INFO:tasks.workunit.client.1.smithi192.stdout:4/665: chown de/d15/d2a/d74/d92/da0/dca/d49/d26/d59 24934210 1 2024-04-23T02:14:01.613 INFO:tasks.workunit.client.0.smithi071.stdout:4/516: fsync d2/d2a/d2d/d39/d42/d49/d67/d1d/f35 0 2024-04-23T02:14:01.614 INFO:tasks.workunit.client.1.smithi192.stdout:3/467: symlink d1/d3/d17/d1f/d33/d24/d34/d6c/l9f 0 2024-04-23T02:14:01.614 INFO:tasks.workunit.client.0.smithi071.stdout:4/517: write d2/d53/f88 [1795255,129728] 0 2024-04-23T02:14:01.615 INFO:tasks.workunit.client.1.smithi192.stdout:0/664: link d6/df/d13/l43 d6/d4b/ldd 0 2024-04-23T02:14:01.617 INFO:tasks.workunit.client.1.smithi192.stdout:2/567: dread d8/d26/d29/f58 [0,4194304] 0 2024-04-23T02:14:01.617 INFO:tasks.workunit.client.1.smithi192.stdout:5/547: creat d2/d6/d2d/fb9 x:0 0 0 2024-04-23T02:14:01.618 INFO:tasks.workunit.client.0.smithi071.stdout:4/518: truncate d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f5b 5077804 0 2024-04-23T02:14:01.618 INFO:tasks.workunit.client.1.smithi192.stdout:0/665: symlink d6/df/d56/d85/da4/lde 0 2024-04-23T02:14:01.620 INFO:tasks.workunit.client.1.smithi192.stdout:4/666: read de/f18 [542683,99721] 0 2024-04-23T02:14:01.620 INFO:tasks.workunit.client.1.smithi192.stdout:5/548: dread - d2/d6/d2d/d41/d43/f9d zero size 2024-04-23T02:14:01.620 INFO:tasks.workunit.client.0.smithi071.stdout:4/519: mkdir d2/d2a/d2d/d39/d42/d49/d67/d3e/da9 0 2024-04-23T02:14:01.621 INFO:tasks.workunit.client.1.smithi192.stdout:3/468: creat d1/d3/d17/d1f/d33/d24/d34/d6c/d84/d5d/fa0 x:0 0 0 2024-04-23T02:14:01.621 INFO:tasks.workunit.client.0.smithi071.stdout:4/520: fsync d2/d2a/f36 0 2024-04-23T02:14:01.622 INFO:tasks.workunit.client.1.smithi192.stdout:2/568: symlink d8/d26/lb2 0 2024-04-23T02:14:01.623 INFO:tasks.workunit.client.0.smithi071.stdout:4/521: write d2/d2a/d2d/d39/d42/d49/d67/d1d/d57/d9c/f9d [429465,78078] 0 2024-04-23T02:14:01.623 INFO:tasks.workunit.client.1.smithi192.stdout:0/666: mknod d6/df/d35/cdf 0 2024-04-23T02:14:01.624 INFO:tasks.workunit.client.1.smithi192.stdout:7/540: dread d5/f15 [4194304,4194304] 0 2024-04-23T02:14:01.625 INFO:tasks.workunit.client.1.smithi192.stdout:2/569: chown d8/d26/d27/d53 2078641351 1 2024-04-23T02:14:01.627 INFO:tasks.workunit.client.1.smithi192.stdout:5/549: symlink d2/d32/lba 0 2024-04-23T02:14:01.627 INFO:tasks.workunit.client.1.smithi192.stdout:7/541: fsync d5/d43/d4c/d5c/d67/f82 0 2024-04-23T02:14:01.627 INFO:tasks.workunit.client.0.smithi071.stdout:4/522: rename d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c/d99 to d2/d2a/daa 0 2024-04-23T02:14:01.628 INFO:tasks.workunit.client.1.smithi192.stdout:0/667: fdatasync d6/df/d13/d25/f3d 0 2024-04-23T02:14:01.629 INFO:tasks.workunit.client.1.smithi192.stdout:4/667: creat de/d15/d19/fe0 x:0 0 0 2024-04-23T02:14:01.629 INFO:tasks.workunit.client.1.smithi192.stdout:7/542: chown d5/d43/d4c/d5c/d67/l6b 4130314 1 2024-04-23T02:14:01.629 INFO:tasks.workunit.client.0.smithi071.stdout:4/523: mkdir d2/d2a/d2d/d39/d42/d49/d67/d1d/d57/d9c/dab 0 2024-04-23T02:14:01.630 INFO:tasks.workunit.client.0.smithi071.stdout:4/524: mknod d2/d2a/da2/cac 0 2024-04-23T02:14:01.631 INFO:tasks.workunit.client.0.smithi071.stdout:4/525: readlink d2/d2a/d2d/l6a 0 2024-04-23T02:14:01.632 INFO:tasks.workunit.client.1.smithi192.stdout:2/570: rmdir d8/d26/d27/d31 39 2024-04-23T02:14:01.632 INFO:tasks.workunit.client.0.smithi071.stdout:4/526: readlink d2/d2a/d2d/d39/d42/d49/d67/d1d/d57/d9c/l90 0 2024-04-23T02:14:01.632 INFO:tasks.workunit.client.0.smithi071.stdout:3/450: sync 2024-04-23T02:14:01.632 INFO:tasks.workunit.client.0.smithi071.stdout:0/454: sync 2024-04-23T02:14:01.633 INFO:tasks.workunit.client.0.smithi071.stdout:9/482: sync 2024-04-23T02:14:01.634 INFO:tasks.workunit.client.0.smithi071.stdout:0/455: stat d0/d3/de/d48/c75 0 2024-04-23T02:14:01.634 INFO:tasks.workunit.client.1.smithi192.stdout:8/693: dwrite d1/d12/d23/d4b/d73/f65 [4194304,4194304] 0 2024-04-23T02:14:01.635 INFO:tasks.workunit.client.0.smithi071.stdout:2/438: dread d0/d2/d26/f4c [0,4194304] 0 2024-04-23T02:14:01.635 INFO:tasks.workunit.client.0.smithi071.stdout:9/483: mkdir d1/d4e/d66/da4 0 2024-04-23T02:14:01.636 INFO:tasks.workunit.client.0.smithi071.stdout:0/456: fsync d0/d3/de/d10/d14/d32/d37/f7b 0 2024-04-23T02:14:01.636 INFO:tasks.workunit.client.1.smithi192.stdout:4/668: rmdir de/d15/d2a/d74/d92/da0/dca 39 2024-04-23T02:14:01.637 INFO:tasks.workunit.client.0.smithi071.stdout:3/451: creat d7/d22/d23/d26/d75/f9b x:0 0 0 2024-04-23T02:14:01.638 INFO:tasks.workunit.client.0.smithi071.stdout:3/452: chown d7/d22/d23 351704 1 2024-04-23T02:14:01.639 INFO:tasks.workunit.client.1.smithi192.stdout:0/668: rename d6/df/d13/c7c to d6/df/d56/d85/d71/ce0 0 2024-04-23T02:14:01.640 INFO:tasks.workunit.client.1.smithi192.stdout:5/550: mknod d2/cbb 0 2024-04-23T02:14:01.642 INFO:tasks.workunit.client.1.smithi192.stdout:2/571: symlink d8/d26/d27/d57/lb3 0 2024-04-23T02:14:01.644 INFO:tasks.workunit.client.1.smithi192.stdout:2/572: chown d8/d26/d29/da4/da8 1 1 2024-04-23T02:14:01.645 INFO:tasks.workunit.client.1.smithi192.stdout:8/694: dread d1/fe [0,4194304] 0 2024-04-23T02:14:01.646 INFO:tasks.workunit.client.1.smithi192.stdout:2/573: chown d8/d26/d29/d43/d50/f76 23020 1 2024-04-23T02:14:01.649 INFO:tasks.workunit.client.1.smithi192.stdout:7/543: mkdir d5/d17/d1c/d59/d5a/d41/d50/d9f 0 2024-04-23T02:14:01.650 INFO:tasks.workunit.client.1.smithi192.stdout:2/574: write d8/f96 [3908935,20304] 0 2024-04-23T02:14:01.651 INFO:tasks.workunit.client.1.smithi192.stdout:0/669: symlink d6/df/d13/d25/d94/le1 0 2024-04-23T02:14:01.653 INFO:tasks.workunit.client.1.smithi192.stdout:8/695: mkdir d1/d12/d7d/df7 0 2024-04-23T02:14:01.653 INFO:tasks.workunit.client.1.smithi192.stdout:4/669: stat de/d15/d2a/d74/d92/da0/dca/d34/d7c/da1 0 2024-04-23T02:14:01.655 INFO:tasks.workunit.client.1.smithi192.stdout:8/696: write d1/d12/d23/d4b/d73/d4c/d82/fb6 [2101465,30025] 0 2024-04-23T02:14:01.660 INFO:tasks.workunit.client.1.smithi192.stdout:3/469: write d1/d3/d17/d2b/f51 [409703,31643] 0 2024-04-23T02:14:01.660 INFO:tasks.workunit.client.1.smithi192.stdout:0/670: write d6/df/d13/d25/d26/f27 [8849519,104854] 0 2024-04-23T02:14:01.665 INFO:tasks.workunit.client.1.smithi192.stdout:8/697: mknod d1/d12/d68/cf8 0 2024-04-23T02:14:01.668 INFO:tasks.workunit.client.1.smithi192.stdout:7/544: mkdir d5/d17/d1c/d59/d5a/da0 0 2024-04-23T02:14:01.669 INFO:tasks.workunit.client.1.smithi192.stdout:8/698: truncate d1/d9/d8f/d90/f93 783830 0 2024-04-23T02:14:01.669 INFO:tasks.workunit.client.1.smithi192.stdout:3/470: symlink d1/la1 0 2024-04-23T02:14:01.671 INFO:tasks.workunit.client.1.smithi192.stdout:4/670: symlink de/d15/d2a/d74/d92/da0/dca/d49/d93/le1 0 2024-04-23T02:14:01.674 INFO:tasks.workunit.client.0.smithi071.stdout:4/527: rmdir d2/d2a/daa 39 2024-04-23T02:14:01.676 INFO:tasks.workunit.client.0.smithi071.stdout:2/439: creat d0/d2/d45/d7d/d56/f89 x:0 0 0 2024-04-23T02:14:01.676 INFO:tasks.workunit.client.1.smithi192.stdout:2/575: creat d8/d26/d27/d31/dae/fb4 x:0 0 0 2024-04-23T02:14:01.677 INFO:tasks.workunit.client.1.smithi192.stdout:7/545: fsync d5/d17/d1c/d59/d5a/d41/d50/f7b 0 2024-04-23T02:14:01.678 INFO:tasks.workunit.client.1.smithi192.stdout:5/551: truncate d2/db/d16/d1b/f29 2387947 0 2024-04-23T02:14:01.680 INFO:tasks.workunit.client.0.smithi071.stdout:0/457: symlink d0/d3/de/d10/d14/d32/d44/d31/l8c 0 2024-04-23T02:14:01.681 INFO:tasks.workunit.client.1.smithi192.stdout:8/699: dwrite d1/d9/d3e/f55 [0,4194304] 0 2024-04-23T02:14:01.681 INFO:tasks.workunit.client.1.smithi192.stdout:4/671: truncate de/d15/d53/d81/fa6 237157 0 2024-04-23T02:14:01.684 INFO:tasks.workunit.client.1.smithi192.stdout:3/471: truncate d1/d3/d17/d1f/d33/d24/d34/d6c/d84/d5d/f9c 91011 0 2024-04-23T02:14:01.684 INFO:tasks.workunit.client.1.smithi192.stdout:4/672: dread - de/d15/d19/fe0 zero size 2024-04-23T02:14:01.685 INFO:tasks.workunit.client.1.smithi192.stdout:2/576: mkdir d8/d26/d27/db5 0 2024-04-23T02:14:01.685 INFO:tasks.workunit.client.1.smithi192.stdout:3/472: write d1/d3/d17/d1a/d7c/f91 [1827386,32359] 0 2024-04-23T02:14:01.686 INFO:tasks.workunit.client.0.smithi071.stdout:3/453: dwrite d7/fd [0,4194304] 0 2024-04-23T02:14:01.691 INFO:tasks.workunit.client.1.smithi192.stdout:2/577: readlink d8/d14/l21 0 2024-04-23T02:14:01.692 INFO:tasks.workunit.client.0.smithi071.stdout:2/440: rename d0/l1b to d0/d2/d45/d58/l8a 0 2024-04-23T02:14:01.692 INFO:tasks.workunit.client.0.smithi071.stdout:0/458: dwrite d0/d3/de/d10/d14/d32/f35 [0,4194304] 0 2024-04-23T02:14:01.696 INFO:tasks.workunit.client.0.smithi071.stdout:4/528: rmdir d2/d2a/d73/d5e 39 2024-04-23T02:14:01.699 INFO:tasks.workunit.client.0.smithi071.stdout:3/454: fdatasync f0 0 2024-04-23T02:14:01.699 INFO:tasks.workunit.client.1.smithi192.stdout:0/671: write d6/df/d35/d49/fc9 [843419,13519] 0 2024-04-23T02:14:01.699 INFO:tasks.workunit.client.1.smithi192.stdout:8/700: symlink d1/d9/d8f/d90/lf9 0 2024-04-23T02:14:01.701 INFO:tasks.workunit.client.0.smithi071.stdout:0/459: symlink d0/d3/de/d10/d14/d32/d44/d31/l8d 0 2024-04-23T02:14:01.704 INFO:tasks.workunit.client.0.smithi071.stdout:3/455: mknod d7/d8/d42/d5f/c9c 0 2024-04-23T02:14:01.705 INFO:tasks.workunit.client.1.smithi192.stdout:3/473: creat d1/d3/d17/d1f/d33/d24/d34/d6c/fa2 x:0 0 0 2024-04-23T02:14:01.705 INFO:tasks.workunit.client.0.smithi071.stdout:9/484: creat d1/d1a/d39/fa5 x:0 0 0 2024-04-23T02:14:01.706 INFO:tasks.workunit.client.0.smithi071.stdout:9/485: readlink d1/d32/l59 0 2024-04-23T02:14:01.706 INFO:tasks.workunit.client.1.smithi192.stdout:7/546: rename d5/d17/d1c/d4a/c4f to d5/d17/d1c/d59/ca1 0 2024-04-23T02:14:01.708 INFO:tasks.workunit.client.1.smithi192.stdout:2/578: read d8/d26/d27/f87 [481685,100443] 0 2024-04-23T02:14:01.711 INFO:tasks.workunit.client.0.smithi071.stdout:9/486: mkdir d1/d4e/d54/d75/da6 0 2024-04-23T02:14:01.711 INFO:tasks.workunit.client.1.smithi192.stdout:4/673: creat de/d15/d2a/fe2 x:0 0 0 2024-04-23T02:14:01.712 INFO:tasks.workunit.client.0.smithi071.stdout:9/487: fsync d1/d32/d3a/d94/f9c 0 2024-04-23T02:14:01.712 INFO:tasks.workunit.client.1.smithi192.stdout:5/552: dwrite d2/db/d16/d1b/f89 [0,4194304] 0 2024-04-23T02:14:01.713 INFO:tasks.workunit.client.1.smithi192.stdout:7/547: write d5/d43/d4c/d5c/d67/f82 [248235,118909] 0 2024-04-23T02:14:01.716 INFO:tasks.workunit.client.1.smithi192.stdout:2/579: dread - d8/d26/d27/d31/f8b zero size 2024-04-23T02:14:01.717 INFO:tasks.workunit.client.1.smithi192.stdout:7/548: write d5/f15 [6864409,29176] 0 2024-04-23T02:14:01.718 INFO:tasks.workunit.client.1.smithi192.stdout:5/553: dread - d2/db/d16/d1b/f79 zero size 2024-04-23T02:14:01.718 INFO:tasks.workunit.client.1.smithi192.stdout:2/580: chown d8/d26 0 1 2024-04-23T02:14:01.718 INFO:tasks.workunit.client.1.smithi192.stdout:4/674: stat de/d15/d53/l96 0 2024-04-23T02:14:01.719 INFO:tasks.workunit.client.1.smithi192.stdout:3/474: write d1/d3/d17/d1f/d33/f8d [1905486,68420] 0 2024-04-23T02:14:01.722 INFO:tasks.workunit.client.0.smithi071.stdout:9/488: dwrite d1/d32/d3a/d77/f88 [0,4194304] 0 2024-04-23T02:14:01.728 INFO:tasks.workunit.client.1.smithi192.stdout:5/554: chown d2/db/d2b/f3a 9640814 1 2024-04-23T02:14:01.729 INFO:tasks.workunit.client.1.smithi192.stdout:7/549: dread d5/d43/d4c/d5c/d67/f81 [0,4194304] 0 2024-04-23T02:14:01.730 INFO:tasks.workunit.client.0.smithi071.stdout:4/529: dread d2/f43 [0,4194304] 0 2024-04-23T02:14:01.731 INFO:tasks.workunit.client.1.smithi192.stdout:4/675: unlink de/d15/d2a/d74/d92/da0/dca/d49/f60 0 2024-04-23T02:14:01.731 INFO:tasks.workunit.client.1.smithi192.stdout:4/676: readlink de/l17 0 2024-04-23T02:14:01.732 INFO:tasks.workunit.client.0.smithi071.stdout:9/489: getdents d1/d1a/d39/d42 0 2024-04-23T02:14:01.732 INFO:tasks.workunit.client.1.smithi192.stdout:0/672: truncate d6/df/d13/d25/d26/f90 2612490 0 2024-04-23T02:14:01.732 INFO:tasks.workunit.client.0.smithi071.stdout:9/490: readlink d1/d1a/d39/l67 0 2024-04-23T02:14:01.732 INFO:tasks.workunit.client.0.smithi071.stdout:4/530: stat d2/d2a/d2d/d39/d42/d49/d67/d1d/d57/d9c/dab 0 2024-04-23T02:14:01.733 INFO:tasks.workunit.client.0.smithi071.stdout:9/491: dread - d1/d4e/d66/f73 zero size 2024-04-23T02:14:01.737 INFO:tasks.workunit.client.0.smithi071.stdout:4/531: creat d2/d2a/daa/fad x:0 0 0 2024-04-23T02:14:01.737 INFO:tasks.workunit.client.1.smithi192.stdout:4/677: fdatasync de/d15/d2a/f2c 0 2024-04-23T02:14:01.739 INFO:tasks.workunit.client.1.smithi192.stdout:2/581: dwrite d8/d26/d29/d43/d50/d5e/f5f [0,4194304] 0 2024-04-23T02:14:01.740 INFO:tasks.workunit.client.1.smithi192.stdout:3/475: dread d1/d3/d17/d1f/d33/d24/d34/d6c/d84/d46/f85 [0,4194304] 0 2024-04-23T02:14:01.741 INFO:tasks.workunit.client.0.smithi071.stdout:4/532: unlink d2/f5a 0 2024-04-23T02:14:01.742 INFO:tasks.workunit.client.1.smithi192.stdout:7/550: write d5/d43/d4c/d5c/d67/f82 [1432962,115640] 0 2024-04-23T02:14:01.743 INFO:tasks.workunit.client.1.smithi192.stdout:5/555: fsync d2/db/d16/d1b/d3d/f65 0 2024-04-23T02:14:01.743 INFO:tasks.workunit.client.0.smithi071.stdout:4/533: truncate d2/d2a/daa/fad 915285 0 2024-04-23T02:14:01.750 INFO:tasks.workunit.client.1.smithi192.stdout:2/582: creat d8/d26/d29/d43/d5d/fb6 x:0 0 0 2024-04-23T02:14:01.753 INFO:tasks.workunit.client.0.smithi071.stdout:2/441: creat d0/d2/d26/d14/d1f/d4f/f8b x:0 0 0 2024-04-23T02:14:01.754 INFO:tasks.workunit.client.1.smithi192.stdout:4/678: rmdir de/d15/d2a/d74/d92/da0/dca/d49/d70/dd8/d89 39 2024-04-23T02:14:01.754 INFO:tasks.workunit.client.0.smithi071.stdout:2/442: write d0/d2/d45/d7d/d56/f89 [679097,39154] 0 2024-04-23T02:14:01.755 INFO:tasks.workunit.client.1.smithi192.stdout:4/679: readlink de/l6b 0 2024-04-23T02:14:01.756 INFO:tasks.workunit.client.1.smithi192.stdout:2/583: rename d8/l2b to d8/d26/d74/d7c/lb7 0 2024-04-23T02:14:01.757 INFO:tasks.workunit.client.1.smithi192.stdout:4/680: readlink de/d15/d53/l9c 0 2024-04-23T02:14:01.757 INFO:tasks.workunit.client.1.smithi192.stdout:5/556: truncate d2/db/d16/d48/d4f/f84 550623 0 2024-04-23T02:14:01.758 INFO:tasks.workunit.client.0.smithi071.stdout:5/576: sync 2024-04-23T02:14:01.759 INFO:tasks.workunit.client.0.smithi071.stdout:8/505: sync 2024-04-23T02:14:01.760 INFO:tasks.workunit.client.0.smithi071.stdout:0/460: rename d0/d3/de/l78 to d0/d3/de/d10/d14/l8e 0 2024-04-23T02:14:01.760 INFO:tasks.workunit.client.1.smithi192.stdout:5/557: read - d2/d6/d2d/fac zero size 2024-04-23T02:14:01.762 INFO:tasks.workunit.client.1.smithi192.stdout:3/476: dwrite d1/d3/d17/d1f/d33/d24/f45 [0,4194304] 0 2024-04-23T02:14:01.764 INFO:tasks.workunit.client.1.smithi192.stdout:2/584: creat d8/d26/d81/fb8 x:0 0 0 2024-04-23T02:14:01.764 INFO:tasks.workunit.client.0.smithi071.stdout:5/577: creat d3/d1d/d43/d4b/fe0 x:0 0 0 2024-04-23T02:14:01.765 INFO:tasks.workunit.client.0.smithi071.stdout:5/578: chown d3/da/d1a/d1c 212 1 2024-04-23T02:14:01.766 INFO:tasks.workunit.client.1.smithi192.stdout:2/585: dread - d8/d26/d27/d31/f8b zero size 2024-04-23T02:14:01.766 INFO:tasks.workunit.client.1.smithi192.stdout:0/673: dread d6/df/f32 [0,4194304] 0 2024-04-23T02:14:01.767 INFO:tasks.workunit.client.0.smithi071.stdout:4/534: rename d2/d2a/d2d/d39/d42/d49/d67/d1d/d57/d9c to d2/d2a/d2d/d39/d42/d49/d67/d7e/dae 0 2024-04-23T02:14:01.767 INFO:tasks.workunit.client.1.smithi192.stdout:7/551: creat d5/d17/d1c/d59/d5a/fa2 x:0 0 0 2024-04-23T02:14:01.767 INFO:tasks.workunit.client.0.smithi071.stdout:4/535: stat d2/l11 0 2024-04-23T02:14:01.770 INFO:tasks.workunit.client.1.smithi192.stdout:5/558: mkdir d2/d6/d2d/d41/d43/dbc 0 2024-04-23T02:14:01.770 INFO:tasks.workunit.client.0.smithi071.stdout:5/579: write d3/d1d/f20 [196626,34351] 0 2024-04-23T02:14:01.772 INFO:tasks.workunit.client.0.smithi071.stdout:0/461: dwrite d0/d3/f3f [0,4194304] 0 2024-04-23T02:14:01.772 INFO:tasks.workunit.client.1.smithi192.stdout:7/552: read d5/d17/d1c/d4a/f76 [42293,34520] 0 2024-04-23T02:14:01.774 INFO:tasks.workunit.client.0.smithi071.stdout:3/456: unlink d7/d22/d23/f24 0 2024-04-23T02:14:01.774 INFO:tasks.workunit.client.1.smithi192.stdout:7/553: fdatasync d5/d17/d1c/d59/d5a/d41/d50/f62 0 2024-04-23T02:14:01.777 INFO:tasks.workunit.client.0.smithi071.stdout:5/580: creat d3/da/d1a/d6d/fe1 x:0 0 0 2024-04-23T02:14:01.778 INFO:tasks.workunit.client.1.smithi192.stdout:3/477: fdatasync d1/d3/d17/d1f/d33/d24/d34/d38/d7b/f96 0 2024-04-23T02:14:01.781 INFO:tasks.workunit.client.0.smithi071.stdout:4/536: write d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f5d [1501880,10050] 0 2024-04-23T02:14:01.781 INFO:tasks.workunit.client.0.smithi071.stdout:0/462: write d0/d3/de/d10/d14/f5b [1556456,103063] 0 2024-04-23T02:14:01.781 INFO:tasks.workunit.client.1.smithi192.stdout:3/478: write d1/d3/d17/d1f/d33/d24/d34/f54 [1485257,129406] 0 2024-04-23T02:14:01.781 INFO:tasks.workunit.client.1.smithi192.stdout:5/559: creat d2/d6/d2d/d41/d43/fbd x:0 0 0 2024-04-23T02:14:01.782 INFO:tasks.workunit.client.1.smithi192.stdout:7/554: rename d5/d43/l4d to d5/d17/d1c/d59/d5a/d41/d50/d3d/la3 0 2024-04-23T02:14:01.783 INFO:tasks.workunit.client.1.smithi192.stdout:4/681: getdents de/d15/d2a/d74/d92/da0/dca/d34/d7c 0 2024-04-23T02:14:01.783 INFO:tasks.workunit.client.0.smithi071.stdout:3/457: symlink d7/d22/d23/d26/d75/l9d 0 2024-04-23T02:14:01.784 INFO:tasks.workunit.client.1.smithi192.stdout:4/682: chown de/d15/dbb 25 1 2024-04-23T02:14:01.784 INFO:tasks.workunit.client.1.smithi192.stdout:5/560: dread - d2/d6/d2d/d41/fa1 zero size 2024-04-23T02:14:01.788 INFO:tasks.workunit.client.0.smithi071.stdout:0/463: mknod d0/d3/de/d48/d7e/c8f 0 2024-04-23T02:14:01.788 INFO:tasks.workunit.client.0.smithi071.stdout:9/492: dwrite d1/d1a/f31 [0,4194304] 0 2024-04-23T02:14:01.791 INFO:tasks.workunit.client.1.smithi192.stdout:4/683: fdatasync de/d15/d2a/d74/d92/da0/dca/d49/d26/d59/f63 0 2024-04-23T02:14:01.792 INFO:tasks.workunit.client.0.smithi071.stdout:2/443: dwrite d0/d2/d45/d7d/f53 [0,4194304] 0 2024-04-23T02:14:01.792 INFO:tasks.workunit.client.1.smithi192.stdout:5/561: mknod d2/d6/cbe 0 2024-04-23T02:14:01.792 INFO:tasks.workunit.client.1.smithi192.stdout:0/674: dread d6/df/d35/d8b/f9b [0,4194304] 0 2024-04-23T02:14:01.792 INFO:tasks.workunit.client.1.smithi192.stdout:3/479: dread d1/d3/d17/d1a/f4f [0,4194304] 0 2024-04-23T02:14:01.795 INFO:tasks.workunit.client.0.smithi071.stdout:2/444: rename d0/d2/d26/df to d0/d2/d26/df/d30/d8c 22 2024-04-23T02:14:01.796 INFO:tasks.workunit.client.0.smithi071.stdout:8/506: dread dc/d15/f39 [0,4194304] 0 2024-04-23T02:14:01.797 INFO:tasks.workunit.client.0.smithi071.stdout:9/493: mknod d1/d4e/d66/ca7 0 2024-04-23T02:14:01.799 INFO:tasks.workunit.client.1.smithi192.stdout:4/684: chown de/d15/d19/d1a/d95 432 1 2024-04-23T02:14:01.799 INFO:tasks.workunit.client.1.smithi192.stdout:7/555: dwrite d5/d17/d1c/d59/d5a/d70/f7e [0,4194304] 0 2024-04-23T02:14:01.800 INFO:tasks.workunit.client.0.smithi071.stdout:0/464: symlink d0/d3/de/d10/d14/d84/l90 0 2024-04-23T02:14:01.803 INFO:tasks.workunit.client.0.smithi071.stdout:8/507: mknod dc/d15/d5d/ca2 0 2024-04-23T02:14:01.804 INFO:tasks.workunit.client.0.smithi071.stdout:8/508: write dc/f54 [122084,36371] 0 2024-04-23T02:14:01.807 INFO:tasks.workunit.client.0.smithi071.stdout:4/537: link d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f59 d2/d2a/d2d/d39/d42/d49/d67/d1d/faf 0 2024-04-23T02:14:01.807 INFO:tasks.workunit.client.1.smithi192.stdout:3/480: mknod d1/d3/d17/d65/ca3 0 2024-04-23T02:14:01.808 INFO:tasks.workunit.client.1.smithi192.stdout:3/481: write d1/d3/d17/f27 [397855,110904] 0 2024-04-23T02:14:01.808 INFO:tasks.workunit.client.1.smithi192.stdout:2/586: dread d8/d26/f61 [0,4194304] 0 2024-04-23T02:14:01.808 INFO:tasks.workunit.client.1.smithi192.stdout:0/675: symlink d6/df/d13/d25/le2 0 2024-04-23T02:14:01.808 INFO:tasks.workunit.client.1.smithi192.stdout:5/562: rename d2/d6/c28 to d2/d6/db4/cbf 0 2024-04-23T02:14:01.808 INFO:tasks.workunit.client.1.smithi192.stdout:3/482: dread - d1/d3/d17/d1f/d33/d24/d34/d6c/d84/f63 zero size 2024-04-23T02:14:01.810 INFO:tasks.workunit.client.0.smithi071.stdout:8/509: rename dc/d15/d5d/ca2 to dc/d15/d72/d95/ca3 0 2024-04-23T02:14:01.811 INFO:tasks.workunit.client.0.smithi071.stdout:4/538: stat d2/d2a/d2d/d39/d42/d49/f77 0 2024-04-23T02:14:01.811 INFO:tasks.workunit.client.1.smithi192.stdout:3/483: write d1/d3/d17/d1f/d33/d24/d34/d6c/d84/d5d/fa0 [4969,40723] 0 2024-04-23T02:14:01.812 INFO:tasks.workunit.client.0.smithi071.stdout:5/581: dread d3/d1d/d43/f69 [0,4194304] 0 2024-04-23T02:14:01.816 INFO:tasks.workunit.client.0.smithi071.stdout:8/510: mkdir dc/d3a/da4 0 2024-04-23T02:14:01.817 INFO:tasks.workunit.client.1.smithi192.stdout:0/676: dwrite d6/df/d47/f61 [0,4194304] 0 2024-04-23T02:14:01.818 INFO:tasks.workunit.client.0.smithi071.stdout:7/457: sync 2024-04-23T02:14:01.818 INFO:tasks.workunit.client.0.smithi071.stdout:1/471: sync 2024-04-23T02:14:01.818 INFO:tasks.workunit.client.0.smithi071.stdout:6/473: sync 2024-04-23T02:14:01.818 INFO:tasks.workunit.client.0.smithi071.stdout:3/458: dwrite d7/d22/d23/d26/f2b [0,4194304] 0 2024-04-23T02:14:01.818 INFO:tasks.workunit.client.0.smithi071.stdout:5/582: write d3/d1d/d43/d4b/fe0 [267868,18384] 0 2024-04-23T02:14:01.821 INFO:tasks.workunit.client.0.smithi071.stdout:4/539: creat d2/d2a/d73/d5e/fb0 x:0 0 0 2024-04-23T02:14:01.821 INFO:tasks.workunit.client.0.smithi071.stdout:8/511: write dc/d15/da0/d30/f66 [975642,75880] 0 2024-04-23T02:14:01.824 INFO:tasks.workunit.client.0.smithi071.stdout:7/458: truncate d3/f69 1850406 0 2024-04-23T02:14:01.825 INFO:tasks.workunit.client.0.smithi071.stdout:3/459: chown d7/d8/d42/d5f/d68/d3b/d4c/f63 14 1 2024-04-23T02:14:01.826 INFO:tasks.workunit.client.1.smithi192.stdout:6/624: sync 2024-04-23T02:14:01.827 INFO:tasks.workunit.client.0.smithi071.stdout:1/472: mkdir db/df/d13/d20/d75/d96 0 2024-04-23T02:14:01.827 INFO:tasks.workunit.client.0.smithi071.stdout:4/540: write d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/ff [642272,112965] 0 2024-04-23T02:14:01.828 INFO:tasks.workunit.client.0.smithi071.stdout:6/474: creat d6/d17/d77/d85/f9b x:0 0 0 2024-04-23T02:14:01.828 INFO:tasks.workunit.client.1.smithi192.stdout:4/685: rename de/d15/d2a to de/d15/d53/d81/de3 0 2024-04-23T02:14:01.829 INFO:tasks.workunit.client.0.smithi071.stdout:2/445: read d0/d2/d26/f18 [45673,76053] 0 2024-04-23T02:14:01.830 INFO:tasks.workunit.client.0.smithi071.stdout:4/541: fsync d2/d2a/d2d/d39/f46 0 2024-04-23T02:14:01.831 INFO:tasks.workunit.client.0.smithi071.stdout:3/460: unlink d7/d8/d42/d5f/l58 0 2024-04-23T02:14:01.831 INFO:tasks.workunit.client.1.smithi192.stdout:2/587: creat d8/d26/d27/d53/da2/fb9 x:0 0 0 2024-04-23T02:14:01.832 INFO:tasks.workunit.client.1.smithi192.stdout:3/484: dwrite d1/d3/d17/d1f/d33/d24/f45 [0,4194304] 0 2024-04-23T02:14:01.833 INFO:tasks.workunit.client.0.smithi071.stdout:5/583: creat d3/da/d1a/fe2 x:0 0 0 2024-04-23T02:14:01.835 INFO:tasks.workunit.client.0.smithi071.stdout:3/461: read - d7/d8/d42/f7c zero size 2024-04-23T02:14:01.835 INFO:tasks.workunit.client.0.smithi071.stdout:5/584: fsync d3/da/fcf 0 2024-04-23T02:14:01.836 INFO:tasks.workunit.client.1.smithi192.stdout:4/686: mkdir de/d15/d53/d81/de3/de4 0 2024-04-23T02:14:01.840 INFO:tasks.workunit.client.0.smithi071.stdout:5/585: fsync d3/d1d/f20 0 2024-04-23T02:14:01.841 INFO:tasks.workunit.client.1.smithi192.stdout:3/485: getdents d1 0 2024-04-23T02:14:01.842 INFO:tasks.workunit.client.1.smithi192.stdout:6/625: creat d3/d11/fce x:0 0 0 2024-04-23T02:14:01.843 INFO:tasks.workunit.client.0.smithi071.stdout:1/473: dwrite db/d34/d68/f87 [0,4194304] 0 2024-04-23T02:14:01.843 INFO:tasks.workunit.client.0.smithi071.stdout:9/494: truncate d1/d32/d3a/d77/f88 558920 0 2024-04-23T02:14:01.843 INFO:tasks.workunit.client.0.smithi071.stdout:4/542: truncate d2/d2a/d2d/d39/d42/d49/d67/d1d/f61 5444241 0 2024-04-23T02:14:01.844 INFO:tasks.workunit.client.0.smithi071.stdout:7/459: link d3/d17/d41/d34/f35 d3/d17/d41/fa0 0 2024-04-23T02:14:01.846 INFO:tasks.workunit.client.1.smithi192.stdout:2/588: dwrite d8/d26/d27/d31/f8b [0,4194304] 0 2024-04-23T02:14:01.848 INFO:tasks.workunit.client.1.smithi192.stdout:6/626: rmdir d3/d15/d1b/d21/d3a/dbd 39 2024-04-23T02:14:01.848 INFO:tasks.workunit.client.1.smithi192.stdout:3/486: creat d1/d3/d17/d65/fa4 x:0 0 0 2024-04-23T02:14:01.849 INFO:tasks.workunit.client.0.smithi071.stdout:3/462: mknod d7/c9e 0 2024-04-23T02:14:01.850 INFO:tasks.workunit.client.1.smithi192.stdout:2/589: mkdir d8/d26/d29/da4/da8/dba 0 2024-04-23T02:14:01.851 INFO:tasks.workunit.client.0.smithi071.stdout:7/460: dread - d3/d17/d1c/d2a/d43/d86/f88 zero size 2024-04-23T02:14:01.851 INFO:tasks.workunit.client.0.smithi071.stdout:9/495: rmdir d1/d32/d3a/d94 39 2024-04-23T02:14:01.852 INFO:tasks.workunit.client.1.smithi192.stdout:4/687: read de/d15/d53/d81/fa6 [107895,59348] 0 2024-04-23T02:14:01.852 INFO:tasks.workunit.client.1.smithi192.stdout:2/590: stat d8/d26/d74 0 2024-04-23T02:14:01.852 INFO:tasks.workunit.client.1.smithi192.stdout:5/563: dread d2/db/d16/d1b/d3d/d40/d49/f70 [0,4194304] 0 2024-04-23T02:14:01.853 INFO:tasks.workunit.client.0.smithi071.stdout:4/543: creat d2/d2a/d2d/d39/d42/d49/d67/d7e/d82/fb1 x:0 0 0 2024-04-23T02:14:01.854 INFO:tasks.workunit.client.1.smithi192.stdout:3/487: fdatasync d1/d3/d17/d1f/d33/d24/d34/d38/f48 0 2024-04-23T02:14:01.854 INFO:tasks.workunit.client.1.smithi192.stdout:6/627: mkdir d3/d15/d1b/dcf 0 2024-04-23T02:14:01.855 INFO:tasks.workunit.client.0.smithi071.stdout:2/446: dread d0/f2e [0,4194304] 0 2024-04-23T02:14:01.856 INFO:tasks.workunit.client.0.smithi071.stdout:3/463: mkdir d7/d8/d42/d5f/d68/d3b/d9f 0 2024-04-23T02:14:01.856 INFO:tasks.workunit.client.0.smithi071.stdout:7/461: fdatasync d3/d17/d41/f66 0 2024-04-23T02:14:01.858 INFO:tasks.workunit.client.0.smithi071.stdout:3/464: write d7/d8/f1a [3743133,96402] 0 2024-04-23T02:14:01.858 INFO:tasks.workunit.client.1.smithi192.stdout:2/591: dread - d8/d26/d29/d43/d5d/fb6 zero size 2024-04-23T02:14:01.861 INFO:tasks.workunit.client.1.smithi192.stdout:3/488: rmdir d1/d3/d17/d1f/d33 39 2024-04-23T02:14:01.861 INFO:tasks.workunit.client.1.smithi192.stdout:6/628: creat d3/d6c/fd0 x:0 0 0 2024-04-23T02:14:01.862 INFO:tasks.workunit.client.1.smithi192.stdout:3/489: dread - d1/d3/f8c zero size 2024-04-23T02:14:01.863 INFO:tasks.workunit.client.1.smithi192.stdout:5/564: creat d2/d6/d2d/d41/d43/d98/fc0 x:0 0 0 2024-04-23T02:14:01.863 INFO:tasks.workunit.client.0.smithi071.stdout:3/465: getdents d7/d8/d42/d5f/d66 0 2024-04-23T02:14:01.864 INFO:tasks.workunit.client.0.smithi071.stdout:2/447: dread d0/f2e [0,4194304] 0 2024-04-23T02:14:01.866 INFO:tasks.workunit.client.0.smithi071.stdout:0/465: dread d0/d3/de/ff [4194304,4194304] 0 2024-04-23T02:14:01.866 INFO:tasks.workunit.client.0.smithi071.stdout:4/544: rename d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d29/f31 to d2/d2a/d2d/d39/d42/d49/d67/d1d/d57/fb2 0 2024-04-23T02:14:01.866 INFO:tasks.workunit.client.1.smithi192.stdout:6/629: rmdir d3/d15/d1b/d21/d3a/dbd 39 2024-04-23T02:14:01.866 INFO:tasks.workunit.client.1.smithi192.stdout:5/565: getdents d2/db/d46 0 2024-04-23T02:14:01.866 INFO:tasks.workunit.client.0.smithi071.stdout:2/448: chown d0/d2/d26/d14/d1f/d36/c46 21825104 1 2024-04-23T02:14:01.867 INFO:tasks.workunit.client.1.smithi192.stdout:3/490: creat d1/d3/d17/d65/d99/fa5 x:0 0 0 2024-04-23T02:14:01.868 INFO:tasks.workunit.client.0.smithi071.stdout:4/545: mkdir d2/d2a/d2d/d39/db3 0 2024-04-23T02:14:01.870 INFO:tasks.workunit.client.0.smithi071.stdout:3/466: creat d7/d22/d23/d26/fa0 x:0 0 0 2024-04-23T02:14:01.871 INFO:tasks.workunit.client.0.smithi071.stdout:2/449: mknod d0/d2/d26/d14/d1f/d36/c8d 0 2024-04-23T02:14:01.872 INFO:tasks.workunit.client.1.smithi192.stdout:3/491: creat d1/d3/d6/fa6 x:0 0 0 2024-04-23T02:14:01.873 INFO:tasks.workunit.client.0.smithi071.stdout:2/450: readlink d0/d2/d26/d14/d1f/d36/l7f 0 2024-04-23T02:14:01.875 INFO:tasks.workunit.client.0.smithi071.stdout:0/466: creat d0/d3/de/d10/d14/d32/d44/d31/d52/f91 x:0 0 0 2024-04-23T02:14:01.877 INFO:tasks.workunit.client.0.smithi071.stdout:8/512: write dc/d78/f8f [154446,127075] 0 2024-04-23T02:14:01.880 INFO:tasks.workunit.client.0.smithi071.stdout:8/513: truncate dc/d15/da0/d8d/f9e 863766 0 2024-04-23T02:14:01.880 INFO:tasks.workunit.client.0.smithi071.stdout:4/546: getdents d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d10 0 2024-04-23T02:14:01.882 INFO:tasks.workunit.client.1.smithi192.stdout:7/556: truncate d5/d17/d1c/d57/f48 1285133 0 2024-04-23T02:14:01.886 INFO:tasks.workunit.client.1.smithi192.stdout:7/557: fdatasync d5/d17/d1c/d57/f90 0 2024-04-23T02:14:01.889 INFO:tasks.workunit.client.0.smithi071.stdout:4/547: mknod d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d4f/cb4 0 2024-04-23T02:14:01.890 INFO:tasks.workunit.client.0.smithi071.stdout:8/514: link dc/d15/d2d/c33 dc/d15/d72/ca5 0 2024-04-23T02:14:01.890 INFO:tasks.workunit.client.1.smithi192.stdout:3/492: dwrite d1/d3/d17/d1f/d33/d24/d34/d6c/d84/d46/f5e [0,4194304] 0 2024-04-23T02:14:01.891 INFO:tasks.workunit.client.0.smithi071.stdout:7/462: dread d3/d17/d41/f2f [0,4194304] 0 2024-04-23T02:14:01.891 INFO:tasks.workunit.client.0.smithi071.stdout:4/548: stat d2/d2a/d2d/d7b/f8c 0 2024-04-23T02:14:01.891 INFO:tasks.workunit.client.0.smithi071.stdout:6/475: truncate d6/d22/f35 982230 0 2024-04-23T02:14:01.892 INFO:tasks.workunit.client.1.smithi192.stdout:7/558: fsync d5/d43/d4c/d4e/f7f 0 2024-04-23T02:14:01.892 INFO:tasks.workunit.client.1.smithi192.stdout:6/630: dread d3/d15/d5b/f5e [0,4194304] 0 2024-04-23T02:14:01.893 INFO:tasks.workunit.client.1.smithi192.stdout:6/631: readlink d3/d15/d5b/l62 0 2024-04-23T02:14:01.895 INFO:tasks.workunit.client.0.smithi071.stdout:4/549: mkdir d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d4f/db5 0 2024-04-23T02:14:01.895 INFO:tasks.workunit.client.1.smithi192.stdout:3/493: creat d1/d3/d17/d1f/d33/d24/d34/d6c/d84/d5d/fa7 x:0 0 0 2024-04-23T02:14:01.896 INFO:tasks.workunit.client.0.smithi071.stdout:1/474: write db/df/d13/d72/f79 [624633,103343] 0 2024-04-23T02:14:01.898 INFO:tasks.workunit.client.1.smithi192.stdout:3/494: write d1/d3/d6/d7/f82 [303340,125572] 0 2024-04-23T02:14:01.900 INFO:tasks.workunit.client.1.smithi192.stdout:7/559: rename d5/d43/d4c/d4e/f7f to d5/d17/d1c/d59/d5a/d41/d50/d9f/fa4 0 2024-04-23T02:14:01.901 INFO:tasks.workunit.client.1.smithi192.stdout:5/566: dwrite d2/f2c [0,4194304] 0 2024-04-23T02:14:01.902 INFO:tasks.workunit.client.0.smithi071.stdout:4/550: mkdir d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d4f/db5/db6 0 2024-04-23T02:14:01.904 INFO:tasks.workunit.client.1.smithi192.stdout:6/632: link d3/d18/f78 d3/d15/d1b/d21/d52/d64/fd1 0 2024-04-23T02:14:01.905 INFO:tasks.workunit.client.0.smithi071.stdout:2/451: dread d0/d2/d26/df/d30/d80/f5f [0,4194304] 0 2024-04-23T02:14:01.905 INFO:tasks.workunit.client.1.smithi192.stdout:7/560: chown d5/d17/d1c/d57/l19 236918548 1 2024-04-23T02:14:01.907 INFO:tasks.workunit.client.0.smithi071.stdout:4/551: write d2/d2a/d2d/d39/d42/d49/d67/d3e/f7d [477239,93980] 0 2024-04-23T02:14:01.908 INFO:tasks.workunit.client.0.smithi071.stdout:3/467: truncate d7/d8/f4e 1326307 0 2024-04-23T02:14:01.908 INFO:tasks.workunit.client.1.smithi192.stdout:5/567: rename d2/db/d16/d1b/d3d/d40/d49/f58 to d2/d6/d2d/d41/d43/dbc/fc1 0 2024-04-23T02:14:01.910 INFO:tasks.workunit.client.1.smithi192.stdout:7/561: write d5/d17/d1c/d59/d5a/d41/d50/f62 [5094993,35956] 0 2024-04-23T02:14:01.915 INFO:tasks.workunit.client.0.smithi071.stdout:2/452: write d0/d2/d26/d3a/f81 [199439,126188] 0 2024-04-23T02:14:01.917 INFO:tasks.workunit.client.0.smithi071.stdout:2/453: read - d0/d2/d26/d3a/f77 zero size 2024-04-23T02:14:01.918 INFO:tasks.workunit.client.1.smithi192.stdout:6/633: link d3/d11/l6a d3/d15/d1b/d21/d52/d64/ld2 0 2024-04-23T02:14:01.918 INFO:tasks.workunit.client.0.smithi071.stdout:3/468: chown d7/d10/d38/d67/d6f/c94 1678473 1 2024-04-23T02:14:01.918 INFO:tasks.workunit.client.0.smithi071.stdout:0/467: fsync d0/d3/de/d10/d14/d32/d44/d31/d52/f5c 0 2024-04-23T02:14:01.920 INFO:tasks.workunit.client.1.smithi192.stdout:7/562: rename d5/d43/d4c/c77 to d5/d43/d4c/d5c/ca5 0 2024-04-23T02:14:01.920 INFO:tasks.workunit.client.1.smithi192.stdout:6/634: write d3/d15/d1b/fb4 [302479,60612] 0 2024-04-23T02:14:01.921 INFO:tasks.workunit.client.0.smithi071.stdout:4/552: dread d2/d2a/d2d/f54 [0,4194304] 0 2024-04-23T02:14:01.921 INFO:tasks.workunit.client.1.smithi192.stdout:5/568: dwrite d2/d6/d2d/d41/f97 [0,4194304] 0 2024-04-23T02:14:01.921 INFO:tasks.workunit.client.0.smithi071.stdout:8/515: dread dc/d15/da0/f3d [0,4194304] 0 2024-04-23T02:14:01.922 INFO:tasks.workunit.client.0.smithi071.stdout:2/454: mknod d0/c8e 0 2024-04-23T02:14:01.922 INFO:tasks.workunit.client.0.smithi071.stdout:3/469: mkdir d7/d8/d42/d5f/d68/d3b/d4c/da1 0 2024-04-23T02:14:01.923 INFO:tasks.workunit.client.0.smithi071.stdout:0/468: chown d0/d3/de/d48/c49 640 1 2024-04-23T02:14:01.923 INFO:tasks.workunit.client.1.smithi192.stdout:3/495: link d1/d3/d17/d1f/d33/d24/d34/d6c/d84/f83 d1/d3/d17/fa8 0 2024-04-23T02:14:01.925 INFO:tasks.workunit.client.1.smithi192.stdout:5/569: fsync d2/d6/d14/d60/fa4 0 2024-04-23T02:14:01.926 INFO:tasks.workunit.client.1.smithi192.stdout:7/563: chown d5/d17/d1c/d59/d5a/d70/f7e 29652 1 2024-04-23T02:14:01.927 INFO:tasks.workunit.client.0.smithi071.stdout:2/455: write d0/d2/d45/f5a [2621108,96692] 0 2024-04-23T02:14:01.928 INFO:tasks.workunit.client.1.smithi192.stdout:6/635: read d3/d28/f2e [2183698,56876] 0 2024-04-23T02:14:01.928 INFO:tasks.workunit.client.0.smithi071.stdout:8/516: rename dc/d15/da0/d55/f94 to dc/d15/da0/d30/fa6 0 2024-04-23T02:14:01.928 INFO:tasks.workunit.client.0.smithi071.stdout:1/475: dread db/df/d5f/d57/f89 [0,4194304] 0 2024-04-23T02:14:01.928 INFO:tasks.workunit.client.0.smithi071.stdout:0/469: truncate d0/d3/de/d10/d14/f64 1133908 0 2024-04-23T02:14:01.929 INFO:tasks.workunit.client.0.smithi071.stdout:3/470: creat d7/d22/d23/d26/d75/fa2 x:0 0 0 2024-04-23T02:14:01.934 INFO:tasks.workunit.client.1.smithi192.stdout:3/496: rename d1/d3/d17/d1f/d33/d24/d34/d38 to d1/d3/d6/d7/d29/d3b/da9 0 2024-04-23T02:14:01.936 INFO:tasks.workunit.client.1.smithi192.stdout:6/636: rmdir d3/d15/d1b/d66/d80/d8e 39 2024-04-23T02:14:01.936 INFO:tasks.workunit.client.0.smithi071.stdout:4/553: truncate d2/d2a/f36 1676842 0 2024-04-23T02:14:01.938 INFO:tasks.workunit.client.1.smithi192.stdout:7/564: dread d5/d17/d1c/d26/f34 [0,4194304] 0 2024-04-23T02:14:01.939 INFO:tasks.workunit.client.0.smithi071.stdout:0/470: readlink d0/d3/de/d48/l6d 0 2024-04-23T02:14:01.939 INFO:tasks.workunit.client.1.smithi192.stdout:7/565: chown d5/d43/d4c/d5c 1005202 1 2024-04-23T02:14:01.942 INFO:tasks.workunit.client.1.smithi192.stdout:5/570: rename d2/db/d16/d1b/d3d/d40 to d2/db/d2b/d52/dc2 0 2024-04-23T02:14:01.942 INFO:tasks.workunit.client.1.smithi192.stdout:3/497: read d1/d3/d17/f3d [701220,110961] 0 2024-04-23T02:14:01.942 INFO:tasks.workunit.client.0.smithi071.stdout:2/456: dwrite d0/d2/d26/d14/f23 [4194304,4194304] 0 2024-04-23T02:14:01.943 INFO:tasks.workunit.client.0.smithi071.stdout:0/471: dread d0/f6b [0,4194304] 0 2024-04-23T02:14:01.943 INFO:tasks.workunit.client.1.smithi192.stdout:6/637: dwrite d3/d11/f4a [0,4194304] 0 2024-04-23T02:14:01.944 INFO:tasks.workunit.client.0.smithi071.stdout:4/554: mknod d2/d2a/cb7 0 2024-04-23T02:14:01.945 INFO:tasks.workunit.client.0.smithi071.stdout:4/555: readlink d2/l11 0 2024-04-23T02:14:01.951 INFO:tasks.workunit.client.0.smithi071.stdout:1/476: truncate db/df/d13/d2f/d48/f6e 4086785 0 2024-04-23T02:14:01.953 INFO:tasks.workunit.client.0.smithi071.stdout:2/457: creat d0/d2/d26/d3a/d75/f8f x:0 0 0 2024-04-23T02:14:01.954 INFO:tasks.workunit.client.1.smithi192.stdout:5/571: creat d2/d2e/d4e/fc3 x:0 0 0 2024-04-23T02:14:01.954 INFO:tasks.workunit.client.1.smithi192.stdout:7/566: unlink d5/d17/d1c/d59/c6d 0 2024-04-23T02:14:01.955 INFO:tasks.workunit.client.0.smithi071.stdout:0/472: dwrite d0/d63/f66 [0,4194304] 0 2024-04-23T02:14:01.956 INFO:tasks.workunit.client.0.smithi071.stdout:1/477: mkdir db/d11/d8d/d97 0 2024-04-23T02:14:01.957 INFO:tasks.workunit.client.1.smithi192.stdout:6/638: creat d3/d15/d1b/dcf/fd3 x:0 0 0 2024-04-23T02:14:01.959 INFO:tasks.workunit.client.1.smithi192.stdout:7/567: mkdir d5/d43/d4c/d5c/d67/da6 0 2024-04-23T02:14:01.962 INFO:tasks.workunit.client.0.smithi071.stdout:8/517: dwrite dc/d3a/f47 [0,4194304] 0 2024-04-23T02:14:01.963 INFO:tasks.workunit.client.0.smithi071.stdout:8/518: read - dc/f9f zero size 2024-04-23T02:14:01.964 INFO:tasks.workunit.client.0.smithi071.stdout:0/473: mkdir d0/d92 0 2024-04-23T02:14:01.965 INFO:tasks.workunit.client.0.smithi071.stdout:8/519: rmdir dc/d15/d5d 39 2024-04-23T02:14:01.965 INFO:tasks.workunit.client.1.smithi192.stdout:7/568: unlink d5/d17/c42 0 2024-04-23T02:14:01.966 INFO:tasks.workunit.client.0.smithi071.stdout:8/520: write dc/d56/f61 [2506624,102922] 0 2024-04-23T02:14:01.967 INFO:tasks.workunit.client.1.smithi192.stdout:7/569: chown d5/c13 153665 1 2024-04-23T02:14:01.970 INFO:tasks.workunit.client.0.smithi071.stdout:0/474: mkdir d0/d3/de/d10/d14/d32/d44/d87/d93 0 2024-04-23T02:14:01.970 INFO:tasks.workunit.client.0.smithi071.stdout:1/478: link db/d34/f61 db/df/d13/d2f/d48/d85/f98 0 2024-04-23T02:14:01.973 INFO:tasks.workunit.client.1.smithi192.stdout:6/639: dwrite d3/d15/d1b/da2/fc3 [0,4194304] 0 2024-04-23T02:14:01.974 INFO:tasks.workunit.client.0.smithi071.stdout:1/479: symlink db/d34/d68/l99 0 2024-04-23T02:14:01.978 INFO:tasks.workunit.client.0.smithi071.stdout:8/521: dwrite dc/d15/da0/d8d/f9e [0,4194304] 0 2024-04-23T02:14:01.978 INFO:tasks.workunit.client.1.smithi192.stdout:6/640: dread - d3/d15/d1b/dcf/fd3 zero size 2024-04-23T02:14:01.978 INFO:tasks.workunit.client.1.smithi192.stdout:3/498: dread d1/d3/d6/f10 [0,4194304] 0 2024-04-23T02:14:01.979 INFO:tasks.workunit.client.1.smithi192.stdout:7/570: rename d5/d17/d1c/d26/c27 to d5/ca7 0 2024-04-23T02:14:01.980 INFO:tasks.workunit.client.0.smithi071.stdout:0/475: rename d0/d3/de/d10/d14/d32/f45 to d0/d3/de/d48/d7e/d8b/f94 0 2024-04-23T02:14:01.981 INFO:tasks.workunit.client.0.smithi071.stdout:1/480: mknod db/d34/d8a/c9a 0 2024-04-23T02:14:01.983 INFO:tasks.workunit.client.1.smithi192.stdout:3/499: dread - d1/d3/d17/d1f/d33/d24/f50 zero size 2024-04-23T02:14:01.984 INFO:tasks.workunit.client.0.smithi071.stdout:8/522: fsync dc/d15/da0/f3d 0 2024-04-23T02:14:01.987 INFO:tasks.workunit.client.0.smithi071.stdout:0/476: mkdir d0/d3/d95 0 2024-04-23T02:14:01.990 INFO:tasks.workunit.client.0.smithi071.stdout:1/481: symlink db/d34/l9b 0 2024-04-23T02:14:01.991 INFO:tasks.workunit.client.0.smithi071.stdout:0/477: creat d0/d3/de/d10/d14/d32/d44/d87/f96 x:0 0 0 2024-04-23T02:14:01.991 INFO:tasks.workunit.client.1.smithi192.stdout:6/641: rename d3/d15/d1b/d21/d52/d64 to d3/d18/dd4 0 2024-04-23T02:14:01.992 INFO:tasks.workunit.client.1.smithi192.stdout:3/500: rmdir d1/d3/d17/d2b/d44 39 2024-04-23T02:14:01.992 INFO:tasks.workunit.client.1.smithi192.stdout:7/571: mkdir d5/d17/d1c/d59/da8 0 2024-04-23T02:14:01.992 INFO:tasks.workunit.client.0.smithi071.stdout:4/556: truncate d2/d2a/d2d/d39/d42/f55 717781 0 2024-04-23T02:14:01.993 INFO:tasks.workunit.client.0.smithi071.stdout:1/482: stat db/df/f1c 0 2024-04-23T02:14:01.993 INFO:tasks.workunit.client.0.smithi071.stdout:4/557: dread - d2/d53/fa5 zero size 2024-04-23T02:14:01.994 INFO:tasks.workunit.client.0.smithi071.stdout:1/483: stat db/d34/d68/f87 0 2024-04-23T02:14:01.994 INFO:tasks.workunit.client.1.smithi192.stdout:3/501: creat d1/d3/d17/d1f/d33/faa x:0 0 0 2024-04-23T02:14:01.994 INFO:tasks.workunit.client.1.smithi192.stdout:7/572: write d5/d17/d1c/d59/d5a/d70/f7e [940294,72922] 0 2024-04-23T02:14:01.994 INFO:tasks.workunit.client.1.smithi192.stdout:6/642: creat d3/d28/d79/dbe/dcb/fd5 x:0 0 0 2024-04-23T02:14:01.995 INFO:tasks.workunit.client.1.smithi192.stdout:6/643: read d3/d15/d1b/fb4 [343806,64240] 0 2024-04-23T02:14:01.996 INFO:tasks.workunit.client.0.smithi071.stdout:8/523: read dc/d15/f23 [3573320,115948] 0 2024-04-23T02:14:01.998 INFO:tasks.workunit.client.0.smithi071.stdout:0/478: rename d0/d3/de/d10/d14/d32/f35 to d0/d3/de/d10/d16/f97 0 2024-04-23T02:14:02.000 INFO:tasks.workunit.client.1.smithi192.stdout:6/644: creat d3/d15/d1b/d66/d80/dbb/fd6 x:0 0 0 2024-04-23T02:14:02.001 INFO:tasks.workunit.client.1.smithi192.stdout:6/645: readlink d3/d15/d1b/l74 0 2024-04-23T02:14:02.002 INFO:tasks.workunit.client.1.smithi192.stdout:6/646: write d3/d15/d5b/fb8 [364381,78976] 0 2024-04-23T02:14:02.002 INFO:tasks.workunit.client.0.smithi071.stdout:1/484: creat db/df/d13/d20/d75/f9c x:0 0 0 2024-04-23T02:14:02.003 INFO:tasks.workunit.client.1.smithi192.stdout:5/572: dread d2/d6/f3b [0,4194304] 0 2024-04-23T02:14:02.010 INFO:tasks.workunit.client.0.smithi071.stdout:2/458: dread d0/d2/fb [0,4194304] 0 2024-04-23T02:14:02.021 INFO:tasks.workunit.client.0.smithi071.stdout:4/558: dwrite d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f8e [0,4194304] 0 2024-04-23T02:14:02.022 INFO:tasks.workunit.client.0.smithi071.stdout:2/459: unlink d0/d2/d26/d14/d1f/d36/f4a 0 2024-04-23T02:14:02.023 INFO:tasks.workunit.client.0.smithi071.stdout:0/479: creat d0/d76/f98 x:0 0 0 2024-04-23T02:14:02.051 INFO:tasks.workunit.client.0.smithi071.stdout:8/524: dwrite dc/d15/da0/d55/f65 [4194304,4194304] 0 2024-04-23T02:14:02.056 INFO:tasks.workunit.client.1.smithi192.stdout:9/496: sync 2024-04-23T02:14:02.056 INFO:tasks.workunit.client.1.smithi192.stdout:1/332: sync 2024-04-23T02:14:02.059 INFO:tasks.workunit.client.0.smithi071.stdout:1/485: link db/df/d5f/d6f/f6b db/df/d13/d2f/d7b/f9d 0 2024-04-23T02:14:02.068 INFO:tasks.workunit.client.0.smithi071.stdout:8/525: dwrite dc/f57 [0,4194304] 0 2024-04-23T02:14:02.068 INFO:tasks.workunit.client.0.smithi071.stdout:0/480: rename d0/d3/de/d10/d14/l8e to d0/d76/l99 0 2024-04-23T02:14:02.070 INFO:tasks.workunit.client.0.smithi071.stdout:8/526: fsync dc/d15/da0/d55/f80 0 2024-04-23T02:14:02.071 INFO:tasks.workunit.client.0.smithi071.stdout:0/481: dread - d0/d3/de/d10/d14/d32/d44/d87/f96 zero size 2024-04-23T02:14:02.072 INFO:tasks.workunit.client.0.smithi071.stdout:8/527: chown dc/d15/d50/d5f/l84 175080567 1 2024-04-23T02:14:02.072 INFO:tasks.workunit.client.0.smithi071.stdout:0/482: stat d0/d3/de/d10 0 2024-04-23T02:14:02.072 INFO:tasks.workunit.client.0.smithi071.stdout:2/460: getdents d0/d2/d34 0 2024-04-23T02:14:02.073 INFO:tasks.workunit.client.0.smithi071.stdout:0/483: chown d0/f20 0 1 2024-04-23T02:14:02.075 INFO:tasks.workunit.client.0.smithi071.stdout:2/461: write d0/d2/d34/f50 [4491619,83473] 0 2024-04-23T02:14:02.078 INFO:tasks.workunit.client.0.smithi071.stdout:1/486: rename db/df/d3a/c6d to db/df/d5f/c9e 0 2024-04-23T02:14:02.078 INFO:tasks.workunit.client.0.smithi071.stdout:2/462: fsync d0/d2/f1d 0 2024-04-23T02:14:02.080 INFO:tasks.workunit.client.0.smithi071.stdout:4/559: dwrite d2/d2a/d2d/d39/f63 [0,4194304] 0 2024-04-23T02:14:02.081 INFO:tasks.workunit.client.0.smithi071.stdout:2/463: write d0/d2/d26/d3a/d75/f8f [190746,125929] 0 2024-04-23T02:14:02.084 INFO:tasks.workunit.client.0.smithi071.stdout:1/487: mkdir db/df/d13/d2f/d48/d9f 0 2024-04-23T02:14:02.084 INFO:tasks.workunit.client.0.smithi071.stdout:2/464: write d0/d2/f29 [455854,108195] 0 2024-04-23T02:14:02.085 INFO:tasks.workunit.client.0.smithi071.stdout:0/484: creat d0/d3/de/d10/d14/d32/d37/d26/f9a x:0 0 0 2024-04-23T02:14:02.086 INFO:tasks.workunit.client.0.smithi071.stdout:4/560: mknod d2/d2a/cb8 0 2024-04-23T02:14:02.087 INFO:tasks.workunit.client.0.smithi071.stdout:2/465: rmdir d0 39 2024-04-23T02:14:02.089 INFO:tasks.workunit.client.0.smithi071.stdout:0/485: mknod d0/d63/c9b 0 2024-04-23T02:14:02.094 INFO:tasks.workunit.client.1.smithi192.stdout:6/647: symlink d3/d18/dd4/ld7 0 2024-04-23T02:14:02.095 INFO:tasks.workunit.client.1.smithi192.stdout:5/573: fdatasync d2/db/d16/d1b/f9f 0 2024-04-23T02:14:02.095 INFO:tasks.workunit.client.0.smithi071.stdout:4/561: dwrite d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f5d [0,4194304] 0 2024-04-23T02:14:02.095 INFO:tasks.workunit.client.0.smithi071.stdout:2/466: creat d0/d2/f90 x:0 0 0 2024-04-23T02:14:02.096 INFO:tasks.workunit.client.0.smithi071.stdout:0/486: mknod d0/d3/de/d10/d14/d32/d44/d87/d93/c9c 0 2024-04-23T02:14:02.097 INFO:tasks.workunit.client.1.smithi192.stdout:7/573: write d5/d17/d1c/d4a/f76 [707583,5370] 0 2024-04-23T02:14:02.098 INFO:tasks.workunit.client.1.smithi192.stdout:6/648: read - d3/f76 zero size 2024-04-23T02:14:02.099 INFO:tasks.workunit.client.0.smithi071.stdout:4/562: mkdir d2/db9 0 2024-04-23T02:14:02.100 INFO:tasks.workunit.client.0.smithi071.stdout:4/563: readlink d2/d2a/d2d/d39/d42/d49/d67/d3e/l52 0 2024-04-23T02:14:02.101 INFO:tasks.workunit.client.1.smithi192.stdout:1/333: symlink dc/d16/d19/d30/d32/l60 0 2024-04-23T02:14:02.102 INFO:tasks.workunit.client.0.smithi071.stdout:2/467: link d0/d2/d26/d14/d1f/d4f/f8b d0/d2/d34/f91 0 2024-04-23T02:14:02.102 INFO:tasks.workunit.client.1.smithi192.stdout:7/574: chown d5/d17/d1c/d26/f34 3517 1 2024-04-23T02:14:02.104 INFO:tasks.workunit.client.0.smithi071.stdout:0/487: rename d0/d3/de/d48/d7e/c8f to d0/d3/de/d10/d14/c9d 0 2024-04-23T02:14:02.108 INFO:tasks.workunit.client.0.smithi071.stdout:0/488: symlink d0/d3/l9e 0 2024-04-23T02:14:02.109 INFO:tasks.workunit.client.0.smithi071.stdout:4/564: dwrite d2/d2a/d2d/f66 [0,4194304] 0 2024-04-23T02:14:02.109 INFO:tasks.workunit.client.1.smithi192.stdout:3/502: rename d1/d3/d17/d1f/d33/d24/d34 to d1/d3/d17/d1a/d7c/dab 0 2024-04-23T02:14:02.110 INFO:tasks.workunit.client.0.smithi071.stdout:2/468: creat d0/d2/d26/d14/d1f/f92 x:0 0 0 2024-04-23T02:14:02.110 INFO:tasks.workunit.client.0.smithi071.stdout:1/488: write db/df/d13/d2f/d48/f6e [4235516,54078] 0 2024-04-23T02:14:02.111 INFO:tasks.workunit.client.0.smithi071.stdout:1/489: dread - db/d34/d68/f81 zero size 2024-04-23T02:14:02.111 INFO:tasks.workunit.client.0.smithi071.stdout:2/469: chown d0/d2/d45/c62 84916371 1 2024-04-23T02:14:02.112 INFO:tasks.workunit.client.1.smithi192.stdout:1/334: readlink dc/d16/d19/d30/d32/l48 0 2024-04-23T02:14:02.112 INFO:tasks.workunit.client.1.smithi192.stdout:5/574: dwrite d2/db/fad [0,4194304] 0 2024-04-23T02:14:02.113 INFO:tasks.workunit.client.0.smithi071.stdout:4/565: dread - d2/d2a/d2d/d39/d42/d49/d67/d7e/d82/fb1 zero size 2024-04-23T02:14:02.113 INFO:tasks.workunit.client.0.smithi071.stdout:2/470: stat d0/d2/d26 0 2024-04-23T02:14:02.117 INFO:tasks.workunit.client.1.smithi192.stdout:9/497: dwrite d0/d9/d44/d51/d64/fac [0,4194304] 0 2024-04-23T02:14:02.118 INFO:tasks.workunit.client.0.smithi071.stdout:2/471: write d0/f2e [5126158,77316] 0 2024-04-23T02:14:02.121 INFO:tasks.workunit.client.1.smithi192.stdout:1/335: fdatasync dc/d16/d19/d30/d32/f33 0 2024-04-23T02:14:02.123 INFO:tasks.workunit.client.0.smithi071.stdout:1/490: link db/df/d13/d2f/c2e db/df/d13/d2f/d48/ca0 0 2024-04-23T02:14:02.124 INFO:tasks.workunit.client.1.smithi192.stdout:5/575: mknod d2/d32/cc4 0 2024-04-23T02:14:02.125 INFO:tasks.workunit.client.1.smithi192.stdout:6/649: rename d3/d18/dd4/l7b to d3/d15/d1b/d21/d3a/ld8 0 2024-04-23T02:14:02.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:01 smithi192 ceph-mon[49662]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:14:02.127 INFO:tasks.workunit.client.0.smithi071.stdout:4/566: link d2/d2a/d2d/d7b/c9a d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c/cba 0 2024-04-23T02:14:02.127 INFO:tasks.workunit.client.0.smithi071.stdout:0/489: dwrite d0/d3/de/d10/d14/d32/d44/d31/d52/f5c [0,4194304] 0 2024-04-23T02:14:02.128 INFO:tasks.workunit.client.1.smithi192.stdout:3/503: creat d1/d3/d6/d7/d29/d3b/da9/d8b/fac x:0 0 0 2024-04-23T02:14:02.129 INFO:tasks.workunit.client.1.smithi192.stdout:3/504: readlink d1/d3/l6f 0 2024-04-23T02:14:02.130 INFO:tasks.workunit.client.1.smithi192.stdout:7/575: dwrite d5/d17/d1c/d59/d5a/d41/d50/f7b [0,4194304] 0 2024-04-23T02:14:02.130 INFO:tasks.workunit.client.1.smithi192.stdout:9/498: creat d0/d9/d44/d51/fae x:0 0 0 2024-04-23T02:14:02.132 INFO:tasks.workunit.client.0.smithi071.stdout:1/491: symlink db/la1 0 2024-04-23T02:14:02.134 INFO:tasks.workunit.client.0.smithi071.stdout:2/472: rename d0/d2/d26/d3a/c41 to d0/d2/d26/d3a/c93 0 2024-04-23T02:14:02.134 INFO:tasks.workunit.client.0.smithi071.stdout:4/567: mkdir d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d10/dbb 0 2024-04-23T02:14:02.137 INFO:tasks.workunit.client.1.smithi192.stdout:1/336: symlink dc/d16/d19/d30/d32/d34/l61 0 2024-04-23T02:14:02.137 INFO:tasks.workunit.client.0.smithi071.stdout:1/492: mknod db/df/d13/d72/d86/ca2 0 2024-04-23T02:14:02.141 INFO:tasks.workunit.client.0.smithi071.stdout:0/490: dwrite d0/d3/de/d10/d14/d32/f82 [0,4194304] 0 2024-04-23T02:14:02.142 INFO:tasks.workunit.client.0.smithi071.stdout:4/568: write d2/d2a/d2d/d39/d42/d49/d67/d1d/f61 [2059430,52450] 0 2024-04-23T02:14:02.147 INFO:tasks.workunit.client.1.smithi192.stdout:7/576: write d5/d17/d1c/d59/d5a/d41/d50/f69 [1206212,119831] 0 2024-04-23T02:14:02.149 INFO:tasks.workunit.client.1.smithi192.stdout:9/499: dwrite d0/d9/f32 [0,4194304] 0 2024-04-23T02:14:02.149 INFO:tasks.workunit.client.1.smithi192.stdout:3/505: truncate d1/d3/d17/d2b/d44/f93 161398 0 2024-04-23T02:14:02.151 INFO:tasks.workunit.client.0.smithi071.stdout:4/569: symlink d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/lbc 0 2024-04-23T02:14:02.159 INFO:tasks.workunit.client.1.smithi192.stdout:6/650: dwrite d3/d28/f9a [4194304,4194304] 0 2024-04-23T02:14:02.159 INFO:tasks.workunit.client.1.smithi192.stdout:9/500: chown d0/d9/d44/d51/d5b/d96 0 1 2024-04-23T02:14:02.159 INFO:tasks.workunit.client.0.smithi071.stdout:4/570: dwrite d2/f89 [0,4194304] 0 2024-04-23T02:14:02.164 INFO:tasks.workunit.client.1.smithi192.stdout:1/337: dwrite dc/d16/d19/d30/f55 [0,4194304] 0 2024-04-23T02:14:02.166 INFO:tasks.workunit.client.1.smithi192.stdout:3/506: creat d1/d3/d17/d1f/d33/fad x:0 0 0 2024-04-23T02:14:02.168 INFO:tasks.workunit.client.0.smithi071.stdout:4/571: getdents d2/d2a/d2d/d39/d42/d49/d67/d7e 0 2024-04-23T02:14:02.169 INFO:tasks.workunit.client.1.smithi192.stdout:1/338: truncate dc/d16/d19/f3a 941875 0 2024-04-23T02:14:02.169 INFO:tasks.workunit.client.0.smithi071.stdout:4/572: write d2/d2a/d2d/f5c [1182552,835] 0 2024-04-23T02:14:02.170 INFO:tasks.workunit.client.1.smithi192.stdout:6/651: getdents d3/d15/d1b/d21/d52/db2 0 2024-04-23T02:14:02.176 INFO:tasks.workunit.client.1.smithi192.stdout:9/501: dread d0/d9/d44/d51/d5b/f69 [0,4194304] 0 2024-04-23T02:14:02.176 INFO:tasks.workunit.client.0.smithi071.stdout:4/573: rename d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d29/f85 to d2/d2a/d2d/d39/d42/d49/d67/d7e/dae/fbd 0 2024-04-23T02:14:02.177 INFO:tasks.workunit.client.1.smithi192.stdout:3/507: creat d1/d3/d17/d1a/d7c/dab/d6c/d84/d4c/fae x:0 0 0 2024-04-23T02:14:02.179 INFO:tasks.workunit.client.0.smithi071.stdout:2/473: dread d0/d2/d26/f18 [0,4194304] 0 2024-04-23T02:14:02.180 INFO:tasks.workunit.client.1.smithi192.stdout:9/502: rmdir d0/d9/d44/d51/d64 39 2024-04-23T02:14:02.180 INFO:tasks.workunit.client.1.smithi192.stdout:6/652: dread d3/f2b [4194304,4194304] 0 2024-04-23T02:14:02.181 INFO:tasks.workunit.client.0.smithi071.stdout:4/574: truncate d2/d2a/d73/d5e/fb0 785480 0 2024-04-23T02:14:02.182 INFO:tasks.workunit.client.1.smithi192.stdout:3/508: mkdir d1/d3/d6/d7/d29/d3b/da9/daf 0 2024-04-23T02:14:02.183 INFO:tasks.workunit.client.1.smithi192.stdout:9/503: creat d0/d9/d44/d51/d5b/d61/faf x:0 0 0 2024-04-23T02:14:02.185 INFO:tasks.workunit.client.0.smithi071.stdout:2/474: rename d0/d2/d26/f18 to d0/d70/f94 0 2024-04-23T02:14:02.185 INFO:tasks.workunit.client.1.smithi192.stdout:5/576: dread d2/d32/f6e [0,4194304] 0 2024-04-23T02:14:02.186 INFO:tasks.workunit.client.1.smithi192.stdout:6/653: rename d3/d18/dd4/fa7 to d3/d15/d1b/d66/d80/dbb/fd9 0 2024-04-23T02:14:02.186 INFO:tasks.workunit.client.1.smithi192.stdout:5/577: fsync d2/db/d2b/d52/dc2/d49/f91 0 2024-04-23T02:14:02.188 INFO:tasks.workunit.client.1.smithi192.stdout:9/504: symlink d0/lb0 0 2024-04-23T02:14:02.189 INFO:tasks.workunit.client.0.smithi071.stdout:4/575: dwrite d2/d2a/d2d/d39/d42/d49/d67/d3e/f4d [0,4194304] 0 2024-04-23T02:14:02.190 INFO:tasks.workunit.client.1.smithi192.stdout:6/654: mkdir d3/d15/d1b/d21/d7c/dda 0 2024-04-23T02:14:02.190 INFO:tasks.workunit.client.1.smithi192.stdout:3/509: getdents d1/d3/d17/d1a/d7c/dab/d6c/d84/d5d/d98 0 2024-04-23T02:14:02.195 INFO:tasks.workunit.client.0.smithi071.stdout:2/475: dwrite d0/d82/f87 [0,4194304] 0 2024-04-23T02:14:02.197 INFO:tasks.workunit.client.0.smithi071.stdout:4/576: symlink d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d4f/lbe 0 2024-04-23T02:14:02.198 INFO:tasks.workunit.client.0.smithi071.stdout:4/577: dread - d2/d53/fa5 zero size 2024-04-23T02:14:02.199 INFO:tasks.workunit.client.0.smithi071.stdout:2/476: creat d0/d2/d26/d14/d1f/f95 x:0 0 0 2024-04-23T02:14:02.205 INFO:tasks.workunit.client.1.smithi192.stdout:5/578: dwrite d2/d6/d2d/faa [0,4194304] 0 2024-04-23T02:14:02.205 INFO:tasks.workunit.client.1.smithi192.stdout:3/510: write d1/d3/f7a [1681276,28784] 0 2024-04-23T02:14:02.207 INFO:tasks.workunit.client.0.smithi071.stdout:1/493: dread db/df/d13/f71 [0,4194304] 0 2024-04-23T02:14:02.207 INFO:tasks.workunit.client.0.smithi071.stdout:1/494: stat db/d11 0 2024-04-23T02:14:02.210 INFO:tasks.workunit.client.1.smithi192.stdout:3/511: stat d1/d3/d6/f79 0 2024-04-23T02:14:02.210 INFO:tasks.workunit.client.0.smithi071.stdout:4/578: rename d2/d2a/d2d/d39/d42/d49/d67/d3e/l44 to d2/d2a/d73/d5e/lbf 0 2024-04-23T02:14:02.217 INFO:tasks.workunit.client.1.smithi192.stdout:5/579: getdents d2/db/d16/d48/db7 0 2024-04-23T02:14:02.217 INFO:tasks.workunit.client.1.smithi192.stdout:6/655: dwrite d3/d15/d1b/d21/d3a/fa6 [0,4194304] 0 2024-04-23T02:14:02.217 INFO:tasks.workunit.client.1.smithi192.stdout:3/512: write d1/d3/d6/f81 [1281415,102031] 0 2024-04-23T02:14:02.218 INFO:tasks.workunit.client.0.smithi071.stdout:2/477: dwrite d0/d2/d34/d55/f72 [0,4194304] 0 2024-04-23T02:14:02.218 INFO:tasks.workunit.client.1.smithi192.stdout:7/577: dwrite d5/d17/d1c/f3e [0,4194304] 0 2024-04-23T02:14:02.220 INFO:tasks.workunit.client.1.smithi192.stdout:1/339: dread dc/f29 [0,4194304] 0 2024-04-23T02:14:02.223 INFO:tasks.workunit.client.0.smithi071.stdout:4/579: dwrite d2/d2a/d2d/d39/d42/d49/d67/d1d/d57/f72 [0,4194304] 0 2024-04-23T02:14:02.224 INFO:tasks.workunit.client.1.smithi192.stdout:6/656: write d3/d28/d79/dbe/dcb/fd5 [368788,24017] 0 2024-04-23T02:14:02.225 INFO:tasks.workunit.client.1.smithi192.stdout:6/657: chown d3/d15/d1b/d21/d7c/cb3 41109 1 2024-04-23T02:14:02.227 INFO:tasks.workunit.client.1.smithi192.stdout:6/658: chown d3/d18/d58/cb5 0 1 2024-04-23T02:14:02.227 INFO:tasks.workunit.client.1.smithi192.stdout:8/701: sync 2024-04-23T02:14:02.228 INFO:tasks.workunit.client.0.smithi071.stdout:2/478: dwrite d0/d2/d45/f60 [0,4194304] 0 2024-04-23T02:14:02.228 INFO:tasks.workunit.client.1.smithi192.stdout:5/580: rename d2/d6/d2d/db3 to d2/db/d16/d48/dc5 0 2024-04-23T02:14:02.231 INFO:tasks.workunit.client.1.smithi192.stdout:1/340: mkdir dc/d16/d19/d30/d32/d62 0 2024-04-23T02:14:02.234 INFO:tasks.workunit.client.0.smithi071.stdout:2/479: mkdir d0/d2/d34/d55/d96 0 2024-04-23T02:14:02.234 INFO:tasks.workunit.client.1.smithi192.stdout:7/578: mkdir d5/d43/d4c/d5c/d67/da6/da9 0 2024-04-23T02:14:02.237 INFO:tasks.workunit.client.1.smithi192.stdout:9/505: dread d0/d9/fb [0,4194304] 0 2024-04-23T02:14:02.239 INFO:tasks.workunit.client.1.smithi192.stdout:8/702: truncate d1/d12/d23/d2c/d4d/f52 1710521 0 2024-04-23T02:14:02.239 INFO:tasks.workunit.client.0.smithi071.stdout:1/495: dwrite db/df/d13/d2f/f29 [0,4194304] 0 2024-04-23T02:14:02.241 INFO:tasks.workunit.client.0.smithi071.stdout:2/480: read d0/d2/d45/d58/f7c [811705,26585] 0 2024-04-23T02:14:02.241 INFO:tasks.workunit.client.0.smithi071.stdout:1/496: write db/d34/d68/f76 [1150518,127423] 0 2024-04-23T02:14:02.243 INFO:tasks.workunit.client.0.smithi071.stdout:2/481: dread - d0/d2/d26/d3a/f77 zero size 2024-04-23T02:14:02.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:01 smithi071 ceph-mon[39837]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:14:02.244 INFO:tasks.workunit.client.0.smithi071.stdout:1/497: write db/df/d13/d20/d75/f95 [462809,11892] 0 2024-04-23T02:14:02.245 INFO:tasks.workunit.client.1.smithi192.stdout:3/513: dwrite d1/d3/d17/d1f/d33/faa [0,4194304] 0 2024-04-23T02:14:02.246 INFO:tasks.workunit.client.1.smithi192.stdout:6/659: mknod d3/d15/d1b/d66/d80/d8e/d9d/cdb 0 2024-04-23T02:14:02.248 INFO:tasks.workunit.client.0.smithi071.stdout:2/482: link d0/d2/d26/c11 d0/d2/d45/c97 0 2024-04-23T02:14:02.251 INFO:tasks.workunit.client.0.smithi071.stdout:2/483: getdents d0/d2/d26/d14/d1f/d36/d5c/d85 0 2024-04-23T02:14:02.252 INFO:tasks.workunit.client.0.smithi071.stdout:2/484: write d0/d2/d26/d3a/f44 [666769,70484] 0 2024-04-23T02:14:02.253 INFO:tasks.workunit.client.1.smithi192.stdout:8/703: creat d1/d9/ffa x:0 0 0 2024-04-23T02:14:02.256 INFO:tasks.workunit.client.0.smithi071.stdout:2/485: read d0/d2/d26/d14/f1a [2007031,2631] 0 2024-04-23T02:14:02.259 INFO:tasks.workunit.client.0.smithi071.stdout:2/486: mkdir d0/d2/d26/d14/d1f/d4f/d98 0 2024-04-23T02:14:02.260 INFO:tasks.workunit.client.0.smithi071.stdout:2/487: stat d0/d2/d45/d7d/d56/c7b 0 2024-04-23T02:14:02.261 INFO:tasks.workunit.client.1.smithi192.stdout:9/506: dwrite d0/d9/d44/d74/f8a [4194304,4194304] 0 2024-04-23T02:14:02.262 INFO:tasks.workunit.client.1.smithi192.stdout:6/660: truncate d3/d18/dd4/d8d/fba 442527 0 2024-04-23T02:14:02.262 INFO:tasks.workunit.client.0.smithi071.stdout:2/488: symlink d0/d2/d26/d3a/d75/l99 0 2024-04-23T02:14:02.263 INFO:tasks.workunit.client.1.smithi192.stdout:5/581: truncate d2/db/d2b/f5b 1953115 0 2024-04-23T02:14:02.263 INFO:tasks.workunit.client.1.smithi192.stdout:8/704: rmdir d1/d12/d23/d4b/d73/d45/d46 39 2024-04-23T02:14:02.264 INFO:tasks.workunit.client.1.smithi192.stdout:5/582: fdatasync d2/f5 0 2024-04-23T02:14:02.266 INFO:tasks.workunit.client.1.smithi192.stdout:1/341: link f9 dc/d16/d19/f63 0 2024-04-23T02:14:02.266 INFO:tasks.workunit.client.1.smithi192.stdout:8/705: write d1/d9/d3e/d59/f75 [1661075,130084] 0 2024-04-23T02:14:02.268 INFO:tasks.workunit.client.0.smithi071.stdout:5/586: sync 2024-04-23T02:14:02.269 INFO:tasks.workunit.client.1.smithi192.stdout:1/342: dread dc/d16/d19/f3a [0,4194304] 0 2024-04-23T02:14:02.269 INFO:tasks.workunit.client.0.smithi071.stdout:5/587: write d3/d1d/d43/dd2/f78 [4449832,62889] 0 2024-04-23T02:14:02.271 INFO:tasks.workunit.client.0.smithi071.stdout:2/489: dwrite d0/d2/d26/df/f40 [0,4194304] 0 2024-04-23T02:14:02.272 INFO:tasks.workunit.client.1.smithi192.stdout:9/507: rename d0/d9/d44/d74/d3c/d5f/d7d to d0/d9/d44/d51/d5b/d61/db1 0 2024-04-23T02:14:02.272 INFO:tasks.workunit.client.0.smithi071.stdout:4/580: dread d2/d2a/d2d/d39/d42/d49/d67/f1e [0,4194304] 0 2024-04-23T02:14:02.273 INFO:tasks.workunit.client.1.smithi192.stdout:5/583: creat d2/db/d16/d1b/fc6 x:0 0 0 2024-04-23T02:14:02.274 INFO:tasks.workunit.client.0.smithi071.stdout:5/588: fsync d3/da/d1a/d1c/d46/d4e/f50 0 2024-04-23T02:14:02.276 INFO:tasks.workunit.client.1.smithi192.stdout:8/706: symlink d1/d12/d7d/dd2/lfb 0 2024-04-23T02:14:02.276 INFO:tasks.workunit.client.0.smithi071.stdout:4/581: creat d2/d53/fc0 x:0 0 0 2024-04-23T02:14:02.279 INFO:tasks.workunit.client.1.smithi192.stdout:3/514: rmdir d1/d3/d17/d1a/d7c/dab/d6c/d84/d5d/d98 0 2024-04-23T02:14:02.279 INFO:tasks.workunit.client.1.smithi192.stdout:8/707: chown d1/d12/d23/d4b/d73/d4c/d82 159882403 1 2024-04-23T02:14:02.279 INFO:tasks.workunit.client.1.smithi192.stdout:9/508: truncate d0/d9/d44/f49 1207441 0 2024-04-23T02:14:02.279 INFO:tasks.workunit.client.1.smithi192.stdout:5/584: creat d2/db/d16/d48/db7/fc7 x:0 0 0 2024-04-23T02:14:02.280 INFO:tasks.workunit.client.0.smithi071.stdout:2/490: rename d0/d2/d26/df/d30/l38 to d0/d2/d26/d73/l9a 0 2024-04-23T02:14:02.281 INFO:tasks.workunit.client.1.smithi192.stdout:1/343: creat dc/d16/f64 x:0 0 0 2024-04-23T02:14:02.285 INFO:tasks.workunit.client.0.smithi071.stdout:2/491: rename d0/d2/d26/d14/d1f/f92 to d0/d2/d26/d73/f9b 0 2024-04-23T02:14:02.285 INFO:tasks.workunit.client.1.smithi192.stdout:1/344: fdatasync dc/d14/d23/f50 0 2024-04-23T02:14:02.285 INFO:tasks.workunit.client.0.smithi071.stdout:5/589: dwrite d3/da/d1a/f2b [0,4194304] 0 2024-04-23T02:14:02.286 INFO:tasks.workunit.client.1.smithi192.stdout:5/585: fsync d2/db/f18 0 2024-04-23T02:14:02.288 INFO:tasks.workunit.client.0.smithi071.stdout:4/582: link d2/d2a/d2d/d39/d42/d49/d67/f22 d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d4f/d8b/fc1 0 2024-04-23T02:14:02.288 INFO:tasks.workunit.client.1.smithi192.stdout:8/708: dwrite d1/d12/d7d/fce [0,4194304] 0 2024-04-23T02:14:02.289 INFO:tasks.workunit.client.0.smithi071.stdout:2/492: mknod d0/d2/d26/d14/d1f/d36/d6a/c9c 0 2024-04-23T02:14:02.292 INFO:tasks.workunit.client.1.smithi192.stdout:0/677: sync 2024-04-23T02:14:02.292 INFO:tasks.workunit.client.1.smithi192.stdout:2/592: sync 2024-04-23T02:14:02.293 INFO:tasks.workunit.client.1.smithi192.stdout:1/345: mknod dc/d14/d15/c65 0 2024-04-23T02:14:02.294 INFO:tasks.workunit.client.1.smithi192.stdout:4/688: sync 2024-04-23T02:14:02.296 INFO:tasks.workunit.client.1.smithi192.stdout:1/346: fdatasync f3 0 2024-04-23T02:14:02.296 INFO:tasks.workunit.client.1.smithi192.stdout:7/579: truncate d5/d17/d1c/d59/d5a/d70/f7e 4097985 0 2024-04-23T02:14:02.298 INFO:tasks.workunit.client.1.smithi192.stdout:8/709: chown d1/d9/fa 107 1 2024-04-23T02:14:02.300 INFO:tasks.workunit.client.1.smithi192.stdout:8/710: chown d1/l5b 529 1 2024-04-23T02:14:02.301 INFO:tasks.workunit.client.0.smithi071.stdout:2/493: dread d0/d70/f94 [0,4194304] 0 2024-04-23T02:14:02.302 INFO:tasks.workunit.client.0.smithi071.stdout:4/583: dwrite d2/d2a/d2d/d39/fa6 [0,4194304] 0 2024-04-23T02:14:02.305 INFO:tasks.workunit.client.0.smithi071.stdout:2/494: dread d0/d2/d26/d14/d1f/d4f/f66 [0,4194304] 0 2024-04-23T02:14:02.305 INFO:tasks.workunit.client.1.smithi192.stdout:5/586: dwrite d2/db/d2b/fa5 [0,4194304] 0 2024-04-23T02:14:02.307 INFO:tasks.workunit.client.1.smithi192.stdout:6/661: truncate d3/d18/f94 4184622 0 2024-04-23T02:14:02.309 INFO:tasks.workunit.client.0.smithi071.stdout:2/495: read - d0/d2/d26/d14/d1f/d36/d6a/f6e zero size 2024-04-23T02:14:02.309 INFO:tasks.workunit.client.1.smithi192.stdout:6/662: read - d3/d15/fb6 zero size 2024-04-23T02:14:02.311 INFO:tasks.workunit.client.1.smithi192.stdout:0/678: dwrite d6/df/d13/d25/d3a/f58 [8388608,4194304] 0 2024-04-23T02:14:02.312 INFO:tasks.workunit.client.1.smithi192.stdout:2/593: rename d8/d26/d29/d43/d50/fa9 to d8/daf/fbb 0 2024-04-23T02:14:02.313 INFO:tasks.workunit.client.1.smithi192.stdout:9/509: truncate d0/f4e 19498 0 2024-04-23T02:14:02.314 INFO:tasks.workunit.client.1.smithi192.stdout:7/580: rmdir d5/d43/d4c/d5c/d67/da6 39 2024-04-23T02:14:02.314 INFO:tasks.workunit.client.1.smithi192.stdout:0/679: stat d6/df/d56/l7f 0 2024-04-23T02:14:02.316 INFO:tasks.workunit.client.1.smithi192.stdout:8/711: truncate d1/df/f9b 179937 0 2024-04-23T02:14:02.316 INFO:tasks.workunit.client.0.smithi071.stdout:2/496: dwrite d0/d2/d26/df/d30/d80/f5f [0,4194304] 0 2024-04-23T02:14:02.317 INFO:tasks.workunit.client.1.smithi192.stdout:8/712: chown d1/f72 11714 1 2024-04-23T02:14:02.319 INFO:tasks.workunit.client.1.smithi192.stdout:4/689: mkdir de/d15/d53/d81/de3/d74/d92/da0/dca/de5 0 2024-04-23T02:14:02.321 INFO:tasks.workunit.client.0.smithi071.stdout:2/497: chown d0/d2/d26/d3a/d75 2 1 2024-04-23T02:14:02.322 INFO:tasks.workunit.client.0.smithi071.stdout:2/498: chown d0/d2/d26/d14/f1c 1964894 1 2024-04-23T02:14:02.322 INFO:tasks.workunit.client.1.smithi192.stdout:6/663: mkdir d3/d15/d1b/d21/d3a/d51/ddc 0 2024-04-23T02:14:02.323 INFO:tasks.workunit.client.0.smithi071.stdout:4/584: write d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f59 [461155,73812] 0 2024-04-23T02:14:02.324 INFO:tasks.workunit.client.0.smithi071.stdout:4/585: write d2/d53/fa5 [332746,20278] 0 2024-04-23T02:14:02.326 INFO:tasks.workunit.client.1.smithi192.stdout:5/587: mkdir d2/db/d2b/dc8 0 2024-04-23T02:14:02.329 INFO:tasks.workunit.client.1.smithi192.stdout:9/510: dwrite d0/d9/d44/d74/f82 [0,4194304] 0 2024-04-23T02:14:02.330 INFO:tasks.workunit.client.1.smithi192.stdout:0/680: fsync d6/d4b/d9d/dc7/f80 0 2024-04-23T02:14:02.330 INFO:tasks.workunit.client.0.smithi071.stdout:5/590: dread d3/f22 [0,4194304] 0 2024-04-23T02:14:02.332 INFO:tasks.workunit.client.0.smithi071.stdout:2/499: dwrite d0/d2/f90 [0,4194304] 0 2024-04-23T02:14:02.332 INFO:tasks.workunit.client.1.smithi192.stdout:8/713: mknod d1/d9/d3e/d59/cfc 0 2024-04-23T02:14:02.333 INFO:tasks.workunit.client.1.smithi192.stdout:0/681: fdatasync d6/df/d56/d85/d71/f84 0 2024-04-23T02:14:02.335 INFO:tasks.workunit.client.1.smithi192.stdout:2/594: mkdir d8/d26/d29/da4/dbc 0 2024-04-23T02:14:02.335 INFO:tasks.workunit.client.1.smithi192.stdout:2/595: chown d8/d14/l21 210 1 2024-04-23T02:14:02.337 INFO:tasks.workunit.client.0.smithi071.stdout:2/500: creat d0/d82/f9d x:0 0 0 2024-04-23T02:14:02.341 INFO:tasks.workunit.client.0.smithi071.stdout:2/501: mknod d0/d2/d34/c9e 0 2024-04-23T02:14:02.342 INFO:tasks.workunit.client.1.smithi192.stdout:5/588: fsync d2/d32/f6e 0 2024-04-23T02:14:02.346 INFO:tasks.workunit.client.0.smithi071.stdout:2/502: rename d0/d2/c8 to d0/d2/d26/d3a/c9f 0 2024-04-23T02:14:02.347 INFO:tasks.workunit.client.0.smithi071.stdout:2/503: read d0/d2/d26/df/f40 [839877,47330] 0 2024-04-23T02:14:02.348 INFO:tasks.workunit.client.0.smithi071.stdout:2/504: fdatasync d0/d2/d26/df/f40 0 2024-04-23T02:14:02.349 INFO:tasks.workunit.client.1.smithi192.stdout:0/682: mknod d6/df/d13/d25/d3a/da6/ce3 0 2024-04-23T02:14:02.350 INFO:tasks.workunit.client.1.smithi192.stdout:4/690: mkdir de/d15/d53/d81/de3/d74/d92/da0/dca/d49/de6 0 2024-04-23T02:14:02.350 INFO:tasks.workunit.client.1.smithi192.stdout:0/683: chown d6/df/d13/d25/d3a/d93/dd6 833574 1 2024-04-23T02:14:02.350 INFO:tasks.workunit.client.1.smithi192.stdout:1/347: dread dc/d16/d19/f63 [4194304,4194304] 0 2024-04-23T02:14:02.350 INFO:tasks.workunit.client.1.smithi192.stdout:2/596: mkdir d8/d26/d29/d43/d5d/dbd 0 2024-04-23T02:14:02.350 INFO:tasks.workunit.client.1.smithi192.stdout:0/684: read d6/df/d13/d25/f3d [430896,3630] 0 2024-04-23T02:14:02.352 INFO:tasks.workunit.client.0.smithi071.stdout:2/505: unlink d0/d2/d45/d58/f7c 0 2024-04-23T02:14:02.353 INFO:tasks.workunit.client.1.smithi192.stdout:9/511: unlink d0/d9/d44/d4f/f1c 0 2024-04-23T02:14:02.354 INFO:tasks.workunit.client.0.smithi071.stdout:2/506: mkdir d0/d2/da0 0 2024-04-23T02:14:02.356 INFO:tasks.workunit.client.1.smithi192.stdout:6/664: creat d3/d15/d1b/d66/d80/fdd x:0 0 0 2024-04-23T02:14:02.356 INFO:tasks.workunit.client.0.smithi071.stdout:2/507: mkdir d0/d2/d26/d14/d1f/d36/d6a/da1 0 2024-04-23T02:14:02.359 INFO:tasks.workunit.client.0.smithi071.stdout:2/508: creat d0/d2/d26/df/d30/fa2 x:0 0 0 2024-04-23T02:14:02.360 INFO:tasks.workunit.client.1.smithi192.stdout:0/685: stat d6/df/d56/d85/l98 0 2024-04-23T02:14:02.361 INFO:tasks.workunit.client.1.smithi192.stdout:0/686: chown d6/d4b/d9d 129197274 1 2024-04-23T02:14:02.361 INFO:tasks.workunit.client.0.smithi071.stdout:4/586: dread d2/d2a/d2d/d39/d42/d49/d67/f22 [0,4194304] 0 2024-04-23T02:14:02.363 INFO:tasks.workunit.client.1.smithi192.stdout:2/597: dwrite d8/daf/fbb [0,4194304] 0 2024-04-23T02:14:02.364 INFO:tasks.workunit.client.1.smithi192.stdout:0/687: fsync d6/df/d13/d25/d94/fb3 0 2024-04-23T02:14:02.364 INFO:tasks.workunit.client.1.smithi192.stdout:1/348: symlink dc/d14/d23/d5e/l66 0 2024-04-23T02:14:02.366 INFO:tasks.workunit.client.1.smithi192.stdout:4/691: mkdir de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/de7 0 2024-04-23T02:14:02.368 INFO:tasks.workunit.client.0.smithi071.stdout:5/591: dread d3/f1f [4194304,4194304] 0 2024-04-23T02:14:02.368 INFO:tasks.workunit.client.0.smithi071.stdout:4/587: dwrite d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d29/f9e [0,4194304] 0 2024-04-23T02:14:02.370 INFO:tasks.workunit.client.0.smithi071.stdout:5/592: write d3/d1d/daf/d10/da5/d5f/fd1 [233673,95209] 0 2024-04-23T02:14:02.372 INFO:tasks.workunit.client.1.smithi192.stdout:5/589: link d2/f27 d2/d32/fc9 0 2024-04-23T02:14:02.372 INFO:tasks.workunit.client.0.smithi071.stdout:4/588: truncate d2/d2a/d2d/d39/d42/d49/d91/f92 907786 0 2024-04-23T02:14:02.373 INFO:tasks.workunit.client.0.smithi071.stdout:5/593: read d3/d1d/d43/dd2/fda [109882,121831] 0 2024-04-23T02:14:02.375 INFO:tasks.workunit.client.0.smithi071.stdout:5/594: write d3/da/f4d [729617,86135] 0 2024-04-23T02:14:02.375 INFO:tasks.workunit.client.0.smithi071.stdout:4/589: read d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c/f7c [343283,99337] 0 2024-04-23T02:14:02.377 INFO:tasks.workunit.client.0.smithi071.stdout:5/595: dread - d3/d1d/daf/d10/da5/f94 zero size 2024-04-23T02:14:02.378 INFO:tasks.workunit.client.0.smithi071.stdout:5/596: stat d3/d1d/f20 0 2024-04-23T02:14:02.378 INFO:tasks.workunit.client.1.smithi192.stdout:2/598: rename d8/d25/fad to d8/d26/fbe 0 2024-04-23T02:14:02.382 INFO:tasks.workunit.client.1.smithi192.stdout:5/590: unlink d2/db/d16/d48/db7/fc7 0 2024-04-23T02:14:02.384 INFO:tasks.workunit.client.0.smithi071.stdout:5/597: dwrite d3/da/fcf [0,4194304] 0 2024-04-23T02:14:02.386 INFO:tasks.workunit.client.0.smithi071.stdout:5/598: write d3/d1d/d43/d4b/fe0 [246829,51408] 0 2024-04-23T02:14:02.386 INFO:tasks.workunit.client.1.smithi192.stdout:1/349: rename dc/d14/d23/l2b to dc/d14/d15/d35/l67 0 2024-04-23T02:14:02.387 INFO:tasks.workunit.client.1.smithi192.stdout:7/581: dwrite d5/d17/f9e [0,4194304] 0 2024-04-23T02:14:02.388 INFO:tasks.workunit.client.0.smithi071.stdout:5/599: mknod d3/d1d/d43/d4b/ce3 0 2024-04-23T02:14:02.390 INFO:tasks.workunit.client.0.smithi071.stdout:5/600: write d3/da/d1a/d1c/f41 [535456,76930] 0 2024-04-23T02:14:02.394 INFO:tasks.workunit.client.0.smithi071.stdout:5/601: rename d3/d1d/d43/dd2/d5a/da7/da8 to d3/d34/d6b/de4 0 2024-04-23T02:14:02.398 INFO:tasks.workunit.client.1.smithi192.stdout:4/692: link de/d15/d53/d81/de3/db2/f86 de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/fe8 0 2024-04-23T02:14:02.399 INFO:tasks.workunit.client.0.smithi071.stdout:4/590: truncate d2/d2a/f80 1338323 0 2024-04-23T02:14:02.402 INFO:tasks.workunit.client.0.smithi071.stdout:4/591: stat d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d4f/d8b 0 2024-04-23T02:14:02.403 INFO:tasks.workunit.client.0.smithi071.stdout:2/509: dwrite d0/d2/d26/d14/d1f/d4f/f66 [0,4194304] 0 2024-04-23T02:14:02.405 INFO:tasks.workunit.client.1.smithi192.stdout:8/714: dwrite d1/d9/fee [0,4194304] 0 2024-04-23T02:14:02.405 INFO:tasks.workunit.client.1.smithi192.stdout:0/688: dread d6/df/d47/d55/f70 [4194304,4194304] 0 2024-04-23T02:14:02.405 INFO:tasks.workunit.client.1.smithi192.stdout:6/665: dread d3/d15/d1b/f72 [0,4194304] 0 2024-04-23T02:14:02.407 INFO:tasks.workunit.client.1.smithi192.stdout:4/693: dread - de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/fd0 zero size 2024-04-23T02:14:02.409 INFO:tasks.workunit.client.0.smithi071.stdout:2/510: creat d0/d2/fa3 x:0 0 0 2024-04-23T02:14:02.413 INFO:tasks.workunit.client.0.smithi071.stdout:4/592: dwrite d2/d2a/d2d/d39/d42/d49/d67/d7e/f81 [0,4194304] 0 2024-04-23T02:14:02.415 INFO:tasks.workunit.client.0.smithi071.stdout:2/511: creat d0/d2/d34/d83/fa4 x:0 0 0 2024-04-23T02:14:02.416 INFO:tasks.workunit.client.1.smithi192.stdout:5/591: rmdir d2 39 2024-04-23T02:14:02.418 INFO:tasks.workunit.client.0.smithi071.stdout:2/512: symlink d0/d2/d26/d14/la5 0 2024-04-23T02:14:02.421 INFO:tasks.workunit.client.0.smithi071.stdout:2/513: read d0/d2/d26/d3a/f3d [2134082,50722] 0 2024-04-23T02:14:02.421 INFO:tasks.workunit.client.0.smithi071.stdout:2/514: chown d0/d82 222836292 1 2024-04-23T02:14:02.424 INFO:tasks.workunit.client.0.smithi071.stdout:4/593: getdents d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d4f/db5/db6 0 2024-04-23T02:14:02.426 INFO:tasks.workunit.client.0.smithi071.stdout:2/515: rename d0/d2/d34/f91 to d0/d2/d26/d14/d1f/d4f/d98/fa6 0 2024-04-23T02:14:02.428 INFO:tasks.workunit.client.0.smithi071.stdout:4/594: mkdir d2/d2a/d2d/d39/d42/d49/d67/d7e/dc2 0 2024-04-23T02:14:02.430 INFO:tasks.workunit.client.1.smithi192.stdout:1/350: mknod dc/d16/d19/c68 0 2024-04-23T02:14:02.430 INFO:tasks.workunit.client.0.smithi071.stdout:4/595: chown d2/d2a/f3b 1154085 1 2024-04-23T02:14:02.431 INFO:tasks.workunit.client.0.smithi071.stdout:2/516: creat d0/d2/d26/d14/fa7 x:0 0 0 2024-04-23T02:14:02.431 INFO:tasks.workunit.client.1.smithi192.stdout:2/599: link d8/d26/d74/d7c/f9b d8/d26/d29/d43/d50/fbf 0 2024-04-23T02:14:02.433 INFO:tasks.workunit.client.0.smithi071.stdout:5/602: dwrite d3/d34/d6b/fa3 [0,4194304] 0 2024-04-23T02:14:02.433 INFO:tasks.workunit.client.0.smithi071.stdout:2/517: dread - d0/d2/d26/df/f3b zero size 2024-04-23T02:14:02.434 INFO:tasks.workunit.client.0.smithi071.stdout:4/596: chown d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/ff 91441 1 2024-04-23T02:14:02.435 INFO:tasks.workunit.client.1.smithi192.stdout:8/715: unlink d1/d12/d23/d4b/d73/d4c/d82/fef 0 2024-04-23T02:14:02.436 INFO:tasks.workunit.client.0.smithi071.stdout:2/518: write d0/d2/d26/f32 [2056154,2424] 0 2024-04-23T02:14:02.436 INFO:tasks.workunit.client.1.smithi192.stdout:6/666: readlink d3/d28/d79/da0/lc0 0 2024-04-23T02:14:02.437 INFO:tasks.workunit.client.1.smithi192.stdout:4/694: rmdir de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/d59 39 2024-04-23T02:14:02.438 INFO:tasks.workunit.client.0.smithi071.stdout:5/603: symlink d3/d1d/daf/d10/da5/d5f/dc0/le5 0 2024-04-23T02:14:02.441 INFO:tasks.workunit.client.0.smithi071.stdout:5/604: mkdir d3/d8f/de6 0 2024-04-23T02:14:02.441 INFO:tasks.workunit.client.0.smithi071.stdout:4/597: dread d2/d2a/d2d/d39/d42/d49/d67/d1d/d57/f72 [0,4194304] 0 2024-04-23T02:14:02.442 INFO:tasks.workunit.client.0.smithi071.stdout:5/605: dread - d3/d1d/daf/d83/fac zero size 2024-04-23T02:14:02.445 INFO:tasks.workunit.client.1.smithi192.stdout:8/716: creat d1/d12/d23/d4b/d73/d45/d46/ffd x:0 0 0 2024-04-23T02:14:02.445 INFO:tasks.workunit.client.1.smithi192.stdout:4/695: getdents de/d15/d53/d81/de3/da3 0 2024-04-23T02:14:02.445 INFO:tasks.workunit.client.1.smithi192.stdout:6/667: truncate d3/d15/d1b/d66/d80/f8b 959647 0 2024-04-23T02:14:02.447 INFO:tasks.workunit.client.1.smithi192.stdout:2/600: getdents d8/d26/d74/d7c 0 2024-04-23T02:14:02.449 INFO:tasks.workunit.client.1.smithi192.stdout:6/668: chown d3/d18/d58/fcd 30483681 1 2024-04-23T02:14:02.451 INFO:tasks.workunit.client.1.smithi192.stdout:2/601: creat d8/d26/d27/d53/da2/fc0 x:0 0 0 2024-04-23T02:14:02.453 INFO:tasks.workunit.client.1.smithi192.stdout:6/669: symlink d3/d15/d1b/lde 0 2024-04-23T02:14:02.454 INFO:tasks.workunit.client.1.smithi192.stdout:6/670: write d3/d15/d1b/f8f [265511,123356] 0 2024-04-23T02:14:02.455 INFO:tasks.workunit.client.1.smithi192.stdout:4/696: link de/d15/d53/d81/de3/d74/d92/da0/dca/f5f de/d15/d53/d81/de3/d74/d92/da0/dca/de5/fe9 0 2024-04-23T02:14:02.457 INFO:tasks.workunit.client.1.smithi192.stdout:2/602: rename d8/daf to d8/d26/d27/d53/da2/dc1 0 2024-04-23T02:14:02.458 INFO:tasks.workunit.client.1.smithi192.stdout:7/582: dread d5/d17/d1c/d59/d5a/d41/d50/f35 [4194304,4194304] 0 2024-04-23T02:14:02.458 INFO:tasks.workunit.client.1.smithi192.stdout:6/671: write d3/d15/d1b/d66/d80/d8e/faf [447317,82035] 0 2024-04-23T02:14:02.461 INFO:tasks.workunit.client.0.smithi071.stdout:4/598: rename d2/d2a/cb7 to d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d4f/cc3 0 2024-04-23T02:14:02.461 INFO:tasks.workunit.client.1.smithi192.stdout:8/717: dread d1/d12/f1e [4194304,4194304] 0 2024-04-23T02:14:02.461 INFO:tasks.workunit.client.1.smithi192.stdout:6/672: fdatasync d3/d28/d79/fa4 0 2024-04-23T02:14:02.466 INFO:tasks.workunit.client.1.smithi192.stdout:2/603: mkdir d8/d26/d29/d43/dc2 0 2024-04-23T02:14:02.466 INFO:tasks.workunit.client.1.smithi192.stdout:7/583: write d5/d17/d1c/d59/d5a/d41/d50/d9f/fa4 [258484,17670] 0 2024-04-23T02:14:02.467 INFO:tasks.workunit.client.0.smithi071.stdout:5/606: dread d3/d1d/daf/d10/f48 [0,4194304] 0 2024-04-23T02:14:02.467 INFO:tasks.workunit.client.0.smithi071.stdout:4/599: dread d2/d2a/d2d/f54 [0,4194304] 0 2024-04-23T02:14:02.467 INFO:tasks.workunit.client.0.smithi071.stdout:5/607: chown d3/da/d1a/d1c/c52 3662649 1 2024-04-23T02:14:02.467 INFO:tasks.workunit.client.0.smithi071.stdout:4/600: chown d2/d2a/daa/l62 1339439098 1 2024-04-23T02:14:02.467 INFO:tasks.workunit.client.0.smithi071.stdout:5/608: readlink d3/d86/lb1 0 2024-04-23T02:14:02.467 INFO:tasks.workunit.client.0.smithi071.stdout:5/609: chown d3/f22 43298 1 2024-04-23T02:14:02.467 INFO:tasks.workunit.client.1.smithi192.stdout:6/673: mkdir d3/d15/d1b/dcf/ddf 0 2024-04-23T02:14:02.469 INFO:tasks.workunit.client.1.smithi192.stdout:6/674: chown d3/d15/d1b/d21/d3a/d51/f93 32 1 2024-04-23T02:14:02.470 INFO:tasks.workunit.client.1.smithi192.stdout:8/718: rename d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/fd9 to d1/d12/d68/ddf/ffe 0 2024-04-23T02:14:02.473 INFO:tasks.workunit.client.1.smithi192.stdout:5/592: truncate d2/db/d16/f96 469814 0 2024-04-23T02:14:02.475 INFO:tasks.workunit.client.1.smithi192.stdout:8/719: rmdir d1/d9/d3e 39 2024-04-23T02:14:02.476 INFO:tasks.workunit.client.1.smithi192.stdout:6/675: symlink d3/d15/d1b/d66/d80/d8e/le0 0 2024-04-23T02:14:02.477 INFO:tasks.workunit.client.1.smithi192.stdout:7/584: dwrite d5/d17/d1c/d59/d5a/d41/f54 [0,4194304] 0 2024-04-23T02:14:02.479 INFO:tasks.workunit.client.1.smithi192.stdout:8/720: chown d1/d12/d23/fd3 4767 1 2024-04-23T02:14:02.479 INFO:tasks.workunit.client.1.smithi192.stdout:2/604: dwrite d8/d26/fbe [0,4194304] 0 2024-04-23T02:14:02.480 INFO:tasks.workunit.client.1.smithi192.stdout:5/593: getdents d2/db/d16/d48/dc5 0 2024-04-23T02:14:02.482 INFO:tasks.workunit.client.1.smithi192.stdout:7/585: write d5/d43/f99 [285877,36493] 0 2024-04-23T02:14:02.487 INFO:tasks.workunit.client.1.smithi192.stdout:4/697: write de/d15/d19/d1a/f23 [4101817,121738] 0 2024-04-23T02:14:02.490 INFO:tasks.workunit.client.1.smithi192.stdout:1/351: dwrite dc/d14/d15/d35/f37 [0,4194304] 0 2024-04-23T02:14:02.490 INFO:tasks.workunit.client.1.smithi192.stdout:6/676: symlink d3/d15/d1b/d21/d3a/d51/le1 0 2024-04-23T02:14:02.497 INFO:tasks.workunit.client.1.smithi192.stdout:4/698: truncate de/d15/d53/d81/de3/d74/d92/da0/dca/d34/d4a/fc4 377518 0 2024-04-23T02:14:02.501 INFO:tasks.workunit.client.1.smithi192.stdout:7/586: getdents d5/d17/d1c/d59/d5a 0 2024-04-23T02:14:02.502 INFO:tasks.workunit.client.1.smithi192.stdout:4/699: mknod de/d15/d53/d81/de3/d74/cea 0 2024-04-23T02:14:02.507 INFO:tasks.workunit.client.1.smithi192.stdout:4/700: mkdir de/d15/d53/d81/de3/d74/d92/da0/dca/d9f/deb 0 2024-04-23T02:14:02.509 INFO:tasks.workunit.client.1.smithi192.stdout:1/352: dwrite f1 [8388608,4194304] 0 2024-04-23T02:14:02.510 INFO:tasks.workunit.client.1.smithi192.stdout:2/605: dwrite d8/d25/f6c [0,4194304] 0 2024-04-23T02:14:02.513 INFO:tasks.workunit.client.1.smithi192.stdout:4/701: chown de/d15/d19/l3e 392 1 2024-04-23T02:14:02.518 INFO:tasks.workunit.client.1.smithi192.stdout:7/587: write d5/d17/d1c/f74 [8912081,76956] 0 2024-04-23T02:14:02.518 INFO:tasks.workunit.client.1.smithi192.stdout:4/702: write de/f21 [1645193,54431] 0 2024-04-23T02:14:02.521 INFO:tasks.workunit.client.1.smithi192.stdout:1/353: truncate dc/d16/f1f 2348185 0 2024-04-23T02:14:02.521 INFO:tasks.workunit.client.1.smithi192.stdout:2/606: getdents d8/d26/d29/da4/da8/dba 0 2024-04-23T02:14:02.524 INFO:tasks.workunit.client.1.smithi192.stdout:7/588: getdents d5/d17/d1c/d59/d5a/d9b 0 2024-04-23T02:14:02.526 INFO:tasks.workunit.client.1.smithi192.stdout:2/607: creat d8/fc3 x:0 0 0 2024-04-23T02:14:02.527 INFO:tasks.workunit.client.1.smithi192.stdout:4/703: write de/d15/d53/d81/de3/d74/d92/da0/dca/f57 [3291703,25046] 0 2024-04-23T02:14:02.527 INFO:tasks.workunit.client.1.smithi192.stdout:5/594: dread d2/db/d16/d1b/f54 [0,4194304] 0 2024-04-23T02:14:02.528 INFO:tasks.workunit.client.1.smithi192.stdout:7/589: mkdir d5/d17/d1c/d4a/daa 0 2024-04-23T02:14:02.528 INFO:tasks.workunit.client.1.smithi192.stdout:2/608: chown d8/d26/d27/db5 2 1 2024-04-23T02:14:02.529 INFO:tasks.workunit.client.1.smithi192.stdout:8/721: dread d1/f10 [0,4194304] 0 2024-04-23T02:14:02.529 INFO:tasks.workunit.client.1.smithi192.stdout:1/354: dread dc/d16/d19/d30/f55 [0,4194304] 0 2024-04-23T02:14:02.529 INFO:tasks.workunit.client.1.smithi192.stdout:5/595: chown d2/cbb 31664 1 2024-04-23T02:14:02.532 INFO:tasks.workunit.client.1.smithi192.stdout:2/609: symlink d8/d26/d81/lc4 0 2024-04-23T02:14:02.534 INFO:tasks.workunit.client.1.smithi192.stdout:8/722: creat d1/d9/d3e/d59/fff x:0 0 0 2024-04-23T02:14:02.537 INFO:tasks.workunit.client.1.smithi192.stdout:5/596: unlink d2/db/d16/d1b/f54 0 2024-04-23T02:14:02.537 INFO:tasks.workunit.client.1.smithi192.stdout:8/723: truncate d1/d12/d23/d4b/d73/d7e/d88/fa4 1679139 0 2024-04-23T02:14:02.537 INFO:tasks.workunit.client.1.smithi192.stdout:5/597: read - d2/d32/f9c zero size 2024-04-23T02:14:02.541 INFO:tasks.workunit.client.0.smithi071.stdout:9/496: sync 2024-04-23T02:14:02.541 INFO:tasks.workunit.client.1.smithi192.stdout:2/610: chown d8/d26/d29/d43/l44 8226 1 2024-04-23T02:14:02.541 INFO:tasks.workunit.client.0.smithi071.stdout:9/497: chown d1/d1a/f22 4276 1 2024-04-23T02:14:02.542 INFO:tasks.workunit.client.1.smithi192.stdout:7/590: creat d5/d17/d1c/d59/d5a/d41/fab x:0 0 0 2024-04-23T02:14:02.543 INFO:tasks.workunit.client.0.smithi071.stdout:9/498: mkdir d1/d4e/d66/da8 0 2024-04-23T02:14:02.544 INFO:tasks.workunit.client.1.smithi192.stdout:7/591: truncate d5/d17/d1c/d59/d5a/fa2 776797 0 2024-04-23T02:14:02.545 INFO:tasks.workunit.client.1.smithi192.stdout:7/592: write d5/d17/d1c/d57/f90 [965318,100791] 0 2024-04-23T02:14:02.545 INFO:tasks.workunit.client.0.smithi071.stdout:9/499: mkdir d1/d4e/d54/d75/da6/da9 0 2024-04-23T02:14:02.546 INFO:tasks.workunit.client.1.smithi192.stdout:5/598: unlink d2/db/d16/d48/d4f/ca0 0 2024-04-23T02:14:02.546 INFO:tasks.workunit.client.1.smithi192.stdout:5/599: stat d2/c6c 0 2024-04-23T02:14:02.547 INFO:tasks.workunit.client.1.smithi192.stdout:7/593: fdatasync d5/d17/d1c/d59/d5a/d41/f98 0 2024-04-23T02:14:02.548 INFO:tasks.workunit.client.1.smithi192.stdout:4/704: write de/d15/d53/d81/de3/d74/d92/da0/dca/d49/f50 [1968762,15513] 0 2024-04-23T02:14:02.549 INFO:tasks.workunit.client.1.smithi192.stdout:2/611: creat d8/d26/d74/fc5 x:0 0 0 2024-04-23T02:14:02.551 INFO:tasks.workunit.client.1.smithi192.stdout:5/600: readlink d2/d6/d14/d60/l75 0 2024-04-23T02:14:02.553 INFO:tasks.workunit.client.1.smithi192.stdout:1/355: rename dc/d14/d23/l42 to dc/d14/l69 0 2024-04-23T02:14:02.553 INFO:tasks.workunit.client.1.smithi192.stdout:7/594: mknod d5/cac 0 2024-04-23T02:14:02.556 INFO:tasks.workunit.client.1.smithi192.stdout:8/724: getdents d1/d12/d7d/dd2 0 2024-04-23T02:14:02.556 INFO:tasks.workunit.client.1.smithi192.stdout:2/612: creat d8/d26/d27/d57/da0/fc6 x:0 0 0 2024-04-23T02:14:02.560 INFO:tasks.workunit.client.1.smithi192.stdout:7/595: symlink d5/d43/d4c/d5c/d97/lad 0 2024-04-23T02:14:02.561 INFO:tasks.workunit.client.1.smithi192.stdout:5/601: dwrite d2/d32/f9c [0,4194304] 0 2024-04-23T02:14:02.562 INFO:tasks.workunit.client.1.smithi192.stdout:2/613: creat d8/d26/d29/d80/fc7 x:0 0 0 2024-04-23T02:14:02.563 INFO:tasks.workunit.client.1.smithi192.stdout:7/596: readlink d5/d43/d4c/d4e/l5b 0 2024-04-23T02:14:02.569 INFO:tasks.workunit.client.1.smithi192.stdout:8/725: symlink d1/d12/d23/d4b/d73/d7e/d88/l100 0 2024-04-23T02:14:02.570 INFO:tasks.workunit.client.1.smithi192.stdout:1/356: unlink dc/d16/d19/d30/d32/d34/f3e 0 2024-04-23T02:14:02.572 INFO:tasks.workunit.client.1.smithi192.stdout:1/357: chown dc/d14/f21 36747505 1 2024-04-23T02:14:02.573 INFO:tasks.workunit.client.1.smithi192.stdout:2/614: dwrite d8/d26/d29/d80/f95 [0,4194304] 0 2024-04-23T02:14:02.573 INFO:tasks.workunit.client.1.smithi192.stdout:5/602: rename d2/db/d2b/d52/f99 to d2/d6/d2d/d41/d43/d98/fca 0 2024-04-23T02:14:02.576 INFO:tasks.workunit.client.1.smithi192.stdout:7/597: creat d5/d17/d1c/d59/d5a/da0/fae x:0 0 0 2024-04-23T02:14:02.577 INFO:tasks.workunit.client.1.smithi192.stdout:2/615: rmdir d8/d26 39 2024-04-23T02:14:02.580 INFO:tasks.workunit.client.1.smithi192.stdout:2/616: chown d8/d14/f1a 28038727 1 2024-04-23T02:14:02.580 INFO:tasks.workunit.client.1.smithi192.stdout:7/598: write d5/d17/d1c/d59/f9a [781565,19047] 0 2024-04-23T02:14:02.580 INFO:tasks.workunit.client.1.smithi192.stdout:1/358: rename dc/d51 to dc/d14/d23/d5e/d6a 0 2024-04-23T02:14:02.584 INFO:tasks.workunit.client.1.smithi192.stdout:1/359: write dc/d16/d19/d30/d32/d44/f53 [285982,6109] 0 2024-04-23T02:14:02.584 INFO:tasks.workunit.client.1.smithi192.stdout:4/705: dwrite de/d15/d53/d81/de3/d74/d92/da0/dca/f32 [0,4194304] 0 2024-04-23T02:14:02.586 INFO:tasks.workunit.client.1.smithi192.stdout:8/726: rename d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d8b/lb7 to d1/d12/d7d/l101 0 2024-04-23T02:14:02.587 INFO:tasks.workunit.client.1.smithi192.stdout:5/603: creat d2/d6/fcb x:0 0 0 2024-04-23T02:14:02.588 INFO:tasks.workunit.client.1.smithi192.stdout:7/599: unlink d5/d17/d1c/d59/d5a/da0/fae 0 2024-04-23T02:14:02.589 INFO:tasks.workunit.client.1.smithi192.stdout:5/604: chown d2/d2e/d4e/l53 1649466148 1 2024-04-23T02:14:02.597 INFO:tasks.workunit.client.1.smithi192.stdout:4/706: truncate de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/f8e 3603188 0 2024-04-23T02:14:02.598 INFO:tasks.workunit.client.1.smithi192.stdout:8/727: creat d1/d12/d23/d4b/d73/d7e/f102 x:0 0 0 2024-04-23T02:14:02.600 INFO:tasks.workunit.client.1.smithi192.stdout:4/707: unlink de/f8c 0 2024-04-23T02:14:02.602 INFO:tasks.workunit.client.1.smithi192.stdout:8/728: rename d1/d12/d23/d4b/d73/d4c/d82/fb6 to d1/d9/d8f/f103 0 2024-04-23T02:14:02.604 INFO:tasks.workunit.client.1.smithi192.stdout:4/708: dread - de/d15/d53/d81/de3/d74/d92/da0/dca/d34/f76 zero size 2024-04-23T02:14:02.605 INFO:tasks.workunit.client.1.smithi192.stdout:1/360: write dc/f29 [857061,20868] 0 2024-04-23T02:14:02.606 INFO:tasks.workunit.client.1.smithi192.stdout:4/709: stat de/f4d 0 2024-04-23T02:14:02.607 INFO:tasks.workunit.client.1.smithi192.stdout:8/729: unlink d1/d12/d23/d4b/d73/l9d 0 2024-04-23T02:14:02.609 INFO:tasks.workunit.client.1.smithi192.stdout:4/710: creat de/d15/d53/d81/de3/db2/fec x:0 0 0 2024-04-23T02:14:02.612 INFO:tasks.workunit.client.1.smithi192.stdout:1/361: rename dc/l1e to dc/l6b 0 2024-04-23T02:14:02.616 INFO:tasks.workunit.client.1.smithi192.stdout:4/711: write de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d93/fc6 [129643,66641] 0 2024-04-23T02:14:02.616 INFO:tasks.workunit.client.1.smithi192.stdout:1/362: mknod dc/d16/d19/d30/d32/c6c 0 2024-04-23T02:14:02.617 INFO:tasks.workunit.client.1.smithi192.stdout:8/730: dwrite d1/d9/d8f/fc0 [0,4194304] 0 2024-04-23T02:14:02.619 INFO:tasks.workunit.client.1.smithi192.stdout:4/712: dread - de/d15/d53/d81/dac/fd7 zero size 2024-04-23T02:14:02.619 INFO:tasks.workunit.client.1.smithi192.stdout:7/600: dread d5/d17/d1c/f2d [0,4194304] 0 2024-04-23T02:14:02.624 INFO:tasks.workunit.client.1.smithi192.stdout:3/515: sync 2024-04-23T02:14:02.626 INFO:tasks.workunit.client.1.smithi192.stdout:4/713: chown de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/l85 61554438 1 2024-04-23T02:14:02.627 INFO:tasks.workunit.client.1.smithi192.stdout:8/731: creat d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/f104 x:0 0 0 2024-04-23T02:14:02.629 INFO:tasks.workunit.client.1.smithi192.stdout:4/714: readlink de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/l3b 0 2024-04-23T02:14:02.629 INFO:tasks.workunit.client.1.smithi192.stdout:3/516: stat d1/d3/f49 0 2024-04-23T02:14:02.631 INFO:tasks.workunit.client.1.smithi192.stdout:1/363: rename dc/d16/l38 to dc/d16/d19/d30/l6d 0 2024-04-23T02:14:02.633 INFO:tasks.workunit.client.1.smithi192.stdout:7/601: link d5/d43/f99 d5/d43/d4c/d5c/d97/faf 0 2024-04-23T02:14:02.635 INFO:tasks.workunit.client.1.smithi192.stdout:1/364: chown dc/d16/d19/d30/d32/d44 2272 1 2024-04-23T02:14:02.642 INFO:tasks.workunit.client.1.smithi192.stdout:8/732: dwrite d1/d9/d8f/d90/fa2 [0,4194304] 0 2024-04-23T02:14:02.642 INFO:tasks.workunit.client.1.smithi192.stdout:3/517: dread d1/d3/d17/f6e [0,4194304] 0 2024-04-23T02:14:02.649 INFO:tasks.workunit.client.1.smithi192.stdout:8/733: dread d1/d12/d23/d4b/d73/d4c/d82/fb4 [0,4194304] 0 2024-04-23T02:14:02.651 INFO:tasks.workunit.client.0.smithi071.stdout:6/476: sync 2024-04-23T02:14:02.651 INFO:tasks.workunit.client.0.smithi071.stdout:3/471: sync 2024-04-23T02:14:02.651 INFO:tasks.workunit.client.0.smithi071.stdout:7/463: sync 2024-04-23T02:14:02.653 INFO:tasks.workunit.client.0.smithi071.stdout:6/477: fsync d6/f12 0 2024-04-23T02:14:02.654 INFO:tasks.workunit.client.0.smithi071.stdout:7/464: symlink d3/d17/d41/d37/la1 0 2024-04-23T02:14:02.658 INFO:tasks.workunit.client.0.smithi071.stdout:7/465: creat d3/d17/d41/d37/d4f/fa2 x:0 0 0 2024-04-23T02:14:02.659 INFO:tasks.workunit.client.0.smithi071.stdout:3/472: link d7/d8/d42/d47/c4d d7/d27/ca3 0 2024-04-23T02:14:02.661 INFO:tasks.workunit.client.0.smithi071.stdout:6/478: dwrite d6/d1f/f9a [0,4194304] 0 2024-04-23T02:14:02.665 INFO:tasks.workunit.client.0.smithi071.stdout:6/479: chown d6/d1f/d5d/l72 7167 1 2024-04-23T02:14:02.666 INFO:tasks.workunit.client.0.smithi071.stdout:3/473: mknod d7/d22/d23/d26/d7e/ca4 0 2024-04-23T02:14:02.667 INFO:tasks.workunit.client.0.smithi071.stdout:3/474: fsync d7/d8/d42/d5f/d68/f84 0 2024-04-23T02:14:02.669 INFO:tasks.workunit.client.0.smithi071.stdout:3/475: chown d7/d8/d42/d80/f89 111035351 1 2024-04-23T02:14:02.670 INFO:tasks.workunit.client.0.smithi071.stdout:3/476: fsync d7/d8/d42/d5f/f5e 0 2024-04-23T02:14:02.676 INFO:tasks.workunit.client.0.smithi071.stdout:7/466: link d3/d17/d41/d34/d59/d5f/f90 d3/d17/d41/d34/fa3 0 2024-04-23T02:14:02.678 INFO:tasks.workunit.client.0.smithi071.stdout:3/477: mkdir d7/da5 0 2024-04-23T02:14:02.679 INFO:tasks.workunit.client.0.smithi071.stdout:3/478: dread - d7/d8/d42/f87 zero size 2024-04-23T02:14:02.680 INFO:tasks.workunit.client.0.smithi071.stdout:7/467: symlink d3/d51/la4 0 2024-04-23T02:14:02.682 INFO:tasks.workunit.client.0.smithi071.stdout:7/468: dread - d3/d17/d41/d37/d4f/fa2 zero size 2024-04-23T02:14:02.682 INFO:tasks.workunit.client.0.smithi071.stdout:7/469: chown d3/f1d 922017839 1 2024-04-23T02:14:02.684 INFO:tasks.workunit.client.0.smithi071.stdout:7/470: write d3/d17/d41/d34/d39/d3b/f49 [111536,84479] 0 2024-04-23T02:14:02.685 INFO:tasks.workunit.client.0.smithi071.stdout:8/528: sync 2024-04-23T02:14:02.685 INFO:tasks.workunit.client.0.smithi071.stdout:2/519: sync 2024-04-23T02:14:02.685 INFO:tasks.workunit.client.0.smithi071.stdout:0/491: sync 2024-04-23T02:14:02.685 INFO:tasks.workunit.client.0.smithi071.stdout:1/498: sync 2024-04-23T02:14:02.686 INFO:tasks.workunit.client.0.smithi071.stdout:9/500: sync 2024-04-23T02:14:02.686 INFO:tasks.workunit.client.0.smithi071.stdout:4/601: sync 2024-04-23T02:14:02.686 INFO:tasks.workunit.client.0.smithi071.stdout:5/610: sync 2024-04-23T02:14:02.690 INFO:tasks.workunit.client.0.smithi071.stdout:5/611: read - d3/d1d/daf/d83/f9b zero size 2024-04-23T02:14:02.691 INFO:tasks.workunit.client.0.smithi071.stdout:9/501: stat d1/d4e/f4f 0 2024-04-23T02:14:02.691 INFO:tasks.workunit.client.1.smithi192.stdout:3/518: mkdir d1/d3/d17/d2b/d44/d57/db0 0 2024-04-23T02:14:02.692 INFO:tasks.workunit.client.1.smithi192.stdout:3/519: chown d1/d3/d17/d1a/f4f 2873 1 2024-04-23T02:14:02.693 INFO:tasks.workunit.client.1.smithi192.stdout:8/734: unlink d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/lcc 0 2024-04-23T02:14:02.693 INFO:tasks.workunit.client.0.smithi071.stdout:6/480: dwrite d6/d22/f35 [0,4194304] 0 2024-04-23T02:14:02.694 INFO:tasks.workunit.client.0.smithi071.stdout:7/471: rename d3/d74 to d3/d17/d41/d3a/da5 0 2024-04-23T02:14:02.694 INFO:tasks.workunit.client.1.smithi192.stdout:4/715: fdatasync de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/fe8 0 2024-04-23T02:14:02.694 INFO:tasks.workunit.client.0.smithi071.stdout:0/492: fdatasync d0/d3/de/d10/d14/f4d 0 2024-04-23T02:14:02.695 INFO:tasks.workunit.client.1.smithi192.stdout:8/735: write d1/d9/d3e/d59/fff [876773,33639] 0 2024-04-23T02:14:02.697 INFO:tasks.workunit.client.0.smithi071.stdout:1/499: fsync db/df/d13/d72/f79 0 2024-04-23T02:14:02.698 INFO:tasks.workunit.client.1.smithi192.stdout:7/602: creat d5/d17/d1c/d59/da8/fb0 x:0 0 0 2024-04-23T02:14:02.699 INFO:tasks.workunit.client.0.smithi071.stdout:4/602: write d2/d2a/d2d/d39/d42/d49/d67/f1e [2613189,37212] 0 2024-04-23T02:14:02.699 INFO:tasks.workunit.client.0.smithi071.stdout:5/612: mknod d3/d86/ce7 0 2024-04-23T02:14:02.702 INFO:tasks.workunit.client.1.smithi192.stdout:3/520: dread d1/d3/d17/d2b/f32 [0,4194304] 0 2024-04-23T02:14:02.704 INFO:tasks.workunit.client.1.smithi192.stdout:4/716: rmdir de/d15/d53/d81/de3/d80 39 2024-04-23T02:14:02.704 INFO:tasks.workunit.client.0.smithi071.stdout:7/472: stat d3/d17/d41/l7a 0 2024-04-23T02:14:02.706 INFO:tasks.workunit.client.0.smithi071.stdout:1/500: creat db/df/d13/d20/fa3 x:0 0 0 2024-04-23T02:14:02.706 INFO:tasks.workunit.client.1.smithi192.stdout:8/736: mknod d1/d12/d23/d4b/d73/d7e/d88/da9/db1/c105 0 2024-04-23T02:14:02.707 INFO:tasks.workunit.client.0.smithi071.stdout:2/520: rename d0/d2/d26/d73/l9a to d0/d2/d26/d73/la8 0 2024-04-23T02:14:02.708 INFO:tasks.workunit.client.0.smithi071.stdout:1/501: read fa [2956129,86490] 0 2024-04-23T02:14:02.709 INFO:tasks.workunit.client.0.smithi071.stdout:3/479: dread d7/d8/f1a [0,4194304] 0 2024-04-23T02:14:02.711 INFO:tasks.workunit.client.0.smithi071.stdout:5/613: symlink d3/d1d/d43/d4b/le8 0 2024-04-23T02:14:02.711 INFO:tasks.workunit.client.1.smithi192.stdout:7/603: read d5/d17/d1c/d59/d5a/d41/d50/f69 [1241630,120159] 0 2024-04-23T02:14:02.712 INFO:tasks.workunit.client.0.smithi071.stdout:1/502: read db/df/f58 [1758903,70070] 0 2024-04-23T02:14:02.712 INFO:tasks.workunit.client.0.smithi071.stdout:7/473: creat d3/d17/d1c/d2a/fa6 x:0 0 0 2024-04-23T02:14:02.714 INFO:tasks.workunit.client.0.smithi071.stdout:0/493: mknod d0/d3/de/d10/d14/d32/d44/d31/d52/c9f 0 2024-04-23T02:14:02.715 INFO:tasks.workunit.client.0.smithi071.stdout:1/503: fdatasync db/df/d13/d2f/f3c 0 2024-04-23T02:14:02.716 INFO:tasks.workunit.client.1.smithi192.stdout:3/521: symlink d1/d3/d6/d7/d29/d3b/da9/lb1 0 2024-04-23T02:14:02.716 INFO:tasks.workunit.client.0.smithi071.stdout:1/504: chown db/la1 2 1 2024-04-23T02:14:02.717 INFO:tasks.workunit.client.0.smithi071.stdout:6/481: write d6/d17/f2c [3252346,119340] 0 2024-04-23T02:14:02.720 INFO:tasks.workunit.client.1.smithi192.stdout:4/717: dwrite de/d15/d53/d81/de3/d97/fbf [0,4194304] 0 2024-04-23T02:14:02.721 INFO:tasks.workunit.client.0.smithi071.stdout:5/614: dread d3/d1d/f3b [0,4194304] 0 2024-04-23T02:14:02.722 INFO:tasks.workunit.client.1.smithi192.stdout:7/604: fdatasync d5/d17/d1c/d59/d5a/d41/d50/f35 0 2024-04-23T02:14:02.722 INFO:tasks.workunit.client.0.smithi071.stdout:9/502: write d1/d1a/d5a/f6b [3443468,11127] 0 2024-04-23T02:14:02.723 INFO:tasks.workunit.client.0.smithi071.stdout:9/503: read - d1/d4e/d66/f73 zero size 2024-04-23T02:14:02.723 INFO:tasks.workunit.client.0.smithi071.stdout:2/521: symlink d0/d2/d34/d55/d96/la9 0 2024-04-23T02:14:02.725 INFO:tasks.workunit.client.0.smithi071.stdout:6/482: rmdir d6/d17 39 2024-04-23T02:14:02.727 INFO:tasks.workunit.client.0.smithi071.stdout:3/480: symlink d7/d8/d42/d5f/d68/d3b/d98/la6 0 2024-04-23T02:14:02.728 INFO:tasks.workunit.client.0.smithi071.stdout:2/522: write d0/d2/d45/d7d/d56/f89 [1412525,1690] 0 2024-04-23T02:14:02.731 INFO:tasks.workunit.client.0.smithi071.stdout:8/529: dread dc/d15/da0/f1c [0,4194304] 0 2024-04-23T02:14:02.733 INFO:tasks.workunit.client.0.smithi071.stdout:9/504: creat d1/d32/d3a/d78/faa x:0 0 0 2024-04-23T02:14:02.734 INFO:tasks.workunit.client.0.smithi071.stdout:4/603: dread d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c/f2e [0,4194304] 0 2024-04-23T02:14:02.734 INFO:tasks.workunit.client.1.smithi192.stdout:3/522: link d1/d3/d17/d1f/l4d d1/d3/d17/d2b/d44/d57/db0/lb2 0 2024-04-23T02:14:02.738 INFO:tasks.workunit.client.1.smithi192.stdout:7/605: symlink d5/d17/d1c/d4a/daa/lb1 0 2024-04-23T02:14:02.739 INFO:tasks.workunit.client.1.smithi192.stdout:3/523: mknod d1/d3/d6/cb3 0 2024-04-23T02:14:02.739 INFO:tasks.workunit.client.0.smithi071.stdout:2/523: mkdir d0/daa 0 2024-04-23T02:14:02.740 INFO:tasks.workunit.client.1.smithi192.stdout:3/524: readlink d1/d3/d17/d65/l9e 0 2024-04-23T02:14:02.740 INFO:tasks.workunit.client.0.smithi071.stdout:4/604: fsync d2/d2a/d73/d5e/fb0 0 2024-04-23T02:14:02.743 INFO:tasks.workunit.client.0.smithi071.stdout:6/483: dwrite d6/d1f/d46/f62 [0,4194304] 0 2024-04-23T02:14:02.744 INFO:tasks.workunit.client.0.smithi071.stdout:9/505: mkdir d1/d32/d3a/d94/dab 0 2024-04-23T02:14:02.745 INFO:tasks.workunit.client.1.smithi192.stdout:3/525: creat d1/d3/d17/d2b/d44/d57/fb4 x:0 0 0 2024-04-23T02:14:02.746 INFO:tasks.workunit.client.0.smithi071.stdout:2/524: read - d0/d2/d26/df/f3b zero size 2024-04-23T02:14:02.748 INFO:tasks.workunit.client.1.smithi192.stdout:8/737: dread d1/d12/d23/d2c/d4d/fc8 [0,4194304] 0 2024-04-23T02:14:02.749 INFO:tasks.workunit.client.1.smithi192.stdout:3/526: chown d1/d3/d6/d7/d71/c89 1157474 1 2024-04-23T02:14:02.749 INFO:tasks.workunit.client.0.smithi071.stdout:9/506: creat d1/d32/fac x:0 0 0 2024-04-23T02:14:02.754 INFO:tasks.workunit.client.0.smithi071.stdout:5/615: write d3/d1d/d43/d4b/f7e [576616,16732] 0 2024-04-23T02:14:02.754 INFO:tasks.workunit.client.1.smithi192.stdout:7/606: dwrite d5/d17/d1c/d59/d5a/d41/d50/d9f/fa4 [0,4194304] 0 2024-04-23T02:14:02.756 INFO:tasks.workunit.client.0.smithi071.stdout:0/494: truncate d0/d3/de/d48/d7e/d8b/f22 6896606 0 2024-04-23T02:14:02.756 INFO:tasks.workunit.client.0.smithi071.stdout:3/481: dwrite d7/fd [0,4194304] 0 2024-04-23T02:14:02.759 INFO:tasks.workunit.client.0.smithi071.stdout:8/530: dwrite dc/d15/da0/d8d/f92 [0,4194304] 0 2024-04-23T02:14:02.761 INFO:tasks.workunit.client.1.smithi192.stdout:7/607: fdatasync d5/d17/d1c/d4a/f76 0 2024-04-23T02:14:02.764 INFO:tasks.workunit.client.0.smithi071.stdout:1/505: dwrite db/df/f58 [0,4194304] 0 2024-04-23T02:14:02.766 INFO:tasks.workunit.client.0.smithi071.stdout:7/474: dread d3/d17/d1c/f20 [0,4194304] 0 2024-04-23T02:14:02.769 INFO:tasks.workunit.client.0.smithi071.stdout:0/495: creat d0/d3/de/d10/d14/d32/d44/d87/fa0 x:0 0 0 2024-04-23T02:14:02.769 INFO:tasks.workunit.client.0.smithi071.stdout:9/507: link d1/d4e/l93 d1/d32/lad 0 2024-04-23T02:14:02.770 INFO:tasks.workunit.client.0.smithi071.stdout:5/616: write d3/d34/d6b/de4/fbb [1167155,85979] 0 2024-04-23T02:14:02.770 INFO:tasks.workunit.client.0.smithi071.stdout:4/605: creat d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d4f/d8b/fc4 x:0 0 0 2024-04-23T02:14:02.771 INFO:tasks.workunit.client.0.smithi071.stdout:6/484: link d6/d1f/d5d/d95/f98 d6/d91/f9c 0 2024-04-23T02:14:02.772 INFO:tasks.workunit.client.0.smithi071.stdout:9/508: stat d1/d32/d3a/d78 0 2024-04-23T02:14:02.773 INFO:tasks.workunit.client.0.smithi071.stdout:2/525: dwrite d0/d2/d26/d3a/f44 [0,4194304] 0 2024-04-23T02:14:02.773 INFO:tasks.workunit.client.0.smithi071.stdout:9/509: chown d1/d8d 6355 1 2024-04-23T02:14:02.777 INFO:tasks.workunit.client.0.smithi071.stdout:5/617: fdatasync d3/d34/f8b 0 2024-04-23T02:14:02.777 INFO:tasks.workunit.client.0.smithi071.stdout:6/485: read d6/d3e/d3f/f47 [209145,80239] 0 2024-04-23T02:14:02.779 INFO:tasks.workunit.client.0.smithi071.stdout:2/526: chown d0/d2/d34/d55 1945896 1 2024-04-23T02:14:02.779 INFO:tasks.workunit.client.0.smithi071.stdout:4/606: stat d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d10/l26 0 2024-04-23T02:14:02.780 INFO:tasks.workunit.client.0.smithi071.stdout:9/510: mknod d1/d32/d3a/cae 0 2024-04-23T02:14:02.781 INFO:tasks.workunit.client.0.smithi071.stdout:6/486: chown d6/d1f/d4e/f74 114047 1 2024-04-23T02:14:02.784 INFO:tasks.workunit.client.0.smithi071.stdout:7/475: dwrite d3/d17/d41/d34/f9b [0,4194304] 0 2024-04-23T02:14:02.787 INFO:tasks.workunit.client.0.smithi071.stdout:2/527: unlink d0/d2/d26/d14/d1f/d36/l7f 0 2024-04-23T02:14:02.790 INFO:tasks.workunit.client.0.smithi071.stdout:4/607: rename d2/d2a/d2d/d39/d42/d49/d67/d1d/faf to d2/d2a/d2d/d39/d42/d49/d91/fc5 0 2024-04-23T02:14:02.791 INFO:tasks.workunit.client.0.smithi071.stdout:8/531: dread dc/d56/d64/f69 [0,4194304] 0 2024-04-23T02:14:02.793 INFO:tasks.workunit.client.0.smithi071.stdout:7/476: mkdir d3/d17/d41/d34/d39/d3b/da7 0 2024-04-23T02:14:02.794 INFO:tasks.workunit.client.0.smithi071.stdout:5/618: dwrite d3/da/d1a/d1c/d46/d4e/f84 [0,4194304] 0 2024-04-23T02:14:02.795 INFO:tasks.workunit.client.0.smithi071.stdout:4/608: unlink d2/d2a/d2d/d7b/la1 0 2024-04-23T02:14:02.796 INFO:tasks.workunit.client.0.smithi071.stdout:7/477: write d3/d17/d41/d37/f6f [937726,30614] 0 2024-04-23T02:14:02.797 INFO:tasks.workunit.client.0.smithi071.stdout:2/528: creat d0/fab x:0 0 0 2024-04-23T02:14:02.798 INFO:tasks.workunit.client.0.smithi071.stdout:5/619: chown d3/d1d/d43/dd2/f91 828548 1 2024-04-23T02:14:02.798 INFO:tasks.workunit.client.0.smithi071.stdout:8/532: fsync dc/d15/da0/f3d 0 2024-04-23T02:14:02.799 INFO:tasks.workunit.client.0.smithi071.stdout:2/529: dread - d0/d2/fa3 zero size 2024-04-23T02:14:02.799 INFO:tasks.workunit.client.0.smithi071.stdout:4/609: symlink d2/d2a/d2d/d39/d42/d49/d91/lc6 0 2024-04-23T02:14:02.800 INFO:tasks.workunit.client.0.smithi071.stdout:7/478: mkdir d3/d17/d41/da8 0 2024-04-23T02:14:02.800 INFO:tasks.workunit.client.0.smithi071.stdout:2/530: chown d0/d2/f1d 69 1 2024-04-23T02:14:02.802 INFO:tasks.workunit.client.0.smithi071.stdout:5/620: read d3/d1d/daf/f89 [15094,84134] 0 2024-04-23T02:14:02.804 INFO:tasks.workunit.client.0.smithi071.stdout:7/479: unlink d3/d17/d1c/d2a/fa6 0 2024-04-23T02:14:02.806 INFO:tasks.workunit.client.0.smithi071.stdout:5/621: mkdir d3/d86/de9 0 2024-04-23T02:14:02.809 INFO:tasks.workunit.client.0.smithi071.stdout:2/531: rename d0/d2/d45/f60 to d0/d2/d45/d7d/d56/fac 0 2024-04-23T02:14:02.810 INFO:tasks.workunit.client.0.smithi071.stdout:7/480: mknod d3/d17/d41/d3a/da5/ca9 0 2024-04-23T02:14:02.812 INFO:tasks.workunit.client.0.smithi071.stdout:4/610: dwrite d2/d2a/d2d/d39/d42/d49/f77 [0,4194304] 0 2024-04-23T02:14:02.812 INFO:tasks.workunit.client.0.smithi071.stdout:8/533: dwrite dc/d15/da0/d30/fa6 [0,4194304] 0 2024-04-23T02:14:02.814 INFO:tasks.workunit.client.1.smithi192.stdout:9/512: sync 2024-04-23T02:14:02.815 INFO:tasks.workunit.client.0.smithi071.stdout:8/534: write dc/f54 [540734,20764] 0 2024-04-23T02:14:02.819 INFO:tasks.workunit.client.0.smithi071.stdout:4/611: stat d2/d2a/d2d/d39/d42/f55 0 2024-04-23T02:14:02.824 INFO:tasks.workunit.client.0.smithi071.stdout:2/532: dwrite d0/d2/d26/d14/f1c [0,4194304] 0 2024-04-23T02:14:02.824 INFO:tasks.workunit.client.0.smithi071.stdout:8/535: truncate dc/d15/da0/f1c 3057075 0 2024-04-23T02:14:02.828 INFO:tasks.workunit.client.0.smithi071.stdout:4/612: fdatasync d2/d2a/d2d/d39/d42/f55 0 2024-04-23T02:14:02.828 INFO:tasks.workunit.client.0.smithi071.stdout:8/536: write dc/f57 [3665864,49855] 0 2024-04-23T02:14:02.830 INFO:tasks.workunit.client.1.smithi192.stdout:9/513: write d0/d9/d44/d51/d64/f9c [869113,30858] 0 2024-04-23T02:14:02.830 INFO:tasks.workunit.client.1.smithi192.stdout:7/608: fdatasync d5/d17/d1c/d59/d5a/d70/f7e 0 2024-04-23T02:14:02.833 INFO:tasks.workunit.client.0.smithi071.stdout:2/533: creat d0/d2/d26/df/d30/fad x:0 0 0 2024-04-23T02:14:02.834 INFO:tasks.workunit.client.0.smithi071.stdout:4/613: mknod d2/d2a/da2/cc7 0 2024-04-23T02:14:02.834 INFO:tasks.workunit.client.1.smithi192.stdout:7/609: chown d5/d17/d1c/d59/d5a/d70 0 1 2024-04-23T02:14:02.837 INFO:tasks.workunit.client.0.smithi071.stdout:2/534: creat d0/d2/d26/d14/fae x:0 0 0 2024-04-23T02:14:02.839 INFO:tasks.workunit.client.1.smithi192.stdout:3/527: dread d1/d3/f7a [0,4194304] 0 2024-04-23T02:14:02.841 INFO:tasks.workunit.client.0.smithi071.stdout:8/537: dwrite dc/d15/da0/d55/f65 [8388608,4194304] 0 2024-04-23T02:14:02.841 INFO:tasks.workunit.client.1.smithi192.stdout:1/365: creat dc/d14/f6e x:0 0 0 2024-04-23T02:14:02.843 INFO:tasks.workunit.client.0.smithi071.stdout:4/614: dread d2/d2a/d2d/d39/d42/d49/d67/d1d/d57/f72 [0,4194304] 0 2024-04-23T02:14:02.846 INFO:tasks.workunit.client.0.smithi071.stdout:8/538: chown dc/d15/d2d/c5a 943138 1 2024-04-23T02:14:02.850 INFO:tasks.workunit.client.0.smithi071.stdout:8/539: symlink dc/d3a/da4/la7 0 2024-04-23T02:14:02.852 INFO:tasks.workunit.client.1.smithi192.stdout:9/514: chown d0/d9/d44/l5d 29820230 1 2024-04-23T02:14:02.854 INFO:tasks.workunit.client.1.smithi192.stdout:3/528: dread - d1/d3/d17/d1f/d33/d24/f50 zero size 2024-04-23T02:14:02.855 INFO:tasks.workunit.client.0.smithi071.stdout:8/540: creat dc/d15/d72/fa8 x:0 0 0 2024-04-23T02:14:02.856 INFO:tasks.workunit.client.0.smithi071.stdout:3/482: write d7/d1d/f72 [8103019,37460] 0 2024-04-23T02:14:02.857 INFO:tasks.workunit.client.0.smithi071.stdout:4/615: dwrite d2/d2a/d2d/f54 [0,4194304] 0 2024-04-23T02:14:02.858 INFO:tasks.workunit.client.1.smithi192.stdout:3/529: dread - d1/d3/d17/d2b/f76 zero size 2024-04-23T02:14:02.860 INFO:tasks.workunit.client.1.smithi192.stdout:1/366: dwrite dc/d14/d23/f3d [4194304,4194304] 0 2024-04-23T02:14:02.861 INFO:tasks.workunit.client.0.smithi071.stdout:6/487: write d6/d3e/f42 [2719045,123564] 0 2024-04-23T02:14:02.861 INFO:tasks.workunit.client.0.smithi071.stdout:8/541: write dc/f54 [696052,124331] 0 2024-04-23T02:14:02.863 INFO:tasks.workunit.client.1.smithi192.stdout:7/610: link d5/d17/d1c/d59/d5a/l8a d5/d17/d1c/d59/d5a/da0/lb2 0 2024-04-23T02:14:02.865 INFO:tasks.workunit.client.0.smithi071.stdout:1/506: dread db/df/d3a/f47 [0,4194304] 0 2024-04-23T02:14:02.866 INFO:tasks.workunit.client.1.smithi192.stdout:8/738: rmdir d1/d12/d23/d4b/d73/d7e/d88 39 2024-04-23T02:14:02.871 INFO:tasks.workunit.client.1.smithi192.stdout:3/530: rename d1/d3/f49 to d1/d3/d17/d1a/d7c/fb5 0 2024-04-23T02:14:02.872 INFO:tasks.workunit.client.0.smithi071.stdout:4/616: dread d2/d2a/d2d/d39/d42/d49/f77 [0,4194304] 0 2024-04-23T02:14:02.875 INFO:tasks.workunit.client.1.smithi192.stdout:3/531: chown d1/d3/d6/d7/d5f/c68 8107381 1 2024-04-23T02:14:02.876 INFO:tasks.workunit.client.1.smithi192.stdout:1/367: dwrite dc/d16/d19/d30/d32/f36 [0,4194304] 0 2024-04-23T02:14:02.877 INFO:tasks.workunit.client.1.smithi192.stdout:3/532: dread - d1/d3/d17/d1a/d7c/dab/d6c/d84/f63 zero size 2024-04-23T02:14:02.879 INFO:tasks.workunit.client.0.smithi071.stdout:3/483: rename d7/d22/d23/d26/d75 to d7/d22/da7 0 2024-04-23T02:14:02.879 INFO:tasks.workunit.client.1.smithi192.stdout:1/368: dread fa [0,4194304] 0 2024-04-23T02:14:02.880 INFO:tasks.workunit.client.1.smithi192.stdout:3/533: write d1/d3/d6/d7/d29/d3b/da9/d7b/f96 [726040,112552] 0 2024-04-23T02:14:02.881 INFO:tasks.workunit.client.1.smithi192.stdout:3/534: write d1/d3/d17/d2b/f51 [131891,99842] 0 2024-04-23T02:14:02.881 INFO:tasks.workunit.client.0.smithi071.stdout:4/617: mkdir d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/dc8 0 2024-04-23T02:14:02.882 INFO:tasks.workunit.client.0.smithi071.stdout:3/484: rmdir d7/d8/d42/d5f/d68/d3b 39 2024-04-23T02:14:02.882 INFO:tasks.workunit.client.1.smithi192.stdout:1/369: dread dc/d16/d19/f3a [0,4194304] 0 2024-04-23T02:14:02.885 INFO:tasks.workunit.client.1.smithi192.stdout:3/535: creat d1/d3/d6/d7/d29/d3b/da9/d8b/fb6 x:0 0 0 2024-04-23T02:14:02.885 INFO:tasks.workunit.client.0.smithi071.stdout:6/488: getdents d6/d17/d77/d85 0 2024-04-23T02:14:02.886 INFO:tasks.workunit.client.0.smithi071.stdout:4/618: write d2/d2a/d2d/d39/d42/d49/d67/d3e/f93 [439734,24051] 0 2024-04-23T02:14:02.886 INFO:tasks.workunit.client.0.smithi071.stdout:0/496: dread d0/d3/de/d10/d14/d32/f41 [0,4194304] 0 2024-04-23T02:14:02.887 INFO:tasks.workunit.client.0.smithi071.stdout:0/497: stat d0/d3/de/f1d 0 2024-04-23T02:14:02.889 INFO:tasks.workunit.client.0.smithi071.stdout:6/489: mknod d6/d1f/c9d 0 2024-04-23T02:14:02.891 INFO:tasks.workunit.client.1.smithi192.stdout:1/370: dwrite dc/d14/d15/d35/f37 [0,4194304] 0 2024-04-23T02:14:02.893 INFO:tasks.workunit.client.1.smithi192.stdout:7/611: rename d5/d43/c60 to d5/d17/d1c/d59/d5a/d41/d50/cb3 0 2024-04-23T02:14:02.896 INFO:tasks.workunit.client.1.smithi192.stdout:3/536: rename d1/d3/d6/d7/d29/d3b/da9/l69 to d1/d3/d6/d7/d29/d3b/lb7 0 2024-04-23T02:14:02.897 INFO:tasks.workunit.client.1.smithi192.stdout:7/612: truncate d5/d17/d1c/d59/d5a/d41/f54 4619060 0 2024-04-23T02:14:02.898 INFO:tasks.workunit.client.1.smithi192.stdout:7/613: fsync d5/d17/d1c/f3c 0 2024-04-23T02:14:02.900 INFO:tasks.workunit.client.0.smithi071.stdout:3/485: rename d7/d27 to d7/d8/d42/d5f/d68/d3b/d4c/da1/da8 0 2024-04-23T02:14:02.900 INFO:tasks.workunit.client.1.smithi192.stdout:3/537: mknod d1/d3/d6/d7/cb8 0 2024-04-23T02:14:02.900 INFO:tasks.workunit.client.0.smithi071.stdout:7/481: write d3/d17/d41/d34/d59/f67 [145506,91606] 0 2024-04-23T02:14:02.901 INFO:tasks.workunit.client.0.smithi071.stdout:5/622: dwrite d3/da/d1a/d1c/d46/d4e/f70 [0,4194304] 0 2024-04-23T02:14:02.902 INFO:tasks.workunit.client.0.smithi071.stdout:4/619: dread d2/d2a/d2d/f5c [0,4194304] 0 2024-04-23T02:14:02.902 INFO:tasks.workunit.client.0.smithi071.stdout:7/482: readlink d3/l4b 0 2024-04-23T02:14:02.903 INFO:tasks.workunit.client.0.smithi071.stdout:3/486: stat d7/d8/d42/d5f/d68/f84 0 2024-04-23T02:14:02.903 INFO:tasks.workunit.client.1.smithi192.stdout:3/538: write d1/d3/d6/f10 [2440298,129287] 0 2024-04-23T02:14:02.904 INFO:tasks.workunit.client.1.smithi192.stdout:7/614: write d5/d17/d1c/d59/d5a/d41/d50/d3d/f83 [715959,17162] 0 2024-04-23T02:14:02.905 INFO:tasks.workunit.client.0.smithi071.stdout:4/620: chown d2/c6 0 1 2024-04-23T02:14:02.907 INFO:tasks.workunit.client.0.smithi071.stdout:9/511: dread d1/d1a/f22 [0,4194304] 0 2024-04-23T02:14:02.907 INFO:tasks.workunit.client.0.smithi071.stdout:3/487: write d7/d8/d42/f7c [27670,71827] 0 2024-04-23T02:14:02.909 INFO:tasks.workunit.client.0.smithi071.stdout:0/498: dread d0/d3/de/d48/d7e/d8b/f21 [0,4194304] 0 2024-04-23T02:14:02.910 INFO:tasks.workunit.client.1.smithi192.stdout:7/615: symlink d5/d17/lb4 0 2024-04-23T02:14:02.911 INFO:tasks.workunit.client.0.smithi071.stdout:2/535: write d0/d2/d26/d14/d1f/d4f/f54 [684224,29025] 0 2024-04-23T02:14:02.913 INFO:tasks.workunit.client.1.smithi192.stdout:3/539: getdents d1/d3/d17/d1f 0 2024-04-23T02:14:02.913 INFO:tasks.workunit.client.0.smithi071.stdout:8/542: truncate f6 860798 0 2024-04-23T02:14:02.915 INFO:tasks.workunit.client.1.smithi192.stdout:3/540: rmdir d1/d3/d17/d1a/d7c 39 2024-04-23T02:14:02.915 INFO:tasks.workunit.client.1.smithi192.stdout:7/616: fsync d5/d17/d1c/d59/d5a/d41/f98 0 2024-04-23T02:14:02.916 INFO:tasks.workunit.client.0.smithi071.stdout:4/621: write d2/d2a/d2d/d39/d42/d49/d91/f92 [160408,99810] 0 2024-04-23T02:14:02.919 INFO:tasks.workunit.client.1.smithi192.stdout:7/617: mkdir d5/d17/db5 0 2024-04-23T02:14:02.921 INFO:tasks.workunit.client.0.smithi071.stdout:3/488: mknod d7/d8/d42/d5f/d68/d3b/d4c/da1/da8/ca9 0 2024-04-23T02:14:02.924 INFO:tasks.workunit.client.1.smithi192.stdout:7/618: chown d5/d43/d4c/d5c/d67 1053909281 1 2024-04-23T02:14:02.924 INFO:tasks.workunit.client.1.smithi192.stdout:9/515: dwrite d0/d9/d44/d51/d64/f78 [0,4194304] 0 2024-04-23T02:14:02.925 INFO:tasks.workunit.client.0.smithi071.stdout:7/483: dwrite d3/d17/d41/f33 [0,4194304] 0 2024-04-23T02:14:02.925 INFO:tasks.workunit.client.0.smithi071.stdout:0/499: link d0/d63/f74 d0/d3/de/d10/d14/d32/d37/fa1 0 2024-04-23T02:14:02.926 INFO:tasks.workunit.client.1.smithi192.stdout:3/541: creat d1/d3/d17/d1f/d33/d24/fb9 x:0 0 0 2024-04-23T02:14:02.928 INFO:tasks.workunit.client.1.smithi192.stdout:8/739: dwrite d1/d9/d3e/d59/d8a/f8d [0,4194304] 0 2024-04-23T02:14:02.929 INFO:tasks.workunit.client.1.smithi192.stdout:1/371: dread f9 [0,4194304] 0 2024-04-23T02:14:02.930 INFO:tasks.workunit.client.1.smithi192.stdout:7/619: write d5/d17/d1c/d59/d5a/d41/f54 [2204212,87824] 0 2024-04-23T02:14:02.934 INFO:tasks.workunit.client.0.smithi071.stdout:3/489: mkdir d7/d10/d38/daa 0 2024-04-23T02:14:02.934 INFO:tasks.workunit.client.0.smithi071.stdout:1/507: dwrite f5 [0,4194304] 0 2024-04-23T02:14:02.934 INFO:tasks.workunit.client.1.smithi192.stdout:9/516: unlink d0/d9/d44/d53/c94 0 2024-04-23T02:14:02.935 INFO:tasks.workunit.client.1.smithi192.stdout:7/620: rmdir d5/d17/d1c/d59/d5a/d41/d50/d3d 39 2024-04-23T02:14:02.936 INFO:tasks.workunit.client.1.smithi192.stdout:3/542: write d1/d3/d6/d7/d29/d3b/da9/d7b/f96 [353045,73735] 0 2024-04-23T02:14:02.937 INFO:tasks.workunit.client.1.smithi192.stdout:1/372: symlink dc/d14/d15/l6f 0 2024-04-23T02:14:02.939 INFO:tasks.workunit.client.1.smithi192.stdout:1/373: chown dc/d14/d15/c65 1 1 2024-04-23T02:14:02.941 INFO:tasks.workunit.client.1.smithi192.stdout:9/517: creat d0/d9/d44/d4f/fb2 x:0 0 0 2024-04-23T02:14:02.942 INFO:tasks.workunit.client.0.smithi071.stdout:8/543: dwrite dc/d3a/f3c [4194304,4194304] 0 2024-04-23T02:14:02.943 INFO:tasks.workunit.client.1.smithi192.stdout:3/543: creat d1/d3/d6/d7/d29/d3b/fba x:0 0 0 2024-04-23T02:14:02.945 INFO:tasks.workunit.client.1.smithi192.stdout:9/518: write d0/d9/d44/d4f/f28 [1467850,30855] 0 2024-04-23T02:14:02.948 INFO:tasks.workunit.client.0.smithi071.stdout:6/490: dwrite d6/d22/f41 [0,4194304] 0 2024-04-23T02:14:02.949 INFO:tasks.workunit.client.1.smithi192.stdout:1/374: unlink dc/d16/c2d 0 2024-04-23T02:14:02.950 INFO:tasks.workunit.client.1.smithi192.stdout:9/519: fsync d0/d9/d44/d51/d5b/d61/f84 0 2024-04-23T02:14:02.951 INFO:tasks.workunit.client.0.smithi071.stdout:4/622: dread d2/d2a/d2d/d39/d42/d49/d91/f92 [0,4194304] 0 2024-04-23T02:14:02.953 INFO:tasks.workunit.client.1.smithi192.stdout:7/621: link d5/d17/d1c/d59/d5a/d41/d50/d9f/fa4 d5/d43/d4c/d5c/d67/da6/fb6 0 2024-04-23T02:14:02.954 INFO:tasks.workunit.client.1.smithi192.stdout:1/375: read dc/d16/d19/d30/d32/d44/f53 [91665,129933] 0 2024-04-23T02:14:02.955 INFO:tasks.workunit.client.0.smithi071.stdout:8/544: creat dc/d15/d50/fa9 x:0 0 0 2024-04-23T02:14:02.956 INFO:tasks.workunit.client.1.smithi192.stdout:7/622: chown d5/d43/d4c/d5c/d67/da6 130232 1 2024-04-23T02:14:02.957 INFO:tasks.workunit.client.0.smithi071.stdout:0/500: creat d0/d3/de/d10/d14/d32/d37/fa2 x:0 0 0 2024-04-23T02:14:02.958 INFO:tasks.workunit.client.0.smithi071.stdout:7/484: dwrite d3/d17/d1c/f26 [4194304,4194304] 0 2024-04-23T02:14:02.958 INFO:tasks.workunit.client.0.smithi071.stdout:6/491: unlink d6/d1f/l52 0 2024-04-23T02:14:02.959 INFO:tasks.workunit.client.0.smithi071.stdout:4/623: fsync d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d10/f87 0 2024-04-23T02:14:02.959 INFO:tasks.workunit.client.0.smithi071.stdout:5/623: dread d3/d34/f40 [0,4194304] 0 2024-04-23T02:14:02.961 INFO:tasks.workunit.client.1.smithi192.stdout:3/544: link d1/f5 d1/d3/d6/d7/fbb 0 2024-04-23T02:14:02.962 INFO:tasks.workunit.client.0.smithi071.stdout:8/545: truncate dc/d78/f8f 4969991 0 2024-04-23T02:14:02.964 INFO:tasks.workunit.client.1.smithi192.stdout:3/545: stat d1/d3/d17/f3d 0 2024-04-23T02:14:02.967 INFO:tasks.workunit.client.0.smithi071.stdout:7/485: mknod d3/d17/d41/d34/d39/d3b/caa 0 2024-04-23T02:14:02.968 INFO:tasks.workunit.client.1.smithi192.stdout:7/623: truncate d5/d17/d1c/d59/d5a/d41/f98 64708 0 2024-04-23T02:14:02.969 INFO:tasks.workunit.client.1.smithi192.stdout:7/624: stat d5/d17/f9e 0 2024-04-23T02:14:02.969 INFO:tasks.workunit.client.1.smithi192.stdout:9/520: rename d0/d9/d44/d74/d3c to d0/d9/db3 0 2024-04-23T02:14:02.971 INFO:tasks.workunit.client.1.smithi192.stdout:3/546: unlink d1/d3/d6/d7/d29/d3b/fba 0 2024-04-23T02:14:02.971 INFO:tasks.workunit.client.1.smithi192.stdout:1/376: dwrite dc/d14/f21 [0,4194304] 0 2024-04-23T02:14:02.971 INFO:tasks.workunit.client.1.smithi192.stdout:7/625: read d5/d43/d4c/d5c/d67/da6/fb6 [2066918,111753] 0 2024-04-23T02:14:02.972 INFO:tasks.workunit.client.0.smithi071.stdout:6/492: stat d6/d17/d77/d85/f89 0 2024-04-23T02:14:02.972 INFO:tasks.workunit.client.0.smithi071.stdout:5/624: rmdir d3/d34/d74 39 2024-04-23T02:14:02.974 INFO:tasks.workunit.client.0.smithi071.stdout:9/512: dread d1/f4 [0,4194304] 0 2024-04-23T02:14:02.978 INFO:tasks.workunit.client.1.smithi192.stdout:3/547: fsync d1/d3/d17/d65/fa4 0 2024-04-23T02:14:02.978 INFO:tasks.workunit.client.1.smithi192.stdout:8/740: truncate d1/d9/d3e/d59/fb0 3399766 0 2024-04-23T02:14:02.978 INFO:tasks.workunit.client.0.smithi071.stdout:3/490: dread d7/d10/d38/f41 [0,4194304] 0 2024-04-23T02:14:02.978 INFO:tasks.workunit.client.0.smithi071.stdout:5/625: fdatasync d3/d1d/daf/f98 0 2024-04-23T02:14:02.980 INFO:tasks.workunit.client.0.smithi071.stdout:4/624: creat d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/fc9 x:0 0 0 2024-04-23T02:14:02.981 INFO:tasks.workunit.client.0.smithi071.stdout:6/493: read d6/d3e/f8a [3461796,72094] 0 2024-04-23T02:14:02.982 INFO:tasks.workunit.client.1.smithi192.stdout:7/626: symlink d5/d17/d1c/d59/d5a/da0/lb7 0 2024-04-23T02:14:02.984 INFO:tasks.workunit.client.1.smithi192.stdout:8/741: creat d1/d12/d7d/df7/f106 x:0 0 0 2024-04-23T02:14:02.984 INFO:tasks.workunit.client.0.smithi071.stdout:8/546: truncate dc/d15/da0/f1e 1633930 0 2024-04-23T02:14:02.984 INFO:tasks.workunit.client.0.smithi071.stdout:7/486: getdents d3 0 2024-04-23T02:14:02.985 INFO:tasks.workunit.client.0.smithi071.stdout:9/513: truncate d1/d32/d3a/f3b 638346 0 2024-04-23T02:14:02.985 INFO:tasks.workunit.client.1.smithi192.stdout:7/627: fdatasync d5/d43/d4c/d5c/d97/faf 0 2024-04-23T02:14:02.986 INFO:tasks.workunit.client.0.smithi071.stdout:5/626: unlink d3/d1d/cbc 0 2024-04-23T02:14:02.987 INFO:tasks.workunit.client.0.smithi071.stdout:7/487: stat d3/d17/d41/d34/d39/d3b/f53 0 2024-04-23T02:14:02.988 INFO:tasks.workunit.client.1.smithi192.stdout:3/548: rename d1/d3/d17/d2b/d44/d57/f97 to d1/d3/d17/d1a/d7c/dab/d6c/d84/d46/fbc 0 2024-04-23T02:14:02.992 INFO:tasks.workunit.client.1.smithi192.stdout:8/742: dread d1/d9/fee [0,4194304] 0 2024-04-23T02:14:02.993 INFO:tasks.workunit.client.0.smithi071.stdout:3/491: rename d7/d8/d42/d5f/l5a to d7/d8/d42/d5f/d68/d3b/d4c/d74/lab 0 2024-04-23T02:14:02.993 INFO:tasks.workunit.client.0.smithi071.stdout:7/488: write d3/f69 [552393,50762] 0 2024-04-23T02:14:02.999 INFO:tasks.workunit.client.0.smithi071.stdout:3/492: rmdir d7 39 2024-04-23T02:14:03.005 INFO:tasks.workunit.client.1.smithi192.stdout:3/549: getdents d1/d3/d17/d1f 0 2024-04-23T02:14:03.005 INFO:tasks.workunit.client.1.smithi192.stdout:8/743: dread d1/d9/d3e/d59/fff [0,4194304] 0 2024-04-23T02:14:03.006 INFO:tasks.workunit.client.0.smithi071.stdout:7/489: write d3/d17/d41/d37/f50 [230272,78766] 0 2024-04-23T02:14:03.006 INFO:tasks.workunit.client.0.smithi071.stdout:9/514: rename d1/d4e/f5b to d1/d32/d41/faf 0 2024-04-23T02:14:03.007 INFO:tasks.workunit.client.0.smithi071.stdout:6/494: dwrite d6/d1f/d4e/f54 [0,4194304] 0 2024-04-23T02:14:03.008 INFO:tasks.workunit.client.1.smithi192.stdout:7/628: dwrite d5/d43/d4c/d5c/d67/f81 [0,4194304] 0 2024-04-23T02:14:03.011 INFO:tasks.workunit.client.1.smithi192.stdout:9/521: dread d0/d9/d44/f45 [0,4194304] 0 2024-04-23T02:14:03.011 INFO:tasks.workunit.client.1.smithi192.stdout:8/744: write d1/d9/d3e/f55 [886703,47302] 0 2024-04-23T02:14:03.011 INFO:tasks.workunit.client.0.smithi071.stdout:4/625: dwrite d2/d53/f88 [0,4194304] 0 2024-04-23T02:14:03.015 INFO:tasks.workunit.client.0.smithi071.stdout:1/508: dread db/df/d13/d2f/d48/f6e [0,4194304] 0 2024-04-23T02:14:03.016 INFO:tasks.workunit.client.0.smithi071.stdout:7/490: stat d3/fc 0 2024-04-23T02:14:03.018 INFO:tasks.workunit.client.0.smithi071.stdout:8/547: dwrite dc/d15/da0/f14 [0,4194304] 0 2024-04-23T02:14:03.019 INFO:tasks.workunit.client.1.smithi192.stdout:7/629: readlink d5/d17/d1c/d59/d5a/d41/l78 0 2024-04-23T02:14:03.021 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:02 smithi071 ceph-mon[39837]: pgmap v28: 65 pgs: 65 active+clean; 1.7 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 43 MiB/s rd, 219 MiB/s wr, 365 op/s 2024-04-23T02:14:03.021 INFO:tasks.workunit.client.1.smithi192.stdout:7/630: fdatasync d5/d17/f9e 0 2024-04-23T02:14:03.023 INFO:tasks.workunit.client.1.smithi192.stdout:1/377: dread dc/d14/d15/d35/f40 [0,4194304] 0 2024-04-23T02:14:03.025 INFO:tasks.workunit.client.1.smithi192.stdout:9/522: mkdir d0/d9/d44/d51/d5b/d96/da0/db4 0 2024-04-23T02:14:03.026 INFO:tasks.workunit.client.1.smithi192.stdout:8/745: fdatasync d1/d9/d3e/f55 0 2024-04-23T02:14:03.027 INFO:tasks.workunit.client.0.smithi071.stdout:5/627: rename d3/d34/d6b/f8d to d3/da/d1a/d1c/d46/d4e/fea 0 2024-04-23T02:14:03.028 INFO:tasks.workunit.client.0.smithi071.stdout:6/495: unlink d6/d1f/d4e/l65 0 2024-04-23T02:14:03.028 INFO:tasks.workunit.client.1.smithi192.stdout:3/550: dwrite d1/d3/d17/d1f/d33/d24/f50 [0,4194304] 0 2024-04-23T02:14:03.029 INFO:tasks.workunit.client.0.smithi071.stdout:8/548: stat dc/d15 0 2024-04-23T02:14:03.029 INFO:tasks.workunit.client.0.smithi071.stdout:9/515: creat d1/d4e/d54/d75/fb0 x:0 0 0 2024-04-23T02:14:03.029 INFO:tasks.workunit.client.1.smithi192.stdout:9/523: chown d0/l12 1088441 1 2024-04-23T02:14:03.030 INFO:tasks.workunit.client.1.smithi192.stdout:0/689: sync 2024-04-23T02:14:03.034 INFO:tasks.workunit.client.1.smithi192.stdout:1/378: mkdir dc/d16/d19/d30/d32/d34/d70 0 2024-04-23T02:14:03.036 INFO:tasks.workunit.client.1.smithi192.stdout:9/524: write d0/d9/d44/f45 [2370375,26956] 0 2024-04-23T02:14:03.037 INFO:tasks.workunit.client.1.smithi192.stdout:8/746: read d1/d12/d23/d4b/d73/d45/d46/f6d [2630431,127698] 0 2024-04-23T02:14:03.038 INFO:tasks.workunit.client.1.smithi192.stdout:0/690: rename d6/df/f32 to d6/df/d13/d25/d86/fe4 0 2024-04-23T02:14:03.039 INFO:tasks.workunit.client.0.smithi071.stdout:0/501: dread d0/d3/de/d10/d14/d32/d37/f53 [0,4194304] 0 2024-04-23T02:14:03.039 INFO:tasks.workunit.client.0.smithi071.stdout:3/493: dwrite d7/d8/f61 [0,4194304] 0 2024-04-23T02:14:03.040 INFO:tasks.workunit.client.1.smithi192.stdout:3/551: mkdir d1/d3/d6/d7/d29/d3b/da9/dbd 0 2024-04-23T02:14:03.050 INFO:tasks.workunit.client.0.smithi071.stdout:4/626: read d2/d2a/d2d/d39/d42/d49/d67/f1a [4168595,45971] 0 2024-04-23T02:14:03.050 INFO:tasks.workunit.client.0.smithi071.stdout:3/494: dwrite d7/fd [0,4194304] 0 2024-04-23T02:14:03.051 INFO:tasks.workunit.client.0.smithi071.stdout:1/509: symlink db/d92/la4 0 2024-04-23T02:14:03.051 INFO:tasks.workunit.client.1.smithi192.stdout:7/631: dwrite d5/d17/d1c/d4a/f76 [0,4194304] 0 2024-04-23T02:14:03.051 INFO:tasks.workunit.client.1.smithi192.stdout:0/691: creat d6/df/d13/d6b/d73/da9/fe5 x:0 0 0 2024-04-23T02:14:03.053 INFO:tasks.workunit.client.1.smithi192.stdout:0/692: fsync d6/df/d13/d25/d3a/f58 0 2024-04-23T02:14:03.053 INFO:tasks.workunit.client.1.smithi192.stdout:3/552: unlink d1/d3/d17/d2b/f76 0 2024-04-23T02:14:03.054 INFO:tasks.workunit.client.0.smithi071.stdout:0/502: creat d0/d3/de/d10/d14/d32/d44/d87/d93/fa3 x:0 0 0 2024-04-23T02:14:03.056 INFO:tasks.workunit.client.1.smithi192.stdout:8/747: truncate d1/d12/d23/d4b/d73/d4c/d82/fb8 174152 0 2024-04-23T02:14:03.058 INFO:tasks.workunit.client.0.smithi071.stdout:8/549: rename dc/d15/da0/d30/f66 to dc/d15/d5d/faa 0 2024-04-23T02:14:03.059 INFO:tasks.workunit.client.0.smithi071.stdout:8/550: chown dc/d3a/f47 61463201 1 2024-04-23T02:14:03.059 INFO:tasks.workunit.client.1.smithi192.stdout:7/632: write d5/d17/d1c/d57/f22 [6339638,123098] 0 2024-04-23T02:14:03.060 INFO:tasks.workunit.client.1.smithi192.stdout:7/633: write d5/d17/d1c/d59/f93 [96074,50212] 0 2024-04-23T02:14:03.060 INFO:tasks.workunit.client.1.smithi192.stdout:1/379: symlink dc/d16/d19/d30/d32/d34/d70/l71 0 2024-04-23T02:14:03.060 INFO:tasks.workunit.client.0.smithi071.stdout:4/627: dread d2/d53/fa5 [0,4194304] 0 2024-04-23T02:14:03.060 INFO:tasks.workunit.client.0.smithi071.stdout:0/503: fsync d0/d3/de/d48/d7e/d8b/f94 0 2024-04-23T02:14:03.060 INFO:tasks.workunit.client.0.smithi071.stdout:9/516: dread - d1/d32/d41/faf zero size 2024-04-23T02:14:03.061 INFO:tasks.workunit.client.0.smithi071.stdout:7/491: getdents d3/d17/d41/d34/d59/d5f 0 2024-04-23T02:14:03.063 INFO:tasks.workunit.client.0.smithi071.stdout:1/510: creat db/df/d13/d2f/d48/d9f/fa5 x:0 0 0 2024-04-23T02:14:03.064 INFO:tasks.workunit.client.0.smithi071.stdout:5/628: dread d3/d1d/d43/dd2/f58 [0,4194304] 0 2024-04-23T02:14:03.065 INFO:tasks.workunit.client.1.smithi192.stdout:0/693: mkdir d6/df/d13/d25/d94/dd0/de6 0 2024-04-23T02:14:03.066 INFO:tasks.workunit.client.0.smithi071.stdout:8/551: write dc/d15/d50/fa9 [675467,34266] 0 2024-04-23T02:14:03.068 INFO:tasks.workunit.client.0.smithi071.stdout:3/495: dread d7/d8/fc [0,4194304] 0 2024-04-23T02:14:03.068 INFO:tasks.workunit.client.0.smithi071.stdout:0/504: read - d0/d3/de/d10/d14/d32/d44/d31/d52/f91 zero size 2024-04-23T02:14:03.071 INFO:tasks.workunit.client.1.smithi192.stdout:0/694: creat d6/df/d35/d49/fe7 x:0 0 0 2024-04-23T02:14:03.072 INFO:tasks.workunit.client.0.smithi071.stdout:8/552: readlink dc/d3a/da4/la7 0 2024-04-23T02:14:03.072 INFO:tasks.workunit.client.1.smithi192.stdout:3/553: dwrite d1/d3/d17/d1a/d7c/dab/d6c/d84/d5d/f9c [0,4194304] 0 2024-04-23T02:14:03.074 INFO:tasks.workunit.client.1.smithi192.stdout:0/695: symlink d6/df/d47/le8 0 2024-04-23T02:14:03.075 INFO:tasks.workunit.client.1.smithi192.stdout:3/554: chown d1/d3/d17/d1a/d7c/dab/d6c/d84/d5d/c6a 1098 1 2024-04-23T02:14:03.076 INFO:tasks.workunit.client.0.smithi071.stdout:3/496: creat d7/d8/d42/fac x:0 0 0 2024-04-23T02:14:03.076 INFO:tasks.workunit.client.1.smithi192.stdout:0/696: chown d6/d4b/d9d/dc7/l78 49085630 1 2024-04-23T02:14:03.077 INFO:tasks.workunit.client.0.smithi071.stdout:5/629: rename d3/da/d1a/d1c/f31 to d3/d1d/daf/d10/da5/feb 0 2024-04-23T02:14:03.078 INFO:tasks.workunit.client.1.smithi192.stdout:0/697: read - d6/df/d35/d49/f9f zero size 2024-04-23T02:14:03.081 INFO:tasks.workunit.client.1.smithi192.stdout:3/555: link d1/d3/f19 d1/d3/d6/d7/d29/fbe 0 2024-04-23T02:14:03.082 INFO:tasks.workunit.client.0.smithi071.stdout:4/628: dwrite d2/d2a/d2d/d39/d42/d49/d67/d1d/f61 [4194304,4194304] 0 2024-04-23T02:14:03.083 INFO:tasks.workunit.client.0.smithi071.stdout:0/505: chown d0/d3/de/d10/d14/d32/d37/c61 7819688 1 2024-04-23T02:14:03.085 INFO:tasks.workunit.client.1.smithi192.stdout:0/698: link d6/df/d35/d49/f54 d6/df/d13/d25/d3a/da6/fe9 0 2024-04-23T02:14:03.085 INFO:tasks.workunit.client.1.smithi192.stdout:3/556: mknod d1/d3/d6/d7/d71/cbf 0 2024-04-23T02:14:03.087 INFO:tasks.workunit.client.0.smithi071.stdout:7/492: dwrite d3/f1d [0,4194304] 0 2024-04-23T02:14:03.089 INFO:tasks.workunit.client.0.smithi071.stdout:7/493: readlink d3/l44 0 2024-04-23T02:14:03.091 INFO:tasks.workunit.client.1.smithi192.stdout:3/557: mknod d1/d3/d17/d1a/cc0 0 2024-04-23T02:14:03.091 INFO:tasks.workunit.client.0.smithi071.stdout:4/629: unlink d2/d2a/d2d/d39/d42/d49/d67/d7e/dae/l90 0 2024-04-23T02:14:03.091 INFO:tasks.workunit.client.0.smithi071.stdout:3/497: link d7/d8/d42/f87 d7/d8/d42/d5f/d68/d3b/d4c/fad 0 2024-04-23T02:14:03.094 INFO:tasks.workunit.client.0.smithi071.stdout:6/496: dwrite d6/d91/f9c [0,4194304] 0 2024-04-23T02:14:03.094 INFO:tasks.workunit.client.0.smithi071.stdout:0/506: creat d0/d3/de/d10/d14/d32/fa4 x:0 0 0 2024-04-23T02:14:03.097 INFO:tasks.workunit.client.0.smithi071.stdout:7/494: rename d3/d17/d41/d34/d39/d3b/f46 to d3/d51/d87/fab 0 2024-04-23T02:14:03.098 INFO:tasks.workunit.client.1.smithi192.stdout:3/558: mknod d1/d3/d17/d1a/d7c/dab/d6c/d84/d46/cc1 0 2024-04-23T02:14:03.098 INFO:tasks.workunit.client.1.smithi192.stdout:9/525: dread d0/d9/d44/d51/f66 [0,4194304] 0 2024-04-23T02:14:03.099 INFO:tasks.workunit.client.1.smithi192.stdout:9/526: chown d0/d9/d44/d51/la9 6870586 1 2024-04-23T02:14:03.099 INFO:tasks.workunit.client.0.smithi071.stdout:6/497: fsync d6/d3e/f8a 0 2024-04-23T02:14:03.101 INFO:tasks.workunit.client.1.smithi192.stdout:1/380: dread dc/d16/d19/d30/f3f [0,4194304] 0 2024-04-23T02:14:03.102 INFO:tasks.workunit.client.0.smithi071.stdout:6/498: chown d6/d1f/d5d/l97 31 1 2024-04-23T02:14:03.103 INFO:tasks.workunit.client.1.smithi192.stdout:8/748: dread d1/d12/d23/d4b/d73/d5d/fa6 [0,4194304] 0 2024-04-23T02:14:03.103 INFO:tasks.workunit.client.0.smithi071.stdout:0/507: mknod d0/d3/de/d10/d14/d32/d44/d87/ca5 0 2024-04-23T02:14:03.109 INFO:tasks.workunit.client.1.smithi192.stdout:1/381: symlink dc/d16/d19/d30/d32/d34/d70/l72 0 2024-04-23T02:14:03.109 INFO:tasks.workunit.client.0.smithi071.stdout:4/630: dwrite d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d10/f87 [0,4194304] 0 2024-04-23T02:14:03.110 INFO:tasks.workunit.client.0.smithi071.stdout:9/517: write d1/d32/d3a/d77/f88 [292232,78427] 0 2024-04-23T02:14:03.111 INFO:tasks.workunit.client.0.smithi071.stdout:0/508: symlink d0/d3/de/d10/d14/d32/d44/d87/la6 0 2024-04-23T02:14:03.111 INFO:tasks.workunit.client.0.smithi071.stdout:7/495: dread d3/d17/d41/d37/f50 [0,4194304] 0 2024-04-23T02:14:03.112 INFO:tasks.workunit.client.0.smithi071.stdout:1/511: truncate db/d34/f61 3114443 0 2024-04-23T02:14:03.113 INFO:tasks.workunit.client.0.smithi071.stdout:5/630: write d3/da/d1a/d1c/d46/d4e/f50 [669973,1655] 0 2024-04-23T02:14:03.115 INFO:tasks.workunit.client.1.smithi192.stdout:9/527: dwrite d0/d9/d44/d51/d5b/d61/f84 [0,4194304] 0 2024-04-23T02:14:03.115 INFO:tasks.workunit.client.0.smithi071.stdout:1/512: fdatasync db/d11/f64 0 2024-04-23T02:14:03.115 INFO:tasks.workunit.client.0.smithi071.stdout:8/553: truncate dc/d15/d50/f82 2647350 0 2024-04-23T02:14:03.115 INFO:tasks.workunit.client.0.smithi071.stdout:4/631: dread - d2/d2a/d2d/d39/d42/d49/d67/d1d/d57/f94 zero size 2024-04-23T02:14:03.121 INFO:tasks.workunit.client.0.smithi071.stdout:6/499: dwrite d6/f16 [4194304,4194304] 0 2024-04-23T02:14:03.121 INFO:tasks.workunit.client.0.smithi071.stdout:6/500: stat d6/d3e/d3f/l66 0 2024-04-23T02:14:03.122 INFO:tasks.workunit.client.0.smithi071.stdout:6/501: dread - d6/d1f/f32 zero size 2024-04-23T02:14:03.125 INFO:tasks.workunit.client.0.smithi071.stdout:1/513: creat db/d34/fa6 x:0 0 0 2024-04-23T02:14:03.126 INFO:tasks.workunit.client.0.smithi071.stdout:7/496: rmdir d3/d17 39 2024-04-23T02:14:03.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:02 smithi192 ceph-mon[49662]: pgmap v28: 65 pgs: 65 active+clean; 1.7 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 43 MiB/s rd, 219 MiB/s wr, 365 op/s 2024-04-23T02:14:03.127 INFO:tasks.workunit.client.1.smithi192.stdout:8/749: rename d1/d12/d23/d4b/d73/d7e/d88/l100 to d1/dcb/l107 0 2024-04-23T02:14:03.129 INFO:tasks.workunit.client.1.smithi192.stdout:8/750: chown d1/d12/d7d/dd2/lfb 7298 1 2024-04-23T02:14:03.131 INFO:tasks.workunit.client.1.smithi192.stdout:9/528: creat d0/d9/d44/d51/fb5 x:0 0 0 2024-04-23T02:14:03.132 INFO:tasks.workunit.client.1.smithi192.stdout:8/751: fsync d1/d12/d23/d4b/d73/d4c/f6e 0 2024-04-23T02:14:03.133 INFO:tasks.workunit.client.0.smithi071.stdout:7/497: readlink d3/l10 0 2024-04-23T02:14:03.136 INFO:tasks.workunit.client.1.smithi192.stdout:9/529: unlink d0/d9/f25 0 2024-04-23T02:14:03.136 INFO:tasks.workunit.client.0.smithi071.stdout:6/502: dwrite d6/d3e/f75 [0,4194304] 0 2024-04-23T02:14:03.139 INFO:tasks.workunit.client.0.smithi071.stdout:1/514: dwrite db/df/d5f/f82 [0,4194304] 0 2024-04-23T02:14:03.139 INFO:tasks.workunit.client.1.smithi192.stdout:3/559: write d1/d3/d17/d2b/d44/d57/f78 [38859,85421] 0 2024-04-23T02:14:03.141 INFO:tasks.workunit.client.1.smithi192.stdout:3/560: readlink d1/d3/d17/l30 0 2024-04-23T02:14:03.142 INFO:tasks.workunit.client.1.smithi192.stdout:8/752: dwrite d1/f4a [4194304,4194304] 0 2024-04-23T02:14:03.146 INFO:tasks.workunit.client.1.smithi192.stdout:8/753: chown d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81 282361804 1 2024-04-23T02:14:03.148 INFO:tasks.workunit.client.1.smithi192.stdout:3/561: mkdir d1/d3/d17/d1f/dc2 0 2024-04-23T02:14:03.149 INFO:tasks.workunit.client.0.smithi071.stdout:6/503: mkdir d6/d17/d77/d9e 0 2024-04-23T02:14:03.150 INFO:tasks.workunit.client.1.smithi192.stdout:8/754: mkdir d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/d108 0 2024-04-23T02:14:03.151 INFO:tasks.workunit.client.0.smithi071.stdout:3/498: dread d7/d8/d42/d5f/d68/d3b/d4c/f59 [0,4194304] 0 2024-04-23T02:14:03.151 INFO:tasks.workunit.client.1.smithi192.stdout:3/562: read d1/d3/d17/fa8 [3332395,80517] 0 2024-04-23T02:14:03.153 INFO:tasks.workunit.client.0.smithi071.stdout:6/504: mkdir d6/d17/d77/d9f 0 2024-04-23T02:14:03.154 INFO:tasks.workunit.client.0.smithi071.stdout:0/509: write d0/d3/de/d10/f1f [4807186,93360] 0 2024-04-23T02:14:03.155 INFO:tasks.workunit.client.1.smithi192.stdout:8/755: rmdir d1/d12/d7d 39 2024-04-23T02:14:03.156 INFO:tasks.workunit.client.0.smithi071.stdout:3/499: mkdir d7/d8/d42/d5f/d68/dae 0 2024-04-23T02:14:03.157 INFO:tasks.workunit.client.0.smithi071.stdout:5/631: write d3/d1d/d43/f75 [406796,7364] 0 2024-04-23T02:14:03.157 INFO:tasks.workunit.client.1.smithi192.stdout:3/563: unlink d1/d3/d17/d1a/d7c/dab/d6c/d84/d5d/f9c 0 2024-04-23T02:14:03.158 INFO:tasks.workunit.client.0.smithi071.stdout:0/510: chown d0/d3/de/d10/d14/d32/l2a 3 1 2024-04-23T02:14:03.161 INFO:tasks.workunit.client.0.smithi071.stdout:8/554: truncate dc/f57 1014711 0 2024-04-23T02:14:03.162 INFO:tasks.workunit.client.0.smithi071.stdout:5/632: write d3/d1d/d43/d4b/fe0 [380957,61169] 0 2024-04-23T02:14:03.162 INFO:tasks.workunit.client.0.smithi071.stdout:4/632: truncate d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f5d 1885449 0 2024-04-23T02:14:03.162 INFO:tasks.workunit.client.1.smithi192.stdout:1/382: truncate dc/d16/d19/d30/d32/d44/f52 3991749 0 2024-04-23T02:14:03.164 INFO:tasks.workunit.client.0.smithi071.stdout:0/511: creat d0/d3/de/d48/d7e/d8b/fa7 x:0 0 0 2024-04-23T02:14:03.165 INFO:tasks.workunit.client.1.smithi192.stdout:1/383: write dc/d16/d19/d30/d32/f36 [71778,67053] 0 2024-04-23T02:14:03.167 INFO:tasks.workunit.client.0.smithi071.stdout:0/512: rmdir d0/d3/de/d10/d16 39 2024-04-23T02:14:03.168 INFO:tasks.workunit.client.0.smithi071.stdout:8/555: truncate dc/d15/da0/d55/f98 187412 0 2024-04-23T02:14:03.169 INFO:tasks.workunit.client.1.smithi192.stdout:3/564: rmdir d1/d3/d17/d2b/d44/d57/db0 39 2024-04-23T02:14:03.169 INFO:tasks.workunit.client.1.smithi192.stdout:8/756: creat d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/d108/f109 x:0 0 0 2024-04-23T02:14:03.171 INFO:tasks.workunit.client.0.smithi071.stdout:4/633: write d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/f5b [4386544,42430] 0 2024-04-23T02:14:03.172 INFO:tasks.workunit.client.1.smithi192.stdout:8/757: creat d1/d9/d8f/d90/f10a x:0 0 0 2024-04-23T02:14:03.172 INFO:tasks.workunit.client.1.smithi192.stdout:3/565: dread d1/d3/f7a [0,4194304] 0 2024-04-23T02:14:03.173 INFO:tasks.workunit.client.0.smithi071.stdout:3/500: dwrite d7/d8/d42/d5f/d68/d3b/d4c/f63 [0,4194304] 0 2024-04-23T02:14:03.173 INFO:tasks.workunit.client.1.smithi192.stdout:9/530: truncate d0/d9/d44/f45 1395165 0 2024-04-23T02:14:03.176 INFO:tasks.workunit.client.1.smithi192.stdout:3/566: fdatasync d1/d3/d17/d1a/d7c/dab/d6c/d84/d46/f5e 0 2024-04-23T02:14:03.180 INFO:tasks.workunit.client.1.smithi192.stdout:8/758: creat d1/d12/d7d/f10b x:0 0 0 2024-04-23T02:14:03.182 INFO:tasks.workunit.client.0.smithi071.stdout:6/505: truncate d6/d3e/f5b 817004 0 2024-04-23T02:14:03.184 INFO:tasks.workunit.client.1.smithi192.stdout:3/567: creat d1/d3/d6/d7/d29/d3b/da9/d8b/fc3 x:0 0 0 2024-04-23T02:14:03.188 INFO:tasks.workunit.client.1.smithi192.stdout:9/531: dwrite d0/d9/fb [4194304,4194304] 0 2024-04-23T02:14:03.189 INFO:tasks.workunit.client.0.smithi071.stdout:6/506: read - d6/d17/d77/f96 zero size 2024-04-23T02:14:03.190 INFO:tasks.workunit.client.0.smithi071.stdout:5/633: dwrite d3/d1d/daf/d83/fc9 [0,4194304] 0 2024-04-23T02:14:03.191 INFO:tasks.workunit.client.0.smithi071.stdout:3/501: fsync d7/d8/f4e 0 2024-04-23T02:14:03.192 INFO:tasks.workunit.client.0.smithi071.stdout:0/513: write d0/d3/de/d10/d16/f5e [92449,115824] 0 2024-04-23T02:14:03.192 INFO:tasks.workunit.client.1.smithi192.stdout:8/759: write d1/d12/d23/d4b/d73/f43 [1936019,10815] 0 2024-04-23T02:14:03.192 INFO:tasks.workunit.client.0.smithi071.stdout:7/498: dwrite d3/d17/d41/d34/f3c [0,4194304] 0 2024-04-23T02:14:03.193 INFO:tasks.workunit.client.1.smithi192.stdout:9/532: readlink d0/d9/d44/d4f/l40 0 2024-04-23T02:14:03.196 INFO:tasks.workunit.client.0.smithi071.stdout:6/507: rmdir d6/d1f/d4e 39 2024-04-23T02:14:03.196 INFO:tasks.workunit.client.0.smithi071.stdout:8/556: dwrite dc/d15/da0/f14 [4194304,4194304] 0 2024-04-23T02:14:03.200 INFO:tasks.workunit.client.1.smithi192.stdout:1/384: dwrite fa [0,4194304] 0 2024-04-23T02:14:03.203 INFO:tasks.workunit.client.1.smithi192.stdout:1/385: write dc/fd [1109675,70606] 0 2024-04-23T02:14:03.205 INFO:tasks.workunit.client.0.smithi071.stdout:3/502: write d7/d8/d42/d5f/d68/d3b/f76 [604609,26071] 0 2024-04-23T02:14:03.207 INFO:tasks.workunit.client.0.smithi071.stdout:0/514: write d0/d3/de/d10/d14/d32/fa4 [725225,108351] 0 2024-04-23T02:14:03.208 INFO:tasks.workunit.client.0.smithi071.stdout:5/634: fsync d3/d1d/daf/f98 0 2024-04-23T02:14:03.210 INFO:tasks.workunit.client.1.smithi192.stdout:8/760: mknod d1/d12/d23/d4b/d73/d7e/d88/c10c 0 2024-04-23T02:14:03.210 INFO:tasks.workunit.client.1.smithi192.stdout:9/533: creat d0/d9/d44/d51/d64/fb6 x:0 0 0 2024-04-23T02:14:03.212 INFO:tasks.workunit.client.1.smithi192.stdout:8/761: stat d1/d9/d3e/fb2 0 2024-04-23T02:14:03.212 INFO:tasks.workunit.client.0.smithi071.stdout:4/634: dread d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/ff [0,4194304] 0 2024-04-23T02:14:03.212 INFO:tasks.workunit.client.0.smithi071.stdout:6/508: rmdir d6/d3e 39 2024-04-23T02:14:03.213 INFO:tasks.workunit.client.1.smithi192.stdout:1/386: creat dc/d14/d15/d35/f73 x:0 0 0 2024-04-23T02:14:03.213 INFO:tasks.workunit.client.0.smithi071.stdout:3/503: symlink d7/d8/d42/d5f/d68/d3b/d4c/d74/laf 0 2024-04-23T02:14:03.215 INFO:tasks.workunit.client.1.smithi192.stdout:3/568: write d1/d3/d17/d1f/d33/f8d [4584157,30136] 0 2024-04-23T02:14:03.215 INFO:tasks.workunit.client.1.smithi192.stdout:8/762: creat d1/de1/f10d x:0 0 0 2024-04-23T02:14:03.216 INFO:tasks.workunit.client.1.smithi192.stdout:1/387: write dc/d14/d15/d35/f49 [240140,59075] 0 2024-04-23T02:14:03.216 INFO:tasks.workunit.client.0.smithi071.stdout:6/509: stat d6/d17/c86 0 2024-04-23T02:14:03.217 INFO:tasks.workunit.client.1.smithi192.stdout:8/763: stat d1/de1/ff1 0 2024-04-23T02:14:03.217 INFO:tasks.workunit.client.0.smithi071.stdout:0/515: mkdir d0/d3/de/d10/d14/d32/da8 0 2024-04-23T02:14:03.219 INFO:tasks.workunit.client.1.smithi192.stdout:3/569: mknod d1/d3/d6/d7/d29/d3b/cc4 0 2024-04-23T02:14:03.219 INFO:tasks.workunit.client.1.smithi192.stdout:8/764: write d1/fc9 [56166,63501] 0 2024-04-23T02:14:03.221 INFO:tasks.workunit.client.1.smithi192.stdout:9/534: mknod d0/d9/d44/d51/d5b/d96/da0/db4/cb7 0 2024-04-23T02:14:03.223 INFO:tasks.workunit.client.0.smithi071.stdout:4/635: dwrite d2/d2a/d2d/d39/d42/d49/d67/f8f [0,4194304] 0 2024-04-23T02:14:03.223 INFO:tasks.workunit.client.0.smithi071.stdout:0/516: getdents d0/d92 0 2024-04-23T02:14:03.224 INFO:tasks.workunit.client.1.smithi192.stdout:9/535: creat d0/d98/fb8 x:0 0 0 2024-04-23T02:14:03.224 INFO:tasks.workunit.client.1.smithi192.stdout:1/388: getdents dc/d16/d19 0 2024-04-23T02:14:03.224 INFO:tasks.workunit.client.0.smithi071.stdout:6/510: getdents d6/d17 0 2024-04-23T02:14:03.227 INFO:tasks.workunit.client.1.smithi192.stdout:1/389: mkdir dc/d16/d19/d74 0 2024-04-23T02:14:03.228 INFO:tasks.workunit.client.1.smithi192.stdout:8/765: dread d1/d12/d23/d4b/d73/d45/f80 [0,4194304] 0 2024-04-23T02:14:03.229 INFO:tasks.workunit.client.1.smithi192.stdout:9/536: creat d0/d9/d44/d51/d5b/d61/fb9 x:0 0 0 2024-04-23T02:14:03.229 INFO:tasks.workunit.client.0.smithi071.stdout:3/504: rename d7/d8/d42/d5f/d68/d3b/d4c/d74/lab to d7/d10/lb0 0 2024-04-23T02:14:03.231 INFO:tasks.workunit.client.1.smithi192.stdout:1/390: symlink dc/d16/d19/d30/l75 0 2024-04-23T02:14:03.231 INFO:tasks.workunit.client.1.smithi192.stdout:9/537: chown d0/d9/db3/d9e/lad 30828 1 2024-04-23T02:14:03.232 INFO:tasks.workunit.client.0.smithi071.stdout:0/517: symlink d0/d3/de/la9 0 2024-04-23T02:14:03.234 INFO:tasks.workunit.client.0.smithi071.stdout:6/511: chown d6/d3e/d3f/l4b 65806789 1 2024-04-23T02:14:03.234 INFO:tasks.workunit.client.0.smithi071.stdout:5/635: dwrite d3/d1d/daf/d10/f48 [0,4194304] 0 2024-04-23T02:14:03.234 INFO:tasks.workunit.client.0.smithi071.stdout:3/505: mknod d7/d10/d38/cb1 0 2024-04-23T02:14:03.239 INFO:tasks.workunit.client.0.smithi071.stdout:3/506: write d7/d8/d42/d5f/d68/d3b/d4c/d74/f92 [908429,85799] 0 2024-04-23T02:14:03.240 INFO:tasks.workunit.client.0.smithi071.stdout:6/512: unlink d6/d22/f35 0 2024-04-23T02:14:03.241 INFO:tasks.workunit.client.0.smithi071.stdout:4/636: creat d2/d2a/d2d/d39/d42/d49/d67/d7e/dae/dab/fca x:0 0 0 2024-04-23T02:14:03.244 INFO:tasks.workunit.client.0.smithi071.stdout:3/507: fdatasync d7/d8/d42/d5f/f5b 0 2024-04-23T02:14:03.245 INFO:tasks.workunit.client.0.smithi071.stdout:5/636: link d3/d1d/d43/dd2/f91 d3/da/d1a/d1c/d46/fec 0 2024-04-23T02:14:03.247 INFO:tasks.workunit.client.0.smithi071.stdout:4/637: readlink d2/d2a/d2d/d39/d42/d49/d67/l68 0 2024-04-23T02:14:03.250 INFO:tasks.workunit.client.0.smithi071.stdout:7/499: dread d3/d17/f5a [0,4194304] 0 2024-04-23T02:14:03.250 INFO:tasks.workunit.client.1.smithi192.stdout:8/766: truncate d1/d12/d23/d4b/d73/d7e/d88/fa4 1461423 0 2024-04-23T02:14:03.252 INFO:tasks.workunit.client.0.smithi071.stdout:7/500: write d3/d17/d41/d37/f75 [6205141,7813] 0 2024-04-23T02:14:03.252 INFO:tasks.workunit.client.0.smithi071.stdout:8/557: dread dc/d3a/f47 [0,4194304] 0 2024-04-23T02:14:03.254 INFO:tasks.workunit.client.1.smithi192.stdout:1/391: dread f3 [4194304,4194304] 0 2024-04-23T02:14:03.254 INFO:tasks.workunit.client.1.smithi192.stdout:8/767: getdents d1/d9/d8f 0 2024-04-23T02:14:03.255 INFO:tasks.workunit.client.0.smithi071.stdout:7/501: mknod d3/d17/d1c/d2a/d73/d83/cac 0 2024-04-23T02:14:03.256 INFO:tasks.workunit.client.1.smithi192.stdout:8/768: write d1/d9/fee [2259922,99252] 0 2024-04-23T02:14:03.259 INFO:tasks.workunit.client.0.smithi071.stdout:7/502: stat d3/c30 0 2024-04-23T02:14:03.259 INFO:tasks.workunit.client.1.smithi192.stdout:8/769: chown d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/fba 142522786 1 2024-04-23T02:14:03.262 INFO:tasks.workunit.client.1.smithi192.stdout:8/770: mknod d1/d9/d3e/c10e 0 2024-04-23T02:14:03.264 INFO:tasks.workunit.client.0.smithi071.stdout:7/503: rename d3/d17/d41/d34/fa3 to d3/fad 0 2024-04-23T02:14:03.264 INFO:tasks.workunit.client.1.smithi192.stdout:8/771: mkdir d1/d12/d23/d4b/d73/dda/d10f 0 2024-04-23T02:14:03.265 INFO:tasks.workunit.client.0.smithi071.stdout:8/558: dwrite dc/f54 [0,4194304] 0 2024-04-23T02:14:03.265 INFO:tasks.workunit.client.1.smithi192.stdout:8/772: stat d1/d12/d23/d4b/d73/d4c/f64 0 2024-04-23T02:14:03.267 INFO:tasks.workunit.client.1.smithi192.stdout:8/773: creat d1/d12/d23/d4b/d73/d7e/d88/da9/f110 x:0 0 0 2024-04-23T02:14:03.268 INFO:tasks.workunit.client.0.smithi071.stdout:5/637: write d3/d8f/fa2 [610622,62784] 0 2024-04-23T02:14:03.269 INFO:tasks.workunit.client.0.smithi071.stdout:7/504: write d3/d17/d41/f33 [2938005,10975] 0 2024-04-23T02:14:03.269 INFO:tasks.workunit.client.0.smithi071.stdout:8/559: symlink dc/d56/lab 0 2024-04-23T02:14:03.270 INFO:tasks.workunit.client.1.smithi192.stdout:8/774: creat d1/d12/d7d/dd2/de3/f111 x:0 0 0 2024-04-23T02:14:03.270 INFO:tasks.workunit.client.1.smithi192.stdout:6/677: sync 2024-04-23T02:14:03.273 INFO:tasks.workunit.client.0.smithi071.stdout:7/505: mkdir d3/dae 0 2024-04-23T02:14:03.274 INFO:tasks.workunit.client.1.smithi192.stdout:1/392: dread dc/d16/d19/d30/d32/f4d [0,4194304] 0 2024-04-23T02:14:03.274 INFO:tasks.workunit.client.0.smithi071.stdout:5/638: mkdir d3/da/d1a/ded 0 2024-04-23T02:14:03.275 INFO:tasks.workunit.client.1.smithi192.stdout:8/775: write d1/d12/d7d/fd5 [464517,3965] 0 2024-04-23T02:14:03.276 INFO:tasks.workunit.client.1.smithi192.stdout:1/393: chown dc/d16/d19/d30/d32/d62 30028614 1 2024-04-23T02:14:03.278 INFO:tasks.workunit.client.0.smithi071.stdout:8/560: mknod dc/d15/d5d/cac 0 2024-04-23T02:14:03.279 INFO:tasks.workunit.client.1.smithi192.stdout:1/394: write dc/d14/f21 [975791,1292] 0 2024-04-23T02:14:03.280 INFO:tasks.workunit.client.0.smithi071.stdout:6/513: dread d6/d17/d77/f84 [4194304,4194304] 0 2024-04-23T02:14:03.280 INFO:tasks.workunit.client.0.smithi071.stdout:0/518: dwrite d0/d3/de/f28 [0,4194304] 0 2024-04-23T02:14:03.281 INFO:tasks.workunit.client.1.smithi192.stdout:6/678: dwrite d3/f5c [0,4194304] 0 2024-04-23T02:14:03.284 INFO:tasks.workunit.client.1.smithi192.stdout:1/395: creat dc/f76 x:0 0 0 2024-04-23T02:14:03.286 INFO:tasks.workunit.client.1.smithi192.stdout:8/776: dwrite d1/de1/f10d [0,4194304] 0 2024-04-23T02:14:03.287 INFO:tasks.workunit.client.0.smithi071.stdout:8/561: readlink dc/d15/da0/l4e 0 2024-04-23T02:14:03.287 INFO:tasks.workunit.client.0.smithi071.stdout:3/508: dwrite d7/d8/d42/f87 [0,4194304] 0 2024-04-23T02:14:03.289 INFO:tasks.workunit.client.0.smithi071.stdout:5/639: creat d3/d1d/daf/d10/da5/fee x:0 0 0 2024-04-23T02:14:03.289 INFO:tasks.workunit.client.0.smithi071.stdout:7/506: unlink d3/d17/d41/d34/d59/f67 0 2024-04-23T02:14:03.292 INFO:tasks.workunit.client.0.smithi071.stdout:3/509: stat d7/d8/d42 0 2024-04-23T02:14:03.294 INFO:tasks.workunit.client.1.smithi192.stdout:1/396: dwrite dc/d14/d23/f50 [8388608,4194304] 0 2024-04-23T02:14:03.299 INFO:tasks.workunit.client.0.smithi071.stdout:0/519: symlink d0/d3/de/d10/d14/d32/d37/laa 0 2024-04-23T02:14:03.300 INFO:tasks.workunit.client.1.smithi192.stdout:1/397: dread dc/d16/d19/d30/d32/f36 [0,4194304] 0 2024-04-23T02:14:03.301 INFO:tasks.workunit.client.1.smithi192.stdout:6/679: write d3/d18/f2d [2178537,101052] 0 2024-04-23T02:14:03.303 INFO:tasks.workunit.client.1.smithi192.stdout:1/398: stat dc/d14/d15/c20 0 2024-04-23T02:14:03.310 INFO:tasks.workunit.client.1.smithi192.stdout:1/399: mkdir dc/d16/d19/d74/d77 0 2024-04-23T02:14:03.311 INFO:tasks.workunit.client.0.smithi071.stdout:7/507: creat d3/d17/d41/d34/d39/d3b/da7/faf x:0 0 0 2024-04-23T02:14:03.311 INFO:tasks.workunit.client.0.smithi071.stdout:0/520: creat d0/d76/fab x:0 0 0 2024-04-23T02:14:03.314 INFO:tasks.workunit.client.0.smithi071.stdout:7/508: symlink d3/d17/d1c/d31/lb0 0 2024-04-23T02:14:03.315 INFO:tasks.workunit.client.0.smithi071.stdout:6/514: dwrite d6/d1f/d5d/f67 [0,4194304] 0 2024-04-23T02:14:03.316 INFO:tasks.workunit.client.0.smithi071.stdout:3/510: dwrite d7/d8/d42/d80/d88/f57 [0,4194304] 0 2024-04-23T02:14:03.317 INFO:tasks.workunit.client.0.smithi071.stdout:6/515: stat d6/fc 0 2024-04-23T02:14:03.319 INFO:tasks.workunit.client.0.smithi071.stdout:5/640: dwrite d3/d34/f6a [0,4194304] 0 2024-04-23T02:14:03.320 INFO:tasks.workunit.client.0.smithi071.stdout:7/509: write d3/d17/f1f [2091728,87411] 0 2024-04-23T02:14:03.323 INFO:tasks.workunit.client.0.smithi071.stdout:3/511: write d7/d8/d42/d5f/d68/f84 [2717642,95122] 0 2024-04-23T02:14:03.324 INFO:tasks.workunit.client.0.smithi071.stdout:6/516: mknod d6/d1f/d46/ca0 0 2024-04-23T02:14:03.328 INFO:tasks.workunit.client.0.smithi071.stdout:5/641: mkdir d3/da/def 0 2024-04-23T02:14:03.330 INFO:tasks.workunit.client.1.smithi192.stdout:6/680: dread d3/d15/f4b [0,4194304] 0 2024-04-23T02:14:03.332 INFO:tasks.workunit.client.1.smithi192.stdout:6/681: truncate d3/d18/dd4/d8d/fba 1467945 0 2024-04-23T02:14:03.336 INFO:tasks.workunit.client.1.smithi192.stdout:6/682: mkdir d3/d15/d1b/d66/d80/de2 0 2024-04-23T02:14:03.338 INFO:tasks.workunit.client.1.smithi192.stdout:6/683: symlink d3/d15/d1b/le3 0 2024-04-23T02:14:03.339 INFO:tasks.workunit.client.0.smithi071.stdout:8/562: dread dc/f46 [0,4194304] 0 2024-04-23T02:14:03.340 INFO:tasks.workunit.client.0.smithi071.stdout:8/563: write dc/d15/da0/f3d [1543333,50738] 0 2024-04-23T02:14:03.345 INFO:tasks.workunit.client.1.smithi192.stdout:8/777: dread d1/d12/fa5 [0,4194304] 0 2024-04-23T02:14:03.346 INFO:tasks.workunit.client.0.smithi071.stdout:8/564: dwrite dc/d15/da0/f14 [8388608,4194304] 0 2024-04-23T02:14:03.348 INFO:tasks.workunit.client.0.smithi071.stdout:8/565: chown dc/d15/da0/d32/f4b 7266212 1 2024-04-23T02:14:03.349 INFO:tasks.workunit.client.1.smithi192.stdout:1/400: dread dc/d14/d23/f50 [0,4194304] 0 2024-04-23T02:14:03.350 INFO:tasks.workunit.client.1.smithi192.stdout:1/401: readlink dc/d16/d19/d30/d32/l48 0 2024-04-23T02:14:03.354 INFO:tasks.workunit.client.1.smithi192.stdout:8/778: link d1/d12/d23/d4b/d73/d5d/c94 d1/d9/c112 0 2024-04-23T02:14:03.355 INFO:tasks.workunit.client.1.smithi192.stdout:1/402: symlink dc/d14/l78 0 2024-04-23T02:14:03.359 INFO:tasks.workunit.client.1.smithi192.stdout:1/403: fsync dc/d16/d19/d30/d32/d44/f52 0 2024-04-23T02:14:03.360 INFO:tasks.workunit.client.0.smithi071.stdout:6/517: dread d6/d22/f28 [0,4194304] 0 2024-04-23T02:14:03.361 INFO:tasks.workunit.client.1.smithi192.stdout:8/779: dwrite d1/d12/d7d/fd5 [0,4194304] 0 2024-04-23T02:14:03.368 INFO:tasks.workunit.client.1.smithi192.stdout:1/404: getdents dc/d14/d23/d5e 0 2024-04-23T02:14:03.369 INFO:tasks.workunit.client.1.smithi192.stdout:8/780: getdents d1/d9/d3e/d59/d8a/daa 0 2024-04-23T02:14:03.370 INFO:tasks.workunit.client.1.smithi192.stdout:1/405: write dc/f10 [2874517,48349] 0 2024-04-23T02:14:03.374 INFO:tasks.workunit.client.1.smithi192.stdout:1/406: getdents dc/d14/d23/d5e 0 2024-04-23T02:14:03.376 INFO:tasks.workunit.client.1.smithi192.stdout:1/407: creat dc/d16/d19/f79 x:0 0 0 2024-04-23T02:14:03.378 INFO:tasks.workunit.client.1.smithi192.stdout:1/408: dread - dc/d16/d19/d30/d32/d34/f56 zero size 2024-04-23T02:14:03.380 INFO:tasks.workunit.client.1.smithi192.stdout:1/409: chown dc/d16/d19/d30/d32/d44/l5f 13 1 2024-04-23T02:14:03.380 INFO:tasks.workunit.client.1.smithi192.stdout:1/410: read dc/d16/d19/f63 [886323,75878] 0 2024-04-23T02:14:03.382 INFO:tasks.workunit.client.1.smithi192.stdout:1/411: rmdir dc/d16/d19/d30/d32/d34 39 2024-04-23T02:14:03.386 INFO:tasks.workunit.client.1.smithi192.stdout:8/781: dread d1/d12/d23/d4b/d73/d45/d46/f6d [0,4194304] 0 2024-04-23T02:14:03.386 INFO:tasks.workunit.client.1.smithi192.stdout:1/412: rename dc/d14/d15/d35/c4e to dc/d16/d19/c7a 0 2024-04-23T02:14:03.389 INFO:tasks.workunit.client.1.smithi192.stdout:1/413: symlink dc/d16/d19/l7b 0 2024-04-23T02:14:03.391 INFO:tasks.workunit.client.1.smithi192.stdout:8/782: dwrite d1/df/f50 [0,4194304] 0 2024-04-23T02:14:03.392 INFO:tasks.workunit.client.1.smithi192.stdout:1/414: symlink dc/d16/d19/d74/d77/l7c 0 2024-04-23T02:14:03.393 INFO:tasks.workunit.client.1.smithi192.stdout:1/415: stat dc/d14/d15/l25 0 2024-04-23T02:14:03.394 INFO:tasks.workunit.client.1.smithi192.stdout:8/783: mkdir d1/d12/d23/d4b/d73/dda/d113 0 2024-04-23T02:14:03.396 INFO:tasks.workunit.client.1.smithi192.stdout:8/784: fdatasync d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/fe0 0 2024-04-23T02:14:03.398 INFO:tasks.workunit.client.1.smithi192.stdout:1/416: rename dc/d16/d19/f63 to dc/d14/d23/f7d 0 2024-04-23T02:14:03.410 INFO:tasks.workunit.client.1.smithi192.stdout:1/417: truncate dc/d16/f1f 2637684 0 2024-04-23T02:14:03.410 INFO:tasks.workunit.client.1.smithi192.stdout:8/785: dwrite d1/df/ff4 [0,4194304] 0 2024-04-23T02:14:03.412 INFO:tasks.workunit.client.1.smithi192.stdout:8/786: stat d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/fec 0 2024-04-23T02:14:03.414 INFO:tasks.workunit.client.1.smithi192.stdout:1/418: mknod dc/d14/d23/c7e 0 2024-04-23T02:14:03.419 INFO:tasks.workunit.client.0.smithi071.stdout:7/510: unlink d3/d17/d41/d37/d4f/c6d 0 2024-04-23T02:14:03.419 INFO:tasks.workunit.client.1.smithi192.stdout:8/787: rename d1/d12/d7d/dd2/de3 to d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/da3/d114 0 2024-04-23T02:14:03.421 INFO:tasks.workunit.client.1.smithi192.stdout:1/419: dwrite dc/d16/d19/d30/f3f [0,4194304] 0 2024-04-23T02:14:03.423 INFO:tasks.workunit.client.1.smithi192.stdout:1/420: chown dc/d16/d19/d30/d32/d44/f53 19388865 1 2024-04-23T02:14:03.425 INFO:tasks.workunit.client.0.smithi071.stdout:6/518: creat d6/d17/d77/d85/fa1 x:0 0 0 2024-04-23T02:14:03.426 INFO:tasks.workunit.client.0.smithi071.stdout:6/519: read - d6/d3e/f8d zero size 2024-04-23T02:14:03.427 INFO:tasks.workunit.client.1.smithi192.stdout:8/788: rename d1/d12/f1e to d1/d9/d3e/d59/d8a/f115 0 2024-04-23T02:14:03.428 INFO:tasks.workunit.client.0.smithi071.stdout:0/521: rmdir d0/d76 39 2024-04-23T02:14:03.430 INFO:tasks.workunit.client.1.smithi192.stdout:1/421: mknod dc/d16/d19/d30/d32/d62/c7f 0 2024-04-23T02:14:03.432 INFO:tasks.workunit.client.0.smithi071.stdout:8/566: mkdir dc/d15/da0/d8d/d9a/dad 0 2024-04-23T02:14:03.433 INFO:tasks.workunit.client.0.smithi071.stdout:6/520: mkdir d6/d17/d77/da2 0 2024-04-23T02:14:03.433 INFO:tasks.workunit.client.1.smithi192.stdout:8/789: dread d1/d9/d3e/d59/d8a/f8d [0,4194304] 0 2024-04-23T02:14:03.435 INFO:tasks.workunit.client.0.smithi071.stdout:0/522: mkdir d0/d3/de/d10/d14/d32/dac 0 2024-04-23T02:14:03.437 INFO:tasks.workunit.client.1.smithi192.stdout:8/790: dread d1/d12/d23/fd3 [4194304,4194304] 0 2024-04-23T02:14:03.439 INFO:tasks.workunit.client.0.smithi071.stdout:6/521: creat d6/d1f/d4e/fa3 x:0 0 0 2024-04-23T02:14:03.440 INFO:tasks.workunit.client.0.smithi071.stdout:6/522: fsync d6/d1f/f68 0 2024-04-23T02:14:03.442 INFO:tasks.workunit.client.1.smithi192.stdout:8/791: unlink d1/d12/d23/d4b/d73/d7e/d88/da9/lbc 0 2024-04-23T02:14:03.442 INFO:tasks.workunit.client.0.smithi071.stdout:5/642: truncate d3/d1d/d43/d4b/f72 43834 0 2024-04-23T02:14:03.443 INFO:tasks.workunit.client.0.smithi071.stdout:6/523: write d6/d17/d77/d85/fa1 [724448,57134] 0 2024-04-23T02:14:03.443 INFO:tasks.workunit.client.0.smithi071.stdout:3/512: truncate d7/d8/d42/d5f/d68/d3b/d4c/fad 495945 0 2024-04-23T02:14:03.444 INFO:tasks.workunit.client.0.smithi071.stdout:5/643: readlink d3/d86/lb1 0 2024-04-23T02:14:03.444 INFO:tasks.workunit.client.0.smithi071.stdout:7/511: dread d3/d17/d1c/f26 [0,4194304] 0 2024-04-23T02:14:03.444 INFO:tasks.workunit.client.0.smithi071.stdout:0/523: dwrite d0/d3/de/d10/d16/f55 [0,4194304] 0 2024-04-23T02:14:03.447 INFO:tasks.workunit.client.0.smithi071.stdout:3/513: chown d7/d8/d42/d5f/f5b 1533 1 2024-04-23T02:14:03.449 INFO:tasks.workunit.client.0.smithi071.stdout:0/524: chown d0/d3/de/d10/d14/d32/d44/d31/l46 249886 1 2024-04-23T02:14:03.449 INFO:tasks.workunit.client.0.smithi071.stdout:7/512: creat d3/d17/d41/d34/d39/d3b/fb1 x:0 0 0 2024-04-23T02:14:03.452 INFO:tasks.workunit.client.0.smithi071.stdout:7/513: chown d3/d17/d41/d34 1 1 2024-04-23T02:14:03.453 INFO:tasks.workunit.client.0.smithi071.stdout:5/644: truncate d3/da/d1a/d1c/d46/fec 498305 0 2024-04-23T02:14:03.453 INFO:tasks.workunit.client.0.smithi071.stdout:8/567: dwrite dc/f28 [4194304,4194304] 0 2024-04-23T02:14:03.456 INFO:tasks.workunit.client.1.smithi192.stdout:8/792: dread d1/d9/d3e/fb2 [0,4194304] 0 2024-04-23T02:14:03.457 INFO:tasks.workunit.client.0.smithi071.stdout:8/568: chown l3 2062205 1 2024-04-23T02:14:03.458 INFO:tasks.workunit.client.1.smithi192.stdout:8/793: dread - d1/d12/d23/d4b/d73/d7e/f102 zero size 2024-04-23T02:14:03.459 INFO:tasks.workunit.client.0.smithi071.stdout:0/525: unlink d0/d3/de/d10/d14/d84/l90 0 2024-04-23T02:14:03.460 INFO:tasks.workunit.client.1.smithi192.stdout:8/794: mknod d1/d12/d23/d4b/d73/d7e/d88/da9/db1/c116 0 2024-04-23T02:14:03.460 INFO:tasks.workunit.client.0.smithi071.stdout:3/514: dwrite d7/d8/d42/fac [0,4194304] 0 2024-04-23T02:14:03.464 INFO:tasks.workunit.client.0.smithi071.stdout:8/569: mknod dc/d15/da0/d32/cae 0 2024-04-23T02:14:03.464 INFO:tasks.workunit.client.0.smithi071.stdout:0/526: truncate d0/f6b 1295403 0 2024-04-23T02:14:03.465 INFO:tasks.workunit.client.0.smithi071.stdout:3/515: mkdir d7/da5/db2 0 2024-04-23T02:14:03.468 INFO:tasks.workunit.client.0.smithi071.stdout:0/527: mknod d0/d3/de/d48/d7e/d8b/cad 0 2024-04-23T02:14:03.469 INFO:tasks.workunit.client.0.smithi071.stdout:3/516: truncate d7/d22/da7/f9b 422094 0 2024-04-23T02:14:03.469 INFO:tasks.workunit.client.0.smithi071.stdout:8/570: creat dc/d15/da0/d8d/faf x:0 0 0 2024-04-23T02:14:03.470 INFO:tasks.workunit.client.0.smithi071.stdout:5/645: dwrite d3/f1f [4194304,4194304] 0 2024-04-23T02:14:03.477 INFO:tasks.workunit.client.0.smithi071.stdout:3/517: rename d7/d10/d64 to d7/d8/d42/d5f/d68/d3b/d4c/d8b/db3 0 2024-04-23T02:14:03.478 INFO:tasks.workunit.client.0.smithi071.stdout:3/518: fdatasync d7/d8/f11 0 2024-04-23T02:14:03.480 INFO:tasks.workunit.client.0.smithi071.stdout:5/646: unlink d3/d1d/daf/d10/da5/d5f/fd1 0 2024-04-23T02:14:03.482 INFO:tasks.workunit.client.1.smithi192.stdout:8/795: write d1/d12/d23/d2c/d4d/fcf [100174,72207] 0 2024-04-23T02:14:03.482 INFO:tasks.workunit.client.0.smithi071.stdout:0/528: rename d0/d3/de/d10/d14/d32/d37/f53 to d0/d3/de/d10/d14/d32/dac/fae 0 2024-04-23T02:14:03.485 INFO:tasks.workunit.client.1.smithi192.stdout:8/796: creat d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/d108/f117 x:0 0 0 2024-04-23T02:14:03.494 INFO:tasks.workunit.client.0.smithi071.stdout:0/529: rename d0/l69 to d0/d3/de/laf 0 2024-04-23T02:14:03.495 INFO:tasks.workunit.client.0.smithi071.stdout:5/647: dwrite d3/d34/d74/faa [0,4194304] 0 2024-04-23T02:14:03.501 INFO:tasks.workunit.client.0.smithi071.stdout:5/648: write d3/d1d/daf/d83/fac [543078,87309] 0 2024-04-23T02:14:03.502 INFO:tasks.workunit.client.0.smithi071.stdout:7/514: dwrite d3/fc [0,4194304] 0 2024-04-23T02:14:03.504 INFO:tasks.workunit.client.0.smithi071.stdout:7/515: chown d3/fc 85727 1 2024-04-23T02:14:03.504 INFO:tasks.workunit.client.0.smithi071.stdout:8/571: dread dc/d15/da0/d30/fa6 [0,4194304] 0 2024-04-23T02:14:03.508 INFO:tasks.workunit.client.0.smithi071.stdout:0/530: dwrite d0/d3/f7a [0,4194304] 0 2024-04-23T02:14:03.508 INFO:tasks.workunit.client.0.smithi071.stdout:5/649: creat d3/d34/d6b/de4/ff0 x:0 0 0 2024-04-23T02:14:03.516 INFO:tasks.workunit.client.0.smithi071.stdout:7/516: read d3/d17/d41/f2f [128982,91402] 0 2024-04-23T02:14:03.517 INFO:tasks.workunit.client.0.smithi071.stdout:8/572: dwrite dc/d15/f39 [0,4194304] 0 2024-04-23T02:14:03.521 INFO:tasks.workunit.client.0.smithi071.stdout:8/573: mkdir dc/d3a/da4/db0 0 2024-04-23T02:14:03.530 INFO:tasks.workunit.client.0.smithi071.stdout:0/531: dwrite d0/d3/de/d10/d14/d32/d37/d26/f4b [0,4194304] 0 2024-04-23T02:14:03.534 INFO:tasks.workunit.client.0.smithi071.stdout:0/532: fdatasync d0/d3/de/d10/d14/d32/d37/fa1 0 2024-04-23T02:14:03.534 INFO:tasks.workunit.client.0.smithi071.stdout:7/517: write d3/d17/d41/d34/f35 [4947547,7034] 0 2024-04-23T02:14:03.542 INFO:tasks.workunit.client.0.smithi071.stdout:8/574: dwrite f6 [0,4194304] 0 2024-04-23T02:14:03.542 INFO:tasks.workunit.client.0.smithi071.stdout:8/575: readlink dc/d3a/l4d 0 2024-04-23T02:14:03.542 INFO:tasks.workunit.client.0.smithi071.stdout:8/576: fsync dc/d56/f61 0 2024-04-23T02:14:03.545 INFO:tasks.workunit.client.0.smithi071.stdout:5/650: dread d3/da/d1a/db8/fca [0,4194304] 0 2024-04-23T02:14:03.545 INFO:tasks.workunit.client.0.smithi071.stdout:7/518: link d3/l44 d3/d17/d41/d37/lb2 0 2024-04-23T02:14:03.547 INFO:tasks.workunit.client.0.smithi071.stdout:5/651: truncate d3/da/d1a/d6d/fd3 901956 0 2024-04-23T02:14:03.553 INFO:tasks.workunit.client.0.smithi071.stdout:0/533: truncate d0/d3/de/d10/d14/d32/dac/fae 2119410 0 2024-04-23T02:14:03.554 INFO:tasks.workunit.client.0.smithi071.stdout:7/519: truncate d3/fc 543506 0 2024-04-23T02:14:03.554 INFO:tasks.workunit.client.0.smithi071.stdout:8/577: write dc/f57 [135058,55812] 0 2024-04-23T02:14:03.557 INFO:tasks.workunit.client.0.smithi071.stdout:8/578: mknod dc/d15/d72/d95/cb1 0 2024-04-23T02:14:03.558 INFO:tasks.workunit.client.0.smithi071.stdout:5/652: dwrite d3/da/d1a/d1c/f41 [0,4194304] 0 2024-04-23T02:14:03.558 INFO:tasks.workunit.client.0.smithi071.stdout:0/534: symlink d0/d3/de/lb0 0 2024-04-23T02:14:03.565 INFO:tasks.workunit.client.0.smithi071.stdout:5/653: mkdir d3/da/d1a/df1 0 2024-04-23T02:14:03.565 INFO:tasks.workunit.client.1.smithi192.stdout:0/699: sync 2024-04-23T02:14:03.565 INFO:tasks.workunit.client.1.smithi192.stdout:7/634: sync 2024-04-23T02:14:03.565 INFO:tasks.workunit.client.1.smithi192.stdout:2/617: sync 2024-04-23T02:14:03.565 INFO:tasks.workunit.client.1.smithi192.stdout:4/718: sync 2024-04-23T02:14:03.565 INFO:tasks.workunit.client.1.smithi192.stdout:5/605: sync 2024-04-23T02:14:03.568 INFO:tasks.workunit.client.0.smithi071.stdout:5/654: chown d3/d1d/d43/d4b/ce3 3 1 2024-04-23T02:14:03.569 INFO:tasks.workunit.client.0.smithi071.stdout:8/579: dwrite dc/d15/da0/f3d [0,4194304] 0 2024-04-23T02:14:03.572 INFO:tasks.workunit.client.0.smithi071.stdout:0/535: dwrite d0/d3/de/d10/d14/d32/d37/d26/f4b [0,4194304] 0 2024-04-23T02:14:03.573 INFO:tasks.workunit.client.1.smithi192.stdout:5/606: write d2/db/d16/d48/d4f/f5c [1399362,74415] 0 2024-04-23T02:14:03.574 INFO:tasks.workunit.client.1.smithi192.stdout:0/700: truncate d6/df/d13/d25/d3a/f62 688723 0 2024-04-23T02:14:03.577 INFO:tasks.workunit.client.1.smithi192.stdout:7/635: dwrite d5/d17/d1c/f3e [0,4194304] 0 2024-04-23T02:14:03.577 INFO:tasks.workunit.client.0.smithi071.stdout:5/655: rename d3/da/d1a/d6d/fd3 to d3/d8f/ff2 0 2024-04-23T02:14:03.579 INFO:tasks.workunit.client.1.smithi192.stdout:5/607: stat d2/c7b 0 2024-04-23T02:14:03.582 INFO:tasks.workunit.client.0.smithi071.stdout:0/536: symlink d0/d3/de/d10/d14/d32/d37/lb1 0 2024-04-23T02:14:03.582 INFO:tasks.workunit.client.1.smithi192.stdout:7/636: symlink d5/d43/lb8 0 2024-04-23T02:14:03.583 INFO:tasks.workunit.client.1.smithi192.stdout:0/701: mkdir d6/df/d35/d8b/d8d/dea 0 2024-04-23T02:14:03.583 INFO:tasks.workunit.client.0.smithi071.stdout:0/537: chown d0/d63/f74 4122031 1 2024-04-23T02:14:03.584 INFO:tasks.workunit.client.1.smithi192.stdout:5/608: truncate d2/d6/d2d/d41/d43/d98/fc0 964011 0 2024-04-23T02:14:03.585 INFO:tasks.workunit.client.1.smithi192.stdout:4/719: truncate de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/d59/fd3 1755533 0 2024-04-23T02:14:03.585 INFO:tasks.workunit.client.1.smithi192.stdout:2/618: getdents d8/d26/d27/d53/da2/dc1 0 2024-04-23T02:14:03.586 INFO:tasks.workunit.client.0.smithi071.stdout:0/538: mkdir d0/d92/db2 0 2024-04-23T02:14:03.587 INFO:tasks.workunit.client.1.smithi192.stdout:0/702: creat d6/df/d13/d25/d26/feb x:0 0 0 2024-04-23T02:14:03.595 INFO:tasks.workunit.client.1.smithi192.stdout:7/637: truncate d5/d17/d1c/d59/d5a/d70/f7e 4884204 0 2024-04-23T02:14:03.596 INFO:tasks.workunit.client.0.smithi071.stdout:0/539: mkdir d0/d3/de/d10/db3 0 2024-04-23T02:14:03.596 INFO:tasks.workunit.client.0.smithi071.stdout:8/580: truncate f6 2289579 0 2024-04-23T02:14:03.596 INFO:tasks.workunit.client.0.smithi071.stdout:7/520: dread d3/d17/d1c/f29 [0,4194304] 0 2024-04-23T02:14:03.596 INFO:tasks.workunit.client.0.smithi071.stdout:5/656: truncate d3/da/f92 719005 0 2024-04-23T02:14:03.596 INFO:tasks.workunit.client.0.smithi071.stdout:8/581: creat dc/d15/da0/d8d/d9a/dad/fb2 x:0 0 0 2024-04-23T02:14:03.597 INFO:tasks.workunit.client.0.smithi071.stdout:8/582: read dc/d3a/f47 [3644517,102306] 0 2024-04-23T02:14:03.597 INFO:tasks.workunit.client.0.smithi071.stdout:0/540: creat d0/d3/d95/fb4 x:0 0 0 2024-04-23T02:14:03.598 INFO:tasks.workunit.client.1.smithi192.stdout:2/619: dwrite d8/f36 [0,4194304] 0 2024-04-23T02:14:03.598 INFO:tasks.workunit.client.1.smithi192.stdout:4/720: dread de/d15/d53/d81/de3/d97/fbf [0,4194304] 0 2024-04-23T02:14:03.599 INFO:tasks.workunit.client.1.smithi192.stdout:5/609: dwrite d2/d6/f8a [0,4194304] 0 2024-04-23T02:14:03.601 INFO:tasks.workunit.client.1.smithi192.stdout:2/620: chown d8/d26/d29/d43/d5d/fb6 3403246 1 2024-04-23T02:14:03.601 INFO:tasks.workunit.client.0.smithi071.stdout:8/583: link dc/d15/da0/d55/l6d dc/d15/da0/d32/d63/lb3 0 2024-04-23T02:14:03.602 INFO:tasks.workunit.client.1.smithi192.stdout:7/638: mkdir d5/d43/d4c/d5c/d67/da6/da9/db9 0 2024-04-23T02:14:03.602 INFO:tasks.workunit.client.0.smithi071.stdout:8/584: fdatasync dc/d15/da0/d30/f52 0 2024-04-23T02:14:03.604 INFO:tasks.workunit.client.1.smithi192.stdout:7/639: read - d5/d17/d1c/d4a/f84 zero size 2024-04-23T02:14:03.607 INFO:tasks.workunit.client.1.smithi192.stdout:5/610: dread d2/d2e/f64 [0,4194304] 0 2024-04-23T02:14:03.607 INFO:tasks.workunit.client.0.smithi071.stdout:8/585: mknod dc/d3a/da4/db0/cb4 0 2024-04-23T02:14:03.608 INFO:tasks.workunit.client.0.smithi071.stdout:0/541: dwrite d0/d3/de/d10/d14/d32/d44/d31/f72 [0,4194304] 0 2024-04-23T02:14:03.608 INFO:tasks.workunit.client.1.smithi192.stdout:5/611: chown d2/db/d2b/f82 12 1 2024-04-23T02:14:03.610 INFO:tasks.workunit.client.1.smithi192.stdout:7/640: write d5/d17/d1c/d59/d5a/d41/d50/f7b [2814598,77091] 0 2024-04-23T02:14:03.612 INFO:tasks.workunit.client.1.smithi192.stdout:7/641: chown d5/d17/d1c/d57/l19 69115 1 2024-04-23T02:14:03.615 INFO:tasks.workunit.client.1.smithi192.stdout:2/621: rename d8/d26/d29/d43/d50/fbf to d8/d26/d29/da4/dbc/fc8 0 2024-04-23T02:14:03.615 INFO:tasks.workunit.client.0.smithi071.stdout:8/586: dread dc/d15/da0/d8d/f9e [4194304,4194304] 0 2024-04-23T02:14:03.616 INFO:tasks.workunit.client.0.smithi071.stdout:0/542: creat d0/d3/de/d10/d14/d84/fb5 x:0 0 0 2024-04-23T02:14:03.618 INFO:tasks.workunit.client.0.smithi071.stdout:8/587: mknod dc/d56/cb5 0 2024-04-23T02:14:03.619 INFO:tasks.workunit.client.1.smithi192.stdout:0/703: dread d6/f6d [0,4194304] 0 2024-04-23T02:14:03.620 INFO:tasks.workunit.client.0.smithi071.stdout:8/588: symlink dc/d15/da0/d55/lb6 0 2024-04-23T02:14:03.621 INFO:tasks.workunit.client.1.smithi192.stdout:4/721: creat de/d15/d53/d81/de3/d74/d92/da0/dca/d34/fed x:0 0 0 2024-04-23T02:14:03.621 INFO:tasks.workunit.client.0.smithi071.stdout:8/589: chown dc/c2f 1847329 1 2024-04-23T02:14:03.622 INFO:tasks.workunit.client.0.smithi071.stdout:0/543: rename d0/d3/de/d48/d7e/d8b/f94 to d0/d3/de/d48/d7e/d8b/fb6 0 2024-04-23T02:14:03.622 INFO:tasks.workunit.client.1.smithi192.stdout:7/642: dread - d5/d43/d4c/f6c zero size 2024-04-23T02:14:03.625 INFO:tasks.workunit.client.0.smithi071.stdout:0/544: creat d0/d3/de/d10/d14/d32/d8a/fb7 x:0 0 0 2024-04-23T02:14:03.625 INFO:tasks.workunit.client.1.smithi192.stdout:0/704: unlink d6/df/c5e 0 2024-04-23T02:14:03.626 INFO:tasks.workunit.client.1.smithi192.stdout:4/722: rmdir de/d15/d19/d1a 39 2024-04-23T02:14:03.627 INFO:tasks.workunit.client.0.smithi071.stdout:2/536: sync 2024-04-23T02:14:03.629 INFO:tasks.workunit.client.1.smithi192.stdout:5/612: getdents d2/d6/d2d/d41/d43/d98 0 2024-04-23T02:14:03.629 INFO:tasks.workunit.client.1.smithi192.stdout:4/723: chown de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d93/le1 12604656 1 2024-04-23T02:14:03.630 INFO:tasks.workunit.client.0.smithi071.stdout:2/537: read d0/d70/f94 [425070,50536] 0 2024-04-23T02:14:03.632 INFO:tasks.workunit.client.1.smithi192.stdout:0/705: rename d6/df/d13/d25/d26/c4e to d6/df/d35/d8b/d8d/dea/cec 0 2024-04-23T02:14:03.635 INFO:tasks.workunit.client.1.smithi192.stdout:5/613: creat d2/db/d16/d48/dc5/fcc x:0 0 0 2024-04-23T02:14:03.635 INFO:tasks.workunit.client.0.smithi071.stdout:7/521: dread d3/d17/d1c/f3f [0,4194304] 0 2024-04-23T02:14:03.636 INFO:tasks.workunit.client.0.smithi071.stdout:8/590: write dc/d15/d2d/f3f [1335910,68277] 0 2024-04-23T02:14:03.637 INFO:tasks.workunit.client.1.smithi192.stdout:4/724: mknod de/d15/d53/d81/de3/d74/d92/da0/dca/d9f/cee 0 2024-04-23T02:14:03.639 INFO:tasks.workunit.client.1.smithi192.stdout:0/706: symlink d6/df/d13/d25/led 0 2024-04-23T02:14:03.640 INFO:tasks.workunit.client.0.smithi071.stdout:8/591: symlink dc/d15/da0/d32/d63/lb7 0 2024-04-23T02:14:03.641 INFO:tasks.workunit.client.1.smithi192.stdout:7/643: truncate d5/d17/d1c/d59/f9a 742038 0 2024-04-23T02:14:03.641 INFO:tasks.workunit.client.0.smithi071.stdout:7/522: rename d3/d17/d1c/d2a/d73/d83/d92/d96 to d3/d51/d87/db3 0 2024-04-23T02:14:03.642 INFO:tasks.workunit.client.1.smithi192.stdout:5/614: symlink d2/db/lcd 0 2024-04-23T02:14:03.642 INFO:tasks.workunit.client.1.smithi192.stdout:7/644: dread d5/d17/d1c/d59/d5a/fa2 [0,4194304] 0 2024-04-23T02:14:03.643 INFO:tasks.workunit.client.0.smithi071.stdout:8/592: link dc/d15/da0/d32/f4b dc/d15/d72/d95/fb8 0 2024-04-23T02:14:03.643 INFO:tasks.workunit.client.1.smithi192.stdout:7/645: chown d5/d43/f6e 193452619 1 2024-04-23T02:14:03.645 INFO:tasks.workunit.client.1.smithi192.stdout:4/725: dread de/d15/d53/d81/de3/f5c [0,4194304] 0 2024-04-23T02:14:03.646 INFO:tasks.workunit.client.0.smithi071.stdout:7/523: getdents d3/d17 0 2024-04-23T02:14:03.646 INFO:tasks.workunit.client.0.smithi071.stdout:7/524: chown d3/d17/d1c/d2a/d73/d83 2144 1 2024-04-23T02:14:03.647 INFO:tasks.workunit.client.0.smithi071.stdout:8/593: link dc/d15/d2d/f3f dc/d15/da0/d32/fb9 0 2024-04-23T02:14:03.648 INFO:tasks.workunit.client.0.smithi071.stdout:7/525: symlink d3/d17/d41/d3a/da5/lb4 0 2024-04-23T02:14:03.649 INFO:tasks.workunit.client.0.smithi071.stdout:8/594: symlink dc/d15/d50/d5f/lba 0 2024-04-23T02:14:03.650 INFO:tasks.workunit.client.1.smithi192.stdout:5/615: dwrite d2/db/f11 [0,4194304] 0 2024-04-23T02:14:03.653 INFO:tasks.workunit.client.1.smithi192.stdout:4/726: chown de/d15/d19/d1a/fc7 21800596 1 2024-04-23T02:14:03.657 INFO:tasks.workunit.client.1.smithi192.stdout:5/616: creat d2/db/d16/fce x:0 0 0 2024-04-23T02:14:03.660 INFO:tasks.workunit.client.1.smithi192.stdout:7/646: truncate d5/d43/f6e 1506881 0 2024-04-23T02:14:03.661 INFO:tasks.workunit.client.1.smithi192.stdout:5/617: mknod d2/ccf 0 2024-04-23T02:14:03.661 INFO:tasks.workunit.client.1.smithi192.stdout:0/707: dwrite d6/df/d13/d25/d3a/da6/fe9 [4194304,4194304] 0 2024-04-23T02:14:03.662 INFO:tasks.workunit.client.0.smithi071.stdout:7/526: write d3/d17/d41/d3a/f58 [1029694,2465] 0 2024-04-23T02:14:03.663 INFO:tasks.workunit.client.0.smithi071.stdout:8/595: write dc/d15/da0/f1c [2695667,130837] 0 2024-04-23T02:14:03.663 INFO:tasks.workunit.client.1.smithi192.stdout:5/618: stat d2/db/d2b/fa5 0 2024-04-23T02:14:03.664 INFO:tasks.workunit.client.1.smithi192.stdout:0/708: write d6/f10 [4221306,125000] 0 2024-04-23T02:14:03.665 INFO:tasks.workunit.client.1.smithi192.stdout:4/727: dwrite de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/f91 [0,4194304] 0 2024-04-23T02:14:03.667 INFO:tasks.workunit.client.1.smithi192.stdout:5/619: chown d2/db/d16/d48/dc5/fcc 1757724659 1 2024-04-23T02:14:03.669 INFO:tasks.workunit.client.1.smithi192.stdout:4/728: chown de/c14 48519 1 2024-04-23T02:14:03.672 INFO:tasks.workunit.client.0.smithi071.stdout:7/527: truncate d3/d17/d1c/d31/f5d 70642 0 2024-04-23T02:14:03.673 INFO:tasks.workunit.client.0.smithi071.stdout:7/528: write d3/d17/d1c/f64 [3096588,88582] 0 2024-04-23T02:14:03.677 INFO:tasks.workunit.client.0.smithi071.stdout:7/529: creat d3/fb5 x:0 0 0 2024-04-23T02:14:03.677 INFO:tasks.workunit.client.1.smithi192.stdout:4/729: link de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d93/fc6 de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d93/fef 0 2024-04-23T02:14:03.678 INFO:tasks.workunit.client.0.smithi071.stdout:7/530: write d3/d17/d41/f2f [4558995,80658] 0 2024-04-23T02:14:03.680 INFO:tasks.workunit.client.0.smithi071.stdout:7/531: dread - d3/d17/d41/d34/d39/d3b/da7/faf zero size 2024-04-23T02:14:03.683 INFO:tasks.workunit.client.1.smithi192.stdout:4/730: readlink de/d15/d53/d81/de3/d74/d92/da0/dca/db6/lbc 0 2024-04-23T02:14:03.683 INFO:tasks.workunit.client.0.smithi071.stdout:7/532: mknod d3/d17/d1c/d31/cb6 0 2024-04-23T02:14:03.685 INFO:tasks.workunit.client.1.smithi192.stdout:7/647: truncate d5/d43/d4c/d5c/d67/f82 744281 0 2024-04-23T02:14:03.685 INFO:tasks.workunit.client.0.smithi071.stdout:7/533: mkdir d3/dae/db7 0 2024-04-23T02:14:03.687 INFO:tasks.workunit.client.1.smithi192.stdout:0/709: dread d6/df/d56/d85/f2e [0,4194304] 0 2024-04-23T02:14:03.692 INFO:tasks.workunit.client.0.smithi071.stdout:7/534: truncate d3/d17/d41/d37/f75 5927713 0 2024-04-23T02:14:03.693 INFO:tasks.workunit.client.1.smithi192.stdout:7/648: creat d5/d17/d1c/d59/d5a/d41/d50/fba x:0 0 0 2024-04-23T02:14:03.695 INFO:tasks.workunit.client.1.smithi192.stdout:7/649: chown d5/d17/d1c/d26/l9c 1729998 1 2024-04-23T02:14:03.695 INFO:tasks.workunit.client.1.smithi192.stdout:4/731: write de/d15/d53/d81/de3/d74/d92/da0/dca/f54 [2656905,51524] 0 2024-04-23T02:14:03.696 INFO:tasks.workunit.client.1.smithi192.stdout:5/620: dread d2/db/fad [0,4194304] 0 2024-04-23T02:14:03.699 INFO:tasks.workunit.client.0.smithi071.stdout:7/535: write d3/d17/d41/d37/f75 [3792956,12765] 0 2024-04-23T02:14:03.700 INFO:tasks.workunit.client.1.smithi192.stdout:7/650: mkdir d5/d17/d1c/d57/dbb 0 2024-04-23T02:14:03.701 INFO:tasks.workunit.client.0.smithi071.stdout:7/536: write d3/d17/d1c/f26 [6665255,93792] 0 2024-04-23T02:14:03.701 INFO:tasks.workunit.client.1.smithi192.stdout:5/621: mknod d2/d6/d14/d60/cd0 0 2024-04-23T02:14:03.703 INFO:tasks.workunit.client.1.smithi192.stdout:0/710: dwrite d6/df/f83 [0,4194304] 0 2024-04-23T02:14:03.704 INFO:tasks.workunit.client.1.smithi192.stdout:5/622: dread - d2/db/d16/d1b/f79 zero size 2024-04-23T02:14:03.707 INFO:tasks.workunit.client.0.smithi071.stdout:7/537: truncate d3/d17/d1c/f45 5099785 0 2024-04-23T02:14:03.707 INFO:tasks.workunit.client.1.smithi192.stdout:5/623: mkdir d2/db/d16/d48/d4f/dd1 0 2024-04-23T02:14:03.708 INFO:tasks.workunit.client.0.smithi071.stdout:7/538: mknod d3/d51/cb8 0 2024-04-23T02:14:03.709 INFO:tasks.workunit.client.1.smithi192.stdout:7/651: unlink d5/d17/d1c/d59/f9a 0 2024-04-23T02:14:03.709 INFO:tasks.workunit.client.1.smithi192.stdout:0/711: mknod d6/df/d47/d55/dae/cee 0 2024-04-23T02:14:03.710 INFO:tasks.workunit.client.0.smithi071.stdout:7/539: creat d3/d17/fb9 x:0 0 0 2024-04-23T02:14:03.711 INFO:tasks.workunit.client.1.smithi192.stdout:4/732: rename de/d15/d53/d81/de3/d74/d92/da0/dca/d34/d4a/f75 to de/d15/d53/d81/de3/d74/d92/da0/dca/d34/d4a/ff0 0 2024-04-23T02:14:03.712 INFO:tasks.workunit.client.1.smithi192.stdout:5/624: chown d2/d6/d2d/f6b 2694 1 2024-04-23T02:14:03.713 INFO:tasks.workunit.client.1.smithi192.stdout:4/733: readlink de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/l3b 0 2024-04-23T02:14:03.714 INFO:tasks.workunit.client.1.smithi192.stdout:0/712: creat d6/d4b/d9d/dc7/fef x:0 0 0 2024-04-23T02:14:03.716 INFO:tasks.workunit.client.1.smithi192.stdout:4/734: chown de/d15/d53/d81/de3/d74/d92/da0/dca/d34 120 1 2024-04-23T02:14:03.717 INFO:tasks.workunit.client.1.smithi192.stdout:4/735: chown de/d15/d19/la4 1 1 2024-04-23T02:14:03.717 INFO:tasks.workunit.client.1.smithi192.stdout:0/713: creat d6/df/d13/d25/d3a/d93/ff0 x:0 0 0 2024-04-23T02:14:03.718 INFO:tasks.workunit.client.0.smithi071.stdout:7/540: dwrite d3/d51/d80/f8a [0,4194304] 0 2024-04-23T02:14:03.718 INFO:tasks.workunit.client.0.smithi071.stdout:7/541: chown d3/d17/l8d 26 1 2024-04-23T02:14:03.718 INFO:tasks.workunit.client.0.smithi071.stdout:7/542: fdatasync d3/d17/d41/d34/f35 0 2024-04-23T02:14:03.721 INFO:tasks.workunit.client.1.smithi192.stdout:0/714: mknod d6/df/d56/db1/cf1 0 2024-04-23T02:14:03.725 INFO:tasks.workunit.client.1.smithi192.stdout:0/715: write d6/df/d13/d6b/d73/f7e [5496,100493] 0 2024-04-23T02:14:03.725 INFO:tasks.workunit.client.0.smithi071.stdout:7/543: dwrite d3/ff [0,4194304] 0 2024-04-23T02:14:03.727 INFO:tasks.workunit.client.1.smithi192.stdout:0/716: chown d6/df/d56/c99 1415220238 1 2024-04-23T02:14:03.729 INFO:tasks.workunit.client.1.smithi192.stdout:7/652: dwrite d5/d17/d1c/d59/d5a/d41/d50/f35 [4194304,4194304] 0 2024-04-23T02:14:03.730 INFO:tasks.workunit.client.0.smithi071.stdout:7/544: rename d3/d17/d41/d37/d4f to d3/d17/d1c/d2a/d73/dba 0 2024-04-23T02:14:03.731 INFO:tasks.workunit.client.1.smithi192.stdout:7/653: chown d5/d17/d1c/d59/d5a/l5e 215 1 2024-04-23T02:14:03.734 INFO:tasks.workunit.client.1.smithi192.stdout:0/717: dread d6/df/d56/d85/f2e [0,4194304] 0 2024-04-23T02:14:03.734 INFO:tasks.workunit.client.0.smithi071.stdout:7/545: link d3/d17/d41/l7a d3/lbb 0 2024-04-23T02:14:03.736 INFO:tasks.workunit.client.0.smithi071.stdout:7/546: write d3/d17/f1f [1415269,52790] 0 2024-04-23T02:14:03.736 INFO:tasks.workunit.client.0.smithi071.stdout:7/547: stat d3/d17/d41/d3a/l4a 0 2024-04-23T02:14:03.741 INFO:tasks.workunit.client.1.smithi192.stdout:0/718: dwrite d6/f10 [0,4194304] 0 2024-04-23T02:14:03.743 INFO:tasks.workunit.client.1.smithi192.stdout:3/570: sync 2024-04-23T02:14:03.743 INFO:tasks.workunit.client.1.smithi192.stdout:9/538: sync 2024-04-23T02:14:03.743 INFO:tasks.workunit.client.1.smithi192.stdout:6/684: sync 2024-04-23T02:14:03.744 INFO:tasks.workunit.client.0.smithi071.stdout:7/548: dwrite d3/d17/d41/f66 [0,4194304] 0 2024-04-23T02:14:03.746 INFO:tasks.workunit.client.0.smithi071.stdout:7/549: creat d3/d51/d87/db3/fbc x:0 0 0 2024-04-23T02:14:03.749 INFO:tasks.workunit.client.1.smithi192.stdout:5/625: dread d2/d6/d2d/d41/f97 [0,4194304] 0 2024-04-23T02:14:03.749 INFO:tasks.workunit.client.1.smithi192.stdout:9/539: stat d0/d9/d44/d51/f66 0 2024-04-23T02:14:03.749 INFO:tasks.workunit.client.1.smithi192.stdout:5/626: write d2/db/d2b/d52/dc2/d49/fb1 [497209,16622] 0 2024-04-23T02:14:03.749 INFO:tasks.workunit.client.1.smithi192.stdout:7/654: truncate d5/d17/d1c/d59/f93 42297 0 2024-04-23T02:14:03.763 INFO:tasks.workunit.client.1.smithi192.stdout:4/736: dwrite de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/f8e [0,4194304] 0 2024-04-23T02:14:03.765 INFO:tasks.workunit.client.1.smithi192.stdout:0/719: getdents d6/df/d56/d85 0 2024-04-23T02:14:03.766 INFO:tasks.workunit.client.1.smithi192.stdout:0/720: dread - d6/fda zero size 2024-04-23T02:14:03.767 INFO:tasks.workunit.client.1.smithi192.stdout:5/627: creat d2/d6/d2d/d41/fd2 x:0 0 0 2024-04-23T02:14:03.769 INFO:tasks.workunit.client.1.smithi192.stdout:4/737: write de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/f8e [2093379,29760] 0 2024-04-23T02:14:03.771 INFO:tasks.workunit.client.1.smithi192.stdout:5/628: chown d2/d6/cbe 7497361 1 2024-04-23T02:14:03.772 INFO:tasks.workunit.client.1.smithi192.stdout:9/540: link d0/d9/d44/d51/d5b/d61/db1/f93 d0/d9/db3/fba 0 2024-04-23T02:14:03.773 INFO:tasks.workunit.client.1.smithi192.stdout:6/685: truncate d3/d18/f2d 1386124 0 2024-04-23T02:14:03.773 INFO:tasks.workunit.client.1.smithi192.stdout:5/629: chown d2/db/d2b/d52/f8f 31 1 2024-04-23T02:14:03.774 INFO:tasks.workunit.client.0.smithi071.stdout:7/550: truncate d3/d17/d41/f33 1073434 0 2024-04-23T02:14:03.775 INFO:tasks.workunit.client.0.smithi071.stdout:7/551: chown d3/d17/d41/d3a/da5/ca9 117 1 2024-04-23T02:14:03.775 INFO:tasks.workunit.client.1.smithi192.stdout:0/721: creat d6/df/d56/d85/d45/ff2 x:0 0 0 2024-04-23T02:14:03.778 INFO:tasks.workunit.client.0.smithi071.stdout:7/552: truncate d3/d17/d1c/f3f 2587287 0 2024-04-23T02:14:03.780 INFO:tasks.workunit.client.1.smithi192.stdout:5/630: creat d2/db/d2b/d52/dc2/fd3 x:0 0 0 2024-04-23T02:14:03.780 INFO:tasks.workunit.client.0.smithi071.stdout:7/553: symlink d3/lbd 0 2024-04-23T02:14:03.781 INFO:tasks.workunit.client.1.smithi192.stdout:4/738: truncate de/d15/d53/d81/de3/d74/d92/faa 424705 0 2024-04-23T02:14:03.784 INFO:tasks.workunit.client.1.smithi192.stdout:6/686: dwrite d3/d28/d79/dbe/dcb/fd5 [0,4194304] 0 2024-04-23T02:14:03.785 INFO:tasks.workunit.client.1.smithi192.stdout:9/541: getdents d0/d9/d44/d58 0 2024-04-23T02:14:03.786 INFO:tasks.workunit.client.1.smithi192.stdout:7/655: dwrite d5/d17/d1c/d26/f28 [0,4194304] 0 2024-04-23T02:14:03.787 INFO:tasks.workunit.client.0.smithi071.stdout:7/554: dwrite d3/d17/d41/d34/d39/d3b/da7/faf [0,4194304] 0 2024-04-23T02:14:03.788 INFO:tasks.workunit.client.1.smithi192.stdout:0/722: truncate d6/f18 1905254 0 2024-04-23T02:14:03.793 INFO:tasks.workunit.client.0.smithi071.stdout:7/555: rename d3/d17/d41/d34/f9b to d3/d17/d41/d3a/da5/fbe 0 2024-04-23T02:14:03.797 INFO:tasks.workunit.client.1.smithi192.stdout:5/631: symlink d2/db/d16/d48/d4f/dd1/ld4 0 2024-04-23T02:14:03.797 INFO:tasks.workunit.client.1.smithi192.stdout:5/632: write d2/d6/d2d/d41/fa1 [753653,46855] 0 2024-04-23T02:14:03.797 INFO:tasks.workunit.client.0.smithi071.stdout:7/556: creat d3/d17/d41/d3a/da5/fbf x:0 0 0 2024-04-23T02:14:03.798 INFO:tasks.workunit.client.1.smithi192.stdout:6/687: dread d3/d15/d1b/d66/d80/d8e/faf [0,4194304] 0 2024-04-23T02:14:03.800 INFO:tasks.workunit.client.0.smithi071.stdout:7/557: write d3/d17/d41/d34/d39/d3b/f49 [1222643,60229] 0 2024-04-23T02:14:03.803 INFO:tasks.workunit.client.0.smithi071.stdout:7/558: creat d3/d17/d1c/d2a/d73/fc0 x:0 0 0 2024-04-23T02:14:03.804 INFO:tasks.workunit.client.1.smithi192.stdout:4/739: getdents de/d15/d53/d81/de3/d74/d92/da0/dca/d34/d7c/da1 0 2024-04-23T02:14:03.805 INFO:tasks.workunit.client.1.smithi192.stdout:0/723: creat d6/df/d56/ff3 x:0 0 0 2024-04-23T02:14:03.806 INFO:tasks.workunit.client.1.smithi192.stdout:6/688: fsync d3/d15/d1b/d21/fa1 0 2024-04-23T02:14:03.807 INFO:tasks.workunit.client.0.smithi071.stdout:7/559: rename d3/d17/d41/d34/d39/d3b/l70 to d3/d17/d1c/d2a/d43/d86/lc1 0 2024-04-23T02:14:03.809 INFO:tasks.workunit.client.1.smithi192.stdout:5/633: rename d2/f2c to d2/d6/fd5 0 2024-04-23T02:14:03.810 INFO:tasks.workunit.client.0.smithi071.stdout:7/560: rename d3/ff to d3/d17/d1c/d31/fc2 0 2024-04-23T02:14:03.812 INFO:tasks.workunit.client.1.smithi192.stdout:7/656: dread d5/d17/d1c/d59/d5a/d41/d50/d3d/f83 [0,4194304] 0 2024-04-23T02:14:03.815 INFO:tasks.workunit.client.1.smithi192.stdout:4/740: creat de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/ff1 x:0 0 0 2024-04-23T02:14:03.817 INFO:tasks.workunit.client.1.smithi192.stdout:6/689: rename d3/d28/d79/da0/dc2/lc9 to d3/d28/d79/dbe/dcb/le4 0 2024-04-23T02:14:03.817 INFO:tasks.workunit.client.1.smithi192.stdout:0/724: dwrite d6/df/d47/f89 [0,4194304] 0 2024-04-23T02:14:03.824 INFO:tasks.workunit.client.0.smithi071.stdout:7/561: write d3/fad [330851,130140] 0 2024-04-23T02:14:03.825 INFO:tasks.workunit.client.0.smithi071.stdout:7/562: write d3/d17/d41/f2f [883826,55655] 0 2024-04-23T02:14:03.826 INFO:tasks.workunit.client.1.smithi192.stdout:5/634: dwrite d2/db/d16/d1b/f79 [0,4194304] 0 2024-04-23T02:14:03.826 INFO:tasks.workunit.client.1.smithi192.stdout:0/725: creat d6/df/d56/d85/ff4 x:0 0 0 2024-04-23T02:14:03.830 INFO:tasks.workunit.client.0.smithi071.stdout:7/563: rmdir d3/d17/d1c/d2a/d73/d83/d92 0 2024-04-23T02:14:03.831 INFO:tasks.workunit.client.1.smithi192.stdout:9/542: dwrite d0/d9/f50 [4194304,4194304] 0 2024-04-23T02:14:03.832 INFO:tasks.workunit.client.0.smithi071.stdout:7/564: creat d3/d17/d1c/d2a/d43/d86/fc3 x:0 0 0 2024-04-23T02:14:03.834 INFO:tasks.workunit.client.0.smithi071.stdout:7/565: mknod d3/d17/d1c/d2a/d73/dba/cc4 0 2024-04-23T02:14:03.838 INFO:tasks.workunit.client.0.smithi071.stdout:7/566: mkdir d3/d17/d41/d34/d59/dc5 0 2024-04-23T02:14:03.838 INFO:tasks.workunit.client.1.smithi192.stdout:5/635: creat d2/db/d2b/d52/dc2/d49/fd6 x:0 0 0 2024-04-23T02:14:03.840 INFO:tasks.workunit.client.1.smithi192.stdout:6/690: dwrite d3/d15/d1b/d66/d80/d8e/faf [0,4194304] 0 2024-04-23T02:14:03.841 INFO:tasks.workunit.client.0.smithi071.stdout:7/567: write d3/d17/d1c/f29 [2582560,60617] 0 2024-04-23T02:14:03.841 INFO:tasks.workunit.client.1.smithi192.stdout:5/636: dread - d2/db/d2b/d52/dc2/d49/f91 zero size 2024-04-23T02:14:03.847 INFO:tasks.workunit.client.1.smithi192.stdout:7/657: write d5/d43/d4c/d5c/d97/faf [134890,51787] 0 2024-04-23T02:14:03.847 INFO:tasks.workunit.client.1.smithi192.stdout:0/726: getdents d6/df/d35/d8b/d8d/dea 0 2024-04-23T02:14:03.848 INFO:tasks.workunit.client.1.smithi192.stdout:5/637: fdatasync d2/db/d2b/d52/f8f 0 2024-04-23T02:14:03.848 INFO:tasks.workunit.client.1.smithi192.stdout:6/691: mknod d3/d18/dca/ce5 0 2024-04-23T02:14:03.852 INFO:tasks.workunit.client.1.smithi192.stdout:0/727: creat d6/df/d35/d8b/d8d/ff5 x:0 0 0 2024-04-23T02:14:03.853 INFO:tasks.workunit.client.1.smithi192.stdout:5/638: creat d2/db/d16/d1b/fd7 x:0 0 0 2024-04-23T02:14:03.853 INFO:tasks.workunit.client.1.smithi192.stdout:1/422: sync 2024-04-23T02:14:03.853 INFO:tasks.workunit.client.1.smithi192.stdout:8/797: sync 2024-04-23T02:14:03.855 INFO:tasks.workunit.client.1.smithi192.stdout:6/692: creat d3/d18/dd4/d8d/fe6 x:0 0 0 2024-04-23T02:14:03.856 INFO:tasks.workunit.client.1.smithi192.stdout:4/741: dwrite de/d15/d53/d81/de3/d80/fc2 [0,4194304] 0 2024-04-23T02:14:03.856 INFO:tasks.workunit.client.1.smithi192.stdout:0/728: write d6/df/f50 [2321217,99483] 0 2024-04-23T02:14:03.857 INFO:tasks.workunit.client.1.smithi192.stdout:7/658: link d5/c13 d5/d43/d4c/d5c/d97/cbc 0 2024-04-23T02:14:03.858 INFO:tasks.workunit.client.1.smithi192.stdout:8/798: chown d1/d12/c16 14712 1 2024-04-23T02:14:03.863 INFO:tasks.workunit.client.1.smithi192.stdout:6/693: rename d3/d6c/c81 to d3/d11/db0/ce7 0 2024-04-23T02:14:03.864 INFO:tasks.workunit.client.1.smithi192.stdout:7/659: symlink d5/d43/d4c/d5c/d97/lbd 0 2024-04-23T02:14:03.864 INFO:tasks.workunit.client.0.smithi071.stdout:9/518: sync 2024-04-23T02:14:03.865 INFO:tasks.workunit.client.1.smithi192.stdout:5/639: creat d2/d2e/fd8 x:0 0 0 2024-04-23T02:14:03.870 INFO:tasks.workunit.client.1.smithi192.stdout:7/660: mknod d5/d17/d1c/d57/cbe 0 2024-04-23T02:14:03.870 INFO:tasks.workunit.client.1.smithi192.stdout:5/640: rmdir d2/d6/d2d 39 2024-04-23T02:14:03.870 INFO:tasks.workunit.client.1.smithi192.stdout:8/799: dwrite d1/d9/d8f/fc0 [4194304,4194304] 0 2024-04-23T02:14:03.872 INFO:tasks.workunit.client.0.smithi071.stdout:9/519: getdents d1 0 2024-04-23T02:14:03.874 INFO:tasks.workunit.client.1.smithi192.stdout:7/661: write d5/d17/d1c/d59/d5a/d41/d50/f69 [2674601,127427] 0 2024-04-23T02:14:03.875 INFO:tasks.workunit.client.1.smithi192.stdout:5/641: mkdir d2/d6/d2d/dd9 0 2024-04-23T02:14:03.875 INFO:tasks.workunit.client.0.smithi071.stdout:9/520: creat d1/d4e/d66/d8b/fb1 x:0 0 0 2024-04-23T02:14:03.876 INFO:tasks.workunit.client.1.smithi192.stdout:5/642: readlink d2/db/d2b/d52/dc2/d49/l5d 0 2024-04-23T02:14:03.876 INFO:tasks.workunit.client.1.smithi192.stdout:8/800: link d1/d12/d23/d2c/c3b d1/d12/d23/d4b/c118 0 2024-04-23T02:14:03.879 INFO:tasks.workunit.client.0.smithi071.stdout:9/521: write d1/d4e/d66/d8b/f95 [520422,97566] 0 2024-04-23T02:14:03.883 INFO:tasks.workunit.client.0.smithi071.stdout:9/522: rename d1/d32/d3a/d5f/c61 to d1/d32/d3a/d5f/cb2 0 2024-04-23T02:14:03.883 INFO:tasks.workunit.client.0.smithi071.stdout:9/523: chown d1/d1a/d5a/f5e 661 1 2024-04-23T02:14:03.884 INFO:tasks.workunit.client.0.smithi071.stdout:9/524: read - d1/f86 zero size 2024-04-23T02:14:03.884 INFO:tasks.workunit.client.1.smithi192.stdout:8/801: link d1/d12/d23/d4b/d73/d7e/d88/da9/db1/c116 d1/d12/d7d/df7/c119 0 2024-04-23T02:14:03.885 INFO:tasks.workunit.client.0.smithi071.stdout:9/525: stat d1/d1a/f70 0 2024-04-23T02:14:03.886 INFO:tasks.workunit.client.1.smithi192.stdout:8/802: rmdir d1/d12/d23/d4b/d73/d45 39 2024-04-23T02:14:03.886 INFO:tasks.workunit.client.0.smithi071.stdout:9/526: truncate d1/d32/d3a/d77/f89 782096 0 2024-04-23T02:14:03.886 INFO:tasks.workunit.client.1.smithi192.stdout:5/643: read d2/d6/d2d/faa [3795697,129124] 0 2024-04-23T02:14:03.888 INFO:tasks.workunit.client.1.smithi192.stdout:5/644: write d2/d6/f3b [664215,726] 0 2024-04-23T02:14:03.888 INFO:tasks.workunit.client.0.smithi071.stdout:9/527: rmdir d1/d4e/d54/d75/da6/da9 0 2024-04-23T02:14:03.889 INFO:tasks.workunit.client.1.smithi192.stdout:5/645: read - d2/db/d2b/d52/dc2/d49/fd6 zero size 2024-04-23T02:14:03.890 INFO:tasks.workunit.client.0.smithi071.stdout:9/528: dread - d1/d1a/d71/f9e zero size 2024-04-23T02:14:03.892 INFO:tasks.workunit.client.1.smithi192.stdout:5/646: dread d2/db/fad [0,4194304] 0 2024-04-23T02:14:03.895 INFO:tasks.workunit.client.1.smithi192.stdout:5/647: symlink d2/d6/d2d/d41/d43/d98/lda 0 2024-04-23T02:14:03.896 INFO:tasks.workunit.client.1.smithi192.stdout:5/648: rmdir d2/db/d2b 39 2024-04-23T02:14:03.897 INFO:tasks.workunit.client.1.smithi192.stdout:5/649: symlink d2/db/ldb 0 2024-04-23T02:14:03.898 INFO:tasks.workunit.client.1.smithi192.stdout:9/543: write d0/d9/d44/f49 [550625,109871] 0 2024-04-23T02:14:03.902 INFO:tasks.workunit.client.1.smithi192.stdout:9/544: rename d0/d9/d44/d51/l68 to d0/d9/d44/d51/d5b/d96/lbb 0 2024-04-23T02:14:03.905 INFO:tasks.workunit.client.1.smithi192.stdout:9/545: symlink d0/d9/d44/d51/d5b/d96/da0/db4/lbc 0 2024-04-23T02:14:03.906 INFO:tasks.workunit.client.1.smithi192.stdout:5/650: getdents d2/d6/d2d/d41/d43/dbc 0 2024-04-23T02:14:03.908 INFO:tasks.workunit.client.1.smithi192.stdout:0/729: write d6/df/d13/d25/d26/f90 [575527,37613] 0 2024-04-23T02:14:03.911 INFO:tasks.workunit.client.0.smithi071.stdout:7/568: dread d3/d17/d1c/d31/f6a [0,4194304] 0 2024-04-23T02:14:03.912 INFO:tasks.workunit.client.1.smithi192.stdout:9/546: truncate d0/d9/d44/d4f/f8 1491590 0 2024-04-23T02:14:03.912 INFO:tasks.workunit.client.0.smithi071.stdout:9/529: dread d1/fb [0,4194304] 0 2024-04-23T02:14:03.913 INFO:tasks.workunit.client.1.smithi192.stdout:1/423: dwrite dc/d16/d19/f3a [0,4194304] 0 2024-04-23T02:14:03.913 INFO:tasks.workunit.client.0.smithi071.stdout:7/569: write d3/d17/d1c/d31/f56 [1117438,54696] 0 2024-04-23T02:14:03.914 INFO:tasks.workunit.client.1.smithi192.stdout:5/651: creat d2/d6/d2d/d41/d43/fdc x:0 0 0 2024-04-23T02:14:03.914 INFO:tasks.workunit.client.1.smithi192.stdout:7/662: truncate d5/f15 3850340 0 2024-04-23T02:14:03.914 INFO:tasks.workunit.client.0.smithi071.stdout:9/530: fsync d1/d1a/d39/f8f 0 2024-04-23T02:14:03.914 INFO:tasks.workunit.client.0.smithi071.stdout:9/531: stat d1/f2b 0 2024-04-23T02:14:03.915 INFO:tasks.workunit.client.0.smithi071.stdout:9/532: truncate d1/d1a/d39/fa5 118965 0 2024-04-23T02:14:03.916 INFO:tasks.workunit.client.1.smithi192.stdout:6/694: dread d3/d18/f6f [0,4194304] 0 2024-04-23T02:14:03.917 INFO:tasks.workunit.client.1.smithi192.stdout:2/622: sync 2024-04-23T02:14:03.917 INFO:tasks.workunit.client.1.smithi192.stdout:3/571: sync 2024-04-23T02:14:03.920 INFO:tasks.workunit.client.0.smithi071.stdout:9/533: rename d1/d4e/d66/f73 to d1/d32/d3a/d94/dab/fb3 0 2024-04-23T02:14:03.922 INFO:tasks.workunit.client.1.smithi192.stdout:8/803: truncate d1/df/f50 3712508 0 2024-04-23T02:14:03.922 INFO:tasks.workunit.client.1.smithi192.stdout:1/424: fdatasync f1 0 2024-04-23T02:14:03.922 INFO:tasks.workunit.client.1.smithi192.stdout:0/730: mknod d6/df/d13/d25/d94/dd0/cf6 0 2024-04-23T02:14:03.923 INFO:tasks.workunit.client.0.smithi071.stdout:9/534: symlink d1/d8d/lb4 0 2024-04-23T02:14:03.923 INFO:tasks.workunit.client.1.smithi192.stdout:4/742: dwrite de/d15/d53/d81/de3/fcb [0,4194304] 0 2024-04-23T02:14:03.925 INFO:tasks.workunit.client.0.smithi071.stdout:9/535: creat d1/d1a/d68/fb5 x:0 0 0 2024-04-23T02:14:03.926 INFO:tasks.workunit.client.1.smithi192.stdout:3/572: dread d1/f56 [0,4194304] 0 2024-04-23T02:14:03.926 INFO:tasks.workunit.client.1.smithi192.stdout:7/663: creat d5/d17/d1c/d59/d5a/d41/d50/d3d/fbf x:0 0 0 2024-04-23T02:14:03.927 INFO:tasks.workunit.client.1.smithi192.stdout:6/695: symlink d3/d15/d1b/d66/d80/d8e/le8 0 2024-04-23T02:14:03.928 INFO:tasks.workunit.client.1.smithi192.stdout:2/623: creat d8/d26/d29/da4/da8/fc9 x:0 0 0 2024-04-23T02:14:03.932 INFO:tasks.workunit.client.1.smithi192.stdout:1/425: mkdir dc/d80 0 2024-04-23T02:14:03.933 INFO:tasks.workunit.client.0.smithi071.stdout:7/570: truncate d3/d17/d1c/f29 2822341 0 2024-04-23T02:14:03.933 INFO:tasks.workunit.client.0.smithi071.stdout:9/536: mkdir d1/d32/d3a/d78/d81/db6 0 2024-04-23T02:14:03.934 INFO:tasks.workunit.client.1.smithi192.stdout:3/573: dread d1/d3/d17/d2b/d44/f93 [0,4194304] 0 2024-04-23T02:14:03.936 INFO:tasks.workunit.client.1.smithi192.stdout:0/731: rename d6/df/d13/d25/d86/da5/lb0 to d6/df/d13/d6b/d73/lf7 0 2024-04-23T02:14:03.937 INFO:tasks.workunit.client.0.smithi071.stdout:7/571: rmdir d3/d17/d1c/d2a/d43 39 2024-04-23T02:14:03.937 INFO:tasks.workunit.client.0.smithi071.stdout:7/572: stat d3/d17/d1c/d2a/d73/d83 0 2024-04-23T02:14:03.942 INFO:tasks.workunit.client.0.smithi071.stdout:7/573: dwrite d3/d17/d41/d3a/da5/fbe [0,4194304] 0 2024-04-23T02:14:03.943 INFO:tasks.workunit.client.1.smithi192.stdout:2/624: mkdir d8/d26/d29/da4/dca 0 2024-04-23T02:14:03.945 INFO:tasks.workunit.client.1.smithi192.stdout:3/574: mknod d1/cc5 0 2024-04-23T02:14:03.945 INFO:tasks.workunit.client.1.smithi192.stdout:5/652: dwrite d2/fe [0,4194304] 0 2024-04-23T02:14:03.946 INFO:tasks.workunit.client.0.smithi071.stdout:7/574: creat d3/d17/d1c/d2a/d73/fc6 x:0 0 0 2024-04-23T02:14:03.947 INFO:tasks.workunit.client.0.smithi071.stdout:7/575: fsync d3/d17/d41/d34/d39/d3b/f84 0 2024-04-23T02:14:03.950 INFO:tasks.workunit.client.0.smithi071.stdout:9/537: rename d1/d1a to d1/d32/d41/db7 0 2024-04-23T02:14:03.951 INFO:tasks.workunit.client.0.smithi071.stdout:7/576: creat d3/d17/fc7 x:0 0 0 2024-04-23T02:14:03.955 INFO:tasks.workunit.client.1.smithi192.stdout:0/732: dread d6/df/d47/f4c [0,4194304] 0 2024-04-23T02:14:03.956 INFO:tasks.workunit.client.1.smithi192.stdout:8/804: rename d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/lbd to d1/d9/d3e/d59/d8a/daa/l11a 0 2024-04-23T02:14:03.956 INFO:tasks.workunit.client.1.smithi192.stdout:4/743: mknod de/d15/d53/d81/de3/d74/d92/da0/dca/d49/de6/cf2 0 2024-04-23T02:14:03.956 INFO:tasks.workunit.client.1.smithi192.stdout:7/664: mkdir d5/d43/d4c/d5c/d67/da6/da9/db9/dc0 0 2024-04-23T02:14:03.957 INFO:tasks.workunit.client.1.smithi192.stdout:9/547: truncate d0/d9/d44/d4f/f41 1537100 0 2024-04-23T02:14:03.958 INFO:tasks.workunit.client.1.smithi192.stdout:0/733: chown d6/df/d13/d25/d26/c64 1983681314 1 2024-04-23T02:14:03.959 INFO:tasks.workunit.client.1.smithi192.stdout:8/805: fsync d1/d12/d23/d4b/f51 0 2024-04-23T02:14:03.960 INFO:tasks.workunit.client.1.smithi192.stdout:4/744: stat l7 0 2024-04-23T02:14:03.962 INFO:tasks.workunit.client.1.smithi192.stdout:7/665: chown d5/c13 19 1 2024-04-23T02:14:03.962 INFO:tasks.workunit.client.1.smithi192.stdout:8/806: chown d1/d9/d3e/c4f 4084 1 2024-04-23T02:14:03.963 INFO:tasks.workunit.client.1.smithi192.stdout:2/625: creat d8/d26/d29/d43/fcb x:0 0 0 2024-04-23T02:14:03.963 INFO:tasks.workunit.client.1.smithi192.stdout:0/734: write d6/df/d13/d6b/d73/da9/fe5 [661120,49722] 0 2024-04-23T02:14:03.966 INFO:tasks.workunit.client.1.smithi192.stdout:5/653: creat d2/db/d2b/dc8/fdd x:0 0 0 2024-04-23T02:14:03.967 INFO:tasks.workunit.client.1.smithi192.stdout:4/745: write de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/fae [75808,101480] 0 2024-04-23T02:14:03.969 INFO:tasks.workunit.client.1.smithi192.stdout:2/626: mknod d8/ccc 0 2024-04-23T02:14:03.969 INFO:tasks.workunit.client.1.smithi192.stdout:0/735: fsync d6/df/d13/d6b/d73/f7e 0 2024-04-23T02:14:03.970 INFO:tasks.workunit.client.0.smithi071.stdout:9/538: mkdir d1/db8 0 2024-04-23T02:14:03.970 INFO:tasks.workunit.client.0.smithi071.stdout:7/577: truncate d3/d51/d80/f8a 1998403 0 2024-04-23T02:14:03.970 INFO:tasks.workunit.client.1.smithi192.stdout:8/807: fdatasync d1/df/f9b 0 2024-04-23T02:14:03.973 INFO:tasks.workunit.client.0.smithi071.stdout:7/578: creat d3/d17/d41/d34/d39/fc8 x:0 0 0 2024-04-23T02:14:03.974 INFO:tasks.workunit.client.1.smithi192.stdout:7/666: rename d5/d43/d4c/d5c/d67/l6b to d5/d17/d1c/d57/d8b/lc1 0 2024-04-23T02:14:03.976 INFO:tasks.workunit.client.1.smithi192.stdout:9/548: getdents d0/d9/d44/d51/d5b 0 2024-04-23T02:14:03.979 INFO:tasks.workunit.client.1.smithi192.stdout:6/696: dread d3/d15/d1b/da2/faa [0,4194304] 0 2024-04-23T02:14:03.979 INFO:tasks.workunit.client.0.smithi071.stdout:7/579: dwrite d3/d51/d87/fab [0,4194304] 0 2024-04-23T02:14:03.982 INFO:tasks.workunit.client.1.smithi192.stdout:4/746: fdatasync de/d15/d19/d1a/f72 0 2024-04-23T02:14:03.982 INFO:tasks.workunit.client.1.smithi192.stdout:1/426: write dc/d16/f1f [3632470,15020] 0 2024-04-23T02:14:03.985 INFO:tasks.workunit.client.1.smithi192.stdout:7/667: creat d5/d17/d1c/d4a/fc2 x:0 0 0 2024-04-23T02:14:03.986 INFO:tasks.workunit.client.0.smithi071.stdout:9/539: dread d1/fc [0,4194304] 0 2024-04-23T02:14:03.987 INFO:tasks.workunit.client.1.smithi192.stdout:9/549: rmdir d0/d9/d44/d51/d5b/d61/db1 39 2024-04-23T02:14:03.987 INFO:tasks.workunit.client.1.smithi192.stdout:1/427: dread - dc/d16/d19/d30/d32/f47 zero size 2024-04-23T02:14:03.988 INFO:tasks.workunit.client.0.smithi071.stdout:7/580: link d3/d17/d41/d34/d59/d5f/f6c d3/d51/fc9 0 2024-04-23T02:14:03.989 INFO:tasks.workunit.client.1.smithi192.stdout:6/697: unlink d3/d15/d1b/d21/d3a/l5d 0 2024-04-23T02:14:03.990 INFO:tasks.workunit.client.0.smithi071.stdout:9/540: write d1/f2b [354947,73368] 0 2024-04-23T02:14:03.991 INFO:tasks.workunit.client.1.smithi192.stdout:2/627: dwrite d8/d25/f71 [0,4194304] 0 2024-04-23T02:14:03.991 INFO:tasks.workunit.client.0.smithi071.stdout:7/581: mkdir d3/d17/dca 0 2024-04-23T02:14:03.991 INFO:tasks.workunit.client.1.smithi192.stdout:0/736: symlink d6/df/d13/d25/d3a/lf8 0 2024-04-23T02:14:03.992 INFO:tasks.workunit.client.0.smithi071.stdout:7/582: symlink d3/d17/d1c/d31/lcb 0 2024-04-23T02:14:03.999 INFO:tasks.workunit.client.1.smithi192.stdout:8/808: dread d1/d12/d23/fd3 [0,4194304] 0 2024-04-23T02:14:04.000 INFO:tasks.workunit.client.1.smithi192.stdout:8/809: chown d1/df/fa8 2358 1 2024-04-23T02:14:04.000 INFO:tasks.workunit.client.1.smithi192.stdout:7/668: stat d5/d17/d1c/d59/d5a/d70/f7e 0 2024-04-23T02:14:04.001 INFO:tasks.workunit.client.1.smithi192.stdout:8/810: readlink d1/d9/d3e/l6f 0 2024-04-23T02:14:04.003 INFO:tasks.workunit.client.1.smithi192.stdout:0/737: rmdir d6/df/d13/d6b/dc1 39 2024-04-23T02:14:04.004 INFO:tasks.workunit.client.1.smithi192.stdout:2/628: unlink d8/d26/f78 0 2024-04-23T02:14:04.009 INFO:tasks.workunit.client.1.smithi192.stdout:5/654: getdents d2/d32 0 2024-04-23T02:14:04.010 INFO:tasks.workunit.client.1.smithi192.stdout:3/575: mknod d1/d3/d17/d2b/d44/d57/cc6 0 2024-04-23T02:14:04.012 INFO:tasks.workunit.client.1.smithi192.stdout:1/428: mkdir dc/d80/d81 0 2024-04-23T02:14:04.015 INFO:tasks.workunit.client.1.smithi192.stdout:1/429: fsync dc/d16/d19/f3a 0 2024-04-23T02:14:04.015 INFO:tasks.workunit.client.1.smithi192.stdout:8/811: stat d1/d12/d68/fad 0 2024-04-23T02:14:04.017 INFO:tasks.workunit.client.1.smithi192.stdout:0/738: unlink d6/d4b/d9d/lbc 0 2024-04-23T02:14:04.018 INFO:tasks.workunit.client.0.smithi071.stdout:1/515: sync 2024-04-23T02:14:04.018 INFO:tasks.workunit.client.0.smithi071.stdout:4/638: sync 2024-04-23T02:14:04.020 INFO:tasks.workunit.client.0.smithi071.stdout:7/583: rename d3/d17/d41/d37/f38 to d3/dae/fcc 0 2024-04-23T02:14:04.020 INFO:tasks.workunit.client.0.smithi071.stdout:9/541: getdents d1/d32/d41/db7 0 2024-04-23T02:14:04.021 INFO:tasks.workunit.client.1.smithi192.stdout:5/655: chown d2/db/d16/f96 14 1 2024-04-23T02:14:04.021 INFO:tasks.workunit.client.1.smithi192.stdout:1/430: creat dc/d16/d19/d30/f82 x:0 0 0 2024-04-23T02:14:04.022 INFO:tasks.workunit.client.1.smithi192.stdout:0/739: chown d6/df/d56/l9a 762 1 2024-04-23T02:14:04.022 INFO:tasks.workunit.client.1.smithi192.stdout:2/629: mkdir d8/d26/d29/dcd 0 2024-04-23T02:14:04.023 INFO:tasks.workunit.client.0.smithi071.stdout:1/516: chown db/d11/d7f/l91 132 1 2024-04-23T02:14:04.024 INFO:tasks.workunit.client.0.smithi071.stdout:1/517: dread - db/df/d13/d20/d75/f9c zero size 2024-04-23T02:14:04.024 INFO:tasks.workunit.client.0.smithi071.stdout:9/542: unlink d1/d32/d3a/d5f/ca0 0 2024-04-23T02:14:04.025 INFO:tasks.workunit.client.1.smithi192.stdout:3/576: symlink d1/d3/d17/d1f/d33/lc7 0 2024-04-23T02:14:04.025 INFO:tasks.workunit.client.1.smithi192.stdout:4/747: dwrite de/d15/d53/d81/de3/f5c [0,4194304] 0 2024-04-23T02:14:04.026 INFO:tasks.workunit.client.0.smithi071.stdout:1/518: write db/df/d13/d20/f53 [374415,24406] 0 2024-04-23T02:14:04.027 INFO:tasks.workunit.client.1.smithi192.stdout:1/431: creat dc/d14/d23/f83 x:0 0 0 2024-04-23T02:14:04.028 INFO:tasks.workunit.client.1.smithi192.stdout:2/630: read - d8/d26/d29/da4/f7e zero size 2024-04-23T02:14:04.028 INFO:tasks.workunit.client.1.smithi192.stdout:8/812: creat d1/d12/d23/d4b/d73/d45/d46/f11b x:0 0 0 2024-04-23T02:14:04.030 INFO:tasks.workunit.client.1.smithi192.stdout:5/656: creat d2/d6/d2d/fde x:0 0 0 2024-04-23T02:14:04.032 INFO:tasks.workunit.client.1.smithi192.stdout:0/740: creat d6/df/d56/ff9 x:0 0 0 2024-04-23T02:14:04.034 INFO:tasks.workunit.client.1.smithi192.stdout:1/432: creat dc/d16/d19/d30/f84 x:0 0 0 2024-04-23T02:14:04.034 INFO:tasks.workunit.client.0.smithi071.stdout:4/639: creat d2/d2a/d2d/d39/d42/d49/d91/fcb x:0 0 0 2024-04-23T02:14:04.034 INFO:tasks.workunit.client.0.smithi071.stdout:9/543: mkdir d1/d32/db9 0 2024-04-23T02:14:04.035 INFO:tasks.workunit.client.1.smithi192.stdout:6/698: write d3/d11/f43 [53432,98466] 0 2024-04-23T02:14:04.036 INFO:tasks.workunit.client.1.smithi192.stdout:8/813: rmdir d1/d12/d23/d2c/d8c 39 2024-04-23T02:14:04.037 INFO:tasks.workunit.client.1.smithi192.stdout:5/657: rmdir d2/db 39 2024-04-23T02:14:04.037 INFO:tasks.workunit.client.1.smithi192.stdout:3/577: truncate d1/d3/d17/d1f/d33/fad 259257 0 2024-04-23T02:14:04.039 INFO:tasks.workunit.client.1.smithi192.stdout:7/669: dread d5/d17/d1c/d59/d5a/d41/d50/f7b [0,4194304] 0 2024-04-23T02:14:04.042 INFO:tasks.workunit.client.0.smithi071.stdout:7/584: link d3/d17/d1c/d2a/d43/f4e d3/d17/d1c/d2a/d73/dba/fcd 0 2024-04-23T02:14:04.043 INFO:tasks.workunit.client.0.smithi071.stdout:9/544: mknod d1/d32/cba 0 2024-04-23T02:14:04.043 INFO:tasks.workunit.client.1.smithi192.stdout:6/699: dread - d3/d28/d79/da0/fae zero size 2024-04-23T02:14:04.043 INFO:tasks.workunit.client.1.smithi192.stdout:3/578: readlink d1/d3/d17/d65/l7e 0 2024-04-23T02:14:04.045 INFO:tasks.workunit.client.0.smithi071.stdout:7/585: mknod d3/d17/d1c/d2a/d73/d83/cce 0 2024-04-23T02:14:04.046 INFO:tasks.workunit.client.1.smithi192.stdout:8/814: stat d1/d12/d23/d4b/d73/d4c/l7a 0 2024-04-23T02:14:04.047 INFO:tasks.workunit.client.0.smithi071.stdout:9/545: creat d1/d32/d3a/d94/fbb x:0 0 0 2024-04-23T02:14:04.048 INFO:tasks.workunit.client.0.smithi071.stdout:4/640: getdents d2/d2a/da2 0 2024-04-23T02:14:04.048 INFO:tasks.workunit.client.1.smithi192.stdout:0/741: getdents d6/df/d13/d25/d26 0 2024-04-23T02:14:04.049 INFO:tasks.workunit.client.1.smithi192.stdout:5/658: mkdir d2/d6/d2d/d41/d43/d98/ddf 0 2024-04-23T02:14:04.050 INFO:tasks.workunit.client.0.smithi071.stdout:7/586: chown d3/d17/f36 72731761 1 2024-04-23T02:14:04.050 INFO:tasks.workunit.client.0.smithi071.stdout:9/546: fdatasync d1/d32/d41/db7/d68/f9d 0 2024-04-23T02:14:04.051 INFO:tasks.workunit.client.0.smithi071.stdout:4/641: creat d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/fcc x:0 0 0 2024-04-23T02:14:04.052 INFO:tasks.workunit.client.1.smithi192.stdout:2/631: dwrite d8/d26/d29/d80/fc7 [0,4194304] 0 2024-04-23T02:14:04.054 INFO:tasks.workunit.client.0.smithi071.stdout:9/547: creat d1/d32/d3a/d78/fbc x:0 0 0 2024-04-23T02:14:04.054 INFO:tasks.workunit.client.1.smithi192.stdout:4/748: write de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d93/fc6 [425295,56077] 0 2024-04-23T02:14:04.056 INFO:tasks.workunit.client.1.smithi192.stdout:8/815: chown d1/d12/d23/d4b/d73/d45/d46/fd6 2447 1 2024-04-23T02:14:04.057 INFO:tasks.workunit.client.1.smithi192.stdout:1/433: dwrite dc/f76 [0,4194304] 0 2024-04-23T02:14:04.058 INFO:tasks.workunit.client.1.smithi192.stdout:9/550: dread d0/d9/d44/d51/d5b/d61/db1/f93 [0,4194304] 0 2024-04-23T02:14:04.060 INFO:tasks.workunit.client.0.smithi071.stdout:4/642: fdatasync d2/d2a/d2d/d39/d42/f96 0 2024-04-23T02:14:04.061 INFO:tasks.workunit.client.0.smithi071.stdout:7/587: dread d3/d51/d80/f8a [0,4194304] 0 2024-04-23T02:14:04.062 INFO:tasks.workunit.client.0.smithi071.stdout:7/588: chown d3/d17/c19 504845 1 2024-04-23T02:14:04.063 INFO:tasks.workunit.client.0.smithi071.stdout:9/548: creat d1/db8/fbd x:0 0 0 2024-04-23T02:14:04.067 INFO:tasks.workunit.client.1.smithi192.stdout:9/551: write d0/d9/f32 [1124355,104802] 0 2024-04-23T02:14:04.067 INFO:tasks.workunit.client.0.smithi071.stdout:9/549: read d1/d32/d41/db7/d5a/f6a [3324946,84801] 0 2024-04-23T02:14:04.067 INFO:tasks.workunit.client.0.smithi071.stdout:7/589: creat d3/d17/d1c/d31/fcf x:0 0 0 2024-04-23T02:14:04.068 INFO:tasks.workunit.client.0.smithi071.stdout:7/590: chown d3/f1d 14 1 2024-04-23T02:14:04.068 INFO:tasks.workunit.client.0.smithi071.stdout:9/550: dread - d1/db8/fbd zero size 2024-04-23T02:14:04.069 INFO:tasks.workunit.client.1.smithi192.stdout:2/632: mknod d8/d26/d81/cce 0 2024-04-23T02:14:04.071 INFO:tasks.workunit.client.1.smithi192.stdout:8/816: mkdir d1/d12/d23/d2c/d11c 0 2024-04-23T02:14:04.072 INFO:tasks.workunit.client.1.smithi192.stdout:7/670: dwrite d5/d17/d1c/f74 [8388608,4194304] 0 2024-04-23T02:14:04.072 INFO:tasks.workunit.client.1.smithi192.stdout:0/742: getdents d6/df/d13/d25/d86 0 2024-04-23T02:14:04.074 INFO:tasks.workunit.client.1.smithi192.stdout:3/579: dwrite d1/d3/d6/f10 [0,4194304] 0 2024-04-23T02:14:04.075 INFO:tasks.workunit.client.1.smithi192.stdout:9/552: creat d0/d9/d44/d58/fbd x:0 0 0 2024-04-23T02:14:04.076 INFO:tasks.workunit.client.0.smithi071.stdout:7/591: dwrite d3/d17/d1c/f8e [0,4194304] 0 2024-04-23T02:14:04.076 INFO:tasks.workunit.client.1.smithi192.stdout:8/817: readlink d1/dcb/l107 0 2024-04-23T02:14:04.076 INFO:tasks.workunit.client.0.smithi071.stdout:9/551: getdents d1/d32/d7e 0 2024-04-23T02:14:04.078 INFO:tasks.workunit.client.0.smithi071.stdout:9/552: symlink d1/d32/d3a/d5f/lbe 0 2024-04-23T02:14:04.084 INFO:tasks.workunit.client.1.smithi192.stdout:7/671: readlink d5/d17/d1c/d57/l36 0 2024-04-23T02:14:04.085 INFO:tasks.workunit.client.1.smithi192.stdout:7/672: readlink d5/l71 0 2024-04-23T02:14:04.085 INFO:tasks.workunit.client.1.smithi192.stdout:8/818: write d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/f104 [1013200,110410] 0 2024-04-23T02:14:04.085 INFO:tasks.workunit.client.1.smithi192.stdout:8/819: readlink d1/d12/d23/d4b/d73/d45/l54 0 2024-04-23T02:14:04.085 INFO:tasks.workunit.client.0.smithi071.stdout:7/592: mknod d3/d17/d1c/d2a/d43/cd0 0 2024-04-23T02:14:04.086 INFO:tasks.workunit.client.1.smithi192.stdout:6/700: truncate d3/f34 63758 0 2024-04-23T02:14:04.088 INFO:tasks.workunit.client.1.smithi192.stdout:2/633: rename d8/d26/d81/cce to d8/d26/d27/d57/ccf 0 2024-04-23T02:14:04.089 INFO:tasks.workunit.client.0.smithi071.stdout:9/553: mkdir d1/d32/d41/db7/d71/dbf 0 2024-04-23T02:14:04.089 INFO:tasks.workunit.client.0.smithi071.stdout:7/593: fdatasync d3/d17/d41/d34/d59/f9d 0 2024-04-23T02:14:04.089 INFO:tasks.workunit.client.1.smithi192.stdout:0/743: mkdir d6/df/d13/d25/d94/dd0/de6/dfa 0 2024-04-23T02:14:04.090 INFO:tasks.workunit.client.0.smithi071.stdout:9/554: chown d1/d32/d41/faf 56136 1 2024-04-23T02:14:04.091 INFO:tasks.workunit.client.1.smithi192.stdout:8/820: write d1/d9/d3e/d59/fff [364148,57836] 0 2024-04-23T02:14:04.092 INFO:tasks.workunit.client.1.smithi192.stdout:0/744: stat d6/df/d13/d25/d94/dd0/de6/dfa 0 2024-04-23T02:14:04.092 INFO:tasks.workunit.client.1.smithi192.stdout:3/580: link d1/d3/c86 d1/d3/d17/d1f/cc8 0 2024-04-23T02:14:04.093 INFO:tasks.workunit.client.1.smithi192.stdout:6/701: mkdir d3/d28/d79/dbe/dcb/de9 0 2024-04-23T02:14:04.093 INFO:tasks.workunit.client.0.smithi071.stdout:9/555: mknod d1/d32/d3a/d94/cc0 0 2024-04-23T02:14:04.093 INFO:tasks.workunit.client.1.smithi192.stdout:5/659: write d2/db/d2b/d52/f76 [476772,53226] 0 2024-04-23T02:14:04.094 INFO:tasks.workunit.client.0.smithi071.stdout:9/556: stat d1/d32/d41/db7/d5a 0 2024-04-23T02:14:04.095 INFO:tasks.workunit.client.1.smithi192.stdout:8/821: write d1/d12/f24 [3122090,104179] 0 2024-04-23T02:14:04.096 INFO:tasks.workunit.client.1.smithi192.stdout:2/634: mkdir d8/d26/d27/db5/dd0 0 2024-04-23T02:14:04.096 INFO:tasks.workunit.client.1.smithi192.stdout:6/702: chown d3/d15/d1b/d21/d7c 18 1 2024-04-23T02:14:04.098 INFO:tasks.workunit.client.0.smithi071.stdout:9/557: read d1/d32/d41/db7/d68/f80 [3212548,60144] 0 2024-04-23T02:14:04.100 INFO:tasks.workunit.client.1.smithi192.stdout:5/660: mknod d2/d6/d2d/d41/d43/dbc/ce0 0 2024-04-23T02:14:04.100 INFO:tasks.workunit.client.0.smithi071.stdout:9/558: symlink d1/d32/d41/db7/d39/lc1 0 2024-04-23T02:14:04.100 INFO:tasks.workunit.client.1.smithi192.stdout:2/635: read - d8/d14/faa zero size 2024-04-23T02:14:04.101 INFO:tasks.workunit.client.0.smithi071.stdout:9/559: chown d1/c21 1506 1 2024-04-23T02:14:04.101 INFO:tasks.workunit.client.1.smithi192.stdout:8/822: dread - d1/d12/d7d/df7/f106 zero size 2024-04-23T02:14:04.102 INFO:tasks.workunit.client.0.smithi071.stdout:9/560: fdatasync d1/d32/d41/faf 0 2024-04-23T02:14:04.104 INFO:tasks.workunit.client.1.smithi192.stdout:0/745: unlink d6/df/d56/d85/f19 0 2024-04-23T02:14:04.105 INFO:tasks.workunit.client.0.smithi071.stdout:9/561: mkdir d1/d32/d3a/d90/dc2 0 2024-04-23T02:14:04.105 INFO:tasks.workunit.client.1.smithi192.stdout:6/703: symlink d3/d28/d79/da0/dc2/lea 0 2024-04-23T02:14:04.107 INFO:tasks.workunit.client.0.smithi071.stdout:9/562: write d1/f30 [3432308,14174] 0 2024-04-23T02:14:04.110 INFO:tasks.workunit.client.1.smithi192.stdout:1/434: dread dc/ff [0,4194304] 0 2024-04-23T02:14:04.110 INFO:tasks.workunit.client.1.smithi192.stdout:3/581: rename d1/d3/d17/d1a/d7c/dab/d6c/d84/d46/f85 to d1/d3/d6/d7/d29/d3b/da9/dbd/fc9 0 2024-04-23T02:14:04.114 INFO:tasks.workunit.client.1.smithi192.stdout:7/673: link d5/d17/d1c/d59/f93 d5/d43/d4c/d5c/d67/da6/da9/fc3 0 2024-04-23T02:14:04.115 INFO:tasks.workunit.client.0.smithi071.stdout:7/594: write d3/d17/d41/d34/d6e/f79 [755610,121514] 0 2024-04-23T02:14:04.116 INFO:tasks.workunit.client.1.smithi192.stdout:0/746: mknod d6/df/d35/d8b/d8d/cfb 0 2024-04-23T02:14:04.116 INFO:tasks.workunit.client.0.smithi071.stdout:9/563: write d1/d32/f4a [849173,123694] 0 2024-04-23T02:14:04.116 INFO:tasks.workunit.client.1.smithi192.stdout:5/661: dread d2/db/d16/d48/d4f/f5c [0,4194304] 0 2024-04-23T02:14:04.119 INFO:tasks.workunit.client.1.smithi192.stdout:2/636: rename d8/d26/d29/d43/d5d/fb6 to d8/d26/d74/fd1 0 2024-04-23T02:14:04.119 INFO:tasks.workunit.client.1.smithi192.stdout:4/749: dread f4 [4194304,4194304] 0 2024-04-23T02:14:04.119 INFO:tasks.workunit.client.0.smithi071.stdout:9/564: symlink d1/d32/d3a/d78/d81/db6/lc3 0 2024-04-23T02:14:04.122 INFO:tasks.workunit.client.1.smithi192.stdout:2/637: chown d8/d14 63 1 2024-04-23T02:14:04.124 INFO:tasks.workunit.client.1.smithi192.stdout:6/704: dwrite d3/d15/d1b/dcf/fd3 [0,4194304] 0 2024-04-23T02:14:04.124 INFO:tasks.workunit.client.1.smithi192.stdout:5/662: chown d2/db/d16/f8b 8 1 2024-04-23T02:14:04.124 INFO:tasks.workunit.client.1.smithi192.stdout:1/435: rmdir dc/d16/d19/d74 39 2024-04-23T02:14:04.129 INFO:tasks.workunit.client.1.smithi192.stdout:0/747: creat d6/df/d13/d25/d94/dd0/ffc x:0 0 0 2024-04-23T02:14:04.130 INFO:tasks.workunit.client.1.smithi192.stdout:0/748: stat d6/fb5 0 2024-04-23T02:14:04.130 INFO:tasks.workunit.client.1.smithi192.stdout:9/553: dread d0/d9/d44/d51/d64/f78 [0,4194304] 0 2024-04-23T02:14:04.132 INFO:tasks.workunit.client.1.smithi192.stdout:3/582: read d1/d3/d6/d7/d29/fbe [162414,33615] 0 2024-04-23T02:14:04.133 INFO:tasks.workunit.client.1.smithi192.stdout:4/750: creat de/d15/d53/d81/de3/da3/ff3 x:0 0 0 2024-04-23T02:14:04.136 INFO:tasks.workunit.client.1.smithi192.stdout:5/663: truncate d2/d6/d2d/fde 595198 0 2024-04-23T02:14:04.138 INFO:tasks.workunit.client.1.smithi192.stdout:7/674: dwrite d5/d17/d1c/d59/d5a/d41/fab [0,4194304] 0 2024-04-23T02:14:04.141 INFO:tasks.workunit.client.0.smithi071.stdout:9/565: dread d1/d32/d41/db7/f25 [8388608,4194304] 0 2024-04-23T02:14:04.145 INFO:tasks.workunit.client.1.smithi192.stdout:7/675: stat d5/d17 0 2024-04-23T02:14:04.146 INFO:tasks.workunit.client.1.smithi192.stdout:0/749: chown d6/df/d13/d25/le2 323 1 2024-04-23T02:14:04.148 INFO:tasks.workunit.client.1.smithi192.stdout:1/436: rename dc/d14/d23/f83 to dc/d16/d19/d30/d32/d34/d70/f85 0 2024-04-23T02:14:04.148 INFO:tasks.workunit.client.1.smithi192.stdout:5/664: mkdir d2/db/de1 0 2024-04-23T02:14:04.150 INFO:tasks.workunit.client.1.smithi192.stdout:3/583: symlink d1/d3/d6/d7/d29/d3b/da9/dbd/lca 0 2024-04-23T02:14:04.151 INFO:tasks.workunit.client.1.smithi192.stdout:2/638: link d8/d26/d27/d57/d69/l90 d8/d26/d29/d43/d50/ld2 0 2024-04-23T02:14:04.153 INFO:tasks.workunit.client.1.smithi192.stdout:6/705: dread d3/d18/f6f [0,4194304] 0 2024-04-23T02:14:04.154 INFO:tasks.workunit.client.1.smithi192.stdout:8/823: dread d1/d9/fee [0,4194304] 0 2024-04-23T02:14:04.154 INFO:tasks.workunit.client.1.smithi192.stdout:7/676: write d5/d17/d1c/d26/f34 [528780,13259] 0 2024-04-23T02:14:04.155 INFO:tasks.workunit.client.1.smithi192.stdout:9/554: link d0/d9/d44/d4f/f28 d0/d9/d44/d51/d5b/d96/da0/fbe 0 2024-04-23T02:14:04.155 INFO:tasks.workunit.client.1.smithi192.stdout:5/665: symlink d2/db/d16/d48/dc5/le2 0 2024-04-23T02:14:04.157 INFO:tasks.workunit.client.1.smithi192.stdout:5/666: readlink d2/db/d16/l22 0 2024-04-23T02:14:04.158 INFO:tasks.workunit.client.1.smithi192.stdout:8/824: write d1/f72 [1841291,127266] 0 2024-04-23T02:14:04.158 INFO:tasks.workunit.client.1.smithi192.stdout:1/437: read dc/fd [9608689,14773] 0 2024-04-23T02:14:04.159 INFO:tasks.workunit.client.1.smithi192.stdout:4/751: dread de/d15/d53/d81/de3/d74/d92/da0/dca/d34/d4a/f71 [0,4194304] 0 2024-04-23T02:14:04.162 INFO:tasks.workunit.client.1.smithi192.stdout:5/667: chown d2/d32/f9c 361491 1 2024-04-23T02:14:04.162 INFO:tasks.workunit.client.1.smithi192.stdout:3/584: creat d1/d3/d17/d1a/d7c/dab/d6c/d84/fcb x:0 0 0 2024-04-23T02:14:04.162 INFO:tasks.workunit.client.1.smithi192.stdout:0/750: dread d6/f6d [0,4194304] 0 2024-04-23T02:14:04.165 INFO:tasks.workunit.client.1.smithi192.stdout:6/706: truncate d3/d15/d1b/d21/d52/f67 204530 0 2024-04-23T02:14:04.166 INFO:tasks.workunit.client.1.smithi192.stdout:7/677: fsync d5/d17/d1c/d59/d5a/d41/fab 0 2024-04-23T02:14:04.167 INFO:tasks.workunit.client.1.smithi192.stdout:5/668: rename d2/db/lc to d2/db/de1/le3 0 2024-04-23T02:14:04.169 INFO:tasks.workunit.client.1.smithi192.stdout:0/751: dread - d6/d4b/d9d/dc7/f80 zero size 2024-04-23T02:14:04.170 INFO:tasks.workunit.client.1.smithi192.stdout:7/678: write d5/d43/f99 [657437,70263] 0 2024-04-23T02:14:04.172 INFO:tasks.workunit.client.1.smithi192.stdout:3/585: chown d1/d3/d6/d7/d29/d3b/da9/d7b/f96 27205471 1 2024-04-23T02:14:04.174 INFO:tasks.workunit.client.1.smithi192.stdout:8/825: creat d1/d9/f11d x:0 0 0 2024-04-23T02:14:04.176 INFO:tasks.workunit.client.1.smithi192.stdout:6/707: creat d3/feb x:0 0 0 2024-04-23T02:14:04.179 INFO:tasks.workunit.client.1.smithi192.stdout:0/752: chown d6/df/d56/d85/d45 847 1 2024-04-23T02:14:04.179 INFO:tasks.workunit.client.1.smithi192.stdout:3/586: fdatasync d1/d3/d17/d2b/d44/d57/fb4 0 2024-04-23T02:14:04.181 INFO:tasks.workunit.client.1.smithi192.stdout:1/438: dwrite dc/d16/d19/d30/f3f [0,4194304] 0 2024-04-23T02:14:04.183 INFO:tasks.workunit.client.1.smithi192.stdout:2/639: dread d8/d26/f61 [0,4194304] 0 2024-04-23T02:14:04.189 INFO:tasks.workunit.client.1.smithi192.stdout:7/679: mkdir d5/d17/d1c/d59/d5a/d41/dc4 0 2024-04-23T02:14:04.194 INFO:tasks.workunit.client.1.smithi192.stdout:0/753: stat d6/df/d56/d85/cd9 0 2024-04-23T02:14:04.197 INFO:tasks.workunit.client.1.smithi192.stdout:4/752: dwrite de/d15/d53/d81/de3/d74/d92/da0/dca/d34/d4a/fc4 [0,4194304] 0 2024-04-23T02:14:04.197 INFO:tasks.workunit.client.1.smithi192.stdout:8/826: stat d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/d108/f109 0 2024-04-23T02:14:04.198 INFO:tasks.workunit.client.1.smithi192.stdout:5/669: dwrite d2/db/d2b/d52/dc2/d49/fd6 [0,4194304] 0 2024-04-23T02:14:04.199 INFO:tasks.workunit.client.1.smithi192.stdout:3/587: write d1/d3/d17/f27 [232537,5045] 0 2024-04-23T02:14:04.203 INFO:tasks.workunit.client.0.smithi071.stdout:3/519: sync 2024-04-23T02:14:04.203 INFO:tasks.workunit.client.0.smithi071.stdout:0/545: sync 2024-04-23T02:14:04.203 INFO:tasks.workunit.client.0.smithi071.stdout:5/657: sync 2024-04-23T02:14:04.203 INFO:tasks.workunit.client.0.smithi071.stdout:6/524: sync 2024-04-23T02:14:04.204 INFO:tasks.workunit.client.1.smithi192.stdout:7/680: write d5/d17/d1c/d59/d5a/d41/f54 [1117123,25973] 0 2024-04-23T02:14:04.205 INFO:tasks.workunit.client.0.smithi071.stdout:6/525: chown d6/d17/d77/l8e 78 1 2024-04-23T02:14:04.205 INFO:tasks.workunit.client.0.smithi071.stdout:7/595: link d3/d17/d41/d34/l4d d3/d17/d41/d34/d39/d3b/da7/ld1 0 2024-04-23T02:14:04.205 INFO:tasks.workunit.client.0.smithi071.stdout:5/658: dread - d3/da/d1a/fe2 zero size 2024-04-23T02:14:04.206 INFO:tasks.workunit.client.1.smithi192.stdout:6/708: mkdir d3/d15/d1b/d21/d3a/dbd/dec 0 2024-04-23T02:14:04.207 INFO:tasks.workunit.client.1.smithi192.stdout:2/640: rename d8/d26/c37 to d8/d26/d27/d57/da0/cd3 0 2024-04-23T02:14:04.207 INFO:tasks.workunit.client.1.smithi192.stdout:9/555: dwrite d0/d9/d44/f45 [0,4194304] 0 2024-04-23T02:14:04.208 INFO:tasks.workunit.client.0.smithi071.stdout:0/546: chown d0/d3/de/d10/d14/d32/d44/d31/d52 0 1 2024-04-23T02:14:04.209 INFO:tasks.workunit.client.0.smithi071.stdout:5/659: write d3/d34/d74/fdd [129862,84012] 0 2024-04-23T02:14:04.210 INFO:tasks.workunit.client.1.smithi192.stdout:0/754: chown d6/df/d35/l60 0 1 2024-04-23T02:14:04.211 INFO:tasks.workunit.client.0.smithi071.stdout:9/566: read - d1/d4e/d54/d75/fb0 zero size 2024-04-23T02:14:04.211 INFO:tasks.workunit.client.0.smithi071.stdout:6/526: mkdir d6/d3e/da4 0 2024-04-23T02:14:04.212 INFO:tasks.workunit.client.1.smithi192.stdout:7/681: chown d5/d17/d1c/d59/d5a/d41 12 1 2024-04-23T02:14:04.212 INFO:tasks.workunit.client.1.smithi192.stdout:5/670: chown d2/d6/c25 11 1 2024-04-23T02:14:04.212 INFO:tasks.workunit.client.1.smithi192.stdout:6/709: write d3/d28/f56 [2809607,121406] 0 2024-04-23T02:14:04.213 INFO:tasks.workunit.client.1.smithi192.stdout:2/641: truncate d8/fc3 277113 0 2024-04-23T02:14:04.215 INFO:tasks.workunit.client.1.smithi192.stdout:1/439: dwrite dc/fd [8388608,4194304] 0 2024-04-23T02:14:04.215 INFO:tasks.workunit.client.1.smithi192.stdout:4/753: mknod de/d15/d53/d81/de3/d74/d92/da0/dca/d9f/cf4 0 2024-04-23T02:14:04.216 INFO:tasks.workunit.client.1.smithi192.stdout:2/642: write d8/fc3 [245509,16867] 0 2024-04-23T02:14:04.219 INFO:tasks.workunit.client.1.smithi192.stdout:2/643: stat d8/d26/d27/d57/da0 0 2024-04-23T02:14:04.224 INFO:tasks.workunit.client.1.smithi192.stdout:3/588: mknod d1/d3/d6/d7/d29/d3b/da9/daf/ccc 0 2024-04-23T02:14:04.225 INFO:tasks.workunit.client.0.smithi071.stdout:3/520: rename d7/d1d/c8f to d7/d8/d42/d5f/d68/d3b/d98/cb4 0 2024-04-23T02:14:04.225 INFO:tasks.workunit.client.0.smithi071.stdout:0/547: creat d0/d3/de/d10/d14/d32/d44/d31/fb8 x:0 0 0 2024-04-23T02:14:04.225 INFO:tasks.workunit.client.0.smithi071.stdout:6/527: unlink d6/d1f/d46/f56 0 2024-04-23T02:14:04.226 INFO:tasks.workunit.client.0.smithi071.stdout:7/596: creat d3/d17/d1c/d2a/fd2 x:0 0 0 2024-04-23T02:14:04.227 INFO:tasks.workunit.client.0.smithi071.stdout:3/521: chown d7/d8/d42/d5f/d68/d3b/d4c/d8b/db3/c7d 38 1 2024-04-23T02:14:04.229 INFO:tasks.workunit.client.0.smithi071.stdout:6/528: write d6/fc [49835,68300] 0 2024-04-23T02:14:04.229 INFO:tasks.workunit.client.0.smithi071.stdout:5/660: mkdir d3/da/d1a/d1c/d46/d4e/dd8/df3 0 2024-04-23T02:14:04.230 INFO:tasks.workunit.client.0.smithi071.stdout:0/548: read d0/d63/f66 [2586233,10921] 0 2024-04-23T02:14:04.231 INFO:tasks.workunit.client.1.smithi192.stdout:0/755: creat d6/df/d35/d8b/d8d/dea/ffd x:0 0 0 2024-04-23T02:14:04.231 INFO:tasks.workunit.client.1.smithi192.stdout:5/671: rename d2/d6/d2d/faa to d2/d2e/d4e/fe4 0 2024-04-23T02:14:04.231 INFO:tasks.workunit.client.1.smithi192.stdout:3/589: chown d1/d3/d17/d1a/d7c/f91 14476 1 2024-04-23T02:14:04.233 INFO:tasks.workunit.client.0.smithi071.stdout:7/597: symlink d3/d17/d1c/d31/ld3 0 2024-04-23T02:14:04.233 INFO:tasks.workunit.client.1.smithi192.stdout:4/754: mknod de/d15/d53/d81/de3/d74/d92/cf5 0 2024-04-23T02:14:04.233 INFO:tasks.workunit.client.1.smithi192.stdout:8/827: write d1/d12/d23/d4b/f69 [140917,110177] 0 2024-04-23T02:14:04.233 INFO:tasks.workunit.client.0.smithi071.stdout:3/522: mkdir d7/da5/db5 0 2024-04-23T02:14:04.233 INFO:tasks.workunit.client.0.smithi071.stdout:0/549: chown d0/d3/de/d10/d16/f77 72 1 2024-04-23T02:14:04.235 INFO:tasks.workunit.client.0.smithi071.stdout:0/550: fdatasync d0/d3/de/d10/d14/d32/d37/d26/f9a 0 2024-04-23T02:14:04.236 INFO:tasks.workunit.client.1.smithi192.stdout:6/710: dwrite d3/d15/d1b/d21/d3a/fa6 [0,4194304] 0 2024-04-23T02:14:04.236 INFO:tasks.workunit.client.0.smithi071.stdout:6/529: mknod d6/d3e/ca5 0 2024-04-23T02:14:04.236 INFO:tasks.workunit.client.0.smithi071.stdout:5/661: mknod d3/d1d/d43/dd2/d5a/cf4 0 2024-04-23T02:14:04.237 INFO:tasks.workunit.client.1.smithi192.stdout:7/682: rename d5/d43/lb8 to d5/d43/d4c/d5c/d67/da6/da9/db9/lc5 0 2024-04-23T02:14:04.238 INFO:tasks.workunit.client.0.smithi071.stdout:3/523: unlink d7/d10/d38/f43 0 2024-04-23T02:14:04.240 INFO:tasks.workunit.client.0.smithi071.stdout:7/598: symlink d3/ld4 0 2024-04-23T02:14:04.242 INFO:tasks.workunit.client.0.smithi071.stdout:0/551: mkdir d0/d3/de/d10/db9 0 2024-04-23T02:14:04.245 INFO:tasks.workunit.client.0.smithi071.stdout:7/599: truncate d3/d17/d41/d34/d59/d5f/f60 4108875 0 2024-04-23T02:14:04.245 INFO:tasks.workunit.client.0.smithi071.stdout:6/530: link d6/d1f/c9d d6/d3e/d3f/ca6 0 2024-04-23T02:14:04.246 INFO:tasks.workunit.client.1.smithi192.stdout:2/644: creat d8/d26/d29/da4/dca/fd4 x:0 0 0 2024-04-23T02:14:04.246 INFO:tasks.workunit.client.1.smithi192.stdout:5/672: creat d2/db/d16/fe5 x:0 0 0 2024-04-23T02:14:04.246 INFO:tasks.workunit.client.1.smithi192.stdout:0/756: truncate d6/df/d35/d8b/d8d/dea/ffd 27753 0 2024-04-23T02:14:04.246 INFO:tasks.workunit.client.1.smithi192.stdout:6/711: mkdir d3/d15/d1b/d66/d80/d8e/d9d/ded 0 2024-04-23T02:14:04.247 INFO:tasks.workunit.client.1.smithi192.stdout:5/673: readlink d2/db/d16/d48/dc5/le2 0 2024-04-23T02:14:04.248 INFO:tasks.workunit.client.0.smithi071.stdout:0/552: mkdir d0/d3/de/d10/db3/dba 0 2024-04-23T02:14:04.248 INFO:tasks.workunit.client.0.smithi071.stdout:7/600: write d3/d17/d41/f2f [3872997,32540] 0 2024-04-23T02:14:04.249 INFO:tasks.workunit.client.0.smithi071.stdout:0/553: chown d0/d3/de/d10/d14/d32/fa4 7015758 1 2024-04-23T02:14:04.250 INFO:tasks.workunit.client.1.smithi192.stdout:2/645: rename d8/d25/f32 to d8/d26/d29/d43/d5d/fd5 0 2024-04-23T02:14:04.251 INFO:tasks.workunit.client.0.smithi071.stdout:6/531: rename d6/d22/f53 to d6/d17/d77/da2/fa7 0 2024-04-23T02:14:04.252 INFO:tasks.workunit.client.0.smithi071.stdout:6/532: chown d6/d3e/f4a 51471 1 2024-04-23T02:14:04.253 INFO:tasks.workunit.client.1.smithi192.stdout:5/674: unlink d2/l9 0 2024-04-23T02:14:04.254 INFO:tasks.workunit.client.1.smithi192.stdout:4/755: creat de/d15/d19/ff6 x:0 0 0 2024-04-23T02:14:04.254 INFO:tasks.workunit.client.0.smithi071.stdout:0/554: symlink d0/d3/de/d10/d14/d32/d44/d31/d52/lbb 0 2024-04-23T02:14:04.255 INFO:tasks.workunit.client.0.smithi071.stdout:5/662: dread d3/da/d1a/f2b [0,4194304] 0 2024-04-23T02:14:04.255 INFO:tasks.workunit.client.0.smithi071.stdout:6/533: write d6/d1f/f83 [928365,18915] 0 2024-04-23T02:14:04.255 INFO:tasks.workunit.client.0.smithi071.stdout:7/601: truncate d3/d17/d1c/d31/f5d 640799 0 2024-04-23T02:14:04.256 INFO:tasks.workunit.client.0.smithi071.stdout:5/663: chown d3/d1d/f42 5 1 2024-04-23T02:14:04.257 INFO:tasks.workunit.client.1.smithi192.stdout:2/646: fdatasync d8/d26/d29/d43/f83 0 2024-04-23T02:14:04.258 INFO:tasks.workunit.client.0.smithi071.stdout:7/602: dread d3/d17/d41/d34/d39/d3b/f84 [0,4194304] 0 2024-04-23T02:14:04.261 INFO:tasks.workunit.client.0.smithi071.stdout:5/664: rename d3/d8f/fa2 to d3/d1d/daf/d83/ff5 0 2024-04-23T02:14:04.261 INFO:tasks.workunit.client.1.smithi192.stdout:0/757: truncate d6/df/d13/d25/f91 5223768 0 2024-04-23T02:14:04.263 INFO:tasks.workunit.client.1.smithi192.stdout:7/683: unlink d5/c49 0 2024-04-23T02:14:04.264 INFO:tasks.workunit.client.1.smithi192.stdout:9/556: dread d0/d9/d44/f47 [0,4194304] 0 2024-04-23T02:14:04.265 INFO:tasks.workunit.client.0.smithi071.stdout:7/603: symlink d3/ld5 0 2024-04-23T02:14:04.265 INFO:tasks.workunit.client.0.smithi071.stdout:0/555: dwrite d0/d3/de/d10/d14/d32/f41 [0,4194304] 0 2024-04-23T02:14:04.266 INFO:tasks.workunit.client.1.smithi192.stdout:6/712: rename d3/f22 to d3/d15/d1b/d66/d80/d8e/fee 0 2024-04-23T02:14:04.266 INFO:tasks.workunit.client.1.smithi192.stdout:8/828: dwrite d1/d9/d3e/d59/d8a/f8d [4194304,4194304] 0 2024-04-23T02:14:04.267 INFO:tasks.workunit.client.0.smithi071.stdout:5/665: mkdir d3/da/df6 0 2024-04-23T02:14:04.270 INFO:tasks.workunit.client.1.smithi192.stdout:3/590: link d1/d3/d17/d1a/d7c/fb5 d1/d3/d6/d7/d5f/fcd 0 2024-04-23T02:14:04.273 INFO:tasks.workunit.client.0.smithi071.stdout:5/666: getdents d3/d34/d74 0 2024-04-23T02:14:04.277 INFO:tasks.workunit.client.0.smithi071.stdout:3/524: dread d7/d8/f1a [0,4194304] 0 2024-04-23T02:14:04.277 INFO:tasks.workunit.client.0.smithi071.stdout:0/556: dwrite d0/d3/de/d10/d14/d32/d44/d31/fb8 [0,4194304] 0 2024-04-23T02:14:04.281 INFO:tasks.workunit.client.1.smithi192.stdout:4/756: mknod de/d15/d53/d81/de3/d74/d92/cf7 0 2024-04-23T02:14:04.284 INFO:tasks.workunit.client.1.smithi192.stdout:3/591: readlink d1/d3/d6/d7/d29/d3b/l77 0 2024-04-23T02:14:04.284 INFO:tasks.workunit.client.0.smithi071.stdout:7/604: write d3/d17/d1c/d2a/d43/f4e [1505590,32071] 0 2024-04-23T02:14:04.285 INFO:tasks.workunit.client.1.smithi192.stdout:6/713: creat d3/d15/d1b/d21/d7c/dda/fef x:0 0 0 2024-04-23T02:14:04.285 INFO:tasks.workunit.client.1.smithi192.stdout:0/758: creat d6/df/d13/d25/d26/d67/ffe x:0 0 0 2024-04-23T02:14:04.285 INFO:tasks.workunit.client.1.smithi192.stdout:2/647: creat d8/d26/d29/d43/fd6 x:0 0 0 2024-04-23T02:14:04.286 INFO:tasks.workunit.client.0.smithi071.stdout:5/667: truncate d3/d1d/d43/f75 1013921 0 2024-04-23T02:14:04.287 INFO:tasks.workunit.client.1.smithi192.stdout:8/829: rename d1/df/ff4 to d1/d9/d3e/d59/d8a/daa/f11e 0 2024-04-23T02:14:04.288 INFO:tasks.workunit.client.1.smithi192.stdout:0/759: write d6/fb5 [844040,59072] 0 2024-04-23T02:14:04.289 INFO:tasks.workunit.client.0.smithi071.stdout:3/525: link d7/fd d7/da5/db5/fb6 0 2024-04-23T02:14:04.290 INFO:tasks.workunit.client.1.smithi192.stdout:3/592: creat d1/d3/d17/d1f/fce x:0 0 0 2024-04-23T02:14:04.290 INFO:tasks.workunit.client.0.smithi071.stdout:0/557: creat d0/d3/de/d10/d14/d32/da8/fbc x:0 0 0 2024-04-23T02:14:04.290 INFO:tasks.workunit.client.0.smithi071.stdout:6/534: dwrite d6/d1f/f21 [0,4194304] 0 2024-04-23T02:14:04.291 INFO:tasks.workunit.client.1.smithi192.stdout:2/648: stat d8/d26/d27/d53/da2/fb9 0 2024-04-23T02:14:04.291 INFO:tasks.workunit.client.1.smithi192.stdout:0/760: fdatasync d6/df/f83 0 2024-04-23T02:14:04.292 INFO:tasks.workunit.client.1.smithi192.stdout:7/684: dread d5/d17/d1c/f3c [0,4194304] 0 2024-04-23T02:14:04.292 INFO:tasks.workunit.client.0.smithi071.stdout:3/526: read d7/d8/d42/d5f/d68/d3b/d4c/f63 [2071510,28873] 0 2024-04-23T02:14:04.292 INFO:tasks.workunit.client.1.smithi192.stdout:5/675: dwrite d2/db/d2b/d52/dc2/f81 [0,4194304] 0 2024-04-23T02:14:04.295 INFO:tasks.workunit.client.1.smithi192.stdout:8/830: write d1/d12/d23/d4b/fdc [253614,34478] 0 2024-04-23T02:14:04.296 INFO:tasks.workunit.client.0.smithi071.stdout:7/605: mknod d3/d17/d41/d34/d6e/cd6 0 2024-04-23T02:14:04.296 INFO:tasks.workunit.client.0.smithi071.stdout:6/535: symlink d6/d1f/d5d/la8 0 2024-04-23T02:14:04.296 INFO:tasks.workunit.client.0.smithi071.stdout:5/668: dread d3/d1d/d43/dd2/f58 [0,4194304] 0 2024-04-23T02:14:04.297 INFO:tasks.workunit.client.1.smithi192.stdout:6/714: write d3/d15/d1b/d21/d3a/fa6 [1554826,103946] 0 2024-04-23T02:14:04.299 INFO:tasks.workunit.client.0.smithi071.stdout:3/527: rename d7/d8/d42/d5f to d7/da5/db2/db7 0 2024-04-23T02:14:04.300 INFO:tasks.workunit.client.0.smithi071.stdout:7/606: rmdir d3/d17/d1c/d2a/d43/d86 39 2024-04-23T02:14:04.303 INFO:tasks.workunit.client.1.smithi192.stdout:7/685: chown d5/d17/d1c/d59/d5a/d41/dc4 1838 1 2024-04-23T02:14:04.303 INFO:tasks.workunit.client.0.smithi071.stdout:6/536: mknod d6/d1f/d5d/ca9 0 2024-04-23T02:14:04.304 INFO:tasks.workunit.client.0.smithi071.stdout:3/528: chown d7/d8/d42/d80/d88/l39 1 1 2024-04-23T02:14:04.304 INFO:tasks.workunit.client.1.smithi192.stdout:0/761: read d6/df/d13/d25/d3a/f6f [3917229,1116] 0 2024-04-23T02:14:04.306 INFO:tasks.workunit.client.1.smithi192.stdout:4/757: mkdir de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/de7/df8 0 2024-04-23T02:14:04.307 INFO:tasks.workunit.client.0.smithi071.stdout:6/537: write d6/d1f/f32 [752535,92405] 0 2024-04-23T02:14:04.307 INFO:tasks.workunit.client.1.smithi192.stdout:7/686: stat d5/d43/d4c/d5c/d97/faf 0 2024-04-23T02:14:04.307 INFO:tasks.workunit.client.1.smithi192.stdout:3/593: creat d1/d3/d17/d1f/d33/fcf x:0 0 0 2024-04-23T02:14:04.308 INFO:tasks.workunit.client.1.smithi192.stdout:6/715: symlink d3/d15/d1b/lf0 0 2024-04-23T02:14:04.309 INFO:tasks.workunit.client.0.smithi071.stdout:7/607: dread d3/d17/d41/d3a/da5/fbe [0,4194304] 0 2024-04-23T02:14:04.309 INFO:tasks.workunit.client.0.smithi071.stdout:5/669: creat d3/da/d1a/d1c/d46/d4e/dd8/ff7 x:0 0 0 2024-04-23T02:14:04.310 INFO:tasks.workunit.client.1.smithi192.stdout:9/557: write d0/d9/f5a [215162,125954] 0 2024-04-23T02:14:04.310 INFO:tasks.workunit.client.0.smithi071.stdout:6/538: mknod d6/d3e/caa 0 2024-04-23T02:14:04.313 INFO:tasks.workunit.client.1.smithi192.stdout:9/558: dread - d0/d98/fb8 zero size 2024-04-23T02:14:04.313 INFO:tasks.workunit.client.0.smithi071.stdout:6/539: fsync d6/d3e/f5e 0 2024-04-23T02:14:04.313 INFO:tasks.workunit.client.0.smithi071.stdout:7/608: write d3/d17/d1c/d31/f56 [5176745,81942] 0 2024-04-23T02:14:04.313 INFO:tasks.workunit.client.0.smithi071.stdout:5/670: chown d3/da/d1a/d6d/fbe 51143 1 2024-04-23T02:14:04.314 INFO:tasks.workunit.client.1.smithi192.stdout:3/594: creat d1/d3/fd0 x:0 0 0 2024-04-23T02:14:04.315 INFO:tasks.workunit.client.1.smithi192.stdout:7/687: creat d5/d43/d4c/d4e/fc6 x:0 0 0 2024-04-23T02:14:04.316 INFO:tasks.workunit.client.1.smithi192.stdout:2/649: dwrite d8/d26/f6b [0,4194304] 0 2024-04-23T02:14:04.316 INFO:tasks.workunit.client.1.smithi192.stdout:0/762: chown d6/df/d56/d85/da4/lde 242 1 2024-04-23T02:14:04.316 INFO:tasks.workunit.client.0.smithi071.stdout:3/529: dwrite d7/da5/db2/db7/d68/f91 [0,4194304] 0 2024-04-23T02:14:04.317 INFO:tasks.workunit.client.0.smithi071.stdout:6/540: write d6/d3e/f8d [37482,4527] 0 2024-04-23T02:14:04.319 INFO:tasks.workunit.client.1.smithi192.stdout:9/559: chown d0/c29 102 1 2024-04-23T02:14:04.320 INFO:tasks.workunit.client.0.smithi071.stdout:5/671: creat d3/d1d/d43/dd2/ff8 x:0 0 0 2024-04-23T02:14:04.322 INFO:tasks.workunit.client.0.smithi071.stdout:3/530: fdatasync d7/da5/db2/db7/d68/d3b/d4c/f63 0 2024-04-23T02:14:04.324 INFO:tasks.workunit.client.0.smithi071.stdout:3/531: chown d7/d8/f1a 345021 1 2024-04-23T02:14:04.325 INFO:tasks.workunit.client.1.smithi192.stdout:3/595: mkdir d1/d3/d17/dd1 0 2024-04-23T02:14:04.325 INFO:tasks.workunit.client.1.smithi192.stdout:0/763: stat d6/df/d56/d85/d71 0 2024-04-23T02:14:04.327 INFO:tasks.workunit.client.1.smithi192.stdout:9/560: write d0/d9/d44/f47 [4247720,64568] 0 2024-04-23T02:14:04.327 INFO:tasks.workunit.client.0.smithi071.stdout:5/672: write d3/da/d1a/d6d/fba [796266,119192] 0 2024-04-23T02:14:04.327 INFO:tasks.workunit.client.0.smithi071.stdout:7/609: creat d3/d17/d1c/d2a/d43/d86/fd7 x:0 0 0 2024-04-23T02:14:04.327 INFO:tasks.workunit.client.0.smithi071.stdout:5/673: stat d3/d34/l6f 0 2024-04-23T02:14:04.328 INFO:tasks.workunit.client.1.smithi192.stdout:4/758: truncate de/d15/d53/d81/de3/d74/d92/da0/dca/f54 4723580 0 2024-04-23T02:14:04.330 INFO:tasks.workunit.client.0.smithi071.stdout:3/532: creat d7/da5/db2/db7/d68/d3b/fb8 x:0 0 0 2024-04-23T02:14:04.331 INFO:tasks.workunit.client.0.smithi071.stdout:5/674: mknod d3/d1d/daf/d83/cf9 0 2024-04-23T02:14:04.331 INFO:tasks.workunit.client.0.smithi071.stdout:6/541: link d6/f3c d6/d17/d77/d85/fab 0 2024-04-23T02:14:04.332 INFO:tasks.workunit.client.1.smithi192.stdout:9/561: unlink d0/d98/fb8 0 2024-04-23T02:14:04.333 INFO:tasks.workunit.client.0.smithi071.stdout:6/542: fsync d6/d17/d77/d85/f80 0 2024-04-23T02:14:04.333 INFO:tasks.workunit.client.1.smithi192.stdout:0/764: stat d6/df/d13/l43 0 2024-04-23T02:14:04.333 INFO:tasks.workunit.client.0.smithi071.stdout:7/610: rename d3/d17/d41/fa0 to d3/d17/d1c/d31/fd8 0 2024-04-23T02:14:04.334 INFO:tasks.workunit.client.1.smithi192.stdout:3/596: creat d1/d3/d17/d1a/d7c/fd2 x:0 0 0 2024-04-23T02:14:04.336 INFO:tasks.workunit.client.1.smithi192.stdout:0/765: write d6/df/f4a [1318350,10976] 0 2024-04-23T02:14:04.337 INFO:tasks.workunit.client.1.smithi192.stdout:8/831: dread d1/d9/d3e/fb2 [0,4194304] 0 2024-04-23T02:14:04.337 INFO:tasks.workunit.client.1.smithi192.stdout:9/562: symlink d0/d98/lbf 0 2024-04-23T02:14:04.337 INFO:tasks.workunit.client.1.smithi192.stdout:8/832: chown d1/de1 8772 1 2024-04-23T02:14:04.338 INFO:tasks.workunit.client.0.smithi071.stdout:5/675: mkdir d3/d1d/daf/d10/da5/d5f/dc0/dfa 0 2024-04-23T02:14:04.338 INFO:tasks.workunit.client.1.smithi192.stdout:7/688: dread d5/d17/d1c/d57/f90 [0,4194304] 0 2024-04-23T02:14:04.339 INFO:tasks.workunit.client.0.smithi071.stdout:7/611: symlink d3/d17/ld9 0 2024-04-23T02:14:04.341 INFO:tasks.workunit.client.1.smithi192.stdout:0/766: creat d6/df/d56/fff x:0 0 0 2024-04-23T02:14:04.342 INFO:tasks.workunit.client.0.smithi071.stdout:5/676: truncate d3/d34/f8b 853216 0 2024-04-23T02:14:04.342 INFO:tasks.workunit.client.0.smithi071.stdout:7/612: getdents d3/d17/d41/d34/d59/dc5 0 2024-04-23T02:14:04.344 INFO:tasks.workunit.client.0.smithi071.stdout:5/677: write d3/d34/d6b/de4/ff0 [872343,56356] 0 2024-04-23T02:14:04.344 INFO:tasks.workunit.client.1.smithi192.stdout:4/759: rename de/d15/d53/d81/de3/d74/d92/da0/dca/d34/d7c to de/d15/d53/d81/de3/d74/d92/da0/dca/d34/df9 0 2024-04-23T02:14:04.345 INFO:tasks.workunit.client.1.smithi192.stdout:8/833: dread - d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/da3/d114/f111 zero size 2024-04-23T02:14:04.345 INFO:tasks.workunit.client.0.smithi071.stdout:1/519: sync 2024-04-23T02:14:04.345 INFO:tasks.workunit.client.0.smithi071.stdout:8/596: sync 2024-04-23T02:14:04.345 INFO:tasks.workunit.client.0.smithi071.stdout:2/538: sync 2024-04-23T02:14:04.346 INFO:tasks.workunit.client.1.smithi192.stdout:0/767: mkdir d6/d4b/d9d/dc7/d100 0 2024-04-23T02:14:04.347 INFO:tasks.workunit.client.1.smithi192.stdout:6/716: dread d3/f2b [4194304,4194304] 0 2024-04-23T02:14:04.348 INFO:tasks.workunit.client.1.smithi192.stdout:5/676: dread d2/f5 [0,4194304] 0 2024-04-23T02:14:04.349 INFO:tasks.workunit.client.0.smithi071.stdout:3/533: write d7/da5/db2/db7/f40 [9190206,41965] 0 2024-04-23T02:14:04.350 INFO:tasks.workunit.client.0.smithi071.stdout:1/520: unlink db/d34/fa6 0 2024-04-23T02:14:04.353 INFO:tasks.workunit.client.0.smithi071.stdout:8/597: mknod dc/d15/d5d/cbb 0 2024-04-23T02:14:04.357 INFO:tasks.workunit.client.0.smithi071.stdout:3/534: stat d7/da5/db2/db7/d68/d3b/d4c/d8b/db3 0 2024-04-23T02:14:04.360 INFO:tasks.workunit.client.0.smithi071.stdout:5/678: dwrite d3/d1d/daf/d83/fc9 [4194304,4194304] 0 2024-04-23T02:14:04.361 INFO:tasks.workunit.client.0.smithi071.stdout:8/598: creat dc/d15/d72/d95/fbc x:0 0 0 2024-04-23T02:14:04.362 INFO:tasks.workunit.client.0.smithi071.stdout:2/539: dwrite d0/d2/d34/d55/f72 [0,4194304] 0 2024-04-23T02:14:04.363 INFO:tasks.workunit.client.0.smithi071.stdout:6/543: dread d6/f8 [0,4194304] 0 2024-04-23T02:14:04.364 INFO:tasks.workunit.client.0.smithi071.stdout:3/535: creat d7/d1d/fb9 x:0 0 0 2024-04-23T02:14:04.367 INFO:tasks.workunit.client.0.smithi071.stdout:1/521: write db/df/d13/d2f/f3e [994594,57595] 0 2024-04-23T02:14:04.368 INFO:tasks.workunit.client.0.smithi071.stdout:3/536: chown f0 0 1 2024-04-23T02:14:04.370 INFO:tasks.workunit.client.0.smithi071.stdout:6/544: write d6/d91/f9c [1589512,93919] 0 2024-04-23T02:14:04.372 INFO:tasks.workunit.client.0.smithi071.stdout:8/599: creat dc/d56/d64/fbd x:0 0 0 2024-04-23T02:14:04.375 INFO:tasks.workunit.client.0.smithi071.stdout:2/540: write d0/d2/d45/f52 [531781,48066] 0 2024-04-23T02:14:04.378 INFO:tasks.workunit.client.0.smithi071.stdout:2/541: fdatasync d0/d2/d26/df/f3b 0 2024-04-23T02:14:04.379 INFO:tasks.workunit.client.0.smithi071.stdout:7/613: dread d3/dae/fcc [0,4194304] 0 2024-04-23T02:14:04.380 INFO:tasks.workunit.client.0.smithi071.stdout:1/522: dwrite db/df/d3a/f44 [0,4194304] 0 2024-04-23T02:14:04.382 INFO:tasks.workunit.client.0.smithi071.stdout:1/523: dread - db/d34/d68/f77 zero size 2024-04-23T02:14:04.387 INFO:tasks.workunit.client.0.smithi071.stdout:1/524: truncate db/df/d13/d20/f32 4558588 0 2024-04-23T02:14:04.389 INFO:tasks.workunit.client.0.smithi071.stdout:1/525: truncate db/d34/d8a/f8f 319889 0 2024-04-23T02:14:04.390 INFO:tasks.workunit.client.0.smithi071.stdout:7/614: dwrite d3/d17/d41/d34/d39/d3b/f49 [0,4194304] 0 2024-04-23T02:14:04.390 INFO:tasks.workunit.client.0.smithi071.stdout:1/526: stat db/f54 0 2024-04-23T02:14:04.391 INFO:tasks.workunit.client.0.smithi071.stdout:1/527: fsync db/d11/f1d 0 2024-04-23T02:14:04.398 INFO:tasks.workunit.client.0.smithi071.stdout:7/615: dread d3/fd [0,4194304] 0 2024-04-23T02:14:04.413 INFO:tasks.workunit.client.0.smithi071.stdout:8/600: creat dc/d15/d50/fbe x:0 0 0 2024-04-23T02:14:04.414 INFO:tasks.workunit.client.0.smithi071.stdout:6/545: getdents d6/d17/d77/d9e 0 2024-04-23T02:14:04.414 INFO:tasks.workunit.client.0.smithi071.stdout:6/546: readlink d6/d3e/d3f/l6a 0 2024-04-23T02:14:04.417 INFO:tasks.workunit.client.0.smithi071.stdout:3/537: write d7/d8/f4e [322325,76539] 0 2024-04-23T02:14:04.420 INFO:tasks.workunit.client.0.smithi071.stdout:7/616: chown d3/d17/d41/d34/d59/d5f/f60 1 1 2024-04-23T02:14:04.423 INFO:tasks.workunit.client.0.smithi071.stdout:2/542: link d0/d2/d26/d14/d1f/f95 d0/d2/d26/d14/d1f/d36/d6d/faf 0 2024-04-23T02:14:04.424 INFO:tasks.workunit.client.0.smithi071.stdout:3/538: rmdir d7/da5/db2/db7/d68/d3b/d4c/da1 39 2024-04-23T02:14:04.426 INFO:tasks.workunit.client.0.smithi071.stdout:7/617: rmdir d3/d51/d87/db3 39 2024-04-23T02:14:04.429 INFO:tasks.workunit.client.0.smithi071.stdout:1/528: getdents db/d11/d8d 0 2024-04-23T02:14:04.429 INFO:tasks.workunit.client.0.smithi071.stdout:3/539: write d7/da5/db2/db7/d68/d3b/d4c/d8b/db3/f7a [779054,86657] 0 2024-04-23T02:14:04.432 INFO:tasks.workunit.client.0.smithi071.stdout:8/601: dwrite dc/d15/d72/d95/fb8 [0,4194304] 0 2024-04-23T02:14:04.435 INFO:tasks.workunit.client.0.smithi071.stdout:7/618: dread d3/d17/d41/d34/d6e/f79 [0,4194304] 0 2024-04-23T02:14:04.435 INFO:tasks.workunit.client.0.smithi071.stdout:8/602: truncate dc/d15/da0/d55/f98 832992 0 2024-04-23T02:14:04.436 INFO:tasks.workunit.client.0.smithi071.stdout:1/529: unlink db/df/d5f/d6f/f6b 0 2024-04-23T02:14:04.436 INFO:tasks.workunit.client.0.smithi071.stdout:3/540: mkdir d7/d22/d23/dba 0 2024-04-23T02:14:04.437 INFO:tasks.workunit.client.0.smithi071.stdout:8/603: fdatasync dc/d15/f86 0 2024-04-23T02:14:04.439 INFO:tasks.workunit.client.0.smithi071.stdout:1/530: mkdir db/d11/d7f/da7 0 2024-04-23T02:14:04.439 INFO:tasks.workunit.client.0.smithi071.stdout:7/619: write d3/d17/d1c/d2a/d43/d86/fd7 [121911,66903] 0 2024-04-23T02:14:04.440 INFO:tasks.workunit.client.0.smithi071.stdout:6/547: dread d6/d17/f2c [0,4194304] 0 2024-04-23T02:14:04.442 INFO:tasks.workunit.client.0.smithi071.stdout:3/541: unlink d7/d22/f55 0 2024-04-23T02:14:04.443 INFO:tasks.workunit.client.0.smithi071.stdout:6/548: symlink d6/d1f/d46/lac 0 2024-04-23T02:14:04.444 INFO:tasks.workunit.client.0.smithi071.stdout:3/542: write d7/d10/f2f [3600596,63513] 0 2024-04-23T02:14:04.444 INFO:tasks.workunit.client.0.smithi071.stdout:6/549: dread - d6/d1f/d5d/f87 zero size 2024-04-23T02:14:04.452 INFO:tasks.workunit.client.0.smithi071.stdout:6/550: unlink d6/d1f/d46/c7d 0 2024-04-23T02:14:04.455 INFO:tasks.workunit.client.0.smithi071.stdout:8/604: dwrite dc/d15/f6b [0,4194304] 0 2024-04-23T02:14:04.456 INFO:tasks.workunit.client.0.smithi071.stdout:7/620: truncate d3/d17/d1c/f64 3489257 0 2024-04-23T02:14:04.459 INFO:tasks.workunit.client.0.smithi071.stdout:3/543: unlink d7/da5/db2/db7/d68/d3b/c6e 0 2024-04-23T02:14:04.461 INFO:tasks.workunit.client.0.smithi071.stdout:3/544: chown d7/da5/db2/db7/f8c 9058469 1 2024-04-23T02:14:04.464 INFO:tasks.workunit.client.0.smithi071.stdout:6/551: rename d6/d3e to d6/d17/d77/d9f/dad 0 2024-04-23T02:14:04.466 INFO:tasks.workunit.client.0.smithi071.stdout:8/605: creat dc/d15/da0/d32/fbf x:0 0 0 2024-04-23T02:14:04.467 INFO:tasks.workunit.client.0.smithi071.stdout:7/621: mkdir d3/d17/d1c/d2a/d73/dda 0 2024-04-23T02:14:04.468 INFO:tasks.workunit.client.0.smithi071.stdout:2/543: dread d0/f1e [0,4194304] 0 2024-04-23T02:14:04.469 INFO:tasks.workunit.client.0.smithi071.stdout:6/552: creat d6/d22/d92/fae x:0 0 0 2024-04-23T02:14:04.470 INFO:tasks.workunit.client.0.smithi071.stdout:3/545: dwrite d7/da5/db2/db7/d68/f84 [0,4194304] 0 2024-04-23T02:14:04.472 INFO:tasks.workunit.client.0.smithi071.stdout:7/622: chown d3/d17/d41/d34/d39/d3b/f49 1070799 1 2024-04-23T02:14:04.473 INFO:tasks.workunit.client.1.smithi192.stdout:7/689: symlink d5/d17/d1c/d57/d75/lc7 0 2024-04-23T02:14:04.478 INFO:tasks.workunit.client.0.smithi071.stdout:7/623: symlink d3/d17/ldb 0 2024-04-23T02:14:04.482 INFO:tasks.workunit.client.0.smithi071.stdout:8/606: dwrite dc/d15/da0/f85 [0,4194304] 0 2024-04-23T02:14:04.483 INFO:tasks.workunit.client.1.smithi192.stdout:8/834: mkdir d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/d108/d11f 0 2024-04-23T02:14:04.484 INFO:tasks.workunit.client.0.smithi071.stdout:2/544: rename d0/d2/d26/df to d0/d2/d45/d7d/d56/db0 0 2024-04-23T02:14:04.485 INFO:tasks.workunit.client.0.smithi071.stdout:6/553: dwrite d6/d17/d77/d85/fa1 [0,4194304] 0 2024-04-23T02:14:04.487 INFO:tasks.workunit.client.0.smithi071.stdout:6/554: write d6/fc [45165,4240] 0 2024-04-23T02:14:04.488 INFO:tasks.workunit.client.0.smithi071.stdout:1/531: truncate db/df/d13/d2f/d7b/f9d 636892 0 2024-04-23T02:14:04.488 INFO:tasks.workunit.client.0.smithi071.stdout:3/546: dread d7/da5/db2/db7/d68/d3b/d4c/f63 [0,4194304] 0 2024-04-23T02:14:04.489 INFO:tasks.workunit.client.1.smithi192.stdout:2/650: dwrite d8/d26/d27/d53/f54 [0,4194304] 0 2024-04-23T02:14:04.489 INFO:tasks.workunit.client.0.smithi071.stdout:3/547: stat d7/d8/d42 0 2024-04-23T02:14:04.490 INFO:tasks.workunit.client.0.smithi071.stdout:7/624: rename d3/d17/d41/d37/c9c to d3/d51/d80/cdc 0 2024-04-23T02:14:04.491 INFO:tasks.workunit.client.0.smithi071.stdout:6/555: creat d6/d17/d77/faf x:0 0 0 2024-04-23T02:14:04.492 INFO:tasks.workunit.client.0.smithi071.stdout:3/548: truncate d7/d8/f1a 4779313 0 2024-04-23T02:14:04.493 INFO:tasks.workunit.client.1.smithi192.stdout:9/563: truncate d0/d9/d44/d51/d5b/f69 2031245 0 2024-04-23T02:14:04.499 INFO:tasks.workunit.client.0.smithi071.stdout:6/556: read d6/d1f/d4e/f54 [740219,66869] 0 2024-04-23T02:14:04.499 INFO:tasks.workunit.client.0.smithi071.stdout:7/625: mkdir d3/d17/d1c/d2a/d73/d83/ddd 0 2024-04-23T02:14:04.499 INFO:tasks.workunit.client.0.smithi071.stdout:1/532: link db/d34/f7d db/df/d3a/fa8 0 2024-04-23T02:14:04.502 INFO:tasks.workunit.client.0.smithi071.stdout:3/549: symlink d7/da5/db2/db7/d68/d3b/d4c/lbb 0 2024-04-23T02:14:04.502 INFO:tasks.workunit.client.1.smithi192.stdout:4/760: dwrite de/d15/d53/d81/de3/f4c [0,4194304] 0 2024-04-23T02:14:04.503 INFO:tasks.workunit.client.0.smithi071.stdout:2/545: rename d0/d2/d26/d14/d1f/c33 to d0/d2/cb1 0 2024-04-23T02:14:04.504 INFO:tasks.workunit.client.1.smithi192.stdout:2/651: dwrite d8/d26/d27/d53/da2/dc1/fbb [0,4194304] 0 2024-04-23T02:14:04.505 INFO:tasks.workunit.client.0.smithi071.stdout:1/533: symlink db/d34/d68/la9 0 2024-04-23T02:14:04.507 INFO:tasks.workunit.client.0.smithi071.stdout:1/534: chown db/df/d13/d2b 2800362 1 2024-04-23T02:14:04.507 INFO:tasks.workunit.client.0.smithi071.stdout:7/626: dwrite d3/d17/d41/d34/d39/d3b/fb1 [0,4194304] 0 2024-04-23T02:14:04.507 INFO:tasks.workunit.client.1.smithi192.stdout:2/652: fdatasync d8/d14/f15 0 2024-04-23T02:14:04.510 INFO:tasks.workunit.client.1.smithi192.stdout:9/564: dwrite d0/d9/d44/d51/d5b/d61/f6a [0,4194304] 0 2024-04-23T02:14:04.511 INFO:tasks.workunit.client.0.smithi071.stdout:6/557: getdents d6/d17/d77/d9f/dad/d3f 0 2024-04-23T02:14:04.511 INFO:tasks.workunit.client.0.smithi071.stdout:2/546: read d0/d2/f9 [4035531,99761] 0 2024-04-23T02:14:04.515 INFO:tasks.workunit.client.1.smithi192.stdout:8/835: creat d1/d12/d23/f120 x:0 0 0 2024-04-23T02:14:04.515 INFO:tasks.workunit.client.1.smithi192.stdout:6/717: symlink d3/d15/d1b/lf1 0 2024-04-23T02:14:04.516 INFO:tasks.workunit.client.1.smithi192.stdout:3/597: link d1/d3/d6/d7/d29/d3b/l77 d1/d3/d6/d7/d29/d3b/da9/d7b/ld3 0 2024-04-23T02:14:04.520 INFO:tasks.workunit.client.1.smithi192.stdout:0/768: creat d6/df/f101 x:0 0 0 2024-04-23T02:14:04.521 INFO:tasks.workunit.client.1.smithi192.stdout:6/718: dread d3/d15/d1b/d21/d52/f92 [0,4194304] 0 2024-04-23T02:14:04.522 INFO:tasks.workunit.client.0.smithi071.stdout:1/535: read db/f22 [83299,98705] 0 2024-04-23T02:14:04.523 INFO:tasks.workunit.client.1.smithi192.stdout:8/836: rename d1/d9/d3e/d59/cfc to d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/c121 0 2024-04-23T02:14:04.524 INFO:tasks.workunit.client.0.smithi071.stdout:7/627: chown d3/d17/d41/d34/d39/d3b/da7/ld1 2041 1 2024-04-23T02:14:04.526 INFO:tasks.workunit.client.1.smithi192.stdout:9/565: mknod d0/d9/db3/cc0 0 2024-04-23T02:14:04.526 INFO:tasks.workunit.client.1.smithi192.stdout:0/769: readlink d6/lb 0 2024-04-23T02:14:04.527 INFO:tasks.workunit.client.1.smithi192.stdout:9/566: dread - d0/d9/db3/d5f/f7e zero size 2024-04-23T02:14:04.527 INFO:tasks.workunit.client.0.smithi071.stdout:8/607: dread dc/d15/f86 [0,4194304] 0 2024-04-23T02:14:04.528 INFO:tasks.workunit.client.1.smithi192.stdout:5/677: getdents d2/db/d16/d1b/d3d 0 2024-04-23T02:14:04.528 INFO:tasks.workunit.client.0.smithi071.stdout:6/558: getdents d6/d1f/d46 0 2024-04-23T02:14:04.529 INFO:tasks.workunit.client.1.smithi192.stdout:3/598: mknod d1/d3/d6/d7/cd4 0 2024-04-23T02:14:04.530 INFO:tasks.workunit.client.1.smithi192.stdout:8/837: symlink d1/d12/d23/d4b/d73/d7e/d88/da9/db1/l122 0 2024-04-23T02:14:04.530 INFO:tasks.workunit.client.1.smithi192.stdout:2/653: link d8/d26/l8e d8/d26/d29/dcd/ld7 0 2024-04-23T02:14:04.531 INFO:tasks.workunit.client.0.smithi071.stdout:8/608: chown dc/d15/da0/d8d/f92 550 1 2024-04-23T02:14:04.531 INFO:tasks.workunit.client.0.smithi071.stdout:7/628: dread d3/d17/d41/f2f [4194304,4194304] 0 2024-04-23T02:14:04.532 INFO:tasks.workunit.client.0.smithi071.stdout:2/547: dwrite d0/d2/fb [0,4194304] 0 2024-04-23T02:14:04.532 INFO:tasks.workunit.client.1.smithi192.stdout:9/567: symlink d0/d9/d44/d74/lc1 0 2024-04-23T02:14:04.533 INFO:tasks.workunit.client.0.smithi071.stdout:6/559: creat d6/d17/d77/d9f/dad/d3f/fb0 x:0 0 0 2024-04-23T02:14:04.534 INFO:tasks.workunit.client.0.smithi071.stdout:7/629: fsync d3/d17/d41/d34/d59/d5f/f90 0 2024-04-23T02:14:04.535 INFO:tasks.workunit.client.1.smithi192.stdout:5/678: write d2/db/d2b/f93 [84666,10378] 0 2024-04-23T02:14:04.535 INFO:tasks.workunit.client.1.smithi192.stdout:8/838: symlink d1/d12/d68/l123 0 2024-04-23T02:14:04.537 INFO:tasks.workunit.client.0.smithi071.stdout:2/548: symlink d0/d2/d34/d55/lb2 0 2024-04-23T02:14:04.538 INFO:tasks.workunit.client.0.smithi071.stdout:8/609: mknod dc/cc0 0 2024-04-23T02:14:04.538 INFO:tasks.workunit.client.1.smithi192.stdout:9/568: stat d0/f4e 0 2024-04-23T02:14:04.541 INFO:tasks.workunit.client.0.smithi071.stdout:2/549: rmdir d0/d2/d45/d7d/d56 39 2024-04-23T02:14:04.544 INFO:tasks.workunit.client.1.smithi192.stdout:3/599: rename d1/d3/d17/d1a/d7c/dab/d6c/d84/d5d/fa0 to d1/d3/d17/d2b/fd5 0 2024-04-23T02:14:04.544 INFO:tasks.workunit.client.1.smithi192.stdout:2/654: dwrite d8/d26/d81/fb8 [0,4194304] 0 2024-04-23T02:14:04.544 INFO:tasks.workunit.client.0.smithi071.stdout:7/630: dwrite d3/d17/d1c/d31/fcf [0,4194304] 0 2024-04-23T02:14:04.546 INFO:tasks.workunit.client.1.smithi192.stdout:8/839: fdatasync d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/fc3 0 2024-04-23T02:14:04.547 INFO:tasks.workunit.client.0.smithi071.stdout:8/610: rename dc/d15/da0/d8d/f9e to dc/d15/d50/d5f/fc1 0 2024-04-23T02:14:04.548 INFO:tasks.workunit.client.1.smithi192.stdout:2/655: chown d8/d26/d27/d57/da0/cd3 27016763 1 2024-04-23T02:14:04.549 INFO:tasks.workunit.client.0.smithi071.stdout:8/611: write dc/d15/da0/f3d [5023614,127102] 0 2024-04-23T02:14:04.550 INFO:tasks.workunit.client.1.smithi192.stdout:9/569: symlink d0/d9/d44/d51/d5b/d61/lc2 0 2024-04-23T02:14:04.552 INFO:tasks.workunit.client.0.smithi071.stdout:7/631: chown d3/d17/d41/d37/lb2 1727596 1 2024-04-23T02:14:04.555 INFO:tasks.workunit.client.0.smithi071.stdout:7/632: symlink d3/d51/lde 0 2024-04-23T02:14:04.559 INFO:tasks.workunit.client.0.smithi071.stdout:7/633: symlink d3/dae/db7/ldf 0 2024-04-23T02:14:04.560 INFO:tasks.workunit.client.0.smithi071.stdout:7/634: chown d3/d17/d1c/f26 2 1 2024-04-23T02:14:04.562 INFO:tasks.workunit.client.0.smithi071.stdout:6/560: dwrite d6/d17/d77/d85/f88 [0,4194304] 0 2024-04-23T02:14:04.568 INFO:tasks.workunit.client.0.smithi071.stdout:7/635: dwrite d3/d17/d1c/d2a/d43/d86/fd7 [0,4194304] 0 2024-04-23T02:14:04.569 INFO:tasks.workunit.client.0.smithi071.stdout:7/636: write d3/fad [306656,24084] 0 2024-04-23T02:14:04.572 INFO:tasks.workunit.client.0.smithi071.stdout:7/637: fdatasync d3/fd 0 2024-04-23T02:14:04.575 INFO:tasks.workunit.client.0.smithi071.stdout:7/638: rmdir d3/d17/d1c/d2a/d73 39 2024-04-23T02:14:04.581 INFO:tasks.workunit.client.1.smithi192.stdout:3/600: mkdir d1/d3/d17/d1f/d33/dd6 0 2024-04-23T02:14:04.582 INFO:tasks.workunit.client.0.smithi071.stdout:7/639: dwrite d3/d51/d87/fab [0,4194304] 0 2024-04-23T02:14:04.584 INFO:tasks.workunit.client.1.smithi192.stdout:3/601: fsync d1/d3/d17/d1f/fce 0 2024-04-23T02:14:04.586 INFO:tasks.workunit.client.1.smithi192.stdout:0/770: dread d6/df/d13/d25/d26/f27 [4194304,4194304] 0 2024-04-23T02:14:04.587 INFO:tasks.workunit.client.1.smithi192.stdout:9/570: fdatasync d0/d9/d44/d51/d5b/d96/da0/fbe 0 2024-04-23T02:14:04.588 INFO:tasks.workunit.client.1.smithi192.stdout:0/771: fsync d6/df/d13/d25/d26/f82 0 2024-04-23T02:14:04.588 INFO:tasks.workunit.client.1.smithi192.stdout:2/656: creat d8/d26/d27/db5/dd0/fd8 x:0 0 0 2024-04-23T02:14:04.590 INFO:tasks.workunit.client.1.smithi192.stdout:3/602: mkdir d1/d3/d6/d7/d29/d3b/da9/d7b/dd7 0 2024-04-23T02:14:04.590 INFO:tasks.workunit.client.0.smithi071.stdout:6/561: dread d6/d1f/f83 [0,4194304] 0 2024-04-23T02:14:04.590 INFO:tasks.workunit.client.0.smithi071.stdout:7/640: rmdir d3/d17/d1c/d2a/d43 39 2024-04-23T02:14:04.591 INFO:tasks.workunit.client.1.smithi192.stdout:9/571: write d0/d9/d44/d51/d64/fb6 [676207,39220] 0 2024-04-23T02:14:04.592 INFO:tasks.workunit.client.1.smithi192.stdout:6/719: truncate d3/d18/f9f 1542853 0 2024-04-23T02:14:04.592 INFO:tasks.workunit.client.0.smithi071.stdout:6/562: rmdir d6/d1f/d5d 39 2024-04-23T02:14:04.593 INFO:tasks.workunit.client.1.smithi192.stdout:9/572: read d0/d9/f50 [4947523,45196] 0 2024-04-23T02:14:04.594 INFO:tasks.workunit.client.0.smithi071.stdout:7/641: chown d3/d17/d1c/d2a/d43 462839 1 2024-04-23T02:14:04.594 INFO:tasks.workunit.client.1.smithi192.stdout:8/840: rmdir d1/d12/d23/d2c/d11c 0 2024-04-23T02:14:04.595 INFO:tasks.workunit.client.0.smithi071.stdout:7/642: mkdir d3/d17/d41/d3a/da5/de0 0 2024-04-23T02:14:04.595 INFO:tasks.workunit.client.1.smithi192.stdout:8/841: write d1/d12/d7d/df7/f106 [579475,84371] 0 2024-04-23T02:14:04.597 INFO:tasks.workunit.client.0.smithi071.stdout:6/563: rename d6/d17/c40 to d6/d1f/cb1 0 2024-04-23T02:14:04.599 INFO:tasks.workunit.client.1.smithi192.stdout:3/603: dread d1/d3/d17/d2b/f32 [0,4194304] 0 2024-04-23T02:14:04.600 INFO:tasks.workunit.client.1.smithi192.stdout:6/720: creat d3/d18/dd4/d8d/ff2 x:0 0 0 2024-04-23T02:14:04.600 INFO:tasks.workunit.client.1.smithi192.stdout:2/657: mknod d8/cd9 0 2024-04-23T02:14:04.601 INFO:tasks.workunit.client.0.smithi071.stdout:6/564: rename d6/d17/d77/d85/f88 to d6/d1f/d5d/d95/fb2 0 2024-04-23T02:14:04.602 INFO:tasks.workunit.client.1.smithi192.stdout:0/772: mknod d6/d4b/c102 0 2024-04-23T02:14:04.603 INFO:tasks.workunit.client.1.smithi192.stdout:6/721: write d3/d15/d1b/d21/d7c/dda/fef [645673,23548] 0 2024-04-23T02:14:04.604 INFO:tasks.workunit.client.0.smithi071.stdout:6/565: mknod d6/d17/d77/d9f/dad/d8f/cb3 0 2024-04-23T02:14:04.605 INFO:tasks.workunit.client.1.smithi192.stdout:6/722: write d3/f5c [228918,46629] 0 2024-04-23T02:14:04.607 INFO:tasks.workunit.client.1.smithi192.stdout:8/842: creat d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/d108/d11f/f124 x:0 0 0 2024-04-23T02:14:04.608 INFO:tasks.workunit.client.0.smithi071.stdout:1/536: write db/df/d13/d72/f79 [262588,95670] 0 2024-04-23T02:14:04.608 INFO:tasks.workunit.client.0.smithi071.stdout:1/537: stat db/df/f2a 0 2024-04-23T02:14:04.609 INFO:tasks.workunit.client.1.smithi192.stdout:3/604: symlink d1/d3/d17/d2b/d9a/ld8 0 2024-04-23T02:14:04.609 INFO:tasks.workunit.client.0.smithi071.stdout:7/643: truncate d3/d17/d1c/d31/fc2 393320 0 2024-04-23T02:14:04.612 INFO:tasks.workunit.client.1.smithi192.stdout:2/658: mkdir d8/d26/d29/d43/d5d/dbd/dda 0 2024-04-23T02:14:04.613 INFO:tasks.workunit.client.0.smithi071.stdout:7/644: truncate d3/d17/d41/d34/f89 809618 0 2024-04-23T02:14:04.614 INFO:tasks.workunit.client.1.smithi192.stdout:8/843: symlink d1/d9/d8f/d90/l125 0 2024-04-23T02:14:04.614 INFO:tasks.workunit.client.0.smithi071.stdout:7/645: write d3/fc [938405,49912] 0 2024-04-23T02:14:04.615 INFO:tasks.workunit.client.0.smithi071.stdout:1/538: rename db/df/d13/d2b/l62 to db/df/d13/d72/d86/laa 0 2024-04-23T02:14:04.616 INFO:tasks.workunit.client.1.smithi192.stdout:8/844: chown d1/d12/d23/d4b/d73/d7e/l8e 638440 1 2024-04-23T02:14:04.616 INFO:tasks.workunit.client.1.smithi192.stdout:3/605: symlink d1/d3/d17/d1a/d7c/ld9 0 2024-04-23T02:14:04.616 INFO:tasks.workunit.client.0.smithi071.stdout:7/646: fsync d3/d17/f36 0 2024-04-23T02:14:04.618 INFO:tasks.workunit.client.0.smithi071.stdout:1/539: creat db/df/d13/d2b/fab x:0 0 0 2024-04-23T02:14:04.619 INFO:tasks.workunit.client.0.smithi071.stdout:7/647: chown d3/d17/d41/d3a/f58 83258 1 2024-04-23T02:14:04.620 INFO:tasks.workunit.client.1.smithi192.stdout:6/723: link d3/d28/l9b d3/d18/d91/lf3 0 2024-04-23T02:14:04.621 INFO:tasks.workunit.client.0.smithi071.stdout:6/566: dread d6/d91/f9c [0,4194304] 0 2024-04-23T02:14:04.621 INFO:tasks.workunit.client.0.smithi071.stdout:1/540: mknod db/df/d13/d2f/d48/d9f/cac 0 2024-04-23T02:14:04.622 INFO:tasks.workunit.client.0.smithi071.stdout:1/541: chown db/df/d13/d2b 22387 1 2024-04-23T02:14:04.622 INFO:tasks.workunit.client.1.smithi192.stdout:8/845: rename d1/d12/d7d/f9a to d1/d12/d23/f126 0 2024-04-23T02:14:04.622 INFO:tasks.workunit.client.1.smithi192.stdout:6/724: chown d3/d28/f96 505208134 1 2024-04-23T02:14:04.623 INFO:tasks.workunit.client.1.smithi192.stdout:8/846: readlink d1/d9/l3a 0 2024-04-23T02:14:04.626 INFO:tasks.workunit.client.1.smithi192.stdout:4/761: dwrite de/d15/d53/d81/de3/d74/d92/da0/dca/f5f [0,4194304] 0 2024-04-23T02:14:04.627 INFO:tasks.workunit.client.0.smithi071.stdout:6/567: rename d6/d22/l39 to d6/d22/d92/lb4 0 2024-04-23T02:14:04.628 INFO:tasks.workunit.client.0.smithi071.stdout:1/542: symlink db/df/d13/d2f/d48/d85/lad 0 2024-04-23T02:14:04.630 INFO:tasks.workunit.client.1.smithi192.stdout:3/606: truncate d1/f56 717916 0 2024-04-23T02:14:04.630 INFO:tasks.workunit.client.0.smithi071.stdout:6/568: mkdir d6/d17/d77/d9f/db5 0 2024-04-23T02:14:04.631 INFO:tasks.workunit.client.1.smithi192.stdout:2/659: dread d8/d26/d27/d31/f8b [0,4194304] 0 2024-04-23T02:14:04.631 INFO:tasks.workunit.client.0.smithi071.stdout:1/543: mkdir db/d11/d7f/dae 0 2024-04-23T02:14:04.632 INFO:tasks.workunit.client.1.smithi192.stdout:6/725: getdents d3/d15/d1b/d66/d80/de2 0 2024-04-23T02:14:04.633 INFO:tasks.workunit.client.0.smithi071.stdout:7/648: truncate d3/d17/d1c/f26 1257039 0 2024-04-23T02:14:04.634 INFO:tasks.workunit.client.1.smithi192.stdout:5/679: truncate d2/db/d16/d1b/f1f 4183981 0 2024-04-23T02:14:04.635 INFO:tasks.workunit.client.0.smithi071.stdout:4/643: sync 2024-04-23T02:14:04.635 INFO:tasks.workunit.client.1.smithi192.stdout:8/847: truncate d1/df/f50 969378 0 2024-04-23T02:14:04.635 INFO:tasks.workunit.client.0.smithi071.stdout:1/544: unlink db/df/d13/d2b/c37 0 2024-04-23T02:14:04.644 INFO:tasks.workunit.client.1.smithi192.stdout:6/726: unlink d3/d28/d79/fa4 0 2024-04-23T02:14:04.645 INFO:tasks.workunit.client.1.smithi192.stdout:4/762: dread - de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/d89/fdc zero size 2024-04-23T02:14:04.645 INFO:tasks.workunit.client.1.smithi192.stdout:5/680: mkdir d2/d2e/d4e/de6 0 2024-04-23T02:14:04.645 INFO:tasks.workunit.client.1.smithi192.stdout:2/660: symlink d8/d26/d27/ldb 0 2024-04-23T02:14:04.645 INFO:tasks.workunit.client.0.smithi071.stdout:1/545: rmdir db/df/d3a 39 2024-04-23T02:14:04.646 INFO:tasks.workunit.client.0.smithi071.stdout:6/569: dwrite d6/d17/d77/d85/f89 [0,4194304] 0 2024-04-23T02:14:04.646 INFO:tasks.workunit.client.0.smithi071.stdout:1/546: creat db/d11/d7f/faf x:0 0 0 2024-04-23T02:14:04.646 INFO:tasks.workunit.client.0.smithi071.stdout:4/644: creat d2/d2a/d2d/d39/d42/fcd x:0 0 0 2024-04-23T02:14:04.646 INFO:tasks.workunit.client.0.smithi071.stdout:1/547: read - db/df/d13/d2b/f7e zero size 2024-04-23T02:14:04.647 INFO:tasks.workunit.client.1.smithi192.stdout:2/661: write d8/d26/d27/d53/da2/fc0 [468124,63622] 0 2024-04-23T02:14:04.647 INFO:tasks.workunit.client.0.smithi071.stdout:4/645: read d2/d2a/d2d/d39/d42/f55 [498947,125282] 0 2024-04-23T02:14:04.648 INFO:tasks.workunit.client.1.smithi192.stdout:6/727: symlink d3/d18/lf4 0 2024-04-23T02:14:04.649 INFO:tasks.workunit.client.0.smithi071.stdout:1/548: write db/df/d3a/f83 [1902840,24854] 0 2024-04-23T02:14:04.649 INFO:tasks.workunit.client.1.smithi192.stdout:2/662: fdatasync d8/d26/d29/d43/fd6 0 2024-04-23T02:14:04.650 INFO:tasks.workunit.client.1.smithi192.stdout:5/681: rmdir d2/d6 39 2024-04-23T02:14:04.651 INFO:tasks.workunit.client.0.smithi071.stdout:4/646: mknod d2/d2a/d2d/d39/d42/cce 0 2024-04-23T02:14:04.651 INFO:tasks.workunit.client.1.smithi192.stdout:2/663: read d8/d26/d29/f41 [2104120,20061] 0 2024-04-23T02:14:04.656 INFO:tasks.workunit.client.0.smithi071.stdout:7/649: write d3/d17/d41/d34/d59/d5f/f6c [3086135,30443] 0 2024-04-23T02:14:04.656 INFO:tasks.workunit.client.1.smithi192.stdout:5/682: fsync d2/db/d16/d1b/d3d/f65 0 2024-04-23T02:14:04.658 INFO:tasks.workunit.client.0.smithi071.stdout:1/549: dwrite db/df/d13/f1b [0,4194304] 0 2024-04-23T02:14:04.659 INFO:tasks.workunit.client.0.smithi071.stdout:4/647: mknod d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d4f/db5/db6/ccf 0 2024-04-23T02:14:04.662 INFO:tasks.workunit.client.0.smithi071.stdout:6/570: truncate d6/d17/f2c 1101069 0 2024-04-23T02:14:04.663 INFO:tasks.workunit.client.0.smithi071.stdout:7/650: write d3/d17/d1c/d2a/d73/fc0 [270862,66770] 0 2024-04-23T02:14:04.668 INFO:tasks.workunit.client.0.smithi071.stdout:7/651: symlink d3/d17/d1c/d2a/d73/d83/ddd/le1 0 2024-04-23T02:14:04.668 INFO:tasks.workunit.client.0.smithi071.stdout:6/571: truncate d6/d1f/f3b 3831456 0 2024-04-23T02:14:04.671 INFO:tasks.workunit.client.0.smithi071.stdout:7/652: truncate d3/fd 2476831 0 2024-04-23T02:14:04.671 INFO:tasks.workunit.client.1.smithi192.stdout:4/763: dwrite de/d15/d53/d81/de3/da3/ff3 [0,4194304] 0 2024-04-23T02:14:04.671 INFO:tasks.workunit.client.1.smithi192.stdout:8/848: dwrite d1/d9/d3e/d59/d8a/f115 [0,4194304] 0 2024-04-23T02:14:04.672 INFO:tasks.workunit.client.0.smithi071.stdout:6/572: creat d6/fb6 x:0 0 0 2024-04-23T02:14:04.673 INFO:tasks.workunit.client.0.smithi071.stdout:4/648: dread d2/d2a/d2d/d39/d42/d49/d91/fc5 [0,4194304] 0 2024-04-23T02:14:04.675 INFO:tasks.workunit.client.1.smithi192.stdout:5/683: creat d2/db/d46/fe7 x:0 0 0 2024-04-23T02:14:04.675 INFO:tasks.workunit.client.0.smithi071.stdout:7/653: mkdir d3/d17/d41/de2 0 2024-04-23T02:14:04.676 INFO:tasks.workunit.client.0.smithi071.stdout:4/649: chown d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c/f7c 6 1 2024-04-23T02:14:04.677 INFO:tasks.workunit.client.1.smithi192.stdout:0/773: dwrite d6/df/d13/d25/d3a/d7b/f88 [0,4194304] 0 2024-04-23T02:14:04.678 INFO:tasks.workunit.client.0.smithi071.stdout:6/573: symlink d6/lb7 0 2024-04-23T02:14:04.679 INFO:tasks.workunit.client.0.smithi071.stdout:7/654: write d3/d17/d1c/f20 [4881284,95233] 0 2024-04-23T02:14:04.680 INFO:tasks.workunit.client.1.smithi192.stdout:3/607: dread d1/d3/d17/d1a/d7c/dab/d6c/f90 [0,4194304] 0 2024-04-23T02:14:04.682 INFO:tasks.workunit.client.0.smithi071.stdout:4/650: mkdir d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d4f/db5/db6/dd0 0 2024-04-23T02:14:04.684 INFO:tasks.workunit.client.1.smithi192.stdout:2/664: dwrite d8/d26/d29/f58 [4194304,4194304] 0 2024-04-23T02:14:04.684 INFO:tasks.workunit.client.1.smithi192.stdout:8/849: mknod d1/d9/d8f/d90/c127 0 2024-04-23T02:14:04.684 INFO:tasks.workunit.client.0.smithi071.stdout:7/655: creat d3/d17/d1c/d2a/d73/dba/fe3 x:0 0 0 2024-04-23T02:14:04.685 INFO:tasks.workunit.client.0.smithi071.stdout:6/574: creat d6/d1f/d5d/d95/fb8 x:0 0 0 2024-04-23T02:14:04.685 INFO:tasks.workunit.client.0.smithi071.stdout:4/651: dread - d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/fc9 zero size 2024-04-23T02:14:04.688 INFO:tasks.workunit.client.0.smithi071.stdout:4/652: chown d2/d2a/d2d/d39/d42/d49/d67/d7e/dae/dab 26 1 2024-04-23T02:14:04.691 INFO:tasks.workunit.client.0.smithi071.stdout:4/653: mkdir d2/d2a/da2/dd1 0 2024-04-23T02:14:04.691 INFO:tasks.workunit.client.0.smithi071.stdout:7/656: truncate d3/d17/d1c/f29 2987649 0 2024-04-23T02:14:04.692 INFO:tasks.workunit.client.0.smithi071.stdout:7/657: truncate d3/fb5 1036824 0 2024-04-23T02:14:04.693 INFO:tasks.workunit.client.0.smithi071.stdout:4/654: creat d2/d2a/d2d/d39/d42/d49/d91/fd2 x:0 0 0 2024-04-23T02:14:04.694 INFO:tasks.workunit.client.0.smithi071.stdout:7/658: read d3/f69 [267072,52413] 0 2024-04-23T02:14:04.694 INFO:tasks.workunit.client.1.smithi192.stdout:3/608: write d1/d3/d6/f81 [225635,52399] 0 2024-04-23T02:14:04.695 INFO:tasks.workunit.client.1.smithi192.stdout:6/728: dwrite d3/d11/f29 [0,4194304] 0 2024-04-23T02:14:04.695 INFO:tasks.workunit.client.0.smithi071.stdout:7/659: stat d3/d51 0 2024-04-23T02:14:04.696 INFO:tasks.workunit.client.0.smithi071.stdout:4/655: fsync d2/d2a/f3b 0 2024-04-23T02:14:04.697 INFO:tasks.workunit.client.0.smithi071.stdout:4/656: chown d2/d2a/d2d/d39/d42/d49/d67/d1d/d23 1163535 1 2024-04-23T02:14:04.698 INFO:tasks.workunit.client.1.smithi192.stdout:2/665: mknod d8/d26/d29/d80/cdc 0 2024-04-23T02:14:04.698 INFO:tasks.workunit.client.0.smithi071.stdout:4/657: fsync d2/d2a/d2d/d39/d42/d49/d91/fcb 0 2024-04-23T02:14:04.706 INFO:tasks.workunit.client.1.smithi192.stdout:8/850: symlink d1/l128 0 2024-04-23T02:14:04.708 INFO:tasks.workunit.client.1.smithi192.stdout:2/666: mknod d8/d26/d29/d43/d5d/dbd/cdd 0 2024-04-23T02:14:04.710 INFO:tasks.workunit.client.1.smithi192.stdout:3/609: getdents d1/d3/d6/d7/d29/d3b/da9/d7b/dd7 0 2024-04-23T02:14:04.712 INFO:tasks.workunit.client.1.smithi192.stdout:5/684: link d2/d6/d14/d60/cd0 d2/db/d2b/dc8/ce8 0 2024-04-23T02:14:04.714 INFO:tasks.workunit.client.1.smithi192.stdout:5/685: truncate d2/db/d16/fce 654800 0 2024-04-23T02:14:04.716 INFO:tasks.workunit.client.1.smithi192.stdout:4/764: getdents de/d15/d53/d81/de3/d74/d92/da0/dca/d34/d4a 0 2024-04-23T02:14:04.720 INFO:tasks.workunit.client.1.smithi192.stdout:4/765: rmdir de/d15/d19/d1a/d95 39 2024-04-23T02:14:04.720 INFO:tasks.workunit.client.1.smithi192.stdout:5/686: dread d2/db/d16/d48/d4f/f84 [0,4194304] 0 2024-04-23T02:14:04.720 INFO:tasks.workunit.client.1.smithi192.stdout:2/667: getdents d8/d26/d29/da4/dca 0 2024-04-23T02:14:04.721 INFO:tasks.workunit.client.1.smithi192.stdout:6/729: dread d3/d28/d79/dbe/dcb/fd5 [0,4194304] 0 2024-04-23T02:14:04.723 INFO:tasks.workunit.client.1.smithi192.stdout:4/766: write de/d15/d53/d81/de3/da3/ff3 [3495695,62411] 0 2024-04-23T02:14:04.726 INFO:tasks.workunit.client.1.smithi192.stdout:5/687: getdents d2/d2e 0 2024-04-23T02:14:04.732 INFO:tasks.workunit.client.1.smithi192.stdout:5/688: symlink d2/db/d16/d48/d4f/dd1/le9 0 2024-04-23T02:14:04.734 INFO:tasks.workunit.client.1.smithi192.stdout:2/668: dwrite d8/f86 [0,4194304] 0 2024-04-23T02:14:04.734 INFO:tasks.workunit.client.1.smithi192.stdout:8/851: write d1/d12/d23/d4b/d73/d45/f80 [1196874,64083] 0 2024-04-23T02:14:04.734 INFO:tasks.workunit.client.1.smithi192.stdout:6/730: getdents d3/d15/d1b/d66 0 2024-04-23T02:14:04.735 INFO:tasks.workunit.client.0.smithi071.stdout:7/660: dwrite d3/d17/d1c/f3f [0,4194304] 0 2024-04-23T02:14:04.737 INFO:tasks.workunit.client.1.smithi192.stdout:4/767: mkdir de/d15/d53/d81/de3/dfa 0 2024-04-23T02:14:04.741 INFO:tasks.workunit.client.0.smithi071.stdout:7/661: dwrite d3/d17/d1c/d2a/d43/d86/fc3 [0,4194304] 0 2024-04-23T02:14:04.744 INFO:tasks.workunit.client.0.smithi071.stdout:7/662: stat d3/d51/d87/db3/l9a 0 2024-04-23T02:14:04.746 INFO:tasks.workunit.client.1.smithi192.stdout:4/768: write de/d15/d53/d81/de3/f5c [5043781,54545] 0 2024-04-23T02:14:04.748 INFO:tasks.workunit.client.0.smithi071.stdout:7/663: symlink d3/d17/d1c/d2a/le4 0 2024-04-23T02:14:04.749 INFO:tasks.workunit.client.1.smithi192.stdout:4/769: write de/d15/d19/d1a/f72 [2639318,72319] 0 2024-04-23T02:14:04.750 INFO:tasks.workunit.client.0.smithi071.stdout:7/664: mknod d3/d17/dca/ce5 0 2024-04-23T02:14:04.751 INFO:tasks.workunit.client.1.smithi192.stdout:6/731: mkdir d3/d15/d1b/d21/d52/d7f/df5 0 2024-04-23T02:14:04.751 INFO:tasks.workunit.client.1.smithi192.stdout:2/669: link d8/d26/d29/d43/d5d/dbd/cdd d8/d26/d27/d53/da2/cde 0 2024-04-23T02:14:04.753 INFO:tasks.workunit.client.1.smithi192.stdout:8/852: getdents d1/d12/d68 0 2024-04-23T02:14:04.754 INFO:tasks.workunit.client.1.smithi192.stdout:0/774: dwrite d6/d4b/d9d/dc7/f80 [0,4194304] 0 2024-04-23T02:14:04.754 INFO:tasks.workunit.client.1.smithi192.stdout:5/689: getdents d2/d6 0 2024-04-23T02:14:04.754 INFO:tasks.workunit.client.1.smithi192.stdout:3/610: dwrite d1/d3/d17/d1a/d7c/fb5 [0,4194304] 0 2024-04-23T02:14:04.756 INFO:tasks.workunit.client.1.smithi192.stdout:6/732: stat d3/d18/d58/fcd 0 2024-04-23T02:14:04.759 INFO:tasks.workunit.client.1.smithi192.stdout:5/690: stat d2/db/d2b/f93 0 2024-04-23T02:14:04.759 INFO:tasks.workunit.client.1.smithi192.stdout:4/770: mknod de/d15/d53/d81/de3/d80/cfb 0 2024-04-23T02:14:04.762 INFO:tasks.workunit.client.1.smithi192.stdout:0/775: creat d6/df/d13/d25/d86/da5/f103 x:0 0 0 2024-04-23T02:14:04.762 INFO:tasks.workunit.client.1.smithi192.stdout:8/853: write d1/d12/d23/d4b/d73/d45/d46/fd6 [811704,17646] 0 2024-04-23T02:14:04.763 INFO:tasks.workunit.client.1.smithi192.stdout:3/611: write d1/d3/f7a [3615116,93035] 0 2024-04-23T02:14:04.765 INFO:tasks.workunit.client.1.smithi192.stdout:6/733: creat d3/d18/d91/ff6 x:0 0 0 2024-04-23T02:14:04.766 INFO:tasks.workunit.client.1.smithi192.stdout:2/670: creat d8/d26/d29/d43/fdf x:0 0 0 2024-04-23T02:14:04.766 INFO:tasks.workunit.client.1.smithi192.stdout:8/854: fsync d1/d12/d23/f120 0 2024-04-23T02:14:04.767 INFO:tasks.workunit.client.1.smithi192.stdout:4/771: read de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/f91 [1603494,108952] 0 2024-04-23T02:14:04.769 INFO:tasks.workunit.client.1.smithi192.stdout:0/776: rename d6/df/d13/d25/d26/d67 to d6/df/d13/d25/d26/d67/d104 22 2024-04-23T02:14:04.770 INFO:tasks.workunit.client.1.smithi192.stdout:2/671: creat d8/d26/d29/d43/d5d/fe0 x:0 0 0 2024-04-23T02:14:04.771 INFO:tasks.workunit.client.1.smithi192.stdout:0/777: stat d6/df/f101 0 2024-04-23T02:14:04.773 INFO:tasks.workunit.client.1.smithi192.stdout:2/672: chown d8/d26/d29/da4/da8/fc9 1795685 1 2024-04-23T02:14:04.773 INFO:tasks.workunit.client.1.smithi192.stdout:0/778: chown d6/d4b/d9d/dc7/lba 513 1 2024-04-23T02:14:04.774 INFO:tasks.workunit.client.1.smithi192.stdout:8/855: stat d1/d12/d23/d2c/d8c/d95/c97 0 2024-04-23T02:14:04.776 INFO:tasks.workunit.client.1.smithi192.stdout:1/440: sync 2024-04-23T02:14:04.777 INFO:tasks.workunit.client.1.smithi192.stdout:8/856: fdatasync d1/f15 0 2024-04-23T02:14:04.778 INFO:tasks.workunit.client.1.smithi192.stdout:2/673: read d8/d26/d29/d43/f7f [739556,6012] 0 2024-04-23T02:14:04.778 INFO:tasks.workunit.client.1.smithi192.stdout:3/612: dwrite d1/d3/d17/d1a/d7c/dab/f54 [0,4194304] 0 2024-04-23T02:14:04.781 INFO:tasks.workunit.client.1.smithi192.stdout:8/857: stat d1/de1/f10d 0 2024-04-23T02:14:04.782 INFO:tasks.workunit.client.1.smithi192.stdout:1/441: write dc/d16/d19/d30/d32/f4d [775282,90134] 0 2024-04-23T02:14:04.787 INFO:tasks.workunit.client.0.smithi071.stdout:9/567: sync 2024-04-23T02:14:04.787 INFO:tasks.workunit.client.0.smithi071.stdout:0/558: sync 2024-04-23T02:14:04.788 INFO:tasks.workunit.client.1.smithi192.stdout:2/674: rename d8/d26/d27/ldb to d8/d26/d29/da4/le1 0 2024-04-23T02:14:04.789 INFO:tasks.workunit.client.1.smithi192.stdout:1/442: mknod dc/d14/d15/d35/c86 0 2024-04-23T02:14:04.789 INFO:tasks.workunit.client.1.smithi192.stdout:4/772: dwrite de/d15/d53/d81/de3/d74/d92/da0/dca/d34/d4a/fc4 [0,4194304] 0 2024-04-23T02:14:04.796 INFO:tasks.workunit.client.1.smithi192.stdout:2/675: rename d8/d26/l7b to d8/d26/d74/le2 0 2024-04-23T02:14:04.799 INFO:tasks.workunit.client.1.smithi192.stdout:3/613: dwrite d1/d3/d6/f81 [0,4194304] 0 2024-04-23T02:14:04.803 INFO:tasks.workunit.client.1.smithi192.stdout:8/858: dwrite d1/d12/d68/fad [0,4194304] 0 2024-04-23T02:14:04.806 INFO:tasks.workunit.client.1.smithi192.stdout:8/859: stat d1/f4a 0 2024-04-23T02:14:04.809 INFO:tasks.workunit.client.1.smithi192.stdout:3/614: stat d1/d3/d17/d1a/d7c/dab/d6c/d84/d4c 0 2024-04-23T02:14:04.810 INFO:tasks.workunit.client.1.smithi192.stdout:0/779: dwrite d6/df/d13/d25/d26/fac [0,4194304] 0 2024-04-23T02:14:04.816 INFO:tasks.workunit.client.1.smithi192.stdout:3/615: mkdir d1/d3/d17/dd1/dda 0 2024-04-23T02:14:04.821 INFO:tasks.workunit.client.1.smithi192.stdout:4/773: dwrite de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d93/fef [0,4194304] 0 2024-04-23T02:14:04.825 INFO:tasks.workunit.client.1.smithi192.stdout:3/616: creat d1/d3/d17/d2b/d44/d57/db0/fdb x:0 0 0 2024-04-23T02:14:04.830 INFO:tasks.workunit.client.1.smithi192.stdout:0/780: dwrite d6/d4b/f79 [4194304,4194304] 0 2024-04-23T02:14:04.831 INFO:tasks.workunit.client.1.smithi192.stdout:3/617: write d1/d3/d17/d1f/d33/fcf [313014,50683] 0 2024-04-23T02:14:04.831 INFO:tasks.workunit.client.1.smithi192.stdout:4/774: rename de/d15/d53/d81/de3/d74/d92 to de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dfc 22 2024-04-23T02:14:04.831 INFO:tasks.workunit.client.1.smithi192.stdout:2/676: dread d8/d26/fbe [0,4194304] 0 2024-04-23T02:14:04.833 INFO:tasks.workunit.client.1.smithi192.stdout:2/677: chown d8/d26/fbe 1634032 1 2024-04-23T02:14:04.836 INFO:tasks.workunit.client.1.smithi192.stdout:0/781: symlink d6/df/d47/d55/l105 0 2024-04-23T02:14:04.838 INFO:tasks.workunit.client.1.smithi192.stdout:4/775: mknod de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/cfd 0 2024-04-23T02:14:04.842 INFO:tasks.workunit.client.1.smithi192.stdout:2/678: dwrite d8/d26/d27/f70 [4194304,4194304] 0 2024-04-23T02:14:04.843 INFO:tasks.workunit.client.1.smithi192.stdout:0/782: link d6/df/f4a d6/df/d13/d25/d94/dd0/de6/f106 0 2024-04-23T02:14:04.844 INFO:tasks.workunit.client.1.smithi192.stdout:2/679: dread - d8/d26/d27/d57/f93 zero size 2024-04-23T02:14:04.846 INFO:tasks.workunit.client.1.smithi192.stdout:4/776: creat de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/de7/ffe x:0 0 0 2024-04-23T02:14:04.852 INFO:tasks.workunit.client.0.smithi071.stdout:9/568: stat d1/d32/f40 0 2024-04-23T02:14:04.853 INFO:tasks.workunit.client.0.smithi071.stdout:7/665: truncate d3/fd 2260147 0 2024-04-23T02:14:04.853 INFO:tasks.workunit.client.1.smithi192.stdout:4/777: symlink de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d93/lff 0 2024-04-23T02:14:04.856 INFO:tasks.workunit.client.0.smithi071.stdout:9/569: mkdir d1/d4e/d66/da4/dc4 0 2024-04-23T02:14:04.857 INFO:tasks.workunit.client.1.smithi192.stdout:8/860: dread d1/d12/d7d/f89 [0,4194304] 0 2024-04-23T02:14:04.857 INFO:tasks.workunit.client.1.smithi192.stdout:2/680: dwrite d8/d26/d29/d80/fc7 [0,4194304] 0 2024-04-23T02:14:04.857 INFO:tasks.workunit.client.0.smithi071.stdout:7/666: creat d3/d17/d41/d34/d59/fe6 x:0 0 0 2024-04-23T02:14:04.858 INFO:tasks.workunit.client.1.smithi192.stdout:0/783: write d6/df/d56/d85/f2e [663501,125637] 0 2024-04-23T02:14:04.859 INFO:tasks.workunit.client.1.smithi192.stdout:8/861: fsync d1/d12/d23/d4b/d73/d7e/d88/fc6 0 2024-04-23T02:14:04.860 INFO:tasks.workunit.client.0.smithi071.stdout:7/667: dread - d3/d17/d41/d34/d59/f9d zero size 2024-04-23T02:14:04.860 INFO:tasks.workunit.client.1.smithi192.stdout:0/784: chown d6/l24 9443 1 2024-04-23T02:14:04.860 INFO:tasks.workunit.client.0.smithi071.stdout:0/559: dwrite d0/f6b [0,4194304] 0 2024-04-23T02:14:04.860 INFO:tasks.workunit.client.1.smithi192.stdout:2/681: read - d8/d26/d29/d43/d50/f76 zero size 2024-04-23T02:14:04.861 INFO:tasks.workunit.client.1.smithi192.stdout:4/778: truncate de/d15/d53/d81/de3/d74/d92/da0/dca/d34/fed 436561 0 2024-04-23T02:14:04.863 INFO:tasks.workunit.client.1.smithi192.stdout:8/862: write d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/fc3 [1037563,55910] 0 2024-04-23T02:14:04.863 INFO:tasks.workunit.client.1.smithi192.stdout:2/682: unlink d8/d25/f38 0 2024-04-23T02:14:04.865 INFO:tasks.workunit.client.1.smithi192.stdout:8/863: mknod d1/d9/d8f/c129 0 2024-04-23T02:14:04.866 INFO:tasks.workunit.client.1.smithi192.stdout:2/683: rmdir d8/d26/d29/d43/d50 39 2024-04-23T02:14:04.866 INFO:tasks.workunit.client.0.smithi071.stdout:7/668: rmdir d3/d17/d1c/d2a 39 2024-04-23T02:14:04.869 INFO:tasks.workunit.client.0.smithi071.stdout:0/560: dread d0/d3/de/d10/d14/d32/d44/d31/f72 [0,4194304] 0 2024-04-23T02:14:04.874 INFO:tasks.workunit.client.0.smithi071.stdout:0/561: rmdir d0/d3/de/d10/d14/d84 39 2024-04-23T02:14:04.874 INFO:tasks.workunit.client.1.smithi192.stdout:2/684: symlink d8/d26/d29/d43/d50/d5e/le3 0 2024-04-23T02:14:04.875 INFO:tasks.workunit.client.0.smithi071.stdout:7/669: dwrite d3/d17/fb9 [0,4194304] 0 2024-04-23T02:14:04.877 INFO:tasks.workunit.client.1.smithi192.stdout:8/864: write d1/d9/f22 [3459149,61123] 0 2024-04-23T02:14:04.877 INFO:tasks.workunit.client.0.smithi071.stdout:0/562: stat d0/d3/de/d10/d14/d32/d37/d26/c54 0 2024-04-23T02:14:04.877 INFO:tasks.workunit.client.0.smithi071.stdout:9/570: truncate d1/d32/d41/db7/f31 3391332 0 2024-04-23T02:14:04.880 INFO:tasks.workunit.client.0.smithi071.stdout:7/670: read d3/fc [404156,105669] 0 2024-04-23T02:14:04.880 INFO:tasks.workunit.client.1.smithi192.stdout:8/865: symlink d1/l12a 0 2024-04-23T02:14:04.880 INFO:tasks.workunit.client.0.smithi071.stdout:0/563: mknod d0/d3/de/d10/db3/dba/cbd 0 2024-04-23T02:14:04.882 INFO:tasks.workunit.client.1.smithi192.stdout:8/866: dread - d1/d9/d8f/d90/f10a zero size 2024-04-23T02:14:04.883 INFO:tasks.workunit.client.1.smithi192.stdout:2/685: dwrite d8/d26/d29/d43/fd6 [0,4194304] 0 2024-04-23T02:14:04.884 INFO:tasks.workunit.client.0.smithi071.stdout:7/671: write d3/d51/d87/db3/fbc [933525,109960] 0 2024-04-23T02:14:04.886 INFO:tasks.workunit.client.0.smithi071.stdout:9/571: dwrite d1/d32/d3a/d94/fbb [0,4194304] 0 2024-04-23T02:14:04.890 INFO:tasks.workunit.client.0.smithi071.stdout:9/572: truncate d1/d32/d3a/d77/f88 1546275 0 2024-04-23T02:14:04.890 INFO:tasks.workunit.client.0.smithi071.stdout:7/672: mkdir d3/d17/d1c/d2a/d73/d83/de7 0 2024-04-23T02:14:04.891 INFO:tasks.workunit.client.0.smithi071.stdout:7/673: chown d3/d17/d41/d3a/l85 180 1 2024-04-23T02:14:04.893 INFO:tasks.workunit.client.0.smithi071.stdout:9/573: unlink d1/d32/d41/la3 0 2024-04-23T02:14:04.897 INFO:tasks.workunit.client.1.smithi192.stdout:2/686: link d8/d14/l49 d8/d26/d27/d31/le4 0 2024-04-23T02:14:04.897 INFO:tasks.workunit.client.0.smithi071.stdout:9/574: mkdir d1/d32/dc5 0 2024-04-23T02:14:04.897 INFO:tasks.workunit.client.0.smithi071.stdout:7/674: dread d3/d17/d41/f66 [0,4194304] 0 2024-04-23T02:14:04.899 INFO:tasks.workunit.client.0.smithi071.stdout:7/675: write d3/d17/d41/d34/d39/fc8 [677799,47484] 0 2024-04-23T02:14:04.899 INFO:tasks.workunit.client.1.smithi192.stdout:2/687: creat d8/d26/d27/d57/da0/fe5 x:0 0 0 2024-04-23T02:14:04.900 INFO:tasks.workunit.client.0.smithi071.stdout:7/676: readlink d3/d17/d41/d37/lb2 0 2024-04-23T02:14:04.900 INFO:tasks.workunit.client.1.smithi192.stdout:2/688: write d8/d25/f71 [3331537,18003] 0 2024-04-23T02:14:04.903 INFO:tasks.workunit.client.0.smithi071.stdout:9/575: rename d1/d32/d3a/d78 to d1/d32/d41/db7/d5a/dc6 0 2024-04-23T02:14:04.903 INFO:tasks.workunit.client.1.smithi192.stdout:2/689: mkdir d8/d26/d27/d57/d69/d99/de6 0 2024-04-23T02:14:04.905 INFO:tasks.workunit.client.1.smithi192.stdout:2/690: readlink d8/d26/d27/d53/l75 0 2024-04-23T02:14:04.905 INFO:tasks.workunit.client.0.smithi071.stdout:7/677: rename d3/d17/d1c/d31/f98 to d3/d17/d41/d34/d39/fe8 0 2024-04-23T02:14:04.906 INFO:tasks.workunit.client.0.smithi071.stdout:9/576: mknod d1/d32/d41/db7/cc7 0 2024-04-23T02:14:04.907 INFO:tasks.workunit.client.1.smithi192.stdout:2/691: unlink d8/f1d 0 2024-04-23T02:14:04.911 INFO:tasks.workunit.client.1.smithi192.stdout:8/867: write d1/d12/f92 [232047,4453] 0 2024-04-23T02:14:04.912 INFO:tasks.workunit.client.1.smithi192.stdout:2/692: dread d8/d26/d27/f87 [0,4194304] 0 2024-04-23T02:14:04.912 INFO:tasks.workunit.client.1.smithi192.stdout:8/868: truncate d1/d9/ffa 911713 0 2024-04-23T02:14:04.914 INFO:tasks.workunit.client.1.smithi192.stdout:8/869: mknod d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/d108/c12b 0 2024-04-23T02:14:04.915 INFO:tasks.workunit.client.0.smithi071.stdout:0/564: dwrite d0/d3/de/d10/d14/d32/d37/fa1 [0,4194304] 0 2024-04-23T02:14:04.916 INFO:tasks.workunit.client.1.smithi192.stdout:8/870: write d1/d12/d23/d4b/d73/d45/f80 [417766,21623] 0 2024-04-23T02:14:04.918 INFO:tasks.workunit.client.1.smithi192.stdout:8/871: mknod d1/d12/d68/ddf/c12c 0 2024-04-23T02:14:04.918 INFO:tasks.workunit.client.0.smithi071.stdout:0/565: dread - d0/d3/de/d10/d14/d32/d44/d87/d93/fa3 zero size 2024-04-23T02:14:04.919 INFO:tasks.workunit.client.0.smithi071.stdout:9/577: chown d1/d32/d41/db7/d39/d42 970833657 1 2024-04-23T02:14:04.921 INFO:tasks.workunit.client.0.smithi071.stdout:0/566: creat d0/d3/de/d10/d14/d32/d37/d26/fbe x:0 0 0 2024-04-23T02:14:04.922 INFO:tasks.workunit.client.0.smithi071.stdout:9/578: truncate d1/d32/d41/db7/d71/f9e 334582 0 2024-04-23T02:14:04.922 INFO:tasks.workunit.client.0.smithi071.stdout:7/678: dread d3/d17/d41/d34/d59/d5f/f60 [0,4194304] 0 2024-04-23T02:14:04.923 INFO:tasks.workunit.client.1.smithi192.stdout:8/872: mkdir d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/da3/d12d 0 2024-04-23T02:14:04.925 INFO:tasks.workunit.client.1.smithi192.stdout:8/873: read d1/d12/d7d/fce [199485,61684] 0 2024-04-23T02:14:04.926 INFO:tasks.workunit.client.0.smithi071.stdout:9/579: mknod d1/d32/d41/db7/d68/cc8 0 2024-04-23T02:14:04.928 INFO:tasks.workunit.client.1.smithi192.stdout:8/874: creat d1/d12/d23/d4b/d73/d45/f12e x:0 0 0 2024-04-23T02:14:04.930 INFO:tasks.workunit.client.0.smithi071.stdout:7/679: dwrite d3/d17/d1c/d31/f48 [0,4194304] 0 2024-04-23T02:14:04.932 INFO:tasks.workunit.client.0.smithi071.stdout:9/580: rename d1/d32/d41/db7/c56 to d1/d4e/d66/da4/cc9 0 2024-04-23T02:14:04.932 INFO:tasks.workunit.client.1.smithi192.stdout:8/875: creat d1/d9/f12f x:0 0 0 2024-04-23T02:14:04.934 INFO:tasks.workunit.client.1.smithi192.stdout:8/876: chown d1/d12/d23/d2c/d8c/d95/ldb 1278255 1 2024-04-23T02:14:04.936 INFO:tasks.workunit.client.0.smithi071.stdout:7/680: dread d3/d17/fb9 [0,4194304] 0 2024-04-23T02:14:04.937 INFO:tasks.workunit.client.1.smithi192.stdout:8/877: mknod d1/d12/d7d/c130 0 2024-04-23T02:14:04.937 INFO:tasks.workunit.client.0.smithi071.stdout:0/567: dread d0/d3/de/d48/d7e/d8b/f21 [0,4194304] 0 2024-04-23T02:14:04.938 INFO:tasks.workunit.client.0.smithi071.stdout:7/681: truncate d3/d51/f7f 876259 0 2024-04-23T02:14:04.938 INFO:tasks.workunit.client.1.smithi192.stdout:8/878: creat d1/d12/d7d/f131 x:0 0 0 2024-04-23T02:14:04.940 INFO:tasks.workunit.client.0.smithi071.stdout:9/581: write d1/d32/d41/db7/d68/fb5 [146731,26114] 0 2024-04-23T02:14:04.941 INFO:tasks.workunit.client.0.smithi071.stdout:0/568: rename d0/f6b to d0/d63/fbf 0 2024-04-23T02:14:04.942 INFO:tasks.workunit.client.1.smithi192.stdout:8/879: creat d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d8b/f132 x:0 0 0 2024-04-23T02:14:04.942 INFO:tasks.workunit.client.0.smithi071.stdout:0/569: readlink d0/l40 0 2024-04-23T02:14:04.943 INFO:tasks.workunit.client.0.smithi071.stdout:9/582: creat d1/d4e/d66/fca x:0 0 0 2024-04-23T02:14:04.944 INFO:tasks.workunit.client.1.smithi192.stdout:8/880: creat d1/d12/d23/d2c/d4d/f133 x:0 0 0 2024-04-23T02:14:04.945 INFO:tasks.workunit.client.0.smithi071.stdout:0/570: chown d0/d3/de/d48/d7e/d8b/l79 690800 1 2024-04-23T02:14:04.946 INFO:tasks.workunit.client.1.smithi192.stdout:8/881: mknod d1/d12/d7d/dd2/c134 0 2024-04-23T02:14:04.947 INFO:tasks.workunit.client.0.smithi071.stdout:7/682: dwrite d3/d17/d1c/d2a/d73/fc6 [0,4194304] 0 2024-04-23T02:14:04.948 INFO:tasks.workunit.client.0.smithi071.stdout:0/571: dread - d0/d3/de/d10/d14/d32/d44/d87/d93/fa3 zero size 2024-04-23T02:14:04.949 INFO:tasks.workunit.client.1.smithi192.stdout:8/882: rename d1/d12/c16 to d1/d9/d3e/d59/d8a/daa/c135 0 2024-04-23T02:14:04.950 INFO:tasks.workunit.client.0.smithi071.stdout:9/583: creat d1/d32/d41/db7/d39/d42/fcb x:0 0 0 2024-04-23T02:14:04.950 INFO:tasks.workunit.client.0.smithi071.stdout:0/572: fsync d0/f2 0 2024-04-23T02:14:04.951 INFO:tasks.workunit.client.1.smithi192.stdout:8/883: symlink d1/d9/d3e/d59/d8a/daa/l136 0 2024-04-23T02:14:04.951 INFO:tasks.workunit.client.0.smithi071.stdout:9/584: truncate d1/db8/fbd 127010 0 2024-04-23T02:14:04.952 INFO:tasks.workunit.client.1.smithi192.stdout:8/884: write d1/d12/d68/fad [3864424,106351] 0 2024-04-23T02:14:04.952 INFO:tasks.workunit.client.0.smithi071.stdout:7/683: rename d3/d17/d1c/d2a/d73/d83/ddd to d3/d51/d80/de9 0 2024-04-23T02:14:04.954 INFO:tasks.workunit.client.0.smithi071.stdout:9/585: truncate d1/d32/d41/db7/d39/fa5 665051 0 2024-04-23T02:14:04.957 INFO:tasks.workunit.client.0.smithi071.stdout:9/586: read d1/d32/d3a/d5f/f63 [2060127,93427] 0 2024-04-23T02:14:04.958 INFO:tasks.workunit.client.1.smithi192.stdout:8/885: creat d1/d12/d23/d4b/d73/d4c/f137 x:0 0 0 2024-04-23T02:14:04.960 INFO:tasks.workunit.client.0.smithi071.stdout:7/684: dwrite d3/d17/d1c/d31/f48 [0,4194304] 0 2024-04-23T02:14:04.960 INFO:tasks.workunit.client.1.smithi192.stdout:8/886: unlink d1/df/c2b 0 2024-04-23T02:14:04.961 INFO:tasks.workunit.client.0.smithi071.stdout:9/587: mkdir d1/d32/d41/db7/d39/d42/dcc 0 2024-04-23T02:14:04.962 INFO:tasks.workunit.client.1.smithi192.stdout:8/887: unlink d1/df/fa8 0 2024-04-23T02:14:04.962 INFO:tasks.workunit.client.0.smithi071.stdout:0/573: link d0/d76/l99 d0/d3/de/d10/d14/d32/d37/lc0 0 2024-04-23T02:14:04.967 INFO:tasks.workunit.client.0.smithi071.stdout:7/685: rename d3/d17/d41/d34/d59 to d3/dae/db7/dea 0 2024-04-23T02:14:04.970 INFO:tasks.workunit.client.1.smithi192.stdout:8/888: dwrite d1/d12/d7d/f10b [0,4194304] 0 2024-04-23T02:14:04.971 INFO:tasks.workunit.client.0.smithi071.stdout:9/588: dwrite d1/d32/d41/db7/d39/f8f [0,4194304] 0 2024-04-23T02:14:04.972 INFO:tasks.workunit.client.0.smithi071.stdout:0/574: link d0/d3/de/d10/d14/d32/d37/d26/c54 d0/cc1 0 2024-04-23T02:14:04.974 INFO:tasks.workunit.client.1.smithi192.stdout:8/889: mknod d1/d12/d68/ddf/c138 0 2024-04-23T02:14:04.974 INFO:tasks.workunit.client.0.smithi071.stdout:7/686: creat d3/feb x:0 0 0 2024-04-23T02:14:04.977 INFO:tasks.workunit.client.0.smithi071.stdout:9/589: creat d1/d32/d41/db7/d39/d42/dcc/fcd x:0 0 0 2024-04-23T02:14:04.979 INFO:tasks.workunit.client.0.smithi071.stdout:0/575: dread d0/d63/fbf [0,4194304] 0 2024-04-23T02:14:04.980 INFO:tasks.workunit.client.0.smithi071.stdout:7/687: creat d3/d17/d1c/d2a/d73/d83/de7/fec x:0 0 0 2024-04-23T02:14:04.982 INFO:tasks.workunit.client.0.smithi071.stdout:9/590: rename d1/c21 to d1/d4e/d66/da4/cce 0 2024-04-23T02:14:04.985 INFO:tasks.workunit.client.0.smithi071.stdout:7/688: rename d3/d17/d41/d3a/l85 to d3/d17/dca/led 0 2024-04-23T02:14:04.985 INFO:tasks.workunit.client.1.smithi192.stdout:8/890: dwrite d1/d12/d23/d4b/d73/d4c/f137 [0,4194304] 0 2024-04-23T02:14:04.986 INFO:tasks.workunit.client.0.smithi071.stdout:9/591: creat d1/d32/d3a/d90/d99/fcf x:0 0 0 2024-04-23T02:14:04.988 INFO:tasks.workunit.client.0.smithi071.stdout:0/576: rename d0/d3/de/d10/d14/d84/fb5 to d0/d3/de/d10/d14/d32/d37/d26/fc2 0 2024-04-23T02:14:04.990 INFO:tasks.workunit.client.0.smithi071.stdout:9/592: write d1/f16 [1258275,104236] 0 2024-04-23T02:14:04.991 INFO:tasks.workunit.client.0.smithi071.stdout:7/689: creat d3/d17/d1c/fee x:0 0 0 2024-04-23T02:14:04.994 INFO:tasks.workunit.client.0.smithi071.stdout:9/593: symlink d1/d32/d7e/ld0 0 2024-04-23T02:14:04.996 INFO:tasks.workunit.client.0.smithi071.stdout:0/577: rename d0/d3/de/d10/d14/d32/d44/d31/d52/c9f to d0/d3/de/d10/d14/d32/d8a/cc3 0 2024-04-23T02:14:05.002 INFO:tasks.workunit.client.0.smithi071.stdout:7/690: link d3/d17/d1c/f45 d3/d17/d41/d3a/da5/fef 0 2024-04-23T02:14:05.006 INFO:tasks.workunit.client.0.smithi071.stdout:0/578: getdents d0/d3 0 2024-04-23T02:14:05.007 INFO:tasks.workunit.client.0.smithi071.stdout:7/691: dread d3/d17/d1c/f8e [0,4194304] 0 2024-04-23T02:14:05.009 INFO:tasks.workunit.client.1.smithi192.stdout:8/891: mkdir d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d139 0 2024-04-23T02:14:05.015 INFO:tasks.workunit.client.0.smithi071.stdout:0/579: rename d0/d3/de/d10/d14/d32/d37/fa1 to d0/d3/de/d10/db9/fc4 0 2024-04-23T02:14:05.017 INFO:tasks.workunit.client.1.smithi192.stdout:8/892: symlink d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/d108/d11f/l13a 0 2024-04-23T02:14:05.019 INFO:tasks.workunit.client.0.smithi071.stdout:5/679: sync 2024-04-23T02:14:05.020 INFO:tasks.workunit.client.1.smithi192.stdout:8/893: mkdir d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/da3/d12d/d13b 0 2024-04-23T02:14:05.020 INFO:tasks.workunit.client.0.smithi071.stdout:7/692: dwrite d3/d17/d1c/d2a/d43/d86/fd7 [0,4194304] 0 2024-04-23T02:14:05.021 INFO:tasks.workunit.client.0.smithi071.stdout:7/693: readlink d3/d17/l8d 0 2024-04-23T02:14:05.024 INFO:tasks.workunit.client.1.smithi192.stdout:8/894: rename d1/df/f50 to d1/d12/d23/d4b/d73/dda/d113/f13c 0 2024-04-23T02:14:05.026 INFO:tasks.workunit.client.1.smithi192.stdout:8/895: mkdir d1/d12/d23/d4b/d73/dda/d10f/d13d 0 2024-04-23T02:14:05.027 INFO:tasks.workunit.client.0.smithi071.stdout:0/580: creat d0/d3/de/d10/d14/d32/d44/fc5 x:0 0 0 2024-04-23T02:14:05.028 INFO:tasks.workunit.client.0.smithi071.stdout:0/581: mknod d0/d92/db2/cc6 0 2024-04-23T02:14:05.030 INFO:tasks.workunit.client.0.smithi071.stdout:9/594: dwrite d1/d32/d41/db7/f25 [12582912,4194304] 0 2024-04-23T02:14:05.032 INFO:tasks.workunit.client.0.smithi071.stdout:7/694: dread d3/d17/d41/d34/d39/d3b/fb1 [0,4194304] 0 2024-04-23T02:14:05.032 INFO:tasks.workunit.client.0.smithi071.stdout:9/595: chown d1/d32/d7e/c92 30994 1 2024-04-23T02:14:05.038 INFO:tasks.workunit.client.1.smithi192.stdout:7/690: sync 2024-04-23T02:14:05.038 INFO:tasks.workunit.client.1.smithi192.stdout:7/691: write d5/d17/d1c/d59/d5a/d41/f98 [93970,119086] 0 2024-04-23T02:14:05.039 INFO:tasks.workunit.client.0.smithi071.stdout:0/582: symlink d0/d3/de/d10/db9/lc7 0 2024-04-23T02:14:05.040 INFO:tasks.workunit.client.0.smithi071.stdout:7/695: symlink d3/d17/d41/d3a/da5/lf0 0 2024-04-23T02:14:05.041 INFO:tasks.workunit.client.1.smithi192.stdout:7/692: mknod d5/d17/d1c/d57/cc8 0 2024-04-23T02:14:05.042 INFO:tasks.workunit.client.1.smithi192.stdout:8/896: truncate d1/d9/d3e/d59/d8a/daa/f11e 3440080 0 2024-04-23T02:14:05.042 INFO:tasks.workunit.client.0.smithi071.stdout:0/583: write d0/d3/de/d10/d14/d32/d44/fc5 [504313,16874] 0 2024-04-23T02:14:05.048 INFO:tasks.workunit.client.0.smithi071.stdout:9/596: rename d1/d32/d41/db7/d68 to d1/d32/dc5/dd1 0 2024-04-23T02:14:05.050 INFO:tasks.workunit.client.0.smithi071.stdout:0/584: creat d0/d3/de/d10/d16/fc8 x:0 0 0 2024-04-23T02:14:05.051 INFO:tasks.workunit.client.0.smithi071.stdout:7/696: rename d3/d17/d41/d34/l7c to d3/d17/d41/d3a/da5/lf1 0 2024-04-23T02:14:05.059 INFO:tasks.workunit.client.0.smithi071.stdout:0/585: dwrite d0/d3/de/d10/d14/f64 [0,4194304] 0 2024-04-23T02:14:05.064 INFO:tasks.workunit.client.0.smithi071.stdout:9/597: write d1/d32/d3a/d94/f9b [1064614,117185] 0 2024-04-23T02:14:05.068 INFO:tasks.workunit.client.0.smithi071.stdout:9/598: creat d1/d32/d41/fd2 x:0 0 0 2024-04-23T02:14:05.071 INFO:tasks.workunit.client.0.smithi071.stdout:7/697: dwrite d3/d17/d1c/d31/fc2 [0,4194304] 0 2024-04-23T02:14:05.072 INFO:tasks.workunit.client.1.smithi192.stdout:7/693: creat d5/fc9 x:0 0 0 2024-04-23T02:14:05.074 INFO:tasks.workunit.client.0.smithi071.stdout:9/599: dread d1/d32/d41/db7/d71/f9e [0,4194304] 0 2024-04-23T02:14:05.074 INFO:tasks.workunit.client.0.smithi071.stdout:7/698: symlink d3/d17/d41/de2/lf2 0 2024-04-23T02:14:05.074 INFO:tasks.workunit.client.1.smithi192.stdout:8/897: rmdir d1/d12/d23/d4b/de6 0 2024-04-23T02:14:05.076 INFO:tasks.workunit.client.0.smithi071.stdout:9/600: dread - d1/d32/d41/fd2 zero size 2024-04-23T02:14:05.076 INFO:tasks.workunit.client.0.smithi071.stdout:7/699: chown d3/d17/d1c/d2a/d73/fc0 68037 1 2024-04-23T02:14:05.076 INFO:tasks.workunit.client.0.smithi071.stdout:9/601: chown d1 1280 1 2024-04-23T02:14:05.079 INFO:tasks.workunit.client.0.smithi071.stdout:9/602: write d1/d32/d41/db7/f70 [1375832,50599] 0 2024-04-23T02:14:05.079 INFO:tasks.workunit.client.1.smithi192.stdout:8/898: chown d1/d9/d3e/d59/d8a/daa/f11e 23490404 1 2024-04-23T02:14:05.082 INFO:tasks.workunit.client.1.smithi192.stdout:8/899: creat d1/de1/f13e x:0 0 0 2024-04-23T02:14:05.084 INFO:tasks.workunit.client.0.smithi071.stdout:7/700: dwrite d3/dae/db7/dea/f9d [0,4194304] 0 2024-04-23T02:14:05.091 INFO:tasks.workunit.client.0.smithi071.stdout:7/701: dwrite d3/feb [0,4194304] 0 2024-04-23T02:14:05.093 INFO:tasks.workunit.client.1.smithi192.stdout:7/694: dread d5/d17/d1c/f74 [4194304,4194304] 0 2024-04-23T02:14:05.094 INFO:tasks.workunit.client.0.smithi071.stdout:7/702: link d3/d17/d1c/d31/f48 d3/d17/d41/d3a/da5/ff3 0 2024-04-23T02:14:05.096 INFO:tasks.workunit.client.1.smithi192.stdout:7/695: rename d5/d17/d1c/d57/l23 to d5/d17/d1c/d59/d5a/d41/d50/d9f/lca 0 2024-04-23T02:14:05.098 INFO:tasks.workunit.client.1.smithi192.stdout:7/696: creat d5/fcb x:0 0 0 2024-04-23T02:14:05.100 INFO:tasks.workunit.client.0.smithi071.stdout:7/703: dread d3/d17/d1c/d31/f5d [0,4194304] 0 2024-04-23T02:14:05.102 INFO:tasks.workunit.client.1.smithi192.stdout:7/697: link d5/d43/d4c/d5c/d97/lbd d5/d17/d1c/d57/d75/lcc 0 2024-04-23T02:14:05.102 INFO:tasks.workunit.client.0.smithi071.stdout:7/704: mknod d3/d51/d87/db3/cf4 0 2024-04-23T02:14:05.103 INFO:tasks.workunit.client.1.smithi192.stdout:7/698: write d5/d17/d1c/d59/d5a/d41/f54 [219490,40960] 0 2024-04-23T02:14:05.103 INFO:tasks.workunit.client.0.smithi071.stdout:7/705: mkdir d3/d51/df5 0 2024-04-23T02:14:05.106 INFO:tasks.workunit.client.0.smithi071.stdout:7/706: creat d3/d17/d1c/d2a/d73/dba/ff6 x:0 0 0 2024-04-23T02:14:05.108 INFO:tasks.workunit.client.0.smithi071.stdout:7/707: mknod d3/cf7 0 2024-04-23T02:14:05.109 INFO:tasks.workunit.client.1.smithi192.stdout:7/699: dread d5/d17/d1c/d26/f34 [0,4194304] 0 2024-04-23T02:14:05.110 INFO:tasks.workunit.client.0.smithi071.stdout:7/708: creat d3/d51/df5/ff8 x:0 0 0 2024-04-23T02:14:05.110 INFO:tasks.workunit.client.1.smithi192.stdout:7/700: chown d5/d17/d1c/f74 1921121751 1 2024-04-23T02:14:05.112 INFO:tasks.workunit.client.0.smithi071.stdout:7/709: read d3/d17/d41/f33 [555985,94599] 0 2024-04-23T02:14:05.112 INFO:tasks.workunit.client.1.smithi192.stdout:7/701: symlink d5/d17/d1c/lcd 0 2024-04-23T02:14:05.115 INFO:tasks.workunit.client.0.smithi071.stdout:7/710: rename d3/d17/d41/d3a/f58 to d3/d17/d1c/d2a/d73/ff9 0 2024-04-23T02:14:05.117 INFO:tasks.workunit.client.0.smithi071.stdout:7/711: symlink d3/d17/d1c/d2a/d43/lfa 0 2024-04-23T02:14:05.117 INFO:tasks.workunit.client.1.smithi192.stdout:7/702: dread d5/d43/d4c/d5c/d67/f82 [0,4194304] 0 2024-04-23T02:14:05.118 INFO:tasks.workunit.client.0.smithi071.stdout:7/712: chown d3/d17/d1c/d31/cb6 173425 1 2024-04-23T02:14:05.120 INFO:tasks.workunit.client.1.smithi192.stdout:7/703: creat d5/d17/d1c/d59/d5a/d41/d50/d9f/fce x:0 0 0 2024-04-23T02:14:05.120 INFO:tasks.workunit.client.1.smithi192.stdout:7/704: chown d5/d43/d4c/d4e 402073 1 2024-04-23T02:14:05.120 INFO:tasks.workunit.client.0.smithi071.stdout:7/713: rename d3/d51/d87/db3/l9a to d3/d17/d1c/d2a/d43/d86/lfb 0 2024-04-23T02:14:05.121 INFO:tasks.workunit.client.1.smithi192.stdout:7/705: write d5/d17/d1c/d26/f28 [872282,26516] 0 2024-04-23T02:14:05.122 INFO:tasks.workunit.client.0.smithi071.stdout:7/714: symlink d3/d51/df5/lfc 0 2024-04-23T02:14:05.123 INFO:tasks.workunit.client.1.smithi192.stdout:7/706: symlink d5/d17/d1c/d59/d5a/d9b/lcf 0 2024-04-23T02:14:05.123 INFO:tasks.workunit.client.1.smithi192.stdout:7/707: chown d5/d43/d4c/d4e/c5d 31092 1 2024-04-23T02:14:05.126 INFO:tasks.workunit.client.1.smithi192.stdout:7/708: creat d5/d17/d1c/d59/fd0 x:0 0 0 2024-04-23T02:14:05.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:04 smithi192 ceph-mon[49662]: pgmap v29: 65 pgs: 65 active+clean; 2.4 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 87 MiB/s rd, 345 MiB/s wr, 614 op/s 2024-04-23T02:14:05.128 INFO:tasks.workunit.client.0.smithi071.stdout:7/715: dwrite d3/d51/d80/f9e [0,4194304] 0 2024-04-23T02:14:05.128 INFO:tasks.workunit.client.1.smithi192.stdout:7/709: symlink d5/d17/d1c/d59/d5a/d41/d50/d3d/ld1 0 2024-04-23T02:14:05.129 INFO:tasks.workunit.client.1.smithi192.stdout:7/710: mkdir d5/d17/d1c/d4a/dd2 0 2024-04-23T02:14:05.131 INFO:tasks.workunit.client.0.smithi071.stdout:7/716: unlink d3/feb 0 2024-04-23T02:14:05.134 INFO:tasks.workunit.client.0.smithi071.stdout:7/717: truncate d3/d17/d1c/d31/f5d 1480585 0 2024-04-23T02:14:05.135 INFO:tasks.workunit.client.0.smithi071.stdout:7/718: mknod d3/d17/d41/d34/d6e/cfd 0 2024-04-23T02:14:05.137 INFO:tasks.workunit.client.0.smithi071.stdout:7/719: creat d3/dae/ffe x:0 0 0 2024-04-23T02:14:05.160 INFO:tasks.workunit.client.1.smithi192.stdout:8/900: dwrite d1/d12/d23/d4b/d73/d4c/f64 [0,4194304] 0 2024-04-23T02:14:05.162 INFO:tasks.workunit.client.1.smithi192.stdout:7/711: truncate d5/d17/d1c/d26/f28 388859 0 2024-04-23T02:14:05.164 INFO:tasks.workunit.client.1.smithi192.stdout:7/712: symlink d5/d43/d4c/d4e/ld3 0 2024-04-23T02:14:05.166 INFO:tasks.workunit.client.1.smithi192.stdout:7/713: fsync d5/d17/d1c/d26/f34 0 2024-04-23T02:14:05.167 INFO:tasks.workunit.client.1.smithi192.stdout:7/714: dread - d5/d17/d1c/d59/fd0 zero size 2024-04-23T02:14:05.168 INFO:tasks.workunit.client.1.smithi192.stdout:8/901: rename d1/d12/d23/d4b/d73/d7e/df0 to d1/df/dd0/d13f 0 2024-04-23T02:14:05.169 INFO:tasks.workunit.client.1.smithi192.stdout:8/902: chown d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/c85 658 1 2024-04-23T02:14:05.179 INFO:tasks.workunit.client.1.smithi192.stdout:7/715: write d5/d43/d4c/d5c/f72 [823351,8732] 0 2024-04-23T02:14:05.180 INFO:tasks.workunit.client.1.smithi192.stdout:7/716: readlink d5/d17/d1c/d59/d5a/d41/d50/l39 0 2024-04-23T02:14:05.181 INFO:tasks.workunit.client.0.smithi071.stdout:8/612: sync 2024-04-23T02:14:05.181 INFO:tasks.workunit.client.0.smithi071.stdout:1/550: sync 2024-04-23T02:14:05.181 INFO:tasks.workunit.client.0.smithi071.stdout:6/575: sync 2024-04-23T02:14:05.181 INFO:tasks.workunit.client.0.smithi071.stdout:4/658: sync 2024-04-23T02:14:05.181 INFO:tasks.workunit.client.0.smithi071.stdout:3/550: sync 2024-04-23T02:14:05.181 INFO:tasks.workunit.client.0.smithi071.stdout:2/550: sync 2024-04-23T02:14:05.183 INFO:tasks.workunit.client.0.smithi071.stdout:8/613: chown dc/d78/f91 15223592 1 2024-04-23T02:14:05.184 INFO:tasks.workunit.client.0.smithi071.stdout:3/551: mkdir d7/d1d/dbc 0 2024-04-23T02:14:05.186 INFO:tasks.workunit.client.0.smithi071.stdout:2/551: write d0/d2/d34/f84 [1389321,112941] 0 2024-04-23T02:14:05.186 INFO:tasks.workunit.client.0.smithi071.stdout:1/551: creat db/df/d13/d2f/d48/d85/fb0 x:0 0 0 2024-04-23T02:14:05.187 INFO:tasks.workunit.client.1.smithi192.stdout:7/717: dwrite d5/d17/d1c/d4a/f58 [0,4194304] 0 2024-04-23T02:14:05.188 INFO:tasks.workunit.client.0.smithi071.stdout:4/659: creat d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d29/fd3 x:0 0 0 2024-04-23T02:14:05.189 INFO:tasks.workunit.client.0.smithi071.stdout:2/552: chown d0/d2/d26/d14/f1c 120992 1 2024-04-23T02:14:05.191 INFO:tasks.workunit.client.1.smithi192.stdout:7/718: creat d5/d17/d1c/d59/d5a/d41/fd4 x:0 0 0 2024-04-23T02:14:05.194 INFO:tasks.workunit.client.0.smithi071.stdout:4/660: dread - d2/d2a/d2d/d39/d42/d49/d67/d7e/dae/fbd zero size 2024-04-23T02:14:05.194 INFO:tasks.workunit.client.0.smithi071.stdout:8/614: dwrite dc/d15/da0/d32/f4b [0,4194304] 0 2024-04-23T02:14:05.194 INFO:tasks.workunit.client.0.smithi071.stdout:2/553: truncate d0/d70/f86 310317 0 2024-04-23T02:14:05.197 INFO:tasks.workunit.client.0.smithi071.stdout:6/576: truncate d6/fc 4616823 0 2024-04-23T02:14:05.197 INFO:tasks.workunit.client.0.smithi071.stdout:3/552: dread d7/da5/db2/db7/d68/d3b/d4c/f63 [0,4194304] 0 2024-04-23T02:14:05.198 INFO:tasks.workunit.client.0.smithi071.stdout:6/577: fdatasync d6/d22/f41 0 2024-04-23T02:14:05.198 INFO:tasks.workunit.client.0.smithi071.stdout:2/554: mknod d0/d2/d26/d14/d1f/cb3 0 2024-04-23T02:14:05.199 INFO:tasks.workunit.client.0.smithi071.stdout:8/615: symlink dc/d15/d72/lc2 0 2024-04-23T02:14:05.200 INFO:tasks.workunit.client.0.smithi071.stdout:4/661: write d2/d2a/d2d/d39/d42/d49/d67/f8f [2790086,124209] 0 2024-04-23T02:14:05.202 INFO:tasks.workunit.client.0.smithi071.stdout:3/553: symlink d7/da5/db2/db7/d68/d3b/d4c/d8b/lbd 0 2024-04-23T02:14:05.203 INFO:tasks.workunit.client.0.smithi071.stdout:8/616: write dc/d15/da0/d32/f4b [4981237,26343] 0 2024-04-23T02:14:05.204 INFO:tasks.workunit.client.0.smithi071.stdout:2/555: dread - d0/d2/d26/d14/d1f/d4f/d98/fa6 zero size 2024-04-23T02:14:05.204 INFO:tasks.workunit.client.0.smithi071.stdout:6/578: chown d6/d17/d77/d9f/dad/d3f/f47 10 1 2024-04-23T02:14:05.207 INFO:tasks.workunit.client.1.smithi192.stdout:7/719: dread d5/d17/d1c/d59/d5a/d70/f7e [0,4194304] 0 2024-04-23T02:14:05.208 INFO:tasks.workunit.client.0.smithi071.stdout:2/556: mknod d0/d2/d26/d73/cb4 0 2024-04-23T02:14:05.209 INFO:tasks.workunit.client.0.smithi071.stdout:8/617: creat dc/d15/da0/d8d/fc3 x:0 0 0 2024-04-23T02:14:05.209 INFO:tasks.workunit.client.0.smithi071.stdout:6/579: dread d6/d17/d77/d9f/dad/f8d [0,4194304] 0 2024-04-23T02:14:05.209 INFO:tasks.workunit.client.1.smithi192.stdout:7/720: fsync d5/d17/f92 0 2024-04-23T02:14:05.210 INFO:tasks.workunit.client.0.smithi071.stdout:3/554: creat d7/da5/db2/db7/d68/d3b/fbe x:0 0 0 2024-04-23T02:14:05.211 INFO:tasks.workunit.client.0.smithi071.stdout:8/618: stat dc/c13 0 2024-04-23T02:14:05.213 INFO:tasks.workunit.client.1.smithi192.stdout:7/721: dread d5/d17/d1c/f74 [4194304,4194304] 0 2024-04-23T02:14:05.213 INFO:tasks.workunit.client.0.smithi071.stdout:2/557: mknod d0/d2/da0/cb5 0 2024-04-23T02:14:05.213 INFO:tasks.workunit.client.0.smithi071.stdout:6/580: creat d6/d22/fb9 x:0 0 0 2024-04-23T02:14:05.214 INFO:tasks.workunit.client.0.smithi071.stdout:4/662: dwrite d2/d2a/d2d/f54 [0,4194304] 0 2024-04-23T02:14:05.215 INFO:tasks.workunit.client.0.smithi071.stdout:3/555: truncate d7/d8/f16 4875571 0 2024-04-23T02:14:05.215 INFO:tasks.workunit.client.1.smithi192.stdout:7/722: mknod d5/d17/d1c/d26/cd5 0 2024-04-23T02:14:05.216 INFO:tasks.workunit.client.1.smithi192.stdout:7/723: mknod d5/d43/d4c/cd6 0 2024-04-23T02:14:05.216 INFO:tasks.workunit.client.0.smithi071.stdout:8/619: dread dc/d15/d2d/f38 [0,4194304] 0 2024-04-23T02:14:05.217 INFO:tasks.workunit.client.0.smithi071.stdout:2/558: symlink d0/d2/d26/d3a/d75/lb6 0 2024-04-23T02:14:05.219 INFO:tasks.workunit.client.1.smithi192.stdout:9/573: sync 2024-04-23T02:14:05.223 INFO:tasks.workunit.client.0.smithi071.stdout:4/663: symlink d2/d2a/d2d/d39/ld4 0 2024-04-23T02:14:05.223 INFO:tasks.workunit.client.1.smithi192.stdout:9/574: write d0/d9/d44/d74/f82 [3215381,102595] 0 2024-04-23T02:14:05.224 INFO:tasks.workunit.client.0.smithi071.stdout:2/559: mkdir d0/d2/da0/db7 0 2024-04-23T02:14:05.224 INFO:tasks.workunit.client.0.smithi071.stdout:8/620: creat dc/d3a/da4/db0/fc4 x:0 0 0 2024-04-23T02:14:05.225 INFO:tasks.workunit.client.1.smithi192.stdout:7/724: dwrite d5/d17/d1c/d59/d5a/d41/fd4 [0,4194304] 0 2024-04-23T02:14:05.228 INFO:tasks.workunit.client.0.smithi071.stdout:2/560: unlink d0/d2/d26/d3a/d75/lb6 0 2024-04-23T02:14:05.229 INFO:tasks.workunit.client.0.smithi071.stdout:2/561: chown d0/fab 1840 1 2024-04-23T02:14:05.230 INFO:tasks.workunit.client.1.smithi192.stdout:9/575: rename d0/d9/d44/d51/d64/f77 to d0/d9/d44/d51/d5b/d96/da0/fc3 0 2024-04-23T02:14:05.232 INFO:tasks.workunit.client.1.smithi192.stdout:7/725: creat d5/d17/d1c/d59/d5a/d41/fd7 x:0 0 0 2024-04-23T02:14:05.234 INFO:tasks.workunit.client.0.smithi071.stdout:2/562: mkdir d0/d2/d45/d7d/d56/db0/d30/db8 0 2024-04-23T02:14:05.238 INFO:tasks.workunit.client.1.smithi192.stdout:9/576: rename d0/f81 to d0/d9/d44/d51/fc4 0 2024-04-23T02:14:05.238 INFO:tasks.workunit.client.1.smithi192.stdout:9/577: truncate d0/d9/db3/d5f/f7e 766339 0 2024-04-23T02:14:05.241 INFO:tasks.workunit.client.1.smithi192.stdout:9/578: mknod d0/d9/d44/d51/cc5 0 2024-04-23T02:14:05.242 INFO:tasks.workunit.client.1.smithi192.stdout:9/579: write d0/d9/d44/f7c [5204214,81819] 0 2024-04-23T02:14:05.242 INFO:tasks.workunit.client.0.smithi071.stdout:8/621: truncate dc/d15/da0/f85 2440940 0 2024-04-23T02:14:05.243 INFO:tasks.workunit.client.1.smithi192.stdout:9/580: write d0/d9/d44/f49 [2250342,93262] 0 2024-04-23T02:14:05.244 INFO:tasks.workunit.client.0.smithi071.stdout:2/563: dwrite d0/d2/d49/f5b [0,4194304] 0 2024-04-23T02:14:05.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:04 smithi071 ceph-mon[39837]: pgmap v29: 65 pgs: 65 active+clean; 2.4 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 87 MiB/s rd, 345 MiB/s wr, 614 op/s 2024-04-23T02:14:05.244 INFO:tasks.workunit.client.1.smithi192.stdout:9/581: fsync d0/d9/d44/d74/f82 0 2024-04-23T02:14:05.245 INFO:tasks.workunit.client.0.smithi071.stdout:6/581: dwrite d6/d1f/f25 [4194304,4194304] 0 2024-04-23T02:14:05.246 INFO:tasks.workunit.client.1.smithi192.stdout:9/582: chown d0/d9/f26 27763304 1 2024-04-23T02:14:05.247 INFO:tasks.workunit.client.0.smithi071.stdout:6/582: chown d6/f8 2278142 1 2024-04-23T02:14:05.249 INFO:tasks.workunit.client.0.smithi071.stdout:8/622: creat dc/d3a/da4/db0/fc5 x:0 0 0 2024-04-23T02:14:05.251 INFO:tasks.workunit.client.0.smithi071.stdout:2/564: mknod d0/d2/d45/d7d/d56/db0/d30/d80/cb9 0 2024-04-23T02:14:05.252 INFO:tasks.workunit.client.0.smithi071.stdout:6/583: symlink d6/lba 0 2024-04-23T02:14:05.253 INFO:tasks.workunit.client.0.smithi071.stdout:8/623: mkdir dc/d15/d5d/dc6 0 2024-04-23T02:14:05.253 INFO:tasks.workunit.client.0.smithi071.stdout:8/624: dread - dc/f96 zero size 2024-04-23T02:14:05.254 INFO:tasks.workunit.client.0.smithi071.stdout:2/565: symlink d0/d2/d45/d7d/d56/db0/d30/d80/lba 0 2024-04-23T02:14:05.255 INFO:tasks.workunit.client.0.smithi071.stdout:2/566: chown d0/d2/d26/d3a/d75/f8f 12118 1 2024-04-23T02:14:05.255 INFO:tasks.workunit.client.1.smithi192.stdout:7/726: dread d5/d43/d4c/d5c/d67/da6/fb6 [0,4194304] 0 2024-04-23T02:14:05.256 INFO:tasks.workunit.client.0.smithi071.stdout:8/625: creat dc/d15/da0/d55/fc7 x:0 0 0 2024-04-23T02:14:05.258 INFO:tasks.workunit.client.0.smithi071.stdout:4/664: dread d2/d2a/d2d/d39/d42/f96 [0,4194304] 0 2024-04-23T02:14:05.258 INFO:tasks.workunit.client.0.smithi071.stdout:8/626: creat dc/d56/d64/fc8 x:0 0 0 2024-04-23T02:14:05.260 INFO:tasks.workunit.client.0.smithi071.stdout:2/567: read d0/d70/f94 [2368376,128881] 0 2024-04-23T02:14:05.261 INFO:tasks.workunit.client.0.smithi071.stdout:8/627: creat dc/d15/d50/d5f/fc9 x:0 0 0 2024-04-23T02:14:05.262 INFO:tasks.workunit.client.0.smithi071.stdout:4/665: rmdir d2/d2a/d2d/d39/d42/d49/d67/d7e/dae/dab 39 2024-04-23T02:14:05.263 INFO:tasks.workunit.client.0.smithi071.stdout:8/628: truncate dc/d15/da0/d8d/faf 405217 0 2024-04-23T02:14:05.263 INFO:tasks.workunit.client.0.smithi071.stdout:2/568: fsync d0/d2/d26/f4c 0 2024-04-23T02:14:05.265 INFO:tasks.workunit.client.1.smithi192.stdout:9/583: dread d0/d9/fb [0,4194304] 0 2024-04-23T02:14:05.266 INFO:tasks.workunit.client.0.smithi071.stdout:4/666: creat d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/fd5 x:0 0 0 2024-04-23T02:14:05.269 INFO:tasks.workunit.client.0.smithi071.stdout:8/629: link dc/d15/da0/d8d/faf dc/d15/da0/d55/fca 0 2024-04-23T02:14:05.270 INFO:tasks.workunit.client.0.smithi071.stdout:4/667: write d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c/f7c [1417427,73839] 0 2024-04-23T02:14:05.272 INFO:tasks.workunit.client.0.smithi071.stdout:8/630: write dc/f46 [3710877,83549] 0 2024-04-23T02:14:05.273 INFO:tasks.workunit.client.0.smithi071.stdout:2/569: dwrite d0/d2/d45/d7d/d56/db0/d30/d80/f5f [0,4194304] 0 2024-04-23T02:14:05.274 INFO:tasks.workunit.client.0.smithi071.stdout:4/668: dread - d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d10/f60 zero size 2024-04-23T02:14:05.276 INFO:tasks.workunit.client.0.smithi071.stdout:8/631: read dc/d15/f87 [314223,113207] 0 2024-04-23T02:14:05.280 INFO:tasks.workunit.client.0.smithi071.stdout:4/669: dread d2/d2a/d2d/d39/d42/f55 [0,4194304] 0 2024-04-23T02:14:05.283 INFO:tasks.workunit.client.0.smithi071.stdout:4/670: creat d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d4f/db5/db6/fd6 x:0 0 0 2024-04-23T02:14:05.287 INFO:tasks.workunit.client.0.smithi071.stdout:4/671: creat d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d4f/fd7 x:0 0 0 2024-04-23T02:14:05.289 INFO:tasks.workunit.client.0.smithi071.stdout:4/672: dread - d2/d2a/d2d/d39/d42/d49/d91/fcb zero size 2024-04-23T02:14:05.291 INFO:tasks.workunit.client.0.smithi071.stdout:6/584: dread d6/d22/f28 [0,4194304] 0 2024-04-23T02:14:05.293 INFO:tasks.workunit.client.0.smithi071.stdout:6/585: chown d6/d17/d77/d85/l8b 9078 1 2024-04-23T02:14:05.293 INFO:tasks.workunit.client.0.smithi071.stdout:4/673: read d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c/f2e [7216219,3501] 0 2024-04-23T02:14:05.293 INFO:tasks.workunit.client.0.smithi071.stdout:6/586: chown d6/d17/c60 149 1 2024-04-23T02:14:05.294 INFO:tasks.workunit.client.1.smithi192.stdout:9/584: dread d0/d9/d44/d51/d5b/d61/db1/f8d [0,4194304] 0 2024-04-23T02:14:05.296 INFO:tasks.workunit.client.0.smithi071.stdout:4/674: creat d2/d2a/d2d/d39/fd8 x:0 0 0 2024-04-23T02:14:05.297 INFO:tasks.workunit.client.1.smithi192.stdout:9/585: creat d0/d9/d44/d51/d5b/d96/fc6 x:0 0 0 2024-04-23T02:14:05.300 INFO:tasks.workunit.client.0.smithi071.stdout:4/675: read d2/d2a/d2d/d39/d42/d49/d67/d1d/d57/f72 [499563,58143] 0 2024-04-23T02:14:05.301 INFO:tasks.workunit.client.0.smithi071.stdout:6/587: rename d6/d17/d77/d9f/dad/c64 to d6/d1f/d4e/cbb 0 2024-04-23T02:14:05.303 INFO:tasks.workunit.client.0.smithi071.stdout:8/632: dread dc/d3a/f47 [0,4194304] 0 2024-04-23T02:14:05.304 INFO:tasks.workunit.client.0.smithi071.stdout:4/676: symlink d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c/ld9 0 2024-04-23T02:14:05.304 INFO:tasks.workunit.client.0.smithi071.stdout:6/588: symlink d6/d91/lbc 0 2024-04-23T02:14:05.305 INFO:tasks.workunit.client.0.smithi071.stdout:2/570: dread d0/d2/d26/d14/f23 [4194304,4194304] 0 2024-04-23T02:14:05.306 INFO:tasks.workunit.client.0.smithi071.stdout:2/571: dread - d0/d2/d26/d3a/f77 zero size 2024-04-23T02:14:05.307 INFO:tasks.workunit.client.0.smithi071.stdout:8/633: creat dc/d15/da0/fcb x:0 0 0 2024-04-23T02:14:05.308 INFO:tasks.workunit.client.0.smithi071.stdout:8/634: read - dc/d56/d64/fc8 zero size 2024-04-23T02:14:05.308 INFO:tasks.workunit.client.0.smithi071.stdout:2/572: mknod d0/d2/d49/cbb 0 2024-04-23T02:14:05.308 INFO:tasks.workunit.client.1.smithi192.stdout:9/586: rename d0/d9/d44/d51/d5b/d96 to d0/d9/dc7 0 2024-04-23T02:14:05.315 INFO:tasks.workunit.client.1.smithi192.stdout:9/587: dwrite d0/d9/d44/d58/fbd [0,4194304] 0 2024-04-23T02:14:05.422 INFO:tasks.workunit.client.1.smithi192.stdout:5/691: sync 2024-04-23T02:14:05.422 INFO:tasks.workunit.client.1.smithi192.stdout:1/443: sync 2024-04-23T02:14:05.422 INFO:tasks.workunit.client.1.smithi192.stdout:6/734: sync 2024-04-23T02:14:05.424 INFO:tasks.workunit.client.0.smithi071.stdout:4/677: symlink d2/d2a/d2d/d39/d42/d49/d67/d1d/d23/d4f/db5/db6/dd0/lda 0 2024-04-23T02:14:05.424 INFO:tasks.workunit.client.0.smithi071.stdout:8/635: fsync dc/f57 0 2024-04-23T02:14:05.426 INFO:tasks.workunit.client.0.smithi071.stdout:2/573: symlink d0/daa/lbc 0 2024-04-23T02:14:05.427 INFO:tasks.workunit.client.0.smithi071.stdout:4/678: write d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c/f2e [4391984,89661] 0 2024-04-23T02:14:05.430 INFO:tasks.workunit.client.0.smithi071.stdout:4/679: mkdir d2/d2a/d2d/d39/d42/d49/d91/ddb 0 2024-04-23T02:14:05.431 INFO:tasks.workunit.client.1.smithi192.stdout:6/735: rmdir d3/d28/d79/da0 39 2024-04-23T02:14:05.432 INFO:tasks.workunit.client.1.smithi192.stdout:1/444: creat dc/d14/d23/f87 x:0 0 0 2024-04-23T02:14:05.434 INFO:tasks.workunit.client.1.smithi192.stdout:6/736: creat d3/d15/d1b/d66/d80/d8e/d9d/ff7 x:0 0 0 2024-04-23T02:14:05.435 INFO:tasks.workunit.client.1.smithi192.stdout:1/445: chown dc/d14/d23/d5e/d6a/l5c 1232490 1 2024-04-23T02:14:05.435 INFO:tasks.workunit.client.1.smithi192.stdout:5/692: getdents d2/db/d16/d48/d4f/dd1 0 2024-04-23T02:14:05.438 INFO:tasks.workunit.client.1.smithi192.stdout:6/737: rename d3/d15/d1b/d21 to d3/d28/df8 0 2024-04-23T02:14:05.442 INFO:tasks.workunit.client.1.smithi192.stdout:6/738: symlink d3/d28/df8/d3a/dbd/lf9 0 2024-04-23T02:14:05.443 INFO:tasks.workunit.client.1.smithi192.stdout:5/693: creat d2/db8/fea x:0 0 0 2024-04-23T02:14:05.444 INFO:tasks.workunit.client.1.smithi192.stdout:6/739: mkdir d3/d28/df8/d3a/dbd/dec/dfa 0 2024-04-23T02:14:05.447 INFO:tasks.workunit.client.1.smithi192.stdout:6/740: creat d3/d18/dd4/ffb x:0 0 0 2024-04-23T02:14:05.452 INFO:tasks.workunit.client.1.smithi192.stdout:1/446: truncate dc/f76 3510028 0 2024-04-23T02:14:05.456 INFO:tasks.workunit.client.1.smithi192.stdout:1/447: write dc/d14/d15/f54 [259877,23482] 0 2024-04-23T02:14:05.460 INFO:tasks.workunit.client.1.smithi192.stdout:6/741: dwrite d3/d28/df8/fa1 [0,4194304] 0 2024-04-23T02:14:05.460 INFO:tasks.workunit.client.1.smithi192.stdout:1/448: truncate dc/d16/d19/d30/d32/f47 432378 0 2024-04-23T02:14:05.462 INFO:tasks.workunit.client.1.smithi192.stdout:1/449: truncate dc/d14/d23/f87 1016534 0 2024-04-23T02:14:05.463 INFO:tasks.workunit.client.1.smithi192.stdout:1/450: stat dc/d16/c3c 0 2024-04-23T02:14:05.465 INFO:tasks.workunit.client.1.smithi192.stdout:6/742: link d3/d28/df8/d52/f92 d3/d15/d1b/ffc 0 2024-04-23T02:14:05.466 INFO:tasks.workunit.client.1.smithi192.stdout:1/451: mknod dc/d16/c88 0 2024-04-23T02:14:05.466 INFO:tasks.workunit.client.1.smithi192.stdout:6/743: write d3/d28/df8/d52/d7f/fc8 [728235,15241] 0 2024-04-23T02:14:05.468 INFO:tasks.workunit.client.1.smithi192.stdout:1/452: creat dc/d16/d19/d30/f89 x:0 0 0 2024-04-23T02:14:05.469 INFO:tasks.workunit.client.1.smithi192.stdout:6/744: truncate d3/d15/d1b/d66/d80/dbb/fd6 842643 0 2024-04-23T02:14:05.469 INFO:tasks.workunit.client.1.smithi192.stdout:1/453: dread - dc/d16/d19/d30/f82 zero size 2024-04-23T02:14:05.470 INFO:tasks.workunit.client.1.smithi192.stdout:6/745: chown d3/d15/d1b/dcf 48847 1 2024-04-23T02:14:05.472 INFO:tasks.workunit.client.1.smithi192.stdout:1/454: symlink dc/l8a 0 2024-04-23T02:14:05.476 INFO:tasks.workunit.client.1.smithi192.stdout:1/455: unlink dc/d14/f6e 0 2024-04-23T02:14:05.478 INFO:tasks.workunit.client.1.smithi192.stdout:6/746: dwrite d3/d28/df8/d52/f67 [0,4194304] 0 2024-04-23T02:14:05.478 INFO:tasks.workunit.client.1.smithi192.stdout:1/456: write dc/f39 [3924659,13705] 0 2024-04-23T02:14:05.486 INFO:tasks.workunit.client.0.smithi071.stdout:2/574: dread d0/d2/d26/d14/f22 [0,4194304] 0 2024-04-23T02:14:05.489 INFO:tasks.workunit.client.0.smithi071.stdout:4/680: mknod d2/d2a/d2d/d39/d42/d49/d91/ddb/cdc 0 2024-04-23T02:14:05.489 INFO:tasks.workunit.client.1.smithi192.stdout:6/747: write d3/d18/f2d [915171,116375] 0 2024-04-23T02:14:05.490 INFO:tasks.workunit.client.0.smithi071.stdout:2/575: creat d0/d2/d45/d7d/fbd x:0 0 0 2024-04-23T02:14:05.494 INFO:tasks.workunit.client.0.smithi071.stdout:2/576: read d0/d82/f87 [2256712,107127] 0 2024-04-23T02:14:05.495 INFO:tasks.workunit.client.0.smithi071.stdout:4/681: link d2/d2a/d2d/d39/d42/d49/d67/d3e/c79 d2/d2a/d2d/d39/d42/d49/d67/d1d/d2c/cdd 0 2024-04-23T02:14:05.495 INFO:tasks.workunit.client.0.smithi071.stdout:2/577: stat d0/d2/d45/d7d/d56/db0 0 2024-04-23T02:14:05.496 INFO:tasks.workunit.client.0.smithi071.stdout:4/682: chown d2/d53/fc0 51 1 2024-04-23T02:14:05.497 INFO:tasks.workunit.client.0.smithi071.stdout:2/578: dread - d0/d2/d26/d14/fa7 zero size 2024-04-23T02:14:05.499 INFO:tasks.workunit.client.0.smithi071.stdout:8/636: dread dc/d15/d50/f82 [0,4194304] 0 2024-04-23T02:14:05.499 INFO:tasks.workunit.client.0.smithi071.stdout:2/579: rmdir d0/d2/d34/d55 39 2024-04-23T02:14:05.500 INFO:tasks.workunit.client.1.smithi192.stdout:6/748: dwrite d3/d15/d5b/f82 [0,4194304] 0 2024-04-23T02:14:05.501 INFO:tasks.workunit.client.0.smithi071.stdout:2/580: write d0/d2/d26/d3a/f3d [1461179,56643] 0 2024-04-23T02:14:05.503 INFO:tasks.workunit.client.1.smithi192.stdout:1/457: dread dc/d14/f21 [0,4194304] 0 2024-04-23T02:14:05.504 INFO:tasks.workunit.client.1.smithi192.stdout:6/749: mknod d3/d28/df8/d3a/cfd 0 2024-04-23T02:14:05.507 INFO:tasks.workunit.client.1.smithi192.stdout:6/750: unlink d3/d15/d1b/ffc 0 2024-04-23T02:14:05.510 INFO:tasks.workunit.client.1.smithi192.stdout:1/458: rename dc/d16/d19/d30/d32 to dc/d8b 0 2024-04-23T02:14:05.511 INFO:tasks.workunit.client.0.smithi071.stdout:8/637: unlink dc/d15/d50/d5f/fc1 0 2024-04-23T02:14:05.512 INFO:tasks.workunit.client.1.smithi192.stdout:1/459: mkdir dc/d8b/d34/d70/d8c 0 2024-04-23T02:14:05.512 INFO:tasks.workunit.client.0.smithi071.stdout:8/638: creat dc/d78/fcc x:0 0 0 2024-04-23T02:14:05.514 INFO:tasks.workunit.client.0.smithi071.stdout:4/683: rename d2/d2a/d2d/d39/d42/d49/d67/d1d to d2/d2a/da2/dd1/dde 0 2024-04-23T02:14:05.515 INFO:tasks.workunit.client.1.smithi192.stdout:1/460: read dc/d8b/f4d [355760,18750] 0 2024-04-23T02:14:05.517 INFO:tasks.workunit.client.0.smithi071.stdout:2/581: rename d0/d2/d45/d7d/d56/db0/f40 to d0/d2/d26/d14/d1f/d36/fbe 0 2024-04-23T02:14:05.518 INFO:tasks.workunit.client.0.smithi071.stdout:2/582: write d0/d2/f1d [384685,111155] 0 2024-04-23T02:14:05.520 INFO:tasks.workunit.client.0.smithi071.stdout:2/583: write d0/d2/d45/d7d/fbd [364188,45649] 0 2024-04-23T02:14:05.522 INFO:tasks.workunit.client.1.smithi192.stdout:1/461: dwrite dc/f10 [0,4194304] 0 2024-04-23T02:14:05.525 INFO:tasks.workunit.client.0.smithi071.stdout:2/584: mkdir d0/d2/d34/dbf 0 2024-04-23T02:14:05.526 INFO:tasks.workunit.client.1.smithi192.stdout:1/462: mknod dc/c8d 0 2024-04-23T02:14:05.527 INFO:tasks.workunit.client.1.smithi192.stdout:1/463: readlink dc/d14/d15/d35/l3b 0 2024-04-23T02:14:05.529 INFO:tasks.workunit.client.0.smithi071.stdout:8/639: dwrite dc/d15/d2d/f3f [0,4194304] 0 2024-04-23T02:14:05.530 INFO:tasks.workunit.client.1.smithi192.stdout:1/464: rename dc/d14/d23/c7e to dc/d14/d23/c8e 0 2024-04-23T02:14:05.532 INFO:tasks.workunit.client.0.smithi071.stdout:4/684: dwrite d2/d2a/d2d/d39/d42/d49/d67/d3e/f76 [0,4194304] 0 2024-04-23T02:14:05.534 INFO:tasks.workunit.client.1.smithi192.stdout:1/465: creat dc/d16/d19/d74/f8f x:0 0 0 2024-04-23T02:14:05.534 INFO:tasks.workunit.client.0.smithi071.stdout:4/685: write d2/d2a/da2/dd1/dde/d23/f5b [2417435,123793] 0 2024-04-23T02:14:05.535 INFO:tasks.workunit.client.0.smithi071.stdout:8/640: creat dc/d15/da0/d8d/fcd x:0 0 0 2024-04-23T02:14:05.536 INFO:tasks.workunit.client.0.smithi071.stdout:4/686: stat d2/d2a/da2/dd1/dde/c98 0 2024-04-23T02:14:05.536 INFO:tasks.workunit.client.1.smithi192.stdout:6/751: dread d3/d28/f56 [0,4194304] 0 2024-04-23T02:14:05.537 INFO:tasks.workunit.client.1.smithi192.stdout:1/466: fdatasync f9 0 2024-04-23T02:14:05.537 INFO:tasks.workunit.client.0.smithi071.stdout:8/641: mknod dc/d15/d72/d95/cce 0 2024-04-23T02:14:05.540 INFO:tasks.workunit.client.0.smithi071.stdout:8/642: symlink dc/d3a/da4/lcf 0 2024-04-23T02:14:05.545 INFO:tasks.workunit.client.0.smithi071.stdout:8/643: mknod dc/d15/d50/cd0 0 2024-04-23T02:14:05.546 INFO:tasks.workunit.client.1.smithi192.stdout:6/752: write d3/d18/f2d [1089428,82672] 0 2024-04-23T02:14:05.547 INFO:tasks.workunit.client.0.smithi071.stdout:4/687: dwrite d2/d2a/da2/dd1/dde/d23/d29/f83 [0,4194304] 0 2024-04-23T02:14:05.548 INFO:tasks.workunit.client.0.smithi071.stdout:2/585: dwrite d0/d2/d26/d14/f22 [0,4194304] 0 2024-04-23T02:14:05.549 INFO:tasks.workunit.client.1.smithi192.stdout:1/467: symlink dc/d16/d19/d30/l90 0 2024-04-23T02:14:05.551 INFO:tasks.workunit.client.0.smithi071.stdout:2/586: chown d0/d2/d45/d7d/d56/db0/d30/d80/f5f 67322 1 2024-04-23T02:14:05.551 INFO:tasks.workunit.client.1.smithi192.stdout:1/468: unlink dc/d8b/d44/f53 0 2024-04-23T02:14:05.552 INFO:tasks.workunit.client.1.smithi192.stdout:1/469: chown dc/c2c 1518129 1 2024-04-23T02:14:05.554 INFO:tasks.workunit.client.0.smithi071.stdout:8/644: getdents dc 0 2024-04-23T02:14:05.555 INFO:tasks.workunit.client.0.smithi071.stdout:8/645: fdatasync dc/d56/f61 0 2024-04-23T02:14:05.558 INFO:tasks.workunit.client.0.smithi071.stdout:8/646: getdents dc/d15/d72 0 2024-04-23T02:14:05.560 INFO:tasks.workunit.client.0.smithi071.stdout:8/647: write dc/d15/da0/d30/fa6 [2059085,75980] 0 2024-04-23T02:14:05.561 INFO:tasks.workunit.client.0.smithi071.stdout:8/648: write dc/d15/da0/d8d/fcd [596029,97231] 0 2024-04-23T02:14:05.565 INFO:tasks.workunit.client.1.smithi192.stdout:6/753: truncate d3/d15/d1b/f8f 4124818 0 2024-04-23T02:14:05.565 INFO:tasks.workunit.client.0.smithi071.stdout:0/586: sync 2024-04-23T02:14:05.566 INFO:tasks.workunit.client.0.smithi071.stdout:5/680: sync 2024-04-23T02:14:05.567 INFO:tasks.workunit.client.1.smithi192.stdout:6/754: creat d3/d28/df8/d52/db2/ffe x:0 0 0 2024-04-23T02:14:05.568 INFO:tasks.workunit.client.0.smithi071.stdout:8/649: getdents dc/d15/d5d 0 2024-04-23T02:14:05.568 INFO:tasks.workunit.client.0.smithi071.stdout:0/587: mknod d0/d92/db2/cc9 0 2024-04-23T02:14:05.569 INFO:tasks.workunit.client.0.smithi071.stdout:5/681: creat d3/d1d/daf/d83/ffb x:0 0 0 2024-04-23T02:14:05.570 INFO:tasks.workunit.client.0.smithi071.stdout:8/650: truncate dc/d15/da0/d8d/fcd 1335660 0 2024-04-23T02:14:05.571 INFO:tasks.workunit.client.0.smithi071.stdout:8/651: chown dc/d15 1 1 2024-04-23T02:14:05.571 INFO:tasks.workunit.client.1.smithi192.stdout:6/755: dread d3/d28/df8/d52/d7f/fc8 [0,4194304] 0 2024-04-23T02:14:05.572 INFO:tasks.workunit.client.0.smithi071.stdout:5/682: creat d3/d1d/d43/d4b/ffc x:0 0 0 2024-04-23T02:14:05.573 INFO:tasks.workunit.client.0.smithi071.stdout:0/588: mknod d0/d3/de/d10/d14/d32/d44/d31/cca 0 2024-04-23T02:14:05.574 INFO:tasks.workunit.client.0.smithi071.stdout:0/589: readlink d0/d3/de/d10/l3b 0 2024-04-23T02:14:05.575 INFO:tasks.workunit.client.0.smithi071.stdout:2/587: write d0/d2/d26/d14/d1f/d36/d6d/faf [563899,46874] 0 2024-04-23T02:14:05.576 INFO:tasks.workunit.client.0.smithi071.stdout:0/590: rmdir d0/d3/de/d10/db3/dba 39 2024-04-23T02:14:05.579 INFO:tasks.workunit.client.1.smithi192.stdout:6/756: dwrite d3/d18/dd4/d8d/ff2 [0,4194304] 0 2024-04-23T02:14:05.582 INFO:tasks.workunit.client.0.smithi071.stdout:0/591: symlink d0/d3/de/d10/db3/dba/lcb 0 2024-04-23T02:14:05.583 INFO:tasks.workunit.client.0.smithi071.stdout:8/652: write dc/d78/f91 [411389,77008] 0 2024-04-23T02:14:05.585 INFO:tasks.workunit.client.1.smithi192.stdout:6/757: write d3/d18/d58/fcd [290395,126605] 0 2024-04-23T02:14:05.585 INFO:tasks.workunit.client.0.smithi071.stdout:2/588: dwrite d0/d2/d45/d7d/d56/fac [4194304,4194304] 0 2024-04-23T02:14:05.589 INFO:tasks.workunit.client.1.smithi192.stdout:6/758: rename d3/d28/df8/d52/d7f to d3/d18/dd4/dff 0 2024-04-23T02:14:05.589 INFO:tasks.workunit.client.0.smithi071.stdout:8/653: symlink dc/d15/d72/ld1 0 2024-04-23T02:14:05.590 INFO:tasks.workunit.client.0.smithi071.stdout:4/688: dread d2/d2a/da2/dd1/dde/d23/d29/f9e [0,4194304] 0 2024-04-23T02:14:05.591 INFO:tasks.workunit.client.1.smithi192.stdout:6/759: mkdir d3/d15/d1b/d66/d80/d8e/d9d/ded/d100 0 2024-04-23T02:14:05.591 INFO:tasks.workunit.client.1.smithi192.stdout:6/760: dread - d3/d18/d91/ff6 zero size 2024-04-23T02:14:05.595 INFO:tasks.workunit.client.0.smithi071.stdout:0/592: mkdir d0/d3/de/d10/d14/d32/d44/d87/d93/dcc 0 2024-04-23T02:14:05.595 INFO:tasks.workunit.client.0.smithi071.stdout:4/689: mknod d2/d2a/d2d/d39/d42/d49/d91/cdf 0 2024-04-23T02:14:05.597 INFO:tasks.workunit.client.0.smithi071.stdout:0/593: fdatasync d0/d3/de/f28 0 2024-04-23T02:14:05.598 INFO:tasks.workunit.client.0.smithi071.stdout:8/654: dwrite dc/d15/f39 [0,4194304] 0 2024-04-23T02:14:05.599 INFO:tasks.workunit.client.0.smithi071.stdout:0/594: dread - d0/d3/de/d10/d14/d32/d37/d26/fbe zero size 2024-04-23T02:14:05.600 INFO:tasks.workunit.client.0.smithi071.stdout:4/690: truncate d2/d2a/d73/f7f 174885 0 2024-04-23T02:14:05.604 INFO:tasks.workunit.client.0.smithi071.stdout:4/691: creat d2/d2a/daa/fe0 x:0 0 0 2024-04-23T02:14:05.609 INFO:tasks.workunit.client.1.smithi192.stdout:6/761: write d3/d15/d1b/f77 [3646738,106959] 0 2024-04-23T02:14:05.610 INFO:tasks.workunit.client.1.smithi192.stdout:6/762: chown d3/d18/d58/fcd 6 1 2024-04-23T02:14:05.612 INFO:tasks.workunit.client.1.smithi192.stdout:6/763: write d3/d28/f56 [4684460,80122] 0 2024-04-23T02:14:05.612 INFO:tasks.workunit.client.0.smithi071.stdout:4/692: dwrite d2/d2a/da2/dd1/dde/d57/f72 [0,4194304] 0 2024-04-23T02:14:05.614 INFO:tasks.workunit.client.0.smithi071.stdout:4/693: write d2/d2a/da2/dd1/dde/d23/d4f/fd7 [438650,29561] 0 2024-04-23T02:14:05.616 INFO:tasks.workunit.client.0.smithi071.stdout:0/595: dread d0/d3/de/d10/d14/d32/dac/fae [0,4194304] 0 2024-04-23T02:14:05.618 INFO:tasks.workunit.client.1.smithi192.stdout:6/764: rename d3/d15/d1b/d66 to d3/d28/d79/da0/dc2/d101 0 2024-04-23T02:14:05.620 INFO:tasks.workunit.client.0.smithi071.stdout:0/596: unlink d0/d3/de/d48/d7e/d8b/f21 0 2024-04-23T02:14:05.620 INFO:tasks.workunit.client.1.smithi192.stdout:6/765: stat d3/d28/d79/da0/dc2/d101/d80/d8e/d9d/cdb 0 2024-04-23T02:14:05.623 INFO:tasks.workunit.client.1.smithi192.stdout:0/785: sync 2024-04-23T02:14:05.623 INFO:tasks.workunit.client.1.smithi192.stdout:3/618: sync 2024-04-23T02:14:05.624 INFO:tasks.workunit.client.0.smithi071.stdout:8/655: dread dc/d15/da0/d30/fa6 [0,4194304] 0 2024-04-23T02:14:05.625 INFO:tasks.workunit.client.1.smithi192.stdout:3/619: write d1/d3/d6/d7/d71/f95 [224113,47296] 0 2024-04-23T02:14:05.626 INFO:tasks.workunit.client.0.smithi071.stdout:8/656: creat dc/d3a/da4/db0/fd2 x:0 0 0 2024-04-23T02:14:05.627 INFO:tasks.workunit.client.1.smithi192.stdout:0/786: mkdir d6/df/d13/d25/d107 0 2024-04-23T02:14:05.627 INFO:tasks.workunit.client.1.smithi192.stdout:3/620: dread - d1/d3/d17/d2b/d44/d57/db0/fdb zero size 2024-04-23T02:14:05.628 INFO:tasks.workunit.client.0.smithi071.stdout:8/657: mkdir dc/d3a/da4/dd3 0 2024-04-23T02:14:05.629 INFO:tasks.workunit.client.1.smithi192.stdout:0/787: mknod d6/df/d35/d8b/d8d/dea/c108 0 2024-04-23T02:14:05.629 INFO:tasks.workunit.client.0.smithi071.stdout:4/694: dread d2/d2a/d2d/d39/d42/d49/d67/f1e [0,4194304] 0 2024-04-23T02:14:05.631 INFO:tasks.workunit.client.1.smithi192.stdout:6/766: dread d3/d11/f43 [0,4194304] 0 2024-04-23T02:14:05.632 INFO:tasks.workunit.client.1.smithi192.stdout:0/788: symlink d6/df/d13/d25/d3a/l109 0 2024-04-23T02:14:05.632 INFO:tasks.workunit.client.0.smithi071.stdout:4/695: symlink d2/d2a/da2/dd1/dde/d23/d10/dbb/le1 0 2024-04-23T02:14:05.637 INFO:tasks.workunit.client.1.smithi192.stdout:3/621: dwrite d1/d3/d17/d1a/d7c/dab/d6c/d84/d46/fbc [0,4194304] 0 2024-04-23T02:14:05.647 INFO:tasks.workunit.client.0.smithi071.stdout:4/696: write d2/d2a/d73/f7f [1053903,118001] 0 2024-04-23T02:14:05.648 INFO:tasks.workunit.client.0.smithi071.stdout:4/697: write d2/d2a/da2/dd1/dde/d23/d10/f87 [517878,103334] 0 2024-04-23T02:14:05.648 INFO:tasks.workunit.client.0.smithi071.stdout:0/597: dread d0/d3/de/d10/d14/f1e [0,4194304] 0 2024-04-23T02:14:05.648 INFO:tasks.workunit.client.1.smithi192.stdout:3/622: dread d1/d3/d17/f6e [0,4194304] 0 2024-04-23T02:14:05.649 INFO:tasks.workunit.client.0.smithi071.stdout:0/598: write d0/f20 [232312,58375] 0 2024-04-23T02:14:05.651 INFO:tasks.workunit.client.0.smithi071.stdout:0/599: chown d0/d3/de/d48/d7e 0 1 2024-04-23T02:14:05.653 INFO:tasks.workunit.client.0.smithi071.stdout:0/600: rmdir d0/d3/d95 39 2024-04-23T02:14:05.659 INFO:tasks.workunit.client.0.smithi071.stdout:0/601: rename d0/d3/de/d10/d14/f1e to d0/d3/d95/fcd 0 2024-04-23T02:14:05.659 INFO:tasks.workunit.client.0.smithi071.stdout:4/698: truncate d2/d2a/d2d/d39/d42/d49/d67/d7e/dae/f9d 284468 0 2024-04-23T02:14:05.661 INFO:tasks.workunit.client.0.smithi071.stdout:4/699: chown d2/d2a/da2/dd1/dde/d23/d29/f9e 15409572 1 2024-04-23T02:14:05.661 INFO:tasks.workunit.client.1.smithi192.stdout:6/767: dread d3/d28/d79/da0/dc2/d101/d80/fb9 [0,4194304] 0 2024-04-23T02:14:05.662 INFO:tasks.workunit.client.0.smithi071.stdout:4/700: stat d2/d2a/da2/dd1/dde/l7a 0 2024-04-23T02:14:05.663 INFO:tasks.workunit.client.1.smithi192.stdout:0/789: dread d6/df/d13/d25/d3a/f58 [0,4194304] 0 2024-04-23T02:14:05.663 INFO:tasks.workunit.client.1.smithi192.stdout:6/768: readlink d3/d28/d79/da0/dc2/d101/d80/la3 0 2024-04-23T02:14:05.663 INFO:tasks.workunit.client.0.smithi071.stdout:4/701: write d2/d2a/daa/fe0 [894685,118025] 0 2024-04-23T02:14:05.664 INFO:tasks.workunit.client.0.smithi071.stdout:4/702: readlink d2/d2a/da2/dd1/dde/l7a 0 2024-04-23T02:14:05.667 INFO:tasks.workunit.client.1.smithi192.stdout:0/790: mknod d6/c10a 0 2024-04-23T02:14:05.668 INFO:tasks.workunit.client.1.smithi192.stdout:6/769: creat d3/d28/df8/d7c/f102 x:0 0 0 2024-04-23T02:14:05.670 INFO:tasks.workunit.client.1.smithi192.stdout:0/791: unlink d6/df/d56/d85/da4/lde 0 2024-04-23T02:14:05.672 INFO:tasks.workunit.client.0.smithi071.stdout:4/703: truncate d2/d2a/da2/dd1/dde/d2c/f7c 1051283 0 2024-04-23T02:14:05.673 INFO:tasks.workunit.client.1.smithi192.stdout:0/792: chown d6/df/d13/d25/d94/le1 312 1 2024-04-23T02:14:05.673 INFO:tasks.workunit.client.0.smithi071.stdout:0/602: dwrite d0/d3/de/d10/d14/d32/d37/d26/f2e [0,4194304] 0 2024-04-23T02:14:05.674 INFO:tasks.workunit.client.1.smithi192.stdout:6/770: dwrite d3/d15/d1b/da2/fc3 [0,4194304] 0 2024-04-23T02:14:05.674 INFO:tasks.workunit.client.1.smithi192.stdout:0/793: chown d6/df/d13/d25/d26/d67/cca 2 1 2024-04-23T02:14:05.676 INFO:tasks.workunit.client.0.smithi071.stdout:0/603: creat d0/d92/fce x:0 0 0 2024-04-23T02:14:05.678 INFO:tasks.workunit.client.1.smithi192.stdout:6/771: dread - d3/d28/d79/da0/dc2/d101/d80/d8e/d9d/ff7 zero size 2024-04-23T02:14:05.678 INFO:tasks.workunit.client.0.smithi071.stdout:4/704: symlink d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/dc8/le2 0 2024-04-23T02:14:05.680 INFO:tasks.workunit.client.1.smithi192.stdout:6/772: write d3/d28/df8/d52/db2/ffe [529131,99998] 0 2024-04-23T02:14:05.681 INFO:tasks.workunit.client.1.smithi192.stdout:6/773: stat d3/d11/f6b 0 2024-04-23T02:14:05.683 INFO:tasks.workunit.client.1.smithi192.stdout:6/774: fsync d3/d28/f56 0 2024-04-23T02:14:05.686 INFO:tasks.workunit.client.1.smithi192.stdout:6/775: creat d3/d28/df8/f103 x:0 0 0 2024-04-23T02:14:05.690 INFO:tasks.workunit.client.1.smithi192.stdout:6/776: mknod d3/d28/d79/da0/dc2/d101/d80/de2/c104 0 2024-04-23T02:14:05.690 INFO:tasks.workunit.client.1.smithi192.stdout:6/777: chown d3/d15/d5b 0 1 2024-04-23T02:14:05.690 INFO:tasks.workunit.client.1.smithi192.stdout:6/778: readlink d3/d15/d5b/lad 0 2024-04-23T02:14:05.692 INFO:tasks.workunit.client.0.smithi071.stdout:0/604: dread d0/f20 [0,4194304] 0 2024-04-23T02:14:05.692 INFO:tasks.workunit.client.1.smithi192.stdout:6/779: fdatasync d3/d28/df8/d52/db2/fbf 0 2024-04-23T02:14:05.703 INFO:tasks.workunit.client.1.smithi192.stdout:6/780: truncate d3/d28/d79/dbe/dcb/fd5 1670860 0 2024-04-23T02:14:05.704 INFO:tasks.workunit.client.1.smithi192.stdout:0/794: dread d6/df/fc0 [0,4194304] 0 2024-04-23T02:14:05.706 INFO:tasks.workunit.client.1.smithi192.stdout:6/781: mknod d3/d28/d79/dbe/dcb/c105 0 2024-04-23T02:14:05.706 INFO:tasks.workunit.client.1.smithi192.stdout:0/795: dread - d6/df/d47/f92 zero size 2024-04-23T02:14:05.707 INFO:tasks.workunit.client.1.smithi192.stdout:6/782: write d3/f2b [7634865,40457] 0 2024-04-23T02:14:05.708 INFO:tasks.workunit.client.1.smithi192.stdout:0/796: dread - d6/df/d13/d25/d94/dd0/fd4 zero size 2024-04-23T02:14:05.708 INFO:tasks.workunit.client.1.smithi192.stdout:6/783: chown d3/d28/d79/da0/dc2/lea 51 1 2024-04-23T02:14:05.712 INFO:tasks.workunit.client.1.smithi192.stdout:6/784: dread d3/d18/d58/fcd [0,4194304] 0 2024-04-23T02:14:05.715 INFO:tasks.workunit.client.1.smithi192.stdout:0/797: write d6/f18 [1433721,66552] 0 2024-04-23T02:14:05.721 INFO:tasks.workunit.client.1.smithi192.stdout:6/785: dwrite d3/d15/f4b [0,4194304] 0 2024-04-23T02:14:05.725 INFO:tasks.workunit.client.1.smithi192.stdout:0/798: dwrite d6/d4b/f6c [0,4194304] 0 2024-04-23T02:14:05.726 INFO:tasks.workunit.client.1.smithi192.stdout:6/786: mkdir d3/d28/d79/da0/d106 0 2024-04-23T02:14:05.727 INFO:tasks.workunit.client.1.smithi192.stdout:6/787: chown d3/d15/d1b/l74 87424129 1 2024-04-23T02:14:05.730 INFO:tasks.workunit.client.1.smithi192.stdout:6/788: dread - d3/d28/d79/da0/dc2/d101/d80/dbb/fd9 zero size 2024-04-23T02:14:05.732 INFO:tasks.workunit.client.1.smithi192.stdout:6/789: write d3/d28/df8/d52/db2/ffe [1058759,93563] 0 2024-04-23T02:14:05.740 INFO:tasks.workunit.client.1.smithi192.stdout:6/790: dwrite d3/d28/d79/da0/dc2/d101/d80/f8b [0,4194304] 0 2024-04-23T02:14:05.741 INFO:tasks.workunit.client.1.smithi192.stdout:6/791: write d3/d28/d79/da0/dc2/d101/d80/fdd [719060,35558] 0 2024-04-23T02:14:05.743 INFO:tasks.workunit.client.1.smithi192.stdout:6/792: readlink d3/d28/d79/da0/dc2/d101/d80/d8e/le0 0 2024-04-23T02:14:05.744 INFO:tasks.workunit.client.1.smithi192.stdout:6/793: chown d3/d28/d79/da0/dc2/d101/d80/fb9 1025626039 1 2024-04-23T02:14:05.747 INFO:tasks.workunit.client.1.smithi192.stdout:6/794: mkdir d3/d28/d79/da0/dc2/d101/d80/d107 0 2024-04-23T02:14:05.749 INFO:tasks.workunit.client.1.smithi192.stdout:6/795: symlink d3/d28/d79/da0/dc2/l108 0 2024-04-23T02:14:05.754 INFO:tasks.workunit.client.1.smithi192.stdout:0/799: dread d6/df/d35/d49/f54 [4194304,4194304] 0 2024-04-23T02:14:05.754 INFO:tasks.workunit.client.1.smithi192.stdout:6/796: dread d3/d28/d79/da0/dc2/d101/d80/fb9 [4194304,4194304] 0 2024-04-23T02:14:05.756 INFO:tasks.workunit.client.1.smithi192.stdout:6/797: fdatasync d3/d28/d79/da0/dc2/d101/d80/dbb/fd9 0 2024-04-23T02:14:05.757 INFO:tasks.workunit.client.0.smithi071.stdout:9/603: sync 2024-04-23T02:14:05.757 INFO:tasks.workunit.client.0.smithi071.stdout:7/720: sync 2024-04-23T02:14:05.758 INFO:tasks.workunit.client.0.smithi071.stdout:9/604: stat d1/d4e/d54 0 2024-04-23T02:14:05.758 INFO:tasks.workunit.client.1.smithi192.stdout:0/800: truncate d6/df/d56/d85/d71/f84 40493 0 2024-04-23T02:14:05.759 INFO:tasks.workunit.client.1.smithi192.stdout:6/798: fdatasync d3/f12 0 2024-04-23T02:14:05.760 INFO:tasks.workunit.client.1.smithi192.stdout:0/801: write d6/df/d35/d49/fc9 [1302950,41440] 0 2024-04-23T02:14:05.763 INFO:tasks.workunit.client.1.smithi192.stdout:2/693: sync 2024-04-23T02:14:05.763 INFO:tasks.workunit.client.1.smithi192.stdout:8/903: sync 2024-04-23T02:14:05.763 INFO:tasks.workunit.client.1.smithi192.stdout:7/727: sync 2024-04-23T02:14:05.763 INFO:tasks.workunit.client.1.smithi192.stdout:4/779: sync 2024-04-23T02:14:05.763 INFO:tasks.workunit.client.1.smithi192.stdout:9/588: sync 2024-04-23T02:14:05.764 INFO:tasks.workunit.client.1.smithi192.stdout:6/799: creat d3/d28/d79/da0/dc2/f109 x:0 0 0 2024-04-23T02:14:05.764 INFO:tasks.workunit.client.1.smithi192.stdout:8/904: readlink d1/l128 0 2024-04-23T02:14:05.767 INFO:tasks.workunit.client.1.smithi192.stdout:2/694: symlink d8/d26/d81/le7 0 2024-04-23T02:14:05.770 INFO:tasks.workunit.client.1.smithi192.stdout:7/728: unlink d5/d17/d1c/d59/d5a/d41/d50/d9f/fce 0 2024-04-23T02:14:05.773 INFO:tasks.workunit.client.1.smithi192.stdout:0/802: dwrite d6/fb5 [0,4194304] 0 2024-04-23T02:14:05.775 INFO:tasks.workunit.client.1.smithi192.stdout:8/905: write d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/da3/d114/f111 [798466,19680] 0 2024-04-23T02:14:05.775 INFO:tasks.workunit.client.1.smithi192.stdout:4/780: symlink de/d15/d53/d81/de3/d74/d92/da0/dca/l100 0 2024-04-23T02:14:05.777 INFO:tasks.workunit.client.1.smithi192.stdout:2/695: creat d8/d26/d29/d43/d50/fe8 x:0 0 0 2024-04-23T02:14:05.777 INFO:tasks.workunit.client.1.smithi192.stdout:9/589: truncate d0/d9/d44/d51/d5b/d61/db1/f8d 3212365 0 2024-04-23T02:14:05.778 INFO:tasks.workunit.client.1.smithi192.stdout:7/729: creat d5/d43/d4c/fd8 x:0 0 0 2024-04-23T02:14:05.779 INFO:tasks.workunit.client.1.smithi192.stdout:4/781: fsync f4 0 2024-04-23T02:14:05.779 INFO:tasks.workunit.client.1.smithi192.stdout:6/800: dwrite f1 [4194304,4194304] 0 2024-04-23T02:14:05.781 INFO:tasks.workunit.client.1.smithi192.stdout:8/906: read - d1/d12/d23/d4b/d73/d45/f12e zero size 2024-04-23T02:14:05.782 INFO:tasks.workunit.client.1.smithi192.stdout:5/694: sync 2024-04-23T02:14:05.782 INFO:tasks.workunit.client.1.smithi192.stdout:3/623: sync 2024-04-23T02:14:05.782 INFO:tasks.workunit.client.1.smithi192.stdout:1/470: sync 2024-04-23T02:14:05.788 INFO:tasks.workunit.client.1.smithi192.stdout:2/696: creat d8/d26/d29/d43/d50/fe9 x:0 0 0 2024-04-23T02:14:05.789 INFO:tasks.workunit.client.0.smithi071.stdout:4/705: mknod d2/db9/ce3 0 2024-04-23T02:14:05.791 INFO:tasks.workunit.client.1.smithi192.stdout:1/471: unlink dc/d8b/d44/l5f 0 2024-04-23T02:14:05.792 INFO:tasks.workunit.client.1.smithi192.stdout:7/730: write d5/d17/d1c/d59/fd0 [609861,100509] 0 2024-04-23T02:14:05.792 INFO:tasks.workunit.client.0.smithi071.stdout:0/605: dread d0/d3/de/d10/f57 [0,4194304] 0 2024-04-23T02:14:05.792 INFO:tasks.workunit.client.0.smithi071.stdout:9/605: symlink d1/d32/d41/db7/d71/ld3 0 2024-04-23T02:14:05.793 INFO:tasks.workunit.client.1.smithi192.stdout:0/803: creat d6/df/d13/d25/d94/dd0/de6/dfa/f10b x:0 0 0 2024-04-23T02:14:05.795 INFO:tasks.workunit.client.0.smithi071.stdout:9/606: dread - d1/d32/d3a/d90/d99/fcf zero size 2024-04-23T02:14:05.798 INFO:tasks.workunit.client.1.smithi192.stdout:0/804: stat d6/d4b/f79 0 2024-04-23T02:14:05.798 INFO:tasks.workunit.client.1.smithi192.stdout:2/697: rename d8/d26/l72 to d8/d26/d27/d31/dae/lea 0 2024-04-23T02:14:05.800 INFO:tasks.workunit.client.1.smithi192.stdout:3/624: dwrite d1/d3/d17/d1f/d33/f8d [0,4194304] 0 2024-04-23T02:14:05.801 INFO:tasks.workunit.client.1.smithi192.stdout:2/698: readlink d8/d25/l6e 0 2024-04-23T02:14:05.802 INFO:tasks.workunit.client.1.smithi192.stdout:1/472: truncate dc/d14/d15/d35/f73 185174 0 2024-04-23T02:14:05.804 INFO:tasks.workunit.client.1.smithi192.stdout:9/590: dwrite d0/d9/d44/d51/d5b/d61/fb9 [0,4194304] 0 2024-04-23T02:14:05.806 INFO:tasks.workunit.client.1.smithi192.stdout:6/801: creat d3/d28/d79/da0/dc2/d101/d80/d8e/d9d/ded/d100/f10a x:0 0 0 2024-04-23T02:14:05.810 INFO:tasks.workunit.client.1.smithi192.stdout:2/699: chown d8/d26/d81/lc4 33664 1 2024-04-23T02:14:05.811 INFO:tasks.workunit.client.1.smithi192.stdout:1/473: unlink dc/d16/d19/d30/f82 0 2024-04-23T02:14:05.815 INFO:tasks.workunit.client.1.smithi192.stdout:3/625: write d1/d3/d17/f3d [559148,124665] 0 2024-04-23T02:14:05.817 INFO:tasks.workunit.client.1.smithi192.stdout:0/805: dwrite d6/fda [0,4194304] 0 2024-04-23T02:14:05.821 INFO:tasks.workunit.client.1.smithi192.stdout:8/907: write d1/d12/d23/d4b/d73/d5d/fa6 [2888403,36242] 0 2024-04-23T02:14:05.829 INFO:tasks.workunit.client.1.smithi192.stdout:3/626: creat d1/d3/d17/fdc x:0 0 0 2024-04-23T02:14:05.831 INFO:tasks.workunit.client.1.smithi192.stdout:2/700: getdents d8/d26/d27/db5 0 2024-04-23T02:14:05.833 INFO:tasks.workunit.client.1.smithi192.stdout:4/782: dwrite de/d15/f2d [0,4194304] 0 2024-04-23T02:14:05.836 INFO:tasks.workunit.client.1.smithi192.stdout:0/806: rename d6/df/d56/d85/ff4 to d6/df/d56/d85/f10c 0 2024-04-23T02:14:05.838 INFO:tasks.workunit.client.1.smithi192.stdout:1/474: dread dc/d14/d23/f3d [0,4194304] 0 2024-04-23T02:14:05.838 INFO:tasks.workunit.client.1.smithi192.stdout:6/802: dread d3/d15/d1b/f77 [0,4194304] 0 2024-04-23T02:14:05.839 INFO:tasks.workunit.client.1.smithi192.stdout:3/627: creat d1/d3/d6/d7/d29/d3b/fdd x:0 0 0 2024-04-23T02:14:05.840 INFO:tasks.workunit.client.1.smithi192.stdout:2/701: mkdir d8/d26/d27/d57/d69/d99/deb 0 2024-04-23T02:14:05.841 INFO:tasks.workunit.client.1.smithi192.stdout:4/783: write de/d15/d53/d81/de3/d74/d92/da0/dca/f5f [2118851,97048] 0 2024-04-23T02:14:05.841 INFO:tasks.workunit.client.1.smithi192.stdout:3/628: readlink d1/d3/l74 0 2024-04-23T02:14:05.843 INFO:tasks.workunit.client.1.smithi192.stdout:0/807: unlink d6/df/d13/d25/d26/l81 0 2024-04-23T02:14:05.850 INFO:tasks.workunit.client.1.smithi192.stdout:8/908: getdents d1/d9/d8f 0 2024-04-23T02:14:05.850 INFO:tasks.workunit.client.1.smithi192.stdout:6/803: creat d3/d28/df8/d7c/f10b x:0 0 0 2024-04-23T02:14:05.851 INFO:tasks.workunit.client.1.smithi192.stdout:2/702: read d8/d26/f61 [1034109,122331] 0 2024-04-23T02:14:05.852 INFO:tasks.workunit.client.1.smithi192.stdout:9/591: dread d0/d9/d44/d4f/f3e [0,4194304] 0 2024-04-23T02:14:05.854 INFO:tasks.workunit.client.1.smithi192.stdout:0/808: dread d6/f68 [0,4194304] 0 2024-04-23T02:14:05.855 INFO:tasks.workunit.client.1.smithi192.stdout:2/703: write d8/f96 [2068002,49112] 0 2024-04-23T02:14:05.856 INFO:tasks.workunit.client.1.smithi192.stdout:8/909: write d1/d12/d23/d4b/d73/d4c/f64 [3261429,70159] 0 2024-04-23T02:14:05.858 INFO:tasks.workunit.client.1.smithi192.stdout:9/592: fsync d0/d9/d44/d4f/fb2 0 2024-04-23T02:14:05.860 INFO:tasks.workunit.client.1.smithi192.stdout:6/804: creat d3/f10c x:0 0 0 2024-04-23T02:14:05.861 INFO:tasks.workunit.client.1.smithi192.stdout:4/784: rename de/d15/d53/d81/de3/d74/d92/da0/dca/d34/d4a to de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/de7/d101 0 2024-04-23T02:14:05.861 INFO:tasks.workunit.client.1.smithi192.stdout:2/704: readlink d8/d26/la5 0 2024-04-23T02:14:05.862 INFO:tasks.workunit.client.1.smithi192.stdout:3/629: dwrite d1/d3/d17/d1f/fce [0,4194304] 0 2024-04-23T02:14:05.863 INFO:tasks.workunit.client.1.smithi192.stdout:9/593: unlink d0/l2f 0 2024-04-23T02:14:05.864 INFO:tasks.workunit.client.1.smithi192.stdout:2/705: fdatasync d8/d26/d29/da4/dca/fd4 0 2024-04-23T02:14:05.868 INFO:tasks.workunit.client.1.smithi192.stdout:0/809: rename d6/df/d13/d6b/f8c to d6/df/d13/d25/d26/f10d 0 2024-04-23T02:14:05.869 INFO:tasks.workunit.client.1.smithi192.stdout:1/475: dwrite dc/d14/f27 [0,4194304] 0 2024-04-23T02:14:05.871 INFO:tasks.workunit.client.1.smithi192.stdout:1/476: write dc/d16/d19/f79 [404419,86784] 0 2024-04-23T02:14:05.871 INFO:tasks.workunit.client.1.smithi192.stdout:9/594: readlink d0/l24 0 2024-04-23T02:14:05.873 INFO:tasks.workunit.client.1.smithi192.stdout:8/910: creat d1/d12/d23/d4b/d73/dda/d113/f140 x:0 0 0 2024-04-23T02:14:05.874 INFO:tasks.workunit.client.1.smithi192.stdout:9/595: chown d0/d9/d44/d4f/f8 2948108 1 2024-04-23T02:14:05.875 INFO:tasks.workunit.client.1.smithi192.stdout:8/911: chown d1/d12/d23/d4b/d73/d45/d46/ffd 5535 1 2024-04-23T02:14:05.875 INFO:tasks.workunit.client.0.smithi071.stdout:7/721: getdents d3 0 2024-04-23T02:14:05.876 INFO:tasks.workunit.client.1.smithi192.stdout:4/785: write de/d15/d53/d81/fa6 [194894,80043] 0 2024-04-23T02:14:05.876 INFO:tasks.workunit.client.1.smithi192.stdout:8/912: chown d1/d12/d68/ddf/c138 14287 1 2024-04-23T02:14:05.878 INFO:tasks.workunit.client.1.smithi192.stdout:6/805: unlink d3/d15/d1b/c24 0 2024-04-23T02:14:05.878 INFO:tasks.workunit.client.0.smithi071.stdout:7/722: dread - d3/d17/d1c/fee zero size 2024-04-23T02:14:05.880 INFO:tasks.workunit.client.0.smithi071.stdout:0/606: read d0/d3/de/d10/d14/d32/d44/d31/d52/f5c [3682244,21021] 0 2024-04-23T02:14:05.880 INFO:tasks.workunit.client.1.smithi192.stdout:1/477: creat dc/d16/d19/d74/d77/f91 x:0 0 0 2024-04-23T02:14:05.881 INFO:tasks.workunit.client.1.smithi192.stdout:6/806: chown d3/d28/d79/da0/dc2/d101/d80/d8e/f50 3033194 1 2024-04-23T02:14:05.881 INFO:tasks.workunit.client.1.smithi192.stdout:3/630: symlink d1/d3/d6/lde 0 2024-04-23T02:14:05.882 INFO:tasks.workunit.client.1.smithi192.stdout:1/478: stat dc/d16/d19/l7b 0 2024-04-23T02:14:05.882 INFO:tasks.workunit.client.0.smithi071.stdout:9/607: symlink d1/d32/d41/db7/d5a/dc6/d9f/ld4 0 2024-04-23T02:14:05.883 INFO:tasks.workunit.client.1.smithi192.stdout:9/596: creat d0/d9/db3/d5f/fc8 x:0 0 0 2024-04-23T02:14:05.884 INFO:tasks.workunit.client.1.smithi192.stdout:2/706: mkdir d8/d26/d29/d43/dc2/dec 0 2024-04-23T02:14:05.884 INFO:tasks.workunit.client.1.smithi192.stdout:3/631: stat d1/d3/d17/d65/l7e 0 2024-04-23T02:14:05.884 INFO:tasks.workunit.client.0.smithi071.stdout:4/706: truncate d2/d2a/da2/dd1/dde/d57/f72 1155138 0 2024-04-23T02:14:05.885 INFO:tasks.workunit.client.0.smithi071.stdout:0/607: creat d0/d3/de/d10/d14/d32/d44/d87/d93/fcf x:0 0 0 2024-04-23T02:14:05.886 INFO:tasks.workunit.client.0.smithi071.stdout:4/707: write d2/d2a/da2/dd1/dde/d23/f8e [2365994,37800] 0 2024-04-23T02:14:05.886 INFO:tasks.workunit.client.1.smithi192.stdout:1/479: fdatasync dc/d8b/d34/d70/f85 0 2024-04-23T02:14:05.887 INFO:tasks.workunit.client.0.smithi071.stdout:9/608: mknod d1/d32/d3a/d5f/cd5 0 2024-04-23T02:14:05.888 INFO:tasks.workunit.client.0.smithi071.stdout:4/708: write d2/d2a/d73/d5e/fb0 [977009,17474] 0 2024-04-23T02:14:05.888 INFO:tasks.workunit.client.1.smithi192.stdout:2/707: fdatasync d8/d26/d74/fd1 0 2024-04-23T02:14:05.889 INFO:tasks.workunit.client.0.smithi071.stdout:0/608: creat d0/d3/de/d10/d14/d84/fd0 x:0 0 0 2024-04-23T02:14:05.889 INFO:tasks.workunit.client.1.smithi192.stdout:1/480: write dc/fd [11795454,119575] 0 2024-04-23T02:14:05.890 INFO:tasks.workunit.client.0.smithi071.stdout:9/609: dread - d1/d32/d41/db7/d39/d42/dcc/fcd zero size 2024-04-23T02:14:05.890 INFO:tasks.workunit.client.0.smithi071.stdout:4/709: readlink d2/d2a/da2/dd1/dde/d23/d29/l95 0 2024-04-23T02:14:05.891 INFO:tasks.workunit.client.1.smithi192.stdout:1/481: fdatasync dc/d8b/d34/f56 0 2024-04-23T02:14:05.891 INFO:tasks.workunit.client.1.smithi192.stdout:0/810: creat d6/df/d13/d25/f10e x:0 0 0 2024-04-23T02:14:05.892 INFO:tasks.workunit.client.0.smithi071.stdout:0/609: creat d0/d3/de/d10/d14/d84/fd1 x:0 0 0 2024-04-23T02:14:05.892 INFO:tasks.workunit.client.1.smithi192.stdout:0/811: chown d6/fb5 14359909 1 2024-04-23T02:14:05.893 INFO:tasks.workunit.client.1.smithi192.stdout:0/812: dread - d6/df/d56/d85/f10c zero size 2024-04-23T02:14:05.894 INFO:tasks.workunit.client.0.smithi071.stdout:0/610: dread - d0/d3/de/d10/d14/f85 zero size 2024-04-23T02:14:05.895 INFO:tasks.workunit.client.0.smithi071.stdout:4/710: mknod d2/d2a/d2d/d39/d42/d49/d67/ce4 0 2024-04-23T02:14:05.896 INFO:tasks.workunit.client.0.smithi071.stdout:0/611: mknod d0/d76/cd2 0 2024-04-23T02:14:05.896 INFO:tasks.workunit.client.1.smithi192.stdout:2/708: unlink d8/d26/d27/c94 0 2024-04-23T02:14:05.898 INFO:tasks.workunit.client.1.smithi192.stdout:4/786: write de/f18 [1337317,49438] 0 2024-04-23T02:14:05.898 INFO:tasks.workunit.client.0.smithi071.stdout:4/711: write d2/d2a/d2d/d39/d42/d49/d67/f1e [1105127,34955] 0 2024-04-23T02:14:05.899 INFO:tasks.workunit.client.0.smithi071.stdout:9/610: link d1/d32/d41/db7/c2e d1/d32/d3a/d5f/cd6 0 2024-04-23T02:14:05.899 INFO:tasks.workunit.client.1.smithi192.stdout:3/632: rename d1/d3/d17/d1a/d7c/dab/d6c/d84/d46/d94 to d1/d3/d6/d7/d29/d3b/da9/daf/ddf 0 2024-04-23T02:14:05.901 INFO:tasks.workunit.client.0.smithi071.stdout:4/712: readlink d2/d2a/da2/dd1/dde/d23/d10/dbb/le1 0 2024-04-23T02:14:05.901 INFO:tasks.workunit.client.0.smithi071.stdout:9/611: chown d1/d32/dc5/dd1/f9d 40243 1 2024-04-23T02:14:05.902 INFO:tasks.workunit.client.0.smithi071.stdout:9/612: stat d1/d32/d3a/cae 0 2024-04-23T02:14:05.903 INFO:tasks.workunit.client.1.smithi192.stdout:1/482: symlink dc/d14/d15/d35/l92 0 2024-04-23T02:14:05.903 INFO:tasks.workunit.client.0.smithi071.stdout:0/612: mknod d0/d3/de/d10/d14/d32/d44/cd3 0 2024-04-23T02:14:05.904 INFO:tasks.workunit.client.1.smithi192.stdout:6/807: creat d3/d28/d79/da0/dc2/d101/d80/f10d x:0 0 0 2024-04-23T02:14:05.905 INFO:tasks.workunit.client.0.smithi071.stdout:9/613: mkdir d1/d4e/d66/da4/dd7 0 2024-04-23T02:14:05.905 INFO:tasks.workunit.client.1.smithi192.stdout:8/913: dwrite d1/d12/d23/d4b/d73/d7e/fe7 [0,4194304] 0 2024-04-23T02:14:05.905 INFO:tasks.workunit.client.0.smithi071.stdout:4/713: dread - d2/d2a/da2/dd1/dde/d23/d10/f60 zero size 2024-04-23T02:14:05.906 INFO:tasks.workunit.client.1.smithi192.stdout:1/483: rename dc to dc/d14/d23/d5e/d93 22 2024-04-23T02:14:05.906 INFO:tasks.workunit.client.1.smithi192.stdout:6/808: fsync d3/d18/dd4/d8d/fba 0 2024-04-23T02:14:05.908 INFO:tasks.workunit.client.1.smithi192.stdout:2/709: fdatasync d8/d26/d29/d43/d50/f76 0 2024-04-23T02:14:05.908 INFO:tasks.workunit.client.0.smithi071.stdout:9/614: mkdir d1/d32/dc5/dd8 0 2024-04-23T02:14:05.909 INFO:tasks.workunit.client.1.smithi192.stdout:6/809: dread - d3/feb zero size 2024-04-23T02:14:05.910 INFO:tasks.workunit.client.0.smithi071.stdout:9/615: write d1/d32/d41/db7/f70 [1626482,49315] 0 2024-04-23T02:14:05.910 INFO:tasks.workunit.client.0.smithi071.stdout:4/714: mkdir d2/d2a/da2/dd1/dde/d57/de5 0 2024-04-23T02:14:05.911 INFO:tasks.workunit.client.1.smithi192.stdout:6/810: write d3/d18/dd4/ffb [762009,89297] 0 2024-04-23T02:14:05.912 INFO:tasks.workunit.client.0.smithi071.stdout:4/715: write d2/d53/f88 [3848759,95659] 0 2024-04-23T02:14:05.913 INFO:tasks.workunit.client.0.smithi071.stdout:0/613: rename d0/d3/de/d10/d16/f97 to d0/d3/de/d10/d14/d32/d44/d31/fd4 0 2024-04-23T02:14:05.916 INFO:tasks.workunit.client.1.smithi192.stdout:6/811: read d3/d28/df8/d7c/dda/fef [452035,110042] 0 2024-04-23T02:14:05.916 INFO:tasks.workunit.client.1.smithi192.stdout:4/787: rename de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/d59/l9e to de/d15/d53/d81/de3/d80/l102 0 2024-04-23T02:14:05.917 INFO:tasks.workunit.client.0.smithi071.stdout:7/723: dread d3/d17/d1c/f20 [0,4194304] 0 2024-04-23T02:14:05.917 INFO:tasks.workunit.client.0.smithi071.stdout:9/616: rename d1/d32/d41/c85 to d1/d32/d41/db7/d5a/dc6/cd9 0 2024-04-23T02:14:05.918 INFO:tasks.workunit.client.0.smithi071.stdout:4/716: truncate d2/d2a/da2/dd1/dde/d23/d4f/db5/db6/fd6 679772 0 2024-04-23T02:14:05.919 INFO:tasks.workunit.client.0.smithi071.stdout:0/614: chown d0/d3/de/d48/d7e/d8b/f22 18077634 1 2024-04-23T02:14:05.920 INFO:tasks.workunit.client.0.smithi071.stdout:7/724: chown d3/d17/d1c/d31/f5d 15759777 1 2024-04-23T02:14:05.921 INFO:tasks.workunit.client.1.smithi192.stdout:4/788: fsync de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/fae 0 2024-04-23T02:14:05.923 INFO:tasks.workunit.client.1.smithi192.stdout:8/914: rmdir d1/d12/d23 39 2024-04-23T02:14:05.924 INFO:tasks.workunit.client.0.smithi071.stdout:4/717: dread d2/d2a/da2/dd1/dde/d23/d29/f83 [0,4194304] 0 2024-04-23T02:14:05.925 INFO:tasks.workunit.client.0.smithi071.stdout:0/615: mkdir d0/d3/de/d10/d14/dd5 0 2024-04-23T02:14:05.926 INFO:tasks.workunit.client.0.smithi071.stdout:0/616: chown d0/l7 2 1 2024-04-23T02:14:05.926 INFO:tasks.workunit.client.1.smithi192.stdout:6/812: dwrite d3/d28/d79/da0/dc2/d101/d80/fdd [0,4194304] 0 2024-04-23T02:14:05.926 INFO:tasks.workunit.client.0.smithi071.stdout:9/617: link d1/d4e/d54/c58 d1/d32/d41/db7/d5a/cda 0 2024-04-23T02:14:05.928 INFO:tasks.workunit.client.1.smithi192.stdout:9/597: getdents d0/d9/d44/d58/d89 0 2024-04-23T02:14:05.933 INFO:tasks.workunit.client.0.smithi071.stdout:6/589: sync 2024-04-23T02:14:05.933 INFO:tasks.workunit.client.0.smithi071.stdout:1/552: sync 2024-04-23T02:14:05.933 INFO:tasks.workunit.client.0.smithi071.stdout:3/556: sync 2024-04-23T02:14:05.935 INFO:tasks.workunit.client.1.smithi192.stdout:2/710: dread d8/d14/f1a [0,4194304] 0 2024-04-23T02:14:05.938 INFO:tasks.workunit.client.1.smithi192.stdout:4/789: mknod de/d15/d53/d81/de3/db2/c103 0 2024-04-23T02:14:05.938 INFO:tasks.workunit.client.0.smithi071.stdout:3/557: fdatasync d7/da5/db2/db7/d68/f84 0 2024-04-23T02:14:05.939 INFO:tasks.workunit.client.1.smithi192.stdout:1/484: dread dc/d16/d19/f3a [0,4194304] 0 2024-04-23T02:14:05.939 INFO:tasks.workunit.client.1.smithi192.stdout:6/813: dwrite d3/d28/d79/da0/dc2/d101/d80/d8e/d9d/ff7 [0,4194304] 0 2024-04-23T02:14:05.940 INFO:tasks.workunit.client.1.smithi192.stdout:2/711: write d8/d26/d29/d43/f9f [467811,26877] 0 2024-04-23T02:14:05.940 INFO:tasks.workunit.client.0.smithi071.stdout:1/553: creat db/df/d13/d20/d75/fb1 x:0 0 0 2024-04-23T02:14:05.941 INFO:tasks.workunit.client.0.smithi071.stdout:3/558: write d7/d1d/f72 [8261766,18714] 0 2024-04-23T02:14:05.942 INFO:tasks.workunit.client.1.smithi192.stdout:3/633: getdents d1/d3/d17/d2b/d44/d57/db0 0 2024-04-23T02:14:05.943 INFO:tasks.workunit.client.0.smithi071.stdout:3/559: readlink d7/d10/l85 0 2024-04-23T02:14:05.946 INFO:tasks.workunit.client.0.smithi071.stdout:7/725: truncate d3/d17/d1c/f3f 278773 0 2024-04-23T02:14:05.947 INFO:tasks.workunit.client.1.smithi192.stdout:1/485: creat dc/d8b/d62/f94 x:0 0 0 2024-04-23T02:14:05.948 INFO:tasks.workunit.client.0.smithi071.stdout:9/618: write d1/f3d [1126062,60728] 0 2024-04-23T02:14:05.949 INFO:tasks.workunit.client.1.smithi192.stdout:4/790: stat de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/de7/d101 0 2024-04-23T02:14:05.949 INFO:tasks.workunit.client.0.smithi071.stdout:0/617: dwrite d0/d3/f62 [4194304,4194304] 0 2024-04-23T02:14:05.950 INFO:tasks.workunit.client.1.smithi192.stdout:1/486: write dc/d16/d19/f3a [193738,66170] 0 2024-04-23T02:14:05.951 INFO:tasks.workunit.client.0.smithi071.stdout:4/718: dread d2/d2a/d2d/d39/d42/d49/d67/f8f [0,4194304] 0 2024-04-23T02:14:05.952 INFO:tasks.workunit.client.0.smithi071.stdout:1/554: rename db/df/d13/d72 to db/df/d5f/d6f/db2 0 2024-04-23T02:14:05.952 INFO:tasks.workunit.client.0.smithi071.stdout:3/560: mknod d7/da5/db2/db7/cbf 0 2024-04-23T02:14:05.953 INFO:tasks.workunit.client.0.smithi071.stdout:7/726: truncate d3/fc 1432900 0 2024-04-23T02:14:05.954 INFO:tasks.workunit.client.1.smithi192.stdout:6/814: symlink d3/d15/d1b/l10e 0 2024-04-23T02:14:05.954 INFO:tasks.workunit.client.1.smithi192.stdout:2/712: mkdir d8/d26/d29/d43/ded 0 2024-04-23T02:14:05.955 INFO:tasks.workunit.client.0.smithi071.stdout:6/590: write d6/d17/d77/d85/fab [1823468,70443] 0 2024-04-23T02:14:05.959 INFO:tasks.workunit.client.1.smithi192.stdout:1/487: write dc/d16/d19/f79 [626572,49556] 0 2024-04-23T02:14:05.959 INFO:tasks.workunit.client.1.smithi192.stdout:2/713: rename d8 to d8/dee 22 2024-04-23T02:14:05.960 INFO:tasks.workunit.client.0.smithi071.stdout:6/591: dread - d6/f94 zero size 2024-04-23T02:14:05.960 INFO:tasks.workunit.client.1.smithi192.stdout:6/815: truncate d3/d28/d79/da0/dc2/f109 169064 0 2024-04-23T02:14:05.961 INFO:tasks.workunit.client.1.smithi192.stdout:1/488: unlink dc/d14/d15/d35/l92 0 2024-04-23T02:14:05.962 INFO:tasks.workunit.client.0.smithi071.stdout:1/555: unlink db/df/d13/d20/c8e 0 2024-04-23T02:14:05.962 INFO:tasks.workunit.client.1.smithi192.stdout:3/634: getdents d1/d3/d6/d7/d29/d3b/da9 0 2024-04-23T02:14:05.963 INFO:tasks.workunit.client.0.smithi071.stdout:0/618: creat d0/d3/de/d10/d14/d32/d44/fd6 x:0 0 0 2024-04-23T02:14:05.963 INFO:tasks.workunit.client.1.smithi192.stdout:2/714: fsync d8/d25/f9d 0 2024-04-23T02:14:05.966 INFO:tasks.workunit.client.1.smithi192.stdout:1/489: write dc/d14/d15/d35/f49 [1053767,48898] 0 2024-04-23T02:14:05.969 INFO:tasks.workunit.client.1.smithi192.stdout:3/635: stat d1/d3/d6/d7/d29 0 2024-04-23T02:14:05.969 INFO:tasks.workunit.client.0.smithi071.stdout:9/619: dwrite d1/d32/d3a/d90/d99/fcf [0,4194304] 0 2024-04-23T02:14:05.969 INFO:tasks.workunit.client.0.smithi071.stdout:1/556: chown db/df/d13/d20/f53 1324 1 2024-04-23T02:14:05.971 INFO:tasks.workunit.client.1.smithi192.stdout:8/915: dwrite d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/f7f [0,4194304] 0 2024-04-23T02:14:05.971 INFO:tasks.workunit.client.0.smithi071.stdout:4/719: dwrite d2/d2a/d73/f7f [0,4194304] 0 2024-04-23T02:14:05.972 INFO:tasks.workunit.client.1.smithi192.stdout:2/715: write d8/d26/d27/d57/d69/f8a [3696447,77812] 0 2024-04-23T02:14:05.972 INFO:tasks.workunit.client.0.smithi071.stdout:6/592: creat d6/d17/d77/d9f/dad/da4/fbd x:0 0 0 2024-04-23T02:14:05.973 INFO:tasks.workunit.client.0.smithi071.stdout:1/557: stat db/df/d5f/d6f/db2/d86/ca2 0 2024-04-23T02:14:05.975 INFO:tasks.workunit.client.0.smithi071.stdout:3/561: link d7/da5/db2/db7/d68/d3b/d4c/d8b/db3/c7d d7/d8/cc0 0 2024-04-23T02:14:05.977 INFO:tasks.workunit.client.0.smithi071.stdout:3/562: chown d7/d22/d23/f6b 287966536 1 2024-04-23T02:14:05.978 INFO:tasks.workunit.client.1.smithi192.stdout:4/791: truncate de/d15/d53/d81/de3/d74/d92/da0/dca/d34/fed 345506 0 2024-04-23T02:14:05.980 INFO:tasks.workunit.client.0.smithi071.stdout:2/589: sync 2024-04-23T02:14:05.980 INFO:tasks.workunit.client.0.smithi071.stdout:5/683: sync 2024-04-23T02:14:05.981 INFO:tasks.workunit.client.0.smithi071.stdout:0/619: rename d0/d3/c4 to d0/d3/de/d10/d14/cd7 0 2024-04-23T02:14:05.981 INFO:tasks.workunit.client.0.smithi071.stdout:1/558: symlink db/df/d13/d2f/d48/lb3 0 2024-04-23T02:14:05.981 INFO:tasks.workunit.client.1.smithi192.stdout:8/916: stat d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/d108/d11f/l13a 0 2024-04-23T02:14:05.981 INFO:tasks.workunit.client.1.smithi192.stdout:9/598: dwrite d0/d9/d44/d51/d5b/d61/db1/f93 [0,4194304] 0 2024-04-23T02:14:05.983 INFO:tasks.workunit.client.0.smithi071.stdout:2/590: chown d0/d2/d49/f5b 2569 1 2024-04-23T02:14:05.983 INFO:tasks.workunit.client.1.smithi192.stdout:4/792: dread - de/d15/d53/d81/de3/db2/fec zero size 2024-04-23T02:14:05.983 INFO:tasks.workunit.client.1.smithi192.stdout:8/917: readlink d1/d9/l3a 0 2024-04-23T02:14:05.984 INFO:tasks.workunit.client.1.smithi192.stdout:6/816: dread d3/d28/d79/da0/dc2/d101/d80/d8e/d9d/ff7 [0,4194304] 0 2024-04-23T02:14:05.984 INFO:tasks.workunit.client.1.smithi192.stdout:9/599: stat d0/d9/d44/d74/l99 0 2024-04-23T02:14:05.985 INFO:tasks.workunit.client.0.smithi071.stdout:7/727: dread d3/d17/d41/d34/d39/d3b/da7/faf [0,4194304] 0 2024-04-23T02:14:05.985 INFO:tasks.workunit.client.1.smithi192.stdout:8/918: fsync d1/d12/d23/d4b/f69 0 2024-04-23T02:14:05.986 INFO:tasks.workunit.client.0.smithi071.stdout:1/559: symlink db/df/d5f/d6f/lb4 0 2024-04-23T02:14:05.987 INFO:tasks.workunit.client.0.smithi071.stdout:2/591: chown d0/d2/f1d 8 1 2024-04-23T02:14:05.988 INFO:tasks.workunit.client.0.smithi071.stdout:4/720: link d2/d2a/d2d/d39/d42/d49/d91/cdf d2/d2a/d2d/d39/db3/ce6 0 2024-04-23T02:14:05.989 INFO:tasks.workunit.client.1.smithi192.stdout:4/793: chown de/d15/f2d 3694098 1 2024-04-23T02:14:05.990 INFO:tasks.workunit.client.0.smithi071.stdout:0/620: mknod d0/d3/de/d10/d14/d32/d44/d31/d52/cd8 0 2024-04-23T02:14:05.990 INFO:tasks.workunit.client.1.smithi192.stdout:9/600: dread - d0/d9/d44/d51/d5b/d61/faf zero size 2024-04-23T02:14:05.992 INFO:tasks.workunit.client.0.smithi071.stdout:7/728: unlink d3/d17/d41/d3a/da5/lb4 0 2024-04-23T02:14:05.996 INFO:tasks.workunit.client.0.smithi071.stdout:1/560: symlink db/df/d5f/d6f/db2/d86/lb5 0 2024-04-23T02:14:05.996 INFO:tasks.workunit.client.0.smithi071.stdout:2/592: symlink d0/d2/d26/d14/d1f/d36/d5c/lc0 0 2024-04-23T02:14:05.996 INFO:tasks.workunit.client.0.smithi071.stdout:0/621: symlink d0/d92/db2/ld9 0 2024-04-23T02:14:05.998 INFO:tasks.workunit.client.0.smithi071.stdout:1/561: fsync db/df/f2a 0 2024-04-23T02:14:06.001 INFO:tasks.workunit.client.0.smithi071.stdout:4/721: rename d2/d2a/da2/dd1/dde/d23/d4f/db5/db6/dd0 to d2/d2a/da2/dd1/dde/d23/d4f/db5/de7 0 2024-04-23T02:14:06.005 INFO:tasks.workunit.client.0.smithi071.stdout:0/622: creat d0/fda x:0 0 0 2024-04-23T02:14:06.006 INFO:tasks.workunit.client.0.smithi071.stdout:0/623: fdatasync d0/d3/de/d10/d14/f4d 0 2024-04-23T02:14:06.007 INFO:tasks.workunit.client.0.smithi071.stdout:2/593: getdents d0/d2/d26/d14/d1f/d4f/d98 0 2024-04-23T02:14:06.008 INFO:tasks.workunit.client.0.smithi071.stdout:5/684: dread d3/d1d/daf/f24 [0,4194304] 0 2024-04-23T02:14:06.008 INFO:tasks.workunit.client.0.smithi071.stdout:9/620: dread d1/d32/d41/db7/d5a/f6b [0,4194304] 0 2024-04-23T02:14:06.010 INFO:tasks.workunit.client.0.smithi071.stdout:2/594: mkdir d0/d2/da0/dc1 0 2024-04-23T02:14:06.013 INFO:tasks.workunit.client.1.smithi192.stdout:2/716: dread d8/d26/d29/d43/fd6 [0,4194304] 0 2024-04-23T02:14:06.053 INFO:tasks.workunit.client.1.smithi192.stdout:1/490: mkdir dc/d16/d95 0 2024-04-23T02:14:06.054 INFO:tasks.workunit.client.1.smithi192.stdout:5/695: sync 2024-04-23T02:14:06.054 INFO:tasks.workunit.client.1.smithi192.stdout:3/636: creat d1/d3/d17/d1a/d7c/fe0 x:0 0 0 2024-04-23T02:14:06.054 INFO:tasks.workunit.client.1.smithi192.stdout:3/637: write d1/d3/d6/d7/f82 [1181758,110635] 0 2024-04-23T02:14:06.057 INFO:tasks.workunit.client.1.smithi192.stdout:6/817: unlink d3/d28/d79/da0/dc2/d101/d80/d8e/c54 0 2024-04-23T02:14:06.059 INFO:tasks.workunit.client.1.smithi192.stdout:9/601: mkdir d0/d9/db3/d5f/dc9 0 2024-04-23T02:14:06.066 INFO:tasks.workunit.client.1.smithi192.stdout:1/491: dread dc/d8b/f4d [0,4194304] 0 2024-04-23T02:14:06.066 INFO:tasks.workunit.client.1.smithi192.stdout:4/794: mknod de/d15/d53/d81/de3/c104 0 2024-04-23T02:14:06.070 INFO:tasks.workunit.client.1.smithi192.stdout:2/717: write d8/d26/d29/f41 [1049691,106573] 0 2024-04-23T02:14:06.071 INFO:tasks.workunit.client.1.smithi192.stdout:3/638: mknod d1/d3/d17/d1f/d33/ce1 0 2024-04-23T02:14:06.072 INFO:tasks.workunit.client.1.smithi192.stdout:2/718: truncate d8/d26/d27/d57/f93 387646 0 2024-04-23T02:14:06.073 INFO:tasks.workunit.client.1.smithi192.stdout:8/919: rename d1/d12/d68/cf8 to d1/c141 0 2024-04-23T02:14:06.073 INFO:tasks.workunit.client.1.smithi192.stdout:5/696: truncate d2/db/f11 14223 0 2024-04-23T02:14:06.074 INFO:tasks.workunit.client.1.smithi192.stdout:6/818: getdents d3/d15/d1b/dcf/ddf 0 2024-04-23T02:14:06.075 INFO:tasks.workunit.client.1.smithi192.stdout:1/492: symlink dc/d14/d15/d35/l96 0 2024-04-23T02:14:06.076 INFO:tasks.workunit.client.1.smithi192.stdout:4/795: creat de/d15/d53/d81/dac/f105 x:0 0 0 2024-04-23T02:14:06.076 INFO:tasks.workunit.client.1.smithi192.stdout:1/493: fsync f1 0 2024-04-23T02:14:06.076 INFO:tasks.workunit.client.1.smithi192.stdout:3/639: rmdir d1/d3/d6/d7/d29 39 2024-04-23T02:14:06.078 INFO:tasks.workunit.client.1.smithi192.stdout:2/719: fdatasync d8/d14/faa 0 2024-04-23T02:14:06.079 INFO:tasks.workunit.client.1.smithi192.stdout:3/640: write d1/d3/d17/f3d [348215,112162] 0 2024-04-23T02:14:06.080 INFO:tasks.workunit.client.1.smithi192.stdout:2/720: dread - d8/d26/d29/da4/da8/fc9 zero size 2024-04-23T02:14:06.081 INFO:tasks.workunit.client.1.smithi192.stdout:5/697: truncate d2/db/d2b/d52/f92 207235 0 2024-04-23T02:14:06.081 INFO:tasks.workunit.client.1.smithi192.stdout:2/721: chown d8/d26/d29/d43/l66 591152421 1 2024-04-23T02:14:06.083 INFO:tasks.workunit.client.1.smithi192.stdout:2/722: truncate d8/d26/d74/fc5 6072 0 2024-04-23T02:14:06.086 INFO:tasks.workunit.client.1.smithi192.stdout:1/494: symlink dc/d14/d23/d5e/d6a/l97 0 2024-04-23T02:14:06.087 INFO:tasks.workunit.client.1.smithi192.stdout:9/602: getdents d0/d9/d44/d53 0 2024-04-23T02:14:06.088 INFO:tasks.workunit.client.1.smithi192.stdout:6/819: write d3/f12 [166875,34438] 0 2024-04-23T02:14:06.092 INFO:tasks.workunit.client.1.smithi192.stdout:4/796: mkdir de/d15/d19/d1a/d106 0 2024-04-23T02:14:06.093 INFO:tasks.workunit.client.1.smithi192.stdout:2/723: rmdir d8/d26/d29/da4/dca 39 2024-04-23T02:14:06.095 INFO:tasks.workunit.client.1.smithi192.stdout:9/603: creat d0/d98/fca x:0 0 0 2024-04-23T02:14:06.096 INFO:tasks.workunit.client.1.smithi192.stdout:9/604: readlink d0/d9/d44/d58/d89/la1 0 2024-04-23T02:14:06.097 INFO:tasks.workunit.client.1.smithi192.stdout:3/641: creat d1/d3/d6/d7/d29/fe2 x:0 0 0 2024-04-23T02:14:06.097 INFO:tasks.workunit.client.1.smithi192.stdout:8/920: truncate d1/d12/d23/d4b/d73/d5d/fa6 1766050 0 2024-04-23T02:14:06.098 INFO:tasks.workunit.client.1.smithi192.stdout:9/605: chown d0/d9/db3/d5f/dc9 12620486 1 2024-04-23T02:14:06.099 INFO:tasks.workunit.client.1.smithi192.stdout:5/698: write d2/d32/f44 [4035361,80097] 0 2024-04-23T02:14:06.101 INFO:tasks.workunit.client.1.smithi192.stdout:1/495: mknod dc/d8b/d34/d70/d8c/c98 0 2024-04-23T02:14:06.102 INFO:tasks.workunit.client.1.smithi192.stdout:1/496: chown dc/d14/d15/d35 0 1 2024-04-23T02:14:06.102 INFO:tasks.workunit.client.1.smithi192.stdout:8/921: rmdir d1/d12/d23/d2c/d8c 39 2024-04-23T02:14:06.103 INFO:tasks.workunit.client.1.smithi192.stdout:4/797: creat de/d15/d19/d1a/d106/f107 x:0 0 0 2024-04-23T02:14:06.103 INFO:tasks.workunit.client.1.smithi192.stdout:3/642: truncate d1/d3/d17/d1a/d7c/dab/d6c/d84/d4c/fae 525685 0 2024-04-23T02:14:06.105 INFO:tasks.workunit.client.1.smithi192.stdout:2/724: mknod d8/d14/cef 0 2024-04-23T02:14:06.106 INFO:tasks.workunit.client.1.smithi192.stdout:3/643: chown d1/d3/d6/d7/cf 14610 1 2024-04-23T02:14:06.108 INFO:tasks.workunit.client.1.smithi192.stdout:8/922: symlink d1/de1/l142 0 2024-04-23T02:14:06.108 INFO:tasks.workunit.client.1.smithi192.stdout:4/798: chown de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/fae 11556 1 2024-04-23T02:14:06.110 INFO:tasks.workunit.client.1.smithi192.stdout:8/923: fsync d1/d12/d23/d4b/d73/d4c/f64 0 2024-04-23T02:14:06.111 INFO:tasks.workunit.client.1.smithi192.stdout:1/497: rename dc/d14/d23/d5e/l66 to dc/d16/d19/d74/d77/l99 0 2024-04-23T02:14:06.112 INFO:tasks.workunit.client.1.smithi192.stdout:6/820: link d3/d28/c5a d3/d28/df8/d3a/c10f 0 2024-04-23T02:14:06.112 INFO:tasks.workunit.client.1.smithi192.stdout:4/799: chown de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/d59/f63 40 1 2024-04-23T02:14:06.115 INFO:tasks.workunit.client.1.smithi192.stdout:2/725: creat d8/d26/d27/d53/da2/dc1/ff0 x:0 0 0 2024-04-23T02:14:06.117 INFO:tasks.workunit.client.1.smithi192.stdout:2/726: write d8/d26/d27/d53/da2/dc1/fbb [148349,66554] 0 2024-04-23T02:14:06.120 INFO:tasks.workunit.client.1.smithi192.stdout:6/821: fsync d3/d28/d79/da0/dc2/d101/d80/d8e/d9d/ff7 0 2024-04-23T02:14:06.120 INFO:tasks.workunit.client.1.smithi192.stdout:5/699: dread d2/db/d16/d1b/f90 [0,4194304] 0 2024-04-23T02:14:06.122 INFO:tasks.workunit.client.1.smithi192.stdout:4/800: read de/d15/d53/d81/de3/d74/d92/da0/dca/f8b [595244,118860] 0 2024-04-23T02:14:06.123 INFO:tasks.workunit.client.1.smithi192.stdout:9/606: dread d0/d9/d44/d74/f23 [0,4194304] 0 2024-04-23T02:14:06.125 INFO:tasks.workunit.client.1.smithi192.stdout:8/924: mkdir d1/d9/d3e/d143 0 2024-04-23T02:14:06.127 INFO:tasks.workunit.client.1.smithi192.stdout:8/925: chown d1/d9/d8f/f103 464634 1 2024-04-23T02:14:06.127 INFO:tasks.workunit.client.1.smithi192.stdout:6/822: readlink d3/lc 0 2024-04-23T02:14:06.128 INFO:tasks.workunit.client.1.smithi192.stdout:5/700: mknod d2/d6/d2d/d41/d43/d98/ceb 0 2024-04-23T02:14:06.130 INFO:tasks.workunit.client.1.smithi192.stdout:9/607: fdatasync d0/d9/d44/d4f/f3e 0 2024-04-23T02:14:06.132 INFO:tasks.workunit.client.1.smithi192.stdout:2/727: symlink d8/d26/d29/d43/ded/lf1 0 2024-04-23T02:14:06.133 INFO:tasks.workunit.client.1.smithi192.stdout:1/498: dread dc/f76 [0,4194304] 0 2024-04-23T02:14:06.134 INFO:tasks.workunit.client.1.smithi192.stdout:2/728: stat d8/d25/f6c 0 2024-04-23T02:14:06.136 INFO:tasks.workunit.client.1.smithi192.stdout:3/644: getdents d1/d3/d6 0 2024-04-23T02:14:06.138 INFO:tasks.workunit.client.1.smithi192.stdout:2/729: read - d8/d26/d27/db5/dd0/fd8 zero size 2024-04-23T02:14:06.138 INFO:tasks.workunit.client.1.smithi192.stdout:3/645: fsync d1/d3/d6/d7/d29/fe2 0 2024-04-23T02:14:06.139 INFO:tasks.workunit.client.1.smithi192.stdout:5/701: dwrite d2/d6/f8a [4194304,4194304] 0 2024-04-23T02:14:06.141 INFO:tasks.workunit.client.1.smithi192.stdout:5/702: dread - d2/db/d46/fe7 zero size 2024-04-23T02:14:06.141 INFO:tasks.workunit.client.1.smithi192.stdout:8/926: mkdir d1/d144 0 2024-04-23T02:14:06.147 INFO:tasks.workunit.client.1.smithi192.stdout:4/801: truncate de/d15/d53/d81/de3/da3/ff3 2963369 0 2024-04-23T02:14:06.149 INFO:tasks.workunit.client.1.smithi192.stdout:8/927: write d1/d9/d3e/d59/d8a/f8d [5690824,14244] 0 2024-04-23T02:14:06.150 INFO:tasks.workunit.client.1.smithi192.stdout:2/730: creat d8/d26/d29/da4/da8/dba/ff2 x:0 0 0 2024-04-23T02:14:06.152 INFO:tasks.workunit.client.1.smithi192.stdout:2/731: dread - d8/fa6 zero size 2024-04-23T02:14:06.152 INFO:tasks.workunit.client.1.smithi192.stdout:5/703: dwrite d2/db/d16/d1b/f9f [0,4194304] 0 2024-04-23T02:14:06.154 INFO:tasks.workunit.client.1.smithi192.stdout:1/499: getdents dc/d14/d23 0 2024-04-23T02:14:06.156 INFO:tasks.workunit.client.1.smithi192.stdout:6/823: write d3/d28/df8/d52/f92 [472825,70242] 0 2024-04-23T02:14:06.157 INFO:tasks.workunit.client.1.smithi192.stdout:9/608: write d0/d9/f42 [667749,98783] 0 2024-04-23T02:14:06.158 INFO:tasks.workunit.client.1.smithi192.stdout:5/704: symlink d2/db/d16/d48/lec 0 2024-04-23T02:14:06.159 INFO:tasks.workunit.client.1.smithi192.stdout:2/732: fdatasync d8/d26/d27/d53/f64 0 2024-04-23T02:14:06.159 INFO:tasks.workunit.client.1.smithi192.stdout:1/500: chown dc/d16/d19/d30/f3f 85981233 1 2024-04-23T02:14:06.160 INFO:tasks.workunit.client.1.smithi192.stdout:8/928: dread - d1/d12/d23/d2c/d8c/d95/fb5 zero size 2024-04-23T02:14:06.160 INFO:tasks.workunit.client.0.smithi071.stdout:8/658: sync 2024-04-23T02:14:06.184 INFO:tasks.workunit.client.0.smithi071.stdout:7/729: write d3/d17/d41/f33 [1610266,62519] 0 2024-04-23T02:14:06.187 INFO:tasks.workunit.client.0.smithi071.stdout:2/595: symlink d0/d2/d49/lc2 0 2024-04-23T02:14:06.187 INFO:tasks.workunit.client.0.smithi071.stdout:5/685: write d3/d1d/f3b [258158,107144] 0 2024-04-23T02:14:06.187 INFO:tasks.workunit.client.0.smithi071.stdout:8/659: mkdir dc/d15/da0/d32/d63/dd4 0 2024-04-23T02:14:06.190 INFO:tasks.workunit.client.0.smithi071.stdout:6/593: dread d6/d1f/d4e/f54 [0,4194304] 0 2024-04-23T02:14:06.190 INFO:tasks.workunit.client.0.smithi071.stdout:0/624: dwrite d0/d63/fbf [0,4194304] 0 2024-04-23T02:14:06.192 INFO:tasks.workunit.client.0.smithi071.stdout:8/660: chown dc/d15/d50 5953781 1 2024-04-23T02:14:06.193 INFO:tasks.workunit.client.0.smithi071.stdout:0/625: readlink d0/d63/l83 0 2024-04-23T02:14:06.194 INFO:tasks.workunit.client.0.smithi071.stdout:2/596: write d0/d2/d26/d14/fa7 [973203,38757] 0 2024-04-23T02:14:06.195 INFO:tasks.workunit.client.0.smithi071.stdout:5/686: rmdir d3/d1d/d43/d4b 39 2024-04-23T02:14:06.195 INFO:tasks.workunit.client.0.smithi071.stdout:7/730: write d3/d17/d1c/d2a/d43/d86/f88 [70373,12328] 0 2024-04-23T02:14:06.198 INFO:tasks.workunit.client.0.smithi071.stdout:9/621: dwrite d1/f9a [0,4194304] 0 2024-04-23T02:14:06.199 INFO:tasks.workunit.client.0.smithi071.stdout:8/661: mknod dc/d15/d5d/dc6/cd5 0 2024-04-23T02:14:06.200 INFO:tasks.workunit.client.0.smithi071.stdout:1/562: dwrite db/f54 [0,4194304] 0 2024-04-23T02:14:06.208 INFO:tasks.workunit.client.0.smithi071.stdout:9/622: chown d1/d32/d41/db7/d39/d42/fcb 51686 1 2024-04-23T02:14:06.211 INFO:tasks.workunit.client.0.smithi071.stdout:2/597: rename d0/d2/d26/d14/d1f to d0/d2/d26/d3a/dc3 0 2024-04-23T02:14:06.212 INFO:tasks.workunit.client.0.smithi071.stdout:1/563: creat db/df/d13/d2f/fb6 x:0 0 0 2024-04-23T02:14:06.214 INFO:tasks.workunit.client.0.smithi071.stdout:6/594: dwrite d6/d17/d77/d9f/dad/f75 [0,4194304] 0 2024-04-23T02:14:06.214 INFO:tasks.workunit.client.0.smithi071.stdout:9/623: mknod d1/d32/d3a/d77/cdb 0 2024-04-23T02:14:06.215 INFO:tasks.workunit.client.0.smithi071.stdout:7/731: link d3/d17/d1c/f3f d3/d17/d41/d3a/fff 0 2024-04-23T02:14:06.215 INFO:tasks.workunit.client.1.smithi192.stdout:7/731: sync 2024-04-23T02:14:06.216 INFO:tasks.workunit.client.0.smithi071.stdout:0/626: creat d0/d3/de/d10/d14/d32/d37/fdb x:0 0 0 2024-04-23T02:14:06.218 INFO:tasks.workunit.client.0.smithi071.stdout:1/564: rename db/df/d13/d20/d75/fb1 to db/df/d5f/d6f/fb7 0 2024-04-23T02:14:06.220 INFO:tasks.workunit.client.0.smithi071.stdout:9/624: symlink d1/d32/d41/db7/d39/d42/ldc 0 2024-04-23T02:14:06.221 INFO:tasks.workunit.client.0.smithi071.stdout:0/627: mknod d0/d3/de/d10/db9/cdc 0 2024-04-23T02:14:06.223 INFO:tasks.workunit.client.0.smithi071.stdout:6/595: symlink d6/lbe 0 2024-04-23T02:14:06.223 INFO:tasks.workunit.client.0.smithi071.stdout:0/628: write d0/d92/fce [996796,7990] 0 2024-04-23T02:14:06.223 INFO:tasks.workunit.client.0.smithi071.stdout:1/565: symlink db/df/d13/d2b/lb8 0 2024-04-23T02:14:06.224 INFO:tasks.workunit.client.1.smithi192.stdout:4/802: dwrite de/f18 [0,4194304] 0 2024-04-23T02:14:06.226 INFO:tasks.workunit.client.0.smithi071.stdout:0/629: mknod d0/d76/cdd 0 2024-04-23T02:14:06.227 INFO:tasks.workunit.client.0.smithi071.stdout:6/596: stat d6/c7 0 2024-04-23T02:14:06.227 INFO:tasks.workunit.client.0.smithi071.stdout:3/563: dread d7/da5/db2/db7/d68/f84 [0,4194304] 0 2024-04-23T02:14:06.230 INFO:tasks.workunit.client.1.smithi192.stdout:7/732: write d5/d17/f9e [3817205,9828] 0 2024-04-23T02:14:06.231 INFO:tasks.workunit.client.1.smithi192.stdout:1/501: unlink dc/f10 0 2024-04-23T02:14:06.233 INFO:tasks.workunit.client.0.smithi071.stdout:0/630: creat d0/d3/de/d10/d14/d32/d37/d26/fde x:0 0 0 2024-04-23T02:14:06.235 INFO:tasks.workunit.client.0.smithi071.stdout:3/564: creat d7/d22/fc1 x:0 0 0 2024-04-23T02:14:06.237 INFO:tasks.workunit.client.1.smithi192.stdout:8/929: symlink d1/d12/d23/d4b/d73/dda/d10f/l145 0 2024-04-23T02:14:06.238 INFO:tasks.workunit.client.0.smithi071.stdout:1/566: rmdir db/d11/d7f/dae 0 2024-04-23T02:14:06.239 INFO:tasks.workunit.client.1.smithi192.stdout:1/502: creat dc/d14/d23/d5e/d6a/f9a x:0 0 0 2024-04-23T02:14:06.241 INFO:tasks.workunit.client.1.smithi192.stdout:4/803: symlink de/d15/d53/d81/de3/d74/d92/da0/dca/d34/df9/da1/l108 0 2024-04-23T02:14:06.242 INFO:tasks.workunit.client.0.smithi071.stdout:6/597: dwrite d6/d17/d77/faf [0,4194304] 0 2024-04-23T02:14:06.242 INFO:tasks.workunit.client.1.smithi192.stdout:5/705: dwrite d2/db/d2b/d52/dc2/d49/fb1 [0,4194304] 0 2024-04-23T02:14:06.245 INFO:tasks.workunit.client.0.smithi071.stdout:0/631: dwrite d0/fda [0,4194304] 0 2024-04-23T02:14:06.245 INFO:tasks.workunit.client.0.smithi071.stdout:3/565: creat d7/da5/db2/db7/d68/d3b/d4c/da1/fc2 x:0 0 0 2024-04-23T02:14:06.246 INFO:tasks.workunit.client.1.smithi192.stdout:6/824: dread d3/d18/dd4/d8d/ff2 [0,4194304] 0 2024-04-23T02:14:06.247 INFO:tasks.workunit.client.1.smithi192.stdout:7/733: mkdir d5/d43/d4c/d5c/d67/da6/da9/db9/dd9 0 2024-04-23T02:14:06.248 INFO:tasks.workunit.client.1.smithi192.stdout:8/930: write d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/d108/d11f/f124 [295312,12546] 0 2024-04-23T02:14:06.249 INFO:tasks.workunit.client.0.smithi071.stdout:0/632: mkdir d0/ddf 0 2024-04-23T02:14:06.250 INFO:tasks.workunit.client.1.smithi192.stdout:1/503: mkdir dc/d16/d9b 0 2024-04-23T02:14:06.250 INFO:tasks.workunit.client.1.smithi192.stdout:7/734: fdatasync d5/d17/d1c/d4a/fc2 0 2024-04-23T02:14:06.251 INFO:tasks.workunit.client.0.smithi071.stdout:3/566: symlink d7/da5/db2/db7/d68/d3b/d4c/d8b/lc3 0 2024-04-23T02:14:06.252 INFO:tasks.workunit.client.0.smithi071.stdout:0/633: stat d0/d3/de/d10/d14/d32/l2a 0 2024-04-23T02:14:06.252 INFO:tasks.workunit.client.0.smithi071.stdout:3/567: write d7/d22/d23/d26/f2b [3817949,94801] 0 2024-04-23T02:14:06.258 INFO:tasks.workunit.client.1.smithi192.stdout:6/825: truncate d3/d28/df8/d3a/f4e 1438110 0 2024-04-23T02:14:06.259 INFO:tasks.workunit.client.1.smithi192.stdout:5/706: dwrite d2/d32/f44 [0,4194304] 0 2024-04-23T02:14:06.260 INFO:tasks.workunit.client.1.smithi192.stdout:4/804: truncate de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/fae 298051 0 2024-04-23T02:14:06.264 INFO:tasks.workunit.client.1.smithi192.stdout:9/609: truncate d0/d9/d44/f49 64150 0 2024-04-23T02:14:06.265 INFO:tasks.workunit.client.1.smithi192.stdout:8/931: mknod d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d139/c146 0 2024-04-23T02:14:06.266 INFO:tasks.workunit.client.1.smithi192.stdout:9/610: fdatasync d0/d9/dc7/fc6 0 2024-04-23T02:14:06.266 INFO:tasks.workunit.client.1.smithi192.stdout:4/805: fdatasync de/d15/d53/f83 0 2024-04-23T02:14:06.266 INFO:tasks.workunit.client.1.smithi192.stdout:5/707: symlink d2/d6/d14/led 0 2024-04-23T02:14:06.270 INFO:tasks.workunit.client.1.smithi192.stdout:7/735: truncate d5/d17/d1c/d59/d5a/d41/d50/d9f/fa4 3767104 0 2024-04-23T02:14:06.270 INFO:tasks.workunit.client.1.smithi192.stdout:8/932: creat d1/d9/d8f/d90/f147 x:0 0 0 2024-04-23T02:14:06.272 INFO:tasks.workunit.client.1.smithi192.stdout:4/806: symlink de/d15/d53/d81/de3/d74/d92/da0/dca/d34/df9/l109 0 2024-04-23T02:14:06.272 INFO:tasks.workunit.client.1.smithi192.stdout:8/933: stat d1/d9/d3e/d59/f75 0 2024-04-23T02:14:06.273 INFO:tasks.workunit.client.1.smithi192.stdout:2/733: dread d8/d26/d27/d53/da2/dc1/fbb [0,4194304] 0 2024-04-23T02:14:06.273 INFO:tasks.workunit.client.1.smithi192.stdout:4/807: chown de/d15/dc3 667 1 2024-04-23T02:14:06.274 INFO:tasks.workunit.client.0.smithi071.stdout:1/567: dread db/df/d5f/d57/f89 [0,4194304] 0 2024-04-23T02:14:06.274 INFO:tasks.workunit.client.1.smithi192.stdout:7/736: unlink d5/d17/d1c/d4a/fc2 0 2024-04-23T02:14:06.275 INFO:tasks.workunit.client.0.smithi071.stdout:1/568: rmdir db/df/d5f/d6f/db2/d86 39 2024-04-23T02:14:06.276 INFO:tasks.workunit.client.0.smithi071.stdout:5/687: dread d3/da/d1a/d1c/f62 [0,4194304] 0 2024-04-23T02:14:06.277 INFO:tasks.workunit.client.1.smithi192.stdout:2/734: mkdir d8/d26/d27/db5/df3 0 2024-04-23T02:14:06.277 INFO:tasks.workunit.client.1.smithi192.stdout:7/737: dread - d5/d17/d1c/d59/d5a/d41/d50/fba zero size 2024-04-23T02:14:06.277 INFO:tasks.workunit.client.1.smithi192.stdout:8/934: mkdir d1/d12/d23/d4b/d73/d7e/d88/da9/db1/d148 0 2024-04-23T02:14:06.279 INFO:tasks.workunit.client.1.smithi192.stdout:8/935: chown d1/d9/d3e/d59/d8a/daa 256 1 2024-04-23T02:14:06.279 INFO:tasks.workunit.client.1.smithi192.stdout:2/735: read - d8/d26/d29/d43/d5d/fe0 zero size 2024-04-23T02:14:06.280 INFO:tasks.workunit.client.1.smithi192.stdout:7/738: creat d5/d17/d1c/d59/d5a/d9b/fda x:0 0 0 2024-04-23T02:14:06.280 INFO:tasks.workunit.client.1.smithi192.stdout:4/808: symlink de/d15/d53/d81/de3/de4/l10a 0 2024-04-23T02:14:06.281 INFO:tasks.workunit.client.0.smithi071.stdout:5/688: mknod d3/d1d/d43/d4b/cfd 0 2024-04-23T02:14:06.282 INFO:tasks.workunit.client.1.smithi192.stdout:8/936: chown d1/d9/f12f 10573750 1 2024-04-23T02:14:06.284 INFO:tasks.workunit.client.1.smithi192.stdout:4/809: mkdir de/d15/d53/d81/d10b 0 2024-04-23T02:14:06.285 INFO:tasks.workunit.client.0.smithi071.stdout:5/689: rename d3/d34/d6b/de4/lb3 to d3/d1d/daf/d10/da5/lfe 0 2024-04-23T02:14:06.287 INFO:tasks.workunit.client.1.smithi192.stdout:7/739: rename d5/d17/d1c/d4a/f58 to d5/d43/d4c/d5c/d67/da6/da9/db9/dc0/fdb 0 2024-04-23T02:14:06.288 INFO:tasks.workunit.client.0.smithi071.stdout:5/690: fdatasync d3/d34/f8b 0 2024-04-23T02:14:06.288 INFO:tasks.workunit.client.0.smithi071.stdout:5/691: stat d3/da/d1a/d6d 0 2024-04-23T02:14:06.290 INFO:tasks.workunit.client.1.smithi192.stdout:2/736: dread d8/d25/f3f [0,4194304] 0 2024-04-23T02:14:06.291 INFO:tasks.workunit.client.0.smithi071.stdout:5/692: readlink d3/d1d/d43/d4b/l9d 0 2024-04-23T02:14:06.293 INFO:tasks.workunit.client.1.smithi192.stdout:8/937: truncate d1/d12/fc1 641323 0 2024-04-23T02:14:06.294 INFO:tasks.workunit.client.1.smithi192.stdout:8/938: write d1/de1/f13e [368316,27431] 0 2024-04-23T02:14:06.295 INFO:tasks.workunit.client.1.smithi192.stdout:6/826: dread d3/d15/d1b/da2/faa [0,4194304] 0 2024-04-23T02:14:06.296 INFO:tasks.workunit.client.1.smithi192.stdout:7/740: mkdir d5/d17/d1c/d59/d5a/d41/d50/d3d/ddc 0 2024-04-23T02:14:06.298 INFO:tasks.workunit.client.1.smithi192.stdout:5/708: dread d2/fd [0,4194304] 0 2024-04-23T02:14:06.299 INFO:tasks.workunit.client.1.smithi192.stdout:2/737: fdatasync d8/d26/d29/d43/d50/d5e/f89 0 2024-04-23T02:14:06.301 INFO:tasks.workunit.client.1.smithi192.stdout:2/738: write d8/d26/d27/d57/f93 [280932,87915] 0 2024-04-23T02:14:06.304 INFO:tasks.workunit.client.1.smithi192.stdout:8/939: creat d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/d108/f149 x:0 0 0 2024-04-23T02:14:06.306 INFO:tasks.workunit.client.1.smithi192.stdout:5/709: mknod d2/db/d16/d1b/cee 0 2024-04-23T02:14:06.309 INFO:tasks.workunit.client.1.smithi192.stdout:4/810: getdents de/d15/d53/d81/de3/d74/d92/da0/dca/d9f 0 2024-04-23T02:14:06.311 INFO:tasks.workunit.client.1.smithi192.stdout:6/827: mkdir d3/d28/d79/da0/dc2/d101/d80/d107/d110 0 2024-04-23T02:14:06.313 INFO:tasks.workunit.client.1.smithi192.stdout:2/739: rename d8/d26/d27/d57/lb3 to d8/d26/d29/d43/dc2/lf4 0 2024-04-23T02:14:06.314 INFO:tasks.workunit.client.1.smithi192.stdout:8/940: rmdir d1/d12/d23/d2c 39 2024-04-23T02:14:06.316 INFO:tasks.workunit.client.1.smithi192.stdout:8/941: write d1/de1/ff1 [833804,68696] 0 2024-04-23T02:14:06.317 INFO:tasks.workunit.client.1.smithi192.stdout:4/811: dwrite de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d93/fef [0,4194304] 0 2024-04-23T02:14:06.317 INFO:tasks.workunit.client.1.smithi192.stdout:7/741: creat d5/d17/d1c/d59/fdd x:0 0 0 2024-04-23T02:14:06.319 INFO:tasks.workunit.client.1.smithi192.stdout:8/942: readlink d1/d12/d23/d4b/d73/d7e/d88/da9/db1/l122 0 2024-04-23T02:14:06.321 INFO:tasks.workunit.client.1.smithi192.stdout:8/943: chown d1/d9/d8f/lbe 166 1 2024-04-23T02:14:06.321 INFO:tasks.workunit.client.1.smithi192.stdout:5/710: getdents d2/db/d16/d1b/d3d 0 2024-04-23T02:14:06.322 INFO:tasks.workunit.client.1.smithi192.stdout:4/812: chown de/d15/d53/d81/de3/d74/f7e 1 1 2024-04-23T02:14:06.322 INFO:tasks.workunit.client.1.smithi192.stdout:8/944: stat d1/d12/d68/ddf/ffe 0 2024-04-23T02:14:06.324 INFO:tasks.workunit.client.1.smithi192.stdout:7/742: truncate d5/d17/d1c/d59/fdd 275119 0 2024-04-23T02:14:06.324 INFO:tasks.workunit.client.1.smithi192.stdout:6/828: mknod d3/d15/d1b/dcf/ddf/c111 0 2024-04-23T02:14:06.325 INFO:tasks.workunit.client.1.smithi192.stdout:6/829: fsync d3/d6c/fd0 0 2024-04-23T02:14:06.326 INFO:tasks.workunit.client.1.smithi192.stdout:2/740: symlink d8/d26/lf5 0 2024-04-23T02:14:06.326 INFO:tasks.workunit.client.1.smithi192.stdout:4/813: read de/f21 [3766589,89577] 0 2024-04-23T02:14:06.327 INFO:tasks.workunit.client.1.smithi192.stdout:5/711: mknod d2/d6/d14/cef 0 2024-04-23T02:14:06.331 INFO:tasks.workunit.client.1.smithi192.stdout:4/814: chown de/d15/d53/d81/de3/da3 388856480 1 2024-04-23T02:14:06.335 INFO:tasks.workunit.client.1.smithi192.stdout:6/830: mknod d3/d28/d79/da0/dc2/d101/d80/de2/c112 0 2024-04-23T02:14:06.335 INFO:tasks.workunit.client.1.smithi192.stdout:7/743: dwrite d5/d17/d1c/d59/d5a/d41/fd7 [0,4194304] 0 2024-04-23T02:14:06.339 INFO:tasks.workunit.client.1.smithi192.stdout:2/741: rename d8/d26/d27/f70 to d8/d26/d29/d43/d5d/dbd/ff6 0 2024-04-23T02:14:06.345 INFO:tasks.workunit.client.1.smithi192.stdout:4/815: rmdir de/d15/d53/d81/de3/d74/d92/da0/dca/db6 39 2024-04-23T02:14:06.345 INFO:tasks.workunit.client.1.smithi192.stdout:5/712: dread d2/db/d2b/f3a [0,4194304] 0 2024-04-23T02:14:06.345 INFO:tasks.workunit.client.1.smithi192.stdout:2/742: readlink d8/d14/l49 0 2024-04-23T02:14:06.346 INFO:tasks.workunit.client.1.smithi192.stdout:5/713: fdatasync d2/db/d46/fe7 0 2024-04-23T02:14:06.349 INFO:tasks.workunit.client.1.smithi192.stdout:4/816: read de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/fae [194089,17564] 0 2024-04-23T02:14:06.350 INFO:tasks.workunit.client.1.smithi192.stdout:0/813: sync 2024-04-23T02:14:06.351 INFO:tasks.workunit.client.0.smithi071.stdout:5/693: mkdir d3/da/def/dff 0 2024-04-23T02:14:06.352 INFO:tasks.workunit.client.1.smithi192.stdout:5/714: read d2/db/d2b/d52/dc2/f8c [2124220,72740] 0 2024-04-23T02:14:06.355 INFO:tasks.workunit.client.1.smithi192.stdout:2/743: write d8/d26/d74/d7c/f9b [1504606,54676] 0 2024-04-23T02:14:06.356 INFO:tasks.workunit.client.0.smithi071.stdout:8/662: truncate dc/d15/da0/d30/fa6 3903363 0 2024-04-23T02:14:06.357 INFO:tasks.workunit.client.1.smithi192.stdout:8/945: dwrite d1/df/f9b [0,4194304] 0 2024-04-23T02:14:06.358 INFO:tasks.workunit.client.0.smithi071.stdout:5/694: dwrite d3/d1d/daf/d10/f48 [0,4194304] 0 2024-04-23T02:14:06.358 INFO:tasks.workunit.client.1.smithi192.stdout:6/831: dread d3/f12 [0,4194304] 0 2024-04-23T02:14:06.359 INFO:tasks.workunit.client.0.smithi071.stdout:8/663: creat dc/d15/d5d/fd6 x:0 0 0 2024-04-23T02:14:06.360 INFO:tasks.workunit.client.1.smithi192.stdout:5/715: rmdir d2/d6/d2d/d41/d43/dbc 39 2024-04-23T02:14:06.361 INFO:tasks.workunit.client.0.smithi071.stdout:5/695: mknod d3/d8f/de6/c100 0 2024-04-23T02:14:06.362 INFO:tasks.workunit.client.0.smithi071.stdout:9/625: write d1/f37 [4895114,88640] 0 2024-04-23T02:14:06.364 INFO:tasks.workunit.client.0.smithi071.stdout:8/664: creat dc/d56/d64/fd7 x:0 0 0 2024-04-23T02:14:06.365 INFO:tasks.workunit.client.1.smithi192.stdout:2/744: read - d8/d26/d29/da4/dca/fd4 zero size 2024-04-23T02:14:06.371 INFO:tasks.workunit.client.0.smithi071.stdout:5/696: creat d3/d1d/daf/d10/da5/d5f/dc0/dfa/f101 x:0 0 0 2024-04-23T02:14:06.372 INFO:tasks.workunit.client.0.smithi071.stdout:9/626: symlink d1/d32/d41/db7/ldd 0 2024-04-23T02:14:06.374 INFO:tasks.workunit.client.1.smithi192.stdout:0/814: link d6/df/d47/d55/l5f d6/df/l10f 0 2024-04-23T02:14:06.375 INFO:tasks.workunit.client.1.smithi192.stdout:6/832: dread d3/d28/df8/d7c/dda/fef [0,4194304] 0 2024-04-23T02:14:06.376 INFO:tasks.workunit.client.0.smithi071.stdout:5/697: symlink d3/d1d/d43/dd2/l102 0 2024-04-23T02:14:06.376 INFO:tasks.workunit.client.0.smithi071.stdout:8/665: dwrite dc/d15/da0/d8d/fc3 [0,4194304] 0 2024-04-23T02:14:06.377 INFO:tasks.workunit.client.0.smithi071.stdout:9/627: mkdir d1/d32/d3a/d5f/dde 0 2024-04-23T02:14:06.377 INFO:tasks.workunit.client.1.smithi192.stdout:8/946: write d1/d12/d68/f83 [551218,5816] 0 2024-04-23T02:14:06.379 INFO:tasks.workunit.client.1.smithi192.stdout:5/716: dread d2/db/d2b/d52/f76 [0,4194304] 0 2024-04-23T02:14:06.379 INFO:tasks.workunit.client.0.smithi071.stdout:3/568: write d7/fd [4028311,104817] 0 2024-04-23T02:14:06.379 INFO:tasks.workunit.client.0.smithi071.stdout:0/634: truncate d0/d3/de/d10/d16/f55 2247633 0 2024-04-23T02:14:06.379 INFO:tasks.workunit.client.1.smithi192.stdout:4/817: truncate de/d15/d53/d81/de3/d74/d92/da0/dca/f54 1852379 0 2024-04-23T02:14:06.381 INFO:tasks.workunit.client.0.smithi071.stdout:1/569: write db/d34/f7d [3221921,105662] 0 2024-04-23T02:14:06.381 INFO:tasks.workunit.client.1.smithi192.stdout:6/833: mknod d3/d28/d79/dbe/c113 0 2024-04-23T02:14:06.383 INFO:tasks.workunit.client.0.smithi071.stdout:5/698: write d3/d1d/d43/fcb [642009,119314] 0 2024-04-23T02:14:06.383 INFO:tasks.workunit.client.0.smithi071.stdout:3/569: chown d7/d22/d23/d26/d52 3151 1 2024-04-23T02:14:06.383 INFO:tasks.workunit.client.0.smithi071.stdout:0/635: symlink d0/d63/le0 0 2024-04-23T02:14:06.385 INFO:tasks.workunit.client.1.smithi192.stdout:2/745: dwrite d8/d26/d27/d53/da2/dc1/fbb [0,4194304] 0 2024-04-23T02:14:06.385 INFO:tasks.workunit.client.1.smithi192.stdout:5/717: stat d2/d6/d2d/cb2 0 2024-04-23T02:14:06.385 INFO:tasks.workunit.client.0.smithi071.stdout:6/598: dwrite f1 [0,4194304] 0 2024-04-23T02:14:06.388 INFO:tasks.workunit.client.1.smithi192.stdout:6/834: write d3/d28/df8/d7c/f10b [782802,115718] 0 2024-04-23T02:14:06.392 INFO:tasks.workunit.client.1.smithi192.stdout:0/815: truncate d6/fda 1658872 0 2024-04-23T02:14:06.393 INFO:tasks.workunit.client.1.smithi192.stdout:4/818: rmdir de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/de7 39 2024-04-23T02:14:06.393 INFO:tasks.workunit.client.1.smithi192.stdout:5/718: truncate d2/db/d2b/d52/dc2/d49/f83 189668 0 2024-04-23T02:14:06.395 INFO:tasks.workunit.client.1.smithi192.stdout:2/746: dread d8/d26/d27/d53/da2/fc0 [0,4194304] 0 2024-04-23T02:14:06.397 INFO:tasks.workunit.client.1.smithi192.stdout:0/816: creat d6/df/d13/d25/d26/d67/f110 x:0 0 0 2024-04-23T02:14:06.399 INFO:tasks.workunit.client.1.smithi192.stdout:4/819: unlink de/d15/f2d 0 2024-04-23T02:14:06.399 INFO:tasks.workunit.client.1.smithi192.stdout:0/817: readlink d6/df/l1d 0 2024-04-23T02:14:06.401 INFO:tasks.workunit.client.1.smithi192.stdout:5/719: rename d2/db/d16/d1b/d3d to d2/d2e/df0 0 2024-04-23T02:14:06.402 INFO:tasks.workunit.client.1.smithi192.stdout:2/747: truncate d8/d26/d29/d43/d50/fe8 349229 0 2024-04-23T02:14:06.405 INFO:tasks.workunit.client.1.smithi192.stdout:6/835: dwrite d3/d15/d5b/fb8 [0,4194304] 0 2024-04-23T02:14:06.406 INFO:tasks.workunit.client.1.smithi192.stdout:0/818: chown d6/df/d13/d6b/d73/da9/fe5 3996 1 2024-04-23T02:14:06.408 INFO:tasks.workunit.client.1.smithi192.stdout:4/820: write de/d15/d53/d81/de3/f5c [1363403,58789] 0 2024-04-23T02:14:06.410 INFO:tasks.workunit.client.1.smithi192.stdout:2/748: creat d8/d26/d27/d57/da0/ff7 x:0 0 0 2024-04-23T02:14:06.411 INFO:tasks.workunit.client.1.smithi192.stdout:0/819: chown d6/df/d56/d85/da4 14129 1 2024-04-23T02:14:06.422 INFO:tasks.workunit.client.1.smithi192.stdout:0/820: creat d6/df/d13/d25/d94/dd0/f111 x:0 0 0 2024-04-23T02:14:06.427 INFO:tasks.workunit.client.1.smithi192.stdout:0/821: dread - d6/df/d13/d25/d26/d67/ffe zero size 2024-04-23T02:14:06.427 INFO:tasks.workunit.client.1.smithi192.stdout:6/836: dwrite d3/d18/dd4/d8d/ff2 [0,4194304] 0 2024-04-23T02:14:06.432 INFO:tasks.workunit.client.1.smithi192.stdout:8/947: dread d1/d12/d23/d4b/d73/d45/d46/f6d [0,4194304] 0 2024-04-23T02:14:06.433 INFO:tasks.workunit.client.1.smithi192.stdout:2/749: write d8/d26/d27/d31/f2a [4491516,112950] 0 2024-04-23T02:14:06.435 INFO:tasks.workunit.client.1.smithi192.stdout:2/750: chown d8/c62 66732 1 2024-04-23T02:14:06.438 INFO:tasks.workunit.client.1.smithi192.stdout:5/720: dread d2/db/d16/f26 [0,4194304] 0 2024-04-23T02:14:06.439 INFO:tasks.workunit.client.1.smithi192.stdout:2/751: rmdir d8/d26/d27/db5 39 2024-04-23T02:14:06.440 INFO:tasks.workunit.client.0.smithi071.stdout:3/570: write d7/d8/f11 [1201695,61252] 0 2024-04-23T02:14:06.440 INFO:tasks.workunit.client.1.smithi192.stdout:2/752: readlink d8/d26/la5 0 2024-04-23T02:14:06.443 INFO:tasks.workunit.client.0.smithi071.stdout:3/571: chown d7/da5/db2/db7/d68 302688238 1 2024-04-23T02:14:06.443 INFO:tasks.workunit.client.0.smithi071.stdout:5/699: creat d3/da/df6/f103 x:0 0 0 2024-04-23T02:14:06.444 INFO:tasks.workunit.client.1.smithi192.stdout:5/721: creat d2/db/d2b/d52/dc2/d49/ff1 x:0 0 0 2024-04-23T02:14:06.445 INFO:tasks.workunit.client.0.smithi071.stdout:0/636: creat d0/d3/de/d10/d14/d32/d44/d31/fe1 x:0 0 0 2024-04-23T02:14:06.446 INFO:tasks.workunit.client.0.smithi071.stdout:6/599: mkdir d6/d22/dbf 0 2024-04-23T02:14:06.447 INFO:tasks.workunit.client.0.smithi071.stdout:6/600: chown d6/d1f/d5d 1490582443 1 2024-04-23T02:14:06.448 INFO:tasks.workunit.client.0.smithi071.stdout:0/637: chown d0/d3/de/d10/d14/d32/d44/d87/fa0 124960006 1 2024-04-23T02:14:06.449 INFO:tasks.workunit.client.0.smithi071.stdout:8/666: rename dc/d15/da0/d55/l6d to dc/d15/ld8 0 2024-04-23T02:14:06.450 INFO:tasks.workunit.client.1.smithi192.stdout:8/948: rmdir d1/d144 0 2024-04-23T02:14:06.450 INFO:tasks.workunit.client.1.smithi192.stdout:4/821: link de/c7a de/d15/d53/d81/de3/d74/d92/da0/dca/d34/c10c 0 2024-04-23T02:14:06.450 INFO:tasks.workunit.client.0.smithi071.stdout:3/572: dread - d7/d10/d38/d67/f82 zero size 2024-04-23T02:14:06.452 INFO:tasks.workunit.client.0.smithi071.stdout:5/700: mknod d3/d1d/daf/d10/da5/c104 0 2024-04-23T02:14:06.452 INFO:tasks.workunit.client.1.smithi192.stdout:5/722: rename d2/d6/d2d/d41/d43/c94 to d2/db/d16/d48/cf2 0 2024-04-23T02:14:06.452 INFO:tasks.workunit.client.0.smithi071.stdout:6/601: mknod d6/d1f/d5d/cc0 0 2024-04-23T02:14:06.454 INFO:tasks.workunit.client.1.smithi192.stdout:4/822: mknod de/d15/d53/d81/c10d 0 2024-04-23T02:14:06.455 INFO:tasks.workunit.client.0.smithi071.stdout:0/638: creat d0/d3/de/d10/d14/d32/d37/d26/fe2 x:0 0 0 2024-04-23T02:14:06.457 INFO:tasks.workunit.client.0.smithi071.stdout:3/573: readlink d7/d22/da7/l90 0 2024-04-23T02:14:06.458 INFO:tasks.workunit.client.0.smithi071.stdout:1/570: dwrite db/df/d13/d2b/f70 [0,4194304] 0 2024-04-23T02:14:06.458 INFO:tasks.workunit.client.1.smithi192.stdout:8/949: rename d1/d12/d23/d2c/d8c/d95/c97 to d1/d9/d8f/c14a 0 2024-04-23T02:14:06.462 INFO:tasks.workunit.client.1.smithi192.stdout:8/950: dread - d1/d12/d23/d4b/d73/d45/d46/f11b zero size 2024-04-23T02:14:06.462 INFO:tasks.workunit.client.1.smithi192.stdout:2/753: link d8/c20 d8/d14/cf8 0 2024-04-23T02:14:06.463 INFO:tasks.workunit.client.1.smithi192.stdout:8/951: mknod d1/d9/d3e/d59/c14b 0 2024-04-23T02:14:06.464 INFO:tasks.workunit.client.0.smithi071.stdout:0/639: mkdir d0/d3/de/d48/d7e/d8b/de3 0 2024-04-23T02:14:06.465 INFO:tasks.workunit.client.1.smithi192.stdout:0/822: write d6/df/d13/d25/d3a/f62 [1563380,63109] 0 2024-04-23T02:14:06.466 INFO:tasks.workunit.client.1.smithi192.stdout:0/823: chown d6/df/d47/d55/dae/cb9 147 1 2024-04-23T02:14:06.466 INFO:tasks.workunit.client.1.smithi192.stdout:8/952: truncate d1/d12/d23/d2c/fe5 49427 0 2024-04-23T02:14:06.468 INFO:tasks.workunit.client.0.smithi071.stdout:3/574: creat d7/da5/db2/db7/d68/d3b/d4c/d8b/db3/fc4 x:0 0 0 2024-04-23T02:14:06.469 INFO:tasks.workunit.client.0.smithi071.stdout:0/640: chown d0/d3/de/d10/d14/f4d 413673017 1 2024-04-23T02:14:06.469 INFO:tasks.workunit.client.1.smithi192.stdout:0/824: symlink d6/df/d13/d25/d94/dd0/de6/l112 0 2024-04-23T02:14:06.469 INFO:tasks.workunit.client.0.smithi071.stdout:6/602: dwrite d6/d17/d77/d85/f9b [0,4194304] 0 2024-04-23T02:14:06.469 INFO:tasks.workunit.client.0.smithi071.stdout:1/571: rmdir db/df/d13/d2f/d7b 39 2024-04-23T02:14:06.471 INFO:tasks.workunit.client.1.smithi192.stdout:0/825: mkdir d6/df/d35/d8b/d113 0 2024-04-23T02:14:06.472 INFO:tasks.workunit.client.0.smithi071.stdout:6/603: write f4 [4522656,53460] 0 2024-04-23T02:14:06.472 INFO:tasks.workunit.client.0.smithi071.stdout:5/701: dwrite d3/da/d1a/f2b [0,4194304] 0 2024-04-23T02:14:06.474 INFO:tasks.workunit.client.0.smithi071.stdout:1/572: truncate db/d11/d7f/faf 243097 0 2024-04-23T02:14:06.474 INFO:tasks.workunit.client.1.smithi192.stdout:6/837: dread d3/d15/d1b/f8f [0,4194304] 0 2024-04-23T02:14:06.475 INFO:tasks.workunit.client.1.smithi192.stdout:0/826: creat d6/df/d13/d25/d86/f114 x:0 0 0 2024-04-23T02:14:06.475 INFO:tasks.workunit.client.0.smithi071.stdout:0/641: symlink d0/d3/de/d48/d7e/le4 0 2024-04-23T02:14:06.476 INFO:tasks.workunit.client.1.smithi192.stdout:8/953: dwrite d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/d108/d11f/f124 [0,4194304] 0 2024-04-23T02:14:06.477 INFO:tasks.workunit.client.1.smithi192.stdout:8/954: chown d1/d9/d3e/d59/d8a/daa 335530 1 2024-04-23T02:14:06.479 INFO:tasks.workunit.client.0.smithi071.stdout:5/702: creat d3/d1d/daf/d10/da5/d5f/dc0/f105 x:0 0 0 2024-04-23T02:14:06.480 INFO:tasks.workunit.client.0.smithi071.stdout:0/642: chown d0/d3/f62 23523 1 2024-04-23T02:14:06.481 INFO:tasks.workunit.client.1.smithi192.stdout:8/955: read - d1/d12/d23/f120 zero size 2024-04-23T02:14:06.481 INFO:tasks.workunit.client.1.smithi192.stdout:6/838: mknod d3/d28/d79/dbe/dcb/de9/c114 0 2024-04-23T02:14:06.481 INFO:tasks.workunit.client.0.smithi071.stdout:5/703: chown d3/d1d/daf/d10/da5/d5f/dc0/dfa 139 1 2024-04-23T02:14:06.482 INFO:tasks.workunit.client.1.smithi192.stdout:8/956: chown d1/d9/d3e/c10e 2 1 2024-04-23T02:14:06.483 INFO:tasks.workunit.client.0.smithi071.stdout:5/704: chown d3/d1d/d43/f95 256586692 1 2024-04-23T02:14:06.483 INFO:tasks.workunit.client.0.smithi071.stdout:1/573: read f5 [3352497,76078] 0 2024-04-23T02:14:06.485 INFO:tasks.workunit.client.1.smithi192.stdout:6/839: fsync d3/d18/dd4/d8d/f9e 0 2024-04-23T02:14:06.485 INFO:tasks.workunit.client.1.smithi192.stdout:8/957: write d1/f10 [6640544,81040] 0 2024-04-23T02:14:06.489 INFO:tasks.workunit.client.0.smithi071.stdout:5/705: dread d3/d1d/f71 [0,4194304] 0 2024-04-23T02:14:06.491 INFO:tasks.workunit.client.1.smithi192.stdout:5/723: write d2/d6/f8d [540441,20805] 0 2024-04-23T02:14:06.492 INFO:tasks.workunit.client.0.smithi071.stdout:5/706: mkdir d3/da/d1a/db8/d106 0 2024-04-23T02:14:06.494 INFO:tasks.workunit.client.1.smithi192.stdout:4/823: write de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/de7/d101/ff0 [1406145,11685] 0 2024-04-23T02:14:06.495 INFO:tasks.workunit.client.0.smithi071.stdout:5/707: truncate d3/da/f92 585614 0 2024-04-23T02:14:06.495 INFO:tasks.workunit.client.0.smithi071.stdout:5/708: fdatasync d3/d1d/d43/dd2/ff8 0 2024-04-23T02:14:06.497 INFO:tasks.workunit.client.0.smithi071.stdout:5/709: symlink d3/da/d1a/l107 0 2024-04-23T02:14:06.498 INFO:tasks.workunit.client.1.smithi192.stdout:6/840: getdents d3/d28/d79/da0/dc2/d101/d80/dbb 0 2024-04-23T02:14:06.499 INFO:tasks.workunit.client.0.smithi071.stdout:5/710: creat d3/d1d/db9/f108 x:0 0 0 2024-04-23T02:14:06.501 INFO:tasks.workunit.client.1.smithi192.stdout:2/754: dwrite d8/d25/f79 [0,4194304] 0 2024-04-23T02:14:06.503 INFO:tasks.workunit.client.0.smithi071.stdout:6/604: truncate d6/d17/d77/d85/fab 3171409 0 2024-04-23T02:14:06.503 INFO:tasks.workunit.client.1.smithi192.stdout:6/841: truncate d3/d11/f43 992879 0 2024-04-23T02:14:06.504 INFO:tasks.workunit.client.0.smithi071.stdout:5/711: creat d3/d1d/d43/f109 x:0 0 0 2024-04-23T02:14:06.506 INFO:tasks.workunit.client.0.smithi071.stdout:6/605: truncate d6/d17/d77/f96 181339 0 2024-04-23T02:14:06.507 INFO:tasks.workunit.client.1.smithi192.stdout:0/827: getdents d6/df/d13/d25/d94/dd0/de6 0 2024-04-23T02:14:06.507 INFO:tasks.workunit.client.0.smithi071.stdout:4/722: sync 2024-04-23T02:14:06.507 INFO:tasks.workunit.client.0.smithi071.stdout:4/723: chown d2/db9 1813 1 2024-04-23T02:14:06.508 INFO:tasks.workunit.client.1.smithi192.stdout:2/755: mknod d8/d26/d27/d31/dae/cf9 0 2024-04-23T02:14:06.509 INFO:tasks.workunit.client.1.smithi192.stdout:2/756: chown d8/d26/d27/d53/f5b 2923 1 2024-04-23T02:14:06.510 INFO:tasks.workunit.client.0.smithi071.stdout:1/574: dwrite db/d34/d68/f80 [0,4194304] 0 2024-04-23T02:14:06.510 INFO:tasks.workunit.client.0.smithi071.stdout:3/575: dread d7/d8/f11 [0,4194304] 0 2024-04-23T02:14:06.511 INFO:tasks.workunit.client.1.smithi192.stdout:2/757: dread - d8/d26/d27/d57/da0/ff7 zero size 2024-04-23T02:14:06.512 INFO:tasks.workunit.client.1.smithi192.stdout:0/828: unlink d6/d4b/c102 0 2024-04-23T02:14:06.512 INFO:tasks.workunit.client.0.smithi071.stdout:1/575: chown db/df/d5f/l28 13 1 2024-04-23T02:14:06.513 INFO:tasks.workunit.client.0.smithi071.stdout:3/576: rename d7 to d7/d1d/dbc/dc5 22 2024-04-23T02:14:06.513 INFO:tasks.workunit.client.1.smithi192.stdout:0/829: chown d6/df/d35/d8b/d8d/dea/cec 82591104 1 2024-04-23T02:14:06.515 INFO:tasks.workunit.client.0.smithi071.stdout:5/712: dread d3/d1d/daf/d83/f9b [0,4194304] 0 2024-04-23T02:14:06.515 INFO:tasks.workunit.client.0.smithi071.stdout:3/577: mkdir d7/da5/db2/db7/d68/dae/dc6 0 2024-04-23T02:14:06.516 INFO:tasks.workunit.client.1.smithi192.stdout:2/758: symlink d8/d26/d27/d57/d69/d99/lfa 0 2024-04-23T02:14:06.517 INFO:tasks.workunit.client.1.smithi192.stdout:6/842: link d3/d18/f9f d3/d28/d79/da0/dc2/d101/d80/d8e/f115 0 2024-04-23T02:14:06.517 INFO:tasks.workunit.client.0.smithi071.stdout:1/576: dread db/f54 [0,4194304] 0 2024-04-23T02:14:06.518 INFO:tasks.workunit.client.1.smithi192.stdout:8/958: write d1/d12/d7d/f89 [4854617,48388] 0 2024-04-23T02:14:06.520 INFO:tasks.workunit.client.1.smithi192.stdout:0/830: rename d6/df/f4a to d6/df/d35/d8b/f115 0 2024-04-23T02:14:06.521 INFO:tasks.workunit.client.0.smithi071.stdout:5/713: creat d3/d1d/f10a x:0 0 0 2024-04-23T02:14:06.522 INFO:tasks.workunit.client.1.smithi192.stdout:6/843: mkdir d3/d18/d91/d116 0 2024-04-23T02:14:06.524 INFO:tasks.workunit.client.1.smithi192.stdout:4/824: write de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/de7/d101/f8d [1325104,35107] 0 2024-04-23T02:14:06.525 INFO:tasks.workunit.client.1.smithi192.stdout:2/759: rename d8/d26/d29/d43/l66 to d8/d26/d27/d31/dae/lfb 0 2024-04-23T02:14:06.527 INFO:tasks.workunit.client.1.smithi192.stdout:0/831: mkdir d6/df/d13/d25/d3a/da6/d116 0 2024-04-23T02:14:06.527 INFO:tasks.workunit.client.0.smithi071.stdout:1/577: dwrite db/d34/d68/f76 [0,4194304] 0 2024-04-23T02:14:06.529 INFO:tasks.workunit.client.0.smithi071.stdout:5/714: rename d3/d1d/daf/d10/f48 to d3/da/d1a/d1c/f10b 0 2024-04-23T02:14:06.532 INFO:tasks.workunit.client.1.smithi192.stdout:5/724: dwrite d2/f5 [4194304,4194304] 0 2024-04-23T02:14:06.533 INFO:tasks.workunit.client.0.smithi071.stdout:3/578: write d7/d22/d23/f6b [305726,87113] 0 2024-04-23T02:14:06.533 INFO:tasks.workunit.client.0.smithi071.stdout:4/724: dwrite d2/d2a/da2/dd1/dde/d57/f72 [0,4194304] 0 2024-04-23T02:14:06.539 INFO:tasks.workunit.client.0.smithi071.stdout:3/579: fdatasync d7/d8/d42/d80/d88/f57 0 2024-04-23T02:14:06.539 INFO:tasks.workunit.client.1.smithi192.stdout:0/832: creat d6/df/d56/db1/f117 x:0 0 0 2024-04-23T02:14:06.541 INFO:tasks.workunit.client.1.smithi192.stdout:4/825: creat de/d15/d19/d1a/d95/dd5/f10e x:0 0 0 2024-04-23T02:14:06.543 INFO:tasks.workunit.client.0.smithi071.stdout:1/578: dwrite db/d11/f64 [0,4194304] 0 2024-04-23T02:14:06.544 INFO:tasks.workunit.client.1.smithi192.stdout:2/760: link d8/d26/d29/f41 d8/d26/d29/da4/dca/ffc 0 2024-04-23T02:14:06.546 INFO:tasks.workunit.client.1.smithi192.stdout:0/833: mknod d6/df/d13/d6b/d73/c118 0 2024-04-23T02:14:06.548 INFO:tasks.workunit.client.0.smithi071.stdout:1/579: creat db/d11/d8d/d97/fb9 x:0 0 0 2024-04-23T02:14:06.549 INFO:tasks.workunit.client.0.smithi071.stdout:4/725: dwrite d2/d2a/d2d/d39/fd8 [0,4194304] 0 2024-04-23T02:14:06.549 INFO:tasks.workunit.client.1.smithi192.stdout:6/844: write d3/d18/f78 [5205740,55553] 0 2024-04-23T02:14:06.549 INFO:tasks.workunit.client.0.smithi071.stdout:4/726: chown d2/d2a/daa 133149 1 2024-04-23T02:14:06.550 INFO:tasks.workunit.client.1.smithi192.stdout:2/761: creat d8/d26/d29/d43/d5d/dbd/ffd x:0 0 0 2024-04-23T02:14:06.552 INFO:tasks.workunit.client.1.smithi192.stdout:4/826: symlink de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/l10f 0 2024-04-23T02:14:06.553 INFO:tasks.workunit.client.1.smithi192.stdout:4/827: stat de 0 2024-04-23T02:14:06.553 INFO:tasks.workunit.client.1.smithi192.stdout:0/834: fsync d6/df/d35/d49/f54 0 2024-04-23T02:14:06.557 INFO:tasks.workunit.client.1.smithi192.stdout:0/835: chown d6/df/d35/d8b/d113 107705 1 2024-04-23T02:14:06.558 INFO:tasks.workunit.client.0.smithi071.stdout:4/727: getdents d2/d2a/da2 0 2024-04-23T02:14:06.558 INFO:tasks.workunit.client.0.smithi071.stdout:1/580: dwrite db/df/d13/d2b/f7e [0,4194304] 0 2024-04-23T02:14:06.560 INFO:tasks.workunit.client.1.smithi192.stdout:3/646: sync 2024-04-23T02:14:06.562 INFO:tasks.workunit.client.1.smithi192.stdout:2/762: dread d8/d26/d29/d43/f7f [0,4194304] 0 2024-04-23T02:14:06.563 INFO:tasks.workunit.client.1.smithi192.stdout:4/828: creat de/d15/d53/d81/de3/d74/d92/f110 x:0 0 0 2024-04-23T02:14:06.564 INFO:tasks.workunit.client.0.smithi071.stdout:4/728: creat d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/dc8/fe8 x:0 0 0 2024-04-23T02:14:06.565 INFO:tasks.workunit.client.1.smithi192.stdout:5/725: dwrite d2/d6/f68 [0,4194304] 0 2024-04-23T02:14:06.566 INFO:tasks.workunit.client.0.smithi071.stdout:1/581: rename db/df/d5f/d6f/db2/d86/ca2 to db/df/d5f/d57/cba 0 2024-04-23T02:14:06.569 INFO:tasks.workunit.client.0.smithi071.stdout:3/580: write d7/da5/db2/db7/f8c [146225,130051] 0 2024-04-23T02:14:06.569 INFO:tasks.workunit.client.1.smithi192.stdout:3/647: mkdir d1/d3/d17/d1a/d7c/dab/d6c/d84/d4c/de3 0 2024-04-23T02:14:06.572 INFO:tasks.workunit.client.1.smithi192.stdout:8/959: dread d1/f4a [4194304,4194304] 0 2024-04-23T02:14:06.574 INFO:tasks.workunit.client.1.smithi192.stdout:0/836: rename d6/df/d13/d25/d26/l53 to d6/df/d47/d55/l119 0 2024-04-23T02:14:06.575 INFO:tasks.workunit.client.1.smithi192.stdout:5/726: fdatasync d2/db/d2b/f7f 0 2024-04-23T02:14:06.575 INFO:tasks.workunit.client.0.smithi071.stdout:4/729: symlink d2/d2a/da2/dd1/dde/d23/d4f/db5/db6/le9 0 2024-04-23T02:14:06.577 INFO:tasks.workunit.client.0.smithi071.stdout:3/581: read d7/da5/db2/db7/d68/d3b/d4c/f65 [64276,46688] 0 2024-04-23T02:14:06.577 INFO:tasks.workunit.client.0.smithi071.stdout:5/715: dwrite d3/da/d1a/d1c/f2e [0,4194304] 0 2024-04-23T02:14:06.577 INFO:tasks.workunit.client.1.smithi192.stdout:3/648: fdatasync d1/d3/d17/d1a/d7c/dab/d6c/f90 0 2024-04-23T02:14:06.579 INFO:tasks.workunit.client.1.smithi192.stdout:8/960: write d1/d9/d3e/fb2 [1363774,31062] 0 2024-04-23T02:14:06.580 INFO:tasks.workunit.client.0.smithi071.stdout:4/730: write d2/d2a/d2d/f5c [38985,87883] 0 2024-04-23T02:14:06.581 INFO:tasks.workunit.client.0.smithi071.stdout:5/716: chown d3/d1d/d43/d4b/cad 9122 1 2024-04-23T02:14:06.581 INFO:tasks.workunit.client.0.smithi071.stdout:3/582: mknod d7/d8/d42/d80/d88/cc7 0 2024-04-23T02:14:06.582 INFO:tasks.workunit.client.0.smithi071.stdout:1/582: dread db/df/d13/d2f/d7b/f9d [0,4194304] 0 2024-04-23T02:14:06.583 INFO:tasks.workunit.client.0.smithi071.stdout:4/731: rmdir d2/d2a/daa 39 2024-04-23T02:14:06.583 INFO:tasks.workunit.client.0.smithi071.stdout:3/583: chown d7/d10/d38/daa 549547938 1 2024-04-23T02:14:06.585 INFO:tasks.workunit.client.0.smithi071.stdout:3/584: fsync d7/d22/da7/f9b 0 2024-04-23T02:14:06.585 INFO:tasks.workunit.client.0.smithi071.stdout:5/717: creat d3/d1d/daf/d10/da5/d5f/dc0/f10c x:0 0 0 2024-04-23T02:14:06.587 INFO:tasks.workunit.client.1.smithi192.stdout:2/763: dwrite d8/d26/d29/d43/d5d/fd5 [4194304,4194304] 0 2024-04-23T02:14:06.588 INFO:tasks.workunit.client.1.smithi192.stdout:4/829: creat de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/f111 x:0 0 0 2024-04-23T02:14:06.588 INFO:tasks.workunit.client.0.smithi071.stdout:1/583: creat db/df/d5f/d6f/db2/fbb x:0 0 0 2024-04-23T02:14:06.591 INFO:tasks.workunit.client.1.smithi192.stdout:0/837: link d6/df/d13/d25/d26/f82 d6/df/d35/d8b/f11a 0 2024-04-23T02:14:06.592 INFO:tasks.workunit.client.0.smithi071.stdout:1/584: mkdir db/df/d5f/d6f/db2/d86/dbc 0 2024-04-23T02:14:06.594 INFO:tasks.workunit.client.1.smithi192.stdout:2/764: rmdir d8/d26/d29/d43/d5d 39 2024-04-23T02:14:06.594 INFO:tasks.workunit.client.0.smithi071.stdout:3/585: link d7/da5/db2/db7/d68/d3b/d4c/f59 d7/d8/d42/d47/fc8 0 2024-04-23T02:14:06.595 INFO:tasks.workunit.client.1.smithi192.stdout:3/649: mkdir d1/d3/d17/de4 0 2024-04-23T02:14:06.596 INFO:tasks.workunit.client.0.smithi071.stdout:1/585: mknod db/df/d13/d20/cbd 0 2024-04-23T02:14:06.596 INFO:tasks.workunit.client.1.smithi192.stdout:0/838: mknod d6/df/d13/d6b/d73/c11b 0 2024-04-23T02:14:06.597 INFO:tasks.workunit.client.0.smithi071.stdout:3/586: chown d7/c37 2016 1 2024-04-23T02:14:06.597 INFO:tasks.workunit.client.1.smithi192.stdout:6/845: dread d3/d11/f29 [0,4194304] 0 2024-04-23T02:14:06.598 INFO:tasks.workunit.client.1.smithi192.stdout:0/839: write d6/df/d56/ff3 [698364,109103] 0 2024-04-23T02:14:06.599 INFO:tasks.workunit.client.1.smithi192.stdout:6/846: readlink d3/d28/d79/da0/dc2/lea 0 2024-04-23T02:14:06.600 INFO:tasks.workunit.client.0.smithi071.stdout:3/587: unlink d7/da5/db2/db7/d68/d3b/d4c/da1/da8/ca9 0 2024-04-23T02:14:06.600 INFO:tasks.workunit.client.1.smithi192.stdout:4/830: symlink de/d15/d19/d1a/d95/dd5/l112 0 2024-04-23T02:14:06.605 INFO:tasks.workunit.client.1.smithi192.stdout:3/650: creat d1/d3/d17/d65/fe5 x:0 0 0 2024-04-23T02:14:06.606 INFO:tasks.workunit.client.0.smithi071.stdout:3/588: dwrite d7/d1d/f72 [8388608,4194304] 0 2024-04-23T02:14:06.607 INFO:tasks.workunit.client.1.smithi192.stdout:6/847: readlink d3/d28/df8/d3a/l46 0 2024-04-23T02:14:06.608 INFO:tasks.workunit.client.1.smithi192.stdout:2/765: creat d8/d26/d29/d43/ffe x:0 0 0 2024-04-23T02:14:06.609 INFO:tasks.workunit.client.1.smithi192.stdout:0/840: dwrite f5 [4194304,4194304] 0 2024-04-23T02:14:06.609 INFO:tasks.workunit.client.1.smithi192.stdout:4/831: symlink de/d15/d53/d81/de3/dfa/l113 0 2024-04-23T02:14:06.611 INFO:tasks.workunit.client.0.smithi071.stdout:3/589: creat d7/fc9 x:0 0 0 2024-04-23T02:14:06.614 INFO:tasks.workunit.client.1.smithi192.stdout:2/766: write d8/d25/f3f [996478,27904] 0 2024-04-23T02:14:06.619 INFO:tasks.workunit.client.0.smithi071.stdout:4/732: dread d2/d2a/d2d/d39/d42/d49/d67/d3e/f4d [0,4194304] 0 2024-04-23T02:14:06.621 INFO:tasks.workunit.client.0.smithi071.stdout:4/733: chown d2/d2a/d2d/d39/db3/ce6 59033399 1 2024-04-23T02:14:06.622 INFO:tasks.workunit.client.1.smithi192.stdout:5/727: dwrite d2/db/f13 [0,4194304] 0 2024-04-23T02:14:06.622 INFO:tasks.workunit.client.0.smithi071.stdout:4/734: chown d2/d2a/da2/dd1/dde/d23/f16 2591 1 2024-04-23T02:14:06.624 INFO:tasks.workunit.client.1.smithi192.stdout:5/728: write d2/db/d2b/d52/dc2/d49/ff1 [74084,41426] 0 2024-04-23T02:14:06.625 INFO:tasks.workunit.client.0.smithi071.stdout:4/735: mknod d2/d2a/d2d/d39/d42/d49/d67/d3e/cea 0 2024-04-23T02:14:06.626 INFO:tasks.workunit.client.1.smithi192.stdout:6/848: symlink d3/d28/d79/da0/l117 0 2024-04-23T02:14:06.627 INFO:tasks.workunit.client.0.smithi071.stdout:4/736: fdatasync d2/d2a/da2/dd1/dde/d23/d29/fd3 0 2024-04-23T02:14:06.628 INFO:tasks.workunit.client.0.smithi071.stdout:5/718: dread d3/d1d/daf/d10/da5/f66 [0,4194304] 0 2024-04-23T02:14:06.628 INFO:tasks.workunit.client.1.smithi192.stdout:6/849: write d3/d28/d79/da0/dc2/d101/d80/dbb/fd6 [1008686,17080] 0 2024-04-23T02:14:06.629 INFO:tasks.workunit.client.1.smithi192.stdout:2/767: symlink d8/d26/d27/d53/da2/dc1/lff 0 2024-04-23T02:14:06.631 INFO:tasks.workunit.client.0.smithi071.stdout:4/737: mkdir d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/dc8/deb 0 2024-04-23T02:14:06.632 INFO:tasks.workunit.client.1.smithi192.stdout:8/961: dread d1/d12/d7d/fce [0,4194304] 0 2024-04-23T02:14:06.633 INFO:tasks.workunit.client.1.smithi192.stdout:6/850: mknod d3/d28/d79/da0/c118 0 2024-04-23T02:14:06.635 INFO:tasks.workunit.client.1.smithi192.stdout:6/851: stat d3/d15/d1b/da2 0 2024-04-23T02:14:06.635 INFO:tasks.workunit.client.1.smithi192.stdout:4/832: dread de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/de7/d101/f8d [0,4194304] 0 2024-04-23T02:14:06.635 INFO:tasks.workunit.client.0.smithi071.stdout:5/719: link d3/d1d/daf/d10/da5/d5f/c8e d3/d1d/daf/d10/da5/d5f/dc0/c10d 0 2024-04-23T02:14:06.637 INFO:tasks.workunit.client.0.smithi071.stdout:4/738: getdents d2/d2a/da2/dd1/dde/d23/d4f/d8b 0 2024-04-23T02:14:06.638 INFO:tasks.workunit.client.0.smithi071.stdout:5/720: mknod d3/d1d/c10e 0 2024-04-23T02:14:06.639 INFO:tasks.workunit.client.1.smithi192.stdout:6/852: mknod d3/d15/d5b/c119 0 2024-04-23T02:14:06.639 INFO:tasks.workunit.client.0.smithi071.stdout:5/721: write d3/d34/d6b/fa3 [4014969,1467] 0 2024-04-23T02:14:06.641 INFO:tasks.workunit.client.0.smithi071.stdout:5/722: fsync d3/d1d/d43/d4b/ffc 0 2024-04-23T02:14:06.644 INFO:tasks.workunit.client.1.smithi192.stdout:8/962: link d1/d12/d23/d4b/d73/d45/f12e d1/d12/d23/d4b/d73/d7e/d88/f14c 0 2024-04-23T02:14:06.645 INFO:tasks.workunit.client.0.smithi071.stdout:4/739: dwrite d2/d2a/d2d/d39/d42/d49/d67/f1e [0,4194304] 0 2024-04-23T02:14:06.646 INFO:tasks.workunit.client.1.smithi192.stdout:8/963: unlink d1/d9/d3e/d59/d8a/daa/l136 0 2024-04-23T02:14:06.648 INFO:tasks.workunit.client.1.smithi192.stdout:4/833: rename de/d15/d53/f58 to de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/f114 0 2024-04-23T02:14:06.648 INFO:tasks.workunit.client.1.smithi192.stdout:6/853: read d3/d15/d1b/f72 [1959825,129911] 0 2024-04-23T02:14:06.655 INFO:tasks.workunit.client.1.smithi192.stdout:3/651: truncate d1/d3/d17/d2b/f51 380205 0 2024-04-23T02:14:06.658 INFO:tasks.workunit.client.1.smithi192.stdout:4/834: creat de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/f115 x:0 0 0 2024-04-23T02:14:06.661 INFO:tasks.workunit.client.1.smithi192.stdout:3/652: symlink d1/d3/d6/d7/d29/d3b/da9/dbd/le6 0 2024-04-23T02:14:06.662 INFO:tasks.workunit.client.1.smithi192.stdout:2/768: write d8/d26/d29/f41 [4438192,22010] 0 2024-04-23T02:14:06.663 INFO:tasks.workunit.client.0.smithi071.stdout:5/723: dread d3/d1d/d43/f69 [0,4194304] 0 2024-04-23T02:14:06.665 INFO:tasks.workunit.client.0.smithi071.stdout:5/724: stat d3/da/d1a/d1c/d46/d4e/f50 0 2024-04-23T02:14:06.666 INFO:tasks.workunit.client.1.smithi192.stdout:2/769: symlink d8/d26/d74/l100 0 2024-04-23T02:14:06.667 INFO:tasks.workunit.client.1.smithi192.stdout:2/770: write d8/d25/f3a [2463688,126846] 0 2024-04-23T02:14:06.673 INFO:tasks.workunit.client.1.smithi192.stdout:8/964: write d1/d12/d23/d4b/d73/d4c/d82/fb8 [58564,101077] 0 2024-04-23T02:14:06.673 INFO:tasks.workunit.client.1.smithi192.stdout:3/653: mknod d1/d3/d17/d1a/d7c/dab/d6c/d84/ce7 0 2024-04-23T02:14:06.679 INFO:tasks.workunit.client.1.smithi192.stdout:3/654: mknod d1/d3/d6/d7/d5f/ce8 0 2024-04-23T02:14:06.679 INFO:tasks.workunit.client.0.smithi071.stdout:4/740: creat d2/d2a/da2/dd1/dde/d23/d10/fec x:0 0 0 2024-04-23T02:14:06.680 INFO:tasks.workunit.client.1.smithi192.stdout:2/771: dwrite d8/d26/d27/d53/f64 [0,4194304] 0 2024-04-23T02:14:06.681 INFO:tasks.workunit.client.0.smithi071.stdout:4/741: write d2/d2a/d2d/d39/d42/d49/d67/d3e/f93 [1485561,70002] 0 2024-04-23T02:14:06.686 INFO:tasks.workunit.client.1.smithi192.stdout:2/772: symlink d8/d26/d29/d80/l101 0 2024-04-23T02:14:06.686 INFO:tasks.workunit.client.1.smithi192.stdout:3/655: creat d1/d3/d17/dd1/fe9 x:0 0 0 2024-04-23T02:14:06.690 INFO:tasks.workunit.client.1.smithi192.stdout:2/773: unlink d8/d26/d29/d80/l101 0 2024-04-23T02:14:06.691 INFO:tasks.workunit.client.1.smithi192.stdout:3/656: rmdir d1/d3/d17/d2b/d9a 39 2024-04-23T02:14:06.694 INFO:tasks.workunit.client.1.smithi192.stdout:2/774: rmdir d8/d26/d29/d43 39 2024-04-23T02:14:06.695 INFO:tasks.workunit.client.1.smithi192.stdout:8/965: dread d1/d9/fa [0,4194304] 0 2024-04-23T02:14:06.695 INFO:tasks.workunit.client.1.smithi192.stdout:3/657: symlink d1/d3/d6/d7/d29/d3b/lea 0 2024-04-23T02:14:06.700 INFO:tasks.workunit.client.1.smithi192.stdout:3/658: chown d1/d3/d17/fdc 8042244 1 2024-04-23T02:14:06.700 INFO:tasks.workunit.client.1.smithi192.stdout:4/835: dread de/d15/d53/d81/de3/db2/f86 [0,4194304] 0 2024-04-23T02:14:06.700 INFO:tasks.workunit.client.1.smithi192.stdout:8/966: chown d1/d12/d23/d4b/d73/d4c/f137 442914 1 2024-04-23T02:14:06.703 INFO:tasks.workunit.client.1.smithi192.stdout:2/775: getdents d8/d26/d27/d53/da2 0 2024-04-23T02:14:06.704 INFO:tasks.workunit.client.1.smithi192.stdout:3/659: dread - d1/d3/d17/d1a/d7c/dab/d6c/d84/d5d/fa7 zero size 2024-04-23T02:14:06.705 INFO:tasks.workunit.client.1.smithi192.stdout:8/967: unlink d1/d9/d8f/d90/fa2 0 2024-04-23T02:14:06.710 INFO:tasks.workunit.client.1.smithi192.stdout:2/776: truncate d8/d26/d27/d57/d69/f6a 477992 0 2024-04-23T02:14:06.717 INFO:tasks.workunit.client.1.smithi192.stdout:8/968: getdents d1/d12/d23/d4b/d73/dda/d10f 0 2024-04-23T02:14:06.718 INFO:tasks.workunit.client.1.smithi192.stdout:2/777: unlink d8/d26/d74/le2 0 2024-04-23T02:14:06.721 INFO:tasks.workunit.client.1.smithi192.stdout:8/969: mknod d1/d12/d23/d2c/c14d 0 2024-04-23T02:14:06.725 INFO:tasks.workunit.client.1.smithi192.stdout:4/836: dread de/d15/d53/d81/de3/d74/d92/da0/dca/de5/fe9 [0,4194304] 0 2024-04-23T02:14:06.726 INFO:tasks.workunit.client.1.smithi192.stdout:3/660: dread d1/d3/d17/d1a/d7c/dab/f7f [0,4194304] 0 2024-04-23T02:14:06.726 INFO:tasks.workunit.client.1.smithi192.stdout:8/970: dread - d1/d12/d23/d4b/d73/d7e/f102 zero size 2024-04-23T02:14:06.727 INFO:tasks.workunit.client.1.smithi192.stdout:2/778: dwrite d8/d26/d29/f41 [0,4194304] 0 2024-04-23T02:14:06.727 INFO:tasks.workunit.client.0.smithi071.stdout:2/598: sync 2024-04-23T02:14:06.727 INFO:tasks.workunit.client.0.smithi071.stdout:7/732: sync 2024-04-23T02:14:06.728 INFO:tasks.workunit.client.0.smithi071.stdout:2/599: chown d0/d2/d45/d58/c5d 538249063 1 2024-04-23T02:14:06.729 INFO:tasks.workunit.client.0.smithi071.stdout:7/733: creat d3/d51/df5/f100 x:0 0 0 2024-04-23T02:14:06.729 INFO:tasks.workunit.client.0.smithi071.stdout:2/600: read d0/d2/d26/d14/f1c [3216531,20878] 0 2024-04-23T02:14:06.730 INFO:tasks.workunit.client.1.smithi192.stdout:4/837: dread - de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/fd0 zero size 2024-04-23T02:14:06.730 INFO:tasks.workunit.client.0.smithi071.stdout:2/601: dread - d0/d82/f9d zero size 2024-04-23T02:14:06.730 INFO:tasks.workunit.client.0.smithi071.stdout:7/734: chown d3/d17/d41/d37/f6f 2637053 1 2024-04-23T02:14:06.732 INFO:tasks.workunit.client.1.smithi192.stdout:2/779: read d8/d26/d27/d53/fa3 [491849,112365] 0 2024-04-23T02:14:06.733 INFO:tasks.workunit.client.0.smithi071.stdout:7/735: chown d3/d17/d41/d37/f75 7562 1 2024-04-23T02:14:06.734 INFO:tasks.workunit.client.1.smithi192.stdout:3/661: creat d1/d3/d17/d1f/d33/d24/feb x:0 0 0 2024-04-23T02:14:06.734 INFO:tasks.workunit.client.1.smithi192.stdout:2/780: chown d8/d25/f9d 0 1 2024-04-23T02:14:06.734 INFO:tasks.workunit.client.1.smithi192.stdout:8/971: mkdir d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d139/d14e 0 2024-04-23T02:14:06.736 INFO:tasks.workunit.client.1.smithi192.stdout:3/662: chown d1/d3/d6/d7/d29/d3b/da9/d8b/l8f 2 1 2024-04-23T02:14:06.737 INFO:tasks.workunit.client.1.smithi192.stdout:4/838: mkdir de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/d116 0 2024-04-23T02:14:06.739 INFO:tasks.workunit.client.0.smithi071.stdout:2/602: creat d0/d2/fc4 x:0 0 0 2024-04-23T02:14:06.739 INFO:tasks.workunit.client.1.smithi192.stdout:8/972: creat d1/d9/d3e/f14f x:0 0 0 2024-04-23T02:14:06.741 INFO:tasks.workunit.client.1.smithi192.stdout:2/781: rename d8/d26/d29/c2d to d8/d14/c102 0 2024-04-23T02:14:06.741 INFO:tasks.workunit.client.0.smithi071.stdout:2/603: creat d0/d2/d34/d83/fc5 x:0 0 0 2024-04-23T02:14:06.742 INFO:tasks.workunit.client.0.smithi071.stdout:4/742: mkdir d2/d2a/d2d/d39/d42/d49/d67/ded 0 2024-04-23T02:14:06.744 INFO:tasks.workunit.client.0.smithi071.stdout:2/604: fdatasync d0/f2e 0 2024-04-23T02:14:06.745 INFO:tasks.workunit.client.1.smithi192.stdout:3/663: rename d1/d3/f19 to d1/d3/d6/d7/d29/d3b/da9/fec 0 2024-04-23T02:14:06.745 INFO:tasks.workunit.client.0.smithi071.stdout:2/605: chown d0/d70/f94 29477 1 2024-04-23T02:14:06.746 INFO:tasks.workunit.client.0.smithi071.stdout:5/725: rename d3/da/d1a/df1 to d3/d10f 0 2024-04-23T02:14:06.747 INFO:tasks.workunit.client.0.smithi071.stdout:2/606: truncate d0/d2/fa3 186380 0 2024-04-23T02:14:06.747 INFO:tasks.workunit.client.1.smithi192.stdout:4/839: symlink de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/l117 0 2024-04-23T02:14:06.749 INFO:tasks.workunit.client.1.smithi192.stdout:4/840: mknod de/d15/d53/c118 0 2024-04-23T02:14:06.750 INFO:tasks.workunit.client.0.smithi071.stdout:2/607: creat d0/d2/d26/d3a/dc3/d36/d6a/fc6 x:0 0 0 2024-04-23T02:14:06.751 INFO:tasks.workunit.client.0.smithi071.stdout:5/726: mkdir d3/d1d/daf/d110 0 2024-04-23T02:14:06.752 INFO:tasks.workunit.client.1.smithi192.stdout:4/841: rmdir de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/de7 39 2024-04-23T02:14:06.754 INFO:tasks.workunit.client.0.smithi071.stdout:5/727: rmdir d3/d86/de9 0 2024-04-23T02:14:06.758 INFO:tasks.workunit.client.1.smithi192.stdout:4/842: rename de/d15/d19/d1a/d95/dd5/l112 to de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/l119 0 2024-04-23T02:14:06.758 INFO:tasks.workunit.client.0.smithi071.stdout:4/743: dread d2/d2a/da2/dd1/dde/d23/f5d [0,4194304] 0 2024-04-23T02:14:06.759 INFO:tasks.workunit.client.1.smithi192.stdout:2/782: dread d8/d26/d29/d43/d50/d5e/f89 [0,4194304] 0 2024-04-23T02:14:06.761 INFO:tasks.workunit.client.1.smithi192.stdout:4/843: dread - de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/fb1 zero size 2024-04-23T02:14:06.764 INFO:tasks.workunit.client.0.smithi071.stdout:4/744: link d2/d2a/da2/dd1/dde/d23/l4a d2/d2a/da2/dd1/dde/d23/d10/lee 0 2024-04-23T02:14:06.767 INFO:tasks.workunit.client.0.smithi071.stdout:4/745: mkdir d2/d2a/d73/d5e/def 0 2024-04-23T02:14:06.773 INFO:tasks.workunit.client.0.smithi071.stdout:5/728: dread d3/d1d/d43/dd2/da9/f96 [0,4194304] 0 2024-04-23T02:14:06.775 INFO:tasks.workunit.client.0.smithi071.stdout:5/729: mknod d3/d1d/daf/d10/da5/d5f/c111 0 2024-04-23T02:14:06.778 INFO:tasks.workunit.client.0.smithi071.stdout:5/730: dread d3/d1d/d43/dd2/f58 [0,4194304] 0 2024-04-23T02:14:06.780 INFO:tasks.workunit.client.0.smithi071.stdout:5/731: mkdir d3/d1d/d43/d112 0 2024-04-23T02:14:06.782 INFO:tasks.workunit.client.1.smithi192.stdout:9/611: sync 2024-04-23T02:14:06.782 INFO:tasks.workunit.client.1.smithi192.stdout:1/504: sync 2024-04-23T02:14:06.783 INFO:tasks.workunit.client.1.smithi192.stdout:2/783: symlink d8/l103 0 2024-04-23T02:14:06.785 INFO:tasks.workunit.client.1.smithi192.stdout:8/973: dread d1/d12/d23/d4b/d73/d4c/f84 [0,4194304] 0 2024-04-23T02:14:06.785 INFO:tasks.workunit.client.1.smithi192.stdout:9/612: write d0/d9/f32 [2292867,35977] 0 2024-04-23T02:14:06.786 INFO:tasks.workunit.client.1.smithi192.stdout:8/974: readlink d1/d9/d8f/d90/lf9 0 2024-04-23T02:14:06.787 INFO:tasks.workunit.client.0.smithi071.stdout:4/746: write d2/d2a/da2/dd1/dde/d23/f71 [1040977,94787] 0 2024-04-23T02:14:06.787 INFO:tasks.workunit.client.0.smithi071.stdout:5/732: write d3/f29 [94126,14157] 0 2024-04-23T02:14:06.789 INFO:tasks.workunit.client.1.smithi192.stdout:1/505: mknod dc/d8b/d62/c9c 0 2024-04-23T02:14:06.790 INFO:tasks.workunit.client.1.smithi192.stdout:3/664: write d1/d3/d17/d1a/f23 [8681645,90766] 0 2024-04-23T02:14:06.791 INFO:tasks.workunit.client.0.smithi071.stdout:2/608: dread d0/d2/d26/d3a/f44 [0,4194304] 0 2024-04-23T02:14:06.791 INFO:tasks.workunit.client.1.smithi192.stdout:9/613: symlink d0/d9/db3/d9e/lcb 0 2024-04-23T02:14:06.792 INFO:tasks.workunit.client.0.smithi071.stdout:5/733: symlink d3/d1d/d43/d112/l113 0 2024-04-23T02:14:06.792 INFO:tasks.workunit.client.1.smithi192.stdout:8/975: mknod d1/d12/d68/ddf/c150 0 2024-04-23T02:14:06.793 INFO:tasks.workunit.client.1.smithi192.stdout:2/784: mknod d8/d26/d29/dcd/c104 0 2024-04-23T02:14:06.793 INFO:tasks.workunit.client.0.smithi071.stdout:2/609: symlink d0/d2/d26/d3a/dc3/d36/d5c/d85/lc7 0 2024-04-23T02:14:06.794 INFO:tasks.workunit.client.1.smithi192.stdout:8/976: fdatasync d1/d9/f22 0 2024-04-23T02:14:06.794 INFO:tasks.workunit.client.0.smithi071.stdout:4/747: truncate d2/d2a/d2d/d39/fa6 2082912 0 2024-04-23T02:14:06.795 INFO:tasks.workunit.client.1.smithi192.stdout:9/614: fdatasync d0/d9/d44/d4f/f3d 0 2024-04-23T02:14:06.796 INFO:tasks.workunit.client.1.smithi192.stdout:2/785: dread d8/d25/f3f [0,4194304] 0 2024-04-23T02:14:06.798 INFO:tasks.workunit.client.0.smithi071.stdout:5/734: link d3/d34/d6b/fa3 d3/d1d/d43/dd2/da9/db4/f114 0 2024-04-23T02:14:06.799 INFO:tasks.workunit.client.1.smithi192.stdout:9/615: dread d0/d9/d44/f9d [0,4194304] 0 2024-04-23T02:14:06.800 INFO:tasks.workunit.client.0.smithi071.stdout:4/748: creat d2/d2a/d73/d5e/ff0 x:0 0 0 2024-04-23T02:14:06.800 INFO:tasks.workunit.client.1.smithi192.stdout:8/977: creat d1/d12/d23/d2c/d4d/f151 x:0 0 0 2024-04-23T02:14:06.801 INFO:tasks.workunit.client.0.smithi071.stdout:2/610: dwrite d0/d2/d49/f5b [0,4194304] 0 2024-04-23T02:14:06.802 INFO:tasks.workunit.client.1.smithi192.stdout:1/506: creat dc/d14/f9d x:0 0 0 2024-04-23T02:14:06.803 INFO:tasks.workunit.client.0.smithi071.stdout:4/749: stat d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/fd5 0 2024-04-23T02:14:06.803 INFO:tasks.workunit.client.1.smithi192.stdout:1/507: fdatasync dc/d16/d19/d30/f84 0 2024-04-23T02:14:06.804 INFO:tasks.workunit.client.1.smithi192.stdout:1/508: dread - dc/d16/f64 zero size 2024-04-23T02:14:06.804 INFO:tasks.workunit.client.0.smithi071.stdout:2/611: symlink d0/d2/da0/db7/lc8 0 2024-04-23T02:14:06.804 INFO:tasks.workunit.client.1.smithi192.stdout:2/786: creat d8/d26/d29/da4/dab/f105 x:0 0 0 2024-04-23T02:14:06.806 INFO:tasks.workunit.client.0.smithi071.stdout:4/750: truncate d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/fcc 469260 0 2024-04-23T02:14:06.806 INFO:tasks.workunit.client.1.smithi192.stdout:1/509: truncate dc/d16/d19/f79 934848 0 2024-04-23T02:14:06.806 INFO:tasks.workunit.client.0.smithi071.stdout:2/612: fsync d0/f2e 0 2024-04-23T02:14:06.808 INFO:tasks.workunit.client.0.smithi071.stdout:4/751: dread - d2/d2a/d73/d5e/ff0 zero size 2024-04-23T02:14:06.808 INFO:tasks.workunit.client.1.smithi192.stdout:9/616: unlink d0/d9/d44/d4f/l35 0 2024-04-23T02:14:06.810 INFO:tasks.workunit.client.0.smithi071.stdout:4/752: dread - d2/d2a/d2d/d39/d42/d49/d67/d7e/d82/fb1 zero size 2024-04-23T02:14:06.810 INFO:tasks.workunit.client.1.smithi192.stdout:9/617: write d0/d9/d44/d74/f82 [3370755,124249] 0 2024-04-23T02:14:06.812 INFO:tasks.workunit.client.0.smithi071.stdout:5/735: write d3/d1d/daf/f89 [941916,65501] 0 2024-04-23T02:14:06.814 INFO:tasks.workunit.client.1.smithi192.stdout:3/665: write d1/d3/d17/d2b/f51 [867968,103562] 0 2024-04-23T02:14:06.815 INFO:tasks.workunit.client.1.smithi192.stdout:1/510: creat dc/d80/d81/f9e x:0 0 0 2024-04-23T02:14:06.815 INFO:tasks.workunit.client.0.smithi071.stdout:4/753: creat d2/d2a/da2/dd1/dde/d57/ff1 x:0 0 0 2024-04-23T02:14:06.816 INFO:tasks.workunit.client.1.smithi192.stdout:8/978: creat d1/d12/d23/d4b/d73/d45/f152 x:0 0 0 2024-04-23T02:14:06.816 INFO:tasks.workunit.client.0.smithi071.stdout:5/736: rename d3/d1d/daf/d10/da5/c104 to d3/da/d1a/d6d/c115 0 2024-04-23T02:14:06.817 INFO:tasks.workunit.client.1.smithi192.stdout:2/787: link d8/d14/f1a d8/d26/f106 0 2024-04-23T02:14:06.819 INFO:tasks.workunit.client.1.smithi192.stdout:9/618: unlink d0/d9/d44/d51/d5b/ca7 0 2024-04-23T02:14:06.819 INFO:tasks.workunit.client.0.smithi071.stdout:2/613: dwrite d0/d2/d26/d3a/dc3/d36/fbe [4194304,4194304] 0 2024-04-23T02:14:06.819 INFO:tasks.workunit.client.0.smithi071.stdout:4/754: mkdir d2/d2a/da2/dd1/dde/d57/df2 0 2024-04-23T02:14:06.821 INFO:tasks.workunit.client.0.smithi071.stdout:2/614: chown d0/fab 12 1 2024-04-23T02:14:06.821 INFO:tasks.workunit.client.0.smithi071.stdout:5/737: symlink d3/d8f/l116 0 2024-04-23T02:14:06.822 INFO:tasks.workunit.client.0.smithi071.stdout:5/738: chown d3/d1d/d43/dd2/fda 1945393 1 2024-04-23T02:14:06.822 INFO:tasks.workunit.client.0.smithi071.stdout:4/755: creat d2/d2a/d2d/d39/ff3 x:0 0 0 2024-04-23T02:14:06.823 INFO:tasks.workunit.client.1.smithi192.stdout:8/979: rename d1/d9/fea to d1/d12/d23/d4b/d73/d7e/d88/da9/db1/d148/f153 0 2024-04-23T02:14:06.823 INFO:tasks.workunit.client.0.smithi071.stdout:2/615: creat d0/d2/d34/d83/fc9 x:0 0 0 2024-04-23T02:14:06.825 INFO:tasks.workunit.client.1.smithi192.stdout:9/619: creat d0/d9/d44/d53/fcc x:0 0 0 2024-04-23T02:14:06.825 INFO:tasks.workunit.client.0.smithi071.stdout:4/756: write d2/d2a/da2/dd1/dde/d2c/f2e [4439593,105937] 0 2024-04-23T02:14:06.827 INFO:tasks.workunit.client.0.smithi071.stdout:5/739: truncate d3/da/d1a/d1c/f62 4471043 0 2024-04-23T02:14:06.828 INFO:tasks.workunit.client.1.smithi192.stdout:1/511: creat dc/d16/d19/f9f x:0 0 0 2024-04-23T02:14:06.829 INFO:tasks.workunit.client.1.smithi192.stdout:9/620: dread d0/f4e [0,4194304] 0 2024-04-23T02:14:06.830 INFO:tasks.workunit.client.0.smithi071.stdout:5/740: fsync d3/d1d/daf/d10/f14 0 2024-04-23T02:14:06.830 INFO:tasks.workunit.client.0.smithi071.stdout:2/616: link d0/d2/d45/d7d/d56/db0/l76 d0/d2/d45/d7d/d56/db0/lca 0 2024-04-23T02:14:06.832 INFO:tasks.workunit.client.1.smithi192.stdout:8/980: creat d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/f154 x:0 0 0 2024-04-23T02:14:06.832 INFO:tasks.workunit.client.0.smithi071.stdout:5/741: write d3/d1d/daf/d10/da5/feb [426757,37031] 0 2024-04-23T02:14:06.833 INFO:tasks.workunit.client.1.smithi192.stdout:9/621: symlink d0/d9/d44/d58/lcd 0 2024-04-23T02:14:06.835 INFO:tasks.workunit.client.1.smithi192.stdout:8/981: rmdir d1/d12/d23/d4b/d73/d7e 39 2024-04-23T02:14:06.836 INFO:tasks.workunit.client.0.smithi071.stdout:2/617: rename d0/d2/d45/d7d/d56/db0/l76 to d0/d2/d34/dbf/lcb 0 2024-04-23T02:14:06.838 INFO:tasks.workunit.client.1.smithi192.stdout:2/788: dread d8/d26/f6b [0,4194304] 0 2024-04-23T02:14:06.838 INFO:tasks.workunit.client.1.smithi192.stdout:1/512: rename dc/d8b/d34/d70/f85 to dc/d8b/fa0 0 2024-04-23T02:14:06.839 INFO:tasks.workunit.client.0.smithi071.stdout:2/618: unlink d0/d2/d26/d3a/f3d 0 2024-04-23T02:14:06.839 INFO:tasks.workunit.client.1.smithi192.stdout:3/666: write d1/d3/d17/d1a/d7c/dab/f92 [812509,44184] 0 2024-04-23T02:14:06.840 INFO:tasks.workunit.client.0.smithi071.stdout:5/742: dread d3/d34/d74/fdd [0,4194304] 0 2024-04-23T02:14:06.840 INFO:tasks.workunit.client.0.smithi071.stdout:2/619: chown d0/d2/d34/d83/fa4 1 1 2024-04-23T02:14:06.840 INFO:tasks.workunit.client.1.smithi192.stdout:9/622: mknod d0/d9/d44/d51/d5b/cce 0 2024-04-23T02:14:06.842 INFO:tasks.workunit.client.1.smithi192.stdout:3/667: rmdir d1 39 2024-04-23T02:14:06.843 INFO:tasks.workunit.client.0.smithi071.stdout:2/620: creat d0/d2/d45/d58/fcc x:0 0 0 2024-04-23T02:14:06.843 INFO:tasks.workunit.client.1.smithi192.stdout:1/513: rmdir dc/d8b/d34/d70 39 2024-04-23T02:14:06.843 INFO:tasks.workunit.client.0.smithi071.stdout:5/743: truncate d3/d1d/daf/d10/f60 1112568 0 2024-04-23T02:14:06.844 INFO:tasks.workunit.client.1.smithi192.stdout:8/982: mknod d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d139/c155 0 2024-04-23T02:14:06.846 INFO:tasks.workunit.client.1.smithi192.stdout:1/514: mknod dc/d16/d19/d74/ca1 0 2024-04-23T02:14:06.848 INFO:tasks.workunit.client.1.smithi192.stdout:8/983: creat d1/d12/d23/d4b/d73/d4c/d82/f156 x:0 0 0 2024-04-23T02:14:06.850 INFO:tasks.workunit.client.1.smithi192.stdout:1/515: mkdir dc/d14/d23/da2 0 2024-04-23T02:14:06.851 INFO:tasks.workunit.client.1.smithi192.stdout:8/984: fsync d1/d12/d68/ddf/ffe 0 2024-04-23T02:14:06.852 INFO:tasks.workunit.client.0.smithi071.stdout:5/744: write d3/d1d/d43/d4b/f72 [728060,17447] 0 2024-04-23T02:14:06.854 INFO:tasks.workunit.client.1.smithi192.stdout:1/516: truncate dc/d14/d15/d35/f37 5159694 0 2024-04-23T02:14:06.855 INFO:tasks.workunit.client.0.smithi071.stdout:5/745: rmdir d3/d34/d74 39 2024-04-23T02:14:06.855 INFO:tasks.workunit.client.1.smithi192.stdout:3/668: mkdir d1/d3/d6/d7/d29/d3b/da9/daf/ded 0 2024-04-23T02:14:06.856 INFO:tasks.workunit.client.1.smithi192.stdout:8/985: mkdir d1/d12/d23/d2c/d157 0 2024-04-23T02:14:06.857 INFO:tasks.workunit.client.1.smithi192.stdout:1/517: fdatasync dc/d14/d23/d5e/d6a/f9a 0 2024-04-23T02:14:06.857 INFO:tasks.workunit.client.1.smithi192.stdout:3/669: stat d1/d3/d17/d1a/d7c/dab 0 2024-04-23T02:14:06.858 INFO:tasks.workunit.client.1.smithi192.stdout:2/789: dwrite d8/d25/f9d [0,4194304] 0 2024-04-23T02:14:06.859 INFO:tasks.workunit.client.0.smithi071.stdout:5/746: dread d3/d1d/daf/d83/ff5 [0,4194304] 0 2024-04-23T02:14:06.859 INFO:tasks.workunit.client.1.smithi192.stdout:1/518: read f1 [2487297,3075] 0 2024-04-23T02:14:06.860 INFO:tasks.workunit.client.1.smithi192.stdout:8/986: truncate d1/d12/d23/f3c 7302962 0 2024-04-23T02:14:06.861 INFO:tasks.workunit.client.0.smithi071.stdout:5/747: mknod d3/d1d/d43/d4b/dce/c117 0 2024-04-23T02:14:06.862 INFO:tasks.workunit.client.1.smithi192.stdout:2/790: unlink d8/d26/d29/f41 0 2024-04-23T02:14:06.862 INFO:tasks.workunit.client.0.smithi071.stdout:5/748: mkdir d3/d1d/d43/d4b/d118 0 2024-04-23T02:14:06.863 INFO:tasks.workunit.client.1.smithi192.stdout:2/791: readlink d8/d26/d29/d43/l44 0 2024-04-23T02:14:06.863 INFO:tasks.workunit.client.1.smithi192.stdout:3/670: mknod d1/d3/d6/d7/d5f/cee 0 2024-04-23T02:14:06.864 INFO:tasks.workunit.client.0.smithi071.stdout:5/749: read d3/da/f33 [8353004,119656] 0 2024-04-23T02:14:06.865 INFO:tasks.workunit.client.0.smithi071.stdout:5/750: symlink d3/d1d/d43/d112/l119 0 2024-04-23T02:14:06.866 INFO:tasks.workunit.client.1.smithi192.stdout:8/987: readlink d1/d12/d23/d4b/d73/d7e/l8e 0 2024-04-23T02:14:06.867 INFO:tasks.workunit.client.1.smithi192.stdout:2/792: mkdir d8/d26/d27/d57/da0/d107 0 2024-04-23T02:14:06.867 INFO:tasks.workunit.client.0.smithi071.stdout:5/751: rmdir d3/d1d/daf/d10/da5/d5f/dc0 39 2024-04-23T02:14:06.868 INFO:tasks.workunit.client.1.smithi192.stdout:3/671: fdatasync d1/d3/d6/d7/d29/d3b/f8e 0 2024-04-23T02:14:06.869 INFO:tasks.workunit.client.1.smithi192.stdout:2/793: chown d8/d26/d29/da4/da8 44141 1 2024-04-23T02:14:06.869 INFO:tasks.workunit.client.1.smithi192.stdout:1/519: mkdir dc/d8b/d34/d70/d8c/da3 0 2024-04-23T02:14:06.875 INFO:tasks.workunit.client.0.smithi071.stdout:5/752: truncate d3/da/d1a/f1b 4808734 0 2024-04-23T02:14:06.875 INFO:tasks.workunit.client.0.smithi071.stdout:5/753: write d3/d1d/f3b [1306245,56078] 0 2024-04-23T02:14:06.876 INFO:tasks.workunit.client.1.smithi192.stdout:1/520: readlink dc/l1b 0 2024-04-23T02:14:06.877 INFO:tasks.workunit.client.1.smithi192.stdout:8/988: fdatasync d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/fec 0 2024-04-23T02:14:06.877 INFO:tasks.workunit.client.1.smithi192.stdout:2/794: write d8/d26/d27/d31/f8b [574929,123865] 0 2024-04-23T02:14:06.877 INFO:tasks.workunit.client.1.smithi192.stdout:3/672: symlink d1/d3/d17/d65/d99/lef 0 2024-04-23T02:14:06.877 INFO:tasks.workunit.client.1.smithi192.stdout:8/989: dread - d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/d9c/fec zero size 2024-04-23T02:14:06.877 INFO:tasks.workunit.client.1.smithi192.stdout:8/990: creat d1/d12/d68/f158 x:0 0 0 2024-04-23T02:14:06.878 INFO:tasks.workunit.client.1.smithi192.stdout:2/795: symlink d8/d26/d29/d43/d5d/dbd/l108 0 2024-04-23T02:14:06.879 INFO:tasks.workunit.client.1.smithi192.stdout:3/673: creat d1/d3/d6/d7/d29/d3b/da9/dbd/ff0 x:0 0 0 2024-04-23T02:14:06.881 INFO:tasks.workunit.client.1.smithi192.stdout:8/991: mknod d1/c159 0 2024-04-23T02:14:06.882 INFO:tasks.workunit.client.1.smithi192.stdout:3/674: fdatasync d1/d3/d17/d1f/d33/d24/f50 0 2024-04-23T02:14:06.882 INFO:tasks.workunit.client.1.smithi192.stdout:1/521: dwrite dc/d14/d23/f3d [0,4194304] 0 2024-04-23T02:14:06.884 INFO:tasks.workunit.client.1.smithi192.stdout:1/522: readlink dc/d14/d23/l45 0 2024-04-23T02:14:06.890 INFO:tasks.workunit.client.1.smithi192.stdout:2/796: dread d8/d26/d27/d53/da2/dc1/fbb [0,4194304] 0 2024-04-23T02:14:06.890 INFO:tasks.workunit.client.1.smithi192.stdout:8/992: mknod d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d8b/c15a 0 2024-04-23T02:14:06.897 INFO:tasks.workunit.client.1.smithi192.stdout:1/523: rename dc/d16/d19/d74/f8f to dc/d8b/d34/d70/d8c/fa4 0 2024-04-23T02:14:06.901 INFO:tasks.workunit.client.1.smithi192.stdout:2/797: creat d8/d26/d29/d43/d5d/dbd/dda/f109 x:0 0 0 2024-04-23T02:14:06.903 INFO:tasks.workunit.client.1.smithi192.stdout:8/993: creat d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/da3/d12d/d13b/f15b x:0 0 0 2024-04-23T02:14:06.906 INFO:tasks.workunit.client.1.smithi192.stdout:1/524: dwrite dc/d16/d19/d74/d77/f91 [0,4194304] 0 2024-04-23T02:14:06.907 INFO:tasks.workunit.client.1.smithi192.stdout:8/994: dread - d1/d12/d23/d4b/d73/d7e/d88/da9/d7c/d81/fe0 zero size 2024-04-23T02:14:06.908 INFO:tasks.workunit.client.1.smithi192.stdout:1/525: write dc/d14/d23/f87 [1097112,99706] 0 2024-04-23T02:14:06.910 INFO:tasks.workunit.client.1.smithi192.stdout:3/675: truncate d1/d3/f7a 4012413 0 2024-04-23T02:14:06.912 INFO:tasks.workunit.client.1.smithi192.stdout:2/798: getdents d8/d26/d27 0 2024-04-23T02:14:06.915 INFO:tasks.workunit.client.1.smithi192.stdout:8/995: creat d1/df/dd0/d13f/f15c x:0 0 0 2024-04-23T02:14:06.918 INFO:tasks.workunit.client.1.smithi192.stdout:2/799: rmdir d8/d26/d29/da4 39 2024-04-23T02:14:06.918 INFO:tasks.workunit.client.1.smithi192.stdout:3/676: dwrite d1/d3/d6/f10 [0,4194304] 0 2024-04-23T02:14:06.919 INFO:tasks.workunit.client.1.smithi192.stdout:8/996: write d1/d12/d23/d4b/d73/d45/d46/ffd [987175,118297] 0 2024-04-23T02:14:06.921 INFO:tasks.workunit.client.1.smithi192.stdout:1/526: mknod dc/d16/ca5 0 2024-04-23T02:14:06.924 INFO:tasks.workunit.client.1.smithi192.stdout:3/677: mkdir d1/d3/d6/d7/df1 0 2024-04-23T02:14:06.925 INFO:tasks.workunit.client.1.smithi192.stdout:2/800: creat d8/f10a x:0 0 0 2024-04-23T02:14:06.927 INFO:tasks.workunit.client.1.smithi192.stdout:8/997: symlink d1/d12/d23/d4b/d73/d4c/l15d 0 2024-04-23T02:14:06.928 INFO:tasks.workunit.client.1.smithi192.stdout:3/678: write d1/d3/d17/d1a/d7c/dab/f92 [1684331,7542] 0 2024-04-23T02:14:06.930 INFO:tasks.workunit.client.1.smithi192.stdout:2/801: unlink d8/l3d 0 2024-04-23T02:14:06.930 INFO:tasks.workunit.client.1.smithi192.stdout:3/679: write d1/d3/d6/d7/f82 [180877,75850] 0 2024-04-23T02:14:06.933 INFO:tasks.workunit.client.1.smithi192.stdout:8/998: rmdir d1/d9 39 2024-04-23T02:14:06.936 INFO:tasks.workunit.client.1.smithi192.stdout:3/680: symlink d1/d3/d6/d7/d29/d3b/da9/dbd/lf2 0 2024-04-23T02:14:06.941 INFO:tasks.workunit.client.1.smithi192.stdout:8/999: dwrite d1/d12/d23/d4b/d73/d45/d46/ffd [0,4194304] 0 2024-04-23T02:14:06.943 INFO:tasks.workunit.client.1.smithi192.stdout:2/802: creat d8/d26/d29/d43/f10b x:0 0 0 2024-04-23T02:14:06.944 INFO:tasks.workunit.client.1.smithi192.stdout:3/681: dread d1/d3/d17/f3d [0,4194304] 0 2024-04-23T02:14:06.945 INFO:tasks.workunit.client.0.smithi071.stdout:8/667: sync 2024-04-23T02:14:06.945 INFO:tasks.workunit.client.0.smithi071.stdout:0/643: sync 2024-04-23T02:14:06.945 INFO:tasks.workunit.client.0.smithi071.stdout:9/628: sync 2024-04-23T02:14:06.946 INFO:tasks.workunit.client.0.smithi071.stdout:8/668: fdatasync dc/d3a/f3c 0 2024-04-23T02:14:06.948 INFO:tasks.workunit.client.1.smithi192.stdout:1/527: dread dc/d14/f27 [0,4194304] 0 2024-04-23T02:14:06.948 INFO:tasks.workunit.client.0.smithi071.stdout:8/669: read dc/f20 [55294,57479] 0 2024-04-23T02:14:06.949 INFO:tasks.workunit.client.0.smithi071.stdout:9/629: mkdir d1/d8d/ddf 0 2024-04-23T02:14:06.950 INFO:tasks.workunit.client.1.smithi192.stdout:3/682: mkdir d1/d3/d17/df3 0 2024-04-23T02:14:06.950 INFO:tasks.workunit.client.0.smithi071.stdout:0/644: rmdir d0/d3/de/d10/d14/d32/d44/d87/d93 39 2024-04-23T02:14:06.951 INFO:tasks.workunit.client.1.smithi192.stdout:1/528: creat dc/d16/d19/d74/fa6 x:0 0 0 2024-04-23T02:14:06.952 INFO:tasks.workunit.client.0.smithi071.stdout:0/645: write d0/d3/de/d10/d14/d32/d37/d26/fde [713210,88860] 0 2024-04-23T02:14:06.954 INFO:tasks.workunit.client.0.smithi071.stdout:0/646: chown d0/d3/de/d10/d14/d32/d44/d87 1 1 2024-04-23T02:14:06.955 INFO:tasks.workunit.client.1.smithi192.stdout:1/529: read dc/d14/d15/d35/f49 [779794,130857] 0 2024-04-23T02:14:06.955 INFO:tasks.workunit.client.1.smithi192.stdout:2/803: dwrite d8/d26/d29/da4/dbc/fc8 [0,4194304] 0 2024-04-23T02:14:06.956 INFO:tasks.workunit.client.0.smithi071.stdout:9/630: creat d1/d32/d3a/d90/dc2/fe0 x:0 0 0 2024-04-23T02:14:06.957 INFO:tasks.workunit.client.1.smithi192.stdout:3/683: creat d1/d3/d6/d7/ff4 x:0 0 0 2024-04-23T02:14:06.957 INFO:tasks.workunit.client.0.smithi071.stdout:0/647: write d0/d3/de/d10/d14/d32/d37/d26/f9a [904759,47202] 0 2024-04-23T02:14:06.961 INFO:tasks.workunit.client.0.smithi071.stdout:0/648: creat d0/d3/de/d10/d14/d32/d44/d59/fe5 x:0 0 0 2024-04-23T02:14:06.962 INFO:tasks.workunit.client.0.smithi071.stdout:0/649: readlink d0/d3/de/d10/l1b 0 2024-04-23T02:14:06.962 INFO:tasks.workunit.client.1.smithi192.stdout:1/530: rename dc/d14/d15/d35/f37 to dc/fa7 0 2024-04-23T02:14:06.962 INFO:tasks.workunit.client.0.smithi071.stdout:9/631: rename d1/d4e/l52 to d1/d32/d3a/le1 0 2024-04-23T02:14:06.963 INFO:tasks.workunit.client.0.smithi071.stdout:0/650: dread - d0/d3/de/d10/d14/d32/d37/fdb zero size 2024-04-23T02:14:06.964 INFO:tasks.workunit.client.1.smithi192.stdout:2/804: getdents d8/d26/d29/d43/ded 0 2024-04-23T02:14:06.964 INFO:tasks.workunit.client.0.smithi071.stdout:9/632: symlink d1/d4e/le2 0 2024-04-23T02:14:06.965 INFO:tasks.workunit.client.1.smithi192.stdout:1/531: creat dc/d80/fa8 x:0 0 0 2024-04-23T02:14:06.966 INFO:tasks.workunit.client.1.smithi192.stdout:3/684: rmdir d1/d3/d6/d7/d29/d3b/da9/d7b 39 2024-04-23T02:14:06.967 INFO:tasks.workunit.client.0.smithi071.stdout:9/633: mkdir d1/d32/d41/db7/d5a/de3 0 2024-04-23T02:14:06.968 INFO:tasks.workunit.client.0.smithi071.stdout:0/651: rename d0/d92/fce to d0/d3/de/d10/d16/fe6 0 2024-04-23T02:14:06.970 INFO:tasks.workunit.client.0.smithi071.stdout:0/652: fdatasync d0/d3/de/d48/d7e/d8b/fb6 0 2024-04-23T02:14:06.971 INFO:tasks.workunit.client.1.smithi192.stdout:3/685: mknod d1/d3/d6/d7/d29/d3b/da9/cf5 0 2024-04-23T02:14:06.971 INFO:tasks.workunit.client.0.smithi071.stdout:0/653: chown d0/d3/de/d10/d14/d32/d8a 1 1 2024-04-23T02:14:06.972 INFO:tasks.workunit.client.1.smithi192.stdout:2/805: dread d8/d25/f9d [0,4194304] 0 2024-04-23T02:14:06.972 INFO:tasks.workunit.client.0.smithi071.stdout:0/654: chown d0/d63/c9b 1 1 2024-04-23T02:14:06.974 INFO:tasks.workunit.client.1.smithi192.stdout:3/686: creat d1/ff6 x:0 0 0 2024-04-23T02:14:06.975 INFO:tasks.workunit.client.0.smithi071.stdout:0/655: creat d0/d3/de/d10/d14/d32/d37/fe7 x:0 0 0 2024-04-23T02:14:06.978 INFO:tasks.workunit.client.1.smithi192.stdout:1/532: rmdir dc/d16/d19 39 2024-04-23T02:14:06.981 INFO:tasks.workunit.client.0.smithi071.stdout:8/670: dread f6 [0,4194304] 0 2024-04-23T02:14:06.981 INFO:tasks.workunit.client.1.smithi192.stdout:1/533: mkdir dc/d8b/d44/da9 0 2024-04-23T02:14:06.982 INFO:tasks.workunit.client.1.smithi192.stdout:1/534: write dc/d8b/d34/f56 [248541,8] 0 2024-04-23T02:14:06.982 INFO:tasks.workunit.client.0.smithi071.stdout:8/671: dread - dc/d15/da0/d8d/d9a/dad/fb2 zero size 2024-04-23T02:14:06.987 INFO:tasks.workunit.client.1.smithi192.stdout:2/806: truncate d8/d26/d29/d43/fd6 1509580 0 2024-04-23T02:14:06.989 INFO:tasks.workunit.client.0.smithi071.stdout:9/634: dread d1/f4 [0,4194304] 0 2024-04-23T02:14:06.989 INFO:tasks.workunit.client.0.smithi071.stdout:8/672: dread dc/d15/da0/d8d/fc3 [0,4194304] 0 2024-04-23T02:14:06.990 INFO:tasks.workunit.client.1.smithi192.stdout:1/535: creat dc/d8b/d34/faa x:0 0 0 2024-04-23T02:14:06.993 INFO:tasks.workunit.client.1.smithi192.stdout:1/536: mkdir dc/d14/d23/d5e/dab 0 2024-04-23T02:14:06.993 INFO:tasks.workunit.client.1.smithi192.stdout:3/687: mkdir d1/d3/d6/d7/df1/df7 0 2024-04-23T02:14:06.994 INFO:tasks.workunit.client.0.smithi071.stdout:8/673: link dc/d15/d50/c7b dc/d3a/cd9 0 2024-04-23T02:14:06.995 INFO:tasks.workunit.client.0.smithi071.stdout:8/674: dread - dc/f96 zero size 2024-04-23T02:14:06.995 INFO:tasks.workunit.client.0.smithi071.stdout:0/656: dread d0/d3/de/d10/d14/d32/d37/d26/f4b [0,4194304] 0 2024-04-23T02:14:06.996 INFO:tasks.workunit.client.0.smithi071.stdout:8/675: chown dc/d15/da0/d55/fc7 578291 1 2024-04-23T02:14:06.998 INFO:tasks.workunit.client.0.smithi071.stdout:8/676: chown dc/d15/d72/ca5 492 1 2024-04-23T02:14:06.998 INFO:tasks.workunit.client.0.smithi071.stdout:0/657: truncate d0/d3/de/d10/d14/f85 738071 0 2024-04-23T02:14:06.999 INFO:tasks.workunit.client.0.smithi071.stdout:8/677: readlink dc/d56/l9b 0 2024-04-23T02:14:06.999 INFO:tasks.workunit.client.1.smithi192.stdout:1/537: getdents dc/d14/d23 0 2024-04-23T02:14:07.002 INFO:tasks.workunit.client.1.smithi192.stdout:1/538: creat dc/d80/fac x:0 0 0 2024-04-23T02:14:07.002 INFO:tasks.workunit.client.1.smithi192.stdout:3/688: link d1/d3/d17/d1f/d33/ce1 d1/d3/d6/d7/d29/d3b/da9/dbd/cf8 0 2024-04-23T02:14:07.004 INFO:tasks.workunit.client.1.smithi192.stdout:2/807: dwrite d8/d14/faa [0,4194304] 0 2024-04-23T02:14:07.004 INFO:tasks.workunit.client.0.smithi071.stdout:8/678: creat dc/d15/da0/d32/d63/dd4/fda x:0 0 0 2024-04-23T02:14:07.007 INFO:tasks.workunit.client.0.smithi071.stdout:8/679: creat dc/d3a/da4/fdb x:0 0 0 2024-04-23T02:14:07.010 INFO:tasks.workunit.client.0.smithi071.stdout:8/680: rmdir dc/d15/da0/d32/d63 39 2024-04-23T02:14:07.011 INFO:tasks.workunit.client.1.smithi192.stdout:3/689: write d1/d3/d17/d1a/d7c/dab/d6c/d84/d5d/fa7 [5621,29263] 0 2024-04-23T02:14:07.011 INFO:tasks.workunit.client.1.smithi192.stdout:2/808: creat d8/d26/d29/dcd/f10c x:0 0 0 2024-04-23T02:14:07.011 INFO:tasks.workunit.client.0.smithi071.stdout:0/658: dwrite d0/d3/de/d10/d14/d32/d44/d31/fd4 [0,4194304] 0 2024-04-23T02:14:07.014 INFO:tasks.workunit.client.1.smithi192.stdout:1/539: truncate dc/d14/d23/f7d 6102325 0 2024-04-23T02:14:07.014 INFO:tasks.workunit.client.1.smithi192.stdout:7/744: sync 2024-04-23T02:14:07.014 INFO:tasks.workunit.client.1.smithi192.stdout:2/809: creat d8/d26/d29/da4/da8/dba/f10d x:0 0 0 2024-04-23T02:14:07.014 INFO:tasks.workunit.client.1.smithi192.stdout:3/690: readlink d1/d3/d17/d1a/d7c/ld9 0 2024-04-23T02:14:07.016 INFO:tasks.workunit.client.0.smithi071.stdout:8/681: getdents dc/d56/d64 0 2024-04-23T02:14:07.018 INFO:tasks.workunit.client.0.smithi071.stdout:8/682: creat dc/d56/fdc x:0 0 0 2024-04-23T02:14:07.021 INFO:tasks.workunit.client.0.smithi071.stdout:8/683: unlink dc/d3a/l40 0 2024-04-23T02:14:07.021 INFO:tasks.workunit.client.1.smithi192.stdout:7/745: unlink d5/d17/d1c/d57/d8b/f96 0 2024-04-23T02:14:07.024 INFO:tasks.workunit.client.1.smithi192.stdout:1/540: creat dc/d8b/fad x:0 0 0 2024-04-23T02:14:07.024 INFO:tasks.workunit.client.1.smithi192.stdout:7/746: rmdir d5/d17/d1c/d59/d5a 39 2024-04-23T02:14:07.025 INFO:tasks.workunit.client.0.smithi071.stdout:8/684: rename dc/d15/d5d/dc6/cd5 to dc/d78/cdd 0 2024-04-23T02:14:07.025 INFO:tasks.workunit.client.1.smithi192.stdout:2/810: creat d8/f10e x:0 0 0 2024-04-23T02:14:07.025 INFO:tasks.workunit.client.0.smithi071.stdout:8/685: stat dc/d15/d5d/l83 0 2024-04-23T02:14:07.027 INFO:tasks.workunit.client.1.smithi192.stdout:3/691: dwrite d1/d3/d17/d1f/d33/d24/feb [0,4194304] 0 2024-04-23T02:14:07.028 INFO:tasks.workunit.client.0.smithi071.stdout:8/686: mkdir dc/d56/d64/dde 0 2024-04-23T02:14:07.028 INFO:tasks.workunit.client.0.smithi071.stdout:8/687: stat dc/d15/da0/d55/l9d 0 2024-04-23T02:14:07.030 INFO:tasks.workunit.client.0.smithi071.stdout:8/688: creat dc/fdf x:0 0 0 2024-04-23T02:14:07.032 INFO:tasks.workunit.client.1.smithi192.stdout:3/692: link d1/d3/d6/d7/cf d1/d3/d6/cf9 0 2024-04-23T02:14:07.033 INFO:tasks.workunit.client.1.smithi192.stdout:2/811: dread d8/d26/d27/d53/f5b [0,4194304] 0 2024-04-23T02:14:07.033 INFO:tasks.workunit.client.1.smithi192.stdout:3/693: stat d1/l16 0 2024-04-23T02:14:07.036 INFO:tasks.workunit.client.1.smithi192.stdout:3/694: write d1/d3/d17/d1f/d33/d24/f50 [4750980,84770] 0 2024-04-23T02:14:07.037 INFO:tasks.workunit.client.1.smithi192.stdout:2/812: creat d8/d26/d29/da4/dab/f10f x:0 0 0 2024-04-23T02:14:07.049 INFO:tasks.workunit.client.0.smithi071.stdout:0/659: dwrite d0/d3/de/d10/d14/d32/d44/f43 [0,4194304] 0 2024-04-23T02:14:07.050 INFO:tasks.workunit.client.0.smithi071.stdout:8/689: dread - dc/d3a/da4/db0/fd2 zero size 2024-04-23T02:14:07.053 INFO:tasks.workunit.client.0.smithi071.stdout:0/660: truncate d0/d3/de/d10/d14/d84/fd0 22013 0 2024-04-23T02:14:07.054 INFO:tasks.workunit.client.0.smithi071.stdout:8/690: stat dc/d15/d72/d95/ca3 0 2024-04-23T02:14:07.058 INFO:tasks.workunit.client.0.smithi071.stdout:0/661: rename d0/d3/de/d10/d14/d32/d44/d31/l46 to d0/d3/de/d10/d14/d32/d44/d31/le8 0 2024-04-23T02:14:07.059 INFO:tasks.workunit.client.0.smithi071.stdout:8/691: creat dc/d56/fe0 x:0 0 0 2024-04-23T02:14:07.062 INFO:tasks.workunit.client.0.smithi071.stdout:8/692: fsync dc/d15/da0/d55/f6c 0 2024-04-23T02:14:07.063 INFO:tasks.workunit.client.0.smithi071.stdout:8/693: chown c7 127087 1 2024-04-23T02:14:07.065 INFO:tasks.workunit.client.0.smithi071.stdout:8/694: read - dc/d15/d72/d95/fbc zero size 2024-04-23T02:14:07.066 INFO:tasks.workunit.client.0.smithi071.stdout:0/662: dwrite d0/d3/f3f [0,4194304] 0 2024-04-23T02:14:07.067 INFO:tasks.workunit.client.0.smithi071.stdout:8/695: dread - dc/d15/da0/d30/f52 zero size 2024-04-23T02:14:07.069 INFO:tasks.workunit.client.0.smithi071.stdout:8/696: write dc/d15/da0/d55/f65 [13289878,41620] 0 2024-04-23T02:14:07.073 INFO:tasks.workunit.client.0.smithi071.stdout:0/663: creat d0/d3/de/d10/d14/d32/d44/d31/fe9 x:0 0 0 2024-04-23T02:14:07.078 INFO:tasks.workunit.client.0.smithi071.stdout:0/664: fsync d0/d3/de/d10/db9/fc4 0 2024-04-23T02:14:07.080 INFO:tasks.workunit.client.0.smithi071.stdout:8/697: dwrite dc/d15/da0/d8d/d9a/dad/fb2 [0,4194304] 0 2024-04-23T02:14:07.082 INFO:tasks.workunit.client.1.smithi192.stdout:3/695: fdatasync d1/f55 0 2024-04-23T02:14:07.083 INFO:tasks.workunit.client.1.smithi192.stdout:2/813: mknod d8/d26/d29/d43/c110 0 2024-04-23T02:14:07.086 INFO:tasks.workunit.client.1.smithi192.stdout:3/696: creat d1/d3/d17/d2b/d44/ffa x:0 0 0 2024-04-23T02:14:07.088 INFO:tasks.workunit.client.1.smithi192.stdout:3/697: write d1/d3/d17/d1a/d7c/dab/d6c/d84/d4c/fae [251686,27276] 0 2024-04-23T02:14:07.093 INFO:tasks.workunit.client.1.smithi192.stdout:2/814: dwrite d8/d26/d29/d43/d5d/dbd/ff6 [0,4194304] 0 2024-04-23T02:14:07.099 INFO:tasks.workunit.client.1.smithi192.stdout:3/698: mkdir d1/d3/d6/d7/d29/d3b/da9/daf/ddf/dfb 0 2024-04-23T02:14:07.101 INFO:tasks.workunit.client.1.smithi192.stdout:3/699: stat d1/d3/d17/d1a/d7c/dab/d6c/d84/f83 0 2024-04-23T02:14:07.102 INFO:tasks.workunit.client.1.smithi192.stdout:2/815: dread d8/d26/d29/da4/dbc/fc8 [0,4194304] 0 2024-04-23T02:14:07.102 INFO:tasks.workunit.client.1.smithi192.stdout:3/700: chown d1/d3/d6/d7/d5f/c68 66544209 1 2024-04-23T02:14:07.105 INFO:tasks.workunit.client.0.smithi071.stdout:0/665: dread d0/d3/de/d10/f1f [0,4194304] 0 2024-04-23T02:14:07.105 INFO:tasks.workunit.client.1.smithi192.stdout:2/816: mknod d8/d26/d29/da4/c111 0 2024-04-23T02:14:07.106 INFO:tasks.workunit.client.1.smithi192.stdout:3/701: mkdir d1/d3/d6/d7/d29/d3b/da9/d8b/dfc 0 2024-04-23T02:14:07.107 INFO:tasks.workunit.client.1.smithi192.stdout:2/817: write d8/d26/d29/d43/ffe [628128,71166] 0 2024-04-23T02:14:07.109 INFO:tasks.workunit.client.0.smithi071.stdout:0/666: creat d0/d3/de/d10/db3/fea x:0 0 0 2024-04-23T02:14:07.111 INFO:tasks.workunit.client.0.smithi071.stdout:0/667: chown d0/d3/de/d10/d14/d32/d44/l5a 3320 1 2024-04-23T02:14:07.113 INFO:tasks.workunit.client.0.smithi071.stdout:0/668: mknod d0/d3/de/d48/d7e/d8b/de3/ceb 0 2024-04-23T02:14:07.114 INFO:tasks.workunit.client.0.smithi071.stdout:0/669: chown d0/d3/d95/fb4 1625 1 2024-04-23T02:14:07.115 INFO:tasks.workunit.client.0.smithi071.stdout:0/670: read - d0/d3/de/d10/d14/d32/d37/d26/fc2 zero size 2024-04-23T02:14:07.116 INFO:tasks.workunit.client.1.smithi192.stdout:2/818: dwrite d8/d26/d29/da4/da8/dba/f10d [0,4194304] 0 2024-04-23T02:14:07.117 INFO:tasks.workunit.client.0.smithi071.stdout:0/671: dread d0/d3/de/d10/f1f [4194304,4194304] 0 2024-04-23T02:14:07.119 INFO:tasks.workunit.client.1.smithi192.stdout:2/819: mknod d8/d25/c112 0 2024-04-23T02:14:07.119 INFO:tasks.workunit.client.0.smithi071.stdout:0/672: write d0/d3/de/d10/d14/d84/fd0 [998037,23535] 0 2024-04-23T02:14:07.120 INFO:tasks.workunit.client.1.smithi192.stdout:3/702: dwrite d1/f5 [0,4194304] 0 2024-04-23T02:14:07.121 INFO:tasks.workunit.client.1.smithi192.stdout:2/820: chown d8/d26/d29/d43/ded/lf1 0 1 2024-04-23T02:14:07.123 INFO:tasks.workunit.client.1.smithi192.stdout:3/703: write d1/d3/d17/d1f/d33/f8d [1879619,48382] 0 2024-04-23T02:14:07.126 INFO:tasks.workunit.client.1.smithi192.stdout:3/704: rmdir d1/d3/d6/d7/d71 39 2024-04-23T02:14:07.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:06 smithi192 ceph-mon[49662]: pgmap v30: 65 pgs: 65 active+clean; 2.4 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 83 MiB/s rd, 261 MiB/s wr, 489 op/s 2024-04-23T02:14:07.129 INFO:tasks.workunit.client.0.smithi071.stdout:0/673: creat d0/d3/de/d10/d14/d32/d37/fec x:0 0 0 2024-04-23T02:14:07.131 INFO:tasks.workunit.client.0.smithi071.stdout:0/674: write d0/d3/de/d10/d14/d32/d37/fec [744278,130324] 0 2024-04-23T02:14:07.132 INFO:tasks.workunit.client.0.smithi071.stdout:0/675: fdatasync d0/d3/de/d10/d14/d32/d44/d87/f96 0 2024-04-23T02:14:07.134 INFO:tasks.workunit.client.1.smithi192.stdout:2/821: truncate d8/d26/d29/d43/d5d/dbd/ff6 6971459 0 2024-04-23T02:14:07.135 INFO:tasks.workunit.client.1.smithi192.stdout:3/705: dwrite d1/d3/d17/d1a/d7c/dab/f7f [0,4194304] 0 2024-04-23T02:14:07.135 INFO:tasks.workunit.client.1.smithi192.stdout:2/822: fdatasync d8/d14/faa 0 2024-04-23T02:14:07.139 INFO:tasks.workunit.client.1.smithi192.stdout:2/823: read d8/d25/f71 [1443428,24652] 0 2024-04-23T02:14:07.141 INFO:tasks.workunit.client.1.smithi192.stdout:2/824: write d8/f96 [3585957,12342] 0 2024-04-23T02:14:07.144 INFO:tasks.workunit.client.1.smithi192.stdout:3/706: rmdir d1/d3/d6/d7/d29/d3b/da9/daf/ddf/dfb 0 2024-04-23T02:14:07.146 INFO:tasks.workunit.client.1.smithi192.stdout:2/825: mkdir d8/d26/d29/d43/d113 0 2024-04-23T02:14:07.148 INFO:tasks.workunit.client.1.smithi192.stdout:3/707: mknod d1/d3/d17/d2b/d44/d57/db0/cfd 0 2024-04-23T02:14:07.148 INFO:tasks.workunit.client.1.smithi192.stdout:2/826: mkdir d8/d26/d27/d57/d69/d99/d114 0 2024-04-23T02:14:07.149 INFO:tasks.workunit.client.1.smithi192.stdout:3/708: mknod d1/d3/d17/d1f/dc2/cfe 0 2024-04-23T02:14:07.151 INFO:tasks.workunit.client.1.smithi192.stdout:3/709: dread - d1/d3/d17/d1a/d7c/fe0 zero size 2024-04-23T02:14:07.152 INFO:tasks.workunit.client.1.smithi192.stdout:2/827: dread d8/d26/f106 [0,4194304] 0 2024-04-23T02:14:07.154 INFO:tasks.workunit.client.0.smithi071.stdout:0/676: creat d0/d3/de/d10/d16/fed x:0 0 0 2024-04-23T02:14:07.155 INFO:tasks.workunit.client.0.smithi071.stdout:0/677: stat d0/d3/de/d10/d16/fe6 0 2024-04-23T02:14:07.156 INFO:tasks.workunit.client.0.smithi071.stdout:0/678: creat d0/d63/fee x:0 0 0 2024-04-23T02:14:07.157 INFO:tasks.workunit.client.1.smithi192.stdout:2/828: getdents d8/d26/d27/d57 0 2024-04-23T02:14:07.157 INFO:tasks.workunit.client.1.smithi192.stdout:2/829: chown d8/f86 10 1 2024-04-23T02:14:07.158 INFO:tasks.workunit.client.1.smithi192.stdout:2/830: readlink d8/d26/d29/da4/da8/lac 0 2024-04-23T02:14:07.158 INFO:tasks.workunit.client.0.smithi071.stdout:0/679: creat d0/d63/fef x:0 0 0 2024-04-23T02:14:07.158 INFO:tasks.workunit.client.0.smithi071.stdout:0/680: fdatasync d0/d3/f3f 0 2024-04-23T02:14:07.159 INFO:tasks.workunit.client.1.smithi192.stdout:2/831: creat d8/d26/d29/d43/d5d/dbd/dda/f115 x:0 0 0 2024-04-23T02:14:07.161 INFO:tasks.workunit.client.1.smithi192.stdout:2/832: truncate d8/d26/d27/d53/da2/fc0 1181832 0 2024-04-23T02:14:07.166 INFO:tasks.workunit.client.1.smithi192.stdout:2/833: fdatasync d8/d26/d29/d43/d5d/dbd/ff6 0 2024-04-23T02:14:07.168 INFO:tasks.workunit.client.1.smithi192.stdout:2/834: chown d8/d26/d29/da4/dca/ffc 13877 1 2024-04-23T02:14:07.170 INFO:tasks.workunit.client.1.smithi192.stdout:2/835: creat d8/d26/d27/d31/f116 x:0 0 0 2024-04-23T02:14:07.171 INFO:tasks.workunit.client.1.smithi192.stdout:2/836: unlink d8/d26/f6b 0 2024-04-23T02:14:07.172 INFO:tasks.workunit.client.1.smithi192.stdout:2/837: dread - d8/d26/d27/d31/f116 zero size 2024-04-23T02:14:07.173 INFO:tasks.workunit.client.1.smithi192.stdout:2/838: truncate d8/d26/d29/d43/d5d/dbd/ffd 506090 0 2024-04-23T02:14:07.174 INFO:tasks.workunit.client.1.smithi192.stdout:2/839: stat d8/d26/d27/d53/da2/dc1/fbb 0 2024-04-23T02:14:07.174 INFO:tasks.workunit.client.0.smithi071.stdout:0/681: rename d0/d3/de/d10/d16 to d0/d3/de/d10/d14/d32/d8a/df0 0 2024-04-23T02:14:07.175 INFO:tasks.workunit.client.1.smithi192.stdout:2/840: symlink d8/d26/d27/db5/dd0/l117 0 2024-04-23T02:14:07.176 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:06 smithi071 ceph-mon[39837]: pgmap v30: 65 pgs: 65 active+clean; 2.4 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 83 MiB/s rd, 261 MiB/s wr, 489 op/s 2024-04-23T02:14:07.176 INFO:tasks.workunit.client.0.smithi071.stdout:0/682: creat d0/d3/de/d10/d14/d84/ff1 x:0 0 0 2024-04-23T02:14:07.177 INFO:tasks.workunit.client.1.smithi192.stdout:2/841: unlink d8/d26/d29/d43/fdf 0 2024-04-23T02:14:07.178 INFO:tasks.workunit.client.1.smithi192.stdout:2/842: fdatasync d8/d26/d27/d53/da2/fb9 0 2024-04-23T02:14:07.178 INFO:tasks.workunit.client.0.smithi071.stdout:0/683: fsync d0/d3/de/d10/d14/d32/d37/d26/f2e 0 2024-04-23T02:14:07.179 INFO:tasks.workunit.client.1.smithi192.stdout:2/843: unlink d8/d26/la1 0 2024-04-23T02:14:07.181 INFO:tasks.workunit.client.0.smithi071.stdout:0/684: getdents d0/d76 0 2024-04-23T02:14:07.181 INFO:tasks.workunit.client.1.smithi192.stdout:2/844: mkdir d8/d26/d29/d43/dc2/dec/d118 0 2024-04-23T02:14:07.183 INFO:tasks.workunit.client.0.smithi071.stdout:0/685: mknod d0/ddf/cf2 0 2024-04-23T02:14:07.184 INFO:tasks.workunit.client.0.smithi071.stdout:0/686: stat d0/d3/de/d10/d14/l7d 0 2024-04-23T02:14:07.186 INFO:tasks.workunit.client.1.smithi192.stdout:0/841: sync 2024-04-23T02:14:07.188 INFO:tasks.workunit.client.0.smithi071.stdout:0/687: truncate d0/d3/de/d10/db9/fc4 3234589 0 2024-04-23T02:14:07.189 INFO:tasks.workunit.client.1.smithi192.stdout:0/842: rename d6/f68 to d6/df/d13/d25/d86/f11c 0 2024-04-23T02:14:07.194 INFO:tasks.workunit.client.1.smithi192.stdout:2/845: dread d8/f36 [4194304,4194304] 0 2024-04-23T02:14:07.201 INFO:tasks.workunit.client.1.smithi192.stdout:2/846: dwrite d8/d26/d29/da4/f7e [0,4194304] 0 2024-04-23T02:14:07.203 INFO:tasks.workunit.client.1.smithi192.stdout:0/843: dread d6/df/d13/d25/d3a/f4f [0,4194304] 0 2024-04-23T02:14:07.204 INFO:tasks.workunit.client.1.smithi192.stdout:2/847: mkdir d8/d26/d29/d43/d5d/d119 0 2024-04-23T02:14:07.207 INFO:tasks.workunit.client.1.smithi192.stdout:0/844: fdatasync d6/df/d13/d25/d3a/f6f 0 2024-04-23T02:14:07.209 INFO:tasks.workunit.client.1.smithi192.stdout:2/848: mkdir d8/d26/d27/d31/dae/d11a 0 2024-04-23T02:14:07.211 INFO:tasks.workunit.client.1.smithi192.stdout:0/845: read - d6/df/d35/d49/fd2 zero size 2024-04-23T02:14:07.212 INFO:tasks.workunit.client.1.smithi192.stdout:2/849: readlink d8/d14/l17 0 2024-04-23T02:14:07.212 INFO:tasks.workunit.client.1.smithi192.stdout:0/846: write d6/f10 [341080,120379] 0 2024-04-23T02:14:07.214 INFO:tasks.workunit.client.1.smithi192.stdout:0/847: truncate d6/df/d13/d25/d26/d67/ffe 388111 0 2024-04-23T02:14:07.215 INFO:tasks.workunit.client.1.smithi192.stdout:2/850: dread d8/d25/f3f [0,4194304] 0 2024-04-23T02:14:07.218 INFO:tasks.workunit.client.1.smithi192.stdout:2/851: creat d8/d26/d27/f11b x:0 0 0 2024-04-23T02:14:07.219 INFO:tasks.workunit.client.1.smithi192.stdout:0/848: rmdir d6/d4b/d9d/dc7/d100 0 2024-04-23T02:14:07.221 INFO:tasks.workunit.client.1.smithi192.stdout:2/852: rename d8/d26/d29/da4/dab/cb1 to d8/d26/d29/d80/c11c 0 2024-04-23T02:14:07.221 INFO:tasks.workunit.client.1.smithi192.stdout:2/853: chown d8/d26/d29/da4/f7e 880800 1 2024-04-23T02:14:07.222 INFO:tasks.workunit.client.1.smithi192.stdout:0/849: creat d6/d4b/d9d/f11d x:0 0 0 2024-04-23T02:14:07.223 INFO:tasks.workunit.client.1.smithi192.stdout:0/850: write d6/df/d13/d25/d26/d67/f69 [3148418,56379] 0 2024-04-23T02:14:07.225 INFO:tasks.workunit.client.1.smithi192.stdout:0/851: chown d6/df/d13/d25/d94/le1 500407137 1 2024-04-23T02:14:07.228 INFO:tasks.workunit.client.1.smithi192.stdout:2/854: dwrite d8/d26/d29/da4/dbc/fc8 [0,4194304] 0 2024-04-23T02:14:07.230 INFO:tasks.workunit.client.1.smithi192.stdout:0/852: rename d6/df/d13/l57 to d6/df/d47/l11e 0 2024-04-23T02:14:07.232 INFO:tasks.workunit.client.1.smithi192.stdout:2/855: symlink d8/d26/d29/d43/dc2/dec/d118/l11d 0 2024-04-23T02:14:07.240 INFO:tasks.workunit.client.1.smithi192.stdout:2/856: dwrite d8/d26/d29/da4/dca/fd4 [0,4194304] 0 2024-04-23T02:14:07.243 INFO:tasks.workunit.client.1.smithi192.stdout:2/857: rename d8/d26/d29/da4/c84 to d8/d26/d81/c11e 0 2024-04-23T02:14:07.243 INFO:tasks.workunit.client.1.smithi192.stdout:2/858: read - d8/d26/d74/fd1 zero size 2024-04-23T02:14:07.249 INFO:tasks.workunit.client.0.smithi071.stdout:6/606: sync 2024-04-23T02:14:07.249 INFO:tasks.workunit.client.1.smithi192.stdout:2/859: dwrite d8/d26/d27/f11b [0,4194304] 0 2024-04-23T02:14:07.251 INFO:tasks.workunit.client.1.smithi192.stdout:0/853: dread d6/df/f83 [0,4194304] 0 2024-04-23T02:14:07.253 INFO:tasks.workunit.client.1.smithi192.stdout:2/860: creat d8/d26/d27/d53/da2/dc1/f11f x:0 0 0 2024-04-23T02:14:07.254 INFO:tasks.workunit.client.1.smithi192.stdout:2/861: stat d8/d26/d29/d43/d5d/fe0 0 2024-04-23T02:14:07.255 INFO:tasks.workunit.client.1.smithi192.stdout:2/862: truncate d8/d26/d29/da4/dab/f105 447077 0 2024-04-23T02:14:07.256 INFO:tasks.workunit.client.1.smithi192.stdout:2/863: write d8/d26/d29/d43/f9f [299981,4775] 0 2024-04-23T02:14:07.257 INFO:tasks.workunit.client.1.smithi192.stdout:2/864: chown d8/d26/d74/d7c 37547 1 2024-04-23T02:14:07.259 INFO:tasks.workunit.client.0.smithi071.stdout:6/607: dwrite d6/f94 [0,4194304] 0 2024-04-23T02:14:07.263 INFO:tasks.workunit.client.0.smithi071.stdout:6/608: fdatasync d6/d1f/d4e/f54 0 2024-04-23T02:14:07.266 INFO:tasks.workunit.client.0.smithi071.stdout:6/609: getdents d6/d17/d77/d9e 0 2024-04-23T02:14:07.267 INFO:tasks.workunit.client.1.smithi192.stdout:2/865: dwrite d8/d14/f1a [4194304,4194304] 0 2024-04-23T02:14:07.270 INFO:tasks.workunit.client.0.smithi071.stdout:6/610: link d6/d1f/l57 d6/d22/lc1 0 2024-04-23T02:14:07.271 INFO:tasks.workunit.client.1.smithi192.stdout:2/866: write d8/d26/d27/f87 [1520148,62774] 0 2024-04-23T02:14:07.271 INFO:tasks.workunit.client.0.smithi071.stdout:6/611: chown d6/d1f/f68 1413 1 2024-04-23T02:14:07.273 INFO:tasks.workunit.client.0.smithi071.stdout:6/612: symlink d6/d1f/d5d/lc2 0 2024-04-23T02:14:07.274 INFO:tasks.workunit.client.0.smithi071.stdout:6/613: dread - d6/d1f/d46/f6e zero size 2024-04-23T02:14:07.282 INFO:tasks.workunit.client.1.smithi192.stdout:2/867: dread d8/f18 [0,4194304] 0 2024-04-23T02:14:07.282 INFO:tasks.workunit.client.1.smithi192.stdout:2/868: creat d8/d26/d29/d43/dc2/f120 x:0 0 0 2024-04-23T02:14:07.284 INFO:tasks.workunit.client.1.smithi192.stdout:2/869: dread d8/d26/d74/fc5 [0,4194304] 0 2024-04-23T02:14:07.289 INFO:tasks.workunit.client.1.smithi192.stdout:2/870: fsync d8/d26/d27/d53/da2/fc0 0 2024-04-23T02:14:07.290 INFO:tasks.workunit.client.1.smithi192.stdout:2/871: chown d8/f10e 245221866 1 2024-04-23T02:14:07.292 INFO:tasks.workunit.client.1.smithi192.stdout:2/872: read d8/d25/f71 [217914,53566] 0 2024-04-23T02:14:07.293 INFO:tasks.workunit.client.1.smithi192.stdout:2/873: mknod d8/d26/d29/d80/c121 0 2024-04-23T02:14:07.296 INFO:tasks.workunit.client.1.smithi192.stdout:0/854: dread d6/df/d13/d25/d3a/f58 [4194304,4194304] 0 2024-04-23T02:14:07.297 INFO:tasks.workunit.client.1.smithi192.stdout:2/874: dread d8/f36 [4194304,4194304] 0 2024-04-23T02:14:07.297 INFO:tasks.workunit.client.1.smithi192.stdout:0/855: write d6/df/d13/d6b/d73/da9/fe5 [1422829,71872] 0 2024-04-23T02:14:07.301 INFO:tasks.workunit.client.1.smithi192.stdout:0/856: rmdir d6/df/d13/d25/d26/d67 39 2024-04-23T02:14:07.301 INFO:tasks.workunit.client.0.smithi071.stdout:6/614: dread d6/d91/f9c [0,4194304] 0 2024-04-23T02:14:07.302 INFO:tasks.workunit.client.1.smithi192.stdout:0/857: write d6/d4b/d9d/dc7/f80 [1176643,7343] 0 2024-04-23T02:14:07.303 INFO:tasks.workunit.client.1.smithi192.stdout:0/858: fdatasync d6/df/d13/d25/d26/f90 0 2024-04-23T02:14:07.303 INFO:tasks.workunit.client.1.smithi192.stdout:2/875: truncate d8/d14/faa 2553134 0 2024-04-23T02:14:07.305 INFO:tasks.workunit.client.1.smithi192.stdout:2/876: read d8/d26/d27/d53/da2/dc1/fbb [3590848,73767] 0 2024-04-23T02:14:07.305 INFO:tasks.workunit.client.1.smithi192.stdout:0/859: chown d6/df/d13/d25/lcd 0 1 2024-04-23T02:14:07.306 INFO:tasks.workunit.client.0.smithi071.stdout:6/615: rename d6/d17/d77/d9f/dad/f5b to d6/d1f/d4e/fc3 0 2024-04-23T02:14:07.307 INFO:tasks.workunit.client.0.smithi071.stdout:6/616: dread - d6/d17/d77/f8c zero size 2024-04-23T02:14:07.309 INFO:tasks.workunit.client.0.smithi071.stdout:6/617: getdents d6/d1f/d46 0 2024-04-23T02:14:07.310 INFO:tasks.workunit.client.1.smithi192.stdout:2/877: dread d8/d26/d29/da4/f7e [0,4194304] 0 2024-04-23T02:14:07.311 INFO:tasks.workunit.client.1.smithi192.stdout:0/860: getdents d6/df/d13/d25/d94/dd0/de6 0 2024-04-23T02:14:07.311 INFO:tasks.workunit.client.0.smithi071.stdout:6/618: symlink d6/d17/d77/d9f/dad/lc4 0 2024-04-23T02:14:07.313 INFO:tasks.workunit.client.0.smithi071.stdout:6/619: unlink d6/d17/d77/d9f/dad/f82 0 2024-04-23T02:14:07.314 INFO:tasks.workunit.client.0.smithi071.stdout:6/620: chown d6/d1f/c9d 95 1 2024-04-23T02:14:07.315 INFO:tasks.workunit.client.1.smithi192.stdout:0/861: rename d6/df/d47/d55/f70 to d6/df/d56/d85/da4/f11f 0 2024-04-23T02:14:07.315 INFO:tasks.workunit.client.1.smithi192.stdout:2/878: write d8/d26/d29/d43/d50/d5e/f89 [4409961,15856] 0 2024-04-23T02:14:07.316 INFO:tasks.workunit.client.1.smithi192.stdout:0/862: truncate d6/df/d13/d25/f91 5391098 0 2024-04-23T02:14:07.317 INFO:tasks.workunit.client.1.smithi192.stdout:2/879: dread - d8/d26/d29/da4/da8/fc9 zero size 2024-04-23T02:14:07.317 INFO:tasks.workunit.client.0.smithi071.stdout:6/621: creat d6/d17/d77/d9e/fc5 x:0 0 0 2024-04-23T02:14:07.317 INFO:tasks.workunit.client.1.smithi192.stdout:0/863: stat d6/df/d35/l60 0 2024-04-23T02:14:07.319 INFO:tasks.workunit.client.0.smithi071.stdout:6/622: creat d6/d17/d77/da2/fc6 x:0 0 0 2024-04-23T02:14:07.321 INFO:tasks.workunit.client.0.smithi071.stdout:6/623: read d6/d1f/f9a [2057844,126546] 0 2024-04-23T02:14:07.321 INFO:tasks.workunit.client.0.smithi071.stdout:6/624: chown d6/d1f/d46/f6e 239 1 2024-04-23T02:14:07.322 INFO:tasks.workunit.client.0.smithi071.stdout:6/625: readlink d6/d1f/d46/l7e 0 2024-04-23T02:14:07.324 INFO:tasks.workunit.client.1.smithi192.stdout:2/880: dwrite d8/d26/d29/d43/d5d/fd5 [0,4194304] 0 2024-04-23T02:14:07.324 INFO:tasks.workunit.client.0.smithi071.stdout:6/626: mknod d6/d22/dbf/cc7 0 2024-04-23T02:14:07.325 INFO:tasks.workunit.client.1.smithi192.stdout:0/864: dwrite d6/df/d35/d8b/d8d/ff5 [0,4194304] 0 2024-04-23T02:14:07.327 INFO:tasks.workunit.client.0.smithi071.stdout:6/627: creat d6/d91/fc8 x:0 0 0 2024-04-23T02:14:07.327 INFO:tasks.workunit.client.0.smithi071.stdout:6/628: stat d6/d22/dbf 0 2024-04-23T02:14:07.327 INFO:tasks.workunit.client.1.smithi192.stdout:0/865: write d6/df/d13/d25/d26/feb [1004306,91647] 0 2024-04-23T02:14:07.332 INFO:tasks.workunit.client.1.smithi192.stdout:2/881: write d8/d26/d27/d53/f5b [125192,99860] 0 2024-04-23T02:14:07.332 INFO:tasks.workunit.client.1.smithi192.stdout:0/866: symlink d6/df/d56/d85/d45/l120 0 2024-04-23T02:14:07.332 INFO:tasks.workunit.client.1.smithi192.stdout:5/729: sync 2024-04-23T02:14:07.333 INFO:tasks.workunit.client.1.smithi192.stdout:6/854: sync 2024-04-23T02:14:07.333 INFO:tasks.workunit.client.1.smithi192.stdout:0/867: chown d6/df/d13/d25/d3a/d93/ff0 703 1 2024-04-23T02:14:07.335 INFO:tasks.workunit.client.1.smithi192.stdout:0/868: write f5 [1701134,115870] 0 2024-04-23T02:14:07.335 INFO:tasks.workunit.client.1.smithi192.stdout:2/882: creat d8/d26/d74/d7c/f122 x:0 0 0 2024-04-23T02:14:07.336 INFO:tasks.workunit.client.1.smithi192.stdout:5/730: mknod d2/cf3 0 2024-04-23T02:14:07.338 INFO:tasks.workunit.client.1.smithi192.stdout:2/883: write d8/f10e [235185,45648] 0 2024-04-23T02:14:07.338 INFO:tasks.workunit.client.1.smithi192.stdout:0/869: chown d6/df/d56/c65 550779293 1 2024-04-23T02:14:07.339 INFO:tasks.workunit.client.1.smithi192.stdout:0/870: chown d6/f6d 866272 1 2024-04-23T02:14:07.343 INFO:tasks.workunit.client.1.smithi192.stdout:2/884: creat d8/d26/d27/d53/da2/f123 x:0 0 0 2024-04-23T02:14:07.345 INFO:tasks.workunit.client.1.smithi192.stdout:0/871: symlink d6/df/d13/d25/d3a/da6/l121 0 2024-04-23T02:14:07.346 INFO:tasks.workunit.client.1.smithi192.stdout:2/885: creat d8/d26/d74/f124 x:0 0 0 2024-04-23T02:14:07.347 INFO:tasks.workunit.client.1.smithi192.stdout:5/731: mknod d2/d2e/d4e/de6/cf4 0 2024-04-23T02:14:07.347 INFO:tasks.workunit.client.1.smithi192.stdout:5/732: fdatasync d2/f5 0 2024-04-23T02:14:07.350 INFO:tasks.workunit.client.1.smithi192.stdout:2/886: unlink d8/d25/f3a 0 2024-04-23T02:14:07.352 INFO:tasks.workunit.client.0.smithi071.stdout:6/629: dread d6/d17/d77/d9f/dad/f8a [0,4194304] 0 2024-04-23T02:14:07.354 INFO:tasks.workunit.client.0.smithi071.stdout:6/630: mknod d6/d17/d77/d85/cc9 0 2024-04-23T02:14:07.356 INFO:tasks.workunit.client.1.smithi192.stdout:6/855: truncate d3/d11/f4a 1112 0 2024-04-23T02:14:07.356 INFO:tasks.workunit.client.1.smithi192.stdout:5/733: dwrite d2/db/d2b/d52/dc2/fd3 [0,4194304] 0 2024-04-23T02:14:07.360 INFO:tasks.workunit.client.0.smithi071.stdout:6/631: dwrite d6/f8 [0,4194304] 0 2024-04-23T02:14:07.361 INFO:tasks.workunit.client.1.smithi192.stdout:6/856: creat d3/d28/d79/dbe/dcb/de9/f11a x:0 0 0 2024-04-23T02:14:07.361 INFO:tasks.workunit.client.0.smithi071.stdout:6/632: chown d6/c13 789036588 1 2024-04-23T02:14:07.363 INFO:tasks.workunit.client.0.smithi071.stdout:6/633: creat d6/d17/d77/d9f/dad/d3f/fca x:0 0 0 2024-04-23T02:14:07.365 INFO:tasks.workunit.client.0.smithi071.stdout:6/634: chown d6/d1f/c27 168781974 1 2024-04-23T02:14:07.366 INFO:tasks.workunit.client.1.smithi192.stdout:6/857: readlink d3/d18/dd4/ld2 0 2024-04-23T02:14:07.366 INFO:tasks.workunit.client.0.smithi071.stdout:6/635: rmdir d6/d1f/d46 39 2024-04-23T02:14:07.367 INFO:tasks.workunit.client.0.smithi071.stdout:6/636: chown d6/d1f/f68 1808818940 1 2024-04-23T02:14:07.368 INFO:tasks.workunit.client.1.smithi192.stdout:2/887: truncate d8/d26/d27/f11b 3162822 0 2024-04-23T02:14:07.369 INFO:tasks.workunit.client.1.smithi192.stdout:9/623: sync 2024-04-23T02:14:07.369 INFO:tasks.workunit.client.1.smithi192.stdout:4/844: sync 2024-04-23T02:14:07.369 INFO:tasks.workunit.client.1.smithi192.stdout:3/710: sync 2024-04-23T02:14:07.369 INFO:tasks.workunit.client.1.smithi192.stdout:5/734: rmdir d2/d6/d2d/d41/d43/d98/ddf 0 2024-04-23T02:14:07.369 INFO:tasks.workunit.client.1.smithi192.stdout:6/858: write d3/d6c/fd0 [919591,61061] 0 2024-04-23T02:14:07.370 INFO:tasks.workunit.client.0.smithi071.stdout:6/637: rmdir d6/d1f/d5d/d95 39 2024-04-23T02:14:07.371 INFO:tasks.workunit.client.1.smithi192.stdout:1/541: sync 2024-04-23T02:14:07.371 INFO:tasks.workunit.client.1.smithi192.stdout:7/747: sync 2024-04-23T02:14:07.371 INFO:tasks.workunit.client.1.smithi192.stdout:0/872: dwrite d6/df/d13/f51 [0,4194304] 0 2024-04-23T02:14:07.374 INFO:tasks.workunit.client.0.smithi071.stdout:6/638: symlink d6/d17/d77/d9f/db5/lcb 0 2024-04-23T02:14:07.375 INFO:tasks.workunit.client.1.smithi192.stdout:4/845: chown de/d15/d53/d81/de3/d74/d92/da0/lc1 132784 1 2024-04-23T02:14:07.375 INFO:tasks.workunit.client.0.smithi071.stdout:6/639: creat d6/d91/fcc x:0 0 0 2024-04-23T02:14:07.378 INFO:tasks.workunit.client.1.smithi192.stdout:0/873: write d6/df/d13/d25/d3a/f62 [1764748,124673] 0 2024-04-23T02:14:07.380 INFO:tasks.workunit.client.1.smithi192.stdout:2/888: mkdir d8/d26/d125 0 2024-04-23T02:14:07.380 INFO:tasks.workunit.client.1.smithi192.stdout:4/846: chown de/d15/d53/d81/de3/db2/lc0 47 1 2024-04-23T02:14:07.382 INFO:tasks.workunit.client.0.smithi071.stdout:6/640: dwrite d6/f16 [0,4194304] 0 2024-04-23T02:14:07.387 INFO:tasks.workunit.client.1.smithi192.stdout:3/711: dwrite d1/d3/d6/d7/f82 [0,4194304] 0 2024-04-23T02:14:07.389 INFO:tasks.workunit.client.1.smithi192.stdout:5/735: mknod d2/db/d16/d1b/cf5 0 2024-04-23T02:14:07.390 INFO:tasks.workunit.client.1.smithi192.stdout:5/736: dread - d2/d2e/d4e/fc3 zero size 2024-04-23T02:14:07.396 INFO:tasks.workunit.client.1.smithi192.stdout:6/859: unlink d3/d18/f94 0 2024-04-23T02:14:07.397 INFO:tasks.workunit.client.0.smithi071.stdout:6/641: dread d6/d1f/f83 [0,4194304] 0 2024-04-23T02:14:07.398 INFO:tasks.workunit.client.1.smithi192.stdout:2/889: dwrite d8/d26/d27/d57/da0/fe5 [0,4194304] 0 2024-04-23T02:14:07.399 INFO:tasks.workunit.client.0.smithi071.stdout:6/642: truncate d6/f12 393193 0 2024-04-23T02:14:07.400 INFO:tasks.workunit.client.1.smithi192.stdout:9/624: link d0/d9/fb d0/d9/d44/d51/d5b/d61/db1/fcf 0 2024-04-23T02:14:07.405 INFO:tasks.workunit.client.0.smithi071.stdout:6/643: dwrite d6/d1f/d4e/f54 [0,4194304] 0 2024-04-23T02:14:07.405 INFO:tasks.workunit.client.1.smithi192.stdout:5/737: rmdir d2/db/d2b 39 2024-04-23T02:14:07.407 INFO:tasks.workunit.client.0.smithi071.stdout:6/644: fsync d6/d1f/d4e/f74 0 2024-04-23T02:14:07.408 INFO:tasks.workunit.client.1.smithi192.stdout:4/847: symlink de/d15/d53/d81/de3/d74/d92/da0/dca/d34/l11a 0 2024-04-23T02:14:07.409 INFO:tasks.workunit.client.0.smithi071.stdout:6/645: truncate d6/d22/f41 4957779 0 2024-04-23T02:14:07.414 INFO:tasks.workunit.client.1.smithi192.stdout:2/890: mknod d8/d26/d29/da4/dbc/c126 0 2024-04-23T02:14:07.414 INFO:tasks.workunit.client.1.smithi192.stdout:0/874: creat d6/df/d13/d6b/dc1/dd7/f122 x:0 0 0 2024-04-23T02:14:07.419 INFO:tasks.workunit.client.1.smithi192.stdout:5/738: symlink d2/db/d16/d48/d4f/lf6 0 2024-04-23T02:14:07.424 INFO:tasks.workunit.client.0.smithi071.stdout:6/646: dread d6/d1f/f25 [0,4194304] 0 2024-04-23T02:14:07.426 INFO:tasks.workunit.client.0.smithi071.stdout:6/647: creat d6/d17/d77/d9e/fcd x:0 0 0 2024-04-23T02:14:07.429 INFO:tasks.workunit.client.0.smithi071.stdout:3/590: sync 2024-04-23T02:14:07.429 INFO:tasks.workunit.client.0.smithi071.stdout:1/586: sync 2024-04-23T02:14:07.429 INFO:tasks.workunit.client.0.smithi071.stdout:6/648: rename d6/d17/d77/d9f/dad/d3f/l49 to d6/d1f/lce 0 2024-04-23T02:14:07.432 INFO:tasks.workunit.client.0.smithi071.stdout:1/587: write db/df/d13/d2f/d48/d85/fb0 [938683,120632] 0 2024-04-23T02:14:07.432 INFO:tasks.workunit.client.0.smithi071.stdout:6/649: chown d6/d1f/d46 172 1 2024-04-23T02:14:07.433 INFO:tasks.workunit.client.1.smithi192.stdout:7/748: dwrite d5/d43/d4c/f6c [0,4194304] 0 2024-04-23T02:14:07.434 INFO:tasks.workunit.client.0.smithi071.stdout:3/591: rename d7/d10/d38/d67 to d7/d8/d42/d80/dca 0 2024-04-23T02:14:07.435 INFO:tasks.workunit.client.1.smithi192.stdout:1/542: truncate dc/d14/d15/d35/f40 1403322 0 2024-04-23T02:14:07.435 INFO:tasks.workunit.client.0.smithi071.stdout:6/650: dread - d6/d17/d77/d9f/dad/da4/fbd zero size 2024-04-23T02:14:07.436 INFO:tasks.workunit.client.0.smithi071.stdout:1/588: unlink db/df/d13/d2f/d48/f49 0 2024-04-23T02:14:07.439 INFO:tasks.workunit.client.0.smithi071.stdout:6/651: fsync d6/d1f/d5d/d95/f98 0 2024-04-23T02:14:07.440 INFO:tasks.workunit.client.0.smithi071.stdout:1/589: read db/df/d5f/d57/f5e [3941953,685] 0 2024-04-23T02:14:07.440 INFO:tasks.workunit.client.1.smithi192.stdout:0/875: dread d6/df/d56/ff3 [0,4194304] 0 2024-04-23T02:14:07.441 INFO:tasks.workunit.client.1.smithi192.stdout:9/625: dwrite d0/d9/dc7/fc6 [0,4194304] 0 2024-04-23T02:14:07.441 INFO:tasks.workunit.client.0.smithi071.stdout:3/592: getdents d7/d22/da7 0 2024-04-23T02:14:07.445 INFO:tasks.workunit.client.0.smithi071.stdout:1/590: creat db/df/d5f/fbe x:0 0 0 2024-04-23T02:14:07.446 INFO:tasks.workunit.client.0.smithi071.stdout:3/593: symlink d7/da5/db2/db7/d68/d3b/d4c/d8b/db3/lcb 0 2024-04-23T02:14:07.446 INFO:tasks.workunit.client.0.smithi071.stdout:6/652: dread d6/d17/d77/d9f/dad/f75 [0,4194304] 0 2024-04-23T02:14:07.447 INFO:tasks.workunit.client.1.smithi192.stdout:3/712: dwrite d1/d3/d17/fa8 [0,4194304] 0 2024-04-23T02:14:07.448 INFO:tasks.workunit.client.0.smithi071.stdout:6/653: write d6/fb6 [851667,99958] 0 2024-04-23T02:14:07.449 INFO:tasks.workunit.client.0.smithi071.stdout:6/654: readlink d6/lbe 0 2024-04-23T02:14:07.452 INFO:tasks.workunit.client.0.smithi071.stdout:6/655: fsync d6/d1f/f9a 0 2024-04-23T02:14:07.452 INFO:tasks.workunit.client.0.smithi071.stdout:3/594: creat d7/da5/db2/db7/d68/d3b/d9f/fcc x:0 0 0 2024-04-23T02:14:07.454 INFO:tasks.workunit.client.0.smithi071.stdout:6/656: mkdir d6/d22/dcf 0 2024-04-23T02:14:07.454 INFO:tasks.workunit.client.0.smithi071.stdout:3/595: mknod d7/da5/db2/db7/d68/d3b/ccd 0 2024-04-23T02:14:07.462 INFO:tasks.workunit.client.1.smithi192.stdout:4/848: fdatasync de/d15/d53/d81/de3/d74/d92/da0/dca/d34/df9/da1/fbe 0 2024-04-23T02:14:07.464 INFO:tasks.workunit.client.1.smithi192.stdout:0/876: dwrite d6/df/d13/d6b/d73/f7e [0,4194304] 0 2024-04-23T02:14:07.466 INFO:tasks.workunit.client.1.smithi192.stdout:3/713: stat d1/d3/d6/cc 0 2024-04-23T02:14:07.468 INFO:tasks.workunit.client.1.smithi192.stdout:6/860: dread d3/d15/d1b/f72 [0,4194304] 0 2024-04-23T02:14:07.469 INFO:tasks.workunit.client.1.smithi192.stdout:5/739: mkdir d2/db/d2b/df7 0 2024-04-23T02:14:07.469 INFO:tasks.workunit.client.1.smithi192.stdout:2/891: getdents d8/d26/d29/d43/d5d/dbd/dda 0 2024-04-23T02:14:07.472 INFO:tasks.workunit.client.1.smithi192.stdout:4/849: dread - de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/fb1 zero size 2024-04-23T02:14:07.473 INFO:tasks.workunit.client.1.smithi192.stdout:0/877: truncate d6/df/f14 2164320 0 2024-04-23T02:14:07.474 INFO:tasks.workunit.client.1.smithi192.stdout:5/740: chown d2/db/d2b/l37 3 1 2024-04-23T02:14:07.478 INFO:tasks.workunit.client.1.smithi192.stdout:1/543: dwrite dc/fa7 [4194304,4194304] 0 2024-04-23T02:14:07.480 INFO:tasks.workunit.client.1.smithi192.stdout:6/861: read d3/d28/df8/f60 [2912061,115302] 0 2024-04-23T02:14:07.481 INFO:tasks.workunit.client.1.smithi192.stdout:4/850: dread - de/d15/d19/d1a/d95/dd5/f10e zero size 2024-04-23T02:14:07.481 INFO:tasks.workunit.client.1.smithi192.stdout:0/878: chown d6/df/d13/d25/d26/fac 757 1 2024-04-23T02:14:07.487 INFO:tasks.workunit.client.0.smithi071.stdout:3/596: dread d7/da5/db5/fb6 [0,4194304] 0 2024-04-23T02:14:07.489 INFO:tasks.workunit.client.1.smithi192.stdout:0/879: mknod d6/df/d56/c123 0 2024-04-23T02:14:07.490 INFO:tasks.workunit.client.1.smithi192.stdout:2/892: dwrite d8/d26/d74/d7c/f122 [0,4194304] 0 2024-04-23T02:14:07.499 INFO:tasks.workunit.client.1.smithi192.stdout:2/893: truncate d8/d26/d29/da4/dab/f105 690258 0 2024-04-23T02:14:07.501 INFO:tasks.workunit.client.1.smithi192.stdout:9/626: write d0/d9/d44/d4f/f79 [199621,56566] 0 2024-04-23T02:14:07.505 INFO:tasks.workunit.client.1.smithi192.stdout:1/544: dwrite f3 [0,4194304] 0 2024-04-23T02:14:07.505 INFO:tasks.workunit.client.1.smithi192.stdout:4/851: rename de/d15/d53/d81/de3/d80 to de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d11b 0 2024-04-23T02:14:07.506 INFO:tasks.workunit.client.1.smithi192.stdout:6/862: dwrite d3/d28/d79/da0/dc2/d101/d80/f8b [0,4194304] 0 2024-04-23T02:14:07.509 INFO:tasks.workunit.client.0.smithi071.stdout:3/597: dread d7/d10/f15 [0,4194304] 0 2024-04-23T02:14:07.510 INFO:tasks.workunit.client.1.smithi192.stdout:5/741: write d2/d6/f4d [2452503,124995] 0 2024-04-23T02:14:07.512 INFO:tasks.workunit.client.1.smithi192.stdout:6/863: write d3/f57 [4207446,109441] 0 2024-04-23T02:14:07.512 INFO:tasks.workunit.client.0.smithi071.stdout:3/598: mkdir d7/d8/d42/d80/dce 0 2024-04-23T02:14:07.515 INFO:tasks.workunit.client.1.smithi192.stdout:9/627: mknod d0/d9/d44/d4f/cd0 0 2024-04-23T02:14:07.515 INFO:tasks.workunit.client.1.smithi192.stdout:3/714: dwrite d1/d3/d17/d1f/d33/d24/f45 [0,4194304] 0 2024-04-23T02:14:07.518 INFO:tasks.workunit.client.1.smithi192.stdout:1/545: mkdir dc/d16/d19/dae 0 2024-04-23T02:14:07.518 INFO:tasks.workunit.client.1.smithi192.stdout:5/742: write d2/db8/fea [884250,31162] 0 2024-04-23T02:14:07.521 INFO:tasks.workunit.client.1.smithi192.stdout:7/749: dread d5/d17/d1c/d59/d5a/d41/d50/d9f/fa4 [0,4194304] 0 2024-04-23T02:14:07.522 INFO:tasks.workunit.client.0.smithi071.stdout:3/599: dwrite d7/da5/db2/db7/f40 [0,4194304] 0 2024-04-23T02:14:07.526 INFO:tasks.workunit.client.1.smithi192.stdout:1/546: creat dc/d16/d19/d74/faf x:0 0 0 2024-04-23T02:14:07.527 INFO:tasks.workunit.client.1.smithi192.stdout:7/750: fdatasync d5/d17/d1c/d59/d5a/fa2 0 2024-04-23T02:14:07.529 INFO:tasks.workunit.client.0.smithi071.stdout:3/600: rename d7/d10/l85 to d7/d8/d42/d80/lcf 0 2024-04-23T02:14:07.531 INFO:tasks.workunit.client.1.smithi192.stdout:5/743: symlink d2/db/d16/d48/d4f/dd1/lf8 0 2024-04-23T02:14:07.532 INFO:tasks.workunit.client.1.smithi192.stdout:3/715: write d1/d3/d17/d1a/d7c/dab/d6c/d84/d46/fbc [4530917,107124] 0 2024-04-23T02:14:07.533 INFO:tasks.workunit.client.1.smithi192.stdout:1/547: mkdir dc/d16/db0 0 2024-04-23T02:14:07.536 INFO:tasks.workunit.client.0.smithi071.stdout:3/601: dwrite d7/d22/da7/f9b [0,4194304] 0 2024-04-23T02:14:07.536 INFO:tasks.workunit.client.1.smithi192.stdout:0/880: dwrite d6/df/d13/d25/d26/fcf [0,4194304] 0 2024-04-23T02:14:07.537 INFO:tasks.workunit.client.1.smithi192.stdout:7/751: mknod d5/d17/d1c/d59/d5a/cde 0 2024-04-23T02:14:07.539 INFO:tasks.workunit.client.1.smithi192.stdout:5/744: chown d2/d2e/fd8 41940215 1 2024-04-23T02:14:07.543 INFO:tasks.workunit.client.1.smithi192.stdout:1/548: mknod dc/d14/d23/d5e/cb1 0 2024-04-23T02:14:07.545 INFO:tasks.workunit.client.1.smithi192.stdout:6/864: dwrite d3/d18/dd4/f65 [0,4194304] 0 2024-04-23T02:14:07.546 INFO:tasks.workunit.client.1.smithi192.stdout:7/752: dread - d5/d17/d1c/d4a/f84 zero size 2024-04-23T02:14:07.548 INFO:tasks.workunit.client.1.smithi192.stdout:2/894: dread d8/d26/d29/f85 [0,4194304] 0 2024-04-23T02:14:07.553 INFO:tasks.workunit.client.1.smithi192.stdout:1/549: chown dc/d16/d19/l7b 1 1 2024-04-23T02:14:07.555 INFO:tasks.workunit.client.1.smithi192.stdout:7/753: read d5/d17/d1c/d59/d5a/d41/f98 [93027,116865] 0 2024-04-23T02:14:07.555 INFO:tasks.workunit.client.1.smithi192.stdout:5/745: truncate d2/db/d16/d48/f9e 4586356 0 2024-04-23T02:14:07.556 INFO:tasks.workunit.client.1.smithi192.stdout:0/881: truncate d6/df/d56/d85/da4/f11f 5597901 0 2024-04-23T02:14:07.557 INFO:tasks.workunit.client.1.smithi192.stdout:3/716: dwrite d1/d3/d6/d7/d29/fe2 [0,4194304] 0 2024-04-23T02:14:07.559 INFO:tasks.workunit.client.1.smithi192.stdout:9/628: write d0/d9/d44/d51/fae [679459,91065] 0 2024-04-23T02:14:07.559 INFO:tasks.workunit.client.1.smithi192.stdout:1/550: fsync dc/d8b/fa0 0 2024-04-23T02:14:07.562 INFO:tasks.workunit.client.1.smithi192.stdout:6/865: link d3/d6c/l83 d3/d15/d1b/dcf/l11b 0 2024-04-23T02:14:07.563 INFO:tasks.workunit.client.1.smithi192.stdout:9/629: chown d0/d9/c6f 17285 1 2024-04-23T02:14:07.565 INFO:tasks.workunit.client.1.smithi192.stdout:4/852: dread de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/f91 [0,4194304] 0 2024-04-23T02:14:07.565 INFO:tasks.workunit.client.1.smithi192.stdout:0/882: creat d6/df/d13/d25/d3a/d7b/f124 x:0 0 0 2024-04-23T02:14:07.566 INFO:tasks.workunit.client.1.smithi192.stdout:1/551: symlink dc/lb2 0 2024-04-23T02:14:07.570 INFO:tasks.workunit.client.1.smithi192.stdout:6/866: symlink d3/d15/d1b/da2/l11c 0 2024-04-23T02:14:07.570 INFO:tasks.workunit.client.1.smithi192.stdout:3/717: fsync d1/d3/d17/d1f/d33/fcf 0 2024-04-23T02:14:07.571 INFO:tasks.workunit.client.1.smithi192.stdout:2/895: dwrite d8/d26/d27/d31/f8b [0,4194304] 0 2024-04-23T02:14:07.580 INFO:tasks.workunit.client.1.smithi192.stdout:6/867: chown d3/d18/d58/cb5 21685789 1 2024-04-23T02:14:07.580 INFO:tasks.workunit.client.1.smithi192.stdout:0/883: mkdir d6/df/d13/d6b/d73/d125 0 2024-04-23T02:14:07.581 INFO:tasks.workunit.client.1.smithi192.stdout:5/746: dwrite d2/d6/d2d/d41/fd2 [0,4194304] 0 2024-04-23T02:14:07.583 INFO:tasks.workunit.client.1.smithi192.stdout:7/754: rmdir d5/d43/d4c/d5c/d67/da6/da9/db9/dc0 39 2024-04-23T02:14:07.585 INFO:tasks.workunit.client.1.smithi192.stdout:2/896: mknod d8/d26/d27/db5/df3/c127 0 2024-04-23T02:14:07.592 INFO:tasks.workunit.client.1.smithi192.stdout:1/552: dwrite dc/d8b/d34/d70/d8c/fa4 [0,4194304] 0 2024-04-23T02:14:07.593 INFO:tasks.workunit.client.1.smithi192.stdout:4/853: dread de/d15/d53/d81/de3/d74/d92/da0/dca/f5f [0,4194304] 0 2024-04-23T02:14:07.593 INFO:tasks.workunit.client.1.smithi192.stdout:6/868: rmdir d3/d28/d79/da0/dc2/d101/d80 39 2024-04-23T02:14:07.594 INFO:tasks.workunit.client.1.smithi192.stdout:0/884: rename d6/df/d13/d25/d94/dd0/de6 to d6/df/d13/d25/d94/dd0/de6/dfa/d126 22 2024-04-23T02:14:07.596 INFO:tasks.workunit.client.1.smithi192.stdout:5/747: symlink d2/db/d16/lf9 0 2024-04-23T02:14:07.598 INFO:tasks.workunit.client.1.smithi192.stdout:3/718: getdents d1/d3/d17/d2b/d44/d57 0 2024-04-23T02:14:07.600 INFO:tasks.workunit.client.1.smithi192.stdout:6/869: unlink d3/f57 0 2024-04-23T02:14:07.601 INFO:tasks.workunit.client.1.smithi192.stdout:9/630: dwrite d0/d9/f37 [0,4194304] 0 2024-04-23T02:14:07.602 INFO:tasks.workunit.client.1.smithi192.stdout:6/870: chown d3/d15/d5b/lad 28 1 2024-04-23T02:14:07.608 INFO:tasks.workunit.client.1.smithi192.stdout:1/553: mknod dc/d16/d95/cb3 0 2024-04-23T02:14:07.609 INFO:tasks.workunit.client.1.smithi192.stdout:4/854: symlink de/d15/d53/d81/de3/de4/l11c 0 2024-04-23T02:14:07.610 INFO:tasks.workunit.client.1.smithi192.stdout:7/755: write d5/d17/d1c/d59/d5a/d41/d50/d9f/fa4 [1606154,88603] 0 2024-04-23T02:14:07.612 INFO:tasks.workunit.client.1.smithi192.stdout:5/748: dread - d2/d6/d2d/fac zero size 2024-04-23T02:14:07.613 INFO:tasks.workunit.client.1.smithi192.stdout:0/885: write d6/df/d47/f89 [3236835,94723] 0 2024-04-23T02:14:07.613 INFO:tasks.workunit.client.1.smithi192.stdout:3/719: chown d1/d3/d17/d1f/l4d 13 1 2024-04-23T02:14:07.614 INFO:tasks.workunit.client.1.smithi192.stdout:1/554: creat dc/d80/fb4 x:0 0 0 2024-04-23T02:14:07.617 INFO:tasks.workunit.client.1.smithi192.stdout:3/720: chown d1/d3/d17/d65/d99/fa5 4368430 1 2024-04-23T02:14:07.618 INFO:tasks.workunit.client.1.smithi192.stdout:2/897: dwrite d8/d26/d74/fc5 [0,4194304] 0 2024-04-23T02:14:07.626 INFO:tasks.workunit.client.1.smithi192.stdout:0/886: dwrite d6/df/d13/d25/d3a/da6/fe9 [0,4194304] 0 2024-04-23T02:14:07.690 INFO:tasks.workunit.client.1.smithi192.stdout:4/855: read de/d15/d53/d81/de3/da3/ff3 [49995,84175] 0 2024-04-23T02:14:07.691 INFO:tasks.workunit.client.0.smithi071.stdout:7/736: sync 2024-04-23T02:14:07.694 INFO:tasks.workunit.client.0.smithi071.stdout:7/737: fsync d3/d17/d1c/f26 0 2024-04-23T02:14:07.696 INFO:tasks.workunit.client.0.smithi071.stdout:7/738: dread d3/d17/d41/d34/d39/d3b/f84 [0,4194304] 0 2024-04-23T02:14:07.698 INFO:tasks.workunit.client.0.smithi071.stdout:7/739: creat d3/dae/f101 x:0 0 0 2024-04-23T02:14:07.702 INFO:tasks.workunit.client.0.smithi071.stdout:7/740: creat d3/d17/d1c/d2a/d43/f102 x:0 0 0 2024-04-23T02:14:07.705 INFO:tasks.workunit.client.0.smithi071.stdout:7/741: creat d3/d17/f103 x:0 0 0 2024-04-23T02:14:07.706 INFO:tasks.workunit.client.0.smithi071.stdout:7/742: fdatasync d3/d17/d41/d37/f6f 0 2024-04-23T02:14:07.707 INFO:tasks.workunit.client.0.smithi071.stdout:7/743: chown d3/d17/d41/d34/d39/fe8 994786 1 2024-04-23T02:14:07.714 INFO:tasks.workunit.client.0.smithi071.stdout:7/744: dwrite d3/f1d [0,4194304] 0 2024-04-23T02:14:07.720 INFO:tasks.workunit.client.1.smithi192.stdout:7/756: fdatasync d5/d17/d1c/d59/d5a/d41/d50/d3d/f83 0 2024-04-23T02:14:07.721 INFO:tasks.workunit.client.0.smithi071.stdout:7/745: creat d3/d17/d1c/d2a/f104 x:0 0 0 2024-04-23T02:14:07.724 INFO:tasks.workunit.client.1.smithi192.stdout:5/749: write d2/d6/d2d/d41/d43/d98/fca [1188739,28537] 0 2024-04-23T02:14:07.725 INFO:tasks.workunit.client.0.smithi071.stdout:7/746: creat d3/d17/d41/da8/f105 x:0 0 0 2024-04-23T02:14:07.727 INFO:tasks.workunit.client.0.smithi071.stdout:7/747: readlink d3/d17/d41/d3a/da5/lf1 0 2024-04-23T02:14:07.728 INFO:tasks.workunit.client.1.smithi192.stdout:3/721: mknod d1/d3/d6/d7/d29/d3b/da9/daf/ddf/cff 0 2024-04-23T02:14:07.731 INFO:tasks.workunit.client.1.smithi192.stdout:2/898: rename d8/d26/d29/d43/d5d/dbd/dda/f115 to d8/d26/d29/d43/d5d/d119/f128 0 2024-04-23T02:14:07.734 INFO:tasks.workunit.client.0.smithi071.stdout:7/748: truncate d3/d17/d41/f33 1447518 0 2024-04-23T02:14:07.738 INFO:tasks.workunit.client.1.smithi192.stdout:7/757: chown d5/d17/d1c/f3e 11 1 2024-04-23T02:14:07.738 INFO:tasks.workunit.client.0.smithi071.stdout:7/749: rename d3/d17/d1c/d2a/fd2 to d3/d51/d80/de9/f106 0 2024-04-23T02:14:07.739 INFO:tasks.workunit.client.0.smithi071.stdout:7/750: truncate d3/d51/d87/db3/fbc 1157538 0 2024-04-23T02:14:07.741 INFO:tasks.workunit.client.0.smithi071.stdout:7/751: write d3/d51/df5/ff8 [713140,120647] 0 2024-04-23T02:14:07.744 INFO:tasks.workunit.client.0.smithi071.stdout:7/752: creat d3/d17/d41/d34/d39/f107 x:0 0 0 2024-04-23T02:14:07.745 INFO:tasks.workunit.client.1.smithi192.stdout:4/856: creat de/d15/d53/d81/de3/d74/d92/da0/dca/d34/db9/f11d x:0 0 0 2024-04-23T02:14:07.746 INFO:tasks.workunit.client.1.smithi192.stdout:4/857: chown de/d15/d53/d81/de3/d74/d92/da0/dca/f8b 348899 1 2024-04-23T02:14:07.747 INFO:tasks.workunit.client.0.smithi071.stdout:7/753: mknod d3/d17/d1c/d2a/d73/d83/de7/c108 0 2024-04-23T02:14:07.747 INFO:tasks.workunit.client.1.smithi192.stdout:1/555: link dc/l1c dc/d14/d23/d5e/lb5 0 2024-04-23T02:14:07.753 INFO:tasks.workunit.client.1.smithi192.stdout:4/858: chown de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/d59/fd3 63995 1 2024-04-23T02:14:07.754 INFO:tasks.workunit.client.0.smithi071.stdout:7/754: dwrite d3/d51/f7f [0,4194304] 0 2024-04-23T02:14:07.756 INFO:tasks.workunit.client.1.smithi192.stdout:2/899: creat d8/d26/d29/da4/dca/f129 x:0 0 0 2024-04-23T02:14:07.759 INFO:tasks.workunit.client.0.smithi071.stdout:7/755: mknod d3/d17/d1c/d2a/d43/d86/c109 0 2024-04-23T02:14:07.759 INFO:tasks.workunit.client.1.smithi192.stdout:1/556: link dc/d8b/fad dc/d16/d19/d74/fb6 0 2024-04-23T02:14:07.764 INFO:tasks.workunit.client.0.smithi071.stdout:7/756: rename d3/d17/d41/d34/d39/fc8 to d3/dae/db7/dea/dc5/f10a 0 2024-04-23T02:14:07.769 INFO:tasks.workunit.client.1.smithi192.stdout:2/900: dwrite d8/d26/d27/d57/da0/fc6 [0,4194304] 0 2024-04-23T02:14:07.771 INFO:tasks.workunit.client.1.smithi192.stdout:1/557: dwrite dc/d16/d19/d74/fa6 [0,4194304] 0 2024-04-23T02:14:07.772 INFO:tasks.workunit.client.1.smithi192.stdout:2/901: chown d8/d26/d29/d43/d5d/l68 2991073 1 2024-04-23T02:14:07.775 INFO:tasks.workunit.client.1.smithi192.stdout:2/902: write d8/d26/d29/d43/d5d/fd5 [9375043,25430] 0 2024-04-23T02:14:07.775 INFO:tasks.workunit.client.1.smithi192.stdout:1/558: mknod dc/d16/d95/cb7 0 2024-04-23T02:14:07.776 INFO:tasks.workunit.client.1.smithi192.stdout:2/903: chown d8/d26/d27/c9a 30660722 1 2024-04-23T02:14:07.778 INFO:tasks.workunit.client.1.smithi192.stdout:2/904: read - d8/d26/d27/d53/da2/dc1/f11f zero size 2024-04-23T02:14:07.781 INFO:tasks.workunit.client.0.smithi071.stdout:7/757: dwrite d3/d17/d41/d3a/da5/ff3 [0,4194304] 0 2024-04-23T02:14:07.784 INFO:tasks.workunit.client.1.smithi192.stdout:1/559: dwrite dc/d16/d19/d30/f84 [0,4194304] 0 2024-04-23T02:14:07.785 INFO:tasks.workunit.client.0.smithi071.stdout:7/758: fsync d3/d17/d41/d37/f50 0 2024-04-23T02:14:07.786 INFO:tasks.workunit.client.1.smithi192.stdout:1/560: read dc/d16/d19/d30/f84 [2011857,43624] 0 2024-04-23T02:14:07.789 INFO:tasks.workunit.client.0.smithi071.stdout:7/759: fsync d3/d17/d41/d34/d39/f97 0 2024-04-23T02:14:07.793 INFO:tasks.workunit.client.0.smithi071.stdout:7/760: rename d3/d51/cb8 to d3/d17/d1c/c10b 0 2024-04-23T02:14:07.795 INFO:tasks.workunit.client.1.smithi192.stdout:2/905: truncate d8/d26/d29/d43/d50/d5e/f5f 108789 0 2024-04-23T02:14:07.802 INFO:tasks.workunit.client.1.smithi192.stdout:3/722: creat d1/d3/d17/d2b/d44/f100 x:0 0 0 2024-04-23T02:14:07.814 INFO:tasks.workunit.client.1.smithi192.stdout:3/723: dwrite d1/d3/d17/d2b/d44/d57/db0/fdb [0,4194304] 0 2024-04-23T02:14:07.817 INFO:tasks.workunit.client.1.smithi192.stdout:5/750: truncate d2/db/d16/d1b/f90 3088878 0 2024-04-23T02:14:07.818 INFO:tasks.workunit.client.1.smithi192.stdout:1/561: dread dc/d14/d23/f50 [8388608,4194304] 0 2024-04-23T02:14:07.820 INFO:tasks.workunit.client.1.smithi192.stdout:0/887: dwrite d6/df/d13/d25/d3a/f87 [0,4194304] 0 2024-04-23T02:14:07.823 INFO:tasks.workunit.client.1.smithi192.stdout:3/724: rename d1/d3/d17/d1f/fce to d1/d3/d6/d7/d29/d3b/da9/daf/f101 0 2024-04-23T02:14:07.828 INFO:tasks.workunit.client.1.smithi192.stdout:3/725: chown d1/d3/d17 1171735562 1 2024-04-23T02:14:07.829 INFO:tasks.workunit.client.0.smithi071.stdout:7/761: dwrite d3/dae/db7/dea/f72 [4194304,4194304] 0 2024-04-23T02:14:07.831 INFO:tasks.workunit.client.1.smithi192.stdout:0/888: mkdir d6/df/d56/db1/d127 0 2024-04-23T02:14:07.836 INFO:tasks.workunit.client.1.smithi192.stdout:7/758: dwrite d5/d17/d1c/d59/f93 [0,4194304] 0 2024-04-23T02:14:07.836 INFO:tasks.workunit.client.1.smithi192.stdout:0/889: rmdir d6/df/d56/d85/da4 39 2024-04-23T02:14:07.839 INFO:tasks.workunit.client.1.smithi192.stdout:7/759: write d5/d43/d4c/d5c/d67/da6/fb6 [4760407,51327] 0 2024-04-23T02:14:07.840 INFO:tasks.workunit.client.0.smithi071.stdout:7/762: dwrite d3/dae/db7/dea/dc5/f10a [0,4194304] 0 2024-04-23T02:14:07.845 INFO:tasks.workunit.client.1.smithi192.stdout:0/890: mkdir d6/df/d35/d8b/d8d/dea/d128 0 2024-04-23T02:14:07.846 INFO:tasks.workunit.client.1.smithi192.stdout:5/751: dread d2/db/d16/d48/d4f/f5c [0,4194304] 0 2024-04-23T02:14:07.847 INFO:tasks.workunit.client.1.smithi192.stdout:9/631: creat d0/fd1 x:0 0 0 2024-04-23T02:14:07.847 INFO:tasks.workunit.client.1.smithi192.stdout:7/760: dread d5/d43/d4c/d5c/d67/da6/fb6 [0,4194304] 0 2024-04-23T02:14:07.848 INFO:tasks.workunit.client.1.smithi192.stdout:5/752: chown d2/d6/f12 46419 1 2024-04-23T02:14:07.852 INFO:tasks.workunit.client.1.smithi192.stdout:4/859: symlink de/d15/d53/d81/l11e 0 2024-04-23T02:14:07.855 INFO:tasks.workunit.client.1.smithi192.stdout:9/632: chown d0/d9/d44/d53/c7a 237 1 2024-04-23T02:14:07.858 INFO:tasks.workunit.client.1.smithi192.stdout:5/753: mknod d2/db/d2b/cfa 0 2024-04-23T02:14:07.858 INFO:tasks.workunit.client.1.smithi192.stdout:9/633: truncate d0/d9/d44/d51/d5b/d61/f6a 5235335 0 2024-04-23T02:14:07.860 INFO:tasks.workunit.client.1.smithi192.stdout:7/761: mkdir d5/d43/d4c/d5c/d67/da6/ddf 0 2024-04-23T02:14:07.861 INFO:tasks.workunit.client.1.smithi192.stdout:3/726: creat d1/d3/d6/d7/f102 x:0 0 0 2024-04-23T02:14:07.864 INFO:tasks.workunit.client.1.smithi192.stdout:2/906: write d8/d26/d27/f11b [4113267,7343] 0 2024-04-23T02:14:07.865 INFO:tasks.workunit.client.1.smithi192.stdout:0/891: dwrite d6/df/d35/d49/fe7 [0,4194304] 0 2024-04-23T02:14:07.866 INFO:tasks.workunit.client.1.smithi192.stdout:4/860: rename de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/l117 to de/d15/d53/d81/de3/d74/d92/da0/dca/l11f 0 2024-04-23T02:14:07.866 INFO:tasks.workunit.client.1.smithi192.stdout:3/727: chown d1/d3/d17/d1f/dc2/cfe 873585693 1 2024-04-23T02:14:07.874 INFO:tasks.workunit.client.1.smithi192.stdout:3/728: read d1/d3/d6/f79 [4165725,62683] 0 2024-04-23T02:14:07.874 INFO:tasks.workunit.client.1.smithi192.stdout:0/892: fdatasync d6/df/d13/d25/d3a/da6/fe9 0 2024-04-23T02:14:07.876 INFO:tasks.workunit.client.1.smithi192.stdout:9/634: getdents d0/d9/dc7/da0/db4 0 2024-04-23T02:14:07.876 INFO:tasks.workunit.client.1.smithi192.stdout:1/562: truncate dc/d14/f21 3640706 0 2024-04-23T02:14:07.879 INFO:tasks.workunit.client.1.smithi192.stdout:0/893: write d6/df/d56/d85/f2e [2585636,93877] 0 2024-04-23T02:14:07.879 INFO:tasks.workunit.client.1.smithi192.stdout:3/729: chown d1/d3/d17/d1a/d7c/dab/f92 8 1 2024-04-23T02:14:07.881 INFO:tasks.workunit.client.1.smithi192.stdout:9/635: mkdir d0/dd2 0 2024-04-23T02:14:07.882 INFO:tasks.workunit.client.1.smithi192.stdout:2/907: dread d8/d26/f106 [0,4194304] 0 2024-04-23T02:14:07.882 INFO:tasks.workunit.client.1.smithi192.stdout:3/730: read - d1/d3/d6/d7/d29/d3b/f8e zero size 2024-04-23T02:14:07.883 INFO:tasks.workunit.client.0.smithi071.stdout:7/763: dread d3/d17/d1c/d2a/d73/fc6 [0,4194304] 0 2024-04-23T02:14:07.883 INFO:tasks.workunit.client.0.smithi071.stdout:7/764: chown d3/d17/d41/d34/d39/f107 2932 1 2024-04-23T02:14:07.884 INFO:tasks.workunit.client.1.smithi192.stdout:9/636: chown d0/fd1 578851962 1 2024-04-23T02:14:07.885 INFO:tasks.workunit.client.1.smithi192.stdout:0/894: readlink d6/df/d13/d25/d3a/d93/ld3 0 2024-04-23T02:14:07.885 INFO:tasks.workunit.client.1.smithi192.stdout:2/908: write d8/d26/d81/fb8 [2007029,98966] 0 2024-04-23T02:14:07.885 INFO:tasks.workunit.client.0.smithi071.stdout:7/765: mkdir d3/d17/d1c/d31/d10c 0 2024-04-23T02:14:07.886 INFO:tasks.workunit.client.0.smithi071.stdout:7/766: chown d3/d17/d41/d34/d39/d3b/f49 56 1 2024-04-23T02:14:07.886 INFO:tasks.workunit.client.1.smithi192.stdout:1/563: rename dc/d14/d23/d5e/d6a/f9a to dc/d8b/d44/da9/fb8 0 2024-04-23T02:14:07.888 INFO:tasks.workunit.client.1.smithi192.stdout:2/909: dread - d8/d26/d27/db5/dd0/fd8 zero size 2024-04-23T02:14:07.889 INFO:tasks.workunit.client.0.smithi071.stdout:7/767: getdents d3/d17/d41/da8 0 2024-04-23T02:14:07.889 INFO:tasks.workunit.client.0.smithi071.stdout:7/768: fdatasync d3/f1d 0 2024-04-23T02:14:07.890 INFO:tasks.workunit.client.1.smithi192.stdout:1/564: read dc/fd [9505091,58743] 0 2024-04-23T02:14:07.894 INFO:tasks.workunit.client.0.smithi071.stdout:7/769: mkdir d3/d51/df5/d10d 0 2024-04-23T02:14:07.894 INFO:tasks.workunit.client.0.smithi071.stdout:7/770: unlink d3/fad 0 2024-04-23T02:14:07.896 INFO:tasks.workunit.client.1.smithi192.stdout:7/762: truncate d5/d17/d1c/d59/d5a/d41/d50/f7b 1304240 0 2024-04-23T02:14:07.899 INFO:tasks.workunit.client.1.smithi192.stdout:1/565: creat dc/d14/d23/d5e/dab/fb9 x:0 0 0 2024-04-23T02:14:07.900 INFO:tasks.workunit.client.0.smithi071.stdout:7/771: read - d3/d17/d41/d34/d39/fe8 zero size 2024-04-23T02:14:07.901 INFO:tasks.workunit.client.1.smithi192.stdout:4/861: dread de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/fa8 [0,4194304] 0 2024-04-23T02:14:07.904 INFO:tasks.workunit.client.1.smithi192.stdout:1/566: mknod dc/d14/d23/d5e/d6a/cba 0 2024-04-23T02:14:07.905 INFO:tasks.workunit.client.1.smithi192.stdout:7/763: dread d5/d43/f99 [0,4194304] 0 2024-04-23T02:14:07.906 INFO:tasks.workunit.client.0.smithi071.stdout:7/772: dwrite d3/d17/d41/d34/d39/d3b/f49 [0,4194304] 0 2024-04-23T02:14:07.908 INFO:tasks.workunit.client.1.smithi192.stdout:1/567: write dc/d16/d19/d30/f55 [5063842,86698] 0 2024-04-23T02:14:07.908 INFO:tasks.workunit.client.1.smithi192.stdout:3/731: dread d1/d3/d17/d1f/d33/f8d [0,4194304] 0 2024-04-23T02:14:07.909 INFO:tasks.workunit.client.1.smithi192.stdout:2/910: dread d8/d26/d29/da4/dca/ffc [0,4194304] 0 2024-04-23T02:14:07.909 INFO:tasks.workunit.client.0.smithi071.stdout:7/773: fsync d3/dae/db7/dea/fe6 0 2024-04-23T02:14:07.910 INFO:tasks.workunit.client.1.smithi192.stdout:4/862: rename de/d15/d53/d81/de3/db2/fec to de/d15/d53/d81/de3/d97/f120 0 2024-04-23T02:14:07.910 INFO:tasks.workunit.client.1.smithi192.stdout:9/637: write d0/d9/f65 [3001270,90071] 0 2024-04-23T02:14:07.911 INFO:tasks.workunit.client.0.smithi071.stdout:7/774: dread - d3/d17/d41/da8/f105 zero size 2024-04-23T02:14:07.911 INFO:tasks.workunit.client.0.smithi071.stdout:7/775: dread - d3/dae/f101 zero size 2024-04-23T02:14:07.912 INFO:tasks.workunit.client.1.smithi192.stdout:2/911: dread - d8/d26/d29/dcd/f10c zero size 2024-04-23T02:14:07.912 INFO:tasks.workunit.client.1.smithi192.stdout:7/764: symlink d5/d17/d1c/d59/da8/le0 0 2024-04-23T02:14:07.913 INFO:tasks.workunit.client.1.smithi192.stdout:1/568: chown dc/d16/d19/d74/d77/f91 41732 1 2024-04-23T02:14:07.915 INFO:tasks.workunit.client.1.smithi192.stdout:3/732: mkdir d1/d3/d6/d7/d29/d3b/da9/dbd/d103 0 2024-04-23T02:14:07.918 INFO:tasks.workunit.client.0.smithi071.stdout:7/776: dwrite d3/d17/d41/f2f [4194304,4194304] 0 2024-04-23T02:14:07.919 INFO:tasks.workunit.client.1.smithi192.stdout:9/638: read d0/d9/f50 [2139344,16689] 0 2024-04-23T02:14:07.920 INFO:tasks.workunit.client.1.smithi192.stdout:3/733: write d1/d3/d17/d2b/d44/d57/fb4 [394076,85271] 0 2024-04-23T02:14:07.922 INFO:tasks.workunit.client.0.smithi071.stdout:7/777: read d3/d17/d41/d34/d39/d3b/f53 [101695,26942] 0 2024-04-23T02:14:07.922 INFO:tasks.workunit.client.1.smithi192.stdout:4/863: creat de/d15/d53/d81/de3/d74/d92/da0/dca/de5/f121 x:0 0 0 2024-04-23T02:14:07.924 INFO:tasks.workunit.client.0.smithi071.stdout:7/778: unlink d3/d17/d1c/d2a/d43/cd0 0 2024-04-23T02:14:07.926 INFO:tasks.workunit.client.0.smithi071.stdout:7/779: mknod d3/d17/d1c/d2a/d73/dba/c10e 0 2024-04-23T02:14:07.927 INFO:tasks.workunit.client.1.smithi192.stdout:1/569: unlink dc/d16/d19/d30/f5b 0 2024-04-23T02:14:07.927 INFO:tasks.workunit.client.1.smithi192.stdout:3/734: chown d1/d3/d6/d7/d29/d3b/da9 21642 1 2024-04-23T02:14:07.927 INFO:tasks.workunit.client.1.smithi192.stdout:2/912: creat d8/d26/d27/d31/dae/d11a/f12a x:0 0 0 2024-04-23T02:14:07.928 INFO:tasks.workunit.client.1.smithi192.stdout:0/895: dwrite d6/df/d13/d25/d26/f10d [4194304,4194304] 0 2024-04-23T02:14:07.928 INFO:tasks.workunit.client.1.smithi192.stdout:4/864: fdatasync de/d15/d53/d81/dac/fd7 0 2024-04-23T02:14:07.930 INFO:tasks.workunit.client.1.smithi192.stdout:7/765: dread d5/d43/d4c/d5c/f72 [0,4194304] 0 2024-04-23T02:14:07.933 INFO:tasks.workunit.client.0.smithi071.stdout:7/780: write d3/d17/d1c/f3f [1213713,20652] 0 2024-04-23T02:14:07.934 INFO:tasks.workunit.client.1.smithi192.stdout:1/570: write dc/d16/d19/d74/fa6 [94123,101662] 0 2024-04-23T02:14:07.934 INFO:tasks.workunit.client.0.smithi071.stdout:7/781: chown d3/d17/d1c/d31/f56 900 1 2024-04-23T02:14:07.935 INFO:tasks.workunit.client.1.smithi192.stdout:2/913: symlink d8/d26/d29/d43/l12b 0 2024-04-23T02:14:07.936 INFO:tasks.workunit.client.1.smithi192.stdout:3/735: chown d1/d3/d17/d1a/d7c/dab/d6c/d84/d5d/fa7 1424 1 2024-04-23T02:14:07.936 INFO:tasks.workunit.client.0.smithi071.stdout:7/782: mknod d3/d17/d41/da8/c10f 0 2024-04-23T02:14:07.938 INFO:tasks.workunit.client.1.smithi192.stdout:7/766: chown d5/d43/d4c/d5c/l7c 29576 1 2024-04-23T02:14:07.938 INFO:tasks.workunit.client.1.smithi192.stdout:2/914: write d8/d26/d27/d57/f93 [715956,36148] 0 2024-04-23T02:14:07.939 INFO:tasks.workunit.client.1.smithi192.stdout:1/571: truncate dc/d16/d19/d74/fb6 490252 0 2024-04-23T02:14:07.942 INFO:tasks.workunit.client.0.smithi071.stdout:7/783: creat d3/d17/d41/d34/f110 x:0 0 0 2024-04-23T02:14:07.943 INFO:tasks.workunit.client.1.smithi192.stdout:4/865: write de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/de7/d101/ff0 [285020,95793] 0 2024-04-23T02:14:07.955 INFO:tasks.workunit.client.1.smithi192.stdout:3/736: dwrite d1/d3/d6/d7/d29/d3b/da9/dbd/ff0 [0,4194304] 0 2024-04-23T02:14:07.955 INFO:tasks.workunit.client.1.smithi192.stdout:2/915: dread d8/d26/d27/d31/f8b [0,4194304] 0 2024-04-23T02:14:07.955 INFO:tasks.workunit.client.1.smithi192.stdout:4/866: creat de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/f122 x:0 0 0 2024-04-23T02:14:07.955 INFO:tasks.workunit.client.1.smithi192.stdout:0/896: dread d6/df/d13/d25/d26/f82 [0,4194304] 0 2024-04-23T02:14:07.957 INFO:tasks.workunit.client.1.smithi192.stdout:2/916: chown d8/d14/f1a 39351068 1 2024-04-23T02:14:07.960 INFO:tasks.workunit.client.1.smithi192.stdout:4/867: chown de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/d59/fd3 69 1 2024-04-23T02:14:07.961 INFO:tasks.workunit.client.1.smithi192.stdout:0/897: truncate d6/df/d35/d8b/d8d/ff5 4934643 0 2024-04-23T02:14:07.963 INFO:tasks.workunit.client.1.smithi192.stdout:4/868: fsync de/d15/d53/d81/dac/f105 0 2024-04-23T02:14:07.964 INFO:tasks.workunit.client.1.smithi192.stdout:1/572: write dc/d14/d15/d35/f40 [870072,15974] 0 2024-04-23T02:14:07.964 INFO:tasks.workunit.client.1.smithi192.stdout:9/639: dread d0/d9/d44/f7c [0,4194304] 0 2024-04-23T02:14:07.966 INFO:tasks.workunit.client.1.smithi192.stdout:0/898: rmdir d6/df/d35/d8b/d8d 39 2024-04-23T02:14:07.967 INFO:tasks.workunit.client.1.smithi192.stdout:7/767: dread d5/d43/f6e [0,4194304] 0 2024-04-23T02:14:07.967 INFO:tasks.workunit.client.1.smithi192.stdout:3/737: write d1/d3/d17/d1f/d33/faa [1350146,56215] 0 2024-04-23T02:14:07.969 INFO:tasks.workunit.client.1.smithi192.stdout:4/869: readlink de/d15/d53/d81/de3/d74/d92/da0/dca/d34/df9/da1/l108 0 2024-04-23T02:14:07.970 INFO:tasks.workunit.client.1.smithi192.stdout:1/573: symlink dc/d14/d23/lbb 0 2024-04-23T02:14:07.971 INFO:tasks.workunit.client.1.smithi192.stdout:9/640: mkdir d0/d9/d44/d51/d5b/dd3 0 2024-04-23T02:14:07.971 INFO:tasks.workunit.client.0.smithi071.stdout:7/784: dread d3/dae/db7/dea/d5f/f6c [0,4194304] 0 2024-04-23T02:14:07.972 INFO:tasks.workunit.client.0.smithi071.stdout:7/785: dread - d3/dae/f101 zero size 2024-04-23T02:14:07.973 INFO:tasks.workunit.client.1.smithi192.stdout:9/641: chown d0/d9/f26 1 1 2024-04-23T02:14:07.973 INFO:tasks.workunit.client.0.smithi071.stdout:7/786: write d3/d17/d41/da8/f105 [605281,43956] 0 2024-04-23T02:14:07.975 INFO:tasks.workunit.client.0.smithi071.stdout:7/787: write d3/d17/d1c/d2a/d73/dba/ff6 [757257,1960] 0 2024-04-23T02:14:07.976 INFO:tasks.workunit.client.0.smithi071.stdout:7/788: dread - d3/dae/ffe zero size 2024-04-23T02:14:07.976 INFO:tasks.workunit.client.1.smithi192.stdout:7/768: fdatasync d5/d17/d1c/f45 0 2024-04-23T02:14:07.976 INFO:tasks.workunit.client.1.smithi192.stdout:1/574: fdatasync dc/d16/d19/d30/f84 0 2024-04-23T02:14:07.977 INFO:tasks.workunit.client.1.smithi192.stdout:4/870: mknod de/d15/d53/d81/de3/d74/d92/da0/dca/c123 0 2024-04-23T02:14:07.978 INFO:tasks.workunit.client.1.smithi192.stdout:3/738: dread d1/d3/d6/d7/f82 [0,4194304] 0 2024-04-23T02:14:07.981 INFO:tasks.workunit.client.0.smithi071.stdout:7/789: dread d3/d17/d1c/d2a/d73/ff9 [0,4194304] 0 2024-04-23T02:14:07.981 INFO:tasks.workunit.client.1.smithi192.stdout:9/642: write d0/d9/d44/f9d [654253,113252] 0 2024-04-23T02:14:07.982 INFO:tasks.workunit.client.0.smithi071.stdout:8/698: sync 2024-04-23T02:14:07.982 INFO:tasks.workunit.client.0.smithi071.stdout:9/635: sync 2024-04-23T02:14:07.982 INFO:tasks.workunit.client.0.smithi071.stdout:4/757: sync 2024-04-23T02:14:07.982 INFO:tasks.workunit.client.0.smithi071.stdout:0/688: sync 2024-04-23T02:14:07.982 INFO:tasks.workunit.client.0.smithi071.stdout:2/621: sync 2024-04-23T02:14:07.982 INFO:tasks.workunit.client.0.smithi071.stdout:5/754: sync 2024-04-23T02:14:07.985 INFO:tasks.workunit.client.1.smithi192.stdout:9/643: write d0/d9/db3/d5f/f7e [1654077,9436] 0 2024-04-23T02:14:07.985 INFO:tasks.workunit.client.0.smithi071.stdout:9/636: dread - d1/d32/d41/db7/d5a/dc6/faa zero size 2024-04-23T02:14:07.985 INFO:tasks.workunit.client.1.smithi192.stdout:4/871: creat de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d93/f124 x:0 0 0 2024-04-23T02:14:07.986 INFO:tasks.workunit.client.0.smithi071.stdout:0/689: mknod d0/d92/db2/cf3 0 2024-04-23T02:14:07.986 INFO:tasks.workunit.client.1.smithi192.stdout:2/917: dread d8/d26/d29/d80/f95 [0,4194304] 0 2024-04-23T02:14:07.988 INFO:tasks.workunit.client.1.smithi192.stdout:3/739: rmdir d1/d3/d6/d7/d29/d3b/da9/d7b/dd7 0 2024-04-23T02:14:07.988 INFO:tasks.workunit.client.0.smithi071.stdout:4/758: mkdir d2/d2a/d2d/d39/d42/d49/d67/d7e/df4 0 2024-04-23T02:14:07.989 INFO:tasks.workunit.client.0.smithi071.stdout:7/790: write d3/d17/d1c/d2a/f78 [45717,115099] 0 2024-04-23T02:14:07.990 INFO:tasks.workunit.client.1.smithi192.stdout:4/872: mkdir de/d15/d19/d1a/d106/d125 0 2024-04-23T02:14:07.993 INFO:tasks.workunit.client.0.smithi071.stdout:5/755: rename d3/d1d/db9/fd5 to d3/d34/d6b/f11a 0 2024-04-23T02:14:07.994 INFO:tasks.workunit.client.0.smithi071.stdout:2/622: dwrite d0/d2/d45/d7d/fbd [0,4194304] 0 2024-04-23T02:14:07.995 INFO:tasks.workunit.client.1.smithi192.stdout:3/740: truncate d1/d3/d17/d2b/d44/ffa 52454 0 2024-04-23T02:14:07.995 INFO:tasks.workunit.client.0.smithi071.stdout:4/759: write d2/d2a/da2/dd1/dde/f35 [2298176,2533] 0 2024-04-23T02:14:07.996 INFO:tasks.workunit.client.0.smithi071.stdout:8/699: getdents dc/d15/d50/d5f 0 2024-04-23T02:14:07.999 INFO:tasks.workunit.client.1.smithi192.stdout:4/873: creat de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/d59/d8a/f126 x:0 0 0 2024-04-23T02:14:08.000 INFO:tasks.workunit.client.1.smithi192.stdout:6/871: sync 2024-04-23T02:14:08.000 INFO:tasks.workunit.client.1.smithi192.stdout:9/644: truncate d0/d9/d44/d51/d5b/d61/fb9 2503 0 2024-04-23T02:14:08.000 INFO:tasks.workunit.client.1.smithi192.stdout:2/918: dwrite d8/d26/d74/d7c/f122 [0,4194304] 0 2024-04-23T02:14:08.000 INFO:tasks.workunit.client.1.smithi192.stdout:7/769: dread d5/d17/d1c/d59/d5a/d41/fd7 [0,4194304] 0 2024-04-23T02:14:08.001 INFO:tasks.workunit.client.0.smithi071.stdout:9/637: getdents d1/d4e 0 2024-04-23T02:14:08.003 INFO:tasks.workunit.client.0.smithi071.stdout:5/756: rename d3/d1d/daf/d10/da5/c97 to d3/d34/d6b/de4/c11b 0 2024-04-23T02:14:08.004 INFO:tasks.workunit.client.1.smithi192.stdout:3/741: symlink d1/d3/d6/d7/d29/d3b/da9/d8b/dfc/l104 0 2024-04-23T02:14:08.007 INFO:tasks.workunit.client.1.smithi192.stdout:1/575: dread f1 [4194304,4194304] 0 2024-04-23T02:14:08.007 INFO:tasks.workunit.client.1.smithi192.stdout:4/874: truncate de/d15/d19/fe0 323577 0 2024-04-23T02:14:08.008 INFO:tasks.workunit.client.1.smithi192.stdout:2/919: dread d8/d26/d29/d43/d5d/dbd/ffd [0,4194304] 0 2024-04-23T02:14:08.009 INFO:tasks.workunit.client.1.smithi192.stdout:9/645: creat d0/d9/d44/d51/d5b/d61/db1/fd4 x:0 0 0 2024-04-23T02:14:08.009 INFO:tasks.workunit.client.0.smithi071.stdout:8/700: unlink dc/f96 0 2024-04-23T02:14:08.009 INFO:tasks.workunit.client.0.smithi071.stdout:0/690: dwrite d0/d3/de/d10/d14/d32/d8a/df0/fc8 [0,4194304] 0 2024-04-23T02:14:08.011 INFO:tasks.workunit.client.1.smithi192.stdout:7/770: creat d5/d43/d4c/d5c/d67/da6/ddf/fe1 x:0 0 0 2024-04-23T02:14:08.012 INFO:tasks.workunit.client.1.smithi192.stdout:1/576: creat dc/d14/d15/fbc x:0 0 0 2024-04-23T02:14:08.014 INFO:tasks.workunit.client.1.smithi192.stdout:4/875: mknod de/d15/d53/d81/de3/d74/d92/c127 0 2024-04-23T02:14:08.015 INFO:tasks.workunit.client.1.smithi192.stdout:4/876: stat de/d15/d53/d81/de3/db2/lc0 0 2024-04-23T02:14:08.015 INFO:tasks.workunit.client.1.smithi192.stdout:9/646: mkdir d0/d9/dc7/da0/dd5 0 2024-04-23T02:14:08.017 INFO:tasks.workunit.client.0.smithi071.stdout:7/791: link d3/d51/df5/ff8 d3/d51/df5/f111 0 2024-04-23T02:14:08.017 INFO:tasks.workunit.client.0.smithi071.stdout:2/623: dwrite d0/d82/f87 [0,4194304] 0 2024-04-23T02:14:08.017 INFO:tasks.workunit.client.0.smithi071.stdout:9/638: rename d1/d4e/d66/d8b to d1/d32/d41/db7/d5a/dc6/d81/db6/de4 0 2024-04-23T02:14:08.018 INFO:tasks.workunit.client.1.smithi192.stdout:7/771: symlink d5/d43/le2 0 2024-04-23T02:14:08.018 INFO:tasks.workunit.client.1.smithi192.stdout:4/877: stat de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/l67 0 2024-04-23T02:14:08.018 INFO:tasks.workunit.client.1.smithi192.stdout:2/920: mknod d8/d26/d29/da4/dca/c12c 0 2024-04-23T02:14:08.020 INFO:tasks.workunit.client.1.smithi192.stdout:1/577: symlink dc/d16/d19/dae/lbd 0 2024-04-23T02:14:08.021 INFO:tasks.workunit.client.1.smithi192.stdout:7/772: fsync d5/d43/d4c/d5c/f72 0 2024-04-23T02:14:08.023 INFO:tasks.workunit.client.0.smithi071.stdout:2/624: write d0/d2/f90 [984626,35534] 0 2024-04-23T02:14:08.023 INFO:tasks.workunit.client.1.smithi192.stdout:7/773: chown d5/d43/d4c/f6c 25511 1 2024-04-23T02:14:08.024 INFO:tasks.workunit.client.1.smithi192.stdout:4/878: write de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/f115 [778796,30954] 0 2024-04-23T02:14:08.025 INFO:tasks.workunit.client.1.smithi192.stdout:9/647: link d0/d9/f26 d0/d9/d44/d51/d5b/d61/fd6 0 2024-04-23T02:14:08.026 INFO:tasks.workunit.client.0.smithi071.stdout:0/691: fsync d0/d3/de/d10/d14/d32/d37/d26/fc2 0 2024-04-23T02:14:08.028 INFO:tasks.workunit.client.0.smithi071.stdout:9/639: symlink d1/d4e/le5 0 2024-04-23T02:14:08.029 INFO:tasks.workunit.client.0.smithi071.stdout:7/792: unlink d3/d17/d1c/f20 0 2024-04-23T02:14:08.029 INFO:tasks.workunit.client.1.smithi192.stdout:1/578: rename dc/d16/d19/d30/c4a to dc/d14/d15/d35/cbe 0 2024-04-23T02:14:08.030 INFO:tasks.workunit.client.1.smithi192.stdout:9/648: symlink d0/d98/ld7 0 2024-04-23T02:14:08.030 INFO:tasks.workunit.client.1.smithi192.stdout:7/774: fdatasync d5/d17/d1c/d26/f28 0 2024-04-23T02:14:08.031 INFO:tasks.workunit.client.1.smithi192.stdout:6/872: dread d3/d28/d79/da0/dc2/d101/d80/dbb/fd6 [0,4194304] 0 2024-04-23T02:14:08.031 INFO:tasks.workunit.client.1.smithi192.stdout:4/879: creat de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d93/f128 x:0 0 0 2024-04-23T02:14:08.032 INFO:tasks.workunit.client.0.smithi071.stdout:4/760: dread d2/d2a/d2d/d39/d42/d49/d67/d3e/f93 [0,4194304] 0 2024-04-23T02:14:08.032 INFO:tasks.workunit.client.0.smithi071.stdout:2/625: readlink d0/d2/d26/d73/la8 0 2024-04-23T02:14:08.033 INFO:tasks.workunit.client.1.smithi192.stdout:9/649: chown d0/d9/d44/d51/cc5 15409909 1 2024-04-23T02:14:08.033 INFO:tasks.workunit.client.1.smithi192.stdout:1/579: write dc/d8b/d44/da9/fb8 [481994,27812] 0 2024-04-23T02:14:08.034 INFO:tasks.workunit.client.1.smithi192.stdout:7/775: write d5/d17/d1c/d59/fdd [1064576,112448] 0 2024-04-23T02:14:08.034 INFO:tasks.workunit.client.1.smithi192.stdout:1/580: readlink dc/d8b/l48 0 2024-04-23T02:14:08.035 INFO:tasks.workunit.client.0.smithi071.stdout:7/793: mkdir d3/d17/d1c/d31/d112 0 2024-04-23T02:14:08.036 INFO:tasks.workunit.client.0.smithi071.stdout:7/794: stat d3/d17/d1c/f3f 0 2024-04-23T02:14:08.037 INFO:tasks.workunit.client.0.smithi071.stdout:4/761: write d2/d2a/d73/d5e/ff0 [623183,98458] 0 2024-04-23T02:14:08.037 INFO:tasks.workunit.client.1.smithi192.stdout:9/650: write d0/d9/d44/d51/d5b/d61/db1/f93 [2262558,9729] 0 2024-04-23T02:14:08.038 INFO:tasks.workunit.client.0.smithi071.stdout:9/640: mknod d1/d32/db9/ce6 0 2024-04-23T02:14:08.038 INFO:tasks.workunit.client.1.smithi192.stdout:1/581: write dc/d14/d15/d35/f40 [1012931,78732] 0 2024-04-23T02:14:08.038 INFO:tasks.workunit.client.0.smithi071.stdout:8/701: dwrite dc/d15/da0/d55/f65 [0,4194304] 0 2024-04-23T02:14:08.040 INFO:tasks.workunit.client.0.smithi071.stdout:9/641: write d1/f37 [3869878,79676] 0 2024-04-23T02:14:08.043 INFO:tasks.workunit.client.1.smithi192.stdout:7/776: rmdir d5/d17/d1c/d4a/dd2 0 2024-04-23T02:14:08.044 INFO:tasks.workunit.client.0.smithi071.stdout:7/795: mknod d3/d17/d41/d34/d39/c113 0 2024-04-23T02:14:08.044 INFO:tasks.workunit.client.1.smithi192.stdout:6/873: getdents d3/d28/d79/da0/dc2/d101/d80/d8e/d9d/ded 0 2024-04-23T02:14:08.044 INFO:tasks.workunit.client.0.smithi071.stdout:0/692: rename d0/d3/de/d10/d14/d32/d44/l4c to d0/d3/de/d10/d14/d32/d44/d31/d52/lf4 0 2024-04-23T02:14:08.045 INFO:tasks.workunit.client.0.smithi071.stdout:4/762: read d2/d2a/d2d/d39/d42/f55 [405230,80285] 0 2024-04-23T02:14:08.047 INFO:tasks.workunit.client.1.smithi192.stdout:9/651: rename d0/d9/d44/d51/d64 to d0/d9/d44/d58/d89/dd8 0 2024-04-23T02:14:08.047 INFO:tasks.workunit.client.0.smithi071.stdout:7/796: write d3/d17/d1c/d2a/d43/f102 [342916,98182] 0 2024-04-23T02:14:08.048 INFO:tasks.workunit.client.0.smithi071.stdout:8/702: read dc/f28 [6314576,65554] 0 2024-04-23T02:14:08.049 INFO:tasks.workunit.client.0.smithi071.stdout:9/642: unlink d1/d32/d41/db7/f31 0 2024-04-23T02:14:08.049 INFO:tasks.workunit.client.1.smithi192.stdout:1/582: dwrite dc/d80/fb4 [0,4194304] 0 2024-04-23T02:14:08.050 INFO:tasks.workunit.client.0.smithi071.stdout:8/703: chown dc/f54 514004 1 2024-04-23T02:14:08.051 INFO:tasks.workunit.client.1.smithi192.stdout:1/583: readlink dc/d14/d15/l6f 0 2024-04-23T02:14:08.053 INFO:tasks.workunit.client.0.smithi071.stdout:7/797: write d3/d51/df5/ff8 [1866776,56873] 0 2024-04-23T02:14:08.054 INFO:tasks.workunit.client.1.smithi192.stdout:4/880: getdents de/d15/d53/d81/de3/d74/d92/da0/dca/d49 0 2024-04-23T02:14:08.054 INFO:tasks.workunit.client.0.smithi071.stdout:9/643: symlink d1/d32/d41/db7/d5a/le7 0 2024-04-23T02:14:08.056 INFO:tasks.workunit.client.1.smithi192.stdout:6/874: fsync d3/d28/d79/da0/dc2/d101/d80/dbb/fd6 0 2024-04-23T02:14:08.057 INFO:tasks.workunit.client.1.smithi192.stdout:1/584: rmdir dc/d8b/d34/d70 39 2024-04-23T02:14:08.058 INFO:tasks.workunit.client.0.smithi071.stdout:7/798: symlink d3/d17/d41/d34/d39/l114 0 2024-04-23T02:14:08.058 INFO:tasks.workunit.client.1.smithi192.stdout:9/652: read d0/d9/f42 [755178,121615] 0 2024-04-23T02:14:08.059 INFO:tasks.workunit.client.0.smithi071.stdout:7/799: dread - d3/d51/d80/de9/f106 zero size 2024-04-23T02:14:08.059 INFO:tasks.workunit.client.1.smithi192.stdout:1/585: write dc/d14/f9d [490195,100825] 0 2024-04-23T02:14:08.061 INFO:tasks.workunit.client.1.smithi192.stdout:4/881: unlink de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/f115 0 2024-04-23T02:14:08.062 INFO:tasks.workunit.client.0.smithi071.stdout:7/800: creat d3/dae/db7/dea/dc5/f115 x:0 0 0 2024-04-23T02:14:08.062 INFO:tasks.workunit.client.1.smithi192.stdout:9/653: mkdir d0/d9/d44/d58/dd9 0 2024-04-23T02:14:08.064 INFO:tasks.workunit.client.1.smithi192.stdout:6/875: creat d3/d28/d79/da0/dc2/d101/d80/d8e/d9d/f11d x:0 0 0 2024-04-23T02:14:08.064 INFO:tasks.workunit.client.0.smithi071.stdout:5/757: dread d3/da/f33 [4194304,4194304] 0 2024-04-23T02:14:08.064 INFO:tasks.workunit.client.0.smithi071.stdout:7/801: mknod d3/dae/db7/c116 0 2024-04-23T02:14:08.065 INFO:tasks.workunit.client.1.smithi192.stdout:4/882: mknod de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/c129 0 2024-04-23T02:14:08.066 INFO:tasks.workunit.client.0.smithi071.stdout:7/802: mkdir d3/d17/dca/d117 0 2024-04-23T02:14:08.069 INFO:tasks.workunit.client.0.smithi071.stdout:4/763: read d2/d2a/d73/d5e/ff0 [563624,381] 0 2024-04-23T02:14:08.069 INFO:tasks.workunit.client.1.smithi192.stdout:6/876: rename d3/c99 to d3/d28/df8/d7c/dda/c11e 0 2024-04-23T02:14:08.069 INFO:tasks.workunit.client.0.smithi071.stdout:7/803: write d3/d17/d1c/d2a/d43/f4e [3847175,28696] 0 2024-04-23T02:14:08.070 INFO:tasks.workunit.client.1.smithi192.stdout:4/883: mknod de/d15/d53/d81/de3/d74/d92/da0/dca/d49/de6/c12a 0 2024-04-23T02:14:08.071 INFO:tasks.workunit.client.0.smithi071.stdout:7/804: dread - d3/d17/d1c/d2a/d73/dba/fe3 zero size 2024-04-23T02:14:08.076 INFO:tasks.workunit.client.1.smithi192.stdout:7/777: dread d5/d17/d1c/d57/f48 [0,4194304] 0 2024-04-23T02:14:08.076 INFO:tasks.workunit.client.0.smithi071.stdout:4/764: mkdir d2/d2a/d2d/d39/d42/d49/d67/d7e/d82/df5 0 2024-04-23T02:14:08.077 INFO:tasks.workunit.client.1.smithi192.stdout:6/877: fsync d3/d15/d1b/f2a 0 2024-04-23T02:14:08.079 INFO:tasks.workunit.client.1.smithi192.stdout:4/884: creat de/d15/d53/d81/de3/f12b x:0 0 0 2024-04-23T02:14:08.080 INFO:tasks.workunit.client.1.smithi192.stdout:6/878: mknod d3/d28/df8/c11f 0 2024-04-23T02:14:08.081 INFO:tasks.workunit.client.0.smithi071.stdout:4/765: mkdir d2/df6 0 2024-04-23T02:14:08.081 INFO:tasks.workunit.client.0.smithi071.stdout:9/644: dread d1/d32/dc5/dd1/f9d [0,4194304] 0 2024-04-23T02:14:08.082 INFO:tasks.workunit.client.0.smithi071.stdout:8/704: dread dc/d15/da0/f14 [4194304,4194304] 0 2024-04-23T02:14:08.082 INFO:tasks.workunit.client.1.smithi192.stdout:4/885: chown de/l4b 2546743 1 2024-04-23T02:14:08.084 INFO:tasks.workunit.client.1.smithi192.stdout:7/778: rename d5/d17/d1c/d59/d5a/d41/fd7 to d5/d43/d4c/d5c/d97/fe3 0 2024-04-23T02:14:08.086 INFO:tasks.workunit.client.1.smithi192.stdout:7/779: write d5/d17/d1c/d26/f34 [678764,36163] 0 2024-04-23T02:14:08.086 INFO:tasks.workunit.client.0.smithi071.stdout:7/805: dwrite d3/d17/d1c/d2a/d73/fc6 [0,4194304] 0 2024-04-23T02:14:08.086 INFO:tasks.workunit.client.1.smithi192.stdout:6/879: read d3/d28/d79/da0/dc2/d101/d80/d8e/f115 [97527,90434] 0 2024-04-23T02:14:08.088 INFO:tasks.workunit.client.0.smithi071.stdout:4/766: mknod d2/d2a/da2/dd1/cf7 0 2024-04-23T02:14:08.088 INFO:tasks.workunit.client.0.smithi071.stdout:5/758: write d3/da/f92 [114188,61695] 0 2024-04-23T02:14:08.089 INFO:tasks.workunit.client.1.smithi192.stdout:6/880: write d3/d15/d5b/f82 [2128020,99027] 0 2024-04-23T02:14:08.089 INFO:tasks.workunit.client.0.smithi071.stdout:8/705: mkdir dc/d56/de1 0 2024-04-23T02:14:08.090 INFO:tasks.workunit.client.1.smithi192.stdout:4/886: getdents de/d15/d19/d1a/d106/d125 0 2024-04-23T02:14:08.090 INFO:tasks.workunit.client.0.smithi071.stdout:9/645: getdents d1/d32/d41/db7/d71/dbf 0 2024-04-23T02:14:08.092 INFO:tasks.workunit.client.0.smithi071.stdout:8/706: fsync dc/d56/d64/fd7 0 2024-04-23T02:14:08.093 INFO:tasks.workunit.client.0.smithi071.stdout:7/806: mkdir d3/d17/d41/d118 0 2024-04-23T02:14:08.094 INFO:tasks.workunit.client.1.smithi192.stdout:7/780: dread d5/d43/d4c/d5c/d67/da6/fb6 [0,4194304] 0 2024-04-23T02:14:08.098 INFO:tasks.workunit.client.0.smithi071.stdout:0/693: dwrite d0/d3/de/d10/d14/d32/d8a/df0/f77 [0,4194304] 0 2024-04-23T02:14:08.098 INFO:tasks.workunit.client.0.smithi071.stdout:5/759: mknod d3/da/def/dff/c11c 0 2024-04-23T02:14:08.099 INFO:tasks.workunit.client.0.smithi071.stdout:7/807: mknod d3/d17/c119 0 2024-04-23T02:14:08.099 INFO:tasks.workunit.client.0.smithi071.stdout:7/808: stat d3/dae/fcc 0 2024-04-23T02:14:08.100 INFO:tasks.workunit.client.1.smithi192.stdout:6/881: creat d3/d6c/f120 x:0 0 0 2024-04-23T02:14:08.101 INFO:tasks.workunit.client.0.smithi071.stdout:9/646: symlink d1/d32/dc5/dd8/le8 0 2024-04-23T02:14:08.102 INFO:tasks.workunit.client.0.smithi071.stdout:9/647: fdatasync d1/f37 0 2024-04-23T02:14:08.103 INFO:tasks.workunit.client.0.smithi071.stdout:5/760: unlink d3/d1d/daf/d10/c88 0 2024-04-23T02:14:08.107 INFO:tasks.workunit.client.0.smithi071.stdout:5/761: rmdir d3/d1d/daf/d83 39 2024-04-23T02:14:08.108 INFO:tasks.workunit.client.1.smithi192.stdout:6/882: mkdir d3/d18/dd4/dff/df5/d121 0 2024-04-23T02:14:08.108 INFO:tasks.workunit.client.0.smithi071.stdout:7/809: dwrite d3/d51/f7f [0,4194304] 0 2024-04-23T02:14:08.110 INFO:tasks.workunit.client.0.smithi071.stdout:9/648: creat d1/d32/d41/db7/d39/fe9 x:0 0 0 2024-04-23T02:14:08.113 INFO:tasks.workunit.client.0.smithi071.stdout:8/707: rmdir dc/d56 39 2024-04-23T02:14:08.115 INFO:tasks.workunit.client.0.smithi071.stdout:9/649: creat d1/d4e/d54/fea x:0 0 0 2024-04-23T02:14:08.117 INFO:tasks.workunit.client.0.smithi071.stdout:8/708: write dc/d15/da0/d8d/fcd [2056460,32448] 0 2024-04-23T02:14:08.118 INFO:tasks.workunit.client.0.smithi071.stdout:7/810: mkdir d3/d17/d1c/d31/d10c/d11a 0 2024-04-23T02:14:08.119 INFO:tasks.workunit.client.1.smithi192.stdout:7/781: write d5/d43/f6e [555472,58605] 0 2024-04-23T02:14:08.122 INFO:tasks.workunit.client.0.smithi071.stdout:8/709: symlink dc/le2 0 2024-04-23T02:14:08.122 INFO:tasks.workunit.client.1.smithi192.stdout:7/782: dread d5/d17/d1c/d59/d5a/d41/d50/d9f/fa4 [4194304,4194304] 0 2024-04-23T02:14:08.123 INFO:tasks.workunit.client.0.smithi071.stdout:7/811: truncate d3/d51/df5/ff8 1988930 0 2024-04-23T02:14:08.123 INFO:tasks.workunit.client.0.smithi071.stdout:9/650: mknod d1/d32/d41/db7/ceb 0 2024-04-23T02:14:08.124 INFO:tasks.workunit.client.0.smithi071.stdout:4/767: dread d2/d2a/d2d/d39/fd8 [0,4194304] 0 2024-04-23T02:14:08.125 INFO:tasks.workunit.client.0.smithi071.stdout:5/762: dwrite d3/da/f92 [0,4194304] 0 2024-04-23T02:14:08.125 INFO:tasks.workunit.client.0.smithi071.stdout:9/651: readlink d1/d32/l48 0 2024-04-23T02:14:08.126 INFO:tasks.workunit.client.0.smithi071.stdout:7/812: fsync d3/dae/ffe 0 2024-04-23T02:14:08.126 INFO:tasks.workunit.client.1.smithi192.stdout:6/883: dwrite d3/d18/d58/f75 [4194304,4194304] 0 2024-04-23T02:14:08.128 INFO:tasks.workunit.client.0.smithi071.stdout:5/763: fsync d3/f22 0 2024-04-23T02:14:08.131 INFO:tasks.workunit.client.0.smithi071.stdout:8/710: rename dc/d3a/c8e to dc/d3a/ce3 0 2024-04-23T02:14:08.133 INFO:tasks.workunit.client.0.smithi071.stdout:7/813: mkdir d3/d17/d11b 0 2024-04-23T02:14:08.134 INFO:tasks.workunit.client.1.smithi192.stdout:6/884: symlink d3/d28/df8/d3a/d51/ddc/l122 0 2024-04-23T02:14:08.135 INFO:tasks.workunit.client.0.smithi071.stdout:4/768: symlink d2/d2a/da2/dd1/dde/d23/d4f/db5/db6/lf8 0 2024-04-23T02:14:08.135 INFO:tasks.workunit.client.1.smithi192.stdout:7/783: link d5/d17/d1c/d59/d5a/d9b/lcf d5/d17/d1c/d59/d5a/d41/le4 0 2024-04-23T02:14:08.136 INFO:tasks.workunit.client.0.smithi071.stdout:9/652: dread d1/d32/f4a [0,4194304] 0 2024-04-23T02:14:08.136 INFO:tasks.workunit.client.0.smithi071.stdout:0/694: dwrite d0/f2 [4194304,4194304] 0 2024-04-23T02:14:08.136 INFO:tasks.workunit.client.0.smithi071.stdout:5/764: fsync d3/d8f/ff2 0 2024-04-23T02:14:08.138 INFO:tasks.workunit.client.1.smithi192.stdout:6/885: truncate d3/d28/d79/da0/dc2/d101/d80/d8e/d9d/f11d 30855 0 2024-04-23T02:14:08.138 INFO:tasks.workunit.client.1.smithi192.stdout:7/784: fsync d5/d43/d4c/d5c/d67/f82 0 2024-04-23T02:14:08.140 INFO:tasks.workunit.client.0.smithi071.stdout:0/695: truncate d0/d76/fab 999583 0 2024-04-23T02:14:08.140 INFO:tasks.workunit.client.1.smithi192.stdout:7/785: fsync d5/d17/d1c/d59/fd0 0 2024-04-23T02:14:08.144 INFO:tasks.workunit.client.1.smithi192.stdout:6/886: truncate d3/d15/d1b/f2a 4183194 0 2024-04-23T02:14:08.144 INFO:tasks.workunit.client.1.smithi192.stdout:7/786: rmdir d5/d17/d1c/d57/d8b 39 2024-04-23T02:14:08.145 INFO:tasks.workunit.client.1.smithi192.stdout:7/787: write d5/d17/d1c/d59/d5a/d41/d50/f69 [1328264,81143] 0 2024-04-23T02:14:08.146 INFO:tasks.workunit.client.0.smithi071.stdout:9/653: fsync d1/d32/d3a/d90/dc2/fe0 0 2024-04-23T02:14:08.147 INFO:tasks.workunit.client.1.smithi192.stdout:7/788: mkdir d5/d17/d1c/d4a/daa/de5 0 2024-04-23T02:14:08.149 INFO:tasks.workunit.client.0.smithi071.stdout:0/696: fsync d0/d3/de/d10/d14/d32/d37/d26/fe2 0 2024-04-23T02:14:08.149 INFO:tasks.workunit.client.1.smithi192.stdout:7/789: write d5/d17/d1c/d59/da8/fb0 [558610,56550] 0 2024-04-23T02:14:08.150 INFO:tasks.workunit.client.0.smithi071.stdout:9/654: readlink d1/d32/d41/db7/d5a/dc6/d9f/ld4 0 2024-04-23T02:14:08.154 INFO:tasks.workunit.client.1.smithi192.stdout:7/790: rename d5/d43/c7d to d5/d17/d1c/d59/d5a/d41/d50/ce6 0 2024-04-23T02:14:08.161 INFO:tasks.workunit.client.1.smithi192.stdout:7/791: dwrite d5/d17/d1c/d57/f90 [0,4194304] 0 2024-04-23T02:14:08.163 INFO:tasks.workunit.client.0.smithi071.stdout:8/711: dread dc/d15/da0/d8d/f92 [0,4194304] 0 2024-04-23T02:14:08.164 INFO:tasks.workunit.client.1.smithi192.stdout:7/792: mkdir d5/d43/d4c/d5c/de7 0 2024-04-23T02:14:08.167 INFO:tasks.workunit.client.1.smithi192.stdout:6/887: dwrite d3/d18/f6f [0,4194304] 0 2024-04-23T02:14:08.168 INFO:tasks.workunit.client.1.smithi192.stdout:7/793: fdatasync d5/d17/d1c/d57/f48 0 2024-04-23T02:14:08.171 INFO:tasks.workunit.client.1.smithi192.stdout:6/888: truncate d3/d28/df8/f60 2400118 0 2024-04-23T02:14:08.172 INFO:tasks.workunit.client.0.smithi071.stdout:7/814: dread d3/d17/d41/d34/f35 [0,4194304] 0 2024-04-23T02:14:08.174 INFO:tasks.workunit.client.0.smithi071.stdout:1/591: sync 2024-04-23T02:14:08.174 INFO:tasks.workunit.client.1.smithi192.stdout:6/889: creat d3/d18/dd4/dff/f123 x:0 0 0 2024-04-23T02:14:08.176 INFO:tasks.workunit.client.1.smithi192.stdout:6/890: symlink d3/d28/d79/dbe/dcb/de9/l124 0 2024-04-23T02:14:08.180 INFO:tasks.workunit.client.0.smithi071.stdout:0/697: mknod d0/d3/de/d10/d14/d32/d44/d59/cf5 0 2024-04-23T02:14:08.181 INFO:tasks.workunit.client.0.smithi071.stdout:5/765: fdatasync d3/d1d/daf/d83/f9b 0 2024-04-23T02:14:08.186 INFO:tasks.workunit.client.0.smithi071.stdout:4/769: truncate d2/d2a/d2d/d39/d42/d49/d67/f3a 2100026 0 2024-04-23T02:14:08.187 INFO:tasks.workunit.client.0.smithi071.stdout:9/655: getdents d1/d8d 0 2024-04-23T02:14:08.188 INFO:tasks.workunit.client.0.smithi071.stdout:0/698: dread d0/d3/de/d10/d14/d32/fa4 [0,4194304] 0 2024-04-23T02:14:08.192 INFO:tasks.workunit.client.0.smithi071.stdout:9/656: creat d1/d32/d3a/d90/fec x:0 0 0 2024-04-23T02:14:08.192 INFO:tasks.workunit.client.0.smithi071.stdout:8/712: dwrite dc/d56/d64/f69 [0,4194304] 0 2024-04-23T02:14:08.193 INFO:tasks.workunit.client.0.smithi071.stdout:5/766: link d3/d1d/d43/f65 d3/d1d/d43/dd2/da9/f11d 0 2024-04-23T02:14:08.194 INFO:tasks.workunit.client.0.smithi071.stdout:4/770: mknod d2/d2a/d73/cf9 0 2024-04-23T02:14:08.195 INFO:tasks.workunit.client.0.smithi071.stdout:8/713: chown dc/d15/d72/d95/cce 185 1 2024-04-23T02:14:08.196 INFO:tasks.workunit.client.0.smithi071.stdout:0/699: symlink d0/d3/de/d10/d14/d32/d37/d26/lf6 0 2024-04-23T02:14:08.198 INFO:tasks.workunit.client.0.smithi071.stdout:9/657: rmdir d1/d4e/d54/d75 39 2024-04-23T02:14:08.200 INFO:tasks.workunit.client.0.smithi071.stdout:0/700: write d0/d3/de/d10/d14/f4d [2187918,108547] 0 2024-04-23T02:14:08.200 INFO:tasks.workunit.client.1.smithi192.stdout:7/794: dread d5/d43/d4c/d5c/d67/da6/da9/db9/dc0/fdb [0,4194304] 0 2024-04-23T02:14:08.200 INFO:tasks.workunit.client.1.smithi192.stdout:5/754: sync 2024-04-23T02:14:08.201 INFO:tasks.workunit.client.1.smithi192.stdout:5/755: fsync d2/db/d2b/f7f 0 2024-04-23T02:14:08.204 INFO:tasks.workunit.client.0.smithi071.stdout:4/771: rename d2/d2a/d2d/d39/d42/f55 to d2/d2a/d2d/d7b/ffa 0 2024-04-23T02:14:08.204 INFO:tasks.workunit.client.1.smithi192.stdout:5/756: creat d2/db/d2b/df7/ffb x:0 0 0 2024-04-23T02:14:08.206 INFO:tasks.workunit.client.1.smithi192.stdout:5/757: write d2/db/d16/d1b/f9f [3524257,50294] 0 2024-04-23T02:14:08.206 INFO:tasks.workunit.client.0.smithi071.stdout:8/714: dwrite dc/d15/da0/d32/f4b [0,4194304] 0 2024-04-23T02:14:08.209 INFO:tasks.workunit.client.1.smithi192.stdout:5/758: symlink d2/d6/d2d/d41/d43/d98/lfc 0 2024-04-23T02:14:08.210 INFO:tasks.workunit.client.0.smithi071.stdout:9/658: rmdir d1/d32/d41/db7/d39/d42/dcc 39 2024-04-23T02:14:08.211 INFO:tasks.workunit.client.1.smithi192.stdout:5/759: write d2/d2e/fd8 [462118,14014] 0 2024-04-23T02:14:08.212 INFO:tasks.workunit.client.1.smithi192.stdout:5/760: readlink d2/db/d16/d48/d4f/dd1/ld4 0 2024-04-23T02:14:08.212 INFO:tasks.workunit.client.0.smithi071.stdout:5/767: rmdir d3/da/d1a/d1c/d46/d4e/dd8/df3 0 2024-04-23T02:14:08.212 INFO:tasks.workunit.client.0.smithi071.stdout:9/659: chown d1/d32/d3a/d5f/cd5 488353033 1 2024-04-23T02:14:08.214 INFO:tasks.workunit.client.0.smithi071.stdout:8/715: mkdir dc/d15/d5d/de4 0 2024-04-23T02:14:08.215 INFO:tasks.workunit.client.1.smithi192.stdout:7/795: write d5/d17/d1c/d59/d5a/d70/f88 [627069,77707] 0 2024-04-23T02:14:08.216 INFO:tasks.workunit.client.0.smithi071.stdout:0/701: dwrite d0/d63/f66 [0,4194304] 0 2024-04-23T02:14:08.218 INFO:tasks.workunit.client.0.smithi071.stdout:5/768: mkdir d3/da/def/dff/d11e 0 2024-04-23T02:14:08.220 INFO:tasks.workunit.client.0.smithi071.stdout:8/716: symlink dc/d15/da0/d30/le5 0 2024-04-23T02:14:08.221 INFO:tasks.workunit.client.0.smithi071.stdout:5/769: write d3/da/d1a/d1c/d46/d4e/dd8/ff7 [979514,51423] 0 2024-04-23T02:14:08.223 INFO:tasks.workunit.client.0.smithi071.stdout:4/772: dread d2/d2a/da2/dd1/dde/d23/f5d [0,4194304] 0 2024-04-23T02:14:08.223 INFO:tasks.workunit.client.0.smithi071.stdout:5/770: chown d3/d1d/daf/f89 1363 1 2024-04-23T02:14:08.226 INFO:tasks.workunit.client.0.smithi071.stdout:8/717: dread - dc/d15/d72/fa8 zero size 2024-04-23T02:14:08.228 INFO:tasks.workunit.client.1.smithi192.stdout:7/796: truncate d5/d17/d1c/d26/f34 88605 0 2024-04-23T02:14:08.229 INFO:tasks.workunit.client.0.smithi071.stdout:5/771: symlink d3/d1d/daf/l11f 0 2024-04-23T02:14:08.230 INFO:tasks.workunit.client.1.smithi192.stdout:7/797: unlink d5/d17/d1c/d59/c91 0 2024-04-23T02:14:08.230 INFO:tasks.workunit.client.0.smithi071.stdout:4/773: mknod d2/d2a/d73/d5e/cfb 0 2024-04-23T02:14:08.231 INFO:tasks.workunit.client.1.smithi192.stdout:7/798: write d5/d17/d1c/d57/f90 [4219409,65103] 0 2024-04-23T02:14:08.233 INFO:tasks.workunit.client.0.smithi071.stdout:8/718: mkdir dc/d15/d72/de6 0 2024-04-23T02:14:08.234 INFO:tasks.workunit.client.0.smithi071.stdout:5/772: read d3/d1d/d43/dd2/fda [93354,119956] 0 2024-04-23T02:14:08.235 INFO:tasks.workunit.client.0.smithi071.stdout:8/719: stat dc/c1b 0 2024-04-23T02:14:08.236 INFO:tasks.workunit.client.0.smithi071.stdout:8/720: dread - dc/d15/d72/d95/fbc zero size 2024-04-23T02:14:08.240 INFO:tasks.workunit.client.0.smithi071.stdout:9/660: dread d1/d32/d41/db7/d39/f53 [0,4194304] 0 2024-04-23T02:14:08.242 INFO:tasks.workunit.client.0.smithi071.stdout:8/721: unlink dc/d3a/da4/db0/fc5 0 2024-04-23T02:14:08.242 INFO:tasks.workunit.client.0.smithi071.stdout:5/773: link d3/da/d1a/d1c/d46/l79 d3/d1d/daf/d10/l120 0 2024-04-23T02:14:08.243 INFO:tasks.workunit.client.0.smithi071.stdout:5/774: readlink d3/d1d/d43/d4b/le8 0 2024-04-23T02:14:08.243 INFO:tasks.workunit.client.1.smithi192.stdout:5/761: dread d2/db/fad [0,4194304] 0 2024-04-23T02:14:08.244 INFO:tasks.workunit.client.0.smithi071.stdout:5/775: chown d3/cc8 2218 1 2024-04-23T02:14:08.244 INFO:tasks.workunit.client.0.smithi071.stdout:5/776: chown d3/d1d/d43/f109 517673 1 2024-04-23T02:14:08.245 INFO:tasks.workunit.client.0.smithi071.stdout:9/661: mkdir d1/d32/d41/db7/d5a/dc6/d9f/ded 0 2024-04-23T02:14:08.245 INFO:tasks.workunit.client.1.smithi192.stdout:7/799: dwrite d5/f15 [0,4194304] 0 2024-04-23T02:14:08.245 INFO:tasks.workunit.client.0.smithi071.stdout:1/592: write fa [4610001,90797] 0 2024-04-23T02:14:08.246 INFO:tasks.workunit.client.1.smithi192.stdout:7/800: write d5/f15 [2276223,45323] 0 2024-04-23T02:14:08.247 INFO:tasks.workunit.client.0.smithi071.stdout:4/774: dread d2/d2a/da2/dd1/dde/d23/ff [0,4194304] 0 2024-04-23T02:14:08.247 INFO:tasks.workunit.client.0.smithi071.stdout:8/722: creat dc/d15/d5d/dc6/fe7 x:0 0 0 2024-04-23T02:14:08.248 INFO:tasks.workunit.client.0.smithi071.stdout:7/815: truncate d3/d17/d1c/d2a/d43/d86/fc3 2832740 0 2024-04-23T02:14:08.249 INFO:tasks.workunit.client.0.smithi071.stdout:5/777: creat d3/d1d/db9/f121 x:0 0 0 2024-04-23T02:14:08.250 INFO:tasks.workunit.client.0.smithi071.stdout:9/662: chown d1/fe 21604694 1 2024-04-23T02:14:08.251 INFO:tasks.workunit.client.1.smithi192.stdout:7/801: mknod d5/d43/d4c/d5c/d97/ce8 0 2024-04-23T02:14:08.251 INFO:tasks.workunit.client.0.smithi071.stdout:1/593: chown db/d34/l84 14 1 2024-04-23T02:14:08.252 INFO:tasks.workunit.client.1.smithi192.stdout:7/802: write d5/d17/d1c/d59/d5a/d41/d50/f35 [2152218,25931] 0 2024-04-23T02:14:08.254 INFO:tasks.workunit.client.0.smithi071.stdout:8/723: read dc/d78/f91 [308622,88326] 0 2024-04-23T02:14:08.255 INFO:tasks.workunit.client.0.smithi071.stdout:4/775: fsync d2/d2a/f3b 0 2024-04-23T02:14:08.256 INFO:tasks.workunit.client.0.smithi071.stdout:7/816: creat d3/d17/d1c/d2a/d73/d83/de7/f11c x:0 0 0 2024-04-23T02:14:08.259 INFO:tasks.workunit.client.0.smithi071.stdout:1/594: rmdir db/d11 39 2024-04-23T02:14:08.262 INFO:tasks.workunit.client.0.smithi071.stdout:9/663: dwrite d1/d32/d3a/d90/fec [0,4194304] 0 2024-04-23T02:14:08.266 INFO:tasks.workunit.client.0.smithi071.stdout:1/595: unlink db/df/d13/c17 0 2024-04-23T02:14:08.270 INFO:tasks.workunit.client.0.smithi071.stdout:9/664: dread d1/fe [0,4194304] 0 2024-04-23T02:14:08.272 INFO:tasks.workunit.client.0.smithi071.stdout:1/596: creat db/df/d13/d2b/fbf x:0 0 0 2024-04-23T02:14:08.274 INFO:tasks.workunit.client.0.smithi071.stdout:0/702: dwrite d0/d3/de/d10/db9/fc4 [0,4194304] 0 2024-04-23T02:14:08.276 INFO:tasks.workunit.client.0.smithi071.stdout:1/597: truncate db/df/d13/d2f/f3e 2063754 0 2024-04-23T02:14:08.277 INFO:tasks.workunit.client.0.smithi071.stdout:4/776: rename d2/d2a/f36 to d2/d2a/d2d/d39/d42/d49/d67/d7e/dae/dab/ffc 0 2024-04-23T02:14:08.279 INFO:tasks.workunit.client.0.smithi071.stdout:0/703: read d0/d3/de/d10/d14/f4d [288592,19014] 0 2024-04-23T02:14:08.280 INFO:tasks.workunit.client.0.smithi071.stdout:9/665: dread d1/d32/d41/db7/d39/f53 [0,4194304] 0 2024-04-23T02:14:08.285 INFO:tasks.workunit.client.0.smithi071.stdout:9/666: fsync d1/f86 0 2024-04-23T02:14:08.288 INFO:tasks.workunit.client.0.smithi071.stdout:8/724: dread dc/d3a/f47 [0,4194304] 0 2024-04-23T02:14:08.289 INFO:tasks.workunit.client.0.smithi071.stdout:5/778: dwrite d3/da/d1a/d1c/fb2 [0,4194304] 0 2024-04-23T02:14:08.290 INFO:tasks.workunit.client.0.smithi071.stdout:8/725: truncate dc/d15/d5d/dc6/fe7 371744 0 2024-04-23T02:14:08.336 INFO:tasks.workunit.client.0.smithi071.stdout:7/817: truncate d3/d17/d41/d34/f35 2185900 0 2024-04-23T02:14:08.337 INFO:tasks.workunit.client.0.smithi071.stdout:9/667: symlink d1/d4e/lee 0 2024-04-23T02:14:08.338 INFO:tasks.workunit.client.0.smithi071.stdout:9/668: stat d1/d32/d41/db7/d39/d42/c65 0 2024-04-23T02:14:08.339 INFO:tasks.workunit.client.0.smithi071.stdout:5/779: symlink d3/da/d1a/d1c/d46/d4e/l122 0 2024-04-23T02:14:08.342 INFO:tasks.workunit.client.0.smithi071.stdout:1/598: truncate db/df/d13/d20/f32 4495026 0 2024-04-23T02:14:08.342 INFO:tasks.workunit.client.0.smithi071.stdout:9/669: mknod d1/d32/dc5/cef 0 2024-04-23T02:14:08.345 INFO:tasks.workunit.client.0.smithi071.stdout:9/670: fdatasync d1/d32/d3a/d90/dc2/fe0 0 2024-04-23T02:14:08.347 INFO:tasks.workunit.client.0.smithi071.stdout:0/704: dwrite d0/d3/de/d48/d7e/d8b/f22 [4194304,4194304] 0 2024-04-23T02:14:08.348 INFO:tasks.workunit.client.0.smithi071.stdout:0/705: fsync d0/d3/de/f28 0 2024-04-23T02:14:08.350 INFO:tasks.workunit.client.0.smithi071.stdout:1/599: mkdir db/df/d13/d20/dc0 0 2024-04-23T02:14:08.351 INFO:tasks.workunit.client.0.smithi071.stdout:8/726: creat dc/d15/fe8 x:0 0 0 2024-04-23T02:14:08.353 INFO:tasks.workunit.client.0.smithi071.stdout:7/818: link d3/d17/d1c/d2a/d73/dba/c10e d3/d17/d41/da8/c11d 0 2024-04-23T02:14:08.355 INFO:tasks.workunit.client.0.smithi071.stdout:7/819: chown d3/d17/d1c/d2a/d73/dba/fcd 8 1 2024-04-23T02:14:08.356 INFO:tasks.workunit.client.0.smithi071.stdout:8/727: stat dc/d15/da0/d32/c71 0 2024-04-23T02:14:08.358 INFO:tasks.workunit.client.0.smithi071.stdout:1/600: symlink db/df/d13/d2f/d48/lc1 0 2024-04-23T02:14:08.360 INFO:tasks.workunit.client.0.smithi071.stdout:3/602: sync 2024-04-23T02:14:08.360 INFO:tasks.workunit.client.0.smithi071.stdout:6/657: sync 2024-04-23T02:14:08.360 INFO:tasks.workunit.client.0.smithi071.stdout:0/706: rename d0/d63/l65 to d0/d3/de/d10/lf7 0 2024-04-23T02:14:08.361 INFO:tasks.workunit.client.0.smithi071.stdout:8/728: mknod dc/d15/da0/d30/ce9 0 2024-04-23T02:14:08.362 INFO:tasks.workunit.client.0.smithi071.stdout:6/658: creat d6/d1f/d5d/d95/fd0 x:0 0 0 2024-04-23T02:14:08.363 INFO:tasks.workunit.client.0.smithi071.stdout:3/603: creat d7/d22/d23/d26/d7e/fd0 x:0 0 0 2024-04-23T02:14:08.364 INFO:tasks.workunit.client.0.smithi071.stdout:7/820: dread d3/d17/d41/d37/f6f [0,4194304] 0 2024-04-23T02:14:08.364 INFO:tasks.workunit.client.0.smithi071.stdout:1/601: mkdir db/df/dc2 0 2024-04-23T02:14:08.365 INFO:tasks.workunit.client.0.smithi071.stdout:5/780: dread d3/da/fcf [0,4194304] 0 2024-04-23T02:14:08.366 INFO:tasks.workunit.client.0.smithi071.stdout:8/729: mkdir dc/d56/de1/dea 0 2024-04-23T02:14:08.367 INFO:tasks.workunit.client.0.smithi071.stdout:5/781: stat d3/d1d/d43/d4b/cd9 0 2024-04-23T02:14:08.368 INFO:tasks.workunit.client.0.smithi071.stdout:7/821: unlink d3/d17/d41/d37/lb2 0 2024-04-23T02:14:08.368 INFO:tasks.workunit.client.0.smithi071.stdout:0/707: dread d0/d3/de/d10/f1f [0,4194304] 0 2024-04-23T02:14:08.369 INFO:tasks.workunit.client.0.smithi071.stdout:8/730: truncate f6 1147402 0 2024-04-23T02:14:08.369 INFO:tasks.workunit.client.0.smithi071.stdout:3/604: fdatasync d7/d22/d23/d26/f78 0 2024-04-23T02:14:08.370 INFO:tasks.workunit.client.0.smithi071.stdout:0/708: stat d0/d63/f66 0 2024-04-23T02:14:08.373 INFO:tasks.workunit.client.0.smithi071.stdout:5/782: creat d3/d1d/daf/f123 x:0 0 0 2024-04-23T02:14:08.374 INFO:tasks.workunit.client.0.smithi071.stdout:8/731: read dc/d78/f8f [2699532,129460] 0 2024-04-23T02:14:08.374 INFO:tasks.workunit.client.0.smithi071.stdout:7/822: rename d3/dae/f101 to d3/dae/f11e 0 2024-04-23T02:14:08.376 INFO:tasks.workunit.client.0.smithi071.stdout:3/605: mknod d7/d22/d23/d26/cd1 0 2024-04-23T02:14:08.376 INFO:tasks.workunit.client.0.smithi071.stdout:5/783: read d3/d34/f6a [1874636,93371] 0 2024-04-23T02:14:08.376 INFO:tasks.workunit.client.0.smithi071.stdout:7/823: readlink d3/d17/d1c/d31/l65 0 2024-04-23T02:14:08.377 INFO:tasks.workunit.client.1.smithi192.stdout:5/762: dwrite d2/db/f11 [0,4194304] 0 2024-04-23T02:14:08.380 INFO:tasks.workunit.client.1.smithi192.stdout:7/803: dwrite d5/d43/d4c/d4e/fc6 [0,4194304] 0 2024-04-23T02:14:08.381 INFO:tasks.workunit.client.0.smithi071.stdout:3/606: creat d7/d22/da7/fd2 x:0 0 0 2024-04-23T02:14:08.381 INFO:tasks.workunit.client.0.smithi071.stdout:1/602: dread db/d34/f7d [0,4194304] 0 2024-04-23T02:14:08.386 INFO:tasks.workunit.client.1.smithi192.stdout:5/763: rename d2/d6/d2d/d41/d43/d98/lda to d2/d6/d2d/d41/d43/lfd 0 2024-04-23T02:14:08.386 INFO:tasks.workunit.client.0.smithi071.stdout:1/603: rmdir db/df 39 2024-04-23T02:14:08.387 INFO:tasks.workunit.client.1.smithi192.stdout:5/764: truncate d2/db/d16/fce 979785 0 2024-04-23T02:14:08.390 INFO:tasks.workunit.client.1.smithi192.stdout:7/804: rename d5/d43/d4c/d5c/d67 to d5/d17/d1c/d59/d5a/d70/de9 0 2024-04-23T02:14:08.391 INFO:tasks.workunit.client.1.smithi192.stdout:7/805: stat d5/d17/d1c/d59/d5a/d70/de9/da6/da9/db9/dc0 0 2024-04-23T02:14:08.392 INFO:tasks.workunit.client.1.smithi192.stdout:5/765: link d2/db/d16/fce d2/db/de1/ffe 0 2024-04-23T02:14:08.392 INFO:tasks.workunit.client.1.smithi192.stdout:7/806: write d5/fcb [486838,90000] 0 2024-04-23T02:14:08.395 INFO:tasks.workunit.client.1.smithi192.stdout:5/766: dread d2/db/d2b/d52/dc2/d49/ff1 [0,4194304] 0 2024-04-23T02:14:08.396 INFO:tasks.workunit.client.1.smithi192.stdout:5/767: dread - d2/db/d16/fe5 zero size 2024-04-23T02:14:08.398 INFO:tasks.workunit.client.1.smithi192.stdout:5/768: chown d2/d6/d2d/d41/d43/f9d 250673493 1 2024-04-23T02:14:08.400 INFO:tasks.workunit.client.1.smithi192.stdout:7/807: dwrite d5/d17/d1c/d59/d5a/d70/de9/da6/da9/fc3 [0,4194304] 0 2024-04-23T02:14:08.400 INFO:tasks.workunit.client.1.smithi192.stdout:5/769: truncate d2/fd 172848 0 2024-04-23T02:14:08.402 INFO:tasks.workunit.client.1.smithi192.stdout:5/770: chown d2/d6/f3b 7072055 1 2024-04-23T02:14:08.403 INFO:tasks.workunit.client.1.smithi192.stdout:7/808: unlink d5/l71 0 2024-04-23T02:14:08.416 INFO:tasks.workunit.client.1.smithi192.stdout:0/899: sync 2024-04-23T02:14:08.418 INFO:tasks.workunit.client.1.smithi192.stdout:7/809: truncate d5/d17/d1c/d59/d5a/d41/f54 2272147 0 2024-04-23T02:14:08.421 INFO:tasks.workunit.client.1.smithi192.stdout:0/900: mknod d6/df/d56/d85/d45/c129 0 2024-04-23T02:14:08.422 INFO:tasks.workunit.client.1.smithi192.stdout:7/810: creat d5/d43/d4c/d5c/d97/fea x:0 0 0 2024-04-23T02:14:08.423 INFO:tasks.workunit.client.1.smithi192.stdout:5/771: dwrite d2/d2e/df0/f67 [0,4194304] 0 2024-04-23T02:14:08.425 INFO:tasks.workunit.client.1.smithi192.stdout:0/901: creat d6/df/d13/d25/d3a/d93/f12a x:0 0 0 2024-04-23T02:14:08.426 INFO:tasks.workunit.client.1.smithi192.stdout:0/902: chown d6/df/d35/d8b/d113 23 1 2024-04-23T02:14:08.427 INFO:tasks.workunit.client.0.smithi071.stdout:3/607: rename d7/da5/db2/db7/d68/d3b/d9f to d7/d22/d23/d26/d52/dd3 0 2024-04-23T02:14:08.427 INFO:tasks.workunit.client.1.smithi192.stdout:5/772: creat d2/d2e/d4e/de6/fff x:0 0 0 2024-04-23T02:14:08.429 INFO:tasks.workunit.client.0.smithi071.stdout:7/824: creat d3/d17/d1c/d31/d112/f11f x:0 0 0 2024-04-23T02:14:08.431 INFO:tasks.workunit.client.1.smithi192.stdout:5/773: dread - d2/db/d2b/d52/f8f zero size 2024-04-23T02:14:08.433 INFO:tasks.workunit.client.0.smithi071.stdout:3/608: creat d7/d8/d42/d80/dca/fd4 x:0 0 0 2024-04-23T02:14:08.435 INFO:tasks.workunit.client.0.smithi071.stdout:7/825: chown d3/lbd 1023 1 2024-04-23T02:14:08.436 INFO:tasks.workunit.client.0.smithi071.stdout:1/604: truncate db/df/f2a 1969900 0 2024-04-23T02:14:08.437 INFO:tasks.workunit.client.0.smithi071.stdout:3/609: mknod d7/da5/cd5 0 2024-04-23T02:14:08.438 INFO:tasks.workunit.client.1.smithi192.stdout:0/903: creat d6/df/d35/d8b/d8d/f12b x:0 0 0 2024-04-23T02:14:08.438 INFO:tasks.workunit.client.1.smithi192.stdout:5/774: chown d2/f95 10012184 1 2024-04-23T02:14:08.440 INFO:tasks.workunit.client.0.smithi071.stdout:1/605: mknod db/d34/cc3 0 2024-04-23T02:14:08.440 INFO:tasks.workunit.client.1.smithi192.stdout:5/775: chown d2/cf3 1047869332 1 2024-04-23T02:14:08.440 INFO:tasks.workunit.client.1.smithi192.stdout:7/811: dwrite d5/d43/f99 [0,4194304] 0 2024-04-23T02:14:08.441 INFO:tasks.workunit.client.0.smithi071.stdout:0/709: write d0/d3/de/d10/d14/d32/d44/d87/fa0 [147021,91950] 0 2024-04-23T02:14:08.442 INFO:tasks.workunit.client.0.smithi071.stdout:7/826: mknod d3/d51/d87/c120 0 2024-04-23T02:14:08.444 INFO:tasks.workunit.client.0.smithi071.stdout:1/606: creat db/df/d13/d20/d75/fc4 x:0 0 0 2024-04-23T02:14:08.444 INFO:tasks.workunit.client.1.smithi192.stdout:5/776: dread d2/d32/fc9 [0,4194304] 0 2024-04-23T02:14:08.445 INFO:tasks.workunit.client.0.smithi071.stdout:7/827: creat d3/d17/dca/d117/f121 x:0 0 0 2024-04-23T02:14:08.446 INFO:tasks.workunit.client.0.smithi071.stdout:5/784: write d3/d34/f40 [1514318,77295] 0 2024-04-23T02:14:08.453 INFO:tasks.workunit.client.1.smithi192.stdout:0/904: chown d6/df/f14 100077 1 2024-04-23T02:14:08.454 INFO:tasks.workunit.client.0.smithi071.stdout:7/828: write d3/fb5 [167647,164] 0 2024-04-23T02:14:08.456 INFO:tasks.workunit.client.0.smithi071.stdout:8/732: dwrite dc/f9f [0,4194304] 0 2024-04-23T02:14:08.458 INFO:tasks.workunit.client.1.smithi192.stdout:0/905: rename d6/df/d13/d25/d26 to d6/df/d13/d25/d94/dd0/d12c 0 2024-04-23T02:14:08.458 INFO:tasks.workunit.client.1.smithi192.stdout:7/812: link d5/d17/d1c/d57/cc8 d5/d17/d1c/d59/d5a/d70/de9/da6/ddf/ceb 0 2024-04-23T02:14:08.458 INFO:tasks.workunit.client.0.smithi071.stdout:0/710: rename d0/d63/f74 to d0/d3/de/d10/d14/d32/d37/ff8 0 2024-04-23T02:14:08.458 INFO:tasks.workunit.client.0.smithi071.stdout:8/733: readlink dc/d15/da0/l4e 0 2024-04-23T02:14:08.462 INFO:tasks.workunit.client.1.smithi192.stdout:7/813: truncate d5/d17/f92 389885 0 2024-04-23T02:14:08.464 INFO:tasks.workunit.client.0.smithi071.stdout:7/829: creat d3/d17/d1c/d2a/d73/f122 x:0 0 0 2024-04-23T02:14:08.466 INFO:tasks.workunit.client.0.smithi071.stdout:1/607: write db/df/d3a/f66 [999416,25345] 0 2024-04-23T02:14:08.466 INFO:tasks.workunit.client.0.smithi071.stdout:3/610: dwrite f0 [0,4194304] 0 2024-04-23T02:14:08.469 INFO:tasks.workunit.client.0.smithi071.stdout:5/785: getdents d3/d1d/d43/d112 0 2024-04-23T02:14:08.469 INFO:tasks.workunit.client.0.smithi071.stdout:3/611: write d7/d22/d23/d26/f78 [3743455,75080] 0 2024-04-23T02:14:08.470 INFO:tasks.workunit.client.1.smithi192.stdout:0/906: creat d6/df/d35/d8b/d8d/dea/d128/f12d x:0 0 0 2024-04-23T02:14:08.473 INFO:tasks.workunit.client.0.smithi071.stdout:7/830: rename d3/d17/d1c/d31/f48 to d3/d17/d1c/d2a/d73/d83/de7/f123 0 2024-04-23T02:14:08.474 INFO:tasks.workunit.client.1.smithi192.stdout:0/907: truncate d6/df/d13/d25/d3a/d7b/f88 5156310 0 2024-04-23T02:14:08.477 INFO:tasks.workunit.client.0.smithi071.stdout:7/831: dread d3/d17/d41/d34/d39/d3b/f53 [0,4194304] 0 2024-04-23T02:14:08.477 INFO:tasks.workunit.client.0.smithi071.stdout:0/711: link d0/d3/d95/fb4 d0/d3/de/d10/d14/d32/d44/d87/d93/dcc/ff9 0 2024-04-23T02:14:08.477 INFO:tasks.workunit.client.0.smithi071.stdout:3/612: creat d7/d1d/fd6 x:0 0 0 2024-04-23T02:14:08.481 INFO:tasks.workunit.client.0.smithi071.stdout:7/832: truncate d3/d17/d1c/d2a/d43/d86/f88 383846 0 2024-04-23T02:14:08.481 INFO:tasks.workunit.client.0.smithi071.stdout:5/786: rename d3/da/d1a/ded to d3/d86/d124 0 2024-04-23T02:14:08.482 INFO:tasks.workunit.client.1.smithi192.stdout:0/908: rmdir d6/df/d13/d25/d94/dd0/d12c/d67 39 2024-04-23T02:14:08.482 INFO:tasks.workunit.client.0.smithi071.stdout:8/734: write dc/d15/da0/f27 [1704332,65279] 0 2024-04-23T02:14:08.485 INFO:tasks.workunit.client.0.smithi071.stdout:7/833: symlink d3/d17/d41/d118/l124 0 2024-04-23T02:14:08.485 INFO:tasks.workunit.client.1.smithi192.stdout:5/777: dread d2/db/d2b/d52/dc2/f81 [0,4194304] 0 2024-04-23T02:14:08.485 INFO:tasks.workunit.client.1.smithi192.stdout:5/778: chown d2/d32/fc9 1 1 2024-04-23T02:14:08.486 INFO:tasks.workunit.client.0.smithi071.stdout:1/608: dread db/df/d5f/f82 [0,4194304] 0 2024-04-23T02:14:08.487 INFO:tasks.workunit.client.1.smithi192.stdout:0/909: dread d6/df/d13/d25/d86/da5/faf [0,4194304] 0 2024-04-23T02:14:08.489 INFO:tasks.workunit.client.1.smithi192.stdout:5/779: unlink d2/db/f11 0 2024-04-23T02:14:08.489 INFO:tasks.workunit.client.1.smithi192.stdout:0/910: read - d6/df/d35/d49/fd2 zero size 2024-04-23T02:14:08.489 INFO:tasks.workunit.client.0.smithi071.stdout:3/613: link d7/c79 d7/d22/d23/dba/cd7 0 2024-04-23T02:14:08.489 INFO:tasks.workunit.client.0.smithi071.stdout:1/609: read db/f22 [135802,102640] 0 2024-04-23T02:14:08.490 INFO:tasks.workunit.client.1.smithi192.stdout:5/780: write d2/f5 [8829826,5921] 0 2024-04-23T02:14:08.490 INFO:tasks.workunit.client.0.smithi071.stdout:8/735: creat dc/d15/da0/feb x:0 0 0 2024-04-23T02:14:08.493 INFO:tasks.workunit.client.1.smithi192.stdout:0/911: creat d6/df/d13/d25/d94/dd0/f12e x:0 0 0 2024-04-23T02:14:08.496 INFO:tasks.workunit.client.0.smithi071.stdout:8/736: rename dc/d15/d2d/f49 to dc/d15/d50/fec 0 2024-04-23T02:14:08.496 INFO:tasks.workunit.client.0.smithi071.stdout:5/787: dwrite d3/d1d/d43/d4b/fe0 [0,4194304] 0 2024-04-23T02:14:08.496 INFO:tasks.workunit.client.0.smithi071.stdout:7/834: link d3/d17/d41/d34/f35 d3/d17/d1c/d31/d10c/f125 0 2024-04-23T02:14:08.499 INFO:tasks.workunit.client.1.smithi192.stdout:5/781: getdents d2/db/d2b/dc8 0 2024-04-23T02:14:08.508 INFO:tasks.workunit.client.1.smithi192.stdout:5/782: getdents d2/db/d16 0 2024-04-23T02:14:08.508 INFO:tasks.workunit.client.1.smithi192.stdout:7/814: dwrite d5/d17/d1c/d57/f33 [0,4194304] 0 2024-04-23T02:14:08.508 INFO:tasks.workunit.client.1.smithi192.stdout:0/912: dwrite d6/df/d35/d8b/f11a [4194304,4194304] 0 2024-04-23T02:14:08.509 INFO:tasks.workunit.client.0.smithi071.stdout:3/614: dwrite d7/d10/f2f [0,4194304] 0 2024-04-23T02:14:08.509 INFO:tasks.workunit.client.0.smithi071.stdout:8/737: creat dc/d15/da0/d32/d63/dd4/fed x:0 0 0 2024-04-23T02:14:08.509 INFO:tasks.workunit.client.0.smithi071.stdout:8/738: readlink dc/le2 0 2024-04-23T02:14:08.509 INFO:tasks.workunit.client.0.smithi071.stdout:1/610: dwrite db/d34/d8a/f8f [0,4194304] 0 2024-04-23T02:14:08.509 INFO:tasks.workunit.client.0.smithi071.stdout:7/835: rename d3/lbb to d3/d17/d1c/d31/l126 0 2024-04-23T02:14:08.512 INFO:tasks.workunit.client.0.smithi071.stdout:1/611: rename db/df/d5f/d6f/db2 to db/df/d5f/d6f/db2/d86/dc5 22 2024-04-23T02:14:08.513 INFO:tasks.workunit.client.1.smithi192.stdout:7/815: dread d5/d43/d4c/d5c/f72 [0,4194304] 0 2024-04-23T02:14:08.514 INFO:tasks.workunit.client.0.smithi071.stdout:3/615: symlink d7/da5/db2/db7/d68/d3b/d4c/d74/ld8 0 2024-04-23T02:14:08.518 INFO:tasks.workunit.client.0.smithi071.stdout:8/739: mknod dc/d15/da0/d32/d63/cee 0 2024-04-23T02:14:08.518 INFO:tasks.workunit.client.1.smithi192.stdout:5/783: dwrite d2/d6/f8a [4194304,4194304] 0 2024-04-23T02:14:08.521 INFO:tasks.workunit.client.1.smithi192.stdout:7/816: creat d5/d17/d1c/fec x:0 0 0 2024-04-23T02:14:08.521 INFO:tasks.workunit.client.1.smithi192.stdout:5/784: chown d2/db/d2b/d52/dc2/d49/ff1 623 1 2024-04-23T02:14:08.523 INFO:tasks.workunit.client.0.smithi071.stdout:1/612: dread db/df/d13/d2b/f78 [0,4194304] 0 2024-04-23T02:14:08.524 INFO:tasks.workunit.client.0.smithi071.stdout:3/616: rename d7/da5/db2/db7/f5e to d7/d22/d23/d26/d7e/fd9 0 2024-04-23T02:14:08.526 INFO:tasks.workunit.client.0.smithi071.stdout:8/740: mkdir dc/d15/da0/d8d/def 0 2024-04-23T02:14:08.527 INFO:tasks.workunit.client.0.smithi071.stdout:5/788: link d3/d34/f8b d3/d8f/de6/f125 0 2024-04-23T02:14:08.528 INFO:tasks.workunit.client.1.smithi192.stdout:7/817: write d5/d17/d1c/d57/f90 [1073918,98461] 0 2024-04-23T02:14:08.529 INFO:tasks.workunit.client.1.smithi192.stdout:5/785: symlink d2/db/d16/d48/d4f/l100 0 2024-04-23T02:14:08.529 INFO:tasks.workunit.client.1.smithi192.stdout:2/921: sync 2024-04-23T02:14:08.529 INFO:tasks.workunit.client.1.smithi192.stdout:3/742: sync 2024-04-23T02:14:08.530 INFO:tasks.workunit.client.0.smithi071.stdout:8/741: write dc/d3a/da4/db0/fd2 [237294,43854] 0 2024-04-23T02:14:08.530 INFO:tasks.workunit.client.0.smithi071.stdout:5/789: dread - d3/d1d/daf/d10/da5/f94 zero size 2024-04-23T02:14:08.533 INFO:tasks.workunit.client.1.smithi192.stdout:2/922: stat d8/d26/d29/d43/d50/d5e/f5f 0 2024-04-23T02:14:08.535 INFO:tasks.workunit.client.0.smithi071.stdout:1/613: symlink db/df/d13/d2f/lc6 0 2024-04-23T02:14:08.537 INFO:tasks.workunit.client.1.smithi192.stdout:7/818: link d5/f15 d5/d43/d4c/d5c/de7/fed 0 2024-04-23T02:14:08.538 INFO:tasks.workunit.client.1.smithi192.stdout:2/923: symlink d8/d26/d27/d31/dae/l12d 0 2024-04-23T02:14:08.538 INFO:tasks.workunit.client.0.smithi071.stdout:5/790: creat d3/da/def/dff/f126 x:0 0 0 2024-04-23T02:14:08.540 INFO:tasks.workunit.client.1.smithi192.stdout:7/819: write d5/d17/d1c/d59/d5a/d70/f88 [799415,103642] 0 2024-04-23T02:14:08.540 INFO:tasks.workunit.client.0.smithi071.stdout:8/742: creat dc/d15/da0/d32/d63/ff0 x:0 0 0 2024-04-23T02:14:08.541 INFO:tasks.workunit.client.1.smithi192.stdout:2/924: creat d8/d26/d29/d80/f12e x:0 0 0 2024-04-23T02:14:08.541 INFO:tasks.workunit.client.0.smithi071.stdout:3/617: creat d7/d10/fda x:0 0 0 2024-04-23T02:14:08.542 INFO:tasks.workunit.client.1.smithi192.stdout:2/925: fdatasync d8/d26/d29/d43/d50/fe9 0 2024-04-23T02:14:08.542 INFO:tasks.workunit.client.0.smithi071.stdout:1/614: rename db/df/d13/d2f/d48/lc1 to db/df/d13/d20/d75/lc7 0 2024-04-23T02:14:08.543 INFO:tasks.workunit.client.1.smithi192.stdout:7/820: fdatasync d5/d17/d1c/d26/f86 0 2024-04-23T02:14:08.544 INFO:tasks.workunit.client.0.smithi071.stdout:5/791: mkdir d3/d1d/d43/dd2/d5a/da7/d127 0 2024-04-23T02:14:08.545 INFO:tasks.workunit.client.0.smithi071.stdout:8/743: mknod dc/d15/da0/d30/cf1 0 2024-04-23T02:14:08.546 INFO:tasks.workunit.client.1.smithi192.stdout:2/926: unlink d8/f10e 0 2024-04-23T02:14:08.546 INFO:tasks.workunit.client.0.smithi071.stdout:7/836: mkdir d3/d51/df5/d10d/d127 0 2024-04-23T02:14:08.547 INFO:tasks.workunit.client.0.smithi071.stdout:5/792: dread - d3/d8f/fc6 zero size 2024-04-23T02:14:08.549 INFO:tasks.workunit.client.1.smithi192.stdout:2/927: fdatasync d8/d25/f3f 0 2024-04-23T02:14:08.550 INFO:tasks.workunit.client.1.smithi192.stdout:7/821: truncate d5/d17/f92 718960 0 2024-04-23T02:14:08.550 INFO:tasks.workunit.client.0.smithi071.stdout:7/837: creat d3/d51/df5/f128 x:0 0 0 2024-04-23T02:14:08.551 INFO:tasks.workunit.client.0.smithi071.stdout:1/615: getdents db/d11/d7f/da7 0 2024-04-23T02:14:08.553 INFO:tasks.workunit.client.1.smithi192.stdout:2/928: symlink d8/d26/d27/d57/d69/d99/de6/l12f 0 2024-04-23T02:14:08.553 INFO:tasks.workunit.client.0.smithi071.stdout:1/616: stat db/d34/f7d 0 2024-04-23T02:14:08.554 INFO:tasks.workunit.client.0.smithi071.stdout:1/617: chown db/df/d13/d2f/d48/lb3 27625770 1 2024-04-23T02:14:08.556 INFO:tasks.workunit.client.1.smithi192.stdout:2/929: creat d8/d26/d29/da4/dbc/f130 x:0 0 0 2024-04-23T02:14:08.558 INFO:tasks.workunit.client.0.smithi071.stdout:1/618: symlink db/df/d5f/d6f/db2/d86/lc8 0 2024-04-23T02:14:08.560 INFO:tasks.workunit.client.0.smithi071.stdout:3/618: dread d7/d10/d38/f41 [0,4194304] 0 2024-04-23T02:14:08.562 INFO:tasks.workunit.client.1.smithi192.stdout:3/743: dread d1/d3/d6/d7/d5f/fcd [0,4194304] 0 2024-04-23T02:14:08.563 INFO:tasks.workunit.client.0.smithi071.stdout:1/619: rename db/df/d13/d2f/d48/f6e to db/d34/fc9 0 2024-04-23T02:14:08.563 INFO:tasks.workunit.client.0.smithi071.stdout:1/620: chown db/la1 15718 1 2024-04-23T02:14:08.565 INFO:tasks.workunit.client.0.smithi071.stdout:5/793: getdents d3/da/def/dff 0 2024-04-23T02:14:08.566 INFO:tasks.workunit.client.0.smithi071.stdout:3/619: creat d7/d86/fdb x:0 0 0 2024-04-23T02:14:08.567 INFO:tasks.workunit.client.1.smithi192.stdout:5/786: truncate d2/db/d16/d1b/f90 2558251 0 2024-04-23T02:14:08.568 INFO:tasks.workunit.client.0.smithi071.stdout:1/621: symlink db/df/d13/d20/d75/lca 0 2024-04-23T02:14:08.572 INFO:tasks.workunit.client.1.smithi192.stdout:5/787: rename d2/db/d2b/d52/dc2/d49/f91 to d2/d32/f101 0 2024-04-23T02:14:08.572 INFO:tasks.workunit.client.0.smithi071.stdout:3/620: mknod d7/d10/cdc 0 2024-04-23T02:14:08.573 INFO:tasks.workunit.client.0.smithi071.stdout:1/622: mknod db/df/d13/ccb 0 2024-04-23T02:14:08.574 INFO:tasks.workunit.client.0.smithi071.stdout:7/838: dread d3/dae/db7/dea/f9d [0,4194304] 0 2024-04-23T02:14:08.575 INFO:tasks.workunit.client.1.smithi192.stdout:2/930: dread d8/d26/d29/da4/dca/fd4 [0,4194304] 0 2024-04-23T02:14:08.576 INFO:tasks.workunit.client.1.smithi192.stdout:7/822: dread d5/d17/d1c/d59/d5a/d70/de9/f81 [0,4194304] 0 2024-04-23T02:14:08.577 INFO:tasks.workunit.client.1.smithi192.stdout:2/931: chown d8/d26/d74 0 1 2024-04-23T02:14:08.579 INFO:tasks.workunit.client.0.smithi071.stdout:5/794: rmdir d3/d1d/d43/d4b/d118 0 2024-04-23T02:14:08.580 INFO:tasks.workunit.client.0.smithi071.stdout:3/621: mknod d7/da5/db2/cdd 0 2024-04-23T02:14:08.580 INFO:tasks.workunit.client.1.smithi192.stdout:5/788: dwrite d2/db/d2b/f7f [0,4194304] 0 2024-04-23T02:14:08.580 INFO:tasks.workunit.client.1.smithi192.stdout:2/932: mknod d8/d26/d29/dcd/c131 0 2024-04-23T02:14:08.581 INFO:tasks.workunit.client.0.smithi071.stdout:1/623: mknod db/d34/ccc 0 2024-04-23T02:14:08.582 INFO:tasks.workunit.client.0.smithi071.stdout:5/795: creat d3/d1d/d43/dd2/d5a/da7/f128 x:0 0 0 2024-04-23T02:14:08.583 INFO:tasks.workunit.client.1.smithi192.stdout:7/823: dread d5/d43/d4c/d5c/d97/fe3 [0,4194304] 0 2024-04-23T02:14:08.584 INFO:tasks.workunit.client.0.smithi071.stdout:1/624: rmdir db/d92 39 2024-04-23T02:14:08.584 INFO:tasks.workunit.client.1.smithi192.stdout:5/789: symlink d2/d6/db4/l102 0 2024-04-23T02:14:08.585 INFO:tasks.workunit.client.0.smithi071.stdout:7/839: symlink d3/d17/d1c/d2a/d73/dba/l129 0 2024-04-23T02:14:08.586 INFO:tasks.workunit.client.1.smithi192.stdout:2/933: fdatasync d8/d14/faa 0 2024-04-23T02:14:08.587 INFO:tasks.workunit.client.0.smithi071.stdout:1/625: fsync db/f22 0 2024-04-23T02:14:08.588 INFO:tasks.workunit.client.1.smithi192.stdout:2/934: dread - d8/d26/d29/da4/dbc/f130 zero size 2024-04-23T02:14:08.589 INFO:tasks.workunit.client.0.smithi071.stdout:5/796: creat d3/da/f129 x:0 0 0 2024-04-23T02:14:08.590 INFO:tasks.workunit.client.1.smithi192.stdout:2/935: rmdir d8/d26/d27/d57/d69 39 2024-04-23T02:14:08.591 INFO:tasks.workunit.client.0.smithi071.stdout:7/840: mkdir d3/d12a 0 2024-04-23T02:14:08.592 INFO:tasks.workunit.client.0.smithi071.stdout:3/622: dwrite d7/da5/db5/fb6 [4194304,4194304] 0 2024-04-23T02:14:08.593 INFO:tasks.workunit.client.0.smithi071.stdout:3/623: chown d7/d10/d38/l69 54874166 1 2024-04-23T02:14:08.594 INFO:tasks.workunit.client.1.smithi192.stdout:2/936: symlink d8/l132 0 2024-04-23T02:14:08.595 INFO:tasks.workunit.client.0.smithi071.stdout:2/626: sync 2024-04-23T02:14:08.597 INFO:tasks.workunit.client.1.smithi192.stdout:7/824: dread d5/d17/d1c/f3c [0,4194304] 0 2024-04-23T02:14:08.598 INFO:tasks.workunit.client.0.smithi071.stdout:2/627: write d0/d2/f1d [1806502,84376] 0 2024-04-23T02:14:08.599 INFO:tasks.workunit.client.1.smithi192.stdout:7/825: read d5/d17/d1c/d59/d5a/d70/de9/f81 [2489941,4957] 0 2024-04-23T02:14:08.599 INFO:tasks.workunit.client.0.smithi071.stdout:3/624: dread d7/d8/f4e [0,4194304] 0 2024-04-23T02:14:08.602 INFO:tasks.workunit.client.1.smithi192.stdout:5/790: dread d2/d6/f12 [0,4194304] 0 2024-04-23T02:14:08.603 INFO:tasks.workunit.client.1.smithi192.stdout:7/826: write d5/d17/d1c/d59/d5a/d70/de9/da6/da9/db9/dc0/fdb [4527765,3917] 0 2024-04-23T02:14:08.605 INFO:tasks.workunit.client.1.smithi192.stdout:2/937: truncate d8/d25/f6c 7148270 0 2024-04-23T02:14:08.606 INFO:tasks.workunit.client.0.smithi071.stdout:5/797: getdents d3/d1d/d43/dd2 0 2024-04-23T02:14:08.606 INFO:tasks.workunit.client.1.smithi192.stdout:7/827: mkdir d5/d17/d1c/d57/d75/dee 0 2024-04-23T02:14:08.607 INFO:tasks.workunit.client.0.smithi071.stdout:7/841: dwrite d3/dae/db7/dea/d5f/f60 [0,4194304] 0 2024-04-23T02:14:08.608 INFO:tasks.workunit.client.1.smithi192.stdout:7/828: stat d5/d43/f6e 0 2024-04-23T02:14:08.608 INFO:tasks.workunit.client.0.smithi071.stdout:2/628: mkdir d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd 0 2024-04-23T02:14:08.610 INFO:tasks.workunit.client.1.smithi192.stdout:2/938: creat d8/d26/d29/f133 x:0 0 0 2024-04-23T02:14:08.610 INFO:tasks.workunit.client.1.smithi192.stdout:5/791: dwrite d2/fe [0,4194304] 0 2024-04-23T02:14:08.612 INFO:tasks.workunit.client.0.smithi071.stdout:3/625: dwrite d7/da5/db2/db7/d68/f84 [0,4194304] 0 2024-04-23T02:14:08.612 INFO:tasks.workunit.client.0.smithi071.stdout:5/798: fsync d3/d1d/daf/d10/da5/f94 0 2024-04-23T02:14:08.613 INFO:tasks.workunit.client.1.smithi192.stdout:2/939: dread - d8/d26/d27/d53/da2/dc1/f11f zero size 2024-04-23T02:14:08.613 INFO:tasks.workunit.client.1.smithi192.stdout:2/940: stat d8/c62 0 2024-04-23T02:14:08.617 INFO:tasks.workunit.client.1.smithi192.stdout:7/829: mkdir d5/d17/d1c/d59/d5a/d70/de9/da6/da9/db9/dc0/def 0 2024-04-23T02:14:08.617 INFO:tasks.workunit.client.1.smithi192.stdout:5/792: truncate d2/d2e/df0/f65 5257 0 2024-04-23T02:14:08.617 INFO:tasks.workunit.client.0.smithi071.stdout:3/626: chown d7/da5/db2/db7/cbf 1371541 1 2024-04-23T02:14:08.617 INFO:tasks.workunit.client.0.smithi071.stdout:1/626: dwrite db/df/d5f/d57/f5b [0,4194304] 0 2024-04-23T02:14:08.618 INFO:tasks.workunit.client.0.smithi071.stdout:5/799: fsync d3/d1d/daf/d10/da5/d5f/fd6 0 2024-04-23T02:14:08.619 INFO:tasks.workunit.client.1.smithi192.stdout:7/830: fsync d5/d17/d1c/d59/d5a/d9b/fda 0 2024-04-23T02:14:08.620 INFO:tasks.workunit.client.0.smithi071.stdout:3/627: write d7/d22/fc1 [986109,121160] 0 2024-04-23T02:14:08.621 INFO:tasks.workunit.client.1.smithi192.stdout:2/941: truncate d8/d26/d29/da4/dab/f105 1352401 0 2024-04-23T02:14:08.622 INFO:tasks.workunit.client.1.smithi192.stdout:5/793: mkdir d2/db/d2b/dc8/d103 0 2024-04-23T02:14:08.622 INFO:tasks.workunit.client.0.smithi071.stdout:5/800: truncate d3/da/f129 79145 0 2024-04-23T02:14:08.623 INFO:tasks.workunit.client.1.smithi192.stdout:7/831: dread d5/d17/d1c/d57/f48 [0,4194304] 0 2024-04-23T02:14:08.624 INFO:tasks.workunit.client.0.smithi071.stdout:3/628: chown d7/da5/db2/db7/c9c 55 1 2024-04-23T02:14:08.625 INFO:tasks.workunit.client.0.smithi071.stdout:7/842: dwrite d3/d17/fb9 [0,4194304] 0 2024-04-23T02:14:08.625 INFO:tasks.workunit.client.1.smithi192.stdout:7/832: write d5/d17/d1c/f3e [4049356,32267] 0 2024-04-23T02:14:08.626 INFO:tasks.workunit.client.1.smithi192.stdout:5/794: creat d2/d2e/df0/f104 x:0 0 0 2024-04-23T02:14:08.626 INFO:tasks.workunit.client.1.smithi192.stdout:2/942: creat d8/d26/d125/f134 x:0 0 0 2024-04-23T02:14:08.629 INFO:tasks.workunit.client.0.smithi071.stdout:1/627: write db/df/d5f/d6f/db2/f79 [1107574,88729] 0 2024-04-23T02:14:08.631 INFO:tasks.workunit.client.0.smithi071.stdout:7/843: chown d3/f12 2084981 1 2024-04-23T02:14:08.632 INFO:tasks.workunit.client.0.smithi071.stdout:3/629: dread d7/d8/d42/f5d [0,4194304] 0 2024-04-23T02:14:08.634 INFO:tasks.workunit.client.1.smithi192.stdout:7/833: symlink d5/d43/d4c/d5c/de7/lf0 0 2024-04-23T02:14:08.636 INFO:tasks.workunit.client.0.smithi071.stdout:3/630: mknod d7/da5/db2/db7/d68/d3b/d4c/d74/cde 0 2024-04-23T02:14:08.638 INFO:tasks.workunit.client.0.smithi071.stdout:3/631: fdatasync d7/d22/d23/d26/f78 0 2024-04-23T02:14:08.638 INFO:tasks.workunit.client.1.smithi192.stdout:7/834: creat d5/d43/d4c/d5c/ff1 x:0 0 0 2024-04-23T02:14:08.639 INFO:tasks.workunit.client.1.smithi192.stdout:2/943: dwrite d8/d26/d29/d43/d50/fe8 [0,4194304] 0 2024-04-23T02:14:08.640 INFO:tasks.workunit.client.0.smithi071.stdout:7/844: rename d3/fb5 to d3/d17/d41/d3a/f12b 0 2024-04-23T02:14:08.640 INFO:tasks.workunit.client.1.smithi192.stdout:2/944: readlink d8/d26/d29/da4/da8/lac 0 2024-04-23T02:14:08.642 INFO:tasks.workunit.client.1.smithi192.stdout:7/835: mknod d5/d17/d1c/cf2 0 2024-04-23T02:14:08.643 INFO:tasks.workunit.client.1.smithi192.stdout:2/945: readlink d8/d26/d29/d43/d50/ld2 0 2024-04-23T02:14:08.644 INFO:tasks.workunit.client.1.smithi192.stdout:7/836: write d5/d43/d4c/d5c/f72 [967240,41662] 0 2024-04-23T02:14:08.648 INFO:tasks.workunit.client.0.smithi071.stdout:3/632: dwrite d7/d10/f15 [0,4194304] 0 2024-04-23T02:14:08.648 INFO:tasks.workunit.client.1.smithi192.stdout:7/837: symlink d5/d17/d1c/d59/d5a/d70/de9/da6/lf3 0 2024-04-23T02:14:08.650 INFO:tasks.workunit.client.1.smithi192.stdout:7/838: mkdir d5/d17/d1c/d57/dbb/df4 0 2024-04-23T02:14:08.651 INFO:tasks.workunit.client.1.smithi192.stdout:7/839: chown d5/d17/d1c/f3e 1234768582 1 2024-04-23T02:14:08.653 INFO:tasks.workunit.client.0.smithi071.stdout:3/633: creat d7/da5/db2/db7/d68/fdf x:0 0 0 2024-04-23T02:14:08.654 INFO:tasks.workunit.client.0.smithi071.stdout:3/634: creat d7/da5/fe0 x:0 0 0 2024-04-23T02:14:08.654 INFO:tasks.workunit.client.0.smithi071.stdout:3/635: dread - d7/fc9 zero size 2024-04-23T02:14:08.655 INFO:tasks.workunit.client.0.smithi071.stdout:5/801: dread d3/d1d/d43/dd2/f78 [0,4194304] 0 2024-04-23T02:14:08.657 INFO:tasks.workunit.client.0.smithi071.stdout:3/636: unlink d7/d10/d38/l5c 0 2024-04-23T02:14:08.658 INFO:tasks.workunit.client.0.smithi071.stdout:3/637: readlink d7/d10/lb0 0 2024-04-23T02:14:08.658 INFO:tasks.workunit.client.1.smithi192.stdout:7/840: dwrite d5/d43/f6e [0,4194304] 0 2024-04-23T02:14:08.659 INFO:tasks.workunit.client.0.smithi071.stdout:3/638: chown d7/da5/db2/db7/d68/d3b/fb8 29802688 1 2024-04-23T02:14:08.659 INFO:tasks.workunit.client.0.smithi071.stdout:3/639: dread - d7/da5/fe0 zero size 2024-04-23T02:14:08.660 INFO:tasks.workunit.client.1.smithi192.stdout:7/841: readlink d5/d17/d1c/d59/d5a/da0/lb2 0 2024-04-23T02:14:08.662 INFO:tasks.workunit.client.0.smithi071.stdout:3/640: unlink d7/da5/db2/db7/d68/d3b/d4c/f60 0 2024-04-23T02:14:08.664 INFO:tasks.workunit.client.1.smithi192.stdout:7/842: dread d5/d43/f99 [0,4194304] 0 2024-04-23T02:14:08.665 INFO:tasks.workunit.client.0.smithi071.stdout:3/641: unlink d7/d8/d42/d80/f89 0 2024-04-23T02:14:08.665 INFO:tasks.workunit.client.0.smithi071.stdout:3/642: fdatasync d7/d22/d23/f6b 0 2024-04-23T02:14:08.666 INFO:tasks.workunit.client.1.smithi192.stdout:7/843: write d5/d17/d1c/d59/da8/fb0 [40147,116692] 0 2024-04-23T02:14:08.669 INFO:tasks.workunit.client.0.smithi071.stdout:3/643: rename d7/d1d/f73 to d7/d8/d42/d80/dca/fe1 0 2024-04-23T02:14:08.670 INFO:tasks.workunit.client.0.smithi071.stdout:3/644: write d7/da5/db2/db7/f40 [5955471,99419] 0 2024-04-23T02:14:08.671 INFO:tasks.workunit.client.0.smithi071.stdout:1/628: write db/df/d13/d2f/d48/d85/f98 [2764163,23783] 0 2024-04-23T02:14:08.673 INFO:tasks.workunit.client.1.smithi192.stdout:7/844: rename d5/d43/d4c/d5c/de7/fed to d5/d17/d1c/d59/d5a/d70/de9/da6/da9/ff5 0 2024-04-23T02:14:08.673 INFO:tasks.workunit.client.0.smithi071.stdout:7/845: write d3/d17/d1c/d31/fd8 [2453507,99254] 0 2024-04-23T02:14:08.675 INFO:tasks.workunit.client.1.smithi192.stdout:2/946: dread d8/d26/d29/da4/dbc/fc8 [0,4194304] 0 2024-04-23T02:14:08.676 INFO:tasks.workunit.client.0.smithi071.stdout:3/645: rename d7/d86/fdb to d7/da5/db2/db7/d68/d3b/d4c/da1/fe2 0 2024-04-23T02:14:08.677 INFO:tasks.workunit.client.1.smithi192.stdout:7/845: symlink d5/d43/d4c/d5c/de7/lf6 0 2024-04-23T02:14:08.679 INFO:tasks.workunit.client.1.smithi192.stdout:5/795: write d2/db/d16/d1b/fc6 [546066,17838] 0 2024-04-23T02:14:08.680 INFO:tasks.workunit.client.0.smithi071.stdout:1/629: dwrite db/df/d3a/f66 [0,4194304] 0 2024-04-23T02:14:08.682 INFO:tasks.workunit.client.0.smithi071.stdout:7/846: dread d3/d17/d1c/d2a/d73/fc6 [0,4194304] 0 2024-04-23T02:14:08.683 INFO:tasks.workunit.client.1.smithi192.stdout:7/846: creat d5/d17/d1c/d59/d5a/d41/d50/d3d/ff7 x:0 0 0 2024-04-23T02:14:08.683 INFO:tasks.workunit.client.0.smithi071.stdout:3/646: symlink d7/da5/db2/db7/d68/d3b/d4c/d8b/db3/le3 0 2024-04-23T02:14:08.683 INFO:tasks.workunit.client.0.smithi071.stdout:7/847: stat d3/d17/f1f 0 2024-04-23T02:14:08.686 INFO:tasks.workunit.client.1.smithi192.stdout:5/796: write d2/db/d2b/d52/f76 [398178,44863] 0 2024-04-23T02:14:08.688 INFO:tasks.workunit.client.1.smithi192.stdout:2/947: dwrite d8/d26/d27/db5/dd0/fd8 [0,4194304] 0 2024-04-23T02:14:08.694 INFO:tasks.workunit.client.0.smithi071.stdout:3/647: link d7/da5/db2/db7/d68/d3b/d4c/da1/fe2 d7/da5/db2/db7/d68/dae/fe4 0 2024-04-23T02:14:08.695 INFO:tasks.workunit.client.1.smithi192.stdout:2/948: symlink d8/d26/d29/d43/l135 0 2024-04-23T02:14:08.697 INFO:tasks.workunit.client.0.smithi071.stdout:1/630: dread db/df/d13/d2b/f70 [0,4194304] 0 2024-04-23T02:14:08.698 INFO:tasks.workunit.client.1.smithi192.stdout:5/797: dread d2/db8/fea [0,4194304] 0 2024-04-23T02:14:08.699 INFO:tasks.workunit.client.0.smithi071.stdout:3/648: creat d7/d22/fe5 x:0 0 0 2024-04-23T02:14:08.699 INFO:tasks.workunit.client.0.smithi071.stdout:7/848: dwrite d3/d51/d80/de9/f106 [0,4194304] 0 2024-04-23T02:14:08.702 INFO:tasks.workunit.client.0.smithi071.stdout:1/631: rename db/d34/l9b to db/d92/lcd 0 2024-04-23T02:14:08.702 INFO:tasks.workunit.client.1.smithi192.stdout:2/949: dwrite d8/d26/d29/d43/d50/fe8 [4194304,4194304] 0 2024-04-23T02:14:08.703 INFO:tasks.workunit.client.1.smithi192.stdout:5/798: dread d2/db/d16/d48/d4f/f5c [0,4194304] 0 2024-04-23T02:14:08.703 INFO:tasks.workunit.client.0.smithi071.stdout:3/649: unlink d7/da5/db2/db7/d68/l9a 0 2024-04-23T02:14:08.704 INFO:tasks.workunit.client.0.smithi071.stdout:1/632: dread - db/d34/f8c zero size 2024-04-23T02:14:08.705 INFO:tasks.workunit.client.0.smithi071.stdout:7/849: mkdir d3/d51/d87/d12c 0 2024-04-23T02:14:08.711 INFO:tasks.workunit.client.0.smithi071.stdout:3/650: truncate d7/d10/d38/f41 3497481 0 2024-04-23T02:14:08.711 INFO:tasks.workunit.client.0.smithi071.stdout:7/850: mknod d3/d51/df5/d10d/c12d 0 2024-04-23T02:14:08.711 INFO:tasks.workunit.client.0.smithi071.stdout:3/651: dread - d7/da5/db2/db7/d68/d3b/d4c/da1/fe2 zero size 2024-04-23T02:14:08.711 INFO:tasks.workunit.client.0.smithi071.stdout:7/851: creat d3/d51/df5/d10d/f12e x:0 0 0 2024-04-23T02:14:08.713 INFO:tasks.workunit.client.1.smithi192.stdout:2/950: rename d8/d26/d29/f85 to d8/d26/d29/d43/d50/f136 0 2024-04-23T02:14:08.713 INFO:tasks.workunit.client.1.smithi192.stdout:2/951: write d8/f36 [7069069,118661] 0 2024-04-23T02:14:08.713 INFO:tasks.workunit.client.1.smithi192.stdout:5/799: rename d2/d2e/d4e/l53 to d2/db/d2b/d52/dc2/l105 0 2024-04-23T02:14:08.714 INFO:tasks.workunit.client.0.smithi071.stdout:3/652: unlink d7/d8/d42/d80/d88/l6a 0 2024-04-23T02:14:08.714 INFO:tasks.workunit.client.0.smithi071.stdout:3/653: chown d7/d22/d23/d26/fa0 61231 1 2024-04-23T02:14:08.715 INFO:tasks.workunit.client.1.smithi192.stdout:2/952: symlink d8/d26/d29/d43/dc2/dec/l137 0 2024-04-23T02:14:08.715 INFO:tasks.workunit.client.1.smithi192.stdout:5/800: mkdir d2/d32/d106 0 2024-04-23T02:14:08.715 INFO:tasks.workunit.client.1.smithi192.stdout:2/953: fsync d8/d26/d29/d43/f9f 0 2024-04-23T02:14:08.717 INFO:tasks.workunit.client.0.smithi071.stdout:3/654: creat d7/da5/db2/fe6 x:0 0 0 2024-04-23T02:14:08.717 INFO:tasks.workunit.client.1.smithi192.stdout:2/954: chown d8/d26/d74/d7c/lb7 5 1 2024-04-23T02:14:08.717 INFO:tasks.workunit.client.0.smithi071.stdout:3/655: write d7/d22/da7/f9b [2452336,91216] 0 2024-04-23T02:14:08.724 INFO:tasks.workunit.client.0.smithi071.stdout:3/656: read d7/da5/db2/db7/d68/d3b/d4c/f59 [2632606,129130] 0 2024-04-23T02:14:08.725 INFO:tasks.workunit.client.1.smithi192.stdout:2/955: dwrite d8/d26/d125/f134 [0,4194304] 0 2024-04-23T02:14:08.725 INFO:tasks.workunit.client.0.smithi071.stdout:3/657: truncate d7/da5/db2/db7/f8c 659417 0 2024-04-23T02:14:08.727 INFO:tasks.workunit.client.1.smithi192.stdout:2/956: read d8/d26/d74/d7c/f9b [395939,2403] 0 2024-04-23T02:14:08.727 INFO:tasks.workunit.client.1.smithi192.stdout:2/957: chown d8/d26/d29/d80/cdc 1 1 2024-04-23T02:14:08.729 INFO:tasks.workunit.client.0.smithi071.stdout:3/658: fsync d7/d8/fe 0 2024-04-23T02:14:08.731 INFO:tasks.workunit.client.0.smithi071.stdout:1/633: write db/df/d3a/f47 [573395,64044] 0 2024-04-23T02:14:08.732 INFO:tasks.workunit.client.0.smithi071.stdout:1/634: stat db/df/d5f/d57/f89 0 2024-04-23T02:14:08.735 INFO:tasks.workunit.client.1.smithi192.stdout:2/958: dwrite d8/d26/d29/d43/d5d/d119/f128 [0,4194304] 0 2024-04-23T02:14:08.735 INFO:tasks.workunit.client.0.smithi071.stdout:3/659: link l3 d7/da5/db2/db7/d68/d3b/d4c/da1/le7 0 2024-04-23T02:14:08.739 INFO:tasks.workunit.client.0.smithi071.stdout:7/852: truncate d3/d17/d1c/d31/fc2 3302636 0 2024-04-23T02:14:08.742 INFO:tasks.workunit.client.0.smithi071.stdout:7/853: mknod d3/d17/d41/d34/d39/d3b/da7/c12f 0 2024-04-23T02:14:08.744 INFO:tasks.workunit.client.0.smithi071.stdout:3/660: rename d7/da5/db2/db7/d68/d3b/d4c/d8b/db3/c6d to d7/d8/d42/d80/dce/ce8 0 2024-04-23T02:14:08.748 INFO:tasks.workunit.client.0.smithi071.stdout:7/854: dwrite d3/d17/fb9 [0,4194304] 0 2024-04-23T02:14:08.751 INFO:tasks.workunit.client.0.smithi071.stdout:7/855: symlink d3/d17/dca/l130 0 2024-04-23T02:14:08.752 INFO:tasks.workunit.client.0.smithi071.stdout:7/856: truncate d3/d17/d41/d3a/f12b 1617540 0 2024-04-23T02:14:08.754 INFO:tasks.workunit.client.0.smithi071.stdout:7/857: mknod d3/d51/df5/d10d/c131 0 2024-04-23T02:14:08.755 INFO:tasks.workunit.client.0.smithi071.stdout:7/858: mknod d3/d51/d87/d12c/c132 0 2024-04-23T02:14:08.758 INFO:tasks.workunit.client.0.smithi071.stdout:7/859: creat d3/d51/df5/d10d/d127/f133 x:0 0 0 2024-04-23T02:14:08.760 INFO:tasks.workunit.client.0.smithi071.stdout:3/661: dread d7/d22/da7/f9b [0,4194304] 0 2024-04-23T02:14:08.762 INFO:tasks.workunit.client.0.smithi071.stdout:3/662: truncate d7/d8/d42/d80/dca/fd4 887679 0 2024-04-23T02:14:08.764 INFO:tasks.workunit.client.0.smithi071.stdout:3/663: mknod d7/d22/ce9 0 2024-04-23T02:14:08.764 INFO:tasks.workunit.client.0.smithi071.stdout:3/664: chown d7/da5/db2/db7/f40 0 1 2024-04-23T02:14:08.765 INFO:tasks.workunit.client.0.smithi071.stdout:3/665: fdatasync d7/d22/d23/f6b 0 2024-04-23T02:14:08.766 INFO:tasks.workunit.client.0.smithi071.stdout:3/666: mkdir d7/d8/dea 0 2024-04-23T02:14:08.773 INFO:tasks.workunit.client.1.smithi192.stdout:9/654: sync 2024-04-23T02:14:08.773 INFO:tasks.workunit.client.1.smithi192.stdout:1/586: sync 2024-04-23T02:14:08.776 INFO:tasks.workunit.client.1.smithi192.stdout:9/655: write d0/d9/d44/f45 [971216,79882] 0 2024-04-23T02:14:08.777 INFO:tasks.workunit.client.0.smithi071.stdout:1/635: truncate db/df/d13/d2f/f3e 1353384 0 2024-04-23T02:14:08.778 INFO:tasks.workunit.client.1.smithi192.stdout:1/587: mkdir dc/d16/d19/d30/dbf 0 2024-04-23T02:14:08.779 INFO:tasks.workunit.client.0.smithi071.stdout:1/636: creat db/d34/d68/fce x:0 0 0 2024-04-23T02:14:08.780 INFO:tasks.workunit.client.1.smithi192.stdout:1/588: dread - dc/d80/fa8 zero size 2024-04-23T02:14:08.780 INFO:tasks.workunit.client.0.smithi071.stdout:1/637: chown db/df/d13/d2f/lc6 26110144 1 2024-04-23T02:14:08.780 INFO:tasks.workunit.client.1.smithi192.stdout:9/656: rmdir d0 39 2024-04-23T02:14:08.782 INFO:tasks.workunit.client.1.smithi192.stdout:1/589: fsync dc/d14/d23/f7d 0 2024-04-23T02:14:08.783 INFO:tasks.workunit.client.0.smithi071.stdout:1/638: creat db/d11/d7f/fcf x:0 0 0 2024-04-23T02:14:08.783 INFO:tasks.workunit.client.0.smithi071.stdout:7/860: write d3/d17/f36 [1066503,104498] 0 2024-04-23T02:14:08.784 INFO:tasks.workunit.client.1.smithi192.stdout:1/590: read - dc/d16/d19/d74/faf zero size 2024-04-23T02:14:08.784 INFO:tasks.workunit.client.0.smithi071.stdout:3/667: write d7/d22/d23/d26/fa0 [297898,17443] 0 2024-04-23T02:14:08.788 INFO:tasks.workunit.client.0.smithi071.stdout:3/668: mknod d7/d22/ceb 0 2024-04-23T02:14:08.791 INFO:tasks.workunit.client.0.smithi071.stdout:3/669: mkdir d7/da5/db5/dec 0 2024-04-23T02:14:08.792 INFO:tasks.workunit.client.0.smithi071.stdout:3/670: rmdir d7/d1d 39 2024-04-23T02:14:08.797 INFO:tasks.workunit.client.0.smithi071.stdout:3/671: symlink d7/d1d/led 0 2024-04-23T02:14:08.798 INFO:tasks.workunit.client.1.smithi192.stdout:2/959: dread d8/fc3 [0,4194304] 0 2024-04-23T02:14:08.801 INFO:tasks.workunit.client.1.smithi192.stdout:2/960: write d8/d26/d27/d31/dae/d11a/f12a [309326,25753] 0 2024-04-23T02:14:08.801 INFO:tasks.workunit.client.0.smithi071.stdout:3/672: mknod d7/da5/db2/db7/d68/d3b/d4c/cee 0 2024-04-23T02:14:08.803 INFO:tasks.workunit.client.1.smithi192.stdout:1/591: dread dc/d14/f21 [0,4194304] 0 2024-04-23T02:14:08.804 INFO:tasks.workunit.client.1.smithi192.stdout:2/961: symlink d8/d26/d29/d43/dc2/dec/l138 0 2024-04-23T02:14:08.806 INFO:tasks.workunit.client.1.smithi192.stdout:1/592: fdatasync dc/d14/d15/d35/f49 0 2024-04-23T02:14:08.808 INFO:tasks.workunit.client.1.smithi192.stdout:2/962: symlink d8/d26/d27/d53/da2/dc1/l139 0 2024-04-23T02:14:08.810 INFO:tasks.workunit.client.0.smithi071.stdout:7/861: dread d3/dae/db7/dea/d5f/f90 [0,4194304] 0 2024-04-23T02:14:08.813 INFO:tasks.workunit.client.1.smithi192.stdout:7/847: write d5/d17/d1c/d59/d5a/d70/f7e [5154005,28059] 0 2024-04-23T02:14:08.814 INFO:tasks.workunit.client.0.smithi071.stdout:7/862: rmdir d3/d17/d1c 39 2024-04-23T02:14:08.817 INFO:tasks.workunit.client.0.smithi071.stdout:7/863: rename d3/f1d to d3/d17/d41/d34/d39/f134 0 2024-04-23T02:14:08.819 INFO:tasks.workunit.client.1.smithi192.stdout:9/657: write d0/d9/f3a [654831,86884] 0 2024-04-23T02:14:08.821 INFO:tasks.workunit.client.1.smithi192.stdout:9/658: fsync d0/d9/d44/f9d 0 2024-04-23T02:14:08.822 INFO:tasks.workunit.client.1.smithi192.stdout:5/801: dwrite d2/db/d16/f8b [0,4194304] 0 2024-04-23T02:14:08.824 INFO:tasks.workunit.client.0.smithi071.stdout:7/864: dwrite d3/d17/d1c/f8e [0,4194304] 0 2024-04-23T02:14:08.825 INFO:tasks.workunit.client.1.smithi192.stdout:5/802: stat d2/db/d16/ca7 0 2024-04-23T02:14:08.826 INFO:tasks.workunit.client.1.smithi192.stdout:9/659: mkdir d0/d9/d44/d58/dd9/dda 0 2024-04-23T02:14:08.833 INFO:tasks.workunit.client.1.smithi192.stdout:5/803: dread d2/db/d16/fce [0,4194304] 0 2024-04-23T02:14:08.834 INFO:tasks.workunit.client.1.smithi192.stdout:2/963: dwrite d8/d26/d29/d43/d50/d5e/f5f [0,4194304] 0 2024-04-23T02:14:08.834 INFO:tasks.workunit.client.0.smithi071.stdout:7/865: dwrite d3/d17/d1c/d2a/f104 [0,4194304] 0 2024-04-23T02:14:08.836 INFO:tasks.workunit.client.1.smithi192.stdout:9/660: dread - d0/d9/d44/d51/fc4 zero size 2024-04-23T02:14:08.837 INFO:tasks.workunit.client.1.smithi192.stdout:1/593: dwrite dc/d14/f21 [4194304,4194304] 0 2024-04-23T02:14:08.841 INFO:tasks.workunit.client.1.smithi192.stdout:9/661: write d0/d9/d44/d51/d5b/d61/db1/fd4 [671765,7717] 0 2024-04-23T02:14:08.845 INFO:tasks.workunit.client.1.smithi192.stdout:2/964: symlink d8/l13a 0 2024-04-23T02:14:08.851 INFO:tasks.workunit.client.1.smithi192.stdout:1/594: creat dc/d14/d23/da2/fc0 x:0 0 0 2024-04-23T02:14:08.851 INFO:tasks.workunit.client.1.smithi192.stdout:5/804: dwrite d2/d32/f101 [0,4194304] 0 2024-04-23T02:14:08.857 INFO:tasks.workunit.client.1.smithi192.stdout:5/805: rename d2/db/d2b/f82 to d2/d6/d2d/d41/d43/d98/f107 0 2024-04-23T02:14:08.858 INFO:tasks.workunit.client.1.smithi192.stdout:5/806: fdatasync d2/db/f9a 0 2024-04-23T02:14:08.859 INFO:tasks.workunit.client.1.smithi192.stdout:5/807: fsync d2/db/d2b/f93 0 2024-04-23T02:14:08.861 INFO:tasks.workunit.client.1.smithi192.stdout:5/808: mkdir d2/d6/d14/d108 0 2024-04-23T02:14:08.863 INFO:tasks.workunit.client.1.smithi192.stdout:5/809: symlink d2/d6/d2d/d41/l109 0 2024-04-23T02:14:08.864 INFO:tasks.workunit.client.1.smithi192.stdout:5/810: write d2/db/d2b/df7/ffb [864497,98606] 0 2024-04-23T02:14:08.871 INFO:tasks.workunit.client.1.smithi192.stdout:5/811: dwrite d2/db/d2b/f3a [0,4194304] 0 2024-04-23T02:14:08.874 INFO:tasks.workunit.client.1.smithi192.stdout:5/812: chown d2/db/d2b/d52/l62 64 1 2024-04-23T02:14:08.875 INFO:tasks.workunit.client.1.smithi192.stdout:5/813: chown d2/d32/f44 215325 1 2024-04-23T02:14:08.876 INFO:tasks.workunit.client.1.smithi192.stdout:5/814: dread - d2/d6/d2d/d41/d43/fdc zero size 2024-04-23T02:14:08.896 INFO:tasks.workunit.client.0.smithi071.stdout:4/777: sync 2024-04-23T02:14:08.896 INFO:tasks.workunit.client.0.smithi071.stdout:9/671: sync 2024-04-23T02:14:08.897 INFO:tasks.workunit.client.0.smithi071.stdout:6/659: sync 2024-04-23T02:14:08.899 INFO:tasks.workunit.client.0.smithi071.stdout:9/672: write d1/d32/d3a/d77/f88 [90099,98141] 0 2024-04-23T02:14:08.901 INFO:tasks.workunit.client.0.smithi071.stdout:6/660: rmdir d6/d17/d77/d9f/dad/d3f 39 2024-04-23T02:14:08.901 INFO:tasks.workunit.client.1.smithi192.stdout:9/662: write d0/d9/d44/d51/fc4 [328751,125715] 0 2024-04-23T02:14:08.902 INFO:tasks.workunit.client.1.smithi192.stdout:1/595: rmdir dc/d14/d23/da2 39 2024-04-23T02:14:08.902 INFO:tasks.workunit.client.0.smithi071.stdout:9/673: creat d1/d32/d41/ff0 x:0 0 0 2024-04-23T02:14:08.903 INFO:tasks.workunit.client.0.smithi071.stdout:9/674: fsync d1/d32/d41/fd2 0 2024-04-23T02:14:08.904 INFO:tasks.workunit.client.0.smithi071.stdout:7/866: truncate d3/d51/d87/fab 2967895 0 2024-04-23T02:14:08.904 INFO:tasks.workunit.client.1.smithi192.stdout:1/596: dread - dc/d14/d15/fbc zero size 2024-04-23T02:14:08.905 INFO:tasks.workunit.client.0.smithi071.stdout:9/675: chown d1/d32/d7e/ld0 61 1 2024-04-23T02:14:08.905 INFO:tasks.workunit.client.1.smithi192.stdout:2/965: write d8/d26/d29/d43/d5d/dbd/ffd [684111,105062] 0 2024-04-23T02:14:08.905 INFO:tasks.workunit.client.0.smithi071.stdout:6/661: chown d6/d17/d77/d9f/dad/d3f/l66 300 1 2024-04-23T02:14:08.907 INFO:tasks.workunit.client.0.smithi071.stdout:4/778: link d2/d2a/da2/dd1/dde/d57/f72 d2/d2a/da2/dd1/dde/d23/d10/ffd 0 2024-04-23T02:14:08.907 INFO:tasks.workunit.client.1.smithi192.stdout:9/663: read d0/d9/d44/d51/d5b/d61/f84 [272941,40574] 0 2024-04-23T02:14:08.908 INFO:tasks.workunit.client.0.smithi071.stdout:7/867: rmdir d3/d17/d41/d34/d6e 39 2024-04-23T02:14:08.909 INFO:tasks.workunit.client.0.smithi071.stdout:4/779: chown d2/d2a/da2/dd1/dde/l7a 27377252 1 2024-04-23T02:14:08.914 INFO:tasks.workunit.client.1.smithi192.stdout:1/597: mkdir dc/d8b/d34/d70/d8c/da3/dc1 0 2024-04-23T02:14:08.915 INFO:tasks.workunit.client.1.smithi192.stdout:9/664: creat d0/fdb x:0 0 0 2024-04-23T02:14:08.915 INFO:tasks.workunit.client.1.smithi192.stdout:1/598: write f1 [13595554,79430] 0 2024-04-23T02:14:08.917 INFO:tasks.workunit.client.0.smithi071.stdout:9/676: dwrite d1/d4e/d54/d75/fb0 [0,4194304] 0 2024-04-23T02:14:08.919 INFO:tasks.workunit.client.1.smithi192.stdout:5/815: dwrite d2/db/d16/fce [0,4194304] 0 2024-04-23T02:14:08.920 INFO:tasks.workunit.client.1.smithi192.stdout:1/599: mkdir dc/d16/d19/d30/dbf/dc2 0 2024-04-23T02:14:08.921 INFO:tasks.workunit.client.1.smithi192.stdout:1/600: dread - dc/d14/d15/fbc zero size 2024-04-23T02:14:08.923 INFO:tasks.workunit.client.1.smithi192.stdout:9/665: rename d0/d9/d44/d58/la2 to d0/d98/ldc 0 2024-04-23T02:14:08.924 INFO:tasks.workunit.client.1.smithi192.stdout:5/816: symlink d2/l10a 0 2024-04-23T02:14:08.926 INFO:tasks.workunit.client.0.smithi071.stdout:4/780: link d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/dc8/fe8 d2/d2a/d2d/d39/d42/d49/d67/d7e/dae/ffe 0 2024-04-23T02:14:08.927 INFO:tasks.workunit.client.1.smithi192.stdout:1/601: unlink dc/d14/d15/d35/l96 0 2024-04-23T02:14:08.928 INFO:tasks.workunit.client.0.smithi071.stdout:7/868: dwrite d3/d17/d41/d37/f75 [0,4194304] 0 2024-04-23T02:14:08.931 INFO:tasks.workunit.client.0.smithi071.stdout:9/677: creat d1/ff1 x:0 0 0 2024-04-23T02:14:08.932 INFO:tasks.workunit.client.1.smithi192.stdout:5/817: mknod d2/d6/d2d/d41/d43/c10b 0 2024-04-23T02:14:08.933 INFO:tasks.workunit.client.1.smithi192.stdout:5/818: readlink d2/d6/d2d/d41/l45 0 2024-04-23T02:14:08.935 INFO:tasks.workunit.client.0.smithi071.stdout:7/869: creat d3/d17/d41/d34/f135 x:0 0 0 2024-04-23T02:14:08.937 INFO:tasks.workunit.client.1.smithi192.stdout:5/819: symlink d2/d32/l10c 0 2024-04-23T02:14:08.937 INFO:tasks.workunit.client.1.smithi192.stdout:1/602: dwrite dc/d14/d23/d5e/dab/fb9 [0,4194304] 0 2024-04-23T02:14:08.939 INFO:tasks.workunit.client.0.smithi071.stdout:6/662: dread d6/d17/d77/faf [0,4194304] 0 2024-04-23T02:14:08.940 INFO:tasks.workunit.client.1.smithi192.stdout:5/820: stat d2/d2e/df0/f65 0 2024-04-23T02:14:08.941 INFO:tasks.workunit.client.0.smithi071.stdout:4/781: dwrite d2/d2a/d2d/d39/fd8 [0,4194304] 0 2024-04-23T02:14:08.943 INFO:tasks.workunit.client.0.smithi071.stdout:7/870: fsync d3/dae/db7/dea/f9d 0 2024-04-23T02:14:08.943 INFO:tasks.workunit.client.0.smithi071.stdout:9/678: getdents d1/d32/dc5/dd1 0 2024-04-23T02:14:08.945 INFO:tasks.workunit.client.1.smithi192.stdout:2/966: dread d8/d26/d29/da4/da8/dba/f10d [0,4194304] 0 2024-04-23T02:14:08.946 INFO:tasks.workunit.client.1.smithi192.stdout:2/967: rename d8/d26 to d8/d26/d27/d53/da2/d13b 22 2024-04-23T02:14:08.949 INFO:tasks.workunit.client.1.smithi192.stdout:2/968: mkdir d8/d26/d29/d43/dc2/dec/d118/d13c 0 2024-04-23T02:14:08.949 INFO:tasks.workunit.client.1.smithi192.stdout:2/969: dread - d8/f10a zero size 2024-04-23T02:14:08.950 INFO:tasks.workunit.client.0.smithi071.stdout:4/782: write d2/d2a/d2d/d39/d42/d49/d91/fc5 [840985,37076] 0 2024-04-23T02:14:08.953 INFO:tasks.workunit.client.1.smithi192.stdout:6/891: sync 2024-04-23T02:14:08.953 INFO:tasks.workunit.client.1.smithi192.stdout:4/887: sync 2024-04-23T02:14:08.954 INFO:tasks.workunit.client.1.smithi192.stdout:2/970: creat d8/d26/d29/d43/d5d/d119/f13d x:0 0 0 2024-04-23T02:14:08.954 INFO:tasks.workunit.client.0.smithi071.stdout:9/679: creat d1/d32/d41/db7/d39/d42/dcc/ff2 x:0 0 0 2024-04-23T02:14:08.956 INFO:tasks.workunit.client.0.smithi071.stdout:4/783: mknod d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/cff 0 2024-04-23T02:14:08.956 INFO:tasks.workunit.client.0.smithi071.stdout:6/663: dwrite d6/d17/d77/d9e/fc5 [0,4194304] 0 2024-04-23T02:14:08.957 INFO:tasks.workunit.client.0.smithi071.stdout:6/664: chown d6/d1f/d5d/l97 34 1 2024-04-23T02:14:08.957 INFO:tasks.workunit.client.1.smithi192.stdout:2/971: dread - d8/d26/d29/d43/f10b zero size 2024-04-23T02:14:08.959 INFO:tasks.workunit.client.0.smithi071.stdout:6/665: write d6/d17/d77/f96 [462702,119278] 0 2024-04-23T02:14:08.959 INFO:tasks.workunit.client.0.smithi071.stdout:4/784: creat d2/d2a/d2d/d39/d42/d49/d67/d7e/d82/f100 x:0 0 0 2024-04-23T02:14:08.962 INFO:tasks.workunit.client.1.smithi192.stdout:2/972: fdatasync d8/d26/d74/fd1 0 2024-04-23T02:14:08.963 INFO:tasks.workunit.client.0.smithi071.stdout:6/666: mkdir d6/d17/d77/d9f/dad/dd1 0 2024-04-23T02:14:08.964 INFO:tasks.workunit.client.1.smithi192.stdout:6/892: mknod d3/d28/df8/d3a/d51/ddc/c125 0 2024-04-23T02:14:08.964 INFO:tasks.workunit.client.1.smithi192.stdout:4/888: truncate de/d15/d19/d1a/f23 3115945 0 2024-04-23T02:14:08.965 INFO:tasks.workunit.client.0.smithi071.stdout:4/785: creat d2/d2a/d2d/d39/d42/f101 x:0 0 0 2024-04-23T02:14:08.966 INFO:tasks.workunit.client.1.smithi192.stdout:9/666: dwrite d0/d9/d44/d51/d5b/d61/f7b [0,4194304] 0 2024-04-23T02:14:08.966 INFO:tasks.workunit.client.1.smithi192.stdout:1/603: truncate dc/fd 7577049 0 2024-04-23T02:14:08.967 INFO:tasks.workunit.client.1.smithi192.stdout:2/973: mknod d8/d26/d29/d43/d50/c13e 0 2024-04-23T02:14:08.967 INFO:tasks.workunit.client.1.smithi192.stdout:6/893: readlink d3/d28/df8/d3a/dbd/lf9 0 2024-04-23T02:14:08.968 INFO:tasks.workunit.client.0.smithi071.stdout:6/667: rename d6/d17/d77/d85/l63 to d6/d17/d77/d9f/ld2 0 2024-04-23T02:14:08.971 INFO:tasks.workunit.client.1.smithi192.stdout:2/974: dread - d8/d26/d27/d57/da0/ff7 zero size 2024-04-23T02:14:08.971 INFO:tasks.workunit.client.1.smithi192.stdout:9/667: read - d0/d9/d44/d53/fcc zero size 2024-04-23T02:14:08.974 INFO:tasks.workunit.client.0.smithi071.stdout:6/668: mkdir d6/d17/d77/d9f/dad/dd1/dd3 0 2024-04-23T02:14:08.975 INFO:tasks.workunit.client.1.smithi192.stdout:4/889: chown de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/d59/c90 1825734 1 2024-04-23T02:14:08.976 INFO:tasks.workunit.client.0.smithi071.stdout:4/786: dwrite d2/d2a/d2d/f54 [0,4194304] 0 2024-04-23T02:14:08.976 INFO:tasks.workunit.client.1.smithi192.stdout:5/821: dwrite d2/db/d16/f96 [0,4194304] 0 2024-04-23T02:14:08.979 INFO:tasks.workunit.client.1.smithi192.stdout:2/975: unlink d8/d26/d29/da4/da8/lac 0 2024-04-23T02:14:08.979 INFO:tasks.workunit.client.1.smithi192.stdout:4/890: dread de/d15/d53/f83 [0,4194304] 0 2024-04-23T02:14:08.981 INFO:tasks.workunit.client.0.smithi071.stdout:4/787: truncate d2/d2a/d2d/d39/d42/d49/d67/d7e/dae/fbd 638839 0 2024-04-23T02:14:08.981 INFO:tasks.workunit.client.1.smithi192.stdout:4/891: readlink de/d15/l4e 0 2024-04-23T02:14:08.982 INFO:tasks.workunit.client.1.smithi192.stdout:9/668: symlink d0/d9/db3/d5f/dc9/ldd 0 2024-04-23T02:14:08.982 INFO:tasks.workunit.client.1.smithi192.stdout:2/976: creat d8/d26/d27/d31/f13f x:0 0 0 2024-04-23T02:14:08.983 INFO:tasks.workunit.client.0.smithi071.stdout:9/680: dread d1/f16 [0,4194304] 0 2024-04-23T02:14:08.983 INFO:tasks.workunit.client.1.smithi192.stdout:5/822: mkdir d2/db/d2b/d10d 0 2024-04-23T02:14:08.985 INFO:tasks.workunit.client.0.smithi071.stdout:7/871: dread d3/d17/d41/f2f [0,4194304] 0 2024-04-23T02:14:08.986 INFO:tasks.workunit.client.1.smithi192.stdout:4/892: creat de/d15/d53/d81/d10b/f12c x:0 0 0 2024-04-23T02:14:08.987 INFO:tasks.workunit.client.0.smithi071.stdout:4/788: write d2/d2a/da2/dd1/dde/d23/d29/fd3 [874704,67406] 0 2024-04-23T02:14:08.987 INFO:tasks.workunit.client.0.smithi071.stdout:9/681: creat d1/d8d/ff3 x:0 0 0 2024-04-23T02:14:08.987 INFO:tasks.workunit.client.1.smithi192.stdout:5/823: symlink d2/db/d16/d48/d4f/l10e 0 2024-04-23T02:14:08.989 INFO:tasks.workunit.client.1.smithi192.stdout:2/977: rename d8/d26/d29/d43/dc2/dec/d118/d13c to d8/d26/d29/d43/d113/d140 0 2024-04-23T02:14:08.993 INFO:tasks.workunit.client.1.smithi192.stdout:2/978: chown d8/d26/d27/d53/da2/f123 21 1 2024-04-23T02:14:08.993 INFO:tasks.workunit.client.1.smithi192.stdout:6/894: dread d3/d28/d79/da0/dc2/d101/d80/d8e/f8c [0,4194304] 0 2024-04-23T02:14:08.995 INFO:tasks.workunit.client.0.smithi071.stdout:9/682: creat d1/ff4 x:0 0 0 2024-04-23T02:14:08.995 INFO:tasks.workunit.client.0.smithi071.stdout:9/683: readlink d1/d32/d41/db7/ldd 0 2024-04-23T02:14:08.998 INFO:tasks.workunit.client.0.smithi071.stdout:6/669: write d6/f3c [1710763,76771] 0 2024-04-23T02:14:08.998 INFO:tasks.workunit.client.1.smithi192.stdout:4/893: mkdir de/d15/d53/d81/de3/d74/d92/da0/dca/d34/d12d 0 2024-04-23T02:14:08.999 INFO:tasks.workunit.client.1.smithi192.stdout:1/604: write dc/f76 [4317944,658] 0 2024-04-23T02:14:09.000 INFO:tasks.workunit.client.0.smithi071.stdout:9/684: creat d1/d32/d3a/d94/ff5 x:0 0 0 2024-04-23T02:14:09.000 INFO:tasks.workunit.client.0.smithi071.stdout:7/872: getdents d3/dae 0 2024-04-23T02:14:09.001 INFO:tasks.workunit.client.1.smithi192.stdout:6/895: rmdir d3/d18/dd4/dff 39 2024-04-23T02:14:09.002 INFO:tasks.workunit.client.0.smithi071.stdout:7/873: write d3/d17/d1c/d31/fd8 [1166408,78933] 0 2024-04-23T02:14:09.003 INFO:tasks.workunit.client.0.smithi071.stdout:6/670: dread d6/d17/d77/d9f/dad/f8d [0,4194304] 0 2024-04-23T02:14:09.004 INFO:tasks.workunit.client.0.smithi071.stdout:9/685: fdatasync d1/d32/d41/db7/f2d 0 2024-04-23T02:14:09.005 INFO:tasks.workunit.client.1.smithi192.stdout:1/605: write dc/d8b/f4d [5211663,80521] 0 2024-04-23T02:14:09.006 INFO:tasks.workunit.client.1.smithi192.stdout:5/824: rename d2/d6/d14/d60/cd0 to d2/d6/d2d/d41/d43/c10f 0 2024-04-23T02:14:09.006 INFO:tasks.workunit.client.0.smithi071.stdout:7/874: write d3/d17/d41/d34/d39/f134 [4378380,102945] 0 2024-04-23T02:14:09.009 INFO:tasks.workunit.client.1.smithi192.stdout:9/669: dread d0/d9/f26 [0,4194304] 0 2024-04-23T02:14:09.009 INFO:tasks.workunit.client.1.smithi192.stdout:2/979: getdents d8/d26/d27/d31/dae 0 2024-04-23T02:14:09.010 INFO:tasks.workunit.client.0.smithi071.stdout:6/671: dread d6/d17/d77/faf [0,4194304] 0 2024-04-23T02:14:09.011 INFO:tasks.workunit.client.0.smithi071.stdout:7/875: symlink d3/d51/df5/d10d/l136 0 2024-04-23T02:14:09.012 INFO:tasks.workunit.client.1.smithi192.stdout:6/896: link d3/d15/d5b/fb8 d3/d28/d79/da0/dc2/f126 0 2024-04-23T02:14:09.013 INFO:tasks.workunit.client.0.smithi071.stdout:6/672: rmdir d6/d17/d77/d9f/db5 39 2024-04-23T02:14:09.014 INFO:tasks.workunit.client.1.smithi192.stdout:1/606: symlink dc/d14/d23/d5e/d6a/lc3 0 2024-04-23T02:14:09.016 INFO:tasks.workunit.client.1.smithi192.stdout:9/670: stat d0/d9/d44/d51/d5b/d61/db1/fcf 0 2024-04-23T02:14:09.016 INFO:tasks.workunit.client.1.smithi192.stdout:1/607: chown dc/d14/c43 2616456 1 2024-04-23T02:14:09.017 INFO:tasks.workunit.client.1.smithi192.stdout:4/894: truncate de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/d59/f63 4718698 0 2024-04-23T02:14:09.018 INFO:tasks.workunit.client.0.smithi071.stdout:9/686: write d1/d32/d41/db7/f22 [4708880,130937] 0 2024-04-23T02:14:09.019 INFO:tasks.workunit.client.0.smithi071.stdout:6/673: mknod d6/d17/d77/d9f/dad/dd1/dd3/cd4 0 2024-04-23T02:14:09.019 INFO:tasks.workunit.client.1.smithi192.stdout:9/671: fdatasync d0/d9/d44/d4f/fb2 0 2024-04-23T02:14:09.020 INFO:tasks.workunit.client.1.smithi192.stdout:1/608: chown dc/d8b/f33 7569274 1 2024-04-23T02:14:09.020 INFO:tasks.workunit.client.1.smithi192.stdout:5/825: dread d2/db/d16/fce [0,4194304] 0 2024-04-23T02:14:09.020 INFO:tasks.workunit.client.0.smithi071.stdout:7/876: getdents d3/d17/d41/d37 0 2024-04-23T02:14:09.020 INFO:tasks.workunit.client.0.smithi071.stdout:4/789: dread d2/d2a/da2/dd1/dde/d23/f16 [0,4194304] 0 2024-04-23T02:14:09.021 INFO:tasks.workunit.client.1.smithi192.stdout:2/980: rename d8/d26/d27/d53/da2/dc1/ff0 to d8/d26/d29/d43/d50/f141 0 2024-04-23T02:14:09.021 INFO:tasks.workunit.client.0.smithi071.stdout:7/877: dread - d3/d51/df5/d10d/d127/f133 zero size 2024-04-23T02:14:09.022 INFO:tasks.workunit.client.1.smithi192.stdout:6/897: mknod d3/d15/d1b/c127 0 2024-04-23T02:14:09.022 INFO:tasks.workunit.client.0.smithi071.stdout:6/674: getdents d6/d22/dbf 0 2024-04-23T02:14:09.023 INFO:tasks.workunit.client.1.smithi192.stdout:6/898: readlink d3/lc 0 2024-04-23T02:14:09.024 INFO:tasks.workunit.client.0.smithi071.stdout:4/790: getdents d2 0 2024-04-23T02:14:09.024 INFO:tasks.workunit.client.0.smithi071.stdout:6/675: chown d6/d17/c1b 13058626 1 2024-04-23T02:14:09.024 INFO:tasks.workunit.client.0.smithi071.stdout:2/629: sync 2024-04-23T02:14:09.024 INFO:tasks.workunit.client.0.smithi071.stdout:0/712: sync 2024-04-23T02:14:09.024 INFO:tasks.workunit.client.0.smithi071.stdout:8/744: sync 2024-04-23T02:14:09.025 INFO:tasks.workunit.client.0.smithi071.stdout:7/878: write d3/d17/d41/d3a/f12b [1749099,21137] 0 2024-04-23T02:14:09.025 INFO:tasks.workunit.client.0.smithi071.stdout:2/630: stat d0/d70 0 2024-04-23T02:14:09.027 INFO:tasks.workunit.client.0.smithi071.stdout:8/745: unlink dc/d15/da0/d55/c9c 0 2024-04-23T02:14:09.028 INFO:tasks.workunit.client.1.smithi192.stdout:5/826: mkdir d2/db/d2b/d52/dc2/d49/d110 0 2024-04-23T02:14:09.030 INFO:tasks.workunit.client.1.smithi192.stdout:6/899: mkdir d3/d28/d79/da0/dc2/d101/d80/d8e/d128 0 2024-04-23T02:14:09.031 INFO:tasks.workunit.client.1.smithi192.stdout:4/895: truncate de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/fdb 702242 0 2024-04-23T02:14:09.032 INFO:tasks.workunit.client.0.smithi071.stdout:6/676: dread d6/d1f/d5d/f73 [0,4194304] 0 2024-04-23T02:14:09.033 INFO:tasks.workunit.client.1.smithi192.stdout:4/896: stat de/d15/d53/d81/de3/d74/d92/da0/dca/f5f 0 2024-04-23T02:14:09.034 INFO:tasks.workunit.client.0.smithi071.stdout:7/879: rmdir d3/d17/d41/d34/d39 39 2024-04-23T02:14:09.035 INFO:tasks.workunit.client.0.smithi071.stdout:2/631: dwrite d0/d70/f86 [0,4194304] 0 2024-04-23T02:14:09.036 INFO:tasks.workunit.client.1.smithi192.stdout:2/981: creat d8/d26/d27/d57/d69/d99/de6/f142 x:0 0 0 2024-04-23T02:14:09.037 INFO:tasks.workunit.client.0.smithi071.stdout:2/632: write d0/d2/d26/d3a/dc3/f95 [134575,48150] 0 2024-04-23T02:14:09.037 INFO:tasks.workunit.client.1.smithi192.stdout:6/900: dwrite d3/d28/df8/d52/f67 [0,4194304] 0 2024-04-23T02:14:09.039 INFO:tasks.workunit.client.0.smithi071.stdout:6/677: unlink d6/d1f/c27 0 2024-04-23T02:14:09.040 INFO:tasks.workunit.client.0.smithi071.stdout:4/791: creat d2/d2a/da2/dd1/dde/d57/de5/f102 x:0 0 0 2024-04-23T02:14:09.041 INFO:tasks.workunit.client.1.smithi192.stdout:4/897: mkdir de/d15/d53/d12e 0 2024-04-23T02:14:09.041 INFO:tasks.workunit.client.0.smithi071.stdout:0/713: truncate d0/d3/f62 1167372 0 2024-04-23T02:14:09.041 INFO:tasks.workunit.client.1.smithi192.stdout:2/982: creat d8/d26/d27/d53/da2/dc1/f143 x:0 0 0 2024-04-23T02:14:09.042 INFO:tasks.workunit.client.0.smithi071.stdout:2/633: symlink d0/d2/d26/d73/lce 0 2024-04-23T02:14:09.043 INFO:tasks.workunit.client.1.smithi192.stdout:9/672: getdents d0/d9 0 2024-04-23T02:14:09.046 INFO:tasks.workunit.client.0.smithi071.stdout:2/634: write d0/d2/d26/d14/f22 [194859,128791] 0 2024-04-23T02:14:09.049 INFO:tasks.workunit.client.1.smithi192.stdout:4/898: stat de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/cd6 0 2024-04-23T02:14:09.049 INFO:tasks.workunit.client.1.smithi192.stdout:9/673: mknod d0/d9/d44/d53/cde 0 2024-04-23T02:14:09.049 INFO:tasks.workunit.client.1.smithi192.stdout:1/609: dwrite dc/f29 [0,4194304] 0 2024-04-23T02:14:09.049 INFO:tasks.workunit.client.0.smithi071.stdout:4/792: creat d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/f103 x:0 0 0 2024-04-23T02:14:09.051 INFO:tasks.workunit.client.1.smithi192.stdout:2/983: truncate d8/d26/d27/d57/d69/d99/de6/f142 44915 0 2024-04-23T02:14:09.052 INFO:tasks.workunit.client.0.smithi071.stdout:8/746: dwrite dc/d15/d72/fa8 [0,4194304] 0 2024-04-23T02:14:09.052 INFO:tasks.workunit.client.0.smithi071.stdout:0/714: fdatasync d0/d3/de/d10/d14/f85 0 2024-04-23T02:14:09.054 INFO:tasks.workunit.client.0.smithi071.stdout:0/715: chown d0/d3/de/d10/d14/l58 577933 1 2024-04-23T02:14:09.055 INFO:tasks.workunit.client.1.smithi192.stdout:4/899: rmdir de/d15/d53/d81/d10b 39 2024-04-23T02:14:09.056 INFO:tasks.workunit.client.0.smithi071.stdout:0/716: fsync d0/d63/fef 0 2024-04-23T02:14:09.057 INFO:tasks.workunit.client.1.smithi192.stdout:1/610: write dc/d8b/d34/d70/d8c/fa4 [3173282,130153] 0 2024-04-23T02:14:09.057 INFO:tasks.workunit.client.0.smithi071.stdout:8/747: creat dc/d15/d2d/ff2 x:0 0 0 2024-04-23T02:14:09.058 INFO:tasks.workunit.client.0.smithi071.stdout:0/717: stat d0/d3/de/d10/d14/d32/d37/ff8 0 2024-04-23T02:14:09.061 INFO:tasks.workunit.client.1.smithi192.stdout:1/611: truncate dc/d80/fac 92190 0 2024-04-23T02:14:09.062 INFO:tasks.workunit.client.0.smithi071.stdout:0/718: write d0/d3/de/d10/d14/d32/d37/ff8 [5029383,67281] 0 2024-04-23T02:14:09.062 INFO:tasks.workunit.client.1.smithi192.stdout:4/900: write de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/d59/d8a/f126 [71014,83055] 0 2024-04-23T02:14:09.063 INFO:tasks.workunit.client.1.smithi192.stdout:4/901: readlink l2 0 2024-04-23T02:14:09.066 INFO:tasks.workunit.client.1.smithi192.stdout:1/612: dread dc/d14/d15/d35/f73 [0,4194304] 0 2024-04-23T02:14:09.069 INFO:tasks.workunit.client.1.smithi192.stdout:9/674: dread d0/d9/f3a [0,4194304] 0 2024-04-23T02:14:09.069 INFO:tasks.workunit.client.1.smithi192.stdout:1/613: mkdir dc/d16/d19/d30/dc4 0 2024-04-23T02:14:09.070 INFO:tasks.workunit.client.0.smithi071.stdout:0/719: link d0/d3/de/d10/d14/d32/d44/d87/d93/fa3 d0/d3/de/d48/d7e/d8b/ffa 0 2024-04-23T02:14:09.071 INFO:tasks.workunit.client.1.smithi192.stdout:1/614: write dc/d14/d23/d5e/dab/fb9 [457130,63519] 0 2024-04-23T02:14:09.071 INFO:tasks.workunit.client.1.smithi192.stdout:4/902: symlink de/d15/d53/d81/de3/d74/d92/da0/dca/d34/l12f 0 2024-04-23T02:14:09.074 INFO:tasks.workunit.client.1.smithi192.stdout:1/615: mknod dc/d16/d19/d74/cc5 0 2024-04-23T02:14:09.074 INFO:tasks.workunit.client.1.smithi192.stdout:4/903: fsync de/d15/d53/d81/de3/f5c 0 2024-04-23T02:14:09.076 INFO:tasks.workunit.client.0.smithi071.stdout:0/720: dwrite d0/d3/de/d10/d14/d32/d8a/df0/f77 [0,4194304] 0 2024-04-23T02:14:09.078 INFO:tasks.workunit.client.0.smithi071.stdout:0/721: chown d0/d3/de/d10/d14/d32/d8a/df0/fe6 0 1 2024-04-23T02:14:09.078 INFO:tasks.workunit.client.0.smithi071.stdout:7/880: dread d3/dae/db7/dea/d5f/f60 [4194304,4194304] 0 2024-04-23T02:14:09.081 INFO:tasks.workunit.client.0.smithi071.stdout:0/722: mknod d0/d3/de/cfb 0 2024-04-23T02:14:09.084 INFO:tasks.workunit.client.0.smithi071.stdout:0/723: dread d0/d3/de/d10/d14/d32/fa4 [0,4194304] 0 2024-04-23T02:14:09.092 INFO:tasks.workunit.client.0.smithi071.stdout:2/635: dwrite d0/d2/d45/d7d/d56/f89 [0,4194304] 0 2024-04-23T02:14:09.095 INFO:tasks.workunit.client.0.smithi071.stdout:2/636: chown d0/d2/d26/d3a/dc3/d36/d5c/d85 13426735 1 2024-04-23T02:14:09.096 INFO:tasks.workunit.client.0.smithi071.stdout:2/637: readlink d0/d2/da0/db7/lc8 0 2024-04-23T02:14:09.096 INFO:tasks.workunit.client.0.smithi071.stdout:0/724: dwrite d0/d3/de/d10/d14/d32/d44/d87/d93/fa3 [0,4194304] 0 2024-04-23T02:14:09.098 INFO:tasks.workunit.client.0.smithi071.stdout:0/725: dread - d0/d3/de/d48/d7e/d8b/fa7 zero size 2024-04-23T02:14:09.101 INFO:tasks.workunit.client.0.smithi071.stdout:0/726: write d0/d3/de/d48/d7e/d8b/fb6 [2294098,127984] 0 2024-04-23T02:14:09.101 INFO:tasks.workunit.client.0.smithi071.stdout:2/638: rename d0/d2/c47 to d0/d2/da0/ccf 0 2024-04-23T02:14:09.104 INFO:tasks.workunit.client.1.smithi192.stdout:0/913: sync 2024-04-23T02:14:09.108 INFO:tasks.workunit.client.0.smithi071.stdout:2/639: dwrite d0/d2/d26/d3a/dc3/d36/d6a/fc6 [0,4194304] 0 2024-04-23T02:14:09.110 INFO:tasks.workunit.client.1.smithi192.stdout:1/616: readlink dc/l8a 0 2024-04-23T02:14:09.112 INFO:tasks.workunit.client.0.smithi071.stdout:7/881: dread d3/d17/d41/da8/f105 [0,4194304] 0 2024-04-23T02:14:09.114 INFO:tasks.workunit.client.1.smithi192.stdout:1/617: chown dc/d14/f27 1216281502 1 2024-04-23T02:14:09.114 INFO:tasks.workunit.client.0.smithi071.stdout:2/640: getdents d0/d2/d26/d3a 0 2024-04-23T02:14:09.116 INFO:tasks.workunit.client.1.smithi192.stdout:9/675: dwrite d0/d9/dc7/da0/fbe [0,4194304] 0 2024-04-23T02:14:09.117 INFO:tasks.workunit.client.1.smithi192.stdout:6/901: dread d3/d28/df8/d3a/d51/f93 [0,4194304] 0 2024-04-23T02:14:09.122 INFO:tasks.workunit.client.1.smithi192.stdout:6/902: chown d3/d28/d79/da0/dc2/d101/d80/d8e/l70 368 1 2024-04-23T02:14:09.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:08 smithi192 ceph-mon[49662]: pgmap v31: 65 pgs: 65 active+clean; 2.9 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail; 111 MiB/s rd, 325 MiB/s wr, 656 op/s 2024-04-23T02:14:09.128 INFO:tasks.workunit.client.0.smithi071.stdout:0/727: write d0/d3/de/d10/d14/d32/d37/fa2 [535619,110213] 0 2024-04-23T02:14:09.130 INFO:tasks.workunit.client.1.smithi192.stdout:9/676: creat d0/d9/d44/d4f/fdf x:0 0 0 2024-04-23T02:14:09.131 INFO:tasks.workunit.client.1.smithi192.stdout:1/618: getdents dc 0 2024-04-23T02:14:09.133 INFO:tasks.workunit.client.1.smithi192.stdout:9/677: truncate d0/d9/f5a 606931 0 2024-04-23T02:14:09.134 INFO:tasks.workunit.client.1.smithi192.stdout:2/984: dread d8/d26/d27/d53/f54 [0,4194304] 0 2024-04-23T02:14:09.134 INFO:tasks.workunit.client.1.smithi192.stdout:1/619: creat dc/d8b/d62/fc6 x:0 0 0 2024-04-23T02:14:09.135 INFO:tasks.workunit.client.0.smithi071.stdout:0/728: dwrite d0/fda [0,4194304] 0 2024-04-23T02:14:09.135 INFO:tasks.workunit.client.1.smithi192.stdout:9/678: write d0/d9/dc7/fc6 [1906761,118763] 0 2024-04-23T02:14:09.136 INFO:tasks.workunit.client.1.smithi192.stdout:4/904: dread de/d15/d53/d81/de3/d74/d92/faa [0,4194304] 0 2024-04-23T02:14:09.137 INFO:tasks.workunit.client.0.smithi071.stdout:0/729: read d0/d3/de/d10/d14/d32/d44/d87/fa0 [193976,112658] 0 2024-04-23T02:14:09.139 INFO:tasks.workunit.client.1.smithi192.stdout:1/620: unlink dc/d14/d15/d35/f49 0 2024-04-23T02:14:09.142 INFO:tasks.workunit.client.1.smithi192.stdout:9/679: creat d0/d9/db3/d9e/fe0 x:0 0 0 2024-04-23T02:14:09.143 INFO:tasks.workunit.client.1.smithi192.stdout:1/621: readlink dc/d16/d19/l22 0 2024-04-23T02:14:09.144 INFO:tasks.workunit.client.1.smithi192.stdout:9/680: dread - d0/fdb zero size 2024-04-23T02:14:09.148 INFO:tasks.workunit.client.1.smithi192.stdout:1/622: fsync dc/fd 0 2024-04-23T02:14:09.150 INFO:tasks.workunit.client.1.smithi192.stdout:2/985: dwrite d8/d26/d29/d43/d50/f141 [0,4194304] 0 2024-04-23T02:14:09.151 INFO:tasks.workunit.client.1.smithi192.stdout:6/903: dread d3/d28/df8/d3a/f4e [0,4194304] 0 2024-04-23T02:14:09.152 INFO:tasks.workunit.client.1.smithi192.stdout:0/914: dread d6/d4b/d9d/dc7/f80 [0,4194304] 0 2024-04-23T02:14:09.157 INFO:tasks.workunit.client.0.smithi071.stdout:7/882: write d3/d17/d1c/d2a/f4c [966178,111724] 0 2024-04-23T02:14:09.157 INFO:tasks.workunit.client.1.smithi192.stdout:2/986: rename d8/d26/d74/fc5 to d8/d26/d27/d31/dae/d11a/f144 0 2024-04-23T02:14:09.161 INFO:tasks.workunit.client.0.smithi071.stdout:7/883: chown d3/d17/d41/d3a/da5/de0 16462746 1 2024-04-23T02:14:09.166 INFO:tasks.workunit.client.1.smithi192.stdout:6/904: getdents d3/d18 0 2024-04-23T02:14:09.168 INFO:tasks.workunit.client.0.smithi071.stdout:2/641: dwrite d0/d2/d26/d3a/d75/f8f [0,4194304] 0 2024-04-23T02:14:09.170 INFO:tasks.workunit.client.0.smithi071.stdout:7/884: rename d3/dae/db7 to d3/d17/d41/d3a/d137 0 2024-04-23T02:14:09.171 INFO:tasks.workunit.client.0.smithi071.stdout:2/642: chown d0/d2/d45/d7d/d56/db0/d30/fa2 195 1 2024-04-23T02:14:09.172 INFO:tasks.workunit.client.1.smithi192.stdout:2/987: dwrite d8/d26/f106 [4194304,4194304] 0 2024-04-23T02:14:09.173 INFO:tasks.workunit.client.1.smithi192.stdout:6/905: creat d3/d15/d1b/da2/f129 x:0 0 0 2024-04-23T02:14:09.174 INFO:tasks.workunit.client.1.smithi192.stdout:2/988: creat d8/d26/d81/f145 x:0 0 0 2024-04-23T02:14:09.175 INFO:tasks.workunit.client.0.smithi071.stdout:2/643: creat d0/d2/d45/d7d/fd0 x:0 0 0 2024-04-23T02:14:09.176 INFO:tasks.workunit.client.0.smithi071.stdout:7/885: rename d3/d17/d1c/d2a/d73/fc0 to d3/d17/d41/d34/d6e/f138 0 2024-04-23T02:14:09.178 INFO:tasks.workunit.client.1.smithi192.stdout:4/905: truncate de/d15/d53/d81/de3/d74/d92/da0/dca/de5/fe9 1290785 0 2024-04-23T02:14:09.180 INFO:tasks.workunit.client.0.smithi071.stdout:7/886: creat d3/d17/d41/de2/f139 x:0 0 0 2024-04-23T02:14:09.180 INFO:tasks.workunit.client.1.smithi192.stdout:2/989: creat d8/d26/d27/d57/da0/d107/f146 x:0 0 0 2024-04-23T02:14:09.181 INFO:tasks.workunit.client.1.smithi192.stdout:1/623: truncate dc/f29 1814945 0 2024-04-23T02:14:09.181 INFO:tasks.workunit.client.1.smithi192.stdout:9/681: dread d0/d9/db3/fba [0,4194304] 0 2024-04-23T02:14:09.182 INFO:tasks.workunit.client.1.smithi192.stdout:2/990: chown d8/d26/d74/d7c/f122 12 1 2024-04-23T02:14:09.185 INFO:tasks.workunit.client.0.smithi071.stdout:2/644: link d0/d2/d26/d14/f42 d0/d2/d26/d3a/d75/fd1 0 2024-04-23T02:14:09.187 INFO:tasks.workunit.client.0.smithi071.stdout:7/887: mkdir d3/d17/d1c/d31/d10c/d11a/d13a 0 2024-04-23T02:14:09.188 INFO:tasks.workunit.client.0.smithi071.stdout:2/645: mknod d0/d2/d34/d83/cd2 0 2024-04-23T02:14:09.188 INFO:tasks.workunit.client.1.smithi192.stdout:9/682: rmdir d0/d9/db3/d5f/dc9 39 2024-04-23T02:14:09.189 INFO:tasks.workunit.client.1.smithi192.stdout:2/991: creat d8/d26/d29/da4/da8/f147 x:0 0 0 2024-04-23T02:14:09.189 INFO:tasks.workunit.client.1.smithi192.stdout:6/906: link d3/d15/c3d d3/d28/d79/da0/dc2/d101/d80/d8e/d128/c12a 0 2024-04-23T02:14:09.190 INFO:tasks.workunit.client.1.smithi192.stdout:1/624: creat dc/d16/db0/fc7 x:0 0 0 2024-04-23T02:14:09.193 INFO:tasks.workunit.client.1.smithi192.stdout:0/915: dwrite d6/df/d13/d25/d3a/f4f [4194304,4194304] 0 2024-04-23T02:14:09.194 INFO:tasks.workunit.client.0.smithi071.stdout:7/888: link d3/c6 d3/d51/d87/db3/c13b 0 2024-04-23T02:14:09.195 INFO:tasks.workunit.client.1.smithi192.stdout:6/907: read d3/d18/f78 [2424002,61612] 0 2024-04-23T02:14:09.195 INFO:tasks.workunit.client.0.smithi071.stdout:7/889: chown d3/d51/d80/de9 24 1 2024-04-23T02:14:09.196 INFO:tasks.workunit.client.1.smithi192.stdout:2/992: truncate d8/d26/d29/d43/d5d/dbd/dda/f109 370450 0 2024-04-23T02:14:09.198 INFO:tasks.workunit.client.0.smithi071.stdout:7/890: unlink d3/d51/df5/d10d/c12d 0 2024-04-23T02:14:09.200 INFO:tasks.workunit.client.0.smithi071.stdout:7/891: write d3/d17/d41/d34/f35 [42039,129067] 0 2024-04-23T02:14:09.200 INFO:tasks.workunit.client.1.smithi192.stdout:0/916: unlink d6/df/d56/fff 0 2024-04-23T02:14:09.203 INFO:tasks.workunit.client.1.smithi192.stdout:1/625: rename dc/d14/d15/d35/c86 to dc/d8b/d44/da9/cc8 0 2024-04-23T02:14:09.204 INFO:tasks.workunit.client.0.smithi071.stdout:5/802: sync 2024-04-23T02:14:09.205 INFO:tasks.workunit.client.0.smithi071.stdout:7/892: fsync d3/d17/d1c/f45 0 2024-04-23T02:14:09.205 INFO:tasks.workunit.client.0.smithi071.stdout:5/803: stat d3/d1d/d43/d4b/f7e 0 2024-04-23T02:14:09.208 INFO:tasks.workunit.client.1.smithi192.stdout:1/626: symlink dc/d16/d19/d74/d77/lc9 0 2024-04-23T02:14:09.208 INFO:tasks.workunit.client.0.smithi071.stdout:5/804: rename d3/d1d/d43/d4b/cd9 to d3/d1d/c12a 0 2024-04-23T02:14:09.214 INFO:tasks.workunit.client.0.smithi071.stdout:5/805: truncate d3/da/d1a/d1c/f62 270258 0 2024-04-23T02:14:09.214 INFO:tasks.workunit.client.1.smithi192.stdout:0/917: getdents d6/df/d13 0 2024-04-23T02:14:09.214 INFO:tasks.workunit.client.1.smithi192.stdout:4/906: dwrite de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/fae [0,4194304] 0 2024-04-23T02:14:09.215 INFO:tasks.workunit.client.0.smithi071.stdout:5/806: write d3/d1d/d43/fcb [602382,95917] 0 2024-04-23T02:14:09.216 INFO:tasks.workunit.client.1.smithi192.stdout:0/918: readlink d6/df/d56/l7f 0 2024-04-23T02:14:09.217 INFO:tasks.workunit.client.0.smithi071.stdout:2/646: dread d0/d2/d45/d7d/d56/fac [4194304,4194304] 0 2024-04-23T02:14:09.218 INFO:tasks.workunit.client.0.smithi071.stdout:2/647: stat d0/d2/da0/cb5 0 2024-04-23T02:14:09.220 INFO:tasks.workunit.client.1.smithi192.stdout:9/683: dwrite d0/d9/d44/d51/fb5 [0,4194304] 0 2024-04-23T02:14:09.221 INFO:tasks.workunit.client.0.smithi071.stdout:2/648: dread - d0/d2/d26/d3a/dc3/d4f/d98/fa6 zero size 2024-04-23T02:14:09.222 INFO:tasks.workunit.client.1.smithi192.stdout:1/627: rename dc/d16/d19/d30/l6d to dc/d80/lca 0 2024-04-23T02:14:09.223 INFO:tasks.workunit.client.1.smithi192.stdout:6/908: truncate d3/d28/df8/d3a/fa6 2720454 0 2024-04-23T02:14:09.225 INFO:tasks.workunit.client.1.smithi192.stdout:1/628: chown dc/d8b/d34/l61 1454650 1 2024-04-23T02:14:09.225 INFO:tasks.workunit.client.1.smithi192.stdout:4/907: chown de/d15/d19/d1a/l52 231146 1 2024-04-23T02:14:09.226 INFO:tasks.workunit.client.0.smithi071.stdout:7/893: dread d3/d17/d1c/f45 [0,4194304] 0 2024-04-23T02:14:09.226 INFO:tasks.workunit.client.0.smithi071.stdout:5/807: dwrite d3/d1d/d43/dd2/f78 [0,4194304] 0 2024-04-23T02:14:09.227 INFO:tasks.workunit.client.0.smithi071.stdout:2/649: read d0/d2/d45/d7d/d56/fac [747245,24946] 0 2024-04-23T02:14:09.229 INFO:tasks.workunit.client.0.smithi071.stdout:7/894: write d3/d17/d1c/f8f [4070792,89380] 0 2024-04-23T02:14:09.232 INFO:tasks.workunit.client.1.smithi192.stdout:2/993: dread d8/d26/d29/d43/d5d/dbd/ff6 [0,4194304] 0 2024-04-23T02:14:09.232 INFO:tasks.workunit.client.1.smithi192.stdout:6/909: chown d3/d28/df8/d7c/dda/c11e 511952815 1 2024-04-23T02:14:09.232 INFO:tasks.workunit.client.1.smithi192.stdout:0/919: dread d6/df/d35/d49/fc9 [0,4194304] 0 2024-04-23T02:14:09.233 INFO:tasks.workunit.client.1.smithi192.stdout:1/629: rmdir dc/d14/d23/d5e/dab 39 2024-04-23T02:14:09.236 INFO:tasks.workunit.client.0.smithi071.stdout:5/808: link d3/d1d/daf/d10/ca1 d3/d1d/c12b 0 2024-04-23T02:14:09.237 INFO:tasks.workunit.client.1.smithi192.stdout:0/920: mkdir d6/df/d13/d25/d107/d12f 0 2024-04-23T02:14:09.238 INFO:tasks.workunit.client.0.smithi071.stdout:2/650: getdents d0/d2/d45 0 2024-04-23T02:14:09.239 INFO:tasks.workunit.client.0.smithi071.stdout:5/809: creat d3/d1d/daf/d10/da5/f12c x:0 0 0 2024-04-23T02:14:09.239 INFO:tasks.workunit.client.0.smithi071.stdout:7/895: link d3/d17/d41/d34/d39/c113 d3/d17/d41/d3a/d137/dea/d5f/c13c 0 2024-04-23T02:14:09.239 INFO:tasks.workunit.client.1.smithi192.stdout:1/630: creat dc/d14/d23/da2/fcb x:0 0 0 2024-04-23T02:14:09.240 INFO:tasks.workunit.client.0.smithi071.stdout:5/810: stat d3/d8f/de6/c100 0 2024-04-23T02:14:09.240 INFO:tasks.workunit.client.1.smithi192.stdout:1/631: fsync dc/d8b/f4d 0 2024-04-23T02:14:09.240 INFO:tasks.workunit.client.0.smithi071.stdout:2/651: symlink d0/d2/d26/d3a/dc3/d4f/ld3 0 2024-04-23T02:14:09.241 INFO:tasks.workunit.client.1.smithi192.stdout:4/908: getdents de/d15/d53/d81/de3/d74 0 2024-04-23T02:14:09.241 INFO:tasks.workunit.client.1.smithi192.stdout:1/632: chown dc/d16/d19/l2f 7 1 2024-04-23T02:14:09.241 INFO:tasks.workunit.client.0.smithi071.stdout:5/811: truncate d3/d1d/daf/f89 1185830 0 2024-04-23T02:14:09.242 INFO:tasks.workunit.client.1.smithi192.stdout:6/910: symlink d3/d18/dd4/dff/df5/d121/l12b 0 2024-04-23T02:14:09.243 INFO:tasks.workunit.client.0.smithi071.stdout:2/652: write d0/d2/d26/d73/f9b [184902,29860] 0 2024-04-23T02:14:09.243 INFO:tasks.workunit.client.1.smithi192.stdout:1/633: fdatasync dc/d14/d23/da2/fc0 0 2024-04-23T02:14:09.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:08 smithi071 ceph-mon[39837]: pgmap v31: 65 pgs: 65 active+clean; 2.9 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail; 111 MiB/s rd, 325 MiB/s wr, 656 op/s 2024-04-23T02:14:09.245 INFO:tasks.workunit.client.1.smithi192.stdout:0/921: dread d6/df/d13/d6b/fab [0,4194304] 0 2024-04-23T02:14:09.245 INFO:tasks.workunit.client.0.smithi071.stdout:7/896: rmdir d3/d17/d41/d3a/da5/de0 0 2024-04-23T02:14:09.245 INFO:tasks.workunit.client.0.smithi071.stdout:5/812: symlink d3/d1d/daf/d10/da5/l12d 0 2024-04-23T02:14:09.247 INFO:tasks.workunit.client.0.smithi071.stdout:5/813: fdatasync d3/d1d/d43/d4b/ffc 0 2024-04-23T02:14:09.248 INFO:tasks.workunit.client.1.smithi192.stdout:1/634: write dc/d8b/d34/faa [300542,34224] 0 2024-04-23T02:14:09.248 INFO:tasks.workunit.client.1.smithi192.stdout:0/922: write d6/d4b/d9d/f11d [516266,113407] 0 2024-04-23T02:14:09.253 INFO:tasks.workunit.client.1.smithi192.stdout:2/994: truncate d8/d26/d29/d43/d5d/d119/f128 1357309 0 2024-04-23T02:14:09.253 INFO:tasks.workunit.client.1.smithi192.stdout:4/909: dwrite de/d15/d53/d81/de3/f4c [0,4194304] 0 2024-04-23T02:14:09.254 INFO:tasks.workunit.client.1.smithi192.stdout:3/744: sync 2024-04-23T02:14:09.254 INFO:tasks.workunit.client.1.smithi192.stdout:7/848: sync 2024-04-23T02:14:09.255 INFO:tasks.workunit.client.0.smithi071.stdout:7/897: dwrite d3/d17/d1c/fee [0,4194304] 0 2024-04-23T02:14:09.258 INFO:tasks.workunit.client.1.smithi192.stdout:1/635: rename dc/c8d to dc/d16/d19/d74/ccc 0 2024-04-23T02:14:09.258 INFO:tasks.workunit.client.1.smithi192.stdout:3/745: chown d1/d3/d17/dd1 5905 1 2024-04-23T02:14:09.259 INFO:tasks.workunit.client.0.smithi071.stdout:5/814: dread d3/d1d/d43/d4b/f7e [0,4194304] 0 2024-04-23T02:14:09.261 INFO:tasks.workunit.client.0.smithi071.stdout:7/898: write d3/d17/d41/d3a/d137/dea/fe6 [75133,128468] 0 2024-04-23T02:14:09.262 INFO:tasks.workunit.client.0.smithi071.stdout:2/653: dread d0/d82/f87 [0,4194304] 0 2024-04-23T02:14:09.263 INFO:tasks.workunit.client.1.smithi192.stdout:2/995: write d8/d26/d27/d57/da0/fc6 [2919100,13118] 0 2024-04-23T02:14:09.264 INFO:tasks.workunit.client.0.smithi071.stdout:7/899: symlink d3/d51/df5/d10d/d127/l13d 0 2024-04-23T02:14:09.266 INFO:tasks.workunit.client.1.smithi192.stdout:0/923: rename d6/df/d13/d25/d94/dd0/de6/dfa to d6/df/d13/d25/d94/d130 0 2024-04-23T02:14:09.266 INFO:tasks.workunit.client.0.smithi071.stdout:5/815: creat d3/da/d1a/db8/d106/f12e x:0 0 0 2024-04-23T02:14:09.268 INFO:tasks.workunit.client.1.smithi192.stdout:9/684: dwrite d0/f2c [0,4194304] 0 2024-04-23T02:14:09.270 INFO:tasks.workunit.client.1.smithi192.stdout:6/911: dwrite d3/d18/dd4/dff/fc8 [0,4194304] 0 2024-04-23T02:14:09.271 INFO:tasks.workunit.client.0.smithi071.stdout:7/900: unlink d3/d51/d87/d12c/c132 0 2024-04-23T02:14:09.271 INFO:tasks.workunit.client.1.smithi192.stdout:7/849: rename d5/d17/d1c/d59/d5a/d41/d50/d3d/c79 to d5/d17/d1c/d4a/daa/cf8 0 2024-04-23T02:14:09.272 INFO:tasks.workunit.client.1.smithi192.stdout:0/924: write d6/f6d [2321264,117868] 0 2024-04-23T02:14:09.272 INFO:tasks.workunit.client.0.smithi071.stdout:5/816: fdatasync d3/da/d1a/d1c/f41 0 2024-04-23T02:14:09.272 INFO:tasks.workunit.client.0.smithi071.stdout:7/901: chown d3/d51/d87/db3 0 1 2024-04-23T02:14:09.273 INFO:tasks.workunit.client.1.smithi192.stdout:4/910: getdents de/d15/d53/d81/de3/d74/d92/da0/dca/d9f/deb 0 2024-04-23T02:14:09.275 INFO:tasks.workunit.client.0.smithi071.stdout:5/817: symlink d3/d1d/d43/l12f 0 2024-04-23T02:14:09.279 INFO:tasks.workunit.client.1.smithi192.stdout:3/746: dread d1/d3/d17/d2b/d44/d57/f78 [0,4194304] 0 2024-04-23T02:14:09.279 INFO:tasks.workunit.client.1.smithi192.stdout:0/925: fdatasync d6/df/d13/d25/d3a/d93/ff0 0 2024-04-23T02:14:09.280 INFO:tasks.workunit.client.0.smithi071.stdout:2/654: dread d0/d2/d45/f52 [0,4194304] 0 2024-04-23T02:14:09.280 INFO:tasks.workunit.client.1.smithi192.stdout:9/685: creat d0/d9/dc7/fe1 x:0 0 0 2024-04-23T02:14:09.280 INFO:tasks.workunit.client.1.smithi192.stdout:2/996: link d8/d26/f106 d8/d26/d74/f148 0 2024-04-23T02:14:09.281 INFO:tasks.workunit.client.0.smithi071.stdout:5/818: truncate d3/da/d1a/db8/f2d 4961617 0 2024-04-23T02:14:09.281 INFO:tasks.workunit.client.0.smithi071.stdout:7/902: rename d3/d51/d80/f8a to d3/d17/d1c/d2a/d43/f13e 0 2024-04-23T02:14:09.282 INFO:tasks.workunit.client.1.smithi192.stdout:6/912: dread d3/d11/f43 [0,4194304] 0 2024-04-23T02:14:09.283 INFO:tasks.workunit.client.1.smithi192.stdout:7/850: dread d5/d17/d1c/d4a/f84 [0,4194304] 0 2024-04-23T02:14:09.286 INFO:tasks.workunit.client.0.smithi071.stdout:7/903: mkdir d3/d17/d41/d3a/d137/dea/d5f/d13f 0 2024-04-23T02:14:09.286 INFO:tasks.workunit.client.1.smithi192.stdout:9/686: symlink d0/d9/d44/d51/d5b/dd3/le2 0 2024-04-23T02:14:09.287 INFO:tasks.workunit.client.0.smithi071.stdout:2/655: mkdir d0/d2/d34/dd4 0 2024-04-23T02:14:09.288 INFO:tasks.workunit.client.1.smithi192.stdout:3/747: rename d1/d3/d17/d65/l9e to d1/d3/d17/df3/l105 0 2024-04-23T02:14:09.288 INFO:tasks.workunit.client.0.smithi071.stdout:5/819: truncate d3/d1d/f20 4652854 0 2024-04-23T02:14:09.288 INFO:tasks.workunit.client.1.smithi192.stdout:9/687: stat d0/d9/d44/d58/lcd 0 2024-04-23T02:14:09.289 INFO:tasks.workunit.client.1.smithi192.stdout:3/748: chown d1/d3/fd0 2079032 1 2024-04-23T02:14:09.289 INFO:tasks.workunit.client.1.smithi192.stdout:2/997: mknod d8/d26/d27/d57/da0/d107/c149 0 2024-04-23T02:14:09.291 INFO:tasks.workunit.client.0.smithi071.stdout:2/656: symlink d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/ld5 0 2024-04-23T02:14:09.291 INFO:tasks.workunit.client.1.smithi192.stdout:3/749: chown d1/d3/d6/d7/d29/d3b/da9/daf/ded 1980 1 2024-04-23T02:14:09.292 INFO:tasks.workunit.client.0.smithi071.stdout:5/820: symlink d3/d1d/d43/d4b/dce/l130 0 2024-04-23T02:14:09.295 INFO:tasks.workunit.client.0.smithi071.stdout:2/657: mknod d0/d2/da0/db7/cd6 0 2024-04-23T02:14:09.295 INFO:tasks.workunit.client.1.smithi192.stdout:9/688: symlink d0/d9/d44/d51/d5b/dd3/le3 0 2024-04-23T02:14:09.295 INFO:tasks.workunit.client.0.smithi071.stdout:5/821: creat d3/da/d1a/d1c/f131 x:0 0 0 2024-04-23T02:14:09.297 INFO:tasks.workunit.client.1.smithi192.stdout:2/998: creat d8/d26/d74/f14a x:0 0 0 2024-04-23T02:14:09.298 INFO:tasks.workunit.client.0.smithi071.stdout:7/904: dwrite d3/f69 [0,4194304] 0 2024-04-23T02:14:09.298 INFO:tasks.workunit.client.1.smithi192.stdout:9/689: write d0/d9/db3/d9e/fe0 [419434,57302] 0 2024-04-23T02:14:09.299 INFO:tasks.workunit.client.0.smithi071.stdout:2/658: creat d0/d2/d34/dbf/fd7 x:0 0 0 2024-04-23T02:14:09.303 INFO:tasks.workunit.client.0.smithi071.stdout:2/659: chown d0/d2/c16 205592 1 2024-04-23T02:14:09.305 INFO:tasks.workunit.client.1.smithi192.stdout:7/851: dwrite d5/d17/d1c/d59/d5a/d70/de9/da6/fb6 [4194304,4194304] 0 2024-04-23T02:14:09.305 INFO:tasks.workunit.client.1.smithi192.stdout:2/999: write d8/d26/d29/d43/d50/d5e/f89 [4291637,106] 0 2024-04-23T02:14:09.306 INFO:tasks.workunit.client.0.smithi071.stdout:2/660: creat d0/fd8 x:0 0 0 2024-04-23T02:14:09.307 INFO:tasks.workunit.client.1.smithi192.stdout:1/636: dread dc/d14/f27 [0,4194304] 0 2024-04-23T02:14:09.309 INFO:tasks.workunit.client.0.smithi071.stdout:2/661: truncate d0/d2/f9 2085458 0 2024-04-23T02:14:09.317 INFO:tasks.workunit.client.0.smithi071.stdout:5/822: write d3/da/d1a/f80 [947631,106288] 0 2024-04-23T02:14:09.318 INFO:tasks.workunit.client.1.smithi192.stdout:1/637: creat dc/d14/d15/fcd x:0 0 0 2024-04-23T02:14:09.318 INFO:tasks.workunit.client.0.smithi071.stdout:7/905: write d3/d17/fc7 [663026,14022] 0 2024-04-23T02:14:09.323 INFO:tasks.workunit.client.1.smithi192.stdout:3/750: truncate d1/d3/d17/d1a/d7c/dab/f54 946231 0 2024-04-23T02:14:09.323 INFO:tasks.workunit.client.1.smithi192.stdout:9/690: dwrite d0/fdb [0,4194304] 0 2024-04-23T02:14:09.324 INFO:tasks.workunit.client.1.smithi192.stdout:1/638: rmdir dc/d8b/d34/d70 39 2024-04-23T02:14:09.324 INFO:tasks.workunit.client.0.smithi071.stdout:7/906: write d3/d17/d41/de2/f139 [674004,10866] 0 2024-04-23T02:14:09.324 INFO:tasks.workunit.client.0.smithi071.stdout:5/823: chown d3/d1d/d43/dd2/d5a/da7/cc5 202 1 2024-04-23T02:14:09.324 INFO:tasks.workunit.client.0.smithi071.stdout:2/662: dwrite d0/d70/f86 [4194304,4194304] 0 2024-04-23T02:14:09.324 INFO:tasks.workunit.client.0.smithi071.stdout:2/663: write d0/fd8 [48487,19827] 0 2024-04-23T02:14:09.327 INFO:tasks.workunit.client.0.smithi071.stdout:7/907: creat d3/d17/d1c/d2a/d43/f140 x:0 0 0 2024-04-23T02:14:09.331 INFO:tasks.workunit.client.1.smithi192.stdout:7/852: dwrite d5/d17/d1c/f3e [0,4194304] 0 2024-04-23T02:14:09.331 INFO:tasks.workunit.client.0.smithi071.stdout:5/824: dwrite d3/d1d/daf/f89 [0,4194304] 0 2024-04-23T02:14:09.336 INFO:tasks.workunit.client.0.smithi071.stdout:5/825: creat d3/da/d1a/d6d/f132 x:0 0 0 2024-04-23T02:14:09.337 INFO:tasks.workunit.client.0.smithi071.stdout:7/908: creat d3/d17/d1c/f141 x:0 0 0 2024-04-23T02:14:09.340 INFO:tasks.workunit.client.1.smithi192.stdout:5/827: sync 2024-04-23T02:14:09.340 INFO:tasks.workunit.client.1.smithi192.stdout:6/913: dwrite d3/d15/d1b/f72 [0,4194304] 0 2024-04-23T02:14:09.341 INFO:tasks.workunit.client.1.smithi192.stdout:9/691: creat d0/d9/d44/d58/fe4 x:0 0 0 2024-04-23T02:14:09.341 INFO:tasks.workunit.client.1.smithi192.stdout:1/639: symlink dc/d8b/d44/da9/lce 0 2024-04-23T02:14:09.341 INFO:tasks.workunit.client.1.smithi192.stdout:5/828: creat d2/db/d2b/df7/f111 x:0 0 0 2024-04-23T02:14:09.342 INFO:tasks.workunit.client.0.smithi071.stdout:7/909: truncate d3/d17/d41/d37/f6f 1034946 0 2024-04-23T02:14:09.343 INFO:tasks.workunit.client.1.smithi192.stdout:4/911: dread de/d15/d53/d81/de3/f2c [0,4194304] 0 2024-04-23T02:14:09.343 INFO:tasks.workunit.client.0.smithi071.stdout:5/826: mknod d3/d1d/d43/dd2/c133 0 2024-04-23T02:14:09.343 INFO:tasks.workunit.client.1.smithi192.stdout:0/926: dread d6/df/d56/d85/da4/f11f [0,4194304] 0 2024-04-23T02:14:09.345 INFO:tasks.workunit.client.1.smithi192.stdout:3/751: creat d1/d3/d17/d1a/d7c/dab/d6c/d84/d4c/de3/f106 x:0 0 0 2024-04-23T02:14:09.345 INFO:tasks.workunit.client.1.smithi192.stdout:6/914: write d3/d18/dd4/dff/fc8 [3745076,97642] 0 2024-04-23T02:14:09.348 INFO:tasks.workunit.client.0.smithi071.stdout:7/910: mknod d3/d17/d41/d34/c142 0 2024-04-23T02:14:09.349 INFO:tasks.workunit.client.0.smithi071.stdout:7/911: rmdir d3/d17/d1c/d31/d112 39 2024-04-23T02:14:09.351 INFO:tasks.workunit.client.1.smithi192.stdout:4/912: fdatasync de/d15/d53/d81/de3/fe2 0 2024-04-23T02:14:09.351 INFO:tasks.workunit.client.1.smithi192.stdout:0/927: read - d6/df/d35/d49/fd2 zero size 2024-04-23T02:14:09.353 INFO:tasks.workunit.client.1.smithi192.stdout:7/853: rename d5/d17/d1c/d59/d5a/d41/d50/ce6 to d5/d43/d4c/cf9 0 2024-04-23T02:14:09.353 INFO:tasks.workunit.client.1.smithi192.stdout:5/829: unlink d2/d2e/f64 0 2024-04-23T02:14:09.356 INFO:tasks.workunit.client.1.smithi192.stdout:9/692: dread d0/fdb [0,4194304] 0 2024-04-23T02:14:09.356 INFO:tasks.workunit.client.1.smithi192.stdout:6/915: mknod d3/d11/c12c 0 2024-04-23T02:14:09.357 INFO:tasks.workunit.client.1.smithi192.stdout:0/928: read - d6/df/d56/d85/d45/ff2 zero size 2024-04-23T02:14:09.361 INFO:tasks.workunit.client.1.smithi192.stdout:5/830: read - d2/db/d16/d48/dc5/fcc zero size 2024-04-23T02:14:09.364 INFO:tasks.workunit.client.1.smithi192.stdout:9/693: rename d0/d9/d44/f7c to d0/d9/db3/d9e/fe5 0 2024-04-23T02:14:09.365 INFO:tasks.workunit.client.1.smithi192.stdout:7/854: getdents d5/d17/d1c/d4a/daa/de5 0 2024-04-23T02:14:09.365 INFO:tasks.workunit.client.1.smithi192.stdout:5/831: write d2/db/d2b/f7f [3741588,93222] 0 2024-04-23T02:14:09.366 INFO:tasks.workunit.client.1.smithi192.stdout:1/640: dwrite dc/d14/d23/f50 [12582912,4194304] 0 2024-04-23T02:14:09.366 INFO:tasks.workunit.client.1.smithi192.stdout:6/916: symlink d3/d28/d79/da0/dc2/d101/d80/de2/l12d 0 2024-04-23T02:14:09.367 INFO:tasks.workunit.client.1.smithi192.stdout:9/694: fdatasync d0/d9/f32 0 2024-04-23T02:14:09.369 INFO:tasks.workunit.client.0.smithi071.stdout:5/827: write d3/d1d/d43/f65 [134949,77136] 0 2024-04-23T02:14:09.370 INFO:tasks.workunit.client.0.smithi071.stdout:7/912: truncate d3/d51/df5/ff8 1209142 0 2024-04-23T02:14:09.371 INFO:tasks.workunit.client.1.smithi192.stdout:0/929: unlink d6/df/d13/d25/le2 0 2024-04-23T02:14:09.373 INFO:tasks.workunit.client.1.smithi192.stdout:1/641: truncate dc/d8b/fad 1007314 0 2024-04-23T02:14:09.374 INFO:tasks.workunit.client.0.smithi071.stdout:7/913: mkdir d3/d17/d41/d3a/d137/dea/d5f/d13f/d143 0 2024-04-23T02:14:09.375 INFO:tasks.workunit.client.1.smithi192.stdout:0/930: chown d6/df/d13/d25/d94/dd0/de6/l112 6829 1 2024-04-23T02:14:09.376 INFO:tasks.workunit.client.0.smithi071.stdout:2/664: dread d0/d2/d45/d7d/d56/f61 [0,4194304] 0 2024-04-23T02:14:09.376 INFO:tasks.workunit.client.0.smithi071.stdout:7/914: readlink d3/d51/df5/d10d/l136 0 2024-04-23T02:14:09.376 INFO:tasks.workunit.client.1.smithi192.stdout:0/931: dread - d6/df/d13/d25/d94/dd0/ffc zero size 2024-04-23T02:14:09.378 INFO:tasks.workunit.client.1.smithi192.stdout:3/752: rename d1/d3/d17/d1a/d7c/dab/d6c/d84/f63 to d1/d3/d6/d7/d29/d3b/da9/daf/ddf/f107 0 2024-04-23T02:14:09.379 INFO:tasks.workunit.client.0.smithi071.stdout:7/915: unlink d3/d17/f103 0 2024-04-23T02:14:09.382 INFO:tasks.workunit.client.1.smithi192.stdout:4/913: link de/d15/d53/cb8 de/d15/d53/d81/de3/c130 0 2024-04-23T02:14:09.383 INFO:tasks.workunit.client.1.smithi192.stdout:1/642: mknod dc/d8b/d62/ccf 0 2024-04-23T02:14:09.384 INFO:tasks.workunit.client.1.smithi192.stdout:1/643: chown dc/d14/f27 165 1 2024-04-23T02:14:09.386 INFO:tasks.workunit.client.0.smithi071.stdout:7/916: dwrite d3/d51/df5/f100 [0,4194304] 0 2024-04-23T02:14:09.386 INFO:tasks.workunit.client.1.smithi192.stdout:4/914: dread - de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/ff1 zero size 2024-04-23T02:14:09.386 INFO:tasks.workunit.client.1.smithi192.stdout:1/644: readlink dc/d14/d23/d5e/d6a/l5c 0 2024-04-23T02:14:09.387 INFO:tasks.workunit.client.0.smithi071.stdout:7/917: fsync d3/d17/d41/d3a/da5/fef 0 2024-04-23T02:14:09.389 INFO:tasks.workunit.client.1.smithi192.stdout:3/753: dread - d1/d3/d6/d7/d29/d3b/da9/d8b/fac zero size 2024-04-23T02:14:09.394 INFO:tasks.workunit.client.1.smithi192.stdout:1/645: chown dc/d8b/d34/d70 4233 1 2024-04-23T02:14:09.395 INFO:tasks.workunit.client.0.smithi071.stdout:2/665: truncate d0/d2/d26/d3a/d75/f8f 2759152 0 2024-04-23T02:14:09.397 INFO:tasks.workunit.client.0.smithi071.stdout:2/666: symlink d0/d82/ld9 0 2024-04-23T02:14:09.399 INFO:tasks.workunit.client.0.smithi071.stdout:2/667: creat d0/d2/d26/d3a/dc3/d4f/fda x:0 0 0 2024-04-23T02:14:09.400 INFO:tasks.workunit.client.1.smithi192.stdout:4/915: dwrite de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d11b/fc2 [0,4194304] 0 2024-04-23T02:14:09.400 INFO:tasks.workunit.client.1.smithi192.stdout:7/855: link d5/d17/d1c/d57/f48 d5/d17/d1c/d59/d5a/d41/ffa 0 2024-04-23T02:14:09.400 INFO:tasks.workunit.client.0.smithi071.stdout:2/668: readlink d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/ld5 0 2024-04-23T02:14:09.402 INFO:tasks.workunit.client.0.smithi071.stdout:7/918: write d3/dae/fcc [3718341,78506] 0 2024-04-23T02:14:09.403 INFO:tasks.workunit.client.1.smithi192.stdout:3/754: creat d1/d3/d6/d7/d29/d3b/da9/dbd/d103/f108 x:0 0 0 2024-04-23T02:14:09.403 INFO:tasks.workunit.client.0.smithi071.stdout:7/919: fsync d3/d17/d1c/f26 0 2024-04-23T02:14:09.404 INFO:tasks.workunit.client.1.smithi192.stdout:1/646: symlink dc/d16/d19/dae/ld0 0 2024-04-23T02:14:09.405 INFO:tasks.workunit.client.1.smithi192.stdout:4/916: mkdir de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d11b/d131 0 2024-04-23T02:14:09.405 INFO:tasks.workunit.client.0.smithi071.stdout:7/920: chown d3/d17/d41/f33 2007584185 1 2024-04-23T02:14:09.409 INFO:tasks.workunit.client.1.smithi192.stdout:3/755: mknod d1/d3/c109 0 2024-04-23T02:14:09.411 INFO:tasks.workunit.client.1.smithi192.stdout:1/647: mknod dc/d16/d19/d30/dbf/cd1 0 2024-04-23T02:14:09.414 INFO:tasks.workunit.client.1.smithi192.stdout:9/695: dread d0/d9/d44/f45 [0,4194304] 0 2024-04-23T02:14:09.416 INFO:tasks.workunit.client.1.smithi192.stdout:1/648: fsync dc/d16/db0/fc7 0 2024-04-23T02:14:09.417 INFO:tasks.workunit.client.1.smithi192.stdout:4/917: fsync de/d15/d19/fe0 0 2024-04-23T02:14:09.417 INFO:tasks.workunit.client.1.smithi192.stdout:5/832: dwrite d2/db/d2b/d52/dc2/d49/fb1 [4194304,4194304] 0 2024-04-23T02:14:09.419 INFO:tasks.workunit.client.1.smithi192.stdout:1/649: stat dc/d16/d19/d30/f3f 0 2024-04-23T02:14:09.421 INFO:tasks.workunit.client.1.smithi192.stdout:6/917: dwrite d3/d15/d1b/f8f [0,4194304] 0 2024-04-23T02:14:09.424 INFO:tasks.workunit.client.1.smithi192.stdout:4/918: stat de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/f122 0 2024-04-23T02:14:09.425 INFO:tasks.workunit.client.1.smithi192.stdout:1/650: read dc/d14/d15/d35/f40 [1007408,12763] 0 2024-04-23T02:14:09.426 INFO:tasks.workunit.client.1.smithi192.stdout:3/756: link d1/d3/l6f d1/d3/d17/d2b/l10a 0 2024-04-23T02:14:09.429 INFO:tasks.workunit.client.1.smithi192.stdout:1/651: write dc/d14/d23/da2/fcb [793155,26637] 0 2024-04-23T02:14:09.432 INFO:tasks.workunit.client.1.smithi192.stdout:4/919: creat de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/d116/f132 x:0 0 0 2024-04-23T02:14:09.433 INFO:tasks.workunit.client.1.smithi192.stdout:6/918: getdents d3/d18/dd4 0 2024-04-23T02:14:09.434 INFO:tasks.workunit.client.1.smithi192.stdout:3/757: read - d1/d3/d6/d7/d29/d3b/da9/d8b/fb6 zero size 2024-04-23T02:14:09.434 INFO:tasks.workunit.client.1.smithi192.stdout:1/652: write dc/d8b/f47 [1147105,68725] 0 2024-04-23T02:14:09.446 INFO:tasks.workunit.client.1.smithi192.stdout:7/856: dread d5/d43/d4c/d4e/fc6 [0,4194304] 0 2024-04-23T02:14:09.446 INFO:tasks.workunit.client.1.smithi192.stdout:9/696: dwrite d0/f54 [0,4194304] 0 2024-04-23T02:14:09.472 INFO:tasks.workunit.client.0.smithi071.stdout:7/921: truncate d3/d17/d1c/d2a/d43/f140 30631 0 2024-04-23T02:14:09.474 INFO:tasks.workunit.client.0.smithi071.stdout:7/922: creat d3/f144 x:0 0 0 2024-04-23T02:14:09.474 INFO:tasks.workunit.client.1.smithi192.stdout:3/758: readlink d1/d3/d17/d1f/d33/l28 0 2024-04-23T02:14:09.477 INFO:tasks.workunit.client.1.smithi192.stdout:1/653: rename dc/d16/d19/d30/dbf to dc/d16/d19/d74/dd2 0 2024-04-23T02:14:09.478 INFO:tasks.workunit.client.1.smithi192.stdout:9/697: write d0/d9/db3/fba [1903033,5442] 0 2024-04-23T02:14:09.479 INFO:tasks.workunit.client.0.smithi071.stdout:7/923: dwrite d3/d17/f36 [0,4194304] 0 2024-04-23T02:14:09.481 INFO:tasks.workunit.client.1.smithi192.stdout:7/857: write d5/d17/d1c/d59/d5a/d41/d50/d3d/fbf [898221,82237] 0 2024-04-23T02:14:09.482 INFO:tasks.workunit.client.0.smithi071.stdout:7/924: mkdir d3/d17/dca/d145 0 2024-04-23T02:14:09.483 INFO:tasks.workunit.client.0.smithi071.stdout:7/925: stat d3/d17/d41/d34/f89 0 2024-04-23T02:14:09.485 INFO:tasks.workunit.client.1.smithi192.stdout:9/698: readlink d0/d9/d44/d4f/l8b 0 2024-04-23T02:14:09.489 INFO:tasks.workunit.client.0.smithi071.stdout:7/926: dwrite d3/d17/d1c/f8f [0,4194304] 0 2024-04-23T02:14:09.491 INFO:tasks.workunit.client.0.smithi071.stdout:7/927: truncate d3/d17/d1c/f26 5214846 0 2024-04-23T02:14:09.514 INFO:tasks.workunit.client.0.smithi071.stdout:7/928: dread d3/d17/d41/d3a/d137/dea/dc5/f10a [0,4194304] 0 2024-04-23T02:14:09.516 INFO:tasks.workunit.client.0.smithi071.stdout:7/929: creat d3/d17/d11b/f146 x:0 0 0 2024-04-23T02:14:09.530 INFO:tasks.workunit.client.1.smithi192.stdout:6/919: getdents d3/d28/df8 0 2024-04-23T02:14:09.531 INFO:tasks.workunit.client.1.smithi192.stdout:9/699: unlink d0/c21 0 2024-04-23T02:14:09.532 INFO:tasks.workunit.client.1.smithi192.stdout:1/654: getdents dc/d8b/d34/d70 0 2024-04-23T02:14:09.532 INFO:tasks.workunit.client.1.smithi192.stdout:6/920: readlink d3/d15/l4d 0 2024-04-23T02:14:09.547 INFO:tasks.workunit.client.1.smithi192.stdout:1/655: rmdir dc/d16/d19/d74/d77 39 2024-04-23T02:14:09.550 INFO:tasks.workunit.client.1.smithi192.stdout:6/921: write d3/d28/d79/da0/dc2/f109 [161809,19478] 0 2024-04-23T02:14:09.552 INFO:tasks.workunit.client.1.smithi192.stdout:1/656: fdatasync dc/d14/d23/f87 0 2024-04-23T02:14:09.552 INFO:tasks.workunit.client.1.smithi192.stdout:3/759: dwrite d1/d3/d17/d1a/f4f [0,4194304] 0 2024-04-23T02:14:09.554 INFO:tasks.workunit.client.1.smithi192.stdout:7/858: dwrite d5/d17/f92 [0,4194304] 0 2024-04-23T02:14:09.561 INFO:tasks.workunit.client.1.smithi192.stdout:7/859: chown d5/d17/d1c/d26/l55 76626718 1 2024-04-23T02:14:09.561 INFO:tasks.workunit.client.1.smithi192.stdout:9/700: dread d0/d9/dc7/da0/fbe [0,4194304] 0 2024-04-23T02:14:09.562 INFO:tasks.workunit.client.1.smithi192.stdout:3/760: creat d1/d3/d17/d1a/d7c/dab/d6c/d84/d5d/f10b x:0 0 0 2024-04-23T02:14:09.562 INFO:tasks.workunit.client.1.smithi192.stdout:1/657: write dc/d16/d19/f79 [1855333,86652] 0 2024-04-23T02:14:09.564 INFO:tasks.workunit.client.1.smithi192.stdout:6/922: creat d3/d28/d79/da0/dc2/d101/d80/d8e/d9d/ded/f12e x:0 0 0 2024-04-23T02:14:09.565 INFO:tasks.workunit.client.1.smithi192.stdout:9/701: write d0/d9/d44/d4f/fdf [130448,60020] 0 2024-04-23T02:14:09.569 INFO:tasks.workunit.client.1.smithi192.stdout:6/923: fdatasync d3/d18/d58/f75 0 2024-04-23T02:14:09.569 INFO:tasks.workunit.client.1.smithi192.stdout:9/702: chown d0/dd2 29 1 2024-04-23T02:14:09.569 INFO:tasks.workunit.client.1.smithi192.stdout:7/860: rmdir d5/d17/d1c/d4a 39 2024-04-23T02:14:09.569 INFO:tasks.workunit.client.1.smithi192.stdout:3/761: mknod d1/d3/d6/d7/d5f/c10c 0 2024-04-23T02:14:09.570 INFO:tasks.workunit.client.1.smithi192.stdout:9/703: creat d0/d9/d44/d58/dd9/fe6 x:0 0 0 2024-04-23T02:14:09.572 INFO:tasks.workunit.client.0.smithi071.stdout:1/639: sync 2024-04-23T02:14:09.573 INFO:tasks.workunit.client.1.smithi192.stdout:9/704: dread d0/d9/f3a [0,4194304] 0 2024-04-23T02:14:09.574 INFO:tasks.workunit.client.0.smithi071.stdout:1/640: rmdir db/df/d5f/d6f/db2/d86 39 2024-04-23T02:14:09.575 INFO:tasks.workunit.client.1.smithi192.stdout:9/705: truncate d0/fd1 130661 0 2024-04-23T02:14:09.577 INFO:tasks.workunit.client.0.smithi071.stdout:1/641: symlink db/df/d5f/d6f/db2/ld0 0 2024-04-23T02:14:09.579 INFO:tasks.workunit.client.1.smithi192.stdout:3/762: fsync d1/d3/d6/d7/d29/d3b/da9/f48 0 2024-04-23T02:14:09.580 INFO:tasks.workunit.client.1.smithi192.stdout:9/706: truncate d0/d9/d44/d51/d5b/d61/f7b 5011634 0 2024-04-23T02:14:09.580 INFO:tasks.workunit.client.0.smithi071.stdout:1/642: fsync db/df/d13/d2f/d48/d85/f90 0 2024-04-23T02:14:09.581 INFO:tasks.workunit.client.1.smithi192.stdout:9/707: read d0/f54 [3870230,106913] 0 2024-04-23T02:14:09.582 INFO:tasks.workunit.client.1.smithi192.stdout:6/924: write d3/d15/d1b/da2/faa [1999665,44046] 0 2024-04-23T02:14:09.584 INFO:tasks.workunit.client.1.smithi192.stdout:3/763: chown d1/d3/d6/d7/d71/c89 39 1 2024-04-23T02:14:09.587 INFO:tasks.workunit.client.1.smithi192.stdout:3/764: chown d1/d3/fd0 2855 1 2024-04-23T02:14:09.589 INFO:tasks.workunit.client.1.smithi192.stdout:6/925: getdents d3/d28/d79/dbe 0 2024-04-23T02:14:09.590 INFO:tasks.workunit.client.1.smithi192.stdout:7/861: dwrite d5/d17/d1c/d59/d5a/d41/d50/f62 [4194304,4194304] 0 2024-04-23T02:14:09.592 INFO:tasks.workunit.client.1.smithi192.stdout:1/658: dread dc/f76 [0,4194304] 0 2024-04-23T02:14:09.594 INFO:tasks.workunit.client.1.smithi192.stdout:1/659: chown dc/d80/fac 502 1 2024-04-23T02:14:09.595 INFO:tasks.workunit.client.1.smithi192.stdout:6/926: mknod d3/d28/d79/da0/dc2/d101/d80/d8e/d9d/c12f 0 2024-04-23T02:14:09.596 INFO:tasks.workunit.client.1.smithi192.stdout:6/927: read - d3/feb zero size 2024-04-23T02:14:09.599 INFO:tasks.workunit.client.1.smithi192.stdout:3/765: link d1/d3/d6/d7/d71/c89 d1/d3/d6/d7/d29/d3b/da9/d7b/c10d 0 2024-04-23T02:14:09.600 INFO:tasks.workunit.client.1.smithi192.stdout:7/862: link d5/d17/d1c/d26/l95 d5/d43/d4c/d5c/lfb 0 2024-04-23T02:14:09.605 INFO:tasks.workunit.client.1.smithi192.stdout:1/660: dwrite dc/d80/d81/f9e [0,4194304] 0 2024-04-23T02:14:09.606 INFO:tasks.workunit.client.1.smithi192.stdout:7/863: read d5/d17/d1c/d59/da8/fb0 [139114,110160] 0 2024-04-23T02:14:09.606 INFO:tasks.workunit.client.1.smithi192.stdout:3/766: stat d1/d3/d6/d7/d29/d3b/da9/d7b/ld3 0 2024-04-23T02:14:09.607 INFO:tasks.workunit.client.1.smithi192.stdout:1/661: write dc/d8b/d62/fc6 [81421,95628] 0 2024-04-23T02:14:09.609 INFO:tasks.workunit.client.1.smithi192.stdout:6/928: dread d3/d18/dd4/ffb [0,4194304] 0 2024-04-23T02:14:09.611 INFO:tasks.workunit.client.1.smithi192.stdout:7/864: chown d5/d17/c2c 0 1 2024-04-23T02:14:09.613 INFO:tasks.workunit.client.1.smithi192.stdout:1/662: rename dc/ff to dc/d14/d23/d5e/dab/fd3 0 2024-04-23T02:14:09.615 INFO:tasks.workunit.client.1.smithi192.stdout:3/767: rename d1/d3/d6/d7/d29/d3b/f8e to d1/d3/d6/d7/d29/d3b/da9/daf/f10e 0 2024-04-23T02:14:09.617 INFO:tasks.workunit.client.1.smithi192.stdout:1/663: creat dc/d8b/d34/d70/d8c/da3/fd4 x:0 0 0 2024-04-23T02:14:09.618 INFO:tasks.workunit.client.1.smithi192.stdout:1/664: read dc/d16/d19/f79 [1022518,99687] 0 2024-04-23T02:14:09.620 INFO:tasks.workunit.client.1.smithi192.stdout:1/665: unlink dc/d14/d23/d5e/d6a/c57 0 2024-04-23T02:14:09.623 INFO:tasks.workunit.client.1.smithi192.stdout:1/666: dread dc/f29 [0,4194304] 0 2024-04-23T02:14:09.624 INFO:tasks.workunit.client.1.smithi192.stdout:1/667: write dc/d16/d19/f9f [810822,111681] 0 2024-04-23T02:14:09.628 INFO:tasks.workunit.client.1.smithi192.stdout:1/668: mknod dc/d8b/d34/d70/d8c/da3/dc1/cd5 0 2024-04-23T02:14:09.630 INFO:tasks.workunit.client.1.smithi192.stdout:1/669: creat dc/d16/fd6 x:0 0 0 2024-04-23T02:14:09.634 INFO:tasks.workunit.client.1.smithi192.stdout:1/670: link dc/d8b/d44/c4b dc/d16/d19/d74/dd2/cd7 0 2024-04-23T02:14:09.635 INFO:tasks.workunit.client.0.smithi071.stdout:6/678: sync 2024-04-23T02:14:09.635 INFO:tasks.workunit.client.0.smithi071.stdout:3/673: sync 2024-04-23T02:14:09.635 INFO:tasks.workunit.client.0.smithi071.stdout:9/687: sync 2024-04-23T02:14:09.637 INFO:tasks.workunit.client.0.smithi071.stdout:9/688: chown d1/f28 152894 1 2024-04-23T02:14:09.637 INFO:tasks.workunit.client.0.smithi071.stdout:6/679: read d6/d17/d77/f96 [118417,30893] 0 2024-04-23T02:14:09.637 INFO:tasks.workunit.client.1.smithi192.stdout:1/671: getdents dc/d16/d19/d74/d77 0 2024-04-23T02:14:09.639 INFO:tasks.workunit.client.1.smithi192.stdout:1/672: rmdir dc 39 2024-04-23T02:14:09.639 INFO:tasks.workunit.client.1.smithi192.stdout:3/768: dread d1/f5 [0,4194304] 0 2024-04-23T02:14:09.640 INFO:tasks.workunit.client.1.smithi192.stdout:6/929: dread d3/d15/d1b/dcf/fd3 [0,4194304] 0 2024-04-23T02:14:09.640 INFO:tasks.workunit.client.0.smithi071.stdout:6/680: fsync d6/d17/d77/d85/fa1 0 2024-04-23T02:14:09.640 INFO:tasks.workunit.client.0.smithi071.stdout:9/689: chown d1/d32/d3a/c51 59103819 1 2024-04-23T02:14:09.640 INFO:tasks.workunit.client.1.smithi192.stdout:6/930: read - d3/f10c zero size 2024-04-23T02:14:09.642 INFO:tasks.workunit.client.0.smithi071.stdout:6/681: mkdir d6/d91/dd5 0 2024-04-23T02:14:09.643 INFO:tasks.workunit.client.1.smithi192.stdout:3/769: unlink d1/d3/d17/d1a/d7c/dab/d6c/d84/d46/f5e 0 2024-04-23T02:14:09.643 INFO:tasks.workunit.client.0.smithi071.stdout:3/674: dwrite d7/d8/f11 [0,4194304] 0 2024-04-23T02:14:09.644 INFO:tasks.workunit.client.1.smithi192.stdout:6/931: mknod d3/d28/df8/d52/c130 0 2024-04-23T02:14:09.644 INFO:tasks.workunit.client.0.smithi071.stdout:6/682: readlink d6/d17/d77/d9f/ld2 0 2024-04-23T02:14:09.646 INFO:tasks.workunit.client.1.smithi192.stdout:1/673: truncate dc/d8b/f4d 2466081 0 2024-04-23T02:14:09.646 INFO:tasks.workunit.client.0.smithi071.stdout:3/675: dread - d7/da5/db2/db7/d68/d3b/d4c/da1/fc2 zero size 2024-04-23T02:14:09.647 INFO:tasks.workunit.client.1.smithi192.stdout:3/770: symlink d1/d3/d17/d1a/d7c/dab/d6c/l10f 0 2024-04-23T02:14:09.648 INFO:tasks.workunit.client.1.smithi192.stdout:6/932: creat d3/d18/d58/f131 x:0 0 0 2024-04-23T02:14:09.649 INFO:tasks.workunit.client.0.smithi071.stdout:3/676: write d7/d1d/fb9 [625283,121462] 0 2024-04-23T02:14:09.651 INFO:tasks.workunit.client.0.smithi071.stdout:9/690: dwrite d1/d32/d41/db7/d5a/dc6/f7a [0,4194304] 0 2024-04-23T02:14:09.653 INFO:tasks.workunit.client.0.smithi071.stdout:3/677: dread d7/da5/db2/db7/d68/d3b/d4c/f65 [0,4194304] 0 2024-04-23T02:14:09.654 INFO:tasks.workunit.client.0.smithi071.stdout:6/683: link d6/d1f/lce d6/d22/dbf/ld6 0 2024-04-23T02:14:09.655 INFO:tasks.workunit.client.1.smithi192.stdout:3/771: chown d1/d3/d6/d7/d29/d3b/da9/dbd/d103/f108 179 1 2024-04-23T02:14:09.655 INFO:tasks.workunit.client.0.smithi071.stdout:6/684: stat d6/d22/d92/l93 0 2024-04-23T02:14:09.658 INFO:tasks.workunit.client.0.smithi071.stdout:9/691: unlink d1/d32/d41/db7/d5a/cda 0 2024-04-23T02:14:09.659 INFO:tasks.workunit.client.1.smithi192.stdout:1/674: dwrite dc/d16/d19/d30/f55 [0,4194304] 0 2024-04-23T02:14:09.659 INFO:tasks.workunit.client.0.smithi071.stdout:3/678: mkdir d7/d22/def 0 2024-04-23T02:14:09.661 INFO:tasks.workunit.client.0.smithi071.stdout:9/692: rmdir d1/d32/d3a/d5f 39 2024-04-23T02:14:09.662 INFO:tasks.workunit.client.1.smithi192.stdout:1/675: creat dc/d16/db0/fd8 x:0 0 0 2024-04-23T02:14:09.664 INFO:tasks.workunit.client.1.smithi192.stdout:3/772: dwrite d1/d3/d17/d2b/d44/f100 [0,4194304] 0 2024-04-23T02:14:09.666 INFO:tasks.workunit.client.0.smithi071.stdout:6/685: dwrite d6/d1f/f25 [4194304,4194304] 0 2024-04-23T02:14:09.667 INFO:tasks.workunit.client.1.smithi192.stdout:3/773: readlink d1/d3/l6f 0 2024-04-23T02:14:09.668 INFO:tasks.workunit.client.0.smithi071.stdout:9/693: rename d1/d32/d3a/d94/dab to d1/d32/d41/db7/d39/df6 0 2024-04-23T02:14:09.669 INFO:tasks.workunit.client.1.smithi192.stdout:6/933: dread d3/d28/d79/da0/dc2/d101/d80/d8e/f50 [0,4194304] 0 2024-04-23T02:14:09.670 INFO:tasks.workunit.client.1.smithi192.stdout:3/774: readlink d1/d3/d17/d1f/d33/l28 0 2024-04-23T02:14:09.672 INFO:tasks.workunit.client.1.smithi192.stdout:6/934: chown d3/d28/d79/da0/dc2/d101/d80/d8e/d9d/ded 40 1 2024-04-23T02:14:09.672 INFO:tasks.workunit.client.0.smithi071.stdout:3/679: dwrite d7/da5/db2/db7/d68/d3b/d4c/f63 [0,4194304] 0 2024-04-23T02:14:09.675 INFO:tasks.workunit.client.1.smithi192.stdout:1/676: rename dc/d8b/d34/d70/d8c/da3/fd4 to dc/d16/d19/d30/dc4/fd9 0 2024-04-23T02:14:09.676 INFO:tasks.workunit.client.1.smithi192.stdout:3/775: creat d1/d3/d17/dd1/f110 x:0 0 0 2024-04-23T02:14:09.678 INFO:tasks.workunit.client.1.smithi192.stdout:1/677: unlink dc/d14/d23/d5e/d6a/lc3 0 2024-04-23T02:14:09.679 INFO:tasks.workunit.client.0.smithi071.stdout:3/680: creat d7/da5/db2/db7/d68/d3b/d4c/da1/da8/ff0 x:0 0 0 2024-04-23T02:14:09.679 INFO:tasks.workunit.client.1.smithi192.stdout:1/678: chown dc/d14/d15/c65 490 1 2024-04-23T02:14:09.680 INFO:tasks.workunit.client.1.smithi192.stdout:6/935: unlink d3/d28/d79/dbe/dcb/de9/l124 0 2024-04-23T02:14:09.680 INFO:tasks.workunit.client.0.smithi071.stdout:6/686: link d6/d17/l43 d6/d22/dbf/ld7 0 2024-04-23T02:14:09.682 INFO:tasks.workunit.client.1.smithi192.stdout:3/776: symlink d1/d3/d17/d65/d99/l111 0 2024-04-23T02:14:09.682 INFO:tasks.workunit.client.0.smithi071.stdout:3/681: rmdir d7/d22 39 2024-04-23T02:14:09.683 INFO:tasks.workunit.client.0.smithi071.stdout:3/682: chown d7/d8/f1b 10 1 2024-04-23T02:14:09.684 INFO:tasks.workunit.client.1.smithi192.stdout:1/679: read dc/f39 [3542921,83219] 0 2024-04-23T02:14:09.684 INFO:tasks.workunit.client.0.smithi071.stdout:9/694: link d1/d32/lad d1/d32/d41/db7/d5a/lf7 0 2024-04-23T02:14:09.685 INFO:tasks.workunit.client.1.smithi192.stdout:1/680: chown dc/d14/d23/da2/fc0 32910154 1 2024-04-23T02:14:09.686 INFO:tasks.workunit.client.1.smithi192.stdout:6/936: creat d3/d28/df8/d3a/dbd/f132 x:0 0 0 2024-04-23T02:14:09.688 INFO:tasks.workunit.client.0.smithi071.stdout:9/695: rmdir d1/d32/d41/db7/d5a/dc6/d81 39 2024-04-23T02:14:09.688 INFO:tasks.workunit.client.1.smithi192.stdout:1/681: readlink dc/d16/d19/d30/l41 0 2024-04-23T02:14:09.692 INFO:tasks.workunit.client.1.smithi192.stdout:6/937: creat d3/d28/d79/da0/dc2/d101/d80/d8e/d9d/ded/f133 x:0 0 0 2024-04-23T02:14:09.693 INFO:tasks.workunit.client.1.smithi192.stdout:3/777: getdents d1/d3/d17/d1a/d7c/dab/d6c/d84/d5d 0 2024-04-23T02:14:09.696 INFO:tasks.workunit.client.1.smithi192.stdout:1/682: dwrite dc/d14/f27 [4194304,4194304] 0 2024-04-23T02:14:09.700 INFO:tasks.workunit.client.1.smithi192.stdout:1/683: symlink dc/d14/d23/lda 0 2024-04-23T02:14:09.703 INFO:tasks.workunit.client.1.smithi192.stdout:1/684: dread - dc/d14/d23/da2/fc0 zero size 2024-04-23T02:14:09.703 INFO:tasks.workunit.client.1.smithi192.stdout:3/778: dwrite d1/d3/d6/d7/f102 [0,4194304] 0 2024-04-23T02:14:09.707 INFO:tasks.workunit.client.0.smithi071.stdout:3/683: write d7/d10/d38/f41 [357280,49958] 0 2024-04-23T02:14:09.712 INFO:tasks.workunit.client.0.smithi071.stdout:6/687: dwrite d6/d1f/f3b [0,4194304] 0 2024-04-23T02:14:09.715 INFO:tasks.workunit.client.0.smithi071.stdout:3/684: rename d7/da5/db2/db7/d68/d3b/ccd to d7/d8/d42/d80/cf1 0 2024-04-23T02:14:09.716 INFO:tasks.workunit.client.1.smithi192.stdout:1/685: write fa [793263,90940] 0 2024-04-23T02:14:09.717 INFO:tasks.workunit.client.0.smithi071.stdout:6/688: read - d6/d22/f90 zero size 2024-04-23T02:14:09.717 INFO:tasks.workunit.client.1.smithi192.stdout:3/779: write d1/d3/d6/d7/d29/d3b/da9/f48 [85606,40900] 0 2024-04-23T02:14:09.717 INFO:tasks.workunit.client.1.smithi192.stdout:1/686: readlink dc/d16/d19/d74/d77/l7c 0 2024-04-23T02:14:09.719 INFO:tasks.workunit.client.0.smithi071.stdout:9/696: dread d1/d32/d41/db7/f2d [0,4194304] 0 2024-04-23T02:14:09.719 INFO:tasks.workunit.client.0.smithi071.stdout:6/689: mkdir d6/d17/d77/d9f/dd8 0 2024-04-23T02:14:09.720 INFO:tasks.workunit.client.0.smithi071.stdout:9/697: dread - d1/d32/d41/faf zero size 2024-04-23T02:14:09.720 INFO:tasks.workunit.client.1.smithi192.stdout:3/780: write d1/d3/d6/d7/d29/fe2 [4295526,8510] 0 2024-04-23T02:14:09.720 INFO:tasks.workunit.client.1.smithi192.stdout:1/687: mknod dc/d8b/d34/d70/d8c/cdb 0 2024-04-23T02:14:09.721 INFO:tasks.workunit.client.0.smithi071.stdout:3/685: truncate d7/d8/f1a 3670528 0 2024-04-23T02:14:09.721 INFO:tasks.workunit.client.1.smithi192.stdout:3/781: write d1/d3/d6/d7/ff4 [1002258,120469] 0 2024-04-23T02:14:09.722 INFO:tasks.workunit.client.1.smithi192.stdout:1/688: stat dc/d16/d19/d30/dc4 0 2024-04-23T02:14:09.725 INFO:tasks.workunit.client.1.smithi192.stdout:3/782: mknod d1/d3/d17/d1f/d33/c112 0 2024-04-23T02:14:09.726 INFO:tasks.workunit.client.0.smithi071.stdout:3/686: truncate d7/da5/db2/db7/d68/d3b/d4c/da1/fe2 783320 0 2024-04-23T02:14:09.727 INFO:tasks.workunit.client.0.smithi071.stdout:3/687: fdatasync d7/d8/f11 0 2024-04-23T02:14:09.728 INFO:tasks.workunit.client.0.smithi071.stdout:6/690: dread d6/d17/d77/f96 [0,4194304] 0 2024-04-23T02:14:09.734 INFO:tasks.workunit.client.0.smithi071.stdout:3/688: rename d7/da5/db2/db7/d68/c34 to d7/da5/db5/dec/cf2 0 2024-04-23T02:14:09.734 INFO:tasks.workunit.client.0.smithi071.stdout:6/691: creat d6/d17/d77/d9f/dad/d3f/fd9 x:0 0 0 2024-04-23T02:14:09.734 INFO:tasks.workunit.client.0.smithi071.stdout:6/692: write d6/d17/d77/f84 [8191230,1776] 0 2024-04-23T02:14:09.736 INFO:tasks.workunit.client.0.smithi071.stdout:3/689: fdatasync d7/d22/d23/d26/d7e/fd9 0 2024-04-23T02:14:09.737 INFO:tasks.workunit.client.0.smithi071.stdout:3/690: stat d7/d1d/led 0 2024-04-23T02:14:09.738 INFO:tasks.workunit.client.0.smithi071.stdout:6/693: fdatasync d6/d17/d77/f8c 0 2024-04-23T02:14:09.742 INFO:tasks.workunit.client.0.smithi071.stdout:9/698: truncate d1/d32/d41/db7/d5a/dc6/f7a 3316983 0 2024-04-23T02:14:09.743 INFO:tasks.workunit.client.0.smithi071.stdout:6/694: symlink d6/d17/d77/d9f/dad/dd1/lda 0 2024-04-23T02:14:09.743 INFO:tasks.workunit.client.0.smithi071.stdout:9/699: read d1/d32/dc5/dd1/f9d [2327300,51309] 0 2024-04-23T02:14:09.745 INFO:tasks.workunit.client.0.smithi071.stdout:9/700: write d1/f28 [1035848,113999] 0 2024-04-23T02:14:09.746 INFO:tasks.workunit.client.0.smithi071.stdout:9/701: dread - d1/d32/d41/db7/d39/fe9 zero size 2024-04-23T02:14:09.747 INFO:tasks.workunit.client.0.smithi071.stdout:6/695: creat d6/d22/fdb x:0 0 0 2024-04-23T02:14:09.749 INFO:tasks.workunit.client.0.smithi071.stdout:3/691: getdents d7/d22/d23 0 2024-04-23T02:14:09.751 INFO:tasks.workunit.client.0.smithi071.stdout:9/702: mkdir d1/d32/d3a/d77/df8 0 2024-04-23T02:14:09.751 INFO:tasks.workunit.client.0.smithi071.stdout:3/692: mkdir d7/da5/db2/db7/d68/dae/df3 0 2024-04-23T02:14:09.755 INFO:tasks.workunit.client.0.smithi071.stdout:3/693: rename d7/d8/d42/d80 to d7/d22/d23/dba/df4 0 2024-04-23T02:14:09.760 INFO:tasks.workunit.client.0.smithi071.stdout:9/703: write d1/d32/d41/db7/d5a/f5e [2617948,75507] 0 2024-04-23T02:14:09.760 INFO:tasks.workunit.client.0.smithi071.stdout:8/748: sync 2024-04-23T02:14:09.761 INFO:tasks.workunit.client.0.smithi071.stdout:4/793: sync 2024-04-23T02:14:09.761 INFO:tasks.workunit.client.0.smithi071.stdout:3/694: dwrite d7/d8/f4e [0,4194304] 0 2024-04-23T02:14:09.765 INFO:tasks.workunit.client.0.smithi071.stdout:4/794: read - d2/d2a/d2d/d7b/f8c zero size 2024-04-23T02:14:09.772 INFO:tasks.workunit.client.0.smithi071.stdout:8/749: write f9 [4589414,50085] 0 2024-04-23T02:14:09.774 INFO:tasks.workunit.client.0.smithi071.stdout:8/750: chown dc/d15/da0/d32/c7c 1 1 2024-04-23T02:14:09.774 INFO:tasks.workunit.client.0.smithi071.stdout:4/795: write d2/d2a/d2d/d39/fa6 [174090,656] 0 2024-04-23T02:14:09.775 INFO:tasks.workunit.client.0.smithi071.stdout:3/695: link d7/d8/fc d7/d22/d23/dba/df4/dca/d6f/ff5 0 2024-04-23T02:14:09.776 INFO:tasks.workunit.client.0.smithi071.stdout:9/704: dread d1/d32/d41/db7/d5a/dc6/f7a [0,4194304] 0 2024-04-23T02:14:09.781 INFO:tasks.workunit.client.0.smithi071.stdout:9/705: unlink d1/d32/d41/f4c 0 2024-04-23T02:14:09.783 INFO:tasks.workunit.client.0.smithi071.stdout:4/796: link d2/d2a/daa/fe0 d2/d2a/d73/f104 0 2024-04-23T02:14:09.784 INFO:tasks.workunit.client.0.smithi071.stdout:8/751: dwrite dc/d15/d2d/f3f [0,4194304] 0 2024-04-23T02:14:09.784 INFO:tasks.workunit.client.0.smithi071.stdout:3/696: dwrite d7/da5/db2/db7/d68/d3b/d4c/f63 [0,4194304] 0 2024-04-23T02:14:09.786 INFO:tasks.workunit.client.0.smithi071.stdout:8/752: write dc/d15/d2d/f3f [3176138,22594] 0 2024-04-23T02:14:09.791 INFO:tasks.workunit.client.0.smithi071.stdout:8/753: link dc/d15/d2d/ff2 dc/d15/d50/d5f/ff3 0 2024-04-23T02:14:09.793 INFO:tasks.workunit.client.0.smithi071.stdout:8/754: mknod dc/d3a/da4/db0/cf4 0 2024-04-23T02:14:09.795 INFO:tasks.workunit.client.0.smithi071.stdout:8/755: creat dc/d15/da0/d8d/d9a/dad/ff5 x:0 0 0 2024-04-23T02:14:09.797 INFO:tasks.workunit.client.0.smithi071.stdout:8/756: symlink dc/d15/da0/d8d/d9a/dad/lf6 0 2024-04-23T02:14:09.812 INFO:tasks.workunit.client.0.smithi071.stdout:8/757: dread dc/d15/f6b [0,4194304] 0 2024-04-23T02:14:09.813 INFO:tasks.workunit.client.0.smithi071.stdout:8/758: fsync dc/d15/da0/d8d/f92 0 2024-04-23T02:14:09.815 INFO:tasks.workunit.client.0.smithi071.stdout:8/759: mknod dc/d15/cf7 0 2024-04-23T02:14:09.816 INFO:tasks.workunit.client.0.smithi071.stdout:8/760: mkdir dc/d15/da0/d8d/df8 0 2024-04-23T02:14:09.817 INFO:tasks.workunit.client.0.smithi071.stdout:8/761: write dc/d15/d72/fa8 [62488,76153] 0 2024-04-23T02:14:09.821 INFO:tasks.workunit.client.1.smithi192.stdout:1/689: link fa dc/d16/d95/fdc 0 2024-04-23T02:14:09.824 INFO:tasks.workunit.client.1.smithi192.stdout:3/783: truncate d1/d3/d6/d7/d29/d3b/da9/f70 681430 0 2024-04-23T02:14:09.825 INFO:tasks.workunit.client.1.smithi192.stdout:1/690: dread dc/d8b/d34/faa [0,4194304] 0 2024-04-23T02:14:09.827 INFO:tasks.workunit.client.0.smithi071.stdout:4/797: dread d2/d2a/d2d/d39/d42/d49/d67/d7e/dae/dab/ffc [0,4194304] 0 2024-04-23T02:14:09.828 INFO:tasks.workunit.client.1.smithi192.stdout:1/691: rename dc/l8a to dc/d80/ldd 0 2024-04-23T02:14:09.830 INFO:tasks.workunit.client.1.smithi192.stdout:1/692: mkdir dc/d16/dde 0 2024-04-23T02:14:09.832 INFO:tasks.workunit.client.1.smithi192.stdout:0/932: sync 2024-04-23T02:14:09.838 INFO:tasks.workunit.client.1.smithi192.stdout:1/693: readlink dc/d80/ldd 0 2024-04-23T02:14:09.838 INFO:tasks.workunit.client.1.smithi192.stdout:3/784: write d1/d3/d6/d7/d29/d3b/da9/dbd/fc9 [31195,76170] 0 2024-04-23T02:14:09.839 INFO:tasks.workunit.client.1.smithi192.stdout:1/694: readlink dc/d14/d23/d5e/d6a/l97 0 2024-04-23T02:14:09.840 INFO:tasks.workunit.client.1.smithi192.stdout:1/695: stat dc/d8b/d34/faa 0 2024-04-23T02:14:09.840 INFO:tasks.workunit.client.1.smithi192.stdout:0/933: truncate f5 1120092 0 2024-04-23T02:14:09.841 INFO:tasks.workunit.client.1.smithi192.stdout:3/785: chown d1/d3/d17/d2b/f51 130099166 1 2024-04-23T02:14:09.841 INFO:tasks.workunit.client.1.smithi192.stdout:1/696: chown dc/d14/f21 166337192 1 2024-04-23T02:14:09.843 INFO:tasks.workunit.client.1.smithi192.stdout:3/786: chown d1/d3/d17/d1f/d33/d24/feb 5525614 1 2024-04-23T02:14:09.844 INFO:tasks.workunit.client.1.smithi192.stdout:1/697: mknod dc/d16/d19/d74/d77/cdf 0 2024-04-23T02:14:09.844 INFO:tasks.workunit.client.1.smithi192.stdout:0/934: dread d6/df/d13/d25/d94/dd0/d12c/d67/ffe [0,4194304] 0 2024-04-23T02:14:09.845 INFO:tasks.workunit.client.0.smithi071.stdout:8/762: dwrite dc/d15/d50/fa9 [0,4194304] 0 2024-04-23T02:14:09.848 INFO:tasks.workunit.client.0.smithi071.stdout:8/763: write dc/d3a/f47 [2940540,40446] 0 2024-04-23T02:14:09.849 INFO:tasks.workunit.client.1.smithi192.stdout:3/787: creat d1/d3/d17/d1a/f113 x:0 0 0 2024-04-23T02:14:09.850 INFO:tasks.workunit.client.0.smithi071.stdout:8/764: creat dc/d15/d72/ff9 x:0 0 0 2024-04-23T02:14:09.852 INFO:tasks.workunit.client.1.smithi192.stdout:0/935: rename d6/df/d13/d25/d3a/d93/lbb to d6/df/d13/d25/d94/dd0/d12c/d67/l131 0 2024-04-23T02:14:09.853 INFO:tasks.workunit.client.1.smithi192.stdout:1/698: getdents dc/d8b/d44 0 2024-04-23T02:14:09.854 INFO:tasks.workunit.client.0.smithi071.stdout:8/765: getdents dc/d15/da0/d32/d63 0 2024-04-23T02:14:09.857 INFO:tasks.workunit.client.0.smithi071.stdout:8/766: creat dc/d56/de1/dea/ffa x:0 0 0 2024-04-23T02:14:09.857 INFO:tasks.workunit.client.1.smithi192.stdout:0/936: mknod d6/d4b/c132 0 2024-04-23T02:14:09.858 INFO:tasks.workunit.client.0.smithi071.stdout:3/697: dread d7/d8/d42/fac [0,4194304] 0 2024-04-23T02:14:09.860 INFO:tasks.workunit.client.1.smithi192.stdout:0/937: chown d6/df/d35/d49/fd2 27326633 1 2024-04-23T02:14:09.861 INFO:tasks.workunit.client.1.smithi192.stdout:0/938: dread - d6/df/d13/d25/d94/dd0/f12e zero size 2024-04-23T02:14:09.864 INFO:tasks.workunit.client.0.smithi071.stdout:3/698: getdents d7/da5/db2/db7/d68/d3b/d4c/da1 0 2024-04-23T02:14:09.864 INFO:tasks.workunit.client.0.smithi071.stdout:3/699: chown d7/d10/cdc 1582992 1 2024-04-23T02:14:09.866 INFO:tasks.workunit.client.1.smithi192.stdout:0/939: link d6/df/d13/d25/d94/d130/f10b d6/df/d56/d85/d71/f133 0 2024-04-23T02:14:09.869 INFO:tasks.workunit.client.1.smithi192.stdout:0/940: mkdir d6/df/d56/d134 0 2024-04-23T02:14:09.870 INFO:tasks.workunit.client.1.smithi192.stdout:0/941: dread - d6/df/d13/d25/d94/dd0/fd4 zero size 2024-04-23T02:14:09.871 INFO:tasks.workunit.client.1.smithi192.stdout:0/942: dread - d6/df/d56/d85/d45/ff2 zero size 2024-04-23T02:14:09.873 INFO:tasks.workunit.client.0.smithi071.stdout:4/798: dread d2/d2a/da2/dd1/dde/d57/f72 [0,4194304] 0 2024-04-23T02:14:09.875 INFO:tasks.workunit.client.0.smithi071.stdout:3/700: dwrite d7/d22/d23/dba/df4/dca/d6f/ff5 [0,4194304] 0 2024-04-23T02:14:09.877 INFO:tasks.workunit.client.1.smithi192.stdout:0/943: rename d6/df/d13/d25/d86/da5/faf to d6/df/d56/d85/da4/f135 0 2024-04-23T02:14:09.879 INFO:tasks.workunit.client.1.smithi192.stdout:3/788: dread d1/d3/d17/d1f/d33/d24/feb [0,4194304] 0 2024-04-23T02:14:09.881 INFO:tasks.workunit.client.1.smithi192.stdout:0/944: unlink d6/df/d13/d25/c76 0 2024-04-23T02:14:09.881 INFO:tasks.workunit.client.0.smithi071.stdout:4/799: truncate d2/d2a/d2d/d39/fd8 3274167 0 2024-04-23T02:14:09.884 INFO:tasks.workunit.client.1.smithi192.stdout:3/789: unlink d1/d3/d6/d7/d29/d3b/da9/dbd/lf2 0 2024-04-23T02:14:09.885 INFO:tasks.workunit.client.1.smithi192.stdout:1/699: dread dc/d14/d23/d5e/dab/fb9 [0,4194304] 0 2024-04-23T02:14:09.885 INFO:tasks.workunit.client.0.smithi071.stdout:3/701: dwrite d7/d8/f61 [0,4194304] 0 2024-04-23T02:14:09.887 INFO:tasks.workunit.client.1.smithi192.stdout:3/790: dread d1/f56 [0,4194304] 0 2024-04-23T02:14:09.892 INFO:tasks.workunit.client.0.smithi071.stdout:3/702: dwrite f0 [0,4194304] 0 2024-04-23T02:14:09.897 INFO:tasks.workunit.client.0.smithi071.stdout:4/800: write d2/d2a/d2d/d39/d42/d49/d67/d7e/d82/fa3 [530233,92259] 0 2024-04-23T02:14:09.897 INFO:tasks.workunit.client.1.smithi192.stdout:1/700: dwrite dc/d16/f64 [0,4194304] 0 2024-04-23T02:14:09.898 INFO:tasks.workunit.client.0.smithi071.stdout:4/801: stat d2/d2a/da2/dd1/dde/d23/d29/f9e 0 2024-04-23T02:14:09.903 INFO:tasks.workunit.client.1.smithi192.stdout:1/701: unlink dc/d16/d19/d74/d77/cdf 0 2024-04-23T02:14:09.904 INFO:tasks.workunit.client.0.smithi071.stdout:8/767: dread dc/d15/da0/f22 [0,4194304] 0 2024-04-23T02:14:09.904 INFO:tasks.workunit.client.1.smithi192.stdout:1/702: chown dc/d14/c43 219 1 2024-04-23T02:14:09.907 INFO:tasks.workunit.client.0.smithi071.stdout:8/768: creat dc/d56/de1/ffb x:0 0 0 2024-04-23T02:14:09.908 INFO:tasks.workunit.client.0.smithi071.stdout:8/769: chown dc/d15/da0/d8d/faf 7029 1 2024-04-23T02:14:09.908 INFO:tasks.workunit.client.1.smithi192.stdout:1/703: mkdir dc/d14/d23/de0 0 2024-04-23T02:14:09.909 INFO:tasks.workunit.client.0.smithi071.stdout:8/770: stat dc/d15/fe8 0 2024-04-23T02:14:09.909 INFO:tasks.workunit.client.1.smithi192.stdout:0/945: dwrite d6/df/d13/d25/d86/fe4 [0,4194304] 0 2024-04-23T02:14:09.913 INFO:tasks.workunit.client.0.smithi071.stdout:8/771: mknod dc/d15/da0/d32/cfc 0 2024-04-23T02:14:09.915 INFO:tasks.workunit.client.1.smithi192.stdout:1/704: rename dc/d14/d15/d35/cbe to dc/d14/d23/d5e/dab/ce1 0 2024-04-23T02:14:09.915 INFO:tasks.workunit.client.0.smithi071.stdout:3/703: dread d7/da5/db2/db7/f40 [8388608,4194304] 0 2024-04-23T02:14:09.917 INFO:tasks.workunit.client.0.smithi071.stdout:3/704: mkdir d7/d8/dea/df6 0 2024-04-23T02:14:09.932 INFO:tasks.workunit.client.1.smithi192.stdout:0/946: truncate d6/df/d13/d25/d3a/f4f 7835352 0 2024-04-23T02:14:09.935 INFO:tasks.workunit.client.0.smithi071.stdout:4/802: write d2/d2a/d2d/d39/fd8 [959382,60469] 0 2024-04-23T02:14:09.935 INFO:tasks.workunit.client.1.smithi192.stdout:0/947: creat d6/df/d13/d25/d107/d12f/f136 x:0 0 0 2024-04-23T02:14:09.938 INFO:tasks.workunit.client.0.smithi071.stdout:4/803: truncate d2/d2a/da2/dd1/dde/d23/d10/fec 243797 0 2024-04-23T02:14:09.938 INFO:tasks.workunit.client.0.smithi071.stdout:8/772: dread dc/f28 [4194304,4194304] 0 2024-04-23T02:14:09.938 INFO:tasks.workunit.client.1.smithi192.stdout:1/705: dwrite dc/d8b/fad [0,4194304] 0 2024-04-23T02:14:09.940 INFO:tasks.workunit.client.0.smithi071.stdout:4/804: mknod d2/d2a/da2/dd1/c105 0 2024-04-23T02:14:09.941 INFO:tasks.workunit.client.0.smithi071.stdout:8/773: creat dc/d15/da0/d32/d63/dd4/ffd x:0 0 0 2024-04-23T02:14:09.943 INFO:tasks.workunit.client.1.smithi192.stdout:0/948: rename d6/df/d13/d25/d94/dd0/d12c to d6/df/d13/d6b/d73/d125/d137 0 2024-04-23T02:14:09.943 INFO:tasks.workunit.client.0.smithi071.stdout:8/774: mknod dc/d15/da0/d8d/d9a/dad/cfe 0 2024-04-23T02:14:09.945 INFO:tasks.workunit.client.1.smithi192.stdout:1/706: mknod dc/d16/d19/d30/ce2 0 2024-04-23T02:14:09.946 INFO:tasks.workunit.client.1.smithi192.stdout:0/949: chown d6/df/d13/d6b/d73/d125/d137/f90 40069 1 2024-04-23T02:14:09.946 INFO:tasks.workunit.client.1.smithi192.stdout:1/707: fdatasync dc/d14/f21 0 2024-04-23T02:14:09.948 INFO:tasks.workunit.client.0.smithi071.stdout:4/805: dwrite d2/d2a/d2d/d39/d42/d49/d67/d3e/f76 [0,4194304] 0 2024-04-23T02:14:09.950 INFO:tasks.workunit.client.1.smithi192.stdout:0/950: truncate d6/df/d56/d85/da4/f11f 1070548 0 2024-04-23T02:14:09.951 INFO:tasks.workunit.client.0.smithi071.stdout:4/806: creat d2/d2a/d2d/d39/f106 x:0 0 0 2024-04-23T02:14:09.952 INFO:tasks.workunit.client.0.smithi071.stdout:0/730: sync 2024-04-23T02:14:09.953 INFO:tasks.workunit.client.1.smithi192.stdout:1/708: getdents dc/d8b/d34/d70/d8c/da3 0 2024-04-23T02:14:09.955 INFO:tasks.workunit.client.0.smithi071.stdout:4/807: creat d2/d2a/da2/dd1/dde/d23/d4f/f107 x:0 0 0 2024-04-23T02:14:09.957 INFO:tasks.workunit.client.1.smithi192.stdout:0/951: creat d6/df/d13/d6b/dc1/dd7/f138 x:0 0 0 2024-04-23T02:14:09.958 INFO:tasks.workunit.client.1.smithi192.stdout:1/709: mkdir dc/d16/d19/d30/dc4/de3 0 2024-04-23T02:14:09.961 INFO:tasks.workunit.client.1.smithi192.stdout:0/952: symlink d6/df/d56/d85/da4/l139 0 2024-04-23T02:14:09.963 INFO:tasks.workunit.client.0.smithi071.stdout:0/731: truncate d0/d3/de/d10/d14/f64 2530608 0 2024-04-23T02:14:09.965 INFO:tasks.workunit.client.1.smithi192.stdout:0/953: rename d6/df/d56/d85/da4/f135 to d6/df/d13/d25/d107/d12f/f13a 0 2024-04-23T02:14:09.965 INFO:tasks.workunit.client.0.smithi071.stdout:0/732: mkdir d0/d3/de/d10/dfc 0 2024-04-23T02:14:09.966 INFO:tasks.workunit.client.1.smithi192.stdout:0/954: read - d6/df/d56/ff9 zero size 2024-04-23T02:14:09.968 INFO:tasks.workunit.client.1.smithi192.stdout:0/955: write d6/f10 [3529625,68556] 0 2024-04-23T02:14:09.969 INFO:tasks.workunit.client.0.smithi071.stdout:0/733: rename d0/d3/de/d10/d14/d32/d37/c61 to d0/d76/cfd 0 2024-04-23T02:14:09.972 INFO:tasks.workunit.client.1.smithi192.stdout:0/956: rename d6/df/d56/l9a to d6/df/d13/d25/d94/dd0/l13b 0 2024-04-23T02:14:09.977 INFO:tasks.workunit.client.1.smithi192.stdout:0/957: link d6/df/d13/d6b/d73/c118 d6/df/d13/d25/d86/da5/c13c 0 2024-04-23T02:14:09.978 INFO:tasks.workunit.client.0.smithi071.stdout:8/775: dread dc/d15/da0/d55/f80 [0,4194304] 0 2024-04-23T02:14:09.981 INFO:tasks.workunit.client.0.smithi071.stdout:0/734: dwrite d0/d3/de/d10/d14/d32/dac/fae [0,4194304] 0 2024-04-23T02:14:09.984 INFO:tasks.workunit.client.1.smithi192.stdout:1/710: dread dc/d16/d19/d74/d77/f91 [0,4194304] 0 2024-04-23T02:14:09.985 INFO:tasks.workunit.client.0.smithi071.stdout:8/776: link dc/d15/da0/d32/d63/c8a dc/d15/d5d/cff 0 2024-04-23T02:14:09.986 INFO:tasks.workunit.client.0.smithi071.stdout:0/735: symlink d0/d3/de/d10/d14/d32/lfe 0 2024-04-23T02:14:09.986 INFO:tasks.workunit.client.0.smithi071.stdout:8/777: dread - dc/d15/d72/d95/fbc zero size 2024-04-23T02:14:09.987 INFO:tasks.workunit.client.0.smithi071.stdout:8/778: write dc/d15/f86 [4045219,48159] 0 2024-04-23T02:14:09.987 INFO:tasks.workunit.client.1.smithi192.stdout:1/711: mkdir dc/d8b/d34/d70/d8c/da3/de4 0 2024-04-23T02:14:09.990 INFO:tasks.workunit.client.0.smithi071.stdout:0/736: symlink d0/d3/de/d10/d14/d32/d44/d31/d52/lff 0 2024-04-23T02:14:09.991 INFO:tasks.workunit.client.0.smithi071.stdout:0/737: chown d0/d63/f66 57 1 2024-04-23T02:14:09.991 INFO:tasks.workunit.client.0.smithi071.stdout:8/779: mknod dc/d56/d64/dde/c100 0 2024-04-23T02:14:09.992 INFO:tasks.workunit.client.0.smithi071.stdout:8/780: chown dc/d56/c5b 5978 1 2024-04-23T02:14:09.995 INFO:tasks.workunit.client.0.smithi071.stdout:8/781: symlink dc/d56/d64/dde/l101 0 2024-04-23T02:14:09.996 INFO:tasks.workunit.client.0.smithi071.stdout:8/782: write dc/d15/d2d/f3f [3521367,29113] 0 2024-04-23T02:14:09.997 INFO:tasks.workunit.client.1.smithi192.stdout:0/958: dread d6/fb5 [0,4194304] 0 2024-04-23T02:14:09.998 INFO:tasks.workunit.client.0.smithi071.stdout:8/783: mkdir dc/d15/d50/d5f/d102 0 2024-04-23T02:14:09.999 INFO:tasks.workunit.client.1.smithi192.stdout:0/959: mknod d6/df/d13/d6b/d73/da9/c13d 0 2024-04-23T02:14:10.000 INFO:tasks.workunit.client.1.smithi192.stdout:1/712: truncate dc/d16/f64 2062627 0 2024-04-23T02:14:10.001 INFO:tasks.workunit.client.1.smithi192.stdout:0/960: write d6/df/d13/d25/f10e [934517,50821] 0 2024-04-23T02:14:10.001 INFO:tasks.workunit.client.1.smithi192.stdout:1/713: dread - dc/d16/db0/fd8 zero size 2024-04-23T02:14:10.002 INFO:tasks.workunit.client.1.smithi192.stdout:5/833: sync 2024-04-23T02:14:10.002 INFO:tasks.workunit.client.1.smithi192.stdout:9/708: sync 2024-04-23T02:14:10.002 INFO:tasks.workunit.client.1.smithi192.stdout:4/920: sync 2024-04-23T02:14:10.003 INFO:tasks.workunit.client.1.smithi192.stdout:1/714: rmdir dc 39 2024-04-23T02:14:10.003 INFO:tasks.workunit.client.1.smithi192.stdout:4/921: readlink de/l45 0 2024-04-23T02:14:10.011 INFO:tasks.workunit.client.1.smithi192.stdout:4/922: rmdir de/d15/d53/d81/dac 39 2024-04-23T02:14:10.012 INFO:tasks.workunit.client.1.smithi192.stdout:0/961: link d6/df/l1d d6/df/d13/d25/d3a/da6/d116/l13e 0 2024-04-23T02:14:10.012 INFO:tasks.workunit.client.1.smithi192.stdout:5/834: dwrite d2/db/d2b/df7/f111 [0,4194304] 0 2024-04-23T02:14:10.017 INFO:tasks.workunit.client.0.smithi071.stdout:8/784: dread dc/d15/da0/d30/fa6 [0,4194304] 0 2024-04-23T02:14:10.017 INFO:tasks.workunit.client.1.smithi192.stdout:1/715: link dc/l1c dc/d8b/d34/d70/d8c/le5 0 2024-04-23T02:14:10.017 INFO:tasks.workunit.client.0.smithi071.stdout:0/738: dread d0/d3/de/f1d [4194304,4194304] 0 2024-04-23T02:14:10.018 INFO:tasks.workunit.client.0.smithi071.stdout:8/785: read dc/d15/da0/f22 [3310210,78773] 0 2024-04-23T02:14:10.020 INFO:tasks.workunit.client.1.smithi192.stdout:0/962: creat d6/df/d13/d25/d94/dd0/de6/f13f x:0 0 0 2024-04-23T02:14:10.020 INFO:tasks.workunit.client.1.smithi192.stdout:9/709: write d0/d9/d44/d51/d5b/d61/fb9 [111689,23648] 0 2024-04-23T02:14:10.022 INFO:tasks.workunit.client.1.smithi192.stdout:1/716: truncate dc/d14/f9d 1530551 0 2024-04-23T02:14:10.023 INFO:tasks.workunit.client.1.smithi192.stdout:1/717: stat dc/d14/f9d 0 2024-04-23T02:14:10.024 INFO:tasks.workunit.client.0.smithi071.stdout:8/786: link dc/d15/da0/d55/lb6 dc/d15/d72/d95/l103 0 2024-04-23T02:14:10.024 INFO:tasks.workunit.client.1.smithi192.stdout:5/835: rename d2/db/d16/d48/d4f/dd1/le9 to d2/db/d2b/d52/dc2/d49/l112 0 2024-04-23T02:14:10.026 INFO:tasks.workunit.client.0.smithi071.stdout:0/739: dwrite d0/d3/de/d48/d7e/d8b/ffa [0,4194304] 0 2024-04-23T02:14:10.026 INFO:tasks.workunit.client.0.smithi071.stdout:8/787: getdents dc/d3a/da4/dd3 0 2024-04-23T02:14:10.027 INFO:tasks.workunit.client.1.smithi192.stdout:9/710: mkdir d0/d9/d44/d74/de7 0 2024-04-23T02:14:10.029 INFO:tasks.workunit.client.0.smithi071.stdout:8/788: rmdir dc/d15/da0/d8d/d9a 39 2024-04-23T02:14:10.029 INFO:tasks.workunit.client.1.smithi192.stdout:0/963: mknod d6/df/d56/db1/d127/c140 0 2024-04-23T02:14:10.029 INFO:tasks.workunit.client.1.smithi192.stdout:7/865: sync 2024-04-23T02:14:10.030 INFO:tasks.workunit.client.1.smithi192.stdout:6/938: sync 2024-04-23T02:14:10.030 INFO:tasks.workunit.client.1.smithi192.stdout:3/791: sync 2024-04-23T02:14:10.030 INFO:tasks.workunit.client.0.smithi071.stdout:8/789: dread - dc/d56/fe0 zero size 2024-04-23T02:14:10.031 INFO:tasks.workunit.client.1.smithi192.stdout:1/718: read dc/d14/d23/d5e/dab/fd3 [244962,74945] 0 2024-04-23T02:14:10.031 INFO:tasks.workunit.client.1.smithi192.stdout:5/836: mkdir d2/d6/d14/d113 0 2024-04-23T02:14:10.031 INFO:tasks.workunit.client.0.smithi071.stdout:0/740: dread d0/d3/de/d10/d14/d32/d37/fa2 [0,4194304] 0 2024-04-23T02:14:10.032 INFO:tasks.workunit.client.0.smithi071.stdout:8/790: fdatasync dc/d15/da0/d8d/faf 0 2024-04-23T02:14:10.032 INFO:tasks.workunit.client.1.smithi192.stdout:9/711: symlink d0/d9/dc7/le8 0 2024-04-23T02:14:10.033 INFO:tasks.workunit.client.1.smithi192.stdout:1/719: stat dc/d80/d81 0 2024-04-23T02:14:10.035 INFO:tasks.workunit.client.0.smithi071.stdout:0/741: write d0/d3/de/d10/d14/d32/d8a/df0/fe6 [289715,21976] 0 2024-04-23T02:14:10.036 INFO:tasks.workunit.client.1.smithi192.stdout:4/923: write de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/fa8 [4287015,6305] 0 2024-04-23T02:14:10.037 INFO:tasks.workunit.client.0.smithi071.stdout:8/791: unlink dc/d15/d5d/l83 0 2024-04-23T02:14:10.038 INFO:tasks.workunit.client.1.smithi192.stdout:0/964: unlink d6/df/d56/lc4 0 2024-04-23T02:14:10.040 INFO:tasks.workunit.client.0.smithi071.stdout:8/792: read dc/d3a/f3c [978178,59210] 0 2024-04-23T02:14:10.041 INFO:tasks.workunit.client.1.smithi192.stdout:9/712: fsync d0/d9/d44/f5e 0 2024-04-23T02:14:10.042 INFO:tasks.workunit.client.1.smithi192.stdout:1/720: symlink dc/d16/d19/d30/dc4/le6 0 2024-04-23T02:14:10.042 INFO:tasks.workunit.client.0.smithi071.stdout:0/742: link d0/d3/de/d10/d14/d32/d44/d31/d52/lff d0/d3/de/d10/d14/d32/d37/d26/l100 0 2024-04-23T02:14:10.043 INFO:tasks.workunit.client.1.smithi192.stdout:3/792: getdents d1/d3/d17/d1f/d33/dd6 0 2024-04-23T02:14:10.043 INFO:tasks.workunit.client.1.smithi192.stdout:4/924: mknod de/d15/d53/d81/de3/db2/c133 0 2024-04-23T02:14:10.044 INFO:tasks.workunit.client.1.smithi192.stdout:5/837: chown d2/db/d16/d48/d4f/l10e 129534 1 2024-04-23T02:14:10.045 INFO:tasks.workunit.client.1.smithi192.stdout:6/939: dwrite d3/d18/d91/ff6 [0,4194304] 0 2024-04-23T02:14:10.045 INFO:tasks.workunit.client.0.smithi071.stdout:8/793: rename dc/d15/d5d/cbb to dc/d15/da0/d8d/def/c104 0 2024-04-23T02:14:10.047 INFO:tasks.workunit.client.1.smithi192.stdout:7/866: rename d5/d17/d1c/d59/d5a/l8a to d5/d17/d1c/d59/d5a/d70/de9/da6/da9/lfc 0 2024-04-23T02:14:10.048 INFO:tasks.workunit.client.0.smithi071.stdout:8/794: write dc/d15/da0/d8d/d9a/dad/fb2 [4252476,17515] 0 2024-04-23T02:14:10.050 INFO:tasks.workunit.client.0.smithi071.stdout:0/743: write d0/d3/f7a [541921,24457] 0 2024-04-23T02:14:10.051 INFO:tasks.workunit.client.1.smithi192.stdout:5/838: creat d2/db8/f114 x:0 0 0 2024-04-23T02:14:10.052 INFO:tasks.workunit.client.1.smithi192.stdout:7/867: chown d5/d43/l85 955894 1 2024-04-23T02:14:10.052 INFO:tasks.workunit.client.1.smithi192.stdout:9/713: symlink d0/d9/d44/d4f/le9 0 2024-04-23T02:14:10.052 INFO:tasks.workunit.client.0.smithi071.stdout:8/795: mkdir dc/d15/d5d/dc6/d105 0 2024-04-23T02:14:10.053 INFO:tasks.workunit.client.0.smithi071.stdout:0/744: creat d0/d63/f101 x:0 0 0 2024-04-23T02:14:10.058 INFO:tasks.workunit.client.0.smithi071.stdout:0/745: creat d0/d3/de/d10/d14/d32/d44/d31/d52/f102 x:0 0 0 2024-04-23T02:14:10.059 INFO:tasks.workunit.client.0.smithi071.stdout:0/746: fsync d0/d3/de/d10/d14/d32/d8a/df0/fed 0 2024-04-23T02:14:10.062 INFO:tasks.workunit.client.1.smithi192.stdout:9/714: creat d0/d9/d44/d58/d89/fea x:0 0 0 2024-04-23T02:14:10.062 INFO:tasks.workunit.client.1.smithi192.stdout:5/839: creat d2/db/d2b/df7/f115 x:0 0 0 2024-04-23T02:14:10.063 INFO:tasks.workunit.client.0.smithi071.stdout:8/796: getdents dc/d15/d5d/dc6 0 2024-04-23T02:14:10.063 INFO:tasks.workunit.client.1.smithi192.stdout:1/721: rename dc/d16/d19/f9f to dc/d8b/fe7 0 2024-04-23T02:14:10.063 INFO:tasks.workunit.client.1.smithi192.stdout:3/793: dwrite d1/d3/d17/d1a/d7c/dab/d6c/d84/d5d/f10b [0,4194304] 0 2024-04-23T02:14:10.066 INFO:tasks.workunit.client.0.smithi071.stdout:0/747: link d0/d3/de/d10/d14/d32/d37/laa d0/d3/de/d10/d14/d32/d44/d87/l103 0 2024-04-23T02:14:10.066 INFO:tasks.workunit.client.1.smithi192.stdout:9/715: chown d0/d9/db3/d5f/dc9 53039373 1 2024-04-23T02:14:10.067 INFO:tasks.workunit.client.0.smithi071.stdout:8/797: truncate dc/f20 1226030 0 2024-04-23T02:14:10.069 INFO:tasks.workunit.client.1.smithi192.stdout:7/868: dwrite d5/d17/d1c/d59/fdd [0,4194304] 0 2024-04-23T02:14:10.069 INFO:tasks.workunit.client.0.smithi071.stdout:0/748: mknod d0/d3/de/d10/d14/d32/d44/c104 0 2024-04-23T02:14:10.070 INFO:tasks.workunit.client.1.smithi192.stdout:4/925: rename de/l6b to de/d15/d53/d81/de3/d74/d92/da0/dca/d49/de6/l134 0 2024-04-23T02:14:10.074 INFO:tasks.workunit.client.1.smithi192.stdout:6/940: write d3/d11/f6b [855401,77454] 0 2024-04-23T02:14:10.077 INFO:tasks.workunit.client.0.smithi071.stdout:0/749: mknod d0/d3/de/d10/d14/d32/d44/d31/d52/c105 0 2024-04-23T02:14:10.078 INFO:tasks.workunit.client.1.smithi192.stdout:3/794: rename d1/d3/d6/d7/cb8 to d1/d3/d17/d1a/d7c/dab/d6c/c114 0 2024-04-23T02:14:10.083 INFO:tasks.workunit.client.1.smithi192.stdout:0/965: dwrite d6/df/d56/d85/d45/fcb [0,4194304] 0 2024-04-23T02:14:10.085 INFO:tasks.workunit.client.1.smithi192.stdout:7/869: mkdir d5/d17/d1c/d59/d5a/d41/dfd 0 2024-04-23T02:14:10.086 INFO:tasks.workunit.client.1.smithi192.stdout:6/941: write d3/d15/d1b/f8f [250902,16495] 0 2024-04-23T02:14:10.086 INFO:tasks.workunit.client.1.smithi192.stdout:4/926: read de/d15/d53/d81/de3/d74/d92/da0/dca/f57 [1335082,43099] 0 2024-04-23T02:14:10.096 INFO:tasks.workunit.client.1.smithi192.stdout:1/722: dread dc/d80/fb4 [0,4194304] 0 2024-04-23T02:14:10.097 INFO:tasks.workunit.client.1.smithi192.stdout:3/795: dwrite d1/d3/d17/d1a/d7c/fb5 [0,4194304] 0 2024-04-23T02:14:10.099 INFO:tasks.workunit.client.1.smithi192.stdout:7/870: chown d5/d17/d1c/d59/d5a/d41/c8c 2188 1 2024-04-23T02:14:10.099 INFO:tasks.workunit.client.1.smithi192.stdout:6/942: truncate d3/d18/f9f 1307821 0 2024-04-23T02:14:10.100 INFO:tasks.workunit.client.1.smithi192.stdout:0/966: dwrite d6/df/f83 [0,4194304] 0 2024-04-23T02:14:10.100 INFO:tasks.workunit.client.1.smithi192.stdout:5/840: truncate d2/d6/d2d/d41/d43/dbc/fc1 3586378 0 2024-04-23T02:14:10.103 INFO:tasks.workunit.client.0.smithi071.stdout:0/750: dwrite d0/d3/de/d10/d14/d32/d8a/fb7 [0,4194304] 0 2024-04-23T02:14:10.103 INFO:tasks.workunit.client.1.smithi192.stdout:4/927: creat de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/de7/f135 x:0 0 0 2024-04-23T02:14:10.106 INFO:tasks.workunit.client.1.smithi192.stdout:3/796: rmdir d1/d3/d17/d1f 39 2024-04-23T02:14:10.108 INFO:tasks.workunit.client.1.smithi192.stdout:5/841: stat d2/db/d2b/d52/dc2/f81 0 2024-04-23T02:14:10.109 INFO:tasks.workunit.client.1.smithi192.stdout:6/943: readlink d3/d15/d1b/l10e 0 2024-04-23T02:14:10.110 INFO:tasks.workunit.client.0.smithi071.stdout:0/751: mkdir d0/d3/de/d10/db9/d106 0 2024-04-23T02:14:10.111 INFO:tasks.workunit.client.1.smithi192.stdout:1/723: mkdir dc/d8b/d34/d70/d8c/da3/de4/de8 0 2024-04-23T02:14:10.112 INFO:tasks.workunit.client.1.smithi192.stdout:0/967: chown d6/df/d47/d55/dae/lb2 6023 1 2024-04-23T02:14:10.113 INFO:tasks.workunit.client.0.smithi071.stdout:0/752: truncate d0/d3/de/d10/d14/d32/d44/d87/f96 81783 0 2024-04-23T02:14:10.113 INFO:tasks.workunit.client.0.smithi071.stdout:0/753: stat d0/d3/de/d10/d14/d32/dac 0 2024-04-23T02:14:10.114 INFO:tasks.workunit.client.1.smithi192.stdout:9/716: dwrite d0/d9/d44/d4f/f8e [0,4194304] 0 2024-04-23T02:14:10.115 INFO:tasks.workunit.client.1.smithi192.stdout:5/842: symlink d2/db/d2b/d52/dc2/d49/l116 0 2024-04-23T02:14:10.115 INFO:tasks.workunit.client.1.smithi192.stdout:6/944: write d3/d28/f56 [366905,8317] 0 2024-04-23T02:14:10.117 INFO:tasks.workunit.client.1.smithi192.stdout:7/871: link d5/d43/d4c/d4e/c89 d5/d43/d4c/d5c/cfe 0 2024-04-23T02:14:10.121 INFO:tasks.workunit.client.1.smithi192.stdout:1/724: truncate dc/d14/d15/fbc 421969 0 2024-04-23T02:14:10.122 INFO:tasks.workunit.client.1.smithi192.stdout:9/717: creat d0/d9/d44/d58/dd9/feb x:0 0 0 2024-04-23T02:14:10.122 INFO:tasks.workunit.client.1.smithi192.stdout:7/872: chown d5/d17/d1c/d59/d5a/d70/de9/da6/da9 0 1 2024-04-23T02:14:10.123 INFO:tasks.workunit.client.1.smithi192.stdout:3/797: creat d1/d3/d17/d1f/dc2/f115 x:0 0 0 2024-04-23T02:14:10.123 INFO:tasks.workunit.client.1.smithi192.stdout:6/945: dread - d3/f49 zero size 2024-04-23T02:14:10.124 INFO:tasks.workunit.client.1.smithi192.stdout:7/873: stat d5/d17/d1c/d59/d5a/d41/d50/l38 0 2024-04-23T02:14:10.127 INFO:tasks.workunit.client.1.smithi192.stdout:9/718: creat d0/d9/d44/d51/d5b/dd3/fec x:0 0 0 2024-04-23T02:14:10.127 INFO:tasks.workunit.client.1.smithi192.stdout:7/874: chown d5/d43/d4c/d5c/d97/fea 120744 1 2024-04-23T02:14:10.129 INFO:tasks.workunit.client.1.smithi192.stdout:4/928: fsync de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/fdb 0 2024-04-23T02:14:10.130 INFO:tasks.workunit.client.1.smithi192.stdout:1/725: mkdir dc/d16/dde/de9 0 2024-04-23T02:14:10.132 INFO:tasks.workunit.client.1.smithi192.stdout:6/946: mkdir d3/d28/d79/da0/dc2/d101/d80/d8e/d128/d134 0 2024-04-23T02:14:10.138 INFO:tasks.workunit.client.0.smithi071.stdout:0/754: write d0/d3/d95/fb4 [894924,37844] 0 2024-04-23T02:14:10.146 INFO:tasks.workunit.client.0.smithi071.stdout:0/755: dwrite d0/d3/de/d10/d14/d32/d8a/df0/f5e [0,4194304] 0 2024-04-23T02:14:10.147 INFO:tasks.workunit.client.0.smithi071.stdout:0/756: fsync d0/d3/d95/fb4 0 2024-04-23T02:14:10.165 INFO:tasks.workunit.client.0.smithi071.stdout:0/757: dread d0/d3/f2d [0,4194304] 0 2024-04-23T02:14:10.179 INFO:tasks.workunit.client.0.smithi071.stdout:7/930: sync 2024-04-23T02:14:10.179 INFO:tasks.workunit.client.0.smithi071.stdout:1/643: sync 2024-04-23T02:14:10.179 INFO:tasks.workunit.client.0.smithi071.stdout:5/828: sync 2024-04-23T02:14:10.179 INFO:tasks.workunit.client.0.smithi071.stdout:2/669: sync 2024-04-23T02:14:10.180 INFO:tasks.workunit.client.0.smithi071.stdout:7/931: stat d3/d17/d1c/l32 0 2024-04-23T02:14:10.185 INFO:tasks.workunit.client.0.smithi071.stdout:5/829: dread d3/d1d/f3b [0,4194304] 0 2024-04-23T02:14:10.186 INFO:tasks.workunit.client.0.smithi071.stdout:0/758: rmdir d0/d3/de/d10/d14/d32/d44/d31 39 2024-04-23T02:14:10.188 INFO:tasks.workunit.client.0.smithi071.stdout:1/644: readlink db/l4c 0 2024-04-23T02:14:10.188 INFO:tasks.workunit.client.1.smithi192.stdout:4/929: fdatasync de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d73/fe8 0 2024-04-23T02:14:10.188 INFO:tasks.workunit.client.0.smithi071.stdout:7/932: dwrite d3/d51/df5/d10d/d127/f133 [0,4194304] 0 2024-04-23T02:14:10.189 INFO:tasks.workunit.client.0.smithi071.stdout:5/830: chown d3/d1d/d43/f69 5061 1 2024-04-23T02:14:10.190 INFO:tasks.workunit.client.0.smithi071.stdout:1/645: chown db/d34/f61 268321306 1 2024-04-23T02:14:10.191 INFO:tasks.workunit.client.1.smithi192.stdout:7/875: symlink d5/d17/d1c/d59/d5a/d70/de9/da6/da9/db9/dc0/def/lff 0 2024-04-23T02:14:10.191 INFO:tasks.workunit.client.0.smithi071.stdout:7/933: truncate d3/d17/d41/d3a/d137/dea/fe6 1043486 0 2024-04-23T02:14:10.193 INFO:tasks.workunit.client.0.smithi071.stdout:1/646: dread - db/df/d13/d20/d75/fc4 zero size 2024-04-23T02:14:10.194 INFO:tasks.workunit.client.0.smithi071.stdout:0/759: creat d0/d3/de/d48/d7e/d8b/de3/f107 x:0 0 0 2024-04-23T02:14:10.197 INFO:tasks.workunit.client.0.smithi071.stdout:2/670: creat d0/d2/d45/d7d/fdb x:0 0 0 2024-04-23T02:14:10.198 INFO:tasks.workunit.client.1.smithi192.stdout:3/798: getdents d1/d3/d17/d1f/d33 0 2024-04-23T02:14:10.200 INFO:tasks.workunit.client.0.smithi071.stdout:7/934: mkdir d3/d17/d41/d34/d147 0 2024-04-23T02:14:10.201 INFO:tasks.workunit.client.1.smithi192.stdout:7/876: truncate d5/d17/d1c/d59/d5a/d41/d50/fba 456377 0 2024-04-23T02:14:10.201 INFO:tasks.workunit.client.1.smithi192.stdout:4/930: mkdir de/d15/dbb/d136 0 2024-04-23T02:14:10.201 INFO:tasks.workunit.client.1.smithi192.stdout:1/726: getdents dc/d8b/d44 0 2024-04-23T02:14:10.204 INFO:tasks.workunit.client.0.smithi071.stdout:2/671: creat d0/d2/d26/d3a/dc3/d36/d5c/d85/fdc x:0 0 0 2024-04-23T02:14:10.204 INFO:tasks.workunit.client.0.smithi071.stdout:2/672: readlink d0/daa/lbc 0 2024-04-23T02:14:10.205 INFO:tasks.workunit.client.1.smithi192.stdout:3/799: chown d1/d3/d17/d2b/fd5 3 1 2024-04-23T02:14:10.205 INFO:tasks.workunit.client.0.smithi071.stdout:7/935: unlink d3/d17/d1c/d31/ld3 0 2024-04-23T02:14:10.208 INFO:tasks.workunit.client.1.smithi192.stdout:0/968: dwrite d6/df/d47/f4c [0,4194304] 0 2024-04-23T02:14:10.208 INFO:tasks.workunit.client.0.smithi071.stdout:2/673: symlink d0/d2/d26/d3a/dc3/d36/d6a/da1/ldd 0 2024-04-23T02:14:10.211 INFO:tasks.workunit.client.1.smithi192.stdout:7/877: creat d5/d17/d1c/d59/d5a/d41/dfd/f100 x:0 0 0 2024-04-23T02:14:10.211 INFO:tasks.workunit.client.1.smithi192.stdout:5/843: dwrite d2/d6/d2d/d41/d43/f9d [0,4194304] 0 2024-04-23T02:14:10.212 INFO:tasks.workunit.client.0.smithi071.stdout:1/647: link db/df/d13/d20/f32 db/df/d13/d20/dc0/fd1 0 2024-04-23T02:14:10.214 INFO:tasks.workunit.client.0.smithi071.stdout:7/936: creat d3/d17/d41/d3a/d137/dea/d5f/d13f/d143/f148 x:0 0 0 2024-04-23T02:14:10.216 INFO:tasks.workunit.client.1.smithi192.stdout:4/931: creat de/d15/d53/d81/de3/d74/d92/da0/dca/d34/df9/f137 x:0 0 0 2024-04-23T02:14:10.216 INFO:tasks.workunit.client.0.smithi071.stdout:0/760: rename d0/d3/de/d10/d14/d32/d44/d87/l103 to d0/d3/de/d10/d14/d32/d44/d87/l108 0 2024-04-23T02:14:10.217 INFO:tasks.workunit.client.0.smithi071.stdout:1/648: fdatasync db/df/d13/d2b/fab 0 2024-04-23T02:14:10.217 INFO:tasks.workunit.client.1.smithi192.stdout:6/947: write d3/d18/dd4/d8d/f9e [759247,10061] 0 2024-04-23T02:14:10.218 INFO:tasks.workunit.client.0.smithi071.stdout:1/649: chown db/df/d13/ccb 38642161 1 2024-04-23T02:14:10.220 INFO:tasks.workunit.client.1.smithi192.stdout:3/800: creat d1/d3/d17/d1a/d7c/dab/d6c/d84/d4c/de3/f116 x:0 0 0 2024-04-23T02:14:10.220 INFO:tasks.workunit.client.1.smithi192.stdout:5/844: chown d2/db/d16/d48/dc5/le2 252121 1 2024-04-23T02:14:10.222 INFO:tasks.workunit.client.0.smithi071.stdout:0/761: symlink d0/ddf/l109 0 2024-04-23T02:14:10.223 INFO:tasks.workunit.client.1.smithi192.stdout:1/727: dwrite dc/d14/d15/d35/f40 [0,4194304] 0 2024-04-23T02:14:10.224 INFO:tasks.workunit.client.1.smithi192.stdout:0/969: creat d6/d4b/f141 x:0 0 0 2024-04-23T02:14:10.224 INFO:tasks.workunit.client.0.smithi071.stdout:1/650: mknod db/df/d3a/cd2 0 2024-04-23T02:14:10.226 INFO:tasks.workunit.client.0.smithi071.stdout:1/651: chown db/f3b 1 1 2024-04-23T02:14:10.226 INFO:tasks.workunit.client.0.smithi071.stdout:2/674: link d0/f1e d0/d2/d34/d55/d96/fde 0 2024-04-23T02:14:10.227 INFO:tasks.workunit.client.1.smithi192.stdout:4/932: symlink de/d15/d53/d81/de3/d74/d92/l138 0 2024-04-23T02:14:10.227 INFO:tasks.workunit.client.0.smithi071.stdout:7/937: dwrite d3/d17/d1c/d31/d112/f11f [0,4194304] 0 2024-04-23T02:14:10.230 INFO:tasks.workunit.client.0.smithi071.stdout:0/762: mknod d0/d3/de/d10/d14/d32/d44/d87/c10a 0 2024-04-23T02:14:10.231 INFO:tasks.workunit.client.1.smithi192.stdout:0/970: chown d6/df/d13/d6b/d73/d125/d137/cb8 1722202 1 2024-04-23T02:14:10.232 INFO:tasks.workunit.client.0.smithi071.stdout:7/938: mknod d3/d17/d1c/d2a/d73/dba/c149 0 2024-04-23T02:14:10.234 INFO:tasks.workunit.client.0.smithi071.stdout:0/763: creat d0/d3/de/d10/db9/f10b x:0 0 0 2024-04-23T02:14:10.236 INFO:tasks.workunit.client.0.smithi071.stdout:1/652: dwrite db/df/d3a/f47 [0,4194304] 0 2024-04-23T02:14:10.237 INFO:tasks.workunit.client.1.smithi192.stdout:6/948: link d3/d15/l4d d3/d28/l135 0 2024-04-23T02:14:10.238 INFO:tasks.workunit.client.1.smithi192.stdout:1/728: truncate dc/d14/d15/d35/f73 56969 0 2024-04-23T02:14:10.238 INFO:tasks.workunit.client.1.smithi192.stdout:5/845: dwrite d2/db/d16/f8b [0,4194304] 0 2024-04-23T02:14:10.238 INFO:tasks.workunit.client.0.smithi071.stdout:7/939: symlink d3/d17/d41/de2/l14a 0 2024-04-23T02:14:10.239 INFO:tasks.workunit.client.1.smithi192.stdout:3/801: link d1/d3/d6/cf9 d1/d3/d17/d1f/d33/d24/c117 0 2024-04-23T02:14:10.241 INFO:tasks.workunit.client.1.smithi192.stdout:4/933: mknod de/d15/d53/d81/de3/d74/d92/da0/dca/de5/c139 0 2024-04-23T02:14:10.241 INFO:tasks.workunit.client.0.smithi071.stdout:1/653: symlink db/df/d13/d2f/d48/d85/ld3 0 2024-04-23T02:14:10.244 INFO:tasks.workunit.client.0.smithi071.stdout:7/940: rename d3/d17/d1c/d2a/c40 to d3/d17/d1c/d2a/d73/dba/c14b 0 2024-04-23T02:14:10.245 INFO:tasks.workunit.client.1.smithi192.stdout:6/949: truncate d3/d28/d79/da0/dc2/d101/d80/d8e/faf 3276024 0 2024-04-23T02:14:10.245 INFO:tasks.workunit.client.0.smithi071.stdout:0/764: link d0/d3/de/d48/l6d d0/d92/db2/l10c 0 2024-04-23T02:14:10.246 INFO:tasks.workunit.client.1.smithi192.stdout:0/971: getdents d6/df/d56/d85/d45 0 2024-04-23T02:14:10.248 INFO:tasks.workunit.client.0.smithi071.stdout:0/765: chown d0/d3/de/d10/d14/d32/d37/d26/f9a 622649 1 2024-04-23T02:14:10.248 INFO:tasks.workunit.client.1.smithi192.stdout:5/846: mknod d2/d6/d2d/d41/d43/c117 0 2024-04-23T02:14:10.249 INFO:tasks.workunit.client.0.smithi071.stdout:1/654: creat db/df/d13/d20/d75/d96/fd4 x:0 0 0 2024-04-23T02:14:10.250 INFO:tasks.workunit.client.1.smithi192.stdout:6/950: creat d3/d28/df8/d7c/f136 x:0 0 0 2024-04-23T02:14:10.250 INFO:tasks.workunit.client.0.smithi071.stdout:7/941: truncate d3/d17/d41/d34/f89 953843 0 2024-04-23T02:14:10.253 INFO:tasks.workunit.client.1.smithi192.stdout:3/802: truncate d1/d3/d6/d7/d29/d3b/da9/d7b/f96 2848928 0 2024-04-23T02:14:10.254 INFO:tasks.workunit.client.1.smithi192.stdout:6/951: chown d3/c10 11 1 2024-04-23T02:14:10.254 INFO:tasks.workunit.client.0.smithi071.stdout:1/655: symlink db/df/d13/ld5 0 2024-04-23T02:14:10.255 INFO:tasks.workunit.client.1.smithi192.stdout:5/847: rename d2/d6/d2d/fb9 to d2/d6/d2d/f118 0 2024-04-23T02:14:10.256 INFO:tasks.workunit.client.1.smithi192.stdout:5/848: chown d2/d6/cbe 1647670432 1 2024-04-23T02:14:10.257 INFO:tasks.workunit.client.0.smithi071.stdout:1/656: fdatasync db/d34/d68/f81 0 2024-04-23T02:14:10.258 INFO:tasks.workunit.client.1.smithi192.stdout:4/934: dwrite de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d11b/fc2 [0,4194304] 0 2024-04-23T02:14:10.260 INFO:tasks.workunit.client.1.smithi192.stdout:6/952: mknod d3/d15/c137 0 2024-04-23T02:14:10.261 INFO:tasks.workunit.client.0.smithi071.stdout:0/766: dwrite d0/d3/de/d10/d14/d32/d37/d26/fc2 [0,4194304] 0 2024-04-23T02:14:10.261 INFO:tasks.workunit.client.0.smithi071.stdout:1/657: truncate db/df/d13/d2f/f3c 7536661 0 2024-04-23T02:14:10.262 INFO:tasks.workunit.client.1.smithi192.stdout:4/935: chown de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/l85 3186095 1 2024-04-23T02:14:10.262 INFO:tasks.workunit.client.1.smithi192.stdout:5/849: symlink d2/db/d16/d48/l119 0 2024-04-23T02:14:10.263 INFO:tasks.workunit.client.1.smithi192.stdout:6/953: write d3/f2b [3971183,97606] 0 2024-04-23T02:14:10.265 INFO:tasks.workunit.client.1.smithi192.stdout:6/954: fdatasync d3/d18/dd4/d8d/fba 0 2024-04-23T02:14:10.265 INFO:tasks.workunit.client.1.smithi192.stdout:4/936: creat de/d15/d19/d1a/d95/dd5/f13a x:0 0 0 2024-04-23T02:14:10.266 INFO:tasks.workunit.client.1.smithi192.stdout:5/850: dread - d2/db/d16/d1b/fd7 zero size 2024-04-23T02:14:10.268 INFO:tasks.workunit.client.1.smithi192.stdout:6/955: write d3/d15/d1b/f72 [3876982,47097] 0 2024-04-23T02:14:10.268 INFO:tasks.workunit.client.0.smithi071.stdout:0/767: unlink d0/d3/de/d10/d14/d32/d44/d31/l36 0 2024-04-23T02:14:10.270 INFO:tasks.workunit.client.1.smithi192.stdout:4/937: dread - de/d15/d53/d81/de3/d74/d92/da0/dca/de5/f121 zero size 2024-04-23T02:14:10.270 INFO:tasks.workunit.client.1.smithi192.stdout:5/851: symlink d2/d2e/l11a 0 2024-04-23T02:14:10.271 INFO:tasks.workunit.client.0.smithi071.stdout:0/768: write d0/d3/de/d10/f1f [1368333,41640] 0 2024-04-23T02:14:10.271 INFO:tasks.workunit.client.1.smithi192.stdout:6/956: chown d3/d28/d79/da0/dc2/d101/d80/d8e/l70 3689927 1 2024-04-23T02:14:10.273 INFO:tasks.workunit.client.1.smithi192.stdout:5/852: truncate d2/db/d2b/f7f 5233360 0 2024-04-23T02:14:10.273 INFO:tasks.workunit.client.0.smithi071.stdout:0/769: dread d0/d3/de/d10/d14/d32/d44/d87/f96 [0,4194304] 0 2024-04-23T02:14:10.274 INFO:tasks.workunit.client.0.smithi071.stdout:0/770: chown d0/f20 0 1 2024-04-23T02:14:10.275 INFO:tasks.workunit.client.1.smithi192.stdout:6/957: creat d3/d28/d79/da0/dc2/d101/d80/d8e/d128/f138 x:0 0 0 2024-04-23T02:14:10.275 INFO:tasks.workunit.client.0.smithi071.stdout:0/771: chown d0/d3/de/d10/db3 11084316 1 2024-04-23T02:14:10.276 INFO:tasks.workunit.client.0.smithi071.stdout:0/772: fsync d0/d3/d95/fb4 0 2024-04-23T02:14:10.277 INFO:tasks.workunit.client.0.smithi071.stdout:0/773: dread - d0/d3/de/d10/d14/d84/fd1 zero size 2024-04-23T02:14:10.277 INFO:tasks.workunit.client.1.smithi192.stdout:6/958: mknod d3/d28/d79/da0/c139 0 2024-04-23T02:14:10.278 INFO:tasks.workunit.client.1.smithi192.stdout:4/938: getdents de/d15/d53/d81/de3/d74/d92/da0/dca/d49 0 2024-04-23T02:14:10.280 INFO:tasks.workunit.client.1.smithi192.stdout:6/959: symlink d3/d15/d1b/dcf/ddf/l13a 0 2024-04-23T02:14:10.281 INFO:tasks.workunit.client.1.smithi192.stdout:4/939: mknod de/d15/d53/d81/de3/d74/d92/da0/dca/de5/c13b 0 2024-04-23T02:14:10.283 INFO:tasks.workunit.client.1.smithi192.stdout:6/960: rename d3/d15/d1b/da2/fc3 to d3/d28/df8/d3a/dbd/dec/dfa/f13b 0 2024-04-23T02:14:10.286 INFO:tasks.workunit.client.1.smithi192.stdout:6/961: creat d3/d28/df8/d3a/dbd/dec/dfa/f13c x:0 0 0 2024-04-23T02:14:10.287 INFO:tasks.workunit.client.1.smithi192.stdout:6/962: rmdir d3/d15/d1b 39 2024-04-23T02:14:10.289 INFO:tasks.workunit.client.1.smithi192.stdout:6/963: mkdir d3/d11/db0/d13d 0 2024-04-23T02:14:10.297 INFO:tasks.workunit.client.1.smithi192.stdout:3/803: rmdir d1/d3/d17/d1a/d7c/dab/d6c/d84 39 2024-04-23T02:14:10.300 INFO:tasks.workunit.client.1.smithi192.stdout:3/804: mkdir d1/d3/d6/d7/d29/d3b/da9/d8b/d118 0 2024-04-23T02:14:10.304 INFO:tasks.workunit.client.1.smithi192.stdout:1/729: write dc/d14/d23/d5e/dab/fd3 [2755102,28428] 0 2024-04-23T02:14:10.305 INFO:tasks.workunit.client.1.smithi192.stdout:1/730: readlink dc/l6b 0 2024-04-23T02:14:10.307 INFO:tasks.workunit.client.1.smithi192.stdout:1/731: dread - dc/d14/d15/fcd zero size 2024-04-23T02:14:10.308 INFO:tasks.workunit.client.0.smithi071.stdout:7/942: dread d3/d17/d41/d34/f89 [0,4194304] 0 2024-04-23T02:14:10.311 INFO:tasks.workunit.client.0.smithi071.stdout:1/658: write db/df/d5f/d57/f89 [3842323,128106] 0 2024-04-23T02:14:10.311 INFO:tasks.workunit.client.1.smithi192.stdout:3/805: dwrite d1/f55 [0,4194304] 0 2024-04-23T02:14:10.313 INFO:tasks.workunit.client.0.smithi071.stdout:0/774: write d0/d3/de/d10/d14/d32/d44/d31/d52/f5c [423872,128956] 0 2024-04-23T02:14:10.314 INFO:tasks.workunit.client.0.smithi071.stdout:1/659: dread - db/df/d13/d2b/f43 zero size 2024-04-23T02:14:10.316 INFO:tasks.workunit.client.1.smithi192.stdout:0/972: dwrite d6/df/d13/d6b/f6e [0,4194304] 0 2024-04-23T02:14:10.319 INFO:tasks.workunit.client.1.smithi192.stdout:1/732: dread dc/f76 [0,4194304] 0 2024-04-23T02:14:10.321 INFO:tasks.workunit.client.0.smithi071.stdout:1/660: creat db/df/d5f/d6f/db2/d86/fd6 x:0 0 0 2024-04-23T02:14:10.322 INFO:tasks.workunit.client.0.smithi071.stdout:0/775: symlink d0/d3/de/d10/d14/d32/d44/d87/d93/dcc/l10d 0 2024-04-23T02:14:10.322 INFO:tasks.workunit.client.0.smithi071.stdout:7/943: dwrite d3/d17/d1c/d2a/d43/f102 [0,4194304] 0 2024-04-23T02:14:10.322 INFO:tasks.workunit.client.1.smithi192.stdout:0/973: symlink d6/d4b/d9d/l142 0 2024-04-23T02:14:10.323 INFO:tasks.workunit.client.1.smithi192.stdout:1/733: mknod dc/d8b/d44/da9/cea 0 2024-04-23T02:14:10.324 INFO:tasks.workunit.client.1.smithi192.stdout:5/853: write d2/d6/d2d/f118 [230775,22991] 0 2024-04-23T02:14:10.325 INFO:tasks.workunit.client.0.smithi071.stdout:1/661: symlink db/df/d13/d20/dc0/ld7 0 2024-04-23T02:14:10.326 INFO:tasks.workunit.client.0.smithi071.stdout:0/776: mknod d0/d3/de/d10/d14/d32/c10e 0 2024-04-23T02:14:10.327 INFO:tasks.workunit.client.0.smithi071.stdout:7/944: creat d3/d17/d1c/d2a/f14c x:0 0 0 2024-04-23T02:14:10.328 INFO:tasks.workunit.client.0.smithi071.stdout:1/662: creat db/d11/d8d/fd8 x:0 0 0 2024-04-23T02:14:10.328 INFO:tasks.workunit.client.0.smithi071.stdout:7/945: fsync d3/d51/df5/d10d/f12e 0 2024-04-23T02:14:10.329 INFO:tasks.workunit.client.1.smithi192.stdout:5/854: creat d2/db/d16/d48/d4f/f11b x:0 0 0 2024-04-23T02:14:10.330 INFO:tasks.workunit.client.0.smithi071.stdout:7/946: stat d3/d17/d41/d3a/d137/dea/fe6 0 2024-04-23T02:14:10.331 INFO:tasks.workunit.client.0.smithi071.stdout:1/663: mkdir db/d11/d7f/da7/dd9 0 2024-04-23T02:14:10.331 INFO:tasks.workunit.client.1.smithi192.stdout:3/806: link d1/d3/d17/d1f/d33/fad d1/d3/d17/d1a/d7c/dab/d6c/d84/d4c/f119 0 2024-04-23T02:14:10.331 INFO:tasks.workunit.client.1.smithi192.stdout:1/734: symlink dc/d16/d19/d30/dc4/de3/leb 0 2024-04-23T02:14:10.332 INFO:tasks.workunit.client.1.smithi192.stdout:0/974: link d6/df/d47/d55/l105 d6/df/d56/d85/d71/l143 0 2024-04-23T02:14:10.333 INFO:tasks.workunit.client.1.smithi192.stdout:5/855: creat d2/db/d16/d48/d4f/f11c x:0 0 0 2024-04-23T02:14:10.333 INFO:tasks.workunit.client.1.smithi192.stdout:0/975: chown d6/df/d56/c65 8704226 1 2024-04-23T02:14:10.334 INFO:tasks.workunit.client.1.smithi192.stdout:3/807: creat d1/d3/d6/d7/d29/d3b/da9/daf/f11a x:0 0 0 2024-04-23T02:14:10.334 INFO:tasks.workunit.client.0.smithi071.stdout:0/777: dwrite d0/d3/de/d10/d14/d32/d44/d31/fe9 [0,4194304] 0 2024-04-23T02:14:10.335 INFO:tasks.workunit.client.0.smithi071.stdout:1/664: unlink db/df/d5f/l28 0 2024-04-23T02:14:10.335 INFO:tasks.workunit.client.1.smithi192.stdout:1/735: mknod dc/d8b/d34/d70/d8c/da3/dc1/cec 0 2024-04-23T02:14:10.337 INFO:tasks.workunit.client.1.smithi192.stdout:5/856: creat d2/db/d2b/d52/dc2/d49/f11d x:0 0 0 2024-04-23T02:14:10.338 INFO:tasks.workunit.client.0.smithi071.stdout:0/778: stat d0/d3/de/d10/d14/d32/d44/d31/d52/lff 0 2024-04-23T02:14:10.338 INFO:tasks.workunit.client.1.smithi192.stdout:3/808: creat d1/d3/d6/d7/d71/f11b x:0 0 0 2024-04-23T02:14:10.339 INFO:tasks.workunit.client.0.smithi071.stdout:0/779: write d0/d3/f3f [4580982,107654] 0 2024-04-23T02:14:10.339 INFO:tasks.workunit.client.0.smithi071.stdout:7/947: link d3/d17/d1c/d2a/d73/dba/cc4 d3/d17/d1c/d2a/d73/c14d 0 2024-04-23T02:14:10.339 INFO:tasks.workunit.client.0.smithi071.stdout:7/948: stat d3/dae 0 2024-04-23T02:14:10.340 INFO:tasks.workunit.client.1.smithi192.stdout:0/976: mkdir d6/df/d144 0 2024-04-23T02:14:10.340 INFO:tasks.workunit.client.0.smithi071.stdout:0/780: chown d0/d3/de/d48/d7e/d8b/f22 0 1 2024-04-23T02:14:10.341 INFO:tasks.workunit.client.0.smithi071.stdout:1/665: rename db/df/d13/d20/d75/f9c to db/df/d13/d2f/fda 0 2024-04-23T02:14:10.342 INFO:tasks.workunit.client.1.smithi192.stdout:1/736: symlink dc/d16/dde/led 0 2024-04-23T02:14:10.342 INFO:tasks.workunit.client.1.smithi192.stdout:3/809: read d1/d3/d17/d1f/d33/fcf [162261,109302] 0 2024-04-23T02:14:10.342 INFO:tasks.workunit.client.0.smithi071.stdout:7/949: mkdir d3/d12a/d14e 0 2024-04-23T02:14:10.343 INFO:tasks.workunit.client.0.smithi071.stdout:7/950: fsync d3/d51/df5/d10d/d127/f133 0 2024-04-23T02:14:10.344 INFO:tasks.workunit.client.0.smithi071.stdout:0/781: symlink d0/d3/de/d10/d14/d32/d44/d87/d93/dcc/l10f 0 2024-04-23T02:14:10.346 INFO:tasks.workunit.client.1.smithi192.stdout:5/857: creat d2/d6/d2d/d41/f11e x:0 0 0 2024-04-23T02:14:10.352 INFO:tasks.workunit.client.1.smithi192.stdout:3/810: mknod d1/d3/d17/d2b/d44/d57/c11c 0 2024-04-23T02:14:10.352 INFO:tasks.workunit.client.1.smithi192.stdout:1/737: creat dc/d14/fee x:0 0 0 2024-04-23T02:14:10.352 INFO:tasks.workunit.client.1.smithi192.stdout:1/738: chown dc/d14/d15/d35 4180 1 2024-04-23T02:14:10.352 INFO:tasks.workunit.client.1.smithi192.stdout:0/977: getdents d6/df/d13/d6b/d73/d125/d137/d67 0 2024-04-23T02:14:10.352 INFO:tasks.workunit.client.1.smithi192.stdout:5/858: read d2/db/d16/d48/f9e [1337021,50140] 0 2024-04-23T02:14:10.352 INFO:tasks.workunit.client.0.smithi071.stdout:0/782: chown d0/d3/de/d48/c75 7 1 2024-04-23T02:14:10.353 INFO:tasks.workunit.client.0.smithi071.stdout:0/783: readlink d0/d3/de/d10/db9/lc7 0 2024-04-23T02:14:10.353 INFO:tasks.workunit.client.0.smithi071.stdout:7/951: rename d3/d51/lde to d3/d17/d11b/l14f 0 2024-04-23T02:14:10.353 INFO:tasks.workunit.client.0.smithi071.stdout:1/666: dwrite db/df/d5f/d6f/db2/d86/fd6 [0,4194304] 0 2024-04-23T02:14:10.353 INFO:tasks.workunit.client.0.smithi071.stdout:7/952: mknod d3/dae/c150 0 2024-04-23T02:14:10.353 INFO:tasks.workunit.client.0.smithi071.stdout:0/784: rmdir d0/d3/de/d10/db3 39 2024-04-23T02:14:10.353 INFO:tasks.workunit.client.1.smithi192.stdout:3/811: mkdir d1/d3/d17/d11d 0 2024-04-23T02:14:10.355 INFO:tasks.workunit.client.1.smithi192.stdout:1/739: dread dc/d14/f27 [4194304,4194304] 0 2024-04-23T02:14:10.355 INFO:tasks.workunit.client.1.smithi192.stdout:5/859: creat d2/db/d46/f11f x:0 0 0 2024-04-23T02:14:10.355 INFO:tasks.workunit.client.1.smithi192.stdout:0/978: readlink d6/df/d13/d25/d3a/da6/d116/l13e 0 2024-04-23T02:14:10.357 INFO:tasks.workunit.client.0.smithi071.stdout:1/667: read db/d34/f61 [3025607,5337] 0 2024-04-23T02:14:10.358 INFO:tasks.workunit.client.1.smithi192.stdout:5/860: stat d2/db/d2b/d52/dc2/d49/la9 0 2024-04-23T02:14:10.361 INFO:tasks.workunit.client.1.smithi192.stdout:0/979: rename d6/df/d56/d85/d71/f133 to d6/df/d56/f145 0 2024-04-23T02:14:10.362 INFO:tasks.workunit.client.1.smithi192.stdout:3/812: getdents d1/d3/d17/dd1 0 2024-04-23T02:14:10.362 INFO:tasks.workunit.client.1.smithi192.stdout:0/980: read - d6/df/d47/f92 zero size 2024-04-23T02:14:10.362 INFO:tasks.workunit.client.1.smithi192.stdout:3/813: readlink d1/d3/l74 0 2024-04-23T02:14:10.363 INFO:tasks.workunit.client.1.smithi192.stdout:5/861: mkdir d2/db/d2b/d52/dc2/d49/d120 0 2024-04-23T02:14:10.364 INFO:tasks.workunit.client.1.smithi192.stdout:5/862: readlink d2/db/d16/d48/l119 0 2024-04-23T02:14:10.365 INFO:tasks.workunit.client.1.smithi192.stdout:3/814: creat d1/d3/d6/d7/d29/d3b/f11e x:0 0 0 2024-04-23T02:14:10.366 INFO:tasks.workunit.client.1.smithi192.stdout:3/815: chown d1/d3/d6/d7/d29/d3b/da9/daf 2942 1 2024-04-23T02:14:10.367 INFO:tasks.workunit.client.1.smithi192.stdout:0/981: rename d6/f6d to d6/df/d13/d25/d3a/d7b/f146 0 2024-04-23T02:14:10.370 INFO:tasks.workunit.client.1.smithi192.stdout:0/982: creat d6/df/d35/d8b/d8d/f147 x:0 0 0 2024-04-23T02:14:10.370 INFO:tasks.workunit.client.1.smithi192.stdout:3/816: mkdir d1/d3/d17/d1f/d33/dd6/d11f 0 2024-04-23T02:14:10.375 INFO:tasks.workunit.client.1.smithi192.stdout:0/983: link d6/df/d13/d25/d107/d12f/f13a d6/df/d13/d6b/d73/f148 0 2024-04-23T02:14:10.381 INFO:tasks.workunit.client.1.smithi192.stdout:3/817: dwrite d1/d3/d17/d2b/f51 [0,4194304] 0 2024-04-23T02:14:10.381 INFO:tasks.workunit.client.1.smithi192.stdout:0/984: write d6/df/d35/d8b/d8d/f147 [25959,115940] 0 2024-04-23T02:14:10.383 INFO:tasks.workunit.client.0.smithi071.stdout:6/696: sync 2024-04-23T02:14:10.383 INFO:tasks.workunit.client.0.smithi071.stdout:9/706: sync 2024-04-23T02:14:10.383 INFO:tasks.workunit.client.0.smithi071.stdout:3/705: sync 2024-04-23T02:14:10.384 INFO:tasks.workunit.client.0.smithi071.stdout:4/808: sync 2024-04-23T02:14:10.386 INFO:tasks.workunit.client.1.smithi192.stdout:3/818: mkdir d1/d3/d6/d7/d29/d3b/d120 0 2024-04-23T02:14:10.386 INFO:tasks.workunit.client.0.smithi071.stdout:7/953: mknod d3/d17/d41/d118/c151 0 2024-04-23T02:14:10.387 INFO:tasks.workunit.client.1.smithi192.stdout:3/819: dread - d1/d3/d17/d1a/d7c/fe0 zero size 2024-04-23T02:14:10.389 INFO:tasks.workunit.client.1.smithi192.stdout:0/985: getdents d6/d4b/d9d/dc7 0 2024-04-23T02:14:10.392 INFO:tasks.workunit.client.0.smithi071.stdout:7/954: write d3/d51/d87/db3/fbc [805065,36929] 0 2024-04-23T02:14:10.395 INFO:tasks.workunit.client.0.smithi071.stdout:6/697: read d6/d17/d77/d85/f9b [1593858,60193] 0 2024-04-23T02:14:10.396 INFO:tasks.workunit.client.1.smithi192.stdout:3/820: dwrite d1/d3/d17/d1a/d7c/dab/f92 [0,4194304] 0 2024-04-23T02:14:10.396 INFO:tasks.workunit.client.0.smithi071.stdout:9/707: stat d1/d32/d41/db7/d5a/dc6/d81 0 2024-04-23T02:14:10.397 INFO:tasks.workunit.client.0.smithi071.stdout:3/706: rename d7/d22/d23/d26/l8d to d7/d10/d38/lf7 0 2024-04-23T02:14:10.403 INFO:tasks.workunit.client.0.smithi071.stdout:9/708: chown d1/d32/d3a/d5f/cd5 3 1 2024-04-23T02:14:10.405 INFO:tasks.workunit.client.0.smithi071.stdout:4/809: symlink d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/dc8/deb/l108 0 2024-04-23T02:14:10.405 INFO:tasks.workunit.client.0.smithi071.stdout:9/709: write d1/d32/d41/db7/d5a/dc6/faa [962256,35709] 0 2024-04-23T02:14:10.406 INFO:tasks.workunit.client.0.smithi071.stdout:6/698: rename d6/d1f/d46/c48 to d6/d17/cdc 0 2024-04-23T02:14:10.407 INFO:tasks.workunit.client.1.smithi192.stdout:1/740: write dc/fd [8115923,20687] 0 2024-04-23T02:14:10.408 INFO:tasks.workunit.client.0.smithi071.stdout:0/785: dread d0/d3/de/d10/d14/d32/d37/f7b [0,4194304] 0 2024-04-23T02:14:10.408 INFO:tasks.workunit.client.1.smithi192.stdout:3/821: dwrite d1/d3/d17/d1a/d7c/dab/d6c/d84/d5d/f10b [4194304,4194304] 0 2024-04-23T02:14:10.409 INFO:tasks.workunit.client.0.smithi071.stdout:7/955: fsync d3/d17/d1c/f64 0 2024-04-23T02:14:10.410 INFO:tasks.workunit.client.0.smithi071.stdout:3/707: rename d7/d22/da7/l9d to d7/d22/d23/d26/d7e/lf8 0 2024-04-23T02:14:10.411 INFO:tasks.workunit.client.1.smithi192.stdout:1/741: symlink dc/d8b/d34/d70/lef 0 2024-04-23T02:14:10.412 INFO:tasks.workunit.client.0.smithi071.stdout:9/710: rmdir d1/d32/d3a/d5f 39 2024-04-23T02:14:10.414 INFO:tasks.workunit.client.1.smithi192.stdout:1/742: dread - dc/d14/d15/fcd zero size 2024-04-23T02:14:10.415 INFO:tasks.workunit.client.1.smithi192.stdout:3/822: creat d1/d3/d6/d7/d29/d3b/da9/f121 x:0 0 0 2024-04-23T02:14:10.416 INFO:tasks.workunit.client.0.smithi071.stdout:6/699: unlink d6/d22/d92/lb4 0 2024-04-23T02:14:10.417 INFO:tasks.workunit.client.0.smithi071.stdout:7/956: dwrite d3/d17/d1c/f8f [0,4194304] 0 2024-04-23T02:14:10.418 INFO:tasks.workunit.client.1.smithi192.stdout:5/863: dwrite d2/d6/f3b [0,4194304] 0 2024-04-23T02:14:10.420 INFO:tasks.workunit.client.0.smithi071.stdout:3/708: unlink d7/da5/db2/db7/d68/d3b/d98/la6 0 2024-04-23T02:14:10.423 INFO:tasks.workunit.client.1.smithi192.stdout:5/864: mkdir d2/db/d16/d48/dc5/d121 0 2024-04-23T02:14:10.425 INFO:tasks.workunit.client.0.smithi071.stdout:0/786: creat d0/d3/de/d10/db3/dba/f110 x:0 0 0 2024-04-23T02:14:10.426 INFO:tasks.workunit.client.0.smithi071.stdout:0/787: write d0/d3/f3f [4665849,16549] 0 2024-04-23T02:14:10.427 INFO:tasks.workunit.client.1.smithi192.stdout:5/865: readlink d2/db/d2b/d52/dc2/d49/l116 0 2024-04-23T02:14:10.428 INFO:tasks.workunit.client.1.smithi192.stdout:1/743: dwrite dc/d8b/d34/d70/d8c/fa4 [4194304,4194304] 0 2024-04-23T02:14:10.428 INFO:tasks.workunit.client.1.smithi192.stdout:3/823: dwrite d1/d3/d17/d1a/d7c/dab/f92 [0,4194304] 0 2024-04-23T02:14:10.430 INFO:tasks.workunit.client.0.smithi071.stdout:3/709: symlink d7/d8/lf9 0 2024-04-23T02:14:10.430 INFO:tasks.workunit.client.0.smithi071.stdout:9/711: rename d1/d32/d41/db7/c33 to d1/d32/d3a/d90/dc2/cf9 0 2024-04-23T02:14:10.430 INFO:tasks.workunit.client.1.smithi192.stdout:5/866: readlink d2/db/d16/l22 0 2024-04-23T02:14:10.432 INFO:tasks.workunit.client.0.smithi071.stdout:3/710: chown d7/d8/l8a 4460 1 2024-04-23T02:14:10.433 INFO:tasks.workunit.client.1.smithi192.stdout:1/744: write dc/d14/f27 [5251161,118482] 0 2024-04-23T02:14:10.433 INFO:tasks.workunit.client.0.smithi071.stdout:9/712: unlink d1/d32/dc5/cef 0 2024-04-23T02:14:10.437 INFO:tasks.workunit.client.0.smithi071.stdout:9/713: unlink d1/d32/d41/db7/d71/ld3 0 2024-04-23T02:14:10.438 INFO:tasks.workunit.client.1.smithi192.stdout:3/824: truncate d1/d3/d6/d7/d29/d3b/da9/daf/f11a 19065 0 2024-04-23T02:14:10.439 INFO:tasks.workunit.client.0.smithi071.stdout:9/714: write d1/d32/d41/db7/d39/d42/dcc/ff2 [760769,78535] 0 2024-04-23T02:14:10.439 INFO:tasks.workunit.client.0.smithi071.stdout:3/711: link d7/d22/da7/f9b d7/d22/d23/d26/d7e/ffa 0 2024-04-23T02:14:10.443 INFO:tasks.workunit.client.0.smithi071.stdout:3/712: mknod d7/cfb 0 2024-04-23T02:14:10.444 INFO:tasks.workunit.client.1.smithi192.stdout:3/825: rename d1/d3/d6/d7/d5f/fcd to d1/d3/d17/d1f/d33/d24/f122 0 2024-04-23T02:14:10.444 INFO:tasks.workunit.client.0.smithi071.stdout:3/713: write d7/d22/d23/dba/df4/d88/f57 [388707,31816] 0 2024-04-23T02:14:10.450 INFO:tasks.workunit.client.0.smithi071.stdout:3/714: dwrite d7/da5/fe0 [0,4194304] 0 2024-04-23T02:14:10.453 INFO:tasks.workunit.client.0.smithi071.stdout:3/715: mknod d7/da5/db2/db7/d68/d3b/cfc 0 2024-04-23T02:14:10.454 INFO:tasks.workunit.client.0.smithi071.stdout:3/716: fdatasync d7/d8/f61 0 2024-04-23T02:14:10.456 INFO:tasks.workunit.client.0.smithi071.stdout:3/717: fdatasync d7/da5/db2/db7/d68/d3b/d4c/d8b/db3/fc4 0 2024-04-23T02:14:10.460 INFO:tasks.workunit.client.0.smithi071.stdout:3/718: getdents d7/da5/db2/db7 0 2024-04-23T02:14:10.462 INFO:tasks.workunit.client.0.smithi071.stdout:3/719: mkdir d7/dfd 0 2024-04-23T02:14:10.462 INFO:tasks.workunit.client.0.smithi071.stdout:3/720: readlink d7/d10/l2d 0 2024-04-23T02:14:10.463 INFO:tasks.workunit.client.0.smithi071.stdout:3/721: write d7/da5/db2/db7/f8c [1095449,41222] 0 2024-04-23T02:14:10.465 INFO:tasks.workunit.client.0.smithi071.stdout:3/722: symlink d7/dfd/lfe 0 2024-04-23T02:14:10.467 INFO:tasks.workunit.client.0.smithi071.stdout:3/723: symlink d7/d22/d23/d26/d52/dd3/lff 0 2024-04-23T02:14:10.468 INFO:tasks.workunit.client.0.smithi071.stdout:3/724: chown d7/d22/d23/d26/d52/dd3 482355 1 2024-04-23T02:14:10.469 INFO:tasks.workunit.client.0.smithi071.stdout:3/725: write d7/d22/d23/f6b [709583,12231] 0 2024-04-23T02:14:10.471 INFO:tasks.workunit.client.0.smithi071.stdout:3/726: mkdir d7/da5/d100 0 2024-04-23T02:14:10.473 INFO:tasks.workunit.client.0.smithi071.stdout:3/727: mknod d7/d22/d23/dba/df4/c101 0 2024-04-23T02:14:10.475 INFO:tasks.workunit.client.0.smithi071.stdout:3/728: mkdir d7/d22/def/d102 0 2024-04-23T02:14:10.477 INFO:tasks.workunit.client.0.smithi071.stdout:3/729: truncate d7/d22/d23/d26/d52/dd3/fcc 923678 0 2024-04-23T02:14:10.477 INFO:tasks.workunit.client.0.smithi071.stdout:7/957: dread d3/d17/fb9 [0,4194304] 0 2024-04-23T02:14:10.479 INFO:tasks.workunit.client.0.smithi071.stdout:7/958: write d3/d17/d41/f2f [1797727,54354] 0 2024-04-23T02:14:10.481 INFO:tasks.workunit.client.0.smithi071.stdout:7/959: creat d3/d17/d1c/d31/d10c/f152 x:0 0 0 2024-04-23T02:14:10.484 INFO:tasks.workunit.client.0.smithi071.stdout:7/960: dread d3/d17/d1c/fee [0,4194304] 0 2024-04-23T02:14:10.487 INFO:tasks.workunit.client.0.smithi071.stdout:7/961: mknod d3/d17/d41/d3a/d137/dea/d5f/c153 0 2024-04-23T02:14:10.488 INFO:tasks.workunit.client.0.smithi071.stdout:8/798: sync 2024-04-23T02:14:10.489 INFO:tasks.workunit.client.0.smithi071.stdout:8/799: fsync dc/f9f 0 2024-04-23T02:14:10.490 INFO:tasks.workunit.client.0.smithi071.stdout:8/800: chown dc/d15/da0/d55/fc7 1 1 2024-04-23T02:14:10.490 INFO:tasks.workunit.client.0.smithi071.stdout:7/962: rmdir d3/d12a/d14e 0 2024-04-23T02:14:10.491 INFO:tasks.workunit.client.0.smithi071.stdout:1/668: write db/df/d13/d2f/f3e [1590815,112032] 0 2024-04-23T02:14:10.493 INFO:tasks.workunit.client.0.smithi071.stdout:8/801: fdatasync dc/d15/f7e 0 2024-04-23T02:14:10.494 INFO:tasks.workunit.client.1.smithi192.stdout:9/719: sync 2024-04-23T02:14:10.494 INFO:tasks.workunit.client.0.smithi071.stdout:7/963: mknod d3/d51/d80/de9/c154 0 2024-04-23T02:14:10.495 INFO:tasks.workunit.client.0.smithi071.stdout:7/964: read d3/d17/d41/d34/d39/d3b/f84 [1488,67099] 0 2024-04-23T02:14:10.495 INFO:tasks.workunit.client.0.smithi071.stdout:1/669: symlink db/df/d13/d20/d75/d96/ldb 0 2024-04-23T02:14:10.502 INFO:tasks.workunit.client.0.smithi071.stdout:1/670: mkdir db/df/d13/ddc 0 2024-04-23T02:14:10.503 INFO:tasks.workunit.client.1.smithi192.stdout:0/986: write d6/df/d13/d25/d3a/f58 [1712605,13097] 0 2024-04-23T02:14:10.503 INFO:tasks.workunit.client.0.smithi071.stdout:7/965: creat d3/d17/d1c/f155 x:0 0 0 2024-04-23T02:14:10.505 INFO:tasks.workunit.client.0.smithi071.stdout:6/700: write d6/d1f/d4e/fa3 [313676,46538] 0 2024-04-23T02:14:10.505 INFO:tasks.workunit.client.1.smithi192.stdout:0/987: read d6/df/d13/d25/d3a/d7b/f88 [4065083,32485] 0 2024-04-23T02:14:10.507 INFO:tasks.workunit.client.1.smithi192.stdout:0/988: write d6/df/d13/d6b/d73/d125/d137/fcf [3156050,47232] 0 2024-04-23T02:14:10.508 INFO:tasks.workunit.client.1.smithi192.stdout:0/989: fdatasync d6/df/d13/d25/d86/f114 0 2024-04-23T02:14:10.510 INFO:tasks.workunit.client.0.smithi071.stdout:0/788: truncate d0/d3/de/d10/d14/d32/d37/d26/f4b 3862306 0 2024-04-23T02:14:10.510 INFO:tasks.workunit.client.1.smithi192.stdout:5/867: write d2/d32/fc9 [521760,62813] 0 2024-04-23T02:14:10.512 INFO:tasks.workunit.client.0.smithi071.stdout:5/831: sync 2024-04-23T02:14:10.512 INFO:tasks.workunit.client.0.smithi071.stdout:2/675: sync 2024-04-23T02:14:10.514 INFO:tasks.workunit.client.0.smithi071.stdout:9/715: write d1/f10 [4859290,122449] 0 2024-04-23T02:14:10.521 INFO:tasks.workunit.client.0.smithi071.stdout:1/671: dread db/df/d13/d2f/f3c [0,4194304] 0 2024-04-23T02:14:10.521 INFO:tasks.workunit.client.0.smithi071.stdout:9/716: symlink d1/d32/d41/db7/d39/lfa 0 2024-04-23T02:14:10.522 INFO:tasks.workunit.client.0.smithi071.stdout:5/832: fsync d3/d1d/d43/dd2/d5a/f82 0 2024-04-23T02:14:10.523 INFO:tasks.workunit.client.1.smithi192.stdout:3/826: unlink d1/d3/d6/d7/d29/fbe 0 2024-04-23T02:14:10.524 INFO:tasks.workunit.client.1.smithi192.stdout:9/720: symlink d0/d9/dc7/led 0 2024-04-23T02:14:10.526 INFO:tasks.workunit.client.0.smithi071.stdout:1/672: mknod db/df/d13/d2f/d48/cdd 0 2024-04-23T02:14:10.526 INFO:tasks.workunit.client.1.smithi192.stdout:0/990: mkdir d6/df/d56/d85/d71/d149 0 2024-04-23T02:14:10.527 INFO:tasks.workunit.client.1.smithi192.stdout:5/868: fsync d2/d6/d2d/d41/d43/fbd 0 2024-04-23T02:14:10.527 INFO:tasks.workunit.client.0.smithi071.stdout:9/717: mkdir d1/d32/d41/db7/d71/dfb 0 2024-04-23T02:14:10.528 INFO:tasks.workunit.client.1.smithi192.stdout:3/827: rmdir d1/d3/d17/d65/d99 39 2024-04-23T02:14:10.530 INFO:tasks.workunit.client.0.smithi071.stdout:5/833: creat d3/d1d/d43/dd2/da9/f134 x:0 0 0 2024-04-23T02:14:10.534 INFO:tasks.workunit.client.1.smithi192.stdout:0/991: symlink d6/df/d56/d85/d45/l14a 0 2024-04-23T02:14:10.535 INFO:tasks.workunit.client.0.smithi071.stdout:1/673: mknod db/df/d13/d2b/cde 0 2024-04-23T02:14:10.535 INFO:tasks.workunit.client.0.smithi071.stdout:2/676: link d0/d2/d26/d3a/c88 d0/d2/d45/d7d/d56/cdf 0 2024-04-23T02:14:10.535 INFO:tasks.workunit.client.0.smithi071.stdout:2/677: fdatasync d0/d2/d45/d7d/fdb 0 2024-04-23T02:14:10.535 INFO:tasks.workunit.client.0.smithi071.stdout:2/678: chown d0/d2/d34/dbf 3 1 2024-04-23T02:14:10.535 INFO:tasks.workunit.client.0.smithi071.stdout:5/834: chown d3/d1d/f20 64223 1 2024-04-23T02:14:10.536 INFO:tasks.workunit.client.1.smithi192.stdout:3/828: mkdir d1/d3/d6/d7/d29/d3b/da9/d7b/d123 0 2024-04-23T02:14:10.536 INFO:tasks.workunit.client.0.smithi071.stdout:1/674: mkdir db/df/d13/d20/dc0/ddf 0 2024-04-23T02:14:10.536 INFO:tasks.workunit.client.0.smithi071.stdout:1/675: chown db/d34/d68 64736 1 2024-04-23T02:14:10.537 INFO:tasks.workunit.client.1.smithi192.stdout:5/869: symlink d2/d6/d2d/d41/l122 0 2024-04-23T02:14:10.539 INFO:tasks.workunit.client.1.smithi192.stdout:9/721: creat d0/d9/d44/fee x:0 0 0 2024-04-23T02:14:10.539 INFO:tasks.workunit.client.0.smithi071.stdout:8/802: write dc/d15/da0/d55/f80 [3604727,116575] 0 2024-04-23T02:14:10.540 INFO:tasks.workunit.client.0.smithi071.stdout:1/676: rmdir db/df/d13/d20/d75/d96 39 2024-04-23T02:14:10.540 INFO:tasks.workunit.client.0.smithi071.stdout:8/803: write dc/d15/da0/d55/f65 [749534,34929] 0 2024-04-23T02:14:10.542 INFO:tasks.workunit.client.0.smithi071.stdout:1/677: fdatasync db/df/d5f/d6f/db2/f79 0 2024-04-23T02:14:10.542 INFO:tasks.workunit.client.0.smithi071.stdout:8/804: read dc/d15/da0/d55/f80 [906131,17230] 0 2024-04-23T02:14:10.543 INFO:tasks.workunit.client.0.smithi071.stdout:5/835: mknod d3/d1d/daf/d10/da5/d5f/dc0/c135 0 2024-04-23T02:14:10.544 INFO:tasks.workunit.client.0.smithi071.stdout:1/678: read db/f54 [755097,787] 0 2024-04-23T02:14:10.545 INFO:tasks.workunit.client.1.smithi192.stdout:9/722: mkdir d0/d9/d44/d58/dd9/def 0 2024-04-23T02:14:10.545 INFO:tasks.workunit.client.0.smithi071.stdout:2/679: rename d0/daa to d0/d2/d45/d7d/de0 0 2024-04-23T02:14:10.545 INFO:tasks.workunit.client.1.smithi192.stdout:0/992: link d6/df/d13/d25/d3a/lf8 d6/df/d47/l14b 0 2024-04-23T02:14:10.547 INFO:tasks.workunit.client.1.smithi192.stdout:3/829: link d1/d3/d17/d1f/c42 d1/d3/d17/dd1/dda/c124 0 2024-04-23T02:14:10.547 INFO:tasks.workunit.client.1.smithi192.stdout:5/870: dwrite d2/db/d16/fce [0,4194304] 0 2024-04-23T02:14:10.548 INFO:tasks.workunit.client.0.smithi071.stdout:0/789: rmdir d0/d3/de/d10/d14/d32/d37 39 2024-04-23T02:14:10.548 INFO:tasks.workunit.client.1.smithi192.stdout:5/871: stat d2/d6/f68 0 2024-04-23T02:14:10.550 INFO:tasks.workunit.client.0.smithi071.stdout:6/701: write d6/d17/d77/d9f/dad/f42 [4916319,116234] 0 2024-04-23T02:14:10.550 INFO:tasks.workunit.client.1.smithi192.stdout:9/723: unlink d0/d9/d44/d4f/l8b 0 2024-04-23T02:14:10.550 INFO:tasks.workunit.client.1.smithi192.stdout:3/830: write d1/d3/d6/d7/d29/d3b/f11e [288637,30106] 0 2024-04-23T02:14:10.551 INFO:tasks.workunit.client.1.smithi192.stdout:5/872: readlink d2/d6/db4/l102 0 2024-04-23T02:14:10.552 INFO:tasks.workunit.client.1.smithi192.stdout:9/724: write d0/d98/fca [993477,75881] 0 2024-04-23T02:14:10.553 INFO:tasks.workunit.client.0.smithi071.stdout:8/805: fdatasync dc/d15/d50/f82 0 2024-04-23T02:14:10.554 INFO:tasks.workunit.client.0.smithi071.stdout:7/966: dwrite d3/d17/d1c/d31/fc2 [0,4194304] 0 2024-04-23T02:14:10.556 INFO:tasks.workunit.client.0.smithi071.stdout:5/836: dwrite d3/f11 [0,4194304] 0 2024-04-23T02:14:10.560 INFO:tasks.workunit.client.0.smithi071.stdout:0/790: creat d0/d76/f111 x:0 0 0 2024-04-23T02:14:10.560 INFO:tasks.workunit.client.0.smithi071.stdout:5/837: truncate d3/d1d/daf/f89 5218590 0 2024-04-23T02:14:10.561 INFO:tasks.workunit.client.0.smithi071.stdout:1/679: mknod db/d11/d7f/da7/dd9/ce0 0 2024-04-23T02:14:10.561 INFO:tasks.workunit.client.1.smithi192.stdout:5/873: dwrite d2/db/d16/f96 [0,4194304] 0 2024-04-23T02:14:10.563 INFO:tasks.workunit.client.0.smithi071.stdout:9/718: write d1/d4e/d54/f69 [596351,87094] 0 2024-04-23T02:14:10.563 INFO:tasks.workunit.client.0.smithi071.stdout:6/702: dread - d6/d17/d77/d9f/dad/f5e zero size 2024-04-23T02:14:10.567 INFO:tasks.workunit.client.0.smithi071.stdout:7/967: rmdir d3/d17/d1c/d2a/d43 39 2024-04-23T02:14:10.567 INFO:tasks.workunit.client.0.smithi071.stdout:0/791: write d0/d3/de/d10/d14/d32/d8a/df0/fed [54740,103219] 0 2024-04-23T02:14:10.570 INFO:tasks.workunit.client.1.smithi192.stdout:5/874: creat d2/d6/d14/d108/f123 x:0 0 0 2024-04-23T02:14:10.571 INFO:tasks.workunit.client.0.smithi071.stdout:6/703: read d6/d1f/d4e/fa3 [349170,48869] 0 2024-04-23T02:14:10.573 INFO:tasks.workunit.client.1.smithi192.stdout:3/831: link d1/d3/d6/d7/d29/d3b/da9/daf/f10e d1/d3/d6/d7/d29/d3b/da9/dbd/d103/f125 0 2024-04-23T02:14:10.575 INFO:tasks.workunit.client.0.smithi071.stdout:5/838: dread d3/d1d/d43/dd2/f5e [0,4194304] 0 2024-04-23T02:14:10.575 INFO:tasks.workunit.client.1.smithi192.stdout:3/832: dread d1/d3/d6/d7/d29/d3b/da9/daf/f11a [0,4194304] 0 2024-04-23T02:14:10.576 INFO:tasks.workunit.client.0.smithi071.stdout:5/839: chown d3/l4 448 1 2024-04-23T02:14:10.578 INFO:tasks.workunit.client.0.smithi071.stdout:9/719: symlink d1/d32/db9/lfc 0 2024-04-23T02:14:10.579 INFO:tasks.workunit.client.1.smithi192.stdout:9/725: write d0/d9/f36 [547670,117883] 0 2024-04-23T02:14:10.580 INFO:tasks.workunit.client.0.smithi071.stdout:7/968: creat d3/d17/d1c/d2a/d73/d83/de7/f156 x:0 0 0 2024-04-23T02:14:10.583 INFO:tasks.workunit.client.0.smithi071.stdout:0/792: truncate d0/d3/de/d10/d14/d32/d37/d26/fbe 428323 0 2024-04-23T02:14:10.583 INFO:tasks.workunit.client.1.smithi192.stdout:3/833: fsync d1/d3/d6/d7/d29/d3b/da9/d7b/f96 0 2024-04-23T02:14:10.584 INFO:tasks.workunit.client.1.smithi192.stdout:0/993: dread d6/df/d13/d6b/d73/d125/d137/f90 [0,4194304] 0 2024-04-23T02:14:10.584 INFO:tasks.workunit.client.1.smithi192.stdout:9/726: read d0/d9/fb [7894964,83055] 0 2024-04-23T02:14:10.585 INFO:tasks.workunit.client.0.smithi071.stdout:9/720: mkdir d1/dfd 0 2024-04-23T02:14:10.586 INFO:tasks.workunit.client.0.smithi071.stdout:7/969: symlink d3/d17/l157 0 2024-04-23T02:14:10.588 INFO:tasks.workunit.client.1.smithi192.stdout:0/994: rmdir d6/df/d13/d25 39 2024-04-23T02:14:10.588 INFO:tasks.workunit.client.0.smithi071.stdout:9/721: mkdir d1/d32/d41/db7/d5a/dc6/d9f/dfe 0 2024-04-23T02:14:10.589 INFO:tasks.workunit.client.0.smithi071.stdout:7/970: creat d3/d17/d41/de2/f158 x:0 0 0 2024-04-23T02:14:10.590 INFO:tasks.workunit.client.0.smithi071.stdout:0/793: link d0/d3/de/d10/d14/d32/d44/d31/d52/f91 d0/ddf/f112 0 2024-04-23T02:14:10.590 INFO:tasks.workunit.client.1.smithi192.stdout:9/727: creat d0/d9/d44/d51/d5b/dd3/ff0 x:0 0 0 2024-04-23T02:14:10.590 INFO:tasks.workunit.client.0.smithi071.stdout:5/840: dread d3/da/d1a/d1c/d46/d4e/dd8/ff7 [0,4194304] 0 2024-04-23T02:14:10.592 INFO:tasks.workunit.client.1.smithi192.stdout:3/834: creat d1/d3/d17/d1f/d33/d24/f126 x:0 0 0 2024-04-23T02:14:10.594 INFO:tasks.workunit.client.1.smithi192.stdout:9/728: creat d0/d9/d44/d58/d89/ff1 x:0 0 0 2024-04-23T02:14:10.594 INFO:tasks.workunit.client.0.smithi071.stdout:7/971: creat d3/d17/d41/d34/d39/d3b/f159 x:0 0 0 2024-04-23T02:14:10.597 INFO:tasks.workunit.client.0.smithi071.stdout:5/841: read d3/da/d1a/d1c/f41 [3208771,87604] 0 2024-04-23T02:14:10.599 INFO:tasks.workunit.client.1.smithi192.stdout:9/729: unlink d0/c29 0 2024-04-23T02:14:10.601 INFO:tasks.workunit.client.1.smithi192.stdout:0/995: mknod d6/df/d13/d25/d3a/d93/dd6/c14c 0 2024-04-23T02:14:10.601 INFO:tasks.workunit.client.1.smithi192.stdout:9/730: readlink d0/d9/d44/l7f 0 2024-04-23T02:14:10.601 INFO:tasks.workunit.client.0.smithi071.stdout:9/722: dwrite d1/d32/d3a/d94/ff5 [0,4194304] 0 2024-04-23T02:14:10.602 INFO:tasks.workunit.client.1.smithi192.stdout:0/996: fdatasync d6/df/d13/d25/d94/dd0/fd4 0 2024-04-23T02:14:10.603 INFO:tasks.workunit.client.0.smithi071.stdout:2/680: write d0/d2/d49/f5b [4701858,106002] 0 2024-04-23T02:14:10.605 INFO:tasks.workunit.client.1.smithi192.stdout:9/731: unlink d0/d9/d44/d51/d5b/d61/lc2 0 2024-04-23T02:14:10.607 INFO:tasks.workunit.client.1.smithi192.stdout:0/997: creat d6/df/d56/db1/d127/f14d x:0 0 0 2024-04-23T02:14:10.609 INFO:tasks.workunit.client.1.smithi192.stdout:5/875: dread d2/d2e/d4e/fe4 [0,4194304] 0 2024-04-23T02:14:10.610 INFO:tasks.workunit.client.1.smithi192.stdout:3/835: dwrite d1/d3/d6/f81 [0,4194304] 0 2024-04-23T02:14:10.610 INFO:tasks.workunit.client.1.smithi192.stdout:9/732: fsync d0/d9/d44/d4f/f41 0 2024-04-23T02:14:10.611 INFO:tasks.workunit.client.0.smithi071.stdout:5/842: creat d3/da/d1a/f136 x:0 0 0 2024-04-23T02:14:10.614 INFO:tasks.workunit.client.1.smithi192.stdout:9/733: write d0/d9/d44/d51/d5b/dd3/ff0 [527744,11522] 0 2024-04-23T02:14:10.616 INFO:tasks.workunit.client.0.smithi071.stdout:0/794: dwrite d0/d3/de/d48/d7e/d8b/fb6 [0,4194304] 0 2024-04-23T02:14:10.618 INFO:tasks.workunit.client.1.smithi192.stdout:5/876: getdents d2/db8 0 2024-04-23T02:14:10.618 INFO:tasks.workunit.client.0.smithi071.stdout:9/723: mknod d1/d4e/d66/cff 0 2024-04-23T02:14:10.619 INFO:tasks.workunit.client.1.smithi192.stdout:5/877: stat d2/db/d46/f11f 0 2024-04-23T02:14:10.620 INFO:tasks.workunit.client.0.smithi071.stdout:6/704: dread d6/d17/d77/d9f/dad/f42 [0,4194304] 0 2024-04-23T02:14:10.621 INFO:tasks.workunit.client.1.smithi192.stdout:3/836: link d1/f55 d1/d3/d17/d1a/d7c/dab/d6c/d84/d4c/de3/f127 0 2024-04-23T02:14:10.622 INFO:tasks.workunit.client.0.smithi071.stdout:8/806: dwrite dc/d15/d2d/f38 [0,4194304] 0 2024-04-23T02:14:10.622 INFO:tasks.workunit.client.1.smithi192.stdout:9/734: symlink d0/d9/dc7/da0/dd5/lf2 0 2024-04-23T02:14:10.623 INFO:tasks.workunit.client.0.smithi071.stdout:6/705: readlink d6/d17/d77/d9f/dad/lc4 0 2024-04-23T02:14:10.625 INFO:tasks.workunit.client.0.smithi071.stdout:5/843: chown d3/d34/d74/faa 6 1 2024-04-23T02:14:10.625 INFO:tasks.workunit.client.0.smithi071.stdout:0/795: fsync d0/d76/fab 0 2024-04-23T02:14:10.627 INFO:tasks.workunit.client.0.smithi071.stdout:9/724: rmdir d1/d4e 39 2024-04-23T02:14:10.630 INFO:tasks.workunit.client.0.smithi071.stdout:7/972: dwrite d3/d17/d1c/f3f [0,4194304] 0 2024-04-23T02:14:10.630 INFO:tasks.workunit.client.0.smithi071.stdout:0/796: chown d0/d3/de/d10/d14/d32/d44/d87/fa0 481 1 2024-04-23T02:14:10.633 INFO:tasks.workunit.client.1.smithi192.stdout:3/837: mknod d1/d3/d6/d7/d29/c128 0 2024-04-23T02:14:10.633 INFO:tasks.workunit.client.0.smithi071.stdout:7/973: dread - d3/d51/df5/d10d/f12e zero size 2024-04-23T02:14:10.633 INFO:tasks.workunit.client.0.smithi071.stdout:9/725: chown d1/d32/d41/db7/d39/l6e 278215646 1 2024-04-23T02:14:10.634 INFO:tasks.workunit.client.0.smithi071.stdout:1/680: dread db/d34/fc9 [0,4194304] 0 2024-04-23T02:14:10.642 INFO:tasks.workunit.client.0.smithi071.stdout:5/844: creat d3/d1d/d43/dd2/d5a/da7/f137 x:0 0 0 2024-04-23T02:14:10.643 INFO:tasks.workunit.client.0.smithi071.stdout:0/797: creat d0/d3/de/d48/d7e/d8b/de3/f113 x:0 0 0 2024-04-23T02:14:10.644 INFO:tasks.workunit.client.0.smithi071.stdout:1/681: mkdir db/d34/d8a/de1 0 2024-04-23T02:14:10.644 INFO:tasks.workunit.client.0.smithi071.stdout:9/726: write d1/fb [8369145,58907] 0 2024-04-23T02:14:10.645 INFO:tasks.workunit.client.0.smithi071.stdout:7/974: dwrite d3/d17/d41/d34/d39/f107 [0,4194304] 0 2024-04-23T02:14:10.645 INFO:tasks.workunit.client.1.smithi192.stdout:3/838: getdents d1/d3/d17 0 2024-04-23T02:14:10.645 INFO:tasks.workunit.client.1.smithi192.stdout:9/735: dwrite d0/d9/d44/d51/d5b/d61/f7b [0,4194304] 0 2024-04-23T02:14:10.646 INFO:tasks.workunit.client.0.smithi071.stdout:0/798: dread - d0/d3/de/d10/d14/d32/d44/fd6 zero size 2024-04-23T02:14:10.647 INFO:tasks.workunit.client.1.smithi192.stdout:0/998: dread d6/df/fc0 [0,4194304] 0 2024-04-23T02:14:10.649 INFO:tasks.workunit.client.0.smithi071.stdout:5/845: rmdir d3/da/df6 39 2024-04-23T02:14:10.650 INFO:tasks.workunit.client.0.smithi071.stdout:8/807: getdents dc/d56/d64 0 2024-04-23T02:14:10.651 INFO:tasks.workunit.client.1.smithi192.stdout:0/999: fsync d6/df/d47/f61 0 2024-04-23T02:14:10.658 INFO:tasks.workunit.client.0.smithi071.stdout:0/799: unlink d0/d3/de/d10/d14/d32/d37/d26/fbe 0 2024-04-23T02:14:10.658 INFO:tasks.workunit.client.0.smithi071.stdout:1/682: symlink db/df/d5f/d57/le2 0 2024-04-23T02:14:10.660 INFO:tasks.workunit.client.1.smithi192.stdout:3/839: link d1/d3/d17/d2b/d44/d57/fb4 d1/d3/d6/d7/d29/d3b/da9/d8b/dfc/f129 0 2024-04-23T02:14:10.661 INFO:tasks.workunit.client.0.smithi071.stdout:9/727: symlink d1/d4e/l100 0 2024-04-23T02:14:10.661 INFO:tasks.workunit.client.1.smithi192.stdout:5/878: dread d2/d32/f6e [0,4194304] 0 2024-04-23T02:14:10.662 INFO:tasks.workunit.client.0.smithi071.stdout:5/846: rename d3/da/d1a/db8/f2d to d3/da/d1a/d1c/d46/d4e/f138 0 2024-04-23T02:14:10.664 INFO:tasks.workunit.client.0.smithi071.stdout:1/683: chown db/df/d13/d20/d75/fc4 889302178 1 2024-04-23T02:14:10.666 INFO:tasks.workunit.client.1.smithi192.stdout:3/840: rename d1/d3/l74 to d1/d3/d17/d1a/d7c/dab/d6c/d84/d5d/l12a 0 2024-04-23T02:14:10.666 INFO:tasks.workunit.client.1.smithi192.stdout:5/879: mkdir d2/db/d16/d124 0 2024-04-23T02:14:10.666 INFO:tasks.workunit.client.0.smithi071.stdout:9/728: readlink d1/l7 0 2024-04-23T02:14:10.666 INFO:tasks.workunit.client.0.smithi071.stdout:0/800: getdents d0/d63 0 2024-04-23T02:14:10.667 INFO:tasks.workunit.client.0.smithi071.stdout:1/684: fsync db/df/d13/d2f/f3c 0 2024-04-23T02:14:10.667 INFO:tasks.workunit.client.1.smithi192.stdout:3/841: chown d1/d3/d17/d2b/d9a 0 1 2024-04-23T02:14:10.667 INFO:tasks.workunit.client.0.smithi071.stdout:0/801: rmdir d0/d3 39 2024-04-23T02:14:10.668 INFO:tasks.workunit.client.1.smithi192.stdout:5/880: unlink d2/db/d16/d48/lec 0 2024-04-23T02:14:10.669 INFO:tasks.workunit.client.0.smithi071.stdout:1/685: symlink db/d11/le3 0 2024-04-23T02:14:10.670 INFO:tasks.workunit.client.1.smithi192.stdout:3/842: unlink d1/d3/d6/d7/d29/d3b/da9/dbd/le6 0 2024-04-23T02:14:10.670 INFO:tasks.workunit.client.0.smithi071.stdout:9/729: creat d1/d32/d41/db7/d5a/dc6/d81/db6/f101 x:0 0 0 2024-04-23T02:14:10.671 INFO:tasks.workunit.client.1.smithi192.stdout:3/843: chown d1/d3/d6/d7/d5f 3650 1 2024-04-23T02:14:10.671 INFO:tasks.workunit.client.0.smithi071.stdout:1/686: creat db/d34/d8a/fe4 x:0 0 0 2024-04-23T02:14:10.672 INFO:tasks.workunit.client.1.smithi192.stdout:5/881: getdents d2/d6 0 2024-04-23T02:14:10.673 INFO:tasks.workunit.client.0.smithi071.stdout:9/730: symlink d1/d32/d41/db7/d39/df6/l102 0 2024-04-23T02:14:10.675 INFO:tasks.workunit.client.0.smithi071.stdout:1/687: mkdir db/df/d13/d20/de5 0 2024-04-23T02:14:10.677 INFO:tasks.workunit.client.0.smithi071.stdout:0/802: dwrite d0/d3/de/d10/db9/f10b [0,4194304] 0 2024-04-23T02:14:10.680 INFO:tasks.workunit.client.0.smithi071.stdout:1/688: creat db/df/d13/ddc/fe6 x:0 0 0 2024-04-23T02:14:10.681 INFO:tasks.workunit.client.0.smithi071.stdout:6/706: write d6/d17/d77/f96 [1349164,119588] 0 2024-04-23T02:14:10.684 INFO:tasks.workunit.client.0.smithi071.stdout:1/689: mkdir db/df/d3a/de7 0 2024-04-23T02:14:10.685 INFO:tasks.workunit.client.0.smithi071.stdout:9/731: link d1/d32/d41/db7/d39/f8f d1/d32/d41/db7/d5a/dc6/f103 0 2024-04-23T02:14:10.686 INFO:tasks.workunit.client.0.smithi071.stdout:0/803: dwrite d0/d3/de/d10/d14/d32/d37/d26/f9a [0,4194304] 0 2024-04-23T02:14:10.686 INFO:tasks.workunit.client.1.smithi192.stdout:7/878: sync 2024-04-23T02:14:10.688 INFO:tasks.workunit.client.0.smithi071.stdout:1/690: unlink db/df/d13/ld5 0 2024-04-23T02:14:10.690 INFO:tasks.workunit.client.1.smithi192.stdout:7/879: mkdir d5/d43/d4c/d4e/d101 0 2024-04-23T02:14:10.691 INFO:tasks.workunit.client.0.smithi071.stdout:9/732: mkdir d1/d4e/d54/d75/d104 0 2024-04-23T02:14:10.691 INFO:tasks.workunit.client.1.smithi192.stdout:7/880: fsync d5/d17/d1c/d59/d5a/d41/d50/d3d/fbf 0 2024-04-23T02:14:10.692 INFO:tasks.workunit.client.0.smithi071.stdout:7/975: write d3/d17/d41/d3a/da5/fbf [517434,58187] 0 2024-04-23T02:14:10.693 INFO:tasks.workunit.client.1.smithi192.stdout:7/881: chown d5/d17/d1c/d59/d5a/d41/d50/d3d/fbf 2 1 2024-04-23T02:14:10.694 INFO:tasks.workunit.client.0.smithi071.stdout:1/691: mkdir db/df/d13/d20/dc0/ddf/de8 0 2024-04-23T02:14:10.700 INFO:tasks.workunit.client.0.smithi071.stdout:0/804: dwrite d0/d3/de/d48/d7e/d8b/f22 [0,4194304] 0 2024-04-23T02:14:10.702 INFO:tasks.workunit.client.1.smithi192.stdout:9/736: symlink d0/d9/d44/d51/lf3 0 2024-04-23T02:14:10.703 INFO:tasks.workunit.client.0.smithi071.stdout:8/808: dwrite f6 [0,4194304] 0 2024-04-23T02:14:10.704 INFO:tasks.workunit.client.1.smithi192.stdout:9/737: chown d0/d9/d44/d74/l4b 9632376 1 2024-04-23T02:14:10.705 INFO:tasks.workunit.client.0.smithi071.stdout:1/692: creat db/d34/d8a/fe9 x:0 0 0 2024-04-23T02:14:10.708 INFO:tasks.workunit.client.0.smithi071.stdout:5/847: write d3/d8f/de6/f125 [1714891,76109] 0 2024-04-23T02:14:10.716 INFO:tasks.workunit.client.0.smithi071.stdout:5/848: mknod d3/da/d1a/d1c/db6/c139 0 2024-04-23T02:14:10.718 INFO:tasks.workunit.client.0.smithi071.stdout:8/809: dwrite dc/d15/d5d/dc6/fe7 [0,4194304] 0 2024-04-23T02:14:10.721 INFO:tasks.workunit.client.0.smithi071.stdout:6/707: write d6/d17/d77/d9f/dad/d3f/fb0 [911702,89550] 0 2024-04-23T02:14:10.721 INFO:tasks.workunit.client.0.smithi071.stdout:7/976: write d3/d51/d80/f9e [440517,118382] 0 2024-04-23T02:14:10.725 INFO:tasks.workunit.client.1.smithi192.stdout:3/844: dwrite d1/d3/d17/d1a/d7c/dab/d6c/d84/fcb [0,4194304] 0 2024-04-23T02:14:10.727 INFO:tasks.workunit.client.0.smithi071.stdout:5/849: write d3/d1d/daf/f89 [684593,57848] 0 2024-04-23T02:14:10.728 INFO:tasks.workunit.client.0.smithi071.stdout:0/805: dwrite d0/d3/de/d10/d14/d32/fa4 [0,4194304] 0 2024-04-23T02:14:10.730 INFO:tasks.workunit.client.0.smithi071.stdout:8/810: fsync dc/d15/da0/f1c 0 2024-04-23T02:14:10.730 INFO:tasks.workunit.client.0.smithi071.stdout:1/693: dwrite db/df/d13/d2f/d48/d85/fb0 [0,4194304] 0 2024-04-23T02:14:10.733 INFO:tasks.workunit.client.1.smithi192.stdout:3/845: rename d1/d3/d6/d7/d29/d3b/l3c to d1/d3/d17/d1a/d7c/dab/d6c/d84/l12b 0 2024-04-23T02:14:10.734 INFO:tasks.workunit.client.1.smithi192.stdout:7/882: dread d5/d17/d1c/d59/d5a/d70/de9/da6/da9/ff5 [0,4194304] 0 2024-04-23T02:14:10.739 INFO:tasks.workunit.client.1.smithi192.stdout:7/883: mkdir d5/d17/d1c/d57/d75/d102 0 2024-04-23T02:14:10.739 INFO:tasks.workunit.client.0.smithi071.stdout:8/811: write dc/d56/d64/f69 [2592076,119278] 0 2024-04-23T02:14:10.740 INFO:tasks.workunit.client.1.smithi192.stdout:3/846: truncate d1/d3/d17/d1f/d33/fad 439570 0 2024-04-23T02:14:10.743 INFO:tasks.workunit.client.0.smithi071.stdout:6/708: rename d6/c31 to d6/d17/d77/d9f/dad/da4/cdd 0 2024-04-23T02:14:10.743 INFO:tasks.workunit.client.0.smithi071.stdout:9/733: dwrite d1/d32/d41/db7/f55 [0,4194304] 0 2024-04-23T02:14:10.743 INFO:tasks.workunit.client.0.smithi071.stdout:5/850: mkdir d3/d10f/d13a 0 2024-04-23T02:14:10.744 INFO:tasks.workunit.client.1.smithi192.stdout:5/882: truncate d2/d6/d2d/f6b 2522707 0 2024-04-23T02:14:10.746 INFO:tasks.workunit.client.1.smithi192.stdout:7/884: symlink d5/d17/d1c/d57/d75/d102/l103 0 2024-04-23T02:14:10.747 INFO:tasks.workunit.client.0.smithi071.stdout:5/851: stat d3/d1d/d43/d4b/f72 0 2024-04-23T02:14:10.747 INFO:tasks.workunit.client.1.smithi192.stdout:4/940: sync 2024-04-23T02:14:10.748 INFO:tasks.workunit.client.0.smithi071.stdout:0/806: rename d0/d92/db2 to d0/d3/d95/d114 0 2024-04-23T02:14:10.749 INFO:tasks.workunit.client.1.smithi192.stdout:5/883: creat d2/d2e/df0/f125 x:0 0 0 2024-04-23T02:14:10.750 INFO:tasks.workunit.client.0.smithi071.stdout:0/807: read - d0/d3/de/d10/d14/d84/fd1 zero size 2024-04-23T02:14:10.750 INFO:tasks.workunit.client.1.smithi192.stdout:5/884: stat d2/d32/cc4 0 2024-04-23T02:14:10.751 INFO:tasks.workunit.client.0.smithi071.stdout:9/734: mkdir d1/d32/dc5/d105 0 2024-04-23T02:14:10.751 INFO:tasks.workunit.client.1.smithi192.stdout:7/885: creat d5/d43/d4c/f104 x:0 0 0 2024-04-23T02:14:10.752 INFO:tasks.workunit.client.0.smithi071.stdout:1/694: creat db/df/fea x:0 0 0 2024-04-23T02:14:10.754 INFO:tasks.workunit.client.1.smithi192.stdout:3/847: dread d1/d3/d17/d1a/d7c/dab/d6c/d84/d4c/fae [0,4194304] 0 2024-04-23T02:14:10.754 INFO:tasks.workunit.client.0.smithi071.stdout:6/709: dread d6/d17/d77/d9f/dad/f42 [0,4194304] 0 2024-04-23T02:14:10.758 INFO:tasks.workunit.client.0.smithi071.stdout:6/710: write d6/d1f/d5d/d95/fd0 [970284,12785] 0 2024-04-23T02:14:10.759 INFO:tasks.workunit.client.1.smithi192.stdout:7/886: mknod d5/d17/d1c/d57/d75/c105 0 2024-04-23T02:14:10.759 INFO:tasks.workunit.client.0.smithi071.stdout:1/695: creat db/df/d13/d20/d75/d96/feb x:0 0 0 2024-04-23T02:14:10.759 INFO:tasks.workunit.client.0.smithi071.stdout:0/808: creat d0/d3/de/d10/d14/d32/d44/f115 x:0 0 0 2024-04-23T02:14:10.760 INFO:tasks.workunit.client.0.smithi071.stdout:6/711: mkdir d6/d1f/d5d/dde 0 2024-04-23T02:14:10.761 INFO:tasks.workunit.client.1.smithi192.stdout:4/941: dwrite de/d15/d19/d1a/d95/dd5/f10e [0,4194304] 0 2024-04-23T02:14:10.761 INFO:tasks.workunit.client.1.smithi192.stdout:3/848: dread - d1/d3/d6/d7/d29/d3b/da9/dbd/d103/f108 zero size 2024-04-23T02:14:10.763 INFO:tasks.workunit.client.1.smithi192.stdout:9/738: dwrite d0/f54 [0,4194304] 0 2024-04-23T02:14:10.763 INFO:tasks.workunit.client.0.smithi071.stdout:6/712: rename d6/d17/d77/faf to d6/d17/fdf 0 2024-04-23T02:14:10.764 INFO:tasks.workunit.client.0.smithi071.stdout:1/696: dread db/f22 [0,4194304] 0 2024-04-23T02:14:10.765 INFO:tasks.workunit.client.0.smithi071.stdout:0/809: getdents d0/d3 0 2024-04-23T02:14:10.766 INFO:tasks.workunit.client.0.smithi071.stdout:0/810: fdatasync d0/fda 0 2024-04-23T02:14:10.766 INFO:tasks.workunit.client.0.smithi071.stdout:0/811: chown d0/d3/de 2131 1 2024-04-23T02:14:10.767 INFO:tasks.workunit.client.1.smithi192.stdout:7/887: symlink d5/d43/l106 0 2024-04-23T02:14:10.769 INFO:tasks.workunit.client.1.smithi192.stdout:4/942: symlink de/d15/d53/d81/de3/da3/l13c 0 2024-04-23T02:14:10.771 INFO:tasks.workunit.client.1.smithi192.stdout:3/849: chown d1/d3/d6/d7/d29/d3b/da9/dbd/fc9 24557341 1 2024-04-23T02:14:10.773 INFO:tasks.workunit.client.0.smithi071.stdout:9/735: dread d1/d32/d3a/d77/f88 [0,4194304] 0 2024-04-23T02:14:10.774 INFO:tasks.workunit.client.0.smithi071.stdout:1/697: dwrite db/df/d13/d2f/fb6 [0,4194304] 0 2024-04-23T02:14:10.774 INFO:tasks.workunit.client.0.smithi071.stdout:9/736: chown d1/f2b 416178604 1 2024-04-23T02:14:10.776 INFO:tasks.workunit.client.1.smithi192.stdout:6/964: sync 2024-04-23T02:14:10.777 INFO:tasks.workunit.client.1.smithi192.stdout:1/745: sync 2024-04-23T02:14:10.778 INFO:tasks.workunit.client.1.smithi192.stdout:9/739: dwrite d0/fd1 [0,4194304] 0 2024-04-23T02:14:10.778 INFO:tasks.workunit.client.0.smithi071.stdout:1/698: truncate db/df/d13/d2f/f3e 2483304 0 2024-04-23T02:14:10.779 INFO:tasks.workunit.client.1.smithi192.stdout:6/965: fdatasync d3/d28/df8/d52/f67 0 2024-04-23T02:14:10.779 INFO:tasks.workunit.client.0.smithi071.stdout:9/737: symlink d1/d32/d41/db7/d5a/dc6/d81/l106 0 2024-04-23T02:14:10.781 INFO:tasks.workunit.client.1.smithi192.stdout:3/850: symlink d1/d3/d17/d2b/d44/d57/db0/l12c 0 2024-04-23T02:14:10.781 INFO:tasks.workunit.client.0.smithi071.stdout:6/713: dread d6/d17/d77/da2/fa7 [0,4194304] 0 2024-04-23T02:14:10.785 INFO:tasks.workunit.client.0.smithi071.stdout:9/738: fdatasync d1/d32/d3a/d5f/f63 0 2024-04-23T02:14:10.786 INFO:tasks.workunit.client.1.smithi192.stdout:3/851: stat d1/d3/d17/d1a/d7c/dab/d6c/fa2 0 2024-04-23T02:14:10.786 INFO:tasks.workunit.client.0.smithi071.stdout:6/714: write d6/d17/d77/da2/fc6 [674333,84756] 0 2024-04-23T02:14:10.787 INFO:tasks.workunit.client.1.smithi192.stdout:6/966: rename d3/c4 to d3/d28/d79/dbe/dcb/c13e 0 2024-04-23T02:14:10.787 INFO:tasks.workunit.client.0.smithi071.stdout:8/812: write dc/d56/d64/fbd [980690,2048] 0 2024-04-23T02:14:10.788 INFO:tasks.workunit.client.1.smithi192.stdout:3/852: chown d1/d3/d17/d1a/d7c/dab/d6c/d84/d4c 556 1 2024-04-23T02:14:10.789 INFO:tasks.workunit.client.1.smithi192.stdout:9/740: mknod d0/d9/dc7/da0/dd5/cf4 0 2024-04-23T02:14:10.789 INFO:tasks.workunit.client.0.smithi071.stdout:7/977: dwrite d3/d17/d41/d34/f89 [0,4194304] 0 2024-04-23T02:14:10.794 INFO:tasks.workunit.client.0.smithi071.stdout:6/715: creat d6/d17/d77/fe0 x:0 0 0 2024-04-23T02:14:10.795 INFO:tasks.workunit.client.0.smithi071.stdout:9/739: creat d1/d32/d41/db7/d39/d42/f107 x:0 0 0 2024-04-23T02:14:10.796 INFO:tasks.workunit.client.0.smithi071.stdout:6/716: truncate d6/d1f/d5d/d95/f98 4778974 0 2024-04-23T02:14:10.796 INFO:tasks.workunit.client.1.smithi192.stdout:9/741: write d0/d9/db3/d9e/fe0 [1337239,49978] 0 2024-04-23T02:14:10.796 INFO:tasks.workunit.client.0.smithi071.stdout:9/740: chown d1/d32/d7e/ca1 0 1 2024-04-23T02:14:10.797 INFO:tasks.workunit.client.0.smithi071.stdout:9/741: chown d1 41894 1 2024-04-23T02:14:10.798 INFO:tasks.workunit.client.0.smithi071.stdout:6/717: symlink d6/d1f/d4e/le1 0 2024-04-23T02:14:10.798 INFO:tasks.workunit.client.1.smithi192.stdout:9/742: truncate d0/d98/fca 1132688 0 2024-04-23T02:14:10.799 INFO:tasks.workunit.client.0.smithi071.stdout:9/742: write d1/d32/d41/db7/d39/d42/f107 [909412,24208] 0 2024-04-23T02:14:10.801 INFO:tasks.workunit.client.1.smithi192.stdout:5/885: truncate d2/db/d2b/f3a 2980358 0 2024-04-23T02:14:10.802 INFO:tasks.workunit.client.0.smithi071.stdout:6/718: write d6/d17/d77/d9f/dad/d3f/f47 [568277,67214] 0 2024-04-23T02:14:10.805 INFO:tasks.workunit.client.1.smithi192.stdout:9/743: creat d0/d9/d44/d51/ff5 x:0 0 0 2024-04-23T02:14:10.806 INFO:tasks.workunit.client.1.smithi192.stdout:3/853: getdents d1/d3/d17/d65/d99 0 2024-04-23T02:14:10.808 INFO:tasks.workunit.client.0.smithi071.stdout:5/852: dwrite d3/da/d1a/db8/f8a [4194304,4194304] 0 2024-04-23T02:14:10.809 INFO:tasks.workunit.client.1.smithi192.stdout:6/967: write d3/d28/f9a [2546504,71395] 0 2024-04-23T02:14:10.811 INFO:tasks.workunit.client.0.smithi071.stdout:3/730: sync 2024-04-23T02:14:10.811 INFO:tasks.workunit.client.0.smithi071.stdout:4/810: sync 2024-04-23T02:14:10.813 INFO:tasks.workunit.client.0.smithi071.stdout:0/812: dwrite d0/d3/de/f1d [0,4194304] 0 2024-04-23T02:14:10.818 INFO:tasks.workunit.client.1.smithi192.stdout:6/968: fsync d3/d18/d58/f131 0 2024-04-23T02:14:10.818 INFO:tasks.workunit.client.0.smithi071.stdout:6/719: dwrite d6/d17/d77/d9e/fcd [0,4194304] 0 2024-04-23T02:14:10.818 INFO:tasks.workunit.client.0.smithi071.stdout:5/853: mkdir d3/d1d/daf/d10/da5/d5f/dc0/dfa/d13b 0 2024-04-23T02:14:10.818 INFO:tasks.workunit.client.0.smithi071.stdout:4/811: fdatasync d2/d2a/da2/dd1/dde/d23/d4f/fd7 0 2024-04-23T02:14:10.821 INFO:tasks.workunit.client.0.smithi071.stdout:3/731: mkdir d7/d22/d23/d26/d7e/d103 0 2024-04-23T02:14:10.821 INFO:tasks.workunit.client.1.smithi192.stdout:5/886: dwrite d2/d6/fd5 [4194304,4194304] 0 2024-04-23T02:14:10.822 INFO:tasks.workunit.client.0.smithi071.stdout:1/699: write db/df/d13/f71 [2285814,83019] 0 2024-04-23T02:14:10.822 INFO:tasks.workunit.client.1.smithi192.stdout:1/746: dwrite dc/f29 [0,4194304] 0 2024-04-23T02:14:10.823 INFO:tasks.workunit.client.0.smithi071.stdout:9/743: write d1/d32/d3a/d5f/f63 [3696753,108806] 0 2024-04-23T02:14:10.823 INFO:tasks.workunit.client.1.smithi192.stdout:7/888: dwrite d5/d17/d1c/d59/d5a/d41/fab [0,4194304] 0 2024-04-23T02:14:10.825 INFO:tasks.workunit.client.0.smithi071.stdout:3/732: chown d7/d8/f16 42 1 2024-04-23T02:14:10.825 INFO:tasks.workunit.client.0.smithi071.stdout:5/854: creat d3/d1d/daf/d10/da5/f13c x:0 0 0 2024-04-23T02:14:10.825 INFO:tasks.workunit.client.0.smithi071.stdout:1/700: dread - db/d34/d8a/fe9 zero size 2024-04-23T02:14:10.826 INFO:tasks.workunit.client.0.smithi071.stdout:9/744: fdatasync d1/d32/d3a/d94/ff5 0 2024-04-23T02:14:10.826 INFO:tasks.workunit.client.1.smithi192.stdout:1/747: write dc/d16/d19/d74/faf [262296,95901] 0 2024-04-23T02:14:10.827 INFO:tasks.workunit.client.0.smithi071.stdout:0/813: mknod d0/d3/de/d10/db3/c116 0 2024-04-23T02:14:10.829 INFO:tasks.workunit.client.0.smithi071.stdout:5/855: fdatasync d3/d1d/daf/d83/fc9 0 2024-04-23T02:14:10.831 INFO:tasks.workunit.client.0.smithi071.stdout:3/733: mkdir d7/da5/db2/db7/d68/d3b/d98/d104 0 2024-04-23T02:14:10.831 INFO:tasks.workunit.client.1.smithi192.stdout:1/748: write dc/d16/d19/f3a [5061166,93041] 0 2024-04-23T02:14:10.832 INFO:tasks.workunit.client.1.smithi192.stdout:5/887: chown d2/db/d2b/f7f 207 1 2024-04-23T02:14:10.832 INFO:tasks.workunit.client.1.smithi192.stdout:5/888: stat d2/f95 0 2024-04-23T02:14:10.834 INFO:tasks.workunit.client.0.smithi071.stdout:0/814: mknod d0/d3/de/d10/d14/d32/d37/d26/c117 0 2024-04-23T02:14:10.834 INFO:tasks.workunit.client.0.smithi071.stdout:6/720: read d6/d22/f28 [3888716,114303] 0 2024-04-23T02:14:10.838 INFO:tasks.workunit.client.0.smithi071.stdout:0/815: write d0/d3/de/d10/d14/d84/ff1 [996824,92480] 0 2024-04-23T02:14:10.838 INFO:tasks.workunit.client.0.smithi071.stdout:5/856: dread d3/d1d/d43/f65 [0,4194304] 0 2024-04-23T02:14:10.839 INFO:tasks.workunit.client.0.smithi071.stdout:6/721: readlink d6/d17/d77/d9f/dad/d3f/l66 0 2024-04-23T02:14:10.840 INFO:tasks.workunit.client.0.smithi071.stdout:0/816: creat d0/ddf/f118 x:0 0 0 2024-04-23T02:14:10.841 INFO:tasks.workunit.client.1.smithi192.stdout:1/749: symlink dc/d16/d19/d74/dd2/lf0 0 2024-04-23T02:14:10.841 INFO:tasks.workunit.client.0.smithi071.stdout:4/812: link d2/d2a/da2/dd1/dde/d23/d29/c6e d2/d2a/da2/dd1/dde/d23/d29/c109 0 2024-04-23T02:14:10.842 INFO:tasks.workunit.client.0.smithi071.stdout:9/745: creat d1/d32/d41/db7/d39/d42/f108 x:0 0 0 2024-04-23T02:14:10.843 INFO:tasks.workunit.client.0.smithi071.stdout:3/734: truncate d7/d22/d23/d26/d7e/ffa 1406489 0 2024-04-23T02:14:10.845 INFO:tasks.workunit.client.0.smithi071.stdout:4/813: fsync d2/d2a/d2d/d39/d42/d49/d91/fc5 0 2024-04-23T02:14:10.846 INFO:tasks.workunit.client.0.smithi071.stdout:0/817: symlink d0/d3/de/d10/d14/d32/d44/d31/l119 0 2024-04-23T02:14:10.848 INFO:tasks.workunit.client.0.smithi071.stdout:3/735: mkdir d7/da5/db5/d105 0 2024-04-23T02:14:10.849 INFO:tasks.workunit.client.0.smithi071.stdout:6/722: fsync d6/d1f/d5d/f6f 0 2024-04-23T02:14:10.849 INFO:tasks.workunit.client.0.smithi071.stdout:3/736: dread - d7/d1d/fd6 zero size 2024-04-23T02:14:10.850 INFO:tasks.workunit.client.0.smithi071.stdout:4/814: unlink d2/d2a/d2d/d39/ff3 0 2024-04-23T02:14:10.851 INFO:tasks.workunit.client.0.smithi071.stdout:5/857: link d3/d1d/d43/d4b/le8 d3/da/d1a/d1c/l13d 0 2024-04-23T02:14:10.852 INFO:tasks.workunit.client.1.smithi192.stdout:3/854: dwrite d1/d3/d6/d7/f2a [0,4194304] 0 2024-04-23T02:14:10.854 INFO:tasks.workunit.client.1.smithi192.stdout:6/969: dwrite d3/d18/f2d [0,4194304] 0 2024-04-23T02:14:10.855 INFO:tasks.workunit.client.0.smithi071.stdout:3/737: creat d7/d22/d23/dba/df4/dca/f106 x:0 0 0 2024-04-23T02:14:10.856 INFO:tasks.workunit.client.0.smithi071.stdout:6/723: rename d6/d17/d77/d9f/dad/da4 to d6/d17/d77/d9f/dad/d8f/de2 0 2024-04-23T02:14:10.858 INFO:tasks.workunit.client.1.smithi192.stdout:1/750: dwrite dc/d16/db0/fc7 [0,4194304] 0 2024-04-23T02:14:10.858 INFO:tasks.workunit.client.1.smithi192.stdout:3/855: fdatasync d1/d3/d6/d7/d29/d3b/da9/daf/f101 0 2024-04-23T02:14:10.863 INFO:tasks.workunit.client.0.smithi071.stdout:6/724: mknod d6/d17/d77/d9f/dad/d8f/ce3 0 2024-04-23T02:14:10.866 INFO:tasks.workunit.client.1.smithi192.stdout:1/751: creat dc/d8b/d44/ff1 x:0 0 0 2024-04-23T02:14:10.867 INFO:tasks.workunit.client.0.smithi071.stdout:5/858: rename d3/d34/d74/fdd to d3/d1d/daf/f13e 0 2024-04-23T02:14:10.869 INFO:tasks.workunit.client.0.smithi071.stdout:6/725: creat d6/d17/d77/d9f/dad/d8f/de2/fe4 x:0 0 0 2024-04-23T02:14:10.869 INFO:tasks.workunit.client.1.smithi192.stdout:6/970: link d3/feb d3/d28/d79/da0/dc2/d101/d80/d107/d110/f13f 0 2024-04-23T02:14:10.870 INFO:tasks.workunit.client.0.smithi071.stdout:6/726: stat d6/d1f/d5d/l72 0 2024-04-23T02:14:10.871 INFO:tasks.workunit.client.0.smithi071.stdout:6/727: unlink d6/d1f/f3b 0 2024-04-23T02:14:10.872 INFO:tasks.workunit.client.1.smithi192.stdout:3/856: dwrite d1/d3/d17/fa8 [4194304,4194304] 0 2024-04-23T02:14:10.872 INFO:tasks.workunit.client.0.smithi071.stdout:6/728: stat d6/d17/d77/d9f/dad/f5e 0 2024-04-23T02:14:10.873 INFO:tasks.workunit.client.0.smithi071.stdout:6/729: dread - d6/d17/d77/d9f/dad/f5e zero size 2024-04-23T02:14:10.874 INFO:tasks.workunit.client.0.smithi071.stdout:5/859: read d3/da/d1a/d1c/f41 [337814,115041] 0 2024-04-23T02:14:10.876 INFO:tasks.workunit.client.1.smithi192.stdout:6/971: creat d3/d15/d1b/dcf/f140 x:0 0 0 2024-04-23T02:14:10.877 INFO:tasks.workunit.client.0.smithi071.stdout:6/730: dread d6/d17/d77/da2/fc6 [0,4194304] 0 2024-04-23T02:14:10.877 INFO:tasks.workunit.client.0.smithi071.stdout:5/860: unlink d3/da/d1a/db8/l68 0 2024-04-23T02:14:10.880 INFO:tasks.workunit.client.1.smithi192.stdout:3/857: link d1/d3/d17/d1f/cc8 d1/d3/d17/d1a/d7c/dab/d6c/d84/c12d 0 2024-04-23T02:14:10.880 INFO:tasks.workunit.client.0.smithi071.stdout:6/731: mkdir d6/d22/d92/de5 0 2024-04-23T02:14:10.882 INFO:tasks.workunit.client.1.smithi192.stdout:3/858: dread - d1/d3/d17/d65/fe5 zero size 2024-04-23T02:14:10.884 INFO:tasks.workunit.client.1.smithi192.stdout:9/744: dread d0/d9/d44/d58/fbd [0,4194304] 0 2024-04-23T02:14:10.885 INFO:tasks.workunit.client.1.smithi192.stdout:6/972: dwrite d3/d18/dd4/d8d/fba [0,4194304] 0 2024-04-23T02:14:10.889 INFO:tasks.workunit.client.0.smithi071.stdout:6/732: dwrite d6/d22/f41 [0,4194304] 0 2024-04-23T02:14:10.890 INFO:tasks.workunit.client.1.smithi192.stdout:6/973: creat d3/d28/df8/d7c/dda/f141 x:0 0 0 2024-04-23T02:14:10.891 INFO:tasks.workunit.client.1.smithi192.stdout:3/859: creat d1/d3/d6/d7/f12e x:0 0 0 2024-04-23T02:14:10.892 INFO:tasks.workunit.client.0.smithi071.stdout:5/861: dwrite d3/d1d/d43/dd2/da9/f134 [0,4194304] 0 2024-04-23T02:14:10.893 INFO:tasks.workunit.client.1.smithi192.stdout:9/745: dread d0/d98/fca [0,4194304] 0 2024-04-23T02:14:10.894 INFO:tasks.workunit.client.1.smithi192.stdout:6/974: write d3/d28/d79/da0/dc2/d101/d80/f10d [757588,114629] 0 2024-04-23T02:14:10.894 INFO:tasks.workunit.client.1.smithi192.stdout:3/860: dread - d1/d3/d6/fa6 zero size 2024-04-23T02:14:10.898 INFO:tasks.workunit.client.0.smithi071.stdout:5/862: mkdir d3/da/df6/d13f 0 2024-04-23T02:14:10.898 INFO:tasks.workunit.client.1.smithi192.stdout:6/975: readlink d3/d18/d91/lf3 0 2024-04-23T02:14:10.899 INFO:tasks.workunit.client.0.smithi071.stdout:5/863: fdatasync d3/da/d1a/d6d/f9a 0 2024-04-23T02:14:10.902 INFO:tasks.workunit.client.0.smithi071.stdout:5/864: fsync d3/d1d/daf/d10/fde 0 2024-04-23T02:14:10.905 INFO:tasks.workunit.client.1.smithi192.stdout:3/861: dread d1/d3/d17/d1f/d33/d24/f122 [0,4194304] 0 2024-04-23T02:14:10.905 INFO:tasks.workunit.client.0.smithi071.stdout:5/865: read d3/da/d1a/d6d/fbe [77861,9961] 0 2024-04-23T02:14:10.905 INFO:tasks.workunit.client.0.smithi071.stdout:5/866: readlink d3/l4 0 2024-04-23T02:14:10.906 INFO:tasks.workunit.client.1.smithi192.stdout:6/976: rename d3/d28/d79/da0/dc2/d101/d80/la3 to d3/d18/l142 0 2024-04-23T02:14:10.906 INFO:tasks.workunit.client.0.smithi071.stdout:5/867: dread - d3/da/d1a/f136 zero size 2024-04-23T02:14:10.913 INFO:tasks.workunit.client.1.smithi192.stdout:6/977: mknod d3/d28/d79/da0/dc2/d101/c143 0 2024-04-23T02:14:10.914 INFO:tasks.workunit.client.1.smithi192.stdout:5/889: write d2/d6/d2d/d41/d43/dbc/fc1 [429631,111046] 0 2024-04-23T02:14:10.916 INFO:tasks.workunit.client.0.smithi071.stdout:8/813: dwrite dc/d78/f8f [4194304,4194304] 0 2024-04-23T02:14:10.917 INFO:tasks.workunit.client.0.smithi071.stdout:8/814: write dc/d15/f86 [3972985,111932] 0 2024-04-23T02:14:10.918 INFO:tasks.workunit.client.1.smithi192.stdout:7/889: dwrite d5/d17/d1c/f45 [0,4194304] 0 2024-04-23T02:14:10.918 INFO:tasks.workunit.client.1.smithi192.stdout:5/890: dread - d2/db/d16/d48/d4f/f11c zero size 2024-04-23T02:14:10.918 INFO:tasks.workunit.client.1.smithi192.stdout:3/862: dwrite d1/d3/d6/f10 [0,4194304] 0 2024-04-23T02:14:10.921 INFO:tasks.workunit.client.1.smithi192.stdout:7/890: chown d5/d43/d4c/cd6 4 1 2024-04-23T02:14:10.922 INFO:tasks.workunit.client.1.smithi192.stdout:7/891: dread - d5/d43/d4c/f104 zero size 2024-04-23T02:14:10.928 INFO:tasks.workunit.client.1.smithi192.stdout:9/746: dread d0/d9/dc7/fc6 [0,4194304] 0 2024-04-23T02:14:10.929 INFO:tasks.workunit.client.1.smithi192.stdout:6/978: symlink d3/d28/d79/da0/dc2/d101/d80/d8e/l144 0 2024-04-23T02:14:10.929 INFO:tasks.workunit.client.1.smithi192.stdout:3/863: mkdir d1/d3/d6/d7/d29/d3b/da9/daf/ddf/d12f 0 2024-04-23T02:14:10.934 INFO:tasks.workunit.client.1.smithi192.stdout:5/891: truncate d2/d6/d2d/d41/d43/d98/f107 49158 0 2024-04-23T02:14:10.936 INFO:tasks.workunit.client.1.smithi192.stdout:5/892: write d2/d6/f3b [1469798,81940] 0 2024-04-23T02:14:10.940 INFO:tasks.workunit.client.1.smithi192.stdout:3/864: creat d1/d3/d17/d2b/f130 x:0 0 0 2024-04-23T02:14:10.941 INFO:tasks.workunit.client.1.smithi192.stdout:5/893: mknod d2/d6/d2d/d41/d43/d98/c126 0 2024-04-23T02:14:10.943 INFO:tasks.workunit.client.1.smithi192.stdout:3/865: read - d1/d3/d17/dd1/fe9 zero size 2024-04-23T02:14:10.945 INFO:tasks.workunit.client.1.smithi192.stdout:9/747: dwrite d0/d9/d44/d51/d5b/d61/db1/f93 [8388608,4194304] 0 2024-04-23T02:14:10.945 INFO:tasks.workunit.client.1.smithi192.stdout:5/894: symlink d2/db/d2b/d10d/l127 0 2024-04-23T02:14:10.956 INFO:tasks.workunit.client.1.smithi192.stdout:5/895: dwrite d2/d6/f59 [4194304,4194304] 0 2024-04-23T02:14:10.957 INFO:tasks.workunit.client.1.smithi192.stdout:7/892: dread d5/d17/d1c/d59/d5a/d70/de9/da6/da9/fc3 [0,4194304] 0 2024-04-23T02:14:10.957 INFO:tasks.workunit.client.1.smithi192.stdout:3/866: dwrite d1/d3/d17/d1a/d7c/dab/d6c/d84/fcb [0,4194304] 0 2024-04-23T02:14:10.966 INFO:tasks.workunit.client.1.smithi192.stdout:5/896: creat d2/db/d16/d48/d4f/f128 x:0 0 0 2024-04-23T02:14:10.966 INFO:tasks.workunit.client.1.smithi192.stdout:7/893: getdents d5/d17 0 2024-04-23T02:14:10.967 INFO:tasks.workunit.client.1.smithi192.stdout:3/867: truncate d1/d3/d6/d7/d71/f11b 668233 0 2024-04-23T02:14:10.968 INFO:tasks.workunit.client.1.smithi192.stdout:3/868: symlink d1/d3/d17/d1a/d7c/dab/d6c/d84/d5d/l131 2 2024-04-23T02:14:10.968 INFO:tasks.workunit.client.1.smithi192.stdout:5/897: truncate d2/db/d46/fe7 834719 0 2024-04-23T02:14:10.970 INFO:tasks.workunit.client.1.smithi192.stdout:7/894: chown d5/d17/d1c/d59/d5a/d70/de9/da6/ddf/ceb 27226952 1 2024-04-23T02:14:10.971 INFO:tasks.workunit.client.1.smithi192.stdout:3/869: symlink d1/d3/d17/d1f/d33/dd6/d11f/l132 0 2024-04-23T02:14:10.972 INFO:tasks.workunit.client.0.smithi071.stdout:4/815: dread d2/d2a/da2/dd1/dde/f61 [4194304,4194304] 0 2024-04-23T02:14:10.972 INFO:tasks.workunit.client.1.smithi192.stdout:7/895: chown d5/d17/d1c/d59/d5a/d70/de9 1 1 2024-04-23T02:14:10.976 INFO:tasks.workunit.client.1.smithi192.stdout:5/898: getdents d2/d6/d2d/d41/d43 0 2024-04-23T02:14:10.976 INFO:tasks.workunit.client.0.smithi071.stdout:1/701: dwrite db/d34/fc9 [0,4194304] 0 2024-04-23T02:14:10.977 INFO:tasks.workunit.client.1.smithi192.stdout:7/896: mkdir d5/d17/d1c/d59/d5a/d70/de9/da6/da9/db9/dc0/d107 0 2024-04-23T02:14:10.977 INFO:tasks.workunit.client.1.smithi192.stdout:5/899: dread - d2/d2e/df0/f104 zero size 2024-04-23T02:14:10.978 INFO:tasks.workunit.client.1.smithi192.stdout:5/900: chown d2/db/de1 1658 1 2024-04-23T02:14:10.979 INFO:tasks.workunit.client.0.smithi071.stdout:5/868: rename d3/d1d/d43 to d3/da/d1a/db8/d140 0 2024-04-23T02:14:10.979 INFO:tasks.workunit.client.0.smithi071.stdout:9/746: dread d1/f10 [0,4194304] 0 2024-04-23T02:14:10.980 INFO:tasks.workunit.client.1.smithi192.stdout:7/897: stat d5/d17/d1c/d59/d5a/d41/d50/l39 0 2024-04-23T02:14:10.980 INFO:tasks.workunit.client.1.smithi192.stdout:3/870: rename d1/d3/d17/d1a/d7c/dab/d6c/d84/c12d to d1/d3/d17/df3/c133 0 2024-04-23T02:14:10.980 INFO:tasks.workunit.client.0.smithi071.stdout:4/816: fsync d2/d2a/d2d/d39/d42/d49/d67/d7e/f81 0 2024-04-23T02:14:10.981 INFO:tasks.workunit.client.1.smithi192.stdout:5/901: mkdir d2/db/d16/d48/dc5/d129 0 2024-04-23T02:14:10.982 INFO:tasks.workunit.client.0.smithi071.stdout:5/869: symlink d3/d86/l141 0 2024-04-23T02:14:10.983 INFO:tasks.workunit.client.1.smithi192.stdout:3/871: symlink d1/d3/d17/d1a/d7c/dab/d6c/d84/d4c/l134 0 2024-04-23T02:14:10.985 INFO:tasks.workunit.client.0.smithi071.stdout:9/747: mknod d1/d32/d41/db7/d5a/dc6/d81/c109 0 2024-04-23T02:14:10.986 INFO:tasks.workunit.client.1.smithi192.stdout:5/902: symlink d2/db/d16/d48/d4f/dd1/l12a 0 2024-04-23T02:14:10.988 INFO:tasks.workunit.client.1.smithi192.stdout:3/872: dread - d1/d3/d6/d7/d29/d3b/da9/daf/ddf/f107 zero size 2024-04-23T02:14:10.989 INFO:tasks.workunit.client.1.smithi192.stdout:7/898: link d5/d17/d1c/d59/fd0 d5/d17/d1c/d59/f108 0 2024-04-23T02:14:10.989 INFO:tasks.workunit.client.0.smithi071.stdout:9/748: truncate d1/d32/f40 1440807 0 2024-04-23T02:14:10.991 INFO:tasks.workunit.client.1.smithi192.stdout:5/903: creat d2/d6/d2d/d41/f12b x:0 0 0 2024-04-23T02:14:10.991 INFO:tasks.workunit.client.0.smithi071.stdout:5/870: creat d3/d10f/d13a/f142 x:0 0 0 2024-04-23T02:14:10.992 INFO:tasks.workunit.client.0.smithi071.stdout:3/738: truncate d7/d22/d23/f6b 355919 0 2024-04-23T02:14:10.993 INFO:tasks.workunit.client.1.smithi192.stdout:3/873: unlink d1/d3/d17/d1f/d33/l4e 0 2024-04-23T02:14:10.993 INFO:tasks.workunit.client.0.smithi071.stdout:0/818: dwrite d0/d3/de/d10/d14/d32/d8a/df0/f55 [0,4194304] 0 2024-04-23T02:14:10.998 INFO:tasks.workunit.client.1.smithi192.stdout:3/874: symlink d1/d3/d17/d2b/d44/d57/l135 0 2024-04-23T02:14:10.998 INFO:tasks.workunit.client.1.smithi192.stdout:7/899: link d5/l16 d5/d17/d1c/d59/d5a/d41/l109 0 2024-04-23T02:14:10.998 INFO:tasks.workunit.client.0.smithi071.stdout:3/739: creat d7/d22/d23/dba/df4/f107 x:0 0 0 2024-04-23T02:14:11.001 INFO:tasks.workunit.client.0.smithi071.stdout:9/749: rename d1/d32/d41/db7/c2e to d1/d32/dc5/dd8/c10a 0 2024-04-23T02:14:11.001 INFO:tasks.workunit.client.1.smithi192.stdout:3/875: mkdir d1/d3/d17/df3/d136 0 2024-04-23T02:14:11.002 INFO:tasks.workunit.client.0.smithi071.stdout:0/819: symlink d0/d3/de/d10/db9/d106/l11a 0 2024-04-23T02:14:11.004 INFO:tasks.workunit.client.0.smithi071.stdout:1/702: dread db/df/d3a/f44 [0,4194304] 0 2024-04-23T02:14:11.007 INFO:tasks.workunit.client.0.smithi071.stdout:0/820: truncate d0/d3/de/d10/d14/f85 507850 0 2024-04-23T02:14:11.007 INFO:tasks.workunit.client.0.smithi071.stdout:1/703: chown db/df/d5f/f82 502702553 1 2024-04-23T02:14:11.007 INFO:tasks.workunit.client.1.smithi192.stdout:9/748: dread d0/d9/f37 [0,4194304] 0 2024-04-23T02:14:11.009 INFO:tasks.workunit.client.1.smithi192.stdout:9/749: truncate d0/d9/d44/d53/fcc 350335 0 2024-04-23T02:14:11.012 INFO:tasks.workunit.client.1.smithi192.stdout:9/750: rename d0/d9/d44/d74/l99 to d0/d9/d44/d58/dd9/lf6 0 2024-04-23T02:14:11.012 INFO:tasks.workunit.client.0.smithi071.stdout:1/704: mkdir db/df/dc2/dec 0 2024-04-23T02:14:11.014 INFO:tasks.workunit.client.1.smithi192.stdout:7/900: dread d5/d17/d1c/d57/f48 [0,4194304] 0 2024-04-23T02:14:11.017 INFO:tasks.workunit.client.0.smithi071.stdout:1/705: link db/d34/d68/l6c db/d34/d8a/led 0 2024-04-23T02:14:11.018 INFO:tasks.workunit.client.1.smithi192.stdout:5/904: dread d2/d6/d2d/d41/f97 [0,4194304] 0 2024-04-23T02:14:11.019 INFO:tasks.workunit.client.0.smithi071.stdout:1/706: readlink db/d92/lcd 0 2024-04-23T02:14:11.020 INFO:tasks.workunit.client.1.smithi192.stdout:7/901: rename d5/d17/d1c/d57/d75/d102/l103 to d5/d43/d4c/d5c/l10a 0 2024-04-23T02:14:11.022 INFO:tasks.workunit.client.1.smithi192.stdout:5/905: mkdir d2/d2e/d12c 0 2024-04-23T02:14:11.026 INFO:tasks.workunit.client.1.smithi192.stdout:7/902: rename d5/d43/d4c/d5c/d97/faf to d5/d17/d1c/d57/d8b/f10b 0 2024-04-23T02:14:11.030 INFO:tasks.workunit.client.1.smithi192.stdout:7/903: creat d5/d17/d1c/d59/f10c x:0 0 0 2024-04-23T02:14:11.034 INFO:tasks.workunit.client.1.smithi192.stdout:7/904: symlink d5/d43/l10d 0 2024-04-23T02:14:11.035 INFO:tasks.workunit.client.0.smithi071.stdout:5/871: dread d3/d34/d74/faa [0,4194304] 0 2024-04-23T02:14:11.040 INFO:tasks.workunit.client.1.smithi192.stdout:6/979: write d3/d28/d79/da0/dc2/d101/d80/d8e/f59 [5023919,39034] 0 2024-04-23T02:14:11.041 INFO:tasks.workunit.client.0.smithi071.stdout:6/733: truncate d6/d17/d77/da2/fc6 351689 0 2024-04-23T02:14:11.044 INFO:tasks.workunit.client.0.smithi071.stdout:6/734: rename d6/d1f/d46/c70 to d6/d1f/d5d/d95/ce6 0 2024-04-23T02:14:11.045 INFO:tasks.workunit.client.1.smithi192.stdout:7/905: dwrite d5/d17/d1c/d59/d5a/d70/de9/da6/fb6 [4194304,4194304] 0 2024-04-23T02:14:11.045 INFO:tasks.workunit.client.0.smithi071.stdout:8/815: write dc/d78/f91 [1059271,24029] 0 2024-04-23T02:14:11.046 INFO:tasks.workunit.client.1.smithi192.stdout:6/980: chown d3/d28/d79/da0/dc2/d101/d80/d8e/d9d/ded/d100 430181 1 2024-04-23T02:14:11.047 INFO:tasks.workunit.client.0.smithi071.stdout:9/750: dread d1/d32/d41/db7/d39/f8f [0,4194304] 0 2024-04-23T02:14:11.049 INFO:tasks.workunit.client.1.smithi192.stdout:6/981: stat d3/d15/d1b/dcf 0 2024-04-23T02:14:11.049 INFO:tasks.workunit.client.0.smithi071.stdout:8/816: dread dc/d15/f87 [0,4194304] 0 2024-04-23T02:14:11.050 INFO:tasks.workunit.client.0.smithi071.stdout:9/751: symlink d1/d32/d3a/d90/dc2/l10b 0 2024-04-23T02:14:11.050 INFO:tasks.workunit.client.0.smithi071.stdout:6/735: dread d6/fb6 [0,4194304] 0 2024-04-23T02:14:11.058 INFO:tasks.workunit.client.1.smithi192.stdout:7/906: getdents d5/d17/d1c/d59/d5a/d41/d50/d9f 0 2024-04-23T02:14:11.061 INFO:tasks.workunit.client.1.smithi192.stdout:6/982: dwrite d3/d28/f9a [0,4194304] 0 2024-04-23T02:14:11.063 INFO:tasks.workunit.client.1.smithi192.stdout:9/751: dwrite d0/d9/d44/d51/d5b/d61/db1/fcf [4194304,4194304] 0 2024-04-23T02:14:11.064 INFO:tasks.workunit.client.1.smithi192.stdout:7/907: creat d5/d17/d1c/d59/d5a/f10e x:0 0 0 2024-04-23T02:14:11.069 INFO:tasks.workunit.client.1.smithi192.stdout:6/983: chown d3/d18/dd4/d8d/dab/lc5 42856 1 2024-04-23T02:14:11.069 INFO:tasks.workunit.client.1.smithi192.stdout:5/906: dwrite d2/d32/f6e [0,4194304] 0 2024-04-23T02:14:11.071 INFO:tasks.workunit.client.1.smithi192.stdout:9/752: read d0/d9/d44/d74/f23 [785585,31543] 0 2024-04-23T02:14:11.072 INFO:tasks.workunit.client.1.smithi192.stdout:5/907: read - d2/db/d16/d48/d4f/f128 zero size 2024-04-23T02:14:11.077 INFO:tasks.workunit.client.1.smithi192.stdout:6/984: creat d3/d28/d79/da0/dc2/d101/d80/d8e/f145 x:0 0 0 2024-04-23T02:14:11.079 INFO:tasks.workunit.client.1.smithi192.stdout:7/908: mkdir d5/d17/d1c/d59/d5a/d41/d50/d3d/d10f 0 2024-04-23T02:14:11.079 INFO:tasks.workunit.client.1.smithi192.stdout:6/985: symlink d3/d18/d58/l146 0 2024-04-23T02:14:11.082 INFO:tasks.workunit.client.1.smithi192.stdout:7/909: mknod d5/d17/d1c/d57/dbb/df4/c110 0 2024-04-23T02:14:11.082 INFO:tasks.workunit.client.1.smithi192.stdout:9/753: dwrite d0/d9/d44/d51/fae [0,4194304] 0 2024-04-23T02:14:11.084 INFO:tasks.workunit.client.1.smithi192.stdout:9/754: fdatasync d0/d9/d44/d4f/fb2 0 2024-04-23T02:14:11.087 INFO:tasks.workunit.client.1.smithi192.stdout:6/986: creat d3/d28/d79/da0/dc2/d101/d80/d8e/d128/d134/f147 x:0 0 0 2024-04-23T02:14:11.091 INFO:tasks.workunit.client.1.smithi192.stdout:6/987: read d3/d15/d1b/fb4 [65104,86981] 0 2024-04-23T02:14:11.094 INFO:tasks.workunit.client.1.smithi192.stdout:7/910: dwrite d5/d17/d1c/d59/d5a/f10e [0,4194304] 0 2024-04-23T02:14:11.100 INFO:tasks.workunit.client.1.smithi192.stdout:7/911: write d5/d17/d1c/d57/f33 [3411734,89365] 0 2024-04-23T02:14:11.101 INFO:tasks.workunit.client.1.smithi192.stdout:9/755: dwrite d0/d9/d44/d51/ff5 [0,4194304] 0 2024-04-23T02:14:11.104 INFO:tasks.workunit.client.1.smithi192.stdout:9/756: fdatasync d0/d9/d44/d51/d5b/d61/faf 0 2024-04-23T02:14:11.106 INFO:tasks.workunit.client.1.smithi192.stdout:9/757: mknod d0/dd2/cf7 0 2024-04-23T02:14:11.107 INFO:tasks.workunit.client.1.smithi192.stdout:4/943: sync 2024-04-23T02:14:11.111 INFO:tasks.workunit.client.1.smithi192.stdout:5/908: dread d2/db/d2b/d52/dc2/d49/fd6 [0,4194304] 0 2024-04-23T02:14:11.111 INFO:tasks.workunit.client.1.smithi192.stdout:7/912: dwrite d5/d17/d1c/d59/d5a/d70/de9/da6/fb6 [8388608,4194304] 0 2024-04-23T02:14:11.115 INFO:tasks.workunit.client.1.smithi192.stdout:5/909: symlink d2/d6/d14/l12d 0 2024-04-23T02:14:11.116 INFO:tasks.workunit.client.1.smithi192.stdout:4/944: write de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/de7/f135 [353785,23614] 0 2024-04-23T02:14:11.118 INFO:tasks.workunit.client.1.smithi192.stdout:7/913: write d5/d17/d1c/f2d [8080447,123939] 0 2024-04-23T02:14:11.119 INFO:tasks.workunit.client.1.smithi192.stdout:5/910: mknod d2/d6/d2d/d41/d43/c12e 0 2024-04-23T02:14:11.120 INFO:tasks.workunit.client.1.smithi192.stdout:4/945: write de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/d116/f132 [323163,105270] 0 2024-04-23T02:14:11.122 INFO:tasks.workunit.client.1.smithi192.stdout:7/914: mknod d5/d17/c111 0 2024-04-23T02:14:11.122 INFO:tasks.workunit.client.1.smithi192.stdout:5/911: truncate d2/d6/d2d/d41/f11e 372266 0 2024-04-23T02:14:11.125 INFO:tasks.workunit.client.1.smithi192.stdout:4/946: symlink de/d15/d53/d81/l13d 0 2024-04-23T02:14:11.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:10 smithi192 ceph-mon[49662]: pgmap v32: 65 pgs: 65 active+clean; 3.1 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 126 MiB/s rd, 313 MiB/s wr, 682 op/s 2024-04-23T02:14:11.129 INFO:tasks.workunit.client.1.smithi192.stdout:4/947: symlink de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d11b/d131/l13e 0 2024-04-23T02:14:11.132 INFO:tasks.workunit.client.1.smithi192.stdout:6/988: dread d3/d18/dd4/d8d/ff2 [0,4194304] 0 2024-04-23T02:14:11.133 INFO:tasks.workunit.client.0.smithi071.stdout:3/740: truncate d7/da5/db2/db7/d68/d3b/d4c/f63 2917582 0 2024-04-23T02:14:11.134 INFO:tasks.workunit.client.1.smithi192.stdout:4/948: read de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/de7/d101/ff0 [2212464,111433] 0 2024-04-23T02:14:11.135 INFO:tasks.workunit.client.1.smithi192.stdout:6/989: symlink d3/d15/l148 0 2024-04-23T02:14:11.136 INFO:tasks.workunit.client.1.smithi192.stdout:6/990: dread - d3/d15/d1b/dcf/f140 zero size 2024-04-23T02:14:11.137 INFO:tasks.workunit.client.1.smithi192.stdout:4/949: rename de/d15/d53/d81/de3/d74/d92/da0 to de/d15/d53/d81/de3/d74/d92/da0/dca/db6/d13f 22 2024-04-23T02:14:11.138 INFO:tasks.workunit.client.1.smithi192.stdout:6/991: creat d3/d15/d5b/f149 x:0 0 0 2024-04-23T02:14:11.141 INFO:tasks.workunit.client.1.smithi192.stdout:6/992: creat d3/d28/d79/da0/dc2/d101/d80/d107/f14a x:0 0 0 2024-04-23T02:14:11.144 INFO:tasks.workunit.client.1.smithi192.stdout:4/950: dread de/d15/d53/d81/de3/d74/d92/faa [0,4194304] 0 2024-04-23T02:14:11.154 INFO:tasks.workunit.client.1.smithi192.stdout:9/758: write d0/d9/d44/d51/d5b/d61/db1/f8d [2366075,17695] 0 2024-04-23T02:14:11.154 INFO:tasks.workunit.client.1.smithi192.stdout:4/951: dwrite de/d15/d19/d1a/d95/dd5/f10e [0,4194304] 0 2024-04-23T02:14:11.156 INFO:tasks.workunit.client.1.smithi192.stdout:5/912: dread d2/d6/d2d/f6b [0,4194304] 0 2024-04-23T02:14:11.161 INFO:tasks.workunit.client.1.smithi192.stdout:4/952: mknod de/d15/d53/d81/de3/d74/d92/da0/dca/de5/c140 0 2024-04-23T02:14:11.163 INFO:tasks.workunit.client.1.smithi192.stdout:9/759: write d0/d9/d44/f47 [3430382,43842] 0 2024-04-23T02:14:11.164 INFO:tasks.workunit.client.1.smithi192.stdout:7/915: symlink d5/d17/d1c/d57/d75/dee/l112 0 2024-04-23T02:14:11.166 INFO:tasks.workunit.client.1.smithi192.stdout:7/916: readlink d5/d17/d1c/d59/l8e 0 2024-04-23T02:14:11.166 INFO:tasks.workunit.client.1.smithi192.stdout:5/913: dwrite d2/f5 [4194304,4194304] 0 2024-04-23T02:14:11.167 INFO:tasks.workunit.client.1.smithi192.stdout:9/760: write d0/d9/d44/d58/dd9/fe6 [106431,72655] 0 2024-04-23T02:14:11.167 INFO:tasks.workunit.client.0.smithi071.stdout:2/681: sync 2024-04-23T02:14:11.168 INFO:tasks.workunit.client.1.smithi192.stdout:5/914: dread - d2/d6/d2d/fac zero size 2024-04-23T02:14:11.173 INFO:tasks.workunit.client.1.smithi192.stdout:9/761: symlink d0/d9/d44/d58/d89/lf8 0 2024-04-23T02:14:11.174 INFO:tasks.workunit.client.1.smithi192.stdout:7/917: creat d5/d17/d1c/d59/d5a/d70/f113 x:0 0 0 2024-04-23T02:14:11.175 INFO:tasks.workunit.client.1.smithi192.stdout:4/953: creat de/d15/f141 x:0 0 0 2024-04-23T02:14:11.176 INFO:tasks.workunit.client.1.smithi192.stdout:7/918: mkdir d5/d17/d114 0 2024-04-23T02:14:11.177 INFO:tasks.workunit.client.1.smithi192.stdout:9/762: symlink d0/d9/d44/d74/de7/lf9 0 2024-04-23T02:14:11.178 INFO:tasks.workunit.client.1.smithi192.stdout:9/763: chown d0/d9/d44/d58/fe4 129246 1 2024-04-23T02:14:11.179 INFO:tasks.workunit.client.1.smithi192.stdout:5/915: getdents d2/db/d16/d48/d4f/dd1 0 2024-04-23T02:14:11.180 INFO:tasks.workunit.client.1.smithi192.stdout:4/954: mknod de/d15/d19/d1a/d106/d125/c142 0 2024-04-23T02:14:11.183 INFO:tasks.workunit.client.1.smithi192.stdout:7/919: unlink d5/d17/d1c/d26/f34 0 2024-04-23T02:14:11.185 INFO:tasks.workunit.client.1.smithi192.stdout:4/955: mknod de/d15/d53/d81/de3/d74/d92/da0/dca/c143 0 2024-04-23T02:14:11.189 INFO:tasks.workunit.client.1.smithi192.stdout:7/920: dread d5/d17/d1c/d59/d5a/d41/d50/d9f/fa4 [4194304,4194304] 0 2024-04-23T02:14:11.189 INFO:tasks.workunit.client.1.smithi192.stdout:4/956: read de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/de7/f135 [171734,73499] 0 2024-04-23T02:14:11.191 INFO:tasks.workunit.client.0.smithi071.stdout:6/736: mkdir d6/d91/de7 0 2024-04-23T02:14:11.193 INFO:tasks.workunit.client.1.smithi192.stdout:7/921: creat d5/d43/d4c/d5c/d97/f115 x:0 0 0 2024-04-23T02:14:11.195 INFO:tasks.workunit.client.1.smithi192.stdout:4/957: fsync de/d15/d53/d81/dac/f105 0 2024-04-23T02:14:11.195 INFO:tasks.workunit.client.1.smithi192.stdout:9/764: dread d0/d9/d44/d58/d89/dd8/fb6 [0,4194304] 0 2024-04-23T02:14:11.195 INFO:tasks.workunit.client.0.smithi071.stdout:8/817: dwrite dc/d56/d64/fd7 [0,4194304] 0 2024-04-23T02:14:11.197 INFO:tasks.workunit.client.0.smithi071.stdout:4/817: dwrite d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/dc8/fe8 [0,4194304] 0 2024-04-23T02:14:11.199 INFO:tasks.workunit.client.1.smithi192.stdout:7/922: creat d5/d17/d1c/d4a/f116 x:0 0 0 2024-04-23T02:14:11.201 INFO:tasks.workunit.client.0.smithi071.stdout:1/707: dwrite db/d34/d68/f87 [4194304,4194304] 0 2024-04-23T02:14:11.201 INFO:tasks.workunit.client.1.smithi192.stdout:9/765: rename d0/fa6 to d0/d9/db3/d9e/ffa 0 2024-04-23T02:14:11.203 INFO:tasks.workunit.client.1.smithi192.stdout:4/958: unlink de/d15/d53/d81/de3/d74/d92/da0/dca/d49/de6/c12a 0 2024-04-23T02:14:11.206 INFO:tasks.workunit.client.0.smithi071.stdout:0/821: dwrite d0/d3/de/d10/d14/d32/d44/d31/f72 [0,4194304] 0 2024-04-23T02:14:11.206 INFO:tasks.workunit.client.0.smithi071.stdout:0/822: fdatasync d0/d63/f101 0 2024-04-23T02:14:11.207 INFO:tasks.workunit.client.0.smithi071.stdout:4/818: readlink d2/d2a/da2/dd1/dde/d23/d4f/db5/de7/lda 0 2024-04-23T02:14:11.212 INFO:tasks.workunit.client.0.smithi071.stdout:3/741: dread d7/da5/db2/db7/d68/d3b/d4c/da1/fe2 [0,4194304] 0 2024-04-23T02:14:11.214 INFO:tasks.workunit.client.0.smithi071.stdout:8/818: rmdir dc/d15/da0/d8d/d9a 39 2024-04-23T02:14:11.215 INFO:tasks.workunit.client.1.smithi192.stdout:5/916: dread d2/fe [0,4194304] 0 2024-04-23T02:14:11.216 INFO:tasks.workunit.client.0.smithi071.stdout:8/819: dread - dc/d56/de1/ffb zero size 2024-04-23T02:14:11.217 INFO:tasks.workunit.client.0.smithi071.stdout:0/823: creat d0/d3/de/d48/d7e/d8b/f11b x:0 0 0 2024-04-23T02:14:11.219 INFO:tasks.workunit.client.0.smithi071.stdout:9/752: rmdir d1/d4e/d54/d75/da6 0 2024-04-23T02:14:11.219 INFO:tasks.workunit.client.0.smithi071.stdout:1/708: dwrite db/df/d13/d20/f32 [0,4194304] 0 2024-04-23T02:14:11.220 INFO:tasks.workunit.client.0.smithi071.stdout:8/820: chown dc/d15/da0/d30/cf1 7178 1 2024-04-23T02:14:11.220 INFO:tasks.workunit.client.0.smithi071.stdout:6/737: unlink d6/d17/d77/d9f/dad/d8f/de2/cdd 0 2024-04-23T02:14:11.221 INFO:tasks.workunit.client.0.smithi071.stdout:8/821: write dc/d56/d64/f69 [4661122,32408] 0 2024-04-23T02:14:11.223 INFO:tasks.workunit.client.0.smithi071.stdout:3/742: creat d7/da5/db2/db7/d68/d3b/d4c/d8b/f108 x:0 0 0 2024-04-23T02:14:11.224 INFO:tasks.workunit.client.0.smithi071.stdout:0/824: creat d0/d3/d95/f11c x:0 0 0 2024-04-23T02:14:11.224 INFO:tasks.workunit.client.0.smithi071.stdout:8/822: stat dc/d15/d50/fbe 0 2024-04-23T02:14:11.225 INFO:tasks.workunit.client.0.smithi071.stdout:1/709: creat db/df/d13/d20/dc0/ddf/fee x:0 0 0 2024-04-23T02:14:11.226 INFO:tasks.workunit.client.0.smithi071.stdout:8/823: chown dc/d15/da0/d55/fc7 21 1 2024-04-23T02:14:11.230 INFO:tasks.workunit.client.0.smithi071.stdout:0/825: creat d0/d3/de/d10/d14/d32/d8a/df0/f11d x:0 0 0 2024-04-23T02:14:11.235 INFO:tasks.workunit.client.0.smithi071.stdout:1/710: dwrite db/d34/d8a/f8f [4194304,4194304] 0 2024-04-23T02:14:11.235 INFO:tasks.workunit.client.0.smithi071.stdout:3/743: unlink d7/d8/cc0 0 2024-04-23T02:14:11.235 INFO:tasks.workunit.client.0.smithi071.stdout:8/824: creat dc/d15/da0/d55/f106 x:0 0 0 2024-04-23T02:14:11.236 INFO:tasks.workunit.client.0.smithi071.stdout:6/738: rmdir d6/d22/d92/de5 0 2024-04-23T02:14:11.236 INFO:tasks.workunit.client.0.smithi071.stdout:0/826: symlink d0/d3/de/d10/d14/d32/d8a/df0/l11e 0 2024-04-23T02:14:11.237 INFO:tasks.workunit.client.1.smithi192.stdout:7/923: dread d5/d17/d1c/d4a/f84 [0,4194304] 0 2024-04-23T02:14:11.237 INFO:tasks.workunit.client.0.smithi071.stdout:9/753: read d1/fb [8331970,96709] 0 2024-04-23T02:14:11.242 INFO:tasks.workunit.client.0.smithi071.stdout:8/825: creat dc/d15/da0/d32/d63/dd4/f107 x:0 0 0 2024-04-23T02:14:11.242 INFO:tasks.workunit.client.1.smithi192.stdout:7/924: rmdir d5/d17/d1c/d59/d5a/d70/de9/da6/da9/db9/dd9 0 2024-04-23T02:14:11.243 INFO:tasks.workunit.client.0.smithi071.stdout:6/739: write d6/d17/d77/d9e/fc5 [4321873,50519] 0 2024-04-23T02:14:11.243 INFO:tasks.workunit.client.1.smithi192.stdout:7/925: fdatasync d5/d43/d4c/d5c/d97/f115 0 2024-04-23T02:14:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:10 smithi071 ceph-mon[39837]: pgmap v32: 65 pgs: 65 active+clean; 3.1 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 126 MiB/s rd, 313 MiB/s wr, 682 op/s 2024-04-23T02:14:11.245 INFO:tasks.workunit.client.0.smithi071.stdout:9/754: truncate d1/f4 2900255 0 2024-04-23T02:14:11.245 INFO:tasks.workunit.client.1.smithi192.stdout:9/766: dread d0/fdb [0,4194304] 0 2024-04-23T02:14:11.246 INFO:tasks.workunit.client.0.smithi071.stdout:6/740: dread - d6/d17/d77/d9f/dad/d3f/fca zero size 2024-04-23T02:14:11.246 INFO:tasks.workunit.client.1.smithi192.stdout:9/767: fdatasync d0/f54 0 2024-04-23T02:14:11.246 INFO:tasks.workunit.client.1.smithi192.stdout:7/926: write d5/d17/d1c/d57/d8b/f10b [2509664,101264] 0 2024-04-23T02:14:11.248 INFO:tasks.workunit.client.0.smithi071.stdout:1/711: rename db/df/d13/d20/fa3 to db/df/d13/fef 0 2024-04-23T02:14:11.249 INFO:tasks.workunit.client.0.smithi071.stdout:3/744: getdents d7/da5/db2/db7/d68/d3b 0 2024-04-23T02:14:11.250 INFO:tasks.workunit.client.0.smithi071.stdout:6/741: mknod d6/d17/d77/d9f/dad/d8f/de2/ce8 0 2024-04-23T02:14:11.250 INFO:tasks.workunit.client.1.smithi192.stdout:7/927: creat d5/d17/d1c/d59/d5a/d41/d50/d9f/f117 x:0 0 0 2024-04-23T02:14:11.251 INFO:tasks.workunit.client.0.smithi071.stdout:9/755: mknod d1/d32/d3a/d5f/dde/c10c 0 2024-04-23T02:14:11.251 INFO:tasks.workunit.client.1.smithi192.stdout:9/768: link d0/d9/d44/d74/c27 d0/d9/db3/d9e/cfb 0 2024-04-23T02:14:11.252 INFO:tasks.workunit.client.1.smithi192.stdout:7/928: write d5/d17/d1c/d59/d5a/d41/f98 [563754,81275] 0 2024-04-23T02:14:11.253 INFO:tasks.workunit.client.1.smithi192.stdout:9/769: write d0/d9/d44/d51/d5b/dd3/ff0 [236578,108927] 0 2024-04-23T02:14:11.254 INFO:tasks.workunit.client.0.smithi071.stdout:1/712: rmdir db/df/d5f 39 2024-04-23T02:14:11.255 INFO:tasks.workunit.client.1.smithi192.stdout:7/929: dread - d5/d43/d4c/d5c/d97/fea zero size 2024-04-23T02:14:11.255 INFO:tasks.workunit.client.1.smithi192.stdout:9/770: dread - d0/d9/d44/d51/d5b/d61/faf zero size 2024-04-23T02:14:11.256 INFO:tasks.workunit.client.0.smithi071.stdout:9/756: creat d1/d32/d3a/d90/f10d x:0 0 0 2024-04-23T02:14:11.257 INFO:tasks.workunit.client.0.smithi071.stdout:9/757: stat d1/d4e/d66/da4/dd7 0 2024-04-23T02:14:11.259 INFO:tasks.workunit.client.0.smithi071.stdout:9/758: readlink d1/l4d 0 2024-04-23T02:14:11.259 INFO:tasks.workunit.client.1.smithi192.stdout:9/771: rename d0/d9/dc7/da0/dd5/cf4 to d0/d9/d44/d58/dd9/def/cfc 0 2024-04-23T02:14:11.259 INFO:tasks.workunit.client.0.smithi071.stdout:1/713: fdatasync db/df/d5f/d6f/fb7 0 2024-04-23T02:14:11.261 INFO:tasks.workunit.client.0.smithi071.stdout:6/742: link d6/d17/d77/d9f/ld2 d6/d17/d77/d9f/le9 0 2024-04-23T02:14:11.262 INFO:tasks.workunit.client.0.smithi071.stdout:1/714: mkdir db/df/d5f/df0 0 2024-04-23T02:14:11.264 INFO:tasks.workunit.client.0.smithi071.stdout:6/743: mknod d6/d17/d77/d9f/dad/cea 0 2024-04-23T02:14:11.268 INFO:tasks.workunit.client.0.smithi071.stdout:4/819: dread d2/d2a/d2d/f66 [0,4194304] 0 2024-04-23T02:14:11.271 INFO:tasks.workunit.client.0.smithi071.stdout:4/820: creat d2/d2a/da2/dd1/dde/d23/d4f/db5/f10a x:0 0 0 2024-04-23T02:14:11.271 INFO:tasks.workunit.client.0.smithi071.stdout:0/827: dread d0/d63/fbf [0,4194304] 0 2024-04-23T02:14:11.273 INFO:tasks.workunit.client.0.smithi071.stdout:4/821: creat d2/d2a/d2d/d7b/f10b x:0 0 0 2024-04-23T02:14:11.274 INFO:tasks.workunit.client.0.smithi071.stdout:0/828: symlink d0/d3/de/d10/d14/l11f 0 2024-04-23T02:14:11.275 INFO:tasks.workunit.client.0.smithi071.stdout:4/822: readlink d2/d2a/da2/dd1/dde/d23/l4c 0 2024-04-23T02:14:11.276 INFO:tasks.workunit.client.1.smithi192.stdout:9/772: dread d0/d9/d44/d51/d5b/dd3/ff0 [0,4194304] 0 2024-04-23T02:14:11.277 INFO:tasks.workunit.client.1.smithi192.stdout:9/773: chown d0/d9/d44/d58/l6c 320388 1 2024-04-23T02:14:11.277 INFO:tasks.workunit.client.0.smithi071.stdout:0/829: mkdir d0/d3/de/d48/d120 0 2024-04-23T02:14:11.278 INFO:tasks.workunit.client.1.smithi192.stdout:9/774: dread - d0/d9/dc7/fe1 zero size 2024-04-23T02:14:11.279 INFO:tasks.workunit.client.0.smithi071.stdout:4/823: read d2/d2a/d2d/d39/d42/d49/f77 [1021322,91256] 0 2024-04-23T02:14:11.280 INFO:tasks.workunit.client.0.smithi071.stdout:4/824: readlink d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/dc8/le2 0 2024-04-23T02:14:11.282 INFO:tasks.workunit.client.1.smithi192.stdout:5/917: dwrite d2/d32/f9c [0,4194304] 0 2024-04-23T02:14:11.288 INFO:tasks.workunit.client.1.smithi192.stdout:7/930: dwrite d5/d17/d1c/d59/d5a/d70/de9/f81 [0,4194304] 0 2024-04-23T02:14:11.289 INFO:tasks.workunit.client.0.smithi071.stdout:2/682: truncate d0/d2/d45/d7d/fbd 405248 0 2024-04-23T02:14:11.291 INFO:tasks.workunit.client.1.smithi192.stdout:9/775: dwrite d0/d9/d44/d58/dd9/feb [0,4194304] 0 2024-04-23T02:14:11.292 INFO:tasks.workunit.client.0.smithi071.stdout:2/683: rmdir d0/d2/d26/d3a/dc3/d36/d6a 39 2024-04-23T02:14:11.296 INFO:tasks.workunit.client.0.smithi071.stdout:2/684: getdents d0/d2/d34/dbf 0 2024-04-23T02:14:11.296 INFO:tasks.workunit.client.1.smithi192.stdout:9/776: truncate d0/d9/d44/d74/f82 4757718 0 2024-04-23T02:14:11.296 INFO:tasks.workunit.client.1.smithi192.stdout:7/931: creat d5/d17/d1c/d59/d5a/d41/d50/d3d/f118 x:0 0 0 2024-04-23T02:14:11.298 INFO:tasks.workunit.client.1.smithi192.stdout:5/918: link d2/d6/d2d/f118 d2/db/d46/f12f 0 2024-04-23T02:14:11.304 INFO:tasks.workunit.client.0.smithi071.stdout:3/745: write d7/d8/f16 [675932,64716] 0 2024-04-23T02:14:11.304 INFO:tasks.workunit.client.0.smithi071.stdout:8/826: write dc/d15/da0/f22 [2429073,92811] 0 2024-04-23T02:14:11.308 INFO:tasks.workunit.client.0.smithi071.stdout:2/685: dwrite d0/d82/f9d [0,4194304] 0 2024-04-23T02:14:11.317 INFO:tasks.workunit.client.0.smithi071.stdout:6/744: dwrite d6/d17/d77/d9f/dad/f5e [0,4194304] 0 2024-04-23T02:14:11.318 INFO:tasks.workunit.client.0.smithi071.stdout:1/715: dwrite db/df/d13/d2f/f29 [0,4194304] 0 2024-04-23T02:14:11.318 INFO:tasks.workunit.client.0.smithi071.stdout:3/746: rename d7/c37 to d7/da5/db2/db7/d68/dae/dc6/c109 0 2024-04-23T02:14:11.328 INFO:tasks.workunit.client.1.smithi192.stdout:1/752: sync 2024-04-23T02:14:11.329 INFO:tasks.workunit.client.0.smithi071.stdout:6/745: dread - d6/d1f/d46/f6e zero size 2024-04-23T02:14:11.329 INFO:tasks.workunit.client.0.smithi071.stdout:7/978: sync 2024-04-23T02:14:11.329 INFO:tasks.workunit.client.1.smithi192.stdout:1/753: write f1 [6639850,25989] 0 2024-04-23T02:14:11.331 INFO:tasks.workunit.client.0.smithi071.stdout:1/716: symlink db/df/d3a/lf1 0 2024-04-23T02:14:11.334 INFO:tasks.workunit.client.0.smithi071.stdout:4/825: dwrite d2/d2a/da2/dd1/dde/f6d [0,4194304] 0 2024-04-23T02:14:11.334 INFO:tasks.workunit.client.0.smithi071.stdout:0/830: dread d0/d3/de/ff [4194304,4194304] 0 2024-04-23T02:14:11.335 INFO:tasks.workunit.client.1.smithi192.stdout:1/754: creat dc/d14/d15/ff2 x:0 0 0 2024-04-23T02:14:11.335 INFO:tasks.workunit.client.1.smithi192.stdout:9/777: dread d0/d9/d44/d58/d89/dd8/fac [0,4194304] 0 2024-04-23T02:14:11.336 INFO:tasks.workunit.client.1.smithi192.stdout:1/755: read dc/d80/fac [65206,30635] 0 2024-04-23T02:14:11.336 INFO:tasks.workunit.client.0.smithi071.stdout:3/747: getdents d7/d22/d23/dba 0 2024-04-23T02:14:11.336 INFO:tasks.workunit.client.1.smithi192.stdout:1/756: dread - dc/d80/fa8 zero size 2024-04-23T02:14:11.336 INFO:tasks.workunit.client.1.smithi192.stdout:9/778: chown d0/d9/d44/d51/d5b/d61/fb9 9306193 1 2024-04-23T02:14:11.337 INFO:tasks.workunit.client.0.smithi071.stdout:6/746: write d6/d17/d77/f84 [3126075,117482] 0 2024-04-23T02:14:11.338 INFO:tasks.workunit.client.0.smithi071.stdout:7/979: mkdir d3/d51/d80/d15a 0 2024-04-23T02:14:11.339 INFO:tasks.workunit.client.1.smithi192.stdout:1/757: creat dc/d16/d19/d30/ff3 x:0 0 0 2024-04-23T02:14:11.341 INFO:tasks.workunit.client.0.smithi071.stdout:3/748: creat d7/d22/d23/f10a x:0 0 0 2024-04-23T02:14:11.341 INFO:tasks.workunit.client.0.smithi071.stdout:4/826: symlink d2/d2a/daa/l10c 0 2024-04-23T02:14:11.341 INFO:tasks.workunit.client.0.smithi071.stdout:2/686: dwrite d0/d70/f86 [4194304,4194304] 0 2024-04-23T02:14:11.342 INFO:tasks.workunit.client.0.smithi071.stdout:0/831: dread - d0/d3/de/d48/d7e/d8b/de3/f113 zero size 2024-04-23T02:14:11.345 INFO:tasks.workunit.client.0.smithi071.stdout:6/747: symlink d6/d22/dbf/leb 0 2024-04-23T02:14:11.346 INFO:tasks.workunit.client.0.smithi071.stdout:7/980: symlink d3/d17/d11b/l15b 0 2024-04-23T02:14:11.346 INFO:tasks.workunit.client.1.smithi192.stdout:1/758: dwrite dc/d16/db0/fd8 [0,4194304] 0 2024-04-23T02:14:11.347 INFO:tasks.workunit.client.0.smithi071.stdout:3/749: symlink d7/d22/d23/dba/df4/dce/l10b 0 2024-04-23T02:14:11.347 INFO:tasks.workunit.client.0.smithi071.stdout:1/717: link db/f22 db/df/d13/d20/de5/ff2 0 2024-04-23T02:14:11.348 INFO:tasks.workunit.client.0.smithi071.stdout:4/827: fsync d2/d2a/da2/dd1/dde/d57/f72 0 2024-04-23T02:14:11.350 INFO:tasks.workunit.client.0.smithi071.stdout:2/687: rename d0/c8e to d0/d2/da0/ce1 0 2024-04-23T02:14:11.353 INFO:tasks.workunit.client.0.smithi071.stdout:3/750: mkdir d7/d22/da7/d10c 0 2024-04-23T02:14:11.355 INFO:tasks.workunit.client.1.smithi192.stdout:9/779: fsync d0/d9/d44/d58/dd9/feb 0 2024-04-23T02:14:11.358 INFO:tasks.workunit.client.0.smithi071.stdout:7/981: dwrite d3/d17/d41/f2f [0,4194304] 0 2024-04-23T02:14:11.360 INFO:tasks.workunit.client.0.smithi071.stdout:2/688: chown d0/d2/d26/d3a/d75/f8f 91 1 2024-04-23T02:14:11.361 INFO:tasks.workunit.client.0.smithi071.stdout:6/748: link d6/d22/d92/l93 d6/d91/lec 0 2024-04-23T02:14:11.362 INFO:tasks.workunit.client.0.smithi071.stdout:7/982: symlink d3/d17/d1c/d2a/d73/l15c 0 2024-04-23T02:14:11.363 INFO:tasks.workunit.client.1.smithi192.stdout:9/780: dwrite d0/d9/dc7/da0/fbe [4194304,4194304] 0 2024-04-23T02:14:11.367 INFO:tasks.workunit.client.0.smithi071.stdout:0/832: rename d0/d3/de/la9 to d0/d3/de/d10/d14/d32/d37/l121 0 2024-04-23T02:14:11.370 INFO:tasks.workunit.client.0.smithi071.stdout:6/749: creat d6/d17/d77/fed x:0 0 0 2024-04-23T02:14:11.374 INFO:tasks.workunit.client.0.smithi071.stdout:2/689: mknod d0/d2/d26/d3a/dc3/d36/d5c/d85/ce2 0 2024-04-23T02:14:11.376 INFO:tasks.workunit.client.0.smithi071.stdout:3/751: rename d7/d1d/fd6 to d7/da5/d100/f10d 0 2024-04-23T02:14:11.379 INFO:tasks.workunit.client.1.smithi192.stdout:1/759: dread dc/f76 [0,4194304] 0 2024-04-23T02:14:11.379 INFO:tasks.workunit.client.0.smithi071.stdout:3/752: read d7/d8/d42/f5d [238318,108232] 0 2024-04-23T02:14:11.380 INFO:tasks.workunit.client.0.smithi071.stdout:0/833: symlink d0/d3/de/d10/d14/d32/d44/d87/l122 0 2024-04-23T02:14:11.382 INFO:tasks.workunit.client.0.smithi071.stdout:8/827: dwrite dc/d15/f23 [4194304,4194304] 0 2024-04-23T02:14:11.384 INFO:tasks.workunit.client.1.smithi192.stdout:1/760: mkdir dc/d16/d19/d74/df4 0 2024-04-23T02:14:11.384 INFO:tasks.workunit.client.1.smithi192.stdout:7/932: rmdir d5/d17/d1c/d59/d5a/d41/d50 39 2024-04-23T02:14:11.384 INFO:tasks.workunit.client.0.smithi071.stdout:2/690: creat d0/d2/d26/d3a/dc3/d36/fe3 x:0 0 0 2024-04-23T02:14:11.384 INFO:tasks.workunit.client.0.smithi071.stdout:3/753: mkdir d7/da5/db2/db7/d68/d3b/d98/d10e 0 2024-04-23T02:14:11.385 INFO:tasks.workunit.client.0.smithi071.stdout:0/834: creat d0/d3/de/d10/db9/f123 x:0 0 0 2024-04-23T02:14:11.386 INFO:tasks.workunit.client.0.smithi071.stdout:8/828: creat dc/d15/d50/d5f/f108 x:0 0 0 2024-04-23T02:14:11.391 INFO:tasks.workunit.client.0.smithi071.stdout:8/829: symlink dc/d15/da0/d8d/l109 0 2024-04-23T02:14:11.393 INFO:tasks.workunit.client.1.smithi192.stdout:9/781: truncate d0/d9/d44/d51/d5b/d61/f6a 5201385 0 2024-04-23T02:14:11.394 INFO:tasks.workunit.client.0.smithi071.stdout:8/830: fsync dc/d15/d2d/f38 0 2024-04-23T02:14:11.394 INFO:tasks.workunit.client.1.smithi192.stdout:7/933: write d5/d17/d1c/d4a/f76 [2859621,102200] 0 2024-04-23T02:14:11.396 INFO:tasks.workunit.client.1.smithi192.stdout:7/934: readlink d5/d43/l85 0 2024-04-23T02:14:11.397 INFO:tasks.workunit.client.0.smithi071.stdout:3/754: dwrite d7/da5/d100/f10d [0,4194304] 0 2024-04-23T02:14:11.397 INFO:tasks.workunit.client.1.smithi192.stdout:9/782: mkdir d0/d9/db3/d5f/dfd 0 2024-04-23T02:14:11.398 INFO:tasks.workunit.client.0.smithi071.stdout:7/983: dread d3/d17/d41/d34/d39/d3b/fb1 [0,4194304] 0 2024-04-23T02:14:11.398 INFO:tasks.workunit.client.0.smithi071.stdout:0/835: link d0/d3/de/laf d0/d3/de/d48/l124 0 2024-04-23T02:14:11.398 INFO:tasks.workunit.client.0.smithi071.stdout:8/831: chown dc/d15/d50/d5f/ff3 58369 1 2024-04-23T02:14:11.399 INFO:tasks.workunit.client.1.smithi192.stdout:5/919: dwrite d2/d6/d2d/f6b [0,4194304] 0 2024-04-23T02:14:11.400 INFO:tasks.workunit.client.0.smithi071.stdout:8/832: dread - dc/d3a/da4/fdb zero size 2024-04-23T02:14:11.400 INFO:tasks.workunit.client.0.smithi071.stdout:0/836: write d0/d3/de/d10/db9/fc4 [1052405,83905] 0 2024-04-23T02:14:11.403 INFO:tasks.workunit.client.1.smithi192.stdout:5/920: dread - d2/db/d2b/d52/f8f zero size 2024-04-23T02:14:11.405 INFO:tasks.workunit.client.0.smithi071.stdout:7/984: mknod d3/d17/d1c/d2a/d73/dda/c15d 0 2024-04-23T02:14:11.408 INFO:tasks.workunit.client.0.smithi071.stdout:8/833: symlink dc/d15/d50/l10a 0 2024-04-23T02:14:11.409 INFO:tasks.workunit.client.1.smithi192.stdout:7/935: rename d5/d17/d1c/d57/d75/dee/l112 to d5/d17/d1c/d57/l119 0 2024-04-23T02:14:11.410 INFO:tasks.workunit.client.0.smithi071.stdout:1/718: truncate db/df/d13/d20/f32 636946 0 2024-04-23T02:14:11.411 INFO:tasks.workunit.client.0.smithi071.stdout:4/828: dwrite d2/d2a/daa/fad [0,4194304] 0 2024-04-23T02:14:11.415 INFO:tasks.workunit.client.0.smithi071.stdout:4/829: write d2/d2a/d2d/d39/d42/d49/d67/f1a [4370940,99710] 0 2024-04-23T02:14:11.416 INFO:tasks.workunit.client.0.smithi071.stdout:8/834: stat dc/l99 0 2024-04-23T02:14:11.417 INFO:tasks.workunit.client.0.smithi071.stdout:7/985: dread d3/d17/d41/d3a/d137/dea/d5f/f7e [0,4194304] 0 2024-04-23T02:14:11.420 INFO:tasks.workunit.client.1.smithi192.stdout:5/921: dwrite d2/d32/fc9 [0,4194304] 0 2024-04-23T02:14:11.420 INFO:tasks.workunit.client.0.smithi071.stdout:1/719: write db/df/d13/d2f/d48/d85/f98 [920516,66831] 0 2024-04-23T02:14:11.420 INFO:tasks.workunit.client.0.smithi071.stdout:7/986: write d3/d17/d1c/d2a/d73/dba/ff6 [523451,102918] 0 2024-04-23T02:14:11.421 INFO:tasks.workunit.client.1.smithi192.stdout:9/783: dwrite d0/d9/d44/d51/d5b/d61/db1/f8d [0,4194304] 0 2024-04-23T02:14:11.422 INFO:tasks.workunit.client.0.smithi071.stdout:6/750: dread d6/f16 [0,4194304] 0 2024-04-23T02:14:11.423 INFO:tasks.workunit.client.1.smithi192.stdout:9/784: stat d0/d9/d44/d53/c7a 0 2024-04-23T02:14:11.423 INFO:tasks.workunit.client.1.smithi192.stdout:1/761: dread dc/d8b/d44/f52 [0,4194304] 0 2024-04-23T02:14:11.423 INFO:tasks.workunit.client.0.smithi071.stdout:7/987: write d3/d51/d80/f9e [5034430,79020] 0 2024-04-23T02:14:11.424 INFO:tasks.workunit.client.1.smithi192.stdout:1/762: chown dc/d14/d23/da2/fcb 698 1 2024-04-23T02:14:11.425 INFO:tasks.workunit.client.0.smithi071.stdout:4/830: fdatasync d2/d2a/d2d/d39/d42/d49/d91/fd2 0 2024-04-23T02:14:11.426 INFO:tasks.workunit.client.0.smithi071.stdout:7/988: dread - d3/dae/f11e zero size 2024-04-23T02:14:11.426 INFO:tasks.workunit.client.1.smithi192.stdout:5/922: creat d2/db/d16/d48/dc5/d121/f130 x:0 0 0 2024-04-23T02:14:11.427 INFO:tasks.workunit.client.0.smithi071.stdout:7/989: readlink d3/d17/l95 0 2024-04-23T02:14:11.429 INFO:tasks.workunit.client.1.smithi192.stdout:1/763: creat dc/d16/d19/d74/dd2/ff5 x:0 0 0 2024-04-23T02:14:11.430 INFO:tasks.workunit.client.1.smithi192.stdout:7/936: dread d5/d17/d1c/d26/f28 [0,4194304] 0 2024-04-23T02:14:11.431 INFO:tasks.workunit.client.1.smithi192.stdout:9/785: link d0/dd2/cf7 d0/d9/dc7/da0/cfe 0 2024-04-23T02:14:11.433 INFO:tasks.workunit.client.1.smithi192.stdout:7/937: write d5/d43/d4c/f6c [2985031,64086] 0 2024-04-23T02:14:11.433 INFO:tasks.workunit.client.1.smithi192.stdout:9/786: stat d0/d9/d44/d4f/cd0 0 2024-04-23T02:14:11.436 INFO:tasks.workunit.client.1.smithi192.stdout:9/787: mkdir d0/d9/db3/d9e/dff 0 2024-04-23T02:14:11.437 INFO:tasks.workunit.client.0.smithi071.stdout:6/751: creat d6/d17/d77/d9f/dad/d3f/fee x:0 0 0 2024-04-23T02:14:11.439 INFO:tasks.workunit.client.1.smithi192.stdout:3/876: sync 2024-04-23T02:14:11.439 INFO:tasks.workunit.client.0.smithi071.stdout:0/837: link d0/d3/de/d10/d14/d32/d8a/cc3 d0/d3/de/d10/db3/c125 0 2024-04-23T02:14:11.440 INFO:tasks.workunit.client.0.smithi071.stdout:6/752: read d6/f12 [28398,16053] 0 2024-04-23T02:14:11.440 INFO:tasks.workunit.client.1.smithi192.stdout:7/938: creat d5/d17/d1c/d59/d5a/d41/d50/d3d/d10f/f11a x:0 0 0 2024-04-23T02:14:11.441 INFO:tasks.workunit.client.1.smithi192.stdout:9/788: getdents d0/d9/d44/d74 0 2024-04-23T02:14:11.444 INFO:tasks.workunit.client.1.smithi192.stdout:9/789: chown d0/d9/d44/d51/d5b/d61 67125582 1 2024-04-23T02:14:11.444 INFO:tasks.workunit.client.1.smithi192.stdout:7/939: write d5/d17/d1c/d26/f28 [1264961,80876] 0 2024-04-23T02:14:11.444 INFO:tasks.workunit.client.1.smithi192.stdout:3/877: getdents d1/d3/d17/d2b 0 2024-04-23T02:14:11.445 INFO:tasks.workunit.client.0.smithi071.stdout:3/755: write d7/da5/db2/db7/d68/dae/fe4 [988163,129302] 0 2024-04-23T02:14:11.447 INFO:tasks.workunit.client.0.smithi071.stdout:1/720: dread db/df/d13/f71 [0,4194304] 0 2024-04-23T02:14:11.448 INFO:tasks.workunit.client.0.smithi071.stdout:3/756: symlink d7/d22/d23/dba/l10f 0 2024-04-23T02:14:11.449 INFO:tasks.workunit.client.0.smithi071.stdout:3/757: write d7/d10/f15 [1179751,23318] 0 2024-04-23T02:14:11.453 INFO:tasks.workunit.client.1.smithi192.stdout:3/878: mknod d1/d3/d6/d7/d29/d3b/da9/c137 0 2024-04-23T02:14:11.454 INFO:tasks.workunit.client.0.smithi071.stdout:8/835: write dc/f20 [396282,98127] 0 2024-04-23T02:14:11.455 INFO:tasks.workunit.client.1.smithi192.stdout:3/879: creat d1/d3/d6/d7/d29/d3b/da9/daf/f138 x:0 0 0 2024-04-23T02:14:11.456 INFO:tasks.workunit.client.1.smithi192.stdout:3/880: chown d1/d3/d17/d1f/d33/d24 1958877785 1 2024-04-23T02:14:11.460 INFO:tasks.workunit.client.1.smithi192.stdout:3/881: symlink d1/l139 0 2024-04-23T02:14:11.460 INFO:tasks.workunit.client.0.smithi071.stdout:8/836: mknod dc/d15/da0/d8d/df8/c10b 0 2024-04-23T02:14:11.461 INFO:tasks.workunit.client.1.smithi192.stdout:7/940: write d5/d17/d1c/f74 [10602057,13121] 0 2024-04-23T02:14:11.463 INFO:tasks.workunit.client.0.smithi071.stdout:3/758: dwrite d7/d22/da7/fd2 [0,4194304] 0 2024-04-23T02:14:11.464 INFO:tasks.workunit.client.1.smithi192.stdout:6/993: sync 2024-04-23T02:14:11.465 INFO:tasks.workunit.client.1.smithi192.stdout:4/959: sync 2024-04-23T02:14:11.468 INFO:tasks.workunit.client.1.smithi192.stdout:7/941: mkdir d5/d43/d4c/d11b 0 2024-04-23T02:14:11.469 INFO:tasks.workunit.client.1.smithi192.stdout:1/764: dwrite dc/d8b/d44/f52 [0,4194304] 0 2024-04-23T02:14:11.469 INFO:tasks.workunit.client.1.smithi192.stdout:5/923: dwrite d2/d6/d2d/d41/fa1 [0,4194304] 0 2024-04-23T02:14:11.472 INFO:tasks.workunit.client.0.smithi071.stdout:4/831: dwrite d2/d2a/d2d/d7b/f8c [0,4194304] 0 2024-04-23T02:14:11.473 INFO:tasks.workunit.client.0.smithi071.stdout:6/753: dwrite d6/d1f/d5d/f87 [0,4194304] 0 2024-04-23T02:14:11.476 INFO:tasks.workunit.client.0.smithi071.stdout:8/837: rename dc/d15/d50/d5f/fc9 to dc/d15/da0/d8d/def/f10c 0 2024-04-23T02:14:11.476 INFO:tasks.workunit.client.0.smithi071.stdout:1/721: dwrite db/f22 [0,4194304] 0 2024-04-23T02:14:11.479 INFO:tasks.workunit.client.0.smithi071.stdout:0/838: dread d0/f2 [0,4194304] 0 2024-04-23T02:14:11.481 INFO:tasks.workunit.client.1.smithi192.stdout:5/924: dread d2/d2e/df0/f65 [0,4194304] 0 2024-04-23T02:14:11.482 INFO:tasks.workunit.client.0.smithi071.stdout:8/838: mknod dc/d15/da0/c10d 0 2024-04-23T02:14:11.483 INFO:tasks.workunit.client.0.smithi071.stdout:1/722: mknod db/df/d13/d2f/d48/d85/cf3 0 2024-04-23T02:14:11.484 INFO:tasks.workunit.client.0.smithi071.stdout:0/839: symlink d0/d3/d95/l126 0 2024-04-23T02:14:11.486 INFO:tasks.workunit.client.0.smithi071.stdout:8/839: symlink dc/d56/d64/dde/l10e 0 2024-04-23T02:14:11.490 INFO:tasks.workunit.client.0.smithi071.stdout:6/754: dwrite d6/d1f/f83 [0,4194304] 0 2024-04-23T02:14:11.490 INFO:tasks.workunit.client.0.smithi071.stdout:0/840: rmdir d0/d3/de/d10/d14/d32/d37 39 2024-04-23T02:14:11.491 INFO:tasks.workunit.client.1.smithi192.stdout:7/942: rename d5/d43/l106 to d5/d17/db5/l11c 0 2024-04-23T02:14:11.491 INFO:tasks.workunit.client.1.smithi192.stdout:4/960: dread - de/d15/d53/d81/d10b/f12c zero size 2024-04-23T02:14:11.492 INFO:tasks.workunit.client.0.smithi071.stdout:1/723: creat db/df/ff4 x:0 0 0 2024-04-23T02:14:11.492 INFO:tasks.workunit.client.0.smithi071.stdout:4/832: dread d2/d2a/d2d/d39/d42/d49/d91/f92 [0,4194304] 0 2024-04-23T02:14:11.493 INFO:tasks.workunit.client.1.smithi192.stdout:9/790: dread d0/d9/d44/f47 [0,4194304] 0 2024-04-23T02:14:11.493 INFO:tasks.workunit.client.1.smithi192.stdout:1/765: truncate dc/d16/d19/d30/dc4/fd9 940 0 2024-04-23T02:14:11.493 INFO:tasks.workunit.client.0.smithi071.stdout:0/841: write d0/d3/d95/fcd [1365363,28113] 0 2024-04-23T02:14:11.495 INFO:tasks.workunit.client.1.smithi192.stdout:5/925: rename d2/db/d2b/d52 to d2/db/d2b/d52/dc2/d49/d110/d131 22 2024-04-23T02:14:11.496 INFO:tasks.workunit.client.0.smithi071.stdout:8/840: mkdir dc/d15/da0/d8d/d10f 0 2024-04-23T02:14:11.499 INFO:tasks.workunit.client.0.smithi071.stdout:1/724: creat db/df/d3a/de7/ff5 x:0 0 0 2024-04-23T02:14:11.500 INFO:tasks.workunit.client.0.smithi071.stdout:4/833: rmdir d2/df6 0 2024-04-23T02:14:11.505 INFO:tasks.workunit.client.0.smithi071.stdout:1/725: mkdir db/df/d5f/df6 0 2024-04-23T02:14:11.510 INFO:tasks.workunit.client.0.smithi071.stdout:1/726: creat db/d11/ff7 x:0 0 0 2024-04-23T02:14:11.514 INFO:tasks.workunit.client.1.smithi192.stdout:9/791: chown d0/d9/d44/d58/d89/la1 1363 1 2024-04-23T02:14:11.514 INFO:tasks.workunit.client.1.smithi192.stdout:3/882: dwrite d1/f5 [0,4194304] 0 2024-04-23T02:14:11.515 INFO:tasks.workunit.client.1.smithi192.stdout:9/792: chown d0/d9/dc7/da0 1071296160 1 2024-04-23T02:14:11.515 INFO:tasks.workunit.client.1.smithi192.stdout:7/943: symlink d5/d17/d1c/d57/dbb/l11d 0 2024-04-23T02:14:11.517 INFO:tasks.workunit.client.1.smithi192.stdout:6/994: write d3/d15/d1b/dcf/fd3 [507334,73192] 0 2024-04-23T02:14:11.520 INFO:tasks.workunit.client.1.smithi192.stdout:7/944: truncate d5/d17/d1c/d59/d5a/d41/d50/d3d/f118 965981 0 2024-04-23T02:14:11.521 INFO:tasks.workunit.client.1.smithi192.stdout:1/766: write fa [2294010,42330] 0 2024-04-23T02:14:11.522 INFO:tasks.workunit.client.1.smithi192.stdout:1/767: chown dc/d16/d19/d74/fa6 0 1 2024-04-23T02:14:11.523 INFO:tasks.workunit.client.1.smithi192.stdout:1/768: chown dc/d16/d19/d74/df4 11522076 1 2024-04-23T02:14:11.525 INFO:tasks.workunit.client.0.smithi071.stdout:3/759: dwrite d7/d8/d42/f87 [0,4194304] 0 2024-04-23T02:14:11.527 INFO:tasks.workunit.client.0.smithi071.stdout:3/760: write d7/da5/db2/db7/d68/fdf [25077,108790] 0 2024-04-23T02:14:11.547 INFO:tasks.workunit.client.1.smithi192.stdout:5/926: creat d2/d6/d14/d113/f132 x:0 0 0 2024-04-23T02:14:11.547 INFO:tasks.workunit.client.1.smithi192.stdout:3/883: write d1/d3/d17/d65/fe5 [466819,91418] 0 2024-04-23T02:14:11.548 INFO:tasks.workunit.client.1.smithi192.stdout:9/793: unlink d0/d9/d44/d51/d5b/la5 0 2024-04-23T02:14:11.548 INFO:tasks.workunit.client.1.smithi192.stdout:4/961: dread de/d15/f31 [0,4194304] 0 2024-04-23T02:14:11.549 INFO:tasks.workunit.client.1.smithi192.stdout:6/995: mknod d3/d28/d79/da0/dc2/d101/d80/d8e/d9d/ded/c14b 0 2024-04-23T02:14:11.550 INFO:tasks.workunit.client.1.smithi192.stdout:9/794: chown d0/d9/d44/d74/c88 0 1 2024-04-23T02:14:11.552 INFO:tasks.workunit.client.1.smithi192.stdout:6/996: chown d3/d28/df8/d3a/d51/ddc/c125 486326171 1 2024-04-23T02:14:11.553 INFO:tasks.workunit.client.1.smithi192.stdout:4/962: chown de/d15/d53/d81/de3/d74/d92/da0/dca/d49/c44 105 1 2024-04-23T02:14:11.553 INFO:tasks.workunit.client.1.smithi192.stdout:3/884: mknod d1/d3/d6/d7/d71/c13a 0 2024-04-23T02:14:11.554 INFO:tasks.workunit.client.1.smithi192.stdout:6/997: chown d3/d28/df8/d7c/cb3 240985 1 2024-04-23T02:14:11.555 INFO:tasks.workunit.client.1.smithi192.stdout:7/945: getdents d5/d17/d1c/d57/dbb/df4 0 2024-04-23T02:14:11.555 INFO:tasks.workunit.client.1.smithi192.stdout:7/946: fsync d5/d43/d4c/d5c/ff1 0 2024-04-23T02:14:11.557 INFO:tasks.workunit.client.1.smithi192.stdout:4/963: fsync de/d15/d53/d81/fa6 0 2024-04-23T02:14:11.558 INFO:tasks.workunit.client.1.smithi192.stdout:9/795: rename d0/d9/d44/d53/c8f to d0/d9/d44/d58/dd9/c100 0 2024-04-23T02:14:11.559 INFO:tasks.workunit.client.1.smithi192.stdout:4/964: chown de/d15/d53/d81/de3/d74/d92/da0/dca/cdd 9813 1 2024-04-23T02:14:11.559 INFO:tasks.workunit.client.1.smithi192.stdout:9/796: chown d0/d9/d44/l5d 1634264 1 2024-04-23T02:14:11.560 INFO:tasks.workunit.client.1.smithi192.stdout:4/965: chown de/d15/d19/d1a/l52 931 1 2024-04-23T02:14:11.615 INFO:tasks.workunit.client.1.smithi192.stdout:7/947: creat d5/d17/d1c/d59/d5a/d70/de9/da6/da9/db9/dc0/f11e x:0 0 0 2024-04-23T02:14:11.615 INFO:tasks.workunit.client.1.smithi192.stdout:9/797: read d0/d9/db3/fba [5064242,73868] 0 2024-04-23T02:14:11.615 INFO:tasks.workunit.client.1.smithi192.stdout:6/998: truncate d3/d28/d79/da0/dc2/f126 1417033 0 2024-04-23T02:14:11.617 INFO:tasks.workunit.client.1.smithi192.stdout:9/798: readlink d0/d9/d44/d4f/le9 0 2024-04-23T02:14:11.618 INFO:tasks.workunit.client.1.smithi192.stdout:6/999: rename d3/d28/d79 to d3/d28/d79/da0/dc2/d101/d80/d107/d110/d14c 22 2024-04-23T02:14:11.618 INFO:tasks.workunit.client.1.smithi192.stdout:1/769: write dc/d16/d19/d30/f3f [7197263,29612] 0 2024-04-23T02:14:11.621 INFO:tasks.workunit.client.1.smithi192.stdout:7/948: mknod d5/d17/d1c/d57/dbb/c11f 0 2024-04-23T02:14:11.621 INFO:tasks.workunit.client.1.smithi192.stdout:4/966: fsync de/d15/f31 0 2024-04-23T02:14:11.622 INFO:tasks.workunit.client.1.smithi192.stdout:9/799: mknod d0/d9/dc7/da0/dd5/c101 0 2024-04-23T02:14:11.624 INFO:tasks.workunit.client.1.smithi192.stdout:1/770: creat dc/d8b/d34/d70/d8c/ff6 x:0 0 0 2024-04-23T02:14:11.625 INFO:tasks.workunit.client.1.smithi192.stdout:5/927: dwrite d2/db/d16/d1b/f79 [4194304,4194304] 0 2024-04-23T02:14:11.629 INFO:tasks.workunit.client.1.smithi192.stdout:5/928: write d2/db/d2b/d52/dc2/d49/fb1 [1973455,58262] 0 2024-04-23T02:14:11.631 INFO:tasks.workunit.client.1.smithi192.stdout:1/771: creat dc/d8b/d34/d70/d8c/da3/de4/de8/ff7 x:0 0 0 2024-04-23T02:14:11.632 INFO:tasks.workunit.client.1.smithi192.stdout:9/800: getdents d0/d9/d44/d53 0 2024-04-23T02:14:11.633 INFO:tasks.workunit.client.1.smithi192.stdout:9/801: fdatasync d0/d9/d44/d4f/fb2 0 2024-04-23T02:14:11.634 INFO:tasks.workunit.client.1.smithi192.stdout:5/929: mkdir d2/db/d133 0 2024-04-23T02:14:11.636 INFO:tasks.workunit.client.1.smithi192.stdout:1/772: mkdir dc/d14/d23/de0/df8 0 2024-04-23T02:14:11.638 INFO:tasks.workunit.client.1.smithi192.stdout:5/930: mknod d2/d2e/d4e/c134 0 2024-04-23T02:14:11.639 INFO:tasks.workunit.client.0.smithi071.stdout:6/755: dwrite d6/d1f/d5d/d95/fb8 [0,4194304] 0 2024-04-23T02:14:11.639 INFO:tasks.workunit.client.0.smithi071.stdout:0/842: dread d0/f20 [0,4194304] 0 2024-04-23T02:14:11.640 INFO:tasks.workunit.client.1.smithi192.stdout:5/931: fsync d2/db/d16/d48/d4f/f128 0 2024-04-23T02:14:11.641 INFO:tasks.workunit.client.1.smithi192.stdout:1/773: unlink dc/d8b/d44/da9/cc8 0 2024-04-23T02:14:11.641 INFO:tasks.workunit.client.0.smithi071.stdout:6/756: readlink d6/d1f/d5d/l72 0 2024-04-23T02:14:11.642 INFO:tasks.workunit.client.0.smithi071.stdout:6/757: chown d6/d91/f9c 0 1 2024-04-23T02:14:11.643 INFO:tasks.workunit.client.0.smithi071.stdout:1/727: dwrite db/df/d13/d20/f5c [0,4194304] 0 2024-04-23T02:14:11.644 INFO:tasks.workunit.client.1.smithi192.stdout:1/774: symlink dc/d80/lf9 0 2024-04-23T02:14:11.645 INFO:tasks.workunit.client.1.smithi192.stdout:9/802: dwrite d0/d9/db3/fba [4194304,4194304] 0 2024-04-23T02:14:11.645 INFO:tasks.workunit.client.0.smithi071.stdout:3/761: dwrite d7/d22/d23/f10a [0,4194304] 0 2024-04-23T02:14:11.650 INFO:tasks.workunit.client.1.smithi192.stdout:1/775: creat dc/d8b/d34/d70/d8c/da3/dc1/ffa x:0 0 0 2024-04-23T02:14:11.652 INFO:tasks.workunit.client.1.smithi192.stdout:1/776: chown dc/d14/d23/d5e/d6a 126 1 2024-04-23T02:14:11.671 INFO:tasks.workunit.client.1.smithi192.stdout:7/949: dread d5/d17/d1c/d59/d5a/d41/d50/f35 [4194304,4194304] 0 2024-04-23T02:14:11.673 INFO:tasks.workunit.client.1.smithi192.stdout:7/950: write d5/d17/d1c/d57/f33 [2456698,20970] 0 2024-04-23T02:14:11.674 INFO:tasks.workunit.client.1.smithi192.stdout:5/932: write d2/db/d16/f26 [2663543,99202] 0 2024-04-23T02:14:11.676 INFO:tasks.workunit.client.1.smithi192.stdout:5/933: write d2/d32/f9c [4528404,28679] 0 2024-04-23T02:14:11.678 INFO:tasks.workunit.client.1.smithi192.stdout:5/934: read - d2/d6/fcb zero size 2024-04-23T02:14:11.679 INFO:tasks.workunit.client.1.smithi192.stdout:7/951: mknod d5/d17/d1c/d59/d5a/d70/de9/c120 0 2024-04-23T02:14:11.712 INFO:tasks.workunit.client.0.smithi071.stdout:4/834: dread d2/d2a/d2d/d39/d42/d49/d67/d3e/f7d [0,4194304] 0 2024-04-23T02:14:11.714 INFO:tasks.workunit.client.0.smithi071.stdout:6/758: mknod d6/d17/d77/d9f/dad/d3f/cef 0 2024-04-23T02:14:11.715 INFO:tasks.workunit.client.0.smithi071.stdout:0/843: getdents d0/d3/de/d48/d120 0 2024-04-23T02:14:11.720 INFO:tasks.workunit.client.0.smithi071.stdout:1/728: mknod db/df/d13/d20/de5/cf8 0 2024-04-23T02:14:11.724 INFO:tasks.workunit.client.0.smithi071.stdout:4/835: fdatasync d2/f89 0 2024-04-23T02:14:11.728 INFO:tasks.workunit.client.0.smithi071.stdout:4/836: chown d2/d2a/da2/dd1/dde/d23/f5d 866 1 2024-04-23T02:14:11.730 INFO:tasks.workunit.client.0.smithi071.stdout:1/729: dwrite db/d11/d8d/fd8 [0,4194304] 0 2024-04-23T02:14:11.734 INFO:tasks.workunit.client.0.smithi071.stdout:1/730: rmdir db/d11/d7f/da7/dd9 39 2024-04-23T02:14:11.736 INFO:tasks.workunit.client.0.smithi071.stdout:1/731: symlink db/df/d5f/df0/lf9 0 2024-04-23T02:14:11.739 INFO:tasks.workunit.client.0.smithi071.stdout:1/732: creat db/df/d13/d20/dc0/ffa x:0 0 0 2024-04-23T02:14:11.739 INFO:tasks.workunit.client.0.smithi071.stdout:1/733: readlink db/l23 0 2024-04-23T02:14:11.741 INFO:tasks.workunit.client.0.smithi071.stdout:1/734: mkdir db/df/d13/d2f/d48/d9f/dfb 0 2024-04-23T02:14:11.744 INFO:tasks.workunit.client.0.smithi071.stdout:1/735: truncate db/d34/d68/f77 907975 0 2024-04-23T02:14:11.747 INFO:tasks.workunit.client.0.smithi071.stdout:1/736: symlink db/df/dc2/lfc 0 2024-04-23T02:14:11.750 INFO:tasks.workunit.client.0.smithi071.stdout:1/737: mkdir db/d11/d7f/da7/dd9/dfd 0 2024-04-23T02:14:11.757 INFO:tasks.workunit.client.0.smithi071.stdout:1/738: dwrite db/df/ff4 [0,4194304] 0 2024-04-23T02:14:11.757 INFO:tasks.workunit.client.1.smithi192.stdout:5/935: chown d2/db/d2b/d52/f92 8 1 2024-04-23T02:14:11.760 INFO:tasks.workunit.client.1.smithi192.stdout:7/952: creat d5/d17/f121 x:0 0 0 2024-04-23T02:14:11.761 INFO:tasks.workunit.client.1.smithi192.stdout:9/803: dread d0/d9/db3/d9e/ffa [0,4194304] 0 2024-04-23T02:14:11.763 INFO:tasks.workunit.client.0.smithi071.stdout:1/739: creat db/df/d13/d20/dc0/ddf/de8/ffe x:0 0 0 2024-04-23T02:14:11.765 INFO:tasks.workunit.client.0.smithi071.stdout:1/740: mknod db/d92/cff 0 2024-04-23T02:14:11.766 INFO:tasks.workunit.client.1.smithi192.stdout:7/953: mknod d5/d17/db5/c122 0 2024-04-23T02:14:11.767 INFO:tasks.workunit.client.0.smithi071.stdout:1/741: fsync db/df/d3a/fa8 0 2024-04-23T02:14:11.768 INFO:tasks.workunit.client.1.smithi192.stdout:5/936: dwrite d2/d6/d2d/d41/f12b [0,4194304] 0 2024-04-23T02:14:11.775 INFO:tasks.workunit.client.0.smithi071.stdout:3/762: mknod d7/d8/d42/c110 0 2024-04-23T02:14:11.775 INFO:tasks.workunit.client.0.smithi071.stdout:3/763: read - d7/d22/fe5 zero size 2024-04-23T02:14:11.777 INFO:tasks.workunit.client.1.smithi192.stdout:7/954: rename d5/d43/d4c/d5c/d97/c9d to d5/d17/d1c/d4a/daa/de5/c123 0 2024-04-23T02:14:11.777 INFO:tasks.workunit.client.1.smithi192.stdout:9/804: link d0/d9/d44/d58/d89/dd8/f78 d0/f102 0 2024-04-23T02:14:11.777 INFO:tasks.workunit.client.1.smithi192.stdout:7/955: dread - d5/d43/d4c/d5c/d97/fea zero size 2024-04-23T02:14:11.778 INFO:tasks.workunit.client.1.smithi192.stdout:5/937: rename d2/d32/d106 to d2/db/d16/d48/db7/d135 0 2024-04-23T02:14:11.782 INFO:tasks.workunit.client.1.smithi192.stdout:9/805: fsync d0/d9/d44/d4f/f3d 0 2024-04-23T02:14:11.784 INFO:tasks.workunit.client.0.smithi071.stdout:3/764: read d7/d8/fc [3694216,98985] 0 2024-04-23T02:14:11.784 INFO:tasks.workunit.client.1.smithi192.stdout:9/806: fsync d0/d9/d44/d58/d89/ff1 0 2024-04-23T02:14:11.784 INFO:tasks.workunit.client.1.smithi192.stdout:7/956: creat d5/d43/d4c/d5c/f124 x:0 0 0 2024-04-23T02:14:11.785 INFO:tasks.workunit.client.0.smithi071.stdout:3/765: fsync d7/d10/f2f 0 2024-04-23T02:14:11.788 INFO:tasks.workunit.client.0.smithi071.stdout:0/844: mkdir d0/d92/d127 0 2024-04-23T02:14:11.788 INFO:tasks.workunit.client.1.smithi192.stdout:7/957: readlink d5/l16 0 2024-04-23T02:14:11.789 INFO:tasks.workunit.client.1.smithi192.stdout:5/938: dwrite d2/db/d16/d1b/f1f [0,4194304] 0 2024-04-23T02:14:11.790 INFO:tasks.workunit.client.1.smithi192.stdout:5/939: chown d2/db/f9a 0 1 2024-04-23T02:14:11.792 INFO:tasks.workunit.client.1.smithi192.stdout:9/807: getdents d0/d9/d44/d58/dd9/dda 0 2024-04-23T02:14:11.793 INFO:tasks.workunit.client.0.smithi071.stdout:0/845: write d0/d3/de/d10/d14/d32/d37/d26/fc2 [563877,45994] 0 2024-04-23T02:14:11.794 INFO:tasks.workunit.client.1.smithi192.stdout:7/958: creat d5/d17/d1c/d59/d5a/d9b/f125 x:0 0 0 2024-04-23T02:14:11.794 INFO:tasks.workunit.client.0.smithi071.stdout:3/766: dwrite d7/d1d/f72 [0,4194304] 0 2024-04-23T02:14:11.795 INFO:tasks.workunit.client.1.smithi192.stdout:9/808: rename d0/d9/d44 to d0/d9/d44/d51/d103 22 2024-04-23T02:14:11.796 INFO:tasks.workunit.client.0.smithi071.stdout:3/767: stat d7/d22/d23/d26/d52 0 2024-04-23T02:14:11.797 INFO:tasks.workunit.client.1.smithi192.stdout:7/959: mknod d5/d17/d1c/d59/d5a/d41/d50/d9f/c126 0 2024-04-23T02:14:11.797 INFO:tasks.workunit.client.1.smithi192.stdout:9/809: fsync d0/d9/d44/d51/d5b/f67 0 2024-04-23T02:14:11.800 INFO:tasks.workunit.client.1.smithi192.stdout:7/960: symlink d5/d17/d1c/d57/d8b/l127 0 2024-04-23T02:14:11.801 INFO:tasks.workunit.client.0.smithi071.stdout:3/768: creat d7/da5/d100/f111 x:0 0 0 2024-04-23T02:14:11.801 INFO:tasks.workunit.client.1.smithi192.stdout:7/961: chown d5/d17/d1c/d59/d5a/d41/dfd/f100 400264 1 2024-04-23T02:14:11.802 INFO:tasks.workunit.client.1.smithi192.stdout:9/810: symlink d0/d9/db3/d5f/l104 0 2024-04-23T02:14:11.805 INFO:tasks.workunit.client.1.smithi192.stdout:9/811: dread - d0/d9/d44/d51/d5b/d61/faf zero size 2024-04-23T02:14:11.806 INFO:tasks.workunit.client.1.smithi192.stdout:3/885: sync 2024-04-23T02:14:11.806 INFO:tasks.workunit.client.1.smithi192.stdout:9/812: fsync d0/d9/d44/d51/d5b/d61/faf 0 2024-04-23T02:14:11.806 INFO:tasks.workunit.client.1.smithi192.stdout:7/962: mkdir d5/d17/d1c/d59/d5a/d41/dfd/d128 0 2024-04-23T02:14:11.810 INFO:tasks.workunit.client.1.smithi192.stdout:3/886: creat d1/d3/d6/d7/d71/f13b x:0 0 0 2024-04-23T02:14:11.811 INFO:tasks.workunit.client.0.smithi071.stdout:1/742: rename db/d34/d68 to db/d11/d7f/da7/dd9/d100 0 2024-04-23T02:14:11.812 INFO:tasks.workunit.client.1.smithi192.stdout:7/963: chown d5/d17/d1c/d59/d5a/d70/de9/da6/da9/lfc 63761 1 2024-04-23T02:14:11.812 INFO:tasks.workunit.client.1.smithi192.stdout:5/940: truncate d2/f27 2566943 0 2024-04-23T02:14:11.813 INFO:tasks.workunit.client.0.smithi071.stdout:1/743: mkdir db/d34/d8a/d101 0 2024-04-23T02:14:11.814 INFO:tasks.workunit.client.1.smithi192.stdout:3/887: write d1/d3/d6/d7/d71/f13b [617190,123356] 0 2024-04-23T02:14:11.815 INFO:tasks.workunit.client.0.smithi071.stdout:1/744: read db/df/d5f/d6f/db2/f79 [610724,107936] 0 2024-04-23T02:14:11.816 INFO:tasks.workunit.client.1.smithi192.stdout:9/813: dwrite d0/f2c [0,4194304] 0 2024-04-23T02:14:11.816 INFO:tasks.workunit.client.1.smithi192.stdout:7/964: creat d5/d17/d1c/d57/dbb/df4/f129 x:0 0 0 2024-04-23T02:14:11.818 INFO:tasks.workunit.client.0.smithi071.stdout:1/745: creat db/d11/d7f/da7/dd9/d100/f102 x:0 0 0 2024-04-23T02:14:11.818 INFO:tasks.workunit.client.1.smithi192.stdout:9/814: write d0/d9/db3/d9e/fe5 [5873877,4333] 0 2024-04-23T02:14:11.819 INFO:tasks.workunit.client.0.smithi071.stdout:1/746: chown db/df/d13/d20/cbd 47701 1 2024-04-23T02:14:11.821 INFO:tasks.workunit.client.0.smithi071.stdout:0/846: mknod d0/d3/de/d10/d14/d32/c128 0 2024-04-23T02:14:11.821 INFO:tasks.workunit.client.1.smithi192.stdout:9/815: write d0/d9/f32 [2521921,119100] 0 2024-04-23T02:14:11.824 INFO:tasks.workunit.client.1.smithi192.stdout:3/888: fsync d1/d3/d6/d7/d29/d3b/da9/d8b/fb6 0 2024-04-23T02:14:11.825 INFO:tasks.workunit.client.1.smithi192.stdout:9/816: chown d0/d9/dc7/lbb 38884 1 2024-04-23T02:14:11.825 INFO:tasks.workunit.client.0.smithi071.stdout:0/847: rename d0/d3/de/d10/d14/d32/lfe to d0/d3/de/d10/d14/d32/da8/l129 0 2024-04-23T02:14:11.826 INFO:tasks.workunit.client.1.smithi192.stdout:3/889: stat d1/d3/d17/d1f/cc8 0 2024-04-23T02:14:11.827 INFO:tasks.workunit.client.1.smithi192.stdout:3/890: chown d1/d3/d17/d65 4901 1 2024-04-23T02:14:11.827 INFO:tasks.workunit.client.0.smithi071.stdout:0/848: creat d0/d3/d95/f12a x:0 0 0 2024-04-23T02:14:11.828 INFO:tasks.workunit.client.0.smithi071.stdout:3/769: creat d7/da5/db2/f112 x:0 0 0 2024-04-23T02:14:11.829 INFO:tasks.workunit.client.1.smithi192.stdout:3/891: rmdir d1/d3/d17/d2b/d44/d57/db0 39 2024-04-23T02:14:11.830 INFO:tasks.workunit.client.1.smithi192.stdout:7/965: dread d5/d17/d1c/d59/d5a/d41/f98 [0,4194304] 0 2024-04-23T02:14:11.831 INFO:tasks.workunit.client.1.smithi192.stdout:7/966: stat d5/d17/d1c/d59/d5a/d70/de9/da6/fb6 0 2024-04-23T02:14:11.832 INFO:tasks.workunit.client.1.smithi192.stdout:3/892: creat d1/d3/d6/d7/df1/df7/f13c x:0 0 0 2024-04-23T02:14:11.839 INFO:tasks.workunit.client.1.smithi192.stdout:3/893: rename d1/d3/d17/d1a/d7c/dab/d6c/d84/d4c to d1/d3/d6/d7/d29/d3b/da9/daf/ddf/d12f/d13d 0 2024-04-23T02:14:11.840 INFO:tasks.workunit.client.1.smithi192.stdout:7/967: dwrite d5/d17/d1c/d59/d5a/d70/f7e [0,4194304] 0 2024-04-23T02:14:11.841 INFO:tasks.workunit.client.1.smithi192.stdout:9/817: write d0/d9/d44/d58/d89/dd8/fac [3526373,1453] 0 2024-04-23T02:14:11.842 INFO:tasks.workunit.client.1.smithi192.stdout:3/894: readlink d1/d3/d17/d65/d99/lef 0 2024-04-23T02:14:11.845 INFO:tasks.workunit.client.1.smithi192.stdout:9/818: chown d0/d9/db3/d5f/l9a 18 1 2024-04-23T02:14:11.847 INFO:tasks.workunit.client.1.smithi192.stdout:9/819: chown d0/d9/d44/d58/d89/c8c 620783 1 2024-04-23T02:14:11.848 INFO:tasks.workunit.client.1.smithi192.stdout:9/820: dread - d0/d9/d44/d51/d5b/d61/faf zero size 2024-04-23T02:14:11.849 INFO:tasks.workunit.client.1.smithi192.stdout:9/821: chown d0/fdb 17204 1 2024-04-23T02:14:11.885 INFO:tasks.workunit.client.0.smithi071.stdout:5/872: sync 2024-04-23T02:14:11.885 INFO:tasks.workunit.client.0.smithi071.stdout:9/759: sync 2024-04-23T02:14:11.887 INFO:tasks.workunit.client.0.smithi071.stdout:0/849: creat d0/d3/de/d10/d14/d32/dac/f12b x:0 0 0 2024-04-23T02:14:11.890 INFO:tasks.workunit.client.0.smithi071.stdout:3/770: truncate d7/d8/f61 507326 0 2024-04-23T02:14:11.891 INFO:tasks.workunit.client.1.smithi192.stdout:9/822: write d0/d9/db3/d5f/fc8 [65809,116337] 0 2024-04-23T02:14:11.891 INFO:tasks.workunit.client.0.smithi071.stdout:0/850: read d0/d3/de/d10/d14/d32/d44/f43 [3184966,85066] 0 2024-04-23T02:14:11.892 INFO:tasks.workunit.client.1.smithi192.stdout:9/823: chown d0/d9/d44/d51/fc4 11 1 2024-04-23T02:14:11.893 INFO:tasks.workunit.client.0.smithi071.stdout:5/873: rename d3/da/def to d3/d8f/de6/d143 0 2024-04-23T02:14:11.894 INFO:tasks.workunit.client.0.smithi071.stdout:5/874: chown d3/d1d/daf/d83/ffb 6945298 1 2024-04-23T02:14:11.895 INFO:tasks.workunit.client.1.smithi192.stdout:3/895: dwrite d1/d3/d6/d7/d29/d3b/da9/daf/ddf/f107 [0,4194304] 0 2024-04-23T02:14:11.896 INFO:tasks.workunit.client.0.smithi071.stdout:3/771: rmdir d7/da5/db2/db7/d68/d3b/d98 39 2024-04-23T02:14:11.897 INFO:tasks.workunit.client.1.smithi192.stdout:3/896: write d1/d3/d17/d1a/f4f [4801792,24095] 0 2024-04-23T02:14:11.897 INFO:tasks.workunit.client.1.smithi192.stdout:9/824: creat d0/d9/db3/d9e/dff/f105 x:0 0 0 2024-04-23T02:14:11.899 INFO:tasks.workunit.client.0.smithi071.stdout:0/851: unlink d0/d3/de/d10/d14/d32/d37/fa2 0 2024-04-23T02:14:11.900 INFO:tasks.workunit.client.1.smithi192.stdout:9/825: mkdir d0/d9/dc7/d106 0 2024-04-23T02:14:11.901 INFO:tasks.workunit.client.1.smithi192.stdout:3/897: symlink d1/d3/d6/d7/d29/d3b/da9/daf/l13e 0 2024-04-23T02:14:11.901 INFO:tasks.workunit.client.0.smithi071.stdout:0/852: chown d0/d3/de/d10/d14/d32/d37/d26/lf6 49 1 2024-04-23T02:14:11.902 INFO:tasks.workunit.client.0.smithi071.stdout:9/760: mknod d1/d32/d41/db7/d5a/dc6/d9f/ded/c10e 0 2024-04-23T02:14:11.903 INFO:tasks.workunit.client.1.smithi192.stdout:3/898: fsync d1/d3/d17/d1a/d7c/dab/d6c/d84/fcb 0 2024-04-23T02:14:11.904 INFO:tasks.workunit.client.0.smithi071.stdout:5/875: truncate d3/da/d1a/db8/fca 2805125 0 2024-04-23T02:14:11.906 INFO:tasks.workunit.client.0.smithi071.stdout:5/876: read d3/d34/f8b [118583,123756] 0 2024-04-23T02:14:11.906 INFO:tasks.workunit.client.0.smithi071.stdout:3/772: creat d7/d1d/f113 x:0 0 0 2024-04-23T02:14:11.907 INFO:tasks.workunit.client.0.smithi071.stdout:0/853: creat d0/d3/de/d10/d14/d32/da8/f12c x:0 0 0 2024-04-23T02:14:11.908 INFO:tasks.workunit.client.1.smithi192.stdout:7/968: dread d5/d43/f6e [0,4194304] 0 2024-04-23T02:14:11.909 INFO:tasks.workunit.client.1.smithi192.stdout:9/826: dwrite d0/d9/d44/d74/f82 [4194304,4194304] 0 2024-04-23T02:14:11.911 INFO:tasks.workunit.client.0.smithi071.stdout:1/747: write db/df/f39 [3351460,37161] 0 2024-04-23T02:14:11.912 INFO:tasks.workunit.client.1.smithi192.stdout:9/827: write d0/d9/d44/d51/d5b/d61/db1/fcf [3087763,79276] 0 2024-04-23T02:14:11.912 INFO:tasks.workunit.client.0.smithi071.stdout:1/748: chown db/d11/f7a 0 1 2024-04-23T02:14:11.913 INFO:tasks.workunit.client.0.smithi071.stdout:5/877: unlink d3/d1d/daf/d83/fac 0 2024-04-23T02:14:11.913 INFO:tasks.workunit.client.0.smithi071.stdout:1/749: chown db/d11/l94 0 1 2024-04-23T02:14:11.914 INFO:tasks.workunit.client.1.smithi192.stdout:7/969: truncate d5/d17/d1c/d59/d5a/d70/de9/da6/da9/db9/dc0/fdb 5450794 0 2024-04-23T02:14:11.915 INFO:tasks.workunit.client.1.smithi192.stdout:9/828: mknod d0/dd2/c107 0 2024-04-23T02:14:11.916 INFO:tasks.workunit.client.1.smithi192.stdout:3/899: write d1/d3/d6/d7/d29/d3b/fdd [675502,5012] 0 2024-04-23T02:14:11.916 INFO:tasks.workunit.client.0.smithi071.stdout:3/773: dread d7/da5/db2/db7/f8c [0,4194304] 0 2024-04-23T02:14:11.917 INFO:tasks.workunit.client.0.smithi071.stdout:0/854: mkdir d0/d63/d12d 0 2024-04-23T02:14:11.918 INFO:tasks.workunit.client.1.smithi192.stdout:9/829: mknod d0/d9/dc7/d106/c108 0 2024-04-23T02:14:11.918 INFO:tasks.workunit.client.1.smithi192.stdout:3/900: write d1/d3/d6/d7/d29/d3b/da9/f121 [499514,25734] 0 2024-04-23T02:14:11.919 INFO:tasks.workunit.client.1.smithi192.stdout:7/970: symlink d5/d17/d1c/d59/d5a/d70/de9/da6/da9/db9/dc0/def/l12a 0 2024-04-23T02:14:11.923 INFO:tasks.workunit.client.1.smithi192.stdout:3/901: unlink d1/d3/d17/d65/d99/fa5 0 2024-04-23T02:14:11.924 INFO:tasks.workunit.client.1.smithi192.stdout:9/830: truncate d0/d9/d44/f5e 439961 0 2024-04-23T02:14:11.927 INFO:tasks.workunit.client.0.smithi071.stdout:9/761: dread d1/d32/d3a/d90/fec [0,4194304] 0 2024-04-23T02:14:11.927 INFO:tasks.workunit.client.1.smithi192.stdout:9/831: write d0/d9/d44/d51/ff5 [267054,47041] 0 2024-04-23T02:14:11.928 INFO:tasks.workunit.client.0.smithi071.stdout:0/855: dread d0/d3/de/d10/d14/d32/d8a/df0/fe6 [0,4194304] 0 2024-04-23T02:14:11.929 INFO:tasks.workunit.client.1.smithi192.stdout:3/902: getdents d1/d3/d17/d1a 0 2024-04-23T02:14:11.929 INFO:tasks.workunit.client.0.smithi071.stdout:1/750: mknod db/d34/d8a/de1/c103 0 2024-04-23T02:14:11.930 INFO:tasks.workunit.client.1.smithi192.stdout:9/832: stat d0/d9/d44/d51/lf3 0 2024-04-23T02:14:11.930 INFO:tasks.workunit.client.0.smithi071.stdout:3/774: mkdir d7/d8/dea/df6/d114 0 2024-04-23T02:14:11.932 INFO:tasks.workunit.client.1.smithi192.stdout:7/971: dwrite d5/d17/d1c/d59/d5a/d70/f113 [0,4194304] 0 2024-04-23T02:14:11.932 INFO:tasks.workunit.client.0.smithi071.stdout:9/762: creat d1/d32/d41/f10f x:0 0 0 2024-04-23T02:14:11.933 INFO:tasks.workunit.client.1.smithi192.stdout:9/833: write d0/d9/d44/d51/fb5 [818939,49808] 0 2024-04-23T02:14:11.936 INFO:tasks.workunit.client.1.smithi192.stdout:9/834: write d0/d9/db3/d5f/fc8 [972813,104487] 0 2024-04-23T02:14:11.936 INFO:tasks.workunit.client.0.smithi071.stdout:0/856: symlink d0/d3/de/d10/d14/l12e 0 2024-04-23T02:14:11.938 INFO:tasks.workunit.client.0.smithi071.stdout:1/751: creat db/df/d5f/df6/f104 x:0 0 0 2024-04-23T02:14:11.938 INFO:tasks.workunit.client.1.smithi192.stdout:9/835: chown d0/d9/dc7/lbb 13794275 1 2024-04-23T02:14:11.939 INFO:tasks.workunit.client.0.smithi071.stdout:9/763: truncate d1/d32/d3a/d94/f9b 686740 0 2024-04-23T02:14:11.939 INFO:tasks.workunit.client.1.smithi192.stdout:7/972: unlink d5/d17/d1c/d57/d8b/f10b 0 2024-04-23T02:14:11.940 INFO:tasks.workunit.client.0.smithi071.stdout:0/857: dread d0/d76/fab [0,4194304] 0 2024-04-23T02:14:11.941 INFO:tasks.workunit.client.1.smithi192.stdout:3/903: getdents d1/d3/d17/dd1/dda 0 2024-04-23T02:14:11.941 INFO:tasks.workunit.client.0.smithi071.stdout:1/752: stat db/d34/d8a/led 0 2024-04-23T02:14:11.942 INFO:tasks.workunit.client.0.smithi071.stdout:9/764: readlink d1/d32/d41/l47 0 2024-04-23T02:14:11.943 INFO:tasks.workunit.client.1.smithi192.stdout:9/836: dread - d0/d9/d44/d51/d5b/d61/db1/fa3 zero size 2024-04-23T02:14:11.944 INFO:tasks.workunit.client.1.smithi192.stdout:9/837: fsync d0/fd1 0 2024-04-23T02:14:11.944 INFO:tasks.workunit.client.0.smithi071.stdout:5/878: dwrite d3/da/d1a/db8/d140/dd2/fda [0,4194304] 0 2024-04-23T02:14:11.944 INFO:tasks.workunit.client.1.smithi192.stdout:7/973: symlink d5/d17/d1c/d57/dbb/df4/l12b 0 2024-04-23T02:14:11.945 INFO:tasks.workunit.client.1.smithi192.stdout:9/838: chown d0/d9/db3/c60 2 1 2024-04-23T02:14:11.946 INFO:tasks.workunit.client.0.smithi071.stdout:9/765: creat d1/dfd/f110 x:0 0 0 2024-04-23T02:14:11.946 INFO:tasks.workunit.client.1.smithi192.stdout:3/904: mknod d1/d3/d6/d7/d29/d3b/da9/daf/ddf/d12f/d13d/c13f 0 2024-04-23T02:14:11.947 INFO:tasks.workunit.client.1.smithi192.stdout:4/967: sync 2024-04-23T02:14:11.947 INFO:tasks.workunit.client.0.smithi071.stdout:5/879: dread - d3/d1d/daf/d10/da5/d5f/dc0/f10c zero size 2024-04-23T02:14:11.950 INFO:tasks.workunit.client.1.smithi192.stdout:9/839: mkdir d0/d9/dc7/da0/dd5/d109 0 2024-04-23T02:14:11.951 INFO:tasks.workunit.client.0.smithi071.stdout:1/753: rename db/df/d13/d20/f32 to db/d92/f105 0 2024-04-23T02:14:11.951 INFO:tasks.workunit.client.0.smithi071.stdout:5/880: write d3/da/d1a/db8/d140/dd2/f78 [6689119,86829] 0 2024-04-23T02:14:11.952 INFO:tasks.workunit.client.0.smithi071.stdout:3/775: link d7/da5/db2/db7/d68/d3b/d98/c99 d7/da5/db2/db7/d68/d3b/d98/d104/c115 0 2024-04-23T02:14:11.953 INFO:tasks.workunit.client.0.smithi071.stdout:9/766: creat d1/d32/d41/db7/d5a/dc6/d9f/ded/f111 x:0 0 0 2024-04-23T02:14:11.953 INFO:tasks.workunit.client.1.smithi192.stdout:3/905: rmdir d1/d3/d17/d11d 0 2024-04-23T02:14:11.954 INFO:tasks.workunit.client.1.smithi192.stdout:4/968: readlink de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d11b/l102 0 2024-04-23T02:14:11.954 INFO:tasks.workunit.client.1.smithi192.stdout:7/974: dwrite d5/d17/d1c/f74 [12582912,4194304] 0 2024-04-23T02:14:11.955 INFO:tasks.workunit.client.0.smithi071.stdout:0/858: creat d0/d3/de/d10/d14/d32/d44/d31/d52/f12f x:0 0 0 2024-04-23T02:14:11.957 INFO:tasks.workunit.client.1.smithi192.stdout:9/840: rename d0/d9/d44/d74/de7/lf9 to d0/d9/d44/d74/de7/l10a 0 2024-04-23T02:14:11.958 INFO:tasks.workunit.client.0.smithi071.stdout:0/859: dread d0/d3/de/d10/d14/d32/d8a/df0/fe6 [0,4194304] 0 2024-04-23T02:14:11.960 INFO:tasks.workunit.client.1.smithi192.stdout:3/906: dread - d1/d3/d6/d7/d29/d3b/da9/d8b/fc3 zero size 2024-04-23T02:14:11.961 INFO:tasks.workunit.client.1.smithi192.stdout:9/841: write d0/d9/d44/d4f/f8e [4359056,28583] 0 2024-04-23T02:14:11.961 INFO:tasks.workunit.client.1.smithi192.stdout:3/907: dread - d1/d3/d17/d2b/f130 zero size 2024-04-23T02:14:11.962 INFO:tasks.workunit.client.0.smithi071.stdout:3/776: read d7/da5/db2/db7/d68/f62 [499412,90869] 0 2024-04-23T02:14:11.962 INFO:tasks.workunit.client.1.smithi192.stdout:7/975: creat d5/d17/d1c/d59/d5a/d70/de9/da6/da9/db9/dc0/f12c x:0 0 0 2024-04-23T02:14:11.963 INFO:tasks.workunit.client.0.smithi071.stdout:5/881: mkdir d3/d1d/daf/d10/da5/d5f/dc0/dfa/d13b/d144 0 2024-04-23T02:14:11.964 INFO:tasks.workunit.client.1.smithi192.stdout:4/969: mknod de/d15/d53/d81/de3/d74/d92/da0/dca/d34/d12d/c144 0 2024-04-23T02:14:11.965 INFO:tasks.workunit.client.1.smithi192.stdout:9/842: symlink d0/d9/d44/d74/de7/l10b 0 2024-04-23T02:14:11.966 INFO:tasks.workunit.client.1.smithi192.stdout:4/970: stat de/d15/d53/d81/de3/d97/f120 0 2024-04-23T02:14:11.969 INFO:tasks.workunit.client.1.smithi192.stdout:3/908: rename d1/d3/d6/d7/d29/d3b/da9/f70 to d1/d3/d17/d1f/f140 0 2024-04-23T02:14:11.970 INFO:tasks.workunit.client.1.smithi192.stdout:3/909: fdatasync d1/d3/d17/d1f/dc2/f115 0 2024-04-23T02:14:11.970 INFO:tasks.workunit.client.0.smithi071.stdout:1/754: write db/df/d13/d2f/d48/d9f/fa5 [291488,83325] 0 2024-04-23T02:14:11.970 INFO:tasks.workunit.client.1.smithi192.stdout:7/976: link d5/d17/d1c/d59/d5a/d9b/f125 d5/d43/d4c/d5c/f12d 0 2024-04-23T02:14:11.973 INFO:tasks.workunit.client.1.smithi192.stdout:4/971: creat de/d15/d53/d81/de3/d74/d92/da0/dca/d49/f145 x:0 0 0 2024-04-23T02:14:11.973 INFO:tasks.workunit.client.0.smithi071.stdout:9/767: write d1/d32/dc5/dd1/f9d [782137,119312] 0 2024-04-23T02:14:11.974 INFO:tasks.workunit.client.1.smithi192.stdout:9/843: creat d0/d9/d44/d4f/f10c x:0 0 0 2024-04-23T02:14:11.974 INFO:tasks.workunit.client.0.smithi071.stdout:1/755: mkdir db/d34/d106 0 2024-04-23T02:14:11.975 INFO:tasks.workunit.client.0.smithi071.stdout:3/777: getdents d7/da5/db2/db7/d68/d3b 0 2024-04-23T02:14:11.977 INFO:tasks.workunit.client.1.smithi192.stdout:3/910: getdents d1/d3/d17/d2b/d9a 0 2024-04-23T02:14:11.978 INFO:tasks.workunit.client.0.smithi071.stdout:0/860: truncate d0/d3/de/d10/f1f 2056382 0 2024-04-23T02:14:11.978 INFO:tasks.workunit.client.1.smithi192.stdout:4/972: symlink de/d15/dbb/d136/l146 0 2024-04-23T02:14:11.981 INFO:tasks.workunit.client.1.smithi192.stdout:7/977: dwrite d5/d43/d4c/d5c/f72 [0,4194304] 0 2024-04-23T02:14:11.983 INFO:tasks.workunit.client.0.smithi071.stdout:3/778: write d7/d8/f1a [3975157,116758] 0 2024-04-23T02:14:11.983 INFO:tasks.workunit.client.0.smithi071.stdout:9/768: creat d1/d4e/d54/f112 x:0 0 0 2024-04-23T02:14:11.984 INFO:tasks.workunit.client.0.smithi071.stdout:1/756: dwrite db/df/f58 [0,4194304] 0 2024-04-23T02:14:11.986 INFO:tasks.workunit.client.1.smithi192.stdout:7/978: readlink d5/d17/d1c/d26/l95 0 2024-04-23T02:14:11.987 INFO:tasks.workunit.client.1.smithi192.stdout:3/911: dwrite d1/d3/d6/d7/d29/fe2 [0,4194304] 0 2024-04-23T02:14:11.988 INFO:tasks.workunit.client.1.smithi192.stdout:7/979: chown d5/d43/d4c/d4e/fc6 0 1 2024-04-23T02:14:11.991 INFO:tasks.workunit.client.0.smithi071.stdout:3/779: read d7/d22/d23/d26/f2b [54105,31357] 0 2024-04-23T02:14:11.996 INFO:tasks.workunit.client.0.smithi071.stdout:5/882: dwrite d3/da/d1a/d1c/f62 [0,4194304] 0 2024-04-23T02:14:11.997 INFO:tasks.workunit.client.1.smithi192.stdout:3/912: mkdir d1/d3/d17/d1f/d141 0 2024-04-23T02:14:11.998 INFO:tasks.workunit.client.1.smithi192.stdout:7/980: fsync d5/d17/d1c/d59/da8/fb0 0 2024-04-23T02:14:11.998 INFO:tasks.workunit.client.1.smithi192.stdout:4/973: link de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d93/fef de/d15/d53/d81/de3/f147 0 2024-04-23T02:14:11.999 INFO:tasks.workunit.client.0.smithi071.stdout:0/861: dwrite d0/d63/f101 [0,4194304] 0 2024-04-23T02:14:12.001 INFO:tasks.workunit.client.1.smithi192.stdout:3/913: unlink d1/d3/d6/d7/d29/d3b/da9/d8b/fac 0 2024-04-23T02:14:12.001 INFO:tasks.workunit.client.0.smithi071.stdout:3/780: creat d7/da5/db2/db7/f116 x:0 0 0 2024-04-23T02:14:12.002 INFO:tasks.workunit.client.1.smithi192.stdout:9/844: dwrite d0/d9/d44/d51/d5b/f67 [0,4194304] 0 2024-04-23T02:14:12.002 INFO:tasks.workunit.client.1.smithi192.stdout:4/974: fsync de/d15/f31 0 2024-04-23T02:14:12.004 INFO:tasks.workunit.client.1.smithi192.stdout:7/981: fsync d5/d17/d1c/d59/d5a/d41/d50/d9f/f117 0 2024-04-23T02:14:12.004 INFO:tasks.workunit.client.0.smithi071.stdout:0/862: dread - d0/d3/de/d10/db9/f123 zero size 2024-04-23T02:14:12.004 INFO:tasks.workunit.client.0.smithi071.stdout:3/781: write d7/da5/db2/fe6 [1004859,121433] 0 2024-04-23T02:14:12.006 INFO:tasks.workunit.client.0.smithi071.stdout:9/769: dread d1/d32/d41/db7/d39/d42/dcc/ff2 [0,4194304] 0 2024-04-23T02:14:12.009 INFO:tasks.workunit.client.1.smithi192.stdout:7/982: mkdir d5/d17/d1c/d57/d75/dee/d12e 0 2024-04-23T02:14:12.011 INFO:tasks.workunit.client.1.smithi192.stdout:7/983: dread - d5/d17/d1c/d59/d5a/d9b/f125 zero size 2024-04-23T02:14:12.011 INFO:tasks.workunit.client.1.smithi192.stdout:3/914: mkdir d1/d3/d17/d2b/d44/d57/db0/d142 0 2024-04-23T02:14:12.013 INFO:tasks.workunit.client.1.smithi192.stdout:9/845: creat d0/d9/d44/f10d x:0 0 0 2024-04-23T02:14:12.016 INFO:tasks.workunit.client.1.smithi192.stdout:9/846: write d0/d9/d44/d53/fcc [1324046,74801] 0 2024-04-23T02:14:12.016 INFO:tasks.workunit.client.1.smithi192.stdout:3/915: symlink d1/d3/d6/d7/d71/l143 0 2024-04-23T02:14:12.017 INFO:tasks.workunit.client.0.smithi071.stdout:3/782: creat d7/da5/db2/db7/d68/dae/f117 x:0 0 0 2024-04-23T02:14:12.017 INFO:tasks.workunit.client.1.smithi192.stdout:7/984: rename d5/d17/d1c/d4a/f76 to d5/d17/d1c/d4a/f12f 0 2024-04-23T02:14:12.017 INFO:tasks.workunit.client.1.smithi192.stdout:4/975: read de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/de7/d101/fc4 [809399,33634] 0 2024-04-23T02:14:12.018 INFO:tasks.workunit.client.0.smithi071.stdout:1/757: dread db/df/d13/d2f/f3e [0,4194304] 0 2024-04-23T02:14:12.020 INFO:tasks.workunit.client.0.smithi071.stdout:5/883: getdents d3/d1d/daf/d10/da5 0 2024-04-23T02:14:12.022 INFO:tasks.workunit.client.0.smithi071.stdout:1/758: symlink db/df/d13/d20/de5/l107 0 2024-04-23T02:14:12.023 INFO:tasks.workunit.client.1.smithi192.stdout:9/847: symlink d0/d9/d44/d51/l10e 0 2024-04-23T02:14:12.025 INFO:tasks.workunit.client.0.smithi071.stdout:5/884: link d3/cc8 d3/d1d/db9/c145 0 2024-04-23T02:14:12.026 INFO:tasks.workunit.client.0.smithi071.stdout:3/783: dread d7/da5/db2/db7/d68/d3b/d4c/d74/f92 [0,4194304] 0 2024-04-23T02:14:12.027 INFO:tasks.workunit.client.1.smithi192.stdout:4/976: rename de/d15/d53/d81/de3/d74/d92/da0/dca/l11f to de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/l148 0 2024-04-23T02:14:12.029 INFO:tasks.workunit.client.0.smithi071.stdout:9/770: write d1/d32/d3a/d94/f9b [1683686,108812] 0 2024-04-23T02:14:12.031 INFO:tasks.workunit.client.0.smithi071.stdout:1/759: dwrite db/d34/d8a/fe4 [0,4194304] 0 2024-04-23T02:14:12.031 INFO:tasks.workunit.client.1.smithi192.stdout:7/985: link d5/d17/d1c/d59/d5a/fa2 d5/d17/d1c/d59/d5a/d41/f130 0 2024-04-23T02:14:12.032 INFO:tasks.workunit.client.1.smithi192.stdout:4/977: write de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/ff1 [156867,86374] 0 2024-04-23T02:14:12.033 INFO:tasks.workunit.client.0.smithi071.stdout:5/885: chown d3/da/d1a/d6d/c115 31 1 2024-04-23T02:14:12.033 INFO:tasks.workunit.client.1.smithi192.stdout:9/848: link d0/d9/db3/c83 d0/d9/dc7/da0/dd5/d109/c10f 0 2024-04-23T02:14:12.035 INFO:tasks.workunit.client.0.smithi071.stdout:1/760: readlink db/df/d13/d20/dc0/ld7 0 2024-04-23T02:14:12.040 INFO:tasks.workunit.client.0.smithi071.stdout:5/886: mknod d3/d1d/c146 0 2024-04-23T02:14:12.041 INFO:tasks.workunit.client.1.smithi192.stdout:4/978: rmdir de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/d11b 39 2024-04-23T02:14:12.044 INFO:tasks.workunit.client.1.smithi192.stdout:7/986: dwrite d5/d17/d1c/d59/d5a/d70/de9/da6/da9/db9/dc0/fdb [0,4194304] 0 2024-04-23T02:14:12.046 INFO:tasks.workunit.client.1.smithi192.stdout:7/987: read - d5/d43/d4c/d5c/ff1 zero size 2024-04-23T02:14:12.048 INFO:tasks.workunit.client.0.smithi071.stdout:3/784: dwrite d7/da5/db2/db7/f116 [0,4194304] 0 2024-04-23T02:14:12.048 INFO:tasks.workunit.client.0.smithi071.stdout:1/761: dread db/d11/d7f/da7/dd9/d100/f77 [0,4194304] 0 2024-04-23T02:14:12.048 INFO:tasks.workunit.client.0.smithi071.stdout:9/771: dwrite d1/d32/d41/fd2 [0,4194304] 0 2024-04-23T02:14:12.049 INFO:tasks.workunit.client.0.smithi071.stdout:5/887: creat d3/d8f/de6/d143/dff/f147 x:0 0 0 2024-04-23T02:14:12.051 INFO:tasks.workunit.client.1.smithi192.stdout:4/979: rename de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/cc9 to de/d15/d53/d81/dac/c149 0 2024-04-23T02:14:12.052 INFO:tasks.workunit.client.1.smithi192.stdout:7/988: mknod d5/d43/d4c/d4e/c131 0 2024-04-23T02:14:12.054 INFO:tasks.workunit.client.1.smithi192.stdout:4/980: mkdir de/d15/d53/d81/de3/dfa/d14a 0 2024-04-23T02:14:12.054 INFO:tasks.workunit.client.0.smithi071.stdout:3/785: mknod d7/d8/d42/c118 0 2024-04-23T02:14:12.057 INFO:tasks.workunit.client.1.smithi192.stdout:1/777: sync 2024-04-23T02:14:12.058 INFO:tasks.workunit.client.0.smithi071.stdout:3/786: rmdir d7/d1d 39 2024-04-23T02:14:12.059 INFO:tasks.workunit.client.1.smithi192.stdout:3/916: dread d1/d3/d17/d1f/d33/d24/f45 [0,4194304] 0 2024-04-23T02:14:12.061 INFO:tasks.workunit.client.1.smithi192.stdout:1/778: mkdir dc/d16/dde/de9/dfb 0 2024-04-23T02:14:12.062 INFO:tasks.workunit.client.0.smithi071.stdout:5/888: link d3/d1d/daf/d10/da5/f13c d3/da/d1a/d1c/db6/f148 0 2024-04-23T02:14:12.063 INFO:tasks.workunit.client.0.smithi071.stdout:1/762: dwrite db/df/d13/d20/f5c [0,4194304] 0 2024-04-23T02:14:12.065 INFO:tasks.workunit.client.0.smithi071.stdout:1/763: read - db/df/d5f/d6f/fb7 zero size 2024-04-23T02:14:12.065 INFO:tasks.workunit.client.0.smithi071.stdout:2/691: sync 2024-04-23T02:14:12.065 INFO:tasks.workunit.client.1.smithi192.stdout:3/917: creat d1/d3/d17/d1f/d33/d24/f144 x:0 0 0 2024-04-23T02:14:12.069 INFO:tasks.workunit.client.1.smithi192.stdout:3/918: chown d1/d3/d17/d65/c87 1087245366 1 2024-04-23T02:14:12.069 INFO:tasks.workunit.client.0.smithi071.stdout:5/889: rename d3/da/d1a/db8/d140/d4b/dce/l130 to d3/da/d1a/l149 0 2024-04-23T02:14:12.069 INFO:tasks.workunit.client.0.smithi071.stdout:3/787: truncate d7/da5/db2/db7/d68/d3b/d4c/d8b/db3/f7a 1660979 0 2024-04-23T02:14:12.069 INFO:tasks.workunit.client.0.smithi071.stdout:2/692: chown d0/d2/d26/d3a/dc3/d36/d6a/f6e 14173 1 2024-04-23T02:14:12.071 INFO:tasks.workunit.client.0.smithi071.stdout:5/890: mknod d3/da/d1a/d1c/d46/d4e/dd8/c14a 0 2024-04-23T02:14:12.071 INFO:tasks.workunit.client.0.smithi071.stdout:3/788: fsync d7/da5/db2/db7/d68/d3b/d4c/f59 0 2024-04-23T02:14:12.072 INFO:tasks.workunit.client.0.smithi071.stdout:5/891: truncate d3/da/f129 584664 0 2024-04-23T02:14:12.073 INFO:tasks.workunit.client.0.smithi071.stdout:5/892: chown d3/d1d 0 1 2024-04-23T02:14:12.075 INFO:tasks.workunit.client.0.smithi071.stdout:3/789: creat d7/da5/db2/db7/d68/f119 x:0 0 0 2024-04-23T02:14:12.075 INFO:tasks.workunit.client.1.smithi192.stdout:3/919: mkdir d1/d3/d6/d7/d29/d145 0 2024-04-23T02:14:12.076 INFO:tasks.workunit.client.1.smithi192.stdout:9/849: truncate d0/d9/d44/d51/d5b/d61/db1/f8d 549592 0 2024-04-23T02:14:12.078 INFO:tasks.workunit.client.0.smithi071.stdout:3/790: dread d7/da5/db2/db7/d68/d3b/d4c/f65 [0,4194304] 0 2024-04-23T02:14:12.079 INFO:tasks.workunit.client.0.smithi071.stdout:5/893: link d3/da/d1a/db8/d140/d4b/fa0 d3/da/d1a/db8/d106/f14b 0 2024-04-23T02:14:12.079 INFO:tasks.workunit.client.1.smithi192.stdout:3/920: mkdir d1/d3/d17/d65/d146 0 2024-04-23T02:14:12.084 INFO:tasks.workunit.client.0.smithi071.stdout:3/791: rename d7/da5/db5/d105 to d7/da5/db2/db7/d68/d3b/d98/d11a 0 2024-04-23T02:14:12.085 INFO:tasks.workunit.client.1.smithi192.stdout:9/850: dwrite d0/d9/d44/d51/d5b/d61/f7b [4194304,4194304] 0 2024-04-23T02:14:12.086 INFO:tasks.workunit.client.0.smithi071.stdout:5/894: unlink d3/d34/d6b/de4/c11b 0 2024-04-23T02:14:12.088 INFO:tasks.workunit.client.0.smithi071.stdout:9/772: dwrite d1/d32/d3a/d94/f9c [0,4194304] 0 2024-04-23T02:14:12.092 INFO:tasks.workunit.client.1.smithi192.stdout:9/851: readlink d0/l73 0 2024-04-23T02:14:12.093 INFO:tasks.workunit.client.0.smithi071.stdout:9/773: rename d1/d4e/d54/fea to d1/d32/dc5/dd1/f113 0 2024-04-23T02:14:12.096 INFO:tasks.workunit.client.1.smithi192.stdout:9/852: mknod d0/d9/d44/d53/c110 0 2024-04-23T02:14:12.096 INFO:tasks.workunit.client.1.smithi192.stdout:9/853: fdatasync d0/d9/d44/d51/ff5 0 2024-04-23T02:14:12.097 INFO:tasks.workunit.client.1.smithi192.stdout:9/854: readlink d0/l92 0 2024-04-23T02:14:12.097 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:11 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:14:12.097 INFO:tasks.workunit.client.0.smithi071.stdout:9/774: write d1/f37 [2408253,31952] 0 2024-04-23T02:14:12.099 INFO:tasks.workunit.client.1.smithi192.stdout:3/921: dread d1/d3/d17/d1a/d7c/dab/f54 [0,4194304] 0 2024-04-23T02:14:12.100 INFO:tasks.workunit.client.0.smithi071.stdout:9/775: mkdir d1/d32/d7e/d114 0 2024-04-23T02:14:12.102 INFO:tasks.workunit.client.1.smithi192.stdout:1/779: dread dc/d14/d23/d5e/dab/fd3 [0,4194304] 0 2024-04-23T02:14:12.106 INFO:tasks.workunit.client.1.smithi192.stdout:9/855: dwrite d0/f4e [0,4194304] 0 2024-04-23T02:14:12.107 INFO:tasks.workunit.client.1.smithi192.stdout:1/780: rmdir dc/d8b/d34/d70/d8c/da3/de4 39 2024-04-23T02:14:12.107 INFO:tasks.workunit.client.1.smithi192.stdout:1/781: write f3 [6522083,73582] 0 2024-04-23T02:14:12.108 INFO:tasks.workunit.client.1.smithi192.stdout:1/782: readlink dc/lb2 0 2024-04-23T02:14:12.109 INFO:tasks.workunit.client.0.smithi071.stdout:2/693: dread d0/d2/d26/d14/f23 [4194304,4194304] 0 2024-04-23T02:14:12.112 INFO:tasks.workunit.client.0.smithi071.stdout:3/792: write d7/d22/da7/f9b [1178279,79443] 0 2024-04-23T02:14:12.114 INFO:tasks.workunit.client.0.smithi071.stdout:5/895: write d3/d34/f6a [773863,29340] 0 2024-04-23T02:14:12.117 INFO:tasks.workunit.client.1.smithi192.stdout:3/922: getdents d1/d3/d17/d1f/d33/d24 0 2024-04-23T02:14:12.118 INFO:tasks.workunit.client.0.smithi071.stdout:5/896: chown d3/da/d1a/db8/d140/d4b 0 1 2024-04-23T02:14:12.118 INFO:tasks.workunit.client.1.smithi192.stdout:1/783: dwrite dc/d16/db0/fd8 [4194304,4194304] 0 2024-04-23T02:14:12.119 INFO:tasks.workunit.client.0.smithi071.stdout:3/793: fsync d7/da5/db2/db7/d68/d3b/d4c/f63 0 2024-04-23T02:14:12.121 INFO:tasks.workunit.client.0.smithi071.stdout:5/897: dread - d3/d1d/daf/d83/ffb zero size 2024-04-23T02:14:12.121 INFO:tasks.workunit.client.0.smithi071.stdout:9/776: dwrite d1/d32/d3a/d94/fbb [0,4194304] 0 2024-04-23T02:14:12.123 INFO:tasks.workunit.client.0.smithi071.stdout:3/794: write d7/da5/d100/f111 [815222,80559] 0 2024-04-23T02:14:12.123 INFO:tasks.workunit.client.0.smithi071.stdout:5/898: chown d3/d1d/c146 8824 1 2024-04-23T02:14:12.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:11 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:14:12.129 INFO:tasks.workunit.client.0.smithi071.stdout:3/795: rmdir d7/d8/dea/df6 39 2024-04-23T02:14:12.130 INFO:tasks.workunit.client.1.smithi192.stdout:9/856: truncate d0/d9/f37 4569485 0 2024-04-23T02:14:12.131 INFO:tasks.workunit.client.0.smithi071.stdout:9/777: creat d1/d32/d41/db7/d5a/dc6/d9f/ded/f115 x:0 0 0 2024-04-23T02:14:12.138 INFO:tasks.workunit.client.0.smithi071.stdout:9/778: creat d1/d32/d3a/d90/d99/f116 x:0 0 0 2024-04-23T02:14:12.138 INFO:tasks.workunit.client.1.smithi192.stdout:3/923: write d1/d3/d17/d2b/d44/d57/f78 [257274,4576] 0 2024-04-23T02:14:12.140 INFO:tasks.workunit.client.1.smithi192.stdout:9/857: write d0/d9/db3/f90 [619424,4824] 0 2024-04-23T02:14:12.140 INFO:tasks.workunit.client.0.smithi071.stdout:3/796: unlink d7/da5/db2/db7/d68/d3b/d4c/d8b/db3/c7d 0 2024-04-23T02:14:12.141 INFO:tasks.workunit.client.0.smithi071.stdout:9/779: read d1/d32/d41/db7/d71/f9e [87873,89939] 0 2024-04-23T02:14:12.141 INFO:tasks.workunit.client.0.smithi071.stdout:2/694: dread d0/d2/d26/d3a/dc3/d4f/f66 [0,4194304] 0 2024-04-23T02:14:12.143 INFO:tasks.workunit.client.1.smithi192.stdout:9/858: rename d0/d9/d44/d58/fe4 to d0/d98/f111 0 2024-04-23T02:14:12.146 INFO:tasks.workunit.client.0.smithi071.stdout:9/780: mknod d1/d32/d41/c117 0 2024-04-23T02:14:12.146 INFO:tasks.workunit.client.0.smithi071.stdout:5/899: dwrite d3/da/d1a/d1c/d46/d4e/f70 [0,4194304] 0 2024-04-23T02:14:12.146 INFO:tasks.workunit.client.1.smithi192.stdout:3/924: rename d1/d3/d17/d1a/d7c/ld9 to d1/d3/d17/d1a/d7c/dab/l147 0 2024-04-23T02:14:12.148 INFO:tasks.workunit.client.0.smithi071.stdout:3/797: rename d7/d22/da7/d10c to d7/d8/d42/d11b 0 2024-04-23T02:14:12.148 INFO:tasks.workunit.client.0.smithi071.stdout:9/781: write d1/d32/d3a/d94/fbb [2158180,73091] 0 2024-04-23T02:14:12.152 INFO:tasks.workunit.client.0.smithi071.stdout:3/798: creat d7/d22/d23/d26/d52/f11c x:0 0 0 2024-04-23T02:14:12.152 INFO:tasks.workunit.client.0.smithi071.stdout:9/782: dread - d1/d32/d41/db7/d39/d42/dcc/fcd zero size 2024-04-23T02:14:12.153 INFO:tasks.workunit.client.0.smithi071.stdout:2/695: read d0/d2/d26/d3a/dc3/d4f/f54 [187954,57223] 0 2024-04-23T02:14:12.154 INFO:tasks.workunit.client.1.smithi192.stdout:3/925: dwrite d1/d3/d17/d1a/d7c/dab/d6c/d84/d46/fbc [4194304,4194304] 0 2024-04-23T02:14:12.154 INFO:tasks.workunit.client.0.smithi071.stdout:9/783: chown d1/d32/d3a/d94/f9b 28 1 2024-04-23T02:14:12.156 INFO:tasks.workunit.client.0.smithi071.stdout:2/696: truncate d0/d2/d45/d7d/fd0 361428 0 2024-04-23T02:14:12.156 INFO:tasks.workunit.client.0.smithi071.stdout:9/784: truncate d1/d32/f4a 1877285 0 2024-04-23T02:14:12.159 INFO:tasks.workunit.client.0.smithi071.stdout:9/785: creat d1/d32/d41/db7/d71/f118 x:0 0 0 2024-04-23T02:14:12.160 INFO:tasks.workunit.client.0.smithi071.stdout:9/786: readlink d1/d32/d7e/ld0 0 2024-04-23T02:14:12.164 INFO:tasks.workunit.client.1.smithi192.stdout:5/941: sync 2024-04-23T02:14:12.166 INFO:tasks.workunit.client.0.smithi071.stdout:9/787: mknod d1/d32/d41/db7/d71/dfb/c119 0 2024-04-23T02:14:12.166 INFO:tasks.workunit.client.1.smithi192.stdout:5/942: write d2/d32/f6e [4111563,122618] 0 2024-04-23T02:14:12.168 INFO:tasks.workunit.client.0.smithi071.stdout:9/788: fsync d1/d32/d3a/d90/fec 0 2024-04-23T02:14:12.171 INFO:tasks.workunit.client.1.smithi192.stdout:9/859: dread d0/d9/d44/d4f/f8e [0,4194304] 0 2024-04-23T02:14:12.171 INFO:tasks.workunit.client.0.smithi071.stdout:9/789: chown d1/d32/d41/db7/d39/d42/f49 770504156 1 2024-04-23T02:14:12.171 INFO:tasks.workunit.client.0.smithi071.stdout:3/799: dwrite d7/da5/db2/db7/d68/d3b/d4c/f63 [0,4194304] 0 2024-04-23T02:14:12.173 INFO:tasks.workunit.client.1.smithi192.stdout:5/943: mkdir d2/db/d2b/dc8/d103/d136 0 2024-04-23T02:14:12.173 INFO:tasks.workunit.client.1.smithi192.stdout:3/926: dwrite d1/d3/d17/fdc [0,4194304] 0 2024-04-23T02:14:12.173 INFO:tasks.workunit.client.0.smithi071.stdout:9/790: fsync d1/d32/dc5/dd1/f113 0 2024-04-23T02:14:12.174 INFO:tasks.workunit.client.1.smithi192.stdout:3/927: dread - d1/d3/d6/d7/f12e zero size 2024-04-23T02:14:12.176 INFO:tasks.workunit.client.1.smithi192.stdout:3/928: chown d1/d3/d6/d7/d5f/cee 229748 1 2024-04-23T02:14:12.180 INFO:tasks.workunit.client.0.smithi071.stdout:9/791: readlink d1/d32/d41/db7/d5a/lf7 0 2024-04-23T02:14:12.180 INFO:tasks.workunit.client.0.smithi071.stdout:3/800: read d7/da5/db2/fe6 [980352,55483] 0 2024-04-23T02:14:12.180 INFO:tasks.workunit.client.1.smithi192.stdout:3/929: dread d1/f56 [0,4194304] 0 2024-04-23T02:14:12.180 INFO:tasks.workunit.client.1.smithi192.stdout:9/860: dread d0/d9/d44/d58/d89/dd8/f9c [0,4194304] 0 2024-04-23T02:14:12.182 INFO:tasks.workunit.client.0.smithi071.stdout:3/801: creat d7/d10/d38/f11d x:0 0 0 2024-04-23T02:14:12.183 INFO:tasks.workunit.client.1.smithi192.stdout:9/861: mknod d0/c112 0 2024-04-23T02:14:12.185 INFO:tasks.workunit.client.1.smithi192.stdout:9/862: mknod d0/d9/d44/d51/c113 0 2024-04-23T02:14:12.186 INFO:tasks.workunit.client.0.smithi071.stdout:3/802: mkdir d7/d22/d11e 0 2024-04-23T02:14:12.187 INFO:tasks.workunit.client.1.smithi192.stdout:9/863: mkdir d0/d9/d44/d4f/d114 0 2024-04-23T02:14:12.194 INFO:tasks.workunit.client.1.smithi192.stdout:9/864: dwrite d0/d9/d44/d51/d5b/f67 [0,4194304] 0 2024-04-23T02:14:12.198 INFO:tasks.workunit.client.0.smithi071.stdout:9/792: dwrite d1/d32/d41/db7/d39/d42/f108 [0,4194304] 0 2024-04-23T02:14:12.198 INFO:tasks.workunit.client.0.smithi071.stdout:2/697: dread d0/d2/d34/d55/f72 [0,4194304] 0 2024-04-23T02:14:12.202 INFO:tasks.workunit.client.1.smithi192.stdout:9/865: dwrite d0/d9/db3/d5f/fc8 [0,4194304] 0 2024-04-23T02:14:12.202 INFO:tasks.workunit.client.0.smithi071.stdout:2/698: readlink d0/d2/d34/d55/d96/la9 0 2024-04-23T02:14:12.202 INFO:tasks.workunit.client.0.smithi071.stdout:2/699: stat d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/ld5 0 2024-04-23T02:14:12.203 INFO:tasks.workunit.client.1.smithi192.stdout:3/930: dread d1/d3/f7a [0,4194304] 0 2024-04-23T02:14:12.206 INFO:tasks.workunit.client.1.smithi192.stdout:9/866: mkdir d0/d9/db3/d5f/d115 0 2024-04-23T02:14:12.209 INFO:tasks.workunit.client.1.smithi192.stdout:3/931: getdents d1/d3/d6/d7/d29/d3b/da9/daf/ded 0 2024-04-23T02:14:12.211 INFO:tasks.workunit.client.1.smithi192.stdout:5/944: dread d2/db/d16/d1b/f90 [0,4194304] 0 2024-04-23T02:14:12.216 INFO:tasks.workunit.client.1.smithi192.stdout:5/945: getdents d2/db/d2b 0 2024-04-23T02:14:12.219 INFO:tasks.workunit.client.1.smithi192.stdout:5/946: link d2/db/d2b/d52/dc2/d49/caf d2/db/d2b/d52/dc2/d49/c137 0 2024-04-23T02:14:12.220 INFO:tasks.workunit.client.1.smithi192.stdout:5/947: rmdir d2/d2e/d4e/de6 39 2024-04-23T02:14:12.222 INFO:tasks.workunit.client.1.smithi192.stdout:5/948: chown d2/f27 533643 1 2024-04-23T02:14:12.223 INFO:tasks.workunit.client.1.smithi192.stdout:5/949: chown d2/d6/d2d/d41/d43/c12e 127810 1 2024-04-23T02:14:12.223 INFO:tasks.workunit.client.1.smithi192.stdout:5/950: chown d2/ccf 478172 1 2024-04-23T02:14:12.225 INFO:tasks.workunit.client.1.smithi192.stdout:5/951: creat d2/db/de1/f138 x:0 0 0 2024-04-23T02:14:12.227 INFO:tasks.workunit.client.1.smithi192.stdout:5/952: unlink d2/d32/fc9 0 2024-04-23T02:14:12.227 INFO:tasks.workunit.client.1.smithi192.stdout:5/953: write d2/d32/f9c [1034814,55929] 0 2024-04-23T02:14:12.231 INFO:tasks.workunit.client.1.smithi192.stdout:5/954: rename d2/db/d16/d48/db7/d135 to d2/d2e/d139 0 2024-04-23T02:14:12.236 INFO:tasks.workunit.client.1.smithi192.stdout:3/932: truncate d1/d3/d6/d7/d71/f11b 637442 0 2024-04-23T02:14:12.238 INFO:tasks.workunit.client.1.smithi192.stdout:3/933: stat d1/d3/d17/d1f/c42 0 2024-04-23T02:14:12.240 INFO:tasks.workunit.client.1.smithi192.stdout:9/867: dwrite d0/d9/dc7/da0/fc3 [0,4194304] 0 2024-04-23T02:14:12.241 INFO:tasks.workunit.client.1.smithi192.stdout:9/868: chown d0/d9/d44/d51/c113 69124 1 2024-04-23T02:14:12.241 INFO:tasks.workunit.client.1.smithi192.stdout:3/934: symlink d1/d3/d6/d7/l148 0 2024-04-23T02:14:12.243 INFO:tasks.workunit.client.1.smithi192.stdout:9/869: creat d0/d9/d44/d58/f116 x:0 0 0 2024-04-23T02:14:12.245 INFO:tasks.workunit.client.1.smithi192.stdout:9/870: creat d0/d9/dc7/da0/dd5/d109/f117 x:0 0 0 2024-04-23T02:14:12.249 INFO:tasks.workunit.client.1.smithi192.stdout:9/871: rename d0/dd2/cf7 to d0/d9/d44/d58/d89/c118 0 2024-04-23T02:14:12.251 INFO:tasks.workunit.client.1.smithi192.stdout:9/872: creat d0/d9/db3/d9e/f119 x:0 0 0 2024-04-23T02:14:12.253 INFO:tasks.workunit.client.1.smithi192.stdout:9/873: creat d0/d9/d44/d58/d89/dd8/f11a x:0 0 0 2024-04-23T02:14:12.256 INFO:tasks.workunit.client.1.smithi192.stdout:3/935: dread d1/d3/d17/f3d [0,4194304] 0 2024-04-23T02:14:12.257 INFO:tasks.workunit.client.1.smithi192.stdout:3/936: readlink d1/d3/d6/d7/d29/d3b/da9/daf/l13e 0 2024-04-23T02:14:12.265 INFO:tasks.workunit.client.1.smithi192.stdout:9/874: rmdir d0/d9/d44 39 2024-04-23T02:14:12.266 INFO:tasks.workunit.client.1.smithi192.stdout:9/875: chown d0/d9/db3/d9e/lcb 2136574468 1 2024-04-23T02:14:12.266 INFO:tasks.workunit.client.1.smithi192.stdout:3/937: truncate d1/d3/d17/d2b/d44/f100 3892092 0 2024-04-23T02:14:12.268 INFO:tasks.workunit.client.1.smithi192.stdout:3/938: mknod d1/d3/d17/d1f/c149 0 2024-04-23T02:14:12.274 INFO:tasks.workunit.client.1.smithi192.stdout:3/939: link d1/d3/d17/d1f/c42 d1/d3/d6/d7/d29/d3b/da9/d7b/c14a 0 2024-04-23T02:14:12.278 INFO:tasks.workunit.client.0.smithi071.stdout:4/837: sync 2024-04-23T02:14:12.278 INFO:tasks.workunit.client.0.smithi071.stdout:8/841: sync 2024-04-23T02:14:12.278 INFO:tasks.workunit.client.0.smithi071.stdout:7/990: sync 2024-04-23T02:14:12.279 INFO:tasks.workunit.client.0.smithi071.stdout:6/759: sync 2024-04-23T02:14:12.279 INFO:tasks.workunit.client.1.smithi192.stdout:9/876: dwrite d0/d9/d44/f5e [0,4194304] 0 2024-04-23T02:14:12.281 INFO:tasks.workunit.client.1.smithi192.stdout:9/877: chown d0/d9/d44/d51/d5b/dd3/ff0 125 1 2024-04-23T02:14:12.281 INFO:tasks.workunit.client.1.smithi192.stdout:3/940: link d1/d3/d17/d65/l7e d1/d3/d17/d1a/d7c/dab/d6c/d84/d46/l14b 0 2024-04-23T02:14:12.284 INFO:tasks.workunit.client.1.smithi192.stdout:9/878: readlink d0/d9/d44/d4f/l6 0 2024-04-23T02:14:12.286 INFO:tasks.workunit.client.1.smithi192.stdout:3/941: rename d1/d3/d17/d1a/d7c/dab/d6c/d84/d5d/c6a to d1/d3/c14c 0 2024-04-23T02:14:12.290 INFO:tasks.workunit.client.0.smithi071.stdout:8/842: dread dc/d15/da0/d55/f6c [0,4194304] 0 2024-04-23T02:14:12.297 INFO:tasks.workunit.client.1.smithi192.stdout:3/942: link d1/d3/c14c d1/d3/d6/c14d 0 2024-04-23T02:14:12.299 INFO:tasks.workunit.client.0.smithi071.stdout:2/700: mknod d0/d2/d45/ce4 0 2024-04-23T02:14:12.301 INFO:tasks.workunit.client.0.smithi071.stdout:9/793: rename d1/d4e/d54/f112 to d1/db8/f11a 0 2024-04-23T02:14:12.303 INFO:tasks.workunit.client.1.smithi192.stdout:9/879: dread d0/d9/fb [0,4194304] 0 2024-04-23T02:14:12.303 INFO:tasks.workunit.client.0.smithi071.stdout:4/838: mkdir d2/d2a/d2d/d39/d42/d49/d91/ddb/d10d 0 2024-04-23T02:14:12.305 INFO:tasks.workunit.client.0.smithi071.stdout:3/803: creat d7/da5/db2/db7/d68/d3b/d4c/f11f x:0 0 0 2024-04-23T02:14:12.305 INFO:tasks.workunit.client.1.smithi192.stdout:3/943: dwrite d1/d3/d6/d7/d29/d3b/fdd [0,4194304] 0 2024-04-23T02:14:12.306 INFO:tasks.workunit.client.0.smithi071.stdout:9/794: dread - d1/d32/d41/db7/d5a/dc6/d81/db6/de4/fb1 zero size 2024-04-23T02:14:12.307 INFO:tasks.workunit.client.1.smithi192.stdout:3/944: chown d1/d3/d6/d7/d29/d3b/da9/daf/f138 0 1 2024-04-23T02:14:12.307 INFO:tasks.workunit.client.0.smithi071.stdout:6/760: creat d6/d17/d77/d9f/ff0 x:0 0 0 2024-04-23T02:14:12.312 INFO:tasks.workunit.client.1.smithi192.stdout:3/945: creat d1/d3/d6/d7/d29/d3b/da9/d7b/d123/f14e x:0 0 0 2024-04-23T02:14:12.312 INFO:tasks.workunit.client.0.smithi071.stdout:9/795: creat d1/d32/dc5/f11b x:0 0 0 2024-04-23T02:14:12.313 INFO:tasks.workunit.client.1.smithi192.stdout:9/880: getdents d0/d9/db3/d9e 0 2024-04-23T02:14:12.314 INFO:tasks.workunit.client.0.smithi071.stdout:2/701: link d0/d2/d45/d7d/d56/db0/f3b d0/d2/d45/d7d/d56/db0/fe5 0 2024-04-23T02:14:12.314 INFO:tasks.workunit.client.1.smithi192.stdout:9/881: chown d0/f4e 176 1 2024-04-23T02:14:12.316 INFO:tasks.workunit.client.1.smithi192.stdout:9/882: write d0/d9/d44/d74/f82 [5622312,93220] 0 2024-04-23T02:14:12.316 INFO:tasks.workunit.client.1.smithi192.stdout:3/946: symlink d1/d3/d6/d7/l14f 0 2024-04-23T02:14:12.316 INFO:tasks.workunit.client.0.smithi071.stdout:3/804: creat d7/d10/d38/daa/f120 x:0 0 0 2024-04-23T02:14:12.318 INFO:tasks.workunit.client.0.smithi071.stdout:4/839: link d2/d2a/da2/dd1/dde/d23/ff d2/f10e 0 2024-04-23T02:14:12.318 INFO:tasks.workunit.client.1.smithi192.stdout:9/883: creat d0/d9/d44/d74/de7/f11b x:0 0 0 2024-04-23T02:14:12.319 INFO:tasks.workunit.client.0.smithi071.stdout:6/761: symlink d6/lf1 0 2024-04-23T02:14:12.319 INFO:tasks.workunit.client.0.smithi071.stdout:9/796: creat d1/d32/d41/db7/d5a/dc6/d9f/ded/f11c x:0 0 0 2024-04-23T02:14:12.319 INFO:tasks.workunit.client.1.smithi192.stdout:9/884: stat d0/d9/d44/d51/d5b/dd3/le3 0 2024-04-23T02:14:12.320 INFO:tasks.workunit.client.0.smithi071.stdout:2/702: rmdir d0/d2/d26/d3a/dc3/d36/d5c 39 2024-04-23T02:14:12.321 INFO:tasks.workunit.client.0.smithi071.stdout:9/797: write d1/d32/d3a/d94/ff5 [3318175,32095] 0 2024-04-23T02:14:12.321 INFO:tasks.workunit.client.1.smithi192.stdout:9/885: fdatasync d0/d9/d44/d4f/f3e 0 2024-04-23T02:14:12.322 INFO:tasks.workunit.client.0.smithi071.stdout:3/805: mkdir d7/d10/d38/d121 0 2024-04-23T02:14:12.322 INFO:tasks.workunit.client.1.smithi192.stdout:9/886: chown d0/d9/db3/d9e/dff 16987009 1 2024-04-23T02:14:12.324 INFO:tasks.workunit.client.0.smithi071.stdout:6/762: truncate d6/d22/f28 2376531 0 2024-04-23T02:14:12.324 INFO:tasks.workunit.client.1.smithi192.stdout:9/887: fdatasync d0/f102 0 2024-04-23T02:14:12.325 INFO:tasks.workunit.client.0.smithi071.stdout:2/703: mknod d0/d2/d34/dbf/ce6 0 2024-04-23T02:14:12.325 INFO:tasks.workunit.client.1.smithi192.stdout:9/888: chown d0/d9/dc7/d106/c108 1349 1 2024-04-23T02:14:12.326 INFO:tasks.workunit.client.0.smithi071.stdout:6/763: fsync d6/d17/d77/d9f/dad/d3f/fca 0 2024-04-23T02:14:12.328 INFO:tasks.workunit.client.0.smithi071.stdout:3/806: mknod d7/d22/c122 0 2024-04-23T02:14:12.329 INFO:tasks.workunit.client.0.smithi071.stdout:3/807: write d7/fd [376421,83601] 0 2024-04-23T02:14:12.330 INFO:tasks.workunit.client.1.smithi192.stdout:9/889: rename d0/d9/db3/d9e/cfb to d0/d9/d44/d51/c11c 0 2024-04-23T02:14:12.330 INFO:tasks.workunit.client.0.smithi071.stdout:4/840: creat d2/d2a/d2d/d39/d42/d49/d67/d7e/dc2/f10f x:0 0 0 2024-04-23T02:14:12.332 INFO:tasks.workunit.client.1.smithi192.stdout:9/890: creat d0/d9/d44/d58/dd9/f11d x:0 0 0 2024-04-23T02:14:12.333 INFO:tasks.workunit.client.0.smithi071.stdout:6/764: rename d6/d91 to d6/d17/d77/d9f/dd8/df2 0 2024-04-23T02:14:12.335 INFO:tasks.workunit.client.0.smithi071.stdout:6/765: read - d6/d17/d77/d9f/dad/d3f/fee zero size 2024-04-23T02:14:12.336 INFO:tasks.workunit.client.0.smithi071.stdout:6/766: chown d6/lb7 44826529 1 2024-04-23T02:14:12.336 INFO:tasks.workunit.client.1.smithi192.stdout:9/891: rename d0/d9/d44/d51/la9 to d0/d9/d44/d58/dd9/l11e 0 2024-04-23T02:14:12.337 INFO:tasks.workunit.client.0.smithi071.stdout:2/704: mkdir d0/d2/d26/d3a/dc3/d36/de7 0 2024-04-23T02:14:12.337 INFO:tasks.workunit.client.1.smithi192.stdout:9/892: stat d0/d9/d44/f9d 0 2024-04-23T02:14:12.338 INFO:tasks.workunit.client.1.smithi192.stdout:9/893: fdatasync d0/d9/d44/d4f/f3d 0 2024-04-23T02:14:12.340 INFO:tasks.workunit.client.0.smithi071.stdout:2/705: symlink d0/d2/d45/le8 0 2024-04-23T02:14:12.340 INFO:tasks.workunit.client.0.smithi071.stdout:4/841: mknod d2/d2a/d2d/d39/d42/d49/d67/ded/c110 0 2024-04-23T02:14:12.341 INFO:tasks.workunit.client.1.smithi192.stdout:9/894: creat d0/d9/dc7/f11f x:0 0 0 2024-04-23T02:14:12.341 INFO:tasks.workunit.client.0.smithi071.stdout:3/808: dread d7/d10/d38/f41 [0,4194304] 0 2024-04-23T02:14:12.343 INFO:tasks.workunit.client.1.smithi192.stdout:9/895: unlink d0/d9/d44/d4f/f79 0 2024-04-23T02:14:12.345 INFO:tasks.workunit.client.1.smithi192.stdout:9/896: chown d0/d9/d44/d58/d89/dd8 3750019 1 2024-04-23T02:14:12.345 INFO:tasks.workunit.client.0.smithi071.stdout:3/809: creat d7/d22/d23/dba/df4/f123 x:0 0 0 2024-04-23T02:14:12.346 INFO:tasks.workunit.client.1.smithi192.stdout:9/897: mkdir d0/d9/db3/d5f/dfd/d120 0 2024-04-23T02:14:12.348 INFO:tasks.workunit.client.1.smithi192.stdout:9/898: truncate d0/d9/d44/d58/d89/ff1 219432 0 2024-04-23T02:14:12.349 INFO:tasks.workunit.client.0.smithi071.stdout:3/810: creat d7/d22/d23/dba/df4/dca/f124 x:0 0 0 2024-04-23T02:14:12.349 INFO:tasks.workunit.client.0.smithi071.stdout:4/842: getdents d2/d2a/da2/dd1/dde/d23/d10 0 2024-04-23T02:14:12.350 INFO:tasks.workunit.client.0.smithi071.stdout:6/767: dread d6/d17/d77/d85/fab [0,4194304] 0 2024-04-23T02:14:12.350 INFO:tasks.workunit.client.1.smithi192.stdout:9/899: creat d0/d9/d44/d4f/d114/f121 x:0 0 0 2024-04-23T02:14:12.353 INFO:tasks.workunit.client.1.smithi192.stdout:9/900: rename d0/d9/dc7/da0/db4 to d0/d9/dc7/da0/dd5/d109/d122 0 2024-04-23T02:14:12.353 INFO:tasks.workunit.client.0.smithi071.stdout:3/811: rename d7/da5/db2/db7/d68/d3b/d4c/d74 to d7/d8/dea/df6/d114/d125 0 2024-04-23T02:14:12.355 INFO:tasks.workunit.client.1.smithi192.stdout:9/901: chown d0/d9/dc7/da0/fbe 463 1 2024-04-23T02:14:12.355 INFO:tasks.workunit.client.0.smithi071.stdout:4/843: mkdir d2/d2a/d73/d111 0 2024-04-23T02:14:12.355 INFO:tasks.workunit.client.0.smithi071.stdout:3/812: read d7/da5/db2/fe6 [670128,10493] 0 2024-04-23T02:14:12.356 INFO:tasks.workunit.client.1.smithi192.stdout:9/902: write d0/d9/dc7/fc6 [1123605,62234] 0 2024-04-23T02:14:12.357 INFO:tasks.workunit.client.0.smithi071.stdout:3/813: write d7/d22/d23/d26/d52/dd3/fcc [718399,111778] 0 2024-04-23T02:14:12.358 INFO:tasks.workunit.client.1.smithi192.stdout:9/903: chown d0/d9/d44/d58/dd9/lf6 5990 1 2024-04-23T02:14:12.360 INFO:tasks.workunit.client.1.smithi192.stdout:9/904: mknod d0/d9/d44/d51/d5b/c123 0 2024-04-23T02:14:12.362 INFO:tasks.workunit.client.0.smithi071.stdout:3/814: creat d7/da5/db2/db7/d68/d3b/d4c/da1/f126 x:0 0 0 2024-04-23T02:14:12.362 INFO:tasks.workunit.client.0.smithi071.stdout:7/991: write d3/d17/d41/d3a/d137/dea/d5f/f7e [121358,97945] 0 2024-04-23T02:14:12.362 INFO:tasks.workunit.client.1.smithi192.stdout:9/905: dread d0/d9/d44/d51/d5b/d61/db1/f8d [0,4194304] 0 2024-04-23T02:14:12.363 INFO:tasks.workunit.client.0.smithi071.stdout:4/844: symlink d2/d2a/d73/d5e/d84/l112 0 2024-04-23T02:14:12.364 INFO:tasks.workunit.client.1.smithi192.stdout:9/906: write d0/d9/dc7/da0/fc3 [398276,35016] 0 2024-04-23T02:14:12.364 INFO:tasks.workunit.client.1.smithi192.stdout:9/907: chown d0/d9/d44/f10d 1690518 1 2024-04-23T02:14:12.368 INFO:tasks.workunit.client.0.smithi071.stdout:7/992: creat d3/d17/d1c/f15e x:0 0 0 2024-04-23T02:14:12.369 INFO:tasks.workunit.client.0.smithi071.stdout:6/768: dread d6/d17/d77/d85/fa1 [0,4194304] 0 2024-04-23T02:14:12.369 INFO:tasks.workunit.client.0.smithi071.stdout:3/815: rename d7/d22/d23/l3d to d7/d22/d23/dba/df4/dce/l127 0 2024-04-23T02:14:12.371 INFO:tasks.workunit.client.1.smithi192.stdout:9/908: rmdir d0/d9/dc7 39 2024-04-23T02:14:12.373 INFO:tasks.workunit.client.0.smithi071.stdout:8/843: dwrite dc/d15/da0/d8d/faf [0,4194304] 0 2024-04-23T02:14:12.376 INFO:tasks.workunit.client.1.smithi192.stdout:9/909: dread d0/d9/d44/d58/dd9/fe6 [0,4194304] 0 2024-04-23T02:14:12.377 INFO:tasks.workunit.client.1.smithi192.stdout:9/910: write d0/d9/d44/f10d [213397,116180] 0 2024-04-23T02:14:12.380 INFO:tasks.workunit.client.1.smithi192.stdout:9/911: symlink d0/d9/d44/d51/d5b/l124 0 2024-04-23T02:14:12.382 INFO:tasks.workunit.client.0.smithi071.stdout:9/798: truncate d1/d32/d41/db7/f55 371095 0 2024-04-23T02:14:12.385 INFO:tasks.workunit.client.0.smithi071.stdout:3/816: dwrite d7/da5/db2/db7/d68/d3b/d4c/da1/f126 [0,4194304] 0 2024-04-23T02:14:12.392 INFO:tasks.workunit.client.0.smithi071.stdout:6/769: creat d6/d17/d77/d9f/dad/ff3 x:0 0 0 2024-04-23T02:14:12.393 INFO:tasks.workunit.client.0.smithi071.stdout:4/845: link d2/d2a/da2/dd1/dde/d23/f5d d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/dc8/f113 0 2024-04-23T02:14:12.394 INFO:tasks.workunit.client.0.smithi071.stdout:9/799: unlink d1/d32/d41/db7/d71/f9e 0 2024-04-23T02:14:12.395 INFO:tasks.workunit.client.0.smithi071.stdout:9/800: chown d1/d32/dc5 1 1 2024-04-23T02:14:12.395 INFO:tasks.workunit.client.1.smithi192.stdout:9/912: dwrite d0/d9/db3/d5f/fc8 [4194304,4194304] 0 2024-04-23T02:14:12.396 INFO:tasks.workunit.client.0.smithi071.stdout:8/844: truncate dc/d15/d50/fec 335482 0 2024-04-23T02:14:12.396 INFO:tasks.workunit.client.0.smithi071.stdout:2/706: write d0/d2/d34/f84 [4133045,33581] 0 2024-04-23T02:14:12.400 INFO:tasks.workunit.client.0.smithi071.stdout:4/846: mknod d2/d2a/d2d/d39/d42/d49/d91/ddb/c114 0 2024-04-23T02:14:12.401 INFO:tasks.workunit.client.1.smithi192.stdout:9/913: rename d0/d9/c6f to d0/d9/d44/d51/c125 0 2024-04-23T02:14:12.405 INFO:tasks.workunit.client.0.smithi071.stdout:8/845: write dc/d15/da0/d8d/d9a/dad/ff5 [221046,87284] 0 2024-04-23T02:14:12.405 INFO:tasks.workunit.client.1.smithi192.stdout:9/914: getdents d0/d9/d44/d51/d5b/d61/db1 0 2024-04-23T02:14:12.406 INFO:tasks.workunit.client.0.smithi071.stdout:9/801: mknod d1/d32/d41/db7/d39/d42/c11d 0 2024-04-23T02:14:12.406 INFO:tasks.workunit.client.0.smithi071.stdout:8/846: chown dc/d3a/f47 4565523 1 2024-04-23T02:14:12.408 INFO:tasks.workunit.client.0.smithi071.stdout:2/707: mkdir d0/d2/d26/d3a/dc3/d36/d5c/d85/de9 0 2024-04-23T02:14:12.409 INFO:tasks.workunit.client.0.smithi071.stdout:3/817: rename d7/da5/db2/db7 to d7/da5/d128 0 2024-04-23T02:14:12.410 INFO:tasks.workunit.client.1.smithi192.stdout:9/915: dwrite d0/d9/d44/f45 [4194304,4194304] 0 2024-04-23T02:14:12.412 INFO:tasks.workunit.client.0.smithi071.stdout:3/818: chown d7/d22/d23/dba/df4/dce 3937 1 2024-04-23T02:14:12.412 INFO:tasks.workunit.client.1.smithi192.stdout:9/916: write d0/d9/f5a [878100,13554] 0 2024-04-23T02:14:12.414 INFO:tasks.workunit.client.0.smithi071.stdout:8/847: unlink dc/d15/da0/d55/fc7 0 2024-04-23T02:14:12.415 INFO:tasks.workunit.client.1.smithi192.stdout:9/917: mknod d0/dd2/c126 0 2024-04-23T02:14:12.417 INFO:tasks.workunit.client.0.smithi071.stdout:7/993: write d3/d17/d1c/d2a/d43/f13e [2577714,17267] 0 2024-04-23T02:14:12.417 INFO:tasks.workunit.client.0.smithi071.stdout:2/708: dwrite d0/fd8 [0,4194304] 0 2024-04-23T02:14:12.418 INFO:tasks.workunit.client.0.smithi071.stdout:9/802: unlink d1/d4e/d54/c58 0 2024-04-23T02:14:12.419 INFO:tasks.workunit.client.1.smithi192.stdout:9/918: dread d0/f2c [0,4194304] 0 2024-04-23T02:14:12.421 INFO:tasks.workunit.client.1.smithi192.stdout:9/919: creat d0/d9/db3/d5f/dfd/f127 x:0 0 0 2024-04-23T02:14:12.421 INFO:tasks.workunit.client.0.smithi071.stdout:4/847: creat d2/d2a/d2d/d39/d42/d49/d67/f115 x:0 0 0 2024-04-23T02:14:12.422 INFO:tasks.workunit.client.0.smithi071.stdout:9/803: readlink d1/d32/d41/db7/d5a/dc6/d81/l106 0 2024-04-23T02:14:12.422 INFO:tasks.workunit.client.1.smithi192.stdout:9/920: truncate d0/d9/d44/fee 499861 0 2024-04-23T02:14:12.424 INFO:tasks.workunit.client.1.smithi192.stdout:9/921: dread d0/d9/d44/d4f/fdf [0,4194304] 0 2024-04-23T02:14:12.425 INFO:tasks.workunit.client.1.smithi192.stdout:9/922: chown d0/d9/d44/d4f/f3d 27 1 2024-04-23T02:14:12.450 INFO:tasks.workunit.client.1.smithi192.stdout:7/989: sync 2024-04-23T02:14:12.453 INFO:tasks.workunit.client.1.smithi192.stdout:7/990: truncate d5/d17/d1c/d59/f10c 299865 0 2024-04-23T02:14:12.460 INFO:tasks.workunit.client.1.smithi192.stdout:7/991: dwrite d5/d17/d1c/d59/d5a/d70/de9/da6/da9/db9/dc0/fdb [4194304,4194304] 0 2024-04-23T02:14:12.461 INFO:tasks.workunit.client.1.smithi192.stdout:7/992: chown d5/d17/d1c/d57/dbb/df4 24743 1 2024-04-23T02:14:12.463 INFO:tasks.workunit.client.1.smithi192.stdout:7/993: symlink d5/d17/d1c/d59/d5a/d70/de9/da6/ddf/l132 0 2024-04-23T02:14:12.464 INFO:tasks.workunit.client.1.smithi192.stdout:7/994: chown d5/d43/d4c/d5c/d97/fea 37 1 2024-04-23T02:14:12.468 INFO:tasks.workunit.client.1.smithi192.stdout:7/995: link d5/d17/db5/c122 d5/d17/d1c/d57/d8b/c133 0 2024-04-23T02:14:12.477 INFO:tasks.workunit.client.1.smithi192.stdout:7/996: write d5/d43/d4c/d5c/f12d [650604,97753] 0 2024-04-23T02:14:12.479 INFO:tasks.workunit.client.1.smithi192.stdout:7/997: mknod d5/d17/d1c/d59/d5a/d41/d50/d3d/d10f/c134 0 2024-04-23T02:14:12.480 INFO:tasks.workunit.client.1.smithi192.stdout:7/998: fdatasync d5/d43/d4c/d4e/fc6 0 2024-04-23T02:14:12.487 INFO:tasks.workunit.client.0.smithi071.stdout:3/819: creat d7/da5/d128/d68/d3b/d4c/d8b/db3/f129 x:0 0 0 2024-04-23T02:14:12.488 INFO:tasks.workunit.client.1.smithi192.stdout:7/999: dread d5/d17/d1c/d59/d5a/d41/d50/d3d/f83 [0,4194304] 0 2024-04-23T02:14:12.489 INFO:tasks.workunit.client.0.smithi071.stdout:3/820: read d7/da5/d128/f40 [5525862,51671] 0 2024-04-23T02:14:12.490 INFO:tasks.workunit.client.0.smithi071.stdout:2/709: creat d0/d2/d34/dbf/fea x:0 0 0 2024-04-23T02:14:12.491 INFO:tasks.workunit.client.0.smithi071.stdout:3/821: dread - d7/d22/d23/d26/d7e/fd0 zero size 2024-04-23T02:14:12.492 INFO:tasks.workunit.client.0.smithi071.stdout:0/863: sync 2024-04-23T02:14:12.494 INFO:tasks.workunit.client.0.smithi071.stdout:2/710: dread - d0/d2/d26/d3a/dc3/d36/d5c/d85/fdc zero size 2024-04-23T02:14:12.494 INFO:tasks.workunit.client.0.smithi071.stdout:9/804: mknod d1/d32/d41/db7/d5a/dc6/d81/db6/c11e 0 2024-04-23T02:14:12.496 INFO:tasks.workunit.client.0.smithi071.stdout:0/864: rmdir d0/d3/de/d10/d14/d32 39 2024-04-23T02:14:12.497 INFO:tasks.workunit.client.0.smithi071.stdout:9/805: dread d1/d32/d41/db7/d39/d42/dcc/ff2 [0,4194304] 0 2024-04-23T02:14:12.497 INFO:tasks.workunit.client.0.smithi071.stdout:3/822: truncate d7/d22/d23/d26/d7e/fd9 1128582 0 2024-04-23T02:14:12.498 INFO:tasks.workunit.client.0.smithi071.stdout:2/711: dread - d0/d2/d26/d14/fae zero size 2024-04-23T02:14:12.501 INFO:tasks.workunit.client.0.smithi071.stdout:6/770: rename d6/d17/cdc to d6/d1f/cf4 0 2024-04-23T02:14:12.502 INFO:tasks.workunit.client.0.smithi071.stdout:0/865: stat d0/d3/de/d10/d14/d32/d44/d87/la6 0 2024-04-23T02:14:12.504 INFO:tasks.workunit.client.0.smithi071.stdout:3/823: readlink d7/l77 0 2024-04-23T02:14:12.504 INFO:tasks.workunit.client.0.smithi071.stdout:3/824: chown d7/d22/d23/d26/d7e 1459701 1 2024-04-23T02:14:12.505 INFO:tasks.workunit.client.0.smithi071.stdout:3/825: readlink d7/da5/d128/d68/l95 0 2024-04-23T02:14:12.508 INFO:tasks.workunit.client.0.smithi071.stdout:0/866: creat d0/d3/d95/d114/f130 x:0 0 0 2024-04-23T02:14:12.511 INFO:tasks.workunit.client.0.smithi071.stdout:4/848: write d2/d2a/d2d/d39/f46 [1990580,115841] 0 2024-04-23T02:14:12.515 INFO:tasks.workunit.client.0.smithi071.stdout:7/994: rename d3/d17/d1c/d2a/d73/d83/de7 to d3/d17/d1c/d15f 0 2024-04-23T02:14:12.519 INFO:tasks.workunit.client.0.smithi071.stdout:4/849: mknod d2/d2a/d2d/d39/d42/d49/c116 0 2024-04-23T02:14:12.522 INFO:tasks.workunit.client.0.smithi071.stdout:9/806: rename d1/d32/d41/db7/d39/d42/f107 to d1/d32/d41/db7/d39/f11f 0 2024-04-23T02:14:12.525 INFO:tasks.workunit.client.0.smithi071.stdout:7/995: dwrite d3/d51/d80/f9e [0,4194304] 0 2024-04-23T02:14:12.527 INFO:tasks.workunit.client.0.smithi071.stdout:6/771: getdents d6/d1f/d5d 0 2024-04-23T02:14:12.530 INFO:tasks.workunit.client.0.smithi071.stdout:2/712: dwrite d0/d2/d45/d7d/f53 [0,4194304] 0 2024-04-23T02:14:12.530 INFO:tasks.workunit.client.0.smithi071.stdout:4/850: creat d2/d2a/da2/f117 x:0 0 0 2024-04-23T02:14:12.533 INFO:tasks.workunit.client.0.smithi071.stdout:0/867: getdents d0/d3/de/d10/d14/d32/dac 0 2024-04-23T02:14:12.534 INFO:tasks.workunit.client.0.smithi071.stdout:9/807: readlink d1/d32/d41/db7/d5a/dc6/d81/db6/lc3 0 2024-04-23T02:14:12.535 INFO:tasks.workunit.client.0.smithi071.stdout:3/826: dread d7/da5/d128/d68/f62 [0,4194304] 0 2024-04-23T02:14:12.536 INFO:tasks.workunit.client.0.smithi071.stdout:6/772: mknod d6/d22/d92/cf5 0 2024-04-23T02:14:12.539 INFO:tasks.workunit.client.0.smithi071.stdout:2/713: stat d0/d2/d45/d7d/d56/db0/fe5 0 2024-04-23T02:14:12.550 INFO:tasks.workunit.client.0.smithi071.stdout:2/714: symlink d0/d2/d34/d55/d96/leb 0 2024-04-23T02:14:12.552 INFO:tasks.workunit.client.0.smithi071.stdout:0/868: creat d0/d3/de/d10/db3/f131 x:0 0 0 2024-04-23T02:14:12.553 INFO:tasks.workunit.client.0.smithi071.stdout:7/996: dwrite d3/d17/d41/d3a/da5/fbf [0,4194304] 0 2024-04-23T02:14:12.553 INFO:tasks.workunit.client.0.smithi071.stdout:3/827: getdents d7 0 2024-04-23T02:14:12.553 INFO:tasks.workunit.client.0.smithi071.stdout:9/808: dwrite d1/d32/d3a/d5f/f63 [0,4194304] 0 2024-04-23T02:14:12.553 INFO:tasks.workunit.client.0.smithi071.stdout:7/997: stat d3/d17/d1c/d31/d112 0 2024-04-23T02:14:12.559 INFO:tasks.workunit.client.0.smithi071.stdout:6/773: getdents d6/d1f 0 2024-04-23T02:14:12.559 INFO:tasks.workunit.client.0.smithi071.stdout:4/851: getdents d2/d2a/d2d/d39/d42/d49 0 2024-04-23T02:14:12.559 INFO:tasks.workunit.client.0.smithi071.stdout:6/774: write d6/d1f/d5d/d95/f98 [3827705,68340] 0 2024-04-23T02:14:12.559 INFO:tasks.workunit.client.0.smithi071.stdout:2/715: mknod d0/d2/da0/dc1/cec 0 2024-04-23T02:14:12.561 INFO:tasks.workunit.client.0.smithi071.stdout:3/828: chown d7/da5/db5/dec/cf2 11637828 1 2024-04-23T02:14:12.563 INFO:tasks.workunit.client.0.smithi071.stdout:9/809: creat d1/d32/d41/db7/d71/dbf/f120 x:0 0 0 2024-04-23T02:14:12.563 INFO:tasks.workunit.client.0.smithi071.stdout:2/716: symlink d0/d2/d26/d73/led 0 2024-04-23T02:14:12.564 INFO:tasks.workunit.client.0.smithi071.stdout:4/852: rmdir d2/d2a/da2/dd1/dde/d23/d4f/db5/db6 39 2024-04-23T02:14:12.564 INFO:tasks.workunit.client.0.smithi071.stdout:0/869: mknod d0/d3/de/d10/d14/d32/c132 0 2024-04-23T02:14:12.566 INFO:tasks.workunit.client.0.smithi071.stdout:3/829: unlink d7/d22/ceb 0 2024-04-23T02:14:12.572 INFO:tasks.workunit.client.0.smithi071.stdout:7/998: truncate d3/d17/d41/d34/f135 391471 0 2024-04-23T02:14:12.574 INFO:tasks.workunit.client.0.smithi071.stdout:7/999: readlink d3/d17/d41/d37/la1 0 2024-04-23T02:14:12.578 INFO:tasks.workunit.client.0.smithi071.stdout:6/775: rename d6/d17/d77/d85/f89 to d6/d17/d77/d9f/dad/ff6 0 2024-04-23T02:14:12.581 INFO:tasks.workunit.client.0.smithi071.stdout:6/776: chown d6/d1f/d5d/f71 7121 1 2024-04-23T02:14:12.584 INFO:tasks.workunit.client.0.smithi071.stdout:9/810: write d1/d32/d41/f57 [5227285,98102] 0 2024-04-23T02:14:12.584 INFO:tasks.workunit.client.0.smithi071.stdout:0/870: symlink d0/d3/de/d10/d14/d32/d44/d87/l133 0 2024-04-23T02:14:12.584 INFO:tasks.workunit.client.0.smithi071.stdout:6/777: read - d6/d17/d77/d9f/dad/ff3 zero size 2024-04-23T02:14:12.590 INFO:tasks.workunit.client.0.smithi071.stdout:9/811: rmdir d1/d32/d41/db7/d39/d42 39 2024-04-23T02:14:12.592 INFO:tasks.workunit.client.0.smithi071.stdout:2/717: dwrite d0/d2/d34/d55/d96/fde [0,4194304] 0 2024-04-23T02:14:12.599 INFO:tasks.workunit.client.0.smithi071.stdout:4/853: dwrite d2/d2a/d2d/d7b/ffa [0,4194304] 0 2024-04-23T02:14:12.600 INFO:tasks.workunit.client.0.smithi071.stdout:2/718: fdatasync d0/d2/d26/d3a/dc3/d36/d5c/d85/fdc 0 2024-04-23T02:14:12.600 INFO:tasks.workunit.client.0.smithi071.stdout:3/830: dwrite d7/d22/d23/d26/f2b [4194304,4194304] 0 2024-04-23T02:14:12.601 INFO:tasks.workunit.client.1.smithi192.stdout:4/981: sync 2024-04-23T02:14:12.602 INFO:tasks.workunit.client.1.smithi192.stdout:1/784: sync 2024-04-23T02:14:12.604 INFO:tasks.workunit.client.1.smithi192.stdout:4/982: chown de/d15/d53/d81/de3/f147 291 1 2024-04-23T02:14:12.606 INFO:tasks.workunit.client.1.smithi192.stdout:1/785: creat dc/d16/d19/d74/df4/ffc x:0 0 0 2024-04-23T02:14:12.608 INFO:tasks.workunit.client.1.smithi192.stdout:1/786: dread - dc/d14/fee zero size 2024-04-23T02:14:12.608 INFO:tasks.workunit.client.1.smithi192.stdout:4/983: stat de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/d59/d8a/ca9 0 2024-04-23T02:14:12.609 INFO:tasks.workunit.client.0.smithi071.stdout:3/831: fdatasync d7/d22/d23/f6b 0 2024-04-23T02:14:12.610 INFO:tasks.workunit.client.1.smithi192.stdout:1/787: fdatasync dc/d16/db0/fc7 0 2024-04-23T02:14:12.612 INFO:tasks.workunit.client.1.smithi192.stdout:1/788: write dc/d8b/d34/d70/d8c/da3/dc1/ffa [369132,95316] 0 2024-04-23T02:14:12.613 INFO:tasks.workunit.client.0.smithi071.stdout:4/854: rmdir d2/d2a/d2d/d39/d42/d49/d67/d7e/df4 0 2024-04-23T02:14:12.615 INFO:tasks.workunit.client.1.smithi192.stdout:1/789: stat dc/d8b/d34/d70/d8c/le5 0 2024-04-23T02:14:12.615 INFO:tasks.workunit.client.0.smithi071.stdout:3/832: mkdir d7/da5/d128/d68/d3b/d4c/d8b/d12a 0 2024-04-23T02:14:12.615 INFO:tasks.workunit.client.0.smithi071.stdout:0/871: dwrite d0/f20 [4194304,4194304] 0 2024-04-23T02:14:12.615 INFO:tasks.workunit.client.1.smithi192.stdout:1/790: readlink dc/d8b/d34/l61 0 2024-04-23T02:14:12.617 INFO:tasks.workunit.client.1.smithi192.stdout:1/791: unlink dc/d8b/d34/d70/l71 0 2024-04-23T02:14:12.619 INFO:tasks.workunit.client.1.smithi192.stdout:4/984: dread de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/f88 [0,4194304] 0 2024-04-23T02:14:12.619 INFO:tasks.workunit.client.0.smithi071.stdout:3/833: symlink d7/d10/d38/daa/l12b 0 2024-04-23T02:14:12.621 INFO:tasks.workunit.client.1.smithi192.stdout:4/985: chown de/d15/d19/d1a/d106/d125/c142 456268056 1 2024-04-23T02:14:12.621 INFO:tasks.workunit.client.0.smithi071.stdout:0/872: dread - d0/d3/de/d10/db3/fea zero size 2024-04-23T02:14:12.623 INFO:tasks.workunit.client.0.smithi071.stdout:6/778: dread d6/d1f/f9a [0,4194304] 0 2024-04-23T02:14:12.623 INFO:tasks.workunit.client.1.smithi192.stdout:1/792: getdents dc/d8b/d34/d70/d8c/da3 0 2024-04-23T02:14:12.624 INFO:tasks.workunit.client.0.smithi071.stdout:4/855: rename d2/d2a/d2d/d39/d42/d49/d67/f115 to d2/d2a/d2d/d39/d42/d49/d67/d7e/dae/f118 0 2024-04-23T02:14:12.625 INFO:tasks.workunit.client.1.smithi192.stdout:1/793: write dc/d14/d15/fcd [23477,37783] 0 2024-04-23T02:14:12.626 INFO:tasks.workunit.client.0.smithi071.stdout:4/856: chown d2/d2a/l78 18 1 2024-04-23T02:14:12.626 INFO:tasks.workunit.client.1.smithi192.stdout:1/794: write dc/d8b/fad [3372269,85866] 0 2024-04-23T02:14:12.627 INFO:tasks.workunit.client.0.smithi071.stdout:0/873: fsync d0/d3/de/d10/d14/f4d 0 2024-04-23T02:14:12.630 INFO:tasks.workunit.client.1.smithi192.stdout:1/795: read dc/d14/d15/d35/f40 [3819170,18868] 0 2024-04-23T02:14:12.631 INFO:tasks.workunit.client.0.smithi071.stdout:3/834: link d7/d8/l7f d7/da5/d128/d68/dae/dc6/l12c 0 2024-04-23T02:14:12.633 INFO:tasks.workunit.client.0.smithi071.stdout:0/874: mkdir d0/d3/d95/d134 0 2024-04-23T02:14:12.634 INFO:tasks.workunit.client.0.smithi071.stdout:4/857: write d2/d2a/da2/dd1/dde/d23/d4f/db5/f10a [691700,130984] 0 2024-04-23T02:14:12.634 INFO:tasks.workunit.client.0.smithi071.stdout:9/812: dread d1/f3d [0,4194304] 0 2024-04-23T02:14:12.636 INFO:tasks.workunit.client.0.smithi071.stdout:6/779: dwrite d6/d17/d77/d9f/dad/d3f/f47 [0,4194304] 0 2024-04-23T02:14:12.638 INFO:tasks.workunit.client.0.smithi071.stdout:2/719: dwrite d0/d2/d26/d14/f23 [0,4194304] 0 2024-04-23T02:14:12.638 INFO:tasks.workunit.client.1.smithi192.stdout:1/796: dwrite dc/d14/d15/fbc [0,4194304] 0 2024-04-23T02:14:12.645 INFO:tasks.workunit.client.1.smithi192.stdout:4/986: dwrite de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/de7/d101/f8d [0,4194304] 0 2024-04-23T02:14:12.646 INFO:tasks.workunit.client.0.smithi071.stdout:9/813: creat d1/d32/d3a/d77/f121 x:0 0 0 2024-04-23T02:14:12.649 INFO:tasks.workunit.client.0.smithi071.stdout:2/720: symlink d0/d2/d45/d7d/de0/lee 0 2024-04-23T02:14:12.649 INFO:tasks.workunit.client.0.smithi071.stdout:6/780: mknod d6/d1f/d46/cf7 0 2024-04-23T02:14:12.650 INFO:tasks.workunit.client.0.smithi071.stdout:4/858: truncate d2/d2a/d2d/d39/d42/d49/d67/f22 2105009 0 2024-04-23T02:14:12.651 INFO:tasks.workunit.client.0.smithi071.stdout:6/781: read - d6/d17/d77/d9f/dad/d3f/fee zero size 2024-04-23T02:14:12.652 INFO:tasks.workunit.client.1.smithi192.stdout:4/987: creat de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/d89/f14b x:0 0 0 2024-04-23T02:14:12.658 INFO:tasks.workunit.client.1.smithi192.stdout:4/988: symlink de/d15/d53/d81/de3/d74/d92/da0/dca/d34/l14c 0 2024-04-23T02:14:12.658 INFO:tasks.workunit.client.0.smithi071.stdout:0/875: link d0/d3/de/ff d0/d3/d95/d114/f135 0 2024-04-23T02:14:12.658 INFO:tasks.workunit.client.0.smithi071.stdout:9/814: creat d1/d32/d41/db7/d5a/dc6/d9f/dfe/f122 x:0 0 0 2024-04-23T02:14:12.659 INFO:tasks.workunit.client.0.smithi071.stdout:3/835: link d7/d22/d23/dba/df4/dce/ce8 d7/da5/d128/d68/d3b/d4c/d8b/d12a/c12d 0 2024-04-23T02:14:12.660 INFO:tasks.workunit.client.0.smithi071.stdout:4/859: mkdir d2/d2a/d2d/d39/d42/d49/d91/ddb/d10d/d119 0 2024-04-23T02:14:12.661 INFO:tasks.workunit.client.1.smithi192.stdout:1/797: dwrite dc/d8b/d62/f94 [0,4194304] 0 2024-04-23T02:14:12.662 INFO:tasks.workunit.client.0.smithi071.stdout:3/836: rmdir d7/dfd 39 2024-04-23T02:14:12.663 INFO:tasks.workunit.client.1.smithi192.stdout:4/989: fsync de/d15/d53/d81/de3/d74/d92/f110 0 2024-04-23T02:14:12.664 INFO:tasks.workunit.client.0.smithi071.stdout:0/876: write d0/d3/de/d10/d14/d32/dac/f12b [793406,115475] 0 2024-04-23T02:14:12.664 INFO:tasks.workunit.client.0.smithi071.stdout:5/900: sync 2024-04-23T02:14:12.665 INFO:tasks.workunit.client.0.smithi071.stdout:1/764: sync 2024-04-23T02:14:12.666 INFO:tasks.workunit.client.0.smithi071.stdout:1/765: readlink db/d92/la4 0 2024-04-23T02:14:12.667 INFO:tasks.workunit.client.1.smithi192.stdout:1/798: read dc/d80/d81/f9e [3829923,121244] 0 2024-04-23T02:14:12.669 INFO:tasks.workunit.client.0.smithi071.stdout:6/782: dwrite d6/d17/d77/f96 [0,4194304] 0 2024-04-23T02:14:12.669 INFO:tasks.workunit.client.0.smithi071.stdout:4/860: write d2/d2a/da2/dd1/dde/d23/d10/ffd [2252256,92352] 0 2024-04-23T02:14:12.675 INFO:tasks.workunit.client.0.smithi071.stdout:9/815: creat d1/d32/d41/db7/d39/d42/f123 x:0 0 0 2024-04-23T02:14:12.678 INFO:tasks.workunit.client.0.smithi071.stdout:6/783: mkdir d6/d1f/df8 0 2024-04-23T02:14:12.678 INFO:tasks.workunit.client.0.smithi071.stdout:3/837: rename d7/d22/d23/dba/df4/dca/fd4 to d7/d22/d23/d26/d52/f12e 0 2024-04-23T02:14:12.679 INFO:tasks.workunit.client.0.smithi071.stdout:6/784: chown d6/d17/d77/d9f/dad 282269039 1 2024-04-23T02:14:12.679 INFO:tasks.workunit.client.0.smithi071.stdout:9/816: read - d1/d32/d41/db7/d39/d42/f123 zero size 2024-04-23T02:14:12.680 INFO:tasks.workunit.client.0.smithi071.stdout:6/785: readlink d6/lba 0 2024-04-23T02:14:12.681 INFO:tasks.workunit.client.0.smithi071.stdout:4/861: read d2/d2a/d2d/d39/d42/d49/d67/d7e/dae/ffe [1294310,29213] 0 2024-04-23T02:14:12.683 INFO:tasks.workunit.client.0.smithi071.stdout:0/877: link d0/d3/de/d10/d14/f4d d0/d3/de/d10/f136 0 2024-04-23T02:14:12.707 INFO:tasks.workunit.client.1.smithi192.stdout:5/955: sync 2024-04-23T02:14:12.713 INFO:tasks.workunit.client.1.smithi192.stdout:5/956: dwrite d2/d32/f101 [0,4194304] 0 2024-04-23T02:14:12.734 INFO:tasks.workunit.client.1.smithi192.stdout:5/957: unlink d2/d32/cc4 0 2024-04-23T02:14:12.738 INFO:tasks.workunit.client.1.smithi192.stdout:4/990: creat de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d70/dd8/f14d x:0 0 0 2024-04-23T02:14:12.741 INFO:tasks.workunit.client.1.smithi192.stdout:4/991: unlink de/d15/d53/d81/de3/d74/d92/da0/dca/d49/f50 0 2024-04-23T02:14:12.742 INFO:tasks.workunit.client.1.smithi192.stdout:4/992: dread - de/d15/d53/d81/d10b/f12c zero size 2024-04-23T02:14:12.743 INFO:tasks.workunit.client.1.smithi192.stdout:4/993: write de/d15/d53/d81/d10b/f12c [737700,61247] 0 2024-04-23T02:14:12.750 INFO:tasks.workunit.client.1.smithi192.stdout:4/994: getdents de/d15/d53 0 2024-04-23T02:14:12.754 INFO:tasks.workunit.client.0.smithi071.stdout:8/848: sync 2024-04-23T02:14:12.757 INFO:tasks.workunit.client.1.smithi192.stdout:3/947: sync 2024-04-23T02:14:12.757 INFO:tasks.workunit.client.1.smithi192.stdout:9/923: sync 2024-04-23T02:14:12.759 INFO:tasks.workunit.client.1.smithi192.stdout:9/924: mknod d0/d9/d44/d74/c128 0 2024-04-23T02:14:12.761 INFO:tasks.workunit.client.0.smithi071.stdout:6/786: unlink d6/d17/c60 0 2024-04-23T02:14:12.762 INFO:tasks.workunit.client.1.smithi192.stdout:3/948: creat d1/d3/d6/d7/d29/d3b/da9/dbd/f150 x:0 0 0 2024-04-23T02:14:12.763 INFO:tasks.workunit.client.0.smithi071.stdout:3/838: truncate d7/da5/d128/d68/d3b/fb8 848201 0 2024-04-23T02:14:12.765 INFO:tasks.workunit.client.1.smithi192.stdout:3/949: write d1/d3/d17/d1a/d7c/dab/d6c/d84/fcb [4450433,38377] 0 2024-04-23T02:14:12.766 INFO:tasks.workunit.client.1.smithi192.stdout:9/925: dread d0/fdb [0,4194304] 0 2024-04-23T02:14:12.767 INFO:tasks.workunit.client.0.smithi071.stdout:8/849: symlink dc/d15/da0/d8d/df8/l110 0 2024-04-23T02:14:12.769 INFO:tasks.workunit.client.0.smithi071.stdout:3/839: write d7/d22/d23/d26/d7e/ffa [2328991,28008] 0 2024-04-23T02:14:12.770 INFO:tasks.workunit.client.0.smithi071.stdout:4/862: unlink d2/d2a/d2d/d39/d42/d49/d67/d7e/d82/fb1 0 2024-04-23T02:14:12.772 INFO:tasks.workunit.client.1.smithi192.stdout:9/926: symlink d0/d9/dc7/d106/l129 0 2024-04-23T02:14:12.780 INFO:tasks.workunit.client.0.smithi071.stdout:6/787: creat d6/d17/d77/d9f/dd8/ff9 x:0 0 0 2024-04-23T02:14:12.780 INFO:tasks.workunit.client.0.smithi071.stdout:2/721: dread d0/d2/d49/f5b [0,4194304] 0 2024-04-23T02:14:12.783 INFO:tasks.workunit.client.0.smithi071.stdout:8/850: creat dc/d56/d64/f111 x:0 0 0 2024-04-23T02:14:12.783 INFO:tasks.workunit.client.0.smithi071.stdout:5/901: dread d3/f11 [0,4194304] 0 2024-04-23T02:14:12.783 INFO:tasks.workunit.client.0.smithi071.stdout:8/851: chown dc/d15/da0/d8d/d9a/dad/fb2 887278 1 2024-04-23T02:14:12.784 INFO:tasks.workunit.client.0.smithi071.stdout:8/852: dread - dc/d15/da0/feb zero size 2024-04-23T02:14:12.786 INFO:tasks.workunit.client.0.smithi071.stdout:3/840: mkdir d7/d22/d23/d26/d52/d12f 0 2024-04-23T02:14:12.786 INFO:tasks.workunit.client.0.smithi071.stdout:0/878: write d0/ddf/f112 [300273,62420] 0 2024-04-23T02:14:12.789 INFO:tasks.workunit.client.0.smithi071.stdout:2/722: mkdir d0/d2/d49/def 0 2024-04-23T02:14:12.792 INFO:tasks.workunit.client.0.smithi071.stdout:0/879: read d0/d3/de/d10/d14/d32/dac/fae [2427243,41286] 0 2024-04-23T02:14:12.793 INFO:tasks.workunit.client.0.smithi071.stdout:5/902: fdatasync d3/da/d1a/db8/d140/dd2/f58 0 2024-04-23T02:14:12.793 INFO:tasks.workunit.client.0.smithi071.stdout:8/853: mkdir dc/d56/de1/dea/d112 0 2024-04-23T02:14:12.794 INFO:tasks.workunit.client.0.smithi071.stdout:0/880: chown d0/d3/f2d 648393 1 2024-04-23T02:14:12.796 INFO:tasks.workunit.client.0.smithi071.stdout:3/841: readlink d7/d22/d23/dba/df4/lcf 0 2024-04-23T02:14:12.796 INFO:tasks.workunit.client.0.smithi071.stdout:8/854: dread - dc/d56/d64/f111 zero size 2024-04-23T02:14:12.803 INFO:tasks.workunit.client.0.smithi071.stdout:4/863: dwrite d2/d2a/da2/dd1/dde/d23/d29/f9e [0,4194304] 0 2024-04-23T02:14:12.803 INFO:tasks.workunit.client.0.smithi071.stdout:2/723: symlink d0/d2/d45/d7d/d56/db0/d30/d80/lf0 0 2024-04-23T02:14:12.804 INFO:tasks.workunit.client.0.smithi071.stdout:3/842: truncate d7/da5/d128/d68/d3b/d4c/da1/fc2 396501 0 2024-04-23T02:14:12.806 INFO:tasks.workunit.client.0.smithi071.stdout:2/724: chown d0/d70 0 1 2024-04-23T02:14:12.807 INFO:tasks.workunit.client.0.smithi071.stdout:5/903: mkdir d3/d8f/de6/d143/d14c 0 2024-04-23T02:14:12.815 INFO:tasks.workunit.client.1.smithi192.stdout:5/958: write d2/d2e/d4e/fe4 [166187,100126] 0 2024-04-23T02:14:12.816 INFO:tasks.workunit.client.0.smithi071.stdout:3/843: unlink d7/da5/d128/f5b 0 2024-04-23T02:14:12.816 INFO:tasks.workunit.client.0.smithi071.stdout:5/904: unlink d3/lc 0 2024-04-23T02:14:12.817 INFO:tasks.workunit.client.0.smithi071.stdout:8/855: link dc/d15/f23 dc/d15/da0/d8d/d9a/f113 0 2024-04-23T02:14:12.818 INFO:tasks.workunit.client.0.smithi071.stdout:0/881: dwrite d0/d3/de/d10/d14/d32/dac/f12b [0,4194304] 0 2024-04-23T02:14:12.823 INFO:tasks.workunit.client.1.smithi192.stdout:5/959: creat d2/d6/d2d/d41/d43/f13a x:0 0 0 2024-04-23T02:14:12.825 INFO:tasks.workunit.client.1.smithi192.stdout:9/927: symlink d0/d9/d44/d51/l12a 0 2024-04-23T02:14:12.826 INFO:tasks.workunit.client.0.smithi071.stdout:4/864: dread d2/d2a/d2d/d39/d42/d49/d67/d3e/f93 [0,4194304] 0 2024-04-23T02:14:12.826 INFO:tasks.workunit.client.0.smithi071.stdout:8/856: creat dc/d15/d72/d95/f114 x:0 0 0 2024-04-23T02:14:12.827 INFO:tasks.workunit.client.0.smithi071.stdout:2/725: getdents d0/d2/d45/d7d/d56/db0/d30/d80 0 2024-04-23T02:14:12.828 INFO:tasks.workunit.client.1.smithi192.stdout:9/928: chown d0/d9/db3/d9e/fe0 223356 1 2024-04-23T02:14:12.829 INFO:tasks.workunit.client.0.smithi071.stdout:3/844: rmdir d7/d22/d11e 0 2024-04-23T02:14:12.829 INFO:tasks.workunit.client.1.smithi192.stdout:5/960: mkdir d2/d6/db4/db6/d13b 0 2024-04-23T02:14:12.830 INFO:tasks.workunit.client.1.smithi192.stdout:3/950: truncate d1/d3/d6/d7/f102 2943757 0 2024-04-23T02:14:12.831 INFO:tasks.workunit.client.0.smithi071.stdout:0/882: dread - d0/d3/de/d10/d14/d32/d44/d31/fe1 zero size 2024-04-23T02:14:12.832 INFO:tasks.workunit.client.0.smithi071.stdout:3/845: dread - d7/d22/d23/d26/d52/f11c zero size 2024-04-23T02:14:12.832 INFO:tasks.workunit.client.1.smithi192.stdout:9/929: write d0/d9/d44/d58/dd9/fe6 [796756,110784] 0 2024-04-23T02:14:12.832 INFO:tasks.workunit.client.0.smithi071.stdout:8/857: write dc/d15/f6b [2005120,40538] 0 2024-04-23T02:14:12.834 INFO:tasks.workunit.client.0.smithi071.stdout:2/726: creat d0/ff1 x:0 0 0 2024-04-23T02:14:12.834 INFO:tasks.workunit.client.1.smithi192.stdout:5/961: creat d2/db/d133/f13c x:0 0 0 2024-04-23T02:14:12.835 INFO:tasks.workunit.client.0.smithi071.stdout:5/905: link d3/c6 d3/da/d1a/db8/d106/c14d 0 2024-04-23T02:14:12.835 INFO:tasks.workunit.client.0.smithi071.stdout:6/788: dwrite d6/d1f/f68 [0,4194304] 0 2024-04-23T02:14:12.838 INFO:tasks.workunit.client.1.smithi192.stdout:5/962: symlink d2/d6/d14/d108/l13d 0 2024-04-23T02:14:12.844 INFO:tasks.workunit.client.0.smithi071.stdout:0/883: write d0/d3/de/d48/d7e/d8b/de3/f107 [491423,45625] 0 2024-04-23T02:14:12.845 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:12 smithi071 ceph-mon[39837]: pgmap v33: 65 pgs: 65 active+clean; 3.1 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 104 MiB/s rd, 238 MiB/s wr, 518 op/s 2024-04-23T02:14:12.847 INFO:tasks.workunit.client.1.smithi192.stdout:4/995: rename de/d15/d53/d81/de3/d74/d92/da0/dca/c123 to de/d15/d53/c14e 0 2024-04-23T02:14:12.849 INFO:tasks.workunit.client.0.smithi071.stdout:6/789: chown d6/d1f/d4e/fc3 47956988 1 2024-04-23T02:14:12.849 INFO:tasks.workunit.client.1.smithi192.stdout:1/799: mkdir dc/d14/dfd 0 2024-04-23T02:14:12.849 INFO:tasks.workunit.client.1.smithi192.stdout:5/963: dwrite d2/db/d16/d1b/f29 [0,4194304] 0 2024-04-23T02:14:12.852 INFO:tasks.workunit.client.0.smithi071.stdout:3/846: mkdir d7/da5/d128/d68/d3b/d4c/d130 0 2024-04-23T02:14:12.853 INFO:tasks.workunit.client.0.smithi071.stdout:3/847: truncate d7/d22/fc1 2085429 0 2024-04-23T02:14:12.854 INFO:tasks.workunit.client.1.smithi192.stdout:3/951: rename d1/d3/d17/d1a/d7c/dab/l75 to d1/d3/d6/d7/d29/d3b/da9/d8b/d118/l151 0 2024-04-23T02:14:12.854 INFO:tasks.workunit.client.1.smithi192.stdout:1/800: fsync dc/d8b/d34/d70/d8c/fa4 0 2024-04-23T02:14:12.855 INFO:tasks.workunit.client.0.smithi071.stdout:8/858: link dc/d78/f91 dc/d15/da0/f115 0 2024-04-23T02:14:12.855 INFO:tasks.workunit.client.0.smithi071.stdout:5/906: dread d3/da/d1a/db8/d140/dd2/f5e [0,4194304] 0 2024-04-23T02:14:12.855 INFO:tasks.workunit.client.0.smithi071.stdout:3/848: write d7/d8/f1a [1925509,43935] 0 2024-04-23T02:14:12.856 INFO:tasks.workunit.client.0.smithi071.stdout:0/884: chown d0/d3/de/d10/f1f 104330 1 2024-04-23T02:14:12.857 INFO:tasks.workunit.client.0.smithi071.stdout:8/859: stat dc/d56/d64/dde 0 2024-04-23T02:14:12.859 INFO:tasks.workunit.client.0.smithi071.stdout:8/860: fsync dc/d15/da0/feb 0 2024-04-23T02:14:12.859 INFO:tasks.workunit.client.0.smithi071.stdout:6/790: creat d6/d22/ffa x:0 0 0 2024-04-23T02:14:12.860 INFO:tasks.workunit.client.0.smithi071.stdout:4/865: write d2/d2a/d2d/d39/d42/d49/d91/fd2 [190382,113614] 0 2024-04-23T02:14:12.860 INFO:tasks.workunit.client.0.smithi071.stdout:2/727: write d0/d2/d45/d7d/d56/db0/f65 [4345231,58500] 0 2024-04-23T02:14:12.862 INFO:tasks.workunit.client.0.smithi071.stdout:2/728: chown d0/d2/d34/d55/lb2 5159815 1 2024-04-23T02:14:12.864 INFO:tasks.workunit.client.1.smithi192.stdout:1/801: rename dc/d16/d19/d30/l90 to dc/d8b/d44/lfe 0 2024-04-23T02:14:12.865 INFO:tasks.workunit.client.1.smithi192.stdout:9/930: symlink d0/d9/dc7/l12b 0 2024-04-23T02:14:12.865 INFO:tasks.workunit.client.0.smithi071.stdout:3/849: mknod d7/d22/d23/d26/d52/c131 0 2024-04-23T02:14:12.866 INFO:tasks.workunit.client.1.smithi192.stdout:1/802: fsync dc/d8b/fa0 0 2024-04-23T02:14:12.866 INFO:tasks.workunit.client.0.smithi071.stdout:6/791: mkdir d6/d17/d77/d85/dfb 0 2024-04-23T02:14:12.867 INFO:tasks.workunit.client.1.smithi192.stdout:1/803: readlink dc/d16/d19/l26 0 2024-04-23T02:14:12.867 INFO:tasks.workunit.client.0.smithi071.stdout:0/885: mkdir d0/d3/de/d137 0 2024-04-23T02:14:12.867 INFO:tasks.workunit.client.0.smithi071.stdout:8/861: write dc/d15/d5d/faa [1299952,14547] 0 2024-04-23T02:14:12.870 INFO:tasks.workunit.client.0.smithi071.stdout:4/866: chown d2/d2a/d2d/d39/d42/d49/d67/d3e/f6f 5066 1 2024-04-23T02:14:12.872 INFO:tasks.workunit.client.1.smithi192.stdout:1/804: dread dc/d16/d19/d74/faf [0,4194304] 0 2024-04-23T02:14:12.873 INFO:tasks.workunit.client.0.smithi071.stdout:6/792: dread - d6/d22/d92/fae zero size 2024-04-23T02:14:12.874 INFO:tasks.workunit.client.0.smithi071.stdout:2/729: dwrite d0/d2/d34/d55/d96/fde [0,4194304] 0 2024-04-23T02:14:12.875 INFO:tasks.workunit.client.0.smithi071.stdout:5/907: chown d3/d34/c3c 8023 1 2024-04-23T02:14:12.876 INFO:tasks.workunit.client.1.smithi192.stdout:3/952: truncate d1/d3/d17/d1a/d7c/dab/d6c/d84/d46/fbc 7251003 0 2024-04-23T02:14:12.876 INFO:tasks.workunit.client.0.smithi071.stdout:2/730: chown d0/d2/fa3 88 1 2024-04-23T02:14:12.879 INFO:tasks.workunit.client.0.smithi071.stdout:4/867: read d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/fcc [293130,95523] 0 2024-04-23T02:14:12.881 INFO:tasks.workunit.client.1.smithi192.stdout:1/805: dread dc/d14/d15/d35/f73 [0,4194304] 0 2024-04-23T02:14:12.882 INFO:tasks.workunit.client.1.smithi192.stdout:3/953: stat d1/d3/d6/d7/d29/d3b/da9/daf/ddf/d12f/d13d/c13f 0 2024-04-23T02:14:12.882 INFO:tasks.workunit.client.1.smithi192.stdout:4/996: dwrite de/d15/d53/d81/de3/d74/d92/da0/dca/d49/d26/d59/fcd [0,4194304] 0 2024-04-23T02:14:12.883 INFO:tasks.workunit.client.0.smithi071.stdout:4/868: chown d2/d53/l5f 1450361410 1 2024-04-23T02:14:12.884 INFO:tasks.workunit.client.1.smithi192.stdout:4/997: stat de/d15/d53/d81/de3/f2c 0 2024-04-23T02:14:12.885 INFO:tasks.workunit.client.0.smithi071.stdout:2/731: truncate d0/d2/d34/f50 5569144 0 2024-04-23T02:14:12.888 INFO:tasks.workunit.client.0.smithi071.stdout:5/908: getdents d3/d34 0 2024-04-23T02:14:12.891 INFO:tasks.workunit.client.0.smithi071.stdout:3/850: dread d7/d8/f16 [0,4194304] 0 2024-04-23T02:14:12.893 INFO:tasks.workunit.client.0.smithi071.stdout:3/851: fdatasync d7/da5/d128/d68/f84 0 2024-04-23T02:14:12.894 INFO:tasks.workunit.client.1.smithi192.stdout:1/806: rename dc/fd to dc/d16/fff 0 2024-04-23T02:14:12.897 INFO:tasks.workunit.client.1.smithi192.stdout:1/807: link dc/d14/d23/lda dc/d8b/d44/da9/l100 0 2024-04-23T02:14:12.899 INFO:tasks.workunit.client.1.smithi192.stdout:3/954: dread d1/d3/d6/d7/d71/f95 [0,4194304] 0 2024-04-23T02:14:12.902 INFO:tasks.workunit.client.0.smithi071.stdout:4/869: write d2/d2a/da2/dd1/dde/d23/f71 [1241304,91455] 0 2024-04-23T02:14:12.903 INFO:tasks.workunit.client.1.smithi192.stdout:1/808: link dc/d16/c3c dc/d16/d95/c101 0 2024-04-23T02:14:12.904 INFO:tasks.workunit.client.0.smithi071.stdout:5/909: dwrite d3/d8f/de6/d143/dff/f126 [0,4194304] 0 2024-04-23T02:14:12.904 INFO:tasks.workunit.client.0.smithi071.stdout:3/852: getdents d7/da5/d128/d68/d3b/d4c 0 2024-04-23T02:14:12.904 INFO:tasks.workunit.client.1.smithi192.stdout:3/955: dread d1/d3/d17/d2b/f32 [0,4194304] 0 2024-04-23T02:14:12.905 INFO:tasks.workunit.client.1.smithi192.stdout:4/998: dwrite de/f18 [0,4194304] 0 2024-04-23T02:14:12.906 INFO:tasks.workunit.client.0.smithi071.stdout:8/862: dwrite dc/d15/da0/d30/fa6 [0,4194304] 0 2024-04-23T02:14:12.906 INFO:tasks.workunit.client.0.smithi071.stdout:0/886: dread d0/d3/de/d10/d14/d32/d8a/df0/f55 [0,4194304] 0 2024-04-23T02:14:12.907 INFO:tasks.workunit.client.1.smithi192.stdout:1/809: readlink dc/d80/lca 0 2024-04-23T02:14:12.908 INFO:tasks.workunit.client.0.smithi071.stdout:6/793: dread d6/d1f/d5d/d95/fb8 [0,4194304] 0 2024-04-23T02:14:12.909 INFO:tasks.workunit.client.1.smithi192.stdout:4/999: chown de/d15/d53/d81/de3/d74/d92/da0/dca/d34 611159 1 2024-04-23T02:14:12.911 INFO:tasks.workunit.client.1.smithi192.stdout:3/956: rename d1/d3/d17/d1a/d7c/dab/d6c/d84/d46/c66 to d1/d3/d17/d1f/d33/dd6/d11f/c152 0 2024-04-23T02:14:12.912 INFO:tasks.workunit.client.0.smithi071.stdout:5/910: stat d3/d1d/daf/d10/da5/d5f/fd6 0 2024-04-23T02:14:12.912 INFO:tasks.workunit.client.1.smithi192.stdout:1/810: dread dc/d8b/d62/fc6 [0,4194304] 0 2024-04-23T02:14:12.913 INFO:tasks.workunit.client.0.smithi071.stdout:3/853: creat d7/d22/d23/d26/d52/dd3/f132 x:0 0 0 2024-04-23T02:14:12.913 INFO:tasks.workunit.client.0.smithi071.stdout:8/863: read dc/d15/da0/f14 [1627205,106042] 0 2024-04-23T02:14:12.913 INFO:tasks.workunit.client.1.smithi192.stdout:1/811: chown dc/d16/d19/d74/dd2/dc2 5 1 2024-04-23T02:14:12.915 INFO:tasks.workunit.client.1.smithi192.stdout:1/812: write dc/d8b/fe7 [1454420,103385] 0 2024-04-23T02:14:12.916 INFO:tasks.workunit.client.0.smithi071.stdout:2/732: dwrite d0/d2/d26/d14/f1c [0,4194304] 0 2024-04-23T02:14:12.919 INFO:tasks.workunit.client.0.smithi071.stdout:3/854: chown d7/d22/da7/l90 132964 1 2024-04-23T02:14:12.920 INFO:tasks.workunit.client.0.smithi071.stdout:8/864: chown dc/d15/da0/d8d/d9a/dad/ff5 15 1 2024-04-23T02:14:12.920 INFO:tasks.workunit.client.1.smithi192.stdout:3/957: dwrite d1/d3/d17/d2b/fd5 [0,4194304] 0 2024-04-23T02:14:12.925 INFO:tasks.workunit.client.0.smithi071.stdout:6/794: dwrite d6/d17/d77/d85/fa1 [0,4194304] 0 2024-04-23T02:14:12.925 INFO:tasks.workunit.client.0.smithi071.stdout:0/887: mknod d0/d3/de/d10/d14/d32/d44/d59/c138 0 2024-04-23T02:14:12.928 INFO:tasks.workunit.client.0.smithi071.stdout:5/911: unlink d3/da/d1a/db8/d140/f65 0 2024-04-23T02:14:12.928 INFO:tasks.workunit.client.0.smithi071.stdout:0/888: dread - d0/ddf/f118 zero size 2024-04-23T02:14:12.928 INFO:tasks.workunit.client.1.smithi192.stdout:1/813: link dc/d16/d19/d74/dd2/lf0 dc/d16/d19/d74/dd2/l102 0 2024-04-23T02:14:12.931 INFO:tasks.workunit.client.0.smithi071.stdout:4/870: link d2/d2a/da2/dd1/dde/d23/d10/lee d2/d2a/d2d/d39/d42/d49/d67/l11a 0 2024-04-23T02:14:12.932 INFO:tasks.workunit.client.1.smithi192.stdout:1/814: symlink dc/l103 0 2024-04-23T02:14:12.932 INFO:tasks.workunit.client.1.smithi192.stdout:3/958: fsync d1/d3/d6/d7/d29/d3b/da9/daf/ddf/d12f/d13d/fae 0 2024-04-23T02:14:12.934 INFO:tasks.workunit.client.1.smithi192.stdout:3/959: mkdir d1/d3/d6/d7/d29/d3b/da9/daf/d153 0 2024-04-23T02:14:12.934 INFO:tasks.workunit.client.0.smithi071.stdout:3/855: symlink d7/da5/d128/l133 0 2024-04-23T02:14:12.936 INFO:tasks.workunit.client.1.smithi192.stdout:1/815: getdents dc/d8b/d34/d70/d8c/da3/de4/de8 0 2024-04-23T02:14:12.937 INFO:tasks.workunit.client.0.smithi071.stdout:2/733: mkdir d0/d2/d34/dd4/df2 0 2024-04-23T02:14:12.938 INFO:tasks.workunit.client.0.smithi071.stdout:0/889: write d0/d3/de/d10/d14/d32/d8a/df0/fe6 [286151,59262] 0 2024-04-23T02:14:12.939 INFO:tasks.workunit.client.0.smithi071.stdout:2/734: dread - d0/d2/d26/d3a/dc3/d36/d5c/d85/fdc zero size 2024-04-23T02:14:12.942 INFO:tasks.workunit.client.0.smithi071.stdout:4/871: stat d2/d2a/da2/dd1/dde/d23/d4f/cb4 0 2024-04-23T02:14:12.942 INFO:tasks.workunit.client.1.smithi192.stdout:1/816: creat dc/d14/d23/de0/df8/f104 x:0 0 0 2024-04-23T02:14:12.942 INFO:tasks.workunit.client.1.smithi192.stdout:3/960: rename d1/d3/d6/d7/d29/d3b/da9/daf/ddf/d12f/d13d to d1/d3/d17/d1f/d33/d154 0 2024-04-23T02:14:12.943 INFO:tasks.workunit.client.0.smithi071.stdout:0/890: creat d0/d3/de/d10/db9/f139 x:0 0 0 2024-04-23T02:14:12.944 INFO:tasks.workunit.client.1.smithi192.stdout:3/961: mkdir d1/d3/d17/d1a/d7c/dab/d155 0 2024-04-23T02:14:12.946 INFO:tasks.workunit.client.0.smithi071.stdout:2/735: getdents d0/d2/d34/dbf 0 2024-04-23T02:14:12.947 INFO:tasks.workunit.client.0.smithi071.stdout:3/856: dwrite d7/da5/db2/fe6 [0,4194304] 0 2024-04-23T02:14:12.947 INFO:tasks.workunit.client.1.smithi192.stdout:1/817: read dc/d16/d19/d30/f3f [189654,66337] 0 2024-04-23T02:14:12.948 INFO:tasks.workunit.client.0.smithi071.stdout:0/891: read d0/d3/de/d10/d14/d32/dac/f12b [564577,112230] 0 2024-04-23T02:14:12.948 INFO:tasks.workunit.client.0.smithi071.stdout:4/872: mkdir d2/d2a/d2d/d39/d42/d49/d67/ded/d11b 0 2024-04-23T02:14:12.950 INFO:tasks.workunit.client.1.smithi192.stdout:1/818: fsync f9 0 2024-04-23T02:14:12.951 INFO:tasks.workunit.client.0.smithi071.stdout:2/736: symlink d0/d2/d45/d7d/d56/lf3 0 2024-04-23T02:14:12.954 INFO:tasks.workunit.client.0.smithi071.stdout:3/857: creat d7/da5/d128/d68/dae/df3/f134 x:0 0 0 2024-04-23T02:14:12.956 INFO:tasks.workunit.client.0.smithi071.stdout:4/873: rename d2/d2a/d2d/d39/fa6 to d2/d2a/d2d/d39/d42/d49/d91/ddb/d10d/f11c 0 2024-04-23T02:14:12.956 INFO:tasks.workunit.client.1.smithi192.stdout:3/962: dwrite d1/d3/d17/d1a/d7c/fe0 [0,4194304] 0 2024-04-23T02:14:12.960 INFO:tasks.workunit.client.1.smithi192.stdout:1/819: link dc/d16/d19/f3a dc/d8b/d34/d70/f105 0 2024-04-23T02:14:12.961 INFO:tasks.workunit.client.1.smithi192.stdout:3/963: creat d1/d3/d6/d7/d29/d3b/da9/d8b/f156 x:0 0 0 2024-04-23T02:14:12.961 INFO:tasks.workunit.client.0.smithi071.stdout:6/795: write d6/d1f/f21 [430809,127756] 0 2024-04-23T02:14:12.962 INFO:tasks.workunit.client.1.smithi192.stdout:3/964: fdatasync d1/d3/d17/d65/fe5 0 2024-04-23T02:14:12.963 INFO:tasks.workunit.client.0.smithi071.stdout:4/874: chown d2/d2a/d73/c86 50803280 1 2024-04-23T02:14:12.967 INFO:tasks.workunit.client.0.smithi071.stdout:6/796: mknod d6/d1f/df8/cfc 0 2024-04-23T02:14:12.967 INFO:tasks.workunit.client.0.smithi071.stdout:4/875: write d2/d2a/d2d/d7b/f8c [1891479,70270] 0 2024-04-23T02:14:12.970 INFO:tasks.workunit.client.0.smithi071.stdout:6/797: stat d6/d17/d77/da2/fc6 0 2024-04-23T02:14:12.971 INFO:tasks.workunit.client.0.smithi071.stdout:5/912: dread d3/d1d/daf/d10/da5/f66 [0,4194304] 0 2024-04-23T02:14:12.972 INFO:tasks.workunit.client.0.smithi071.stdout:5/913: chown d3/da/f33 325563 1 2024-04-23T02:14:12.975 INFO:tasks.workunit.client.0.smithi071.stdout:6/798: stat d6/d1f/d46/f6e 0 2024-04-23T02:14:12.977 INFO:tasks.workunit.client.0.smithi071.stdout:4/876: dwrite d2/d2a/d2d/d39/d42/f101 [0,4194304] 0 2024-04-23T02:14:12.977 INFO:tasks.workunit.client.1.smithi192.stdout:1/820: truncate dc/d16/d19/d74/fb6 3046281 0 2024-04-23T02:14:12.978 INFO:tasks.workunit.client.1.smithi192.stdout:3/965: write d1/d3/d6/d7/d29/d3b/da9/daf/f11a [233573,83309] 0 2024-04-23T02:14:12.979 INFO:tasks.workunit.client.0.smithi071.stdout:6/799: symlink d6/d1f/d46/lfd 0 2024-04-23T02:14:12.979 INFO:tasks.workunit.client.0.smithi071.stdout:5/914: link d3/d1d/daf/l11f d3/d8f/de6/l14e 0 2024-04-23T02:14:12.981 INFO:tasks.workunit.client.0.smithi071.stdout:0/892: dwrite d0/d3/de/ff [4194304,4194304] 0 2024-04-23T02:14:12.982 INFO:tasks.workunit.client.0.smithi071.stdout:4/877: stat d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/dc8/fe8 0 2024-04-23T02:14:12.982 INFO:tasks.workunit.client.0.smithi071.stdout:5/915: chown d3/da/d1a/db8/d140/c90 0 1 2024-04-23T02:14:12.984 INFO:tasks.workunit.client.0.smithi071.stdout:6/800: creat d6/d17/d77/d9f/dad/d8f/ffe x:0 0 0 2024-04-23T02:14:12.985 INFO:tasks.workunit.client.0.smithi071.stdout:0/893: chown d0/d3/de/d10/d14/d32/d44/d31/l71 1719514 1 2024-04-23T02:14:12.985 INFO:tasks.workunit.client.0.smithi071.stdout:4/878: chown d2/d2a/da2/dd1/dde/c98 28307179 1 2024-04-23T02:14:12.986 INFO:tasks.workunit.client.0.smithi071.stdout:5/916: unlink d3/da/d1a/db8/d140/c90 0 2024-04-23T02:14:12.987 INFO:tasks.workunit.client.1.smithi192.stdout:3/966: dwrite d1/d3/d17/d1f/d33/d24/feb [0,4194304] 0 2024-04-23T02:14:12.988 INFO:tasks.workunit.client.0.smithi071.stdout:4/879: fdatasync d2/d2a/d2d/d39/d42/d49/d67/f1a 0 2024-04-23T02:14:12.988 INFO:tasks.workunit.client.0.smithi071.stdout:6/801: creat d6/d17/d77/d9f/dad/d3f/fff x:0 0 0 2024-04-23T02:14:12.991 INFO:tasks.workunit.client.1.smithi192.stdout:3/967: truncate d1/d3/fd0 459542 0 2024-04-23T02:14:12.991 INFO:tasks.workunit.client.0.smithi071.stdout:4/880: readlink d2/d2a/d2d/d39/d42/d49/d67/d3e/l52 0 2024-04-23T02:14:12.993 INFO:tasks.workunit.client.0.smithi071.stdout:0/894: dread d0/d3/de/d10/d14/d32/d8a/df0/f55 [0,4194304] 0 2024-04-23T02:14:12.994 INFO:tasks.workunit.client.0.smithi071.stdout:0/895: stat d0/d63 0 2024-04-23T02:14:12.996 INFO:tasks.workunit.client.0.smithi071.stdout:6/802: rename d6/d17/d77/d9f/dad/d3f/f47 to d6/d17/d77/d9f/f100 0 2024-04-23T02:14:12.997 INFO:tasks.workunit.client.1.smithi192.stdout:3/968: dwrite d1/f55 [0,4194304] 0 2024-04-23T02:14:12.998 INFO:tasks.workunit.client.0.smithi071.stdout:5/917: link d3/d34/f40 d3/da/d1a/db8/d140/dd2/da9/f14f 0 2024-04-23T02:14:13.002 INFO:tasks.workunit.client.1.smithi192.stdout:3/969: creat d1/d3/d17/d2b/d44/d57/f157 x:0 0 0 2024-04-23T02:14:13.002 INFO:tasks.workunit.client.0.smithi071.stdout:4/881: read d2/d2a/da2/dd1/dde/d2c/f7c [331862,74805] 0 2024-04-23T02:14:13.002 INFO:tasks.workunit.client.0.smithi071.stdout:6/803: mknod d6/d17/d77/d9f/dad/d3f/c101 0 2024-04-23T02:14:13.003 INFO:tasks.workunit.client.1.smithi192.stdout:3/970: truncate d1/d3/f7a 4990135 0 2024-04-23T02:14:13.007 INFO:tasks.workunit.client.0.smithi071.stdout:6/804: mknod d6/d17/d77/d9f/dd8/c102 0 2024-04-23T02:14:13.008 INFO:tasks.workunit.client.0.smithi071.stdout:4/882: stat d2/d2a/da2/dd1/dde/d23/d4f/db5/db6/ccf 0 2024-04-23T02:14:13.009 INFO:tasks.workunit.client.1.smithi192.stdout:3/971: rename d1/d3/d6/d7/d29/d3b/fdd to d1/d3/d17/d1f/d33/d24/f158 0 2024-04-23T02:14:13.010 INFO:tasks.workunit.client.0.smithi071.stdout:4/883: write d2/d2a/d2d/d7b/f10b [712517,69230] 0 2024-04-23T02:14:13.010 INFO:tasks.workunit.client.0.smithi071.stdout:0/896: creat d0/d3/de/d10/d14/d32/d44/f13a x:0 0 0 2024-04-23T02:14:13.012 INFO:tasks.workunit.client.0.smithi071.stdout:6/805: chown d6/d17/d77/d9f/dad/d3f/fd9 16951 1 2024-04-23T02:14:13.014 INFO:tasks.workunit.client.0.smithi071.stdout:5/918: getdents d3/d1d/daf 0 2024-04-23T02:14:13.015 INFO:tasks.workunit.client.0.smithi071.stdout:5/919: chown d3/d34/f6a 827 1 2024-04-23T02:14:13.017 INFO:tasks.workunit.client.1.smithi192.stdout:3/972: dwrite d1/d3/d17/d1a/f113 [0,4194304] 0 2024-04-23T02:14:13.018 INFO:tasks.workunit.client.1.smithi192.stdout:1/821: dread dc/d16/d19/d74/fa6 [0,4194304] 0 2024-04-23T02:14:13.018 INFO:tasks.workunit.client.0.smithi071.stdout:0/897: truncate d0/d3/de/d10/d14/d32/d37/d26/fde 257107 0 2024-04-23T02:14:13.019 INFO:tasks.workunit.client.1.smithi192.stdout:1/822: truncate f1 8662026 0 2024-04-23T02:14:13.019 INFO:tasks.workunit.client.1.smithi192.stdout:3/973: dread - d1/d3/d17/d1a/d7c/dab/d6c/fa2 zero size 2024-04-23T02:14:13.023 INFO:tasks.workunit.client.0.smithi071.stdout:6/806: creat d6/d17/d77/d9f/db5/f103 x:0 0 0 2024-04-23T02:14:13.024 INFO:tasks.workunit.client.0.smithi071.stdout:0/898: creat d0/d3/de/d10/d14/d32/d8a/f13b x:0 0 0 2024-04-23T02:14:13.025 INFO:tasks.workunit.client.1.smithi192.stdout:3/974: link d1/d3/d6/d7/d5f/c10c d1/d3/d17/d1f/c159 0 2024-04-23T02:14:13.027 INFO:tasks.workunit.client.0.smithi071.stdout:6/807: write d6/d17/d77/d9f/ff0 [98341,93885] 0 2024-04-23T02:14:13.028 INFO:tasks.workunit.client.0.smithi071.stdout:5/920: rename d3/da/d1a/db8/d140/dd2/c5d to d3/d10f/d13a/c150 0 2024-04-23T02:14:13.029 INFO:tasks.workunit.client.0.smithi071.stdout:0/899: mknod d0/d3/de/d10/d14/dd5/c13c 0 2024-04-23T02:14:13.032 INFO:tasks.workunit.client.1.smithi192.stdout:3/975: link d1/d3/d17/d1a/f113 d1/d3/d17/d65/f15a 0 2024-04-23T02:14:13.036 INFO:tasks.workunit.client.1.smithi192.stdout:3/976: chown d1/d3/d6/d7/d29/d3b/da9/daf/ddf/f107 73 1 2024-04-23T02:14:13.037 INFO:tasks.workunit.client.1.smithi192.stdout:1/823: dwrite dc/d14/d15/f54 [0,4194304] 0 2024-04-23T02:14:13.038 INFO:tasks.workunit.client.0.smithi071.stdout:6/808: dwrite d6/d1f/f21 [0,4194304] 0 2024-04-23T02:14:13.041 INFO:tasks.workunit.client.0.smithi071.stdout:0/900: rename d0/d3/de/d10/d14/d32/d44/d87/fa0 to d0/d3/de/d10/d14/f13d 0 2024-04-23T02:14:13.042 INFO:tasks.workunit.client.1.smithi192.stdout:3/977: rename d1/d3/d17/d2b/d44/d57/db0/d142 to d1/d3/d6/d7/d29/d145/d15b 0 2024-04-23T02:14:13.043 INFO:tasks.workunit.client.1.smithi192.stdout:3/978: fdatasync d1/d3/d17/d2b/fd5 0 2024-04-23T02:14:13.044 INFO:tasks.workunit.client.0.smithi071.stdout:4/884: dread d2/d2a/daa/fad [0,4194304] 0 2024-04-23T02:14:13.045 INFO:tasks.workunit.client.1.smithi192.stdout:3/979: read d1/d3/f7a [3188605,8941] 0 2024-04-23T02:14:13.047 INFO:tasks.workunit.client.1.smithi192.stdout:1/824: read dc/d16/d19/d74/fb6 [416151,106003] 0 2024-04-23T02:14:13.053 INFO:tasks.workunit.client.0.smithi071.stdout:0/901: creat d0/d3/de/d10/d14/d32/d44/d31/d52/f13e x:0 0 0 2024-04-23T02:14:13.054 INFO:tasks.workunit.client.0.smithi071.stdout:4/885: getdents d2/d2a/d2d/d39/d42/d49/d91/ddb 0 2024-04-23T02:14:13.056 INFO:tasks.workunit.client.0.smithi071.stdout:5/921: dwrite d3/da/d1a/f1b [0,4194304] 0 2024-04-23T02:14:13.057 INFO:tasks.workunit.client.0.smithi071.stdout:4/886: fdatasync d2/d2a/da2/dd1/dde/d23/f59 0 2024-04-23T02:14:13.057 INFO:tasks.workunit.client.0.smithi071.stdout:5/922: chown d3/d8f/fc6 207301 1 2024-04-23T02:14:13.064 INFO:tasks.workunit.client.0.smithi071.stdout:0/902: getdents d0/d3/de/d10/d14/d32/d8a/df0 0 2024-04-23T02:14:13.068 INFO:tasks.workunit.client.1.smithi192.stdout:1/825: truncate f1 7207850 0 2024-04-23T02:14:13.071 INFO:tasks.workunit.client.0.smithi071.stdout:5/923: dwrite d3/da/d1a/db8/d140/dd2/da9/f11d [0,4194304] 0 2024-04-23T02:14:13.072 INFO:tasks.workunit.client.1.smithi192.stdout:1/826: rename dc/d14/d23/de0/df8 to dc/d16/dde/d106 0 2024-04-23T02:14:13.075 INFO:tasks.workunit.client.0.smithi071.stdout:5/924: creat d3/da/d1a/d1c/d46/f151 x:0 0 0 2024-04-23T02:14:13.078 INFO:tasks.workunit.client.0.smithi071.stdout:5/925: mkdir d3/da/d1a/d1c/db6/d152 0 2024-04-23T02:14:13.081 INFO:tasks.workunit.client.0.smithi071.stdout:6/809: dread d6/d1f/d5d/f67 [0,4194304] 0 2024-04-23T02:14:13.081 INFO:tasks.workunit.client.0.smithi071.stdout:5/926: read - d3/d1d/daf/d10/da5/d5f/dc0/dfa/f101 zero size 2024-04-23T02:14:13.083 INFO:tasks.workunit.client.0.smithi071.stdout:5/927: mkdir d3/da/df6/d153 0 2024-04-23T02:14:13.087 INFO:tasks.workunit.client.0.smithi071.stdout:5/928: creat d3/d1d/daf/d10/da5/d5f/dc0/dfa/f154 x:0 0 0 2024-04-23T02:14:13.088 INFO:tasks.workunit.client.1.smithi192.stdout:1/827: dwrite dc/d16/f1f [0,4194304] 0 2024-04-23T02:14:13.088 INFO:tasks.workunit.client.0.smithi071.stdout:6/810: dwrite d6/d17/d77/fed [0,4194304] 0 2024-04-23T02:14:13.091 INFO:tasks.workunit.client.0.smithi071.stdout:5/929: symlink d3/d1d/daf/d10/da5/d5f/l155 0 2024-04-23T02:14:13.092 INFO:tasks.workunit.client.0.smithi071.stdout:5/930: chown d3/d1d/daf/c8c 115784446 1 2024-04-23T02:14:13.093 INFO:tasks.workunit.client.1.smithi192.stdout:1/828: symlink dc/d14/d15/d35/l107 0 2024-04-23T02:14:13.097 INFO:tasks.workunit.client.0.smithi071.stdout:6/811: rename d6/d17/d77/d9f to d6/d104 0 2024-04-23T02:14:13.097 INFO:tasks.workunit.client.0.smithi071.stdout:6/812: dread - d6/d104/db5/f103 zero size 2024-04-23T02:14:13.098 INFO:tasks.workunit.client.1.smithi192.stdout:1/829: symlink dc/d16/d19/d74/df4/l108 0 2024-04-23T02:14:13.099 INFO:tasks.workunit.client.0.smithi071.stdout:5/931: mkdir d3/d8f/de6/d143/dff/d156 0 2024-04-23T02:14:13.100 INFO:tasks.workunit.client.0.smithi071.stdout:5/932: fsync d3/da/d1a/f80 0 2024-04-23T02:14:13.101 INFO:tasks.workunit.client.0.smithi071.stdout:5/933: chown d3/da/d1a/db8/d140/dd2/c133 5560616 1 2024-04-23T02:14:13.101 INFO:tasks.workunit.client.0.smithi071.stdout:6/813: unlink d6/d17/d77/l8e 0 2024-04-23T02:14:13.103 INFO:tasks.workunit.client.0.smithi071.stdout:6/814: mkdir d6/d17/d77/d105 0 2024-04-23T02:14:13.104 INFO:tasks.workunit.client.0.smithi071.stdout:5/934: dread - d3/da/d1a/db8/d106/f14b zero size 2024-04-23T02:14:13.107 INFO:tasks.workunit.client.0.smithi071.stdout:6/815: rename f4 to d6/f106 0 2024-04-23T02:14:13.109 INFO:tasks.workunit.client.0.smithi071.stdout:6/816: rename d6/lbe to d6/d104/dd8/l107 0 2024-04-23T02:14:13.110 INFO:tasks.workunit.client.0.smithi071.stdout:6/817: readlink d6/d22/lc1 0 2024-04-23T02:14:13.112 INFO:tasks.workunit.client.0.smithi071.stdout:6/818: mkdir d6/d17/d77/d105/d108 0 2024-04-23T02:14:13.113 INFO:tasks.workunit.client.0.smithi071.stdout:6/819: fdatasync d6/d1f/d5d/f6f 0 2024-04-23T02:14:13.121 INFO:tasks.workunit.client.0.smithi071.stdout:6/820: dwrite d6/d17/d77/da2/fa7 [0,4194304] 0 2024-04-23T02:14:13.122 INFO:tasks.workunit.client.0.smithi071.stdout:6/821: write d6/d17/d77/da2/fa7 [4102541,80800] 0 2024-04-23T02:14:13.124 INFO:tasks.workunit.client.0.smithi071.stdout:6/822: dread d6/d104/dad/f8d [0,4194304] 0 2024-04-23T02:14:13.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:12 smithi192 ceph-mon[49662]: pgmap v33: 65 pgs: 65 active+clean; 3.1 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 104 MiB/s rd, 238 MiB/s wr, 518 op/s 2024-04-23T02:14:13.179 INFO:tasks.workunit.client.0.smithi071.stdout:1/766: sync 2024-04-23T02:14:13.180 INFO:tasks.workunit.client.0.smithi071.stdout:9/817: sync 2024-04-23T02:14:13.183 INFO:tasks.workunit.client.0.smithi071.stdout:1/767: dread - db/f6a zero size 2024-04-23T02:14:13.186 INFO:tasks.workunit.client.0.smithi071.stdout:9/818: symlink d1/d4e/d66/da4/dd7/l124 0 2024-04-23T02:14:13.188 INFO:tasks.workunit.client.0.smithi071.stdout:9/819: rmdir d1/d32/d41/db7/d5a 39 2024-04-23T02:14:13.189 INFO:tasks.workunit.client.0.smithi071.stdout:6/823: dwrite d6/f34 [0,4194304] 0 2024-04-23T02:14:13.190 INFO:tasks.workunit.client.0.smithi071.stdout:6/824: dread - d6/d22/ffa zero size 2024-04-23T02:14:13.191 INFO:tasks.workunit.client.0.smithi071.stdout:9/820: mknod d1/d32/d41/db7/d71/dbf/c125 0 2024-04-23T02:14:13.193 INFO:tasks.workunit.client.0.smithi071.stdout:6/825: write d6/d1f/d5d/d95/f98 [4074631,53884] 0 2024-04-23T02:14:13.203 INFO:tasks.workunit.client.0.smithi071.stdout:6/826: read d6/d17/d77/d85/fa1 [3559623,11026] 0 2024-04-23T02:14:13.203 INFO:tasks.workunit.client.0.smithi071.stdout:1/768: dwrite db/fd [0,4194304] 0 2024-04-23T02:14:13.212 INFO:tasks.workunit.client.0.smithi071.stdout:6/827: creat d6/d17/d77/da2/f109 x:0 0 0 2024-04-23T02:14:13.213 INFO:tasks.workunit.client.0.smithi071.stdout:6/828: readlink d6/d22/dbf/leb 0 2024-04-23T02:14:13.214 INFO:tasks.workunit.client.0.smithi071.stdout:6/829: write d6/d104/db5/f103 [190822,2178] 0 2024-04-23T02:14:13.214 INFO:tasks.workunit.client.0.smithi071.stdout:1/769: creat db/df/d13/d2f/d48/d9f/dfb/f108 x:0 0 0 2024-04-23T02:14:13.215 INFO:tasks.workunit.client.0.smithi071.stdout:6/830: readlink d6/d22/l76 0 2024-04-23T02:14:13.217 INFO:tasks.workunit.client.0.smithi071.stdout:6/831: write d6/d17/d77/fe0 [964404,81872] 0 2024-04-23T02:14:13.218 INFO:tasks.workunit.client.0.smithi071.stdout:6/832: read f1 [6714226,79805] 0 2024-04-23T02:14:13.220 INFO:tasks.workunit.client.0.smithi071.stdout:9/821: dread d1/d32/d41/f57 [0,4194304] 0 2024-04-23T02:14:13.221 INFO:tasks.workunit.client.0.smithi071.stdout:9/822: chown d1/f3d 0 1 2024-04-23T02:14:13.226 INFO:tasks.workunit.client.0.smithi071.stdout:6/833: dread d6/d1f/f21 [0,4194304] 0 2024-04-23T02:14:13.226 INFO:tasks.workunit.client.0.smithi071.stdout:6/834: chown d6/d1f/d4e/le1 9 1 2024-04-23T02:14:13.227 INFO:tasks.workunit.client.0.smithi071.stdout:6/835: mkdir d6/d17/d77/d85/dfb/d10a 0 2024-04-23T02:14:13.229 INFO:tasks.workunit.client.0.smithi071.stdout:1/770: dread db/df/d13/d2b/f70 [0,4194304] 0 2024-04-23T02:14:13.229 INFO:tasks.workunit.client.0.smithi071.stdout:6/836: write d6/d1f/f83 [2473550,69886] 0 2024-04-23T02:14:13.231 INFO:tasks.workunit.client.0.smithi071.stdout:6/837: write d6/d17/d77/fe0 [1396813,43915] 0 2024-04-23T02:14:13.240 INFO:tasks.workunit.client.0.smithi071.stdout:9/823: write d1/fc [5196195,69958] 0 2024-04-23T02:14:13.244 INFO:tasks.workunit.client.0.smithi071.stdout:9/824: creat d1/d32/d41/db7/d39/d42/f126 x:0 0 0 2024-04-23T02:14:13.247 INFO:tasks.workunit.client.0.smithi071.stdout:1/771: dwrite db/df/d13/d2f/f3c [0,4194304] 0 2024-04-23T02:14:13.248 INFO:tasks.workunit.client.0.smithi071.stdout:9/825: write d1/f10 [2282577,48757] 0 2024-04-23T02:14:13.248 INFO:tasks.workunit.client.0.smithi071.stdout:6/838: dwrite d6/d17/d77/f8c [0,4194304] 0 2024-04-23T02:14:13.252 INFO:tasks.workunit.client.0.smithi071.stdout:6/839: rename d6/d1f/d4e to d6/d104/dad/dd1/d10b 0 2024-04-23T02:14:13.255 INFO:tasks.workunit.client.0.smithi071.stdout:6/840: unlink d6/d17/c23 0 2024-04-23T02:14:13.256 INFO:tasks.workunit.client.0.smithi071.stdout:6/841: fsync d6/f16 0 2024-04-23T02:14:13.257 INFO:tasks.workunit.client.1.smithi192.stdout:9/931: sync 2024-04-23T02:14:13.257 INFO:tasks.workunit.client.1.smithi192.stdout:5/964: sync 2024-04-23T02:14:13.259 INFO:tasks.workunit.client.0.smithi071.stdout:6/842: symlink d6/d104/dad/dd1/d10b/l10c 0 2024-04-23T02:14:13.260 INFO:tasks.workunit.client.1.smithi192.stdout:5/965: creat d2/d2e/df0/f13e x:0 0 0 2024-04-23T02:14:13.260 INFO:tasks.workunit.client.0.smithi071.stdout:6/843: truncate d6/d17/d77/f8c 4435810 0 2024-04-23T02:14:13.260 INFO:tasks.workunit.client.1.smithi192.stdout:9/932: symlink d0/l12c 0 2024-04-23T02:14:13.262 INFO:tasks.workunit.client.1.smithi192.stdout:5/966: mknod d2/d6/d14/d113/c13f 0 2024-04-23T02:14:13.263 INFO:tasks.workunit.client.1.smithi192.stdout:5/967: write d2/d6/d14/d113/f132 [6452,96769] 0 2024-04-23T02:14:13.270 INFO:tasks.workunit.client.1.smithi192.stdout:9/933: dwrite d0/d9/f3a [0,4194304] 0 2024-04-23T02:14:13.270 INFO:tasks.workunit.client.1.smithi192.stdout:5/968: mknod d2/d6/d2d/d41/d43/d98/c140 0 2024-04-23T02:14:13.271 INFO:tasks.workunit.client.1.smithi192.stdout:9/934: readlink d0/d9/d44/l7f 0 2024-04-23T02:14:13.274 INFO:tasks.workunit.client.1.smithi192.stdout:5/969: symlink d2/d6/db4/l141 0 2024-04-23T02:14:13.278 INFO:tasks.workunit.client.1.smithi192.stdout:5/970: symlink d2/db/d16/d48/db7/l142 0 2024-04-23T02:14:13.291 INFO:tasks.workunit.client.1.smithi192.stdout:9/935: dread d0/d9/d44/d74/f82 [0,4194304] 0 2024-04-23T02:14:13.293 INFO:tasks.workunit.client.1.smithi192.stdout:9/936: mkdir d0/d9/d44/d58/d89/d12d 0 2024-04-23T02:14:13.294 INFO:tasks.workunit.client.1.smithi192.stdout:5/971: dread d2/d6/f8a [4194304,4194304] 0 2024-04-23T02:14:13.295 INFO:tasks.workunit.client.1.smithi192.stdout:9/937: mknod d0/d9/d44/d58/dd9/def/c12e 0 2024-04-23T02:14:13.295 INFO:tasks.workunit.client.1.smithi192.stdout:5/972: chown d2/d6/d14/l51 0 1 2024-04-23T02:14:13.297 INFO:tasks.workunit.client.1.smithi192.stdout:5/973: creat d2/db/d16/d48/f143 x:0 0 0 2024-04-23T02:14:13.300 INFO:tasks.workunit.client.1.smithi192.stdout:5/974: mkdir d2/db/d2b/d52/dc2/d49/d110/d144 0 2024-04-23T02:14:13.303 INFO:tasks.workunit.client.1.smithi192.stdout:5/975: creat d2/db/f145 x:0 0 0 2024-04-23T02:14:13.304 INFO:tasks.workunit.client.1.smithi192.stdout:5/976: write d2/db/d2b/df7/ffb [681677,46394] 0 2024-04-23T02:14:13.307 INFO:tasks.workunit.client.1.smithi192.stdout:9/938: dread d0/d9/d44/d58/dd9/feb [0,4194304] 0 2024-04-23T02:14:13.310 INFO:tasks.workunit.client.1.smithi192.stdout:5/977: dwrite d2/d6/d2d/d41/f97 [0,4194304] 0 2024-04-23T02:14:13.311 INFO:tasks.workunit.client.1.smithi192.stdout:5/978: chown d2/d32/f35 1054 1 2024-04-23T02:14:13.314 INFO:tasks.workunit.client.1.smithi192.stdout:5/979: dread d2/db/d46/f12f [0,4194304] 0 2024-04-23T02:14:13.318 INFO:tasks.workunit.client.1.smithi192.stdout:5/980: rename d2/db/d16/d1b/f89 to d2/d6/d14/f146 0 2024-04-23T02:14:13.320 INFO:tasks.workunit.client.1.smithi192.stdout:5/981: mknod d2/db/d2b/d52/dc2/c147 0 2024-04-23T02:14:13.323 INFO:tasks.workunit.client.1.smithi192.stdout:5/982: rmdir d2/db/d16/d48/dc5/d121 39 2024-04-23T02:14:13.325 INFO:tasks.workunit.client.1.smithi192.stdout:3/980: sync 2024-04-23T02:14:13.325 INFO:tasks.workunit.client.1.smithi192.stdout:1/830: sync 2024-04-23T02:14:13.327 INFO:tasks.workunit.client.1.smithi192.stdout:5/983: rename d2/d6/d2d/c39 to d2/db/d2b/d52/dc2/d49/d110/c148 0 2024-04-23T02:14:13.329 INFO:tasks.workunit.client.1.smithi192.stdout:9/939: dread d0/d9/d44/d51/ff5 [0,4194304] 0 2024-04-23T02:14:13.331 INFO:tasks.workunit.client.1.smithi192.stdout:3/981: dread d1/d3/d17/d65/fe5 [0,4194304] 0 2024-04-23T02:14:13.333 INFO:tasks.workunit.client.1.smithi192.stdout:5/984: symlink d2/db/d16/d48/dc5/l149 0 2024-04-23T02:14:13.333 INFO:tasks.workunit.client.1.smithi192.stdout:9/940: chown d0/d9/db3/d5f/l6b 27 1 2024-04-23T02:14:13.334 INFO:tasks.workunit.client.1.smithi192.stdout:1/831: dwrite dc/d14/d15/fbc [0,4194304] 0 2024-04-23T02:14:13.336 INFO:tasks.workunit.client.1.smithi192.stdout:9/941: write d0/d9/d44/d51/d5b/d61/faf [1031569,77822] 0 2024-04-23T02:14:13.340 INFO:tasks.workunit.client.1.smithi192.stdout:9/942: readlink d0/d9/d44/d51/d5b/l124 0 2024-04-23T02:14:13.340 INFO:tasks.workunit.client.1.smithi192.stdout:5/985: unlink d2/d2e/d4e/de6/cf4 0 2024-04-23T02:14:13.340 INFO:tasks.workunit.client.1.smithi192.stdout:3/982: link d1/d3/d17/f6e d1/d3/d17/d1a/d7c/dab/f15c 0 2024-04-23T02:14:13.343 INFO:tasks.workunit.client.1.smithi192.stdout:1/832: dread dc/d14/d23/f87 [0,4194304] 0 2024-04-23T02:14:13.346 INFO:tasks.workunit.client.0.smithi071.stdout:8/865: sync 2024-04-23T02:14:13.347 INFO:tasks.workunit.client.1.smithi192.stdout:9/943: write d0/d9/d44/d51/d5b/f69 [944656,37571] 0 2024-04-23T02:14:13.348 INFO:tasks.workunit.client.1.smithi192.stdout:3/983: write d1/d3/d17/d2b/d44/d57/fb4 [238553,67438] 0 2024-04-23T02:14:13.353 INFO:tasks.workunit.client.1.smithi192.stdout:9/944: getdents d0/d9/d44/d58/d89/dd8 0 2024-04-23T02:14:13.355 INFO:tasks.workunit.client.1.smithi192.stdout:3/984: write d1/d3/d17/d1a/f23 [3020047,42142] 0 2024-04-23T02:14:13.356 INFO:tasks.workunit.client.1.smithi192.stdout:5/986: dread d2/d32/f44 [0,4194304] 0 2024-04-23T02:14:13.362 INFO:tasks.workunit.client.1.smithi192.stdout:3/985: rename d1/d3/d17/d2b/d44/d57/l61 to d1/d3/d17/df3/d136/l15d 0 2024-04-23T02:14:13.362 INFO:tasks.workunit.client.1.smithi192.stdout:9/945: dwrite d0/d9/d44/f49 [0,4194304] 0 2024-04-23T02:14:13.367 INFO:tasks.workunit.client.1.smithi192.stdout:5/987: rename d2/d6/d2d/d41/d43/dbc/fc1 to d2/d6/d14/d60/f14a 0 2024-04-23T02:14:13.367 INFO:tasks.workunit.client.0.smithi071.stdout:8/866: dread dc/d15/f6b [0,4194304] 0 2024-04-23T02:14:13.368 INFO:tasks.workunit.client.1.smithi192.stdout:9/946: unlink d0/d9/d44/l7f 0 2024-04-23T02:14:13.369 INFO:tasks.workunit.client.1.smithi192.stdout:5/988: read d2/db/d16/f8b [1964371,26583] 0 2024-04-23T02:14:13.370 INFO:tasks.workunit.client.1.smithi192.stdout:3/986: creat d1/d3/d6/d7/d29/d3b/da9/dbd/f15e x:0 0 0 2024-04-23T02:14:13.370 INFO:tasks.workunit.client.0.smithi071.stdout:8/867: mkdir dc/d15/da0/d32/d116 0 2024-04-23T02:14:13.375 INFO:tasks.workunit.client.0.smithi071.stdout:8/868: read dc/d15/f39 [2863545,70959] 0 2024-04-23T02:14:13.377 INFO:tasks.workunit.client.1.smithi192.stdout:3/987: dwrite d1/d3/d17/d2b/d44/d57/db0/fdb [0,4194304] 0 2024-04-23T02:14:13.378 INFO:tasks.workunit.client.0.smithi071.stdout:8/869: creat dc/d15/da0/f117 x:0 0 0 2024-04-23T02:14:13.387 INFO:tasks.workunit.client.0.smithi071.stdout:8/870: write dc/d15/da0/d55/f6c [5373481,83979] 0 2024-04-23T02:14:13.390 INFO:tasks.workunit.client.0.smithi071.stdout:8/871: creat dc/f118 x:0 0 0 2024-04-23T02:14:13.395 INFO:tasks.workunit.client.0.smithi071.stdout:8/872: dwrite dc/d15/d5d/faa [0,4194304] 0 2024-04-23T02:14:13.396 INFO:tasks.workunit.client.1.smithi192.stdout:3/988: dwrite d1/d3/d17/d1a/d7c/fd2 [0,4194304] 0 2024-04-23T02:14:13.396 INFO:tasks.workunit.client.1.smithi192.stdout:9/947: dwrite d0/d9/d44/d51/d5b/d61/f6a [0,4194304] 0 2024-04-23T02:14:13.398 INFO:tasks.workunit.client.0.smithi071.stdout:8/873: truncate dc/d56/d64/fc8 491003 0 2024-04-23T02:14:13.399 INFO:tasks.workunit.client.1.smithi192.stdout:3/989: dread - d1/d3/d17/d2b/f130 zero size 2024-04-23T02:14:13.405 INFO:tasks.workunit.client.1.smithi192.stdout:3/990: mkdir d1/d3/d6/d15f 0 2024-04-23T02:14:13.409 INFO:tasks.workunit.client.1.smithi192.stdout:9/948: dwrite d0/f4e [0,4194304] 0 2024-04-23T02:14:13.411 INFO:tasks.workunit.client.1.smithi192.stdout:9/949: chown d0/d98/ldc 261179 1 2024-04-23T02:14:13.413 INFO:tasks.workunit.client.1.smithi192.stdout:9/950: chown d0/d9/d44/d58/dd9/def/cfc 1139 1 2024-04-23T02:14:13.433 INFO:tasks.workunit.client.1.smithi192.stdout:9/951: write d0/d9/d44/d4f/f28 [748184,130403] 0 2024-04-23T02:14:13.436 INFO:tasks.workunit.client.1.smithi192.stdout:9/952: creat d0/f12f x:0 0 0 2024-04-23T02:14:13.437 INFO:tasks.workunit.client.1.smithi192.stdout:9/953: stat d0/d9/d44/d51/l12a 0 2024-04-23T02:14:13.438 INFO:tasks.workunit.client.1.smithi192.stdout:9/954: stat d0/l12 0 2024-04-23T02:14:13.438 INFO:tasks.workunit.client.1.smithi192.stdout:3/991: dwrite d1/d3/d17/d1a/d7c/dab/d6c/fa2 [0,4194304] 0 2024-04-23T02:14:13.444 INFO:tasks.workunit.client.1.smithi192.stdout:9/955: getdents d0/d9/db3/d5f 0 2024-04-23T02:14:13.450 INFO:tasks.workunit.client.1.smithi192.stdout:9/956: dwrite d0/f2c [0,4194304] 0 2024-04-23T02:14:13.455 INFO:tasks.workunit.client.1.smithi192.stdout:9/957: dread d0/d9/d44/d51/d5b/d61/faf [0,4194304] 0 2024-04-23T02:14:13.459 INFO:tasks.workunit.client.1.smithi192.stdout:9/958: dread d0/d9/db3/d5f/fc8 [4194304,4194304] 0 2024-04-23T02:14:13.461 INFO:tasks.workunit.client.1.smithi192.stdout:9/959: creat d0/d9/d44/d58/f130 x:0 0 0 2024-04-23T02:14:13.462 INFO:tasks.workunit.client.1.smithi192.stdout:9/960: mknod d0/d9/db3/d5f/c131 0 2024-04-23T02:14:13.468 INFO:tasks.workunit.client.1.smithi192.stdout:9/961: dwrite d0/d9/db3/d9e/dff/f105 [0,4194304] 0 2024-04-23T02:14:13.474 INFO:tasks.workunit.client.1.smithi192.stdout:9/962: mknod d0/d9/db3/d5f/dfd/c132 0 2024-04-23T02:14:13.477 INFO:tasks.workunit.client.1.smithi192.stdout:9/963: unlink d0/d9/d44/d74/de7/l10a 0 2024-04-23T02:14:13.480 INFO:tasks.workunit.client.1.smithi192.stdout:9/964: mkdir d0/d9/db3/d5f/dfd/d120/d133 0 2024-04-23T02:14:13.490 INFO:tasks.workunit.client.1.smithi192.stdout:9/965: truncate d0/f5c 853399 0 2024-04-23T02:14:13.492 INFO:tasks.workunit.client.1.smithi192.stdout:1/833: sync 2024-04-23T02:14:13.493 INFO:tasks.workunit.client.1.smithi192.stdout:9/966: link d0/d98/lbf d0/d9/d44/d53/l134 0 2024-04-23T02:14:13.494 INFO:tasks.workunit.client.1.smithi192.stdout:9/967: fdatasync d0/d9/db3/fba 0 2024-04-23T02:14:13.502 INFO:tasks.workunit.client.1.smithi192.stdout:9/968: dwrite d0/d9/dc7/fe1 [0,4194304] 0 2024-04-23T02:14:13.503 INFO:tasks.workunit.client.1.smithi192.stdout:9/969: readlink d0/d9/d44/d51/d5b/dd3/le2 0 2024-04-23T02:14:13.503 INFO:tasks.workunit.client.1.smithi192.stdout:1/834: dwrite dc/d16/d19/f3a [0,4194304] 0 2024-04-23T02:14:13.508 INFO:tasks.workunit.client.1.smithi192.stdout:1/835: rmdir dc/d8b/d44 39 2024-04-23T02:14:13.511 INFO:tasks.workunit.client.1.smithi192.stdout:1/836: truncate dc/d16/d19/d30/f89 696224 0 2024-04-23T02:14:13.512 INFO:tasks.workunit.client.1.smithi192.stdout:1/837: chown dc/d14/d23/f50 104537 1 2024-04-23T02:14:13.514 INFO:tasks.workunit.client.1.smithi192.stdout:1/838: creat dc/d8b/d34/d70/d8c/da3/dc1/f109 x:0 0 0 2024-04-23T02:14:13.514 INFO:tasks.workunit.client.1.smithi192.stdout:1/839: chown dc/d80 1 1 2024-04-23T02:14:13.517 INFO:tasks.workunit.client.1.smithi192.stdout:1/840: rename dc/d16/d19/f3a to dc/d16/dde/d106/f10a 0 2024-04-23T02:14:13.523 INFO:tasks.workunit.client.1.smithi192.stdout:1/841: readlink dc/d14/d15/d35/l3b 0 2024-04-23T02:14:13.523 INFO:tasks.workunit.client.1.smithi192.stdout:9/970: truncate d0/d9/d44/f45 3636382 0 2024-04-23T02:14:13.524 INFO:tasks.workunit.client.1.smithi192.stdout:1/842: chown dc/d16/d19/dae/lbd 62863953 1 2024-04-23T02:14:13.530 INFO:tasks.workunit.client.1.smithi192.stdout:1/843: dread dc/d16/d19/f79 [0,4194304] 0 2024-04-23T02:14:13.531 INFO:tasks.workunit.client.1.smithi192.stdout:1/844: write dc/d14/d15/ff2 [83046,19545] 0 2024-04-23T02:14:13.532 INFO:tasks.workunit.client.1.smithi192.stdout:1/845: write dc/d14/d15/ff2 [412029,97376] 0 2024-04-23T02:14:13.547 INFO:tasks.workunit.client.1.smithi192.stdout:9/971: link d0/d9/d44/d58/d89/fea d0/d9/d44/d58/f135 0 2024-04-23T02:14:13.548 INFO:tasks.workunit.client.1.smithi192.stdout:9/972: dread - d0/d9/db3/d5f/dfd/f127 zero size 2024-04-23T02:14:13.548 INFO:tasks.workunit.client.1.smithi192.stdout:1/846: rename dc/d8b/d62/ccf to dc/d14/c10b 0 2024-04-23T02:14:13.550 INFO:tasks.workunit.client.1.smithi192.stdout:9/973: creat d0/d9/d44/d51/d5b/f136 x:0 0 0 2024-04-23T02:14:13.550 INFO:tasks.workunit.client.1.smithi192.stdout:1/847: rmdir dc/d16/d19/d30 39 2024-04-23T02:14:13.554 INFO:tasks.workunit.client.1.smithi192.stdout:1/848: mkdir dc/d16/d10c 0 2024-04-23T02:14:13.555 INFO:tasks.workunit.client.1.smithi192.stdout:9/974: rmdir d0/d9/db3/d5f/d115 0 2024-04-23T02:14:13.556 INFO:tasks.workunit.client.1.smithi192.stdout:9/975: write d0/d9/f3a [2359240,102392] 0 2024-04-23T02:14:13.560 INFO:tasks.workunit.client.1.smithi192.stdout:1/849: rename dc/d8b/d44/ff1 to dc/d16/d19/f10d 0 2024-04-23T02:14:13.562 INFO:tasks.workunit.client.1.smithi192.stdout:1/850: write dc/d16/db0/fd8 [2467155,86378] 0 2024-04-23T02:14:13.565 INFO:tasks.workunit.client.1.smithi192.stdout:1/851: rmdir dc/d8b/d34/d70/d8c/da3 39 2024-04-23T02:14:13.567 INFO:tasks.workunit.client.1.smithi192.stdout:9/976: dwrite d0/d9/d44/d58/dd9/f11d [0,4194304] 0 2024-04-23T02:14:13.573 INFO:tasks.workunit.client.1.smithi192.stdout:9/977: unlink d0/d9/db3/d5f/dc9/ldd 0 2024-04-23T02:14:13.575 INFO:tasks.workunit.client.1.smithi192.stdout:1/852: rename dc/d16/d19/d74/d77/l7c to dc/d16/d19/l10e 0 2024-04-23T02:14:13.576 INFO:tasks.workunit.client.1.smithi192.stdout:3/992: sync 2024-04-23T02:14:13.576 INFO:tasks.workunit.client.1.smithi192.stdout:5/989: sync 2024-04-23T02:14:13.578 INFO:tasks.workunit.client.0.smithi071.stdout:2/737: sync 2024-04-23T02:14:13.578 INFO:tasks.workunit.client.1.smithi192.stdout:1/853: chown dc/d16/d19/d30/ff3 4566625 1 2024-04-23T02:14:13.580 INFO:tasks.workunit.client.0.smithi071.stdout:2/738: mknod d0/d2/d45/d7d/d56/db0/cf4 0 2024-04-23T02:14:13.580 INFO:tasks.workunit.client.1.smithi192.stdout:9/978: getdents d0/d9/d44/d51/d5b/dd3 0 2024-04-23T02:14:13.581 INFO:tasks.workunit.client.1.smithi192.stdout:9/979: write d0/d9/dc7/fe1 [2404609,5778] 0 2024-04-23T02:14:13.581 INFO:tasks.workunit.client.0.smithi071.stdout:2/739: fsync d0/d2/d45/d7d/d56/db0/d30/fad 0 2024-04-23T02:14:13.582 INFO:tasks.workunit.client.1.smithi192.stdout:9/980: read - d0/d9/d44/d4f/d114/f121 zero size 2024-04-23T02:14:13.585 INFO:tasks.workunit.client.1.smithi192.stdout:5/990: getdents d2/d6/d2d/d41/d43/d98 0 2024-04-23T02:14:13.586 INFO:tasks.workunit.client.0.smithi071.stdout:2/740: rename d0/d2/fc4 to d0/d2/d26/d3a/dc3/d36/d5c/d85/ff5 0 2024-04-23T02:14:13.588 INFO:tasks.workunit.client.1.smithi192.stdout:9/981: rename d0/d9/d44/d51/c113 to d0/d9/db3/d5f/dfd/d120/d133/c137 0 2024-04-23T02:14:13.589 INFO:tasks.workunit.client.0.smithi071.stdout:2/741: rename d0/d2/d26/d3a/dc3/d36/d6a/da1/ldd to d0/d2/d26/d3a/dc3/d36/d5c/lf6 0 2024-04-23T02:14:13.591 INFO:tasks.workunit.client.0.smithi071.stdout:2/742: mknod d0/d2/cf7 0 2024-04-23T02:14:13.591 INFO:tasks.workunit.client.0.smithi071.stdout:2/743: fdatasync d0/d70/f86 0 2024-04-23T02:14:13.592 INFO:tasks.workunit.client.0.smithi071.stdout:2/744: fsync d0/d2/d34/dbf/fea 0 2024-04-23T02:14:13.592 INFO:tasks.workunit.client.1.smithi192.stdout:5/991: getdents d2/db/d2b/d52/dc2 0 2024-04-23T02:14:13.593 INFO:tasks.workunit.client.0.smithi071.stdout:2/745: readlink d0/d2/d26/l17 0 2024-04-23T02:14:13.593 INFO:tasks.workunit.client.1.smithi192.stdout:5/992: chown d2/db/d16/d48/d4f/l100 1940945107 1 2024-04-23T02:14:13.594 INFO:tasks.workunit.client.0.smithi071.stdout:2/746: creat d0/d2/d34/d83/ff8 x:0 0 0 2024-04-23T02:14:13.595 INFO:tasks.workunit.client.0.smithi071.stdout:2/747: symlink d0/d2/d34/d55/lf9 0 2024-04-23T02:14:13.596 INFO:tasks.workunit.client.1.smithi192.stdout:5/993: creat d2/db/d2b/f14b x:0 0 0 2024-04-23T02:14:13.597 INFO:tasks.workunit.client.0.smithi071.stdout:2/748: symlink d0/d2/da0/db7/lfa 0 2024-04-23T02:14:13.601 INFO:tasks.workunit.client.0.smithi071.stdout:2/749: dread d0/d2/d49/f5b [0,4194304] 0 2024-04-23T02:14:13.603 INFO:tasks.workunit.client.0.smithi071.stdout:2/750: symlink d0/d2/d26/d3a/dc3/d36/d5c/d85/lfb 0 2024-04-23T02:14:13.604 INFO:tasks.workunit.client.0.smithi071.stdout:2/751: chown d0/d2/d34/d55/d96 0 1 2024-04-23T02:14:13.606 INFO:tasks.workunit.client.0.smithi071.stdout:2/752: mkdir d0/d2/d26/d3a/dc3/d36/d5c/d85/de9/dfc 0 2024-04-23T02:14:13.607 INFO:tasks.workunit.client.1.smithi192.stdout:9/982: dread d0/d9/db3/d9e/fe5 [0,4194304] 0 2024-04-23T02:14:13.608 INFO:tasks.workunit.client.0.smithi071.stdout:2/753: read d0/d2/d45/d7d/d56/f89 [2711742,120817] 0 2024-04-23T02:14:13.610 INFO:tasks.workunit.client.0.smithi071.stdout:2/754: mknod d0/d2/d45/d7d/d56/db0/d30/db8/cfd 0 2024-04-23T02:14:13.611 INFO:tasks.workunit.client.1.smithi192.stdout:1/854: truncate dc/d80/d81/f9e 1518731 0 2024-04-23T02:14:13.612 INFO:tasks.workunit.client.0.smithi071.stdout:2/755: chown d0/d2/d45/c97 1 1 2024-04-23T02:14:13.614 INFO:tasks.workunit.client.1.smithi192.stdout:1/855: write dc/d14/d15/ff2 [1347199,110476] 0 2024-04-23T02:14:13.614 INFO:tasks.workunit.client.1.smithi192.stdout:9/983: link d0/d9/db3/d5f/fc8 d0/d9/d44/d58/d89/d12d/f138 0 2024-04-23T02:14:13.615 INFO:tasks.workunit.client.1.smithi192.stdout:1/856: stat dc/d16/d19 0 2024-04-23T02:14:13.617 INFO:tasks.workunit.client.1.smithi192.stdout:5/994: dwrite d2/d6/f4d [0,4194304] 0 2024-04-23T02:14:13.617 INFO:tasks.workunit.client.1.smithi192.stdout:1/857: dread - dc/d16/d19/d74/dd2/ff5 zero size 2024-04-23T02:14:13.633 INFO:tasks.workunit.client.0.smithi071.stdout:2/756: dread d0/d2/d26/d3a/dc3/d4f/f66 [0,4194304] 0 2024-04-23T02:14:13.633 INFO:tasks.workunit.client.0.smithi071.stdout:2/757: mkdir d0/d2/d26/d3a/dc3/d36/d5c/dfe 0 2024-04-23T02:14:13.633 INFO:tasks.workunit.client.0.smithi071.stdout:2/758: chown d0/d2/d34/dd4/df2 554719802 1 2024-04-23T02:14:13.634 INFO:tasks.workunit.client.1.smithi192.stdout:5/995: creat d2/db/d16/d48/dc5/d121/f14c x:0 0 0 2024-04-23T02:14:13.634 INFO:tasks.workunit.client.1.smithi192.stdout:1/858: dwrite dc/d16/d19/f10d [0,4194304] 0 2024-04-23T02:14:13.634 INFO:tasks.workunit.client.1.smithi192.stdout:5/996: truncate d2/db8/f114 862467 0 2024-04-23T02:14:13.634 INFO:tasks.workunit.client.1.smithi192.stdout:1/859: write dc/d16/d19/d30/f55 [3240903,58952] 0 2024-04-23T02:14:13.635 INFO:tasks.workunit.client.1.smithi192.stdout:1/860: creat dc/d16/d19/d30/dc4/f10f x:0 0 0 2024-04-23T02:14:13.635 INFO:tasks.workunit.client.0.smithi071.stdout:2/759: mkdir d0/d2/d45/d7d/d56/db0/d30/dff 0 2024-04-23T02:14:13.636 INFO:tasks.workunit.client.1.smithi192.stdout:1/861: symlink dc/d16/d19/d74/df4/l110 0 2024-04-23T02:14:13.638 INFO:tasks.workunit.client.1.smithi192.stdout:1/862: creat dc/d8b/f111 x:0 0 0 2024-04-23T02:14:13.639 INFO:tasks.workunit.client.0.smithi071.stdout:3/858: sync 2024-04-23T02:14:13.640 INFO:tasks.workunit.client.0.smithi071.stdout:0/903: sync 2024-04-23T02:14:13.640 INFO:tasks.workunit.client.0.smithi071.stdout:9/826: sync 2024-04-23T02:14:13.640 INFO:tasks.workunit.client.0.smithi071.stdout:8/874: sync 2024-04-23T02:14:13.640 INFO:tasks.workunit.client.0.smithi071.stdout:4/887: sync 2024-04-23T02:14:13.640 INFO:tasks.workunit.client.0.smithi071.stdout:5/935: sync 2024-04-23T02:14:13.640 INFO:tasks.workunit.client.0.smithi071.stdout:1/772: sync 2024-04-23T02:14:13.640 INFO:tasks.workunit.client.0.smithi071.stdout:6/844: sync 2024-04-23T02:14:13.641 INFO:tasks.workunit.client.1.smithi192.stdout:1/863: mkdir dc/d16/dde/de9/d112 0 2024-04-23T02:14:13.641 INFO:tasks.workunit.client.1.smithi192.stdout:9/984: dread d0/d9/d44/f45 [0,4194304] 0 2024-04-23T02:14:13.644 INFO:tasks.workunit.client.0.smithi071.stdout:8/875: mknod dc/d15/d50/c119 0 2024-04-23T02:14:13.645 INFO:tasks.workunit.client.0.smithi071.stdout:5/936: dread - d3/d1d/daf/d10/da5/f94 zero size 2024-04-23T02:14:13.646 INFO:tasks.workunit.client.1.smithi192.stdout:1/864: creat dc/d16/d19/d30/f113 x:0 0 0 2024-04-23T02:14:13.646 INFO:tasks.workunit.client.0.smithi071.stdout:3/859: dread - d7/d22/d23/d26/d52/dd3/f132 zero size 2024-04-23T02:14:13.647 INFO:tasks.workunit.client.0.smithi071.stdout:1/773: read - db/d11/d7f/da7/dd9/d100/f81 zero size 2024-04-23T02:14:13.648 INFO:tasks.workunit.client.0.smithi071.stdout:2/760: dwrite d0/d2/d34/dbf/fd7 [0,4194304] 0 2024-04-23T02:14:13.650 INFO:tasks.workunit.client.0.smithi071.stdout:9/827: fsync d1/d32/d41/db7/d39/d42/f49 0 2024-04-23T02:14:13.653 INFO:tasks.workunit.client.0.smithi071.stdout:4/888: dread d2/d2a/d2d/d7b/f10b [0,4194304] 0 2024-04-23T02:14:13.656 INFO:tasks.workunit.client.0.smithi071.stdout:3/860: rmdir d7/da5/d100 39 2024-04-23T02:14:13.656 INFO:tasks.workunit.client.1.smithi192.stdout:9/985: truncate d0/d9/d44/d4f/f8e 2245586 0 2024-04-23T02:14:13.657 INFO:tasks.workunit.client.0.smithi071.stdout:5/937: mknod d3/d1d/daf/d10/da5/c157 0 2024-04-23T02:14:13.658 INFO:tasks.workunit.client.0.smithi071.stdout:9/828: truncate d1/fe 1925534 0 2024-04-23T02:14:13.658 INFO:tasks.workunit.client.1.smithi192.stdout:9/986: fdatasync d0/d9/dc7/da0/fc3 0 2024-04-23T02:14:13.659 INFO:tasks.workunit.client.0.smithi071.stdout:2/761: creat d0/d2/d26/d3a/dc3/d36/d5c/d85/de9/f100 x:0 0 0 2024-04-23T02:14:13.660 INFO:tasks.workunit.client.0.smithi071.stdout:0/904: creat d0/d3/de/d10/db3/f13f x:0 0 0 2024-04-23T02:14:13.661 INFO:tasks.workunit.client.1.smithi192.stdout:9/987: fdatasync d0/d9/d44/f45 0 2024-04-23T02:14:13.662 INFO:tasks.workunit.client.0.smithi071.stdout:5/938: chown d3/da/d1a/db8/d140/d4b/ffc 61948174 1 2024-04-23T02:14:13.664 INFO:tasks.workunit.client.0.smithi071.stdout:4/889: write d2/d2a/d2d/d39/d42/d49/d91/ddb/d10d/f11c [1142351,88518] 0 2024-04-23T02:14:13.667 INFO:tasks.workunit.client.0.smithi071.stdout:3/861: creat d7/da5/db2/f135 x:0 0 0 2024-04-23T02:14:13.667 INFO:tasks.workunit.client.1.smithi192.stdout:9/988: dread d0/d9/d44/d58/d89/dd8/fb6 [0,4194304] 0 2024-04-23T02:14:13.669 INFO:tasks.workunit.client.0.smithi071.stdout:6/845: dwrite d6/d22/f90 [0,4194304] 0 2024-04-23T02:14:13.670 INFO:tasks.workunit.client.1.smithi192.stdout:9/989: dread d0/d9/d44/f10d [0,4194304] 0 2024-04-23T02:14:13.671 INFO:tasks.workunit.client.0.smithi071.stdout:8/876: dread dc/d15/da0/f1c [0,4194304] 0 2024-04-23T02:14:13.672 INFO:tasks.workunit.client.0.smithi071.stdout:1/774: getdents db/d11/d7f 0 2024-04-23T02:14:13.673 INFO:tasks.workunit.client.1.smithi192.stdout:9/990: truncate d0/d9/dc7/da0/fc3 1647806 0 2024-04-23T02:14:13.676 INFO:tasks.workunit.client.0.smithi071.stdout:8/877: fdatasync dc/d15/d2d/f38 0 2024-04-23T02:14:13.676 INFO:tasks.workunit.client.0.smithi071.stdout:6/846: dread - d6/d22/fb9 zero size 2024-04-23T02:14:13.679 INFO:tasks.workunit.client.0.smithi071.stdout:1/775: fdatasync db/df/d13/d2f/d48/d9f/fa5 0 2024-04-23T02:14:13.680 INFO:tasks.workunit.client.1.smithi192.stdout:9/991: dwrite d0/d9/fb [4194304,4194304] 0 2024-04-23T02:14:13.682 INFO:tasks.workunit.client.0.smithi071.stdout:2/762: truncate d0/d2/d26/d14/f22 2334987 0 2024-04-23T02:14:13.683 INFO:tasks.workunit.client.1.smithi192.stdout:9/992: mknod d0/d9/d44/d58/dd9/c139 0 2024-04-23T02:14:13.686 INFO:tasks.workunit.client.0.smithi071.stdout:6/847: fsync d6/d1f/f32 0 2024-04-23T02:14:13.687 INFO:tasks.workunit.client.0.smithi071.stdout:5/939: dread d3/d1d/f3b [0,4194304] 0 2024-04-23T02:14:13.689 INFO:tasks.workunit.client.0.smithi071.stdout:9/829: dwrite d1/d32/d41/db7/d71/dbf/f120 [0,4194304] 0 2024-04-23T02:14:13.689 INFO:tasks.workunit.client.0.smithi071.stdout:1/776: fdatasync db/df/d5f/d57/f5e 0 2024-04-23T02:14:13.691 INFO:tasks.workunit.client.0.smithi071.stdout:3/862: link d7/dfd/lfe d7/d22/d23/dba/df4/dca/l136 0 2024-04-23T02:14:13.698 INFO:tasks.workunit.client.0.smithi071.stdout:0/905: dwrite d0/d3/de/d10/d14/d32/d8a/df0/fc8 [0,4194304] 0 2024-04-23T02:14:13.699 INFO:tasks.workunit.client.0.smithi071.stdout:4/890: link d2/d2a/da2/dd1/dde/d2c/c45 d2/d2a/d2d/d39/d42/d49/d67/d3e/c11d 0 2024-04-23T02:14:13.700 INFO:tasks.workunit.client.0.smithi071.stdout:3/863: read d7/da5/d128/d68/d3b/d4c/f63 [1814637,90984] 0 2024-04-23T02:14:13.701 INFO:tasks.workunit.client.0.smithi071.stdout:5/940: fsync d3/da/d1a/d1c/d46/d4e/fea 0 2024-04-23T02:14:13.704 INFO:tasks.workunit.client.0.smithi071.stdout:0/906: fdatasync d0/d3/de/d10/d14/d32/d8a/fb7 0 2024-04-23T02:14:13.704 INFO:tasks.workunit.client.1.smithi192.stdout:9/993: dread d0/d9/d44/d51/d5b/d61/f84 [0,4194304] 0 2024-04-23T02:14:13.705 INFO:tasks.workunit.client.0.smithi071.stdout:5/941: fsync d3/da/d1a/d6d/f132 0 2024-04-23T02:14:13.705 INFO:tasks.workunit.client.0.smithi071.stdout:1/777: unlink db/d11/d7f/da7/dd9/d100/f80 0 2024-04-23T02:14:13.707 INFO:tasks.workunit.client.0.smithi071.stdout:6/848: link d6/d104/dad/dd1/d10b/fc3 d6/d1f/df8/f10d 0 2024-04-23T02:14:13.708 INFO:tasks.workunit.client.0.smithi071.stdout:3/864: chown d7/d22/d23/dba/df4/dca/f82 138 1 2024-04-23T02:14:13.709 INFO:tasks.workunit.client.1.smithi192.stdout:9/994: creat d0/d9/dc7/da0/dd5/d109/d122/f13a x:0 0 0 2024-04-23T02:14:13.710 INFO:tasks.workunit.client.0.smithi071.stdout:8/878: dwrite dc/d15/da0/d30/f52 [0,4194304] 0 2024-04-23T02:14:13.713 INFO:tasks.workunit.client.1.smithi192.stdout:9/995: link d0/d9/d44/d53/c110 d0/d9/d44/d51/d5b/c13b 0 2024-04-23T02:14:13.716 INFO:tasks.workunit.client.0.smithi071.stdout:6/849: mknod d6/d104/dad/dd1/d10b/c10e 0 2024-04-23T02:14:13.718 INFO:tasks.workunit.client.0.smithi071.stdout:3/865: chown d7/da5/d128/d68/d3b/d4c/da1 1075899 1 2024-04-23T02:14:13.718 INFO:tasks.workunit.client.1.smithi192.stdout:9/996: rename d0/d9/db3/d5f/dfd/f127 to d0/d9/d44/d58/dd9/def/f13c 0 2024-04-23T02:14:13.719 INFO:tasks.workunit.client.0.smithi071.stdout:5/942: truncate d3/d34/f40 3362378 0 2024-04-23T02:14:13.719 INFO:tasks.workunit.client.0.smithi071.stdout:1/778: truncate db/df/d13/d2b/fab 294101 0 2024-04-23T02:14:13.721 INFO:tasks.workunit.client.0.smithi071.stdout:6/850: mknod d6/d17/d77/d105/c10f 0 2024-04-23T02:14:13.722 INFO:tasks.workunit.client.0.smithi071.stdout:9/830: dread d1/d32/d3a/d77/f88 [0,4194304] 0 2024-04-23T02:14:13.724 INFO:tasks.workunit.client.1.smithi192.stdout:9/997: dwrite d0/d9/dc7/fe1 [4194304,4194304] 0 2024-04-23T02:14:13.725 INFO:tasks.workunit.client.0.smithi071.stdout:2/763: dwrite d0/d2/d26/d3a/dc3/d36/d6a/f6e [0,4194304] 0 2024-04-23T02:14:13.727 INFO:tasks.workunit.client.0.smithi071.stdout:1/779: chown db/df/d3a/l5a 336103798 1 2024-04-23T02:14:13.727 INFO:tasks.workunit.client.0.smithi071.stdout:6/851: read - d6/d104/dad/d8f/de2/fe4 zero size 2024-04-23T02:14:13.728 INFO:tasks.workunit.client.1.smithi192.stdout:9/998: symlink d0/d9/d44/d51/l13d 0 2024-04-23T02:14:13.732 INFO:tasks.workunit.client.1.smithi192.stdout:9/999: mknod d0/dd2/c13e 0 2024-04-23T02:14:13.734 INFO:tasks.workunit.client.0.smithi071.stdout:5/943: rename d3/da/d1a/db8/d140/dd2/d5a/da7/f128 to d3/d8f/f158 0 2024-04-23T02:14:13.739 INFO:tasks.workunit.client.0.smithi071.stdout:4/891: dread d2/f89 [0,4194304] 0 2024-04-23T02:14:13.742 INFO:tasks.workunit.client.0.smithi071.stdout:5/944: creat d3/da/df6/f159 x:0 0 0 2024-04-23T02:14:13.744 INFO:tasks.workunit.client.0.smithi071.stdout:3/866: write d7/da5/d128/d68/d3b/fb8 [71670,130799] 0 2024-04-23T02:14:13.747 INFO:tasks.workunit.client.0.smithi071.stdout:0/907: dread d0/d3/de/d10/d14/d32/d8a/df0/f77 [0,4194304] 0 2024-04-23T02:14:13.747 INFO:tasks.workunit.client.0.smithi071.stdout:8/879: truncate dc/d15/d72/d95/fb8 1882056 0 2024-04-23T02:14:13.749 INFO:tasks.workunit.client.0.smithi071.stdout:5/945: dread d3/da/d1a/f1b [0,4194304] 0 2024-04-23T02:14:13.750 INFO:tasks.workunit.client.0.smithi071.stdout:3/867: creat d7/da5/db2/f137 x:0 0 0 2024-04-23T02:14:13.752 INFO:tasks.workunit.client.0.smithi071.stdout:2/764: write d0/d2/d26/d14/f42 [649656,26073] 0 2024-04-23T02:14:13.752 INFO:tasks.workunit.client.0.smithi071.stdout:9/831: link d1/d32/d41/db7/d5a/dc6/cd9 d1/d32/d7e/d114/c127 0 2024-04-23T02:14:13.753 INFO:tasks.workunit.client.0.smithi071.stdout:5/946: readlink d3/da/d1a/db8/d140/dd2/d5a/da7/lcc 0 2024-04-23T02:14:13.755 INFO:tasks.workunit.client.0.smithi071.stdout:4/892: write d2/d2a/f3b [830968,120820] 0 2024-04-23T02:14:13.756 INFO:tasks.workunit.client.0.smithi071.stdout:0/908: symlink d0/d3/de/d10/dfc/l140 0 2024-04-23T02:14:13.758 INFO:tasks.workunit.client.0.smithi071.stdout:3/868: mkdir d7/da5/d128/d68/d3b/d4c/da1/da8/d138 0 2024-04-23T02:14:13.758 INFO:tasks.workunit.client.0.smithi071.stdout:5/947: chown d3/d34/l6f 546723656 1 2024-04-23T02:14:13.760 INFO:tasks.workunit.client.0.smithi071.stdout:9/832: mknod d1/d32/d3a/d77/c128 0 2024-04-23T02:14:13.763 INFO:tasks.workunit.client.0.smithi071.stdout:3/869: write d7/da5/db2/fe6 [5131222,66797] 0 2024-04-23T02:14:13.763 INFO:tasks.workunit.client.0.smithi071.stdout:8/880: dwrite dc/d15/da0/d32/fb9 [0,4194304] 0 2024-04-23T02:14:13.766 INFO:tasks.workunit.client.0.smithi071.stdout:5/948: creat d3/d1d/daf/f15a x:0 0 0 2024-04-23T02:14:13.767 INFO:tasks.workunit.client.0.smithi071.stdout:4/893: chown d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/fd5 200315665 1 2024-04-23T02:14:13.768 INFO:tasks.workunit.client.0.smithi071.stdout:0/909: creat d0/d92/d127/f141 x:0 0 0 2024-04-23T02:14:13.769 INFO:tasks.workunit.client.0.smithi071.stdout:9/833: creat d1/d32/dc5/dd1/f129 x:0 0 0 2024-04-23T02:14:13.770 INFO:tasks.workunit.client.0.smithi071.stdout:1/780: dread db/df/d3a/f83 [0,4194304] 0 2024-04-23T02:14:13.773 INFO:tasks.workunit.client.0.smithi071.stdout:8/881: mkdir dc/d78/d11a 0 2024-04-23T02:14:13.776 INFO:tasks.workunit.client.0.smithi071.stdout:3/870: fdatasync d7/da5/d128/d68/d3b/d4c/d8b/db3/f7a 0 2024-04-23T02:14:13.779 INFO:tasks.workunit.client.0.smithi071.stdout:4/894: rmdir d2/d2a/d2d/d39/d42/d49/d67/d7e 39 2024-04-23T02:14:13.780 INFO:tasks.workunit.client.0.smithi071.stdout:2/765: dread d0/d2/d45/f5a [0,4194304] 0 2024-04-23T02:14:13.789 INFO:tasks.workunit.client.0.smithi071.stdout:0/910: link d0/l7 d0/d3/de/d10/db9/l142 0 2024-04-23T02:14:13.790 INFO:tasks.workunit.client.0.smithi071.stdout:3/871: dwrite d7/da5/d128/d68/d3b/d4c/d8b/db3/f129 [0,4194304] 0 2024-04-23T02:14:13.791 INFO:tasks.workunit.client.0.smithi071.stdout:0/911: truncate d0/d3/f62 2021288 0 2024-04-23T02:14:13.792 INFO:tasks.workunit.client.0.smithi071.stdout:1/781: write db/df/d13/d2b/f7e [698655,69305] 0 2024-04-23T02:14:13.795 INFO:tasks.workunit.client.0.smithi071.stdout:3/872: fsync d7/d22/da7/fa2 0 2024-04-23T02:14:13.800 INFO:tasks.workunit.client.1.smithi192.stdout:3/993: sync 2024-04-23T02:14:13.801 INFO:tasks.workunit.client.0.smithi071.stdout:4/895: rename d2/d2a/d73/d5e/c69 to d2/d2a/d73/c11e 0 2024-04-23T02:14:13.802 INFO:tasks.workunit.client.0.smithi071.stdout:9/834: dread d1/d32/d3a/f8a [0,4194304] 0 2024-04-23T02:14:13.803 INFO:tasks.workunit.client.1.smithi192.stdout:3/994: creat d1/d3/d6/d7/d29/d3b/da9/daf/ddf/f160 x:0 0 0 2024-04-23T02:14:13.804 INFO:tasks.workunit.client.0.smithi071.stdout:5/949: write d3/d1d/daf/d10/da5/feb [2011959,90479] 0 2024-04-23T02:14:13.805 INFO:tasks.workunit.client.0.smithi071.stdout:8/882: dwrite dc/d15/da0/d55/f80 [0,4194304] 0 2024-04-23T02:14:13.809 INFO:tasks.workunit.client.1.smithi192.stdout:3/995: read d1/d3/d6/d7/d29/d3b/da9/dbd/fc9 [477832,102850] 0 2024-04-23T02:14:13.810 INFO:tasks.workunit.client.1.smithi192.stdout:3/996: stat d1/d3/d17/d1a/d7c/dab 0 2024-04-23T02:14:13.810 INFO:tasks.workunit.client.0.smithi071.stdout:5/950: stat d3/d8f/de6/c100 0 2024-04-23T02:14:13.810 INFO:tasks.workunit.client.0.smithi071.stdout:2/766: rename d0/d2/d45/d7d to d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101 0 2024-04-23T02:14:13.810 INFO:tasks.workunit.client.1.smithi192.stdout:3/997: chown d1/d3/d6/d7/d29/d3b/da9/d8b/dfc 6 1 2024-04-23T02:14:13.812 INFO:tasks.workunit.client.1.smithi192.stdout:3/998: chown d1/d3/d6/d7/d29/d3b/da9/dbd/f15e 3504 1 2024-04-23T02:14:13.814 INFO:tasks.workunit.client.0.smithi071.stdout:1/782: write db/d11/d7f/faf [595690,31518] 0 2024-04-23T02:14:13.815 INFO:tasks.workunit.client.1.smithi192.stdout:3/999: creat d1/d3/d17/d65/d99/f161 x:0 0 0 2024-04-23T02:14:13.815 INFO:tasks.workunit.client.0.smithi071.stdout:8/883: symlink dc/d15/d50/d5f/l11b 0 2024-04-23T02:14:13.820 INFO:tasks.workunit.client.0.smithi071.stdout:0/912: rename d0/d3/de/d10/d14/d32/d44/d87/l108 to d0/d3/de/d10/d14/dd5/l143 0 2024-04-23T02:14:13.821 INFO:tasks.workunit.client.0.smithi071.stdout:4/896: dwrite d2/d2a/da2/dd1/dde/f6d [0,4194304] 0 2024-04-23T02:14:13.824 INFO:tasks.workunit.client.0.smithi071.stdout:1/783: unlink db/d11/d7f/faf 0 2024-04-23T02:14:13.828 INFO:tasks.workunit.client.0.smithi071.stdout:9/835: dwrite d1/d32/d41/db7/d71/dbf/f120 [0,4194304] 0 2024-04-23T02:14:13.828 INFO:tasks.workunit.client.0.smithi071.stdout:8/884: stat dc/d15/da0/f85 0 2024-04-23T02:14:13.833 INFO:tasks.workunit.client.0.smithi071.stdout:9/836: write d1/f28 [172193,28405] 0 2024-04-23T02:14:13.833 INFO:tasks.workunit.client.0.smithi071.stdout:3/873: dread d7/d22/da7/f9b [0,4194304] 0 2024-04-23T02:14:13.834 INFO:tasks.workunit.client.0.smithi071.stdout:2/767: write d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/d80/f5f [1357151,85953] 0 2024-04-23T02:14:13.838 INFO:tasks.workunit.client.0.smithi071.stdout:4/897: truncate d2/d2a/da2/dd1/dde/d23/d29/fd3 1223651 0 2024-04-23T02:14:13.839 INFO:tasks.workunit.client.0.smithi071.stdout:1/784: rename db/df/d13/d2f/d48/d9f/fa5 to db/df/d5f/df0/f109 0 2024-04-23T02:14:13.840 INFO:tasks.workunit.client.0.smithi071.stdout:9/837: mknod d1/d4e/d66/da4/dd7/c12a 0 2024-04-23T02:14:13.843 INFO:tasks.workunit.client.0.smithi071.stdout:2/768: fdatasync d0/d2/d26/d3a/dc3/f25 0 2024-04-23T02:14:13.843 INFO:tasks.workunit.client.0.smithi071.stdout:4/898: symlink d2/d53/l11f 0 2024-04-23T02:14:13.846 INFO:tasks.workunit.client.0.smithi071.stdout:8/885: mknod dc/d15/d50/d5f/d102/c11c 0 2024-04-23T02:14:13.848 INFO:tasks.workunit.client.0.smithi071.stdout:0/913: getdents d0/d3/de/d10/db9/d106 0 2024-04-23T02:14:13.849 INFO:tasks.workunit.client.0.smithi071.stdout:3/874: dread d7/da5/d128/d68/d3b/d4c/d8b/db3/f7a [0,4194304] 0 2024-04-23T02:14:13.849 INFO:tasks.workunit.client.0.smithi071.stdout:9/838: mknod d1/d32/d41/db7/d5a/de3/c12b 0 2024-04-23T02:14:13.850 INFO:tasks.workunit.client.0.smithi071.stdout:1/785: dwrite db/df/d5f/d57/f89 [0,4194304] 0 2024-04-23T02:14:13.852 INFO:tasks.workunit.client.0.smithi071.stdout:2/769: mknod d0/d2/d26/d3a/c102 0 2024-04-23T02:14:13.853 INFO:tasks.workunit.client.0.smithi071.stdout:8/886: symlink dc/d15/da0/d8d/def/l11d 0 2024-04-23T02:14:13.853 INFO:tasks.workunit.client.0.smithi071.stdout:5/951: dread d3/da/d1a/db8/d140/d4b/fe0 [0,4194304] 0 2024-04-23T02:14:13.857 INFO:tasks.workunit.client.0.smithi071.stdout:2/770: dread d0/d2/d26/d73/f9b [0,4194304] 0 2024-04-23T02:14:13.859 INFO:tasks.workunit.client.0.smithi071.stdout:0/914: creat d0/d3/d95/d114/f144 x:0 0 0 2024-04-23T02:14:13.861 INFO:tasks.workunit.client.0.smithi071.stdout:0/915: dread - d0/d3/de/d48/d7e/d8b/f11b zero size 2024-04-23T02:14:13.864 INFO:tasks.workunit.client.0.smithi071.stdout:9/839: mkdir d1/d32/d41/db7/d5a/dc6/d81/d12c 0 2024-04-23T02:14:13.865 INFO:tasks.workunit.client.0.smithi071.stdout:5/952: write d3/f1f [584326,22408] 0 2024-04-23T02:14:13.868 INFO:tasks.workunit.client.0.smithi071.stdout:8/887: creat dc/d15/d50/d5f/f11e x:0 0 0 2024-04-23T02:14:13.868 INFO:tasks.workunit.client.0.smithi071.stdout:2/771: unlink d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/fa2 0 2024-04-23T02:14:13.869 INFO:tasks.workunit.client.1.smithi192.stdout:1/865: sync 2024-04-23T02:14:13.869 INFO:tasks.workunit.client.1.smithi192.stdout:5/997: sync 2024-04-23T02:14:13.869 INFO:tasks.workunit.client.0.smithi071.stdout:0/916: mknod d0/d3/de/d48/d7e/d8b/de3/c145 0 2024-04-23T02:14:13.871 INFO:tasks.workunit.client.1.smithi192.stdout:5/998: symlink d2/d6/db4/db6/l14d 0 2024-04-23T02:14:13.875 INFO:tasks.workunit.client.1.smithi192.stdout:1/866: rename dc/d14/d15/c20 to dc/d8b/d62/c114 0 2024-04-23T02:14:13.876 INFO:tasks.workunit.client.0.smithi071.stdout:3/875: write d7/da5/d128/d68/d3b/f76 [1196427,114485] 0 2024-04-23T02:14:13.878 INFO:tasks.workunit.client.0.smithi071.stdout:3/876: write d7/d22/d23/dba/df4/d88/f57 [6030207,129580] 0 2024-04-23T02:14:13.879 INFO:tasks.workunit.client.1.smithi192.stdout:1/867: unlink dc/d16/d19/d74/dd2/cd1 0 2024-04-23T02:14:13.881 INFO:tasks.workunit.client.0.smithi071.stdout:2/772: mknod d0/d2/d26/d3a/dc3/d36/d6d/c103 0 2024-04-23T02:14:13.882 INFO:tasks.workunit.client.0.smithi071.stdout:5/953: creat d3/d86/f15b x:0 0 0 2024-04-23T02:14:13.882 INFO:tasks.workunit.client.0.smithi071.stdout:4/899: rename d2/d2a/d2d/d39/d42/d49/d67/d7e/dae/dab to d2/d2a/d2d/d39/d42/d120 0 2024-04-23T02:14:13.883 INFO:tasks.workunit.client.1.smithi192.stdout:5/999: mknod d2/db/d2b/dc8/d103/d136/c14e 0 2024-04-23T02:14:13.883 INFO:tasks.workunit.client.0.smithi071.stdout:0/917: creat d0/d3/de/d10/d14/d32/d44/d59/f146 x:0 0 0 2024-04-23T02:14:13.886 INFO:tasks.workunit.client.0.smithi071.stdout:3/877: write d7/d8/f16 [5494093,70072] 0 2024-04-23T02:14:13.886 INFO:tasks.workunit.client.0.smithi071.stdout:5/954: write d3/d1d/daf/d10/da5/f12c [233131,41133] 0 2024-04-23T02:14:13.891 INFO:tasks.workunit.client.0.smithi071.stdout:3/878: chown d7/d22/ce9 974092 1 2024-04-23T02:14:13.891 INFO:tasks.workunit.client.0.smithi071.stdout:9/840: dread d1/fc [0,4194304] 0 2024-04-23T02:14:13.892 INFO:tasks.workunit.client.0.smithi071.stdout:8/888: write dc/d15/da0/f1e [32873,109888] 0 2024-04-23T02:14:13.893 INFO:tasks.workunit.client.0.smithi071.stdout:0/918: symlink d0/d3/de/d10/d14/d32/d8a/l147 0 2024-04-23T02:14:13.894 INFO:tasks.workunit.client.1.smithi192.stdout:1/868: dread dc/d16/d19/d30/f3f [4194304,4194304] 0 2024-04-23T02:14:13.894 INFO:tasks.workunit.client.0.smithi071.stdout:2/773: creat d0/d2/d26/d3a/dc3/d36/d5c/d85/de9/dfc/f104 x:0 0 0 2024-04-23T02:14:13.895 INFO:tasks.workunit.client.1.smithi192.stdout:1/869: chown dc/d8b/d34/d70/lef 13 1 2024-04-23T02:14:13.897 INFO:tasks.workunit.client.1.smithi192.stdout:1/870: write dc/d14/d15/f54 [1581372,35506] 0 2024-04-23T02:14:13.897 INFO:tasks.workunit.client.0.smithi071.stdout:3/879: read - d7/d22/d23/dba/df4/dca/f82 zero size 2024-04-23T02:14:13.898 INFO:tasks.workunit.client.0.smithi071.stdout:9/841: symlink d1/d4e/d54/d75/l12d 0 2024-04-23T02:14:13.899 INFO:tasks.workunit.client.0.smithi071.stdout:5/955: dread d3/d1d/daf/d10/da5/d5f/fd6 [0,4194304] 0 2024-04-23T02:14:13.902 INFO:tasks.workunit.client.1.smithi192.stdout:1/871: dread dc/d14/d23/da2/fcb [0,4194304] 0 2024-04-23T02:14:13.904 INFO:tasks.workunit.client.1.smithi192.stdout:1/872: read dc/d16/d19/f10d [1303290,18631] 0 2024-04-23T02:14:13.907 INFO:tasks.workunit.client.1.smithi192.stdout:1/873: link dc/c28 dc/d16/dde/de9/d112/c115 0 2024-04-23T02:14:13.908 INFO:tasks.workunit.client.0.smithi071.stdout:0/919: rename d0/d3/de/d10/d14/d32/d44/d31/d52 to d0/ddf/d148 0 2024-04-23T02:14:13.909 INFO:tasks.workunit.client.1.smithi192.stdout:1/874: write dc/d14/d15/d35/f40 [1896667,46378] 0 2024-04-23T02:14:13.910 INFO:tasks.workunit.client.0.smithi071.stdout:5/956: creat d3/da/df6/d153/f15c x:0 0 0 2024-04-23T02:14:13.910 INFO:tasks.workunit.client.1.smithi192.stdout:1/875: write dc/d14/f9d [456548,64596] 0 2024-04-23T02:14:13.910 INFO:tasks.workunit.client.0.smithi071.stdout:4/900: dwrite d2/d2a/da2/dd1/dde/d23/d29/f9e [4194304,4194304] 0 2024-04-23T02:14:13.913 INFO:tasks.workunit.client.1.smithi192.stdout:1/876: creat dc/d8b/d34/d70/d8c/da3/dc1/f116 x:0 0 0 2024-04-23T02:14:13.914 INFO:tasks.workunit.client.0.smithi071.stdout:9/842: mknod d1/d32/d41/db7/d5a/de3/c12e 0 2024-04-23T02:14:13.914 INFO:tasks.workunit.client.1.smithi192.stdout:1/877: rmdir dc 39 2024-04-23T02:14:13.918 INFO:tasks.workunit.client.1.smithi192.stdout:1/878: unlink dc/d8b/d44/da9/l100 0 2024-04-23T02:14:13.919 INFO:tasks.workunit.client.0.smithi071.stdout:3/880: rename d7/d22/d23/dba/df4/dca/f82 to d7/d22/d23/d26/d52/d12f/f139 0 2024-04-23T02:14:13.919 INFO:tasks.workunit.client.0.smithi071.stdout:0/920: creat d0/d3/de/d48/d7e/d8b/de3/f149 x:0 0 0 2024-04-23T02:14:13.920 INFO:tasks.workunit.client.0.smithi071.stdout:2/774: dwrite d0/d2/d26/d3a/d75/fd1 [4194304,4194304] 0 2024-04-23T02:14:13.920 INFO:tasks.workunit.client.0.smithi071.stdout:8/889: dwrite dc/d15/d50/d5f/f108 [0,4194304] 0 2024-04-23T02:14:13.923 INFO:tasks.workunit.client.1.smithi192.stdout:1/879: rename dc/d14/d23/d5e/cb1 to dc/d16/d19/d74/c117 0 2024-04-23T02:14:13.926 INFO:tasks.workunit.client.0.smithi071.stdout:8/890: write dc/d15/d72/ff9 [212987,76571] 0 2024-04-23T02:14:13.927 INFO:tasks.workunit.client.0.smithi071.stdout:5/957: dread d3/da/d1a/d6d/fbe [0,4194304] 0 2024-04-23T02:14:13.927 INFO:tasks.workunit.client.0.smithi071.stdout:0/921: creat d0/d3/de/d10/dfc/f14a x:0 0 0 2024-04-23T02:14:13.930 INFO:tasks.workunit.client.0.smithi071.stdout:5/958: chown d3/d86/d124 11167 1 2024-04-23T02:14:13.935 INFO:tasks.workunit.client.0.smithi071.stdout:0/922: creat d0/d3/de/d48/d120/f14b x:0 0 0 2024-04-23T02:14:13.937 INFO:tasks.workunit.client.0.smithi071.stdout:5/959: dread - d3/d1d/daf/d10/da5/f94 zero size 2024-04-23T02:14:13.939 INFO:tasks.workunit.client.0.smithi071.stdout:0/923: chown d0/fda 19 1 2024-04-23T02:14:13.940 INFO:tasks.workunit.client.0.smithi071.stdout:3/881: dwrite d7/d22/d23/dba/df4/d88/f57 [8388608,4194304] 0 2024-04-23T02:14:13.943 INFO:tasks.workunit.client.0.smithi071.stdout:9/843: dwrite d1/d32/d41/db7/f70 [0,4194304] 0 2024-04-23T02:14:13.944 INFO:tasks.workunit.client.0.smithi071.stdout:2/775: dwrite d0/d2/d26/d3a/f81 [0,4194304] 0 2024-04-23T02:14:13.947 INFO:tasks.workunit.client.0.smithi071.stdout:0/924: mkdir d0/d3/de/d10/d14/d32/da8/d14c 0 2024-04-23T02:14:13.947 INFO:tasks.workunit.client.0.smithi071.stdout:3/882: write d7/da5/d100/f10d [2500461,87547] 0 2024-04-23T02:14:13.949 INFO:tasks.workunit.client.0.smithi071.stdout:9/844: creat d1/d32/dc5/f12f x:0 0 0 2024-04-23T02:14:13.950 INFO:tasks.workunit.client.0.smithi071.stdout:2/776: fdatasync d0/d2/fb 0 2024-04-23T02:14:13.954 INFO:tasks.workunit.client.0.smithi071.stdout:0/925: creat d0/d76/f14d x:0 0 0 2024-04-23T02:14:13.955 INFO:tasks.workunit.client.0.smithi071.stdout:3/883: rename d7/d22/d23/dba/df4/d88/f57 to d7/da5/d128/d68/d3b/f13a 0 2024-04-23T02:14:13.960 INFO:tasks.workunit.client.0.smithi071.stdout:6/852: sync 2024-04-23T02:14:13.962 INFO:tasks.workunit.client.0.smithi071.stdout:5/960: dread d3/da/d1a/d1c/d46/d4e/dd8/ff7 [0,4194304] 0 2024-04-23T02:14:13.964 INFO:tasks.workunit.client.0.smithi071.stdout:0/926: mkdir d0/d3/de/d10/d14/d32/d44/d31/d14e 0 2024-04-23T02:14:13.964 INFO:tasks.workunit.client.0.smithi071.stdout:5/961: stat d3/da/d1a/f80 0 2024-04-23T02:14:13.965 INFO:tasks.workunit.client.0.smithi071.stdout:9/845: dwrite d1/d32/dc5/f11b [0,4194304] 0 2024-04-23T02:14:13.965 INFO:tasks.workunit.client.0.smithi071.stdout:2/777: truncate d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/fdb 392348 0 2024-04-23T02:14:13.969 INFO:tasks.workunit.client.0.smithi071.stdout:6/853: truncate d6/d17/d77/d85/fa1 4690290 0 2024-04-23T02:14:13.969 INFO:tasks.workunit.client.0.smithi071.stdout:5/962: stat d3/da/d1a/db8/d140/fcb 0 2024-04-23T02:14:13.975 INFO:tasks.workunit.client.0.smithi071.stdout:5/963: rmdir d3/da/d1a 39 2024-04-23T02:14:13.979 INFO:tasks.workunit.client.0.smithi071.stdout:6/854: link d6/f106 d6/d104/dd8/f110 0 2024-04-23T02:14:13.979 INFO:tasks.workunit.client.0.smithi071.stdout:2/778: dwrite d0/d2/d26/d14/f23 [4194304,4194304] 0 2024-04-23T02:14:13.983 INFO:tasks.workunit.client.0.smithi071.stdout:0/927: dwrite d0/d3/de/d10/d14/d32/d8a/df0/f5e [0,4194304] 0 2024-04-23T02:14:13.984 INFO:tasks.workunit.client.0.smithi071.stdout:9/846: getdents d1/d4e/d66/da4 0 2024-04-23T02:14:13.985 INFO:tasks.workunit.client.0.smithi071.stdout:9/847: chown d1/d4e 103 1 2024-04-23T02:14:13.990 INFO:tasks.workunit.client.0.smithi071.stdout:5/964: getdents d3/d34 0 2024-04-23T02:14:13.991 INFO:tasks.workunit.client.0.smithi071.stdout:5/965: stat d3/d1d/daf 0 2024-04-23T02:14:13.992 INFO:tasks.workunit.client.0.smithi071.stdout:6/855: truncate d6/d104/dad/f8a 1757472 0 2024-04-23T02:14:13.995 INFO:tasks.workunit.client.0.smithi071.stdout:0/928: mknod d0/d3/de/d10/d14/d32/da8/c14f 0 2024-04-23T02:14:13.998 INFO:tasks.workunit.client.0.smithi071.stdout:2/779: write d0/d2/d26/d3a/dc3/d36/d6d/faf [1000652,50235] 0 2024-04-23T02:14:13.999 INFO:tasks.workunit.client.0.smithi071.stdout:3/884: dwrite d7/d8/f61 [0,4194304] 0 2024-04-23T02:14:14.007 INFO:tasks.workunit.client.0.smithi071.stdout:9/848: dwrite d1/d32/dc5/dd1/f113 [0,4194304] 0 2024-04-23T02:14:14.009 INFO:tasks.workunit.client.0.smithi071.stdout:9/849: write d1/ff1 [1020712,12597] 0 2024-04-23T02:14:14.011 INFO:tasks.workunit.client.0.smithi071.stdout:5/966: readlink d3/da/d1a/db8/d140/dd2/d5a/l9f 0 2024-04-23T02:14:14.012 INFO:tasks.workunit.client.0.smithi071.stdout:3/885: mkdir d7/da5/d128/d68/d3b/d4c/d8b/d13b 0 2024-04-23T02:14:14.014 INFO:tasks.workunit.client.0.smithi071.stdout:6/856: write d6/d17/d77/da2/fc6 [631869,90887] 0 2024-04-23T02:14:14.015 INFO:tasks.workunit.client.0.smithi071.stdout:9/850: stat d1/d32/dc5/dd8/c10a 0 2024-04-23T02:14:14.017 INFO:tasks.workunit.client.0.smithi071.stdout:3/886: mknod d7/d86/c13c 0 2024-04-23T02:14:14.022 INFO:tasks.workunit.client.0.smithi071.stdout:9/851: fdatasync d1/fb 0 2024-04-23T02:14:14.029 INFO:tasks.workunit.client.0.smithi071.stdout:5/967: rename d3/d8f/de6 to d3/da/d1a/db8/d140/dd2/da9/d15d 0 2024-04-23T02:14:14.029 INFO:tasks.workunit.client.0.smithi071.stdout:0/929: dwrite d0/d3/de/d10/d14/d32/d37/d26/f2e [0,4194304] 0 2024-04-23T02:14:14.030 INFO:tasks.workunit.client.0.smithi071.stdout:9/852: creat d1/d32/db9/f130 x:0 0 0 2024-04-23T02:14:14.032 INFO:tasks.workunit.client.0.smithi071.stdout:5/968: unlink d3/d1d/c10e 0 2024-04-23T02:14:14.035 INFO:tasks.workunit.client.0.smithi071.stdout:6/857: dwrite d6/d17/d77/d85/fab [0,4194304] 0 2024-04-23T02:14:14.040 INFO:tasks.workunit.client.0.smithi071.stdout:5/969: read - d3/da/d1a/fe2 zero size 2024-04-23T02:14:14.041 INFO:tasks.workunit.client.0.smithi071.stdout:3/887: rename d7/da5/d128/d68/dae/dc6/c109 to d7/d8/dea/c13d 0 2024-04-23T02:14:14.046 INFO:tasks.workunit.client.1.smithi192.stdout:1/880: sync 2024-04-23T02:14:14.046 INFO:tasks.workunit.client.0.smithi071.stdout:5/970: mknod d3/da/d1a/db8/d140/dd2/da9/d15d/c15e 0 2024-04-23T02:14:14.046 INFO:tasks.workunit.client.0.smithi071.stdout:3/888: creat d7/da5/f13e x:0 0 0 2024-04-23T02:14:14.047 INFO:tasks.workunit.client.0.smithi071.stdout:5/971: truncate d3/d10f/d13a/f142 422705 0 2024-04-23T02:14:14.050 INFO:tasks.workunit.client.0.smithi071.stdout:3/889: symlink d7/d22/d23/d26/d52/l13f 0 2024-04-23T02:14:14.051 INFO:tasks.workunit.client.0.smithi071.stdout:5/972: creat d3/d1d/daf/d10/da5/f15f x:0 0 0 2024-04-23T02:14:14.052 INFO:tasks.workunit.client.1.smithi192.stdout:1/881: dread dc/d14/d15/ff2 [0,4194304] 0 2024-04-23T02:14:14.053 INFO:tasks.workunit.client.1.smithi192.stdout:1/882: fdatasync dc/d16/fd6 0 2024-04-23T02:14:14.053 INFO:tasks.workunit.client.0.smithi071.stdout:5/973: readlink d3/d1d/daf/d10/da5/d5f/dc0/le5 0 2024-04-23T02:14:14.053 INFO:tasks.workunit.client.0.smithi071.stdout:3/890: mkdir d7/da5/d128/d140 0 2024-04-23T02:14:14.055 INFO:tasks.workunit.client.1.smithi192.stdout:1/883: mkdir dc/d16/d19/dae/d118 0 2024-04-23T02:14:14.056 INFO:tasks.workunit.client.1.smithi192.stdout:1/884: dread - dc/d14/fee zero size 2024-04-23T02:14:14.058 INFO:tasks.workunit.client.0.smithi071.stdout:3/891: creat d7/da5/d128/d68/d3b/d4c/d8b/db3/f141 x:0 0 0 2024-04-23T02:14:14.060 INFO:tasks.workunit.client.0.smithi071.stdout:3/892: readlink d7/d8/l51 0 2024-04-23T02:14:14.063 INFO:tasks.workunit.client.0.smithi071.stdout:5/974: write d3/d34/d74/faa [1148343,65029] 0 2024-04-23T02:14:14.065 INFO:tasks.workunit.client.0.smithi071.stdout:3/893: dread d7/da5/d128/d68/d3b/d4c/d8b/db3/f7a [0,4194304] 0 2024-04-23T02:14:14.065 INFO:tasks.workunit.client.0.smithi071.stdout:9/853: dread d1/d32/d41/db7/d39/f8f [0,4194304] 0 2024-04-23T02:14:14.068 INFO:tasks.workunit.client.0.smithi071.stdout:6/858: dread d6/d17/fdf [0,4194304] 0 2024-04-23T02:14:14.068 INFO:tasks.workunit.client.0.smithi071.stdout:3/894: mkdir d7/d22/d23/dba/df4/dca/d142 0 2024-04-23T02:14:14.069 INFO:tasks.workunit.client.1.smithi192.stdout:1/885: dread dc/d14/d23/d5e/dab/fd3 [0,4194304] 0 2024-04-23T02:14:14.071 INFO:tasks.workunit.client.0.smithi071.stdout:6/859: write d6/f34 [2582557,120801] 0 2024-04-23T02:14:14.071 INFO:tasks.workunit.client.0.smithi071.stdout:0/930: dread d0/f20 [0,4194304] 0 2024-04-23T02:14:14.071 INFO:tasks.workunit.client.1.smithi192.stdout:1/886: write dc/d8b/d62/f94 [4513983,54564] 0 2024-04-23T02:14:14.073 INFO:tasks.workunit.client.1.smithi192.stdout:1/887: creat dc/d80/d81/f119 x:0 0 0 2024-04-23T02:14:14.074 INFO:tasks.workunit.client.0.smithi071.stdout:9/854: link d1/d32/d41/db7/d5a/dc6/d81/db6/f101 d1/d32/d3a/d77/f131 0 2024-04-23T02:14:14.075 INFO:tasks.workunit.client.0.smithi071.stdout:3/895: rmdir d7/da5/d128/d140 0 2024-04-23T02:14:14.076 INFO:tasks.workunit.client.1.smithi192.stdout:1/888: read dc/d8b/fad [2057525,86943] 0 2024-04-23T02:14:14.077 INFO:tasks.workunit.client.1.smithi192.stdout:1/889: dread - dc/d80/fa8 zero size 2024-04-23T02:14:14.077 INFO:tasks.workunit.client.0.smithi071.stdout:6/860: rmdir d6/d17/d77/da2 39 2024-04-23T02:14:14.077 INFO:tasks.workunit.client.0.smithi071.stdout:9/855: readlink d1/d32/d7e/ld0 0 2024-04-23T02:14:14.078 INFO:tasks.workunit.client.0.smithi071.stdout:0/931: unlink d0/d3/d95/d114/cf3 0 2024-04-23T02:14:14.078 INFO:tasks.workunit.client.0.smithi071.stdout:5/975: dwrite d3/da/d1a/d6d/fba [4194304,4194304] 0 2024-04-23T02:14:14.080 INFO:tasks.workunit.client.0.smithi071.stdout:3/896: write d7/da5/d128/f40 [10232440,118896] 0 2024-04-23T02:14:14.085 INFO:tasks.workunit.client.0.smithi071.stdout:0/932: chown d0/d3/de/d48/l6d 17782 1 2024-04-23T02:14:14.087 INFO:tasks.workunit.client.0.smithi071.stdout:5/976: creat d3/da/d1a/db8/d140/dd2/f160 x:0 0 0 2024-04-23T02:14:14.088 INFO:tasks.workunit.client.0.smithi071.stdout:6/861: dread d6/f3c [0,4194304] 0 2024-04-23T02:14:14.088 INFO:tasks.workunit.client.0.smithi071.stdout:3/897: creat d7/da5/d128/d68/d3b/d98/d11a/f143 x:0 0 0 2024-04-23T02:14:14.092 INFO:tasks.workunit.client.0.smithi071.stdout:3/898: mkdir d7/da5/db5/d144 0 2024-04-23T02:14:14.093 INFO:tasks.workunit.client.0.smithi071.stdout:9/856: dwrite d1/db8/f11a [0,4194304] 0 2024-04-23T02:14:14.095 INFO:tasks.workunit.client.0.smithi071.stdout:3/899: write d7/da5/d128/d68/f119 [303218,16332] 0 2024-04-23T02:14:14.095 INFO:tasks.workunit.client.0.smithi071.stdout:9/857: readlink d1/d32/d41/db7/d39/lfa 0 2024-04-23T02:14:14.095 INFO:tasks.workunit.client.0.smithi071.stdout:5/977: unlink d3/d1d/c12a 0 2024-04-23T02:14:14.097 INFO:tasks.workunit.client.0.smithi071.stdout:0/933: creat d0/d3/de/d10/db3/f150 x:0 0 0 2024-04-23T02:14:14.097 INFO:tasks.workunit.client.0.smithi071.stdout:3/900: write d7/d22/d23/d26/f78 [2480151,24471] 0 2024-04-23T02:14:14.099 INFO:tasks.workunit.client.0.smithi071.stdout:6/862: creat d6/d104/dad/f111 x:0 0 0 2024-04-23T02:14:14.100 INFO:tasks.workunit.client.0.smithi071.stdout:9/858: symlink d1/d32/d41/db7/d5a/dc6/d9f/ded/l132 0 2024-04-23T02:14:14.101 INFO:tasks.workunit.client.0.smithi071.stdout:5/978: symlink d3/da/d1a/db8/d140/dd2/da9/d15d/d143/dff/l161 0 2024-04-23T02:14:14.105 INFO:tasks.workunit.client.0.smithi071.stdout:6/863: symlink d6/d1f/l112 0 2024-04-23T02:14:14.108 INFO:tasks.workunit.client.0.smithi071.stdout:6/864: creat d6/d17/d77/d85/dfb/f113 x:0 0 0 2024-04-23T02:14:14.111 INFO:tasks.workunit.client.0.smithi071.stdout:9/859: link d1/c29 d1/d32/d41/c133 0 2024-04-23T02:14:14.113 INFO:tasks.workunit.client.0.smithi071.stdout:3/901: truncate f0 1339501 0 2024-04-23T02:14:14.115 INFO:tasks.workunit.client.0.smithi071.stdout:9/860: read d1/d32/d41/db7/d39/d42/f108 [2780127,27143] 0 2024-04-23T02:14:14.115 INFO:tasks.workunit.client.0.smithi071.stdout:9/861: readlink d1/l7 0 2024-04-23T02:14:14.120 INFO:tasks.workunit.client.0.smithi071.stdout:9/862: mkdir d1/d32/d41/d134 0 2024-04-23T02:14:14.121 INFO:tasks.workunit.client.0.smithi071.stdout:0/934: dread d0/d3/de/d10/d14/d32/d8a/fb7 [0,4194304] 0 2024-04-23T02:14:14.124 INFO:tasks.workunit.client.0.smithi071.stdout:5/979: dwrite d3/d1d/daf/d10/da5/d5f/dc0/f105 [0,4194304] 0 2024-04-23T02:14:14.126 INFO:tasks.workunit.client.0.smithi071.stdout:9/863: creat d1/d32/d41/db7/d5a/f135 x:0 0 0 2024-04-23T02:14:14.126 INFO:tasks.workunit.client.0.smithi071.stdout:6/865: dwrite d6/d1f/d46/f5a [0,4194304] 0 2024-04-23T02:14:14.127 INFO:tasks.workunit.client.0.smithi071.stdout:3/902: write d7/d22/d23/d26/d7e/fd9 [357500,99624] 0 2024-04-23T02:14:14.128 INFO:tasks.workunit.client.0.smithi071.stdout:0/935: write d0/d76/f98 [1006515,91146] 0 2024-04-23T02:14:14.128 INFO:tasks.workunit.client.0.smithi071.stdout:5/980: symlink d3/d34/d6b/de4/l162 0 2024-04-23T02:14:14.130 INFO:tasks.workunit.client.0.smithi071.stdout:6/866: fdatasync d6/d17/d77/d85/dfb/f113 0 2024-04-23T02:14:14.131 INFO:tasks.workunit.client.0.smithi071.stdout:0/936: symlink d0/d63/l151 0 2024-04-23T02:14:14.131 INFO:tasks.workunit.client.0.smithi071.stdout:9/864: write d1/d32/d41/db7/d5a/dc6/faa [316853,56624] 0 2024-04-23T02:14:14.132 INFO:tasks.workunit.client.0.smithi071.stdout:3/903: mkdir d7/da5/d128/d68/d3b/d4c/da1/da8/d145 0 2024-04-23T02:14:14.134 INFO:tasks.workunit.client.0.smithi071.stdout:5/981: mkdir d3/da/d1a/d1c/db6/d152/d163 0 2024-04-23T02:14:14.136 INFO:tasks.workunit.client.0.smithi071.stdout:0/937: creat d0/d3/de/d10/d14/dd5/f152 x:0 0 0 2024-04-23T02:14:14.136 INFO:tasks.workunit.client.0.smithi071.stdout:0/938: readlink d0/d3/de/d10/dfc/l140 0 2024-04-23T02:14:14.137 INFO:tasks.workunit.client.0.smithi071.stdout:6/867: creat d6/d104/dad/f114 x:0 0 0 2024-04-23T02:14:14.138 INFO:tasks.workunit.client.0.smithi071.stdout:9/865: creat d1/d32/d3a/d77/df8/f136 x:0 0 0 2024-04-23T02:14:14.139 INFO:tasks.workunit.client.0.smithi071.stdout:5/982: symlink d3/d1d/l164 0 2024-04-23T02:14:14.140 INFO:tasks.workunit.client.0.smithi071.stdout:5/983: readlink d3/da/d1a/db8/d140/dd2/d5a/da7/lcc 0 2024-04-23T02:14:14.141 INFO:tasks.workunit.client.0.smithi071.stdout:3/904: truncate d7/da5/d128/d68/d3b/d4c/f59 4019467 0 2024-04-23T02:14:14.145 INFO:tasks.workunit.client.0.smithi071.stdout:5/984: stat d3/da/d1a/db8/d140/dd2/da9/d15d/d143/dff/d156 0 2024-04-23T02:14:14.147 INFO:tasks.workunit.client.0.smithi071.stdout:9/866: dwrite d1/d32/d41/db7/d5a/f135 [0,4194304] 0 2024-04-23T02:14:14.148 INFO:tasks.workunit.client.0.smithi071.stdout:9/867: readlink d1/d32/d41/db7/d5a/le7 0 2024-04-23T02:14:14.148 INFO:tasks.workunit.client.0.smithi071.stdout:0/939: getdents d0/d3/de/d10/dfc 0 2024-04-23T02:14:14.150 INFO:tasks.workunit.client.0.smithi071.stdout:9/868: chown d1/d32/d41/db7/d39/fa5 95 1 2024-04-23T02:14:14.150 INFO:tasks.workunit.client.0.smithi071.stdout:5/985: symlink d3/d1d/daf/d10/l165 0 2024-04-23T02:14:14.153 INFO:tasks.workunit.client.0.smithi071.stdout:5/986: symlink d3/d1d/db9/l166 0 2024-04-23T02:14:14.155 INFO:tasks.workunit.client.0.smithi071.stdout:9/869: fsync d1/d32/d41/db7/d5a/dc6/fbc 0 2024-04-23T02:14:14.155 INFO:tasks.workunit.client.0.smithi071.stdout:6/868: write d6/f94 [3322093,96320] 0 2024-04-23T02:14:14.158 INFO:tasks.workunit.client.0.smithi071.stdout:3/905: truncate d7/d10/f15 434570 0 2024-04-23T02:14:14.160 INFO:tasks.workunit.client.0.smithi071.stdout:9/870: symlink d1/d32/d41/db7/d71/l137 0 2024-04-23T02:14:14.162 INFO:tasks.workunit.client.0.smithi071.stdout:3/906: creat d7/d8/d42/f146 x:0 0 0 2024-04-23T02:14:14.163 INFO:tasks.workunit.client.0.smithi071.stdout:9/871: creat d1/d32/d3a/d94/f138 x:0 0 0 2024-04-23T02:14:14.164 INFO:tasks.workunit.client.0.smithi071.stdout:6/869: link d6/d17/d77/d85/l8b d6/d17/d77/d85/l115 0 2024-04-23T02:14:14.165 INFO:tasks.workunit.client.0.smithi071.stdout:3/907: symlink d7/da5/db2/l147 0 2024-04-23T02:14:14.167 INFO:tasks.workunit.client.0.smithi071.stdout:5/987: dwrite d3/da/d1a/d1c/f2e [0,4194304] 0 2024-04-23T02:14:14.168 INFO:tasks.workunit.client.0.smithi071.stdout:6/870: unlink d6/d104/dd8/df2/fcc 0 2024-04-23T02:14:14.170 INFO:tasks.workunit.client.0.smithi071.stdout:3/908: symlink d7/d10/d38/l148 0 2024-04-23T02:14:14.171 INFO:tasks.workunit.client.0.smithi071.stdout:6/871: creat d6/d17/d77/d85/dfb/f116 x:0 0 0 2024-04-23T02:14:14.174 INFO:tasks.workunit.client.0.smithi071.stdout:5/988: dread d3/da/d1a/f1b [0,4194304] 0 2024-04-23T02:14:14.174 INFO:tasks.workunit.client.0.smithi071.stdout:3/909: fdatasync d7/d22/da7/fa2 0 2024-04-23T02:14:14.175 INFO:tasks.workunit.client.0.smithi071.stdout:5/989: dread - d3/da/d1a/fe2 zero size 2024-04-23T02:14:14.178 INFO:tasks.workunit.client.0.smithi071.stdout:6/872: rmdir d6/d22/dbf 39 2024-04-23T02:14:14.179 INFO:tasks.workunit.client.0.smithi071.stdout:5/990: dread - d3/d1d/f10a zero size 2024-04-23T02:14:14.179 INFO:tasks.workunit.client.0.smithi071.stdout:3/910: stat d7/da5/d128/d68/d3b/d4c 0 2024-04-23T02:14:14.183 INFO:tasks.workunit.client.0.smithi071.stdout:6/873: write d6/d104/dad/f111 [433259,23628] 0 2024-04-23T02:14:14.183 INFO:tasks.workunit.client.0.smithi071.stdout:9/872: truncate d1/f37 4061256 0 2024-04-23T02:14:14.186 INFO:tasks.workunit.client.0.smithi071.stdout:6/874: write d6/d104/dad/d8f/ffe [471228,41544] 0 2024-04-23T02:14:14.190 INFO:tasks.workunit.client.0.smithi071.stdout:6/875: read d6/d1f/d5d/d95/fd0 [700745,26124] 0 2024-04-23T02:14:14.192 INFO:tasks.workunit.client.0.smithi071.stdout:6/876: readlink d6/d104/dad/dd1/lda 0 2024-04-23T02:14:14.193 INFO:tasks.workunit.client.0.smithi071.stdout:9/873: dwrite d1/ff4 [0,4194304] 0 2024-04-23T02:14:14.217 INFO:tasks.workunit.client.1.smithi192.stdout:1/890: getdents dc/d16/d19/dae 0 2024-04-23T02:14:14.222 INFO:tasks.workunit.client.1.smithi192.stdout:1/891: truncate dc/d16/dde/d106/f10a 813958 0 2024-04-23T02:14:14.223 INFO:tasks.workunit.client.1.smithi192.stdout:1/892: mknod dc/d16/d19/d30/c11a 0 2024-04-23T02:14:14.248 INFO:tasks.workunit.client.0.smithi071.stdout:3/911: unlink d7/d10/fda 0 2024-04-23T02:14:14.249 INFO:tasks.workunit.client.0.smithi071.stdout:1/786: sync 2024-04-23T02:14:14.250 INFO:tasks.workunit.client.0.smithi071.stdout:5/991: creat d3/da/d1a/d1c/d46/f167 x:0 0 0 2024-04-23T02:14:14.252 INFO:tasks.workunit.client.0.smithi071.stdout:5/992: dread d3/da/f129 [0,4194304] 0 2024-04-23T02:14:14.253 INFO:tasks.workunit.client.0.smithi071.stdout:6/877: unlink d6/f8 0 2024-04-23T02:14:14.295 INFO:tasks.workunit.client.0.smithi071.stdout:3/912: symlink d7/da5/d100/l149 0 2024-04-23T02:14:14.296 INFO:tasks.workunit.client.0.smithi071.stdout:1/787: truncate db/df/d3a/f44 1268568 0 2024-04-23T02:14:14.297 INFO:tasks.workunit.client.0.smithi071.stdout:5/993: write d3/d1d/db9/f121 [682007,105134] 0 2024-04-23T02:14:14.299 INFO:tasks.workunit.client.0.smithi071.stdout:6/878: mknod d6/d104/dad/dd1/c117 0 2024-04-23T02:14:14.300 INFO:tasks.workunit.client.0.smithi071.stdout:3/913: mkdir d7/d22/d23/dba/df4/d14a 0 2024-04-23T02:14:14.303 INFO:tasks.workunit.client.0.smithi071.stdout:5/994: symlink d3/d8f/l168 0 2024-04-23T02:14:14.309 INFO:tasks.workunit.client.1.smithi192.stdout:1/893: sync 2024-04-23T02:14:14.310 INFO:tasks.workunit.client.0.smithi071.stdout:6/879: getdents d6/d22/dbf 0 2024-04-23T02:14:14.311 INFO:tasks.workunit.client.1.smithi192.stdout:1/894: mkdir dc/d16/db0/d11b 0 2024-04-23T02:14:14.312 INFO:tasks.workunit.client.1.smithi192.stdout:1/895: symlink dc/d16/db0/l11c 0 2024-04-23T02:14:14.316 INFO:tasks.workunit.client.1.smithi192.stdout:1/896: rename dc/d80/fa8 to dc/d16/d19/f11d 0 2024-04-23T02:14:14.323 INFO:tasks.workunit.client.0.smithi071.stdout:1/788: write db/df/d13/f1b [3528268,120951] 0 2024-04-23T02:14:14.323 INFO:tasks.workunit.client.0.smithi071.stdout:5/995: truncate d3/d34/f6a 646063 0 2024-04-23T02:14:14.325 INFO:tasks.workunit.client.0.smithi071.stdout:6/880: write d6/d17/f2c [1310668,91639] 0 2024-04-23T02:14:14.329 INFO:tasks.workunit.client.0.smithi071.stdout:9/874: dwrite d1/d32/d41/db7/f25 [12582912,4194304] 0 2024-04-23T02:14:14.329 INFO:tasks.workunit.client.1.smithi192.stdout:1/897: dwrite dc/d8b/d44/da9/fb8 [0,4194304] 0 2024-04-23T02:14:14.330 INFO:tasks.workunit.client.0.smithi071.stdout:5/996: dread d3/d8f/ff2 [0,4194304] 0 2024-04-23T02:14:14.334 INFO:tasks.workunit.client.0.smithi071.stdout:3/914: dwrite d7/da5/d128/d68/d3b/d4c/d8b/db3/f7a [0,4194304] 0 2024-04-23T02:14:14.334 INFO:tasks.workunit.client.0.smithi071.stdout:6/881: creat d6/d104/dd8/df2/dd5/f118 x:0 0 0 2024-04-23T02:14:14.335 INFO:tasks.workunit.client.0.smithi071.stdout:5/997: rmdir d3/da/d1a/d6d 39 2024-04-23T02:14:14.337 INFO:tasks.workunit.client.0.smithi071.stdout:5/998: chown d3/d1d/daf/d10/da5/feb 18646025 1 2024-04-23T02:14:14.337 INFO:tasks.workunit.client.0.smithi071.stdout:9/875: fsync d1/d32/d41/f57 0 2024-04-23T02:14:14.340 INFO:tasks.workunit.client.0.smithi071.stdout:3/915: creat d7/da5/d128/f14b x:0 0 0 2024-04-23T02:14:14.343 INFO:tasks.workunit.client.0.smithi071.stdout:9/876: creat d1/d32/d3a/d77/df8/f139 x:0 0 0 2024-04-23T02:14:14.345 INFO:tasks.workunit.client.0.smithi071.stdout:3/916: fdatasync d7/da5/d128/d68/d3b/d4c/d8b/f108 0 2024-04-23T02:14:14.348 INFO:tasks.workunit.client.0.smithi071.stdout:6/882: dwrite d6/d22/fdb [0,4194304] 0 2024-04-23T02:14:14.357 INFO:tasks.workunit.client.0.smithi071.stdout:6/883: mknod d6/d17/d77/d105/c119 0 2024-04-23T02:14:14.357 INFO:tasks.workunit.client.0.smithi071.stdout:5/999: dwrite d3/da/d1a/d1c/d46/d4e/dd8/ff7 [0,4194304] 0 2024-04-23T02:14:14.358 INFO:tasks.workunit.client.0.smithi071.stdout:1/789: dread db/df/d3a/f47 [0,4194304] 0 2024-04-23T02:14:14.358 INFO:tasks.workunit.client.0.smithi071.stdout:6/884: readlink d6/d104/dad/dd1/d10b/le1 0 2024-04-23T02:14:14.359 INFO:tasks.workunit.client.0.smithi071.stdout:9/877: write d1/f3d [600802,72535] 0 2024-04-23T02:14:14.359 INFO:tasks.workunit.client.0.smithi071.stdout:3/917: truncate d7/d22/d23/f10a 3756421 0 2024-04-23T02:14:14.363 INFO:tasks.workunit.client.0.smithi071.stdout:1/790: truncate db/df/d13/d2b/f78 1131454 0 2024-04-23T02:14:14.365 INFO:tasks.workunit.client.0.smithi071.stdout:9/878: truncate d1/f37 3186909 0 2024-04-23T02:14:14.370 INFO:tasks.workunit.client.0.smithi071.stdout:6/885: dwrite d6/fb6 [0,4194304] 0 2024-04-23T02:14:14.370 INFO:tasks.workunit.client.0.smithi071.stdout:1/791: getdents db/d11/d7f/da7 0 2024-04-23T02:14:14.372 INFO:tasks.workunit.client.0.smithi071.stdout:6/886: chown d6/d1f/df8/cfc 51744 1 2024-04-23T02:14:14.377 INFO:tasks.workunit.client.0.smithi071.stdout:1/792: creat db/d11/d7f/da7/f10a x:0 0 0 2024-04-23T02:14:14.378 INFO:tasks.workunit.client.0.smithi071.stdout:1/793: readlink db/df/d13/d2f/l74 0 2024-04-23T02:14:14.379 INFO:tasks.workunit.client.0.smithi071.stdout:9/879: creat d1/d32/d41/db7/f13a x:0 0 0 2024-04-23T02:14:14.380 INFO:tasks.workunit.client.0.smithi071.stdout:6/887: symlink d6/d1f/d46/l11a 0 2024-04-23T02:14:14.381 INFO:tasks.workunit.client.0.smithi071.stdout:3/918: dread d7/d1d/fb9 [0,4194304] 0 2024-04-23T02:14:14.382 INFO:tasks.workunit.client.0.smithi071.stdout:1/794: fsync db/df/d5f/d6f/db2/fbb 0 2024-04-23T02:14:14.384 INFO:tasks.workunit.client.1.smithi192.stdout:1/898: sync 2024-04-23T02:14:14.387 INFO:tasks.workunit.client.0.smithi071.stdout:3/919: rename d7/da5/d128/d68/dae/dc6 to d7/da5/d128/d14c 0 2024-04-23T02:14:14.387 INFO:tasks.workunit.client.1.smithi192.stdout:1/899: creat dc/d8b/d44/da9/f11e x:0 0 0 2024-04-23T02:14:14.388 INFO:tasks.workunit.client.0.smithi071.stdout:3/920: chown d7/d1d/f72 158 1 2024-04-23T02:14:14.390 INFO:tasks.workunit.client.1.smithi192.stdout:1/900: creat dc/d14/d23/d5e/dab/f11f x:0 0 0 2024-04-23T02:14:14.391 INFO:tasks.workunit.client.0.smithi071.stdout:9/880: rmdir d1/d32/d41/d134 0 2024-04-23T02:14:14.392 INFO:tasks.workunit.client.1.smithi192.stdout:1/901: creat dc/d8b/d44/da9/f120 x:0 0 0 2024-04-23T02:14:14.392 INFO:tasks.workunit.client.0.smithi071.stdout:3/921: creat d7/d22/d23/d26/d52/f14d x:0 0 0 2024-04-23T02:14:14.394 INFO:tasks.workunit.client.0.smithi071.stdout:3/922: chown d7/d22/d23/dba/df4/dce/l10b 636537 1 2024-04-23T02:14:14.394 INFO:tasks.workunit.client.1.smithi192.stdout:1/902: dread dc/d14/d15/ff2 [0,4194304] 0 2024-04-23T02:14:14.394 INFO:tasks.workunit.client.0.smithi071.stdout:9/881: mkdir d1/d32/d7e/d13b 0 2024-04-23T02:14:14.397 INFO:tasks.workunit.client.0.smithi071.stdout:3/923: read f0 [699173,54122] 0 2024-04-23T02:14:14.398 INFO:tasks.workunit.client.1.smithi192.stdout:1/903: creat dc/d16/d19/f121 x:0 0 0 2024-04-23T02:14:14.402 INFO:tasks.workunit.client.1.smithi192.stdout:1/904: fsync dc/d16/d19/f121 0 2024-04-23T02:14:14.403 INFO:tasks.workunit.client.0.smithi071.stdout:3/924: link d7/d8/dea/c13d d7/da5/d128/d68/d3b/d4c/d8b/d12a/c14e 0 2024-04-23T02:14:14.404 INFO:tasks.workunit.client.0.smithi071.stdout:1/795: write db/d11/d7f/da7/dd9/d100/f77 [204208,83005] 0 2024-04-23T02:14:14.405 INFO:tasks.workunit.client.1.smithi192.stdout:1/905: link dc/d8b/l48 dc/d16/d19/dae/l122 0 2024-04-23T02:14:14.407 INFO:tasks.workunit.client.0.smithi071.stdout:3/925: creat d7/d8/d42/d11b/f14f x:0 0 0 2024-04-23T02:14:14.407 INFO:tasks.workunit.client.1.smithi192.stdout:1/906: dread dc/d14/d15/ff2 [0,4194304] 0 2024-04-23T02:14:14.408 INFO:tasks.workunit.client.0.smithi071.stdout:6/888: dread d6/d17/d77/d9e/fcd [0,4194304] 0 2024-04-23T02:14:14.412 INFO:tasks.workunit.client.0.smithi071.stdout:6/889: creat d6/d104/dd8/f11b x:0 0 0 2024-04-23T02:14:14.412 INFO:tasks.workunit.client.0.smithi071.stdout:1/796: dwrite db/d11/ff7 [0,4194304] 0 2024-04-23T02:14:14.413 INFO:tasks.workunit.client.1.smithi192.stdout:1/907: truncate dc/d16/d19/d74/faf 333192 0 2024-04-23T02:14:14.414 INFO:tasks.workunit.client.0.smithi071.stdout:3/926: rmdir d7/d22/d23/dba/df4/dca/d142 0 2024-04-23T02:14:14.415 INFO:tasks.workunit.client.1.smithi192.stdout:1/908: mkdir dc/d16/db0/d123 0 2024-04-23T02:14:14.417 INFO:tasks.workunit.client.0.smithi071.stdout:3/927: readlink d7/dfd/lfe 0 2024-04-23T02:14:14.418 INFO:tasks.workunit.client.1.smithi192.stdout:1/909: truncate dc/d80/fac 590799 0 2024-04-23T02:14:14.418 INFO:tasks.workunit.client.0.smithi071.stdout:1/797: mknod db/df/d13/d2f/d48/d9f/dfb/c10b 0 2024-04-23T02:14:14.420 INFO:tasks.workunit.client.0.smithi071.stdout:3/928: mkdir d7/d8/d42/d47/d150 0 2024-04-23T02:14:14.423 INFO:tasks.workunit.client.0.smithi071.stdout:1/798: unlink db/df/d13/d2f/fda 0 2024-04-23T02:14:14.425 INFO:tasks.workunit.client.0.smithi071.stdout:3/929: link d7/d22/c122 d7/d10/d38/daa/c151 0 2024-04-23T02:14:14.427 INFO:tasks.workunit.client.0.smithi071.stdout:3/930: write d7/da5/d128/f40 [1285415,86592] 0 2024-04-23T02:14:14.428 INFO:tasks.workunit.client.0.smithi071.stdout:3/931: chown d7/dfd 166923 1 2024-04-23T02:14:14.429 INFO:tasks.workunit.client.0.smithi071.stdout:3/932: chown d7/da5/d128/d68/d3b/f13a 969131750 1 2024-04-23T02:14:14.432 INFO:tasks.workunit.client.0.smithi071.stdout:3/933: creat d7/d22/d23/d26/f152 x:0 0 0 2024-04-23T02:14:14.437 INFO:tasks.workunit.client.0.smithi071.stdout:1/799: dwrite db/df/d13/d20/f53 [0,4194304] 0 2024-04-23T02:14:14.438 INFO:tasks.workunit.client.0.smithi071.stdout:6/890: dread d6/d17/d77/f96 [0,4194304] 0 2024-04-23T02:14:14.440 INFO:tasks.workunit.client.0.smithi071.stdout:1/800: chown db/df/d13/d20/d75/d96/fd4 107342 1 2024-04-23T02:14:14.441 INFO:tasks.workunit.client.0.smithi071.stdout:6/891: symlink d6/d1f/d5d/l11c 0 2024-04-23T02:14:14.443 INFO:tasks.workunit.client.0.smithi071.stdout:1/801: creat db/df/d13/d2f/d7b/f10c x:0 0 0 2024-04-23T02:14:14.444 INFO:tasks.workunit.client.0.smithi071.stdout:6/892: creat d6/d1f/d46/f11d x:0 0 0 2024-04-23T02:14:14.445 INFO:tasks.workunit.client.0.smithi071.stdout:1/802: chown db/d11/d7f/da7/dd9/d100/f102 1053909752 1 2024-04-23T02:14:14.448 INFO:tasks.workunit.client.0.smithi071.stdout:6/893: symlink d6/d17/d77/d105/d108/l11e 0 2024-04-23T02:14:14.448 INFO:tasks.workunit.client.0.smithi071.stdout:3/934: dread d7/d22/d23/d26/d52/dd3/fcc [0,4194304] 0 2024-04-23T02:14:14.449 INFO:tasks.workunit.client.0.smithi071.stdout:6/894: write d6/d1f/d46/f5a [2343253,33419] 0 2024-04-23T02:14:14.454 INFO:tasks.workunit.client.0.smithi071.stdout:6/895: chown d6/d17/d77/da2/fa7 0 1 2024-04-23T02:14:14.454 INFO:tasks.workunit.client.0.smithi071.stdout:3/935: creat d7/da5/d128/d68/d3b/d4c/d130/f153 x:0 0 0 2024-04-23T02:14:14.454 INFO:tasks.workunit.client.0.smithi071.stdout:3/936: fsync d7/d8/f61 0 2024-04-23T02:14:14.456 INFO:tasks.workunit.client.0.smithi071.stdout:6/896: unlink d6/d104/dad/d8f/cb3 0 2024-04-23T02:14:14.456 INFO:tasks.workunit.client.0.smithi071.stdout:6/897: fsync d6/d104/dad/ff3 0 2024-04-23T02:14:14.457 INFO:tasks.workunit.client.0.smithi071.stdout:3/937: symlink d7/d8/dea/df6/d114/l154 0 2024-04-23T02:14:14.460 INFO:tasks.workunit.client.0.smithi071.stdout:3/938: symlink d7/d22/d23/d26/d52/d12f/l155 0 2024-04-23T02:14:14.463 INFO:tasks.workunit.client.0.smithi071.stdout:3/939: fsync d7/d8/f1b 0 2024-04-23T02:14:14.465 INFO:tasks.workunit.client.0.smithi071.stdout:1/803: dwrite db/df/d13/fef [0,4194304] 0 2024-04-23T02:14:14.466 INFO:tasks.workunit.client.0.smithi071.stdout:1/804: chown db/d11/c16 2 1 2024-04-23T02:14:14.469 INFO:tasks.workunit.client.0.smithi071.stdout:1/805: read - db/df/d13/d20/dc0/ddf/fee zero size 2024-04-23T02:14:14.472 INFO:tasks.workunit.client.0.smithi071.stdout:6/898: dwrite d6/d1f/f32 [0,4194304] 0 2024-04-23T02:14:14.475 INFO:tasks.workunit.client.0.smithi071.stdout:1/806: creat db/df/d13/d2f/f10d x:0 0 0 2024-04-23T02:14:14.476 INFO:tasks.workunit.client.0.smithi071.stdout:6/899: creat d6/d1f/d46/f11f x:0 0 0 2024-04-23T02:14:14.477 INFO:tasks.workunit.client.0.smithi071.stdout:3/940: dwrite d7/da5/d128/d68/f91 [0,4194304] 0 2024-04-23T02:14:14.477 INFO:tasks.workunit.client.0.smithi071.stdout:6/900: fdatasync d6/d17/d77/fe0 0 2024-04-23T02:14:14.479 INFO:tasks.workunit.client.0.smithi071.stdout:3/941: write d7/d22/d23/d26/d52/f14d [385460,94362] 0 2024-04-23T02:14:14.480 INFO:tasks.workunit.client.0.smithi071.stdout:1/807: creat db/d11/d7f/da7/dd9/f10e x:0 0 0 2024-04-23T02:14:14.482 INFO:tasks.workunit.client.0.smithi071.stdout:3/942: dread - d7/da5/db2/f112 zero size 2024-04-23T02:14:14.486 INFO:tasks.workunit.client.0.smithi071.stdout:3/943: truncate d7/d8/d42/f7c 455155 0 2024-04-23T02:14:14.487 INFO:tasks.workunit.client.0.smithi071.stdout:3/944: write d7/d10/d38/daa/f120 [788830,6709] 0 2024-04-23T02:14:14.489 INFO:tasks.workunit.client.0.smithi071.stdout:3/945: mknod d7/da5/d128/d68/dae/c156 0 2024-04-23T02:14:14.491 INFO:tasks.workunit.client.0.smithi071.stdout:3/946: creat d7/da5/db5/d144/f157 x:0 0 0 2024-04-23T02:14:14.492 INFO:tasks.workunit.client.0.smithi071.stdout:3/947: mknod d7/d22/d23/d26/d7e/c158 0 2024-04-23T02:14:14.494 INFO:tasks.workunit.client.0.smithi071.stdout:3/948: creat d7/da5/d128/d68/d3b/f159 x:0 0 0 2024-04-23T02:14:14.495 INFO:tasks.workunit.client.0.smithi071.stdout:3/949: write d7/d8/f11 [429556,6164] 0 2024-04-23T02:14:14.497 INFO:tasks.workunit.client.0.smithi071.stdout:3/950: creat d7/d22/d23/d26/d52/d12f/f15a x:0 0 0 2024-04-23T02:14:14.499 INFO:tasks.workunit.client.0.smithi071.stdout:3/951: write d7/da5/d128/d68/f91 [3188183,66025] 0 2024-04-23T02:14:14.500 INFO:tasks.workunit.client.0.smithi071.stdout:1/808: truncate db/d11/ff7 2751663 0 2024-04-23T02:14:14.501 INFO:tasks.workunit.client.0.smithi071.stdout:1/809: chown fa 886964614 1 2024-04-23T02:14:14.504 INFO:tasks.workunit.client.0.smithi071.stdout:3/952: rename d7/d22/d23/d26/c93 to d7/da5/d128/d68/d3b/c15b 0 2024-04-23T02:14:14.505 INFO:tasks.workunit.client.0.smithi071.stdout:1/810: symlink db/l10f 0 2024-04-23T02:14:14.505 INFO:tasks.workunit.client.0.smithi071.stdout:6/901: dread d6/d1f/f9a [0,4194304] 0 2024-04-23T02:14:14.505 INFO:tasks.workunit.client.0.smithi071.stdout:3/953: fdatasync d7/d10/d38/f11d 0 2024-04-23T02:14:14.506 INFO:tasks.workunit.client.0.smithi071.stdout:6/902: write d6/f34 [3691587,102894] 0 2024-04-23T02:14:14.507 INFO:tasks.workunit.client.0.smithi071.stdout:1/811: creat db/d11/d8d/f110 x:0 0 0 2024-04-23T02:14:14.508 INFO:tasks.workunit.client.0.smithi071.stdout:1/812: readlink db/df/l56 0 2024-04-23T02:14:14.508 INFO:tasks.workunit.client.0.smithi071.stdout:3/954: symlink d7/d22/def/l15c 0 2024-04-23T02:14:14.510 INFO:tasks.workunit.client.0.smithi071.stdout:1/813: readlink db/df/d5f/d6f/db2/l93 0 2024-04-23T02:14:14.512 INFO:tasks.workunit.client.0.smithi071.stdout:3/955: rename d7/d22/d23/dba/df4/c101 to d7/da5/d128/d68/d3b/d4c/d8b/db3/c15d 0 2024-04-23T02:14:14.513 INFO:tasks.workunit.client.1.smithi192.stdout:1/910: sync 2024-04-23T02:14:14.514 INFO:tasks.workunit.client.0.smithi071.stdout:3/956: write d7/d22/d23/d26/d7e/fd0 [794946,8834] 0 2024-04-23T02:14:14.514 INFO:tasks.workunit.client.0.smithi071.stdout:1/814: symlink db/df/dc2/dec/l111 0 2024-04-23T02:14:14.515 INFO:tasks.workunit.client.0.smithi071.stdout:3/957: readlink d7/d22/d23/dba/l10f 0 2024-04-23T02:14:14.515 INFO:tasks.workunit.client.0.smithi071.stdout:1/815: chown db/d11/f64 1748735843 1 2024-04-23T02:14:14.516 INFO:tasks.workunit.client.1.smithi192.stdout:1/911: symlink dc/d16/d19/dae/d118/l124 0 2024-04-23T02:14:14.517 INFO:tasks.workunit.client.1.smithi192.stdout:1/912: chown dc/d16/d19 2981 1 2024-04-23T02:14:14.519 INFO:tasks.workunit.client.1.smithi192.stdout:1/913: symlink dc/d80/d81/l125 0 2024-04-23T02:14:14.519 INFO:tasks.workunit.client.0.smithi071.stdout:3/958: creat d7/f15e x:0 0 0 2024-04-23T02:14:14.520 INFO:tasks.workunit.client.0.smithi071.stdout:6/903: write d6/d104/dad/d8f/de2/fbd [981885,61676] 0 2024-04-23T02:14:14.521 INFO:tasks.workunit.client.1.smithi192.stdout:1/914: rename dc/d14/d23/d5e/dab/fd3 to dc/d80/d81/f126 0 2024-04-23T02:14:14.530 INFO:tasks.workunit.client.1.smithi192.stdout:1/915: dread dc/d14/d23/f87 [0,4194304] 0 2024-04-23T02:14:14.531 INFO:tasks.workunit.client.0.smithi071.stdout:3/959: dwrite d7/da5/d128/d68/d3b/d98/d11a/f143 [0,4194304] 0 2024-04-23T02:14:14.531 INFO:tasks.workunit.client.0.smithi071.stdout:1/816: dwrite db/d11/f7a [0,4194304] 0 2024-04-23T02:14:14.533 INFO:tasks.workunit.client.0.smithi071.stdout:6/904: dwrite d6/f18 [0,4194304] 0 2024-04-23T02:14:14.535 INFO:tasks.workunit.client.0.smithi071.stdout:3/960: rmdir d7/dfd 39 2024-04-23T02:14:14.540 INFO:tasks.workunit.client.0.smithi071.stdout:6/905: creat d6/d22/d92/f120 x:0 0 0 2024-04-23T02:14:14.545 INFO:tasks.workunit.client.0.smithi071.stdout:1/817: dwrite db/df/d13/d2f/d48/d85/fb0 [0,4194304] 0 2024-04-23T02:14:14.548 INFO:tasks.workunit.client.0.smithi071.stdout:1/818: creat db/df/d13/ddc/f112 x:0 0 0 2024-04-23T02:14:14.556 INFO:tasks.workunit.client.0.smithi071.stdout:3/961: truncate d7/da5/d100/f10d 84877 0 2024-04-23T02:14:14.558 INFO:tasks.workunit.client.0.smithi071.stdout:3/962: mkdir d7/d22/d23/d26/d7e/d15f 0 2024-04-23T02:14:14.566 INFO:tasks.workunit.client.0.smithi071.stdout:6/906: dread d6/d104/dad/f8a [0,4194304] 0 2024-04-23T02:14:14.567 INFO:tasks.workunit.client.0.smithi071.stdout:6/907: mkdir d6/d1f/d5d/dde/d121 0 2024-04-23T02:14:14.574 INFO:tasks.workunit.client.1.smithi192.stdout:1/916: rename dc/d8b/d34/d70/d8c/da3/de4/de8/ff7 to dc/d16/d19/d74/df4/f127 0 2024-04-23T02:14:14.611 INFO:tasks.workunit.client.0.smithi071.stdout:6/908: mkdir d6/d104/d122 0 2024-04-23T02:14:14.614 INFO:tasks.workunit.client.0.smithi071.stdout:3/963: write f0 [1583430,40565] 0 2024-04-23T02:14:14.614 INFO:tasks.workunit.client.0.smithi071.stdout:6/909: getdents d6/d17/d77/d85/dfb/d10a 0 2024-04-23T02:14:14.616 INFO:tasks.workunit.client.0.smithi071.stdout:3/964: creat d7/d10/d38/f160 x:0 0 0 2024-04-23T02:14:14.617 INFO:tasks.workunit.client.0.smithi071.stdout:6/910: getdents d6/d22/dcf 0 2024-04-23T02:14:14.619 INFO:tasks.workunit.client.0.smithi071.stdout:3/965: creat d7/d22/def/d102/f161 x:0 0 0 2024-04-23T02:14:14.620 INFO:tasks.workunit.client.0.smithi071.stdout:6/911: symlink d6/l123 0 2024-04-23T02:14:14.620 INFO:tasks.workunit.client.0.smithi071.stdout:3/966: chown d7/da5/d128/d68/d3b/d4c/da1/fc2 31367 1 2024-04-23T02:14:14.622 INFO:tasks.workunit.client.0.smithi071.stdout:6/912: mknod d6/d1f/d5d/dde/c124 0 2024-04-23T02:14:14.622 INFO:tasks.workunit.client.0.smithi071.stdout:3/967: truncate d7/d22/d23/d26/fa0 864310 0 2024-04-23T02:14:14.624 INFO:tasks.workunit.client.0.smithi071.stdout:6/913: unlink d6/d1f/d5d/f67 0 2024-04-23T02:14:14.625 INFO:tasks.workunit.client.0.smithi071.stdout:3/968: mknod d7/d22/d23/d26/d7e/d15f/c162 0 2024-04-23T02:14:14.626 INFO:tasks.workunit.client.0.smithi071.stdout:6/914: creat d6/d1f/d46/f125 x:0 0 0 2024-04-23T02:14:14.627 INFO:tasks.workunit.client.0.smithi071.stdout:6/915: symlink d6/d104/dd8/df2/l126 0 2024-04-23T02:14:14.629 INFO:tasks.workunit.client.0.smithi071.stdout:3/969: getdents d7/da5/d128/d68/d3b/d4c/d8b 0 2024-04-23T02:14:14.630 INFO:tasks.workunit.client.0.smithi071.stdout:3/970: creat d7/d8/dea/df6/f163 x:0 0 0 2024-04-23T02:14:14.632 INFO:tasks.workunit.client.0.smithi071.stdout:6/916: link d6/d22/lc1 d6/d1f/d46/l127 0 2024-04-23T02:14:14.635 INFO:tasks.workunit.client.0.smithi071.stdout:6/917: creat d6/d104/d122/f128 x:0 0 0 2024-04-23T02:14:14.638 INFO:tasks.workunit.client.0.smithi071.stdout:6/918: rename d6/d104/dd8/df2/dd5/f118 to d6/d1f/d5d/f129 0 2024-04-23T02:14:14.644 INFO:tasks.workunit.client.0.smithi071.stdout:6/919: dwrite d6/d104/dad/d3f/fd9 [0,4194304] 0 2024-04-23T02:14:14.647 INFO:tasks.workunit.client.0.smithi071.stdout:6/920: symlink d6/d104/db5/l12a 0 2024-04-23T02:14:14.648 INFO:tasks.workunit.client.0.smithi071.stdout:3/971: dwrite d7/d8/d42/d47/fc8 [0,4194304] 0 2024-04-23T02:14:14.657 INFO:tasks.workunit.client.0.smithi071.stdout:6/921: truncate d6/d17/d77/d9e/fc5 156945 0 2024-04-23T02:14:14.657 INFO:tasks.workunit.client.0.smithi071.stdout:3/972: dwrite d7/da5/d128/d68/d3b/d4c/d8b/db3/fc4 [0,4194304] 0 2024-04-23T02:14:14.660 INFO:tasks.workunit.client.0.smithi071.stdout:3/973: creat d7/da5/d128/d68/d3b/d4c/d8b/f164 x:0 0 0 2024-04-23T02:14:14.662 INFO:tasks.workunit.client.0.smithi071.stdout:6/922: creat d6/d104/f12b x:0 0 0 2024-04-23T02:14:14.664 INFO:tasks.workunit.client.0.smithi071.stdout:3/974: mknod d7/da5/d128/d68/c165 0 2024-04-23T02:14:14.665 INFO:tasks.workunit.client.0.smithi071.stdout:6/923: write d6/d1f/f32 [4948492,56263] 0 2024-04-23T02:14:14.669 INFO:tasks.workunit.client.1.smithi192.stdout:1/917: sync 2024-04-23T02:14:14.674 INFO:tasks.workunit.client.1.smithi192.stdout:1/918: dwrite f3 [0,4194304] 0 2024-04-23T02:14:14.678 INFO:tasks.workunit.client.1.smithi192.stdout:1/919: truncate dc/d80/d81/f9e 2098631 0 2024-04-23T02:14:14.680 INFO:tasks.workunit.client.0.smithi071.stdout:6/924: link d6/d17/d77/d9e/fc5 d6/d104/dad/f12c 0 2024-04-23T02:14:14.682 INFO:tasks.workunit.client.1.smithi192.stdout:1/920: rmdir dc/d16/db0/d11b 0 2024-04-23T02:14:14.687 INFO:tasks.workunit.client.0.smithi071.stdout:6/925: rename d6/d1f/df8/f10d to d6/d1f/d5d/d95/f12d 0 2024-04-23T02:14:14.687 INFO:tasks.workunit.client.1.smithi192.stdout:1/921: dwrite dc/f29 [0,4194304] 0 2024-04-23T02:14:14.689 INFO:tasks.workunit.client.0.smithi071.stdout:6/926: truncate d6/d104/dad/dd1/d10b/fa3 1039871 0 2024-04-23T02:14:14.690 INFO:tasks.workunit.client.1.smithi192.stdout:1/922: symlink dc/d16/d19/d30/dc4/l128 0 2024-04-23T02:14:14.691 INFO:tasks.workunit.client.1.smithi192.stdout:1/923: stat dc/d8b/d34/d70/d8c/ff6 0 2024-04-23T02:14:14.693 INFO:tasks.workunit.client.1.smithi192.stdout:1/924: symlink dc/l129 0 2024-04-23T02:14:14.695 INFO:tasks.workunit.client.0.smithi071.stdout:6/927: dwrite d6/d1f/d46/f125 [0,4194304] 0 2024-04-23T02:14:14.696 INFO:tasks.workunit.client.0.smithi071.stdout:4/901: sync 2024-04-23T02:14:14.698 INFO:tasks.workunit.client.1.smithi192.stdout:1/925: symlink dc/d16/d19/d74/d77/l12a 0 2024-04-23T02:14:14.699 INFO:tasks.workunit.client.0.smithi071.stdout:6/928: creat d6/d104/dad/d3f/f12e x:0 0 0 2024-04-23T02:14:14.700 INFO:tasks.workunit.client.0.smithi071.stdout:6/929: fsync d6/d104/ff0 0 2024-04-23T02:14:14.701 INFO:tasks.workunit.client.0.smithi071.stdout:6/930: stat d6/d1f/d5d/cc0 0 2024-04-23T02:14:14.704 INFO:tasks.workunit.client.0.smithi071.stdout:6/931: mknod d6/d104/dd8/df2/de7/c12f 0 2024-04-23T02:14:14.706 INFO:tasks.workunit.client.0.smithi071.stdout:6/932: read d6/d22/f41 [947191,117457] 0 2024-04-23T02:14:14.706 INFO:tasks.workunit.client.1.smithi192.stdout:1/926: dwrite dc/d16/d19/f11d [0,4194304] 0 2024-04-23T02:14:14.708 INFO:tasks.workunit.client.0.smithi071.stdout:6/933: dread - d6/d104/dd8/df2/fc8 zero size 2024-04-23T02:14:14.712 INFO:tasks.workunit.client.0.smithi071.stdout:6/934: dread d6/d104/dad/f111 [0,4194304] 0 2024-04-23T02:14:14.713 INFO:tasks.workunit.client.0.smithi071.stdout:6/935: chown d6/d1f/d46/l11a 597 1 2024-04-23T02:14:14.713 INFO:tasks.workunit.client.1.smithi192.stdout:1/927: rmdir dc/d14/dfd 0 2024-04-23T02:14:14.720 INFO:tasks.workunit.client.0.smithi071.stdout:4/902: dread d2/d2a/d2d/d39/f63 [0,4194304] 0 2024-04-23T02:14:14.724 INFO:tasks.workunit.client.1.smithi192.stdout:1/928: dread f9 [4194304,4194304] 0 2024-04-23T02:14:14.725 INFO:tasks.workunit.client.0.smithi071.stdout:4/903: link d2/d2a/d2d/d7b/f8c d2/d2a/d2d/d39/d42/d49/d67/d7e/f121 0 2024-04-23T02:14:14.726 INFO:tasks.workunit.client.0.smithi071.stdout:4/904: fdatasync d2/d2a/d2d/d39/d42/d49/d67/d7e/dc2/f10f 0 2024-04-23T02:14:14.731 INFO:tasks.workunit.client.0.smithi071.stdout:4/905: dwrite d2/d2a/da2/dd1/dde/d23/d29/fd3 [0,4194304] 0 2024-04-23T02:14:14.732 INFO:tasks.workunit.client.1.smithi192.stdout:1/929: dwrite dc/d8b/f111 [0,4194304] 0 2024-04-23T02:14:14.735 INFO:tasks.workunit.client.0.smithi071.stdout:4/906: write d2/d2a/d2d/d39/d42/d49/d67/d7e/dc2/f10f [780738,815] 0 2024-04-23T02:14:14.737 INFO:tasks.workunit.client.0.smithi071.stdout:6/936: dread d6/d17/d77/f8c [0,4194304] 0 2024-04-23T02:14:14.738 INFO:tasks.workunit.client.0.smithi071.stdout:4/907: write d2/d2a/d2d/d7b/f10b [1794060,49071] 0 2024-04-23T02:14:14.738 INFO:tasks.workunit.client.1.smithi192.stdout:1/930: creat dc/d14/d23/d5e/dab/f12b x:0 0 0 2024-04-23T02:14:14.740 INFO:tasks.workunit.client.0.smithi071.stdout:6/937: creat d6/d104/dd8/df2/de7/f130 x:0 0 0 2024-04-23T02:14:14.741 INFO:tasks.workunit.client.0.smithi071.stdout:6/938: readlink d6/d22/dbf/leb 0 2024-04-23T02:14:14.744 INFO:tasks.workunit.client.0.smithi071.stdout:6/939: getdents d6/d22 0 2024-04-23T02:14:14.746 INFO:tasks.workunit.client.1.smithi192.stdout:1/931: dwrite dc/d16/d19/d30/f3f [4194304,4194304] 0 2024-04-23T02:14:14.748 INFO:tasks.workunit.client.0.smithi071.stdout:6/940: unlink d6/d1f/d5d/d95/f12d 0 2024-04-23T02:14:14.749 INFO:tasks.workunit.client.0.smithi071.stdout:4/908: write d2/d2a/d73/d5e/fb0 [166406,60834] 0 2024-04-23T02:14:14.750 INFO:tasks.workunit.client.1.smithi192.stdout:1/932: creat dc/d8b/d34/d70/d8c/da3/de4/de8/f12c x:0 0 0 2024-04-23T02:14:14.751 INFO:tasks.workunit.client.0.smithi071.stdout:6/941: mkdir d6/d104/dd8/df2/d131 0 2024-04-23T02:14:14.751 INFO:tasks.workunit.client.1.smithi192.stdout:1/933: rmdir dc 39 2024-04-23T02:14:14.754 INFO:tasks.workunit.client.0.smithi071.stdout:6/942: symlink d6/d104/dad/l132 0 2024-04-23T02:14:14.755 INFO:tasks.workunit.client.1.smithi192.stdout:1/934: creat dc/d8b/d34/d70/d8c/da3/de4/f12d x:0 0 0 2024-04-23T02:14:14.755 INFO:tasks.workunit.client.0.smithi071.stdout:6/943: write d6/d17/d77/da2/fc6 [199349,43582] 0 2024-04-23T02:14:14.756 INFO:tasks.workunit.client.1.smithi192.stdout:1/935: fdatasync f9 0 2024-04-23T02:14:14.757 INFO:tasks.workunit.client.0.smithi071.stdout:6/944: write d6/d1f/d46/f125 [2223774,97344] 0 2024-04-23T02:14:14.757 INFO:tasks.workunit.client.1.smithi192.stdout:1/936: fsync dc/d14/d23/f50 0 2024-04-23T02:14:14.758 INFO:tasks.workunit.client.1.smithi192.stdout:1/937: stat dc/d14/d15/f54 0 2024-04-23T02:14:14.758 INFO:tasks.workunit.client.0.smithi071.stdout:6/945: chown d6/d104/dd8/df2/d131 290072555 1 2024-04-23T02:14:14.759 INFO:tasks.workunit.client.1.smithi192.stdout:1/938: symlink dc/d8b/d44/l12e 0 2024-04-23T02:14:14.761 INFO:tasks.workunit.client.0.smithi071.stdout:6/946: rename d6/c7c to d6/d22/d92/c133 0 2024-04-23T02:14:14.761 INFO:tasks.workunit.client.1.smithi192.stdout:1/939: truncate dc/d8b/d34/d70/d8c/da3/dc1/f109 297121 0 2024-04-23T02:14:14.763 INFO:tasks.workunit.client.0.smithi071.stdout:6/947: creat d6/d104/db5/f134 x:0 0 0 2024-04-23T02:14:14.764 INFO:tasks.workunit.client.0.smithi071.stdout:4/909: write d2/d2a/d2d/d39/d42/d49/d67/d7e/f121 [4741142,78087] 0 2024-04-23T02:14:14.764 INFO:tasks.workunit.client.1.smithi192.stdout:1/940: symlink dc/d16/db0/d123/l12f 0 2024-04-23T02:14:14.766 INFO:tasks.workunit.client.1.smithi192.stdout:1/941: creat dc/d14/f130 x:0 0 0 2024-04-23T02:14:14.767 INFO:tasks.workunit.client.0.smithi071.stdout:6/948: rename d6/d17/d77/d105 to d6/d1f/d46/d135 0 2024-04-23T02:14:14.767 INFO:tasks.workunit.client.1.smithi192.stdout:1/942: write dc/d8b/fe7 [559769,21710] 0 2024-04-23T02:14:14.768 INFO:tasks.workunit.client.0.smithi071.stdout:4/910: creat d2/d2a/d73/d111/f122 x:0 0 0 2024-04-23T02:14:14.769 INFO:tasks.workunit.client.0.smithi071.stdout:4/911: fdatasync d2/d2a/da2/dd1/dde/d23/d4f/db5/f10a 0 2024-04-23T02:14:14.770 INFO:tasks.workunit.client.0.smithi071.stdout:6/949: creat d6/d22/f136 x:0 0 0 2024-04-23T02:14:14.770 INFO:tasks.workunit.client.1.smithi192.stdout:1/943: rename dc/d14/d15/fcd to dc/d14/d23/da2/f131 0 2024-04-23T02:14:14.771 INFO:tasks.workunit.client.0.smithi071.stdout:4/912: chown d2/d2a/d2d/d39/d42/d49/d67/d7e/dc2 506363 1 2024-04-23T02:14:14.771 INFO:tasks.workunit.client.1.smithi192.stdout:1/944: chown dc/d16/d19/d30/f55 72631 1 2024-04-23T02:14:14.773 INFO:tasks.workunit.client.0.smithi071.stdout:4/913: fdatasync d2/d2a/d2d/d39/d42/d49/d67/d7e/d82/f100 0 2024-04-23T02:14:14.773 INFO:tasks.workunit.client.0.smithi071.stdout:4/914: chown d2/d2a/d73/f104 251336 1 2024-04-23T02:14:14.775 INFO:tasks.workunit.client.0.smithi071.stdout:4/915: dread - d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/fc9 zero size 2024-04-23T02:14:14.778 INFO:tasks.workunit.client.1.smithi192.stdout:1/945: dwrite dc/d16/d19/d30/dc4/fd9 [0,4194304] 0 2024-04-23T02:14:14.780 INFO:tasks.workunit.client.0.smithi071.stdout:4/916: rename d2/d2a/da2/dd1/dde/d23/f8e to d2/d2a/d2d/d39/d42/d49/d91/ddb/d10d/d119/f123 0 2024-04-23T02:14:14.783 INFO:tasks.workunit.client.0.smithi071.stdout:4/917: creat d2/d2a/d2d/d39/d42/d49/d91/ddb/f124 x:0 0 0 2024-04-23T02:14:14.785 INFO:tasks.workunit.client.0.smithi071.stdout:4/918: write d2/d2a/da2/dd1/dde/d23/d4f/fd7 [272314,6152] 0 2024-04-23T02:14:14.785 INFO:tasks.workunit.client.1.smithi192.stdout:1/946: rename dc/d16/d19/dae to dc/d16/d132 0 2024-04-23T02:14:14.786 INFO:tasks.workunit.client.0.smithi071.stdout:4/919: stat d2/d2a/da2/dd1/dde/d23/d4f/d8b 0 2024-04-23T02:14:14.788 INFO:tasks.workunit.client.1.smithi192.stdout:1/947: getdents dc/d8b/d44/da9 0 2024-04-23T02:14:14.791 INFO:tasks.workunit.client.0.smithi071.stdout:6/950: dwrite d6/d22/d92/fae [0,4194304] 0 2024-04-23T02:14:14.795 INFO:tasks.workunit.client.0.smithi071.stdout:6/951: symlink d6/d17/d77/d9e/l137 0 2024-04-23T02:14:14.795 INFO:tasks.workunit.client.0.smithi071.stdout:4/920: link d2/d2a/da2/dd1/dde/f61 d2/d2a/d73/f125 0 2024-04-23T02:14:14.799 INFO:tasks.workunit.client.0.smithi071.stdout:6/952: dread d6/d104/dad/f12c [0,4194304] 0 2024-04-23T02:14:14.802 INFO:tasks.workunit.client.0.smithi071.stdout:6/953: symlink d6/d104/dd8/df2/l138 0 2024-04-23T02:14:14.803 INFO:tasks.workunit.client.0.smithi071.stdout:4/921: dwrite d2/d2a/d2d/d39/d42/d49/d67/d3e/f76 [4194304,4194304] 0 2024-04-23T02:14:14.805 INFO:tasks.workunit.client.0.smithi071.stdout:6/954: mknod d6/d104/dad/dd1/c139 0 2024-04-23T02:14:14.806 INFO:tasks.workunit.client.0.smithi071.stdout:4/922: symlink d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/dc8/l126 0 2024-04-23T02:14:14.808 INFO:tasks.workunit.client.0.smithi071.stdout:4/923: truncate d2/d2a/d73/d5e/fb0 1205898 0 2024-04-23T02:14:14.811 INFO:tasks.workunit.client.0.smithi071.stdout:6/955: creat d6/d22/dcf/f13a x:0 0 0 2024-04-23T02:14:14.811 INFO:tasks.workunit.client.0.smithi071.stdout:4/924: dread - d2/d2a/da2/dd1/dde/d57/ff1 zero size 2024-04-23T02:14:14.813 INFO:tasks.workunit.client.0.smithi071.stdout:4/925: write d2/d2a/daa/fad [956141,41293] 0 2024-04-23T02:14:14.815 INFO:tasks.workunit.client.0.smithi071.stdout:4/926: symlink d2/d2a/daa/l127 0 2024-04-23T02:14:14.816 INFO:tasks.workunit.client.0.smithi071.stdout:4/927: fdatasync d2/d53/fc0 0 2024-04-23T02:14:14.817 INFO:tasks.workunit.client.0.smithi071.stdout:4/928: symlink d2/d2a/d2d/l128 0 2024-04-23T02:14:14.824 INFO:tasks.workunit.client.0.smithi071.stdout:4/929: truncate d2/d2a/d2d/d7b/f8c 3416347 0 2024-04-23T02:14:14.826 INFO:tasks.workunit.client.0.smithi071.stdout:4/930: symlink d2/l129 0 2024-04-23T02:14:14.828 INFO:tasks.workunit.client.0.smithi071.stdout:4/931: creat d2/d2a/d2d/d39/d42/d49/d91/f12a x:0 0 0 2024-04-23T02:14:14.837 INFO:tasks.workunit.client.0.smithi071.stdout:2/780: sync 2024-04-23T02:14:14.837 INFO:tasks.workunit.client.0.smithi071.stdout:8/891: sync 2024-04-23T02:14:14.837 INFO:tasks.workunit.client.0.smithi071.stdout:0/940: sync 2024-04-23T02:14:14.839 INFO:tasks.workunit.client.0.smithi071.stdout:4/932: dwrite d2/d2a/da2/dd1/dde/f61 [4194304,4194304] 0 2024-04-23T02:14:14.840 INFO:tasks.workunit.client.0.smithi071.stdout:8/892: symlink dc/d3a/l11f 0 2024-04-23T02:14:14.842 INFO:tasks.workunit.client.0.smithi071.stdout:2/781: rename d0/d2/d26/d73/led to d0/d2/d49/l105 0 2024-04-23T02:14:14.845 INFO:tasks.workunit.client.0.smithi071.stdout:8/893: rename dc/d15/d50/fa9 to dc/d56/d64/f120 0 2024-04-23T02:14:14.846 INFO:tasks.workunit.client.0.smithi071.stdout:2/782: symlink d0/d2/d45/l106 0 2024-04-23T02:14:14.847 INFO:tasks.workunit.client.0.smithi071.stdout:2/783: chown d0/d2/d34/d55/f72 8562 1 2024-04-23T02:14:14.848 INFO:tasks.workunit.client.0.smithi071.stdout:2/784: stat d0/d2/d26/d3a/dc3/d36 0 2024-04-23T02:14:14.848 INFO:tasks.workunit.client.0.smithi071.stdout:8/894: mknod dc/d3a/da4/c121 0 2024-04-23T02:14:14.849 INFO:tasks.workunit.client.0.smithi071.stdout:4/933: chown d2/d2a/d2d/d39/d42/d49/d67/d7e/f121 229528145 1 2024-04-23T02:14:14.854 INFO:tasks.workunit.client.0.smithi071.stdout:8/895: rename dc/d56/de1/dea/ffa to dc/d15/da0/d32/d63/f122 0 2024-04-23T02:14:14.855 INFO:tasks.workunit.client.0.smithi071.stdout:2/785: fdatasync d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/fad 0 2024-04-23T02:14:14.856 INFO:tasks.workunit.client.0.smithi071.stdout:4/934: mkdir d2/d2a/d2d/d39/d42/d49/d91/ddb/d12b 0 2024-04-23T02:14:14.857 INFO:tasks.workunit.client.0.smithi071.stdout:2/786: chown d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/f61 3 1 2024-04-23T02:14:14.858 INFO:tasks.workunit.client.0.smithi071.stdout:8/896: rename dc/d78 to dc/d15/da0/d8d/d9a/d123 0 2024-04-23T02:14:14.861 INFO:tasks.workunit.client.0.smithi071.stdout:0/941: dread d0/d3/de/d10/d14/d32/dac/fae [0,4194304] 0 2024-04-23T02:14:14.862 INFO:tasks.workunit.client.0.smithi071.stdout:8/897: mknod dc/d56/d64/dde/c124 0 2024-04-23T02:14:14.864 INFO:tasks.workunit.client.0.smithi071.stdout:2/787: creat d0/d2/d26/f107 x:0 0 0 2024-04-23T02:14:14.868 INFO:tasks.workunit.client.0.smithi071.stdout:8/898: unlink dc/d15/da0/d32/d63/dd4/ffd 0 2024-04-23T02:14:14.872 INFO:tasks.workunit.client.0.smithi071.stdout:8/899: symlink dc/d15/da0/d8d/df8/l125 0 2024-04-23T02:14:14.872 INFO:tasks.workunit.client.0.smithi071.stdout:0/942: write d0/d3/de/d10/d14/d32/d37/fec [1057280,37781] 0 2024-04-23T02:14:14.873 INFO:tasks.workunit.client.0.smithi071.stdout:4/935: dread d2/d2a/d2d/d39/d42/d49/d67/d7e/f121 [0,4194304] 0 2024-04-23T02:14:14.875 INFO:tasks.workunit.client.0.smithi071.stdout:8/900: chown dc/d3a/ce3 49698259 1 2024-04-23T02:14:14.876 INFO:tasks.workunit.client.0.smithi071.stdout:2/788: link d0/d2/d26/d3a/c88 d0/d70/c108 0 2024-04-23T02:14:14.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:14 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:14.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:14 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:14.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:14 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-23T02:14:14.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:14 smithi192 ceph-mon[49662]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-23T02:14:14.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:14 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:14:14.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:14 smithi192 ceph-mon[49662]: Upgrade: Need to upgrade myself (mgr.smithi192.qgecnv) 2024-04-23T02:14:14.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:14 smithi192 ceph-mon[49662]: pgmap v34: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 156 MiB/s rd, 348 MiB/s wr, 774 op/s 2024-04-23T02:14:14.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:14 smithi192 ceph-mon[49662]: Upgrade: Need to upgrade myself (mgr.smithi192.qgecnv) 2024-04-23T02:14:14.878 INFO:tasks.workunit.client.0.smithi071.stdout:8/901: write dc/f9f [4494118,18721] 0 2024-04-23T02:14:14.880 INFO:tasks.workunit.client.0.smithi071.stdout:8/902: chown dc/d15/d72/d95/ca3 855430 1 2024-04-23T02:14:14.881 INFO:tasks.workunit.client.0.smithi071.stdout:0/943: dwrite d0/ddf/f118 [0,4194304] 0 2024-04-23T02:14:14.888 INFO:tasks.workunit.client.0.smithi071.stdout:4/936: dwrite d2/d2a/d73/d5e/ff0 [0,4194304] 0 2024-04-23T02:14:14.889 INFO:tasks.workunit.client.0.smithi071.stdout:8/903: getdents dc/d15/d2d 0 2024-04-23T02:14:14.892 INFO:tasks.workunit.client.0.smithi071.stdout:2/789: dwrite d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/fd0 [0,4194304] 0 2024-04-23T02:14:14.898 INFO:tasks.workunit.client.0.smithi071.stdout:0/944: dwrite d0/d3/de/d10/d14/d32/d44/d31/fe9 [0,4194304] 0 2024-04-23T02:14:14.903 INFO:tasks.workunit.client.0.smithi071.stdout:0/945: rmdir d0/d3/de/d10/d14/d32/d44/d87 39 2024-04-23T02:14:14.903 INFO:tasks.workunit.client.0.smithi071.stdout:0/946: dread - d0/d3/de/d10/db3/fea zero size 2024-04-23T02:14:14.905 INFO:tasks.workunit.client.0.smithi071.stdout:0/947: chown d0/d3/de/d10/d14 871 1 2024-04-23T02:14:14.908 INFO:tasks.workunit.client.0.smithi071.stdout:0/948: fsync d0/d3/de/d10/d14/d32/d44/fd6 0 2024-04-23T02:14:14.909 INFO:tasks.workunit.client.0.smithi071.stdout:0/949: fdatasync d0/fda 0 2024-04-23T02:14:14.910 INFO:tasks.workunit.client.0.smithi071.stdout:8/904: dwrite dc/d3a/da4/db0/fd2 [0,4194304] 0 2024-04-23T02:14:14.913 INFO:tasks.workunit.client.0.smithi071.stdout:0/950: creat d0/d3/de/d137/f153 x:0 0 0 2024-04-23T02:14:14.914 INFO:tasks.workunit.client.0.smithi071.stdout:8/905: creat dc/d56/f126 x:0 0 0 2024-04-23T02:14:14.916 INFO:tasks.workunit.client.0.smithi071.stdout:8/906: write dc/d15/f87 [163547,73106] 0 2024-04-23T02:14:14.918 INFO:tasks.workunit.client.0.smithi071.stdout:8/907: fdatasync dc/d15/da0/d32/d63/dd4/f107 0 2024-04-23T02:14:14.919 INFO:tasks.workunit.client.0.smithi071.stdout:0/951: rename d0/d3/de/d10/d14/dd5 to d0/d3/d95/d114/d154 0 2024-04-23T02:14:14.919 INFO:tasks.workunit.client.0.smithi071.stdout:8/908: stat dc/d15/d72/d95/cb1 0 2024-04-23T02:14:14.922 INFO:tasks.workunit.client.0.smithi071.stdout:8/909: unlink dc/d15/d5d/fd6 0 2024-04-23T02:14:14.924 INFO:tasks.workunit.client.0.smithi071.stdout:0/952: creat d0/d3/de/d10/d14/d32/d44/d31/f155 x:0 0 0 2024-04-23T02:14:14.924 INFO:tasks.workunit.client.0.smithi071.stdout:8/910: dread - dc/d15/da0/fcb zero size 2024-04-23T02:14:14.931 INFO:tasks.workunit.client.0.smithi071.stdout:0/953: rename d0/d3/de/d10/d14/d32/d37/d26/c54 to d0/d3/d95/d114/d154/c156 0 2024-04-23T02:14:14.935 INFO:tasks.workunit.client.1.smithi192.stdout:1/948: sync 2024-04-23T02:14:14.936 INFO:tasks.workunit.client.0.smithi071.stdout:8/911: write dc/d15/da0/f1c [915835,120273] 0 2024-04-23T02:14:14.936 INFO:tasks.workunit.client.0.smithi071.stdout:0/954: link d0/d63/l151 d0/d3/de/d48/d7e/d8b/de3/l157 0 2024-04-23T02:14:14.938 INFO:tasks.workunit.client.0.smithi071.stdout:4/937: dread d2/d2a/d2d/d39/d42/d49/d67/d7e/dae/ffe [0,4194304] 0 2024-04-23T02:14:14.939 INFO:tasks.workunit.client.0.smithi071.stdout:8/912: creat dc/d15/f127 x:0 0 0 2024-04-23T02:14:14.940 INFO:tasks.workunit.client.0.smithi071.stdout:0/955: fsync d0/d3/de/d10/d14/d32/da8/fbc 0 2024-04-23T02:14:14.940 INFO:tasks.workunit.client.1.smithi192.stdout:1/949: write dc/d16/f64 [2997307,65582] 0 2024-04-23T02:14:14.941 INFO:tasks.workunit.client.0.smithi071.stdout:2/790: dread d0/f2e [4194304,4194304] 0 2024-04-23T02:14:14.943 INFO:tasks.workunit.client.0.smithi071.stdout:4/938: creat d2/d2a/da2/dd1/dde/d23/d29/f12c x:0 0 0 2024-04-23T02:14:14.944 INFO:tasks.workunit.client.0.smithi071.stdout:8/913: truncate dc/d15/f7e 641552 0 2024-04-23T02:14:14.947 INFO:tasks.workunit.client.0.smithi071.stdout:8/914: rmdir dc/d56/d64 39 2024-04-23T02:14:14.949 INFO:tasks.workunit.client.0.smithi071.stdout:2/791: symlink d0/d2/d49/l109 0 2024-04-23T02:14:14.952 INFO:tasks.workunit.client.0.smithi071.stdout:4/939: mkdir d2/d2a/d2d/d39/d42/d12d 0 2024-04-23T02:14:14.954 INFO:tasks.workunit.client.1.smithi192.stdout:1/950: dread dc/d14/f21 [4194304,4194304] 0 2024-04-23T02:14:14.955 INFO:tasks.workunit.client.0.smithi071.stdout:0/956: truncate d0/d3/de/f1d 831959 0 2024-04-23T02:14:14.955 INFO:tasks.workunit.client.0.smithi071.stdout:8/915: dwrite dc/d56/de1/ffb [0,4194304] 0 2024-04-23T02:14:14.956 INFO:tasks.workunit.client.1.smithi192.stdout:1/951: creat dc/d8b/f133 x:0 0 0 2024-04-23T02:14:14.957 INFO:tasks.workunit.client.1.smithi192.stdout:1/952: fsync dc/d8b/d34/d70/d8c/da3/de4/de8/f12c 0 2024-04-23T02:14:14.959 INFO:tasks.workunit.client.0.smithi071.stdout:8/916: chown dc/d56/d64/dde/l10e 8 1 2024-04-23T02:14:14.961 INFO:tasks.workunit.client.0.smithi071.stdout:0/957: symlink d0/d3/de/d10/d14/d32/d8a/l158 0 2024-04-23T02:14:14.961 INFO:tasks.workunit.client.0.smithi071.stdout:8/917: dread - dc/d15/d50/d5f/ff3 zero size 2024-04-23T02:14:14.961 INFO:tasks.workunit.client.0.smithi071.stdout:2/792: link d0/d2/d34/d55/lf9 d0/d2/d45/d58/l10a 0 2024-04-23T02:14:14.962 INFO:tasks.workunit.client.0.smithi071.stdout:4/940: dread d2/d2a/d2d/d39/d42/d49/d91/fc5 [0,4194304] 0 2024-04-23T02:14:14.962 INFO:tasks.workunit.client.0.smithi071.stdout:0/958: chown d0/d3/de/d10/d14/d32/d44/f43 404053424 1 2024-04-23T02:14:14.964 INFO:tasks.workunit.client.0.smithi071.stdout:2/793: unlink d0/d2/d34/c9e 0 2024-04-23T02:14:14.965 INFO:tasks.workunit.client.0.smithi071.stdout:8/918: getdents dc/d15/d5d/dc6/d105 0 2024-04-23T02:14:14.966 INFO:tasks.workunit.client.0.smithi071.stdout:0/959: truncate d0/d3/de/d10/db3/dba/f110 832962 0 2024-04-23T02:14:14.967 INFO:tasks.workunit.client.0.smithi071.stdout:4/941: symlink d2/d2a/d2d/d39/d42/d49/d91/l12e 0 2024-04-23T02:14:14.970 INFO:tasks.workunit.client.1.smithi192.stdout:1/953: dread dc/f39 [0,4194304] 0 2024-04-23T02:14:14.974 INFO:tasks.workunit.client.0.smithi071.stdout:2/794: rename d0/d2/d34/d55/lf9 to d0/d2/d26/d73/l10b 0 2024-04-23T02:14:14.978 INFO:tasks.workunit.client.0.smithi071.stdout:4/942: rename d2/d2a/f3b to d2/d2a/da2/f12f 0 2024-04-23T02:14:14.981 INFO:tasks.workunit.client.1.smithi192.stdout:1/954: dwrite dc/d16/d19/d30/f89 [0,4194304] 0 2024-04-23T02:14:14.982 INFO:tasks.workunit.client.0.smithi071.stdout:4/943: dread d2/d2a/d2d/d39/f63 [0,4194304] 0 2024-04-23T02:14:14.983 INFO:tasks.workunit.client.0.smithi071.stdout:0/960: rename d0/ddf/d148/f91 to d0/d3/de/d48/d7e/d8b/de3/f159 0 2024-04-23T02:14:14.983 INFO:tasks.workunit.client.0.smithi071.stdout:4/944: read - d2/d53/fc0 zero size 2024-04-23T02:14:14.983 INFO:tasks.workunit.client.1.smithi192.stdout:1/955: mkdir dc/d16/d19/d30/d134 0 2024-04-23T02:14:14.986 INFO:tasks.workunit.client.0.smithi071.stdout:8/919: write dc/f54 [2966239,54850] 0 2024-04-23T02:14:14.988 INFO:tasks.workunit.client.1.smithi192.stdout:1/956: rename dc/d80/fac to dc/d16/d19/d30/dc4/de3/f135 0 2024-04-23T02:14:14.988 INFO:tasks.workunit.client.0.smithi071.stdout:2/795: link d0/l39 d0/d2/d26/d3a/dc3/d36/d5c/d85/de9/dfc/l10c 0 2024-04-23T02:14:14.989 INFO:tasks.workunit.client.0.smithi071.stdout:8/920: truncate dc/d15/da0/d55/f106 73076 0 2024-04-23T02:14:14.990 INFO:tasks.workunit.client.0.smithi071.stdout:4/945: mknod d2/d2a/d2d/d39/d42/d49/d67/c130 0 2024-04-23T02:14:14.990 INFO:tasks.workunit.client.1.smithi192.stdout:1/957: creat dc/d16/dde/de9/dfb/f136 x:0 0 0 2024-04-23T02:14:14.992 INFO:tasks.workunit.client.0.smithi071.stdout:0/961: dwrite d0/d3/de/d10/d14/d32/d8a/df0/f11d [0,4194304] 0 2024-04-23T02:14:14.993 INFO:tasks.workunit.client.0.smithi071.stdout:8/921: chown dc/d15/d50/d5f/c67 762390 1 2024-04-23T02:14:14.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:14 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:14.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:14 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:14.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:14 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-23T02:14:14.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:14 smithi071 ceph-mon[39837]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-23T02:14:14.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:14 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:14:14.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:14 smithi071 ceph-mon[39837]: Upgrade: Need to upgrade myself (mgr.smithi192.qgecnv) 2024-04-23T02:14:14.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:14 smithi071 ceph-mon[39837]: pgmap v34: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 156 MiB/s rd, 348 MiB/s wr, 774 op/s 2024-04-23T02:14:14.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:14 smithi071 ceph-mon[39837]: Upgrade: Need to upgrade myself (mgr.smithi192.qgecnv) 2024-04-23T02:14:14.995 INFO:tasks.workunit.client.0.smithi071.stdout:0/962: fdatasync d0/d3/d95/d114/f135 0 2024-04-23T02:14:14.997 INFO:tasks.workunit.client.0.smithi071.stdout:4/946: creat d2/d2a/da2/dd1/dde/d23/d4f/db5/de7/f131 x:0 0 0 2024-04-23T02:14:14.997 INFO:tasks.workunit.client.0.smithi071.stdout:0/963: fdatasync d0/d3/de/d10/d14/d32/da8/f12c 0 2024-04-23T02:14:14.999 INFO:tasks.workunit.client.0.smithi071.stdout:0/964: chown d0/d3/de/d10/dfc/l140 102 1 2024-04-23T02:14:15.005 INFO:tasks.workunit.client.0.smithi071.stdout:0/965: rename d0/d3/de/d10/d14/l12e to d0/d3/de/d10/d14/d32/d8a/l15a 0 2024-04-23T02:14:15.007 INFO:tasks.workunit.client.0.smithi071.stdout:4/947: link d2/d2a/d2d/d39/d42/d49/d67/d7e/dae/f118 d2/d2a/da2/dd1/dde/d23/d10/f132 0 2024-04-23T02:14:15.008 INFO:tasks.workunit.client.0.smithi071.stdout:2/796: write d0/d2/d34/d55/f72 [1996308,126732] 0 2024-04-23T02:14:15.011 INFO:tasks.workunit.client.0.smithi071.stdout:4/948: unlink d2/d2a/d2d/d39/d42/d49/d91/fcb 0 2024-04-23T02:14:15.011 INFO:tasks.workunit.client.0.smithi071.stdout:9/882: sync 2024-04-23T02:14:15.016 INFO:tasks.workunit.client.0.smithi071.stdout:0/966: dwrite d0/ddf/d148/f13e [0,4194304] 0 2024-04-23T02:14:15.021 INFO:tasks.workunit.client.0.smithi071.stdout:8/922: dread dc/d15/da0/d55/f65 [0,4194304] 0 2024-04-23T02:14:15.023 INFO:tasks.workunit.client.0.smithi071.stdout:4/949: creat d2/d2a/d2d/d39/d42/d49/d67/d7e/f133 x:0 0 0 2024-04-23T02:14:15.024 INFO:tasks.workunit.client.0.smithi071.stdout:8/923: readlink dc/d15/da0/d8d/d9a/dad/lf6 0 2024-04-23T02:14:15.026 INFO:tasks.workunit.client.0.smithi071.stdout:4/950: read d2/f10e [648806,24033] 0 2024-04-23T02:14:15.027 INFO:tasks.workunit.client.0.smithi071.stdout:2/797: link d0/d2/d26/d73/l10b d0/d2/d26/d3a/l10d 0 2024-04-23T02:14:15.027 INFO:tasks.workunit.client.0.smithi071.stdout:9/883: dwrite d1/d32/d41/db7/d5a/dc6/faa [0,4194304] 0 2024-04-23T02:14:15.030 INFO:tasks.workunit.client.0.smithi071.stdout:2/798: rmdir d0/d2/d26/d3a 39 2024-04-23T02:14:15.034 INFO:tasks.workunit.client.0.smithi071.stdout:4/951: creat d2/d2a/d2d/d39/d42/d49/d91/ddb/d10d/f134 x:0 0 0 2024-04-23T02:14:15.036 INFO:tasks.workunit.client.0.smithi071.stdout:9/884: truncate d1/d32/d41/db7/d5a/dc6/f103 191107 0 2024-04-23T02:14:15.040 INFO:tasks.workunit.client.0.smithi071.stdout:2/799: symlink d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/l10e 0 2024-04-23T02:14:15.045 INFO:tasks.workunit.client.0.smithi071.stdout:9/885: dwrite d1/d32/d41/f10f [0,4194304] 0 2024-04-23T02:14:15.045 INFO:tasks.workunit.client.0.smithi071.stdout:4/952: rename d2/d2a/da2/dd1/dde/d23/l4a to d2/d2a/d73/d5e/l135 0 2024-04-23T02:14:15.048 INFO:tasks.workunit.client.0.smithi071.stdout:9/886: chown d1/f9a 5 1 2024-04-23T02:14:15.050 INFO:tasks.workunit.client.0.smithi071.stdout:8/924: dwrite dc/d15/da0/d32/d63/ff0 [0,4194304] 0 2024-04-23T02:14:15.055 INFO:tasks.workunit.client.0.smithi071.stdout:2/800: write d0/d2/d26/d3a/f77 [722420,18578] 0 2024-04-23T02:14:15.055 INFO:tasks.workunit.client.0.smithi071.stdout:9/887: truncate d1/d32/d41/db7/d71/f118 808870 0 2024-04-23T02:14:15.058 INFO:tasks.workunit.client.0.smithi071.stdout:8/925: mkdir dc/d15/d128 0 2024-04-23T02:14:15.059 INFO:tasks.workunit.client.0.smithi071.stdout:4/953: dwrite d2/d2a/da2/dd1/dde/f6d [0,4194304] 0 2024-04-23T02:14:15.060 INFO:tasks.workunit.client.0.smithi071.stdout:8/926: dread - dc/d56/f126 zero size 2024-04-23T02:14:15.061 INFO:tasks.workunit.client.0.smithi071.stdout:9/888: fsync d1/d32/d3a/d90/d99/fcf 0 2024-04-23T02:14:15.063 INFO:tasks.workunit.client.0.smithi071.stdout:9/889: chown d1/d4e/d66/da4/cc9 11136 1 2024-04-23T02:14:15.065 INFO:tasks.workunit.client.0.smithi071.stdout:4/954: creat d2/d2a/d2d/f136 x:0 0 0 2024-04-23T02:14:15.070 INFO:tasks.workunit.client.0.smithi071.stdout:9/890: symlink d1/d32/d3a/d77/df8/l13c 0 2024-04-23T02:14:15.070 INFO:tasks.workunit.client.0.smithi071.stdout:4/955: dread d2/d2a/d73/d5e/ff0 [0,4194304] 0 2024-04-23T02:14:15.073 INFO:tasks.workunit.client.0.smithi071.stdout:2/801: write d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/f69 [3426155,86925] 0 2024-04-23T02:14:15.077 INFO:tasks.workunit.client.0.smithi071.stdout:8/927: truncate dc/f9f 4277410 0 2024-04-23T02:14:15.080 INFO:tasks.workunit.client.0.smithi071.stdout:4/956: dread d2/d2a/d2d/f5c [0,4194304] 0 2024-04-23T02:14:15.081 INFO:tasks.workunit.client.0.smithi071.stdout:2/802: dwrite d0/f1e [4194304,4194304] 0 2024-04-23T02:14:15.087 INFO:tasks.workunit.client.0.smithi071.stdout:8/928: dwrite dc/d15/f86 [0,4194304] 0 2024-04-23T02:14:15.087 INFO:tasks.workunit.client.0.smithi071.stdout:2/803: chown d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/f65 157 1 2024-04-23T02:14:15.089 INFO:tasks.workunit.client.0.smithi071.stdout:2/804: write d0/d2/d26/d14/f42 [4980152,7367] 0 2024-04-23T02:14:15.090 INFO:tasks.workunit.client.0.smithi071.stdout:8/929: symlink dc/d56/d64/dde/l129 0 2024-04-23T02:14:15.094 INFO:tasks.workunit.client.0.smithi071.stdout:8/930: rmdir dc/d15/d50/d5f/d102 39 2024-04-23T02:14:15.097 INFO:tasks.workunit.client.0.smithi071.stdout:8/931: mkdir dc/d56/d64/dde/d12a 0 2024-04-23T02:14:15.099 INFO:tasks.workunit.client.0.smithi071.stdout:4/957: dwrite d2/d2a/d2d/d39/d42/d49/d67/d3e/f4d [4194304,4194304] 0 2024-04-23T02:14:15.100 INFO:tasks.workunit.client.0.smithi071.stdout:8/932: dread dc/d56/d64/fc8 [0,4194304] 0 2024-04-23T02:14:15.108 INFO:tasks.workunit.client.0.smithi071.stdout:4/958: dwrite d2/d2a/d2d/d39/d42/d49/d67/d3e/f93 [0,4194304] 0 2024-04-23T02:14:15.110 INFO:tasks.workunit.client.0.smithi071.stdout:4/959: readlink d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/dc8/le2 0 2024-04-23T02:14:15.111 INFO:tasks.workunit.client.0.smithi071.stdout:4/960: stat d2/d2a/d73/d5e 0 2024-04-23T02:14:15.116 INFO:tasks.workunit.client.0.smithi071.stdout:2/805: dwrite d0/d2/d26/d14/f1a [0,4194304] 0 2024-04-23T02:14:15.119 INFO:tasks.workunit.client.0.smithi071.stdout:2/806: mkdir d0/d2/d26/d3a/dc3/d36/d5c/dfe/d10f 0 2024-04-23T02:14:15.120 INFO:tasks.workunit.client.0.smithi071.stdout:2/807: chown d0/d70 96334 1 2024-04-23T02:14:15.123 INFO:tasks.workunit.client.0.smithi071.stdout:4/961: write d2/d2a/d2d/d39/d42/fcd [1041333,86172] 0 2024-04-23T02:14:15.127 INFO:tasks.workunit.client.0.smithi071.stdout:8/933: dwrite dc/d15/da0/d8d/d9a/f113 [0,4194304] 0 2024-04-23T02:14:15.129 INFO:tasks.workunit.client.0.smithi071.stdout:2/808: dwrite d0/d2/d26/d3a/dc3/f95 [0,4194304] 0 2024-04-23T02:14:15.130 INFO:tasks.workunit.client.0.smithi071.stdout:4/962: mkdir d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/dc8/deb/d137 0 2024-04-23T02:14:15.133 INFO:tasks.workunit.client.0.smithi071.stdout:2/809: dread - d0/d2/d34/d83/fc9 zero size 2024-04-23T02:14:15.134 INFO:tasks.workunit.client.1.smithi192.stdout:1/958: sync 2024-04-23T02:14:15.135 INFO:tasks.workunit.client.0.smithi071.stdout:4/963: mknod d2/d2a/da2/dd1/c138 0 2024-04-23T02:14:15.135 INFO:tasks.workunit.client.0.smithi071.stdout:4/964: dread - d2/d2a/da2/f117 zero size 2024-04-23T02:14:15.136 INFO:tasks.workunit.client.0.smithi071.stdout:2/810: mknod d0/d2/d26/d3a/dc3/c110 0 2024-04-23T02:14:15.137 INFO:tasks.workunit.client.0.smithi071.stdout:8/934: write dc/d15/da0/d32/fbf [708211,123279] 0 2024-04-23T02:14:15.138 INFO:tasks.workunit.client.1.smithi192.stdout:1/959: rmdir dc/d8b/d34/d70/d8c/da3/dc1 39 2024-04-23T02:14:15.139 INFO:tasks.workunit.client.0.smithi071.stdout:4/965: write d2/d2a/d2d/d39/d42/d49/d91/ddb/d10d/f11c [715693,68226] 0 2024-04-23T02:14:15.140 INFO:tasks.workunit.client.1.smithi192.stdout:1/960: chown dc/d16/d19/d30/l75 451 1 2024-04-23T02:14:15.142 INFO:tasks.workunit.client.0.smithi071.stdout:8/935: symlink dc/d15/da0/d8d/d9a/d123/d11a/l12b 0 2024-04-23T02:14:15.142 INFO:tasks.workunit.client.1.smithi192.stdout:1/961: stat dc/d16/d19/d30/dc4/fd9 0 2024-04-23T02:14:15.144 INFO:tasks.workunit.client.1.smithi192.stdout:1/962: creat dc/d16/db0/f137 x:0 0 0 2024-04-23T02:14:15.145 INFO:tasks.workunit.client.0.smithi071.stdout:8/936: creat dc/d15/da0/d8d/d9a/d123/f12c x:0 0 0 2024-04-23T02:14:15.145 INFO:tasks.workunit.client.0.smithi071.stdout:4/966: link d2/d2a/da2/dd1/dde/d23/d10/c4e d2/d2a/da2/c139 0 2024-04-23T02:14:15.146 INFO:tasks.workunit.client.1.smithi192.stdout:1/963: unlink dc/d16/d19/d74/df4/f127 0 2024-04-23T02:14:15.147 INFO:tasks.workunit.client.0.smithi071.stdout:4/967: fsync d2/d53/fa5 0 2024-04-23T02:14:15.148 INFO:tasks.workunit.client.0.smithi071.stdout:2/811: link d0/d2/d45/c7e d0/d2/d34/d55/d96/c111 0 2024-04-23T02:14:15.149 INFO:tasks.workunit.client.0.smithi071.stdout:8/937: symlink dc/d15/d128/l12d 0 2024-04-23T02:14:15.150 INFO:tasks.workunit.client.0.smithi071.stdout:8/938: fdatasync dc/d15/da0/d8d/d9a/d123/f12c 0 2024-04-23T02:14:15.151 INFO:tasks.workunit.client.0.smithi071.stdout:4/968: creat d2/d2a/da2/dd1/dde/d57/df2/f13a x:0 0 0 2024-04-23T02:14:15.151 INFO:tasks.workunit.client.1.smithi192.stdout:1/964: link dc/d8b/d34/d70/d8c/da3/de4/f12d dc/d8b/f138 0 2024-04-23T02:14:15.152 INFO:tasks.workunit.client.1.smithi192.stdout:1/965: dread - dc/d8b/fa0 zero size 2024-04-23T02:14:15.154 INFO:tasks.workunit.client.0.smithi071.stdout:2/812: creat d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/f112 x:0 0 0 2024-04-23T02:14:15.155 INFO:tasks.workunit.client.0.smithi071.stdout:8/939: truncate dc/d15/da0/f3d 4934521 0 2024-04-23T02:14:15.159 INFO:tasks.workunit.client.0.smithi071.stdout:8/940: write dc/d15/da0/d8d/d9a/dad/ff5 [661020,19956] 0 2024-04-23T02:14:15.160 INFO:tasks.workunit.client.0.smithi071.stdout:4/969: dwrite d2/d2a/d2d/d39/f106 [0,4194304] 0 2024-04-23T02:14:15.163 INFO:tasks.workunit.client.0.smithi071.stdout:8/941: rmdir dc/d15/da0/d32 39 2024-04-23T02:14:15.165 INFO:tasks.workunit.client.0.smithi071.stdout:2/813: link d0/d2/d26/d3a/dc3/d36/fbe d0/d2/da0/db7/f113 0 2024-04-23T02:14:15.166 INFO:tasks.workunit.client.1.smithi192.stdout:1/966: dread dc/d16/d19/d74/d77/f91 [0,4194304] 0 2024-04-23T02:14:15.171 INFO:tasks.workunit.client.0.smithi071.stdout:4/970: getdents d2/d2a/d2d/d39/d42/d49/d67/d3e 0 2024-04-23T02:14:15.172 INFO:tasks.workunit.client.1.smithi192.stdout:1/967: creat dc/d16/d19/f139 x:0 0 0 2024-04-23T02:14:15.173 INFO:tasks.workunit.client.0.smithi071.stdout:4/971: rmdir d2/d2a/da2/dd1/dde/d23/d4f/db5/db6 39 2024-04-23T02:14:15.174 INFO:tasks.workunit.client.1.smithi192.stdout:1/968: write dc/d14/d23/d5e/dab/f12b [371727,60793] 0 2024-04-23T02:14:15.175 INFO:tasks.workunit.client.1.smithi192.stdout:1/969: chown dc/d8b/f4d 99203 1 2024-04-23T02:14:15.177 INFO:tasks.workunit.client.0.smithi071.stdout:4/972: getdents d2/d2a/d2d/d39/d42/d49/d67 0 2024-04-23T02:14:15.178 INFO:tasks.workunit.client.0.smithi071.stdout:4/973: chown d2/d2a/d2d/d39/d42/d49/d91/lc6 649881 1 2024-04-23T02:14:15.184 INFO:tasks.workunit.client.0.smithi071.stdout:8/942: dwrite dc/f9f [0,4194304] 0 2024-04-23T02:14:15.187 INFO:tasks.workunit.client.0.smithi071.stdout:8/943: dread - dc/d15/fe8 zero size 2024-04-23T02:14:15.188 INFO:tasks.workunit.client.0.smithi071.stdout:4/974: dwrite d2/d2a/da2/dd1/dde/d23/d4f/f107 [0,4194304] 0 2024-04-23T02:14:15.189 INFO:tasks.workunit.client.0.smithi071.stdout:2/814: dread d0/d2/d26/d3a/d75/f8f [0,4194304] 0 2024-04-23T02:14:15.192 INFO:tasks.workunit.client.0.smithi071.stdout:2/815: dread - d0/fab zero size 2024-04-23T02:14:15.196 INFO:tasks.workunit.client.0.smithi071.stdout:2/816: creat d0/d2/d34/d83/f114 x:0 0 0 2024-04-23T02:14:15.197 INFO:tasks.workunit.client.0.smithi071.stdout:8/944: dwrite dc/d15/d72/d95/f114 [0,4194304] 0 2024-04-23T02:14:15.199 INFO:tasks.workunit.client.0.smithi071.stdout:2/817: creat d0/d2/d26/d3a/d75/f115 x:0 0 0 2024-04-23T02:14:15.202 INFO:tasks.workunit.client.0.smithi071.stdout:8/945: link dc/d15/d50/d5f/l90 dc/d15/d128/l12e 0 2024-04-23T02:14:15.220 INFO:tasks.workunit.client.0.smithi071.stdout:4/975: dwrite d2/d2a/d73/f7f [0,4194304] 0 2024-04-23T02:14:15.223 INFO:tasks.workunit.client.0.smithi071.stdout:4/976: truncate d2/d2a/da2/dd1/dde/d23/d29/f9e 8999783 0 2024-04-23T02:14:15.234 INFO:tasks.workunit.client.0.smithi071.stdout:4/977: readlink d2/d2a/d73/d5e/l135 0 2024-04-23T02:14:15.236 INFO:tasks.workunit.client.0.smithi071.stdout:4/978: fsync d2/d2a/d2d/f54 0 2024-04-23T02:14:15.245 INFO:tasks.workunit.client.0.smithi071.stdout:8/946: dread dc/d56/d64/fd7 [0,4194304] 0 2024-04-23T02:14:15.250 INFO:tasks.workunit.client.0.smithi071.stdout:4/979: truncate d2/d2a/d2d/d39/d42/fcd 98150 0 2024-04-23T02:14:15.251 INFO:tasks.workunit.client.0.smithi071.stdout:8/947: write dc/d15/d50/fec [467295,79669] 0 2024-04-23T02:14:15.254 INFO:tasks.workunit.client.0.smithi071.stdout:4/980: mknod d2/d2a/d2d/d39/d42/d49/d67/d7e/dae/c13b 0 2024-04-23T02:14:15.256 INFO:tasks.workunit.client.0.smithi071.stdout:4/981: creat d2/d2a/da2/dd1/dde/d57/f13c x:0 0 0 2024-04-23T02:14:15.258 INFO:tasks.workunit.client.0.smithi071.stdout:4/982: truncate d2/d2a/d2d/f66 957799 0 2024-04-23T02:14:15.260 INFO:tasks.workunit.client.0.smithi071.stdout:4/983: truncate d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/fd5 316855 0 2024-04-23T02:14:15.262 INFO:tasks.workunit.client.0.smithi071.stdout:4/984: symlink d2/d2a/d73/d5e/def/l13d 0 2024-04-23T02:14:15.264 INFO:tasks.workunit.client.0.smithi071.stdout:4/985: write d2/d2a/d2d/d39/d42/d49/d67/d3e/f4d [5639604,102900] 0 2024-04-23T02:14:15.266 INFO:tasks.workunit.client.0.smithi071.stdout:4/986: fsync d2/d2a/da2/dd1/dde/d23/f5b 0 2024-04-23T02:14:15.267 INFO:tasks.workunit.client.0.smithi071.stdout:8/948: dread dc/d15/da0/d32/fb9 [0,4194304] 0 2024-04-23T02:14:15.270 INFO:tasks.workunit.client.0.smithi071.stdout:4/987: fsync d2/d2a/d2d/d39/d42/d49/d67/d7e/dae/ffe 0 2024-04-23T02:14:15.271 INFO:tasks.workunit.client.0.smithi071.stdout:8/949: truncate dc/d15/da0/d55/f98 1252332 0 2024-04-23T02:14:15.271 INFO:tasks.workunit.client.0.smithi071.stdout:4/988: chown d2/d2a/d2d/d39/d42/cce 87 1 2024-04-23T02:14:15.276 INFO:tasks.workunit.client.0.smithi071.stdout:8/950: write dc/f20 [502312,74965] 0 2024-04-23T02:14:15.276 INFO:tasks.workunit.client.0.smithi071.stdout:8/951: stat dc/d56/d64/dde 0 2024-04-23T02:14:15.276 INFO:tasks.workunit.client.0.smithi071.stdout:4/989: truncate d2/d2a/da2/dd1/dde/d23/d29/f83 501668 0 2024-04-23T02:14:15.278 INFO:tasks.workunit.client.1.smithi192.stdout:1/970: sync 2024-04-23T02:14:15.278 INFO:tasks.workunit.client.0.smithi071.stdout:8/952: unlink dc/d15/da0/c21 0 2024-04-23T02:14:15.280 INFO:tasks.workunit.client.0.smithi071.stdout:4/990: dread d2/d53/fa5 [0,4194304] 0 2024-04-23T02:14:15.283 INFO:tasks.workunit.client.0.smithi071.stdout:4/991: creat d2/d2a/d2d/d39/d42/d49/d91/ddb/d10d/d119/f13e x:0 0 0 2024-04-23T02:14:15.284 INFO:tasks.workunit.client.0.smithi071.stdout:8/953: rmdir dc/d15/d5d/de4 0 2024-04-23T02:14:15.288 INFO:tasks.workunit.client.0.smithi071.stdout:4/992: link d2/l11 d2/d2a/da2/dd1/dde/d23/d10/l13f 0 2024-04-23T02:14:15.290 INFO:tasks.workunit.client.1.smithi192.stdout:1/971: dread dc/d14/d23/f50 [4194304,4194304] 0 2024-04-23T02:14:15.292 INFO:tasks.workunit.client.1.smithi192.stdout:1/972: creat dc/d16/d95/f13a x:0 0 0 2024-04-23T02:14:15.295 INFO:tasks.workunit.client.1.smithi192.stdout:1/973: write dc/d8b/d34/d70/d8c/da3/dc1/f109 [124790,25040] 0 2024-04-23T02:14:15.302 INFO:tasks.workunit.client.0.smithi071.stdout:4/993: dread d2/d2a/d2d/f66 [0,4194304] 0 2024-04-23T02:14:15.302 INFO:tasks.workunit.client.0.smithi071.stdout:8/954: dread dc/d15/d2d/f38 [0,4194304] 0 2024-04-23T02:14:15.304 INFO:tasks.workunit.client.0.smithi071.stdout:8/955: getdents dc/d56/de1/dea/d112 0 2024-04-23T02:14:15.308 INFO:tasks.workunit.client.0.smithi071.stdout:8/956: dread dc/d15/da0/d32/fb9 [0,4194304] 0 2024-04-23T02:14:15.310 INFO:tasks.workunit.client.1.smithi192.stdout:1/974: dwrite dc/d16/d19/d74/faf [0,4194304] 0 2024-04-23T02:14:15.329 INFO:tasks.workunit.client.1.smithi192.stdout:1/975: dread dc/d16/d95/fdc [0,4194304] 0 2024-04-23T02:14:15.335 INFO:tasks.workunit.client.1.smithi192.stdout:1/976: dwrite dc/d14/d15/f54 [4194304,4194304] 0 2024-04-23T02:14:15.338 INFO:tasks.workunit.client.1.smithi192.stdout:1/977: truncate dc/d16/d19/d74/fb6 3837186 0 2024-04-23T02:14:15.340 INFO:tasks.workunit.client.1.smithi192.stdout:1/978: symlink dc/d16/d132/d118/l13b 0 2024-04-23T02:14:15.342 INFO:tasks.workunit.client.1.smithi192.stdout:1/979: symlink dc/d80/d81/l13c 0 2024-04-23T02:14:15.348 INFO:tasks.workunit.client.1.smithi192.stdout:1/980: dwrite dc/d16/d19/d74/d77/f91 [0,4194304] 0 2024-04-23T02:14:15.350 INFO:tasks.workunit.client.1.smithi192.stdout:1/981: rmdir dc/d8b/d34/d70 39 2024-04-23T02:14:15.351 INFO:tasks.workunit.client.1.smithi192.stdout:1/982: rmdir dc/d14/d15 39 2024-04-23T02:14:15.357 INFO:tasks.workunit.client.1.smithi192.stdout:1/983: rename dc/d16/d19/d74/dd2/l102 to dc/d14/d23/de0/l13d 0 2024-04-23T02:14:15.359 INFO:tasks.workunit.client.1.smithi192.stdout:1/984: fdatasync dc/d8b/d34/d70/d8c/da3/de4/f12d 0 2024-04-23T02:14:15.362 INFO:tasks.workunit.client.1.smithi192.stdout:1/985: getdents dc/d80 0 2024-04-23T02:14:15.368 INFO:tasks.workunit.client.1.smithi192.stdout:1/986: dwrite dc/d8b/d34/d70/d8c/fa4 [0,4194304] 0 2024-04-23T02:14:15.370 INFO:tasks.workunit.client.1.smithi192.stdout:1/987: creat dc/d16/d19/d74/dd2/f13e x:0 0 0 2024-04-23T02:14:15.371 INFO:tasks.workunit.client.1.smithi192.stdout:1/988: symlink dc/d14/d23/da2/l13f 0 2024-04-23T02:14:15.373 INFO:tasks.workunit.client.1.smithi192.stdout:1/989: mknod dc/d80/d81/c140 0 2024-04-23T02:14:15.374 INFO:tasks.workunit.client.1.smithi192.stdout:1/990: creat dc/d16/d19/d30/dc4/f141 x:0 0 0 2024-04-23T02:14:15.380 INFO:tasks.workunit.client.1.smithi192.stdout:1/991: dwrite dc/d16/db0/fd8 [4194304,4194304] 0 2024-04-23T02:14:15.383 INFO:tasks.workunit.client.1.smithi192.stdout:1/992: fdatasync dc/d14/d15/d35/f40 0 2024-04-23T02:14:15.404 INFO:tasks.workunit.client.0.smithi071.stdout:8/957: symlink dc/l12f 0 2024-04-23T02:14:15.409 INFO:tasks.workunit.client.0.smithi071.stdout:4/994: dwrite d2/d2a/da2/dd1/dde/d23/d29/f83 [0,4194304] 0 2024-04-23T02:14:15.413 INFO:tasks.workunit.client.0.smithi071.stdout:4/995: stat d2/d2a/d2d/d39/d42/d49/d67/d3e/da9/dc8/le2 0 2024-04-23T02:14:15.416 INFO:tasks.workunit.client.0.smithi071.stdout:8/958: dwrite dc/d56/fdc [0,4194304] 0 2024-04-23T02:14:15.416 INFO:tasks.workunit.client.0.smithi071.stdout:8/959: fsync dc/d15/f87 0 2024-04-23T02:14:15.417 INFO:tasks.workunit.client.0.smithi071.stdout:8/960: dread - dc/f118 zero size 2024-04-23T02:14:15.418 INFO:tasks.workunit.client.0.smithi071.stdout:4/996: rename d2/d2a/da2/cc7 to d2/d2a/d2d/d39/d42/d49/d91/ddb/c140 0 2024-04-23T02:14:15.422 INFO:tasks.workunit.client.0.smithi071.stdout:4/997: fdatasync d2/d2a/da2/dd1/dde/d23/f5d 0 2024-04-23T02:14:15.424 INFO:tasks.workunit.client.0.smithi071.stdout:4/998: write d2/d2a/d2d/d39/d42/d49/d67/d7e/f133 [114309,29686] 0 2024-04-23T02:14:15.424 INFO:tasks.workunit.client.1.smithi192.stdout:1/993: dwrite dc/d80/d81/f9e [0,4194304] 0 2024-04-23T02:14:15.426 INFO:tasks.workunit.client.1.smithi192.stdout:1/994: chown dc/d16/d19/d74/cc5 7999 1 2024-04-23T02:14:15.426 INFO:tasks.workunit.client.0.smithi071.stdout:8/961: getdents dc/d15/da0/d32/d63/dd4 0 2024-04-23T02:14:15.428 INFO:tasks.workunit.client.0.smithi071.stdout:8/962: write dc/d15/da0/f14 [835626,121152] 0 2024-04-23T02:14:15.430 INFO:tasks.workunit.client.0.smithi071.stdout:8/963: dread - dc/d15/da0/feb zero size 2024-04-23T02:14:15.431 INFO:tasks.workunit.client.1.smithi192.stdout:1/995: write dc/d8b/d62/f94 [3201003,130461] 0 2024-04-23T02:14:15.434 INFO:tasks.workunit.client.0.smithi071.stdout:4/999: dwrite d2/d2a/d73/d5e/ff0 [0,4194304] 0 2024-04-23T02:14:15.441 INFO:tasks.workunit.client.0.smithi071.stdout:8/964: dwrite dc/d15/da0/f14 [0,4194304] 0 2024-04-23T02:14:15.448 INFO:tasks.workunit.client.0.smithi071.stdout:8/965: dwrite dc/d15/d72/ff9 [0,4194304] 0 2024-04-23T02:14:15.450 INFO:tasks.workunit.client.0.smithi071.stdout:8/966: mknod dc/d15/da0/d30/c130 0 2024-04-23T02:14:15.482 INFO:tasks.workunit.client.0.smithi071.stdout:8/967: dread dc/d15/da0/f3d [0,4194304] 0 2024-04-23T02:14:15.484 INFO:tasks.workunit.client.0.smithi071.stdout:8/968: mkdir dc/d15/da0/d8d/d131 0 2024-04-23T02:14:15.485 INFO:tasks.workunit.client.0.smithi071.stdout:8/969: symlink dc/d15/da0/d8d/def/l132 0 2024-04-23T02:14:15.486 INFO:tasks.workunit.client.0.smithi071.stdout:8/970: write dc/f20 [1424299,62951] 0 2024-04-23T02:14:15.487 INFO:tasks.workunit.client.0.smithi071.stdout:8/971: chown dc/d15/da0/f3d 53 1 2024-04-23T02:14:15.492 INFO:tasks.workunit.client.0.smithi071.stdout:8/972: dwrite dc/d15/d5d/dc6/fe7 [0,4194304] 0 2024-04-23T02:14:15.494 INFO:tasks.workunit.client.0.smithi071.stdout:8/973: truncate dc/d15/da0/d8d/d9a/d123/f12c 661072 0 2024-04-23T02:14:15.498 INFO:tasks.workunit.client.0.smithi071.stdout:8/974: dread dc/d15/da0/d32/fbf [0,4194304] 0 2024-04-23T02:14:15.499 INFO:tasks.workunit.client.0.smithi071.stdout:8/975: truncate dc/d56/d64/f111 787215 0 2024-04-23T02:14:15.502 INFO:tasks.workunit.client.0.smithi071.stdout:8/976: read dc/d15/da0/d55/f80 [2562064,99028] 0 2024-04-23T02:14:15.502 INFO:tasks.workunit.client.0.smithi071.stdout:8/977: stat dc/d56/de1 0 2024-04-23T02:14:15.506 INFO:tasks.workunit.client.0.smithi071.stdout:8/978: write dc/d15/da0/d8d/fc3 [3253569,121580] 0 2024-04-23T02:14:15.511 INFO:tasks.workunit.client.0.smithi071.stdout:8/979: dwrite dc/d15/f23 [4194304,4194304] 0 2024-04-23T02:14:15.512 INFO:tasks.workunit.client.0.smithi071.stdout:8/980: chown dc/d15/da0/f115 75433 1 2024-04-23T02:14:15.517 INFO:tasks.workunit.client.0.smithi071.stdout:8/981: write dc/f57 [1855021,44991] 0 2024-04-23T02:14:15.520 INFO:tasks.workunit.client.1.smithi192.stdout:1/996: sync 2024-04-23T02:14:15.522 INFO:tasks.workunit.client.1.smithi192.stdout:1/997: unlink dc/d14/f130 0 2024-04-23T02:14:15.524 INFO:tasks.workunit.client.1.smithi192.stdout:1/998: mkdir dc/d80/d81/d142 0 2024-04-23T02:14:15.528 INFO:tasks.workunit.client.0.smithi071.stdout:8/982: dwrite dc/d15/fe8 [0,4194304] 0 2024-04-23T02:14:15.530 INFO:tasks.workunit.client.0.smithi071.stdout:8/983: write dc/d56/f126 [443054,92052] 0 2024-04-23T02:14:15.531 INFO:tasks.workunit.client.0.smithi071.stdout:8/984: symlink dc/d56/de1/dea/l133 0 2024-04-23T02:14:15.533 INFO:tasks.workunit.client.0.smithi071.stdout:8/985: mkdir dc/d15/da0/d8d/d9a/d123/d134 0 2024-04-23T02:14:15.534 INFO:tasks.workunit.client.1.smithi192.stdout:1/999: truncate dc/d16/d19/d30/f3f 8843896 0 2024-04-23T02:14:15.536 INFO:tasks.workunit.client.0.smithi071.stdout:8/986: truncate dc/d15/da0/d8d/d9a/dad/fb2 261174 0 2024-04-23T02:14:15.537 INFO:tasks.workunit.client.0.smithi071.stdout:0/967: sync 2024-04-23T02:14:15.537 INFO:tasks.workunit.client.0.smithi071.stdout:1/819: sync 2024-04-23T02:14:15.538 INFO:tasks.workunit.client.0.smithi071.stdout:6/956: sync 2024-04-23T02:14:15.538 INFO:tasks.workunit.client.0.smithi071.stdout:3/975: sync 2024-04-23T02:14:15.538 INFO:tasks.workunit.client.0.smithi071.stdout:8/987: creat dc/d15/d50/f135 x:0 0 0 2024-04-23T02:14:15.538 INFO:tasks.workunit.client.1.smithi192.stderr:+ rm -rf -- ./tmp.W1wvVtvA74 2024-04-23T02:14:15.540 INFO:tasks.workunit.client.0.smithi071.stdout:0/968: dread - d0/d3/de/d10/db3/f150 zero size 2024-04-23T02:14:15.540 INFO:tasks.workunit.client.0.smithi071.stdout:3/976: chown d7/d8/d42/d11b/f14f 251 1 2024-04-23T02:14:15.541 INFO:tasks.workunit.client.0.smithi071.stdout:1/820: unlink db/d11/d8d/f110 0 2024-04-23T02:14:15.542 INFO:tasks.workunit.client.0.smithi071.stdout:6/957: mknod d6/d1f/d46/d135/c13b 0 2024-04-23T02:14:15.544 INFO:tasks.workunit.client.0.smithi071.stdout:0/969: mknod d0/d92/d127/c15b 0 2024-04-23T02:14:15.546 INFO:tasks.workunit.client.0.smithi071.stdout:3/977: rename d7/d8/d42 to d7/da5/d128/d68/d3b/d4c/d166 0 2024-04-23T02:14:15.552 INFO:tasks.workunit.client.0.smithi071.stdout:1/821: mkdir db/d34/d106/d113 0 2024-04-23T02:14:15.554 INFO:tasks.workunit.client.0.smithi071.stdout:3/978: write d7/d22/d23/d26/d7e/ffa [3123522,90913] 0 2024-04-23T02:14:15.557 INFO:tasks.workunit.client.0.smithi071.stdout:0/970: dwrite d0/d3/de/d10/d14/d32/d8a/df0/f77 [0,4194304] 0 2024-04-23T02:14:15.557 INFO:tasks.workunit.client.0.smithi071.stdout:3/979: dread - d7/d22/def/d102/f161 zero size 2024-04-23T02:14:15.558 INFO:tasks.workunit.client.0.smithi071.stdout:6/958: write d6/d104/dad/f12c [287133,21544] 0 2024-04-23T02:14:15.560 INFO:tasks.workunit.client.0.smithi071.stdout:1/822: getdents db/df/d13/d2f/d48/d9f 0 2024-04-23T02:14:15.563 INFO:tasks.workunit.client.0.smithi071.stdout:8/988: dwrite dc/d15/da0/d8d/d9a/d123/fcc [0,4194304] 0 2024-04-23T02:14:15.564 INFO:tasks.workunit.client.0.smithi071.stdout:1/823: write db/df/d13/ddc/f112 [224631,32911] 0 2024-04-23T02:14:15.565 INFO:tasks.workunit.client.0.smithi071.stdout:3/980: dread d7/d22/d23/f6b [0,4194304] 0 2024-04-23T02:14:15.566 INFO:tasks.workunit.client.0.smithi071.stdout:8/989: chown dc/d15/d50/d5f/f108 39038164 1 2024-04-23T02:14:15.566 INFO:tasks.workunit.client.0.smithi071.stdout:0/971: truncate d0/d3/de/d10/d14/d32/d44/d59/f146 548719 0 2024-04-23T02:14:15.567 INFO:tasks.workunit.client.0.smithi071.stdout:1/824: dread - db/df/d13/d2f/f10d zero size 2024-04-23T02:14:15.569 INFO:tasks.workunit.client.0.smithi071.stdout:8/990: creat dc/d56/de1/dea/d112/f136 x:0 0 0 2024-04-23T02:14:15.572 INFO:tasks.workunit.client.0.smithi071.stdout:8/991: dread - dc/d15/da0/d32/d63/dd4/f107 zero size 2024-04-23T02:14:15.573 INFO:tasks.workunit.client.0.smithi071.stdout:1/825: mknod db/d11/d7f/da7/dd9/dfd/c114 0 2024-04-23T02:14:15.574 INFO:tasks.workunit.client.0.smithi071.stdout:6/959: write d6/d17/d77/d85/fa1 [4482104,44089] 0 2024-04-23T02:14:15.577 INFO:tasks.workunit.client.0.smithi071.stdout:8/992: truncate dc/d15/da0/f1c 3750064 0 2024-04-23T02:14:15.577 INFO:tasks.workunit.client.0.smithi071.stdout:6/960: readlink d6/d104/dd8/df2/l126 0 2024-04-23T02:14:15.578 INFO:tasks.workunit.client.0.smithi071.stdout:3/981: link d7/l77 d7/d10/l167 0 2024-04-23T02:14:15.579 INFO:tasks.workunit.client.0.smithi071.stdout:0/972: dwrite d0/d3/de/d10/d14/d32/d44/f115 [0,4194304] 0 2024-04-23T02:14:15.579 INFO:tasks.workunit.client.0.smithi071.stdout:1/826: readlink db/df/d5f/d6f/db2/d86/laa 0 2024-04-23T02:14:15.581 INFO:tasks.workunit.client.0.smithi071.stdout:6/961: chown d6/d1f/d46/d135/c10f 0 1 2024-04-23T02:14:15.582 INFO:tasks.workunit.client.0.smithi071.stdout:8/993: mknod dc/d15/d72/d95/c137 0 2024-04-23T02:14:15.583 INFO:tasks.workunit.client.0.smithi071.stdout:0/973: fsync d0/d3/de/d137/f153 0 2024-04-23T02:14:15.583 INFO:tasks.workunit.client.0.smithi071.stdout:6/962: write d6/d104/dad/f114 [385982,75656] 0 2024-04-23T02:14:15.584 INFO:tasks.workunit.client.0.smithi071.stdout:3/982: unlink d7/da5/d128/d68/d3b/d4c/d166/d47/l7b 0 2024-04-23T02:14:15.585 INFO:tasks.workunit.client.0.smithi071.stdout:0/974: stat d0/d3/de/d10/d14/d32/d37/lb1 0 2024-04-23T02:14:15.585 INFO:tasks.workunit.client.0.smithi071.stdout:1/827: mknod db/df/d13/d20/dc0/ddf/c115 0 2024-04-23T02:14:15.588 INFO:tasks.workunit.client.0.smithi071.stdout:8/994: creat dc/d15/d50/f138 x:0 0 0 2024-04-23T02:14:15.591 INFO:tasks.workunit.client.0.smithi071.stdout:3/983: symlink d7/dfd/l168 0 2024-04-23T02:14:15.592 INFO:tasks.workunit.client.0.smithi071.stdout:8/995: readlink dc/d15/d50/d5f/l90 0 2024-04-23T02:14:15.593 INFO:tasks.workunit.client.0.smithi071.stdout:1/828: symlink db/l116 0 2024-04-23T02:14:15.594 INFO:tasks.workunit.client.0.smithi071.stdout:0/975: rmdir d0/d3/de/d10/d14/d32/d44/d87 39 2024-04-23T02:14:15.596 INFO:tasks.workunit.client.0.smithi071.stdout:8/996: mknod dc/d15/da0/d8d/def/c139 0 2024-04-23T02:14:15.599 INFO:tasks.workunit.client.0.smithi071.stdout:1/829: chown db/df/d13/d2f/c2e 5616 1 2024-04-23T02:14:15.599 INFO:tasks.workunit.client.0.smithi071.stdout:8/997: write dc/d15/d50/d5f/f108 [4620646,57319] 0 2024-04-23T02:14:15.599 INFO:tasks.workunit.client.0.smithi071.stdout:3/984: dwrite d7/d22/d23/d26/d52/f14d [0,4194304] 0 2024-04-23T02:14:15.601 INFO:tasks.workunit.client.0.smithi071.stdout:0/976: creat d0/d3/de/d10/d14/d32/d44/d31/f15c x:0 0 0 2024-04-23T02:14:15.602 INFO:tasks.workunit.client.0.smithi071.stdout:6/963: write d6/d17/d77/d85/f80 [275745,74801] 0 2024-04-23T02:14:15.602 INFO:tasks.workunit.client.0.smithi071.stdout:8/998: write dc/d56/de1/ffb [2283907,99422] 0 2024-04-23T02:14:15.604 INFO:tasks.workunit.client.0.smithi071.stdout:9/891: sync 2024-04-23T02:14:15.605 INFO:tasks.workunit.client.0.smithi071.stdout:1/830: creat db/d11/d7f/da7/dd9/f117 x:0 0 0 2024-04-23T02:14:15.606 INFO:tasks.workunit.client.0.smithi071.stdout:3/985: readlink d7/d22/d23/d26/d7e/lf8 0 2024-04-23T02:14:15.607 INFO:tasks.workunit.client.0.smithi071.stdout:6/964: stat d6/d22/d92/l93 0 2024-04-23T02:14:15.608 INFO:tasks.workunit.client.0.smithi071.stdout:3/986: chown d7/d10/d38/f11d 28656054 1 2024-04-23T02:14:15.611 INFO:tasks.workunit.client.0.smithi071.stdout:1/831: mkdir db/df/d118 0 2024-04-23T02:14:15.612 INFO:tasks.workunit.client.0.smithi071.stdout:6/965: rmdir d6/d1f/d46/d135 39 2024-04-23T02:14:15.613 INFO:tasks.workunit.client.0.smithi071.stdout:1/832: dread - db/df/d13/d20/d75/d96/fd4 zero size 2024-04-23T02:14:15.616 INFO:tasks.workunit.client.0.smithi071.stdout:0/977: write d0/d3/de/d10/d14/d32/d44/d59/fe5 [579433,20506] 0 2024-04-23T02:14:15.618 INFO:tasks.workunit.client.0.smithi071.stdout:1/833: creat db/d34/d8a/f119 x:0 0 0 2024-04-23T02:14:15.620 INFO:tasks.workunit.client.0.smithi071.stdout:8/999: link dc/d3a/da4/c121 dc/d15/da0/d8d/d131/c13a 0 2024-04-23T02:14:15.620 INFO:tasks.workunit.client.0.smithi071.stdout:6/966: fdatasync d6/d22/f41 0 2024-04-23T02:14:15.621 INFO:tasks.workunit.client.0.smithi071.stdout:0/978: symlink d0/d3/de/d10/d14/d32/d8a/df0/l15d 0 2024-04-23T02:14:15.624 INFO:tasks.workunit.client.0.smithi071.stdout:9/892: dwrite d1/d32/d41/db7/d39/f8f [0,4194304] 0 2024-04-23T02:14:15.626 INFO:tasks.workunit.client.0.smithi071.stdout:3/987: write d7/d22/d23/d26/fa0 [299375,114658] 0 2024-04-23T02:14:15.628 INFO:tasks.workunit.client.0.smithi071.stdout:9/893: readlink d1/d32/d3a/d90/dc2/l10b 0 2024-04-23T02:14:15.628 INFO:tasks.workunit.client.0.smithi071.stdout:0/979: truncate d0/d3/d95/fcd 430094 0 2024-04-23T02:14:15.630 INFO:tasks.workunit.client.0.smithi071.stdout:1/834: rmdir db/df/d5f/d6f/db2/d86/dbc 0 2024-04-23T02:14:15.632 INFO:tasks.workunit.client.0.smithi071.stdout:0/980: write d0/d3/de/d10/d14/d32/d8a/df0/f77 [4720891,30602] 0 2024-04-23T02:14:15.633 INFO:tasks.workunit.client.0.smithi071.stdout:1/835: readlink db/df/dc2/dec/l111 0 2024-04-23T02:14:15.633 INFO:tasks.workunit.client.0.smithi071.stdout:6/967: dwrite d6/d104/dad/ff3 [0,4194304] 0 2024-04-23T02:14:15.634 INFO:tasks.workunit.client.0.smithi071.stdout:6/968: fdatasync d6/f94 0 2024-04-23T02:14:15.636 INFO:tasks.workunit.client.0.smithi071.stdout:3/988: getdents d7/da5 0 2024-04-23T02:14:15.638 INFO:tasks.workunit.client.0.smithi071.stdout:0/981: mkdir d0/d3/d95/d114/d154/d15e 0 2024-04-23T02:14:15.638 INFO:tasks.workunit.client.0.smithi071.stdout:6/969: write d6/d104/dd8/df2/de7/f130 [934893,123173] 0 2024-04-23T02:14:15.639 INFO:tasks.workunit.client.0.smithi071.stdout:3/989: truncate d7/da5/db2/f137 905480 0 2024-04-23T02:14:15.641 INFO:tasks.workunit.client.0.smithi071.stdout:6/970: write d6/d22/ffa [429956,82060] 0 2024-04-23T02:14:15.642 INFO:tasks.workunit.client.0.smithi071.stdout:1/836: rename db/d11/d7f/da7/dd9/d100/fce to db/df/d5f/f11a 0 2024-04-23T02:14:15.644 INFO:tasks.workunit.client.0.smithi071.stdout:1/837: fsync db/df/d13/ddc/f112 0 2024-04-23T02:14:15.645 INFO:tasks.workunit.client.0.smithi071.stdout:6/971: chown d6/d104/db5/f134 2067 1 2024-04-23T02:14:15.646 INFO:tasks.workunit.client.0.smithi071.stdout:3/990: unlink d7/d22/da7/fa2 0 2024-04-23T02:14:15.650 INFO:tasks.workunit.client.0.smithi071.stdout:1/838: rmdir db/df/d13/ddc 39 2024-04-23T02:14:15.651 INFO:tasks.workunit.client.0.smithi071.stdout:1/839: chown db/d11/f1d 27 1 2024-04-23T02:14:15.652 INFO:tasks.workunit.client.0.smithi071.stdout:6/972: truncate d6/d1f/d5d/f73 1175223 0 2024-04-23T02:14:15.653 INFO:tasks.workunit.client.0.smithi071.stdout:6/973: dread - d6/d104/dad/d3f/fff zero size 2024-04-23T02:14:15.654 INFO:tasks.workunit.client.0.smithi071.stdout:6/974: fsync d6/d104/dad/d8f/de2/fbd 0 2024-04-23T02:14:15.655 INFO:tasks.workunit.client.0.smithi071.stdout:1/840: mknod db/d34/d8a/de1/c11b 0 2024-04-23T02:14:15.657 INFO:tasks.workunit.client.0.smithi071.stdout:1/841: write db/df/d13/d2f/f29 [4040902,50766] 0 2024-04-23T02:14:15.659 INFO:tasks.workunit.client.0.smithi071.stdout:6/975: creat d6/d1f/d5d/f13c x:0 0 0 2024-04-23T02:14:15.667 INFO:tasks.workunit.client.0.smithi071.stdout:0/982: dread d0/d3/de/d48/d7e/d8b/f22 [4194304,4194304] 0 2024-04-23T02:14:15.669 INFO:tasks.workunit.client.0.smithi071.stdout:9/894: dread d1/d32/d41/db7/f25 [8388608,4194304] 0 2024-04-23T02:14:15.674 INFO:tasks.workunit.client.0.smithi071.stdout:9/895: creat d1/d32/d3a/d90/dc2/f13d x:0 0 0 2024-04-23T02:14:15.674 INFO:tasks.workunit.client.0.smithi071.stdout:3/991: dwrite d7/da5/d100/f10d [0,4194304] 0 2024-04-23T02:14:15.677 INFO:tasks.workunit.client.0.smithi071.stdout:1/842: dwrite f5 [4194304,4194304] 0 2024-04-23T02:14:15.677 INFO:tasks.workunit.client.0.smithi071.stdout:6/976: dread d6/d104/f100 [0,4194304] 0 2024-04-23T02:14:15.680 INFO:tasks.workunit.client.0.smithi071.stdout:1/843: readlink db/df/d13/d2f/l74 0 2024-04-23T02:14:15.682 INFO:tasks.workunit.client.0.smithi071.stdout:6/977: write d6/d17/fdf [1514300,108009] 0 2024-04-23T02:14:15.684 INFO:tasks.workunit.client.0.smithi071.stdout:3/992: rename d7/d1d/fb9 to d7/da5/d128/d68/f169 0 2024-04-23T02:14:15.688 INFO:tasks.workunit.client.0.smithi071.stdout:6/978: getdents d6/d22/dbf 0 2024-04-23T02:14:15.688 INFO:tasks.workunit.client.0.smithi071.stdout:1/844: getdents db/d11/d7f/da7/dd9/dfd 0 2024-04-23T02:14:15.690 INFO:tasks.workunit.client.0.smithi071.stdout:1/845: fdatasync db/df/ff4 0 2024-04-23T02:14:15.691 INFO:tasks.workunit.client.0.smithi071.stdout:6/979: unlink d6/d1f/d5d/l72 0 2024-04-23T02:14:15.692 INFO:tasks.workunit.client.0.smithi071.stdout:3/993: dwrite d7/d8/f1a [0,4194304] 0 2024-04-23T02:14:15.696 INFO:tasks.workunit.client.0.smithi071.stdout:3/994: mknod d7/d22/da7/c16a 0 2024-04-23T02:14:15.697 INFO:tasks.workunit.client.0.smithi071.stdout:3/995: write f0 [2607854,14114] 0 2024-04-23T02:14:15.698 INFO:tasks.workunit.client.0.smithi071.stdout:3/996: fdatasync d7/d22/d23/d26/d52/d12f/f15a 0 2024-04-23T02:14:15.699 INFO:tasks.workunit.client.0.smithi071.stdout:1/846: getdents db/df/d13/d20/dc0/ddf/de8 0 2024-04-23T02:14:15.701 INFO:tasks.workunit.client.0.smithi071.stdout:1/847: fsync db/d34/f8c 0 2024-04-23T02:14:15.702 INFO:tasks.workunit.client.0.smithi071.stdout:6/980: read d6/d22/f90 [3198653,118167] 0 2024-04-23T02:14:15.703 INFO:tasks.workunit.client.0.smithi071.stdout:6/981: write d6/d1f/f83 [2663878,9848] 0 2024-04-23T02:14:15.705 INFO:tasks.workunit.client.0.smithi071.stdout:3/997: rename d7/da5/d128/d68/d3b/d4c/d166/d47/fc8 to d7/d22/d23/dba/df4/dca/f16b 0 2024-04-23T02:14:15.708 INFO:tasks.workunit.client.0.smithi071.stdout:3/998: mkdir d7/da5/d128/d68/d3b/d98/d11a/d16c 0 2024-04-23T02:14:15.709 INFO:tasks.workunit.client.0.smithi071.stdout:1/848: truncate db/df/d13/d2b/f78 1872537 0 2024-04-23T02:14:15.709 INFO:tasks.workunit.client.0.smithi071.stdout:9/896: write d1/d32/d41/db7/d5a/dc6/d81/db6/f101 [187066,120618] 0 2024-04-23T02:14:15.712 INFO:tasks.workunit.client.0.smithi071.stdout:3/999: mkdir d7/da5/db5/dec/d16d 0 2024-04-23T02:14:15.715 INFO:tasks.workunit.client.0.smithi071.stdout:1/849: mkdir db/df/d5f/d6f/db2/d11c 0 2024-04-23T02:14:15.716 INFO:tasks.workunit.client.0.smithi071.stdout:9/897: symlink d1/d32/d41/db7/d39/l13e 0 2024-04-23T02:14:15.719 INFO:tasks.workunit.client.0.smithi071.stdout:9/898: mkdir d1/d32/dc5/dd8/d13f 0 2024-04-23T02:14:15.719 INFO:tasks.workunit.client.0.smithi071.stdout:9/899: chown d1/d32/d3a/d90/dc2/fe0 957656757 1 2024-04-23T02:14:15.721 INFO:tasks.workunit.client.0.smithi071.stdout:9/900: mkdir d1/d32/d3a/d5f/d140 0 2024-04-23T02:14:15.722 INFO:tasks.workunit.client.0.smithi071.stdout:6/982: write d6/d104/dad/dd1/d10b/fc3 [276416,55344] 0 2024-04-23T02:14:15.723 INFO:tasks.workunit.client.0.smithi071.stdout:6/983: chown d6/f11 136566 1 2024-04-23T02:14:15.723 INFO:tasks.workunit.client.0.smithi071.stdout:9/901: write d1/d32/d41/db7/f22 [1886720,127995] 0 2024-04-23T02:14:15.725 INFO:tasks.workunit.client.0.smithi071.stdout:1/850: truncate db/d11/d8d/fd8 605331 0 2024-04-23T02:14:15.726 INFO:tasks.workunit.client.0.smithi071.stdout:6/984: symlink d6/d22/d92/l13d 0 2024-04-23T02:14:15.729 INFO:tasks.workunit.client.0.smithi071.stdout:1/851: truncate db/df/d13/d20/dc0/ffa 749215 0 2024-04-23T02:14:15.731 INFO:tasks.workunit.client.0.smithi071.stdout:9/902: dwrite d1/d32/d3a/d90/f10d [0,4194304] 0 2024-04-23T02:14:15.733 INFO:tasks.workunit.client.0.smithi071.stdout:6/985: getdents d6/d1f/d5d 0 2024-04-23T02:14:15.734 INFO:tasks.workunit.client.0.smithi071.stdout:1/852: creat db/d11/d7f/da7/dd9/d100/f11d x:0 0 0 2024-04-23T02:14:15.735 INFO:tasks.workunit.client.0.smithi071.stdout:1/853: chown db/d11/d8d 181193 1 2024-04-23T02:14:15.736 INFO:tasks.workunit.client.0.smithi071.stdout:9/903: creat d1/d8d/f141 x:0 0 0 2024-04-23T02:14:15.739 INFO:tasks.workunit.client.0.smithi071.stdout:6/986: symlink d6/d22/l13e 0 2024-04-23T02:14:15.742 INFO:tasks.workunit.client.0.smithi071.stdout:1/854: truncate db/df/d13/d2b/fbf 987204 0 2024-04-23T02:14:15.744 INFO:tasks.workunit.client.0.smithi071.stdout:1/855: chown db/d34/d8a/led 4 1 2024-04-23T02:14:15.745 INFO:tasks.workunit.client.0.smithi071.stdout:6/987: dread d6/d17/d77/d9e/fc5 [0,4194304] 0 2024-04-23T02:14:15.745 INFO:tasks.workunit.client.0.smithi071.stdout:9/904: dwrite d1/ff1 [0,4194304] 0 2024-04-23T02:14:15.748 INFO:tasks.workunit.client.0.smithi071.stdout:1/856: unlink db/df/d13/d2b/fab 0 2024-04-23T02:14:15.750 INFO:tasks.workunit.client.0.smithi071.stdout:9/905: mknod d1/d32/dc5/dd8/c142 0 2024-04-23T02:14:15.758 INFO:tasks.workunit.client.0.smithi071.stdout:6/988: write d6/d104/dd8/df2/fc8 [719516,27095] 0 2024-04-23T02:14:15.760 INFO:tasks.workunit.client.0.smithi071.stdout:1/857: dread db/df/f2a [0,4194304] 0 2024-04-23T02:14:15.762 INFO:tasks.workunit.client.0.smithi071.stdout:6/989: truncate d6/f11 1191215 0 2024-04-23T02:14:15.764 INFO:tasks.workunit.client.0.smithi071.stdout:9/906: dwrite d1/d32/dc5/dd1/fb5 [0,4194304] 0 2024-04-23T02:14:15.764 INFO:tasks.workunit.client.0.smithi071.stdout:1/858: rmdir db/df/d13/d2f/d48/d85 39 2024-04-23T02:14:15.765 INFO:tasks.workunit.client.0.smithi071.stdout:9/907: read - d1/d32/d3a/d90/dc2/f13d zero size 2024-04-23T02:14:15.766 INFO:tasks.workunit.client.0.smithi071.stdout:6/990: creat d6/d1f/d5d/dde/d121/f13f x:0 0 0 2024-04-23T02:14:15.773 INFO:tasks.workunit.client.0.smithi071.stdout:1/859: truncate db/df/d13/d20/dc0/fd1 1259282 0 2024-04-23T02:14:15.775 INFO:tasks.workunit.client.0.smithi071.stdout:9/908: write d1/f37 [4119075,13381] 0 2024-04-23T02:14:15.778 INFO:tasks.workunit.client.0.smithi071.stdout:9/909: rename d1/d32/lad to d1/d32/d3a/d77/l143 0 2024-04-23T02:14:15.780 INFO:tasks.workunit.client.0.smithi071.stdout:9/910: creat d1/d32/d41/db7/d5a/de3/f144 x:0 0 0 2024-04-23T02:14:15.782 INFO:tasks.workunit.client.0.smithi071.stdout:9/911: rmdir d1/d32/d7e 39 2024-04-23T02:14:15.784 INFO:tasks.workunit.client.0.smithi071.stdout:9/912: fdatasync d1/d32/d41/db7/d5a/dc6/d9f/dfe/f122 0 2024-04-23T02:14:15.785 INFO:tasks.workunit.client.0.smithi071.stdout:1/860: dwrite db/df/f1c [0,4194304] 0 2024-04-23T02:14:15.787 INFO:tasks.workunit.client.0.smithi071.stdout:6/991: dread d6/d1f/f68 [0,4194304] 0 2024-04-23T02:14:15.787 INFO:tasks.workunit.client.0.smithi071.stdout:9/913: mknod d1/d32/d41/db7/d71/c145 0 2024-04-23T02:14:15.788 INFO:tasks.workunit.client.0.smithi071.stdout:1/861: mknod db/df/d5f/df6/c11e 0 2024-04-23T02:14:15.793 INFO:tasks.workunit.client.0.smithi071.stdout:9/914: mkdir d1/d4e/d66/da4/dc4/d146 0 2024-04-23T02:14:15.794 INFO:tasks.workunit.client.0.smithi071.stdout:6/992: creat d6/d1f/d46/d135/f140 x:0 0 0 2024-04-23T02:14:15.796 INFO:tasks.workunit.client.0.smithi071.stdout:9/915: unlink d1/d8d/lb4 0 2024-04-23T02:14:15.796 INFO:tasks.workunit.client.0.smithi071.stdout:1/862: dwrite db/df/f39 [4194304,4194304] 0 2024-04-23T02:14:15.798 INFO:tasks.workunit.client.0.smithi071.stdout:1/863: write db/d34/d8a/fe4 [3377506,114792] 0 2024-04-23T02:14:15.799 INFO:tasks.workunit.client.0.smithi071.stdout:6/993: symlink d6/d1f/l141 0 2024-04-23T02:14:15.800 INFO:tasks.workunit.client.0.smithi071.stdout:9/916: truncate d1/f16 192381 0 2024-04-23T02:14:15.801 INFO:tasks.workunit.client.0.smithi071.stdout:1/864: symlink db/df/dc2/dec/l11f 0 2024-04-23T02:14:15.801 INFO:tasks.workunit.client.0.smithi071.stdout:9/917: chown d1/d32/d3a/d5f/dde/c10c 501465 1 2024-04-23T02:14:15.807 INFO:tasks.workunit.client.0.smithi071.stdout:6/994: dread d6/f12 [0,4194304] 0 2024-04-23T02:14:15.810 INFO:tasks.workunit.client.0.smithi071.stdout:9/918: dwrite d1/d32/d3a/d77/f88 [0,4194304] 0 2024-04-23T02:14:15.813 INFO:tasks.workunit.client.0.smithi071.stdout:9/919: stat d1/d32/d3a/le1 0 2024-04-23T02:14:15.814 INFO:tasks.workunit.client.0.smithi071.stdout:9/920: write d1/d32/d3a/d94/ff5 [1215047,28689] 0 2024-04-23T02:14:15.817 INFO:tasks.workunit.client.0.smithi071.stdout:9/921: dread - d1/f86 zero size 2024-04-23T02:14:15.819 INFO:tasks.workunit.client.0.smithi071.stdout:9/922: chown d1/d32/d3a/f3b 32442 1 2024-04-23T02:14:15.822 INFO:tasks.workunit.client.0.smithi071.stdout:9/923: mknod d1/d32/d3a/d5f/d140/c147 0 2024-04-23T02:14:15.822 INFO:tasks.workunit.client.0.smithi071.stdout:1/865: write db/df/d13/d20/d75/f95 [748941,67974] 0 2024-04-23T02:14:15.823 INFO:tasks.workunit.client.0.smithi071.stdout:1/866: stat db/d11/f7a 0 2024-04-23T02:14:15.826 INFO:tasks.workunit.client.0.smithi071.stdout:1/867: mknod db/df/d13/d2f/d48/c120 0 2024-04-23T02:14:15.830 INFO:tasks.workunit.client.0.smithi071.stdout:2/818: sync 2024-04-23T02:14:15.831 INFO:tasks.workunit.client.0.smithi071.stdout:9/924: write d1/d32/d41/db7/d5a/dc6/d9f/f98 [654055,73182] 0 2024-04-23T02:14:15.833 INFO:tasks.workunit.client.0.smithi071.stdout:2/819: stat d0/d2/d34/d83/fc5 0 2024-04-23T02:14:15.834 INFO:tasks.workunit.client.0.smithi071.stdout:6/995: dread d6/d22/f28 [0,4194304] 0 2024-04-23T02:14:15.835 INFO:tasks.workunit.client.0.smithi071.stdout:9/925: chown d1/d32/d41/db7/d39/fe9 228556580 1 2024-04-23T02:14:15.835 INFO:tasks.workunit.client.0.smithi071.stdout:1/868: dwrite db/d11/d7f/da7/dd9/f117 [0,4194304] 0 2024-04-23T02:14:15.839 INFO:tasks.workunit.client.0.smithi071.stdout:1/869: mknod db/df/d13/d2b/c121 0 2024-04-23T02:14:15.842 INFO:tasks.workunit.client.0.smithi071.stdout:6/996: rename d6/d1f/f83 to d6/d1f/d5d/d95/f142 0 2024-04-23T02:14:15.843 INFO:tasks.workunit.client.0.smithi071.stdout:1/870: truncate db/df/d13/d2f/d48/d9f/dfb/f108 349088 0 2024-04-23T02:14:15.844 INFO:tasks.workunit.client.0.smithi071.stdout:6/997: write d6/d17/f2c [335227,119842] 0 2024-04-23T02:14:15.844 INFO:tasks.workunit.client.0.smithi071.stdout:2/820: truncate d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/fbd 822091 0 2024-04-23T02:14:15.846 INFO:tasks.workunit.client.0.smithi071.stdout:1/871: unlink db/df/d13/d2f/d7b/f9d 0 2024-04-23T02:14:15.848 INFO:tasks.workunit.client.0.smithi071.stdout:2/821: symlink d0/d2/d26/d3a/dc3/d36/d5c/d85/l116 0 2024-04-23T02:14:15.849 INFO:tasks.workunit.client.0.smithi071.stdout:1/872: mkdir db/df/d13/d20/dc0/d122 0 2024-04-23T02:14:15.850 INFO:tasks.workunit.client.0.smithi071.stdout:6/998: creat d6/d17/d77/f143 x:0 0 0 2024-04-23T02:14:15.851 INFO:tasks.workunit.client.0.smithi071.stdout:1/873: readlink db/df/dc2/dec/l111 0 2024-04-23T02:14:15.852 INFO:tasks.workunit.client.0.smithi071.stdout:2/822: mknod d0/d2/d26/d3a/dc3/d36/d5c/dfe/d10f/c117 0 2024-04-23T02:14:15.854 INFO:tasks.workunit.client.0.smithi071.stdout:6/999: dread d6/d104/dad/f111 [0,4194304] 0 2024-04-23T02:14:15.855 INFO:tasks.workunit.client.0.smithi071.stdout:9/926: dread d1/d32/dc5/dd1/f9d [0,4194304] 0 2024-04-23T02:14:15.855 INFO:tasks.workunit.client.0.smithi071.stdout:2/823: chown d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/f3b 198818722 1 2024-04-23T02:14:15.856 INFO:tasks.workunit.client.0.smithi071.stdout:9/927: chown d1/d32/d41/db7/d5a/f6b 11 1 2024-04-23T02:14:15.859 INFO:tasks.workunit.client.0.smithi071.stdout:2/824: symlink d0/d2/d49/def/l118 0 2024-04-23T02:14:15.861 INFO:tasks.workunit.client.0.smithi071.stdout:0/983: sync 2024-04-23T02:14:15.861 INFO:tasks.workunit.client.0.smithi071.stdout:9/928: mkdir d1/d32/d7e/d13b/d148 0 2024-04-23T02:14:15.862 INFO:tasks.workunit.client.0.smithi071.stdout:0/984: stat d0/d3/de/d10/d14/d32/d8a 0 2024-04-23T02:14:15.864 INFO:tasks.workunit.client.0.smithi071.stdout:2/825: symlink d0/d2/d26/l119 0 2024-04-23T02:14:15.868 INFO:tasks.workunit.client.0.smithi071.stdout:0/985: creat d0/d3/de/d10/d14/d32/dac/f15f x:0 0 0 2024-04-23T02:14:15.869 INFO:tasks.workunit.client.0.smithi071.stdout:9/929: dwrite d1/d32/d41/fd2 [4194304,4194304] 0 2024-04-23T02:14:15.876 INFO:tasks.workunit.client.0.smithi071.stdout:0/986: link d0/d3/d95/l126 d0/d3/de/d10/dfc/l160 0 2024-04-23T02:14:15.879 INFO:tasks.workunit.client.0.smithi071.stdout:0/987: chown d0/d3/de/d10/d14/d32/c2f 479 1 2024-04-23T02:14:15.881 INFO:tasks.workunit.client.0.smithi071.stdout:9/930: dwrite d1/d32/d41/db7/d39/d42/f123 [0,4194304] 0 2024-04-23T02:14:15.886 INFO:tasks.workunit.client.0.smithi071.stdout:1/874: dwrite db/df/d13/d2b/fbf [0,4194304] 0 2024-04-23T02:14:15.888 INFO:tasks.workunit.client.0.smithi071.stdout:0/988: write d0/d3/de/d48/d7e/d8b/fa7 [518302,45580] 0 2024-04-23T02:14:15.891 INFO:tasks.workunit.client.0.smithi071.stdout:2/826: dwrite d0/d2/d45/f5a [4194304,4194304] 0 2024-04-23T02:14:15.893 INFO:tasks.workunit.client.0.smithi071.stdout:1/875: mkdir db/df/d13/d2f/d48/d9f/d123 0 2024-04-23T02:14:15.895 INFO:tasks.workunit.client.0.smithi071.stdout:2/827: symlink d0/d2/da0/l11a 0 2024-04-23T02:14:15.896 INFO:tasks.workunit.client.0.smithi071.stdout:1/876: mknod db/d11/c124 0 2024-04-23T02:14:15.899 INFO:tasks.workunit.client.0.smithi071.stdout:2/828: read d0/d2/f9 [1050706,8051] 0 2024-04-23T02:14:15.900 INFO:tasks.workunit.client.0.smithi071.stdout:1/877: chown db/df/d5f/d6f/db2/d86/lc8 4 1 2024-04-23T02:14:15.905 INFO:tasks.workunit.client.0.smithi071.stdout:1/878: dread db/d11/d7f/da7/dd9/f117 [0,4194304] 0 2024-04-23T02:14:15.908 INFO:tasks.workunit.client.0.smithi071.stdout:1/879: creat db/df/d13/d20/dc0/ddf/de8/f125 x:0 0 0 2024-04-23T02:14:15.911 INFO:tasks.workunit.client.0.smithi071.stdout:1/880: fdatasync db/df/d13/f71 0 2024-04-23T02:14:15.912 INFO:tasks.workunit.client.0.smithi071.stdout:9/931: dread d1/fb [0,4194304] 0 2024-04-23T02:14:15.917 INFO:tasks.workunit.client.0.smithi071.stdout:1/881: read db/f54 [4122659,79547] 0 2024-04-23T02:14:15.919 INFO:tasks.workunit.client.0.smithi071.stdout:1/882: unlink db/df/d5f/df6/f104 0 2024-04-23T02:14:15.920 INFO:tasks.workunit.client.0.smithi071.stdout:1/883: stat db/df/d13/d20/f53 0 2024-04-23T02:14:15.921 INFO:tasks.workunit.client.0.smithi071.stdout:1/884: readlink db/df/d13/d2b/l41 0 2024-04-23T02:14:15.922 INFO:tasks.workunit.client.0.smithi071.stdout:9/932: fsync d1/d32/f4a 0 2024-04-23T02:14:15.923 INFO:tasks.workunit.client.0.smithi071.stdout:1/885: dread - db/d34/d8a/f119 zero size 2024-04-23T02:14:15.925 INFO:tasks.workunit.client.0.smithi071.stdout:1/886: mkdir db/d34/d126 0 2024-04-23T02:14:15.926 INFO:tasks.workunit.client.0.smithi071.stdout:9/933: mkdir d1/d32/d41/db7/d5a/dc6/d9f/d149 0 2024-04-23T02:14:15.932 INFO:tasks.workunit.client.0.smithi071.stdout:2/829: dwrite d0/d2/d26/d14/fa7 [0,4194304] 0 2024-04-23T02:14:15.932 INFO:tasks.workunit.client.0.smithi071.stdout:1/887: write db/df/d5f/df0/f109 [108713,67724] 0 2024-04-23T02:14:15.936 INFO:tasks.workunit.client.0.smithi071.stdout:2/830: symlink d0/d2/da0/db7/l11b 0 2024-04-23T02:14:15.938 INFO:tasks.workunit.client.0.smithi071.stdout:1/888: creat db/df/d13/d20/dc0/ddf/de8/f127 x:0 0 0 2024-04-23T02:14:15.938 INFO:tasks.workunit.client.0.smithi071.stdout:9/934: getdents d1/d32/d7e/d13b/d148 0 2024-04-23T02:14:15.939 INFO:tasks.workunit.client.0.smithi071.stdout:2/831: mknod d0/d2/d26/d3a/dc3/d36/d5c/c11c 0 2024-04-23T02:14:15.940 INFO:tasks.workunit.client.0.smithi071.stdout:1/889: write db/d11/d7f/da7/dd9/d100/f102 [909442,108999] 0 2024-04-23T02:14:15.941 INFO:tasks.workunit.client.0.smithi071.stdout:9/935: creat d1/d32/d3a/d5f/f14a x:0 0 0 2024-04-23T02:14:15.942 INFO:tasks.workunit.client.0.smithi071.stdout:1/890: chown db/d11/d7f/da7/dd9 4 1 2024-04-23T02:14:15.944 INFO:tasks.workunit.client.0.smithi071.stdout:9/936: symlink d1/d32/d7e/d13b/l14b 0 2024-04-23T02:14:15.949 INFO:tasks.workunit.client.0.smithi071.stdout:2/832: unlink d0/d2/d26/d3a/dc3/d4f/fda 0 2024-04-23T02:14:15.950 INFO:tasks.workunit.client.0.smithi071.stdout:9/937: fdatasync d1/d32/d41/db7/d39/d42/f108 0 2024-04-23T02:14:15.952 INFO:tasks.workunit.client.0.smithi071.stdout:9/938: creat d1/dfd/f14c x:0 0 0 2024-04-23T02:14:15.956 INFO:tasks.workunit.client.0.smithi071.stdout:1/891: link db/df/d13/d2b/cde db/df/d13/d2f/c128 0 2024-04-23T02:14:15.957 INFO:tasks.workunit.client.0.smithi071.stdout:9/939: mkdir d1/d32/d3a/d77/d14d 0 2024-04-23T02:14:15.960 INFO:tasks.workunit.client.0.smithi071.stdout:1/892: rename db/f54 to db/df/f129 0 2024-04-23T02:14:15.960 INFO:tasks.workunit.client.0.smithi071.stdout:1/893: stat db/d34/f8c 0 2024-04-23T02:14:15.961 INFO:tasks.workunit.client.0.smithi071.stdout:9/940: creat d1/d32/d41/db7/d39/df6/f14e x:0 0 0 2024-04-23T02:14:15.962 INFO:tasks.workunit.client.0.smithi071.stdout:9/941: rmdir d1/d32/dc5/dd1 39 2024-04-23T02:14:15.964 INFO:tasks.workunit.client.0.smithi071.stdout:1/894: rename db/df/d13/d20/d75/fc4 to db/df/d13/d20/dc0/ddf/f12a 0 2024-04-23T02:14:15.966 INFO:tasks.workunit.client.0.smithi071.stdout:2/833: dread d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/f53 [0,4194304] 0 2024-04-23T02:14:15.966 INFO:tasks.workunit.client.0.smithi071.stdout:1/895: unlink db/d92/lcd 0 2024-04-23T02:14:15.969 INFO:tasks.workunit.client.0.smithi071.stdout:2/834: creat d0/d2/d26/d3a/dc3/d36/d5c/f11d x:0 0 0 2024-04-23T02:14:15.973 INFO:tasks.workunit.client.0.smithi071.stdout:1/896: rename db/d11/d8d to db/df/d118/d12b 0 2024-04-23T02:14:15.979 INFO:tasks.workunit.client.0.smithi071.stdout:2/835: write d0/d2/f9 [2774811,95143] 0 2024-04-23T02:14:15.980 INFO:tasks.workunit.client.0.smithi071.stdout:1/897: getdents db/df/d5f/d57 0 2024-04-23T02:14:15.981 INFO:tasks.workunit.client.0.smithi071.stdout:1/898: fdatasync db/df/d13/d20/de5/ff2 0 2024-04-23T02:14:15.981 INFO:tasks.workunit.client.0.smithi071.stdout:2/836: write d0/d2/d26/d3a/dc3/d36/d6a/f6e [1571855,122152] 0 2024-04-23T02:14:15.984 INFO:tasks.workunit.client.0.smithi071.stdout:1/899: symlink db/d11/d7f/l12c 0 2024-04-23T02:14:15.986 INFO:tasks.workunit.client.0.smithi071.stdout:1/900: symlink db/df/dc2/l12d 0 2024-04-23T02:14:15.990 INFO:tasks.workunit.client.0.smithi071.stdout:2/837: dwrite d0/d2/d26/d3a/f77 [0,4194304] 0 2024-04-23T02:14:15.990 INFO:tasks.workunit.client.0.smithi071.stdout:1/901: mkdir db/df/d13/ddc/d12e 0 2024-04-23T02:14:15.992 INFO:tasks.workunit.client.0.smithi071.stdout:2/838: read d0/d70/f94 [3170491,92856] 0 2024-04-23T02:14:15.993 INFO:tasks.workunit.client.0.smithi071.stdout:1/902: mkdir db/df/d118/d12b/d12f 0 2024-04-23T02:14:15.996 INFO:tasks.workunit.client.0.smithi071.stdout:1/903: creat db/df/d118/d12b/f130 x:0 0 0 2024-04-23T02:14:15.996 INFO:tasks.workunit.client.0.smithi071.stdout:2/839: creat d0/d2/d26/d3a/dc3/d36/d5c/f11e x:0 0 0 2024-04-23T02:14:15.998 INFO:tasks.workunit.client.0.smithi071.stdout:1/904: truncate fa 5357510 0 2024-04-23T02:14:16.006 INFO:tasks.workunit.client.0.smithi071.stdout:2/840: write d0/d2/f29 [1624836,66327] 0 2024-04-23T02:14:16.006 INFO:tasks.workunit.client.0.smithi071.stdout:1/905: dwrite db/df/d13/d20/dc0/ffa [0,4194304] 0 2024-04-23T02:14:16.008 INFO:tasks.workunit.client.0.smithi071.stdout:2/841: write d0/d2/d34/f84 [1252526,85647] 0 2024-04-23T02:14:16.014 INFO:tasks.workunit.client.0.smithi071.stdout:2/842: truncate d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/fac 3713571 0 2024-04-23T02:14:16.014 INFO:tasks.workunit.client.0.smithi071.stdout:1/906: write db/df/d5f/fbe [917894,81708] 0 2024-04-23T02:14:16.016 INFO:tasks.workunit.client.0.smithi071.stdout:2/843: chown d0/d2/d34/d55/lb2 6111319 1 2024-04-23T02:14:16.018 INFO:tasks.workunit.client.0.smithi071.stdout:2/844: readlink d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/d80/lf0 0 2024-04-23T02:14:16.021 INFO:tasks.workunit.client.0.smithi071.stdout:2/845: chown d0/d2/d26/d3a/dc3/d36/d5c/d85/lfb 15501567 1 2024-04-23T02:14:16.028 INFO:tasks.workunit.client.0.smithi071.stdout:1/907: dread db/df/d3a/f83 [0,4194304] 0 2024-04-23T02:14:16.031 INFO:tasks.workunit.client.0.smithi071.stdout:2/846: write d0/d2/d26/d3a/dc3/d36/fbe [909204,53733] 0 2024-04-23T02:14:16.031 INFO:tasks.workunit.client.0.smithi071.stdout:1/908: chown db/df/d13/d20/dc0/ddf 37 1 2024-04-23T02:14:16.034 INFO:tasks.workunit.client.0.smithi071.stdout:1/909: creat db/df/d5f/d6f/db2/d86/f131 x:0 0 0 2024-04-23T02:14:16.040 INFO:tasks.workunit.client.0.smithi071.stdout:1/910: dwrite db/df/fea [0,4194304] 0 2024-04-23T02:14:16.046 INFO:tasks.workunit.client.0.smithi071.stdout:1/911: truncate db/d11/f7a 299579 0 2024-04-23T02:14:16.051 INFO:tasks.workunit.client.0.smithi071.stdout:1/912: dwrite db/df/f58 [0,4194304] 0 2024-04-23T02:14:16.065 INFO:tasks.workunit.client.0.smithi071.stdout:1/913: dread db/df/d13/fef [0,4194304] 0 2024-04-23T02:14:16.068 INFO:tasks.workunit.client.0.smithi071.stdout:1/914: rmdir db/df/d118/d12b/d12f 0 2024-04-23T02:14:16.068 INFO:tasks.workunit.client.0.smithi071.stdout:1/915: read - db/df/d5f/d6f/fb7 zero size 2024-04-23T02:14:16.070 INFO:tasks.workunit.client.0.smithi071.stdout:1/916: symlink db/df/dc2/dec/l132 0 2024-04-23T02:14:16.072 INFO:tasks.workunit.client.0.smithi071.stdout:1/917: mknod db/df/d13/d2f/d48/d9f/dfb/c133 0 2024-04-23T02:14:16.073 INFO:tasks.workunit.client.0.smithi071.stdout:1/918: symlink db/d11/d7f/da7/dd9/d100/l134 0 2024-04-23T02:14:16.074 INFO:tasks.workunit.client.0.smithi071.stdout:1/919: truncate db/d34/d8a/f119 592413 0 2024-04-23T02:14:16.075 INFO:tasks.workunit.client.0.smithi071.stdout:1/920: write db/df/d5f/d6f/db2/d86/fd6 [2896404,25893] 0 2024-04-23T02:14:16.076 INFO:tasks.workunit.client.0.smithi071.stdout:1/921: truncate db/df/d13/d2f/f10d 503370 0 2024-04-23T02:14:16.078 INFO:tasks.workunit.client.0.smithi071.stdout:1/922: write db/df/d13/d20/dc0/ddf/de8/f127 [108952,52024] 0 2024-04-23T02:14:16.081 INFO:tasks.workunit.client.0.smithi071.stdout:1/923: symlink db/df/d5f/d6f/db2/d11c/l135 0 2024-04-23T02:14:16.083 INFO:tasks.workunit.client.0.smithi071.stdout:1/924: creat db/df/d118/f136 x:0 0 0 2024-04-23T02:14:16.085 INFO:tasks.workunit.client.0.smithi071.stdout:1/925: getdents db/d11/d7f/da7/dd9/dfd 0 2024-04-23T02:14:16.086 INFO:tasks.workunit.client.0.smithi071.stdout:1/926: chown db/d11/d7f/da7/dd9/d100/l134 1157880 1 2024-04-23T02:14:16.088 INFO:tasks.workunit.client.0.smithi071.stdout:1/927: link db/df/d13/fef db/df/d13/f137 0 2024-04-23T02:14:16.094 INFO:tasks.workunit.client.0.smithi071.stdout:1/928: dread db/d34/d8a/f8b [0,4194304] 0 2024-04-23T02:14:16.099 INFO:tasks.workunit.client.0.smithi071.stdout:1/929: truncate db/df/d5f/d6f/db2/fbb 328855 0 2024-04-23T02:14:16.099 INFO:tasks.workunit.client.0.smithi071.stdout:1/930: symlink db/df/d5f/df0/l138 0 2024-04-23T02:14:16.100 INFO:tasks.workunit.client.0.smithi071.stdout:1/931: fdatasync db/df/d3a/f83 0 2024-04-23T02:14:16.103 INFO:tasks.workunit.client.0.smithi071.stdout:1/932: symlink db/df/d13/d2f/d48/d9f/d123/l139 0 2024-04-23T02:14:16.106 INFO:tasks.workunit.client.0.smithi071.stdout:1/933: mknod db/d34/d8a/d101/c13a 0 2024-04-23T02:14:16.108 INFO:tasks.workunit.client.0.smithi071.stdout:1/934: mkdir db/df/d5f/d6f/db2/d13b 0 2024-04-23T02:14:16.109 INFO:tasks.workunit.client.0.smithi071.stdout:1/935: chown db/df/d13/d2f/d48/d85/fb0 185100 1 2024-04-23T02:14:16.120 INFO:tasks.workunit.client.0.smithi071.stdout:1/936: dwrite db/df/d13/f71 [0,4194304] 0 2024-04-23T02:14:16.124 INFO:tasks.workunit.client.0.smithi071.stdout:1/937: unlink db/df/f58 0 2024-04-23T02:14:16.126 INFO:tasks.workunit.client.0.smithi071.stdout:1/938: mkdir db/df/d13/ddc/d12e/d13c 0 2024-04-23T02:14:16.129 INFO:tasks.workunit.client.0.smithi071.stdout:1/939: getdents db/df/d118/d12b/d97 0 2024-04-23T02:14:16.394 INFO:tasks.workunit.client.0.smithi071.stdout:0/989: sync 2024-04-23T02:14:16.394 INFO:tasks.workunit.client.0.smithi071.stdout:9/942: sync 2024-04-23T02:14:16.396 INFO:tasks.workunit.client.0.smithi071.stdout:9/943: fsync d1/d32/d41/db7/d39/d42/f108 0 2024-04-23T02:14:16.397 INFO:tasks.workunit.client.0.smithi071.stdout:0/990: rename d0/d3/de/d48/d7e/d8b/fa7 to d0/d3/de/d10/db9/f161 0 2024-04-23T02:14:16.399 INFO:tasks.workunit.client.0.smithi071.stdout:9/944: fsync d1/d32/dc5/dd1/f113 0 2024-04-23T02:14:16.405 INFO:tasks.workunit.client.0.smithi071.stdout:0/991: symlink d0/d63/d12d/l162 0 2024-04-23T02:14:16.405 INFO:tasks.workunit.client.0.smithi071.stdout:9/945: dwrite d1/d32/dc5/dd1/fb5 [0,4194304] 0 2024-04-23T02:14:16.406 INFO:tasks.workunit.client.0.smithi071.stdout:0/992: stat d0/ddf/d148/f5c 0 2024-04-23T02:14:16.407 INFO:tasks.workunit.client.0.smithi071.stdout:9/946: dread - d1/d32/d41/db7/d5a/dc6/d9f/ded/f115 zero size 2024-04-23T02:14:16.409 INFO:tasks.workunit.client.0.smithi071.stdout:0/993: mkdir d0/d3/de/d137/d163 0 2024-04-23T02:14:16.411 INFO:tasks.workunit.client.0.smithi071.stdout:9/947: mknod d1/d4e/d54/c14f 0 2024-04-23T02:14:16.413 INFO:tasks.workunit.client.0.smithi071.stdout:0/994: symlink d0/d3/de/d48/d7e/d8b/de3/l164 0 2024-04-23T02:14:16.414 INFO:tasks.workunit.client.0.smithi071.stdout:9/948: symlink d1/d32/dc5/dd1/l150 0 2024-04-23T02:14:16.418 INFO:tasks.workunit.client.0.smithi071.stdout:9/949: mkdir d1/d32/d41/db7/d5a/dc6/d81/db6/de4/d151 0 2024-04-23T02:14:16.420 INFO:tasks.workunit.client.0.smithi071.stdout:9/950: symlink d1/d32/d41/db7/d5a/dc6/d81/db6/de4/l152 0 2024-04-23T02:14:16.424 INFO:tasks.workunit.client.0.smithi071.stdout:9/951: unlink d1/l2a 0 2024-04-23T02:14:16.436 INFO:tasks.workunit.client.0.smithi071.stdout:0/995: dread d0/d3/de/d10/d14/d32/d37/d26/f2e [0,4194304] 0 2024-04-23T02:14:16.437 INFO:tasks.workunit.client.0.smithi071.stdout:2/847: sync 2024-04-23T02:14:16.437 INFO:tasks.workunit.client.0.smithi071.stdout:1/940: sync 2024-04-23T02:14:16.439 INFO:tasks.workunit.client.0.smithi071.stdout:1/941: stat db/df/d5f/d6f/db2/d11c 0 2024-04-23T02:14:16.439 INFO:tasks.workunit.client.0.smithi071.stdout:0/996: chown d0/d3/de/d10/d14/f64 9954 1 2024-04-23T02:14:16.440 INFO:tasks.workunit.client.0.smithi071.stdout:9/952: dread d1/d32/dc5/dd1/f113 [0,4194304] 0 2024-04-23T02:14:16.442 INFO:tasks.workunit.client.0.smithi071.stdout:1/942: chown db/df/d13/d2b/c121 46452 1 2024-04-23T02:14:16.442 INFO:tasks.workunit.client.0.smithi071.stdout:2/848: creat d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/f11f x:0 0 0 2024-04-23T02:14:16.442 INFO:tasks.workunit.client.0.smithi071.stdout:0/997: rmdir d0/d3/de/d10/db9/d106 39 2024-04-23T02:14:16.446 INFO:tasks.workunit.client.0.smithi071.stdout:2/849: write d0/d2/d26/d3a/d75/fd1 [6549187,25391] 0 2024-04-23T02:14:16.446 INFO:tasks.workunit.client.0.smithi071.stdout:1/943: truncate db/df/d13/d2b/f7e 4548275 0 2024-04-23T02:14:16.448 INFO:tasks.workunit.client.0.smithi071.stdout:2/850: fdatasync d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/f53 0 2024-04-23T02:14:16.450 INFO:tasks.workunit.client.0.smithi071.stdout:1/944: symlink db/df/d13/d2f/d48/d9f/d123/l13d 0 2024-04-23T02:14:16.450 INFO:tasks.workunit.client.0.smithi071.stdout:0/998: write d0/d3/de/d10/d14/d32/dac/f12b [1891581,21061] 0 2024-04-23T02:14:16.452 INFO:tasks.workunit.client.0.smithi071.stdout:9/953: dread d1/d32/d41/db7/d39/f11f [0,4194304] 0 2024-04-23T02:14:16.456 INFO:tasks.workunit.client.0.smithi071.stdout:2/851: creat d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/db8/f120 x:0 0 0 2024-04-23T02:14:16.459 INFO:tasks.workunit.client.0.smithi071.stdout:1/945: creat db/df/d13/d2f/d48/f13e x:0 0 0 2024-04-23T02:14:16.463 INFO:tasks.workunit.client.0.smithi071.stdout:0/999: dwrite d0/d3/de/d10/d14/d84/ff1 [0,4194304] 0 2024-04-23T02:14:16.468 INFO:tasks.workunit.client.0.smithi071.stdout:2/852: dwrite d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/f65 [0,4194304] 0 2024-04-23T02:14:16.470 INFO:tasks.workunit.client.0.smithi071.stdout:9/954: dread d1/d4e/f4f [0,4194304] 0 2024-04-23T02:14:16.475 INFO:tasks.workunit.client.0.smithi071.stdout:9/955: mknod d1/d4e/d66/da8/c153 0 2024-04-23T02:14:16.477 INFO:tasks.workunit.client.0.smithi071.stdout:2/853: dwrite d0/d2/d26/d3a/dc3/f95 [0,4194304] 0 2024-04-23T02:14:16.481 INFO:tasks.workunit.client.0.smithi071.stdout:9/956: link d1/d32/d41/db7/d71/c145 d1/d32/d41/db7/d5a/dc6/d81/db6/c154 0 2024-04-23T02:14:16.482 INFO:tasks.workunit.client.0.smithi071.stdout:2/854: mknod d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/dff/c121 0 2024-04-23T02:14:16.483 INFO:tasks.workunit.client.0.smithi071.stdout:9/957: creat d1/d32/d3a/d77/f155 x:0 0 0 2024-04-23T02:14:16.483 INFO:tasks.workunit.client.0.smithi071.stdout:2/855: read - d0/d2/d34/d83/ff8 zero size 2024-04-23T02:14:16.484 INFO:tasks.workunit.client.0.smithi071.stdout:9/958: stat d1/d4e/d54/c6c 0 2024-04-23T02:14:16.486 INFO:tasks.workunit.client.0.smithi071.stdout:2/856: mknod d0/d2/d26/d3a/dc3/d36/d5c/dfe/c122 0 2024-04-23T02:14:16.487 INFO:tasks.workunit.client.0.smithi071.stdout:9/959: mknod d1/d32/d7e/d114/c156 0 2024-04-23T02:14:16.489 INFO:tasks.workunit.client.0.smithi071.stdout:9/960: mkdir d1/d4e/d157 0 2024-04-23T02:14:16.498 INFO:tasks.workunit.client.0.smithi071.stdout:9/961: write d1/d8d/ff3 [453528,27329] 0 2024-04-23T02:14:16.506 INFO:tasks.workunit.client.0.smithi071.stdout:2/857: dread d0/d70/f86 [0,4194304] 0 2024-04-23T02:14:16.506 INFO:tasks.workunit.client.0.smithi071.stdout:9/962: dwrite d1/d32/d41/db7/d5a/f6b [0,4194304] 0 2024-04-23T02:14:16.507 INFO:tasks.workunit.client.0.smithi071.stdout:9/963: readlink d1/d4e/le2 0 2024-04-23T02:14:16.508 INFO:tasks.workunit.client.0.smithi071.stdout:9/964: chown d1/d32/d41/faf 51 1 2024-04-23T02:14:16.512 INFO:tasks.workunit.client.0.smithi071.stdout:9/965: truncate d1/d32/d3a/d90/dc2/f13d 942773 0 2024-04-23T02:14:16.515 INFO:tasks.workunit.client.0.smithi071.stdout:2/858: dwrite d0/d2/d26/f4c [0,4194304] 0 2024-04-23T02:14:16.517 INFO:tasks.workunit.client.0.smithi071.stdout:2/859: chown d0/d2/d26/d14/f19 0 1 2024-04-23T02:14:16.521 INFO:tasks.workunit.client.0.smithi071.stdout:9/966: dwrite d1/f3d [0,4194304] 0 2024-04-23T02:14:16.526 INFO:tasks.workunit.client.0.smithi071.stdout:9/967: getdents d1/d4e/d66/da8 0 2024-04-23T02:14:16.530 INFO:tasks.workunit.client.0.smithi071.stdout:2/860: dwrite d0/d2/fb [4194304,4194304] 0 2024-04-23T02:14:16.532 INFO:tasks.workunit.client.0.smithi071.stdout:9/968: link d1/d32/d41/db7/cc7 d1/d4e/d66/c158 0 2024-04-23T02:14:16.536 INFO:tasks.workunit.client.0.smithi071.stdout:2/861: creat d0/d2/d49/f123 x:0 0 0 2024-04-23T02:14:16.539 INFO:tasks.workunit.client.0.smithi071.stdout:2/862: mknod d0/d2/d26/d3a/dc3/d36/d6a/c124 0 2024-04-23T02:14:16.546 INFO:tasks.workunit.client.0.smithi071.stdout:9/969: write d1/d32/d3a/f8a [1806671,47544] 0 2024-04-23T02:14:16.547 INFO:tasks.workunit.client.0.smithi071.stdout:9/970: dread - d1/d32/d41/db7/d5a/dc6/d9f/ded/f111 zero size 2024-04-23T02:14:16.572 INFO:tasks.workunit.client.0.smithi071.stdout:2/863: dread d0/d2/d26/d14/f1c [0,4194304] 0 2024-04-23T02:14:16.573 INFO:tasks.workunit.client.0.smithi071.stdout:2/864: fdatasync d0/d2/d26/d3a/dc3/d36/d5c/f11d 0 2024-04-23T02:14:16.576 INFO:tasks.workunit.client.0.smithi071.stdout:2/865: rename d0/d2/d26/f107 to d0/d2/d26/d3a/f125 0 2024-04-23T02:14:16.577 INFO:tasks.workunit.client.0.smithi071.stdout:2/866: dread - d0/d2/d26/d3a/dc3/d36/fe3 zero size 2024-04-23T02:14:16.578 INFO:tasks.workunit.client.0.smithi071.stdout:2/867: chown d0/c35 5591206 1 2024-04-23T02:14:16.579 INFO:tasks.workunit.client.0.smithi071.stdout:2/868: creat d0/d70/f126 x:0 0 0 2024-04-23T02:14:16.584 INFO:tasks.workunit.client.0.smithi071.stdout:2/869: dwrite d0/d82/f9d [0,4194304] 0 2024-04-23T02:14:16.588 INFO:tasks.workunit.client.0.smithi071.stdout:2/870: rename d0/f2e to d0/d2/d26/d3a/dc3/d36/d6a/f127 0 2024-04-23T02:14:16.615 INFO:tasks.workunit.client.0.smithi071.stdout:2/871: dread d0/d2/d26/d14/f23 [0,4194304] 0 2024-04-23T02:14:16.617 INFO:tasks.workunit.client.0.smithi071.stdout:2/872: symlink d0/d2/d34/d83/l128 0 2024-04-23T02:14:16.618 INFO:tasks.workunit.client.0.smithi071.stdout:2/873: write d0/d2/f29 [1341130,129474] 0 2024-04-23T02:14:16.620 INFO:tasks.workunit.client.0.smithi071.stdout:2/874: creat d0/d2/da0/f129 x:0 0 0 2024-04-23T02:14:16.633 INFO:tasks.workunit.client.0.smithi071.stdout:2/875: dread d0/d2/d34/f50 [0,4194304] 0 2024-04-23T02:14:16.633 INFO:tasks.workunit.client.0.smithi071.stdout:2/876: readlink d0/d2/l13 0 2024-04-23T02:14:16.672 INFO:tasks.workunit.client.0.smithi071.stdout:9/971: sync 2024-04-23T02:14:16.673 INFO:tasks.workunit.client.0.smithi071.stdout:1/946: sync 2024-04-23T02:14:16.674 INFO:tasks.workunit.client.0.smithi071.stdout:9/972: write d1/d32/d41/db7/d5a/de3/f144 [37154,30285] 0 2024-04-23T02:14:16.675 INFO:tasks.workunit.client.0.smithi071.stdout:9/973: chown d1/d32/d7e/d13b/l14b 116441746 1 2024-04-23T02:14:16.681 INFO:tasks.workunit.client.0.smithi071.stdout:9/974: dwrite d1/d32/d41/db7/d5a/dc6/d9f/dfe/f122 [0,4194304] 0 2024-04-23T02:14:16.683 INFO:tasks.workunit.client.0.smithi071.stdout:2/877: write d0/d2/f1d [930423,37551] 0 2024-04-23T02:14:16.685 INFO:tasks.workunit.client.0.smithi071.stdout:2/878: dread - d0/d2/d26/d14/fae zero size 2024-04-23T02:14:16.688 INFO:tasks.workunit.client.0.smithi071.stdout:9/975: write d1/d32/d41/db7/d39/f11f [1240291,65001] 0 2024-04-23T02:14:16.692 INFO:tasks.workunit.client.0.smithi071.stdout:1/947: creat db/d34/d8a/de1/f13f x:0 0 0 2024-04-23T02:14:16.695 INFO:tasks.workunit.client.0.smithi071.stdout:1/948: rmdir db/df/d5f/d6f/db2/d11c 39 2024-04-23T02:14:16.701 INFO:tasks.workunit.client.0.smithi071.stdout:1/949: symlink db/df/d13/d20/de5/l140 0 2024-04-23T02:14:16.702 INFO:tasks.workunit.client.0.smithi071.stdout:2/879: getdents d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/db8 0 2024-04-23T02:14:16.706 INFO:tasks.workunit.client.0.smithi071.stdout:2/880: rename d0/d2/d26/d3a/c93 to d0/d2/d26/d3a/dc3/d4f/c12a 0 2024-04-23T02:14:16.709 INFO:tasks.workunit.client.0.smithi071.stdout:2/881: fsync d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/f65 0 2024-04-23T02:14:16.710 INFO:tasks.workunit.client.0.smithi071.stdout:2/882: chown d0/d2/d45/le8 18442414 1 2024-04-23T02:14:16.711 INFO:tasks.workunit.client.0.smithi071.stdout:2/883: write d0/d2/d26/d3a/dc3/f95 [4315430,24471] 0 2024-04-23T02:14:16.712 INFO:tasks.workunit.client.0.smithi071.stdout:2/884: read d0/d2/d26/f4c [1044747,85010] 0 2024-04-23T02:14:16.716 INFO:tasks.workunit.client.0.smithi071.stdout:1/950: dread db/df/d13/d2b/f7e [0,4194304] 0 2024-04-23T02:14:16.717 INFO:tasks.workunit.client.0.smithi071.stdout:2/885: rename d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/de0/lee to d0/d2/d49/def/l12b 0 2024-04-23T02:14:16.718 INFO:tasks.workunit.client.0.smithi071.stdout:2/886: read d0/d2/f9 [1233386,4287] 0 2024-04-23T02:14:16.721 INFO:tasks.workunit.client.0.smithi071.stdout:1/951: mkdir db/df/d13/d2f/d48/d141 0 2024-04-23T02:14:16.721 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:16 smithi071 ceph-mon[39837]: pgmap v35: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 112 MiB/s rd, 222 MiB/s wr, 525 op/s 2024-04-23T02:14:16.722 INFO:tasks.workunit.client.0.smithi071.stdout:2/887: mkdir d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d12c 0 2024-04-23T02:14:16.724 INFO:tasks.workunit.client.0.smithi071.stdout:1/952: fdatasync db/d11/f7a 0 2024-04-23T02:14:16.724 INFO:tasks.workunit.client.0.smithi071.stdout:1/953: chown db/df/d3a/l67 5939 1 2024-04-23T02:14:16.726 INFO:tasks.workunit.client.0.smithi071.stdout:2/888: rename d0/d2/d49/f123 to d0/d2/d45/f12d 0 2024-04-23T02:14:16.730 INFO:tasks.workunit.client.0.smithi071.stdout:2/889: fsync d0/d2/d26/d14/fa7 0 2024-04-23T02:14:16.731 INFO:tasks.workunit.client.0.smithi071.stdout:2/890: stat d0/d2/d26/d3a/dc3 0 2024-04-23T02:14:16.752 INFO:tasks.workunit.client.0.smithi071.stdout:9/976: mkdir d1/d32/d3a/d159 0 2024-04-23T02:14:16.752 INFO:tasks.workunit.client.0.smithi071.stdout:9/977: chown d1/d4e/d54/l64 1003 1 2024-04-23T02:14:16.758 INFO:tasks.workunit.client.0.smithi071.stdout:9/978: link d1/d32/d41/db7/d5a/dc6/d81/db6/de4/f95 d1/d32/d41/db7/d5a/dc6/d9f/dfe/f15a 0 2024-04-23T02:14:16.761 INFO:tasks.workunit.client.0.smithi071.stdout:1/954: write db/d11/d7f/da7/dd9/d100/f81 [227081,89409] 0 2024-04-23T02:14:16.767 INFO:tasks.workunit.client.0.smithi071.stdout:2/891: mknod d0/d2/d26/d3a/dc3/d36/c12e 0 2024-04-23T02:14:16.768 INFO:tasks.workunit.client.0.smithi071.stdout:9/979: truncate d1/f28 1519296 0 2024-04-23T02:14:16.769 INFO:tasks.workunit.client.0.smithi071.stdout:9/980: dread - d1/d32/d41/db7/d39/df6/fb3 zero size 2024-04-23T02:14:16.769 INFO:tasks.workunit.client.0.smithi071.stdout:2/892: mknod d0/d2/d34/dbf/c12f 0 2024-04-23T02:14:16.771 INFO:tasks.workunit.client.0.smithi071.stdout:1/955: write db/d34/d8a/fe9 [727,98207] 0 2024-04-23T02:14:16.773 INFO:tasks.workunit.client.0.smithi071.stdout:9/981: rename d1/d32/d3a/d5f/lbe to d1/d32/d41/db7/d71/l15b 0 2024-04-23T02:14:16.775 INFO:tasks.workunit.client.0.smithi071.stdout:9/982: truncate d1/f2b 7725854 0 2024-04-23T02:14:16.780 INFO:tasks.workunit.client.0.smithi071.stdout:1/956: dwrite db/df/d5f/d6f/db2/d86/f131 [0,4194304] 0 2024-04-23T02:14:16.784 INFO:tasks.workunit.client.0.smithi071.stdout:1/957: dread - db/df/d3a/de7/ff5 zero size 2024-04-23T02:14:16.787 INFO:tasks.workunit.client.0.smithi071.stdout:9/983: dread d1/d32/d3a/d94/f9b [0,4194304] 0 2024-04-23T02:14:16.789 INFO:tasks.workunit.client.0.smithi071.stdout:9/984: read d1/d32/d41/db7/d5a/f135 [826251,5210] 0 2024-04-23T02:14:16.792 INFO:tasks.workunit.client.0.smithi071.stdout:9/985: mkdir d1/d32/d41/db7/d39/d15c 0 2024-04-23T02:14:16.793 INFO:tasks.workunit.client.0.smithi071.stdout:2/893: dread d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/f61 [0,4194304] 0 2024-04-23T02:14:16.795 INFO:tasks.workunit.client.0.smithi071.stdout:9/986: creat d1/d32/d3a/d90/d99/f15d x:0 0 0 2024-04-23T02:14:16.796 INFO:tasks.workunit.client.0.smithi071.stdout:9/987: chown d1/d32/d41/db7/d5a/dc6/d81/db6/f101 1134 1 2024-04-23T02:14:16.797 INFO:tasks.workunit.client.0.smithi071.stdout:9/988: mkdir d1/d32/d7e/d13b/d148/d15e 0 2024-04-23T02:14:16.798 INFO:tasks.workunit.client.0.smithi071.stdout:9/989: rmdir d1/d32/d3a/d5f 39 2024-04-23T02:14:16.800 INFO:tasks.workunit.client.0.smithi071.stdout:9/990: creat d1/d8d/f15f x:0 0 0 2024-04-23T02:14:16.806 INFO:tasks.workunit.client.0.smithi071.stdout:9/991: dwrite d1/d8d/f141 [0,4194304] 0 2024-04-23T02:14:16.807 INFO:tasks.workunit.client.0.smithi071.stdout:9/992: chown d1/d32/d41/db7/d39/l13e 17101 1 2024-04-23T02:14:16.808 INFO:tasks.workunit.client.0.smithi071.stdout:2/894: write d0/d82/f87 [3447430,74413] 0 2024-04-23T02:14:16.811 INFO:tasks.workunit.client.0.smithi071.stdout:9/993: creat d1/d32/d41/db7/d5a/de3/f160 x:0 0 0 2024-04-23T02:14:16.813 INFO:tasks.workunit.client.0.smithi071.stdout:1/958: dread db/df/d13/d2b/f35 [0,4194304] 0 2024-04-23T02:14:16.814 INFO:tasks.workunit.client.0.smithi071.stdout:9/994: stat d1/d32/d3a/d5f/f60 0 2024-04-23T02:14:16.816 INFO:tasks.workunit.client.0.smithi071.stdout:1/959: fdatasync db/d34/f7d 0 2024-04-23T02:14:16.816 INFO:tasks.workunit.client.0.smithi071.stdout:2/895: dwrite d0/d82/f9d [4194304,4194304] 0 2024-04-23T02:14:16.818 INFO:tasks.workunit.client.0.smithi071.stdout:9/995: creat d1/d8d/ddf/f161 x:0 0 0 2024-04-23T02:14:16.820 INFO:tasks.workunit.client.0.smithi071.stdout:2/896: mknod d0/d2/d26/d3a/dc3/d4f/c130 0 2024-04-23T02:14:16.821 INFO:tasks.workunit.client.0.smithi071.stdout:9/996: symlink d1/db8/l162 0 2024-04-23T02:14:16.822 INFO:tasks.workunit.client.0.smithi071.stdout:1/960: creat db/df/d13/d20/dc0/d122/f142 x:0 0 0 2024-04-23T02:14:16.823 INFO:tasks.workunit.client.0.smithi071.stdout:2/897: dread - d0/d2/d34/d83/fa4 zero size 2024-04-23T02:14:16.825 INFO:tasks.workunit.client.0.smithi071.stdout:2/898: write d0/d2/d26/d3a/dc3/d36/fe3 [596504,16442] 0 2024-04-23T02:14:16.826 INFO:tasks.workunit.client.0.smithi071.stdout:9/997: unlink d1/d32/d41/db7/d39/d42/f49 0 2024-04-23T02:14:16.831 INFO:tasks.workunit.client.0.smithi071.stdout:1/961: dwrite db/df/d5f/df0/f109 [0,4194304] 0 2024-04-23T02:14:16.831 INFO:tasks.workunit.client.0.smithi071.stdout:2/899: symlink d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/dff/l131 0 2024-04-23T02:14:16.832 INFO:tasks.workunit.client.0.smithi071.stdout:2/900: chown d0/d2/d34/dd4/df2 176774730 1 2024-04-23T02:14:16.840 INFO:tasks.workunit.client.0.smithi071.stdout:2/901: dwrite d0/d2/d34/d55/f72 [0,4194304] 0 2024-04-23T02:14:16.840 INFO:tasks.workunit.client.0.smithi071.stdout:1/962: dread db/df/d13/d20/dc0/ffa [0,4194304] 0 2024-04-23T02:14:16.846 INFO:tasks.workunit.client.0.smithi071.stdout:1/963: getdents db/d92 0 2024-04-23T02:14:16.846 INFO:tasks.workunit.client.0.smithi071.stdout:2/902: dread d0/d2/d34/f50 [0,4194304] 0 2024-04-23T02:14:16.847 INFO:tasks.workunit.client.0.smithi071.stdout:2/903: readlink d0/d2/d45/le8 0 2024-04-23T02:14:16.848 INFO:tasks.workunit.client.0.smithi071.stdout:1/964: symlink db/df/d5f/d6f/db2/l143 0 2024-04-23T02:14:16.850 INFO:tasks.workunit.client.0.smithi071.stdout:2/904: mkdir d0/d2/d49/d132 0 2024-04-23T02:14:16.851 INFO:tasks.workunit.client.0.smithi071.stdout:1/965: truncate db/df/d5f/d57/f5b 3738298 0 2024-04-23T02:14:16.860 INFO:tasks.workunit.client.0.smithi071.stdout:1/966: dwrite db/df/d13/d2f/f10d [0,4194304] 0 2024-04-23T02:14:16.862 INFO:tasks.workunit.client.0.smithi071.stdout:2/905: dwrite d0/d2/d26/d3a/dc3/d36/d6a/fc6 [4194304,4194304] 0 2024-04-23T02:14:16.869 INFO:tasks.workunit.client.0.smithi071.stdout:2/906: read - d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/db8/f120 zero size 2024-04-23T02:14:16.870 INFO:tasks.workunit.client.0.smithi071.stdout:1/967: dwrite db/d34/f7d [4194304,4194304] 0 2024-04-23T02:14:16.873 INFO:tasks.workunit.client.0.smithi071.stdout:1/968: creat db/df/d13/d20/dc0/d122/f144 x:0 0 0 2024-04-23T02:14:16.877 INFO:tasks.workunit.client.0.smithi071.stdout:2/907: truncate d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/db8/f120 780967 0 2024-04-23T02:14:16.878 INFO:tasks.workunit.client.0.smithi071.stdout:2/908: chown d0/d2/d34/d55/d96/la9 43 1 2024-04-23T02:14:16.879 INFO:tasks.workunit.client.0.smithi071.stdout:2/909: chown d0/d2/d26/d3a/dc3/d36/d5c/d85/lfb 7 1 2024-04-23T02:14:16.879 INFO:tasks.workunit.client.0.smithi071.stdout:2/910: write d0/d2/d34/f84 [4085174,53340] 0 2024-04-23T02:14:16.881 INFO:tasks.workunit.client.0.smithi071.stdout:2/911: write d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/f112 [15638,108753] 0 2024-04-23T02:14:16.884 INFO:tasks.workunit.client.0.smithi071.stdout:2/912: creat d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d12c/f133 x:0 0 0 2024-04-23T02:14:16.933 INFO:tasks.workunit.client.0.smithi071.stdout:1/969: truncate db/df/d3a/fa8 2431257 0 2024-04-23T02:14:16.933 INFO:tasks.workunit.client.0.smithi071.stdout:2/913: stat d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/f69 0 2024-04-23T02:14:16.935 INFO:tasks.workunit.client.0.smithi071.stdout:2/914: dread - d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/fad zero size 2024-04-23T02:14:16.936 INFO:tasks.workunit.client.0.smithi071.stdout:1/970: rename db/d11/f1d to db/df/d13/ddc/d12e/f145 0 2024-04-23T02:14:16.940 INFO:tasks.workunit.client.0.smithi071.stdout:1/971: getdents db/d11/d7f/da7/dd9/d100 0 2024-04-23T02:14:16.944 INFO:tasks.workunit.client.0.smithi071.stdout:1/972: unlink db/df/d13/d2f/d48/d9f/dfb/f108 0 2024-04-23T02:14:16.945 INFO:tasks.workunit.client.0.smithi071.stdout:1/973: mkdir db/d34/d8a/d146 0 2024-04-23T02:14:16.952 INFO:tasks.workunit.client.0.smithi071.stdout:1/974: dwrite db/df/d13/d20/dc0/d122/f144 [0,4194304] 0 2024-04-23T02:14:16.954 INFO:tasks.workunit.client.0.smithi071.stdout:1/975: symlink db/df/d13/d20/de5/l147 0 2024-04-23T02:14:16.964 INFO:tasks.workunit.client.0.smithi071.stdout:1/976: dwrite db/df/d13/d2f/d48/d85/f90 [4194304,4194304] 0 2024-04-23T02:14:16.966 INFO:tasks.workunit.client.0.smithi071.stdout:1/977: read db/d11/d7f/da7/dd9/d100/f81 [98115,30266] 0 2024-04-23T02:14:16.977 INFO:tasks.workunit.client.0.smithi071.stdout:1/978: write db/df/d3a/f66 [4985227,51426] 0 2024-04-23T02:14:16.978 INFO:tasks.workunit.client.0.smithi071.stdout:1/979: stat db/df/d3a/de7/ff5 0 2024-04-23T02:14:16.979 INFO:tasks.workunit.client.0.smithi071.stdout:1/980: dread - db/d11/d7f/da7/dd9/f10e zero size 2024-04-23T02:14:16.991 INFO:tasks.workunit.client.0.smithi071.stdout:1/981: dread db/df/d13/d2f/f3c [0,4194304] 0 2024-04-23T02:14:16.996 INFO:tasks.workunit.client.0.smithi071.stdout:1/982: rename db/df/d13/d20/dc0/ddf to db/d11/d7f/da7/d148 0 2024-04-23T02:14:16.998 INFO:tasks.workunit.client.0.smithi071.stdout:1/983: symlink db/d34/d106/d113/l149 0 2024-04-23T02:14:16.999 INFO:tasks.workunit.client.0.smithi071.stdout:1/984: fsync db/df/d118/f136 0 2024-04-23T02:14:17.000 INFO:tasks.workunit.client.0.smithi071.stdout:1/985: mkdir db/df/d13/d2f/d48/d141/d14a 0 2024-04-23T02:14:17.017 INFO:tasks.workunit.client.0.smithi071.stdout:1/986: dread db/df/d5f/d6f/db2/d86/fd6 [0,4194304] 0 2024-04-23T02:14:17.021 INFO:tasks.workunit.client.0.smithi071.stdout:1/987: creat db/df/d13/d2f/d48/d9f/d123/f14b x:0 0 0 2024-04-23T02:14:17.023 INFO:tasks.workunit.client.0.smithi071.stdout:1/988: creat db/df/d5f/f14c x:0 0 0 2024-04-23T02:14:17.026 INFO:tasks.workunit.client.0.smithi071.stdout:1/989: dread db/df/d5f/d6f/db2/d86/fd6 [0,4194304] 0 2024-04-23T02:14:17.029 INFO:tasks.workunit.client.0.smithi071.stdout:1/990: creat db/df/d5f/f14d x:0 0 0 2024-04-23T02:14:17.030 INFO:tasks.workunit.client.0.smithi071.stdout:1/991: symlink db/d34/d8a/d146/l14e 0 2024-04-23T02:14:17.036 INFO:tasks.workunit.client.0.smithi071.stdout:1/992: dwrite db/df/d5f/df0/f109 [0,4194304] 0 2024-04-23T02:14:17.038 INFO:tasks.workunit.client.0.smithi071.stdout:1/993: truncate db/df/d118/d12b/f130 632826 0 2024-04-23T02:14:17.040 INFO:tasks.workunit.client.0.smithi071.stdout:1/994: mkdir db/d11/d14f 0 2024-04-23T02:14:17.098 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:16 smithi192 ceph-mon[49662]: pgmap v35: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 112 MiB/s rd, 222 MiB/s wr, 525 op/s 2024-04-23T02:14:17.214 INFO:tasks.workunit.client.0.smithi071.stdout:9/998: sync 2024-04-23T02:14:17.218 INFO:tasks.workunit.client.0.smithi071.stdout:9/999: rename d1/d32/d41/db7/d39/df6/fb3 to d1/d4e/d54/d75/d104/f163 0 2024-04-23T02:14:17.278 INFO:tasks.workunit.client.0.smithi071.stdout:2/915: sync 2024-04-23T02:14:17.279 INFO:tasks.workunit.client.0.smithi071.stdout:1/995: sync 2024-04-23T02:14:17.279 INFO:tasks.workunit.client.0.smithi071.stdout:1/996: stat db/df/d13/d2b/f7e 0 2024-04-23T02:14:17.281 INFO:tasks.workunit.client.0.smithi071.stdout:2/916: creat d0/d2/d26/d3a/dc3/f134 x:0 0 0 2024-04-23T02:14:17.283 INFO:tasks.workunit.client.0.smithi071.stdout:2/917: creat d0/d2/da0/dc1/f135 x:0 0 0 2024-04-23T02:14:17.285 INFO:tasks.workunit.client.0.smithi071.stdout:2/918: rmdir d0/d2/d34/dbf 39 2024-04-23T02:14:17.286 INFO:tasks.workunit.client.0.smithi071.stdout:2/919: read - d0/d2/d26/d3a/f125 zero size 2024-04-23T02:14:17.288 INFO:tasks.workunit.client.0.smithi071.stdout:2/920: write d0/d2/da0/dc1/f135 [182008,100529] 0 2024-04-23T02:14:17.291 INFO:tasks.workunit.client.0.smithi071.stdout:2/921: truncate d0/d2/d26/d3a/dc3/d4f/f66 3948373 0 2024-04-23T02:14:17.303 INFO:tasks.workunit.client.0.smithi071.stdout:1/997: dwrite db/df/d13/f137 [0,4194304] 0 2024-04-23T02:14:17.305 INFO:tasks.workunit.client.0.smithi071.stdout:2/922: dwrite d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/f61 [0,4194304] 0 2024-04-23T02:14:17.307 INFO:tasks.workunit.client.0.smithi071.stdout:1/998: mkdir db/df/d13/ddc/d12e/d150 0 2024-04-23T02:14:17.307 INFO:tasks.workunit.client.0.smithi071.stdout:2/923: chown d0/d2/d34/d83/f114 266562154 1 2024-04-23T02:14:17.308 INFO:tasks.workunit.client.0.smithi071.stdout:2/924: write d0/d2/d26/d14/f1a [1448093,20228] 0 2024-04-23T02:14:17.309 INFO:tasks.workunit.client.0.smithi071.stdout:1/999: fsync db/df/d13/d2f/f3c 0 2024-04-23T02:14:17.312 INFO:tasks.workunit.client.0.smithi071.stdout:2/925: mkdir d0/d136 0 2024-04-23T02:14:17.316 INFO:tasks.workunit.client.0.smithi071.stdout:2/926: creat d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/dff/f137 x:0 0 0 2024-04-23T02:14:17.317 INFO:tasks.workunit.client.0.smithi071.stdout:2/927: mknod d0/d82/c138 0 2024-04-23T02:14:17.321 INFO:tasks.workunit.client.0.smithi071.stdout:2/928: getdents d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/de0 0 2024-04-23T02:14:17.323 INFO:tasks.workunit.client.0.smithi071.stdout:2/929: mkdir d0/d2/d26/d3a/dc3/d36/d139 0 2024-04-23T02:14:17.325 INFO:tasks.workunit.client.0.smithi071.stdout:2/930: dread - d0/d2/d26/d3a/dc3/d36/d5c/d85/fdc zero size 2024-04-23T02:14:17.325 INFO:tasks.workunit.client.0.smithi071.stdout:2/931: chown d0/d2/d26/d3a/dc3/d4f/c12a 19843 1 2024-04-23T02:14:17.328 INFO:tasks.workunit.client.0.smithi071.stdout:2/932: mkdir d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/db8/d13a 0 2024-04-23T02:14:17.329 INFO:tasks.workunit.client.0.smithi071.stdout:2/933: mknod d0/d2/c13b 0 2024-04-23T02:14:17.331 INFO:tasks.workunit.client.0.smithi071.stdout:2/934: creat d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/f13c x:0 0 0 2024-04-23T02:14:17.332 INFO:tasks.workunit.client.0.smithi071.stdout:2/935: truncate d0/d2/d34/f84 5046698 0 2024-04-23T02:14:17.335 INFO:tasks.workunit.client.0.smithi071.stdout:2/936: creat d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/de0/f13d x:0 0 0 2024-04-23T02:14:17.337 INFO:tasks.workunit.client.0.smithi071.stdout:2/937: symlink d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d12c/l13e 0 2024-04-23T02:14:17.338 INFO:tasks.workunit.client.0.smithi071.stdout:2/938: chown d0/d2/d26/d14/f1c 15009 1 2024-04-23T02:14:17.340 INFO:tasks.workunit.client.0.smithi071.stdout:2/939: symlink d0/d2/d26/d3a/d75/l13f 0 2024-04-23T02:14:17.349 INFO:tasks.workunit.client.0.smithi071.stdout:2/940: truncate d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/fd0 3274700 0 2024-04-23T02:14:17.352 INFO:tasks.workunit.client.0.smithi071.stdout:2/941: truncate d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/db8/f120 1651801 0 2024-04-23T02:14:17.549 INFO:tasks.workunit.client.0.smithi071.stdout:2/942: sync 2024-04-23T02:14:17.554 INFO:tasks.workunit.client.0.smithi071.stdout:2/943: write d0/d2/d45/f52 [925113,119019] 0 2024-04-23T02:14:17.561 INFO:tasks.workunit.client.0.smithi071.stdout:2/944: dwrite d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/d80/f5f [4194304,4194304] 0 2024-04-23T02:14:17.569 INFO:tasks.workunit.client.0.smithi071.stdout:2/945: dwrite d0/d2/d26/d3a/d75/f115 [0,4194304] 0 2024-04-23T02:14:17.574 INFO:tasks.workunit.client.0.smithi071.stdout:2/946: rename d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/db8/f120 to d0/d2/d34/dbf/f140 0 2024-04-23T02:14:17.577 INFO:tasks.workunit.client.0.smithi071.stdout:2/947: creat d0/d2/d26/f141 x:0 0 0 2024-04-23T02:14:17.579 INFO:tasks.workunit.client.0.smithi071.stdout:2/948: stat d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/de0/f13d 0 2024-04-23T02:14:17.583 INFO:tasks.workunit.client.0.smithi071.stdout:2/949: write d0/d2/d26/d73/f9b [149823,3411] 0 2024-04-23T02:14:17.584 INFO:tasks.workunit.client.0.smithi071.stdout:2/950: chown d0/d2/d45/d58/fcc 119 1 2024-04-23T02:14:17.588 INFO:tasks.workunit.client.0.smithi071.stdout:2/951: creat d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/db8/f142 x:0 0 0 2024-04-23T02:14:17.591 INFO:tasks.workunit.client.0.smithi071.stdout:2/952: dread d0/d82/f9d [4194304,4194304] 0 2024-04-23T02:14:17.593 INFO:tasks.workunit.client.0.smithi071.stdout:2/953: mkdir d0/d2/d26/d3a/dc3/d36/d5c/d85/de9/d143 0 2024-04-23T02:14:17.602 INFO:tasks.workunit.client.0.smithi071.stdout:2/954: dwrite d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/d80/f4e [0,4194304] 0 2024-04-23T02:14:17.607 INFO:tasks.workunit.client.0.smithi071.stdout:2/955: truncate d0/d82/f9d 4805532 0 2024-04-23T02:14:17.608 INFO:tasks.workunit.client.0.smithi071.stdout:2/956: chown d0/d2/d26/d3a/dc3/d36/c12e 381 1 2024-04-23T02:14:17.614 INFO:tasks.workunit.client.0.smithi071.stdout:2/957: dwrite d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/f69 [0,4194304] 0 2024-04-23T02:14:17.617 INFO:tasks.workunit.client.0.smithi071.stdout:2/958: write d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/fdb [266134,64707] 0 2024-04-23T02:14:17.621 INFO:tasks.workunit.client.0.smithi071.stdout:2/959: rename d0/d2/d26/d3a/dc3/d36/d5c/dfe to d0/d2/d49/d132/d144 0 2024-04-23T02:14:17.624 INFO:tasks.workunit.client.0.smithi071.stdout:2/960: rename d0/d2/fa3 to d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/f145 0 2024-04-23T02:14:17.650 INFO:tasks.workunit.client.0.smithi071.stdout:2/961: dread d0/d2/d26/d3a/f44 [0,4194304] 0 2024-04-23T02:14:17.652 INFO:tasks.workunit.client.0.smithi071.stdout:2/962: unlink d0/d2/d34/dbf/fd7 0 2024-04-23T02:14:17.653 INFO:tasks.workunit.client.0.smithi071.stdout:2/963: dread - d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/fe5 zero size 2024-04-23T02:14:17.655 INFO:tasks.workunit.client.0.smithi071.stdout:2/964: creat d0/d2/d26/d3a/dc3/d36/d6a/f146 x:0 0 0 2024-04-23T02:14:17.656 INFO:tasks.workunit.client.0.smithi071.stdout:2/965: stat d0/d2/d26 0 2024-04-23T02:14:17.660 INFO:tasks.workunit.client.0.smithi071.stdout:2/966: dread d0/d2/d45/f52 [0,4194304] 0 2024-04-23T02:14:17.661 INFO:tasks.workunit.client.0.smithi071.stdout:2/967: chown d0/d2/d26/d3a/l10d 3 1 2024-04-23T02:14:17.663 INFO:tasks.workunit.client.0.smithi071.stdout:2/968: dread - d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/db8/f142 zero size 2024-04-23T02:14:17.665 INFO:tasks.workunit.client.0.smithi071.stdout:2/969: symlink d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/l147 0 2024-04-23T02:14:17.665 INFO:tasks.workunit.client.0.smithi071.stdout:2/970: chown d0/d2/da0/f129 515676824 1 2024-04-23T02:14:17.670 INFO:tasks.workunit.client.0.smithi071.stdout:2/971: link d0/d2/d26/d14/f23 d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d30/d80/f148 0 2024-04-23T02:14:17.674 INFO:tasks.workunit.client.0.smithi071.stdout:2/972: dread d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/f112 [0,4194304] 0 2024-04-23T02:14:17.682 INFO:tasks.workunit.client.0.smithi071.stdout:2/973: truncate d0/d2/d26/d3a/dc3/d36/d6a/f127 6627438 0 2024-04-23T02:14:17.683 INFO:tasks.workunit.client.0.smithi071.stdout:2/974: fdatasync d0/d2/d26/d3a/dc3/f134 0 2024-04-23T02:14:17.685 INFO:tasks.workunit.client.0.smithi071.stdout:2/975: fsync d0/d2/f9 0 2024-04-23T02:14:17.687 INFO:tasks.workunit.client.0.smithi071.stdout:2/976: mkdir d0/d2/d49/d132/d149 0 2024-04-23T02:14:17.690 INFO:tasks.workunit.client.0.smithi071.stdout:2/977: read d0/d2/d26/d3a/dc3/d4f/f66 [3603213,105177] 0 2024-04-23T02:14:17.692 INFO:tasks.workunit.client.0.smithi071.stdout:2/978: chown d0/d2/d34/d83/fc9 1332 1 2024-04-23T02:14:17.693 INFO:tasks.workunit.client.0.smithi071.stdout:2/979: rmdir d0 39 2024-04-23T02:14:17.695 INFO:tasks.workunit.client.0.smithi071.stdout:2/980: stat d0/d2/cb1 0 2024-04-23T02:14:17.696 INFO:tasks.workunit.client.0.smithi071.stdout:2/981: stat d0/d2/d45 0 2024-04-23T02:14:17.699 INFO:tasks.workunit.client.0.smithi071.stdout:2/982: creat d0/d2/d26/d3a/dc3/d36/d139/f14a x:0 0 0 2024-04-23T02:14:17.706 INFO:tasks.workunit.client.0.smithi071.stdout:2/983: dwrite d0/f1e [0,4194304] 0 2024-04-23T02:14:17.711 INFO:tasks.workunit.client.0.smithi071.stdout:2/984: rename d0/d2/d26/d3a/d75/f8f to d0/d2/d26/d3a/dc3/d36/d6a/da1/f14b 0 2024-04-23T02:14:17.714 INFO:tasks.workunit.client.0.smithi071.stdout:2/985: symlink d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/de0/l14c 0 2024-04-23T02:14:17.724 INFO:tasks.workunit.client.0.smithi071.stdout:2/986: dwrite d0/d2/d26/d3a/dc3/d36/d5c/d85/de9/dfc/f104 [0,4194304] 0 2024-04-23T02:14:17.726 INFO:tasks.workunit.client.0.smithi071.stdout:2/987: chown d0/l39 1 1 2024-04-23T02:14:17.729 INFO:tasks.workunit.client.0.smithi071.stdout:2/988: symlink d0/d2/d34/dd4/l14d 0 2024-04-23T02:14:17.736 INFO:tasks.workunit.client.0.smithi071.stdout:2/989: link d0/d2/f29 d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/d12c/f14e 0 2024-04-23T02:14:17.740 INFO:tasks.workunit.client.0.smithi071.stdout:2/990: chown d0/d2/d45/f12d 1683580526 1 2024-04-23T02:14:17.743 INFO:tasks.workunit.client.0.smithi071.stdout:2/991: creat d0/d2/da0/db7/f14f x:0 0 0 2024-04-23T02:14:17.744 INFO:tasks.workunit.client.0.smithi071.stdout:2/992: stat d0/fab 0 2024-04-23T02:14:17.746 INFO:tasks.workunit.client.0.smithi071.stdout:2/993: fdatasync d0/d2/f29 0 2024-04-23T02:14:17.753 INFO:tasks.workunit.client.0.smithi071.stdout:2/994: rename d0/d2/l13 to d0/d2/d26/d3a/dc3/d36/d6a/da1/dcd/d101/d56/db0/l150 0 2024-04-23T02:14:17.756 INFO:tasks.workunit.client.0.smithi071.stdout:2/995: getdents d0/d2/d26/d3a/dc3/d4f/d98 0 2024-04-23T02:14:17.758 INFO:tasks.workunit.client.0.smithi071.stdout:2/996: mkdir d0/d2/d45/d151 0 2024-04-23T02:14:17.759 INFO:tasks.workunit.client.0.smithi071.stdout:2/997: fsync d0/d2/d26/d3a/dc3/d36/d5c/d85/de9/dfc/f104 0 2024-04-23T02:14:17.761 INFO:tasks.workunit.client.0.smithi071.stdout:2/998: getdents d0/d136 0 2024-04-23T02:14:17.762 INFO:tasks.workunit.client.0.smithi071.stdout:2/999: chown d0/d2/cf7 215420 1 2024-04-23T02:14:17.766 INFO:tasks.workunit.client.0.smithi071.stderr:+ rm -rf -- ./tmp.oALTRHA6YO 2024-04-23T02:14:19.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:19 smithi071 ceph-mon[39837]: pgmap v36: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 129 MiB/s rd, 263 MiB/s wr, 585 op/s 2024-04-23T02:14:19.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:19 smithi071 ceph-mon[39837]: Upgrade: Updating mgr.smithi071.ukdjvn 2024-04-23T02:14:19.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:19 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:19.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:19 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi071.ukdjvn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T02:14:19.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:19 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi071.ukdjvn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T02:14:19.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:19 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T02:14:19.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:19 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:14:19.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:19 smithi071 ceph-mon[39837]: Deploying daemon mgr.smithi071.ukdjvn on smithi071 2024-04-23T02:14:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:19 smithi192 ceph-mon[49662]: pgmap v36: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 129 MiB/s rd, 263 MiB/s wr, 585 op/s 2024-04-23T02:14:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:19 smithi192 ceph-mon[49662]: Upgrade: Updating mgr.smithi071.ukdjvn 2024-04-23T02:14:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:19 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:19 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi071.ukdjvn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T02:14:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:19 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi071.ukdjvn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T02:14:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:19 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T02:14:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:19 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:14:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:19 smithi192 ceph-mon[49662]: Deploying daemon mgr.smithi071.ukdjvn on smithi071 2024-04-23T02:14:21.407 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:21 smithi071 ceph-mon[39837]: pgmap v37: 65 pgs: 65 active+clean; 4.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 117 MiB/s rd, 230 MiB/s wr, 543 op/s 2024-04-23T02:14:21.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:21 smithi192 ceph-mon[49662]: pgmap v37: 65 pgs: 65 active+clean; 4.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 117 MiB/s rd, 230 MiB/s wr, 543 op/s 2024-04-23T02:14:23.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:23 smithi071 ceph-mon[39837]: pgmap v38: 65 pgs: 65 active+clean; 4.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 85 MiB/s rd, 182 MiB/s wr, 441 op/s 2024-04-23T02:14:23.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:23 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:23.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:23 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:23.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:23 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:14:23.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:23 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:23.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:23 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:23 smithi192 ceph-mon[49662]: pgmap v38: 65 pgs: 65 active+clean; 4.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 85 MiB/s rd, 182 MiB/s wr, 441 op/s 2024-04-23T02:14:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:23 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:23 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:23 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:14:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:23 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:23 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:25.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:25 smithi192 ceph-mon[49662]: pgmap v39: 65 pgs: 65 active+clean; 2.7 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 86 MiB/s rd, 185 MiB/s wr, 574 op/s 2024-04-23T02:14:25.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:25 smithi071 ceph-mon[39837]: pgmap v39: 65 pgs: 65 active+clean; 2.7 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 86 MiB/s rd, 185 MiB/s wr, 574 op/s 2024-04-23T02:14:26.977 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-23T02:14:26.977 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-23T02:14:27.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:27 smithi192 ceph-mon[49662]: pgmap v40: 65 pgs: 65 active+clean; 2.7 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 34 MiB/s rd, 74 MiB/s wr, 317 op/s 2024-04-23T02:14:27.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:27 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:14:27.570 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:27 smithi071 ceph-mon[39837]: pgmap v40: 65 pgs: 65 active+clean; 2.7 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 34 MiB/s rd, 74 MiB/s wr, 317 op/s 2024-04-23T02:14:27.570 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:27 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:14:29.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:29 smithi192 ceph-mon[49662]: pgmap v41: 65 pgs: 65 active+clean; 1.9 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 35 MiB/s rd, 75 MiB/s wr, 385 op/s 2024-04-23T02:14:29.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:29 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:29.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:29 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:29.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:29 smithi071 ceph-mon[39837]: pgmap v41: 65 pgs: 65 active+clean; 1.9 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 35 MiB/s rd, 75 MiB/s wr, 385 op/s 2024-04-23T02:14:29.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:29 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:29.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:29 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:30.729 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-23T02:14:30.729 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-23T02:14:30.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.780+0000 7f9697881700 1 -- 172.21.15.71:0/2831952791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96901040d0 msgr2=0x7f96901064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:30.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.780+0000 7f9697881700 1 --2- 172.21.15.71:0/2831952791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96901040d0 0x7f96901064c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9680009910 tx=0x7f9680009c20 comp rx=0 tx=0).stop 2024-04-23T02:14:30.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.781+0000 7f9697881700 1 -- 172.21.15.71:0/2831952791 shutdown_connections 2024-04-23T02:14:30.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.781+0000 7f9697881700 1 --2- 172.21.15.71:0/2831952791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96901040d0 0x7f96901064c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:30.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.781+0000 7f9697881700 1 --2- 172.21.15.71:0/2831952791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96901017a0 0x7f9690103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:30.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.781+0000 7f9697881700 1 -- 172.21.15.71:0/2831952791 >> 172.21.15.71:0/2831952791 conn(0x7f96900fb190 msgr2=0x7f96900fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:14:30.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.781+0000 7f9697881700 1 -- 172.21.15.71:0/2831952791 shutdown_connections 2024-04-23T02:14:30.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.781+0000 7f9697881700 1 -- 172.21.15.71:0/2831952791 wait complete. 2024-04-23T02:14:30.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.781+0000 7f9697881700 1 Processor -- start 2024-04-23T02:14:30.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.781+0000 7f9697881700 1 -- start start 2024-04-23T02:14:30.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.781+0000 7f9697881700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96901017a0 0x7f9690101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:14:30.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.781+0000 7f9697881700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96901040d0 0x7f96900ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:14:30.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.781+0000 7f9697881700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96900ffc90 con 0x7f96901040d0 2024-04-23T02:14:30.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.781+0000 7f9697881700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96900ffdd0 con 0x7f96901017a0 2024-04-23T02:14:30.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.782+0000 7f9694e1c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96901040d0 0x7f96900ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:14:30.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.782+0000 7f9694e1c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96901040d0 0x7f96900ff750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60860/0 (socket says 172.21.15.71:60860) 2024-04-23T02:14:30.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.782+0000 7f969561d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96901017a0 0x7f9690101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:14:30.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.782+0000 7f9694e1c700 1 -- 172.21.15.71:0/3759382220 learned_addr learned my addr 172.21.15.71:0/3759382220 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:14:30.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.782+0000 7f9694e1c700 1 -- 172.21.15.71:0/3759382220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96901017a0 msgr2=0x7f9690101100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:30.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.782+0000 7f9694e1c700 1 --2- 172.21.15.71:0/3759382220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96901017a0 0x7f9690101100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:30.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.782+0000 7f9694e1c700 1 -- 172.21.15.71:0/3759382220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96800095f0 con 0x7f96901040d0 2024-04-23T02:14:30.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.782+0000 7f969561d700 1 --2- 172.21.15.71:0/3759382220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96901017a0 0x7f9690101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T02:14:30.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.782+0000 7f9694e1c700 1 --2- 172.21.15.71:0/3759382220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96901040d0 0x7f96900ff750 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f968000bde0 tx=0x7f968000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:14:30.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.782+0000 7f96867fc700 1 -- 172.21.15.71:0/3759382220 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f968001d070 con 0x7f96901040d0 2024-04-23T02:14:30.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.782+0000 7f96867fc700 1 -- 172.21.15.71:0/3759382220 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f9680005dc0 con 0x7f96901040d0 2024-04-23T02:14:30.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.782+0000 7f9697881700 1 -- 172.21.15.71:0/3759382220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9690100050 con 0x7f96901040d0 2024-04-23T02:14:30.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.782+0000 7f96867fc700 1 -- 172.21.15.71:0/3759382220 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9680003b10 con 0x7f96901040d0 2024-04-23T02:14:30.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.782+0000 7f9697881700 1 -- 172.21.15.71:0/3759382220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96901004e0 con 0x7f96901040d0 2024-04-23T02:14:30.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.783+0000 7f9697881700 1 -- 172.21.15.71:0/3759382220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96901904d0 con 0x7f96901040d0 2024-04-23T02:14:30.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.784+0000 7f96867fc700 1 -- 172.21.15.71:0/3759382220 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) v1 ==== 90681+0+0 (secure 0 0 0) 0x7f96800048e0 con 0x7f96901040d0 2024-04-23T02:14:30.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.784+0000 7f96867fc700 1 --2- 172.21.15.71:0/3759382220 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f967c06ca20 0x7f967c06eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:14:30.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.785+0000 7f96867fc700 1 -- 172.21.15.71:0/3759382220 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f968008d880 con 0x7f96901040d0 2024-04-23T02:14:30.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.785+0000 7f969561d700 1 --2- 172.21.15.71:0/3759382220 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f967c06ca20 0x7f967c06eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:14:30.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.785+0000 7f969561d700 1 --2- 172.21.15.71:0/3759382220 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f967c06ca20 0x7f967c06eee0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f968c011490 tx=0x7f968c009260 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:14:30.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:30.789+0000 7f96867fc700 1 -- 172.21.15.71:0/3759382220 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f968005b360 con 0x7f96901040d0 2024-04-23T02:14:31.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.085+0000 7f9697881700 1 -- 172.21.15.71:0/3759382220 --> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f96900611d0 con 0x7f967c06ca20 2024-04-23T02:14:31.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.086+0000 7f96867fc700 1 -- 172.21.15.71:0/3759382220 <== mgr.24351 v2:172.21.15.192:6828/966231696 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f96900611d0 con 0x7f967c06ca20 2024-04-23T02:14:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.089+0000 7f9697881700 1 -- 172.21.15.71:0/3759382220 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f967c06ca20 msgr2=0x7f967c06eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.089+0000 7f9697881700 1 --2- 172.21.15.71:0/3759382220 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f967c06ca20 0x7f967c06eee0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f968c011490 tx=0x7f968c009260 comp rx=0 tx=0).stop 2024-04-23T02:14:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.089+0000 7f9697881700 1 -- 172.21.15.71:0/3759382220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96901040d0 msgr2=0x7f96900ff750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.089+0000 7f9697881700 1 --2- 172.21.15.71:0/3759382220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96901040d0 0x7f96900ff750 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f968000bde0 tx=0x7f968000be10 comp rx=0 tx=0).stop 2024-04-23T02:14:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.089+0000 7f9697881700 1 -- 172.21.15.71:0/3759382220 shutdown_connections 2024-04-23T02:14:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.089+0000 7f9697881700 1 --2- 172.21.15.71:0/3759382220 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f967c06ca20 0x7f967c06eee0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.089+0000 7f9697881700 1 --2- 172.21.15.71:0/3759382220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96901017a0 0x7f9690101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:31.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.089+0000 7f9697881700 1 --2- 172.21.15.71:0/3759382220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96901040d0 0x7f96900ff750 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:31.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.089+0000 7f9697881700 1 -- 172.21.15.71:0/3759382220 >> 172.21.15.71:0/3759382220 conn(0x7f96900fb190 msgr2=0x7f96900fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:14:31.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.089+0000 7f9697881700 1 -- 172.21.15.71:0/3759382220 shutdown_connections 2024-04-23T02:14:31.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.089+0000 7f9697881700 1 -- 172.21.15.71:0/3759382220 wait complete. 2024-04-23T02:14:31.104 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:14:31.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:30 smithi071 ceph-mon[39837]: pgmap v42: 65 pgs: 65 active+clean; 1.3 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 18 MiB/s rd, 34 MiB/s wr, 362 op/s 2024-04-23T02:14:31.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:30 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:31.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:30 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:31.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.247+0000 7f0bdb483700 1 -- 172.21.15.71:0/2477360284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd4101770 msgr2=0x7f0bd4103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:31.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.247+0000 7f0bdb483700 1 --2- 172.21.15.71:0/2477360284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd4101770 0x7f0bd4103b60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009910 tx=0x7f0bd0009c20 comp rx=0 tx=0).stop 2024-04-23T02:14:31.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.247+0000 7f0bdb483700 1 -- 172.21.15.71:0/2477360284 shutdown_connections 2024-04-23T02:14:31.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.247+0000 7f0bdb483700 1 --2- 172.21.15.71:0/2477360284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bd41040a0 0x7f0bd4106490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:31.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.247+0000 7f0bdb483700 1 --2- 172.21.15.71:0/2477360284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd4101770 0x7f0bd4103b60 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:31.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.247+0000 7f0bdb483700 1 -- 172.21.15.71:0/2477360284 >> 172.21.15.71:0/2477360284 conn(0x7f0bd40fb120 msgr2=0x7f0bd40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:14:31.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.247+0000 7f0bdb483700 1 -- 172.21.15.71:0/2477360284 shutdown_connections 2024-04-23T02:14:31.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.247+0000 7f0bdb483700 1 -- 172.21.15.71:0/2477360284 wait complete. 2024-04-23T02:14:31.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.248+0000 7f0bdb483700 1 Processor -- start 2024-04-23T02:14:31.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.248+0000 7f0bdb483700 1 -- start start 2024-04-23T02:14:31.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.248+0000 7f0bdb483700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bd4101770 0x7f0bd41963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:14:31.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.248+0000 7f0bdb483700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd41040a0 0x7f0bd41968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:14:31.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.248+0000 7f0bdb483700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd4196f00 con 0x7f0bd41040a0 2024-04-23T02:14:31.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.248+0000 7f0bdb483700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd4197040 con 0x7f0bd4101770 2024-04-23T02:14:31.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.248+0000 7f0bd8a1e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd41040a0 0x7f0bd41968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:14:31.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.248+0000 7f0bd921f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bd4101770 0x7f0bd41963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:14:31.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.248+0000 7f0bd8a1e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd41040a0 0x7f0bd41968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60876/0 (socket says 172.21.15.71:60876) 2024-04-23T02:14:31.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.248+0000 7f0bd8a1e700 1 -- 172.21.15.71:0/2257793631 learned_addr learned my addr 172.21.15.71:0/2257793631 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:14:31.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.248+0000 7f0bd8a1e700 1 -- 172.21.15.71:0/2257793631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bd4101770 msgr2=0x7f0bd41963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:31.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.248+0000 7f0bd8a1e700 1 --2- 172.21.15.71:0/2257793631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bd4101770 0x7f0bd41963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:31.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.248+0000 7f0bd8a1e700 1 -- 172.21.15.71:0/2257793631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd00095f0 con 0x7f0bd41040a0 2024-04-23T02:14:31.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.248+0000 7f0bd921f700 1 --2- 172.21.15.71:0/2257793631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bd4101770 0x7f0bd41963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T02:14:31.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.249+0000 7f0bd8a1e700 1 --2- 172.21.15.71:0/2257793631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd41040a0 0x7f0bd41968e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400e920 tx=0x7f0bc400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:14:31.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.249+0000 7f0bca7fc700 1 -- 172.21.15.71:0/2257793631 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bc400c9d0 con 0x7f0bd41040a0 2024-04-23T02:14:31.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.249+0000 7f0bdb483700 1 -- 172.21.15.71:0/2257793631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd406a950 con 0x7f0bd41040a0 2024-04-23T02:14:31.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.249+0000 7f0bca7fc700 1 -- 172.21.15.71:0/2257793631 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f0bc4010430 con 0x7f0bd41040a0 2024-04-23T02:14:31.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.249+0000 7f0bca7fc700 1 -- 172.21.15.71:0/2257793631 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bc4018630 con 0x7f0bd41040a0 2024-04-23T02:14:31.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.249+0000 7f0bdb483700 1 -- 172.21.15.71:0/2257793631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd406aea0 con 0x7f0bd41040a0 2024-04-23T02:14:31.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.250+0000 7f0bdb483700 1 -- 172.21.15.71:0/2257793631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd41096c0 con 0x7f0bd41040a0 2024-04-23T02:14:31.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.251+0000 7f0bca7fc700 1 -- 172.21.15.71:0/2257793631 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) v1 ==== 90681+0+0 (secure 0 0 0) 0x7f0bc40105a0 con 0x7f0bd41040a0 2024-04-23T02:14:31.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.252+0000 7f0bca7fc700 1 --2- 172.21.15.71:0/2257793631 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f0bc006ca20 0x7f0bc006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:14:31.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.252+0000 7f0bca7fc700 1 -- 172.21.15.71:0/2257793631 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f0bc4014070 con 0x7f0bd41040a0 2024-04-23T02:14:31.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.252+0000 7f0bd921f700 1 --2- 172.21.15.71:0/2257793631 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f0bc006ca20 0x7f0bc006eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:14:31.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.252+0000 7f0bd921f700 1 --2- 172.21.15.71:0/2257793631 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f0bc006ca20 0x7f0bc006eee0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009fd0 tx=0x7f0bd000b5a0 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:14:31.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.257+0000 7f0bca7fc700 1 -- 172.21.15.71:0/2257793631 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f0bc40529b0 con 0x7f0bd41040a0 2024-04-23T02:14:31.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:30 smithi192 ceph-mon[49662]: pgmap v42: 65 pgs: 65 active+clean; 1.3 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 18 MiB/s rd, 34 MiB/s wr, 362 op/s 2024-04-23T02:14:31.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:30 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:31.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:30 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:31.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.573+0000 7f0bdb483700 1 -- 172.21.15.71:0/2257793631 --> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0bd406aae0 con 0x7f0bc006ca20 2024-04-23T02:14:31.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.574+0000 7f0bca7fc700 1 -- 172.21.15.71:0/2257793631 <== mgr.24351 v2:172.21.15.192:6828/966231696 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f0bd406aae0 con 0x7f0bc006ca20 2024-04-23T02:14:31.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.577+0000 7f0bdb483700 1 -- 172.21.15.71:0/2257793631 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f0bc006ca20 msgr2=0x7f0bc006eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:31.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.577+0000 7f0bdb483700 1 --2- 172.21.15.71:0/2257793631 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f0bc006ca20 0x7f0bc006eee0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009fd0 tx=0x7f0bd000b5a0 comp rx=0 tx=0).stop 2024-04-23T02:14:31.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.577+0000 7f0bdb483700 1 -- 172.21.15.71:0/2257793631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd41040a0 msgr2=0x7f0bd41968e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:31.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.577+0000 7f0bdb483700 1 --2- 172.21.15.71:0/2257793631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd41040a0 0x7f0bd41968e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400e920 tx=0x7f0bc400ec30 comp rx=0 tx=0).stop 2024-04-23T02:14:31.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.577+0000 7f0bdb483700 1 -- 172.21.15.71:0/2257793631 shutdown_connections 2024-04-23T02:14:31.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.577+0000 7f0bdb483700 1 --2- 172.21.15.71:0/2257793631 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f0bc006ca20 0x7f0bc006eee0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:31.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.577+0000 7f0bdb483700 1 --2- 172.21.15.71:0/2257793631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bd4101770 0x7f0bd41963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:31.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.577+0000 7f0bdb483700 1 --2- 172.21.15.71:0/2257793631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd41040a0 0x7f0bd41968e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:31.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.577+0000 7f0bdb483700 1 -- 172.21.15.71:0/2257793631 >> 172.21.15.71:0/2257793631 conn(0x7f0bd40fb120 msgr2=0x7f0bd40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:14:31.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.577+0000 7f0bdb483700 1 -- 172.21.15.71:0/2257793631 shutdown_connections 2024-04-23T02:14:31.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.577+0000 7f0bdb483700 1 -- 172.21.15.71:0/2257793631 wait complete. 2024-04-23T02:14:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.715+0000 7f6cd4f16700 1 -- 172.21.15.71:0/295345217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cd0102000 msgr2=0x7f6cd0102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.715+0000 7f6cd4f16700 1 --2- 172.21.15.71:0/295345217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cd0102000 0x7f6cd0102460 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009910 tx=0x7f6cc4009c20 comp rx=0 tx=0).stop 2024-04-23T02:14:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.715+0000 7f6cd4f16700 1 -- 172.21.15.71:0/295345217 shutdown_connections 2024-04-23T02:14:31.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.715+0000 7f6cd4f16700 1 --2- 172.21.15.71:0/295345217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cd0102000 0x7f6cd0102460 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:31.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.715+0000 7f6cd4f16700 1 --2- 172.21.15.71:0/295345217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cd00fef80 0x7f6cd00ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:31.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.715+0000 7f6cd4f16700 1 -- 172.21.15.71:0/295345217 >> 172.21.15.71:0/295345217 conn(0x7f6cd00fa8c0 msgr2=0x7f6cd00fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:14:31.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.715+0000 7f6cd4f16700 1 -- 172.21.15.71:0/295345217 shutdown_connections 2024-04-23T02:14:31.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.715+0000 7f6cd4f16700 1 -- 172.21.15.71:0/295345217 wait complete. 2024-04-23T02:14:31.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.716+0000 7f6cd4f16700 1 Processor -- start 2024-04-23T02:14:31.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.716+0000 7f6cd4f16700 1 -- start start 2024-04-23T02:14:31.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.716+0000 7f6cd4f16700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cd00fef80 0x7f6cd01963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:14:31.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.716+0000 7f6cd4f16700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cd0102000 0x7f6cd0196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:14:31.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.716+0000 7f6cd4f16700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd0196f20 con 0x7f6cd0102000 2024-04-23T02:14:31.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.716+0000 7f6cd4f16700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd0197060 con 0x7f6cd00fef80 2024-04-23T02:14:31.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.717+0000 7f6ccdd9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cd0102000 0x7f6cd0196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:14:31.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.717+0000 7f6ccdd9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cd0102000 0x7f6cd0196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60904/0 (socket says 172.21.15.71:60904) 2024-04-23T02:14:31.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.717+0000 7f6ccdd9b700 1 -- 172.21.15.71:0/679496193 learned_addr learned my addr 172.21.15.71:0/679496193 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:14:31.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.717+0000 7f6cce59c700 1 --2- 172.21.15.71:0/679496193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cd00fef80 0x7f6cd01963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:14:31.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.717+0000 7f6ccdd9b700 1 -- 172.21.15.71:0/679496193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cd00fef80 msgr2=0x7f6cd01963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:31.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.717+0000 7f6ccdd9b700 1 --2- 172.21.15.71:0/679496193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cd00fef80 0x7f6cd01963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:31.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.717+0000 7f6ccdd9b700 1 -- 172.21.15.71:0/679496193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc40095f0 con 0x7f6cd0102000 2024-04-23T02:14:31.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.717+0000 7f6ccdd9b700 1 --2- 172.21.15.71:0/679496193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cd0102000 0x7f6cd0196900 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400bde0 tx=0x7f6cc400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:14:31.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.718+0000 7f6cbf7fe700 1 -- 172.21.15.71:0/679496193 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cc401d070 con 0x7f6cd0102000 2024-04-23T02:14:31.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.718+0000 7f6cbf7fe700 1 -- 172.21.15.71:0/679496193 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f6cc4005dc0 con 0x7f6cd0102000 2024-04-23T02:14:31.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.718+0000 7f6cd4f16700 1 -- 172.21.15.71:0/679496193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd019bab0 con 0x7f6cd0102000 2024-04-23T02:14:31.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.718+0000 7f6cbf7fe700 1 -- 172.21.15.71:0/679496193 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cc4003b10 con 0x7f6cd0102000 2024-04-23T02:14:31.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.718+0000 7f6cd4f16700 1 -- 172.21.15.71:0/679496193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd019bf40 con 0x7f6cd0102000 2024-04-23T02:14:31.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.719+0000 7f6cd4f16700 1 -- 172.21.15.71:0/679496193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd01096c0 con 0x7f6cd0102000 2024-04-23T02:14:31.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.720+0000 7f6cbf7fe700 1 -- 172.21.15.71:0/679496193 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) v1 ==== 90681+0+0 (secure 0 0 0) 0x7f6cc40048e0 con 0x7f6cd0102000 2024-04-23T02:14:31.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.720+0000 7f6cbf7fe700 1 --2- 172.21.15.71:0/679496193 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f6cb806ca20 0x7f6cb806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:14:31.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.723+0000 7f6cbf7fe700 1 -- 172.21.15.71:0/679496193 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f6cc405b5f0 con 0x7f6cd0102000 2024-04-23T02:14:31.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.723+0000 7f6cce59c700 1 --2- 172.21.15.71:0/679496193 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f6cb806ca20 0x7f6cb806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:14:31.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.723+0000 7f6cce59c700 1 --2- 172.21.15.71:0/679496193 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f6cb806ca20 0x7f6cb806eee0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0068b50 tx=0x7f6cc0009260 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:14:31.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.728+0000 7f6cbf7fe700 1 -- 172.21.15.71:0/679496193 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f6cc4092060 con 0x7f6cd0102000 2024-04-23T02:14:31.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:31.943+0000 7f6cbf7fe700 1 -- 172.21.15.71:0/679496193 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 25) v1 ==== 90946+0+0 (secure 0 0 0) 0x7f6cc408b690 con 0x7f6cd0102000 2024-04-23T02:14:32.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.013+0000 7f6cd4f16700 1 -- 172.21.15.71:0/679496193 --> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cd0106950 con 0x7f6cb806ca20 2024-04-23T02:14:32.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.026+0000 7f6cbf7fe700 1 -- 172.21.15.71:0/679496193 <== mgr.24351 v2:172.21.15.192:6828/966231696 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6cd0106950 con 0x7f6cb806ca20 2024-04-23T02:14:32.030 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:14:32.030 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (7m) 2s ago 9m 26.4M - 0.25.0 c8568f914cd2 8261e17319fe 2024-04-23T02:14:32.030 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (9m) 2s ago 9m 7805k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:14:32.030 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (8m) 71s ago 8m 7616k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:14:32.030 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (9m) 2s ago 9m 6656k - 18.2.1 5be31c24972a 258512dcbc00 2024-04-23T02:14:32.030 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (8m) 71s ago 8m 6656k - 18.2.1 5be31c24972a feea2c9896a6 2024-04-23T02:14:32.030 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (7m) 2s ago 9m 87.7M - 9.4.7 954c08fa6188 48ea3e02b806 2024-04-23T02:14:32.030 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (3m) 2s ago 3m 153M - 18.2.1 5be31c24972a a515de2a841c 2024-04-23T02:14:32.030 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (3m) 2s ago 3m 174M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:14:32.031 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (3m) 71s ago 3m 63.8M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:14:32.031 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (3m) 71s ago 3m 76.6M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:14:32.031 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (10s) 2s ago 11m 72.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:14:32.031 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (90s) 71s ago 8m 483M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 151cb2226032 2024-04-23T02:14:32.031 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (11m) 2s ago 11m 58.0M 2048M 18.2.1 5be31c24972a ff40abe63fd9 2024-04-23T02:14:32.031 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (8m) 71s ago 8m 40.3M 2048M 18.2.1 5be31c24972a 9c03e90acc95 2024-04-23T02:14:32.031 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (9m) 2s ago 9m 19.3M - 1.5.0 0da6a335fe13 3f4d0aaee04e 2024-04-23T02:14:32.031 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (8m) 71s ago 8m 16.7M - 1.5.0 0da6a335fe13 cfd0b49d770c 2024-04-23T02:14:32.031 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (7m) 2s ago 7m 161M 1227M 18.2.1 5be31c24972a d3d5f2fa7267 2024-04-23T02:14:32.031 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (7m) 2s ago 7m 182M 1227M 18.2.1 5be31c24972a 3992157ff0fd 2024-04-23T02:14:32.031 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (6m) 2s ago 6m 139M 1227M 18.2.1 5be31c24972a 12380b6a15a2 2024-04-23T02:14:32.031 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (6m) 71s ago 6m 79.5M 2251M 18.2.1 5be31c24972a d5f587ee3a22 2024-04-23T02:14:32.031 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (5m) 71s ago 5m 72.5M 2251M 18.2.1 5be31c24972a 958d046005f0 2024-04-23T02:14:32.031 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (5m) 71s ago 5m 69.2M 2251M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:14:32.032 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (24s) 2s ago 8m 42.2M - 2.43.0 a07b618ecd1d 44bef23ad991 2024-04-23T02:14:32.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.029+0000 7f6cd4f16700 1 -- 172.21.15.71:0/679496193 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f6cb806ca20 msgr2=0x7f6cb806eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:32.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.029+0000 7f6cd4f16700 1 --2- 172.21.15.71:0/679496193 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f6cb806ca20 0x7f6cb806eee0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0068b50 tx=0x7f6cc0009260 comp rx=0 tx=0).stop 2024-04-23T02:14:32.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.029+0000 7f6cd4f16700 1 -- 172.21.15.71:0/679496193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cd0102000 msgr2=0x7f6cd0196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:32.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.029+0000 7f6cd4f16700 1 --2- 172.21.15.71:0/679496193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cd0102000 0x7f6cd0196900 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400bde0 tx=0x7f6cc400be10 comp rx=0 tx=0).stop 2024-04-23T02:14:32.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.030+0000 7f6cd4f16700 1 -- 172.21.15.71:0/679496193 shutdown_connections 2024-04-23T02:14:32.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.030+0000 7f6cd4f16700 1 --2- 172.21.15.71:0/679496193 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f6cb806ca20 0x7f6cb806eee0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:32.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.030+0000 7f6cd4f16700 1 --2- 172.21.15.71:0/679496193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cd00fef80 0x7f6cd01963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:32.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.030+0000 7f6cd4f16700 1 --2- 172.21.15.71:0/679496193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cd0102000 0x7f6cd0196900 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:32.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.030+0000 7f6cd4f16700 1 -- 172.21.15.71:0/679496193 >> 172.21.15.71:0/679496193 conn(0x7f6cd00fa8c0 msgr2=0x7f6cd0105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:14:32.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.033+0000 7f6cd4f16700 1 -- 172.21.15.71:0/679496193 shutdown_connections 2024-04-23T02:14:32.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.033+0000 7f6cd4f16700 1 -- 172.21.15.71:0/679496193 wait complete. 2024-04-23T02:14:32.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.178+0000 7fbada613700 1 -- 172.21.15.71:0/2786128378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbad41084a0 msgr2=0x7fbad4108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:32.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.178+0000 7fbada613700 1 --2- 172.21.15.71:0/2786128378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbad41084a0 0x7fbad4108900 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbac8009870 tx=0x7fbac8009b80 comp rx=0 tx=0).stop 2024-04-23T02:14:32.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.179+0000 7fbada613700 1 -- 172.21.15.71:0/2786128378 shutdown_connections 2024-04-23T02:14:32.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.179+0000 7fbada613700 1 --2- 172.21.15.71:0/2786128378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbad41084a0 0x7fbad4108900 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:32.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.179+0000 7fbada613700 1 --2- 172.21.15.71:0/2786128378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbad41072a0 0x7fbad41076c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:32.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.179+0000 7fbada613700 1 -- 172.21.15.71:0/2786128378 >> 172.21.15.71:0/2786128378 conn(0x7fbad4076250 msgr2=0x7fbad40786b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:14:32.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.182+0000 7fbada613700 1 -- 172.21.15.71:0/2786128378 shutdown_connections 2024-04-23T02:14:32.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.182+0000 7fbada613700 1 -- 172.21.15.71:0/2786128378 wait complete. 2024-04-23T02:14:32.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.183+0000 7fbada613700 1 Processor -- start 2024-04-23T02:14:32.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.183+0000 7fbada613700 1 -- start start 2024-04-23T02:14:32.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.183+0000 7fbada613700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbad41072a0 0x7fbad4080c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:14:32.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.183+0000 7fbada613700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbad41084a0 0x7fbad4081190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:14:32.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.183+0000 7fbada613700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad40817b0 con 0x7fbad41084a0 2024-04-23T02:14:32.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.183+0000 7fbada613700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad40818f0 con 0x7fbad41072a0 2024-04-23T02:14:32.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.184+0000 7fbad3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbad41072a0 0x7fbad4080c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:14:32.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.184+0000 7fbad37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbad41084a0 0x7fbad4081190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:14:32.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.184+0000 7fbad37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbad41084a0 0x7fbad4081190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60920/0 (socket says 172.21.15.71:60920) 2024-04-23T02:14:32.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.184+0000 7fbad37fe700 1 -- 172.21.15.71:0/1307117689 learned_addr learned my addr 172.21.15.71:0/1307117689 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:14:32.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.185+0000 7fbad37fe700 1 -- 172.21.15.71:0/1307117689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbad41072a0 msgr2=0x7fbad4080c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:32.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.185+0000 7fbad37fe700 1 --2- 172.21.15.71:0/1307117689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbad41072a0 0x7fbad4080c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:32.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.185+0000 7fbad37fe700 1 -- 172.21.15.71:0/1307117689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac8009520 con 0x7fbad41084a0 2024-04-23T02:14:32.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.185+0000 7fbad37fe700 1 --2- 172.21.15.71:0/1307117689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbad41084a0 0x7fbad4081190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbac8005950 tx=0x7fbac800f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:14:32.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.185+0000 7fbad17fa700 1 -- 172.21.15.71:0/1307117689 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbac801c070 con 0x7fbad41084a0 2024-04-23T02:14:32.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.185+0000 7fbad17fa700 1 -- 172.21.15.71:0/1307117689 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fbac800fda0 con 0x7fbad41084a0 2024-04-23T02:14:32.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.185+0000 7fbad17fa700 1 -- 172.21.15.71:0/1307117689 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbac80173f0 con 0x7fbad41084a0 2024-04-23T02:14:32.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.186+0000 7fbada613700 1 -- 172.21.15.71:0/1307117689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad406a8f0 con 0x7fbad41084a0 2024-04-23T02:14:32.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.186+0000 7fbada613700 1 -- 172.21.15.71:0/1307117689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad406ade0 con 0x7fbad41084a0 2024-04-23T02:14:32.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.187+0000 7fbada613700 1 -- 172.21.15.71:0/1307117689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad410fcc0 con 0x7fbad41084a0 2024-04-23T02:14:32.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.194+0000 7fbad17fa700 1 -- 172.21.15.71:0/1307117689 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 90946+0+0 (secure 0 0 0) 0x7fbac8020070 con 0x7fbad41084a0 2024-04-23T02:14:32.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.194+0000 7fbad17fa700 1 --2- 172.21.15.71:0/1307117689 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7fbabc06cf20 0x7fbabc06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:14:32.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.194+0000 7fbad17fa700 1 -- 172.21.15.71:0/1307117689 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fbac808e5a0 con 0x7fbad41084a0 2024-04-23T02:14:32.199 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:31 smithi071 ceph-mon[39837]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:14:32.200 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:31 smithi071 ceph-mon[39837]: Standby manager daemon smithi071.ukdjvn restarted 2024-04-23T02:14:32.200 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:31 smithi071 ceph-mon[39837]: Standby manager daemon smithi071.ukdjvn started 2024-04-23T02:14:32.200 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:31 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi071.ukdjvn/crt"}]: dispatch 2024-04-23T02:14:32.200 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:31 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T02:14:32.200 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:31 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi071.ukdjvn/key"}]: dispatch 2024-04-23T02:14:32.200 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:31 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T02:14:32.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.196+0000 7fbad17fa700 1 -- 172.21.15.71:0/1307117689 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fbac805c0e0 con 0x7fbad41084a0 2024-04-23T02:14:32.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.198+0000 7fbad3fff700 1 --2- 172.21.15.71:0/1307117689 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7fbabc06cf20 0x7fbabc06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:14:32.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.200+0000 7fbad3fff700 1 --2- 172.21.15.71:0/1307117689 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7fbabc06cf20 0x7fbabc06f3e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbad4108300 tx=0x7fbac4009310 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:14:32.311 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:31 smithi192 ceph-mon[49662]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:14:32.311 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:31 smithi192 ceph-mon[49662]: Standby manager daemon smithi071.ukdjvn restarted 2024-04-23T02:14:32.311 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:31 smithi192 ceph-mon[49662]: Standby manager daemon smithi071.ukdjvn started 2024-04-23T02:14:32.312 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:31 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi071.ukdjvn/crt"}]: dispatch 2024-04-23T02:14:32.312 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:31 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T02:14:32.312 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:31 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi071.ukdjvn/key"}]: dispatch 2024-04-23T02:14:32.312 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:31 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T02:14:32.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.666+0000 7fbada613700 1 -- 172.21.15.71:0/1307117689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbad410feb0 con 0x7fbad41084a0 2024-04-23T02:14:32.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.666+0000 7fbad17fa700 1 -- 172.21.15.71:0/1307117689 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7fbac805bc70 con 0x7fbad41084a0 2024-04-23T02:14:32.671 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:14:32.671 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:14:32.671 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-23T02:14:32.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:14:32.671 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:14:32.671 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:14:32.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:14:32.671 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:14:32.671 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-23T02:14:32.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:14:32.671 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:14:32.672 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-23T02:14:32.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:14:32.672 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:14:32.672 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-23T02:14:32.672 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:14:32.672 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:14:32.672 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:14:32.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.669+0000 7fbada613700 1 -- 172.21.15.71:0/1307117689 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7fbabc06cf20 msgr2=0x7fbabc06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:32.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.669+0000 7fbada613700 1 --2- 172.21.15.71:0/1307117689 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7fbabc06cf20 0x7fbabc06f3e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbad4108300 tx=0x7fbac4009310 comp rx=0 tx=0).stop 2024-04-23T02:14:32.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.670+0000 7fbada613700 1 -- 172.21.15.71:0/1307117689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbad41084a0 msgr2=0x7fbad4081190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:32.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.670+0000 7fbada613700 1 --2- 172.21.15.71:0/1307117689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbad41084a0 0x7fbad4081190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbac8005950 tx=0x7fbac800f460 comp rx=0 tx=0).stop 2024-04-23T02:14:32.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.670+0000 7fbada613700 1 -- 172.21.15.71:0/1307117689 shutdown_connections 2024-04-23T02:14:32.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.670+0000 7fbada613700 1 --2- 172.21.15.71:0/1307117689 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7fbabc06cf20 0x7fbabc06f3e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:32.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.670+0000 7fbada613700 1 --2- 172.21.15.71:0/1307117689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbad41072a0 0x7fbad4080c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:32.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.670+0000 7fbada613700 1 --2- 172.21.15.71:0/1307117689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbad41084a0 0x7fbad4081190 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:32.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.670+0000 7fbada613700 1 -- 172.21.15.71:0/1307117689 >> 172.21.15.71:0/1307117689 conn(0x7fbad4076250 msgr2=0x7fbad410b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:14:32.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.670+0000 7fbada613700 1 -- 172.21.15.71:0/1307117689 shutdown_connections 2024-04-23T02:14:32.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.670+0000 7fbada613700 1 -- 172.21.15.71:0/1307117689 wait complete. 2024-04-23T02:14:32.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.804+0000 7f114ec26700 1 -- 172.21.15.71:0/2948224416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11480ff590 msgr2=0x7f11481019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:32.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.804+0000 7f114ec26700 1 --2- 172.21.15.71:0/2948224416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11480ff590 0x7f11481019c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f113c009910 tx=0x7f113c009c20 comp rx=0 tx=0).stop 2024-04-23T02:14:32.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.804+0000 7f114ec26700 1 -- 172.21.15.71:0/2948224416 shutdown_connections 2024-04-23T02:14:32.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.804+0000 7f114ec26700 1 --2- 172.21.15.71:0/2948224416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1148101f00 0x7f1148104390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:32.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.804+0000 7f114ec26700 1 --2- 172.21.15.71:0/2948224416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11480ff590 0x7f11481019c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:32.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.804+0000 7f114ec26700 1 -- 172.21.15.71:0/2948224416 >> 172.21.15.71:0/2948224416 conn(0x7f11480fb190 msgr2=0x7f11480fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:14:32.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.804+0000 7f114ec26700 1 -- 172.21.15.71:0/2948224416 shutdown_connections 2024-04-23T02:14:32.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.804+0000 7f114ec26700 1 -- 172.21.15.71:0/2948224416 wait complete. 2024-04-23T02:14:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.805+0000 7f114ec26700 1 Processor -- start 2024-04-23T02:14:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.805+0000 7f114ec26700 1 -- start start 2024-04-23T02:14:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.805+0000 7f114ec26700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11480ff590 0x7f114819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:14:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.805+0000 7f114ec26700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1148101f00 0x7f114819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:14:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.805+0000 7f114ec26700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114819d600 con 0x7f1148101f00 2024-04-23T02:14:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.805+0000 7f114ec26700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114819d740 con 0x7f11480ff590 2024-04-23T02:14:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.805+0000 7f1147fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1148101f00 0x7f114819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:14:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.805+0000 7f1147fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1148101f00 0x7f114819cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60938/0 (socket says 172.21.15.71:60938) 2024-04-23T02:14:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.805+0000 7f1147fff700 1 -- 172.21.15.71:0/903798220 learned_addr learned my addr 172.21.15.71:0/903798220 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:14:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.805+0000 7f114c9c2700 1 --2- 172.21.15.71:0/903798220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11480ff590 0x7f114819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:14:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.806+0000 7f1147fff700 1 -- 172.21.15.71:0/903798220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11480ff590 msgr2=0x7f114819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.806+0000 7f1147fff700 1 --2- 172.21.15.71:0/903798220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11480ff590 0x7f114819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.806+0000 7f1147fff700 1 -- 172.21.15.71:0/903798220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113c0095f0 con 0x7f1148101f00 2024-04-23T02:14:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.806+0000 7f1147fff700 1 --2- 172.21.15.71:0/903798220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1148101f00 0x7f114819cfe0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f113400b4b0 tx=0x7f113400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:14:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.807+0000 7f1145ffb700 1 -- 172.21.15.71:0/903798220 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1134011800 con 0x7f1148101f00 2024-04-23T02:14:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.807+0000 7f1145ffb700 1 -- 172.21.15.71:0/903798220 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1134011df0 con 0x7f1148101f00 2024-04-23T02:14:32.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.807+0000 7f1145ffb700 1 -- 172.21.15.71:0/903798220 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1134010e50 con 0x7f1148101f00 2024-04-23T02:14:32.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.808+0000 7f114ec26700 1 -- 172.21.15.71:0/903798220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11481a21f0 con 0x7f1148101f00 2024-04-23T02:14:32.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.808+0000 7f114ec26700 1 -- 172.21.15.71:0/903798220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11481a26c0 con 0x7f1148101f00 2024-04-23T02:14:32.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.809+0000 7f114ec26700 1 -- 172.21.15.71:0/903798220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1148196b40 con 0x7f1148101f00 2024-04-23T02:14:32.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.810+0000 7f1145ffb700 1 -- 172.21.15.71:0/903798220 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 90946+0+0 (secure 0 0 0) 0x7f1134010380 con 0x7f1148101f00 2024-04-23T02:14:32.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.810+0000 7f1145ffb700 1 --2- 172.21.15.71:0/903798220 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f113806ce10 0x7f113806f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:14:32.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.810+0000 7f1145ffb700 1 -- 172.21.15.71:0/903798220 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f113408c100 con 0x7f1148101f00 2024-04-23T02:14:32.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.811+0000 7f114c9c2700 1 --2- 172.21.15.71:0/903798220 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f113806ce10 0x7f113806f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:14:32.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.811+0000 7f114c9c2700 1 --2- 172.21.15.71:0/903798220 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f113806ce10 0x7f113806f2d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f113c006010 tx=0x7f113c00b560 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:14:32.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:32.815+0000 7f1145ffb700 1 -- 172.21.15.71:0/903798220 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f1134059c50 con 0x7f1148101f00 2024-04-23T02:14:33.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:33.142+0000 7f114ec26700 1 -- 172.21.15.71:0/903798220 --> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11480611d0 con 0x7f113806ce10 2024-04-23T02:14:33.148 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:14:33.148 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:14:33.149 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:14:33.149 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-23T02:14:33.149 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:14:33.149 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr" 2024-04-23T02:14:33.149 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:14:33.149 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "2/2 daemons upgraded", 2024-04-23T02:14:33.149 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mgr daemons", 2024-04-23T02:14:33.149 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:14:33.149 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:14:33.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:33.144+0000 7f1145ffb700 1 -- 172.21.15.71:0/903798220 <== mgr.24351 v2:172.21.15.192:6828/966231696 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f11480611d0 con 0x7f113806ce10 2024-04-23T02:14:33.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:33.148+0000 7f114ec26700 1 -- 172.21.15.71:0/903798220 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f113806ce10 msgr2=0x7f113806f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:33.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:33.148+0000 7f114ec26700 1 --2- 172.21.15.71:0/903798220 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f113806ce10 0x7f113806f2d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f113c006010 tx=0x7f113c00b560 comp rx=0 tx=0).stop 2024-04-23T02:14:33.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:33.148+0000 7f114ec26700 1 -- 172.21.15.71:0/903798220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1148101f00 msgr2=0x7f114819cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:14:33.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:33.148+0000 7f114ec26700 1 --2- 172.21.15.71:0/903798220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1148101f00 0x7f114819cfe0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f113400b4b0 tx=0x7f113400b7c0 comp rx=0 tx=0).stop 2024-04-23T02:14:33.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:33.148+0000 7f114ec26700 1 -- 172.21.15.71:0/903798220 shutdown_connections 2024-04-23T02:14:33.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:33.148+0000 7f114ec26700 1 --2- 172.21.15.71:0/903798220 >> [v2:172.21.15.192:6828/966231696,v1:172.21.15.192:6829/966231696] conn(0x7f113806ce10 0x7f113806f2d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:33.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:33.148+0000 7f114ec26700 1 --2- 172.21.15.71:0/903798220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11480ff590 0x7f114819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:14:33.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:33.148+0000 7f114ec26700 1 --2- 172.21.15.71:0/903798220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1148101f00 0x7f114819cfe0 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f113400b4b0 tx=0x7f113400b7c0 comp rx=0 tx=0).stop 2024-04-23T02:14:33.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:33.148+0000 7f114ec26700 1 -- 172.21.15.71:0/903798220 >> 172.21.15.71:0/903798220 conn(0x7f11480fb190 msgr2=0x7f11480fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:14:33.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:33.148+0000 7f114ec26700 1 -- 172.21.15.71:0/903798220 shutdown_connections 2024-04-23T02:14:33.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:14:33.148+0000 7f114ec26700 1 -- 172.21.15.71:0/903798220 wait complete. 2024-04-23T02:14:33.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:32 smithi071 ceph-mon[39837]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:14:33.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:32 smithi071 ceph-mon[39837]: pgmap v43: 65 pgs: 65 active+clean; 1.3 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 237 op/s 2024-04-23T02:14:33.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:32 smithi071 ceph-mon[39837]: mgrmap e25: smithi192.qgecnv(active, since 80s), standbys: smithi071.ukdjvn 2024-04-23T02:14:33.246 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:32 smithi071 ceph-mon[39837]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:14:33.246 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:32 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1307117689' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:14:33.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:32 smithi192 ceph-mon[49662]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:14:33.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:32 smithi192 ceph-mon[49662]: pgmap v43: 65 pgs: 65 active+clean; 1.3 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 237 op/s 2024-04-23T02:14:33.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:32 smithi192 ceph-mon[49662]: mgrmap e25: smithi192.qgecnv(active, since 80s), standbys: smithi071.ukdjvn 2024-04-23T02:14:33.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:32 smithi192 ceph-mon[49662]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:14:33.379 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:32 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1307117689' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:14:34.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:33 smithi071 ceph-mon[39837]: from='client.14678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:14:34.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:33 smithi192 ceph-mon[49662]: from='client.14678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:14:34.949 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: pgmap v44: 65 pgs: 65 active+clean; 409 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 338 op/s 2024-04-23T02:14:34.949 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:34.949 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:34.949 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:14:34.949 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:14:34.949 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:34.949 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:14:34.949 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mgr fail", "who": "smithi192.qgecnv"}]: dispatch 2024-04-23T02:14:34.949 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mgr fail", "who": "smithi192.qgecnv"}]: dispatch 2024-04-23T02:14:34.949 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: osdmap e48: 6 total, 6 up, 6 in 2024-04-23T02:14:34.949 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd='[{"prefix": "mgr fail", "who": "smithi192.qgecnv"}]': finished 2024-04-23T02:14:34.949 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: mgrmap e26: smithi071.ukdjvn(active, starting, since 0.00798944s) 2024-04-23T02:14:34.950 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi071"}]: dispatch 2024-04-23T02:14:34.950 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:14:34.950 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.vxejye"}]: dispatch 2024-04-23T02:14:34.950 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.qusrrb"}]: dispatch 2024-04-23T02:14:34.950 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.gtmyst"}]: dispatch 2024-04-23T02:14:34.950 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.azdyze"}]: dispatch 2024-04-23T02:14:34.950 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr metadata", "who": "smithi071.ukdjvn", "id": "smithi071.ukdjvn"}]: dispatch 2024-04-23T02:14:34.950 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T02:14:34.950 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T02:14:34.950 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T02:14:34.950 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T02:14:34.950 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T02:14:34.950 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T02:14:34.951 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T02:14:34.951 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T02:14:34.951 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T02:14:34.951 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: Manager daemon smithi071.ukdjvn is now available 2024-04-23T02:14:34.951 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:14:34.951 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:14:34.951 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi071.ukdjvn/mirror_snapshot_schedule"}]: dispatch 2024-04-23T02:14:34.951 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi071.ukdjvn/mirror_snapshot_schedule"}]: dispatch 2024-04-23T02:14:34.951 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi071.ukdjvn/trash_purge_schedule"}]: dispatch 2024-04-23T02:14:34.951 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:34 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi071.ukdjvn/trash_purge_schedule"}]: dispatch 2024-04-23T02:14:35.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: pgmap v44: 65 pgs: 65 active+clean; 409 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 338 op/s 2024-04-23T02:14:35.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:35.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:35.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:14:35.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:14:35.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' 2024-04-23T02:14:35.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:14:35.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.24351 172.21.15.192:0/4128629096' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mgr fail", "who": "smithi192.qgecnv"}]: dispatch 2024-04-23T02:14:35.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "mgr fail", "who": "smithi192.qgecnv"}]: dispatch 2024-04-23T02:14:35.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: osdmap e48: 6 total, 6 up, 6 in 2024-04-23T02:14:35.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.24351 ' entity='mgr.smithi192.qgecnv' cmd='[{"prefix": "mgr fail", "who": "smithi192.qgecnv"}]': finished 2024-04-23T02:14:35.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: mgrmap e26: smithi071.ukdjvn(active, starting, since 0.00798944s) 2024-04-23T02:14:35.384 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi071"}]: dispatch 2024-04-23T02:14:35.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:14:35.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.vxejye"}]: dispatch 2024-04-23T02:14:35.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.qusrrb"}]: dispatch 2024-04-23T02:14:35.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.gtmyst"}]: dispatch 2024-04-23T02:14:35.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.azdyze"}]: dispatch 2024-04-23T02:14:35.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr metadata", "who": "smithi071.ukdjvn", "id": "smithi071.ukdjvn"}]: dispatch 2024-04-23T02:14:35.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T02:14:35.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T02:14:35.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T02:14:35.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T02:14:35.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T02:14:35.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T02:14:35.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T02:14:35.385 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T02:14:35.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T02:14:35.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: Manager daemon smithi071.ukdjvn is now available 2024-04-23T02:14:35.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:14:35.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:14:35.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi071.ukdjvn/mirror_snapshot_schedule"}]: dispatch 2024-04-23T02:14:35.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi071.ukdjvn/mirror_snapshot_schedule"}]: dispatch 2024-04-23T02:14:35.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi071.ukdjvn/trash_purge_schedule"}]: dispatch 2024-04-23T02:14:35.386 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:34 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi071.ukdjvn/trash_purge_schedule"}]: dispatch 2024-04-23T02:14:36.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:36 smithi192 ceph-mon[49662]: mgrmap e27: smithi071.ukdjvn(active, since 1.00727s) 2024-04-23T02:14:36.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:36 smithi192 ceph-mon[49662]: pgmap v3: 65 pgs: 65 active+clean; 296 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:14:36.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:36 smithi071 ceph-mon[39837]: mgrmap e27: smithi071.ukdjvn(active, since 1.00727s) 2024-04-23T02:14:36.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:36 smithi071 ceph-mon[39837]: pgmap v3: 65 pgs: 65 active+clean; 296 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:14:37.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:37 smithi071 ceph-mon[39837]: [23/Apr/2024:02:14:36] ENGINE Bus STARTING 2024-04-23T02:14:37.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:37 smithi071 ceph-mon[39837]: [23/Apr/2024:02:14:36] ENGINE Serving on http://172.21.15.71:8765 2024-04-23T02:14:37.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:37 smithi071 ceph-mon[39837]: [23/Apr/2024:02:14:36] ENGINE Serving on https://172.21.15.71:7150 2024-04-23T02:14:37.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:37 smithi071 ceph-mon[39837]: [23/Apr/2024:02:14:36] ENGINE Bus STARTED 2024-04-23T02:14:37.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:37 smithi071 ceph-mon[39837]: pgmap v4: 65 pgs: 65 active+clean; 296 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:14:37.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:37 smithi071 ceph-mon[39837]: mgrmap e28: smithi071.ukdjvn(active, since 2s) 2024-04-23T02:14:38.052 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:37 smithi192 ceph-mon[49662]: [23/Apr/2024:02:14:36] ENGINE Bus STARTING 2024-04-23T02:14:38.052 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:37 smithi192 ceph-mon[49662]: [23/Apr/2024:02:14:36] ENGINE Serving on http://172.21.15.71:8765 2024-04-23T02:14:38.052 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:37 smithi192 ceph-mon[49662]: [23/Apr/2024:02:14:36] ENGINE Serving on https://172.21.15.71:7150 2024-04-23T02:14:38.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:37 smithi192 ceph-mon[49662]: [23/Apr/2024:02:14:36] ENGINE Bus STARTED 2024-04-23T02:14:38.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:37 smithi192 ceph-mon[49662]: pgmap v4: 65 pgs: 65 active+clean; 296 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:14:38.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:37 smithi192 ceph-mon[49662]: mgrmap e28: smithi071.ukdjvn(active, since 2s) 2024-04-23T02:14:39.804 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:39 smithi192 ceph-mon[49662]: pgmap v5: 65 pgs: 65 active+clean; 296 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:14:39.804 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:39 smithi192 ceph-mon[49662]: mgrmap e29: smithi071.ukdjvn(active, since 4s) 2024-04-23T02:14:39.804 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:39 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:39.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:39 smithi071 ceph-mon[39837]: pgmap v5: 65 pgs: 65 active+clean; 296 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:14:39.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:39 smithi071 ceph-mon[39837]: mgrmap e29: smithi071.ukdjvn(active, since 4s) 2024-04-23T02:14:39.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:39 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:40.865 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:40 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:40.865 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:40 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:40.866 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:40 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:40.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:40 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:40.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:40 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:40.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:40 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:41.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:41 smithi192 ceph-mon[49662]: pgmap v6: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 173 op/s 2024-04-23T02:14:41.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:41 smithi071 ceph-mon[39837]: pgmap v6: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 173 op/s 2024-04-23T02:14:43.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:43 smithi192 ceph-mon[49662]: pgmap v7: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 123 op/s 2024-04-23T02:14:43.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:43 smithi192 ceph-mon[49662]: Standby manager daemon smithi192.qgecnv started 2024-04-23T02:14:43.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:43 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.192:0/1224851957' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.qgecnv/crt"}]: dispatch 2024-04-23T02:14:43.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:43 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.192:0/1224851957' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T02:14:43.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:43 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.192:0/1224851957' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.qgecnv/key"}]: dispatch 2024-04-23T02:14:43.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:43 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.192:0/1224851957' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T02:14:43.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:43 smithi071 ceph-mon[39837]: pgmap v7: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 123 op/s 2024-04-23T02:14:43.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:43 smithi071 ceph-mon[39837]: Standby manager daemon smithi192.qgecnv started 2024-04-23T02:14:43.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:43 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.192:0/1224851957' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.qgecnv/crt"}]: dispatch 2024-04-23T02:14:43.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:43 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.192:0/1224851957' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T02:14:43.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:43 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.192:0/1224851957' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.qgecnv/key"}]: dispatch 2024-04-23T02:14:43.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:43 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.192:0/1224851957' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T02:14:44.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:44 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr metadata", "who": "smithi192.qgecnv", "id": "smithi192.qgecnv"}]: dispatch 2024-04-23T02:14:44.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:44 smithi071 ceph-mon[39837]: mgrmap e30: smithi071.ukdjvn(active, since 9s), standbys: smithi192.qgecnv 2024-04-23T02:14:44.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:44 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:44.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:44 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:45.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:44 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr metadata", "who": "smithi192.qgecnv", "id": "smithi192.qgecnv"}]: dispatch 2024-04-23T02:14:45.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:44 smithi192 ceph-mon[49662]: mgrmap e30: smithi071.ukdjvn(active, since 9s), standbys: smithi192.qgecnv 2024-04-23T02:14:45.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:44 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:45.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:44 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:46.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:45 smithi192 ceph-mon[49662]: pgmap v8: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-04-23T02:14:46.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:45 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:46.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:45 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:46.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:45 smithi071 ceph-mon[39837]: pgmap v8: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-04-23T02:14:46.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:45 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:46.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:45 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:47.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:47 smithi071 ceph-mon[39837]: pgmap v9: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 154 op/s 2024-04-23T02:14:48.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:47 smithi192 ceph-mon[49662]: pgmap v9: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 154 op/s 2024-04-23T02:14:50.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:50 smithi071 ceph-mon[39837]: pgmap v10: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 141 op/s 2024-04-23T02:14:50.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:50 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:50.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:50 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:50.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:50 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:14:50.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:50 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:14:50.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:50 smithi071 ceph-mon[39837]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-23T02:14:50.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:50 smithi071 ceph-mon[39837]: Updating smithi192:/etc/ceph/ceph.conf 2024-04-23T02:14:50.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:50 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:14:50.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:50 smithi192 ceph-mon[49662]: pgmap v10: 65 pgs: 65 active+clean; 316 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 141 op/s 2024-04-23T02:14:50.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:50 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:50.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:50 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:50.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:50 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:14:50.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:50 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:14:50.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:50 smithi192 ceph-mon[49662]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-23T02:14:50.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:50 smithi192 ceph-mon[49662]: Updating smithi192:/etc/ceph/ceph.conf 2024-04-23T02:14:50.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:50 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:14:51.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:51 smithi071 ceph-mon[39837]: Updating smithi192:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T02:14:51.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:51 smithi071 ceph-mon[39837]: Updating smithi071:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T02:14:51.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:51 smithi071 ceph-mon[39837]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-04-23T02:14:51.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:51 smithi071 ceph-mon[39837]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-23T02:14:51.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:51 smithi071 ceph-mon[39837]: Updating smithi192:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T02:14:51.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:51 smithi071 ceph-mon[39837]: Updating smithi071:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T02:14:51.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:51 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:51.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:51 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:51.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:51 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:51.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:51 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:51.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:51 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:51.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:51 smithi192 ceph-mon[49662]: Updating smithi192:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T02:14:51.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:51 smithi192 ceph-mon[49662]: Updating smithi071:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T02:14:51.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:51 smithi192 ceph-mon[49662]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-04-23T02:14:51.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:51 smithi192 ceph-mon[49662]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-23T02:14:51.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:51 smithi192 ceph-mon[49662]: Updating smithi192:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T02:14:51.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:51 smithi192 ceph-mon[49662]: Updating smithi071:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T02:14:51.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:51 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:51.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:51 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:51.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:51 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:51.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:51 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:51.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:51 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:14:52.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:52 smithi071 ceph-mon[39837]: pgmap v11: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 221 op/s 2024-04-23T02:14:52.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:52 smithi071 ceph-mon[39837]: Reconfiguring prometheus.smithi071 (dependencies changed)... 2024-04-23T02:14:52.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:52 smithi071 ceph-mon[39837]: Reconfiguring daemon prometheus.smithi071 on smithi071 2024-04-23T02:14:52.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:52 smithi192 ceph-mon[49662]: pgmap v11: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 221 op/s 2024-04-23T02:14:52.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:52 smithi192 ceph-mon[49662]: Reconfiguring prometheus.smithi071 (dependencies changed)... 2024-04-23T02:14:52.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:52 smithi192 ceph-mon[49662]: Reconfiguring daemon prometheus.smithi071 on smithi071 2024-04-23T02:14:54.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:54 smithi192 ceph-mon[49662]: pgmap v12: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-04-23T02:14:54.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:54 smithi071 ceph-mon[39837]: pgmap v12: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-04-23T02:14:56.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:56 smithi071 ceph-mon[39837]: pgmap v13: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-04-23T02:14:56.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:56 smithi192 ceph-mon[49662]: pgmap v13: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-04-23T02:14:58.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:14:58 smithi192 ceph-mon[49662]: pgmap v14: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 211 op/s 2024-04-23T02:14:58.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:14:58 smithi071 ceph-mon[39837]: pgmap v14: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 211 op/s 2024-04-23T02:15:00.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:00 smithi192 ceph-mon[49662]: pgmap v15: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 141 op/s 2024-04-23T02:15:00.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:00 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:00.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:00 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:00.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:00 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-23T02:15:00.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:00 smithi192 ceph-mon[49662]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-23T02:15:00.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:00 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:15:00.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:00 smithi071 ceph-mon[39837]: pgmap v15: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 141 op/s 2024-04-23T02:15:00.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:00 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:00.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:00 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:00.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:00 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-23T02:15:00.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:00 smithi071 ceph-mon[39837]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-23T02:15:00.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:00 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:15:02.288 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:02 smithi192 ceph-mon[49662]: pgmap v16: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 212 op/s 2024-04-23T02:15:02.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:02 smithi071 ceph-mon[39837]: pgmap v16: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 212 op/s 2024-04-23T02:15:03.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.281+0000 7f7864691700 1 -- 172.21.15.71:0/1951125485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f785c100e10 msgr2=0x7f785c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:03.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.281+0000 7f7864691700 1 --2- 172.21.15.71:0/1951125485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f785c100e10 0x7f785c101270 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f784c009910 tx=0x7f784c009c20 comp rx=0 tx=0).stop 2024-04-23T02:15:03.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.281+0000 7f7864691700 1 -- 172.21.15.71:0/1951125485 shutdown_connections 2024-04-23T02:15:03.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.281+0000 7f7864691700 1 --2- 172.21.15.71:0/1951125485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f785c100e10 0x7f785c101270 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:03.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.281+0000 7f7864691700 1 --2- 172.21.15.71:0/1951125485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f785c0ffc10 0x7f785c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:03.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.281+0000 7f7864691700 1 -- 172.21.15.71:0/1951125485 >> 172.21.15.71:0/1951125485 conn(0x7f785c0fb190 msgr2=0x7f785c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:03.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.281+0000 7f7864691700 1 -- 172.21.15.71:0/1951125485 shutdown_connections 2024-04-23T02:15:03.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.281+0000 7f7864691700 1 -- 172.21.15.71:0/1951125485 wait complete. 2024-04-23T02:15:03.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.282+0000 7f7864691700 1 Processor -- start 2024-04-23T02:15:03.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.283+0000 7f7864691700 1 -- start start 2024-04-23T02:15:03.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.283+0000 7f7864691700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f785c0ffc10 0x7f785c1963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:03.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.283+0000 7f7864691700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f785c100e10 0x7f785c196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:03.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.283+0000 7f7864691700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c196f20 con 0x7f785c0ffc10 2024-04-23T02:15:03.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.283+0000 7f7864691700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c197060 con 0x7f785c100e10 2024-04-23T02:15:03.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.283+0000 7f786242d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f785c0ffc10 0x7f785c1963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:03.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.283+0000 7f786242d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f785c0ffc10 0x7f785c1963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58044/0 (socket says 172.21.15.71:58044) 2024-04-23T02:15:03.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.283+0000 7f786242d700 1 -- 172.21.15.71:0/3011385481 learned_addr learned my addr 172.21.15.71:0/3011385481 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:15:03.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.283+0000 7f7861c2c700 1 --2- 172.21.15.71:0/3011385481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f785c100e10 0x7f785c196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:03.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.284+0000 7f786242d700 1 -- 172.21.15.71:0/3011385481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f785c100e10 msgr2=0x7f785c196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:03.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.284+0000 7f786242d700 1 --2- 172.21.15.71:0/3011385481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f785c100e10 0x7f785c196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:03.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.284+0000 7f786242d700 1 -- 172.21.15.71:0/3011385481 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784c0095f0 con 0x7f785c0ffc10 2024-04-23T02:15:03.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.284+0000 7f786242d700 1 --2- 172.21.15.71:0/3011385481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f785c0ffc10 0x7f785c1963c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f785800e8f0 tx=0x7f785800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:03.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.284+0000 7f78537fe700 1 -- 172.21.15.71:0/3011385481 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7858009790 con 0x7f785c0ffc10 2024-04-23T02:15:03.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.284+0000 7f78537fe700 1 -- 172.21.15.71:0/3011385481 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f785800c810 con 0x7f785c0ffc10 2024-04-23T02:15:03.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.284+0000 7f7864691700 1 -- 172.21.15.71:0/3011385481 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785c19bb10 con 0x7f785c0ffc10 2024-04-23T02:15:03.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.284+0000 7f78537fe700 1 -- 172.21.15.71:0/3011385481 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7858003c20 con 0x7f785c0ffc10 2024-04-23T02:15:03.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.284+0000 7f7864691700 1 -- 172.21.15.71:0/3011385481 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785c19c060 con 0x7f785c0ffc10 2024-04-23T02:15:03.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.285+0000 7f7864691700 1 -- 172.21.15.71:0/3011385481 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f785c1096c0 con 0x7f785c0ffc10 2024-04-23T02:15:03.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.286+0000 7f78537fe700 1 -- 172.21.15.71:0/3011385481 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f785800c980 con 0x7f785c0ffc10 2024-04-23T02:15:03.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.286+0000 7f78537fe700 1 --2- 172.21.15.71:0/3011385481 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f784806ce20 0x7f784806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:03.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.286+0000 7f78537fe700 1 -- 172.21.15.71:0/3011385481 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7858014070 con 0x7f785c0ffc10 2024-04-23T02:15:03.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.286+0000 7f7861c2c700 1 --2- 172.21.15.71:0/3011385481 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f784806ce20 0x7f784806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:03.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.287+0000 7f7861c2c700 1 --2- 172.21.15.71:0/3011385481 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f784806ce20 0x7f784806f2e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f784c009fd0 tx=0x7f784c00b5a0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:03.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.291+0000 7f78537fe700 1 -- 172.21.15.71:0/3011385481 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f785805a3d0 con 0x7f785c0ffc10 2024-04-23T02:15:03.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.609+0000 7f7864691700 1 -- 172.21.15.71:0/3011385481 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f785c105760 con 0x7f784806ce20 2024-04-23T02:15:03.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.611+0000 7f78537fe700 1 -- 172.21.15.71:0/3011385481 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f785c105760 con 0x7f784806ce20 2024-04-23T02:15:03.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.615+0000 7f7864691700 1 -- 172.21.15.71:0/3011385481 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f784806ce20 msgr2=0x7f784806f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:03.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.615+0000 7f7864691700 1 --2- 172.21.15.71:0/3011385481 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f784806ce20 0x7f784806f2e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f784c009fd0 tx=0x7f784c00b5a0 comp rx=0 tx=0).stop 2024-04-23T02:15:03.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.615+0000 7f7864691700 1 -- 172.21.15.71:0/3011385481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f785c0ffc10 msgr2=0x7f785c1963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:03.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.615+0000 7f7864691700 1 --2- 172.21.15.71:0/3011385481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f785c0ffc10 0x7f785c1963c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f785800e8f0 tx=0x7f785800ec00 comp rx=0 tx=0).stop 2024-04-23T02:15:03.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.615+0000 7f7864691700 1 -- 172.21.15.71:0/3011385481 shutdown_connections 2024-04-23T02:15:03.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.615+0000 7f7864691700 1 --2- 172.21.15.71:0/3011385481 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f784806ce20 0x7f784806f2e0 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f784c009fd0 tx=0x7f784c00b5a0 comp rx=0 tx=0).stop 2024-04-23T02:15:03.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.615+0000 7f7864691700 1 --2- 172.21.15.71:0/3011385481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f785c0ffc10 0x7f785c1963c0 secure :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f785800e8f0 tx=0x7f785800ec00 comp rx=0 tx=0).stop 2024-04-23T02:15:03.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.615+0000 7f7864691700 1 --2- 172.21.15.71:0/3011385481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f785c100e10 0x7f785c196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:03.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.615+0000 7f7864691700 1 -- 172.21.15.71:0/3011385481 >> 172.21.15.71:0/3011385481 conn(0x7f785c0fb190 msgr2=0x7f785c104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:03.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.618+0000 7f7864691700 1 -- 172.21.15.71:0/3011385481 shutdown_connections 2024-04-23T02:15:03.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.619+0000 7f7864691700 1 -- 172.21.15.71:0/3011385481 wait complete. 2024-04-23T02:15:03.642 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:15:03.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.786+0000 7fea8680b700 1 -- 172.21.15.71:0/863892188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea80100da0 msgr2=0x7fea80101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:03.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.786+0000 7fea8680b700 1 --2- 172.21.15.71:0/863892188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea80100da0 0x7fea80101200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fea74009910 tx=0x7fea74009c20 comp rx=0 tx=0).stop 2024-04-23T02:15:03.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.789+0000 7fea8680b700 1 -- 172.21.15.71:0/863892188 shutdown_connections 2024-04-23T02:15:03.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.789+0000 7fea8680b700 1 --2- 172.21.15.71:0/863892188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea80100da0 0x7fea80101200 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:03.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.789+0000 7fea8680b700 1 --2- 172.21.15.71:0/863892188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea800ffba0 0x7fea800fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:03.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.789+0000 7fea8680b700 1 -- 172.21.15.71:0/863892188 >> 172.21.15.71:0/863892188 conn(0x7fea800fb120 msgr2=0x7fea800fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:03.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.789+0000 7fea8680b700 1 -- 172.21.15.71:0/863892188 shutdown_connections 2024-04-23T02:15:03.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.789+0000 7fea8680b700 1 -- 172.21.15.71:0/863892188 wait complete. 2024-04-23T02:15:03.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.790+0000 7fea8680b700 1 Processor -- start 2024-04-23T02:15:03.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.790+0000 7fea8680b700 1 -- start start 2024-04-23T02:15:03.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.790+0000 7fea8680b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea800ffba0 0x7fea80198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:03.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.790+0000 7fea8680b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea80100da0 0x7fea80198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:03.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.790+0000 7fea8680b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea80198070 con 0x7fea80100da0 2024-04-23T02:15:03.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.790+0000 7fea8680b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea801990f0 con 0x7fea800ffba0 2024-04-23T02:15:03.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.790+0000 7fea7f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea80100da0 0x7fea80198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:03.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.791+0000 7fea7f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea80100da0 0x7fea80198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58064/0 (socket says 172.21.15.71:58064) 2024-04-23T02:15:03.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.791+0000 7fea7f7fe700 1 -- 172.21.15.71:0/499597101 learned_addr learned my addr 172.21.15.71:0/499597101 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:15:03.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.791+0000 7fea7ffff700 1 --2- 172.21.15.71:0/499597101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea800ffba0 0x7fea80198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:03.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.791+0000 7fea7f7fe700 1 -- 172.21.15.71:0/499597101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea800ffba0 msgr2=0x7fea80198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:03.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.791+0000 7fea7f7fe700 1 --2- 172.21.15.71:0/499597101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea800ffba0 0x7fea80198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:03.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.791+0000 7fea7f7fe700 1 -- 172.21.15.71:0/499597101 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea740095f0 con 0x7fea80100da0 2024-04-23T02:15:03.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.791+0000 7fea7f7fe700 1 --2- 172.21.15.71:0/499597101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea80100da0 0x7fea80198bb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fea74000c00 tx=0x7fea74004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:03.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.792+0000 7fea7d7fa700 1 -- 172.21.15.71:0/499597101 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea74004d80 con 0x7fea80100da0 2024-04-23T02:15:03.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.792+0000 7fea7d7fa700 1 -- 172.21.15.71:0/499597101 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fea74004ee0 con 0x7fea80100da0 2024-04-23T02:15:03.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.792+0000 7fea7d7fa700 1 -- 172.21.15.71:0/499597101 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea7401d620 con 0x7fea80100da0 2024-04-23T02:15:03.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.792+0000 7fea8680b700 1 -- 172.21.15.71:0/499597101 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea801a2330 con 0x7fea80100da0 2024-04-23T02:15:03.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.792+0000 7fea8680b700 1 -- 172.21.15.71:0/499597101 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea801a26e0 con 0x7fea80100da0 2024-04-23T02:15:03.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.793+0000 7fea8680b700 1 -- 172.21.15.71:0/499597101 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea80107550 con 0x7fea80100da0 2024-04-23T02:15:03.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.796+0000 7fea7d7fa700 1 -- 172.21.15.71:0/499597101 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) v1 ==== 90944+0+0 (secure 0 0 0) 0x7fea740154b0 con 0x7fea80100da0 2024-04-23T02:15:03.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.796+0000 7fea7d7fa700 1 --2- 172.21.15.71:0/499597101 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fea6806cf70 0x7fea6806f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:03.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.796+0000 7fea7d7fa700 1 -- 172.21.15.71:0/499597101 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fea7408e700 con 0x7fea80100da0 2024-04-23T02:15:03.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.797+0000 7fea7ffff700 1 --2- 172.21.15.71:0/499597101 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fea6806cf70 0x7fea6806f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:03.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.797+0000 7fea7ffff700 1 --2- 172.21.15.71:0/499597101 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fea6806cf70 0x7fea6806f430 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fea70006fd0 tx=0x7fea70008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:03.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:03.804+0000 7fea7d7fa700 1 -- 172.21.15.71:0/499597101 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fea7405c0f0 con 0x7fea80100da0 2024-04-23T02:15:04.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.099+0000 7fea8680b700 1 -- 172.21.15.71:0/499597101 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fea801a2db0 con 0x7fea6806cf70 2024-04-23T02:15:04.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.101+0000 7fea7d7fa700 1 -- 172.21.15.71:0/499597101 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fea801a2db0 con 0x7fea6806cf70 2024-04-23T02:15:04.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.104+0000 7fea8680b700 1 -- 172.21.15.71:0/499597101 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fea6806cf70 msgr2=0x7fea6806f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:04.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.104+0000 7fea8680b700 1 --2- 172.21.15.71:0/499597101 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fea6806cf70 0x7fea6806f430 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fea70006fd0 tx=0x7fea70008040 comp rx=0 tx=0).stop 2024-04-23T02:15:04.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.104+0000 7fea8680b700 1 -- 172.21.15.71:0/499597101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea80100da0 msgr2=0x7fea80198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:04.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.104+0000 7fea8680b700 1 --2- 172.21.15.71:0/499597101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea80100da0 0x7fea80198bb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fea74000c00 tx=0x7fea74004690 comp rx=0 tx=0).stop 2024-04-23T02:15:04.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.104+0000 7fea8680b700 1 -- 172.21.15.71:0/499597101 shutdown_connections 2024-04-23T02:15:04.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.104+0000 7fea8680b700 1 --2- 172.21.15.71:0/499597101 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fea6806cf70 0x7fea6806f430 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.104+0000 7fea8680b700 1 --2- 172.21.15.71:0/499597101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea800ffba0 0x7fea80198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.104+0000 7fea8680b700 1 --2- 172.21.15.71:0/499597101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea80100da0 0x7fea80198bb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.104+0000 7fea8680b700 1 -- 172.21.15.71:0/499597101 >> 172.21.15.71:0/499597101 conn(0x7fea800fb120 msgr2=0x7fea80103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.104+0000 7fea8680b700 1 -- 172.21.15.71:0/499597101 shutdown_connections 2024-04-23T02:15:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.104+0000 7fea8680b700 1 -- 172.21.15.71:0/499597101 wait complete. 2024-04-23T02:15:04.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.228+0000 7f477b17c700 1 -- 172.21.15.71:0/1791748409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4774101740 msgr2=0x7f4774103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:04.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.228+0000 7f477b17c700 1 --2- 172.21.15.71:0/1791748409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4774101740 0x7f4774103b30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4768009910 tx=0x7f4768009c20 comp rx=0 tx=0).stop 2024-04-23T02:15:04.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.228+0000 7f477b17c700 1 -- 172.21.15.71:0/1791748409 shutdown_connections 2024-04-23T02:15:04.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.228+0000 7f477b17c700 1 --2- 172.21.15.71:0/1791748409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4774104070 0x7f4774106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:04.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.228+0000 7f477b17c700 1 --2- 172.21.15.71:0/1791748409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4774101740 0x7f4774103b30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:04.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.228+0000 7f477b17c700 1 -- 172.21.15.71:0/1791748409 >> 172.21.15.71:0/1791748409 conn(0x7f47740fb150 msgr2=0x7f47740fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:04.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.228+0000 7f477b17c700 1 -- 172.21.15.71:0/1791748409 shutdown_connections 2024-04-23T02:15:04.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.228+0000 7f477b17c700 1 -- 172.21.15.71:0/1791748409 wait complete. 2024-04-23T02:15:04.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.229+0000 7f477b17c700 1 Processor -- start 2024-04-23T02:15:04.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.229+0000 7f477b17c700 1 -- start start 2024-04-23T02:15:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.229+0000 7f477b17c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4774101740 0x7f47741963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.229+0000 7f477b17c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4774104070 0x7f47741968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.229+0000 7f477b17c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4774196f00 con 0x7f4774101740 2024-04-23T02:15:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.229+0000 7f477b17c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4774197040 con 0x7f4774104070 2024-04-23T02:15:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.229+0000 7f4778f18700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4774101740 0x7f47741963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.229+0000 7f4778f18700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4774101740 0x7f47741963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58078/0 (socket says 172.21.15.71:58078) 2024-04-23T02:15:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.229+0000 7f4778f18700 1 -- 172.21.15.71:0/160213868 learned_addr learned my addr 172.21.15.71:0/160213868 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:15:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.229+0000 7f4773fff700 1 --2- 172.21.15.71:0/160213868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4774104070 0x7f47741968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.229+0000 7f4778f18700 1 -- 172.21.15.71:0/160213868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4774104070 msgr2=0x7f47741968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.229+0000 7f4778f18700 1 --2- 172.21.15.71:0/160213868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4774104070 0x7f47741968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.229+0000 7f4778f18700 1 -- 172.21.15.71:0/160213868 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47680095f0 con 0x7f4774101740 2024-04-23T02:15:04.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.229+0000 7f4773fff700 1 --2- 172.21.15.71:0/160213868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4774104070 0x7f47741968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T02:15:04.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.229+0000 7f4778f18700 1 --2- 172.21.15.71:0/160213868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4774101740 0x7f47741963a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f476800bde0 tx=0x7f476800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:04.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.230+0000 7f4771ffb700 1 -- 172.21.15.71:0/160213868 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f476801d070 con 0x7f4774101740 2024-04-23T02:15:04.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.230+0000 7f477b17c700 1 -- 172.21.15.71:0/160213868 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477419ba90 con 0x7f4774101740 2024-04-23T02:15:04.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.230+0000 7f4771ffb700 1 -- 172.21.15.71:0/160213868 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4768005dc0 con 0x7f4774101740 2024-04-23T02:15:04.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.230+0000 7f4771ffb700 1 -- 172.21.15.71:0/160213868 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4768003b10 con 0x7f4774101740 2024-04-23T02:15:04.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.230+0000 7f477b17c700 1 -- 172.21.15.71:0/160213868 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477419bf20 con 0x7f4774101740 2024-04-23T02:15:04.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.231+0000 7f477b17c700 1 -- 172.21.15.71:0/160213868 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47741904e0 con 0x7f4774101740 2024-04-23T02:15:04.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.232+0000 7f4771ffb700 1 -- 172.21.15.71:0/160213868 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f4768003c70 con 0x7f4774101740 2024-04-23T02:15:04.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.232+0000 7f4771ffb700 1 --2- 172.21.15.71:0/160213868 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f476406ce20 0x7f476406f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:04.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.232+0000 7f4771ffb700 1 -- 172.21.15.71:0/160213868 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f476808de40 con 0x7f4774101740 2024-04-23T02:15:04.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.232+0000 7f4773fff700 1 --2- 172.21.15.71:0/160213868 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f476406ce20 0x7f476406f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:04.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.233+0000 7f4773fff700 1 --2- 172.21.15.71:0/160213868 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f476406ce20 0x7f476406f2e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4760011490 tx=0x7f4760009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:04.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.239+0000 7f4771ffb700 1 -- 172.21.15.71:0/160213868 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f476805b850 con 0x7f4774101740 2024-04-23T02:15:04.353 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:04 smithi192 ceph-mon[49662]: pgmap v17: 65 pgs: 65 active+clean; 320 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 157 op/s 2024-04-23T02:15:04.354 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:04 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.qgecnv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T02:15:04.354 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:04 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.qgecnv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T02:15:04.354 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:04 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T02:15:04.354 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:04 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:15:04.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:04 smithi071 ceph-mon[39837]: pgmap v17: 65 pgs: 65 active+clean; 320 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 157 op/s 2024-04-23T02:15:04.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:04 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.qgecnv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T02:15:04.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:04 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi192.qgecnv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T02:15:04.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:04 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T02:15:04.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:04 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:15:04.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.505+0000 7f477b17c700 1 -- 172.21.15.71:0/160213868 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f47740611d0 con 0x7f476406ce20 2024-04-23T02:15:04.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.523+0000 7f4771ffb700 1 -- 172.21.15.71:0/160213868 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f47740611d0 con 0x7f476406ce20 2024-04-23T02:15:04.530 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:15:04.530 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (8m) 20s ago 10m 26.4M - 0.25.0 c8568f914cd2 8261e17319fe 2024-04-23T02:15:04.530 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (10m) 20s ago 10m 7818k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:15:04.531 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (9m) 24s ago 9m 7726k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:15:04.531 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (10m) 20s ago 10m 6656k - 18.2.1 5be31c24972a 258512dcbc00 2024-04-23T02:15:04.531 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (9m) 24s ago 9m 6656k - 18.2.1 5be31c24972a feea2c9896a6 2024-04-23T02:15:04.531 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (8m) 20s ago 9m 88.0M - 9.4.7 954c08fa6188 48ea3e02b806 2024-04-23T02:15:04.531 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (4m) 20s ago 4m 160M - 18.2.1 5be31c24972a a515de2a841c 2024-04-23T02:15:04.531 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (4m) 20s ago 4m 177M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:15:04.531 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (4m) 24s ago 4m 146M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:15:04.531 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (4m) 24s ago 4m 167M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:15:04.531 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (42s) 20s ago 11m 469M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:15:04.531 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (2m) 24s ago 9m 62.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 151cb2226032 2024-04-23T02:15:04.531 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (11m) 20s ago 11m 58.1M 2048M 18.2.1 5be31c24972a ff40abe63fd9 2024-04-23T02:15:04.531 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (8m) 24s ago 8m 43.1M 2048M 18.2.1 5be31c24972a 9c03e90acc95 2024-04-23T02:15:04.531 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (10m) 20s ago 10m 19.2M - 1.5.0 0da6a335fe13 3f4d0aaee04e 2024-04-23T02:15:04.532 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (9m) 24s ago 9m 16.9M - 1.5.0 0da6a335fe13 cfd0b49d770c 2024-04-23T02:15:04.532 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (8m) 20s ago 8m 169M 1227M 18.2.1 5be31c24972a d3d5f2fa7267 2024-04-23T02:15:04.532 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (7m) 20s ago 7m 178M 1227M 18.2.1 5be31c24972a 3992157ff0fd 2024-04-23T02:15:04.532 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (7m) 20s ago 7m 137M 1227M 18.2.1 5be31c24972a 12380b6a15a2 2024-04-23T02:15:04.532 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (6m) 24s ago 6m 201M 2251M 18.2.1 5be31c24972a d5f587ee3a22 2024-04-23T02:15:04.532 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (6m) 24s ago 6m 183M 2251M 18.2.1 5be31c24972a 958d046005f0 2024-04-23T02:15:04.532 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (5m) 24s ago 5m 170M 2251M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:15:04.532 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 starting - - - - 2024-04-23T02:15:04.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.527+0000 7f477b17c700 1 -- 172.21.15.71:0/160213868 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f476406ce20 msgr2=0x7f476406f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:04.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.527+0000 7f477b17c700 1 --2- 172.21.15.71:0/160213868 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f476406ce20 0x7f476406f2e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4760011490 tx=0x7f4760009260 comp rx=0 tx=0).stop 2024-04-23T02:15:04.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.527+0000 7f477b17c700 1 -- 172.21.15.71:0/160213868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4774101740 msgr2=0x7f47741963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:04.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.527+0000 7f477b17c700 1 --2- 172.21.15.71:0/160213868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4774101740 0x7f47741963a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f476800bde0 tx=0x7f476800be10 comp rx=0 tx=0).stop 2024-04-23T02:15:04.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.527+0000 7f477b17c700 1 -- 172.21.15.71:0/160213868 shutdown_connections 2024-04-23T02:15:04.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.527+0000 7f477b17c700 1 --2- 172.21.15.71:0/160213868 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f476406ce20 0x7f476406f2e0 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4760011490 tx=0x7f4760009260 comp rx=0 tx=0).stop 2024-04-23T02:15:04.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.527+0000 7f477b17c700 1 --2- 172.21.15.71:0/160213868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4774101740 0x7f47741963a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:04.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.527+0000 7f477b17c700 1 --2- 172.21.15.71:0/160213868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4774104070 0x7f47741968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:04.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.527+0000 7f477b17c700 1 -- 172.21.15.71:0/160213868 >> 172.21.15.71:0/160213868 conn(0x7f47740fb150 msgr2=0x7f47740fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:04.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.527+0000 7f477b17c700 1 -- 172.21.15.71:0/160213868 shutdown_connections 2024-04-23T02:15:04.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.528+0000 7f477b17c700 1 -- 172.21.15.71:0/160213868 wait complete. 2024-04-23T02:15:04.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.695+0000 7f98b6bb0700 1 -- 172.21.15.71:0/2217334625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98b00ffe10 msgr2=0x7f98b0100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:04.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.695+0000 7f98b6bb0700 1 --2- 172.21.15.71:0/2217334625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98b00ffe10 0x7f98b0100290 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f989c00fa70 tx=0x7f989c00fd80 comp rx=0 tx=0).stop 2024-04-23T02:15:04.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.696+0000 7f98b6bb0700 1 -- 172.21.15.71:0/2217334625 shutdown_connections 2024-04-23T02:15:04.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.696+0000 7f98b6bb0700 1 --2- 172.21.15.71:0/2217334625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98b00ffe10 0x7f98b0100290 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:04.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.696+0000 7f98b6bb0700 1 --2- 172.21.15.71:0/2217334625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98b01011b0 0x7f98b01015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:04.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.696+0000 7f98b6bb0700 1 -- 172.21.15.71:0/2217334625 >> 172.21.15.71:0/2217334625 conn(0x7f98b00fa8c0 msgr2=0x7f98b00fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:04.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.696+0000 7f98b6bb0700 1 -- 172.21.15.71:0/2217334625 shutdown_connections 2024-04-23T02:15:04.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.696+0000 7f98b6bb0700 1 -- 172.21.15.71:0/2217334625 wait complete. 2024-04-23T02:15:04.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.697+0000 7f98b6bb0700 1 Processor -- start 2024-04-23T02:15:04.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.697+0000 7f98b6bb0700 1 -- start start 2024-04-23T02:15:04.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.697+0000 7f98b6bb0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98b00ffe10 0x7f98b019ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:04.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.697+0000 7f98b6bb0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98b01011b0 0x7f98b019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:04.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.697+0000 7f98b6bb0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b019d570 con 0x7f98b00ffe10 2024-04-23T02:15:04.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.697+0000 7f98b6bb0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b019d6b0 con 0x7f98b01011b0 2024-04-23T02:15:04.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.698+0000 7f98b494c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98b00ffe10 0x7f98b019ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:04.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.698+0000 7f98affff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98b01011b0 0x7f98b019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:04.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.698+0000 7f98affff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98b01011b0 0x7f98b019cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.71:40346/0 (socket says 172.21.15.71:40346) 2024-04-23T02:15:04.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.698+0000 7f98affff700 1 -- 172.21.15.71:0/389285006 learned_addr learned my addr 172.21.15.71:0/389285006 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:15:04.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.698+0000 7f98b494c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98b00ffe10 0x7f98b019ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58096/0 (socket says 172.21.15.71:58096) 2024-04-23T02:15:04.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.698+0000 7f98b494c700 1 -- 172.21.15.71:0/389285006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98b01011b0 msgr2=0x7f98b019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:04.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.698+0000 7f98b494c700 1 --2- 172.21.15.71:0/389285006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98b01011b0 0x7f98b019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:04.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.698+0000 7f98b494c700 1 -- 172.21.15.71:0/389285006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c00f750 con 0x7f98b00ffe10 2024-04-23T02:15:04.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.698+0000 7f98b494c700 1 --2- 172.21.15.71:0/389285006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98b00ffe10 0x7f98b019ca10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f98a400bfd0 tx=0x7f98a4007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:04.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.698+0000 7f98adffb700 1 -- 172.21.15.71:0/389285006 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98a4010040 con 0x7f98b00ffe10 2024-04-23T02:15:04.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.698+0000 7f98b6bb0700 1 -- 172.21.15.71:0/389285006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b01a2160 con 0x7f98b00ffe10 2024-04-23T02:15:04.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.699+0000 7f98b6bb0700 1 -- 172.21.15.71:0/389285006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b01a26b0 con 0x7f98b00ffe10 2024-04-23T02:15:04.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.699+0000 7f98adffb700 1 -- 172.21.15.71:0/389285006 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f98a40044b0 con 0x7f98b00ffe10 2024-04-23T02:15:04.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.699+0000 7f98adffb700 1 -- 172.21.15.71:0/389285006 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98a40158b0 con 0x7f98b00ffe10 2024-04-23T02:15:04.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.700+0000 7f98b6bb0700 1 -- 172.21.15.71:0/389285006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9894005320 con 0x7f98b00ffe10 2024-04-23T02:15:04.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.701+0000 7f98adffb700 1 -- 172.21.15.71:0/389285006 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f98a4003680 con 0x7f98b00ffe10 2024-04-23T02:15:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.702+0000 7f98adffb700 1 --2- 172.21.15.71:0/389285006 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98a006ce20 0x7f98a006f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.702+0000 7f98adffb700 1 -- 172.21.15.71:0/389285006 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f98a408cd60 con 0x7f98b00ffe10 2024-04-23T02:15:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.702+0000 7f98affff700 1 --2- 172.21.15.71:0/389285006 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98a006ce20 0x7f98a006f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.703+0000 7f98affff700 1 --2- 172.21.15.71:0/389285006 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98a006ce20 0x7f98a006f2e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f989c009870 tx=0x7f989c005ab0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:04.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:04.709+0000 7f98adffb700 1 -- 172.21.15.71:0/389285006 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f98a405a750 con 0x7f98b00ffe10 2024-04-23T02:15:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.080+0000 7f98b6bb0700 1 -- 172.21.15.71:0/389285006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9894005cc0 con 0x7f98b00ffe10 2024-04-23T02:15:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.080+0000 7f98adffb700 1 -- 172.21.15.71:0/389285006 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f98a405a2e0 con 0x7f98b00ffe10 2024-04-23T02:15:05.085 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:15:05.085 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:15:05.085 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-23T02:15:05.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:15:05.085 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:15:05.085 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:15:05.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:15:05.085 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:15:05.085 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-23T02:15:05.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:15:05.085 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:15:05.085 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-23T02:15:05.086 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:15:05.086 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:15:05.086 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-23T02:15:05.086 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:15:05.086 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:15:05.086 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:15:05.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.083+0000 7f98b6bb0700 1 -- 172.21.15.71:0/389285006 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98a006ce20 msgr2=0x7f98a006f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:05.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.083+0000 7f98b6bb0700 1 --2- 172.21.15.71:0/389285006 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98a006ce20 0x7f98a006f2e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f989c009870 tx=0x7f989c005ab0 comp rx=0 tx=0).stop 2024-04-23T02:15:05.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.083+0000 7f98b6bb0700 1 -- 172.21.15.71:0/389285006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98b00ffe10 msgr2=0x7f98b019ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:05.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.083+0000 7f98b6bb0700 1 --2- 172.21.15.71:0/389285006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98b00ffe10 0x7f98b019ca10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f98a400bfd0 tx=0x7f98a4007bd0 comp rx=0 tx=0).stop 2024-04-23T02:15:05.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.083+0000 7f98b6bb0700 1 -- 172.21.15.71:0/389285006 shutdown_connections 2024-04-23T02:15:05.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.083+0000 7f98b6bb0700 1 --2- 172.21.15.71:0/389285006 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98a006ce20 0x7f98a006f2e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:05.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.083+0000 7f98b6bb0700 1 --2- 172.21.15.71:0/389285006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98b00ffe10 0x7f98b019ca10 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:05.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.083+0000 7f98b6bb0700 1 --2- 172.21.15.71:0/389285006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98b01011b0 0x7f98b019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:05.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.083+0000 7f98b6bb0700 1 -- 172.21.15.71:0/389285006 >> 172.21.15.71:0/389285006 conn(0x7f98b00fa8c0 msgr2=0x7f98b0104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:05.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.083+0000 7f98b6bb0700 1 -- 172.21.15.71:0/389285006 shutdown_connections 2024-04-23T02:15:05.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.083+0000 7f98b6bb0700 1 -- 172.21.15.71:0/389285006 wait complete. 2024-04-23T02:15:05.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:05 smithi192 ceph-mon[49662]: Upgrade: Updating mgr.smithi192.qgecnv 2024-04-23T02:15:05.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:05 smithi192 ceph-mon[49662]: Deploying daemon mgr.smithi192.qgecnv on smithi192 2024-04-23T02:15:05.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:05 smithi192 ceph-mon[49662]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:15:05.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:05 smithi192 ceph-mon[49662]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:15:05.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:05 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:15:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.206+0000 7f4d7b4ae700 1 -- 172.21.15.71:0/2818044134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d740ffba0 msgr2=0x7f4d740fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.206+0000 7f4d7b4ae700 1 --2- 172.21.15.71:0/2818044134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d740ffba0 0x7f4d740fffc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009910 tx=0x7f4d70009c20 comp rx=0 tx=0).stop 2024-04-23T02:15:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.206+0000 7f4d7b4ae700 1 -- 172.21.15.71:0/2818044134 shutdown_connections 2024-04-23T02:15:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.206+0000 7f4d7b4ae700 1 --2- 172.21.15.71:0/2818044134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d74100da0 0x7f4d74101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.206+0000 7f4d7b4ae700 1 --2- 172.21.15.71:0/2818044134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d740ffba0 0x7f4d740fffc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.206+0000 7f4d7b4ae700 1 -- 172.21.15.71:0/2818044134 >> 172.21.15.71:0/2818044134 conn(0x7f4d740fb120 msgr2=0x7f4d740fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.206+0000 7f4d7b4ae700 1 -- 172.21.15.71:0/2818044134 shutdown_connections 2024-04-23T02:15:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.206+0000 7f4d7b4ae700 1 -- 172.21.15.71:0/2818044134 wait complete. 2024-04-23T02:15:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.207+0000 7f4d7b4ae700 1 Processor -- start 2024-04-23T02:15:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.207+0000 7f4d7b4ae700 1 -- start start 2024-04-23T02:15:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.207+0000 7f4d7b4ae700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d740ffba0 0x7f4d74071e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.207+0000 7f4d7b4ae700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d74100da0 0x7f4d74072360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.207+0000 7f4d7b4ae700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740729a0 con 0x7f4d740ffba0 2024-04-23T02:15:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.207+0000 7f4d7b4ae700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d74191c50 con 0x7f4d74100da0 2024-04-23T02:15:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.207+0000 7f4d7924a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d740ffba0 0x7f4d74071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.207+0000 7f4d7924a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d740ffba0 0x7f4d74071e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58114/0 (socket says 172.21.15.71:58114) 2024-04-23T02:15:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.207+0000 7f4d7924a700 1 -- 172.21.15.71:0/115630562 learned_addr learned my addr 172.21.15.71:0/115630562 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:15:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.207+0000 7f4d78a49700 1 --2- 172.21.15.71:0/115630562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d74100da0 0x7f4d74072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.207+0000 7f4d7924a700 1 -- 172.21.15.71:0/115630562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d74100da0 msgr2=0x7f4d74072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.208+0000 7f4d7924a700 1 --2- 172.21.15.71:0/115630562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d74100da0 0x7f4d74072360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.208+0000 7f4d7924a700 1 -- 172.21.15.71:0/115630562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d700095f0 con 0x7f4d740ffba0 2024-04-23T02:15:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.208+0000 7f4d7924a700 1 --2- 172.21.15.71:0/115630562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d740ffba0 0x7f4d74071e00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4d70000c00 tx=0x7f4d70005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.208+0000 7f4d6a7fc700 1 -- 172.21.15.71:0/115630562 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d70003850 con 0x7f4d740ffba0 2024-04-23T02:15:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.208+0000 7f4d7b4ae700 1 -- 172.21.15.71:0/115630562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d74068e50 con 0x7f4d740ffba0 2024-04-23T02:15:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.208+0000 7f4d6a7fc700 1 -- 172.21.15.71:0/115630562 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4d70003e40 con 0x7f4d740ffba0 2024-04-23T02:15:05.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.208+0000 7f4d6a7fc700 1 -- 172.21.15.71:0/115630562 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d7000f690 con 0x7f4d740ffba0 2024-04-23T02:15:05.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.208+0000 7f4d7b4ae700 1 -- 172.21.15.71:0/115630562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d74069340 con 0x7f4d740ffba0 2024-04-23T02:15:05.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.209+0000 7f4d7b4ae700 1 -- 172.21.15.71:0/115630562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d741085c0 con 0x7f4d740ffba0 2024-04-23T02:15:05.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.209+0000 7f4d6a7fc700 1 -- 172.21.15.71:0/115630562 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 30) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f4d70020480 con 0x7f4d740ffba0 2024-04-23T02:15:05.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.210+0000 7f4d6a7fc700 1 --2- 172.21.15.71:0/115630562 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4d6006ce10 0x7f4d6006f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:05.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.210+0000 7f4d6a7fc700 1 -- 172.21.15.71:0/115630562 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4d70090290 con 0x7f4d740ffba0 2024-04-23T02:15:05.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.210+0000 7f4d78a49700 1 --2- 172.21.15.71:0/115630562 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4d6006ce10 0x7f4d6006f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:05.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.211+0000 7f4d78a49700 1 --2- 172.21.15.71:0/115630562 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4d6006ce10 0x7f4d6006f2d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4d74072ee0 tx=0x7f4d64009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:05.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.215+0000 7f4d6a7fc700 1 -- 172.21.15.71:0/115630562 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f4d7005dcd0 con 0x7f4d740ffba0 2024-04-23T02:15:05.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:05 smithi071 ceph-mon[39837]: Upgrade: Updating mgr.smithi192.qgecnv 2024-04-23T02:15:05.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:05 smithi071 ceph-mon[39837]: Deploying daemon mgr.smithi192.qgecnv on smithi192 2024-04-23T02:15:05.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:05 smithi071 ceph-mon[39837]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:15:05.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:05 smithi071 ceph-mon[39837]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:15:05.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:05 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:15:05.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.517+0000 7f4d7b4ae700 1 -- 172.21.15.71:0/115630562 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d741056f0 con 0x7f4d6006ce10 2024-04-23T02:15:05.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.518+0000 7f4d6a7fc700 1 -- 172.21.15.71:0/115630562 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f4d741056f0 con 0x7f4d6006ce10 2024-04-23T02:15:05.523 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:15:05.523 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:15:05.523 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:15:05.523 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-23T02:15:05.523 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:15:05.523 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr" 2024-04-23T02:15:05.523 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:15:05.523 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "2/2 daemons upgraded", 2024-04-23T02:15:05.523 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mgr daemons", 2024-04-23T02:15:05.523 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:15:05.523 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:15:05.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.521+0000 7f4d7b4ae700 1 -- 172.21.15.71:0/115630562 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4d6006ce10 msgr2=0x7f4d6006f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:05.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.521+0000 7f4d7b4ae700 1 --2- 172.21.15.71:0/115630562 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4d6006ce10 0x7f4d6006f2d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4d74072ee0 tx=0x7f4d64009260 comp rx=0 tx=0).stop 2024-04-23T02:15:05.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.521+0000 7f4d7b4ae700 1 -- 172.21.15.71:0/115630562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d740ffba0 msgr2=0x7f4d74071e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:05.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.521+0000 7f4d7b4ae700 1 --2- 172.21.15.71:0/115630562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d740ffba0 0x7f4d74071e00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4d70000c00 tx=0x7f4d70005e20 comp rx=0 tx=0).stop 2024-04-23T02:15:05.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.522+0000 7f4d7b4ae700 1 -- 172.21.15.71:0/115630562 shutdown_connections 2024-04-23T02:15:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.522+0000 7f4d7b4ae700 1 --2- 172.21.15.71:0/115630562 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4d6006ce10 0x7f4d6006f2d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.522+0000 7f4d7b4ae700 1 --2- 172.21.15.71:0/115630562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d740ffba0 0x7f4d74071e00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.522+0000 7f4d7b4ae700 1 --2- 172.21.15.71:0/115630562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d74100da0 0x7f4d74072360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.522+0000 7f4d7b4ae700 1 -- 172.21.15.71:0/115630562 >> 172.21.15.71:0/115630562 conn(0x7f4d740fb120 msgr2=0x7f4d74103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.522+0000 7f4d7b4ae700 1 -- 172.21.15.71:0/115630562 shutdown_connections 2024-04-23T02:15:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:05.523+0000 7f4d7b4ae700 1 -- 172.21.15.71:0/115630562 wait complete. 2024-04-23T02:15:06.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:06 smithi192 ceph-mon[49662]: from='client.14710 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:15:06.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:06 smithi192 ceph-mon[49662]: pgmap v18: 65 pgs: 65 active+clean; 320 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 157 op/s 2024-04-23T02:15:06.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:06 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/389285006' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:06.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:06 smithi071 ceph-mon[39837]: from='client.14710 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:15:06.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:06 smithi071 ceph-mon[39837]: pgmap v18: 65 pgs: 65 active+clean; 320 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 157 op/s 2024-04-23T02:15:06.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:06 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/389285006' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:07 smithi192 ceph-mon[49662]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:15:07.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:07 smithi071 ceph-mon[39837]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:15:08.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:08 smithi192 ceph-mon[49662]: pgmap v19: 65 pgs: 65 active+clean; 327 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 263 op/s 2024-04-23T02:15:08.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:08 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:08.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:08 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:08.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:08 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:15:08.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:08 smithi071 ceph-mon[39837]: pgmap v19: 65 pgs: 65 active+clean; 327 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 263 op/s 2024-04-23T02:15:08.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:08 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:08.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:08 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:08.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:08 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:15:10.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:10 smithi192 ceph-mon[49662]: pgmap v20: 65 pgs: 65 active+clean; 327 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-04-23T02:15:10.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:10 smithi071 ceph-mon[39837]: pgmap v20: 65 pgs: 65 active+clean; 327 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-04-23T02:15:12.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:12 smithi192 ceph-mon[49662]: pgmap v21: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 280 op/s 2024-04-23T02:15:12.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:12 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:12.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:12 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:12.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:12 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:12.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:12 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:12.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:12 smithi071 ceph-mon[39837]: pgmap v21: 65 pgs: 65 active+clean; 324 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 280 op/s 2024-04-23T02:15:12.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:12 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:12.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:12 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:12.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:12 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:12.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:12 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:14.436 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:14 smithi192 ceph-mon[49662]: pgmap v22: 65 pgs: 65 active+clean; 322 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 263 op/s 2024-04-23T02:15:14.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:14 smithi071 ceph-mon[39837]: pgmap v22: 65 pgs: 65 active+clean; 322 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 263 op/s 2024-04-23T02:15:16.241 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-23T02:15:16.241 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-23T02:15:16.337 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:16 smithi192 ceph-mon[49662]: pgmap v23: 65 pgs: 65 active+clean; 322 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-04-23T02:15:16.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:16 smithi071 ceph-mon[39837]: pgmap v23: 65 pgs: 65 active+clean; 322 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-04-23T02:15:16.564 DEBUG:teuthology.parallel:result is None 2024-04-23T02:15:17.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:17 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:17.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:17 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:17.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:17 smithi192 ceph-mon[49662]: pgmap v24: 65 pgs: 65 active+clean; 324 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 387 op/s 2024-04-23T02:15:17.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:17 smithi192 ceph-mon[49662]: Standby manager daemon smithi192.qgecnv restarted 2024-04-23T02:15:17.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:17 smithi192 ceph-mon[49662]: Standby manager daemon smithi192.qgecnv started 2024-04-23T02:15:17.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:17 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.192:0/2816868064' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.qgecnv/crt"}]: dispatch 2024-04-23T02:15:17.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:17 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.192:0/2816868064' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T02:15:17.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:17 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.192:0/2816868064' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.qgecnv/key"}]: dispatch 2024-04-23T02:15:17.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:17 smithi192 ceph-mon[49662]: from='mgr.? 172.21.15.192:0/2816868064' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T02:15:17.733 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-04-23T02:15:17.733 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-23T02:15:17.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:17 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:17.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:17 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:17.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:17 smithi071 ceph-mon[39837]: pgmap v24: 65 pgs: 65 active+clean; 324 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 387 op/s 2024-04-23T02:15:17.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:17 smithi071 ceph-mon[39837]: Standby manager daemon smithi192.qgecnv restarted 2024-04-23T02:15:17.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:17 smithi071 ceph-mon[39837]: Standby manager daemon smithi192.qgecnv started 2024-04-23T02:15:17.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:17 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.192:0/2816868064' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.qgecnv/crt"}]: dispatch 2024-04-23T02:15:17.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:17 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.192:0/2816868064' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T02:15:17.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:17 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.192:0/2816868064' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi192.qgecnv/key"}]: dispatch 2024-04-23T02:15:17.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:17 smithi071 ceph-mon[39837]: from='mgr.? 172.21.15.192:0/2816868064' entity='mgr.smithi192.qgecnv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T02:15:18.107 DEBUG:teuthology.parallel:result is None 2024-04-23T02:15:18.107 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-23T02:15:18.141 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-23T02:15:18.141 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-23T02:15:18.209 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-23T02:15:18.210 DEBUG:teuthology.parallel:result is None 2024-04-23T02:15:18.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:18 smithi192 ceph-mon[49662]: mgrmap e31: smithi071.ukdjvn(active, since 42s), standbys: smithi192.qgecnv 2024-04-23T02:15:18.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:18 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:18.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:18 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:18.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:18 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:15:18.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:18 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:15:18.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:18 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:18.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:18 smithi071 ceph-mon[39837]: mgrmap e31: smithi071.ukdjvn(active, since 42s), standbys: smithi192.qgecnv 2024-04-23T02:15:18.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:18 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:18.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:18 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:18.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:18 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:15:18.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:18 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:15:18.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:18 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:19.621 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:15:19.621 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:19.621 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: Upgrade: Setting container_image for all mgr 2024-04-23T02:15:19.621 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:19.621 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi071.ukdjvn"}]: dispatch 2024-04-23T02:15:19.621 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi071.ukdjvn"}]: dispatch 2024-04-23T02:15:19.621 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi071.ukdjvn"}]': finished 2024-04-23T02:15:19.621 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi192.qgecnv"}]: dispatch 2024-04-23T02:15:19.621 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi192.qgecnv"}]: dispatch 2024-04-23T02:15:19.621 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi192.qgecnv"}]': finished 2024-04-23T02:15:19.622 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:19.622 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: Upgrade: Setting container_image for all rgw 2024-04-23T02:15:19.622 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:19.622 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:19.622 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: Upgrade: Setting container_image for all rbd-mirror 2024-04-23T02:15:19.622 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:19.622 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:19.622 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-23T02:15:19.622 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:19.622 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:19.622 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: Upgrade: Setting container_image for all iscsi 2024-04-23T02:15:19.622 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:19.622 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:19.622 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: Upgrade: Setting container_image for all nfs 2024-04-23T02:15:19.623 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:19.623 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:19.623 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: Upgrade: Setting container_image for all nvmeof 2024-04-23T02:15:19.623 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:19.623 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:19 smithi071 ceph-mon[39837]: pgmap v25: 65 pgs: 65 active+clean; 324 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-04-23T02:15:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:15:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: Upgrade: Setting container_image for all mgr 2024-04-23T02:15:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi071.ukdjvn"}]: dispatch 2024-04-23T02:15:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi071.ukdjvn"}]: dispatch 2024-04-23T02:15:19.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi071.ukdjvn"}]': finished 2024-04-23T02:15:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi192.qgecnv"}]: dispatch 2024-04-23T02:15:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi192.qgecnv"}]: dispatch 2024-04-23T02:15:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi192.qgecnv"}]': finished 2024-04-23T02:15:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: Upgrade: Setting container_image for all rgw 2024-04-23T02:15:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: Upgrade: Setting container_image for all rbd-mirror 2024-04-23T02:15:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-23T02:15:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:19.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: Upgrade: Setting container_image for all iscsi 2024-04-23T02:15:19.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:19.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:19.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: Upgrade: Setting container_image for all nfs 2024-04-23T02:15:19.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:19.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:19.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: Upgrade: Setting container_image for all nvmeof 2024-04-23T02:15:19.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:19.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:19 smithi192 ceph-mon[49662]: pgmap v25: 65 pgs: 65 active+clean; 324 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-04-23T02:15:20.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:20 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:15:20.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:20 smithi071 ceph-mon[39837]: Upgrade: Updating node-exporter.smithi071 (1/2) 2024-04-23T02:15:20.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:20 smithi071 ceph-mon[39837]: Deploying daemon node-exporter.smithi071 on smithi071 2024-04-23T02:15:20.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:20 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:15:20.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:20 smithi192 ceph-mon[49662]: Upgrade: Updating node-exporter.smithi071 (1/2) 2024-04-23T02:15:20.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:20 smithi192 ceph-mon[49662]: Deploying daemon node-exporter.smithi071 on smithi071 2024-04-23T02:15:21.811 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:21 smithi071 ceph-mon[39837]: pgmap v26: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 349 op/s 2024-04-23T02:15:22.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:21 smithi192 ceph-mon[49662]: pgmap v26: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 349 op/s 2024-04-23T02:15:23.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:23 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:23.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:23 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:23.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:23 smithi071 ceph-mon[39837]: pgmap v27: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 294 op/s 2024-04-23T02:15:23.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:23 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:23.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:23 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:23.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:23 smithi192 ceph-mon[49662]: pgmap v27: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 294 op/s 2024-04-23T02:15:24.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:24 smithi071 ceph-mon[39837]: fsmap cephfs:2 {0=cephfs.smithi071.vxejye=up:active,1=cephfs.smithi192.gtmyst=up:active} 2 up:standby-replay 2024-04-23T02:15:24.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:24 smithi192 ceph-mon[49662]: fsmap cephfs:2 {0=cephfs.smithi071.vxejye=up:active,1=cephfs.smithi192.gtmyst=up:active} 2 up:standby-replay 2024-04-23T02:15:25.742 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:25 smithi192 ceph-mon[49662]: pgmap v28: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 240 op/s 2024-04-23T02:15:25.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:25 smithi071 ceph-mon[39837]: pgmap v28: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 240 op/s 2024-04-23T02:15:27.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:27 smithi071 ceph-mon[39837]: pgmap v29: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 244 op/s 2024-04-23T02:15:27.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:27 smithi071 ceph-mon[39837]: Upgrade: Updating node-exporter.smithi192 (2/2) 2024-04-23T02:15:27.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:27 smithi071 ceph-mon[39837]: Deploying daemon node-exporter.smithi192 on smithi192 2024-04-23T02:15:28.115 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:27 smithi192 ceph-mon[49662]: pgmap v29: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 244 op/s 2024-04-23T02:15:28.115 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:27 smithi192 ceph-mon[49662]: Upgrade: Updating node-exporter.smithi192 (2/2) 2024-04-23T02:15:28.115 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:27 smithi192 ceph-mon[49662]: Deploying daemon node-exporter.smithi192 on smithi192 2024-04-23T02:15:30.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:30 smithi071 ceph-mon[39837]: pgmap v30: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 393 KiB/s rd, 347 KiB/s wr, 95 op/s 2024-04-23T02:15:30.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:30 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:30.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:30 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:30.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:30 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:15:30.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:30 smithi192 ceph-mon[49662]: pgmap v30: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 393 KiB/s rd, 347 KiB/s wr, 95 op/s 2024-04-23T02:15:30.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:30 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:30.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:30 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:30.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:30 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:15:32.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:32 smithi071 ceph-mon[39837]: pgmap v31: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 394 KiB/s rd, 348 KiB/s wr, 97 op/s 2024-04-23T02:15:32.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:32 smithi192 ceph-mon[49662]: pgmap v31: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 394 KiB/s rd, 348 KiB/s wr, 97 op/s 2024-04-23T02:15:33.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:33 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:33.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:33 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:33.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:33 smithi071 ceph-mon[39837]: pgmap v32: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 111 KiB/s rd, 109 KiB/s wr, 30 op/s 2024-04-23T02:15:33.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:33 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:33.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:33 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:33.526 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:33 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:33.526 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:33 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:33.526 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:33 smithi192 ceph-mon[49662]: pgmap v32: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 111 KiB/s rd, 109 KiB/s wr, 30 op/s 2024-04-23T02:15:33.526 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:33 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:33.526 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:33 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:35.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.649+0000 7fd746145700 1 -- 172.21.15.71:0/72339495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7401040d0 msgr2=0x7fd7401064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:35.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.649+0000 7fd746145700 1 --2- 172.21.15.71:0/72339495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7401040d0 0x7fd7401064c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd734009910 tx=0x7fd734009c20 comp rx=0 tx=0).stop 2024-04-23T02:15:35.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.649+0000 7fd746145700 1 -- 172.21.15.71:0/72339495 shutdown_connections 2024-04-23T02:15:35.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.649+0000 7fd746145700 1 --2- 172.21.15.71:0/72339495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7401040d0 0x7fd7401064c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:35.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.649+0000 7fd746145700 1 --2- 172.21.15.71:0/72339495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7401017a0 0x7fd740103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:35.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.649+0000 7fd746145700 1 -- 172.21.15.71:0/72339495 >> 172.21.15.71:0/72339495 conn(0x7fd7400fb190 msgr2=0x7fd7400fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:35.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.649+0000 7fd746145700 1 -- 172.21.15.71:0/72339495 shutdown_connections 2024-04-23T02:15:35.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.649+0000 7fd746145700 1 -- 172.21.15.71:0/72339495 wait complete. 2024-04-23T02:15:35.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.649+0000 7fd746145700 1 Processor -- start 2024-04-23T02:15:35.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.649+0000 7fd746145700 1 -- start start 2024-04-23T02:15:35.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.650+0000 7fd746145700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7401017a0 0x7fd7401963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:35.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.650+0000 7fd746145700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7401040d0 0x7fd740196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:35.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.650+0000 7fd746145700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd740196f20 con 0x7fd7401017a0 2024-04-23T02:15:35.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.650+0000 7fd746145700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd740197060 con 0x7fd7401040d0 2024-04-23T02:15:35.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.650+0000 7fd73f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7401017a0 0x7fd7401963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:35.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.650+0000 7fd73f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7401017a0 0x7fd7401963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42390/0 (socket says 172.21.15.71:42390) 2024-04-23T02:15:35.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.650+0000 7fd73f7fe700 1 -- 172.21.15.71:0/4017615006 learned_addr learned my addr 172.21.15.71:0/4017615006 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:15:35.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.650+0000 7fd73effd700 1 --2- 172.21.15.71:0/4017615006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7401040d0 0x7fd740196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:35.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.650+0000 7fd73f7fe700 1 -- 172.21.15.71:0/4017615006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7401040d0 msgr2=0x7fd740196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:35.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.650+0000 7fd73f7fe700 1 --2- 172.21.15.71:0/4017615006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7401040d0 0x7fd740196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:35.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.650+0000 7fd73f7fe700 1 -- 172.21.15.71:0/4017615006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7340095f0 con 0x7fd7401017a0 2024-04-23T02:15:35.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.650+0000 7fd73f7fe700 1 --2- 172.21.15.71:0/4017615006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7401017a0 0x7fd7401963c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd73000e920 tx=0x7fd73000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:35.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.650+0000 7fd73cff9700 1 -- 172.21.15.71:0/4017615006 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd73000c9d0 con 0x7fd7401017a0 2024-04-23T02:15:35.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.650+0000 7fd73cff9700 1 -- 172.21.15.71:0/4017615006 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd730010430 con 0x7fd7401017a0 2024-04-23T02:15:35.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.650+0000 7fd746145700 1 -- 172.21.15.71:0/4017615006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74019bb10 con 0x7fd7401017a0 2024-04-23T02:15:35.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.650+0000 7fd73cff9700 1 -- 172.21.15.71:0/4017615006 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd730018630 con 0x7fd7401017a0 2024-04-23T02:15:35.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.650+0000 7fd746145700 1 -- 172.21.15.71:0/4017615006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74019c060 con 0x7fd7401017a0 2024-04-23T02:15:35.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.651+0000 7fd746145700 1 -- 172.21.15.71:0/4017615006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7401904d0 con 0x7fd7401017a0 2024-04-23T02:15:35.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.652+0000 7fd73cff9700 1 -- 172.21.15.71:0/4017615006 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 31) v1 ==== 90944+0+0 (secure 0 0 0) 0x7fd7300105a0 con 0x7fd7401017a0 2024-04-23T02:15:35.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.652+0000 7fd73cff9700 1 --2- 172.21.15.71:0/4017615006 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd72806ce20 0x7fd72806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:35.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.652+0000 7fd73effd700 1 --2- 172.21.15.71:0/4017615006 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd72806ce20 0x7fd72806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:35.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.653+0000 7fd73cff9700 1 -- 172.21.15.71:0/4017615006 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd730014070 con 0x7fd7401017a0 2024-04-23T02:15:35.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.653+0000 7fd73effd700 1 --2- 172.21.15.71:0/4017615006 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd72806ce20 0x7fd72806f2e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd734009fd0 tx=0x7fd73400b5a0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:35.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.657+0000 7fd73cff9700 1 -- 172.21.15.71:0/4017615006 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fd7300529b0 con 0x7fd7401017a0 2024-04-23T02:15:35.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.872+0000 7fd746145700 1 -- 172.21.15.71:0/4017615006 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7400611d0 con 0x7fd72806ce20 2024-04-23T02:15:35.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.874+0000 7fd73cff9700 1 -- 172.21.15.71:0/4017615006 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fd7400611d0 con 0x7fd72806ce20 2024-04-23T02:15:35.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.878+0000 7fd746145700 1 -- 172.21.15.71:0/4017615006 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd72806ce20 msgr2=0x7fd72806f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:35.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.878+0000 7fd746145700 1 --2- 172.21.15.71:0/4017615006 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd72806ce20 0x7fd72806f2e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd734009fd0 tx=0x7fd73400b5a0 comp rx=0 tx=0).stop 2024-04-23T02:15:35.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.878+0000 7fd746145700 1 -- 172.21.15.71:0/4017615006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7401017a0 msgr2=0x7fd7401963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:35.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.878+0000 7fd746145700 1 --2- 172.21.15.71:0/4017615006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7401017a0 0x7fd7401963c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd73000e920 tx=0x7fd73000ec30 comp rx=0 tx=0).stop 2024-04-23T02:15:35.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.878+0000 7fd746145700 1 -- 172.21.15.71:0/4017615006 shutdown_connections 2024-04-23T02:15:35.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.878+0000 7fd746145700 1 --2- 172.21.15.71:0/4017615006 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd72806ce20 0x7fd72806f2e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:35.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.878+0000 7fd746145700 1 --2- 172.21.15.71:0/4017615006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7401017a0 0x7fd7401963c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:35.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.878+0000 7fd746145700 1 --2- 172.21.15.71:0/4017615006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7401040d0 0x7fd740196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:35.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.878+0000 7fd746145700 1 -- 172.21.15.71:0/4017615006 >> 172.21.15.71:0/4017615006 conn(0x7fd7400fb190 msgr2=0x7fd7400fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:35.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.878+0000 7fd746145700 1 -- 172.21.15.71:0/4017615006 shutdown_connections 2024-04-23T02:15:35.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.878+0000 7fd746145700 1 -- 172.21.15.71:0/4017615006 wait complete. 2024-04-23T02:15:35.897 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:15:35.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:35 smithi071 ceph-mon[39837]: pgmap v33: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 33 KiB/s rd, 31 KiB/s wr, 5 op/s 2024-04-23T02:15:35.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:35 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:15:35.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.993+0000 7f3e01708700 1 -- 172.21.15.71:0/1522676910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3dfc101eb0 msgr2=0x7f3dfc102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:35.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.993+0000 7f3e01708700 1 --2- 172.21.15.71:0/1522676910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3dfc101eb0 0x7f3dfc102310 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009910 tx=0x7f3de4009c20 comp rx=0 tx=0).stop 2024-04-23T02:15:35.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.994+0000 7f3e01708700 1 -- 172.21.15.71:0/1522676910 shutdown_connections 2024-04-23T02:15:35.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.994+0000 7f3e01708700 1 --2- 172.21.15.71:0/1522676910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3dfc101eb0 0x7f3dfc102310 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:35.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.994+0000 7f3e01708700 1 --2- 172.21.15.71:0/1522676910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3dfc100cb0 0x7f3dfc1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:35.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.994+0000 7f3e01708700 1 -- 172.21.15.71:0/1522676910 >> 172.21.15.71:0/1522676910 conn(0x7f3dfc0fc230 msgr2=0x7f3dfc0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:35.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.994+0000 7f3e01708700 1 -- 172.21.15.71:0/1522676910 shutdown_connections 2024-04-23T02:15:35.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.994+0000 7f3e01708700 1 -- 172.21.15.71:0/1522676910 wait complete. 2024-04-23T02:15:36.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.994+0000 7f3e01708700 1 Processor -- start 2024-04-23T02:15:36.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.994+0000 7f3e01708700 1 -- start start 2024-04-23T02:15:36.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.995+0000 7f3e01708700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3dfc100cb0 0x7f3dfc074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:36.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.995+0000 7f3e01708700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3dfc101eb0 0x7f3dfc073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:36.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.995+0000 7f3e01708700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc0736c0 con 0x7f3dfc101eb0 2024-04-23T02:15:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.995+0000 7f3e01708700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc073800 con 0x7f3dfc100cb0 2024-04-23T02:15:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.995+0000 7f3dfa7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3dfc101eb0 0x7f3dfc073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.995+0000 7f3dfa7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3dfc101eb0 0x7f3dfc073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42414/0 (socket says 172.21.15.71:42414) 2024-04-23T02:15:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.995+0000 7f3dfa7fc700 1 -- 172.21.15.71:0/3438149460 learned_addr learned my addr 172.21.15.71:0/3438149460 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:15:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.995+0000 7f3dfaffd700 1 --2- 172.21.15.71:0/3438149460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3dfc100cb0 0x7f3dfc074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.995+0000 7f3dfa7fc700 1 -- 172.21.15.71:0/3438149460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3dfc100cb0 msgr2=0x7f3dfc074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.995+0000 7f3dfa7fc700 1 --2- 172.21.15.71:0/3438149460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3dfc100cb0 0x7f3dfc074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.995+0000 7f3dfa7fc700 1 -- 172.21.15.71:0/3438149460 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de40095f0 con 0x7f3dfc101eb0 2024-04-23T02:15:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.995+0000 7f3dfa7fc700 1 --2- 172.21.15.71:0/3438149460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3dfc101eb0 0x7f3dfc073180 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3de400bde0 tx=0x7f3de400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.995+0000 7f3df3fff700 1 -- 172.21.15.71:0/3438149460 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3de401d070 con 0x7f3dfc101eb0 2024-04-23T02:15:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.995+0000 7f3df3fff700 1 -- 172.21.15.71:0/3438149460 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f3de4005dc0 con 0x7f3dfc101eb0 2024-04-23T02:15:36.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.995+0000 7f3e01708700 1 -- 172.21.15.71:0/3438149460 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dfc073a80 con 0x7f3dfc101eb0 2024-04-23T02:15:36.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.995+0000 7f3df3fff700 1 -- 172.21.15.71:0/3438149460 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3de4003b10 con 0x7f3dfc101eb0 2024-04-23T02:15:36.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.995+0000 7f3e01708700 1 -- 172.21.15.71:0/3438149460 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dfc073f10 con 0x7f3dfc101eb0 2024-04-23T02:15:36.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.996+0000 7f3e01708700 1 -- 172.21.15.71:0/3438149460 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dfc1096f0 con 0x7f3dfc101eb0 2024-04-23T02:15:36.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.997+0000 7f3df3fff700 1 -- 172.21.15.71:0/3438149460 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 31) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f3de40048e0 con 0x7f3dfc101eb0 2024-04-23T02:15:36.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.997+0000 7f3df3fff700 1 --2- 172.21.15.71:0/3438149460 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3de806ce20 0x7f3de806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:36.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.997+0000 7f3df3fff700 1 -- 172.21.15.71:0/3438149460 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3de408dea0 con 0x7f3dfc101eb0 2024-04-23T02:15:36.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.997+0000 7f3dfaffd700 1 --2- 172.21.15.71:0/3438149460 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3de806ce20 0x7f3de806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:36.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:35.998+0000 7f3dfaffd700 1 --2- 172.21.15.71:0/3438149460 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3de806ce20 0x7f3de806f2e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc101d10 tx=0x7f3df400b040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:36.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.002+0000 7f3df3fff700 1 -- 172.21.15.71:0/3438149460 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f3de405b830 con 0x7f3dfc101eb0 2024-04-23T02:15:36.023 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:35 smithi192 ceph-mon[49662]: pgmap v33: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 33 KiB/s rd, 31 KiB/s wr, 5 op/s 2024-04-23T02:15:36.024 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:35 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:15:36.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.214+0000 7f3e01708700 1 -- 172.21.15.71:0/3438149460 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3dfc19e890 con 0x7f3de806ce20 2024-04-23T02:15:36.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.215+0000 7f3df3fff700 1 -- 172.21.15.71:0/3438149460 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f3dfc19e890 con 0x7f3de806ce20 2024-04-23T02:15:36.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.218+0000 7f3e01708700 1 -- 172.21.15.71:0/3438149460 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3de806ce20 msgr2=0x7f3de806f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:36.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.218+0000 7f3e01708700 1 --2- 172.21.15.71:0/3438149460 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3de806ce20 0x7f3de806f2e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc101d10 tx=0x7f3df400b040 comp rx=0 tx=0).stop 2024-04-23T02:15:36.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.218+0000 7f3e01708700 1 -- 172.21.15.71:0/3438149460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3dfc101eb0 msgr2=0x7f3dfc073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:36.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.218+0000 7f3e01708700 1 --2- 172.21.15.71:0/3438149460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3dfc101eb0 0x7f3dfc073180 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3de400bde0 tx=0x7f3de400be10 comp rx=0 tx=0).stop 2024-04-23T02:15:36.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.218+0000 7f3e01708700 1 -- 172.21.15.71:0/3438149460 shutdown_connections 2024-04-23T02:15:36.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.218+0000 7f3e01708700 1 --2- 172.21.15.71:0/3438149460 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3de806ce20 0x7f3de806f2e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:36.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.218+0000 7f3e01708700 1 --2- 172.21.15.71:0/3438149460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3dfc100cb0 0x7f3dfc074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:36.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.218+0000 7f3e01708700 1 --2- 172.21.15.71:0/3438149460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3dfc101eb0 0x7f3dfc073180 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:36.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.218+0000 7f3e01708700 1 -- 172.21.15.71:0/3438149460 >> 172.21.15.71:0/3438149460 conn(0x7f3dfc0fc230 msgr2=0x7f3dfc1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:36.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.218+0000 7f3e01708700 1 -- 172.21.15.71:0/3438149460 shutdown_connections 2024-04-23T02:15:36.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.218+0000 7f3e01708700 1 -- 172.21.15.71:0/3438149460 wait complete. 2024-04-23T02:15:36.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.322+0000 7f04e84ee700 1 -- 172.21.15.71:0/3502122511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e00ffc10 msgr2=0x7f04e0100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:36.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.322+0000 7f04e84ee700 1 --2- 172.21.15.71:0/3502122511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e00ffc10 0x7f04e0100030 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009910 tx=0x7f04d4009c20 comp rx=0 tx=0).stop 2024-04-23T02:15:36.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.322+0000 7f04e84ee700 1 -- 172.21.15.71:0/3502122511 shutdown_connections 2024-04-23T02:15:36.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.322+0000 7f04e84ee700 1 --2- 172.21.15.71:0/3502122511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04e0100e10 0x7f04e0101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:36.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.322+0000 7f04e84ee700 1 --2- 172.21.15.71:0/3502122511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e00ffc10 0x7f04e0100030 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:36.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.322+0000 7f04e84ee700 1 -- 172.21.15.71:0/3502122511 >> 172.21.15.71:0/3502122511 conn(0x7f04e00fb190 msgr2=0x7f04e00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:36.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.322+0000 7f04e84ee700 1 -- 172.21.15.71:0/3502122511 shutdown_connections 2024-04-23T02:15:36.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.322+0000 7f04e84ee700 1 -- 172.21.15.71:0/3502122511 wait complete. 2024-04-23T02:15:36.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.322+0000 7f04e84ee700 1 Processor -- start 2024-04-23T02:15:36.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.322+0000 7f04e84ee700 1 -- start start 2024-04-23T02:15:36.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.323+0000 7f04e84ee700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04e00ffc10 0x7f04e0196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:36.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.323+0000 7f04e84ee700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e0100e10 0x7f04e0196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:36.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.323+0000 7f04e84ee700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e0196f70 con 0x7f04e0100e10 2024-04-23T02:15:36.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.323+0000 7f04e84ee700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e01970b0 con 0x7f04e00ffc10 2024-04-23T02:15:36.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.323+0000 7f04e5a89700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e0100e10 0x7f04e0196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:36.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.323+0000 7f04e5a89700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e0100e10 0x7f04e0196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42432/0 (socket says 172.21.15.71:42432) 2024-04-23T02:15:36.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.323+0000 7f04e5a89700 1 -- 172.21.15.71:0/2733597765 learned_addr learned my addr 172.21.15.71:0/2733597765 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:15:36.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.323+0000 7f04e628a700 1 --2- 172.21.15.71:0/2733597765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04e00ffc10 0x7f04e0196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:36.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.323+0000 7f04e5a89700 1 -- 172.21.15.71:0/2733597765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04e00ffc10 msgr2=0x7f04e0196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:36.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.323+0000 7f04e5a89700 1 --2- 172.21.15.71:0/2733597765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04e00ffc10 0x7f04e0196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:36.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.323+0000 7f04e5a89700 1 -- 172.21.15.71:0/2733597765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d40095f0 con 0x7f04e0100e10 2024-04-23T02:15:36.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.323+0000 7f04e5a89700 1 --2- 172.21.15.71:0/2733597765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e0100e10 0x7f04e0196950 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f04dc00c8a0 tx=0x7f04dc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:36.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.323+0000 7f04d37fe700 1 -- 172.21.15.71:0/2733597765 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04dc00cea0 con 0x7f04e0100e10 2024-04-23T02:15:36.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.323+0000 7f04d37fe700 1 -- 172.21.15.71:0/2733597765 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f04dc004830 con 0x7f04e0100e10 2024-04-23T02:15:36.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.323+0000 7f04e84ee700 1 -- 172.21.15.71:0/2733597765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e019bb60 con 0x7f04e0100e10 2024-04-23T02:15:36.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.323+0000 7f04d37fe700 1 -- 172.21.15.71:0/2733597765 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04dc005540 con 0x7f04e0100e10 2024-04-23T02:15:36.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.323+0000 7f04e84ee700 1 -- 172.21.15.71:0/2733597765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e019c0b0 con 0x7f04e0100e10 2024-04-23T02:15:36.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.324+0000 7f04e84ee700 1 -- 172.21.15.71:0/2733597765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e01085c0 con 0x7f04e0100e10 2024-04-23T02:15:36.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.325+0000 7f04d37fe700 1 -- 172.21.15.71:0/2733597765 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 31) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f04dc0049a0 con 0x7f04e0100e10 2024-04-23T02:15:36.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.325+0000 7f04d37fe700 1 --2- 172.21.15.71:0/2733597765 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f04cc06ce20 0x7f04cc06f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:36.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.325+0000 7f04d37fe700 1 -- 172.21.15.71:0/2733597765 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f04dc08c330 con 0x7f04e0100e10 2024-04-23T02:15:36.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.325+0000 7f04e628a700 1 --2- 172.21.15.71:0/2733597765 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f04cc06ce20 0x7f04cc06f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:36.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.326+0000 7f04e628a700 1 --2- 172.21.15.71:0/2733597765 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f04cc06ce20 0x7f04cc06f2e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f04d4006010 tx=0x7f04d400b580 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:36.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.330+0000 7f04d37fe700 1 -- 172.21.15.71:0/2733597765 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f04dc059d40 con 0x7f04e0100e10 2024-04-23T02:15:36.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.571+0000 7f04e84ee700 1 -- 172.21.15.71:0/2733597765 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f04e0068c90 con 0x7f04cc06ce20 2024-04-23T02:15:36.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.583+0000 7f04d37fe700 1 -- 172.21.15.71:0/2733597765 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f04e0068c90 con 0x7f04cc06ce20 2024-04-23T02:15:36.587 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:15:36.587 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (9m) 20s ago 10m 26.5M - 0.25.0 c8568f914cd2 8261e17319fe 2024-04-23T02:15:36.587 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (10m) 20s ago 10m 7864k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:15:36.588 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (9m) 4s ago 9m 7796k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:15:36.588 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (10m) 20s ago 10m 6656k - 18.2.1 5be31c24972a 258512dcbc00 2024-04-23T02:15:36.588 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (9m) 4s ago 9m 6656k - 18.2.1 5be31c24972a feea2c9896a6 2024-04-23T02:15:36.588 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (8m) 20s ago 10m 87.3M - 9.4.7 954c08fa6188 48ea3e02b806 2024-04-23T02:15:36.588 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (4m) 20s ago 4m 100M - 18.2.1 5be31c24972a a515de2a841c 2024-04-23T02:15:36.588 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (4m) 20s ago 4m 148M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:15:36.588 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (4m) 4s ago 4m 68.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:15:36.588 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (4m) 4s ago 4m 125M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:15:36.588 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (74s) 20s ago 12m 490M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:15:36.588 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (29s) 4s ago 9m 449M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:15:36.588 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (12m) 20s ago 12m 58.5M 2048M 18.2.1 5be31c24972a ff40abe63fd9 2024-04-23T02:15:36.588 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (9m) 4s ago 9m 46.7M 2048M 18.2.1 5be31c24972a 9c03e90acc95 2024-04-23T02:15:36.588 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 starting - - - - 2024-04-23T02:15:36.588 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (8s) 4s ago 9m 3510k - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:15:36.589 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (8m) 20s ago 8m 187M 1227M 18.2.1 5be31c24972a d3d5f2fa7267 2024-04-23T02:15:36.589 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (8m) 20s ago 8m 202M 1227M 18.2.1 5be31c24972a 3992157ff0fd 2024-04-23T02:15:36.589 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (7m) 20s ago 7m 158M 1227M 18.2.1 5be31c24972a 12380b6a15a2 2024-04-23T02:15:36.589 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (7m) 4s ago 7m 240M 2251M 18.2.1 5be31c24972a d5f587ee3a22 2024-04-23T02:15:36.589 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (6m) 4s ago 6m 217M 2251M 18.2.1 5be31c24972a 958d046005f0 2024-04-23T02:15:36.589 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (6m) 4s ago 6m 195M 2251M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:15:36.589 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (39s) 20s ago 10m 41.2M - 2.43.0 a07b618ecd1d 749acbfb7c5a 2024-04-23T02:15:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.585+0000 7f04e84ee700 1 -- 172.21.15.71:0/2733597765 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f04cc06ce20 msgr2=0x7f04cc06f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.585+0000 7f04e84ee700 1 --2- 172.21.15.71:0/2733597765 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f04cc06ce20 0x7f04cc06f2e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f04d4006010 tx=0x7f04d400b580 comp rx=0 tx=0).stop 2024-04-23T02:15:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.585+0000 7f04e84ee700 1 -- 172.21.15.71:0/2733597765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e0100e10 msgr2=0x7f04e0196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.585+0000 7f04e84ee700 1 --2- 172.21.15.71:0/2733597765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e0100e10 0x7f04e0196950 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f04dc00c8a0 tx=0x7f04dc00cc60 comp rx=0 tx=0).stop 2024-04-23T02:15:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.585+0000 7f04e84ee700 1 -- 172.21.15.71:0/2733597765 shutdown_connections 2024-04-23T02:15:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.585+0000 7f04e84ee700 1 --2- 172.21.15.71:0/2733597765 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f04cc06ce20 0x7f04cc06f2e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.585+0000 7f04e84ee700 1 --2- 172.21.15.71:0/2733597765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04e00ffc10 0x7f04e0196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.585+0000 7f04e84ee700 1 --2- 172.21.15.71:0/2733597765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e0100e10 0x7f04e0196950 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.585+0000 7f04e84ee700 1 -- 172.21.15.71:0/2733597765 >> 172.21.15.71:0/2733597765 conn(0x7f04e00fb190 msgr2=0x7f04e0068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.585+0000 7f04e84ee700 1 -- 172.21.15.71:0/2733597765 shutdown_connections 2024-04-23T02:15:36.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.585+0000 7f04e84ee700 1 -- 172.21.15.71:0/2733597765 wait complete. 2024-04-23T02:15:36.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.695+0000 7f22b5899700 1 -- 172.21.15.71:0/2880651260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22b0101f40 msgr2=0x7f22b01023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:36.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.695+0000 7f22b5899700 1 --2- 172.21.15.71:0/2880651260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22b0101f40 0x7f22b01023a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009910 tx=0x7f22a4009c20 comp rx=0 tx=0).stop 2024-04-23T02:15:36.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.695+0000 7f22b5899700 1 -- 172.21.15.71:0/2880651260 shutdown_connections 2024-04-23T02:15:36.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.695+0000 7f22b5899700 1 --2- 172.21.15.71:0/2880651260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22b0101f40 0x7f22b01023a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:36.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.695+0000 7f22b5899700 1 --2- 172.21.15.71:0/2880651260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22b00fef60 0x7f22b00ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:36.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.695+0000 7f22b5899700 1 -- 172.21.15.71:0/2880651260 >> 172.21.15.71:0/2880651260 conn(0x7f22b00fa8c0 msgr2=0x7f22b00fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:36.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.695+0000 7f22b5899700 1 -- 172.21.15.71:0/2880651260 shutdown_connections 2024-04-23T02:15:36.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.695+0000 7f22b5899700 1 -- 172.21.15.71:0/2880651260 wait complete. 2024-04-23T02:15:36.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.695+0000 7f22b5899700 1 Processor -- start 2024-04-23T02:15:36.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.695+0000 7f22b5899700 1 -- start start 2024-04-23T02:15:36.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.696+0000 7f22b5899700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22b00fef60 0x7f22b01963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:36.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.696+0000 7f22b5899700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22b0101f40 0x7f22b0196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:36.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.696+0000 7f22b5899700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b0196f30 con 0x7f22b00fef60 2024-04-23T02:15:36.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.696+0000 7f22b5899700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b0197070 con 0x7f22b0101f40 2024-04-23T02:15:36.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.696+0000 7f22aeffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22b00fef60 0x7f22b01963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:36.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.696+0000 7f22aeffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22b00fef60 0x7f22b01963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42454/0 (socket says 172.21.15.71:42454) 2024-04-23T02:15:36.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.696+0000 7f22aeffd700 1 -- 172.21.15.71:0/1858333401 learned_addr learned my addr 172.21.15.71:0/1858333401 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:15:36.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.696+0000 7f22ae7fc700 1 --2- 172.21.15.71:0/1858333401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22b0101f40 0x7f22b0196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:36.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.696+0000 7f22aeffd700 1 -- 172.21.15.71:0/1858333401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22b0101f40 msgr2=0x7f22b0196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:36.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.696+0000 7f22aeffd700 1 --2- 172.21.15.71:0/1858333401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22b0101f40 0x7f22b0196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:36.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.696+0000 7f22aeffd700 1 -- 172.21.15.71:0/1858333401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a40095f0 con 0x7f22b00fef60 2024-04-23T02:15:36.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.696+0000 7f22aeffd700 1 --2- 172.21.15.71:0/1858333401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22b00fef60 0x7f22b01963d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f22a000e8f0 tx=0x7f22a000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:36.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.696+0000 7f22b4897700 1 -- 172.21.15.71:0/1858333401 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22a0009790 con 0x7f22b00fef60 2024-04-23T02:15:36.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.696+0000 7f22b4897700 1 -- 172.21.15.71:0/1858333401 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f22a00044b0 con 0x7f22b00fef60 2024-04-23T02:15:36.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.696+0000 7f22b5899700 1 -- 172.21.15.71:0/1858333401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b019bb20 con 0x7f22b00fef60 2024-04-23T02:15:36.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.696+0000 7f22b4897700 1 -- 172.21.15.71:0/1858333401 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22a0003d30 con 0x7f22b00fef60 2024-04-23T02:15:36.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.696+0000 7f22b5899700 1 -- 172.21.15.71:0/1858333401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b019c070 con 0x7f22b00fef60 2024-04-23T02:15:36.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.697+0000 7f22b5899700 1 -- 172.21.15.71:0/1858333401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b01096c0 con 0x7f22b00fef60 2024-04-23T02:15:36.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.698+0000 7f22b4897700 1 -- 172.21.15.71:0/1858333401 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 31) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f22a0004000 con 0x7f22b00fef60 2024-04-23T02:15:36.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.698+0000 7f22b4897700 1 --2- 172.21.15.71:0/1858333401 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f229806ce20 0x7f229806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:36.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.698+0000 7f22b4897700 1 -- 172.21.15.71:0/1858333401 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f22a0014070 con 0x7f22b00fef60 2024-04-23T02:15:36.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.699+0000 7f22ae7fc700 1 --2- 172.21.15.71:0/1858333401 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f229806ce20 0x7f229806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:36.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.699+0000 7f22ae7fc700 1 --2- 172.21.15.71:0/1858333401 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f229806ce20 0x7f229806f2e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f22a400b600 tx=0x7f22a4009f70 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:36.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:36.704+0000 7f22b4897700 1 -- 172.21.15.71:0/1858333401 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f22a005a420 con 0x7f22b00fef60 2024-04-23T02:15:36.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:36 smithi071 ceph-mon[39837]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:15:36.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:36 smithi071 ceph-mon[39837]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:15:36.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:36 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:36.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:36 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:37.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.009+0000 7f22b5899700 1 -- 172.21.15.71:0/1858333401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f22b019c680 con 0x7f22b00fef60 2024-04-23T02:15:37.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.009+0000 7f22b4897700 1 -- 172.21.15.71:0/1858333401 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f22a0059fb0 con 0x7f22b00fef60 2024-04-23T02:15:37.014 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:15:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:15:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-23T02:15:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:15:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:15:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:15:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:15:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:15:37.014 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-23T02:15:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:15:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:15:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-23T02:15:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:15:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:15:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-23T02:15:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:15:37.015 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:15:37.015 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:15:37.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.011+0000 7f22b5899700 1 -- 172.21.15.71:0/1858333401 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f229806ce20 msgr2=0x7f229806f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:37.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.011+0000 7f22b5899700 1 --2- 172.21.15.71:0/1858333401 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f229806ce20 0x7f229806f2e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f22a400b600 tx=0x7f22a4009f70 comp rx=0 tx=0).stop 2024-04-23T02:15:37.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.011+0000 7f22b5899700 1 -- 172.21.15.71:0/1858333401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22b00fef60 msgr2=0x7f22b01963d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:37.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.011+0000 7f22b5899700 1 --2- 172.21.15.71:0/1858333401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22b00fef60 0x7f22b01963d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f22a000e8f0 tx=0x7f22a000ecb0 comp rx=0 tx=0).stop 2024-04-23T02:15:37.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.012+0000 7f22b5899700 1 -- 172.21.15.71:0/1858333401 shutdown_connections 2024-04-23T02:15:37.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.012+0000 7f22b5899700 1 --2- 172.21.15.71:0/1858333401 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f229806ce20 0x7f229806f2e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:37.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.012+0000 7f22b5899700 1 --2- 172.21.15.71:0/1858333401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22b00fef60 0x7f22b01963d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:37.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.012+0000 7f22b5899700 1 --2- 172.21.15.71:0/1858333401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22b0101f40 0x7f22b0196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:37.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.012+0000 7f22b5899700 1 -- 172.21.15.71:0/1858333401 >> 172.21.15.71:0/1858333401 conn(0x7f22b00fa8c0 msgr2=0x7f22b0105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:37.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.012+0000 7f22b5899700 1 -- 172.21.15.71:0/1858333401 shutdown_connections 2024-04-23T02:15:37.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.012+0000 7f22b5899700 1 -- 172.21.15.71:0/1858333401 wait complete. 2024-04-23T02:15:37.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.119+0000 7f7630483700 1 -- 172.21.15.71:0/2340263262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7628104130 msgr2=0x7f7628106520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:37.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.119+0000 7f7630483700 1 --2- 172.21.15.71:0/2340263262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7628104130 0x7f7628106520 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7618009910 tx=0x7f7618009c20 comp rx=0 tx=0).stop 2024-04-23T02:15:37.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.119+0000 7f7630483700 1 -- 172.21.15.71:0/2340263262 shutdown_connections 2024-04-23T02:15:37.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.119+0000 7f7630483700 1 --2- 172.21.15.71:0/2340263262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7628104130 0x7f7628106520 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:37.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.119+0000 7f7630483700 1 --2- 172.21.15.71:0/2340263262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7628101800 0x7f7628103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:37.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.119+0000 7f7630483700 1 -- 172.21.15.71:0/2340263262 >> 172.21.15.71:0/2340263262 conn(0x7f76280fb190 msgr2=0x7f76280fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:37.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.119+0000 7f7630483700 1 -- 172.21.15.71:0/2340263262 shutdown_connections 2024-04-23T02:15:37.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.119+0000 7f7630483700 1 -- 172.21.15.71:0/2340263262 wait complete. 2024-04-23T02:15:37.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.120+0000 7f7630483700 1 Processor -- start 2024-04-23T02:15:37.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.120+0000 7f7630483700 1 -- start start 2024-04-23T02:15:37.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.120+0000 7f7630483700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7628101800 0x7f7628101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:37.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.120+0000 7f7630483700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7628104130 0x7f76280ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:37.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.120+0000 7f7630483700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76280ffcf0 con 0x7f7628101800 2024-04-23T02:15:37.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.120+0000 7f7630483700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76280ffe30 con 0x7f7628104130 2024-04-23T02:15:37.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.120+0000 7f762e21f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7628101800 0x7f7628101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:37.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.120+0000 7f762e21f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7628101800 0x7f7628101160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42476/0 (socket says 172.21.15.71:42476) 2024-04-23T02:15:37.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.120+0000 7f762e21f700 1 -- 172.21.15.71:0/1226379656 learned_addr learned my addr 172.21.15.71:0/1226379656 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:15:37.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.120+0000 7f762da1e700 1 --2- 172.21.15.71:0/1226379656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7628104130 0x7f76280ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:37.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.120+0000 7f762e21f700 1 -- 172.21.15.71:0/1226379656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7628104130 msgr2=0x7f76280ff7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:37.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.120+0000 7f762e21f700 1 --2- 172.21.15.71:0/1226379656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7628104130 0x7f76280ff7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:37.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.120+0000 7f762e21f700 1 -- 172.21.15.71:0/1226379656 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76180095f0 con 0x7f7628101800 2024-04-23T02:15:37.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.120+0000 7f762e21f700 1 --2- 172.21.15.71:0/1226379656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7628101800 0x7f7628101160 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f762400e920 tx=0x7f762400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:37.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.121+0000 7f761f7fe700 1 -- 172.21.15.71:0/1226379656 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f762400c9d0 con 0x7f7628101800 2024-04-23T02:15:37.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.121+0000 7f761f7fe700 1 -- 172.21.15.71:0/1226379656 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7624010430 con 0x7f7628101800 2024-04-23T02:15:37.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.121+0000 7f7630483700 1 -- 172.21.15.71:0/1226379656 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7628100110 con 0x7f7628101800 2024-04-23T02:15:37.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.121+0000 7f761f7fe700 1 -- 172.21.15.71:0/1226379656 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7624018630 con 0x7f7628101800 2024-04-23T02:15:37.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.121+0000 7f7630483700 1 -- 172.21.15.71:0/1226379656 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762819e420 con 0x7f7628101800 2024-04-23T02:15:37.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:36 smithi192 ceph-mon[49662]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:15:37.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:36 smithi192 ceph-mon[49662]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:15:37.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:36 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:37.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:36 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:37.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.122+0000 7f7630483700 1 -- 172.21.15.71:0/1226379656 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76281904d0 con 0x7f7628101800 2024-04-23T02:15:37.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.122+0000 7f761f7fe700 1 -- 172.21.15.71:0/1226379656 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 31) v1 ==== 90944+0+0 (secure 0 0 0) 0x7f762400cb30 con 0x7f7628101800 2024-04-23T02:15:37.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.122+0000 7f761f7fe700 1 --2- 172.21.15.71:0/1226379656 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f761406ce20 0x7f761406f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:15:37.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.123+0000 7f761f7fe700 1 -- 172.21.15.71:0/1226379656 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7624014070 con 0x7f7628101800 2024-04-23T02:15:37.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.123+0000 7f762da1e700 1 --2- 172.21.15.71:0/1226379656 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f761406ce20 0x7f761406f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:15:37.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.123+0000 7f762da1e700 1 --2- 172.21.15.71:0/1226379656 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f761406ce20 0x7f761406f2e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7618009fd0 tx=0x7f761800b5a0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:15:37.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.127+0000 7f761f7fe700 1 -- 172.21.15.71:0/1226379656 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f76240529b0 con 0x7f7628101800 2024-04-23T02:15:37.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.365+0000 7f7630483700 1 -- 172.21.15.71:0/1226379656 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f76280611d0 con 0x7f761406ce20 2024-04-23T02:15:37.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.366+0000 7f761f7fe700 1 -- 172.21.15.71:0/1226379656 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f76280611d0 con 0x7f761406ce20 2024-04-23T02:15:37.370 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:15:37.370 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:15:37.370 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:15:37.370 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-23T02:15:37.370 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:15:37.371 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr" 2024-04-23T02:15:37.371 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:15:37.371 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "2/2 daemons upgraded", 2024-04-23T02:15:37.371 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading node-exporter daemons", 2024-04-23T02:15:37.371 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:15:37.371 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:15:37.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.368+0000 7f7630483700 1 -- 172.21.15.71:0/1226379656 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f761406ce20 msgr2=0x7f761406f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:37.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.368+0000 7f7630483700 1 --2- 172.21.15.71:0/1226379656 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f761406ce20 0x7f761406f2e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7618009fd0 tx=0x7f761800b5a0 comp rx=0 tx=0).stop 2024-04-23T02:15:37.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.368+0000 7f7630483700 1 -- 172.21.15.71:0/1226379656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7628101800 msgr2=0x7f7628101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:15:37.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.368+0000 7f7630483700 1 --2- 172.21.15.71:0/1226379656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7628101800 0x7f7628101160 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f762400e920 tx=0x7f762400ec30 comp rx=0 tx=0).stop 2024-04-23T02:15:37.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.368+0000 7f7630483700 1 -- 172.21.15.71:0/1226379656 shutdown_connections 2024-04-23T02:15:37.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.368+0000 7f7630483700 1 --2- 172.21.15.71:0/1226379656 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f761406ce20 0x7f761406f2e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:37.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.368+0000 7f7630483700 1 --2- 172.21.15.71:0/1226379656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7628101800 0x7f7628101160 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:37.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.368+0000 7f7630483700 1 --2- 172.21.15.71:0/1226379656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7628104130 0x7f76280ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:15:37.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.368+0000 7f7630483700 1 -- 172.21.15.71:0/1226379656 >> 172.21.15.71:0/1226379656 conn(0x7f76280fb190 msgr2=0x7f76280fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:15:37.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.368+0000 7f7630483700 1 -- 172.21.15.71:0/1226379656 shutdown_connections 2024-04-23T02:15:37.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:15:37.368+0000 7f7630483700 1 -- 172.21.15.71:0/1226379656 wait complete. 2024-04-23T02:15:38.088 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:37 smithi071 ceph-mon[39837]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:15:38.088 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:37 smithi071 ceph-mon[39837]: pgmap v34: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 31 KiB/s wr, 7 op/s 2024-04-23T02:15:38.089 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:37 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:38.089 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:37 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:38.089 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:37 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1858333401' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:38.089 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:37 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:38.089 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:37 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:38.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:37 smithi192 ceph-mon[49662]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:15:38.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:37 smithi192 ceph-mon[49662]: pgmap v34: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 31 KiB/s wr, 7 op/s 2024-04-23T02:15:38.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:37 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:38.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:37 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:38.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:37 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1858333401' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:38.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:37 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:38.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:37 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:38.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:38 smithi071 ceph-mon[39837]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:15:39.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:38 smithi192 ceph-mon[49662]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:15:40.084 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:39 smithi071 ceph-mon[39837]: pgmap v35: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 426 B/s wr, 3 op/s 2024-04-23T02:15:40.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:39 smithi192 ceph-mon[49662]: pgmap v35: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 426 B/s wr, 3 op/s 2024-04-23T02:15:41.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:41 smithi071 ceph-mon[39837]: pgmap v36: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-23T02:15:42.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:41 smithi192 ceph-mon[49662]: pgmap v36: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-23T02:15:43.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:43 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:43.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:43 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:43.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:43 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:15:43.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:43 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:15:43.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:43 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:43.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:43 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:15:43.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:43 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:43.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:43 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:43.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:43 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:43.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:43 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:43.496 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:43 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:43.496 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:43 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:43.496 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:43 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:43.496 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:43 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:43.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:43 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:43.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:43 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:43.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:43 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:15:43.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:43 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:15:43.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:43 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:15:43.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:43 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:15:43.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:43 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:43.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:43 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:43.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:43 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:43.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:43 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:43.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:43 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:43.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:43 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:43.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:43 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:43.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:43 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:15:44.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:44 smithi071 ceph-mon[39837]: pgmap v37: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:15:44.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:44 smithi192 ceph-mon[49662]: pgmap v37: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:15:45.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:45 smithi071 ceph-mon[39837]: Upgrade: Updating prometheus.smithi071 2024-04-23T02:15:45.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:45 smithi071 ceph-mon[39837]: Deploying daemon prometheus.smithi071 on smithi071 2024-04-23T02:15:45.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:45 smithi192 ceph-mon[49662]: Upgrade: Updating prometheus.smithi071 2024-04-23T02:15:45.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:45 smithi192 ceph-mon[49662]: Deploying daemon prometheus.smithi071 on smithi071 2024-04-23T02:15:46.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:46 smithi071 ceph-mon[39837]: pgmap v38: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:15:46.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:46 smithi192 ceph-mon[49662]: pgmap v38: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:15:48.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:48 smithi071 ceph-mon[39837]: pgmap v39: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:15:48.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:48 smithi192 ceph-mon[49662]: pgmap v39: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:15:50.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:50 smithi071 ceph-mon[39837]: pgmap v40: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:15:50.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:50 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:15:50.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:50 smithi192 ceph-mon[49662]: pgmap v40: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:15:50.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:50 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:15:52.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:52 smithi071 ceph-mon[39837]: pgmap v41: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:15:52.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:52 smithi192 ceph-mon[49662]: pgmap v41: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:15:54.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:54 smithi071 ceph-mon[39837]: pgmap v42: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:15:54.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:54 smithi192 ceph-mon[49662]: pgmap v42: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:15:56.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:56 smithi071 ceph-mon[39837]: pgmap v43: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:15:56.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:56 smithi192 ceph-mon[49662]: pgmap v43: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:15:58.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:15:58 smithi071 ceph-mon[39837]: pgmap v44: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:15:58.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:15:58 smithi192 ceph-mon[49662]: pgmap v44: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:00.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:00 smithi071 ceph-mon[39837]: pgmap v45: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:00.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:00 smithi192 ceph-mon[49662]: pgmap v45: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:01.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:01 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:01.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:01 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:01.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:01 smithi071 ceph-mon[39837]: pgmap v46: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:01.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:01 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:16:01.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:01 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:01.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:01 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:02.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:01 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:02.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:01 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:02.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:01 smithi192 ceph-mon[49662]: pgmap v46: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:02.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:01 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:16:02.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:01 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:02.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:01 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:03.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:03 smithi192 ceph-mon[49662]: pgmap v47: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:03.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:03 smithi071 ceph-mon[39837]: pgmap v47: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:04.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:04 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:16:05.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:04 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:16:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:05 smithi071 ceph-mon[39837]: pgmap v48: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:06.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:05 smithi192 ceph-mon[49662]: pgmap v48: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:07 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:07 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:07.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:07 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:07.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:07 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:07.426 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:07 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:07.426 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:07 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:07.426 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:07 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:07.427 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:07 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:07.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.479+0000 7f76c2c28700 1 -- 172.21.15.71:0/233832907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76bc101ea0 msgr2=0x7f76bc102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.479+0000 7f76c2c28700 1 --2- 172.21.15.71:0/233832907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76bc101ea0 0x7f76bc102300 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009910 tx=0x7f76a8009c20 comp rx=0 tx=0).stop 2024-04-23T02:16:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.479+0000 7f76c2c28700 1 -- 172.21.15.71:0/233832907 shutdown_connections 2024-04-23T02:16:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.479+0000 7f76c2c28700 1 --2- 172.21.15.71:0/233832907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76bc101ea0 0x7f76bc102300 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.479+0000 7f76c2c28700 1 --2- 172.21.15.71:0/233832907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76bc100ca0 0x7f76bc1010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.479+0000 7f76c2c28700 1 -- 172.21.15.71:0/233832907 >> 172.21.15.71:0/233832907 conn(0x7f76bc0fc260 msgr2=0x7f76bc0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.479+0000 7f76c2c28700 1 -- 172.21.15.71:0/233832907 shutdown_connections 2024-04-23T02:16:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.479+0000 7f76c2c28700 1 -- 172.21.15.71:0/233832907 wait complete. 2024-04-23T02:16:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.480+0000 7f76c2c28700 1 Processor -- start 2024-04-23T02:16:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.480+0000 7f76c2c28700 1 -- start start 2024-04-23T02:16:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.480+0000 7f76c2c28700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76bc100ca0 0x7f76bc1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.480+0000 7f76c2c28700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76bc101ea0 0x7f76bc198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.480+0000 7f76c2c28700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc199230 con 0x7f76bc100ca0 2024-04-23T02:16:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.480+0000 7f76c2c28700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc199370 con 0x7f76bc101ea0 2024-04-23T02:16:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.480+0000 7f76c09c4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76bc100ca0 0x7f76bc1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:07.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.480+0000 7f76c09c4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76bc100ca0 0x7f76bc1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44620/0 (socket says 172.21.15.71:44620) 2024-04-23T02:16:07.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.480+0000 7f76c09c4700 1 -- 172.21.15.71:0/452404055 learned_addr learned my addr 172.21.15.71:0/452404055 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:16:07.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.480+0000 7f76bbfff700 1 --2- 172.21.15.71:0/452404055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76bc101ea0 0x7f76bc198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:07.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.480+0000 7f76c09c4700 1 -- 172.21.15.71:0/452404055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76bc101ea0 msgr2=0x7f76bc198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:07.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.480+0000 7f76c09c4700 1 --2- 172.21.15.71:0/452404055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76bc101ea0 0x7f76bc198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:07.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.480+0000 7f76c09c4700 1 -- 172.21.15.71:0/452404055 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a80095f0 con 0x7f76bc100ca0 2024-04-23T02:16:07.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.481+0000 7f76c09c4700 1 --2- 172.21.15.71:0/452404055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76bc100ca0 0x7f76bc1986d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f76b000b480 tx=0x7f76b000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:07.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.481+0000 7f76b9ffb700 1 -- 172.21.15.71:0/452404055 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76b00118c0 con 0x7f76bc100ca0 2024-04-23T02:16:07.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.481+0000 7f76c2c28700 1 -- 172.21.15.71:0/452404055 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76bc19de20 con 0x7f76bc100ca0 2024-04-23T02:16:07.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.481+0000 7f76b9ffb700 1 -- 172.21.15.71:0/452404055 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f76b0010380 con 0x7f76bc100ca0 2024-04-23T02:16:07.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.481+0000 7f76b9ffb700 1 -- 172.21.15.71:0/452404055 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76b000f510 con 0x7f76bc100ca0 2024-04-23T02:16:07.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.481+0000 7f76c2c28700 1 -- 172.21.15.71:0/452404055 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76bc19e2e0 con 0x7f76bc100ca0 2024-04-23T02:16:07.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.482+0000 7f76c2c28700 1 -- 172.21.15.71:0/452404055 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76bc105f80 con 0x7f76bc100ca0 2024-04-23T02:16:07.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.482+0000 7f76b9ffb700 1 -- 172.21.15.71:0/452404055 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f76b00104f0 con 0x7f76bc100ca0 2024-04-23T02:16:07.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.483+0000 7f76b9ffb700 1 --2- 172.21.15.71:0/452404055 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f76ac06cce0 0x7f76ac06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:07.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.483+0000 7f76b9ffb700 1 -- 172.21.15.71:0/452404055 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f76b008c080 con 0x7f76bc100ca0 2024-04-23T02:16:07.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.483+0000 7f76bbfff700 1 --2- 172.21.15.71:0/452404055 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f76ac06cce0 0x7f76ac06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:07.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.483+0000 7f76bbfff700 1 --2- 172.21.15.71:0/452404055 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f76ac06cce0 0x7f76ac06f1a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f76a800b5c0 tx=0x7f76a8005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:07.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.488+0000 7f76b9ffb700 1 -- 172.21.15.71:0/452404055 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f76b0059cb0 con 0x7f76bc100ca0 2024-04-23T02:16:07.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.705+0000 7f76c2c28700 1 -- 172.21.15.71:0/452404055 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f76bc0611d0 con 0x7f76ac06cce0 2024-04-23T02:16:07.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.706+0000 7f76b9ffb700 1 -- 172.21.15.71:0/452404055 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f76bc0611d0 con 0x7f76ac06cce0 2024-04-23T02:16:07.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.708+0000 7f76c2c28700 1 -- 172.21.15.71:0/452404055 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f76ac06cce0 msgr2=0x7f76ac06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:07.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.708+0000 7f76c2c28700 1 --2- 172.21.15.71:0/452404055 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f76ac06cce0 0x7f76ac06f1a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f76a800b5c0 tx=0x7f76a8005960 comp rx=0 tx=0).stop 2024-04-23T02:16:07.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.708+0000 7f76c2c28700 1 -- 172.21.15.71:0/452404055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76bc100ca0 msgr2=0x7f76bc1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:07.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.708+0000 7f76c2c28700 1 --2- 172.21.15.71:0/452404055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76bc100ca0 0x7f76bc1986d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f76b000b480 tx=0x7f76b000b840 comp rx=0 tx=0).stop 2024-04-23T02:16:07.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.708+0000 7f76c2c28700 1 -- 172.21.15.71:0/452404055 shutdown_connections 2024-04-23T02:16:07.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.708+0000 7f76c2c28700 1 --2- 172.21.15.71:0/452404055 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f76ac06cce0 0x7f76ac06f1a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:07.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.708+0000 7f76c2c28700 1 --2- 172.21.15.71:0/452404055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76bc100ca0 0x7f76bc1986d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:07.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.708+0000 7f76c2c28700 1 --2- 172.21.15.71:0/452404055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76bc101ea0 0x7f76bc198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:07.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.708+0000 7f76c2c28700 1 -- 172.21.15.71:0/452404055 >> 172.21.15.71:0/452404055 conn(0x7f76bc0fc260 msgr2=0x7f76bc1030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:07.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.709+0000 7f76c2c28700 1 -- 172.21.15.71:0/452404055 shutdown_connections 2024-04-23T02:16:07.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.709+0000 7f76c2c28700 1 -- 172.21.15.71:0/452404055 wait complete. 2024-04-23T02:16:07.724 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:16:07.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.817+0000 7fd1f985f700 1 -- 172.21.15.71:0/3999864779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1f4100cb0 msgr2=0x7fd1f41010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:07.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.817+0000 7fd1f985f700 1 --2- 172.21.15.71:0/3999864779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1f4100cb0 0x7fd1f41010d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009910 tx=0x7fd1e8009c20 comp rx=0 tx=0).stop 2024-04-23T02:16:07.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.817+0000 7fd1f985f700 1 -- 172.21.15.71:0/3999864779 shutdown_connections 2024-04-23T02:16:07.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.817+0000 7fd1f985f700 1 --2- 172.21.15.71:0/3999864779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f4101eb0 0x7fd1f4102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:07.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.817+0000 7fd1f985f700 1 --2- 172.21.15.71:0/3999864779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1f4100cb0 0x7fd1f41010d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:07.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.817+0000 7fd1f985f700 1 -- 172.21.15.71:0/3999864779 >> 172.21.15.71:0/3999864779 conn(0x7fd1f40fc230 msgr2=0x7fd1f40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:07.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.817+0000 7fd1f985f700 1 -- 172.21.15.71:0/3999864779 shutdown_connections 2024-04-23T02:16:07.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.817+0000 7fd1f985f700 1 -- 172.21.15.71:0/3999864779 wait complete. 2024-04-23T02:16:07.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.817+0000 7fd1f985f700 1 Processor -- start 2024-04-23T02:16:07.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.818+0000 7fd1f985f700 1 -- start start 2024-04-23T02:16:07.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.818+0000 7fd1f985f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1f4100cb0 0x7fd1f41963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:07.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.818+0000 7fd1f985f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f4101eb0 0x7fd1f4196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:07.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.818+0000 7fd1f985f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f4196f40 con 0x7fd1f4100cb0 2024-04-23T02:16:07.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.818+0000 7fd1f985f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f4197080 con 0x7fd1f4101eb0 2024-04-23T02:16:07.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.818+0000 7fd1f2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1f4100cb0 0x7fd1f41963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:07.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.818+0000 7fd1f2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1f4100cb0 0x7fd1f41963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44636/0 (socket says 172.21.15.71:44636) 2024-04-23T02:16:07.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.818+0000 7fd1f2ffd700 1 -- 172.21.15.71:0/2255800773 learned_addr learned my addr 172.21.15.71:0/2255800773 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:16:07.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.818+0000 7fd1f27fc700 1 --2- 172.21.15.71:0/2255800773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f4101eb0 0x7fd1f4196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:07.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.818+0000 7fd1f2ffd700 1 -- 172.21.15.71:0/2255800773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f4101eb0 msgr2=0x7fd1f4196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:07.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.818+0000 7fd1f2ffd700 1 --2- 172.21.15.71:0/2255800773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f4101eb0 0x7fd1f4196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:07.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.818+0000 7fd1f2ffd700 1 -- 172.21.15.71:0/2255800773 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e80095f0 con 0x7fd1f4100cb0 2024-04-23T02:16:07.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.818+0000 7fd1f27fc700 1 --2- 172.21.15.71:0/2255800773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f4101eb0 0x7fd1f4196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:16:07.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.818+0000 7fd1f2ffd700 1 --2- 172.21.15.71:0/2255800773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1f4100cb0 0x7fd1f41963e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800bda0 tx=0x7fd1e800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:07.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.819+0000 7fd1f885d700 1 -- 172.21.15.71:0/2255800773 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1e801d070 con 0x7fd1f4100cb0 2024-04-23T02:16:07.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.819+0000 7fd1f985f700 1 -- 172.21.15.71:0/2255800773 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f419bad0 con 0x7fd1f4100cb0 2024-04-23T02:16:07.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.819+0000 7fd1f885d700 1 -- 172.21.15.71:0/2255800773 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd1e800f460 con 0x7fd1f4100cb0 2024-04-23T02:16:07.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.819+0000 7fd1f885d700 1 -- 172.21.15.71:0/2255800773 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1e8021640 con 0x7fd1f4100cb0 2024-04-23T02:16:07.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.819+0000 7fd1f985f700 1 -- 172.21.15.71:0/2255800773 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f419bfc0 con 0x7fd1f4100cb0 2024-04-23T02:16:07.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.820+0000 7fd1f985f700 1 -- 172.21.15.71:0/2255800773 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f41096d0 con 0x7fd1f4100cb0 2024-04-23T02:16:07.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.820+0000 7fd1f885d700 1 -- 172.21.15.71:0/2255800773 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd1e800f9b0 con 0x7fd1f4100cb0 2024-04-23T02:16:07.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.820+0000 7fd1f885d700 1 --2- 172.21.15.71:0/2255800773 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd1dc06cce0 0x7fd1dc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:07.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.821+0000 7fd1f885d700 1 -- 172.21.15.71:0/2255800773 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd1e808d7d0 con 0x7fd1f4100cb0 2024-04-23T02:16:07.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.821+0000 7fd1f27fc700 1 --2- 172.21.15.71:0/2255800773 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd1dc06cce0 0x7fd1dc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:07.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.821+0000 7fd1f27fc700 1 --2- 172.21.15.71:0/2255800773 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd1dc06cce0 0x7fd1dc06f1a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4005950 tx=0x7fd1e40058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:07.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:07.825+0000 7fd1f885d700 1 -- 172.21.15.71:0/2255800773 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fd1e805b380 con 0x7fd1f4100cb0 2024-04-23T02:16:08.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.035+0000 7fd1f985f700 1 -- 172.21.15.71:0/2255800773 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1f419c630 con 0x7fd1dc06cce0 2024-04-23T02:16:08.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.037+0000 7fd1f885d700 1 -- 172.21.15.71:0/2255800773 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7fd1f419c630 con 0x7fd1dc06cce0 2024-04-23T02:16:08.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.039+0000 7fd1f985f700 1 -- 172.21.15.71:0/2255800773 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd1dc06cce0 msgr2=0x7fd1dc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:08.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.039+0000 7fd1f985f700 1 --2- 172.21.15.71:0/2255800773 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd1dc06cce0 0x7fd1dc06f1a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4005950 tx=0x7fd1e40058e0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.039+0000 7fd1f985f700 1 -- 172.21.15.71:0/2255800773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1f4100cb0 msgr2=0x7fd1f41963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:08.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.039+0000 7fd1f985f700 1 --2- 172.21.15.71:0/2255800773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1f4100cb0 0x7fd1f41963e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800bda0 tx=0x7fd1e800be80 comp rx=0 tx=0).stop 2024-04-23T02:16:08.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.039+0000 7fd1f985f700 1 -- 172.21.15.71:0/2255800773 shutdown_connections 2024-04-23T02:16:08.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.039+0000 7fd1f985f700 1 --2- 172.21.15.71:0/2255800773 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd1dc06cce0 0x7fd1dc06f1a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.039+0000 7fd1f985f700 1 --2- 172.21.15.71:0/2255800773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1f4100cb0 0x7fd1f41963e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.039+0000 7fd1f985f700 1 --2- 172.21.15.71:0/2255800773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1f4101eb0 0x7fd1f4196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.039+0000 7fd1f985f700 1 -- 172.21.15.71:0/2255800773 >> 172.21.15.71:0/2255800773 conn(0x7fd1f40fc230 msgr2=0x7fd1f41050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:08.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.039+0000 7fd1f985f700 1 -- 172.21.15.71:0/2255800773 shutdown_connections 2024-04-23T02:16:08.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.039+0000 7fd1f985f700 1 -- 172.21.15.71:0/2255800773 wait complete. 2024-04-23T02:16:08.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.145+0000 7ff2ba023700 1 -- 172.21.15.71:0/3607866348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2b40ffc10 msgr2=0x7ff2b4100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:08.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.145+0000 7ff2ba023700 1 --2- 172.21.15.71:0/3607866348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2b40ffc10 0x7ff2b4100030 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009910 tx=0x7ff2a4009c20 comp rx=0 tx=0).stop 2024-04-23T02:16:08.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.145+0000 7ff2ba023700 1 -- 172.21.15.71:0/3607866348 shutdown_connections 2024-04-23T02:16:08.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.145+0000 7ff2ba023700 1 --2- 172.21.15.71:0/3607866348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b4100e10 0x7ff2b4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.145+0000 7ff2ba023700 1 --2- 172.21.15.71:0/3607866348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2b40ffc10 0x7ff2b4100030 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.145+0000 7ff2ba023700 1 -- 172.21.15.71:0/3607866348 >> 172.21.15.71:0/3607866348 conn(0x7ff2b40fb190 msgr2=0x7ff2b40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:08.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.145+0000 7ff2ba023700 1 -- 172.21.15.71:0/3607866348 shutdown_connections 2024-04-23T02:16:08.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.145+0000 7ff2ba023700 1 -- 172.21.15.71:0/3607866348 wait complete. 2024-04-23T02:16:08.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.146+0000 7ff2ba023700 1 Processor -- start 2024-04-23T02:16:08.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.146+0000 7ff2ba023700 1 -- start start 2024-04-23T02:16:08.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.146+0000 7ff2ba023700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b40ffc10 0x7ff2b4196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:08.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.146+0000 7ff2ba023700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2b4100e10 0x7ff2b4196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:08.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.146+0000 7ff2ba023700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b4196ee0 con 0x7ff2b4100e10 2024-04-23T02:16:08.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.146+0000 7ff2ba023700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b4197020 con 0x7ff2b40ffc10 2024-04-23T02:16:08.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.146+0000 7ff2b2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2b4100e10 0x7ff2b4196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:08.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.146+0000 7ff2b2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2b4100e10 0x7ff2b4196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44654/0 (socket says 172.21.15.71:44654) 2024-04-23T02:16:08.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.146+0000 7ff2b2ffd700 1 -- 172.21.15.71:0/11809720 learned_addr learned my addr 172.21.15.71:0/11809720 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:16:08.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.146+0000 7ff2b2ffd700 1 -- 172.21.15.71:0/11809720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b40ffc10 msgr2=0x7ff2b4196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:08.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.146+0000 7ff2b2ffd700 1 --2- 172.21.15.71:0/11809720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b40ffc10 0x7ff2b4196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.146+0000 7ff2b2ffd700 1 -- 172.21.15.71:0/11809720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a40095f0 con 0x7ff2b4100e10 2024-04-23T02:16:08.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.146+0000 7ff2b2ffd700 1 --2- 172.21.15.71:0/11809720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2b4100e10 0x7ff2b4196950 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800c930 tx=0x7ff2a800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:08.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.147+0000 7ff2b0ff9700 1 -- 172.21.15.71:0/11809720 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2a8007ad0 con 0x7ff2b4100e10 2024-04-23T02:16:08.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.147+0000 7ff2ba023700 1 -- 172.21.15.71:0/11809720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b419bae0 con 0x7ff2b4100e10 2024-04-23T02:16:08.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.147+0000 7ff2b0ff9700 1 -- 172.21.15.71:0/11809720 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ff2a8007c30 con 0x7ff2b4100e10 2024-04-23T02:16:08.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.147+0000 7ff2b0ff9700 1 -- 172.21.15.71:0/11809720 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2a8018630 con 0x7ff2b4100e10 2024-04-23T02:16:08.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.147+0000 7ff2ba023700 1 -- 172.21.15.71:0/11809720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b419c000 con 0x7ff2b4100e10 2024-04-23T02:16:08.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.148+0000 7ff2ba023700 1 -- 172.21.15.71:0/11809720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b41085c0 con 0x7ff2b4100e10 2024-04-23T02:16:08.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.148+0000 7ff2b0ff9700 1 -- 172.21.15.71:0/11809720 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7ff2a800f450 con 0x7ff2b4100e10 2024-04-23T02:16:08.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.149+0000 7ff2b0ff9700 1 --2- 172.21.15.71:0/11809720 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff29c06cce0 0x7ff29c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:08.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.149+0000 7ff2b0ff9700 1 -- 172.21.15.71:0/11809720 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff2a808c6a0 con 0x7ff2b4100e10 2024-04-23T02:16:08.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.149+0000 7ff2b37fe700 1 --2- 172.21.15.71:0/11809720 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff29c06cce0 0x7ff29c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:08.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.149+0000 7ff2b37fe700 1 --2- 172.21.15.71:0/11809720 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff29c06cce0 0x7ff29c06f1a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400b5c0 tx=0x7ff2a4005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:08.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.153+0000 7ff2b0ff9700 1 -- 172.21.15.71:0/11809720 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7ff2a805a300 con 0x7ff2b4100e10 2024-04-23T02:16:08.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.359+0000 7ff2ba023700 1 -- 172.21.15.71:0/11809720 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2b4068c90 con 0x7ff29c06cce0 2024-04-23T02:16:08.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.370+0000 7ff2b0ff9700 1 -- 172.21.15.71:0/11809720 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7ff2b4068c90 con 0x7ff29c06cce0 2024-04-23T02:16:08.375 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:16:08.375 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (9m) 2s ago 11m 26.5M - 0.25.0 c8568f914cd2 8261e17319fe 2024-04-23T02:16:08.375 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (11m) 2s ago 11m 7931k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:16:08.375 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (10m) 36s ago 10m 7796k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:16:08.376 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (11m) 2s ago 11m 6656k - 18.2.1 5be31c24972a 258512dcbc00 2024-04-23T02:16:08.376 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (10m) 36s ago 10m 6656k - 18.2.1 5be31c24972a feea2c9896a6 2024-04-23T02:16:08.376 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (9m) 2s ago 10m 87.3M - 9.4.7 954c08fa6188 48ea3e02b806 2024-04-23T02:16:08.376 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (5m) 2s ago 5m 64.7M - 18.2.1 5be31c24972a a515de2a841c 2024-04-23T02:16:08.376 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (5m) 2s ago 5m 135M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:16:08.376 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (5m) 36s ago 5m 68.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:16:08.376 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (5m) 36s ago 5m 125M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:16:08.376 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (106s) 2s ago 12m 495M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:16:08.376 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (61s) 36s ago 10m 449M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:16:08.376 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (12m) 2s ago 12m 61.4M 2048M 18.2.1 5be31c24972a ff40abe63fd9 2024-04-23T02:16:08.376 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (9m) 36s ago 9m 46.7M 2048M 18.2.1 5be31c24972a 9c03e90acc95 2024-04-23T02:16:08.376 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (46s) 2s ago 11m 9508k - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:16:08.376 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (39s) 36s ago 10m 3510k - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:16:08.377 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (9m) 2s ago 9m 196M 1227M 18.2.1 5be31c24972a d3d5f2fa7267 2024-04-23T02:16:08.377 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (8m) 2s ago 8m 208M 1227M 18.2.1 5be31c24972a 3992157ff0fd 2024-04-23T02:16:08.377 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (8m) 2s ago 8m 163M 1227M 18.2.1 5be31c24972a 12380b6a15a2 2024-04-23T02:16:08.377 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (7m) 36s ago 7m 240M 2251M 18.2.1 5be31c24972a d5f587ee3a22 2024-04-23T02:16:08.377 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (7m) 36s ago 7m 217M 2251M 18.2.1 5be31c24972a 958d046005f0 2024-04-23T02:16:08.377 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (6m) 36s ago 6m 195M 2251M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:16:08.377 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (8s) 2s ago 10m 38.9M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:16:08.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:08 smithi192 ceph-mon[49662]: pgmap v49: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-23T02:16:08.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:08 smithi192 ceph-mon[49662]: mgrmap e32: smithi071.ukdjvn(active, since 92s), standbys: smithi192.qgecnv 2024-04-23T02:16:08.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.372+0000 7ff2ba023700 1 -- 172.21.15.71:0/11809720 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff29c06cce0 msgr2=0x7ff29c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:08.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.372+0000 7ff2ba023700 1 --2- 172.21.15.71:0/11809720 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff29c06cce0 0x7ff29c06f1a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400b5c0 tx=0x7ff2a4005fd0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.372+0000 7ff2ba023700 1 -- 172.21.15.71:0/11809720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2b4100e10 msgr2=0x7ff2b4196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:08.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.372+0000 7ff2ba023700 1 --2- 172.21.15.71:0/11809720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2b4100e10 0x7ff2b4196950 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800c930 tx=0x7ff2a800ccf0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.372+0000 7ff2ba023700 1 -- 172.21.15.71:0/11809720 shutdown_connections 2024-04-23T02:16:08.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.372+0000 7ff2ba023700 1 --2- 172.21.15.71:0/11809720 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff29c06cce0 0x7ff29c06f1a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.372+0000 7ff2ba023700 1 --2- 172.21.15.71:0/11809720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2b40ffc10 0x7ff2b4196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.372+0000 7ff2ba023700 1 --2- 172.21.15.71:0/11809720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2b4100e10 0x7ff2b4196950 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.372+0000 7ff2ba023700 1 -- 172.21.15.71:0/11809720 >> 172.21.15.71:0/11809720 conn(0x7ff2b40fb190 msgr2=0x7ff2b4068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:08.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.372+0000 7ff2ba023700 1 -- 172.21.15.71:0/11809720 shutdown_connections 2024-04-23T02:16:08.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.372+0000 7ff2ba023700 1 -- 172.21.15.71:0/11809720 wait complete. 2024-04-23T02:16:08.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.475+0000 7f38a36f9700 1 -- 172.21.15.71:0/1981336018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f389c101ea0 msgr2=0x7f389c102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:08.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.475+0000 7f38a36f9700 1 --2- 172.21.15.71:0/1981336018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f389c101ea0 0x7f389c102300 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f388c009910 tx=0x7f388c009c20 comp rx=0 tx=0).stop 2024-04-23T02:16:08.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.475+0000 7f38a36f9700 1 -- 172.21.15.71:0/1981336018 shutdown_connections 2024-04-23T02:16:08.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.475+0000 7f38a36f9700 1 --2- 172.21.15.71:0/1981336018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f389c101ea0 0x7f389c102300 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.475+0000 7f38a36f9700 1 --2- 172.21.15.71:0/1981336018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f389c100ca0 0x7f389c1010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.475+0000 7f38a36f9700 1 -- 172.21.15.71:0/1981336018 >> 172.21.15.71:0/1981336018 conn(0x7f389c0fc260 msgr2=0x7f389c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:08.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.475+0000 7f38a36f9700 1 -- 172.21.15.71:0/1981336018 shutdown_connections 2024-04-23T02:16:08.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.475+0000 7f38a36f9700 1 -- 172.21.15.71:0/1981336018 wait complete. 2024-04-23T02:16:08.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.476+0000 7f38a36f9700 1 Processor -- start 2024-04-23T02:16:08.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.476+0000 7f38a36f9700 1 -- start start 2024-04-23T02:16:08.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.476+0000 7f38a36f9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f389c100ca0 0x7f389c196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:08.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.476+0000 7f38a36f9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f389c101ea0 0x7f389c196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:08.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.476+0000 7f38a36f9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c196f60 con 0x7f389c101ea0 2024-04-23T02:16:08.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.476+0000 7f38a36f9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c1970a0 con 0x7f389c100ca0 2024-04-23T02:16:08.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.476+0000 7f38a0c94700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f389c101ea0 0x7f389c196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:08.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.476+0000 7f38a0c94700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f389c101ea0 0x7f389c196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44664/0 (socket says 172.21.15.71:44664) 2024-04-23T02:16:08.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.476+0000 7f38a0c94700 1 -- 172.21.15.71:0/2079358811 learned_addr learned my addr 172.21.15.71:0/2079358811 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:16:08.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.476+0000 7f38a1495700 1 --2- 172.21.15.71:0/2079358811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f389c100ca0 0x7f389c196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:08.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.476+0000 7f38a0c94700 1 -- 172.21.15.71:0/2079358811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f389c100ca0 msgr2=0x7f389c196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:08.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.476+0000 7f38a0c94700 1 --2- 172.21.15.71:0/2079358811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f389c100ca0 0x7f389c196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.476+0000 7f38a0c94700 1 -- 172.21.15.71:0/2079358811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c0095f0 con 0x7f389c101ea0 2024-04-23T02:16:08.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.477+0000 7f38a0c94700 1 --2- 172.21.15.71:0/2079358811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f389c101ea0 0x7f389c196940 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f388c006010 tx=0x7f388c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:08.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.477+0000 7f38927fc700 1 -- 172.21.15.71:0/2079358811 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f388c01d070 con 0x7f389c101ea0 2024-04-23T02:16:08.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.477+0000 7f38927fc700 1 -- 172.21.15.71:0/2079358811 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f388c00f460 con 0x7f389c101ea0 2024-04-23T02:16:08.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.477+0000 7f38a36f9700 1 -- 172.21.15.71:0/2079358811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f389c19baf0 con 0x7f389c101ea0 2024-04-23T02:16:08.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.477+0000 7f38927fc700 1 -- 172.21.15.71:0/2079358811 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f388c021640 con 0x7f389c101ea0 2024-04-23T02:16:08.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.477+0000 7f38a36f9700 1 -- 172.21.15.71:0/2079358811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389c19bfe0 con 0x7f389c101ea0 2024-04-23T02:16:08.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.478+0000 7f38a36f9700 1 -- 172.21.15.71:0/2079358811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f389c1096c0 con 0x7f389c101ea0 2024-04-23T02:16:08.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.478+0000 7f38927fc700 1 -- 172.21.15.71:0/2079358811 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f388c00f9b0 con 0x7f389c101ea0 2024-04-23T02:16:08.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.478+0000 7f38927fc700 1 --2- 172.21.15.71:0/2079358811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f388806cce0 0x7f388806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:08.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.479+0000 7f38927fc700 1 -- 172.21.15.71:0/2079358811 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f388c08d720 con 0x7f389c101ea0 2024-04-23T02:16:08.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.479+0000 7f38a1495700 1 --2- 172.21.15.71:0/2079358811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f388806cce0 0x7f388806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:08.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.479+0000 7f38a1495700 1 --2- 172.21.15.71:0/2079358811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f388806cce0 0x7f388806f1a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f389c101d00 tx=0x7f389800b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:08.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.483+0000 7f38927fc700 1 -- 172.21.15.71:0/2079358811 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f388c05b2d0 con 0x7f389c101ea0 2024-04-23T02:16:08.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:08 smithi071 ceph-mon[39837]: pgmap v49: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-23T02:16:08.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:08 smithi071 ceph-mon[39837]: mgrmap e32: smithi071.ukdjvn(active, since 92s), standbys: smithi192.qgecnv 2024-04-23T02:16:08.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.783+0000 7f38a36f9700 1 -- 172.21.15.71:0/2079358811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f389c109880 con 0x7f389c101ea0 2024-04-23T02:16:08.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.783+0000 7f38927fc700 1 -- 172.21.15.71:0/2079358811 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f388c05ae60 con 0x7f389c101ea0 2024-04-23T02:16:08.789 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:16:08.789 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:16:08.789 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-23T02:16:08.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:16:08.789 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:16:08.789 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:16:08.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:16:08.789 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:16:08.789 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-23T02:16:08.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:16:08.790 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:16:08.790 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-23T02:16:08.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:16:08.790 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:16:08.790 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-23T02:16:08.790 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:16:08.790 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:16:08.790 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:16:08.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.786+0000 7f38a36f9700 1 -- 172.21.15.71:0/2079358811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f388806cce0 msgr2=0x7f388806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:08.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.786+0000 7f38a36f9700 1 --2- 172.21.15.71:0/2079358811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f388806cce0 0x7f388806f1a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f389c101d00 tx=0x7f389800b410 comp rx=0 tx=0).stop 2024-04-23T02:16:08.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.786+0000 7f38a36f9700 1 -- 172.21.15.71:0/2079358811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f389c101ea0 msgr2=0x7f389c196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:08.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.786+0000 7f38a36f9700 1 --2- 172.21.15.71:0/2079358811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f389c101ea0 0x7f389c196940 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f388c006010 tx=0x7f388c00bd70 comp rx=0 tx=0).stop 2024-04-23T02:16:08.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.786+0000 7f38a36f9700 1 -- 172.21.15.71:0/2079358811 shutdown_connections 2024-04-23T02:16:08.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.786+0000 7f38a36f9700 1 --2- 172.21.15.71:0/2079358811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f388806cce0 0x7f388806f1a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.786+0000 7f38a36f9700 1 --2- 172.21.15.71:0/2079358811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f389c100ca0 0x7f389c196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.786+0000 7f38a36f9700 1 --2- 172.21.15.71:0/2079358811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f389c101ea0 0x7f389c196940 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.786+0000 7f38a36f9700 1 -- 172.21.15.71:0/2079358811 >> 172.21.15.71:0/2079358811 conn(0x7f389c0fc260 msgr2=0x7f389c1050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:08.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.786+0000 7f38a36f9700 1 -- 172.21.15.71:0/2079358811 shutdown_connections 2024-04-23T02:16:08.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.786+0000 7f38a36f9700 1 -- 172.21.15.71:0/2079358811 wait complete. 2024-04-23T02:16:08.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.891+0000 7f58d5bf5700 1 -- 172.21.15.71:0/60692101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58d0100cb0 msgr2=0x7f58d01010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:08.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.891+0000 7f58d5bf5700 1 --2- 172.21.15.71:0/60692101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58d0100cb0 0x7f58d01010d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009910 tx=0x7f58c4009c20 comp rx=0 tx=0).stop 2024-04-23T02:16:08.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.891+0000 7f58d5bf5700 1 -- 172.21.15.71:0/60692101 shutdown_connections 2024-04-23T02:16:08.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.891+0000 7f58d5bf5700 1 --2- 172.21.15.71:0/60692101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58d0101eb0 0x7f58d0102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.891+0000 7f58d5bf5700 1 --2- 172.21.15.71:0/60692101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58d0100cb0 0x7f58d01010d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.891+0000 7f58d5bf5700 1 -- 172.21.15.71:0/60692101 >> 172.21.15.71:0/60692101 conn(0x7f58d00fc230 msgr2=0x7f58d00fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.892+0000 7f58d5bf5700 1 -- 172.21.15.71:0/60692101 shutdown_connections 2024-04-23T02:16:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.892+0000 7f58d5bf5700 1 -- 172.21.15.71:0/60692101 wait complete. 2024-04-23T02:16:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.892+0000 7f58d5bf5700 1 Processor -- start 2024-04-23T02:16:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.892+0000 7f58d5bf5700 1 -- start start 2024-04-23T02:16:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.892+0000 7f58d5bf5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58d0100cb0 0x7f58d01964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.892+0000 7f58d5bf5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58d0101eb0 0x7f58d0196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.892+0000 7f58d5bf5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d0197040 con 0x7f58d0100cb0 2024-04-23T02:16:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.892+0000 7f58d5bf5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d0197180 con 0x7f58d0101eb0 2024-04-23T02:16:08.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.893+0000 7f58cf7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58d0100cb0 0x7f58d01964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:08.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.893+0000 7f58cf7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58d0100cb0 0x7f58d01964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44680/0 (socket says 172.21.15.71:44680) 2024-04-23T02:16:08.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.893+0000 7f58cf7fe700 1 -- 172.21.15.71:0/2714381819 learned_addr learned my addr 172.21.15.71:0/2714381819 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:16:08.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.893+0000 7f58ceffd700 1 --2- 172.21.15.71:0/2714381819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58d0101eb0 0x7f58d0196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:08.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.893+0000 7f58cf7fe700 1 -- 172.21.15.71:0/2714381819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58d0101eb0 msgr2=0x7f58d0196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:08.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.893+0000 7f58cf7fe700 1 --2- 172.21.15.71:0/2714381819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58d0101eb0 0x7f58d0196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:08.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.893+0000 7f58cf7fe700 1 -- 172.21.15.71:0/2714381819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c40095f0 con 0x7f58d0100cb0 2024-04-23T02:16:08.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.893+0000 7f58ceffd700 1 --2- 172.21.15.71:0/2714381819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58d0101eb0 0x7f58d0196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:16:08.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.893+0000 7f58cf7fe700 1 --2- 172.21.15.71:0/2714381819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58d0100cb0 0x7f58d01964e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f58c400bda0 tx=0x7f58c400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:08.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.893+0000 7f58ccff9700 1 -- 172.21.15.71:0/2714381819 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58c401d070 con 0x7f58d0100cb0 2024-04-23T02:16:08.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.893+0000 7f58ccff9700 1 -- 172.21.15.71:0/2714381819 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f58c4022470 con 0x7f58d0100cb0 2024-04-23T02:16:08.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.893+0000 7f58d5bf5700 1 -- 172.21.15.71:0/2714381819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d019bbd0 con 0x7f58d0100cb0 2024-04-23T02:16:08.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.893+0000 7f58ccff9700 1 -- 172.21.15.71:0/2714381819 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58c400f690 con 0x7f58d0100cb0 2024-04-23T02:16:08.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.893+0000 7f58d5bf5700 1 -- 172.21.15.71:0/2714381819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d019c0c0 con 0x7f58d0100cb0 2024-04-23T02:16:08.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.894+0000 7f58d5bf5700 1 -- 172.21.15.71:0/2714381819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d0109750 con 0x7f58d0100cb0 2024-04-23T02:16:08.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.895+0000 7f58ccff9700 1 -- 172.21.15.71:0/2714381819 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f58c40229c0 con 0x7f58d0100cb0 2024-04-23T02:16:08.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.895+0000 7f58ccff9700 1 --2- 172.21.15.71:0/2714381819 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f58b806cce0 0x7f58b806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:08.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.895+0000 7f58ccff9700 1 -- 172.21.15.71:0/2714381819 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f58c408dd30 con 0x7f58d0100cb0 2024-04-23T02:16:08.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.895+0000 7f58ceffd700 1 --2- 172.21.15.71:0/2714381819 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f58b806cce0 0x7f58b806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:08.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.896+0000 7f58ceffd700 1 --2- 172.21.15.71:0/2714381819 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f58b806cce0 0x7f58b806f1a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f58c00099d0 tx=0x7f58c0008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:08.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:08.900+0000 7f58ccff9700 1 -- 172.21.15.71:0/2714381819 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f58c405bab0 con 0x7f58d0100cb0 2024-04-23T02:16:09.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:09.119+0000 7f58d5bf5700 1 -- 172.21.15.71:0/2714381819 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f58d0106800 con 0x7f58b806cce0 2024-04-23T02:16:09.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:09.120+0000 7f58ccff9700 1 -- 172.21.15.71:0/2714381819 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f58d0106800 con 0x7f58b806cce0 2024-04-23T02:16:09.125 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:16:09.125 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:16:09.125 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:16:09.126 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-23T02:16:09.126 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:16:09.126 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr" 2024-04-23T02:16:09.126 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:16:09.126 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "2/2 daemons upgraded", 2024-04-23T02:16:09.126 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading prometheus daemons", 2024-04-23T02:16:09.126 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:16:09.126 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:16:09.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:09.122+0000 7f58d5bf5700 1 -- 172.21.15.71:0/2714381819 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f58b806cce0 msgr2=0x7f58b806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:09.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:09.122+0000 7f58d5bf5700 1 --2- 172.21.15.71:0/2714381819 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f58b806cce0 0x7f58b806f1a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f58c00099d0 tx=0x7f58c0008040 comp rx=0 tx=0).stop 2024-04-23T02:16:09.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:09.122+0000 7f58d5bf5700 1 -- 172.21.15.71:0/2714381819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58d0100cb0 msgr2=0x7f58d01964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:09.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:09.122+0000 7f58d5bf5700 1 --2- 172.21.15.71:0/2714381819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58d0100cb0 0x7f58d01964e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f58c400bda0 tx=0x7f58c400be80 comp rx=0 tx=0).stop 2024-04-23T02:16:09.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:09.122+0000 7f58d5bf5700 1 -- 172.21.15.71:0/2714381819 shutdown_connections 2024-04-23T02:16:09.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:09.122+0000 7f58d5bf5700 1 --2- 172.21.15.71:0/2714381819 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f58b806cce0 0x7f58b806f1a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:09.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:09.122+0000 7f58d5bf5700 1 --2- 172.21.15.71:0/2714381819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58d0100cb0 0x7f58d01964e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:09.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:09.122+0000 7f58d5bf5700 1 --2- 172.21.15.71:0/2714381819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58d0101eb0 0x7f58d0196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:09.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:09.122+0000 7f58d5bf5700 1 -- 172.21.15.71:0/2714381819 >> 172.21.15.71:0/2714381819 conn(0x7f58d00fc230 msgr2=0x7f58d01050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:09.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:09.122+0000 7f58d5bf5700 1 -- 172.21.15.71:0/2714381819 shutdown_connections 2024-04-23T02:16:09.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:09.122+0000 7f58d5bf5700 1 -- 172.21.15.71:0/2714381819 wait complete. 2024-04-23T02:16:09.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:09 smithi071 ceph-mon[39837]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:16:09.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:09 smithi071 ceph-mon[39837]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:16:09.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:09 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/2079358811' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:09.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:09 smithi192 ceph-mon[49662]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:16:09.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:09 smithi192 ceph-mon[49662]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:16:09.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:09 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/2079358811' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:10.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:10 smithi071 ceph-mon[39837]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:16:10.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:10 smithi071 ceph-mon[39837]: pgmap v50: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-23T02:16:10.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:10 smithi071 ceph-mon[39837]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:16:10.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:10 smithi192 ceph-mon[49662]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:16:10.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:10 smithi192 ceph-mon[49662]: pgmap v50: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-23T02:16:10.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:10 smithi192 ceph-mon[49662]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:16:12.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:11 smithi192 ceph-mon[49662]: pgmap v51: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-23T02:16:12.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:11 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:12.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:11 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:12.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:11 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:16:12.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:11 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:16:12.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:11 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:12.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:11 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-23T02:16:12.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:11 smithi192 ceph-mon[49662]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-23T02:16:12.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:11 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:16:12.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:11 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:12.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:11 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:12.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:11 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:12.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:11 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:12.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:11 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:12.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:11 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:12.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:11 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:12.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:11 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:12.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:11 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:12.219 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:11 smithi071 ceph-mon[39837]: pgmap v51: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-23T02:16:12.219 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:11 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:12.219 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:11 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:12.219 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:11 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:16:12.219 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:11 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:16:12.220 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:11 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:12.220 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:11 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-23T02:16:12.220 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:11 smithi071 ceph-mon[39837]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-23T02:16:12.220 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:11 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:16:12.220 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:11 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:12.220 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:11 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:12.220 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:11 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:12.220 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:11 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:12.220 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:11 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:12.220 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:11 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:12.220 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:11 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:12.220 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:11 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:12.220 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:11 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:13.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:13 smithi192 ceph-mon[49662]: pgmap v52: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-23T02:16:13.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:13 smithi192 ceph-mon[49662]: Upgrade: Updating alertmanager.smithi071 2024-04-23T02:16:13.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:13 smithi192 ceph-mon[49662]: Deploying daemon alertmanager.smithi071 on smithi071 2024-04-23T02:16:13.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:13 smithi071 ceph-mon[39837]: pgmap v52: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-23T02:16:13.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:13 smithi071 ceph-mon[39837]: Upgrade: Updating alertmanager.smithi071 2024-04-23T02:16:13.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:13 smithi071 ceph-mon[39837]: Deploying daemon alertmanager.smithi071 on smithi071 2024-04-23T02:16:16.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:15 smithi192 ceph-mon[49662]: pgmap v53: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-23T02:16:16.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:15 smithi071 ceph-mon[39837]: pgmap v53: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-23T02:16:18.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:17 smithi192 ceph-mon[49662]: pgmap v54: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-23T02:16:18.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:17 smithi071 ceph-mon[39837]: pgmap v54: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-23T02:16:20.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:19 smithi071 ceph-mon[39837]: pgmap v55: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:20.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:19 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:20.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:19 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:20.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:19 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:16:20.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:19 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:16:20.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:19 smithi192 ceph-mon[49662]: pgmap v55: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:20.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:19 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:20.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:19 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:20.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:19 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:16:20.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:19 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:16:22.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:21 smithi071 ceph-mon[39837]: pgmap v56: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:22.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:21 smithi192 ceph-mon[49662]: pgmap v56: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:24.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:23 smithi192 ceph-mon[49662]: pgmap v57: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:24.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:23 smithi071 ceph-mon[39837]: pgmap v57: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:25.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:25 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:25.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:25 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:25.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:25 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:25.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:25 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:25.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:25 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:25.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:25 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:25.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:25 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:25.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:25 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:26.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:26 smithi192 ceph-mon[49662]: pgmap v58: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:26.689 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:26 smithi071 ceph-mon[39837]: pgmap v58: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:28.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:28 smithi192 ceph-mon[49662]: pgmap v59: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:28.686 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:28 smithi071 ceph-mon[39837]: pgmap v59: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:29.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: pgmap v60: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:29.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:29.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:29.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:16:29.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:16:29.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:29.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-23T02:16:29.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-23T02:16:29.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:16:29.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:29.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:29.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:29.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:29.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:29.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:29.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:29.996 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:29.996 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:29.996 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:29 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:30.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: pgmap v60: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:30.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:30.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:30.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:16:30.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:16:30.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:30.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-23T02:16:30.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-23T02:16:30.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:16:30.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:30.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:30.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:30.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:30.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:30.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:30.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:30.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:30.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:30.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:29 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:31 smithi192 ceph-mon[49662]: Upgrade: Updating grafana.smithi071 2024-04-23T02:16:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:31 smithi192 ceph-mon[49662]: Deploying daemon grafana.smithi071 on smithi071 2024-04-23T02:16:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:31 smithi192 ceph-mon[49662]: pgmap v61: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:32.234 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:31 smithi071 ceph-mon[39837]: Upgrade: Updating grafana.smithi071 2024-04-23T02:16:32.235 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:31 smithi071 ceph-mon[39837]: Deploying daemon grafana.smithi071 on smithi071 2024-04-23T02:16:32.235 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:31 smithi071 ceph-mon[39837]: pgmap v61: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:33.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:33 smithi071 ceph-mon[39837]: pgmap v62: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:34.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:33 smithi192 ceph-mon[49662]: pgmap v62: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:35.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:16:35.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:16:35.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:35 smithi071 ceph-mon[39837]: pgmap v63: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:35.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:35 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:35.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:35 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:36.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:35 smithi192 ceph-mon[49662]: pgmap v63: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:36.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:35 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:36.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:35 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:36.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:36 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:16:37.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:36 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:16:38.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:37 smithi192 ceph-mon[49662]: pgmap v64: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:38.226 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:37 smithi071 ceph-mon[39837]: pgmap v64: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:39.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.233+0000 7f9f27a35700 1 -- 172.21.15.71:0/3314995938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f200ffc10 msgr2=0x7f9f20100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:39.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.233+0000 7f9f27a35700 1 --2- 172.21.15.71:0/3314995938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f200ffc10 0x7f9f20100030 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009910 tx=0x7f9f10009c20 comp rx=0 tx=0).stop 2024-04-23T02:16:39.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.233+0000 7f9f27a35700 1 -- 172.21.15.71:0/3314995938 shutdown_connections 2024-04-23T02:16:39.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.233+0000 7f9f27a35700 1 --2- 172.21.15.71:0/3314995938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f20100e10 0x7f9f20101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:39.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.233+0000 7f9f27a35700 1 --2- 172.21.15.71:0/3314995938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f200ffc10 0x7f9f20100030 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:39.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.233+0000 7f9f27a35700 1 -- 172.21.15.71:0/3314995938 >> 172.21.15.71:0/3314995938 conn(0x7f9f200fb190 msgr2=0x7f9f200fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:39.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.233+0000 7f9f27a35700 1 -- 172.21.15.71:0/3314995938 shutdown_connections 2024-04-23T02:16:39.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.233+0000 7f9f27a35700 1 -- 172.21.15.71:0/3314995938 wait complete. 2024-04-23T02:16:39.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.234+0000 7f9f27a35700 1 Processor -- start 2024-04-23T02:16:39.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.234+0000 7f9f27a35700 1 -- start start 2024-04-23T02:16:39.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.234+0000 7f9f27a35700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f200ffc10 0x7f9f200690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:39.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.234+0000 7f9f27a35700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f20100e10 0x7f9f2019b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:39.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.234+0000 7f9f27a35700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2019b730 con 0x7f9f20100e10 2024-04-23T02:16:39.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.234+0000 7f9f27a35700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2019b870 con 0x7f9f200ffc10 2024-04-23T02:16:39.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.234+0000 7f9f24fd0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f20100e10 0x7f9f2019b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:39.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.234+0000 7f9f24fd0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f20100e10 0x7f9f2019b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40192/0 (socket says 172.21.15.71:40192) 2024-04-23T02:16:39.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.234+0000 7f9f24fd0700 1 -- 172.21.15.71:0/3785742692 learned_addr learned my addr 172.21.15.71:0/3785742692 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:16:39.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.234+0000 7f9f257d1700 1 --2- 172.21.15.71:0/3785742692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f200ffc10 0x7f9f200690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:39.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.234+0000 7f9f24fd0700 1 -- 172.21.15.71:0/3785742692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f200ffc10 msgr2=0x7f9f200690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:39.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.234+0000 7f9f24fd0700 1 --2- 172.21.15.71:0/3785742692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f200ffc10 0x7f9f200690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:39.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.234+0000 7f9f24fd0700 1 -- 172.21.15.71:0/3785742692 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f100095f0 con 0x7f9f20100e10 2024-04-23T02:16:39.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.234+0000 7f9f24fd0700 1 --2- 172.21.15.71:0/3785742692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f20100e10 0x7f9f2019b160 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00d8d0 tx=0x7f9f1c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:39.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.235+0000 7f9f167fc700 1 -- 172.21.15.71:0/3785742692 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f1c0098a0 con 0x7f9f20100e10 2024-04-23T02:16:39.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.235+0000 7f9f27a35700 1 -- 172.21.15.71:0/3785742692 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2019bb50 con 0x7f9f20100e10 2024-04-23T02:16:39.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.235+0000 7f9f167fc700 1 -- 172.21.15.71:0/3785742692 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f9f1c009e90 con 0x7f9f20100e10 2024-04-23T02:16:39.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.235+0000 7f9f167fc700 1 -- 172.21.15.71:0/3785742692 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f1c00f5f0 con 0x7f9f20100e10 2024-04-23T02:16:39.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.235+0000 7f9f27a35700 1 -- 172.21.15.71:0/3785742692 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2019c0a0 con 0x7f9f20100e10 2024-04-23T02:16:39.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.236+0000 7f9f27a35700 1 -- 172.21.15.71:0/3785742692 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f201085c0 con 0x7f9f20100e10 2024-04-23T02:16:39.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.236+0000 7f9f167fc700 1 -- 172.21.15.71:0/3785742692 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f9f1c010460 con 0x7f9f20100e10 2024-04-23T02:16:39.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.236+0000 7f9f167fc700 1 --2- 172.21.15.71:0/3785742692 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9f0c06cce0 0x7f9f0c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:39.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.237+0000 7f9f167fc700 1 -- 172.21.15.71:0/3785742692 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9f1c08c360 con 0x7f9f20100e10 2024-04-23T02:16:39.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.237+0000 7f9f257d1700 1 --2- 172.21.15.71:0/3785742692 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9f0c06cce0 0x7f9f0c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:39.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.237+0000 7f9f257d1700 1 --2- 172.21.15.71:0/3785742692 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9f0c06cce0 0x7f9f0c06f1a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000bf80 tx=0x7f9f1000b560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:39.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.241+0000 7f9f167fc700 1 -- 172.21.15.71:0/3785742692 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f9f1c059f10 con 0x7f9f20100e10 2024-04-23T02:16:39.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.457+0000 7f9f27a35700 1 -- 172.21.15.71:0/3785742692 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f20105760 con 0x7f9f0c06cce0 2024-04-23T02:16:39.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.458+0000 7f9f167fc700 1 -- 172.21.15.71:0/3785742692 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f9f20105760 con 0x7f9f0c06cce0 2024-04-23T02:16:39.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.460+0000 7f9f27a35700 1 -- 172.21.15.71:0/3785742692 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9f0c06cce0 msgr2=0x7f9f0c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:39.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.460+0000 7f9f27a35700 1 --2- 172.21.15.71:0/3785742692 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9f0c06cce0 0x7f9f0c06f1a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000bf80 tx=0x7f9f1000b560 comp rx=0 tx=0).stop 2024-04-23T02:16:39.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.460+0000 7f9f27a35700 1 -- 172.21.15.71:0/3785742692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f20100e10 msgr2=0x7f9f2019b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:39.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.460+0000 7f9f27a35700 1 --2- 172.21.15.71:0/3785742692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f20100e10 0x7f9f2019b160 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00d8d0 tx=0x7f9f1c00dbe0 comp rx=0 tx=0).stop 2024-04-23T02:16:39.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.460+0000 7f9f27a35700 1 -- 172.21.15.71:0/3785742692 shutdown_connections 2024-04-23T02:16:39.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.460+0000 7f9f27a35700 1 --2- 172.21.15.71:0/3785742692 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9f0c06cce0 0x7f9f0c06f1a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:39.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.460+0000 7f9f27a35700 1 --2- 172.21.15.71:0/3785742692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f200ffc10 0x7f9f200690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:39.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.460+0000 7f9f27a35700 1 --2- 172.21.15.71:0/3785742692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f20100e10 0x7f9f2019b160 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:39.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.460+0000 7f9f27a35700 1 -- 172.21.15.71:0/3785742692 >> 172.21.15.71:0/3785742692 conn(0x7f9f200fb190 msgr2=0x7f9f20104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:39.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.460+0000 7f9f27a35700 1 -- 172.21.15.71:0/3785742692 shutdown_connections 2024-04-23T02:16:39.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.460+0000 7f9f27a35700 1 -- 172.21.15.71:0/3785742692 wait complete. 2024-04-23T02:16:39.475 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:16:39.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.577+0000 7f1bffc07700 1 -- 172.21.15.71:0/1998852964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bf80ffe70 msgr2=0x7f1bf81002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:39.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.577+0000 7f1bffc07700 1 --2- 172.21.15.71:0/1998852964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bf80ffe70 0x7f1bf81002d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009910 tx=0x7f1be8009c20 comp rx=0 tx=0).stop 2024-04-23T02:16:39.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.578+0000 7f1bffc07700 1 -- 172.21.15.71:0/1998852964 shutdown_connections 2024-04-23T02:16:39.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.578+0000 7f1bffc07700 1 --2- 172.21.15.71:0/1998852964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bf80ffe70 0x7f1bf81002d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:39.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.578+0000 7f1bffc07700 1 --2- 172.21.15.71:0/1998852964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf8101130 0x7f1bf8101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:39.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.578+0000 7f1bffc07700 1 -- 172.21.15.71:0/1998852964 >> 172.21.15.71:0/1998852964 conn(0x7f1bf80fa8c0 msgr2=0x7f1bf80fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:39.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.578+0000 7f1bffc07700 1 -- 172.21.15.71:0/1998852964 shutdown_connections 2024-04-23T02:16:39.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.578+0000 7f1bffc07700 1 -- 172.21.15.71:0/1998852964 wait complete. 2024-04-23T02:16:39.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.578+0000 7f1bffc07700 1 Processor -- start 2024-04-23T02:16:39.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.578+0000 7f1bffc07700 1 -- start start 2024-04-23T02:16:39.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.579+0000 7f1bffc07700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf80ffe70 0x7f1bf8196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:39.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.579+0000 7f1bffc07700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bf8101130 0x7f1bf8196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:39.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.579+0000 7f1bffc07700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf8196f80 con 0x7f1bf8101130 2024-04-23T02:16:39.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.579+0000 7f1bffc07700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf81970c0 con 0x7f1bf80ffe70 2024-04-23T02:16:39.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.579+0000 7f1bfd1a2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bf8101130 0x7f1bf8196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:39.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.579+0000 7f1bfd1a2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bf8101130 0x7f1bf8196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40220/0 (socket says 172.21.15.71:40220) 2024-04-23T02:16:39.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.579+0000 7f1bfd1a2700 1 -- 172.21.15.71:0/3424119519 learned_addr learned my addr 172.21.15.71:0/3424119519 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:16:39.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.579+0000 7f1bfd9a3700 1 --2- 172.21.15.71:0/3424119519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf80ffe70 0x7f1bf8196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:39.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.579+0000 7f1bfd1a2700 1 -- 172.21.15.71:0/3424119519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf80ffe70 msgr2=0x7f1bf8196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:39.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.579+0000 7f1bfd1a2700 1 --2- 172.21.15.71:0/3424119519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf80ffe70 0x7f1bf8196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:39.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.579+0000 7f1bfd1a2700 1 -- 172.21.15.71:0/3424119519 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be80095f0 con 0x7f1bf8101130 2024-04-23T02:16:39.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.579+0000 7f1bfd1a2700 1 --2- 172.21.15.71:0/3424119519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bf8101130 0x7f1bf8196960 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1be8006010 tx=0x7f1be800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:39.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.579+0000 7f1beeffd700 1 -- 172.21.15.71:0/3424119519 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1be801d070 con 0x7f1bf8101130 2024-04-23T02:16:39.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.580+0000 7f1bffc07700 1 -- 172.21.15.71:0/3424119519 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf819bb10 con 0x7f1bf8101130 2024-04-23T02:16:39.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.580+0000 7f1beeffd700 1 -- 172.21.15.71:0/3424119519 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1be800f460 con 0x7f1bf8101130 2024-04-23T02:16:39.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.580+0000 7f1bffc07700 1 -- 172.21.15.71:0/3424119519 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf819c000 con 0x7f1bf8101130 2024-04-23T02:16:39.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.580+0000 7f1beeffd700 1 -- 172.21.15.71:0/3424119519 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1be8021640 con 0x7f1bf8101130 2024-04-23T02:16:39.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.581+0000 7f1bffc07700 1 -- 172.21.15.71:0/3424119519 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bf81085d0 con 0x7f1bf8101130 2024-04-23T02:16:39.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.581+0000 7f1beeffd700 1 -- 172.21.15.71:0/3424119519 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f1be802b440 con 0x7f1bf8101130 2024-04-23T02:16:39.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.581+0000 7f1beeffd700 1 --2- 172.21.15.71:0/3424119519 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1be406cce0 0x7f1be406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:39.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.582+0000 7f1beeffd700 1 -- 172.21.15.71:0/3424119519 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1be808d650 con 0x7f1bf8101130 2024-04-23T02:16:39.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.582+0000 7f1bfd9a3700 1 --2- 172.21.15.71:0/3424119519 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1be406cce0 0x7f1be406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:39.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.582+0000 7f1bfd9a3700 1 --2- 172.21.15.71:0/3424119519 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1be406cce0 0x7f1be406f1a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8068300 tx=0x7f1bf400b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:39.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.586+0000 7f1beeffd700 1 -- 172.21.15.71:0/3424119519 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f1be805b280 con 0x7f1bf8101130 2024-04-23T02:16:39.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.803+0000 7f1bffc07700 1 -- 172.21.15.71:0/3424119519 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bf80ff450 con 0x7f1be406cce0 2024-04-23T02:16:39.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.804+0000 7f1beeffd700 1 -- 172.21.15.71:0/3424119519 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f1bf80ff450 con 0x7f1be406cce0 2024-04-23T02:16:39.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.806+0000 7f1bffc07700 1 -- 172.21.15.71:0/3424119519 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1be406cce0 msgr2=0x7f1be406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:39.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.806+0000 7f1bffc07700 1 --2- 172.21.15.71:0/3424119519 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1be406cce0 0x7f1be406f1a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8068300 tx=0x7f1bf400b410 comp rx=0 tx=0).stop 2024-04-23T02:16:39.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.806+0000 7f1bffc07700 1 -- 172.21.15.71:0/3424119519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bf8101130 msgr2=0x7f1bf8196960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:39.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.806+0000 7f1bffc07700 1 --2- 172.21.15.71:0/3424119519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bf8101130 0x7f1bf8196960 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1be8006010 tx=0x7f1be800bd70 comp rx=0 tx=0).stop 2024-04-23T02:16:39.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.806+0000 7f1bffc07700 1 -- 172.21.15.71:0/3424119519 shutdown_connections 2024-04-23T02:16:39.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.806+0000 7f1bffc07700 1 --2- 172.21.15.71:0/3424119519 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1be406cce0 0x7f1be406f1a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:39.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.806+0000 7f1bffc07700 1 --2- 172.21.15.71:0/3424119519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf80ffe70 0x7f1bf8196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:39.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.806+0000 7f1bffc07700 1 --2- 172.21.15.71:0/3424119519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bf8101130 0x7f1bf8196960 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:39.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.806+0000 7f1bffc07700 1 -- 172.21.15.71:0/3424119519 >> 172.21.15.71:0/3424119519 conn(0x7f1bf80fa8c0 msgr2=0x7f1bf80fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:39.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.806+0000 7f1bffc07700 1 -- 172.21.15.71:0/3424119519 shutdown_connections 2024-04-23T02:16:39.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.807+0000 7f1bffc07700 1 -- 172.21.15.71:0/3424119519 wait complete. 2024-04-23T02:16:39.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.936+0000 7fd4b0042700 1 -- 172.21.15.71:0/977422251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a80fef60 msgr2=0x7fd4a80ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:39.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.936+0000 7fd4b0042700 1 --2- 172.21.15.71:0/977422251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a80fef60 0x7fd4a80ff380 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009910 tx=0x7fd4a4009c20 comp rx=0 tx=0).stop 2024-04-23T02:16:39.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.936+0000 7fd4b0042700 1 -- 172.21.15.71:0/977422251 shutdown_connections 2024-04-23T02:16:39.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.936+0000 7fd4b0042700 1 --2- 172.21.15.71:0/977422251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4a8101f40 0x7fd4a81023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:39.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.936+0000 7fd4b0042700 1 --2- 172.21.15.71:0/977422251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a80fef60 0x7fd4a80ff380 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:39.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.936+0000 7fd4b0042700 1 -- 172.21.15.71:0/977422251 >> 172.21.15.71:0/977422251 conn(0x7fd4a80fa8c0 msgr2=0x7fd4a80fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:39.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.936+0000 7fd4b0042700 1 -- 172.21.15.71:0/977422251 shutdown_connections 2024-04-23T02:16:39.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.936+0000 7fd4b0042700 1 -- 172.21.15.71:0/977422251 wait complete. 2024-04-23T02:16:39.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.936+0000 7fd4b0042700 1 Processor -- start 2024-04-23T02:16:39.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.937+0000 7fd4b0042700 1 -- start start 2024-04-23T02:16:39.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.937+0000 7fd4b0042700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4a80fef60 0x7fd4a81018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:39.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.937+0000 7fd4b0042700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a8101f40 0x7fd4a80ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:39.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.937+0000 7fd4b0042700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a8100430 con 0x7fd4a8101f40 2024-04-23T02:16:39.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.937+0000 7fd4b0042700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a8100570 con 0x7fd4a80fef60 2024-04-23T02:16:39.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.937+0000 7fd4ad5dd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a8101f40 0x7fd4a80ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:39.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.937+0000 7fd4ad5dd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a8101f40 0x7fd4a80ffef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40242/0 (socket says 172.21.15.71:40242) 2024-04-23T02:16:39.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.937+0000 7fd4ad5dd700 1 -- 172.21.15.71:0/4277170097 learned_addr learned my addr 172.21.15.71:0/4277170097 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:16:39.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.937+0000 7fd4addde700 1 --2- 172.21.15.71:0/4277170097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4a80fef60 0x7fd4a81018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:39.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.937+0000 7fd4ad5dd700 1 -- 172.21.15.71:0/4277170097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4a80fef60 msgr2=0x7fd4a81018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:39.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.937+0000 7fd4ad5dd700 1 --2- 172.21.15.71:0/4277170097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4a80fef60 0x7fd4a81018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:39.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.937+0000 7fd4ad5dd700 1 -- 172.21.15.71:0/4277170097 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a40095f0 con 0x7fd4a8101f40 2024-04-23T02:16:39.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.937+0000 7fd4addde700 1 --2- 172.21.15.71:0/4277170097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4a80fef60 0x7fd4a81018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:16:39.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.937+0000 7fd4ad5dd700 1 --2- 172.21.15.71:0/4277170097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a8101f40 0x7fd4a80ffef0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd49800c8a0 tx=0x7fd49800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:39.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.937+0000 7fd49effd700 1 -- 172.21.15.71:0/4277170097 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd49800cea0 con 0x7fd4a8101f40 2024-04-23T02:16:39.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.938+0000 7fd49effd700 1 -- 172.21.15.71:0/4277170097 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd498004830 con 0x7fd4a8101f40 2024-04-23T02:16:39.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.938+0000 7fd49effd700 1 -- 172.21.15.71:0/4277170097 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd498005540 con 0x7fd4a8101f40 2024-04-23T02:16:39.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.938+0000 7fd4b0042700 1 -- 172.21.15.71:0/4277170097 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a8100850 con 0x7fd4a8101f40 2024-04-23T02:16:39.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.938+0000 7fd4b0042700 1 -- 172.21.15.71:0/4277170097 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a819e460 con 0x7fd4a8101f40 2024-04-23T02:16:39.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.939+0000 7fd4b0042700 1 -- 172.21.15.71:0/4277170097 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a81096c0 con 0x7fd4a8101f40 2024-04-23T02:16:39.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.939+0000 7fd49effd700 1 -- 172.21.15.71:0/4277170097 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd4980049a0 con 0x7fd4a8101f40 2024-04-23T02:16:39.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.940+0000 7fd49effd700 1 --2- 172.21.15.71:0/4277170097 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd49406cce0 0x7fd49406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:39.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.940+0000 7fd49effd700 1 -- 172.21.15.71:0/4277170097 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd49808c1c0 con 0x7fd4a8101f40 2024-04-23T02:16:39.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.940+0000 7fd4addde700 1 --2- 172.21.15.71:0/4277170097 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd49406cce0 0x7fd49406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:39.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.940+0000 7fd4addde700 1 --2- 172.21.15.71:0/4277170097 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd49406cce0 0x7fd49406f1a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4006010 tx=0x7fd4a4005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:39.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:39.944+0000 7fd49effd700 1 -- 172.21.15.71:0/4277170097 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fd498059d70 con 0x7fd4a8101f40 2024-04-23T02:16:40.069 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:39 smithi071 ceph-mon[39837]: pgmap v65: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:40.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:39 smithi192 ceph-mon[49662]: pgmap v65: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:40.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.158+0000 7fd4b0042700 1 -- 172.21.15.71:0/4277170097 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4a8106890 con 0x7fd49406cce0 2024-04-23T02:16:40.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.169+0000 7fd49effd700 1 -- 172.21.15.71:0/4277170097 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd4a8106890 con 0x7fd49406cce0 2024-04-23T02:16:40.173 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:16:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (22s) 15s ago 11m 16.9M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:16:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (11m) 15s ago 11m 7956k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:16:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (10m) 67s ago 10m 7796k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:16:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (11m) 15s ago 11m 6656k - 18.2.1 5be31c24972a 258512dcbc00 2024-04-23T02:16:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (10m) 67s ago 10m 6656k - 18.2.1 5be31c24972a feea2c9896a6 2024-04-23T02:16:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 starting - - - - 2024-04-23T02:16:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (5m) 15s ago 5m 64.7M - 18.2.1 5be31c24972a a515de2a841c 2024-04-23T02:16:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (5m) 15s ago 5m 133M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:16:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (5m) 67s ago 5m 68.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:16:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (5m) 67s ago 5m 125M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:16:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (2m) 15s ago 13m 496M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:16:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (93s) 67s ago 10m 449M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:16:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (13m) 15s ago 13m 62.9M 2048M 18.2.1 5be31c24972a ff40abe63fd9 2024-04-23T02:16:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (10m) 67s ago 10m 46.7M 2048M 18.2.1 5be31c24972a 9c03e90acc95 2024-04-23T02:16:40.175 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (78s) 15s ago 11m 9387k - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:16:40.175 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (71s) 67s ago 10m 3510k - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:16:40.175 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (9m) 15s ago 9m 197M 1227M 18.2.1 5be31c24972a d3d5f2fa7267 2024-04-23T02:16:40.175 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (9m) 15s ago 9m 209M 1227M 18.2.1 5be31c24972a 3992157ff0fd 2024-04-23T02:16:40.175 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (8m) 15s ago 8m 164M 1227M 18.2.1 5be31c24972a 12380b6a15a2 2024-04-23T02:16:40.175 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (8m) 67s ago 8m 240M 2251M 18.2.1 5be31c24972a d5f587ee3a22 2024-04-23T02:16:40.175 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (7m) 67s ago 7m 217M 2251M 18.2.1 5be31c24972a 958d046005f0 2024-04-23T02:16:40.175 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (7m) 67s ago 7m 195M 2251M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:16:40.175 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (40s) 15s ago 11m 46.4M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:16:40.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.171+0000 7fd4b0042700 1 -- 172.21.15.71:0/4277170097 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd49406cce0 msgr2=0x7fd49406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:40.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.171+0000 7fd4b0042700 1 --2- 172.21.15.71:0/4277170097 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd49406cce0 0x7fd49406f1a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4006010 tx=0x7fd4a4005960 comp rx=0 tx=0).stop 2024-04-23T02:16:40.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.171+0000 7fd4b0042700 1 -- 172.21.15.71:0/4277170097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a8101f40 msgr2=0x7fd4a80ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:40.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.171+0000 7fd4b0042700 1 --2- 172.21.15.71:0/4277170097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a8101f40 0x7fd4a80ffef0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd49800c8a0 tx=0x7fd49800cc60 comp rx=0 tx=0).stop 2024-04-23T02:16:40.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.171+0000 7fd4b0042700 1 -- 172.21.15.71:0/4277170097 shutdown_connections 2024-04-23T02:16:40.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.171+0000 7fd4b0042700 1 --2- 172.21.15.71:0/4277170097 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd49406cce0 0x7fd49406f1a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:40.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.171+0000 7fd4b0042700 1 --2- 172.21.15.71:0/4277170097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4a80fef60 0x7fd4a81018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:40.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.171+0000 7fd4b0042700 1 --2- 172.21.15.71:0/4277170097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4a8101f40 0x7fd4a80ffef0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:40.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.171+0000 7fd4b0042700 1 -- 172.21.15.71:0/4277170097 >> 172.21.15.71:0/4277170097 conn(0x7fd4a80fa8c0 msgr2=0x7fd4a8105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:40.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.172+0000 7fd4b0042700 1 -- 172.21.15.71:0/4277170097 shutdown_connections 2024-04-23T02:16:40.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.172+0000 7fd4b0042700 1 -- 172.21.15.71:0/4277170097 wait complete. 2024-04-23T02:16:40.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.274+0000 7fbf825bc700 1 -- 172.21.15.71:0/3877435329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf7c100db0 msgr2=0x7fbf7c101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:40.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.274+0000 7fbf825bc700 1 --2- 172.21.15.71:0/3877435329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf7c100db0 0x7fbf7c101210 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009910 tx=0x7fbf64009c20 comp rx=0 tx=0).stop 2024-04-23T02:16:40.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.274+0000 7fbf825bc700 1 -- 172.21.15.71:0/3877435329 shutdown_connections 2024-04-23T02:16:40.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.274+0000 7fbf825bc700 1 --2- 172.21.15.71:0/3877435329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf7c100db0 0x7fbf7c101210 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:40.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.274+0000 7fbf825bc700 1 --2- 172.21.15.71:0/3877435329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf7c0ffbb0 0x7fbf7c0fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:40.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.274+0000 7fbf825bc700 1 -- 172.21.15.71:0/3877435329 >> 172.21.15.71:0/3877435329 conn(0x7fbf7c0fb150 msgr2=0x7fbf7c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:40.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.274+0000 7fbf825bc700 1 -- 172.21.15.71:0/3877435329 shutdown_connections 2024-04-23T02:16:40.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.274+0000 7fbf825bc700 1 -- 172.21.15.71:0/3877435329 wait complete. 2024-04-23T02:16:40.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.274+0000 7fbf825bc700 1 Processor -- start 2024-04-23T02:16:40.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.274+0000 7fbf825bc700 1 -- start start 2024-04-23T02:16:40.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.275+0000 7fbf825bc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf7c0ffbb0 0x7fbf7c103b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:40.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.275+0000 7fbf825bc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf7c100db0 0x7fbf7c102190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:40.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.275+0000 7fbf825bc700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c1026d0 con 0x7fbf7c0ffbb0 2024-04-23T02:16:40.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.275+0000 7fbf825bc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c102810 con 0x7fbf7c100db0 2024-04-23T02:16:40.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.275+0000 7fbf7bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf7c0ffbb0 0x7fbf7c103b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:40.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.275+0000 7fbf7bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf7c0ffbb0 0x7fbf7c103b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40248/0 (socket says 172.21.15.71:40248) 2024-04-23T02:16:40.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.275+0000 7fbf7bfff700 1 -- 172.21.15.71:0/31368480 learned_addr learned my addr 172.21.15.71:0/31368480 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:16:40.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.275+0000 7fbf7b7fe700 1 --2- 172.21.15.71:0/31368480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf7c100db0 0x7fbf7c102190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:40.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.275+0000 7fbf7bfff700 1 -- 172.21.15.71:0/31368480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf7c100db0 msgr2=0x7fbf7c102190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:40.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.275+0000 7fbf7bfff700 1 --2- 172.21.15.71:0/31368480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf7c100db0 0x7fbf7c102190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:40.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.275+0000 7fbf7bfff700 1 -- 172.21.15.71:0/31368480 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf640095f0 con 0x7fbf7c0ffbb0 2024-04-23T02:16:40.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.275+0000 7fbf7b7fe700 1 --2- 172.21.15.71:0/31368480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf7c100db0 0x7fbf7c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:16:40.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.275+0000 7fbf7bfff700 1 --2- 172.21.15.71:0/31368480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf7c0ffbb0 0x7fbf7c103b40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00b480 tx=0x7fbf6c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:40.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.275+0000 7fbf797fa700 1 -- 172.21.15.71:0/31368480 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf6c0118c0 con 0x7fbf7c0ffbb0 2024-04-23T02:16:40.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.275+0000 7fbf797fa700 1 -- 172.21.15.71:0/31368480 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fbf6c010380 con 0x7fbf7c0ffbb0 2024-04-23T02:16:40.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.275+0000 7fbf825bc700 1 -- 172.21.15.71:0/31368480 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7c102af0 con 0x7fbf7c0ffbb0 2024-04-23T02:16:40.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.275+0000 7fbf797fa700 1 -- 172.21.15.71:0/31368480 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf6c00f510 con 0x7fbf7c0ffbb0 2024-04-23T02:16:40.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.275+0000 7fbf825bc700 1 -- 172.21.15.71:0/31368480 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7c19e540 con 0x7fbf7c0ffbb0 2024-04-23T02:16:40.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.276+0000 7fbf825bc700 1 -- 172.21.15.71:0/31368480 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7c108670 con 0x7fbf7c0ffbb0 2024-04-23T02:16:40.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.277+0000 7fbf797fa700 1 -- 172.21.15.71:0/31368480 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fbf6c0104f0 con 0x7fbf7c0ffbb0 2024-04-23T02:16:40.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.277+0000 7fbf797fa700 1 --2- 172.21.15.71:0/31368480 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbf6806cce0 0x7fbf6806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:40.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.277+0000 7fbf797fa700 1 -- 172.21.15.71:0/31368480 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbf6c08c170 con 0x7fbf7c0ffbb0 2024-04-23T02:16:40.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.277+0000 7fbf7b7fe700 1 --2- 172.21.15.71:0/31368480 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbf6806cce0 0x7fbf6806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:40.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.278+0000 7fbf7b7fe700 1 --2- 172.21.15.71:0/31368480 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbf6806cce0 0x7fbf6806f1a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbf640059d0 tx=0x7fbf64005920 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:40.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.282+0000 7fbf797fa700 1 -- 172.21.15.71:0/31368480 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fbf6c059d20 con 0x7fbf7c0ffbb0 2024-04-23T02:16:40.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.579+0000 7fbf825bc700 1 -- 172.21.15.71:0/31368480 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbf7c19ed10 con 0x7fbf7c0ffbb0 2024-04-23T02:16:40.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.580+0000 7fbf797fa700 1 -- 172.21.15.71:0/31368480 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7fbf6c0107a0 con 0x7fbf7c0ffbb0 2024-04-23T02:16:40.585 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:16:40.585 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:16:40.585 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-23T02:16:40.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:16:40.585 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:16:40.585 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:16:40.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:16:40.585 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:16:40.585 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-23T02:16:40.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:16:40.585 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:16:40.585 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-23T02:16:40.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:16:40.586 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:16:40.586 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-23T02:16:40.586 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:16:40.586 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:16:40.586 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:16:40.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.582+0000 7fbf825bc700 1 -- 172.21.15.71:0/31368480 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbf6806cce0 msgr2=0x7fbf6806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:40.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.582+0000 7fbf825bc700 1 --2- 172.21.15.71:0/31368480 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbf6806cce0 0x7fbf6806f1a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbf640059d0 tx=0x7fbf64005920 comp rx=0 tx=0).stop 2024-04-23T02:16:40.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.582+0000 7fbf825bc700 1 -- 172.21.15.71:0/31368480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf7c0ffbb0 msgr2=0x7fbf7c103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:40.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.582+0000 7fbf825bc700 1 --2- 172.21.15.71:0/31368480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf7c0ffbb0 0x7fbf7c103b40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00b480 tx=0x7fbf6c00b840 comp rx=0 tx=0).stop 2024-04-23T02:16:40.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.582+0000 7fbf825bc700 1 -- 172.21.15.71:0/31368480 shutdown_connections 2024-04-23T02:16:40.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.582+0000 7fbf825bc700 1 --2- 172.21.15.71:0/31368480 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbf6806cce0 0x7fbf6806f1a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:40.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.582+0000 7fbf825bc700 1 --2- 172.21.15.71:0/31368480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf7c0ffbb0 0x7fbf7c103b40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:40.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.582+0000 7fbf825bc700 1 --2- 172.21.15.71:0/31368480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf7c100db0 0x7fbf7c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:40.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.582+0000 7fbf825bc700 1 -- 172.21.15.71:0/31368480 >> 172.21.15.71:0/31368480 conn(0x7fbf7c0fb150 msgr2=0x7fbf7c068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:40.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.582+0000 7fbf825bc700 1 -- 172.21.15.71:0/31368480 shutdown_connections 2024-04-23T02:16:40.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.582+0000 7fbf825bc700 1 -- 172.21.15.71:0/31368480 wait complete. 2024-04-23T02:16:40.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.698+0000 7f2f08462700 1 -- 172.21.15.71:0/3646056031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f00102000 msgr2=0x7f2f00102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:40.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.698+0000 7f2f08462700 1 --2- 172.21.15.71:0/3646056031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f00102000 0x7f2f00102460 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009910 tx=0x7f2ef0009c20 comp rx=0 tx=0).stop 2024-04-23T02:16:40.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.699+0000 7f2f08462700 1 -- 172.21.15.71:0/3646056031 shutdown_connections 2024-04-23T02:16:40.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.699+0000 7f2f08462700 1 --2- 172.21.15.71:0/3646056031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f00102000 0x7f2f00102460 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:40.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.699+0000 7f2f08462700 1 --2- 172.21.15.71:0/3646056031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f000fef80 0x7f2f000ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:40.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.699+0000 7f2f08462700 1 -- 172.21.15.71:0/3646056031 >> 172.21.15.71:0/3646056031 conn(0x7f2f000fa8c0 msgr2=0x7f2f000fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:40.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.699+0000 7f2f08462700 1 -- 172.21.15.71:0/3646056031 shutdown_connections 2024-04-23T02:16:40.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.699+0000 7f2f08462700 1 -- 172.21.15.71:0/3646056031 wait complete. 2024-04-23T02:16:40.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.699+0000 7f2f08462700 1 Processor -- start 2024-04-23T02:16:40.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.699+0000 7f2f08462700 1 -- start start 2024-04-23T02:16:40.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.699+0000 7f2f08462700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f000fef80 0x7f2f001963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:40.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.699+0000 7f2f08462700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f00102000 0x7f2f00196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:40.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.699+0000 7f2f08462700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f00196f20 con 0x7f2f00102000 2024-04-23T02:16:40.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.699+0000 7f2f08462700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f00197060 con 0x7f2f000fef80 2024-04-23T02:16:40.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.699+0000 7f2f059fd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f00102000 0x7f2f00196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:40.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.699+0000 7f2f059fd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f00102000 0x7f2f00196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40266/0 (socket says 172.21.15.71:40266) 2024-04-23T02:16:40.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.699+0000 7f2f059fd700 1 -- 172.21.15.71:0/1670920714 learned_addr learned my addr 172.21.15.71:0/1670920714 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:16:40.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.700+0000 7f2f061fe700 1 --2- 172.21.15.71:0/1670920714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f000fef80 0x7f2f001963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:40.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.700+0000 7f2f059fd700 1 -- 172.21.15.71:0/1670920714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f000fef80 msgr2=0x7f2f001963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:40.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.700+0000 7f2f059fd700 1 --2- 172.21.15.71:0/1670920714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f000fef80 0x7f2f001963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:40.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.700+0000 7f2f059fd700 1 -- 172.21.15.71:0/1670920714 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef00095f0 con 0x7f2f00102000 2024-04-23T02:16:40.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.700+0000 7f2f059fd700 1 --2- 172.21.15.71:0/1670920714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f00102000 0x7f2f00196900 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000bde0 tx=0x7f2ef000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:40.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.700+0000 7f2ef77fe700 1 -- 172.21.15.71:0/1670920714 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ef001d070 con 0x7f2f00102000 2024-04-23T02:16:40.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.700+0000 7f2ef77fe700 1 -- 172.21.15.71:0/1670920714 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f2ef0005dc0 con 0x7f2f00102000 2024-04-23T02:16:40.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.700+0000 7f2f08462700 1 -- 172.21.15.71:0/1670920714 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0019bab0 con 0x7f2f00102000 2024-04-23T02:16:40.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.700+0000 7f2ef77fe700 1 -- 172.21.15.71:0/1670920714 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ef0003b10 con 0x7f2f00102000 2024-04-23T02:16:40.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.700+0000 7f2f08462700 1 -- 172.21.15.71:0/1670920714 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0019bf40 con 0x7f2f00102000 2024-04-23T02:16:40.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.701+0000 7f2f08462700 1 -- 172.21.15.71:0/1670920714 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f001096c0 con 0x7f2f00102000 2024-04-23T02:16:40.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.702+0000 7f2ef77fe700 1 -- 172.21.15.71:0/1670920714 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2ef00048e0 con 0x7f2f00102000 2024-04-23T02:16:40.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.702+0000 7f2ef77fe700 1 --2- 172.21.15.71:0/1670920714 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2eec06cce0 0x7f2eec06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:16:40.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.703+0000 7f2f061fe700 1 --2- 172.21.15.71:0/1670920714 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2eec06cce0 0x7f2eec06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:16:40.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.703+0000 7f2ef77fe700 1 -- 172.21.15.71:0/1670920714 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2ef008e430 con 0x7f2f00102000 2024-04-23T02:16:40.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.703+0000 7f2f061fe700 1 --2- 172.21.15.71:0/1670920714 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2eec06cce0 0x7f2eec06f1a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2f00068b50 tx=0x7f2efc009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:16:40.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.707+0000 7f2ef77fe700 1 -- 172.21.15.71:0/1670920714 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f2ef005c000 con 0x7f2f00102000 2024-04-23T02:16:40.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.923+0000 7f2f08462700 1 -- 172.21.15.71:0/1670920714 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f00106950 con 0x7f2eec06cce0 2024-04-23T02:16:40.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.924+0000 7f2ef77fe700 1 -- 172.21.15.71:0/1670920714 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f2f00106950 con 0x7f2eec06cce0 2024-04-23T02:16:40.929 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:16:40.929 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:16:40.929 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:16:40.929 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-23T02:16:40.929 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:16:40.929 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr" 2024-04-23T02:16:40.929 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:16:40.929 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "2/2 daemons upgraded", 2024-04-23T02:16:40.929 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading grafana daemons", 2024-04-23T02:16:40.930 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:16:40.930 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:16:40.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.926+0000 7f2f08462700 1 -- 172.21.15.71:0/1670920714 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2eec06cce0 msgr2=0x7f2eec06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:40.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.926+0000 7f2f08462700 1 --2- 172.21.15.71:0/1670920714 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2eec06cce0 0x7f2eec06f1a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2f00068b50 tx=0x7f2efc009260 comp rx=0 tx=0).stop 2024-04-23T02:16:40.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.926+0000 7f2f08462700 1 -- 172.21.15.71:0/1670920714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f00102000 msgr2=0x7f2f00196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:16:40.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.926+0000 7f2f08462700 1 --2- 172.21.15.71:0/1670920714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f00102000 0x7f2f00196900 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000bde0 tx=0x7f2ef000be10 comp rx=0 tx=0).stop 2024-04-23T02:16:40.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.927+0000 7f2f08462700 1 -- 172.21.15.71:0/1670920714 shutdown_connections 2024-04-23T02:16:40.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.927+0000 7f2f08462700 1 --2- 172.21.15.71:0/1670920714 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2eec06cce0 0x7f2eec06f1a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:40.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.927+0000 7f2f08462700 1 --2- 172.21.15.71:0/1670920714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f000fef80 0x7f2f001963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:40.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.927+0000 7f2f08462700 1 --2- 172.21.15.71:0/1670920714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f00102000 0x7f2f00196900 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:16:40.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.927+0000 7f2f08462700 1 -- 172.21.15.71:0/1670920714 >> 172.21.15.71:0/1670920714 conn(0x7f2f000fa8c0 msgr2=0x7f2f00105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:16:40.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.927+0000 7f2f08462700 1 -- 172.21.15.71:0/1670920714 shutdown_connections 2024-04-23T02:16:40.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:16:40.927+0000 7f2f08462700 1 -- 172.21.15.71:0/1670920714 wait complete. 2024-04-23T02:16:40.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:40 smithi071 ceph-mon[39837]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:16:40.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:40 smithi071 ceph-mon[39837]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:16:40.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:40 smithi071 ceph-mon[39837]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:16:40.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:40 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/31368480' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:41.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:40 smithi192 ceph-mon[49662]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:16:41.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:40 smithi192 ceph-mon[49662]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:16:41.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:40 smithi192 ceph-mon[49662]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:16:41.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:40 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/31368480' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:42.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:42 smithi192 ceph-mon[49662]: pgmap v66: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:42.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:42 smithi192 ceph-mon[49662]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:16:42.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:42 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:42.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:42 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:42.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:42 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:42.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:42 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:42.634 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:42 smithi071 ceph-mon[39837]: pgmap v66: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:42.634 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:42 smithi071 ceph-mon[39837]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:16:42.635 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:42 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:42.635 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:42 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:42.635 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:42 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:42.635 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:42 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:44.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:44 smithi192 ceph-mon[49662]: pgmap v67: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:44.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:44 smithi071 ceph-mon[39837]: pgmap v67: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:46.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: pgmap v68: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:46.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:46.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:46.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:16:46.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:16:46.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:46.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-23T02:16:46.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:16:46.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.629 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.630 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.630 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.630 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.630 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.630 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:46.630 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-23T02:16:46.630 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-23T02:16:46.630 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-23T02:16:46.630 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.630 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.630 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-23T02:16:46.630 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-23T02:16:46.630 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-23T02:16:46.631 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-23T02:16:46.631 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-23T02:16:46.631 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-23T02:16:46.631 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-23T02:16:46.631 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-23T02:16:46.631 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-23T02:16:46.631 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-23T02:16:46.631 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-23T02:16:46.631 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-23T02:16:46.631 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-23T02:16:46.631 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.631 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.631 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-23T02:16:46.631 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-23T02:16:46.632 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-23T02:16:46.632 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-23T02:16:46.632 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-23T02:16:46.632 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-23T02:16:46.632 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-23T02:16:46.632 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-23T02:16:46.632 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-23T02:16:46.632 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-23T02:16:46.632 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-23T02:16:46.632 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.632 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.632 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.632 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.632 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.633 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.633 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.633 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.633 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.633 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.633 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.633 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.633 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-04-23T02:16:46.633 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-04-23T02:16:46.633 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-04-23T02:16:46.633 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-23T02:16:46.633 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-23T02:16:46.633 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-23T02:16:46.634 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:16:46.634 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:16:46.634 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:16:46.634 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:46.634 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:16:46.634 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:16:46.634 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:16:46.634 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:46 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:46.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: pgmap v68: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:46.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:46.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:46.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:16:46.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:16:46.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:46.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-23T02:16:46.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:16:46.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:16:46.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:46.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-23T02:16:46.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-23T02:16:46.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-23T02:16:46.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-23T02:16:46.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-23T02:16:46.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-23T02:16:46.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-23T02:16:46.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-23T02:16:46.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-23T02:16:46.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-23T02:16:46.747 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-23T02:16:46.748 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-23T02:16:46.748 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-23T02:16:46.748 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-23T02:16:46.748 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-23T02:16:46.748 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-23T02:16:46.748 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.748 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.748 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-23T02:16:46.748 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-23T02:16:46.748 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-23T02:16:46.748 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-23T02:16:46.748 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-23T02:16:46.748 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-23T02:16:46.748 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-23T02:16:46.749 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-23T02:16:46.749 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-23T02:16:46.749 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-23T02:16:46.749 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-23T02:16:46.749 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.749 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.749 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.749 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.749 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.749 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.749 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.749 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.749 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.750 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.750 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.750 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:16:46.750 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-04-23T02:16:46.750 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-04-23T02:16:46.750 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-04-23T02:16:46.750 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-23T02:16:46.750 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-23T02:16:46.750 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-23T02:16:46.750 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:16:46.750 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:16:46.750 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:16:46.750 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:46.750 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:16:46.751 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:16:46.751 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:16:46.751 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:46 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:47.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:47 smithi192 ceph-mon[49662]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-23T02:16:47.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:47 smithi192 ceph-mon[49662]: Upgrade: Finalizing container_image settings 2024-04-23T02:16:47.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:47 smithi192 ceph-mon[49662]: Upgrade: Complete! 2024-04-23T02:16:47.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:47 smithi071 ceph-mon[39837]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-23T02:16:47.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:47 smithi071 ceph-mon[39837]: Upgrade: Finalizing container_image settings 2024-04-23T02:16:47.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:47 smithi071 ceph-mon[39837]: Upgrade: Complete! 2024-04-23T02:16:48.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:48 smithi192 ceph-mon[49662]: pgmap v69: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:48.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:48 smithi071 ceph-mon[39837]: pgmap v69: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:50.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:50 smithi192 ceph-mon[49662]: pgmap v70: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:50.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:50 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:16:50.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:50 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:50.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:50 smithi071 ceph-mon[39837]: pgmap v70: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:50.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:50 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:16:50.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:50 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:16:52.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:52 smithi192 ceph-mon[49662]: pgmap v71: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:52.634 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:52 smithi071 ceph-mon[39837]: pgmap v71: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:54.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:54 smithi192 ceph-mon[49662]: pgmap v72: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:54.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:54 smithi071 ceph-mon[39837]: pgmap v72: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:56.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:56 smithi192 ceph-mon[49662]: pgmap v73: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:56.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:56 smithi071 ceph-mon[39837]: pgmap v73: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:16:57.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:57 smithi071 ceph-mon[39837]: pgmap v74: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:57.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:57 smithi192 ceph-mon[49662]: pgmap v74: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:16:59.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:16:59 smithi071 ceph-mon[39837]: pgmap v75: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:00.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:16:59 smithi192 ceph-mon[49662]: pgmap v75: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:02.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:01 smithi192 ceph-mon[49662]: pgmap v76: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:02.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:01 smithi071 ceph-mon[39837]: pgmap v76: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:04.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:03 smithi192 ceph-mon[49662]: pgmap v77: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:04.147 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:03 smithi071 ceph-mon[39837]: pgmap v77: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:05.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:04 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:17:05.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:04 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:17:06.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:05 smithi192 ceph-mon[49662]: pgmap v78: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:06.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:05 smithi071 ceph-mon[39837]: pgmap v78: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:08.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:07 smithi192 ceph-mon[49662]: pgmap v79: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:08.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:07 smithi071 ceph-mon[39837]: pgmap v79: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:10.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:09 smithi192 ceph-mon[49662]: pgmap v80: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:10.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:09 smithi071 ceph-mon[39837]: pgmap v80: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.042+0000 7f00d9edf700 1 -- 172.21.15.71:0/877862656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00d4100dc0 msgr2=0x7f00d4101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.042+0000 7f00d9edf700 1 --2- 172.21.15.71:0/877862656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00d4100dc0 0x7f00d4101220 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009910 tx=0x7f00c4009c20 comp rx=0 tx=0).stop 2024-04-23T02:17:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.042+0000 7f00d9edf700 1 -- 172.21.15.71:0/877862656 shutdown_connections 2024-04-23T02:17:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.042+0000 7f00d9edf700 1 --2- 172.21.15.71:0/877862656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00d4100dc0 0x7f00d4101220 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.042+0000 7f00d9edf700 1 --2- 172.21.15.71:0/877862656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00d40ffbc0 0x7f00d40fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:11.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.042+0000 7f00d9edf700 1 -- 172.21.15.71:0/877862656 >> 172.21.15.71:0/877862656 conn(0x7f00d40fb160 msgr2=0x7f00d40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:11.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.042+0000 7f00d9edf700 1 -- 172.21.15.71:0/877862656 shutdown_connections 2024-04-23T02:17:11.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.042+0000 7f00d9edf700 1 -- 172.21.15.71:0/877862656 wait complete. 2024-04-23T02:17:11.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.043+0000 7f00d9edf700 1 Processor -- start 2024-04-23T02:17:11.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.043+0000 7f00d9edf700 1 -- start start 2024-04-23T02:17:11.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.043+0000 7f00d9edf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00d40ffbc0 0x7f00d4069970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:11.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.043+0000 7f00d9edf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00d4100dc0 0x7f00d419d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:11.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.043+0000 7f00d9edf700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d4197300 con 0x7f00d4100dc0 2024-04-23T02:17:11.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.043+0000 7f00d9edf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d419ddd0 con 0x7f00d40ffbc0 2024-04-23T02:17:11.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.043+0000 7f00d2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00d4100dc0 0x7f00d419d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:11.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.043+0000 7f00d2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00d4100dc0 0x7f00d419d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56112/0 (socket says 172.21.15.71:56112) 2024-04-23T02:17:11.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.043+0000 7f00d2ffd700 1 -- 172.21.15.71:0/540824461 learned_addr learned my addr 172.21.15.71:0/540824461 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:17:11.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.044+0000 7f00d37fe700 1 --2- 172.21.15.71:0/540824461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00d40ffbc0 0x7f00d4069970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:11.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.044+0000 7f00d2ffd700 1 -- 172.21.15.71:0/540824461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00d40ffbc0 msgr2=0x7f00d4069970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:11.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.044+0000 7f00d2ffd700 1 --2- 172.21.15.71:0/540824461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00d40ffbc0 0x7f00d4069970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:11.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.044+0000 7f00d2ffd700 1 -- 172.21.15.71:0/540824461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c40095f0 con 0x7f00d4100dc0 2024-04-23T02:17:11.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.044+0000 7f00d2ffd700 1 --2- 172.21.15.71:0/540824461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00d4100dc0 0x7f00d419d890 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f00c400bd70 tx=0x7f00c400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:11.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.044+0000 7f00d0ff9700 1 -- 172.21.15.71:0/540824461 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00c401d070 con 0x7f00d4100dc0 2024-04-23T02:17:11.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.044+0000 7f00d9edf700 1 -- 172.21.15.71:0/540824461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d419e020 con 0x7f00d4100dc0 2024-04-23T02:17:11.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.044+0000 7f00d0ff9700 1 -- 172.21.15.71:0/540824461 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f00c400f460 con 0x7f00d4100dc0 2024-04-23T02:17:11.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.044+0000 7f00d0ff9700 1 -- 172.21.15.71:0/540824461 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00c4017630 con 0x7f00d4100dc0 2024-04-23T02:17:11.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.044+0000 7f00d9edf700 1 -- 172.21.15.71:0/540824461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d419e510 con 0x7f00d4100dc0 2024-04-23T02:17:11.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.045+0000 7f00d9edf700 1 -- 172.21.15.71:0/540824461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d418f3f0 con 0x7f00d4100dc0 2024-04-23T02:17:11.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.046+0000 7f00d0ff9700 1 -- 172.21.15.71:0/540824461 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f00c400fa20 con 0x7f00d4100dc0 2024-04-23T02:17:11.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.046+0000 7f00d0ff9700 1 --2- 172.21.15.71:0/540824461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f00bc06cce0 0x7f00bc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:11.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.046+0000 7f00d37fe700 1 --2- 172.21.15.71:0/540824461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f00bc06cce0 0x7f00bc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:11.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.046+0000 7f00d0ff9700 1 -- 172.21.15.71:0/540824461 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f00c408d9a0 con 0x7f00d4100dc0 2024-04-23T02:17:11.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.047+0000 7f00d37fe700 1 --2- 172.21.15.71:0/540824461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f00bc06cce0 0x7f00bc06f1a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f00d4100c20 tx=0x7f00c800b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:11.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.051+0000 7f00d0ff9700 1 -- 172.21.15.71:0/540824461 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f00c405b550 con 0x7f00d4100dc0 2024-04-23T02:17:11.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.260+0000 7f00d9edf700 1 -- 172.21.15.71:0/540824461 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f00d402cff0 con 0x7f00bc06cce0 2024-04-23T02:17:11.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.261+0000 7f00d0ff9700 1 -- 172.21.15.71:0/540824461 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f00d402cff0 con 0x7f00bc06cce0 2024-04-23T02:17:11.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.263+0000 7f00d9edf700 1 -- 172.21.15.71:0/540824461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f00bc06cce0 msgr2=0x7f00bc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:11.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.263+0000 7f00d9edf700 1 --2- 172.21.15.71:0/540824461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f00bc06cce0 0x7f00bc06f1a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f00d4100c20 tx=0x7f00c800b410 comp rx=0 tx=0).stop 2024-04-23T02:17:11.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.263+0000 7f00d9edf700 1 -- 172.21.15.71:0/540824461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00d4100dc0 msgr2=0x7f00d419d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:11.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.263+0000 7f00d9edf700 1 --2- 172.21.15.71:0/540824461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00d4100dc0 0x7f00d419d890 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f00c400bd70 tx=0x7f00c400bda0 comp rx=0 tx=0).stop 2024-04-23T02:17:11.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.263+0000 7f00d9edf700 1 -- 172.21.15.71:0/540824461 shutdown_connections 2024-04-23T02:17:11.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.263+0000 7f00d9edf700 1 --2- 172.21.15.71:0/540824461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f00bc06cce0 0x7f00bc06f1a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:11.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.263+0000 7f00d9edf700 1 --2- 172.21.15.71:0/540824461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00d40ffbc0 0x7f00d4069970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:11.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.263+0000 7f00d9edf700 1 --2- 172.21.15.71:0/540824461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00d4100dc0 0x7f00d419d890 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:11.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.263+0000 7f00d9edf700 1 -- 172.21.15.71:0/540824461 >> 172.21.15.71:0/540824461 conn(0x7f00d40fb160 msgr2=0x7f00d41041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:11.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.263+0000 7f00d9edf700 1 -- 172.21.15.71:0/540824461 shutdown_connections 2024-04-23T02:17:11.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:11.263+0000 7f00d9edf700 1 -- 172.21.15.71:0/540824461 wait complete. 2024-04-23T02:17:11.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:11 smithi071 ceph-mon[39837]: pgmap v81: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:11.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:11 smithi071 ceph-mon[39837]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:17:12.066 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-04-23T02:17:12.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:11 smithi192 ceph-mon[49662]: pgmap v81: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:12.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:11 smithi192 ceph-mon[49662]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:17:12.336 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:17:13.861 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:13 smithi071 ceph-mon[39837]: pgmap v82: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:14.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:13 smithi192 ceph-mon[49662]: pgmap v82: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:14.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.141+0000 7f76b0c3a700 1 -- 172.21.15.71:0/3511717832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76ac105bd0 msgr2=0x7f76ac105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:14.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.141+0000 7f76b0c3a700 1 --2- 172.21.15.71:0/3511717832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76ac105bd0 0x7f76ac105fb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f76a400b0a0 tx=0x7f76a400b3b0 comp rx=0 tx=0).stop 2024-04-23T02:17:14.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.142+0000 7f76b0c3a700 1 -- 172.21.15.71:0/3511717832 shutdown_connections 2024-04-23T02:17:14.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.142+0000 7f76b0c3a700 1 --2- 172.21.15.71:0/3511717832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76ac0ffc20 0x7f76ac100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:14.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.142+0000 7f76b0c3a700 1 --2- 172.21.15.71:0/3511717832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76ac105bd0 0x7f76ac105fb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:14.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.142+0000 7f76b0c3a700 1 -- 172.21.15.71:0/3511717832 >> 172.21.15.71:0/3511717832 conn(0x7f76ac075670 msgr2=0x7f76ac075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:14.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.142+0000 7f76b0c3a700 1 -- 172.21.15.71:0/3511717832 shutdown_connections 2024-04-23T02:17:14.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.142+0000 7f76b0c3a700 1 -- 172.21.15.71:0/3511717832 wait complete. 2024-04-23T02:17:14.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.142+0000 7f76b0c3a700 1 Processor -- start 2024-04-23T02:17:14.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.142+0000 7f76b0c3a700 1 -- start start 2024-04-23T02:17:14.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.143+0000 7f76b0c3a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76ac0ffc20 0x7f76ac197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:14.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.143+0000 7f76b0c3a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76ac105bd0 0x7f76ac197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:14.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.143+0000 7f76b0c3a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac1981b0 con 0x7f76ac105bd0 2024-04-23T02:17:14.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.143+0000 7f76b0c3a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac198320 con 0x7f76ac0ffc20 2024-04-23T02:17:14.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.143+0000 7f76a9d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76ac105bd0 0x7f76ac197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:14.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.143+0000 7f76a9d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76ac105bd0 0x7f76ac197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56134/0 (socket says 172.21.15.71:56134) 2024-04-23T02:17:14.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.143+0000 7f76a9d9b700 1 -- 172.21.15.71:0/3424688867 learned_addr learned my addr 172.21.15.71:0/3424688867 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:17:14.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.143+0000 7f76aa59c700 1 --2- 172.21.15.71:0/3424688867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76ac0ffc20 0x7f76ac197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:14.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.143+0000 7f76a9d9b700 1 -- 172.21.15.71:0/3424688867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76ac0ffc20 msgr2=0x7f76ac197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:14.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.143+0000 7f76a9d9b700 1 --2- 172.21.15.71:0/3424688867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76ac0ffc20 0x7f76ac197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:14.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.143+0000 7f76a9d9b700 1 -- 172.21.15.71:0/3424688867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a4009d00 con 0x7f76ac105bd0 2024-04-23T02:17:14.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.143+0000 7f76a9d9b700 1 --2- 172.21.15.71:0/3424688867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76ac105bd0 0x7f76ac197b90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f769400b4b0 tx=0x7f769400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:14.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.143+0000 7f76a37fe700 1 -- 172.21.15.71:0/3424688867 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7694011800 con 0x7f76ac105bd0 2024-04-23T02:17:14.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.143+0000 7f76b0c3a700 1 -- 172.21.15.71:0/3424688867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76ac1a06c0 con 0x7f76ac105bd0 2024-04-23T02:17:14.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.143+0000 7f76a37fe700 1 -- 172.21.15.71:0/3424688867 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7694011e00 con 0x7f76ac105bd0 2024-04-23T02:17:14.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.143+0000 7f76a37fe700 1 -- 172.21.15.71:0/3424688867 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7694010e50 con 0x7f76ac105bd0 2024-04-23T02:17:14.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.143+0000 7f76b0c3a700 1 -- 172.21.15.71:0/3424688867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ac1a0bb0 con 0x7f76ac105bd0 2024-04-23T02:17:14.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.144+0000 7f76b0c3a700 1 -- 172.21.15.71:0/3424688867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76ac1080d0 con 0x7f76ac105bd0 2024-04-23T02:17:14.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.145+0000 7f76a37fe700 1 -- 172.21.15.71:0/3424688867 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7694011960 con 0x7f76ac105bd0 2024-04-23T02:17:14.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.145+0000 7f76a37fe700 1 --2- 172.21.15.71:0/3424688867 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f769806cce0 0x7f769806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:14.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.145+0000 7f76a37fe700 1 -- 172.21.15.71:0/3424688867 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f769408c140 con 0x7f76ac105bd0 2024-04-23T02:17:14.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.145+0000 7f76aa59c700 1 --2- 172.21.15.71:0/3424688867 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f769806cce0 0x7f769806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:14.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.146+0000 7f76aa59c700 1 --2- 172.21.15.71:0/3424688867 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f769806cce0 0x7f769806f1a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f76a400b830 tx=0x7f76a4006000 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:14.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.150+0000 7f76a37fe700 1 -- 172.21.15.71:0/3424688867 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f7694056260 con 0x7f76ac105bd0 2024-04-23T02:17:14.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.434+0000 7f76b0c3a700 1 -- 172.21.15.71:0/3424688867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f76ac04f2e0 con 0x7f76ac105bd0 2024-04-23T02:17:14.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.435+0000 7f76a37fe700 1 -- 172.21.15.71:0/3424688867 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f7694059880 con 0x7f76ac105bd0 2024-04-23T02:17:14.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.437+0000 7f76b0c3a700 1 -- 172.21.15.71:0/3424688867 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f769806cce0 msgr2=0x7f769806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:14.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.437+0000 7f76b0c3a700 1 --2- 172.21.15.71:0/3424688867 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f769806cce0 0x7f769806f1a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f76a400b830 tx=0x7f76a4006000 comp rx=0 tx=0).stop 2024-04-23T02:17:14.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.437+0000 7f76b0c3a700 1 -- 172.21.15.71:0/3424688867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76ac105bd0 msgr2=0x7f76ac197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:14.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.437+0000 7f76b0c3a700 1 --2- 172.21.15.71:0/3424688867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76ac105bd0 0x7f76ac197b90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f769400b4b0 tx=0x7f769400b870 comp rx=0 tx=0).stop 2024-04-23T02:17:14.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.437+0000 7f76b0c3a700 1 -- 172.21.15.71:0/3424688867 shutdown_connections 2024-04-23T02:17:14.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.437+0000 7f76b0c3a700 1 --2- 172.21.15.71:0/3424688867 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f769806cce0 0x7f769806f1a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:14.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.437+0000 7f76b0c3a700 1 --2- 172.21.15.71:0/3424688867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76ac0ffc20 0x7f76ac197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:14.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.437+0000 7f76b0c3a700 1 --2- 172.21.15.71:0/3424688867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76ac105bd0 0x7f76ac197b90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:14.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.437+0000 7f76b0c3a700 1 -- 172.21.15.71:0/3424688867 >> 172.21.15.71:0/3424688867 conn(0x7f76ac075670 msgr2=0x7f76ac0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:14.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.437+0000 7f76b0c3a700 1 -- 172.21.15.71:0/3424688867 shutdown_connections 2024-04-23T02:17:14.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:14.437+0000 7f76b0c3a700 1 -- 172.21.15.71:0/3424688867 wait complete. 2024-04-23T02:17:14.453 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:17:15.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:14 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/3424688867' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:17:15.187 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:14 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3424688867' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:17:15.208 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-04-23T02:17:15.439 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:17:15.973 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:15 smithi071 ceph-mon[39837]: pgmap v83: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:16.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:15 smithi192 ceph-mon[49662]: pgmap v83: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:17.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.359+0000 7f95e229a700 1 -- 172.21.15.71:0/794745443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95dc0ffb40 msgr2=0x7f95dc0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:17.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.359+0000 7f95e229a700 1 --2- 172.21.15.71:0/794745443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95dc0ffb40 0x7f95dc0fffc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009910 tx=0x7f95d0009c20 comp rx=0 tx=0).stop 2024-04-23T02:17:17.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.359+0000 7f95e229a700 1 -- 172.21.15.71:0/794745443 shutdown_connections 2024-04-23T02:17:17.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.359+0000 7f95e229a700 1 --2- 172.21.15.71:0/794745443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95dc0ffb40 0x7f95dc0fffc0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:17.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.359+0000 7f95e229a700 1 --2- 172.21.15.71:0/794745443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95dc105bf0 0x7f95dc105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:17.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.359+0000 7f95e229a700 1 -- 172.21.15.71:0/794745443 >> 172.21.15.71:0/794745443 conn(0x7f95dc0fb610 msgr2=0x7f95dc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:17.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.359+0000 7f95e229a700 1 -- 172.21.15.71:0/794745443 shutdown_connections 2024-04-23T02:17:17.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.359+0000 7f95e229a700 1 -- 172.21.15.71:0/794745443 wait complete. 2024-04-23T02:17:17.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.360+0000 7f95e229a700 1 Processor -- start 2024-04-23T02:17:17.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.360+0000 7f95e229a700 1 -- start start 2024-04-23T02:17:17.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.360+0000 7f95e229a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95dc0ffb40 0x7f95dc1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:17.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.360+0000 7f95e229a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95dc105bf0 0x7f95dc101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:17.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.360+0000 7f95e229a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc102230 con 0x7f95dc105bf0 2024-04-23T02:17:17.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.360+0000 7f95e229a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc102370 con 0x7f95dc0ffb40 2024-04-23T02:17:17.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.360+0000 7f95db7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95dc105bf0 0x7f95dc101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:17.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.360+0000 7f95db7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95dc105bf0 0x7f95dc101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42446/0 (socket says 172.21.15.71:42446) 2024-04-23T02:17:17.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.360+0000 7f95db7fe700 1 -- 172.21.15.71:0/21128600 learned_addr learned my addr 172.21.15.71:0/21128600 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:17:17.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.360+0000 7f95dbfff700 1 --2- 172.21.15.71:0/21128600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95dc0ffb40 0x7f95dc1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:17.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.361+0000 7f95db7fe700 1 -- 172.21.15.71:0/21128600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95dc0ffb40 msgr2=0x7f95dc1017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:17.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.361+0000 7f95db7fe700 1 --2- 172.21.15.71:0/21128600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95dc0ffb40 0x7f95dc1017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:17.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.361+0000 7f95db7fe700 1 -- 172.21.15.71:0/21128600 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d00095f0 con 0x7f95dc105bf0 2024-04-23T02:17:17.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.361+0000 7f95db7fe700 1 --2- 172.21.15.71:0/21128600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95dc105bf0 0x7f95dc101cf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f95d0006010 tx=0x7f95d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:17.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.361+0000 7f95d97fa700 1 -- 172.21.15.71:0/21128600 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95d001d070 con 0x7f95dc105bf0 2024-04-23T02:17:17.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.361+0000 7f95e229a700 1 -- 172.21.15.71:0/21128600 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95dc1025f0 con 0x7f95dc105bf0 2024-04-23T02:17:17.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.361+0000 7f95d97fa700 1 -- 172.21.15.71:0/21128600 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f95d000f460 con 0x7f95dc105bf0 2024-04-23T02:17:17.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.361+0000 7f95d97fa700 1 -- 172.21.15.71:0/21128600 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95d0021640 con 0x7f95dc105bf0 2024-04-23T02:17:17.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.361+0000 7f95e229a700 1 -- 172.21.15.71:0/21128600 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95dc1a2d50 con 0x7f95dc105bf0 2024-04-23T02:17:17.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.362+0000 7f95e229a700 1 -- 172.21.15.71:0/21128600 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95dc0679c0 con 0x7f95dc105bf0 2024-04-23T02:17:17.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.362+0000 7f95d97fa700 1 -- 172.21.15.71:0/21128600 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f95d000fa20 con 0x7f95dc105bf0 2024-04-23T02:17:17.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.363+0000 7f95d97fa700 1 --2- 172.21.15.71:0/21128600 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f95c406cce0 0x7f95c406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:17.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.363+0000 7f95d97fa700 1 -- 172.21.15.71:0/21128600 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f95d008d800 con 0x7f95dc105bf0 2024-04-23T02:17:17.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.363+0000 7f95dbfff700 1 --2- 172.21.15.71:0/21128600 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f95c406cce0 0x7f95c406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:17.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.363+0000 7f95dbfff700 1 --2- 172.21.15.71:0/21128600 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f95c406cce0 0x7f95c406f1a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f95cc006fd0 tx=0x7f95cc008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:17.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.367+0000 7f95d97fa700 1 -- 172.21.15.71:0/21128600 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f95d0057920 con 0x7f95dc105bf0 2024-04-23T02:17:17.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.641+0000 7f95e229a700 1 -- 172.21.15.71:0/21128600 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f95dc04f2e0 con 0x7f95dc105bf0 2024-04-23T02:17:17.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.642+0000 7f95d97fa700 1 -- 172.21.15.71:0/21128600 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f95d005af40 con 0x7f95dc105bf0 2024-04-23T02:17:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.644+0000 7f95e229a700 1 -- 172.21.15.71:0/21128600 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f95c406cce0 msgr2=0x7f95c406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.644+0000 7f95e229a700 1 --2- 172.21.15.71:0/21128600 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f95c406cce0 0x7f95c406f1a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f95cc006fd0 tx=0x7f95cc008040 comp rx=0 tx=0).stop 2024-04-23T02:17:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.644+0000 7f95e229a700 1 -- 172.21.15.71:0/21128600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95dc105bf0 msgr2=0x7f95dc101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.644+0000 7f95e229a700 1 --2- 172.21.15.71:0/21128600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95dc105bf0 0x7f95dc101cf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f95d0006010 tx=0x7f95d000be20 comp rx=0 tx=0).stop 2024-04-23T02:17:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.644+0000 7f95e229a700 1 -- 172.21.15.71:0/21128600 shutdown_connections 2024-04-23T02:17:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.644+0000 7f95e229a700 1 --2- 172.21.15.71:0/21128600 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f95c406cce0 0x7f95c406f1a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:17.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.644+0000 7f95e229a700 1 --2- 172.21.15.71:0/21128600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95dc0ffb40 0x7f95dc1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:17.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.644+0000 7f95e229a700 1 --2- 172.21.15.71:0/21128600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95dc105bf0 0x7f95dc101cf0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:17.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.644+0000 7f95e229a700 1 -- 172.21.15.71:0/21128600 >> 172.21.15.71:0/21128600 conn(0x7f95dc0fb610 msgr2=0x7f95dc069030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:17.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.644+0000 7f95e229a700 1 -- 172.21.15.71:0/21128600 shutdown_connections 2024-04-23T02:17:17.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:17.644+0000 7f95e229a700 1 -- 172.21.15.71:0/21128600 wait complete. 2024-04-23T02:17:17.659 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)" 2024-04-23T02:17:17.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:17 smithi071 ceph-mon[39837]: pgmap v84: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:17.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:17 smithi071 ceph-mon[39837]: fsmap cephfs:2 {0=cephfs.smithi071.vxejye=up:active,1=cephfs.smithi192.gtmyst=up:active} 2 up:standby-replay 2024-04-23T02:17:17.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:17 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/21128600' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:17:18.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:17 smithi192 ceph-mon[49662]: pgmap v84: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:18.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:17 smithi192 ceph-mon[49662]: fsmap cephfs:2 {0=cephfs.smithi071.vxejye=up:active,1=cephfs.smithi192.gtmyst=up:active} 2 up:standby-replay 2024-04-23T02:17:18.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:17 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/21128600' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:17:18.478 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-04-23T02:17:18.706 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:17:20.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:19 smithi192 ceph-mon[49662]: pgmap v85: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:20.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:19 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:17:20.206 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:19 smithi071 ceph-mon[39837]: pgmap v85: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:20.207 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:19 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:17:20.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.811+0000 7fd18040f700 1 -- 172.21.15.71:0/3790017035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd178100e90 msgr2=0x7fd178101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:20.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.811+0000 7fd18040f700 1 --2- 172.21.15.71:0/3790017035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd178100e90 0x7fd178101270 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd174009910 tx=0x7fd174009c20 comp rx=0 tx=0).stop 2024-04-23T02:17:20.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.812+0000 7fd18040f700 1 -- 172.21.15.71:0/3790017035 shutdown_connections 2024-04-23T02:17:20.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.812+0000 7fd18040f700 1 --2- 172.21.15.71:0/3790017035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd178101840 0x7fd178105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:20.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.812+0000 7fd18040f700 1 --2- 172.21.15.71:0/3790017035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd178100e90 0x7fd178101270 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:20.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.812+0000 7fd18040f700 1 -- 172.21.15.71:0/3790017035 >> 172.21.15.71:0/3790017035 conn(0x7fd1780fc740 msgr2=0x7fd1780feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:20.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.812+0000 7fd18040f700 1 -- 172.21.15.71:0/3790017035 shutdown_connections 2024-04-23T02:17:20.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.812+0000 7fd18040f700 1 -- 172.21.15.71:0/3790017035 wait complete. 2024-04-23T02:17:20.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.812+0000 7fd18040f700 1 Processor -- start 2024-04-23T02:17:20.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.812+0000 7fd18040f700 1 -- start start 2024-04-23T02:17:20.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.812+0000 7fd18040f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd178100e90 0x7fd17819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:20.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.812+0000 7fd18040f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd178101840 0x7fd17819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:20.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.812+0000 7fd18040f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17819d100 con 0x7fd178101840 2024-04-23T02:17:20.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.812+0000 7fd18040f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd178196640 con 0x7fd178100e90 2024-04-23T02:17:20.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.812+0000 7fd17d9aa700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd178101840 0x7fd17819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:20.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.813+0000 7fd17d9aa700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd178101840 0x7fd17819cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42462/0 (socket says 172.21.15.71:42462) 2024-04-23T02:17:20.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.813+0000 7fd17d9aa700 1 -- 172.21.15.71:0/1334813669 learned_addr learned my addr 172.21.15.71:0/1334813669 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:17:20.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.813+0000 7fd17e1ab700 1 --2- 172.21.15.71:0/1334813669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd178100e90 0x7fd17819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:20.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.813+0000 7fd17d9aa700 1 -- 172.21.15.71:0/1334813669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd178100e90 msgr2=0x7fd17819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:20.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.813+0000 7fd17d9aa700 1 --2- 172.21.15.71:0/1334813669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd178100e90 0x7fd17819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:20.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.813+0000 7fd17d9aa700 1 -- 172.21.15.71:0/1334813669 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1740095f0 con 0x7fd178101840 2024-04-23T02:17:20.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.813+0000 7fd17e1ab700 1 --2- 172.21.15.71:0/1334813669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd178100e90 0x7fd17819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:17:20.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.813+0000 7fd17d9aa700 1 --2- 172.21.15.71:0/1334813669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd178101840 0x7fd17819cb00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd16800c960 tx=0x7fd16800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:20.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.813+0000 7fd16f7fe700 1 -- 172.21.15.71:0/1334813669 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd168007a70 con 0x7fd178101840 2024-04-23T02:17:20.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.813+0000 7fd16f7fe700 1 -- 172.21.15.71:0/1334813669 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd168007bd0 con 0x7fd178101840 2024-04-23T02:17:20.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.813+0000 7fd18040f700 1 -- 172.21.15.71:0/1334813669 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd178196920 con 0x7fd178101840 2024-04-23T02:17:20.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.813+0000 7fd16f7fe700 1 -- 172.21.15.71:0/1334813669 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd168018630 con 0x7fd178101840 2024-04-23T02:17:20.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.813+0000 7fd18040f700 1 -- 172.21.15.71:0/1334813669 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd178196e70 con 0x7fd178101840 2024-04-23T02:17:20.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.814+0000 7fd18040f700 1 -- 172.21.15.71:0/1334813669 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd178109230 con 0x7fd178101840 2024-04-23T02:17:20.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.814+0000 7fd16f7fe700 1 -- 172.21.15.71:0/1334813669 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd16800f9f0 con 0x7fd178101840 2024-04-23T02:17:20.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.815+0000 7fd16f7fe700 1 --2- 172.21.15.71:0/1334813669 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd16406cce0 0x7fd16406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:20.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.815+0000 7fd16f7fe700 1 -- 172.21.15.71:0/1334813669 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd16808c4a0 con 0x7fd178101840 2024-04-23T02:17:20.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.815+0000 7fd17e1ab700 1 --2- 172.21.15.71:0/1334813669 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd16406cce0 0x7fd16406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:20.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.815+0000 7fd17e1ab700 1 --2- 172.21.15.71:0/1334813669 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd16406cce0 0x7fd16406f1a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd174006010 tx=0x7fd17401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:20.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:20.819+0000 7fd16f7fe700 1 -- 172.21.15.71:0/1334813669 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fd168059e40 con 0x7fd178101840 2024-04-23T02:17:21.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:21.094+0000 7fd18040f700 1 -- 172.21.15.71:0/1334813669 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd178197c40 con 0x7fd178101840 2024-04-23T02:17:21.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:21.095+0000 7fd16f7fe700 1 -- 172.21.15.71:0/1334813669 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7fd168059c60 con 0x7fd178101840 2024-04-23T02:17:21.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:21.097+0000 7fd18040f700 1 -- 172.21.15.71:0/1334813669 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd16406cce0 msgr2=0x7fd16406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:21.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:21.097+0000 7fd18040f700 1 --2- 172.21.15.71:0/1334813669 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd16406cce0 0x7fd16406f1a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd174006010 tx=0x7fd17401a040 comp rx=0 tx=0).stop 2024-04-23T02:17:21.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:21.097+0000 7fd18040f700 1 -- 172.21.15.71:0/1334813669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd178101840 msgr2=0x7fd17819cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:21.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:21.097+0000 7fd18040f700 1 --2- 172.21.15.71:0/1334813669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd178101840 0x7fd17819cb00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd16800c960 tx=0x7fd16800cd20 comp rx=0 tx=0).stop 2024-04-23T02:17:21.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:21.097+0000 7fd18040f700 1 -- 172.21.15.71:0/1334813669 shutdown_connections 2024-04-23T02:17:21.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:21.097+0000 7fd18040f700 1 --2- 172.21.15.71:0/1334813669 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd16406cce0 0x7fd16406f1a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:21.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:21.097+0000 7fd18040f700 1 --2- 172.21.15.71:0/1334813669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd178100e90 0x7fd17819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:21.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:21.097+0000 7fd18040f700 1 --2- 172.21.15.71:0/1334813669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd178101840 0x7fd17819cb00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:21.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:21.097+0000 7fd18040f700 1 -- 172.21.15.71:0/1334813669 >> 172.21.15.71:0/1334813669 conn(0x7fd1780fc740 msgr2=0x7fd1780fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:21.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:21.097+0000 7fd18040f700 1 -- 172.21.15.71:0/1334813669 shutdown_connections 2024-04-23T02:17:21.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:21.097+0000 7fd18040f700 1 -- 172.21.15.71:0/1334813669 wait complete. 2024-04-23T02:17:21.112 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:17:21.833 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-04-23T02:17:22.062 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:17:22.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:21 smithi192 ceph-mon[49662]: pgmap v86: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:22.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:21 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/1334813669' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:17:22.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:21 smithi071 ceph-mon[39837]: pgmap v86: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:22.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:21 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/1334813669' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:17:23.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.853+0000 7fd6e6551700 1 -- 172.21.15.71:0/3702628672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6e0100e90 msgr2=0x7fd6e0101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:23.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.853+0000 7fd6e6551700 1 --2- 172.21.15.71:0/3702628672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6e0100e90 0x7fd6e0101270 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009910 tx=0x7fd6d0009c20 comp rx=0 tx=0).stop 2024-04-23T02:17:23.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.853+0000 7fd6e6551700 1 -- 172.21.15.71:0/3702628672 shutdown_connections 2024-04-23T02:17:23.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.853+0000 7fd6e6551700 1 --2- 172.21.15.71:0/3702628672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6e0101840 0x7fd6e0105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:23.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.853+0000 7fd6e6551700 1 --2- 172.21.15.71:0/3702628672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6e0100e90 0x7fd6e0101270 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:23.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.853+0000 7fd6e6551700 1 -- 172.21.15.71:0/3702628672 >> 172.21.15.71:0/3702628672 conn(0x7fd6e00fc740 msgr2=0x7fd6e00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:23.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.853+0000 7fd6e6551700 1 -- 172.21.15.71:0/3702628672 shutdown_connections 2024-04-23T02:17:23.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.853+0000 7fd6e6551700 1 -- 172.21.15.71:0/3702628672 wait complete. 2024-04-23T02:17:23.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.854+0000 7fd6e6551700 1 Processor -- start 2024-04-23T02:17:23.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.854+0000 7fd6e6551700 1 -- start start 2024-04-23T02:17:23.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.854+0000 7fd6e6551700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6e0100e90 0x7fd6e019c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:23.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.854+0000 7fd6e6551700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6e0101840 0x7fd6e019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:23.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.854+0000 7fd6e6551700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e019d0b0 con 0x7fd6e0100e90 2024-04-23T02:17:23.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.854+0000 7fd6e6551700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e01965f0 con 0x7fd6e0101840 2024-04-23T02:17:23.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.854+0000 7fd6dffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6e0100e90 0x7fd6e019c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:23.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.854+0000 7fd6dffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6e0100e90 0x7fd6e019c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42490/0 (socket says 172.21.15.71:42490) 2024-04-23T02:17:23.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.854+0000 7fd6dffff700 1 -- 172.21.15.71:0/3404712601 learned_addr learned my addr 172.21.15.71:0/3404712601 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:17:23.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.854+0000 7fd6dffff700 1 -- 172.21.15.71:0/3404712601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6e0101840 msgr2=0x7fd6e019cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:23.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.854+0000 7fd6dffff700 1 --2- 172.21.15.71:0/3404712601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6e0101840 0x7fd6e019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:23.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.854+0000 7fd6dffff700 1 -- 172.21.15.71:0/3404712601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d00095f0 con 0x7fd6e0100e90 2024-04-23T02:17:23.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.854+0000 7fd6dffff700 1 --2- 172.21.15.71:0/3404712601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6e0100e90 0x7fd6e019c570 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0006010 tx=0x7fd6d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:23.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.855+0000 7fd6ddffb700 1 -- 172.21.15.71:0/3404712601 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6d001d070 con 0x7fd6e0100e90 2024-04-23T02:17:23.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.855+0000 7fd6e6551700 1 -- 172.21.15.71:0/3404712601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e0196870 con 0x7fd6e0100e90 2024-04-23T02:17:23.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.855+0000 7fd6ddffb700 1 -- 172.21.15.71:0/3404712601 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd6d000f460 con 0x7fd6e0100e90 2024-04-23T02:17:23.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.855+0000 7fd6ddffb700 1 -- 172.21.15.71:0/3404712601 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6d0021640 con 0x7fd6e0100e90 2024-04-23T02:17:23.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.855+0000 7fd6e6551700 1 -- 172.21.15.71:0/3404712601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e0196d60 con 0x7fd6e0100e90 2024-04-23T02:17:23.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.856+0000 7fd6e6551700 1 -- 172.21.15.71:0/3404712601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e0109230 con 0x7fd6e0100e90 2024-04-23T02:17:23.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.856+0000 7fd6ddffb700 1 -- 172.21.15.71:0/3404712601 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd6d000fa60 con 0x7fd6e0100e90 2024-04-23T02:17:23.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.856+0000 7fd6ddffb700 1 --2- 172.21.15.71:0/3404712601 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6cc06cd30 0x7fd6cc06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:23.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.856+0000 7fd6ddffb700 1 -- 172.21.15.71:0/3404712601 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd6d008d860 con 0x7fd6e0100e90 2024-04-23T02:17:23.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.857+0000 7fd6d7fff700 1 --2- 172.21.15.71:0/3404712601 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6cc06cd30 0x7fd6cc06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:23.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.857+0000 7fd6d7fff700 1 --2- 172.21.15.71:0/3404712601 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6cc06cd30 0x7fd6cc06f1f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8005fd0 tx=0x7fd6c8005d00 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:23.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:23.861+0000 7fd6ddffb700 1 -- 172.21.15.71:0/3404712601 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fd6d005b370 con 0x7fd6e0100e90 2024-04-23T02:17:23.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:23 smithi071 ceph-mon[39837]: pgmap v87: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:24.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:24.071+0000 7fd6e6551700 1 -- 172.21.15.71:0/3404712601 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6e0197be0 con 0x7fd6cc06cd30 2024-04-23T02:17:24.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:23 smithi192 ceph-mon[49662]: pgmap v87: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:25.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:24 smithi192 ceph-mon[49662]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:17:25.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:24 smithi071 ceph-mon[39837]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:17:25.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:25.591+0000 7fd6ddffb700 1 -- 172.21.15.71:0/3404712601 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5393 (secure 0 0 0) 0x7fd6e0197be0 con 0x7fd6cc06cd30 2024-04-23T02:17:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:25.593+0000 7fd6e6551700 1 -- 172.21.15.71:0/3404712601 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6cc06cd30 msgr2=0x7fd6cc06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:25.593+0000 7fd6e6551700 1 --2- 172.21.15.71:0/3404712601 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6cc06cd30 0x7fd6cc06f1f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8005fd0 tx=0x7fd6c8005d00 comp rx=0 tx=0).stop 2024-04-23T02:17:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:25.593+0000 7fd6e6551700 1 -- 172.21.15.71:0/3404712601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6e0100e90 msgr2=0x7fd6e019c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:25.593+0000 7fd6e6551700 1 --2- 172.21.15.71:0/3404712601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6e0100e90 0x7fd6e019c570 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0006010 tx=0x7fd6d000be20 comp rx=0 tx=0).stop 2024-04-23T02:17:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:25.593+0000 7fd6e6551700 1 -- 172.21.15.71:0/3404712601 shutdown_connections 2024-04-23T02:17:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:25.593+0000 7fd6e6551700 1 --2- 172.21.15.71:0/3404712601 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6cc06cd30 0x7fd6cc06f1f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:25.593+0000 7fd6e6551700 1 --2- 172.21.15.71:0/3404712601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6e0100e90 0x7fd6e019c570 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:25.593+0000 7fd6e6551700 1 --2- 172.21.15.71:0/3404712601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6e0101840 0x7fd6e019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:25.593+0000 7fd6e6551700 1 -- 172.21.15.71:0/3404712601 >> 172.21.15.71:0/3404712601 conn(0x7fd6e00fc740 msgr2=0x7fd6e00fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:25.593+0000 7fd6e6551700 1 -- 172.21.15.71:0/3404712601 shutdown_connections 2024-04-23T02:17:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:25.593+0000 7fd6e6551700 1 -- 172.21.15.71:0/3404712601 wait complete. 2024-04-23T02:17:25.608 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:17:26.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:25 smithi192 ceph-mon[49662]: pgmap v88: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:26.166 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:25 smithi071 ceph-mon[39837]: pgmap v88: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:26.185 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph orch ps' 2024-04-23T02:17:26.413 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:17:28.062 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:27 smithi071 ceph-mon[39837]: pgmap v89: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:28.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:27 smithi192 ceph-mon[49662]: pgmap v89: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:28.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.355+0000 7f702c7a1700 1 -- 172.21.15.71:0/3355298853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7024105bf0 msgr2=0x7f7024105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.355+0000 7f702c7a1700 1 --2- 172.21.15.71:0/3355298853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7024105bf0 0x7f7024105fd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7014009910 tx=0x7f7014009c20 comp rx=0 tx=0).stop 2024-04-23T02:17:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.355+0000 7f702c7a1700 1 -- 172.21.15.71:0/3355298853 shutdown_connections 2024-04-23T02:17:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.355+0000 7f702c7a1700 1 --2- 172.21.15.71:0/3355298853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70240ffb40 0x7f70240fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.355+0000 7f702c7a1700 1 --2- 172.21.15.71:0/3355298853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7024105bf0 0x7f7024105fd0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.355+0000 7f702c7a1700 1 -- 172.21.15.71:0/3355298853 >> 172.21.15.71:0/3355298853 conn(0x7f70240fb610 msgr2=0x7f70240fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.355+0000 7f702c7a1700 1 -- 172.21.15.71:0/3355298853 shutdown_connections 2024-04-23T02:17:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.355+0000 7f702c7a1700 1 -- 172.21.15.71:0/3355298853 wait complete. 2024-04-23T02:17:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.355+0000 7f702c7a1700 1 Processor -- start 2024-04-23T02:17:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.355+0000 7f702c7a1700 1 -- start start 2024-04-23T02:17:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.356+0000 7f702c7a1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70240ffb40 0x7f702419b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.356+0000 7f702c7a1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7024105bf0 0x7f702419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.356+0000 7f702c7a1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702419c0e0 con 0x7f70240ffb40 2024-04-23T02:17:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.356+0000 7f702c7a1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7024195590 con 0x7f7024105bf0 2024-04-23T02:17:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.356+0000 7f702a53d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70240ffb40 0x7f702419b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.356+0000 7f702a53d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70240ffb40 0x7f702419b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50382/0 (socket says 172.21.15.71:50382) 2024-04-23T02:17:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.356+0000 7f702a53d700 1 -- 172.21.15.71:0/3371617807 learned_addr learned my addr 172.21.15.71:0/3371617807 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:17:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.356+0000 7f7029d3c700 1 --2- 172.21.15.71:0/3371617807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7024105bf0 0x7f702419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.356+0000 7f702a53d700 1 -- 172.21.15.71:0/3371617807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7024105bf0 msgr2=0x7f702419ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.356+0000 7f702a53d700 1 --2- 172.21.15.71:0/3371617807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7024105bf0 0x7f702419ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.356+0000 7f702a53d700 1 -- 172.21.15.71:0/3371617807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70140095f0 con 0x7f70240ffb40 2024-04-23T02:17:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.356+0000 7f702a53d700 1 --2- 172.21.15.71:0/3371617807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70240ffb40 0x7f702419b510 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7014006010 tx=0x7f701400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.356+0000 7f701b7fe700 1 -- 172.21.15.71:0/3371617807 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f701401f070 con 0x7f70240ffb40 2024-04-23T02:17:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.356+0000 7f701b7fe700 1 -- 172.21.15.71:0/3371617807 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7014019610 con 0x7f70240ffb40 2024-04-23T02:17:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.356+0000 7f702c7a1700 1 -- 172.21.15.71:0/3371617807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7024195810 con 0x7f70240ffb40 2024-04-23T02:17:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.356+0000 7f701b7fe700 1 -- 172.21.15.71:0/3371617807 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f701400f670 con 0x7f70240ffb40 2024-04-23T02:17:28.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.356+0000 7f702c7a1700 1 -- 172.21.15.71:0/3371617807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7024195d00 con 0x7f70240ffb40 2024-04-23T02:17:28.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.357+0000 7f702c7a1700 1 -- 172.21.15.71:0/3371617807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f702404f2e0 con 0x7f70240ffb40 2024-04-23T02:17:28.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.358+0000 7f701b7fe700 1 -- 172.21.15.71:0/3371617807 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f7014019c10 con 0x7f70240ffb40 2024-04-23T02:17:28.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.358+0000 7f701b7fe700 1 --2- 172.21.15.71:0/3371617807 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f701006cce0 0x7f701006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:28.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.358+0000 7f701b7fe700 1 -- 172.21.15.71:0/3371617807 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f701408fa30 con 0x7f70240ffb40 2024-04-23T02:17:28.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.358+0000 7f7029d3c700 1 --2- 172.21.15.71:0/3371617807 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f701006cce0 0x7f701006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:28.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.359+0000 7f7029d3c700 1 --2- 172.21.15.71:0/3371617807 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f701006cce0 0x7f701006f1a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7020006fd0 tx=0x7f7020008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:28.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.363+0000 7f701b7fe700 1 -- 172.21.15.71:0/3371617807 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f701405d5e0 con 0x7f70240ffb40 2024-04-23T02:17:28.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.565+0000 7f702c7a1700 1 -- 172.21.15.71:0/3371617807 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7024196100 con 0x7f701006cce0 2024-04-23T02:17:28.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.575+0000 7f701b7fe700 1 -- 172.21.15.71:0/3371617807 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7024196100 con 0x7f701006cce0 2024-04-23T02:17:28.580 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:17:28.580 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (70s) 47s ago 12m 16.9M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:17:28.580 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (12m) 47s ago 12m 7977k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:17:28.580 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (11m) 116s ago 11m 7796k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:17:28.580 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (12m) 47s ago 12m 6656k - 18.2.1 5be31c24972a 258512dcbc00 2024-04-23T02:17:28.580 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (11m) 116s ago 11m 6656k - 18.2.1 5be31c24972a feea2c9896a6 2024-04-23T02:17:28.580 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (53s) 47s ago 12m 39.0M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:17:28.580 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (6m) 47s ago 6m 64.4M - 18.2.1 5be31c24972a a515de2a841c 2024-04-23T02:17:28.580 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (6m) 47s ago 6m 133M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:17:28.580 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (6m) 116s ago 6m 68.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:17:28.581 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (6m) 116s ago 6m 125M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:17:28.581 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (3m) 47s ago 14m 498M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:17:28.581 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (2m) 116s ago 11m 449M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:17:28.581 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (14m) 47s ago 14m 63.4M 2048M 18.2.1 5be31c24972a ff40abe63fd9 2024-04-23T02:17:28.581 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (11m) 116s ago 11m 46.7M 2048M 18.2.1 5be31c24972a 9c03e90acc95 2024-04-23T02:17:28.581 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (2m) 47s ago 12m 9659k - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:17:28.581 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (2m) 116s ago 11m 3510k - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:17:28.581 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (10m) 47s ago 10m 196M 1227M 18.2.1 5be31c24972a d3d5f2fa7267 2024-04-23T02:17:28.581 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (10m) 47s ago 10m 208M 1227M 18.2.1 5be31c24972a 3992157ff0fd 2024-04-23T02:17:28.581 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (9m) 47s ago 9m 164M 1227M 18.2.1 5be31c24972a 12380b6a15a2 2024-04-23T02:17:28.581 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (9m) 116s ago 9m 240M 2251M 18.2.1 5be31c24972a d5f587ee3a22 2024-04-23T02:17:28.581 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (8m) 116s ago 8m 217M 2251M 18.2.1 5be31c24972a 958d046005f0 2024-04-23T02:17:28.581 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (8m) 116s ago 8m 195M 2251M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:17:28.581 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (88s) 47s ago 11m 46.6M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:17:28.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.577+0000 7f702c7a1700 1 -- 172.21.15.71:0/3371617807 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f701006cce0 msgr2=0x7f701006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:28.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.577+0000 7f702c7a1700 1 --2- 172.21.15.71:0/3371617807 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f701006cce0 0x7f701006f1a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7020006fd0 tx=0x7f7020008040 comp rx=0 tx=0).stop 2024-04-23T02:17:28.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.577+0000 7f702c7a1700 1 -- 172.21.15.71:0/3371617807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70240ffb40 msgr2=0x7f702419b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:28.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.577+0000 7f702c7a1700 1 --2- 172.21.15.71:0/3371617807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70240ffb40 0x7f702419b510 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7014006010 tx=0x7f701400be20 comp rx=0 tx=0).stop 2024-04-23T02:17:28.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.577+0000 7f702c7a1700 1 -- 172.21.15.71:0/3371617807 shutdown_connections 2024-04-23T02:17:28.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.577+0000 7f702c7a1700 1 --2- 172.21.15.71:0/3371617807 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f701006cce0 0x7f701006f1a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:28.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.577+0000 7f702c7a1700 1 --2- 172.21.15.71:0/3371617807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70240ffb40 0x7f702419b510 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:28.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.577+0000 7f702c7a1700 1 --2- 172.21.15.71:0/3371617807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7024105bf0 0x7f702419ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:28.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.577+0000 7f702c7a1700 1 -- 172.21.15.71:0/3371617807 >> 172.21.15.71:0/3371617807 conn(0x7f70240fb610 msgr2=0x7f70240fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:28.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.577+0000 7f702c7a1700 1 -- 172.21.15.71:0/3371617807 shutdown_connections 2024-04-23T02:17:28.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:28.577+0000 7f702c7a1700 1 -- 172.21.15.71:0/3371617807 wait complete. 2024-04-23T02:17:28.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:28 smithi071 ceph-mon[39837]: fsmap cephfs:2 {0=cephfs.smithi071.vxejye=up:active,1=cephfs.smithi192.gtmyst=up:active} 2 up:standby-replay 2024-04-23T02:17:29.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:28 smithi192 ceph-mon[49662]: fsmap cephfs:2 {0=cephfs.smithi071.vxejye=up:active,1=cephfs.smithi192.gtmyst=up:active} 2 up:standby-replay 2024-04-23T02:17:29.370 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-23T02:17:29.371 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi071.front.sepia.ceph.com 2024-04-23T02:17:29.371 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-04-23T02:17:29.608 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:17:30.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:29 smithi192 ceph-mon[49662]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:17:30.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:29 smithi192 ceph-mon[49662]: pgmap v90: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:30.154 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:29 smithi071 ceph-mon[39837]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:17:30.154 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:29 smithi071 ceph-mon[39837]: pgmap v90: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:31.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.546+0000 7f258c3f0700 1 -- 172.21.15.71:0/1318061758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2584100e90 msgr2=0x7f2584101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:31.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.546+0000 7f258c3f0700 1 --2- 172.21.15.71:0/1318061758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2584100e90 0x7f2584101270 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2580009910 tx=0x7f2580009c20 comp rx=0 tx=0).stop 2024-04-23T02:17:31.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.547+0000 7f258c3f0700 1 -- 172.21.15.71:0/1318061758 shutdown_connections 2024-04-23T02:17:31.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.547+0000 7f258c3f0700 1 --2- 172.21.15.71:0/1318061758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2584101840 0x7f2584105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:31.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.547+0000 7f258c3f0700 1 --2- 172.21.15.71:0/1318061758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2584100e90 0x7f2584101270 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:31.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.547+0000 7f258c3f0700 1 -- 172.21.15.71:0/1318061758 >> 172.21.15.71:0/1318061758 conn(0x7f25840fc740 msgr2=0x7f25840feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:31.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.547+0000 7f258c3f0700 1 -- 172.21.15.71:0/1318061758 shutdown_connections 2024-04-23T02:17:31.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.547+0000 7f258c3f0700 1 -- 172.21.15.71:0/1318061758 wait complete. 2024-04-23T02:17:31.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.547+0000 7f258c3f0700 1 Processor -- start 2024-04-23T02:17:31.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.547+0000 7f258c3f0700 1 -- start start 2024-04-23T02:17:31.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.548+0000 7f258c3f0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2584100e90 0x7f258419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:31.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.548+0000 7f258c3f0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2584101840 0x7f258419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:31.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.548+0000 7f258c3f0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258419d230 con 0x7f2584101840 2024-04-23T02:17:31.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.548+0000 7f258c3f0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25841966e0 con 0x7f2584100e90 2024-04-23T02:17:31.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.548+0000 7f258998b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2584101840 0x7f258419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:31.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.548+0000 7f258998b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2584101840 0x7f258419cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50390/0 (socket says 172.21.15.71:50390) 2024-04-23T02:17:31.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.548+0000 7f258998b700 1 -- 172.21.15.71:0/3753946171 learned_addr learned my addr 172.21.15.71:0/3753946171 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:17:31.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.548+0000 7f258a18c700 1 --2- 172.21.15.71:0/3753946171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2584100e90 0x7f258419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:31.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.548+0000 7f258998b700 1 -- 172.21.15.71:0/3753946171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2584100e90 msgr2=0x7f258419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:31.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.548+0000 7f258998b700 1 --2- 172.21.15.71:0/3753946171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2584100e90 0x7f258419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:31.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.548+0000 7f258998b700 1 -- 172.21.15.71:0/3753946171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25800095f0 con 0x7f2584101840 2024-04-23T02:17:31.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.548+0000 7f258998b700 1 --2- 172.21.15.71:0/3753946171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2584101840 0x7f258419cba0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f257400b480 tx=0x7f257400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:31.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.548+0000 7f257b7fe700 1 -- 172.21.15.71:0/3753946171 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25740118c0 con 0x7f2584101840 2024-04-23T02:17:31.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.548+0000 7f257b7fe700 1 -- 172.21.15.71:0/3753946171 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2574010380 con 0x7f2584101840 2024-04-23T02:17:31.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.548+0000 7f258c3f0700 1 -- 172.21.15.71:0/3753946171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25841969c0 con 0x7f2584101840 2024-04-23T02:17:31.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.548+0000 7f257b7fe700 1 -- 172.21.15.71:0/3753946171 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f257400f510 con 0x7f2584101840 2024-04-23T02:17:31.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.548+0000 7f258c3f0700 1 -- 172.21.15.71:0/3753946171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2584196f10 con 0x7f2584101840 2024-04-23T02:17:31.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.549+0000 7f258c3f0700 1 -- 172.21.15.71:0/3753946171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2584109230 con 0x7f2584101840 2024-04-23T02:17:31.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.550+0000 7f257b7fe700 1 -- 172.21.15.71:0/3753946171 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f25740104f0 con 0x7f2584101840 2024-04-23T02:17:31.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.550+0000 7f257b7fe700 1 --2- 172.21.15.71:0/3753946171 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f257006cce0 0x7f257006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:31.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.550+0000 7f257b7fe700 1 -- 172.21.15.71:0/3753946171 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f257408c070 con 0x7f2584101840 2024-04-23T02:17:31.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.550+0000 7f258a18c700 1 --2- 172.21.15.71:0/3753946171 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f257006cce0 0x7f257006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:31.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.551+0000 7f258a18c700 1 --2- 172.21.15.71:0/3753946171 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f257006cce0 0x7f257006f1a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2580006010 tx=0x7f258001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:31.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.555+0000 7f257b7fe700 1 -- 172.21.15.71:0/3753946171 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f2574059ca0 con 0x7f2584101840 2024-04-23T02:17:31.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.746+0000 7f258c3f0700 1 -- 172.21.15.71:0/3753946171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f258404f2e0 con 0x7f2584101840 2024-04-23T02:17:31.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.750+0000 7f257b7fe700 1 -- 172.21.15.71:0/3753946171 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v45)=0 v45) v1 ==== 125+0+0 (secure 0 0 0) 0x7f25740107a0 con 0x7f2584101840 2024-04-23T02:17:31.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.752+0000 7f258c3f0700 1 -- 172.21.15.71:0/3753946171 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f257006cce0 msgr2=0x7f257006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:31.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.752+0000 7f258c3f0700 1 --2- 172.21.15.71:0/3753946171 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f257006cce0 0x7f257006f1a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2580006010 tx=0x7f258001a040 comp rx=0 tx=0).stop 2024-04-23T02:17:31.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.752+0000 7f258c3f0700 1 -- 172.21.15.71:0/3753946171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2584101840 msgr2=0x7f258419cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:31.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.752+0000 7f258c3f0700 1 --2- 172.21.15.71:0/3753946171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2584101840 0x7f258419cba0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f257400b480 tx=0x7f257400b840 comp rx=0 tx=0).stop 2024-04-23T02:17:31.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.752+0000 7f258c3f0700 1 -- 172.21.15.71:0/3753946171 shutdown_connections 2024-04-23T02:17:31.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.752+0000 7f258c3f0700 1 --2- 172.21.15.71:0/3753946171 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f257006cce0 0x7f257006f1a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:31.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.752+0000 7f258c3f0700 1 --2- 172.21.15.71:0/3753946171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2584100e90 0x7f258419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:31.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.752+0000 7f258c3f0700 1 --2- 172.21.15.71:0/3753946171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2584101840 0x7f258419cba0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:31.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.752+0000 7f258c3f0700 1 -- 172.21.15.71:0/3753946171 >> 172.21.15.71:0/3753946171 conn(0x7f25840fc740 msgr2=0x7f25840fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:31.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.752+0000 7f258c3f0700 1 -- 172.21.15.71:0/3753946171 shutdown_connections 2024-04-23T02:17:31.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:31.752+0000 7f258c3f0700 1 -- 172.21.15.71:0/3753946171 wait complete. 2024-04-23T02:17:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:31 smithi192 ceph-mon[49662]: pgmap v91: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:31 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/3753946171' entity='client.admin' 2024-04-23T02:17:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:31 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:17:32.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:31 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:17:32.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:31 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:17:32.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:31 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:17:32.162 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:31 smithi071 ceph-mon[39837]: pgmap v91: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:32.163 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:31 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3753946171' entity='client.admin' 2024-04-23T02:17:32.163 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:31 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:17:32.163 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:31 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:17:32.163 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:31 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:17:32.163 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:31 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:17:32.519 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-23T02:17:32.520 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi071.front.sepia.ceph.com 2024-04-23T02:17:32.520 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-23T02:17:32.751 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:17:34.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:33 smithi192 ceph-mon[49662]: pgmap v92: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:34.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:33 smithi071 ceph-mon[39837]: pgmap v92: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:34.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.686+0000 7fd0fdf68700 1 -- 172.21.15.71:0/1041399575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f80ffb40 msgr2=0x7fd0f80fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:34.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.686+0000 7fd0fdf68700 1 --2- 172.21.15.71:0/1041399575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f80ffb40 0x7fd0f80fffc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009910 tx=0x7fd0e8009c20 comp rx=0 tx=0).stop 2024-04-23T02:17:34.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.686+0000 7fd0fdf68700 1 -- 172.21.15.71:0/1041399575 shutdown_connections 2024-04-23T02:17:34.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.686+0000 7fd0fdf68700 1 --2- 172.21.15.71:0/1041399575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f80ffb40 0x7fd0f80fffc0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:34.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.686+0000 7fd0fdf68700 1 --2- 172.21.15.71:0/1041399575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f8105bf0 0x7fd0f8105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:34.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.686+0000 7fd0fdf68700 1 -- 172.21.15.71:0/1041399575 >> 172.21.15.71:0/1041399575 conn(0x7fd0f80fb610 msgr2=0x7fd0f80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:34.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.686+0000 7fd0fdf68700 1 -- 172.21.15.71:0/1041399575 shutdown_connections 2024-04-23T02:17:34.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.686+0000 7fd0fdf68700 1 -- 172.21.15.71:0/1041399575 wait complete. 2024-04-23T02:17:34.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.686+0000 7fd0fdf68700 1 Processor -- start 2024-04-23T02:17:34.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.687+0000 7fd0fdf68700 1 -- start start 2024-04-23T02:17:34.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.687+0000 7fd0fdf68700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f80ffb40 0x7fd0f819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:34.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.687+0000 7fd0fdf68700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f8105bf0 0x7fd0f819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:34.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.687+0000 7fd0fdf68700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f819d0d0 con 0x7fd0f80ffb40 2024-04-23T02:17:34.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.687+0000 7fd0fdf68700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f8196580 con 0x7fd0f8105bf0 2024-04-23T02:17:34.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.687+0000 7fd0f77fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f80ffb40 0x7fd0f819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:34.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.687+0000 7fd0f77fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f80ffb40 0x7fd0f819c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50394/0 (socket says 172.21.15.71:50394) 2024-04-23T02:17:34.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.687+0000 7fd0f77fe700 1 -- 172.21.15.71:0/2340729538 learned_addr learned my addr 172.21.15.71:0/2340729538 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:17:34.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.687+0000 7fd0f6ffd700 1 --2- 172.21.15.71:0/2340729538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f8105bf0 0x7fd0f819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:34.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.687+0000 7fd0f77fe700 1 -- 172.21.15.71:0/2340729538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f8105bf0 msgr2=0x7fd0f819ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:34.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.687+0000 7fd0f77fe700 1 --2- 172.21.15.71:0/2340729538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f8105bf0 0x7fd0f819ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:34.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.687+0000 7fd0f77fe700 1 -- 172.21.15.71:0/2340729538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e80095f0 con 0x7fd0f80ffb40 2024-04-23T02:17:34.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.687+0000 7fd0f77fe700 1 --2- 172.21.15.71:0/2340729538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f80ffb40 0x7fd0f819c500 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00d900 tx=0x7fd0ec00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:34.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.688+0000 7fd0f4ff9700 1 -- 172.21.15.71:0/2340729538 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0ec004a00 con 0x7fd0f80ffb40 2024-04-23T02:17:34.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.688+0000 7fd0fdf68700 1 -- 172.21.15.71:0/2340729538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f8196860 con 0x7fd0f80ffb40 2024-04-23T02:17:34.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.688+0000 7fd0f4ff9700 1 -- 172.21.15.71:0/2340729538 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd0ec005020 con 0x7fd0f80ffb40 2024-04-23T02:17:34.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.688+0000 7fd0f4ff9700 1 -- 172.21.15.71:0/2340729538 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0ec009b90 con 0x7fd0f80ffb40 2024-04-23T02:17:34.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.688+0000 7fd0fdf68700 1 -- 172.21.15.71:0/2340729538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f8196db0 con 0x7fd0f80ffb40 2024-04-23T02:17:34.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.689+0000 7fd0fdf68700 1 -- 172.21.15.71:0/2340729538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f804f2e0 con 0x7fd0f80ffb40 2024-04-23T02:17:34.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.689+0000 7fd0f4ff9700 1 -- 172.21.15.71:0/2340729538 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd0ec005190 con 0x7fd0f80ffb40 2024-04-23T02:17:34.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.690+0000 7fd0f4ff9700 1 --2- 172.21.15.71:0/2340729538 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd0e006cce0 0x7fd0e006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:34.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.690+0000 7fd0f4ff9700 1 -- 172.21.15.71:0/2340729538 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd0ec08c440 con 0x7fd0f80ffb40 2024-04-23T02:17:34.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.690+0000 7fd0f6ffd700 1 --2- 172.21.15.71:0/2340729538 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd0e006cce0 0x7fd0e006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:34.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.690+0000 7fd0f6ffd700 1 --2- 172.21.15.71:0/2340729538 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd0e006cce0 0x7fd0e006f1a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80febe0 tx=0x7fd0e801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:34.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.694+0000 7fd0f4ff9700 1 -- 172.21.15.71:0/2340729538 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fd0ec059ff0 con 0x7fd0f80ffb40 2024-04-23T02:17:34.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.892+0000 7fd0fdf68700 1 -- 172.21.15.71:0/2340729538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fd0f8197a90 con 0x7fd0f80ffb40 2024-04-23T02:17:34.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.892+0000 7fd0f4ff9700 1 -- 172.21.15.71:0/2340729538 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v45)=0 v45) v1 ==== 155+0+0 (secure 0 0 0) 0x7fd0ec005440 con 0x7fd0f80ffb40 2024-04-23T02:17:34.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.894+0000 7fd0fdf68700 1 -- 172.21.15.71:0/2340729538 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd0e006cce0 msgr2=0x7fd0e006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:34.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.894+0000 7fd0fdf68700 1 --2- 172.21.15.71:0/2340729538 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd0e006cce0 0x7fd0e006f1a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80febe0 tx=0x7fd0e801a040 comp rx=0 tx=0).stop 2024-04-23T02:17:34.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.894+0000 7fd0fdf68700 1 -- 172.21.15.71:0/2340729538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f80ffb40 msgr2=0x7fd0f819c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:34.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.894+0000 7fd0fdf68700 1 --2- 172.21.15.71:0/2340729538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f80ffb40 0x7fd0f819c500 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00d900 tx=0x7fd0ec00dc10 comp rx=0 tx=0).stop 2024-04-23T02:17:34.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.894+0000 7fd0fdf68700 1 -- 172.21.15.71:0/2340729538 shutdown_connections 2024-04-23T02:17:34.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.894+0000 7fd0fdf68700 1 --2- 172.21.15.71:0/2340729538 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd0e006cce0 0x7fd0e006f1a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:34.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.894+0000 7fd0fdf68700 1 --2- 172.21.15.71:0/2340729538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f80ffb40 0x7fd0f819c500 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:34.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.894+0000 7fd0fdf68700 1 --2- 172.21.15.71:0/2340729538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f8105bf0 0x7fd0f819ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:34.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.894+0000 7fd0fdf68700 1 -- 172.21.15.71:0/2340729538 >> 172.21.15.71:0/2340729538 conn(0x7fd0f80fb610 msgr2=0x7fd0f8068ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:34.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.895+0000 7fd0fdf68700 1 -- 172.21.15.71:0/2340729538 shutdown_connections 2024-04-23T02:17:34.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:34.895+0000 7fd0fdf68700 1 -- 172.21.15.71:0/2340729538 wait complete. 2024-04-23T02:17:35.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:34 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:17:35.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:34 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:17:35.625 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-23T02:17:35.857 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:17:35.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:35 smithi071 ceph-mon[39837]: pgmap v93: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:36.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:35 smithi192 ceph-mon[49662]: pgmap v93: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:37.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.650+0000 7fd43ae13700 1 -- 172.21.15.71:0/2655585301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd434105bd0 msgr2=0x7fd434105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:37.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.650+0000 7fd43ae13700 1 --2- 172.21.15.71:0/2655585301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd434105bd0 0x7fd434105fb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd428009910 tx=0x7fd428009c20 comp rx=0 tx=0).stop 2024-04-23T02:17:37.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.650+0000 7fd43ae13700 1 -- 172.21.15.71:0/2655585301 shutdown_connections 2024-04-23T02:17:37.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.650+0000 7fd43ae13700 1 --2- 172.21.15.71:0/2655585301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4340ffc20 0x7fd434100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:37.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.650+0000 7fd43ae13700 1 --2- 172.21.15.71:0/2655585301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd434105bd0 0x7fd434105fb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:37.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.650+0000 7fd43ae13700 1 -- 172.21.15.71:0/2655585301 >> 172.21.15.71:0/2655585301 conn(0x7fd434075670 msgr2=0x7fd434075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:37.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.650+0000 7fd43ae13700 1 -- 172.21.15.71:0/2655585301 shutdown_connections 2024-04-23T02:17:37.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.650+0000 7fd43ae13700 1 -- 172.21.15.71:0/2655585301 wait complete. 2024-04-23T02:17:37.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.651+0000 7fd43ae13700 1 Processor -- start 2024-04-23T02:17:37.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.651+0000 7fd43ae13700 1 -- start start 2024-04-23T02:17:37.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.651+0000 7fd43ae13700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4340ffc20 0x7fd43419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:37.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.651+0000 7fd43ae13700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd434105bd0 0x7fd43419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:37.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.651+0000 7fd43ae13700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43419d160 con 0x7fd4340ffc20 2024-04-23T02:17:37.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.651+0000 7fd43ae13700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd434196610 con 0x7fd434105bd0 2024-04-23T02:17:37.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.651+0000 7fd438baf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4340ffc20 0x7fd43419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:37.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.651+0000 7fd438baf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4340ffc20 0x7fd43419c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51580/0 (socket says 172.21.15.71:51580) 2024-04-23T02:17:37.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.651+0000 7fd438baf700 1 -- 172.21.15.71:0/1702707232 learned_addr learned my addr 172.21.15.71:0/1702707232 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:17:37.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.651+0000 7fd433fff700 1 --2- 172.21.15.71:0/1702707232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd434105bd0 0x7fd43419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:37.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.651+0000 7fd438baf700 1 -- 172.21.15.71:0/1702707232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd434105bd0 msgr2=0x7fd43419cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:37.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.651+0000 7fd438baf700 1 --2- 172.21.15.71:0/1702707232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd434105bd0 0x7fd43419cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:37.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.651+0000 7fd438baf700 1 -- 172.21.15.71:0/1702707232 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4280095f0 con 0x7fd4340ffc20 2024-04-23T02:17:37.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.651+0000 7fd433fff700 1 --2- 172.21.15.71:0/1702707232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd434105bd0 0x7fd43419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:17:37.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.652+0000 7fd438baf700 1 --2- 172.21.15.71:0/1702707232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4340ffc20 0x7fd43419c590 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd428006010 tx=0x7fd42800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:37.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.652+0000 7fd431ffb700 1 -- 172.21.15.71:0/1702707232 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd42801d070 con 0x7fd4340ffc20 2024-04-23T02:17:37.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.652+0000 7fd431ffb700 1 -- 172.21.15.71:0/1702707232 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd42800f460 con 0x7fd4340ffc20 2024-04-23T02:17:37.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.652+0000 7fd43ae13700 1 -- 172.21.15.71:0/1702707232 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd434196890 con 0x7fd4340ffc20 2024-04-23T02:17:37.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.652+0000 7fd431ffb700 1 -- 172.21.15.71:0/1702707232 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd428021620 con 0x7fd4340ffc20 2024-04-23T02:17:37.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.652+0000 7fd43ae13700 1 -- 172.21.15.71:0/1702707232 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd434196d80 con 0x7fd4340ffc20 2024-04-23T02:17:37.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.653+0000 7fd43ae13700 1 -- 172.21.15.71:0/1702707232 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43404f2e0 con 0x7fd4340ffc20 2024-04-23T02:17:37.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.653+0000 7fd431ffb700 1 -- 172.21.15.71:0/1702707232 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fd42800fa40 con 0x7fd4340ffc20 2024-04-23T02:17:37.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.654+0000 7fd431ffb700 1 --2- 172.21.15.71:0/1702707232 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd42406cce0 0x7fd42406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:37.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.654+0000 7fd431ffb700 1 -- 172.21.15.71:0/1702707232 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd42808d6d0 con 0x7fd4340ffc20 2024-04-23T02:17:37.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.654+0000 7fd433fff700 1 --2- 172.21.15.71:0/1702707232 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd42406cce0 0x7fd42406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:37.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.654+0000 7fd433fff700 1 --2- 172.21.15.71:0/1702707232 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd42406cce0 0x7fd42406f1a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd420005950 tx=0x7fd4200058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:37.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.658+0000 7fd431ffb700 1 -- 172.21.15.71:0/1702707232 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fd42805b280 con 0x7fd4340ffc20 2024-04-23T02:17:37.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.851+0000 7fd43ae13700 1 -- 172.21.15.71:0/1702707232 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fd434197b20 con 0x7fd4340ffc20 2024-04-23T02:17:37.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.852+0000 7fd431ffb700 1 -- 172.21.15.71:0/1702707232 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v45)=0 v45) v1 ==== 163+0+0 (secure 0 0 0) 0x7fd42805ae10 con 0x7fd4340ffc20 2024-04-23T02:17:37.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.854+0000 7fd43ae13700 1 -- 172.21.15.71:0/1702707232 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd42406cce0 msgr2=0x7fd42406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:37.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.854+0000 7fd43ae13700 1 --2- 172.21.15.71:0/1702707232 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd42406cce0 0x7fd42406f1a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd420005950 tx=0x7fd4200058e0 comp rx=0 tx=0).stop 2024-04-23T02:17:37.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.854+0000 7fd43ae13700 1 -- 172.21.15.71:0/1702707232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4340ffc20 msgr2=0x7fd43419c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:37.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.854+0000 7fd43ae13700 1 --2- 172.21.15.71:0/1702707232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4340ffc20 0x7fd43419c590 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd428006010 tx=0x7fd42800be20 comp rx=0 tx=0).stop 2024-04-23T02:17:37.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.854+0000 7fd43ae13700 1 -- 172.21.15.71:0/1702707232 shutdown_connections 2024-04-23T02:17:37.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.854+0000 7fd43ae13700 1 --2- 172.21.15.71:0/1702707232 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd42406cce0 0x7fd42406f1a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:37.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.854+0000 7fd43ae13700 1 --2- 172.21.15.71:0/1702707232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4340ffc20 0x7fd43419c590 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:37.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.854+0000 7fd43ae13700 1 --2- 172.21.15.71:0/1702707232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd434105bd0 0x7fd43419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:37.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.854+0000 7fd43ae13700 1 -- 172.21.15.71:0/1702707232 >> 172.21.15.71:0/1702707232 conn(0x7fd434075670 msgr2=0x7fd4340686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:37.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.854+0000 7fd43ae13700 1 -- 172.21.15.71:0/1702707232 shutdown_connections 2024-04-23T02:17:37.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:37.854+0000 7fd43ae13700 1 -- 172.21.15.71:0/1702707232 wait complete. 2024-04-23T02:17:38.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:37 smithi192 ceph-mon[49662]: pgmap v94: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:38.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:37 smithi071 ceph-mon[39837]: pgmap v94: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:38.527 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-23T02:17:38.755 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:17:40.102 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:39 smithi071 ceph-mon[39837]: pgmap v95: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:40.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:39 smithi192 ceph-mon[49662]: pgmap v95: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:40.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.643+0000 7fa7867cc700 1 -- 172.21.15.71:0/2604795703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa780068580 msgr2=0x7fa7800689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:40.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.644+0000 7fa7867cc700 1 --2- 172.21.15.71:0/2604795703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa780068580 0x7fa7800689e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa774009910 tx=0x7fa774009c20 comp rx=0 tx=0).stop 2024-04-23T02:17:40.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.644+0000 7fa7867cc700 1 -- 172.21.15.71:0/2604795703 shutdown_connections 2024-04-23T02:17:40.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.644+0000 7fa7867cc700 1 --2- 172.21.15.71:0/2604795703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa780068580 0x7fa7800689e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:40.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.644+0000 7fa7867cc700 1 --2- 172.21.15.71:0/2604795703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa780106cd0 0x7fa7801070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:40.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.644+0000 7fa7867cc700 1 -- 172.21.15.71:0/2604795703 >> 172.21.15.71:0/2604795703 conn(0x7fa780075670 msgr2=0x7fa780075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:40.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.644+0000 7fa7867cc700 1 -- 172.21.15.71:0/2604795703 shutdown_connections 2024-04-23T02:17:40.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.644+0000 7fa7867cc700 1 -- 172.21.15.71:0/2604795703 wait complete. 2024-04-23T02:17:40.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.644+0000 7fa7867cc700 1 Processor -- start 2024-04-23T02:17:40.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.644+0000 7fa7867cc700 1 -- start start 2024-04-23T02:17:40.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.644+0000 7fa7867cc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa780068580 0x7fa78019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:40.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.645+0000 7fa7867cc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa780106cd0 0x7fa78019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.645+0000 7fa7867cc700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78019d110 con 0x7fa780068580 2024-04-23T02:17:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.645+0000 7fa7867cc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7801965c0 con 0x7fa780106cd0 2024-04-23T02:17:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.645+0000 7fa77ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa780068580 0x7fa78019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.645+0000 7fa77ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa780068580 0x7fa78019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51604/0 (socket says 172.21.15.71:51604) 2024-04-23T02:17:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.645+0000 7fa77ffff700 1 -- 172.21.15.71:0/275556136 learned_addr learned my addr 172.21.15.71:0/275556136 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:17:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.645+0000 7fa77f7fe700 1 --2- 172.21.15.71:0/275556136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa780106cd0 0x7fa78019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.645+0000 7fa77ffff700 1 -- 172.21.15.71:0/275556136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa780106cd0 msgr2=0x7fa78019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.645+0000 7fa77ffff700 1 --2- 172.21.15.71:0/275556136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa780106cd0 0x7fa78019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.645+0000 7fa77ffff700 1 -- 172.21.15.71:0/275556136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7740095f0 con 0x7fa780068580 2024-04-23T02:17:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.645+0000 7fa77ffff700 1 --2- 172.21.15.71:0/275556136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa780068580 0x7fa78019c540 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa77000d8d0 tx=0x7fa77000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.645+0000 7fa77d7fa700 1 -- 172.21.15.71:0/275556136 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa770009940 con 0x7fa780068580 2024-04-23T02:17:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.645+0000 7fa77d7fa700 1 -- 172.21.15.71:0/275556136 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa770010460 con 0x7fa780068580 2024-04-23T02:17:40.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.645+0000 7fa7867cc700 1 -- 172.21.15.71:0/275556136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7801968a0 con 0x7fa780068580 2024-04-23T02:17:40.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.645+0000 7fa77d7fa700 1 -- 172.21.15.71:0/275556136 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa77000f5f0 con 0x7fa780068580 2024-04-23T02:17:40.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.645+0000 7fa7867cc700 1 -- 172.21.15.71:0/275556136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa780196df0 con 0x7fa780068580 2024-04-23T02:17:40.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.646+0000 7fa7867cc700 1 -- 172.21.15.71:0/275556136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7801091d0 con 0x7fa780068580 2024-04-23T02:17:40.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.647+0000 7fa77d7fa700 1 -- 172.21.15.71:0/275556136 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa7700105d0 con 0x7fa780068580 2024-04-23T02:17:40.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.647+0000 7fa77d7fa700 1 --2- 172.21.15.71:0/275556136 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa76806cce0 0x7fa76806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:40.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.647+0000 7fa77d7fa700 1 -- 172.21.15.71:0/275556136 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa77008c430 con 0x7fa780068580 2024-04-23T02:17:40.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.647+0000 7fa77f7fe700 1 --2- 172.21.15.71:0/275556136 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa76806cce0 0x7fa76806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:40.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.648+0000 7fa77f7fe700 1 --2- 172.21.15.71:0/275556136 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa76806cce0 0x7fa76806f1a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa7800fe260 tx=0x7fa77401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:40.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.652+0000 7fa77d7fa700 1 -- 172.21.15.71:0/275556136 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fa77005a060 con 0x7fa780068580 2024-04-23T02:17:40.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.846+0000 7fa7867cc700 1 -- 172.21.15.71:0/275556136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fa78004f2e0 con 0x7fa780068580 2024-04-23T02:17:40.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.848+0000 7fa77d7fa700 1 -- 172.21.15.71:0/275556136 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v45)=0 v45) v1 ==== 135+0+0 (secure 0 0 0) 0x7fa770010880 con 0x7fa780068580 2024-04-23T02:17:40.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.850+0000 7fa7867cc700 1 -- 172.21.15.71:0/275556136 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa76806cce0 msgr2=0x7fa76806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:40.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.850+0000 7fa7867cc700 1 --2- 172.21.15.71:0/275556136 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa76806cce0 0x7fa76806f1a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa7800fe260 tx=0x7fa77401a040 comp rx=0 tx=0).stop 2024-04-23T02:17:40.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.850+0000 7fa7867cc700 1 -- 172.21.15.71:0/275556136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa780068580 msgr2=0x7fa78019c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:40.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.850+0000 7fa7867cc700 1 --2- 172.21.15.71:0/275556136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa780068580 0x7fa78019c540 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa77000d8d0 tx=0x7fa77000dc90 comp rx=0 tx=0).stop 2024-04-23T02:17:40.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.850+0000 7fa7867cc700 1 -- 172.21.15.71:0/275556136 shutdown_connections 2024-04-23T02:17:40.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.850+0000 7fa7867cc700 1 --2- 172.21.15.71:0/275556136 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa76806cce0 0x7fa76806f1a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:40.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.850+0000 7fa7867cc700 1 --2- 172.21.15.71:0/275556136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa780068580 0x7fa78019c540 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:40.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.850+0000 7fa7867cc700 1 --2- 172.21.15.71:0/275556136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa780106cd0 0x7fa78019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:40.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.850+0000 7fa7867cc700 1 -- 172.21.15.71:0/275556136 >> 172.21.15.71:0/275556136 conn(0x7fa780075670 msgr2=0x7fa7800fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:40.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.850+0000 7fa7867cc700 1 -- 172.21.15.71:0/275556136 shutdown_connections 2024-04-23T02:17:40.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:40.850+0000 7fa7867cc700 1 -- 172.21.15.71:0/275556136 wait complete. 2024-04-23T02:17:41.545 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-04-23T02:17:41.771 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:17:42.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:41 smithi192 ceph-mon[49662]: pgmap v96: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:42.231 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:41 smithi071 ceph-mon[39837]: pgmap v96: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:43.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.615+0000 7f666ceeb700 1 -- 172.21.15.71:0/3599096126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66680ffb40 msgr2=0x7f66680fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:43.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.615+0000 7f666ceeb700 1 --2- 172.21.15.71:0/3599096126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66680ffb40 0x7f66680fffc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f665c009910 tx=0x7f665c009c20 comp rx=0 tx=0).stop 2024-04-23T02:17:43.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.615+0000 7f666ceeb700 1 -- 172.21.15.71:0/3599096126 shutdown_connections 2024-04-23T02:17:43.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.615+0000 7f666ceeb700 1 --2- 172.21.15.71:0/3599096126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66680ffb40 0x7f66680fffc0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:43.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.615+0000 7f666ceeb700 1 --2- 172.21.15.71:0/3599096126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6668105bf0 0x7f6668105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:43.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.615+0000 7f666ceeb700 1 -- 172.21.15.71:0/3599096126 >> 172.21.15.71:0/3599096126 conn(0x7f66680fb610 msgr2=0x7f66680fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:43.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.615+0000 7f666ceeb700 1 -- 172.21.15.71:0/3599096126 shutdown_connections 2024-04-23T02:17:43.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.615+0000 7f666ceeb700 1 -- 172.21.15.71:0/3599096126 wait complete. 2024-04-23T02:17:43.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.616+0000 7f666ceeb700 1 Processor -- start 2024-04-23T02:17:43.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.616+0000 7f666ceeb700 1 -- start start 2024-04-23T02:17:43.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.616+0000 7f666ceeb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66680ffb40 0x7f666819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:43.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.616+0000 7f666ceeb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6668105bf0 0x7f666819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:43.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.616+0000 7f666ceeb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666819d250 con 0x7f66680ffb40 2024-04-23T02:17:43.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.616+0000 7f666ceeb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6668196750 con 0x7f6668105bf0 2024-04-23T02:17:43.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.616+0000 7f666659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66680ffb40 0x7f666819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:43.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.616+0000 7f666659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66680ffb40 0x7f666819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51616/0 (socket says 172.21.15.71:51616) 2024-04-23T02:17:43.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.616+0000 7f666659c700 1 -- 172.21.15.71:0/4025897508 learned_addr learned my addr 172.21.15.71:0/4025897508 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:17:43.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.617+0000 7f666659c700 1 -- 172.21.15.71:0/4025897508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6668105bf0 msgr2=0x7f666819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:43.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.617+0000 7f666659c700 1 --2- 172.21.15.71:0/4025897508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6668105bf0 0x7f666819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:43.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.617+0000 7f666659c700 1 -- 172.21.15.71:0/4025897508 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c0095f0 con 0x7f66680ffb40 2024-04-23T02:17:43.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.617+0000 7f666659c700 1 --2- 172.21.15.71:0/4025897508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66680ffb40 0x7f666819c680 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f665800b4b0 tx=0x7f665800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:43.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.617+0000 7f66577fe700 1 -- 172.21.15.71:0/4025897508 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6658011800 con 0x7f66680ffb40 2024-04-23T02:17:43.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.617+0000 7f666ceeb700 1 -- 172.21.15.71:0/4025897508 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6668196a30 con 0x7f66680ffb40 2024-04-23T02:17:43.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.617+0000 7f66577fe700 1 -- 172.21.15.71:0/4025897508 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6658011e00 con 0x7f66680ffb40 2024-04-23T02:17:43.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.617+0000 7f66577fe700 1 -- 172.21.15.71:0/4025897508 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6658010e50 con 0x7f66680ffb40 2024-04-23T02:17:43.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.617+0000 7f666ceeb700 1 -- 172.21.15.71:0/4025897508 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6668196f80 con 0x7f66680ffb40 2024-04-23T02:17:43.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.618+0000 7f666ceeb700 1 -- 172.21.15.71:0/4025897508 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f666804f2e0 con 0x7f66680ffb40 2024-04-23T02:17:43.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.619+0000 7f66577fe700 1 -- 172.21.15.71:0/4025897508 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f6658010380 con 0x7f66680ffb40 2024-04-23T02:17:43.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.619+0000 7f66577fe700 1 --2- 172.21.15.71:0/4025897508 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f665006cce0 0x7f665006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:43.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.619+0000 7f6665d9b700 1 --2- 172.21.15.71:0/4025897508 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f665006cce0 0x7f665006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:43.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.619+0000 7f66577fe700 1 -- 172.21.15.71:0/4025897508 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f665808c550 con 0x7f66680ffb40 2024-04-23T02:17:43.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.620+0000 7f6665d9b700 1 --2- 172.21.15.71:0/4025897508 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f665006cce0 0x7f665006f1a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f665c00b5c0 tx=0x7f665c005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:43.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.623+0000 7f66577fe700 1 -- 172.21.15.71:0/4025897508 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f665805a120 con 0x7f66680ffb40 2024-04-23T02:17:43.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.835+0000 7f666ceeb700 1 -- 172.21.15.71:0/4025897508 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d", "target": ["mon-mgr", ""]}) v1 -- 0x7f6668197310 con 0x7f665006cce0 2024-04-23T02:17:43.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.842+0000 7f66577fe700 1 -- 172.21.15.71:0/4025897508 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f6668197310 con 0x7f665006cce0 2024-04-23T02:17:43.847 INFO:teuthology.orchestra.run.smithi071.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T02:17:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.844+0000 7f666ceeb700 1 -- 172.21.15.71:0/4025897508 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f665006cce0 msgr2=0x7f665006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.844+0000 7f666ceeb700 1 --2- 172.21.15.71:0/4025897508 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f665006cce0 0x7f665006f1a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f665c00b5c0 tx=0x7f665c005960 comp rx=0 tx=0).stop 2024-04-23T02:17:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.844+0000 7f666ceeb700 1 -- 172.21.15.71:0/4025897508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66680ffb40 msgr2=0x7f666819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.844+0000 7f666ceeb700 1 --2- 172.21.15.71:0/4025897508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66680ffb40 0x7f666819c680 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f665800b4b0 tx=0x7f665800b870 comp rx=0 tx=0).stop 2024-04-23T02:17:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.844+0000 7f666ceeb700 1 -- 172.21.15.71:0/4025897508 shutdown_connections 2024-04-23T02:17:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.844+0000 7f666ceeb700 1 --2- 172.21.15.71:0/4025897508 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f665006cce0 0x7f665006f1a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.844+0000 7f666ceeb700 1 --2- 172.21.15.71:0/4025897508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66680ffb40 0x7f666819c680 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.844+0000 7f666ceeb700 1 --2- 172.21.15.71:0/4025897508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6668105bf0 0x7f666819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.845+0000 7f666ceeb700 1 -- 172.21.15.71:0/4025897508 >> 172.21.15.71:0/4025897508 conn(0x7f66680fb610 msgr2=0x7f6668069090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.845+0000 7f666ceeb700 1 -- 172.21.15.71:0/4025897508 shutdown_connections 2024-04-23T02:17:43.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:43.845+0000 7f666ceeb700 1 -- 172.21.15.71:0/4025897508 wait complete. 2024-04-23T02:17:44.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:43 smithi192 ceph-mon[49662]: pgmap v97: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:44.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:43 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:17:44.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:43 smithi071 ceph-mon[39837]: pgmap v97: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:44.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:43 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:17:44.642 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-23T02:17:44.643 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi071.front.sepia.ceph.com 2024-04-23T02:17:44.643 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-04-23T02:17:45.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:44 smithi192 ceph-mon[49662]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:17:45.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:44 smithi192 ceph-mon[49662]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T02:17:45.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:44 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:17:45.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:44 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:17:45.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:44 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:17:45.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:44 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:17:45.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:44 smithi071 ceph-mon[39837]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:17:45.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:44 smithi071 ceph-mon[39837]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T02:17:45.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:44 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:17:45.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:44 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:17:45.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:44 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:17:45.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:44 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:17:45.392 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:17:46.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:45 smithi192 ceph-mon[49662]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T02:17:46.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:45 smithi192 ceph-mon[49662]: pgmap v98: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:46.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:45 smithi071 ceph-mon[39837]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:646b8c7d293d8ec31a40c69f7b3237b19c1be24d 2024-04-23T02:17:46.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:45 smithi071 ceph-mon[39837]: pgmap v98: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:47.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:47 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:17:47.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:47 smithi192 ceph-mon[49662]: Upgrade: Target is version 18.2.2-1237-g646b8c7d (reef) 2024-04-23T02:17:47.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:47 smithi192 ceph-mon[49662]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, digests ['quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210'] 2024-04-23T02:17:47.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:47 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:17:47.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:47 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:17:47.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:47 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:17:47.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:47 smithi192 ceph-mon[49662]: Upgrade: Setting container_image for all mgr 2024-04-23T02:17:47.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:47 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:17:47.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:47 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-23T02:17:47.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:47 smithi192 ceph-mon[49662]: pgmap v99: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:47.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:47 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:17:47.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:47 smithi071 ceph-mon[39837]: Upgrade: Target is version 18.2.2-1237-g646b8c7d (reef) 2024-04-23T02:17:47.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:47 smithi071 ceph-mon[39837]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210, digests ['quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210'] 2024-04-23T02:17:47.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:47 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:17:47.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:47 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:17:47.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:47 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:17:47.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:47 smithi071 ceph-mon[39837]: Upgrade: Setting container_image for all mgr 2024-04-23T02:17:47.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:47 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:17:47.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:47 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-23T02:17:47.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:47 smithi071 ceph-mon[39837]: pgmap v99: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:48.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.864+0000 7f2f0215c700 1 -- 172.21.15.71:0/861920802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2efc101840 msgr2=0x7f2efc105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:48.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.864+0000 7f2f0215c700 1 --2- 172.21.15.71:0/861920802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2efc101840 0x7f2efc105890 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009910 tx=0x7f2ef0009c20 comp rx=0 tx=0).stop 2024-04-23T02:17:48.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.865+0000 7f2f0215c700 1 -- 172.21.15.71:0/861920802 shutdown_connections 2024-04-23T02:17:48.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.865+0000 7f2f0215c700 1 --2- 172.21.15.71:0/861920802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2efc101840 0x7f2efc105890 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:48.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.865+0000 7f2f0215c700 1 --2- 172.21.15.71:0/861920802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2efc100e90 0x7f2efc101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:48.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.865+0000 7f2f0215c700 1 -- 172.21.15.71:0/861920802 >> 172.21.15.71:0/861920802 conn(0x7f2efc0fc740 msgr2=0x7f2efc0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:48.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.865+0000 7f2f0215c700 1 -- 172.21.15.71:0/861920802 shutdown_connections 2024-04-23T02:17:48.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.865+0000 7f2f0215c700 1 -- 172.21.15.71:0/861920802 wait complete. 2024-04-23T02:17:48.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.865+0000 7f2f0215c700 1 Processor -- start 2024-04-23T02:17:48.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.865+0000 7f2f0215c700 1 -- start start 2024-04-23T02:17:48.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.866+0000 7f2f0215c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2efc100e90 0x7f2efc19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:48.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.866+0000 7f2f0215c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2efc101840 0x7f2efc19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:48.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.866+0000 7f2f0215c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc19d170 con 0x7f2efc101840 2024-04-23T02:17:48.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.866+0000 7f2f0215c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2efc1966b0 con 0x7f2efc100e90 2024-04-23T02:17:48.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.866+0000 7f2efaffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2efc101840 0x7f2efc19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:48.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.866+0000 7f2efaffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2efc101840 0x7f2efc19cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33692/0 (socket says 172.21.15.71:33692) 2024-04-23T02:17:48.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.866+0000 7f2efaffd700 1 -- 172.21.15.71:0/3320452359 learned_addr learned my addr 172.21.15.71:0/3320452359 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:17:48.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.866+0000 7f2efb7fe700 1 --2- 172.21.15.71:0/3320452359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2efc100e90 0x7f2efc19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:48.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.866+0000 7f2efaffd700 1 -- 172.21.15.71:0/3320452359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2efc100e90 msgr2=0x7f2efc19c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:48.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.866+0000 7f2efaffd700 1 --2- 172.21.15.71:0/3320452359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2efc100e90 0x7f2efc19c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:48.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.866+0000 7f2efaffd700 1 -- 172.21.15.71:0/3320452359 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef00095f0 con 0x7f2efc101840 2024-04-23T02:17:48.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.866+0000 7f2efaffd700 1 --2- 172.21.15.71:0/3320452359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2efc101840 0x7f2efc19cb70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00092e0 tx=0x7f2ef000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:48.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.867+0000 7f2ef8ff9700 1 -- 172.21.15.71:0/3320452359 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ef001d070 con 0x7f2efc101840 2024-04-23T02:17:48.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.867+0000 7f2f0215c700 1 -- 172.21.15.71:0/3320452359 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2efc196930 con 0x7f2efc101840 2024-04-23T02:17:48.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.867+0000 7f2ef8ff9700 1 -- 172.21.15.71:0/3320452359 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2ef000f460 con 0x7f2efc101840 2024-04-23T02:17:48.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.867+0000 7f2f0215c700 1 -- 172.21.15.71:0/3320452359 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2efc196e20 con 0x7f2efc101840 2024-04-23T02:17:48.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.867+0000 7f2ef8ff9700 1 -- 172.21.15.71:0/3320452359 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ef0021620 con 0x7f2efc101840 2024-04-23T02:17:48.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.868+0000 7f2f0215c700 1 -- 172.21.15.71:0/3320452359 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee8005320 con 0x7f2efc101840 2024-04-23T02:17:48.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.869+0000 7f2ef8ff9700 1 -- 172.21.15.71:0/3320452359 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2ef002b440 con 0x7f2efc101840 2024-04-23T02:17:48.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.869+0000 7f2ef8ff9700 1 --2- 172.21.15.71:0/3320452359 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2ee406cce0 0x7f2ee406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:48.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.869+0000 7f2ef8ff9700 1 -- 172.21.15.71:0/3320452359 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2ef008d580 con 0x7f2efc101840 2024-04-23T02:17:48.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.869+0000 7f2efb7fe700 1 --2- 172.21.15.71:0/3320452359 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2ee406cce0 0x7f2ee406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:48.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.870+0000 7f2efb7fe700 1 --2- 172.21.15.71:0/3320452359 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2ee406cce0 0x7f2ee406f1a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00ac50 tx=0x7f2eec00a380 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:48.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:48.874+0000 7f2ef8ff9700 1 -- 172.21.15.71:0/3320452359 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f2ef005b220 con 0x7f2efc101840 2024-04-23T02:17:49.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.093+0000 7f2f0215c700 1 -- 172.21.15.71:0/3320452359 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ee8000bf0 con 0x7f2ee406cce0 2024-04-23T02:17:49.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.094+0000 7f2ef8ff9700 1 -- 172.21.15.71:0/3320452359 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f2ee8000bf0 con 0x7f2ee406cce0 2024-04-23T02:17:49.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.096+0000 7f2f0215c700 1 -- 172.21.15.71:0/3320452359 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2ee406cce0 msgr2=0x7f2ee406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:49.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.097+0000 7f2f0215c700 1 --2- 172.21.15.71:0/3320452359 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2ee406cce0 0x7f2ee406f1a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00ac50 tx=0x7f2eec00a380 comp rx=0 tx=0).stop 2024-04-23T02:17:49.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.097+0000 7f2f0215c700 1 -- 172.21.15.71:0/3320452359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2efc101840 msgr2=0x7f2efc19cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:49.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.097+0000 7f2f0215c700 1 --2- 172.21.15.71:0/3320452359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2efc101840 0x7f2efc19cb70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00092e0 tx=0x7f2ef000bd70 comp rx=0 tx=0).stop 2024-04-23T02:17:49.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.097+0000 7f2f0215c700 1 -- 172.21.15.71:0/3320452359 shutdown_connections 2024-04-23T02:17:49.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.097+0000 7f2f0215c700 1 --2- 172.21.15.71:0/3320452359 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2ee406cce0 0x7f2ee406f1a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.097+0000 7f2f0215c700 1 --2- 172.21.15.71:0/3320452359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2efc100e90 0x7f2efc19c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.097+0000 7f2f0215c700 1 --2- 172.21.15.71:0/3320452359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2efc101840 0x7f2efc19cb70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.097+0000 7f2f0215c700 1 -- 172.21.15.71:0/3320452359 >> 172.21.15.71:0/3320452359 conn(0x7f2efc0fc740 msgr2=0x7f2efc0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:49.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.097+0000 7f2f0215c700 1 -- 172.21.15.71:0/3320452359 shutdown_connections 2024-04-23T02:17:49.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.097+0000 7f2f0215c700 1 -- 172.21.15.71:0/3320452359 wait complete. 2024-04-23T02:17:49.112 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:17:49.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.209+0000 7f3781e67700 1 -- 172.21.15.71:0/2493705138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f377c1006b0 msgr2=0x7f377c100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.209+0000 7f3781e67700 1 --2- 172.21.15.71:0/2493705138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f377c1006b0 0x7f377c100a90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f376c009910 tx=0x7f376c009c20 comp rx=0 tx=0).stop 2024-04-23T02:17:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.209+0000 7f3781e67700 1 -- 172.21.15.71:0/2493705138 shutdown_connections 2024-04-23T02:17:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.209+0000 7f3781e67700 1 --2- 172.21.15.71:0/2493705138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f377c101060 0x7f377c1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.209+0000 7f3781e67700 1 --2- 172.21.15.71:0/2493705138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f377c1006b0 0x7f377c100a90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.209+0000 7f3781e67700 1 -- 172.21.15.71:0/2493705138 >> 172.21.15.71:0/2493705138 conn(0x7f377c074e30 msgr2=0x7f377c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.209+0000 7f3781e67700 1 -- 172.21.15.71:0/2493705138 shutdown_connections 2024-04-23T02:17:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.209+0000 7f3781e67700 1 -- 172.21.15.71:0/2493705138 wait complete. 2024-04-23T02:17:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.209+0000 7f3781e67700 1 Processor -- start 2024-04-23T02:17:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.210+0000 7f3781e67700 1 -- start start 2024-04-23T02:17:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.210+0000 7f3781e67700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f377c1006b0 0x7f377c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.210+0000 7f3781e67700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f377c101060 0x7f377c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.210+0000 7f3781e67700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c19d0a0 con 0x7f377c101060 2024-04-23T02:17:49.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.210+0000 7f3781e67700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c1965e0 con 0x7f377c1006b0 2024-04-23T02:17:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.210+0000 7f377affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f377c101060 0x7f377c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.210+0000 7f377affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f377c101060 0x7f377c19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33708/0 (socket says 172.21.15.71:33708) 2024-04-23T02:17:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.210+0000 7f377affd700 1 -- 172.21.15.71:0/1983888275 learned_addr learned my addr 172.21.15.71:0/1983888275 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:17:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.210+0000 7f377b7fe700 1 --2- 172.21.15.71:0/1983888275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f377c1006b0 0x7f377c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.210+0000 7f377affd700 1 -- 172.21.15.71:0/1983888275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f377c1006b0 msgr2=0x7f377c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.210+0000 7f377affd700 1 --2- 172.21.15.71:0/1983888275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f377c1006b0 0x7f377c19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.210+0000 7f377affd700 1 -- 172.21.15.71:0/1983888275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376c0095f0 con 0x7f377c101060 2024-04-23T02:17:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.210+0000 7f377b7fe700 1 --2- 172.21.15.71:0/1983888275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f377c1006b0 0x7f377c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:17:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.210+0000 7f377affd700 1 --2- 172.21.15.71:0/1983888275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f377c101060 0x7f377c19caa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f377000c960 tx=0x7f377000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.211+0000 7f3778ff9700 1 -- 172.21.15.71:0/1983888275 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3770007a70 con 0x7f377c101060 2024-04-23T02:17:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.211+0000 7f3778ff9700 1 -- 172.21.15.71:0/1983888275 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3770007bd0 con 0x7f377c101060 2024-04-23T02:17:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.211+0000 7f3781e67700 1 -- 172.21.15.71:0/1983888275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377c1968c0 con 0x7f377c101060 2024-04-23T02:17:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.211+0000 7f3778ff9700 1 -- 172.21.15.71:0/1983888275 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3770018630 con 0x7f377c101060 2024-04-23T02:17:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.211+0000 7f3781e67700 1 -- 172.21.15.71:0/1983888275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377c196e10 con 0x7f377c101060 2024-04-23T02:17:49.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.211+0000 7f3781e67700 1 -- 172.21.15.71:0/1983888275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f377c04f2e0 con 0x7f377c101060 2024-04-23T02:17:49.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.212+0000 7f3778ff9700 1 -- 172.21.15.71:0/1983888275 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f377000f9f0 con 0x7f377c101060 2024-04-23T02:17:49.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.213+0000 7f3778ff9700 1 --2- 172.21.15.71:0/1983888275 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f376406cce0 0x7f376406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:49.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.213+0000 7f3778ff9700 1 -- 172.21.15.71:0/1983888275 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f377008c630 con 0x7f377c101060 2024-04-23T02:17:49.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.213+0000 7f377b7fe700 1 --2- 172.21.15.71:0/1983888275 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f376406cce0 0x7f376406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:49.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.213+0000 7f377b7fe700 1 --2- 172.21.15.71:0/1983888275 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f376406cce0 0x7f376406f1a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f376c006010 tx=0x7f376c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:49.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.217+0000 7f3778ff9700 1 -- 172.21.15.71:0/1983888275 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f377005a070 con 0x7f377c101060 2024-04-23T02:17:49.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.436+0000 7f3781e67700 1 -- 172.21.15.71:0/1983888275 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f377c197190 con 0x7f376406cce0 2024-04-23T02:17:49.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.437+0000 7f3778ff9700 1 -- 172.21.15.71:0/1983888275 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f377c197190 con 0x7f376406cce0 2024-04-23T02:17:49.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.439+0000 7f3781e67700 1 -- 172.21.15.71:0/1983888275 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f376406cce0 msgr2=0x7f376406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:49.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.439+0000 7f3781e67700 1 --2- 172.21.15.71:0/1983888275 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f376406cce0 0x7f376406f1a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f376c006010 tx=0x7f376c01a040 comp rx=0 tx=0).stop 2024-04-23T02:17:49.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.439+0000 7f3781e67700 1 -- 172.21.15.71:0/1983888275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f377c101060 msgr2=0x7f377c19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:49.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.439+0000 7f3781e67700 1 --2- 172.21.15.71:0/1983888275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f377c101060 0x7f377c19caa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f377000c960 tx=0x7f377000cd20 comp rx=0 tx=0).stop 2024-04-23T02:17:49.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.439+0000 7f3781e67700 1 -- 172.21.15.71:0/1983888275 shutdown_connections 2024-04-23T02:17:49.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.439+0000 7f3781e67700 1 --2- 172.21.15.71:0/1983888275 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f376406cce0 0x7f376406f1a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.439+0000 7f3781e67700 1 --2- 172.21.15.71:0/1983888275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f377c1006b0 0x7f377c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.439+0000 7f3781e67700 1 --2- 172.21.15.71:0/1983888275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f377c101060 0x7f377c19caa0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.439+0000 7f3781e67700 1 -- 172.21.15.71:0/1983888275 >> 172.21.15.71:0/1983888275 conn(0x7f377c074e30 msgr2=0x7f377c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:49.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.439+0000 7f3781e67700 1 -- 172.21.15.71:0/1983888275 shutdown_connections 2024-04-23T02:17:49.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.439+0000 7f3781e67700 1 -- 172.21.15.71:0/1983888275 wait complete. 2024-04-23T02:17:49.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.552+0000 7f3e2bfe6700 1 -- 172.21.15.71:0/2023730357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e240ffc20 msgr2=0x7f3e24100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:49.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.552+0000 7f3e2bfe6700 1 --2- 172.21.15.71:0/2023730357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e240ffc20 0x7f3e24100080 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009910 tx=0x7f3e14009c20 comp rx=0 tx=0).stop 2024-04-23T02:17:49.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.552+0000 7f3e2bfe6700 1 -- 172.21.15.71:0/2023730357 shutdown_connections 2024-04-23T02:17:49.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.552+0000 7f3e2bfe6700 1 --2- 172.21.15.71:0/2023730357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e240ffc20 0x7f3e24100080 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.552+0000 7f3e2bfe6700 1 --2- 172.21.15.71:0/2023730357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e24105bd0 0x7f3e24105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.552+0000 7f3e2bfe6700 1 -- 172.21.15.71:0/2023730357 >> 172.21.15.71:0/2023730357 conn(0x7f3e24075670 msgr2=0x7f3e24075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:49.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.552+0000 7f3e2bfe6700 1 -- 172.21.15.71:0/2023730357 shutdown_connections 2024-04-23T02:17:49.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.552+0000 7f3e2bfe6700 1 -- 172.21.15.71:0/2023730357 wait complete. 2024-04-23T02:17:49.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.553+0000 7f3e2bfe6700 1 Processor -- start 2024-04-23T02:17:49.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.553+0000 7f3e2bfe6700 1 -- start start 2024-04-23T02:17:49.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.553+0000 7f3e2bfe6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e240ffc20 0x7f3e241976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:49.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.553+0000 7f3e2bfe6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e24105bd0 0x7f3e24197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:49.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.553+0000 7f3e2bfe6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e24198200 con 0x7f3e240ffc20 2024-04-23T02:17:49.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.553+0000 7f3e2bfe6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e24198370 con 0x7f3e24105bd0 2024-04-23T02:17:49.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.553+0000 7f3e29d82700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e240ffc20 0x7f3e241976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:49.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.553+0000 7f3e29d82700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e240ffc20 0x7f3e241976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33726/0 (socket says 172.21.15.71:33726) 2024-04-23T02:17:49.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.553+0000 7f3e29d82700 1 -- 172.21.15.71:0/4087139592 learned_addr learned my addr 172.21.15.71:0/4087139592 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:17:49.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.553+0000 7f3e29581700 1 --2- 172.21.15.71:0/4087139592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e24105bd0 0x7f3e24197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:49.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.553+0000 7f3e29d82700 1 -- 172.21.15.71:0/4087139592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e24105bd0 msgr2=0x7f3e24197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:49.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.553+0000 7f3e29d82700 1 --2- 172.21.15.71:0/4087139592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e24105bd0 0x7f3e24197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.553+0000 7f3e29d82700 1 -- 172.21.15.71:0/4087139592 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e140095f0 con 0x7f3e240ffc20 2024-04-23T02:17:49.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.554+0000 7f3e29d82700 1 --2- 172.21.15.71:0/4087139592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e240ffc20 0x7f3e241976a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3e2000d8d0 tx=0x7f3e2000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:49.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.554+0000 7f3e1affd700 1 -- 172.21.15.71:0/4087139592 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e200098a0 con 0x7f3e240ffc20 2024-04-23T02:17:49.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.554+0000 7f3e2bfe6700 1 -- 172.21.15.71:0/4087139592 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e241a0710 con 0x7f3e240ffc20 2024-04-23T02:17:49.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.554+0000 7f3e1affd700 1 -- 172.21.15.71:0/4087139592 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3e20010460 con 0x7f3e240ffc20 2024-04-23T02:17:49.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.554+0000 7f3e1affd700 1 -- 172.21.15.71:0/4087139592 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e20009ea0 con 0x7f3e240ffc20 2024-04-23T02:17:49.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.554+0000 7f3e2bfe6700 1 -- 172.21.15.71:0/4087139592 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e241a0c00 con 0x7f3e240ffc20 2024-04-23T02:17:49.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.555+0000 7f3e2bfe6700 1 -- 172.21.15.71:0/4087139592 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e241080d0 con 0x7f3e240ffc20 2024-04-23T02:17:49.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.556+0000 7f3e1affd700 1 -- 172.21.15.71:0/4087139592 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f3e200105d0 con 0x7f3e240ffc20 2024-04-23T02:17:49.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.556+0000 7f3e1affd700 1 --2- 172.21.15.71:0/4087139592 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3e1006cce0 0x7f3e1006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:49.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.556+0000 7f3e1affd700 1 -- 172.21.15.71:0/4087139592 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3e2008c3c0 con 0x7f3e240ffc20 2024-04-23T02:17:49.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.556+0000 7f3e29581700 1 --2- 172.21.15.71:0/4087139592 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3e1006cce0 0x7f3e1006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:49.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.556+0000 7f3e29581700 1 --2- 172.21.15.71:0/4087139592 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3e1006cce0 0x7f3e1006f1a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3e24068e50 tx=0x7f3e1401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:49.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.560+0000 7f3e1affd700 1 -- 172.21.15.71:0/4087139592 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f3e200564e0 con 0x7f3e240ffc20 2024-04-23T02:17:49.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:49 smithi071 ceph-mon[39837]: Upgrade: Updating mon.smithi071 2024-04-23T02:17:49.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:49 smithi071 ceph-mon[39837]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:17:49.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:49 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T02:17:49.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:49 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T02:17:49.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:49 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:17:49.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:49 smithi071 ceph-mon[39837]: Deploying daemon mon.smithi071 on smithi071 2024-04-23T02:17:49.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:49 smithi071 ceph-mon[39837]: pgmap v100: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:49.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:49 smithi071 ceph-mon[39837]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:17:49.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.776+0000 7f3e2bfe6700 1 -- 172.21.15.71:0/4087139592 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e240611d0 con 0x7f3e1006cce0 2024-04-23T02:17:49.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.787+0000 7f3e1affd700 1 -- 172.21.15.71:0/4087139592 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f3e240611d0 con 0x7f3e1006cce0 2024-04-23T02:17:49.791 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:17:49.791 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (91s) 68s ago 12m 16.9M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:17:49.791 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (13m) 68s ago 13m 7977k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:17:49.791 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (11m) 2m ago 11m 7796k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:17:49.791 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (13m) 68s ago 13m 6656k - 18.2.1 5be31c24972a 258512dcbc00 2024-04-23T02:17:49.792 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (11m) 2m ago 11m 6656k - 18.2.1 5be31c24972a feea2c9896a6 2024-04-23T02:17:49.792 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (74s) 68s ago 12m 39.0M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:17:49.792 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (6m) 68s ago 6m 64.4M - 18.2.1 5be31c24972a a515de2a841c 2024-04-23T02:17:49.792 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (6m) 68s ago 6m 133M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:17:49.792 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (6m) 2m ago 6m 68.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:17:49.792 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (6m) 2m ago 6m 125M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:17:49.792 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (3m) 68s ago 14m 498M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:17:49.792 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (2m) 2m ago 11m 449M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:17:49.792 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (14m) 68s ago 14m 63.4M 2048M 18.2.1 5be31c24972a ff40abe63fd9 2024-04-23T02:17:49.792 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (11m) 2m ago 11m 46.7M 2048M 18.2.1 5be31c24972a 9c03e90acc95 2024-04-23T02:17:49.792 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (2m) 68s ago 12m 9659k - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:17:49.793 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (2m) 2m ago 11m 3510k - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:17:49.793 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (10m) 68s ago 10m 196M 1227M 18.2.1 5be31c24972a d3d5f2fa7267 2024-04-23T02:17:49.793 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (10m) 68s ago 10m 208M 1227M 18.2.1 5be31c24972a 3992157ff0fd 2024-04-23T02:17:49.793 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (9m) 68s ago 9m 164M 1227M 18.2.1 5be31c24972a 12380b6a15a2 2024-04-23T02:17:49.793 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (9m) 2m ago 9m 240M 2251M 18.2.1 5be31c24972a d5f587ee3a22 2024-04-23T02:17:49.793 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (8m) 2m ago 8m 217M 2251M 18.2.1 5be31c24972a 958d046005f0 2024-04-23T02:17:49.793 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (8m) 2m ago 8m 195M 2251M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:17:49.793 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (109s) 68s ago 12m 46.6M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:17:49.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.789+0000 7f3e2bfe6700 1 -- 172.21.15.71:0/4087139592 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3e1006cce0 msgr2=0x7f3e1006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:49.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.789+0000 7f3e2bfe6700 1 --2- 172.21.15.71:0/4087139592 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3e1006cce0 0x7f3e1006f1a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3e24068e50 tx=0x7f3e1401a040 comp rx=0 tx=0).stop 2024-04-23T02:17:49.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.789+0000 7f3e2bfe6700 1 -- 172.21.15.71:0/4087139592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e240ffc20 msgr2=0x7f3e241976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:49.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.789+0000 7f3e2bfe6700 1 --2- 172.21.15.71:0/4087139592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e240ffc20 0x7f3e241976a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3e2000d8d0 tx=0x7f3e2000dbe0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.789+0000 7f3e2bfe6700 1 -- 172.21.15.71:0/4087139592 shutdown_connections 2024-04-23T02:17:49.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.789+0000 7f3e2bfe6700 1 --2- 172.21.15.71:0/4087139592 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3e1006cce0 0x7f3e1006f1a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.789+0000 7f3e2bfe6700 1 --2- 172.21.15.71:0/4087139592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e240ffc20 0x7f3e241976a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.789+0000 7f3e2bfe6700 1 --2- 172.21.15.71:0/4087139592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e24105bd0 0x7f3e24197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.789+0000 7f3e2bfe6700 1 -- 172.21.15.71:0/4087139592 >> 172.21.15.71:0/4087139592 conn(0x7f3e24075670 msgr2=0x7f3e240fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:49.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.789+0000 7f3e2bfe6700 1 -- 172.21.15.71:0/4087139592 shutdown_connections 2024-04-23T02:17:49.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.789+0000 7f3e2bfe6700 1 -- 172.21.15.71:0/4087139592 wait complete. 2024-04-23T02:17:49.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:49 smithi192 ceph-mon[49662]: Upgrade: Updating mon.smithi071 2024-04-23T02:17:49.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:49 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:17:49.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:49 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T02:17:49.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:49 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T02:17:49.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:49 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:17:49.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:49 smithi192 ceph-mon[49662]: Deploying daemon mon.smithi071 on smithi071 2024-04-23T02:17:49.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:49 smithi192 ceph-mon[49662]: pgmap v100: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:49.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:49 smithi192 ceph-mon[49662]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:17:49.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.894+0000 7f2dd105b700 1 -- 172.21.15.71:0/3498298236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dcc0fff50 msgr2=0x7f2dcc105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:49.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.894+0000 7f2dd105b700 1 --2- 172.21.15.71:0/3498298236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dcc0fff50 0x7f2dcc105840 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009910 tx=0x7f2dc0009c20 comp rx=0 tx=0).stop 2024-04-23T02:17:49.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.894+0000 7f2dd105b700 1 -- 172.21.15.71:0/3498298236 shutdown_connections 2024-04-23T02:17:49.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.894+0000 7f2dd105b700 1 --2- 172.21.15.71:0/3498298236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dcc0fff50 0x7f2dcc105840 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.894+0000 7f2dd105b700 1 --2- 172.21.15.71:0/3498298236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2dcc0ff5a0 0x7f2dcc0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.894+0000 7f2dd105b700 1 -- 172.21.15.71:0/3498298236 >> 172.21.15.71:0/3498298236 conn(0x7f2dcc074e30 msgr2=0x7f2dcc075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:49.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.894+0000 7f2dd105b700 1 -- 172.21.15.71:0/3498298236 shutdown_connections 2024-04-23T02:17:49.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.894+0000 7f2dd105b700 1 -- 172.21.15.71:0/3498298236 wait complete. 2024-04-23T02:17:49.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.895+0000 7f2dd105b700 1 Processor -- start 2024-04-23T02:17:49.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.895+0000 7f2dd105b700 1 -- start start 2024-04-23T02:17:49.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.895+0000 7f2dd105b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2dcc0ff5a0 0x7f2dcc19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:49.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.895+0000 7f2dd105b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dcc0fff50 0x7f2dcc19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:49.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.895+0000 7f2dd105b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc19d230 con 0x7f2dcc0fff50 2024-04-23T02:17:49.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.895+0000 7f2dd105b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc1966e0 con 0x7f2dcc0ff5a0 2024-04-23T02:17:49.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.896+0000 7f2dca59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dcc0fff50 0x7f2dcc19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:49.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.896+0000 7f2dca59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dcc0fff50 0x7f2dcc19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33746/0 (socket says 172.21.15.71:33746) 2024-04-23T02:17:49.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.896+0000 7f2dca59c700 1 -- 172.21.15.71:0/3082579305 learned_addr learned my addr 172.21.15.71:0/3082579305 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:17:49.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.896+0000 7f2dcad9d700 1 --2- 172.21.15.71:0/3082579305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2dcc0ff5a0 0x7f2dcc19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:49.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.896+0000 7f2dca59c700 1 -- 172.21.15.71:0/3082579305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2dcc0ff5a0 msgr2=0x7f2dcc19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:49.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.896+0000 7f2dca59c700 1 --2- 172.21.15.71:0/3082579305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2dcc0ff5a0 0x7f2dcc19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:49.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.896+0000 7f2dca59c700 1 -- 172.21.15.71:0/3082579305 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc00095f0 con 0x7f2dcc0fff50 2024-04-23T02:17:49.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.896+0000 7f2dca59c700 1 --2- 172.21.15.71:0/3082579305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dcc0fff50 0x7f2dcc19cba0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0005f50 tx=0x7f2dc000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:49.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.896+0000 7f2db3fff700 1 -- 172.21.15.71:0/3082579305 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2dc001d070 con 0x7f2dcc0fff50 2024-04-23T02:17:49.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.896+0000 7f2db3fff700 1 -- 172.21.15.71:0/3082579305 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2dc000f460 con 0x7f2dcc0fff50 2024-04-23T02:17:49.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.896+0000 7f2dd105b700 1 -- 172.21.15.71:0/3082579305 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dcc196960 con 0x7f2dcc0fff50 2024-04-23T02:17:49.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.896+0000 7f2db3fff700 1 -- 172.21.15.71:0/3082579305 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2dc0021640 con 0x7f2dcc0fff50 2024-04-23T02:17:49.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.896+0000 7f2dd105b700 1 -- 172.21.15.71:0/3082579305 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dcc196e50 con 0x7f2dcc0fff50 2024-04-23T02:17:49.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.897+0000 7f2dd105b700 1 -- 172.21.15.71:0/3082579305 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dcc109220 con 0x7f2dcc0fff50 2024-04-23T02:17:49.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.898+0000 7f2db3fff700 1 -- 172.21.15.71:0/3082579305 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f2dc000f9e0 con 0x7f2dcc0fff50 2024-04-23T02:17:49.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.898+0000 7f2db3fff700 1 --2- 172.21.15.71:0/3082579305 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2db406cce0 0x7f2db406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:49.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.898+0000 7f2db3fff700 1 -- 172.21.15.71:0/3082579305 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2dc008d780 con 0x7f2dcc0fff50 2024-04-23T02:17:49.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.898+0000 7f2dcad9d700 1 --2- 172.21.15.71:0/3082579305 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2db406cce0 0x7f2db406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:49.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.899+0000 7f2dcad9d700 1 --2- 172.21.15.71:0/3082579305 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2db406cce0 0x7f2db406f1a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc009a10 tx=0x7f2dbc009190 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:49.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:49.903+0000 7f2db3fff700 1 -- 172.21.15.71:0/3082579305 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f2dc005b2d0 con 0x7f2dcc0fff50 2024-04-23T02:17:50.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.180+0000 7f2dd105b700 1 -- 172.21.15.71:0/3082579305 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2dcc108c00 con 0x7f2dcc0fff50 2024-04-23T02:17:50.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.180+0000 7f2db3fff700 1 -- 172.21.15.71:0/3082579305 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f2dc005ae60 con 0x7f2dcc0fff50 2024-04-23T02:17:50.184 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:17:50.185 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:17:50.185 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-23T02:17:50.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:17:50.185 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:17:50.185 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:17:50.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:17:50.185 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:17:50.185 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-23T02:17:50.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:17:50.185 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:17:50.185 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-23T02:17:50.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:17:50.185 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:17:50.185 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-23T02:17:50.186 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:17:50.186 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:17:50.186 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:17:50.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.182+0000 7f2dd105b700 1 -- 172.21.15.71:0/3082579305 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2db406cce0 msgr2=0x7f2db406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:50.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.182+0000 7f2dd105b700 1 --2- 172.21.15.71:0/3082579305 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2db406cce0 0x7f2db406f1a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc009a10 tx=0x7f2dbc009190 comp rx=0 tx=0).stop 2024-04-23T02:17:50.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.182+0000 7f2dd105b700 1 -- 172.21.15.71:0/3082579305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dcc0fff50 msgr2=0x7f2dcc19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:50.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.182+0000 7f2dd105b700 1 --2- 172.21.15.71:0/3082579305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dcc0fff50 0x7f2dcc19cba0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0005f50 tx=0x7f2dc000bd70 comp rx=0 tx=0).stop 2024-04-23T02:17:50.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.182+0000 7f2dd105b700 1 -- 172.21.15.71:0/3082579305 shutdown_connections 2024-04-23T02:17:50.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.182+0000 7f2dd105b700 1 --2- 172.21.15.71:0/3082579305 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2db406cce0 0x7f2db406f1a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:50.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.182+0000 7f2dd105b700 1 --2- 172.21.15.71:0/3082579305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2dcc0ff5a0 0x7f2dcc19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:50.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.182+0000 7f2dd105b700 1 --2- 172.21.15.71:0/3082579305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dcc0fff50 0x7f2dcc19cba0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:50.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.182+0000 7f2dd105b700 1 -- 172.21.15.71:0/3082579305 >> 172.21.15.71:0/3082579305 conn(0x7f2dcc074e30 msgr2=0x7f2dcc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:50.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.182+0000 7f2dd105b700 1 -- 172.21.15.71:0/3082579305 shutdown_connections 2024-04-23T02:17:50.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.183+0000 7f2dd105b700 1 -- 172.21.15.71:0/3082579305 wait complete. 2024-04-23T02:17:50.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.289+0000 7f479dc2d700 1 -- 172.21.15.71:0/2946561928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4798068580 msgr2=0x7f47980689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:50.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.289+0000 7f479dc2d700 1 --2- 172.21.15.71:0/2946561928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4798068580 0x7f47980689e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4780009910 tx=0x7f4780009c20 comp rx=0 tx=0).stop 2024-04-23T02:17:50.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.289+0000 7f479dc2d700 1 -- 172.21.15.71:0/2946561928 shutdown_connections 2024-04-23T02:17:50.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.289+0000 7f479dc2d700 1 --2- 172.21.15.71:0/2946561928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4798068580 0x7f47980689e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:50.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.289+0000 7f479dc2d700 1 --2- 172.21.15.71:0/2946561928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4798106cd0 0x7f47981070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:50.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.289+0000 7f479dc2d700 1 -- 172.21.15.71:0/2946561928 >> 172.21.15.71:0/2946561928 conn(0x7f4798075670 msgr2=0x7f4798075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:50.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.289+0000 7f479dc2d700 1 -- 172.21.15.71:0/2946561928 shutdown_connections 2024-04-23T02:17:50.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.289+0000 7f479dc2d700 1 -- 172.21.15.71:0/2946561928 wait complete. 2024-04-23T02:17:50.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.289+0000 7f479dc2d700 1 Processor -- start 2024-04-23T02:17:50.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.289+0000 7f479dc2d700 1 -- start start 2024-04-23T02:17:50.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.290+0000 7f479dc2d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4798068580 0x7f47980ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:50.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.290+0000 7f479dc2d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4798106cd0 0x7f4798100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:50.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.290+0000 7f479dc2d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4798100770 con 0x7f4798068580 2024-04-23T02:17:50.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.290+0000 7f479dc2d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47981008b0 con 0x7f4798106cd0 2024-04-23T02:17:50.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.290+0000 7f47977fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4798068580 0x7f47980ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:50.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.290+0000 7f47977fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4798068580 0x7f47980ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33756/0 (socket says 172.21.15.71:33756) 2024-04-23T02:17:50.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.290+0000 7f47977fe700 1 -- 172.21.15.71:0/4046660517 learned_addr learned my addr 172.21.15.71:0/4046660517 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:17:50.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.290+0000 7f4796ffd700 1 --2- 172.21.15.71:0/4046660517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4798106cd0 0x7f4798100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:50.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.290+0000 7f47977fe700 1 -- 172.21.15.71:0/4046660517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4798106cd0 msgr2=0x7f4798100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:50.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.290+0000 7f47977fe700 1 --2- 172.21.15.71:0/4046660517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4798106cd0 0x7f4798100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:50.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.290+0000 7f47977fe700 1 -- 172.21.15.71:0/4046660517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47800095f0 con 0x7f4798068580 2024-04-23T02:17:50.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.290+0000 7f47977fe700 1 --2- 172.21.15.71:0/4046660517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4798068580 0x7f47980ffcf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f478800d8d0 tx=0x7f478800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:50.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.290+0000 7f4794ff9700 1 -- 172.21.15.71:0/4046660517 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47880098a0 con 0x7f4798068580 2024-04-23T02:17:50.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.290+0000 7f479dc2d700 1 -- 172.21.15.71:0/4046660517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47981a2940 con 0x7f4798068580 2024-04-23T02:17:50.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.290+0000 7f4794ff9700 1 -- 172.21.15.71:0/4046660517 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4788010460 con 0x7f4798068580 2024-04-23T02:17:50.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.290+0000 7f4794ff9700 1 -- 172.21.15.71:0/4046660517 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4788009ea0 con 0x7f4798068580 2024-04-23T02:17:50.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.290+0000 7f479dc2d700 1 -- 172.21.15.71:0/4046660517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47981a2d50 con 0x7f4798068580 2024-04-23T02:17:50.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.292+0000 7f4794ff9700 1 -- 172.21.15.71:0/4046660517 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f47880105d0 con 0x7f4798068580 2024-04-23T02:17:50.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.292+0000 7f479dc2d700 1 -- 172.21.15.71:0/4046660517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47981091d0 con 0x7f4798068580 2024-04-23T02:17:50.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.292+0000 7f4794ff9700 1 --2- 172.21.15.71:0/4046660517 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f478406cce0 0x7f478406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:50.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.292+0000 7f4794ff9700 1 -- 172.21.15.71:0/4046660517 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f478808c3c0 con 0x7f4798068580 2024-04-23T02:17:50.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.292+0000 7f4796ffd700 1 --2- 172.21.15.71:0/4046660517 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f478406cce0 0x7f478406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:50.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.293+0000 7f4796ffd700 1 --2- 172.21.15.71:0/4046660517 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f478406cce0 0x7f478406f1a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47981014e0 tx=0x7f478001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:50.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.297+0000 7f4794ff9700 1 -- 172.21.15.71:0/4046660517 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f47880564e0 con 0x7f4798068580 2024-04-23T02:17:50.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:50 smithi071 systemd[1]: Stopping Ceph mon.smithi071 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T02:17:50.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.559+0000 7f479dc2d700 1 -- 172.21.15.71:0/4046660517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4798108bb0 con 0x7f4798068580 2024-04-23T02:17:50.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.559+0000 7f4794ff9700 1 -- 172.21.15.71:0/4046660517 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1848 (secure 0 0 0) 0x7f4788010880 con 0x7f4798068580 2024-04-23T02:17:50.565 INFO:teuthology.orchestra.run.smithi071.stdout:e25 2024-04-23T02:17:50.565 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:17:50.565 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:17:50.565 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:17:50.565 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:17:50.565 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:17:50.565 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:17:50.565 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 25 2024-04-23T02:17:50.565 INFO:teuthology.orchestra.run.smithi071.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T02:17:50.566 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:17:50.566 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:17:27.814232+0000 2024-04-23T02:17:50.566 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:17:50.566 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:17:50.566 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:17:50.566 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:17:50.566 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:17:50.566 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:17:50.566 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:17:50.566 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 46 2024-04-23T02:17:50.567 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:17:50.567 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:17:50.567 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:17:50.567 INFO:teuthology.orchestra.run.smithi071.stdout:up {0=14478,1=24243} 2024-04-23T02:17:50.567 INFO:teuthology.orchestra.run.smithi071.stdout:failed 2024-04-23T02:17:50.567 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:17:50.567 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:17:50.567 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:17:50.567 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:17:50.567 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:17:50.567 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:17:50.567 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:17:50.567 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:17:50.568 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:17:50.568 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/3442634960,v1:172.21.15.71:6827/3442634960] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:17:50.568 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{1:24243} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.192:6824/3780549896,v1:172.21.15.192:6825/3780549896] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:17:50.568 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{1:24255} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2100078284,v1:172.21.15.192:6827/2100078284] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:17:50.568 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:17:50.568 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:17:50.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.562+0000 7f479dc2d700 1 -- 172.21.15.71:0/4046660517 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f478406cce0 msgr2=0x7f478406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:50.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.562+0000 7f479dc2d700 1 --2- 172.21.15.71:0/4046660517 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f478406cce0 0x7f478406f1a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47981014e0 tx=0x7f478001a040 comp rx=0 tx=0).stop 2024-04-23T02:17:50.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.562+0000 7f479dc2d700 1 -- 172.21.15.71:0/4046660517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4798068580 msgr2=0x7f47980ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:50.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.562+0000 7f479dc2d700 1 --2- 172.21.15.71:0/4046660517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4798068580 0x7f47980ffcf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f478800d8d0 tx=0x7f478800dbe0 comp rx=0 tx=0).stop 2024-04-23T02:17:50.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.562+0000 7f479dc2d700 1 -- 172.21.15.71:0/4046660517 shutdown_connections 2024-04-23T02:17:50.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.562+0000 7f479dc2d700 1 --2- 172.21.15.71:0/4046660517 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f478406cce0 0x7f478406f1a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:50.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.562+0000 7f479dc2d700 1 --2- 172.21.15.71:0/4046660517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4798068580 0x7f47980ffcf0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:50.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.562+0000 7f479dc2d700 1 --2- 172.21.15.71:0/4046660517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4798106cd0 0x7f4798100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:50.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.562+0000 7f479dc2d700 1 -- 172.21.15.71:0/4046660517 >> 172.21.15.71:0/4046660517 conn(0x7f4798075670 msgr2=0x7f47980fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:50.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.562+0000 7f479dc2d700 1 -- 172.21.15.71:0/4046660517 shutdown_connections 2024-04-23T02:17:50.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.562+0000 7f479dc2d700 1 -- 172.21.15.71:0/4046660517 wait complete. 2024-04-23T02:17:50.570 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 25 2024-04-23T02:17:50.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.672+0000 7fa396cb2700 1 -- 172.21.15.71:0/205024323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa390068580 msgr2=0x7fa3900689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:50.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.672+0000 7fa396cb2700 1 --2- 172.21.15.71:0/205024323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa390068580 0x7fa3900689e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa378009910 tx=0x7fa378009c20 comp rx=0 tx=0).stop 2024-04-23T02:17:50.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.672+0000 7fa396cb2700 1 -- 172.21.15.71:0/205024323 shutdown_connections 2024-04-23T02:17:50.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.672+0000 7fa396cb2700 1 --2- 172.21.15.71:0/205024323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa390068580 0x7fa3900689e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:50.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.672+0000 7fa396cb2700 1 --2- 172.21.15.71:0/205024323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa390106cd0 0x7fa3901070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:50.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.672+0000 7fa396cb2700 1 -- 172.21.15.71:0/205024323 >> 172.21.15.71:0/205024323 conn(0x7fa390075670 msgr2=0x7fa390075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:50.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.672+0000 7fa396cb2700 1 -- 172.21.15.71:0/205024323 shutdown_connections 2024-04-23T02:17:50.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.672+0000 7fa396cb2700 1 -- 172.21.15.71:0/205024323 wait complete. 2024-04-23T02:17:50.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.673+0000 7fa396cb2700 1 Processor -- start 2024-04-23T02:17:50.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.673+0000 7fa396cb2700 1 -- start start 2024-04-23T02:17:50.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.673+0000 7fa396cb2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa390068580 0x7fa39019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:50.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.673+0000 7fa396cb2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa390106cd0 0x7fa39019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:50.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.673+0000 7fa396cb2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39019d110 con 0x7fa390068580 2024-04-23T02:17:50.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.673+0000 7fa396cb2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3901965c0 con 0x7fa390106cd0 2024-04-23T02:17:50.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.673+0000 7fa394a4e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa390068580 0x7fa39019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:50.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.673+0000 7fa394a4e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa390068580 0x7fa39019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33766/0 (socket says 172.21.15.71:33766) 2024-04-23T02:17:50.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.673+0000 7fa394a4e700 1 -- 172.21.15.71:0/2946573332 learned_addr learned my addr 172.21.15.71:0/2946573332 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:17:50.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.673+0000 7fa394a4e700 1 -- 172.21.15.71:0/2946573332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa390106cd0 msgr2=0x7fa39019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:50.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.673+0000 7fa394a4e700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa390106cd0 0x7fa39019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:50.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.673+0000 7fa394a4e700 1 -- 172.21.15.71:0/2946573332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3780095f0 con 0x7fa390068580 2024-04-23T02:17:50.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.673+0000 7fa394a4e700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa390068580 0x7fa39019c540 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa38000c8a0 tx=0x7fa38000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:50.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.673+0000 7fa38dffb700 1 -- 172.21.15.71:0/2946573332 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa38000cea0 con 0x7fa390068580 2024-04-23T02:17:50.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.674+0000 7fa396cb2700 1 -- 172.21.15.71:0/2946573332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3901968a0 con 0x7fa390068580 2024-04-23T02:17:50.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.674+0000 7fa38dffb700 1 -- 172.21.15.71:0/2946573332 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa380004830 con 0x7fa390068580 2024-04-23T02:17:50.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.674+0000 7fa38dffb700 1 -- 172.21.15.71:0/2946573332 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa380005500 con 0x7fa390068580 2024-04-23T02:17:50.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.674+0000 7fa396cb2700 1 -- 172.21.15.71:0/2946573332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa390196df0 con 0x7fa390068580 2024-04-23T02:17:50.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.675+0000 7fa38dffb700 1 -- 172.21.15.71:0/2946573332 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa3800049a0 con 0x7fa390068580 2024-04-23T02:17:50.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.675+0000 7fa396cb2700 1 -- 172.21.15.71:0/2946573332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3901091d0 con 0x7fa390068580 2024-04-23T02:17:50.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.675+0000 7fa38dffb700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa37c06cce0 0x7fa37c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:50.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.676+0000 7fa38dffb700 1 -- 172.21.15.71:0/2946573332 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa38008c1e0 con 0x7fa390068580 2024-04-23T02:17:50.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.676+0000 7fa38ffff700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa37c06cce0 0x7fa37c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:50.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.676+0000 7fa38ffff700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa37c06cce0 0x7fa37c06f1a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa378006010 tx=0x7fa37800b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:50.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.680+0000 7fa38dffb700 1 -- 172.21.15.71:0/2946573332 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fa380059d90 con 0x7fa390068580 2024-04-23T02:17:50.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.681+0000 7fa394a4e700 1 -- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa390068580 msgr2=0x7fa39019c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-23T02:17:50.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.681+0000 7fa394a4e700 1 -- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa390068580 msgr2=0x7fa39019c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-23T02:17:50.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.681+0000 7fa394a4e700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa390068580 0x7fa39019c540 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa38000c8a0 tx=0x7fa38000cc60 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T02:17:50.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.681+0000 7fa394a4e700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa390068580 0x7fa39019c540 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa38000c8a0 tx=0x7fa38000cc60 comp rx=0 tx=0).stop 2024-04-23T02:17:50.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.681+0000 7fa38dffb700 1 -- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa390068580 msgr2=0x7fa39019c540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-23T02:17:50.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.681+0000 7fa38dffb700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa390068580 0x7fa39019c540 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:50.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.681+0000 7fa38dffb700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa390106cd0 0x7fa37c07ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:50.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.681+0000 7fa38dffb700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c07b460 0x7fa37c07d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:50.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.681+0000 7fa38dffb700 1 -- 172.21.15.71:0/2946573332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c07e310 con 0x7fa37c07b460 2024-04-23T02:17:50.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.681+0000 7fa38dffb700 1 -- 172.21.15.71:0/2946573332 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c07e450 con 0x7fa390106cd0 2024-04-23T02:17:50.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.681+0000 7fa39524f700 1 -- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c07b460 msgr2=0x7fa37c07d8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:3300/0 2024-04-23T02:17:50.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.681+0000 7fa394a4e700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa390106cd0 0x7fa37c07ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:50.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.681+0000 7fa39524f700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c07b460 0x7fa37c07d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T02:17:50.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.682+0000 7fa394a4e700 1 -- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c07b460 msgr2=0x7fa37c07d8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T02:17:50.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.682+0000 7fa394a4e700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c07b460 0x7fa37c07d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:50.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.682+0000 7fa394a4e700 1 -- 172.21.15.71:0/2946573332 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=3+}) v3 -- 0x7fa37c0749a0 con 0x7fa390106cd0 2024-04-23T02:17:50.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.683+0000 7fa394a4e700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa390106cd0 0x7fa37c07ad60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa390197c60 tx=0x7fa38000a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:50.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.683+0000 7fa38dffb700 1 -- 172.21.15.71:0/2946573332 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa38005ddf0 con 0x7fa390106cd0 2024-04-23T02:17:50.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.684+0000 7fa38dffb700 1 -- 172.21.15.71:0/2946573332 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa3800623b0 con 0x7fa390106cd0 2024-04-23T02:17:50.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.685+0000 7fa38dffb700 1 -- 172.21.15.71:0/2946573332 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7fa3800566d0 con 0x7fa390106cd0 2024-04-23T02:17:50.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.900+0000 7fa396cb2700 1 -- 172.21.15.71:0/2946573332 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa390197ad0 con 0x7fa37c06cce0 2024-04-23T02:17:50.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.901+0000 7fa38dffb700 1 -- 172.21.15.71:0/2946573332 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fa390197ad0 con 0x7fa37c06cce0 2024-04-23T02:17:50.905 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:17:50.905 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:17:50.906 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:17:50.906 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:17:50.906 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:17:50.906 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr" 2024-04-23T02:17:50.906 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:17:50.906 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "2/23 daemons upgraded", 2024-04-23T02:17:50.906 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mon daemons", 2024-04-23T02:17:50.906 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:17:50.906 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:17:50.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.903+0000 7fa396cb2700 1 -- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa37c06cce0 msgr2=0x7fa37c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:50.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.903+0000 7fa396cb2700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa37c06cce0 0x7fa37c06f1a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa378006010 tx=0x7fa37800b540 comp rx=0 tx=0).stop 2024-04-23T02:17:50.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.903+0000 7fa396cb2700 1 -- 172.21.15.71:0/2946573332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa390106cd0 msgr2=0x7fa37c07ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:50.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.903+0000 7fa396cb2700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa390106cd0 0x7fa37c07ad60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa390197c60 tx=0x7fa38000a400 comp rx=0 tx=0).stop 2024-04-23T02:17:50.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.903+0000 7fa396cb2700 1 -- 172.21.15.71:0/2946573332 shutdown_connections 2024-04-23T02:17:50.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.903+0000 7fa396cb2700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa37c07b460 0x7fa37c07d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:50.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.903+0000 7fa396cb2700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa390106cd0 0x7fa37c07ad60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:50.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.903+0000 7fa396cb2700 1 --2- 172.21.15.71:0/2946573332 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa37c06cce0 0x7fa37c06f1a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:50.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.903+0000 7fa396cb2700 1 -- 172.21.15.71:0/2946573332 >> 172.21.15.71:0/2946573332 conn(0x7fa390075670 msgr2=0x7fa3900fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:50.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.904+0000 7fa396cb2700 1 -- 172.21.15.71:0/2946573332 shutdown_connections 2024-04-23T02:17:50.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:50.904+0000 7fa396cb2700 1 -- 172.21.15.71:0/2946573332 wait complete. 2024-04-23T02:17:50.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:50 smithi071 ceph-mon[39837]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:17:50.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:50 smithi071 ceph-mon[39837]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:17:50.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:50 smithi071 ceph-mon[39837]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:17:50.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:50 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/3082579305' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:17:50.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:50 smithi071 ceph-mon[39837]: from='client.? 172.21.15.71:0/4046660517' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:17:50.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:50 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[39814]: 2024-04-23T02:17:50.677+0000 7fe2270b5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi071 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T02:17:50.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:50 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[39814]: 2024-04-23T02:17:50.677+0000 7fe2270b5700 -1 mon.smithi071@0(leader) e2 *** Got Signal Terminated *** 2024-04-23T02:17:51.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.010+0000 7f8b70152700 1 -- 172.21.15.71:0/2664964195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b68068580 msgr2=0x7f8b680689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:51.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.010+0000 7f8b70152700 1 --2- 172.21.15.71:0/2664964195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b68068580 0x7f8b680689e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009910 tx=0x7f8b58009c20 comp rx=0 tx=0).stop 2024-04-23T02:17:51.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.010+0000 7f8b70152700 1 -- 172.21.15.71:0/2664964195 shutdown_connections 2024-04-23T02:17:51.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.010+0000 7f8b70152700 1 --2- 172.21.15.71:0/2664964195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b68068580 0x7f8b680689e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:51.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.010+0000 7f8b70152700 1 --2- 172.21.15.71:0/2664964195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b68106cd0 0x7f8b681070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:51.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.010+0000 7f8b70152700 1 -- 172.21.15.71:0/2664964195 >> 172.21.15.71:0/2664964195 conn(0x7f8b68075670 msgr2=0x7f8b68075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:51.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.010+0000 7f8b70152700 1 -- 172.21.15.71:0/2664964195 shutdown_connections 2024-04-23T02:17:51.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.010+0000 7f8b70152700 1 -- 172.21.15.71:0/2664964195 wait complete. 2024-04-23T02:17:51.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.011+0000 7f8b70152700 1 Processor -- start 2024-04-23T02:17:51.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.011+0000 7f8b70152700 1 -- start start 2024-04-23T02:17:51.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.011+0000 7f8b70152700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b68068580 0x7f8b6819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:51.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.011+0000 7f8b70152700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b68106cd0 0x7f8b6819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:51.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.011+0000 7f8b70152700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6819d230 con 0x7f8b68068580 2024-04-23T02:17:51.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.011+0000 7f8b70152700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b681966e0 con 0x7f8b68106cd0 2024-04-23T02:17:51.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.011+0000 7f8b6deee700 1 -- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b68068580 msgr2=0x7f8b6819c660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:3300/0 2024-04-23T02:17:51.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.011+0000 7f8b6deee700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b68068580 0x7f8b6819c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T02:17:51.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.012+0000 7f8b6d6ed700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b68106cd0 0x7f8b6819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:51.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.012+0000 7f8b6d6ed700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b68106cd0 0x7f8b6819cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.71:57160/0 (socket says 172.21.15.71:57160) 2024-04-23T02:17:51.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.012+0000 7f8b6d6ed700 1 -- 172.21.15.71:0/3581080218 learned_addr learned my addr 172.21.15.71:0/3581080218 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:17:51.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.012+0000 7f8b6d6ed700 1 -- 172.21.15.71:0/3581080218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b68068580 msgr2=0x7f8b6819c660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T02:17:51.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.012+0000 7f8b6d6ed700 1 --2- 172.21.15.71:0/3581080218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b68068580 0x7f8b6819c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:51.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.012+0000 7f8b6d6ed700 1 -- 172.21.15.71:0/3581080218 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b580095f0 con 0x7f8b68106cd0 2024-04-23T02:17:51.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.012+0000 7f8b6d6ed700 1 --2- 172.21.15.71:0/3581080218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b68106cd0 0x7f8b6819cba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b58005b40 tx=0x7f8b5800be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:51.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.012+0000 7f8b5effd700 1 -- 172.21.15.71:0/3581080218 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b5801d070 con 0x7f8b68106cd0 2024-04-23T02:17:51.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.012+0000 7f8b70152700 1 -- 172.21.15.71:0/3581080218 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b68196960 con 0x7f8b68106cd0 2024-04-23T02:17:51.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.012+0000 7f8b5effd700 1 -- 172.21.15.71:0/3581080218 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8b580048e0 con 0x7f8b68106cd0 2024-04-23T02:17:51.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.012+0000 7f8b5effd700 1 -- 172.21.15.71:0/3581080218 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b580039d0 con 0x7f8b68106cd0 2024-04-23T02:17:51.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.012+0000 7f8b70152700 1 -- 172.21.15.71:0/3581080218 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b68196df0 con 0x7f8b68106cd0 2024-04-23T02:17:51.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.013+0000 7f8b70152700 1 -- 172.21.15.71:0/3581080218 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b681091f0 con 0x7f8b68106cd0 2024-04-23T02:17:51.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.014+0000 7f8b5effd700 1 -- 172.21.15.71:0/3581080218 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 32) v1 ==== 90842+0+0 (secure 0 0 0) 0x7f8b58004a50 con 0x7f8b68106cd0 2024-04-23T02:17:51.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.014+0000 7f8b5effd700 1 --2- 172.21.15.71:0/3581080218 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8b5406cc40 0x7f8b5406f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:17:51.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.015+0000 7f8b5effd700 1 -- 172.21.15.71:0/3581080218 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8b5808dab0 con 0x7f8b68106cd0 2024-04-23T02:17:51.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.015+0000 7f8b6deee700 1 --2- 172.21.15.71:0/3581080218 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8b5406cc40 0x7f8b5406f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:17:51.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.015+0000 7f8b6deee700 1 --2- 172.21.15.71:0/3581080218 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8b5406cc40 0x7f8b5406f100 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8b64000f20 tx=0x7f8b64006e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:17:51.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.019+0000 7f8b5effd700 1 -- 172.21.15.71:0/3581080218 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f8b5805b660 con 0x7f8b68106cd0 2024-04-23T02:17:51.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:50 smithi192 ceph-mon[49662]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:17:51.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:50 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:17:51.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:50 smithi192 ceph-mon[49662]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:17:51.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:50 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/3082579305' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:17:51.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:50 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/4046660517' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:17:51.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.296+0000 7f8b70152700 1 -- 172.21.15.71:0/3581080218 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8b6802cc70 con 0x7f8b68106cd0 2024-04-23T02:17:51.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.297+0000 7f8b5effd700 1 -- 172.21.15.71:0/3581080218 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f8b5805b1f0 con 0x7f8b68106cd0 2024-04-23T02:17:51.301 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-23T02:17:51.301 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:17:51.301 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:17:51.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.299+0000 7f8b70152700 1 -- 172.21.15.71:0/3581080218 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8b5406cc40 msgr2=0x7f8b5406f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:51.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.299+0000 7f8b70152700 1 --2- 172.21.15.71:0/3581080218 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8b5406cc40 0x7f8b5406f100 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8b64000f20 tx=0x7f8b64006e30 comp rx=0 tx=0).stop 2024-04-23T02:17:51.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.299+0000 7f8b70152700 1 -- 172.21.15.71:0/3581080218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b68106cd0 msgr2=0x7f8b6819cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:17:51.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.299+0000 7f8b70152700 1 --2- 172.21.15.71:0/3581080218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b68106cd0 0x7f8b6819cba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b58005b40 tx=0x7f8b5800be20 comp rx=0 tx=0).stop 2024-04-23T02:17:51.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.299+0000 7f8b70152700 1 -- 172.21.15.71:0/3581080218 shutdown_connections 2024-04-23T02:17:51.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.299+0000 7f8b70152700 1 --2- 172.21.15.71:0/3581080218 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8b5406cc40 0x7f8b5406f100 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:51.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.299+0000 7f8b70152700 1 --2- 172.21.15.71:0/3581080218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b68068580 0x7f8b6819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:51.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.299+0000 7f8b70152700 1 --2- 172.21.15.71:0/3581080218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b68106cd0 0x7f8b6819cba0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:17:51.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.299+0000 7f8b70152700 1 -- 172.21.15.71:0/3581080218 >> 172.21.15.71:0/3581080218 conn(0x7f8b68075670 msgr2=0x7f8b680fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:17:51.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.299+0000 7f8b70152700 1 -- 172.21.15.71:0/3581080218 shutdown_connections 2024-04-23T02:17:51.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:17:51.299+0000 7f8b70152700 1 -- 172.21.15.71:0/3581080218 wait complete. 2024-04-23T02:17:51.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:51 smithi071 bash[144308]: ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071 2024-04-23T02:17:51.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:51 smithi071 systemd[1]: ceph-91f5470c-0115-11ef-bc93-c7b262605968@mon.smithi071.service: Succeeded. 2024-04-23T02:17:51.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:51 smithi071 systemd[1]: Stopped Ceph mon.smithi071 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:17:51.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:51 smithi071 systemd[1]: Starting Ceph mon.smithi071 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T02:17:52.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: set uid:gid to 167:167 (ceph:ceph) 2024-04-23T02:17:52.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable), process ceph-mon, pid 7 2024-04-23T02:17:52.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: pidfile_write: ignore empty --pid-file 2024-04-23T02:17:52.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: load: jerasure load: lrc 2024-04-23T02:17:52.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: RocksDB version: 7.9.2 2024-04-23T02:17:52.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Git sha 0 2024-04-23T02:17:52.496 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Compile date 2024-04-18 11:37:30 2024-04-23T02:17:52.496 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: DB SUMMARY 2024-04-23T02:17:52.496 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: DB Session ID: 9SY7YFICIFNGZ39YK3OE 2024-04-23T02:17:52.496 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: CURRENT file: CURRENT 2024-04-23T02:17:52.496 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: IDENTITY file: IDENTITY 2024-04-23T02:17:52.496 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1509 Bytes 2024-04-23T02:17:52.496 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi071/store.db dir, Total Num: 1, files: 000032.sst 2024-04-23T02:17:52.496 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi071/store.db: 000030.log size: 5185226 ; 2024-04-23T02:17:52.496 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.error_if_exists: 0 2024-04-23T02:17:52.496 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.create_if_missing: 0 2024-04-23T02:17:52.496 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.paranoid_checks: 1 2024-04-23T02:17:52.496 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-23T02:17:52.496 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-23T02:17:52.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-23T02:17:52.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.env: 0x557aff897820 2024-04-23T02:17:52.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.fs: PosixFileSystem 2024-04-23T02:17:52.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.info_log: 0x557b0274f3c0 2024-04-23T02:17:52.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_file_opening_threads: 16 2024-04-23T02:17:52.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.statistics: (nil) 2024-04-23T02:17:52.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.use_fsync: 0 2024-04-23T02:17:52.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_log_file_size: 0 2024-04-23T02:17:52.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-23T02:17:52.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-23T02:17:52.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.keep_log_file_num: 1000 2024-04-23T02:17:52.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.recycle_log_file_num: 0 2024-04-23T02:17:52.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.allow_fallocate: 1 2024-04-23T02:17:52.497 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.allow_mmap_reads: 0 2024-04-23T02:17:52.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.allow_mmap_writes: 0 2024-04-23T02:17:52.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.use_direct_reads: 0 2024-04-23T02:17:52.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-23T02:17:52.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.create_missing_column_families: 0 2024-04-23T02:17:52.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.db_log_dir: 2024-04-23T02:17:52.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.wal_dir: 2024-04-23T02:17:52.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-23T02:17:52.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-23T02:17:52.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-23T02:17:52.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-23T02:17:52.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-23T02:17:52.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-23T02:17:52.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.advise_random_on_open: 1 2024-04-23T02:17:52.498 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.db_write_buffer_size: 0 2024-04-23T02:17:52.499 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.write_buffer_manager: 0x557b019c4140 2024-04-23T02:17:52.499 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-23T02:17:52.499 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-23T02:17:52.499 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-23T02:17:52.499 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.rate_limiter: (nil) 2024-04-23T02:17:52.499 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-23T02:17:52.499 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.wal_recovery_mode: 2 2024-04-23T02:17:52.499 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.enable_thread_tracking: 0 2024-04-23T02:17:52.499 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.enable_pipelined_write: 0 2024-04-23T02:17:52.499 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.unordered_write: 0 2024-04-23T02:17:52.499 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-23T02:17:52.499 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-23T02:17:52.499 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-23T02:17:52.499 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-23T02:17:52.500 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.row_cache: None 2024-04-23T02:17:52.500 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.wal_filter: None 2024-04-23T02:17:52.500 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-23T02:17:52.500 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.allow_ingest_behind: 0 2024-04-23T02:17:52.500 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.two_write_queues: 0 2024-04-23T02:17:52.500 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.manual_wal_flush: 0 2024-04-23T02:17:52.500 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.wal_compression: 0 2024-04-23T02:17:52.500 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.atomic_flush: 0 2024-04-23T02:17:52.501 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-23T02:17:52.501 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-23T02:17:52.501 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-23T02:17:52.501 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.log_readahead_size: 0 2024-04-23T02:17:52.501 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-23T02:17:52.501 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.best_efforts_recovery: 0 2024-04-23T02:17:52.501 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-23T02:17:52.501 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-23T02:17:52.501 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.allow_data_in_errors: 0 2024-04-23T02:17:52.501 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.db_host_id: __hostname__ 2024-04-23T02:17:52.501 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-23T02:17:52.501 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_background_jobs: 2 2024-04-23T02:17:52.501 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_background_compactions: -1 2024-04-23T02:17:52.501 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_subcompactions: 1 2024-04-23T02:17:52.502 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-23T02:17:52.502 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-23T02:17:52.502 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-23T02:17:52.502 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_total_wal_size: 0 2024-04-23T02:17:52.502 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-23T02:17:52.502 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-23T02:17:52.502 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-23T02:17:52.502 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-23T02:17:52.502 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_open_files: -1 2024-04-23T02:17:52.503 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.bytes_per_sync: 0 2024-04-23T02:17:52.503 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-23T02:17:52.503 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-23T02:17:52.503 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compaction_readahead_size: 0 2024-04-23T02:17:52.503 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_background_flushes: -1 2024-04-23T02:17:52.503 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Compression algorithms supported: 2024-04-23T02:17:52.503 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-23T02:17:52.503 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: kZSTD supported: 0 2024-04-23T02:17:52.503 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: kXpressCompression supported: 0 2024-04-23T02:17:52.503 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: kLZ4HCCompression supported: 1 2024-04-23T02:17:52.504 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: kZlibCompression supported: 1 2024-04-23T02:17:52.504 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: kSnappyCompression supported: 1 2024-04-23T02:17:52.504 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: kLZ4Compression supported: 1 2024-04-23T02:17:52.504 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: kBZip2Compression supported: 0 2024-04-23T02:17:52.504 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-23T02:17:52.504 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-23T02:17:52.504 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi071/store.db/MANIFEST-000015 2024-04-23T02:17:52.504 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-23T02:17:52.504 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-23T02:17:52.504 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.merge_operator: 2024-04-23T02:17:52.504 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compaction_filter: None 2024-04-23T02:17:52.505 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compaction_filter_factory: None 2024-04-23T02:17:52.505 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.sst_partitioner_factory: None 2024-04-23T02:17:52.505 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-23T02:17:52.505 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-23T02:17:52.505 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557b0274f4e0) 2024-04-23T02:17:52.505 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: cache_index_and_filter_blocks: 1 2024-04-23T02:17:52.505 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-23T02:17:52.505 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-23T02:17:52.505 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: pin_top_level_index_and_filter: 1 2024-04-23T02:17:52.505 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: index_type: 0 2024-04-23T02:17:52.505 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: data_block_index_type: 0 2024-04-23T02:17:52.505 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: index_shortening: 1 2024-04-23T02:17:52.505 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-23T02:17:52.505 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: checksum: 4 2024-04-23T02:17:52.505 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: no_block_cache: 0 2024-04-23T02:17:52.506 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: block_cache: 0x557b01a651f0 2024-04-23T02:17:52.506 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: block_cache_name: BinnedLRUCache 2024-04-23T02:17:52.506 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: block_cache_options: 2024-04-23T02:17:52.506 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: capacity : 536870912 2024-04-23T02:17:52.506 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: num_shard_bits : 4 2024-04-23T02:17:52.506 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: strict_capacity_limit : 0 2024-04-23T02:17:52.506 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: high_pri_pool_ratio: 0.000 2024-04-23T02:17:52.506 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: block_cache_compressed: (nil) 2024-04-23T02:17:52.506 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: persistent_cache: (nil) 2024-04-23T02:17:52.506 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: block_size: 4096 2024-04-23T02:17:52.506 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: block_size_deviation: 10 2024-04-23T02:17:52.506 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: block_restart_interval: 16 2024-04-23T02:17:52.506 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: index_block_restart_interval: 1 2024-04-23T02:17:52.506 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: metadata_block_size: 4096 2024-04-23T02:17:52.507 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: partition_filters: 0 2024-04-23T02:17:52.507 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: use_delta_encoding: 1 2024-04-23T02:17:52.507 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: filter_policy: bloomfilter 2024-04-23T02:17:52.507 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: whole_key_filtering: 1 2024-04-23T02:17:52.507 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: verify_compression: 0 2024-04-23T02:17:52.507 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: read_amp_bytes_per_bit: 0 2024-04-23T02:17:52.507 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: format_version: 5 2024-04-23T02:17:52.507 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: enable_index_compression: 1 2024-04-23T02:17:52.507 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: block_align: 0 2024-04-23T02:17:52.507 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: max_auto_readahead_size: 262144 2024-04-23T02:17:52.507 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: prepopulate_block_cache: 0 2024-04-23T02:17:52.507 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: initial_auto_readahead_size: 8192 2024-04-23T02:17:52.507 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: num_file_reads_for_auto_readahead: 2 2024-04-23T02:17:52.507 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.write_buffer_size: 33554432 2024-04-23T02:17:52.507 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_write_buffer_number: 2 2024-04-23T02:17:52.508 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compression: NoCompression 2024-04-23T02:17:52.508 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.bottommost_compression: Disabled 2024-04-23T02:17:52.508 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.prefix_extractor: nullptr 2024-04-23T02:17:52.508 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-23T02:17:52.508 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.num_levels: 7 2024-04-23T02:17:52.508 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-23T02:17:52.508 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-23T02:17:52.508 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-23T02:17:52.508 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-23T02:17:52.508 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-23T02:17:52.508 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-23T02:17:52.508 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-23T02:17:52.508 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-23T02:17:52.508 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-23T02:17:52.509 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-23T02:17:52.509 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-23T02:17:52.509 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-23T02:17:52.509 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-23T02:17:52.509 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compression_opts.level: 32767 2024-04-23T02:17:52.509 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compression_opts.strategy: 0 2024-04-23T02:17:52.509 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-23T02:17:52.509 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-23T02:17:52.509 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-23T02:17:52.509 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-23T02:17:52.509 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compression_opts.enabled: false 2024-04-23T02:17:52.509 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-23T02:17:52.509 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-23T02:17:52.509 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-23T02:17:52.509 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-23T02:17:52.510 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.target_file_size_base: 67108864 2024-04-23T02:17:52.510 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-23T02:17:52.510 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-23T02:17:52.510 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-23T02:17:52.510 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-23T02:17:52.510 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-23T02:17:52.510 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-23T02:17:52.510 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-23T02:17:52.511 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-23T02:17:52.511 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-23T02:17:52.511 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-23T02:17:52.511 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-23T02:17:52.511 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-23T02:17:52.511 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-23T02:17:52.511 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-23T02:17:52.511 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.arena_block_size: 1048576 2024-04-23T02:17:52.511 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-23T02:17:52.511 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-23T02:17:52.511 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.disable_auto_compactions: 0 2024-04-23T02:17:52.511 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-23T02:17:52.512 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-23T02:17:52.512 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-23T02:17:52.512 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-23T02:17:52.512 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-23T02:17:52.513 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-23T02:17:52.513 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-23T02:17:52.513 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-23T02:17:52.513 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-23T02:17:52.513 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-23T02:17:52.513 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.table_properties_collectors: 2024-04-23T02:17:52.513 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.inplace_update_support: 0 2024-04-23T02:17:52.513 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-23T02:17:52.513 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-23T02:17:52.513 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-23T02:17:52.513 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-23T02:17:52.513 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.bloom_locality: 0 2024-04-23T02:17:52.513 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.max_successive_merges: 0 2024-04-23T02:17:52.513 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-23T02:17:52.514 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.paranoid_file_checks: 0 2024-04-23T02:17:52.514 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.force_consistency_checks: 1 2024-04-23T02:17:52.514 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.report_bg_io_stats: 0 2024-04-23T02:17:52.514 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.ttl: 2592000 2024-04-23T02:17:52.514 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-23T02:17:52.514 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-23T02:17:52.514 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-23T02:17:52.514 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.enable_blob_files: false 2024-04-23T02:17:52.514 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.min_blob_size: 0 2024-04-23T02:17:52.514 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.blob_file_size: 268435456 2024-04-23T02:17:52.514 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-23T02:17:52.514 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-23T02:17:52.514 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-23T02:17:52.514 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-23T02:17:52.515 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-23T02:17:52.515 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.blob_file_starting_level: 0 2024-04-23T02:17:52.515 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-23T02:17:52.515 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-23T02:17:52.515 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi071/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12106, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-04-23T02:17:52.515 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-04-23T02:17:52.515 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 41362561-8ff1-4644-9c70-3e52d3ce932e 2024-04-23T02:17:52.515 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713838672236672, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-04-23T02:17:52.515 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-04-23T02:17:52.515 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713838672252418, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 4607925, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12107, "largest_seqno": 15727, "table_properties": {"data_size": 4593822, "index_size": 8986, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 4101, "raw_key_size": 35915, "raw_average_key_size": 22, "raw_value_size": 4561775, "raw_average_value_size": 2812, "num_data_blocks": 406, "num_entries": 1622, "num_filter_entries": 1622, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713838672, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "41362561-8ff1-4644-9c70-3e52d3ce932e", "db_session_id": "9SY7YFICIFNGZ39YK3OE", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-04-23T02:17:52.515 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713838672252534, "job": 1, "event": "recovery_finished"} 2024-04-23T02:17:52.515 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-04-23T02:17:52.515 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-23T02:17:52.515 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi071/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-23T02:17:52.515 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557b01b04000 2024-04-23T02:17:52.516 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: DB pointer 0x557b01af2000 2024-04-23T02:17:52.516 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-23T02:17:52.516 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-23T02:17:52.516 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: ** DB Stats ** 2024-04-23T02:17:52.516 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T02:17:52.516 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-23T02:17:52.516 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T02:17:52.516 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T02:17:52.516 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-23T02:17:52.516 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T02:17:52.516 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T02:17:52.516 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: 2024-04-23T02:17:52.516 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: ** Compaction Stats [default] ** 2024-04-23T02:17:52.516 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T02:17:52.517 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-23T02:17:52.517 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: L0 1/0 4.39 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 561.3 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-23T02:17:52.517 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: L6 1/0 4.68 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-23T02:17:52.517 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Sum 2/0 9.08 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 561.3 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-23T02:17:52.517 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 561.3 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-23T02:17:52.517 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: 2024-04-23T02:17:52.517 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: ** Compaction Stats [default] ** 2024-04-23T02:17:52.517 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T02:17:52.517 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-23T02:17:52.517 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 561.3 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-23T02:17:52.517 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: 2024-04-23T02:17:52.517 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-23T02:17:52.517 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: 2024-04-23T02:17:52.517 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T02:17:52.517 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Flush(GB): cumulative 0.004, interval 0.004 2024-04-23T02:17:52.518 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-23T02:17:52.518 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-23T02:17:52.518 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-23T02:17:52.518 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-23T02:17:52.518 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Cumulative compaction: 0.00 GB write, 238.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T02:17:52.518 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Interval compaction: 0.00 GB write, 238.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T02:17:52.518 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-23T02:17:52.518 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Block cache BinnedLRUCache@0x557b01a651f0#7 capacity: 512.00 MB usage: 13.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-23T02:17:52.518 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.53 KB,0.000864267%) IndexBlock(1,9.08 KB,0.00173151%) Misc(1,0.00 KB,0%) 2024-04-23T02:17:52.518 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: 2024-04-23T02:17:52.518 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-23T02:17:52.518 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: starting mon.smithi071 rank 0 at public addrs [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] at bind addrs [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi071 fsid 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T02:17:52.518 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: mon.smithi071@-1(???) e2 preinit fsid 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T02:17:52.518 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: mon.smithi071@-1(???).mds e25 new map 2024-04-23T02:17:52.518 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: mon.smithi071@-1(???).mds e25 print_map 2024-04-23T02:17:52.519 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: e25 2024-04-23T02:17:52.519 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:17:52.519 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:17:52.519 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: legacy client fscid: 1 2024-04-23T02:17:52.519 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: 2024-04-23T02:17:52.519 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: Filesystem 'cephfs' (1) 2024-04-23T02:17:52.519 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: fs_name cephfs 2024-04-23T02:17:52.519 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: epoch 25 2024-04-23T02:17:52.519 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T02:17:52.519 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:17:52.519 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: modified 2024-04-23T02:17:27.814232+0000 2024-04-23T02:17:52.519 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: tableserver 0 2024-04-23T02:17:52.519 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: root 0 2024-04-23T02:17:52.519 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: session_timeout 60 2024-04-23T02:17:52.520 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: session_autoclose 300 2024-04-23T02:17:52.520 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: max_file_size 1099511627776 2024-04-23T02:17:52.520 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: max_xattr_size 65536 2024-04-23T02:17:52.520 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: required_client_features {} 2024-04-23T02:17:52.520 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: last_failure 0 2024-04-23T02:17:52.520 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: last_failure_osd_epoch 46 2024-04-23T02:17:52.520 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:17:52.520 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: max_mds 2 2024-04-23T02:17:52.520 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: in 0,1 2024-04-23T02:17:52.520 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: up {0=14478,1=24243} 2024-04-23T02:17:52.520 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: failed 2024-04-23T02:17:52.520 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: damaged 2024-04-23T02:17:52.520 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: stopped 2024-04-23T02:17:52.520 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: data_pools [3] 2024-04-23T02:17:52.521 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: metadata_pool 2 2024-04-23T02:17:52.521 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: inline_data enabled 2024-04-23T02:17:52.521 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: balancer 2024-04-23T02:17:52.521 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: bal_rank_mask -1 2024-04-23T02:17:52.521 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: standby_count_wanted 1 2024-04-23T02:17:52.521 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: [mds.cephfs.smithi071.vxejye{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:17:52.521 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: [mds.cephfs.smithi071.qusrrb{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/3442634960,v1:172.21.15.71:6827/3442634960] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:17:52.521 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: [mds.cephfs.smithi192.gtmyst{1:24243} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.192:6824/3780549896,v1:172.21.15.192:6825/3780549896] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:17:52.521 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: [mds.cephfs.smithi192.azdyze{1:24255} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2100078284,v1:172.21.15.192:6827/2100078284] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:17:52.521 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: 2024-04-23T02:17:52.521 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout: 2024-04-23T02:17:52.521 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: mon.smithi071@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-23T02:17:52.521 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: mon.smithi071@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-23T02:17:52.521 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: mon.smithi071@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-23T02:17:52.522 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: mon.smithi071@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-23T02:17:52.522 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 ceph-mon[144544]: mon.smithi071@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-04-23T02:17:52.522 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 bash[144506]: 16378807fa69ab891bb14084eb9d5be0959e85174ef956013e7c5c5a22f64fac 2024-04-23T02:17:52.522 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:52 smithi071 systemd[1]: Started Ceph mon.smithi071 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:17:55.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:55 smithi192 ceph-mon[49662]: pgmap v101: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:55.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:55 smithi192 ceph-mon[49662]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:17:55.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:55 smithi192 ceph-mon[49662]: from='client.? 172.21.15.71:0/3581080218' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:17:55.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:55 smithi192 ceph-mon[49662]: pgmap v102: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:55.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:55 smithi192 ceph-mon[49662]: pgmap v103: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:55.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:55 smithi192 ceph-mon[49662]: mon.smithi071 calling monitor election 2024-04-23T02:17:55.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:55 smithi192 ceph-mon[49662]: mon.smithi071 is new leader, mons smithi071,smithi192 in quorum (ranks 0,1) 2024-04-23T02:17:55.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:55 smithi192 ceph-mon[49662]: monmap e2: 2 mons at {smithi071=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],smithi192=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-23T02:17:55.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:55 smithi192 ceph-mon[49662]: fsmap cephfs:2 {0=cephfs.smithi071.vxejye=up:active,1=cephfs.smithi192.gtmyst=up:active} 2 up:standby-replay 2024-04-23T02:17:55.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:55 smithi192 ceph-mon[49662]: osdmap e48: 6 total, 6 up, 6 in 2024-04-23T02:17:55.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:55 smithi192 ceph-mon[49662]: mgrmap e32: smithi071.ukdjvn(active, since 3m), standbys: smithi192.qgecnv 2024-04-23T02:17:55.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:55 smithi192 ceph-mon[49662]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-23T02:17:55.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:55 smithi192 ceph-mon[49662]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:17:55.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:55 smithi192 ceph-mon[49662]: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:17:55.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:55 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='' 2024-04-23T02:17:55.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:55 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:17:55.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:55 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:17:55.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:55 smithi071 ceph-mon[144544]: pgmap v101: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:55.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:55 smithi071 ceph-mon[144544]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:17:55.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:55 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3581080218' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:17:55.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:55 smithi071 ceph-mon[144544]: pgmap v102: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:55.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:55 smithi071 ceph-mon[144544]: pgmap v103: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:17:55.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:55 smithi071 ceph-mon[144544]: mon.smithi071 calling monitor election 2024-04-23T02:17:55.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:55 smithi071 ceph-mon[144544]: mon.smithi071 is new leader, mons smithi071,smithi192 in quorum (ranks 0,1) 2024-04-23T02:17:55.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:55 smithi071 ceph-mon[144544]: monmap e2: 2 mons at {smithi071=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],smithi192=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-23T02:17:55.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:55 smithi071 ceph-mon[144544]: fsmap cephfs:2 {0=cephfs.smithi071.vxejye=up:active,1=cephfs.smithi192.gtmyst=up:active} 2 up:standby-replay 2024-04-23T02:17:55.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:55 smithi071 ceph-mon[144544]: osdmap e48: 6 total, 6 up, 6 in 2024-04-23T02:17:55.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:55 smithi071 ceph-mon[144544]: mgrmap e32: smithi071.ukdjvn(active, since 3m), standbys: smithi192.qgecnv 2024-04-23T02:17:55.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:55 smithi071 ceph-mon[144544]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-23T02:17:55.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:55 smithi071 ceph-mon[144544]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:17:55.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:55 smithi071 ceph-mon[144544]: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:17:55.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:55 smithi071 ceph-mon[144544]: from='mgr.14660 ' entity='' 2024-04-23T02:17:55.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:55 smithi071 ceph-mon[144544]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:17:55.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:55 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:17:57.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:57 smithi071 ceph-mon[144544]: pgmap v104: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:58.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:57 smithi192 ceph-mon[49662]: pgmap v104: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:17:59.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:17:59 smithi071 ceph-mon[144544]: pgmap v105: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:00.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:17:59 smithi192 ceph-mon[49662]: pgmap v105: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:02.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:01 smithi071 ceph-mon[144544]: pgmap v106: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:18:02.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:01 smithi071 ceph-mon[144544]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:02.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:01 smithi071 ceph-mon[144544]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:02.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:01 smithi071 ceph-mon[144544]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:02.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:01 smithi071 ceph-mon[144544]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:02.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:01 smithi192 ceph-mon[49662]: pgmap v106: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:18:02.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:01 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:02.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:01 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:02.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:01 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:02.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:01 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:04.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:03 smithi071 ceph-mon[144544]: pgmap v107: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:04.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:03 smithi192 ceph-mon[49662]: pgmap v107: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:05.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:04 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:18:05.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:04 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:18:06.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:06 smithi192 ceph-mon[49662]: pgmap v108: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:06.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:06 smithi192 ceph-mon[49662]: Detected new or changed devices on smithi071 2024-04-23T02:18:06.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:06 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:06.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:06 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:06.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:06 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:18:06.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:06 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:18:06.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:06 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:06.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:06 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:18:06.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:06 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:18:06.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:06 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-23T02:18:06.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:06 smithi071 ceph-mon[144544]: pgmap v108: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:06.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:06 smithi071 ceph-mon[144544]: Detected new or changed devices on smithi071 2024-04-23T02:18:06.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:06 smithi071 ceph-mon[144544]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:06.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:06 smithi071 ceph-mon[144544]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:06.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:06 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:18:06.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:06 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:18:06.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:06 smithi071 ceph-mon[144544]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:06.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:06 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:18:06.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:06 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:18:06.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:06 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-23T02:18:08.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:08 smithi192 ceph-mon[49662]: pgmap v109: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:18:08.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:08 smithi192 ceph-mon[49662]: Upgrade: Updating mon.smithi192 2024-04-23T02:18:08.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:08 smithi192 ceph-mon[49662]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:08.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:08 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T02:18:08.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:08 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T02:18:08.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:08 smithi192 ceph-mon[49662]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:18:08.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:08 smithi192 ceph-mon[49662]: Deploying daemon mon.smithi192 on smithi192 2024-04-23T02:18:08.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:08 smithi071 ceph-mon[144544]: pgmap v109: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:18:08.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:08 smithi071 ceph-mon[144544]: Upgrade: Updating mon.smithi192 2024-04-23T02:18:08.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:08 smithi071 ceph-mon[144544]: from='mgr.14660 ' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:08.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:08 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T02:18:08.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:08 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T02:18:08.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:08 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:18:08.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:08 smithi071 ceph-mon[144544]: Deploying daemon mon.smithi192 on smithi192 2024-04-23T02:18:09.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:09 smithi192 systemd[1]: Stopping Ceph mon.smithi192 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T02:18:09.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:09 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi192[49628]: 2024-04-23T02:18:09.359+0000 7f856b606700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi192 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T02:18:09.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:09 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi192[49628]: 2024-04-23T02:18:09.359+0000 7f856b606700 -1 mon.smithi192@1(peon) e2 *** Got Signal Terminated *** 2024-04-23T02:18:10.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:09 smithi192 bash[108423]: ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi192 2024-04-23T02:18:10.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 systemd[1]: ceph-91f5470c-0115-11ef-bc93-c7b262605968@mon.smithi192.service: Succeeded. 2024-04-23T02:18:10.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 systemd[1]: Stopped Ceph mon.smithi192 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:18:10.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 systemd[1]: Starting Ceph mon.smithi192 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T02:18:11.047 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: set uid:gid to 167:167 (ceph:ceph) 2024-04-23T02:18:11.047 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable), process ceph-mon, pid 7 2024-04-23T02:18:11.047 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: pidfile_write: ignore empty --pid-file 2024-04-23T02:18:11.047 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: load: jerasure load: lrc 2024-04-23T02:18:11.047 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: RocksDB version: 7.9.2 2024-04-23T02:18:11.047 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Git sha 0 2024-04-23T02:18:11.047 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Compile date 2024-04-18 11:37:30 2024-04-23T02:18:11.048 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: DB SUMMARY 2024-04-23T02:18:11.048 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: DB Session ID: X6ITDR64KQ2WOCUERG4J 2024-04-23T02:18:11.048 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: CURRENT file: CURRENT 2024-04-23T02:18:11.048 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: IDENTITY file: IDENTITY 2024-04-23T02:18:11.048 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1403 Bytes 2024-04-23T02:18:11.048 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi192/store.db dir, Total Num: 1, files: 000027.sst 2024-04-23T02:18:11.048 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi192/store.db: 000025.log size: 8352738 ; 2024-04-23T02:18:11.048 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.error_if_exists: 0 2024-04-23T02:18:11.048 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.create_if_missing: 0 2024-04-23T02:18:11.048 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.paranoid_checks: 1 2024-04-23T02:18:11.048 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-23T02:18:11.048 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-23T02:18:11.048 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-23T02:18:11.049 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.env: 0x55e97b766820 2024-04-23T02:18:11.049 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.fs: PosixFileSystem 2024-04-23T02:18:11.049 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.info_log: 0x55e97e1733c0 2024-04-23T02:18:11.049 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_file_opening_threads: 16 2024-04-23T02:18:11.049 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.statistics: (nil) 2024-04-23T02:18:11.049 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.use_fsync: 0 2024-04-23T02:18:11.049 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_log_file_size: 0 2024-04-23T02:18:11.049 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-23T02:18:11.049 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-23T02:18:11.049 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.keep_log_file_num: 1000 2024-04-23T02:18:11.049 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.recycle_log_file_num: 0 2024-04-23T02:18:11.049 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.allow_fallocate: 1 2024-04-23T02:18:11.049 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.allow_mmap_reads: 0 2024-04-23T02:18:11.049 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.allow_mmap_writes: 0 2024-04-23T02:18:11.050 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.use_direct_reads: 0 2024-04-23T02:18:11.050 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-23T02:18:11.050 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.create_missing_column_families: 0 2024-04-23T02:18:11.050 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.db_log_dir: 2024-04-23T02:18:11.050 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.wal_dir: 2024-04-23T02:18:11.050 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-23T02:18:11.050 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-23T02:18:11.050 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-23T02:18:11.050 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-23T02:18:11.050 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-23T02:18:11.050 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-23T02:18:11.050 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.advise_random_on_open: 1 2024-04-23T02:18:11.050 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.db_write_buffer_size: 0 2024-04-23T02:18:11.051 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.write_buffer_manager: 0x55e97d3e8140 2024-04-23T02:18:11.051 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-23T02:18:11.051 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-23T02:18:11.051 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-23T02:18:11.051 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.rate_limiter: (nil) 2024-04-23T02:18:11.051 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-23T02:18:11.051 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.wal_recovery_mode: 2 2024-04-23T02:18:11.051 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.enable_thread_tracking: 0 2024-04-23T02:18:11.051 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.enable_pipelined_write: 0 2024-04-23T02:18:11.051 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.unordered_write: 0 2024-04-23T02:18:11.051 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-23T02:18:11.051 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-23T02:18:11.051 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-23T02:18:11.051 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-23T02:18:11.051 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.row_cache: None 2024-04-23T02:18:11.052 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.wal_filter: None 2024-04-23T02:18:11.052 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-23T02:18:11.052 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.allow_ingest_behind: 0 2024-04-23T02:18:11.052 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.two_write_queues: 0 2024-04-23T02:18:11.052 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.manual_wal_flush: 0 2024-04-23T02:18:11.052 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.wal_compression: 0 2024-04-23T02:18:11.052 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.atomic_flush: 0 2024-04-23T02:18:11.052 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-23T02:18:11.052 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-23T02:18:11.052 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-23T02:18:11.052 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.log_readahead_size: 0 2024-04-23T02:18:11.052 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-23T02:18:11.052 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.best_efforts_recovery: 0 2024-04-23T02:18:11.052 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-23T02:18:11.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-23T02:18:11.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.allow_data_in_errors: 0 2024-04-23T02:18:11.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.db_host_id: __hostname__ 2024-04-23T02:18:11.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-23T02:18:11.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_background_jobs: 2 2024-04-23T02:18:11.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_background_compactions: -1 2024-04-23T02:18:11.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_subcompactions: 1 2024-04-23T02:18:11.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-23T02:18:11.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-23T02:18:11.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-23T02:18:11.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_total_wal_size: 0 2024-04-23T02:18:11.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-23T02:18:11.053 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-23T02:18:11.054 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-23T02:18:11.054 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-23T02:18:11.054 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_open_files: -1 2024-04-23T02:18:11.054 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.bytes_per_sync: 0 2024-04-23T02:18:11.054 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-23T02:18:11.054 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-23T02:18:11.054 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compaction_readahead_size: 0 2024-04-23T02:18:11.054 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_background_flushes: -1 2024-04-23T02:18:11.054 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Compression algorithms supported: 2024-04-23T02:18:11.054 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-23T02:18:11.054 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: kZSTD supported: 0 2024-04-23T02:18:11.054 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: kXpressCompression supported: 0 2024-04-23T02:18:11.054 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: kLZ4HCCompression supported: 1 2024-04-23T02:18:11.054 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: kZlibCompression supported: 1 2024-04-23T02:18:11.055 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: kSnappyCompression supported: 1 2024-04-23T02:18:11.055 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: kLZ4Compression supported: 1 2024-04-23T02:18:11.055 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: kBZip2Compression supported: 0 2024-04-23T02:18:11.055 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-23T02:18:11.055 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-23T02:18:11.055 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi192/store.db/MANIFEST-000010 2024-04-23T02:18:11.055 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-23T02:18:11.055 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-23T02:18:11.055 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.merge_operator: 2024-04-23T02:18:11.055 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compaction_filter: None 2024-04-23T02:18:11.055 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compaction_filter_factory: None 2024-04-23T02:18:11.055 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.sst_partitioner_factory: None 2024-04-23T02:18:11.055 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-23T02:18:11.055 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-23T02:18:11.055 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e97e1734e0) 2024-04-23T02:18:11.056 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: cache_index_and_filter_blocks: 1 2024-04-23T02:18:11.056 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-23T02:18:11.056 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-23T02:18:11.056 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: pin_top_level_index_and_filter: 1 2024-04-23T02:18:11.056 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: index_type: 0 2024-04-23T02:18:11.056 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: data_block_index_type: 0 2024-04-23T02:18:11.056 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: index_shortening: 1 2024-04-23T02:18:11.056 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-23T02:18:11.056 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: checksum: 4 2024-04-23T02:18:11.056 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: no_block_cache: 0 2024-04-23T02:18:11.056 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_cache: 0x55e97d4891f0 2024-04-23T02:18:11.056 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_cache_name: BinnedLRUCache 2024-04-23T02:18:11.056 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_cache_options: 2024-04-23T02:18:11.056 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: capacity : 536870912 2024-04-23T02:18:11.057 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: num_shard_bits : 4 2024-04-23T02:18:11.057 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: strict_capacity_limit : 0 2024-04-23T02:18:11.057 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: high_pri_pool_ratio: 0.000 2024-04-23T02:18:11.057 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_cache_compressed: (nil) 2024-04-23T02:18:11.057 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: persistent_cache: (nil) 2024-04-23T02:18:11.057 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_size: 4096 2024-04-23T02:18:11.057 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_size_deviation: 10 2024-04-23T02:18:11.057 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_restart_interval: 16 2024-04-23T02:18:11.057 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: index_block_restart_interval: 1 2024-04-23T02:18:11.057 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: metadata_block_size: 4096 2024-04-23T02:18:11.057 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: partition_filters: 0 2024-04-23T02:18:11.057 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: use_delta_encoding: 1 2024-04-23T02:18:11.057 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: filter_policy: bloomfilter 2024-04-23T02:18:11.057 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: whole_key_filtering: 1 2024-04-23T02:18:11.057 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: verify_compression: 0 2024-04-23T02:18:11.058 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: read_amp_bytes_per_bit: 0 2024-04-23T02:18:11.058 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: format_version: 5 2024-04-23T02:18:11.058 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: enable_index_compression: 1 2024-04-23T02:18:11.058 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: block_align: 0 2024-04-23T02:18:11.058 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: max_auto_readahead_size: 262144 2024-04-23T02:18:11.058 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: prepopulate_block_cache: 0 2024-04-23T02:18:11.058 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: initial_auto_readahead_size: 8192 2024-04-23T02:18:11.058 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: num_file_reads_for_auto_readahead: 2 2024-04-23T02:18:11.058 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.write_buffer_size: 33554432 2024-04-23T02:18:11.058 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_write_buffer_number: 2 2024-04-23T02:18:11.058 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compression: NoCompression 2024-04-23T02:18:11.058 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.bottommost_compression: Disabled 2024-04-23T02:18:11.058 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.prefix_extractor: nullptr 2024-04-23T02:18:11.059 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-23T02:18:11.059 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.num_levels: 7 2024-04-23T02:18:11.059 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-23T02:18:11.059 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-23T02:18:11.059 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-23T02:18:11.059 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-23T02:18:11.059 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-23T02:18:11.059 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-23T02:18:11.059 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-23T02:18:11.059 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-23T02:18:11.059 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-23T02:18:11.059 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-23T02:18:11.060 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-23T02:18:11.060 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-23T02:18:11.060 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-23T02:18:11.060 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compression_opts.level: 32767 2024-04-23T02:18:11.060 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compression_opts.strategy: 0 2024-04-23T02:18:11.060 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-23T02:18:11.060 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-23T02:18:11.060 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-23T02:18:11.060 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-23T02:18:11.060 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compression_opts.enabled: false 2024-04-23T02:18:11.060 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-23T02:18:11.060 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-23T02:18:11.060 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-23T02:18:11.060 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-23T02:18:11.060 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.target_file_size_base: 67108864 2024-04-23T02:18:11.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-23T02:18:11.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-23T02:18:11.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-23T02:18:11.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-23T02:18:11.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-23T02:18:11.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-23T02:18:11.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-23T02:18:11.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-23T02:18:11.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-23T02:18:11.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-23T02:18:11.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-23T02:18:11.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-23T02:18:11.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-23T02:18:11.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-23T02:18:11.061 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.arena_block_size: 1048576 2024-04-23T02:18:11.062 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-23T02:18:11.062 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-23T02:18:11.062 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.disable_auto_compactions: 0 2024-04-23T02:18:11.062 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-23T02:18:11.063 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-23T02:18:11.063 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-23T02:18:11.063 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-23T02:18:11.063 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-23T02:18:11.063 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-23T02:18:11.063 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-23T02:18:11.063 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-23T02:18:11.063 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-23T02:18:11.064 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-23T02:18:11.064 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.table_properties_collectors: 2024-04-23T02:18:11.064 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.inplace_update_support: 0 2024-04-23T02:18:11.064 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-23T02:18:11.064 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-23T02:18:11.064 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-23T02:18:11.064 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-23T02:18:11.064 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.bloom_locality: 0 2024-04-23T02:18:11.064 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.max_successive_merges: 0 2024-04-23T02:18:11.064 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-23T02:18:11.064 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.paranoid_file_checks: 0 2024-04-23T02:18:11.064 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.force_consistency_checks: 1 2024-04-23T02:18:11.064 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.report_bg_io_stats: 0 2024-04-23T02:18:11.064 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.ttl: 2592000 2024-04-23T02:18:11.065 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-23T02:18:11.065 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-23T02:18:11.065 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-23T02:18:11.065 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.enable_blob_files: false 2024-04-23T02:18:11.065 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.min_blob_size: 0 2024-04-23T02:18:11.065 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.blob_file_size: 268435456 2024-04-23T02:18:11.065 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-23T02:18:11.065 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-23T02:18:11.065 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-23T02:18:11.065 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-23T02:18:11.065 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-23T02:18:11.065 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.blob_file_starting_level: 0 2024-04-23T02:18:11.065 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-23T02:18:11.065 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-23T02:18:11.066 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi192/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12075, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-04-23T02:18:11.066 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-04-23T02:18:11.066 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 395cc7b7-f26b-46a3-a605-1e908d9b66c1 2024-04-23T02:18:11.066 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713838690940763, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-04-23T02:18:11.066 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-04-23T02:18:11.066 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713838690963977, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 4924044, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12080, "largest_seqno": 16356, "table_properties": {"data_size": 4908980, "index_size": 9691, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 4357, "raw_key_size": 38300, "raw_average_key_size": 22, "raw_value_size": 4874699, "raw_average_value_size": 2825, "num_data_blocks": 438, "num_entries": 1725, "num_filter_entries": 1725, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713838690, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "395cc7b7-f26b-46a3-a605-1e908d9b66c1", "db_session_id": "X6ITDR64KQ2WOCUERG4J", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-04-23T02:18:11.066 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713838690964087, "job": 1, "event": "recovery_finished"} 2024-04-23T02:18:11.066 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-04-23T02:18:11.066 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-23T02:18:11.066 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi192/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-23T02:18:11.066 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e97d528000 2024-04-23T02:18:11.066 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: DB pointer 0x55e97d516000 2024-04-23T02:18:11.066 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-23T02:18:11.067 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-23T02:18:11.067 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** DB Stats ** 2024-04-23T02:18:11.067 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T02:18:11.067 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-23T02:18:11.067 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T02:18:11.067 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T02:18:11.067 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-23T02:18:11.067 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T02:18:11.067 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T02:18:11.067 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-23T02:18:11.067 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** Compaction Stats [default] ** 2024-04-23T02:18:11.067 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T02:18:11.067 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-23T02:18:11.067 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: L0 1/0 4.70 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 569.8 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-23T02:18:11.068 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: L6 1/0 4.68 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-23T02:18:11.068 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Sum 2/0 9.38 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 569.8 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-23T02:18:11.068 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 569.8 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-23T02:18:11.068 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-23T02:18:11.068 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** Compaction Stats [default] ** 2024-04-23T02:18:11.068 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T02:18:11.068 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-23T02:18:11.068 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 569.8 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-23T02:18:11.068 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-23T02:18:11.068 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-23T02:18:11.068 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-23T02:18:11.068 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T02:18:11.068 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Flush(GB): cumulative 0.005, interval 0.005 2024-04-23T02:18:11.068 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-23T02:18:11.069 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-23T02:18:11.069 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-23T02:18:11.069 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-23T02:18:11.069 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Cumulative compaction: 0.00 GB write, 159.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T02:18:11.069 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Interval compaction: 0.00 GB write, 159.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T02:18:11.069 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-23T02:18:11.069 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Block cache BinnedLRUCache@0x55e97d4891f0#7 capacity: 512.00 MB usage: 14.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-04-23T02:18:11.069 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.53 KB,0.000864267%) IndexBlock(1,10.08 KB,0.00192225%) Misc(1,0.00 KB,0%) 2024-04-23T02:18:11.069 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-23T02:18:11.069 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-23T02:18:11.070 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: starting mon.smithi192 rank 1 at public addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] at bind addrs [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi192 fsid 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T02:18:11.070 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: mon.smithi192@-1(???) e2 preinit fsid 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T02:18:11.070 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: mon.smithi192@-1(???).mds e25 new map 2024-04-23T02:18:11.070 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: mon.smithi192@-1(???).mds e25 print_map 2024-04-23T02:18:11.070 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: e25 2024-04-23T02:18:11.070 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:18:11.070 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:18:11.070 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: legacy client fscid: 1 2024-04-23T02:18:11.070 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-23T02:18:11.070 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: Filesystem 'cephfs' (1) 2024-04-23T02:18:11.070 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: fs_name cephfs 2024-04-23T02:18:11.070 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: epoch 25 2024-04-23T02:18:11.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T02:18:11.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:18:11.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: modified 2024-04-23T02:17:27.814232+0000 2024-04-23T02:18:11.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: tableserver 0 2024-04-23T02:18:11.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: root 0 2024-04-23T02:18:11.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: session_timeout 60 2024-04-23T02:18:11.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: session_autoclose 300 2024-04-23T02:18:11.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: max_file_size 1099511627776 2024-04-23T02:18:11.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: max_xattr_size 65536 2024-04-23T02:18:11.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: required_client_features {} 2024-04-23T02:18:11.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: last_failure 0 2024-04-23T02:18:11.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: last_failure_osd_epoch 46 2024-04-23T02:18:11.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:18:11.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: max_mds 2 2024-04-23T02:18:11.071 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: in 0,1 2024-04-23T02:18:11.072 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: up {0=14478,1=24243} 2024-04-23T02:18:11.072 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: failed 2024-04-23T02:18:11.072 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: damaged 2024-04-23T02:18:11.072 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: stopped 2024-04-23T02:18:11.072 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: data_pools [3] 2024-04-23T02:18:11.072 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: metadata_pool 2 2024-04-23T02:18:11.072 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: inline_data enabled 2024-04-23T02:18:11.072 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: balancer 2024-04-23T02:18:11.072 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: bal_rank_mask -1 2024-04-23T02:18:11.072 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: standby_count_wanted 1 2024-04-23T02:18:11.072 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: [mds.cephfs.smithi071.vxejye{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:18:11.072 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: [mds.cephfs.smithi071.qusrrb{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/3442634960,v1:172.21.15.71:6827/3442634960] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:18:11.072 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: [mds.cephfs.smithi192.gtmyst{1:24243} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.192:6824/3780549896,v1:172.21.15.192:6825/3780549896] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:18:11.072 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: [mds.cephfs.smithi192.azdyze{1:24255} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2100078284,v1:172.21.15.192:6827/2100078284] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:18:11.073 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-23T02:18:11.073 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout: 2024-04-23T02:18:11.073 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: mon.smithi192@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-23T02:18:11.073 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: mon.smithi192@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-23T02:18:11.073 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: mon.smithi192@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-23T02:18:11.073 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: mon.smithi192@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-23T02:18:11.073 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:10 smithi192 ceph-mon[108617]: mon.smithi192@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-04-23T02:18:11.073 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:11 smithi192 bash[108579]: a25130f1b27d1ae5295e52b5341b88135aca61a46c6ab6ea460c934bd29a524a 2024-04-23T02:18:11.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:11 smithi192 systemd[1]: Started Ceph mon.smithi192 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:18:11.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:11 smithi071 ceph-mon[144544]: pgmap v110: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:11.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:11 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi071"}]: dispatch 2024-04-23T02:18:11.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:11 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:18:12.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:11 smithi192 ceph-mon[108617]: pgmap v110: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:12.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:11 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi071"}]: dispatch 2024-04-23T02:18:12.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:11 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mon metadata", "id": "smithi192"}]: dispatch 2024-04-23T02:18:12.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:12 smithi192 ceph-mon[108617]: pgmap v111: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:18:12.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:12 smithi192 ceph-mon[108617]: mon.smithi192 calling monitor election 2024-04-23T02:18:12.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:12 smithi192 ceph-mon[108617]: mon.smithi071 calling monitor election 2024-04-23T02:18:12.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:12 smithi192 ceph-mon[108617]: mon.smithi071 is new leader, mons smithi071,smithi192 in quorum (ranks 0,1) 2024-04-23T02:18:12.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:12 smithi192 ceph-mon[108617]: monmap e2: 2 mons at {smithi071=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],smithi192=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-23T02:18:12.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:12 smithi192 ceph-mon[108617]: fsmap cephfs:2 {0=cephfs.smithi071.vxejye=up:active,1=cephfs.smithi192.gtmyst=up:active} 2 up:standby-replay 2024-04-23T02:18:12.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:12 smithi192 ceph-mon[108617]: osdmap e48: 6 total, 6 up, 6 in 2024-04-23T02:18:12.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:12 smithi192 ceph-mon[108617]: mgrmap e32: smithi071.ukdjvn(active, since 3m), standbys: smithi192.qgecnv 2024-04-23T02:18:12.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:12 smithi192 ceph-mon[108617]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-23T02:18:12.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:12 smithi192 ceph-mon[108617]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:18:12.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:12 smithi192 ceph-mon[108617]: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:18:12.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:12 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:12.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:12 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:12.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:12 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:18:12.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:12 smithi071 ceph-mon[144544]: pgmap v111: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:18:12.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:12 smithi071 ceph-mon[144544]: mon.smithi192 calling monitor election 2024-04-23T02:18:12.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:12 smithi071 ceph-mon[144544]: mon.smithi071 calling monitor election 2024-04-23T02:18:12.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:12 smithi071 ceph-mon[144544]: mon.smithi071 is new leader, mons smithi071,smithi192 in quorum (ranks 0,1) 2024-04-23T02:18:12.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:12 smithi071 ceph-mon[144544]: monmap e2: 2 mons at {smithi071=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],smithi192=[v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-23T02:18:12.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:12 smithi071 ceph-mon[144544]: fsmap cephfs:2 {0=cephfs.smithi071.vxejye=up:active,1=cephfs.smithi192.gtmyst=up:active} 2 up:standby-replay 2024-04-23T02:18:12.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:12 smithi071 ceph-mon[144544]: osdmap e48: 6 total, 6 up, 6 in 2024-04-23T02:18:12.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:12 smithi071 ceph-mon[144544]: mgrmap e32: smithi071.ukdjvn(active, since 3m), standbys: smithi192.qgecnv 2024-04-23T02:18:12.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:12 smithi071 ceph-mon[144544]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-23T02:18:12.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:12 smithi071 ceph-mon[144544]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:18:12.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:12 smithi071 ceph-mon[144544]: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:18:12.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:12 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:12.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:12 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:12.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:12 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:18:13.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:13 smithi071 ceph-mon[144544]: pgmap v112: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:14.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:13 smithi192 ceph-mon[108617]: pgmap v112: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:16.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:15 smithi071 ceph-mon[144544]: pgmap v113: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:16.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:15 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:16.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:15 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:16.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:15 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:16.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:15 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:16.370 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:15 smithi192 ceph-mon[108617]: pgmap v113: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:16.370 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:15 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:16.370 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:15 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:16.370 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:15 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:16.370 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:15 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:18.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:17 smithi071 ceph-mon[144544]: pgmap v114: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:18:18.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:17 smithi192 ceph-mon[108617]: pgmap v114: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:18:20.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:19 smithi071 ceph-mon[144544]: pgmap v115: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:20.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:20.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:20.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:18:20.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:18:20.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:20.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:18:20.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:18:20.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:18:20.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:20.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi071"}]: dispatch 2024-04-23T02:18:20.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi071"}]': finished 2024-04-23T02:18:20.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi192"}]: dispatch 2024-04-23T02:18:20.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi192"}]': finished 2024-04-23T02:18:20.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:18:20.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:19 smithi192 ceph-mon[108617]: pgmap v115: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:20.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:20.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:20.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:18:20.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:18:20.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:20.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:18:20.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:18:20.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:18:20.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:20.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi071"}]: dispatch 2024-04-23T02:18:20.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi071"}]': finished 2024-04-23T02:18:20.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi192"}]: dispatch 2024-04-23T02:18:20.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi192"}]': finished 2024-04-23T02:18:20.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:18:21.088 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:20 smithi071 ceph-mon[144544]: Detected new or changed devices on smithi192 2024-04-23T02:18:21.088 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:20 smithi071 ceph-mon[144544]: Upgrade: Setting container_image for all mon 2024-04-23T02:18:21.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:20 smithi192 ceph-mon[108617]: Detected new or changed devices on smithi192 2024-04-23T02:18:21.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:20 smithi192 ceph-mon[108617]: Upgrade: Setting container_image for all mon 2024-04-23T02:18:21.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.415+0000 7f2ad5ff7700 1 -- 172.21.15.71:0/2559936863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 msgr2=0x7f2ad0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:21.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.415+0000 7f2ad5ff7700 1 --2- 172.21.15.71:0/2559936863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 0x7f2ad0105fb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009910 tx=0x7f2ab8009c20 comp rx=0 tx=0).stop 2024-04-23T02:18:21.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.415+0000 7f2ad5ff7700 1 -- 172.21.15.71:0/2559936863 shutdown_connections 2024-04-23T02:18:21.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.415+0000 7f2ad5ff7700 1 --2- 172.21.15.71:0/2559936863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ad00ffc20 0x7f2ad0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:21.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.415+0000 7f2ad5ff7700 1 --2- 172.21.15.71:0/2559936863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 0x7f2ad0105fb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:21.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.415+0000 7f2ad5ff7700 1 -- 172.21.15.71:0/2559936863 >> 172.21.15.71:0/2559936863 conn(0x7f2ad0075670 msgr2=0x7f2ad0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:21.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.415+0000 7f2ad5ff7700 1 -- 172.21.15.71:0/2559936863 shutdown_connections 2024-04-23T02:18:21.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.415+0000 7f2ad5ff7700 1 -- 172.21.15.71:0/2559936863 wait complete. 2024-04-23T02:18:21.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.416+0000 7f2ad5ff7700 1 Processor -- start 2024-04-23T02:18:21.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.416+0000 7f2ad5ff7700 1 -- start start 2024-04-23T02:18:21.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.417+0000 7f2ad5ff7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ad00ffc20 0x7f2ad01975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:21.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.417+0000 7f2ad5ff7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 0x7f2ad0197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:21.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.417+0000 7f2ad5ff7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad0198140 con 0x7f2ad0105bd0 2024-04-23T02:18:21.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.417+0000 7f2ad5ff7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad01982b0 con 0x7f2ad00ffc20 2024-04-23T02:18:21.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.417+0000 7f2aceffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 0x7f2ad0197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:21.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.417+0000 7f2aceffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 0x7f2ad0197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47096/0 (socket says 172.21.15.71:47096) 2024-04-23T02:18:21.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.417+0000 7f2aceffd700 1 -- 172.21.15.71:0/1406273207 learned_addr learned my addr 172.21.15.71:0/1406273207 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:18:21.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.417+0000 7f2acf7fe700 1 --2- 172.21.15.71:0/1406273207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ad00ffc20 0x7f2ad01975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:21.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.417+0000 7f2aceffd700 1 -- 172.21.15.71:0/1406273207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ad00ffc20 msgr2=0x7f2ad01975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:21.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.417+0000 7f2aceffd700 1 --2- 172.21.15.71:0/1406273207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ad00ffc20 0x7f2ad01975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:21.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.417+0000 7f2aceffd700 1 -- 172.21.15.71:0/1406273207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab80095f0 con 0x7f2ad0105bd0 2024-04-23T02:18:21.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.417+0000 7f2aceffd700 1 --2- 172.21.15.71:0/1406273207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 0x7f2ad0197b20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000c8a0 tx=0x7f2ac000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:21.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.417+0000 7f2accff9700 1 -- 172.21.15.71:0/1406273207 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ac000cea0 con 0x7f2ad0105bd0 2024-04-23T02:18:21.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.417+0000 7f2accff9700 1 -- 172.21.15.71:0/1406273207 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2ac0004830 con 0x7f2ad0105bd0 2024-04-23T02:18:21.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.417+0000 7f2ad5ff7700 1 -- 172.21.15.71:0/1406273207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad01a0650 con 0x7f2ad0105bd0 2024-04-23T02:18:21.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.418+0000 7f2accff9700 1 -- 172.21.15.71:0/1406273207 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ac0005500 con 0x7f2ad0105bd0 2024-04-23T02:18:21.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.418+0000 7f2ad5ff7700 1 -- 172.21.15.71:0/1406273207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad01a0b40 con 0x7f2ad0105bd0 2024-04-23T02:18:21.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.418+0000 7f2ad5ff7700 1 -- 172.21.15.71:0/1406273207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad01080d0 con 0x7f2ad0105bd0 2024-04-23T02:18:21.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.419+0000 7f2accff9700 1 -- 172.21.15.71:0/1406273207 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2ac00049a0 con 0x7f2ad0105bd0 2024-04-23T02:18:21.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.419+0000 7f2accff9700 1 --2- 172.21.15.71:0/1406273207 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2abc06cce0 0x7f2abc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:21.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.419+0000 7f2accff9700 1 -- 172.21.15.71:0/1406273207 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2ac008c120 con 0x7f2ad0105bd0 2024-04-23T02:18:21.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.419+0000 7f2acf7fe700 1 --2- 172.21.15.71:0/1406273207 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2abc06cce0 0x7f2abc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:21.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.420+0000 7f2acf7fe700 1 --2- 172.21.15.71:0/1406273207 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2abc06cce0 0x7f2abc06f1a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8006010 tx=0x7f2ab801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:21.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.424+0000 7f2accff9700 1 -- 172.21.15.71:0/1406273207 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2ac00562c0 con 0x7f2ad0105bd0 2024-04-23T02:18:21.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.647+0000 7f2ad5ff7700 1 -- 172.21.15.71:0/1406273207 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ad002ccc0 con 0x7f2abc06cce0 2024-04-23T02:18:21.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.648+0000 7f2accff9700 1 -- 172.21.15.71:0/1406273207 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f2ad002ccc0 con 0x7f2abc06cce0 2024-04-23T02:18:21.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.650+0000 7f2ad5ff7700 1 -- 172.21.15.71:0/1406273207 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2abc06cce0 msgr2=0x7f2abc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:21.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.650+0000 7f2ad5ff7700 1 --2- 172.21.15.71:0/1406273207 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2abc06cce0 0x7f2abc06f1a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8006010 tx=0x7f2ab801a040 comp rx=0 tx=0).stop 2024-04-23T02:18:21.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.650+0000 7f2ad5ff7700 1 -- 172.21.15.71:0/1406273207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 msgr2=0x7f2ad0197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:21.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.650+0000 7f2ad5ff7700 1 --2- 172.21.15.71:0/1406273207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 0x7f2ad0197b20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000c8a0 tx=0x7f2ac000cc60 comp rx=0 tx=0).stop 2024-04-23T02:18:21.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.651+0000 7f2ad5ff7700 1 -- 172.21.15.71:0/1406273207 shutdown_connections 2024-04-23T02:18:21.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.651+0000 7f2ad5ff7700 1 --2- 172.21.15.71:0/1406273207 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2abc06cce0 0x7f2abc06f1a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:21.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.651+0000 7f2ad5ff7700 1 --2- 172.21.15.71:0/1406273207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ad00ffc20 0x7f2ad01975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:21.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.651+0000 7f2ad5ff7700 1 --2- 172.21.15.71:0/1406273207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 0x7f2ad0197b20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:21.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.651+0000 7f2ad5ff7700 1 -- 172.21.15.71:0/1406273207 >> 172.21.15.71:0/1406273207 conn(0x7f2ad0075670 msgr2=0x7f2ad00fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:21.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.651+0000 7f2ad5ff7700 1 -- 172.21.15.71:0/1406273207 shutdown_connections 2024-04-23T02:18:21.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.651+0000 7f2ad5ff7700 1 -- 172.21.15.71:0/1406273207 wait complete. 2024-04-23T02:18:21.665 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:18:21.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.758+0000 7f10821e6700 1 -- 172.21.15.71:0/3600808305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107c100ea0 msgr2=0x7f107c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:21.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.758+0000 7f10821e6700 1 --2- 172.21.15.71:0/3600808305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107c100ea0 0x7f107c101280 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1070009910 tx=0x7f1070009c20 comp rx=0 tx=0).stop 2024-04-23T02:18:21.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.758+0000 7f10821e6700 1 -- 172.21.15.71:0/3600808305 shutdown_connections 2024-04-23T02:18:21.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.758+0000 7f10821e6700 1 --2- 172.21.15.71:0/3600808305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f107c101850 0x7f107c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:21.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.759+0000 7f10821e6700 1 --2- 172.21.15.71:0/3600808305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107c100ea0 0x7f107c101280 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:21.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.759+0000 7f10821e6700 1 -- 172.21.15.71:0/3600808305 >> 172.21.15.71:0/3600808305 conn(0x7f107c0fc750 msgr2=0x7f107c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:21.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.759+0000 7f10821e6700 1 -- 172.21.15.71:0/3600808305 shutdown_connections 2024-04-23T02:18:21.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.759+0000 7f10821e6700 1 -- 172.21.15.71:0/3600808305 wait complete. 2024-04-23T02:18:21.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.759+0000 7f10821e6700 1 Processor -- start 2024-04-23T02:18:21.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.759+0000 7f10821e6700 1 -- start start 2024-04-23T02:18:21.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.759+0000 7f10821e6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f107c100ea0 0x7f107c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:21.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.760+0000 7f10821e6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107c101850 0x7f107c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:21.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.760+0000 7f10821e6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c19d230 con 0x7f107c101850 2024-04-23T02:18:21.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.760+0000 7f10821e6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c1966e0 con 0x7f107c100ea0 2024-04-23T02:18:21.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.760+0000 7f107affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107c101850 0x7f107c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:21.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.760+0000 7f107affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107c101850 0x7f107c19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47106/0 (socket says 172.21.15.71:47106) 2024-04-23T02:18:21.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.760+0000 7f107affd700 1 -- 172.21.15.71:0/1945670121 learned_addr learned my addr 172.21.15.71:0/1945670121 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:18:21.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.760+0000 7f107b7fe700 1 --2- 172.21.15.71:0/1945670121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f107c100ea0 0x7f107c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:21.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.760+0000 7f107affd700 1 -- 172.21.15.71:0/1945670121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f107c100ea0 msgr2=0x7f107c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:21.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.760+0000 7f107affd700 1 --2- 172.21.15.71:0/1945670121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f107c100ea0 0x7f107c19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:21.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.760+0000 7f107affd700 1 -- 172.21.15.71:0/1945670121 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10700095f0 con 0x7f107c101850 2024-04-23T02:18:21.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.760+0000 7f107b7fe700 1 --2- 172.21.15.71:0/1945670121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f107c100ea0 0x7f107c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:18:21.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.760+0000 7f107affd700 1 --2- 172.21.15.71:0/1945670121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107c101850 0x7f107c19cba0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f106c00b480 tx=0x7f106c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:21.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.760+0000 7f1078ff9700 1 -- 172.21.15.71:0/1945670121 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f106c0118c0 con 0x7f107c101850 2024-04-23T02:18:21.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.760+0000 7f1078ff9700 1 -- 172.21.15.71:0/1945670121 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f106c010380 con 0x7f107c101850 2024-04-23T02:18:21.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.760+0000 7f10821e6700 1 -- 172.21.15.71:0/1945670121 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107c1969c0 con 0x7f107c101850 2024-04-23T02:18:21.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.760+0000 7f1078ff9700 1 -- 172.21.15.71:0/1945670121 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f106c00f510 con 0x7f107c101850 2024-04-23T02:18:21.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.760+0000 7f10821e6700 1 -- 172.21.15.71:0/1945670121 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f107c196f10 con 0x7f107c101850 2024-04-23T02:18:21.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.761+0000 7f10821e6700 1 -- 172.21.15.71:0/1945670121 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f107c109240 con 0x7f107c101850 2024-04-23T02:18:21.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.762+0000 7f1078ff9700 1 -- 172.21.15.71:0/1945670121 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f106c011a20 con 0x7f107c101850 2024-04-23T02:18:21.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.762+0000 7f1078ff9700 1 --2- 172.21.15.71:0/1945670121 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f106406cce0 0x7f106406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:21.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.762+0000 7f1078ff9700 1 -- 172.21.15.71:0/1945670121 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f106c08c0c0 con 0x7f107c101850 2024-04-23T02:18:21.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.762+0000 7f107b7fe700 1 --2- 172.21.15.71:0/1945670121 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f106406cce0 0x7f106406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:21.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.762+0000 7f107b7fe700 1 --2- 172.21.15.71:0/1945670121 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f106406cce0 0x7f106406f1a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1070006010 tx=0x7f107001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:21.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.766+0000 7f1078ff9700 1 -- 172.21.15.71:0/1945670121 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f106c059cf0 con 0x7f107c101850 2024-04-23T02:18:21.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.989+0000 7f10821e6700 1 -- 172.21.15.71:0/1945670121 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f107c1971e0 con 0x7f106406cce0 2024-04-23T02:18:21.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.991+0000 7f1078ff9700 1 -- 172.21.15.71:0/1945670121 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f107c1971e0 con 0x7f106406cce0 2024-04-23T02:18:21.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.993+0000 7f10821e6700 1 -- 172.21.15.71:0/1945670121 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f106406cce0 msgr2=0x7f106406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:21.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.993+0000 7f10821e6700 1 --2- 172.21.15.71:0/1945670121 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f106406cce0 0x7f106406f1a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1070006010 tx=0x7f107001a040 comp rx=0 tx=0).stop 2024-04-23T02:18:21.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.993+0000 7f10821e6700 1 -- 172.21.15.71:0/1945670121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107c101850 msgr2=0x7f107c19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:21.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.993+0000 7f10821e6700 1 --2- 172.21.15.71:0/1945670121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107c101850 0x7f107c19cba0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f106c00b480 tx=0x7f106c00b840 comp rx=0 tx=0).stop 2024-04-23T02:18:21.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.993+0000 7f10821e6700 1 -- 172.21.15.71:0/1945670121 shutdown_connections 2024-04-23T02:18:21.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.993+0000 7f10821e6700 1 --2- 172.21.15.71:0/1945670121 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f106406cce0 0x7f106406f1a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:21.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.993+0000 7f10821e6700 1 --2- 172.21.15.71:0/1945670121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f107c100ea0 0x7f107c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:21.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.993+0000 7f10821e6700 1 --2- 172.21.15.71:0/1945670121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107c101850 0x7f107c19cba0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:21.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.993+0000 7f10821e6700 1 -- 172.21.15.71:0/1945670121 >> 172.21.15.71:0/1945670121 conn(0x7f107c0fc750 msgr2=0x7f107c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:21.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.993+0000 7f10821e6700 1 -- 172.21.15.71:0/1945670121 shutdown_connections 2024-04-23T02:18:21.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:21.993+0000 7f10821e6700 1 -- 172.21.15.71:0/1945670121 wait complete. 2024-04-23T02:18:22.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.097+0000 7f0cce196700 1 -- 172.21.15.71:0/1168406311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc8100e90 msgr2=0x7f0cc8101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:22.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.097+0000 7f0cce196700 1 --2- 172.21.15.71:0/1168406311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc8100e90 0x7f0cc8101270 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009910 tx=0x7f0cbc009c20 comp rx=0 tx=0).stop 2024-04-23T02:18:22.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.097+0000 7f0cce196700 1 -- 172.21.15.71:0/1168406311 shutdown_connections 2024-04-23T02:18:22.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.097+0000 7f0cce196700 1 --2- 172.21.15.71:0/1168406311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cc8101840 0x7f0cc8105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:22.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.097+0000 7f0cce196700 1 --2- 172.21.15.71:0/1168406311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc8100e90 0x7f0cc8101270 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:22.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.097+0000 7f0cce196700 1 -- 172.21.15.71:0/1168406311 >> 172.21.15.71:0/1168406311 conn(0x7f0cc80fc740 msgr2=0x7f0cc80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:22.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.097+0000 7f0cce196700 1 -- 172.21.15.71:0/1168406311 shutdown_connections 2024-04-23T02:18:22.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.097+0000 7f0cce196700 1 -- 172.21.15.71:0/1168406311 wait complete. 2024-04-23T02:18:22.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.098+0000 7f0cce196700 1 Processor -- start 2024-04-23T02:18:22.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.098+0000 7f0cce196700 1 -- start start 2024-04-23T02:18:22.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.098+0000 7f0cce196700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cc8100e90 0x7f0cc819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:22.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.098+0000 7f0cce196700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc8101840 0x7f0cc819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:22.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.098+0000 7f0cce196700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc819d190 con 0x7f0cc8101840 2024-04-23T02:18:22.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.098+0000 7f0cce196700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc8196640 con 0x7f0cc8100e90 2024-04-23T02:18:22.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.098+0000 7f0cc6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc8101840 0x7f0cc819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:22.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.098+0000 7f0cc6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc8101840 0x7f0cc819cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47130/0 (socket says 172.21.15.71:47130) 2024-04-23T02:18:22.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.098+0000 7f0cc6ffd700 1 -- 172.21.15.71:0/2695083404 learned_addr learned my addr 172.21.15.71:0/2695083404 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:18:22.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.098+0000 7f0cc77fe700 1 --2- 172.21.15.71:0/2695083404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cc8100e90 0x7f0cc819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:22.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.098+0000 7f0cc6ffd700 1 -- 172.21.15.71:0/2695083404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cc8100e90 msgr2=0x7f0cc819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:22.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.098+0000 7f0cc6ffd700 1 --2- 172.21.15.71:0/2695083404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cc8100e90 0x7f0cc819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:22.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.098+0000 7f0cc6ffd700 1 -- 172.21.15.71:0/2695083404 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc0095f0 con 0x7f0cc8101840 2024-04-23T02:18:22.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.098+0000 7f0cc77fe700 1 --2- 172.21.15.71:0/2695083404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cc8100e90 0x7f0cc819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:18:22.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.099+0000 7f0cc6ffd700 1 --2- 172.21.15.71:0/2695083404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc8101840 0x7f0cc819cb00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0cb800e8f0 tx=0x7f0cb800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:22.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.099+0000 7f0cc4ff9700 1 -- 172.21.15.71:0/2695083404 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cb8009790 con 0x7f0cc8101840 2024-04-23T02:18:22.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.099+0000 7f0cce196700 1 -- 172.21.15.71:0/2695083404 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc8196920 con 0x7f0cc8101840 2024-04-23T02:18:22.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.099+0000 7f0cc4ff9700 1 -- 172.21.15.71:0/2695083404 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0cb8004cd0 con 0x7f0cc8101840 2024-04-23T02:18:22.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.099+0000 7f0cc4ff9700 1 -- 172.21.15.71:0/2695083404 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cb8005580 con 0x7f0cc8101840 2024-04-23T02:18:22.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.099+0000 7f0cce196700 1 -- 172.21.15.71:0/2695083404 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc8196e70 con 0x7f0cc8101840 2024-04-23T02:18:22.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.100+0000 7f0cce196700 1 -- 172.21.15.71:0/2695083404 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc8109230 con 0x7f0cc8101840 2024-04-23T02:18:22.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.100+0000 7f0cc4ff9700 1 -- 172.21.15.71:0/2695083404 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0cb8004750 con 0x7f0cc8101840 2024-04-23T02:18:22.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.100+0000 7f0cc4ff9700 1 --2- 172.21.15.71:0/2695083404 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0cb006cce0 0x7f0cb006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:22.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.101+0000 7f0cc4ff9700 1 -- 172.21.15.71:0/2695083404 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0cb8014070 con 0x7f0cc8101840 2024-04-23T02:18:22.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.101+0000 7f0cc77fe700 1 --2- 172.21.15.71:0/2695083404 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0cb006cce0 0x7f0cb006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:22.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.101+0000 7f0cc77fe700 1 --2- 172.21.15.71:0/2695083404 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0cb006cce0 0x7f0cb006f1a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0059d0 tx=0x7f0cbc005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:22.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.105+0000 7f0cc4ff9700 1 -- 172.21.15.71:0/2695083404 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0cb805a450 con 0x7f0cc8101840 2024-04-23T02:18:22.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.314+0000 7f0cce196700 1 -- 172.21.15.71:0/2695083404 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0cc81970f0 con 0x7f0cb006cce0 2024-04-23T02:18:22.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.324+0000 7f0cc4ff9700 1 -- 172.21.15.71:0/2695083404 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f0cc81970f0 con 0x7f0cb006cce0 2024-04-23T02:18:22.329 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:18:22.329 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (2m) 21s ago 13m 16.9M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:18:22.329 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (13m) 21s ago 13m 8086k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:18:22.329 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (12m) 7s ago 12m 8010k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:18:22.329 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (13m) 21s ago 13m 6656k - 18.2.1 5be31c24972a 258512dcbc00 2024-04-23T02:18:22.329 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (12m) 7s ago 12m 6660k - 18.2.1 5be31c24972a feea2c9896a6 2024-04-23T02:18:22.329 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (107s) 21s ago 13m 78.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:18:22.329 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (7m) 21s ago 7m 64.7M - 18.2.1 5be31c24972a a515de2a841c 2024-04-23T02:18:22.329 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (7m) 21s ago 7m 133M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:18:22.329 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (7m) 7s ago 7m 68.1M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:18:22.329 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (7m) 7s ago 7m 122M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:18:22.329 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (4m) 21s ago 14m 502M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:18:22.329 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (3m) 7s ago 12m 451M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:18:22.330 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (30s) 21s ago 15m 26.9M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:18:22.330 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (11s) 7s ago 12m 27.3M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:18:22.330 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (3m) 21s ago 13m 18.8M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:18:22.330 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (2m) 7s ago 12m 17.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:18:22.330 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (11m) 21s ago 11m 197M 1227M 18.2.1 5be31c24972a d3d5f2fa7267 2024-04-23T02:18:22.330 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (10m) 21s ago 10m 209M 1227M 18.2.1 5be31c24972a 3992157ff0fd 2024-04-23T02:18:22.330 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (10m) 21s ago 10m 166M 1227M 18.2.1 5be31c24972a 12380b6a15a2 2024-04-23T02:18:22.330 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (9m) 7s ago 9m 243M 2251M 18.2.1 5be31c24972a d5f587ee3a22 2024-04-23T02:18:22.330 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (9m) 7s ago 9m 219M 2251M 18.2.1 5be31c24972a 958d046005f0 2024-04-23T02:18:22.330 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (8m) 7s ago 8m 198M 2251M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:18:22.330 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (2m) 21s ago 12m 61.2M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:18:22.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.326+0000 7f0cce196700 1 -- 172.21.15.71:0/2695083404 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0cb006cce0 msgr2=0x7f0cb006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:22.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.326+0000 7f0cce196700 1 --2- 172.21.15.71:0/2695083404 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0cb006cce0 0x7f0cb006f1a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0059d0 tx=0x7f0cbc005940 comp rx=0 tx=0).stop 2024-04-23T02:18:22.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.326+0000 7f0cce196700 1 -- 172.21.15.71:0/2695083404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc8101840 msgr2=0x7f0cc819cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:22.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.326+0000 7f0cce196700 1 --2- 172.21.15.71:0/2695083404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc8101840 0x7f0cc819cb00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0cb800e8f0 tx=0x7f0cb800ecb0 comp rx=0 tx=0).stop 2024-04-23T02:18:22.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.327+0000 7f0cce196700 1 -- 172.21.15.71:0/2695083404 shutdown_connections 2024-04-23T02:18:22.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.327+0000 7f0cce196700 1 --2- 172.21.15.71:0/2695083404 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0cb006cce0 0x7f0cb006f1a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:22.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.327+0000 7f0cce196700 1 --2- 172.21.15.71:0/2695083404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cc8100e90 0x7f0cc819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:22.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.327+0000 7f0cce196700 1 --2- 172.21.15.71:0/2695083404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0cc8101840 0x7f0cc819cb00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:22.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.327+0000 7f0cce196700 1 -- 172.21.15.71:0/2695083404 >> 172.21.15.71:0/2695083404 conn(0x7f0cc80fc740 msgr2=0x7f0cc80fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:22.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.327+0000 7f0cce196700 1 -- 172.21.15.71:0/2695083404 shutdown_connections 2024-04-23T02:18:22.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.327+0000 7f0cce196700 1 -- 172.21.15.71:0/2695083404 wait complete. 2024-04-23T02:18:22.397 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:22 smithi071 ceph-mon[144544]: pgmap v116: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:18:22.397 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:22 smithi071 ceph-mon[144544]: Upgrade: Updating crash.smithi071 (1/2) 2024-04-23T02:18:22.397 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:22 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:22.398 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:22 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi071", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-23T02:18:22.398 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:22 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:18:22.398 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:22 smithi071 ceph-mon[144544]: Deploying daemon crash.smithi071 on smithi071 2024-04-23T02:18:22.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.431+0000 7f1f00cae700 1 -- 172.21.15.71:0/4219093108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1efc0ffc20 msgr2=0x7f1efc100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:22.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.431+0000 7f1f00cae700 1 --2- 172.21.15.71:0/4219093108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1efc0ffc20 0x7f1efc100080 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4009910 tx=0x7f1ee4009c20 comp rx=0 tx=0).stop 2024-04-23T02:18:22.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.431+0000 7f1f00cae700 1 -- 172.21.15.71:0/4219093108 shutdown_connections 2024-04-23T02:18:22.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.431+0000 7f1f00cae700 1 --2- 172.21.15.71:0/4219093108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1efc0ffc20 0x7f1efc100080 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:22.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.431+0000 7f1f00cae700 1 --2- 172.21.15.71:0/4219093108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1efc105bd0 0x7f1efc105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:22.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.431+0000 7f1f00cae700 1 -- 172.21.15.71:0/4219093108 >> 172.21.15.71:0/4219093108 conn(0x7f1efc075670 msgr2=0x7f1efc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:22.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.431+0000 7f1f00cae700 1 -- 172.21.15.71:0/4219093108 shutdown_connections 2024-04-23T02:18:22.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.431+0000 7f1f00cae700 1 -- 172.21.15.71:0/4219093108 wait complete. 2024-04-23T02:18:22.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.432+0000 7f1f00cae700 1 Processor -- start 2024-04-23T02:18:22.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.432+0000 7f1f00cae700 1 -- start start 2024-04-23T02:18:22.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.432+0000 7f1f00cae700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1efc0ffc20 0x7f1efc19c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:22.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.432+0000 7f1f00cae700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1efc105bd0 0x7f1efc19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:22.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.432+0000 7f1f00cae700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc19d1c0 con 0x7f1efc105bd0 2024-04-23T02:18:22.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.432+0000 7f1f00cae700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc196670 con 0x7f1efc0ffc20 2024-04-23T02:18:22.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.432+0000 7f1ef9d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1efc105bd0 0x7f1efc19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:22.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.432+0000 7f1ef9d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1efc105bd0 0x7f1efc19cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47140/0 (socket says 172.21.15.71:47140) 2024-04-23T02:18:22.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.432+0000 7f1ef9d9b700 1 -- 172.21.15.71:0/4191773979 learned_addr learned my addr 172.21.15.71:0/4191773979 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:18:22.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.432+0000 7f1efa59c700 1 --2- 172.21.15.71:0/4191773979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1efc0ffc20 0x7f1efc19c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:22.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.432+0000 7f1ef9d9b700 1 -- 172.21.15.71:0/4191773979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1efc0ffc20 msgr2=0x7f1efc19c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:22.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.432+0000 7f1ef9d9b700 1 --2- 172.21.15.71:0/4191773979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1efc0ffc20 0x7f1efc19c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:22.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.432+0000 7f1ef9d9b700 1 -- 172.21.15.71:0/4191773979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee40095f0 con 0x7f1efc105bd0 2024-04-23T02:18:22.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.432+0000 7f1efa59c700 1 --2- 172.21.15.71:0/4191773979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1efc0ffc20 0x7f1efc19c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:18:22.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.432+0000 7f1ef9d9b700 1 --2- 172.21.15.71:0/4191773979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1efc105bd0 0x7f1efc19cb30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4006010 tx=0x7f1ee400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:22.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.433+0000 7f1ef37fe700 1 -- 172.21.15.71:0/4191773979 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ee401d070 con 0x7f1efc105bd0 2024-04-23T02:18:22.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.433+0000 7f1ef37fe700 1 -- 172.21.15.71:0/4191773979 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1ee400f460 con 0x7f1efc105bd0 2024-04-23T02:18:22.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.433+0000 7f1f00cae700 1 -- 172.21.15.71:0/4191773979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1efc1968f0 con 0x7f1efc105bd0 2024-04-23T02:18:22.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.433+0000 7f1ef37fe700 1 -- 172.21.15.71:0/4191773979 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ee4021640 con 0x7f1efc105bd0 2024-04-23T02:18:22.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.433+0000 7f1f00cae700 1 -- 172.21.15.71:0/4191773979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1efc196de0 con 0x7f1efc105bd0 2024-04-23T02:18:22.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.434+0000 7f1f00cae700 1 -- 172.21.15.71:0/4191773979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1efc04f2e0 con 0x7f1efc105bd0 2024-04-23T02:18:22.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.434+0000 7f1ef37fe700 1 -- 172.21.15.71:0/4191773979 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1ee400fa60 con 0x7f1efc105bd0 2024-04-23T02:18:22.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.434+0000 7f1ef37fe700 1 --2- 172.21.15.71:0/4191773979 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1ee806cce0 0x7f1ee806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:22.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.434+0000 7f1ef37fe700 1 -- 172.21.15.71:0/4191773979 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1ee408d7d0 con 0x7f1efc105bd0 2024-04-23T02:18:22.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.434+0000 7f1efa59c700 1 --2- 172.21.15.71:0/4191773979 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1ee806cce0 0x7f1ee806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:22.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.435+0000 7f1efa59c700 1 --2- 172.21.15.71:0/4191773979 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1ee806cce0 0x7f1ee806f1a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4007fd0 tx=0x7f1ef4007830 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:22.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.439+0000 7f1ef37fe700 1 -- 172.21.15.71:0/4191773979 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1ee405b380 con 0x7f1efc105bd0 2024-04-23T02:18:22.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:22 smithi192 ceph-mon[108617]: pgmap v116: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:18:22.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:22 smithi192 ceph-mon[108617]: Upgrade: Updating crash.smithi071 (1/2) 2024-04-23T02:18:22.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:22 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:22.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:22 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi071", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-23T02:18:22.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:22 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:18:22.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:22 smithi192 ceph-mon[108617]: Deploying daemon crash.smithi071 on smithi071 2024-04-23T02:18:22.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.717+0000 7f1f00cae700 1 -- 172.21.15.71:0/4191773979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1efc197b80 con 0x7f1efc105bd0 2024-04-23T02:18:22.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.717+0000 7f1ef37fe700 1 -- 172.21.15.71:0/4191773979 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+694 (secure 0 0 0) 0x7f1ee405af10 con 0x7f1efc105bd0 2024-04-23T02:18:22.721 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:18:22.722 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:18:22.722 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:18:22.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:18:22.722 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:18:22.722 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:18:22.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:18:22.722 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:18:22.722 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-23T02:18:22.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:18:22.722 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:18:22.722 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-23T02:18:22.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:18:22.722 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:18:22.723 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-04-23T02:18:22.723 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:18:22.723 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:18:22.723 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:18:22.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.719+0000 7f1f00cae700 1 -- 172.21.15.71:0/4191773979 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1ee806cce0 msgr2=0x7f1ee806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:22.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.719+0000 7f1f00cae700 1 --2- 172.21.15.71:0/4191773979 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1ee806cce0 0x7f1ee806f1a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4007fd0 tx=0x7f1ef4007830 comp rx=0 tx=0).stop 2024-04-23T02:18:22.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.719+0000 7f1f00cae700 1 -- 172.21.15.71:0/4191773979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1efc105bd0 msgr2=0x7f1efc19cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:22.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.719+0000 7f1f00cae700 1 --2- 172.21.15.71:0/4191773979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1efc105bd0 0x7f1efc19cb30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4006010 tx=0x7f1ee400be20 comp rx=0 tx=0).stop 2024-04-23T02:18:22.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.719+0000 7f1f00cae700 1 -- 172.21.15.71:0/4191773979 shutdown_connections 2024-04-23T02:18:22.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.719+0000 7f1f00cae700 1 --2- 172.21.15.71:0/4191773979 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1ee806cce0 0x7f1ee806f1a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:22.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.719+0000 7f1f00cae700 1 --2- 172.21.15.71:0/4191773979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1efc0ffc20 0x7f1efc19c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:22.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.719+0000 7f1f00cae700 1 --2- 172.21.15.71:0/4191773979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1efc105bd0 0x7f1efc19cb30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:22.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.719+0000 7f1f00cae700 1 -- 172.21.15.71:0/4191773979 >> 172.21.15.71:0/4191773979 conn(0x7f1efc075670 msgr2=0x7f1efc103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:22.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.719+0000 7f1f00cae700 1 -- 172.21.15.71:0/4191773979 shutdown_connections 2024-04-23T02:18:22.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.719+0000 7f1f00cae700 1 -- 172.21.15.71:0/4191773979 wait complete. 2024-04-23T02:18:22.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.826+0000 7fe82d6cf700 1 -- 172.21.15.71:0/3834106944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8280ff5a0 msgr2=0x7fe8280ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:22.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.826+0000 7fe82d6cf700 1 --2- 172.21.15.71:0/3834106944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8280ff5a0 0x7fe8280ff980 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe818009910 tx=0x7fe818009c20 comp rx=0 tx=0).stop 2024-04-23T02:18:22.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.826+0000 7fe82d6cf700 1 -- 172.21.15.71:0/3834106944 shutdown_connections 2024-04-23T02:18:22.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.826+0000 7fe82d6cf700 1 --2- 172.21.15.71:0/3834106944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8280fff50 0x7fe828105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:22.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.826+0000 7fe82d6cf700 1 --2- 172.21.15.71:0/3834106944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8280ff5a0 0x7fe8280ff980 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:22.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.826+0000 7fe82d6cf700 1 -- 172.21.15.71:0/3834106944 >> 172.21.15.71:0/3834106944 conn(0x7fe828074e30 msgr2=0x7fe828075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:22.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.826+0000 7fe82d6cf700 1 -- 172.21.15.71:0/3834106944 shutdown_connections 2024-04-23T02:18:22.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.826+0000 7fe82d6cf700 1 -- 172.21.15.71:0/3834106944 wait complete. 2024-04-23T02:18:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.827+0000 7fe82d6cf700 1 Processor -- start 2024-04-23T02:18:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.827+0000 7fe82d6cf700 1 -- start start 2024-04-23T02:18:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.828+0000 7fe82d6cf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8280ff5a0 0x7fe828100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.828+0000 7fe82d6cf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8280fff50 0x7fe828100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.828+0000 7fe82d6cf700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe828100fa0 con 0x7fe8280ff5a0 2024-04-23T02:18:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.828+0000 7fe82d6cf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8281010e0 con 0x7fe8280fff50 2024-04-23T02:18:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.828+0000 7fe826ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8280ff5a0 0x7fe828100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.828+0000 7fe826ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8280ff5a0 0x7fe828100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47154/0 (socket says 172.21.15.71:47154) 2024-04-23T02:18:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.828+0000 7fe826ffd700 1 -- 172.21.15.71:0/3712812255 learned_addr learned my addr 172.21.15.71:0/3712812255 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:18:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.828+0000 7fe8267fc700 1 --2- 172.21.15.71:0/3712812255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8280fff50 0x7fe828100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.828+0000 7fe826ffd700 1 -- 172.21.15.71:0/3712812255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8280fff50 msgr2=0x7fe828100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:22.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.828+0000 7fe826ffd700 1 --2- 172.21.15.71:0/3712812255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8280fff50 0x7fe828100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:22.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.828+0000 7fe826ffd700 1 -- 172.21.15.71:0/3712812255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8180095f0 con 0x7fe8280ff5a0 2024-04-23T02:18:22.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.828+0000 7fe826ffd700 1 --2- 172.21.15.71:0/3712812255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8280ff5a0 0x7fe828100520 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe818006010 tx=0x7fe81800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:22.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.829+0000 7fe80ffff700 1 -- 172.21.15.71:0/3712812255 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe81801d070 con 0x7fe8280ff5a0 2024-04-23T02:18:22.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.829+0000 7fe82d6cf700 1 -- 172.21.15.71:0/3712812255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe828101360 con 0x7fe8280ff5a0 2024-04-23T02:18:22.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.829+0000 7fe80ffff700 1 -- 172.21.15.71:0/3712812255 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe818005710 con 0x7fe8280ff5a0 2024-04-23T02:18:22.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.829+0000 7fe80ffff700 1 -- 172.21.15.71:0/3712812255 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe81800fe20 con 0x7fe8280ff5a0 2024-04-23T02:18:22.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.829+0000 7fe82d6cf700 1 -- 172.21.15.71:0/3712812255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8281a2c90 con 0x7fe8280ff5a0 2024-04-23T02:18:22.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.830+0000 7fe82d6cf700 1 -- 172.21.15.71:0/3712812255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe828109270 con 0x7fe8280ff5a0 2024-04-23T02:18:22.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.831+0000 7fe80ffff700 1 -- 172.21.15.71:0/3712812255 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe818003a60 con 0x7fe8280ff5a0 2024-04-23T02:18:22.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.831+0000 7fe80ffff700 1 --2- 172.21.15.71:0/3712812255 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe81006cce0 0x7fe81006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:22.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.831+0000 7fe80ffff700 1 -- 172.21.15.71:0/3712812255 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe81808d770 con 0x7fe8280ff5a0 2024-04-23T02:18:22.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.839+0000 7fe80ffff700 1 -- 172.21.15.71:0/3712812255 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe818057800 con 0x7fe8280ff5a0 2024-04-23T02:18:22.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.844+0000 7fe8267fc700 1 --2- 172.21.15.71:0/3712812255 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe81006cce0 0x7fe81006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:22.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:22.845+0000 7fe8267fc700 1 --2- 172.21.15.71:0/3712812255 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe81006cce0 0x7fe81006f1a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe828101d10 tx=0x7fe81c00b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:23.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.107+0000 7fe82d6cf700 1 -- 172.21.15.71:0/3712812255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe82804f2e0 con 0x7fe8280ff5a0 2024-04-23T02:18:23.112 INFO:teuthology.orchestra.run.smithi071.stdout:e25 2024-04-23T02:18:23.112 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:18:23.112 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:18:23.112 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:18:23.113 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:18:23.113 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:18:23.113 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:18:23.113 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 25 2024-04-23T02:18:23.113 INFO:teuthology.orchestra.run.smithi071.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T02:18:23.113 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:18:23.113 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:17:27.814232+0000 2024-04-23T02:18:23.113 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:18:23.113 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:18:23.113 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:18:23.113 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:18:23.113 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:18:23.114 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:18:23.114 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:18:23.114 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:18:23.114 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 46 2024-04-23T02:18:23.114 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:18:23.114 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:18:23.114 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:18:23.114 INFO:teuthology.orchestra.run.smithi071.stdout:up {0=14478,1=24243} 2024-04-23T02:18:23.114 INFO:teuthology.orchestra.run.smithi071.stdout:failed 2024-04-23T02:18:23.114 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:18:23.114 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:18:23.114 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:18:23.114 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:18:23.115 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:18:23.115 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:18:23.115 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:18:23.115 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:18:23.115 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:18:23.115 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/3442634960,v1:172.21.15.71:6827/3442634960] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:18:23.115 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{1:24243} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.192:6824/3780549896,v1:172.21.15.192:6825/3780549896] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:18:23.115 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{1:24255} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2100078284,v1:172.21.15.192:6827/2100078284] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:18:23.115 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:18:23.115 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:18:23.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.108+0000 7fe80ffff700 1 -- 172.21.15.71:0/3712812255 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1869 (secure 0 0 0) 0x7fe818003d10 con 0x7fe8280ff5a0 2024-04-23T02:18:23.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.110+0000 7fe82d6cf700 1 -- 172.21.15.71:0/3712812255 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe81006cce0 msgr2=0x7fe81006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:23.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.110+0000 7fe82d6cf700 1 --2- 172.21.15.71:0/3712812255 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe81006cce0 0x7fe81006f1a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe828101d10 tx=0x7fe81c00b410 comp rx=0 tx=0).stop 2024-04-23T02:18:23.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.110+0000 7fe82d6cf700 1 -- 172.21.15.71:0/3712812255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8280ff5a0 msgr2=0x7fe828100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:23.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.110+0000 7fe82d6cf700 1 --2- 172.21.15.71:0/3712812255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8280ff5a0 0x7fe828100520 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe818006010 tx=0x7fe81800bd70 comp rx=0 tx=0).stop 2024-04-23T02:18:23.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.110+0000 7fe82d6cf700 1 -- 172.21.15.71:0/3712812255 shutdown_connections 2024-04-23T02:18:23.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.110+0000 7fe82d6cf700 1 --2- 172.21.15.71:0/3712812255 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe81006cce0 0x7fe81006f1a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:23.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.110+0000 7fe82d6cf700 1 --2- 172.21.15.71:0/3712812255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8280ff5a0 0x7fe828100520 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:23.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.110+0000 7fe82d6cf700 1 --2- 172.21.15.71:0/3712812255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8280fff50 0x7fe828100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:23.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.110+0000 7fe82d6cf700 1 -- 172.21.15.71:0/3712812255 >> 172.21.15.71:0/3712812255 conn(0x7fe828074e30 msgr2=0x7fe8280fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:23.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.110+0000 7fe82d6cf700 1 -- 172.21.15.71:0/3712812255 shutdown_connections 2024-04-23T02:18:23.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.110+0000 7fe82d6cf700 1 -- 172.21.15.71:0/3712812255 wait complete. 2024-04-23T02:18:23.117 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 25 2024-04-23T02:18:23.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.217+0000 7fabc9040700 1 -- 172.21.15.71:0/1259195363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40ff5a0 msgr2=0x7fabc40ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:23.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.217+0000 7fabc9040700 1 --2- 172.21.15.71:0/1259195363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40ff5a0 0x7fabc40ff980 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009910 tx=0x7fabb4009c20 comp rx=0 tx=0).stop 2024-04-23T02:18:23.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.217+0000 7fabc9040700 1 -- 172.21.15.71:0/1259195363 shutdown_connections 2024-04-23T02:18:23.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.217+0000 7fabc9040700 1 --2- 172.21.15.71:0/1259195363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc40fff50 0x7fabc4105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:23.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.217+0000 7fabc9040700 1 --2- 172.21.15.71:0/1259195363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40ff5a0 0x7fabc40ff980 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:23.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.217+0000 7fabc9040700 1 -- 172.21.15.71:0/1259195363 >> 172.21.15.71:0/1259195363 conn(0x7fabc4074e30 msgr2=0x7fabc4075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:23.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.218+0000 7fabc9040700 1 -- 172.21.15.71:0/1259195363 shutdown_connections 2024-04-23T02:18:23.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.218+0000 7fabc9040700 1 -- 172.21.15.71:0/1259195363 wait complete. 2024-04-23T02:18:23.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.218+0000 7fabc9040700 1 Processor -- start 2024-04-23T02:18:23.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.219+0000 7fabc9040700 1 -- start start 2024-04-23T02:18:23.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.219+0000 7fabc9040700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc40ff5a0 0x7fabc410f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:23.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.219+0000 7fabc9040700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40fff50 0x7fabc410fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:23.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.219+0000 7fabc9040700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc4110520 con 0x7fabc40fff50 2024-04-23T02:18:23.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.219+0000 7fabc9040700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc41a17d0 con 0x7fabc40ff5a0 2024-04-23T02:18:23.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.219+0000 7fabc259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40fff50 0x7fabc410fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:23.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.219+0000 7fabc259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40fff50 0x7fabc410fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47184/0 (socket says 172.21.15.71:47184) 2024-04-23T02:18:23.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.219+0000 7fabc259c700 1 -- 172.21.15.71:0/880583401 learned_addr learned my addr 172.21.15.71:0/880583401 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:18:23.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.219+0000 7fabc2d9d700 1 --2- 172.21.15.71:0/880583401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc40ff5a0 0x7fabc410f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:23.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.220+0000 7fabc259c700 1 -- 172.21.15.71:0/880583401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc40ff5a0 msgr2=0x7fabc410f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:23.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.220+0000 7fabc259c700 1 --2- 172.21.15.71:0/880583401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc40ff5a0 0x7fabc410f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:23.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.220+0000 7fabc259c700 1 -- 172.21.15.71:0/880583401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb40095f0 con 0x7fabc40fff50 2024-04-23T02:18:23.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.220+0000 7fabc259c700 1 --2- 172.21.15.71:0/880583401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40fff50 0x7fabc410fe40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fabb800d900 tx=0x7fabb800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:23.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.220+0000 7fababfff700 1 -- 172.21.15.71:0/880583401 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabb8009900 con 0x7fabc40fff50 2024-04-23T02:18:23.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.220+0000 7fabc9040700 1 -- 172.21.15.71:0/880583401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc41a1ab0 con 0x7fabc40fff50 2024-04-23T02:18:23.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.220+0000 7fababfff700 1 -- 172.21.15.71:0/880583401 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fabb8010460 con 0x7fabc40fff50 2024-04-23T02:18:23.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.220+0000 7fababfff700 1 -- 172.21.15.71:0/880583401 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabb800f5f0 con 0x7fabc40fff50 2024-04-23T02:18:23.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.220+0000 7fabc9040700 1 -- 172.21.15.71:0/880583401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc41a2000 con 0x7fabc40fff50 2024-04-23T02:18:23.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.221+0000 7fabc9040700 1 -- 172.21.15.71:0/880583401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc4109270 con 0x7fabc40fff50 2024-04-23T02:18:23.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.222+0000 7fababfff700 1 -- 172.21.15.71:0/880583401 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fabb80105d0 con 0x7fabc40fff50 2024-04-23T02:18:23.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.222+0000 7fababfff700 1 --2- 172.21.15.71:0/880583401 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fabac06cce0 0x7fabac06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:23.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.222+0000 7fababfff700 1 -- 172.21.15.71:0/880583401 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fabb808c0d0 con 0x7fabc40fff50 2024-04-23T02:18:23.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.222+0000 7fabc2d9d700 1 --2- 172.21.15.71:0/880583401 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fabac06cce0 0x7fabac06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:23.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.223+0000 7fabc2d9d700 1 --2- 172.21.15.71:0/880583401 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fabac06cce0 0x7fabac06f1a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fabb4006010 tx=0x7fabb401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:23.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.228+0000 7fababfff700 1 -- 172.21.15.71:0/880583401 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fabb8056150 con 0x7fabc40fff50 2024-04-23T02:18:23.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.441+0000 7fabc9040700 1 -- 172.21.15.71:0/880583401 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fabc41a2220 con 0x7fabac06cce0 2024-04-23T02:18:23.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.443+0000 7fababfff700 1 -- 172.21.15.71:0/880583401 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7fabc41a2220 con 0x7fabac06cce0 2024-04-23T02:18:23.447 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:18:23.447 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:18:23.447 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:18:23.447 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:18:23.447 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:18:23.447 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:18:23.447 INFO:teuthology.orchestra.run.smithi071.stdout: "mon" 2024-04-23T02:18:23.447 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:18:23.448 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "4/23 daemons upgraded", 2024-04-23T02:18:23.448 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading crash daemons", 2024-04-23T02:18:23.448 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:18:23.448 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:18:23.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.445+0000 7fabc9040700 1 -- 172.21.15.71:0/880583401 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fabac06cce0 msgr2=0x7fabac06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:23.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.445+0000 7fabc9040700 1 --2- 172.21.15.71:0/880583401 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fabac06cce0 0x7fabac06f1a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fabb4006010 tx=0x7fabb401a040 comp rx=0 tx=0).stop 2024-04-23T02:18:23.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.445+0000 7fabc9040700 1 -- 172.21.15.71:0/880583401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40fff50 msgr2=0x7fabc410fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:23.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.445+0000 7fabc9040700 1 --2- 172.21.15.71:0/880583401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40fff50 0x7fabc410fe40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fabb800d900 tx=0x7fabb800dc10 comp rx=0 tx=0).stop 2024-04-23T02:18:23.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.445+0000 7fabc9040700 1 -- 172.21.15.71:0/880583401 shutdown_connections 2024-04-23T02:18:23.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.445+0000 7fabc9040700 1 --2- 172.21.15.71:0/880583401 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fabac06cce0 0x7fabac06f1a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:23.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.445+0000 7fabc9040700 1 --2- 172.21.15.71:0/880583401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabc40ff5a0 0x7fabc410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:23.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.445+0000 7fabc9040700 1 --2- 172.21.15.71:0/880583401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40fff50 0x7fabc410fe40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:23.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.445+0000 7fabc9040700 1 -- 172.21.15.71:0/880583401 >> 172.21.15.71:0/880583401 conn(0x7fabc4074e30 msgr2=0x7fabc40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:23.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.445+0000 7fabc9040700 1 -- 172.21.15.71:0/880583401 shutdown_connections 2024-04-23T02:18:23.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.445+0000 7fabc9040700 1 -- 172.21.15.71:0/880583401 wait complete. 2024-04-23T02:18:23.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:23 smithi071 ceph-mon[144544]: from='client.34100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:18:23.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:23 smithi071 ceph-mon[144544]: from='client.34104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:18:23.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:23 smithi071 ceph-mon[144544]: from='client.34108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:18:23.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:23 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/4191773979' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:18:23.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:23 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3712812255' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:18:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.551+0000 7fcd875fe700 1 -- 172.21.15.71:0/3150599862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd80101850 msgr2=0x7fcd801058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.551+0000 7fcd875fe700 1 --2- 172.21.15.71:0/3150599862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd80101850 0x7fcd801058a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009910 tx=0x7fcd70009c20 comp rx=0 tx=0).stop 2024-04-23T02:18:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.551+0000 7fcd875fe700 1 -- 172.21.15.71:0/3150599862 shutdown_connections 2024-04-23T02:18:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.551+0000 7fcd875fe700 1 --2- 172.21.15.71:0/3150599862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd80101850 0x7fcd801058a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:23.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.551+0000 7fcd875fe700 1 --2- 172.21.15.71:0/3150599862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd80100ea0 0x7fcd80101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:23.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.551+0000 7fcd875fe700 1 -- 172.21.15.71:0/3150599862 >> 172.21.15.71:0/3150599862 conn(0x7fcd800fc750 msgr2=0x7fcd800feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:23.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.551+0000 7fcd875fe700 1 -- 172.21.15.71:0/3150599862 shutdown_connections 2024-04-23T02:18:23.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.551+0000 7fcd875fe700 1 -- 172.21.15.71:0/3150599862 wait complete. 2024-04-23T02:18:23.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.552+0000 7fcd875fe700 1 Processor -- start 2024-04-23T02:18:23.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.552+0000 7fcd875fe700 1 -- start start 2024-04-23T02:18:23.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.552+0000 7fcd875fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd80100ea0 0x7fcd80073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:23.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.552+0000 7fcd875fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd80101850 0x7fcd800735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:23.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.552+0000 7fcd875fe700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd80073b10 con 0x7fcd80101850 2024-04-23T02:18:23.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.552+0000 7fcd875fe700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd80073c50 con 0x7fcd80100ea0 2024-04-23T02:18:23.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.552+0000 7fcd84b99700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd80101850 0x7fcd800735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:23.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.552+0000 7fcd84b99700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd80101850 0x7fcd800735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47198/0 (socket says 172.21.15.71:47198) 2024-04-23T02:18:23.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.552+0000 7fcd84b99700 1 -- 172.21.15.71:0/1743146709 learned_addr learned my addr 172.21.15.71:0/1743146709 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:18:23.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.552+0000 7fcd8539a700 1 --2- 172.21.15.71:0/1743146709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd80100ea0 0x7fcd80073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:23.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.552+0000 7fcd84b99700 1 -- 172.21.15.71:0/1743146709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd80100ea0 msgr2=0x7fcd80073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:23.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.553+0000 7fcd84b99700 1 --2- 172.21.15.71:0/1743146709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd80100ea0 0x7fcd80073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:23.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.553+0000 7fcd84b99700 1 -- 172.21.15.71:0/1743146709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd700095f0 con 0x7fcd80101850 2024-04-23T02:18:23.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.553+0000 7fcd84b99700 1 --2- 172.21.15.71:0/1743146709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd80101850 0x7fcd800735d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000bd70 tx=0x7fcd7000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:23.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.553+0000 7fcd767fc700 1 -- 172.21.15.71:0/1743146709 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd7001d070 con 0x7fcd80101850 2024-04-23T02:18:23.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.553+0000 7fcd767fc700 1 -- 172.21.15.71:0/1743146709 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcd7000f460 con 0x7fcd80101850 2024-04-23T02:18:23.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.553+0000 7fcd875fe700 1 -- 172.21.15.71:0/1743146709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd80073ed0 con 0x7fcd80101850 2024-04-23T02:18:23.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.553+0000 7fcd767fc700 1 -- 172.21.15.71:0/1743146709 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd70021640 con 0x7fcd80101850 2024-04-23T02:18:23.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.553+0000 7fcd875fe700 1 -- 172.21.15.71:0/1743146709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd801a2d70 con 0x7fcd80101850 2024-04-23T02:18:23.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.554+0000 7fcd875fe700 1 -- 172.21.15.71:0/1743146709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd801092a0 con 0x7fcd80101850 2024-04-23T02:18:23.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.554+0000 7fcd767fc700 1 -- 172.21.15.71:0/1743146709 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fcd7000f9e0 con 0x7fcd80101850 2024-04-23T02:18:23.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.555+0000 7fcd767fc700 1 --2- 172.21.15.71:0/1743146709 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcd6c06cce0 0x7fcd6c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:23.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.555+0000 7fcd767fc700 1 -- 172.21.15.71:0/1743146709 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fcd7008d610 con 0x7fcd80101850 2024-04-23T02:18:23.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.555+0000 7fcd8539a700 1 --2- 172.21.15.71:0/1743146709 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcd6c06cce0 0x7fcd6c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:23.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.555+0000 7fcd8539a700 1 --2- 172.21.15.71:0/1743146709 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcd6c06cce0 0x7fcd6c06f1a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcd78009c10 tx=0x7fcd78009040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:23.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.559+0000 7fcd767fc700 1 -- 172.21.15.71:0/1743146709 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fcd700577b0 con 0x7fcd80101850 2024-04-23T02:18:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:23 smithi192 ceph-mon[108617]: from='client.34100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:18:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:23 smithi192 ceph-mon[108617]: from='client.34104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:18:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:23 smithi192 ceph-mon[108617]: from='client.34108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:18:23.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:23 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/4191773979' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:18:23.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:23 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3712812255' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:18:23.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.858+0000 7fcd875fe700 1 -- 172.21.15.71:0/1743146709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcd8004f2e0 con 0x7fcd80101850 2024-04-23T02:18:23.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.859+0000 7fcd767fc700 1 -- 172.21.15.71:0/1743146709 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fcd7005add0 con 0x7fcd80101850 2024-04-23T02:18:23.863 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-23T02:18:23.863 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:18:23.863 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:18:23.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.861+0000 7fcd875fe700 1 -- 172.21.15.71:0/1743146709 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcd6c06cce0 msgr2=0x7fcd6c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:23.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.861+0000 7fcd875fe700 1 --2- 172.21.15.71:0/1743146709 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcd6c06cce0 0x7fcd6c06f1a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcd78009c10 tx=0x7fcd78009040 comp rx=0 tx=0).stop 2024-04-23T02:18:23.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.861+0000 7fcd875fe700 1 -- 172.21.15.71:0/1743146709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd80101850 msgr2=0x7fcd800735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:23.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.861+0000 7fcd875fe700 1 --2- 172.21.15.71:0/1743146709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd80101850 0x7fcd800735d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000bd70 tx=0x7fcd7000be50 comp rx=0 tx=0).stop 2024-04-23T02:18:23.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.861+0000 7fcd875fe700 1 -- 172.21.15.71:0/1743146709 shutdown_connections 2024-04-23T02:18:23.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.861+0000 7fcd875fe700 1 --2- 172.21.15.71:0/1743146709 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcd6c06cce0 0x7fcd6c06f1a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:23.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.861+0000 7fcd875fe700 1 --2- 172.21.15.71:0/1743146709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd80100ea0 0x7fcd80073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:23.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.861+0000 7fcd875fe700 1 --2- 172.21.15.71:0/1743146709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd80101850 0x7fcd800735d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:23.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.861+0000 7fcd875fe700 1 -- 172.21.15.71:0/1743146709 >> 172.21.15.71:0/1743146709 conn(0x7fcd800fc750 msgr2=0x7fcd800fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:23.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.861+0000 7fcd875fe700 1 -- 172.21.15.71:0/1743146709 shutdown_connections 2024-04-23T02:18:23.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:23.861+0000 7fcd875fe700 1 -- 172.21.15.71:0/1743146709 wait complete. 2024-04-23T02:18:24.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:24 smithi071 ceph-mon[144544]: pgmap v117: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:24.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:24 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1743146709' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:18:24.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:24 smithi192 ceph-mon[108617]: pgmap v117: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:24.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:24 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1743146709' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:18:25.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:25 smithi192 ceph-mon[108617]: from='client.34120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:18:25.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:25.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:25.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:25 smithi071 ceph-mon[144544]: from='client.34120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:18:25.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:25.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:26.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:26 smithi071 ceph-mon[144544]: pgmap v118: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:26.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:26 smithi192 ceph-mon[108617]: pgmap v118: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:27.853 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:27 smithi192 ceph-mon[108617]: pgmap v119: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:18:27.854 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:27 smithi192 ceph-mon[108617]: Upgrade: Updating crash.smithi192 (2/2) 2024-04-23T02:18:27.854 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:27 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:27.854 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:27 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-23T02:18:27.854 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:27 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:18:27.854 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:27 smithi192 ceph-mon[108617]: Deploying daemon crash.smithi192 on smithi192 2024-04-23T02:18:27.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:27 smithi071 ceph-mon[144544]: pgmap v119: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:18:27.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:27 smithi071 ceph-mon[144544]: Upgrade: Updating crash.smithi192 (2/2) 2024-04-23T02:18:27.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:27 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:27.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:27 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-23T02:18:27.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:27 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:18:27.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:27 smithi071 ceph-mon[144544]: Deploying daemon crash.smithi192 on smithi192 2024-04-23T02:18:29.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:29 smithi192 ceph-mon[108617]: pgmap v120: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:29.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:29 smithi071 ceph-mon[144544]: pgmap v120: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:31.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:31 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:31.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:31 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:31.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:31 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:18:31.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:31 smithi071 ceph-mon[144544]: pgmap v121: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:18:31.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:31 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:31.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:31 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:31.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:31 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:18:31.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:31 smithi192 ceph-mon[108617]: pgmap v121: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:18:33.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:33 smithi071 ceph-mon[144544]: pgmap v122: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:33.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:33 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:33.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:33 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:34.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:33 smithi192 ceph-mon[108617]: pgmap v122: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:34.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:33 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:34.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:33 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:35.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:35.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:35.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:18:35.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:35.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:35.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:18:36.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:36 smithi071 ceph-mon[144544]: pgmap v123: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:36.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:36 smithi192 ceph-mon[108617]: pgmap v123: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:37.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:37 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:37.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:37 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:37.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:37 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:37.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:37 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:37.658 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:37 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:37.658 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:37 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:37.658 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:37 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:37.658 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:37 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:38.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:38 smithi192 ceph-mon[108617]: pgmap v124: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:18:38.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:38 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:38.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:38 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:38.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:38 smithi071 ceph-mon[144544]: pgmap v124: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:18:38.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:38 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:38.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:38 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:40.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:40 smithi071 ceph-mon[144544]: pgmap v125: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:40.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:40 smithi192 ceph-mon[108617]: pgmap v125: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:41.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:41.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:41.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:18:41.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:18:41.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:41.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: pgmap v126: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:18:41.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:18:41.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:18:41.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:18:41.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:18:41.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: Upgrade: Setting container_image for all crash 2024-04-23T02:18:41.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:41.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi071"}]: dispatch 2024-04-23T02:18:41.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi071"}]': finished 2024-04-23T02:18:41.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi192"}]: dispatch 2024-04-23T02:18:41.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi192"}]': finished 2024-04-23T02:18:41.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-23T02:18:41.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-23T02:18:41.996 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:41 smithi071 ceph-mon[144544]: Upgrade: osd.0 is safe to restart 2024-04-23T02:18:42.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:42.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:42.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:18:42.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:18:42.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:42.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: pgmap v126: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:18:42.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:18:42.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:18:42.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:18:42.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:18:42.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: Upgrade: Setting container_image for all crash 2024-04-23T02:18:42.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:42.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi071"}]: dispatch 2024-04-23T02:18:42.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi071"}]': finished 2024-04-23T02:18:42.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi192"}]: dispatch 2024-04-23T02:18:42.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi192"}]': finished 2024-04-23T02:18:42.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-23T02:18:42.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-23T02:18:42.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:41 smithi192 ceph-mon[108617]: Upgrade: osd.0 is safe to restart 2024-04-23T02:18:43.461 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:43 smithi071 ceph-mon[144544]: Upgrade: Updating osd.0 2024-04-23T02:18:43.461 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:43 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:43.461 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:43 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T02:18:43.461 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:43 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:18:43.462 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:43 smithi071 ceph-mon[144544]: Deploying daemon osd.0 on smithi071 2024-04-23T02:18:43.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:43 smithi192 ceph-mon[108617]: Upgrade: Updating osd.0 2024-04-23T02:18:43.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:43 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:43.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:43 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T02:18:43.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:43 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:18:43.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:43 smithi192 ceph-mon[108617]: Deploying daemon osd.0 on smithi071 2024-04-23T02:18:44.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:44 smithi071 ceph-mon[144544]: pgmap v127: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:44.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:44 smithi192 ceph-mon[108617]: pgmap v127: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:45.821 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:45 smithi071 systemd[1]: Stopping Ceph osd.0 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T02:18:46.244 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:45 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-0[63099]: 2024-04-23T02:18:45.816+0000 7f24d5271700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T02:18:46.244 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:45 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-0[63099]: 2024-04-23T02:18:45.816+0000 7f24d5271700 -1 osd.0 48 *** Got signal Terminated *** 2024-04-23T02:18:46.244 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:45 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-0[63099]: 2024-04-23T02:18:45.816+0000 7f24d5271700 -1 osd.0 48 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T02:18:46.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:46 smithi192 ceph-mon[108617]: pgmap v128: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:46.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:46 smithi192 ceph-mon[108617]: osd.0 marked itself down and dead 2024-04-23T02:18:46.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:46 smithi071 ceph-mon[144544]: pgmap v128: 65 pgs: 65 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:18:46.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:46 smithi071 ceph-mon[144544]: osd.0 marked itself down and dead 2024-04-23T02:18:47.494 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:47 smithi071 bash[155872]: ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-0 2024-04-23T02:18:47.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:47 smithi071 ceph-mon[144544]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-23T02:18:47.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:47 smithi071 ceph-mon[144544]: osdmap e49: 6 total, 5 up, 6 in 2024-04-23T02:18:47.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:47 smithi192 ceph-mon[108617]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-23T02:18:47.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:47 smithi192 ceph-mon[108617]: osdmap e49: 6 total, 5 up, 6 in 2024-04-23T02:18:48.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:48 smithi071 ceph-mon[144544]: pgmap v130: 65 pgs: 9 stale+active+clean, 56 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-23T02:18:48.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:48 smithi071 ceph-mon[144544]: osdmap e50: 6 total, 5 up, 6 in 2024-04-23T02:18:48.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:48 smithi192 ceph-mon[108617]: pgmap v130: 65 pgs: 9 stale+active+clean, 56 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-23T02:18:48.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:48 smithi192 ceph-mon[108617]: osdmap e50: 6 total, 5 up, 6 in 2024-04-23T02:18:49.494 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:49 smithi071 systemd[1]: ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.0.service: Succeeded. 2024-04-23T02:18:49.494 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:49 smithi071 systemd[1]: Stopped Ceph osd.0 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:18:49.494 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:49 smithi071 systemd[1]: Starting Ceph osd.0 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T02:18:50.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:50 smithi192 ceph-mon[108617]: pgmap v132: 65 pgs: 9 stale+active+clean, 56 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T02:18:50.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:50.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:18:50.695 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:50 smithi071 ceph-mon[144544]: pgmap v132: 65 pgs: 9 stale+active+clean, 56 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T02:18:50.695 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:50.695 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:18:50.695 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:50 smithi071 bash[156144]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-23T02:18:50.695 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:50 smithi071 bash[156144]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-23T02:18:50.696 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:50 smithi071 bash[156144]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-23T02:18:50.696 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:50 smithi071 bash[156144]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-23T02:18:50.696 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:50 smithi071 bash[156144]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-23T02:18:50.696 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:50 smithi071 bash[156144]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-23T02:18:50.696 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:50 smithi071 bash[156144]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-23T02:18:51.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:51 smithi071 ceph-mon[144544]: pgmap v133: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 71/366 objects degraded (19.399%) 2024-04-23T02:18:51.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:51 smithi071 ceph-mon[144544]: Health check failed: Degraded data redundancy: 71/366 objects degraded (19.399%), 20 pgs degraded (PG_DEGRADED) 2024-04-23T02:18:51.744 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:51 smithi071 ceph-osd[156439]: -- 172.21.15.71:0/625978938 <== mon.0 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562382caf1e0 con 0x562382c7f000 2024-04-23T02:18:51.744 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:51 smithi071 bash[156401]: 6ada0e60b90b4f697dfaae524c6498971d21dbe0a36ec3cbe5cb349f9a91a4ce 2024-04-23T02:18:51.744 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:51 smithi071 systemd[1]: Started Ceph osd.0 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:18:51.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:51 smithi192 ceph-mon[108617]: pgmap v133: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 71/366 objects degraded (19.399%) 2024-04-23T02:18:51.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:51 smithi192 ceph-mon[108617]: Health check failed: Degraded data redundancy: 71/366 objects degraded (19.399%), 20 pgs degraded (PG_DEGRADED) 2024-04-23T02:18:52.436 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:52 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-0[156416]: 2024-04-23T02:18:52.091+0000 7f73c97317c0 -1 Falling back to public interface 2024-04-23T02:18:52.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:52 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:52.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:52 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:52.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:52 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:18:53.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:52 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:53.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:52 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:53.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:52 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:18:53.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.967+0000 7f98983d5700 1 -- 172.21.15.71:0/791368447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98900ffc20 msgr2=0x7f9890100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:53.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.967+0000 7f98983d5700 1 --2- 172.21.15.71:0/791368447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98900ffc20 0x7f9890100080 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9880009910 tx=0x7f9880009c20 comp rx=0 tx=0).stop 2024-04-23T02:18:53.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.967+0000 7f98983d5700 1 -- 172.21.15.71:0/791368447 shutdown_connections 2024-04-23T02:18:53.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.967+0000 7f98983d5700 1 --2- 172.21.15.71:0/791368447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98900ffc20 0x7f9890100080 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:53.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.967+0000 7f98983d5700 1 --2- 172.21.15.71:0/791368447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9890105bd0 0x7f9890105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:53.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.967+0000 7f98983d5700 1 -- 172.21.15.71:0/791368447 >> 172.21.15.71:0/791368447 conn(0x7f9890075670 msgr2=0x7f9890075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:53.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.967+0000 7f98983d5700 1 -- 172.21.15.71:0/791368447 shutdown_connections 2024-04-23T02:18:53.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.968+0000 7f98983d5700 1 -- 172.21.15.71:0/791368447 wait complete. 2024-04-23T02:18:53.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.968+0000 7f98983d5700 1 Processor -- start 2024-04-23T02:18:53.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.968+0000 7f98983d5700 1 -- start start 2024-04-23T02:18:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.968+0000 7f98983d5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98900ffc20 0x7f989019b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.968+0000 7f98983d5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9890105bd0 0x7f989019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.968+0000 7f98983d5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989019c060 con 0x7f9890105bd0 2024-04-23T02:18:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.968+0000 7f98983d5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98901955a0 con 0x7f98900ffc20 2024-04-23T02:18:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.968+0000 7f9895970700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9890105bd0 0x7f989019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.968+0000 7f9895970700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9890105bd0 0x7f989019ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55572/0 (socket says 172.21.15.71:55572) 2024-04-23T02:18:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.968+0000 7f9895970700 1 -- 172.21.15.71:0/1700685858 learned_addr learned my addr 172.21.15.71:0/1700685858 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:18:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.969+0000 7f9896171700 1 --2- 172.21.15.71:0/1700685858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98900ffc20 0x7f989019b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.969+0000 7f9895970700 1 -- 172.21.15.71:0/1700685858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98900ffc20 msgr2=0x7f989019b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.969+0000 7f9895970700 1 --2- 172.21.15.71:0/1700685858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98900ffc20 0x7f989019b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.969+0000 7f9895970700 1 -- 172.21.15.71:0/1700685858 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98800095f0 con 0x7f9890105bd0 2024-04-23T02:18:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.969+0000 7f9896171700 1 --2- 172.21.15.71:0/1700685858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98900ffc20 0x7f989019b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:18:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.969+0000 7f9895970700 1 --2- 172.21.15.71:0/1700685858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9890105bd0 0x7f989019ba60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9880006010 tx=0x7f988000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.969+0000 7f98877fe700 1 -- 172.21.15.71:0/1700685858 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f988001d070 con 0x7f9890105bd0 2024-04-23T02:18:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.969+0000 7f98983d5700 1 -- 172.21.15.71:0/1700685858 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9890195820 con 0x7f9890105bd0 2024-04-23T02:18:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.969+0000 7f98877fe700 1 -- 172.21.15.71:0/1700685858 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f988000f460 con 0x7f9890105bd0 2024-04-23T02:18:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.969+0000 7f98877fe700 1 -- 172.21.15.71:0/1700685858 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9880021640 con 0x7f9890105bd0 2024-04-23T02:18:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.969+0000 7f98983d5700 1 -- 172.21.15.71:0/1700685858 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9890195d10 con 0x7f9890105bd0 2024-04-23T02:18:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.970+0000 7f98983d5700 1 -- 172.21.15.71:0/1700685858 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98901080d0 con 0x7f9890105bd0 2024-04-23T02:18:53.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.970+0000 7f98877fe700 1 -- 172.21.15.71:0/1700685858 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f988000fa20 con 0x7f9890105bd0 2024-04-23T02:18:53.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.971+0000 7f98877fe700 1 --2- 172.21.15.71:0/1700685858 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f987c06cce0 0x7f987c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:53.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.971+0000 7f98877fe700 1 -- 172.21.15.71:0/1700685858 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f988008d810 con 0x7f9890105bd0 2024-04-23T02:18:53.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.971+0000 7f9896171700 1 --2- 172.21.15.71:0/1700685858 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f987c06cce0 0x7f987c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:53.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.971+0000 7f9896171700 1 --2- 172.21.15.71:0/1700685858 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f987c06cce0 0x7f987c06f1a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f988c00e300 tx=0x7f988c009040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:53.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:53.976+0000 7f98877fe700 1 -- 172.21.15.71:0/1700685858 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f988005b3c0 con 0x7f9890105bd0 2024-04-23T02:18:53.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:53 smithi071 ceph-mon[144544]: pgmap v134: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 71/366 objects degraded (19.399%) 2024-04-23T02:18:54.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:53 smithi192 ceph-mon[108617]: pgmap v134: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 71/366 objects degraded (19.399%) 2024-04-23T02:18:54.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.213+0000 7f98983d5700 1 -- 172.21.15.71:0/1700685858 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9890196b40 con 0x7f987c06cce0 2024-04-23T02:18:54.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.215+0000 7f98877fe700 1 -- 172.21.15.71:0/1700685858 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9890196b40 con 0x7f987c06cce0 2024-04-23T02:18:54.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.217+0000 7f98983d5700 1 -- 172.21.15.71:0/1700685858 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f987c06cce0 msgr2=0x7f987c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:54.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.217+0000 7f98983d5700 1 --2- 172.21.15.71:0/1700685858 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f987c06cce0 0x7f987c06f1a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f988c00e300 tx=0x7f988c009040 comp rx=0 tx=0).stop 2024-04-23T02:18:54.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.217+0000 7f98983d5700 1 -- 172.21.15.71:0/1700685858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9890105bd0 msgr2=0x7f989019ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:54.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.217+0000 7f98983d5700 1 --2- 172.21.15.71:0/1700685858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9890105bd0 0x7f989019ba60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9880006010 tx=0x7f988000be20 comp rx=0 tx=0).stop 2024-04-23T02:18:54.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.217+0000 7f98983d5700 1 -- 172.21.15.71:0/1700685858 shutdown_connections 2024-04-23T02:18:54.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.217+0000 7f98983d5700 1 --2- 172.21.15.71:0/1700685858 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f987c06cce0 0x7f987c06f1a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:54.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.217+0000 7f98983d5700 1 --2- 172.21.15.71:0/1700685858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98900ffc20 0x7f989019b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:54.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.217+0000 7f98983d5700 1 --2- 172.21.15.71:0/1700685858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9890105bd0 0x7f989019ba60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:54.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.217+0000 7f98983d5700 1 -- 172.21.15.71:0/1700685858 >> 172.21.15.71:0/1700685858 conn(0x7f9890075670 msgr2=0x7f98900fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:54.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.217+0000 7f98983d5700 1 -- 172.21.15.71:0/1700685858 shutdown_connections 2024-04-23T02:18:54.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.217+0000 7f98983d5700 1 -- 172.21.15.71:0/1700685858 wait complete. 2024-04-23T02:18:54.231 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:18:54.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.323+0000 7f100be5b700 1 -- 172.21.15.71:0/2222080256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1004101850 msgr2=0x7f10041058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:54.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.323+0000 7f100be5b700 1 --2- 172.21.15.71:0/2222080256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1004101850 0x7f10041058a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009910 tx=0x7f0ff4009c20 comp rx=0 tx=0).stop 2024-04-23T02:18:54.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.323+0000 7f100be5b700 1 -- 172.21.15.71:0/2222080256 shutdown_connections 2024-04-23T02:18:54.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.323+0000 7f100be5b700 1 --2- 172.21.15.71:0/2222080256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1004101850 0x7f10041058a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:54.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.323+0000 7f100be5b700 1 --2- 172.21.15.71:0/2222080256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1004100ea0 0x7f1004101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:54.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.323+0000 7f100be5b700 1 -- 172.21.15.71:0/2222080256 >> 172.21.15.71:0/2222080256 conn(0x7f10040fc750 msgr2=0x7f10040feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:54.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.323+0000 7f100be5b700 1 -- 172.21.15.71:0/2222080256 shutdown_connections 2024-04-23T02:18:54.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.323+0000 7f100be5b700 1 -- 172.21.15.71:0/2222080256 wait complete. 2024-04-23T02:18:54.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.324+0000 7f100be5b700 1 Processor -- start 2024-04-23T02:18:54.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.324+0000 7f100be5b700 1 -- start start 2024-04-23T02:18:54.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.324+0000 7f100be5b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1004100ea0 0x7f100419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:54.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.324+0000 7f100be5b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1004101850 0x7f100419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:54.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.324+0000 7f100be5b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100419d1f0 con 0x7f1004100ea0 2024-04-23T02:18:54.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.324+0000 7f100be5b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10041966a0 con 0x7f1004101850 2024-04-23T02:18:54.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.324+0000 7f1009bf7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1004100ea0 0x7f100419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:54.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.324+0000 7f1009bf7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1004100ea0 0x7f100419c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55582/0 (socket says 172.21.15.71:55582) 2024-04-23T02:18:54.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.324+0000 7f1009bf7700 1 -- 172.21.15.71:0/3101013818 learned_addr learned my addr 172.21.15.71:0/3101013818 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:18:54.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.324+0000 7f10093f6700 1 --2- 172.21.15.71:0/3101013818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1004101850 0x7f100419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:54.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.324+0000 7f1009bf7700 1 -- 172.21.15.71:0/3101013818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1004101850 msgr2=0x7f100419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:54.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.324+0000 7f1009bf7700 1 --2- 172.21.15.71:0/3101013818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1004101850 0x7f100419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:54.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.324+0000 7f1009bf7700 1 -- 172.21.15.71:0/3101013818 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff40095f0 con 0x7f1004100ea0 2024-04-23T02:18:54.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.324+0000 7f10093f6700 1 --2- 172.21.15.71:0/3101013818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1004101850 0x7f100419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:18:54.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.325+0000 7f1009bf7700 1 --2- 172.21.15.71:0/3101013818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1004100ea0 0x7f100419c620 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f100000d8d0 tx=0x7f100000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:54.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.325+0000 7f0ffaffd700 1 -- 172.21.15.71:0/3101013818 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1000009940 con 0x7f1004100ea0 2024-04-23T02:18:54.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.325+0000 7f0ffaffd700 1 -- 172.21.15.71:0/3101013818 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1000010460 con 0x7f1004100ea0 2024-04-23T02:18:54.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.325+0000 7f0ffaffd700 1 -- 172.21.15.71:0/3101013818 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f100000f5f0 con 0x7f1004100ea0 2024-04-23T02:18:54.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.325+0000 7f100be5b700 1 -- 172.21.15.71:0/3101013818 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1004196980 con 0x7f1004100ea0 2024-04-23T02:18:54.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.325+0000 7f100be5b700 1 -- 172.21.15.71:0/3101013818 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1004196ed0 con 0x7f1004100ea0 2024-04-23T02:18:54.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.326+0000 7f100be5b700 1 -- 172.21.15.71:0/3101013818 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10041092a0 con 0x7f1004100ea0 2024-04-23T02:18:54.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.326+0000 7f0ffaffd700 1 -- 172.21.15.71:0/3101013818 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f10000105d0 con 0x7f1004100ea0 2024-04-23T02:18:54.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.327+0000 7f0ffaffd700 1 --2- 172.21.15.71:0/3101013818 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ff006cce0 0x7f0ff006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:54.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.327+0000 7f0ffaffd700 1 -- 172.21.15.71:0/3101013818 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f100008c4e0 con 0x7f1004100ea0 2024-04-23T02:18:54.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.327+0000 7f10093f6700 1 --2- 172.21.15.71:0/3101013818 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ff006cce0 0x7f0ff006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:54.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.327+0000 7f10093f6700 1 --2- 172.21.15.71:0/3101013818 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ff006cce0 0x7f0ff006f1a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f10040fcfc0 tx=0x7f0ff401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:54.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.331+0000 7f0ffaffd700 1 -- 172.21.15.71:0/3101013818 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f100005a070 con 0x7f1004100ea0 2024-04-23T02:18:54.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.552+0000 7f100be5b700 1 -- 172.21.15.71:0/3101013818 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1004197ce0 con 0x7f0ff006cce0 2024-04-23T02:18:54.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.553+0000 7f0ffaffd700 1 -- 172.21.15.71:0/3101013818 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1004197ce0 con 0x7f0ff006cce0 2024-04-23T02:18:54.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.555+0000 7f100be5b700 1 -- 172.21.15.71:0/3101013818 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ff006cce0 msgr2=0x7f0ff006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:54.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.555+0000 7f100be5b700 1 --2- 172.21.15.71:0/3101013818 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ff006cce0 0x7f0ff006f1a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f10040fcfc0 tx=0x7f0ff401a040 comp rx=0 tx=0).stop 2024-04-23T02:18:54.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.555+0000 7f100be5b700 1 -- 172.21.15.71:0/3101013818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1004100ea0 msgr2=0x7f100419c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:54.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.555+0000 7f100be5b700 1 --2- 172.21.15.71:0/3101013818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1004100ea0 0x7f100419c620 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f100000d8d0 tx=0x7f100000dc90 comp rx=0 tx=0).stop 2024-04-23T02:18:54.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.555+0000 7f100be5b700 1 -- 172.21.15.71:0/3101013818 shutdown_connections 2024-04-23T02:18:54.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.555+0000 7f100be5b700 1 --2- 172.21.15.71:0/3101013818 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ff006cce0 0x7f0ff006f1a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:54.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.555+0000 7f100be5b700 1 --2- 172.21.15.71:0/3101013818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1004100ea0 0x7f100419c620 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:54.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.555+0000 7f100be5b700 1 --2- 172.21.15.71:0/3101013818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1004101850 0x7f100419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:54.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.555+0000 7f100be5b700 1 -- 172.21.15.71:0/3101013818 >> 172.21.15.71:0/3101013818 conn(0x7f10040fc750 msgr2=0x7f10040fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:54.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.556+0000 7f100be5b700 1 -- 172.21.15.71:0/3101013818 shutdown_connections 2024-04-23T02:18:54.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.556+0000 7f100be5b700 1 -- 172.21.15.71:0/3101013818 wait complete. 2024-04-23T02:18:54.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.677+0000 7f05bf5ce700 1 -- 172.21.15.71:0/1835270648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b8073160 msgr2=0x7f05b8073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:54.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.677+0000 7f05bf5ce700 1 --2- 172.21.15.71:0/1835270648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b8073160 0x7f05b8073540 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009910 tx=0x7f05b4009c20 comp rx=0 tx=0).stop 2024-04-23T02:18:54.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.677+0000 7f05bf5ce700 1 -- 172.21.15.71:0/1835270648 shutdown_connections 2024-04-23T02:18:54.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.677+0000 7f05bf5ce700 1 --2- 172.21.15.71:0/1835270648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05b8073a80 0x7f05b810cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:54.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.677+0000 7f05bf5ce700 1 --2- 172.21.15.71:0/1835270648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b8073160 0x7f05b8073540 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:54.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.677+0000 7f05bf5ce700 1 -- 172.21.15.71:0/1835270648 >> 172.21.15.71:0/1835270648 conn(0x7f05b80fc750 msgr2=0x7f05b80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:54.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.677+0000 7f05bf5ce700 1 -- 172.21.15.71:0/1835270648 shutdown_connections 2024-04-23T02:18:54.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.677+0000 7f05bf5ce700 1 -- 172.21.15.71:0/1835270648 wait complete. 2024-04-23T02:18:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.678+0000 7f05bf5ce700 1 Processor -- start 2024-04-23T02:18:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.678+0000 7f05bf5ce700 1 -- start start 2024-04-23T02:18:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.678+0000 7f05bf5ce700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b8073160 0x7f05b81989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.678+0000 7f05bf5ce700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05b8073a80 0x7f05b8198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.679+0000 7f05bf5ce700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b8199590 con 0x7f05b8073160 2024-04-23T02:18:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.679+0000 7f05bf5ce700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b819d350 con 0x7f05b8073a80 2024-04-23T02:18:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.679+0000 7f05bd36a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b8073160 0x7f05b81989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.679+0000 7f05bd36a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b8073160 0x7f05b81989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55598/0 (socket says 172.21.15.71:55598) 2024-04-23T02:18:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.679+0000 7f05bd36a700 1 -- 172.21.15.71:0/3787077708 learned_addr learned my addr 172.21.15.71:0/3787077708 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:18:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.679+0000 7f05bcb69700 1 --2- 172.21.15.71:0/3787077708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05b8073a80 0x7f05b8198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:54.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.679+0000 7f05bd36a700 1 -- 172.21.15.71:0/3787077708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05b8073a80 msgr2=0x7f05b8198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:54.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.679+0000 7f05bd36a700 1 --2- 172.21.15.71:0/3787077708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05b8073a80 0x7f05b8198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:54.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.679+0000 7f05bd36a700 1 -- 172.21.15.71:0/3787077708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b40095f0 con 0x7f05b8073160 2024-04-23T02:18:54.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.679+0000 7f05bd36a700 1 --2- 172.21.15.71:0/3787077708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b8073160 0x7f05b81989c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f05b4006010 tx=0x7f05b400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:54.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.679+0000 7f05ae7fc700 1 -- 172.21.15.71:0/3787077708 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05b401d070 con 0x7f05b8073160 2024-04-23T02:18:54.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.679+0000 7f05ae7fc700 1 -- 172.21.15.71:0/3787077708 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f05b4005dc0 con 0x7f05b8073160 2024-04-23T02:18:54.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.679+0000 7f05bf5ce700 1 -- 172.21.15.71:0/3787077708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b819d5d0 con 0x7f05b8073160 2024-04-23T02:18:54.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.679+0000 7f05ae7fc700 1 -- 172.21.15.71:0/3787077708 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05b4021410 con 0x7f05b8073160 2024-04-23T02:18:54.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.679+0000 7f05bf5ce700 1 -- 172.21.15.71:0/3787077708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b819dac0 con 0x7f05b8073160 2024-04-23T02:18:54.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.680+0000 7f05bf5ce700 1 -- 172.21.15.71:0/3787077708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b810a6a0 con 0x7f05b8073160 2024-04-23T02:18:54.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.681+0000 7f05ae7fc700 1 -- 172.21.15.71:0/3787077708 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f05b4003a60 con 0x7f05b8073160 2024-04-23T02:18:54.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.681+0000 7f05ae7fc700 1 --2- 172.21.15.71:0/3787077708 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f05a406cce0 0x7f05a406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:54.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.681+0000 7f05ae7fc700 1 -- 172.21.15.71:0/3787077708 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f05b408d780 con 0x7f05b8073160 2024-04-23T02:18:54.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.681+0000 7f05bcb69700 1 --2- 172.21.15.71:0/3787077708 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f05a406cce0 0x7f05a406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:54.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.682+0000 7f05bcb69700 1 --2- 172.21.15.71:0/3787077708 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f05a406cce0 0x7f05a406f1a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f05b8199f90 tx=0x7f05a800b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:54.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.686+0000 7f05ae7fc700 1 -- 172.21.15.71:0/3787077708 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f05b405b330 con 0x7f05b8073160 2024-04-23T02:18:54.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.900+0000 7f05bf5ce700 1 -- 172.21.15.71:0/3787077708 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f05b8199e10 con 0x7f05a406cce0 2024-04-23T02:18:54.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.915+0000 7f05ae7fc700 1 -- 172.21.15.71:0/3787077708 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f05a8015070 con 0x7f05a406cce0 2024-04-23T02:18:54.919 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:18:54.919 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (2m) 18s ago 13m 24.2M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:18:54.919 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (14m) 18s ago 14m 8132k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:18:54.919 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (13m) 21s ago 13m 8036k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:18:54.919 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (30s) 18s ago 14m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:18:54.919 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (25s) 21s ago 12m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:18:54.919 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (2m) 18s ago 13m 78.2M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:18:54.919 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (8m) 18s ago 7m 65.0M - 18.2.1 5be31c24972a a515de2a841c 2024-04-23T02:18:54.919 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (7m) 18s ago 7m 133M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:18:54.919 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (7m) 21s ago 7m 68.1M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:18:54.920 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (7m) 21s ago 7m 122M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:18:54.920 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (4m) 18s ago 15m 503M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:18:54.920 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (3m) 21s ago 12m 451M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:18:54.920 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (63s) 18s ago 15m 41.7M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:18:54.920 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (44s) 21s ago 12m 31.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:18:54.920 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (3m) 18s ago 14m 18.8M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:18:54.920 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (3m) 21s ago 12m 17.8M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:18:54.920 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 starting - - - 1227M 2024-04-23T02:18:54.920 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (11m) 18s ago 11m 210M 1227M 18.2.1 5be31c24972a 3992157ff0fd 2024-04-23T02:18:54.920 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (10m) 18s ago 10m 165M 1227M 18.2.1 5be31c24972a 12380b6a15a2 2024-04-23T02:18:54.920 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (10m) 21s ago 10m 243M 2251M 18.2.1 5be31c24972a d5f587ee3a22 2024-04-23T02:18:54.920 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (10m) 21s ago 10m 219M 2251M 18.2.1 5be31c24972a 958d046005f0 2024-04-23T02:18:54.920 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (9m) 21s ago 9m 199M 2251M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:18:54.921 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (2m) 18s ago 13m 63.4M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:18:54.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.917+0000 7f05bf5ce700 1 -- 172.21.15.71:0/3787077708 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f05a406cce0 msgr2=0x7f05a406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:54.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.917+0000 7f05bf5ce700 1 --2- 172.21.15.71:0/3787077708 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f05a406cce0 0x7f05a406f1a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f05b8199f90 tx=0x7f05a800b410 comp rx=0 tx=0).stop 2024-04-23T02:18:54.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.917+0000 7f05bf5ce700 1 -- 172.21.15.71:0/3787077708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b8073160 msgr2=0x7f05b81989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:54.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.917+0000 7f05bf5ce700 1 --2- 172.21.15.71:0/3787077708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b8073160 0x7f05b81989c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f05b4006010 tx=0x7f05b400bd70 comp rx=0 tx=0).stop 2024-04-23T02:18:54.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.917+0000 7f05bf5ce700 1 -- 172.21.15.71:0/3787077708 shutdown_connections 2024-04-23T02:18:54.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.917+0000 7f05bf5ce700 1 --2- 172.21.15.71:0/3787077708 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f05a406cce0 0x7f05a406f1a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:54.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.917+0000 7f05bf5ce700 1 --2- 172.21.15.71:0/3787077708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b8073160 0x7f05b81989c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:54.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.917+0000 7f05bf5ce700 1 --2- 172.21.15.71:0/3787077708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05b8073a80 0x7f05b8198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:54.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.917+0000 7f05bf5ce700 1 -- 172.21.15.71:0/3787077708 >> 172.21.15.71:0/3787077708 conn(0x7f05b80fc750 msgr2=0x7f05b81077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:54.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.917+0000 7f05bf5ce700 1 -- 172.21.15.71:0/3787077708 shutdown_connections 2024-04-23T02:18:54.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:54.917+0000 7f05bf5ce700 1 -- 172.21.15.71:0/3787077708 wait complete. 2024-04-23T02:18:54.961 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:54 smithi071 ceph-mon[144544]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:18:55.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.021+0000 7ff2fb6ee700 1 -- 172.21.15.71:0/2750045202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2f4101850 msgr2=0x7ff2f41058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:55.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.021+0000 7ff2fb6ee700 1 --2- 172.21.15.71:0/2750045202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2f4101850 0x7ff2f41058a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009910 tx=0x7ff2e4009c20 comp rx=0 tx=0).stop 2024-04-23T02:18:55.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.022+0000 7ff2fb6ee700 1 -- 172.21.15.71:0/2750045202 shutdown_connections 2024-04-23T02:18:55.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.022+0000 7ff2fb6ee700 1 --2- 172.21.15.71:0/2750045202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2f4101850 0x7ff2f41058a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:55.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.022+0000 7ff2fb6ee700 1 --2- 172.21.15.71:0/2750045202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2f4100ea0 0x7ff2f4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:55.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.022+0000 7ff2fb6ee700 1 -- 172.21.15.71:0/2750045202 >> 172.21.15.71:0/2750045202 conn(0x7ff2f40fc750 msgr2=0x7ff2f40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:55.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.022+0000 7ff2fb6ee700 1 -- 172.21.15.71:0/2750045202 shutdown_connections 2024-04-23T02:18:55.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.022+0000 7ff2fb6ee700 1 -- 172.21.15.71:0/2750045202 wait complete. 2024-04-23T02:18:55.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.022+0000 7ff2fb6ee700 1 Processor -- start 2024-04-23T02:18:55.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.022+0000 7ff2fb6ee700 1 -- start start 2024-04-23T02:18:55.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.022+0000 7ff2fb6ee700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2f4100ea0 0x7ff2f419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:55.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.022+0000 7ff2fb6ee700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2f4101850 0x7ff2f419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:55.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.022+0000 7ff2fb6ee700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f419d1f0 con 0x7ff2f4100ea0 2024-04-23T02:18:55.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.022+0000 7ff2fb6ee700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f41966a0 con 0x7ff2f4101850 2024-04-23T02:18:55.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.023+0000 7ff2f948a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2f4100ea0 0x7ff2f419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:55.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.023+0000 7ff2f948a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2f4100ea0 0x7ff2f419c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55612/0 (socket says 172.21.15.71:55612) 2024-04-23T02:18:55.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.023+0000 7ff2f948a700 1 -- 172.21.15.71:0/3476513997 learned_addr learned my addr 172.21.15.71:0/3476513997 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:18:55.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.023+0000 7ff2f8c89700 1 --2- 172.21.15.71:0/3476513997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2f4101850 0x7ff2f419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:55.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.023+0000 7ff2f948a700 1 -- 172.21.15.71:0/3476513997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2f4101850 msgr2=0x7ff2f419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:55.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.023+0000 7ff2f948a700 1 --2- 172.21.15.71:0/3476513997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2f4101850 0x7ff2f419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:55.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.023+0000 7ff2f948a700 1 -- 172.21.15.71:0/3476513997 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e40095f0 con 0x7ff2f4100ea0 2024-04-23T02:18:55.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.023+0000 7ff2f948a700 1 --2- 172.21.15.71:0/3476513997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2f4100ea0 0x7ff2f419c620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000d8d0 tx=0x7ff2f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:55.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.023+0000 7ff2ea7fc700 1 -- 172.21.15.71:0/3476513997 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2f0009940 con 0x7ff2f4100ea0 2024-04-23T02:18:55.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.023+0000 7ff2ea7fc700 1 -- 172.21.15.71:0/3476513997 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff2f0010460 con 0x7ff2f4100ea0 2024-04-23T02:18:55.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.023+0000 7ff2fb6ee700 1 -- 172.21.15.71:0/3476513997 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f4196980 con 0x7ff2f4100ea0 2024-04-23T02:18:55.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.023+0000 7ff2ea7fc700 1 -- 172.21.15.71:0/3476513997 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2f000f5f0 con 0x7ff2f4100ea0 2024-04-23T02:18:55.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.023+0000 7ff2fb6ee700 1 -- 172.21.15.71:0/3476513997 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f4196ed0 con 0x7ff2f4100ea0 2024-04-23T02:18:55.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.024+0000 7ff2fb6ee700 1 -- 172.21.15.71:0/3476513997 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f41092a0 con 0x7ff2f4100ea0 2024-04-23T02:18:55.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.025+0000 7ff2ea7fc700 1 -- 172.21.15.71:0/3476513997 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff2f00105d0 con 0x7ff2f4100ea0 2024-04-23T02:18:55.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.025+0000 7ff2ea7fc700 1 --2- 172.21.15.71:0/3476513997 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff2e006cce0 0x7ff2e006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:55.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.025+0000 7ff2ea7fc700 1 -- 172.21.15.71:0/3476513997 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff2f008c530 con 0x7ff2f4100ea0 2024-04-23T02:18:55.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.025+0000 7ff2f8c89700 1 --2- 172.21.15.71:0/3476513997 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff2e006cce0 0x7ff2e006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:55.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.025+0000 7ff2f8c89700 1 --2- 172.21.15.71:0/3476513997 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff2e006cce0 0x7ff2e006f1a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40fcfc0 tx=0x7ff2e401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:55.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.029+0000 7ff2ea7fc700 1 -- 172.21.15.71:0/3476513997 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff2f005a0e0 con 0x7ff2f4100ea0 2024-04-23T02:18:55.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:54 smithi192 ceph-mon[108617]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:18:55.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.355+0000 7ff2fb6ee700 1 -- 172.21.15.71:0/3476513997 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff2f41970f0 con 0x7ff2f4100ea0 2024-04-23T02:18:55.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.356+0000 7ff2ea7fc700 1 -- 172.21.15.71:0/3476513997 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+693 (secure 0 0 0) 0x7ff2f0010880 con 0x7ff2f4100ea0 2024-04-23T02:18:55.360 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:18:55.360 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:18:55.360 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:18:55.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:18:55.360 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:18:55.360 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:18:55.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:18:55.360 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:18:55.360 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5 2024-04-23T02:18:55.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:18:55.360 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:18:55.361 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-23T02:18:55.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:18:55.361 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:18:55.361 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 9, 2024-04-23T02:18:55.361 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:18:55.361 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:18:55.361 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:18:55.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.358+0000 7ff2fb6ee700 1 -- 172.21.15.71:0/3476513997 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff2e006cce0 msgr2=0x7ff2e006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:55.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.358+0000 7ff2fb6ee700 1 --2- 172.21.15.71:0/3476513997 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff2e006cce0 0x7ff2e006f1a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40fcfc0 tx=0x7ff2e401a040 comp rx=0 tx=0).stop 2024-04-23T02:18:55.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.358+0000 7ff2fb6ee700 1 -- 172.21.15.71:0/3476513997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2f4100ea0 msgr2=0x7ff2f419c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:55.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.358+0000 7ff2fb6ee700 1 --2- 172.21.15.71:0/3476513997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2f4100ea0 0x7ff2f419c620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000d8d0 tx=0x7ff2f000dc90 comp rx=0 tx=0).stop 2024-04-23T02:18:55.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.361+0000 7ff2fb6ee700 1 -- 172.21.15.71:0/3476513997 shutdown_connections 2024-04-23T02:18:55.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.361+0000 7ff2fb6ee700 1 --2- 172.21.15.71:0/3476513997 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff2e006cce0 0x7ff2e006f1a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:55.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.361+0000 7ff2fb6ee700 1 --2- 172.21.15.71:0/3476513997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2f4100ea0 0x7ff2f419c620 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:55.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.361+0000 7ff2fb6ee700 1 --2- 172.21.15.71:0/3476513997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2f4101850 0x7ff2f419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:55.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.361+0000 7ff2fb6ee700 1 -- 172.21.15.71:0/3476513997 >> 172.21.15.71:0/3476513997 conn(0x7ff2f40fc750 msgr2=0x7ff2f40fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:55.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.361+0000 7ff2fb6ee700 1 -- 172.21.15.71:0/3476513997 shutdown_connections 2024-04-23T02:18:55.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.361+0000 7ff2fb6ee700 1 -- 172.21.15.71:0/3476513997 wait complete. 2024-04-23T02:18:55.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.473+0000 7efe60ab8700 1 -- 172.21.15.71:0/1542045218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe5c0ffda0 msgr2=0x7efe5c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:55.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.473+0000 7efe60ab8700 1 --2- 172.21.15.71:0/1542045218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe5c0ffda0 0x7efe5c100180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009910 tx=0x7efe4c009c20 comp rx=0 tx=0).stop 2024-04-23T02:18:55.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.473+0000 7efe60ab8700 1 -- 172.21.15.71:0/1542045218 shutdown_connections 2024-04-23T02:18:55.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.473+0000 7efe60ab8700 1 --2- 172.21.15.71:0/1542045218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe5c100750 0x7efe5c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:55.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.473+0000 7efe60ab8700 1 --2- 172.21.15.71:0/1542045218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe5c0ffda0 0x7efe5c100180 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:55.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.473+0000 7efe60ab8700 1 -- 172.21.15.71:0/1542045218 >> 172.21.15.71:0/1542045218 conn(0x7efe5c0fb650 msgr2=0x7efe5c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:55.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.473+0000 7efe60ab8700 1 -- 172.21.15.71:0/1542045218 shutdown_connections 2024-04-23T02:18:55.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.473+0000 7efe60ab8700 1 -- 172.21.15.71:0/1542045218 wait complete. 2024-04-23T02:18:55.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.474+0000 7efe60ab8700 1 Processor -- start 2024-04-23T02:18:55.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.474+0000 7efe60ab8700 1 -- start start 2024-04-23T02:18:55.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.474+0000 7efe60ab8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe5c0ffda0 0x7efe5c197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:55.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.474+0000 7efe60ab8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe5c100750 0x7efe5c197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:55.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.474+0000 7efe60ab8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c1982a0 con 0x7efe5c0ffda0 2024-04-23T02:18:55.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.474+0000 7efe60ab8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c1983e0 con 0x7efe5c100750 2024-04-23T02:18:55.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.475+0000 7efe5a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe5c0ffda0 0x7efe5c197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:55.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.475+0000 7efe5a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe5c0ffda0 0x7efe5c197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47418/0 (socket says 172.21.15.71:47418) 2024-04-23T02:18:55.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.475+0000 7efe5a59c700 1 -- 172.21.15.71:0/2165036409 learned_addr learned my addr 172.21.15.71:0/2165036409 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:18:55.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.475+0000 7efe59d9b700 1 --2- 172.21.15.71:0/2165036409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe5c100750 0x7efe5c197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:55.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.475+0000 7efe5a59c700 1 -- 172.21.15.71:0/2165036409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe5c100750 msgr2=0x7efe5c197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:55.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.475+0000 7efe5a59c700 1 --2- 172.21.15.71:0/2165036409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe5c100750 0x7efe5c197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:55.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.475+0000 7efe5a59c700 1 -- 172.21.15.71:0/2165036409 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c0095f0 con 0x7efe5c0ffda0 2024-04-23T02:18:55.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.475+0000 7efe5a59c700 1 --2- 172.21.15.71:0/2165036409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe5c0ffda0 0x7efe5c197740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0092e0 tx=0x7efe4c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:55.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.475+0000 7efe4b7fe700 1 -- 172.21.15.71:0/2165036409 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe4c01d070 con 0x7efe5c0ffda0 2024-04-23T02:18:55.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.475+0000 7efe60ab8700 1 -- 172.21.15.71:0/2165036409 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5c1a07b0 con 0x7efe5c0ffda0 2024-04-23T02:18:55.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.475+0000 7efe4b7fe700 1 -- 172.21.15.71:0/2165036409 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efe4c00f460 con 0x7efe5c0ffda0 2024-04-23T02:18:55.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.475+0000 7efe4b7fe700 1 -- 172.21.15.71:0/2165036409 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe4c021620 con 0x7efe5c0ffda0 2024-04-23T02:18:55.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.475+0000 7efe60ab8700 1 -- 172.21.15.71:0/2165036409 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5c1a0bc0 con 0x7efe5c0ffda0 2024-04-23T02:18:55.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.476+0000 7efe60ab8700 1 -- 172.21.15.71:0/2165036409 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe5c108140 con 0x7efe5c0ffda0 2024-04-23T02:18:55.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.477+0000 7efe4b7fe700 1 -- 172.21.15.71:0/2165036409 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7efe4c003a60 con 0x7efe5c0ffda0 2024-04-23T02:18:55.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.477+0000 7efe4b7fe700 1 --2- 172.21.15.71:0/2165036409 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efe4406cce0 0x7efe4406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:55.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.477+0000 7efe4b7fe700 1 -- 172.21.15.71:0/2165036409 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7efe4c08d7a0 con 0x7efe5c0ffda0 2024-04-23T02:18:55.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.478+0000 7efe59d9b700 1 --2- 172.21.15.71:0/2165036409 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efe4406cce0 0x7efe4406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:55.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.478+0000 7efe59d9b700 1 --2- 172.21.15.71:0/2165036409 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efe4406cce0 0x7efe4406f1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0696c0 tx=0x7efe50009190 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:55.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.482+0000 7efe4b7fe700 1 -- 172.21.15.71:0/2165036409 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7efe4c0578c0 con 0x7efe5c0ffda0 2024-04-23T02:18:55.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.729+0000 7efe60ab8700 1 -- 172.21.15.71:0/2165036409 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efe5c107b20 con 0x7efe5c0ffda0 2024-04-23T02:18:55.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.729+0000 7efe4b7fe700 1 -- 172.21.15.71:0/2165036409 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1869 (secure 0 0 0) 0x7efe4c05aee0 con 0x7efe5c0ffda0 2024-04-23T02:18:55.733 INFO:teuthology.orchestra.run.smithi071.stdout:e25 2024-04-23T02:18:55.733 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:18:55.733 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:18:55.733 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:18:55.733 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:18:55.734 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:18:55.734 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:18:55.734 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 25 2024-04-23T02:18:55.734 INFO:teuthology.orchestra.run.smithi071.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T02:18:55.734 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:18:55.734 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:17:27.814232+0000 2024-04-23T02:18:55.734 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:18:55.734 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:18:55.734 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:18:55.734 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:18:55.734 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:18:55.734 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:18:55.734 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:18:55.734 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:18:55.735 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 46 2024-04-23T02:18:55.735 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:18:55.735 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:18:55.735 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:18:55.735 INFO:teuthology.orchestra.run.smithi071.stdout:up {0=14478,1=24243} 2024-04-23T02:18:55.735 INFO:teuthology.orchestra.run.smithi071.stdout:failed 2024-04-23T02:18:55.735 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:18:55.735 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:18:55.735 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:18:55.735 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:18:55.735 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:18:55.735 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:18:55.735 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:18:55.735 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:18:55.735 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:18:55.736 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/3442634960,v1:172.21.15.71:6827/3442634960] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:18:55.736 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{1:24243} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.192:6824/3780549896,v1:172.21.15.192:6825/3780549896] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:18:55.736 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{1:24255} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2100078284,v1:172.21.15.192:6827/2100078284] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:18:55.736 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:18:55.736 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:18:55.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.731+0000 7efe60ab8700 1 -- 172.21.15.71:0/2165036409 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efe4406cce0 msgr2=0x7efe4406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:55.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.731+0000 7efe60ab8700 1 --2- 172.21.15.71:0/2165036409 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efe4406cce0 0x7efe4406f1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0696c0 tx=0x7efe50009190 comp rx=0 tx=0).stop 2024-04-23T02:18:55.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.731+0000 7efe60ab8700 1 -- 172.21.15.71:0/2165036409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe5c0ffda0 msgr2=0x7efe5c197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:55.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.731+0000 7efe60ab8700 1 --2- 172.21.15.71:0/2165036409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe5c0ffda0 0x7efe5c197740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0092e0 tx=0x7efe4c00bd70 comp rx=0 tx=0).stop 2024-04-23T02:18:55.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.731+0000 7efe60ab8700 1 -- 172.21.15.71:0/2165036409 shutdown_connections 2024-04-23T02:18:55.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.731+0000 7efe60ab8700 1 --2- 172.21.15.71:0/2165036409 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efe4406cce0 0x7efe4406f1a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:55.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.731+0000 7efe60ab8700 1 --2- 172.21.15.71:0/2165036409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe5c0ffda0 0x7efe5c197740 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:55.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.731+0000 7efe60ab8700 1 --2- 172.21.15.71:0/2165036409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe5c100750 0x7efe5c197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:55.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.731+0000 7efe60ab8700 1 -- 172.21.15.71:0/2165036409 >> 172.21.15.71:0/2165036409 conn(0x7efe5c0fb650 msgr2=0x7efe5c0fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:55.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.731+0000 7efe60ab8700 1 -- 172.21.15.71:0/2165036409 shutdown_connections 2024-04-23T02:18:55.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.731+0000 7efe60ab8700 1 -- 172.21.15.71:0/2165036409 wait complete. 2024-04-23T02:18:55.737 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 25 2024-04-23T02:18:55.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.857+0000 7f0846587700 1 -- 172.21.15.71:0/3549586180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08400ffc20 msgr2=0x7f0840100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.857+0000 7f0846587700 1 --2- 172.21.15.71:0/3549586180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08400ffc20 0x7f0840100080 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0834009910 tx=0x7f0834009c20 comp rx=0 tx=0).stop 2024-04-23T02:18:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.857+0000 7f0846587700 1 -- 172.21.15.71:0/3549586180 shutdown_connections 2024-04-23T02:18:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.857+0000 7f0846587700 1 --2- 172.21.15.71:0/3549586180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08400ffc20 0x7f0840100080 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.857+0000 7f0846587700 1 --2- 172.21.15.71:0/3549586180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0840105bd0 0x7f0840105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.857+0000 7f0846587700 1 -- 172.21.15.71:0/3549586180 >> 172.21.15.71:0/3549586180 conn(0x7f0840075670 msgr2=0x7f0840075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.857+0000 7f0846587700 1 -- 172.21.15.71:0/3549586180 shutdown_connections 2024-04-23T02:18:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.857+0000 7f0846587700 1 -- 172.21.15.71:0/3549586180 wait complete. 2024-04-23T02:18:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.858+0000 7f0846587700 1 Processor -- start 2024-04-23T02:18:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.858+0000 7f0846587700 1 -- start start 2024-04-23T02:18:55.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.858+0000 7f0846587700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08400ffc20 0x7f0840101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.858+0000 7f0846587700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0840105bd0 0x7f0840101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.858+0000 7f0846587700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08401022c0 con 0x7f0840105bd0 2024-04-23T02:18:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.858+0000 7f0846587700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0840102400 con 0x7f08400ffc20 2024-04-23T02:18:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.858+0000 7f083f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0840105bd0 0x7f0840101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.858+0000 7f083f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0840105bd0 0x7f0840101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47432/0 (socket says 172.21.15.71:47432) 2024-04-23T02:18:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.858+0000 7f083f7fe700 1 -- 172.21.15.71:0/1903268952 learned_addr learned my addr 172.21.15.71:0/1903268952 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:18:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.859+0000 7f083ffff700 1 --2- 172.21.15.71:0/1903268952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08400ffc20 0x7f0840101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.859+0000 7f083f7fe700 1 -- 172.21.15.71:0/1903268952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08400ffc20 msgr2=0x7f0840101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.859+0000 7f083f7fe700 1 --2- 172.21.15.71:0/1903268952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08400ffc20 0x7f0840101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.859+0000 7f083f7fe700 1 -- 172.21.15.71:0/1903268952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08340095f0 con 0x7f0840105bd0 2024-04-23T02:18:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.859+0000 7f083ffff700 1 --2- 172.21.15.71:0/1903268952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08400ffc20 0x7f0840101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:18:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.859+0000 7f083f7fe700 1 --2- 172.21.15.71:0/1903268952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0840105bd0 0x7f0840101d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0834006010 tx=0x7f083400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.859+0000 7f083d7fa700 1 -- 172.21.15.71:0/1903268952 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f083401d070 con 0x7f0840105bd0 2024-04-23T02:18:55.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.859+0000 7f083d7fa700 1 -- 172.21.15.71:0/1903268952 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f083400f460 con 0x7f0840105bd0 2024-04-23T02:18:55.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.859+0000 7f083d7fa700 1 -- 172.21.15.71:0/1903268952 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0834021640 con 0x7f0840105bd0 2024-04-23T02:18:55.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.859+0000 7f0846587700 1 -- 172.21.15.71:0/1903268952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0840102680 con 0x7f0840105bd0 2024-04-23T02:18:55.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.859+0000 7f0846587700 1 -- 172.21.15.71:0/1903268952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08401a2d80 con 0x7f0840105bd0 2024-04-23T02:18:55.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.860+0000 7f0846587700 1 -- 172.21.15.71:0/1903268952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f084004f2e0 con 0x7f0840105bd0 2024-04-23T02:18:55.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.862+0000 7f083d7fa700 1 -- 172.21.15.71:0/1903268952 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f083400f9e0 con 0x7f0840105bd0 2024-04-23T02:18:55.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.862+0000 7f083d7fa700 1 --2- 172.21.15.71:0/1903268952 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f082806cce0 0x7f082806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:55.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.862+0000 7f083d7fa700 1 -- 172.21.15.71:0/1903268952 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f083408dc00 con 0x7f0840105bd0 2024-04-23T02:18:55.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.862+0000 7f083ffff700 1 --2- 172.21.15.71:0/1903268952 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f082806cce0 0x7f082806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:55.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.863+0000 7f083ffff700 1 --2- 172.21.15.71:0/1903268952 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f082806cce0 0x7f082806f1a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f08300099d0 tx=0x7f0830008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:55.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:55.866+0000 7f083d7fa700 1 -- 172.21.15.71:0/1903268952 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0834057d20 con 0x7f0840105bd0 2024-04-23T02:18:55.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:55 smithi071 ceph-mon[144544]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:18:55.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:55 smithi071 ceph-mon[144544]: pgmap v135: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 71/366 objects degraded (19.399%) 2024-04-23T02:18:55.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:55 smithi071 ceph-mon[144544]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:18:55.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:55 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3476513997' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:18:56.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.116+0000 7f0846587700 1 -- 172.21.15.71:0/1903268952 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0840104990 con 0x7f082806cce0 2024-04-23T02:18:56.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.117+0000 7f083d7fa700 1 -- 172.21.15.71:0/1903268952 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0840104990 con 0x7f082806cce0 2024-04-23T02:18:56.121 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:18:56.121 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:18:56.121 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:18:56.121 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:18:56.121 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:18:56.121 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:18:56.121 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:18:56.122 INFO:teuthology.orchestra.run.smithi071.stdout: "mon" 2024-04-23T02:18:56.122 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:18:56.122 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "6/23 daemons upgraded", 2024-04-23T02:18:56.122 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading osd daemons", 2024-04-23T02:18:56.122 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:18:56.123 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:18:56.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.119+0000 7f0846587700 1 -- 172.21.15.71:0/1903268952 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f082806cce0 msgr2=0x7f082806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:56.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.119+0000 7f0846587700 1 --2- 172.21.15.71:0/1903268952 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f082806cce0 0x7f082806f1a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f08300099d0 tx=0x7f0830008040 comp rx=0 tx=0).stop 2024-04-23T02:18:56.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.119+0000 7f0846587700 1 -- 172.21.15.71:0/1903268952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0840105bd0 msgr2=0x7f0840101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:56.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.119+0000 7f0846587700 1 --2- 172.21.15.71:0/1903268952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0840105bd0 0x7f0840101d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0834006010 tx=0x7f083400be20 comp rx=0 tx=0).stop 2024-04-23T02:18:56.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.119+0000 7f0846587700 1 -- 172.21.15.71:0/1903268952 shutdown_connections 2024-04-23T02:18:56.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.119+0000 7f0846587700 1 --2- 172.21.15.71:0/1903268952 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f082806cce0 0x7f082806f1a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:56.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.119+0000 7f0846587700 1 --2- 172.21.15.71:0/1903268952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08400ffc20 0x7f0840101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:56.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.119+0000 7f0846587700 1 --2- 172.21.15.71:0/1903268952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0840105bd0 0x7f0840101d80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:56.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.119+0000 7f0846587700 1 -- 172.21.15.71:0/1903268952 >> 172.21.15.71:0/1903268952 conn(0x7f0840075670 msgr2=0x7f08400686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:56.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.119+0000 7f0846587700 1 -- 172.21.15.71:0/1903268952 shutdown_connections 2024-04-23T02:18:56.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.119+0000 7f0846587700 1 -- 172.21.15.71:0/1903268952 wait complete. 2024-04-23T02:18:56.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:55 smithi192 ceph-mon[108617]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:18:56.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:55 smithi192 ceph-mon[108617]: pgmap v135: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 71/366 objects degraded (19.399%) 2024-04-23T02:18:56.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:55 smithi192 ceph-mon[108617]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:18:56.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:55 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3476513997' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:18:56.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.228+0000 7f3407c94700 1 -- 172.21.15.71:0/4126153380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3400105bd0 msgr2=0x7f3400105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:56.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.228+0000 7f3407c94700 1 --2- 172.21.15.71:0/4126153380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3400105bd0 0x7f3400105fb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009910 tx=0x7f33fc009c20 comp rx=0 tx=0).stop 2024-04-23T02:18:56.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.228+0000 7f3407c94700 1 -- 172.21.15.71:0/4126153380 shutdown_connections 2024-04-23T02:18:56.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.228+0000 7f3407c94700 1 --2- 172.21.15.71:0/4126153380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34000ffc20 0x7f3400100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:56.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.228+0000 7f3407c94700 1 --2- 172.21.15.71:0/4126153380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3400105bd0 0x7f3400105fb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:56.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.228+0000 7f3407c94700 1 -- 172.21.15.71:0/4126153380 >> 172.21.15.71:0/4126153380 conn(0x7f3400075670 msgr2=0x7f3400075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:56.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.228+0000 7f3407c94700 1 -- 172.21.15.71:0/4126153380 shutdown_connections 2024-04-23T02:18:56.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.228+0000 7f3407c94700 1 -- 172.21.15.71:0/4126153380 wait complete. 2024-04-23T02:18:56.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.228+0000 7f3407c94700 1 Processor -- start 2024-04-23T02:18:56.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.229+0000 7f3407c94700 1 -- start start 2024-04-23T02:18:56.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.229+0000 7f3407c94700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34000ffc20 0x7f3400101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:56.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.229+0000 7f3407c94700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3400105bd0 0x7f3400101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:56.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.229+0000 7f3407c94700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3400102350 con 0x7f34000ffc20 2024-04-23T02:18:56.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.229+0000 7f3407c94700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34001024c0 con 0x7f3400105bd0 2024-04-23T02:18:56.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.229+0000 7f3405a30700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34000ffc20 0x7f3400101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:56.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.229+0000 7f3405a30700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34000ffc20 0x7f3400101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47446/0 (socket says 172.21.15.71:47446) 2024-04-23T02:18:56.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.229+0000 7f3405a30700 1 -- 172.21.15.71:0/67302612 learned_addr learned my addr 172.21.15.71:0/67302612 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:18:56.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.229+0000 7f340522f700 1 --2- 172.21.15.71:0/67302612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3400105bd0 0x7f3400101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:56.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.229+0000 7f3405a30700 1 -- 172.21.15.71:0/67302612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3400105bd0 msgr2=0x7f3400101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:56.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.229+0000 7f3405a30700 1 --2- 172.21.15.71:0/67302612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3400105bd0 0x7f3400101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:56.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.229+0000 7f3405a30700 1 -- 172.21.15.71:0/67302612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc0095f0 con 0x7f34000ffc20 2024-04-23T02:18:56.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.229+0000 7f340522f700 1 --2- 172.21.15.71:0/67302612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3400105bd0 0x7f3400101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:18:56.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.229+0000 7f3405a30700 1 --2- 172.21.15.71:0/67302612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34000ffc20 0x7f3400101840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f33fc006010 tx=0x7f33fc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:56.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.229+0000 7f33f6ffd700 1 -- 172.21.15.71:0/67302612 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33fc01d070 con 0x7f34000ffc20 2024-04-23T02:18:56.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.230+0000 7f33f6ffd700 1 -- 172.21.15.71:0/67302612 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f33fc00f460 con 0x7f34000ffc20 2024-04-23T02:18:56.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.230+0000 7f3407c94700 1 -- 172.21.15.71:0/67302612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34001a29b0 con 0x7f34000ffc20 2024-04-23T02:18:56.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.230+0000 7f33f6ffd700 1 -- 172.21.15.71:0/67302612 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33fc021620 con 0x7f34000ffc20 2024-04-23T02:18:56.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.230+0000 7f3407c94700 1 -- 172.21.15.71:0/67302612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34001a2d60 con 0x7f34000ffc20 2024-04-23T02:18:56.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.230+0000 7f3407c94700 1 -- 172.21.15.71:0/67302612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340004f2e0 con 0x7f34000ffc20 2024-04-23T02:18:56.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.231+0000 7f33f6ffd700 1 -- 172.21.15.71:0/67302612 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f33fc00fa40 con 0x7f34000ffc20 2024-04-23T02:18:56.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.232+0000 7f33f6ffd700 1 --2- 172.21.15.71:0/67302612 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f33ec06cce0 0x7f33ec06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:18:56.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.232+0000 7f33f6ffd700 1 -- 172.21.15.71:0/67302612 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f33fc08d6e0 con 0x7f34000ffc20 2024-04-23T02:18:56.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.232+0000 7f340522f700 1 --2- 172.21.15.71:0/67302612 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f33ec06cce0 0x7f33ec06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:18:56.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.232+0000 7f340522f700 1 --2- 172.21.15.71:0/67302612 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f33ec06cce0 0x7f33ec06f1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f33f0005950 tx=0x7f33f00058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:18:56.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.237+0000 7f33f6ffd700 1 -- 172.21.15.71:0/67302612 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f33fc057800 con 0x7f34000ffc20 2024-04-23T02:18:56.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.560+0000 7f3407c94700 1 -- 172.21.15.71:0/67302612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f34001a2fd0 con 0x7f34000ffc20 2024-04-23T02:18:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.561+0000 7f33f6ffd700 1 -- 172.21.15.71:0/67302612 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1573 (secure 0 0 0) 0x7f33fc05ae20 con 0x7f34000ffc20 2024-04-23T02:18:56.565 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 71/366 objects degraded (19.399%), 20 pgs degraded 2024-04-23T02:18:56.565 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:18:56.565 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:18:56.565 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-23T02:18:56.565 INFO:teuthology.orchestra.run.smithi071.stdout: osd.0 (root=default,host=smithi071) is down 2024-04-23T02:18:56.565 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 71/366 objects degraded (19.399%), 20 pgs degraded 2024-04-23T02:18:56.565 INFO:teuthology.orchestra.run.smithi071.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-04-23T02:18:56.565 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-04-23T02:18:56.565 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-04-23T02:18:56.565 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-04-23T02:18:56.566 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-04-23T02:18:56.566 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-04-23T02:18:56.566 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-04-23T02:18:56.566 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-04-23T02:18:56.566 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.c is active+undersized+degraded, acting [2,3] 2024-04-23T02:18:56.566 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-04-23T02:18:56.566 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-04-23T02:18:56.566 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-04-23T02:18:56.566 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.12 is active+undersized+degraded, acting [3,1] 2024-04-23T02:18:56.566 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-04-23T02:18:56.566 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-04-23T02:18:56.566 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-04-23T02:18:56.566 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-04-23T02:18:56.566 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-04-23T02:18:56.566 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-04-23T02:18:56.567 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-04-23T02:18:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.564+0000 7f3407c94700 1 -- 172.21.15.71:0/67302612 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f33ec06cce0 msgr2=0x7f33ec06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.564+0000 7f3407c94700 1 --2- 172.21.15.71:0/67302612 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f33ec06cce0 0x7f33ec06f1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f33f0005950 tx=0x7f33f00058e0 comp rx=0 tx=0).stop 2024-04-23T02:18:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.564+0000 7f3407c94700 1 -- 172.21.15.71:0/67302612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34000ffc20 msgr2=0x7f3400101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:18:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.564+0000 7f3407c94700 1 --2- 172.21.15.71:0/67302612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34000ffc20 0x7f3400101840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f33fc006010 tx=0x7f33fc00be20 comp rx=0 tx=0).stop 2024-04-23T02:18:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.564+0000 7f3407c94700 1 -- 172.21.15.71:0/67302612 shutdown_connections 2024-04-23T02:18:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.564+0000 7f3407c94700 1 --2- 172.21.15.71:0/67302612 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f33ec06cce0 0x7f33ec06f1a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.564+0000 7f3407c94700 1 --2- 172.21.15.71:0/67302612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34000ffc20 0x7f3400101840 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:56.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.564+0000 7f3407c94700 1 --2- 172.21.15.71:0/67302612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3400105bd0 0x7f3400101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:18:56.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.564+0000 7f3407c94700 1 -- 172.21.15.71:0/67302612 >> 172.21.15.71:0/67302612 conn(0x7f3400075670 msgr2=0x7f3400068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:18:56.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.564+0000 7f3407c94700 1 -- 172.21.15.71:0/67302612 shutdown_connections 2024-04-23T02:18:56.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:18:56.564+0000 7f3407c94700 1 -- 172.21.15.71:0/67302612 wait complete. 2024-04-23T02:18:56.574 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:56 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-0[156416]: 2024-04-23T02:18:56.275+0000 7f73c97317c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-04-23T02:18:56.994 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:56 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-0[156416]: 2024-04-23T02:18:56.569+0000 7f73c97317c0 -1 osd.0 48 log_to_monitors true 2024-04-23T02:18:56.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:56 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2165036409' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:18:56.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:56 smithi071 ceph-mon[144544]: from='client.34150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:18:56.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:56 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/67302612' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:18:56.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:56 smithi071 ceph-mon[144544]: from='osd.0 [v2:172.21.15.71:6802/2865488005,v1:172.21.15.71:6803/2865488005]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T02:18:57.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:56 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2165036409' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:18:57.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:56 smithi192 ceph-mon[108617]: from='client.34150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:18:57.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:56 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/67302612' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:18:57.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:56 smithi192 ceph-mon[108617]: from='osd.0 [v2:172.21.15.71:6802/2865488005,v1:172.21.15.71:6803/2865488005]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T02:18:57.994 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 02:18:57 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-0[156416]: 2024-04-23T02:18:57.572+0000 7f73b6b83700 -1 osd.0 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T02:18:57.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:57 smithi071 ceph-mon[144544]: from='osd.0 [v2:172.21.15.71:6802/2865488005,v1:172.21.15.71:6803/2865488005]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T02:18:57.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:57 smithi071 ceph-mon[144544]: osdmap e51: 6 total, 5 up, 6 in 2024-04-23T02:18:57.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:57 smithi071 ceph-mon[144544]: from='osd.0 [v2:172.21.15.71:6802/2865488005,v1:172.21.15.71:6803/2865488005]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-23T02:18:57.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:57 smithi071 ceph-mon[144544]: pgmap v137: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 71/366 objects degraded (19.399%) 2024-04-23T02:18:57.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:57 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:57.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:57 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:57.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:57 smithi071 ceph-mon[144544]: from='osd.0 [v2:172.21.15.71:6802/2865488005,v1:172.21.15.71:6803/2865488005]' entity='osd.0' 2024-04-23T02:18:58.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:57 smithi192 ceph-mon[108617]: from='osd.0 [v2:172.21.15.71:6802/2865488005,v1:172.21.15.71:6803/2865488005]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T02:18:58.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:57 smithi192 ceph-mon[108617]: osdmap e51: 6 total, 5 up, 6 in 2024-04-23T02:18:58.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:57 smithi192 ceph-mon[108617]: from='osd.0 [v2:172.21.15.71:6802/2865488005,v1:172.21.15.71:6803/2865488005]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-23T02:18:58.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:57 smithi192 ceph-mon[108617]: pgmap v137: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 313 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 71/366 objects degraded (19.399%) 2024-04-23T02:18:58.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:57 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:58.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:57 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:58.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:57 smithi192 ceph-mon[108617]: from='osd.0 [v2:172.21.15.71:6802/2865488005,v1:172.21.15.71:6803/2865488005]' entity='osd.0' 2024-04-23T02:18:59.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:59.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:59.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:59 smithi071 ceph-mon[144544]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-23T02:18:59.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:59 smithi071 ceph-mon[144544]: osd.0 [v2:172.21.15.71:6802/2865488005,v1:172.21.15.71:6803/2865488005] boot 2024-04-23T02:18:59.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:59 smithi071 ceph-mon[144544]: osdmap e52: 6 total, 6 up, 6 in 2024-04-23T02:18:59.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:18:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T02:18:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:18:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:59 smithi192 ceph-mon[108617]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-23T02:18:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:59 smithi192 ceph-mon[108617]: osd.0 [v2:172.21.15.71:6802/2865488005,v1:172.21.15.71:6803/2865488005] boot 2024-04-23T02:18:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:59 smithi192 ceph-mon[108617]: osdmap e52: 6 total, 6 up, 6 in 2024-04-23T02:18:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:18:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T02:19:00.375 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:00 smithi071 ceph-mon[144544]: pgmap v139: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 71/366 objects degraded (19.399%) 2024-04-23T02:19:00.376 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:00 smithi071 ceph-mon[144544]: osdmap e53: 6 total, 6 up, 6 in 2024-04-23T02:19:00.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:00 smithi192 ceph-mon[108617]: pgmap v139: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 71/366 objects degraded (19.399%) 2024-04-23T02:19:00.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:00 smithi192 ceph-mon[108617]: osdmap e53: 6 total, 6 up, 6 in 2024-04-23T02:19:01.451 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:01 smithi071 ceph-mon[144544]: Health check update: Degraded data redundancy: 13/366 objects degraded (3.552%), 3 pgs degraded (PG_DEGRADED) 2024-04-23T02:19:01.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:01 smithi192 ceph-mon[108617]: Health check update: Degraded data redundancy: 13/366 objects degraded (3.552%), 3 pgs degraded (PG_DEGRADED) 2024-04-23T02:19:02.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:02 smithi071 ceph-mon[144544]: pgmap v141: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 5 op/s; 13/366 objects degraded (3.552%) 2024-04-23T02:19:02.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:02 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:02.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:02 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:02.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:02 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:19:02.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:02 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:19:02.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:02 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:02.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:02 smithi192 ceph-mon[108617]: pgmap v141: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 5 op/s; 13/366 objects degraded (3.552%) 2024-04-23T02:19:02.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:02 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:02.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:02 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:02.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:02 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:19:02.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:02 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:19:02.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:02 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:03.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:03 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:19:03.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:03 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:19:03.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:03 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:19:03.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:03 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:19:03.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:03 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-23T02:19:03.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:03 smithi071 ceph-mon[144544]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-23T02:19:03.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:03 smithi071 ceph-mon[144544]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-04-23T02:19:03.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:03 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:19:03.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:03 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:19:03.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:03 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:19:03.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:03 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:19:03.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:03 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-23T02:19:03.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:03 smithi192 ceph-mon[108617]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-23T02:19:03.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:03 smithi192 ceph-mon[108617]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-04-23T02:19:04.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:04 smithi071 ceph-mon[144544]: pgmap v142: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-04-23T02:19:04.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:04 smithi071 ceph-mon[144544]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 13/366 objects degraded (3.552%), 3 pgs degraded) 2024-04-23T02:19:04.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:04 smithi192 ceph-mon[108617]: pgmap v142: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-04-23T02:19:04.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:04 smithi192 ceph-mon[108617]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 13/366 objects degraded (3.552%), 3 pgs degraded) 2024-04-23T02:19:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:05 smithi071 ceph-mon[144544]: pgmap v143: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-04-23T02:19:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:19:06.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:05 smithi192 ceph-mon[108617]: pgmap v143: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-04-23T02:19:06.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:06.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:19:07.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:07 smithi071 ceph-mon[144544]: pgmap v144: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 2 op/s 2024-04-23T02:19:08.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:07 smithi192 ceph-mon[108617]: pgmap v144: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 2 op/s 2024-04-23T02:19:09.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:09 smithi071 ceph-mon[144544]: pgmap v145: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-04-23T02:19:10.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:09 smithi192 ceph-mon[108617]: pgmap v145: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-04-23T02:19:11.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:10 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:11.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:10 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:12.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:11 smithi192 ceph-mon[108617]: pgmap v146: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 3 op/s 2024-04-23T02:19:12.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:11 smithi071 ceph-mon[144544]: pgmap v146: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 3 op/s 2024-04-23T02:19:14.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:13 smithi192 ceph-mon[108617]: pgmap v147: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-04-23T02:19:14.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:13 smithi071 ceph-mon[144544]: pgmap v147: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-04-23T02:19:15.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:15 smithi192 ceph-mon[108617]: pgmap v148: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:19:15.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:15 smithi071 ceph-mon[144544]: pgmap v148: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:19:18.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:18 smithi071 ceph-mon[144544]: pgmap v149: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:19:18.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:18 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-23T02:19:18.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:18 smithi071 ceph-mon[144544]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-23T02:19:18.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:18 smithi071 ceph-mon[144544]: Upgrade: osd.1 is safe to restart 2024-04-23T02:19:18.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:18 smithi192 ceph-mon[108617]: pgmap v149: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:19:18.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:18 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-23T02:19:18.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:18 smithi192 ceph-mon[108617]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-23T02:19:18.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:18 smithi192 ceph-mon[108617]: Upgrade: osd.1 is safe to restart 2024-04-23T02:19:20.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:20 smithi192 ceph-mon[108617]: pgmap v150: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:19:20.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:20 smithi192 ceph-mon[108617]: Upgrade: Updating osd.1 2024-04-23T02:19:20.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:20.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T02:19:20.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:19:20.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:20 smithi192 ceph-mon[108617]: Deploying daemon osd.1 on smithi071 2024-04-23T02:19:20.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:19:20.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:20 smithi071 ceph-mon[144544]: pgmap v150: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:19:20.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:20 smithi071 ceph-mon[144544]: Upgrade: Updating osd.1 2024-04-23T02:19:20.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:20.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T02:19:20.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:19:20.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:20 smithi071 ceph-mon[144544]: Deploying daemon osd.1 on smithi071 2024-04-23T02:19:20.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:19:22.097 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:22 smithi071 ceph-mon[144544]: pgmap v151: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:19:22.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:22 smithi192 ceph-mon[108617]: pgmap v151: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:19:22.744 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:22 smithi071 systemd[1]: Stopping Ceph osd.1 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T02:19:22.744 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:22 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-1[72855]: 2024-04-23T02:19:22.656+0000 7f8d75989700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T02:19:22.744 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:22 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-1[72855]: 2024-04-23T02:19:22.656+0000 7f8d75989700 -1 osd.1 53 *** Got signal Terminated *** 2024-04-23T02:19:22.744 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:22 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-1[72855]: 2024-04-23T02:19:22.656+0000 7f8d75989700 -1 osd.1 53 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T02:19:23.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:23 smithi192 ceph-mon[108617]: osd.1 marked itself down and dead 2024-04-23T02:19:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:23 smithi071 ceph-mon[144544]: osd.1 marked itself down and dead 2024-04-23T02:19:24.244 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:24 smithi071 bash[162552]: ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-1 2024-04-23T02:19:24.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:24 smithi071 ceph-mon[144544]: pgmap v152: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:19:24.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:24 smithi071 ceph-mon[144544]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-23T02:19:24.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:24 smithi071 ceph-mon[144544]: osdmap e54: 6 total, 5 up, 6 in 2024-04-23T02:19:24.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:24 smithi192 ceph-mon[108617]: pgmap v152: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:19:24.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:24 smithi192 ceph-mon[108617]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-23T02:19:24.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:24 smithi192 ceph-mon[108617]: osdmap e54: 6 total, 5 up, 6 in 2024-04-23T02:19:25.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:25 smithi071 ceph-mon[144544]: osdmap e55: 6 total, 5 up, 6 in 2024-04-23T02:19:25.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:25 smithi192 ceph-mon[108617]: osdmap e55: 6 total, 5 up, 6 in 2024-04-23T02:19:26.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:26 smithi071 ceph-mon[144544]: pgmap v155: 65 pgs: 12 stale+active+clean, 53 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T02:19:26.245 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:25 smithi071 systemd[1]: ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.1.service: Succeeded. 2024-04-23T02:19:26.245 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:25 smithi071 systemd[1]: Stopped Ceph osd.1 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:19:26.245 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:26 smithi071 systemd[1]: Starting Ceph osd.1 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T02:19:26.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:26 smithi192 ceph-mon[108617]: pgmap v155: 65 pgs: 12 stale+active+clean, 53 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T02:19:26.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.678+0000 7f1ba4871700 1 -- 172.21.15.71:0/576253775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba0068580 msgr2=0x7f1ba00689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:26.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.678+0000 7f1ba4871700 1 --2- 172.21.15.71:0/576253775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba0068580 0x7f1ba00689e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009910 tx=0x7f1b94009c20 comp rx=0 tx=0).stop 2024-04-23T02:19:26.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.678+0000 7f1ba4871700 1 -- 172.21.15.71:0/576253775 shutdown_connections 2024-04-23T02:19:26.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.678+0000 7f1ba4871700 1 --2- 172.21.15.71:0/576253775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba0068580 0x7f1ba00689e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:26.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.678+0000 7f1ba4871700 1 --2- 172.21.15.71:0/576253775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ba0106cd0 0x7f1ba01070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:26.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.678+0000 7f1ba4871700 1 -- 172.21.15.71:0/576253775 >> 172.21.15.71:0/576253775 conn(0x7f1ba0075670 msgr2=0x7f1ba0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:26.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.678+0000 7f1ba4871700 1 -- 172.21.15.71:0/576253775 shutdown_connections 2024-04-23T02:19:26.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.678+0000 7f1ba4871700 1 -- 172.21.15.71:0/576253775 wait complete. 2024-04-23T02:19:26.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.679+0000 7f1ba4871700 1 Processor -- start 2024-04-23T02:19:26.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.679+0000 7f1ba4871700 1 -- start start 2024-04-23T02:19:26.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.680+0000 7f1ba4871700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba0068580 0x7f1ba0072920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:26.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.680+0000 7f1ba4871700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ba0106cd0 0x7f1ba006a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:26.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.680+0000 7f1ba4871700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba006ae50 con 0x7f1ba0068580 2024-04-23T02:19:26.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.680+0000 7f1ba4871700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba006afc0 con 0x7f1ba0106cd0 2024-04-23T02:19:26.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.680+0000 7f1b9e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba0068580 0x7f1ba0072920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:26.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.680+0000 7f1b9e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba0068580 0x7f1ba0072920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46396/0 (socket says 172.21.15.71:46396) 2024-04-23T02:19:26.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.680+0000 7f1b9e59c700 1 -- 172.21.15.71:0/1174571707 learned_addr learned my addr 172.21.15.71:0/1174571707 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:19:26.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.680+0000 7f1b9dd9b700 1 --2- 172.21.15.71:0/1174571707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ba0106cd0 0x7f1ba006a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:26.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.680+0000 7f1b9e59c700 1 -- 172.21.15.71:0/1174571707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ba0106cd0 msgr2=0x7f1ba006a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:26.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.680+0000 7f1b9e59c700 1 --2- 172.21.15.71:0/1174571707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ba0106cd0 0x7f1ba006a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:26.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.680+0000 7f1b9e59c700 1 -- 172.21.15.71:0/1174571707 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b940095f0 con 0x7f1ba0068580 2024-04-23T02:19:26.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.680+0000 7f1b9e59c700 1 --2- 172.21.15.71:0/1174571707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba0068580 0x7f1ba0072920 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000b4b0 tx=0x7f1b9000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:26.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.680+0000 7f1b8f7fe700 1 -- 172.21.15.71:0/1174571707 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b90011800 con 0x7f1ba0068580 2024-04-23T02:19:26.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.681+0000 7f1b8f7fe700 1 -- 172.21.15.71:0/1174571707 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1b90011e00 con 0x7f1ba0068580 2024-04-23T02:19:26.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.681+0000 7f1ba4871700 1 -- 172.21.15.71:0/1174571707 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba006b250 con 0x7f1ba0068580 2024-04-23T02:19:26.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.681+0000 7f1b8f7fe700 1 -- 172.21.15.71:0/1174571707 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b90010e60 con 0x7f1ba0068580 2024-04-23T02:19:26.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.681+0000 7f1ba4871700 1 -- 172.21.15.71:0/1174571707 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba01a9670 con 0x7f1ba0068580 2024-04-23T02:19:26.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.682+0000 7f1ba4871700 1 -- 172.21.15.71:0/1174571707 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba01091d0 con 0x7f1ba0068580 2024-04-23T02:19:26.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.682+0000 7f1b8f7fe700 1 -- 172.21.15.71:0/1174571707 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1b90011960 con 0x7f1ba0068580 2024-04-23T02:19:26.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.683+0000 7f1b8f7fe700 1 --2- 172.21.15.71:0/1174571707 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1b8806cce0 0x7f1b8806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:26.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.683+0000 7f1b9dd9b700 1 --2- 172.21.15.71:0/1174571707 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1b8806cce0 0x7f1b8806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:26.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.683+0000 7f1b8f7fe700 1 -- 172.21.15.71:0/1174571707 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1b9008c0c0 con 0x7f1ba0068580 2024-04-23T02:19:26.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.683+0000 7f1b9dd9b700 1 --2- 172.21.15.71:0/1174571707 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1b8806cce0 0x7f1b8806f1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1ba006c070 tx=0x7f1b9401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:26.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.687+0000 7f1b8f7fe700 1 -- 172.21.15.71:0/1174571707 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1b90056140 con 0x7f1ba0068580 2024-04-23T02:19:26.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.908+0000 7f1ba4871700 1 -- 172.21.15.71:0/1174571707 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ba006bee0 con 0x7f1b8806cce0 2024-04-23T02:19:26.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.909+0000 7f1b8f7fe700 1 -- 172.21.15.71:0/1174571707 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1ba006bee0 con 0x7f1b8806cce0 2024-04-23T02:19:26.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.911+0000 7f1ba4871700 1 -- 172.21.15.71:0/1174571707 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1b8806cce0 msgr2=0x7f1b8806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:26.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.911+0000 7f1ba4871700 1 --2- 172.21.15.71:0/1174571707 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1b8806cce0 0x7f1b8806f1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1ba006c070 tx=0x7f1b9401a040 comp rx=0 tx=0).stop 2024-04-23T02:19:26.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.911+0000 7f1ba4871700 1 -- 172.21.15.71:0/1174571707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba0068580 msgr2=0x7f1ba0072920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:26.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.911+0000 7f1ba4871700 1 --2- 172.21.15.71:0/1174571707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba0068580 0x7f1ba0072920 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000b4b0 tx=0x7f1b9000b7c0 comp rx=0 tx=0).stop 2024-04-23T02:19:26.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.911+0000 7f1ba4871700 1 -- 172.21.15.71:0/1174571707 shutdown_connections 2024-04-23T02:19:26.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.911+0000 7f1ba4871700 1 --2- 172.21.15.71:0/1174571707 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1b8806cce0 0x7f1b8806f1a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:26.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.911+0000 7f1ba4871700 1 --2- 172.21.15.71:0/1174571707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ba0068580 0x7f1ba0072920 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:26.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.911+0000 7f1ba4871700 1 --2- 172.21.15.71:0/1174571707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ba0106cd0 0x7f1ba006a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:26.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.911+0000 7f1ba4871700 1 -- 172.21.15.71:0/1174571707 >> 172.21.15.71:0/1174571707 conn(0x7f1ba0075670 msgr2=0x7f1ba00fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:26.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.912+0000 7f1ba4871700 1 -- 172.21.15.71:0/1174571707 shutdown_connections 2024-04-23T02:19:26.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:26.912+0000 7f1ba4871700 1 -- 172.21.15.71:0/1174571707 wait complete. 2024-04-23T02:19:26.926 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:19:27.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.030+0000 7f5496703700 1 -- 172.21.15.71:0/957252685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54900ffda0 msgr2=0x7f5490100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:27.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.030+0000 7f5496703700 1 --2- 172.21.15.71:0/957252685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54900ffda0 0x7f5490100180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5480009910 tx=0x7f5480009c20 comp rx=0 tx=0).stop 2024-04-23T02:19:27.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.030+0000 7f5496703700 1 -- 172.21.15.71:0/957252685 shutdown_connections 2024-04-23T02:19:27.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.030+0000 7f5496703700 1 --2- 172.21.15.71:0/957252685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5490100750 0x7f54901047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:27.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.030+0000 7f5496703700 1 --2- 172.21.15.71:0/957252685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54900ffda0 0x7f5490100180 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:27.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.030+0000 7f5496703700 1 -- 172.21.15.71:0/957252685 >> 172.21.15.71:0/957252685 conn(0x7f54900fb650 msgr2=0x7f54900fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:27.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.030+0000 7f5496703700 1 -- 172.21.15.71:0/957252685 shutdown_connections 2024-04-23T02:19:27.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.030+0000 7f5496703700 1 -- 172.21.15.71:0/957252685 wait complete. 2024-04-23T02:19:27.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.030+0000 7f5496703700 1 Processor -- start 2024-04-23T02:19:27.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.030+0000 7f5496703700 1 -- start start 2024-04-23T02:19:27.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.031+0000 7f5496703700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54900ffda0 0x7f54901956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:27.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.031+0000 7f5496703700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5490100750 0x7f5490195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:27.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.031+0000 7f5496703700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5490196200 con 0x7f5490100750 2024-04-23T02:19:27.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.031+0000 7f5496703700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5490196370 con 0x7f54900ffda0 2024-04-23T02:19:27.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.031+0000 7f548f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5490100750 0x7f5490195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:27.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.031+0000 7f548f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5490100750 0x7f5490195c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46414/0 (socket says 172.21.15.71:46414) 2024-04-23T02:19:27.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.031+0000 7f548f7fe700 1 -- 172.21.15.71:0/1441042609 learned_addr learned my addr 172.21.15.71:0/1441042609 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:19:27.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.031+0000 7f548ffff700 1 --2- 172.21.15.71:0/1441042609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54900ffda0 0x7f54901956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:27.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.031+0000 7f548f7fe700 1 -- 172.21.15.71:0/1441042609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54900ffda0 msgr2=0x7f54901956c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:27.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.031+0000 7f548f7fe700 1 --2- 172.21.15.71:0/1441042609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54900ffda0 0x7f54901956c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:27.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.031+0000 7f548f7fe700 1 -- 172.21.15.71:0/1441042609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54800095f0 con 0x7f5490100750 2024-04-23T02:19:27.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.031+0000 7f548f7fe700 1 --2- 172.21.15.71:0/1441042609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5490100750 0x7f5490195c00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f548400c960 tx=0x7f548400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:27.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.032+0000 7f548d7fa700 1 -- 172.21.15.71:0/1441042609 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5484007a70 con 0x7f5490100750 2024-04-23T02:19:27.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.032+0000 7f5496703700 1 -- 172.21.15.71:0/1441042609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5490199110 con 0x7f5490100750 2024-04-23T02:19:27.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.032+0000 7f548d7fa700 1 -- 172.21.15.71:0/1441042609 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5484007bd0 con 0x7f5490100750 2024-04-23T02:19:27.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.032+0000 7f548d7fa700 1 -- 172.21.15.71:0/1441042609 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5484018630 con 0x7f5490100750 2024-04-23T02:19:27.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.032+0000 7f5496703700 1 -- 172.21.15.71:0/1441042609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5490199660 con 0x7f5490100750 2024-04-23T02:19:27.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.033+0000 7f5496703700 1 -- 172.21.15.71:0/1441042609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5490108140 con 0x7f5490100750 2024-04-23T02:19:27.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.033+0000 7f548d7fa700 1 -- 172.21.15.71:0/1441042609 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5484018790 con 0x7f5490100750 2024-04-23T02:19:27.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.034+0000 7f548d7fa700 1 --2- 172.21.15.71:0/1441042609 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f547806cce0 0x7f547806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:27.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.034+0000 7f548d7fa700 1 -- 172.21.15.71:0/1441042609 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f548408c510 con 0x7f5490100750 2024-04-23T02:19:27.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.034+0000 7f548ffff700 1 --2- 172.21.15.71:0/1441042609 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f547806cce0 0x7f547806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:27.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.034+0000 7f548ffff700 1 --2- 172.21.15.71:0/1441042609 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f547806cce0 0x7f547806f1a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5480006010 tx=0x7f548001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:27.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.038+0000 7f548d7fa700 1 -- 172.21.15.71:0/1441042609 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f54840565d0 con 0x7f5490100750 2024-04-23T02:19:27.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.256+0000 7f5496703700 1 -- 172.21.15.71:0/1441042609 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f549002cce0 con 0x7f547806cce0 2024-04-23T02:19:27.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.257+0000 7f548d7fa700 1 -- 172.21.15.71:0/1441042609 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f549002cce0 con 0x7f547806cce0 2024-04-23T02:19:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.259+0000 7f5496703700 1 -- 172.21.15.71:0/1441042609 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f547806cce0 msgr2=0x7f547806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.259+0000 7f5496703700 1 --2- 172.21.15.71:0/1441042609 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f547806cce0 0x7f547806f1a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5480006010 tx=0x7f548001a040 comp rx=0 tx=0).stop 2024-04-23T02:19:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.259+0000 7f5496703700 1 -- 172.21.15.71:0/1441042609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5490100750 msgr2=0x7f5490195c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.259+0000 7f5496703700 1 --2- 172.21.15.71:0/1441042609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5490100750 0x7f5490195c00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f548400c960 tx=0x7f548400cc70 comp rx=0 tx=0).stop 2024-04-23T02:19:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.259+0000 7f5496703700 1 -- 172.21.15.71:0/1441042609 shutdown_connections 2024-04-23T02:19:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.259+0000 7f5496703700 1 --2- 172.21.15.71:0/1441042609 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f547806cce0 0x7f547806f1a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.259+0000 7f5496703700 1 --2- 172.21.15.71:0/1441042609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54900ffda0 0x7f54901956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.259+0000 7f5496703700 1 --2- 172.21.15.71:0/1441042609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5490100750 0x7f5490195c00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:27.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.259+0000 7f5496703700 1 -- 172.21.15.71:0/1441042609 >> 172.21.15.71:0/1441042609 conn(0x7f54900fb650 msgr2=0x7f54900fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:27.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.259+0000 7f5496703700 1 -- 172.21.15.71:0/1441042609 shutdown_connections 2024-04-23T02:19:27.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.259+0000 7f5496703700 1 -- 172.21.15.71:0/1441042609 wait complete. 2024-04-23T02:19:27.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.367+0000 7fe898ff3700 1 -- 172.21.15.71:0/3868894942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe894101ea0 msgr2=0x7fe894102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:27.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.367+0000 7fe898ff3700 1 --2- 172.21.15.71:0/3868894942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe894101ea0 0x7fe894102280 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00b0a0 tx=0x7fe88c00b3b0 comp rx=0 tx=0).stop 2024-04-23T02:19:27.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.367+0000 7fe898ff3700 1 -- 172.21.15.71:0/3868894942 shutdown_connections 2024-04-23T02:19:27.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.367+0000 7fe898ff3700 1 --2- 172.21.15.71:0/3868894942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8941027c0 0x7fe89410acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:27.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.367+0000 7fe898ff3700 1 --2- 172.21.15.71:0/3868894942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe894101ea0 0x7fe894102280 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:27.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.367+0000 7fe898ff3700 1 -- 172.21.15.71:0/3868894942 >> 172.21.15.71:0/3868894942 conn(0x7fe8940fb650 msgr2=0x7fe8940fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:27.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.367+0000 7fe898ff3700 1 -- 172.21.15.71:0/3868894942 shutdown_connections 2024-04-23T02:19:27.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.367+0000 7fe898ff3700 1 -- 172.21.15.71:0/3868894942 wait complete. 2024-04-23T02:19:27.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.368+0000 7fe898ff3700 1 Processor -- start 2024-04-23T02:19:27.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.368+0000 7fe898ff3700 1 -- start start 2024-04-23T02:19:27.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.368+0000 7fe898ff3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe894101ea0 0x7fe89419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:27.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.368+0000 7fe898ff3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8941027c0 0x7fe89419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:27.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.368+0000 7fe898ff3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89419d1f0 con 0x7fe8941027c0 2024-04-23T02:19:27.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.368+0000 7fe898ff3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8941966a0 con 0x7fe894101ea0 2024-04-23T02:19:27.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.368+0000 7fe891d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8941027c0 0x7fe89419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:27.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.368+0000 7fe891d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8941027c0 0x7fe89419cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46434/0 (socket says 172.21.15.71:46434) 2024-04-23T02:19:27.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.368+0000 7fe891d9b700 1 -- 172.21.15.71:0/260662704 learned_addr learned my addr 172.21.15.71:0/260662704 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:19:27.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.368+0000 7fe89259c700 1 --2- 172.21.15.71:0/260662704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe894101ea0 0x7fe89419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:27.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.368+0000 7fe891d9b700 1 -- 172.21.15.71:0/260662704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe894101ea0 msgr2=0x7fe89419c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:27.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.368+0000 7fe891d9b700 1 --2- 172.21.15.71:0/260662704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe894101ea0 0x7fe89419c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:27.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.369+0000 7fe891d9b700 1 -- 172.21.15.71:0/260662704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c009d00 con 0x7fe8941027c0 2024-04-23T02:19:27.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.369+0000 7fe891d9b700 1 --2- 172.21.15.71:0/260662704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8941027c0 0x7fe89419cb60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00d8d0 tx=0x7fe87c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:27.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.369+0000 7fe88b7fe700 1 -- 172.21.15.71:0/260662704 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe87c0041f0 con 0x7fe8941027c0 2024-04-23T02:19:27.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.369+0000 7fe88b7fe700 1 -- 172.21.15.71:0/260662704 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe87c007620 con 0x7fe8941027c0 2024-04-23T02:19:27.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.369+0000 7fe898ff3700 1 -- 172.21.15.71:0/260662704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe894196980 con 0x7fe8941027c0 2024-04-23T02:19:27.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.369+0000 7fe88b7fe700 1 -- 172.21.15.71:0/260662704 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe87c005610 con 0x7fe8941027c0 2024-04-23T02:19:27.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.369+0000 7fe898ff3700 1 -- 172.21.15.71:0/260662704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe894196ed0 con 0x7fe8941027c0 2024-04-23T02:19:27.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.370+0000 7fe898ff3700 1 -- 172.21.15.71:0/260662704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe894108350 con 0x7fe8941027c0 2024-04-23T02:19:27.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.370+0000 7fe88b7fe700 1 -- 172.21.15.71:0/260662704 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe87c004750 con 0x7fe8941027c0 2024-04-23T02:19:27.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.371+0000 7fe88b7fe700 1 --2- 172.21.15.71:0/260662704 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe88006cce0 0x7fe88006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:27.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.371+0000 7fe89259c700 1 --2- 172.21.15.71:0/260662704 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe88006cce0 0x7fe88006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:27.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.371+0000 7fe88b7fe700 1 -- 172.21.15.71:0/260662704 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe87c08c350 con 0x7fe8941027c0 2024-04-23T02:19:27.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.371+0000 7fe89259c700 1 --2- 172.21.15.71:0/260662704 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe88006cce0 0x7fe88006f1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009fd0 tx=0x7fe88c0060c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:27.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.375+0000 7fe88b7fe700 1 -- 172.21.15.71:0/260662704 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe87c059ee0 con 0x7fe8941027c0 2024-04-23T02:19:27.431 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:27 smithi071 ceph-mon[144544]: Health check failed: Degraded data redundancy: 56/366 objects degraded (15.301%), 16 pgs degraded (PG_DEGRADED) 2024-04-23T02:19:27.432 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:27 smithi071 bash[162834]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-23T02:19:27.432 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:27 smithi071 bash[162834]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-23T02:19:27.432 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:27 smithi071 bash[162834]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-23T02:19:27.432 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:27 smithi071 bash[162834]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-23T02:19:27.432 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:27 smithi071 bash[162834]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-23T02:19:27.432 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:27 smithi071 bash[162834]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-23T02:19:27.432 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:27 smithi071 bash[162834]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-23T02:19:27.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.581+0000 7fe898ff3700 1 -- 172.21.15.71:0/260662704 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe894197140 con 0x7fe88006cce0 2024-04-23T02:19:27.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.591+0000 7fe88b7fe700 1 -- 172.21.15.71:0/260662704 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe894197140 con 0x7fe88006cce0 2024-04-23T02:19:27.595 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:19:27.595 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (3m) 30s ago 14m 26.3M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:19:27.596 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (14m) 30s ago 14m 8158k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:19:27.596 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (13m) 54s ago 13m 8036k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:19:27.596 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (63s) 30s ago 14m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:19:27.596 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (57s) 54s ago 13m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:19:27.596 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (2m) 30s ago 14m 78.2M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:19:27.596 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (8m) 30s ago 8m 64.9M - 18.2.1 5be31c24972a a515de2a841c 2024-04-23T02:19:27.596 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (8m) 30s ago 8m 133M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:19:27.596 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (8m) 54s ago 8m 68.1M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:19:27.596 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (8m) 54s ago 8m 122M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:19:27.596 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (5m) 30s ago 16m 504M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:19:27.596 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (4m) 54s ago 13m 451M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:19:27.596 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (95s) 30s ago 16m 44.3M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:19:27.596 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (77s) 54s ago 13m 31.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:19:27.596 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (4m) 30s ago 14m 19.3M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:19:27.597 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (3m) 54s ago 13m 17.8M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:19:27.597 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (36s) 30s ago 12m 10.9M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:19:27.597 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (12m) 30s ago 11m 209M 1227M 18.2.1 5be31c24972a 3992157ff0fd 2024-04-23T02:19:27.597 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (11m) 30s ago 11m 164M 1227M 18.2.1 5be31c24972a 12380b6a15a2 2024-04-23T02:19:27.597 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (11m) 54s ago 11m 243M 2251M 18.2.1 5be31c24972a d5f587ee3a22 2024-04-23T02:19:27.597 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (10m) 54s ago 10m 219M 2251M 18.2.1 5be31c24972a 958d046005f0 2024-04-23T02:19:27.597 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (10m) 54s ago 10m 199M 2251M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:19:27.597 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (3m) 30s ago 13m 63.4M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:19:27.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.594+0000 7fe898ff3700 1 -- 172.21.15.71:0/260662704 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe88006cce0 msgr2=0x7fe88006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:27.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.594+0000 7fe898ff3700 1 --2- 172.21.15.71:0/260662704 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe88006cce0 0x7fe88006f1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009fd0 tx=0x7fe88c0060c0 comp rx=0 tx=0).stop 2024-04-23T02:19:27.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.594+0000 7fe898ff3700 1 -- 172.21.15.71:0/260662704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8941027c0 msgr2=0x7fe89419cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:27.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.594+0000 7fe898ff3700 1 --2- 172.21.15.71:0/260662704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8941027c0 0x7fe89419cb60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00d8d0 tx=0x7fe87c00dbe0 comp rx=0 tx=0).stop 2024-04-23T02:19:27.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.594+0000 7fe898ff3700 1 -- 172.21.15.71:0/260662704 shutdown_connections 2024-04-23T02:19:27.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.594+0000 7fe898ff3700 1 --2- 172.21.15.71:0/260662704 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe88006cce0 0x7fe88006f1a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:27.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.594+0000 7fe898ff3700 1 --2- 172.21.15.71:0/260662704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe894101ea0 0x7fe89419c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:27.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.594+0000 7fe898ff3700 1 --2- 172.21.15.71:0/260662704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8941027c0 0x7fe89419cb60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:27.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.594+0000 7fe898ff3700 1 -- 172.21.15.71:0/260662704 >> 172.21.15.71:0/260662704 conn(0x7fe8940fb650 msgr2=0x7fe894105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:27.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.594+0000 7fe898ff3700 1 -- 172.21.15.71:0/260662704 shutdown_connections 2024-04-23T02:19:27.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.594+0000 7fe898ff3700 1 -- 172.21.15.71:0/260662704 wait complete. 2024-04-23T02:19:27.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:27 smithi192 ceph-mon[108617]: Health check failed: Degraded data redundancy: 56/366 objects degraded (15.301%), 16 pgs degraded (PG_DEGRADED) 2024-04-23T02:19:27.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.696+0000 7f1105028700 1 -- 172.21.15.71:0/3585952738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1100101e00 msgr2=0x7f11001021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:27.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.696+0000 7f1105028700 1 --2- 172.21.15.71:0/3585952738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1100101e00 0x7f11001021e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009910 tx=0x7f10f0009c20 comp rx=0 tx=0).stop 2024-04-23T02:19:27.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.696+0000 7f1105028700 1 -- 172.21.15.71:0/3585952738 shutdown_connections 2024-04-23T02:19:27.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.696+0000 7f1105028700 1 --2- 172.21.15.71:0/3585952738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1100102720 0x7f110010ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:27.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.696+0000 7f1105028700 1 --2- 172.21.15.71:0/3585952738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1100101e00 0x7f11001021e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:27.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.696+0000 7f1105028700 1 -- 172.21.15.71:0/3585952738 >> 172.21.15.71:0/3585952738 conn(0x7f11000fb650 msgr2=0x7f11000fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:27.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.697+0000 7f1105028700 1 -- 172.21.15.71:0/3585952738 shutdown_connections 2024-04-23T02:19:27.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.697+0000 7f1105028700 1 -- 172.21.15.71:0/3585952738 wait complete. 2024-04-23T02:19:27.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.697+0000 7f1105028700 1 Processor -- start 2024-04-23T02:19:27.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.697+0000 7f1105028700 1 -- start start 2024-04-23T02:19:27.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.698+0000 7f1105028700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1100101e00 0x7f110019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:27.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.698+0000 7f1105028700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1100102720 0x7f110019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:27.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.698+0000 7f1105028700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110019d240 con 0x7f1100102720 2024-04-23T02:19:27.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.698+0000 7f1105028700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11001966f0 con 0x7f1100101e00 2024-04-23T02:19:27.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.698+0000 7f10fe59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1100102720 0x7f110019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:27.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.698+0000 7f10fe59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1100102720 0x7f110019cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46454/0 (socket says 172.21.15.71:46454) 2024-04-23T02:19:27.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.698+0000 7f10fe59c700 1 -- 172.21.15.71:0/3090271154 learned_addr learned my addr 172.21.15.71:0/3090271154 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:19:27.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.698+0000 7f10fed9d700 1 --2- 172.21.15.71:0/3090271154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1100101e00 0x7f110019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:27.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.698+0000 7f10fe59c700 1 -- 172.21.15.71:0/3090271154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1100101e00 msgr2=0x7f110019c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:27.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.698+0000 7f10fe59c700 1 --2- 172.21.15.71:0/3090271154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1100101e00 0x7f110019c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:27.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.698+0000 7f10fe59c700 1 -- 172.21.15.71:0/3090271154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f00095f0 con 0x7f1100102720 2024-04-23T02:19:27.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.698+0000 7f10fed9d700 1 --2- 172.21.15.71:0/3090271154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1100101e00 0x7f110019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:19:27.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.698+0000 7f10fe59c700 1 --2- 172.21.15.71:0/3090271154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1100102720 0x7f110019cbb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f10f400c8f0 tx=0x7f10f400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:27.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.698+0000 7f10e7fff700 1 -- 172.21.15.71:0/3090271154 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10f4004a00 con 0x7f1100102720 2024-04-23T02:19:27.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.698+0000 7f10e7fff700 1 -- 172.21.15.71:0/3090271154 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f10f4005020 con 0x7f1100102720 2024-04-23T02:19:27.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.698+0000 7f1105028700 1 -- 172.21.15.71:0/3090271154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11001969d0 con 0x7f1100102720 2024-04-23T02:19:27.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.698+0000 7f10e7fff700 1 -- 172.21.15.71:0/3090271154 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10f4007c10 con 0x7f1100102720 2024-04-23T02:19:27.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.698+0000 7f1105028700 1 -- 172.21.15.71:0/3090271154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1100196f20 con 0x7f1100102720 2024-04-23T02:19:27.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.699+0000 7f1105028700 1 -- 172.21.15.71:0/3090271154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1100108310 con 0x7f1100102720 2024-04-23T02:19:27.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.700+0000 7f10e7fff700 1 -- 172.21.15.71:0/3090271154 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f10f400f450 con 0x7f1100102720 2024-04-23T02:19:27.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.700+0000 7f10e7fff700 1 --2- 172.21.15.71:0/3090271154 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f10e806cce0 0x7f10e806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:27.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.700+0000 7f10e7fff700 1 -- 172.21.15.71:0/3090271154 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f10f408c0b0 con 0x7f1100102720 2024-04-23T02:19:27.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.700+0000 7f10fed9d700 1 --2- 172.21.15.71:0/3090271154 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f10e806cce0 0x7f10e806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:27.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.701+0000 7f10fed9d700 1 --2- 172.21.15.71:0/3090271154 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f10e806cce0 0x7f10e806f1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f10f000b5c0 tx=0x7f10f001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:27.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.705+0000 7f10e7fff700 1 -- 172.21.15.71:0/3090271154 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f10f4059c60 con 0x7f1100102720 2024-04-23T02:19:28.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.998+0000 7f1105028700 1 -- 172.21.15.71:0/3090271154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f110004f2e0 con 0x7f1100102720 2024-04-23T02:19:28.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:27.999+0000 7f10e7fff700 1 -- 172.21.15.71:0/3090271154 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7f10f40597f0 con 0x7f1100102720 2024-04-23T02:19:28.003 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:19:28.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:19:28.003 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:19:28.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:19:28.003 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:19:28.003 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:19:28.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:19:28.003 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:19:28.003 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4, 2024-04-23T02:19:28.003 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1 2024-04-23T02:19:28.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:19:28.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:19:28.004 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-23T02:19:28.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:19:28.004 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:19:28.004 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 8, 2024-04-23T02:19:28.004 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 5 2024-04-23T02:19:28.004 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:19:28.004 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:19:28.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.001+0000 7f1105028700 1 -- 172.21.15.71:0/3090271154 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f10e806cce0 msgr2=0x7f10e806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:28.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.001+0000 7f1105028700 1 --2- 172.21.15.71:0/3090271154 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f10e806cce0 0x7f10e806f1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f10f000b5c0 tx=0x7f10f001a040 comp rx=0 tx=0).stop 2024-04-23T02:19:28.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.001+0000 7f1105028700 1 -- 172.21.15.71:0/3090271154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1100102720 msgr2=0x7f110019cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:28.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.001+0000 7f1105028700 1 --2- 172.21.15.71:0/3090271154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1100102720 0x7f110019cbb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f10f400c8f0 tx=0x7f10f400ccb0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.001+0000 7f1105028700 1 -- 172.21.15.71:0/3090271154 shutdown_connections 2024-04-23T02:19:28.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.001+0000 7f1105028700 1 --2- 172.21.15.71:0/3090271154 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f10e806cce0 0x7f10e806f1a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.001+0000 7f1105028700 1 --2- 172.21.15.71:0/3090271154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1100101e00 0x7f110019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.001+0000 7f1105028700 1 --2- 172.21.15.71:0/3090271154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1100102720 0x7f110019cbb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.001+0000 7f1105028700 1 -- 172.21.15.71:0/3090271154 >> 172.21.15.71:0/3090271154 conn(0x7f11000fb650 msgr2=0x7f1100105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:28.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.001+0000 7f1105028700 1 -- 172.21.15.71:0/3090271154 shutdown_connections 2024-04-23T02:19:28.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.001+0000 7f1105028700 1 -- 172.21.15.71:0/3090271154 wait complete. 2024-04-23T02:19:28.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.105+0000 7f3f98454700 1 -- 172.21.15.71:0/4141702666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f90068580 msgr2=0x7f3f900689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:28.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.105+0000 7f3f98454700 1 --2- 172.21.15.71:0/4141702666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f90068580 0x7f3f900689e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009910 tx=0x7f3f80009c20 comp rx=0 tx=0).stop 2024-04-23T02:19:28.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.105+0000 7f3f98454700 1 -- 172.21.15.71:0/4141702666 shutdown_connections 2024-04-23T02:19:28.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.105+0000 7f3f98454700 1 --2- 172.21.15.71:0/4141702666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f90068580 0x7f3f900689e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.105+0000 7f3f98454700 1 --2- 172.21.15.71:0/4141702666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f90106cd0 0x7f3f901070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.105+0000 7f3f98454700 1 -- 172.21.15.71:0/4141702666 >> 172.21.15.71:0/4141702666 conn(0x7f3f90075670 msgr2=0x7f3f90075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:28.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.105+0000 7f3f98454700 1 -- 172.21.15.71:0/4141702666 shutdown_connections 2024-04-23T02:19:28.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.105+0000 7f3f98454700 1 -- 172.21.15.71:0/4141702666 wait complete. 2024-04-23T02:19:28.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.105+0000 7f3f98454700 1 Processor -- start 2024-04-23T02:19:28.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.105+0000 7f3f98454700 1 -- start start 2024-04-23T02:19:28.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.106+0000 7f3f98454700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f90068580 0x7f3f9019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:28.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.106+0000 7f3f98454700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f90106cd0 0x7f3f9019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:28.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.106+0000 7f3f98454700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9019d110 con 0x7f3f90106cd0 2024-04-23T02:19:28.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.106+0000 7f3f98454700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f901965c0 con 0x7f3f90068580 2024-04-23T02:19:28.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.106+0000 7f3f959ef700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f90106cd0 0x7f3f9019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:28.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.106+0000 7f3f959ef700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f90106cd0 0x7f3f9019ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46464/0 (socket says 172.21.15.71:46464) 2024-04-23T02:19:28.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.106+0000 7f3f959ef700 1 -- 172.21.15.71:0/2292008326 learned_addr learned my addr 172.21.15.71:0/2292008326 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:19:28.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.106+0000 7f3f961f0700 1 --2- 172.21.15.71:0/2292008326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f90068580 0x7f3f9019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:28.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.106+0000 7f3f959ef700 1 -- 172.21.15.71:0/2292008326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f90068580 msgr2=0x7f3f9019c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:28.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.106+0000 7f3f959ef700 1 --2- 172.21.15.71:0/2292008326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f90068580 0x7f3f9019c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.106+0000 7f3f959ef700 1 -- 172.21.15.71:0/2292008326 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f800095f0 con 0x7f3f90106cd0 2024-04-23T02:19:28.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.106+0000 7f3f961f0700 1 --2- 172.21.15.71:0/2292008326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f90068580 0x7f3f9019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:19:28.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.106+0000 7f3f959ef700 1 --2- 172.21.15.71:0/2292008326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f90106cd0 0x7f3f9019ca80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3f80006010 tx=0x7f3f8000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:28.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.106+0000 7f3f877fe700 1 -- 172.21.15.71:0/2292008326 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f8001d070 con 0x7f3f90106cd0 2024-04-23T02:19:28.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.106+0000 7f3f877fe700 1 -- 172.21.15.71:0/2292008326 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3f8000f460 con 0x7f3f90106cd0 2024-04-23T02:19:28.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.106+0000 7f3f877fe700 1 -- 172.21.15.71:0/2292008326 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f80021620 con 0x7f3f90106cd0 2024-04-23T02:19:28.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.106+0000 7f3f98454700 1 -- 172.21.15.71:0/2292008326 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f90196840 con 0x7f3f90106cd0 2024-04-23T02:19:28.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.106+0000 7f3f98454700 1 -- 172.21.15.71:0/2292008326 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f90196d30 con 0x7f3f90106cd0 2024-04-23T02:19:28.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.107+0000 7f3f98454700 1 -- 172.21.15.71:0/2292008326 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f901091d0 con 0x7f3f90106cd0 2024-04-23T02:19:28.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.108+0000 7f3f877fe700 1 -- 172.21.15.71:0/2292008326 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3f8000fa40 con 0x7f3f90106cd0 2024-04-23T02:19:28.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.108+0000 7f3f877fe700 1 --2- 172.21.15.71:0/2292008326 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3f7c06cce0 0x7f3f7c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:28.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.108+0000 7f3f877fe700 1 -- 172.21.15.71:0/2292008326 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3f8008d610 con 0x7f3f90106cd0 2024-04-23T02:19:28.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.108+0000 7f3f961f0700 1 --2- 172.21.15.71:0/2292008326 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3f7c06cce0 0x7f3f7c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:28.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.109+0000 7f3f961f0700 1 --2- 172.21.15.71:0/2292008326 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3f7c06cce0 0x7f3f7c06f1a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c005fd0 tx=0x7f3f8c005ee0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:28.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.113+0000 7f3f877fe700 1 -- 172.21.15.71:0/2292008326 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3f8005b240 con 0x7f3f90106cd0 2024-04-23T02:19:28.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.361+0000 7f3f98454700 1 -- 172.21.15.71:0/2292008326 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3f9004f2e0 con 0x7f3f90106cd0 2024-04-23T02:19:28.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.361+0000 7f3f877fe700 1 -- 172.21.15.71:0/2292008326 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1869 (secure 0 0 0) 0x7f3f8005add0 con 0x7f3f90106cd0 2024-04-23T02:19:28.367 INFO:teuthology.orchestra.run.smithi071.stdout:e25 2024-04-23T02:19:28.367 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:19:28.367 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:19:28.367 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:19:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:19:28.367 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:19:28.367 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:19:28.367 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 25 2024-04-23T02:19:28.367 INFO:teuthology.orchestra.run.smithi071.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T02:19:28.367 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:19:28.367 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:17:27.814232+0000 2024-04-23T02:19:28.367 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:19:28.367 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:19:28.368 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:19:28.368 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:19:28.368 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:19:28.368 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:19:28.368 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:19:28.368 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:19:28.368 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 46 2024-04-23T02:19:28.368 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:19:28.368 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:19:28.368 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:19:28.368 INFO:teuthology.orchestra.run.smithi071.stdout:up {0=14478,1=24243} 2024-04-23T02:19:28.368 INFO:teuthology.orchestra.run.smithi071.stdout:failed 2024-04-23T02:19:28.368 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:19:28.368 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:19:28.369 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:19:28.369 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:19:28.369 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:19:28.369 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:19:28.369 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:19:28.369 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:19:28.369 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:19:28.369 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/3442634960,v1:172.21.15.71:6827/3442634960] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:19:28.369 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{1:24243} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.192:6824/3780549896,v1:172.21.15.192:6825/3780549896] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:19:28.369 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{1:24255} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2100078284,v1:172.21.15.192:6827/2100078284] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:19:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:19:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:19:28.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.364+0000 7f3f98454700 1 -- 172.21.15.71:0/2292008326 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3f7c06cce0 msgr2=0x7f3f7c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:28.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.364+0000 7f3f98454700 1 --2- 172.21.15.71:0/2292008326 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3f7c06cce0 0x7f3f7c06f1a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c005fd0 tx=0x7f3f8c005ee0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.364+0000 7f3f98454700 1 -- 172.21.15.71:0/2292008326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f90106cd0 msgr2=0x7f3f9019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:28.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.364+0000 7f3f98454700 1 --2- 172.21.15.71:0/2292008326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f90106cd0 0x7f3f9019ca80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3f80006010 tx=0x7f3f8000be20 comp rx=0 tx=0).stop 2024-04-23T02:19:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.364+0000 7f3f98454700 1 -- 172.21.15.71:0/2292008326 shutdown_connections 2024-04-23T02:19:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.364+0000 7f3f98454700 1 --2- 172.21.15.71:0/2292008326 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3f7c06cce0 0x7f3f7c06f1a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.364+0000 7f3f98454700 1 --2- 172.21.15.71:0/2292008326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f90068580 0x7f3f9019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.364+0000 7f3f98454700 1 --2- 172.21.15.71:0/2292008326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f90106cd0 0x7f3f9019ca80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.364+0000 7f3f98454700 1 -- 172.21.15.71:0/2292008326 >> 172.21.15.71:0/2292008326 conn(0x7f3f90075670 msgr2=0x7f3f900fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.364+0000 7f3f98454700 1 -- 172.21.15.71:0/2292008326 shutdown_connections 2024-04-23T02:19:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.364+0000 7f3f98454700 1 -- 172.21.15.71:0/2292008326 wait complete. 2024-04-23T02:19:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 25 2024-04-23T02:19:28.416 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:28 smithi071 ceph-osd[163273]: -- 172.21.15.71:0/2350729613 <== mon.0 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5562b32131e0 con 0x5562b31e3000 2024-04-23T02:19:28.416 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:28 smithi071 ceph-mon[144544]: pgmap v156: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/366 objects degraded (15.301%) 2024-04-23T02:19:28.416 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:28 smithi071 ceph-mon[144544]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:19:28.417 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:28 smithi071 ceph-mon[144544]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:19:28.417 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:28 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3090271154' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:19:28.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.475+0000 7f8be12b9700 1 -- 172.21.15.71:0/2013150462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bdc0ffc20 msgr2=0x7f8bdc100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:28.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.475+0000 7f8be12b9700 1 --2- 172.21.15.71:0/2013150462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bdc0ffc20 0x7f8bdc100080 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009910 tx=0x7f8bcc009c20 comp rx=0 tx=0).stop 2024-04-23T02:19:28.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.475+0000 7f8be12b9700 1 -- 172.21.15.71:0/2013150462 shutdown_connections 2024-04-23T02:19:28.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.475+0000 7f8be12b9700 1 --2- 172.21.15.71:0/2013150462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bdc0ffc20 0x7f8bdc100080 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.475+0000 7f8be12b9700 1 --2- 172.21.15.71:0/2013150462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bdc105bd0 0x7f8bdc105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.475+0000 7f8be12b9700 1 -- 172.21.15.71:0/2013150462 >> 172.21.15.71:0/2013150462 conn(0x7f8bdc075670 msgr2=0x7f8bdc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:28.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.475+0000 7f8be12b9700 1 -- 172.21.15.71:0/2013150462 shutdown_connections 2024-04-23T02:19:28.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.475+0000 7f8be12b9700 1 -- 172.21.15.71:0/2013150462 wait complete. 2024-04-23T02:19:28.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.476+0000 7f8be12b9700 1 Processor -- start 2024-04-23T02:19:28.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.476+0000 7f8be12b9700 1 -- start start 2024-04-23T02:19:28.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.476+0000 7f8be12b9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bdc0ffc20 0x7f8bdc1976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:28.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.476+0000 7f8be12b9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bdc105bd0 0x7f8bdc197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:28.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.476+0000 7f8be12b9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc198220 con 0x7f8bdc0ffc20 2024-04-23T02:19:28.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.476+0000 7f8be12b9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc198360 con 0x7f8bdc105bd0 2024-04-23T02:19:28.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.476+0000 7f8bdaffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bdc0ffc20 0x7f8bdc1976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:28.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.476+0000 7f8bdaffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bdc0ffc20 0x7f8bdc1976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46498/0 (socket says 172.21.15.71:46498) 2024-04-23T02:19:28.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.476+0000 7f8bdaffd700 1 -- 172.21.15.71:0/2576495113 learned_addr learned my addr 172.21.15.71:0/2576495113 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:19:28.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.476+0000 7f8bda7fc700 1 --2- 172.21.15.71:0/2576495113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bdc105bd0 0x7f8bdc197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:28.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.477+0000 7f8bdaffd700 1 -- 172.21.15.71:0/2576495113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bdc105bd0 msgr2=0x7f8bdc197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:28.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.477+0000 7f8bdaffd700 1 --2- 172.21.15.71:0/2576495113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bdc105bd0 0x7f8bdc197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.477+0000 7f8bdaffd700 1 -- 172.21.15.71:0/2576495113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc0095f0 con 0x7f8bdc0ffc20 2024-04-23T02:19:28.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.477+0000 7f8bdaffd700 1 --2- 172.21.15.71:0/2576495113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bdc0ffc20 0x7f8bdc1976c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000d8d0 tx=0x7f8bd000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:28.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.477+0000 7f8bc3fff700 1 -- 172.21.15.71:0/2576495113 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bd00098a0 con 0x7f8bdc0ffc20 2024-04-23T02:19:28.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.477+0000 7f8bc3fff700 1 -- 172.21.15.71:0/2576495113 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8bd0010460 con 0x7f8bdc0ffc20 2024-04-23T02:19:28.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.477+0000 7f8be12b9700 1 -- 172.21.15.71:0/2576495113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bdc1a0730 con 0x7f8bdc0ffc20 2024-04-23T02:19:28.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.477+0000 7f8bc3fff700 1 -- 172.21.15.71:0/2576495113 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bd0009ea0 con 0x7f8bdc0ffc20 2024-04-23T02:19:28.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.477+0000 7f8be12b9700 1 -- 172.21.15.71:0/2576495113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bdc1a0bf0 con 0x7f8bdc0ffc20 2024-04-23T02:19:28.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.478+0000 7f8be12b9700 1 -- 172.21.15.71:0/2576495113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bdc1080d0 con 0x7f8bdc0ffc20 2024-04-23T02:19:28.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.478+0000 7f8bc3fff700 1 -- 172.21.15.71:0/2576495113 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8bd0009a00 con 0x7f8bdc0ffc20 2024-04-23T02:19:28.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.479+0000 7f8bc3fff700 1 --2- 172.21.15.71:0/2576495113 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8bc406cce0 0x7f8bc406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:28.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.479+0000 7f8bda7fc700 1 --2- 172.21.15.71:0/2576495113 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8bc406cce0 0x7f8bc406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:28.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.479+0000 7f8bc3fff700 1 -- 172.21.15.71:0/2576495113 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8bd008c3a0 con 0x7f8bdc0ffc20 2024-04-23T02:19:28.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.479+0000 7f8bda7fc700 1 --2- 172.21.15.71:0/2576495113 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8bc406cce0 0x7f8bc406f1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc068e30 tx=0x7f8bcc01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:28.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.483+0000 7f8bc3fff700 1 -- 172.21.15.71:0/2576495113 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8bd00564c0 con 0x7f8bdc0ffc20 2024-04-23T02:19:28.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:28 smithi192 ceph-mon[108617]: pgmap v156: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/366 objects degraded (15.301%) 2024-04-23T02:19:28.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:28 smithi192 ceph-mon[108617]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:19:28.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:28 smithi192 ceph-mon[108617]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:19:28.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:28 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3090271154' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:19:28.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.718+0000 7f8be12b9700 1 -- 172.21.15.71:0/2576495113 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bdc0611d0 con 0x7f8bc406cce0 2024-04-23T02:19:28.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.719+0000 7f8bc3fff700 1 -- 172.21.15.71:0/2576495113 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8bdc0611d0 con 0x7f8bc406cce0 2024-04-23T02:19:28.723 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:19:28.723 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:19:28.723 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:19:28.724 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:19:28.724 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:19:28.724 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:19:28.724 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:19:28.724 INFO:teuthology.orchestra.run.smithi071.stdout: "mon" 2024-04-23T02:19:28.724 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:19:28.724 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "7/23 daemons upgraded", 2024-04-23T02:19:28.724 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading osd daemons", 2024-04-23T02:19:28.724 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:19:28.724 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:19:28.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.722+0000 7f8be12b9700 1 -- 172.21.15.71:0/2576495113 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8bc406cce0 msgr2=0x7f8bc406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:28.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.722+0000 7f8be12b9700 1 --2- 172.21.15.71:0/2576495113 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8bc406cce0 0x7f8bc406f1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc068e30 tx=0x7f8bcc01a040 comp rx=0 tx=0).stop 2024-04-23T02:19:28.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.722+0000 7f8be12b9700 1 -- 172.21.15.71:0/2576495113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bdc0ffc20 msgr2=0x7f8bdc1976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:28.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.722+0000 7f8be12b9700 1 --2- 172.21.15.71:0/2576495113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bdc0ffc20 0x7f8bdc1976c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000d8d0 tx=0x7f8bd000dbe0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.722+0000 7f8be12b9700 1 -- 172.21.15.71:0/2576495113 shutdown_connections 2024-04-23T02:19:28.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.722+0000 7f8be12b9700 1 --2- 172.21.15.71:0/2576495113 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8bc406cce0 0x7f8bc406f1a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.722+0000 7f8be12b9700 1 --2- 172.21.15.71:0/2576495113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bdc0ffc20 0x7f8bdc1976c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.722+0000 7f8be12b9700 1 --2- 172.21.15.71:0/2576495113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bdc105bd0 0x7f8bdc197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.722+0000 7f8be12b9700 1 -- 172.21.15.71:0/2576495113 >> 172.21.15.71:0/2576495113 conn(0x7f8bdc075670 msgr2=0x7f8bdc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:28.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.722+0000 7f8be12b9700 1 -- 172.21.15.71:0/2576495113 shutdown_connections 2024-04-23T02:19:28.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.722+0000 7f8be12b9700 1 -- 172.21.15.71:0/2576495113 wait complete. 2024-04-23T02:19:28.743 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:28 smithi071 bash[163212]: a626640cb50f93112dda9ec0235872501bc3c7b079403669d06e6a4e9a26fddd 2024-04-23T02:19:28.743 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:28 smithi071 systemd[1]: Started Ceph osd.1 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:19:28.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.843+0000 7f44f9443700 1 -- 172.21.15.71:0/4275895502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f4105bd0 msgr2=0x7f44f4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:28.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.843+0000 7f44f9443700 1 --2- 172.21.15.71:0/4275895502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f4105bd0 0x7f44f4105fb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009910 tx=0x7f44e4009c20 comp rx=0 tx=0).stop 2024-04-23T02:19:28.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.843+0000 7f44f9443700 1 -- 172.21.15.71:0/4275895502 shutdown_connections 2024-04-23T02:19:28.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.843+0000 7f44f9443700 1 --2- 172.21.15.71:0/4275895502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f40ffc20 0x7f44f4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.843+0000 7f44f9443700 1 --2- 172.21.15.71:0/4275895502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f4105bd0 0x7f44f4105fb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.843+0000 7f44f9443700 1 -- 172.21.15.71:0/4275895502 >> 172.21.15.71:0/4275895502 conn(0x7f44f4075670 msgr2=0x7f44f4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:28.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.843+0000 7f44f9443700 1 -- 172.21.15.71:0/4275895502 shutdown_connections 2024-04-23T02:19:28.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.843+0000 7f44f9443700 1 -- 172.21.15.71:0/4275895502 wait complete. 2024-04-23T02:19:28.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.844+0000 7f44f9443700 1 Processor -- start 2024-04-23T02:19:28.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.844+0000 7f44f9443700 1 -- start start 2024-04-23T02:19:28.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.844+0000 7f44f9443700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f40ffc20 0x7f44f41976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:28.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.844+0000 7f44f9443700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f4105bd0 0x7f44f4197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:28.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.844+0000 7f44f9443700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f4198240 con 0x7f44f40ffc20 2024-04-23T02:19:28.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.844+0000 7f44f9443700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f4198380 con 0x7f44f4105bd0 2024-04-23T02:19:28.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.844+0000 7f44f2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f40ffc20 0x7f44f41976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:28.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.844+0000 7f44f2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f40ffc20 0x7f44f41976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46514/0 (socket says 172.21.15.71:46514) 2024-04-23T02:19:28.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.844+0000 7f44f2ffd700 1 -- 172.21.15.71:0/3587271889 learned_addr learned my addr 172.21.15.71:0/3587271889 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:19:28.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.845+0000 7f44f2ffd700 1 -- 172.21.15.71:0/3587271889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f4105bd0 msgr2=0x7f44f4197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:28.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.845+0000 7f44ebfff700 1 --2- 172.21.15.71:0/3587271889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f4105bd0 0x7f44f4197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:28.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.845+0000 7f44f2ffd700 1 --2- 172.21.15.71:0/3587271889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f4105bd0 0x7f44f4197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:28.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.845+0000 7f44f2ffd700 1 -- 172.21.15.71:0/3587271889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e40095f0 con 0x7f44f40ffc20 2024-04-23T02:19:28.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.845+0000 7f44ebfff700 1 --2- 172.21.15.71:0/3587271889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f4105bd0 0x7f44f4197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:19:28.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.845+0000 7f44f2ffd700 1 --2- 172.21.15.71:0/3587271889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f40ffc20 0x7f44f41976e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f44e40092e0 tx=0x7f44e400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:28.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.845+0000 7f44f0ff9700 1 -- 172.21.15.71:0/3587271889 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44e401d070 con 0x7f44f40ffc20 2024-04-23T02:19:28.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.845+0000 7f44f0ff9700 1 -- 172.21.15.71:0/3587271889 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f44e400f460 con 0x7f44f40ffc20 2024-04-23T02:19:28.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.845+0000 7f44f9443700 1 -- 172.21.15.71:0/3587271889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f41a0750 con 0x7f44f40ffc20 2024-04-23T02:19:28.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.845+0000 7f44f0ff9700 1 -- 172.21.15.71:0/3587271889 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44e4021640 con 0x7f44f40ffc20 2024-04-23T02:19:28.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.845+0000 7f44f9443700 1 -- 172.21.15.71:0/3587271889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f41a0b60 con 0x7f44f40ffc20 2024-04-23T02:19:28.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.846+0000 7f44f9443700 1 -- 172.21.15.71:0/3587271889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f41080f0 con 0x7f44f40ffc20 2024-04-23T02:19:28.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.847+0000 7f44f0ff9700 1 -- 172.21.15.71:0/3587271889 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f44e400fa00 con 0x7f44f40ffc20 2024-04-23T02:19:28.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.847+0000 7f44f0ff9700 1 --2- 172.21.15.71:0/3587271889 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f44e006cce0 0x7f44e006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:28.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.848+0000 7f44f0ff9700 1 -- 172.21.15.71:0/3587271889 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f44e408d870 con 0x7f44f40ffc20 2024-04-23T02:19:28.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.848+0000 7f44ebfff700 1 --2- 172.21.15.71:0/3587271889 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f44e006cce0 0x7f44e006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:28.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.848+0000 7f44ebfff700 1 --2- 172.21.15.71:0/3587271889 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f44e006cce0 0x7f44e006f1a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f44f4068e30 tx=0x7f44dc008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:28.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:28.853+0000 7f44f0ff9700 1 -- 172.21.15.71:0/3587271889 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f44e40578f0 con 0x7f44f40ffc20 2024-04-23T02:19:29.136 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:28 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-1[163250]: 2024-04-23T02:19:28.865+0000 7f2508c227c0 -1 Falling back to public interface 2024-04-23T02:19:29.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:29.145+0000 7f44f9443700 1 -- 172.21.15.71:0/3587271889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f44f404f2e0 con 0x7f44f40ffc20 2024-04-23T02:19:29.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:29.145+0000 7f44f0ff9700 1 -- 172.21.15.71:0/3587271889 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1350 (secure 0 0 0) 0x7f44e405af10 con 0x7f44f40ffc20 2024-04-23T02:19:29.149 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 56/366 objects degraded (15.301%), 16 pgs degraded 2024-04-23T02:19:29.149 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:19:29.150 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:19:29.150 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-23T02:19:29.150 INFO:teuthology.orchestra.run.smithi071.stdout: osd.1 (root=default,host=smithi071) is down 2024-04-23T02:19:29.150 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 56/366 objects degraded (15.301%), 16 pgs degraded 2024-04-23T02:19:29.150 INFO:teuthology.orchestra.run.smithi071.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-04-23T02:19:29.150 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-04-23T02:19:29.150 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-04-23T02:19:29.150 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-04-23T02:19:29.150 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-04-23T02:19:29.150 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-04-23T02:19:29.150 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-04-23T02:19:29.150 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-04-23T02:19:29.150 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-04-23T02:19:29.151 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-04-23T02:19:29.151 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-04-23T02:19:29.151 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.11 is active+undersized+degraded, acting [4,3] 2024-04-23T02:19:29.151 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.12 is active+undersized+degraded, acting [3,0] 2024-04-23T02:19:29.151 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-04-23T02:19:29.151 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.17 is active+undersized+degraded, acting [5,2] 2024-04-23T02:19:29.151 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-04-23T02:19:29.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:29.147+0000 7f44f9443700 1 -- 172.21.15.71:0/3587271889 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f44e006cce0 msgr2=0x7f44e006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:29.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:29.147+0000 7f44f9443700 1 --2- 172.21.15.71:0/3587271889 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f44e006cce0 0x7f44e006f1a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f44f4068e30 tx=0x7f44dc008040 comp rx=0 tx=0).stop 2024-04-23T02:19:29.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:29.147+0000 7f44f9443700 1 -- 172.21.15.71:0/3587271889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f40ffc20 msgr2=0x7f44f41976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:29.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:29.147+0000 7f44f9443700 1 --2- 172.21.15.71:0/3587271889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f40ffc20 0x7f44f41976e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f44e40092e0 tx=0x7f44e400be20 comp rx=0 tx=0).stop 2024-04-23T02:19:29.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:29.148+0000 7f44f9443700 1 -- 172.21.15.71:0/3587271889 shutdown_connections 2024-04-23T02:19:29.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:29.148+0000 7f44f9443700 1 --2- 172.21.15.71:0/3587271889 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f44e006cce0 0x7f44e006f1a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:29.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:29.148+0000 7f44f9443700 1 --2- 172.21.15.71:0/3587271889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f40ffc20 0x7f44f41976e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:29.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:29.148+0000 7f44f9443700 1 --2- 172.21.15.71:0/3587271889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f4105bd0 0x7f44f4197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:29.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:29.148+0000 7f44f9443700 1 -- 172.21.15.71:0/3587271889 >> 172.21.15.71:0/3587271889 conn(0x7f44f4075670 msgr2=0x7f44f40fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:29.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:29.148+0000 7f44f9443700 1 -- 172.21.15.71:0/3587271889 shutdown_connections 2024-04-23T02:19:29.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:29.148+0000 7f44f9443700 1 -- 172.21.15.71:0/3587271889 wait complete. 2024-04-23T02:19:29.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:29 smithi071 ceph-mon[144544]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:19:29.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:29 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2292008326' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:19:29.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:29 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:29.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:29 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:29.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:29 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:19:29.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:29 smithi192 ceph-mon[108617]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:19:29.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:29 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2292008326' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:19:29.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:29 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:29.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:29 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:29.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:29 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:19:30.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:30 smithi071 ceph-mon[144544]: pgmap v157: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/366 objects degraded (15.301%) 2024-04-23T02:19:30.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:30 smithi071 ceph-mon[144544]: from='client.34182 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:19:30.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:30 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3587271889' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:19:30.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:30 smithi192 ceph-mon[108617]: pgmap v157: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/366 objects degraded (15.301%) 2024-04-23T02:19:30.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:30 smithi192 ceph-mon[108617]: from='client.34182 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:19:30.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:30 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3587271889' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:19:32.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:32 smithi071 ceph-mon[144544]: pgmap v158: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/366 objects degraded (15.301%) 2024-04-23T02:19:32.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:32 smithi192 ceph-mon[108617]: pgmap v158: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/366 objects degraded (15.301%) 2024-04-23T02:19:33.694 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:33 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-1[163250]: 2024-04-23T02:19:33.383+0000 7f2508c227c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-04-23T02:19:33.993 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:33 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-1[163250]: 2024-04-23T02:19:33.729+0000 7f2508c227c0 -1 osd.1 53 log_to_monitors true 2024-04-23T02:19:34.493 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 02:19:34 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-1[163250]: 2024-04-23T02:19:34.156+0000 7f24fa07c700 -1 osd.1 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T02:19:34.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:34 smithi071 ceph-mon[144544]: pgmap v159: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/366 objects degraded (15.301%) 2024-04-23T02:19:34.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:34 smithi071 ceph-mon[144544]: from='osd.1 [v2:172.21.15.71:6810/2293979729,v1:172.21.15.71:6811/2293979729]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T02:19:34.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:34 smithi192 ceph-mon[108617]: pgmap v159: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/366 objects degraded (15.301%) 2024-04-23T02:19:34.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:34 smithi192 ceph-mon[108617]: from='osd.1 [v2:172.21.15.71:6810/2293979729,v1:172.21.15.71:6811/2293979729]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T02:19:35.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:35 smithi071 ceph-mon[144544]: from='osd.1 [v2:172.21.15.71:6810/2293979729,v1:172.21.15.71:6811/2293979729]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T02:19:35.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:35 smithi071 ceph-mon[144544]: osdmap e56: 6 total, 5 up, 6 in 2024-04-23T02:19:35.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:35 smithi071 ceph-mon[144544]: from='osd.1 [v2:172.21.15.71:6810/2293979729,v1:172.21.15.71:6811/2293979729]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-23T02:19:35.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:35.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:35.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:35.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:19:35.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:35.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:35.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:35 smithi192 ceph-mon[108617]: from='osd.1 [v2:172.21.15.71:6810/2293979729,v1:172.21.15.71:6811/2293979729]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T02:19:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:35 smithi192 ceph-mon[108617]: osdmap e56: 6 total, 5 up, 6 in 2024-04-23T02:19:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:35 smithi192 ceph-mon[108617]: from='osd.1 [v2:172.21.15.71:6810/2293979729,v1:172.21.15.71:6811/2293979729]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-23T02:19:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:19:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:35.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:36.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:36 smithi071 ceph-mon[144544]: pgmap v161: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/366 objects degraded (15.301%) 2024-04-23T02:19:36.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:36 smithi071 ceph-mon[144544]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-23T02:19:36.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:36 smithi071 ceph-mon[144544]: osd.1 [v2:172.21.15.71:6810/2293979729,v1:172.21.15.71:6811/2293979729] boot 2024-04-23T02:19:36.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:36 smithi071 ceph-mon[144544]: osdmap e57: 6 total, 6 up, 6 in 2024-04-23T02:19:36.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:36 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T02:19:36.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:36 smithi071 ceph-mon[144544]: osdmap e58: 6 total, 6 up, 6 in 2024-04-23T02:19:36.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:36 smithi192 ceph-mon[108617]: pgmap v161: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/366 objects degraded (15.301%) 2024-04-23T02:19:36.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:36 smithi192 ceph-mon[108617]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-23T02:19:36.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:36 smithi192 ceph-mon[108617]: osd.1 [v2:172.21.15.71:6810/2293979729,v1:172.21.15.71:6811/2293979729] boot 2024-04-23T02:19:36.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:36 smithi192 ceph-mon[108617]: osdmap e57: 6 total, 6 up, 6 in 2024-04-23T02:19:36.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:36 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T02:19:36.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:36 smithi192 ceph-mon[108617]: osdmap e58: 6 total, 6 up, 6 in 2024-04-23T02:19:37.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:37 smithi071 ceph-mon[144544]: Health check update: Degraded data redundancy: 23/366 objects degraded (6.284%), 7 pgs degraded (PG_DEGRADED) 2024-04-23T02:19:37.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:37 smithi192 ceph-mon[108617]: Health check update: Degraded data redundancy: 23/366 objects degraded (6.284%), 7 pgs degraded (PG_DEGRADED) 2024-04-23T02:19:38.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:38 smithi071 ceph-mon[144544]: pgmap v164: 65 pgs: 20 peering, 7 active+undersized+degraded, 5 active+undersized, 33 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s; 23/366 objects degraded (6.284%) 2024-04-23T02:19:38.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:38 smithi192 ceph-mon[108617]: pgmap v164: 65 pgs: 20 peering, 7 active+undersized+degraded, 5 active+undersized, 33 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s; 23/366 objects degraded (6.284%) 2024-04-23T02:19:40.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:39 smithi071 ceph-mon[144544]: pgmap v165: 65 pgs: 20 peering, 7 active+undersized+degraded, 5 active+undersized, 33 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s; 23/366 objects degraded (6.284%) 2024-04-23T02:19:40.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:39 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:40.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:39 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:40.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:39 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:19:40.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:39 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:19:40.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:39 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:40.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:39 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:19:40.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:39 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:19:40.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:39 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:19:40.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:39 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:19:40.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:39 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-23T02:19:40.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:39 smithi071 ceph-mon[144544]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-23T02:19:40.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:39 smithi071 ceph-mon[144544]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-23T02:19:40.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:39 smithi192 ceph-mon[108617]: pgmap v165: 65 pgs: 20 peering, 7 active+undersized+degraded, 5 active+undersized, 33 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s; 23/366 objects degraded (6.284%) 2024-04-23T02:19:40.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:39 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:40.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:39 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:40.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:39 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:19:40.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:39 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:19:40.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:39 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:40.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:39 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:19:40.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:39 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:19:40.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:39 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:19:40.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:39 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:19:40.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:39 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-23T02:19:40.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:39 smithi192 ceph-mon[108617]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-23T02:19:40.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:39 smithi192 ceph-mon[108617]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-23T02:19:41.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:40 smithi192 ceph-mon[108617]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/366 objects degraded (6.284%), 7 pgs degraded) 2024-04-23T02:19:41.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:40 smithi071 ceph-mon[144544]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/366 objects degraded (6.284%), 7 pgs degraded) 2024-04-23T02:19:42.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:41 smithi071 ceph-mon[144544]: pgmap v166: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T02:19:42.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:41 smithi192 ceph-mon[108617]: pgmap v166: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T02:19:44.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:43 smithi071 ceph-mon[144544]: pgmap v167: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-23T02:19:44.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:43 smithi192 ceph-mon[108617]: pgmap v167: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-23T02:19:46.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:45 smithi192 ceph-mon[108617]: pgmap v168: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-04-23T02:19:46.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:45 smithi071 ceph-mon[144544]: pgmap v168: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-04-23T02:19:48.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:47 smithi071 ceph-mon[144544]: pgmap v169: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-04-23T02:19:48.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:47 smithi192 ceph-mon[108617]: pgmap v169: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-04-23T02:19:50.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:49 smithi071 ceph-mon[144544]: pgmap v170: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:19:50.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:50.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:19:50.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:50.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:49 smithi192 ceph-mon[108617]: pgmap v170: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:19:50.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:50.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:19:50.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:52.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:51 smithi071 ceph-mon[144544]: pgmap v171: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:19:52.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:51 smithi192 ceph-mon[108617]: pgmap v171: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:19:54.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:53 smithi071 ceph-mon[144544]: pgmap v172: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:19:54.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:53 smithi192 ceph-mon[108617]: pgmap v172: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:19:55.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:54 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-23T02:19:55.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:54 smithi071 ceph-mon[144544]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-23T02:19:55.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:54 smithi071 ceph-mon[144544]: Upgrade: osd.2 is safe to restart 2024-04-23T02:19:55.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:54 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-23T02:19:55.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:54 smithi192 ceph-mon[108617]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-23T02:19:55.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:54 smithi192 ceph-mon[108617]: Upgrade: osd.2 is safe to restart 2024-04-23T02:19:55.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:55 smithi071 ceph-mon[144544]: pgmap v173: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:19:55.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:55 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:55.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:55 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T02:19:55.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:55 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:19:56.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:55 smithi192 ceph-mon[108617]: pgmap v173: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:19:56.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:55 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:19:56.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:55 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T02:19:56.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:55 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:19:57.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:56 smithi071 ceph-mon[144544]: Upgrade: Updating osd.2 2024-04-23T02:19:57.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:56 smithi071 ceph-mon[144544]: Deploying daemon osd.2 on smithi071 2024-04-23T02:19:57.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:56 smithi192 ceph-mon[108617]: Upgrade: Updating osd.2 2024-04-23T02:19:57.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:56 smithi192 ceph-mon[108617]: Deploying daemon osd.2 on smithi071 2024-04-23T02:19:58.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:19:57 smithi192 ceph-mon[108617]: pgmap v174: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:19:58.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:19:57 smithi071 ceph-mon[144544]: pgmap v174: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:19:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.265+0000 7f6dff25f700 1 -- 172.21.15.71:0/2169398165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6df8101850 msgr2=0x7f6df81058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.265+0000 7f6dff25f700 1 --2- 172.21.15.71:0/2169398165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6df8101850 0x7f6df81058a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009910 tx=0x7f6de4009c20 comp rx=0 tx=0).stop 2024-04-23T02:19:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.265+0000 7f6dff25f700 1 -- 172.21.15.71:0/2169398165 shutdown_connections 2024-04-23T02:19:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.265+0000 7f6dff25f700 1 --2- 172.21.15.71:0/2169398165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6df8101850 0x7f6df81058a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.265+0000 7f6dff25f700 1 --2- 172.21.15.71:0/2169398165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6df8100ea0 0x7f6df8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.265+0000 7f6dff25f700 1 -- 172.21.15.71:0/2169398165 >> 172.21.15.71:0/2169398165 conn(0x7f6df80fc750 msgr2=0x7f6df80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.265+0000 7f6dff25f700 1 -- 172.21.15.71:0/2169398165 shutdown_connections 2024-04-23T02:19:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.265+0000 7f6dff25f700 1 -- 172.21.15.71:0/2169398165 wait complete. 2024-04-23T02:19:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.265+0000 7f6dff25f700 1 Processor -- start 2024-04-23T02:19:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.265+0000 7f6dff25f700 1 -- start start 2024-04-23T02:19:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.266+0000 7f6dff25f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6df8100ea0 0x7f6df8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.266+0000 7f6dff25f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6df8101850 0x7f6df80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:59.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.266+0000 7f6dff25f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df8073b10 con 0x7f6df8100ea0 2024-04-23T02:19:59.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.266+0000 7f6dff25f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df8073c50 con 0x7f6df8101850 2024-04-23T02:19:59.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.266+0000 7f6dfcffb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6df8100ea0 0x7f6df8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:59.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.266+0000 7f6dfcffb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6df8100ea0 0x7f6df8073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47058/0 (socket says 172.21.15.71:47058) 2024-04-23T02:19:59.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.266+0000 7f6dfcffb700 1 -- 172.21.15.71:0/1933823904 learned_addr learned my addr 172.21.15.71:0/1933823904 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:19:59.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.266+0000 7f6dfcffb700 1 -- 172.21.15.71:0/1933823904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6df8101850 msgr2=0x7f6df80735d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T02:19:59.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.266+0000 7f6dfcffb700 1 --2- 172.21.15.71:0/1933823904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6df8101850 0x7f6df80735d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:59.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.266+0000 7f6dfcffb700 1 -- 172.21.15.71:0/1933823904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de40095f0 con 0x7f6df8100ea0 2024-04-23T02:19:59.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.266+0000 7f6dfcffb700 1 --2- 172.21.15.71:0/1933823904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6df8100ea0 0x7f6df8073090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00e940 tx=0x7f6dec00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:59.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.266+0000 7f6df5ffb700 1 -- 172.21.15.71:0/1933823904 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dec0097b0 con 0x7f6df8100ea0 2024-04-23T02:19:59.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.266+0000 7f6dff25f700 1 -- 172.21.15.71:0/1933823904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df81a2a40 con 0x7f6df8100ea0 2024-04-23T02:19:59.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.266+0000 7f6df5ffb700 1 -- 172.21.15.71:0/1933823904 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6dec004830 con 0x7f6df8100ea0 2024-04-23T02:19:59.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.266+0000 7f6df5ffb700 1 -- 172.21.15.71:0/1933823904 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dec0055b0 con 0x7f6df8100ea0 2024-04-23T02:19:59.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.267+0000 7f6dff25f700 1 -- 172.21.15.71:0/1933823904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df81a2e50 con 0x7f6df8100ea0 2024-04-23T02:19:59.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.268+0000 7f6df5ffb700 1 -- 172.21.15.71:0/1933823904 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6dec00cba0 con 0x7f6df8100ea0 2024-04-23T02:19:59.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.268+0000 7f6dff25f700 1 -- 172.21.15.71:0/1933823904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df81092a0 con 0x7f6df8100ea0 2024-04-23T02:19:59.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.268+0000 7f6df5ffb700 1 --2- 172.21.15.71:0/1933823904 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6de806cce0 0x7f6de806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:59.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.268+0000 7f6df5ffb700 1 -- 172.21.15.71:0/1933823904 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6dec014070 con 0x7f6df8100ea0 2024-04-23T02:19:59.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.268+0000 7f6df7fff700 1 --2- 172.21.15.71:0/1933823904 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6de806cce0 0x7f6de806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:59.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.269+0000 7f6df7fff700 1 --2- 172.21.15.71:0/1933823904 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6de806cce0 0x7f6de806f1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6de4005cb0 tx=0x7f6de4005c40 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:59.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.273+0000 7f6df5ffb700 1 -- 172.21.15.71:0/1933823904 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6dec056990 con 0x7f6df8100ea0 2024-04-23T02:19:59.494 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:19:59 smithi071 systemd[1]: Stopping Ceph osd.2 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T02:19:59.494 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:19:59 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-2[82024]: 2024-04-23T02:19:59.366+0000 7fa47df8f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T02:19:59.494 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:19:59 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-2[82024]: 2024-04-23T02:19:59.366+0000 7fa47df8f700 -1 osd.2 58 *** Got signal Terminated *** 2024-04-23T02:19:59.494 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:19:59 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-2[82024]: 2024-04-23T02:19:59.366+0000 7fa47df8f700 -1 osd.2 58 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T02:19:59.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.489+0000 7f6dff25f700 1 -- 172.21.15.71:0/1933823904 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6df80611d0 con 0x7f6de806cce0 2024-04-23T02:19:59.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.490+0000 7f6df5ffb700 1 -- 172.21.15.71:0/1933823904 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f6df80611d0 con 0x7f6de806cce0 2024-04-23T02:19:59.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.492+0000 7f6dff25f700 1 -- 172.21.15.71:0/1933823904 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6de806cce0 msgr2=0x7f6de806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:59.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.492+0000 7f6dff25f700 1 --2- 172.21.15.71:0/1933823904 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6de806cce0 0x7f6de806f1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6de4005cb0 tx=0x7f6de4005c40 comp rx=0 tx=0).stop 2024-04-23T02:19:59.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.492+0000 7f6dff25f700 1 -- 172.21.15.71:0/1933823904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6df8100ea0 msgr2=0x7f6df8073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:59.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.492+0000 7f6dff25f700 1 --2- 172.21.15.71:0/1933823904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6df8100ea0 0x7f6df8073090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00e940 tx=0x7f6dec00ed00 comp rx=0 tx=0).stop 2024-04-23T02:19:59.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.492+0000 7f6dff25f700 1 -- 172.21.15.71:0/1933823904 shutdown_connections 2024-04-23T02:19:59.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.492+0000 7f6dff25f700 1 --2- 172.21.15.71:0/1933823904 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6de806cce0 0x7f6de806f1a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:59.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.492+0000 7f6dff25f700 1 --2- 172.21.15.71:0/1933823904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6df8100ea0 0x7f6df8073090 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:59.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.492+0000 7f6dff25f700 1 --2- 172.21.15.71:0/1933823904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6df8101850 0x7f6df80735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:59.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.492+0000 7f6dff25f700 1 -- 172.21.15.71:0/1933823904 >> 172.21.15.71:0/1933823904 conn(0x7f6df80fc750 msgr2=0x7f6df80fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:59.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.492+0000 7f6dff25f700 1 -- 172.21.15.71:0/1933823904 shutdown_connections 2024-04-23T02:19:59.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.493+0000 7f6dff25f700 1 -- 172.21.15.71:0/1933823904 wait complete. 2024-04-23T02:19:59.508 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:19:59.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.597+0000 7f13270b9700 1 -- 172.21.15.71:0/436511749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13200ff5a0 msgr2=0x7f13200ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:59.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.597+0000 7f13270b9700 1 --2- 172.21.15.71:0/436511749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13200ff5a0 0x7f13200ff980 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1314009910 tx=0x7f1314009c20 comp rx=0 tx=0).stop 2024-04-23T02:19:59.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.597+0000 7f13270b9700 1 -- 172.21.15.71:0/436511749 shutdown_connections 2024-04-23T02:19:59.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.597+0000 7f13270b9700 1 --2- 172.21.15.71:0/436511749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13200fff50 0x7f1320105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:59.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.597+0000 7f13270b9700 1 --2- 172.21.15.71:0/436511749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13200ff5a0 0x7f13200ff980 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:59.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.597+0000 7f13270b9700 1 -- 172.21.15.71:0/436511749 >> 172.21.15.71:0/436511749 conn(0x7f1320074e30 msgr2=0x7f1320075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:59.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.597+0000 7f13270b9700 1 -- 172.21.15.71:0/436511749 shutdown_connections 2024-04-23T02:19:59.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.597+0000 7f13270b9700 1 -- 172.21.15.71:0/436511749 wait complete. 2024-04-23T02:19:59.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.598+0000 7f13270b9700 1 Processor -- start 2024-04-23T02:19:59.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.598+0000 7f13270b9700 1 -- start start 2024-04-23T02:19:59.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.598+0000 7f13270b9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13200ff5a0 0x7f132019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:59.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.598+0000 7f13270b9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13200fff50 0x7f132019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:59.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.598+0000 7f13270b9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132019f360 con 0x7f13200fff50 2024-04-23T02:19:59.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.598+0000 7f13270b9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1320198810 con 0x7f13200ff5a0 2024-04-23T02:19:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.598+0000 7f131ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13200fff50 0x7f132019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.598+0000 7f131ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13200fff50 0x7f132019ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47076/0 (socket says 172.21.15.71:47076) 2024-04-23T02:19:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.598+0000 7f131ffff700 1 -- 172.21.15.71:0/2553377929 learned_addr learned my addr 172.21.15.71:0/2553377929 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:19:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.598+0000 7f1324e55700 1 --2- 172.21.15.71:0/2553377929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13200ff5a0 0x7f132019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.598+0000 7f131ffff700 1 -- 172.21.15.71:0/2553377929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13200ff5a0 msgr2=0x7f132019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.598+0000 7f131ffff700 1 --2- 172.21.15.71:0/2553377929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13200ff5a0 0x7f132019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.598+0000 7f131ffff700 1 -- 172.21.15.71:0/2553377929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13140095f0 con 0x7f13200fff50 2024-04-23T02:19:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.598+0000 7f131ffff700 1 --2- 172.21.15.71:0/2553377929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13200fff50 0x7f132019ecd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f130c00d900 tx=0x7f130c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.599+0000 7f131dffb700 1 -- 172.21.15.71:0/2553377929 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130c009900 con 0x7f13200fff50 2024-04-23T02:19:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.599+0000 7f131dffb700 1 -- 172.21.15.71:0/2553377929 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f130c010460 con 0x7f13200fff50 2024-04-23T02:19:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.599+0000 7f13270b9700 1 -- 172.21.15.71:0/2553377929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1320198af0 con 0x7f13200fff50 2024-04-23T02:19:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.599+0000 7f131dffb700 1 -- 172.21.15.71:0/2553377929 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130c00f5f0 con 0x7f13200fff50 2024-04-23T02:19:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.599+0000 7f13270b9700 1 -- 172.21.15.71:0/2553377929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1320199040 con 0x7f13200fff50 2024-04-23T02:19:59.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.600+0000 7f13270b9700 1 -- 172.21.15.71:0/2553377929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1320104fc0 con 0x7f13200fff50 2024-04-23T02:19:59.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.600+0000 7f131dffb700 1 -- 172.21.15.71:0/2553377929 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f130c0105d0 con 0x7f13200fff50 2024-04-23T02:19:59.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.600+0000 7f131dffb700 1 --2- 172.21.15.71:0/2553377929 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f131006cce0 0x7f131006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:59.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.600+0000 7f131dffb700 1 -- 172.21.15.71:0/2553377929 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f130c08c310 con 0x7f13200fff50 2024-04-23T02:19:59.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.601+0000 7f1324e55700 1 --2- 172.21.15.71:0/2553377929 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f131006cce0 0x7f131006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:59.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.601+0000 7f1324e55700 1 --2- 172.21.15.71:0/2553377929 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f131006cce0 0x7f131006f1a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1314006010 tx=0x7f131401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:59.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.605+0000 7f131dffb700 1 -- 172.21.15.71:0/2553377929 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f130c059ec0 con 0x7f13200fff50 2024-04-23T02:19:59.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.819+0000 7f13270b9700 1 -- 172.21.15.71:0/2553377929 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1320199260 con 0x7f131006cce0 2024-04-23T02:19:59.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.821+0000 7f131dffb700 1 -- 172.21.15.71:0/2553377929 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1320199260 con 0x7f131006cce0 2024-04-23T02:19:59.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.823+0000 7f13270b9700 1 -- 172.21.15.71:0/2553377929 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f131006cce0 msgr2=0x7f131006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:59.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.823+0000 7f13270b9700 1 --2- 172.21.15.71:0/2553377929 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f131006cce0 0x7f131006f1a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1314006010 tx=0x7f131401a040 comp rx=0 tx=0).stop 2024-04-23T02:19:59.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.823+0000 7f13270b9700 1 -- 172.21.15.71:0/2553377929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13200fff50 msgr2=0x7f132019ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:59.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.823+0000 7f13270b9700 1 --2- 172.21.15.71:0/2553377929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13200fff50 0x7f132019ecd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f130c00d900 tx=0x7f130c00dcc0 comp rx=0 tx=0).stop 2024-04-23T02:19:59.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.823+0000 7f13270b9700 1 -- 172.21.15.71:0/2553377929 shutdown_connections 2024-04-23T02:19:59.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.823+0000 7f13270b9700 1 --2- 172.21.15.71:0/2553377929 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f131006cce0 0x7f131006f1a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:59.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.823+0000 7f13270b9700 1 --2- 172.21.15.71:0/2553377929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13200ff5a0 0x7f132019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:59.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.823+0000 7f13270b9700 1 --2- 172.21.15.71:0/2553377929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13200fff50 0x7f132019ecd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:59.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.823+0000 7f13270b9700 1 -- 172.21.15.71:0/2553377929 >> 172.21.15.71:0/2553377929 conn(0x7f1320074e30 msgr2=0x7f13200fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:59.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.823+0000 7f13270b9700 1 -- 172.21.15.71:0/2553377929 shutdown_connections 2024-04-23T02:19:59.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.823+0000 7f13270b9700 1 -- 172.21.15.71:0/2553377929 wait complete. 2024-04-23T02:19:59.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.926+0000 7fd833b0a700 1 -- 172.21.15.71:0/12850440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd82c105bf0 msgr2=0x7fd82c105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:59.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.926+0000 7fd833b0a700 1 --2- 172.21.15.71:0/12850440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd82c105bf0 0x7fd82c105fd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd81c009910 tx=0x7fd81c009c20 comp rx=0 tx=0).stop 2024-04-23T02:19:59.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.926+0000 7fd833b0a700 1 -- 172.21.15.71:0/12850440 shutdown_connections 2024-04-23T02:19:59.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.926+0000 7fd833b0a700 1 --2- 172.21.15.71:0/12850440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd82c0ffb40 0x7fd82c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:59.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.926+0000 7fd833b0a700 1 --2- 172.21.15.71:0/12850440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd82c105bf0 0x7fd82c105fd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:59.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.926+0000 7fd833b0a700 1 -- 172.21.15.71:0/12850440 >> 172.21.15.71:0/12850440 conn(0x7fd82c0fb610 msgr2=0x7fd82c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:19:59.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.926+0000 7fd833b0a700 1 -- 172.21.15.71:0/12850440 shutdown_connections 2024-04-23T02:19:59.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.926+0000 7fd833b0a700 1 -- 172.21.15.71:0/12850440 wait complete. 2024-04-23T02:19:59.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.927+0000 7fd833b0a700 1 Processor -- start 2024-04-23T02:19:59.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.927+0000 7fd833b0a700 1 -- start start 2024-04-23T02:19:59.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.927+0000 7fd833b0a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd82c0ffb40 0x7fd82c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:59.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.927+0000 7fd833b0a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd82c105bf0 0x7fd82c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:59.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.927+0000 7fd833b0a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c198110 con 0x7fd82c0ffb40 2024-04-23T02:19:59.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.927+0000 7fd833b0a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c198250 con 0x7fd82c105bf0 2024-04-23T02:19:59.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.927+0000 7fd8318a6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd82c0ffb40 0x7fd82c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:59.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.927+0000 7fd8318a6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd82c0ffb40 0x7fd82c1975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47092/0 (socket says 172.21.15.71:47092) 2024-04-23T02:19:59.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.927+0000 7fd8318a6700 1 -- 172.21.15.71:0/2623639647 learned_addr learned my addr 172.21.15.71:0/2623639647 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:19:59.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.927+0000 7fd8318a6700 1 -- 172.21.15.71:0/2623639647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd82c105bf0 msgr2=0x7fd82c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:19:59.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.927+0000 7fd8318a6700 1 --2- 172.21.15.71:0/2623639647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd82c105bf0 0x7fd82c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:19:59.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.927+0000 7fd8318a6700 1 -- 172.21.15.71:0/2623639647 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c0095f0 con 0x7fd82c0ffb40 2024-04-23T02:19:59.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.927+0000 7fd8318a6700 1 --2- 172.21.15.71:0/2623639647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd82c0ffb40 0x7fd82c1975b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd81c006010 tx=0x7fd81c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:59.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.927+0000 7fd822ffd700 1 -- 172.21.15.71:0/2623639647 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd81c01d070 con 0x7fd82c0ffb40 2024-04-23T02:19:59.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.927+0000 7fd833b0a700 1 -- 172.21.15.71:0/2623639647 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82c1a0620 con 0x7fd82c0ffb40 2024-04-23T02:19:59.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.927+0000 7fd822ffd700 1 -- 172.21.15.71:0/2623639647 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd81c00f460 con 0x7fd82c0ffb40 2024-04-23T02:19:59.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.928+0000 7fd822ffd700 1 -- 172.21.15.71:0/2623639647 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd81c021640 con 0x7fd82c0ffb40 2024-04-23T02:19:59.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.928+0000 7fd833b0a700 1 -- 172.21.15.71:0/2623639647 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82c1a0a30 con 0x7fd82c0ffb40 2024-04-23T02:19:59.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.929+0000 7fd833b0a700 1 -- 172.21.15.71:0/2623639647 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd82c108020 con 0x7fd82c0ffb40 2024-04-23T02:19:59.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.929+0000 7fd822ffd700 1 -- 172.21.15.71:0/2623639647 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd81c00fa20 con 0x7fd82c0ffb40 2024-04-23T02:19:59.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.929+0000 7fd822ffd700 1 --2- 172.21.15.71:0/2623639647 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd81806cce0 0x7fd81806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:19:59.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.929+0000 7fd822ffd700 1 -- 172.21.15.71:0/2623639647 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd81c08d700 con 0x7fd82c0ffb40 2024-04-23T02:19:59.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.929+0000 7fd8310a5700 1 --2- 172.21.15.71:0/2623639647 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd81806cce0 0x7fd81806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:19:59.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.930+0000 7fd8310a5700 1 --2- 172.21.15.71:0/2623639647 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd81806cce0 0x7fd81806f1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0696c0 tx=0x7fd828005c80 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:19:59.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:19:59.934+0000 7fd822ffd700 1 -- 172.21.15.71:0/2623639647 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd81c0578a0 con 0x7fd82c0ffb40 2024-04-23T02:20:00.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.149+0000 7fd833b0a700 1 -- 172.21.15.71:0/2623639647 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd82c02ccc0 con 0x7fd81806cce0 2024-04-23T02:20:00.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.159+0000 7fd822ffd700 1 -- 172.21.15.71:0/2623639647 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fd82c02ccc0 con 0x7fd81806cce0 2024-04-23T02:20:00.164 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:20:00.164 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (3m) 25s ago 14m 26.3M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:20:00.164 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (15m) 25s ago 15m 8204k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:20:00.164 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (14m) 86s ago 14m 8036k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:20:00.164 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (95s) 25s ago 15m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:20:00.164 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (90s) 86s ago 14m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:20:00.164 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (3m) 25s ago 14m 78.5M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:20:00.164 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (9m) 25s ago 9m 64.9M - 18.2.1 5be31c24972a a515de2a841c 2024-04-23T02:20:00.165 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (8m) 25s ago 8m 133M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:20:00.165 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (8m) 86s ago 8m 68.1M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:20:00.165 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (9m) 86s ago 9m 122M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:20:00.165 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (5m) 25s ago 16m 506M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:20:00.165 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (4m) 86s ago 13m 451M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:20:00.165 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (2m) 25s ago 16m 46.1M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:20:00.165 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (109s) 86s ago 13m 31.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:20:00.165 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (4m) 25s ago 15m 18.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:20:00.165 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (4m) 86s ago 13m 17.8M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:20:00.165 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (69s) 25s ago 13m 84.9M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:20:00.165 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (32s) 25s ago 12m 13.9M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:20:00.165 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (12m) 25s ago 12m 165M 1227M 18.2.1 5be31c24972a 12380b6a15a2 2024-04-23T02:20:00.165 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (11m) 86s ago 11m 243M 2251M 18.2.1 5be31c24972a d5f587ee3a22 2024-04-23T02:20:00.165 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (11m) 86s ago 11m 219M 2251M 18.2.1 5be31c24972a 958d046005f0 2024-04-23T02:20:00.166 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (10m) 86s ago 10m 199M 2251M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:20:00.166 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (4m) 25s ago 14m 65.6M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:20:00.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.161+0000 7fd833b0a700 1 -- 172.21.15.71:0/2623639647 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd81806cce0 msgr2=0x7fd81806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:00.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.161+0000 7fd833b0a700 1 --2- 172.21.15.71:0/2623639647 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd81806cce0 0x7fd81806f1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0696c0 tx=0x7fd828005c80 comp rx=0 tx=0).stop 2024-04-23T02:20:00.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.161+0000 7fd833b0a700 1 -- 172.21.15.71:0/2623639647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd82c0ffb40 msgr2=0x7fd82c1975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:00.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.161+0000 7fd833b0a700 1 --2- 172.21.15.71:0/2623639647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd82c0ffb40 0x7fd82c1975b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd81c006010 tx=0x7fd81c00be20 comp rx=0 tx=0).stop 2024-04-23T02:20:00.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.162+0000 7fd833b0a700 1 -- 172.21.15.71:0/2623639647 shutdown_connections 2024-04-23T02:20:00.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.162+0000 7fd833b0a700 1 --2- 172.21.15.71:0/2623639647 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd81806cce0 0x7fd81806f1a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:00.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.162+0000 7fd833b0a700 1 --2- 172.21.15.71:0/2623639647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd82c0ffb40 0x7fd82c1975b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:00.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.162+0000 7fd833b0a700 1 --2- 172.21.15.71:0/2623639647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd82c105bf0 0x7fd82c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:00.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.162+0000 7fd833b0a700 1 -- 172.21.15.71:0/2623639647 >> 172.21.15.71:0/2623639647 conn(0x7fd82c0fb610 msgr2=0x7fd82c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:00.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.162+0000 7fd833b0a700 1 -- 172.21.15.71:0/2623639647 shutdown_connections 2024-04-23T02:20:00.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.162+0000 7fd833b0a700 1 -- 172.21.15.71:0/2623639647 wait complete. 2024-04-23T02:20:00.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.265+0000 7f399ad51700 1 -- 172.21.15.71:0/2160962244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39940ffc20 msgr2=0x7f3994100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:00.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.265+0000 7f399ad51700 1 --2- 172.21.15.71:0/2160962244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39940ffc20 0x7f3994100080 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3980009910 tx=0x7f3980009c20 comp rx=0 tx=0).stop 2024-04-23T02:20:00.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.265+0000 7f399ad51700 1 -- 172.21.15.71:0/2160962244 shutdown_connections 2024-04-23T02:20:00.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.265+0000 7f399ad51700 1 --2- 172.21.15.71:0/2160962244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39940ffc20 0x7f3994100080 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:00.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.265+0000 7f399ad51700 1 --2- 172.21.15.71:0/2160962244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3994105bd0 0x7f3994105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:00.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.265+0000 7f399ad51700 1 -- 172.21.15.71:0/2160962244 >> 172.21.15.71:0/2160962244 conn(0x7f3994075670 msgr2=0x7f3994075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:00.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.265+0000 7f399ad51700 1 -- 172.21.15.71:0/2160962244 shutdown_connections 2024-04-23T02:20:00.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.265+0000 7f399ad51700 1 -- 172.21.15.71:0/2160962244 wait complete. 2024-04-23T02:20:00.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.266+0000 7f399ad51700 1 Processor -- start 2024-04-23T02:20:00.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.266+0000 7f399ad51700 1 -- start start 2024-04-23T02:20:00.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.266+0000 7f399ad51700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39940ffc20 0x7f3994197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:00.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.266+0000 7f399ad51700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3994105bd0 0x7f3994197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:00.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.266+0000 7f399ad51700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39941982a0 con 0x7f3994105bd0 2024-04-23T02:20:00.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.266+0000 7f399ad51700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3994198410 con 0x7f39940ffc20 2024-04-23T02:20:00.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.266+0000 7f3993fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3994105bd0 0x7f3994197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:00.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.266+0000 7f3993fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3994105bd0 0x7f3994197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47100/0 (socket says 172.21.15.71:47100) 2024-04-23T02:20:00.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.266+0000 7f3993fff700 1 -- 172.21.15.71:0/204633892 learned_addr learned my addr 172.21.15.71:0/204633892 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:20:00.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.266+0000 7f3998aed700 1 --2- 172.21.15.71:0/204633892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39940ffc20 0x7f3994197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:00.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.266+0000 7f3993fff700 1 -- 172.21.15.71:0/204633892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39940ffc20 msgr2=0x7f3994197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:00.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.266+0000 7f3993fff700 1 --2- 172.21.15.71:0/204633892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39940ffc20 0x7f3994197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:00.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.266+0000 7f3993fff700 1 -- 172.21.15.71:0/204633892 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39800095f0 con 0x7f3994105bd0 2024-04-23T02:20:00.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.266+0000 7f3998aed700 1 --2- 172.21.15.71:0/204633892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39940ffc20 0x7f3994197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:20:00.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.266+0000 7f3993fff700 1 --2- 172.21.15.71:0/204633892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3994105bd0 0x7f3994197c80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3980006010 tx=0x7f398000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:00.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.267+0000 7f3991ffb700 1 -- 172.21.15.71:0/204633892 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f398001d070 con 0x7f3994105bd0 2024-04-23T02:20:00.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.267+0000 7f399ad51700 1 -- 172.21.15.71:0/204633892 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39941a07b0 con 0x7f3994105bd0 2024-04-23T02:20:00.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.267+0000 7f3991ffb700 1 -- 172.21.15.71:0/204633892 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f398000f460 con 0x7f3994105bd0 2024-04-23T02:20:00.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.267+0000 7f3991ffb700 1 -- 172.21.15.71:0/204633892 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3980021640 con 0x7f3994105bd0 2024-04-23T02:20:00.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.267+0000 7f399ad51700 1 -- 172.21.15.71:0/204633892 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39941a0c20 con 0x7f3994105bd0 2024-04-23T02:20:00.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.268+0000 7f399ad51700 1 -- 172.21.15.71:0/204633892 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39941080f0 con 0x7f3994105bd0 2024-04-23T02:20:00.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.268+0000 7f3991ffb700 1 -- 172.21.15.71:0/204633892 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f398000f9e0 con 0x7f3994105bd0 2024-04-23T02:20:00.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.269+0000 7f3991ffb700 1 --2- 172.21.15.71:0/204633892 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f398406cce0 0x7f398406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:00.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.269+0000 7f3991ffb700 1 -- 172.21.15.71:0/204633892 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f398008d8f0 con 0x7f3994105bd0 2024-04-23T02:20:00.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.269+0000 7f3998aed700 1 --2- 172.21.15.71:0/204633892 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f398406cce0 0x7f398406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:00.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.269+0000 7f3998aed700 1 --2- 172.21.15.71:0/204633892 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f398406cce0 0x7f398406f1a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3988007900 tx=0x7f3988008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:00.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.273+0000 7f3991ffb700 1 -- 172.21.15.71:0/204633892 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3980057970 con 0x7f3994105bd0 2024-04-23T02:20:00.290 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:00 smithi071 ceph-mon[144544]: pgmap v175: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:20:00.290 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:00 smithi071 ceph-mon[144544]: osd.2 marked itself down and dead 2024-04-23T02:20:00.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:00 smithi192 ceph-mon[108617]: pgmap v175: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:20:00.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:00 smithi192 ceph-mon[108617]: osd.2 marked itself down and dead 2024-04-23T02:20:00.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.571+0000 7f399ad51700 1 -- 172.21.15.71:0/204633892 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3994068c80 con 0x7f3994105bd0 2024-04-23T02:20:00.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.571+0000 7f3991ffb700 1 -- 172.21.15.71:0/204633892 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7f398005af90 con 0x7f3994105bd0 2024-04-23T02:20:00.577 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:20:00.577 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:20:00.577 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:20:00.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:20:00.577 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:20:00.577 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:20:00.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:20:00.577 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:20:00.577 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-04-23T02:20:00.577 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:20:00.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:20:00.578 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:20:00.578 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-23T02:20:00.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:20:00.578 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:20:00.578 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 7, 2024-04-23T02:20:00.578 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:20:00.578 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:20:00.578 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:20:00.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.574+0000 7f399ad51700 1 -- 172.21.15.71:0/204633892 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f398406cce0 msgr2=0x7f398406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:00.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.574+0000 7f399ad51700 1 --2- 172.21.15.71:0/204633892 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f398406cce0 0x7f398406f1a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3988007900 tx=0x7f3988008040 comp rx=0 tx=0).stop 2024-04-23T02:20:00.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.574+0000 7f399ad51700 1 -- 172.21.15.71:0/204633892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3994105bd0 msgr2=0x7f3994197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:00.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.574+0000 7f399ad51700 1 --2- 172.21.15.71:0/204633892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3994105bd0 0x7f3994197c80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3980006010 tx=0x7f398000be20 comp rx=0 tx=0).stop 2024-04-23T02:20:00.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.574+0000 7f399ad51700 1 -- 172.21.15.71:0/204633892 shutdown_connections 2024-04-23T02:20:00.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.574+0000 7f399ad51700 1 --2- 172.21.15.71:0/204633892 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f398406cce0 0x7f398406f1a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:00.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.574+0000 7f399ad51700 1 --2- 172.21.15.71:0/204633892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39940ffc20 0x7f3994197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:00.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.574+0000 7f399ad51700 1 --2- 172.21.15.71:0/204633892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3994105bd0 0x7f3994197c80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:00.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.574+0000 7f399ad51700 1 -- 172.21.15.71:0/204633892 >> 172.21.15.71:0/204633892 conn(0x7f3994075670 msgr2=0x7f39940fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:00.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.574+0000 7f399ad51700 1 -- 172.21.15.71:0/204633892 shutdown_connections 2024-04-23T02:20:00.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.574+0000 7f399ad51700 1 -- 172.21.15.71:0/204633892 wait complete. 2024-04-23T02:20:00.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.678+0000 7fb69360c700 1 -- 172.21.15.71:0/3511726805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb68c101840 msgr2=0x7fb68c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:00.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.678+0000 7fb69360c700 1 --2- 172.21.15.71:0/3511726805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb68c101840 0x7fb68c105890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009910 tx=0x7fb67c009c20 comp rx=0 tx=0).stop 2024-04-23T02:20:00.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.678+0000 7fb69360c700 1 -- 172.21.15.71:0/3511726805 shutdown_connections 2024-04-23T02:20:00.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.678+0000 7fb69360c700 1 --2- 172.21.15.71:0/3511726805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb68c101840 0x7fb68c105890 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:00.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.678+0000 7fb69360c700 1 --2- 172.21.15.71:0/3511726805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb68c100e90 0x7fb68c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:00.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.678+0000 7fb69360c700 1 -- 172.21.15.71:0/3511726805 >> 172.21.15.71:0/3511726805 conn(0x7fb68c0fc740 msgr2=0x7fb68c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:00.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.679+0000 7fb69360c700 1 -- 172.21.15.71:0/3511726805 shutdown_connections 2024-04-23T02:20:00.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.679+0000 7fb69360c700 1 -- 172.21.15.71:0/3511726805 wait complete. 2024-04-23T02:20:00.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.679+0000 7fb69360c700 1 Processor -- start 2024-04-23T02:20:00.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.679+0000 7fb69360c700 1 -- start start 2024-04-23T02:20:00.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.679+0000 7fb69360c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb68c100e90 0x7fb68c19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:00.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.679+0000 7fb69360c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb68c101840 0x7fb68c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:00.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.679+0000 7fb69360c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c19d200 con 0x7fb68c100e90 2024-04-23T02:20:00.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.679+0000 7fb69360c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c1966b0 con 0x7fb68c101840 2024-04-23T02:20:00.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.679+0000 7fb6913a8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb68c100e90 0x7fb68c19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:00.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.680+0000 7fb6913a8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb68c100e90 0x7fb68c19c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47128/0 (socket says 172.21.15.71:47128) 2024-04-23T02:20:00.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.680+0000 7fb6913a8700 1 -- 172.21.15.71:0/3620965941 learned_addr learned my addr 172.21.15.71:0/3620965941 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:20:00.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.680+0000 7fb690ba7700 1 --2- 172.21.15.71:0/3620965941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb68c101840 0x7fb68c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:00.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.680+0000 7fb6913a8700 1 -- 172.21.15.71:0/3620965941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb68c101840 msgr2=0x7fb68c19cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:00.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.680+0000 7fb6913a8700 1 --2- 172.21.15.71:0/3620965941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb68c101840 0x7fb68c19cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:00.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.680+0000 7fb6913a8700 1 -- 172.21.15.71:0/3620965941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c0095f0 con 0x7fb68c100e90 2024-04-23T02:20:00.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.680+0000 7fb6913a8700 1 --2- 172.21.15.71:0/3620965941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb68c100e90 0x7fb68c19c630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb68800d8d0 tx=0x7fb68800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:00.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.680+0000 7fb6827fc700 1 -- 172.21.15.71:0/3620965941 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6880098a0 con 0x7fb68c100e90 2024-04-23T02:20:00.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.680+0000 7fb6827fc700 1 -- 172.21.15.71:0/3620965941 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb688010460 con 0x7fb68c100e90 2024-04-23T02:20:00.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.680+0000 7fb69360c700 1 -- 172.21.15.71:0/3620965941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68c196990 con 0x7fb68c100e90 2024-04-23T02:20:00.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.680+0000 7fb6827fc700 1 -- 172.21.15.71:0/3620965941 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb688009ea0 con 0x7fb68c100e90 2024-04-23T02:20:00.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.680+0000 7fb69360c700 1 -- 172.21.15.71:0/3620965941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68c196ee0 con 0x7fb68c100e90 2024-04-23T02:20:00.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.681+0000 7fb69360c700 1 -- 172.21.15.71:0/3620965941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68c109290 con 0x7fb68c100e90 2024-04-23T02:20:00.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.682+0000 7fb6827fc700 1 -- 172.21.15.71:0/3620965941 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb6880105d0 con 0x7fb68c100e90 2024-04-23T02:20:00.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.682+0000 7fb6827fc700 1 --2- 172.21.15.71:0/3620965941 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb67806cce0 0x7fb67806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:00.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.682+0000 7fb6827fc700 1 -- 172.21.15.71:0/3620965941 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb68808c3d0 con 0x7fb68c100e90 2024-04-23T02:20:00.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.682+0000 7fb690ba7700 1 --2- 172.21.15.71:0/3620965941 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb67806cce0 0x7fb67806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:00.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.682+0000 7fb690ba7700 1 --2- 172.21.15.71:0/3620965941 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb67806cce0 0x7fb67806f1a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0fcfb0 tx=0x7fb67c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:00.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.686+0000 7fb6827fc700 1 -- 172.21.15.71:0/3620965941 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb688059f80 con 0x7fb68c100e90 2024-04-23T02:20:00.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.937+0000 7fb69360c700 1 -- 172.21.15.71:0/3620965941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb68c197100 con 0x7fb68c100e90 2024-04-23T02:20:00.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.938+0000 7fb6827fc700 1 -- 172.21.15.71:0/3620965941 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1869 (secure 0 0 0) 0x7fb688010880 con 0x7fb68c100e90 2024-04-23T02:20:00.943 INFO:teuthology.orchestra.run.smithi071.stdout:e25 2024-04-23T02:20:00.943 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:20:00.943 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:20:00.943 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:20:00.943 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:20:00.943 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:20:00.943 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:20:00.943 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 25 2024-04-23T02:20:00.943 INFO:teuthology.orchestra.run.smithi071.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T02:20:00.943 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:20:00.943 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:17:27.814232+0000 2024-04-23T02:20:00.944 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:20:00.944 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:20:00.944 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:20:00.944 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:20:00.944 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:20:00.944 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:20:00.944 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:20:00.944 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:20:00.944 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 46 2024-04-23T02:20:00.944 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:20:00.944 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:20:00.944 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:20:00.944 INFO:teuthology.orchestra.run.smithi071.stdout:up {0=14478,1=24243} 2024-04-23T02:20:00.944 INFO:teuthology.orchestra.run.smithi071.stdout:failed 2024-04-23T02:20:00.944 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:20:00.945 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:20:00.945 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:20:00.945 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:20:00.945 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:20:00.945 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:20:00.945 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:20:00.945 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:20:00.945 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:20:00.945 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/3442634960,v1:172.21.15.71:6827/3442634960] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:20:00.945 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{1:24243} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.192:6824/3780549896,v1:172.21.15.192:6825/3780549896] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:20:00.945 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{1:24255} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2100078284,v1:172.21.15.192:6827/2100078284] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:20:00.945 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:20:00.945 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:20:00.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.941+0000 7fb69360c700 1 -- 172.21.15.71:0/3620965941 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb67806cce0 msgr2=0x7fb67806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:00.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.941+0000 7fb69360c700 1 --2- 172.21.15.71:0/3620965941 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb67806cce0 0x7fb67806f1a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0fcfb0 tx=0x7fb67c01a040 comp rx=0 tx=0).stop 2024-04-23T02:20:00.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.941+0000 7fb69360c700 1 -- 172.21.15.71:0/3620965941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb68c100e90 msgr2=0x7fb68c19c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:00.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.941+0000 7fb69360c700 1 --2- 172.21.15.71:0/3620965941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb68c100e90 0x7fb68c19c630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb68800d8d0 tx=0x7fb68800dbe0 comp rx=0 tx=0).stop 2024-04-23T02:20:00.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.941+0000 7fb69360c700 1 -- 172.21.15.71:0/3620965941 shutdown_connections 2024-04-23T02:20:00.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.941+0000 7fb69360c700 1 --2- 172.21.15.71:0/3620965941 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb67806cce0 0x7fb67806f1a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:00.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.941+0000 7fb69360c700 1 --2- 172.21.15.71:0/3620965941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb68c100e90 0x7fb68c19c630 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:00.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.941+0000 7fb69360c700 1 --2- 172.21.15.71:0/3620965941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb68c101840 0x7fb68c19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:00.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.941+0000 7fb69360c700 1 -- 172.21.15.71:0/3620965941 >> 172.21.15.71:0/3620965941 conn(0x7fb68c0fc740 msgr2=0x7fb68c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:00.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.941+0000 7fb69360c700 1 -- 172.21.15.71:0/3620965941 shutdown_connections 2024-04-23T02:20:00.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:00.941+0000 7fb69360c700 1 -- 172.21.15.71:0/3620965941 wait complete. 2024-04-23T02:20:00.947 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 25 2024-04-23T02:20:01.015 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:20:00 smithi071 bash[169432]: ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-2 2024-04-23T02:20:01.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.063+0000 7fe40a023700 1 -- 172.21.15.71:0/834677939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe404068580 msgr2=0x7fe4040689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:01.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.063+0000 7fe40a023700 1 --2- 172.21.15.71:0/834677939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe404068580 0x7fe4040689e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009910 tx=0x7fe3ec009c20 comp rx=0 tx=0).stop 2024-04-23T02:20:01.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.063+0000 7fe40a023700 1 -- 172.21.15.71:0/834677939 shutdown_connections 2024-04-23T02:20:01.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.063+0000 7fe40a023700 1 --2- 172.21.15.71:0/834677939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe404068580 0x7fe4040689e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:01.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.063+0000 7fe40a023700 1 --2- 172.21.15.71:0/834677939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe404106cd0 0x7fe4041070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:01.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.063+0000 7fe40a023700 1 -- 172.21.15.71:0/834677939 >> 172.21.15.71:0/834677939 conn(0x7fe404075670 msgr2=0x7fe404075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:01.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.063+0000 7fe40a023700 1 -- 172.21.15.71:0/834677939 shutdown_connections 2024-04-23T02:20:01.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.063+0000 7fe40a023700 1 -- 172.21.15.71:0/834677939 wait complete. 2024-04-23T02:20:01.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.063+0000 7fe40a023700 1 Processor -- start 2024-04-23T02:20:01.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.064+0000 7fe40a023700 1 -- start start 2024-04-23T02:20:01.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.064+0000 7fe40a023700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe404068580 0x7fe40419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:01.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.064+0000 7fe40a023700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe404106cd0 0x7fe40419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:01.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.064+0000 7fe40a023700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40419d110 con 0x7fe404068580 2024-04-23T02:20:01.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.064+0000 7fe40a023700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4041965c0 con 0x7fe404106cd0 2024-04-23T02:20:01.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.064+0000 7fe4037fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe404068580 0x7fe40419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:01.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.064+0000 7fe4037fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe404068580 0x7fe40419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47146/0 (socket says 172.21.15.71:47146) 2024-04-23T02:20:01.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.064+0000 7fe4037fe700 1 -- 172.21.15.71:0/2586240559 learned_addr learned my addr 172.21.15.71:0/2586240559 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:20:01.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.064+0000 7fe402ffd700 1 --2- 172.21.15.71:0/2586240559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe404106cd0 0x7fe40419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:01.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.064+0000 7fe4037fe700 1 -- 172.21.15.71:0/2586240559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe404106cd0 msgr2=0x7fe40419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:01.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.064+0000 7fe4037fe700 1 --2- 172.21.15.71:0/2586240559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe404106cd0 0x7fe40419ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:01.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.064+0000 7fe4037fe700 1 -- 172.21.15.71:0/2586240559 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec0095f0 con 0x7fe404068580 2024-04-23T02:20:01.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.064+0000 7fe4037fe700 1 --2- 172.21.15.71:0/2586240559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe404068580 0x7fe40419c540 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400ef10 tx=0x7fe3f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:01.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.065+0000 7fe400ff9700 1 -- 172.21.15.71:0/2586240559 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3f400c9f0 con 0x7fe404068580 2024-04-23T02:20:01.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.065+0000 7fe400ff9700 1 -- 172.21.15.71:0/2586240559 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe3f400f040 con 0x7fe404068580 2024-04-23T02:20:01.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.065+0000 7fe40a023700 1 -- 172.21.15.71:0/2586240559 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe404196840 con 0x7fe404068580 2024-04-23T02:20:01.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.065+0000 7fe400ff9700 1 -- 172.21.15.71:0/2586240559 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3f4014480 con 0x7fe404068580 2024-04-23T02:20:01.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.065+0000 7fe40a023700 1 -- 172.21.15.71:0/2586240559 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe404196d90 con 0x7fe404068580 2024-04-23T02:20:01.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.066+0000 7fe40a023700 1 -- 172.21.15.71:0/2586240559 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4041091d0 con 0x7fe404068580 2024-04-23T02:20:01.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.066+0000 7fe400ff9700 1 -- 172.21.15.71:0/2586240559 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe3f4007500 con 0x7fe404068580 2024-04-23T02:20:01.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.067+0000 7fe400ff9700 1 --2- 172.21.15.71:0/2586240559 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe3f006cce0 0x7fe3f006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:01.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.067+0000 7fe402ffd700 1 --2- 172.21.15.71:0/2586240559 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe3f006cce0 0x7fe3f006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:01.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.067+0000 7fe400ff9700 1 -- 172.21.15.71:0/2586240559 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe3f408c830 con 0x7fe404068580 2024-04-23T02:20:01.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.067+0000 7fe402ffd700 1 --2- 172.21.15.71:0/2586240559 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe3f006cce0 0x7fe3f006f1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe4040fe260 tx=0x7fe3ec005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:01.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.071+0000 7fe400ff9700 1 -- 172.21.15.71:0/2586240559 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe3f405a3e0 con 0x7fe404068580 2024-04-23T02:20:01.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.289+0000 7fe40a023700 1 -- 172.21.15.71:0/2586240559 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe404196fb0 con 0x7fe3f006cce0 2024-04-23T02:20:01.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.290+0000 7fe400ff9700 1 -- 172.21.15.71:0/2586240559 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fe404196fb0 con 0x7fe3f006cce0 2024-04-23T02:20:01.295 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:20:01.295 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:20:01.295 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:20:01.295 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:20:01.295 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:20:01.295 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:20:01.295 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:20:01.295 INFO:teuthology.orchestra.run.smithi071.stdout: "mon" 2024-04-23T02:20:01.296 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:20:01.296 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "8/23 daemons upgraded", 2024-04-23T02:20:01.296 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading osd daemons", 2024-04-23T02:20:01.296 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:20:01.296 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:20:01.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.293+0000 7fe40a023700 1 -- 172.21.15.71:0/2586240559 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe3f006cce0 msgr2=0x7fe3f006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:01.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.293+0000 7fe40a023700 1 --2- 172.21.15.71:0/2586240559 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe3f006cce0 0x7fe3f006f1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe4040fe260 tx=0x7fe3ec005940 comp rx=0 tx=0).stop 2024-04-23T02:20:01.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.293+0000 7fe40a023700 1 -- 172.21.15.71:0/2586240559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe404068580 msgr2=0x7fe40419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:01.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.293+0000 7fe40a023700 1 --2- 172.21.15.71:0/2586240559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe404068580 0x7fe40419c540 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400ef10 tx=0x7fe3f400c3a0 comp rx=0 tx=0).stop 2024-04-23T02:20:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.293+0000 7fe40a023700 1 -- 172.21.15.71:0/2586240559 shutdown_connections 2024-04-23T02:20:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.293+0000 7fe40a023700 1 --2- 172.21.15.71:0/2586240559 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe3f006cce0 0x7fe3f006f1a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.293+0000 7fe40a023700 1 --2- 172.21.15.71:0/2586240559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe404068580 0x7fe40419c540 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.293+0000 7fe40a023700 1 --2- 172.21.15.71:0/2586240559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe404106cd0 0x7fe40419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.293+0000 7fe40a023700 1 -- 172.21.15.71:0/2586240559 >> 172.21.15.71:0/2586240559 conn(0x7fe404075670 msgr2=0x7fe4040fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.293+0000 7fe40a023700 1 -- 172.21.15.71:0/2586240559 shutdown_connections 2024-04-23T02:20:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.293+0000 7fe40a023700 1 -- 172.21.15.71:0/2586240559 wait complete. 2024-04-23T02:20:01.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:01 smithi192 ceph-mon[108617]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:20:01.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:01 smithi192 ceph-mon[108617]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:20:01.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:01 smithi192 ceph-mon[108617]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-23T02:20:01.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:01 smithi192 ceph-mon[108617]: osdmap e59: 6 total, 5 up, 6 in 2024-04-23T02:20:01.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:01 smithi192 ceph-mon[108617]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down 2024-04-23T02:20:01.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:01 smithi192 ceph-mon[108617]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:20:01.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:01 smithi192 ceph-mon[108617]: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:20:01.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:01 smithi192 ceph-mon[108617]: [WRN] OSD_DOWN: 1 osds down 2024-04-23T02:20:01.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:01 smithi192 ceph-mon[108617]: osd.2 (root=default,host=smithi071) is down 2024-04-23T02:20:01.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:01 smithi192 ceph-mon[108617]: from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:20:01.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:01 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/204633892' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:20:01.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:01 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3620965941' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:20:01.378 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:01 smithi192 ceph-mon[108617]: osdmap e60: 6 total, 5 up, 6 in 2024-04-23T02:20:01.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.413+0000 7f867b760700 1 -- 172.21.15.71:0/366484033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674068580 msgr2=0x7f86740689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:01.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.413+0000 7f867b760700 1 --2- 172.21.15.71:0/366484033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674068580 0x7f86740689e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8664009910 tx=0x7f8664009c20 comp rx=0 tx=0).stop 2024-04-23T02:20:01.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.413+0000 7f867b760700 1 -- 172.21.15.71:0/366484033 shutdown_connections 2024-04-23T02:20:01.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.413+0000 7f867b760700 1 --2- 172.21.15.71:0/366484033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674068580 0x7f86740689e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:01.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.413+0000 7f867b760700 1 --2- 172.21.15.71:0/366484033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8674106cd0 0x7f86741070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:01.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.413+0000 7f867b760700 1 -- 172.21.15.71:0/366484033 >> 172.21.15.71:0/366484033 conn(0x7f8674075670 msgr2=0x7f8674075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:01.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.414+0000 7f867b760700 1 -- 172.21.15.71:0/366484033 shutdown_connections 2024-04-23T02:20:01.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.414+0000 7f867b760700 1 -- 172.21.15.71:0/366484033 wait complete. 2024-04-23T02:20:01.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.414+0000 7f867b760700 1 Processor -- start 2024-04-23T02:20:01.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.414+0000 7f867b760700 1 -- start start 2024-04-23T02:20:01.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.414+0000 7f867b760700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674068580 0x7f86740ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:01.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.414+0000 7f867b760700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8674106cd0 0x7f8674100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:01.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.414+0000 7f867b760700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8674100770 con 0x7f8674068580 2024-04-23T02:20:01.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.414+0000 7f867b760700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86741008b0 con 0x7f8674106cd0 2024-04-23T02:20:01.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.414+0000 7f86794fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674068580 0x7f86740ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:01.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.414+0000 7f86794fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674068580 0x7f86740ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47178/0 (socket says 172.21.15.71:47178) 2024-04-23T02:20:01.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.415+0000 7f86794fc700 1 -- 172.21.15.71:0/2696865124 learned_addr learned my addr 172.21.15.71:0/2696865124 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:20:01.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.415+0000 7f8678cfb700 1 --2- 172.21.15.71:0/2696865124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8674106cd0 0x7f8674100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:01.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.415+0000 7f86794fc700 1 -- 172.21.15.71:0/2696865124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8674106cd0 msgr2=0x7f8674100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:01.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.415+0000 7f86794fc700 1 --2- 172.21.15.71:0/2696865124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8674106cd0 0x7f8674100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:01.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.415+0000 7f86794fc700 1 -- 172.21.15.71:0/2696865124 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86640095f0 con 0x7f8674068580 2024-04-23T02:20:01.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.415+0000 7f86794fc700 1 --2- 172.21.15.71:0/2696865124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674068580 0x7f86740ffcf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f867000c930 tx=0x7f867000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:01.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.415+0000 7f866a7fc700 1 -- 172.21.15.71:0/2696865124 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8670007ad0 con 0x7f8674068580 2024-04-23T02:20:01.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.415+0000 7f866a7fc700 1 -- 172.21.15.71:0/2696865124 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8670007c30 con 0x7f8674068580 2024-04-23T02:20:01.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.415+0000 7f867b760700 1 -- 172.21.15.71:0/2696865124 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86741a2940 con 0x7f8674068580 2024-04-23T02:20:01.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.415+0000 7f866a7fc700 1 -- 172.21.15.71:0/2696865124 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8670018630 con 0x7f8674068580 2024-04-23T02:20:01.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.415+0000 7f867b760700 1 -- 172.21.15.71:0/2696865124 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86741a2ca0 con 0x7f8674068580 2024-04-23T02:20:01.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.416+0000 7f867b760700 1 -- 172.21.15.71:0/2696865124 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86741091d0 con 0x7f8674068580 2024-04-23T02:20:01.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.417+0000 7f866a7fc700 1 -- 172.21.15.71:0/2696865124 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f867000f450 con 0x7f8674068580 2024-04-23T02:20:01.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.417+0000 7f866a7fc700 1 --2- 172.21.15.71:0/2696865124 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f866006cce0 0x7f866006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:01.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.417+0000 7f866a7fc700 1 -- 172.21.15.71:0/2696865124 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f867008c720 con 0x7f8674068580 2024-04-23T02:20:01.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.417+0000 7f8678cfb700 1 --2- 172.21.15.71:0/2696865124 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f866006cce0 0x7f866006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:01.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.417+0000 7f8678cfb700 1 --2- 172.21.15.71:0/2696865124 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f866006cce0 0x7f866006f1a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f86741014e0 tx=0x7f866401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:01.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.421+0000 7f866a7fc700 1 -- 172.21.15.71:0/2696865124 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f86700568f0 con 0x7f8674068580 2024-04-23T02:20:01.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:01 smithi071 ceph-mon[144544]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:20:01.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:01 smithi071 ceph-mon[144544]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:20:01.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:01 smithi071 ceph-mon[144544]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-23T02:20:01.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:01 smithi071 ceph-mon[144544]: osdmap e59: 6 total, 5 up, 6 in 2024-04-23T02:20:01.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:01 smithi071 ceph-mon[144544]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down 2024-04-23T02:20:01.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:01 smithi071 ceph-mon[144544]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:20:01.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:01 smithi071 ceph-mon[144544]: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:20:01.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:01 smithi071 ceph-mon[144544]: [WRN] OSD_DOWN: 1 osds down 2024-04-23T02:20:01.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:01 smithi071 ceph-mon[144544]: osd.2 (root=default,host=smithi071) is down 2024-04-23T02:20:01.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:01 smithi071 ceph-mon[144544]: from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:20:01.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:01 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/204633892' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:20:01.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:01 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3620965941' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:20:01.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:01 smithi071 ceph-mon[144544]: osdmap e60: 6 total, 5 up, 6 in 2024-04-23T02:20:01.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.700+0000 7f867b760700 1 -- 172.21.15.71:0/2696865124 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f867404f2e0 con 0x7f8674068580 2024-04-23T02:20:01.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.700+0000 7f866a7fc700 1 -- 172.21.15.71:0/2696865124 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+551 (secure 0 0 0) 0x7f8670059f10 con 0x7f8674068580 2024-04-23T02:20:01.706 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Reduced data availability: 2 pgs peering 2024-04-23T02:20:01.706 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:20:01.706 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:20:01.706 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-23T02:20:01.706 INFO:teuthology.orchestra.run.smithi071.stdout: osd.2 (root=default,host=smithi071) is down 2024-04-23T02:20:01.706 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 2 pgs peering 2024-04-23T02:20:01.706 INFO:teuthology.orchestra.run.smithi071.stdout: pg 2.c is stuck peering for 3m, current state peering, last acting [3,0] 2024-04-23T02:20:01.706 INFO:teuthology.orchestra.run.smithi071.stdout: pg 3.1e is stuck peering for 4m, current state peering, last acting [3,1] 2024-04-23T02:20:01.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.703+0000 7f867b760700 1 -- 172.21.15.71:0/2696865124 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f866006cce0 msgr2=0x7f866006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:01.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.703+0000 7f867b760700 1 --2- 172.21.15.71:0/2696865124 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f866006cce0 0x7f866006f1a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f86741014e0 tx=0x7f866401a040 comp rx=0 tx=0).stop 2024-04-23T02:20:01.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.703+0000 7f867b760700 1 -- 172.21.15.71:0/2696865124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674068580 msgr2=0x7f86740ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:01.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.703+0000 7f867b760700 1 --2- 172.21.15.71:0/2696865124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674068580 0x7f86740ffcf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f867000c930 tx=0x7f867000ccf0 comp rx=0 tx=0).stop 2024-04-23T02:20:01.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.703+0000 7f867b760700 1 -- 172.21.15.71:0/2696865124 shutdown_connections 2024-04-23T02:20:01.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.703+0000 7f867b760700 1 --2- 172.21.15.71:0/2696865124 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f866006cce0 0x7f866006f1a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:01.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.703+0000 7f867b760700 1 --2- 172.21.15.71:0/2696865124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674068580 0x7f86740ffcf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:01.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.703+0000 7f867b760700 1 --2- 172.21.15.71:0/2696865124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8674106cd0 0x7f8674100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:01.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.703+0000 7f867b760700 1 -- 172.21.15.71:0/2696865124 >> 172.21.15.71:0/2696865124 conn(0x7f8674075670 msgr2=0x7f86740fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:01.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.703+0000 7f867b760700 1 -- 172.21.15.71:0/2696865124 shutdown_connections 2024-04-23T02:20:01.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:01.703+0000 7f867b760700 1 -- 172.21.15.71:0/2696865124 wait complete. 2024-04-23T02:20:02.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:02 smithi192 ceph-mon[108617]: pgmap v177: 65 pgs: 12 peering, 7 stale+active+clean, 46 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:20:02.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:02 smithi192 ceph-mon[108617]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-23T02:20:02.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:02 smithi192 ceph-mon[108617]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:20:02.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:02 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2696865124' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:20:02.457 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:02 smithi071 ceph-mon[144544]: pgmap v177: 65 pgs: 12 peering, 7 stale+active+clean, 46 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:20:02.457 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:02 smithi071 ceph-mon[144544]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-23T02:20:02.457 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:02 smithi071 ceph-mon[144544]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:20:02.457 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:02 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2696865124' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:20:03.067 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:20:02 smithi071 systemd[1]: ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.2.service: Succeeded. 2024-04-23T02:20:03.067 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:20:02 smithi071 systemd[1]: Stopped Ceph osd.2 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:20:03.067 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:20:02 smithi071 systemd[1]: Starting Ceph osd.2 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T02:20:03.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:03 smithi192 ceph-mon[108617]: Health check failed: Degraded data redundancy: 14/366 objects degraded (3.825%), 4 pgs degraded (PG_DEGRADED) 2024-04-23T02:20:03.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:03 smithi071 ceph-mon[144544]: Health check failed: Degraded data redundancy: 14/366 objects degraded (3.825%), 4 pgs degraded (PG_DEGRADED) 2024-04-23T02:20:04.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:04 smithi071 ceph-mon[144544]: pgmap v179: 65 pgs: 4 active+undersized+degraded, 12 peering, 4 stale+active+clean, 4 active+undersized, 41 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 14/366 objects degraded (3.825%) 2024-04-23T02:20:04.244 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:20:03 smithi071 bash[169870]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-23T02:20:04.244 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:20:03 smithi071 bash[169870]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-23T02:20:04.244 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:20:03 smithi071 bash[169870]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-23T02:20:04.244 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:20:03 smithi071 bash[169870]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-23T02:20:04.244 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:20:03 smithi071 bash[169870]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-23T02:20:04.245 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:20:03 smithi071 bash[169870]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-23T02:20:04.245 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:20:03 smithi071 bash[169870]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-23T02:20:04.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:04 smithi192 ceph-mon[108617]: pgmap v179: 65 pgs: 4 active+undersized+degraded, 12 peering, 4 stale+active+clean, 4 active+undersized, 41 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 14/366 objects degraded (3.825%) 2024-04-23T02:20:05.202 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:20:05 smithi071 ceph-osd[170193]: -- 172.21.15.71:0/2064416404 <== mon.0 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5576c24a51e0 con 0x5576c2474c00 2024-04-23T02:20:05.494 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:20:05 smithi071 bash[170155]: 7e44c427695b110a383fef708e7d5186db015d7902b388eba929f672946d6d3d 2024-04-23T02:20:05.494 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:20:05 smithi071 systemd[1]: Started Ceph osd.2 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:20:05.993 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:20:05 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-2[170169]: 2024-04-23T02:20:05.653+0000 7f14920de7c0 -1 Falling back to public interface 2024-04-23T02:20:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:05 smithi071 ceph-mon[144544]: pgmap v180: 65 pgs: 7 active+undersized+degraded, 12 peering, 8 active+undersized, 38 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 27/366 objects degraded (7.377%) 2024-04-23T02:20:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:20:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:20:06.081 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:05 smithi192 ceph-mon[108617]: pgmap v180: 65 pgs: 7 active+undersized+degraded, 12 peering, 8 active+undersized, 38 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 27/366 objects degraded (7.377%) 2024-04-23T02:20:06.082 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:06.082 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:20:06.082 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:06.082 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:06.082 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:20:07.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:06 smithi192 ceph-mon[108617]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-23T02:20:07.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:06 smithi071 ceph-mon[144544]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-23T02:20:07.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:07 smithi071 ceph-mon[144544]: pgmap v181: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45/366 objects degraded (12.295%) 2024-04-23T02:20:08.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:07 smithi192 ceph-mon[108617]: pgmap v181: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45/366 objects degraded (12.295%) 2024-04-23T02:20:09.743 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:20:09 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-2[170169]: 2024-04-23T02:20:09.553+0000 7f14920de7c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-04-23T02:20:10.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:09 smithi192 ceph-mon[108617]: pgmap v182: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45/366 objects degraded (12.295%) 2024-04-23T02:20:10.149 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:09 smithi071 ceph-mon[144544]: pgmap v182: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45/366 objects degraded (12.295%) 2024-04-23T02:20:10.149 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:20:09 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-2[170169]: 2024-04-23T02:20:09.788+0000 7f14920de7c0 -1 osd.2 58 log_to_monitors true 2024-04-23T02:20:11.115 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:10 smithi071 ceph-mon[144544]: from='osd.2 [v2:172.21.15.71:6818/3388560056,v1:172.21.15.71:6819/3388560056]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T02:20:11.115 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:10 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:11.115 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:10 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:11.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:10 smithi192 ceph-mon[108617]: from='osd.2 [v2:172.21.15.71:6818/3388560056,v1:172.21.15.71:6819/3388560056]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T02:20:11.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:10 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:11.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:10 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:11.994 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 02:20:11 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-2[170169]: 2024-04-23T02:20:11.672+0000 7f147f530700 -1 osd.2 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T02:20:11.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:11 smithi071 ceph-mon[144544]: pgmap v183: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 45/366 objects degraded (12.295%) 2024-04-23T02:20:11.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:11 smithi071 ceph-mon[144544]: from='osd.2 [v2:172.21.15.71:6818/3388560056,v1:172.21.15.71:6819/3388560056]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T02:20:11.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:11 smithi071 ceph-mon[144544]: osdmap e61: 6 total, 5 up, 6 in 2024-04-23T02:20:11.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:11 smithi071 ceph-mon[144544]: from='osd.2 [v2:172.21.15.71:6818/3388560056,v1:172.21.15.71:6819/3388560056]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-23T02:20:11.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:11 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:11.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:11 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:11.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:11 smithi071 ceph-mon[144544]: from='osd.2 [v2:172.21.15.71:6818/3388560056,v1:172.21.15.71:6819/3388560056]' entity='osd.2' 2024-04-23T02:20:12.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:11 smithi192 ceph-mon[108617]: pgmap v183: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 45/366 objects degraded (12.295%) 2024-04-23T02:20:12.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:11 smithi192 ceph-mon[108617]: from='osd.2 [v2:172.21.15.71:6818/3388560056,v1:172.21.15.71:6819/3388560056]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T02:20:12.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:11 smithi192 ceph-mon[108617]: osdmap e61: 6 total, 5 up, 6 in 2024-04-23T02:20:12.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:11 smithi192 ceph-mon[108617]: from='osd.2 [v2:172.21.15.71:6818/3388560056,v1:172.21.15.71:6819/3388560056]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-23T02:20:12.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:11 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:12.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:11 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:12.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:11 smithi192 ceph-mon[108617]: from='osd.2 [v2:172.21.15.71:6818/3388560056,v1:172.21.15.71:6819/3388560056]' entity='osd.2' 2024-04-23T02:20:12.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:12 smithi071 ceph-mon[144544]: Health check update: Degraded data redundancy: 45/366 objects degraded (12.295%), 13 pgs degraded (PG_DEGRADED) 2024-04-23T02:20:12.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:12 smithi071 ceph-mon[144544]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-23T02:20:12.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:12 smithi071 ceph-mon[144544]: osd.2 [v2:172.21.15.71:6818/3388560056,v1:172.21.15.71:6819/3388560056] boot 2024-04-23T02:20:12.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:12 smithi071 ceph-mon[144544]: osdmap e62: 6 total, 6 up, 6 in 2024-04-23T02:20:12.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:12 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T02:20:13.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:12 smithi192 ceph-mon[108617]: Health check update: Degraded data redundancy: 45/366 objects degraded (12.295%), 13 pgs degraded (PG_DEGRADED) 2024-04-23T02:20:13.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:12 smithi192 ceph-mon[108617]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-23T02:20:13.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:12 smithi192 ceph-mon[108617]: osd.2 [v2:172.21.15.71:6818/3388560056,v1:172.21.15.71:6819/3388560056] boot 2024-04-23T02:20:13.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:12 smithi192 ceph-mon[108617]: osdmap e62: 6 total, 6 up, 6 in 2024-04-23T02:20:13.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:12 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T02:20:14.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:13 smithi192 ceph-mon[108617]: pgmap v186: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 45/366 objects degraded (12.295%) 2024-04-23T02:20:14.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:13 smithi192 ceph-mon[108617]: osdmap e63: 6 total, 6 up, 6 in 2024-04-23T02:20:14.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:13 smithi071 ceph-mon[144544]: pgmap v186: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 45/366 objects degraded (12.295%) 2024-04-23T02:20:14.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:13 smithi071 ceph-mon[144544]: osdmap e63: 6 total, 6 up, 6 in 2024-04-23T02:20:16.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:16 smithi071 ceph-mon[144544]: pgmap v188: 65 pgs: 12 active+undersized+degraded, 12 active+undersized, 41 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 42/366 objects degraded (11.475%) 2024-04-23T02:20:16.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:16 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:16.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:16 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:16.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:16 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:20:16.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:16 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:20:16.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:16 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:16.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:16 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:20:16.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:16 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:20:16.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:16 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:20:16.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:16 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:20:16.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:16 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-23T02:20:16.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:16 smithi071 ceph-mon[144544]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-23T02:20:16.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:16 smithi071 ceph-mon[144544]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-04-23T02:20:16.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:16 smithi192 ceph-mon[108617]: pgmap v188: 65 pgs: 12 active+undersized+degraded, 12 active+undersized, 41 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 42/366 objects degraded (11.475%) 2024-04-23T02:20:16.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:16 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:16.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:16 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:16.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:16 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:20:16.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:16 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:20:16.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:16 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:16.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:16 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:20:16.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:16 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:20:16.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:16 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:20:16.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:16 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:20:16.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:16 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-23T02:20:16.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:16 smithi192 ceph-mon[108617]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-23T02:20:16.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:16 smithi192 ceph-mon[108617]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-04-23T02:20:17.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:17 smithi192 ceph-mon[108617]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 42/366 objects degraded (11.475%), 12 pgs degraded) 2024-04-23T02:20:17.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:17 smithi071 ceph-mon[144544]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 42/366 objects degraded (11.475%), 12 pgs degraded) 2024-04-23T02:20:18.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:18 smithi192 ceph-mon[108617]: pgmap v189: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-23T02:20:18.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:18 smithi071 ceph-mon[144544]: pgmap v189: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-23T02:20:20.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:20 smithi192 ceph-mon[108617]: pgmap v190: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 774 B/s rd, 0 op/s 2024-04-23T02:20:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:20:20.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:20 smithi071 ceph-mon[144544]: pgmap v190: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 774 B/s rd, 0 op/s 2024-04-23T02:20:20.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:20.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:20:22.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:22 smithi192 ceph-mon[108617]: pgmap v191: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-04-23T02:20:22.634 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:22 smithi071 ceph-mon[144544]: pgmap v191: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-04-23T02:20:24.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:24 smithi192 ceph-mon[108617]: pgmap v192: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2 op/s 2024-04-23T02:20:24.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:24 smithi071 ceph-mon[144544]: pgmap v192: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2 op/s 2024-04-23T02:20:26.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:25 smithi192 ceph-mon[108617]: pgmap v193: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-04-23T02:20:26.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:26.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:25 smithi071 ceph-mon[144544]: pgmap v193: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-04-23T02:20:26.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:28.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:27 smithi192 ceph-mon[108617]: pgmap v194: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-04-23T02:20:28.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:27 smithi071 ceph-mon[144544]: pgmap v194: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-04-23T02:20:30.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:29 smithi192 ceph-mon[108617]: pgmap v195: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:20:30.247 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:29 smithi071 ceph-mon[144544]: pgmap v195: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:20:31.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:30 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-23T02:20:31.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:30 smithi192 ceph-mon[108617]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-23T02:20:31.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:30 smithi192 ceph-mon[108617]: Upgrade: osd.3 is safe to restart 2024-04-23T02:20:31.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:30 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-23T02:20:31.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:30 smithi071 ceph-mon[144544]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-23T02:20:31.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:30 smithi071 ceph-mon[144544]: Upgrade: osd.3 is safe to restart 2024-04-23T02:20:31.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.823+0000 7f31ca7b3700 1 -- 172.21.15.71:0/1233521269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c4101850 msgr2=0x7f31c41058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:31.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.823+0000 7f31ca7b3700 1 --2- 172.21.15.71:0/1233521269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c4101850 0x7f31c41058a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009910 tx=0x7f31b4009c20 comp rx=0 tx=0).stop 2024-04-23T02:20:31.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.823+0000 7f31ca7b3700 1 -- 172.21.15.71:0/1233521269 shutdown_connections 2024-04-23T02:20:31.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.823+0000 7f31ca7b3700 1 --2- 172.21.15.71:0/1233521269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c4101850 0x7f31c41058a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:31.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.823+0000 7f31ca7b3700 1 --2- 172.21.15.71:0/1233521269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31c4100ea0 0x7f31c4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:31.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.823+0000 7f31ca7b3700 1 -- 172.21.15.71:0/1233521269 >> 172.21.15.71:0/1233521269 conn(0x7f31c40fc750 msgr2=0x7f31c40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:31.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.823+0000 7f31ca7b3700 1 -- 172.21.15.71:0/1233521269 shutdown_connections 2024-04-23T02:20:31.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.823+0000 7f31ca7b3700 1 -- 172.21.15.71:0/1233521269 wait complete. 2024-04-23T02:20:31.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.824+0000 7f31ca7b3700 1 Processor -- start 2024-04-23T02:20:31.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.824+0000 7f31ca7b3700 1 -- start start 2024-04-23T02:20:31.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.824+0000 7f31ca7b3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c4100ea0 0x7f31c419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:31.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.824+0000 7f31ca7b3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31c4101850 0x7f31c419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:31.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.824+0000 7f31ca7b3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c419d1f0 con 0x7f31c4100ea0 2024-04-23T02:20:31.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.824+0000 7f31ca7b3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c41966a0 con 0x7f31c4101850 2024-04-23T02:20:31.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.824+0000 7f31c3fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c4100ea0 0x7f31c419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:31.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.824+0000 7f31c3fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c4100ea0 0x7f31c419c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51644/0 (socket says 172.21.15.71:51644) 2024-04-23T02:20:31.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.824+0000 7f31c3fff700 1 -- 172.21.15.71:0/828844099 learned_addr learned my addr 172.21.15.71:0/828844099 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:20:31.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.824+0000 7f31c37fe700 1 --2- 172.21.15.71:0/828844099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31c4101850 0x7f31c419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:31.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.825+0000 7f31c3fff700 1 -- 172.21.15.71:0/828844099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31c4101850 msgr2=0x7f31c419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:31.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.825+0000 7f31c3fff700 1 --2- 172.21.15.71:0/828844099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31c4101850 0x7f31c419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:31.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.825+0000 7f31c3fff700 1 -- 172.21.15.71:0/828844099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b40095f0 con 0x7f31c4100ea0 2024-04-23T02:20:31.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.825+0000 7f31c3fff700 1 --2- 172.21.15.71:0/828844099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c4100ea0 0x7f31c419c620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f31b800d8d0 tx=0x7f31b800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:31.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.825+0000 7f31c17fa700 1 -- 172.21.15.71:0/828844099 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31b80098a0 con 0x7f31c4100ea0 2024-04-23T02:20:31.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.825+0000 7f31ca7b3700 1 -- 172.21.15.71:0/828844099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31c4196980 con 0x7f31c4100ea0 2024-04-23T02:20:31.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.825+0000 7f31c17fa700 1 -- 172.21.15.71:0/828844099 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f31b8010460 con 0x7f31c4100ea0 2024-04-23T02:20:31.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.825+0000 7f31c17fa700 1 -- 172.21.15.71:0/828844099 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31b8009ea0 con 0x7f31c4100ea0 2024-04-23T02:20:31.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.825+0000 7f31ca7b3700 1 -- 172.21.15.71:0/828844099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31c4196ed0 con 0x7f31c4100ea0 2024-04-23T02:20:31.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.826+0000 7f31ca7b3700 1 -- 172.21.15.71:0/828844099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31c41092a0 con 0x7f31c4100ea0 2024-04-23T02:20:31.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.827+0000 7f31c17fa700 1 -- 172.21.15.71:0/828844099 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f31b80105d0 con 0x7f31c4100ea0 2024-04-23T02:20:31.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.827+0000 7f31c17fa700 1 --2- 172.21.15.71:0/828844099 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f31ac06cce0 0x7f31ac06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:31.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.827+0000 7f31c37fe700 1 --2- 172.21.15.71:0/828844099 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f31ac06cce0 0x7f31ac06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:31.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.827+0000 7f31c17fa700 1 -- 172.21.15.71:0/828844099 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f31b808c3d0 con 0x7f31c4100ea0 2024-04-23T02:20:31.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.827+0000 7f31c37fe700 1 --2- 172.21.15.71:0/828844099 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f31ac06cce0 0x7f31ac06f1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f31c40fcfc0 tx=0x7f31b401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:31.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:31.832+0000 7f31c17fa700 1 -- 172.21.15.71:0/828844099 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f31b8059f80 con 0x7f31c4100ea0 2024-04-23T02:20:32.010 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:31 smithi192 ceph-mon[108617]: pgmap v196: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:20:32.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.042+0000 7f31ca7b3700 1 -- 172.21.15.71:0/828844099 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f31c4197ce0 con 0x7f31ac06cce0 2024-04-23T02:20:32.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.043+0000 7f31c17fa700 1 -- 172.21.15.71:0/828844099 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f31c4197ce0 con 0x7f31ac06cce0 2024-04-23T02:20:32.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.045+0000 7f31ca7b3700 1 -- 172.21.15.71:0/828844099 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f31ac06cce0 msgr2=0x7f31ac06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:32.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.045+0000 7f31ca7b3700 1 --2- 172.21.15.71:0/828844099 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f31ac06cce0 0x7f31ac06f1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f31c40fcfc0 tx=0x7f31b401a040 comp rx=0 tx=0).stop 2024-04-23T02:20:32.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.045+0000 7f31ca7b3700 1 -- 172.21.15.71:0/828844099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c4100ea0 msgr2=0x7f31c419c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:32.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.045+0000 7f31ca7b3700 1 --2- 172.21.15.71:0/828844099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c4100ea0 0x7f31c419c620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f31b800d8d0 tx=0x7f31b800dbe0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.045+0000 7f31ca7b3700 1 -- 172.21.15.71:0/828844099 shutdown_connections 2024-04-23T02:20:32.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.045+0000 7f31ca7b3700 1 --2- 172.21.15.71:0/828844099 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f31ac06cce0 0x7f31ac06f1a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.045+0000 7f31ca7b3700 1 --2- 172.21.15.71:0/828844099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c4100ea0 0x7f31c419c620 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.045+0000 7f31ca7b3700 1 --2- 172.21.15.71:0/828844099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31c4101850 0x7f31c419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.045+0000 7f31ca7b3700 1 -- 172.21.15.71:0/828844099 >> 172.21.15.71:0/828844099 conn(0x7f31c40fc750 msgr2=0x7f31c40fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:32.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.045+0000 7f31ca7b3700 1 -- 172.21.15.71:0/828844099 shutdown_connections 2024-04-23T02:20:32.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.045+0000 7f31ca7b3700 1 -- 172.21.15.71:0/828844099 wait complete. 2024-04-23T02:20:32.059 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:20:32.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.151+0000 7f6a50293700 1 -- 172.21.15.71:0/1197317057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a48105bd0 msgr2=0x7f6a48105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:32.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.151+0000 7f6a50293700 1 --2- 172.21.15.71:0/1197317057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a48105bd0 0x7f6a48105fb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009910 tx=0x7f6a44009c20 comp rx=0 tx=0).stop 2024-04-23T02:20:32.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.152+0000 7f6a50293700 1 -- 172.21.15.71:0/1197317057 shutdown_connections 2024-04-23T02:20:32.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.152+0000 7f6a50293700 1 --2- 172.21.15.71:0/1197317057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a480ffc20 0x7f6a48100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.152+0000 7f6a50293700 1 --2- 172.21.15.71:0/1197317057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a48105bd0 0x7f6a48105fb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.152+0000 7f6a50293700 1 -- 172.21.15.71:0/1197317057 >> 172.21.15.71:0/1197317057 conn(0x7f6a48075670 msgr2=0x7f6a48075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:32.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.152+0000 7f6a50293700 1 -- 172.21.15.71:0/1197317057 shutdown_connections 2024-04-23T02:20:32.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.152+0000 7f6a50293700 1 -- 172.21.15.71:0/1197317057 wait complete. 2024-04-23T02:20:32.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.152+0000 7f6a50293700 1 Processor -- start 2024-04-23T02:20:32.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.152+0000 7f6a50293700 1 -- start start 2024-04-23T02:20:32.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.152+0000 7f6a50293700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a480ffc20 0x7f6a4819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:32.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.152+0000 7f6a50293700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a48105bd0 0x7f6a4819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:32.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.152+0000 7f6a50293700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4819d150 con 0x7f6a48105bd0 2024-04-23T02:20:32.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.152+0000 7f6a50293700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a48196600 con 0x7f6a480ffc20 2024-04-23T02:20:32.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.153+0000 7f6a4d82e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a48105bd0 0x7f6a4819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:32.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.153+0000 7f6a4d82e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a48105bd0 0x7f6a4819cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51662/0 (socket says 172.21.15.71:51662) 2024-04-23T02:20:32.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.153+0000 7f6a4d82e700 1 -- 172.21.15.71:0/656308432 learned_addr learned my addr 172.21.15.71:0/656308432 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:20:32.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.153+0000 7f6a4e02f700 1 --2- 172.21.15.71:0/656308432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a480ffc20 0x7f6a4819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:32.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.153+0000 7f6a4d82e700 1 -- 172.21.15.71:0/656308432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a480ffc20 msgr2=0x7f6a4819c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:32.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.153+0000 7f6a4d82e700 1 --2- 172.21.15.71:0/656308432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a480ffc20 0x7f6a4819c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.153+0000 7f6a4d82e700 1 -- 172.21.15.71:0/656308432 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a440095f0 con 0x7f6a48105bd0 2024-04-23T02:20:32.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.153+0000 7f6a4d82e700 1 --2- 172.21.15.71:0/656308432 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a48105bd0 0x7f6a4819cac0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800d8d0 tx=0x7f6a3800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:32.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.153+0000 7f6a3f7fe700 1 -- 172.21.15.71:0/656308432 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a380098a0 con 0x7f6a48105bd0 2024-04-23T02:20:32.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.153+0000 7f6a3f7fe700 1 -- 172.21.15.71:0/656308432 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6a38010460 con 0x7f6a48105bd0 2024-04-23T02:20:32.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.153+0000 7f6a50293700 1 -- 172.21.15.71:0/656308432 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a481968e0 con 0x7f6a48105bd0 2024-04-23T02:20:32.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.153+0000 7f6a3f7fe700 1 -- 172.21.15.71:0/656308432 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a38009ea0 con 0x7f6a48105bd0 2024-04-23T02:20:32.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.153+0000 7f6a50293700 1 -- 172.21.15.71:0/656308432 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a48196e30 con 0x7f6a48105bd0 2024-04-23T02:20:32.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.154+0000 7f6a50293700 1 -- 172.21.15.71:0/656308432 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4804f2e0 con 0x7f6a48105bd0 2024-04-23T02:20:32.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.155+0000 7f6a3f7fe700 1 -- 172.21.15.71:0/656308432 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6a380105d0 con 0x7f6a48105bd0 2024-04-23T02:20:32.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.155+0000 7f6a3f7fe700 1 --2- 172.21.15.71:0/656308432 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6a3406cce0 0x7f6a3406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:32.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.155+0000 7f6a3f7fe700 1 -- 172.21.15.71:0/656308432 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6a3808c2f0 con 0x7f6a48105bd0 2024-04-23T02:20:32.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.155+0000 7f6a4e02f700 1 --2- 172.21.15.71:0/656308432 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6a3406cce0 0x7f6a3406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:32.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.155+0000 7f6a4e02f700 1 --2- 172.21.15.71:0/656308432 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6a3406cce0 0x7f6a3406f1a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6a44006010 tx=0x7f6a4401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:32.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.160+0000 7f6a3f7fe700 1 -- 172.21.15.71:0/656308432 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6a38059f20 con 0x7f6a48105bd0 2024-04-23T02:20:32.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:31 smithi071 ceph-mon[144544]: pgmap v196: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T02:20:32.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.374+0000 7f6a50293700 1 -- 172.21.15.71:0/656308432 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a48197100 con 0x7f6a3406cce0 2024-04-23T02:20:32.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.375+0000 7f6a3f7fe700 1 -- 172.21.15.71:0/656308432 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f6a48197100 con 0x7f6a3406cce0 2024-04-23T02:20:32.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.377+0000 7f6a50293700 1 -- 172.21.15.71:0/656308432 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6a3406cce0 msgr2=0x7f6a3406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:32.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.377+0000 7f6a50293700 1 --2- 172.21.15.71:0/656308432 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6a3406cce0 0x7f6a3406f1a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6a44006010 tx=0x7f6a4401a040 comp rx=0 tx=0).stop 2024-04-23T02:20:32.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.377+0000 7f6a50293700 1 -- 172.21.15.71:0/656308432 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a48105bd0 msgr2=0x7f6a4819cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:32.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.377+0000 7f6a50293700 1 --2- 172.21.15.71:0/656308432 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a48105bd0 0x7f6a4819cac0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800d8d0 tx=0x7f6a3800dbe0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.377+0000 7f6a50293700 1 -- 172.21.15.71:0/656308432 shutdown_connections 2024-04-23T02:20:32.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.377+0000 7f6a50293700 1 --2- 172.21.15.71:0/656308432 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6a3406cce0 0x7f6a3406f1a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.377+0000 7f6a50293700 1 --2- 172.21.15.71:0/656308432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a480ffc20 0x7f6a4819c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.377+0000 7f6a50293700 1 --2- 172.21.15.71:0/656308432 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a48105bd0 0x7f6a4819cac0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.377+0000 7f6a50293700 1 -- 172.21.15.71:0/656308432 >> 172.21.15.71:0/656308432 conn(0x7f6a48075670 msgr2=0x7f6a48068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:32.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.377+0000 7f6a50293700 1 -- 172.21.15.71:0/656308432 shutdown_connections 2024-04-23T02:20:32.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.377+0000 7f6a50293700 1 -- 172.21.15.71:0/656308432 wait complete. 2024-04-23T02:20:32.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.479+0000 7ffa93df3700 1 -- 172.21.15.71:0/1323607290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa8c0ffc20 msgr2=0x7ffa8c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:32.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.479+0000 7ffa93df3700 1 --2- 172.21.15.71:0/1323607290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa8c0ffc20 0x7ffa8c100080 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009910 tx=0x7ffa7c009c20 comp rx=0 tx=0).stop 2024-04-23T02:20:32.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.479+0000 7ffa93df3700 1 -- 172.21.15.71:0/1323607290 shutdown_connections 2024-04-23T02:20:32.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.479+0000 7ffa93df3700 1 --2- 172.21.15.71:0/1323607290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa8c0ffc20 0x7ffa8c100080 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.479+0000 7ffa93df3700 1 --2- 172.21.15.71:0/1323607290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa8c105bd0 0x7ffa8c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.479+0000 7ffa93df3700 1 -- 172.21.15.71:0/1323607290 >> 172.21.15.71:0/1323607290 conn(0x7ffa8c075670 msgr2=0x7ffa8c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:32.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.480+0000 7ffa93df3700 1 -- 172.21.15.71:0/1323607290 shutdown_connections 2024-04-23T02:20:32.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.480+0000 7ffa93df3700 1 -- 172.21.15.71:0/1323607290 wait complete. 2024-04-23T02:20:32.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.480+0000 7ffa93df3700 1 Processor -- start 2024-04-23T02:20:32.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.480+0000 7ffa93df3700 1 -- start start 2024-04-23T02:20:32.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.480+0000 7ffa93df3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa8c0ffc20 0x7ffa8c19b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:32.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.480+0000 7ffa93df3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa8c105bd0 0x7ffa8c19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:32.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.480+0000 7ffa93df3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c19c0f0 con 0x7ffa8c0ffc20 2024-04-23T02:20:32.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.480+0000 7ffa93df3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c1955a0 con 0x7ffa8c105bd0 2024-04-23T02:20:32.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.480+0000 7ffa91b8f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa8c0ffc20 0x7ffa8c19b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:32.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.481+0000 7ffa91b8f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa8c0ffc20 0x7ffa8c19b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51668/0 (socket says 172.21.15.71:51668) 2024-04-23T02:20:32.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.481+0000 7ffa91b8f700 1 -- 172.21.15.71:0/4261113388 learned_addr learned my addr 172.21.15.71:0/4261113388 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:20:32.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.481+0000 7ffa9138e700 1 --2- 172.21.15.71:0/4261113388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa8c105bd0 0x7ffa8c19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:32.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.481+0000 7ffa91b8f700 1 -- 172.21.15.71:0/4261113388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa8c105bd0 msgr2=0x7ffa8c19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:32.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.481+0000 7ffa91b8f700 1 --2- 172.21.15.71:0/4261113388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa8c105bd0 0x7ffa8c19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.481+0000 7ffa91b8f700 1 -- 172.21.15.71:0/4261113388 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7c0095f0 con 0x7ffa8c0ffc20 2024-04-23T02:20:32.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.481+0000 7ffa91b8f700 1 --2- 172.21.15.71:0/4261113388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa8c0ffc20 0x7ffa8c19b520 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800d8d0 tx=0x7ffa8800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:32.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.481+0000 7ffa82ffd700 1 -- 172.21.15.71:0/4261113388 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa880098a0 con 0x7ffa8c0ffc20 2024-04-23T02:20:32.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.481+0000 7ffa82ffd700 1 -- 172.21.15.71:0/4261113388 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffa88010460 con 0x7ffa8c0ffc20 2024-04-23T02:20:32.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.481+0000 7ffa93df3700 1 -- 172.21.15.71:0/4261113388 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa8c195880 con 0x7ffa8c0ffc20 2024-04-23T02:20:32.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.481+0000 7ffa82ffd700 1 -- 172.21.15.71:0/4261113388 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa88009ea0 con 0x7ffa8c0ffc20 2024-04-23T02:20:32.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.481+0000 7ffa93df3700 1 -- 172.21.15.71:0/4261113388 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8c195dd0 con 0x7ffa8c0ffc20 2024-04-23T02:20:32.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.482+0000 7ffa93df3700 1 -- 172.21.15.71:0/4261113388 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8c1080d0 con 0x7ffa8c0ffc20 2024-04-23T02:20:32.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.483+0000 7ffa82ffd700 1 -- 172.21.15.71:0/4261113388 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ffa880105d0 con 0x7ffa8c0ffc20 2024-04-23T02:20:32.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.483+0000 7ffa82ffd700 1 --2- 172.21.15.71:0/4261113388 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ffa7806cce0 0x7ffa7806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:32.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.483+0000 7ffa82ffd700 1 -- 172.21.15.71:0/4261113388 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ffa8808c3d0 con 0x7ffa8c0ffc20 2024-04-23T02:20:32.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.483+0000 7ffa9138e700 1 --2- 172.21.15.71:0/4261113388 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ffa7806cce0 0x7ffa7806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:32.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.483+0000 7ffa9138e700 1 --2- 172.21.15.71:0/4261113388 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ffa7806cce0 0x7ffa7806f1a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0fe2d0 tx=0x7ffa7c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:32.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.488+0000 7ffa82ffd700 1 -- 172.21.15.71:0/4261113388 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ffa88059f80 con 0x7ffa8c0ffc20 2024-04-23T02:20:32.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.691+0000 7ffa93df3700 1 -- 172.21.15.71:0/4261113388 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa8c196bc0 con 0x7ffa7806cce0 2024-04-23T02:20:32.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.702+0000 7ffa82ffd700 1 -- 172.21.15.71:0/4261113388 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7ffa8c196bc0 con 0x7ffa7806cce0 2024-04-23T02:20:32.706 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:20:32.706 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (4m) 21s ago 15m 26.3M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:20:32.706 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (15m) 21s ago 15m 8250k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:20:32.706 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (14m) 119s ago 14m 8036k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:20:32.706 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (2m) 21s ago 15m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:20:32.706 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (2m) 119s ago 14m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:20:32.706 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (3m) 21s ago 15m 80.0M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:20:32.706 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (9m) 21s ago 9m 64.9M - 18.2.1 5be31c24972a a515de2a841c 2024-04-23T02:20:32.706 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (9m) 21s ago 9m 133M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:20:32.707 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (9m) 119s ago 9m 68.1M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:20:32.707 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (9m) 119s ago 9m 122M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:20:32.707 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (6m) 21s ago 17m 507M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:20:32.707 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (5m) 119s ago 14m 451M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:20:32.707 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (2m) 21s ago 17m 47.1M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:20:32.707 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (2m) 119s ago 14m 31.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:20:32.707 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (5m) 21s ago 15m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:20:32.707 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (5m) 119s ago 14m 17.8M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:20:32.707 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (101s) 21s ago 13m 85.3M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:20:32.707 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (64s) 21s ago 13m 85.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:20:32.707 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (28s) 21s ago 12m 10.9M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:20:32.707 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (12m) 119s ago 12m 243M 2251M 18.2.1 5be31c24972a d5f587ee3a22 2024-04-23T02:20:32.707 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (11m) 119s ago 11m 219M 2251M 18.2.1 5be31c24972a 958d046005f0 2024-04-23T02:20:32.707 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (11m) 119s ago 11m 199M 2251M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:20:32.708 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (4m) 21s ago 14m 65.6M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:20:32.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.704+0000 7ffa93df3700 1 -- 172.21.15.71:0/4261113388 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ffa7806cce0 msgr2=0x7ffa7806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:32.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.704+0000 7ffa93df3700 1 --2- 172.21.15.71:0/4261113388 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ffa7806cce0 0x7ffa7806f1a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0fe2d0 tx=0x7ffa7c01a040 comp rx=0 tx=0).stop 2024-04-23T02:20:32.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.704+0000 7ffa93df3700 1 -- 172.21.15.71:0/4261113388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa8c0ffc20 msgr2=0x7ffa8c19b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:32.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.704+0000 7ffa93df3700 1 --2- 172.21.15.71:0/4261113388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa8c0ffc20 0x7ffa8c19b520 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800d8d0 tx=0x7ffa8800dbe0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.704+0000 7ffa93df3700 1 -- 172.21.15.71:0/4261113388 shutdown_connections 2024-04-23T02:20:32.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.704+0000 7ffa93df3700 1 --2- 172.21.15.71:0/4261113388 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ffa7806cce0 0x7ffa7806f1a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.704+0000 7ffa93df3700 1 --2- 172.21.15.71:0/4261113388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa8c0ffc20 0x7ffa8c19b520 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.704+0000 7ffa93df3700 1 --2- 172.21.15.71:0/4261113388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa8c105bd0 0x7ffa8c19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.704+0000 7ffa93df3700 1 -- 172.21.15.71:0/4261113388 >> 172.21.15.71:0/4261113388 conn(0x7ffa8c075670 msgr2=0x7ffa8c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:32.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.704+0000 7ffa93df3700 1 -- 172.21.15.71:0/4261113388 shutdown_connections 2024-04-23T02:20:32.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.704+0000 7ffa93df3700 1 -- 172.21.15.71:0/4261113388 wait complete. 2024-04-23T02:20:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.806+0000 7f80fda8d700 1 -- 172.21.15.71:0/3142352293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80f8106cd0 msgr2=0x7f80f81070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.806+0000 7f80fda8d700 1 --2- 172.21.15.71:0/3142352293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80f8106cd0 0x7f80f81070b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009910 tx=0x7f80e0009c20 comp rx=0 tx=0).stop 2024-04-23T02:20:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.806+0000 7f80fda8d700 1 -- 172.21.15.71:0/3142352293 shutdown_connections 2024-04-23T02:20:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.806+0000 7f80fda8d700 1 --2- 172.21.15.71:0/3142352293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80f8068580 0x7f80f80689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.806+0000 7f80fda8d700 1 --2- 172.21.15.71:0/3142352293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80f8106cd0 0x7f80f81070b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.806+0000 7f80fda8d700 1 -- 172.21.15.71:0/3142352293 >> 172.21.15.71:0/3142352293 conn(0x7f80f8075670 msgr2=0x7f80f8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.806+0000 7f80fda8d700 1 -- 172.21.15.71:0/3142352293 shutdown_connections 2024-04-23T02:20:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.806+0000 7f80fda8d700 1 -- 172.21.15.71:0/3142352293 wait complete. 2024-04-23T02:20:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.806+0000 7f80fda8d700 1 Processor -- start 2024-04-23T02:20:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.807+0000 7f80fda8d700 1 -- start start 2024-04-23T02:20:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.807+0000 7f80fda8d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80f8068580 0x7f80f819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.807+0000 7f80fda8d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80f8106cd0 0x7f80f819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.807+0000 7f80fda8d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f819d130 con 0x7f80f8068580 2024-04-23T02:20:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.807+0000 7f80fda8d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f8196630 con 0x7f80f8106cd0 2024-04-23T02:20:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.807+0000 7f80f77fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80f8068580 0x7f80f819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.807+0000 7f80f77fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80f8068580 0x7f80f819c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51678/0 (socket says 172.21.15.71:51678) 2024-04-23T02:20:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.807+0000 7f80f77fe700 1 -- 172.21.15.71:0/3678384583 learned_addr learned my addr 172.21.15.71:0/3678384583 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:20:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.807+0000 7f80f6ffd700 1 --2- 172.21.15.71:0/3678384583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80f8106cd0 0x7f80f819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.807+0000 7f80f77fe700 1 -- 172.21.15.71:0/3678384583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80f8106cd0 msgr2=0x7f80f819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.807+0000 7f80f77fe700 1 --2- 172.21.15.71:0/3678384583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80f8106cd0 0x7f80f819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:32.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.807+0000 7f80f77fe700 1 -- 172.21.15.71:0/3678384583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e00095f0 con 0x7f80f8068580 2024-04-23T02:20:32.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.807+0000 7f80f77fe700 1 --2- 172.21.15.71:0/3678384583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80f8068580 0x7f80f819c560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f80e00092e0 tx=0x7f80e000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:32.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.808+0000 7f80f4ff9700 1 -- 172.21.15.71:0/3678384583 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80e001d070 con 0x7f80f8068580 2024-04-23T02:20:32.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.808+0000 7f80f4ff9700 1 -- 172.21.15.71:0/3678384583 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f80e0005e00 con 0x7f80f8068580 2024-04-23T02:20:32.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.808+0000 7f80fda8d700 1 -- 172.21.15.71:0/3678384583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f81968b0 con 0x7f80f8068580 2024-04-23T02:20:32.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.808+0000 7f80f4ff9700 1 -- 172.21.15.71:0/3678384583 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80e000fe90 con 0x7f80f8068580 2024-04-23T02:20:32.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.808+0000 7f80fda8d700 1 -- 172.21.15.71:0/3678384583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f8196da0 con 0x7f80f8068580 2024-04-23T02:20:32.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.809+0000 7f80fda8d700 1 -- 172.21.15.71:0/3678384583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f81091f0 con 0x7f80f8068580 2024-04-23T02:20:32.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.809+0000 7f80f4ff9700 1 -- 172.21.15.71:0/3678384583 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f80e0003a60 con 0x7f80f8068580 2024-04-23T02:20:32.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.809+0000 7f80f4ff9700 1 --2- 172.21.15.71:0/3678384583 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f80e406cce0 0x7f80e406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:32.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.810+0000 7f80f6ffd700 1 --2- 172.21.15.71:0/3678384583 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f80e406cce0 0x7f80e406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:32.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.810+0000 7f80f4ff9700 1 -- 172.21.15.71:0/3678384583 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f80e008d780 con 0x7f80f8068580 2024-04-23T02:20:32.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.810+0000 7f80f6ffd700 1 --2- 172.21.15.71:0/3678384583 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f80e406cce0 0x7f80e406f1a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f80f80fe390 tx=0x7f80e800b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:32.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:32.814+0000 7f80f4ff9700 1 -- 172.21.15.71:0/3678384583 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f80e005b330 con 0x7f80f8068580 2024-04-23T02:20:33.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.090+0000 7f80fda8d700 1 -- 172.21.15.71:0/3678384583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f80f8197bd0 con 0x7f80f8068580 2024-04-23T02:20:33.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.091+0000 7f80f4ff9700 1 -- 172.21.15.71:0/3678384583 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7f80e0057030 con 0x7f80f8068580 2024-04-23T02:20:33.094 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:20:33.094 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:20:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:20:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:20:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:20:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:20:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:20:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:20:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-04-23T02:20:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:20:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:20:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:20:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-23T02:20:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:20:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:20:33.096 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 7, 2024-04-23T02:20:33.096 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 7 2024-04-23T02:20:33.096 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:20:33.096 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:20:33.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.093+0000 7f80fda8d700 1 -- 172.21.15.71:0/3678384583 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f80e406cce0 msgr2=0x7f80e406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:33.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.093+0000 7f80fda8d700 1 --2- 172.21.15.71:0/3678384583 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f80e406cce0 0x7f80e406f1a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f80f80fe390 tx=0x7f80e800b410 comp rx=0 tx=0).stop 2024-04-23T02:20:33.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.093+0000 7f80fda8d700 1 -- 172.21.15.71:0/3678384583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80f8068580 msgr2=0x7f80f819c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:33.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.093+0000 7f80fda8d700 1 --2- 172.21.15.71:0/3678384583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80f8068580 0x7f80f819c560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f80e00092e0 tx=0x7f80e000bd70 comp rx=0 tx=0).stop 2024-04-23T02:20:33.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.093+0000 7f80fda8d700 1 -- 172.21.15.71:0/3678384583 shutdown_connections 2024-04-23T02:20:33.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.093+0000 7f80fda8d700 1 --2- 172.21.15.71:0/3678384583 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f80e406cce0 0x7f80e406f1a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.093+0000 7f80fda8d700 1 --2- 172.21.15.71:0/3678384583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80f8068580 0x7f80f819c560 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.093+0000 7f80fda8d700 1 --2- 172.21.15.71:0/3678384583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80f8106cd0 0x7f80f819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.093+0000 7f80fda8d700 1 -- 172.21.15.71:0/3678384583 >> 172.21.15.71:0/3678384583 conn(0x7f80f8075670 msgr2=0x7f80f80fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:33.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.093+0000 7f80fda8d700 1 -- 172.21.15.71:0/3678384583 shutdown_connections 2024-04-23T02:20:33.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.093+0000 7f80fda8d700 1 -- 172.21.15.71:0/3678384583 wait complete. 2024-04-23T02:20:33.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.195+0000 7f363539f700 1 -- 172.21.15.71:0/3820210880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3630101e00 msgr2=0x7f36301021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:33.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.195+0000 7f363539f700 1 --2- 172.21.15.71:0/3820210880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3630101e00 0x7f36301021e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3624009910 tx=0x7f3624009c20 comp rx=0 tx=0).stop 2024-04-23T02:20:33.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.196+0000 7f363539f700 1 -- 172.21.15.71:0/3820210880 shutdown_connections 2024-04-23T02:20:33.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.196+0000 7f363539f700 1 --2- 172.21.15.71:0/3820210880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3630102720 0x7f363010ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.196+0000 7f363539f700 1 --2- 172.21.15.71:0/3820210880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3630101e00 0x7f36301021e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.196+0000 7f363539f700 1 -- 172.21.15.71:0/3820210880 >> 172.21.15.71:0/3820210880 conn(0x7f36300fb650 msgr2=0x7f36300fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:33.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.196+0000 7f363539f700 1 -- 172.21.15.71:0/3820210880 shutdown_connections 2024-04-23T02:20:33.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.196+0000 7f363539f700 1 -- 172.21.15.71:0/3820210880 wait complete. 2024-04-23T02:20:33.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.196+0000 7f363539f700 1 Processor -- start 2024-04-23T02:20:33.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.196+0000 7f363539f700 1 -- start start 2024-04-23T02:20:33.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.197+0000 7f363539f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3630101e00 0x7f363019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:33.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.197+0000 7f363539f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3630102720 0x7f363019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:33.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.197+0000 7f363539f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363019d200 con 0x7f3630101e00 2024-04-23T02:20:33.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.197+0000 7f363539f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36301966b0 con 0x7f3630102720 2024-04-23T02:20:33.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.197+0000 7f362effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3630101e00 0x7f363019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:33.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.197+0000 7f362e7fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3630102720 0x7f363019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:33.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.197+0000 7f362effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3630101e00 0x7f363019c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51698/0 (socket says 172.21.15.71:51698) 2024-04-23T02:20:33.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.197+0000 7f362effd700 1 -- 172.21.15.71:0/1649498405 learned_addr learned my addr 172.21.15.71:0/1649498405 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:20:33.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.197+0000 7f362effd700 1 -- 172.21.15.71:0/1649498405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3630102720 msgr2=0x7f363019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:33.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.197+0000 7f362effd700 1 --2- 172.21.15.71:0/1649498405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3630102720 0x7f363019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.197+0000 7f362effd700 1 -- 172.21.15.71:0/1649498405 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36240095f0 con 0x7f3630101e00 2024-04-23T02:20:33.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.197+0000 7f362e7fc700 1 --2- 172.21.15.71:0/1649498405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3630102720 0x7f363019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T02:20:33.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.197+0000 7f362effd700 1 --2- 172.21.15.71:0/1649498405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3630101e00 0x7f363019c630 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3624000c00 tx=0x7f362400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:33.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.197+0000 7f3617fff700 1 -- 172.21.15.71:0/1649498405 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f362401d070 con 0x7f3630101e00 2024-04-23T02:20:33.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.197+0000 7f3617fff700 1 -- 172.21.15.71:0/1649498405 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3624005dc0 con 0x7f3630101e00 2024-04-23T02:20:33.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.197+0000 7f3617fff700 1 -- 172.21.15.71:0/1649498405 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3624003940 con 0x7f3630101e00 2024-04-23T02:20:33.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.197+0000 7f363539f700 1 -- 172.21.15.71:0/1649498405 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3630196930 con 0x7f3630101e00 2024-04-23T02:20:33.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.197+0000 7f363539f700 1 -- 172.21.15.71:0/1649498405 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3630196dc0 con 0x7f3630101e00 2024-04-23T02:20:33.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.198+0000 7f363539f700 1 -- 172.21.15.71:0/1649498405 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3630108310 con 0x7f3630101e00 2024-04-23T02:20:33.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.199+0000 7f3617fff700 1 -- 172.21.15.71:0/1649498405 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3624004ae0 con 0x7f3630101e00 2024-04-23T02:20:33.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.199+0000 7f3617fff700 1 --2- 172.21.15.71:0/1649498405 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f361806cce0 0x7f361806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:33.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.199+0000 7f362e7fc700 1 --2- 172.21.15.71:0/1649498405 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f361806cce0 0x7f361806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:33.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.199+0000 7f3617fff700 1 -- 172.21.15.71:0/1649498405 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f362408da20 con 0x7f3630101e00 2024-04-23T02:20:33.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.200+0000 7f362e7fc700 1 --2- 172.21.15.71:0/1649498405 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f361806cce0 0x7f361806f1a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f36300fce80 tx=0x7f3620009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:33.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.204+0000 7f3617fff700 1 -- 172.21.15.71:0/1649498405 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f362405b5d0 con 0x7f3630101e00 2024-04-23T02:20:33.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:33 smithi192 ceph-mon[108617]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:20:33.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:33 smithi192 ceph-mon[108617]: Upgrade: Updating osd.3 2024-04-23T02:20:33.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:33 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:33.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:33 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T02:20:33.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:33 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:20:33.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:33 smithi192 ceph-mon[108617]: Deploying daemon osd.3 on smithi192 2024-04-23T02:20:33.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.436+0000 7f363539f700 1 -- 172.21.15.71:0/1649498405 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3630197bf0 con 0x7f3630101e00 2024-04-23T02:20:33.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.436+0000 7f3617fff700 1 -- 172.21.15.71:0/1649498405 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1869 (secure 0 0 0) 0x7f362405b160 con 0x7f3630101e00 2024-04-23T02:20:33.440 INFO:teuthology.orchestra.run.smithi071.stdout:e25 2024-04-23T02:20:33.440 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:20:33.440 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:20:33.440 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:20:33.440 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:20:33.440 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:20:33.440 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:20:33.441 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 25 2024-04-23T02:20:33.441 INFO:teuthology.orchestra.run.smithi071.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T02:20:33.441 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:20:33.441 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:17:27.814232+0000 2024-04-23T02:20:33.441 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:20:33.441 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:20:33.441 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:20:33.441 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:20:33.441 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:20:33.441 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:20:33.441 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:20:33.441 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:20:33.441 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 46 2024-04-23T02:20:33.441 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:20:33.441 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:20:33.442 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:20:33.442 INFO:teuthology.orchestra.run.smithi071.stdout:up {0=14478,1=24243} 2024-04-23T02:20:33.442 INFO:teuthology.orchestra.run.smithi071.stdout:failed 2024-04-23T02:20:33.442 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:20:33.442 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:20:33.442 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:20:33.442 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:20:33.442 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:20:33.442 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:20:33.442 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:20:33.442 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:20:33.442 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:20:33.442 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/3442634960,v1:172.21.15.71:6827/3442634960] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:20:33.442 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{1:24243} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.192:6824/3780549896,v1:172.21.15.192:6825/3780549896] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:20:33.442 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{1:24255} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2100078284,v1:172.21.15.192:6827/2100078284] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:20:33.443 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:20:33.443 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:20:33.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.438+0000 7f363539f700 1 -- 172.21.15.71:0/1649498405 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f361806cce0 msgr2=0x7f361806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:33.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.438+0000 7f363539f700 1 --2- 172.21.15.71:0/1649498405 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f361806cce0 0x7f361806f1a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f36300fce80 tx=0x7f3620009260 comp rx=0 tx=0).stop 2024-04-23T02:20:33.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.438+0000 7f363539f700 1 -- 172.21.15.71:0/1649498405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3630101e00 msgr2=0x7f363019c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:33.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.438+0000 7f363539f700 1 --2- 172.21.15.71:0/1649498405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3630101e00 0x7f363019c630 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3624000c00 tx=0x7f362400bde0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.439+0000 7f363539f700 1 -- 172.21.15.71:0/1649498405 shutdown_connections 2024-04-23T02:20:33.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.439+0000 7f363539f700 1 --2- 172.21.15.71:0/1649498405 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f361806cce0 0x7f361806f1a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.439+0000 7f363539f700 1 --2- 172.21.15.71:0/1649498405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3630101e00 0x7f363019c630 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.439+0000 7f363539f700 1 --2- 172.21.15.71:0/1649498405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3630102720 0x7f363019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.439+0000 7f363539f700 1 -- 172.21.15.71:0/1649498405 >> 172.21.15.71:0/1649498405 conn(0x7f36300fb650 msgr2=0x7f3630105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:33.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.439+0000 7f363539f700 1 -- 172.21.15.71:0/1649498405 shutdown_connections 2024-04-23T02:20:33.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.439+0000 7f363539f700 1 -- 172.21.15.71:0/1649498405 wait complete. 2024-04-23T02:20:33.444 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 25 2024-04-23T02:20:33.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:33 smithi071 ceph-mon[144544]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:20:33.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:33 smithi071 ceph-mon[144544]: Upgrade: Updating osd.3 2024-04-23T02:20:33.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:33 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:33.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:33 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T02:20:33.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:33 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:20:33.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:33 smithi071 ceph-mon[144544]: Deploying daemon osd.3 on smithi192 2024-04-23T02:20:33.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.544+0000 7fb72bfde700 1 -- 172.21.15.71:0/665877056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7240ffc20 msgr2=0x7fb724100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:33.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.544+0000 7fb72bfde700 1 --2- 172.21.15.71:0/665877056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7240ffc20 0x7fb724100080 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb720009910 tx=0x7fb720009c20 comp rx=0 tx=0).stop 2024-04-23T02:20:33.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.544+0000 7fb72bfde700 1 -- 172.21.15.71:0/665877056 shutdown_connections 2024-04-23T02:20:33.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.544+0000 7fb72bfde700 1 --2- 172.21.15.71:0/665877056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7240ffc20 0x7fb724100080 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.544+0000 7fb72bfde700 1 --2- 172.21.15.71:0/665877056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb724105bd0 0x7fb724105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.544+0000 7fb72bfde700 1 -- 172.21.15.71:0/665877056 >> 172.21.15.71:0/665877056 conn(0x7fb724075670 msgr2=0x7fb724075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:33.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.544+0000 7fb72bfde700 1 -- 172.21.15.71:0/665877056 shutdown_connections 2024-04-23T02:20:33.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.544+0000 7fb72bfde700 1 -- 172.21.15.71:0/665877056 wait complete. 2024-04-23T02:20:33.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.544+0000 7fb72bfde700 1 Processor -- start 2024-04-23T02:20:33.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.544+0000 7fb72bfde700 1 -- start start 2024-04-23T02:20:33.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.544+0000 7fb72bfde700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7240ffc20 0x7fb724197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:33.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.544+0000 7fb72bfde700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb724105bd0 0x7fb724197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:33.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.544+0000 7fb72bfde700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7241981f0 con 0x7fb7240ffc20 2024-04-23T02:20:33.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.544+0000 7fb72bfde700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb724198330 con 0x7fb724105bd0 2024-04-23T02:20:33.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.545+0000 7fb729d7a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7240ffc20 0x7fb724197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:33.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.545+0000 7fb729d7a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7240ffc20 0x7fb724197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51720/0 (socket says 172.21.15.71:51720) 2024-04-23T02:20:33.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.545+0000 7fb729d7a700 1 -- 172.21.15.71:0/1948902091 learned_addr learned my addr 172.21.15.71:0/1948902091 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:20:33.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.545+0000 7fb729579700 1 --2- 172.21.15.71:0/1948902091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb724105bd0 0x7fb724197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:33.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.545+0000 7fb729d7a700 1 -- 172.21.15.71:0/1948902091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb724105bd0 msgr2=0x7fb724197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:33.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.545+0000 7fb729d7a700 1 --2- 172.21.15.71:0/1948902091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb724105bd0 0x7fb724197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.545+0000 7fb729d7a700 1 -- 172.21.15.71:0/1948902091 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7200095f0 con 0x7fb7240ffc20 2024-04-23T02:20:33.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.545+0000 7fb729d7a700 1 --2- 172.21.15.71:0/1948902091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7240ffc20 0x7fb724197660 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb71400c930 tx=0x7fb71400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:33.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.545+0000 7fb71affd700 1 -- 172.21.15.71:0/1948902091 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb714007ad0 con 0x7fb7240ffc20 2024-04-23T02:20:33.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.545+0000 7fb71affd700 1 -- 172.21.15.71:0/1948902091 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb71400ce80 con 0x7fb7240ffc20 2024-04-23T02:20:33.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.545+0000 7fb72bfde700 1 -- 172.21.15.71:0/1948902091 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7241a06d0 con 0x7fb7240ffc20 2024-04-23T02:20:33.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.545+0000 7fb71affd700 1 -- 172.21.15.71:0/1948902091 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb714018630 con 0x7fb7240ffc20 2024-04-23T02:20:33.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.545+0000 7fb72bfde700 1 -- 172.21.15.71:0/1948902091 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7241a0b70 con 0x7fb7240ffc20 2024-04-23T02:20:33.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.546+0000 7fb72bfde700 1 -- 172.21.15.71:0/1948902091 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7241080d0 con 0x7fb7240ffc20 2024-04-23T02:20:33.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.547+0000 7fb71affd700 1 -- 172.21.15.71:0/1948902091 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb714007c30 con 0x7fb7240ffc20 2024-04-23T02:20:33.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.547+0000 7fb71affd700 1 --2- 172.21.15.71:0/1948902091 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb71006cce0 0x7fb71006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:33.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.547+0000 7fb71affd700 1 -- 172.21.15.71:0/1948902091 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb71408c4f0 con 0x7fb7240ffc20 2024-04-23T02:20:33.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.547+0000 7fb729579700 1 --2- 172.21.15.71:0/1948902091 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb71006cce0 0x7fb71006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:33.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.547+0000 7fb729579700 1 --2- 172.21.15.71:0/1948902091 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb71006cce0 0x7fb71006f1a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb724069660 tx=0x7fb72001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:33.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.552+0000 7fb71affd700 1 -- 172.21.15.71:0/1948902091 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb714056610 con 0x7fb7240ffc20 2024-04-23T02:20:33.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.764+0000 7fb72bfde700 1 -- 172.21.15.71:0/1948902091 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7240611d0 con 0x7fb71006cce0 2024-04-23T02:20:33.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.766+0000 7fb71affd700 1 -- 172.21.15.71:0/1948902091 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb7240611d0 con 0x7fb71006cce0 2024-04-23T02:20:33.769 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:20:33.769 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:20:33.769 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:20:33.769 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:20:33.769 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:20:33.770 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:20:33.770 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:20:33.770 INFO:teuthology.orchestra.run.smithi071.stdout: "mon" 2024-04-23T02:20:33.770 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:20:33.770 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "9/23 daemons upgraded", 2024-04-23T02:20:33.770 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading osd daemons", 2024-04-23T02:20:33.770 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:20:33.770 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:20:33.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.768+0000 7fb72bfde700 1 -- 172.21.15.71:0/1948902091 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb71006cce0 msgr2=0x7fb71006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:33.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.768+0000 7fb72bfde700 1 --2- 172.21.15.71:0/1948902091 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb71006cce0 0x7fb71006f1a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb724069660 tx=0x7fb72001a040 comp rx=0 tx=0).stop 2024-04-23T02:20:33.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.768+0000 7fb72bfde700 1 -- 172.21.15.71:0/1948902091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7240ffc20 msgr2=0x7fb724197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:33.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.768+0000 7fb72bfde700 1 --2- 172.21.15.71:0/1948902091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7240ffc20 0x7fb724197660 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb71400c930 tx=0x7fb71400cc40 comp rx=0 tx=0).stop 2024-04-23T02:20:33.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.768+0000 7fb72bfde700 1 -- 172.21.15.71:0/1948902091 shutdown_connections 2024-04-23T02:20:33.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.768+0000 7fb72bfde700 1 --2- 172.21.15.71:0/1948902091 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb71006cce0 0x7fb71006f1a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.768+0000 7fb72bfde700 1 --2- 172.21.15.71:0/1948902091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7240ffc20 0x7fb724197660 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.768+0000 7fb72bfde700 1 --2- 172.21.15.71:0/1948902091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb724105bd0 0x7fb724197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.768+0000 7fb72bfde700 1 -- 172.21.15.71:0/1948902091 >> 172.21.15.71:0/1948902091 conn(0x7fb724075670 msgr2=0x7fb7240fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:33.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.768+0000 7fb72bfde700 1 -- 172.21.15.71:0/1948902091 shutdown_connections 2024-04-23T02:20:33.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.768+0000 7fb72bfde700 1 -- 172.21.15.71:0/1948902091 wait complete. 2024-04-23T02:20:33.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.871+0000 7f3db9e3c700 1 -- 172.21.15.71:0/2703822934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3db4068580 msgr2=0x7f3db4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:33.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.871+0000 7f3db9e3c700 1 --2- 172.21.15.71:0/2703822934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3db4068580 0x7f3db4068960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009910 tx=0x7f3da4009c20 comp rx=0 tx=0).stop 2024-04-23T02:20:33.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.871+0000 7f3db9e3c700 1 -- 172.21.15.71:0/2703822934 shutdown_connections 2024-04-23T02:20:33.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.871+0000 7f3db9e3c700 1 --2- 172.21.15.71:0/2703822934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3db4068ea0 0x7f3db410d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.871+0000 7f3db9e3c700 1 --2- 172.21.15.71:0/2703822934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3db4068580 0x7f3db4068960 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.871+0000 7f3db9e3c700 1 -- 172.21.15.71:0/2703822934 >> 172.21.15.71:0/2703822934 conn(0x7f3db4075670 msgr2=0x7f3db4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:33.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.871+0000 7f3db9e3c700 1 -- 172.21.15.71:0/2703822934 shutdown_connections 2024-04-23T02:20:33.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.871+0000 7f3db9e3c700 1 -- 172.21.15.71:0/2703822934 wait complete. 2024-04-23T02:20:33.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.871+0000 7f3db9e3c700 1 Processor -- start 2024-04-23T02:20:33.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.872+0000 7f3db9e3c700 1 -- start start 2024-04-23T02:20:33.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.872+0000 7f3db9e3c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3db4068580 0x7f3db4198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:33.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.872+0000 7f3db9e3c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3db4068ea0 0x7f3db4198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:33.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.872+0000 7f3db9e3c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db4199550 con 0x7f3db4068ea0 2024-04-23T02:20:33.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.872+0000 7f3db9e3c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db419d360 con 0x7f3db4068580 2024-04-23T02:20:33.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.872+0000 7f3db2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3db4068ea0 0x7f3db4198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:33.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.872+0000 7f3db2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3db4068ea0 0x7f3db4198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51732/0 (socket says 172.21.15.71:51732) 2024-04-23T02:20:33.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.872+0000 7f3db2ffd700 1 -- 172.21.15.71:0/2536111127 learned_addr learned my addr 172.21.15.71:0/2536111127 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:20:33.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.872+0000 7f3db37fe700 1 --2- 172.21.15.71:0/2536111127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3db4068580 0x7f3db4198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:33.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.872+0000 7f3db2ffd700 1 -- 172.21.15.71:0/2536111127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3db4068580 msgr2=0x7f3db4198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:33.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.872+0000 7f3db2ffd700 1 --2- 172.21.15.71:0/2536111127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3db4068580 0x7f3db4198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:33.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.872+0000 7f3db2ffd700 1 -- 172.21.15.71:0/2536111127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da40095f0 con 0x7f3db4068ea0 2024-04-23T02:20:33.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.873+0000 7f3db2ffd700 1 --2- 172.21.15.71:0/2536111127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3db4068ea0 0x7f3db4198ec0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3da800d8d0 tx=0x7f3da800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:33.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.873+0000 7f3db0ff9700 1 -- 172.21.15.71:0/2536111127 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3da80098a0 con 0x7f3db4068ea0 2024-04-23T02:20:33.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.873+0000 7f3db0ff9700 1 -- 172.21.15.71:0/2536111127 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3da8010460 con 0x7f3db4068ea0 2024-04-23T02:20:33.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.873+0000 7f3db0ff9700 1 -- 172.21.15.71:0/2536111127 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3da8009ea0 con 0x7f3db4068ea0 2024-04-23T02:20:33.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.873+0000 7f3db9e3c700 1 -- 172.21.15.71:0/2536111127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db419d640 con 0x7f3db4068ea0 2024-04-23T02:20:33.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.873+0000 7f3db9e3c700 1 -- 172.21.15.71:0/2536111127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db419db90 con 0x7f3db4068ea0 2024-04-23T02:20:33.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.874+0000 7f3db9e3c700 1 -- 172.21.15.71:0/2536111127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db410aa00 con 0x7f3db4068ea0 2024-04-23T02:20:33.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.874+0000 7f3db0ff9700 1 -- 172.21.15.71:0/2536111127 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3da8009a00 con 0x7f3db4068ea0 2024-04-23T02:20:33.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.875+0000 7f3db0ff9700 1 --2- 172.21.15.71:0/2536111127 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3d9c06cce0 0x7f3d9c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:20:33.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.875+0000 7f3db37fe700 1 --2- 172.21.15.71:0/2536111127 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3d9c06cce0 0x7f3d9c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:20:33.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.875+0000 7f3db0ff9700 1 -- 172.21.15.71:0/2536111127 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3da808c3a0 con 0x7f3db4068ea0 2024-04-23T02:20:33.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.875+0000 7f3db37fe700 1 --2- 172.21.15.71:0/2536111127 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3d9c06cce0 0x7f3d9c06f1a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3da4006010 tx=0x7f3da401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:20:33.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:33.879+0000 7f3db0ff9700 1 -- 172.21.15.71:0/2536111127 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3da8059f50 con 0x7f3db4068ea0 2024-04-23T02:20:34.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:34.153+0000 7f3db9e3c700 1 -- 172.21.15.71:0/2536111127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3db404f2e0 con 0x7f3db4068ea0 2024-04-23T02:20:34.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:34.153+0000 7f3db0ff9700 1 -- 172.21.15.71:0/2536111127 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f3da8059ae0 con 0x7f3db4068ea0 2024-04-23T02:20:34.157 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-23T02:20:34.157 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:20:34.158 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:20:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:34.156+0000 7f3db9e3c700 1 -- 172.21.15.71:0/2536111127 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3d9c06cce0 msgr2=0x7f3d9c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:34.156+0000 7f3db9e3c700 1 --2- 172.21.15.71:0/2536111127 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3d9c06cce0 0x7f3d9c06f1a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3da4006010 tx=0x7f3da401a040 comp rx=0 tx=0).stop 2024-04-23T02:20:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:34.156+0000 7f3db9e3c700 1 -- 172.21.15.71:0/2536111127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3db4068ea0 msgr2=0x7f3db4198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:20:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:34.156+0000 7f3db9e3c700 1 --2- 172.21.15.71:0/2536111127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3db4068ea0 0x7f3db4198ec0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3da800d8d0 tx=0x7f3da800dbe0 comp rx=0 tx=0).stop 2024-04-23T02:20:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:34.156+0000 7f3db9e3c700 1 -- 172.21.15.71:0/2536111127 shutdown_connections 2024-04-23T02:20:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:34.156+0000 7f3db9e3c700 1 --2- 172.21.15.71:0/2536111127 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3d9c06cce0 0x7f3d9c06f1a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:34.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:34.156+0000 7f3db9e3c700 1 --2- 172.21.15.71:0/2536111127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3db4068580 0x7f3db4198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:34.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:34.156+0000 7f3db9e3c700 1 --2- 172.21.15.71:0/2536111127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3db4068ea0 0x7f3db4198ec0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:20:34.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:34.156+0000 7f3db9e3c700 1 -- 172.21.15.71:0/2536111127 >> 172.21.15.71:0/2536111127 conn(0x7f3db4075670 msgr2=0x7f3db40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:20:34.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:34.156+0000 7f3db9e3c700 1 -- 172.21.15.71:0/2536111127 shutdown_connections 2024-04-23T02:20:34.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:20:34.156+0000 7f3db9e3c700 1 -- 172.21.15.71:0/2536111127 wait complete. 2024-04-23T02:20:34.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:34 smithi192 ceph-mon[108617]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:20:34.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:34 smithi192 ceph-mon[108617]: from='client.34232 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:20:34.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:34 smithi192 ceph-mon[108617]: pgmap v197: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-04-23T02:20:34.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:34 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3678384583' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:20:34.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:34 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1649498405' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:20:34.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:34 smithi071 ceph-mon[144544]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:20:34.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:34 smithi071 ceph-mon[144544]: from='client.34232 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:20:34.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:34 smithi071 ceph-mon[144544]: pgmap v197: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-04-23T02:20:34.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:34 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3678384583' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:20:34.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:34 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1649498405' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:20:35.334 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:35 smithi192 ceph-mon[108617]: from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:20:35.334 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:35 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2536111127' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:20:35.334 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:20:35.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:35 smithi071 ceph-mon[144544]: from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:20:35.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:35 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2536111127' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:20:35.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:20:36.092 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:35 smithi192 systemd[1]: Stopping Ceph osd.3 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T02:20:36.092 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:35 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-3[56562]: 2024-04-23T02:20:35.902+0000 7f780e549700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T02:20:36.092 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:35 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-3[56562]: 2024-04-23T02:20:35.902+0000 7f780e549700 -1 osd.3 63 *** Got signal Terminated *** 2024-04-23T02:20:36.093 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:35 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-3[56562]: 2024-04-23T02:20:35.902+0000 7f780e549700 -1 osd.3 63 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T02:20:36.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:36 smithi192 ceph-mon[108617]: pgmap v198: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:20:36.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:36 smithi192 ceph-mon[108617]: osd.3 marked itself down and dead 2024-04-23T02:20:36.392 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:36 smithi071 ceph-mon[144544]: pgmap v198: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:20:36.392 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:36 smithi071 ceph-mon[144544]: osd.3 marked itself down and dead 2024-04-23T02:20:37.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:37 smithi192 ceph-mon[108617]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-23T02:20:37.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:37 smithi192 ceph-mon[108617]: osdmap e64: 6 total, 5 up, 6 in 2024-04-23T02:20:37.377 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:37 smithi192 bash[119244]: ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-3 2024-04-23T02:20:37.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:37 smithi071 ceph-mon[144544]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-23T02:20:37.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:37 smithi071 ceph-mon[144544]: osdmap e64: 6 total, 5 up, 6 in 2024-04-23T02:20:38.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:38 smithi192 ceph-mon[108617]: pgmap v200: 65 pgs: 7 peering, 13 stale+active+clean, 45 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-04-23T02:20:38.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:38 smithi192 ceph-mon[108617]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-23T02:20:38.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:38 smithi192 ceph-mon[108617]: osdmap e65: 6 total, 5 up, 6 in 2024-04-23T02:20:38.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:38 smithi071 ceph-mon[144544]: pgmap v200: 65 pgs: 7 peering, 13 stale+active+clean, 45 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-04-23T02:20:38.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:38 smithi071 ceph-mon[144544]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-23T02:20:38.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:38 smithi071 ceph-mon[144544]: osdmap e65: 6 total, 5 up, 6 in 2024-04-23T02:20:39.626 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:39 smithi192 systemd[1]: ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.3.service: Succeeded. 2024-04-23T02:20:39.627 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:39 smithi192 systemd[1]: Stopped Ceph osd.3 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:20:39.627 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:39 smithi192 systemd[1]: Starting Ceph osd.3 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T02:20:40.352 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:40 smithi192 ceph-mon[108617]: pgmap v202: 65 pgs: 7 peering, 13 stale+active+clean, 45 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-23T02:20:40.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:40 smithi071 ceph-mon[144544]: pgmap v202: 65 pgs: 7 peering, 13 stale+active+clean, 45 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-23T02:20:40.626 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:40 smithi192 bash[119517]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-23T02:20:40.626 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:40 smithi192 bash[119517]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-23T02:20:40.626 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:40 smithi192 bash[119517]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-23T02:20:40.626 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:40 smithi192 bash[119517]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-23T02:20:40.627 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:40 smithi192 bash[119517]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-23T02:20:40.627 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:40 smithi192 bash[119517]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-23T02:20:40.627 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:40 smithi192 bash[119517]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-23T02:20:41.446 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:41 smithi071 ceph-mon[144544]: Health check failed: Degraded data redundancy: 63/366 objects degraded (17.213%), 15 pgs degraded (PG_DEGRADED) 2024-04-23T02:20:41.448 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:41 smithi192 ceph-mon[108617]: Health check failed: Degraded data redundancy: 63/366 objects degraded (17.213%), 15 pgs degraded (PG_DEGRADED) 2024-04-23T02:20:42.126 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:41 smithi192 ceph-osd[119812]: -- 172.21.15.192:0/2070768773 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c79964b1e0 con 0x55c79961b000 2024-04-23T02:20:42.126 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:41 smithi192 bash[119774]: aa11be9f1148bfa1214eaa5264c4217d4bc87e59266e213a8aaca92bc3b47604 2024-04-23T02:20:42.126 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:41 smithi192 systemd[1]: Started Ceph osd.3 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:20:42.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:42 smithi071 ceph-mon[144544]: pgmap v203: 65 pgs: 15 active+undersized+degraded, 7 peering, 1 stale+active+clean, 12 active+undersized, 30 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 63/366 objects degraded (17.213%) 2024-04-23T02:20:42.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:42 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:42.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:42 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:42.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:42 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:20:42.626 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:42 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-3[119789]: 2024-04-23T02:20:42.265+0000 7f578eb3c7c0 -1 Falling back to public interface 2024-04-23T02:20:42.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:42 smithi192 ceph-mon[108617]: pgmap v203: 65 pgs: 15 active+undersized+degraded, 7 peering, 1 stale+active+clean, 12 active+undersized, 30 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 63/366 objects degraded (17.213%) 2024-04-23T02:20:42.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:42 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:42.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:42 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:42.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:42 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:20:43.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:43 smithi071 ceph-mon[144544]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-23T02:20:43.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:43 smithi192 ceph-mon[108617]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-23T02:20:44.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:44 smithi071 ceph-mon[144544]: pgmap v204: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 76/366 objects degraded (20.765%) 2024-04-23T02:20:44.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:44 smithi192 ceph-mon[108617]: pgmap v204: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 76/366 objects degraded (20.765%) 2024-04-23T02:20:46.218 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:45 smithi192 ceph-mon[108617]: pgmap v205: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 76/366 objects degraded (20.765%) 2024-04-23T02:20:46.218 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:45 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:46.218 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:45 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:46.218 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:45 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:46.218 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:45 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:46.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:45 smithi071 ceph-mon[144544]: pgmap v205: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 76/366 objects degraded (20.765%) 2024-04-23T02:20:46.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:45 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:46.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:45 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:46.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:45 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:46.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:45 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:47.126 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:46 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-3[119789]: 2024-04-23T02:20:46.765+0000 7f578eb3c7c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-04-23T02:20:47.601 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:47 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-3[119789]: 2024-04-23T02:20:47.136+0000 7f578eb3c7c0 -1 osd.3 63 log_to_monitors true 2024-04-23T02:20:48.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:47 smithi071 ceph-mon[144544]: pgmap v206: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 76/366 objects degraded (20.765%) 2024-04-23T02:20:48.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:47 smithi071 ceph-mon[144544]: from='osd.3 [v2:172.21.15.192:6800/4106623627,v1:172.21.15.192:6801/4106623627]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T02:20:48.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:47 smithi071 ceph-mon[144544]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T02:20:48.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:47 smithi071 ceph-mon[144544]: Health check update: Degraded data redundancy: 76/366 objects degraded (20.765%), 20 pgs degraded (PG_DEGRADED) 2024-04-23T02:20:48.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:47 smithi192 ceph-mon[108617]: pgmap v206: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 76/366 objects degraded (20.765%) 2024-04-23T02:20:48.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:47 smithi192 ceph-mon[108617]: from='osd.3 [v2:172.21.15.192:6800/4106623627,v1:172.21.15.192:6801/4106623627]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T02:20:48.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:47 smithi192 ceph-mon[108617]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T02:20:48.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:47 smithi192 ceph-mon[108617]: Health check update: Degraded data redundancy: 76/366 objects degraded (20.765%), 20 pgs degraded (PG_DEGRADED) 2024-04-23T02:20:49.126 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 02:20:48 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-3[119789]: 2024-04-23T02:20:48.844+0000 7f577bf8e700 -1 osd.3 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T02:20:49.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:48 smithi192 ceph-mon[108617]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T02:20:49.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:48 smithi192 ceph-mon[108617]: osdmap e66: 6 total, 5 up, 6 in 2024-04-23T02:20:49.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:48 smithi192 ceph-mon[108617]: from='osd.3 [v2:172.21.15.192:6800/4106623627,v1:172.21.15.192:6801/4106623627]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:20:49.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:48 smithi192 ceph-mon[108617]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:20:49.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:48 smithi192 ceph-mon[108617]: from='osd.3 ' entity='osd.3' 2024-04-23T02:20:49.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:48 smithi071 ceph-mon[144544]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T02:20:49.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:48 smithi071 ceph-mon[144544]: osdmap e66: 6 total, 5 up, 6 in 2024-04-23T02:20:49.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:48 smithi071 ceph-mon[144544]: from='osd.3 [v2:172.21.15.192:6800/4106623627,v1:172.21.15.192:6801/4106623627]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:20:49.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:48 smithi071 ceph-mon[144544]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:20:49.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:48 smithi071 ceph-mon[144544]: from='osd.3 ' entity='osd.3' 2024-04-23T02:20:50.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:49 smithi071 ceph-mon[144544]: pgmap v208: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 76/366 objects degraded (20.765%) 2024-04-23T02:20:50.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:50.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:50.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:20:50.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:20:50.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:50.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:50.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:20:50.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:20:50.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:20:50.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:20:50.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:20:50.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-23T02:20:50.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:49 smithi071 ceph-mon[144544]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-23T02:20:50.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:49 smithi071 ceph-mon[144544]: osd.3 [v2:172.21.15.192:6800/4106623627,v1:172.21.15.192:6801/4106623627] boot 2024-04-23T02:20:50.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:49 smithi071 ceph-mon[144544]: osdmap e67: 6 total, 6 up, 6 in 2024-04-23T02:20:50.245 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T02:20:50.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:49 smithi192 ceph-mon[108617]: pgmap v208: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 76/366 objects degraded (20.765%) 2024-04-23T02:20:50.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:50.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:50.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:20:50.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:20:50.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:50.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:20:50.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:20:50.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:20:50.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:20:50.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:20:50.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:20:50.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-23T02:20:50.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:49 smithi192 ceph-mon[108617]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-23T02:20:50.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:49 smithi192 ceph-mon[108617]: osd.3 [v2:172.21.15.192:6800/4106623627,v1:172.21.15.192:6801/4106623627] boot 2024-04-23T02:20:50.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:49 smithi192 ceph-mon[108617]: osdmap e67: 6 total, 6 up, 6 in 2024-04-23T02:20:50.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T02:20:51.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:50 smithi071 ceph-mon[144544]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-23T02:20:51.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:50 smithi071 ceph-mon[144544]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-23T02:20:51.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:50 smithi071 ceph-mon[144544]: osdmap e68: 6 total, 6 up, 6 in 2024-04-23T02:20:51.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:50 smithi192 ceph-mon[108617]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-23T02:20:51.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:50 smithi192 ceph-mon[108617]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-23T02:20:51.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:50 smithi192 ceph-mon[108617]: osdmap e68: 6 total, 6 up, 6 in 2024-04-23T02:20:52.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:51 smithi071 ceph-mon[144544]: pgmap v210: 65 pgs: 8 peering, 17 active+undersized+degraded, 13 active+undersized, 27 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 65/366 objects degraded (17.760%) 2024-04-23T02:20:52.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:51 smithi192 ceph-mon[108617]: pgmap v210: 65 pgs: 8 peering, 17 active+undersized+degraded, 13 active+undersized, 27 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 65/366 objects degraded (17.760%) 2024-04-23T02:20:53.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:52 smithi071 ceph-mon[144544]: Health check update: Degraded data redundancy: 65/366 objects degraded (17.760%), 17 pgs degraded (PG_DEGRADED) 2024-04-23T02:20:53.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:52 smithi192 ceph-mon[108617]: Health check update: Degraded data redundancy: 65/366 objects degraded (17.760%), 17 pgs degraded (PG_DEGRADED) 2024-04-23T02:20:54.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:53 smithi071 ceph-mon[144544]: pgmap v212: 65 pgs: 11 peering, 14 active+undersized+degraded, 9 active+undersized, 31 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 57/366 objects degraded (15.574%) 2024-04-23T02:20:54.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:53 smithi192 ceph-mon[108617]: pgmap v212: 65 pgs: 11 peering, 14 active+undersized+degraded, 9 active+undersized, 31 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 57/366 objects degraded (15.574%) 2024-04-23T02:20:55.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:55 smithi192 ceph-mon[108617]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 57/366 objects degraded (15.574%), 14 pgs degraded) 2024-04-23T02:20:55.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:55 smithi071 ceph-mon[144544]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 57/366 objects degraded (15.574%), 14 pgs degraded) 2024-04-23T02:20:56.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:56 smithi192 ceph-mon[108617]: pgmap v213: 65 pgs: 11 peering, 54 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-23T02:20:56.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:56 smithi071 ceph-mon[144544]: pgmap v213: 65 pgs: 11 peering, 54 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-23T02:20:58.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:20:58 smithi192 ceph-mon[108617]: pgmap v214: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-23T02:20:58.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:20:58 smithi071 ceph-mon[144544]: pgmap v214: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-23T02:21:00.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:00 smithi192 ceph-mon[108617]: pgmap v215: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-23T02:21:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:00 smithi071 ceph-mon[144544]: pgmap v215: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-23T02:21:02.380 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:02 smithi192 ceph-mon[108617]: pgmap v216: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-23T02:21:02.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:02 smithi071 ceph-mon[144544]: pgmap v216: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-23T02:21:04.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.271+0000 7f33eedd0700 1 -- 172.21.15.71:0/605240702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33e80fff50 msgr2=0x7f33e8105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:04.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.271+0000 7f33eedd0700 1 --2- 172.21.15.71:0/605240702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33e80fff50 0x7f33e8105840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009910 tx=0x7f33d4009c20 comp rx=0 tx=0).stop 2024-04-23T02:21:04.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.271+0000 7f33eedd0700 1 -- 172.21.15.71:0/605240702 shutdown_connections 2024-04-23T02:21:04.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.271+0000 7f33eedd0700 1 --2- 172.21.15.71:0/605240702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33e80fff50 0x7f33e8105840 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:04.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.271+0000 7f33eedd0700 1 --2- 172.21.15.71:0/605240702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e80ff5a0 0x7f33e80ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:04.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.271+0000 7f33eedd0700 1 -- 172.21.15.71:0/605240702 >> 172.21.15.71:0/605240702 conn(0x7f33e8074e30 msgr2=0x7f33e8075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:04.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.271+0000 7f33eedd0700 1 -- 172.21.15.71:0/605240702 shutdown_connections 2024-04-23T02:21:04.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.271+0000 7f33eedd0700 1 -- 172.21.15.71:0/605240702 wait complete. 2024-04-23T02:21:04.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.272+0000 7f33eedd0700 1 Processor -- start 2024-04-23T02:21:04.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.272+0000 7f33eedd0700 1 -- start start 2024-04-23T02:21:04.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.272+0000 7f33eedd0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33e80ff5a0 0x7f33e819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:04.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.272+0000 7f33eedd0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e80fff50 0x7f33e819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:04.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.272+0000 7f33eedd0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e819d250 con 0x7f33e80ff5a0 2024-04-23T02:21:04.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.272+0000 7f33eedd0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e8196700 con 0x7f33e80fff50 2024-04-23T02:21:04.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.272+0000 7f33ecb6c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33e80ff5a0 0x7f33e819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:04.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.272+0000 7f33ecb6c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33e80ff5a0 0x7f33e819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39764/0 (socket says 172.21.15.71:39764) 2024-04-23T02:21:04.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.272+0000 7f33ecb6c700 1 -- 172.21.15.71:0/180828593 learned_addr learned my addr 172.21.15.71:0/180828593 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:21:04.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.273+0000 7f33e7fff700 1 --2- 172.21.15.71:0/180828593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e80fff50 0x7f33e819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:04.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.273+0000 7f33ecb6c700 1 -- 172.21.15.71:0/180828593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e80fff50 msgr2=0x7f33e819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:04.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.273+0000 7f33ecb6c700 1 --2- 172.21.15.71:0/180828593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e80fff50 0x7f33e819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:04.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.273+0000 7f33ecb6c700 1 -- 172.21.15.71:0/180828593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d40095f0 con 0x7f33e80ff5a0 2024-04-23T02:21:04.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.273+0000 7f33ecb6c700 1 --2- 172.21.15.71:0/180828593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33e80ff5a0 0x7f33e819c680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00b4b0 tx=0x7f33dc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:04.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.273+0000 7f33e5ffb700 1 -- 172.21.15.71:0/180828593 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33dc011800 con 0x7f33e80ff5a0 2024-04-23T02:21:04.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.273+0000 7f33e5ffb700 1 -- 172.21.15.71:0/180828593 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f33dc011e00 con 0x7f33e80ff5a0 2024-04-23T02:21:04.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.273+0000 7f33eedd0700 1 -- 172.21.15.71:0/180828593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e81969e0 con 0x7f33e80ff5a0 2024-04-23T02:21:04.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.273+0000 7f33e5ffb700 1 -- 172.21.15.71:0/180828593 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33dc010e50 con 0x7f33e80ff5a0 2024-04-23T02:21:04.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.273+0000 7f33eedd0700 1 -- 172.21.15.71:0/180828593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e8196f30 con 0x7f33e80ff5a0 2024-04-23T02:21:04.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.274+0000 7f33eedd0700 1 -- 172.21.15.71:0/180828593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e8109200 con 0x7f33e80ff5a0 2024-04-23T02:21:04.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.275+0000 7f33e5ffb700 1 -- 172.21.15.71:0/180828593 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f33dc010380 con 0x7f33e80ff5a0 2024-04-23T02:21:04.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.275+0000 7f33e5ffb700 1 --2- 172.21.15.71:0/180828593 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f33d806cce0 0x7f33d806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:04.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.275+0000 7f33e5ffb700 1 -- 172.21.15.71:0/180828593 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f33dc08c0a0 con 0x7f33e80ff5a0 2024-04-23T02:21:04.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.275+0000 7f33e7fff700 1 --2- 172.21.15.71:0/180828593 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f33d806cce0 0x7f33d806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:04.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.276+0000 7f33e7fff700 1 --2- 172.21.15.71:0/180828593 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f33d806cce0 0x7f33d806f1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f33e8197b70 tx=0x7f33d401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:04.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.280+0000 7f33e5ffb700 1 -- 172.21.15.71:0/180828593 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f33dc059c50 con 0x7f33e80ff5a0 2024-04-23T02:21:04.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:04 smithi192 ceph-mon[108617]: pgmap v217: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:21:04.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:04 smithi071 ceph-mon[144544]: pgmap v217: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:21:04.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.495+0000 7f33eedd0700 1 -- 172.21.15.71:0/180828593 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f33e81971b0 con 0x7f33d806cce0 2024-04-23T02:21:04.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.497+0000 7f33e5ffb700 1 -- 172.21.15.71:0/180828593 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f33e81971b0 con 0x7f33d806cce0 2024-04-23T02:21:04.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.499+0000 7f33eedd0700 1 -- 172.21.15.71:0/180828593 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f33d806cce0 msgr2=0x7f33d806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:04.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.499+0000 7f33eedd0700 1 --2- 172.21.15.71:0/180828593 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f33d806cce0 0x7f33d806f1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f33e8197b70 tx=0x7f33d401a040 comp rx=0 tx=0).stop 2024-04-23T02:21:04.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.499+0000 7f33eedd0700 1 -- 172.21.15.71:0/180828593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33e80ff5a0 msgr2=0x7f33e819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:04.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.499+0000 7f33eedd0700 1 --2- 172.21.15.71:0/180828593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33e80ff5a0 0x7f33e819c680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00b4b0 tx=0x7f33dc00b870 comp rx=0 tx=0).stop 2024-04-23T02:21:04.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.499+0000 7f33eedd0700 1 -- 172.21.15.71:0/180828593 shutdown_connections 2024-04-23T02:21:04.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.499+0000 7f33eedd0700 1 --2- 172.21.15.71:0/180828593 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f33d806cce0 0x7f33d806f1a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:04.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.499+0000 7f33eedd0700 1 --2- 172.21.15.71:0/180828593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33e80ff5a0 0x7f33e819c680 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:04.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.499+0000 7f33eedd0700 1 --2- 172.21.15.71:0/180828593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e80fff50 0x7f33e819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:04.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.499+0000 7f33eedd0700 1 -- 172.21.15.71:0/180828593 >> 172.21.15.71:0/180828593 conn(0x7f33e8074e30 msgr2=0x7f33e80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:04.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.499+0000 7f33eedd0700 1 -- 172.21.15.71:0/180828593 shutdown_connections 2024-04-23T02:21:04.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.499+0000 7f33eedd0700 1 -- 172.21.15.71:0/180828593 wait complete. 2024-04-23T02:21:04.514 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:21:04.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.602+0000 7f7284688700 1 -- 172.21.15.71:0/897705747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f727c0ffc20 msgr2=0x7f727c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:04.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.602+0000 7f7284688700 1 --2- 172.21.15.71:0/897705747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f727c0ffc20 0x7f727c100080 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7278009910 tx=0x7f7278009c20 comp rx=0 tx=0).stop 2024-04-23T02:21:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.602+0000 7f7284688700 1 -- 172.21.15.71:0/897705747 shutdown_connections 2024-04-23T02:21:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.602+0000 7f7284688700 1 --2- 172.21.15.71:0/897705747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f727c0ffc20 0x7f727c100080 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.602+0000 7f7284688700 1 --2- 172.21.15.71:0/897705747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f727c105bd0 0x7f727c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.602+0000 7f7284688700 1 -- 172.21.15.71:0/897705747 >> 172.21.15.71:0/897705747 conn(0x7f727c075670 msgr2=0x7f727c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.602+0000 7f7284688700 1 -- 172.21.15.71:0/897705747 shutdown_connections 2024-04-23T02:21:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.602+0000 7f7284688700 1 -- 172.21.15.71:0/897705747 wait complete. 2024-04-23T02:21:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.602+0000 7f7284688700 1 Processor -- start 2024-04-23T02:21:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.603+0000 7f7284688700 1 -- start start 2024-04-23T02:21:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.603+0000 7f7284688700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f727c0ffc20 0x7f727c19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.603+0000 7f7284688700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f727c105bd0 0x7f727c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.603+0000 7f7284688700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c19d150 con 0x7f727c0ffc20 2024-04-23T02:21:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.603+0000 7f7284688700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c196600 con 0x7f727c105bd0 2024-04-23T02:21:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.603+0000 7f7282424700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f727c0ffc20 0x7f727c19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.603+0000 7f7282424700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f727c0ffc20 0x7f727c19c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39782/0 (socket says 172.21.15.71:39782) 2024-04-23T02:21:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.603+0000 7f7282424700 1 -- 172.21.15.71:0/3098094487 learned_addr learned my addr 172.21.15.71:0/3098094487 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:21:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.603+0000 7f7281c23700 1 --2- 172.21.15.71:0/3098094487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f727c105bd0 0x7f727c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.603+0000 7f7282424700 1 -- 172.21.15.71:0/3098094487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f727c105bd0 msgr2=0x7f727c19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.603+0000 7f7282424700 1 --2- 172.21.15.71:0/3098094487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f727c105bd0 0x7f727c19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.603+0000 7f7282424700 1 -- 172.21.15.71:0/3098094487 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72780095f0 con 0x7f727c0ffc20 2024-04-23T02:21:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.603+0000 7f7282424700 1 --2- 172.21.15.71:0/3098094487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f727c0ffc20 0x7f727c19c580 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f726c00d900 tx=0x7f726c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.603+0000 7f72737fe700 1 -- 172.21.15.71:0/3098094487 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f726c004a00 con 0x7f727c0ffc20 2024-04-23T02:21:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.603+0000 7f72737fe700 1 -- 172.21.15.71:0/3098094487 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f726c005020 con 0x7f727c0ffc20 2024-04-23T02:21:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.603+0000 7f7284688700 1 -- 172.21.15.71:0/3098094487 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727c1968e0 con 0x7f727c0ffc20 2024-04-23T02:21:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.603+0000 7f72737fe700 1 -- 172.21.15.71:0/3098094487 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f726c009c00 con 0x7f727c0ffc20 2024-04-23T02:21:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.604+0000 7f7284688700 1 -- 172.21.15.71:0/3098094487 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727c196e30 con 0x7f727c0ffc20 2024-04-23T02:21:04.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.604+0000 7f7284688700 1 -- 172.21.15.71:0/3098094487 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f727c04f2e0 con 0x7f727c0ffc20 2024-04-23T02:21:04.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.605+0000 7f72737fe700 1 -- 172.21.15.71:0/3098094487 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f726c005190 con 0x7f727c0ffc20 2024-04-23T02:21:04.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.605+0000 7f72737fe700 1 --2- 172.21.15.71:0/3098094487 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f726806cce0 0x7f726806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:04.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.605+0000 7f72737fe700 1 -- 172.21.15.71:0/3098094487 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f726c08c6c0 con 0x7f727c0ffc20 2024-04-23T02:21:04.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.605+0000 7f7281c23700 1 --2- 172.21.15.71:0/3098094487 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f726806cce0 0x7f726806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:04.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.606+0000 7f7281c23700 1 --2- 172.21.15.71:0/3098094487 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f726806cce0 0x7f726806f1a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f727c0ffb20 tx=0x7f727801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:04.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.611+0000 7f72737fe700 1 -- 172.21.15.71:0/3098094487 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f726c05a270 con 0x7f727c0ffc20 2024-04-23T02:21:04.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.823+0000 7f7284688700 1 -- 172.21.15.71:0/3098094487 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f727c1970e0 con 0x7f726806cce0 2024-04-23T02:21:04.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.824+0000 7f72737fe700 1 -- 172.21.15.71:0/3098094487 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f727c1970e0 con 0x7f726806cce0 2024-04-23T02:21:04.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.827+0000 7f7284688700 1 -- 172.21.15.71:0/3098094487 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f726806cce0 msgr2=0x7f726806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:04.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.827+0000 7f7284688700 1 --2- 172.21.15.71:0/3098094487 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f726806cce0 0x7f726806f1a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f727c0ffb20 tx=0x7f727801a040 comp rx=0 tx=0).stop 2024-04-23T02:21:04.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.827+0000 7f7284688700 1 -- 172.21.15.71:0/3098094487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f727c0ffc20 msgr2=0x7f727c19c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:04.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.827+0000 7f7284688700 1 --2- 172.21.15.71:0/3098094487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f727c0ffc20 0x7f727c19c580 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f726c00d900 tx=0x7f726c00dcc0 comp rx=0 tx=0).stop 2024-04-23T02:21:04.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.827+0000 7f7284688700 1 -- 172.21.15.71:0/3098094487 shutdown_connections 2024-04-23T02:21:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.827+0000 7f7284688700 1 --2- 172.21.15.71:0/3098094487 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f726806cce0 0x7f726806f1a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.827+0000 7f7284688700 1 --2- 172.21.15.71:0/3098094487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f727c0ffc20 0x7f727c19c580 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.827+0000 7f7284688700 1 --2- 172.21.15.71:0/3098094487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f727c105bd0 0x7f727c19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.827+0000 7f7284688700 1 -- 172.21.15.71:0/3098094487 >> 172.21.15.71:0/3098094487 conn(0x7f727c075670 msgr2=0x7f727c0686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.827+0000 7f7284688700 1 -- 172.21.15.71:0/3098094487 shutdown_connections 2024-04-23T02:21:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.827+0000 7f7284688700 1 -- 172.21.15.71:0/3098094487 wait complete. 2024-04-23T02:21:04.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.931+0000 7f9912555700 1 -- 172.21.15.71:0/1568191260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f990c106cd0 msgr2=0x7f990c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:04.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.931+0000 7f9912555700 1 --2- 172.21.15.71:0/1568191260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f990c106cd0 0x7f990c1070b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009910 tx=0x7f98fc009c20 comp rx=0 tx=0).stop 2024-04-23T02:21:04.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.931+0000 7f9912555700 1 -- 172.21.15.71:0/1568191260 shutdown_connections 2024-04-23T02:21:04.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.931+0000 7f9912555700 1 --2- 172.21.15.71:0/1568191260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f990c068580 0x7f990c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:04.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.931+0000 7f9912555700 1 --2- 172.21.15.71:0/1568191260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f990c106cd0 0x7f990c1070b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:04.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.931+0000 7f9912555700 1 -- 172.21.15.71:0/1568191260 >> 172.21.15.71:0/1568191260 conn(0x7f990c075670 msgr2=0x7f990c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:04.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.931+0000 7f9912555700 1 -- 172.21.15.71:0/1568191260 shutdown_connections 2024-04-23T02:21:04.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.931+0000 7f9912555700 1 -- 172.21.15.71:0/1568191260 wait complete. 2024-04-23T02:21:04.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.931+0000 7f9912555700 1 Processor -- start 2024-04-23T02:21:04.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.931+0000 7f9912555700 1 -- start start 2024-04-23T02:21:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.932+0000 7f9912555700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f990c068580 0x7f990c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.932+0000 7f9912555700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f990c106cd0 0x7f990c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.932+0000 7f9912555700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c19d130 con 0x7f990c068580 2024-04-23T02:21:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.932+0000 7f9912555700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c196630 con 0x7f990c106cd0 2024-04-23T02:21:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.932+0000 7f990bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f990c068580 0x7f990c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.932+0000 7f990bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f990c068580 0x7f990c19c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39806/0 (socket says 172.21.15.71:39806) 2024-04-23T02:21:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.932+0000 7f990bfff700 1 -- 172.21.15.71:0/3644839423 learned_addr learned my addr 172.21.15.71:0/3644839423 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:21:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.932+0000 7f990b7fe700 1 --2- 172.21.15.71:0/3644839423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f990c106cd0 0x7f990c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.932+0000 7f990bfff700 1 -- 172.21.15.71:0/3644839423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f990c106cd0 msgr2=0x7f990c19caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.932+0000 7f990bfff700 1 --2- 172.21.15.71:0/3644839423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f990c106cd0 0x7f990c19caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.932+0000 7f990bfff700 1 -- 172.21.15.71:0/3644839423 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc0095f0 con 0x7f990c068580 2024-04-23T02:21:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.932+0000 7f990bfff700 1 --2- 172.21.15.71:0/3644839423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f990c068580 0x7f990c19c560 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0092e0 tx=0x7f98fc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:04.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.932+0000 7f99097fa700 1 -- 172.21.15.71:0/3644839423 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98fc01d070 con 0x7f990c068580 2024-04-23T02:21:04.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.932+0000 7f99097fa700 1 -- 172.21.15.71:0/3644839423 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f98fc005e00 con 0x7f990c068580 2024-04-23T02:21:04.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.932+0000 7f9912555700 1 -- 172.21.15.71:0/3644839423 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990c1968b0 con 0x7f990c068580 2024-04-23T02:21:04.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.932+0000 7f99097fa700 1 -- 172.21.15.71:0/3644839423 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98fc00fe90 con 0x7f990c068580 2024-04-23T02:21:04.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.932+0000 7f9912555700 1 -- 172.21.15.71:0/3644839423 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990c196da0 con 0x7f990c068580 2024-04-23T02:21:04.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.933+0000 7f9912555700 1 -- 172.21.15.71:0/3644839423 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f990c1091f0 con 0x7f990c068580 2024-04-23T02:21:04.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.934+0000 7f99097fa700 1 -- 172.21.15.71:0/3644839423 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f98fc00f6d0 con 0x7f990c068580 2024-04-23T02:21:04.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.934+0000 7f99097fa700 1 --2- 172.21.15.71:0/3644839423 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98f406cce0 0x7f98f406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:04.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.934+0000 7f99097fa700 1 -- 172.21.15.71:0/3644839423 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f98fc08d6b0 con 0x7f990c068580 2024-04-23T02:21:04.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.934+0000 7f990b7fe700 1 --2- 172.21.15.71:0/3644839423 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98f406cce0 0x7f98f406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:04.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.935+0000 7f990b7fe700 1 --2- 172.21.15.71:0/3644839423 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98f406cce0 0x7f98f406f1a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f990c0fe390 tx=0x7f990000b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:04.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:04.939+0000 7f99097fa700 1 -- 172.21.15.71:0/3644839423 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f98fc05b2e0 con 0x7f990c068580 2024-04-23T02:21:05.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.138+0000 7f9912555700 1 -- 172.21.15.71:0/3644839423 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f990c197010 con 0x7f98f406cce0 2024-04-23T02:21:05.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.148+0000 7f99097fa700 1 -- 172.21.15.71:0/3644839423 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f990c197010 con 0x7f98f406cce0 2024-04-23T02:21:05.153 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:21:05.153 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (4m) 54s ago 16m 26.3M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:21:05.153 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (16m) 54s ago 16m 8250k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:21:05.153 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (15m) 20s ago 15m 8212k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:21:05.153 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (2m) 54s ago 16m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:21:05.153 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (2m) 20s ago 15m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:21:05.154 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (4m) 54s ago 15m 80.0M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:21:05.154 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (10m) 54s ago 10m 64.9M - 18.2.1 5be31c24972a a515de2a841c 2024-04-23T02:21:05.154 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (10m) 54s ago 10m 133M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:21:05.154 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (10m) 20s ago 10m 68.2M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:21:05.154 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (10m) 20s ago 10m 122M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:21:05.154 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (6m) 54s ago 17m 507M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:21:05.154 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (5m) 20s ago 15m 452M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:21:05.154 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (3m) 54s ago 17m 47.1M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:21:05.154 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (2m) 20s ago 14m 41.1M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:21:05.154 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (5m) 54s ago 16m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:21:05.154 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (5m) 20s ago 15m 18.1M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:21:05.154 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (2m) 54s ago 14m 85.3M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:21:05.155 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (97s) 54s ago 13m 85.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:21:05.155 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (60s) 54s ago 13m 10.9M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:21:05.155 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (23s) 20s ago 12m 11.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:21:05.155 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (12m) 20s ago 12m 222M 2251M 18.2.1 5be31c24972a 958d046005f0 2024-04-23T02:21:05.155 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (11m) 20s ago 11m 199M 2251M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:21:05.155 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (5m) 54s ago 15m 65.6M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:21:05.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.151+0000 7f9912555700 1 -- 172.21.15.71:0/3644839423 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98f406cce0 msgr2=0x7f98f406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:05.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.151+0000 7f9912555700 1 --2- 172.21.15.71:0/3644839423 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98f406cce0 0x7f98f406f1a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f990c0fe390 tx=0x7f990000b410 comp rx=0 tx=0).stop 2024-04-23T02:21:05.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.151+0000 7f9912555700 1 -- 172.21.15.71:0/3644839423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f990c068580 msgr2=0x7f990c19c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:05.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.151+0000 7f9912555700 1 --2- 172.21.15.71:0/3644839423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f990c068580 0x7f990c19c560 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0092e0 tx=0x7f98fc00bd70 comp rx=0 tx=0).stop 2024-04-23T02:21:05.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.151+0000 7f9912555700 1 -- 172.21.15.71:0/3644839423 shutdown_connections 2024-04-23T02:21:05.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.151+0000 7f9912555700 1 --2- 172.21.15.71:0/3644839423 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98f406cce0 0x7f98f406f1a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:05.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.151+0000 7f9912555700 1 --2- 172.21.15.71:0/3644839423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f990c068580 0x7f990c19c560 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:05.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.151+0000 7f9912555700 1 --2- 172.21.15.71:0/3644839423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f990c106cd0 0x7f990c19caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:05.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.151+0000 7f9912555700 1 -- 172.21.15.71:0/3644839423 >> 172.21.15.71:0/3644839423 conn(0x7f990c075670 msgr2=0x7f990c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:05.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.151+0000 7f9912555700 1 -- 172.21.15.71:0/3644839423 shutdown_connections 2024-04-23T02:21:05.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.151+0000 7f9912555700 1 -- 172.21.15.71:0/3644839423 wait complete. 2024-04-23T02:21:05.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.252+0000 7f7d8de58700 1 -- 172.21.15.71:0/594472862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d880fff50 msgr2=0x7f7d88105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:05.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.252+0000 7f7d8de58700 1 --2- 172.21.15.71:0/594472862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d880fff50 0x7f7d88105840 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009910 tx=0x7f7d78009c20 comp rx=0 tx=0).stop 2024-04-23T02:21:05.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.252+0000 7f7d8de58700 1 -- 172.21.15.71:0/594472862 shutdown_connections 2024-04-23T02:21:05.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.252+0000 7f7d8de58700 1 --2- 172.21.15.71:0/594472862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d880fff50 0x7f7d88105840 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:05.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.252+0000 7f7d8de58700 1 --2- 172.21.15.71:0/594472862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d880ff5a0 0x7f7d880ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:05.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.252+0000 7f7d8de58700 1 -- 172.21.15.71:0/594472862 >> 172.21.15.71:0/594472862 conn(0x7f7d88074e30 msgr2=0x7f7d88075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:05.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.252+0000 7f7d8de58700 1 -- 172.21.15.71:0/594472862 shutdown_connections 2024-04-23T02:21:05.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.252+0000 7f7d8de58700 1 -- 172.21.15.71:0/594472862 wait complete. 2024-04-23T02:21:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.253+0000 7f7d8de58700 1 Processor -- start 2024-04-23T02:21:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.253+0000 7f7d8de58700 1 -- start start 2024-04-23T02:21:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.253+0000 7f7d8de58700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d880ff5a0 0x7f7d8819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.253+0000 7f7d8de58700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d880fff50 0x7f7d8819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.253+0000 7f7d8de58700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8819d150 con 0x7f7d880ff5a0 2024-04-23T02:21:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.253+0000 7f7d8de58700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d88196650 con 0x7f7d880fff50 2024-04-23T02:21:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.253+0000 7f7d877fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d880ff5a0 0x7f7d8819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.253+0000 7f7d877fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d880ff5a0 0x7f7d8819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39812/0 (socket says 172.21.15.71:39812) 2024-04-23T02:21:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.253+0000 7f7d877fe700 1 -- 172.21.15.71:0/4067509248 learned_addr learned my addr 172.21.15.71:0/4067509248 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:21:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.254+0000 7f7d86ffd700 1 --2- 172.21.15.71:0/4067509248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d880fff50 0x7f7d8819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.254+0000 7f7d877fe700 1 -- 172.21.15.71:0/4067509248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d880fff50 msgr2=0x7f7d8819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.254+0000 7f7d877fe700 1 --2- 172.21.15.71:0/4067509248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d880fff50 0x7f7d8819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.254+0000 7f7d877fe700 1 -- 172.21.15.71:0/4067509248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d780095f0 con 0x7f7d880ff5a0 2024-04-23T02:21:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.254+0000 7f7d877fe700 1 --2- 172.21.15.71:0/4067509248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d880ff5a0 0x7f7d8819c580 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00e940 tx=0x7f7d7c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.254+0000 7f7d84ff9700 1 -- 172.21.15.71:0/4067509248 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d7c00cba0 con 0x7f7d880ff5a0 2024-04-23T02:21:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.254+0000 7f7d84ff9700 1 -- 172.21.15.71:0/4067509248 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7d7c00ee90 con 0x7f7d880ff5a0 2024-04-23T02:21:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.254+0000 7f7d8de58700 1 -- 172.21.15.71:0/4067509248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d88196930 con 0x7f7d880ff5a0 2024-04-23T02:21:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.254+0000 7f7d84ff9700 1 -- 172.21.15.71:0/4067509248 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d7c0055f0 con 0x7f7d880ff5a0 2024-04-23T02:21:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.254+0000 7f7d8de58700 1 -- 172.21.15.71:0/4067509248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d88196e50 con 0x7f7d880ff5a0 2024-04-23T02:21:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.255+0000 7f7d8de58700 1 -- 172.21.15.71:0/4067509248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d88109200 con 0x7f7d880ff5a0 2024-04-23T02:21:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.256+0000 7f7d84ff9700 1 -- 172.21.15.71:0/4067509248 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7d7c004750 con 0x7f7d880ff5a0 2024-04-23T02:21:05.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.256+0000 7f7d84ff9700 1 --2- 172.21.15.71:0/4067509248 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7d7006cce0 0x7f7d7006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:05.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.256+0000 7f7d84ff9700 1 -- 172.21.15.71:0/4067509248 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7d7c014070 con 0x7f7d880ff5a0 2024-04-23T02:21:05.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.256+0000 7f7d86ffd700 1 --2- 172.21.15.71:0/4067509248 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7d7006cce0 0x7f7d7006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:05.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.257+0000 7f7d86ffd700 1 --2- 172.21.15.71:0/4067509248 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7d7006cce0 0x7f7d7006f1a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009fd0 tx=0x7f7d78005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:05.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.261+0000 7f7d84ff9700 1 -- 172.21.15.71:0/4067509248 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7d7c05a340 con 0x7f7d880ff5a0 2024-04-23T02:21:05.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.530+0000 7f7d8de58700 1 -- 172.21.15.71:0/4067509248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7d8804f2e0 con 0x7f7d880ff5a0 2024-04-23T02:21:05.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.530+0000 7f7d84ff9700 1 -- 172.21.15.71:0/4067509248 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7f7d7c059ed0 con 0x7f7d880ff5a0 2024-04-23T02:21:05.535 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:21:05.535 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:21:05.535 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:21:05.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:21:05.535 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:21:05.535 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:21:05.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:21:05.535 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:21:05.535 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-04-23T02:21:05.536 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:21:05.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:21:05.536 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:21:05.536 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-23T02:21:05.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:21:05.536 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:21:05.536 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6, 2024-04-23T02:21:05.536 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 8 2024-04-23T02:21:05.536 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:21:05.536 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:21:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.532+0000 7f7d8de58700 1 -- 172.21.15.71:0/4067509248 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7d7006cce0 msgr2=0x7f7d7006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.532+0000 7f7d8de58700 1 --2- 172.21.15.71:0/4067509248 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7d7006cce0 0x7f7d7006f1a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009fd0 tx=0x7f7d78005940 comp rx=0 tx=0).stop 2024-04-23T02:21:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.532+0000 7f7d8de58700 1 -- 172.21.15.71:0/4067509248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d880ff5a0 msgr2=0x7f7d8819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.532+0000 7f7d8de58700 1 --2- 172.21.15.71:0/4067509248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d880ff5a0 0x7f7d8819c580 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00e940 tx=0x7f7d7c00ec50 comp rx=0 tx=0).stop 2024-04-23T02:21:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.532+0000 7f7d8de58700 1 -- 172.21.15.71:0/4067509248 shutdown_connections 2024-04-23T02:21:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.532+0000 7f7d8de58700 1 --2- 172.21.15.71:0/4067509248 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7d7006cce0 0x7f7d7006f1a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.532+0000 7f7d8de58700 1 --2- 172.21.15.71:0/4067509248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d880ff5a0 0x7f7d8819c580 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.532+0000 7f7d8de58700 1 --2- 172.21.15.71:0/4067509248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d880fff50 0x7f7d8819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:05.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.532+0000 7f7d8de58700 1 -- 172.21.15.71:0/4067509248 >> 172.21.15.71:0/4067509248 conn(0x7f7d88074e30 msgr2=0x7f7d880fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:05.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.532+0000 7f7d8de58700 1 -- 172.21.15.71:0/4067509248 shutdown_connections 2024-04-23T02:21:05.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.532+0000 7f7d8de58700 1 -- 172.21.15.71:0/4067509248 wait complete. 2024-04-23T02:21:05.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.636+0000 7fa64da90700 1 -- 172.21.15.71:0/337000753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa648100750 msgr2=0x7fa6481047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:05.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.636+0000 7fa64da90700 1 --2- 172.21.15.71:0/337000753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa648100750 0x7fa6481047a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009910 tx=0x7fa63c009c20 comp rx=0 tx=0).stop 2024-04-23T02:21:05.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.636+0000 7fa64da90700 1 -- 172.21.15.71:0/337000753 shutdown_connections 2024-04-23T02:21:05.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.636+0000 7fa64da90700 1 --2- 172.21.15.71:0/337000753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa648100750 0x7fa6481047a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:05.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.636+0000 7fa64da90700 1 --2- 172.21.15.71:0/337000753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6480ffda0 0x7fa648100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:05.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.636+0000 7fa64da90700 1 -- 172.21.15.71:0/337000753 >> 172.21.15.71:0/337000753 conn(0x7fa6480fb650 msgr2=0x7fa6480fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:05.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.636+0000 7fa64da90700 1 -- 172.21.15.71:0/337000753 shutdown_connections 2024-04-23T02:21:05.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.636+0000 7fa64da90700 1 -- 172.21.15.71:0/337000753 wait complete. 2024-04-23T02:21:05.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.637+0000 7fa64da90700 1 Processor -- start 2024-04-23T02:21:05.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.637+0000 7fa64da90700 1 -- start start 2024-04-23T02:21:05.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.637+0000 7fa64da90700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6480ffda0 0x7fa6481977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:05.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.637+0000 7fa64da90700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa648100750 0x7fa648197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:05.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.637+0000 7fa64da90700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6481983c0 con 0x7fa648100750 2024-04-23T02:21:05.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.637+0000 7fa64da90700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa648198500 con 0x7fa6480ffda0 2024-04-23T02:21:05.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.637+0000 7fa646ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa648100750 0x7fa648197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:05.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.637+0000 7fa646ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa648100750 0x7fa648197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55326/0 (socket says 172.21.15.71:55326) 2024-04-23T02:21:05.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.637+0000 7fa646ffd700 1 -- 172.21.15.71:0/84445861 learned_addr learned my addr 172.21.15.71:0/84445861 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:21:05.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.637+0000 7fa6477fe700 1 --2- 172.21.15.71:0/84445861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6480ffda0 0x7fa6481977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:05.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.638+0000 7fa646ffd700 1 -- 172.21.15.71:0/84445861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6480ffda0 msgr2=0x7fa6481977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:05.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.638+0000 7fa646ffd700 1 --2- 172.21.15.71:0/84445861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6480ffda0 0x7fa6481977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:05.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.638+0000 7fa646ffd700 1 -- 172.21.15.71:0/84445861 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c0095f0 con 0x7fa648100750 2024-04-23T02:21:05.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.638+0000 7fa646ffd700 1 --2- 172.21.15.71:0/84445861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa648100750 0x7fa648197ce0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa63c005f50 tx=0x7fa63c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:05.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.638+0000 7fa644ff9700 1 -- 172.21.15.71:0/84445861 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa63c01d070 con 0x7fa648100750 2024-04-23T02:21:05.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.638+0000 7fa644ff9700 1 -- 172.21.15.71:0/84445861 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa63c00f460 con 0x7fa648100750 2024-04-23T02:21:05.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.638+0000 7fa64da90700 1 -- 172.21.15.71:0/84445861 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6481a0810 con 0x7fa648100750 2024-04-23T02:21:05.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.638+0000 7fa644ff9700 1 -- 172.21.15.71:0/84445861 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa63c021640 con 0x7fa648100750 2024-04-23T02:21:05.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.638+0000 7fa64da90700 1 -- 172.21.15.71:0/84445861 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6481a0c80 con 0x7fa648100750 2024-04-23T02:21:05.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.639+0000 7fa64da90700 1 -- 172.21.15.71:0/84445861 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6481080f0 con 0x7fa648100750 2024-04-23T02:21:05.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.639+0000 7fa644ff9700 1 -- 172.21.15.71:0/84445861 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa63c003a60 con 0x7fa648100750 2024-04-23T02:21:05.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.640+0000 7fa644ff9700 1 --2- 172.21.15.71:0/84445861 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa63006cce0 0x7fa63006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:05.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.640+0000 7fa644ff9700 1 -- 172.21.15.71:0/84445861 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa63c08d730 con 0x7fa648100750 2024-04-23T02:21:05.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.640+0000 7fa6477fe700 1 --2- 172.21.15.71:0/84445861 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa63006cce0 0x7fa63006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:05.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.640+0000 7fa6477fe700 1 --2- 172.21.15.71:0/84445861 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa63006cce0 0x7fa63006f1a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa638009a10 tx=0x7fa638009190 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:05.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.644+0000 7fa644ff9700 1 -- 172.21.15.71:0/84445861 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa63c057850 con 0x7fa648100750 2024-04-23T02:21:05.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.883+0000 7fa64da90700 1 -- 172.21.15.71:0/84445861 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa64804f2e0 con 0x7fa648100750 2024-04-23T02:21:05.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.883+0000 7fa644ff9700 1 -- 172.21.15.71:0/84445861 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1869 (secure 0 0 0) 0x7fa63c003d10 con 0x7fa648100750 2024-04-23T02:21:05.887 INFO:teuthology.orchestra.run.smithi071.stdout:e25 2024-04-23T02:21:05.887 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:21:05.887 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:21:05.887 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:21:05.888 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:21:05.888 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:21:05.888 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:21:05.888 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 25 2024-04-23T02:21:05.888 INFO:teuthology.orchestra.run.smithi071.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T02:21:05.888 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:21:05.888 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:17:27.814232+0000 2024-04-23T02:21:05.888 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:21:05.888 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:21:05.888 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:21:05.888 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:21:05.888 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:21:05.889 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:21:05.889 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:21:05.889 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:21:05.889 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 46 2024-04-23T02:21:05.889 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:21:05.889 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:21:05.889 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:21:05.889 INFO:teuthology.orchestra.run.smithi071.stdout:up {0=14478,1=24243} 2024-04-23T02:21:05.889 INFO:teuthology.orchestra.run.smithi071.stdout:failed 2024-04-23T02:21:05.889 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:21:05.889 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:21:05.889 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:21:05.890 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:21:05.890 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:21:05.890 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:21:05.890 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:21:05.890 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:21:05.890 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:21:05.890 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/3442634960,v1:172.21.15.71:6827/3442634960] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:21:05.890 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{1:24243} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.192:6824/3780549896,v1:172.21.15.192:6825/3780549896] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:21:05.890 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{1:24255} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2100078284,v1:172.21.15.192:6827/2100078284] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:21:05.890 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:21:05.890 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:21:05.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.885+0000 7fa64da90700 1 -- 172.21.15.71:0/84445861 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa63006cce0 msgr2=0x7fa63006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:05.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.885+0000 7fa64da90700 1 --2- 172.21.15.71:0/84445861 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa63006cce0 0x7fa63006f1a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa638009a10 tx=0x7fa638009190 comp rx=0 tx=0).stop 2024-04-23T02:21:05.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.885+0000 7fa64da90700 1 -- 172.21.15.71:0/84445861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa648100750 msgr2=0x7fa648197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:05.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.885+0000 7fa64da90700 1 --2- 172.21.15.71:0/84445861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa648100750 0x7fa648197ce0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa63c005f50 tx=0x7fa63c00bd70 comp rx=0 tx=0).stop 2024-04-23T02:21:05.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.886+0000 7fa64da90700 1 -- 172.21.15.71:0/84445861 shutdown_connections 2024-04-23T02:21:05.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.886+0000 7fa64da90700 1 --2- 172.21.15.71:0/84445861 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa63006cce0 0x7fa63006f1a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:05.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.886+0000 7fa64da90700 1 --2- 172.21.15.71:0/84445861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6480ffda0 0x7fa6481977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:05.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.886+0000 7fa64da90700 1 --2- 172.21.15.71:0/84445861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa648100750 0x7fa648197ce0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:05.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.886+0000 7fa64da90700 1 -- 172.21.15.71:0/84445861 >> 172.21.15.71:0/84445861 conn(0x7fa6480fb650 msgr2=0x7fa6480fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:05.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.886+0000 7fa64da90700 1 -- 172.21.15.71:0/84445861 shutdown_connections 2024-04-23T02:21:05.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.886+0000 7fa64da90700 1 -- 172.21.15.71:0/84445861 wait complete. 2024-04-23T02:21:05.892 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 25 2024-04-23T02:21:05.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:05 smithi071 ceph-mon[144544]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:21:05.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:21:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:05 smithi071 ceph-mon[144544]: pgmap v218: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:21:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-23T02:21:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:05 smithi071 ceph-mon[144544]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-23T02:21:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:05 smithi071 ceph-mon[144544]: Upgrade: osd.4 is safe to restart 2024-04-23T02:21:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:05 smithi071 ceph-mon[144544]: from='client.34258 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:21:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:05 smithi071 ceph-mon[144544]: from='client.34262 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:21:05.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:05 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/4067509248' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:21:05.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.990+0000 7f59bb868700 1 -- 172.21.15.71:0/4267812039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59b40ffb40 msgr2=0x7f59b40fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:05.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.990+0000 7f59bb868700 1 --2- 172.21.15.71:0/4267812039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59b40ffb40 0x7f59b40fffc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009910 tx=0x7f59b0009c20 comp rx=0 tx=0).stop 2024-04-23T02:21:05.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.990+0000 7f59bb868700 1 -- 172.21.15.71:0/4267812039 shutdown_connections 2024-04-23T02:21:05.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.990+0000 7f59bb868700 1 --2- 172.21.15.71:0/4267812039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59b40ffb40 0x7f59b40fffc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:05.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.990+0000 7f59bb868700 1 --2- 172.21.15.71:0/4267812039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59b4105bf0 0x7f59b4105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:05.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.990+0000 7f59bb868700 1 -- 172.21.15.71:0/4267812039 >> 172.21.15.71:0/4267812039 conn(0x7f59b40fb610 msgr2=0x7f59b40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:05.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.990+0000 7f59bb868700 1 -- 172.21.15.71:0/4267812039 shutdown_connections 2024-04-23T02:21:05.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.990+0000 7f59bb868700 1 -- 172.21.15.71:0/4267812039 wait complete. 2024-04-23T02:21:05.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.991+0000 7f59bb868700 1 Processor -- start 2024-04-23T02:21:05.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.991+0000 7f59bb868700 1 -- start start 2024-04-23T02:21:05.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.991+0000 7f59bb868700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59b40ffb40 0x7f59b419c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:05.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.991+0000 7f59bb868700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59b4105bf0 0x7f59b419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:05.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.991+0000 7f59bb868700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b419d1a0 con 0x7f59b4105bf0 2024-04-23T02:21:05.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.991+0000 7f59bb868700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b4196650 con 0x7f59b40ffb40 2024-04-23T02:21:05.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.991+0000 7f59b8e03700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59b4105bf0 0x7f59b419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:05.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.991+0000 7f59b8e03700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59b4105bf0 0x7f59b419cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55342/0 (socket says 172.21.15.71:55342) 2024-04-23T02:21:05.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.991+0000 7f59b8e03700 1 -- 172.21.15.71:0/1041053192 learned_addr learned my addr 172.21.15.71:0/1041053192 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:21:05.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.991+0000 7f59b9604700 1 --2- 172.21.15.71:0/1041053192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59b40ffb40 0x7f59b419c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:05.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.991+0000 7f59b8e03700 1 -- 172.21.15.71:0/1041053192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59b40ffb40 msgr2=0x7f59b419c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:05.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.991+0000 7f59b8e03700 1 --2- 172.21.15.71:0/1041053192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59b40ffb40 0x7f59b419c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:05.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.992+0000 7f59b8e03700 1 -- 172.21.15.71:0/1041053192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b00095f0 con 0x7f59b4105bf0 2024-04-23T02:21:05.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.992+0000 7f59b8e03700 1 --2- 172.21.15.71:0/1041053192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59b4105bf0 0x7f59b419cb10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f59b0006010 tx=0x7f59b000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:05.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.992+0000 7f59aa7fc700 1 -- 172.21.15.71:0/1041053192 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59b001d070 con 0x7f59b4105bf0 2024-04-23T02:21:05.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.992+0000 7f59aa7fc700 1 -- 172.21.15.71:0/1041053192 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f59b0005710 con 0x7f59b4105bf0 2024-04-23T02:21:05.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.992+0000 7f59bb868700 1 -- 172.21.15.71:0/1041053192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b41968d0 con 0x7f59b4105bf0 2024-04-23T02:21:05.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.992+0000 7f59aa7fc700 1 -- 172.21.15.71:0/1041053192 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59b000fe20 con 0x7f59b4105bf0 2024-04-23T02:21:05.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.992+0000 7f59bb868700 1 -- 172.21.15.71:0/1041053192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b4196dc0 con 0x7f59b4105bf0 2024-04-23T02:21:05.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.993+0000 7f59bb868700 1 -- 172.21.15.71:0/1041053192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b40679c0 con 0x7f59b4105bf0 2024-04-23T02:21:05.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.993+0000 7f59aa7fc700 1 -- 172.21.15.71:0/1041053192 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f59b002b440 con 0x7f59b4105bf0 2024-04-23T02:21:05.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.994+0000 7f59aa7fc700 1 --2- 172.21.15.71:0/1041053192 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f59a006cce0 0x7f59a006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:05.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.994+0000 7f59aa7fc700 1 -- 172.21.15.71:0/1041053192 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f59b008d670 con 0x7f59b4105bf0 2024-04-23T02:21:05.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.994+0000 7f59b9604700 1 --2- 172.21.15.71:0/1041053192 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f59a006cce0 0x7f59a006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:06.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.994+0000 7f59b9604700 1 --2- 172.21.15.71:0/1041053192 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f59a006cce0 0x7f59a006f1a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f59a4004630 tx=0x7f59a400b3c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:06.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:05.998+0000 7f59aa7fc700 1 -- 172.21.15.71:0/1041053192 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f59b005b290 con 0x7f59b4105bf0 2024-04-23T02:21:06.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:05 smithi192 ceph-mon[108617]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:21:06.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:06.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:21:06.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:05 smithi192 ceph-mon[108617]: pgmap v218: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:21:06.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:06.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-23T02:21:06.037 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:05 smithi192 ceph-mon[108617]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-23T02:21:06.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:05 smithi192 ceph-mon[108617]: Upgrade: osd.4 is safe to restart 2024-04-23T02:21:06.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:05 smithi192 ceph-mon[108617]: from='client.34258 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:21:06.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:05 smithi192 ceph-mon[108617]: from='client.34262 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:21:06.038 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:05 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/4067509248' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:21:06.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.209+0000 7f59bb868700 1 -- 172.21.15.71:0/1041053192 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f59b41970a0 con 0x7f59a006cce0 2024-04-23T02:21:06.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.210+0000 7f59aa7fc700 1 -- 172.21.15.71:0/1041053192 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f59b41970a0 con 0x7f59a006cce0 2024-04-23T02:21:06.214 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:21:06.214 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:21:06.214 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:21:06.214 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:21:06.214 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:21:06.214 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:21:06.214 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:21:06.214 INFO:teuthology.orchestra.run.smithi071.stdout: "mon" 2024-04-23T02:21:06.214 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:21:06.214 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "10/23 daemons upgraded", 2024-04-23T02:21:06.214 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading osd daemons", 2024-04-23T02:21:06.214 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:21:06.214 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:21:06.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.212+0000 7f59bb868700 1 -- 172.21.15.71:0/1041053192 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f59a006cce0 msgr2=0x7f59a006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:06.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.212+0000 7f59bb868700 1 --2- 172.21.15.71:0/1041053192 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f59a006cce0 0x7f59a006f1a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f59a4004630 tx=0x7f59a400b3c0 comp rx=0 tx=0).stop 2024-04-23T02:21:06.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.212+0000 7f59bb868700 1 -- 172.21.15.71:0/1041053192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59b4105bf0 msgr2=0x7f59b419cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:06.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.212+0000 7f59bb868700 1 --2- 172.21.15.71:0/1041053192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59b4105bf0 0x7f59b419cb10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f59b0006010 tx=0x7f59b000bd70 comp rx=0 tx=0).stop 2024-04-23T02:21:06.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.212+0000 7f59bb868700 1 -- 172.21.15.71:0/1041053192 shutdown_connections 2024-04-23T02:21:06.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.212+0000 7f59bb868700 1 --2- 172.21.15.71:0/1041053192 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f59a006cce0 0x7f59a006f1a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:06.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.212+0000 7f59bb868700 1 --2- 172.21.15.71:0/1041053192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59b40ffb40 0x7f59b419c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:06.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.212+0000 7f59bb868700 1 --2- 172.21.15.71:0/1041053192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59b4105bf0 0x7f59b419cb10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:06.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.212+0000 7f59bb868700 1 -- 172.21.15.71:0/1041053192 >> 172.21.15.71:0/1041053192 conn(0x7f59b40fb610 msgr2=0x7f59b4103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:06.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.212+0000 7f59bb868700 1 -- 172.21.15.71:0/1041053192 shutdown_connections 2024-04-23T02:21:06.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.212+0000 7f59bb868700 1 -- 172.21.15.71:0/1041053192 wait complete. 2024-04-23T02:21:06.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.314+0000 7f14f3fff700 1 -- 172.21.15.71:0/2576979308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14f4105950 msgr2=0x7f14f4105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:06.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.314+0000 7f14f3fff700 1 --2- 172.21.15.71:0/2576979308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14f4105950 0x7f14f4105d30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009910 tx=0x7f14e4009c20 comp rx=0 tx=0).stop 2024-04-23T02:21:06.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.314+0000 7f14f3fff700 1 -- 172.21.15.71:0/2576979308 shutdown_connections 2024-04-23T02:21:06.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.314+0000 7f14f3fff700 1 --2- 172.21.15.71:0/2576979308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14f40ffa20 0x7f14f40ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:06.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.314+0000 7f14f3fff700 1 --2- 172.21.15.71:0/2576979308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14f4105950 0x7f14f4105d30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:06.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.314+0000 7f14f3fff700 1 -- 172.21.15.71:0/2576979308 >> 172.21.15.71:0/2576979308 conn(0x7f14f4075460 msgr2=0x7f14f4075870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:06.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.315+0000 7f14f3fff700 1 -- 172.21.15.71:0/2576979308 shutdown_connections 2024-04-23T02:21:06.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.315+0000 7f14f3fff700 1 -- 172.21.15.71:0/2576979308 wait complete. 2024-04-23T02:21:06.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.315+0000 7f14f3fff700 1 Processor -- start 2024-04-23T02:21:06.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.315+0000 7f14f3fff700 1 -- start start 2024-04-23T02:21:06.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.315+0000 7f14f3fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14f40ffa20 0x7f14f41954e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:06.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.315+0000 7f14f3fff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14f4105950 0x7f14f4195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:06.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.315+0000 7f14f3fff700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f4196100 con 0x7f14f40ffa20 2024-04-23T02:21:06.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.315+0000 7f14f3fff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f4199e90 con 0x7f14f4105950 2024-04-23T02:21:06.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.316+0000 7f14f2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14f40ffa20 0x7f14f41954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:06.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.316+0000 7f14f2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14f40ffa20 0x7f14f41954e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55364/0 (socket says 172.21.15.71:55364) 2024-04-23T02:21:06.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.316+0000 7f14f2ffd700 1 -- 172.21.15.71:0/572156636 learned_addr learned my addr 172.21.15.71:0/572156636 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:21:06.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.316+0000 7f14f27fc700 1 --2- 172.21.15.71:0/572156636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14f4105950 0x7f14f4195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:06.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.316+0000 7f14f2ffd700 1 -- 172.21.15.71:0/572156636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14f4105950 msgr2=0x7f14f4195a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:06.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.316+0000 7f14f2ffd700 1 --2- 172.21.15.71:0/572156636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14f4105950 0x7f14f4195a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:06.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.316+0000 7f14f2ffd700 1 -- 172.21.15.71:0/572156636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e40095f0 con 0x7f14f40ffa20 2024-04-23T02:21:06.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.316+0000 7f14f2ffd700 1 --2- 172.21.15.71:0/572156636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14f40ffa20 0x7f14f41954e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f14e4000c00 tx=0x7f14e400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:06.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.316+0000 7f14dbfff700 1 -- 172.21.15.71:0/572156636 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14e401d070 con 0x7f14f40ffa20 2024-04-23T02:21:06.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.316+0000 7f14dbfff700 1 -- 172.21.15.71:0/572156636 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f14e4005dc0 con 0x7f14f40ffa20 2024-04-23T02:21:06.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.316+0000 7f14f3fff700 1 -- 172.21.15.71:0/572156636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f419a110 con 0x7f14f40ffa20 2024-04-23T02:21:06.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.316+0000 7f14dbfff700 1 -- 172.21.15.71:0/572156636 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14e40039d0 con 0x7f14f40ffa20 2024-04-23T02:21:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.316+0000 7f14f3fff700 1 -- 172.21.15.71:0/572156636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f419a5a0 con 0x7f14f40ffa20 2024-04-23T02:21:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.317+0000 7f14f3fff700 1 -- 172.21.15.71:0/572156636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f4107e90 con 0x7f14f40ffa20 2024-04-23T02:21:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.318+0000 7f14dbfff700 1 -- 172.21.15.71:0/572156636 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f14e4003b30 con 0x7f14f40ffa20 2024-04-23T02:21:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.318+0000 7f14dbfff700 1 --2- 172.21.15.71:0/572156636 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f14dc06cce0 0x7f14dc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.318+0000 7f14dbfff700 1 -- 172.21.15.71:0/572156636 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f14e408dae0 con 0x7f14f40ffa20 2024-04-23T02:21:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.318+0000 7f14f27fc700 1 --2- 172.21.15.71:0/572156636 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f14dc06cce0 0x7f14dc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.318+0000 7f14f27fc700 1 --2- 172.21.15.71:0/572156636 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f14dc06cce0 0x7f14dc06f1a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f14f4196b00 tx=0x7f14e8009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:06.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.322+0000 7f14dbfff700 1 -- 172.21.15.71:0/572156636 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f14e4057c00 con 0x7f14f40ffa20 2024-04-23T02:21:06.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.595+0000 7f14f3fff700 1 -- 172.21.15.71:0/572156636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f14f4107870 con 0x7f14f40ffa20 2024-04-23T02:21:06.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.595+0000 7f14dbfff700 1 -- 172.21.15.71:0/572156636 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f14e405b220 con 0x7f14f40ffa20 2024-04-23T02:21:06.599 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-23T02:21:06.599 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:21:06.599 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:21:06.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.597+0000 7f14f3fff700 1 -- 172.21.15.71:0/572156636 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f14dc06cce0 msgr2=0x7f14dc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:06.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.597+0000 7f14f3fff700 1 --2- 172.21.15.71:0/572156636 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f14dc06cce0 0x7f14dc06f1a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f14f4196b00 tx=0x7f14e8009260 comp rx=0 tx=0).stop 2024-04-23T02:21:06.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.597+0000 7f14f3fff700 1 -- 172.21.15.71:0/572156636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14f40ffa20 msgr2=0x7f14f41954e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:06.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.597+0000 7f14f3fff700 1 --2- 172.21.15.71:0/572156636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14f40ffa20 0x7f14f41954e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f14e4000c00 tx=0x7f14e400bde0 comp rx=0 tx=0).stop 2024-04-23T02:21:06.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.597+0000 7f14f3fff700 1 -- 172.21.15.71:0/572156636 shutdown_connections 2024-04-23T02:21:06.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.597+0000 7f14f3fff700 1 --2- 172.21.15.71:0/572156636 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f14dc06cce0 0x7f14dc06f1a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:06.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.597+0000 7f14f3fff700 1 --2- 172.21.15.71:0/572156636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14f40ffa20 0x7f14f41954e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:06.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.597+0000 7f14f3fff700 1 --2- 172.21.15.71:0/572156636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14f4105950 0x7f14f4195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:06.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.597+0000 7f14f3fff700 1 -- 172.21.15.71:0/572156636 >> 172.21.15.71:0/572156636 conn(0x7f14f4075460 msgr2=0x7f14f40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:06.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.598+0000 7f14f3fff700 1 -- 172.21.15.71:0/572156636 shutdown_connections 2024-04-23T02:21:06.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:06.598+0000 7f14f3fff700 1 -- 172.21.15.71:0/572156636 wait complete. 2024-04-23T02:21:06.966 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:06 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/84445861' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:21:06.966 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:06 smithi192 ceph-mon[108617]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:21:06.966 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:06 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:06.966 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:06 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T02:21:06.966 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:06 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:21:06.966 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:06 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/572156636' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:21:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:06 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/84445861' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:21:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:06 smithi071 ceph-mon[144544]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:21:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:06 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:06 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T02:21:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:06 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:21:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:06 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/572156636' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:21:07.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:07 smithi192 ceph-mon[108617]: Upgrade: Updating osd.4 2024-04-23T02:21:07.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:07 smithi192 ceph-mon[108617]: Deploying daemon osd.4 on smithi192 2024-04-23T02:21:07.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:07 smithi192 ceph-mon[108617]: pgmap v219: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:21:07.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:07 smithi071 ceph-mon[144544]: Upgrade: Updating osd.4 2024-04-23T02:21:07.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:07 smithi071 ceph-mon[144544]: Deploying daemon osd.4 on smithi192 2024-04-23T02:21:07.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:07 smithi071 ceph-mon[144544]: pgmap v219: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:21:10.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:09 smithi192 ceph-mon[108617]: pgmap v220: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:21:10.127 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:09 smithi192 systemd[1]: Stopping Ceph osd.4 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T02:21:10.127 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:09 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[64749]: 2024-04-23T02:21:09.881+0000 7f9ec56eb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T02:21:10.127 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:09 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[64749]: 2024-04-23T02:21:09.881+0000 7f9ec56eb700 -1 osd.4 68 *** Got signal Terminated *** 2024-04-23T02:21:10.127 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:09 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[64749]: 2024-04-23T02:21:09.881+0000 7f9ec56eb700 -1 osd.4 68 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T02:21:10.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:09 smithi071 ceph-mon[144544]: pgmap v220: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:21:11.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:10 smithi192 ceph-mon[108617]: osd.4 marked itself down and dead 2024-04-23T02:21:11.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:10 smithi071 ceph-mon[144544]: osd.4 marked itself down and dead 2024-04-23T02:21:11.755 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:11 smithi192 bash[125391]: ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4 2024-04-23T02:21:12.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:11 smithi192 ceph-mon[108617]: pgmap v221: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:21:12.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:11 smithi192 ceph-mon[108617]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-23T02:21:12.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:11 smithi192 ceph-mon[108617]: osdmap e69: 6 total, 5 up, 6 in 2024-04-23T02:21:12.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:11 smithi071 ceph-mon[144544]: pgmap v221: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:21:12.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:11 smithi071 ceph-mon[144544]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-23T02:21:12.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:11 smithi071 ceph-mon[144544]: osdmap e69: 6 total, 5 up, 6 in 2024-04-23T02:21:12.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:12 smithi071 ceph-mon[144544]: osdmap e70: 6 total, 5 up, 6 in 2024-04-23T02:21:13.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:12 smithi192 ceph-mon[108617]: osdmap e70: 6 total, 5 up, 6 in 2024-04-23T02:21:13.824 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:13 smithi192 systemd[1]: ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.4.service: Succeeded. 2024-04-23T02:21:13.824 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:13 smithi192 systemd[1]: Stopped Ceph osd.4 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:21:13.824 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:13 smithi192 systemd[1]: Starting Ceph osd.4 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T02:21:14.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:13 smithi192 ceph-mon[108617]: pgmap v224: 65 pgs: 10 peering, 3 stale+active+clean, 52 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s 2024-04-23T02:21:14.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:13 smithi192 ceph-mon[108617]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-23T02:21:14.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:13 smithi071 ceph-mon[144544]: pgmap v224: 65 pgs: 10 peering, 3 stale+active+clean, 52 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s 2024-04-23T02:21:14.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:13 smithi071 ceph-mon[144544]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-23T02:21:14.875 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:14 smithi192 bash[125670]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-23T02:21:14.875 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:14 smithi192 bash[125670]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-23T02:21:14.875 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:14 smithi192 bash[125670]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-23T02:21:14.875 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:14 smithi192 bash[125670]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-23T02:21:14.876 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:14 smithi192 bash[125670]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-23T02:21:14.876 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:14 smithi192 bash[125670]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-23T02:21:14.876 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:14 smithi192 bash[125670]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-23T02:21:15.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:14 smithi192 ceph-mon[108617]: Health check failed: Degraded data redundancy: 41/366 objects degraded (11.202%), 10 pgs degraded (PG_DEGRADED) 2024-04-23T02:21:15.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:14 smithi071 ceph-mon[144544]: Health check failed: Degraded data redundancy: 41/366 objects degraded (11.202%), 10 pgs degraded (PG_DEGRADED) 2024-04-23T02:21:16.186 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:15 smithi192 ceph-mon[108617]: pgmap v225: 65 pgs: 10 active+undersized+degraded, 10 peering, 1 stale+active+clean, 9 active+undersized, 35 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/366 objects degraded (11.202%) 2024-04-23T02:21:16.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:15 smithi071 ceph-mon[144544]: pgmap v225: 65 pgs: 10 active+undersized+degraded, 10 peering, 1 stale+active+clean, 9 active+undersized, 35 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/366 objects degraded (11.202%) 2024-04-23T02:21:16.606 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:16 smithi192 bash[125957]: c4eff10fbc303282a3ba4f5008f3c046126c182e504fd51c1533975ea1069a3c 2024-04-23T02:21:16.606 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:16 smithi192 systemd[1]: Started Ceph osd.4 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:21:16.607 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:16 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[125971]: 2024-04-23T02:21:16.601+0000 7fa25d98d7c0 -1 Falling back to public interface 2024-04-23T02:21:17.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:17 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:17.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:17 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:17.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:17 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:21:17.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:17 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:17.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:17 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:17.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:17 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:21:18.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:18 smithi071 ceph-mon[144544]: pgmap v226: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 62/366 objects degraded (16.940%) 2024-04-23T02:21:18.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:18 smithi071 ceph-mon[144544]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-23T02:21:18.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:18 smithi192 ceph-mon[108617]: pgmap v226: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 62/366 objects degraded (16.940%) 2024-04-23T02:21:18.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:18 smithi192 ceph-mon[108617]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-23T02:21:20.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:20 smithi071 ceph-mon[144544]: pgmap v227: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 62/366 objects degraded (16.940%) 2024-04-23T02:21:20.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:20.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:20.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:20.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:21:20.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:20.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:20.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:20 smithi192 ceph-mon[108617]: pgmap v227: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 62/366 objects degraded (16.940%) 2024-04-23T02:21:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:21:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:21.376 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:21 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[125971]: 2024-04-23T02:21:21.120+0000 7fa25d98d7c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-04-23T02:21:21.829 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:21 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[125971]: 2024-04-23T02:21:21.453+0000 7fa25d98d7c0 -1 osd.4 68 log_to_monitors true 2024-04-23T02:21:22.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:22 smithi192 ceph-mon[108617]: pgmap v228: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 62/366 objects degraded (16.940%) 2024-04-23T02:21:22.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:22 smithi192 ceph-mon[108617]: from='osd.4 [v2:172.21.15.192:6808/1707328698,v1:172.21.15.192:6809/1707328698]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T02:21:22.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:22 smithi192 ceph-mon[108617]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T02:21:22.633 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:22 smithi071 ceph-mon[144544]: pgmap v228: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 62/366 objects degraded (16.940%) 2024-04-23T02:21:22.633 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:22 smithi071 ceph-mon[144544]: from='osd.4 [v2:172.21.15.192:6808/1707328698,v1:172.21.15.192:6809/1707328698]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T02:21:22.633 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:22 smithi071 ceph-mon[144544]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T02:21:23.427 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 02:21:23 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[125971]: 2024-04-23T02:21:23.122+0000 7fa24addf700 -1 osd.4 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T02:21:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:23 smithi192 ceph-mon[108617]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T02:21:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:23 smithi192 ceph-mon[108617]: osdmap e71: 6 total, 5 up, 6 in 2024-04-23T02:21:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:23 smithi192 ceph-mon[108617]: from='osd.4 [v2:172.21.15.192:6808/1707328698,v1:172.21.15.192:6809/1707328698]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:21:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:23 smithi192 ceph-mon[108617]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:21:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:23 smithi192 ceph-mon[108617]: Health check update: Degraded data redundancy: 62/366 objects degraded (16.940%), 16 pgs degraded (PG_DEGRADED) 2024-04-23T02:21:23.428 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:23 smithi192 ceph-mon[108617]: from='osd.4 ' entity='osd.4' 2024-04-23T02:21:23.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:23 smithi071 ceph-mon[144544]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T02:21:23.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:23 smithi071 ceph-mon[144544]: osdmap e71: 6 total, 5 up, 6 in 2024-04-23T02:21:23.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:23 smithi071 ceph-mon[144544]: from='osd.4 [v2:172.21.15.192:6808/1707328698,v1:172.21.15.192:6809/1707328698]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:21:23.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:23 smithi071 ceph-mon[144544]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:21:23.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:23 smithi071 ceph-mon[144544]: Health check update: Degraded data redundancy: 62/366 objects degraded (16.940%), 16 pgs degraded (PG_DEGRADED) 2024-04-23T02:21:23.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:23 smithi071 ceph-mon[144544]: from='osd.4 ' entity='osd.4' 2024-04-23T02:21:24.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:24 smithi192 ceph-mon[108617]: pgmap v230: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 62/366 objects degraded (16.940%) 2024-04-23T02:21:24.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:24 smithi192 ceph-mon[108617]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-23T02:21:24.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:24 smithi192 ceph-mon[108617]: osd.4 [v2:172.21.15.192:6808/1707328698,v1:172.21.15.192:6809/1707328698] boot 2024-04-23T02:21:24.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:24 smithi192 ceph-mon[108617]: osdmap e72: 6 total, 6 up, 6 in 2024-04-23T02:21:24.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:24 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T02:21:24.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:24 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:24.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:24 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:24.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:24 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:21:24.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:24 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:21:24.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:24 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:24.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:24 smithi071 ceph-mon[144544]: pgmap v230: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 62/366 objects degraded (16.940%) 2024-04-23T02:21:24.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:24 smithi071 ceph-mon[144544]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-23T02:21:24.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:24 smithi071 ceph-mon[144544]: osd.4 [v2:172.21.15.192:6808/1707328698,v1:172.21.15.192:6809/1707328698] boot 2024-04-23T02:21:24.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:24 smithi071 ceph-mon[144544]: osdmap e72: 6 total, 6 up, 6 in 2024-04-23T02:21:24.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:24 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T02:21:24.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:24 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:24.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:24 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:24.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:24 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:21:24.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:24 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:21:24.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:24 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:25.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:21:25.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:21:25.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:21:25.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:21:25.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-23T02:21:25.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:25 smithi192 ceph-mon[108617]: osdmap e73: 6 total, 6 up, 6 in 2024-04-23T02:21:25.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:21:25.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:21:25.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:21:25.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:21:25.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-23T02:21:25.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:25 smithi071 ceph-mon[144544]: osdmap e73: 6 total, 6 up, 6 in 2024-04-23T02:21:26.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:26 smithi192 ceph-mon[108617]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-23T02:21:26.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:26 smithi192 ceph-mon[108617]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-23T02:21:26.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:26 smithi192 ceph-mon[108617]: pgmap v232: 65 pgs: 7 peering, 12 active+undersized+degraded, 15 active+undersized, 31 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 50/366 objects degraded (13.661%) 2024-04-23T02:21:26.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:26 smithi071 ceph-mon[144544]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-23T02:21:26.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:26 smithi071 ceph-mon[144544]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-23T02:21:26.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:26 smithi071 ceph-mon[144544]: pgmap v232: 65 pgs: 7 peering, 12 active+undersized+degraded, 15 active+undersized, 31 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 50/366 objects degraded (13.661%) 2024-04-23T02:21:28.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:28 smithi192 ceph-mon[108617]: pgmap v234: 65 pgs: 7 peering, 8 active+undersized+degraded, 8 active+undersized, 42 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s; 33/366 objects degraded (9.016%) 2024-04-23T02:21:28.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:28 smithi192 ceph-mon[108617]: Health check update: Degraded data redundancy: 33/366 objects degraded (9.016%), 8 pgs degraded (PG_DEGRADED) 2024-04-23T02:21:28.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:28 smithi071 ceph-mon[144544]: pgmap v234: 65 pgs: 7 peering, 8 active+undersized+degraded, 8 active+undersized, 42 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s; 33/366 objects degraded (9.016%) 2024-04-23T02:21:28.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:28 smithi071 ceph-mon[144544]: Health check update: Degraded data redundancy: 33/366 objects degraded (9.016%), 8 pgs degraded (PG_DEGRADED) 2024-04-23T02:21:29.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:29 smithi192 ceph-mon[108617]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 33/366 objects degraded (9.016%), 8 pgs degraded) 2024-04-23T02:21:29.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:29 smithi071 ceph-mon[144544]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 33/366 objects degraded (9.016%), 8 pgs degraded) 2024-04-23T02:21:30.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:30 smithi192 ceph-mon[108617]: pgmap v235: 65 pgs: 7 peering, 58 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-23T02:21:30.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:30 smithi071 ceph-mon[144544]: pgmap v235: 65 pgs: 7 peering, 58 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-23T02:21:32.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:32 smithi192 ceph-mon[108617]: pgmap v236: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:21:32.633 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:32 smithi071 ceph-mon[144544]: pgmap v236: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:21:34.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:34 smithi192 ceph-mon[108617]: pgmap v237: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 3 op/s 2024-04-23T02:21:34.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:34 smithi071 ceph-mon[144544]: pgmap v237: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 3 op/s 2024-04-23T02:21:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:21:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:35 smithi071 ceph-mon[144544]: pgmap v238: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:21:36.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:36.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:21:36.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:35 smithi192 ceph-mon[108617]: pgmap v238: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:21:36.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.715+0000 7f5a5ee4b700 1 -- 172.21.15.71:0/4044517705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a58101850 msgr2=0x7f5a581058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:36.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.715+0000 7f5a5ee4b700 1 --2- 172.21.15.71:0/4044517705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a58101850 0x7f5a581058a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009910 tx=0x7f5a44009c20 comp rx=0 tx=0).stop 2024-04-23T02:21:36.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.715+0000 7f5a5ee4b700 1 -- 172.21.15.71:0/4044517705 shutdown_connections 2024-04-23T02:21:36.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.715+0000 7f5a5ee4b700 1 --2- 172.21.15.71:0/4044517705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a58101850 0x7f5a581058a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:36.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.715+0000 7f5a5ee4b700 1 --2- 172.21.15.71:0/4044517705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a58100ea0 0x7f5a58101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:36.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.715+0000 7f5a5ee4b700 1 -- 172.21.15.71:0/4044517705 >> 172.21.15.71:0/4044517705 conn(0x7f5a580fc750 msgr2=0x7f5a580feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:36.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.715+0000 7f5a5ee4b700 1 -- 172.21.15.71:0/4044517705 shutdown_connections 2024-04-23T02:21:36.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.715+0000 7f5a5ee4b700 1 -- 172.21.15.71:0/4044517705 wait complete. 2024-04-23T02:21:36.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.716+0000 7f5a5ee4b700 1 Processor -- start 2024-04-23T02:21:36.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.716+0000 7f5a5ee4b700 1 -- start start 2024-04-23T02:21:36.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.716+0000 7f5a5ee4b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a58100ea0 0x7f5a5819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:36.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.716+0000 7f5a5ee4b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a58101850 0x7f5a5819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:36.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.716+0000 7f5a5ee4b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5819d160 con 0x7f5a58101850 2024-04-23T02:21:36.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.716+0000 7f5a5ee4b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a581966a0 con 0x7f5a58100ea0 2024-04-23T02:21:36.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.717+0000 7f5a57fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a58101850 0x7f5a5819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:36.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.717+0000 7f5a57fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a58101850 0x7f5a5819cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57184/0 (socket says 172.21.15.71:57184) 2024-04-23T02:21:36.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.717+0000 7f5a57fff700 1 -- 172.21.15.71:0/2352179420 learned_addr learned my addr 172.21.15.71:0/2352179420 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:21:36.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.717+0000 7f5a5cbe7700 1 --2- 172.21.15.71:0/2352179420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a58100ea0 0x7f5a5819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:36.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.717+0000 7f5a57fff700 1 -- 172.21.15.71:0/2352179420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a58100ea0 msgr2=0x7f5a5819c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:36.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.717+0000 7f5a57fff700 1 --2- 172.21.15.71:0/2352179420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a58100ea0 0x7f5a5819c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:36.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.717+0000 7f5a57fff700 1 -- 172.21.15.71:0/2352179420 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a440095f0 con 0x7f5a58101850 2024-04-23T02:21:36.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.717+0000 7f5a5cbe7700 1 --2- 172.21.15.71:0/2352179420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a58100ea0 0x7f5a5819c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:21:36.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.717+0000 7f5a57fff700 1 --2- 172.21.15.71:0/2352179420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a58101850 0x7f5a5819cb60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400bda0 tx=0x7f5a4400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:36.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.717+0000 7f5a55ffb700 1 -- 172.21.15.71:0/2352179420 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a4401d070 con 0x7f5a58101850 2024-04-23T02:21:36.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.717+0000 7f5a55ffb700 1 -- 172.21.15.71:0/2352179420 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5a4400f460 con 0x7f5a58101850 2024-04-23T02:21:36.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.717+0000 7f5a5ee4b700 1 -- 172.21.15.71:0/2352179420 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a58196920 con 0x7f5a58101850 2024-04-23T02:21:36.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.717+0000 7f5a55ffb700 1 -- 172.21.15.71:0/2352179420 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a44021640 con 0x7f5a58101850 2024-04-23T02:21:36.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.718+0000 7f5a5ee4b700 1 -- 172.21.15.71:0/2352179420 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a58196e10 con 0x7f5a58101850 2024-04-23T02:21:36.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.718+0000 7f5a5ee4b700 1 -- 172.21.15.71:0/2352179420 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a581092a0 con 0x7f5a58101850 2024-04-23T02:21:36.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.719+0000 7f5a55ffb700 1 -- 172.21.15.71:0/2352179420 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5a4400f9b0 con 0x7f5a58101850 2024-04-23T02:21:36.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.719+0000 7f5a55ffb700 1 --2- 172.21.15.71:0/2352179420 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5a4806cd30 0x7f5a4806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:36.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.719+0000 7f5a55ffb700 1 -- 172.21.15.71:0/2352179420 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5a4408d760 con 0x7f5a58101850 2024-04-23T02:21:36.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.719+0000 7f5a5cbe7700 1 --2- 172.21.15.71:0/2352179420 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5a4806cd30 0x7f5a4806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:36.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.720+0000 7f5a5cbe7700 1 --2- 172.21.15.71:0/2352179420 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5a4806cd30 0x7f5a4806f1f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c005fd0 tx=0x7f5a4c005e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:36.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.724+0000 7f5a55ffb700 1 -- 172.21.15.71:0/2352179420 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5a4405b2f0 con 0x7f5a58101850 2024-04-23T02:21:36.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.936+0000 7f5a5ee4b700 1 -- 172.21.15.71:0/2352179420 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a58197c70 con 0x7f5a4806cd30 2024-04-23T02:21:36.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.937+0000 7f5a55ffb700 1 -- 172.21.15.71:0/2352179420 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f5a58197c70 con 0x7f5a4806cd30 2024-04-23T02:21:36.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.939+0000 7f5a5ee4b700 1 -- 172.21.15.71:0/2352179420 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5a4806cd30 msgr2=0x7f5a4806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:36.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.939+0000 7f5a5ee4b700 1 --2- 172.21.15.71:0/2352179420 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5a4806cd30 0x7f5a4806f1f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c005fd0 tx=0x7f5a4c005e30 comp rx=0 tx=0).stop 2024-04-23T02:21:36.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.939+0000 7f5a5ee4b700 1 -- 172.21.15.71:0/2352179420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a58101850 msgr2=0x7f5a5819cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:36.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.939+0000 7f5a5ee4b700 1 --2- 172.21.15.71:0/2352179420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a58101850 0x7f5a5819cb60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400bda0 tx=0x7f5a4400be80 comp rx=0 tx=0).stop 2024-04-23T02:21:36.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.939+0000 7f5a5ee4b700 1 -- 172.21.15.71:0/2352179420 shutdown_connections 2024-04-23T02:21:36.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.939+0000 7f5a5ee4b700 1 --2- 172.21.15.71:0/2352179420 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5a4806cd30 0x7f5a4806f1f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:36.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.939+0000 7f5a5ee4b700 1 --2- 172.21.15.71:0/2352179420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a58100ea0 0x7f5a5819c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:36.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.939+0000 7f5a5ee4b700 1 --2- 172.21.15.71:0/2352179420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a58101850 0x7f5a5819cb60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:36.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.939+0000 7f5a5ee4b700 1 -- 172.21.15.71:0/2352179420 >> 172.21.15.71:0/2352179420 conn(0x7f5a580fc750 msgr2=0x7f5a580fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:36.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.939+0000 7f5a5ee4b700 1 -- 172.21.15.71:0/2352179420 shutdown_connections 2024-04-23T02:21:36.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:36.940+0000 7f5a5ee4b700 1 -- 172.21.15.71:0/2352179420 wait complete. 2024-04-23T02:21:36.953 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:21:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.043+0000 7fdd460f1700 1 -- 172.21.15.71:0/3929532693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd40102760 msgr2=0x7fdd4010ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.043+0000 7fdd460f1700 1 --2- 172.21.15.71:0/3929532693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd40102760 0x7fdd4010ac50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009940 tx=0x7fdd34009c50 comp rx=0 tx=0).stop 2024-04-23T02:21:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.043+0000 7fdd460f1700 1 -- 172.21.15.71:0/3929532693 shutdown_connections 2024-04-23T02:21:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.043+0000 7fdd460f1700 1 --2- 172.21.15.71:0/3929532693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd40102760 0x7fdd4010ac50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.043+0000 7fdd460f1700 1 --2- 172.21.15.71:0/3929532693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd40101e40 0x7fdd40102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.043+0000 7fdd460f1700 1 -- 172.21.15.71:0/3929532693 >> 172.21.15.71:0/3929532693 conn(0x7fdd400fb650 msgr2=0x7fdd400fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.043+0000 7fdd460f1700 1 -- 172.21.15.71:0/3929532693 shutdown_connections 2024-04-23T02:21:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.043+0000 7fdd460f1700 1 -- 172.21.15.71:0/3929532693 wait complete. 2024-04-23T02:21:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.044+0000 7fdd460f1700 1 Processor -- start 2024-04-23T02:21:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.044+0000 7fdd460f1700 1 -- start start 2024-04-23T02:21:37.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.044+0000 7fdd460f1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd40101e40 0x7fdd4019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:37.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.044+0000 7fdd460f1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd40102760 0x7fdd4019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:37.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.044+0000 7fdd460f1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4019d210 con 0x7fdd40101e40 2024-04-23T02:21:37.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.044+0000 7fdd460f1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd401966c0 con 0x7fdd40102760 2024-04-23T02:21:37.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.044+0000 7fdd3f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd40101e40 0x7fdd4019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:37.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.044+0000 7fdd3f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd40101e40 0x7fdd4019c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57196/0 (socket says 172.21.15.71:57196) 2024-04-23T02:21:37.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.044+0000 7fdd3f7fe700 1 -- 172.21.15.71:0/2798956370 learned_addr learned my addr 172.21.15.71:0/2798956370 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:21:37.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.044+0000 7fdd3effd700 1 --2- 172.21.15.71:0/2798956370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd40102760 0x7fdd4019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:37.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.044+0000 7fdd3f7fe700 1 -- 172.21.15.71:0/2798956370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd40102760 msgr2=0x7fdd4019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:37.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.044+0000 7fdd3f7fe700 1 --2- 172.21.15.71:0/2798956370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd40102760 0x7fdd4019cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.044+0000 7fdd3f7fe700 1 -- 172.21.15.71:0/2798956370 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd340095f0 con 0x7fdd40101e40 2024-04-23T02:21:37.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.045+0000 7fdd3effd700 1 --2- 172.21.15.71:0/2798956370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd40102760 0x7fdd4019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:21:37.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.045+0000 7fdd3f7fe700 1 --2- 172.21.15.71:0/2798956370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd40101e40 0x7fdd4019c640 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000d8d0 tx=0x7fdd3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:37.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.045+0000 7fdd3cff9700 1 -- 172.21.15.71:0/2798956370 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd30009940 con 0x7fdd40101e40 2024-04-23T02:21:37.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.045+0000 7fdd3cff9700 1 -- 172.21.15.71:0/2798956370 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd30010460 con 0x7fdd40101e40 2024-04-23T02:21:37.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.045+0000 7fdd460f1700 1 -- 172.21.15.71:0/2798956370 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd401969a0 con 0x7fdd40101e40 2024-04-23T02:21:37.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.045+0000 7fdd3cff9700 1 -- 172.21.15.71:0/2798956370 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd3000f5f0 con 0x7fdd40101e40 2024-04-23T02:21:37.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.045+0000 7fdd460f1700 1 -- 172.21.15.71:0/2798956370 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd40196ec0 con 0x7fdd40101e40 2024-04-23T02:21:37.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.046+0000 7fdd460f1700 1 -- 172.21.15.71:0/2798956370 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd40108350 con 0x7fdd40101e40 2024-04-23T02:21:37.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.047+0000 7fdd3cff9700 1 -- 172.21.15.71:0/2798956370 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fdd300105d0 con 0x7fdd40101e40 2024-04-23T02:21:37.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.047+0000 7fdd3cff9700 1 --2- 172.21.15.71:0/2798956370 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdd2806cce0 0x7fdd2806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:37.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.047+0000 7fdd3cff9700 1 -- 172.21.15.71:0/2798956370 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fdd3008c530 con 0x7fdd40101e40 2024-04-23T02:21:37.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.047+0000 7fdd3effd700 1 --2- 172.21.15.71:0/2798956370 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdd2806cce0 0x7fdd2806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:37.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.048+0000 7fdd3effd700 1 --2- 172.21.15.71:0/2798956370 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdd2806cce0 0x7fdd2806f1a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009910 tx=0x7fdd34005ad0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:37.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.052+0000 7fdd3cff9700 1 -- 172.21.15.71:0/2798956370 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fdd3005a0e0 con 0x7fdd40101e40 2024-04-23T02:21:37.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.271+0000 7fdd460f1700 1 -- 172.21.15.71:0/2798956370 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd40197ce0 con 0x7fdd2806cce0 2024-04-23T02:21:37.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.272+0000 7fdd3cff9700 1 -- 172.21.15.71:0/2798956370 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fdd40197ce0 con 0x7fdd2806cce0 2024-04-23T02:21:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.274+0000 7fdd460f1700 1 -- 172.21.15.71:0/2798956370 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdd2806cce0 msgr2=0x7fdd2806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.274+0000 7fdd460f1700 1 --2- 172.21.15.71:0/2798956370 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdd2806cce0 0x7fdd2806f1a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009910 tx=0x7fdd34005ad0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.274+0000 7fdd460f1700 1 -- 172.21.15.71:0/2798956370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd40101e40 msgr2=0x7fdd4019c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.274+0000 7fdd460f1700 1 --2- 172.21.15.71:0/2798956370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd40101e40 0x7fdd4019c640 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000d8d0 tx=0x7fdd3000dc90 comp rx=0 tx=0).stop 2024-04-23T02:21:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.274+0000 7fdd460f1700 1 -- 172.21.15.71:0/2798956370 shutdown_connections 2024-04-23T02:21:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.274+0000 7fdd460f1700 1 --2- 172.21.15.71:0/2798956370 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdd2806cce0 0x7fdd2806f1a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.274+0000 7fdd460f1700 1 --2- 172.21.15.71:0/2798956370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd40101e40 0x7fdd4019c640 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.274+0000 7fdd460f1700 1 --2- 172.21.15.71:0/2798956370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd40102760 0x7fdd4019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.274+0000 7fdd460f1700 1 -- 172.21.15.71:0/2798956370 >> 172.21.15.71:0/2798956370 conn(0x7fdd400fb650 msgr2=0x7fdd40105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:37.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.275+0000 7fdd460f1700 1 -- 172.21.15.71:0/2798956370 shutdown_connections 2024-04-23T02:21:37.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.275+0000 7fdd460f1700 1 -- 172.21.15.71:0/2798956370 wait complete. 2024-04-23T02:21:37.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.377+0000 7fccce8c0700 1 -- 172.21.15.71:0/3116491959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccc80fff50 msgr2=0x7fccc8105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:37.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.377+0000 7fccce8c0700 1 --2- 172.21.15.71:0/3116491959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccc80fff50 0x7fccc8105840 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009910 tx=0x7fccb8009c20 comp rx=0 tx=0).stop 2024-04-23T02:21:37.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.378+0000 7fccce8c0700 1 -- 172.21.15.71:0/3116491959 shutdown_connections 2024-04-23T02:21:37.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.378+0000 7fccce8c0700 1 --2- 172.21.15.71:0/3116491959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccc80fff50 0x7fccc8105840 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.378+0000 7fccce8c0700 1 --2- 172.21.15.71:0/3116491959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccc80ff5a0 0x7fccc80ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.378+0000 7fccce8c0700 1 -- 172.21.15.71:0/3116491959 >> 172.21.15.71:0/3116491959 conn(0x7fccc8074e30 msgr2=0x7fccc8075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:37.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.378+0000 7fccce8c0700 1 -- 172.21.15.71:0/3116491959 shutdown_connections 2024-04-23T02:21:37.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.378+0000 7fccce8c0700 1 -- 172.21.15.71:0/3116491959 wait complete. 2024-04-23T02:21:37.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.378+0000 7fccce8c0700 1 Processor -- start 2024-04-23T02:21:37.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.378+0000 7fccce8c0700 1 -- start start 2024-04-23T02:21:37.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.379+0000 7fccce8c0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccc80ff5a0 0x7fccc8100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:37.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.379+0000 7fccce8c0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccc80fff50 0x7fccc8100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:37.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.379+0000 7fccce8c0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc8100fa0 con 0x7fccc80fff50 2024-04-23T02:21:37.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.379+0000 7fccce8c0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc81010e0 con 0x7fccc80ff5a0 2024-04-23T02:21:37.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.379+0000 7fccc77fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccc80fff50 0x7fccc8100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:37.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.379+0000 7fccc77fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccc80fff50 0x7fccc8100a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57206/0 (socket says 172.21.15.71:57206) 2024-04-23T02:21:37.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.379+0000 7fccc77fe700 1 -- 172.21.15.71:0/3717640835 learned_addr learned my addr 172.21.15.71:0/3717640835 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:21:37.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.379+0000 7fccc7fff700 1 --2- 172.21.15.71:0/3717640835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccc80ff5a0 0x7fccc8100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:37.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.379+0000 7fccc77fe700 1 -- 172.21.15.71:0/3717640835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccc80ff5a0 msgr2=0x7fccc8100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:37.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.379+0000 7fccc77fe700 1 --2- 172.21.15.71:0/3717640835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccc80ff5a0 0x7fccc8100520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.379+0000 7fccc77fe700 1 -- 172.21.15.71:0/3717640835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb80095f0 con 0x7fccc80fff50 2024-04-23T02:21:37.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.379+0000 7fccc77fe700 1 --2- 172.21.15.71:0/3717640835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccc80fff50 0x7fccc8100a60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fccb80092e0 tx=0x7fccb800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:37.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.379+0000 7fccc57fa700 1 -- 172.21.15.71:0/3717640835 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccb801d070 con 0x7fccc80fff50 2024-04-23T02:21:37.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.379+0000 7fccc57fa700 1 -- 172.21.15.71:0/3717640835 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fccb800f460 con 0x7fccc80fff50 2024-04-23T02:21:37.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.379+0000 7fccce8c0700 1 -- 172.21.15.71:0/3717640835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc8101360 con 0x7fccc80fff50 2024-04-23T02:21:37.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.379+0000 7fccc57fa700 1 -- 172.21.15.71:0/3717640835 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccb8021620 con 0x7fccc80fff50 2024-04-23T02:21:37.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.379+0000 7fccce8c0700 1 -- 172.21.15.71:0/3717640835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc81a2d90 con 0x7fccc80fff50 2024-04-23T02:21:37.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.380+0000 7fccce8c0700 1 -- 172.21.15.71:0/3717640835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc8109220 con 0x7fccc80fff50 2024-04-23T02:21:37.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.381+0000 7fccc57fa700 1 -- 172.21.15.71:0/3717640835 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fccb8003a60 con 0x7fccc80fff50 2024-04-23T02:21:37.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.381+0000 7fccc57fa700 1 --2- 172.21.15.71:0/3717640835 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fccb406cce0 0x7fccb406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:37.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.381+0000 7fccc57fa700 1 -- 172.21.15.71:0/3717640835 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fccb808d610 con 0x7fccc80fff50 2024-04-23T02:21:37.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.381+0000 7fccc7fff700 1 --2- 172.21.15.71:0/3717640835 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fccb406cce0 0x7fccb406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:37.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.382+0000 7fccc7fff700 1 --2- 172.21.15.71:0/3717640835 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fccb406cce0 0x7fccb406f1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009a10 tx=0x7fccb0009190 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:37.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.386+0000 7fccc57fa700 1 -- 172.21.15.71:0/3717640835 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fccb80577b0 con 0x7fccc80fff50 2024-04-23T02:21:37.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.585+0000 7fccce8c0700 1 -- 172.21.15.71:0/3717640835 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fccc80611d0 con 0x7fccb406cce0 2024-04-23T02:21:37.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.595+0000 7fccc57fa700 1 -- 172.21.15.71:0/3717640835 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fccc80611d0 con 0x7fccb406cce0 2024-04-23T02:21:37.599 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:21:37.599 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (5m) 86s ago 16m 26.3M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:21:37.599 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (16m) 86s ago 16m 8250k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:21:37.599 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (15m) 18s ago 15m 8254k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:21:37.599 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (3m) 86s ago 16m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:21:37.599 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (3m) 18s ago 15m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:21:37.599 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (5m) 86s ago 16m 80.0M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:21:37.599 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (10m) 86s ago 10m 64.9M - 18.2.1 5be31c24972a a515de2a841c 2024-04-23T02:21:37.600 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (10m) 86s ago 10m 133M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:21:37.600 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (10m) 18s ago 10m 68.2M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:21:37.600 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (10m) 18s ago 10m 122M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:21:37.600 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (7m) 86s ago 18m 507M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:21:37.600 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (6m) 18s ago 15m 453M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:21:37.600 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (3m) 86s ago 18m 47.1M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:21:37.600 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (3m) 18s ago 15m 41.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:21:37.600 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (6m) 86s ago 16m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:21:37.600 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (6m) 18s ago 15m 17.7M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:21:37.600 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (2m) 86s ago 14m 85.3M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:21:37.601 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (2m) 86s ago 14m 85.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:21:37.601 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (92s) 86s ago 13m 10.9M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:21:37.601 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (56s) 18s ago 13m 100M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:21:37.601 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (22s) 18s ago 12m 12.5M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:21:37.601 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (12m) 18s ago 12m 198M 2251M 18.2.1 5be31c24972a a5a427d25ab6 2024-04-23T02:21:37.601 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (5m) 86s ago 16m 65.6M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:21:37.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.597+0000 7fccce8c0700 1 -- 172.21.15.71:0/3717640835 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fccb406cce0 msgr2=0x7fccb406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:37.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.597+0000 7fccce8c0700 1 --2- 172.21.15.71:0/3717640835 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fccb406cce0 0x7fccb406f1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009a10 tx=0x7fccb0009190 comp rx=0 tx=0).stop 2024-04-23T02:21:37.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.597+0000 7fccce8c0700 1 -- 172.21.15.71:0/3717640835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccc80fff50 msgr2=0x7fccc8100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:37.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.597+0000 7fccce8c0700 1 --2- 172.21.15.71:0/3717640835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccc80fff50 0x7fccc8100a60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fccb80092e0 tx=0x7fccb800bd70 comp rx=0 tx=0).stop 2024-04-23T02:21:37.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.597+0000 7fccce8c0700 1 -- 172.21.15.71:0/3717640835 shutdown_connections 2024-04-23T02:21:37.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.597+0000 7fccce8c0700 1 --2- 172.21.15.71:0/3717640835 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fccb406cce0 0x7fccb406f1a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.597+0000 7fccce8c0700 1 --2- 172.21.15.71:0/3717640835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccc80ff5a0 0x7fccc8100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.597+0000 7fccce8c0700 1 --2- 172.21.15.71:0/3717640835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccc80fff50 0x7fccc8100a60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.597+0000 7fccce8c0700 1 -- 172.21.15.71:0/3717640835 >> 172.21.15.71:0/3717640835 conn(0x7fccc8074e30 msgr2=0x7fccc80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:37.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.598+0000 7fccce8c0700 1 -- 172.21.15.71:0/3717640835 shutdown_connections 2024-04-23T02:21:37.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.598+0000 7fccce8c0700 1 -- 172.21.15.71:0/3717640835 wait complete. 2024-04-23T02:21:37.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.699+0000 7f58cfb79700 1 -- 172.21.15.71:0/768894548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58c8105bf0 msgr2=0x7f58c8105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:37.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.699+0000 7f58cfb79700 1 --2- 172.21.15.71:0/768894548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58c8105bf0 0x7f58c8105fd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009910 tx=0x7f58b8009c20 comp rx=0 tx=0).stop 2024-04-23T02:21:37.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.699+0000 7f58cfb79700 1 -- 172.21.15.71:0/768894548 shutdown_connections 2024-04-23T02:21:37.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.699+0000 7f58cfb79700 1 --2- 172.21.15.71:0/768894548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58c80ffb40 0x7f58c80fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.699+0000 7f58cfb79700 1 --2- 172.21.15.71:0/768894548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58c8105bf0 0x7f58c8105fd0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.699+0000 7f58cfb79700 1 -- 172.21.15.71:0/768894548 >> 172.21.15.71:0/768894548 conn(0x7f58c80fb610 msgr2=0x7f58c80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:37.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.699+0000 7f58cfb79700 1 -- 172.21.15.71:0/768894548 shutdown_connections 2024-04-23T02:21:37.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.699+0000 7f58cfb79700 1 -- 172.21.15.71:0/768894548 wait complete. 2024-04-23T02:21:37.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.700+0000 7f58cfb79700 1 Processor -- start 2024-04-23T02:21:37.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.700+0000 7f58cfb79700 1 -- start start 2024-04-23T02:21:37.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.700+0000 7f58cfb79700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58c80ffb40 0x7f58c8197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:37.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.700+0000 7f58cfb79700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58c8105bf0 0x7f58c8197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:37.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.700+0000 7f58cfb79700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c81980f0 con 0x7f58c8105bf0 2024-04-23T02:21:37.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.700+0000 7f58cfb79700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c8198260 con 0x7f58c80ffb40 2024-04-23T02:21:37.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.700+0000 7f58cd114700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58c8105bf0 0x7f58c8197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.700+0000 7f58cd114700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58c8105bf0 0x7f58c8197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57226/0 (socket says 172.21.15.71:57226) 2024-04-23T02:21:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.700+0000 7f58cd114700 1 -- 172.21.15.71:0/256605440 learned_addr learned my addr 172.21.15.71:0/256605440 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:21:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.700+0000 7f58cd915700 1 --2- 172.21.15.71:0/256605440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58c80ffb40 0x7f58c8197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.700+0000 7f58cd114700 1 -- 172.21.15.71:0/256605440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58c80ffb40 msgr2=0x7f58c8197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.700+0000 7f58cd114700 1 --2- 172.21.15.71:0/256605440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58c80ffb40 0x7f58c8197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.700+0000 7f58cd114700 1 -- 172.21.15.71:0/256605440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b80095f0 con 0x7f58c8105bf0 2024-04-23T02:21:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.700+0000 7f58cd114700 1 --2- 172.21.15.71:0/256605440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58c8105bf0 0x7f58c8197ad0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f58c400c8a0 tx=0x7f58c400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.700+0000 7f58beffd700 1 -- 172.21.15.71:0/256605440 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58c40078e0 con 0x7f58c8105bf0 2024-04-23T02:21:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.700+0000 7f58beffd700 1 -- 172.21.15.71:0/256605440 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f58c400f450 con 0x7f58c8105bf0 2024-04-23T02:21:37.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.700+0000 7f58cfb79700 1 -- 172.21.15.71:0/256605440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c81a0600 con 0x7f58c8105bf0 2024-04-23T02:21:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.701+0000 7f58beffd700 1 -- 172.21.15.71:0/256605440 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58c400e5e0 con 0x7f58c8105bf0 2024-04-23T02:21:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.701+0000 7f58cfb79700 1 -- 172.21.15.71:0/256605440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c81a0af0 con 0x7f58c8105bf0 2024-04-23T02:21:37.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.701+0000 7f58cfb79700 1 -- 172.21.15.71:0/256605440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c8108020 con 0x7f58c8105bf0 2024-04-23T02:21:37.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.702+0000 7f58beffd700 1 -- 172.21.15.71:0/256605440 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f58c400f5c0 con 0x7f58c8105bf0 2024-04-23T02:21:37.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.702+0000 7f58beffd700 1 --2- 172.21.15.71:0/256605440 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f58b406cce0 0x7f58b406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:37.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.702+0000 7f58beffd700 1 -- 172.21.15.71:0/256605440 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f58c408c120 con 0x7f58c8105bf0 2024-04-23T02:21:37.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.702+0000 7f58cd915700 1 --2- 172.21.15.71:0/256605440 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f58b406cce0 0x7f58b406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:37.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.703+0000 7f58cd915700 1 --2- 172.21.15.71:0/256605440 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f58b406cce0 0x7f58b406f1a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f58b8006010 tx=0x7f58b801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:37.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.707+0000 7f58beffd700 1 -- 172.21.15.71:0/256605440 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f58c4056240 con 0x7f58c8105bf0 2024-04-23T02:21:37.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.983+0000 7f58cfb79700 1 -- 172.21.15.71:0/256605440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f58c804f2e0 con 0x7f58c8105bf0 2024-04-23T02:21:37.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.983+0000 7f58beffd700 1 -- 172.21.15.71:0/256605440 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7f58c4059860 con 0x7f58c8105bf0 2024-04-23T02:21:37.987 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:21:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:21:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:21:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:21:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:21:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:21:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:21:37.987 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:21:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:21:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 5 2024-04-23T02:21:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:21:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:21:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-23T02:21:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:21:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:21:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-04-23T02:21:37.988 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 9 2024-04-23T02:21:37.989 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:21:37.989 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:21:37.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.985+0000 7f58cfb79700 1 -- 172.21.15.71:0/256605440 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f58b406cce0 msgr2=0x7f58b406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:37.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.985+0000 7f58cfb79700 1 --2- 172.21.15.71:0/256605440 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f58b406cce0 0x7f58b406f1a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f58b8006010 tx=0x7f58b801a040 comp rx=0 tx=0).stop 2024-04-23T02:21:37.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.985+0000 7f58cfb79700 1 -- 172.21.15.71:0/256605440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58c8105bf0 msgr2=0x7f58c8197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:37.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.985+0000 7f58cfb79700 1 --2- 172.21.15.71:0/256605440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58c8105bf0 0x7f58c8197ad0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f58c400c8a0 tx=0x7f58c400cbb0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.986+0000 7f58cfb79700 1 -- 172.21.15.71:0/256605440 shutdown_connections 2024-04-23T02:21:37.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.986+0000 7f58cfb79700 1 --2- 172.21.15.71:0/256605440 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f58b406cce0 0x7f58b406f1a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.986+0000 7f58cfb79700 1 --2- 172.21.15.71:0/256605440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58c80ffb40 0x7f58c8197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.986+0000 7f58cfb79700 1 --2- 172.21.15.71:0/256605440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58c8105bf0 0x7f58c8197ad0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:37.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.986+0000 7f58cfb79700 1 -- 172.21.15.71:0/256605440 >> 172.21.15.71:0/256605440 conn(0x7f58c80fb610 msgr2=0x7f58c80fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:37.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.986+0000 7f58cfb79700 1 -- 172.21.15.71:0/256605440 shutdown_connections 2024-04-23T02:21:37.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:37.986+0000 7f58cfb79700 1 -- 172.21.15.71:0/256605440 wait complete. 2024-04-23T02:21:38.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.088+0000 7f9364420700 1 -- 172.21.15.71:0/3056282655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f935c0ff5a0 msgr2=0x7f935c0ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:38.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.088+0000 7f9364420700 1 --2- 172.21.15.71:0/3056282655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f935c0ff5a0 0x7f935c0ff980 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f934c009910 tx=0x7f934c009c20 comp rx=0 tx=0).stop 2024-04-23T02:21:38.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.088+0000 7f9364420700 1 -- 172.21.15.71:0/3056282655 shutdown_connections 2024-04-23T02:21:38.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.088+0000 7f9364420700 1 --2- 172.21.15.71:0/3056282655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f935c0fff50 0x7f935c105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:38.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.088+0000 7f9364420700 1 --2- 172.21.15.71:0/3056282655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f935c0ff5a0 0x7f935c0ff980 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:38.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.088+0000 7f9364420700 1 -- 172.21.15.71:0/3056282655 >> 172.21.15.71:0/3056282655 conn(0x7f935c074e30 msgr2=0x7f935c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:38.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.088+0000 7f9364420700 1 -- 172.21.15.71:0/3056282655 shutdown_connections 2024-04-23T02:21:38.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.088+0000 7f9364420700 1 -- 172.21.15.71:0/3056282655 wait complete. 2024-04-23T02:21:38.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.088+0000 7f9364420700 1 Processor -- start 2024-04-23T02:21:38.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.088+0000 7f9364420700 1 -- start start 2024-04-23T02:21:38.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.089+0000 7f9364420700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f935c0ff5a0 0x7f935c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:38.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.089+0000 7f9364420700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f935c0fff50 0x7f935c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:38.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.089+0000 7f9364420700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c19d130 con 0x7f935c0ff5a0 2024-04-23T02:21:38.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.089+0000 7f9364420700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c1965e0 con 0x7f935c0fff50 2024-04-23T02:21:38.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.089+0000 7f93621bc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f935c0ff5a0 0x7f935c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:38.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.089+0000 7f93621bc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f935c0ff5a0 0x7f935c19c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57234/0 (socket says 172.21.15.71:57234) 2024-04-23T02:21:38.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.089+0000 7f93621bc700 1 -- 172.21.15.71:0/1539175943 learned_addr learned my addr 172.21.15.71:0/1539175943 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:21:38.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.089+0000 7f93621bc700 1 -- 172.21.15.71:0/1539175943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f935c0fff50 msgr2=0x7f935c19caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:38.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.089+0000 7f93619bb700 1 --2- 172.21.15.71:0/1539175943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f935c0fff50 0x7f935c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:38.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.089+0000 7f93621bc700 1 --2- 172.21.15.71:0/1539175943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f935c0fff50 0x7f935c19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:38.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.089+0000 7f93621bc700 1 -- 172.21.15.71:0/1539175943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c0095f0 con 0x7f935c0ff5a0 2024-04-23T02:21:38.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.089+0000 7f93621bc700 1 --2- 172.21.15.71:0/1539175943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f935c0ff5a0 0x7f935c19c560 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f934c006010 tx=0x7f934c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:38.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.089+0000 7f93537fe700 1 -- 172.21.15.71:0/1539175943 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f934c01d070 con 0x7f935c0ff5a0 2024-04-23T02:21:38.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.089+0000 7f93537fe700 1 -- 172.21.15.71:0/1539175943 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f934c00f460 con 0x7f935c0ff5a0 2024-04-23T02:21:38.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.089+0000 7f9364420700 1 -- 172.21.15.71:0/1539175943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935c196860 con 0x7f935c0ff5a0 2024-04-23T02:21:38.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.089+0000 7f93537fe700 1 -- 172.21.15.71:0/1539175943 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f934c021640 con 0x7f935c0ff5a0 2024-04-23T02:21:38.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.089+0000 7f9364420700 1 -- 172.21.15.71:0/1539175943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935c196d50 con 0x7f935c0ff5a0 2024-04-23T02:21:38.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.090+0000 7f9364420700 1 -- 172.21.15.71:0/1539175943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935c109270 con 0x7f935c0ff5a0 2024-04-23T02:21:38.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.091+0000 7f93537fe700 1 -- 172.21.15.71:0/1539175943 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f934c00f9e0 con 0x7f935c0ff5a0 2024-04-23T02:21:38.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.091+0000 7f93537fe700 1 --2- 172.21.15.71:0/1539175943 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f934806cce0 0x7f934806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:38.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.091+0000 7f93537fe700 1 -- 172.21.15.71:0/1539175943 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f934c08d780 con 0x7f935c0ff5a0 2024-04-23T02:21:38.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.091+0000 7f93619bb700 1 --2- 172.21.15.71:0/1539175943 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f934806cce0 0x7f934806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:38.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.091+0000 7f93619bb700 1 --2- 172.21.15.71:0/1539175943 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f934806cce0 0x7f934806f1a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f935c107770 tx=0x7f9358005cf0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:38.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.096+0000 7f93537fe700 1 -- 172.21.15.71:0/1539175943 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f934c05b330 con 0x7f935c0ff5a0 2024-04-23T02:21:38.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:37 smithi192 ceph-mon[108617]: pgmap v239: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-04-23T02:21:38.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:37 smithi192 ceph-mon[108617]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:21:38.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:37 smithi192 ceph-mon[108617]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:21:38.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:37 smithi071 ceph-mon[144544]: pgmap v239: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-04-23T02:21:38.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:37 smithi071 ceph-mon[144544]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:21:38.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:37 smithi071 ceph-mon[144544]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:21:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.331+0000 7f9364420700 1 -- 172.21.15.71:0/1539175943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f935c197030 con 0x7f935c0ff5a0 2024-04-23T02:21:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.331+0000 7f93537fe700 1 -- 172.21.15.71:0/1539175943 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1869 (secure 0 0 0) 0x7f934c057020 con 0x7f935c0ff5a0 2024-04-23T02:21:38.335 INFO:teuthology.orchestra.run.smithi071.stdout:e25 2024-04-23T02:21:38.335 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:21:38.335 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:21:38.335 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:21:38.335 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:21:38.335 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:21:38.335 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:21:38.335 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 25 2024-04-23T02:21:38.335 INFO:teuthology.orchestra.run.smithi071.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T02:21:38.335 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:21:38.335 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:17:27.814232+0000 2024-04-23T02:21:38.335 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:21:38.335 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:21:38.336 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:21:38.336 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:21:38.336 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:21:38.336 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:21:38.336 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:21:38.336 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:21:38.336 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 46 2024-04-23T02:21:38.336 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:21:38.336 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:21:38.336 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:21:38.336 INFO:teuthology.orchestra.run.smithi071.stdout:up {0=14478,1=24243} 2024-04-23T02:21:38.337 INFO:teuthology.orchestra.run.smithi071.stdout:failed 2024-04-23T02:21:38.337 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:21:38.337 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:21:38.337 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:21:38.337 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:21:38.337 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:21:38.337 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:21:38.337 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:21:38.337 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:21:38.337 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.71:6828/467545394,v1:172.21.15.71:6829/467545394] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:21:38.337 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{0:14490} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/3442634960,v1:172.21.15.71:6827/3442634960] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:21:38.337 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{1:24243} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.192:6824/3780549896,v1:172.21.15.192:6825/3780549896] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:21:38.337 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{1:24255} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.192:6826/2100078284,v1:172.21.15.192:6827/2100078284] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:21:38.337 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:21:38.337 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:21:38.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.333+0000 7f9364420700 1 -- 172.21.15.71:0/1539175943 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f934806cce0 msgr2=0x7f934806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:38.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.333+0000 7f9364420700 1 --2- 172.21.15.71:0/1539175943 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f934806cce0 0x7f934806f1a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f935c107770 tx=0x7f9358005cf0 comp rx=0 tx=0).stop 2024-04-23T02:21:38.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.333+0000 7f9364420700 1 -- 172.21.15.71:0/1539175943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f935c0ff5a0 msgr2=0x7f935c19c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:38.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.333+0000 7f9364420700 1 --2- 172.21.15.71:0/1539175943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f935c0ff5a0 0x7f935c19c560 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f934c006010 tx=0x7f934c00be20 comp rx=0 tx=0).stop 2024-04-23T02:21:38.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.333+0000 7f9364420700 1 -- 172.21.15.71:0/1539175943 shutdown_connections 2024-04-23T02:21:38.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.333+0000 7f9364420700 1 --2- 172.21.15.71:0/1539175943 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f934806cce0 0x7f934806f1a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:38.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.333+0000 7f9364420700 1 --2- 172.21.15.71:0/1539175943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f935c0ff5a0 0x7f935c19c560 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:38.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.333+0000 7f9364420700 1 --2- 172.21.15.71:0/1539175943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f935c0fff50 0x7f935c19caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:38.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.333+0000 7f9364420700 1 -- 172.21.15.71:0/1539175943 >> 172.21.15.71:0/1539175943 conn(0x7f935c074e30 msgr2=0x7f935c0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:38.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.333+0000 7f9364420700 1 -- 172.21.15.71:0/1539175943 shutdown_connections 2024-04-23T02:21:38.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.333+0000 7f9364420700 1 -- 172.21.15.71:0/1539175943 wait complete. 2024-04-23T02:21:38.339 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 25 2024-04-23T02:21:38.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.437+0000 7f72b03d1700 1 -- 172.21.15.71:0/4243309592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a8101840 msgr2=0x7f72a8105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:38.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.437+0000 7f72b03d1700 1 --2- 172.21.15.71:0/4243309592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a8101840 0x7f72a8105890 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7298009910 tx=0x7f7298009c20 comp rx=0 tx=0).stop 2024-04-23T02:21:38.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.437+0000 7f72b03d1700 1 -- 172.21.15.71:0/4243309592 shutdown_connections 2024-04-23T02:21:38.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.438+0000 7f72b03d1700 1 --2- 172.21.15.71:0/4243309592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a8101840 0x7f72a8105890 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:38.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.438+0000 7f72b03d1700 1 --2- 172.21.15.71:0/4243309592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72a8100e90 0x7f72a8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:38.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.438+0000 7f72b03d1700 1 -- 172.21.15.71:0/4243309592 >> 172.21.15.71:0/4243309592 conn(0x7f72a80fc740 msgr2=0x7f72a80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:38.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.438+0000 7f72b03d1700 1 -- 172.21.15.71:0/4243309592 shutdown_connections 2024-04-23T02:21:38.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.438+0000 7f72b03d1700 1 -- 172.21.15.71:0/4243309592 wait complete. 2024-04-23T02:21:38.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.438+0000 7f72b03d1700 1 Processor -- start 2024-04-23T02:21:38.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.438+0000 7f72b03d1700 1 -- start start 2024-04-23T02:21:38.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.438+0000 7f72b03d1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a8100e90 0x7f72a819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:38.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.438+0000 7f72b03d1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72a8101840 0x7f72a819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:38.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.438+0000 7f72b03d1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a819d200 con 0x7f72a8100e90 2024-04-23T02:21:38.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.438+0000 7f72b03d1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a81966b0 con 0x7f72a8101840 2024-04-23T02:21:38.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.438+0000 7f72ae16d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a8100e90 0x7f72a819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:38.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.439+0000 7f72ae16d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a8100e90 0x7f72a819c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57258/0 (socket says 172.21.15.71:57258) 2024-04-23T02:21:38.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.439+0000 7f72ae16d700 1 -- 172.21.15.71:0/889087654 learned_addr learned my addr 172.21.15.71:0/889087654 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:21:38.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.439+0000 7f72ad96c700 1 --2- 172.21.15.71:0/889087654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72a8101840 0x7f72a819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:38.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.439+0000 7f72ae16d700 1 -- 172.21.15.71:0/889087654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72a8101840 msgr2=0x7f72a819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:38.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.439+0000 7f72ae16d700 1 --2- 172.21.15.71:0/889087654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72a8101840 0x7f72a819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:38.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.439+0000 7f72ae16d700 1 -- 172.21.15.71:0/889087654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72980095f0 con 0x7f72a8100e90 2024-04-23T02:21:38.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.439+0000 7f72ae16d700 1 --2- 172.21.15.71:0/889087654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a8100e90 0x7f72a819c630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f72a400c8a0 tx=0x7f72a400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:38.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.439+0000 7f729f7fe700 1 -- 172.21.15.71:0/889087654 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72a400cea0 con 0x7f72a8100e90 2024-04-23T02:21:38.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.439+0000 7f729f7fe700 1 -- 172.21.15.71:0/889087654 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f72a4004830 con 0x7f72a8100e90 2024-04-23T02:21:38.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.439+0000 7f72b03d1700 1 -- 172.21.15.71:0/889087654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a8196990 con 0x7f72a8100e90 2024-04-23T02:21:38.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.439+0000 7f729f7fe700 1 -- 172.21.15.71:0/889087654 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72a4005500 con 0x7f72a8100e90 2024-04-23T02:21:38.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.439+0000 7f72b03d1700 1 -- 172.21.15.71:0/889087654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a8196ee0 con 0x7f72a8100e90 2024-04-23T02:21:38.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.440+0000 7f72b03d1700 1 -- 172.21.15.71:0/889087654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a8109290 con 0x7f72a8100e90 2024-04-23T02:21:38.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.441+0000 7f729f7fe700 1 -- 172.21.15.71:0/889087654 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f72a40049a0 con 0x7f72a8100e90 2024-04-23T02:21:38.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.441+0000 7f729f7fe700 1 --2- 172.21.15.71:0/889087654 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f729406cce0 0x7f729406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:38.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.441+0000 7f729f7fe700 1 -- 172.21.15.71:0/889087654 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f72a408c200 con 0x7f72a8100e90 2024-04-23T02:21:38.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.441+0000 7f72ad96c700 1 --2- 172.21.15.71:0/889087654 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f729406cce0 0x7f729406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:38.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.441+0000 7f72ad96c700 1 --2- 172.21.15.71:0/889087654 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f729406cce0 0x7f729406f1a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f72a80fcfb0 tx=0x7f729801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:38.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.445+0000 7f729f7fe700 1 -- 172.21.15.71:0/889087654 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f72a4059db0 con 0x7f72a8100e90 2024-04-23T02:21:38.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.655+0000 7f72b03d1700 1 -- 172.21.15.71:0/889087654 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f72a8197cf0 con 0x7f729406cce0 2024-04-23T02:21:38.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.656+0000 7f729f7fe700 1 -- 172.21.15.71:0/889087654 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f72a8197cf0 con 0x7f729406cce0 2024-04-23T02:21:38.659 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:21:38.659 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:21:38.659 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:21:38.659 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:21:38.660 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:21:38.660 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:21:38.660 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:21:38.660 INFO:teuthology.orchestra.run.smithi071.stdout: "mon" 2024-04-23T02:21:38.660 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:21:38.660 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "11/23 daemons upgraded", 2024-04-23T02:21:38.660 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading osd daemons", 2024-04-23T02:21:38.660 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:21:38.660 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:21:38.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.658+0000 7f72b03d1700 1 -- 172.21.15.71:0/889087654 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f729406cce0 msgr2=0x7f729406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:38.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.658+0000 7f72b03d1700 1 --2- 172.21.15.71:0/889087654 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f729406cce0 0x7f729406f1a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f72a80fcfb0 tx=0x7f729801a040 comp rx=0 tx=0).stop 2024-04-23T02:21:38.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.658+0000 7f72b03d1700 1 -- 172.21.15.71:0/889087654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a8100e90 msgr2=0x7f72a819c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:38.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.658+0000 7f72b03d1700 1 --2- 172.21.15.71:0/889087654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a8100e90 0x7f72a819c630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f72a400c8a0 tx=0x7f72a400cc60 comp rx=0 tx=0).stop 2024-04-23T02:21:38.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.658+0000 7f72b03d1700 1 -- 172.21.15.71:0/889087654 shutdown_connections 2024-04-23T02:21:38.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.658+0000 7f72b03d1700 1 --2- 172.21.15.71:0/889087654 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f729406cce0 0x7f729406f1a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:38.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.658+0000 7f72b03d1700 1 --2- 172.21.15.71:0/889087654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a8100e90 0x7f72a819c630 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:38.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.658+0000 7f72b03d1700 1 --2- 172.21.15.71:0/889087654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72a8101840 0x7f72a819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:38.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.658+0000 7f72b03d1700 1 -- 172.21.15.71:0/889087654 >> 172.21.15.71:0/889087654 conn(0x7f72a80fc740 msgr2=0x7f72a80fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:38.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.658+0000 7f72b03d1700 1 -- 172.21.15.71:0/889087654 shutdown_connections 2024-04-23T02:21:38.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.658+0000 7f72b03d1700 1 -- 172.21.15.71:0/889087654 wait complete. 2024-04-23T02:21:38.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.761+0000 7ff7d4324700 1 -- 172.21.15.71:0/3211755951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7cc105bd0 msgr2=0x7ff7cc105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:38.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.762+0000 7ff7d4324700 1 --2- 172.21.15.71:0/3211755951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7cc105bd0 0x7ff7cc105fb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc009910 tx=0x7ff7bc009c20 comp rx=0 tx=0).stop 2024-04-23T02:21:38.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.762+0000 7ff7d4324700 1 -- 172.21.15.71:0/3211755951 shutdown_connections 2024-04-23T02:21:38.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.762+0000 7ff7d4324700 1 --2- 172.21.15.71:0/3211755951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7cc0ffc20 0x7ff7cc100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:38.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.762+0000 7ff7d4324700 1 --2- 172.21.15.71:0/3211755951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7cc105bd0 0x7ff7cc105fb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:38.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.762+0000 7ff7d4324700 1 -- 172.21.15.71:0/3211755951 >> 172.21.15.71:0/3211755951 conn(0x7ff7cc075670 msgr2=0x7ff7cc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:38.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.762+0000 7ff7d4324700 1 -- 172.21.15.71:0/3211755951 shutdown_connections 2024-04-23T02:21:38.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.762+0000 7ff7d4324700 1 -- 172.21.15.71:0/3211755951 wait complete. 2024-04-23T02:21:38.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.762+0000 7ff7d4324700 1 Processor -- start 2024-04-23T02:21:38.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.762+0000 7ff7d4324700 1 -- start start 2024-04-23T02:21:38.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.763+0000 7ff7d4324700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7cc0ffc20 0x7ff7cc19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:38.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.763+0000 7ff7d4324700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7cc105bd0 0x7ff7cc19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:38.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.763+0000 7ff7d4324700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc19d150 con 0x7ff7cc105bd0 2024-04-23T02:21:38.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.763+0000 7ff7d4324700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc196600 con 0x7ff7cc0ffc20 2024-04-23T02:21:38.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.763+0000 7ff7d18bf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7cc105bd0 0x7ff7cc19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:38.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.763+0000 7ff7d18bf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7cc105bd0 0x7ff7cc19cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57278/0 (socket says 172.21.15.71:57278) 2024-04-23T02:21:38.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.763+0000 7ff7d18bf700 1 -- 172.21.15.71:0/1837197624 learned_addr learned my addr 172.21.15.71:0/1837197624 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:21:38.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.763+0000 7ff7d20c0700 1 --2- 172.21.15.71:0/1837197624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7cc0ffc20 0x7ff7cc19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:38.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.763+0000 7ff7d18bf700 1 -- 172.21.15.71:0/1837197624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7cc0ffc20 msgr2=0x7ff7cc19c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:38.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.763+0000 7ff7d18bf700 1 --2- 172.21.15.71:0/1837197624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7cc0ffc20 0x7ff7cc19c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:38.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.763+0000 7ff7d18bf700 1 -- 172.21.15.71:0/1837197624 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc0095f0 con 0x7ff7cc105bd0 2024-04-23T02:21:38.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.763+0000 7ff7d18bf700 1 --2- 172.21.15.71:0/1837197624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7cc105bd0 0x7ff7cc19cac0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800d8d0 tx=0x7ff7c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:38.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.763+0000 7ff7c37fe700 1 -- 172.21.15.71:0/1837197624 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7c8009940 con 0x7ff7cc105bd0 2024-04-23T02:21:38.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.763+0000 7ff7c37fe700 1 -- 172.21.15.71:0/1837197624 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff7c8010460 con 0x7ff7cc105bd0 2024-04-23T02:21:38.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.763+0000 7ff7c37fe700 1 -- 172.21.15.71:0/1837197624 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7c800f5f0 con 0x7ff7cc105bd0 2024-04-23T02:21:38.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.763+0000 7ff7d4324700 1 -- 172.21.15.71:0/1837197624 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7cc1968e0 con 0x7ff7cc105bd0 2024-04-23T02:21:38.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.763+0000 7ff7d4324700 1 -- 172.21.15.71:0/1837197624 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7cc196e30 con 0x7ff7cc105bd0 2024-04-23T02:21:38.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.764+0000 7ff7d4324700 1 -- 172.21.15.71:0/1837197624 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7cc04f2e0 con 0x7ff7cc105bd0 2024-04-23T02:21:38.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.765+0000 7ff7c37fe700 1 -- 172.21.15.71:0/1837197624 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff7c80105d0 con 0x7ff7cc105bd0 2024-04-23T02:21:38.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.765+0000 7ff7c37fe700 1 --2- 172.21.15.71:0/1837197624 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff7b806cce0 0x7ff7b806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:21:38.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.765+0000 7ff7c37fe700 1 -- 172.21.15.71:0/1837197624 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff7c808c5d0 con 0x7ff7cc105bd0 2024-04-23T02:21:38.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.765+0000 7ff7d20c0700 1 --2- 172.21.15.71:0/1837197624 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff7b806cce0 0x7ff7b806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:21:38.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.766+0000 7ff7d20c0700 1 --2- 172.21.15.71:0/1837197624 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff7b806cce0 0x7ff7b806f1a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc006010 tx=0x7ff7bc01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:21:38.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:38.770+0000 7ff7c37fe700 1 -- 172.21.15.71:0/1837197624 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff7c805a0e0 con 0x7ff7cc105bd0 2024-04-23T02:21:39.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:39.046+0000 7ff7d4324700 1 -- 172.21.15.71:0/1837197624 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff7cc197100 con 0x7ff7cc105bd0 2024-04-23T02:21:39.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:39.046+0000 7ff7c37fe700 1 -- 172.21.15.71:0/1837197624 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7ff7c8010880 con 0x7ff7cc105bd0 2024-04-23T02:21:39.049 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-23T02:21:39.050 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:21:39.050 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:21:39.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:39.048+0000 7ff7d4324700 1 -- 172.21.15.71:0/1837197624 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff7b806cce0 msgr2=0x7ff7b806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:39.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:39.048+0000 7ff7d4324700 1 --2- 172.21.15.71:0/1837197624 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff7b806cce0 0x7ff7b806f1a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc006010 tx=0x7ff7bc01a040 comp rx=0 tx=0).stop 2024-04-23T02:21:39.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:39.048+0000 7ff7d4324700 1 -- 172.21.15.71:0/1837197624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7cc105bd0 msgr2=0x7ff7cc19cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:21:39.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:39.048+0000 7ff7d4324700 1 --2- 172.21.15.71:0/1837197624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7cc105bd0 0x7ff7cc19cac0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800d8d0 tx=0x7ff7c800dc90 comp rx=0 tx=0).stop 2024-04-23T02:21:39.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:39.048+0000 7ff7d4324700 1 -- 172.21.15.71:0/1837197624 shutdown_connections 2024-04-23T02:21:39.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:39.048+0000 7ff7d4324700 1 --2- 172.21.15.71:0/1837197624 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff7b806cce0 0x7ff7b806f1a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:39.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:39.048+0000 7ff7d4324700 1 --2- 172.21.15.71:0/1837197624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7cc0ffc20 0x7ff7cc19c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:39.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:39.048+0000 7ff7d4324700 1 --2- 172.21.15.71:0/1837197624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7cc105bd0 0x7ff7cc19cac0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:21:39.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:39.048+0000 7ff7d4324700 1 -- 172.21.15.71:0/1837197624 >> 172.21.15.71:0/1837197624 conn(0x7ff7cc075670 msgr2=0x7ff7cc068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:21:39.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:39.048+0000 7ff7d4324700 1 -- 172.21.15.71:0/1837197624 shutdown_connections 2024-04-23T02:21:39.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:21:39.048+0000 7ff7d4324700 1 -- 172.21.15.71:0/1837197624 wait complete. 2024-04-23T02:21:39.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:38 smithi192 ceph-mon[108617]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:21:39.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:38 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/256605440' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:21:39.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:38 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1539175943' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:21:39.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:38 smithi071 ceph-mon[144544]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:21:39.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:38 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/256605440' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:21:39.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:38 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1539175943' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:21:40.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:39 smithi192 ceph-mon[108617]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:21:40.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:39 smithi192 ceph-mon[108617]: pgmap v240: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-23T02:21:40.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:39 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1837197624' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:21:40.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:39 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-23T02:21:40.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:39 smithi071 ceph-mon[144544]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:21:40.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:39 smithi071 ceph-mon[144544]: pgmap v240: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-23T02:21:40.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:39 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1837197624' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:21:40.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:39 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-23T02:21:41.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:40 smithi192 ceph-mon[108617]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-23T02:21:41.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:40 smithi192 ceph-mon[108617]: Upgrade: osd.5 is safe to restart 2024-04-23T02:21:41.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:40 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:41.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:40 smithi071 ceph-mon[144544]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-23T02:21:41.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:40 smithi071 ceph-mon[144544]: Upgrade: osd.5 is safe to restart 2024-04-23T02:21:41.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:40 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:42.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:41 smithi071 ceph-mon[144544]: pgmap v241: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:21:42.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:41 smithi071 ceph-mon[144544]: Upgrade: Updating osd.5 2024-04-23T02:21:42.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:41 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:42.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:41 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T02:21:42.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:41 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:21:42.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:41 smithi071 ceph-mon[144544]: Deploying daemon osd.5 on smithi192 2024-04-23T02:21:42.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:41 smithi192 ceph-mon[108617]: pgmap v241: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:21:42.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:41 smithi192 ceph-mon[108617]: Upgrade: Updating osd.5 2024-04-23T02:21:42.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:41 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:42.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:41 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T02:21:42.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:41 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:21:42.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:41 smithi192 ceph-mon[108617]: Deploying daemon osd.5 on smithi192 2024-04-23T02:21:44.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:44 smithi192 ceph-mon[108617]: pgmap v242: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-04-23T02:21:44.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:44 smithi071 ceph-mon[144544]: pgmap v242: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-04-23T02:21:45.126 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:44 smithi192 systemd[1]: Stopping Ceph osd.5 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T02:21:45.126 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:44 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[73690]: 2024-04-23T02:21:44.910+0000 7f081229a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T02:21:45.127 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:44 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[73690]: 2024-04-23T02:21:44.910+0000 7f081229a700 -1 osd.5 73 *** Got signal Terminated *** 2024-04-23T02:21:45.127 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:44 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[73690]: 2024-04-23T02:21:44.910+0000 7f081229a700 -1 osd.5 73 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T02:21:45.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:45 smithi192 ceph-mon[108617]: osd.5 marked itself down and dead 2024-04-23T02:21:45.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:45 smithi071 ceph-mon[144544]: osd.5 marked itself down and dead 2024-04-23T02:21:46.312 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:46 smithi192 bash[132365]: ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5 2024-04-23T02:21:46.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:46 smithi192 ceph-mon[108617]: pgmap v243: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:21:46.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:46 smithi192 ceph-mon[108617]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-23T02:21:46.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:46 smithi192 ceph-mon[108617]: osdmap e74: 6 total, 5 up, 6 in 2024-04-23T02:21:46.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:46 smithi071 ceph-mon[144544]: pgmap v243: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:21:46.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:46 smithi071 ceph-mon[144544]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-23T02:21:46.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:46 smithi071 ceph-mon[144544]: osdmap e74: 6 total, 5 up, 6 in 2024-04-23T02:21:47.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:47 smithi192 ceph-mon[108617]: osdmap e75: 6 total, 5 up, 6 in 2024-04-23T02:21:47.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:47 smithi071 ceph-mon[144544]: osdmap e75: 6 total, 5 up, 6 in 2024-04-23T02:21:48.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:48 smithi192 ceph-mon[108617]: pgmap v246: 65 pgs: 5 peering, 8 stale+active+clean, 3 active+undersized, 49 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:21:48.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:48 smithi192 ceph-mon[108617]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-23T02:21:48.627 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:48 smithi192 systemd[1]: ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.5.service: Succeeded. 2024-04-23T02:21:48.627 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:48 smithi192 systemd[1]: Stopped Ceph osd.5 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:21:48.627 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:48 smithi192 systemd[1]: Starting Ceph osd.5 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T02:21:48.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:48 smithi071 ceph-mon[144544]: pgmap v246: 65 pgs: 5 peering, 8 stale+active+clean, 3 active+undersized, 49 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:21:48.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:48 smithi071 ceph-mon[144544]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-23T02:21:49.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:49 smithi192 ceph-mon[108617]: Health check failed: Degraded data redundancy: 40/366 objects degraded (10.929%), 9 pgs degraded (PG_DEGRADED) 2024-04-23T02:21:49.626 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:49 smithi192 bash[132641]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-23T02:21:49.626 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:49 smithi192 bash[132641]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-23T02:21:49.626 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:49 smithi192 bash[132641]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-23T02:21:49.626 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:49 smithi192 bash[132641]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-23T02:21:49.626 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:49 smithi192 bash[132641]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-23T02:21:49.627 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:49 smithi192 bash[132641]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-23T02:21:49.627 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:49 smithi192 bash[132641]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-23T02:21:49.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:49 smithi071 ceph-mon[144544]: Health check failed: Degraded data redundancy: 40/366 objects degraded (10.929%), 9 pgs degraded (PG_DEGRADED) 2024-04-23T02:21:50.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:50 smithi071 ceph-mon[144544]: pgmap v247: 65 pgs: 9 active+undersized+degraded, 5 peering, 1 stale+active+clean, 11 active+undersized, 39 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 40/366 objects degraded (10.929%) 2024-04-23T02:21:50.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:50.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:21:50.750 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:50 smithi192 ceph-mon[108617]: pgmap v247: 65 pgs: 9 active+undersized+degraded, 5 peering, 1 stale+active+clean, 11 active+undersized, 39 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 40/366 objects degraded (10.929%) 2024-04-23T02:21:50.750 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:50.750 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:21:51.020 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:50 smithi192 ceph-osd[132960]: -- 172.21.15.192:0/3576847319 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c2f496b1e0 con 0x55c2f493b000 2024-04-23T02:21:51.376 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:51 smithi192 bash[132922]: 16449d7414855babc5ce314e1cff691b54eaea5b5e5d65cbe66764c8871db485 2024-04-23T02:21:51.376 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:51 smithi192 systemd[1]: Started Ceph osd.5 for 91f5470c-0115-11ef-bc93-c7b262605968. 2024-04-23T02:21:51.854 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:51 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T02:21:51.466+0000 7f0fe50d27c0 -1 Falling back to public interface 2024-04-23T02:21:52.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:52 smithi192 ceph-mon[108617]: pgmap v248: 65 pgs: 11 active+undersized+degraded, 5 peering, 12 active+undersized, 37 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 47/366 objects degraded (12.842%) 2024-04-23T02:21:52.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:52 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:52.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:52 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:52.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:52 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:21:52.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:52 smithi071 ceph-mon[144544]: pgmap v248: 65 pgs: 11 active+undersized+degraded, 5 peering, 12 active+undersized, 37 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 47/366 objects degraded (12.842%) 2024-04-23T02:21:52.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:52 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:52.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:52 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:52.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:52 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:21:53.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:53 smithi192 ceph-mon[108617]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-23T02:21:53.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:53 smithi071 ceph-mon[144544]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-23T02:21:54.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:54 smithi192 ceph-mon[108617]: pgmap v249: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/366 objects degraded (15.301%) 2024-04-23T02:21:54.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:54 smithi071 ceph-mon[144544]: pgmap v249: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/366 objects degraded (15.301%) 2024-04-23T02:21:55.603 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:55 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:55.603 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:55 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:55.603 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:55 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:55.604 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:55 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:55.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:55 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:55.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:55 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:55.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:55 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:55.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:55 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:55.862 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:55 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T02:21:55.633+0000 7f0fe50d27c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-04-23T02:21:56.126 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:55 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T02:21:55.930+0000 7f0fe50d27c0 -1 osd.5 73 log_to_monitors true 2024-04-23T02:21:56.626 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 02:21:56 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T02:21:56.373+0000 7f0fd652c700 -1 osd.5 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T02:21:56.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:56 smithi192 ceph-mon[108617]: pgmap v250: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/366 objects degraded (15.301%) 2024-04-23T02:21:56.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:56 smithi192 ceph-mon[108617]: from='osd.5 [v2:172.21.15.192:6816/868066904,v1:172.21.15.192:6817/868066904]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T02:21:56.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:56 smithi192 ceph-mon[108617]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T02:21:56.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:56 smithi071 ceph-mon[144544]: pgmap v250: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/366 objects degraded (15.301%) 2024-04-23T02:21:56.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:56 smithi071 ceph-mon[144544]: from='osd.5 [v2:172.21.15.192:6816/868066904,v1:172.21.15.192:6817/868066904]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T02:21:56.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:56 smithi071 ceph-mon[144544]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T02:21:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:57 smithi192 ceph-mon[108617]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T02:21:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:57 smithi192 ceph-mon[108617]: osdmap e76: 6 total, 5 up, 6 in 2024-04-23T02:21:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:57 smithi192 ceph-mon[108617]: from='osd.5 [v2:172.21.15.192:6816/868066904,v1:172.21.15.192:6817/868066904]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:21:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:57 smithi192 ceph-mon[108617]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:21:57.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:57 smithi192 ceph-mon[108617]: Health check update: Degraded data redundancy: 56/366 objects degraded (15.301%), 14 pgs degraded (PG_DEGRADED) 2024-04-23T02:21:57.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:57 smithi071 ceph-mon[144544]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T02:21:57.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:57 smithi071 ceph-mon[144544]: osdmap e76: 6 total, 5 up, 6 in 2024-04-23T02:21:57.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:57 smithi071 ceph-mon[144544]: from='osd.5 [v2:172.21.15.192:6816/868066904,v1:172.21.15.192:6817/868066904]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:21:57.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:57 smithi071 ceph-mon[144544]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-04-23T02:21:57.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:57 smithi071 ceph-mon[144544]: Health check update: Degraded data redundancy: 56/366 objects degraded (15.301%), 14 pgs degraded (PG_DEGRADED) 2024-04-23T02:21:58.513 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:58 smithi192 ceph-mon[108617]: pgmap v252: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 56/366 objects degraded (15.301%) 2024-04-23T02:21:58.513 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:58 smithi192 ceph-mon[108617]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-23T02:21:58.513 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:58 smithi192 ceph-mon[108617]: osd.5 [v2:172.21.15.192:6816/868066904,v1:172.21.15.192:6817/868066904] boot 2024-04-23T02:21:58.513 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:58 smithi192 ceph-mon[108617]: osdmap e77: 6 total, 6 up, 6 in 2024-04-23T02:21:58.513 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:58 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T02:21:58.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:58 smithi071 ceph-mon[144544]: pgmap v252: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 56/366 objects degraded (15.301%) 2024-04-23T02:21:58.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:58 smithi071 ceph-mon[144544]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-23T02:21:58.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:58 smithi071 ceph-mon[144544]: osd.5 [v2:172.21.15.192:6816/868066904,v1:172.21.15.192:6817/868066904] boot 2024-04-23T02:21:58.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:58 smithi071 ceph-mon[144544]: osdmap e77: 6 total, 6 up, 6 in 2024-04-23T02:21:58.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:58 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T02:21:59.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: osdmap e78: 6 total, 6 up, 6 in 2024-04-23T02:21:59.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:21:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:21:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:21:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:21:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:21:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:21:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:21:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-23T02:21:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-23T02:21:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-23T02:21:59.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-23T02:21:59.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-23T02:21:59.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-23T02:21:59.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-23T02:21:59.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-23T02:21:59.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-23T02:21:59.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-23T02:21:59.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-23T02:21:59.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-23T02:21:59.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:59.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-04-23T02:21:59.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:21:59 smithi192 ceph-mon[108617]: osdmap e79: 6 total, 6 up, 6 in 2024-04-23T02:21:59.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: osdmap e78: 6 total, 6 up, 6 in 2024-04-23T02:21:59.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:21:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:21:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:21:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:21:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:21:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:21:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:21:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-23T02:21:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-23T02:21:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-23T02:21:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-23T02:21:59.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-23T02:21:59.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-23T02:21:59.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-23T02:21:59.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-23T02:21:59.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-23T02:21:59.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-23T02:21:59.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-23T02:21:59.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-23T02:21:59.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:21:59.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-04-23T02:21:59.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:21:59 smithi071 ceph-mon[144544]: osdmap e79: 6 total, 6 up, 6 in 2024-04-23T02:22:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:22:00.279+0000 7fc177969700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T02:22:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: pgmap v255: 65 pgs: 7 peering, 9 active+undersized+degraded, 12 active+undersized, 37 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s; 28/366 objects degraded (7.650%) 2024-04-23T02:22:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: Upgrade: Setting container_image for all osd 2024-04-23T02:22:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-04-23T02:22:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-23T02:22:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-04-23T02:22:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: fsmap cephfs:2 {0=cephfs.smithi071.vxejye=up:active,1=cephfs.smithi192.gtmyst=up:active} 2024-04-23T02:22:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:22:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:22:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:22:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:22:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:00.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:00.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T02:22:00.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T02:22:00.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T02:22:00.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-23T02:22:00.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: osdmap e80: 6 total, 6 up, 6 in 2024-04-23T02:22:00.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: mds.? [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] up:boot 2024-04-23T02:22:00.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: mds.? [v2:172.21.15.71:6826/1456175422,v1:172.21.15.71:6827/1456175422] up:boot 2024-04-23T02:22:00.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T02:22:00.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-04-23T02:22:00.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.qusrrb"}]: dispatch 2024-04-23T02:22:00.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:00 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.azdyze"}]: dispatch 2024-04-23T02:22:00.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: pgmap v255: 65 pgs: 7 peering, 9 active+undersized+degraded, 12 active+undersized, 37 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s; 28/366 objects degraded (7.650%) 2024-04-23T02:22:00.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: Upgrade: Setting container_image for all osd 2024-04-23T02:22:00.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-04-23T02:22:00.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:00.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-23T02:22:00.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-04-23T02:22:00.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: fsmap cephfs:2 {0=cephfs.smithi071.vxejye=up:active,1=cephfs.smithi192.gtmyst=up:active} 2024-04-23T02:22:00.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:22:00.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:22:00.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:22:00.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:00.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:22:00.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:00.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:00.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:00.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:00.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T02:22:00.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T02:22:00.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T02:22:00.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-23T02:22:00.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: osdmap e80: 6 total, 6 up, 6 in 2024-04-23T02:22:00.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: mds.? [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] up:boot 2024-04-23T02:22:00.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: mds.? [v2:172.21.15.71:6826/1456175422,v1:172.21.15.71:6827/1456175422] up:boot 2024-04-23T02:22:00.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T02:22:00.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-04-23T02:22:00.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.qusrrb"}]: dispatch 2024-04-23T02:22:00.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:00 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.azdyze"}]: dispatch 2024-04-23T02:22:01.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:01 smithi071 ceph-mon[144544]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-04-23T02:22:01.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:01 smithi071 ceph-mon[144544]: mds.? [v2:172.21.15.192:6824/1096782156,v1:172.21.15.192:6825/1096782156] up:boot 2024-04-23T02:22:01.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:01 smithi071 ceph-mon[144544]: mds.? [v2:172.21.15.71:6828/3220156840,v1:172.21.15.71:6829/3220156840] up:boot 2024-04-23T02:22:01.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:01 smithi071 ceph-mon[144544]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-23T02:22:01.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:01 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.gtmyst"}]: dispatch 2024-04-23T02:22:01.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:01 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.vxejye"}]: dispatch 2024-04-23T02:22:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:01 smithi192 ceph-mon[108617]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-04-23T02:22:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:01 smithi192 ceph-mon[108617]: mds.? [v2:172.21.15.192:6824/1096782156,v1:172.21.15.192:6825/1096782156] up:boot 2024-04-23T02:22:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:01 smithi192 ceph-mon[108617]: mds.? [v2:172.21.15.71:6828/3220156840,v1:172.21.15.71:6829/3220156840] up:boot 2024-04-23T02:22:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:01 smithi192 ceph-mon[108617]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-23T02:22:01.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:01 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.gtmyst"}]: dispatch 2024-04-23T02:22:01.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:01 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.vxejye"}]: dispatch 2024-04-23T02:22:02.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:02 smithi192 ceph-mon[108617]: pgmap v258: 65 pgs: 7 peering, 8 active+undersized+degraded, 11 active+undersized, 39 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 0 op/s; 24/366 objects degraded (6.557%) 2024-04-23T02:22:02.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:02 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:02.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:02 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi071.qusrrb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T02:22:02.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:02 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:22:02.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:02 smithi192 ceph-mon[108617]: Health check update: Degraded data redundancy: 24/366 objects degraded (6.557%), 8 pgs degraded (PG_DEGRADED) 2024-04-23T02:22:02.633 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:02 smithi071 ceph-mon[144544]: pgmap v258: 65 pgs: 7 peering, 8 active+undersized+degraded, 11 active+undersized, 39 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 0 op/s; 24/366 objects degraded (6.557%) 2024-04-23T02:22:02.633 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:02 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:02.634 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:02 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi071.qusrrb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T02:22:02.634 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:02 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:22:02.634 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:02 smithi071 ceph-mon[144544]: Health check update: Degraded data redundancy: 24/366 objects degraded (6.557%), 8 pgs degraded (PG_DEGRADED) 2024-04-23T02:22:03.606 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:03 smithi071 ceph-mon[144544]: Upgrade: Updating mds.cephfs.smithi071.qusrrb 2024-04-23T02:22:03.606 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:03 smithi071 ceph-mon[144544]: Deploying daemon mds.cephfs.smithi071.qusrrb on smithi071 2024-04-23T02:22:03.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:03 smithi192 ceph-mon[108617]: Upgrade: Updating mds.cephfs.smithi071.qusrrb 2024-04-23T02:22:03.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:03 smithi192 ceph-mon[108617]: Deploying daemon mds.cephfs.smithi071.qusrrb on smithi071 2024-04-23T02:22:04.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:04 smithi071 ceph-mon[144544]: pgmap v259: 65 pgs: 7 peering, 58 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-23T02:22:04.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:04 smithi071 ceph-mon[144544]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/366 objects degraded (6.557%), 8 pgs degraded) 2024-04-23T02:22:04.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:04 smithi071 ceph-mon[144544]: osdmap e81: 6 total, 6 up, 6 in 2024-04-23T02:22:04.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:04 smithi192 ceph-mon[108617]: pgmap v259: 65 pgs: 7 peering, 58 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-23T02:22:04.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:04 smithi192 ceph-mon[108617]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/366 objects degraded (6.557%), 8 pgs degraded) 2024-04-23T02:22:04.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:04 smithi192 ceph-mon[108617]: osdmap e81: 6 total, 6 up, 6 in 2024-04-23T02:22:05.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:05 smithi071 ceph-mon[144544]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-23T02:22:05.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:05.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:22:05.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:05 smithi071 ceph-mon[144544]: pgmap v261: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-23T02:22:05.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.qusrrb"}]: dispatch 2024-04-23T02:22:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:05 smithi192 ceph-mon[108617]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-23T02:22:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:22:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:05 smithi192 ceph-mon[108617]: pgmap v261: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-23T02:22:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.qusrrb"}]: dispatch 2024-04-23T02:22:08.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:07 smithi192 ceph-mon[108617]: pgmap v262: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-23T02:22:08.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:07 smithi071 ceph-mon[144544]: pgmap v262: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-23T02:22:09.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.157+0000 7f56659f9700 1 -- 172.21.15.71:0/648425358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56600ffda0 msgr2=0x7f5660100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:09.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.157+0000 7f56659f9700 1 --2- 172.21.15.71:0/648425358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56600ffda0 0x7f5660100180 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5654009910 tx=0x7f5654009c20 comp rx=0 tx=0).stop 2024-04-23T02:22:09.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.157+0000 7f56659f9700 1 -- 172.21.15.71:0/648425358 shutdown_connections 2024-04-23T02:22:09.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.157+0000 7f56659f9700 1 --2- 172.21.15.71:0/648425358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5660100750 0x7f56601047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:09.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.157+0000 7f56659f9700 1 --2- 172.21.15.71:0/648425358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56600ffda0 0x7f5660100180 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:09.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.157+0000 7f56659f9700 1 -- 172.21.15.71:0/648425358 >> 172.21.15.71:0/648425358 conn(0x7f56600fb650 msgr2=0x7f56600fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:09.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.157+0000 7f56659f9700 1 -- 172.21.15.71:0/648425358 shutdown_connections 2024-04-23T02:22:09.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.157+0000 7f56659f9700 1 -- 172.21.15.71:0/648425358 wait complete. 2024-04-23T02:22:09.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.158+0000 7f56659f9700 1 Processor -- start 2024-04-23T02:22:09.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.158+0000 7f56659f9700 1 -- start start 2024-04-23T02:22:09.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.159+0000 7f56659f9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56600ffda0 0x7f5660101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:09.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.159+0000 7f56659f9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5660100750 0x7f5660101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:09.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.159+0000 7f56659f9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56601022c0 con 0x7f56600ffda0 2024-04-23T02:22:09.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.159+0000 7f56659f9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5660102400 con 0x7f5660100750 2024-04-23T02:22:09.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.159+0000 7f565effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56600ffda0 0x7f5660101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:09.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.159+0000 7f565effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56600ffda0 0x7f5660101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40568/0 (socket says 172.21.15.71:40568) 2024-04-23T02:22:09.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.159+0000 7f565effd700 1 -- 172.21.15.71:0/3855623162 learned_addr learned my addr 172.21.15.71:0/3855623162 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:22:09.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.159+0000 7f565e7fc700 1 --2- 172.21.15.71:0/3855623162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5660100750 0x7f5660101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:09.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.159+0000 7f565effd700 1 -- 172.21.15.71:0/3855623162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5660100750 msgr2=0x7f5660101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:09.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.159+0000 7f565effd700 1 --2- 172.21.15.71:0/3855623162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5660100750 0x7f5660101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:09.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.159+0000 7f565effd700 1 -- 172.21.15.71:0/3855623162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56540095f0 con 0x7f56600ffda0 2024-04-23T02:22:09.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.159+0000 7f565effd700 1 --2- 172.21.15.71:0/3855623162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56600ffda0 0x7f5660101840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5654006010 tx=0x7f565400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:09.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.159+0000 7f56649f7700 1 -- 172.21.15.71:0/3855623162 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f565401d070 con 0x7f56600ffda0 2024-04-23T02:22:09.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.159+0000 7f56649f7700 1 -- 172.21.15.71:0/3855623162 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f565400f460 con 0x7f56600ffda0 2024-04-23T02:22:09.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.159+0000 7f56659f9700 1 -- 172.21.15.71:0/3855623162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5660102680 con 0x7f56600ffda0 2024-04-23T02:22:09.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.159+0000 7f56649f7700 1 -- 172.21.15.71:0/3855623162 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5654021620 con 0x7f56600ffda0 2024-04-23T02:22:09.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.159+0000 7f56659f9700 1 -- 172.21.15.71:0/3855623162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56601a2de0 con 0x7f56600ffda0 2024-04-23T02:22:09.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.160+0000 7f56659f9700 1 -- 172.21.15.71:0/3855623162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f566004f2e0 con 0x7f56600ffda0 2024-04-23T02:22:09.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.161+0000 7f56649f7700 1 -- 172.21.15.71:0/3855623162 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f565400f5d0 con 0x7f56600ffda0 2024-04-23T02:22:09.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.161+0000 7f56649f7700 1 --2- 172.21.15.71:0/3855623162 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f564806cce0 0x7f564806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:09.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.161+0000 7f56649f7700 1 -- 172.21.15.71:0/3855623162 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f565408d850 con 0x7f56600ffda0 2024-04-23T02:22:09.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.161+0000 7f565e7fc700 1 --2- 172.21.15.71:0/3855623162 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f564806cce0 0x7f564806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:09.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.162+0000 7f565e7fc700 1 --2- 172.21.15.71:0/3855623162 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f564806cce0 0x7f564806f1a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5660103030 tx=0x7f5650009190 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:09.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.166+0000 7f56649f7700 1 -- 172.21.15.71:0/3855623162 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f56540578c0 con 0x7f56600ffda0 2024-04-23T02:22:09.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.385+0000 7f56659f9700 1 -- 172.21.15.71:0/3855623162 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f56600fc560 con 0x7f564806cce0 2024-04-23T02:22:09.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.386+0000 7f56649f7700 1 -- 172.21.15.71:0/3855623162 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f56600fc560 con 0x7f564806cce0 2024-04-23T02:22:09.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.388+0000 7f56659f9700 1 -- 172.21.15.71:0/3855623162 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f564806cce0 msgr2=0x7f564806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:09.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.388+0000 7f56659f9700 1 --2- 172.21.15.71:0/3855623162 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f564806cce0 0x7f564806f1a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5660103030 tx=0x7f5650009190 comp rx=0 tx=0).stop 2024-04-23T02:22:09.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.388+0000 7f56659f9700 1 -- 172.21.15.71:0/3855623162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56600ffda0 msgr2=0x7f5660101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:09.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.388+0000 7f56659f9700 1 --2- 172.21.15.71:0/3855623162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56600ffda0 0x7f5660101840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5654006010 tx=0x7f565400bd70 comp rx=0 tx=0).stop 2024-04-23T02:22:09.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.388+0000 7f56659f9700 1 -- 172.21.15.71:0/3855623162 shutdown_connections 2024-04-23T02:22:09.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.388+0000 7f56659f9700 1 --2- 172.21.15.71:0/3855623162 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f564806cce0 0x7f564806f1a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:09.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.388+0000 7f56659f9700 1 --2- 172.21.15.71:0/3855623162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56600ffda0 0x7f5660101840 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:09.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.388+0000 7f56659f9700 1 --2- 172.21.15.71:0/3855623162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5660100750 0x7f5660101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:09.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.388+0000 7f56659f9700 1 -- 172.21.15.71:0/3855623162 >> 172.21.15.71:0/3855623162 conn(0x7f56600fb650 msgr2=0x7f5660104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:09.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.388+0000 7f56659f9700 1 -- 172.21.15.71:0/3855623162 shutdown_connections 2024-04-23T02:22:09.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.388+0000 7f56659f9700 1 -- 172.21.15.71:0/3855623162 wait complete. 2024-04-23T02:22:09.402 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:22:09.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.497+0000 7fae7c18b700 1 -- 172.21.15.71:0/1556786442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae740ffc20 msgr2=0x7fae74100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:09.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.497+0000 7fae7c18b700 1 --2- 172.21.15.71:0/1556786442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae740ffc20 0x7fae74100080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fae70009910 tx=0x7fae70009c20 comp rx=0 tx=0).stop 2024-04-23T02:22:09.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.497+0000 7fae7c18b700 1 -- 172.21.15.71:0/1556786442 shutdown_connections 2024-04-23T02:22:09.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.497+0000 7fae7c18b700 1 --2- 172.21.15.71:0/1556786442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae740ffc20 0x7fae74100080 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:09.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.497+0000 7fae7c18b700 1 --2- 172.21.15.71:0/1556786442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae74105bd0 0x7fae74105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:09.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.497+0000 7fae7c18b700 1 -- 172.21.15.71:0/1556786442 >> 172.21.15.71:0/1556786442 conn(0x7fae74075670 msgr2=0x7fae74075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:09.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.497+0000 7fae7c18b700 1 -- 172.21.15.71:0/1556786442 shutdown_connections 2024-04-23T02:22:09.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.497+0000 7fae7c18b700 1 -- 172.21.15.71:0/1556786442 wait complete. 2024-04-23T02:22:09.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.498+0000 7fae7c18b700 1 Processor -- start 2024-04-23T02:22:09.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.498+0000 7fae7c18b700 1 -- start start 2024-04-23T02:22:09.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.498+0000 7fae7c18b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae740ffc20 0x7fae7419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:09.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.498+0000 7fae7c18b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae74105bd0 0x7fae7419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:09.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.498+0000 7fae7c18b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7419c0f0 con 0x7fae740ffc20 2024-04-23T02:22:09.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.498+0000 7fae7c18b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae741955a0 con 0x7fae74105bd0 2024-04-23T02:22:09.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.498+0000 7fae79f27700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae740ffc20 0x7fae7419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:09.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.498+0000 7fae79f27700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae740ffc20 0x7fae7419b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40610/0 (socket says 172.21.15.71:40610) 2024-04-23T02:22:09.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.498+0000 7fae79f27700 1 -- 172.21.15.71:0/2610673424 learned_addr learned my addr 172.21.15.71:0/2610673424 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:22:09.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.498+0000 7fae79726700 1 --2- 172.21.15.71:0/2610673424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae74105bd0 0x7fae7419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:09.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.498+0000 7fae79f27700 1 -- 172.21.15.71:0/2610673424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae74105bd0 msgr2=0x7fae7419ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:09.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.498+0000 7fae79f27700 1 --2- 172.21.15.71:0/2610673424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae74105bd0 0x7fae7419ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:09.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.499+0000 7fae79f27700 1 -- 172.21.15.71:0/2610673424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae700095f0 con 0x7fae740ffc20 2024-04-23T02:22:09.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.499+0000 7fae79726700 1 --2- 172.21.15.71:0/2610673424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae74105bd0 0x7fae7419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:22:09.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.499+0000 7fae79f27700 1 --2- 172.21.15.71:0/2610673424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae740ffc20 0x7fae7419b520 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fae64007b50 tx=0x7fae64007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:09.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.499+0000 7fae6affd700 1 -- 172.21.15.71:0/2610673424 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae64011070 con 0x7fae740ffc20 2024-04-23T02:22:09.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.499+0000 7fae6affd700 1 -- 172.21.15.71:0/2610673424 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae6400c2e0 con 0x7fae740ffc20 2024-04-23T02:22:09.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.499+0000 7fae7c18b700 1 -- 172.21.15.71:0/2610673424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae74195820 con 0x7fae740ffc20 2024-04-23T02:22:09.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.499+0000 7fae6affd700 1 -- 172.21.15.71:0/2610673424 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae6400e770 con 0x7fae740ffc20 2024-04-23T02:22:09.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.499+0000 7fae7c18b700 1 -- 172.21.15.71:0/2610673424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae74195d70 con 0x7fae740ffc20 2024-04-23T02:22:09.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.500+0000 7fae7c18b700 1 -- 172.21.15.71:0/2610673424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae7404f2e0 con 0x7fae740ffc20 2024-04-23T02:22:09.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.500+0000 7fae6affd700 1 -- 172.21.15.71:0/2610673424 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fae64004a00 con 0x7fae740ffc20 2024-04-23T02:22:09.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.501+0000 7fae6affd700 1 --2- 172.21.15.71:0/2610673424 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fae6006cce0 0x7fae6006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:09.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.501+0000 7fae6affd700 1 -- 172.21.15.71:0/2610673424 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fae6408dcb0 con 0x7fae740ffc20 2024-04-23T02:22:09.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.501+0000 7fae79726700 1 --2- 172.21.15.71:0/2610673424 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fae6006cce0 0x7fae6006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:09.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.501+0000 7fae79726700 1 --2- 172.21.15.71:0/2610673424 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fae6006cce0 0x7fae6006f1a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fae7000b5c0 tx=0x7fae70005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:09.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.505+0000 7fae6affd700 1 -- 172.21.15.71:0/2610673424 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fae6405b7b0 con 0x7fae740ffc20 2024-04-23T02:22:09.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.729+0000 7fae7c18b700 1 -- 172.21.15.71:0/2610673424 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fae74196070 con 0x7fae6006cce0 2024-04-23T02:22:09.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.731+0000 7fae6affd700 1 -- 172.21.15.71:0/2610673424 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fae74196070 con 0x7fae6006cce0 2024-04-23T02:22:09.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.733+0000 7fae7c18b700 1 -- 172.21.15.71:0/2610673424 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fae6006cce0 msgr2=0x7fae6006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:09.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.733+0000 7fae7c18b700 1 --2- 172.21.15.71:0/2610673424 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fae6006cce0 0x7fae6006f1a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fae7000b5c0 tx=0x7fae70005960 comp rx=0 tx=0).stop 2024-04-23T02:22:09.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.733+0000 7fae7c18b700 1 -- 172.21.15.71:0/2610673424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae740ffc20 msgr2=0x7fae7419b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:09.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.733+0000 7fae7c18b700 1 --2- 172.21.15.71:0/2610673424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae740ffc20 0x7fae7419b520 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fae64007b50 tx=0x7fae64007f10 comp rx=0 tx=0).stop 2024-04-23T02:22:09.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.733+0000 7fae7c18b700 1 -- 172.21.15.71:0/2610673424 shutdown_connections 2024-04-23T02:22:09.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.733+0000 7fae7c18b700 1 --2- 172.21.15.71:0/2610673424 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fae6006cce0 0x7fae6006f1a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:09.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.733+0000 7fae7c18b700 1 --2- 172.21.15.71:0/2610673424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae740ffc20 0x7fae7419b520 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:09.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.733+0000 7fae7c18b700 1 --2- 172.21.15.71:0/2610673424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae74105bd0 0x7fae7419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:09.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.733+0000 7fae7c18b700 1 -- 172.21.15.71:0/2610673424 >> 172.21.15.71:0/2610673424 conn(0x7fae74075670 msgr2=0x7fae740fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:09.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.733+0000 7fae7c18b700 1 -- 172.21.15.71:0/2610673424 shutdown_connections 2024-04-23T02:22:09.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.733+0000 7fae7c18b700 1 -- 172.21.15.71:0/2610673424 wait complete. 2024-04-23T02:22:09.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.843+0000 7f5835c21700 1 -- 172.21.15.71:0/890905254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5830100e90 msgr2=0x7f5830101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:09.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.843+0000 7f5835c21700 1 --2- 172.21.15.71:0/890905254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5830100e90 0x7f5830101270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5820009910 tx=0x7f5820009c20 comp rx=0 tx=0).stop 2024-04-23T02:22:09.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.843+0000 7f5835c21700 1 -- 172.21.15.71:0/890905254 shutdown_connections 2024-04-23T02:22:09.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.843+0000 7f5835c21700 1 --2- 172.21.15.71:0/890905254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5830101840 0x7f5830105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:09.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.843+0000 7f5835c21700 1 --2- 172.21.15.71:0/890905254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5830100e90 0x7f5830101270 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:09.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.843+0000 7f5835c21700 1 -- 172.21.15.71:0/890905254 >> 172.21.15.71:0/890905254 conn(0x7f58300fc740 msgr2=0x7f58300feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:09.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.843+0000 7f5835c21700 1 -- 172.21.15.71:0/890905254 shutdown_connections 2024-04-23T02:22:09.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.843+0000 7f5835c21700 1 -- 172.21.15.71:0/890905254 wait complete. 2024-04-23T02:22:09.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.844+0000 7f5835c21700 1 Processor -- start 2024-04-23T02:22:09.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.844+0000 7f5835c21700 1 -- start start 2024-04-23T02:22:09.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.844+0000 7f5835c21700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5830100e90 0x7f5830073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:09.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.844+0000 7f5835c21700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5830101840 0x7f58300735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:09.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.844+0000 7f5835c21700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5830073b10 con 0x7f5830100e90 2024-04-23T02:22:09.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.844+0000 7f5835c21700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5830073c50 con 0x7f5830101840 2024-04-23T02:22:09.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.844+0000 7f582f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5830100e90 0x7f5830073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:09.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.844+0000 7f582f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5830100e90 0x7f5830073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40632/0 (socket says 172.21.15.71:40632) 2024-04-23T02:22:09.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.844+0000 7f582f7fe700 1 -- 172.21.15.71:0/1585514229 learned_addr learned my addr 172.21.15.71:0/1585514229 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:22:09.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.844+0000 7f582effd700 1 --2- 172.21.15.71:0/1585514229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5830101840 0x7f58300735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:09.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.845+0000 7f582f7fe700 1 -- 172.21.15.71:0/1585514229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5830101840 msgr2=0x7f58300735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:09.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.845+0000 7f582f7fe700 1 --2- 172.21.15.71:0/1585514229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5830101840 0x7f58300735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:09.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.845+0000 7f582f7fe700 1 -- 172.21.15.71:0/1585514229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58200095f0 con 0x7f5830100e90 2024-04-23T02:22:09.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.845+0000 7f582f7fe700 1 --2- 172.21.15.71:0/1585514229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5830100e90 0x7f5830073090 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5820000c00 tx=0x7f582000be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:09.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.845+0000 7f582cff9700 1 -- 172.21.15.71:0/1585514229 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f582001d070 con 0x7f5830100e90 2024-04-23T02:22:09.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.845+0000 7f582cff9700 1 -- 172.21.15.71:0/1585514229 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f58200044c0 con 0x7f5830100e90 2024-04-23T02:22:09.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.845+0000 7f5835c21700 1 -- 172.21.15.71:0/1585514229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5830073ed0 con 0x7f5830100e90 2024-04-23T02:22:09.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.845+0000 7f582cff9700 1 -- 172.21.15.71:0/1585514229 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58200039d0 con 0x7f5830100e90 2024-04-23T02:22:09.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.845+0000 7f5835c21700 1 -- 172.21.15.71:0/1585514229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58301a2c90 con 0x7f5830100e90 2024-04-23T02:22:09.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.846+0000 7f5835c21700 1 -- 172.21.15.71:0/1585514229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5830109230 con 0x7f5830100e90 2024-04-23T02:22:09.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.846+0000 7f582cff9700 1 -- 172.21.15.71:0/1585514229 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f58200048e0 con 0x7f5830100e90 2024-04-23T02:22:09.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.847+0000 7f582cff9700 1 --2- 172.21.15.71:0/1585514229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f581806cce0 0x7f581806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:09.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.847+0000 7f582cff9700 1 -- 172.21.15.71:0/1585514229 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f582008db70 con 0x7f5830100e90 2024-04-23T02:22:09.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.847+0000 7f582effd700 1 --2- 172.21.15.71:0/1585514229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f581806cce0 0x7f581806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:09.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.847+0000 7f582effd700 1 --2- 172.21.15.71:0/1585514229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f581806cce0 0x7f581806f1a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5830074880 tx=0x7f5824009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:09.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:09.852+0000 7f582cff9700 1 -- 172.21.15.71:0/1585514229 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5820057c60 con 0x7f5830100e90 2024-04-23T02:22:10.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.102+0000 7f5835c21700 1 -- 172.21.15.71:0/1585514229 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f58300611d0 con 0x7f581806cce0 2024-04-23T02:22:10.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.113+0000 7f582cff9700 1 -- 172.21.15.71:0/1585514229 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f58300611d0 con 0x7f581806cce0 2024-04-23T02:22:10.116 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:22:10.116 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (5m) 119s ago 17m 26.3M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:22:10.116 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (17m) 119s ago 17m 8250k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:22:10.116 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (16m) 15s ago 16m 8321k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:22:10.116 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (3m) 119s ago 17m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:22:10.116 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (3m) 15s ago 16m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:22:10.116 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (5m) 119s ago 16m 80.0M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:22:10.116 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 starting - - - - 2024-04-23T02:22:10.117 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (11m) 119s ago 11m 133M - 18.2.1 5be31c24972a 08554fb804a2 2024-04-23T02:22:10.117 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (11m) 15s ago 11m 68.2M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:22:10.117 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (11m) 15s ago 11m 122M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:22:10.117 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (7m) 119s ago 18m 507M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:22:10.117 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (7m) 15s ago 16m 453M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:22:10.117 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (4m) 119s ago 18m 47.1M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:22:10.117 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (3m) 15s ago 15m 42.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:22:10.117 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (6m) 119s ago 17m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:22:10.117 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (6m) 15s ago 16m 17.9M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:22:10.117 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (3m) 119s ago 15m 85.3M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:22:10.118 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (2m) 119s ago 14m 85.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:22:10.118 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (2m) 119s ago 14m 10.9M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:22:10.118 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (88s) 15s ago 13m 102M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:22:10.118 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (54s) 15s ago 13m 91.2M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:22:10.118 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (19s) 15s ago 12m 11.4M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:22:10.118 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (6m) 119s ago 16m 65.6M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:22:10.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.115+0000 7f5835c21700 1 -- 172.21.15.71:0/1585514229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f581806cce0 msgr2=0x7f581806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:10.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.115+0000 7f5835c21700 1 --2- 172.21.15.71:0/1585514229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f581806cce0 0x7f581806f1a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5830074880 tx=0x7f5824009260 comp rx=0 tx=0).stop 2024-04-23T02:22:10.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.115+0000 7f5835c21700 1 -- 172.21.15.71:0/1585514229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5830100e90 msgr2=0x7f5830073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:10.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.115+0000 7f5835c21700 1 --2- 172.21.15.71:0/1585514229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5830100e90 0x7f5830073090 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5820000c00 tx=0x7f582000be90 comp rx=0 tx=0).stop 2024-04-23T02:22:10.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.115+0000 7f5835c21700 1 -- 172.21.15.71:0/1585514229 shutdown_connections 2024-04-23T02:22:10.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.115+0000 7f5835c21700 1 --2- 172.21.15.71:0/1585514229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f581806cce0 0x7f581806f1a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:10.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.115+0000 7f5835c21700 1 --2- 172.21.15.71:0/1585514229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5830100e90 0x7f5830073090 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:10.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.115+0000 7f5835c21700 1 --2- 172.21.15.71:0/1585514229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5830101840 0x7f58300735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:10.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.115+0000 7f5835c21700 1 -- 172.21.15.71:0/1585514229 >> 172.21.15.71:0/1585514229 conn(0x7f58300fc740 msgr2=0x7f58300fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:10.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.115+0000 7f5835c21700 1 -- 172.21.15.71:0/1585514229 shutdown_connections 2024-04-23T02:22:10.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.115+0000 7f5835c21700 1 -- 172.21.15.71:0/1585514229 wait complete. 2024-04-23T02:22:10.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:09 smithi192 ceph-mon[108617]: pgmap v263: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-23T02:22:10.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:09 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:10.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:09 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:10.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:09 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:22:10.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.219+0000 7f5468738700 1 -- 172.21.15.71:0/2740897279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54600ffda0 msgr2=0x7f5460100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:10.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.219+0000 7f5468738700 1 --2- 172.21.15.71:0/2740897279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54600ffda0 0x7f5460100180 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5450009910 tx=0x7f5450009c20 comp rx=0 tx=0).stop 2024-04-23T02:22:10.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.219+0000 7f5468738700 1 -- 172.21.15.71:0/2740897279 shutdown_connections 2024-04-23T02:22:10.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.219+0000 7f5468738700 1 --2- 172.21.15.71:0/2740897279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5460100750 0x7f54601047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:10.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.219+0000 7f5468738700 1 --2- 172.21.15.71:0/2740897279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54600ffda0 0x7f5460100180 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:10.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.219+0000 7f5468738700 1 -- 172.21.15.71:0/2740897279 >> 172.21.15.71:0/2740897279 conn(0x7f54600fb650 msgr2=0x7f54600fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:10.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.219+0000 7f5468738700 1 -- 172.21.15.71:0/2740897279 shutdown_connections 2024-04-23T02:22:10.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.219+0000 7f5468738700 1 -- 172.21.15.71:0/2740897279 wait complete. 2024-04-23T02:22:10.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.220+0000 7f5468738700 1 Processor -- start 2024-04-23T02:22:10.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.220+0000 7f5468738700 1 -- start start 2024-04-23T02:22:10.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.220+0000 7f5468738700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54600ffda0 0x7f5460197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:10.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.220+0000 7f5468738700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5460100750 0x7f5460197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:10.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.220+0000 7f5468738700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54601982d0 con 0x7f54600ffda0 2024-04-23T02:22:10.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.220+0000 7f5468738700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5460198410 con 0x7f5460100750 2024-04-23T02:22:10.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.220+0000 7f54664d4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54600ffda0 0x7f5460197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:10.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.220+0000 7f54664d4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54600ffda0 0x7f5460197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40654/0 (socket says 172.21.15.71:40654) 2024-04-23T02:22:10.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.220+0000 7f54664d4700 1 -- 172.21.15.71:0/1080016795 learned_addr learned my addr 172.21.15.71:0/1080016795 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:22:10.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.220+0000 7f5465cd3700 1 --2- 172.21.15.71:0/1080016795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5460100750 0x7f5460197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:10.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.220+0000 7f54664d4700 1 -- 172.21.15.71:0/1080016795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5460100750 msgr2=0x7f5460197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:10.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.220+0000 7f54664d4700 1 --2- 172.21.15.71:0/1080016795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5460100750 0x7f5460197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:10.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.220+0000 7f54664d4700 1 -- 172.21.15.71:0/1080016795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54500095f0 con 0x7f54600ffda0 2024-04-23T02:22:10.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.220+0000 7f5465cd3700 1 --2- 172.21.15.71:0/1080016795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5460100750 0x7f5460197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:22:10.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.221+0000 7f54664d4700 1 --2- 172.21.15.71:0/1080016795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54600ffda0 0x7f5460197770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5450006010 tx=0x7f545000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:10.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.221+0000 7f54577fe700 1 -- 172.21.15.71:0/1080016795 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f545001d070 con 0x7f54600ffda0 2024-04-23T02:22:10.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.221+0000 7f54577fe700 1 -- 172.21.15.71:0/1080016795 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f545000f460 con 0x7f54600ffda0 2024-04-23T02:22:10.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.221+0000 7f54577fe700 1 -- 172.21.15.71:0/1080016795 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5450021640 con 0x7f54600ffda0 2024-04-23T02:22:10.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.221+0000 7f5468738700 1 -- 172.21.15.71:0/1080016795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54601a07e0 con 0x7f54600ffda0 2024-04-23T02:22:10.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.221+0000 7f5468738700 1 -- 172.21.15.71:0/1080016795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54601a0bf0 con 0x7f54600ffda0 2024-04-23T02:22:10.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.222+0000 7f5468738700 1 -- 172.21.15.71:0/1080016795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5460108140 con 0x7f54600ffda0 2024-04-23T02:22:10.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.223+0000 7f54577fe700 1 -- 172.21.15.71:0/1080016795 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f545000fa60 con 0x7f54600ffda0 2024-04-23T02:22:10.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.223+0000 7f54577fe700 1 --2- 172.21.15.71:0/1080016795 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f544c06cce0 0x7f544c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:10.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.223+0000 7f54577fe700 1 -- 172.21.15.71:0/1080016795 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f545008dba0 con 0x7f54600ffda0 2024-04-23T02:22:10.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.223+0000 7f5465cd3700 1 --2- 172.21.15.71:0/1080016795 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f544c06cce0 0x7f544c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:10.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.223+0000 7f5465cd3700 1 --2- 172.21.15.71:0/1080016795 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f544c06cce0 0x7f544c06f1a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f54600696c0 tx=0x7f545c008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:10.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.227+0000 7f54577fe700 1 -- 172.21.15.71:0/1080016795 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5450057d30 con 0x7f54600ffda0 2024-04-23T02:22:10.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:09 smithi071 ceph-mon[144544]: pgmap v263: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-23T02:22:10.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:09 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:10.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:09 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:10.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:09 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:22:10.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.511+0000 7f5468738700 1 -- 172.21.15.71:0/1080016795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f546004f2e0 con 0x7f54600ffda0 2024-04-23T02:22:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.512+0000 7f54577fe700 1 -- 172.21.15.71:0/1080016795 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f545005b350 con 0x7f54600ffda0 2024-04-23T02:22:10.515 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:22:10.515 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:22:10.515 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:22:10.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:22:10.515 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:22:10.515 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:22:10.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:22:10.516 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:22:10.516 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:22:10.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:22:10.516 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:22:10.516 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-04-23T02:22:10.516 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 1 2024-04-23T02:22:10.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:22:10.516 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:22:10.516 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-04-23T02:22:10.516 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 11 2024-04-23T02:22:10.516 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:22:10.516 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:22:10.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.514+0000 7f5468738700 1 -- 172.21.15.71:0/1080016795 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f544c06cce0 msgr2=0x7f544c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:10.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.514+0000 7f5468738700 1 --2- 172.21.15.71:0/1080016795 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f544c06cce0 0x7f544c06f1a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f54600696c0 tx=0x7f545c008040 comp rx=0 tx=0).stop 2024-04-23T02:22:10.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.514+0000 7f5468738700 1 -- 172.21.15.71:0/1080016795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54600ffda0 msgr2=0x7f5460197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:10.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.514+0000 7f5468738700 1 --2- 172.21.15.71:0/1080016795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54600ffda0 0x7f5460197770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5450006010 tx=0x7f545000be20 comp rx=0 tx=0).stop 2024-04-23T02:22:10.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.514+0000 7f5468738700 1 -- 172.21.15.71:0/1080016795 shutdown_connections 2024-04-23T02:22:10.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.514+0000 7f5468738700 1 --2- 172.21.15.71:0/1080016795 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f544c06cce0 0x7f544c06f1a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:10.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.514+0000 7f5468738700 1 --2- 172.21.15.71:0/1080016795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54600ffda0 0x7f5460197770 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:10.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.514+0000 7f5468738700 1 --2- 172.21.15.71:0/1080016795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5460100750 0x7f5460197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:10.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.514+0000 7f5468738700 1 -- 172.21.15.71:0/1080016795 >> 172.21.15.71:0/1080016795 conn(0x7f54600fb650 msgr2=0x7f54600fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:10.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.514+0000 7f5468738700 1 -- 172.21.15.71:0/1080016795 shutdown_connections 2024-04-23T02:22:10.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.514+0000 7f5468738700 1 -- 172.21.15.71:0/1080016795 wait complete. 2024-04-23T02:22:10.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.617+0000 7fe06f7d3700 1 -- 172.21.15.71:0/1212001917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0680ffc20 msgr2=0x7fe068100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:10.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.617+0000 7fe06f7d3700 1 --2- 172.21.15.71:0/1212001917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0680ffc20 0x7fe068100080 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe064009910 tx=0x7fe064009c20 comp rx=0 tx=0).stop 2024-04-23T02:22:10.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.617+0000 7fe06f7d3700 1 -- 172.21.15.71:0/1212001917 shutdown_connections 2024-04-23T02:22:10.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.617+0000 7fe06f7d3700 1 --2- 172.21.15.71:0/1212001917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0680ffc20 0x7fe068100080 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:10.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.617+0000 7fe06f7d3700 1 --2- 172.21.15.71:0/1212001917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe068105bd0 0x7fe068105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:10.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.617+0000 7fe06f7d3700 1 -- 172.21.15.71:0/1212001917 >> 172.21.15.71:0/1212001917 conn(0x7fe068075670 msgr2=0x7fe068075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:10.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.617+0000 7fe06f7d3700 1 -- 172.21.15.71:0/1212001917 shutdown_connections 2024-04-23T02:22:10.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.617+0000 7fe06f7d3700 1 -- 172.21.15.71:0/1212001917 wait complete. 2024-04-23T02:22:10.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.618+0000 7fe06f7d3700 1 Processor -- start 2024-04-23T02:22:10.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.618+0000 7fe06f7d3700 1 -- start start 2024-04-23T02:22:10.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.618+0000 7fe06f7d3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0680ffc20 0x7fe06819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:10.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.618+0000 7fe06f7d3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe068105bd0 0x7fe06819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:10.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.618+0000 7fe06f7d3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06819d140 con 0x7fe068105bd0 2024-04-23T02:22:10.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.618+0000 7fe06f7d3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0681965f0 con 0x7fe0680ffc20 2024-04-23T02:22:10.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.618+0000 7fe06cd6e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe068105bd0 0x7fe06819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:10.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.618+0000 7fe06cd6e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe068105bd0 0x7fe06819cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40662/0 (socket says 172.21.15.71:40662) 2024-04-23T02:22:10.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.618+0000 7fe06cd6e700 1 -- 172.21.15.71:0/3998055765 learned_addr learned my addr 172.21.15.71:0/3998055765 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:22:10.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.619+0000 7fe06d56f700 1 --2- 172.21.15.71:0/3998055765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0680ffc20 0x7fe06819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:10.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.619+0000 7fe06cd6e700 1 -- 172.21.15.71:0/3998055765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0680ffc20 msgr2=0x7fe06819c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:10.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.619+0000 7fe06cd6e700 1 --2- 172.21.15.71:0/3998055765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0680ffc20 0x7fe06819c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:10.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.619+0000 7fe06cd6e700 1 -- 172.21.15.71:0/3998055765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0640095f0 con 0x7fe068105bd0 2024-04-23T02:22:10.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.619+0000 7fe06d56f700 1 --2- 172.21.15.71:0/3998055765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0680ffc20 0x7fe06819c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:22:10.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.619+0000 7fe06cd6e700 1 --2- 172.21.15.71:0/3998055765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe068105bd0 0x7fe06819cab0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe064006010 tx=0x7fe06400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:10.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.619+0000 7fe05a7fc700 1 -- 172.21.15.71:0/3998055765 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe06401d070 con 0x7fe068105bd0 2024-04-23T02:22:10.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.619+0000 7fe05a7fc700 1 -- 172.21.15.71:0/3998055765 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe06400f460 con 0x7fe068105bd0 2024-04-23T02:22:10.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.619+0000 7fe06f7d3700 1 -- 172.21.15.71:0/3998055765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe068196870 con 0x7fe068105bd0 2024-04-23T02:22:10.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.619+0000 7fe05a7fc700 1 -- 172.21.15.71:0/3998055765 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe064021640 con 0x7fe068105bd0 2024-04-23T02:22:10.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.619+0000 7fe06f7d3700 1 -- 172.21.15.71:0/3998055765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe068196d60 con 0x7fe068105bd0 2024-04-23T02:22:10.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.621+0000 7fe06f7d3700 1 -- 172.21.15.71:0/3998055765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0681080d0 con 0x7fe068105bd0 2024-04-23T02:22:10.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.621+0000 7fe05a7fc700 1 -- 172.21.15.71:0/3998055765 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe06400fa60 con 0x7fe068105bd0 2024-04-23T02:22:10.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.621+0000 7fe05a7fc700 1 --2- 172.21.15.71:0/3998055765 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe05406cce0 0x7fe05406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:10.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.621+0000 7fe05a7fc700 1 -- 172.21.15.71:0/3998055765 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe06408d7e0 con 0x7fe068105bd0 2024-04-23T02:22:10.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.621+0000 7fe06d56f700 1 --2- 172.21.15.71:0/3998055765 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe05406cce0 0x7fe05406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:10.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.621+0000 7fe06d56f700 1 --2- 172.21.15.71:0/3998055765 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe05406cce0 0x7fe05406f1a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe05c005fd0 tx=0x7fe05c005dc0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:10.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.626+0000 7fe05a7fc700 1 -- 172.21.15.71:0/3998055765 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe06405b360 con 0x7fe068105bd0 2024-04-23T02:22:10.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.913+0000 7fe06f7d3700 1 -- 172.21.15.71:0/3998055765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe068107ab0 con 0x7fe068105bd0 2024-04-23T02:22:10.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.914+0000 7fe05a7fc700 1 -- 172.21.15.71:0/3998055765 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1838 (secure 0 0 0) 0x7fe06405aef0 con 0x7fe068105bd0 2024-04-23T02:22:10.917 INFO:teuthology.orchestra.run.smithi071.stdout:e30 2024-04-23T02:22:10.918 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:22:10.918 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:22:10.918 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:22:10.918 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:22:10.918 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:22:10.918 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:22:10.918 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:22:10.918 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:22:10.918 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:22:10.918 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:22:10.918 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:22:10.918 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:22:10.918 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:22:10.919 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:22:10.919 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:22:10.919 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:22:10.919 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:22:10.919 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:22:10.919 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:22:10.920 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:22:10.920 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:22:10.920 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:22:10.920 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:22:10.920 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:22:10.920 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:22:10.921 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:22:10.921 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:22:10.921 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:22:10.921 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:22:10.921 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:22:10.921 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:22:10.922 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:22:10.922 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:22:10.922 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:22:10.922 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:22:10.922 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:22:10.922 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:22:10.922 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:22:10.923 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44271} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/1096782156,v1:172.21.15.192:6825/1096782156] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:22:10.923 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:44273} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/3220156840,v1:172.21.15.71:6829/3220156840] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:22:10.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.916+0000 7fe06f7d3700 1 -- 172.21.15.71:0/3998055765 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe05406cce0 msgr2=0x7fe05406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:10.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.916+0000 7fe06f7d3700 1 --2- 172.21.15.71:0/3998055765 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe05406cce0 0x7fe05406f1a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe05c005fd0 tx=0x7fe05c005dc0 comp rx=0 tx=0).stop 2024-04-23T02:22:10.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.916+0000 7fe06f7d3700 1 -- 172.21.15.71:0/3998055765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe068105bd0 msgr2=0x7fe06819cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:10.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.916+0000 7fe06f7d3700 1 --2- 172.21.15.71:0/3998055765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe068105bd0 0x7fe06819cab0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe064006010 tx=0x7fe06400be20 comp rx=0 tx=0).stop 2024-04-23T02:22:10.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.916+0000 7fe06f7d3700 1 -- 172.21.15.71:0/3998055765 shutdown_connections 2024-04-23T02:22:10.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.916+0000 7fe06f7d3700 1 --2- 172.21.15.71:0/3998055765 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe05406cce0 0x7fe05406f1a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:10.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.916+0000 7fe06f7d3700 1 --2- 172.21.15.71:0/3998055765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0680ffc20 0x7fe06819c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:10.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.916+0000 7fe06f7d3700 1 --2- 172.21.15.71:0/3998055765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe068105bd0 0x7fe06819cab0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:10.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.916+0000 7fe06f7d3700 1 -- 172.21.15.71:0/3998055765 >> 172.21.15.71:0/3998055765 conn(0x7fe068075670 msgr2=0x7fe0680fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:10.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.916+0000 7fe06f7d3700 1 -- 172.21.15.71:0/3998055765 shutdown_connections 2024-04-23T02:22:10.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:10.916+0000 7fe06f7d3700 1 -- 172.21.15.71:0/3998055765 wait complete. 2024-04-23T02:22:10.926 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 30 2024-04-23T02:22:11.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.021+0000 7f4432c55700 1 -- 172.21.15.71:0/3909890838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f442c105bd0 msgr2=0x7f442c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:11.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.021+0000 7f4432c55700 1 --2- 172.21.15.71:0/3909890838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f442c105bd0 0x7f442c105fb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4420009910 tx=0x7f4420009c20 comp rx=0 tx=0).stop 2024-04-23T02:22:11.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.021+0000 7f4432c55700 1 -- 172.21.15.71:0/3909890838 shutdown_connections 2024-04-23T02:22:11.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.021+0000 7f4432c55700 1 --2- 172.21.15.71:0/3909890838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f442c0ffc20 0x7f442c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:11.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.021+0000 7f4432c55700 1 --2- 172.21.15.71:0/3909890838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f442c105bd0 0x7f442c105fb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:11.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.021+0000 7f4432c55700 1 -- 172.21.15.71:0/3909890838 >> 172.21.15.71:0/3909890838 conn(0x7f442c075670 msgr2=0x7f442c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:11.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.021+0000 7f4432c55700 1 -- 172.21.15.71:0/3909890838 shutdown_connections 2024-04-23T02:22:11.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.021+0000 7f4432c55700 1 -- 172.21.15.71:0/3909890838 wait complete. 2024-04-23T02:22:11.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.022+0000 7f4432c55700 1 Processor -- start 2024-04-23T02:22:11.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.022+0000 7f4432c55700 1 -- start start 2024-04-23T02:22:11.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.022+0000 7f4432c55700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f442c0ffc20 0x7f442c19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:11.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.022+0000 7f4432c55700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f442c105bd0 0x7f442c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:11.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.022+0000 7f4432c55700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442c19d150 con 0x7f442c0ffc20 2024-04-23T02:22:11.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.022+0000 7f4432c55700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442c196600 con 0x7f442c105bd0 2024-04-23T02:22:11.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.022+0000 7f44309f1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f442c0ffc20 0x7f442c19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:11.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.022+0000 7f44309f1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f442c0ffc20 0x7f442c19c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40686/0 (socket says 172.21.15.71:40686) 2024-04-23T02:22:11.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.022+0000 7f44309f1700 1 -- 172.21.15.71:0/750665560 learned_addr learned my addr 172.21.15.71:0/750665560 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:22:11.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.022+0000 7f442bfff700 1 --2- 172.21.15.71:0/750665560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f442c105bd0 0x7f442c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:11.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.022+0000 7f44309f1700 1 -- 172.21.15.71:0/750665560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f442c105bd0 msgr2=0x7f442c19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:11.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.022+0000 7f44309f1700 1 --2- 172.21.15.71:0/750665560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f442c105bd0 0x7f442c19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:11.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.022+0000 7f44309f1700 1 -- 172.21.15.71:0/750665560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44200095f0 con 0x7f442c0ffc20 2024-04-23T02:22:11.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.022+0000 7f44309f1700 1 --2- 172.21.15.71:0/750665560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f442c0ffc20 0x7f442c19c580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f44200092e0 tx=0x7f442000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:11.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.022+0000 7f4429ffb700 1 -- 172.21.15.71:0/750665560 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f442001d070 con 0x7f442c0ffc20 2024-04-23T02:22:11.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.023+0000 7f4429ffb700 1 -- 172.21.15.71:0/750665560 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4420005e00 con 0x7f442c0ffc20 2024-04-23T02:22:11.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.023+0000 7f4432c55700 1 -- 172.21.15.71:0/750665560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f442c196880 con 0x7f442c0ffc20 2024-04-23T02:22:11.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.023+0000 7f4429ffb700 1 -- 172.21.15.71:0/750665560 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f442000fe60 con 0x7f442c0ffc20 2024-04-23T02:22:11.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.023+0000 7f4432c55700 1 -- 172.21.15.71:0/750665560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442c196d70 con 0x7f442c0ffc20 2024-04-23T02:22:11.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.024+0000 7f4432c55700 1 -- 172.21.15.71:0/750665560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f442c04f2e0 con 0x7f442c0ffc20 2024-04-23T02:22:11.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.024+0000 7f4429ffb700 1 -- 172.21.15.71:0/750665560 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f442000f6d0 con 0x7f442c0ffc20 2024-04-23T02:22:11.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.025+0000 7f4429ffb700 1 --2- 172.21.15.71:0/750665560 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f441c06cce0 0x7f441c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:11.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.025+0000 7f442bfff700 1 --2- 172.21.15.71:0/750665560 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f441c06cce0 0x7f441c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:11.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.025+0000 7f4429ffb700 1 -- 172.21.15.71:0/750665560 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f442008d8b0 con 0x7f442c0ffc20 2024-04-23T02:22:11.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.025+0000 7f442bfff700 1 --2- 172.21.15.71:0/750665560 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f441c06cce0 0x7f441c06f1a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4418005950 tx=0x7f44180058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:11.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.029+0000 7f4429ffb700 1 -- 172.21.15.71:0/750665560 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f442005b3b0 con 0x7f442c0ffc20 2024-04-23T02:22:11.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:10 smithi192 ceph-mon[108617]: from='client.34330 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:22:11.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:10 smithi192 ceph-mon[108617]: from='client.34338 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:22:11.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:10 smithi192 ceph-mon[108617]: mds.? [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] up:boot 2024-04-23T02:22:11.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:10 smithi192 ceph-mon[108617]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-23T02:22:11.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:10 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.qusrrb"}]: dispatch 2024-04-23T02:22:11.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:10 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:11.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:10 smithi192 ceph-mon[108617]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:22:11.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:10 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1080016795' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:11.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:10 smithi071 ceph-mon[144544]: from='client.34330 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:22:11.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:10 smithi071 ceph-mon[144544]: from='client.34338 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:22:11.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:10 smithi071 ceph-mon[144544]: mds.? [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] up:boot 2024-04-23T02:22:11.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:10 smithi071 ceph-mon[144544]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-23T02:22:11.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:10 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.qusrrb"}]: dispatch 2024-04-23T02:22:11.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:10 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:11.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:10 smithi071 ceph-mon[144544]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:22:11.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:10 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1080016795' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:11.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.249+0000 7f4432c55700 1 -- 172.21.15.71:0/750665560 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f442c197020 con 0x7f441c06cce0 2024-04-23T02:22:11.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.250+0000 7f4429ffb700 1 -- 172.21.15.71:0/750665560 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f442c197020 con 0x7f441c06cce0 2024-04-23T02:22:11.254 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:22:11.254 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:22:11.254 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:22:11.254 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:22:11.254 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:22:11.254 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:22:11.254 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:22:11.254 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:22:11.254 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:22:11.254 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:22:11.254 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "12/23 daemons upgraded", 2024-04-23T02:22:11.254 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:22:11.254 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:22:11.255 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:22:11.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.252+0000 7f4432c55700 1 -- 172.21.15.71:0/750665560 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f441c06cce0 msgr2=0x7f441c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:11.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.252+0000 7f4432c55700 1 --2- 172.21.15.71:0/750665560 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f441c06cce0 0x7f441c06f1a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4418005950 tx=0x7f44180058e0 comp rx=0 tx=0).stop 2024-04-23T02:22:11.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.253+0000 7f4432c55700 1 -- 172.21.15.71:0/750665560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f442c0ffc20 msgr2=0x7f442c19c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:11.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.253+0000 7f4432c55700 1 --2- 172.21.15.71:0/750665560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f442c0ffc20 0x7f442c19c580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f44200092e0 tx=0x7f442000be20 comp rx=0 tx=0).stop 2024-04-23T02:22:11.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.253+0000 7f4432c55700 1 -- 172.21.15.71:0/750665560 shutdown_connections 2024-04-23T02:22:11.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.253+0000 7f4432c55700 1 --2- 172.21.15.71:0/750665560 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f441c06cce0 0x7f441c06f1a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:11.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.253+0000 7f4432c55700 1 --2- 172.21.15.71:0/750665560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f442c0ffc20 0x7f442c19c580 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:11.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.253+0000 7f4432c55700 1 --2- 172.21.15.71:0/750665560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f442c105bd0 0x7f442c19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:11.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.253+0000 7f4432c55700 1 -- 172.21.15.71:0/750665560 >> 172.21.15.71:0/750665560 conn(0x7f442c075670 msgr2=0x7f442c068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:11.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.253+0000 7f4432c55700 1 -- 172.21.15.71:0/750665560 shutdown_connections 2024-04-23T02:22:11.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.253+0000 7f4432c55700 1 -- 172.21.15.71:0/750665560 wait complete. 2024-04-23T02:22:11.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.360+0000 7f7964e10700 1 -- 172.21.15.71:0/475579269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7960106cd0 msgr2=0x7f79601070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:11.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.360+0000 7f7964e10700 1 --2- 172.21.15.71:0/475579269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7960106cd0 0x7f79601070b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7950009910 tx=0x7f7950009c20 comp rx=0 tx=0).stop 2024-04-23T02:22:11.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.361+0000 7f7964e10700 1 -- 172.21.15.71:0/475579269 shutdown_connections 2024-04-23T02:22:11.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.361+0000 7f7964e10700 1 --2- 172.21.15.71:0/475579269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7960068580 0x7f79600689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:11.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.361+0000 7f7964e10700 1 --2- 172.21.15.71:0/475579269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7960106cd0 0x7f79601070b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:11.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.361+0000 7f7964e10700 1 -- 172.21.15.71:0/475579269 >> 172.21.15.71:0/475579269 conn(0x7f7960075670 msgr2=0x7f7960075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:11.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.361+0000 7f7964e10700 1 -- 172.21.15.71:0/475579269 shutdown_connections 2024-04-23T02:22:11.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.361+0000 7f7964e10700 1 -- 172.21.15.71:0/475579269 wait complete. 2024-04-23T02:22:11.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.361+0000 7f7964e10700 1 Processor -- start 2024-04-23T02:22:11.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.361+0000 7f7964e10700 1 -- start start 2024-04-23T02:22:11.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.362+0000 7f7964e10700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7960068580 0x7f796019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:11.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.362+0000 7f7964e10700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7960106cd0 0x7f796019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:11.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.362+0000 7f7964e10700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796019d130 con 0x7f7960106cd0 2024-04-23T02:22:11.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.362+0000 7f7964e10700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7960196630 con 0x7f7960068580 2024-04-23T02:22:11.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.362+0000 7f795dd9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7960106cd0 0x7f796019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:11.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.362+0000 7f795dd9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7960106cd0 0x7f796019caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40698/0 (socket says 172.21.15.71:40698) 2024-04-23T02:22:11.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.362+0000 7f795dd9b700 1 -- 172.21.15.71:0/2584601512 learned_addr learned my addr 172.21.15.71:0/2584601512 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:22:11.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.362+0000 7f795e59c700 1 --2- 172.21.15.71:0/2584601512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7960068580 0x7f796019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:11.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.362+0000 7f795dd9b700 1 -- 172.21.15.71:0/2584601512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7960068580 msgr2=0x7f796019c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:11.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.362+0000 7f795dd9b700 1 --2- 172.21.15.71:0/2584601512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7960068580 0x7f796019c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:11.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.362+0000 7f795dd9b700 1 -- 172.21.15.71:0/2584601512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79500095f0 con 0x7f7960106cd0 2024-04-23T02:22:11.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.362+0000 7f795dd9b700 1 --2- 172.21.15.71:0/2584601512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7960106cd0 0x7f796019caa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f795400d8d0 tx=0x7f795400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:11.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.362+0000 7f794f7fe700 1 -- 172.21.15.71:0/2584601512 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79540098a0 con 0x7f7960106cd0 2024-04-23T02:22:11.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.362+0000 7f7964e10700 1 -- 172.21.15.71:0/2584601512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7960196910 con 0x7f7960106cd0 2024-04-23T02:22:11.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.362+0000 7f794f7fe700 1 -- 172.21.15.71:0/2584601512 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7954010460 con 0x7f7960106cd0 2024-04-23T02:22:11.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.362+0000 7f794f7fe700 1 -- 172.21.15.71:0/2584601512 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7954009ea0 con 0x7f7960106cd0 2024-04-23T02:22:11.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.362+0000 7f7964e10700 1 -- 172.21.15.71:0/2584601512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7960196e60 con 0x7f7960106cd0 2024-04-23T02:22:11.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.363+0000 7f7964e10700 1 -- 172.21.15.71:0/2584601512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79601091d0 con 0x7f7960106cd0 2024-04-23T02:22:11.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.364+0000 7f794f7fe700 1 -- 172.21.15.71:0/2584601512 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f795400f6c0 con 0x7f7960106cd0 2024-04-23T02:22:11.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.364+0000 7f794f7fe700 1 --2- 172.21.15.71:0/2584601512 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f794806cce0 0x7f794806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:11.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.364+0000 7f794f7fe700 1 -- 172.21.15.71:0/2584601512 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f795408c510 con 0x7f7960106cd0 2024-04-23T02:22:11.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.364+0000 7f795e59c700 1 --2- 172.21.15.71:0/2584601512 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f794806cce0 0x7f794806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:11.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.365+0000 7f795e59c700 1 --2- 172.21.15.71:0/2584601512 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f794806cce0 0x7f794806f1a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7950006010 tx=0x7f795001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:11.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.368+0000 7f794f7fe700 1 -- 172.21.15.71:0/2584601512 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7954059f70 con 0x7f7960106cd0 2024-04-23T02:22:11.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.650+0000 7f7964e10700 1 -- 172.21.15.71:0/2584601512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f796004f2e0 con 0x7f7960106cd0 2024-04-23T02:22:11.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.651+0000 7f794f7fe700 1 -- 172.21.15.71:0/2584601512 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f7954059b00 con 0x7f7960106cd0 2024-04-23T02:22:11.654 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:22:11.654 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:22:11.655 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:22:11.655 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:22:11.655 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:22:11.655 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:22:11.655 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:22:11.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.653+0000 7f7964e10700 1 -- 172.21.15.71:0/2584601512 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f794806cce0 msgr2=0x7f794806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:11.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.653+0000 7f7964e10700 1 --2- 172.21.15.71:0/2584601512 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f794806cce0 0x7f794806f1a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7950006010 tx=0x7f795001a040 comp rx=0 tx=0).stop 2024-04-23T02:22:11.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.653+0000 7f7964e10700 1 -- 172.21.15.71:0/2584601512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7960106cd0 msgr2=0x7f796019caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:11.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.653+0000 7f7964e10700 1 --2- 172.21.15.71:0/2584601512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7960106cd0 0x7f796019caa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f795400d8d0 tx=0x7f795400dbe0 comp rx=0 tx=0).stop 2024-04-23T02:22:11.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.653+0000 7f7964e10700 1 -- 172.21.15.71:0/2584601512 shutdown_connections 2024-04-23T02:22:11.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.653+0000 7f7964e10700 1 --2- 172.21.15.71:0/2584601512 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f794806cce0 0x7f794806f1a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:11.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.653+0000 7f7964e10700 1 --2- 172.21.15.71:0/2584601512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7960068580 0x7f796019c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:11.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.653+0000 7f7964e10700 1 --2- 172.21.15.71:0/2584601512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7960106cd0 0x7f796019caa0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:11.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.653+0000 7f7964e10700 1 -- 172.21.15.71:0/2584601512 >> 172.21.15.71:0/2584601512 conn(0x7f7960075670 msgr2=0x7f79600fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:11.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.653+0000 7f7964e10700 1 -- 172.21.15.71:0/2584601512 shutdown_connections 2024-04-23T02:22:11.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:11.653+0000 7f7964e10700 1 -- 172.21.15.71:0/2584601512 wait complete. 2024-04-23T02:22:12.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:11 smithi192 ceph-mon[108617]: pgmap v264: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-23T02:22:12.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:11 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3998055765' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:22:12.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:11 smithi192 ceph-mon[108617]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:22:12.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:11 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2584601512' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:22:12.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:11 smithi071 ceph-mon[144544]: pgmap v264: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-23T02:22:12.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:11 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3998055765' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:22:12.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:11 smithi071 ceph-mon[144544]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:22:12.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:11 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2584601512' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:22:14.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:14 smithi192 ceph-mon[108617]: pgmap v265: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 204 B/s rd, 0 op/s 2024-04-23T02:22:14.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:14 smithi071 ceph-mon[144544]: pgmap v265: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 204 B/s rd, 0 op/s 2024-04-23T02:22:16.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:16 smithi071 ceph-mon[144544]: pgmap v266: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:16.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:16 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:16.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:16 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:16.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:16 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:16.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:16 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:16.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:16 smithi192 ceph-mon[108617]: pgmap v266: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:16.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:16 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:16.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:16 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:16.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:16 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:16.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:16 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:18.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:18 smithi071 ceph-mon[144544]: pgmap v267: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:18.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:18 smithi192 ceph-mon[108617]: pgmap v267: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:20.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:20 smithi071 ceph-mon[144544]: pgmap v268: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:20.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:20.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:20.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:22:20.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:22:20.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:20.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:22:20.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:20.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:20.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:20.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:20.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:22:20.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:20 smithi192 ceph-mon[108617]: pgmap v268: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:20.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:20.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:20.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:22:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:22:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:22:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:20.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:22:22.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:22 smithi071 ceph-mon[144544]: pgmap v269: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:22.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:22 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:22.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:22 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi071.vxejye", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T02:22:22.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:22 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:22:22.466 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:22 smithi192 ceph-mon[108617]: pgmap v269: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:22.466 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:22 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:22.467 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:22 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi071.vxejye", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T02:22:22.467 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:22 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:22:23.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:23 smithi071 ceph-mon[144544]: Upgrade: Updating mds.cephfs.smithi071.vxejye 2024-04-23T02:22:23.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:23 smithi071 ceph-mon[144544]: Deploying daemon mds.cephfs.smithi071.vxejye on smithi071 2024-04-23T02:22:23.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:23 smithi192 ceph-mon[108617]: Upgrade: Updating mds.cephfs.smithi071.vxejye 2024-04-23T02:22:23.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:23 smithi192 ceph-mon[108617]: Deploying daemon mds.cephfs.smithi071.vxejye on smithi071 2024-04-23T02:22:24.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:24 smithi071 ceph-mon[144544]: pgmap v270: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:24.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:24 smithi071 ceph-mon[144544]: osdmap e82: 6 total, 6 up, 6 in 2024-04-23T02:22:24.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:24 smithi192 ceph-mon[108617]: pgmap v270: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:24.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:24 smithi192 ceph-mon[108617]: osdmap e82: 6 total, 6 up, 6 in 2024-04-23T02:22:25.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:25 smithi071 ceph-mon[144544]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-23T02:22:25.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:25 smithi192 ceph-mon[108617]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-23T02:22:26.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:26 smithi071 ceph-mon[144544]: pgmap v272: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:26.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:26 smithi192 ceph-mon[108617]: pgmap v272: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:27.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:27 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.vxejye"}]: dispatch 2024-04-23T02:22:27.512 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:27 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.vxejye"}]: dispatch 2024-04-23T02:22:28.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:28 smithi071 ceph-mon[144544]: pgmap v273: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:28.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:28 smithi192 ceph-mon[108617]: pgmap v273: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:30.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:30 smithi071 ceph-mon[144544]: pgmap v274: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:30.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:30 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:30.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:30 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:30.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:30 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:22:30.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:30 smithi192 ceph-mon[108617]: pgmap v274: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:30.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:30 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:30.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:30 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:30.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:30 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:22:31.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:31 smithi071 ceph-mon[144544]: mds.? [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] up:boot 2024-04-23T02:22:31.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:31 smithi071 ceph-mon[144544]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-23T02:22:31.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:31 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.vxejye"}]: dispatch 2024-04-23T02:22:31.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:31 smithi192 ceph-mon[108617]: mds.? [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] up:boot 2024-04-23T02:22:31.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:31 smithi192 ceph-mon[108617]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-23T02:22:31.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:31 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi071.vxejye"}]: dispatch 2024-04-23T02:22:32.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:32 smithi071 ceph-mon[144544]: pgmap v275: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:32.558 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:32 smithi192 ceph-mon[108617]: pgmap v275: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:34.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:34 smithi071 ceph-mon[144544]: pgmap v276: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:34.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:34 smithi192 ceph-mon[108617]: pgmap v276: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:35.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:22:35.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:22:36.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:36 smithi071 ceph-mon[144544]: pgmap v277: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:36.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:36 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:36.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:36 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:36.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:36 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:36.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:36 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:36.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:36 smithi192 ceph-mon[108617]: pgmap v277: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:36.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:36 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:36.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:36 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:36.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:36 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:36.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:36 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:38.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:38 smithi071 ceph-mon[144544]: pgmap v278: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:38.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:38 smithi192 ceph-mon[108617]: pgmap v278: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:40.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:40 smithi071 ceph-mon[144544]: pgmap v279: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:40.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:40 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:40.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:40 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:40.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:40 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:22:40.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:40 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:22:40.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:40 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:40.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:40 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:22:40.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:40 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:40.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:40 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:40.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:40 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:40.244 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:40 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:40.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:40 smithi192 ceph-mon[108617]: pgmap v279: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:40.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:40 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:40.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:40 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:40.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:40 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:22:40.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:40 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:22:40.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:40 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:40.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:40 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:22:40.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:40 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:40.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:40 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:40.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:40 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:40.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:40 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:41.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.772+0000 7fb21b1c2700 1 -- 172.21.15.71:0/2933195878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2140ffc20 msgr2=0x7fb214100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:41.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.772+0000 7fb21b1c2700 1 --2- 172.21.15.71:0/2933195878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2140ffc20 0x7fb214100080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb200009910 tx=0x7fb200009c20 comp rx=0 tx=0).stop 2024-04-23T02:22:41.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.772+0000 7fb21b1c2700 1 -- 172.21.15.71:0/2933195878 shutdown_connections 2024-04-23T02:22:41.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.772+0000 7fb21b1c2700 1 --2- 172.21.15.71:0/2933195878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2140ffc20 0x7fb214100080 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:41.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.772+0000 7fb21b1c2700 1 --2- 172.21.15.71:0/2933195878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb214105bd0 0x7fb214105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:41.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.772+0000 7fb21b1c2700 1 -- 172.21.15.71:0/2933195878 >> 172.21.15.71:0/2933195878 conn(0x7fb214075670 msgr2=0x7fb214075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:41.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.772+0000 7fb21b1c2700 1 -- 172.21.15.71:0/2933195878 shutdown_connections 2024-04-23T02:22:41.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.772+0000 7fb21b1c2700 1 -- 172.21.15.71:0/2933195878 wait complete. 2024-04-23T02:22:41.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.773+0000 7fb21b1c2700 1 Processor -- start 2024-04-23T02:22:41.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.773+0000 7fb21b1c2700 1 -- start start 2024-04-23T02:22:41.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.773+0000 7fb21b1c2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2140ffc20 0x7fb2141989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:41.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.773+0000 7fb21b1c2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb214105bd0 0x7fb214198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:41.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.773+0000 7fb21b1c2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb214199570 con 0x7fb214105bd0 2024-04-23T02:22:41.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.773+0000 7fb21b1c2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21419d380 con 0x7fb2140ffc20 2024-04-23T02:22:41.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.773+0000 7fb213fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb214105bd0 0x7fb214198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:41.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.773+0000 7fb213fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb214105bd0 0x7fb214198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58104/0 (socket says 172.21.15.71:58104) 2024-04-23T02:22:41.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.773+0000 7fb213fff700 1 -- 172.21.15.71:0/409133022 learned_addr learned my addr 172.21.15.71:0/409133022 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:22:41.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.774+0000 7fb218f5e700 1 --2- 172.21.15.71:0/409133022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2140ffc20 0x7fb2141989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:41.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.774+0000 7fb213fff700 1 -- 172.21.15.71:0/409133022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2140ffc20 msgr2=0x7fb2141989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:41.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.774+0000 7fb213fff700 1 --2- 172.21.15.71:0/409133022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2140ffc20 0x7fb2141989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:41.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.774+0000 7fb213fff700 1 -- 172.21.15.71:0/409133022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2000095f0 con 0x7fb214105bd0 2024-04-23T02:22:41.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.774+0000 7fb213fff700 1 --2- 172.21.15.71:0/409133022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb214105bd0 0x7fb214198ee0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb200000c00 tx=0x7fb20000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:41.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.774+0000 7fb211ffb700 1 -- 172.21.15.71:0/409133022 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb20001d070 con 0x7fb214105bd0 2024-04-23T02:22:41.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.774+0000 7fb211ffb700 1 -- 172.21.15.71:0/409133022 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb200005dc0 con 0x7fb214105bd0 2024-04-23T02:22:41.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.774+0000 7fb211ffb700 1 -- 172.21.15.71:0/409133022 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb200003940 con 0x7fb214105bd0 2024-04-23T02:22:41.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.774+0000 7fb21b1c2700 1 -- 172.21.15.71:0/409133022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21419d600 con 0x7fb214105bd0 2024-04-23T02:22:41.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.774+0000 7fb21b1c2700 1 -- 172.21.15.71:0/409133022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb21419da90 con 0x7fb214105bd0 2024-04-23T02:22:41.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.775+0000 7fb21b1c2700 1 -- 172.21.15.71:0/409133022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb21404f2e0 con 0x7fb214105bd0 2024-04-23T02:22:41.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.776+0000 7fb211ffb700 1 -- 172.21.15.71:0/409133022 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb200004ae0 con 0x7fb214105bd0 2024-04-23T02:22:41.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.776+0000 7fb211ffb700 1 --2- 172.21.15.71:0/409133022 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb20406cce0 0x7fb20406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:41.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.776+0000 7fb211ffb700 1 -- 172.21.15.71:0/409133022 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb20008dad0 con 0x7fb214105bd0 2024-04-23T02:22:41.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.776+0000 7fb218f5e700 1 --2- 172.21.15.71:0/409133022 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb20406cce0 0x7fb20406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:41.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.777+0000 7fb218f5e700 1 --2- 172.21.15.71:0/409133022 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb20406cce0 0x7fb20406f1a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb208009c60 tx=0x7fb208009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:41.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.781+0000 7fb211ffb700 1 -- 172.21.15.71:0/409133022 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb20005b5d0 con 0x7fb214105bd0 2024-04-23T02:22:41.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.992+0000 7fb21b1c2700 1 -- 172.21.15.71:0/409133022 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb21419dd60 con 0x7fb20406cce0 2024-04-23T02:22:41.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.993+0000 7fb211ffb700 1 -- 172.21.15.71:0/409133022 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb21419dd60 con 0x7fb20406cce0 2024-04-23T02:22:41.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.996+0000 7fb21b1c2700 1 -- 172.21.15.71:0/409133022 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb20406cce0 msgr2=0x7fb20406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:41.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.996+0000 7fb21b1c2700 1 --2- 172.21.15.71:0/409133022 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb20406cce0 0x7fb20406f1a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb208009c60 tx=0x7fb208009260 comp rx=0 tx=0).stop 2024-04-23T02:22:41.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.996+0000 7fb21b1c2700 1 -- 172.21.15.71:0/409133022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb214105bd0 msgr2=0x7fb214198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:41.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.996+0000 7fb21b1c2700 1 --2- 172.21.15.71:0/409133022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb214105bd0 0x7fb214198ee0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb200000c00 tx=0x7fb20000bde0 comp rx=0 tx=0).stop 2024-04-23T02:22:41.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.996+0000 7fb21b1c2700 1 -- 172.21.15.71:0/409133022 shutdown_connections 2024-04-23T02:22:41.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.996+0000 7fb21b1c2700 1 --2- 172.21.15.71:0/409133022 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb20406cce0 0x7fb20406f1a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:41.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.996+0000 7fb21b1c2700 1 --2- 172.21.15.71:0/409133022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2140ffc20 0x7fb2141989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:41.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.996+0000 7fb21b1c2700 1 --2- 172.21.15.71:0/409133022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb214105bd0 0x7fb214198ee0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:42.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.996+0000 7fb21b1c2700 1 -- 172.21.15.71:0/409133022 >> 172.21.15.71:0/409133022 conn(0x7fb214075670 msgr2=0x7fb2140687e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:42.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.996+0000 7fb21b1c2700 1 -- 172.21.15.71:0/409133022 shutdown_connections 2024-04-23T02:22:42.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:41.996+0000 7fb21b1c2700 1 -- 172.21.15.71:0/409133022 wait complete. 2024-04-23T02:22:42.009 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:22:42.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.100+0000 7f94f8b12700 1 -- 172.21.15.71:0/734860859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94f40ffda0 msgr2=0x7f94f4100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:42.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.100+0000 7f94f8b12700 1 --2- 172.21.15.71:0/734860859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94f40ffda0 0x7f94f4100180 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f94e8009910 tx=0x7f94e8009c20 comp rx=0 tx=0).stop 2024-04-23T02:22:42.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.100+0000 7f94f8b12700 1 -- 172.21.15.71:0/734860859 shutdown_connections 2024-04-23T02:22:42.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.100+0000 7f94f8b12700 1 --2- 172.21.15.71:0/734860859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94f4100750 0x7f94f41047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:42.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.100+0000 7f94f8b12700 1 --2- 172.21.15.71:0/734860859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94f40ffda0 0x7f94f4100180 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:42.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.100+0000 7f94f8b12700 1 -- 172.21.15.71:0/734860859 >> 172.21.15.71:0/734860859 conn(0x7f94f40fb650 msgr2=0x7f94f40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:42.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.100+0000 7f94f8b12700 1 -- 172.21.15.71:0/734860859 shutdown_connections 2024-04-23T02:22:42.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.100+0000 7f94f8b12700 1 -- 172.21.15.71:0/734860859 wait complete. 2024-04-23T02:22:42.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.101+0000 7f94f8b12700 1 Processor -- start 2024-04-23T02:22:42.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.101+0000 7f94f8b12700 1 -- start start 2024-04-23T02:22:42.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.101+0000 7f94f8b12700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94f40ffda0 0x7f94f41956d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:42.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.101+0000 7f94f8b12700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94f4100750 0x7f94f4195c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:42.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.101+0000 7f94f8b12700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f41962a0 con 0x7f94f40ffda0 2024-04-23T02:22:42.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.101+0000 7f94f8b12700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f4198fd0 con 0x7f94f4100750 2024-04-23T02:22:42.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.101+0000 7f94f259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94f40ffda0 0x7f94f41956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:42.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.101+0000 7f94f259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94f40ffda0 0x7f94f41956d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58126/0 (socket says 172.21.15.71:58126) 2024-04-23T02:22:42.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.101+0000 7f94f259c700 1 -- 172.21.15.71:0/3435076094 learned_addr learned my addr 172.21.15.71:0/3435076094 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:22:42.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.101+0000 7f94f1d9b700 1 --2- 172.21.15.71:0/3435076094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94f4100750 0x7f94f4195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:42.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.101+0000 7f94f259c700 1 -- 172.21.15.71:0/3435076094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94f4100750 msgr2=0x7f94f4195c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:42.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.101+0000 7f94f259c700 1 --2- 172.21.15.71:0/3435076094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94f4100750 0x7f94f4195c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:42.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.101+0000 7f94f259c700 1 -- 172.21.15.71:0/3435076094 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e80095f0 con 0x7f94f40ffda0 2024-04-23T02:22:42.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.101+0000 7f94f259c700 1 --2- 172.21.15.71:0/3435076094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94f40ffda0 0x7f94f41956d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f94e8006010 tx=0x7f94e800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:42.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.101+0000 7f94e37fe700 1 -- 172.21.15.71:0/3435076094 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e801d070 con 0x7f94f40ffda0 2024-04-23T02:22:42.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.102+0000 7f94f8b12700 1 -- 172.21.15.71:0/3435076094 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f4199250 con 0x7f94f40ffda0 2024-04-23T02:22:42.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.101+0000 7f94e37fe700 1 -- 172.21.15.71:0/3435076094 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f94e8005710 con 0x7f94f40ffda0 2024-04-23T02:22:42.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.102+0000 7f94e37fe700 1 -- 172.21.15.71:0/3435076094 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e800fe20 con 0x7f94f40ffda0 2024-04-23T02:22:42.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.102+0000 7f94f8b12700 1 -- 172.21.15.71:0/3435076094 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f4199740 con 0x7f94f40ffda0 2024-04-23T02:22:42.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.102+0000 7f94f8b12700 1 -- 172.21.15.71:0/3435076094 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f4108140 con 0x7f94f40ffda0 2024-04-23T02:22:42.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.103+0000 7f94e37fe700 1 -- 172.21.15.71:0/3435076094 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f94e8003a60 con 0x7f94f40ffda0 2024-04-23T02:22:42.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.103+0000 7f94e37fe700 1 --2- 172.21.15.71:0/3435076094 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f94dc06cce0 0x7f94dc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:42.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.103+0000 7f94e37fe700 1 -- 172.21.15.71:0/3435076094 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f94e808d750 con 0x7f94f40ffda0 2024-04-23T02:22:42.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.103+0000 7f94f1d9b700 1 --2- 172.21.15.71:0/3435076094 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f94dc06cce0 0x7f94dc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:42.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.104+0000 7f94f1d9b700 1 --2- 172.21.15.71:0/3435076094 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f94dc06cce0 0x7f94dc06f1a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f94f4196cf0 tx=0x7f94e40073d0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:42.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.108+0000 7f94e37fe700 1 -- 172.21.15.71:0/3435076094 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f94e80577c0 con 0x7f94f40ffda0 2024-04-23T02:22:42.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.320+0000 7f94f8b12700 1 -- 172.21.15.71:0/3435076094 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f94f4196b70 con 0x7f94dc06cce0 2024-04-23T02:22:42.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.321+0000 7f94e37fe700 1 -- 172.21.15.71:0/3435076094 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f94e4004030 con 0x7f94dc06cce0 2024-04-23T02:22:42.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.323+0000 7f94f8b12700 1 -- 172.21.15.71:0/3435076094 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f94dc06cce0 msgr2=0x7f94dc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:42.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.323+0000 7f94f8b12700 1 --2- 172.21.15.71:0/3435076094 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f94dc06cce0 0x7f94dc06f1a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f94f4196cf0 tx=0x7f94e40073d0 comp rx=0 tx=0).stop 2024-04-23T02:22:42.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.323+0000 7f94f8b12700 1 -- 172.21.15.71:0/3435076094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94f40ffda0 msgr2=0x7f94f41956d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:42.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.323+0000 7f94f8b12700 1 --2- 172.21.15.71:0/3435076094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94f40ffda0 0x7f94f41956d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f94e8006010 tx=0x7f94e800bd70 comp rx=0 tx=0).stop 2024-04-23T02:22:42.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.323+0000 7f94f8b12700 1 -- 172.21.15.71:0/3435076094 shutdown_connections 2024-04-23T02:22:42.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.323+0000 7f94f8b12700 1 --2- 172.21.15.71:0/3435076094 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f94dc06cce0 0x7f94dc06f1a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:42.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.323+0000 7f94f8b12700 1 --2- 172.21.15.71:0/3435076094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94f40ffda0 0x7f94f41956d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:42.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.323+0000 7f94f8b12700 1 --2- 172.21.15.71:0/3435076094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94f4100750 0x7f94f4195c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:42.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.323+0000 7f94f8b12700 1 -- 172.21.15.71:0/3435076094 >> 172.21.15.71:0/3435076094 conn(0x7f94f40fb650 msgr2=0x7f94f40fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:42.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.323+0000 7f94f8b12700 1 -- 172.21.15.71:0/3435076094 shutdown_connections 2024-04-23T02:22:42.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.323+0000 7f94f8b12700 1 -- 172.21.15.71:0/3435076094 wait complete. 2024-04-23T02:22:42.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.426+0000 7fb7ded8d700 1 -- 172.21.15.71:0/4255977882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7d8105bd0 msgr2=0x7fb7d8105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:42.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.426+0000 7fb7ded8d700 1 --2- 172.21.15.71:0/4255977882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7d8105bd0 0x7fb7d8105fb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009910 tx=0x7fb7cc009c20 comp rx=0 tx=0).stop 2024-04-23T02:22:42.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.426+0000 7fb7ded8d700 1 -- 172.21.15.71:0/4255977882 shutdown_connections 2024-04-23T02:22:42.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.426+0000 7fb7ded8d700 1 --2- 172.21.15.71:0/4255977882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7d80ffc20 0x7fb7d8100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:42.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.426+0000 7fb7ded8d700 1 --2- 172.21.15.71:0/4255977882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7d8105bd0 0x7fb7d8105fb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:42.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.426+0000 7fb7ded8d700 1 -- 172.21.15.71:0/4255977882 >> 172.21.15.71:0/4255977882 conn(0x7fb7d8075670 msgr2=0x7fb7d8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:42.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.426+0000 7fb7ded8d700 1 -- 172.21.15.71:0/4255977882 shutdown_connections 2024-04-23T02:22:42.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.426+0000 7fb7ded8d700 1 -- 172.21.15.71:0/4255977882 wait complete. 2024-04-23T02:22:42.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7ded8d700 1 Processor -- start 2024-04-23T02:22:42.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7ded8d700 1 -- start start 2024-04-23T02:22:42.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7ded8d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7d80ffc20 0x7fb7d8195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:42.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7ded8d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7d8105bd0 0x7fb7d8195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:42.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7ded8d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d8196260 con 0x7fb7d80ffc20 2024-04-23T02:22:42.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7ded8d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d8198f90 con 0x7fb7d8105bd0 2024-04-23T02:22:42.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7dcb29700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7d80ffc20 0x7fb7d8195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:42.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7dcb29700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7d80ffc20 0x7fb7d8195690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58144/0 (socket says 172.21.15.71:58144) 2024-04-23T02:22:42.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7dcb29700 1 -- 172.21.15.71:0/3637536547 learned_addr learned my addr 172.21.15.71:0/3637536547 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:22:42.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7d7fff700 1 --2- 172.21.15.71:0/3637536547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7d8105bd0 0x7fb7d8195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:42.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7dcb29700 1 -- 172.21.15.71:0/3637536547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7d8105bd0 msgr2=0x7fb7d8195bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:42.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7dcb29700 1 --2- 172.21.15.71:0/3637536547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7d8105bd0 0x7fb7d8195bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:42.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7dcb29700 1 -- 172.21.15.71:0/3637536547 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc0095f0 con 0x7fb7d80ffc20 2024-04-23T02:22:42.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7d7fff700 1 --2- 172.21.15.71:0/3637536547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7d8105bd0 0x7fb7d8195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:22:42.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7dcb29700 1 --2- 172.21.15.71:0/3637536547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7d80ffc20 0x7fb7d8195690 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc006010 tx=0x7fb7cc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:42.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7d5ffb700 1 -- 172.21.15.71:0/3637536547 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7cc01f070 con 0x7fb7d80ffc20 2024-04-23T02:22:42.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7d5ffb700 1 -- 172.21.15.71:0/3637536547 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb7cc019610 con 0x7fb7d80ffc20 2024-04-23T02:22:42.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7ded8d700 1 -- 172.21.15.71:0/3637536547 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d8199210 con 0x7fb7d80ffc20 2024-04-23T02:22:42.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.427+0000 7fb7d5ffb700 1 -- 172.21.15.71:0/3637536547 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7cc00f670 con 0x7fb7d80ffc20 2024-04-23T02:22:42.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.428+0000 7fb7ded8d700 1 -- 172.21.15.71:0/3637536547 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d8199700 con 0x7fb7d80ffc20 2024-04-23T02:22:42.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.429+0000 7fb7ded8d700 1 -- 172.21.15.71:0/3637536547 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d804f2e0 con 0x7fb7d80ffc20 2024-04-23T02:22:42.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.429+0000 7fb7d5ffb700 1 -- 172.21.15.71:0/3637536547 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb7cc019780 con 0x7fb7d80ffc20 2024-04-23T02:22:42.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.429+0000 7fb7d5ffb700 1 --2- 172.21.15.71:0/3637536547 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb7c806cce0 0x7fb7c806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:42.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.429+0000 7fb7d5ffb700 1 -- 172.21.15.71:0/3637536547 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb7cc08fbd0 con 0x7fb7d80ffc20 2024-04-23T02:22:42.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.430+0000 7fb7d7fff700 1 --2- 172.21.15.71:0/3637536547 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb7c806cce0 0x7fb7c806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:42.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.430+0000 7fb7d7fff700 1 --2- 172.21.15.71:0/3637536547 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb7c806cce0 0x7fb7c806f1a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8196cb0 tx=0x7fb7c4005f50 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:42.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.434+0000 7fb7d5ffb700 1 -- 172.21.15.71:0/3637536547 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb7cc059ba0 con 0x7fb7d80ffc20 2024-04-23T02:22:42.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:42 smithi071 ceph-mon[144544]: pgmap v280: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:42.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:42 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:42.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:42 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.gtmyst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T02:22:42.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:42 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:22:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:42 smithi192 ceph-mon[108617]: pgmap v280: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:42 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:42 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.gtmyst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T02:22:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:42 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:22:42.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.638+0000 7fb7ded8d700 1 -- 172.21.15.71:0/3637536547 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7d8199920 con 0x7fb7c806cce0 2024-04-23T02:22:42.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.648+0000 7fb7d5ffb700 1 -- 172.21.15.71:0/3637536547 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb7d8199920 con 0x7fb7c806cce0 2024-04-23T02:22:42.651 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:22:42.651 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (6m) 7s ago 17m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:22:42.652 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (18m) 7s ago 18m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:22:42.652 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (16m) 48s ago 16m 8321k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:22:42.652 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (4m) 7s ago 17m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:22:42.652 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (4m) 48s ago 16m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:22:42.652 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (6m) 7s ago 17m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:22:42.652 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (33s) 7s ago 11m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:22:42.652 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (13s) 7s ago 11m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:22:42.652 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (11m) 48s ago 11m 68.2M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:22:42.652 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (11m) 48s ago 11m 122M - 18.2.1 5be31c24972a e42ed50aca41 2024-04-23T02:22:42.652 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (8m) 7s ago 19m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:22:42.652 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (7m) 48s ago 16m 453M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:22:42.652 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (4m) 7s ago 19m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:22:42.652 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (4m) 48s ago 16m 42.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:22:42.652 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (7m) 7s ago 17m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:22:42.653 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (7m) 48s ago 16m 17.9M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:22:42.653 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (3m) 7s ago 15m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:22:42.653 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (3m) 7s ago 15m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:22:42.653 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (2m) 7s ago 14m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:22:42.653 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (2m) 48s ago 14m 102M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:22:42.653 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (87s) 48s ago 13m 91.2M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:22:42.653 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (52s) 48s ago 13m 11.4M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:22:42.653 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (6m) 7s ago 17m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:22:42.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.650+0000 7fb7ded8d700 1 -- 172.21.15.71:0/3637536547 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb7c806cce0 msgr2=0x7fb7c806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:42.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.650+0000 7fb7ded8d700 1 --2- 172.21.15.71:0/3637536547 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb7c806cce0 0x7fb7c806f1a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8196cb0 tx=0x7fb7c4005f50 comp rx=0 tx=0).stop 2024-04-23T02:22:42.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.650+0000 7fb7ded8d700 1 -- 172.21.15.71:0/3637536547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7d80ffc20 msgr2=0x7fb7d8195690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:42.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.650+0000 7fb7ded8d700 1 --2- 172.21.15.71:0/3637536547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7d80ffc20 0x7fb7d8195690 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc006010 tx=0x7fb7cc00be20 comp rx=0 tx=0).stop 2024-04-23T02:22:42.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.650+0000 7fb7ded8d700 1 -- 172.21.15.71:0/3637536547 shutdown_connections 2024-04-23T02:22:42.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.650+0000 7fb7ded8d700 1 --2- 172.21.15.71:0/3637536547 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb7c806cce0 0x7fb7c806f1a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:42.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.650+0000 7fb7ded8d700 1 --2- 172.21.15.71:0/3637536547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7d80ffc20 0x7fb7d8195690 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:42.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.650+0000 7fb7ded8d700 1 --2- 172.21.15.71:0/3637536547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7d8105bd0 0x7fb7d8195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:42.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.650+0000 7fb7ded8d700 1 -- 172.21.15.71:0/3637536547 >> 172.21.15.71:0/3637536547 conn(0x7fb7d8075670 msgr2=0x7fb7d80fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:42.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.651+0000 7fb7ded8d700 1 -- 172.21.15.71:0/3637536547 shutdown_connections 2024-04-23T02:22:42.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.651+0000 7fb7ded8d700 1 -- 172.21.15.71:0/3637536547 wait complete. 2024-04-23T02:22:42.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.758+0000 7fee7fd55700 1 -- 172.21.15.71:0/693481240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78100ea0 msgr2=0x7fee78101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:42.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.758+0000 7fee7fd55700 1 --2- 172.21.15.71:0/693481240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78100ea0 0x7fee78101280 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fee74009910 tx=0x7fee74009c20 comp rx=0 tx=0).stop 2024-04-23T02:22:42.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.758+0000 7fee7fd55700 1 -- 172.21.15.71:0/693481240 shutdown_connections 2024-04-23T02:22:42.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.758+0000 7fee7fd55700 1 --2- 172.21.15.71:0/693481240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee78101850 0x7fee781058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:42.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.758+0000 7fee7fd55700 1 --2- 172.21.15.71:0/693481240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78100ea0 0x7fee78101280 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:42.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.758+0000 7fee7fd55700 1 -- 172.21.15.71:0/693481240 >> 172.21.15.71:0/693481240 conn(0x7fee780fc750 msgr2=0x7fee780feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:42.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.759+0000 7fee7fd55700 1 -- 172.21.15.71:0/693481240 shutdown_connections 2024-04-23T02:22:42.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.759+0000 7fee7fd55700 1 -- 172.21.15.71:0/693481240 wait complete. 2024-04-23T02:22:42.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.759+0000 7fee7fd55700 1 Processor -- start 2024-04-23T02:22:42.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.759+0000 7fee7fd55700 1 -- start start 2024-04-23T02:22:42.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.759+0000 7fee7fd55700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee78100ea0 0x7fee7819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:42.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.759+0000 7fee7fd55700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78101850 0x7fee7819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:42.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.759+0000 7fee7fd55700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7819d130 con 0x7fee78101850 2024-04-23T02:22:42.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.759+0000 7fee7fd55700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee781965e0 con 0x7fee78100ea0 2024-04-23T02:22:42.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.759+0000 7fee7d2f0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78101850 0x7fee7819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:42.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.760+0000 7fee7d2f0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78101850 0x7fee7819caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58170/0 (socket says 172.21.15.71:58170) 2024-04-23T02:22:42.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.760+0000 7fee7d2f0700 1 -- 172.21.15.71:0/590988625 learned_addr learned my addr 172.21.15.71:0/590988625 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:22:42.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.760+0000 7fee7daf1700 1 --2- 172.21.15.71:0/590988625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee78100ea0 0x7fee7819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:42.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.760+0000 7fee7d2f0700 1 -- 172.21.15.71:0/590988625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee78100ea0 msgr2=0x7fee7819c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:42.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.760+0000 7fee7d2f0700 1 --2- 172.21.15.71:0/590988625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee78100ea0 0x7fee7819c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:42.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.760+0000 7fee7d2f0700 1 -- 172.21.15.71:0/590988625 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee740095f0 con 0x7fee78101850 2024-04-23T02:22:42.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.760+0000 7fee7d2f0700 1 --2- 172.21.15.71:0/590988625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78101850 0x7fee7819caa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fee6800d8d0 tx=0x7fee6800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:42.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.760+0000 7fee6effd700 1 -- 172.21.15.71:0/590988625 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee680041f0 con 0x7fee78101850 2024-04-23T02:22:42.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.760+0000 7fee6effd700 1 -- 172.21.15.71:0/590988625 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fee68007620 con 0x7fee78101850 2024-04-23T02:22:42.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.760+0000 7fee7fd55700 1 -- 172.21.15.71:0/590988625 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee781968c0 con 0x7fee78101850 2024-04-23T02:22:42.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.760+0000 7fee6effd700 1 -- 172.21.15.71:0/590988625 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee68003b50 con 0x7fee78101850 2024-04-23T02:22:42.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.760+0000 7fee7fd55700 1 -- 172.21.15.71:0/590988625 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee78196e10 con 0x7fee78101850 2024-04-23T02:22:42.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.761+0000 7fee7fd55700 1 -- 172.21.15.71:0/590988625 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee78109240 con 0x7fee78101850 2024-04-23T02:22:42.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.762+0000 7fee6effd700 1 -- 172.21.15.71:0/590988625 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fee68004350 con 0x7fee78101850 2024-04-23T02:22:42.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.762+0000 7fee6effd700 1 --2- 172.21.15.71:0/590988625 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee6406cce0 0x7fee6406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:42.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.762+0000 7fee6effd700 1 -- 172.21.15.71:0/590988625 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fee6808c060 con 0x7fee78101850 2024-04-23T02:22:42.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.762+0000 7fee7daf1700 1 --2- 172.21.15.71:0/590988625 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee6406cce0 0x7fee6406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:42.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.762+0000 7fee7daf1700 1 --2- 172.21.15.71:0/590988625 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee6406cce0 0x7fee6406f1a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fee74006010 tx=0x7fee7401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:42.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:42.766+0000 7fee6effd700 1 -- 172.21.15.71:0/590988625 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fee68059be0 con 0x7fee78101850 2024-04-23T02:22:43.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.048+0000 7fee7fd55700 1 -- 172.21.15.71:0/590988625 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fee7804f2e0 con 0x7fee78101850 2024-04-23T02:22:43.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.048+0000 7fee6effd700 1 -- 172.21.15.71:0/590988625 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7fee68059770 con 0x7fee78101850 2024-04-23T02:22:43.052 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:22:43.052 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:22:43.052 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:22:43.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:22:43.052 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:22:43.052 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:22:43.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:22:43.052 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:22:43.052 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:22:43.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:22:43.052 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:22:43.052 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-04-23T02:22:43.053 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:22:43.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:22:43.053 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:22:43.053 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-04-23T02:22:43.053 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 12 2024-04-23T02:22:43.053 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:22:43.053 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:22:43.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.050+0000 7fee7fd55700 1 -- 172.21.15.71:0/590988625 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee6406cce0 msgr2=0x7fee6406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:43.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.051+0000 7fee7fd55700 1 --2- 172.21.15.71:0/590988625 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee6406cce0 0x7fee6406f1a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fee74006010 tx=0x7fee7401a040 comp rx=0 tx=0).stop 2024-04-23T02:22:43.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.051+0000 7fee7fd55700 1 -- 172.21.15.71:0/590988625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78101850 msgr2=0x7fee7819caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:43.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.051+0000 7fee7fd55700 1 --2- 172.21.15.71:0/590988625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78101850 0x7fee7819caa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fee6800d8d0 tx=0x7fee6800dc90 comp rx=0 tx=0).stop 2024-04-23T02:22:43.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.051+0000 7fee7fd55700 1 -- 172.21.15.71:0/590988625 shutdown_connections 2024-04-23T02:22:43.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.051+0000 7fee7fd55700 1 --2- 172.21.15.71:0/590988625 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee6406cce0 0x7fee6406f1a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.051+0000 7fee7fd55700 1 --2- 172.21.15.71:0/590988625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee78100ea0 0x7fee7819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.051+0000 7fee7fd55700 1 --2- 172.21.15.71:0/590988625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78101850 0x7fee7819caa0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.051+0000 7fee7fd55700 1 -- 172.21.15.71:0/590988625 >> 172.21.15.71:0/590988625 conn(0x7fee780fc750 msgr2=0x7fee780fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:43.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.051+0000 7fee7fd55700 1 -- 172.21.15.71:0/590988625 shutdown_connections 2024-04-23T02:22:43.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.051+0000 7fee7fd55700 1 -- 172.21.15.71:0/590988625 wait complete. 2024-04-23T02:22:43.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.154+0000 7f1cd8740700 1 -- 172.21.15.71:0/1038127851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd0101e60 msgr2=0x7f1cd0102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:43.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.154+0000 7f1cd8740700 1 --2- 172.21.15.71:0/1038127851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd0101e60 0x7f1cd0102240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009910 tx=0x7f1cc0009c20 comp rx=0 tx=0).stop 2024-04-23T02:22:43.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.154+0000 7f1cd8740700 1 -- 172.21.15.71:0/1038127851 shutdown_connections 2024-04-23T02:22:43.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.154+0000 7f1cd8740700 1 --2- 172.21.15.71:0/1038127851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cd0102780 0x7f1cd010ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.154+0000 7f1cd8740700 1 --2- 172.21.15.71:0/1038127851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd0101e60 0x7f1cd0102240 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.154+0000 7f1cd8740700 1 -- 172.21.15.71:0/1038127851 >> 172.21.15.71:0/1038127851 conn(0x7f1cd00fb650 msgr2=0x7f1cd00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:43.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.154+0000 7f1cd8740700 1 -- 172.21.15.71:0/1038127851 shutdown_connections 2024-04-23T02:22:43.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.154+0000 7f1cd8740700 1 -- 172.21.15.71:0/1038127851 wait complete. 2024-04-23T02:22:43.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.154+0000 7f1cd8740700 1 Processor -- start 2024-04-23T02:22:43.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.154+0000 7f1cd8740700 1 -- start start 2024-04-23T02:22:43.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.155+0000 7f1cd8740700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cd0101e60 0x7f1cd019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:43.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.155+0000 7f1cd8740700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd0102780 0x7f1cd019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:43.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.155+0000 7f1cd8740700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd019d1f0 con 0x7f1cd0102780 2024-04-23T02:22:43.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.155+0000 7f1cd8740700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd01966a0 con 0x7f1cd0101e60 2024-04-23T02:22:43.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.155+0000 7f1cd5cdb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd0102780 0x7f1cd019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:43.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.155+0000 7f1cd5cdb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd0102780 0x7f1cd019cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58182/0 (socket says 172.21.15.71:58182) 2024-04-23T02:22:43.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.155+0000 7f1cd5cdb700 1 -- 172.21.15.71:0/1623536170 learned_addr learned my addr 172.21.15.71:0/1623536170 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:22:43.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.155+0000 7f1cd64dc700 1 --2- 172.21.15.71:0/1623536170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cd0101e60 0x7f1cd019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:43.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.155+0000 7f1cd5cdb700 1 -- 172.21.15.71:0/1623536170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cd0101e60 msgr2=0x7f1cd019c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:43.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.155+0000 7f1cd5cdb700 1 --2- 172.21.15.71:0/1623536170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cd0101e60 0x7f1cd019c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.155+0000 7f1cd5cdb700 1 -- 172.21.15.71:0/1623536170 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc00095f0 con 0x7f1cd0102780 2024-04-23T02:22:43.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.155+0000 7f1cd64dc700 1 --2- 172.21.15.71:0/1623536170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cd0101e60 0x7f1cd019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T02:22:43.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.155+0000 7f1cd5cdb700 1 --2- 172.21.15.71:0/1623536170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd0102780 0x7f1cd019cb60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00e8f0 tx=0x7f1ccc00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:43.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.155+0000 7f1cc77fe700 1 -- 172.21.15.71:0/1623536170 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ccc009790 con 0x7f1cd0102780 2024-04-23T02:22:43.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.155+0000 7f1cc77fe700 1 -- 172.21.15.71:0/1623536170 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1ccc00c810 con 0x7f1cd0102780 2024-04-23T02:22:43.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.155+0000 7f1cd8740700 1 -- 172.21.15.71:0/1623536170 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd0196980 con 0x7f1cd0102780 2024-04-23T02:22:43.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.155+0000 7f1cc77fe700 1 -- 172.21.15.71:0/1623536170 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ccc005520 con 0x7f1cd0102780 2024-04-23T02:22:43.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.155+0000 7f1cd8740700 1 -- 172.21.15.71:0/1623536170 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd0196ed0 con 0x7f1cd0102780 2024-04-23T02:22:43.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.156+0000 7f1cd8740700 1 -- 172.21.15.71:0/1623536170 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd0108310 con 0x7f1cd0102780 2024-04-23T02:22:43.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.157+0000 7f1cc77fe700 1 -- 172.21.15.71:0/1623536170 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1ccc00c980 con 0x7f1cd0102780 2024-04-23T02:22:43.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.157+0000 7f1cc77fe700 1 --2- 172.21.15.71:0/1623536170 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1cbc06cce0 0x7f1cbc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:43.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.157+0000 7f1cc77fe700 1 -- 172.21.15.71:0/1623536170 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1ccc014070 con 0x7f1cd0102780 2024-04-23T02:22:43.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.157+0000 7f1cd64dc700 1 --2- 172.21.15.71:0/1623536170 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1cbc06cce0 0x7f1cbc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:43.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.158+0000 7f1cd64dc700 1 --2- 172.21.15.71:0/1623536170 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1cbc06cce0 0x7f1cbc06f1a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009fd0 tx=0x7f1cc0005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:43.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.162+0000 7f1cc77fe700 1 -- 172.21.15.71:0/1623536170 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1ccc05a3b0 con 0x7f1cd0102780 2024-04-23T02:22:43.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.396+0000 7f1cd8740700 1 -- 172.21.15.71:0/1623536170 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1cd004f2e0 con 0x7f1cd0102780 2024-04-23T02:22:43.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.396+0000 7f1cc77fe700 1 -- 172.21.15.71:0/1623536170 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f1ccc00cc30 con 0x7f1cd0102780 2024-04-23T02:22:43.399 INFO:teuthology.orchestra.run.smithi071.stdout:e32 2024-04-23T02:22:43.399 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:22:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:22:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:22:43.400 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:22:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:22:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:22:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:22:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:22:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:22:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:22:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:22:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:22:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:22:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:22:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:22:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:22:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:22:43.401 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:22:43.401 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:22:43.401 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:22:43.401 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:22:43.401 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:22:43.401 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:22:43.401 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:22:43.401 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:22:43.401 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:22:43.401 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:22:43.401 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:22:43.401 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:22:43.401 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:22:43.401 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:22:43.401 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:22:43.401 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:22:43.402 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:22:43.402 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:22:43.402 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:22:43.402 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:22:43.402 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:22:43.402 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:22:43.402 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44271} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/1096782156,v1:172.21.15.192:6825/1096782156] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:22:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.398+0000 7f1cd8740700 1 -- 172.21.15.71:0/1623536170 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1cbc06cce0 msgr2=0x7f1cbc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.398+0000 7f1cd8740700 1 --2- 172.21.15.71:0/1623536170 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1cbc06cce0 0x7f1cbc06f1a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009fd0 tx=0x7f1cc0005940 comp rx=0 tx=0).stop 2024-04-23T02:22:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.398+0000 7f1cd8740700 1 -- 172.21.15.71:0/1623536170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd0102780 msgr2=0x7f1cd019cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.398+0000 7f1cd8740700 1 --2- 172.21.15.71:0/1623536170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd0102780 0x7f1cd019cb60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00e8f0 tx=0x7f1ccc00ec00 comp rx=0 tx=0).stop 2024-04-23T02:22:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.398+0000 7f1cd8740700 1 -- 172.21.15.71:0/1623536170 shutdown_connections 2024-04-23T02:22:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.398+0000 7f1cd8740700 1 --2- 172.21.15.71:0/1623536170 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1cbc06cce0 0x7f1cbc06f1a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.398+0000 7f1cd8740700 1 --2- 172.21.15.71:0/1623536170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1cd0101e60 0x7f1cd019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.398+0000 7f1cd8740700 1 --2- 172.21.15.71:0/1623536170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd0102780 0x7f1cd019cb60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.398+0000 7f1cd8740700 1 -- 172.21.15.71:0/1623536170 >> 172.21.15.71:0/1623536170 conn(0x7f1cd00fb650 msgr2=0x7f1cd0105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.398+0000 7f1cd8740700 1 -- 172.21.15.71:0/1623536170 shutdown_connections 2024-04-23T02:22:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.398+0000 7f1cd8740700 1 -- 172.21.15.71:0/1623536170 wait complete. 2024-04-23T02:22:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 32 2024-04-23T02:22:43.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:43 smithi071 ceph-mon[144544]: Upgrade: Updating mds.cephfs.smithi192.gtmyst 2024-04-23T02:22:43.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:43 smithi071 ceph-mon[144544]: Deploying daemon mds.cephfs.smithi192.gtmyst on smithi192 2024-04-23T02:22:43.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:43 smithi071 ceph-mon[144544]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:22:43.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:43 smithi071 ceph-mon[144544]: from='client.34370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:22:43.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:43 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/590988625' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:43.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.502+0000 7fc2c2f48700 1 -- 172.21.15.71:0/2945552964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2bc0ffc20 msgr2=0x7fc2bc100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:43.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.502+0000 7fc2c2f48700 1 --2- 172.21.15.71:0/2945552964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2bc0ffc20 0x7fc2bc100080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009910 tx=0x7fc2a8009c20 comp rx=0 tx=0).stop 2024-04-23T02:22:43.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.502+0000 7fc2c2f48700 1 -- 172.21.15.71:0/2945552964 shutdown_connections 2024-04-23T02:22:43.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.502+0000 7fc2c2f48700 1 --2- 172.21.15.71:0/2945552964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2bc0ffc20 0x7fc2bc100080 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.502+0000 7fc2c2f48700 1 --2- 172.21.15.71:0/2945552964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2bc105bd0 0x7fc2bc105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.502+0000 7fc2c2f48700 1 -- 172.21.15.71:0/2945552964 >> 172.21.15.71:0/2945552964 conn(0x7fc2bc075670 msgr2=0x7fc2bc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:43.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.502+0000 7fc2c2f48700 1 -- 172.21.15.71:0/2945552964 shutdown_connections 2024-04-23T02:22:43.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.502+0000 7fc2c2f48700 1 -- 172.21.15.71:0/2945552964 wait complete. 2024-04-23T02:22:43.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.503+0000 7fc2c2f48700 1 Processor -- start 2024-04-23T02:22:43.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.503+0000 7fc2c2f48700 1 -- start start 2024-04-23T02:22:43.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.503+0000 7fc2c2f48700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2bc0ffc20 0x7fc2bc101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:43.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.503+0000 7fc2c2f48700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2bc105bd0 0x7fc2bc101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:43.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.503+0000 7fc2c2f48700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc1022c0 con 0x7fc2bc105bd0 2024-04-23T02:22:43.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.503+0000 7fc2c2f48700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc102400 con 0x7fc2bc0ffc20 2024-04-23T02:22:43.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.503+0000 7fc2bbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2bc105bd0 0x7fc2bc101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:43.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.503+0000 7fc2bbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2bc105bd0 0x7fc2bc101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58206/0 (socket says 172.21.15.71:58206) 2024-04-23T02:22:43.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.503+0000 7fc2bbfff700 1 -- 172.21.15.71:0/2922537412 learned_addr learned my addr 172.21.15.71:0/2922537412 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:22:43.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.503+0000 7fc2c0ce4700 1 --2- 172.21.15.71:0/2922537412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2bc0ffc20 0x7fc2bc101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:43.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.503+0000 7fc2bbfff700 1 -- 172.21.15.71:0/2922537412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2bc0ffc20 msgr2=0x7fc2bc101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:43.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.503+0000 7fc2bbfff700 1 --2- 172.21.15.71:0/2922537412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2bc0ffc20 0x7fc2bc101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.503+0000 7fc2bbfff700 1 -- 172.21.15.71:0/2922537412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a80095f0 con 0x7fc2bc105bd0 2024-04-23T02:22:43.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.503+0000 7fc2bbfff700 1 --2- 172.21.15.71:0/2922537412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2bc105bd0 0x7fc2bc101d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2a80092e0 tx=0x7fc2a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:43.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.503+0000 7fc2b9ffb700 1 -- 172.21.15.71:0/2922537412 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2a801d070 con 0x7fc2bc105bd0 2024-04-23T02:22:43.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.504+0000 7fc2b9ffb700 1 -- 172.21.15.71:0/2922537412 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc2a8005e00 con 0x7fc2bc105bd0 2024-04-23T02:22:43.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.504+0000 7fc2c2f48700 1 -- 172.21.15.71:0/2922537412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2bc102680 con 0x7fc2bc105bd0 2024-04-23T02:22:43.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.504+0000 7fc2b9ffb700 1 -- 172.21.15.71:0/2922537412 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2a800fe60 con 0x7fc2bc105bd0 2024-04-23T02:22:43.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.504+0000 7fc2c2f48700 1 -- 172.21.15.71:0/2922537412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2bc1a2cc0 con 0x7fc2bc105bd0 2024-04-23T02:22:43.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.505+0000 7fc2c2f48700 1 -- 172.21.15.71:0/2922537412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2bc04f2e0 con 0x7fc2bc105bd0 2024-04-23T02:22:43.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.505+0000 7fc2b9ffb700 1 -- 172.21.15.71:0/2922537412 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc2a8003a60 con 0x7fc2bc105bd0 2024-04-23T02:22:43.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.505+0000 7fc2b9ffb700 1 --2- 172.21.15.71:0/2922537412 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc2ac06cce0 0x7fc2ac06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:43.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.506+0000 7fc2b9ffb700 1 -- 172.21.15.71:0/2922537412 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc2a808d860 con 0x7fc2bc105bd0 2024-04-23T02:22:43.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.506+0000 7fc2c0ce4700 1 --2- 172.21.15.71:0/2922537412 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc2ac06cce0 0x7fc2ac06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:43.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.506+0000 7fc2c0ce4700 1 --2- 172.21.15.71:0/2922537412 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc2ac06cce0 0x7fc2ac06f1a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0005950 tx=0x7fc2b00058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:43.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.510+0000 7fc2b9ffb700 1 -- 172.21.15.71:0/2922537412 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc2a80578d0 con 0x7fc2bc105bd0 2024-04-23T02:22:43.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:43 smithi192 ceph-mon[108617]: Upgrade: Updating mds.cephfs.smithi192.gtmyst 2024-04-23T02:22:43.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:43 smithi192 ceph-mon[108617]: Deploying daemon mds.cephfs.smithi192.gtmyst on smithi192 2024-04-23T02:22:43.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:43 smithi192 ceph-mon[108617]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:22:43.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:43 smithi192 ceph-mon[108617]: from='client.34370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:22:43.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:43 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/590988625' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:22:43.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.720+0000 7fc2c2f48700 1 -- 172.21.15.71:0/2922537412 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2bc068e10 con 0x7fc2ac06cce0 2024-04-23T02:22:43.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.722+0000 7fc2b9ffb700 1 -- 172.21.15.71:0/2922537412 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc2bc068e10 con 0x7fc2ac06cce0 2024-04-23T02:22:43.725 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:22:43.725 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:22:43.725 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:22:43.725 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:22:43.725 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:22:43.725 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:22:43.725 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:22:43.725 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:22:43.725 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:22:43.725 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:22:43.726 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "14/23 daemons upgraded", 2024-04-23T02:22:43.726 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:22:43.726 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:22:43.726 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:22:43.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.724+0000 7fc2c2f48700 1 -- 172.21.15.71:0/2922537412 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc2ac06cce0 msgr2=0x7fc2ac06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:43.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.724+0000 7fc2c2f48700 1 --2- 172.21.15.71:0/2922537412 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc2ac06cce0 0x7fc2ac06f1a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0005950 tx=0x7fc2b00058e0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.724+0000 7fc2c2f48700 1 -- 172.21.15.71:0/2922537412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2bc105bd0 msgr2=0x7fc2bc101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:43.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.724+0000 7fc2c2f48700 1 --2- 172.21.15.71:0/2922537412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2bc105bd0 0x7fc2bc101d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2a80092e0 tx=0x7fc2a800be20 comp rx=0 tx=0).stop 2024-04-23T02:22:43.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.724+0000 7fc2c2f48700 1 -- 172.21.15.71:0/2922537412 shutdown_connections 2024-04-23T02:22:43.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.724+0000 7fc2c2f48700 1 --2- 172.21.15.71:0/2922537412 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc2ac06cce0 0x7fc2ac06f1a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.724+0000 7fc2c2f48700 1 --2- 172.21.15.71:0/2922537412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2bc0ffc20 0x7fc2bc101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.724+0000 7fc2c2f48700 1 --2- 172.21.15.71:0/2922537412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2bc105bd0 0x7fc2bc101d80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.724+0000 7fc2c2f48700 1 -- 172.21.15.71:0/2922537412 >> 172.21.15.71:0/2922537412 conn(0x7fc2bc075670 msgr2=0x7fc2bc068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:43.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.724+0000 7fc2c2f48700 1 -- 172.21.15.71:0/2922537412 shutdown_connections 2024-04-23T02:22:43.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.724+0000 7fc2c2f48700 1 -- 172.21.15.71:0/2922537412 wait complete. 2024-04-23T02:22:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.827+0000 7fb6420e2700 1 -- 172.21.15.71:0/3075320283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb63c0fff50 msgr2=0x7fb63c105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.827+0000 7fb6420e2700 1 --2- 172.21.15.71:0/3075320283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb63c0fff50 0x7fb63c105840 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb630009910 tx=0x7fb630009c20 comp rx=0 tx=0).stop 2024-04-23T02:22:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.827+0000 7fb6420e2700 1 -- 172.21.15.71:0/3075320283 shutdown_connections 2024-04-23T02:22:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.827+0000 7fb6420e2700 1 --2- 172.21.15.71:0/3075320283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb63c0fff50 0x7fb63c105840 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.827+0000 7fb6420e2700 1 --2- 172.21.15.71:0/3075320283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb63c0ff5a0 0x7fb63c0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.827+0000 7fb6420e2700 1 -- 172.21.15.71:0/3075320283 >> 172.21.15.71:0/3075320283 conn(0x7fb63c074e30 msgr2=0x7fb63c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.827+0000 7fb6420e2700 1 -- 172.21.15.71:0/3075320283 shutdown_connections 2024-04-23T02:22:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.827+0000 7fb6420e2700 1 -- 172.21.15.71:0/3075320283 wait complete. 2024-04-23T02:22:43.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.828+0000 7fb6420e2700 1 Processor -- start 2024-04-23T02:22:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.828+0000 7fb6420e2700 1 -- start start 2024-04-23T02:22:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.828+0000 7fb6420e2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb63c0ff5a0 0x7fb63c19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.828+0000 7fb6420e2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb63c19cd70 0x7fb63c1968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.828+0000 7fb6420e2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c19d340 con 0x7fb63c0ff5a0 2024-04-23T02:22:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.828+0000 7fb6420e2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c196df0 con 0x7fb63c19cd70 2024-04-23T02:22:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.829+0000 7fb63b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb63c0ff5a0 0x7fb63c19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.829+0000 7fb63b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb63c0ff5a0 0x7fb63c19c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58222/0 (socket says 172.21.15.71:58222) 2024-04-23T02:22:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.829+0000 7fb63b7fe700 1 -- 172.21.15.71:0/2015013084 learned_addr learned my addr 172.21.15.71:0/2015013084 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:22:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.829+0000 7fb63affd700 1 --2- 172.21.15.71:0/2015013084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb63c19cd70 0x7fb63c1968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.829+0000 7fb63b7fe700 1 -- 172.21.15.71:0/2015013084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb63c19cd70 msgr2=0x7fb63c1968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.829+0000 7fb63b7fe700 1 --2- 172.21.15.71:0/2015013084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb63c19cd70 0x7fb63c1968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.829+0000 7fb63b7fe700 1 -- 172.21.15.71:0/2015013084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6300095f0 con 0x7fb63c0ff5a0 2024-04-23T02:22:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.829+0000 7fb63b7fe700 1 --2- 172.21.15.71:0/2015013084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb63c0ff5a0 0x7fb63c19c830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00e920 tx=0x7fb62c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.829+0000 7fb638ff9700 1 -- 172.21.15.71:0/2015013084 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb62c00c9d0 con 0x7fb63c0ff5a0 2024-04-23T02:22:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.829+0000 7fb638ff9700 1 -- 172.21.15.71:0/2015013084 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb62c010430 con 0x7fb63c0ff5a0 2024-04-23T02:22:43.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.829+0000 7fb638ff9700 1 -- 172.21.15.71:0/2015013084 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb62c018650 con 0x7fb63c0ff5a0 2024-04-23T02:22:43.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.829+0000 7fb6420e2700 1 -- 172.21.15.71:0/2015013084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63c1970d0 con 0x7fb63c0ff5a0 2024-04-23T02:22:43.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.829+0000 7fb6420e2700 1 -- 172.21.15.71:0/2015013084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63c197620 con 0x7fb63c0ff5a0 2024-04-23T02:22:43.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.830+0000 7fb6420e2700 1 -- 172.21.15.71:0/2015013084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb63c109270 con 0x7fb63c0ff5a0 2024-04-23T02:22:43.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.831+0000 7fb638ff9700 1 -- 172.21.15.71:0/2015013084 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb62c0105a0 con 0x7fb63c0ff5a0 2024-04-23T02:22:43.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.831+0000 7fb638ff9700 1 --2- 172.21.15.71:0/2015013084 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb62406cce0 0x7fb62406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:22:43.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.831+0000 7fb638ff9700 1 -- 172.21.15.71:0/2015013084 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb62c014070 con 0x7fb63c0ff5a0 2024-04-23T02:22:43.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.831+0000 7fb63affd700 1 --2- 172.21.15.71:0/2015013084 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb62406cce0 0x7fb62406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:22:43.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.832+0000 7fb63affd700 1 --2- 172.21.15.71:0/2015013084 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb62406cce0 0x7fb62406f1a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb630006010 tx=0x7fb630009f90 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:22:43.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:43.835+0000 7fb638ff9700 1 -- 172.21.15.71:0/2015013084 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb62c08ce60 con 0x7fb63c0ff5a0 2024-04-23T02:22:44.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:44.106+0000 7fb6420e2700 1 -- 172.21.15.71:0/2015013084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb63c197dc0 con 0x7fb63c0ff5a0 2024-04-23T02:22:44.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:44.106+0000 7fb638ff9700 1 -- 172.21.15.71:0/2015013084 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fb62c026090 con 0x7fb63c0ff5a0 2024-04-23T02:22:44.109 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:22:44.109 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:22:44.109 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:22:44.110 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:22:44.110 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:22:44.110 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:22:44.110 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:22:44.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:44.108+0000 7fb6420e2700 1 -- 172.21.15.71:0/2015013084 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb62406cce0 msgr2=0x7fb62406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:44.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:44.108+0000 7fb6420e2700 1 --2- 172.21.15.71:0/2015013084 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb62406cce0 0x7fb62406f1a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb630006010 tx=0x7fb630009f90 comp rx=0 tx=0).stop 2024-04-23T02:22:44.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:44.108+0000 7fb6420e2700 1 -- 172.21.15.71:0/2015013084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb63c0ff5a0 msgr2=0x7fb63c19c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:22:44.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:44.108+0000 7fb6420e2700 1 --2- 172.21.15.71:0/2015013084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb63c0ff5a0 0x7fb63c19c830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00e920 tx=0x7fb62c00ec30 comp rx=0 tx=0).stop 2024-04-23T02:22:44.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:44.108+0000 7fb6420e2700 1 -- 172.21.15.71:0/2015013084 shutdown_connections 2024-04-23T02:22:44.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:44.108+0000 7fb6420e2700 1 --2- 172.21.15.71:0/2015013084 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb62406cce0 0x7fb62406f1a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:44.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:44.108+0000 7fb6420e2700 1 --2- 172.21.15.71:0/2015013084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb63c0ff5a0 0x7fb63c19c830 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:44.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:44.108+0000 7fb6420e2700 1 --2- 172.21.15.71:0/2015013084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb63c19cd70 0x7fb63c1968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:22:44.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:44.108+0000 7fb6420e2700 1 -- 172.21.15.71:0/2015013084 >> 172.21.15.71:0/2015013084 conn(0x7fb63c074e30 msgr2=0x7fb63c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:22:44.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:44.108+0000 7fb6420e2700 1 -- 172.21.15.71:0/2015013084 shutdown_connections 2024-04-23T02:22:44.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:22:44.108+0000 7fb6420e2700 1 -- 172.21.15.71:0/2015013084 wait complete. 2024-04-23T02:22:44.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:44 smithi071 ceph-mon[144544]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:22:44.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:44 smithi071 ceph-mon[144544]: pgmap v281: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:44.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:44 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1623536170' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:22:44.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:44 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2015013084' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:22:44.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:44 smithi071 ceph-mon[144544]: osdmap e83: 6 total, 6 up, 6 in 2024-04-23T02:22:44.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:44 smithi192 ceph-mon[108617]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:22:44.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:44 smithi192 ceph-mon[108617]: pgmap v281: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:44.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:44 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1623536170' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:22:44.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:44 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2015013084' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:22:44.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:44 smithi192 ceph-mon[108617]: osdmap e83: 6 total, 6 up, 6 in 2024-04-23T02:22:45.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:45 smithi071 ceph-mon[144544]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:22:45.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:45 smithi071 ceph-mon[144544]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-23T02:22:45.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:45 smithi192 ceph-mon[108617]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:22:45.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:45 smithi192 ceph-mon[108617]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-23T02:22:46.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:46 smithi071 ceph-mon[144544]: pgmap v283: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:46.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:46 smithi192 ceph-mon[108617]: pgmap v283: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:47.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:47 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.gtmyst"}]: dispatch 2024-04-23T02:22:47.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:47 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.gtmyst"}]: dispatch 2024-04-23T02:22:48.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:48 smithi071 ceph-mon[144544]: pgmap v284: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:48.603 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:48 smithi192 ceph-mon[108617]: pgmap v284: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:50 smithi071 ceph-mon[144544]: pgmap v285: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:22:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:50.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:22:50.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:50 smithi192 ceph-mon[108617]: pgmap v285: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:50.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:22:50.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:50.535 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:50.536 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:22:51.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:51 smithi192 ceph-mon[108617]: mds.? [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] up:boot 2024-04-23T02:22:51.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:51 smithi192 ceph-mon[108617]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-23T02:22:51.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:51 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.gtmyst"}]: dispatch 2024-04-23T02:22:51.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:51 smithi071 ceph-mon[144544]: mds.? [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] up:boot 2024-04-23T02:22:51.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:51 smithi071 ceph-mon[144544]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-23T02:22:51.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:51 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.gtmyst"}]: dispatch 2024-04-23T02:22:52.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:52 smithi192 ceph-mon[108617]: pgmap v286: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:52.633 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:52 smithi071 ceph-mon[144544]: pgmap v286: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:54.165 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:53 smithi192 ceph-mon[108617]: pgmap v287: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:54.165 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:53 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:54.165 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:53 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:54.166 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:53 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:54.166 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:53 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:54.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:53 smithi071 ceph-mon[144544]: pgmap v287: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:54.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:53 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:54.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:53 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:54.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:53 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:54.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:53 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:22:56.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:55 smithi071 ceph-mon[144544]: pgmap v288: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:56.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:55 smithi192 ceph-mon[108617]: pgmap v288: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:58.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:57 smithi192 ceph-mon[108617]: pgmap v289: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:22:58.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:57 smithi071 ceph-mon[144544]: pgmap v289: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:00.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:22:59 smithi071 ceph-mon[144544]: pgmap v290: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:00.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:22:59 smithi192 ceph-mon[108617]: pgmap v290: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:02.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:01 smithi071 ceph-mon[144544]: pgmap v291: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:02.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:01 smithi192 ceph-mon[108617]: pgmap v291: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:04.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:03 smithi071 ceph-mon[144544]: pgmap v292: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:04.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:03 smithi192 ceph-mon[108617]: pgmap v292: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:05.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:04 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:23:05.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:04 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:23:06.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:05 smithi071 ceph-mon[144544]: pgmap v293: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:06.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:05 smithi192 ceph-mon[108617]: pgmap v293: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:08.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:07 smithi071 ceph-mon[144544]: pgmap v294: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:08.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:07 smithi192 ceph-mon[108617]: pgmap v294: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:10.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:09 smithi071 ceph-mon[144544]: pgmap v295: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:10.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:09 smithi192 ceph-mon[108617]: pgmap v295: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:12.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:11 smithi071 ceph-mon[144544]: pgmap v296: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:12.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:11 smithi192 ceph-mon[108617]: pgmap v296: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:14.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.224+0000 7fe2b8ac0700 1 -- 172.21.15.71:0/494671541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2b4101840 msgr2=0x7fe2b4105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:14.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.224+0000 7fe2b8ac0700 1 --2- 172.21.15.71:0/494671541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2b4101840 0x7fe2b4105890 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009910 tx=0x7fe2a4009c20 comp rx=0 tx=0).stop 2024-04-23T02:23:14.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.224+0000 7fe2b8ac0700 1 -- 172.21.15.71:0/494671541 shutdown_connections 2024-04-23T02:23:14.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.224+0000 7fe2b8ac0700 1 --2- 172.21.15.71:0/494671541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2b4101840 0x7fe2b4105890 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:14.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.224+0000 7fe2b8ac0700 1 --2- 172.21.15.71:0/494671541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2b4100e90 0x7fe2b4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:14.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.224+0000 7fe2b8ac0700 1 -- 172.21.15.71:0/494671541 >> 172.21.15.71:0/494671541 conn(0x7fe2b40fc740 msgr2=0x7fe2b40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:14.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.224+0000 7fe2b8ac0700 1 -- 172.21.15.71:0/494671541 shutdown_connections 2024-04-23T02:23:14.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.224+0000 7fe2b8ac0700 1 -- 172.21.15.71:0/494671541 wait complete. 2024-04-23T02:23:14.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.225+0000 7fe2b8ac0700 1 Processor -- start 2024-04-23T02:23:14.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.225+0000 7fe2b8ac0700 1 -- start start 2024-04-23T02:23:14.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.225+0000 7fe2b8ac0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2b4100e90 0x7fe2b419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:14.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.225+0000 7fe2b8ac0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2b4101840 0x7fe2b419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:14.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.225+0000 7fe2b8ac0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b419d200 con 0x7fe2b4100e90 2024-04-23T02:23:14.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.225+0000 7fe2b8ac0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b41966b0 con 0x7fe2b4101840 2024-04-23T02:23:14.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.225+0000 7fe2b259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2b4100e90 0x7fe2b419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:14.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.225+0000 7fe2b259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2b4100e90 0x7fe2b419c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58868/0 (socket says 172.21.15.71:58868) 2024-04-23T02:23:14.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.225+0000 7fe2b259c700 1 -- 172.21.15.71:0/2925760226 learned_addr learned my addr 172.21.15.71:0/2925760226 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:23:14.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.226+0000 7fe2b1d9b700 1 --2- 172.21.15.71:0/2925760226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2b4101840 0x7fe2b419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:14.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.226+0000 7fe2b259c700 1 -- 172.21.15.71:0/2925760226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2b4101840 msgr2=0x7fe2b419cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:14.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.226+0000 7fe2b259c700 1 --2- 172.21.15.71:0/2925760226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2b4101840 0x7fe2b419cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:14.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.226+0000 7fe2b259c700 1 -- 172.21.15.71:0/2925760226 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a40095f0 con 0x7fe2b4100e90 2024-04-23T02:23:14.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.226+0000 7fe2b1d9b700 1 --2- 172.21.15.71:0/2925760226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2b4101840 0x7fe2b419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:23:14.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.226+0000 7fe2b259c700 1 --2- 172.21.15.71:0/2925760226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2b4100e90 0x7fe2b419c630 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800d8d0 tx=0x7fe2a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:14.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.226+0000 7fe2a37fe700 1 -- 172.21.15.71:0/2925760226 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2a8009940 con 0x7fe2b4100e90 2024-04-23T02:23:14.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.226+0000 7fe2a37fe700 1 -- 172.21.15.71:0/2925760226 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe2a8010460 con 0x7fe2b4100e90 2024-04-23T02:23:14.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.226+0000 7fe2b8ac0700 1 -- 172.21.15.71:0/2925760226 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b4196990 con 0x7fe2b4100e90 2024-04-23T02:23:14.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.226+0000 7fe2a37fe700 1 -- 172.21.15.71:0/2925760226 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2a800f5f0 con 0x7fe2b4100e90 2024-04-23T02:23:14.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.226+0000 7fe2b8ac0700 1 -- 172.21.15.71:0/2925760226 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b4196ee0 con 0x7fe2b4100e90 2024-04-23T02:23:14.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.227+0000 7fe2b8ac0700 1 -- 172.21.15.71:0/2925760226 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b4109290 con 0x7fe2b4100e90 2024-04-23T02:23:14.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.228+0000 7fe2a37fe700 1 -- 172.21.15.71:0/2925760226 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe2a80105d0 con 0x7fe2b4100e90 2024-04-23T02:23:14.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.228+0000 7fe2a37fe700 1 --2- 172.21.15.71:0/2925760226 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe29c06cce0 0x7fe29c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:14.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.228+0000 7fe2a37fe700 1 -- 172.21.15.71:0/2925760226 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe2a808c4f0 con 0x7fe2b4100e90 2024-04-23T02:23:14.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.228+0000 7fe2b1d9b700 1 --2- 172.21.15.71:0/2925760226 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe29c06cce0 0x7fe29c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:14.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.228+0000 7fe2b1d9b700 1 --2- 172.21.15.71:0/2925760226 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe29c06cce0 0x7fe29c06f1a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40fcfb0 tx=0x7fe2a401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:14.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.233+0000 7fe2a37fe700 1 -- 172.21.15.71:0/2925760226 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe2a805a070 con 0x7fe2b4100e90 2024-04-23T02:23:14.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:13 smithi071 ceph-mon[144544]: pgmap v297: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:14.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:13 smithi192 ceph-mon[108617]: pgmap v297: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:14.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.445+0000 7fe2b8ac0700 1 -- 172.21.15.71:0/2925760226 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2b4197cf0 con 0x7fe29c06cce0 2024-04-23T02:23:14.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.446+0000 7fe2a37fe700 1 -- 172.21.15.71:0/2925760226 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe2b4197cf0 con 0x7fe29c06cce0 2024-04-23T02:23:14.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.448+0000 7fe2b8ac0700 1 -- 172.21.15.71:0/2925760226 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe29c06cce0 msgr2=0x7fe29c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:14.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.448+0000 7fe2b8ac0700 1 --2- 172.21.15.71:0/2925760226 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe29c06cce0 0x7fe29c06f1a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40fcfb0 tx=0x7fe2a401a040 comp rx=0 tx=0).stop 2024-04-23T02:23:14.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.448+0000 7fe2b8ac0700 1 -- 172.21.15.71:0/2925760226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2b4100e90 msgr2=0x7fe2b419c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:14.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.448+0000 7fe2b8ac0700 1 --2- 172.21.15.71:0/2925760226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2b4100e90 0x7fe2b419c630 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800d8d0 tx=0x7fe2a800dc90 comp rx=0 tx=0).stop 2024-04-23T02:23:14.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.448+0000 7fe2b8ac0700 1 -- 172.21.15.71:0/2925760226 shutdown_connections 2024-04-23T02:23:14.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.448+0000 7fe2b8ac0700 1 --2- 172.21.15.71:0/2925760226 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe29c06cce0 0x7fe29c06f1a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:14.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.448+0000 7fe2b8ac0700 1 --2- 172.21.15.71:0/2925760226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2b4100e90 0x7fe2b419c630 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:14.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.448+0000 7fe2b8ac0700 1 --2- 172.21.15.71:0/2925760226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2b4101840 0x7fe2b419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:14.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.448+0000 7fe2b8ac0700 1 -- 172.21.15.71:0/2925760226 >> 172.21.15.71:0/2925760226 conn(0x7fe2b40fc740 msgr2=0x7fe2b40fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:14.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.448+0000 7fe2b8ac0700 1 -- 172.21.15.71:0/2925760226 shutdown_connections 2024-04-23T02:23:14.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.448+0000 7fe2b8ac0700 1 -- 172.21.15.71:0/2925760226 wait complete. 2024-04-23T02:23:14.462 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:23:14.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.552+0000 7f8c6d3e8700 1 -- 172.21.15.71:0/849184563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c68105bf0 msgr2=0x7f8c68105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:14.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.552+0000 7f8c6d3e8700 1 --2- 172.21.15.71:0/849184563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c68105bf0 0x7f8c68105fd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000b0a0 tx=0x7f8c6000b3b0 comp rx=0 tx=0).stop 2024-04-23T02:23:14.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.552+0000 7f8c6d3e8700 1 -- 172.21.15.71:0/849184563 shutdown_connections 2024-04-23T02:23:14.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.552+0000 7f8c6d3e8700 1 --2- 172.21.15.71:0/849184563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c680ffb40 0x7f8c680fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:14.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.552+0000 7f8c6d3e8700 1 --2- 172.21.15.71:0/849184563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c68105bf0 0x7f8c68105fd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:14.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.552+0000 7f8c6d3e8700 1 -- 172.21.15.71:0/849184563 >> 172.21.15.71:0/849184563 conn(0x7f8c680fb610 msgr2=0x7f8c680fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:14.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.552+0000 7f8c6d3e8700 1 -- 172.21.15.71:0/849184563 shutdown_connections 2024-04-23T02:23:14.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.552+0000 7f8c6d3e8700 1 -- 172.21.15.71:0/849184563 wait complete. 2024-04-23T02:23:14.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.552+0000 7f8c6d3e8700 1 Processor -- start 2024-04-23T02:23:14.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.552+0000 7f8c6d3e8700 1 -- start start 2024-04-23T02:23:14.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.553+0000 7f8c6d3e8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c680ffb40 0x7f8c681017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:14.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.553+0000 7f8c6d3e8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c68105bf0 0x7f8c68101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:14.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.553+0000 7f8c6d3e8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c68102230 con 0x7f8c680ffb40 2024-04-23T02:23:14.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.553+0000 7f8c6d3e8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c68102370 con 0x7f8c68105bf0 2024-04-23T02:23:14.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.553+0000 7f8c66ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c680ffb40 0x7f8c681017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:14.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.553+0000 7f8c66ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c680ffb40 0x7f8c681017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58884/0 (socket says 172.21.15.71:58884) 2024-04-23T02:23:14.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.553+0000 7f8c66ffd700 1 -- 172.21.15.71:0/1659167921 learned_addr learned my addr 172.21.15.71:0/1659167921 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:23:14.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.553+0000 7f8c667fc700 1 --2- 172.21.15.71:0/1659167921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c68105bf0 0x7f8c68101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:14.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.553+0000 7f8c66ffd700 1 -- 172.21.15.71:0/1659167921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c68105bf0 msgr2=0x7f8c68101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:14.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.553+0000 7f8c66ffd700 1 --2- 172.21.15.71:0/1659167921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c68105bf0 0x7f8c68101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:14.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.553+0000 7f8c66ffd700 1 -- 172.21.15.71:0/1659167921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c60009d00 con 0x7f8c680ffb40 2024-04-23T02:23:14.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.553+0000 7f8c66ffd700 1 --2- 172.21.15.71:0/1659167921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c680ffb40 0x7f8c681017b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000d010 tx=0x7f8c60007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:14.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.553+0000 7f8c4ffff700 1 -- 172.21.15.71:0/1659167921 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c6000e040 con 0x7f8c680ffb40 2024-04-23T02:23:14.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.553+0000 7f8c4ffff700 1 -- 172.21.15.71:0/1659167921 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c60003e50 con 0x7f8c680ffb40 2024-04-23T02:23:14.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.553+0000 7f8c6d3e8700 1 -- 172.21.15.71:0/1659167921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c681025f0 con 0x7f8c680ffb40 2024-04-23T02:23:14.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.553+0000 7f8c4ffff700 1 -- 172.21.15.71:0/1659167921 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c6001da10 con 0x7f8c680ffb40 2024-04-23T02:23:14.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.553+0000 7f8c6d3e8700 1 -- 172.21.15.71:0/1659167921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c681a2cb0 con 0x7f8c680ffb40 2024-04-23T02:23:14.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.555+0000 7f8c6d3e8700 1 -- 172.21.15.71:0/1659167921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c680679c0 con 0x7f8c680ffb40 2024-04-23T02:23:14.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.555+0000 7f8c4ffff700 1 -- 172.21.15.71:0/1659167921 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8c60004210 con 0x7f8c680ffb40 2024-04-23T02:23:14.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.555+0000 7f8c4ffff700 1 --2- 172.21.15.71:0/1659167921 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8c5006cce0 0x7f8c5006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:14.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.555+0000 7f8c4ffff700 1 -- 172.21.15.71:0/1659167921 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8c6008e0d0 con 0x7f8c680ffb40 2024-04-23T02:23:14.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.555+0000 7f8c667fc700 1 --2- 172.21.15.71:0/1659167921 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8c5006cce0 0x7f8c5006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:14.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.556+0000 7f8c667fc700 1 --2- 172.21.15.71:0/1659167921 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8c5006cce0 0x7f8c5006f1a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c006fd0 tx=0x7f8c5c008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:14.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.560+0000 7f8c4ffff700 1 -- 172.21.15.71:0/1659167921 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8c60058140 con 0x7f8c680ffb40 2024-04-23T02:23:14.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.767+0000 7f8c6d3e8700 1 -- 172.21.15.71:0/1659167921 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c68069640 con 0x7f8c5006cce0 2024-04-23T02:23:14.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.769+0000 7f8c4ffff700 1 -- 172.21.15.71:0/1659167921 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8c68069640 con 0x7f8c5006cce0 2024-04-23T02:23:14.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.771+0000 7f8c6d3e8700 1 -- 172.21.15.71:0/1659167921 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8c5006cce0 msgr2=0x7f8c5006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:14.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.771+0000 7f8c6d3e8700 1 --2- 172.21.15.71:0/1659167921 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8c5006cce0 0x7f8c5006f1a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c006fd0 tx=0x7f8c5c008040 comp rx=0 tx=0).stop 2024-04-23T02:23:14.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.771+0000 7f8c6d3e8700 1 -- 172.21.15.71:0/1659167921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c680ffb40 msgr2=0x7f8c681017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:14.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.771+0000 7f8c6d3e8700 1 --2- 172.21.15.71:0/1659167921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c680ffb40 0x7f8c681017b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000d010 tx=0x7f8c60007be0 comp rx=0 tx=0).stop 2024-04-23T02:23:14.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.771+0000 7f8c6d3e8700 1 -- 172.21.15.71:0/1659167921 shutdown_connections 2024-04-23T02:23:14.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.771+0000 7f8c6d3e8700 1 --2- 172.21.15.71:0/1659167921 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8c5006cce0 0x7f8c5006f1a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:14.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.771+0000 7f8c6d3e8700 1 --2- 172.21.15.71:0/1659167921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c680ffb40 0x7f8c681017b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:14.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.771+0000 7f8c6d3e8700 1 --2- 172.21.15.71:0/1659167921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c68105bf0 0x7f8c68101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:14.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.771+0000 7f8c6d3e8700 1 -- 172.21.15.71:0/1659167921 >> 172.21.15.71:0/1659167921 conn(0x7f8c680fb610 msgr2=0x7f8c68068f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:14.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.771+0000 7f8c6d3e8700 1 -- 172.21.15.71:0/1659167921 shutdown_connections 2024-04-23T02:23:14.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.771+0000 7f8c6d3e8700 1 -- 172.21.15.71:0/1659167921 wait complete. 2024-04-23T02:23:14.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.875+0000 7fbc2765b700 1 -- 172.21.15.71:0/3843511612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc20101850 msgr2=0x7fbc201058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:14.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.875+0000 7fbc2765b700 1 --2- 172.21.15.71:0/3843511612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc20101850 0x7fbc201058a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009910 tx=0x7fbc10009c20 comp rx=0 tx=0).stop 2024-04-23T02:23:14.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.875+0000 7fbc2765b700 1 -- 172.21.15.71:0/3843511612 shutdown_connections 2024-04-23T02:23:14.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.875+0000 7fbc2765b700 1 --2- 172.21.15.71:0/3843511612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc20101850 0x7fbc201058a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:14.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.875+0000 7fbc2765b700 1 --2- 172.21.15.71:0/3843511612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc20100ea0 0x7fbc20101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:14.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.875+0000 7fbc2765b700 1 -- 172.21.15.71:0/3843511612 >> 172.21.15.71:0/3843511612 conn(0x7fbc200fc750 msgr2=0x7fbc200feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:14.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.875+0000 7fbc2765b700 1 -- 172.21.15.71:0/3843511612 shutdown_connections 2024-04-23T02:23:14.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.875+0000 7fbc2765b700 1 -- 172.21.15.71:0/3843511612 wait complete. 2024-04-23T02:23:14.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.876+0000 7fbc2765b700 1 Processor -- start 2024-04-23T02:23:14.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.876+0000 7fbc2765b700 1 -- start start 2024-04-23T02:23:14.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.876+0000 7fbc2765b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc20100ea0 0x7fbc20073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:14.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.876+0000 7fbc2765b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc20101850 0x7fbc200735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:14.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.876+0000 7fbc2765b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc20073b10 con 0x7fbc20101850 2024-04-23T02:23:14.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.876+0000 7fbc2765b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc20073c50 con 0x7fbc20100ea0 2024-04-23T02:23:14.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.876+0000 7fbc24bf6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc20101850 0x7fbc200735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:14.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.876+0000 7fbc24bf6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc20101850 0x7fbc200735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58898/0 (socket says 172.21.15.71:58898) 2024-04-23T02:23:14.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.876+0000 7fbc24bf6700 1 -- 172.21.15.71:0/4081870274 learned_addr learned my addr 172.21.15.71:0/4081870274 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:23:14.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.876+0000 7fbc253f7700 1 --2- 172.21.15.71:0/4081870274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc20100ea0 0x7fbc20073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:14.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.876+0000 7fbc24bf6700 1 -- 172.21.15.71:0/4081870274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc20100ea0 msgr2=0x7fbc20073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:14.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.876+0000 7fbc24bf6700 1 --2- 172.21.15.71:0/4081870274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc20100ea0 0x7fbc20073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:14.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.876+0000 7fbc24bf6700 1 -- 172.21.15.71:0/4081870274 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc100095f0 con 0x7fbc20101850 2024-04-23T02:23:14.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.876+0000 7fbc24bf6700 1 --2- 172.21.15.71:0/4081870274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc20101850 0x7fbc200735d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000bd70 tx=0x7fbc1000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:14.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.876+0000 7fbc167fc700 1 -- 172.21.15.71:0/4081870274 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc1001d070 con 0x7fbc20101850 2024-04-23T02:23:14.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.876+0000 7fbc167fc700 1 -- 172.21.15.71:0/4081870274 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbc1000f460 con 0x7fbc20101850 2024-04-23T02:23:14.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.876+0000 7fbc2765b700 1 -- 172.21.15.71:0/4081870274 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc20073ed0 con 0x7fbc20101850 2024-04-23T02:23:14.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.876+0000 7fbc167fc700 1 -- 172.21.15.71:0/4081870274 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc10021640 con 0x7fbc20101850 2024-04-23T02:23:14.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.877+0000 7fbc2765b700 1 -- 172.21.15.71:0/4081870274 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc201a2d50 con 0x7fbc20101850 2024-04-23T02:23:14.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.877+0000 7fbc2765b700 1 -- 172.21.15.71:0/4081870274 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc201092a0 con 0x7fbc20101850 2024-04-23T02:23:14.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.878+0000 7fbc167fc700 1 -- 172.21.15.71:0/4081870274 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbc1000f9e0 con 0x7fbc20101850 2024-04-23T02:23:14.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.878+0000 7fbc167fc700 1 --2- 172.21.15.71:0/4081870274 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbc0c06cce0 0x7fbc0c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:14.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.878+0000 7fbc167fc700 1 -- 172.21.15.71:0/4081870274 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbc1008d6b0 con 0x7fbc20101850 2024-04-23T02:23:14.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.878+0000 7fbc253f7700 1 --2- 172.21.15.71:0/4081870274 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbc0c06cce0 0x7fbc0c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:14.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.879+0000 7fbc253f7700 1 --2- 172.21.15.71:0/4081870274 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbc0c06cce0 0x7fbc0c06f1a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c005950 tx=0x7fbc1c00b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:14.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:14.883+0000 7fbc167fc700 1 -- 172.21.15.71:0/4081870274 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbc10057720 con 0x7fbc20101850 2024-04-23T02:23:15.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.084+0000 7fbc2765b700 1 -- 172.21.15.71:0/4081870274 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc200746d0 con 0x7fbc0c06cce0 2024-04-23T02:23:15.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.094+0000 7fbc167fc700 1 -- 172.21.15.71:0/4081870274 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fbc200746d0 con 0x7fbc0c06cce0 2024-04-23T02:23:15.098 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:23:15.098 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (6m) 39s ago 18m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:23:15.098 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (18m) 39s ago 18m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:23:15.098 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (17m) 22s ago 17m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:23:15.098 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (4m) 39s ago 18m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:23:15.098 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (4m) 22s ago 17m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:23:15.098 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (6m) 39s ago 17m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:23:15.098 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (66s) 39s ago 12m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:23:15.098 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (46s) 39s ago 12m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:23:15.098 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (12m) 22s ago 12m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:23:15.098 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (25s) 22s ago 12m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:23:15.098 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (8m) 39s ago 19m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:23:15.099 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (8m) 22s ago 17m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:23:15.099 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (5m) 39s ago 20m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:23:15.099 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (5m) 22s ago 17m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:23:15.099 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (7m) 39s ago 18m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:23:15.099 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (7m) 22s ago 17m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:23:15.099 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (4m) 39s ago 16m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:23:15.099 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (3m) 39s ago 15m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:23:15.099 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (3m) 39s ago 15m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:23:15.099 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (2m) 22s ago 14m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:23:15.099 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (119s) 22s ago 14m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:23:15.099 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (84s) 22s ago 13m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:23:15.099 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (7m) 39s ago 17m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:23:15.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.096+0000 7fbc2765b700 1 -- 172.21.15.71:0/4081870274 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbc0c06cce0 msgr2=0x7fbc0c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:15.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.096+0000 7fbc2765b700 1 --2- 172.21.15.71:0/4081870274 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbc0c06cce0 0x7fbc0c06f1a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c005950 tx=0x7fbc1c00b410 comp rx=0 tx=0).stop 2024-04-23T02:23:15.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.096+0000 7fbc2765b700 1 -- 172.21.15.71:0/4081870274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc20101850 msgr2=0x7fbc200735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:15.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.096+0000 7fbc2765b700 1 --2- 172.21.15.71:0/4081870274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc20101850 0x7fbc200735d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000bd70 tx=0x7fbc1000bda0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.096+0000 7fbc2765b700 1 -- 172.21.15.71:0/4081870274 shutdown_connections 2024-04-23T02:23:15.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.096+0000 7fbc2765b700 1 --2- 172.21.15.71:0/4081870274 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbc0c06cce0 0x7fbc0c06f1a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.096+0000 7fbc2765b700 1 --2- 172.21.15.71:0/4081870274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc20100ea0 0x7fbc20073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.096+0000 7fbc2765b700 1 --2- 172.21.15.71:0/4081870274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc20101850 0x7fbc200735d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.096+0000 7fbc2765b700 1 -- 172.21.15.71:0/4081870274 >> 172.21.15.71:0/4081870274 conn(0x7fbc200fc750 msgr2=0x7fbc200fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:15.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.097+0000 7fbc2765b700 1 -- 172.21.15.71:0/4081870274 shutdown_connections 2024-04-23T02:23:15.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.097+0000 7fbc2765b700 1 -- 172.21.15.71:0/4081870274 wait complete. 2024-04-23T02:23:15.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.198+0000 7fab718ca700 1 -- 172.21.15.71:0/2693639141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab6c0ffda0 msgr2=0x7fab6c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:15.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.198+0000 7fab718ca700 1 --2- 172.21.15.71:0/2693639141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab6c0ffda0 0x7fab6c100180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009910 tx=0x7fab5c009c20 comp rx=0 tx=0).stop 2024-04-23T02:23:15.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.198+0000 7fab718ca700 1 -- 172.21.15.71:0/2693639141 shutdown_connections 2024-04-23T02:23:15.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.198+0000 7fab718ca700 1 --2- 172.21.15.71:0/2693639141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab6c100750 0x7fab6c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.198+0000 7fab718ca700 1 --2- 172.21.15.71:0/2693639141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab6c0ffda0 0x7fab6c100180 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.198+0000 7fab718ca700 1 -- 172.21.15.71:0/2693639141 >> 172.21.15.71:0/2693639141 conn(0x7fab6c0fb650 msgr2=0x7fab6c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:15.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.198+0000 7fab718ca700 1 -- 172.21.15.71:0/2693639141 shutdown_connections 2024-04-23T02:23:15.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.198+0000 7fab718ca700 1 -- 172.21.15.71:0/2693639141 wait complete. 2024-04-23T02:23:15.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.199+0000 7fab718ca700 1 Processor -- start 2024-04-23T02:23:15.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.199+0000 7fab718ca700 1 -- start start 2024-04-23T02:23:15.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.200+0000 7fab718ca700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab6c0ffda0 0x7fab6c197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:15.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.200+0000 7fab718ca700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab6c100750 0x7fab6c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:15.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.200+0000 7fab718ca700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c198180 con 0x7fab6c0ffda0 2024-04-23T02:23:15.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.200+0000 7fab718ca700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c1982f0 con 0x7fab6c100750 2024-04-23T02:23:15.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.200+0000 7fab6affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab6c0ffda0 0x7fab6c197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:15.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.200+0000 7fab6affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab6c0ffda0 0x7fab6c197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58908/0 (socket says 172.21.15.71:58908) 2024-04-23T02:23:15.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.200+0000 7fab6affd700 1 -- 172.21.15.71:0/980120804 learned_addr learned my addr 172.21.15.71:0/980120804 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:23:15.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.200+0000 7fab6a7fc700 1 --2- 172.21.15.71:0/980120804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab6c100750 0x7fab6c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:15.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.200+0000 7fab6affd700 1 -- 172.21.15.71:0/980120804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab6c100750 msgr2=0x7fab6c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:15.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.200+0000 7fab6affd700 1 --2- 172.21.15.71:0/980120804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab6c100750 0x7fab6c197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.200+0000 7fab6affd700 1 -- 172.21.15.71:0/980120804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c0095f0 con 0x7fab6c0ffda0 2024-04-23T02:23:15.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.200+0000 7fab6affd700 1 --2- 172.21.15.71:0/980120804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab6c0ffda0 0x7fab6c197620 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fab5c006010 tx=0x7fab5c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:15.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.200+0000 7fab708c8700 1 -- 172.21.15.71:0/980120804 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab5c01d070 con 0x7fab6c0ffda0 2024-04-23T02:23:15.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.200+0000 7fab718ca700 1 -- 172.21.15.71:0/980120804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c1a0690 con 0x7fab6c0ffda0 2024-04-23T02:23:15.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.200+0000 7fab708c8700 1 -- 172.21.15.71:0/980120804 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fab5c005940 con 0x7fab6c0ffda0 2024-04-23T02:23:15.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.200+0000 7fab708c8700 1 -- 172.21.15.71:0/980120804 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab5c00fd90 con 0x7fab6c0ffda0 2024-04-23T02:23:15.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.201+0000 7fab718ca700 1 -- 172.21.15.71:0/980120804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c1a0b00 con 0x7fab6c0ffda0 2024-04-23T02:23:15.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.201+0000 7fab718ca700 1 -- 172.21.15.71:0/980120804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6c108140 con 0x7fab6c0ffda0 2024-04-23T02:23:15.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.202+0000 7fab708c8700 1 -- 172.21.15.71:0/980120804 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fab5c003a60 con 0x7fab6c0ffda0 2024-04-23T02:23:15.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.202+0000 7fab708c8700 1 --2- 172.21.15.71:0/980120804 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fab5406cce0 0x7fab5406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:15.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.202+0000 7fab708c8700 1 -- 172.21.15.71:0/980120804 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fab5c08d740 con 0x7fab6c0ffda0 2024-04-23T02:23:15.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.202+0000 7fab6a7fc700 1 --2- 172.21.15.71:0/980120804 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fab5406cce0 0x7fab5406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:15.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.203+0000 7fab6a7fc700 1 --2- 172.21.15.71:0/980120804 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fab5406cce0 0x7fab5406f1a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0696e0 tx=0x7fab6000b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:15.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.207+0000 7fab708c8700 1 -- 172.21.15.71:0/980120804 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fab5c0577b0 con 0x7fab6c0ffda0 2024-04-23T02:23:15.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.478+0000 7fab718ca700 1 -- 172.21.15.71:0/980120804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fab6c02cc70 con 0x7fab6c0ffda0 2024-04-23T02:23:15.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.478+0000 7fab708c8700 1 -- 172.21.15.71:0/980120804 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7fab5c05add0 con 0x7fab6c0ffda0 2024-04-23T02:23:15.481 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:23:15.481 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:23:15.482 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:23:15.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:23:15.482 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:23:15.482 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:23:15.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:23:15.482 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:23:15.482 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:23:15.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:23:15.482 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:23:15.482 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:23:15.482 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:23:15.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:23:15.483 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:23:15.483 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:23:15.483 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:23:15.483 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:23:15.483 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:23:15.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.480+0000 7fab718ca700 1 -- 172.21.15.71:0/980120804 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fab5406cce0 msgr2=0x7fab5406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:15.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.480+0000 7fab718ca700 1 --2- 172.21.15.71:0/980120804 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fab5406cce0 0x7fab5406f1a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0696e0 tx=0x7fab6000b410 comp rx=0 tx=0).stop 2024-04-23T02:23:15.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.480+0000 7fab718ca700 1 -- 172.21.15.71:0/980120804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab6c0ffda0 msgr2=0x7fab6c197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:15.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.480+0000 7fab718ca700 1 --2- 172.21.15.71:0/980120804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab6c0ffda0 0x7fab6c197620 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fab5c006010 tx=0x7fab5c00bd70 comp rx=0 tx=0).stop 2024-04-23T02:23:15.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.480+0000 7fab718ca700 1 -- 172.21.15.71:0/980120804 shutdown_connections 2024-04-23T02:23:15.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.480+0000 7fab718ca700 1 --2- 172.21.15.71:0/980120804 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fab5406cce0 0x7fab5406f1a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.480+0000 7fab718ca700 1 --2- 172.21.15.71:0/980120804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab6c0ffda0 0x7fab6c197620 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.480+0000 7fab718ca700 1 --2- 172.21.15.71:0/980120804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab6c100750 0x7fab6c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.480+0000 7fab718ca700 1 -- 172.21.15.71:0/980120804 >> 172.21.15.71:0/980120804 conn(0x7fab6c0fb650 msgr2=0x7fab6c0fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:15.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.480+0000 7fab718ca700 1 -- 172.21.15.71:0/980120804 shutdown_connections 2024-04-23T02:23:15.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.480+0000 7fab718ca700 1 -- 172.21.15.71:0/980120804 wait complete. 2024-04-23T02:23:15.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.584+0000 7fd2dd88a700 1 -- 172.21.15.71:0/790462873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d8102700 msgr2=0x7fd2d810abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:15.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.584+0000 7fd2dd88a700 1 --2- 172.21.15.71:0/790462873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d8102700 0x7fd2d810abf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009910 tx=0x7fd2c8009c20 comp rx=0 tx=0).stop 2024-04-23T02:23:15.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.584+0000 7fd2dd88a700 1 -- 172.21.15.71:0/790462873 shutdown_connections 2024-04-23T02:23:15.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.584+0000 7fd2dd88a700 1 --2- 172.21.15.71:0/790462873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d8102700 0x7fd2d810abf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.584+0000 7fd2dd88a700 1 --2- 172.21.15.71:0/790462873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2d8101de0 0x7fd2d81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.584+0000 7fd2dd88a700 1 -- 172.21.15.71:0/790462873 >> 172.21.15.71:0/790462873 conn(0x7fd2d80fb610 msgr2=0x7fd2d80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:15.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.584+0000 7fd2dd88a700 1 -- 172.21.15.71:0/790462873 shutdown_connections 2024-04-23T02:23:15.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.584+0000 7fd2dd88a700 1 -- 172.21.15.71:0/790462873 wait complete. 2024-04-23T02:23:15.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.584+0000 7fd2dd88a700 1 Processor -- start 2024-04-23T02:23:15.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.585+0000 7fd2dd88a700 1 -- start start 2024-04-23T02:23:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.585+0000 7fd2dd88a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2d8101de0 0x7fd2d819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.585+0000 7fd2dd88a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d8102700 0x7fd2d819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.585+0000 7fd2dd88a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d819d190 con 0x7fd2d8102700 2024-04-23T02:23:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.585+0000 7fd2dd88a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8196640 con 0x7fd2d8101de0 2024-04-23T02:23:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.585+0000 7fd2d67fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d8102700 0x7fd2d819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.585+0000 7fd2d67fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d8102700 0x7fd2d819cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36996/0 (socket says 172.21.15.71:36996) 2024-04-23T02:23:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.585+0000 7fd2d67fc700 1 -- 172.21.15.71:0/2749473590 learned_addr learned my addr 172.21.15.71:0/2749473590 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:23:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.585+0000 7fd2d6ffd700 1 --2- 172.21.15.71:0/2749473590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2d8101de0 0x7fd2d819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.585+0000 7fd2d67fc700 1 -- 172.21.15.71:0/2749473590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2d8101de0 msgr2=0x7fd2d819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.585+0000 7fd2d67fc700 1 --2- 172.21.15.71:0/2749473590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2d8101de0 0x7fd2d819c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.585+0000 7fd2d67fc700 1 -- 172.21.15.71:0/2749473590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c80095f0 con 0x7fd2d8102700 2024-04-23T02:23:15.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.585+0000 7fd2d67fc700 1 --2- 172.21.15.71:0/2749473590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d8102700 0x7fd2d819cb00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80092e0 tx=0x7fd2c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.586+0000 7fd2dc888700 1 -- 172.21.15.71:0/2749473590 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2c801d070 con 0x7fd2d8102700 2024-04-23T02:23:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.586+0000 7fd2dc888700 1 -- 172.21.15.71:0/2749473590 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd2c8003d30 con 0x7fd2d8102700 2024-04-23T02:23:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.586+0000 7fd2dd88a700 1 -- 172.21.15.71:0/2749473590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d81968c0 con 0x7fd2d8102700 2024-04-23T02:23:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.586+0000 7fd2dc888700 1 -- 172.21.15.71:0/2749473590 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2c8021410 con 0x7fd2d8102700 2024-04-23T02:23:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.586+0000 7fd2dd88a700 1 -- 172.21.15.71:0/2749473590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d8196db0 con 0x7fd2d8102700 2024-04-23T02:23:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.587+0000 7fd2dd88a700 1 -- 172.21.15.71:0/2749473590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d81082f0 con 0x7fd2d8102700 2024-04-23T02:23:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.587+0000 7fd2dc888700 1 -- 172.21.15.71:0/2749473590 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd2c800f6f0 con 0x7fd2d8102700 2024-04-23T02:23:15.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.587+0000 7fd2dc888700 1 --2- 172.21.15.71:0/2749473590 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd2c006cce0 0x7fd2c006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:15.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.588+0000 7fd2dc888700 1 -- 172.21.15.71:0/2749473590 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd2c808d8b0 con 0x7fd2d8102700 2024-04-23T02:23:15.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.588+0000 7fd2d6ffd700 1 --2- 172.21.15.71:0/2749473590 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd2c006cce0 0x7fd2c006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:15.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.588+0000 7fd2d6ffd700 1 --2- 172.21.15.71:0/2749473590 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd2c006cce0 0x7fd2c006f1a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc005950 tx=0x7fd2cc0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:15.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.592+0000 7fd2dc888700 1 -- 172.21.15.71:0/2749473590 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd2c805b310 con 0x7fd2d8102700 2024-04-23T02:23:15.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.824+0000 7fd2dd88a700 1 -- 172.21.15.71:0/2749473590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd2d804f2e0 con 0x7fd2d8102700 2024-04-23T02:23:15.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.825+0000 7fd2dc888700 1 -- 172.21.15.71:0/2749473590 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7fd2c805aea0 con 0x7fd2d8102700 2024-04-23T02:23:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:23:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:23:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:23:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:23:15.830 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:23:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:23:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:23:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:23:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:23:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:23:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:23:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:23:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:23:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:23:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:23:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:23:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:23:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:23:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:23:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:23:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:23:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:23:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:23:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:23:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:23:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:23:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:23:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:23:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:23:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:23:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:23:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:23:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:23:15.832 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:23:15.832 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:23:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:23:15.832 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:23:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:23:15.833 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:23:15.833 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:23:15.833 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:23:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.827+0000 7fd2dd88a700 1 -- 172.21.15.71:0/2749473590 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd2c006cce0 msgr2=0x7fd2c006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.827+0000 7fd2dd88a700 1 --2- 172.21.15.71:0/2749473590 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd2c006cce0 0x7fd2c006f1a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc005950 tx=0x7fd2cc0058e0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.827+0000 7fd2dd88a700 1 -- 172.21.15.71:0/2749473590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d8102700 msgr2=0x7fd2d819cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:15.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.827+0000 7fd2dd88a700 1 --2- 172.21.15.71:0/2749473590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d8102700 0x7fd2d819cb00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80092e0 tx=0x7fd2c800be20 comp rx=0 tx=0).stop 2024-04-23T02:23:15.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.827+0000 7fd2dd88a700 1 -- 172.21.15.71:0/2749473590 shutdown_connections 2024-04-23T02:23:15.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.827+0000 7fd2dd88a700 1 --2- 172.21.15.71:0/2749473590 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd2c006cce0 0x7fd2c006f1a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.827+0000 7fd2dd88a700 1 --2- 172.21.15.71:0/2749473590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2d8101de0 0x7fd2d819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.827+0000 7fd2dd88a700 1 --2- 172.21.15.71:0/2749473590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d8102700 0x7fd2d819cb00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.827+0000 7fd2dd88a700 1 -- 172.21.15.71:0/2749473590 >> 172.21.15.71:0/2749473590 conn(0x7fd2d80fb610 msgr2=0x7fd2d8105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:15.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.827+0000 7fd2dd88a700 1 -- 172.21.15.71:0/2749473590 shutdown_connections 2024-04-23T02:23:15.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.827+0000 7fd2dd88a700 1 -- 172.21.15.71:0/2749473590 wait complete. 2024-04-23T02:23:15.834 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:23:15.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.937+0000 7f79e80d4700 1 -- 172.21.15.71:0/1012131358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e0068580 msgr2=0x7f79e00689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:15.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.937+0000 7f79e80d4700 1 --2- 172.21.15.71:0/1012131358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e0068580 0x7f79e00689e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009910 tx=0x7f79dc009c20 comp rx=0 tx=0).stop 2024-04-23T02:23:15.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.937+0000 7f79e80d4700 1 -- 172.21.15.71:0/1012131358 shutdown_connections 2024-04-23T02:23:15.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.937+0000 7f79e80d4700 1 --2- 172.21.15.71:0/1012131358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e0068580 0x7f79e00689e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.937+0000 7f79e80d4700 1 --2- 172.21.15.71:0/1012131358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79e0106cd0 0x7f79e01070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.937+0000 7f79e80d4700 1 -- 172.21.15.71:0/1012131358 >> 172.21.15.71:0/1012131358 conn(0x7f79e0075670 msgr2=0x7f79e0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:15.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.937+0000 7f79e80d4700 1 -- 172.21.15.71:0/1012131358 shutdown_connections 2024-04-23T02:23:15.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.937+0000 7f79e80d4700 1 -- 172.21.15.71:0/1012131358 wait complete. 2024-04-23T02:23:15.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.938+0000 7f79e80d4700 1 Processor -- start 2024-04-23T02:23:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.938+0000 7f79e80d4700 1 -- start start 2024-04-23T02:23:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.938+0000 7f79e80d4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79e0068580 0x7f79e019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.938+0000 7f79e80d4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e0106cd0 0x7f79e019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.938+0000 7f79e80d4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e019d110 con 0x7f79e0106cd0 2024-04-23T02:23:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.938+0000 7f79e80d4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e01965c0 con 0x7f79e0068580 2024-04-23T02:23:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.938+0000 7f79e566f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e0106cd0 0x7f79e019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.938+0000 7f79e566f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e0106cd0 0x7f79e019ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37016/0 (socket says 172.21.15.71:37016) 2024-04-23T02:23:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.938+0000 7f79e566f700 1 -- 172.21.15.71:0/2859706461 learned_addr learned my addr 172.21.15.71:0/2859706461 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:23:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.938+0000 7f79e566f700 1 -- 172.21.15.71:0/2859706461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79e0068580 msgr2=0x7f79e019c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.938+0000 7f79e5e70700 1 --2- 172.21.15.71:0/2859706461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79e0068580 0x7f79e019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.938+0000 7f79e566f700 1 --2- 172.21.15.71:0/2859706461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79e0068580 0x7f79e019c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.938+0000 7f79e566f700 1 -- 172.21.15.71:0/2859706461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc0095f0 con 0x7f79e0106cd0 2024-04-23T02:23:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.938+0000 7f79e5e70700 1 --2- 172.21.15.71:0/2859706461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79e0068580 0x7f79e019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:23:15.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.938+0000 7f79e566f700 1 --2- 172.21.15.71:0/2859706461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e0106cd0 0x7f79e019ca80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f79dc006010 tx=0x7f79dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:15.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.939+0000 7f79d6ffd700 1 -- 172.21.15.71:0/2859706461 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79dc01d070 con 0x7f79e0106cd0 2024-04-23T02:23:15.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.939+0000 7f79d6ffd700 1 -- 172.21.15.71:0/2859706461 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f79dc00f460 con 0x7f79e0106cd0 2024-04-23T02:23:15.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.939+0000 7f79e80d4700 1 -- 172.21.15.71:0/2859706461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e0196840 con 0x7f79e0106cd0 2024-04-23T02:23:15.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.939+0000 7f79d6ffd700 1 -- 172.21.15.71:0/2859706461 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79dc021640 con 0x7f79e0106cd0 2024-04-23T02:23:15.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.939+0000 7f79e80d4700 1 -- 172.21.15.71:0/2859706461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e0196d30 con 0x7f79e0106cd0 2024-04-23T02:23:15.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.940+0000 7f79e80d4700 1 -- 172.21.15.71:0/2859706461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e01091d0 con 0x7f79e0106cd0 2024-04-23T02:23:15.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.940+0000 7f79d6ffd700 1 -- 172.21.15.71:0/2859706461 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f79dc00fa60 con 0x7f79e0106cd0 2024-04-23T02:23:15.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.940+0000 7f79d6ffd700 1 --2- 172.21.15.71:0/2859706461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f79cc06cce0 0x7f79cc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:15.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.940+0000 7f79d6ffd700 1 -- 172.21.15.71:0/2859706461 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f79dc08d880 con 0x7f79e0106cd0 2024-04-23T02:23:15.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.941+0000 7f79e5e70700 1 --2- 172.21.15.71:0/2859706461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f79cc06cce0 0x7f79cc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:15.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.941+0000 7f79e5e70700 1 --2- 172.21.15.71:0/2859706461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f79cc06cce0 0x7f79cc06f1a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f79d0005fd0 tx=0x7f79d0005dc0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:15.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:15.945+0000 7f79d6ffd700 1 -- 172.21.15.71:0/2859706461 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f79dc05b380 con 0x7f79e0106cd0 2024-04-23T02:23:16.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.154+0000 7f79e80d4700 1 -- 172.21.15.71:0/2859706461 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f79e0196f50 con 0x7f79cc06cce0 2024-04-23T02:23:16.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.155+0000 7f79d6ffd700 1 -- 172.21.15.71:0/2859706461 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f79e0196f50 con 0x7f79cc06cce0 2024-04-23T02:23:16.158 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:23:16.158 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:23:16.158 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:23:16.159 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:23:16.159 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:23:16.159 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:23:16.159 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:23:16.159 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:23:16.159 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:23:16.159 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:23:16.159 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:23:16.159 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:23:16.159 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:23:16.159 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:23:16.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.157+0000 7f79e80d4700 1 -- 172.21.15.71:0/2859706461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f79cc06cce0 msgr2=0x7f79cc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:16.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.157+0000 7f79e80d4700 1 --2- 172.21.15.71:0/2859706461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f79cc06cce0 0x7f79cc06f1a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f79d0005fd0 tx=0x7f79d0005dc0 comp rx=0 tx=0).stop 2024-04-23T02:23:16.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.157+0000 7f79e80d4700 1 -- 172.21.15.71:0/2859706461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e0106cd0 msgr2=0x7f79e019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:16.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.157+0000 7f79e80d4700 1 --2- 172.21.15.71:0/2859706461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e0106cd0 0x7f79e019ca80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f79dc006010 tx=0x7f79dc00be20 comp rx=0 tx=0).stop 2024-04-23T02:23:16.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.157+0000 7f79e80d4700 1 -- 172.21.15.71:0/2859706461 shutdown_connections 2024-04-23T02:23:16.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.157+0000 7f79e80d4700 1 --2- 172.21.15.71:0/2859706461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f79cc06cce0 0x7f79cc06f1a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:16.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.157+0000 7f79e80d4700 1 --2- 172.21.15.71:0/2859706461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79e0068580 0x7f79e019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:16.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.157+0000 7f79e80d4700 1 --2- 172.21.15.71:0/2859706461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e0106cd0 0x7f79e019ca80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:16.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.157+0000 7f79e80d4700 1 -- 172.21.15.71:0/2859706461 >> 172.21.15.71:0/2859706461 conn(0x7f79e0075670 msgr2=0x7f79e00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:16.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.157+0000 7f79e80d4700 1 -- 172.21.15.71:0/2859706461 shutdown_connections 2024-04-23T02:23:16.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.157+0000 7f79e80d4700 1 -- 172.21.15.71:0/2859706461 wait complete. 2024-04-23T02:23:16.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:15 smithi071 ceph-mon[144544]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:23:16.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:15 smithi071 ceph-mon[144544]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:23:16.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:15 smithi071 ceph-mon[144544]: pgmap v298: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:16.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:15 smithi071 ceph-mon[144544]: from='client.34406 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:23:16.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:15 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/980120804' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:23:16.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:15 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2749473590' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:23:16.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.260+0000 7efedb4c9700 1 -- 172.21.15.71:0/2381555551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed4101840 msgr2=0x7efed4105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:16.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.260+0000 7efedb4c9700 1 --2- 172.21.15.71:0/2381555551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed4101840 0x7efed4105890 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efed0009910 tx=0x7efed0009c20 comp rx=0 tx=0).stop 2024-04-23T02:23:16.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.260+0000 7efedb4c9700 1 -- 172.21.15.71:0/2381555551 shutdown_connections 2024-04-23T02:23:16.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.260+0000 7efedb4c9700 1 --2- 172.21.15.71:0/2381555551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed4101840 0x7efed4105890 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:16.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.260+0000 7efedb4c9700 1 --2- 172.21.15.71:0/2381555551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efed4100e90 0x7efed4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:16.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.260+0000 7efedb4c9700 1 -- 172.21.15.71:0/2381555551 >> 172.21.15.71:0/2381555551 conn(0x7efed40fc740 msgr2=0x7efed40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:16.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.260+0000 7efedb4c9700 1 -- 172.21.15.71:0/2381555551 shutdown_connections 2024-04-23T02:23:16.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.260+0000 7efedb4c9700 1 -- 172.21.15.71:0/2381555551 wait complete. 2024-04-23T02:23:16.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.261+0000 7efedb4c9700 1 Processor -- start 2024-04-23T02:23:16.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.261+0000 7efedb4c9700 1 -- start start 2024-04-23T02:23:16.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.261+0000 7efedb4c9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed4100e90 0x7efed419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:16.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.261+0000 7efedb4c9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efed4101840 0x7efed419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:16.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.261+0000 7efedb4c9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed419d200 con 0x7efed4100e90 2024-04-23T02:23:16.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.261+0000 7efedb4c9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed41966b0 con 0x7efed4101840 2024-04-23T02:23:16.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.261+0000 7efed9265700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed4100e90 0x7efed419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:16.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.261+0000 7efed9265700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed4100e90 0x7efed419c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37036/0 (socket says 172.21.15.71:37036) 2024-04-23T02:23:16.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.261+0000 7efed9265700 1 -- 172.21.15.71:0/2516655252 learned_addr learned my addr 172.21.15.71:0/2516655252 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:23:16.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.261+0000 7efed9265700 1 -- 172.21.15.71:0/2516655252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efed4101840 msgr2=0x7efed419cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:16.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.261+0000 7efed9265700 1 --2- 172.21.15.71:0/2516655252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efed4101840 0x7efed419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:16.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.261+0000 7efed9265700 1 -- 172.21.15.71:0/2516655252 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed00095f0 con 0x7efed4100e90 2024-04-23T02:23:16.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.261+0000 7efed9265700 1 --2- 172.21.15.71:0/2516655252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed4100e90 0x7efed419c630 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efec400d8d0 tx=0x7efec400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:16.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.261+0000 7efeca7fc700 1 -- 172.21.15.71:0/2516655252 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efec4009940 con 0x7efed4100e90 2024-04-23T02:23:16.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.261+0000 7efeca7fc700 1 -- 172.21.15.71:0/2516655252 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efec4010460 con 0x7efed4100e90 2024-04-23T02:23:16.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.261+0000 7efeca7fc700 1 -- 172.21.15.71:0/2516655252 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efec400f5f0 con 0x7efed4100e90 2024-04-23T02:23:16.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.261+0000 7efedb4c9700 1 -- 172.21.15.71:0/2516655252 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed4196990 con 0x7efed4100e90 2024-04-23T02:23:16.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.262+0000 7efedb4c9700 1 -- 172.21.15.71:0/2516655252 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed4196ee0 con 0x7efed4100e90 2024-04-23T02:23:16.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.263+0000 7efedb4c9700 1 -- 172.21.15.71:0/2516655252 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed4109290 con 0x7efed4100e90 2024-04-23T02:23:16.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.263+0000 7efeca7fc700 1 -- 172.21.15.71:0/2516655252 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7efec40105d0 con 0x7efed4100e90 2024-04-23T02:23:16.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.263+0000 7efeca7fc700 1 --2- 172.21.15.71:0/2516655252 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efec006cce0 0x7efec006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:16.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.263+0000 7efeca7fc700 1 -- 172.21.15.71:0/2516655252 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7efec408c5e0 con 0x7efed4100e90 2024-04-23T02:23:16.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.264+0000 7efed8a64700 1 --2- 172.21.15.71:0/2516655252 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efec006cce0 0x7efec006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:16.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.264+0000 7efed8a64700 1 --2- 172.21.15.71:0/2516655252 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efec006cce0 0x7efec006f1a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efed000b5c0 tx=0x7efed0005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:16.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.268+0000 7efeca7fc700 1 -- 172.21.15.71:0/2516655252 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7efec405a0e0 con 0x7efed4100e90 2024-04-23T02:23:16.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:15 smithi192 ceph-mon[108617]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:23:16.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:15 smithi192 ceph-mon[108617]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:23:16.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:15 smithi192 ceph-mon[108617]: pgmap v298: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:16.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:15 smithi192 ceph-mon[108617]: from='client.34406 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:23:16.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:15 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/980120804' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:23:16.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:15 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2749473590' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:23:16.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.539+0000 7efedb4c9700 1 -- 172.21.15.71:0/2516655252 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efed4197c90 con 0x7efed4100e90 2024-04-23T02:23:16.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.539+0000 7efeca7fc700 1 -- 172.21.15.71:0/2516655252 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7efec4010880 con 0x7efed4100e90 2024-04-23T02:23:16.542 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:23:16.542 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:23:16.542 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:23:16.543 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:23:16.543 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:23:16.543 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:23:16.543 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:23:16.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.541+0000 7efedb4c9700 1 -- 172.21.15.71:0/2516655252 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efec006cce0 msgr2=0x7efec006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:16.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.541+0000 7efedb4c9700 1 --2- 172.21.15.71:0/2516655252 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efec006cce0 0x7efec006f1a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efed000b5c0 tx=0x7efed0005960 comp rx=0 tx=0).stop 2024-04-23T02:23:16.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.541+0000 7efedb4c9700 1 -- 172.21.15.71:0/2516655252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed4100e90 msgr2=0x7efed419c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:16.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.541+0000 7efedb4c9700 1 --2- 172.21.15.71:0/2516655252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed4100e90 0x7efed419c630 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efec400d8d0 tx=0x7efec400dc90 comp rx=0 tx=0).stop 2024-04-23T02:23:16.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.541+0000 7efedb4c9700 1 -- 172.21.15.71:0/2516655252 shutdown_connections 2024-04-23T02:23:16.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.541+0000 7efedb4c9700 1 --2- 172.21.15.71:0/2516655252 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efec006cce0 0x7efec006f1a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:16.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.541+0000 7efedb4c9700 1 --2- 172.21.15.71:0/2516655252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efed4100e90 0x7efed419c630 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:16.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.541+0000 7efedb4c9700 1 --2- 172.21.15.71:0/2516655252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efed4101840 0x7efed419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:16.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.541+0000 7efedb4c9700 1 -- 172.21.15.71:0/2516655252 >> 172.21.15.71:0/2516655252 conn(0x7efed40fc740 msgr2=0x7efed40fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:16.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.541+0000 7efedb4c9700 1 -- 172.21.15.71:0/2516655252 shutdown_connections 2024-04-23T02:23:16.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:16.541+0000 7efedb4c9700 1 -- 172.21.15.71:0/2516655252 wait complete. 2024-04-23T02:23:17.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:16 smithi071 ceph-mon[144544]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:23:17.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:16 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2516655252' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:23:17.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:16 smithi192 ceph-mon[108617]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:23:17.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:16 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2516655252' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:23:18.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:18 smithi192 ceph-mon[108617]: pgmap v299: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:18.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:18 smithi071 ceph-mon[144544]: pgmap v299: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:20.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:20 smithi192 ceph-mon[108617]: pgmap v300: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:20.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:23:20.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:20 smithi071 ceph-mon[144544]: pgmap v300: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:20.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:23:22.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:22 smithi192 ceph-mon[108617]: pgmap v301: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:22.632 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:22 smithi071 ceph-mon[144544]: pgmap v301: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:24.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:24 smithi192 ceph-mon[108617]: pgmap v302: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:24.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:24 smithi071 ceph-mon[144544]: pgmap v302: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:26.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:26 smithi192 ceph-mon[108617]: pgmap v303: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:26.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:26 smithi071 ceph-mon[144544]: pgmap v303: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:28.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:28 smithi071 ceph-mon[144544]: pgmap v304: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:28.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:28 smithi192 ceph-mon[108617]: pgmap v304: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:30.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:30 smithi192 ceph-mon[108617]: pgmap v305: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:30.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:30 smithi071 ceph-mon[144544]: pgmap v305: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:32.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:32 smithi192 ceph-mon[108617]: pgmap v306: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:32.632 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:32 smithi071 ceph-mon[144544]: pgmap v306: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:34.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:34 smithi192 ceph-mon[108617]: pgmap v307: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:34.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:34 smithi071 ceph-mon[144544]: pgmap v307: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:35.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:23:35.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:23:36.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:36 smithi192 ceph-mon[108617]: pgmap v308: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:36.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:36 smithi071 ceph-mon[144544]: pgmap v308: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:38.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:38 smithi071 ceph-mon[144544]: pgmap v309: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:38.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:38 smithi192 ceph-mon[108617]: pgmap v309: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:40.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:40 smithi192 ceph-mon[108617]: pgmap v310: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:40.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:40 smithi071 ceph-mon[144544]: pgmap v310: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:42.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:42 smithi192 ceph-mon[108617]: pgmap v311: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:42.633 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:42 smithi071 ceph-mon[144544]: pgmap v311: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:44.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:44 smithi192 ceph-mon[108617]: pgmap v312: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:44.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:44 smithi071 ceph-mon[144544]: pgmap v312: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:46.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:46 smithi192 ceph-mon[108617]: pgmap v313: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:46.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.658+0000 7fc5d6859700 1 -- 172.21.15.71:0/3303817399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5d01006b0 msgr2=0x7fc5d0100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:46.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.658+0000 7fc5d6859700 1 --2- 172.21.15.71:0/3303817399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5d01006b0 0x7fc5d0100a90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009910 tx=0x7fc5c0009c20 comp rx=0 tx=0).stop 2024-04-23T02:23:46.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.658+0000 7fc5d6859700 1 -- 172.21.15.71:0/3303817399 shutdown_connections 2024-04-23T02:23:46.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.658+0000 7fc5d6859700 1 --2- 172.21.15.71:0/3303817399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5d0101060 0x7fc5d01047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:46.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.658+0000 7fc5d6859700 1 --2- 172.21.15.71:0/3303817399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5d01006b0 0x7fc5d0100a90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:46.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.658+0000 7fc5d6859700 1 -- 172.21.15.71:0/3303817399 >> 172.21.15.71:0/3303817399 conn(0x7fc5d0074e30 msgr2=0x7fc5d0075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:46.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.658+0000 7fc5d6859700 1 -- 172.21.15.71:0/3303817399 shutdown_connections 2024-04-23T02:23:46.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.659+0000 7fc5d6859700 1 -- 172.21.15.71:0/3303817399 wait complete. 2024-04-23T02:23:46.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.659+0000 7fc5d6859700 1 Processor -- start 2024-04-23T02:23:46.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.659+0000 7fc5d6859700 1 -- start start 2024-04-23T02:23:46.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.660+0000 7fc5d6859700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5d01006b0 0x7fc5d0195680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:46.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.660+0000 7fc5d6859700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5d0101060 0x7fc5d0195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:46.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.660+0000 7fc5d6859700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d0196250 con 0x7fc5d0101060 2024-04-23T02:23:46.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.660+0000 7fc5d6859700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d0198f80 con 0x7fc5d01006b0 2024-04-23T02:23:46.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.660+0000 7fc5cf7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5d0101060 0x7fc5d0195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:46.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.660+0000 7fc5cf7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5d0101060 0x7fc5d0195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55870/0 (socket says 172.21.15.71:55870) 2024-04-23T02:23:46.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.660+0000 7fc5cf7fe700 1 -- 172.21.15.71:0/1527547464 learned_addr learned my addr 172.21.15.71:0/1527547464 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:23:46.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.660+0000 7fc5cffff700 1 --2- 172.21.15.71:0/1527547464 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5d01006b0 0x7fc5d0195680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:46.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.660+0000 7fc5cf7fe700 1 -- 172.21.15.71:0/1527547464 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5d01006b0 msgr2=0x7fc5d0195680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:46.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.660+0000 7fc5cf7fe700 1 --2- 172.21.15.71:0/1527547464 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5d01006b0 0x7fc5d0195680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:46.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.660+0000 7fc5cf7fe700 1 -- 172.21.15.71:0/1527547464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c00095f0 con 0x7fc5d0101060 2024-04-23T02:23:46.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.660+0000 7fc5cf7fe700 1 --2- 172.21.15.71:0/1527547464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5d0101060 0x7fc5d0195bc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800ea80 tx=0x7fc5b800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:46.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.661+0000 7fc5cd7fa700 1 -- 172.21.15.71:0/1527547464 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5b8013070 con 0x7fc5d0101060 2024-04-23T02:23:46.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.661+0000 7fc5cd7fa700 1 -- 172.21.15.71:0/1527547464 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc5b8004830 con 0x7fc5d0101060 2024-04-23T02:23:46.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.661+0000 7fc5d6859700 1 -- 172.21.15.71:0/1527547464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d0199200 con 0x7fc5d0101060 2024-04-23T02:23:46.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.661+0000 7fc5cd7fa700 1 -- 172.21.15.71:0/1527547464 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5b8005580 con 0x7fc5d0101060 2024-04-23T02:23:46.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.661+0000 7fc5d6859700 1 -- 172.21.15.71:0/1527547464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d0199750 con 0x7fc5d0101060 2024-04-23T02:23:46.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.662+0000 7fc5d6859700 1 -- 172.21.15.71:0/1527547464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d004f2e0 con 0x7fc5d0101060 2024-04-23T02:23:46.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.662+0000 7fc5cd7fa700 1 -- 172.21.15.71:0/1527547464 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc5b80049a0 con 0x7fc5d0101060 2024-04-23T02:23:46.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.663+0000 7fc5cd7fa700 1 --2- 172.21.15.71:0/1527547464 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc5bc06cd30 0x7fc5bc06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:46.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.663+0000 7fc5cd7fa700 1 -- 172.21.15.71:0/1527547464 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc5b808de20 con 0x7fc5d0101060 2024-04-23T02:23:46.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.663+0000 7fc5cffff700 1 --2- 172.21.15.71:0/1527547464 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc5bc06cd30 0x7fc5bc06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:46.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.663+0000 7fc5cffff700 1 --2- 172.21.15.71:0/1527547464 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc5bc06cd30 0x7fc5bc06f1f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009fd0 tx=0x7fc5c0005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:46.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.667+0000 7fc5cd7fa700 1 -- 172.21.15.71:0/1527547464 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc5b8057df0 con 0x7fc5d0101060 2024-04-23T02:23:46.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:46 smithi071 ceph-mon[144544]: pgmap v313: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:46.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.878+0000 7fc5d6859700 1 -- 172.21.15.71:0/1527547464 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5d01999e0 con 0x7fc5bc06cd30 2024-04-23T02:23:46.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.879+0000 7fc5cd7fa700 1 -- 172.21.15.71:0/1527547464 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc5d01999e0 con 0x7fc5bc06cd30 2024-04-23T02:23:46.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.881+0000 7fc5d6859700 1 -- 172.21.15.71:0/1527547464 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc5bc06cd30 msgr2=0x7fc5bc06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:46.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.881+0000 7fc5d6859700 1 --2- 172.21.15.71:0/1527547464 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc5bc06cd30 0x7fc5bc06f1f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009fd0 tx=0x7fc5c0005940 comp rx=0 tx=0).stop 2024-04-23T02:23:46.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.881+0000 7fc5d6859700 1 -- 172.21.15.71:0/1527547464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5d0101060 msgr2=0x7fc5d0195bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:46.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.881+0000 7fc5d6859700 1 --2- 172.21.15.71:0/1527547464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5d0101060 0x7fc5d0195bc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800ea80 tx=0x7fc5b800ed90 comp rx=0 tx=0).stop 2024-04-23T02:23:46.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.881+0000 7fc5d6859700 1 -- 172.21.15.71:0/1527547464 shutdown_connections 2024-04-23T02:23:46.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.881+0000 7fc5d6859700 1 --2- 172.21.15.71:0/1527547464 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc5bc06cd30 0x7fc5bc06f1f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:46.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.881+0000 7fc5d6859700 1 --2- 172.21.15.71:0/1527547464 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5d01006b0 0x7fc5d0195680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:46.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.881+0000 7fc5d6859700 1 --2- 172.21.15.71:0/1527547464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5d0101060 0x7fc5d0195bc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:46.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.881+0000 7fc5d6859700 1 -- 172.21.15.71:0/1527547464 >> 172.21.15.71:0/1527547464 conn(0x7fc5d0074e30 msgr2=0x7fc5d00fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:46.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.881+0000 7fc5d6859700 1 -- 172.21.15.71:0/1527547464 shutdown_connections 2024-04-23T02:23:46.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.881+0000 7fc5d6859700 1 -- 172.21.15.71:0/1527547464 wait complete. 2024-04-23T02:23:46.895 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:23:46.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.985+0000 7f6512d7c700 1 -- 172.21.15.71:0/3817662809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f650c100ea0 msgr2=0x7f650c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:46.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.985+0000 7f6512d7c700 1 --2- 172.21.15.71:0/3817662809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f650c100ea0 0x7f650c101280 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6500009910 tx=0x7f6500009c20 comp rx=0 tx=0).stop 2024-04-23T02:23:46.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.985+0000 7f6512d7c700 1 -- 172.21.15.71:0/3817662809 shutdown_connections 2024-04-23T02:23:46.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.985+0000 7f6512d7c700 1 --2- 172.21.15.71:0/3817662809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f650c101850 0x7f650c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:46.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.985+0000 7f6512d7c700 1 --2- 172.21.15.71:0/3817662809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f650c100ea0 0x7f650c101280 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:46.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.985+0000 7f6512d7c700 1 -- 172.21.15.71:0/3817662809 >> 172.21.15.71:0/3817662809 conn(0x7f650c0fc750 msgr2=0x7f650c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:46.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.985+0000 7f6512d7c700 1 -- 172.21.15.71:0/3817662809 shutdown_connections 2024-04-23T02:23:46.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.985+0000 7f6512d7c700 1 -- 172.21.15.71:0/3817662809 wait complete. 2024-04-23T02:23:46.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.986+0000 7f6512d7c700 1 Processor -- start 2024-04-23T02:23:46.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.986+0000 7f6512d7c700 1 -- start start 2024-04-23T02:23:46.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.986+0000 7f6512d7c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f650c100ea0 0x7f650c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:46.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.986+0000 7f6512d7c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f650c101850 0x7f650c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:46.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.986+0000 7f6512d7c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c073b10 con 0x7f650c100ea0 2024-04-23T02:23:46.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.986+0000 7f6512d7c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c073c50 con 0x7f650c101850 2024-04-23T02:23:46.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.986+0000 7f6510b18700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f650c100ea0 0x7f650c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:46.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.986+0000 7f6510b18700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f650c100ea0 0x7f650c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55894/0 (socket says 172.21.15.71:55894) 2024-04-23T02:23:46.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.986+0000 7f6510b18700 1 -- 172.21.15.71:0/3301119040 learned_addr learned my addr 172.21.15.71:0/3301119040 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:23:46.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.986+0000 7f650bfff700 1 --2- 172.21.15.71:0/3301119040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f650c101850 0x7f650c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:46.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.986+0000 7f6510b18700 1 -- 172.21.15.71:0/3301119040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f650c101850 msgr2=0x7f650c0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:46.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.986+0000 7f6510b18700 1 --2- 172.21.15.71:0/3301119040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f650c101850 0x7f650c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:46.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.986+0000 7f6510b18700 1 -- 172.21.15.71:0/3301119040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65000095f0 con 0x7f650c100ea0 2024-04-23T02:23:46.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.986+0000 7f6510b18700 1 --2- 172.21.15.71:0/3301119040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f650c100ea0 0x7f650c073090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f650000b5c0 tx=0x7f650000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:46.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.986+0000 7f6509ffb700 1 -- 172.21.15.71:0/3301119040 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f650001d070 con 0x7f650c100ea0 2024-04-23T02:23:46.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.987+0000 7f6509ffb700 1 -- 172.21.15.71:0/3301119040 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6500005e00 con 0x7f650c100ea0 2024-04-23T02:23:46.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.987+0000 7f6512d7c700 1 -- 172.21.15.71:0/3301119040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650c073ed0 con 0x7f650c100ea0 2024-04-23T02:23:46.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.987+0000 7f6509ffb700 1 -- 172.21.15.71:0/3301119040 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f650000fe60 con 0x7f650c100ea0 2024-04-23T02:23:46.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.987+0000 7f6512d7c700 1 -- 172.21.15.71:0/3301119040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650c1a2c90 con 0x7f650c100ea0 2024-04-23T02:23:46.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.988+0000 7f6512d7c700 1 -- 172.21.15.71:0/3301119040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650c109240 con 0x7f650c100ea0 2024-04-23T02:23:46.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.988+0000 7f6509ffb700 1 -- 172.21.15.71:0/3301119040 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f650000f6d0 con 0x7f650c100ea0 2024-04-23T02:23:46.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.989+0000 7f6509ffb700 1 --2- 172.21.15.71:0/3301119040 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f64fc06cce0 0x7f64fc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:46.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.989+0000 7f6509ffb700 1 -- 172.21.15.71:0/3301119040 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f650008d880 con 0x7f650c100ea0 2024-04-23T02:23:46.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.989+0000 7f650bfff700 1 --2- 172.21.15.71:0/3301119040 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f64fc06cce0 0x7f64fc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:46.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.989+0000 7f650bfff700 1 --2- 172.21.15.71:0/3301119040 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f64fc06cce0 0x7f64fc06f1a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f650c074880 tx=0x7f64f80094f0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:46.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:46.993+0000 7f6509ffb700 1 -- 172.21.15.71:0/3301119040 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f65000578f0 con 0x7f650c100ea0 2024-04-23T02:23:47.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.203+0000 7f6512d7c700 1 -- 172.21.15.71:0/3301119040 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f650c0611d0 con 0x7f64fc06cce0 2024-04-23T02:23:47.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.204+0000 7f6509ffb700 1 -- 172.21.15.71:0/3301119040 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f650c0611d0 con 0x7f64fc06cce0 2024-04-23T02:23:47.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.206+0000 7f6512d7c700 1 -- 172.21.15.71:0/3301119040 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f64fc06cce0 msgr2=0x7f64fc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:47.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.206+0000 7f6512d7c700 1 --2- 172.21.15.71:0/3301119040 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f64fc06cce0 0x7f64fc06f1a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f650c074880 tx=0x7f64f80094f0 comp rx=0 tx=0).stop 2024-04-23T02:23:47.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.206+0000 7f6512d7c700 1 -- 172.21.15.71:0/3301119040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f650c100ea0 msgr2=0x7f650c073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:47.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.206+0000 7f6512d7c700 1 --2- 172.21.15.71:0/3301119040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f650c100ea0 0x7f650c073090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f650000b5c0 tx=0x7f650000be20 comp rx=0 tx=0).stop 2024-04-23T02:23:47.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.206+0000 7f6512d7c700 1 -- 172.21.15.71:0/3301119040 shutdown_connections 2024-04-23T02:23:47.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.206+0000 7f6512d7c700 1 --2- 172.21.15.71:0/3301119040 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f64fc06cce0 0x7f64fc06f1a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:47.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.206+0000 7f6512d7c700 1 --2- 172.21.15.71:0/3301119040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f650c100ea0 0x7f650c073090 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:47.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.206+0000 7f6512d7c700 1 --2- 172.21.15.71:0/3301119040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f650c101850 0x7f650c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:47.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.206+0000 7f6512d7c700 1 -- 172.21.15.71:0/3301119040 >> 172.21.15.71:0/3301119040 conn(0x7f650c0fc750 msgr2=0x7f650c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:47.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.207+0000 7f6512d7c700 1 -- 172.21.15.71:0/3301119040 shutdown_connections 2024-04-23T02:23:47.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.207+0000 7f6512d7c700 1 -- 172.21.15.71:0/3301119040 wait complete. 2024-04-23T02:23:47.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.309+0000 7fb21d3c4700 1 -- 172.21.15.71:0/2778746292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb218106cd0 msgr2=0x7fb2181070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:47.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.309+0000 7fb21d3c4700 1 --2- 172.21.15.71:0/2778746292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb218106cd0 0x7fb2181070b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb208009910 tx=0x7fb208009c20 comp rx=0 tx=0).stop 2024-04-23T02:23:47.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.309+0000 7fb21d3c4700 1 -- 172.21.15.71:0/2778746292 shutdown_connections 2024-04-23T02:23:47.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.309+0000 7fb21d3c4700 1 --2- 172.21.15.71:0/2778746292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb218068580 0x7fb2180689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:47.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.309+0000 7fb21d3c4700 1 --2- 172.21.15.71:0/2778746292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb218106cd0 0x7fb2181070b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:47.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.309+0000 7fb21d3c4700 1 -- 172.21.15.71:0/2778746292 >> 172.21.15.71:0/2778746292 conn(0x7fb218075670 msgr2=0x7fb218075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:47.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.309+0000 7fb21d3c4700 1 -- 172.21.15.71:0/2778746292 shutdown_connections 2024-04-23T02:23:47.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.310+0000 7fb21d3c4700 1 -- 172.21.15.71:0/2778746292 wait complete. 2024-04-23T02:23:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.310+0000 7fb21d3c4700 1 Processor -- start 2024-04-23T02:23:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.310+0000 7fb21d3c4700 1 -- start start 2024-04-23T02:23:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.311+0000 7fb21d3c4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb218068580 0x7fb21819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.311+0000 7fb21d3c4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb218106cd0 0x7fb21819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.311+0000 7fb21d3c4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21819d130 con 0x7fb218106cd0 2024-04-23T02:23:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.311+0000 7fb21d3c4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb218196630 con 0x7fb218068580 2024-04-23T02:23:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.311+0000 7fb2167fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb218106cd0 0x7fb21819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.311+0000 7fb2167fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb218106cd0 0x7fb21819caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55918/0 (socket says 172.21.15.71:55918) 2024-04-23T02:23:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.311+0000 7fb2167fc700 1 -- 172.21.15.71:0/1570416026 learned_addr learned my addr 172.21.15.71:0/1570416026 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:23:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.311+0000 7fb216ffd700 1 --2- 172.21.15.71:0/1570416026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb218068580 0x7fb21819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.311+0000 7fb2167fc700 1 -- 172.21.15.71:0/1570416026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb218068580 msgr2=0x7fb21819c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.311+0000 7fb2167fc700 1 --2- 172.21.15.71:0/1570416026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb218068580 0x7fb21819c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.311+0000 7fb2167fc700 1 -- 172.21.15.71:0/1570416026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2080095f0 con 0x7fb218106cd0 2024-04-23T02:23:47.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.311+0000 7fb2167fc700 1 --2- 172.21.15.71:0/1570416026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb218106cd0 0x7fb21819caa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00d8d0 tx=0x7fb20c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:47.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.311+0000 7fb1fffff700 1 -- 172.21.15.71:0/1570416026 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb20c009940 con 0x7fb218106cd0 2024-04-23T02:23:47.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.312+0000 7fb21d3c4700 1 -- 172.21.15.71:0/1570416026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb218196910 con 0x7fb218106cd0 2024-04-23T02:23:47.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.312+0000 7fb1fffff700 1 -- 172.21.15.71:0/1570416026 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb20c010460 con 0x7fb218106cd0 2024-04-23T02:23:47.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.312+0000 7fb1fffff700 1 -- 172.21.15.71:0/1570416026 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb20c00f5f0 con 0x7fb218106cd0 2024-04-23T02:23:47.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.312+0000 7fb21d3c4700 1 -- 172.21.15.71:0/1570416026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb218196e60 con 0x7fb218106cd0 2024-04-23T02:23:47.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.312+0000 7fb21d3c4700 1 -- 172.21.15.71:0/1570416026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2181091d0 con 0x7fb218106cd0 2024-04-23T02:23:47.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.313+0000 7fb1fffff700 1 -- 172.21.15.71:0/1570416026 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb20c0105d0 con 0x7fb218106cd0 2024-04-23T02:23:47.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.313+0000 7fb1fffff700 1 --2- 172.21.15.71:0/1570416026 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb20006cce0 0x7fb20006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:47.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.313+0000 7fb1fffff700 1 -- 172.21.15.71:0/1570416026 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb20c08c680 con 0x7fb218106cd0 2024-04-23T02:23:47.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.313+0000 7fb216ffd700 1 --2- 172.21.15.71:0/1570416026 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb20006cce0 0x7fb20006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:47.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.314+0000 7fb216ffd700 1 --2- 172.21.15.71:0/1570416026 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb20006cce0 0x7fb20006f1a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb20800b5c0 tx=0x7fb208005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:47.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.318+0000 7fb1fffff700 1 -- 172.21.15.71:0/1570416026 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb20c05a0e0 con 0x7fb218106cd0 2024-04-23T02:23:47.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.522+0000 7fb21d3c4700 1 -- 172.21.15.71:0/1570416026 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb218197080 con 0x7fb20006cce0 2024-04-23T02:23:47.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.532+0000 7fb1fffff700 1 -- 172.21.15.71:0/1570416026 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fb218197080 con 0x7fb20006cce0 2024-04-23T02:23:47.536 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:23:47.536 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (7m) 72s ago 18m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:23:47.536 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (19m) 72s ago 19m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:23:47.536 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (17m) 54s ago 17m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:23:47.536 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (5m) 72s ago 19m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:23:47.536 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (5m) 54s ago 17m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:23:47.536 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (7m) 72s ago 18m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:23:47.536 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (98s) 72s ago 12m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:23:47.536 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (78s) 72s ago 12m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:23:47.536 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (12m) 54s ago 12m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:23:47.536 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (58s) 54s ago 12m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:23:47.536 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (9m) 72s ago 20m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:23:47.537 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (8m) 54s ago 17m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:23:47.537 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (5m) 72s ago 20m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:23:47.537 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (5m) 54s ago 17m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:23:47.537 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (8m) 72s ago 18m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:23:47.537 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (8m) 54s ago 17m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:23:47.537 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (4m) 72s ago 16m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:23:47.537 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (4m) 72s ago 16m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:23:47.537 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (3m) 72s ago 15m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:23:47.537 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (3m) 54s ago 15m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:23:47.537 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (2m) 54s ago 14m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:23:47.537 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (117s) 54s ago 14m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:23:47.537 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (7m) 72s ago 18m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:23:47.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.534+0000 7fb21d3c4700 1 -- 172.21.15.71:0/1570416026 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb20006cce0 msgr2=0x7fb20006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:47.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.534+0000 7fb21d3c4700 1 --2- 172.21.15.71:0/1570416026 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb20006cce0 0x7fb20006f1a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb20800b5c0 tx=0x7fb208005fb0 comp rx=0 tx=0).stop 2024-04-23T02:23:47.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.535+0000 7fb21d3c4700 1 -- 172.21.15.71:0/1570416026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb218106cd0 msgr2=0x7fb21819caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:47.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.535+0000 7fb21d3c4700 1 --2- 172.21.15.71:0/1570416026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb218106cd0 0x7fb21819caa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00d8d0 tx=0x7fb20c00dc90 comp rx=0 tx=0).stop 2024-04-23T02:23:47.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.535+0000 7fb21d3c4700 1 -- 172.21.15.71:0/1570416026 shutdown_connections 2024-04-23T02:23:47.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.535+0000 7fb21d3c4700 1 --2- 172.21.15.71:0/1570416026 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb20006cce0 0x7fb20006f1a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:47.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.535+0000 7fb21d3c4700 1 --2- 172.21.15.71:0/1570416026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb218068580 0x7fb21819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:47.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.535+0000 7fb21d3c4700 1 --2- 172.21.15.71:0/1570416026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb218106cd0 0x7fb21819caa0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:47.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.535+0000 7fb21d3c4700 1 -- 172.21.15.71:0/1570416026 >> 172.21.15.71:0/1570416026 conn(0x7fb218075670 msgr2=0x7fb2180fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:47.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.535+0000 7fb21d3c4700 1 -- 172.21.15.71:0/1570416026 shutdown_connections 2024-04-23T02:23:47.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.535+0000 7fb21d3c4700 1 -- 172.21.15.71:0/1570416026 wait complete. 2024-04-23T02:23:47.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.636+0000 7f9d9ade2700 1 -- 172.21.15.71:0/3652653492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d94100ea0 msgr2=0x7f9d94101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:47.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.636+0000 7f9d9ade2700 1 --2- 172.21.15.71:0/3652653492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d94100ea0 0x7f9d94101280 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009910 tx=0x7f9d88009c20 comp rx=0 tx=0).stop 2024-04-23T02:23:47.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.636+0000 7f9d9ade2700 1 -- 172.21.15.71:0/3652653492 shutdown_connections 2024-04-23T02:23:47.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.636+0000 7f9d9ade2700 1 --2- 172.21.15.71:0/3652653492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d94101850 0x7f9d941058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:47.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.636+0000 7f9d9ade2700 1 --2- 172.21.15.71:0/3652653492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d94100ea0 0x7f9d94101280 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:47.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.636+0000 7f9d9ade2700 1 -- 172.21.15.71:0/3652653492 >> 172.21.15.71:0/3652653492 conn(0x7f9d940fc750 msgr2=0x7f9d940feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:47.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.637+0000 7f9d9ade2700 1 -- 172.21.15.71:0/3652653492 shutdown_connections 2024-04-23T02:23:47.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.637+0000 7f9d9ade2700 1 -- 172.21.15.71:0/3652653492 wait complete. 2024-04-23T02:23:47.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.637+0000 7f9d9ade2700 1 Processor -- start 2024-04-23T02:23:47.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.637+0000 7f9d9ade2700 1 -- start start 2024-04-23T02:23:47.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.638+0000 7f9d9ade2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d94100ea0 0x7f9d9419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:47.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.638+0000 7f9d9ade2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d94101850 0x7f9d9419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:47.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.638+0000 7f9d9ade2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9419d130 con 0x7f9d94101850 2024-04-23T02:23:47.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.638+0000 7f9d9ade2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d941965e0 con 0x7f9d94100ea0 2024-04-23T02:23:47.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.638+0000 7f9d93fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d94101850 0x7f9d9419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:47.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.638+0000 7f9d93fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d94101850 0x7f9d9419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55930/0 (socket says 172.21.15.71:55930) 2024-04-23T02:23:47.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.638+0000 7f9d93fff700 1 -- 172.21.15.71:0/2530299629 learned_addr learned my addr 172.21.15.71:0/2530299629 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:23:47.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.638+0000 7f9d98b7e700 1 --2- 172.21.15.71:0/2530299629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d94100ea0 0x7f9d9419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:47.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.638+0000 7f9d93fff700 1 -- 172.21.15.71:0/2530299629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d94100ea0 msgr2=0x7f9d9419c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:47.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.638+0000 7f9d93fff700 1 --2- 172.21.15.71:0/2530299629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d94100ea0 0x7f9d9419c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:47.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.638+0000 7f9d93fff700 1 -- 172.21.15.71:0/2530299629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d880095f0 con 0x7f9d94101850 2024-04-23T02:23:47.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.638+0000 7f9d93fff700 1 --2- 172.21.15.71:0/2530299629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d94101850 0x7f9d9419caa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000d900 tx=0x7f9d8000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:47.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.638+0000 7f9d91ffb700 1 -- 172.21.15.71:0/2530299629 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d80009900 con 0x7f9d94101850 2024-04-23T02:23:47.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.638+0000 7f9d91ffb700 1 -- 172.21.15.71:0/2530299629 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9d80010460 con 0x7f9d94101850 2024-04-23T02:23:47.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.638+0000 7f9d9ade2700 1 -- 172.21.15.71:0/2530299629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d941968c0 con 0x7f9d94101850 2024-04-23T02:23:47.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.638+0000 7f9d91ffb700 1 -- 172.21.15.71:0/2530299629 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d8000f5f0 con 0x7f9d94101850 2024-04-23T02:23:47.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.639+0000 7f9d9ade2700 1 -- 172.21.15.71:0/2530299629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d94196e10 con 0x7f9d94101850 2024-04-23T02:23:47.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.640+0000 7f9d9ade2700 1 -- 172.21.15.71:0/2530299629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d94109240 con 0x7f9d94101850 2024-04-23T02:23:47.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.640+0000 7f9d91ffb700 1 -- 172.21.15.71:0/2530299629 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9d800105d0 con 0x7f9d94101850 2024-04-23T02:23:47.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.640+0000 7f9d91ffb700 1 --2- 172.21.15.71:0/2530299629 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9d8406cce0 0x7f9d8406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:47.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.640+0000 7f9d91ffb700 1 -- 172.21.15.71:0/2530299629 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9d8008c010 con 0x7f9d94101850 2024-04-23T02:23:47.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.640+0000 7f9d98b7e700 1 --2- 172.21.15.71:0/2530299629 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9d8406cce0 0x7f9d8406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:47.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.641+0000 7f9d98b7e700 1 --2- 172.21.15.71:0/2530299629 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9d8406cce0 0x7f9d8406f1a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9d88006010 tx=0x7f9d8801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:47.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.645+0000 7f9d91ffb700 1 -- 172.21.15.71:0/2530299629 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9d80059b90 con 0x7f9d94101850 2024-04-23T02:23:47.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.917+0000 7f9d9ade2700 1 -- 172.21.15.71:0/2530299629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9d9404f2e0 con 0x7f9d94101850 2024-04-23T02:23:47.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.917+0000 7f9d91ffb700 1 -- 172.21.15.71:0/2530299629 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f9d80059720 con 0x7f9d94101850 2024-04-23T02:23:47.921 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:23:47.921 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:23:47.922 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:23:47.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:23:47.922 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:23:47.922 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:23:47.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:23:47.922 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:23:47.922 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:23:47.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:23:47.922 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:23:47.922 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:23:47.922 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:23:47.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:23:47.923 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:23:47.923 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:23:47.923 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:23:47.923 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:23:47.923 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:23:47.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.919+0000 7f9d9ade2700 1 -- 172.21.15.71:0/2530299629 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9d8406cce0 msgr2=0x7f9d8406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:47.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.919+0000 7f9d9ade2700 1 --2- 172.21.15.71:0/2530299629 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9d8406cce0 0x7f9d8406f1a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9d88006010 tx=0x7f9d8801a040 comp rx=0 tx=0).stop 2024-04-23T02:23:47.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.919+0000 7f9d9ade2700 1 -- 172.21.15.71:0/2530299629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d94101850 msgr2=0x7f9d9419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:47.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.919+0000 7f9d9ade2700 1 --2- 172.21.15.71:0/2530299629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d94101850 0x7f9d9419caa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000d900 tx=0x7f9d8000dc10 comp rx=0 tx=0).stop 2024-04-23T02:23:47.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.919+0000 7f9d9ade2700 1 -- 172.21.15.71:0/2530299629 shutdown_connections 2024-04-23T02:23:47.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.919+0000 7f9d9ade2700 1 --2- 172.21.15.71:0/2530299629 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9d8406cce0 0x7f9d8406f1a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:47.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.919+0000 7f9d9ade2700 1 --2- 172.21.15.71:0/2530299629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d94100ea0 0x7f9d9419c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:47.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.919+0000 7f9d9ade2700 1 --2- 172.21.15.71:0/2530299629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d94101850 0x7f9d9419caa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:47.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.919+0000 7f9d9ade2700 1 -- 172.21.15.71:0/2530299629 >> 172.21.15.71:0/2530299629 conn(0x7f9d940fc750 msgr2=0x7f9d940fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:47.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.919+0000 7f9d9ade2700 1 -- 172.21.15.71:0/2530299629 shutdown_connections 2024-04-23T02:23:47.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:47.919+0000 7f9d9ade2700 1 -- 172.21.15.71:0/2530299629 wait complete. 2024-04-23T02:23:48.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.022+0000 7f1f5eb38700 1 -- 172.21.15.71:0/2568467817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f58105350 msgr2=0x7f1f58105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:48.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.022+0000 7f1f5eb38700 1 --2- 172.21.15.71:0/2568467817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f58105350 0x7f1f58105730 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009910 tx=0x7f1f4c009c20 comp rx=0 tx=0).stop 2024-04-23T02:23:48.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.022+0000 7f1f5eb38700 1 -- 172.21.15.71:0/2568467817 shutdown_connections 2024-04-23T02:23:48.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.022+0000 7f1f5eb38700 1 --2- 172.21.15.71:0/2568467817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f580ff340 0x7f1f580ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.023+0000 7f1f5eb38700 1 --2- 172.21.15.71:0/2568467817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f58105350 0x7f1f58105730 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.023+0000 7f1f5eb38700 1 -- 172.21.15.71:0/2568467817 >> 172.21.15.71:0/2568467817 conn(0x7f1f58074e30 msgr2=0x7f1f58075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.023+0000 7f1f5eb38700 1 -- 172.21.15.71:0/2568467817 shutdown_connections 2024-04-23T02:23:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.023+0000 7f1f5eb38700 1 -- 172.21.15.71:0/2568467817 wait complete. 2024-04-23T02:23:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.023+0000 7f1f5eb38700 1 Processor -- start 2024-04-23T02:23:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.023+0000 7f1f5eb38700 1 -- start start 2024-04-23T02:23:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.023+0000 7f1f5eb38700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f580ff340 0x7f1f581976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.023+0000 7f1f5eb38700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f58105350 0x7f1f58197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.023+0000 7f1f5eb38700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f581982d0 con 0x7f1f58105350 2024-04-23T02:23:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.023+0000 7f1f5eb38700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f58198440 con 0x7f1f580ff340 2024-04-23T02:23:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.024+0000 7f1f57fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f58105350 0x7f1f58197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.024+0000 7f1f57fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f58105350 0x7f1f58197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55956/0 (socket says 172.21.15.71:55956) 2024-04-23T02:23:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.024+0000 7f1f57fff700 1 -- 172.21.15.71:0/4033461153 learned_addr learned my addr 172.21.15.71:0/4033461153 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:23:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.024+0000 7f1f5c8d4700 1 --2- 172.21.15.71:0/4033461153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f580ff340 0x7f1f581976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.024+0000 7f1f57fff700 1 -- 172.21.15.71:0/4033461153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f580ff340 msgr2=0x7f1f581976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.024+0000 7f1f57fff700 1 --2- 172.21.15.71:0/4033461153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f580ff340 0x7f1f581976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.024+0000 7f1f57fff700 1 -- 172.21.15.71:0/4033461153 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c0095f0 con 0x7f1f58105350 2024-04-23T02:23:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.024+0000 7f1f57fff700 1 --2- 172.21.15.71:0/4033461153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f58105350 0x7f1f58197bf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400c8f0 tx=0x7f1f4400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.024+0000 7f1f55ffb700 1 -- 172.21.15.71:0/4033461153 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f44007a70 con 0x7f1f58105350 2024-04-23T02:23:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.024+0000 7f1f55ffb700 1 -- 172.21.15.71:0/4033461153 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1f44004cd0 con 0x7f1f58105350 2024-04-23T02:23:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.024+0000 7f1f5eb38700 1 -- 172.21.15.71:0/4033461153 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f581a0720 con 0x7f1f58105350 2024-04-23T02:23:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.024+0000 7f1f55ffb700 1 -- 172.21.15.71:0/4033461153 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f44005580 con 0x7f1f58105350 2024-04-23T02:23:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.024+0000 7f1f5eb38700 1 -- 172.21.15.71:0/4033461153 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f581a0c70 con 0x7f1f58105350 2024-04-23T02:23:48.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.025+0000 7f1f5eb38700 1 -- 172.21.15.71:0/4033461153 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5804f2e0 con 0x7f1f58105350 2024-04-23T02:23:48.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.026+0000 7f1f55ffb700 1 -- 172.21.15.71:0/4033461153 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1f44004750 con 0x7f1f58105350 2024-04-23T02:23:48.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.026+0000 7f1f55ffb700 1 --2- 172.21.15.71:0/4033461153 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1f4806cce0 0x7f1f4806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:48.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.026+0000 7f1f55ffb700 1 -- 172.21.15.71:0/4033461153 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1f44013070 con 0x7f1f58105350 2024-04-23T02:23:48.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.026+0000 7f1f5c8d4700 1 --2- 172.21.15.71:0/4033461153 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1f4806cce0 0x7f1f4806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:48.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.026+0000 7f1f5c8d4700 1 --2- 172.21.15.71:0/4033461153 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1f4806cce0 0x7f1f4806f1a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c006010 tx=0x7f1f4c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:48.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.030+0000 7f1f55ffb700 1 -- 172.21.15.71:0/4033461153 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1f440564e0 con 0x7f1f58105350 2024-04-23T02:23:48.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.264+0000 7f1f5eb38700 1 -- 172.21.15.71:0/4033461153 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1f581a0ef0 con 0x7f1f58105350 2024-04-23T02:23:48.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.264+0000 7f1f55ffb700 1 -- 172.21.15.71:0/4033461153 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f1f44059b00 con 0x7f1f58105350 2024-04-23T02:23:48.268 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:23:48.268 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:23:48.268 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:23:48.268 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:23:48.268 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:23:48.268 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:23:48.268 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:23:48.269 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:23:48.269 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:23:48.269 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:23:48.269 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:23:48.269 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:23:48.269 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:23:48.269 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:23:48.269 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:23:48.269 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:23:48.269 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:23:48.269 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:23:48.269 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:23:48.269 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:23:48.269 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:23:48.270 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:23:48.270 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:23:48.270 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:23:48.270 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:23:48.270 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:23:48.270 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:23:48.270 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:23:48.270 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:23:48.270 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:23:48.270 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:23:48.270 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:23:48.270 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:23:48.270 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:23:48.271 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:23:48.271 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:23:48.271 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:23:48.271 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:23:48.271 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:23:48.271 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:23:48.271 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:23:48.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.266+0000 7f1f5eb38700 1 -- 172.21.15.71:0/4033461153 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1f4806cce0 msgr2=0x7f1f4806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:48.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.266+0000 7f1f5eb38700 1 --2- 172.21.15.71:0/4033461153 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1f4806cce0 0x7f1f4806f1a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c006010 tx=0x7f1f4c01a040 comp rx=0 tx=0).stop 2024-04-23T02:23:48.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.266+0000 7f1f5eb38700 1 -- 172.21.15.71:0/4033461153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f58105350 msgr2=0x7f1f58197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:48.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.266+0000 7f1f5eb38700 1 --2- 172.21.15.71:0/4033461153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f58105350 0x7f1f58197bf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400c8f0 tx=0x7f1f4400ccb0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.266+0000 7f1f5eb38700 1 -- 172.21.15.71:0/4033461153 shutdown_connections 2024-04-23T02:23:48.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.266+0000 7f1f5eb38700 1 --2- 172.21.15.71:0/4033461153 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1f4806cce0 0x7f1f4806f1a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.266+0000 7f1f5eb38700 1 --2- 172.21.15.71:0/4033461153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f580ff340 0x7f1f581976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.266+0000 7f1f5eb38700 1 --2- 172.21.15.71:0/4033461153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f58105350 0x7f1f58197bf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.266+0000 7f1f5eb38700 1 -- 172.21.15.71:0/4033461153 >> 172.21.15.71:0/4033461153 conn(0x7f1f58074e30 msgr2=0x7f1f58103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:48.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.266+0000 7f1f5eb38700 1 -- 172.21.15.71:0/4033461153 shutdown_connections 2024-04-23T02:23:48.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.266+0000 7f1f5eb38700 1 -- 172.21.15.71:0/4033461153 wait complete. 2024-04-23T02:23:48.272 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:23:48.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.378+0000 7fdc4e835700 1 -- 172.21.15.71:0/491421163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc48100ea0 msgr2=0x7fdc48101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:48.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.378+0000 7fdc4e835700 1 --2- 172.21.15.71:0/491421163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc48100ea0 0x7fdc48101280 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009910 tx=0x7fdc38009c20 comp rx=0 tx=0).stop 2024-04-23T02:23:48.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.378+0000 7fdc4e835700 1 -- 172.21.15.71:0/491421163 shutdown_connections 2024-04-23T02:23:48.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.378+0000 7fdc4e835700 1 --2- 172.21.15.71:0/491421163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc48101850 0x7fdc481058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.378+0000 7fdc4e835700 1 --2- 172.21.15.71:0/491421163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc48100ea0 0x7fdc48101280 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.378+0000 7fdc4e835700 1 -- 172.21.15.71:0/491421163 >> 172.21.15.71:0/491421163 conn(0x7fdc480fc750 msgr2=0x7fdc480feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:48.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.378+0000 7fdc4e835700 1 -- 172.21.15.71:0/491421163 shutdown_connections 2024-04-23T02:23:48.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.378+0000 7fdc4e835700 1 -- 172.21.15.71:0/491421163 wait complete. 2024-04-23T02:23:48.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.379+0000 7fdc4e835700 1 Processor -- start 2024-04-23T02:23:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.379+0000 7fdc4e835700 1 -- start start 2024-04-23T02:23:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.379+0000 7fdc4e835700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc48100ea0 0x7fdc4819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.379+0000 7fdc4e835700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc48101850 0x7fdc4819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.379+0000 7fdc4e835700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4819d130 con 0x7fdc48100ea0 2024-04-23T02:23:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.379+0000 7fdc4e835700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc481965e0 con 0x7fdc48101850 2024-04-23T02:23:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.379+0000 7fdc47fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc48100ea0 0x7fdc4819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.379+0000 7fdc47fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc48100ea0 0x7fdc4819c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55974/0 (socket says 172.21.15.71:55974) 2024-04-23T02:23:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.379+0000 7fdc47fff700 1 -- 172.21.15.71:0/4206155932 learned_addr learned my addr 172.21.15.71:0/4206155932 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:23:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.380+0000 7fdc47fff700 1 -- 172.21.15.71:0/4206155932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc48101850 msgr2=0x7fdc4819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.380+0000 7fdc47fff700 1 --2- 172.21.15.71:0/4206155932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc48101850 0x7fdc4819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.380+0000 7fdc47fff700 1 -- 172.21.15.71:0/4206155932 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc380095f0 con 0x7fdc48100ea0 2024-04-23T02:23:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.380+0000 7fdc47fff700 1 --2- 172.21.15.71:0/4206155932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc48100ea0 0x7fdc4819c560 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdc38006010 tx=0x7fdc3800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.380+0000 7fdc457fa700 1 -- 172.21.15.71:0/4206155932 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc3801d070 con 0x7fdc48100ea0 2024-04-23T02:23:48.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.380+0000 7fdc4e835700 1 -- 172.21.15.71:0/4206155932 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc48196860 con 0x7fdc48100ea0 2024-04-23T02:23:48.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.380+0000 7fdc457fa700 1 -- 172.21.15.71:0/4206155932 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdc3800f460 con 0x7fdc48100ea0 2024-04-23T02:23:48.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.380+0000 7fdc457fa700 1 -- 172.21.15.71:0/4206155932 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc38021640 con 0x7fdc48100ea0 2024-04-23T02:23:48.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.380+0000 7fdc4e835700 1 -- 172.21.15.71:0/4206155932 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc48196d50 con 0x7fdc48100ea0 2024-04-23T02:23:48.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.381+0000 7fdc4e835700 1 -- 172.21.15.71:0/4206155932 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc48109240 con 0x7fdc48100ea0 2024-04-23T02:23:48.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.381+0000 7fdc457fa700 1 -- 172.21.15.71:0/4206155932 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fdc3800fa60 con 0x7fdc48100ea0 2024-04-23T02:23:48.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.382+0000 7fdc457fa700 1 --2- 172.21.15.71:0/4206155932 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdc3006cce0 0x7fdc3006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:48.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.382+0000 7fdc457fa700 1 -- 172.21.15.71:0/4206155932 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdc3808d880 con 0x7fdc48100ea0 2024-04-23T02:23:48.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.382+0000 7fdc477fe700 1 --2- 172.21.15.71:0/4206155932 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdc3006cce0 0x7fdc3006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:48.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.382+0000 7fdc477fe700 1 --2- 172.21.15.71:0/4206155932 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdc3006cce0 0x7fdc3006f1a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdc48105b20 tx=0x7fdc3c005c80 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:48.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.386+0000 7fdc457fa700 1 -- 172.21.15.71:0/4206155932 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fdc3805b380 con 0x7fdc48100ea0 2024-04-23T02:23:48.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.597+0000 7fdc4e835700 1 -- 172.21.15.71:0/4206155932 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc48196f70 con 0x7fdc3006cce0 2024-04-23T02:23:48.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.598+0000 7fdc457fa700 1 -- 172.21.15.71:0/4206155932 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdc48196f70 con 0x7fdc3006cce0 2024-04-23T02:23:48.602 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:23:48.602 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:23:48.602 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:23:48.602 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:23:48.603 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:23:48.603 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:23:48.603 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:23:48.603 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:23:48.603 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:23:48.603 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:23:48.603 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:23:48.603 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:23:48.603 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:23:48.603 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:23:48.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.600+0000 7fdc4e835700 1 -- 172.21.15.71:0/4206155932 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdc3006cce0 msgr2=0x7fdc3006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:48.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.600+0000 7fdc4e835700 1 --2- 172.21.15.71:0/4206155932 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdc3006cce0 0x7fdc3006f1a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdc48105b20 tx=0x7fdc3c005c80 comp rx=0 tx=0).stop 2024-04-23T02:23:48.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.600+0000 7fdc4e835700 1 -- 172.21.15.71:0/4206155932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc48100ea0 msgr2=0x7fdc4819c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.600+0000 7fdc4e835700 1 --2- 172.21.15.71:0/4206155932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc48100ea0 0x7fdc4819c560 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdc38006010 tx=0x7fdc3800be20 comp rx=0 tx=0).stop 2024-04-23T02:23:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.600+0000 7fdc4e835700 1 -- 172.21.15.71:0/4206155932 shutdown_connections 2024-04-23T02:23:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.600+0000 7fdc4e835700 1 --2- 172.21.15.71:0/4206155932 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdc3006cce0 0x7fdc3006f1a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.600+0000 7fdc4e835700 1 --2- 172.21.15.71:0/4206155932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc48100ea0 0x7fdc4819c560 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.600+0000 7fdc4e835700 1 --2- 172.21.15.71:0/4206155932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc48101850 0x7fdc4819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.600+0000 7fdc4e835700 1 -- 172.21.15.71:0/4206155932 >> 172.21.15.71:0/4206155932 conn(0x7fdc480fc750 msgr2=0x7fdc480fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.600+0000 7fdc4e835700 1 -- 172.21.15.71:0/4206155932 shutdown_connections 2024-04-23T02:23:48.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.600+0000 7fdc4e835700 1 -- 172.21.15.71:0/4206155932 wait complete. 2024-04-23T02:23:48.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:48 smithi192 ceph-mon[108617]: pgmap v314: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:48.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:48 smithi192 ceph-mon[108617]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:23:48.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:48 smithi192 ceph-mon[108617]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:23:48.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:48 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2530299629' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:23:48.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:48 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/4033461153' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:23:48.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.703+0000 7fd76d761700 1 -- 172.21.15.71:0/2849696461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd768105bd0 msgr2=0x7fd768105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:48.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.703+0000 7fd76d761700 1 --2- 172.21.15.71:0/2849696461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd768105bd0 0x7fd768105fb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd758009910 tx=0x7fd758009c20 comp rx=0 tx=0).stop 2024-04-23T02:23:48.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.703+0000 7fd76d761700 1 -- 172.21.15.71:0/2849696461 shutdown_connections 2024-04-23T02:23:48.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.703+0000 7fd76d761700 1 --2- 172.21.15.71:0/2849696461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7680ffc20 0x7fd768100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.703+0000 7fd76d761700 1 --2- 172.21.15.71:0/2849696461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd768105bd0 0x7fd768105fb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.703+0000 7fd76d761700 1 -- 172.21.15.71:0/2849696461 >> 172.21.15.71:0/2849696461 conn(0x7fd768075670 msgr2=0x7fd768075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:48.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.703+0000 7fd76d761700 1 -- 172.21.15.71:0/2849696461 shutdown_connections 2024-04-23T02:23:48.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.703+0000 7fd76d761700 1 -- 172.21.15.71:0/2849696461 wait complete. 2024-04-23T02:23:48.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.704+0000 7fd76d761700 1 Processor -- start 2024-04-23T02:23:48.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.704+0000 7fd76d761700 1 -- start start 2024-04-23T02:23:48.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.704+0000 7fd76d761700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7680ffc20 0x7fd7681975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:48.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.704+0000 7fd76d761700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd768105bd0 0x7fd768197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:48.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.704+0000 7fd76d761700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7681980b0 con 0x7fd7680ffc20 2024-04-23T02:23:48.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.704+0000 7fd76d761700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd768198220 con 0x7fd768105bd0 2024-04-23T02:23:48.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.705+0000 7fd766ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7680ffc20 0x7fd7681975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:48.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.705+0000 7fd766ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7680ffc20 0x7fd7681975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55990/0 (socket says 172.21.15.71:55990) 2024-04-23T02:23:48.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.705+0000 7fd766ffd700 1 -- 172.21.15.71:0/2658307968 learned_addr learned my addr 172.21.15.71:0/2658307968 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:23:48.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.705+0000 7fd7667fc700 1 --2- 172.21.15.71:0/2658307968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd768105bd0 0x7fd768197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:48.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.705+0000 7fd766ffd700 1 -- 172.21.15.71:0/2658307968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd768105bd0 msgr2=0x7fd768197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:48.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.705+0000 7fd766ffd700 1 --2- 172.21.15.71:0/2658307968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd768105bd0 0x7fd768197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.705+0000 7fd766ffd700 1 -- 172.21.15.71:0/2658307968 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7580095f0 con 0x7fd7680ffc20 2024-04-23T02:23:48.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.705+0000 7fd766ffd700 1 --2- 172.21.15.71:0/2658307968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7680ffc20 0x7fd7681975e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd75800b5c0 tx=0x7fd75800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:48.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.705+0000 7fd74ffff700 1 -- 172.21.15.71:0/2658307968 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd75801d070 con 0x7fd7680ffc20 2024-04-23T02:23:48.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.705+0000 7fd74ffff700 1 -- 172.21.15.71:0/2658307968 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd75800f460 con 0x7fd7680ffc20 2024-04-23T02:23:48.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.705+0000 7fd76d761700 1 -- 172.21.15.71:0/2658307968 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7681a0650 con 0x7fd7680ffc20 2024-04-23T02:23:48.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.705+0000 7fd74ffff700 1 -- 172.21.15.71:0/2658307968 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd758021640 con 0x7fd7680ffc20 2024-04-23T02:23:48.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.705+0000 7fd76d761700 1 -- 172.21.15.71:0/2658307968 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7681a0a00 con 0x7fd7680ffc20 2024-04-23T02:23:48.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.706+0000 7fd76d761700 1 -- 172.21.15.71:0/2658307968 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7681080f0 con 0x7fd7680ffc20 2024-04-23T02:23:48.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.707+0000 7fd74ffff700 1 -- 172.21.15.71:0/2658307968 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd75800fa20 con 0x7fd7680ffc20 2024-04-23T02:23:48.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.707+0000 7fd74ffff700 1 --2- 172.21.15.71:0/2658307968 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd75006cce0 0x7fd75006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:23:48.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.707+0000 7fd74ffff700 1 -- 172.21.15.71:0/2658307968 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd75808d7c0 con 0x7fd7680ffc20 2024-04-23T02:23:48.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.707+0000 7fd7667fc700 1 --2- 172.21.15.71:0/2658307968 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd75006cce0 0x7fd75006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:23:48.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.707+0000 7fd7667fc700 1 --2- 172.21.15.71:0/2658307968 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd75006cce0 0x7fd75006f1a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd768068e50 tx=0x7fd75c00a380 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:23:48.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.712+0000 7fd74ffff700 1 -- 172.21.15.71:0/2658307968 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd758057830 con 0x7fd7680ffc20 2024-04-23T02:23:48.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:48 smithi071 ceph-mon[144544]: pgmap v314: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:48.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:48 smithi071 ceph-mon[144544]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:23:48.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:48 smithi071 ceph-mon[144544]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:23:48.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:48 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2530299629' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:23:48.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:48 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/4033461153' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:23:48.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.983+0000 7fd76d761700 1 -- 172.21.15.71:0/2658307968 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd76804f2e0 con 0x7fd7680ffc20 2024-04-23T02:23:48.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.983+0000 7fd74ffff700 1 -- 172.21.15.71:0/2658307968 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fd75805ae50 con 0x7fd7680ffc20 2024-04-23T02:23:48.987 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:23:48.987 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:23:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:23:48.988 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:23:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:23:48.988 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:23:48.988 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:23:48.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.985+0000 7fd76d761700 1 -- 172.21.15.71:0/2658307968 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd75006cce0 msgr2=0x7fd75006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:48.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.985+0000 7fd76d761700 1 --2- 172.21.15.71:0/2658307968 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd75006cce0 0x7fd75006f1a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd768068e50 tx=0x7fd75c00a380 comp rx=0 tx=0).stop 2024-04-23T02:23:48.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.985+0000 7fd76d761700 1 -- 172.21.15.71:0/2658307968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7680ffc20 msgr2=0x7fd7681975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:23:48.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.985+0000 7fd76d761700 1 --2- 172.21.15.71:0/2658307968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7680ffc20 0x7fd7681975e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd75800b5c0 tx=0x7fd75800bd70 comp rx=0 tx=0).stop 2024-04-23T02:23:48.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.985+0000 7fd76d761700 1 -- 172.21.15.71:0/2658307968 shutdown_connections 2024-04-23T02:23:48.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.985+0000 7fd76d761700 1 --2- 172.21.15.71:0/2658307968 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd75006cce0 0x7fd75006f1a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.985+0000 7fd76d761700 1 --2- 172.21.15.71:0/2658307968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7680ffc20 0x7fd7681975e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.985+0000 7fd76d761700 1 --2- 172.21.15.71:0/2658307968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd768105bd0 0x7fd768197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:23:48.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.985+0000 7fd76d761700 1 -- 172.21.15.71:0/2658307968 >> 172.21.15.71:0/2658307968 conn(0x7fd768075670 msgr2=0x7fd7680fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:23:48.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.985+0000 7fd76d761700 1 -- 172.21.15.71:0/2658307968 shutdown_connections 2024-04-23T02:23:48.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:23:48.986+0000 7fd76d761700 1 -- 172.21.15.71:0/2658307968 wait complete. 2024-04-23T02:23:49.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:49 smithi071 ceph-mon[144544]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:23:49.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:49 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2658307968' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:23:49.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:49 smithi192 ceph-mon[108617]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:23:49.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:49 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2658307968' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:23:50.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:50 smithi071 ceph-mon[144544]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:23:50.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:50 smithi071 ceph-mon[144544]: pgmap v315: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:50.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:23:50.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:50 smithi192 ceph-mon[108617]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:23:50.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:50 smithi192 ceph-mon[108617]: pgmap v315: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:50.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:23:52.633 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:52 smithi071 ceph-mon[144544]: pgmap v316: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:52.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:52 smithi192 ceph-mon[108617]: pgmap v316: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:54.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:54 smithi071 ceph-mon[144544]: pgmap v317: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:54.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:54 smithi192 ceph-mon[108617]: pgmap v317: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:56.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:56 smithi071 ceph-mon[144544]: pgmap v318: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:56.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:56 smithi192 ceph-mon[108617]: pgmap v318: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:58.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:23:58 smithi192 ceph-mon[108617]: pgmap v319: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:23:58.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:23:58 smithi071 ceph-mon[144544]: pgmap v319: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:00.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:00 smithi071 ceph-mon[144544]: pgmap v320: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:00.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:00 smithi192 ceph-mon[108617]: pgmap v320: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:02.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:02 smithi071 ceph-mon[144544]: pgmap v321: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:02.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:02 smithi192 ceph-mon[108617]: pgmap v321: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:04.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:04 smithi071 ceph-mon[144544]: pgmap v322: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:04.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:04 smithi192 ceph-mon[108617]: pgmap v322: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:05.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:24:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:24:06.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:06 smithi071 ceph-mon[144544]: pgmap v323: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:06.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:06 smithi192 ceph-mon[108617]: pgmap v323: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:07.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:07 smithi071 ceph-mon[144544]: pgmap v324: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:07.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:07 smithi192 ceph-mon[108617]: pgmap v324: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:10.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:09 smithi192 ceph-mon[108617]: pgmap v325: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:10.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:09 smithi071 ceph-mon[144544]: pgmap v325: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:12.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:11 smithi192 ceph-mon[108617]: pgmap v326: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:12.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:11 smithi071 ceph-mon[144544]: pgmap v326: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:14.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:13 smithi192 ceph-mon[108617]: pgmap v327: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:14.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:13 smithi071 ceph-mon[144544]: pgmap v327: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:16.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:15 smithi192 ceph-mon[108617]: pgmap v328: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:16.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:15 smithi071 ceph-mon[144544]: pgmap v328: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:18.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:17 smithi192 ceph-mon[108617]: pgmap v329: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:18.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:17 smithi071 ceph-mon[144544]: pgmap v329: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:19.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.102+0000 7f0ae0984700 1 -- 172.21.15.71:0/666466310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 msgr2=0x7f0adc1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:19.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.102+0000 7f0ae0984700 1 --2- 172.21.15.71:0/666466310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 0x7f0adc1070b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009910 tx=0x7f0ad0009c20 comp rx=0 tx=0).stop 2024-04-23T02:24:19.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.102+0000 7f0ae0984700 1 -- 172.21.15.71:0/666466310 shutdown_connections 2024-04-23T02:24:19.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.102+0000 7f0ae0984700 1 --2- 172.21.15.71:0/666466310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0adc068580 0x7f0adc0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.102+0000 7f0ae0984700 1 --2- 172.21.15.71:0/666466310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 0x7f0adc1070b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.102+0000 7f0ae0984700 1 -- 172.21.15.71:0/666466310 >> 172.21.15.71:0/666466310 conn(0x7f0adc075670 msgr2=0x7f0adc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:19.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.102+0000 7f0ae0984700 1 -- 172.21.15.71:0/666466310 shutdown_connections 2024-04-23T02:24:19.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.102+0000 7f0ae0984700 1 -- 172.21.15.71:0/666466310 wait complete. 2024-04-23T02:24:19.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.103+0000 7f0ae0984700 1 Processor -- start 2024-04-23T02:24:19.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.103+0000 7f0ae0984700 1 -- start start 2024-04-23T02:24:19.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.103+0000 7f0ae0984700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0adc068580 0x7f0adc0ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:19.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.103+0000 7f0ae0984700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 0x7f0adc100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:19.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.103+0000 7f0ae0984700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc100770 con 0x7f0adc106cd0 2024-04-23T02:24:19.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.103+0000 7f0ae0984700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc1008b0 con 0x7f0adc068580 2024-04-23T02:24:19.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.103+0000 7f0ad9d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 0x7f0adc100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:19.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.104+0000 7f0ad9d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 0x7f0adc100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38444/0 (socket says 172.21.15.71:38444) 2024-04-23T02:24:19.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.104+0000 7f0ad9d9b700 1 -- 172.21.15.71:0/737943410 learned_addr learned my addr 172.21.15.71:0/737943410 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:24:19.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.104+0000 7f0ada59c700 1 --2- 172.21.15.71:0/737943410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0adc068580 0x7f0adc0ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:19.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.104+0000 7f0ad9d9b700 1 -- 172.21.15.71:0/737943410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0adc068580 msgr2=0x7f0adc0ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:19.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.104+0000 7f0ad9d9b700 1 --2- 172.21.15.71:0/737943410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0adc068580 0x7f0adc0ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.104+0000 7f0ad9d9b700 1 -- 172.21.15.71:0/737943410 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad00095f0 con 0x7f0adc106cd0 2024-04-23T02:24:19.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.104+0000 7f0ad9d9b700 1 --2- 172.21.15.71:0/737943410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 0x7f0adc100230 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00e920 tx=0x7f0acc00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:19.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.104+0000 7f0acb7fe700 1 -- 172.21.15.71:0/737943410 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0acc009790 con 0x7f0adc106cd0 2024-04-23T02:24:19.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.104+0000 7f0acb7fe700 1 -- 172.21.15.71:0/737943410 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0acc00cba0 con 0x7f0adc106cd0 2024-04-23T02:24:19.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.104+0000 7f0ae0984700 1 -- 172.21.15.71:0/737943410 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0adc1a29b0 con 0x7f0adc106cd0 2024-04-23T02:24:19.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.104+0000 7f0acb7fe700 1 -- 172.21.15.71:0/737943410 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0acc018650 con 0x7f0adc106cd0 2024-04-23T02:24:19.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.104+0000 7f0ae0984700 1 -- 172.21.15.71:0/737943410 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0adc1a2d60 con 0x7f0adc106cd0 2024-04-23T02:24:19.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.105+0000 7f0ae0984700 1 -- 172.21.15.71:0/737943410 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0adc1091d0 con 0x7f0adc106cd0 2024-04-23T02:24:19.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.106+0000 7f0acb7fe700 1 -- 172.21.15.71:0/737943410 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0acc00cd10 con 0x7f0adc106cd0 2024-04-23T02:24:19.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.106+0000 7f0acb7fe700 1 --2- 172.21.15.71:0/737943410 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ac406cce0 0x7f0ac406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:19.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.106+0000 7f0acb7fe700 1 -- 172.21.15.71:0/737943410 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0acc014070 con 0x7f0adc106cd0 2024-04-23T02:24:19.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.106+0000 7f0ada59c700 1 --2- 172.21.15.71:0/737943410 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ac406cce0 0x7f0ac406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:19.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.107+0000 7f0ada59c700 1 --2- 172.21.15.71:0/737943410 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ac406cce0 0x7f0ac406f1a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0000c00 tx=0x7f0ad0009f70 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:19.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.111+0000 7f0acb7fe700 1 -- 172.21.15.71:0/737943410 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0acc052c50 con 0x7f0adc106cd0 2024-04-23T02:24:19.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.321+0000 7f0ae0984700 1 -- 172.21.15.71:0/737943410 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0adc02ccc0 con 0x7f0ac406cce0 2024-04-23T02:24:19.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.322+0000 7f0acb7fe700 1 -- 172.21.15.71:0/737943410 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0adc02ccc0 con 0x7f0ac406cce0 2024-04-23T02:24:19.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.324+0000 7f0ae0984700 1 -- 172.21.15.71:0/737943410 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ac406cce0 msgr2=0x7f0ac406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:19.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.324+0000 7f0ae0984700 1 --2- 172.21.15.71:0/737943410 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ac406cce0 0x7f0ac406f1a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0000c00 tx=0x7f0ad0009f70 comp rx=0 tx=0).stop 2024-04-23T02:24:19.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.324+0000 7f0ae0984700 1 -- 172.21.15.71:0/737943410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 msgr2=0x7f0adc100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:19.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.324+0000 7f0ae0984700 1 --2- 172.21.15.71:0/737943410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 0x7f0adc100230 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00e920 tx=0x7f0acc00ece0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.324+0000 7f0ae0984700 1 -- 172.21.15.71:0/737943410 shutdown_connections 2024-04-23T02:24:19.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.324+0000 7f0ae0984700 1 --2- 172.21.15.71:0/737943410 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ac406cce0 0x7f0ac406f1a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.324+0000 7f0ae0984700 1 --2- 172.21.15.71:0/737943410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0adc068580 0x7f0adc0ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.324+0000 7f0ae0984700 1 --2- 172.21.15.71:0/737943410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 0x7f0adc100230 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.324+0000 7f0ae0984700 1 -- 172.21.15.71:0/737943410 >> 172.21.15.71:0/737943410 conn(0x7f0adc075670 msgr2=0x7f0adc0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:19.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.324+0000 7f0ae0984700 1 -- 172.21.15.71:0/737943410 shutdown_connections 2024-04-23T02:24:19.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.324+0000 7f0ae0984700 1 -- 172.21.15.71:0/737943410 wait complete. 2024-04-23T02:24:19.337 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:24:19.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.428+0000 7f0def901700 1 -- 172.21.15.71:0/3362963814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0de8100ea0 msgr2=0x7f0de8101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:19.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.428+0000 7f0def901700 1 --2- 172.21.15.71:0/3362963814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0de8100ea0 0x7f0de8101280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009910 tx=0x7f0de4009c20 comp rx=0 tx=0).stop 2024-04-23T02:24:19.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.428+0000 7f0def901700 1 -- 172.21.15.71:0/3362963814 shutdown_connections 2024-04-23T02:24:19.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.428+0000 7f0def901700 1 --2- 172.21.15.71:0/3362963814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de8101850 0x7f0de81058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.428+0000 7f0def901700 1 --2- 172.21.15.71:0/3362963814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0de8100ea0 0x7f0de8101280 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.428+0000 7f0def901700 1 -- 172.21.15.71:0/3362963814 >> 172.21.15.71:0/3362963814 conn(0x7f0de80fc750 msgr2=0x7f0de80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:19.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.428+0000 7f0def901700 1 -- 172.21.15.71:0/3362963814 shutdown_connections 2024-04-23T02:24:19.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.428+0000 7f0def901700 1 -- 172.21.15.71:0/3362963814 wait complete. 2024-04-23T02:24:19.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.429+0000 7f0def901700 1 Processor -- start 2024-04-23T02:24:19.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.429+0000 7f0def901700 1 -- start start 2024-04-23T02:24:19.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.429+0000 7f0def901700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0de8100ea0 0x7f0de819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:19.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.429+0000 7f0def901700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de8101850 0x7f0de819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:19.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.429+0000 7f0def901700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de819d130 con 0x7f0de8100ea0 2024-04-23T02:24:19.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.429+0000 7f0def901700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81965e0 con 0x7f0de8101850 2024-04-23T02:24:19.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.429+0000 7f0ded69d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0de8100ea0 0x7f0de819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:19.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.429+0000 7f0ded69d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0de8100ea0 0x7f0de819c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38454/0 (socket says 172.21.15.71:38454) 2024-04-23T02:24:19.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.429+0000 7f0ded69d700 1 -- 172.21.15.71:0/2020549814 learned_addr learned my addr 172.21.15.71:0/2020549814 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:24:19.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.430+0000 7f0dece9c700 1 --2- 172.21.15.71:0/2020549814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de8101850 0x7f0de819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:19.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.430+0000 7f0ded69d700 1 -- 172.21.15.71:0/2020549814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de8101850 msgr2=0x7f0de819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:19.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.430+0000 7f0ded69d700 1 --2- 172.21.15.71:0/2020549814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de8101850 0x7f0de819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.430+0000 7f0ded69d700 1 -- 172.21.15.71:0/2020549814 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de40095f0 con 0x7f0de8100ea0 2024-04-23T02:24:19.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.430+0000 7f0dece9c700 1 --2- 172.21.15.71:0/2020549814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de8101850 0x7f0de819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:24:19.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.430+0000 7f0ded69d700 1 --2- 172.21.15.71:0/2020549814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0de8100ea0 0x7f0de819c560 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0de4006010 tx=0x7f0de400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:19.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.430+0000 7f0dde7fc700 1 -- 172.21.15.71:0/2020549814 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0de401d070 con 0x7f0de8100ea0 2024-04-23T02:24:19.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.430+0000 7f0dde7fc700 1 -- 172.21.15.71:0/2020549814 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0de400f460 con 0x7f0de8100ea0 2024-04-23T02:24:19.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.430+0000 7f0def901700 1 -- 172.21.15.71:0/2020549814 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de8196860 con 0x7f0de8100ea0 2024-04-23T02:24:19.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.430+0000 7f0dde7fc700 1 -- 172.21.15.71:0/2020549814 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0de4021640 con 0x7f0de8100ea0 2024-04-23T02:24:19.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.430+0000 7f0def901700 1 -- 172.21.15.71:0/2020549814 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de8196d50 con 0x7f0de8100ea0 2024-04-23T02:24:19.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.431+0000 7f0def901700 1 -- 172.21.15.71:0/2020549814 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de8109240 con 0x7f0de8100ea0 2024-04-23T02:24:19.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.431+0000 7f0dde7fc700 1 -- 172.21.15.71:0/2020549814 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0de400fa20 con 0x7f0de8100ea0 2024-04-23T02:24:19.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.432+0000 7f0dde7fc700 1 --2- 172.21.15.71:0/2020549814 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0dd406cce0 0x7f0dd406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:19.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.432+0000 7f0dde7fc700 1 -- 172.21.15.71:0/2020549814 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0de408d740 con 0x7f0de8100ea0 2024-04-23T02:24:19.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.432+0000 7f0dece9c700 1 --2- 172.21.15.71:0/2020549814 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0dd406cce0 0x7f0dd406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:19.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.432+0000 7f0dece9c700 1 --2- 172.21.15.71:0/2020549814 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0dd406cce0 0x7f0dd406f1a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8005d90 tx=0x7f0dd8005d00 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:19.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.436+0000 7f0dde7fc700 1 -- 172.21.15.71:0/2020549814 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0de405b2c0 con 0x7f0de8100ea0 2024-04-23T02:24:19.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.646+0000 7f0def901700 1 -- 172.21.15.71:0/2020549814 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0de8196f70 con 0x7f0dd406cce0 2024-04-23T02:24:19.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.648+0000 7f0dde7fc700 1 -- 172.21.15.71:0/2020549814 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0de8196f70 con 0x7f0dd406cce0 2024-04-23T02:24:19.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.649+0000 7f0def901700 1 -- 172.21.15.71:0/2020549814 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0dd406cce0 msgr2=0x7f0dd406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:19.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.649+0000 7f0def901700 1 --2- 172.21.15.71:0/2020549814 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0dd406cce0 0x7f0dd406f1a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8005d90 tx=0x7f0dd8005d00 comp rx=0 tx=0).stop 2024-04-23T02:24:19.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.650+0000 7f0def901700 1 -- 172.21.15.71:0/2020549814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0de8100ea0 msgr2=0x7f0de819c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:19.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.650+0000 7f0def901700 1 --2- 172.21.15.71:0/2020549814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0de8100ea0 0x7f0de819c560 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0de4006010 tx=0x7f0de400be20 comp rx=0 tx=0).stop 2024-04-23T02:24:19.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.650+0000 7f0def901700 1 -- 172.21.15.71:0/2020549814 shutdown_connections 2024-04-23T02:24:19.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.650+0000 7f0def901700 1 --2- 172.21.15.71:0/2020549814 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0dd406cce0 0x7f0dd406f1a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.650+0000 7f0def901700 1 --2- 172.21.15.71:0/2020549814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0de8100ea0 0x7f0de819c560 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.650+0000 7f0def901700 1 --2- 172.21.15.71:0/2020549814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0de8101850 0x7f0de819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.650+0000 7f0def901700 1 -- 172.21.15.71:0/2020549814 >> 172.21.15.71:0/2020549814 conn(0x7f0de80fc750 msgr2=0x7f0de80fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:19.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.650+0000 7f0def901700 1 -- 172.21.15.71:0/2020549814 shutdown_connections 2024-04-23T02:24:19.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.650+0000 7f0def901700 1 -- 172.21.15.71:0/2020549814 wait complete. 2024-04-23T02:24:19.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.753+0000 7f804a79f700 1 -- 172.21.15.71:0/938785821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8044101940 msgr2=0x7f8044101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:19.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.753+0000 7f804a79f700 1 --2- 172.21.15.71:0/938785821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8044101940 0x7f8044101d20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8040009910 tx=0x7f8040009c20 comp rx=0 tx=0).stop 2024-04-23T02:24:19.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.754+0000 7f804a79f700 1 -- 172.21.15.71:0/938785821 shutdown_connections 2024-04-23T02:24:19.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.754+0000 7f804a79f700 1 --2- 172.21.15.71:0/938785821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80441002d0 0x7f8044100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.754+0000 7f804a79f700 1 --2- 172.21.15.71:0/938785821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8044101940 0x7f8044101d20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.754+0000 7f804a79f700 1 -- 172.21.15.71:0/938785821 >> 172.21.15.71:0/938785821 conn(0x7f8044074c20 msgr2=0x7f8044075030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:19.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.754+0000 7f804a79f700 1 -- 172.21.15.71:0/938785821 shutdown_connections 2024-04-23T02:24:19.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.754+0000 7f804a79f700 1 -- 172.21.15.71:0/938785821 wait complete. 2024-04-23T02:24:19.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.754+0000 7f804a79f700 1 Processor -- start 2024-04-23T02:24:19.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.754+0000 7f804a79f700 1 -- start start 2024-04-23T02:24:19.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.755+0000 7f804a79f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80441002d0 0x7f80441974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:19.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.755+0000 7f804a79f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8044101940 0x7f8044197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:19.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.755+0000 7f804a79f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044198040 con 0x7f80441002d0 2024-04-23T02:24:19.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.755+0000 7f804a79f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044198180 con 0x7f8044101940 2024-04-23T02:24:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.755+0000 7f804979d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80441002d0 0x7f80441974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.755+0000 7f804979d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80441002d0 0x7f80441974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38470/0 (socket says 172.21.15.71:38470) 2024-04-23T02:24:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.755+0000 7f804979d700 1 -- 172.21.15.71:0/4184126854 learned_addr learned my addr 172.21.15.71:0/4184126854 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:24:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.755+0000 7f8048f9c700 1 --2- 172.21.15.71:0/4184126854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8044101940 0x7f8044197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.755+0000 7f804979d700 1 -- 172.21.15.71:0/4184126854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8044101940 msgr2=0x7f8044197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.755+0000 7f804979d700 1 --2- 172.21.15.71:0/4184126854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8044101940 0x7f8044197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.755+0000 7f804979d700 1 -- 172.21.15.71:0/4184126854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80400095f0 con 0x7f80441002d0 2024-04-23T02:24:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.755+0000 7f804979d700 1 --2- 172.21.15.71:0/4184126854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80441002d0 0x7f80441974e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f80400092e0 tx=0x7f804000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.755+0000 7f803a7fc700 1 -- 172.21.15.71:0/4184126854 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f804001d070 con 0x7f80441002d0 2024-04-23T02:24:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.755+0000 7f804a79f700 1 -- 172.21.15.71:0/4184126854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80441a0550 con 0x7f80441002d0 2024-04-23T02:24:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.755+0000 7f803a7fc700 1 -- 172.21.15.71:0/4184126854 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8040005940 con 0x7f80441002d0 2024-04-23T02:24:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.755+0000 7f803a7fc700 1 -- 172.21.15.71:0/4184126854 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f804000fe50 con 0x7f80441002d0 2024-04-23T02:24:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.755+0000 7f804a79f700 1 -- 172.21.15.71:0/4184126854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80441a0960 con 0x7f80441002d0 2024-04-23T02:24:19.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.756+0000 7f804a79f700 1 -- 172.21.15.71:0/4184126854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80440623c0 con 0x7f80441002d0 2024-04-23T02:24:19.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.757+0000 7f803a7fc700 1 -- 172.21.15.71:0/4184126854 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8040003a60 con 0x7f80441002d0 2024-04-23T02:24:19.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.758+0000 7f803a7fc700 1 --2- 172.21.15.71:0/4184126854 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f803006cce0 0x7f803006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:19.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.758+0000 7f803a7fc700 1 -- 172.21.15.71:0/4184126854 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f804008db90 con 0x7f80441002d0 2024-04-23T02:24:19.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.758+0000 7f8048f9c700 1 --2- 172.21.15.71:0/4184126854 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f803006cce0 0x7f803006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:19.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.758+0000 7f8048f9c700 1 --2- 172.21.15.71:0/4184126854 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f803006cce0 0x7f803006f1a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8034005950 tx=0x7f80340058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:19.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.762+0000 7f803a7fc700 1 -- 172.21.15.71:0/4184126854 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8040057c00 con 0x7f80441002d0 2024-04-23T02:24:19.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.966+0000 7f804a79f700 1 -- 172.21.15.71:0/4184126854 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f80440ff8c0 con 0x7f803006cce0 2024-04-23T02:24:19.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.977+0000 7f803a7fc700 1 -- 172.21.15.71:0/4184126854 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f80440ff8c0 con 0x7f803006cce0 2024-04-23T02:24:19.980 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:24:19.980 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (8m) 104s ago 19m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:24:19.980 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (19m) 104s ago 19m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:24:19.980 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (18m) 87s ago 18m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:24:19.980 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (5m) 104s ago 19m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:24:19.980 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (5m) 87s ago 18m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:24:19.980 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (7m) 104s ago 19m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:24:19.980 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (2m) 104s ago 13m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:24:19.980 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (110s) 104s ago 13m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:24:19.980 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (13m) 87s ago 13m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:24:19.980 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (90s) 87s ago 13m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:24:19.981 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (9m) 104s ago 20m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:24:19.981 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (9m) 87s ago 18m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:24:19.981 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (6m) 104s ago 21m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:24:19.981 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (6m) 87s ago 18m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:24:19.981 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (8m) 104s ago 19m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:24:19.981 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (8m) 87s ago 18m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:24:19.981 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (5m) 104s ago 17m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:24:19.981 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (4m) 104s ago 16m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:24:19.981 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (4m) 104s ago 16m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:24:19.981 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (3m) 87s ago 15m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:24:19.981 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (3m) 87s ago 15m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:24:19.981 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (2m) 87s ago 14m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:24:19.981 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (8m) 104s ago 18m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:24:19.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.979+0000 7f804a79f700 1 -- 172.21.15.71:0/4184126854 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f803006cce0 msgr2=0x7f803006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:19.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.979+0000 7f804a79f700 1 --2- 172.21.15.71:0/4184126854 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f803006cce0 0x7f803006f1a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8034005950 tx=0x7f80340058e0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.979+0000 7f804a79f700 1 -- 172.21.15.71:0/4184126854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80441002d0 msgr2=0x7f80441974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:19.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.979+0000 7f804a79f700 1 --2- 172.21.15.71:0/4184126854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80441002d0 0x7f80441974e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f80400092e0 tx=0x7f804000be20 comp rx=0 tx=0).stop 2024-04-23T02:24:19.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.979+0000 7f804a79f700 1 -- 172.21.15.71:0/4184126854 shutdown_connections 2024-04-23T02:24:19.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.979+0000 7f804a79f700 1 --2- 172.21.15.71:0/4184126854 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f803006cce0 0x7f803006f1a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.979+0000 7f804a79f700 1 --2- 172.21.15.71:0/4184126854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80441002d0 0x7f80441974e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.979+0000 7f804a79f700 1 --2- 172.21.15.71:0/4184126854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8044101940 0x7f8044197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:19.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.979+0000 7f804a79f700 1 -- 172.21.15.71:0/4184126854 >> 172.21.15.71:0/4184126854 conn(0x7f8044074c20 msgr2=0x7f80440fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:19.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.979+0000 7f804a79f700 1 -- 172.21.15.71:0/4184126854 shutdown_connections 2024-04-23T02:24:19.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:19.979+0000 7f804a79f700 1 -- 172.21.15.71:0/4184126854 wait complete. 2024-04-23T02:24:20.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.081+0000 7f26d2778700 1 -- 172.21.15.71:0/2153855988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26cc105bf0 msgr2=0x7f26cc105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:20.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.081+0000 7f26d2778700 1 --2- 172.21.15.71:0/2153855988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26cc105bf0 0x7f26cc105fd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009910 tx=0x7f26bc009c20 comp rx=0 tx=0).stop 2024-04-23T02:24:20.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.081+0000 7f26d2778700 1 -- 172.21.15.71:0/2153855988 shutdown_connections 2024-04-23T02:24:20.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.081+0000 7f26d2778700 1 --2- 172.21.15.71:0/2153855988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26cc0ffb40 0x7f26cc0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:20.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.081+0000 7f26d2778700 1 --2- 172.21.15.71:0/2153855988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26cc105bf0 0x7f26cc105fd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:20.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.081+0000 7f26d2778700 1 -- 172.21.15.71:0/2153855988 >> 172.21.15.71:0/2153855988 conn(0x7f26cc0fb610 msgr2=0x7f26cc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:20.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.081+0000 7f26d2778700 1 -- 172.21.15.71:0/2153855988 shutdown_connections 2024-04-23T02:24:20.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.081+0000 7f26d2778700 1 -- 172.21.15.71:0/2153855988 wait complete. 2024-04-23T02:24:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.082+0000 7f26d2778700 1 Processor -- start 2024-04-23T02:24:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.082+0000 7f26d2778700 1 -- start start 2024-04-23T02:24:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.082+0000 7f26d2778700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26cc0ffb40 0x7f26cc1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.082+0000 7f26d2778700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26cc105bf0 0x7f26cc101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.082+0000 7f26d2778700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc102230 con 0x7f26cc105bf0 2024-04-23T02:24:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.082+0000 7f26d2778700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc102370 con 0x7f26cc0ffb40 2024-04-23T02:24:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.082+0000 7f26cb7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26cc105bf0 0x7f26cc101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.082+0000 7f26cb7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26cc105bf0 0x7f26cc101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38492/0 (socket says 172.21.15.71:38492) 2024-04-23T02:24:20.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.082+0000 7f26cb7fe700 1 -- 172.21.15.71:0/420836165 learned_addr learned my addr 172.21.15.71:0/420836165 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:24:20.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.082+0000 7f26cbfff700 1 --2- 172.21.15.71:0/420836165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26cc0ffb40 0x7f26cc1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:20.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.082+0000 7f26cb7fe700 1 -- 172.21.15.71:0/420836165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26cc0ffb40 msgr2=0x7f26cc1017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:20.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.082+0000 7f26cb7fe700 1 --2- 172.21.15.71:0/420836165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26cc0ffb40 0x7f26cc1017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:20.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.082+0000 7f26cb7fe700 1 -- 172.21.15.71:0/420836165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc0095f0 con 0x7f26cc105bf0 2024-04-23T02:24:20.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.082+0000 7f26cbfff700 1 --2- 172.21.15.71:0/420836165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26cc0ffb40 0x7f26cc1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:24:20.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.083+0000 7f26cb7fe700 1 --2- 172.21.15.71:0/420836165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26cc105bf0 0x7f26cc101cf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f26c000e940 tx=0x7f26c000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:20.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.083+0000 7f26c97fa700 1 -- 172.21.15.71:0/420836165 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26c00097b0 con 0x7f26cc105bf0 2024-04-23T02:24:20.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.083+0000 7f26d2778700 1 -- 172.21.15.71:0/420836165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26cc1a2920 con 0x7f26cc105bf0 2024-04-23T02:24:20.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.083+0000 7f26c97fa700 1 -- 172.21.15.71:0/420836165 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f26c0004830 con 0x7f26cc105bf0 2024-04-23T02:24:20.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.083+0000 7f26c97fa700 1 -- 172.21.15.71:0/420836165 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26c00055b0 con 0x7f26cc105bf0 2024-04-23T02:24:20.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.083+0000 7f26d2778700 1 -- 172.21.15.71:0/420836165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26cc1a2d30 con 0x7f26cc105bf0 2024-04-23T02:24:20.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.084+0000 7f26d2778700 1 -- 172.21.15.71:0/420836165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26cc04f2e0 con 0x7f26cc105bf0 2024-04-23T02:24:20.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.084+0000 7f26c97fa700 1 -- 172.21.15.71:0/420836165 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f26c00049a0 con 0x7f26cc105bf0 2024-04-23T02:24:20.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.085+0000 7f26c97fa700 1 --2- 172.21.15.71:0/420836165 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f26b406cce0 0x7f26b406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:20.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.085+0000 7f26c97fa700 1 -- 172.21.15.71:0/420836165 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f26c0014070 con 0x7f26cc105bf0 2024-04-23T02:24:20.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.085+0000 7f26cbfff700 1 --2- 172.21.15.71:0/420836165 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f26b406cce0 0x7f26b406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:20.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.085+0000 7f26cbfff700 1 --2- 172.21.15.71:0/420836165 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f26b406cce0 0x7f26b406f1a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f26bc0059d0 tx=0x7f26bc005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:20.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.089+0000 7f26c97fa700 1 -- 172.21.15.71:0/420836165 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f26c00569f0 con 0x7f26cc105bf0 2024-04-23T02:24:20.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:19 smithi192 ceph-mon[108617]: pgmap v330: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:20.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:19 smithi192 ceph-mon[108617]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:24:20.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:24:20.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:19 smithi071 ceph-mon[144544]: pgmap v330: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:20.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:19 smithi071 ceph-mon[144544]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:24:20.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:24:20.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.362+0000 7f26d2778700 1 -- 172.21.15.71:0/420836165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f26cc102ce0 con 0x7f26cc105bf0 2024-04-23T02:24:20.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.362+0000 7f26c97fa700 1 -- 172.21.15.71:0/420836165 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f26c005a010 con 0x7f26cc105bf0 2024-04-23T02:24:20.365 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:24:20.365 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:24:20.366 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:24:20.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:24:20.366 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:24:20.366 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:24:20.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:24:20.366 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:24:20.366 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:24:20.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:24:20.366 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:24:20.366 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:24:20.366 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:24:20.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:24:20.366 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:24:20.367 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:24:20.367 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:24:20.367 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:24:20.367 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:24:20.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.364+0000 7f26d2778700 1 -- 172.21.15.71:0/420836165 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f26b406cce0 msgr2=0x7f26b406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:20.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.364+0000 7f26d2778700 1 --2- 172.21.15.71:0/420836165 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f26b406cce0 0x7f26b406f1a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f26bc0059d0 tx=0x7f26bc005940 comp rx=0 tx=0).stop 2024-04-23T02:24:20.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.364+0000 7f26d2778700 1 -- 172.21.15.71:0/420836165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26cc105bf0 msgr2=0x7f26cc101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:20.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.364+0000 7f26d2778700 1 --2- 172.21.15.71:0/420836165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26cc105bf0 0x7f26cc101cf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f26c000e940 tx=0x7f26c000ed00 comp rx=0 tx=0).stop 2024-04-23T02:24:20.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.364+0000 7f26d2778700 1 -- 172.21.15.71:0/420836165 shutdown_connections 2024-04-23T02:24:20.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.364+0000 7f26d2778700 1 --2- 172.21.15.71:0/420836165 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f26b406cce0 0x7f26b406f1a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:20.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.364+0000 7f26d2778700 1 --2- 172.21.15.71:0/420836165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26cc0ffb40 0x7f26cc1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:20.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.364+0000 7f26d2778700 1 --2- 172.21.15.71:0/420836165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26cc105bf0 0x7f26cc101cf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:20.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.364+0000 7f26d2778700 1 -- 172.21.15.71:0/420836165 >> 172.21.15.71:0/420836165 conn(0x7f26cc0fb610 msgr2=0x7f26cc068f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:20.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.364+0000 7f26d2778700 1 -- 172.21.15.71:0/420836165 shutdown_connections 2024-04-23T02:24:20.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.364+0000 7f26d2778700 1 -- 172.21.15.71:0/420836165 wait complete. 2024-04-23T02:24:20.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.467+0000 7f12df18d700 1 -- 172.21.15.71:0/1145975445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d8105bd0 msgr2=0x7f12d8105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:20.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.467+0000 7f12df18d700 1 --2- 172.21.15.71:0/1145975445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d8105bd0 0x7f12d8105fb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009910 tx=0x7f12cc009c20 comp rx=0 tx=0).stop 2024-04-23T02:24:20.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.468+0000 7f12df18d700 1 -- 172.21.15.71:0/1145975445 shutdown_connections 2024-04-23T02:24:20.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.468+0000 7f12df18d700 1 --2- 172.21.15.71:0/1145975445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12d80ffc20 0x7f12d8100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:20.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.468+0000 7f12df18d700 1 --2- 172.21.15.71:0/1145975445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d8105bd0 0x7f12d8105fb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:20.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.468+0000 7f12df18d700 1 -- 172.21.15.71:0/1145975445 >> 172.21.15.71:0/1145975445 conn(0x7f12d8075670 msgr2=0x7f12d8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:20.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.468+0000 7f12df18d700 1 -- 172.21.15.71:0/1145975445 shutdown_connections 2024-04-23T02:24:20.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.468+0000 7f12df18d700 1 -- 172.21.15.71:0/1145975445 wait complete. 2024-04-23T02:24:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.468+0000 7f12df18d700 1 Processor -- start 2024-04-23T02:24:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.468+0000 7f12df18d700 1 -- start start 2024-04-23T02:24:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.469+0000 7f12df18d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d80ffc20 0x7f12d8197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.469+0000 7f12df18d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12d8105bd0 0x7f12d8197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.469+0000 7f12df18d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d81980d0 con 0x7f12d80ffc20 2024-04-23T02:24:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.469+0000 7f12df18d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d8198210 con 0x7f12d8105bd0 2024-04-23T02:24:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.469+0000 7f12dcf29700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d80ffc20 0x7f12d8197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.469+0000 7f12dcf29700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d80ffc20 0x7f12d8197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38514/0 (socket says 172.21.15.71:38514) 2024-04-23T02:24:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.469+0000 7f12dcf29700 1 -- 172.21.15.71:0/245654224 learned_addr learned my addr 172.21.15.71:0/245654224 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:24:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.469+0000 7f12d7fff700 1 --2- 172.21.15.71:0/245654224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12d8105bd0 0x7f12d8197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.469+0000 7f12dcf29700 1 -- 172.21.15.71:0/245654224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12d8105bd0 msgr2=0x7f12d8197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.469+0000 7f12dcf29700 1 --2- 172.21.15.71:0/245654224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12d8105bd0 0x7f12d8197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.469+0000 7f12dcf29700 1 -- 172.21.15.71:0/245654224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc0095f0 con 0x7f12d80ffc20 2024-04-23T02:24:20.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.469+0000 7f12dcf29700 1 --2- 172.21.15.71:0/245654224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d80ffc20 0x7f12d8197600 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f12cc006010 tx=0x7f12cc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:20.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.469+0000 7f12d5ffb700 1 -- 172.21.15.71:0/245654224 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12cc01d070 con 0x7f12d80ffc20 2024-04-23T02:24:20.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.469+0000 7f12d5ffb700 1 -- 172.21.15.71:0/245654224 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f12cc00f460 con 0x7f12d80ffc20 2024-04-23T02:24:20.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.469+0000 7f12df18d700 1 -- 172.21.15.71:0/245654224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d81a0670 con 0x7f12d80ffc20 2024-04-23T02:24:20.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.469+0000 7f12d5ffb700 1 -- 172.21.15.71:0/245654224 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12cc021640 con 0x7f12d80ffc20 2024-04-23T02:24:20.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.470+0000 7f12df18d700 1 -- 172.21.15.71:0/245654224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d81a0a20 con 0x7f12d80ffc20 2024-04-23T02:24:20.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.471+0000 7f12df18d700 1 -- 172.21.15.71:0/245654224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d81080f0 con 0x7f12d80ffc20 2024-04-23T02:24:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.471+0000 7f12d5ffb700 1 -- 172.21.15.71:0/245654224 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f12cc00fa20 con 0x7f12d80ffc20 2024-04-23T02:24:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.471+0000 7f12d5ffb700 1 --2- 172.21.15.71:0/245654224 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f12c806cce0 0x7f12c806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.472+0000 7f12d7fff700 1 --2- 172.21.15.71:0/245654224 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f12c806cce0 0x7f12c806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.472+0000 7f12d5ffb700 1 -- 172.21.15.71:0/245654224 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f12cc08d850 con 0x7f12d80ffc20 2024-04-23T02:24:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.472+0000 7f12d7fff700 1 --2- 172.21.15.71:0/245654224 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f12c806cce0 0x7f12c806f1a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f12c4005ea0 tx=0x7f12c4005e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:20.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.476+0000 7f12d5ffb700 1 -- 172.21.15.71:0/245654224 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f12cc0578a0 con 0x7f12d80ffc20 2024-04-23T02:24:20.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.712+0000 7f12df18d700 1 -- 172.21.15.71:0/245654224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f12d8107ad0 con 0x7f12d80ffc20 2024-04-23T02:24:20.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.713+0000 7f12d5ffb700 1 -- 172.21.15.71:0/245654224 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f12cc057020 con 0x7f12d80ffc20 2024-04-23T02:24:20.716 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:24:20.716 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:24:20.716 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:24:20.716 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:24:20.716 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:24:20.716 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:24:20.716 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:24:20.716 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:24:20.716 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:24:20.716 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:24:20.716 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:24:20.716 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:24:20.717 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:24:20.717 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:24:20.717 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:24:20.717 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:24:20.717 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:24:20.717 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:24:20.717 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:24:20.717 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:24:20.717 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:24:20.717 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:24:20.717 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:24:20.717 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:24:20.717 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:24:20.717 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:24:20.718 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:24:20.718 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:24:20.718 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:24:20.718 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:24:20.718 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:24:20.718 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:24:20.718 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:24:20.718 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:24:20.718 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:24:20.718 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:24:20.718 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:24:20.718 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:24:20.718 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:24:20.718 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:24:20.718 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:24:20.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.715+0000 7f12df18d700 1 -- 172.21.15.71:0/245654224 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f12c806cce0 msgr2=0x7f12c806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:20.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.715+0000 7f12df18d700 1 --2- 172.21.15.71:0/245654224 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f12c806cce0 0x7f12c806f1a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f12c4005ea0 tx=0x7f12c4005e30 comp rx=0 tx=0).stop 2024-04-23T02:24:20.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.715+0000 7f12df18d700 1 -- 172.21.15.71:0/245654224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d80ffc20 msgr2=0x7f12d8197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:20.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.715+0000 7f12df18d700 1 --2- 172.21.15.71:0/245654224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d80ffc20 0x7f12d8197600 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f12cc006010 tx=0x7f12cc00be20 comp rx=0 tx=0).stop 2024-04-23T02:24:20.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.715+0000 7f12df18d700 1 -- 172.21.15.71:0/245654224 shutdown_connections 2024-04-23T02:24:20.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.715+0000 7f12df18d700 1 --2- 172.21.15.71:0/245654224 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f12c806cce0 0x7f12c806f1a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:20.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.715+0000 7f12df18d700 1 --2- 172.21.15.71:0/245654224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12d80ffc20 0x7f12d8197600 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:20.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.715+0000 7f12df18d700 1 --2- 172.21.15.71:0/245654224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12d8105bd0 0x7f12d8197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:20.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.715+0000 7f12df18d700 1 -- 172.21.15.71:0/245654224 >> 172.21.15.71:0/245654224 conn(0x7f12d8075670 msgr2=0x7f12d80fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:20.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.715+0000 7f12df18d700 1 -- 172.21.15.71:0/245654224 shutdown_connections 2024-04-23T02:24:20.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.715+0000 7f12df18d700 1 -- 172.21.15.71:0/245654224 wait complete. 2024-04-23T02:24:20.720 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:24:20.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.819+0000 7fa443271700 1 -- 172.21.15.71:0/601375662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa43c100ea0 msgr2=0x7fa43c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:20.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.819+0000 7fa443271700 1 --2- 172.21.15.71:0/601375662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa43c100ea0 0x7fa43c101280 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009910 tx=0x7fa42c009c20 comp rx=0 tx=0).stop 2024-04-23T02:24:20.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.819+0000 7fa443271700 1 -- 172.21.15.71:0/601375662 shutdown_connections 2024-04-23T02:24:20.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.819+0000 7fa443271700 1 --2- 172.21.15.71:0/601375662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa43c101850 0x7fa43c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:20.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.819+0000 7fa443271700 1 --2- 172.21.15.71:0/601375662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa43c100ea0 0x7fa43c101280 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:20.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.819+0000 7fa443271700 1 -- 172.21.15.71:0/601375662 >> 172.21.15.71:0/601375662 conn(0x7fa43c0fc750 msgr2=0x7fa43c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:20.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.819+0000 7fa443271700 1 -- 172.21.15.71:0/601375662 shutdown_connections 2024-04-23T02:24:20.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.819+0000 7fa443271700 1 -- 172.21.15.71:0/601375662 wait complete. 2024-04-23T02:24:20.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.820+0000 7fa443271700 1 Processor -- start 2024-04-23T02:24:20.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.820+0000 7fa443271700 1 -- start start 2024-04-23T02:24:20.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.820+0000 7fa443271700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa43c100ea0 0x7fa43c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:20.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.820+0000 7fa443271700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa43c101850 0x7fa43c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:20.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.820+0000 7fa443271700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c19d130 con 0x7fa43c101850 2024-04-23T02:24:20.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.820+0000 7fa443271700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c1965e0 con 0x7fa43c100ea0 2024-04-23T02:24:20.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.820+0000 7fa44080c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa43c101850 0x7fa43c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:20.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.820+0000 7fa44080c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa43c101850 0x7fa43c19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38518/0 (socket says 172.21.15.71:38518) 2024-04-23T02:24:20.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.820+0000 7fa44080c700 1 -- 172.21.15.71:0/2769627229 learned_addr learned my addr 172.21.15.71:0/2769627229 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:24:20.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.820+0000 7fa44080c700 1 -- 172.21.15.71:0/2769627229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa43c100ea0 msgr2=0x7fa43c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:20.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.820+0000 7fa44080c700 1 --2- 172.21.15.71:0/2769627229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa43c100ea0 0x7fa43c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:20.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.820+0000 7fa44080c700 1 -- 172.21.15.71:0/2769627229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42c0095f0 con 0x7fa43c101850 2024-04-23T02:24:20.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.820+0000 7fa44080c700 1 --2- 172.21.15.71:0/2769627229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa43c101850 0x7fa43c19caa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa43800c8f0 tx=0x7fa43800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:20.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.821+0000 7fa4327fc700 1 -- 172.21.15.71:0/2769627229 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa438007a50 con 0x7fa43c101850 2024-04-23T02:24:20.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.821+0000 7fa4327fc700 1 -- 172.21.15.71:0/2769627229 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa438007bb0 con 0x7fa43c101850 2024-04-23T02:24:20.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.821+0000 7fa443271700 1 -- 172.21.15.71:0/2769627229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43c1968c0 con 0x7fa43c101850 2024-04-23T02:24:20.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.821+0000 7fa4327fc700 1 -- 172.21.15.71:0/2769627229 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa438018630 con 0x7fa43c101850 2024-04-23T02:24:20.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.821+0000 7fa443271700 1 -- 172.21.15.71:0/2769627229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43c196e10 con 0x7fa43c101850 2024-04-23T02:24:20.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.822+0000 7fa443271700 1 -- 172.21.15.71:0/2769627229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43c109240 con 0x7fa43c101850 2024-04-23T02:24:20.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.822+0000 7fa4327fc700 1 -- 172.21.15.71:0/2769627229 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa438007d20 con 0x7fa43c101850 2024-04-23T02:24:20.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.822+0000 7fa4327fc700 1 --2- 172.21.15.71:0/2769627229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa42806cce0 0x7fa42806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:20.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.823+0000 7fa4327fc700 1 -- 172.21.15.71:0/2769627229 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa43808c590 con 0x7fa43c101850 2024-04-23T02:24:20.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.823+0000 7fa44100d700 1 --2- 172.21.15.71:0/2769627229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa42806cce0 0x7fa42806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:20.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.823+0000 7fa44100d700 1 --2- 172.21.15.71:0/2769627229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa42806cce0 0x7fa42806f1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00b5c0 tx=0x7fa42c005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:20.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:20.827+0000 7fa4327fc700 1 -- 172.21.15.71:0/2769627229 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa43805a090 con 0x7fa43c101850 2024-04-23T02:24:21.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.040+0000 7fa443271700 1 -- 172.21.15.71:0/2769627229 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa43c197090 con 0x7fa42806cce0 2024-04-23T02:24:21.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.041+0000 7fa4327fc700 1 -- 172.21.15.71:0/2769627229 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa43c197090 con 0x7fa42806cce0 2024-04-23T02:24:21.044 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:24:21.044 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:24:21.044 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:24:21.044 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:24:21.044 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:24:21.044 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:24:21.044 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:24:21.044 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:24:21.044 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:24:21.044 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:24:21.045 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:24:21.045 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:24:21.045 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:24:21.045 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:24:21.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.043+0000 7fa443271700 1 -- 172.21.15.71:0/2769627229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa42806cce0 msgr2=0x7fa42806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:21.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.043+0000 7fa443271700 1 --2- 172.21.15.71:0/2769627229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa42806cce0 0x7fa42806f1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00b5c0 tx=0x7fa42c005960 comp rx=0 tx=0).stop 2024-04-23T02:24:21.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.043+0000 7fa443271700 1 -- 172.21.15.71:0/2769627229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa43c101850 msgr2=0x7fa43c19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:21.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.043+0000 7fa443271700 1 --2- 172.21.15.71:0/2769627229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa43c101850 0x7fa43c19caa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa43800c8f0 tx=0x7fa43800ccb0 comp rx=0 tx=0).stop 2024-04-23T02:24:21.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.043+0000 7fa443271700 1 -- 172.21.15.71:0/2769627229 shutdown_connections 2024-04-23T02:24:21.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.043+0000 7fa443271700 1 --2- 172.21.15.71:0/2769627229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa42806cce0 0x7fa42806f1a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:21.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.043+0000 7fa443271700 1 --2- 172.21.15.71:0/2769627229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa43c100ea0 0x7fa43c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:21.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.043+0000 7fa443271700 1 --2- 172.21.15.71:0/2769627229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa43c101850 0x7fa43c19caa0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:21.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.043+0000 7fa443271700 1 -- 172.21.15.71:0/2769627229 >> 172.21.15.71:0/2769627229 conn(0x7fa43c0fc750 msgr2=0x7fa43c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:21.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.043+0000 7fa443271700 1 -- 172.21.15.71:0/2769627229 shutdown_connections 2024-04-23T02:24:21.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.043+0000 7fa443271700 1 -- 172.21.15.71:0/2769627229 wait complete. 2024-04-23T02:24:21.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:20 smithi192 ceph-mon[108617]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:24:21.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:20 smithi192 ceph-mon[108617]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:24:21.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:20 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/420836165' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:24:21.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:20 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/245654224' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:24:21.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.146+0000 7f69a4114700 1 -- 172.21.15.71:0/3474171707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f699c0ffc20 msgr2=0x7f699c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:21.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.146+0000 7f69a4114700 1 --2- 172.21.15.71:0/3474171707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f699c0ffc20 0x7f699c100080 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6998009910 tx=0x7f6998009c20 comp rx=0 tx=0).stop 2024-04-23T02:24:21.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.146+0000 7f69a4114700 1 -- 172.21.15.71:0/3474171707 shutdown_connections 2024-04-23T02:24:21.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.146+0000 7f69a4114700 1 --2- 172.21.15.71:0/3474171707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f699c0ffc20 0x7f699c100080 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:21.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.146+0000 7f69a4114700 1 --2- 172.21.15.71:0/3474171707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f699c105bd0 0x7f699c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:21.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.146+0000 7f69a4114700 1 -- 172.21.15.71:0/3474171707 >> 172.21.15.71:0/3474171707 conn(0x7f699c075670 msgr2=0x7f699c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:21.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.146+0000 7f69a4114700 1 -- 172.21.15.71:0/3474171707 shutdown_connections 2024-04-23T02:24:21.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.146+0000 7f69a4114700 1 -- 172.21.15.71:0/3474171707 wait complete. 2024-04-23T02:24:21.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.147+0000 7f69a4114700 1 Processor -- start 2024-04-23T02:24:21.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.147+0000 7f69a4114700 1 -- start start 2024-04-23T02:24:21.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.147+0000 7f69a4114700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f699c0ffc20 0x7f699c19c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:21.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.147+0000 7f69a4114700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f699c105bd0 0x7f699c19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:21.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.147+0000 7f69a4114700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c19d140 con 0x7f699c105bd0 2024-04-23T02:24:21.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.147+0000 7f69a4114700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c1965f0 con 0x7f699c0ffc20 2024-04-23T02:24:21.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.147+0000 7f69a16af700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f699c105bd0 0x7f699c19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:21.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.147+0000 7f69a16af700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f699c105bd0 0x7f699c19cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38548/0 (socket says 172.21.15.71:38548) 2024-04-23T02:24:21.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.147+0000 7f69a1eb0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f699c0ffc20 0x7f699c19c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:21.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.147+0000 7f69a16af700 1 -- 172.21.15.71:0/3258576506 learned_addr learned my addr 172.21.15.71:0/3258576506 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:24:21.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.148+0000 7f69a16af700 1 -- 172.21.15.71:0/3258576506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f699c0ffc20 msgr2=0x7f699c19c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:21.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.148+0000 7f69a16af700 1 --2- 172.21.15.71:0/3258576506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f699c0ffc20 0x7f699c19c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:21.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.148+0000 7f69a16af700 1 -- 172.21.15.71:0/3258576506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69980095f0 con 0x7f699c105bd0 2024-04-23T02:24:21.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.148+0000 7f69a16af700 1 --2- 172.21.15.71:0/3258576506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f699c105bd0 0x7f699c19cab0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6998006010 tx=0x7f699800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:21.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.148+0000 7f698effd700 1 -- 172.21.15.71:0/3258576506 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f699801d070 con 0x7f699c105bd0 2024-04-23T02:24:21.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.148+0000 7f69a4114700 1 -- 172.21.15.71:0/3258576506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f699c196870 con 0x7f699c105bd0 2024-04-23T02:24:21.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.148+0000 7f698effd700 1 -- 172.21.15.71:0/3258576506 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f699800f460 con 0x7f699c105bd0 2024-04-23T02:24:21.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.148+0000 7f698effd700 1 -- 172.21.15.71:0/3258576506 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6998021620 con 0x7f699c105bd0 2024-04-23T02:24:21.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.148+0000 7f69a4114700 1 -- 172.21.15.71:0/3258576506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699c196d60 con 0x7f699c105bd0 2024-04-23T02:24:21.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.149+0000 7f69a4114700 1 -- 172.21.15.71:0/3258576506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f699c1080d0 con 0x7f699c105bd0 2024-04-23T02:24:21.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.149+0000 7f698effd700 1 -- 172.21.15.71:0/3258576506 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f699800f5d0 con 0x7f699c105bd0 2024-04-23T02:24:21.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.150+0000 7f698effd700 1 --2- 172.21.15.71:0/3258576506 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f698806cce0 0x7f698806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:21.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.150+0000 7f698effd700 1 -- 172.21.15.71:0/3258576506 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f699808d850 con 0x7f699c105bd0 2024-04-23T02:24:21.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.150+0000 7f69a1eb0700 1 --2- 172.21.15.71:0/3258576506 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f698806cce0 0x7f698806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:21.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.150+0000 7f69a1eb0700 1 --2- 172.21.15.71:0/3258576506 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f698806cce0 0x7f698806f1a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6990005950 tx=0x7f699000a300 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:21.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.154+0000 7f698effd700 1 -- 172.21.15.71:0/3258576506 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f699805b350 con 0x7f699c105bd0 2024-04-23T02:24:21.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:20 smithi071 ceph-mon[144544]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:24:21.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:20 smithi071 ceph-mon[144544]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:24:21.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:20 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/420836165' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:24:21.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:20 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/245654224' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:24:21.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.426+0000 7f69a4114700 1 -- 172.21.15.71:0/3258576506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f699c107ab0 con 0x7f699c105bd0 2024-04-23T02:24:21.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.427+0000 7f698effd700 1 -- 172.21.15.71:0/3258576506 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f699805aee0 con 0x7f699c105bd0 2024-04-23T02:24:21.430 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:24:21.430 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:24:21.430 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:24:21.430 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:24:21.430 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:24:21.430 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:24:21.430 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:24:21.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.428+0000 7f69a4114700 1 -- 172.21.15.71:0/3258576506 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f698806cce0 msgr2=0x7f698806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:21.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.428+0000 7f69a4114700 1 --2- 172.21.15.71:0/3258576506 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f698806cce0 0x7f698806f1a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6990005950 tx=0x7f699000a300 comp rx=0 tx=0).stop 2024-04-23T02:24:21.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.428+0000 7f69a4114700 1 -- 172.21.15.71:0/3258576506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f699c105bd0 msgr2=0x7f699c19cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:21.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.428+0000 7f69a4114700 1 --2- 172.21.15.71:0/3258576506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f699c105bd0 0x7f699c19cab0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6998006010 tx=0x7f699800bd70 comp rx=0 tx=0).stop 2024-04-23T02:24:21.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.429+0000 7f69a4114700 1 -- 172.21.15.71:0/3258576506 shutdown_connections 2024-04-23T02:24:21.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.429+0000 7f69a4114700 1 --2- 172.21.15.71:0/3258576506 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f698806cce0 0x7f698806f1a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:21.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.429+0000 7f69a4114700 1 --2- 172.21.15.71:0/3258576506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f699c0ffc20 0x7f699c19c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:21.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.429+0000 7f69a4114700 1 --2- 172.21.15.71:0/3258576506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f699c105bd0 0x7f699c19cab0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:21.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.429+0000 7f69a4114700 1 -- 172.21.15.71:0/3258576506 >> 172.21.15.71:0/3258576506 conn(0x7f699c075670 msgr2=0x7f699c0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:21.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.429+0000 7f69a4114700 1 -- 172.21.15.71:0/3258576506 shutdown_connections 2024-04-23T02:24:21.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:21.429+0000 7f69a4114700 1 -- 172.21.15.71:0/3258576506 wait complete. 2024-04-23T02:24:22.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:21 smithi192 ceph-mon[108617]: pgmap v331: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:22.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:21 smithi192 ceph-mon[108617]: from='client.34474 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:24:22.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:21 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3258576506' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:24:22.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:21 smithi071 ceph-mon[144544]: pgmap v331: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:22.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:21 smithi071 ceph-mon[144544]: from='client.34474 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:24:22.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:21 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3258576506' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:24:24.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:23 smithi192 ceph-mon[108617]: pgmap v332: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:24.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:23 smithi071 ceph-mon[144544]: pgmap v332: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:26.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:25 smithi192 ceph-mon[108617]: pgmap v333: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:26.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:25 smithi071 ceph-mon[144544]: pgmap v333: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:28.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:27 smithi192 ceph-mon[108617]: pgmap v334: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:28.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:27 smithi071 ceph-mon[144544]: pgmap v334: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:30.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:29 smithi071 ceph-mon[144544]: pgmap v335: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:30.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:29 smithi192 ceph-mon[108617]: pgmap v335: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:32.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:31 smithi071 ceph-mon[144544]: pgmap v336: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:32.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:31 smithi192 ceph-mon[108617]: pgmap v336: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:33.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:33 smithi071 ceph-mon[144544]: pgmap v337: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:34.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:33 smithi192 ceph-mon[108617]: pgmap v337: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:35.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:34 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:24:35.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:34 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:24:36.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:35 smithi071 ceph-mon[144544]: pgmap v338: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:36.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:35 smithi192 ceph-mon[108617]: pgmap v338: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:38.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:37 smithi071 ceph-mon[144544]: pgmap v339: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:38.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:37 smithi192 ceph-mon[108617]: pgmap v339: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:40.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:39 smithi071 ceph-mon[144544]: pgmap v340: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:40.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:39 smithi192 ceph-mon[108617]: pgmap v340: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:42.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:41 smithi071 ceph-mon[144544]: pgmap v341: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:42.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:41 smithi192 ceph-mon[108617]: pgmap v341: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:44.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:43 smithi192 ceph-mon[108617]: pgmap v342: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:44.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:43 smithi071 ceph-mon[144544]: pgmap v342: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:46.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:45 smithi071 ceph-mon[144544]: pgmap v343: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:46.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:45 smithi192 ceph-mon[108617]: pgmap v343: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:48.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:47 smithi071 ceph-mon[144544]: pgmap v344: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:48.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:47 smithi192 ceph-mon[108617]: pgmap v344: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:50.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:49 smithi071 ceph-mon[144544]: pgmap v345: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:50.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:24:50.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:49 smithi192 ceph-mon[108617]: pgmap v345: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:50.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:24:51.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.543+0000 7f732ab40700 1 -- 172.21.15.71:0/1563654209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7324068580 msgr2=0x7f7324068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:51.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.543+0000 7f732ab40700 1 --2- 172.21.15.71:0/1563654209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7324068580 0x7f7324068960 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7318009910 tx=0x7f7318009c20 comp rx=0 tx=0).stop 2024-04-23T02:24:51.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.543+0000 7f732ab40700 1 -- 172.21.15.71:0/1563654209 shutdown_connections 2024-04-23T02:24:51.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.543+0000 7f732ab40700 1 --2- 172.21.15.71:0/1563654209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7324068ea0 0x7f732410d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:51.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.543+0000 7f732ab40700 1 --2- 172.21.15.71:0/1563654209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7324068580 0x7f7324068960 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:51.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.543+0000 7f732ab40700 1 -- 172.21.15.71:0/1563654209 >> 172.21.15.71:0/1563654209 conn(0x7f7324075670 msgr2=0x7f7324075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:51.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.543+0000 7f732ab40700 1 -- 172.21.15.71:0/1563654209 shutdown_connections 2024-04-23T02:24:51.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.543+0000 7f732ab40700 1 -- 172.21.15.71:0/1563654209 wait complete. 2024-04-23T02:24:51.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.544+0000 7f732ab40700 1 Processor -- start 2024-04-23T02:24:51.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.544+0000 7f732ab40700 1 -- start start 2024-04-23T02:24:51.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.544+0000 7f732ab40700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7324068580 0x7f7324198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:51.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.544+0000 7f732ab40700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7324068ea0 0x7f7324198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:51.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.544+0000 7f732ab40700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7324199550 con 0x7f7324068580 2024-04-23T02:24:51.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.544+0000 7f732ab40700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732419d360 con 0x7f7324068ea0 2024-04-23T02:24:51.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.544+0000 7f73288dc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7324068580 0x7f7324198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:51.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.544+0000 7f73288dc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7324068580 0x7f7324198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57112/0 (socket says 172.21.15.71:57112) 2024-04-23T02:24:51.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.544+0000 7f73288dc700 1 -- 172.21.15.71:0/3151367859 learned_addr learned my addr 172.21.15.71:0/3151367859 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:24:51.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.544+0000 7f7323fff700 1 --2- 172.21.15.71:0/3151367859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7324068ea0 0x7f7324198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:51.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.544+0000 7f73288dc700 1 -- 172.21.15.71:0/3151367859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7324068ea0 msgr2=0x7f7324198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:51.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.544+0000 7f73288dc700 1 --2- 172.21.15.71:0/3151367859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7324068ea0 0x7f7324198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:51.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.544+0000 7f73288dc700 1 -- 172.21.15.71:0/3151367859 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73180095f0 con 0x7f7324068580 2024-04-23T02:24:51.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.544+0000 7f7323fff700 1 --2- 172.21.15.71:0/3151367859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7324068ea0 0x7f7324198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:24:51.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.545+0000 7f73288dc700 1 --2- 172.21.15.71:0/3151367859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7324068580 0x7f7324198980 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7318006010 tx=0x7f731800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:51.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.545+0000 7f7321ffb700 1 -- 172.21.15.71:0/3151367859 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f731801d070 con 0x7f7324068580 2024-04-23T02:24:51.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.545+0000 7f732ab40700 1 -- 172.21.15.71:0/3151367859 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732419d5e0 con 0x7f7324068580 2024-04-23T02:24:51.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.545+0000 7f7321ffb700 1 -- 172.21.15.71:0/3151367859 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f731800f460 con 0x7f7324068580 2024-04-23T02:24:51.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.545+0000 7f7321ffb700 1 -- 172.21.15.71:0/3151367859 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7318021640 con 0x7f7324068580 2024-04-23T02:24:51.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.545+0000 7f732ab40700 1 -- 172.21.15.71:0/3151367859 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732419dad0 con 0x7f7324068580 2024-04-23T02:24:51.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.546+0000 7f732ab40700 1 -- 172.21.15.71:0/3151367859 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732410aa00 con 0x7f7324068580 2024-04-23T02:24:51.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.547+0000 7f7321ffb700 1 -- 172.21.15.71:0/3151367859 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f731800fa20 con 0x7f7324068580 2024-04-23T02:24:51.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.547+0000 7f7321ffb700 1 --2- 172.21.15.71:0/3151367859 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f731406cce0 0x7f731406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:51.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.547+0000 7f7321ffb700 1 -- 172.21.15.71:0/3151367859 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f731808d830 con 0x7f7324068580 2024-04-23T02:24:51.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.547+0000 7f7323fff700 1 --2- 172.21.15.71:0/3151367859 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f731406cce0 0x7f731406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:51.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.547+0000 7f7323fff700 1 --2- 172.21.15.71:0/3151367859 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f731406cce0 0x7f731406f1a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7324199fa0 tx=0x7f731000b5f0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:51.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.552+0000 7f7321ffb700 1 -- 172.21.15.71:0/3151367859 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f731805b330 con 0x7f7324068580 2024-04-23T02:24:51.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.760+0000 7f732ab40700 1 -- 172.21.15.71:0/3151367859 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7324199d30 con 0x7f731406cce0 2024-04-23T02:24:51.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.761+0000 7f7321ffb700 1 -- 172.21.15.71:0/3151367859 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7324199d30 con 0x7f731406cce0 2024-04-23T02:24:51.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.763+0000 7f732ab40700 1 -- 172.21.15.71:0/3151367859 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f731406cce0 msgr2=0x7f731406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:51.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.763+0000 7f732ab40700 1 --2- 172.21.15.71:0/3151367859 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f731406cce0 0x7f731406f1a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7324199fa0 tx=0x7f731000b5f0 comp rx=0 tx=0).stop 2024-04-23T02:24:51.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.763+0000 7f732ab40700 1 -- 172.21.15.71:0/3151367859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7324068580 msgr2=0x7f7324198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:51.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.763+0000 7f732ab40700 1 --2- 172.21.15.71:0/3151367859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7324068580 0x7f7324198980 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7318006010 tx=0x7f731800be20 comp rx=0 tx=0).stop 2024-04-23T02:24:51.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.763+0000 7f732ab40700 1 -- 172.21.15.71:0/3151367859 shutdown_connections 2024-04-23T02:24:51.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.764+0000 7f732ab40700 1 --2- 172.21.15.71:0/3151367859 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f731406cce0 0x7f731406f1a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:51.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.764+0000 7f732ab40700 1 --2- 172.21.15.71:0/3151367859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7324068580 0x7f7324198980 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:51.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.764+0000 7f732ab40700 1 --2- 172.21.15.71:0/3151367859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7324068ea0 0x7f7324198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:51.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.764+0000 7f732ab40700 1 -- 172.21.15.71:0/3151367859 >> 172.21.15.71:0/3151367859 conn(0x7f7324075670 msgr2=0x7f73240fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:51.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.764+0000 7f732ab40700 1 -- 172.21.15.71:0/3151367859 shutdown_connections 2024-04-23T02:24:51.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.764+0000 7f732ab40700 1 -- 172.21.15.71:0/3151367859 wait complete. 2024-04-23T02:24:51.778 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:24:51.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.867+0000 7fa601a51700 1 -- 172.21.15.71:0/1162449451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5fc105bf0 msgr2=0x7fa5fc105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:51.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.867+0000 7fa601a51700 1 --2- 172.21.15.71:0/1162449451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5fc105bf0 0x7fa5fc105fd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec009910 tx=0x7fa5ec009c20 comp rx=0 tx=0).stop 2024-04-23T02:24:51.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.867+0000 7fa601a51700 1 -- 172.21.15.71:0/1162449451 shutdown_connections 2024-04-23T02:24:51.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.868+0000 7fa601a51700 1 --2- 172.21.15.71:0/1162449451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5fc0ffb40 0x7fa5fc0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:51.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.868+0000 7fa601a51700 1 --2- 172.21.15.71:0/1162449451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5fc105bf0 0x7fa5fc105fd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:51.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.868+0000 7fa601a51700 1 -- 172.21.15.71:0/1162449451 >> 172.21.15.71:0/1162449451 conn(0x7fa5fc0fb650 msgr2=0x7fa5fc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:51.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.868+0000 7fa601a51700 1 -- 172.21.15.71:0/1162449451 shutdown_connections 2024-04-23T02:24:51.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.868+0000 7fa601a51700 1 -- 172.21.15.71:0/1162449451 wait complete. 2024-04-23T02:24:51.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.868+0000 7fa601a51700 1 Processor -- start 2024-04-23T02:24:51.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.869+0000 7fa601a51700 1 -- start start 2024-04-23T02:24:51.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.869+0000 7fa601a51700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5fc0ffb40 0x7fa5fc195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:51.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.869+0000 7fa601a51700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5fc105bf0 0x7fa5fc195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:51.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.869+0000 7fa601a51700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc196260 con 0x7fa5fc0ffb40 2024-04-23T02:24:51.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.869+0000 7fa601a51700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc199ff0 con 0x7fa5fc105bf0 2024-04-23T02:24:51.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.869+0000 7fa5faffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5fc0ffb40 0x7fa5fc195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:51.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.869+0000 7fa5faffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5fc0ffb40 0x7fa5fc195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57128/0 (socket says 172.21.15.71:57128) 2024-04-23T02:24:51.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.869+0000 7fa5faffd700 1 -- 172.21.15.71:0/815657199 learned_addr learned my addr 172.21.15.71:0/815657199 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:24:51.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.869+0000 7fa5fa7fc700 1 --2- 172.21.15.71:0/815657199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5fc105bf0 0x7fa5fc195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:51.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.869+0000 7fa5faffd700 1 -- 172.21.15.71:0/815657199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5fc105bf0 msgr2=0x7fa5fc195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:51.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.869+0000 7fa5faffd700 1 --2- 172.21.15.71:0/815657199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5fc105bf0 0x7fa5fc195b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:51.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.869+0000 7fa5faffd700 1 -- 172.21.15.71:0/815657199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec0095f0 con 0x7fa5fc0ffb40 2024-04-23T02:24:51.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.869+0000 7fa5fa7fc700 1 --2- 172.21.15.71:0/815657199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5fc105bf0 0x7fa5fc195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:24:51.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.869+0000 7fa5faffd700 1 --2- 172.21.15.71:0/815657199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5fc0ffb40 0x7fa5fc195640 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec006010 tx=0x7fa5ec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:51.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.870+0000 7fa600a4f700 1 -- 172.21.15.71:0/815657199 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5ec01f070 con 0x7fa5fc0ffb40 2024-04-23T02:24:51.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.870+0000 7fa600a4f700 1 -- 172.21.15.71:0/815657199 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa5ec019610 con 0x7fa5fc0ffb40 2024-04-23T02:24:51.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.870+0000 7fa600a4f700 1 -- 172.21.15.71:0/815657199 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5ec00f670 con 0x7fa5fc0ffb40 2024-04-23T02:24:51.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.870+0000 7fa601a51700 1 -- 172.21.15.71:0/815657199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5fc19a270 con 0x7fa5fc0ffb40 2024-04-23T02:24:51.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.870+0000 7fa601a51700 1 -- 172.21.15.71:0/815657199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5fc19a760 con 0x7fa5fc0ffb40 2024-04-23T02:24:51.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.871+0000 7fa601a51700 1 -- 172.21.15.71:0/815657199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5fc04f2e0 con 0x7fa5fc0ffb40 2024-04-23T02:24:51.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.871+0000 7fa600a4f700 1 -- 172.21.15.71:0/815657199 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa5ec019780 con 0x7fa5fc0ffb40 2024-04-23T02:24:51.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.872+0000 7fa600a4f700 1 --2- 172.21.15.71:0/815657199 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa5e406cce0 0x7fa5e406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:51.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.872+0000 7fa600a4f700 1 -- 172.21.15.71:0/815657199 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa5ec08fb90 con 0x7fa5fc0ffb40 2024-04-23T02:24:51.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.872+0000 7fa5fa7fc700 1 --2- 172.21.15.71:0/815657199 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa5e406cce0 0x7fa5e406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:51.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.872+0000 7fa5fa7fc700 1 --2- 172.21.15.71:0/815657199 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa5e406cce0 0x7fa5e406f1a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0005fd0 tx=0x7fa5f0005dc0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:51.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:51.876+0000 7fa600a4f700 1 -- 172.21.15.71:0/815657199 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa5ec059ba0 con 0x7fa5fc0ffb40 2024-04-23T02:24:52.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.087+0000 7fa601a51700 1 -- 172.21.15.71:0/815657199 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5fc19aae0 con 0x7fa5e406cce0 2024-04-23T02:24:52.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.088+0000 7fa600a4f700 1 -- 172.21.15.71:0/815657199 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa5fc19aae0 con 0x7fa5e406cce0 2024-04-23T02:24:52.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.091+0000 7fa601a51700 1 -- 172.21.15.71:0/815657199 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa5e406cce0 msgr2=0x7fa5e406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:52.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.091+0000 7fa601a51700 1 --2- 172.21.15.71:0/815657199 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa5e406cce0 0x7fa5e406f1a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0005fd0 tx=0x7fa5f0005dc0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.091+0000 7fa601a51700 1 -- 172.21.15.71:0/815657199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5fc0ffb40 msgr2=0x7fa5fc195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:52.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.091+0000 7fa601a51700 1 --2- 172.21.15.71:0/815657199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5fc0ffb40 0x7fa5fc195640 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec006010 tx=0x7fa5ec00be20 comp rx=0 tx=0).stop 2024-04-23T02:24:52.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.091+0000 7fa601a51700 1 -- 172.21.15.71:0/815657199 shutdown_connections 2024-04-23T02:24:52.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.091+0000 7fa601a51700 1 --2- 172.21.15.71:0/815657199 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa5e406cce0 0x7fa5e406f1a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.091+0000 7fa601a51700 1 --2- 172.21.15.71:0/815657199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5fc0ffb40 0x7fa5fc195640 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.091+0000 7fa601a51700 1 --2- 172.21.15.71:0/815657199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5fc105bf0 0x7fa5fc195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.091+0000 7fa601a51700 1 -- 172.21.15.71:0/815657199 >> 172.21.15.71:0/815657199 conn(0x7fa5fc0fb650 msgr2=0x7fa5fc0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:52.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.091+0000 7fa601a51700 1 -- 172.21.15.71:0/815657199 shutdown_connections 2024-04-23T02:24:52.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.091+0000 7fa601a51700 1 -- 172.21.15.71:0/815657199 wait complete. 2024-04-23T02:24:52.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.193+0000 7f3b904e0700 1 -- 172.21.15.71:0/699290751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b88101840 msgr2=0x7f3b88105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:52.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.193+0000 7f3b904e0700 1 --2- 172.21.15.71:0/699290751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b88101840 0x7f3b88105890 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009910 tx=0x7f3b84009c20 comp rx=0 tx=0).stop 2024-04-23T02:24:52.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.193+0000 7f3b904e0700 1 -- 172.21.15.71:0/699290751 shutdown_connections 2024-04-23T02:24:52.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.193+0000 7f3b904e0700 1 --2- 172.21.15.71:0/699290751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b88101840 0x7f3b88105890 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.193+0000 7f3b904e0700 1 --2- 172.21.15.71:0/699290751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b88100e90 0x7f3b88101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.193+0000 7f3b904e0700 1 -- 172.21.15.71:0/699290751 >> 172.21.15.71:0/699290751 conn(0x7f3b880fc740 msgr2=0x7f3b880feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:52.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.193+0000 7f3b904e0700 1 -- 172.21.15.71:0/699290751 shutdown_connections 2024-04-23T02:24:52.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.193+0000 7f3b904e0700 1 -- 172.21.15.71:0/699290751 wait complete. 2024-04-23T02:24:52.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.194+0000 7f3b904e0700 1 Processor -- start 2024-04-23T02:24:52.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.194+0000 7f3b904e0700 1 -- start start 2024-04-23T02:24:52.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.194+0000 7f3b904e0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b88100e90 0x7f3b88073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.194+0000 7f3b904e0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b88101840 0x7f3b880735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.194+0000 7f3b904e0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b88073b10 con 0x7f3b88100e90 2024-04-23T02:24:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.194+0000 7f3b904e0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b88073c50 con 0x7f3b88101840 2024-04-23T02:24:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.194+0000 7f3b8e27c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b88100e90 0x7f3b88073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.194+0000 7f3b8e27c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b88100e90 0x7f3b88073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57138/0 (socket says 172.21.15.71:57138) 2024-04-23T02:24:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.194+0000 7f3b8e27c700 1 -- 172.21.15.71:0/943465041 learned_addr learned my addr 172.21.15.71:0/943465041 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:24:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.194+0000 7f3b8e27c700 1 -- 172.21.15.71:0/943465041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b88101840 msgr2=0x7f3b880735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.194+0000 7f3b8e27c700 1 --2- 172.21.15.71:0/943465041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b88101840 0x7f3b880735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.194+0000 7f3b8e27c700 1 -- 172.21.15.71:0/943465041 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b840095f0 con 0x7f3b88100e90 2024-04-23T02:24:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.195+0000 7f3b8e27c700 1 --2- 172.21.15.71:0/943465041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b88100e90 0x7f3b88073090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800b4b0 tx=0x7f3b7800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.195+0000 7f3b7f7fe700 1 -- 172.21.15.71:0/943465041 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b78011800 con 0x7f3b88100e90 2024-04-23T02:24:52.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.195+0000 7f3b7f7fe700 1 -- 172.21.15.71:0/943465041 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3b78011e00 con 0x7f3b88100e90 2024-04-23T02:24:52.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.195+0000 7f3b7f7fe700 1 -- 172.21.15.71:0/943465041 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b78010e50 con 0x7f3b88100e90 2024-04-23T02:24:52.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.195+0000 7f3b904e0700 1 -- 172.21.15.71:0/943465041 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b881a2a70 con 0x7f3b88100e90 2024-04-23T02:24:52.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.195+0000 7f3b904e0700 1 -- 172.21.15.71:0/943465041 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b881a2e80 con 0x7f3b88100e90 2024-04-23T02:24:52.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.196+0000 7f3b904e0700 1 -- 172.21.15.71:0/943465041 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b881091c0 con 0x7f3b88100e90 2024-04-23T02:24:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.196+0000 7f3b7f7fe700 1 -- 172.21.15.71:0/943465041 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3b78010380 con 0x7f3b88100e90 2024-04-23T02:24:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.197+0000 7f3b7f7fe700 1 --2- 172.21.15.71:0/943465041 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3b7406cce0 0x7f3b7406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.197+0000 7f3b7f7fe700 1 -- 172.21.15.71:0/943465041 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3b7808c1b0 con 0x7f3b88100e90 2024-04-23T02:24:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.197+0000 7f3b8da7b700 1 --2- 172.21.15.71:0/943465041 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3b7406cce0 0x7f3b7406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:52.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.197+0000 7f3b8da7b700 1 --2- 172.21.15.71:0/943465041 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3b7406cce0 0x7f3b7406f1a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400b5c0 tx=0x7f3b84005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:52.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.201+0000 7f3b7f7fe700 1 -- 172.21.15.71:0/943465041 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3b78056220 con 0x7f3b88100e90 2024-04-23T02:24:52.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:51 smithi071 ceph-mon[144544]: pgmap v346: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:52.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:51 smithi192 ceph-mon[108617]: pgmap v346: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:52.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.403+0000 7f3b904e0700 1 -- 172.21.15.71:0/943465041 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b880611d0 con 0x7f3b7406cce0 2024-04-23T02:24:52.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.413+0000 7f3b7f7fe700 1 -- 172.21.15.71:0/943465041 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f3b880611d0 con 0x7f3b7406cce0 2024-04-23T02:24:52.417 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:24:52.417 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (8m) 2m ago 19m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:24:52.417 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (20m) 2m ago 20m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:24:52.417 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (18m) 119s ago 18m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:24:52.418 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (6m) 2m ago 20m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:24:52.418 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (6m) 119s ago 18m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:24:52.418 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (8m) 2m ago 19m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:24:52.418 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (2m) 2m ago 13m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:24:52.418 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (2m) 2m ago 13m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:24:52.418 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (13m) 119s ago 13m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:24:52.418 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (2m) 119s ago 13m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:24:52.418 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (10m) 2m ago 21m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:24:52.418 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (9m) 119s ago 18m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:24:52.418 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (7m) 2m ago 21m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:24:52.418 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (6m) 119s ago 18m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:24:52.418 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (9m) 2m ago 20m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:24:52.418 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (9m) 119s ago 18m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:24:52.418 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (6m) 2m ago 17m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:24:52.418 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (5m) 2m ago 17m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:24:52.419 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (4m) 2m ago 16m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:24:52.419 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (4m) 119s ago 16m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:24:52.419 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (3m) 119s ago 15m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:24:52.419 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (3m) 119s ago 15m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:24:52.419 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (8m) 2m ago 19m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:24:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.415+0000 7f3b904e0700 1 -- 172.21.15.71:0/943465041 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3b7406cce0 msgr2=0x7f3b7406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.415+0000 7f3b904e0700 1 --2- 172.21.15.71:0/943465041 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3b7406cce0 0x7f3b7406f1a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400b5c0 tx=0x7f3b84005960 comp rx=0 tx=0).stop 2024-04-23T02:24:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.415+0000 7f3b904e0700 1 -- 172.21.15.71:0/943465041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b88100e90 msgr2=0x7f3b88073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.415+0000 7f3b904e0700 1 --2- 172.21.15.71:0/943465041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b88100e90 0x7f3b88073090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800b4b0 tx=0x7f3b7800b870 comp rx=0 tx=0).stop 2024-04-23T02:24:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.415+0000 7f3b904e0700 1 -- 172.21.15.71:0/943465041 shutdown_connections 2024-04-23T02:24:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.415+0000 7f3b904e0700 1 --2- 172.21.15.71:0/943465041 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3b7406cce0 0x7f3b7406f1a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.415+0000 7f3b904e0700 1 --2- 172.21.15.71:0/943465041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b88100e90 0x7f3b88073090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.415+0000 7f3b904e0700 1 --2- 172.21.15.71:0/943465041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b88101840 0x7f3b880735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.415+0000 7f3b904e0700 1 -- 172.21.15.71:0/943465041 >> 172.21.15.71:0/943465041 conn(0x7f3b880fc740 msgr2=0x7f3b880fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.415+0000 7f3b904e0700 1 -- 172.21.15.71:0/943465041 shutdown_connections 2024-04-23T02:24:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.416+0000 7f3b904e0700 1 -- 172.21.15.71:0/943465041 wait complete. 2024-04-23T02:24:52.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.517+0000 7f86997fe700 1 -- 172.21.15.71:0/1692150737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8694105bd0 msgr2=0x7f8694105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:52.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.517+0000 7f86997fe700 1 --2- 172.21.15.71:0/1692150737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8694105bd0 0x7f8694105fb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8688009910 tx=0x7f8688009c20 comp rx=0 tx=0).stop 2024-04-23T02:24:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.517+0000 7f86997fe700 1 -- 172.21.15.71:0/1692150737 shutdown_connections 2024-04-23T02:24:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.517+0000 7f86997fe700 1 --2- 172.21.15.71:0/1692150737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86940ffc20 0x7f8694100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.517+0000 7f86997fe700 1 --2- 172.21.15.71:0/1692150737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8694105bd0 0x7f8694105fb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.517+0000 7f86997fe700 1 -- 172.21.15.71:0/1692150737 >> 172.21.15.71:0/1692150737 conn(0x7f8694075670 msgr2=0x7f8694075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.517+0000 7f86997fe700 1 -- 172.21.15.71:0/1692150737 shutdown_connections 2024-04-23T02:24:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.517+0000 7f86997fe700 1 -- 172.21.15.71:0/1692150737 wait complete. 2024-04-23T02:24:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.517+0000 7f86997fe700 1 Processor -- start 2024-04-23T02:24:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.517+0000 7f86997fe700 1 -- start start 2024-04-23T02:24:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.518+0000 7f86997fe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86940ffc20 0x7f869419c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.518+0000 7f86997fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8694105bd0 0x7f869419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.518+0000 7f86997fe700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869419d140 con 0x7f8694105bd0 2024-04-23T02:24:52.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.518+0000 7f86997fe700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86941965f0 con 0x7f86940ffc20 2024-04-23T02:24:52.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.518+0000 7f86927fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8694105bd0 0x7f869419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:52.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.518+0000 7f86927fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8694105bd0 0x7f869419cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57148/0 (socket says 172.21.15.71:57148) 2024-04-23T02:24:52.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.518+0000 7f86927fc700 1 -- 172.21.15.71:0/2695034917 learned_addr learned my addr 172.21.15.71:0/2695034917 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:24:52.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.518+0000 7f8692ffd700 1 --2- 172.21.15.71:0/2695034917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86940ffc20 0x7f869419c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:52.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.518+0000 7f86927fc700 1 -- 172.21.15.71:0/2695034917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86940ffc20 msgr2=0x7f869419c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:52.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.518+0000 7f86927fc700 1 --2- 172.21.15.71:0/2695034917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86940ffc20 0x7f869419c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.518+0000 7f86927fc700 1 -- 172.21.15.71:0/2695034917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86880095f0 con 0x7f8694105bd0 2024-04-23T02:24:52.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.518+0000 7f86927fc700 1 --2- 172.21.15.71:0/2695034917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8694105bd0 0x7f869419cab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f868400d8d0 tx=0x7f868400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:52.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.518+0000 7f867bfff700 1 -- 172.21.15.71:0/2695034917 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86840098a0 con 0x7f8694105bd0 2024-04-23T02:24:52.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.518+0000 7f867bfff700 1 -- 172.21.15.71:0/2695034917 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8684010460 con 0x7f8694105bd0 2024-04-23T02:24:52.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.518+0000 7f86997fe700 1 -- 172.21.15.71:0/2695034917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86941968d0 con 0x7f8694105bd0 2024-04-23T02:24:52.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.518+0000 7f867bfff700 1 -- 172.21.15.71:0/2695034917 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8684009ea0 con 0x7f8694105bd0 2024-04-23T02:24:52.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.518+0000 7f86997fe700 1 -- 172.21.15.71:0/2695034917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8694196e20 con 0x7f8694105bd0 2024-04-23T02:24:52.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.519+0000 7f86997fe700 1 -- 172.21.15.71:0/2695034917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f869404f2e0 con 0x7f8694105bd0 2024-04-23T02:24:52.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.520+0000 7f867bfff700 1 -- 172.21.15.71:0/2695034917 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f86840105d0 con 0x7f8694105bd0 2024-04-23T02:24:52.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.520+0000 7f867bfff700 1 --2- 172.21.15.71:0/2695034917 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f867c06cce0 0x7f867c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:52.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.520+0000 7f867bfff700 1 -- 172.21.15.71:0/2695034917 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f868408c480 con 0x7f8694105bd0 2024-04-23T02:24:52.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.520+0000 7f8692ffd700 1 --2- 172.21.15.71:0/2695034917 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f867c06cce0 0x7f867c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:52.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.521+0000 7f8692ffd700 1 --2- 172.21.15.71:0/2695034917 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f867c06cce0 0x7f867c06f1a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f868800b5c0 tx=0x7f868801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:52.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.525+0000 7f867bfff700 1 -- 172.21.15.71:0/2695034917 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8684059f80 con 0x7f8694105bd0 2024-04-23T02:24:52.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.801+0000 7f86997fe700 1 -- 172.21.15.71:0/2695034917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f86941970a0 con 0x7f8694105bd0 2024-04-23T02:24:52.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.801+0000 7f867bfff700 1 -- 172.21.15.71:0/2695034917 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f8684010880 con 0x7f8694105bd0 2024-04-23T02:24:52.806 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:24:52.806 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:24:52.806 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:24:52.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:24:52.806 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:24:52.806 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:24:52.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:24:52.806 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:24:52.806 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:24:52.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:24:52.806 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:24:52.807 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:24:52.807 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:24:52.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:24:52.807 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:24:52.807 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:24:52.807 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:24:52.807 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:24:52.807 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:24:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.803+0000 7f86997fe700 1 -- 172.21.15.71:0/2695034917 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f867c06cce0 msgr2=0x7f867c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.803+0000 7f86997fe700 1 --2- 172.21.15.71:0/2695034917 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f867c06cce0 0x7f867c06f1a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f868800b5c0 tx=0x7f868801a040 comp rx=0 tx=0).stop 2024-04-23T02:24:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.803+0000 7f86997fe700 1 -- 172.21.15.71:0/2695034917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8694105bd0 msgr2=0x7f869419cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.803+0000 7f86997fe700 1 --2- 172.21.15.71:0/2695034917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8694105bd0 0x7f869419cab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f868400d8d0 tx=0x7f868400dbe0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.803+0000 7f86997fe700 1 -- 172.21.15.71:0/2695034917 shutdown_connections 2024-04-23T02:24:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.804+0000 7f86997fe700 1 --2- 172.21.15.71:0/2695034917 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f867c06cce0 0x7f867c06f1a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.804+0000 7f86997fe700 1 --2- 172.21.15.71:0/2695034917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86940ffc20 0x7f869419c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.804+0000 7f86997fe700 1 --2- 172.21.15.71:0/2695034917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8694105bd0 0x7f869419cab0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.804+0000 7f86997fe700 1 -- 172.21.15.71:0/2695034917 >> 172.21.15.71:0/2695034917 conn(0x7f8694075670 msgr2=0x7f8694103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.804+0000 7f86997fe700 1 -- 172.21.15.71:0/2695034917 shutdown_connections 2024-04-23T02:24:52.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.804+0000 7f86997fe700 1 -- 172.21.15.71:0/2695034917 wait complete. 2024-04-23T02:24:52.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.910+0000 7f64face1700 1 -- 172.21.15.71:0/2260420526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f4105350 msgr2=0x7f64f4105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:52.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.910+0000 7f64face1700 1 --2- 172.21.15.71:0/2260420526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f4105350 0x7f64f4105730 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009910 tx=0x7f64e8009c20 comp rx=0 tx=0).stop 2024-04-23T02:24:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.910+0000 7f64face1700 1 -- 172.21.15.71:0/2260420526 shutdown_connections 2024-04-23T02:24:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.910+0000 7f64face1700 1 --2- 172.21.15.71:0/2260420526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64f40ff340 0x7f64f40ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.910+0000 7f64face1700 1 --2- 172.21.15.71:0/2260420526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f4105350 0x7f64f4105730 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.910+0000 7f64face1700 1 -- 172.21.15.71:0/2260420526 >> 172.21.15.71:0/2260420526 conn(0x7f64f4074e30 msgr2=0x7f64f4075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.910+0000 7f64face1700 1 -- 172.21.15.71:0/2260420526 shutdown_connections 2024-04-23T02:24:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.910+0000 7f64face1700 1 -- 172.21.15.71:0/2260420526 wait complete. 2024-04-23T02:24:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.910+0000 7f64face1700 1 Processor -- start 2024-04-23T02:24:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.911+0000 7f64face1700 1 -- start start 2024-04-23T02:24:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.911+0000 7f64face1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f40ff340 0x7f64f419c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.911+0000 7f64face1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64f4105350 0x7f64f419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.911+0000 7f64face1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f419d180 con 0x7f64f40ff340 2024-04-23T02:24:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.911+0000 7f64face1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f4196630 con 0x7f64f4105350 2024-04-23T02:24:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.911+0000 7f64f8a7d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f40ff340 0x7f64f419c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.911+0000 7f64f8a7d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f40ff340 0x7f64f419c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57172/0 (socket says 172.21.15.71:57172) 2024-04-23T02:24:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.911+0000 7f64f8a7d700 1 -- 172.21.15.71:0/3388043114 learned_addr learned my addr 172.21.15.71:0/3388043114 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:24:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.911+0000 7f64f3fff700 1 --2- 172.21.15.71:0/3388043114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64f4105350 0x7f64f419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.911+0000 7f64f8a7d700 1 -- 172.21.15.71:0/3388043114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64f4105350 msgr2=0x7f64f419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.911+0000 7f64f8a7d700 1 --2- 172.21.15.71:0/3388043114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64f4105350 0x7f64f419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.911+0000 7f64f8a7d700 1 -- 172.21.15.71:0/3388043114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e80095f0 con 0x7f64f40ff340 2024-04-23T02:24:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.911+0000 7f64f8a7d700 1 --2- 172.21.15.71:0/3388043114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f40ff340 0x7f64f419c5b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f64e8006010 tx=0x7f64e800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.911+0000 7f64f1ffb700 1 -- 172.21.15.71:0/3388043114 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64e801d070 con 0x7f64f40ff340 2024-04-23T02:24:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.911+0000 7f64face1700 1 -- 172.21.15.71:0/3388043114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f41968b0 con 0x7f64f40ff340 2024-04-23T02:24:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.911+0000 7f64f1ffb700 1 -- 172.21.15.71:0/3388043114 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f64e800f460 con 0x7f64f40ff340 2024-04-23T02:24:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.911+0000 7f64f1ffb700 1 -- 172.21.15.71:0/3388043114 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64e8021620 con 0x7f64f40ff340 2024-04-23T02:24:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.912+0000 7f64face1700 1 -- 172.21.15.71:0/3388043114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f4196da0 con 0x7f64f40ff340 2024-04-23T02:24:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.913+0000 7f64face1700 1 -- 172.21.15.71:0/3388043114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f4109880 con 0x7f64f40ff340 2024-04-23T02:24:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.913+0000 7f64f1ffb700 1 -- 172.21.15.71:0/3388043114 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f64e800f5d0 con 0x7f64f40ff340 2024-04-23T02:24:52.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.913+0000 7f64f1ffb700 1 --2- 172.21.15.71:0/3388043114 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f64e406cce0 0x7f64e406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:52.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.914+0000 7f64f1ffb700 1 -- 172.21.15.71:0/3388043114 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f64e808d830 con 0x7f64f40ff340 2024-04-23T02:24:52.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.914+0000 7f64f3fff700 1 --2- 172.21.15.71:0/3388043114 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f64e406cce0 0x7f64e406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:52.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.914+0000 7f64f3fff700 1 --2- 172.21.15.71:0/3388043114 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f64e406cce0 0x7f64e406f1a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f64f40754f0 tx=0x7f64e000a300 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:52.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:52.918+0000 7f64f1ffb700 1 -- 172.21.15.71:0/3388043114 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f64e805b330 con 0x7f64f40ff340 2024-04-23T02:24:53.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.156+0000 7f64face1700 1 -- 172.21.15.71:0/3388043114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f64f404f2e0 con 0x7f64f40ff340 2024-04-23T02:24:53.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.157+0000 7f64f1ffb700 1 -- 172.21.15.71:0/3388043114 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f64e8057020 con 0x7f64f40ff340 2024-04-23T02:24:53.161 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:24:53.161 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:24:53.161 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:24:53.161 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:24:53.161 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:24:53.161 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:24:53.161 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:24:53.161 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:24:53.162 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:24:53.162 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:24:53.162 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:24:53.162 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:24:53.162 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:24:53.162 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:24:53.162 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:24:53.162 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:24:53.162 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:24:53.162 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:24:53.162 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:24:53.162 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:24:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:24:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:24:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:24:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:24:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:24:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:24:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:24:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:24:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:24:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:24:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:24:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:24:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:24:53.163 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:24:53.164 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:24:53.164 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:24:53.164 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:24:53.164 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:24:53.164 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:24:53.164 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:24:53.164 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:24:53.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.159+0000 7f64face1700 1 -- 172.21.15.71:0/3388043114 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f64e406cce0 msgr2=0x7f64e406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:53.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.159+0000 7f64face1700 1 --2- 172.21.15.71:0/3388043114 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f64e406cce0 0x7f64e406f1a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f64f40754f0 tx=0x7f64e000a300 comp rx=0 tx=0).stop 2024-04-23T02:24:53.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.159+0000 7f64face1700 1 -- 172.21.15.71:0/3388043114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f40ff340 msgr2=0x7f64f419c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:53.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.159+0000 7f64face1700 1 --2- 172.21.15.71:0/3388043114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f40ff340 0x7f64f419c5b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f64e8006010 tx=0x7f64e800bd70 comp rx=0 tx=0).stop 2024-04-23T02:24:53.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.159+0000 7f64face1700 1 -- 172.21.15.71:0/3388043114 shutdown_connections 2024-04-23T02:24:53.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.159+0000 7f64face1700 1 --2- 172.21.15.71:0/3388043114 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f64e406cce0 0x7f64e406f1a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:53.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.159+0000 7f64face1700 1 --2- 172.21.15.71:0/3388043114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64f40ff340 0x7f64f419c5b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:53.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.159+0000 7f64face1700 1 --2- 172.21.15.71:0/3388043114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64f4105350 0x7f64f419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:53.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.159+0000 7f64face1700 1 -- 172.21.15.71:0/3388043114 >> 172.21.15.71:0/3388043114 conn(0x7f64f4074e30 msgr2=0x7f64f4103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:53.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.159+0000 7f64face1700 1 -- 172.21.15.71:0/3388043114 shutdown_connections 2024-04-23T02:24:53.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.159+0000 7f64face1700 1 -- 172.21.15.71:0/3388043114 wait complete. 2024-04-23T02:24:53.166 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:24:53.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:52 smithi071 ceph-mon[144544]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:24:53.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:52 smithi071 ceph-mon[144544]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:24:53.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:52 smithi071 ceph-mon[144544]: from='client.34490 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:24:53.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:52 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2695034917' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:24:53.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.263+0000 7fca63e9c700 1 -- 172.21.15.71:0/3434162733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c105bf0 msgr2=0x7fca5c105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:53.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.263+0000 7fca63e9c700 1 --2- 172.21.15.71:0/3434162733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c105bf0 0x7fca5c105fd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009910 tx=0x7fca4c009c20 comp rx=0 tx=0).stop 2024-04-23T02:24:53.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.263+0000 7fca63e9c700 1 -- 172.21.15.71:0/3434162733 shutdown_connections 2024-04-23T02:24:53.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.263+0000 7fca63e9c700 1 --2- 172.21.15.71:0/3434162733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca5c0ffb40 0x7fca5c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:53.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.263+0000 7fca63e9c700 1 --2- 172.21.15.71:0/3434162733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c105bf0 0x7fca5c105fd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:53.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.263+0000 7fca63e9c700 1 -- 172.21.15.71:0/3434162733 >> 172.21.15.71:0/3434162733 conn(0x7fca5c0fb610 msgr2=0x7fca5c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:53.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.263+0000 7fca63e9c700 1 -- 172.21.15.71:0/3434162733 shutdown_connections 2024-04-23T02:24:53.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.263+0000 7fca63e9c700 1 -- 172.21.15.71:0/3434162733 wait complete. 2024-04-23T02:24:53.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.264+0000 7fca63e9c700 1 Processor -- start 2024-04-23T02:24:53.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.264+0000 7fca63e9c700 1 -- start start 2024-04-23T02:24:53.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.264+0000 7fca63e9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca5c0ffb40 0x7fca5c1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:53.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.264+0000 7fca63e9c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c105bf0 0x7fca5c101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:53.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.264+0000 7fca63e9c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c102230 con 0x7fca5c105bf0 2024-04-23T02:24:53.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.264+0000 7fca63e9c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c102370 con 0x7fca5c0ffb40 2024-04-23T02:24:53.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.264+0000 7fca61437700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c105bf0 0x7fca5c101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:53.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.264+0000 7fca61437700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c105bf0 0x7fca5c101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57202/0 (socket says 172.21.15.71:57202) 2024-04-23T02:24:53.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.264+0000 7fca61437700 1 -- 172.21.15.71:0/2253577104 learned_addr learned my addr 172.21.15.71:0/2253577104 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:24:53.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.264+0000 7fca61c38700 1 --2- 172.21.15.71:0/2253577104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca5c0ffb40 0x7fca5c1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:53.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.264+0000 7fca61437700 1 -- 172.21.15.71:0/2253577104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca5c0ffb40 msgr2=0x7fca5c1017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:53.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.264+0000 7fca61437700 1 --2- 172.21.15.71:0/2253577104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca5c0ffb40 0x7fca5c1017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:53.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.264+0000 7fca61437700 1 -- 172.21.15.71:0/2253577104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c0095f0 con 0x7fca5c105bf0 2024-04-23T02:24:53.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.264+0000 7fca61437700 1 --2- 172.21.15.71:0/2253577104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c105bf0 0x7fca5c101cf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fca5800c8a0 tx=0x7fca5800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:53.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.264+0000 7fca52ffd700 1 -- 172.21.15.71:0/2253577104 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca580078e0 con 0x7fca5c105bf0 2024-04-23T02:24:53.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.265+0000 7fca52ffd700 1 -- 172.21.15.71:0/2253577104 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fca5800f450 con 0x7fca5c105bf0 2024-04-23T02:24:53.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.265+0000 7fca63e9c700 1 -- 172.21.15.71:0/2253577104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5c1a28d0 con 0x7fca5c105bf0 2024-04-23T02:24:53.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.265+0000 7fca52ffd700 1 -- 172.21.15.71:0/2253577104 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca5800e5e0 con 0x7fca5c105bf0 2024-04-23T02:24:53.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.265+0000 7fca63e9c700 1 -- 172.21.15.71:0/2253577104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5c1a2ce0 con 0x7fca5c105bf0 2024-04-23T02:24:53.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.265+0000 7fca63e9c700 1 -- 172.21.15.71:0/2253577104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5c04f2e0 con 0x7fca5c105bf0 2024-04-23T02:24:53.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.266+0000 7fca52ffd700 1 -- 172.21.15.71:0/2253577104 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fca5800f5c0 con 0x7fca5c105bf0 2024-04-23T02:24:53.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.266+0000 7fca52ffd700 1 --2- 172.21.15.71:0/2253577104 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fca4806cce0 0x7fca4806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:53.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.266+0000 7fca52ffd700 1 -- 172.21.15.71:0/2253577104 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fca5808c1d0 con 0x7fca5c105bf0 2024-04-23T02:24:53.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.266+0000 7fca61c38700 1 --2- 172.21.15.71:0/2253577104 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fca4806cce0 0x7fca4806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:53.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.267+0000 7fca61c38700 1 --2- 172.21.15.71:0/2253577104 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fca4806cce0 0x7fca4806f1a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fca4c006010 tx=0x7fca4c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:53.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.271+0000 7fca52ffd700 1 -- 172.21.15.71:0/2253577104 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fca58056240 con 0x7fca5c105bf0 2024-04-23T02:24:53.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:52 smithi192 ceph-mon[108617]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:24:53.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:52 smithi192 ceph-mon[108617]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:24:53.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:52 smithi192 ceph-mon[108617]: from='client.34490 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:24:53.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:52 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2695034917' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:24:53.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.480+0000 7fca63e9c700 1 -- 172.21.15.71:0/2253577104 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca5c069620 con 0x7fca4806cce0 2024-04-23T02:24:53.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.481+0000 7fca52ffd700 1 -- 172.21.15.71:0/2253577104 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fca5c069620 con 0x7fca4806cce0 2024-04-23T02:24:53.485 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:24:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:24:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:24:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:24:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:24:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:24:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:24:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:24:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:24:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:24:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:24:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:24:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:24:53.486 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:24:53.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.484+0000 7fca63e9c700 1 -- 172.21.15.71:0/2253577104 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fca4806cce0 msgr2=0x7fca4806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:53.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.484+0000 7fca63e9c700 1 --2- 172.21.15.71:0/2253577104 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fca4806cce0 0x7fca4806f1a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fca4c006010 tx=0x7fca4c01a040 comp rx=0 tx=0).stop 2024-04-23T02:24:53.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.484+0000 7fca63e9c700 1 -- 172.21.15.71:0/2253577104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c105bf0 msgr2=0x7fca5c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:53.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.484+0000 7fca63e9c700 1 --2- 172.21.15.71:0/2253577104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c105bf0 0x7fca5c101cf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fca5800c8a0 tx=0x7fca5800cbb0 comp rx=0 tx=0).stop 2024-04-23T02:24:53.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.484+0000 7fca63e9c700 1 -- 172.21.15.71:0/2253577104 shutdown_connections 2024-04-23T02:24:53.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.484+0000 7fca63e9c700 1 --2- 172.21.15.71:0/2253577104 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fca4806cce0 0x7fca4806f1a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:53.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.484+0000 7fca63e9c700 1 --2- 172.21.15.71:0/2253577104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca5c0ffb40 0x7fca5c1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:53.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.484+0000 7fca63e9c700 1 --2- 172.21.15.71:0/2253577104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c105bf0 0x7fca5c101cf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:53.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.484+0000 7fca63e9c700 1 -- 172.21.15.71:0/2253577104 >> 172.21.15.71:0/2253577104 conn(0x7fca5c0fb610 msgr2=0x7fca5c068f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:53.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.484+0000 7fca63e9c700 1 -- 172.21.15.71:0/2253577104 shutdown_connections 2024-04-23T02:24:53.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.484+0000 7fca63e9c700 1 -- 172.21.15.71:0/2253577104 wait complete. 2024-04-23T02:24:53.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.586+0000 7f7169198700 1 -- 172.21.15.71:0/2070623748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7164101e60 msgr2=0x7f7164102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:53.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.586+0000 7f7169198700 1 --2- 172.21.15.71:0/2070623748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7164101e60 0x7f7164102240 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7158009940 tx=0x7f7158009c50 comp rx=0 tx=0).stop 2024-04-23T02:24:53.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.586+0000 7f7169198700 1 -- 172.21.15.71:0/2070623748 shutdown_connections 2024-04-23T02:24:53.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.586+0000 7f7169198700 1 --2- 172.21.15.71:0/2070623748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7164102780 0x7f716410ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:53.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.586+0000 7f7169198700 1 --2- 172.21.15.71:0/2070623748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7164101e60 0x7f7164102240 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:53.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.586+0000 7f7169198700 1 -- 172.21.15.71:0/2070623748 >> 172.21.15.71:0/2070623748 conn(0x7f71640fb650 msgr2=0x7f71640fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:53.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.586+0000 7f7169198700 1 -- 172.21.15.71:0/2070623748 shutdown_connections 2024-04-23T02:24:53.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.586+0000 7f7169198700 1 -- 172.21.15.71:0/2070623748 wait complete. 2024-04-23T02:24:53.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.587+0000 7f7169198700 1 Processor -- start 2024-04-23T02:24:53.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.587+0000 7f7169198700 1 -- start start 2024-04-23T02:24:53.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.588+0000 7f7169198700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7164101e60 0x7f716419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:53.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.588+0000 7f7169198700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7164102780 0x7f716419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.588+0000 7f7169198700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716419d220 con 0x7f7164102780 2024-04-23T02:24:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.588+0000 7f7169198700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71641966d0 con 0x7f7164101e60 2024-04-23T02:24:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.588+0000 7f716259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7164102780 0x7f716419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.588+0000 7f716259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7164102780 0x7f716419cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57216/0 (socket says 172.21.15.71:57216) 2024-04-23T02:24:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.588+0000 7f716259c700 1 -- 172.21.15.71:0/2068005356 learned_addr learned my addr 172.21.15.71:0/2068005356 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:24:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.588+0000 7f7162d9d700 1 --2- 172.21.15.71:0/2068005356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7164101e60 0x7f716419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.588+0000 7f716259c700 1 -- 172.21.15.71:0/2068005356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7164101e60 msgr2=0x7f716419c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.588+0000 7f716259c700 1 --2- 172.21.15.71:0/2068005356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7164101e60 0x7f716419c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.588+0000 7f716259c700 1 -- 172.21.15.71:0/2068005356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71580095f0 con 0x7f7164102780 2024-04-23T02:24:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.588+0000 7f716259c700 1 --2- 172.21.15.71:0/2068005356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7164102780 0x7f716419cb90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f715400c8f0 tx=0x7f715400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.588+0000 7f714bfff700 1 -- 172.21.15.71:0/2068005356 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71540078a0 con 0x7f7164102780 2024-04-23T02:24:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.588+0000 7f714bfff700 1 -- 172.21.15.71:0/2068005356 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f715400f450 con 0x7f7164102780 2024-04-23T02:24:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.588+0000 7f7169198700 1 -- 172.21.15.71:0/2068005356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71641969b0 con 0x7f7164102780 2024-04-23T02:24:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.588+0000 7f714bfff700 1 -- 172.21.15.71:0/2068005356 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7154007ea0 con 0x7f7164102780 2024-04-23T02:24:53.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.588+0000 7f7169198700 1 -- 172.21.15.71:0/2068005356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7164196f00 con 0x7f7164102780 2024-04-23T02:24:53.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.589+0000 7f7169198700 1 -- 172.21.15.71:0/2068005356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7164108370 con 0x7f7164102780 2024-04-23T02:24:53.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.590+0000 7f714bfff700 1 -- 172.21.15.71:0/2068005356 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f715400f5c0 con 0x7f7164102780 2024-04-23T02:24:53.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.590+0000 7f714bfff700 1 --2- 172.21.15.71:0/2068005356 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f714c06cce0 0x7f714c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:24:53.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.590+0000 7f7162d9d700 1 --2- 172.21.15.71:0/2068005356 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f714c06cce0 0x7f714c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:24:53.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.590+0000 7f714bfff700 1 -- 172.21.15.71:0/2068005356 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f715408c580 con 0x7f7164102780 2024-04-23T02:24:53.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.591+0000 7f7162d9d700 1 --2- 172.21.15.71:0/2068005356 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f714c06cce0 0x7f714c06f1a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f71580059d0 tx=0x7f7158005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:24:53.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.595+0000 7f714bfff700 1 -- 172.21.15.71:0/2068005356 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7154059fe0 con 0x7f7164102780 2024-04-23T02:24:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.866+0000 7f7169198700 1 -- 172.21.15.71:0/2068005356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f716404f2e0 con 0x7f7164102780 2024-04-23T02:24:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.867+0000 7f714bfff700 1 -- 172.21.15.71:0/2068005356 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f7154059b70 con 0x7f7164102780 2024-04-23T02:24:53.871 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:24:53.871 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:24:53.871 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:24:53.871 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:24:53.871 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:24:53.871 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:24:53.871 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:24:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.869+0000 7f7169198700 1 -- 172.21.15.71:0/2068005356 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f714c06cce0 msgr2=0x7f714c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.869+0000 7f7169198700 1 --2- 172.21.15.71:0/2068005356 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f714c06cce0 0x7f714c06f1a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f71580059d0 tx=0x7f7158005960 comp rx=0 tx=0).stop 2024-04-23T02:24:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.869+0000 7f7169198700 1 -- 172.21.15.71:0/2068005356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7164102780 msgr2=0x7f716419cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:24:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.869+0000 7f7169198700 1 --2- 172.21.15.71:0/2068005356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7164102780 0x7f716419cb90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f715400c8f0 tx=0x7f715400cc00 comp rx=0 tx=0).stop 2024-04-23T02:24:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.869+0000 7f7169198700 1 -- 172.21.15.71:0/2068005356 shutdown_connections 2024-04-23T02:24:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.869+0000 7f7169198700 1 --2- 172.21.15.71:0/2068005356 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f714c06cce0 0x7f714c06f1a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.869+0000 7f7169198700 1 --2- 172.21.15.71:0/2068005356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7164101e60 0x7f716419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.869+0000 7f7169198700 1 --2- 172.21.15.71:0/2068005356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7164102780 0x7f716419cb90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:24:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.869+0000 7f7169198700 1 -- 172.21.15.71:0/2068005356 >> 172.21.15.71:0/2068005356 conn(0x7f71640fb650 msgr2=0x7f71641054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:24:53.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.869+0000 7f7169198700 1 -- 172.21.15.71:0/2068005356 shutdown_connections 2024-04-23T02:24:53.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:24:53.869+0000 7f7169198700 1 -- 172.21.15.71:0/2068005356 wait complete. 2024-04-23T02:24:54.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:53 smithi192 ceph-mon[108617]: pgmap v347: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:54.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:53 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3388043114' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:24:54.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:53 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2068005356' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:24:54.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:53 smithi071 ceph-mon[144544]: pgmap v347: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:54.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:53 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3388043114' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:24:54.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:53 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2068005356' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:24:55.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:54 smithi071 ceph-mon[144544]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:24:55.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:54 smithi192 ceph-mon[108617]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:24:56.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:55 smithi071 ceph-mon[144544]: pgmap v348: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:56.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:55 smithi192 ceph-mon[108617]: pgmap v348: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:58.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:57 smithi071 ceph-mon[144544]: pgmap v349: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:24:58.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:57 smithi192 ceph-mon[108617]: pgmap v349: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:00.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:24:59 smithi071 ceph-mon[144544]: pgmap v350: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:00.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:24:59 smithi192 ceph-mon[108617]: pgmap v350: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:02.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:01 smithi071 ceph-mon[144544]: pgmap v351: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:02.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:01 smithi192 ceph-mon[108617]: pgmap v351: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:04.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:03 smithi071 ceph-mon[144544]: pgmap v352: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:04.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:03 smithi192 ceph-mon[108617]: pgmap v352: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:05.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:04 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:25:05.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:04 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:25:06.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:05 smithi071 ceph-mon[144544]: pgmap v353: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:06.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:05 smithi192 ceph-mon[108617]: pgmap v353: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:08.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:07 smithi071 ceph-mon[144544]: pgmap v354: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:08.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:07 smithi192 ceph-mon[108617]: pgmap v354: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:10.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:09 smithi071 ceph-mon[144544]: pgmap v355: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:10.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:09 smithi192 ceph-mon[108617]: pgmap v355: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:12.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:11 smithi071 ceph-mon[144544]: pgmap v356: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:12.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:11 smithi192 ceph-mon[108617]: pgmap v356: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:14.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:13 smithi071 ceph-mon[144544]: pgmap v357: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:14.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:13 smithi192 ceph-mon[108617]: pgmap v357: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:16.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:15 smithi071 ceph-mon[144544]: pgmap v358: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:16.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:15 smithi192 ceph-mon[108617]: pgmap v358: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:18.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:18 smithi192 ceph-mon[108617]: pgmap v359: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:18.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:18 smithi071 ceph-mon[144544]: pgmap v359: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:20.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:20 smithi192 ceph-mon[108617]: pgmap v360: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:20.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:25:20.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:20 smithi071 ceph-mon[144544]: pgmap v360: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:20.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:25:22.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:22 smithi192 ceph-mon[108617]: pgmap v361: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:22.632 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:22 smithi071 ceph-mon[144544]: pgmap v361: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:23.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.988+0000 7fe2f34f9700 1 -- 172.21.15.71:0/2835861750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2ec100e90 msgr2=0x7fe2ec101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:23.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.988+0000 7fe2f34f9700 1 --2- 172.21.15.71:0/2835861750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2ec100e90 0x7fe2ec101270 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009910 tx=0x7fe2e0009c20 comp rx=0 tx=0).stop 2024-04-23T02:25:23.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.988+0000 7fe2f34f9700 1 -- 172.21.15.71:0/2835861750 shutdown_connections 2024-04-23T02:25:23.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.988+0000 7fe2f34f9700 1 --2- 172.21.15.71:0/2835861750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2ec101840 0x7fe2ec105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:23.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.988+0000 7fe2f34f9700 1 --2- 172.21.15.71:0/2835861750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2ec100e90 0x7fe2ec101270 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:23.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.988+0000 7fe2f34f9700 1 -- 172.21.15.71:0/2835861750 >> 172.21.15.71:0/2835861750 conn(0x7fe2ec0fc740 msgr2=0x7fe2ec0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:23.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.988+0000 7fe2f34f9700 1 -- 172.21.15.71:0/2835861750 shutdown_connections 2024-04-23T02:25:23.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.988+0000 7fe2f34f9700 1 -- 172.21.15.71:0/2835861750 wait complete. 2024-04-23T02:25:23.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.989+0000 7fe2f34f9700 1 Processor -- start 2024-04-23T02:25:23.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.989+0000 7fe2f34f9700 1 -- start start 2024-04-23T02:25:23.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.989+0000 7fe2f34f9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2ec100e90 0x7fe2ec073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:23.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.989+0000 7fe2f34f9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2ec101840 0x7fe2ec0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:23.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.989+0000 7fe2f34f9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec073b10 con 0x7fe2ec100e90 2024-04-23T02:25:23.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.989+0000 7fe2f34f9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec073c50 con 0x7fe2ec101840 2024-04-23T02:25:23.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.989+0000 7fe2f1295700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2ec100e90 0x7fe2ec073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:23.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.989+0000 7fe2f1295700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2ec100e90 0x7fe2ec073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52546/0 (socket says 172.21.15.71:52546) 2024-04-23T02:25:23.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.989+0000 7fe2f1295700 1 -- 172.21.15.71:0/2715446248 learned_addr learned my addr 172.21.15.71:0/2715446248 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:25:23.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.990+0000 7fe2f1295700 1 -- 172.21.15.71:0/2715446248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2ec101840 msgr2=0x7fe2ec0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:23.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.990+0000 7fe2f1295700 1 --2- 172.21.15.71:0/2715446248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2ec101840 0x7fe2ec0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:23.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.990+0000 7fe2f1295700 1 -- 172.21.15.71:0/2715446248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e00095f0 con 0x7fe2ec100e90 2024-04-23T02:25:23.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.990+0000 7fe2f1295700 1 --2- 172.21.15.71:0/2715446248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2ec100e90 0x7fe2ec073090 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0006010 tx=0x7fe2e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:23.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.990+0000 7fe2de7fc700 1 -- 172.21.15.71:0/2715446248 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2e001d070 con 0x7fe2ec100e90 2024-04-23T02:25:23.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.990+0000 7fe2de7fc700 1 -- 172.21.15.71:0/2715446248 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe2e000f460 con 0x7fe2ec100e90 2024-04-23T02:25:23.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.990+0000 7fe2f34f9700 1 -- 172.21.15.71:0/2715446248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ec073ed0 con 0x7fe2ec100e90 2024-04-23T02:25:23.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.990+0000 7fe2de7fc700 1 -- 172.21.15.71:0/2715446248 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2e0021640 con 0x7fe2ec100e90 2024-04-23T02:25:23.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.990+0000 7fe2f34f9700 1 -- 172.21.15.71:0/2715446248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ec1a2cf0 con 0x7fe2ec100e90 2024-04-23T02:25:23.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.991+0000 7fe2f34f9700 1 -- 172.21.15.71:0/2715446248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2ec109230 con 0x7fe2ec100e90 2024-04-23T02:25:23.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.992+0000 7fe2de7fc700 1 -- 172.21.15.71:0/2715446248 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe2e000fa60 con 0x7fe2ec100e90 2024-04-23T02:25:23.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.992+0000 7fe2de7fc700 1 --2- 172.21.15.71:0/2715446248 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe2d806cce0 0x7fe2d806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:23.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.992+0000 7fe2de7fc700 1 -- 172.21.15.71:0/2715446248 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe2e008d7b0 con 0x7fe2ec100e90 2024-04-23T02:25:23.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.992+0000 7fe2f0a94700 1 --2- 172.21.15.71:0/2715446248 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe2d806cce0 0x7fe2d806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:23.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.993+0000 7fe2f0a94700 1 --2- 172.21.15.71:0/2715446248 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe2d806cce0 0x7fe2d806f1a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec074880 tx=0x7fe2e8005c80 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:23.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:23.997+0000 7fe2de7fc700 1 -- 172.21.15.71:0/2715446248 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe2e00578a0 con 0x7fe2ec100e90 2024-04-23T02:25:24.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.215+0000 7fe2f34f9700 1 -- 172.21.15.71:0/2715446248 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2ec0611d0 con 0x7fe2d806cce0 2024-04-23T02:25:24.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.216+0000 7fe2de7fc700 1 -- 172.21.15.71:0/2715446248 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe2ec0611d0 con 0x7fe2d806cce0 2024-04-23T02:25:24.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.218+0000 7fe2f34f9700 1 -- 172.21.15.71:0/2715446248 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe2d806cce0 msgr2=0x7fe2d806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:24.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.218+0000 7fe2f34f9700 1 --2- 172.21.15.71:0/2715446248 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe2d806cce0 0x7fe2d806f1a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec074880 tx=0x7fe2e8005c80 comp rx=0 tx=0).stop 2024-04-23T02:25:24.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.218+0000 7fe2f34f9700 1 -- 172.21.15.71:0/2715446248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2ec100e90 msgr2=0x7fe2ec073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:24.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.218+0000 7fe2f34f9700 1 --2- 172.21.15.71:0/2715446248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2ec100e90 0x7fe2ec073090 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0006010 tx=0x7fe2e000be20 comp rx=0 tx=0).stop 2024-04-23T02:25:24.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.219+0000 7fe2f34f9700 1 -- 172.21.15.71:0/2715446248 shutdown_connections 2024-04-23T02:25:24.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.219+0000 7fe2f34f9700 1 --2- 172.21.15.71:0/2715446248 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe2d806cce0 0x7fe2d806f1a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.219+0000 7fe2f34f9700 1 --2- 172.21.15.71:0/2715446248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2ec100e90 0x7fe2ec073090 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.219+0000 7fe2f34f9700 1 --2- 172.21.15.71:0/2715446248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2ec101840 0x7fe2ec0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.219+0000 7fe2f34f9700 1 -- 172.21.15.71:0/2715446248 >> 172.21.15.71:0/2715446248 conn(0x7fe2ec0fc740 msgr2=0x7fe2ec0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:24.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.219+0000 7fe2f34f9700 1 -- 172.21.15.71:0/2715446248 shutdown_connections 2024-04-23T02:25:24.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.219+0000 7fe2f34f9700 1 -- 172.21.15.71:0/2715446248 wait complete. 2024-04-23T02:25:24.232 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:25:24.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.321+0000 7f2329234700 1 -- 172.21.15.71:0/1223864464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2324068580 msgr2=0x7f23240689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:24.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.321+0000 7f2329234700 1 --2- 172.21.15.71:0/1223864464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2324068580 0x7f23240689e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2314009910 tx=0x7f2314009c20 comp rx=0 tx=0).stop 2024-04-23T02:25:24.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.322+0000 7f2329234700 1 -- 172.21.15.71:0/1223864464 shutdown_connections 2024-04-23T02:25:24.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.322+0000 7f2329234700 1 --2- 172.21.15.71:0/1223864464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2324068580 0x7f23240689e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.322+0000 7f2329234700 1 --2- 172.21.15.71:0/1223864464 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2324106cd0 0x7f23241070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.322+0000 7f2329234700 1 -- 172.21.15.71:0/1223864464 >> 172.21.15.71:0/1223864464 conn(0x7f2324075670 msgr2=0x7f2324075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:24.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.322+0000 7f2329234700 1 -- 172.21.15.71:0/1223864464 shutdown_connections 2024-04-23T02:25:24.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.322+0000 7f2329234700 1 -- 172.21.15.71:0/1223864464 wait complete. 2024-04-23T02:25:24.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.322+0000 7f2329234700 1 Processor -- start 2024-04-23T02:25:24.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.323+0000 7f2329234700 1 -- start start 2024-04-23T02:25:24.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.323+0000 7f2329234700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2324068580 0x7f232419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:24.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.323+0000 7f2329234700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2324106cd0 0x7f232419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:24.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.323+0000 7f2329234700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232419d080 con 0x7f2324068580 2024-04-23T02:25:24.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.323+0000 7f2329234700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23241965c0 con 0x7f2324106cd0 2024-04-23T02:25:24.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.323+0000 7f2322d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2324068580 0x7f232419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:24.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.323+0000 7f2322d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2324068580 0x7f232419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52556/0 (socket says 172.21.15.71:52556) 2024-04-23T02:25:24.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.323+0000 7f2322d9d700 1 -- 172.21.15.71:0/1907322659 learned_addr learned my addr 172.21.15.71:0/1907322659 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:25:24.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.323+0000 7f232259c700 1 --2- 172.21.15.71:0/1907322659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2324106cd0 0x7f232419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:24.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.323+0000 7f2322d9d700 1 -- 172.21.15.71:0/1907322659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2324106cd0 msgr2=0x7f232419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:24.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.323+0000 7f2322d9d700 1 --2- 172.21.15.71:0/1907322659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2324106cd0 0x7f232419ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.323+0000 7f2322d9d700 1 -- 172.21.15.71:0/1907322659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23140095f0 con 0x7f2324068580 2024-04-23T02:25:24.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.324+0000 7f2322d9d700 1 --2- 172.21.15.71:0/1907322659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2324068580 0x7f232419c540 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f231800c930 tx=0x7f231800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:24.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.324+0000 7f230bfff700 1 -- 172.21.15.71:0/1907322659 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2318007ad0 con 0x7f2324068580 2024-04-23T02:25:24.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.324+0000 7f2329234700 1 -- 172.21.15.71:0/1907322659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23241968a0 con 0x7f2324068580 2024-04-23T02:25:24.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.324+0000 7f230bfff700 1 -- 172.21.15.71:0/1907322659 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2318007c30 con 0x7f2324068580 2024-04-23T02:25:24.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.324+0000 7f230bfff700 1 -- 172.21.15.71:0/1907322659 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2318018630 con 0x7f2324068580 2024-04-23T02:25:24.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.324+0000 7f2329234700 1 -- 172.21.15.71:0/1907322659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2324196df0 con 0x7f2324068580 2024-04-23T02:25:24.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.325+0000 7f2329234700 1 -- 172.21.15.71:0/1907322659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23241091d0 con 0x7f2324068580 2024-04-23T02:25:24.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.326+0000 7f230bfff700 1 -- 172.21.15.71:0/1907322659 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f231800f450 con 0x7f2324068580 2024-04-23T02:25:24.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.326+0000 7f230bfff700 1 --2- 172.21.15.71:0/1907322659 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f230c06cce0 0x7f230c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:24.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.326+0000 7f230bfff700 1 -- 172.21.15.71:0/1907322659 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f231808ca20 con 0x7f2324068580 2024-04-23T02:25:24.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.326+0000 7f232259c700 1 --2- 172.21.15.71:0/1907322659 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f230c06cce0 0x7f230c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:24.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.327+0000 7f232259c700 1 --2- 172.21.15.71:0/1907322659 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f230c06cce0 0x7f230c06f1a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f231400b5c0 tx=0x7f2314005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:24.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.330+0000 7f230bfff700 1 -- 172.21.15.71:0/1907322659 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f231805a580 con 0x7f2324068580 2024-04-23T02:25:24.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:24 smithi071 ceph-mon[144544]: pgmap v362: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:24.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.545+0000 7f2329234700 1 -- 172.21.15.71:0/1907322659 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f23241970d0 con 0x7f230c06cce0 2024-04-23T02:25:24.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.546+0000 7f230bfff700 1 -- 172.21.15.71:0/1907322659 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f23241970d0 con 0x7f230c06cce0 2024-04-23T02:25:24.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.548+0000 7f2329234700 1 -- 172.21.15.71:0/1907322659 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f230c06cce0 msgr2=0x7f230c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:24.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.548+0000 7f2329234700 1 --2- 172.21.15.71:0/1907322659 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f230c06cce0 0x7f230c06f1a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f231400b5c0 tx=0x7f2314005fb0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.548+0000 7f2329234700 1 -- 172.21.15.71:0/1907322659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2324068580 msgr2=0x7f232419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:24.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.548+0000 7f2329234700 1 --2- 172.21.15.71:0/1907322659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2324068580 0x7f232419c540 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f231800c930 tx=0x7f231800ccf0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.548+0000 7f2329234700 1 -- 172.21.15.71:0/1907322659 shutdown_connections 2024-04-23T02:25:24.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.548+0000 7f2329234700 1 --2- 172.21.15.71:0/1907322659 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f230c06cce0 0x7f230c06f1a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.548+0000 7f2329234700 1 --2- 172.21.15.71:0/1907322659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2324068580 0x7f232419c540 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.548+0000 7f2329234700 1 --2- 172.21.15.71:0/1907322659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2324106cd0 0x7f232419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.548+0000 7f2329234700 1 -- 172.21.15.71:0/1907322659 >> 172.21.15.71:0/1907322659 conn(0x7f2324075670 msgr2=0x7f23240fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:24.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.548+0000 7f2329234700 1 -- 172.21.15.71:0/1907322659 shutdown_connections 2024-04-23T02:25:24.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.548+0000 7f2329234700 1 -- 172.21.15.71:0/1907322659 wait complete. 2024-04-23T02:25:24.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:24 smithi192 ceph-mon[108617]: pgmap v362: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:24.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.652+0000 7f247cf1c700 1 -- 172.21.15.71:0/2925599210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2478100750 msgr2=0x7f24781047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:24.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.652+0000 7f247cf1c700 1 --2- 172.21.15.71:0/2925599210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2478100750 0x7f24781047a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f246c009910 tx=0x7f246c009c20 comp rx=0 tx=0).stop 2024-04-23T02:25:24.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.652+0000 7f247cf1c700 1 -- 172.21.15.71:0/2925599210 shutdown_connections 2024-04-23T02:25:24.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.652+0000 7f247cf1c700 1 --2- 172.21.15.71:0/2925599210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2478100750 0x7f24781047a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.652+0000 7f247cf1c700 1 --2- 172.21.15.71:0/2925599210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24780ffda0 0x7f2478100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.652+0000 7f247cf1c700 1 -- 172.21.15.71:0/2925599210 >> 172.21.15.71:0/2925599210 conn(0x7f24780fb650 msgr2=0x7f24780fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:24.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.652+0000 7f247cf1c700 1 -- 172.21.15.71:0/2925599210 shutdown_connections 2024-04-23T02:25:24.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.652+0000 7f247cf1c700 1 -- 172.21.15.71:0/2925599210 wait complete. 2024-04-23T02:25:24.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.652+0000 7f247cf1c700 1 Processor -- start 2024-04-23T02:25:24.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.653+0000 7f247cf1c700 1 -- start start 2024-04-23T02:25:24.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.653+0000 7f247cf1c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24780ffda0 0x7f2478101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:24.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.653+0000 7f247cf1c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2478100750 0x7f2478101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:24.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.653+0000 7f247cf1c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24781022c0 con 0x7f2478100750 2024-04-23T02:25:24.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.653+0000 7f247cf1c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2478102400 con 0x7f24780ffda0 2024-04-23T02:25:24.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.653+0000 7f2475d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2478100750 0x7f2478101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:24.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.653+0000 7f2475d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2478100750 0x7f2478101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52576/0 (socket says 172.21.15.71:52576) 2024-04-23T02:25:24.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.653+0000 7f2475d9b700 1 -- 172.21.15.71:0/56876331 learned_addr learned my addr 172.21.15.71:0/56876331 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:25:24.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.653+0000 7f247659c700 1 --2- 172.21.15.71:0/56876331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24780ffda0 0x7f2478101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:24.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.653+0000 7f2475d9b700 1 -- 172.21.15.71:0/56876331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24780ffda0 msgr2=0x7f2478101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:24.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.653+0000 7f2475d9b700 1 --2- 172.21.15.71:0/56876331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24780ffda0 0x7f2478101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.653+0000 7f2475d9b700 1 -- 172.21.15.71:0/56876331 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246c0095f0 con 0x7f2478100750 2024-04-23T02:25:24.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.653+0000 7f2475d9b700 1 --2- 172.21.15.71:0/56876331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2478100750 0x7f2478101d80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f246c00bda0 tx=0x7f246c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:24.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.653+0000 7f24677fe700 1 -- 172.21.15.71:0/56876331 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f246c01d070 con 0x7f2478100750 2024-04-23T02:25:24.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.654+0000 7f24677fe700 1 -- 172.21.15.71:0/56876331 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f246c00f460 con 0x7f2478100750 2024-04-23T02:25:24.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.654+0000 7f247cf1c700 1 -- 172.21.15.71:0/56876331 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2478102680 con 0x7f2478100750 2024-04-23T02:25:24.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.654+0000 7f24677fe700 1 -- 172.21.15.71:0/56876331 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f246c021640 con 0x7f2478100750 2024-04-23T02:25:24.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.654+0000 7f247cf1c700 1 -- 172.21.15.71:0/56876331 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24781a2c00 con 0x7f2478100750 2024-04-23T02:25:24.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.654+0000 7f247cf1c700 1 -- 172.21.15.71:0/56876331 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24781091b0 con 0x7f2478100750 2024-04-23T02:25:24.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.655+0000 7f24677fe700 1 -- 172.21.15.71:0/56876331 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f246c00f9b0 con 0x7f2478100750 2024-04-23T02:25:24.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.655+0000 7f24677fe700 1 --2- 172.21.15.71:0/56876331 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f246006cce0 0x7f246006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:24.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.655+0000 7f24677fe700 1 -- 172.21.15.71:0/56876331 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f246c08d830 con 0x7f2478100750 2024-04-23T02:25:24.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.655+0000 7f247659c700 1 --2- 172.21.15.71:0/56876331 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f246006cce0 0x7f246006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:24.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.656+0000 7f247659c700 1 --2- 172.21.15.71:0/56876331 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f246006cce0 0x7f246006f1a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2468005ea0 tx=0x7f2468005e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:24.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.660+0000 7f24677fe700 1 -- 172.21.15.71:0/56876331 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f246c0578a0 con 0x7f2478100750 2024-04-23T02:25:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.867+0000 7f247cf1c700 1 -- 172.21.15.71:0/56876331 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f24780611d0 con 0x7f246006cce0 2024-04-23T02:25:24.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.878+0000 7f24677fe700 1 -- 172.21.15.71:0/56876331 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f24780611d0 con 0x7f246006cce0 2024-04-23T02:25:24.880 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:25:24.881 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (9m) 2m ago 20m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:25:24.881 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (20m) 2m ago 20m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:25:24.881 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (19m) 2m ago 19m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:25:24.881 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (7m) 2m ago 20m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:25:24.881 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (6m) 2m ago 19m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:25:24.881 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (8m) 2m ago 20m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:25:24.881 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (3m) 2m ago 14m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:25:24.881 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (2m) 2m ago 14m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:25:24.881 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (14m) 2m ago 14m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:25:24.881 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (2m) 2m ago 14m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:25:24.881 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (11m) 2m ago 22m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:25:24.881 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (10m) 2m ago 19m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:25:24.881 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (7m) 2m ago 22m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:25:24.882 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (7m) 2m ago 19m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:25:24.882 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (10m) 2m ago 20m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:25:24.882 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (9m) 2m ago 19m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:25:24.882 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (6m) 2m ago 18m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:25:24.882 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (5m) 2m ago 17m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:25:24.882 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (5m) 2m ago 17m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:25:24.882 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (4m) 2m ago 16m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:25:24.882 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (4m) 2m ago 16m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:25:24.882 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (3m) 2m ago 16m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:25:24.882 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (9m) 2m ago 19m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:25:24.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.880+0000 7f247cf1c700 1 -- 172.21.15.71:0/56876331 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f246006cce0 msgr2=0x7f246006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:24.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.880+0000 7f247cf1c700 1 --2- 172.21.15.71:0/56876331 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f246006cce0 0x7f246006f1a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2468005ea0 tx=0x7f2468005e30 comp rx=0 tx=0).stop 2024-04-23T02:25:24.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.880+0000 7f247cf1c700 1 -- 172.21.15.71:0/56876331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2478100750 msgr2=0x7f2478101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:24.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.880+0000 7f247cf1c700 1 --2- 172.21.15.71:0/56876331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2478100750 0x7f2478101d80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f246c00bda0 tx=0x7f246c00be80 comp rx=0 tx=0).stop 2024-04-23T02:25:24.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.880+0000 7f247cf1c700 1 -- 172.21.15.71:0/56876331 shutdown_connections 2024-04-23T02:25:24.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.880+0000 7f247cf1c700 1 --2- 172.21.15.71:0/56876331 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f246006cce0 0x7f246006f1a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.880+0000 7f247cf1c700 1 --2- 172.21.15.71:0/56876331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24780ffda0 0x7f2478101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.880+0000 7f247cf1c700 1 --2- 172.21.15.71:0/56876331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2478100750 0x7f2478101d80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.880+0000 7f247cf1c700 1 -- 172.21.15.71:0/56876331 >> 172.21.15.71:0/56876331 conn(0x7f24780fb650 msgr2=0x7f2478069190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:24.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.880+0000 7f247cf1c700 1 -- 172.21.15.71:0/56876331 shutdown_connections 2024-04-23T02:25:24.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.880+0000 7f247cf1c700 1 -- 172.21.15.71:0/56876331 wait complete. 2024-04-23T02:25:24.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.981+0000 7fc0f15da700 1 -- 172.21.15.71:0/2012737398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec0ffb40 msgr2=0x7fc0ec0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:24.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.982+0000 7fc0f15da700 1 --2- 172.21.15.71:0/2012737398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec0ffb40 0x7fc0ec0fffc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009910 tx=0x7fc0dc009c20 comp rx=0 tx=0).stop 2024-04-23T02:25:24.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.982+0000 7fc0f15da700 1 -- 172.21.15.71:0/2012737398 shutdown_connections 2024-04-23T02:25:24.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.982+0000 7fc0f15da700 1 --2- 172.21.15.71:0/2012737398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec0ffb40 0x7fc0ec0fffc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.982+0000 7fc0f15da700 1 --2- 172.21.15.71:0/2012737398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0ec105bf0 0x7fc0ec105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.982+0000 7fc0f15da700 1 -- 172.21.15.71:0/2012737398 >> 172.21.15.71:0/2012737398 conn(0x7fc0ec0fb650 msgr2=0x7fc0ec0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:24.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.982+0000 7fc0f15da700 1 -- 172.21.15.71:0/2012737398 shutdown_connections 2024-04-23T02:25:24.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.982+0000 7fc0f15da700 1 -- 172.21.15.71:0/2012737398 wait complete. 2024-04-23T02:25:24.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.982+0000 7fc0f15da700 1 Processor -- start 2024-04-23T02:25:24.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.982+0000 7fc0f15da700 1 -- start start 2024-04-23T02:25:24.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.983+0000 7fc0f15da700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec0ffb40 0x7fc0ec195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:24.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.983+0000 7fc0f15da700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0ec105bf0 0x7fc0ec195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:24.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.983+0000 7fc0f15da700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec196230 con 0x7fc0ec0ffb40 2024-04-23T02:25:24.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.983+0000 7fc0f15da700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec199fc0 con 0x7fc0ec105bf0 2024-04-23T02:25:24.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.983+0000 7fc0eaffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec0ffb40 0x7fc0ec195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:24.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.983+0000 7fc0eaffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec0ffb40 0x7fc0ec195610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52594/0 (socket says 172.21.15.71:52594) 2024-04-23T02:25:24.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.983+0000 7fc0eaffd700 1 -- 172.21.15.71:0/3372227986 learned_addr learned my addr 172.21.15.71:0/3372227986 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:25:24.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.983+0000 7fc0ea7fc700 1 --2- 172.21.15.71:0/3372227986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0ec105bf0 0x7fc0ec195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:24.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.983+0000 7fc0eaffd700 1 -- 172.21.15.71:0/3372227986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0ec105bf0 msgr2=0x7fc0ec195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:24.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.983+0000 7fc0eaffd700 1 --2- 172.21.15.71:0/3372227986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0ec105bf0 0x7fc0ec195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:24.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.983+0000 7fc0eaffd700 1 -- 172.21.15.71:0/3372227986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0dc0095f0 con 0x7fc0ec0ffb40 2024-04-23T02:25:24.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.983+0000 7fc0eaffd700 1 --2- 172.21.15.71:0/3372227986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec0ffb40 0x7fc0ec195610 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000c8a0 tx=0x7fc0e000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:24.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.983+0000 7fc0d3fff700 1 -- 172.21.15.71:0/3372227986 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0e00078e0 con 0x7fc0ec0ffb40 2024-04-23T02:25:24.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.983+0000 7fc0d3fff700 1 -- 172.21.15.71:0/3372227986 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc0e000f450 con 0x7fc0ec0ffb40 2024-04-23T02:25:24.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.983+0000 7fc0f15da700 1 -- 172.21.15.71:0/3372227986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ec19a2a0 con 0x7fc0ec0ffb40 2024-04-23T02:25:24.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.983+0000 7fc0d3fff700 1 -- 172.21.15.71:0/3372227986 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0e000e5e0 con 0x7fc0ec0ffb40 2024-04-23T02:25:24.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.983+0000 7fc0f15da700 1 -- 172.21.15.71:0/3372227986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ec19a7f0 con 0x7fc0ec0ffb40 2024-04-23T02:25:24.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.984+0000 7fc0f15da700 1 -- 172.21.15.71:0/3372227986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ec1080a0 con 0x7fc0ec0ffb40 2024-04-23T02:25:24.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.985+0000 7fc0d3fff700 1 -- 172.21.15.71:0/3372227986 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc0e000f5c0 con 0x7fc0ec0ffb40 2024-04-23T02:25:24.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.985+0000 7fc0d3fff700 1 --2- 172.21.15.71:0/3372227986 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc0d406cce0 0x7fc0d406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:24.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.985+0000 7fc0ea7fc700 1 --2- 172.21.15.71:0/3372227986 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc0d406cce0 0x7fc0d406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:24.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.985+0000 7fc0d3fff700 1 -- 172.21.15.71:0/3372227986 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc0e008c1d0 con 0x7fc0ec0ffb40 2024-04-23T02:25:24.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.986+0000 7fc0ea7fc700 1 --2- 172.21.15.71:0/3372227986 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc0d406cce0 0x7fc0d406f1a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec196c30 tx=0x7fc0dc01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:24.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:24.990+0000 7fc0d3fff700 1 -- 172.21.15.71:0/3372227986 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc0e0056240 con 0x7fc0ec0ffb40 2024-04-23T02:25:25.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.264+0000 7fc0f15da700 1 -- 172.21.15.71:0/3372227986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc0ec04f2e0 con 0x7fc0ec0ffb40 2024-04-23T02:25:25.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.265+0000 7fc0d3fff700 1 -- 172.21.15.71:0/3372227986 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7fc0e0059860 con 0x7fc0ec0ffb40 2024-04-23T02:25:25.268 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:25:25.268 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:25:25.268 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:25:25.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:25:25.268 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:25:25.268 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:25:25.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:25:25.268 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:25:25.268 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:25:25.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:25:25.268 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:25:25.268 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:25:25.269 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:25:25.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:25:25.269 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:25:25.269 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:25:25.269 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:25:25.269 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:25:25.269 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:25:25.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.267+0000 7fc0f15da700 1 -- 172.21.15.71:0/3372227986 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc0d406cce0 msgr2=0x7fc0d406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:25.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.267+0000 7fc0f15da700 1 --2- 172.21.15.71:0/3372227986 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc0d406cce0 0x7fc0d406f1a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec196c30 tx=0x7fc0dc01a040 comp rx=0 tx=0).stop 2024-04-23T02:25:25.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.267+0000 7fc0f15da700 1 -- 172.21.15.71:0/3372227986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec0ffb40 msgr2=0x7fc0ec195610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:25.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.267+0000 7fc0f15da700 1 --2- 172.21.15.71:0/3372227986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec0ffb40 0x7fc0ec195610 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000c8a0 tx=0x7fc0e000cbb0 comp rx=0 tx=0).stop 2024-04-23T02:25:25.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.267+0000 7fc0f15da700 1 -- 172.21.15.71:0/3372227986 shutdown_connections 2024-04-23T02:25:25.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.267+0000 7fc0f15da700 1 --2- 172.21.15.71:0/3372227986 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc0d406cce0 0x7fc0d406f1a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:25.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.267+0000 7fc0f15da700 1 --2- 172.21.15.71:0/3372227986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec0ffb40 0x7fc0ec195610 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:25.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.267+0000 7fc0f15da700 1 --2- 172.21.15.71:0/3372227986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0ec105bf0 0x7fc0ec195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:25.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.267+0000 7fc0f15da700 1 -- 172.21.15.71:0/3372227986 >> 172.21.15.71:0/3372227986 conn(0x7fc0ec0fb650 msgr2=0x7fc0ec0fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:25.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.267+0000 7fc0f15da700 1 -- 172.21.15.71:0/3372227986 shutdown_connections 2024-04-23T02:25:25.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.267+0000 7fc0f15da700 1 -- 172.21.15.71:0/3372227986 wait complete. 2024-04-23T02:25:25.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.370+0000 7f98efb20700 1 -- 172.21.15.71:0/253146656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98e8101e80 msgr2=0x7f98e8102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:25.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.370+0000 7f98efb20700 1 --2- 172.21.15.71:0/253146656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98e8101e80 0x7f98e8102260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009910 tx=0x7f98d8009c20 comp rx=0 tx=0).stop 2024-04-23T02:25:25.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.370+0000 7f98efb20700 1 -- 172.21.15.71:0/253146656 shutdown_connections 2024-04-23T02:25:25.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.370+0000 7f98efb20700 1 --2- 172.21.15.71:0/253146656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98e81027a0 0x7f98e810ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:25.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.370+0000 7f98efb20700 1 --2- 172.21.15.71:0/253146656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98e8101e80 0x7f98e8102260 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:25.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.370+0000 7f98efb20700 1 -- 172.21.15.71:0/253146656 >> 172.21.15.71:0/253146656 conn(0x7f98e80fb610 msgr2=0x7f98e80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:25.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.370+0000 7f98efb20700 1 -- 172.21.15.71:0/253146656 shutdown_connections 2024-04-23T02:25:25.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.370+0000 7f98efb20700 1 -- 172.21.15.71:0/253146656 wait complete. 2024-04-23T02:25:25.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.371+0000 7f98efb20700 1 Processor -- start 2024-04-23T02:25:25.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.371+0000 7f98efb20700 1 -- start start 2024-04-23T02:25:25.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.371+0000 7f98efb20700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98e8101e80 0x7f98e80ffae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:25.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.371+0000 7f98efb20700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98e81027a0 0x7f98e8100020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:25.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.371+0000 7f98efb20700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e8100560 con 0x7f98e8101e80 2024-04-23T02:25:25.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.371+0000 7f98efb20700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e81006a0 con 0x7f98e81027a0 2024-04-23T02:25:25.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.371+0000 7f98ed8bc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98e8101e80 0x7f98e80ffae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:25.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.371+0000 7f98ed8bc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98e8101e80 0x7f98e80ffae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38800/0 (socket says 172.21.15.71:38800) 2024-04-23T02:25:25.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.371+0000 7f98ed8bc700 1 -- 172.21.15.71:0/1651473641 learned_addr learned my addr 172.21.15.71:0/1651473641 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:25:25.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.371+0000 7f98ed0bb700 1 --2- 172.21.15.71:0/1651473641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98e81027a0 0x7f98e8100020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:25.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.371+0000 7f98ed8bc700 1 -- 172.21.15.71:0/1651473641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98e81027a0 msgr2=0x7f98e8100020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:25.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.371+0000 7f98ed8bc700 1 --2- 172.21.15.71:0/1651473641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98e81027a0 0x7f98e8100020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:25.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.371+0000 7f98ed8bc700 1 -- 172.21.15.71:0/1651473641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d80095f0 con 0x7f98e8101e80 2024-04-23T02:25:25.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.372+0000 7f98ed8bc700 1 --2- 172.21.15.71:0/1651473641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98e8101e80 0x7f98e80ffae0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f98d8006010 tx=0x7f98d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:25.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.372+0000 7f98deffd700 1 -- 172.21.15.71:0/1651473641 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98d801d070 con 0x7f98e8101e80 2024-04-23T02:25:25.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.372+0000 7f98efb20700 1 -- 172.21.15.71:0/1651473641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e8100920 con 0x7f98e8101e80 2024-04-23T02:25:25.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.372+0000 7f98deffd700 1 -- 172.21.15.71:0/1651473641 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f98d800f460 con 0x7f98e8101e80 2024-04-23T02:25:25.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.372+0000 7f98deffd700 1 -- 172.21.15.71:0/1651473641 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98d8021620 con 0x7f98e8101e80 2024-04-23T02:25:25.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.372+0000 7f98efb20700 1 -- 172.21.15.71:0/1651473641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e81a2de0 con 0x7f98e8101e80 2024-04-23T02:25:25.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.373+0000 7f98efb20700 1 -- 172.21.15.71:0/1651473641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e8108390 con 0x7f98e8101e80 2024-04-23T02:25:25.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.373+0000 7f98deffd700 1 -- 172.21.15.71:0/1651473641 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f98d800f5d0 con 0x7f98e8101e80 2024-04-23T02:25:25.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.374+0000 7f98deffd700 1 --2- 172.21.15.71:0/1651473641 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98d406cce0 0x7f98d406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:25.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.374+0000 7f98deffd700 1 -- 172.21.15.71:0/1651473641 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f98d808d8a0 con 0x7f98e8101e80 2024-04-23T02:25:25.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.374+0000 7f98ed0bb700 1 --2- 172.21.15.71:0/1651473641 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98d406cce0 0x7f98d406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:25.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.374+0000 7f98ed0bb700 1 --2- 172.21.15.71:0/1651473641 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98d406cce0 0x7f98d406f1a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f98e4006fd0 tx=0x7f98e4008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:25.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.378+0000 7f98deffd700 1 -- 172.21.15.71:0/1651473641 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f98d8057910 con 0x7f98e8101e80 2024-04-23T02:25:25.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.611+0000 7f98efb20700 1 -- 172.21.15.71:0/1651473641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f98e8101120 con 0x7f98e8101e80 2024-04-23T02:25:25.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.612+0000 7f98deffd700 1 -- 172.21.15.71:0/1651473641 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f98d805af30 con 0x7f98e8101e80 2024-04-23T02:25:25.615 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:25:25.615 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:25:25.615 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:25:25.615 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:25:25.615 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:25:25.615 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:25:25.615 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:25:25.615 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:25:25.615 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:25:25.615 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:25:25.615 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:25:25.615 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:25:25.616 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:25:25.616 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:25:25.616 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:25:25.616 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:25:25.616 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:25:25.616 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:25:25.616 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:25:25.616 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:25:25.616 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:25:25.616 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:25:25.616 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:25:25.616 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:25:25.616 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:25:25.616 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:25:25.616 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:25:25.617 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:25:25.617 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:25:25.617 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:25:25.617 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:25:25.617 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:25:25.617 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:25:25.617 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:25:25.617 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:25:25.617 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:25:25.617 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:25:25.617 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:25:25.617 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:25:25.617 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:25:25.617 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:25:25.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.614+0000 7f98efb20700 1 -- 172.21.15.71:0/1651473641 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98d406cce0 msgr2=0x7f98d406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:25.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.614+0000 7f98efb20700 1 --2- 172.21.15.71:0/1651473641 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98d406cce0 0x7f98d406f1a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f98e4006fd0 tx=0x7f98e4008040 comp rx=0 tx=0).stop 2024-04-23T02:25:25.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.614+0000 7f98efb20700 1 -- 172.21.15.71:0/1651473641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98e8101e80 msgr2=0x7f98e80ffae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:25.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.614+0000 7f98efb20700 1 --2- 172.21.15.71:0/1651473641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98e8101e80 0x7f98e80ffae0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f98d8006010 tx=0x7f98d800be20 comp rx=0 tx=0).stop 2024-04-23T02:25:25.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.614+0000 7f98efb20700 1 -- 172.21.15.71:0/1651473641 shutdown_connections 2024-04-23T02:25:25.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.614+0000 7f98efb20700 1 --2- 172.21.15.71:0/1651473641 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98d406cce0 0x7f98d406f1a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:25.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.614+0000 7f98efb20700 1 --2- 172.21.15.71:0/1651473641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98e8101e80 0x7f98e80ffae0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:25.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.614+0000 7f98efb20700 1 --2- 172.21.15.71:0/1651473641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98e81027a0 0x7f98e8100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:25.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.614+0000 7f98efb20700 1 -- 172.21.15.71:0/1651473641 >> 172.21.15.71:0/1651473641 conn(0x7f98e80fb610 msgr2=0x7f98e81054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:25.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.614+0000 7f98efb20700 1 -- 172.21.15.71:0/1651473641 shutdown_connections 2024-04-23T02:25:25.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.614+0000 7f98efb20700 1 -- 172.21.15.71:0/1651473641 wait complete. 2024-04-23T02:25:25.619 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:25:25.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:25 smithi192 ceph-mon[108617]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:25:25.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:25 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3372227986' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:25:25.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.717+0000 7fa7adf38700 1 -- 172.21.15.71:0/2213599398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8100e90 msgr2=0x7fa7a8101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:25.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.717+0000 7fa7adf38700 1 --2- 172.21.15.71:0/2213599398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8100e90 0x7fa7a8101270 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa798009910 tx=0x7fa798009c20 comp rx=0 tx=0).stop 2024-04-23T02:25:25.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.718+0000 7fa7adf38700 1 -- 172.21.15.71:0/2213599398 shutdown_connections 2024-04-23T02:25:25.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.718+0000 7fa7adf38700 1 --2- 172.21.15.71:0/2213599398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7a8101840 0x7fa7a8105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:25.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.718+0000 7fa7adf38700 1 --2- 172.21.15.71:0/2213599398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8100e90 0x7fa7a8101270 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:25.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.718+0000 7fa7adf38700 1 -- 172.21.15.71:0/2213599398 >> 172.21.15.71:0/2213599398 conn(0x7fa7a80fc740 msgr2=0x7fa7a80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:25.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.718+0000 7fa7adf38700 1 -- 172.21.15.71:0/2213599398 shutdown_connections 2024-04-23T02:25:25.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.718+0000 7fa7adf38700 1 -- 172.21.15.71:0/2213599398 wait complete. 2024-04-23T02:25:25.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.718+0000 7fa7adf38700 1 Processor -- start 2024-04-23T02:25:25.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.718+0000 7fa7adf38700 1 -- start start 2024-04-23T02:25:25.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.718+0000 7fa7adf38700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7a8100e90 0x7fa7a8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:25.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.718+0000 7fa7adf38700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8101840 0x7fa7a80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:25.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.718+0000 7fa7adf38700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a8073b10 con 0x7fa7a8101840 2024-04-23T02:25:25.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.718+0000 7fa7adf38700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a8073c50 con 0x7fa7a8100e90 2024-04-23T02:25:25.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.719+0000 7fa7a6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8101840 0x7fa7a80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:25.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.719+0000 7fa7a6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8101840 0x7fa7a80735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38814/0 (socket says 172.21.15.71:38814) 2024-04-23T02:25:25.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.719+0000 7fa7a6ffd700 1 -- 172.21.15.71:0/3775377361 learned_addr learned my addr 172.21.15.71:0/3775377361 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:25:25.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.719+0000 7fa7a77fe700 1 --2- 172.21.15.71:0/3775377361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7a8100e90 0x7fa7a8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:25.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.719+0000 7fa7a6ffd700 1 -- 172.21.15.71:0/3775377361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7a8100e90 msgr2=0x7fa7a8073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:25.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.719+0000 7fa7a6ffd700 1 --2- 172.21.15.71:0/3775377361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7a8100e90 0x7fa7a8073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:25.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.719+0000 7fa7a6ffd700 1 -- 172.21.15.71:0/3775377361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7980095f0 con 0x7fa7a8101840 2024-04-23T02:25:25.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.719+0000 7fa7a6ffd700 1 --2- 172.21.15.71:0/3775377361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8101840 0x7fa7a80735d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00b480 tx=0x7fa79c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:25.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.719+0000 7fa7a4ff9700 1 -- 172.21.15.71:0/3775377361 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa79c0118c0 con 0x7fa7a8101840 2024-04-23T02:25:25.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.719+0000 7fa7adf38700 1 -- 172.21.15.71:0/3775377361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a81a2ae0 con 0x7fa7a8101840 2024-04-23T02:25:25.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.719+0000 7fa7a4ff9700 1 -- 172.21.15.71:0/3775377361 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa79c010380 con 0x7fa7a8101840 2024-04-23T02:25:25.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.719+0000 7fa7a4ff9700 1 -- 172.21.15.71:0/3775377361 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa79c00f510 con 0x7fa7a8101840 2024-04-23T02:25:25.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.719+0000 7fa7adf38700 1 -- 172.21.15.71:0/3775377361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a81a2ef0 con 0x7fa7a8101840 2024-04-23T02:25:25.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.720+0000 7fa7adf38700 1 -- 172.21.15.71:0/3775377361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a8109230 con 0x7fa7a8101840 2024-04-23T02:25:25.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.721+0000 7fa7a4ff9700 1 -- 172.21.15.71:0/3775377361 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa79c0104f0 con 0x7fa7a8101840 2024-04-23T02:25:25.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.721+0000 7fa7a4ff9700 1 --2- 172.21.15.71:0/3775377361 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa79006cce0 0x7fa79006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:25.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.721+0000 7fa7a4ff9700 1 -- 172.21.15.71:0/3775377361 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa79c08c210 con 0x7fa7a8101840 2024-04-23T02:25:25.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.721+0000 7fa7a77fe700 1 --2- 172.21.15.71:0/3775377361 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa79006cce0 0x7fa79006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:25.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.721+0000 7fa7a77fe700 1 --2- 172.21.15.71:0/3775377361 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa79006cce0 0x7fa79006f1a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa798006010 tx=0x7fa79801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:25.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.725+0000 7fa7a4ff9700 1 -- 172.21.15.71:0/3775377361 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa79c056280 con 0x7fa7a8101840 2024-04-23T02:25:25.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:25 smithi071 ceph-mon[144544]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:25:25.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:25 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3372227986' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:25:25.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.938+0000 7fa7adf38700 1 -- 172.21.15.71:0/3775377361 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7a80611d0 con 0x7fa79006cce0 2024-04-23T02:25:25.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.939+0000 7fa7a4ff9700 1 -- 172.21.15.71:0/3775377361 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa7a80611d0 con 0x7fa79006cce0 2024-04-23T02:25:25.942 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:25:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:25:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:25:25.942 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:25:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:25:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:25:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:25:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:25:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:25:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:25:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:25:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:25:25.943 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:25:25.943 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:25:25.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.942+0000 7fa7adf38700 1 -- 172.21.15.71:0/3775377361 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa79006cce0 msgr2=0x7fa79006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:25.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.942+0000 7fa7adf38700 1 --2- 172.21.15.71:0/3775377361 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa79006cce0 0x7fa79006f1a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa798006010 tx=0x7fa79801a040 comp rx=0 tx=0).stop 2024-04-23T02:25:25.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.942+0000 7fa7adf38700 1 -- 172.21.15.71:0/3775377361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8101840 msgr2=0x7fa7a80735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:25.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.942+0000 7fa7adf38700 1 --2- 172.21.15.71:0/3775377361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8101840 0x7fa7a80735d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00b480 tx=0x7fa79c00b840 comp rx=0 tx=0).stop 2024-04-23T02:25:25.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.942+0000 7fa7adf38700 1 -- 172.21.15.71:0/3775377361 shutdown_connections 2024-04-23T02:25:25.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.942+0000 7fa7adf38700 1 --2- 172.21.15.71:0/3775377361 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa79006cce0 0x7fa79006f1a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:25.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.942+0000 7fa7adf38700 1 --2- 172.21.15.71:0/3775377361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7a8100e90 0x7fa7a8073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:25.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.942+0000 7fa7adf38700 1 --2- 172.21.15.71:0/3775377361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8101840 0x7fa7a80735d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:25.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.942+0000 7fa7adf38700 1 -- 172.21.15.71:0/3775377361 >> 172.21.15.71:0/3775377361 conn(0x7fa7a80fc740 msgr2=0x7fa7a80fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:25.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.942+0000 7fa7adf38700 1 -- 172.21.15.71:0/3775377361 shutdown_connections 2024-04-23T02:25:25.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:25.942+0000 7fa7adf38700 1 -- 172.21.15.71:0/3775377361 wait complete. 2024-04-23T02:25:26.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.045+0000 7f770867d700 1 -- 172.21.15.71:0/2398268450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7700105bd0 msgr2=0x7f7700105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:26.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.045+0000 7f770867d700 1 --2- 172.21.15.71:0/2398268450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7700105bd0 0x7f7700105fb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009910 tx=0x7f76f0009c20 comp rx=0 tx=0).stop 2024-04-23T02:25:26.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.045+0000 7f770867d700 1 -- 172.21.15.71:0/2398268450 shutdown_connections 2024-04-23T02:25:26.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.045+0000 7f770867d700 1 --2- 172.21.15.71:0/2398268450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77000ffc20 0x7f7700100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:26.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.045+0000 7f770867d700 1 --2- 172.21.15.71:0/2398268450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7700105bd0 0x7f7700105fb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:26.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.045+0000 7f770867d700 1 -- 172.21.15.71:0/2398268450 >> 172.21.15.71:0/2398268450 conn(0x7f7700075670 msgr2=0x7f7700075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:26.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.045+0000 7f770867d700 1 -- 172.21.15.71:0/2398268450 shutdown_connections 2024-04-23T02:25:26.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.045+0000 7f770867d700 1 -- 172.21.15.71:0/2398268450 wait complete. 2024-04-23T02:25:26.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.046+0000 7f770867d700 1 Processor -- start 2024-04-23T02:25:26.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.046+0000 7f770867d700 1 -- start start 2024-04-23T02:25:26.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.046+0000 7f770867d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77000ffc20 0x7f7700101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:26.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.046+0000 7f770867d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7700105bd0 0x7f7700101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:26.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.046+0000 7f770867d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77001022c0 con 0x7f77000ffc20 2024-04-23T02:25:26.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.046+0000 7f770867d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7700102400 con 0x7f7700105bd0 2024-04-23T02:25:26.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.046+0000 7f7706419700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77000ffc20 0x7f7700101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:26.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.046+0000 7f7706419700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77000ffc20 0x7f7700101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38830/0 (socket says 172.21.15.71:38830) 2024-04-23T02:25:26.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.046+0000 7f7706419700 1 -- 172.21.15.71:0/3351737486 learned_addr learned my addr 172.21.15.71:0/3351737486 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:25:26.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.046+0000 7f7706419700 1 -- 172.21.15.71:0/3351737486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7700105bd0 msgr2=0x7f7700101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:26.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.046+0000 7f7706419700 1 --2- 172.21.15.71:0/3351737486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7700105bd0 0x7f7700101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:26.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.046+0000 7f7706419700 1 -- 172.21.15.71:0/3351737486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f00095f0 con 0x7f77000ffc20 2024-04-23T02:25:26.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.046+0000 7f7706419700 1 --2- 172.21.15.71:0/3351737486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77000ffc20 0x7f7700101840 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f76f0006010 tx=0x7f76f000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:26.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.046+0000 7f76f77fe700 1 -- 172.21.15.71:0/3351737486 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76f001d070 con 0x7f77000ffc20 2024-04-23T02:25:26.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.047+0000 7f76f77fe700 1 -- 172.21.15.71:0/3351737486 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f76f000f460 con 0x7f77000ffc20 2024-04-23T02:25:26.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.047+0000 7f770867d700 1 -- 172.21.15.71:0/3351737486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7700102680 con 0x7f77000ffc20 2024-04-23T02:25:26.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.047+0000 7f76f77fe700 1 -- 172.21.15.71:0/3351737486 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76f0021640 con 0x7f77000ffc20 2024-04-23T02:25:26.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.047+0000 7f770867d700 1 -- 172.21.15.71:0/3351737486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77001a2d80 con 0x7f77000ffc20 2024-04-23T02:25:26.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.047+0000 7f770867d700 1 -- 172.21.15.71:0/3351737486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770004f2e0 con 0x7f77000ffc20 2024-04-23T02:25:26.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.048+0000 7f76f77fe700 1 -- 172.21.15.71:0/3351737486 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f76f000fa60 con 0x7f77000ffc20 2024-04-23T02:25:26.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.048+0000 7f76f77fe700 1 --2- 172.21.15.71:0/3351737486 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f76ec06cce0 0x7f76ec06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:26.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.048+0000 7f76f77fe700 1 -- 172.21.15.71:0/3351737486 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f76f008d860 con 0x7f77000ffc20 2024-04-23T02:25:26.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.048+0000 7f7705c18700 1 --2- 172.21.15.71:0/3351737486 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f76ec06cce0 0x7f76ec06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:26.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.049+0000 7f7705c18700 1 --2- 172.21.15.71:0/3351737486 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f76ec06cce0 0x7f76ec06f1a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7700103030 tx=0x7f76fc006d20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:26.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.053+0000 7f76f77fe700 1 -- 172.21.15.71:0/3351737486 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f76f00578d0 con 0x7f77000ffc20 2024-04-23T02:25:26.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.323+0000 7f770867d700 1 -- 172.21.15.71:0/3351737486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f77001a30d0 con 0x7f77000ffc20 2024-04-23T02:25:26.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.323+0000 7f76f77fe700 1 -- 172.21.15.71:0/3351737486 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f76f005aef0 con 0x7f77000ffc20 2024-04-23T02:25:26.326 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:25:26.326 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:25:26.326 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:25:26.326 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:25:26.326 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:25:26.326 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:25:26.326 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:25:26.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.325+0000 7f770867d700 1 -- 172.21.15.71:0/3351737486 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f76ec06cce0 msgr2=0x7f76ec06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:26.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.325+0000 7f770867d700 1 --2- 172.21.15.71:0/3351737486 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f76ec06cce0 0x7f76ec06f1a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7700103030 tx=0x7f76fc006d20 comp rx=0 tx=0).stop 2024-04-23T02:25:26.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.325+0000 7f770867d700 1 -- 172.21.15.71:0/3351737486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77000ffc20 msgr2=0x7f7700101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:26.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.325+0000 7f770867d700 1 --2- 172.21.15.71:0/3351737486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77000ffc20 0x7f7700101840 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f76f0006010 tx=0x7f76f000be20 comp rx=0 tx=0).stop 2024-04-23T02:25:26.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.325+0000 7f770867d700 1 -- 172.21.15.71:0/3351737486 shutdown_connections 2024-04-23T02:25:26.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.325+0000 7f770867d700 1 --2- 172.21.15.71:0/3351737486 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f76ec06cce0 0x7f76ec06f1a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:26.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.325+0000 7f770867d700 1 --2- 172.21.15.71:0/3351737486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77000ffc20 0x7f7700101840 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:26.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.325+0000 7f770867d700 1 --2- 172.21.15.71:0/3351737486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7700105bd0 0x7f7700101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:26.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.325+0000 7f770867d700 1 -- 172.21.15.71:0/3351737486 >> 172.21.15.71:0/3351737486 conn(0x7f7700075670 msgr2=0x7f7700068740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:26.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.325+0000 7f770867d700 1 -- 172.21.15.71:0/3351737486 shutdown_connections 2024-04-23T02:25:26.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:26.325+0000 7f770867d700 1 -- 172.21.15.71:0/3351737486 wait complete. 2024-04-23T02:25:26.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:26 smithi192 ceph-mon[108617]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:25:26.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:26 smithi192 ceph-mon[108617]: pgmap v363: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:26.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:26 smithi192 ceph-mon[108617]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:25:26.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:26 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1651473641' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:25:26.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:26 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3351737486' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:25:26.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:26 smithi071 ceph-mon[144544]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:25:26.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:26 smithi071 ceph-mon[144544]: pgmap v363: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:26.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:26 smithi071 ceph-mon[144544]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:25:26.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:26 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1651473641' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:25:26.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:26 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3351737486' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:25:27.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:27 smithi192 ceph-mon[108617]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:25:27.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:27 smithi071 ceph-mon[144544]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:25:28.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:28 smithi192 ceph-mon[108617]: pgmap v364: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:28.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:28 smithi071 ceph-mon[144544]: pgmap v364: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:30.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:30 smithi192 ceph-mon[108617]: pgmap v365: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:30.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:30 smithi071 ceph-mon[144544]: pgmap v365: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:32.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:32 smithi192 ceph-mon[108617]: pgmap v366: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:32.632 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:32 smithi071 ceph-mon[144544]: pgmap v366: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:34.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:34 smithi192 ceph-mon[108617]: pgmap v367: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:34.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:34 smithi071 ceph-mon[144544]: pgmap v367: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:35.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:25:35.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:25:36.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:36 smithi071 ceph-mon[144544]: pgmap v368: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:36.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:36 smithi192 ceph-mon[108617]: pgmap v368: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:38.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:38 smithi071 ceph-mon[144544]: pgmap v369: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:38.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:38 smithi192 ceph-mon[108617]: pgmap v369: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:40.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:40 smithi071 ceph-mon[144544]: pgmap v370: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:40.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:40 smithi192 ceph-mon[108617]: pgmap v370: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:42.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:42 smithi071 ceph-mon[144544]: pgmap v371: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:42.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:42 smithi192 ceph-mon[108617]: pgmap v371: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:44.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:44 smithi192 ceph-mon[108617]: pgmap v372: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:44.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:44 smithi071 ceph-mon[144544]: pgmap v372: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:46.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:46 smithi071 ceph-mon[144544]: pgmap v373: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:46.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:46 smithi192 ceph-mon[108617]: pgmap v373: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:48.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:48 smithi071 ceph-mon[144544]: pgmap v374: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:48.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:48 smithi192 ceph-mon[108617]: pgmap v374: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:50.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:50 smithi071 ceph-mon[144544]: pgmap v375: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:50.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:25:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:50 smithi192 ceph-mon[108617]: pgmap v375: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:50.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:25:52.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:52 smithi071 ceph-mon[144544]: pgmap v376: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:52.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:52 smithi192 ceph-mon[108617]: pgmap v376: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:54.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:54 smithi071 ceph-mon[144544]: pgmap v377: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:54.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:54 smithi192 ceph-mon[108617]: pgmap v377: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:56.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.442+0000 7ff4d9643700 1 -- 172.21.15.71:0/736685083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4d41006b0 msgr2=0x7ff4d4100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:56.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.442+0000 7ff4d9643700 1 --2- 172.21.15.71:0/736685083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4d41006b0 0x7ff4d4100a90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009910 tx=0x7ff4c8009c20 comp rx=0 tx=0).stop 2024-04-23T02:25:56.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.442+0000 7ff4d9643700 1 -- 172.21.15.71:0/736685083 shutdown_connections 2024-04-23T02:25:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.442+0000 7ff4d9643700 1 --2- 172.21.15.71:0/736685083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4d4101060 0x7ff4d41047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.442+0000 7ff4d9643700 1 --2- 172.21.15.71:0/736685083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4d41006b0 0x7ff4d4100a90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.442+0000 7ff4d9643700 1 -- 172.21.15.71:0/736685083 >> 172.21.15.71:0/736685083 conn(0x7ff4d4074e30 msgr2=0x7ff4d4075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.442+0000 7ff4d9643700 1 -- 172.21.15.71:0/736685083 shutdown_connections 2024-04-23T02:25:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.442+0000 7ff4d9643700 1 -- 172.21.15.71:0/736685083 wait complete. 2024-04-23T02:25:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.443+0000 7ff4d9643700 1 Processor -- start 2024-04-23T02:25:56.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.443+0000 7ff4d9643700 1 -- start start 2024-04-23T02:25:56.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.443+0000 7ff4d9643700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4d41006b0 0x7ff4d410bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:56.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.443+0000 7ff4d9643700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4d4101060 0x7ff4d410c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:56.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.443+0000 7ff4d9643700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d410c760 con 0x7ff4d41006b0 2024-04-23T02:25:56.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.443+0000 7ff4d9643700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d410c8a0 con 0x7ff4d4101060 2024-04-23T02:25:56.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.443+0000 7ff4d2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4d41006b0 0x7ff4d410bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:56.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.443+0000 7ff4d2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4d41006b0 0x7ff4d410bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40394/0 (socket says 172.21.15.71:40394) 2024-04-23T02:25:56.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.443+0000 7ff4d2ffd700 1 -- 172.21.15.71:0/678736192 learned_addr learned my addr 172.21.15.71:0/678736192 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:25:56.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.443+0000 7ff4d27fc700 1 --2- 172.21.15.71:0/678736192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4d4101060 0x7ff4d410c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:56.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.443+0000 7ff4d2ffd700 1 -- 172.21.15.71:0/678736192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4d4101060 msgr2=0x7ff4d410c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:56.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.443+0000 7ff4d2ffd700 1 --2- 172.21.15.71:0/678736192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4d4101060 0x7ff4d410c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.443+0000 7ff4d2ffd700 1 -- 172.21.15.71:0/678736192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c80095f0 con 0x7ff4d41006b0 2024-04-23T02:25:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.444+0000 7ff4d2ffd700 1 --2- 172.21.15.71:0/678736192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4d41006b0 0x7ff4d410bce0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8000c00 tx=0x7ff4c800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.444+0000 7ff4bbfff700 1 -- 172.21.15.71:0/678736192 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4c801d070 con 0x7ff4d41006b0 2024-04-23T02:25:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.444+0000 7ff4bbfff700 1 -- 172.21.15.71:0/678736192 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff4c80044c0 con 0x7ff4d41006b0 2024-04-23T02:25:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.444+0000 7ff4d9643700 1 -- 172.21.15.71:0/678736192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d410cb20 con 0x7ff4d41006b0 2024-04-23T02:25:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.444+0000 7ff4bbfff700 1 -- 172.21.15.71:0/678736192 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4c80039d0 con 0x7ff4d41006b0 2024-04-23T02:25:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.444+0000 7ff4d9643700 1 -- 172.21.15.71:0/678736192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d41a2c20 con 0x7ff4d41006b0 2024-04-23T02:25:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.445+0000 7ff4d9643700 1 -- 172.21.15.71:0/678736192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d4108120 con 0x7ff4d41006b0 2024-04-23T02:25:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.445+0000 7ff4bbfff700 1 -- 172.21.15.71:0/678736192 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff4c80048e0 con 0x7ff4d41006b0 2024-04-23T02:25:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.446+0000 7ff4bbfff700 1 --2- 172.21.15.71:0/678736192 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff4bc06cce0 0x7ff4bc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.446+0000 7ff4bbfff700 1 -- 172.21.15.71:0/678736192 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff4c808dc60 con 0x7ff4d41006b0 2024-04-23T02:25:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.446+0000 7ff4d27fc700 1 --2- 172.21.15.71:0/678736192 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff4bc06cce0 0x7ff4bc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.446+0000 7ff4d27fc700 1 --2- 172.21.15.71:0/678736192 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff4bc06cce0 0x7ff4bc06f1a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff4d410d4d0 tx=0x7ff4c4009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:56.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.450+0000 7ff4bbfff700 1 -- 172.21.15.71:0/678736192 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff4c8057cd0 con 0x7ff4d41006b0 2024-04-23T02:25:56.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.661+0000 7ff4d9643700 1 -- 172.21.15.71:0/678736192 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4d40611d0 con 0x7ff4bc06cce0 2024-04-23T02:25:56.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.663+0000 7ff4bbfff700 1 -- 172.21.15.71:0/678736192 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff4d40611d0 con 0x7ff4bc06cce0 2024-04-23T02:25:56.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.665+0000 7ff4d9643700 1 -- 172.21.15.71:0/678736192 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff4bc06cce0 msgr2=0x7ff4bc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:56.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.665+0000 7ff4d9643700 1 --2- 172.21.15.71:0/678736192 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff4bc06cce0 0x7ff4bc06f1a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff4d410d4d0 tx=0x7ff4c4009260 comp rx=0 tx=0).stop 2024-04-23T02:25:56.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.665+0000 7ff4d9643700 1 -- 172.21.15.71:0/678736192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4d41006b0 msgr2=0x7ff4d410bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:56.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.665+0000 7ff4d9643700 1 --2- 172.21.15.71:0/678736192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4d41006b0 0x7ff4d410bce0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8000c00 tx=0x7ff4c800be90 comp rx=0 tx=0).stop 2024-04-23T02:25:56.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.665+0000 7ff4d9643700 1 -- 172.21.15.71:0/678736192 shutdown_connections 2024-04-23T02:25:56.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.665+0000 7ff4d9643700 1 --2- 172.21.15.71:0/678736192 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff4bc06cce0 0x7ff4bc06f1a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:56.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.665+0000 7ff4d9643700 1 --2- 172.21.15.71:0/678736192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4d41006b0 0x7ff4d410bce0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:56.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.665+0000 7ff4d9643700 1 --2- 172.21.15.71:0/678736192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4d4101060 0x7ff4d410c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:56.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.665+0000 7ff4d9643700 1 -- 172.21.15.71:0/678736192 >> 172.21.15.71:0/678736192 conn(0x7ff4d4074e30 msgr2=0x7ff4d40fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:56.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.665+0000 7ff4d9643700 1 -- 172.21.15.71:0/678736192 shutdown_connections 2024-04-23T02:25:56.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.665+0000 7ff4d9643700 1 -- 172.21.15.71:0/678736192 wait complete. 2024-04-23T02:25:56.678 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:25:56.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:56 smithi071 ceph-mon[144544]: pgmap v378: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:56.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.769+0000 7f05f4c3a700 1 -- 172.21.15.71:0/2926279072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05f00ff340 msgr2=0x7f05f00ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:56.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.769+0000 7f05f4c3a700 1 --2- 172.21.15.71:0/2926279072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05f00ff340 0x7f05f00ff7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009910 tx=0x7f05e0009c20 comp rx=0 tx=0).stop 2024-04-23T02:25:56.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.769+0000 7f05f4c3a700 1 -- 172.21.15.71:0/2926279072 shutdown_connections 2024-04-23T02:25:56.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.769+0000 7f05f4c3a700 1 --2- 172.21.15.71:0/2926279072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05f00ff340 0x7f05f00ff7a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:56.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.769+0000 7f05f4c3a700 1 --2- 172.21.15.71:0/2926279072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05f0105350 0x7f05f0105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:56.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.769+0000 7f05f4c3a700 1 -- 172.21.15.71:0/2926279072 >> 172.21.15.71:0/2926279072 conn(0x7f05f0074e30 msgr2=0x7f05f0075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:56.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.769+0000 7f05f4c3a700 1 -- 172.21.15.71:0/2926279072 shutdown_connections 2024-04-23T02:25:56.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.769+0000 7f05f4c3a700 1 -- 172.21.15.71:0/2926279072 wait complete. 2024-04-23T02:25:56.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.769+0000 7f05f4c3a700 1 Processor -- start 2024-04-23T02:25:56.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.770+0000 7f05f4c3a700 1 -- start start 2024-04-23T02:25:56.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.770+0000 7f05f4c3a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05f00ff340 0x7f05f0100fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:56.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.770+0000 7f05f4c3a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05f0105350 0x7f05f0101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:56.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.770+0000 7f05f4c3a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f0101a40 con 0x7f05f00ff340 2024-04-23T02:25:56.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.770+0000 7f05f4c3a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f0101b80 con 0x7f05f0105350 2024-04-23T02:25:56.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.770+0000 7f05ee59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05f00ff340 0x7f05f0100fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:56.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.770+0000 7f05ee59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05f00ff340 0x7f05f0100fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40412/0 (socket says 172.21.15.71:40412) 2024-04-23T02:25:56.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.770+0000 7f05ee59c700 1 -- 172.21.15.71:0/3692564411 learned_addr learned my addr 172.21.15.71:0/3692564411 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:25:56.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.770+0000 7f05edd9b700 1 --2- 172.21.15.71:0/3692564411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05f0105350 0x7f05f0101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:56.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.770+0000 7f05ee59c700 1 -- 172.21.15.71:0/3692564411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05f0105350 msgr2=0x7f05f0101500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:56.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.770+0000 7f05ee59c700 1 --2- 172.21.15.71:0/3692564411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05f0105350 0x7f05f0101500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:56.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.770+0000 7f05ee59c700 1 -- 172.21.15.71:0/3692564411 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e00095f0 con 0x7f05f00ff340 2024-04-23T02:25:56.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.770+0000 7f05ee59c700 1 --2- 172.21.15.71:0/3692564411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05f00ff340 0x7f05f0100fc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f05e400b4d0 tx=0x7f05e400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:56.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.770+0000 7f05df7fe700 1 -- 172.21.15.71:0/3692564411 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05e4011800 con 0x7f05f00ff340 2024-04-23T02:25:56.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.770+0000 7f05df7fe700 1 -- 172.21.15.71:0/3692564411 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f05e4011e00 con 0x7f05f00ff340 2024-04-23T02:25:56.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.770+0000 7f05f4c3a700 1 -- 172.21.15.71:0/3692564411 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f01a2b40 con 0x7f05f00ff340 2024-04-23T02:25:56.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.771+0000 7f05df7fe700 1 -- 172.21.15.71:0/3692564411 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05e400f2e0 con 0x7f05f00ff340 2024-04-23T02:25:56.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.771+0000 7f05f4c3a700 1 -- 172.21.15.71:0/3692564411 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f01a2f50 con 0x7f05f00ff340 2024-04-23T02:25:56.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.772+0000 7f05f4c3a700 1 -- 172.21.15.71:0/3692564411 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f0109880 con 0x7f05f00ff340 2024-04-23T02:25:56.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.772+0000 7f05df7fe700 1 -- 172.21.15.71:0/3692564411 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f05e4010380 con 0x7f05f00ff340 2024-04-23T02:25:56.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.772+0000 7f05df7fe700 1 --2- 172.21.15.71:0/3692564411 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f05d806cce0 0x7f05d806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:56.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.772+0000 7f05df7fe700 1 -- 172.21.15.71:0/3692564411 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f05e408c160 con 0x7f05f00ff340 2024-04-23T02:25:56.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.773+0000 7f05edd9b700 1 --2- 172.21.15.71:0/3692564411 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f05d806cce0 0x7f05d806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:56.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.773+0000 7f05edd9b700 1 --2- 172.21.15.71:0/3692564411 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f05d806cce0 0x7f05d806f1a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f05f01027b0 tx=0x7f05e000b560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:56.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.777+0000 7f05df7fe700 1 -- 172.21.15.71:0/3692564411 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f05e40561d0 con 0x7f05f00ff340 2024-04-23T02:25:56.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:56 smithi192 ceph-mon[108617]: pgmap v378: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:56.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.988+0000 7f05f4c3a700 1 -- 172.21.15.71:0/3692564411 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f05f00611d0 con 0x7f05d806cce0 2024-04-23T02:25:56.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.989+0000 7f05df7fe700 1 -- 172.21.15.71:0/3692564411 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f05f00611d0 con 0x7f05d806cce0 2024-04-23T02:25:56.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.991+0000 7f05f4c3a700 1 -- 172.21.15.71:0/3692564411 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f05d806cce0 msgr2=0x7f05d806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:56.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.991+0000 7f05f4c3a700 1 --2- 172.21.15.71:0/3692564411 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f05d806cce0 0x7f05d806f1a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f05f01027b0 tx=0x7f05e000b560 comp rx=0 tx=0).stop 2024-04-23T02:25:56.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.991+0000 7f05f4c3a700 1 -- 172.21.15.71:0/3692564411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05f00ff340 msgr2=0x7f05f0100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:56.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.991+0000 7f05f4c3a700 1 --2- 172.21.15.71:0/3692564411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05f00ff340 0x7f05f0100fc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f05e400b4d0 tx=0x7f05e400b890 comp rx=0 tx=0).stop 2024-04-23T02:25:56.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.992+0000 7f05f4c3a700 1 -- 172.21.15.71:0/3692564411 shutdown_connections 2024-04-23T02:25:56.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.992+0000 7f05f4c3a700 1 --2- 172.21.15.71:0/3692564411 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f05d806cce0 0x7f05d806f1a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:56.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.992+0000 7f05f4c3a700 1 --2- 172.21.15.71:0/3692564411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05f00ff340 0x7f05f0100fc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:56.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.992+0000 7f05f4c3a700 1 --2- 172.21.15.71:0/3692564411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05f0105350 0x7f05f0101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:56.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.992+0000 7f05f4c3a700 1 -- 172.21.15.71:0/3692564411 >> 172.21.15.71:0/3692564411 conn(0x7f05f0074e30 msgr2=0x7f05f0103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:56.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.992+0000 7f05f4c3a700 1 -- 172.21.15.71:0/3692564411 shutdown_connections 2024-04-23T02:25:56.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:56.992+0000 7f05f4c3a700 1 -- 172.21.15.71:0/3692564411 wait complete. 2024-04-23T02:25:57.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.093+0000 7f79eec1b700 1 -- 172.21.15.71:0/2710057571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e80ff5a0 msgr2=0x7f79e80ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:57.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.093+0000 7f79eec1b700 1 --2- 172.21.15.71:0/2710057571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e80ff5a0 0x7f79e80ff980 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009910 tx=0x7f79dc009c20 comp rx=0 tx=0).stop 2024-04-23T02:25:57.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.093+0000 7f79eec1b700 1 -- 172.21.15.71:0/2710057571 shutdown_connections 2024-04-23T02:25:57.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.093+0000 7f79eec1b700 1 --2- 172.21.15.71:0/2710057571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79e80fff50 0x7f79e8105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:57.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.093+0000 7f79eec1b700 1 --2- 172.21.15.71:0/2710057571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e80ff5a0 0x7f79e80ff980 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:57.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.093+0000 7f79eec1b700 1 -- 172.21.15.71:0/2710057571 >> 172.21.15.71:0/2710057571 conn(0x7f79e8074e30 msgr2=0x7f79e8075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:57.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.093+0000 7f79eec1b700 1 -- 172.21.15.71:0/2710057571 shutdown_connections 2024-04-23T02:25:57.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.093+0000 7f79eec1b700 1 -- 172.21.15.71:0/2710057571 wait complete. 2024-04-23T02:25:57.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.094+0000 7f79eec1b700 1 Processor -- start 2024-04-23T02:25:57.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.094+0000 7f79eec1b700 1 -- start start 2024-04-23T02:25:57.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.094+0000 7f79eec1b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79e80ff5a0 0x7f79e819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:57.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.094+0000 7f79eec1b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e80fff50 0x7f79e819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:57.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.094+0000 7f79eec1b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e819d260 con 0x7f79e80fff50 2024-04-23T02:25:57.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.094+0000 7f79eec1b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e8196710 con 0x7f79e80ff5a0 2024-04-23T02:25:57.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.094+0000 7f79e7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e80fff50 0x7f79e819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:57.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.094+0000 7f79e7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e80fff50 0x7f79e819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40434/0 (socket says 172.21.15.71:40434) 2024-04-23T02:25:57.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.094+0000 7f79e7fff700 1 -- 172.21.15.71:0/3272914599 learned_addr learned my addr 172.21.15.71:0/3272914599 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:25:57.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.095+0000 7f79e7fff700 1 -- 172.21.15.71:0/3272914599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79e80ff5a0 msgr2=0x7f79e819c690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-23T02:25:57.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.095+0000 7f79e7fff700 1 --2- 172.21.15.71:0/3272914599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79e80ff5a0 0x7f79e819c690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:57.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.095+0000 7f79e7fff700 1 -- 172.21.15.71:0/3272914599 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc0095f0 con 0x7f79e80fff50 2024-04-23T02:25:57.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.095+0000 7f79e7fff700 1 --2- 172.21.15.71:0/3272914599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e80fff50 0x7f79e819cbd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f79d400b4d0 tx=0x7f79d400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:57.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.095+0000 7f79e5ffb700 1 -- 172.21.15.71:0/3272914599 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79d4011800 con 0x7f79e80fff50 2024-04-23T02:25:57.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.095+0000 7f79e5ffb700 1 -- 172.21.15.71:0/3272914599 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f79d4011e00 con 0x7f79e80fff50 2024-04-23T02:25:57.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.095+0000 7f79eec1b700 1 -- 172.21.15.71:0/3272914599 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e81969f0 con 0x7f79e80fff50 2024-04-23T02:25:57.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.095+0000 7f79e5ffb700 1 -- 172.21.15.71:0/3272914599 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79d400f2e0 con 0x7f79e80fff50 2024-04-23T02:25:57.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.095+0000 7f79eec1b700 1 -- 172.21.15.71:0/3272914599 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e8196f40 con 0x7f79e80fff50 2024-04-23T02:25:57.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.096+0000 7f79eec1b700 1 -- 172.21.15.71:0/3272914599 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e8109270 con 0x7f79e80fff50 2024-04-23T02:25:57.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.096+0000 7f79e5ffb700 1 -- 172.21.15.71:0/3272914599 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f79d4010380 con 0x7f79e80fff50 2024-04-23T02:25:57.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.097+0000 7f79e5ffb700 1 --2- 172.21.15.71:0/3272914599 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f79d806cce0 0x7f79d806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:57.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.097+0000 7f79e5ffb700 1 -- 172.21.15.71:0/3272914599 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f79d408c090 con 0x7f79e80fff50 2024-04-23T02:25:57.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.097+0000 7f79ec9b7700 1 --2- 172.21.15.71:0/3272914599 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f79d806cce0 0x7f79d806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:57.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.097+0000 7f79ec9b7700 1 --2- 172.21.15.71:0/3272914599 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f79d806cce0 0x7f79d806f1a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009fd0 tx=0x7f79dc005900 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:57.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.101+0000 7f79e5ffb700 1 -- 172.21.15.71:0/3272914599 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f79d4059c10 con 0x7f79e80fff50 2024-04-23T02:25:57.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.305+0000 7f79eec1b700 1 -- 172.21.15.71:0/3272914599 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f79e8197d30 con 0x7f79d806cce0 2024-04-23T02:25:57.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.316+0000 7f79e5ffb700 1 -- 172.21.15.71:0/3272914599 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f79e8197d30 con 0x7f79d806cce0 2024-04-23T02:25:57.319 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:25:57.319 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (9m) 3m ago 20m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:25:57.319 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (21m) 3m ago 21m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:25:57.319 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (20m) 3m ago 20m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:25:57.319 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (7m) 3m ago 21m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:25:57.319 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (7m) 3m ago 19m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:25:57.319 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (9m) 3m ago 20m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:25:57.319 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (3m) 3m ago 15m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:25:57.319 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (3m) 3m ago 14m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:25:57.320 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (14m) 3m ago 14m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:25:57.320 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (3m) 3m ago 14m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:25:57.320 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (11m) 3m ago 22m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:25:57.320 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (10m) 3m ago 19m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:25:57.320 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (8m) 3m ago 22m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:25:57.320 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (7m) 3m ago 19m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:25:57.320 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (10m) 3m ago 21m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:25:57.320 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (10m) 3m ago 19m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:25:57.320 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (7m) 3m ago 19m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:25:57.320 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (6m) 3m ago 18m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:25:57.320 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (5m) 3m ago 17m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:25:57.320 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (5m) 3m ago 17m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:25:57.320 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (4m) 3m ago 17m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:25:57.320 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (4m) 3m ago 16m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:25:57.321 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (9m) 3m ago 20m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:25:57.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.318+0000 7f79eec1b700 1 -- 172.21.15.71:0/3272914599 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f79d806cce0 msgr2=0x7f79d806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:57.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.318+0000 7f79eec1b700 1 --2- 172.21.15.71:0/3272914599 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f79d806cce0 0x7f79d806f1a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009fd0 tx=0x7f79dc005900 comp rx=0 tx=0).stop 2024-04-23T02:25:57.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.318+0000 7f79eec1b700 1 -- 172.21.15.71:0/3272914599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e80fff50 msgr2=0x7f79e819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:57.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.318+0000 7f79eec1b700 1 --2- 172.21.15.71:0/3272914599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e80fff50 0x7f79e819cbd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f79d400b4d0 tx=0x7f79d400b890 comp rx=0 tx=0).stop 2024-04-23T02:25:57.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.318+0000 7f79eec1b700 1 -- 172.21.15.71:0/3272914599 shutdown_connections 2024-04-23T02:25:57.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.318+0000 7f79eec1b700 1 --2- 172.21.15.71:0/3272914599 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f79d806cce0 0x7f79d806f1a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:57.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.318+0000 7f79eec1b700 1 --2- 172.21.15.71:0/3272914599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79e80ff5a0 0x7f79e819c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:57.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.318+0000 7f79eec1b700 1 --2- 172.21.15.71:0/3272914599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79e80fff50 0x7f79e819cbd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:57.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.318+0000 7f79eec1b700 1 -- 172.21.15.71:0/3272914599 >> 172.21.15.71:0/3272914599 conn(0x7f79e8074e30 msgr2=0x7f79e80fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:57.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.318+0000 7f79eec1b700 1 -- 172.21.15.71:0/3272914599 shutdown_connections 2024-04-23T02:25:57.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.318+0000 7f79eec1b700 1 -- 172.21.15.71:0/3272914599 wait complete. 2024-04-23T02:25:57.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.421+0000 7f86058b4700 1 -- 172.21.15.71:0/4216597803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8600100750 msgr2=0x7f86001047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:57.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.421+0000 7f86058b4700 1 --2- 172.21.15.71:0/4216597803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8600100750 0x7f86001047a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009910 tx=0x7f85f4009c20 comp rx=0 tx=0).stop 2024-04-23T02:25:57.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.421+0000 7f86058b4700 1 -- 172.21.15.71:0/4216597803 shutdown_connections 2024-04-23T02:25:57.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.421+0000 7f86058b4700 1 --2- 172.21.15.71:0/4216597803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8600100750 0x7f86001047a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:57.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.421+0000 7f86058b4700 1 --2- 172.21.15.71:0/4216597803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86000ffda0 0x7f8600100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:57.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.421+0000 7f86058b4700 1 -- 172.21.15.71:0/4216597803 >> 172.21.15.71:0/4216597803 conn(0x7f86000fb650 msgr2=0x7f86000fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:57.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.421+0000 7f86058b4700 1 -- 172.21.15.71:0/4216597803 shutdown_connections 2024-04-23T02:25:57.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.421+0000 7f86058b4700 1 -- 172.21.15.71:0/4216597803 wait complete. 2024-04-23T02:25:57.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.422+0000 7f86058b4700 1 Processor -- start 2024-04-23T02:25:57.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.422+0000 7f86058b4700 1 -- start start 2024-04-23T02:25:57.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.423+0000 7f86058b4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86000ffda0 0x7f860019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:57.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.423+0000 7f86058b4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8600100750 0x7f860019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:57.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.423+0000 7f86058b4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860019d230 con 0x7f86000ffda0 2024-04-23T02:25:57.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.423+0000 7f86058b4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86001966e0 con 0x7f8600100750 2024-04-23T02:25:57.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.423+0000 7f85feffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86000ffda0 0x7f860019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:57.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.423+0000 7f85feffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86000ffda0 0x7f860019c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40448/0 (socket says 172.21.15.71:40448) 2024-04-23T02:25:57.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.423+0000 7f85feffd700 1 -- 172.21.15.71:0/506523280 learned_addr learned my addr 172.21.15.71:0/506523280 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:25:57.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.423+0000 7f85fe7fc700 1 --2- 172.21.15.71:0/506523280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8600100750 0x7f860019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:57.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.423+0000 7f85feffd700 1 -- 172.21.15.71:0/506523280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8600100750 msgr2=0x7f860019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:57.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.423+0000 7f85feffd700 1 --2- 172.21.15.71:0/506523280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8600100750 0x7f860019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:57.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.423+0000 7f85feffd700 1 -- 172.21.15.71:0/506523280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f40095f0 con 0x7f86000ffda0 2024-04-23T02:25:57.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.423+0000 7f85feffd700 1 --2- 172.21.15.71:0/506523280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86000ffda0 0x7f860019c660 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f85f000c8a0 tx=0x7f85f000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:57.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.423+0000 7f86048b2700 1 -- 172.21.15.71:0/506523280 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85f00078e0 con 0x7f86000ffda0 2024-04-23T02:25:57.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.423+0000 7f86048b2700 1 -- 172.21.15.71:0/506523280 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f85f000f450 con 0x7f86000ffda0 2024-04-23T02:25:57.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.423+0000 7f86058b4700 1 -- 172.21.15.71:0/506523280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86001969c0 con 0x7f86000ffda0 2024-04-23T02:25:57.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.423+0000 7f86048b2700 1 -- 172.21.15.71:0/506523280 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85f000e5e0 con 0x7f86000ffda0 2024-04-23T02:25:57.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.423+0000 7f86058b4700 1 -- 172.21.15.71:0/506523280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8600196f10 con 0x7f86000ffda0 2024-04-23T02:25:57.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.424+0000 7f86058b4700 1 -- 172.21.15.71:0/506523280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86001081a0 con 0x7f86000ffda0 2024-04-23T02:25:57.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.425+0000 7f86048b2700 1 -- 172.21.15.71:0/506523280 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f85f000f5c0 con 0x7f86000ffda0 2024-04-23T02:25:57.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.425+0000 7f86048b2700 1 --2- 172.21.15.71:0/506523280 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f85e806cce0 0x7f85e806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:57.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.425+0000 7f86048b2700 1 -- 172.21.15.71:0/506523280 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f85f008c480 con 0x7f86000ffda0 2024-04-23T02:25:57.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.425+0000 7f85fe7fc700 1 --2- 172.21.15.71:0/506523280 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f85e806cce0 0x7f85e806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:57.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.426+0000 7f85fe7fc700 1 --2- 172.21.15.71:0/506523280 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f85e806cce0 0x7f85e806f1a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8600197b50 tx=0x7f85f401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:57.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.430+0000 7f86048b2700 1 -- 172.21.15.71:0/506523280 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f85f005a000 con 0x7f86000ffda0 2024-04-23T02:25:57.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.702+0000 7f86058b4700 1 -- 172.21.15.71:0/506523280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f860004f2e0 con 0x7f86000ffda0 2024-04-23T02:25:57.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.703+0000 7f86048b2700 1 -- 172.21.15.71:0/506523280 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f85f0059b90 con 0x7f86000ffda0 2024-04-23T02:25:57.706 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:25:57.706 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:25:57.706 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:25:57.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:25:57.706 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:25:57.706 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:25:57.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:25:57.706 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:25:57.706 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:25:57.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:25:57.707 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:25:57.707 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:25:57.707 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:25:57.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:25:57.707 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:25:57.707 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:25:57.707 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:25:57.707 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:25:57.707 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:25:57.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.705+0000 7f86058b4700 1 -- 172.21.15.71:0/506523280 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f85e806cce0 msgr2=0x7f85e806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:57.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.705+0000 7f86058b4700 1 --2- 172.21.15.71:0/506523280 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f85e806cce0 0x7f85e806f1a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8600197b50 tx=0x7f85f401a040 comp rx=0 tx=0).stop 2024-04-23T02:25:57.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.705+0000 7f86058b4700 1 -- 172.21.15.71:0/506523280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86000ffda0 msgr2=0x7f860019c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:57.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.705+0000 7f86058b4700 1 --2- 172.21.15.71:0/506523280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86000ffda0 0x7f860019c660 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f85f000c8a0 tx=0x7f85f000cbb0 comp rx=0 tx=0).stop 2024-04-23T02:25:57.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.705+0000 7f86058b4700 1 -- 172.21.15.71:0/506523280 shutdown_connections 2024-04-23T02:25:57.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.705+0000 7f86058b4700 1 --2- 172.21.15.71:0/506523280 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f85e806cce0 0x7f85e806f1a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:57.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.705+0000 7f86058b4700 1 --2- 172.21.15.71:0/506523280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86000ffda0 0x7f860019c660 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:57.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.705+0000 7f86058b4700 1 --2- 172.21.15.71:0/506523280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8600100750 0x7f860019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:57.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.705+0000 7f86058b4700 1 -- 172.21.15.71:0/506523280 >> 172.21.15.71:0/506523280 conn(0x7f86000fb650 msgr2=0x7f86000fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:57.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.705+0000 7f86058b4700 1 -- 172.21.15.71:0/506523280 shutdown_connections 2024-04-23T02:25:57.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.705+0000 7f86058b4700 1 -- 172.21.15.71:0/506523280 wait complete. 2024-04-23T02:25:57.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.808+0000 7fbddd3a8700 1 -- 172.21.15.71:0/2506281395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd81006b0 msgr2=0x7fbdd8100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:57.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.808+0000 7fbddd3a8700 1 --2- 172.21.15.71:0/2506281395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd81006b0 0x7fbdd8100a90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009910 tx=0x7fbdc8009c20 comp rx=0 tx=0).stop 2024-04-23T02:25:57.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.808+0000 7fbddd3a8700 1 -- 172.21.15.71:0/2506281395 shutdown_connections 2024-04-23T02:25:57.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.808+0000 7fbddd3a8700 1 --2- 172.21.15.71:0/2506281395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdd8101060 0x7fbdd81047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:57.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.808+0000 7fbddd3a8700 1 --2- 172.21.15.71:0/2506281395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd81006b0 0x7fbdd8100a90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:57.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.808+0000 7fbddd3a8700 1 -- 172.21.15.71:0/2506281395 >> 172.21.15.71:0/2506281395 conn(0x7fbdd8074e30 msgr2=0x7fbdd8075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:57.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.808+0000 7fbddd3a8700 1 -- 172.21.15.71:0/2506281395 shutdown_connections 2024-04-23T02:25:57.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.808+0000 7fbddd3a8700 1 -- 172.21.15.71:0/2506281395 wait complete. 2024-04-23T02:25:57.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.809+0000 7fbddd3a8700 1 Processor -- start 2024-04-23T02:25:57.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.809+0000 7fbddd3a8700 1 -- start start 2024-04-23T02:25:57.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.809+0000 7fbddd3a8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd81006b0 0x7fbdd819b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:57.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.809+0000 7fbddd3a8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdd8101060 0x7fbdd819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:57.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.809+0000 7fbddd3a8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd819c170 con 0x7fbdd81006b0 2024-04-23T02:25:57.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.809+0000 7fbddd3a8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd8195620 con 0x7fbdd8101060 2024-04-23T02:25:57.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.809+0000 7fbdd6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd81006b0 0x7fbdd819b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:57.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.809+0000 7fbdd6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd81006b0 0x7fbdd819b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40480/0 (socket says 172.21.15.71:40480) 2024-04-23T02:25:57.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.809+0000 7fbdd6ffd700 1 -- 172.21.15.71:0/4136339542 learned_addr learned my addr 172.21.15.71:0/4136339542 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:25:57.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.810+0000 7fbdd6ffd700 1 -- 172.21.15.71:0/4136339542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdd8101060 msgr2=0x7fbdd819bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:57.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.810+0000 7fbdd6ffd700 1 --2- 172.21.15.71:0/4136339542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdd8101060 0x7fbdd819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:57.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.810+0000 7fbdd6ffd700 1 -- 172.21.15.71:0/4136339542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc80095f0 con 0x7fbdd81006b0 2024-04-23T02:25:57.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.810+0000 7fbdd6ffd700 1 --2- 172.21.15.71:0/4136339542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd81006b0 0x7fbdd819b5a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8006010 tx=0x7fbdc800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:57.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.810+0000 7fbdbffff700 1 -- 172.21.15.71:0/4136339542 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdc801d070 con 0x7fbdd81006b0 2024-04-23T02:25:57.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.810+0000 7fbdbffff700 1 -- 172.21.15.71:0/4136339542 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbdc800f460 con 0x7fbdd81006b0 2024-04-23T02:25:57.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.810+0000 7fbdbffff700 1 -- 172.21.15.71:0/4136339542 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdc8021640 con 0x7fbdd81006b0 2024-04-23T02:25:57.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.810+0000 7fbddd3a8700 1 -- 172.21.15.71:0/4136339542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd81958a0 con 0x7fbdd81006b0 2024-04-23T02:25:57.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.810+0000 7fbddd3a8700 1 -- 172.21.15.71:0/4136339542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd8195d90 con 0x7fbdd81006b0 2024-04-23T02:25:57.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.811+0000 7fbddd3a8700 1 -- 172.21.15.71:0/4136339542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd8108120 con 0x7fbdd81006b0 2024-04-23T02:25:57.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.811+0000 7fbdbffff700 1 -- 172.21.15.71:0/4136339542 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbdc800fa60 con 0x7fbdd81006b0 2024-04-23T02:25:57.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.812+0000 7fbdbffff700 1 --2- 172.21.15.71:0/4136339542 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbdc006cce0 0x7fbdc006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:57.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.812+0000 7fbdd67fc700 1 --2- 172.21.15.71:0/4136339542 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbdc006cce0 0x7fbdc006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:57.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.812+0000 7fbdbffff700 1 -- 172.21.15.71:0/4136339542 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbdc808d7e0 con 0x7fbdd81006b0 2024-04-23T02:25:57.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.812+0000 7fbdd67fc700 1 --2- 172.21.15.71:0/4136339542 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbdc006cce0 0x7fbdc006f1a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8106750 tx=0x7fbdcc006d20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:57.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:57.816+0000 7fbdbffff700 1 -- 172.21.15.71:0/4136339542 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbdc805b360 con 0x7fbdd81006b0 2024-04-23T02:25:58.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.053+0000 7fbddd3a8700 1 -- 172.21.15.71:0/4136339542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbdd8107b00 con 0x7fbdd81006b0 2024-04-23T02:25:58.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.054+0000 7fbdbffff700 1 -- 172.21.15.71:0/4136339542 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7fbdc805aef0 con 0x7fbdd81006b0 2024-04-23T02:25:58.057 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:25:58.057 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:25:58.057 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:25:58.057 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:25:58.057 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:25:58.057 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:25:58.057 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:25:58.057 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:25:58.057 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:25:58.057 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:25:58.057 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:25:58.057 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:25:58.058 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:25:58.058 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:25:58.058 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:25:58.058 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:25:58.058 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:25:58.058 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:25:58.058 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:25:58.058 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:25:58.058 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:25:58.058 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:25:58.058 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:25:58.058 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:25:58.058 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:25:58.058 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:25:58.059 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:25:58.059 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:25:58.059 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:25:58.059 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:25:58.059 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:25:58.059 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:25:58.059 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:25:58.059 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:25:58.059 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:25:58.059 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:25:58.059 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:25:58.059 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:25:58.059 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:25:58.059 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:25:58.060 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:25:58.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.056+0000 7fbddd3a8700 1 -- 172.21.15.71:0/4136339542 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbdc006cce0 msgr2=0x7fbdc006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:58.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.056+0000 7fbddd3a8700 1 --2- 172.21.15.71:0/4136339542 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbdc006cce0 0x7fbdc006f1a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8106750 tx=0x7fbdcc006d20 comp rx=0 tx=0).stop 2024-04-23T02:25:58.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.056+0000 7fbddd3a8700 1 -- 172.21.15.71:0/4136339542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd81006b0 msgr2=0x7fbdd819b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:58.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.056+0000 7fbddd3a8700 1 --2- 172.21.15.71:0/4136339542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd81006b0 0x7fbdd819b5a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8006010 tx=0x7fbdc800be20 comp rx=0 tx=0).stop 2024-04-23T02:25:58.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.056+0000 7fbddd3a8700 1 -- 172.21.15.71:0/4136339542 shutdown_connections 2024-04-23T02:25:58.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.056+0000 7fbddd3a8700 1 --2- 172.21.15.71:0/4136339542 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbdc006cce0 0x7fbdc006f1a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:58.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.056+0000 7fbddd3a8700 1 --2- 172.21.15.71:0/4136339542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd81006b0 0x7fbdd819b5a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:58.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.056+0000 7fbddd3a8700 1 --2- 172.21.15.71:0/4136339542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdd8101060 0x7fbdd819bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:58.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.056+0000 7fbddd3a8700 1 -- 172.21.15.71:0/4136339542 >> 172.21.15.71:0/4136339542 conn(0x7fbdd8074e30 msgr2=0x7fbdd80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:58.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.056+0000 7fbddd3a8700 1 -- 172.21.15.71:0/4136339542 shutdown_connections 2024-04-23T02:25:58.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.056+0000 7fbddd3a8700 1 -- 172.21.15.71:0/4136339542 wait complete. 2024-04-23T02:25:58.061 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:25:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.159+0000 7f552c628700 1 -- 172.21.15.71:0/31354369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5524105bf0 msgr2=0x7f5524105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.159+0000 7f552c628700 1 --2- 172.21.15.71:0/31354369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5524105bf0 0x7f5524105fd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5514009910 tx=0x7f5514009c20 comp rx=0 tx=0).stop 2024-04-23T02:25:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.159+0000 7f552c628700 1 -- 172.21.15.71:0/31354369 shutdown_connections 2024-04-23T02:25:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.159+0000 7f552c628700 1 --2- 172.21.15.71:0/31354369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55240ffb40 0x7f55240fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.159+0000 7f552c628700 1 --2- 172.21.15.71:0/31354369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5524105bf0 0x7f5524105fd0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.159+0000 7f552c628700 1 -- 172.21.15.71:0/31354369 >> 172.21.15.71:0/31354369 conn(0x7f55240fb610 msgr2=0x7f55240fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.159+0000 7f552c628700 1 -- 172.21.15.71:0/31354369 shutdown_connections 2024-04-23T02:25:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.159+0000 7f552c628700 1 -- 172.21.15.71:0/31354369 wait complete. 2024-04-23T02:25:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.160+0000 7f552c628700 1 Processor -- start 2024-04-23T02:25:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.160+0000 7f552c628700 1 -- start start 2024-04-23T02:25:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.160+0000 7f552c628700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55240ffb40 0x7f55241017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.160+0000 7f552c628700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5524105bf0 0x7f5524101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.160+0000 7f552c628700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5524102230 con 0x7f5524105bf0 2024-04-23T02:25:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.160+0000 7f552c628700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5524102370 con 0x7f55240ffb40 2024-04-23T02:25:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.160+0000 7f5529bc3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5524105bf0 0x7f5524101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.160+0000 7f5529bc3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5524105bf0 0x7f5524101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40500/0 (socket says 172.21.15.71:40500) 2024-04-23T02:25:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.160+0000 7f5529bc3700 1 -- 172.21.15.71:0/4239099840 learned_addr learned my addr 172.21.15.71:0/4239099840 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:25:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.160+0000 7f552a3c4700 1 --2- 172.21.15.71:0/4239099840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55240ffb40 0x7f55241017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.161+0000 7f5529bc3700 1 -- 172.21.15.71:0/4239099840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55240ffb40 msgr2=0x7f55241017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.161+0000 7f5529bc3700 1 --2- 172.21.15.71:0/4239099840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55240ffb40 0x7f55241017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.161+0000 7f5529bc3700 1 -- 172.21.15.71:0/4239099840 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55140095f0 con 0x7f5524105bf0 2024-04-23T02:25:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.161+0000 7f5529bc3700 1 --2- 172.21.15.71:0/4239099840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5524105bf0 0x7f5524101cf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f552000d900 tx=0x7f552000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.161+0000 7f551b7fe700 1 -- 172.21.15.71:0/4239099840 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5520004a00 con 0x7f5524105bf0 2024-04-23T02:25:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.161+0000 7f551b7fe700 1 -- 172.21.15.71:0/4239099840 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5520005020 con 0x7f5524105bf0 2024-04-23T02:25:58.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.161+0000 7f552c628700 1 -- 172.21.15.71:0/4239099840 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55241a2950 con 0x7f5524105bf0 2024-04-23T02:25:58.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.161+0000 7f551b7fe700 1 -- 172.21.15.71:0/4239099840 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5520009b90 con 0x7f5524105bf0 2024-04-23T02:25:58.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.161+0000 7f552c628700 1 -- 172.21.15.71:0/4239099840 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55241a2d60 con 0x7f5524105bf0 2024-04-23T02:25:58.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.162+0000 7f552c628700 1 -- 172.21.15.71:0/4239099840 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f552404f2e0 con 0x7f5524105bf0 2024-04-23T02:25:58.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.162+0000 7f551b7fe700 1 -- 172.21.15.71:0/4239099840 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5520005190 con 0x7f5524105bf0 2024-04-23T02:25:58.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.163+0000 7f551b7fe700 1 --2- 172.21.15.71:0/4239099840 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f551006cce0 0x7f551006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:58.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.163+0000 7f551b7fe700 1 -- 172.21.15.71:0/4239099840 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f552008c4f0 con 0x7f5524105bf0 2024-04-23T02:25:58.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.163+0000 7f552a3c4700 1 --2- 172.21.15.71:0/4239099840 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f551006cce0 0x7f551006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:58.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.163+0000 7f552a3c4700 1 --2- 172.21.15.71:0/4239099840 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f551006cce0 0x7f551006f1a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5514006010 tx=0x7f551401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:58.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.167+0000 7f551b7fe700 1 -- 172.21.15.71:0/4239099840 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5520056560 con 0x7f5524105bf0 2024-04-23T02:25:58.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.381+0000 7f552c628700 1 -- 172.21.15.71:0/4239099840 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5524069600 con 0x7f551006cce0 2024-04-23T02:25:58.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.383+0000 7f551b7fe700 1 -- 172.21.15.71:0/4239099840 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5524069600 con 0x7f551006cce0 2024-04-23T02:25:58.386 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:25:58.386 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:25:58.386 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:25:58.386 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:25:58.387 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:25:58.387 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:25:58.387 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:25:58.387 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:25:58.387 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:25:58.387 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:25:58.388 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:25:58.388 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:25:58.388 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:25:58.388 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:25:58.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.385+0000 7f552c628700 1 -- 172.21.15.71:0/4239099840 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f551006cce0 msgr2=0x7f551006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:58.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.385+0000 7f552c628700 1 --2- 172.21.15.71:0/4239099840 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f551006cce0 0x7f551006f1a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5514006010 tx=0x7f551401a040 comp rx=0 tx=0).stop 2024-04-23T02:25:58.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.385+0000 7f552c628700 1 -- 172.21.15.71:0/4239099840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5524105bf0 msgr2=0x7f5524101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:58.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.385+0000 7f552c628700 1 --2- 172.21.15.71:0/4239099840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5524105bf0 0x7f5524101cf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f552000d900 tx=0x7f552000dc10 comp rx=0 tx=0).stop 2024-04-23T02:25:58.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.385+0000 7f552c628700 1 -- 172.21.15.71:0/4239099840 shutdown_connections 2024-04-23T02:25:58.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.385+0000 7f552c628700 1 --2- 172.21.15.71:0/4239099840 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f551006cce0 0x7f551006f1a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:58.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.385+0000 7f552c628700 1 --2- 172.21.15.71:0/4239099840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55240ffb40 0x7f55241017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:58.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.385+0000 7f552c628700 1 --2- 172.21.15.71:0/4239099840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5524105bf0 0x7f5524101cf0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:58.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.385+0000 7f552c628700 1 -- 172.21.15.71:0/4239099840 >> 172.21.15.71:0/4239099840 conn(0x7f55240fb610 msgr2=0x7f5524068ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:58.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.385+0000 7f552c628700 1 -- 172.21.15.71:0/4239099840 shutdown_connections 2024-04-23T02:25:58.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.385+0000 7f552c628700 1 -- 172.21.15.71:0/4239099840 wait complete. 2024-04-23T02:25:58.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.488+0000 7fe321ad6700 1 -- 172.21.15.71:0/4275284988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe31c100ea0 msgr2=0x7fe31c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:58.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.488+0000 7fe321ad6700 1 --2- 172.21.15.71:0/4275284988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe31c100ea0 0x7fe31c101280 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe31400b0a0 tx=0x7fe31400b3b0 comp rx=0 tx=0).stop 2024-04-23T02:25:58.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.488+0000 7fe321ad6700 1 -- 172.21.15.71:0/4275284988 shutdown_connections 2024-04-23T02:25:58.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.488+0000 7fe321ad6700 1 --2- 172.21.15.71:0/4275284988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe31c101850 0x7fe31c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:58.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.488+0000 7fe321ad6700 1 --2- 172.21.15.71:0/4275284988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe31c100ea0 0x7fe31c101280 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:58.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.488+0000 7fe321ad6700 1 -- 172.21.15.71:0/4275284988 >> 172.21.15.71:0/4275284988 conn(0x7fe31c0fc750 msgr2=0x7fe31c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:58.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.488+0000 7fe321ad6700 1 -- 172.21.15.71:0/4275284988 shutdown_connections 2024-04-23T02:25:58.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.488+0000 7fe321ad6700 1 -- 172.21.15.71:0/4275284988 wait complete. 2024-04-23T02:25:58.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.489+0000 7fe321ad6700 1 Processor -- start 2024-04-23T02:25:58.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.489+0000 7fe321ad6700 1 -- start start 2024-04-23T02:25:58.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.489+0000 7fe321ad6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe31c100ea0 0x7fe31c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:58.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.489+0000 7fe321ad6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe31c101850 0x7fe31c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:58.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.489+0000 7fe321ad6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c19d130 con 0x7fe31c101850 2024-04-23T02:25:58.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.489+0000 7fe321ad6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c1965e0 con 0x7fe31c100ea0 2024-04-23T02:25:58.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.489+0000 7fe31affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe31c101850 0x7fe31c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:58.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.489+0000 7fe31affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe31c101850 0x7fe31c19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40528/0 (socket says 172.21.15.71:40528) 2024-04-23T02:25:58.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.489+0000 7fe31affd700 1 -- 172.21.15.71:0/1306050865 learned_addr learned my addr 172.21.15.71:0/1306050865 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:25:58.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.489+0000 7fe31b7fe700 1 --2- 172.21.15.71:0/1306050865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe31c100ea0 0x7fe31c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:58.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.490+0000 7fe31affd700 1 -- 172.21.15.71:0/1306050865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe31c100ea0 msgr2=0x7fe31c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:58.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.490+0000 7fe31affd700 1 --2- 172.21.15.71:0/1306050865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe31c100ea0 0x7fe31c19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:58.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.490+0000 7fe31affd700 1 -- 172.21.15.71:0/1306050865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe314009d00 con 0x7fe31c101850 2024-04-23T02:25:58.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.490+0000 7fe31b7fe700 1 --2- 172.21.15.71:0/1306050865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe31c100ea0 0x7fe31c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:25:58.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.490+0000 7fe31affd700 1 --2- 172.21.15.71:0/1306050865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe31c101850 0x7fe31c19caa0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe30400c8f0 tx=0x7fe30400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:58.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.490+0000 7fe318ff9700 1 -- 172.21.15.71:0/1306050865 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe304007a50 con 0x7fe31c101850 2024-04-23T02:25:58.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.490+0000 7fe321ad6700 1 -- 172.21.15.71:0/1306050865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31c1968c0 con 0x7fe31c101850 2024-04-23T02:25:58.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.490+0000 7fe318ff9700 1 -- 172.21.15.71:0/1306050865 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe304007bb0 con 0x7fe31c101850 2024-04-23T02:25:58.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.490+0000 7fe318ff9700 1 -- 172.21.15.71:0/1306050865 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe304018630 con 0x7fe31c101850 2024-04-23T02:25:58.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.490+0000 7fe321ad6700 1 -- 172.21.15.71:0/1306050865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c196e10 con 0x7fe31c101850 2024-04-23T02:25:58.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.491+0000 7fe321ad6700 1 -- 172.21.15.71:0/1306050865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31c109240 con 0x7fe31c101850 2024-04-23T02:25:58.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.491+0000 7fe318ff9700 1 -- 172.21.15.71:0/1306050865 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe304007d20 con 0x7fe31c101850 2024-04-23T02:25:58.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.492+0000 7fe318ff9700 1 --2- 172.21.15.71:0/1306050865 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe30806cce0 0x7fe30806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:25:58.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.492+0000 7fe318ff9700 1 -- 172.21.15.71:0/1306050865 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe30408c590 con 0x7fe31c101850 2024-04-23T02:25:58.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.492+0000 7fe31b7fe700 1 --2- 172.21.15.71:0/1306050865 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe30806cce0 0x7fe30806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:25:58.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.492+0000 7fe31b7fe700 1 --2- 172.21.15.71:0/1306050865 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe30806cce0 0x7fe30806f1a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe314009480 tx=0x7fe314006000 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:25:58.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.496+0000 7fe318ff9700 1 -- 172.21.15.71:0/1306050865 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe30405a090 con 0x7fe31c101850 2024-04-23T02:25:58.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:58 smithi071 ceph-mon[144544]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:25:58.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:58 smithi071 ceph-mon[144544]: pgmap v379: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:58.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:58 smithi071 ceph-mon[144544]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:25:58.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:58 smithi071 ceph-mon[144544]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:25:58.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:58 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/506523280' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:25:58.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:58 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/4136339542' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:25:58.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.767+0000 7fe321ad6700 1 -- 172.21.15.71:0/1306050865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe31c04f2e0 con 0x7fe31c101850 2024-04-23T02:25:58.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.767+0000 7fe318ff9700 1 -- 172.21.15.71:0/1306050865 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fe304059c20 con 0x7fe31c101850 2024-04-23T02:25:58.770 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:25:58.770 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:25:58.770 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:25:58.770 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:25:58.770 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:25:58.770 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:25:58.770 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:25:58.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.769+0000 7fe321ad6700 1 -- 172.21.15.71:0/1306050865 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe30806cce0 msgr2=0x7fe30806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:58.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.769+0000 7fe321ad6700 1 --2- 172.21.15.71:0/1306050865 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe30806cce0 0x7fe30806f1a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe314009480 tx=0x7fe314006000 comp rx=0 tx=0).stop 2024-04-23T02:25:58.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.769+0000 7fe321ad6700 1 -- 172.21.15.71:0/1306050865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe31c101850 msgr2=0x7fe31c19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:25:58.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.769+0000 7fe321ad6700 1 --2- 172.21.15.71:0/1306050865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe31c101850 0x7fe31c19caa0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe30400c8f0 tx=0x7fe30400ccb0 comp rx=0 tx=0).stop 2024-04-23T02:25:58.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.769+0000 7fe321ad6700 1 -- 172.21.15.71:0/1306050865 shutdown_connections 2024-04-23T02:25:58.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.769+0000 7fe321ad6700 1 --2- 172.21.15.71:0/1306050865 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe30806cce0 0x7fe30806f1a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:58.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.769+0000 7fe321ad6700 1 --2- 172.21.15.71:0/1306050865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe31c100ea0 0x7fe31c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:58.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.769+0000 7fe321ad6700 1 --2- 172.21.15.71:0/1306050865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe31c101850 0x7fe31c19caa0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:25:58.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.769+0000 7fe321ad6700 1 -- 172.21.15.71:0/1306050865 >> 172.21.15.71:0/1306050865 conn(0x7fe31c0fc750 msgr2=0x7fe31c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:25:58.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.769+0000 7fe321ad6700 1 -- 172.21.15.71:0/1306050865 shutdown_connections 2024-04-23T02:25:58.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:25:58.769+0000 7fe321ad6700 1 -- 172.21.15.71:0/1306050865 wait complete. 2024-04-23T02:25:58.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:58 smithi192 ceph-mon[108617]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:25:58.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:58 smithi192 ceph-mon[108617]: pgmap v379: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:25:58.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:58 smithi192 ceph-mon[108617]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:25:58.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:58 smithi192 ceph-mon[108617]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:25:58.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:58 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/506523280' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:25:58.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:58 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/4136339542' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:25:59.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:59 smithi071 ceph-mon[144544]: from='client.34558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:25:59.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:25:59 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1306050865' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:25:59.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:59 smithi192 ceph-mon[108617]: from='client.34558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:25:59.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:25:59 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1306050865' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:26:00.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:00 smithi071 ceph-mon[144544]: pgmap v380: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:00 smithi192 ceph-mon[108617]: pgmap v380: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:01.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:01 smithi071 ceph-mon[144544]: pgmap v381: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:01.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:01 smithi192 ceph-mon[108617]: pgmap v381: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:04.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:03 smithi192 ceph-mon[108617]: pgmap v382: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:04.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:03 smithi071 ceph-mon[144544]: pgmap v382: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:05.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:04 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:26:05.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:04 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:26:06.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:05 smithi071 ceph-mon[144544]: pgmap v383: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:06.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:05 smithi192 ceph-mon[108617]: pgmap v383: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:08.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:07 smithi071 ceph-mon[144544]: pgmap v384: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:08.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:07 smithi192 ceph-mon[108617]: pgmap v384: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:10.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:09 smithi071 ceph-mon[144544]: pgmap v385: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:10.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:09 smithi192 ceph-mon[108617]: pgmap v385: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:12.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:11 smithi071 ceph-mon[144544]: pgmap v386: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:12.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:11 smithi192 ceph-mon[108617]: pgmap v386: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:14.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:13 smithi071 ceph-mon[144544]: pgmap v387: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:14.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:13 smithi192 ceph-mon[108617]: pgmap v387: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:16.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:15 smithi071 ceph-mon[144544]: pgmap v388: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:16.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:15 smithi192 ceph-mon[108617]: pgmap v388: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:18.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:17 smithi071 ceph-mon[144544]: pgmap v389: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:18.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:17 smithi192 ceph-mon[108617]: pgmap v389: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:19.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:19 smithi071 ceph-mon[144544]: pgmap v390: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:19.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:26:20.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:19 smithi192 ceph-mon[108617]: pgmap v390: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:20.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:26:22.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:21 smithi071 ceph-mon[144544]: pgmap v391: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:22.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:21 smithi192 ceph-mon[108617]: pgmap v391: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:24.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:23 smithi071 ceph-mon[144544]: pgmap v392: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:24.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:23 smithi192 ceph-mon[108617]: pgmap v392: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:26.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:25 smithi071 ceph-mon[144544]: pgmap v393: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:26.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:25 smithi192 ceph-mon[108617]: pgmap v393: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:28.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:27 smithi071 ceph-mon[144544]: pgmap v394: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:28.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:27 smithi192 ceph-mon[108617]: pgmap v394: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:28.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.881+0000 7fbd457ed700 1 -- 172.21.15.71:0/3857026493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd400fff50 msgr2=0x7fbd40105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:28.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.881+0000 7fbd457ed700 1 --2- 172.21.15.71:0/3857026493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd400fff50 0x7fbd40105840 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009910 tx=0x7fbd30009c20 comp rx=0 tx=0).stop 2024-04-23T02:26:28.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.881+0000 7fbd457ed700 1 -- 172.21.15.71:0/3857026493 shutdown_connections 2024-04-23T02:26:28.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.881+0000 7fbd457ed700 1 --2- 172.21.15.71:0/3857026493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd400fff50 0x7fbd40105840 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:28.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.881+0000 7fbd457ed700 1 --2- 172.21.15.71:0/3857026493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd400ff5a0 0x7fbd400ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:28.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.881+0000 7fbd457ed700 1 -- 172.21.15.71:0/3857026493 >> 172.21.15.71:0/3857026493 conn(0x7fbd40074e30 msgr2=0x7fbd40075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:26:28.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.881+0000 7fbd457ed700 1 -- 172.21.15.71:0/3857026493 shutdown_connections 2024-04-23T02:26:28.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.881+0000 7fbd457ed700 1 -- 172.21.15.71:0/3857026493 wait complete. 2024-04-23T02:26:28.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.882+0000 7fbd457ed700 1 Processor -- start 2024-04-23T02:26:28.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.882+0000 7fbd457ed700 1 -- start start 2024-04-23T02:26:28.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.882+0000 7fbd457ed700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd400ff5a0 0x7fbd4019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:28.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.882+0000 7fbd457ed700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd400fff50 0x7fbd4019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:28.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.882+0000 7fbd457ed700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4019d160 con 0x7fbd400fff50 2024-04-23T02:26:28.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.882+0000 7fbd457ed700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd40196610 con 0x7fbd400ff5a0 2024-04-23T02:26:28.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.882+0000 7fbd3e7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd400fff50 0x7fbd4019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:28.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.882+0000 7fbd3e7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd400fff50 0x7fbd4019cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38288/0 (socket says 172.21.15.71:38288) 2024-04-23T02:26:28.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.882+0000 7fbd3e7fc700 1 -- 172.21.15.71:0/4156718278 learned_addr learned my addr 172.21.15.71:0/4156718278 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:26:28.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.883+0000 7fbd3effd700 1 --2- 172.21.15.71:0/4156718278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd400ff5a0 0x7fbd4019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:28.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.883+0000 7fbd3e7fc700 1 -- 172.21.15.71:0/4156718278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd400ff5a0 msgr2=0x7fbd4019c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:28.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.883+0000 7fbd3e7fc700 1 --2- 172.21.15.71:0/4156718278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd400ff5a0 0x7fbd4019c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:28.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.883+0000 7fbd3e7fc700 1 -- 172.21.15.71:0/4156718278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd300095f0 con 0x7fbd400fff50 2024-04-23T02:26:28.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.883+0000 7fbd3e7fc700 1 --2- 172.21.15.71:0/4156718278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd400fff50 0x7fbd4019cad0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbd30000c00 tx=0x7fbd3000bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:26:28.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.883+0000 7fbd37fff700 1 -- 172.21.15.71:0/4156718278 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd3001d070 con 0x7fbd400fff50 2024-04-23T02:26:28.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.883+0000 7fbd37fff700 1 -- 172.21.15.71:0/4156718278 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbd30003cf0 con 0x7fbd400fff50 2024-04-23T02:26:28.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.883+0000 7fbd457ed700 1 -- 172.21.15.71:0/4156718278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd40196890 con 0x7fbd400fff50 2024-04-23T02:26:28.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.883+0000 7fbd37fff700 1 -- 172.21.15.71:0/4156718278 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd30021410 con 0x7fbd400fff50 2024-04-23T02:26:28.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.883+0000 7fbd457ed700 1 -- 172.21.15.71:0/4156718278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd40196d20 con 0x7fbd400fff50 2024-04-23T02:26:28.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.884+0000 7fbd457ed700 1 -- 172.21.15.71:0/4156718278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd40109200 con 0x7fbd400fff50 2024-04-23T02:26:28.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.884+0000 7fbd37fff700 1 -- 172.21.15.71:0/4156718278 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbd30003850 con 0x7fbd400fff50 2024-04-23T02:26:28.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.885+0000 7fbd37fff700 1 --2- 172.21.15.71:0/4156718278 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbd2c06cce0 0x7fbd2c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:28.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.885+0000 7fbd37fff700 1 -- 172.21.15.71:0/4156718278 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbd3008d740 con 0x7fbd400fff50 2024-04-23T02:26:28.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.885+0000 7fbd3effd700 1 --2- 172.21.15.71:0/4156718278 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbd2c06cce0 0x7fbd2c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:28.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.885+0000 7fbd3effd700 1 --2- 172.21.15.71:0/4156718278 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbd2c06cce0 0x7fbd2c06f1a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009bf0 tx=0x7fbd28009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:26:28.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:28.890+0000 7fbd37fff700 1 -- 172.21.15.71:0/4156718278 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbd3005b2c0 con 0x7fbd400fff50 2024-04-23T02:26:29.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.098+0000 7fbd457ed700 1 -- 172.21.15.71:0/4156718278 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd40196ff0 con 0x7fbd2c06cce0 2024-04-23T02:26:29.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.099+0000 7fbd37fff700 1 -- 172.21.15.71:0/4156718278 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbd40196ff0 con 0x7fbd2c06cce0 2024-04-23T02:26:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.101+0000 7fbd457ed700 1 -- 172.21.15.71:0/4156718278 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbd2c06cce0 msgr2=0x7fbd2c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.101+0000 7fbd457ed700 1 --2- 172.21.15.71:0/4156718278 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbd2c06cce0 0x7fbd2c06f1a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009bf0 tx=0x7fbd28009260 comp rx=0 tx=0).stop 2024-04-23T02:26:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.101+0000 7fbd457ed700 1 -- 172.21.15.71:0/4156718278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd400fff50 msgr2=0x7fbd4019cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.101+0000 7fbd457ed700 1 --2- 172.21.15.71:0/4156718278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd400fff50 0x7fbd4019cad0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbd30000c00 tx=0x7fbd3000bb20 comp rx=0 tx=0).stop 2024-04-23T02:26:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.101+0000 7fbd457ed700 1 -- 172.21.15.71:0/4156718278 shutdown_connections 2024-04-23T02:26:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.101+0000 7fbd457ed700 1 --2- 172.21.15.71:0/4156718278 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbd2c06cce0 0x7fbd2c06f1a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.101+0000 7fbd457ed700 1 --2- 172.21.15.71:0/4156718278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd400ff5a0 0x7fbd4019c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.101+0000 7fbd457ed700 1 --2- 172.21.15.71:0/4156718278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd400fff50 0x7fbd4019cad0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.101+0000 7fbd457ed700 1 -- 172.21.15.71:0/4156718278 >> 172.21.15.71:0/4156718278 conn(0x7fbd40074e30 msgr2=0x7fbd400fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:26:29.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.101+0000 7fbd457ed700 1 -- 172.21.15.71:0/4156718278 shutdown_connections 2024-04-23T02:26:29.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.101+0000 7fbd457ed700 1 -- 172.21.15.71:0/4156718278 wait complete. 2024-04-23T02:26:29.114 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:26:29.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.206+0000 7f04dfe3c700 1 -- 172.21.15.71:0/932468488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04d8101060 msgr2=0x7f04d81047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:29.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.206+0000 7f04dfe3c700 1 --2- 172.21.15.71:0/932468488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04d8101060 0x7f04d81047b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009910 tx=0x7f04d4009c20 comp rx=0 tx=0).stop 2024-04-23T02:26:29.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.206+0000 7f04dfe3c700 1 -- 172.21.15.71:0/932468488 shutdown_connections 2024-04-23T02:26:29.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.206+0000 7f04dfe3c700 1 --2- 172.21.15.71:0/932468488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04d8101060 0x7f04d81047b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:29.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.206+0000 7f04dfe3c700 1 --2- 172.21.15.71:0/932468488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04d81006b0 0x7f04d8100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:29.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.206+0000 7f04dfe3c700 1 -- 172.21.15.71:0/932468488 >> 172.21.15.71:0/932468488 conn(0x7f04d8074e30 msgr2=0x7f04d8075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:26:29.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.206+0000 7f04dfe3c700 1 -- 172.21.15.71:0/932468488 shutdown_connections 2024-04-23T02:26:29.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.206+0000 7f04dfe3c700 1 -- 172.21.15.71:0/932468488 wait complete. 2024-04-23T02:26:29.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.207+0000 7f04dfe3c700 1 Processor -- start 2024-04-23T02:26:29.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.207+0000 7f04dfe3c700 1 -- start start 2024-04-23T02:26:29.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.207+0000 7f04dfe3c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04d81006b0 0x7f04d8197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:29.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.207+0000 7f04dfe3c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04d8101060 0x7f04d8197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:29.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.207+0000 7f04dfe3c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d8198170 con 0x7f04d81006b0 2024-04-23T02:26:29.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.207+0000 7f04dfe3c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d81982b0 con 0x7f04d8101060 2024-04-23T02:26:29.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.207+0000 7f04ddbd8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04d81006b0 0x7f04d8197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:29.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.207+0000 7f04ddbd8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04d81006b0 0x7f04d8197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38318/0 (socket says 172.21.15.71:38318) 2024-04-23T02:26:29.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.207+0000 7f04ddbd8700 1 -- 172.21.15.71:0/1671437087 learned_addr learned my addr 172.21.15.71:0/1671437087 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:26:29.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.208+0000 7f04dd3d7700 1 --2- 172.21.15.71:0/1671437087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04d8101060 0x7f04d8197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:29.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.208+0000 7f04ddbd8700 1 -- 172.21.15.71:0/1671437087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04d8101060 msgr2=0x7f04d8197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:29.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.208+0000 7f04ddbd8700 1 --2- 172.21.15.71:0/1671437087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04d8101060 0x7f04d8197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:29.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.208+0000 7f04ddbd8700 1 -- 172.21.15.71:0/1671437087 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d40095f0 con 0x7f04d81006b0 2024-04-23T02:26:29.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.208+0000 7f04ddbd8700 1 --2- 172.21.15.71:0/1671437087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04d81006b0 0x7f04d8197610 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f04c800d8d0 tx=0x7f04c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:26:29.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.208+0000 7f04ceffd700 1 -- 172.21.15.71:0/1671437087 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04c8009940 con 0x7f04d81006b0 2024-04-23T02:26:29.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.208+0000 7f04dfe3c700 1 -- 172.21.15.71:0/1671437087 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d81a0680 con 0x7f04d81006b0 2024-04-23T02:26:29.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.208+0000 7f04ceffd700 1 -- 172.21.15.71:0/1671437087 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f04c8010460 con 0x7f04d81006b0 2024-04-23T02:26:29.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.208+0000 7f04ceffd700 1 -- 172.21.15.71:0/1671437087 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04c800f5f0 con 0x7f04d81006b0 2024-04-23T02:26:29.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.208+0000 7f04dfe3c700 1 -- 172.21.15.71:0/1671437087 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d81a0b40 con 0x7f04d81006b0 2024-04-23T02:26:29.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.209+0000 7f04dfe3c700 1 -- 172.21.15.71:0/1671437087 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d8108090 con 0x7f04d81006b0 2024-04-23T02:26:29.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.209+0000 7f04ceffd700 1 -- 172.21.15.71:0/1671437087 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f04c80105d0 con 0x7f04d81006b0 2024-04-23T02:26:29.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.210+0000 7f04ceffd700 1 --2- 172.21.15.71:0/1671437087 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f04c406cce0 0x7f04c406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:29.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.210+0000 7f04ceffd700 1 -- 172.21.15.71:0/1671437087 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f04c808c5e0 con 0x7f04d81006b0 2024-04-23T02:26:29.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.210+0000 7f04dd3d7700 1 --2- 172.21.15.71:0/1671437087 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f04c406cce0 0x7f04c406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:29.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.210+0000 7f04dd3d7700 1 --2- 172.21.15.71:0/1671437087 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f04c406cce0 0x7f04c406f1a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f04d40059d0 tx=0x7f04d4005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:26:29.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.214+0000 7f04ceffd700 1 -- 172.21.15.71:0/1671437087 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f04c8056650 con 0x7f04d81006b0 2024-04-23T02:26:29.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.425+0000 7f04dfe3c700 1 -- 172.21.15.71:0/1671437087 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f04d80611d0 con 0x7f04c406cce0 2024-04-23T02:26:29.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.427+0000 7f04ceffd700 1 -- 172.21.15.71:0/1671437087 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f04d80611d0 con 0x7f04c406cce0 2024-04-23T02:26:29.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.429+0000 7f04dfe3c700 1 -- 172.21.15.71:0/1671437087 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f04c406cce0 msgr2=0x7f04c406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:29.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.429+0000 7f04dfe3c700 1 --2- 172.21.15.71:0/1671437087 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f04c406cce0 0x7f04c406f1a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f04d40059d0 tx=0x7f04d4005960 comp rx=0 tx=0).stop 2024-04-23T02:26:29.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.429+0000 7f04dfe3c700 1 -- 172.21.15.71:0/1671437087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04d81006b0 msgr2=0x7f04d8197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:29.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.429+0000 7f04dfe3c700 1 --2- 172.21.15.71:0/1671437087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04d81006b0 0x7f04d8197610 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f04c800d8d0 tx=0x7f04c800dc90 comp rx=0 tx=0).stop 2024-04-23T02:26:29.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.429+0000 7f04dfe3c700 1 -- 172.21.15.71:0/1671437087 shutdown_connections 2024-04-23T02:26:29.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.429+0000 7f04dfe3c700 1 --2- 172.21.15.71:0/1671437087 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f04c406cce0 0x7f04c406f1a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:29.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.429+0000 7f04dfe3c700 1 --2- 172.21.15.71:0/1671437087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04d81006b0 0x7f04d8197610 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:29.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.429+0000 7f04dfe3c700 1 --2- 172.21.15.71:0/1671437087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04d8101060 0x7f04d8197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:29.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.429+0000 7f04dfe3c700 1 -- 172.21.15.71:0/1671437087 >> 172.21.15.71:0/1671437087 conn(0x7f04d8074e30 msgr2=0x7f04d80fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:26:29.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.429+0000 7f04dfe3c700 1 -- 172.21.15.71:0/1671437087 shutdown_connections 2024-04-23T02:26:29.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.429+0000 7f04dfe3c700 1 -- 172.21.15.71:0/1671437087 wait complete. 2024-04-23T02:26:29.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.531+0000 7f1a3c94b700 1 -- 172.21.15.71:0/3951856344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a38105bd0 msgr2=0x7f1a38105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:29.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.531+0000 7f1a3c94b700 1 --2- 172.21.15.71:0/3951856344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a38105bd0 0x7f1a38105fb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009910 tx=0x7f1a28009c20 comp rx=0 tx=0).stop 2024-04-23T02:26:29.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.531+0000 7f1a3c94b700 1 -- 172.21.15.71:0/3951856344 shutdown_connections 2024-04-23T02:26:29.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.531+0000 7f1a3c94b700 1 --2- 172.21.15.71:0/3951856344 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a380ffc20 0x7f1a38100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:29.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.531+0000 7f1a3c94b700 1 --2- 172.21.15.71:0/3951856344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a38105bd0 0x7f1a38105fb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:29.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.531+0000 7f1a3c94b700 1 -- 172.21.15.71:0/3951856344 >> 172.21.15.71:0/3951856344 conn(0x7f1a38075670 msgr2=0x7f1a38075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:26:29.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.531+0000 7f1a3c94b700 1 -- 172.21.15.71:0/3951856344 shutdown_connections 2024-04-23T02:26:29.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.531+0000 7f1a3c94b700 1 -- 172.21.15.71:0/3951856344 wait complete. 2024-04-23T02:26:29.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.532+0000 7f1a3c94b700 1 Processor -- start 2024-04-23T02:26:29.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.532+0000 7f1a3c94b700 1 -- start start 2024-04-23T02:26:29.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.532+0000 7f1a3c94b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a380ffc20 0x7f1a381976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:29.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.532+0000 7f1a3c94b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a38105bd0 0x7f1a38197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:29.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.532+0000 7f1a3c94b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a38198220 con 0x7f1a38105bd0 2024-04-23T02:26:29.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.532+0000 7f1a3c94b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a38198390 con 0x7f1a380ffc20 2024-04-23T02:26:29.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.533+0000 7f1a35d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a38105bd0 0x7f1a38197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:29.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.533+0000 7f1a35d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a38105bd0 0x7f1a38197c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38336/0 (socket says 172.21.15.71:38336) 2024-04-23T02:26:29.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.533+0000 7f1a35d9b700 1 -- 172.21.15.71:0/3838345299 learned_addr learned my addr 172.21.15.71:0/3838345299 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:26:29.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.533+0000 7f1a3659c700 1 --2- 172.21.15.71:0/3838345299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a380ffc20 0x7f1a381976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:29.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.533+0000 7f1a35d9b700 1 -- 172.21.15.71:0/3838345299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a380ffc20 msgr2=0x7f1a381976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:29.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.533+0000 7f1a35d9b700 1 --2- 172.21.15.71:0/3838345299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a380ffc20 0x7f1a381976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:29.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.533+0000 7f1a35d9b700 1 -- 172.21.15.71:0/3838345299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a280095f0 con 0x7f1a38105bd0 2024-04-23T02:26:29.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.533+0000 7f1a3659c700 1 --2- 172.21.15.71:0/3838345299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a380ffc20 0x7f1a381976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:26:29.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.533+0000 7f1a35d9b700 1 --2- 172.21.15.71:0/3838345299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a38105bd0 0x7f1a38197c00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00b4b0 tx=0x7f1a2c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:26:29.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.533+0000 7f1a277fe700 1 -- 172.21.15.71:0/3838345299 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a2c011800 con 0x7f1a38105bd0 2024-04-23T02:26:29.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.533+0000 7f1a3c94b700 1 -- 172.21.15.71:0/3838345299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a381a0730 con 0x7f1a38105bd0 2024-04-23T02:26:29.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.533+0000 7f1a277fe700 1 -- 172.21.15.71:0/3838345299 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1a2c011e00 con 0x7f1a38105bd0 2024-04-23T02:26:29.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.533+0000 7f1a277fe700 1 -- 172.21.15.71:0/3838345299 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a2c010e50 con 0x7f1a38105bd0 2024-04-23T02:26:29.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.533+0000 7f1a3c94b700 1 -- 172.21.15.71:0/3838345299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a381a0c20 con 0x7f1a38105bd0 2024-04-23T02:26:29.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.534+0000 7f1a3c94b700 1 -- 172.21.15.71:0/3838345299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a381080d0 con 0x7f1a38105bd0 2024-04-23T02:26:29.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.535+0000 7f1a277fe700 1 -- 172.21.15.71:0/3838345299 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1a2c010380 con 0x7f1a38105bd0 2024-04-23T02:26:29.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.535+0000 7f1a277fe700 1 --2- 172.21.15.71:0/3838345299 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1a2006cce0 0x7f1a2006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:29.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.535+0000 7f1a277fe700 1 -- 172.21.15.71:0/3838345299 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1a2c08c190 con 0x7f1a38105bd0 2024-04-23T02:26:29.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.535+0000 7f1a3659c700 1 --2- 172.21.15.71:0/3838345299 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1a2006cce0 0x7f1a2006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:29.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.536+0000 7f1a3659c700 1 --2- 172.21.15.71:0/3838345299 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1a2006cce0 0x7f1a2006f1a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1a28006010 tx=0x7f1a2801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:26:29.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.540+0000 7f1a277fe700 1 -- 172.21.15.71:0/3838345299 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1a2c056200 con 0x7f1a38105bd0 2024-04-23T02:26:29.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.741+0000 7f1a3c94b700 1 -- 172.21.15.71:0/3838345299 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a380611d0 con 0x7f1a2006cce0 2024-04-23T02:26:29.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.752+0000 7f1a277fe700 1 -- 172.21.15.71:0/3838345299 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1a380611d0 con 0x7f1a2006cce0 2024-04-23T02:26:29.755 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:26:29.755 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (10m) 3m ago 21m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:26:29.756 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (21m) 3m ago 21m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:26:29.756 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (20m) 3m ago 20m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:26:29.756 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (8m) 3m ago 21m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:26:29.756 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (7m) 3m ago 20m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:26:29.756 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (9m) 3m ago 21m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:26:29.756 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (4m) 3m ago 15m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:26:29.756 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (4m) 3m ago 15m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:26:29.756 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (15m) 3m ago 15m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:26:29.756 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (3m) 3m ago 15m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:26:29.757 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (12m) 3m ago 23m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:26:29.757 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (11m) 3m ago 20m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:26:29.757 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (8m) 3m ago 23m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:26:29.757 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (8m) 3m ago 20m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:26:29.757 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (11m) 3m ago 21m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:26:29.757 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (11m) 3m ago 20m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:26:29.757 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (7m) 3m ago 19m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:26:29.757 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (7m) 3m ago 19m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:26:29.757 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (6m) 3m ago 18m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:26:29.757 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (5m) 3m ago 18m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:26:29.757 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (5m) 3m ago 17m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:26:29.757 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (4m) 3m ago 17m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:26:29.757 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (10m) 3m ago 20m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:26:29.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.754+0000 7f1a3c94b700 1 -- 172.21.15.71:0/3838345299 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1a2006cce0 msgr2=0x7f1a2006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:29.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.754+0000 7f1a3c94b700 1 --2- 172.21.15.71:0/3838345299 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1a2006cce0 0x7f1a2006f1a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1a28006010 tx=0x7f1a2801a040 comp rx=0 tx=0).stop 2024-04-23T02:26:29.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.754+0000 7f1a3c94b700 1 -- 172.21.15.71:0/3838345299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a38105bd0 msgr2=0x7f1a38197c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:29.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.754+0000 7f1a3c94b700 1 --2- 172.21.15.71:0/3838345299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a38105bd0 0x7f1a38197c00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00b4b0 tx=0x7f1a2c00b870 comp rx=0 tx=0).stop 2024-04-23T02:26:29.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.754+0000 7f1a3c94b700 1 -- 172.21.15.71:0/3838345299 shutdown_connections 2024-04-23T02:26:29.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.754+0000 7f1a3c94b700 1 --2- 172.21.15.71:0/3838345299 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1a2006cce0 0x7f1a2006f1a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:29.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.754+0000 7f1a3c94b700 1 --2- 172.21.15.71:0/3838345299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a380ffc20 0x7f1a381976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:29.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.754+0000 7f1a3c94b700 1 --2- 172.21.15.71:0/3838345299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a38105bd0 0x7f1a38197c00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:29.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.754+0000 7f1a3c94b700 1 -- 172.21.15.71:0/3838345299 >> 172.21.15.71:0/3838345299 conn(0x7f1a38075670 msgr2=0x7f1a380fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:26:29.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.755+0000 7f1a3c94b700 1 -- 172.21.15.71:0/3838345299 shutdown_connections 2024-04-23T02:26:29.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.755+0000 7f1a3c94b700 1 -- 172.21.15.71:0/3838345299 wait complete. 2024-04-23T02:26:29.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.860+0000 7f61e17d9700 1 -- 172.21.15.71:0/785100214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61dc0ff5a0 msgr2=0x7f61dc0ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:29.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.860+0000 7f61e17d9700 1 --2- 172.21.15.71:0/785100214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61dc0ff5a0 0x7f61dc0ff980 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009910 tx=0x7f61c4009c20 comp rx=0 tx=0).stop 2024-04-23T02:26:29.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.860+0000 7f61e17d9700 1 -- 172.21.15.71:0/785100214 shutdown_connections 2024-04-23T02:26:29.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.860+0000 7f61e17d9700 1 --2- 172.21.15.71:0/785100214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61dc0fff50 0x7f61dc105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:29.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.860+0000 7f61e17d9700 1 --2- 172.21.15.71:0/785100214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61dc0ff5a0 0x7f61dc0ff980 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:29.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.860+0000 7f61e17d9700 1 -- 172.21.15.71:0/785100214 >> 172.21.15.71:0/785100214 conn(0x7f61dc074e30 msgr2=0x7f61dc075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:26:29.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.860+0000 7f61e17d9700 1 -- 172.21.15.71:0/785100214 shutdown_connections 2024-04-23T02:26:29.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.860+0000 7f61e17d9700 1 -- 172.21.15.71:0/785100214 wait complete. 2024-04-23T02:26:29.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.860+0000 7f61e17d9700 1 Processor -- start 2024-04-23T02:26:29.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.860+0000 7f61e17d9700 1 -- start start 2024-04-23T02:26:29.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.861+0000 7f61e17d9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61dc0ff5a0 0x7f61dc100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:29.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.861+0000 7f61e17d9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61dc0fff50 0x7f61dc100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:29.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.861+0000 7f61e17d9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc100fa0 con 0x7f61dc0ff5a0 2024-04-23T02:26:29.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.861+0000 7f61e17d9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc1010e0 con 0x7f61dc0fff50 2024-04-23T02:26:29.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.861+0000 7f61daffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61dc0ff5a0 0x7f61dc100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:29.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.861+0000 7f61daffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61dc0ff5a0 0x7f61dc100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38356/0 (socket says 172.21.15.71:38356) 2024-04-23T02:26:29.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.861+0000 7f61daffd700 1 -- 172.21.15.71:0/3529709133 learned_addr learned my addr 172.21.15.71:0/3529709133 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:26:29.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.861+0000 7f61da7fc700 1 --2- 172.21.15.71:0/3529709133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61dc0fff50 0x7f61dc100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:29.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.861+0000 7f61daffd700 1 -- 172.21.15.71:0/3529709133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61dc0fff50 msgr2=0x7f61dc100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:29.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.861+0000 7f61daffd700 1 --2- 172.21.15.71:0/3529709133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61dc0fff50 0x7f61dc100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:29.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.861+0000 7f61daffd700 1 -- 172.21.15.71:0/3529709133 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c40095f0 con 0x7f61dc0ff5a0 2024-04-23T02:26:29.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.861+0000 7f61da7fc700 1 --2- 172.21.15.71:0/3529709133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61dc0fff50 0x7f61dc100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:26:29.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.861+0000 7f61daffd700 1 --2- 172.21.15.71:0/3529709133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61dc0ff5a0 0x7f61dc100520 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f61c4006010 tx=0x7f61c400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:26:29.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.861+0000 7f61d3fff700 1 -- 172.21.15.71:0/3529709133 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61c401d070 con 0x7f61dc0ff5a0 2024-04-23T02:26:29.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.861+0000 7f61d3fff700 1 -- 172.21.15.71:0/3529709133 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f61c400f460 con 0x7f61dc0ff5a0 2024-04-23T02:26:29.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.861+0000 7f61e17d9700 1 -- 172.21.15.71:0/3529709133 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61dc101360 con 0x7f61dc0ff5a0 2024-04-23T02:26:29.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.861+0000 7f61d3fff700 1 -- 172.21.15.71:0/3529709133 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61c4021640 con 0x7f61dc0ff5a0 2024-04-23T02:26:29.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.861+0000 7f61e17d9700 1 -- 172.21.15.71:0/3529709133 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61dc1a2c90 con 0x7f61dc0ff5a0 2024-04-23T02:26:29.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.862+0000 7f61e17d9700 1 -- 172.21.15.71:0/3529709133 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61dc109270 con 0x7f61dc0ff5a0 2024-04-23T02:26:29.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.863+0000 7f61d3fff700 1 -- 172.21.15.71:0/3529709133 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f61c400fa60 con 0x7f61dc0ff5a0 2024-04-23T02:26:29.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.863+0000 7f61d3fff700 1 --2- 172.21.15.71:0/3529709133 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f61c806cce0 0x7f61c806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:29.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.863+0000 7f61da7fc700 1 --2- 172.21.15.71:0/3529709133 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f61c806cce0 0x7f61c806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:29.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.863+0000 7f61d3fff700 1 -- 172.21.15.71:0/3529709133 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f61c408d7b0 con 0x7f61dc0ff5a0 2024-04-23T02:26:29.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.864+0000 7f61da7fc700 1 --2- 172.21.15.71:0/3529709133 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f61c806cce0 0x7f61c806f1a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f61cc005fd0 tx=0x7f61cc005dc0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:26:29.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:29.868+0000 7f61d3fff700 1 -- 172.21.15.71:0/3529709133 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f61c40578a0 con 0x7f61dc0ff5a0 2024-04-23T02:26:30.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:29 smithi192 ceph-mon[108617]: pgmap v395: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:30.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:29 smithi192 ceph-mon[108617]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:26:30.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.141+0000 7f61e17d9700 1 -- 172.21.15.71:0/3529709133 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f61dc108c50 con 0x7f61dc0ff5a0 2024-04-23T02:26:30.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.141+0000 7f61d3fff700 1 -- 172.21.15.71:0/3529709133 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f61c4057030 con 0x7f61dc0ff5a0 2024-04-23T02:26:30.144 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:26:30.144 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:26:30.144 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:26:30.144 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:26:30.144 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:26:30.144 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:26:30.144 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:26:30.145 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:26:30.145 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:26:30.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:26:30.145 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:26:30.145 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:26:30.145 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:26:30.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:26:30.145 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:26:30.145 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:26:30.145 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:26:30.145 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:26:30.145 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:26:30.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.143+0000 7f61e17d9700 1 -- 172.21.15.71:0/3529709133 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f61c806cce0 msgr2=0x7f61c806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:30.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.143+0000 7f61e17d9700 1 --2- 172.21.15.71:0/3529709133 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f61c806cce0 0x7f61c806f1a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f61cc005fd0 tx=0x7f61cc005dc0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.143+0000 7f61e17d9700 1 -- 172.21.15.71:0/3529709133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61dc0ff5a0 msgr2=0x7f61dc100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:30.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.143+0000 7f61e17d9700 1 --2- 172.21.15.71:0/3529709133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61dc0ff5a0 0x7f61dc100520 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f61c4006010 tx=0x7f61c400be20 comp rx=0 tx=0).stop 2024-04-23T02:26:30.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.143+0000 7f61e17d9700 1 -- 172.21.15.71:0/3529709133 shutdown_connections 2024-04-23T02:26:30.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.143+0000 7f61e17d9700 1 --2- 172.21.15.71:0/3529709133 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f61c806cce0 0x7f61c806f1a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.143+0000 7f61e17d9700 1 --2- 172.21.15.71:0/3529709133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61dc0ff5a0 0x7f61dc100520 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.143+0000 7f61e17d9700 1 --2- 172.21.15.71:0/3529709133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61dc0fff50 0x7f61dc100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.143+0000 7f61e17d9700 1 -- 172.21.15.71:0/3529709133 >> 172.21.15.71:0/3529709133 conn(0x7f61dc074e30 msgr2=0x7f61dc0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:26:30.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.143+0000 7f61e17d9700 1 -- 172.21.15.71:0/3529709133 shutdown_connections 2024-04-23T02:26:30.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.143+0000 7f61e17d9700 1 -- 172.21.15.71:0/3529709133 wait complete. 2024-04-23T02:26:30.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:29 smithi071 ceph-mon[144544]: pgmap v395: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:30.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:29 smithi071 ceph-mon[144544]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:26:30.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.246+0000 7fbf7fa76700 1 -- 172.21.15.71:0/3910192942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf78100e80 msgr2=0x7fbf78101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:30.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.246+0000 7fbf7fa76700 1 --2- 172.21.15.71:0/3910192942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf78100e80 0x7fbf78101260 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009910 tx=0x7fbf68009c20 comp rx=0 tx=0).stop 2024-04-23T02:26:30.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.246+0000 7fbf7fa76700 1 -- 172.21.15.71:0/3910192942 shutdown_connections 2024-04-23T02:26:30.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.246+0000 7fbf7fa76700 1 --2- 172.21.15.71:0/3910192942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf78101830 0x7fbf78105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.246+0000 7fbf7fa76700 1 --2- 172.21.15.71:0/3910192942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf78100e80 0x7fbf78101260 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.246+0000 7fbf7fa76700 1 -- 172.21.15.71:0/3910192942 >> 172.21.15.71:0/3910192942 conn(0x7fbf780fc710 msgr2=0x7fbf780feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:26:30.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.246+0000 7fbf7fa76700 1 -- 172.21.15.71:0/3910192942 shutdown_connections 2024-04-23T02:26:30.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.246+0000 7fbf7fa76700 1 -- 172.21.15.71:0/3910192942 wait complete. 2024-04-23T02:26:30.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.247+0000 7fbf7fa76700 1 Processor -- start 2024-04-23T02:26:30.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.247+0000 7fbf7fa76700 1 -- start start 2024-04-23T02:26:30.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.247+0000 7fbf7fa76700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf78100e80 0x7fbf7819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:30.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.247+0000 7fbf7fa76700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf78101830 0x7fbf7819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:30.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.247+0000 7fbf7fa76700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7819d150 con 0x7fbf78100e80 2024-04-23T02:26:30.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.247+0000 7fbf7fa76700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf78196600 con 0x7fbf78101830 2024-04-23T02:26:30.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.247+0000 7fbf7d812700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf78100e80 0x7fbf7819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:30.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.247+0000 7fbf7d812700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf78100e80 0x7fbf7819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38380/0 (socket says 172.21.15.71:38380) 2024-04-23T02:26:30.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.247+0000 7fbf7d812700 1 -- 172.21.15.71:0/1636884171 learned_addr learned my addr 172.21.15.71:0/1636884171 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:26:30.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.247+0000 7fbf7d812700 1 -- 172.21.15.71:0/1636884171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf78101830 msgr2=0x7fbf7819cac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-23T02:26:30.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.247+0000 7fbf7d812700 1 --2- 172.21.15.71:0/1636884171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf78101830 0x7fbf7819cac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.247+0000 7fbf7d812700 1 -- 172.21.15.71:0/1636884171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf680095f0 con 0x7fbf78100e80 2024-04-23T02:26:30.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.248+0000 7fbf7d812700 1 --2- 172.21.15.71:0/1636884171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf78100e80 0x7fbf7819c580 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbf68006010 tx=0x7fbf6800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:26:30.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.248+0000 7fbf6effd700 1 -- 172.21.15.71:0/1636884171 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf6801d070 con 0x7fbf78100e80 2024-04-23T02:26:30.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.248+0000 7fbf6effd700 1 -- 172.21.15.71:0/1636884171 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbf6800f460 con 0x7fbf78100e80 2024-04-23T02:26:30.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.248+0000 7fbf7fa76700 1 -- 172.21.15.71:0/1636884171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf78196880 con 0x7fbf78100e80 2024-04-23T02:26:30.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.248+0000 7fbf6effd700 1 -- 172.21.15.71:0/1636884171 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf68021620 con 0x7fbf78100e80 2024-04-23T02:26:30.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.248+0000 7fbf7fa76700 1 -- 172.21.15.71:0/1636884171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf78196d70 con 0x7fbf78100e80 2024-04-23T02:26:30.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.249+0000 7fbf7fa76700 1 -- 172.21.15.71:0/1636884171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf78109220 con 0x7fbf78100e80 2024-04-23T02:26:30.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.249+0000 7fbf6effd700 1 -- 172.21.15.71:0/1636884171 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbf6800fa40 con 0x7fbf78100e80 2024-04-23T02:26:30.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.250+0000 7fbf6effd700 1 --2- 172.21.15.71:0/1636884171 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbf6406cce0 0x7fbf6406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:30.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.250+0000 7fbf7d011700 1 --2- 172.21.15.71:0/1636884171 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbf6406cce0 0x7fbf6406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:30.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.250+0000 7fbf6effd700 1 -- 172.21.15.71:0/1636884171 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbf6808d790 con 0x7fbf78100e80 2024-04-23T02:26:30.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.250+0000 7fbf7d011700 1 --2- 172.21.15.71:0/1636884171 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbf6406cce0 0x7fbf6406f1a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbf78105b00 tx=0x7fbf7400b3f0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:26:30.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.254+0000 7fbf6effd700 1 -- 172.21.15.71:0/1636884171 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbf6805b290 con 0x7fbf78100e80 2024-04-23T02:26:30.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.489+0000 7fbf7fa76700 1 -- 172.21.15.71:0/1636884171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbf7804f2e0 con 0x7fbf78100e80 2024-04-23T02:26:30.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.489+0000 7fbf6effd700 1 -- 172.21.15.71:0/1636884171 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7fbf6805ae20 con 0x7fbf78100e80 2024-04-23T02:26:30.493 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:26:30.493 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:26:30.493 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:26:30.493 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:26:30.493 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:26:30.493 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:26:30.493 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:26:30.493 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:26:30.493 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:26:30.493 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:26:30.493 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:26:30.493 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:26:30.494 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:26:30.494 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:26:30.494 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:26:30.494 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:26:30.494 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:26:30.494 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:26:30.494 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:26:30.494 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:26:30.494 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:26:30.494 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:26:30.494 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:26:30.494 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:26:30.495 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:26:30.495 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:26:30.495 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:26:30.495 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:26:30.495 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:26:30.495 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:26:30.495 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:26:30.495 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:26:30.495 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:26:30.495 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:26:30.495 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:26:30.495 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:26:30.495 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:26:30.495 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:26:30.496 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:26:30.496 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:26:30.496 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:26:30.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.491+0000 7fbf7fa76700 1 -- 172.21.15.71:0/1636884171 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbf6406cce0 msgr2=0x7fbf6406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:30.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.491+0000 7fbf7fa76700 1 --2- 172.21.15.71:0/1636884171 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbf6406cce0 0x7fbf6406f1a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbf78105b00 tx=0x7fbf7400b3f0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.491+0000 7fbf7fa76700 1 -- 172.21.15.71:0/1636884171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf78100e80 msgr2=0x7fbf7819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:30.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.491+0000 7fbf7fa76700 1 --2- 172.21.15.71:0/1636884171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf78100e80 0x7fbf7819c580 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbf68006010 tx=0x7fbf6800be20 comp rx=0 tx=0).stop 2024-04-23T02:26:30.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.492+0000 7fbf7fa76700 1 -- 172.21.15.71:0/1636884171 shutdown_connections 2024-04-23T02:26:30.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.492+0000 7fbf7fa76700 1 --2- 172.21.15.71:0/1636884171 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbf6406cce0 0x7fbf6406f1a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.492+0000 7fbf7fa76700 1 --2- 172.21.15.71:0/1636884171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf78100e80 0x7fbf7819c580 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.492+0000 7fbf7fa76700 1 --2- 172.21.15.71:0/1636884171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf78101830 0x7fbf7819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.492+0000 7fbf7fa76700 1 -- 172.21.15.71:0/1636884171 >> 172.21.15.71:0/1636884171 conn(0x7fbf780fc710 msgr2=0x7fbf780fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:26:30.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.492+0000 7fbf7fa76700 1 -- 172.21.15.71:0/1636884171 shutdown_connections 2024-04-23T02:26:30.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.492+0000 7fbf7fa76700 1 -- 172.21.15.71:0/1636884171 wait complete. 2024-04-23T02:26:30.497 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:26:30.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.595+0000 7ff0afbd6700 1 -- 172.21.15.71:0/642328758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0a8068580 msgr2=0x7ff0a80689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:30.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.595+0000 7ff0afbd6700 1 --2- 172.21.15.71:0/642328758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0a8068580 0x7ff0a80689e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009910 tx=0x7ff0a4009c20 comp rx=0 tx=0).stop 2024-04-23T02:26:30.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.595+0000 7ff0afbd6700 1 -- 172.21.15.71:0/642328758 shutdown_connections 2024-04-23T02:26:30.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.595+0000 7ff0afbd6700 1 --2- 172.21.15.71:0/642328758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0a8068580 0x7ff0a80689e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.595+0000 7ff0afbd6700 1 --2- 172.21.15.71:0/642328758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0a8106cd0 0x7ff0a81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.595+0000 7ff0afbd6700 1 -- 172.21.15.71:0/642328758 >> 172.21.15.71:0/642328758 conn(0x7ff0a8075670 msgr2=0x7ff0a8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:26:30.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.595+0000 7ff0afbd6700 1 -- 172.21.15.71:0/642328758 shutdown_connections 2024-04-23T02:26:30.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.595+0000 7ff0afbd6700 1 -- 172.21.15.71:0/642328758 wait complete. 2024-04-23T02:26:30.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.596+0000 7ff0afbd6700 1 Processor -- start 2024-04-23T02:26:30.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.596+0000 7ff0afbd6700 1 -- start start 2024-04-23T02:26:30.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.596+0000 7ff0afbd6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0a8068580 0x7ff0a819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:30.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.596+0000 7ff0afbd6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0a8106cd0 0x7ff0a819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:30.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.596+0000 7ff0afbd6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a819d290 con 0x7ff0a8106cd0 2024-04-23T02:26:30.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.596+0000 7ff0afbd6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a8196740 con 0x7ff0a8068580 2024-04-23T02:26:30.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.596+0000 7ff0ad171700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0a8106cd0 0x7ff0a819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:30.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.596+0000 7ff0ad171700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0a8106cd0 0x7ff0a819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38398/0 (socket says 172.21.15.71:38398) 2024-04-23T02:26:30.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.596+0000 7ff0ad171700 1 -- 172.21.15.71:0/2836517389 learned_addr learned my addr 172.21.15.71:0/2836517389 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:26:30.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.597+0000 7ff0ad972700 1 --2- 172.21.15.71:0/2836517389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0a8068580 0x7ff0a819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:30.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.597+0000 7ff0ad171700 1 -- 172.21.15.71:0/2836517389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0a8068580 msgr2=0x7ff0a819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:30.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.597+0000 7ff0ad171700 1 --2- 172.21.15.71:0/2836517389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0a8068580 0x7ff0a819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.597+0000 7ff0ad171700 1 -- 172.21.15.71:0/2836517389 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a40095f0 con 0x7ff0a8106cd0 2024-04-23T02:26:30.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.597+0000 7ff0ad972700 1 --2- 172.21.15.71:0/2836517389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0a8068580 0x7ff0a819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:26:30.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.597+0000 7ff0ad171700 1 --2- 172.21.15.71:0/2836517389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0a8106cd0 0x7ff0a819cc00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4006010 tx=0x7ff0a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:26:30.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.597+0000 7ff09affd700 1 -- 172.21.15.71:0/2836517389 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0a401d070 con 0x7ff0a8106cd0 2024-04-23T02:26:30.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.597+0000 7ff09affd700 1 -- 172.21.15.71:0/2836517389 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff0a400f460 con 0x7ff0a8106cd0 2024-04-23T02:26:30.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.597+0000 7ff0afbd6700 1 -- 172.21.15.71:0/2836517389 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a81969c0 con 0x7ff0a8106cd0 2024-04-23T02:26:30.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.597+0000 7ff09affd700 1 -- 172.21.15.71:0/2836517389 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0a4021640 con 0x7ff0a8106cd0 2024-04-23T02:26:30.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.597+0000 7ff0afbd6700 1 -- 172.21.15.71:0/2836517389 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a8196eb0 con 0x7ff0a8106cd0 2024-04-23T02:26:30.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.598+0000 7ff0afbd6700 1 -- 172.21.15.71:0/2836517389 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a81091f0 con 0x7ff0a8106cd0 2024-04-23T02:26:30.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.599+0000 7ff09affd700 1 -- 172.21.15.71:0/2836517389 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff0a400fa60 con 0x7ff0a8106cd0 2024-04-23T02:26:30.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.599+0000 7ff09affd700 1 --2- 172.21.15.71:0/2836517389 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff09406cce0 0x7ff09406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:30.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.599+0000 7ff09affd700 1 -- 172.21.15.71:0/2836517389 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff0a408d880 con 0x7ff0a8106cd0 2024-04-23T02:26:30.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.599+0000 7ff0ad972700 1 --2- 172.21.15.71:0/2836517389 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff09406cce0 0x7ff09406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:30.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.599+0000 7ff0ad972700 1 --2- 172.21.15.71:0/2836517389 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff09406cce0 0x7ff09406f1a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff09c007900 tx=0x7ff09c008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:26:30.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.603+0000 7ff09affd700 1 -- 172.21.15.71:0/2836517389 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff0a405b380 con 0x7ff0a8106cd0 2024-04-23T02:26:30.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.817+0000 7ff0afbd6700 1 -- 172.21.15.71:0/2836517389 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0a81970d0 con 0x7ff09406cce0 2024-04-23T02:26:30.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.818+0000 7ff09affd700 1 -- 172.21.15.71:0/2836517389 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff0a81970d0 con 0x7ff09406cce0 2024-04-23T02:26:30.821 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:26:30.821 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:26:30.821 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:26:30.821 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:26:30.821 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:26:30.821 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:26:30.821 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:26:30.821 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:26:30.821 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:26:30.821 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:26:30.821 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:26:30.821 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:26:30.821 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:26:30.822 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:26:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.820+0000 7ff0afbd6700 1 -- 172.21.15.71:0/2836517389 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff09406cce0 msgr2=0x7ff09406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.820+0000 7ff0afbd6700 1 --2- 172.21.15.71:0/2836517389 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff09406cce0 0x7ff09406f1a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff09c007900 tx=0x7ff09c008040 comp rx=0 tx=0).stop 2024-04-23T02:26:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.820+0000 7ff0afbd6700 1 -- 172.21.15.71:0/2836517389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0a8106cd0 msgr2=0x7ff0a819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.820+0000 7ff0afbd6700 1 --2- 172.21.15.71:0/2836517389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0a8106cd0 0x7ff0a819cc00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4006010 tx=0x7ff0a400be20 comp rx=0 tx=0).stop 2024-04-23T02:26:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.820+0000 7ff0afbd6700 1 -- 172.21.15.71:0/2836517389 shutdown_connections 2024-04-23T02:26:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.820+0000 7ff0afbd6700 1 --2- 172.21.15.71:0/2836517389 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff09406cce0 0x7ff09406f1a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.820+0000 7ff0afbd6700 1 --2- 172.21.15.71:0/2836517389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0a8068580 0x7ff0a819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.820+0000 7ff0afbd6700 1 --2- 172.21.15.71:0/2836517389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0a8106cd0 0x7ff0a819cc00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.820+0000 7ff0afbd6700 1 -- 172.21.15.71:0/2836517389 >> 172.21.15.71:0/2836517389 conn(0x7ff0a8075670 msgr2=0x7ff0a80fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:26:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.820+0000 7ff0afbd6700 1 -- 172.21.15.71:0/2836517389 shutdown_connections 2024-04-23T02:26:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.820+0000 7ff0afbd6700 1 -- 172.21.15.71:0/2836517389 wait complete. 2024-04-23T02:26:30.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.926+0000 7ff65ace4700 1 -- 172.21.15.71:0/3762877347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff654100750 msgr2=0x7ff6541047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:30.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.926+0000 7ff65ace4700 1 --2- 172.21.15.71:0/3762877347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff654100750 0x7ff6541047a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff640009910 tx=0x7ff640009c20 comp rx=0 tx=0).stop 2024-04-23T02:26:30.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.926+0000 7ff65ace4700 1 -- 172.21.15.71:0/3762877347 shutdown_connections 2024-04-23T02:26:30.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.926+0000 7ff65ace4700 1 --2- 172.21.15.71:0/3762877347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff654100750 0x7ff6541047a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.926+0000 7ff65ace4700 1 --2- 172.21.15.71:0/3762877347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6540ffda0 0x7ff654100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.926+0000 7ff65ace4700 1 -- 172.21.15.71:0/3762877347 >> 172.21.15.71:0/3762877347 conn(0x7ff6540fb650 msgr2=0x7ff6540fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:26:30.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.926+0000 7ff65ace4700 1 -- 172.21.15.71:0/3762877347 shutdown_connections 2024-04-23T02:26:30.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.926+0000 7ff65ace4700 1 -- 172.21.15.71:0/3762877347 wait complete. 2024-04-23T02:26:30.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.927+0000 7ff65ace4700 1 Processor -- start 2024-04-23T02:26:30.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.927+0000 7ff65ace4700 1 -- start start 2024-04-23T02:26:30.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.927+0000 7ff65ace4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6540ffda0 0x7ff654101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:30.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.927+0000 7ff65ace4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff654100750 0x7ff654101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:30.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.927+0000 7ff65ace4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6541022c0 con 0x7ff654100750 2024-04-23T02:26:30.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.927+0000 7ff65ace4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff654102400 con 0x7ff6540ffda0 2024-04-23T02:26:30.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.927+0000 7ff653fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff654100750 0x7ff654101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:30.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.927+0000 7ff653fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff654100750 0x7ff654101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38418/0 (socket says 172.21.15.71:38418) 2024-04-23T02:26:30.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.927+0000 7ff653fff700 1 -- 172.21.15.71:0/4059847690 learned_addr learned my addr 172.21.15.71:0/4059847690 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:26:30.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.927+0000 7ff653fff700 1 -- 172.21.15.71:0/4059847690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6540ffda0 msgr2=0x7ff654101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:30.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.927+0000 7ff653fff700 1 --2- 172.21.15.71:0/4059847690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6540ffda0 0x7ff654101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:30.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.927+0000 7ff653fff700 1 -- 172.21.15.71:0/4059847690 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6400095f0 con 0x7ff654100750 2024-04-23T02:26:30.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.928+0000 7ff653fff700 1 --2- 172.21.15.71:0/4059847690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff654100750 0x7ff654101d80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff64000bda0 tx=0x7ff64000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:26:30.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.928+0000 7ff651ffb700 1 -- 172.21.15.71:0/4059847690 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff64001d070 con 0x7ff654100750 2024-04-23T02:26:30.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.928+0000 7ff651ffb700 1 -- 172.21.15.71:0/4059847690 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff64000f460 con 0x7ff654100750 2024-04-23T02:26:30.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.928+0000 7ff65ace4700 1 -- 172.21.15.71:0/4059847690 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff654102680 con 0x7ff654100750 2024-04-23T02:26:30.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.928+0000 7ff651ffb700 1 -- 172.21.15.71:0/4059847690 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff640021640 con 0x7ff654100750 2024-04-23T02:26:30.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.928+0000 7ff65ace4700 1 -- 172.21.15.71:0/4059847690 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6541a2ce0 con 0x7ff654100750 2024-04-23T02:26:30.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.929+0000 7ff65ace4700 1 -- 172.21.15.71:0/4059847690 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff65404f2e0 con 0x7ff654100750 2024-04-23T02:26:30.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.929+0000 7ff651ffb700 1 -- 172.21.15.71:0/4059847690 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff64000fa20 con 0x7ff654100750 2024-04-23T02:26:30.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.930+0000 7ff651ffb700 1 --2- 172.21.15.71:0/4059847690 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff64406cd30 0x7ff64406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:26:30.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.930+0000 7ff651ffb700 1 -- 172.21.15.71:0/4059847690 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff64008d740 con 0x7ff654100750 2024-04-23T02:26:30.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.930+0000 7ff658a80700 1 --2- 172.21.15.71:0/4059847690 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff64406cd30 0x7ff64406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:26:30.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.930+0000 7ff658a80700 1 --2- 172.21.15.71:0/4059847690 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff64406cd30 0x7ff64406f1f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff648005950 tx=0x7ff6480058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:26:30.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:30.934+0000 7ff651ffb700 1 -- 172.21.15.71:0/4059847690 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff640057790 con 0x7ff654100750 2024-04-23T02:26:31.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:31.208+0000 7ff65ace4700 1 -- 172.21.15.71:0/4059847690 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff6541a2ff0 con 0x7ff654100750 2024-04-23T02:26:31.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:31.208+0000 7ff651ffb700 1 -- 172.21.15.71:0/4059847690 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7ff64005adb0 con 0x7ff654100750 2024-04-23T02:26:31.211 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:26:31.211 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:26:31.211 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:26:31.211 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:26:31.211 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:26:31.211 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:26:31.211 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:26:31.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:31.210+0000 7ff65ace4700 1 -- 172.21.15.71:0/4059847690 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff64406cd30 msgr2=0x7ff64406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:31.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:31.210+0000 7ff65ace4700 1 --2- 172.21.15.71:0/4059847690 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff64406cd30 0x7ff64406f1f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff648005950 tx=0x7ff6480058e0 comp rx=0 tx=0).stop 2024-04-23T02:26:31.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:31.210+0000 7ff65ace4700 1 -- 172.21.15.71:0/4059847690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff654100750 msgr2=0x7ff654101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:26:31.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:31.210+0000 7ff65ace4700 1 --2- 172.21.15.71:0/4059847690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff654100750 0x7ff654101d80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff64000bda0 tx=0x7ff64000be80 comp rx=0 tx=0).stop 2024-04-23T02:26:31.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:31.210+0000 7ff65ace4700 1 -- 172.21.15.71:0/4059847690 shutdown_connections 2024-04-23T02:26:31.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:31.210+0000 7ff65ace4700 1 --2- 172.21.15.71:0/4059847690 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff64406cd30 0x7ff64406f1f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:31.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:31.210+0000 7ff65ace4700 1 --2- 172.21.15.71:0/4059847690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6540ffda0 0x7ff654101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:31.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:31.210+0000 7ff65ace4700 1 --2- 172.21.15.71:0/4059847690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff654100750 0x7ff654101d80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:26:31.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:31.210+0000 7ff65ace4700 1 -- 172.21.15.71:0/4059847690 >> 172.21.15.71:0/4059847690 conn(0x7ff6540fb650 msgr2=0x7ff654068f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:26:31.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:31.210+0000 7ff65ace4700 1 -- 172.21.15.71:0/4059847690 shutdown_connections 2024-04-23T02:26:31.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:26:31.210+0000 7ff65ace4700 1 -- 172.21.15.71:0/4059847690 wait complete. 2024-04-23T02:26:31.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:30 smithi071 ceph-mon[144544]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:26:31.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:30 smithi071 ceph-mon[144544]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:26:31.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:30 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3529709133' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:26:31.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:30 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1636884171' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:26:31.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:30 smithi192 ceph-mon[108617]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:26:31.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:30 smithi192 ceph-mon[108617]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:26:31.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:30 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3529709133' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:26:31.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:30 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1636884171' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:26:32.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:31 smithi071 ceph-mon[144544]: from='client.34586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:26:32.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:31 smithi071 ceph-mon[144544]: pgmap v396: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:32.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:31 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/4059847690' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:26:32.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:31 smithi192 ceph-mon[108617]: from='client.34586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:26:32.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:31 smithi192 ceph-mon[108617]: pgmap v396: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:32.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:31 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/4059847690' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:26:34.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:33 smithi071 ceph-mon[144544]: pgmap v397: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:34.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:33 smithi192 ceph-mon[108617]: pgmap v397: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:35.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:34 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:26:35.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:34 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:26:36.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:35 smithi071 ceph-mon[144544]: pgmap v398: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:36.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:35 smithi192 ceph-mon[108617]: pgmap v398: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:38.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:37 smithi071 ceph-mon[144544]: pgmap v399: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:38.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:37 smithi192 ceph-mon[108617]: pgmap v399: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:40.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:39 smithi192 ceph-mon[108617]: pgmap v400: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:40.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:39 smithi071 ceph-mon[144544]: pgmap v400: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:42.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:41 smithi071 ceph-mon[144544]: pgmap v401: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:42.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:41 smithi192 ceph-mon[108617]: pgmap v401: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:44.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:43 smithi071 ceph-mon[144544]: pgmap v402: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:44.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:43 smithi192 ceph-mon[108617]: pgmap v402: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:46.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:45 smithi071 ceph-mon[144544]: pgmap v403: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:46.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:45 smithi192 ceph-mon[108617]: pgmap v403: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:48.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:47 smithi071 ceph-mon[144544]: pgmap v404: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:48.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:47 smithi192 ceph-mon[108617]: pgmap v404: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:50.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:49 smithi071 ceph-mon[144544]: pgmap v405: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:50.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:26:50.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:49 smithi192 ceph-mon[108617]: pgmap v405: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:50.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:26:52.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:51 smithi071 ceph-mon[144544]: pgmap v406: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:52.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:51 smithi192 ceph-mon[108617]: pgmap v406: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:54.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:53 smithi071 ceph-mon[144544]: pgmap v407: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:54.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:53 smithi192 ceph-mon[108617]: pgmap v407: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:56.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:55 smithi071 ceph-mon[144544]: pgmap v408: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:56.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:55 smithi192 ceph-mon[108617]: pgmap v408: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:58.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:57 smithi071 ceph-mon[144544]: pgmap v409: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:26:58.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:57 smithi192 ceph-mon[108617]: pgmap v409: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:00.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:26:59 smithi071 ceph-mon[144544]: pgmap v410: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:00.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:26:59 smithi192 ceph-mon[108617]: pgmap v410: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:01.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.326+0000 7f9802a56700 1 -- 172.21.15.71:0/276545341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97fc100e90 msgr2=0x7f97fc101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:01.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.326+0000 7f9802a56700 1 --2- 172.21.15.71:0/276545341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97fc100e90 0x7f97fc101270 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009910 tx=0x7f97ec009c20 comp rx=0 tx=0).stop 2024-04-23T02:27:01.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.326+0000 7f9802a56700 1 -- 172.21.15.71:0/276545341 shutdown_connections 2024-04-23T02:27:01.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.326+0000 7f9802a56700 1 --2- 172.21.15.71:0/276545341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97fc101840 0x7f97fc105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:01.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.326+0000 7f9802a56700 1 --2- 172.21.15.71:0/276545341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97fc100e90 0x7f97fc101270 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:01.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.326+0000 7f9802a56700 1 -- 172.21.15.71:0/276545341 >> 172.21.15.71:0/276545341 conn(0x7f97fc0fc740 msgr2=0x7f97fc0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:01.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.326+0000 7f9802a56700 1 -- 172.21.15.71:0/276545341 shutdown_connections 2024-04-23T02:27:01.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.326+0000 7f9802a56700 1 -- 172.21.15.71:0/276545341 wait complete. 2024-04-23T02:27:01.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.327+0000 7f9802a56700 1 Processor -- start 2024-04-23T02:27:01.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.327+0000 7f9802a56700 1 -- start start 2024-04-23T02:27:01.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.327+0000 7f9802a56700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97fc100e90 0x7f97fc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:01.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.327+0000 7f9802a56700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97fc101840 0x7f97fc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:01.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.327+0000 7f9802a56700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc073b10 con 0x7f97fc101840 2024-04-23T02:27:01.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.327+0000 7f9802a56700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc073c50 con 0x7f97fc100e90 2024-04-23T02:27:01.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.327+0000 7f97fb7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97fc101840 0x7f97fc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:01.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.327+0000 7f97fb7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97fc101840 0x7f97fc0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56786/0 (socket says 172.21.15.71:56786) 2024-04-23T02:27:01.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.327+0000 7f97fb7fe700 1 -- 172.21.15.71:0/4100331890 learned_addr learned my addr 172.21.15.71:0/4100331890 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:27:01.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.327+0000 7f97fbfff700 1 --2- 172.21.15.71:0/4100331890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97fc100e90 0x7f97fc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:01.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.328+0000 7f97fb7fe700 1 -- 172.21.15.71:0/4100331890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97fc100e90 msgr2=0x7f97fc073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:01.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.328+0000 7f97fb7fe700 1 --2- 172.21.15.71:0/4100331890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97fc100e90 0x7f97fc073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:01.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.328+0000 7f97fb7fe700 1 -- 172.21.15.71:0/4100331890 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec0095f0 con 0x7f97fc101840 2024-04-23T02:27:01.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.328+0000 7f97fb7fe700 1 --2- 172.21.15.71:0/4100331890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97fc101840 0x7f97fc0735d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f97f000c8f0 tx=0x7f97f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:01.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.328+0000 7f97f97fa700 1 -- 172.21.15.71:0/4100331890 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97f0007a50 con 0x7f97fc101840 2024-04-23T02:27:01.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.328+0000 7f9802a56700 1 -- 172.21.15.71:0/4100331890 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97fc1a29c0 con 0x7f97fc101840 2024-04-23T02:27:01.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.328+0000 7f97f97fa700 1 -- 172.21.15.71:0/4100331890 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f97f0007bb0 con 0x7f97fc101840 2024-04-23T02:27:01.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.328+0000 7f97f97fa700 1 -- 172.21.15.71:0/4100331890 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97f0018630 con 0x7f97fc101840 2024-04-23T02:27:01.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.328+0000 7f9802a56700 1 -- 172.21.15.71:0/4100331890 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97fc1a2dd0 con 0x7f97fc101840 2024-04-23T02:27:01.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.329+0000 7f9802a56700 1 -- 172.21.15.71:0/4100331890 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97fc109230 con 0x7f97fc101840 2024-04-23T02:27:01.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.329+0000 7f97f97fa700 1 -- 172.21.15.71:0/4100331890 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f97f0007d20 con 0x7f97fc101840 2024-04-23T02:27:01.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.330+0000 7f97f97fa700 1 --2- 172.21.15.71:0/4100331890 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f97e406cce0 0x7f97e406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:01.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.330+0000 7f97f97fa700 1 -- 172.21.15.71:0/4100331890 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f97f008c590 con 0x7f97fc101840 2024-04-23T02:27:01.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.330+0000 7f97fbfff700 1 --2- 172.21.15.71:0/4100331890 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f97e406cce0 0x7f97e406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:01.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.330+0000 7f97fbfff700 1 --2- 172.21.15.71:0/4100331890 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f97e406cce0 0x7f97e406f1a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f97ec006010 tx=0x7f97ec01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:01.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.334+0000 7f97f97fa700 1 -- 172.21.15.71:0/4100331890 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f97f0056600 con 0x7f97fc101840 2024-04-23T02:27:01.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.544+0000 7f9802a56700 1 -- 172.21.15.71:0/4100331890 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f97fc0611d0 con 0x7f97e406cce0 2024-04-23T02:27:01.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.546+0000 7f97f97fa700 1 -- 172.21.15.71:0/4100331890 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f97fc0611d0 con 0x7f97e406cce0 2024-04-23T02:27:01.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.548+0000 7f9802a56700 1 -- 172.21.15.71:0/4100331890 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f97e406cce0 msgr2=0x7f97e406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:01.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.548+0000 7f9802a56700 1 --2- 172.21.15.71:0/4100331890 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f97e406cce0 0x7f97e406f1a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f97ec006010 tx=0x7f97ec01a040 comp rx=0 tx=0).stop 2024-04-23T02:27:01.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.548+0000 7f9802a56700 1 -- 172.21.15.71:0/4100331890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97fc101840 msgr2=0x7f97fc0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:01.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.548+0000 7f9802a56700 1 --2- 172.21.15.71:0/4100331890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97fc101840 0x7f97fc0735d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f97f000c8f0 tx=0x7f97f000ccb0 comp rx=0 tx=0).stop 2024-04-23T02:27:01.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.548+0000 7f9802a56700 1 -- 172.21.15.71:0/4100331890 shutdown_connections 2024-04-23T02:27:01.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.548+0000 7f9802a56700 1 --2- 172.21.15.71:0/4100331890 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f97e406cce0 0x7f97e406f1a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:01.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.548+0000 7f9802a56700 1 --2- 172.21.15.71:0/4100331890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97fc100e90 0x7f97fc073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:01.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.548+0000 7f9802a56700 1 --2- 172.21.15.71:0/4100331890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97fc101840 0x7f97fc0735d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:01.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.548+0000 7f9802a56700 1 -- 172.21.15.71:0/4100331890 >> 172.21.15.71:0/4100331890 conn(0x7f97fc0fc740 msgr2=0x7f97fc0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:01.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.548+0000 7f9802a56700 1 -- 172.21.15.71:0/4100331890 shutdown_connections 2024-04-23T02:27:01.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.548+0000 7f9802a56700 1 -- 172.21.15.71:0/4100331890 wait complete. 2024-04-23T02:27:01.561 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:27:01.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.653+0000 7fca60273700 1 -- 172.21.15.71:0/1920426417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca58105bd0 msgr2=0x7fca58105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:01.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.653+0000 7fca60273700 1 --2- 172.21.15.71:0/1920426417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca58105bd0 0x7fca58105fb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fca48009910 tx=0x7fca48009c20 comp rx=0 tx=0).stop 2024-04-23T02:27:01.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.653+0000 7fca60273700 1 -- 172.21.15.71:0/1920426417 shutdown_connections 2024-04-23T02:27:01.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.653+0000 7fca60273700 1 --2- 172.21.15.71:0/1920426417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca580ffc20 0x7fca58100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:01.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.653+0000 7fca60273700 1 --2- 172.21.15.71:0/1920426417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca58105bd0 0x7fca58105fb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:01.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.653+0000 7fca60273700 1 -- 172.21.15.71:0/1920426417 >> 172.21.15.71:0/1920426417 conn(0x7fca58075670 msgr2=0x7fca58075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:01.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.653+0000 7fca60273700 1 -- 172.21.15.71:0/1920426417 shutdown_connections 2024-04-23T02:27:01.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.653+0000 7fca60273700 1 -- 172.21.15.71:0/1920426417 wait complete. 2024-04-23T02:27:01.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.653+0000 7fca60273700 1 Processor -- start 2024-04-23T02:27:01.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.654+0000 7fca60273700 1 -- start start 2024-04-23T02:27:01.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.654+0000 7fca60273700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca580ffc20 0x7fca5819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:01.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.654+0000 7fca60273700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca58105bd0 0x7fca5819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:01.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.654+0000 7fca60273700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5819d150 con 0x7fca58105bd0 2024-04-23T02:27:01.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.654+0000 7fca60273700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca58196600 con 0x7fca580ffc20 2024-04-23T02:27:01.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.654+0000 7fca5d80e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca58105bd0 0x7fca5819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:01.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.654+0000 7fca5d80e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca58105bd0 0x7fca5819cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56806/0 (socket says 172.21.15.71:56806) 2024-04-23T02:27:01.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.654+0000 7fca5d80e700 1 -- 172.21.15.71:0/2652267730 learned_addr learned my addr 172.21.15.71:0/2652267730 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:27:01.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.654+0000 7fca5e00f700 1 --2- 172.21.15.71:0/2652267730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca580ffc20 0x7fca5819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:01.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.654+0000 7fca5d80e700 1 -- 172.21.15.71:0/2652267730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca580ffc20 msgr2=0x7fca5819c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:01.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.654+0000 7fca5d80e700 1 --2- 172.21.15.71:0/2652267730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca580ffc20 0x7fca5819c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:01.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.654+0000 7fca5d80e700 1 -- 172.21.15.71:0/2652267730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca480095f0 con 0x7fca58105bd0 2024-04-23T02:27:01.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.654+0000 7fca5d80e700 1 --2- 172.21.15.71:0/2652267730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca58105bd0 0x7fca5819cac0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fca5400e920 tx=0x7fca5400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:01.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.654+0000 7fca4f7fe700 1 -- 172.21.15.71:0/2652267730 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca5400c9d0 con 0x7fca58105bd0 2024-04-23T02:27:01.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.654+0000 7fca4f7fe700 1 -- 172.21.15.71:0/2652267730 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fca54010430 con 0x7fca58105bd0 2024-04-23T02:27:01.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.654+0000 7fca60273700 1 -- 172.21.15.71:0/2652267730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca581968e0 con 0x7fca58105bd0 2024-04-23T02:27:01.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.655+0000 7fca4f7fe700 1 -- 172.21.15.71:0/2652267730 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca54018650 con 0x7fca58105bd0 2024-04-23T02:27:01.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.655+0000 7fca60273700 1 -- 172.21.15.71:0/2652267730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca58196e30 con 0x7fca58105bd0 2024-04-23T02:27:01.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.656+0000 7fca60273700 1 -- 172.21.15.71:0/2652267730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5804f2e0 con 0x7fca58105bd0 2024-04-23T02:27:01.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.656+0000 7fca4f7fe700 1 -- 172.21.15.71:0/2652267730 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fca540105a0 con 0x7fca58105bd0 2024-04-23T02:27:01.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.656+0000 7fca4f7fe700 1 --2- 172.21.15.71:0/2652267730 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fca4406cce0 0x7fca4406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:01.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.656+0000 7fca4f7fe700 1 -- 172.21.15.71:0/2652267730 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fca54014070 con 0x7fca58105bd0 2024-04-23T02:27:01.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.656+0000 7fca5e00f700 1 --2- 172.21.15.71:0/2652267730 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fca4406cce0 0x7fca4406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:01.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.657+0000 7fca5e00f700 1 --2- 172.21.15.71:0/2652267730 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fca4406cce0 0x7fca4406f1a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fca48009fd0 tx=0x7fca48005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:01.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.661+0000 7fca4f7fe700 1 -- 172.21.15.71:0/2652267730 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fca54052c50 con 0x7fca58105bd0 2024-04-23T02:27:01.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.872+0000 7fca60273700 1 -- 172.21.15.71:0/2652267730 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca581970b0 con 0x7fca4406cce0 2024-04-23T02:27:01.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.873+0000 7fca4f7fe700 1 -- 172.21.15.71:0/2652267730 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fca581970b0 con 0x7fca4406cce0 2024-04-23T02:27:01.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.875+0000 7fca60273700 1 -- 172.21.15.71:0/2652267730 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fca4406cce0 msgr2=0x7fca4406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:01.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.875+0000 7fca60273700 1 --2- 172.21.15.71:0/2652267730 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fca4406cce0 0x7fca4406f1a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fca48009fd0 tx=0x7fca48005940 comp rx=0 tx=0).stop 2024-04-23T02:27:01.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.875+0000 7fca60273700 1 -- 172.21.15.71:0/2652267730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca58105bd0 msgr2=0x7fca5819cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:01.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.875+0000 7fca60273700 1 --2- 172.21.15.71:0/2652267730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca58105bd0 0x7fca5819cac0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fca5400e920 tx=0x7fca5400ec30 comp rx=0 tx=0).stop 2024-04-23T02:27:01.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.876+0000 7fca60273700 1 -- 172.21.15.71:0/2652267730 shutdown_connections 2024-04-23T02:27:01.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.876+0000 7fca60273700 1 --2- 172.21.15.71:0/2652267730 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fca4406cce0 0x7fca4406f1a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:01.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.876+0000 7fca60273700 1 --2- 172.21.15.71:0/2652267730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca580ffc20 0x7fca5819c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:01.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.876+0000 7fca60273700 1 --2- 172.21.15.71:0/2652267730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca58105bd0 0x7fca5819cac0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:01.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.876+0000 7fca60273700 1 -- 172.21.15.71:0/2652267730 >> 172.21.15.71:0/2652267730 conn(0x7fca58075670 msgr2=0x7fca58068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:01.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.876+0000 7fca60273700 1 -- 172.21.15.71:0/2652267730 shutdown_connections 2024-04-23T02:27:01.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.876+0000 7fca60273700 1 -- 172.21.15.71:0/2652267730 wait complete. 2024-04-23T02:27:01.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.979+0000 7f177a232700 1 -- 172.21.15.71:0/1224677696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1774101d50 msgr2=0x7f1774102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:01.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.979+0000 7f177a232700 1 --2- 172.21.15.71:0/1224677696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1774101d50 0x7f1774102130 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f176400b0a0 tx=0x7f176400b3b0 comp rx=0 tx=0).stop 2024-04-23T02:27:01.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.979+0000 7f177a232700 1 -- 172.21.15.71:0/1224677696 shutdown_connections 2024-04-23T02:27:01.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.979+0000 7f177a232700 1 --2- 172.21.15.71:0/1224677696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1774102700 0x7f177410abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:01.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.979+0000 7f177a232700 1 --2- 172.21.15.71:0/1224677696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1774101d50 0x7f1774102130 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:01.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.979+0000 7f177a232700 1 -- 172.21.15.71:0/1224677696 >> 172.21.15.71:0/1224677696 conn(0x7f17740fb610 msgr2=0x7f17740fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:01.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.979+0000 7f177a232700 1 -- 172.21.15.71:0/1224677696 shutdown_connections 2024-04-23T02:27:01.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.979+0000 7f177a232700 1 -- 172.21.15.71:0/1224677696 wait complete. 2024-04-23T02:27:01.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.980+0000 7f177a232700 1 Processor -- start 2024-04-23T02:27:01.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.980+0000 7f177a232700 1 -- start start 2024-04-23T02:27:01.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.980+0000 7f177a232700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1774101d50 0x7f177419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.980+0000 7f177a232700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1774102700 0x7f177419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.980+0000 7f177a232700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177419f380 con 0x7f1774102700 2024-04-23T02:27:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.980+0000 7f177a232700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1774198830 con 0x7f1774101d50 2024-04-23T02:27:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.980+0000 7f1772ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1774102700 0x7f177419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.980+0000 7f1772ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1774102700 0x7f177419ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56826/0 (socket says 172.21.15.71:56826) 2024-04-23T02:27:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.980+0000 7f1772ffd700 1 -- 172.21.15.71:0/1616380726 learned_addr learned my addr 172.21.15.71:0/1616380726 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:27:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.980+0000 7f17737fe700 1 --2- 172.21.15.71:0/1616380726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1774101d50 0x7f177419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.981+0000 7f1772ffd700 1 -- 172.21.15.71:0/1616380726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1774101d50 msgr2=0x7f177419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.981+0000 7f1772ffd700 1 --2- 172.21.15.71:0/1616380726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1774101d50 0x7f177419e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.981+0000 7f1772ffd700 1 -- 172.21.15.71:0/1616380726 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1764009d00 con 0x7f1774102700 2024-04-23T02:27:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.981+0000 7f17737fe700 1 --2- 172.21.15.71:0/1616380726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1774101d50 0x7f177419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:27:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.981+0000 7f1772ffd700 1 --2- 172.21.15.71:0/1616380726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1774102700 0x7f177419ecf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f176800e8f0 tx=0x7f176800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.981+0000 7f1770ff9700 1 -- 172.21.15.71:0/1616380726 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1768009790 con 0x7f1774102700 2024-04-23T02:27:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.981+0000 7f1770ff9700 1 -- 172.21.15.71:0/1616380726 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1768004cd0 con 0x7f1774102700 2024-04-23T02:27:01.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.981+0000 7f177a232700 1 -- 172.21.15.71:0/1616380726 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1774198b10 con 0x7f1774102700 2024-04-23T02:27:01.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.981+0000 7f1770ff9700 1 -- 172.21.15.71:0/1616380726 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1768005580 con 0x7f1774102700 2024-04-23T02:27:01.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.981+0000 7f177a232700 1 -- 172.21.15.71:0/1616380726 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1774199060 con 0x7f1774102700 2024-04-23T02:27:01.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.982+0000 7f177a232700 1 -- 172.21.15.71:0/1616380726 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f177410a750 con 0x7f1774102700 2024-04-23T02:27:01.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.982+0000 7f1770ff9700 1 -- 172.21.15.71:0/1616380726 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1768004750 con 0x7f1774102700 2024-04-23T02:27:01.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.983+0000 7f1770ff9700 1 --2- 172.21.15.71:0/1616380726 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f175c06cce0 0x7f175c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:01.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.983+0000 7f1770ff9700 1 -- 172.21.15.71:0/1616380726 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1768014070 con 0x7f1774102700 2024-04-23T02:27:01.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.983+0000 7f17737fe700 1 --2- 172.21.15.71:0/1616380726 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f175c06cce0 0x7f175c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:01.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.983+0000 7f17737fe700 1 --2- 172.21.15.71:0/1616380726 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f175c06cce0 0x7f175c06f1a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f17640062a0 tx=0x7f1764006210 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:01.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:01.989+0000 7f1770ff9700 1 -- 172.21.15.71:0/1616380726 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f176805a400 con 0x7f1774102700 2024-04-23T02:27:02.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.188+0000 7f177a232700 1 -- 172.21.15.71:0/1616380726 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1774199330 con 0x7f175c06cce0 2024-04-23T02:27:02.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.199+0000 7f1770ff9700 1 -- 172.21.15.71:0/1616380726 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1774199330 con 0x7f175c06cce0 2024-04-23T02:27:02.201 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:27:02.202 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (10m) 4m ago 22m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:27:02.202 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (22m) 4m ago 22m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:27:02.202 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (21m) 4m ago 21m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:27:02.202 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (8m) 4m ago 22m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:27:02.202 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (8m) 4m ago 21m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:27:02.202 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (10m) 4m ago 21m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:27:02.202 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (4m) 4m ago 16m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:27:02.202 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (4m) 4m ago 16m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:27:02.202 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (15m) 4m ago 15m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:27:02.202 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (4m) 4m ago 16m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:27:02.202 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (12m) 4m ago 23m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:27:02.202 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (11m) 4m ago 20m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:27:02.202 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (9m) 4m ago 23m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:27:02.202 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (8m) 4m ago 20m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:27:02.203 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (11m) 4m ago 22m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:27:02.203 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (11m) 4m ago 21m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:27:02.203 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (8m) 4m ago 20m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:27:02.203 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (7m) 4m ago 19m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:27:02.203 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (6m) 4m ago 19m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:27:02.203 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (6m) 4m ago 18m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:27:02.203 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (5m) 4m ago 18m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:27:02.203 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (5m) 4m ago 17m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:27:02.203 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (11m) 4m ago 21m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:27:02.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.201+0000 7f177a232700 1 -- 172.21.15.71:0/1616380726 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f175c06cce0 msgr2=0x7f175c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:02.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.201+0000 7f177a232700 1 --2- 172.21.15.71:0/1616380726 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f175c06cce0 0x7f175c06f1a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f17640062a0 tx=0x7f1764006210 comp rx=0 tx=0).stop 2024-04-23T02:27:02.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.201+0000 7f177a232700 1 -- 172.21.15.71:0/1616380726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1774102700 msgr2=0x7f177419ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:02.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.201+0000 7f177a232700 1 --2- 172.21.15.71:0/1616380726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1774102700 0x7f177419ecf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f176800e8f0 tx=0x7f176800ecb0 comp rx=0 tx=0).stop 2024-04-23T02:27:02.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.201+0000 7f177a232700 1 -- 172.21.15.71:0/1616380726 shutdown_connections 2024-04-23T02:27:02.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.201+0000 7f177a232700 1 --2- 172.21.15.71:0/1616380726 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f175c06cce0 0x7f175c06f1a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:02.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.201+0000 7f177a232700 1 --2- 172.21.15.71:0/1616380726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1774101d50 0x7f177419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:02.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.201+0000 7f177a232700 1 --2- 172.21.15.71:0/1616380726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1774102700 0x7f177419ecf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:02.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.201+0000 7f177a232700 1 -- 172.21.15.71:0/1616380726 >> 172.21.15.71:0/1616380726 conn(0x7f17740fb610 msgr2=0x7f17741001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:02.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.201+0000 7f177a232700 1 -- 172.21.15.71:0/1616380726 shutdown_connections 2024-04-23T02:27:02.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.201+0000 7f177a232700 1 -- 172.21.15.71:0/1616380726 wait complete. 2024-04-23T02:27:02.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:01 smithi071 ceph-mon[144544]: pgmap v411: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:02.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.302+0000 7f2ad87be700 1 -- 172.21.15.71:0/228691560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 msgr2=0x7f2ad0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:02.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.302+0000 7f2ad87be700 1 --2- 172.21.15.71:0/228691560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 0x7f2ad0105fb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009910 tx=0x7f2ac4009c20 comp rx=0 tx=0).stop 2024-04-23T02:27:02.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.302+0000 7f2ad87be700 1 -- 172.21.15.71:0/228691560 shutdown_connections 2024-04-23T02:27:02.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.302+0000 7f2ad87be700 1 --2- 172.21.15.71:0/228691560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ad00ffc20 0x7f2ad0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:02.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.302+0000 7f2ad87be700 1 --2- 172.21.15.71:0/228691560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 0x7f2ad0105fb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:02.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.302+0000 7f2ad87be700 1 -- 172.21.15.71:0/228691560 >> 172.21.15.71:0/228691560 conn(0x7f2ad0075670 msgr2=0x7f2ad0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:02.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.302+0000 7f2ad87be700 1 -- 172.21.15.71:0/228691560 shutdown_connections 2024-04-23T02:27:02.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.302+0000 7f2ad87be700 1 -- 172.21.15.71:0/228691560 wait complete. 2024-04-23T02:27:02.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.302+0000 7f2ad87be700 1 Processor -- start 2024-04-23T02:27:02.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.303+0000 7f2ad87be700 1 -- start start 2024-04-23T02:27:02.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.303+0000 7f2ad87be700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ad00ffc20 0x7f2ad01953f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:02.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.303+0000 7f2ad87be700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 0x7f2ad0195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:02.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.303+0000 7f2ad87be700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad0195e70 con 0x7f2ad0105bd0 2024-04-23T02:27:02.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.303+0000 7f2ad87be700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad0195fe0 con 0x7f2ad00ffc20 2024-04-23T02:27:02.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.303+0000 7f2ad5d59700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 0x7f2ad0195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:02.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.303+0000 7f2ad5d59700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 0x7f2ad0195930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56846/0 (socket says 172.21.15.71:56846) 2024-04-23T02:27:02.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.303+0000 7f2ad5d59700 1 -- 172.21.15.71:0/2885043014 learned_addr learned my addr 172.21.15.71:0/2885043014 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:27:02.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.303+0000 7f2ad655a700 1 --2- 172.21.15.71:0/2885043014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ad00ffc20 0x7f2ad01953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:02.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.303+0000 7f2ad5d59700 1 -- 172.21.15.71:0/2885043014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ad00ffc20 msgr2=0x7f2ad01953f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:02.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.303+0000 7f2ad5d59700 1 --2- 172.21.15.71:0/2885043014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ad00ffc20 0x7f2ad01953f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:02.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.303+0000 7f2ad5d59700 1 -- 172.21.15.71:0/2885043014 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac40095f0 con 0x7f2ad0105bd0 2024-04-23T02:27:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.303+0000 7f2ad5d59700 1 --2- 172.21.15.71:0/2885043014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 0x7f2ad0195930 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00d8d0 tx=0x7f2acc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.303+0000 7f2ac37fe700 1 -- 172.21.15.71:0/2885043014 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2acc0098a0 con 0x7f2ad0105bd0 2024-04-23T02:27:02.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.303+0000 7f2ad87be700 1 -- 172.21.15.71:0/2885043014 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad01a2a60 con 0x7f2ad0105bd0 2024-04-23T02:27:02.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.303+0000 7f2ac37fe700 1 -- 172.21.15.71:0/2885043014 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2acc010460 con 0x7f2ad0105bd0 2024-04-23T02:27:02.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.304+0000 7f2ac37fe700 1 -- 172.21.15.71:0/2885043014 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2acc009ea0 con 0x7f2ad0105bd0 2024-04-23T02:27:02.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.304+0000 7f2ad87be700 1 -- 172.21.15.71:0/2885043014 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad01a2dc0 con 0x7f2ad0105bd0 2024-04-23T02:27:02.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.304+0000 7f2ad87be700 1 -- 172.21.15.71:0/2885043014 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad01080d0 con 0x7f2ad0105bd0 2024-04-23T02:27:02.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.305+0000 7f2ac37fe700 1 -- 172.21.15.71:0/2885043014 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2acc0105d0 con 0x7f2ad0105bd0 2024-04-23T02:27:02.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.305+0000 7f2ac37fe700 1 --2- 172.21.15.71:0/2885043014 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2abc06cce0 0x7f2abc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:02.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.305+0000 7f2ac37fe700 1 -- 172.21.15.71:0/2885043014 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2acc08c410 con 0x7f2ad0105bd0 2024-04-23T02:27:02.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.305+0000 7f2ad655a700 1 --2- 172.21.15.71:0/2885043014 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2abc06cce0 0x7f2abc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:02.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.306+0000 7f2ad655a700 1 --2- 172.21.15.71:0/2885043014 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2abc06cce0 0x7f2abc06f1a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400b5c0 tx=0x7f2ac401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:02.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.310+0000 7f2ac37fe700 1 -- 172.21.15.71:0/2885043014 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2acc059f10 con 0x7f2ad0105bd0 2024-04-23T02:27:02.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:01 smithi192 ceph-mon[108617]: pgmap v411: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:02.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.583+0000 7f2ad87be700 1 -- 172.21.15.71:0/2885043014 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2ad0107ab0 con 0x7f2ad0105bd0 2024-04-23T02:27:02.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.584+0000 7f2ac37fe700 1 -- 172.21.15.71:0/2885043014 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f2acc010880 con 0x7f2ad0105bd0 2024-04-23T02:27:02.587 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:27:02.587 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:27:02.587 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:27:02.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:27:02.587 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:27:02.587 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:27:02.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:27:02.587 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:27:02.587 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:27:02.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:27:02.587 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:27:02.588 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:27:02.588 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:27:02.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:27:02.588 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:27:02.588 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:27:02.588 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:27:02.588 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:27:02.588 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:27:02.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.586+0000 7f2ad87be700 1 -- 172.21.15.71:0/2885043014 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2abc06cce0 msgr2=0x7f2abc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:02.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.586+0000 7f2ad87be700 1 --2- 172.21.15.71:0/2885043014 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2abc06cce0 0x7f2abc06f1a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400b5c0 tx=0x7f2ac401a040 comp rx=0 tx=0).stop 2024-04-23T02:27:02.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.586+0000 7f2ad87be700 1 -- 172.21.15.71:0/2885043014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 msgr2=0x7f2ad0195930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:02.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.586+0000 7f2ad87be700 1 --2- 172.21.15.71:0/2885043014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 0x7f2ad0195930 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00d8d0 tx=0x7f2acc00dbe0 comp rx=0 tx=0).stop 2024-04-23T02:27:02.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.586+0000 7f2ad87be700 1 -- 172.21.15.71:0/2885043014 shutdown_connections 2024-04-23T02:27:02.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.586+0000 7f2ad87be700 1 --2- 172.21.15.71:0/2885043014 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2abc06cce0 0x7f2abc06f1a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:02.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.586+0000 7f2ad87be700 1 --2- 172.21.15.71:0/2885043014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ad00ffc20 0x7f2ad01953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:02.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.586+0000 7f2ad87be700 1 --2- 172.21.15.71:0/2885043014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad0105bd0 0x7f2ad0195930 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:02.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.586+0000 7f2ad87be700 1 -- 172.21.15.71:0/2885043014 >> 172.21.15.71:0/2885043014 conn(0x7f2ad0075670 msgr2=0x7f2ad00fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:02.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.586+0000 7f2ad87be700 1 -- 172.21.15.71:0/2885043014 shutdown_connections 2024-04-23T02:27:02.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.586+0000 7f2ad87be700 1 -- 172.21.15.71:0/2885043014 wait complete. 2024-04-23T02:27:02.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.689+0000 7fcfd495e700 1 -- 172.21.15.71:0/644142213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd0105bd0 msgr2=0x7fcfd0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:02.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.689+0000 7fcfd495e700 1 --2- 172.21.15.71:0/644142213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd0105bd0 0x7fcfd0105fb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009910 tx=0x7fcfb8009c20 comp rx=0 tx=0).stop 2024-04-23T02:27:02.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.689+0000 7fcfd495e700 1 -- 172.21.15.71:0/644142213 shutdown_connections 2024-04-23T02:27:02.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.689+0000 7fcfd495e700 1 --2- 172.21.15.71:0/644142213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd00ffc20 0x7fcfd0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:02.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.689+0000 7fcfd495e700 1 --2- 172.21.15.71:0/644142213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd0105bd0 0x7fcfd0105fb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:02.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.689+0000 7fcfd495e700 1 -- 172.21.15.71:0/644142213 >> 172.21.15.71:0/644142213 conn(0x7fcfd0075670 msgr2=0x7fcfd0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:02.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.689+0000 7fcfd495e700 1 -- 172.21.15.71:0/644142213 shutdown_connections 2024-04-23T02:27:02.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.689+0000 7fcfd495e700 1 -- 172.21.15.71:0/644142213 wait complete. 2024-04-23T02:27:02.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.690+0000 7fcfd495e700 1 Processor -- start 2024-04-23T02:27:02.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.690+0000 7fcfd495e700 1 -- start start 2024-04-23T02:27:02.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.690+0000 7fcfd495e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd00ffc20 0x7fcfd019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:02.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.690+0000 7fcfd495e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd0105bd0 0x7fcfd019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:02.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.690+0000 7fcfd495e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd0196620 con 0x7fcfd00ffc20 2024-04-23T02:27:02.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.690+0000 7fcfd495e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd0196790 con 0x7fcfd0105bd0 2024-04-23T02:27:02.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.690+0000 7fcfce59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd00ffc20 0x7fcfd019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:02.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.690+0000 7fcfce59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd00ffc20 0x7fcfd019c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56862/0 (socket says 172.21.15.71:56862) 2024-04-23T02:27:02.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.690+0000 7fcfce59c700 1 -- 172.21.15.71:0/1369410378 learned_addr learned my addr 172.21.15.71:0/1369410378 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:27:02.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.690+0000 7fcfcdd9b700 1 --2- 172.21.15.71:0/1369410378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd0105bd0 0x7fcfd019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:02.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.691+0000 7fcfce59c700 1 -- 172.21.15.71:0/1369410378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd0105bd0 msgr2=0x7fcfd019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:02.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.691+0000 7fcfce59c700 1 --2- 172.21.15.71:0/1369410378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd0105bd0 0x7fcfd019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:02.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.691+0000 7fcfce59c700 1 -- 172.21.15.71:0/1369410378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb80095f0 con 0x7fcfd00ffc20 2024-04-23T02:27:02.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.691+0000 7fcfce59c700 1 --2- 172.21.15.71:0/1369410378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd00ffc20 0x7fcfd019c630 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8006010 tx=0x7fcfb800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:02.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.691+0000 7fcfc77fe700 1 -- 172.21.15.71:0/1369410378 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfb801d070 con 0x7fcfd00ffc20 2024-04-23T02:27:02.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.691+0000 7fcfd495e700 1 -- 172.21.15.71:0/1369410378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd0196a10 con 0x7fcfd00ffc20 2024-04-23T02:27:02.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.691+0000 7fcfc77fe700 1 -- 172.21.15.71:0/1369410378 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcfb8005940 con 0x7fcfd00ffc20 2024-04-23T02:27:02.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.691+0000 7fcfc77fe700 1 -- 172.21.15.71:0/1369410378 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfb800fd90 con 0x7fcfd00ffc20 2024-04-23T02:27:02.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.691+0000 7fcfd495e700 1 -- 172.21.15.71:0/1369410378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd0196f00 con 0x7fcfd00ffc20 2024-04-23T02:27:02.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.692+0000 7fcfd495e700 1 -- 172.21.15.71:0/1369410378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd004f2e0 con 0x7fcfd00ffc20 2024-04-23T02:27:02.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.692+0000 7fcfc77fe700 1 -- 172.21.15.71:0/1369410378 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fcfb802b440 con 0x7fcfd00ffc20 2024-04-23T02:27:02.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.693+0000 7fcfc77fe700 1 --2- 172.21.15.71:0/1369410378 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcfbc06cce0 0x7fcfbc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:02.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.693+0000 7fcfc77fe700 1 -- 172.21.15.71:0/1369410378 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcfb808d750 con 0x7fcfd00ffc20 2024-04-23T02:27:02.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.693+0000 7fcfcdd9b700 1 --2- 172.21.15.71:0/1369410378 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcfbc06cce0 0x7fcfbc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:02.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.693+0000 7fcfcdd9b700 1 --2- 172.21.15.71:0/1369410378 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcfbc06cce0 0x7fcfbc06f1a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0197ec0 tx=0x7fcfc000b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:02.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.697+0000 7fcfc77fe700 1 -- 172.21.15.71:0/1369410378 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fcfb8057830 con 0x7fcfd00ffc20 2024-04-23T02:27:02.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.935+0000 7fcfd495e700 1 -- 172.21.15.71:0/1369410378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcfd0197b50 con 0x7fcfd00ffc20 2024-04-23T02:27:02.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.936+0000 7fcfc77fe700 1 -- 172.21.15.71:0/1369410378 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7fcfb800f560 con 0x7fcfd00ffc20 2024-04-23T02:27:02.939 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:27:02.939 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:27:02.940 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:27:02.940 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:27:02.940 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:27:02.940 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:27:02.940 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:27:02.940 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:27:02.941 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:27:02.941 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:27:02.941 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:27:02.941 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:27:02.941 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:27:02.942 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:27:02.942 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:27:02.942 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:27:02.942 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:27:02.942 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:27:02.942 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:27:02.942 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:27:02.943 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:27:02.943 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:27:02.943 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:27:02.943 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:27:02.943 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:27:02.943 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:27:02.943 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:27:02.943 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:27:02.943 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:27:02.944 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:27:02.944 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:27:02.944 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:27:02.944 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:27:02.944 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:27:02.944 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:27:02.944 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:27:02.944 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:27:02.944 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:27:02.945 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:27:02.945 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:27:02.945 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:27:02.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.938+0000 7fcfd495e700 1 -- 172.21.15.71:0/1369410378 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcfbc06cce0 msgr2=0x7fcfbc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:02.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.938+0000 7fcfd495e700 1 --2- 172.21.15.71:0/1369410378 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcfbc06cce0 0x7fcfbc06f1a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0197ec0 tx=0x7fcfc000b410 comp rx=0 tx=0).stop 2024-04-23T02:27:02.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.938+0000 7fcfd495e700 1 -- 172.21.15.71:0/1369410378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd00ffc20 msgr2=0x7fcfd019c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:02.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.938+0000 7fcfd495e700 1 --2- 172.21.15.71:0/1369410378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd00ffc20 0x7fcfd019c630 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8006010 tx=0x7fcfb800bd70 comp rx=0 tx=0).stop 2024-04-23T02:27:02.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.938+0000 7fcfd495e700 1 -- 172.21.15.71:0/1369410378 shutdown_connections 2024-04-23T02:27:02.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.938+0000 7fcfd495e700 1 --2- 172.21.15.71:0/1369410378 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcfbc06cce0 0x7fcfbc06f1a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:02.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.938+0000 7fcfd495e700 1 --2- 172.21.15.71:0/1369410378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd00ffc20 0x7fcfd019c630 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:02.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.938+0000 7fcfd495e700 1 --2- 172.21.15.71:0/1369410378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd0105bd0 0x7fcfd019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:02.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.938+0000 7fcfd495e700 1 -- 172.21.15.71:0/1369410378 >> 172.21.15.71:0/1369410378 conn(0x7fcfd0075670 msgr2=0x7fcfd0068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:02.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.938+0000 7fcfd495e700 1 -- 172.21.15.71:0/1369410378 shutdown_connections 2024-04-23T02:27:02.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:02.938+0000 7fcfd495e700 1 -- 172.21.15.71:0/1369410378 wait complete. 2024-04-23T02:27:02.947 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:27:03.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.043+0000 7fcfd9d43700 1 -- 172.21.15.71:0/240381116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4106cd0 msgr2=0x7fcfd41070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:03.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.043+0000 7fcfd9d43700 1 --2- 172.21.15.71:0/240381116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4106cd0 0x7fcfd41070b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009910 tx=0x7fcfc4009c20 comp rx=0 tx=0).stop 2024-04-23T02:27:03.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.043+0000 7fcfd9d43700 1 -- 172.21.15.71:0/240381116 shutdown_connections 2024-04-23T02:27:03.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.043+0000 7fcfd9d43700 1 --2- 172.21.15.71:0/240381116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd4068580 0x7fcfd40689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:03.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.043+0000 7fcfd9d43700 1 --2- 172.21.15.71:0/240381116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4106cd0 0x7fcfd41070b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:03.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.043+0000 7fcfd9d43700 1 -- 172.21.15.71:0/240381116 >> 172.21.15.71:0/240381116 conn(0x7fcfd4075670 msgr2=0x7fcfd4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:03.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.043+0000 7fcfd9d43700 1 -- 172.21.15.71:0/240381116 shutdown_connections 2024-04-23T02:27:03.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.043+0000 7fcfd9d43700 1 -- 172.21.15.71:0/240381116 wait complete. 2024-04-23T02:27:03.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.044+0000 7fcfd9d43700 1 Processor -- start 2024-04-23T02:27:03.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.044+0000 7fcfd9d43700 1 -- start start 2024-04-23T02:27:03.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.044+0000 7fcfd9d43700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd4068580 0x7fcfd40ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:03.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.044+0000 7fcfd9d43700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4106cd0 0x7fcfd4100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:03.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.044+0000 7fcfd9d43700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd4100770 con 0x7fcfd4106cd0 2024-04-23T02:27:03.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.044+0000 7fcfd9d43700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd41008b0 con 0x7fcfd4068580 2024-04-23T02:27:03.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.044+0000 7fcfd2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4106cd0 0x7fcfd4100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:03.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.044+0000 7fcfd2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4106cd0 0x7fcfd4100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56880/0 (socket says 172.21.15.71:56880) 2024-04-23T02:27:03.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.044+0000 7fcfd2ffd700 1 -- 172.21.15.71:0/1775442952 learned_addr learned my addr 172.21.15.71:0/1775442952 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:27:03.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.045+0000 7fcfd37fe700 1 --2- 172.21.15.71:0/1775442952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd4068580 0x7fcfd40ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:03.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.045+0000 7fcfd2ffd700 1 -- 172.21.15.71:0/1775442952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd4068580 msgr2=0x7fcfd40ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:03.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.045+0000 7fcfd2ffd700 1 --2- 172.21.15.71:0/1775442952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd4068580 0x7fcfd40ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:03.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.045+0000 7fcfd2ffd700 1 -- 172.21.15.71:0/1775442952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc40095f0 con 0x7fcfd4106cd0 2024-04-23T02:27:03.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.045+0000 7fcfd37fe700 1 --2- 172.21.15.71:0/1775442952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd4068580 0x7fcfd40ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:27:03.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.045+0000 7fcfd2ffd700 1 --2- 172.21.15.71:0/1775442952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4106cd0 0x7fcfd4100230 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800d8d0 tx=0x7fcfc800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:03.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.045+0000 7fcfd0ff9700 1 -- 172.21.15.71:0/1775442952 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc8009940 con 0x7fcfd4106cd0 2024-04-23T02:27:03.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.045+0000 7fcfd9d43700 1 -- 172.21.15.71:0/1775442952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd41a29b0 con 0x7fcfd4106cd0 2024-04-23T02:27:03.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.045+0000 7fcfd0ff9700 1 -- 172.21.15.71:0/1775442952 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcfc8010460 con 0x7fcfd4106cd0 2024-04-23T02:27:03.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.045+0000 7fcfd0ff9700 1 -- 172.21.15.71:0/1775442952 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc800f5f0 con 0x7fcfd4106cd0 2024-04-23T02:27:03.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.045+0000 7fcfd9d43700 1 -- 172.21.15.71:0/1775442952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd41a2d60 con 0x7fcfd4106cd0 2024-04-23T02:27:03.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.046+0000 7fcfd9d43700 1 -- 172.21.15.71:0/1775442952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd41091d0 con 0x7fcfd4106cd0 2024-04-23T02:27:03.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.046+0000 7fcfd0ff9700 1 -- 172.21.15.71:0/1775442952 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fcfc80105d0 con 0x7fcfd4106cd0 2024-04-23T02:27:03.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.047+0000 7fcfd0ff9700 1 --2- 172.21.15.71:0/1775442952 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcfbc06cce0 0x7fcfbc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:03.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.047+0000 7fcfd0ff9700 1 -- 172.21.15.71:0/1775442952 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcfc808c590 con 0x7fcfd4106cd0 2024-04-23T02:27:03.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.047+0000 7fcfd37fe700 1 --2- 172.21.15.71:0/1775442952 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcfbc06cce0 0x7fcfbc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:03.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.047+0000 7fcfd37fe700 1 --2- 172.21.15.71:0/1775442952 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcfbc06cce0 0x7fcfbc06f1a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4006010 tx=0x7fcfc400b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:03.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.051+0000 7fcfd0ff9700 1 -- 172.21.15.71:0/1775442952 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fcfc8056600 con 0x7fcfd4106cd0 2024-04-23T02:27:03.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:02 smithi071 ceph-mon[144544]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:27:03.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:02 smithi071 ceph-mon[144544]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:27:03.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:02 smithi071 ceph-mon[144544]: from='client.34602 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:27:03.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:02 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2885043014' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:27:03.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:02 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1369410378' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:27:03.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.263+0000 7fcfd9d43700 1 -- 172.21.15.71:0/1775442952 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfd402ccc0 con 0x7fcfbc06cce0 2024-04-23T02:27:03.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.264+0000 7fcfd0ff9700 1 -- 172.21.15.71:0/1775442952 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcfd402ccc0 con 0x7fcfbc06cce0 2024-04-23T02:27:03.266 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:27:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:27:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:27:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:27:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:27:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:27:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:27:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:27:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:27:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:27:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:27:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:27:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:27:03.267 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:27:03.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.266+0000 7fcfd9d43700 1 -- 172.21.15.71:0/1775442952 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcfbc06cce0 msgr2=0x7fcfbc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:03.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.266+0000 7fcfd9d43700 1 --2- 172.21.15.71:0/1775442952 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcfbc06cce0 0x7fcfbc06f1a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4006010 tx=0x7fcfc400b540 comp rx=0 tx=0).stop 2024-04-23T02:27:03.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.266+0000 7fcfd9d43700 1 -- 172.21.15.71:0/1775442952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4106cd0 msgr2=0x7fcfd4100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:03.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.266+0000 7fcfd9d43700 1 --2- 172.21.15.71:0/1775442952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4106cd0 0x7fcfd4100230 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800d8d0 tx=0x7fcfc800dc90 comp rx=0 tx=0).stop 2024-04-23T02:27:03.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.266+0000 7fcfd9d43700 1 -- 172.21.15.71:0/1775442952 shutdown_connections 2024-04-23T02:27:03.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.266+0000 7fcfd9d43700 1 --2- 172.21.15.71:0/1775442952 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcfbc06cce0 0x7fcfbc06f1a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:03.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.266+0000 7fcfd9d43700 1 --2- 172.21.15.71:0/1775442952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd4068580 0x7fcfd40ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:03.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.266+0000 7fcfd9d43700 1 --2- 172.21.15.71:0/1775442952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4106cd0 0x7fcfd4100230 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:03.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.266+0000 7fcfd9d43700 1 -- 172.21.15.71:0/1775442952 >> 172.21.15.71:0/1775442952 conn(0x7fcfd4075670 msgr2=0x7fcfd40fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:03.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.266+0000 7fcfd9d43700 1 -- 172.21.15.71:0/1775442952 shutdown_connections 2024-04-23T02:27:03.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.266+0000 7fcfd9d43700 1 -- 172.21.15.71:0/1775442952 wait complete. 2024-04-23T02:27:03.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.368+0000 7fa323441700 1 -- 172.21.15.71:0/466959970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa31c105350 msgr2=0x7fa31c105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:03.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.368+0000 7fa323441700 1 --2- 172.21.15.71:0/466959970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa31c105350 0x7fa31c105730 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa310009910 tx=0x7fa310009c20 comp rx=0 tx=0).stop 2024-04-23T02:27:03.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.368+0000 7fa323441700 1 -- 172.21.15.71:0/466959970 shutdown_connections 2024-04-23T02:27:03.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.368+0000 7fa323441700 1 --2- 172.21.15.71:0/466959970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa31c0ff340 0x7fa31c0ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:03.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.368+0000 7fa323441700 1 --2- 172.21.15.71:0/466959970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa31c105350 0x7fa31c105730 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:03.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.368+0000 7fa323441700 1 -- 172.21.15.71:0/466959970 >> 172.21.15.71:0/466959970 conn(0x7fa31c074e30 msgr2=0x7fa31c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:03.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.368+0000 7fa323441700 1 -- 172.21.15.71:0/466959970 shutdown_connections 2024-04-23T02:27:03.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.368+0000 7fa323441700 1 -- 172.21.15.71:0/466959970 wait complete. 2024-04-23T02:27:03.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.369+0000 7fa323441700 1 Processor -- start 2024-04-23T02:27:03.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.369+0000 7fa323441700 1 -- start start 2024-04-23T02:27:03.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.369+0000 7fa323441700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa31c0ff340 0x7fa31c19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:03.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.369+0000 7fa323441700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa31c105350 0x7fa31c19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:03.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.369+0000 7fa323441700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c19d100 con 0x7fa31c0ff340 2024-04-23T02:27:03.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.369+0000 7fa323441700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c1965b0 con 0x7fa31c105350 2024-04-23T02:27:03.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.369+0000 7fa3211dd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa31c0ff340 0x7fa31c19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:03.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.369+0000 7fa3211dd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa31c0ff340 0x7fa31c19c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56896/0 (socket says 172.21.15.71:56896) 2024-04-23T02:27:03.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.369+0000 7fa3211dd700 1 -- 172.21.15.71:0/3216089103 learned_addr learned my addr 172.21.15.71:0/3216089103 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:27:03.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.369+0000 7fa3209dc700 1 --2- 172.21.15.71:0/3216089103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa31c105350 0x7fa31c19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:03.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.370+0000 7fa3211dd700 1 -- 172.21.15.71:0/3216089103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa31c105350 msgr2=0x7fa31c19ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:03.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.370+0000 7fa3211dd700 1 --2- 172.21.15.71:0/3216089103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa31c105350 0x7fa31c19ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:03.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.370+0000 7fa3211dd700 1 -- 172.21.15.71:0/3216089103 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3100095f0 con 0x7fa31c0ff340 2024-04-23T02:27:03.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.370+0000 7fa3211dd700 1 --2- 172.21.15.71:0/3216089103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa31c0ff340 0x7fa31c19c530 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa310006010 tx=0x7fa31000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:03.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.370+0000 7fa30e7fc700 1 -- 172.21.15.71:0/3216089103 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa31001d070 con 0x7fa31c0ff340 2024-04-23T02:27:03.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.370+0000 7fa30e7fc700 1 -- 172.21.15.71:0/3216089103 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa310005710 con 0x7fa31c0ff340 2024-04-23T02:27:03.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.370+0000 7fa323441700 1 -- 172.21.15.71:0/3216089103 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31c196830 con 0x7fa31c0ff340 2024-04-23T02:27:03.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.370+0000 7fa30e7fc700 1 -- 172.21.15.71:0/3216089103 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa31000fe20 con 0x7fa31c0ff340 2024-04-23T02:27:03.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.370+0000 7fa323441700 1 -- 172.21.15.71:0/3216089103 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31c196d20 con 0x7fa31c0ff340 2024-04-23T02:27:03.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.371+0000 7fa323441700 1 -- 172.21.15.71:0/3216089103 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31c109880 con 0x7fa31c0ff340 2024-04-23T02:27:03.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:02 smithi192 ceph-mon[108617]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:27:03.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:02 smithi192 ceph-mon[108617]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:27:03.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:02 smithi192 ceph-mon[108617]: from='client.34602 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:27:03.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:02 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2885043014' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:27:03.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:02 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1369410378' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:27:03.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.372+0000 7fa30e7fc700 1 -- 172.21.15.71:0/3216089103 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa310003a60 con 0x7fa31c0ff340 2024-04-23T02:27:03.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.372+0000 7fa30e7fc700 1 --2- 172.21.15.71:0/3216089103 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa30806cce0 0x7fa30806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:03.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.372+0000 7fa30e7fc700 1 -- 172.21.15.71:0/3216089103 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa31008d700 con 0x7fa31c0ff340 2024-04-23T02:27:03.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.372+0000 7fa3209dc700 1 --2- 172.21.15.71:0/3216089103 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa30806cce0 0x7fa30806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:03.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.372+0000 7fa3209dc700 1 --2- 172.21.15.71:0/3216089103 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa30806cce0 0x7fa30806f1a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0fd990 tx=0x7fa31800b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:03.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.377+0000 7fa30e7fc700 1 -- 172.21.15.71:0/3216089103 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa31005b200 con 0x7fa31c0ff340 2024-04-23T02:27:03.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.648+0000 7fa323441700 1 -- 172.21.15.71:0/3216089103 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa31c04f2e0 con 0x7fa31c0ff340 2024-04-23T02:27:03.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.649+0000 7fa30e7fc700 1 -- 172.21.15.71:0/3216089103 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fa31005ad90 con 0x7fa31c0ff340 2024-04-23T02:27:03.652 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:27:03.652 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:27:03.652 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:27:03.652 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:27:03.652 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:27:03.652 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:27:03.652 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:27:03.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.651+0000 7fa323441700 1 -- 172.21.15.71:0/3216089103 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa30806cce0 msgr2=0x7fa30806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:03.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.651+0000 7fa323441700 1 --2- 172.21.15.71:0/3216089103 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa30806cce0 0x7fa30806f1a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0fd990 tx=0x7fa31800b410 comp rx=0 tx=0).stop 2024-04-23T02:27:03.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.651+0000 7fa323441700 1 -- 172.21.15.71:0/3216089103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa31c0ff340 msgr2=0x7fa31c19c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:03.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.651+0000 7fa323441700 1 --2- 172.21.15.71:0/3216089103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa31c0ff340 0x7fa31c19c530 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa310006010 tx=0x7fa31000bd70 comp rx=0 tx=0).stop 2024-04-23T02:27:03.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.651+0000 7fa323441700 1 -- 172.21.15.71:0/3216089103 shutdown_connections 2024-04-23T02:27:03.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.651+0000 7fa323441700 1 --2- 172.21.15.71:0/3216089103 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa30806cce0 0x7fa30806f1a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:03.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.651+0000 7fa323441700 1 --2- 172.21.15.71:0/3216089103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa31c0ff340 0x7fa31c19c530 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:03.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.651+0000 7fa323441700 1 --2- 172.21.15.71:0/3216089103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa31c105350 0x7fa31c19ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:03.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.651+0000 7fa323441700 1 -- 172.21.15.71:0/3216089103 >> 172.21.15.71:0/3216089103 conn(0x7fa31c074e30 msgr2=0x7fa31c104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:03.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.651+0000 7fa323441700 1 -- 172.21.15.71:0/3216089103 shutdown_connections 2024-04-23T02:27:03.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:03.651+0000 7fa323441700 1 -- 172.21.15.71:0/3216089103 wait complete. 2024-04-23T02:27:04.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:03 smithi071 ceph-mon[144544]: pgmap v412: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:04.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:03 smithi071 ceph-mon[144544]: from='client.34614 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:27:04.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:03 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3216089103' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:27:04.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:03 smithi192 ceph-mon[108617]: pgmap v412: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:04.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:03 smithi192 ceph-mon[108617]: from='client.34614 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:27:04.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:03 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3216089103' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:27:05.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:04 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:27:05.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:04 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:27:06.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:05 smithi192 ceph-mon[108617]: pgmap v413: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:06.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:05 smithi071 ceph-mon[144544]: pgmap v413: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:08.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:07 smithi192 ceph-mon[108617]: pgmap v414: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:08.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:07 smithi071 ceph-mon[144544]: pgmap v414: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:10.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:10 smithi192 ceph-mon[108617]: pgmap v415: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:10.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:10 smithi071 ceph-mon[144544]: pgmap v415: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:12.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:12 smithi192 ceph-mon[108617]: pgmap v416: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:12.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:12 smithi071 ceph-mon[144544]: pgmap v416: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:14.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:14 smithi192 ceph-mon[108617]: pgmap v417: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:14.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:14 smithi071 ceph-mon[144544]: pgmap v417: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:16.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:16 smithi192 ceph-mon[108617]: pgmap v418: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:16.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:16 smithi071 ceph-mon[144544]: pgmap v418: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:18.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:18 smithi192 ceph-mon[108617]: pgmap v419: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:18.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:18 smithi071 ceph-mon[144544]: pgmap v419: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:20.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:20 smithi071 ceph-mon[144544]: pgmap v420: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:20.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:27:20.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:20 smithi192 ceph-mon[108617]: pgmap v420: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:20.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:27:22.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:22 smithi192 ceph-mon[108617]: pgmap v421: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:22.632 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:22 smithi071 ceph-mon[144544]: pgmap v421: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:24.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:24 smithi192 ceph-mon[108617]: pgmap v422: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:24.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:24 smithi071 ceph-mon[144544]: pgmap v422: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:26.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:26 smithi192 ceph-mon[108617]: pgmap v423: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:26.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:26 smithi071 ceph-mon[144544]: pgmap v423: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:28.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:28 smithi192 ceph-mon[108617]: pgmap v424: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:28.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:28 smithi071 ceph-mon[144544]: pgmap v424: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:30.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:30 smithi192 ceph-mon[108617]: pgmap v425: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:30.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:30 smithi071 ceph-mon[144544]: pgmap v425: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:32.632 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:32 smithi071 ceph-mon[144544]: pgmap v426: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:32.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:32 smithi192 ceph-mon[108617]: pgmap v426: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:33.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.768+0000 7fee75763700 1 -- 172.21.15.71:0/1411701534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee70105bd0 msgr2=0x7fee70105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:33.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.768+0000 7fee75763700 1 --2- 172.21.15.71:0/1411701534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee70105bd0 0x7fee70105fb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fee6000b0a0 tx=0x7fee6000b3b0 comp rx=0 tx=0).stop 2024-04-23T02:27:33.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.768+0000 7fee75763700 1 -- 172.21.15.71:0/1411701534 shutdown_connections 2024-04-23T02:27:33.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.768+0000 7fee75763700 1 --2- 172.21.15.71:0/1411701534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee700ffc20 0x7fee70100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:33.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.768+0000 7fee75763700 1 --2- 172.21.15.71:0/1411701534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee70105bd0 0x7fee70105fb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:33.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.768+0000 7fee75763700 1 -- 172.21.15.71:0/1411701534 >> 172.21.15.71:0/1411701534 conn(0x7fee70075670 msgr2=0x7fee70075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:33.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.768+0000 7fee75763700 1 -- 172.21.15.71:0/1411701534 shutdown_connections 2024-04-23T02:27:33.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.768+0000 7fee75763700 1 -- 172.21.15.71:0/1411701534 wait complete. 2024-04-23T02:27:33.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.769+0000 7fee75763700 1 Processor -- start 2024-04-23T02:27:33.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.769+0000 7fee75763700 1 -- start start 2024-04-23T02:27:33.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.769+0000 7fee75763700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee700ffc20 0x7fee70198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:33.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.769+0000 7fee75763700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee70105bd0 0x7fee70198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:33.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.769+0000 7fee75763700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee70199560 con 0x7fee700ffc20 2024-04-23T02:27:33.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.769+0000 7fee75763700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7019d370 con 0x7fee70105bd0 2024-04-23T02:27:33.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.769+0000 7fee6effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee700ffc20 0x7fee70198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:33.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.769+0000 7fee6effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee700ffc20 0x7fee70198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56532/0 (socket says 172.21.15.71:56532) 2024-04-23T02:27:33.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.770+0000 7fee6effd700 1 -- 172.21.15.71:0/264351068 learned_addr learned my addr 172.21.15.71:0/264351068 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:27:33.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.770+0000 7fee6e7fc700 1 --2- 172.21.15.71:0/264351068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee70105bd0 0x7fee70198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:33.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.770+0000 7fee6effd700 1 -- 172.21.15.71:0/264351068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee70105bd0 msgr2=0x7fee70198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:33.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.770+0000 7fee6effd700 1 --2- 172.21.15.71:0/264351068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee70105bd0 0x7fee70198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:33.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.770+0000 7fee6effd700 1 -- 172.21.15.71:0/264351068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee60009d00 con 0x7fee700ffc20 2024-04-23T02:27:33.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.770+0000 7fee6effd700 1 --2- 172.21.15.71:0/264351068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee700ffc20 0x7fee70198990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fee6000d010 tx=0x7fee60007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:33.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.770+0000 7fee67fff700 1 -- 172.21.15.71:0/264351068 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee6000e040 con 0x7fee700ffc20 2024-04-23T02:27:33.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.770+0000 7fee75763700 1 -- 172.21.15.71:0/264351068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7019d5f0 con 0x7fee700ffc20 2024-04-23T02:27:33.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.770+0000 7fee67fff700 1 -- 172.21.15.71:0/264351068 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fee60003e60 con 0x7fee700ffc20 2024-04-23T02:27:33.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.770+0000 7fee67fff700 1 -- 172.21.15.71:0/264351068 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee6001d7c0 con 0x7fee700ffc20 2024-04-23T02:27:33.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.770+0000 7fee75763700 1 -- 172.21.15.71:0/264351068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7019dae0 con 0x7fee700ffc20 2024-04-23T02:27:33.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.771+0000 7fee75763700 1 -- 172.21.15.71:0/264351068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7004f2e0 con 0x7fee700ffc20 2024-04-23T02:27:33.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.772+0000 7fee67fff700 1 -- 172.21.15.71:0/264351068 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fee600046e0 con 0x7fee700ffc20 2024-04-23T02:27:33.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.772+0000 7fee67fff700 1 --2- 172.21.15.71:0/264351068 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee5c06cce0 0x7fee5c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:33.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.772+0000 7fee67fff700 1 -- 172.21.15.71:0/264351068 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fee6008df80 con 0x7fee700ffc20 2024-04-23T02:27:33.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.772+0000 7fee6e7fc700 1 --2- 172.21.15.71:0/264351068 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee5c06cce0 0x7fee5c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:33.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.773+0000 7fee6e7fc700 1 --2- 172.21.15.71:0/264351068 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee5c06cce0 0x7fee5c06f1a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fee58005950 tx=0x7fee580058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:33.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.777+0000 7fee67fff700 1 -- 172.21.15.71:0/264351068 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fee6005ba80 con 0x7fee700ffc20 2024-04-23T02:27:33.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.991+0000 7fee75763700 1 -- 172.21.15.71:0/264351068 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fee7019dd00 con 0x7fee5c06cce0 2024-04-23T02:27:33.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.992+0000 7fee67fff700 1 -- 172.21.15.71:0/264351068 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fee7019dd00 con 0x7fee5c06cce0 2024-04-23T02:27:33.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.994+0000 7fee75763700 1 -- 172.21.15.71:0/264351068 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee5c06cce0 msgr2=0x7fee5c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:33.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.994+0000 7fee75763700 1 --2- 172.21.15.71:0/264351068 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee5c06cce0 0x7fee5c06f1a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fee58005950 tx=0x7fee580058e0 comp rx=0 tx=0).stop 2024-04-23T02:27:33.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.994+0000 7fee75763700 1 -- 172.21.15.71:0/264351068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee700ffc20 msgr2=0x7fee70198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:33.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.994+0000 7fee75763700 1 --2- 172.21.15.71:0/264351068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee700ffc20 0x7fee70198990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fee6000d010 tx=0x7fee60007be0 comp rx=0 tx=0).stop 2024-04-23T02:27:33.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.994+0000 7fee75763700 1 -- 172.21.15.71:0/264351068 shutdown_connections 2024-04-23T02:27:33.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.994+0000 7fee75763700 1 --2- 172.21.15.71:0/264351068 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee5c06cce0 0x7fee5c06f1a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:33.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.994+0000 7fee75763700 1 --2- 172.21.15.71:0/264351068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee700ffc20 0x7fee70198990 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:33.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.994+0000 7fee75763700 1 --2- 172.21.15.71:0/264351068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee70105bd0 0x7fee70198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:33.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.994+0000 7fee75763700 1 -- 172.21.15.71:0/264351068 >> 172.21.15.71:0/264351068 conn(0x7fee70075670 msgr2=0x7fee70068740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:33.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.994+0000 7fee75763700 1 -- 172.21.15.71:0/264351068 shutdown_connections 2024-04-23T02:27:33.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:33.994+0000 7fee75763700 1 -- 172.21.15.71:0/264351068 wait complete. 2024-04-23T02:27:34.007 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:27:34.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.098+0000 7f48fee01700 1 -- 172.21.15.71:0/1833548862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48f8100ea0 msgr2=0x7f48f8101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:34.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.098+0000 7f48fee01700 1 --2- 172.21.15.71:0/1833548862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48f8100ea0 0x7f48f8101280 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009910 tx=0x7f48ec009c20 comp rx=0 tx=0).stop 2024-04-23T02:27:34.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.098+0000 7f48fee01700 1 -- 172.21.15.71:0/1833548862 shutdown_connections 2024-04-23T02:27:34.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.098+0000 7f48fee01700 1 --2- 172.21.15.71:0/1833548862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48f8101850 0x7f48f81058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:34.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.098+0000 7f48fee01700 1 --2- 172.21.15.71:0/1833548862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48f8100ea0 0x7f48f8101280 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:34.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.098+0000 7f48fee01700 1 -- 172.21.15.71:0/1833548862 >> 172.21.15.71:0/1833548862 conn(0x7f48f80fc750 msgr2=0x7f48f80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:34.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.098+0000 7f48fee01700 1 -- 172.21.15.71:0/1833548862 shutdown_connections 2024-04-23T02:27:34.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.098+0000 7f48fee01700 1 -- 172.21.15.71:0/1833548862 wait complete. 2024-04-23T02:27:34.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.099+0000 7f48fee01700 1 Processor -- start 2024-04-23T02:27:34.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.099+0000 7f48fee01700 1 -- start start 2024-04-23T02:27:34.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.099+0000 7f48fee01700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48f8100ea0 0x7f48f819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:34.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.099+0000 7f48fee01700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48f8101850 0x7f48f819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:34.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.099+0000 7f48fee01700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f819d130 con 0x7f48f8101850 2024-04-23T02:27:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.099+0000 7f48fee01700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f81965e0 con 0x7f48f8100ea0 2024-04-23T02:27:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.099+0000 7f48f7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48f8101850 0x7f48f819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.099+0000 7f48f7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48f8101850 0x7f48f819caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56552/0 (socket says 172.21.15.71:56552) 2024-04-23T02:27:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.099+0000 7f48f7fff700 1 -- 172.21.15.71:0/3870693734 learned_addr learned my addr 172.21.15.71:0/3870693734 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:27:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.099+0000 7f48fcb9d700 1 --2- 172.21.15.71:0/3870693734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48f8100ea0 0x7f48f819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.099+0000 7f48f7fff700 1 -- 172.21.15.71:0/3870693734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48f8100ea0 msgr2=0x7f48f819c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.099+0000 7f48f7fff700 1 --2- 172.21.15.71:0/3870693734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48f8100ea0 0x7f48f819c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.099+0000 7f48f7fff700 1 -- 172.21.15.71:0/3870693734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec0095f0 con 0x7f48f8101850 2024-04-23T02:27:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.100+0000 7f48f7fff700 1 --2- 172.21.15.71:0/3870693734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48f8101850 0x7f48f819caa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f48e400d900 tx=0x7f48e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.100+0000 7f48f5ffb700 1 -- 172.21.15.71:0/3870693734 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48e4009900 con 0x7f48f8101850 2024-04-23T02:27:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.100+0000 7f48f5ffb700 1 -- 172.21.15.71:0/3870693734 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f48e4010460 con 0x7f48f8101850 2024-04-23T02:27:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.100+0000 7f48fee01700 1 -- 172.21.15.71:0/3870693734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f81968c0 con 0x7f48f8101850 2024-04-23T02:27:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.100+0000 7f48f5ffb700 1 -- 172.21.15.71:0/3870693734 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48e400f5f0 con 0x7f48f8101850 2024-04-23T02:27:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.100+0000 7f48fee01700 1 -- 172.21.15.71:0/3870693734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f8196e10 con 0x7f48f8101850 2024-04-23T02:27:34.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.101+0000 7f48fee01700 1 -- 172.21.15.71:0/3870693734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f8109240 con 0x7f48f8101850 2024-04-23T02:27:34.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.101+0000 7f48f5ffb700 1 -- 172.21.15.71:0/3870693734 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f48e40105d0 con 0x7f48f8101850 2024-04-23T02:27:34.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.102+0000 7f48f5ffb700 1 --2- 172.21.15.71:0/3870693734 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f48e806cce0 0x7f48e806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:34.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.102+0000 7f48f5ffb700 1 -- 172.21.15.71:0/3870693734 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f48e408c420 con 0x7f48f8101850 2024-04-23T02:27:34.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.102+0000 7f48fcb9d700 1 --2- 172.21.15.71:0/3870693734 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f48e806cce0 0x7f48e806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:34.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.102+0000 7f48fcb9d700 1 --2- 172.21.15.71:0/3870693734 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f48e806cce0 0x7f48e806f1a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f48ec006010 tx=0x7f48ec01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:34.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.106+0000 7f48f5ffb700 1 -- 172.21.15.71:0/3870693734 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f48e4059ec0 con 0x7f48f8101850 2024-04-23T02:27:34.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.315+0000 7f48fee01700 1 -- 172.21.15.71:0/3870693734 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f48f8197030 con 0x7f48e806cce0 2024-04-23T02:27:34.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.316+0000 7f48f5ffb700 1 -- 172.21.15.71:0/3870693734 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f48f8197030 con 0x7f48e806cce0 2024-04-23T02:27:34.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.318+0000 7f48fee01700 1 -- 172.21.15.71:0/3870693734 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f48e806cce0 msgr2=0x7f48e806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:34.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.318+0000 7f48fee01700 1 --2- 172.21.15.71:0/3870693734 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f48e806cce0 0x7f48e806f1a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f48ec006010 tx=0x7f48ec01a040 comp rx=0 tx=0).stop 2024-04-23T02:27:34.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.318+0000 7f48fee01700 1 -- 172.21.15.71:0/3870693734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48f8101850 msgr2=0x7f48f819caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:34.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.318+0000 7f48fee01700 1 --2- 172.21.15.71:0/3870693734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48f8101850 0x7f48f819caa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f48e400d900 tx=0x7f48e400dcc0 comp rx=0 tx=0).stop 2024-04-23T02:27:34.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.318+0000 7f48fee01700 1 -- 172.21.15.71:0/3870693734 shutdown_connections 2024-04-23T02:27:34.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.318+0000 7f48fee01700 1 --2- 172.21.15.71:0/3870693734 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f48e806cce0 0x7f48e806f1a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:34.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.318+0000 7f48fee01700 1 --2- 172.21.15.71:0/3870693734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48f8100ea0 0x7f48f819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:34.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.318+0000 7f48fee01700 1 --2- 172.21.15.71:0/3870693734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48f8101850 0x7f48f819caa0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:34.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.318+0000 7f48fee01700 1 -- 172.21.15.71:0/3870693734 >> 172.21.15.71:0/3870693734 conn(0x7f48f80fc750 msgr2=0x7f48f80fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:34.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.318+0000 7f48fee01700 1 -- 172.21.15.71:0/3870693734 shutdown_connections 2024-04-23T02:27:34.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.318+0000 7f48fee01700 1 -- 172.21.15.71:0/3870693734 wait complete. 2024-04-23T02:27:34.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.423+0000 7f575aa5d700 1 -- 172.21.15.71:0/2518292076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5754101850 msgr2=0x7f57541058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:34.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.423+0000 7f575aa5d700 1 --2- 172.21.15.71:0/2518292076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5754101850 0x7f57541058a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5744009910 tx=0x7f5744009c20 comp rx=0 tx=0).stop 2024-04-23T02:27:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.423+0000 7f575aa5d700 1 -- 172.21.15.71:0/2518292076 shutdown_connections 2024-04-23T02:27:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.423+0000 7f575aa5d700 1 --2- 172.21.15.71:0/2518292076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5754101850 0x7f57541058a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.423+0000 7f575aa5d700 1 --2- 172.21.15.71:0/2518292076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5754100ea0 0x7f5754101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.423+0000 7f575aa5d700 1 -- 172.21.15.71:0/2518292076 >> 172.21.15.71:0/2518292076 conn(0x7f57540fc750 msgr2=0x7f57540feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.423+0000 7f575aa5d700 1 -- 172.21.15.71:0/2518292076 shutdown_connections 2024-04-23T02:27:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.423+0000 7f575aa5d700 1 -- 172.21.15.71:0/2518292076 wait complete. 2024-04-23T02:27:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.424+0000 7f575aa5d700 1 Processor -- start 2024-04-23T02:27:34.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.424+0000 7f575aa5d700 1 -- start start 2024-04-23T02:27:34.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.424+0000 7f575aa5d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5754100ea0 0x7f5754073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:34.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.424+0000 7f575aa5d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5754101850 0x7f57540735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:34.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.424+0000 7f575aa5d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5754073b10 con 0x7f5754100ea0 2024-04-23T02:27:34.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.424+0000 7f575aa5d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5754073c50 con 0x7f5754101850 2024-04-23T02:27:34.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.424+0000 7f5753fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5754100ea0 0x7f5754073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:34.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.424+0000 7f5753fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5754100ea0 0x7f5754073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56568/0 (socket says 172.21.15.71:56568) 2024-04-23T02:27:34.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.424+0000 7f5753fff700 1 -- 172.21.15.71:0/2052926641 learned_addr learned my addr 172.21.15.71:0/2052926641 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:27:34.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.424+0000 7f57537fe700 1 --2- 172.21.15.71:0/2052926641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5754101850 0x7f57540735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:34.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.425+0000 7f5753fff700 1 -- 172.21.15.71:0/2052926641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5754101850 msgr2=0x7f57540735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:34.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.425+0000 7f5753fff700 1 --2- 172.21.15.71:0/2052926641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5754101850 0x7f57540735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:34.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.425+0000 7f5753fff700 1 -- 172.21.15.71:0/2052926641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57440095f0 con 0x7f5754100ea0 2024-04-23T02:27:34.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.425+0000 7f5753fff700 1 --2- 172.21.15.71:0/2052926641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5754100ea0 0x7f5754073090 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f573c00e920 tx=0x7f573c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:34.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.425+0000 7f57517fa700 1 -- 172.21.15.71:0/2052926641 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f573c00c9d0 con 0x7f5754100ea0 2024-04-23T02:27:34.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.425+0000 7f575aa5d700 1 -- 172.21.15.71:0/2052926641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57541a2a20 con 0x7f5754100ea0 2024-04-23T02:27:34.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.425+0000 7f57517fa700 1 -- 172.21.15.71:0/2052926641 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f573c010430 con 0x7f5754100ea0 2024-04-23T02:27:34.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.425+0000 7f57517fa700 1 -- 172.21.15.71:0/2052926641 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f573c018650 con 0x7f5754100ea0 2024-04-23T02:27:34.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.425+0000 7f575aa5d700 1 -- 172.21.15.71:0/2052926641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57541a2dd0 con 0x7f5754100ea0 2024-04-23T02:27:34.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.426+0000 7f575aa5d700 1 -- 172.21.15.71:0/2052926641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57541092a0 con 0x7f5754100ea0 2024-04-23T02:27:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.426+0000 7f57517fa700 1 -- 172.21.15.71:0/2052926641 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f573c0105a0 con 0x7f5754100ea0 2024-04-23T02:27:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.427+0000 7f57517fa700 1 --2- 172.21.15.71:0/2052926641 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f574006cce0 0x7f574006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.427+0000 7f57517fa700 1 -- 172.21.15.71:0/2052926641 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f573c014070 con 0x7f5754100ea0 2024-04-23T02:27:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.427+0000 7f57537fe700 1 --2- 172.21.15.71:0/2052926641 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f574006cce0 0x7f574006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:34.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.427+0000 7f57537fe700 1 --2- 172.21.15.71:0/2052926641 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f574006cce0 0x7f574006f1a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5744009fd0 tx=0x7f5744005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:34.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.431+0000 7f57517fa700 1 -- 172.21.15.71:0/2052926641 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f573c052c50 con 0x7f5754100ea0 2024-04-23T02:27:34.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.632+0000 7f575aa5d700 1 -- 172.21.15.71:0/2052926641 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f57540611d0 con 0x7f574006cce0 2024-04-23T02:27:34.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.643+0000 7f57517fa700 1 -- 172.21.15.71:0/2052926641 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f57540611d0 con 0x7f574006cce0 2024-04-23T02:27:34.646 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:27:34.646 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (11m) 4m ago 22m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:27:34.646 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (22m) 4m ago 22m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:27:34.646 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (21m) 4m ago 21m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:27:34.646 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (9m) 4m ago 22m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:27:34.647 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (9m) 4m ago 21m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:27:34.647 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (10m) 4m ago 22m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:27:34.647 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (5m) 4m ago 16m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:27:34.647 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (5m) 4m ago 16m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:27:34.647 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (16m) 4m ago 16m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:27:34.647 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (4m) 4m ago 16m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:27:34.647 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (13m) 4m ago 24m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:27:34.647 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (12m) 4m ago 21m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:27:34.647 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (9m) 4m ago 24m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:27:34.647 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (9m) 4m ago 21m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:27:34.647 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (12m) 4m ago 22m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:27:34.647 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (12m) 4m ago 21m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:27:34.648 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (8m) 4m ago 20m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:27:34.648 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (8m) 4m ago 20m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:27:34.648 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (7m) 4m ago 19m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:27:34.648 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (6m) 4m ago 19m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:27:34.648 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (6m) 4m ago 18m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:27:34.648 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (5m) 4m ago 18m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:27:34.648 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (11m) 4m ago 22m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:27:34.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.645+0000 7f575aa5d700 1 -- 172.21.15.71:0/2052926641 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f574006cce0 msgr2=0x7f574006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:34.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.645+0000 7f575aa5d700 1 --2- 172.21.15.71:0/2052926641 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f574006cce0 0x7f574006f1a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5744009fd0 tx=0x7f5744005940 comp rx=0 tx=0).stop 2024-04-23T02:27:34.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.645+0000 7f575aa5d700 1 -- 172.21.15.71:0/2052926641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5754100ea0 msgr2=0x7f5754073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:34.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.645+0000 7f575aa5d700 1 --2- 172.21.15.71:0/2052926641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5754100ea0 0x7f5754073090 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f573c00e920 tx=0x7f573c00ec30 comp rx=0 tx=0).stop 2024-04-23T02:27:34.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.645+0000 7f575aa5d700 1 -- 172.21.15.71:0/2052926641 shutdown_connections 2024-04-23T02:27:34.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.645+0000 7f575aa5d700 1 --2- 172.21.15.71:0/2052926641 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f574006cce0 0x7f574006f1a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:34.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.645+0000 7f575aa5d700 1 --2- 172.21.15.71:0/2052926641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5754100ea0 0x7f5754073090 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:34.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.645+0000 7f575aa5d700 1 --2- 172.21.15.71:0/2052926641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5754101850 0x7f57540735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:34.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.645+0000 7f575aa5d700 1 -- 172.21.15.71:0/2052926641 >> 172.21.15.71:0/2052926641 conn(0x7f57540fc750 msgr2=0x7f57540fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:34.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.645+0000 7f575aa5d700 1 -- 172.21.15.71:0/2052926641 shutdown_connections 2024-04-23T02:27:34.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.645+0000 7f575aa5d700 1 -- 172.21.15.71:0/2052926641 wait complete. 2024-04-23T02:27:34.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:34 smithi071 ceph-mon[144544]: pgmap v427: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:34.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.749+0000 7f93af680700 1 -- 172.21.15.71:0/3690710377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93a80ffec0 msgr2=0x7f93a81048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:34.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.749+0000 7f93af680700 1 --2- 172.21.15.71:0/3690710377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93a80ffec0 0x7f93a81048f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9398009910 tx=0x7f9398009c20 comp rx=0 tx=0).stop 2024-04-23T02:27:34.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.749+0000 7f93af680700 1 -- 172.21.15.71:0/3690710377 shutdown_connections 2024-04-23T02:27:34.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.749+0000 7f93af680700 1 --2- 172.21.15.71:0/3690710377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93a80ffec0 0x7f93a81048f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:34.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.749+0000 7f93af680700 1 --2- 172.21.15.71:0/3690710377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93a80ff5a0 0x7f93a80ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:34.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.749+0000 7f93af680700 1 -- 172.21.15.71:0/3690710377 >> 172.21.15.71:0/3690710377 conn(0x7f93a8074e30 msgr2=0x7f93a8075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:34.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.749+0000 7f93af680700 1 -- 172.21.15.71:0/3690710377 shutdown_connections 2024-04-23T02:27:34.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.749+0000 7f93af680700 1 -- 172.21.15.71:0/3690710377 wait complete. 2024-04-23T02:27:34.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.749+0000 7f93af680700 1 Processor -- start 2024-04-23T02:27:34.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.749+0000 7f93af680700 1 -- start start 2024-04-23T02:27:34.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.749+0000 7f93af680700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93a80ff5a0 0x7f93a819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:34.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.750+0000 7f93af680700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93a80ffec0 0x7f93a819d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:34.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.750+0000 7f93af680700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a819d9f0 con 0x7f93a80ff5a0 2024-04-23T02:27:34.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.750+0000 7f93af680700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a81a1780 con 0x7f93a80ffec0 2024-04-23T02:27:34.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.750+0000 7f93ad41c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93a80ff5a0 0x7f93a819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:34.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.750+0000 7f93ad41c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93a80ff5a0 0x7f93a819cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56580/0 (socket says 172.21.15.71:56580) 2024-04-23T02:27:34.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.750+0000 7f93ad41c700 1 -- 172.21.15.71:0/2641359815 learned_addr learned my addr 172.21.15.71:0/2641359815 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:27:34.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.750+0000 7f93acc1b700 1 --2- 172.21.15.71:0/2641359815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93a80ffec0 0x7f93a819d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:34.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.750+0000 7f93ad41c700 1 -- 172.21.15.71:0/2641359815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93a80ffec0 msgr2=0x7f93a819d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:34.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.750+0000 7f93ad41c700 1 --2- 172.21.15.71:0/2641359815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93a80ffec0 0x7f93a819d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:34.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.750+0000 7f93ad41c700 1 -- 172.21.15.71:0/2641359815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93980095f0 con 0x7f93a80ff5a0 2024-04-23T02:27:34.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.750+0000 7f93ad41c700 1 --2- 172.21.15.71:0/2641359815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93a80ff5a0 0x7f93a819cdd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f93a400c8a0 tx=0x7f93a400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:34.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.750+0000 7f939e7fc700 1 -- 172.21.15.71:0/2641359815 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93a40078e0 con 0x7f93a80ff5a0 2024-04-23T02:27:34.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.750+0000 7f939e7fc700 1 -- 172.21.15.71:0/2641359815 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f93a400f450 con 0x7f93a80ff5a0 2024-04-23T02:27:34.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.750+0000 7f93af680700 1 -- 172.21.15.71:0/2641359815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a81a1a60 con 0x7f93a80ff5a0 2024-04-23T02:27:34.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.750+0000 7f939e7fc700 1 -- 172.21.15.71:0/2641359815 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93a400e5e0 con 0x7f93a80ff5a0 2024-04-23T02:27:34.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.750+0000 7f93af680700 1 -- 172.21.15.71:0/2641359815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a81a1fb0 con 0x7f93a80ff5a0 2024-04-23T02:27:34.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.751+0000 7f93af680700 1 -- 172.21.15.71:0/2641359815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a8102050 con 0x7f93a80ff5a0 2024-04-23T02:27:34.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.752+0000 7f939e7fc700 1 -- 172.21.15.71:0/2641359815 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f93a400f5c0 con 0x7f93a80ff5a0 2024-04-23T02:27:34.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.752+0000 7f939e7fc700 1 --2- 172.21.15.71:0/2641359815 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f939406cce0 0x7f939406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:34.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.752+0000 7f939e7fc700 1 -- 172.21.15.71:0/2641359815 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f93a408c480 con 0x7f93a80ff5a0 2024-04-23T02:27:34.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.752+0000 7f93acc1b700 1 --2- 172.21.15.71:0/2641359815 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f939406cce0 0x7f939406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:34.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.753+0000 7f93acc1b700 1 --2- 172.21.15.71:0/2641359815 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f939406cce0 0x7f939406f1a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f93a819e3f0 tx=0x7f939801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:34.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:34.757+0000 7f939e7fc700 1 -- 172.21.15.71:0/2641359815 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f93a4056570 con 0x7f93a80ff5a0 2024-04-23T02:27:34.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:34 smithi192 ceph-mon[108617]: pgmap v427: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:35.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.032+0000 7f93af680700 1 -- 172.21.15.71:0/2641359815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f93a81a2280 con 0x7f93a80ff5a0 2024-04-23T02:27:35.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.032+0000 7f939e7fc700 1 -- 172.21.15.71:0/2641359815 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f93a4059b90 con 0x7f93a80ff5a0 2024-04-23T02:27:35.035 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:27:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:27:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:27:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:27:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:27:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:27:35.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:27:35.037 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:27:35.037 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:27:35.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:27:35.037 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:27:35.037 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:27:35.037 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:27:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:27:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:27:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:27:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:27:35.038 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:27:35.038 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:27:35.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.034+0000 7f93af680700 1 -- 172.21.15.71:0/2641359815 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f939406cce0 msgr2=0x7f939406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:35.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.034+0000 7f93af680700 1 --2- 172.21.15.71:0/2641359815 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f939406cce0 0x7f939406f1a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f93a819e3f0 tx=0x7f939801a040 comp rx=0 tx=0).stop 2024-04-23T02:27:35.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.034+0000 7f93af680700 1 -- 172.21.15.71:0/2641359815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93a80ff5a0 msgr2=0x7f93a819cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:35.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.034+0000 7f93af680700 1 --2- 172.21.15.71:0/2641359815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93a80ff5a0 0x7f93a819cdd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f93a400c8a0 tx=0x7f93a400cbb0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.034+0000 7f93af680700 1 -- 172.21.15.71:0/2641359815 shutdown_connections 2024-04-23T02:27:35.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.034+0000 7f93af680700 1 --2- 172.21.15.71:0/2641359815 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f939406cce0 0x7f939406f1a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.034+0000 7f93af680700 1 --2- 172.21.15.71:0/2641359815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93a80ff5a0 0x7f93a819cdd0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.034+0000 7f93af680700 1 --2- 172.21.15.71:0/2641359815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93a80ffec0 0x7f93a819d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.034+0000 7f93af680700 1 -- 172.21.15.71:0/2641359815 >> 172.21.15.71:0/2641359815 conn(0x7f93a8074e30 msgr2=0x7f93a8111b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:35.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.034+0000 7f93af680700 1 -- 172.21.15.71:0/2641359815 shutdown_connections 2024-04-23T02:27:35.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.034+0000 7f93af680700 1 -- 172.21.15.71:0/2641359815 wait complete. 2024-04-23T02:27:35.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.136+0000 7f7fc6ccc700 1 -- 172.21.15.71:0/3219957612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc0100e90 msgr2=0x7f7fc0101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:35.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.136+0000 7f7fc6ccc700 1 --2- 172.21.15.71:0/3219957612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc0100e90 0x7f7fc0101270 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009910 tx=0x7f7fb4009c20 comp rx=0 tx=0).stop 2024-04-23T02:27:35.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.136+0000 7f7fc6ccc700 1 -- 172.21.15.71:0/3219957612 shutdown_connections 2024-04-23T02:27:35.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.136+0000 7f7fc6ccc700 1 --2- 172.21.15.71:0/3219957612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fc0101840 0x7f7fc0105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.136+0000 7f7fc6ccc700 1 --2- 172.21.15.71:0/3219957612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc0100e90 0x7f7fc0101270 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.136+0000 7f7fc6ccc700 1 -- 172.21.15.71:0/3219957612 >> 172.21.15.71:0/3219957612 conn(0x7f7fc00fc740 msgr2=0x7f7fc00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:35.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.136+0000 7f7fc6ccc700 1 -- 172.21.15.71:0/3219957612 shutdown_connections 2024-04-23T02:27:35.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.136+0000 7f7fc6ccc700 1 -- 172.21.15.71:0/3219957612 wait complete. 2024-04-23T02:27:35.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.137+0000 7f7fc6ccc700 1 Processor -- start 2024-04-23T02:27:35.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.137+0000 7f7fc6ccc700 1 -- start start 2024-04-23T02:27:35.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.137+0000 7f7fc6ccc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc0100e90 0x7f7fc019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:35.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.137+0000 7f7fc6ccc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fc0101840 0x7f7fc019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:35.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.137+0000 7f7fc6ccc700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc019d190 con 0x7f7fc0100e90 2024-04-23T02:27:35.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.137+0000 7f7fc6ccc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc0196640 con 0x7f7fc0101840 2024-04-23T02:27:35.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.138+0000 7f7fc4a68700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc0100e90 0x7f7fc019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:35.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.138+0000 7f7fc4a68700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc0100e90 0x7f7fc019c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56590/0 (socket says 172.21.15.71:56590) 2024-04-23T02:27:35.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.138+0000 7f7fc4a68700 1 -- 172.21.15.71:0/2772691903 learned_addr learned my addr 172.21.15.71:0/2772691903 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:27:35.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.138+0000 7f7fbffff700 1 --2- 172.21.15.71:0/2772691903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fc0101840 0x7f7fc019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:35.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.138+0000 7f7fc4a68700 1 -- 172.21.15.71:0/2772691903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fc0101840 msgr2=0x7f7fc019cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:35.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.138+0000 7f7fc4a68700 1 --2- 172.21.15.71:0/2772691903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fc0101840 0x7f7fc019cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.138+0000 7f7fc4a68700 1 -- 172.21.15.71:0/2772691903 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb40095f0 con 0x7f7fc0100e90 2024-04-23T02:27:35.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.138+0000 7f7fc4a68700 1 --2- 172.21.15.71:0/2772691903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc0100e90 0x7f7fc019c5c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4006010 tx=0x7f7fb400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:35.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.138+0000 7f7fbdffb700 1 -- 172.21.15.71:0/2772691903 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fb401d070 con 0x7f7fc0100e90 2024-04-23T02:27:35.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.138+0000 7f7fbdffb700 1 -- 172.21.15.71:0/2772691903 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7fb4005710 con 0x7f7fc0100e90 2024-04-23T02:27:35.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.138+0000 7f7fc6ccc700 1 -- 172.21.15.71:0/2772691903 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc01968c0 con 0x7f7fc0100e90 2024-04-23T02:27:35.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.138+0000 7f7fbdffb700 1 -- 172.21.15.71:0/2772691903 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fb400fe20 con 0x7f7fc0100e90 2024-04-23T02:27:35.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.138+0000 7f7fc6ccc700 1 -- 172.21.15.71:0/2772691903 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc0196db0 con 0x7f7fc0100e90 2024-04-23T02:27:35.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.139+0000 7f7fc6ccc700 1 -- 172.21.15.71:0/2772691903 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc0109230 con 0x7f7fc0100e90 2024-04-23T02:27:35.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.140+0000 7f7fbdffb700 1 -- 172.21.15.71:0/2772691903 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7fb4003a60 con 0x7f7fc0100e90 2024-04-23T02:27:35.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.140+0000 7f7fbdffb700 1 --2- 172.21.15.71:0/2772691903 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7fb006cd30 0x7f7fb006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:35.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.140+0000 7f7fbdffb700 1 -- 172.21.15.71:0/2772691903 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7fb408d7f0 con 0x7f7fc0100e90 2024-04-23T02:27:35.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.140+0000 7f7fbffff700 1 --2- 172.21.15.71:0/2772691903 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7fb006cd30 0x7f7fb006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:35.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.141+0000 7f7fbffff700 1 --2- 172.21.15.71:0/2772691903 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7fb006cd30 0x7f7fb006f1f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00fd000 tx=0x7f7fac00b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:35.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.145+0000 7f7fbdffb700 1 -- 172.21.15.71:0/2772691903 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7fb405b250 con 0x7f7fc0100e90 2024-04-23T02:27:35.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.376+0000 7f7fc6ccc700 1 -- 172.21.15.71:0/2772691903 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7fc004f2e0 con 0x7f7fc0100e90 2024-04-23T02:27:35.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.377+0000 7f7fbdffb700 1 -- 172.21.15.71:0/2772691903 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f7fb405ade0 con 0x7f7fc0100e90 2024-04-23T02:27:35.379 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:27:35.380 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:27:35.380 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:27:35.380 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:27:35.380 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:27:35.380 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:27:35.380 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:27:35.380 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:27:35.380 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:27:35.380 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:27:35.380 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:27:35.380 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:27:35.380 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:27:35.380 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:27:35.381 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:27:35.381 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:27:35.381 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:27:35.381 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:27:35.381 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:27:35.381 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:27:35.381 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:27:35.381 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:27:35.381 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:27:35.381 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:27:35.381 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:27:35.381 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:27:35.381 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:27:35.381 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:27:35.381 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:27:35.381 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:27:35.382 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:27:35.382 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:27:35.382 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:27:35.382 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:27:35.382 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:27:35.382 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:27:35.382 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:27:35.382 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:27:35.382 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:27:35.382 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:27:35.382 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:27:35.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.379+0000 7f7fc6ccc700 1 -- 172.21.15.71:0/2772691903 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7fb006cd30 msgr2=0x7f7fb006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:35.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.379+0000 7f7fc6ccc700 1 --2- 172.21.15.71:0/2772691903 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7fb006cd30 0x7f7fb006f1f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00fd000 tx=0x7f7fac00b410 comp rx=0 tx=0).stop 2024-04-23T02:27:35.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.379+0000 7f7fc6ccc700 1 -- 172.21.15.71:0/2772691903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc0100e90 msgr2=0x7f7fc019c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:35.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.379+0000 7f7fc6ccc700 1 --2- 172.21.15.71:0/2772691903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc0100e90 0x7f7fc019c5c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4006010 tx=0x7f7fb400bd70 comp rx=0 tx=0).stop 2024-04-23T02:27:35.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.379+0000 7f7fc6ccc700 1 -- 172.21.15.71:0/2772691903 shutdown_connections 2024-04-23T02:27:35.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.379+0000 7f7fc6ccc700 1 --2- 172.21.15.71:0/2772691903 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7fb006cd30 0x7f7fb006f1f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.379+0000 7f7fc6ccc700 1 --2- 172.21.15.71:0/2772691903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fc0100e90 0x7f7fc019c5c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.379+0000 7f7fc6ccc700 1 --2- 172.21.15.71:0/2772691903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fc0101840 0x7f7fc019cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.379+0000 7f7fc6ccc700 1 -- 172.21.15.71:0/2772691903 >> 172.21.15.71:0/2772691903 conn(0x7f7fc00fc740 msgr2=0x7f7fc00fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:35.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.379+0000 7f7fc6ccc700 1 -- 172.21.15.71:0/2772691903 shutdown_connections 2024-04-23T02:27:35.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.379+0000 7f7fc6ccc700 1 -- 172.21.15.71:0/2772691903 wait complete. 2024-04-23T02:27:35.384 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:27:35.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.486+0000 7f9dbf425700 1 -- 172.21.15.71:0/1509028475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9db8068ea0 msgr2=0x7f9db810d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:35.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.486+0000 7f9dbf425700 1 --2- 172.21.15.71:0/1509028475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9db8068ea0 0x7f9db810d290 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009910 tx=0x7f9db4009c20 comp rx=0 tx=0).stop 2024-04-23T02:27:35.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.486+0000 7f9dbf425700 1 -- 172.21.15.71:0/1509028475 shutdown_connections 2024-04-23T02:27:35.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.486+0000 7f9dbf425700 1 --2- 172.21.15.71:0/1509028475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9db8068ea0 0x7f9db810d290 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.486+0000 7f9dbf425700 1 --2- 172.21.15.71:0/1509028475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9db8068580 0x7f9db8068960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.486+0000 7f9dbf425700 1 -- 172.21.15.71:0/1509028475 >> 172.21.15.71:0/1509028475 conn(0x7f9db8075670 msgr2=0x7f9db8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:35.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.486+0000 7f9dbf425700 1 -- 172.21.15.71:0/1509028475 shutdown_connections 2024-04-23T02:27:35.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.486+0000 7f9dbf425700 1 -- 172.21.15.71:0/1509028475 wait complete. 2024-04-23T02:27:35.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.487+0000 7f9dbf425700 1 Processor -- start 2024-04-23T02:27:35.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.487+0000 7f9dbf425700 1 -- start start 2024-04-23T02:27:35.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.487+0000 7f9dbf425700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9db8068580 0x7f9db819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:35.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.487+0000 7f9dbf425700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9db8068ea0 0x7f9db819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.487+0000 7f9dbf425700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db819f3d0 con 0x7f9db8068ea0 2024-04-23T02:27:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.487+0000 7f9dbf425700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db8198880 con 0x7f9db8068580 2024-04-23T02:27:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.487+0000 7f9dbc9c0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9db8068ea0 0x7f9db819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.487+0000 7f9dbc9c0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9db8068ea0 0x7f9db819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45804/0 (socket says 172.21.15.71:45804) 2024-04-23T02:27:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.487+0000 7f9dbc9c0700 1 -- 172.21.15.71:0/1642003348 learned_addr learned my addr 172.21.15.71:0/1642003348 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:27:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.487+0000 7f9dbd1c1700 1 --2- 172.21.15.71:0/1642003348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9db8068580 0x7f9db819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.487+0000 7f9dbc9c0700 1 -- 172.21.15.71:0/1642003348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9db8068580 msgr2=0x7f9db819e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.487+0000 7f9dbc9c0700 1 --2- 172.21.15.71:0/1642003348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9db8068580 0x7f9db819e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.487+0000 7f9dbc9c0700 1 -- 172.21.15.71:0/1642003348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db40095f0 con 0x7f9db8068ea0 2024-04-23T02:27:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.487+0000 7f9dbc9c0700 1 --2- 172.21.15.71:0/1642003348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9db8068ea0 0x7f9db819ed40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9db4006010 tx=0x7f9db400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.488+0000 7f9dae7fc700 1 -- 172.21.15.71:0/1642003348 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9db401d070 con 0x7f9db8068ea0 2024-04-23T02:27:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.488+0000 7f9dae7fc700 1 -- 172.21.15.71:0/1642003348 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9db400f460 con 0x7f9db8068ea0 2024-04-23T02:27:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.488+0000 7f9dbf425700 1 -- 172.21.15.71:0/1642003348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db8198b00 con 0x7f9db8068ea0 2024-04-23T02:27:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.488+0000 7f9dae7fc700 1 -- 172.21.15.71:0/1642003348 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9db4021620 con 0x7f9db8068ea0 2024-04-23T02:27:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.488+0000 7f9dbf425700 1 -- 172.21.15.71:0/1642003348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db8198ff0 con 0x7f9db8068ea0 2024-04-23T02:27:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.489+0000 7f9dbf425700 1 -- 172.21.15.71:0/1642003348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db810aa00 con 0x7f9db8068ea0 2024-04-23T02:27:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.489+0000 7f9dae7fc700 1 -- 172.21.15.71:0/1642003348 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9db400fa40 con 0x7f9db8068ea0 2024-04-23T02:27:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.489+0000 7f9dae7fc700 1 --2- 172.21.15.71:0/1642003348 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9da406cce0 0x7f9da406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.489+0000 7f9dae7fc700 1 -- 172.21.15.71:0/1642003348 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9db408d6c0 con 0x7f9db8068ea0 2024-04-23T02:27:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.489+0000 7f9dbd1c1700 1 --2- 172.21.15.71:0/1642003348 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9da406cce0 0x7f9da406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.490+0000 7f9dbd1c1700 1 --2- 172.21.15.71:0/1642003348 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9da406cce0 0x7f9da406f1a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9da8005950 tx=0x7f9da80058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.494+0000 7f9dae7fc700 1 -- 172.21.15.71:0/1642003348 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9db405b240 con 0x7f9db8068ea0 2024-04-23T02:27:35.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:35 smithi192 ceph-mon[108617]: from='client.34622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:27:35.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:35 smithi192 ceph-mon[108617]: from='client.34626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:27:35.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:27:35.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:35 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2641359815' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:27:35.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:35 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2772691903' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:27:35.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.702+0000 7f9dbf425700 1 -- 172.21.15.71:0/1642003348 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9db8199e00 con 0x7f9da406cce0 2024-04-23T02:27:35.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.703+0000 7f9dae7fc700 1 -- 172.21.15.71:0/1642003348 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9db8199e00 con 0x7f9da406cce0 2024-04-23T02:27:35.706 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:27:35.706 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:27:35.706 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:27:35.706 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:27:35.706 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:27:35.706 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:27:35.706 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:27:35.706 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:27:35.706 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:27:35.707 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:27:35.707 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:27:35.707 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:27:35.707 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:27:35.707 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:27:35.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.705+0000 7f9dbf425700 1 -- 172.21.15.71:0/1642003348 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9da406cce0 msgr2=0x7f9da406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:35.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.705+0000 7f9dbf425700 1 --2- 172.21.15.71:0/1642003348 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9da406cce0 0x7f9da406f1a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9da8005950 tx=0x7f9da80058e0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.705+0000 7f9dbf425700 1 -- 172.21.15.71:0/1642003348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9db8068ea0 msgr2=0x7f9db819ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:35.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.705+0000 7f9dbf425700 1 --2- 172.21.15.71:0/1642003348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9db8068ea0 0x7f9db819ed40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9db4006010 tx=0x7f9db400be20 comp rx=0 tx=0).stop 2024-04-23T02:27:35.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.705+0000 7f9dbf425700 1 -- 172.21.15.71:0/1642003348 shutdown_connections 2024-04-23T02:27:35.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.705+0000 7f9dbf425700 1 --2- 172.21.15.71:0/1642003348 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9da406cce0 0x7f9da406f1a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.705+0000 7f9dbf425700 1 --2- 172.21.15.71:0/1642003348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9db8068580 0x7f9db819e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.705+0000 7f9dbf425700 1 --2- 172.21.15.71:0/1642003348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9db8068ea0 0x7f9db819ed40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.705+0000 7f9dbf425700 1 -- 172.21.15.71:0/1642003348 >> 172.21.15.71:0/1642003348 conn(0x7f9db8075670 msgr2=0x7f9db80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:35.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.706+0000 7f9dbf425700 1 -- 172.21.15.71:0/1642003348 shutdown_connections 2024-04-23T02:27:35.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.706+0000 7f9dbf425700 1 -- 172.21.15.71:0/1642003348 wait complete. 2024-04-23T02:27:35.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:35 smithi071 ceph-mon[144544]: from='client.34622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:27:35.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:35 smithi071 ceph-mon[144544]: from='client.34626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:27:35.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:27:35.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:35 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2641359815' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:27:35.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:35 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2772691903' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:27:35.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.809+0000 7fa6d2911700 1 -- 172.21.15.71:0/4009966739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6cc102700 msgr2=0x7fa6cc10abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:35.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.809+0000 7fa6d2911700 1 --2- 172.21.15.71:0/4009966739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6cc102700 0x7fa6cc10abf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009910 tx=0x7fa6bc009c20 comp rx=0 tx=0).stop 2024-04-23T02:27:35.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.809+0000 7fa6d2911700 1 -- 172.21.15.71:0/4009966739 shutdown_connections 2024-04-23T02:27:35.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.809+0000 7fa6d2911700 1 --2- 172.21.15.71:0/4009966739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6cc102700 0x7fa6cc10abf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.809+0000 7fa6d2911700 1 --2- 172.21.15.71:0/4009966739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6cc101de0 0x7fa6cc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.809+0000 7fa6d2911700 1 -- 172.21.15.71:0/4009966739 >> 172.21.15.71:0/4009966739 conn(0x7fa6cc0fb610 msgr2=0x7fa6cc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:35.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.809+0000 7fa6d2911700 1 -- 172.21.15.71:0/4009966739 shutdown_connections 2024-04-23T02:27:35.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.809+0000 7fa6d2911700 1 -- 172.21.15.71:0/4009966739 wait complete. 2024-04-23T02:27:35.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.809+0000 7fa6d2911700 1 Processor -- start 2024-04-23T02:27:35.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.809+0000 7fa6d2911700 1 -- start start 2024-04-23T02:27:35.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.810+0000 7fa6d2911700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6cc101de0 0x7fa6cc19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:35.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.810+0000 7fa6d2911700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6cc102700 0x7fa6cc19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:35.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.810+0000 7fa6d2911700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc19d190 con 0x7fa6cc101de0 2024-04-23T02:27:35.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.810+0000 7fa6d2911700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc196640 con 0x7fa6cc102700 2024-04-23T02:27:35.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.810+0000 7fa6cbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6cc101de0 0x7fa6cc19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:35.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.810+0000 7fa6cbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6cc101de0 0x7fa6cc19c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45818/0 (socket says 172.21.15.71:45818) 2024-04-23T02:27:35.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.810+0000 7fa6cbfff700 1 -- 172.21.15.71:0/3273247106 learned_addr learned my addr 172.21.15.71:0/3273247106 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:27:35.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.810+0000 7fa6cb7fe700 1 --2- 172.21.15.71:0/3273247106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6cc102700 0x7fa6cc19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:35.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.810+0000 7fa6cbfff700 1 -- 172.21.15.71:0/3273247106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6cc102700 msgr2=0x7fa6cc19cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:35.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.810+0000 7fa6cbfff700 1 --2- 172.21.15.71:0/3273247106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6cc102700 0x7fa6cc19cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:35.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.810+0000 7fa6cbfff700 1 -- 172.21.15.71:0/3273247106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc0095f0 con 0x7fa6cc101de0 2024-04-23T02:27:35.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.810+0000 7fa6cb7fe700 1 --2- 172.21.15.71:0/3273247106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6cc102700 0x7fa6cc19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:27:35.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.810+0000 7fa6cbfff700 1 --2- 172.21.15.71:0/3273247106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6cc101de0 0x7fa6cc19c5c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400e920 tx=0x7fa6b400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:35.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.811+0000 7fa6c97fa700 1 -- 172.21.15.71:0/3273247106 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6b4009790 con 0x7fa6cc101de0 2024-04-23T02:27:35.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.811+0000 7fa6c97fa700 1 -- 172.21.15.71:0/3273247106 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa6b400cba0 con 0x7fa6cc101de0 2024-04-23T02:27:35.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.811+0000 7fa6d2911700 1 -- 172.21.15.71:0/3273247106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6cc196920 con 0x7fa6cc101de0 2024-04-23T02:27:35.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.811+0000 7fa6c97fa700 1 -- 172.21.15.71:0/3273247106 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6b4018650 con 0x7fa6cc101de0 2024-04-23T02:27:35.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.811+0000 7fa6d2911700 1 -- 172.21.15.71:0/3273247106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6cc196e70 con 0x7fa6cc101de0 2024-04-23T02:27:35.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.811+0000 7fa6d2911700 1 -- 172.21.15.71:0/3273247106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6cc1082f0 con 0x7fa6cc101de0 2024-04-23T02:27:35.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.812+0000 7fa6c97fa700 1 -- 172.21.15.71:0/3273247106 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa6b400cd10 con 0x7fa6cc101de0 2024-04-23T02:27:35.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.812+0000 7fa6c97fa700 1 --2- 172.21.15.71:0/3273247106 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa6b806cce0 0x7fa6b806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:27:35.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.812+0000 7fa6c97fa700 1 -- 172.21.15.71:0/3273247106 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa6b4014070 con 0x7fa6cc101de0 2024-04-23T02:27:35.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.812+0000 7fa6cb7fe700 1 --2- 172.21.15.71:0/3273247106 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa6b806cce0 0x7fa6b806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:27:35.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.813+0000 7fa6cb7fe700 1 --2- 172.21.15.71:0/3273247106 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa6b806cce0 0x7fa6b806f1a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009fd0 tx=0x7fa6bc005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:27:35.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:35.817+0000 7fa6c97fa700 1 -- 172.21.15.71:0/3273247106 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa6b4052c50 con 0x7fa6cc101de0 2024-04-23T02:27:36.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:36.088+0000 7fa6d2911700 1 -- 172.21.15.71:0/3273247106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa6cc197090 con 0x7fa6cc101de0 2024-04-23T02:27:36.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:36.088+0000 7fa6c97fa700 1 -- 172.21.15.71:0/3273247106 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fa6b405a1c0 con 0x7fa6cc101de0 2024-04-23T02:27:36.091 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:27:36.091 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:27:36.091 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:27:36.091 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:27:36.091 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:27:36.091 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:27:36.091 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:27:36.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:36.090+0000 7fa6d2911700 1 -- 172.21.15.71:0/3273247106 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa6b806cce0 msgr2=0x7fa6b806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:36.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:36.090+0000 7fa6d2911700 1 --2- 172.21.15.71:0/3273247106 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa6b806cce0 0x7fa6b806f1a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009fd0 tx=0x7fa6bc005940 comp rx=0 tx=0).stop 2024-04-23T02:27:36.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:36.090+0000 7fa6d2911700 1 -- 172.21.15.71:0/3273247106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6cc101de0 msgr2=0x7fa6cc19c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:27:36.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:36.090+0000 7fa6d2911700 1 --2- 172.21.15.71:0/3273247106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6cc101de0 0x7fa6cc19c5c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400e920 tx=0x7fa6b400ece0 comp rx=0 tx=0).stop 2024-04-23T02:27:36.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:36.090+0000 7fa6d2911700 1 -- 172.21.15.71:0/3273247106 shutdown_connections 2024-04-23T02:27:36.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:36.090+0000 7fa6d2911700 1 --2- 172.21.15.71:0/3273247106 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa6b806cce0 0x7fa6b806f1a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:36.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:36.090+0000 7fa6d2911700 1 --2- 172.21.15.71:0/3273247106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6cc101de0 0x7fa6cc19c5c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:36.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:36.090+0000 7fa6d2911700 1 --2- 172.21.15.71:0/3273247106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6cc102700 0x7fa6cc19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:27:36.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:36.090+0000 7fa6d2911700 1 -- 172.21.15.71:0/3273247106 >> 172.21.15.71:0/3273247106 conn(0x7fa6cc0fb610 msgr2=0x7fa6cc105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:27:36.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:36.090+0000 7fa6d2911700 1 -- 172.21.15.71:0/3273247106 shutdown_connections 2024-04-23T02:27:36.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:27:36.090+0000 7fa6d2911700 1 -- 172.21.15.71:0/3273247106 wait complete. 2024-04-23T02:27:36.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:36 smithi071 ceph-mon[144544]: from='client.34630 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:27:36.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:36 smithi071 ceph-mon[144544]: pgmap v428: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:36.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:36 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3273247106' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:27:36.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:36 smithi192 ceph-mon[108617]: from='client.34630 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:27:36.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:36 smithi192 ceph-mon[108617]: pgmap v428: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:36.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:36 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3273247106' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:27:37.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:37 smithi071 ceph-mon[144544]: from='client.34642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:27:37.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:37 smithi192 ceph-mon[108617]: from='client.34642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:27:38.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:38 smithi071 ceph-mon[144544]: pgmap v429: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:38.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:38 smithi192 ceph-mon[108617]: pgmap v429: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:40.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:40 smithi071 ceph-mon[144544]: pgmap v430: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:40.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:40 smithi192 ceph-mon[108617]: pgmap v430: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:42.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:42 smithi071 ceph-mon[144544]: pgmap v431: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:42.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:42 smithi192 ceph-mon[108617]: pgmap v431: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:44.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:44 smithi071 ceph-mon[144544]: pgmap v432: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:44.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:44 smithi192 ceph-mon[108617]: pgmap v432: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:46.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:46 smithi071 ceph-mon[144544]: pgmap v433: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:46.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:46 smithi192 ceph-mon[108617]: pgmap v433: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:48.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:48 smithi071 ceph-mon[144544]: pgmap v434: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:48.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:48 smithi192 ceph-mon[108617]: pgmap v434: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:50.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:50 smithi071 ceph-mon[144544]: pgmap v435: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:50.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:27:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:50 smithi192 ceph-mon[108617]: pgmap v435: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:27:52.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:52 smithi071 ceph-mon[144544]: pgmap v436: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:52.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:52 smithi192 ceph-mon[108617]: pgmap v436: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:54.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:54 smithi071 ceph-mon[144544]: pgmap v437: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:54.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:54 smithi192 ceph-mon[108617]: pgmap v437: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:56.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:56 smithi071 ceph-mon[144544]: pgmap v438: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:56.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:56 smithi192 ceph-mon[108617]: pgmap v438: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:58.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:27:58 smithi071 ceph-mon[144544]: pgmap v439: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:27:58.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:27:58 smithi192 ceph-mon[108617]: pgmap v439: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:00.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:00 smithi071 ceph-mon[144544]: pgmap v440: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:00.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:00 smithi192 ceph-mon[108617]: pgmap v440: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:02.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:02 smithi071 ceph-mon[144544]: pgmap v441: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:02.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:02 smithi192 ceph-mon[108617]: pgmap v441: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:04.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:04 smithi071 ceph-mon[144544]: pgmap v442: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:04.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:04 smithi192 ceph-mon[108617]: pgmap v442: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:05.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:28:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:28:06.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.206+0000 7ff15ce4f700 1 -- 172.21.15.71:0/320281887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580ffb40 msgr2=0x7ff1580fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:06.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.206+0000 7ff15ce4f700 1 --2- 172.21.15.71:0/320281887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580ffb40 0x7ff1580fffc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff14c009910 tx=0x7ff14c009c20 comp rx=0 tx=0).stop 2024-04-23T02:28:06.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.206+0000 7ff15ce4f700 1 -- 172.21.15.71:0/320281887 shutdown_connections 2024-04-23T02:28:06.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.206+0000 7ff15ce4f700 1 --2- 172.21.15.71:0/320281887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580ffb40 0x7ff1580fffc0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:06.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.206+0000 7ff15ce4f700 1 --2- 172.21.15.71:0/320281887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff158105bf0 0x7ff158105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:06.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.206+0000 7ff15ce4f700 1 -- 172.21.15.71:0/320281887 >> 172.21.15.71:0/320281887 conn(0x7ff1580fb610 msgr2=0x7ff1580fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:06.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.206+0000 7ff15ce4f700 1 -- 172.21.15.71:0/320281887 shutdown_connections 2024-04-23T02:28:06.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.206+0000 7ff15ce4f700 1 -- 172.21.15.71:0/320281887 wait complete. 2024-04-23T02:28:06.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.207+0000 7ff15ce4f700 1 Processor -- start 2024-04-23T02:28:06.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.207+0000 7ff15ce4f700 1 -- start start 2024-04-23T02:28:06.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.207+0000 7ff15ce4f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580ffb40 0x7ff158197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:06.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.207+0000 7ff15ce4f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff158105bf0 0x7ff158197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:06.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.207+0000 7ff15ce4f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1581980d0 con 0x7ff1580ffb40 2024-04-23T02:28:06.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.207+0000 7ff15ce4f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff158198240 con 0x7ff158105bf0 2024-04-23T02:28:06.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.207+0000 7ff15659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580ffb40 0x7ff158197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:06.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.207+0000 7ff15659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580ffb40 0x7ff158197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38322/0 (socket says 172.21.15.71:38322) 2024-04-23T02:28:06.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.207+0000 7ff15659c700 1 -- 172.21.15.71:0/1330195020 learned_addr learned my addr 172.21.15.71:0/1330195020 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:28:06.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.208+0000 7ff155d9b700 1 --2- 172.21.15.71:0/1330195020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff158105bf0 0x7ff158197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:06.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.208+0000 7ff15659c700 1 -- 172.21.15.71:0/1330195020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff158105bf0 msgr2=0x7ff158197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:06.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.208+0000 7ff15659c700 1 --2- 172.21.15.71:0/1330195020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff158105bf0 0x7ff158197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:06.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.208+0000 7ff15659c700 1 -- 172.21.15.71:0/1330195020 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14c0095f0 con 0x7ff1580ffb40 2024-04-23T02:28:06.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.208+0000 7ff15659c700 1 --2- 172.21.15.71:0/1330195020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580ffb40 0x7ff158197570 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff14800d8d0 tx=0x7ff14800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:06.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.208+0000 7ff1477fe700 1 -- 172.21.15.71:0/1330195020 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff148009940 con 0x7ff1580ffb40 2024-04-23T02:28:06.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.208+0000 7ff1477fe700 1 -- 172.21.15.71:0/1330195020 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff148010460 con 0x7ff1580ffb40 2024-04-23T02:28:06.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.208+0000 7ff1477fe700 1 -- 172.21.15.71:0/1330195020 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff14800f5f0 con 0x7ff1580ffb40 2024-04-23T02:28:06.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.208+0000 7ff15ce4f700 1 -- 172.21.15.71:0/1330195020 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1581a05e0 con 0x7ff1580ffb40 2024-04-23T02:28:06.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.208+0000 7ff15ce4f700 1 -- 172.21.15.71:0/1330195020 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1581a0ad0 con 0x7ff1580ffb40 2024-04-23T02:28:06.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.209+0000 7ff15ce4f700 1 -- 172.21.15.71:0/1330195020 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1581080a0 con 0x7ff1580ffb40 2024-04-23T02:28:06.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.209+0000 7ff1477fe700 1 -- 172.21.15.71:0/1330195020 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff1480105d0 con 0x7ff1580ffb40 2024-04-23T02:28:06.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.210+0000 7ff1477fe700 1 --2- 172.21.15.71:0/1330195020 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff14006cce0 0x7ff14006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:06.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.210+0000 7ff1477fe700 1 -- 172.21.15.71:0/1330195020 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff14808c4f0 con 0x7ff1580ffb40 2024-04-23T02:28:06.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.210+0000 7ff155d9b700 1 --2- 172.21.15.71:0/1330195020 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff14006cce0 0x7ff14006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:06.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.210+0000 7ff155d9b700 1 --2- 172.21.15.71:0/1330195020 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff14006cce0 0x7ff14006f1a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff1580696e0 tx=0x7ff14c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:06.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.215+0000 7ff1477fe700 1 -- 172.21.15.71:0/1330195020 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff1480565e0 con 0x7ff1580ffb40 2024-04-23T02:28:06.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.426+0000 7ff15ce4f700 1 -- 172.21.15.71:0/1330195020 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1580611d0 con 0x7ff14006cce0 2024-04-23T02:28:06.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.427+0000 7ff1477fe700 1 -- 172.21.15.71:0/1330195020 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff1580611d0 con 0x7ff14006cce0 2024-04-23T02:28:06.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.430+0000 7ff15ce4f700 1 -- 172.21.15.71:0/1330195020 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff14006cce0 msgr2=0x7ff14006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:06.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.430+0000 7ff15ce4f700 1 --2- 172.21.15.71:0/1330195020 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff14006cce0 0x7ff14006f1a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff1580696e0 tx=0x7ff14c01a040 comp rx=0 tx=0).stop 2024-04-23T02:28:06.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.430+0000 7ff15ce4f700 1 -- 172.21.15.71:0/1330195020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580ffb40 msgr2=0x7ff158197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:06.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.430+0000 7ff15ce4f700 1 --2- 172.21.15.71:0/1330195020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580ffb40 0x7ff158197570 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff14800d8d0 tx=0x7ff14800dc90 comp rx=0 tx=0).stop 2024-04-23T02:28:06.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.430+0000 7ff15ce4f700 1 -- 172.21.15.71:0/1330195020 shutdown_connections 2024-04-23T02:28:06.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.430+0000 7ff15ce4f700 1 --2- 172.21.15.71:0/1330195020 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff14006cce0 0x7ff14006f1a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:06.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.430+0000 7ff15ce4f700 1 --2- 172.21.15.71:0/1330195020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580ffb40 0x7ff158197570 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:06.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.430+0000 7ff15ce4f700 1 --2- 172.21.15.71:0/1330195020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff158105bf0 0x7ff158197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:06.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.430+0000 7ff15ce4f700 1 -- 172.21.15.71:0/1330195020 >> 172.21.15.71:0/1330195020 conn(0x7ff1580fb610 msgr2=0x7ff1580fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:06.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.430+0000 7ff15ce4f700 1 -- 172.21.15.71:0/1330195020 shutdown_connections 2024-04-23T02:28:06.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.430+0000 7ff15ce4f700 1 -- 172.21.15.71:0/1330195020 wait complete. 2024-04-23T02:28:06.443 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:28:06.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.533+0000 7f51ff394700 1 -- 172.21.15.71:0/1687782061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51f8068580 msgr2=0x7f51f80689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:06.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.533+0000 7f51ff394700 1 --2- 172.21.15.71:0/1687782061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51f8068580 0x7f51f80689e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009910 tx=0x7f51f4009c20 comp rx=0 tx=0).stop 2024-04-23T02:28:06.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.533+0000 7f51ff394700 1 -- 172.21.15.71:0/1687782061 shutdown_connections 2024-04-23T02:28:06.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.533+0000 7f51ff394700 1 --2- 172.21.15.71:0/1687782061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51f8068580 0x7f51f80689e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:06.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.533+0000 7f51ff394700 1 --2- 172.21.15.71:0/1687782061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f8106cd0 0x7f51f81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:06.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.533+0000 7f51ff394700 1 -- 172.21.15.71:0/1687782061 >> 172.21.15.71:0/1687782061 conn(0x7f51f8075670 msgr2=0x7f51f8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:06.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.534+0000 7f51ff394700 1 -- 172.21.15.71:0/1687782061 shutdown_connections 2024-04-23T02:28:06.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.534+0000 7f51ff394700 1 -- 172.21.15.71:0/1687782061 wait complete. 2024-04-23T02:28:06.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.534+0000 7f51ff394700 1 Processor -- start 2024-04-23T02:28:06.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.534+0000 7f51ff394700 1 -- start start 2024-04-23T02:28:06.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.534+0000 7f51ff394700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f8068580 0x7f51f80ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:06.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.534+0000 7f51ff394700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51f8106cd0 0x7f51f8100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:06.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.534+0000 7f51ff394700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f8100770 con 0x7f51f8106cd0 2024-04-23T02:28:06.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.534+0000 7f51ff394700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f81008b0 con 0x7f51f8068580 2024-04-23T02:28:06.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.534+0000 7f51fc92f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51f8106cd0 0x7f51f8100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:06.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.534+0000 7f51fc92f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51f8106cd0 0x7f51f8100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38340/0 (socket says 172.21.15.71:38340) 2024-04-23T02:28:06.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.534+0000 7f51fc92f700 1 -- 172.21.15.71:0/946716034 learned_addr learned my addr 172.21.15.71:0/946716034 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:28:06.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.535+0000 7f51fd130700 1 --2- 172.21.15.71:0/946716034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f8068580 0x7f51f80ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:06.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.535+0000 7f51fc92f700 1 -- 172.21.15.71:0/946716034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f8068580 msgr2=0x7f51f80ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:06.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.535+0000 7f51fc92f700 1 --2- 172.21.15.71:0/946716034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f8068580 0x7f51f80ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:06.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.535+0000 7f51fc92f700 1 -- 172.21.15.71:0/946716034 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f40095f0 con 0x7f51f8106cd0 2024-04-23T02:28:06.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.535+0000 7f51fd130700 1 --2- 172.21.15.71:0/946716034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f8068580 0x7f51f80ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:28:06.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.535+0000 7f51fc92f700 1 --2- 172.21.15.71:0/946716034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51f8106cd0 0x7f51f8100230 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f51f4006010 tx=0x7f51f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:06.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.535+0000 7f51ea7fc700 1 -- 172.21.15.71:0/946716034 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51f401d070 con 0x7f51f8106cd0 2024-04-23T02:28:06.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.535+0000 7f51ea7fc700 1 -- 172.21.15.71:0/946716034 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f51f400f460 con 0x7f51f8106cd0 2024-04-23T02:28:06.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.535+0000 7f51ff394700 1 -- 172.21.15.71:0/946716034 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f8100b30 con 0x7f51f8106cd0 2024-04-23T02:28:06.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.535+0000 7f51ea7fc700 1 -- 172.21.15.71:0/946716034 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51f4021640 con 0x7f51f8106cd0 2024-04-23T02:28:06.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.535+0000 7f51ff394700 1 -- 172.21.15.71:0/946716034 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f81a2df0 con 0x7f51f8106cd0 2024-04-23T02:28:06.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.536+0000 7f51ff394700 1 -- 172.21.15.71:0/946716034 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f81091f0 con 0x7f51f8106cd0 2024-04-23T02:28:06.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.536+0000 7f51ea7fc700 1 -- 172.21.15.71:0/946716034 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f51f400fa60 con 0x7f51f8106cd0 2024-04-23T02:28:06.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.537+0000 7f51ea7fc700 1 --2- 172.21.15.71:0/946716034 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f51e406cce0 0x7f51e406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:06.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.537+0000 7f51ea7fc700 1 -- 172.21.15.71:0/946716034 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f51f408d7b0 con 0x7f51f8106cd0 2024-04-23T02:28:06.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.537+0000 7f51fd130700 1 --2- 172.21.15.71:0/946716034 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f51e406cce0 0x7f51e406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:06.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.537+0000 7f51fd130700 1 --2- 172.21.15.71:0/946716034 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f51e406cce0 0x7f51e406f1a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f51ec007900 tx=0x7f51ec008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:06.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.541+0000 7f51ea7fc700 1 -- 172.21.15.71:0/946716034 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f51f40578a0 con 0x7f51f8106cd0 2024-04-23T02:28:06.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:06 smithi071 ceph-mon[144544]: pgmap v443: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:06.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.752+0000 7f51ff394700 1 -- 172.21.15.71:0/946716034 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f51f80611d0 con 0x7f51e406cce0 2024-04-23T02:28:06.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.753+0000 7f51ea7fc700 1 -- 172.21.15.71:0/946716034 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f51f80611d0 con 0x7f51e406cce0 2024-04-23T02:28:06.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.755+0000 7f51ff394700 1 -- 172.21.15.71:0/946716034 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f51e406cce0 msgr2=0x7f51e406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:06.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.755+0000 7f51ff394700 1 --2- 172.21.15.71:0/946716034 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f51e406cce0 0x7f51e406f1a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f51ec007900 tx=0x7f51ec008040 comp rx=0 tx=0).stop 2024-04-23T02:28:06.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.755+0000 7f51ff394700 1 -- 172.21.15.71:0/946716034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51f8106cd0 msgr2=0x7f51f8100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:06.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.755+0000 7f51ff394700 1 --2- 172.21.15.71:0/946716034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51f8106cd0 0x7f51f8100230 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f51f4006010 tx=0x7f51f400be20 comp rx=0 tx=0).stop 2024-04-23T02:28:06.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.755+0000 7f51ff394700 1 -- 172.21.15.71:0/946716034 shutdown_connections 2024-04-23T02:28:06.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.755+0000 7f51ff394700 1 --2- 172.21.15.71:0/946716034 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f51e406cce0 0x7f51e406f1a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:06.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.755+0000 7f51ff394700 1 --2- 172.21.15.71:0/946716034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f8068580 0x7f51f80ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:06.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.755+0000 7f51ff394700 1 --2- 172.21.15.71:0/946716034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51f8106cd0 0x7f51f8100230 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:06.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.756+0000 7f51ff394700 1 -- 172.21.15.71:0/946716034 >> 172.21.15.71:0/946716034 conn(0x7f51f8075670 msgr2=0x7f51f80fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:06.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.756+0000 7f51ff394700 1 -- 172.21.15.71:0/946716034 shutdown_connections 2024-04-23T02:28:06.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.756+0000 7f51ff394700 1 -- 172.21.15.71:0/946716034 wait complete. 2024-04-23T02:28:06.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.859+0000 7fa7cb798700 1 -- 172.21.15.71:0/1281026040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c40fff50 msgr2=0x7fa7c4105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:06.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.859+0000 7fa7cb798700 1 --2- 172.21.15.71:0/1281026040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c40fff50 0x7fa7c4105840 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009910 tx=0x7fa7b4009c20 comp rx=0 tx=0).stop 2024-04-23T02:28:06.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.859+0000 7fa7cb798700 1 -- 172.21.15.71:0/1281026040 shutdown_connections 2024-04-23T02:28:06.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.859+0000 7fa7cb798700 1 --2- 172.21.15.71:0/1281026040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c40fff50 0x7fa7c4105840 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:06.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.859+0000 7fa7cb798700 1 --2- 172.21.15.71:0/1281026040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7c40ff5a0 0x7fa7c40ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:06.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.859+0000 7fa7cb798700 1 -- 172.21.15.71:0/1281026040 >> 172.21.15.71:0/1281026040 conn(0x7fa7c4074e30 msgr2=0x7fa7c4075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:06.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.859+0000 7fa7cb798700 1 -- 172.21.15.71:0/1281026040 shutdown_connections 2024-04-23T02:28:06.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.859+0000 7fa7cb798700 1 -- 172.21.15.71:0/1281026040 wait complete. 2024-04-23T02:28:06.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.860+0000 7fa7cb798700 1 Processor -- start 2024-04-23T02:28:06.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.860+0000 7fa7cb798700 1 -- start start 2024-04-23T02:28:06.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.860+0000 7fa7cb798700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7c40ff5a0 0x7fa7c419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:06.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.860+0000 7fa7cb798700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c40fff50 0x7fa7c419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.860+0000 7fa7cb798700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c419d150 con 0x7fa7c40fff50 2024-04-23T02:28:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.860+0000 7fa7cb798700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c4196650 con 0x7fa7c40ff5a0 2024-04-23T02:28:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.860+0000 7fa7c8d33700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c40fff50 0x7fa7c419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.860+0000 7fa7c8d33700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c40fff50 0x7fa7c419cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38348/0 (socket says 172.21.15.71:38348) 2024-04-23T02:28:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.860+0000 7fa7c8d33700 1 -- 172.21.15.71:0/2444640206 learned_addr learned my addr 172.21.15.71:0/2444640206 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:28:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.860+0000 7fa7c8d33700 1 -- 172.21.15.71:0/2444640206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7c40ff5a0 msgr2=0x7fa7c419c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.860+0000 7fa7c8d33700 1 --2- 172.21.15.71:0/2444640206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7c40ff5a0 0x7fa7c419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.860+0000 7fa7c8d33700 1 -- 172.21.15.71:0/2444640206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b40095f0 con 0x7fa7c40fff50 2024-04-23T02:28:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.861+0000 7fa7c8d33700 1 --2- 172.21.15.71:0/2444640206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c40fff50 0x7fa7c419cac0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa7b40092e0 tx=0x7fa7b400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.861+0000 7fa7ba7fc700 1 -- 172.21.15.71:0/2444640206 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7b401d070 con 0x7fa7c40fff50 2024-04-23T02:28:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.861+0000 7fa7ba7fc700 1 -- 172.21.15.71:0/2444640206 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa7b4005940 con 0x7fa7c40fff50 2024-04-23T02:28:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.861+0000 7fa7cb798700 1 -- 172.21.15.71:0/2444640206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c41968d0 con 0x7fa7c40fff50 2024-04-23T02:28:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.861+0000 7fa7ba7fc700 1 -- 172.21.15.71:0/2444640206 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7b400fe50 con 0x7fa7c40fff50 2024-04-23T02:28:06.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.861+0000 7fa7cb798700 1 -- 172.21.15.71:0/2444640206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c4196dc0 con 0x7fa7c40fff50 2024-04-23T02:28:06.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.862+0000 7fa7cb798700 1 -- 172.21.15.71:0/2444640206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c4109200 con 0x7fa7c40fff50 2024-04-23T02:28:06.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.862+0000 7fa7ba7fc700 1 -- 172.21.15.71:0/2444640206 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa7b4003a60 con 0x7fa7c40fff50 2024-04-23T02:28:06.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.862+0000 7fa7ba7fc700 1 --2- 172.21.15.71:0/2444640206 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa7b006cce0 0x7fa7b006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:06.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.863+0000 7fa7ba7fc700 1 -- 172.21.15.71:0/2444640206 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa7b408d7e0 con 0x7fa7c40fff50 2024-04-23T02:28:06.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.863+0000 7fa7c9534700 1 --2- 172.21.15.71:0/2444640206 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa7b006cce0 0x7fa7b006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:06.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.863+0000 7fa7c9534700 1 --2- 172.21.15.71:0/2444640206 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa7b006cce0 0x7fa7b006f1a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0005950 tx=0x7fa7c00058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:06.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:06.867+0000 7fa7ba7fc700 1 -- 172.21.15.71:0/2444640206 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa7b405b2e0 con 0x7fa7c40fff50 2024-04-23T02:28:06.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:06 smithi192 ceph-mon[108617]: pgmap v443: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:07.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.076+0000 7fa7cb798700 1 -- 172.21.15.71:0/2444640206 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7c4196fe0 con 0x7fa7b006cce0 2024-04-23T02:28:07.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.086+0000 7fa7ba7fc700 1 -- 172.21.15.71:0/2444640206 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa7c4196fe0 con 0x7fa7b006cce0 2024-04-23T02:28:07.089 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:28:07.089 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (11m) 5m ago 23m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:28:07.089 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (23m) 5m ago 23m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:28:07.089 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (22m) 5m ago 22m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:28:07.089 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (9m) 5m ago 23m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:28:07.089 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (9m) 5m ago 22m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:28:07.089 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (11m) 5m ago 22m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:28:07.089 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (5m) 5m ago 17m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:28:07.090 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (5m) 5m ago 17m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:28:07.090 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (17m) 5m ago 17m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:28:07.090 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (5m) 5m ago 17m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:28:07.090 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (13m) 5m ago 24m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:28:07.090 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (13m) 5m ago 22m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:28:07.090 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (10m) 5m ago 24m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:28:07.090 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (9m) 5m ago 21m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:28:07.090 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (12m) 5m ago 23m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:28:07.090 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (12m) 5m ago 22m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:28:07.090 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (9m) 5m ago 21m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:28:07.090 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (8m) 5m ago 20m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:28:07.090 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (8m) 5m ago 20m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:28:07.090 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (7m) 5m ago 19m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:28:07.091 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (6m) 5m ago 19m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:28:07.091 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (6m) 5m ago 18m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:28:07.091 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (12m) 5m ago 22m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:28:07.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.088+0000 7fa7cb798700 1 -- 172.21.15.71:0/2444640206 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa7b006cce0 msgr2=0x7fa7b006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:07.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.088+0000 7fa7cb798700 1 --2- 172.21.15.71:0/2444640206 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa7b006cce0 0x7fa7b006f1a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0005950 tx=0x7fa7c00058e0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.088+0000 7fa7cb798700 1 -- 172.21.15.71:0/2444640206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c40fff50 msgr2=0x7fa7c419cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:07.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.088+0000 7fa7cb798700 1 --2- 172.21.15.71:0/2444640206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c40fff50 0x7fa7c419cac0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa7b40092e0 tx=0x7fa7b400be20 comp rx=0 tx=0).stop 2024-04-23T02:28:07.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.088+0000 7fa7cb798700 1 -- 172.21.15.71:0/2444640206 shutdown_connections 2024-04-23T02:28:07.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.088+0000 7fa7cb798700 1 --2- 172.21.15.71:0/2444640206 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa7b006cce0 0x7fa7b006f1a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.088+0000 7fa7cb798700 1 --2- 172.21.15.71:0/2444640206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7c40ff5a0 0x7fa7c419c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.088+0000 7fa7cb798700 1 --2- 172.21.15.71:0/2444640206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c40fff50 0x7fa7c419cac0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.088+0000 7fa7cb798700 1 -- 172.21.15.71:0/2444640206 >> 172.21.15.71:0/2444640206 conn(0x7fa7c4074e30 msgr2=0x7fa7c40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:07.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.088+0000 7fa7cb798700 1 -- 172.21.15.71:0/2444640206 shutdown_connections 2024-04-23T02:28:07.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.088+0000 7fa7cb798700 1 -- 172.21.15.71:0/2444640206 wait complete. 2024-04-23T02:28:07.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.190+0000 7f9de6bf4700 1 -- 172.21.15.71:0/1807056966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9de00ffc20 msgr2=0x7f9de0100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:07.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.190+0000 7f9de6bf4700 1 --2- 172.21.15.71:0/1807056966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9de00ffc20 0x7f9de0100080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009910 tx=0x7f9dcc009c20 comp rx=0 tx=0).stop 2024-04-23T02:28:07.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.190+0000 7f9de6bf4700 1 -- 172.21.15.71:0/1807056966 shutdown_connections 2024-04-23T02:28:07.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.190+0000 7f9de6bf4700 1 --2- 172.21.15.71:0/1807056966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9de00ffc20 0x7f9de0100080 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.190+0000 7f9de6bf4700 1 --2- 172.21.15.71:0/1807056966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9de0105bd0 0x7f9de0105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.190+0000 7f9de6bf4700 1 -- 172.21.15.71:0/1807056966 >> 172.21.15.71:0/1807056966 conn(0x7f9de0075670 msgr2=0x7f9de0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:07.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.190+0000 7f9de6bf4700 1 -- 172.21.15.71:0/1807056966 shutdown_connections 2024-04-23T02:28:07.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.190+0000 7f9de6bf4700 1 -- 172.21.15.71:0/1807056966 wait complete. 2024-04-23T02:28:07.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.190+0000 7f9de6bf4700 1 Processor -- start 2024-04-23T02:28:07.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.191+0000 7f9de6bf4700 1 -- start start 2024-04-23T02:28:07.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.191+0000 7f9de6bf4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9de00ffc20 0x7f9de019c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:07.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.191+0000 7f9de6bf4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9de0105bd0 0x7f9de019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:07.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.191+0000 7f9de6bf4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de019d1c0 con 0x7f9de00ffc20 2024-04-23T02:28:07.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.191+0000 7f9de6bf4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de0196670 con 0x7f9de0105bd0 2024-04-23T02:28:07.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.191+0000 7f9de4990700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9de00ffc20 0x7f9de019c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:07.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.191+0000 7f9de4990700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9de00ffc20 0x7f9de019c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38360/0 (socket says 172.21.15.71:38360) 2024-04-23T02:28:07.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.191+0000 7f9de4990700 1 -- 172.21.15.71:0/2493048143 learned_addr learned my addr 172.21.15.71:0/2493048143 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:28:07.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.191+0000 7f9ddffff700 1 --2- 172.21.15.71:0/2493048143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9de0105bd0 0x7f9de019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:07.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.191+0000 7f9de4990700 1 -- 172.21.15.71:0/2493048143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9de0105bd0 msgr2=0x7f9de019cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:07.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.191+0000 7f9de4990700 1 --2- 172.21.15.71:0/2493048143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9de0105bd0 0x7f9de019cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.191+0000 7f9de4990700 1 -- 172.21.15.71:0/2493048143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dcc0095f0 con 0x7f9de00ffc20 2024-04-23T02:28:07.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.191+0000 7f9ddffff700 1 --2- 172.21.15.71:0/2493048143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9de0105bd0 0x7f9de019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:28:07.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.191+0000 7f9de4990700 1 --2- 172.21.15.71:0/2493048143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9de00ffc20 0x7f9de019c5f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400d8d0 tx=0x7f9dd400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:07.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.191+0000 7f9dddffb700 1 -- 172.21.15.71:0/2493048143 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dd4009940 con 0x7f9de00ffc20 2024-04-23T02:28:07.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.191+0000 7f9dddffb700 1 -- 172.21.15.71:0/2493048143 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9dd4010460 con 0x7f9de00ffc20 2024-04-23T02:28:07.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.191+0000 7f9de6bf4700 1 -- 172.21.15.71:0/2493048143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de0196950 con 0x7f9de00ffc20 2024-04-23T02:28:07.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.191+0000 7f9dddffb700 1 -- 172.21.15.71:0/2493048143 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dd400f5f0 con 0x7f9de00ffc20 2024-04-23T02:28:07.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.192+0000 7f9de6bf4700 1 -- 172.21.15.71:0/2493048143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de0196ea0 con 0x7f9de00ffc20 2024-04-23T02:28:07.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.193+0000 7f9de6bf4700 1 -- 172.21.15.71:0/2493048143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de004f2e0 con 0x7f9de00ffc20 2024-04-23T02:28:07.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.193+0000 7f9dddffb700 1 -- 172.21.15.71:0/2493048143 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9dd40105d0 con 0x7f9de00ffc20 2024-04-23T02:28:07.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.193+0000 7f9dddffb700 1 --2- 172.21.15.71:0/2493048143 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9dd006cce0 0x7f9dd006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:07.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.193+0000 7f9dddffb700 1 -- 172.21.15.71:0/2493048143 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9dd408c5e0 con 0x7f9de00ffc20 2024-04-23T02:28:07.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.193+0000 7f9ddffff700 1 --2- 172.21.15.71:0/2493048143 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9dd006cce0 0x7f9dd006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:07.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.194+0000 7f9ddffff700 1 --2- 172.21.15.71:0/2493048143 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9dd006cce0 0x7f9dd006f1a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00b5c0 tx=0x7f9dcc005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:07.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.198+0000 7f9dddffb700 1 -- 172.21.15.71:0/2493048143 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9dd405a0e0 con 0x7f9de00ffc20 2024-04-23T02:28:07.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.475+0000 7f9de6bf4700 1 -- 172.21.15.71:0/2493048143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9de01970c0 con 0x7f9de00ffc20 2024-04-23T02:28:07.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.476+0000 7f9dddffb700 1 -- 172.21.15.71:0/2493048143 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f9dd4010880 con 0x7f9de00ffc20 2024-04-23T02:28:07.478 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:28:07.479 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:28:07.479 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:28:07.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:28:07.479 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:28:07.479 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:28:07.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:28:07.479 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:28:07.479 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:28:07.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:28:07.479 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:28:07.479 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:28:07.479 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:28:07.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:28:07.480 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:28:07.480 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:28:07.480 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:28:07.480 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:28:07.480 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:28:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.478+0000 7f9de6bf4700 1 -- 172.21.15.71:0/2493048143 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9dd006cce0 msgr2=0x7f9dd006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.478+0000 7f9de6bf4700 1 --2- 172.21.15.71:0/2493048143 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9dd006cce0 0x7f9dd006f1a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00b5c0 tx=0x7f9dcc005960 comp rx=0 tx=0).stop 2024-04-23T02:28:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.478+0000 7f9de6bf4700 1 -- 172.21.15.71:0/2493048143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9de00ffc20 msgr2=0x7f9de019c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.478+0000 7f9de6bf4700 1 --2- 172.21.15.71:0/2493048143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9de00ffc20 0x7f9de019c5f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400d8d0 tx=0x7f9dd400dc90 comp rx=0 tx=0).stop 2024-04-23T02:28:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.478+0000 7f9de6bf4700 1 -- 172.21.15.71:0/2493048143 shutdown_connections 2024-04-23T02:28:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.478+0000 7f9de6bf4700 1 --2- 172.21.15.71:0/2493048143 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9dd006cce0 0x7f9dd006f1a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.478+0000 7f9de6bf4700 1 --2- 172.21.15.71:0/2493048143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9de00ffc20 0x7f9de019c5f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.478+0000 7f9de6bf4700 1 --2- 172.21.15.71:0/2493048143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9de0105bd0 0x7f9de019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.478+0000 7f9de6bf4700 1 -- 172.21.15.71:0/2493048143 >> 172.21.15.71:0/2493048143 conn(0x7f9de0075670 msgr2=0x7f9de0103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.478+0000 7f9de6bf4700 1 -- 172.21.15.71:0/2493048143 shutdown_connections 2024-04-23T02:28:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.478+0000 7f9de6bf4700 1 -- 172.21.15.71:0/2493048143 wait complete. 2024-04-23T02:28:07.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.581+0000 7f272d72a700 1 -- 172.21.15.71:0/2560872204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2728101850 msgr2=0x7f27281058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:07.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.581+0000 7f272d72a700 1 --2- 172.21.15.71:0/2560872204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2728101850 0x7f27281058a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2718009910 tx=0x7f2718009c20 comp rx=0 tx=0).stop 2024-04-23T02:28:07.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.581+0000 7f272d72a700 1 -- 172.21.15.71:0/2560872204 shutdown_connections 2024-04-23T02:28:07.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.581+0000 7f272d72a700 1 --2- 172.21.15.71:0/2560872204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2728101850 0x7f27281058a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.581+0000 7f272d72a700 1 --2- 172.21.15.71:0/2560872204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2728100ea0 0x7f2728101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.581+0000 7f272d72a700 1 -- 172.21.15.71:0/2560872204 >> 172.21.15.71:0/2560872204 conn(0x7f27280fc750 msgr2=0x7f27280feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:07.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.581+0000 7f272d72a700 1 -- 172.21.15.71:0/2560872204 shutdown_connections 2024-04-23T02:28:07.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.581+0000 7f272d72a700 1 -- 172.21.15.71:0/2560872204 wait complete. 2024-04-23T02:28:07.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.581+0000 7f272d72a700 1 Processor -- start 2024-04-23T02:28:07.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.582+0000 7f272d72a700 1 -- start start 2024-04-23T02:28:07.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.582+0000 7f272d72a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2728100ea0 0x7f272819c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:07.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.582+0000 7f272d72a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2728101850 0x7f272819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:07.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.582+0000 7f272d72a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272819d2b0 con 0x7f2728100ea0 2024-04-23T02:28:07.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.582+0000 7f272d72a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2728196760 con 0x7f2728101850 2024-04-23T02:28:07.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.582+0000 7f2726ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2728100ea0 0x7f272819c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:07.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.582+0000 7f2726ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2728100ea0 0x7f272819c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38374/0 (socket says 172.21.15.71:38374) 2024-04-23T02:28:07.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.582+0000 7f2726ffd700 1 -- 172.21.15.71:0/531870795 learned_addr learned my addr 172.21.15.71:0/531870795 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:28:07.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.582+0000 7f2726ffd700 1 -- 172.21.15.71:0/531870795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2728101850 msgr2=0x7f272819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:07.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.582+0000 7f2726ffd700 1 --2- 172.21.15.71:0/531870795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2728101850 0x7f272819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.582+0000 7f2726ffd700 1 -- 172.21.15.71:0/531870795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27180095f0 con 0x7f2728100ea0 2024-04-23T02:28:07.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.582+0000 7f2726ffd700 1 --2- 172.21.15.71:0/531870795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2728100ea0 0x7f272819c6e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f271c00b4b0 tx=0x7f271c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:07.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.582+0000 7f270ffff700 1 -- 172.21.15.71:0/531870795 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f271c011800 con 0x7f2728100ea0 2024-04-23T02:28:07.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.582+0000 7f270ffff700 1 -- 172.21.15.71:0/531870795 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f271c011e00 con 0x7f2728100ea0 2024-04-23T02:28:07.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.582+0000 7f272d72a700 1 -- 172.21.15.71:0/531870795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2728196a40 con 0x7f2728100ea0 2024-04-23T02:28:07.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.582+0000 7f270ffff700 1 -- 172.21.15.71:0/531870795 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f271c010e50 con 0x7f2728100ea0 2024-04-23T02:28:07.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.583+0000 7f272d72a700 1 -- 172.21.15.71:0/531870795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2728196f90 con 0x7f2728100ea0 2024-04-23T02:28:07.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.583+0000 7f272d72a700 1 -- 172.21.15.71:0/531870795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27281091d0 con 0x7f2728100ea0 2024-04-23T02:28:07.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.584+0000 7f270ffff700 1 -- 172.21.15.71:0/531870795 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f271c010380 con 0x7f2728100ea0 2024-04-23T02:28:07.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.584+0000 7f270ffff700 1 --2- 172.21.15.71:0/531870795 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f271006cce0 0x7f271006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:07.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.584+0000 7f270ffff700 1 -- 172.21.15.71:0/531870795 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f271c08c190 con 0x7f2728100ea0 2024-04-23T02:28:07.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.584+0000 7f27267fc700 1 --2- 172.21.15.71:0/531870795 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f271006cce0 0x7f271006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:07.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.585+0000 7f27267fc700 1 --2- 172.21.15.71:0/531870795 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f271006cce0 0x7f271006f1a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f271800b5c0 tx=0x7f2718005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:07.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.589+0000 7f270ffff700 1 -- 172.21.15.71:0/531870795 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f271c059c90 con 0x7f2728100ea0 2024-04-23T02:28:07.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:07 smithi071 ceph-mon[144544]: from='client.34650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:28:07.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.821+0000 7f272d72a700 1 -- 172.21.15.71:0/531870795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2728197d70 con 0x7f2728100ea0 2024-04-23T02:28:07.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.821+0000 7f270ffff700 1 -- 172.21.15.71:0/531870795 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f2728197d70 con 0x7f2728100ea0 2024-04-23T02:28:07.824 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:28:07.824 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:28:07.824 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:28:07.824 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:28:07.824 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:28:07.825 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:28:07.825 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:28:07.825 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:28:07.825 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:28:07.825 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:28:07.825 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:28:07.825 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:28:07.825 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:28:07.825 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:28:07.825 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:28:07.825 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:28:07.825 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:28:07.825 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:28:07.825 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:28:07.826 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:28:07.826 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:28:07.826 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:28:07.826 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:28:07.826 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:28:07.826 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:28:07.826 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:28:07.826 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:28:07.826 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:28:07.826 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:28:07.826 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:28:07.826 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:28:07.826 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:28:07.826 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:28:07.826 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:28:07.827 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:28:07.827 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:28:07.827 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:28:07.827 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:28:07.827 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:28:07.827 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:28:07.827 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:28:07.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.823+0000 7f272d72a700 1 -- 172.21.15.71:0/531870795 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f271006cce0 msgr2=0x7f271006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:07.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.823+0000 7f272d72a700 1 --2- 172.21.15.71:0/531870795 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f271006cce0 0x7f271006f1a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f271800b5c0 tx=0x7f2718005960 comp rx=0 tx=0).stop 2024-04-23T02:28:07.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.823+0000 7f272d72a700 1 -- 172.21.15.71:0/531870795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2728100ea0 msgr2=0x7f272819c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:07.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.823+0000 7f272d72a700 1 --2- 172.21.15.71:0/531870795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2728100ea0 0x7f272819c6e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f271c00b4b0 tx=0x7f271c00b870 comp rx=0 tx=0).stop 2024-04-23T02:28:07.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.824+0000 7f272d72a700 1 -- 172.21.15.71:0/531870795 shutdown_connections 2024-04-23T02:28:07.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.824+0000 7f272d72a700 1 --2- 172.21.15.71:0/531870795 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f271006cce0 0x7f271006f1a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.824+0000 7f272d72a700 1 --2- 172.21.15.71:0/531870795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2728100ea0 0x7f272819c6e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.824+0000 7f272d72a700 1 --2- 172.21.15.71:0/531870795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2728101850 0x7f272819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.824+0000 7f272d72a700 1 -- 172.21.15.71:0/531870795 >> 172.21.15.71:0/531870795 conn(0x7f27280fc750 msgr2=0x7f27280fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:07.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.824+0000 7f272d72a700 1 -- 172.21.15.71:0/531870795 shutdown_connections 2024-04-23T02:28:07.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.824+0000 7f272d72a700 1 -- 172.21.15.71:0/531870795 wait complete. 2024-04-23T02:28:07.829 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:28:07.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:07 smithi192 ceph-mon[108617]: from='client.34650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:28:07.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.927+0000 7f5e77490700 1 -- 172.21.15.71:0/956508277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e700fff50 msgr2=0x7f5e70105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:07.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.927+0000 7f5e77490700 1 --2- 172.21.15.71:0/956508277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e700fff50 0x7f5e70105840 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009910 tx=0x7f5e6c009c20 comp rx=0 tx=0).stop 2024-04-23T02:28:07.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.927+0000 7f5e77490700 1 -- 172.21.15.71:0/956508277 shutdown_connections 2024-04-23T02:28:07.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.927+0000 7f5e77490700 1 --2- 172.21.15.71:0/956508277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e700fff50 0x7f5e70105840 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.927+0000 7f5e77490700 1 --2- 172.21.15.71:0/956508277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e700ff5a0 0x7f5e700ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.927+0000 7f5e77490700 1 -- 172.21.15.71:0/956508277 >> 172.21.15.71:0/956508277 conn(0x7f5e70074e30 msgr2=0x7f5e70075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:07.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.927+0000 7f5e77490700 1 -- 172.21.15.71:0/956508277 shutdown_connections 2024-04-23T02:28:07.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.927+0000 7f5e77490700 1 -- 172.21.15.71:0/956508277 wait complete. 2024-04-23T02:28:07.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.928+0000 7f5e77490700 1 Processor -- start 2024-04-23T02:28:07.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.928+0000 7f5e77490700 1 -- start start 2024-04-23T02:28:07.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.928+0000 7f5e77490700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e700ff5a0 0x7f5e70100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:07.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.928+0000 7f5e77490700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e700fff50 0x7f5e70100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:07.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.928+0000 7f5e77490700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e70100fa0 con 0x7f5e700fff50 2024-04-23T02:28:07.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.928+0000 7f5e77490700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e701010e0 con 0x7f5e700ff5a0 2024-04-23T02:28:07.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.928+0000 7f5e74a2b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e700fff50 0x7f5e70100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:07.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.928+0000 7f5e74a2b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e700fff50 0x7f5e70100a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38398/0 (socket says 172.21.15.71:38398) 2024-04-23T02:28:07.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.928+0000 7f5e74a2b700 1 -- 172.21.15.71:0/2772764657 learned_addr learned my addr 172.21.15.71:0/2772764657 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:28:07.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.928+0000 7f5e7522c700 1 --2- 172.21.15.71:0/2772764657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e700ff5a0 0x7f5e70100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:07.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.928+0000 7f5e74a2b700 1 -- 172.21.15.71:0/2772764657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e700ff5a0 msgr2=0x7f5e70100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:07.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.928+0000 7f5e74a2b700 1 --2- 172.21.15.71:0/2772764657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e700ff5a0 0x7f5e70100520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:07.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.928+0000 7f5e74a2b700 1 -- 172.21.15.71:0/2772764657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c0095f0 con 0x7f5e700fff50 2024-04-23T02:28:07.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.928+0000 7f5e7522c700 1 --2- 172.21.15.71:0/2772764657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e700ff5a0 0x7f5e70100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:28:07.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.928+0000 7f5e74a2b700 1 --2- 172.21.15.71:0/2772764657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e700fff50 0x7f5e70100a60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00bda0 tx=0x7f5e6c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:07.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.929+0000 7f5e667fc700 1 -- 172.21.15.71:0/2772764657 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e6c01d070 con 0x7f5e700fff50 2024-04-23T02:28:07.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.929+0000 7f5e667fc700 1 -- 172.21.15.71:0/2772764657 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5e6c00f460 con 0x7f5e700fff50 2024-04-23T02:28:07.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.929+0000 7f5e667fc700 1 -- 172.21.15.71:0/2772764657 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e6c021640 con 0x7f5e700fff50 2024-04-23T02:28:07.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.929+0000 7f5e77490700 1 -- 172.21.15.71:0/2772764657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e70101360 con 0x7f5e700fff50 2024-04-23T02:28:07.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.929+0000 7f5e77490700 1 -- 172.21.15.71:0/2772764657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e701a2c70 con 0x7f5e700fff50 2024-04-23T02:28:07.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.930+0000 7f5e77490700 1 -- 172.21.15.71:0/2772764657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e70109200 con 0x7f5e700fff50 2024-04-23T02:28:07.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.930+0000 7f5e667fc700 1 -- 172.21.15.71:0/2772764657 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5e6c00f9b0 con 0x7f5e700fff50 2024-04-23T02:28:07.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.930+0000 7f5e667fc700 1 --2- 172.21.15.71:0/2772764657 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5e5c06cce0 0x7f5e5c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:07.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.931+0000 7f5e667fc700 1 -- 172.21.15.71:0/2772764657 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5e6c08d770 con 0x7f5e700fff50 2024-04-23T02:28:07.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.931+0000 7f5e7522c700 1 --2- 172.21.15.71:0/2772764657 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5e5c06cce0 0x7f5e5c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:07.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.931+0000 7f5e7522c700 1 --2- 172.21.15.71:0/2772764657 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5e5c06cce0 0x7f5e5c06f1a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e60005fd0 tx=0x7f5e60005e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:07.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:07.935+0000 7f5e667fc700 1 -- 172.21.15.71:0/2772764657 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5e6c057860 con 0x7f5e700fff50 2024-04-23T02:28:08.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.145+0000 7f5e77490700 1 -- 172.21.15.71:0/2772764657 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e700611d0 con 0x7f5e5c06cce0 2024-04-23T02:28:08.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.146+0000 7f5e667fc700 1 -- 172.21.15.71:0/2772764657 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5e700611d0 con 0x7f5e5c06cce0 2024-04-23T02:28:08.149 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:28:08.149 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:28:08.149 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:28:08.149 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:28:08.149 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:28:08.149 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:28:08.149 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:28:08.149 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:28:08.149 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:28:08.149 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:28:08.149 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:28:08.149 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:28:08.149 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:28:08.149 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:28:08.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.148+0000 7f5e77490700 1 -- 172.21.15.71:0/2772764657 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5e5c06cce0 msgr2=0x7f5e5c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:08.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.148+0000 7f5e77490700 1 --2- 172.21.15.71:0/2772764657 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5e5c06cce0 0x7f5e5c06f1a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e60005fd0 tx=0x7f5e60005e30 comp rx=0 tx=0).stop 2024-04-23T02:28:08.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.148+0000 7f5e77490700 1 -- 172.21.15.71:0/2772764657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e700fff50 msgr2=0x7f5e70100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:08.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.148+0000 7f5e77490700 1 --2- 172.21.15.71:0/2772764657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e700fff50 0x7f5e70100a60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00bda0 tx=0x7f5e6c00be80 comp rx=0 tx=0).stop 2024-04-23T02:28:08.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.148+0000 7f5e77490700 1 -- 172.21.15.71:0/2772764657 shutdown_connections 2024-04-23T02:28:08.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.148+0000 7f5e77490700 1 --2- 172.21.15.71:0/2772764657 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5e5c06cce0 0x7f5e5c06f1a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:08.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.148+0000 7f5e77490700 1 --2- 172.21.15.71:0/2772764657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e700ff5a0 0x7f5e70100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:08.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.148+0000 7f5e77490700 1 --2- 172.21.15.71:0/2772764657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e700fff50 0x7f5e70100a60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:08.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.148+0000 7f5e77490700 1 -- 172.21.15.71:0/2772764657 >> 172.21.15.71:0/2772764657 conn(0x7f5e70074e30 msgr2=0x7f5e700fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:08.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.148+0000 7f5e77490700 1 -- 172.21.15.71:0/2772764657 shutdown_connections 2024-04-23T02:28:08.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.148+0000 7f5e77490700 1 -- 172.21.15.71:0/2772764657 wait complete. 2024-04-23T02:28:08.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.250+0000 7faca60a4700 1 -- 172.21.15.71:0/2284227833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faca0101850 msgr2=0x7faca01058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:08.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.250+0000 7faca60a4700 1 --2- 172.21.15.71:0/2284227833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faca0101850 0x7faca01058a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fac90009910 tx=0x7fac90009c20 comp rx=0 tx=0).stop 2024-04-23T02:28:08.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.251+0000 7faca60a4700 1 -- 172.21.15.71:0/2284227833 shutdown_connections 2024-04-23T02:28:08.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.251+0000 7faca60a4700 1 --2- 172.21.15.71:0/2284227833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faca0101850 0x7faca01058a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:08.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.251+0000 7faca60a4700 1 --2- 172.21.15.71:0/2284227833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faca0100ea0 0x7faca0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:08.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.251+0000 7faca60a4700 1 -- 172.21.15.71:0/2284227833 >> 172.21.15.71:0/2284227833 conn(0x7faca00fc750 msgr2=0x7faca00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:08.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.251+0000 7faca60a4700 1 -- 172.21.15.71:0/2284227833 shutdown_connections 2024-04-23T02:28:08.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.251+0000 7faca60a4700 1 -- 172.21.15.71:0/2284227833 wait complete. 2024-04-23T02:28:08.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.251+0000 7faca60a4700 1 Processor -- start 2024-04-23T02:28:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.251+0000 7faca60a4700 1 -- start start 2024-04-23T02:28:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.251+0000 7faca60a4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faca0100ea0 0x7faca0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.252+0000 7faca60a4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faca0101850 0x7faca00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.252+0000 7faca60a4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca0073b10 con 0x7faca0100ea0 2024-04-23T02:28:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.252+0000 7faca60a4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca0073c50 con 0x7faca0101850 2024-04-23T02:28:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.252+0000 7fac9f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faca0100ea0 0x7faca0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.252+0000 7fac9f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faca0100ea0 0x7faca0073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38420/0 (socket says 172.21.15.71:38420) 2024-04-23T02:28:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.252+0000 7fac9f7fe700 1 -- 172.21.15.71:0/2597632538 learned_addr learned my addr 172.21.15.71:0/2597632538 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:28:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.252+0000 7fac9effd700 1 --2- 172.21.15.71:0/2597632538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faca0101850 0x7faca00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.252+0000 7fac9f7fe700 1 -- 172.21.15.71:0/2597632538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faca0101850 msgr2=0x7faca00735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.252+0000 7fac9f7fe700 1 --2- 172.21.15.71:0/2597632538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faca0101850 0x7faca00735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.252+0000 7fac9f7fe700 1 -- 172.21.15.71:0/2597632538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac900095f0 con 0x7faca0100ea0 2024-04-23T02:28:08.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.252+0000 7fac9f7fe700 1 --2- 172.21.15.71:0/2597632538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faca0100ea0 0x7faca0073090 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fac9400d8d0 tx=0x7fac9400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:08.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.252+0000 7fac9cff9700 1 -- 172.21.15.71:0/2597632538 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac94009940 con 0x7faca0100ea0 2024-04-23T02:28:08.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.252+0000 7fac9cff9700 1 -- 172.21.15.71:0/2597632538 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fac94010460 con 0x7faca0100ea0 2024-04-23T02:28:08.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.252+0000 7faca60a4700 1 -- 172.21.15.71:0/2597632538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca01a2a20 con 0x7faca0100ea0 2024-04-23T02:28:08.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.252+0000 7fac9cff9700 1 -- 172.21.15.71:0/2597632538 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac9400f5f0 con 0x7faca0100ea0 2024-04-23T02:28:08.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.252+0000 7faca60a4700 1 -- 172.21.15.71:0/2597632538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca01a2e30 con 0x7faca0100ea0 2024-04-23T02:28:08.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.253+0000 7faca60a4700 1 -- 172.21.15.71:0/2597632538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca01092a0 con 0x7faca0100ea0 2024-04-23T02:28:08.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.254+0000 7fac9cff9700 1 -- 172.21.15.71:0/2597632538 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fac940105d0 con 0x7faca0100ea0 2024-04-23T02:28:08.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.254+0000 7fac9cff9700 1 --2- 172.21.15.71:0/2597632538 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fac8806cce0 0x7fac8806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:08.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.254+0000 7fac9cff9700 1 -- 172.21.15.71:0/2597632538 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fac9408c4f0 con 0x7faca0100ea0 2024-04-23T02:28:08.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.254+0000 7fac9effd700 1 --2- 172.21.15.71:0/2597632538 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fac8806cce0 0x7fac8806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:08.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.255+0000 7fac9effd700 1 --2- 172.21.15.71:0/2597632538 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fac8806cce0 0x7fac8806f1a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faca0074880 tx=0x7fac9001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:08.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.259+0000 7fac9cff9700 1 -- 172.21.15.71:0/2597632538 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fac940565e0 con 0x7faca0100ea0 2024-04-23T02:28:08.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.528+0000 7faca60a4700 1 -- 172.21.15.71:0/2597632538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7faca004f2e0 con 0x7faca0100ea0 2024-04-23T02:28:08.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.528+0000 7fac9cff9700 1 -- 172.21.15.71:0/2597632538 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fac94010880 con 0x7faca0100ea0 2024-04-23T02:28:08.531 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:28:08.531 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:28:08.532 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:28:08.532 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:28:08.532 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:28:08.532 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:28:08.532 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:28:08.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.530+0000 7faca60a4700 1 -- 172.21.15.71:0/2597632538 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fac8806cce0 msgr2=0x7fac8806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:08.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.530+0000 7faca60a4700 1 --2- 172.21.15.71:0/2597632538 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fac8806cce0 0x7fac8806f1a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faca0074880 tx=0x7fac9001a040 comp rx=0 tx=0).stop 2024-04-23T02:28:08.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.530+0000 7faca60a4700 1 -- 172.21.15.71:0/2597632538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faca0100ea0 msgr2=0x7faca0073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:08.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.530+0000 7faca60a4700 1 --2- 172.21.15.71:0/2597632538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faca0100ea0 0x7faca0073090 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fac9400d8d0 tx=0x7fac9400dc90 comp rx=0 tx=0).stop 2024-04-23T02:28:08.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.530+0000 7faca60a4700 1 -- 172.21.15.71:0/2597632538 shutdown_connections 2024-04-23T02:28:08.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.530+0000 7faca60a4700 1 --2- 172.21.15.71:0/2597632538 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fac8806cce0 0x7fac8806f1a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:08.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.530+0000 7faca60a4700 1 --2- 172.21.15.71:0/2597632538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faca0100ea0 0x7faca0073090 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:08.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.530+0000 7faca60a4700 1 --2- 172.21.15.71:0/2597632538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faca0101850 0x7faca00735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:08.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.530+0000 7faca60a4700 1 -- 172.21.15.71:0/2597632538 >> 172.21.15.71:0/2597632538 conn(0x7faca00fc750 msgr2=0x7faca00fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:08.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.531+0000 7faca60a4700 1 -- 172.21.15.71:0/2597632538 shutdown_connections 2024-04-23T02:28:08.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:08.531+0000 7faca60a4700 1 -- 172.21.15.71:0/2597632538 wait complete. 2024-04-23T02:28:08.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:08 smithi071 ceph-mon[144544]: from='client.34654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:28:08.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:08 smithi071 ceph-mon[144544]: pgmap v444: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:08.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:08 smithi071 ceph-mon[144544]: from='client.34658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:28:08.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:08 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2493048143' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:28:08.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:08 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/531870795' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:28:08.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:08 smithi071 ceph-mon[144544]: from='client.34670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:28:08.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:08 smithi192 ceph-mon[108617]: from='client.34654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:28:08.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:08 smithi192 ceph-mon[108617]: pgmap v444: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:08.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:08 smithi192 ceph-mon[108617]: from='client.34658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:28:08.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:08 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2493048143' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:28:08.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:08 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/531870795' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:28:08.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:08 smithi192 ceph-mon[108617]: from='client.34670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:28:09.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:09 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2597632538' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:28:09.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:09 smithi071 ceph-mon[144544]: pgmap v445: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:09.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:09 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2597632538' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:28:09.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:09 smithi192 ceph-mon[108617]: pgmap v445: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:12.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:11 smithi071 ceph-mon[144544]: pgmap v446: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:12.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:11 smithi192 ceph-mon[108617]: pgmap v446: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:14.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:13 smithi071 ceph-mon[144544]: pgmap v447: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:14.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:13 smithi192 ceph-mon[108617]: pgmap v447: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:16.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:15 smithi192 ceph-mon[108617]: pgmap v448: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:16.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:15 smithi071 ceph-mon[144544]: pgmap v448: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:18.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:17 smithi071 ceph-mon[144544]: pgmap v449: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:18.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:17 smithi192 ceph-mon[108617]: pgmap v449: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:20.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:19 smithi071 ceph-mon[144544]: pgmap v450: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:20.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:28:20.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:19 smithi192 ceph-mon[108617]: pgmap v450: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:20.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:28:22.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:21 smithi071 ceph-mon[144544]: pgmap v451: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:22.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:21 smithi192 ceph-mon[108617]: pgmap v451: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:24.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:23 smithi071 ceph-mon[144544]: pgmap v452: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:24.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:23 smithi192 ceph-mon[108617]: pgmap v452: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:26.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:25 smithi192 ceph-mon[108617]: pgmap v453: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:26.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:25 smithi071 ceph-mon[144544]: pgmap v453: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:28.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:27 smithi071 ceph-mon[144544]: pgmap v454: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:28.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:27 smithi192 ceph-mon[108617]: pgmap v454: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:30.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:29 smithi071 ceph-mon[144544]: pgmap v455: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:30.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:29 smithi192 ceph-mon[108617]: pgmap v455: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:32.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:31 smithi071 ceph-mon[144544]: pgmap v456: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:32.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:31 smithi192 ceph-mon[108617]: pgmap v456: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:34.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:33 smithi071 ceph-mon[144544]: pgmap v457: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:34.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:33 smithi192 ceph-mon[108617]: pgmap v457: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:35.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:34 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:28:35.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:34 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:28:36.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:35 smithi071 ceph-mon[144544]: pgmap v458: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:36.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:35 smithi192 ceph-mon[108617]: pgmap v458: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:38.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:38 smithi192 ceph-mon[108617]: pgmap v459: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:38.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.646+0000 7fa6878e7700 1 -- 172.21.15.71:0/4264607081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6800ffb40 msgr2=0x7fa6800fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:38.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.646+0000 7fa6878e7700 1 --2- 172.21.15.71:0/4264607081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6800ffb40 0x7fa6800fffc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa674009910 tx=0x7fa674009c20 comp rx=0 tx=0).stop 2024-04-23T02:28:38.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.646+0000 7fa6878e7700 1 -- 172.21.15.71:0/4264607081 shutdown_connections 2024-04-23T02:28:38.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.646+0000 7fa6878e7700 1 --2- 172.21.15.71:0/4264607081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6800ffb40 0x7fa6800fffc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:38.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.646+0000 7fa6878e7700 1 --2- 172.21.15.71:0/4264607081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa680105bf0 0x7fa680105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:38.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.646+0000 7fa6878e7700 1 -- 172.21.15.71:0/4264607081 >> 172.21.15.71:0/4264607081 conn(0x7fa6800fb610 msgr2=0x7fa6800fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:38.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.646+0000 7fa6878e7700 1 -- 172.21.15.71:0/4264607081 shutdown_connections 2024-04-23T02:28:38.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.646+0000 7fa6878e7700 1 -- 172.21.15.71:0/4264607081 wait complete. 2024-04-23T02:28:38.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.647+0000 7fa6878e7700 1 Processor -- start 2024-04-23T02:28:38.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.647+0000 7fa6878e7700 1 -- start start 2024-04-23T02:28:38.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.647+0000 7fa6878e7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6800ffb40 0x7fa680195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:38.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.647+0000 7fa6878e7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa680105bf0 0x7fa680195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:38.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.647+0000 7fa6878e7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa680196240 con 0x7fa680105bf0 2024-04-23T02:28:38.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.647+0000 7fa6878e7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa680199fd0 con 0x7fa6800ffb40 2024-04-23T02:28:38.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.647+0000 7fa684e82700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa680105bf0 0x7fa680195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:38.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.647+0000 7fa684e82700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa680105bf0 0x7fa680195b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48110/0 (socket says 172.21.15.71:48110) 2024-04-23T02:28:38.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.647+0000 7fa684e82700 1 -- 172.21.15.71:0/873942740 learned_addr learned my addr 172.21.15.71:0/873942740 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:28:38.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.648+0000 7fa685683700 1 --2- 172.21.15.71:0/873942740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6800ffb40 0x7fa680195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:38.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.648+0000 7fa684e82700 1 -- 172.21.15.71:0/873942740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6800ffb40 msgr2=0x7fa680195620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:38.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.648+0000 7fa684e82700 1 --2- 172.21.15.71:0/873942740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6800ffb40 0x7fa680195620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:38.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.648+0000 7fa684e82700 1 -- 172.21.15.71:0/873942740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6740095f0 con 0x7fa680105bf0 2024-04-23T02:28:38.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.648+0000 7fa684e82700 1 --2- 172.21.15.71:0/873942740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa680105bf0 0x7fa680195b60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa674006010 tx=0x7fa67400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:38.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.648+0000 7fa6727fc700 1 -- 172.21.15.71:0/873942740 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa67401f070 con 0x7fa680105bf0 2024-04-23T02:28:38.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.648+0000 7fa6727fc700 1 -- 172.21.15.71:0/873942740 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa674019610 con 0x7fa680105bf0 2024-04-23T02:28:38.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.648+0000 7fa6727fc700 1 -- 172.21.15.71:0/873942740 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa67400f670 con 0x7fa680105bf0 2024-04-23T02:28:38.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.648+0000 7fa6878e7700 1 -- 172.21.15.71:0/873942740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa68019a250 con 0x7fa680105bf0 2024-04-23T02:28:38.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.648+0000 7fa6878e7700 1 -- 172.21.15.71:0/873942740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68019a740 con 0x7fa680105bf0 2024-04-23T02:28:38.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.649+0000 7fa6878e7700 1 -- 172.21.15.71:0/873942740 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68004f2e0 con 0x7fa680105bf0 2024-04-23T02:28:38.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.649+0000 7fa6727fc700 1 -- 172.21.15.71:0/873942740 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa674019bf0 con 0x7fa680105bf0 2024-04-23T02:28:38.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.650+0000 7fa6727fc700 1 --2- 172.21.15.71:0/873942740 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa66c06cce0 0x7fa66c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:38.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.650+0000 7fa685683700 1 --2- 172.21.15.71:0/873942740 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa66c06cce0 0x7fa66c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:38.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.650+0000 7fa6727fc700 1 -- 172.21.15.71:0/873942740 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa67408f920 con 0x7fa680105bf0 2024-04-23T02:28:38.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.650+0000 7fa685683700 1 --2- 172.21.15.71:0/873942740 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa66c06cce0 0x7fa66c06f1a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa67c005950 tx=0x7fa67c0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:38.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.655+0000 7fa6727fc700 1 -- 172.21.15.71:0/873942740 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa674059a10 con 0x7fa680105bf0 2024-04-23T02:28:38.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:38 smithi071 ceph-mon[144544]: pgmap v459: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:38.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.867+0000 7fa6878e7700 1 -- 172.21.15.71:0/873942740 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa68019ab20 con 0x7fa66c06cce0 2024-04-23T02:28:38.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.868+0000 7fa6727fc700 1 -- 172.21.15.71:0/873942740 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa68019ab20 con 0x7fa66c06cce0 2024-04-23T02:28:38.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.870+0000 7fa6878e7700 1 -- 172.21.15.71:0/873942740 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa66c06cce0 msgr2=0x7fa66c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:38.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.870+0000 7fa6878e7700 1 --2- 172.21.15.71:0/873942740 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa66c06cce0 0x7fa66c06f1a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa67c005950 tx=0x7fa67c0058e0 comp rx=0 tx=0).stop 2024-04-23T02:28:38.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.870+0000 7fa6878e7700 1 -- 172.21.15.71:0/873942740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa680105bf0 msgr2=0x7fa680195b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:38.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.870+0000 7fa6878e7700 1 --2- 172.21.15.71:0/873942740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa680105bf0 0x7fa680195b60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa674006010 tx=0x7fa67400be20 comp rx=0 tx=0).stop 2024-04-23T02:28:38.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.870+0000 7fa6878e7700 1 -- 172.21.15.71:0/873942740 shutdown_connections 2024-04-23T02:28:38.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.870+0000 7fa6878e7700 1 --2- 172.21.15.71:0/873942740 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa66c06cce0 0x7fa66c06f1a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:38.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.870+0000 7fa6878e7700 1 --2- 172.21.15.71:0/873942740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6800ffb40 0x7fa680195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:38.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.870+0000 7fa6878e7700 1 --2- 172.21.15.71:0/873942740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa680105bf0 0x7fa680195b60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:38.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.870+0000 7fa6878e7700 1 -- 172.21.15.71:0/873942740 >> 172.21.15.71:0/873942740 conn(0x7fa6800fb610 msgr2=0x7fa6800fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:38.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.870+0000 7fa6878e7700 1 -- 172.21.15.71:0/873942740 shutdown_connections 2024-04-23T02:28:38.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.870+0000 7fa6878e7700 1 -- 172.21.15.71:0/873942740 wait complete. 2024-04-23T02:28:38.885 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:28:38.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.973+0000 7f373822f700 1 -- 172.21.15.71:0/1035556128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37300ffc20 msgr2=0x7f3730100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:38.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.973+0000 7f373822f700 1 --2- 172.21.15.71:0/1035556128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37300ffc20 0x7f3730100080 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3720009910 tx=0x7f3720009c20 comp rx=0 tx=0).stop 2024-04-23T02:28:38.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.973+0000 7f373822f700 1 -- 172.21.15.71:0/1035556128 shutdown_connections 2024-04-23T02:28:38.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.973+0000 7f373822f700 1 --2- 172.21.15.71:0/1035556128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37300ffc20 0x7f3730100080 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:38.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.973+0000 7f373822f700 1 --2- 172.21.15.71:0/1035556128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3730105bd0 0x7f3730105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:38.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.973+0000 7f373822f700 1 -- 172.21.15.71:0/1035556128 >> 172.21.15.71:0/1035556128 conn(0x7f3730075670 msgr2=0x7f3730075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:38.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.973+0000 7f373822f700 1 -- 172.21.15.71:0/1035556128 shutdown_connections 2024-04-23T02:28:38.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.973+0000 7f373822f700 1 -- 172.21.15.71:0/1035556128 wait complete. 2024-04-23T02:28:38.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.974+0000 7f373822f700 1 Processor -- start 2024-04-23T02:28:38.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.974+0000 7f373822f700 1 -- start start 2024-04-23T02:28:38.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.974+0000 7f373822f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37300ffc20 0x7f37301976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:38.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.974+0000 7f373822f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3730105bd0 0x7f3730197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:38.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.974+0000 7f373822f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3730198190 con 0x7f3730105bd0 2024-04-23T02:28:38.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.974+0000 7f373822f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37301982d0 con 0x7f37300ffc20 2024-04-23T02:28:38.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.974+0000 7f37357ca700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3730105bd0 0x7f3730197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:38.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.974+0000 7f37357ca700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3730105bd0 0x7f3730197c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48124/0 (socket says 172.21.15.71:48124) 2024-04-23T02:28:38.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.974+0000 7f37357ca700 1 -- 172.21.15.71:0/2512569963 learned_addr learned my addr 172.21.15.71:0/2512569963 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:28:38.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.975+0000 7f37357ca700 1 -- 172.21.15.71:0/2512569963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37300ffc20 msgr2=0x7f37301976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:38.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.975+0000 7f3735fcb700 1 --2- 172.21.15.71:0/2512569963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37300ffc20 0x7f37301976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:38.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.975+0000 7f37357ca700 1 --2- 172.21.15.71:0/2512569963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37300ffc20 0x7f37301976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:38.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.975+0000 7f37357ca700 1 -- 172.21.15.71:0/2512569963 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37200095f0 con 0x7f3730105bd0 2024-04-23T02:28:38.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.975+0000 7f3735fcb700 1 --2- 172.21.15.71:0/2512569963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37300ffc20 0x7f37301976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:28:38.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.975+0000 7f37357ca700 1 --2- 172.21.15.71:0/2512569963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3730105bd0 0x7f3730197c00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3720006010 tx=0x7f372000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:38.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.975+0000 7f3726ffd700 1 -- 172.21.15.71:0/2512569963 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f372001d070 con 0x7f3730105bd0 2024-04-23T02:28:38.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.975+0000 7f373822f700 1 -- 172.21.15.71:0/2512569963 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37301a0730 con 0x7f3730105bd0 2024-04-23T02:28:38.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.975+0000 7f3726ffd700 1 -- 172.21.15.71:0/2512569963 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f372000f460 con 0x7f3730105bd0 2024-04-23T02:28:38.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.975+0000 7f3726ffd700 1 -- 172.21.15.71:0/2512569963 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3720021640 con 0x7f3730105bd0 2024-04-23T02:28:38.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.975+0000 7f373822f700 1 -- 172.21.15.71:0/2512569963 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37301a0ae0 con 0x7f3730105bd0 2024-04-23T02:28:38.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.976+0000 7f373822f700 1 -- 172.21.15.71:0/2512569963 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37301080d0 con 0x7f3730105bd0 2024-04-23T02:28:38.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.976+0000 7f3726ffd700 1 -- 172.21.15.71:0/2512569963 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f372000fa20 con 0x7f3730105bd0 2024-04-23T02:28:38.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.977+0000 7f3726ffd700 1 --2- 172.21.15.71:0/2512569963 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f371c06cce0 0x7f371c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:38.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.977+0000 7f3726ffd700 1 -- 172.21.15.71:0/2512569963 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f372008d8c0 con 0x7f3730105bd0 2024-04-23T02:28:38.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.977+0000 7f3735fcb700 1 --2- 172.21.15.71:0/2512569963 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f371c06cce0 0x7f371c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:38.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.977+0000 7f3735fcb700 1 --2- 172.21.15.71:0/2512569963 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f371c06cce0 0x7f371c06f1a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f372c005fd0 tx=0x7f372c005e20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:38.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:38.981+0000 7f3726ffd700 1 -- 172.21.15.71:0/2512569963 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3720057930 con 0x7f3730105bd0 2024-04-23T02:28:39.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.191+0000 7f373822f700 1 -- 172.21.15.71:0/2512569963 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f37300611d0 con 0x7f371c06cce0 2024-04-23T02:28:39.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.193+0000 7f3726ffd700 1 -- 172.21.15.71:0/2512569963 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f37300611d0 con 0x7f371c06cce0 2024-04-23T02:28:39.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.194+0000 7f373822f700 1 -- 172.21.15.71:0/2512569963 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f371c06cce0 msgr2=0x7f371c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:39.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.195+0000 7f373822f700 1 --2- 172.21.15.71:0/2512569963 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f371c06cce0 0x7f371c06f1a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f372c005fd0 tx=0x7f372c005e20 comp rx=0 tx=0).stop 2024-04-23T02:28:39.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.195+0000 7f373822f700 1 -- 172.21.15.71:0/2512569963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3730105bd0 msgr2=0x7f3730197c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:39.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.195+0000 7f373822f700 1 --2- 172.21.15.71:0/2512569963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3730105bd0 0x7f3730197c00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3720006010 tx=0x7f372000be20 comp rx=0 tx=0).stop 2024-04-23T02:28:39.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.195+0000 7f373822f700 1 -- 172.21.15.71:0/2512569963 shutdown_connections 2024-04-23T02:28:39.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.195+0000 7f373822f700 1 --2- 172.21.15.71:0/2512569963 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f371c06cce0 0x7f371c06f1a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:39.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.195+0000 7f373822f700 1 --2- 172.21.15.71:0/2512569963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37300ffc20 0x7f37301976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:39.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.195+0000 7f373822f700 1 --2- 172.21.15.71:0/2512569963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3730105bd0 0x7f3730197c00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:39.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.195+0000 7f373822f700 1 -- 172.21.15.71:0/2512569963 >> 172.21.15.71:0/2512569963 conn(0x7f3730075670 msgr2=0x7f37300fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:39.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.195+0000 7f373822f700 1 -- 172.21.15.71:0/2512569963 shutdown_connections 2024-04-23T02:28:39.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.195+0000 7f373822f700 1 -- 172.21.15.71:0/2512569963 wait complete. 2024-04-23T02:28:39.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.297+0000 7f9b8b4d5700 1 -- 172.21.15.71:0/2787937114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b84100ea0 msgr2=0x7f9b84101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:39.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.297+0000 7f9b8b4d5700 1 --2- 172.21.15.71:0/2787937114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b84100ea0 0x7f9b84101280 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009910 tx=0x7f9b74009c20 comp rx=0 tx=0).stop 2024-04-23T02:28:39.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.297+0000 7f9b8b4d5700 1 -- 172.21.15.71:0/2787937114 shutdown_connections 2024-04-23T02:28:39.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.297+0000 7f9b8b4d5700 1 --2- 172.21.15.71:0/2787937114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b84101850 0x7f9b841058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:39.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.297+0000 7f9b8b4d5700 1 --2- 172.21.15.71:0/2787937114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b84100ea0 0x7f9b84101280 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:39.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.297+0000 7f9b8b4d5700 1 -- 172.21.15.71:0/2787937114 >> 172.21.15.71:0/2787937114 conn(0x7f9b840fc750 msgr2=0x7f9b840feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:39.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.297+0000 7f9b8b4d5700 1 -- 172.21.15.71:0/2787937114 shutdown_connections 2024-04-23T02:28:39.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.297+0000 7f9b8b4d5700 1 -- 172.21.15.71:0/2787937114 wait complete. 2024-04-23T02:28:39.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.298+0000 7f9b8b4d5700 1 Processor -- start 2024-04-23T02:28:39.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.298+0000 7f9b8b4d5700 1 -- start start 2024-04-23T02:28:39.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.298+0000 7f9b8b4d5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b84100ea0 0x7f9b8419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:39.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.298+0000 7f9b8b4d5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b84101850 0x7f9b8419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:39.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.298+0000 7f9b8b4d5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8419d130 con 0x7f9b84101850 2024-04-23T02:28:39.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.298+0000 7f9b8b4d5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b841965e0 con 0x7f9b84100ea0 2024-04-23T02:28:39.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.298+0000 7f9b88a70700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b84101850 0x7f9b8419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:39.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.298+0000 7f9b88a70700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b84101850 0x7f9b8419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48128/0 (socket says 172.21.15.71:48128) 2024-04-23T02:28:39.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.298+0000 7f9b88a70700 1 -- 172.21.15.71:0/2819919470 learned_addr learned my addr 172.21.15.71:0/2819919470 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:28:39.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.298+0000 7f9b88a70700 1 -- 172.21.15.71:0/2819919470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b84100ea0 msgr2=0x7f9b8419c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:39.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.298+0000 7f9b89271700 1 --2- 172.21.15.71:0/2819919470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b84100ea0 0x7f9b8419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:39.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.299+0000 7f9b88a70700 1 --2- 172.21.15.71:0/2819919470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b84100ea0 0x7f9b8419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:39.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.299+0000 7f9b88a70700 1 -- 172.21.15.71:0/2819919470 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b740095f0 con 0x7f9b84101850 2024-04-23T02:28:39.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.299+0000 7f9b88a70700 1 --2- 172.21.15.71:0/2819919470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b84101850 0x7f9b8419caa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000d8d0 tx=0x7f9b8000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:39.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.299+0000 7f9b7a7fc700 1 -- 172.21.15.71:0/2819919470 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b800041f0 con 0x7f9b84101850 2024-04-23T02:28:39.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.299+0000 7f9b7a7fc700 1 -- 172.21.15.71:0/2819919470 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9b80007620 con 0x7f9b84101850 2024-04-23T02:28:39.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.299+0000 7f9b8b4d5700 1 -- 172.21.15.71:0/2819919470 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b841968c0 con 0x7f9b84101850 2024-04-23T02:28:39.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.299+0000 7f9b7a7fc700 1 -- 172.21.15.71:0/2819919470 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b80003b50 con 0x7f9b84101850 2024-04-23T02:28:39.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.299+0000 7f9b8b4d5700 1 -- 172.21.15.71:0/2819919470 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b84196e10 con 0x7f9b84101850 2024-04-23T02:28:39.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.300+0000 7f9b8b4d5700 1 -- 172.21.15.71:0/2819919470 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b84109240 con 0x7f9b84101850 2024-04-23T02:28:39.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.300+0000 7f9b7a7fc700 1 -- 172.21.15.71:0/2819919470 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9b80004350 con 0x7f9b84101850 2024-04-23T02:28:39.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.301+0000 7f9b7a7fc700 1 --2- 172.21.15.71:0/2819919470 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9b7006cce0 0x7f9b7006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:39.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.301+0000 7f9b7a7fc700 1 -- 172.21.15.71:0/2819919470 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9b8008c060 con 0x7f9b84101850 2024-04-23T02:28:39.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.301+0000 7f9b89271700 1 --2- 172.21.15.71:0/2819919470 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9b7006cce0 0x7f9b7006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:39.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.301+0000 7f9b89271700 1 --2- 172.21.15.71:0/2819919470 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9b7006cce0 0x7f9b7006f1a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400b5c0 tx=0x7f9b74005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:39.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.305+0000 7f9b7a7fc700 1 -- 172.21.15.71:0/2819919470 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9b80059be0 con 0x7f9b84101850 2024-04-23T02:28:39.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.507+0000 7f9b8b4d5700 1 -- 172.21.15.71:0/2819919470 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b84197080 con 0x7f9b7006cce0 2024-04-23T02:28:39.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.517+0000 7f9b7a7fc700 1 -- 172.21.15.71:0/2819919470 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9b84197080 con 0x7f9b7006cce0 2024-04-23T02:28:39.521 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:28:39.521 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (12m) 6m ago 23m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:28:39.521 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (23m) 6m ago 23m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:28:39.522 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (22m) 5m ago 22m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:28:39.522 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (10m) 6m ago 23m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:28:39.522 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (10m) 5m ago 22m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:28:39.522 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (12m) 6m ago 23m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:28:39.522 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (6m) 6m ago 17m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:28:39.522 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (6m) 6m ago 17m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:28:39.522 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (17m) 5m ago 17m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:28:39.522 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (5m) 5m ago 17m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:28:39.522 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (14m) 6m ago 25m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:28:39.522 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (13m) 5m ago 22m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:28:39.522 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (10m) 6m ago 25m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:28:39.522 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (10m) 5m ago 22m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:28:39.522 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (13m) 6m ago 23m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:28:39.522 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (13m) 5m ago 22m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:28:39.523 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (9m) 6m ago 21m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:28:39.523 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (9m) 6m ago 21m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:28:39.523 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (8m) 6m ago 20m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:28:39.523 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (7m) 5m ago 20m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:28:39.523 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (7m) 5m ago 19m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:28:39.523 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (6m) 5m ago 19m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:28:39.523 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (12m) 6m ago 23m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:28:39.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.519+0000 7f9b8b4d5700 1 -- 172.21.15.71:0/2819919470 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9b7006cce0 msgr2=0x7f9b7006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:39.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.519+0000 7f9b8b4d5700 1 --2- 172.21.15.71:0/2819919470 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9b7006cce0 0x7f9b7006f1a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400b5c0 tx=0x7f9b74005fd0 comp rx=0 tx=0).stop 2024-04-23T02:28:39.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.519+0000 7f9b8b4d5700 1 -- 172.21.15.71:0/2819919470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b84101850 msgr2=0x7f9b8419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:39.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.519+0000 7f9b8b4d5700 1 --2- 172.21.15.71:0/2819919470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b84101850 0x7f9b8419caa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000d8d0 tx=0x7f9b8000dc90 comp rx=0 tx=0).stop 2024-04-23T02:28:39.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.520+0000 7f9b8b4d5700 1 -- 172.21.15.71:0/2819919470 shutdown_connections 2024-04-23T02:28:39.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.520+0000 7f9b8b4d5700 1 --2- 172.21.15.71:0/2819919470 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9b7006cce0 0x7f9b7006f1a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:39.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.520+0000 7f9b8b4d5700 1 --2- 172.21.15.71:0/2819919470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b84100ea0 0x7f9b8419c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:39.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.520+0000 7f9b8b4d5700 1 --2- 172.21.15.71:0/2819919470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b84101850 0x7f9b8419caa0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:39.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.520+0000 7f9b8b4d5700 1 -- 172.21.15.71:0/2819919470 >> 172.21.15.71:0/2819919470 conn(0x7f9b840fc750 msgr2=0x7f9b840fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:39.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.520+0000 7f9b8b4d5700 1 -- 172.21.15.71:0/2819919470 shutdown_connections 2024-04-23T02:28:39.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.520+0000 7f9b8b4d5700 1 -- 172.21.15.71:0/2819919470 wait complete. 2024-04-23T02:28:39.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.622+0000 7f6ed34c8700 1 -- 172.21.15.71:0/166014640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ecc0ffb40 msgr2=0x7f6ecc0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:39.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.622+0000 7f6ed34c8700 1 --2- 172.21.15.71:0/166014640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ecc0ffb40 0x7f6ecc0fffc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009910 tx=0x7f6ec8009c20 comp rx=0 tx=0).stop 2024-04-23T02:28:39.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.622+0000 7f6ed34c8700 1 -- 172.21.15.71:0/166014640 shutdown_connections 2024-04-23T02:28:39.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.622+0000 7f6ed34c8700 1 --2- 172.21.15.71:0/166014640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ecc0ffb40 0x7f6ecc0fffc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:39.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.622+0000 7f6ed34c8700 1 --2- 172.21.15.71:0/166014640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ecc105bf0 0x7f6ecc105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:39.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.622+0000 7f6ed34c8700 1 -- 172.21.15.71:0/166014640 >> 172.21.15.71:0/166014640 conn(0x7f6ecc0fb610 msgr2=0x7f6ecc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:39.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.622+0000 7f6ed34c8700 1 -- 172.21.15.71:0/166014640 shutdown_connections 2024-04-23T02:28:39.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.622+0000 7f6ed34c8700 1 -- 172.21.15.71:0/166014640 wait complete. 2024-04-23T02:28:39.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.623+0000 7f6ed34c8700 1 Processor -- start 2024-04-23T02:28:39.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.623+0000 7f6ed34c8700 1 -- start start 2024-04-23T02:28:39.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.623+0000 7f6ed34c8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ecc0ffb40 0x7f6ecc19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:39.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.623+0000 7f6ed34c8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ecc105bf0 0x7f6ecc19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:39.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.623+0000 7f6ed34c8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc19d030 con 0x7f6ecc0ffb40 2024-04-23T02:28:39.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.623+0000 7f6ed34c8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc196570 con 0x7f6ecc105bf0 2024-04-23T02:28:39.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.623+0000 7f6ed1264700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ecc0ffb40 0x7f6ecc19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:39.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.623+0000 7f6ed1264700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ecc0ffb40 0x7f6ecc19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48138/0 (socket says 172.21.15.71:48138) 2024-04-23T02:28:39.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.623+0000 7f6ed1264700 1 -- 172.21.15.71:0/3701023285 learned_addr learned my addr 172.21.15.71:0/3701023285 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:28:39.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.624+0000 7f6ed0a63700 1 --2- 172.21.15.71:0/3701023285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ecc105bf0 0x7f6ecc19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:39.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.624+0000 7f6ed1264700 1 -- 172.21.15.71:0/3701023285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ecc105bf0 msgr2=0x7f6ecc19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:39.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.624+0000 7f6ed1264700 1 --2- 172.21.15.71:0/3701023285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ecc105bf0 0x7f6ecc19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:39.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.624+0000 7f6ed1264700 1 -- 172.21.15.71:0/3701023285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec80095f0 con 0x7f6ecc0ffb40 2024-04-23T02:28:39.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.624+0000 7f6ed1264700 1 --2- 172.21.15.71:0/3701023285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ecc0ffb40 0x7f6ecc19c4f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00c930 tx=0x7f6ebc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:39.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.624+0000 7f6ec27fc700 1 -- 172.21.15.71:0/3701023285 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ebc007ad0 con 0x7f6ecc0ffb40 2024-04-23T02:28:39.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.624+0000 7f6ed34c8700 1 -- 172.21.15.71:0/3701023285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ecc196850 con 0x7f6ecc0ffb40 2024-04-23T02:28:39.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.624+0000 7f6ec27fc700 1 -- 172.21.15.71:0/3701023285 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6ebc007c30 con 0x7f6ecc0ffb40 2024-04-23T02:28:39.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.624+0000 7f6ec27fc700 1 -- 172.21.15.71:0/3701023285 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ebc018630 con 0x7f6ecc0ffb40 2024-04-23T02:28:39.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.624+0000 7f6ed34c8700 1 -- 172.21.15.71:0/3701023285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ecc196da0 con 0x7f6ecc0ffb40 2024-04-23T02:28:39.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.625+0000 7f6ed34c8700 1 -- 172.21.15.71:0/3701023285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ecc04f2e0 con 0x7f6ecc0ffb40 2024-04-23T02:28:39.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.625+0000 7f6ec27fc700 1 -- 172.21.15.71:0/3701023285 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6ebc00f450 con 0x7f6ecc0ffb40 2024-04-23T02:28:39.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.626+0000 7f6ec27fc700 1 --2- 172.21.15.71:0/3701023285 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6eb806cce0 0x7f6eb806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:39.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.626+0000 7f6ec27fc700 1 -- 172.21.15.71:0/3701023285 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6ebc08c870 con 0x7f6ecc0ffb40 2024-04-23T02:28:39.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.626+0000 7f6ed0a63700 1 --2- 172.21.15.71:0/3701023285 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6eb806cce0 0x7f6eb806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:39.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.626+0000 7f6ed0a63700 1 --2- 172.21.15.71:0/3701023285 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6eb806cce0 0x7f6eb806f1a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0feb60 tx=0x7f6ec801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:39.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.630+0000 7f6ec27fc700 1 -- 172.21.15.71:0/3701023285 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6ebc05a380 con 0x7f6ecc0ffb40 2024-04-23T02:28:39.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.903+0000 7f6ed34c8700 1 -- 172.21.15.71:0/3701023285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6ecc197080 con 0x7f6ecc0ffb40 2024-04-23T02:28:39.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.904+0000 7f6ec27fc700 1 -- 172.21.15.71:0/3701023285 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f6ebc059f10 con 0x7f6ecc0ffb40 2024-04-23T02:28:39.907 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:28:39.907 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:28:39.907 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:28:39.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:28:39.908 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:28:39.908 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:28:39.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:28:39.908 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:28:39.908 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:28:39.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:28:39.908 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:28:39.908 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:28:39.908 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:28:39.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:28:39.908 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:28:39.908 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:28:39.908 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:28:39.908 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:28:39.909 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:28:39.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.906+0000 7f6ed34c8700 1 -- 172.21.15.71:0/3701023285 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6eb806cce0 msgr2=0x7f6eb806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:39.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.906+0000 7f6ed34c8700 1 --2- 172.21.15.71:0/3701023285 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6eb806cce0 0x7f6eb806f1a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0feb60 tx=0x7f6ec801a040 comp rx=0 tx=0).stop 2024-04-23T02:28:39.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.906+0000 7f6ed34c8700 1 -- 172.21.15.71:0/3701023285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ecc0ffb40 msgr2=0x7f6ecc19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:39.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.906+0000 7f6ed34c8700 1 --2- 172.21.15.71:0/3701023285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ecc0ffb40 0x7f6ecc19c4f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00c930 tx=0x7f6ebc00ccf0 comp rx=0 tx=0).stop 2024-04-23T02:28:39.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.906+0000 7f6ed34c8700 1 -- 172.21.15.71:0/3701023285 shutdown_connections 2024-04-23T02:28:39.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.906+0000 7f6ed34c8700 1 --2- 172.21.15.71:0/3701023285 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6eb806cce0 0x7f6eb806f1a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:39.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.906+0000 7f6ed34c8700 1 --2- 172.21.15.71:0/3701023285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ecc0ffb40 0x7f6ecc19c4f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:39.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.906+0000 7f6ed34c8700 1 --2- 172.21.15.71:0/3701023285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ecc105bf0 0x7f6ecc19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:39.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.906+0000 7f6ed34c8700 1 -- 172.21.15.71:0/3701023285 >> 172.21.15.71:0/3701023285 conn(0x7f6ecc0fb610 msgr2=0x7f6ecc068ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:39.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.906+0000 7f6ed34c8700 1 -- 172.21.15.71:0/3701023285 shutdown_connections 2024-04-23T02:28:39.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:39.906+0000 7f6ed34c8700 1 -- 172.21.15.71:0/3701023285 wait complete. 2024-04-23T02:28:40.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.008+0000 7f9656180700 1 -- 172.21.15.71:0/1780383109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9650068580 msgr2=0x7f96500689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:40.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.008+0000 7f9656180700 1 --2- 172.21.15.71:0/1780383109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9650068580 0x7f96500689e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9644009910 tx=0x7f9644009c20 comp rx=0 tx=0).stop 2024-04-23T02:28:40.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.008+0000 7f9656180700 1 -- 172.21.15.71:0/1780383109 shutdown_connections 2024-04-23T02:28:40.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.008+0000 7f9656180700 1 --2- 172.21.15.71:0/1780383109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9650068580 0x7f96500689e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.008+0000 7f9656180700 1 --2- 172.21.15.71:0/1780383109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9650106cd0 0x7f96501070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.008+0000 7f9656180700 1 -- 172.21.15.71:0/1780383109 >> 172.21.15.71:0/1780383109 conn(0x7f9650075670 msgr2=0x7f9650075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:40.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.009+0000 7f9656180700 1 -- 172.21.15.71:0/1780383109 shutdown_connections 2024-04-23T02:28:40.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.009+0000 7f9656180700 1 -- 172.21.15.71:0/1780383109 wait complete. 2024-04-23T02:28:40.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.009+0000 7f9656180700 1 Processor -- start 2024-04-23T02:28:40.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.009+0000 7f9656180700 1 -- start start 2024-04-23T02:28:40.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.009+0000 7f9656180700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9650068580 0x7f965019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:40.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.009+0000 7f9656180700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9650106cd0 0x7f965019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:40.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.009+0000 7f9656180700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965019d110 con 0x7f9650068580 2024-04-23T02:28:40.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.010+0000 7f9656180700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96501965c0 con 0x7f9650106cd0 2024-04-23T02:28:40.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.010+0000 7f964f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9650068580 0x7f965019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:40.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.010+0000 7f964f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9650068580 0x7f965019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48156/0 (socket says 172.21.15.71:48156) 2024-04-23T02:28:40.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.010+0000 7f964f7fe700 1 -- 172.21.15.71:0/2422502111 learned_addr learned my addr 172.21.15.71:0/2422502111 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:28:40.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.010+0000 7f964effd700 1 --2- 172.21.15.71:0/2422502111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9650106cd0 0x7f965019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:40.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.010+0000 7f964f7fe700 1 -- 172.21.15.71:0/2422502111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9650106cd0 msgr2=0x7f965019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:40.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.010+0000 7f964f7fe700 1 --2- 172.21.15.71:0/2422502111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9650106cd0 0x7f965019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.010+0000 7f964f7fe700 1 -- 172.21.15.71:0/2422502111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96440095f0 con 0x7f9650068580 2024-04-23T02:28:40.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.010+0000 7f964f7fe700 1 --2- 172.21.15.71:0/2422502111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9650068580 0x7f965019c540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f964000c8a0 tx=0x7f964000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:40.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.010+0000 7f964cff9700 1 -- 172.21.15.71:0/2422502111 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96400078e0 con 0x7f9650068580 2024-04-23T02:28:40.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.010+0000 7f9656180700 1 -- 172.21.15.71:0/2422502111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96501968a0 con 0x7f9650068580 2024-04-23T02:28:40.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.010+0000 7f964cff9700 1 -- 172.21.15.71:0/2422502111 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f964000f450 con 0x7f9650068580 2024-04-23T02:28:40.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.010+0000 7f964cff9700 1 -- 172.21.15.71:0/2422502111 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f964000e5e0 con 0x7f9650068580 2024-04-23T02:28:40.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.010+0000 7f9656180700 1 -- 172.21.15.71:0/2422502111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9650196df0 con 0x7f9650068580 2024-04-23T02:28:40.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.011+0000 7f9656180700 1 -- 172.21.15.71:0/2422502111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96501091d0 con 0x7f9650068580 2024-04-23T02:28:40.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.012+0000 7f964cff9700 1 -- 172.21.15.71:0/2422502111 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f964000f5c0 con 0x7f9650068580 2024-04-23T02:28:40.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.012+0000 7f964cff9700 1 --2- 172.21.15.71:0/2422502111 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f963806cce0 0x7f963806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:40.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.012+0000 7f964cff9700 1 -- 172.21.15.71:0/2422502111 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f964008c1d0 con 0x7f9650068580 2024-04-23T02:28:40.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.012+0000 7f964effd700 1 --2- 172.21.15.71:0/2422502111 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f963806cce0 0x7f963806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:40.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.013+0000 7f964effd700 1 --2- 172.21.15.71:0/2422502111 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f963806cce0 0x7f963806f1a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f96500fe260 tx=0x7f964401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:40.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.017+0000 7f964cff9700 1 -- 172.21.15.71:0/2422502111 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9640059cd0 con 0x7f9650068580 2024-04-23T02:28:40.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.251+0000 7f9656180700 1 -- 172.21.15.71:0/2422502111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9650197070 con 0x7f9650068580 2024-04-23T02:28:40.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.251+0000 7f964cff9700 1 -- 172.21.15.71:0/2422502111 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f9640059860 con 0x7f9650068580 2024-04-23T02:28:40.255 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:28:40.255 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:28:40.255 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:28:40.255 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:28:40.255 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:28:40.255 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:28:40.255 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:28:40.255 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:28:40.255 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:28:40.255 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:28:40.256 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:28:40.256 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:28:40.256 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:28:40.256 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:28:40.256 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:28:40.256 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:28:40.256 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:28:40.256 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:28:40.256 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:28:40.256 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:28:40.257 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:28:40.257 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:28:40.257 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:28:40.257 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:28:40.257 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:28:40.257 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:28:40.257 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:28:40.257 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:28:40.257 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:28:40.257 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:28:40.257 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:28:40.257 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:28:40.257 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:28:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:28:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:28:40.258 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:28:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:28:40.258 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:28:40.258 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:28:40.258 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:28:40.258 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:28:40.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.253+0000 7f9656180700 1 -- 172.21.15.71:0/2422502111 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f963806cce0 msgr2=0x7f963806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:40.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.253+0000 7f9656180700 1 --2- 172.21.15.71:0/2422502111 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f963806cce0 0x7f963806f1a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f96500fe260 tx=0x7f964401a040 comp rx=0 tx=0).stop 2024-04-23T02:28:40.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.253+0000 7f9656180700 1 -- 172.21.15.71:0/2422502111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9650068580 msgr2=0x7f965019c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:40.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.253+0000 7f9656180700 1 --2- 172.21.15.71:0/2422502111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9650068580 0x7f965019c540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f964000c8a0 tx=0x7f964000cbb0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.253+0000 7f9656180700 1 -- 172.21.15.71:0/2422502111 shutdown_connections 2024-04-23T02:28:40.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.253+0000 7f9656180700 1 --2- 172.21.15.71:0/2422502111 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f963806cce0 0x7f963806f1a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.253+0000 7f9656180700 1 --2- 172.21.15.71:0/2422502111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9650068580 0x7f965019c540 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.253+0000 7f9656180700 1 --2- 172.21.15.71:0/2422502111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9650106cd0 0x7f965019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.253+0000 7f9656180700 1 -- 172.21.15.71:0/2422502111 >> 172.21.15.71:0/2422502111 conn(0x7f9650075670 msgr2=0x7f96500fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:40.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.253+0000 7f9656180700 1 -- 172.21.15.71:0/2422502111 shutdown_connections 2024-04-23T02:28:40.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.253+0000 7f9656180700 1 -- 172.21.15.71:0/2422502111 wait complete. 2024-04-23T02:28:40.260 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:28:40.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.357+0000 7fd7e75be700 1 -- 172.21.15.71:0/3538734310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e0101850 msgr2=0x7fd7e01058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:40.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.357+0000 7fd7e75be700 1 --2- 172.21.15.71:0/3538734310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e0101850 0x7fd7e01058a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009910 tx=0x7fd7dc009c20 comp rx=0 tx=0).stop 2024-04-23T02:28:40.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.357+0000 7fd7e75be700 1 -- 172.21.15.71:0/3538734310 shutdown_connections 2024-04-23T02:28:40.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.357+0000 7fd7e75be700 1 --2- 172.21.15.71:0/3538734310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e0101850 0x7fd7e01058a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.357+0000 7fd7e75be700 1 --2- 172.21.15.71:0/3538734310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7e0100ea0 0x7fd7e0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.357+0000 7fd7e75be700 1 -- 172.21.15.71:0/3538734310 >> 172.21.15.71:0/3538734310 conn(0x7fd7e00fc750 msgr2=0x7fd7e00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:40.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.357+0000 7fd7e75be700 1 -- 172.21.15.71:0/3538734310 shutdown_connections 2024-04-23T02:28:40.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.357+0000 7fd7e75be700 1 -- 172.21.15.71:0/3538734310 wait complete. 2024-04-23T02:28:40.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.357+0000 7fd7e75be700 1 Processor -- start 2024-04-23T02:28:40.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.357+0000 7fd7e75be700 1 -- start start 2024-04-23T02:28:40.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.358+0000 7fd7e75be700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7e0100ea0 0x7fd7e0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:40.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.358+0000 7fd7e75be700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e0101850 0x7fd7e00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:40.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.358+0000 7fd7e75be700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e0073b10 con 0x7fd7e0101850 2024-04-23T02:28:40.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.358+0000 7fd7e75be700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e0073c50 con 0x7fd7e0100ea0 2024-04-23T02:28:40.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.358+0000 7fd7e4b59700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e0101850 0x7fd7e00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:40.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.358+0000 7fd7e4b59700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e0101850 0x7fd7e00735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48182/0 (socket says 172.21.15.71:48182) 2024-04-23T02:28:40.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.358+0000 7fd7e4b59700 1 -- 172.21.15.71:0/436696441 learned_addr learned my addr 172.21.15.71:0/436696441 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:28:40.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.358+0000 7fd7e535a700 1 --2- 172.21.15.71:0/436696441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7e0100ea0 0x7fd7e0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:40.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.358+0000 7fd7e4b59700 1 -- 172.21.15.71:0/436696441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7e0100ea0 msgr2=0x7fd7e0073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:40.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.358+0000 7fd7e4b59700 1 --2- 172.21.15.71:0/436696441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7e0100ea0 0x7fd7e0073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.358+0000 7fd7e4b59700 1 -- 172.21.15.71:0/436696441 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7dc0095f0 con 0x7fd7e0101850 2024-04-23T02:28:40.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.358+0000 7fd7e535a700 1 --2- 172.21.15.71:0/436696441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7e0100ea0 0x7fd7e0073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:28:40.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.358+0000 7fd7e4b59700 1 --2- 172.21.15.71:0/436696441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e0101850 0x7fd7e00735d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc000c00 tx=0x7fd7dc00bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:40.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.358+0000 7fd7d67fc700 1 -- 172.21.15.71:0/436696441 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7dc01d070 con 0x7fd7e0101850 2024-04-23T02:28:40.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.358+0000 7fd7e75be700 1 -- 172.21.15.71:0/436696441 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e0073ed0 con 0x7fd7e0101850 2024-04-23T02:28:40.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.358+0000 7fd7d67fc700 1 -- 172.21.15.71:0/436696441 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd7dc003cf0 con 0x7fd7e0101850 2024-04-23T02:28:40.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.358+0000 7fd7d67fc700 1 -- 172.21.15.71:0/436696441 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7dc021410 con 0x7fd7e0101850 2024-04-23T02:28:40.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.358+0000 7fd7e75be700 1 -- 172.21.15.71:0/436696441 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e01a2cf0 con 0x7fd7e0101850 2024-04-23T02:28:40.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.359+0000 7fd7e75be700 1 -- 172.21.15.71:0/436696441 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e01092a0 con 0x7fd7e0101850 2024-04-23T02:28:40.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.360+0000 7fd7d67fc700 1 -- 172.21.15.71:0/436696441 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd7dc003850 con 0x7fd7e0101850 2024-04-23T02:28:40.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.360+0000 7fd7d67fc700 1 --2- 172.21.15.71:0/436696441 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd7cc06cce0 0x7fd7cc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:40.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.360+0000 7fd7d67fc700 1 -- 172.21.15.71:0/436696441 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd7dc08d7e0 con 0x7fd7e0101850 2024-04-23T02:28:40.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.360+0000 7fd7e535a700 1 --2- 172.21.15.71:0/436696441 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd7cc06cce0 0x7fd7cc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:40.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.361+0000 7fd7e535a700 1 --2- 172.21.15.71:0/436696441 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd7cc06cce0 0x7fd7cc06f1a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009cb0 tx=0x7fd7d0009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:40.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.365+0000 7fd7d67fc700 1 -- 172.21.15.71:0/436696441 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd7dc057850 con 0x7fd7e0101850 2024-04-23T02:28:40.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.574+0000 7fd7e75be700 1 -- 172.21.15.71:0/436696441 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7e00611d0 con 0x7fd7cc06cce0 2024-04-23T02:28:40.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.575+0000 7fd7d67fc700 1 -- 172.21.15.71:0/436696441 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd7e00611d0 con 0x7fd7cc06cce0 2024-04-23T02:28:40.579 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:28:40.579 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:28:40.579 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:28:40.579 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:28:40.579 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:28:40.579 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:28:40.580 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:28:40.580 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:28:40.580 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:28:40.580 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:28:40.580 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:28:40.580 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:28:40.580 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:28:40.580 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:28:40.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.577+0000 7fd7e75be700 1 -- 172.21.15.71:0/436696441 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd7cc06cce0 msgr2=0x7fd7cc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:40.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.577+0000 7fd7e75be700 1 --2- 172.21.15.71:0/436696441 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd7cc06cce0 0x7fd7cc06f1a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009cb0 tx=0x7fd7d0009260 comp rx=0 tx=0).stop 2024-04-23T02:28:40.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.577+0000 7fd7e75be700 1 -- 172.21.15.71:0/436696441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e0101850 msgr2=0x7fd7e00735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:40.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.577+0000 7fd7e75be700 1 --2- 172.21.15.71:0/436696441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e0101850 0x7fd7e00735d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc000c00 tx=0x7fd7dc00bb20 comp rx=0 tx=0).stop 2024-04-23T02:28:40.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.578+0000 7fd7e75be700 1 -- 172.21.15.71:0/436696441 shutdown_connections 2024-04-23T02:28:40.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.578+0000 7fd7e75be700 1 --2- 172.21.15.71:0/436696441 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd7cc06cce0 0x7fd7cc06f1a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.578+0000 7fd7e75be700 1 --2- 172.21.15.71:0/436696441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7e0100ea0 0x7fd7e0073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.578+0000 7fd7e75be700 1 --2- 172.21.15.71:0/436696441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e0101850 0x7fd7e00735d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.578+0000 7fd7e75be700 1 -- 172.21.15.71:0/436696441 >> 172.21.15.71:0/436696441 conn(0x7fd7e00fc750 msgr2=0x7fd7e00fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:40.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.578+0000 7fd7e75be700 1 -- 172.21.15.71:0/436696441 shutdown_connections 2024-04-23T02:28:40.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.578+0000 7fd7e75be700 1 -- 172.21.15.71:0/436696441 wait complete. 2024-04-23T02:28:40.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:40 smithi192 ceph-mon[108617]: pgmap v460: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:40.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:40 smithi192 ceph-mon[108617]: from='client.34678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:28:40.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:40 smithi192 ceph-mon[108617]: from='client.34682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:28:40.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:40 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3701023285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:28:40.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:40 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2422502111' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:28:40.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.680+0000 7f2f65fb4700 1 -- 172.21.15.71:0/2646692560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f600ffc20 msgr2=0x7f2f60100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:40.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.680+0000 7f2f65fb4700 1 --2- 172.21.15.71:0/2646692560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f600ffc20 0x7f2f60100080 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2f50009910 tx=0x7f2f50009c20 comp rx=0 tx=0).stop 2024-04-23T02:28:40.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.680+0000 7f2f65fb4700 1 -- 172.21.15.71:0/2646692560 shutdown_connections 2024-04-23T02:28:40.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.680+0000 7f2f65fb4700 1 --2- 172.21.15.71:0/2646692560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f600ffc20 0x7f2f60100080 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.680+0000 7f2f65fb4700 1 --2- 172.21.15.71:0/2646692560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f60105bd0 0x7f2f60105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.680+0000 7f2f65fb4700 1 -- 172.21.15.71:0/2646692560 >> 172.21.15.71:0/2646692560 conn(0x7f2f60075670 msgr2=0x7f2f60075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:40.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.680+0000 7f2f65fb4700 1 -- 172.21.15.71:0/2646692560 shutdown_connections 2024-04-23T02:28:40.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.680+0000 7f2f65fb4700 1 -- 172.21.15.71:0/2646692560 wait complete. 2024-04-23T02:28:40.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.680+0000 7f2f65fb4700 1 Processor -- start 2024-04-23T02:28:40.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.681+0000 7f2f65fb4700 1 -- start start 2024-04-23T02:28:40.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.681+0000 7f2f65fb4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f600ffc20 0x7f2f6019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:40.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.681+0000 7f2f65fb4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f60105bd0 0x7f2f6019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:40.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.681+0000 7f2f65fb4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f601987e0 con 0x7f2f600ffc20 2024-04-23T02:28:40.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.681+0000 7f2f65fb4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f60198950 con 0x7f2f60105bd0 2024-04-23T02:28:40.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.681+0000 7f2f5f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f600ffc20 0x7f2f6019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:40.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.681+0000 7f2f5f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f600ffc20 0x7f2f6019e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48194/0 (socket says 172.21.15.71:48194) 2024-04-23T02:28:40.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.681+0000 7f2f5f7fe700 1 -- 172.21.15.71:0/463679256 learned_addr learned my addr 172.21.15.71:0/463679256 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:28:40.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.681+0000 7f2f5effd700 1 --2- 172.21.15.71:0/463679256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f60105bd0 0x7f2f6019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:40.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.681+0000 7f2f5f7fe700 1 -- 172.21.15.71:0/463679256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f60105bd0 msgr2=0x7f2f6019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:40.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.681+0000 7f2f5f7fe700 1 --2- 172.21.15.71:0/463679256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f60105bd0 0x7f2f6019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.681+0000 7f2f5f7fe700 1 -- 172.21.15.71:0/463679256 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f500095f0 con 0x7f2f600ffc20 2024-04-23T02:28:40.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.681+0000 7f2f5effd700 1 --2- 172.21.15.71:0/463679256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f60105bd0 0x7f2f6019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:28:40.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.681+0000 7f2f5f7fe700 1 --2- 172.21.15.71:0/463679256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f600ffc20 0x7f2f6019e7f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800e920 tx=0x7f2f4800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:40.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.682+0000 7f2f5cff9700 1 -- 172.21.15.71:0/463679256 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f48009790 con 0x7f2f600ffc20 2024-04-23T02:28:40.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.682+0000 7f2f5cff9700 1 -- 172.21.15.71:0/463679256 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2f4800cba0 con 0x7f2f600ffc20 2024-04-23T02:28:40.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.682+0000 7f2f65fb4700 1 -- 172.21.15.71:0/463679256 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f60198c30 con 0x7f2f600ffc20 2024-04-23T02:28:40.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.682+0000 7f2f5cff9700 1 -- 172.21.15.71:0/463679256 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f48018650 con 0x7f2f600ffc20 2024-04-23T02:28:40.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.682+0000 7f2f65fb4700 1 -- 172.21.15.71:0/463679256 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f60199180 con 0x7f2f600ffc20 2024-04-23T02:28:40.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.682+0000 7f2f65fb4700 1 -- 172.21.15.71:0/463679256 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f6004f2e0 con 0x7f2f600ffc20 2024-04-23T02:28:40.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.683+0000 7f2f5cff9700 1 -- 172.21.15.71:0/463679256 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2f4800cd10 con 0x7f2f600ffc20 2024-04-23T02:28:40.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.683+0000 7f2f5cff9700 1 --2- 172.21.15.71:0/463679256 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2f4c06cce0 0x7f2f4c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:28:40.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.683+0000 7f2f5cff9700 1 -- 172.21.15.71:0/463679256 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2f48014070 con 0x7f2f600ffc20 2024-04-23T02:28:40.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.683+0000 7f2f5effd700 1 --2- 172.21.15.71:0/463679256 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2f4c06cce0 0x7f2f4c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:28:40.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.684+0000 7f2f5effd700 1 --2- 172.21.15.71:0/463679256 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2f4c06cce0 0x7f2f4c06f1a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2f50009fd0 tx=0x7f2f50005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:28:40.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.688+0000 7f2f5cff9700 1 -- 172.21.15.71:0/463679256 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2f48052c50 con 0x7f2f600ffc20 2024-04-23T02:28:40.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:40 smithi071 ceph-mon[144544]: pgmap v460: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:40.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:40 smithi071 ceph-mon[144544]: from='client.34678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:28:40.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:40 smithi071 ceph-mon[144544]: from='client.34682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:28:40.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:40 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3701023285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:28:40.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:40 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2422502111' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:28:40.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.959+0000 7f2f65fb4700 1 -- 172.21.15.71:0/463679256 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2f601993a0 con 0x7f2f600ffc20 2024-04-23T02:28:40.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.960+0000 7f2f5cff9700 1 -- 172.21.15.71:0/463679256 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f2f4805a450 con 0x7f2f600ffc20 2024-04-23T02:28:40.963 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:28:40.963 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:28:40.963 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:28:40.963 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:28:40.964 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:28:40.964 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:28:40.964 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:28:40.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.962+0000 7f2f65fb4700 1 -- 172.21.15.71:0/463679256 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2f4c06cce0 msgr2=0x7f2f4c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:40.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.962+0000 7f2f65fb4700 1 --2- 172.21.15.71:0/463679256 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2f4c06cce0 0x7f2f4c06f1a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2f50009fd0 tx=0x7f2f50005940 comp rx=0 tx=0).stop 2024-04-23T02:28:40.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.962+0000 7f2f65fb4700 1 -- 172.21.15.71:0/463679256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f600ffc20 msgr2=0x7f2f6019e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:28:40.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.962+0000 7f2f65fb4700 1 --2- 172.21.15.71:0/463679256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f600ffc20 0x7f2f6019e7f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800e920 tx=0x7f2f4800ece0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.962+0000 7f2f65fb4700 1 -- 172.21.15.71:0/463679256 shutdown_connections 2024-04-23T02:28:40.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.962+0000 7f2f65fb4700 1 --2- 172.21.15.71:0/463679256 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2f4c06cce0 0x7f2f4c06f1a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.962+0000 7f2f65fb4700 1 --2- 172.21.15.71:0/463679256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f600ffc20 0x7f2f6019e7f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.962+0000 7f2f65fb4700 1 --2- 172.21.15.71:0/463679256 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f60105bd0 0x7f2f6019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:28:40.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.962+0000 7f2f65fb4700 1 -- 172.21.15.71:0/463679256 >> 172.21.15.71:0/463679256 conn(0x7f2f60075670 msgr2=0x7f2f600687a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:28:40.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.962+0000 7f2f65fb4700 1 -- 172.21.15.71:0/463679256 shutdown_connections 2024-04-23T02:28:40.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:28:40.962+0000 7f2f65fb4700 1 -- 172.21.15.71:0/463679256 wait complete. 2024-04-23T02:28:41.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:41 smithi071 ceph-mon[144544]: from='client.34686 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:28:41.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:41 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/463679256' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:28:41.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:41 smithi192 ceph-mon[108617]: from='client.34686 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:28:41.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:41 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/463679256' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:28:42.632 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:42 smithi071 ceph-mon[144544]: from='client.34698 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:28:42.632 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:42 smithi071 ceph-mon[144544]: pgmap v461: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:42.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:42 smithi192 ceph-mon[108617]: from='client.34698 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:28:42.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:42 smithi192 ceph-mon[108617]: pgmap v461: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:44.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:44 smithi071 ceph-mon[144544]: pgmap v462: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:44.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:44 smithi192 ceph-mon[108617]: pgmap v462: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:46.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:46 smithi071 ceph-mon[144544]: pgmap v463: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:46.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:46 smithi192 ceph-mon[108617]: pgmap v463: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:48.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:48 smithi071 ceph-mon[144544]: pgmap v464: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:48.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:48 smithi192 ceph-mon[108617]: pgmap v464: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:50.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:50 smithi071 ceph-mon[144544]: pgmap v465: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:50.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:28:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:50 smithi192 ceph-mon[108617]: pgmap v465: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:50.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:28:52.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:52 smithi071 ceph-mon[144544]: pgmap v466: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:52.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:52 smithi192 ceph-mon[108617]: pgmap v466: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:54.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:54 smithi071 ceph-mon[144544]: pgmap v467: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:54.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:54 smithi192 ceph-mon[108617]: pgmap v467: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:56.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:56 smithi071 ceph-mon[144544]: pgmap v468: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:56.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:56 smithi192 ceph-mon[108617]: pgmap v468: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:58.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:28:58 smithi071 ceph-mon[144544]: pgmap v469: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:28:58.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:28:58 smithi192 ceph-mon[108617]: pgmap v469: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:00.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:00 smithi071 ceph-mon[144544]: pgmap v470: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:00.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:00 smithi192 ceph-mon[108617]: pgmap v470: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:02.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:02 smithi071 ceph-mon[144544]: pgmap v471: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:02.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:02 smithi192 ceph-mon[108617]: pgmap v471: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:04.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:04 smithi071 ceph-mon[144544]: pgmap v472: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:04.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:04 smithi192 ceph-mon[108617]: pgmap v472: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:05.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:29:05.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:29:06.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:06 smithi071 ceph-mon[144544]: pgmap v473: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:06.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:06 smithi192 ceph-mon[108617]: pgmap v473: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:08.745 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:08 smithi071 ceph-mon[144544]: pgmap v474: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:08.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:08 smithi192 ceph-mon[108617]: pgmap v474: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:10.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:10 smithi071 ceph-mon[144544]: pgmap v475: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:10.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:10 smithi192 ceph-mon[108617]: pgmap v475: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:11.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.079+0000 7f13d4a19700 1 -- 172.21.15.71:0/301220122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13d0100750 msgr2=0x7f13d01047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:11.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.079+0000 7f13d4a19700 1 --2- 172.21.15.71:0/301220122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13d0100750 0x7f13d01047a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f13b8009910 tx=0x7f13b8009c20 comp rx=0 tx=0).stop 2024-04-23T02:29:11.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.079+0000 7f13d4a19700 1 -- 172.21.15.71:0/301220122 shutdown_connections 2024-04-23T02:29:11.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.079+0000 7f13d4a19700 1 --2- 172.21.15.71:0/301220122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13d0100750 0x7f13d01047a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.079+0000 7f13d4a19700 1 --2- 172.21.15.71:0/301220122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d00ffda0 0x7f13d0100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.079+0000 7f13d4a19700 1 -- 172.21.15.71:0/301220122 >> 172.21.15.71:0/301220122 conn(0x7f13d00fb650 msgr2=0x7f13d00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:11.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.079+0000 7f13d4a19700 1 -- 172.21.15.71:0/301220122 shutdown_connections 2024-04-23T02:29:11.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.079+0000 7f13d4a19700 1 -- 172.21.15.71:0/301220122 wait complete. 2024-04-23T02:29:11.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.080+0000 7f13d4a19700 1 Processor -- start 2024-04-23T02:29:11.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.080+0000 7f13d4a19700 1 -- start start 2024-04-23T02:29:11.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.080+0000 7f13d4a19700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13d00ffda0 0x7f13d01955f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:11.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.080+0000 7f13d4a19700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d0100750 0x7f13d0195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:11.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.080+0000 7f13d4a19700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d0196070 con 0x7f13d00ffda0 2024-04-23T02:29:11.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.080+0000 7f13d4a19700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d01961e0 con 0x7f13d0100750 2024-04-23T02:29:11.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.080+0000 7f13ce59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13d00ffda0 0x7f13d01955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:11.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.080+0000 7f13ce59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13d00ffda0 0x7f13d01955f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52278/0 (socket says 172.21.15.71:52278) 2024-04-23T02:29:11.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.080+0000 7f13ce59c700 1 -- 172.21.15.71:0/2264077306 learned_addr learned my addr 172.21.15.71:0/2264077306 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:29:11.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.081+0000 7f13ce59c700 1 -- 172.21.15.71:0/2264077306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d0100750 msgr2=0x7f13d0195b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:11.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.081+0000 7f13ce59c700 1 --2- 172.21.15.71:0/2264077306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d0100750 0x7f13d0195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.081+0000 7f13ce59c700 1 -- 172.21.15.71:0/2264077306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b80095f0 con 0x7f13d00ffda0 2024-04-23T02:29:11.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.081+0000 7f13ce59c700 1 --2- 172.21.15.71:0/2264077306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13d00ffda0 0x7f13d01955f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f13c000b4b0 tx=0x7f13c000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:11.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.081+0000 7f13c77fe700 1 -- 172.21.15.71:0/2264077306 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13c0011800 con 0x7f13d00ffda0 2024-04-23T02:29:11.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.081+0000 7f13c77fe700 1 -- 172.21.15.71:0/2264077306 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f13c0011e00 con 0x7f13d00ffda0 2024-04-23T02:29:11.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.081+0000 7f13d4a19700 1 -- 172.21.15.71:0/2264077306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d01a2c60 con 0x7f13d00ffda0 2024-04-23T02:29:11.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.081+0000 7f13c77fe700 1 -- 172.21.15.71:0/2264077306 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13c0010e50 con 0x7f13d00ffda0 2024-04-23T02:29:11.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.081+0000 7f13d4a19700 1 -- 172.21.15.71:0/2264077306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d01a2fc0 con 0x7f13d00ffda0 2024-04-23T02:29:11.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.082+0000 7f13d4a19700 1 -- 172.21.15.71:0/2264077306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d01080d0 con 0x7f13d00ffda0 2024-04-23T02:29:11.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.083+0000 7f13c77fe700 1 -- 172.21.15.71:0/2264077306 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f13c0010380 con 0x7f13d00ffda0 2024-04-23T02:29:11.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.083+0000 7f13c77fe700 1 --2- 172.21.15.71:0/2264077306 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f13bc06cce0 0x7f13bc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:11.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.083+0000 7f13cdd9b700 1 --2- 172.21.15.71:0/2264077306 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f13bc06cce0 0x7f13bc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:11.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.083+0000 7f13c77fe700 1 -- 172.21.15.71:0/2264077306 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f13c008c0b0 con 0x7f13d00ffda0 2024-04-23T02:29:11.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.084+0000 7f13cdd9b700 1 --2- 172.21.15.71:0/2264077306 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f13bc06cce0 0x7f13bc06f1a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f13b8006010 tx=0x7f13b8005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:11.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.088+0000 7f13c77fe700 1 -- 172.21.15.71:0/2264077306 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f13c0059c30 con 0x7f13d00ffda0 2024-04-23T02:29:11.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.308+0000 7f13d4a19700 1 -- 172.21.15.71:0/2264077306 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f13d00611d0 con 0x7f13bc06cce0 2024-04-23T02:29:11.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.310+0000 7f13c77fe700 1 -- 172.21.15.71:0/2264077306 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f13d00611d0 con 0x7f13bc06cce0 2024-04-23T02:29:11.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.312+0000 7f13d4a19700 1 -- 172.21.15.71:0/2264077306 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f13bc06cce0 msgr2=0x7f13bc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:11.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.312+0000 7f13d4a19700 1 --2- 172.21.15.71:0/2264077306 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f13bc06cce0 0x7f13bc06f1a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f13b8006010 tx=0x7f13b8005960 comp rx=0 tx=0).stop 2024-04-23T02:29:11.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.312+0000 7f13d4a19700 1 -- 172.21.15.71:0/2264077306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13d00ffda0 msgr2=0x7f13d01955f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:11.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.312+0000 7f13d4a19700 1 --2- 172.21.15.71:0/2264077306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13d00ffda0 0x7f13d01955f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f13c000b4b0 tx=0x7f13c000b870 comp rx=0 tx=0).stop 2024-04-23T02:29:11.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.312+0000 7f13d4a19700 1 -- 172.21.15.71:0/2264077306 shutdown_connections 2024-04-23T02:29:11.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.312+0000 7f13d4a19700 1 --2- 172.21.15.71:0/2264077306 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f13bc06cce0 0x7f13bc06f1a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.312+0000 7f13d4a19700 1 --2- 172.21.15.71:0/2264077306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13d00ffda0 0x7f13d01955f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.312+0000 7f13d4a19700 1 --2- 172.21.15.71:0/2264077306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d0100750 0x7f13d0195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.312+0000 7f13d4a19700 1 -- 172.21.15.71:0/2264077306 >> 172.21.15.71:0/2264077306 conn(0x7f13d00fb650 msgr2=0x7f13d00fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:11.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.312+0000 7f13d4a19700 1 -- 172.21.15.71:0/2264077306 shutdown_connections 2024-04-23T02:29:11.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.312+0000 7f13d4a19700 1 -- 172.21.15.71:0/2264077306 wait complete. 2024-04-23T02:29:11.325 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:29:11.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.416+0000 7f62a25b9700 1 -- 172.21.15.71:0/374341367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f629c105bf0 msgr2=0x7f629c105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:11.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.416+0000 7f62a25b9700 1 --2- 172.21.15.71:0/374341367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f629c105bf0 0x7f629c105fd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f628c009910 tx=0x7f628c009c20 comp rx=0 tx=0).stop 2024-04-23T02:29:11.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.416+0000 7f62a25b9700 1 -- 172.21.15.71:0/374341367 shutdown_connections 2024-04-23T02:29:11.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.416+0000 7f62a25b9700 1 --2- 172.21.15.71:0/374341367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f629c0ffb40 0x7f629c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.416+0000 7f62a25b9700 1 --2- 172.21.15.71:0/374341367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f629c105bf0 0x7f629c105fd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.416+0000 7f62a25b9700 1 -- 172.21.15.71:0/374341367 >> 172.21.15.71:0/374341367 conn(0x7f629c0fb610 msgr2=0x7f629c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:11.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.416+0000 7f62a25b9700 1 -- 172.21.15.71:0/374341367 shutdown_connections 2024-04-23T02:29:11.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.416+0000 7f62a25b9700 1 -- 172.21.15.71:0/374341367 wait complete. 2024-04-23T02:29:11.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.416+0000 7f62a25b9700 1 Processor -- start 2024-04-23T02:29:11.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.416+0000 7f62a25b9700 1 -- start start 2024-04-23T02:29:11.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.417+0000 7f62a25b9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f629c0ffb40 0x7f629c196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:11.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.417+0000 7f62a25b9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f629c105bf0 0x7f629c196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:11.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.417+0000 7f62a25b9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c1972e0 con 0x7f629c105bf0 2024-04-23T02:29:11.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.417+0000 7f62a25b9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c19a010 con 0x7f629c0ffb40 2024-04-23T02:29:11.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.417+0000 7f629b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f629c105bf0 0x7f629c196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:11.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.417+0000 7f629b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f629c105bf0 0x7f629c196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52292/0 (socket says 172.21.15.71:52292) 2024-04-23T02:29:11.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.417+0000 7f629b7fe700 1 -- 172.21.15.71:0/3627813378 learned_addr learned my addr 172.21.15.71:0/3627813378 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:29:11.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.417+0000 7f629bfff700 1 --2- 172.21.15.71:0/3627813378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f629c0ffb40 0x7f629c196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:11.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.417+0000 7f629b7fe700 1 -- 172.21.15.71:0/3627813378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f629c0ffb40 msgr2=0x7f629c196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:11.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.417+0000 7f629b7fe700 1 --2- 172.21.15.71:0/3627813378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f629c0ffb40 0x7f629c196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.417+0000 7f629b7fe700 1 -- 172.21.15.71:0/3627813378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0095f0 con 0x7f629c105bf0 2024-04-23T02:29:11.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.417+0000 7f629bfff700 1 --2- 172.21.15.71:0/3627813378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f629c0ffb40 0x7f629c196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:29:11.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.417+0000 7f629b7fe700 1 --2- 172.21.15.71:0/3627813378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f629c105bf0 0x7f629c196c50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f629000d900 tx=0x7f629000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:11.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.417+0000 7f62997fa700 1 -- 172.21.15.71:0/3627813378 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62900041f0 con 0x7f629c105bf0 2024-04-23T02:29:11.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.417+0000 7f62997fa700 1 -- 172.21.15.71:0/3627813378 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6290004cd0 con 0x7f629c105bf0 2024-04-23T02:29:11.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.417+0000 7f62a25b9700 1 -- 172.21.15.71:0/3627813378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629c19a2f0 con 0x7f629c105bf0 2024-04-23T02:29:11.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.417+0000 7f62997fa700 1 -- 172.21.15.71:0/3627813378 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62900056d0 con 0x7f629c105bf0 2024-04-23T02:29:11.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.417+0000 7f62a25b9700 1 -- 172.21.15.71:0/3627813378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629c19a840 con 0x7f629c105bf0 2024-04-23T02:29:11.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.418+0000 7f62a25b9700 1 -- 172.21.15.71:0/3627813378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629c0679c0 con 0x7f629c105bf0 2024-04-23T02:29:11.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.419+0000 7f62997fa700 1 -- 172.21.15.71:0/3627813378 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6290004e40 con 0x7f629c105bf0 2024-04-23T02:29:11.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.419+0000 7f62997fa700 1 --2- 172.21.15.71:0/3627813378 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f628406cc90 0x7f628406f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:11.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.419+0000 7f62997fa700 1 -- 172.21.15.71:0/3627813378 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f629008c710 con 0x7f629c105bf0 2024-04-23T02:29:11.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.419+0000 7f629bfff700 1 --2- 172.21.15.71:0/3627813378 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f628406cc90 0x7f628406f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:11.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.420+0000 7f629bfff700 1 --2- 172.21.15.71:0/3627813378 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f628406cc90 0x7f628406f150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f628c00b5c0 tx=0x7f628c005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:11.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.424+0000 7f62997fa700 1 -- 172.21.15.71:0/3627813378 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6290056780 con 0x7f629c105bf0 2024-04-23T02:29:11.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.636+0000 7f62a25b9700 1 -- 172.21.15.71:0/3627813378 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f629c19aba0 con 0x7f628406cc90 2024-04-23T02:29:11.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.638+0000 7f62997fa700 1 -- 172.21.15.71:0/3627813378 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f629c19aba0 con 0x7f628406cc90 2024-04-23T02:29:11.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.640+0000 7f62a25b9700 1 -- 172.21.15.71:0/3627813378 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f628406cc90 msgr2=0x7f628406f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:11.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.640+0000 7f62a25b9700 1 --2- 172.21.15.71:0/3627813378 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f628406cc90 0x7f628406f150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f628c00b5c0 tx=0x7f628c005960 comp rx=0 tx=0).stop 2024-04-23T02:29:11.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.640+0000 7f62a25b9700 1 -- 172.21.15.71:0/3627813378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f629c105bf0 msgr2=0x7f629c196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:11.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.640+0000 7f62a25b9700 1 --2- 172.21.15.71:0/3627813378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f629c105bf0 0x7f629c196c50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f629000d900 tx=0x7f629000dcc0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.640+0000 7f62a25b9700 1 -- 172.21.15.71:0/3627813378 shutdown_connections 2024-04-23T02:29:11.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.640+0000 7f62a25b9700 1 --2- 172.21.15.71:0/3627813378 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f628406cc90 0x7f628406f150 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.640+0000 7f62a25b9700 1 --2- 172.21.15.71:0/3627813378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f629c0ffb40 0x7f629c196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.640+0000 7f62a25b9700 1 --2- 172.21.15.71:0/3627813378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f629c105bf0 0x7f629c196c50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.640+0000 7f62a25b9700 1 -- 172.21.15.71:0/3627813378 >> 172.21.15.71:0/3627813378 conn(0x7f629c0fb610 msgr2=0x7f629c068ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:11.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.640+0000 7f62a25b9700 1 -- 172.21.15.71:0/3627813378 shutdown_connections 2024-04-23T02:29:11.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.640+0000 7f62a25b9700 1 -- 172.21.15.71:0/3627813378 wait complete. 2024-04-23T02:29:11.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.743+0000 7f27bd0ed700 1 -- 172.21.15.71:0/3008030974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27b8105bd0 msgr2=0x7f27b8105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:11.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.743+0000 7f27bd0ed700 1 --2- 172.21.15.71:0/3008030974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27b8105bd0 0x7f27b8105fb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009910 tx=0x7f27ac009c20 comp rx=0 tx=0).stop 2024-04-23T02:29:11.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.743+0000 7f27bd0ed700 1 -- 172.21.15.71:0/3008030974 shutdown_connections 2024-04-23T02:29:11.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.743+0000 7f27bd0ed700 1 --2- 172.21.15.71:0/3008030974 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b80ffc20 0x7f27b8100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.743+0000 7f27bd0ed700 1 --2- 172.21.15.71:0/3008030974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27b8105bd0 0x7f27b8105fb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.743+0000 7f27bd0ed700 1 -- 172.21.15.71:0/3008030974 >> 172.21.15.71:0/3008030974 conn(0x7f27b8075670 msgr2=0x7f27b8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:11.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.743+0000 7f27bd0ed700 1 -- 172.21.15.71:0/3008030974 shutdown_connections 2024-04-23T02:29:11.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.743+0000 7f27bd0ed700 1 -- 172.21.15.71:0/3008030974 wait complete. 2024-04-23T02:29:11.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.743+0000 7f27bd0ed700 1 Processor -- start 2024-04-23T02:29:11.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.744+0000 7f27bd0ed700 1 -- start start 2024-04-23T02:29:11.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.744+0000 7f27bd0ed700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27b80ffc20 0x7f27b8197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:11.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.744+0000 7f27bd0ed700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b8105bd0 0x7f27b8197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:11.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.744+0000 7f27bd0ed700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b8198160 con 0x7f27b80ffc20 2024-04-23T02:29:11.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.744+0000 7f27bd0ed700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b81982a0 con 0x7f27b8105bd0 2024-04-23T02:29:11.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.744+0000 7f27b6d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27b80ffc20 0x7f27b8197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:11.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.744+0000 7f27b6d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27b80ffc20 0x7f27b8197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52308/0 (socket says 172.21.15.71:52308) 2024-04-23T02:29:11.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.744+0000 7f27b6d9d700 1 -- 172.21.15.71:0/460948252 learned_addr learned my addr 172.21.15.71:0/460948252 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:29:11.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.744+0000 7f27b6d9d700 1 -- 172.21.15.71:0/460948252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b8105bd0 msgr2=0x7f27b8197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:11.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.744+0000 7f27b6d9d700 1 --2- 172.21.15.71:0/460948252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b8105bd0 0x7f27b8197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.744+0000 7f27b6d9d700 1 -- 172.21.15.71:0/460948252 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac0095f0 con 0x7f27b80ffc20 2024-04-23T02:29:11.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.744+0000 7f27b6d9d700 1 --2- 172.21.15.71:0/460948252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27b80ffc20 0x7f27b8197600 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f27ac006010 tx=0x7f27ac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:11.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.744+0000 7f279ffff700 1 -- 172.21.15.71:0/460948252 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27ac01d070 con 0x7f27b80ffc20 2024-04-23T02:29:11.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.745+0000 7f27bd0ed700 1 -- 172.21.15.71:0/460948252 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b81a0670 con 0x7f27b80ffc20 2024-04-23T02:29:11.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.745+0000 7f279ffff700 1 -- 172.21.15.71:0/460948252 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f27ac00f460 con 0x7f27b80ffc20 2024-04-23T02:29:11.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.745+0000 7f279ffff700 1 -- 172.21.15.71:0/460948252 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27ac021640 con 0x7f27b80ffc20 2024-04-23T02:29:11.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.745+0000 7f27bd0ed700 1 -- 172.21.15.71:0/460948252 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b81a0a80 con 0x7f27b80ffc20 2024-04-23T02:29:11.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.746+0000 7f27bd0ed700 1 -- 172.21.15.71:0/460948252 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b81080f0 con 0x7f27b80ffc20 2024-04-23T02:29:11.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.746+0000 7f279ffff700 1 -- 172.21.15.71:0/460948252 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f27ac00fa60 con 0x7f27b80ffc20 2024-04-23T02:29:11.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.746+0000 7f279ffff700 1 --2- 172.21.15.71:0/460948252 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f27a006cce0 0x7f27a006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:11.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.746+0000 7f279ffff700 1 -- 172.21.15.71:0/460948252 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f27ac08d8c0 con 0x7f27b80ffc20 2024-04-23T02:29:11.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.746+0000 7f27b659c700 1 --2- 172.21.15.71:0/460948252 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f27a006cce0 0x7f27a006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:11.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.747+0000 7f27b659c700 1 --2- 172.21.15.71:0/460948252 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f27a006cce0 0x7f27a006f1a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f27b8068e30 tx=0x7f27a8005c80 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:11.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.751+0000 7f279ffff700 1 -- 172.21.15.71:0/460948252 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f27ac057930 con 0x7f27b80ffc20 2024-04-23T02:29:11.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.955+0000 7f27bd0ed700 1 -- 172.21.15.71:0/460948252 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f27b80611d0 con 0x7f27a006cce0 2024-04-23T02:29:11.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.965+0000 7f279ffff700 1 -- 172.21.15.71:0/460948252 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f27b80611d0 con 0x7f27a006cce0 2024-04-23T02:29:11.968 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:29:11.968 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (12m) 6m ago 24m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:29:11.968 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (24m) 6m ago 24m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:29:11.968 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (23m) 6m ago 23m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:29:11.968 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (10m) 6m ago 24m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:29:11.968 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (10m) 6m ago 23m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:29:11.968 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (12m) 6m ago 23m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:29:11.968 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (7m) 6m ago 18m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:29:11.968 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (6m) 6m ago 18m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:29:11.968 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (18m) 6m ago 18m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:29:11.968 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (6m) 6m ago 18m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:29:11.968 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (14m) 6m ago 25m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:29:11.968 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (14m) 6m ago 23m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:29:11.969 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (11m) 6m ago 25m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:29:11.969 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (11m) 6m ago 23m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:29:11.969 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (13m) 6m ago 24m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:29:11.969 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (13m) 6m ago 23m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:29:11.969 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (10m) 6m ago 22m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:29:11.969 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (9m) 6m ago 21m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:29:11.969 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (9m) 6m ago 21m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:29:11.969 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (8m) 6m ago 20m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:29:11.969 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (7m) 6m ago 20m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:29:11.969 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (7m) 6m ago 19m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:29:11.969 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (13m) 6m ago 23m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:29:11.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.967+0000 7f27bd0ed700 1 -- 172.21.15.71:0/460948252 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f27a006cce0 msgr2=0x7f27a006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:11.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.967+0000 7f27bd0ed700 1 --2- 172.21.15.71:0/460948252 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f27a006cce0 0x7f27a006f1a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f27b8068e30 tx=0x7f27a8005c80 comp rx=0 tx=0).stop 2024-04-23T02:29:11.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.967+0000 7f27bd0ed700 1 -- 172.21.15.71:0/460948252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27b80ffc20 msgr2=0x7f27b8197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:11.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.967+0000 7f27bd0ed700 1 --2- 172.21.15.71:0/460948252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27b80ffc20 0x7f27b8197600 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f27ac006010 tx=0x7f27ac00be20 comp rx=0 tx=0).stop 2024-04-23T02:29:11.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.967+0000 7f27bd0ed700 1 -- 172.21.15.71:0/460948252 shutdown_connections 2024-04-23T02:29:11.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.967+0000 7f27bd0ed700 1 --2- 172.21.15.71:0/460948252 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f27a006cce0 0x7f27a006f1a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.967+0000 7f27bd0ed700 1 --2- 172.21.15.71:0/460948252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27b80ffc20 0x7f27b8197600 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.967+0000 7f27bd0ed700 1 --2- 172.21.15.71:0/460948252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b8105bd0 0x7f27b8197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:11.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.967+0000 7f27bd0ed700 1 -- 172.21.15.71:0/460948252 >> 172.21.15.71:0/460948252 conn(0x7f27b8075670 msgr2=0x7f27b80fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:11.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.967+0000 7f27bd0ed700 1 -- 172.21.15.71:0/460948252 shutdown_connections 2024-04-23T02:29:11.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:11.967+0000 7f27bd0ed700 1 -- 172.21.15.71:0/460948252 wait complete. 2024-04-23T02:29:12.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.069+0000 7f9433de8700 1 -- 172.21.15.71:0/540113670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f942c0ffb40 msgr2=0x7f942c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:12.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.069+0000 7f9433de8700 1 --2- 172.21.15.71:0/540113670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f942c0ffb40 0x7f942c0fffc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f941c009910 tx=0x7f941c009c20 comp rx=0 tx=0).stop 2024-04-23T02:29:12.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.069+0000 7f9433de8700 1 -- 172.21.15.71:0/540113670 shutdown_connections 2024-04-23T02:29:12.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.070+0000 7f9433de8700 1 --2- 172.21.15.71:0/540113670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f942c0ffb40 0x7f942c0fffc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:12.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.070+0000 7f9433de8700 1 --2- 172.21.15.71:0/540113670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f942c105bf0 0x7f942c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:12.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.070+0000 7f9433de8700 1 -- 172.21.15.71:0/540113670 >> 172.21.15.71:0/540113670 conn(0x7f942c0fb610 msgr2=0x7f942c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:12.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.070+0000 7f9433de8700 1 -- 172.21.15.71:0/540113670 shutdown_connections 2024-04-23T02:29:12.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.070+0000 7f9433de8700 1 -- 172.21.15.71:0/540113670 wait complete. 2024-04-23T02:29:12.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.070+0000 7f9433de8700 1 Processor -- start 2024-04-23T02:29:12.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.070+0000 7f9433de8700 1 -- start start 2024-04-23T02:29:12.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.070+0000 7f9433de8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f942c0ffb40 0x7f942c19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:12.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.070+0000 7f9433de8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f942c105bf0 0x7f942c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:12.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.070+0000 7f9433de8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c19d0e0 con 0x7f942c0ffb40 2024-04-23T02:29:12.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.070+0000 7f9433de8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c196590 con 0x7f942c105bf0 2024-04-23T02:29:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.071+0000 7f9431b84700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f942c0ffb40 0x7f942c19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.071+0000 7f9431b84700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f942c0ffb40 0x7f942c19c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52328/0 (socket says 172.21.15.71:52328) 2024-04-23T02:29:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.071+0000 7f9431b84700 1 -- 172.21.15.71:0/2332679710 learned_addr learned my addr 172.21.15.71:0/2332679710 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:29:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.071+0000 7f9431b84700 1 -- 172.21.15.71:0/2332679710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f942c105bf0 msgr2=0x7f942c19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.071+0000 7f9431b84700 1 --2- 172.21.15.71:0/2332679710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f942c105bf0 0x7f942c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.071+0000 7f9431b84700 1 -- 172.21.15.71:0/2332679710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c0095f0 con 0x7f942c0ffb40 2024-04-23T02:29:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.071+0000 7f9431b84700 1 --2- 172.21.15.71:0/2332679710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f942c0ffb40 0x7f942c19c510 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f942800d900 tx=0x7f942800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.071+0000 7f9422ffd700 1 -- 172.21.15.71:0/2332679710 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9428004a00 con 0x7f942c0ffb40 2024-04-23T02:29:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.071+0000 7f9422ffd700 1 -- 172.21.15.71:0/2332679710 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9428005020 con 0x7f942c0ffb40 2024-04-23T02:29:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.071+0000 7f9433de8700 1 -- 172.21.15.71:0/2332679710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c196870 con 0x7f942c0ffb40 2024-04-23T02:29:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.071+0000 7f9422ffd700 1 -- 172.21.15.71:0/2332679710 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9428009c00 con 0x7f942c0ffb40 2024-04-23T02:29:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.071+0000 7f9433de8700 1 -- 172.21.15.71:0/2332679710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c196dc0 con 0x7f942c0ffb40 2024-04-23T02:29:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.072+0000 7f9433de8700 1 -- 172.21.15.71:0/2332679710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942c04f2e0 con 0x7f942c0ffb40 2024-04-23T02:29:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.072+0000 7f9422ffd700 1 -- 172.21.15.71:0/2332679710 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9428005190 con 0x7f942c0ffb40 2024-04-23T02:29:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.073+0000 7f9422ffd700 1 --2- 172.21.15.71:0/2332679710 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f941806cce0 0x7f941806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.073+0000 7f9422ffd700 1 -- 172.21.15.71:0/2332679710 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f942808c770 con 0x7f942c0ffb40 2024-04-23T02:29:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.073+0000 7f9431383700 1 --2- 172.21.15.71:0/2332679710 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f941806cce0 0x7f941806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:12.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.073+0000 7f9431383700 1 --2- 172.21.15.71:0/2332679710 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f941806cce0 0x7f941806f1a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f941c00b5c0 tx=0x7f941c005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:12.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.077+0000 7f9422ffd700 1 -- 172.21.15.71:0/2332679710 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f942805a270 con 0x7f942c0ffb40 2024-04-23T02:29:12.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.352+0000 7f9433de8700 1 -- 172.21.15.71:0/2332679710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f942c197aa0 con 0x7f942c0ffb40 2024-04-23T02:29:12.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.353+0000 7f9422ffd700 1 -- 172.21.15.71:0/2332679710 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f9428005440 con 0x7f942c0ffb40 2024-04-23T02:29:12.356 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:29:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:29:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:29:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:29:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:29:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:29:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:29:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:29:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:29:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:29:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:29:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:29:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:29:12.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:29:12.357 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:29:12.357 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:29:12.357 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:29:12.357 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:29:12.357 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:29:12.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.355+0000 7f9433de8700 1 -- 172.21.15.71:0/2332679710 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f941806cce0 msgr2=0x7f941806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:12.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.355+0000 7f9433de8700 1 --2- 172.21.15.71:0/2332679710 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f941806cce0 0x7f941806f1a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f941c00b5c0 tx=0x7f941c005960 comp rx=0 tx=0).stop 2024-04-23T02:29:12.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.355+0000 7f9433de8700 1 -- 172.21.15.71:0/2332679710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f942c0ffb40 msgr2=0x7f942c19c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:12.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.355+0000 7f9433de8700 1 --2- 172.21.15.71:0/2332679710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f942c0ffb40 0x7f942c19c510 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f942800d900 tx=0x7f942800dcc0 comp rx=0 tx=0).stop 2024-04-23T02:29:12.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.355+0000 7f9433de8700 1 -- 172.21.15.71:0/2332679710 shutdown_connections 2024-04-23T02:29:12.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.355+0000 7f9433de8700 1 --2- 172.21.15.71:0/2332679710 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f941806cce0 0x7f941806f1a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:12.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.355+0000 7f9433de8700 1 --2- 172.21.15.71:0/2332679710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f942c0ffb40 0x7f942c19c510 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:12.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.355+0000 7f9433de8700 1 --2- 172.21.15.71:0/2332679710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f942c105bf0 0x7f942c19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:12.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.355+0000 7f9433de8700 1 -- 172.21.15.71:0/2332679710 >> 172.21.15.71:0/2332679710 conn(0x7f942c0fb610 msgr2=0x7f942c068eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:12.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.355+0000 7f9433de8700 1 -- 172.21.15.71:0/2332679710 shutdown_connections 2024-04-23T02:29:12.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.355+0000 7f9433de8700 1 -- 172.21.15.71:0/2332679710 wait complete. 2024-04-23T02:29:12.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.459+0000 7f1aeca18700 1 -- 172.21.15.71:0/4070374189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae80fff50 msgr2=0x7f1ae8105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:12.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.459+0000 7f1aeca18700 1 --2- 172.21.15.71:0/4070374189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae80fff50 0x7f1ae8105840 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009910 tx=0x7f1ad8009c20 comp rx=0 tx=0).stop 2024-04-23T02:29:12.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.459+0000 7f1aeca18700 1 -- 172.21.15.71:0/4070374189 shutdown_connections 2024-04-23T02:29:12.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.459+0000 7f1aeca18700 1 --2- 172.21.15.71:0/4070374189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae80fff50 0x7f1ae8105840 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:12.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.459+0000 7f1aeca18700 1 --2- 172.21.15.71:0/4070374189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ae80ff5a0 0x7f1ae80ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:12.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.459+0000 7f1aeca18700 1 -- 172.21.15.71:0/4070374189 >> 172.21.15.71:0/4070374189 conn(0x7f1ae8074e30 msgr2=0x7f1ae8075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:12.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.459+0000 7f1aeca18700 1 -- 172.21.15.71:0/4070374189 shutdown_connections 2024-04-23T02:29:12.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.459+0000 7f1aeca18700 1 -- 172.21.15.71:0/4070374189 wait complete. 2024-04-23T02:29:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.460+0000 7f1aeca18700 1 Processor -- start 2024-04-23T02:29:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.460+0000 7f1aeca18700 1 -- start start 2024-04-23T02:29:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.460+0000 7f1aeca18700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae80ff5a0 0x7f1ae8100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.460+0000 7f1aeca18700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ae80fff50 0x7f1ae8100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.460+0000 7f1aeca18700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae8100fa0 con 0x7f1ae80ff5a0 2024-04-23T02:29:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.460+0000 7f1aeca18700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae81010e0 con 0x7f1ae80fff50 2024-04-23T02:29:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.460+0000 7f1ae659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae80ff5a0 0x7f1ae8100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.460+0000 7f1ae659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae80ff5a0 0x7f1ae8100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52350/0 (socket says 172.21.15.71:52350) 2024-04-23T02:29:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.460+0000 7f1ae659c700 1 -- 172.21.15.71:0/1104068389 learned_addr learned my addr 172.21.15.71:0/1104068389 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:29:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.460+0000 7f1ae5d9b700 1 --2- 172.21.15.71:0/1104068389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ae80fff50 0x7f1ae8100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.460+0000 7f1ae659c700 1 -- 172.21.15.71:0/1104068389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ae80fff50 msgr2=0x7f1ae8100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.460+0000 7f1ae659c700 1 --2- 172.21.15.71:0/1104068389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ae80fff50 0x7f1ae8100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.460+0000 7f1ae659c700 1 -- 172.21.15.71:0/1104068389 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad80095f0 con 0x7f1ae80ff5a0 2024-04-23T02:29:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.460+0000 7f1ae659c700 1 --2- 172.21.15.71:0/1104068389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae80ff5a0 0x7f1ae8100520 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00b4b0 tx=0x7f1adc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.461+0000 7f1ad77fe700 1 -- 172.21.15.71:0/1104068389 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1adc011800 con 0x7f1ae80ff5a0 2024-04-23T02:29:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.461+0000 7f1ad77fe700 1 -- 172.21.15.71:0/1104068389 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1adc011e00 con 0x7f1ae80ff5a0 2024-04-23T02:29:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.461+0000 7f1aeca18700 1 -- 172.21.15.71:0/1104068389 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae81a2a60 con 0x7f1ae80ff5a0 2024-04-23T02:29:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.461+0000 7f1ad77fe700 1 -- 172.21.15.71:0/1104068389 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1adc010e50 con 0x7f1ae80ff5a0 2024-04-23T02:29:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.461+0000 7f1aeca18700 1 -- 172.21.15.71:0/1104068389 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae81a2e70 con 0x7f1ae80ff5a0 2024-04-23T02:29:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.462+0000 7f1aeca18700 1 -- 172.21.15.71:0/1104068389 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae8109200 con 0x7f1ae80ff5a0 2024-04-23T02:29:12.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.462+0000 7f1ad77fe700 1 -- 172.21.15.71:0/1104068389 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1adc010380 con 0x7f1ae80ff5a0 2024-04-23T02:29:12.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.462+0000 7f1ad77fe700 1 --2- 172.21.15.71:0/1104068389 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1ad006cce0 0x7f1ad006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:12.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.462+0000 7f1ad77fe700 1 -- 172.21.15.71:0/1104068389 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1adc08c190 con 0x7f1ae80ff5a0 2024-04-23T02:29:12.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.462+0000 7f1ae5d9b700 1 --2- 172.21.15.71:0/1104068389 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1ad006cce0 0x7f1ad006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:12.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.463+0000 7f1ae5d9b700 1 --2- 172.21.15.71:0/1104068389 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1ad006cce0 0x7f1ad006f1a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8101d10 tx=0x7f1ad801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:12.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.467+0000 7f1ad77fe700 1 -- 172.21.15.71:0/1104068389 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1adc056200 con 0x7f1ae80ff5a0 2024-04-23T02:29:12.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.702+0000 7f1aeca18700 1 -- 172.21.15.71:0/1104068389 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1ae8108be0 con 0x7f1ae80ff5a0 2024-04-23T02:29:12.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.702+0000 7f1ad77fe700 1 -- 172.21.15.71:0/1104068389 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f1adc059820 con 0x7f1ae80ff5a0 2024-04-23T02:29:12.705 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:29:12.705 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:29:12.705 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:29:12.705 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:29:12.705 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:29:12.705 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:29:12.705 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:29:12.705 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:29:12.705 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:29:12.706 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:29:12.706 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:29:12.706 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:29:12.706 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:29:12.706 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:29:12.706 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:29:12.706 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:29:12.706 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:29:12.706 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:29:12.706 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:29:12.706 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:29:12.706 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:29:12.706 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:29:12.706 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:29:12.707 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:29:12.707 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:29:12.707 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:29:12.707 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:29:12.707 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:29:12.707 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:29:12.707 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:29:12.707 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:29:12.707 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:29:12.707 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:29:12.707 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:29:12.707 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:29:12.708 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:29:12.708 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:29:12.708 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:29:12.708 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:29:12.708 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:29:12.708 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:29:12.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.704+0000 7f1aeca18700 1 -- 172.21.15.71:0/1104068389 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1ad006cce0 msgr2=0x7f1ad006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:12.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.704+0000 7f1aeca18700 1 --2- 172.21.15.71:0/1104068389 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1ad006cce0 0x7f1ad006f1a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8101d10 tx=0x7f1ad801a040 comp rx=0 tx=0).stop 2024-04-23T02:29:12.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.705+0000 7f1aeca18700 1 -- 172.21.15.71:0/1104068389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae80ff5a0 msgr2=0x7f1ae8100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:12.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.705+0000 7f1aeca18700 1 --2- 172.21.15.71:0/1104068389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae80ff5a0 0x7f1ae8100520 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00b4b0 tx=0x7f1adc00b870 comp rx=0 tx=0).stop 2024-04-23T02:29:12.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.705+0000 7f1aeca18700 1 -- 172.21.15.71:0/1104068389 shutdown_connections 2024-04-23T02:29:12.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.705+0000 7f1aeca18700 1 --2- 172.21.15.71:0/1104068389 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1ad006cce0 0x7f1ad006f1a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:12.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.705+0000 7f1aeca18700 1 --2- 172.21.15.71:0/1104068389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae80ff5a0 0x7f1ae8100520 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:12.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.705+0000 7f1aeca18700 1 --2- 172.21.15.71:0/1104068389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ae80fff50 0x7f1ae8100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:12.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.705+0000 7f1aeca18700 1 -- 172.21.15.71:0/1104068389 >> 172.21.15.71:0/1104068389 conn(0x7f1ae8074e30 msgr2=0x7f1ae80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:12.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.705+0000 7f1aeca18700 1 -- 172.21.15.71:0/1104068389 shutdown_connections 2024-04-23T02:29:12.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.705+0000 7f1aeca18700 1 -- 172.21.15.71:0/1104068389 wait complete. 2024-04-23T02:29:12.709 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:29:12.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:12 smithi071 ceph-mon[144544]: pgmap v476: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:12.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:12 smithi071 ceph-mon[144544]: from='client.34706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:29:12.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:12 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2332679710' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:29:12.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.809+0000 7fcf7675b700 1 -- 172.21.15.71:0/3978231608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf700ffc20 msgr2=0x7fcf70100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:12.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.809+0000 7fcf7675b700 1 --2- 172.21.15.71:0/3978231608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf700ffc20 0x7fcf70100080 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009910 tx=0x7fcf64009c20 comp rx=0 tx=0).stop 2024-04-23T02:29:12.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.809+0000 7fcf7675b700 1 -- 172.21.15.71:0/3978231608 shutdown_connections 2024-04-23T02:29:12.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.809+0000 7fcf7675b700 1 --2- 172.21.15.71:0/3978231608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf700ffc20 0x7fcf70100080 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:12.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.809+0000 7fcf7675b700 1 --2- 172.21.15.71:0/3978231608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf70105bd0 0x7fcf70105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:12.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.809+0000 7fcf7675b700 1 -- 172.21.15.71:0/3978231608 >> 172.21.15.71:0/3978231608 conn(0x7fcf70075670 msgr2=0x7fcf70075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:12.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.809+0000 7fcf7675b700 1 -- 172.21.15.71:0/3978231608 shutdown_connections 2024-04-23T02:29:12.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.809+0000 7fcf7675b700 1 -- 172.21.15.71:0/3978231608 wait complete. 2024-04-23T02:29:12.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.810+0000 7fcf7675b700 1 Processor -- start 2024-04-23T02:29:12.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.810+0000 7fcf7675b700 1 -- start start 2024-04-23T02:29:12.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.810+0000 7fcf7675b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf70105bd0 0x7fcf70197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:12.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.810+0000 7fcf7675b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf70197d80 0x7fcf70198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:12.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.810+0000 7fcf7675b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf701a08b0 con 0x7fcf70105bd0 2024-04-23T02:29:12.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.810+0000 7fcf7675b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf701a09f0 con 0x7fcf70197d80 2024-04-23T02:29:12.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.810+0000 7fcf6ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf70105bd0 0x7fcf70197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:12.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.811+0000 7fcf6ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf70105bd0 0x7fcf70197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52382/0 (socket says 172.21.15.71:52382) 2024-04-23T02:29:12.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.811+0000 7fcf6ffff700 1 -- 172.21.15.71:0/3570168284 learned_addr learned my addr 172.21.15.71:0/3570168284 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:29:12.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.811+0000 7fcf6ffff700 1 -- 172.21.15.71:0/3570168284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf70197d80 msgr2=0x7fcf70198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:12.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.811+0000 7fcf6ffff700 1 --2- 172.21.15.71:0/3570168284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf70197d80 0x7fcf70198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:12.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.811+0000 7fcf6ffff700 1 -- 172.21.15.71:0/3570168284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf640095f0 con 0x7fcf70105bd0 2024-04-23T02:29:12.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.811+0000 7fcf6ffff700 1 --2- 172.21.15.71:0/3570168284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf70105bd0 0x7fcf70197840 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000d8d0 tx=0x7fcf6000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:12.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.811+0000 7fcf6d7fa700 1 -- 172.21.15.71:0/3570168284 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf60009940 con 0x7fcf70105bd0 2024-04-23T02:29:12.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.811+0000 7fcf6d7fa700 1 -- 172.21.15.71:0/3570168284 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcf60010460 con 0x7fcf70105bd0 2024-04-23T02:29:12.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.811+0000 7fcf6d7fa700 1 -- 172.21.15.71:0/3570168284 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf6000f5f0 con 0x7fcf70105bd0 2024-04-23T02:29:12.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.811+0000 7fcf7675b700 1 -- 172.21.15.71:0/3570168284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf701a0c50 con 0x7fcf70105bd0 2024-04-23T02:29:12.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.811+0000 7fcf7675b700 1 -- 172.21.15.71:0/3570168284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf701a1170 con 0x7fcf70105bd0 2024-04-23T02:29:12.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.812+0000 7fcf7675b700 1 -- 172.21.15.71:0/3570168284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf701080d0 con 0x7fcf70105bd0 2024-04-23T02:29:12.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.813+0000 7fcf6d7fa700 1 -- 172.21.15.71:0/3570168284 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fcf600105d0 con 0x7fcf70105bd0 2024-04-23T02:29:12.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.813+0000 7fcf6d7fa700 1 --2- 172.21.15.71:0/3570168284 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcf5806cce0 0x7fcf5806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:12.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.813+0000 7fcf6d7fa700 1 -- 172.21.15.71:0/3570168284 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcf6008c4b0 con 0x7fcf70105bd0 2024-04-23T02:29:12.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.813+0000 7fcf6f7fe700 1 --2- 172.21.15.71:0/3570168284 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcf5806cce0 0x7fcf5806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:12.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.813+0000 7fcf6f7fe700 1 --2- 172.21.15.71:0/3570168284 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcf5806cce0 0x7fcf5806f1a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcf640059d0 tx=0x7fcf64005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:12.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:12.817+0000 7fcf6d7fa700 1 -- 172.21.15.71:0/3570168284 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fcf600565a0 con 0x7fcf70105bd0 2024-04-23T02:29:12.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:12 smithi192 ceph-mon[108617]: pgmap v476: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:12.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:12 smithi192 ceph-mon[108617]: from='client.34706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:29:12.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:12 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2332679710' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:29:13.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.033+0000 7fcf7675b700 1 -- 172.21.15.71:0/3570168284 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf7002cce0 con 0x7fcf5806cce0 2024-04-23T02:29:13.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.034+0000 7fcf6d7fa700 1 -- 172.21.15.71:0/3570168284 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcf7002cce0 con 0x7fcf5806cce0 2024-04-23T02:29:13.037 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:29:13.037 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:29:13.037 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:29:13.037 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:29:13.037 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:29:13.037 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:29:13.037 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:29:13.037 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:29:13.037 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:29:13.037 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:29:13.037 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:29:13.037 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:29:13.037 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:29:13.038 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:29:13.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.036+0000 7fcf7675b700 1 -- 172.21.15.71:0/3570168284 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcf5806cce0 msgr2=0x7fcf5806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:13.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.036+0000 7fcf7675b700 1 --2- 172.21.15.71:0/3570168284 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcf5806cce0 0x7fcf5806f1a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcf640059d0 tx=0x7fcf64005960 comp rx=0 tx=0).stop 2024-04-23T02:29:13.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.036+0000 7fcf7675b700 1 -- 172.21.15.71:0/3570168284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf70105bd0 msgr2=0x7fcf70197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:13.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.036+0000 7fcf7675b700 1 --2- 172.21.15.71:0/3570168284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf70105bd0 0x7fcf70197840 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000d8d0 tx=0x7fcf6000dc90 comp rx=0 tx=0).stop 2024-04-23T02:29:13.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.036+0000 7fcf7675b700 1 -- 172.21.15.71:0/3570168284 shutdown_connections 2024-04-23T02:29:13.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.036+0000 7fcf7675b700 1 --2- 172.21.15.71:0/3570168284 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcf5806cce0 0x7fcf5806f1a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:13.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.036+0000 7fcf7675b700 1 --2- 172.21.15.71:0/3570168284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf70105bd0 0x7fcf70197840 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:13.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.036+0000 7fcf7675b700 1 --2- 172.21.15.71:0/3570168284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf70197d80 0x7fcf70198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:13.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.036+0000 7fcf7675b700 1 -- 172.21.15.71:0/3570168284 >> 172.21.15.71:0/3570168284 conn(0x7fcf70075670 msgr2=0x7fcf70104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:13.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.037+0000 7fcf7675b700 1 -- 172.21.15.71:0/3570168284 shutdown_connections 2024-04-23T02:29:13.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.037+0000 7fcf7675b700 1 -- 172.21.15.71:0/3570168284 wait complete. 2024-04-23T02:29:13.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.140+0000 7f98aaab6700 1 -- 172.21.15.71:0/3909793731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a4105bd0 msgr2=0x7f98a4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:13.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.140+0000 7f98aaab6700 1 --2- 172.21.15.71:0/3909793731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a4105bd0 0x7f98a4105fb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f989c00b0a0 tx=0x7f989c00b3b0 comp rx=0 tx=0).stop 2024-04-23T02:29:13.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.140+0000 7f98aaab6700 1 -- 172.21.15.71:0/3909793731 shutdown_connections 2024-04-23T02:29:13.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.140+0000 7f98aaab6700 1 --2- 172.21.15.71:0/3909793731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98a40ffc20 0x7f98a4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:13.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.140+0000 7f98aaab6700 1 --2- 172.21.15.71:0/3909793731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a4105bd0 0x7f98a4105fb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:13.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.140+0000 7f98aaab6700 1 -- 172.21.15.71:0/3909793731 >> 172.21.15.71:0/3909793731 conn(0x7f98a4075670 msgr2=0x7f98a4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:13.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.140+0000 7f98aaab6700 1 -- 172.21.15.71:0/3909793731 shutdown_connections 2024-04-23T02:29:13.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.140+0000 7f98aaab6700 1 -- 172.21.15.71:0/3909793731 wait complete. 2024-04-23T02:29:13.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.141+0000 7f98aaab6700 1 Processor -- start 2024-04-23T02:29:13.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.141+0000 7f98aaab6700 1 -- start start 2024-04-23T02:29:13.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.141+0000 7f98aaab6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98a40ffc20 0x7f98a419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:13.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.141+0000 7f98aaab6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a4105bd0 0x7f98a419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:13.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.141+0000 7f98aaab6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a4196660 con 0x7f98a4105bd0 2024-04-23T02:29:13.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.141+0000 7f98aaab6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a41967d0 con 0x7f98a40ffc20 2024-04-23T02:29:13.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.141+0000 7f98a3fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a4105bd0 0x7f98a419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:13.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.141+0000 7f98a3fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a4105bd0 0x7f98a419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52398/0 (socket says 172.21.15.71:52398) 2024-04-23T02:29:13.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.141+0000 7f98a3fff700 1 -- 172.21.15.71:0/2684143165 learned_addr learned my addr 172.21.15.71:0/2684143165 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:29:13.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.141+0000 7f98a8852700 1 --2- 172.21.15.71:0/2684143165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98a40ffc20 0x7f98a419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:13.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.141+0000 7f98a3fff700 1 -- 172.21.15.71:0/2684143165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98a40ffc20 msgr2=0x7f98a419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:13.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.141+0000 7f98a3fff700 1 --2- 172.21.15.71:0/2684143165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98a40ffc20 0x7f98a419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:13.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.141+0000 7f98a3fff700 1 -- 172.21.15.71:0/2684143165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c009d00 con 0x7f98a4105bd0 2024-04-23T02:29:13.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.141+0000 7f98a3fff700 1 --2- 172.21.15.71:0/2684143165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a4105bd0 0x7f98a419cbb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f989800b4b0 tx=0x7f989800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:13.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.141+0000 7f98a1ffb700 1 -- 172.21.15.71:0/2684143165 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9898011800 con 0x7f98a4105bd0 2024-04-23T02:29:13.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.141+0000 7f98a1ffb700 1 -- 172.21.15.71:0/2684143165 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9898011e00 con 0x7f98a4105bd0 2024-04-23T02:29:13.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.142+0000 7f98aaab6700 1 -- 172.21.15.71:0/2684143165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a4196ab0 con 0x7f98a4105bd0 2024-04-23T02:29:13.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.142+0000 7f98a1ffb700 1 -- 172.21.15.71:0/2684143165 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9898010e60 con 0x7f98a4105bd0 2024-04-23T02:29:13.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.142+0000 7f98aaab6700 1 -- 172.21.15.71:0/2684143165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a4197000 con 0x7f98a4105bd0 2024-04-23T02:29:13.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.143+0000 7f98aaab6700 1 -- 172.21.15.71:0/2684143165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a404f2e0 con 0x7f98a4105bd0 2024-04-23T02:29:13.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.143+0000 7f98a1ffb700 1 -- 172.21.15.71:0/2684143165 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9898011960 con 0x7f98a4105bd0 2024-04-23T02:29:13.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.143+0000 7f98a1ffb700 1 --2- 172.21.15.71:0/2684143165 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f988c06cce0 0x7f988c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:13.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.144+0000 7f98a8852700 1 --2- 172.21.15.71:0/2684143165 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f988c06cce0 0x7f988c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:13.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.144+0000 7f98a1ffb700 1 -- 172.21.15.71:0/2684143165 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f989808c0f0 con 0x7f98a4105bd0 2024-04-23T02:29:13.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.144+0000 7f98a8852700 1 --2- 172.21.15.71:0/2684143165 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f988c06cce0 0x7f988c06f1a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f989c009fd0 tx=0x7f989c0060c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:13.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.148+0000 7f98a1ffb700 1 -- 172.21.15.71:0/2684143165 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9898056160 con 0x7f98a4105bd0 2024-04-23T02:29:13.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.418+0000 7f98aaab6700 1 -- 172.21.15.71:0/2684143165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f98a4197280 con 0x7f98a4105bd0 2024-04-23T02:29:13.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.418+0000 7f98a1ffb700 1 -- 172.21.15.71:0/2684143165 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f9898059780 con 0x7f98a4105bd0 2024-04-23T02:29:13.421 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:29:13.421 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:29:13.421 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:29:13.421 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:29:13.421 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:29:13.421 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:29:13.421 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:29:13.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.420+0000 7f98aaab6700 1 -- 172.21.15.71:0/2684143165 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f988c06cce0 msgr2=0x7f988c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:13.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.420+0000 7f98aaab6700 1 --2- 172.21.15.71:0/2684143165 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f988c06cce0 0x7f988c06f1a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f989c009fd0 tx=0x7f989c0060c0 comp rx=0 tx=0).stop 2024-04-23T02:29:13.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.420+0000 7f98aaab6700 1 -- 172.21.15.71:0/2684143165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a4105bd0 msgr2=0x7f98a419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:13.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.420+0000 7f98aaab6700 1 --2- 172.21.15.71:0/2684143165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a4105bd0 0x7f98a419cbb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f989800b4b0 tx=0x7f989800b7c0 comp rx=0 tx=0).stop 2024-04-23T02:29:13.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.420+0000 7f98aaab6700 1 -- 172.21.15.71:0/2684143165 shutdown_connections 2024-04-23T02:29:13.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.420+0000 7f98aaab6700 1 --2- 172.21.15.71:0/2684143165 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f988c06cce0 0x7f988c06f1a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:13.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.420+0000 7f98aaab6700 1 --2- 172.21.15.71:0/2684143165 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98a40ffc20 0x7f98a419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:13.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.420+0000 7f98aaab6700 1 --2- 172.21.15.71:0/2684143165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a4105bd0 0x7f98a419cbb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:13.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.420+0000 7f98aaab6700 1 -- 172.21.15.71:0/2684143165 >> 172.21.15.71:0/2684143165 conn(0x7f98a4075670 msgr2=0x7f98a4068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:13.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.420+0000 7f98aaab6700 1 -- 172.21.15.71:0/2684143165 shutdown_connections 2024-04-23T02:29:13.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:13.420+0000 7f98aaab6700 1 -- 172.21.15.71:0/2684143165 wait complete. 2024-04-23T02:29:13.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:13 smithi071 ceph-mon[144544]: from='client.34710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:29:13.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:13 smithi071 ceph-mon[144544]: from='client.34714 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:29:13.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:13 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1104068389' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:29:13.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:13 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2684143165' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:29:13.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:13 smithi192 ceph-mon[108617]: from='client.34710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:29:13.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:13 smithi192 ceph-mon[108617]: from='client.34714 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:29:13.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:13 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1104068389' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:29:13.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:13 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2684143165' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:29:14.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:14 smithi071 ceph-mon[144544]: pgmap v477: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:14.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:14 smithi071 ceph-mon[144544]: from='client.34726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:29:14.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:14 smithi192 ceph-mon[108617]: pgmap v477: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:14.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:14 smithi192 ceph-mon[108617]: from='client.34726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:29:16.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:16 smithi192 ceph-mon[108617]: pgmap v478: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:16.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:16 smithi071 ceph-mon[144544]: pgmap v478: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:18.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:18 smithi071 ceph-mon[144544]: pgmap v479: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:18.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:18 smithi192 ceph-mon[108617]: pgmap v479: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:20.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:20 smithi071 ceph-mon[144544]: pgmap v480: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:20.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:29:20.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:20 smithi192 ceph-mon[108617]: pgmap v480: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:20.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:29:22.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:22 smithi071 ceph-mon[144544]: pgmap v481: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:22.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:22 smithi192 ceph-mon[108617]: pgmap v481: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:24.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:24 smithi071 ceph-mon[144544]: pgmap v482: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:24.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:24 smithi192 ceph-mon[108617]: pgmap v482: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:26.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:26 smithi192 ceph-mon[108617]: pgmap v483: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:26.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:26 smithi071 ceph-mon[144544]: pgmap v483: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:27.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:27 smithi192 ceph-mon[108617]: pgmap v484: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:27.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:27 smithi071 ceph-mon[144544]: pgmap v484: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:30.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:29 smithi071 ceph-mon[144544]: pgmap v485: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:30.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:29 smithi192 ceph-mon[108617]: pgmap v485: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:32.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:31 smithi071 ceph-mon[144544]: pgmap v486: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:32.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:31 smithi192 ceph-mon[108617]: pgmap v486: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:34.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:33 smithi071 ceph-mon[144544]: pgmap v487: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:34.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:33 smithi192 ceph-mon[108617]: pgmap v487: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:35.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:34 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:29:35.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:34 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:29:36.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:35 smithi071 ceph-mon[144544]: pgmap v488: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:36.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:35 smithi192 ceph-mon[108617]: pgmap v488: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:38.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:37 smithi071 ceph-mon[144544]: pgmap v489: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:38.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:37 smithi192 ceph-mon[108617]: pgmap v489: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:40.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:39 smithi071 ceph-mon[144544]: pgmap v490: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:40.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:39 smithi192 ceph-mon[108617]: pgmap v490: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:42.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:41 smithi071 ceph-mon[144544]: pgmap v491: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:42.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:41 smithi192 ceph-mon[108617]: pgmap v491: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:43.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.538+0000 7f6b19f6d700 1 -- 172.21.15.71:0/2073174799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b14101850 msgr2=0x7f6b141058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:43.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.538+0000 7f6b19f6d700 1 --2- 172.21.15.71:0/2073174799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b14101850 0x7f6b141058a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009910 tx=0x7f6b04009c20 comp rx=0 tx=0).stop 2024-04-23T02:29:43.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.538+0000 7f6b19f6d700 1 -- 172.21.15.71:0/2073174799 shutdown_connections 2024-04-23T02:29:43.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.538+0000 7f6b19f6d700 1 --2- 172.21.15.71:0/2073174799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b14101850 0x7f6b141058a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:43.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.538+0000 7f6b19f6d700 1 --2- 172.21.15.71:0/2073174799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b14100ea0 0x7f6b14101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:43.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.538+0000 7f6b19f6d700 1 -- 172.21.15.71:0/2073174799 >> 172.21.15.71:0/2073174799 conn(0x7f6b140fc750 msgr2=0x7f6b140feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:43.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.538+0000 7f6b19f6d700 1 -- 172.21.15.71:0/2073174799 shutdown_connections 2024-04-23T02:29:43.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.538+0000 7f6b19f6d700 1 -- 172.21.15.71:0/2073174799 wait complete. 2024-04-23T02:29:43.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.539+0000 7f6b19f6d700 1 Processor -- start 2024-04-23T02:29:43.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.539+0000 7f6b19f6d700 1 -- start start 2024-04-23T02:29:43.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.539+0000 7f6b19f6d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b14100ea0 0x7f6b1419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:43.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.539+0000 7f6b19f6d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b14101850 0x7f6b1419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:43.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.539+0000 7f6b19f6d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1419d1f0 con 0x7f6b14100ea0 2024-04-23T02:29:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.539+0000 7f6b19f6d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b141966a0 con 0x7f6b14101850 2024-04-23T02:29:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.539+0000 7f6b137fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b14100ea0 0x7f6b1419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.539+0000 7f6b137fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b14100ea0 0x7f6b1419c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43760/0 (socket says 172.21.15.71:43760) 2024-04-23T02:29:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.539+0000 7f6b137fe700 1 -- 172.21.15.71:0/298834763 learned_addr learned my addr 172.21.15.71:0/298834763 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:29:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.539+0000 7f6b12ffd700 1 --2- 172.21.15.71:0/298834763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b14101850 0x7f6b1419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.540+0000 7f6b137fe700 1 -- 172.21.15.71:0/298834763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b14101850 msgr2=0x7f6b1419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.540+0000 7f6b137fe700 1 --2- 172.21.15.71:0/298834763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b14101850 0x7f6b1419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.540+0000 7f6b137fe700 1 -- 172.21.15.71:0/298834763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b040095f0 con 0x7f6b14100ea0 2024-04-23T02:29:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.540+0000 7f6b137fe700 1 --2- 172.21.15.71:0/298834763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b14100ea0 0x7f6b1419c620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800e920 tx=0x7f6b0800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.540+0000 7f6b10ff9700 1 -- 172.21.15.71:0/298834763 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b08009790 con 0x7f6b14100ea0 2024-04-23T02:29:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.540+0000 7f6b10ff9700 1 -- 172.21.15.71:0/298834763 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6b0800cba0 con 0x7f6b14100ea0 2024-04-23T02:29:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.540+0000 7f6b19f6d700 1 -- 172.21.15.71:0/298834763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b14196980 con 0x7f6b14100ea0 2024-04-23T02:29:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.540+0000 7f6b10ff9700 1 -- 172.21.15.71:0/298834763 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b08018650 con 0x7f6b14100ea0 2024-04-23T02:29:43.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.540+0000 7f6b19f6d700 1 -- 172.21.15.71:0/298834763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b14196ed0 con 0x7f6b14100ea0 2024-04-23T02:29:43.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.541+0000 7f6b19f6d700 1 -- 172.21.15.71:0/298834763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b141092a0 con 0x7f6b14100ea0 2024-04-23T02:29:43.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.543+0000 7f6b10ff9700 1 -- 172.21.15.71:0/298834763 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6b0800cd10 con 0x7f6b14100ea0 2024-04-23T02:29:43.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.543+0000 7f6b10ff9700 1 --2- 172.21.15.71:0/298834763 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6afc06cce0 0x7f6afc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:43.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.543+0000 7f6b10ff9700 1 -- 172.21.15.71:0/298834763 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6b08014070 con 0x7f6b14100ea0 2024-04-23T02:29:43.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.543+0000 7f6b12ffd700 1 --2- 172.21.15.71:0/298834763 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6afc06cce0 0x7f6afc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:43.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.543+0000 7f6b12ffd700 1 --2- 172.21.15.71:0/298834763 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6afc06cce0 0x7f6afc06f1a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6b040059d0 tx=0x7f6b04005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:43.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.548+0000 7f6b10ff9700 1 -- 172.21.15.71:0/298834763 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6b08052c50 con 0x7f6b14100ea0 2024-04-23T02:29:43.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.759+0000 7f6b19f6d700 1 -- 172.21.15.71:0/298834763 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b14197140 con 0x7f6afc06cce0 2024-04-23T02:29:43.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.760+0000 7f6b10ff9700 1 -- 172.21.15.71:0/298834763 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6b14197140 con 0x7f6afc06cce0 2024-04-23T02:29:43.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.762+0000 7f6b19f6d700 1 -- 172.21.15.71:0/298834763 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6afc06cce0 msgr2=0x7f6afc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:43.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.762+0000 7f6b19f6d700 1 --2- 172.21.15.71:0/298834763 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6afc06cce0 0x7f6afc06f1a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6b040059d0 tx=0x7f6b04005940 comp rx=0 tx=0).stop 2024-04-23T02:29:43.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.762+0000 7f6b19f6d700 1 -- 172.21.15.71:0/298834763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b14100ea0 msgr2=0x7f6b1419c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:43.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.762+0000 7f6b19f6d700 1 --2- 172.21.15.71:0/298834763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b14100ea0 0x7f6b1419c620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800e920 tx=0x7f6b0800ece0 comp rx=0 tx=0).stop 2024-04-23T02:29:43.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.762+0000 7f6b19f6d700 1 -- 172.21.15.71:0/298834763 shutdown_connections 2024-04-23T02:29:43.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.762+0000 7f6b19f6d700 1 --2- 172.21.15.71:0/298834763 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6afc06cce0 0x7f6afc06f1a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:43.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.762+0000 7f6b19f6d700 1 --2- 172.21.15.71:0/298834763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b14100ea0 0x7f6b1419c620 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:43.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.762+0000 7f6b19f6d700 1 --2- 172.21.15.71:0/298834763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b14101850 0x7f6b1419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:43.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.762+0000 7f6b19f6d700 1 -- 172.21.15.71:0/298834763 >> 172.21.15.71:0/298834763 conn(0x7f6b140fc750 msgr2=0x7f6b140fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:43.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.762+0000 7f6b19f6d700 1 -- 172.21.15.71:0/298834763 shutdown_connections 2024-04-23T02:29:43.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.762+0000 7f6b19f6d700 1 -- 172.21.15.71:0/298834763 wait complete. 2024-04-23T02:29:43.775 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:29:43.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.866+0000 7f27fbcc2700 1 -- 172.21.15.71:0/2394540627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27f4100e80 msgr2=0x7f27f4101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:43.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.866+0000 7f27fbcc2700 1 --2- 172.21.15.71:0/2394540627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27f4100e80 0x7f27f4101260 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f27e4009910 tx=0x7f27e4009c20 comp rx=0 tx=0).stop 2024-04-23T02:29:43.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.866+0000 7f27fbcc2700 1 -- 172.21.15.71:0/2394540627 shutdown_connections 2024-04-23T02:29:43.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.866+0000 7f27fbcc2700 1 --2- 172.21.15.71:0/2394540627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f4101830 0x7f27f4105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:43.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.866+0000 7f27fbcc2700 1 --2- 172.21.15.71:0/2394540627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27f4100e80 0x7f27f4101260 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:43.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.866+0000 7f27fbcc2700 1 -- 172.21.15.71:0/2394540627 >> 172.21.15.71:0/2394540627 conn(0x7f27f40fc710 msgr2=0x7f27f40feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:43.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.866+0000 7f27fbcc2700 1 -- 172.21.15.71:0/2394540627 shutdown_connections 2024-04-23T02:29:43.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.866+0000 7f27fbcc2700 1 -- 172.21.15.71:0/2394540627 wait complete. 2024-04-23T02:29:43.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.867+0000 7f27fbcc2700 1 Processor -- start 2024-04-23T02:29:43.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.867+0000 7f27fbcc2700 1 -- start start 2024-04-23T02:29:43.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.867+0000 7f27fbcc2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f4100e80 0x7f27f419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:43.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.867+0000 7f27fbcc2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27f4101830 0x7f27f419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:43.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.867+0000 7f27fbcc2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f419d150 con 0x7f27f4101830 2024-04-23T02:29:43.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.867+0000 7f27fbcc2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f4196600 con 0x7f27f4100e80 2024-04-23T02:29:43.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.867+0000 7f27f925d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27f4101830 0x7f27f419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:43.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.867+0000 7f27f925d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27f4101830 0x7f27f419cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43774/0 (socket says 172.21.15.71:43774) 2024-04-23T02:29:43.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.867+0000 7f27f925d700 1 -- 172.21.15.71:0/4141614001 learned_addr learned my addr 172.21.15.71:0/4141614001 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:29:43.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.867+0000 7f27f9a5e700 1 --2- 172.21.15.71:0/4141614001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f4100e80 0x7f27f419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:43.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.867+0000 7f27f925d700 1 -- 172.21.15.71:0/4141614001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f4100e80 msgr2=0x7f27f419c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:43.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.867+0000 7f27f925d700 1 --2- 172.21.15.71:0/4141614001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f4100e80 0x7f27f419c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:43.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.867+0000 7f27f925d700 1 -- 172.21.15.71:0/4141614001 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e40095f0 con 0x7f27f4101830 2024-04-23T02:29:43.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.867+0000 7f27f9a5e700 1 --2- 172.21.15.71:0/4141614001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f4100e80 0x7f27f419c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:29:43.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.868+0000 7f27f925d700 1 --2- 172.21.15.71:0/4141614001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27f4101830 0x7f27f419cac0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f27f000d900 tx=0x7f27f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:43.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.868+0000 7f27eaffd700 1 -- 172.21.15.71:0/4141614001 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27f0009900 con 0x7f27f4101830 2024-04-23T02:29:43.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.868+0000 7f27eaffd700 1 -- 172.21.15.71:0/4141614001 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f27f0010460 con 0x7f27f4101830 2024-04-23T02:29:43.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.868+0000 7f27fbcc2700 1 -- 172.21.15.71:0/4141614001 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f41968e0 con 0x7f27f4101830 2024-04-23T02:29:43.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.868+0000 7f27eaffd700 1 -- 172.21.15.71:0/4141614001 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27f000f5f0 con 0x7f27f4101830 2024-04-23T02:29:43.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.868+0000 7f27fbcc2700 1 -- 172.21.15.71:0/4141614001 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f4196e30 con 0x7f27f4101830 2024-04-23T02:29:43.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.869+0000 7f27fbcc2700 1 -- 172.21.15.71:0/4141614001 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f4109220 con 0x7f27f4101830 2024-04-23T02:29:43.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.869+0000 7f27eaffd700 1 -- 172.21.15.71:0/4141614001 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f27f00105d0 con 0x7f27f4101830 2024-04-23T02:29:43.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.869+0000 7f27eaffd700 1 --2- 172.21.15.71:0/4141614001 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f27e006cce0 0x7f27e006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:43.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.870+0000 7f27eaffd700 1 -- 172.21.15.71:0/4141614001 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f27f008c460 con 0x7f27f4101830 2024-04-23T02:29:43.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.870+0000 7f27f9a5e700 1 --2- 172.21.15.71:0/4141614001 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f27e006cce0 0x7f27e006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:43.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.870+0000 7f27f9a5e700 1 --2- 172.21.15.71:0/4141614001 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f27e006cce0 0x7f27e006f1a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f27e400b5c0 tx=0x7f27e401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:43.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:43.874+0000 7f27eaffd700 1 -- 172.21.15.71:0/4141614001 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f27f0059ec0 con 0x7f27f4101830 2024-04-23T02:29:44.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.085+0000 7f27fbcc2700 1 -- 172.21.15.71:0/4141614001 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f27f4197050 con 0x7f27e006cce0 2024-04-23T02:29:44.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.086+0000 7f27eaffd700 1 -- 172.21.15.71:0/4141614001 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f27f4197050 con 0x7f27e006cce0 2024-04-23T02:29:44.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.088+0000 7f27fbcc2700 1 -- 172.21.15.71:0/4141614001 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f27e006cce0 msgr2=0x7f27e006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.088+0000 7f27fbcc2700 1 --2- 172.21.15.71:0/4141614001 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f27e006cce0 0x7f27e006f1a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f27e400b5c0 tx=0x7f27e401a040 comp rx=0 tx=0).stop 2024-04-23T02:29:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.088+0000 7f27fbcc2700 1 -- 172.21.15.71:0/4141614001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27f4101830 msgr2=0x7f27f419cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.088+0000 7f27fbcc2700 1 --2- 172.21.15.71:0/4141614001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27f4101830 0x7f27f419cac0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f27f000d900 tx=0x7f27f000dcc0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.088+0000 7f27fbcc2700 1 -- 172.21.15.71:0/4141614001 shutdown_connections 2024-04-23T02:29:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.088+0000 7f27fbcc2700 1 --2- 172.21.15.71:0/4141614001 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f27e006cce0 0x7f27e006f1a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.088+0000 7f27fbcc2700 1 --2- 172.21.15.71:0/4141614001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f4100e80 0x7f27f419c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.088+0000 7f27fbcc2700 1 --2- 172.21.15.71:0/4141614001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27f4101830 0x7f27f419cac0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.088+0000 7f27fbcc2700 1 -- 172.21.15.71:0/4141614001 >> 172.21.15.71:0/4141614001 conn(0x7f27f40fc710 msgr2=0x7f27f40fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.088+0000 7f27fbcc2700 1 -- 172.21.15.71:0/4141614001 shutdown_connections 2024-04-23T02:29:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.088+0000 7f27fbcc2700 1 -- 172.21.15.71:0/4141614001 wait complete. 2024-04-23T02:29:44.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.192+0000 7f6f1b068700 1 -- 172.21.15.71:0/2078665769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f14105350 msgr2=0x7f6f14105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:44.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.192+0000 7f6f1b068700 1 --2- 172.21.15.71:0/2078665769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f14105350 0x7f6f14105730 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00b0a0 tx=0x7f6f0c00b3b0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.192+0000 7f6f1b068700 1 -- 172.21.15.71:0/2078665769 shutdown_connections 2024-04-23T02:29:44.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.192+0000 7f6f1b068700 1 --2- 172.21.15.71:0/2078665769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f140ff340 0x7f6f140ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.192+0000 7f6f1b068700 1 --2- 172.21.15.71:0/2078665769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f14105350 0x7f6f14105730 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.192+0000 7f6f1b068700 1 -- 172.21.15.71:0/2078665769 >> 172.21.15.71:0/2078665769 conn(0x7f6f14074e30 msgr2=0x7f6f14075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:44.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.192+0000 7f6f1b068700 1 -- 172.21.15.71:0/2078665769 shutdown_connections 2024-04-23T02:29:44.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.192+0000 7f6f1b068700 1 -- 172.21.15.71:0/2078665769 wait complete. 2024-04-23T02:29:44.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.193+0000 7f6f1b068700 1 Processor -- start 2024-04-23T02:29:44.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.193+0000 7f6f1b068700 1 -- start start 2024-04-23T02:29:44.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.193+0000 7f6f1b068700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f140ff340 0x7f6f1419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:44.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.193+0000 7f6f1b068700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f14105350 0x7f6f1419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:44.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.193+0000 7f6f1b068700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1419d150 con 0x7f6f140ff340 2024-04-23T02:29:44.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.193+0000 7f6f1b068700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14196600 con 0x7f6f14105350 2024-04-23T02:29:44.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.193+0000 7f6f18e04700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f140ff340 0x7f6f1419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:44.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.193+0000 7f6f18e04700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f140ff340 0x7f6f1419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43786/0 (socket says 172.21.15.71:43786) 2024-04-23T02:29:44.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.193+0000 7f6f18e04700 1 -- 172.21.15.71:0/1165132300 learned_addr learned my addr 172.21.15.71:0/1165132300 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:29:44.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.193+0000 7f6f13fff700 1 --2- 172.21.15.71:0/1165132300 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f14105350 0x7f6f1419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:44.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.193+0000 7f6f18e04700 1 -- 172.21.15.71:0/1165132300 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f14105350 msgr2=0x7f6f1419cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:44.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.193+0000 7f6f18e04700 1 --2- 172.21.15.71:0/1165132300 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f14105350 0x7f6f1419cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.193+0000 7f6f18e04700 1 -- 172.21.15.71:0/1165132300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0c009d00 con 0x7f6f140ff340 2024-04-23T02:29:44.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.193+0000 7f6f18e04700 1 --2- 172.21.15.71:0/1165132300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f140ff340 0x7f6f1419c580 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c000f80 tx=0x7f6f0c009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:44.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.194+0000 7f6f11ffb700 1 -- 172.21.15.71:0/1165132300 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f0c00e040 con 0x7f6f140ff340 2024-04-23T02:29:44.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.194+0000 7f6f1b068700 1 -- 172.21.15.71:0/1165132300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f14196880 con 0x7f6f140ff340 2024-04-23T02:29:44.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.194+0000 7f6f11ffb700 1 -- 172.21.15.71:0/1165132300 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6f0c00be90 con 0x7f6f140ff340 2024-04-23T02:29:44.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.194+0000 7f6f11ffb700 1 -- 172.21.15.71:0/1165132300 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f0c004470 con 0x7f6f140ff340 2024-04-23T02:29:44.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.194+0000 7f6f1b068700 1 -- 172.21.15.71:0/1165132300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f14196d70 con 0x7f6f140ff340 2024-04-23T02:29:44.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.195+0000 7f6f1b068700 1 -- 172.21.15.71:0/1165132300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f14109880 con 0x7f6f140ff340 2024-04-23T02:29:44.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.195+0000 7f6f11ffb700 1 -- 172.21.15.71:0/1165132300 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6f0c019070 con 0x7f6f140ff340 2024-04-23T02:29:44.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.196+0000 7f6f11ffb700 1 --2- 172.21.15.71:0/1165132300 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6f0006cce0 0x7f6f0006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:44.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.196+0000 7f6f13fff700 1 --2- 172.21.15.71:0/1165132300 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6f0006cce0 0x7f6f0006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:44.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.196+0000 7f6f11ffb700 1 -- 172.21.15.71:0/1165132300 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6f0c08dd00 con 0x7f6f140ff340 2024-04-23T02:29:44.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.196+0000 7f6f13fff700 1 --2- 172.21.15.71:0/1165132300 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6f0006cce0 0x7f6f0006f1a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6f140fda00 tx=0x7f6efc00a370 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:44.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.200+0000 7f6f11ffb700 1 -- 172.21.15.71:0/1165132300 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6f0c091030 con 0x7f6f140ff340 2024-04-23T02:29:44.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:43 smithi071 ceph-mon[144544]: pgmap v492: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:44.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:43 smithi192 ceph-mon[108617]: pgmap v492: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:44.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.402+0000 7f6f1b068700 1 -- 172.21.15.71:0/1165132300 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f14196f90 con 0x7f6f0006cce0 2024-04-23T02:29:44.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.413+0000 7f6f11ffb700 1 -- 172.21.15.71:0/1165132300 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6f14196f90 con 0x7f6f0006cce0 2024-04-23T02:29:44.416 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:29:44.416 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (13m) 7m ago 24m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:29:44.416 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (25m) 7m ago 25m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:29:44.416 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (23m) 6m ago 23m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:29:44.416 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (11m) 7m ago 24m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:29:44.416 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (11m) 6m ago 23m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:29:44.416 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (13m) 7m ago 24m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:29:44.416 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (7m) 7m ago 18m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:29:44.416 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (7m) 7m ago 18m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:29:44.416 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (18m) 6m ago 18m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:29:44.416 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (6m) 6m ago 18m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:29:44.417 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (15m) 7m ago 26m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:29:44.417 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (14m) 6m ago 23m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:29:44.417 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (11m) 7m ago 26m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:29:44.417 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (11m) 6m ago 23m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:29:44.417 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (14m) 7m ago 24m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:29:44.417 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (14m) 6m ago 23m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:29:44.417 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (10m) 7m ago 22m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:29:44.417 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (10m) 7m ago 22m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:29:44.417 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (9m) 7m ago 21m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:29:44.417 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (9m) 6m ago 21m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:29:44.417 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (8m) 6m ago 20m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:29:44.417 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (7m) 6m ago 20m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:29:44.417 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (13m) 7m ago 24m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:29:44.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.415+0000 7f6f1b068700 1 -- 172.21.15.71:0/1165132300 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6f0006cce0 msgr2=0x7f6f0006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:44.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.415+0000 7f6f1b068700 1 --2- 172.21.15.71:0/1165132300 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6f0006cce0 0x7f6f0006f1a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6f140fda00 tx=0x7f6efc00a370 comp rx=0 tx=0).stop 2024-04-23T02:29:44.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.415+0000 7f6f1b068700 1 -- 172.21.15.71:0/1165132300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f140ff340 msgr2=0x7f6f1419c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:44.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.415+0000 7f6f1b068700 1 --2- 172.21.15.71:0/1165132300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f140ff340 0x7f6f1419c580 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c000f80 tx=0x7f6f0c009790 comp rx=0 tx=0).stop 2024-04-23T02:29:44.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.415+0000 7f6f1b068700 1 -- 172.21.15.71:0/1165132300 shutdown_connections 2024-04-23T02:29:44.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.415+0000 7f6f1b068700 1 --2- 172.21.15.71:0/1165132300 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6f0006cce0 0x7f6f0006f1a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.415+0000 7f6f1b068700 1 --2- 172.21.15.71:0/1165132300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f140ff340 0x7f6f1419c580 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.415+0000 7f6f1b068700 1 --2- 172.21.15.71:0/1165132300 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f14105350 0x7f6f1419cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.415+0000 7f6f1b068700 1 -- 172.21.15.71:0/1165132300 >> 172.21.15.71:0/1165132300 conn(0x7f6f14074e30 msgr2=0x7f6f14104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:44.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.415+0000 7f6f1b068700 1 -- 172.21.15.71:0/1165132300 shutdown_connections 2024-04-23T02:29:44.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.415+0000 7f6f1b068700 1 -- 172.21.15.71:0/1165132300 wait complete. 2024-04-23T02:29:44.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.517+0000 7f2784a21700 1 -- 172.21.15.71:0/890353365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2780105bd0 msgr2=0x7f2780105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:44.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.517+0000 7f2784a21700 1 --2- 172.21.15.71:0/890353365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2780105bd0 0x7f2780105fb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2770009910 tx=0x7f2770009c20 comp rx=0 tx=0).stop 2024-04-23T02:29:44.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.518+0000 7f2784a21700 1 -- 172.21.15.71:0/890353365 shutdown_connections 2024-04-23T02:29:44.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.518+0000 7f2784a21700 1 --2- 172.21.15.71:0/890353365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27800ffc20 0x7f2780100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.518+0000 7f2784a21700 1 --2- 172.21.15.71:0/890353365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2780105bd0 0x7f2780105fb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.518+0000 7f2784a21700 1 -- 172.21.15.71:0/890353365 >> 172.21.15.71:0/890353365 conn(0x7f2780075670 msgr2=0x7f2780075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:44.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.518+0000 7f2784a21700 1 -- 172.21.15.71:0/890353365 shutdown_connections 2024-04-23T02:29:44.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.518+0000 7f2784a21700 1 -- 172.21.15.71:0/890353365 wait complete. 2024-04-23T02:29:44.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.519+0000 7f2784a21700 1 Processor -- start 2024-04-23T02:29:44.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.519+0000 7f2784a21700 1 -- start start 2024-04-23T02:29:44.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.519+0000 7f2784a21700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27800ffc20 0x7f2780195770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:44.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.519+0000 7f2784a21700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2780105bd0 0x7f2780195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:44.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.519+0000 7f2784a21700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2780196340 con 0x7f27800ffc20 2024-04-23T02:29:44.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.519+0000 7f2784a21700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2780199000 con 0x7f2780105bd0 2024-04-23T02:29:44.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.519+0000 7f277e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27800ffc20 0x7f2780195770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:44.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.519+0000 7f277e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27800ffc20 0x7f2780195770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43808/0 (socket says 172.21.15.71:43808) 2024-04-23T02:29:44.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.519+0000 7f277e59c700 1 -- 172.21.15.71:0/966862454 learned_addr learned my addr 172.21.15.71:0/966862454 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:29:44.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.519+0000 7f277e59c700 1 -- 172.21.15.71:0/966862454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2780105bd0 msgr2=0x7f2780195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:44.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.519+0000 7f277dd9b700 1 --2- 172.21.15.71:0/966862454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2780105bd0 0x7f2780195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:44.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.519+0000 7f277e59c700 1 --2- 172.21.15.71:0/966862454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2780105bd0 0x7f2780195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.519+0000 7f277e59c700 1 -- 172.21.15.71:0/966862454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27700095f0 con 0x7f27800ffc20 2024-04-23T02:29:44.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.519+0000 7f277e59c700 1 --2- 172.21.15.71:0/966862454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27800ffc20 0x7f2780195770 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2770006010 tx=0x7f277000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:44.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.520+0000 7f276f7fe700 1 -- 172.21.15.71:0/966862454 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f277001d070 con 0x7f27800ffc20 2024-04-23T02:29:44.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.520+0000 7f276f7fe700 1 -- 172.21.15.71:0/966862454 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f277000f460 con 0x7f27800ffc20 2024-04-23T02:29:44.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.520+0000 7f2784a21700 1 -- 172.21.15.71:0/966862454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2780199280 con 0x7f27800ffc20 2024-04-23T02:29:44.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.520+0000 7f276f7fe700 1 -- 172.21.15.71:0/966862454 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2770021640 con 0x7f27800ffc20 2024-04-23T02:29:44.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.520+0000 7f2784a21700 1 -- 172.21.15.71:0/966862454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2780199770 con 0x7f27800ffc20 2024-04-23T02:29:44.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.521+0000 7f2784a21700 1 -- 172.21.15.71:0/966862454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27801080f0 con 0x7f27800ffc20 2024-04-23T02:29:44.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.521+0000 7f276f7fe700 1 -- 172.21.15.71:0/966862454 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2770003a60 con 0x7f27800ffc20 2024-04-23T02:29:44.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.521+0000 7f276f7fe700 1 --2- 172.21.15.71:0/966862454 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f276806cce0 0x7f276806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:44.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.521+0000 7f276f7fe700 1 -- 172.21.15.71:0/966862454 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f277008d7e0 con 0x7f27800ffc20 2024-04-23T02:29:44.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.521+0000 7f277dd9b700 1 --2- 172.21.15.71:0/966862454 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f276806cce0 0x7f276806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:44.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.522+0000 7f277dd9b700 1 --2- 172.21.15.71:0/966862454 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f276806cce0 0x7f276806f1a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2774006fd0 tx=0x7f2774008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:44.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.526+0000 7f276f7fe700 1 -- 172.21.15.71:0/966862454 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f27700578d0 con 0x7f27800ffc20 2024-04-23T02:29:44.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.798+0000 7f2784a21700 1 -- 172.21.15.71:0/966862454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2780196ad0 con 0x7f27800ffc20 2024-04-23T02:29:44.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.799+0000 7f276f7fe700 1 -- 172.21.15.71:0/966862454 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f277005aef0 con 0x7f27800ffc20 2024-04-23T02:29:44.801 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:29:44.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:29:44.803 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:29:44.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.801+0000 7f2784a21700 1 -- 172.21.15.71:0/966862454 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f276806cce0 msgr2=0x7f276806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:44.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.801+0000 7f2784a21700 1 --2- 172.21.15.71:0/966862454 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f276806cce0 0x7f276806f1a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2774006fd0 tx=0x7f2774008040 comp rx=0 tx=0).stop 2024-04-23T02:29:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.801+0000 7f2784a21700 1 -- 172.21.15.71:0/966862454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27800ffc20 msgr2=0x7f2780195770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.801+0000 7f2784a21700 1 --2- 172.21.15.71:0/966862454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27800ffc20 0x7f2780195770 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2770006010 tx=0x7f277000be20 comp rx=0 tx=0).stop 2024-04-23T02:29:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.801+0000 7f2784a21700 1 -- 172.21.15.71:0/966862454 shutdown_connections 2024-04-23T02:29:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.801+0000 7f2784a21700 1 --2- 172.21.15.71:0/966862454 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f276806cce0 0x7f276806f1a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.801+0000 7f2784a21700 1 --2- 172.21.15.71:0/966862454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27800ffc20 0x7f2780195770 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.801+0000 7f2784a21700 1 --2- 172.21.15.71:0/966862454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2780105bd0 0x7f2780195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.801+0000 7f2784a21700 1 -- 172.21.15.71:0/966862454 >> 172.21.15.71:0/966862454 conn(0x7f2780075670 msgr2=0x7f27800fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.801+0000 7f2784a21700 1 -- 172.21.15.71:0/966862454 shutdown_connections 2024-04-23T02:29:44.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.801+0000 7f2784a21700 1 -- 172.21.15.71:0/966862454 wait complete. 2024-04-23T02:29:44.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.904+0000 7f22eebf4700 1 -- 172.21.15.71:0/327676244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22e8105bd0 msgr2=0x7f22e8105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:44.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.904+0000 7f22eebf4700 1 --2- 172.21.15.71:0/327676244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22e8105bd0 0x7f22e8105fb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009910 tx=0x7f22d8009c20 comp rx=0 tx=0).stop 2024-04-23T02:29:44.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.904+0000 7f22eebf4700 1 -- 172.21.15.71:0/327676244 shutdown_connections 2024-04-23T02:29:44.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.904+0000 7f22eebf4700 1 --2- 172.21.15.71:0/327676244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e80ffc20 0x7f22e8100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.904+0000 7f22eebf4700 1 --2- 172.21.15.71:0/327676244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22e8105bd0 0x7f22e8105fb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.904+0000 7f22eebf4700 1 -- 172.21.15.71:0/327676244 >> 172.21.15.71:0/327676244 conn(0x7f22e8075670 msgr2=0x7f22e8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:44.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.904+0000 7f22eebf4700 1 -- 172.21.15.71:0/327676244 shutdown_connections 2024-04-23T02:29:44.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.904+0000 7f22eebf4700 1 -- 172.21.15.71:0/327676244 wait complete. 2024-04-23T02:29:44.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.905+0000 7f22eebf4700 1 Processor -- start 2024-04-23T02:29:44.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.905+0000 7f22eebf4700 1 -- start start 2024-04-23T02:29:44.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.905+0000 7f22eebf4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22e80ffc20 0x7f22e81953f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:44.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.905+0000 7f22eebf4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e8105bd0 0x7f22e8195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:44.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.905+0000 7f22eebf4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e8195e70 con 0x7f22e80ffc20 2024-04-23T02:29:44.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.905+0000 7f22eebf4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e8195fe0 con 0x7f22e8105bd0 2024-04-23T02:29:44.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.905+0000 7f22ec990700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22e80ffc20 0x7f22e81953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:44.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.905+0000 7f22ec990700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22e80ffc20 0x7f22e81953f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43832/0 (socket says 172.21.15.71:43832) 2024-04-23T02:29:44.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.905+0000 7f22ec990700 1 -- 172.21.15.71:0/182262238 learned_addr learned my addr 172.21.15.71:0/182262238 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:29:44.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.905+0000 7f22e7fff700 1 --2- 172.21.15.71:0/182262238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e8105bd0 0x7f22e8195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:44.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.905+0000 7f22ec990700 1 -- 172.21.15.71:0/182262238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e8105bd0 msgr2=0x7f22e8195930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:44.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.905+0000 7f22ec990700 1 --2- 172.21.15.71:0/182262238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e8105bd0 0x7f22e8195930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:44.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.905+0000 7f22ec990700 1 -- 172.21.15.71:0/182262238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d80095f0 con 0x7f22e80ffc20 2024-04-23T02:29:44.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.905+0000 7f22ec990700 1 --2- 172.21.15.71:0/182262238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22e80ffc20 0x7f22e81953f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f22d8006010 tx=0x7f22d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:44.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.906+0000 7f22e5ffb700 1 -- 172.21.15.71:0/182262238 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22d801d070 con 0x7f22e80ffc20 2024-04-23T02:29:44.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.906+0000 7f22eebf4700 1 -- 172.21.15.71:0/182262238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e8196260 con 0x7f22e80ffc20 2024-04-23T02:29:44.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.906+0000 7f22e5ffb700 1 -- 172.21.15.71:0/182262238 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f22d800f460 con 0x7f22e80ffc20 2024-04-23T02:29:44.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.906+0000 7f22e5ffb700 1 -- 172.21.15.71:0/182262238 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22d8021620 con 0x7f22e80ffc20 2024-04-23T02:29:44.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.906+0000 7f22eebf4700 1 -- 172.21.15.71:0/182262238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e81a2ce0 con 0x7f22e80ffc20 2024-04-23T02:29:44.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.907+0000 7f22eebf4700 1 -- 172.21.15.71:0/182262238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e81080f0 con 0x7f22e80ffc20 2024-04-23T02:29:44.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.907+0000 7f22e5ffb700 1 -- 172.21.15.71:0/182262238 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f22d800fa40 con 0x7f22e80ffc20 2024-04-23T02:29:44.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.908+0000 7f22e5ffb700 1 --2- 172.21.15.71:0/182262238 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f22d406cce0 0x7f22d406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:44.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.908+0000 7f22e7fff700 1 --2- 172.21.15.71:0/182262238 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f22d406cce0 0x7f22d406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:44.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.908+0000 7f22e5ffb700 1 -- 172.21.15.71:0/182262238 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f22d808d620 con 0x7f22e80ffc20 2024-04-23T02:29:44.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.908+0000 7f22e7fff700 1 --2- 172.21.15.71:0/182262238 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f22d406cce0 0x7f22d406f1a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f22d0005950 tx=0x7f22d00058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:44.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:44.912+0000 7f22e5ffb700 1 -- 172.21.15.71:0/182262238 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f22d805b120 con 0x7f22e80ffc20 2024-04-23T02:29:45.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.149+0000 7f22eebf4700 1 -- 172.21.15.71:0/182262238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f22e8196260 con 0x7f22e80ffc20 2024-04-23T02:29:45.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.149+0000 7f22e5ffb700 1 -- 172.21.15.71:0/182262238 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f22e8196260 con 0x7f22e80ffc20 2024-04-23T02:29:45.152 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:29:45.152 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:29:45.152 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:29:45.152 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:29:45.152 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:29:45.152 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:29:45.152 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:29:45.152 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:29:45.152 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:29:45.152 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:29:45.152 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:29:45.152 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:29:45.152 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:29:45.153 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:29:45.153 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:29:45.153 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:29:45.153 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:29:45.153 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:29:45.153 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:29:45.153 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:29:45.153 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:29:45.153 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:29:45.153 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:29:45.153 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:29:45.153 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:29:45.153 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:29:45.153 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:29:45.153 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:29:45.154 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:29:45.154 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:29:45.154 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:29:45.154 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:29:45.154 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:29:45.154 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:29:45.154 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:29:45.154 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:29:45.154 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:29:45.154 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:29:45.154 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:29:45.154 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:29:45.154 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:29:45.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.151+0000 7f22eebf4700 1 -- 172.21.15.71:0/182262238 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f22d406cce0 msgr2=0x7f22d406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:45.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.151+0000 7f22eebf4700 1 --2- 172.21.15.71:0/182262238 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f22d406cce0 0x7f22d406f1a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f22d0005950 tx=0x7f22d00058e0 comp rx=0 tx=0).stop 2024-04-23T02:29:45.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.151+0000 7f22eebf4700 1 -- 172.21.15.71:0/182262238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22e80ffc20 msgr2=0x7f22e81953f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:45.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.151+0000 7f22eebf4700 1 --2- 172.21.15.71:0/182262238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22e80ffc20 0x7f22e81953f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f22d8006010 tx=0x7f22d800be20 comp rx=0 tx=0).stop 2024-04-23T02:29:45.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.151+0000 7f22eebf4700 1 -- 172.21.15.71:0/182262238 shutdown_connections 2024-04-23T02:29:45.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.151+0000 7f22eebf4700 1 --2- 172.21.15.71:0/182262238 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f22d406cce0 0x7f22d406f1a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:45.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.151+0000 7f22eebf4700 1 --2- 172.21.15.71:0/182262238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22e80ffc20 0x7f22e81953f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:45.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.151+0000 7f22eebf4700 1 --2- 172.21.15.71:0/182262238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e8105bd0 0x7f22e8195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:45.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.151+0000 7f22eebf4700 1 -- 172.21.15.71:0/182262238 >> 172.21.15.71:0/182262238 conn(0x7f22e8075670 msgr2=0x7f22e80fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:45.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.151+0000 7f22eebf4700 1 -- 172.21.15.71:0/182262238 shutdown_connections 2024-04-23T02:29:45.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.151+0000 7f22eebf4700 1 -- 172.21.15.71:0/182262238 wait complete. 2024-04-23T02:29:45.156 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:29:45.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:44 smithi071 ceph-mon[144544]: from='client.34734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:29:45.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:44 smithi071 ceph-mon[144544]: from='client.34738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:29:45.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:44 smithi071 ceph-mon[144544]: from='client.34742 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:29:45.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:44 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/966862454' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:29:45.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.255+0000 7f72f8223700 1 -- 172.21.15.71:0/3827890311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72f0105bd0 msgr2=0x7f72f0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:45.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.255+0000 7f72f8223700 1 --2- 172.21.15.71:0/3827890311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72f0105bd0 0x7f72f0105fb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f72e000b0a0 tx=0x7f72e000b3b0 comp rx=0 tx=0).stop 2024-04-23T02:29:45.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.255+0000 7f72f8223700 1 -- 172.21.15.71:0/3827890311 shutdown_connections 2024-04-23T02:29:45.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.255+0000 7f72f8223700 1 --2- 172.21.15.71:0/3827890311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72f00ffc20 0x7f72f0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:45.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.255+0000 7f72f8223700 1 --2- 172.21.15.71:0/3827890311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72f0105bd0 0x7f72f0105fb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:45.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.255+0000 7f72f8223700 1 -- 172.21.15.71:0/3827890311 >> 172.21.15.71:0/3827890311 conn(0x7f72f0075670 msgr2=0x7f72f0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:45.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.255+0000 7f72f8223700 1 -- 172.21.15.71:0/3827890311 shutdown_connections 2024-04-23T02:29:45.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.255+0000 7f72f8223700 1 -- 172.21.15.71:0/3827890311 wait complete. 2024-04-23T02:29:45.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.256+0000 7f72f8223700 1 Processor -- start 2024-04-23T02:29:45.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.256+0000 7f72f8223700 1 -- start start 2024-04-23T02:29:45.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.256+0000 7f72f8223700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72f00ffc20 0x7f72f0197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:45.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.256+0000 7f72f8223700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72f0105bd0 0x7f72f0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:45.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.256+0000 7f72f8223700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f0198160 con 0x7f72f00ffc20 2024-04-23T02:29:45.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.256+0000 7f72f8223700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f01982a0 con 0x7f72f0105bd0 2024-04-23T02:29:45.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.256+0000 7f72f5fbf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72f00ffc20 0x7f72f0197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:45.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.256+0000 7f72f5fbf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72f00ffc20 0x7f72f0197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43860/0 (socket says 172.21.15.71:43860) 2024-04-23T02:29:45.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.256+0000 7f72f5fbf700 1 -- 172.21.15.71:0/3609309685 learned_addr learned my addr 172.21.15.71:0/3609309685 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:29:45.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.256+0000 7f72f57be700 1 --2- 172.21.15.71:0/3609309685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72f0105bd0 0x7f72f0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:45.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.256+0000 7f72f5fbf700 1 -- 172.21.15.71:0/3609309685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72f0105bd0 msgr2=0x7f72f0197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:45.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.256+0000 7f72f5fbf700 1 --2- 172.21.15.71:0/3609309685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72f0105bd0 0x7f72f0197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:45.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.256+0000 7f72f5fbf700 1 -- 172.21.15.71:0/3609309685 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e0009d00 con 0x7f72f00ffc20 2024-04-23T02:29:45.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.256+0000 7f72f5fbf700 1 --2- 172.21.15.71:0/3609309685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72f00ffc20 0x7f72f0197600 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f72e0000f80 tx=0x7f72e0009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:45.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.256+0000 7f72eeffd700 1 -- 172.21.15.71:0/3609309685 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72e000e040 con 0x7f72f00ffc20 2024-04-23T02:29:45.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.257+0000 7f72eeffd700 1 -- 172.21.15.71:0/3609309685 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f72e000be90 con 0x7f72f00ffc20 2024-04-23T02:29:45.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.257+0000 7f72f8223700 1 -- 172.21.15.71:0/3609309685 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f01a0670 con 0x7f72f00ffc20 2024-04-23T02:29:45.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.257+0000 7f72eeffd700 1 -- 172.21.15.71:0/3609309685 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72e0004470 con 0x7f72f00ffc20 2024-04-23T02:29:45.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.257+0000 7f72f8223700 1 -- 172.21.15.71:0/3609309685 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f01a0a80 con 0x7f72f00ffc20 2024-04-23T02:29:45.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.258+0000 7f72f8223700 1 -- 172.21.15.71:0/3609309685 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f01080f0 con 0x7f72f00ffc20 2024-04-23T02:29:45.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.258+0000 7f72eeffd700 1 -- 172.21.15.71:0/3609309685 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f72e0019070 con 0x7f72f00ffc20 2024-04-23T02:29:45.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.258+0000 7f72eeffd700 1 --2- 172.21.15.71:0/3609309685 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f72d806cce0 0x7f72d806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:45.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.258+0000 7f72eeffd700 1 -- 172.21.15.71:0/3609309685 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f72e008dc20 con 0x7f72f00ffc20 2024-04-23T02:29:45.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.258+0000 7f72f57be700 1 --2- 172.21.15.71:0/3609309685 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f72d806cce0 0x7f72d806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:45.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.259+0000 7f72f57be700 1 --2- 172.21.15.71:0/3609309685 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f72d806cce0 0x7f72d806f1a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f72f0068e30 tx=0x7f72e4009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:45.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.263+0000 7f72eeffd700 1 -- 172.21.15.71:0/3609309685 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f72e0057d10 con 0x7f72f00ffc20 2024-04-23T02:29:45.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:44 smithi192 ceph-mon[108617]: from='client.34734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:29:45.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:44 smithi192 ceph-mon[108617]: from='client.34738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:29:45.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:44 smithi192 ceph-mon[108617]: from='client.34742 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:29:45.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:44 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/966862454' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:29:45.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.473+0000 7f72f8223700 1 -- 172.21.15.71:0/3609309685 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f72f00611d0 con 0x7f72d806cce0 2024-04-23T02:29:45.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.474+0000 7f72eeffd700 1 -- 172.21.15.71:0/3609309685 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f72f00611d0 con 0x7f72d806cce0 2024-04-23T02:29:45.477 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:29:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:29:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:29:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:29:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:29:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:29:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:29:45.477 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:29:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:29:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:29:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:29:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:29:45.478 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:29:45.478 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:29:45.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.476+0000 7f72f8223700 1 -- 172.21.15.71:0/3609309685 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f72d806cce0 msgr2=0x7f72d806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:45.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.476+0000 7f72f8223700 1 --2- 172.21.15.71:0/3609309685 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f72d806cce0 0x7f72d806f1a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f72f0068e30 tx=0x7f72e4009260 comp rx=0 tx=0).stop 2024-04-23T02:29:45.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.476+0000 7f72f8223700 1 -- 172.21.15.71:0/3609309685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72f00ffc20 msgr2=0x7f72f0197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:45.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.476+0000 7f72f8223700 1 --2- 172.21.15.71:0/3609309685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72f00ffc20 0x7f72f0197600 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f72e0000f80 tx=0x7f72e0009790 comp rx=0 tx=0).stop 2024-04-23T02:29:45.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.476+0000 7f72f8223700 1 -- 172.21.15.71:0/3609309685 shutdown_connections 2024-04-23T02:29:45.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.476+0000 7f72f8223700 1 --2- 172.21.15.71:0/3609309685 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f72d806cce0 0x7f72d806f1a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:45.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.476+0000 7f72f8223700 1 --2- 172.21.15.71:0/3609309685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72f00ffc20 0x7f72f0197600 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:45.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.476+0000 7f72f8223700 1 --2- 172.21.15.71:0/3609309685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72f0105bd0 0x7f72f0197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:45.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.476+0000 7f72f8223700 1 -- 172.21.15.71:0/3609309685 >> 172.21.15.71:0/3609309685 conn(0x7f72f0075670 msgr2=0x7f72f00fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:45.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.477+0000 7f72f8223700 1 -- 172.21.15.71:0/3609309685 shutdown_connections 2024-04-23T02:29:45.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.477+0000 7f72f8223700 1 -- 172.21.15.71:0/3609309685 wait complete. 2024-04-23T02:29:45.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.579+0000 7fa00fc0a700 1 -- 172.21.15.71:0/736689169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0080ffda0 msgr2=0x7fa008100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:45.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.579+0000 7fa00fc0a700 1 --2- 172.21.15.71:0/736689169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0080ffda0 0x7fa008100180 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8009910 tx=0x7f9ff8009c20 comp rx=0 tx=0).stop 2024-04-23T02:29:45.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.580+0000 7fa00fc0a700 1 -- 172.21.15.71:0/736689169 shutdown_connections 2024-04-23T02:29:45.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.580+0000 7fa00fc0a700 1 --2- 172.21.15.71:0/736689169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa008100750 0x7fa0081047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:45.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.580+0000 7fa00fc0a700 1 --2- 172.21.15.71:0/736689169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0080ffda0 0x7fa008100180 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:45.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.580+0000 7fa00fc0a700 1 -- 172.21.15.71:0/736689169 >> 172.21.15.71:0/736689169 conn(0x7fa0080fb650 msgr2=0x7fa0080fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:45.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.580+0000 7fa00fc0a700 1 -- 172.21.15.71:0/736689169 shutdown_connections 2024-04-23T02:29:45.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.580+0000 7fa00fc0a700 1 -- 172.21.15.71:0/736689169 wait complete. 2024-04-23T02:29:45.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.580+0000 7fa00fc0a700 1 Processor -- start 2024-04-23T02:29:45.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.580+0000 7fa00fc0a700 1 -- start start 2024-04-23T02:29:45.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.580+0000 7fa00fc0a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0080ffda0 0x7fa008101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:45.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.580+0000 7fa00fc0a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa008100750 0x7fa008101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:45.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.580+0000 7fa00fc0a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0081022c0 con 0x7fa0080ffda0 2024-04-23T02:29:45.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.580+0000 7fa00fc0a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa008102400 con 0x7fa008100750 2024-04-23T02:29:45.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.581+0000 7fa00d9a6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0080ffda0 0x7fa008101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:45.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.581+0000 7fa00d9a6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0080ffda0 0x7fa008101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49422/0 (socket says 172.21.15.71:49422) 2024-04-23T02:29:45.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.581+0000 7fa00d9a6700 1 -- 172.21.15.71:0/1263102080 learned_addr learned my addr 172.21.15.71:0/1263102080 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:29:45.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.581+0000 7fa00d1a5700 1 --2- 172.21.15.71:0/1263102080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa008100750 0x7fa008101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:45.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.581+0000 7fa00d9a6700 1 -- 172.21.15.71:0/1263102080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa008100750 msgr2=0x7fa008101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:45.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.581+0000 7fa00d9a6700 1 --2- 172.21.15.71:0/1263102080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa008100750 0x7fa008101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:45.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.581+0000 7fa00d9a6700 1 -- 172.21.15.71:0/1263102080 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff80095f0 con 0x7fa0080ffda0 2024-04-23T02:29:45.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.581+0000 7fa00d1a5700 1 --2- 172.21.15.71:0/1263102080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa008100750 0x7fa008101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:29:45.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.581+0000 7fa00d9a6700 1 --2- 172.21.15.71:0/1263102080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0080ffda0 0x7fa008101840 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8006010 tx=0x7f9ff800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:45.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.581+0000 7f9ffeffd700 1 -- 172.21.15.71:0/1263102080 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ff801d070 con 0x7fa0080ffda0 2024-04-23T02:29:45.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.581+0000 7f9ffeffd700 1 -- 172.21.15.71:0/1263102080 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9ff800f460 con 0x7fa0080ffda0 2024-04-23T02:29:45.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.581+0000 7f9ffeffd700 1 -- 172.21.15.71:0/1263102080 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ff8021640 con 0x7fa0080ffda0 2024-04-23T02:29:45.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.581+0000 7fa00fc0a700 1 -- 172.21.15.71:0/1263102080 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa008102680 con 0x7fa0080ffda0 2024-04-23T02:29:45.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.581+0000 7fa00fc0a700 1 -- 172.21.15.71:0/1263102080 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0081a2d50 con 0x7fa0080ffda0 2024-04-23T02:29:45.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.582+0000 7fa00fc0a700 1 -- 172.21.15.71:0/1263102080 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00804f2e0 con 0x7fa0080ffda0 2024-04-23T02:29:45.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.583+0000 7f9ffeffd700 1 -- 172.21.15.71:0/1263102080 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9ff800fa60 con 0x7fa0080ffda0 2024-04-23T02:29:45.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.583+0000 7f9ffeffd700 1 --2- 172.21.15.71:0/1263102080 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9ff406cce0 0x7f9ff406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:29:45.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.583+0000 7f9ffeffd700 1 -- 172.21.15.71:0/1263102080 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9ff808d860 con 0x7fa0080ffda0 2024-04-23T02:29:45.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.583+0000 7fa00d1a5700 1 --2- 172.21.15.71:0/1263102080 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9ff406cce0 0x7f9ff406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:29:45.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.584+0000 7fa00d1a5700 1 --2- 172.21.15.71:0/1263102080 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9ff406cce0 0x7f9ff406f1a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa004005fd0 tx=0x7fa004005dc0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:29:45.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.588+0000 7f9ffeffd700 1 -- 172.21.15.71:0/1263102080 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9ff80578d0 con 0x7fa0080ffda0 2024-04-23T02:29:45.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.857+0000 7fa00fc0a700 1 -- 172.21.15.71:0/1263102080 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa0081a2fa0 con 0x7fa0080ffda0 2024-04-23T02:29:45.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.858+0000 7f9ffeffd700 1 -- 172.21.15.71:0/1263102080 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f9ff805aef0 con 0x7fa0080ffda0 2024-04-23T02:29:45.862 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:29:45.862 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:29:45.863 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:29:45.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.860+0000 7fa00fc0a700 1 -- 172.21.15.71:0/1263102080 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9ff406cce0 msgr2=0x7f9ff406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:45.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.860+0000 7fa00fc0a700 1 --2- 172.21.15.71:0/1263102080 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9ff406cce0 0x7f9ff406f1a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa004005fd0 tx=0x7fa004005dc0 comp rx=0 tx=0).stop 2024-04-23T02:29:45.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.860+0000 7fa00fc0a700 1 -- 172.21.15.71:0/1263102080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0080ffda0 msgr2=0x7fa008101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:29:45.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.860+0000 7fa00fc0a700 1 --2- 172.21.15.71:0/1263102080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0080ffda0 0x7fa008101840 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8006010 tx=0x7f9ff800be20 comp rx=0 tx=0).stop 2024-04-23T02:29:45.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.860+0000 7fa00fc0a700 1 -- 172.21.15.71:0/1263102080 shutdown_connections 2024-04-23T02:29:45.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.860+0000 7fa00fc0a700 1 --2- 172.21.15.71:0/1263102080 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9ff406cce0 0x7f9ff406f1a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:45.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.860+0000 7fa00fc0a700 1 --2- 172.21.15.71:0/1263102080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0080ffda0 0x7fa008101840 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:45.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.860+0000 7fa00fc0a700 1 --2- 172.21.15.71:0/1263102080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa008100750 0x7fa008101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:29:45.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.860+0000 7fa00fc0a700 1 -- 172.21.15.71:0/1263102080 >> 172.21.15.71:0/1263102080 conn(0x7fa0080fb650 msgr2=0x7fa008103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:29:45.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.860+0000 7fa00fc0a700 1 -- 172.21.15.71:0/1263102080 shutdown_connections 2024-04-23T02:29:45.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:29:45.860+0000 7fa00fc0a700 1 -- 172.21.15.71:0/1263102080 wait complete. 2024-04-23T02:29:46.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:45 smithi071 ceph-mon[144544]: pgmap v493: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:46.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:45 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/182262238' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:29:46.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:45 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1263102080' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:29:46.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:45 smithi192 ceph-mon[108617]: pgmap v493: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:46.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:45 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/182262238' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:29:46.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:45 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1263102080' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:29:47.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:46 smithi071 ceph-mon[144544]: from='client.34754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:29:47.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:46 smithi192 ceph-mon[108617]: from='client.34754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:29:48.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:47 smithi071 ceph-mon[144544]: pgmap v494: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:48.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:47 smithi192 ceph-mon[108617]: pgmap v494: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:50.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:49 smithi071 ceph-mon[144544]: pgmap v495: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:50.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:29:50.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:49 smithi192 ceph-mon[108617]: pgmap v495: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:50.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:29:52.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:51 smithi071 ceph-mon[144544]: pgmap v496: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:52.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:51 smithi192 ceph-mon[108617]: pgmap v496: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:54.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:53 smithi071 ceph-mon[144544]: pgmap v497: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:54.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:53 smithi192 ceph-mon[108617]: pgmap v497: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:56.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:55 smithi071 ceph-mon[144544]: pgmap v498: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:56.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:55 smithi192 ceph-mon[108617]: pgmap v498: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:58.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:57 smithi071 ceph-mon[144544]: pgmap v499: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:29:58.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:57 smithi192 ceph-mon[108617]: pgmap v499: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:00.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:29:59 smithi071 ceph-mon[144544]: pgmap v500: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:00.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:29:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:30:00.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:29:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:30:00.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:29:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-04-23T02:30:00.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:29:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:30:00.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:29:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:30:00.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:29:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:30:00.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:29:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-04-23T02:30:00.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:29:59 smithi192 ceph-mon[108617]: pgmap v500: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:01.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:00 smithi071 ceph-mon[144544]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:30:01.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:00 smithi071 ceph-mon[144544]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:30:01.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:00 smithi071 ceph-mon[144544]: fs cephfs is degraded 2024-04-23T02:30:01.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:00 smithi071 ceph-mon[144544]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:30:01.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:00 smithi071 ceph-mon[144544]: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:30:01.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:00 smithi071 ceph-mon[144544]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:30:01.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:00 smithi071 ceph-mon[144544]: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:30:01.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:00 smithi192 ceph-mon[108617]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:30:01.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:00 smithi192 ceph-mon[108617]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:30:01.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:00 smithi192 ceph-mon[108617]: fs cephfs is degraded 2024-04-23T02:30:01.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:00 smithi192 ceph-mon[108617]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:30:01.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:00 smithi192 ceph-mon[108617]: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:30:01.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:00 smithi192 ceph-mon[108617]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:30:01.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:00 smithi192 ceph-mon[108617]: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:30:02.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:01 smithi071 ceph-mon[144544]: pgmap v501: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:02.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:01 smithi192 ceph-mon[108617]: pgmap v501: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:04.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:03 smithi071 ceph-mon[144544]: pgmap v502: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:04.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:03 smithi192 ceph-mon[108617]: pgmap v502: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:05.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:04 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:30:05.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:04 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:30:06.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:05 smithi192 ceph-mon[108617]: pgmap v503: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:06.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:05 smithi071 ceph-mon[144544]: pgmap v503: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:08.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:07 smithi192 ceph-mon[108617]: pgmap v504: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:08.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:07 smithi071 ceph-mon[144544]: pgmap v504: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:10.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:09 smithi192 ceph-mon[108617]: pgmap v505: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:10.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:09 smithi071 ceph-mon[144544]: pgmap v505: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:12.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:12 smithi192 ceph-mon[108617]: pgmap v506: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:12.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:12 smithi071 ceph-mon[144544]: pgmap v506: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:14.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:14 smithi192 ceph-mon[108617]: pgmap v507: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:14.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:14 smithi071 ceph-mon[144544]: pgmap v507: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:15.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.974+0000 7fceb2bd0700 1 -- 172.21.15.71:0/2979913786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceac100750 msgr2=0x7fceac1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:15.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.974+0000 7fceb2bd0700 1 --2- 172.21.15.71:0/2979913786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceac100750 0x7fceac1047a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fce98009910 tx=0x7fce98009c20 comp rx=0 tx=0).stop 2024-04-23T02:30:15.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.974+0000 7fceb2bd0700 1 -- 172.21.15.71:0/2979913786 shutdown_connections 2024-04-23T02:30:15.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.974+0000 7fceb2bd0700 1 --2- 172.21.15.71:0/2979913786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceac100750 0x7fceac1047a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:15.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.974+0000 7fceb2bd0700 1 --2- 172.21.15.71:0/2979913786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fceac0ffda0 0x7fceac100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:15.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.974+0000 7fceb2bd0700 1 -- 172.21.15.71:0/2979913786 >> 172.21.15.71:0/2979913786 conn(0x7fceac0fb650 msgr2=0x7fceac0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:15.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.975+0000 7fceb2bd0700 1 -- 172.21.15.71:0/2979913786 shutdown_connections 2024-04-23T02:30:15.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.975+0000 7fceb2bd0700 1 -- 172.21.15.71:0/2979913786 wait complete. 2024-04-23T02:30:15.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.975+0000 7fceb2bd0700 1 Processor -- start 2024-04-23T02:30:15.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.975+0000 7fceb2bd0700 1 -- start start 2024-04-23T02:30:15.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.975+0000 7fceb2bd0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fceac0ffda0 0x7fceac1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:15.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.975+0000 7fceb2bd0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceac100750 0x7fceac197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:15.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.975+0000 7fceb2bd0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceac198240 con 0x7fceac100750 2024-04-23T02:30:15.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.975+0000 7fceb2bd0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceac1983b0 con 0x7fceac0ffda0 2024-04-23T02:30:15.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.975+0000 7fceabfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceac100750 0x7fceac197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:15.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.976+0000 7fceabfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceac100750 0x7fceac197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58318/0 (socket says 172.21.15.71:58318) 2024-04-23T02:30:15.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.976+0000 7fceabfff700 1 -- 172.21.15.71:0/2302731149 learned_addr learned my addr 172.21.15.71:0/2302731149 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:30:15.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.976+0000 7fceb096c700 1 --2- 172.21.15.71:0/2302731149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fceac0ffda0 0x7fceac1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:15.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.976+0000 7fceabfff700 1 -- 172.21.15.71:0/2302731149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fceac0ffda0 msgr2=0x7fceac1976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:15.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.976+0000 7fceabfff700 1 --2- 172.21.15.71:0/2302731149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fceac0ffda0 0x7fceac1976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:15.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.976+0000 7fceabfff700 1 -- 172.21.15.71:0/2302731149 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce980095f0 con 0x7fceac100750 2024-04-23T02:30:15.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.976+0000 7fceabfff700 1 --2- 172.21.15.71:0/2302731149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceac100750 0x7fceac197c20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fce9800bda0 tx=0x7fce9800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:15.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.976+0000 7fcea9ffb700 1 -- 172.21.15.71:0/2302731149 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce9801d070 con 0x7fceac100750 2024-04-23T02:30:15.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.976+0000 7fcea9ffb700 1 -- 172.21.15.71:0/2302731149 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fce9800f460 con 0x7fceac100750 2024-04-23T02:30:15.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.976+0000 7fceb2bd0700 1 -- 172.21.15.71:0/2302731149 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceac1a0750 con 0x7fceac100750 2024-04-23T02:30:15.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.976+0000 7fcea9ffb700 1 -- 172.21.15.71:0/2302731149 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce98021640 con 0x7fceac100750 2024-04-23T02:30:15.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.976+0000 7fceb2bd0700 1 -- 172.21.15.71:0/2302731149 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceac1a0bc0 con 0x7fceac100750 2024-04-23T02:30:15.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.977+0000 7fceb2bd0700 1 -- 172.21.15.71:0/2302731149 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceac1081a0 con 0x7fceac100750 2024-04-23T02:30:15.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.978+0000 7fcea9ffb700 1 -- 172.21.15.71:0/2302731149 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fce9800fa20 con 0x7fceac100750 2024-04-23T02:30:15.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.978+0000 7fcea9ffb700 1 --2- 172.21.15.71:0/2302731149 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fce9c06cce0 0x7fce9c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:15.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.979+0000 7fceb096c700 1 --2- 172.21.15.71:0/2302731149 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fce9c06cce0 0x7fce9c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:15.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.979+0000 7fcea9ffb700 1 -- 172.21.15.71:0/2302731149 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fce9808d7a0 con 0x7fceac100750 2024-04-23T02:30:15.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.979+0000 7fceb096c700 1 --2- 172.21.15.71:0/2302731149 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fce9c06cce0 0x7fce9c06f1a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcea0005fd0 tx=0x7fcea000a560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:15.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:15.983+0000 7fcea9ffb700 1 -- 172.21.15.71:0/2302731149 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fce980578d0 con 0x7fceac100750 2024-04-23T02:30:16.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.195+0000 7fceb2bd0700 1 -- 172.21.15.71:0/2302731149 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fceac0611d0 con 0x7fce9c06cce0 2024-04-23T02:30:16.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.196+0000 7fcea9ffb700 1 -- 172.21.15.71:0/2302731149 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fceac0611d0 con 0x7fce9c06cce0 2024-04-23T02:30:16.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.198+0000 7fceb2bd0700 1 -- 172.21.15.71:0/2302731149 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fce9c06cce0 msgr2=0x7fce9c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:16.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.198+0000 7fceb2bd0700 1 --2- 172.21.15.71:0/2302731149 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fce9c06cce0 0x7fce9c06f1a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcea0005fd0 tx=0x7fcea000a560 comp rx=0 tx=0).stop 2024-04-23T02:30:16.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.198+0000 7fceb2bd0700 1 -- 172.21.15.71:0/2302731149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceac100750 msgr2=0x7fceac197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:16.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.198+0000 7fceb2bd0700 1 --2- 172.21.15.71:0/2302731149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceac100750 0x7fceac197c20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fce9800bda0 tx=0x7fce9800be80 comp rx=0 tx=0).stop 2024-04-23T02:30:16.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.198+0000 7fceb2bd0700 1 -- 172.21.15.71:0/2302731149 shutdown_connections 2024-04-23T02:30:16.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.198+0000 7fceb2bd0700 1 --2- 172.21.15.71:0/2302731149 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fce9c06cce0 0x7fce9c06f1a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:16.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.198+0000 7fceb2bd0700 1 --2- 172.21.15.71:0/2302731149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fceac0ffda0 0x7fceac1976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:16.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.198+0000 7fceb2bd0700 1 --2- 172.21.15.71:0/2302731149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceac100750 0x7fceac197c20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:16.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.198+0000 7fceb2bd0700 1 -- 172.21.15.71:0/2302731149 >> 172.21.15.71:0/2302731149 conn(0x7fceac0fb650 msgr2=0x7fceac0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:16.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.198+0000 7fceb2bd0700 1 -- 172.21.15.71:0/2302731149 shutdown_connections 2024-04-23T02:30:16.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.198+0000 7fceb2bd0700 1 -- 172.21.15.71:0/2302731149 wait complete. 2024-04-23T02:30:16.211 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:30:16.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.302+0000 7f1170243700 1 -- 172.21.15.71:0/1015141937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11680ffc20 msgr2=0x7f1168100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:16.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.302+0000 7f1170243700 1 --2- 172.21.15.71:0/1015141937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11680ffc20 0x7f1168100080 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1158009910 tx=0x7f1158009c20 comp rx=0 tx=0).stop 2024-04-23T02:30:16.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.302+0000 7f1170243700 1 -- 172.21.15.71:0/1015141937 shutdown_connections 2024-04-23T02:30:16.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.302+0000 7f1170243700 1 --2- 172.21.15.71:0/1015141937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11680ffc20 0x7f1168100080 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:16.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.302+0000 7f1170243700 1 --2- 172.21.15.71:0/1015141937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1168105bd0 0x7f1168105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:16.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.302+0000 7f1170243700 1 -- 172.21.15.71:0/1015141937 >> 172.21.15.71:0/1015141937 conn(0x7f1168075670 msgr2=0x7f1168075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:16.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.302+0000 7f1170243700 1 -- 172.21.15.71:0/1015141937 shutdown_connections 2024-04-23T02:30:16.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.302+0000 7f1170243700 1 -- 172.21.15.71:0/1015141937 wait complete. 2024-04-23T02:30:16.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.302+0000 7f1170243700 1 Processor -- start 2024-04-23T02:30:16.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.302+0000 7f1170243700 1 -- start start 2024-04-23T02:30:16.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.303+0000 7f1170243700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11680ffc20 0x7f116819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:16.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.303+0000 7f1170243700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1168105bd0 0x7f116819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:16.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.303+0000 7f1170243700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116819d150 con 0x7f11680ffc20 2024-04-23T02:30:16.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.303+0000 7f1170243700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1168196600 con 0x7f1168105bd0 2024-04-23T02:30:16.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.303+0000 7f116dfdf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11680ffc20 0x7f116819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:16.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.303+0000 7f116dfdf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11680ffc20 0x7f116819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58332/0 (socket says 172.21.15.71:58332) 2024-04-23T02:30:16.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.303+0000 7f116dfdf700 1 -- 172.21.15.71:0/1770647311 learned_addr learned my addr 172.21.15.71:0/1770647311 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:30:16.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.303+0000 7f116d7de700 1 --2- 172.21.15.71:0/1770647311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1168105bd0 0x7f116819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:16.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.303+0000 7f116dfdf700 1 -- 172.21.15.71:0/1770647311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1168105bd0 msgr2=0x7f116819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:16.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.303+0000 7f116dfdf700 1 --2- 172.21.15.71:0/1770647311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1168105bd0 0x7f116819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:16.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.303+0000 7f116dfdf700 1 -- 172.21.15.71:0/1770647311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11580095f0 con 0x7f11680ffc20 2024-04-23T02:30:16.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.303+0000 7f116d7de700 1 --2- 172.21.15.71:0/1770647311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1168105bd0 0x7f116819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:30:16.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.303+0000 7f116dfdf700 1 --2- 172.21.15.71:0/1770647311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11680ffc20 0x7f116819c580 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f116400d8d0 tx=0x7f116400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:16.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.303+0000 7f115effd700 1 -- 172.21.15.71:0/1770647311 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1164009940 con 0x7f11680ffc20 2024-04-23T02:30:16.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.303+0000 7f115effd700 1 -- 172.21.15.71:0/1770647311 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1164010460 con 0x7f11680ffc20 2024-04-23T02:30:16.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.303+0000 7f1170243700 1 -- 172.21.15.71:0/1770647311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11681968e0 con 0x7f11680ffc20 2024-04-23T02:30:16.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.303+0000 7f115effd700 1 -- 172.21.15.71:0/1770647311 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f116400f5f0 con 0x7f11680ffc20 2024-04-23T02:30:16.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.303+0000 7f1170243700 1 -- 172.21.15.71:0/1770647311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1168196e30 con 0x7f11680ffc20 2024-04-23T02:30:16.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.304+0000 7f1170243700 1 -- 172.21.15.71:0/1770647311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116804f2e0 con 0x7f11680ffc20 2024-04-23T02:30:16.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.305+0000 7f115effd700 1 -- 172.21.15.71:0/1770647311 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f11640105d0 con 0x7f11680ffc20 2024-04-23T02:30:16.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.305+0000 7f115effd700 1 --2- 172.21.15.71:0/1770647311 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f115406cce0 0x7f115406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:16.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.305+0000 7f115effd700 1 -- 172.21.15.71:0/1770647311 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f116408c4f0 con 0x7f11680ffc20 2024-04-23T02:30:16.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.305+0000 7f116d7de700 1 --2- 172.21.15.71:0/1770647311 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f115406cce0 0x7f115406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:16.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.305+0000 7f116d7de700 1 --2- 172.21.15.71:0/1770647311 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f115406cce0 0x7f115406f1a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1158005b40 tx=0x7f115801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:16.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.310+0000 7f115effd700 1 -- 172.21.15.71:0/1770647311 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f116405a070 con 0x7f11680ffc20 2024-04-23T02:30:16.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:16 smithi192 ceph-mon[108617]: pgmap v508: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:16.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:16 smithi071 ceph-mon[144544]: pgmap v508: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:16.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.521+0000 7f1170243700 1 -- 172.21.15.71:0/1770647311 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1168197140 con 0x7f115406cce0 2024-04-23T02:30:16.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.522+0000 7f115effd700 1 -- 172.21.15.71:0/1770647311 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1168197140 con 0x7f115406cce0 2024-04-23T02:30:16.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.524+0000 7f1170243700 1 -- 172.21.15.71:0/1770647311 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f115406cce0 msgr2=0x7f115406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:16.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.524+0000 7f1170243700 1 --2- 172.21.15.71:0/1770647311 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f115406cce0 0x7f115406f1a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1158005b40 tx=0x7f115801a040 comp rx=0 tx=0).stop 2024-04-23T02:30:16.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.524+0000 7f1170243700 1 -- 172.21.15.71:0/1770647311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11680ffc20 msgr2=0x7f116819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:16.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.524+0000 7f1170243700 1 --2- 172.21.15.71:0/1770647311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11680ffc20 0x7f116819c580 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f116400d8d0 tx=0x7f116400dc90 comp rx=0 tx=0).stop 2024-04-23T02:30:16.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.524+0000 7f1170243700 1 -- 172.21.15.71:0/1770647311 shutdown_connections 2024-04-23T02:30:16.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.524+0000 7f1170243700 1 --2- 172.21.15.71:0/1770647311 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f115406cce0 0x7f115406f1a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:16.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.524+0000 7f1170243700 1 --2- 172.21.15.71:0/1770647311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11680ffc20 0x7f116819c580 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:16.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.524+0000 7f1170243700 1 --2- 172.21.15.71:0/1770647311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1168105bd0 0x7f116819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:16.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.524+0000 7f1170243700 1 -- 172.21.15.71:0/1770647311 >> 172.21.15.71:0/1770647311 conn(0x7f1168075670 msgr2=0x7f1168068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:16.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.524+0000 7f1170243700 1 -- 172.21.15.71:0/1770647311 shutdown_connections 2024-04-23T02:30:16.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.524+0000 7f1170243700 1 -- 172.21.15.71:0/1770647311 wait complete. 2024-04-23T02:30:16.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.626+0000 7f0afc130700 1 -- 172.21.15.71:0/247063773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0af4100750 msgr2=0x7f0af41047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:16.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.626+0000 7f0afc130700 1 --2- 172.21.15.71:0/247063773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0af4100750 0x7f0af41047a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009910 tx=0x7f0af0009c20 comp rx=0 tx=0).stop 2024-04-23T02:30:16.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.626+0000 7f0afc130700 1 -- 172.21.15.71:0/247063773 shutdown_connections 2024-04-23T02:30:16.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.626+0000 7f0afc130700 1 --2- 172.21.15.71:0/247063773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0af4100750 0x7f0af41047a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:16.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.626+0000 7f0afc130700 1 --2- 172.21.15.71:0/247063773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0af40ffda0 0x7f0af4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:16.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.626+0000 7f0afc130700 1 -- 172.21.15.71:0/247063773 >> 172.21.15.71:0/247063773 conn(0x7f0af40fb650 msgr2=0x7f0af40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:16.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.626+0000 7f0afc130700 1 -- 172.21.15.71:0/247063773 shutdown_connections 2024-04-23T02:30:16.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.626+0000 7f0afc130700 1 -- 172.21.15.71:0/247063773 wait complete. 2024-04-23T02:30:16.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.627+0000 7f0afc130700 1 Processor -- start 2024-04-23T02:30:16.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.627+0000 7f0afc130700 1 -- start start 2024-04-23T02:30:16.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.627+0000 7f0afc130700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0af40ffda0 0x7f0af4197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:16.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.627+0000 7f0afc130700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0af4100750 0x7f0af4197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:16.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.627+0000 7f0afc130700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af41981d0 con 0x7f0af4100750 2024-04-23T02:30:16.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.627+0000 7f0afc130700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af4198310 con 0x7f0af40ffda0 2024-04-23T02:30:16.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.627+0000 7f0af96cb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0af4100750 0x7f0af4197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:16.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.627+0000 7f0af96cb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0af4100750 0x7f0af4197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58346/0 (socket says 172.21.15.71:58346) 2024-04-23T02:30:16.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.627+0000 7f0af96cb700 1 -- 172.21.15.71:0/2768482044 learned_addr learned my addr 172.21.15.71:0/2768482044 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:30:16.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.627+0000 7f0af9ecc700 1 --2- 172.21.15.71:0/2768482044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0af40ffda0 0x7f0af4197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:16.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.627+0000 7f0af96cb700 1 -- 172.21.15.71:0/2768482044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0af40ffda0 msgr2=0x7f0af4197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:16.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.627+0000 7f0af96cb700 1 --2- 172.21.15.71:0/2768482044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0af40ffda0 0x7f0af4197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:16.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.627+0000 7f0af96cb700 1 -- 172.21.15.71:0/2768482044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af00095f0 con 0x7f0af4100750 2024-04-23T02:30:16.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.627+0000 7f0af9ecc700 1 --2- 172.21.15.71:0/2768482044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0af40ffda0 0x7f0af4197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:30:16.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.628+0000 7f0af96cb700 1 --2- 172.21.15.71:0/2768482044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0af4100750 0x7f0af4197c40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0af000bda0 tx=0x7f0af000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:16.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.628+0000 7f0aeaffd700 1 -- 172.21.15.71:0/2768482044 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0af001d070 con 0x7f0af4100750 2024-04-23T02:30:16.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.628+0000 7f0aeaffd700 1 -- 172.21.15.71:0/2768482044 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0af000f460 con 0x7f0af4100750 2024-04-23T02:30:16.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.628+0000 7f0afc130700 1 -- 172.21.15.71:0/2768482044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af41a0770 con 0x7f0af4100750 2024-04-23T02:30:16.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.628+0000 7f0aeaffd700 1 -- 172.21.15.71:0/2768482044 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0af0021640 con 0x7f0af4100750 2024-04-23T02:30:16.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.628+0000 7f0afc130700 1 -- 172.21.15.71:0/2768482044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af41a0b20 con 0x7f0af4100750 2024-04-23T02:30:16.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.629+0000 7f0afc130700 1 -- 172.21.15.71:0/2768482044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af41081a0 con 0x7f0af4100750 2024-04-23T02:30:16.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.629+0000 7f0aeaffd700 1 -- 172.21.15.71:0/2768482044 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0af000f9b0 con 0x7f0af4100750 2024-04-23T02:30:16.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.630+0000 7f0aeaffd700 1 --2- 172.21.15.71:0/2768482044 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ae006cce0 0x7f0ae006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:16.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.630+0000 7f0aeaffd700 1 -- 172.21.15.71:0/2768482044 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0af008d8d0 con 0x7f0af4100750 2024-04-23T02:30:16.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.630+0000 7f0af9ecc700 1 --2- 172.21.15.71:0/2768482044 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ae006cce0 0x7f0ae006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:16.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.630+0000 7f0af9ecc700 1 --2- 172.21.15.71:0/2768482044 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ae006cce0 0x7f0ae006f1a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4005fd0 tx=0x7f0ae4005e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:16.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.634+0000 7f0aeaffd700 1 -- 172.21.15.71:0/2768482044 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0af00578a0 con 0x7f0af4100750 2024-04-23T02:30:16.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.840+0000 7f0afc130700 1 -- 172.21.15.71:0/2768482044 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0af40611d0 con 0x7f0ae006cce0 2024-04-23T02:30:16.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.850+0000 7f0aeaffd700 1 -- 172.21.15.71:0/2768482044 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0af40611d0 con 0x7f0ae006cce0 2024-04-23T02:30:16.853 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:30:16.853 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (13m) 7m ago 25m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:30:16.853 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (25m) 7m ago 25m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:30:16.853 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (24m) 7m ago 24m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:30:16.853 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (11m) 7m ago 25m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:30:16.853 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (11m) 7m ago 24m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:30:16.853 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (13m) 7m ago 25m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:30:16.853 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (8m) 7m ago 19m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:30:16.853 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (7m) 7m ago 19m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:30:16.853 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (19m) 7m ago 19m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:30:16.853 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (7m) 7m ago 19m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:30:16.853 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (15m) 7m ago 26m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:30:16.853 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (15m) 7m ago 24m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:30:16.854 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (12m) 7m ago 27m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:30:16.854 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (12m) 7m ago 24m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:30:16.854 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (14m) 7m ago 25m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:30:16.854 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (14m) 7m ago 24m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:30:16.854 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (11m) 7m ago 23m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:30:16.854 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (10m) 7m ago 22m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:30:16.854 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (10m) 7m ago 22m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:30:16.854 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (9m) 7m ago 21m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:30:16.854 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (9m) 7m ago 21m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:30:16.854 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (8m) 7m ago 20m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:30:16.854 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (14m) 7m ago 24m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:30:16.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.852+0000 7f0afc130700 1 -- 172.21.15.71:0/2768482044 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ae006cce0 msgr2=0x7f0ae006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:16.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.852+0000 7f0afc130700 1 --2- 172.21.15.71:0/2768482044 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ae006cce0 0x7f0ae006f1a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4005fd0 tx=0x7f0ae4005e30 comp rx=0 tx=0).stop 2024-04-23T02:30:16.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.852+0000 7f0afc130700 1 -- 172.21.15.71:0/2768482044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0af4100750 msgr2=0x7f0af4197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:16.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.852+0000 7f0afc130700 1 --2- 172.21.15.71:0/2768482044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0af4100750 0x7f0af4197c40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0af000bda0 tx=0x7f0af000be80 comp rx=0 tx=0).stop 2024-04-23T02:30:16.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.852+0000 7f0afc130700 1 -- 172.21.15.71:0/2768482044 shutdown_connections 2024-04-23T02:30:16.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.852+0000 7f0afc130700 1 --2- 172.21.15.71:0/2768482044 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ae006cce0 0x7f0ae006f1a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:16.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.852+0000 7f0afc130700 1 --2- 172.21.15.71:0/2768482044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0af40ffda0 0x7f0af4197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:16.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.852+0000 7f0afc130700 1 --2- 172.21.15.71:0/2768482044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0af4100750 0x7f0af4197c40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:16.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.852+0000 7f0afc130700 1 -- 172.21.15.71:0/2768482044 >> 172.21.15.71:0/2768482044 conn(0x7f0af40fb650 msgr2=0x7f0af40fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:16.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.852+0000 7f0afc130700 1 -- 172.21.15.71:0/2768482044 shutdown_connections 2024-04-23T02:30:16.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.852+0000 7f0afc130700 1 -- 172.21.15.71:0/2768482044 wait complete. 2024-04-23T02:30:16.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.956+0000 7fdcf43cf700 1 -- 172.21.15.71:0/1734217116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcec0ffc20 msgr2=0x7fdcec100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:16.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.956+0000 7fdcf43cf700 1 --2- 172.21.15.71:0/1734217116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcec0ffc20 0x7fdcec100080 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009910 tx=0x7fdce8009c20 comp rx=0 tx=0).stop 2024-04-23T02:30:16.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.956+0000 7fdcf43cf700 1 -- 172.21.15.71:0/1734217116 shutdown_connections 2024-04-23T02:30:16.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.956+0000 7fdcf43cf700 1 --2- 172.21.15.71:0/1734217116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcec0ffc20 0x7fdcec100080 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:16.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.956+0000 7fdcf43cf700 1 --2- 172.21.15.71:0/1734217116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcec105bd0 0x7fdcec105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:16.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.956+0000 7fdcf43cf700 1 -- 172.21.15.71:0/1734217116 >> 172.21.15.71:0/1734217116 conn(0x7fdcec075670 msgr2=0x7fdcec075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:16.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.956+0000 7fdcf43cf700 1 -- 172.21.15.71:0/1734217116 shutdown_connections 2024-04-23T02:30:16.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.956+0000 7fdcf43cf700 1 -- 172.21.15.71:0/1734217116 wait complete. 2024-04-23T02:30:16.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdcf43cf700 1 Processor -- start 2024-04-23T02:30:16.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdcf43cf700 1 -- start start 2024-04-23T02:30:16.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdcf43cf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcec0ffc20 0x7fdcec19b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:16.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdcf43cf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcec105bd0 0x7fdcec19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:16.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdcf43cf700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec19c0f0 con 0x7fdcec105bd0 2024-04-23T02:30:16.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdcf43cf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec1955a0 con 0x7fdcec0ffc20 2024-04-23T02:30:16.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdcf196a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcec105bd0 0x7fdcec19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:16.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdcf196a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcec105bd0 0x7fdcec19ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58358/0 (socket says 172.21.15.71:58358) 2024-04-23T02:30:16.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdcf196a700 1 -- 172.21.15.71:0/2437969671 learned_addr learned my addr 172.21.15.71:0/2437969671 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:30:16.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdcf216b700 1 --2- 172.21.15.71:0/2437969671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcec0ffc20 0x7fdcec19b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:16.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdcf196a700 1 -- 172.21.15.71:0/2437969671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcec0ffc20 msgr2=0x7fdcec19b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:16.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdcf196a700 1 --2- 172.21.15.71:0/2437969671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcec0ffc20 0x7fdcec19b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:16.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdcf196a700 1 -- 172.21.15.71:0/2437969671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce80095f0 con 0x7fdcec105bd0 2024-04-23T02:30:16.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdcf216b700 1 --2- 172.21.15.71:0/2437969671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcec0ffc20 0x7fdcec19b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:30:16.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdcf196a700 1 --2- 172.21.15.71:0/2437969671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcec105bd0 0x7fdcec19ba60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdce8006010 tx=0x7fdce800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:16.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdce37fe700 1 -- 172.21.15.71:0/2437969671 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdce801f070 con 0x7fdcec105bd0 2024-04-23T02:30:16.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdce37fe700 1 -- 172.21.15.71:0/2437969671 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdce8019610 con 0x7fdcec105bd0 2024-04-23T02:30:16.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdcf43cf700 1 -- 172.21.15.71:0/2437969671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcec195820 con 0x7fdcec105bd0 2024-04-23T02:30:16.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.957+0000 7fdce37fe700 1 -- 172.21.15.71:0/2437969671 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdce800f690 con 0x7fdcec105bd0 2024-04-23T02:30:16.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.958+0000 7fdcf43cf700 1 -- 172.21.15.71:0/2437969671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcec195d10 con 0x7fdcec105bd0 2024-04-23T02:30:16.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.958+0000 7fdcf43cf700 1 -- 172.21.15.71:0/2437969671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcec04f2e0 con 0x7fdcec105bd0 2024-04-23T02:30:16.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.959+0000 7fdce37fe700 1 -- 172.21.15.71:0/2437969671 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fdce8019bd0 con 0x7fdcec105bd0 2024-04-23T02:30:16.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.960+0000 7fdce37fe700 1 --2- 172.21.15.71:0/2437969671 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdcd806cce0 0x7fdcd806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:16.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.960+0000 7fdce37fe700 1 -- 172.21.15.71:0/2437969671 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdce808fcd0 con 0x7fdcec105bd0 2024-04-23T02:30:16.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.960+0000 7fdcf216b700 1 --2- 172.21.15.71:0/2437969671 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdcd806cce0 0x7fdcd806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:16.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.960+0000 7fdcf216b700 1 --2- 172.21.15.71:0/2437969671 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdcd806cce0 0x7fdcd806f1a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc005950 tx=0x7fdcdc005f50 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:16.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:16.964+0000 7fdce37fe700 1 -- 172.21.15.71:0/2437969671 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fdce805d850 con 0x7fdcec105bd0 2024-04-23T02:30:17.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:17 smithi192 ceph-mon[108617]: from='client.34762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:30:17.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.238+0000 7fdcf43cf700 1 -- 172.21.15.71:0/2437969671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdcec196ab0 con 0x7fdcec105bd0 2024-04-23T02:30:17.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.238+0000 7fdce37fe700 1 -- 172.21.15.71:0/2437969671 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7fdce805d3e0 con 0x7fdcec105bd0 2024-04-23T02:30:17.241 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:30:17.241 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:30:17.241 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:30:17.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:30:17.241 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:30:17.241 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:30:17.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:30:17.241 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:30:17.242 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:30:17.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:30:17.242 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:30:17.242 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:30:17.242 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:30:17.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:30:17.242 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:30:17.242 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:30:17.242 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:30:17.242 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:30:17.242 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:30:17.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:17 smithi071 ceph-mon[144544]: from='client.34762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:30:17.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.240+0000 7fdcf43cf700 1 -- 172.21.15.71:0/2437969671 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdcd806cce0 msgr2=0x7fdcd806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:17.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.240+0000 7fdcf43cf700 1 --2- 172.21.15.71:0/2437969671 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdcd806cce0 0x7fdcd806f1a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc005950 tx=0x7fdcdc005f50 comp rx=0 tx=0).stop 2024-04-23T02:30:17.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.240+0000 7fdcf43cf700 1 -- 172.21.15.71:0/2437969671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcec105bd0 msgr2=0x7fdcec19ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:17.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.240+0000 7fdcf43cf700 1 --2- 172.21.15.71:0/2437969671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcec105bd0 0x7fdcec19ba60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdce8006010 tx=0x7fdce800be20 comp rx=0 tx=0).stop 2024-04-23T02:30:17.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.240+0000 7fdcf43cf700 1 -- 172.21.15.71:0/2437969671 shutdown_connections 2024-04-23T02:30:17.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.240+0000 7fdcf43cf700 1 --2- 172.21.15.71:0/2437969671 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdcd806cce0 0x7fdcd806f1a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:17.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.240+0000 7fdcf43cf700 1 --2- 172.21.15.71:0/2437969671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcec0ffc20 0x7fdcec19b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:17.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.240+0000 7fdcf43cf700 1 --2- 172.21.15.71:0/2437969671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcec105bd0 0x7fdcec19ba60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:17.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.241+0000 7fdcf43cf700 1 -- 172.21.15.71:0/2437969671 >> 172.21.15.71:0/2437969671 conn(0x7fdcec075670 msgr2=0x7fdcec0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:17.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.241+0000 7fdcf43cf700 1 -- 172.21.15.71:0/2437969671 shutdown_connections 2024-04-23T02:30:17.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.241+0000 7fdcf43cf700 1 -- 172.21.15.71:0/2437969671 wait complete. 2024-04-23T02:30:17.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.342+0000 7f2016019700 1 -- 172.21.15.71:0/1645274584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20100ffb40 msgr2=0x7f20100fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:17.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.342+0000 7f2016019700 1 --2- 172.21.15.71:0/1645274584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20100ffb40 0x7f20100fffc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2004009910 tx=0x7f2004009c20 comp rx=0 tx=0).stop 2024-04-23T02:30:17.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.342+0000 7f2016019700 1 -- 172.21.15.71:0/1645274584 shutdown_connections 2024-04-23T02:30:17.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.342+0000 7f2016019700 1 --2- 172.21.15.71:0/1645274584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20100ffb40 0x7f20100fffc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:17.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.342+0000 7f2016019700 1 --2- 172.21.15.71:0/1645274584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2010105bf0 0x7f2010105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:17.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.342+0000 7f2016019700 1 -- 172.21.15.71:0/1645274584 >> 172.21.15.71:0/1645274584 conn(0x7f20100fb610 msgr2=0x7f20100fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:17.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.342+0000 7f2016019700 1 -- 172.21.15.71:0/1645274584 shutdown_connections 2024-04-23T02:30:17.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.342+0000 7f2016019700 1 -- 172.21.15.71:0/1645274584 wait complete. 2024-04-23T02:30:17.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.343+0000 7f2016019700 1 Processor -- start 2024-04-23T02:30:17.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.343+0000 7f2016019700 1 -- start start 2024-04-23T02:30:17.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.343+0000 7f2016019700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20100ffb40 0x7f2010197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:17.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.343+0000 7f2016019700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2010105bf0 0x7f2010197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:17.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.343+0000 7f2016019700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20101980d0 con 0x7f20100ffb40 2024-04-23T02:30:17.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.343+0000 7f2016019700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2010198240 con 0x7f2010105bf0 2024-04-23T02:30:17.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.343+0000 7f200f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20100ffb40 0x7f2010197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:17.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.344+0000 7f200f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20100ffb40 0x7f2010197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58376/0 (socket says 172.21.15.71:58376) 2024-04-23T02:30:17.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.344+0000 7f200f7fe700 1 -- 172.21.15.71:0/1307790339 learned_addr learned my addr 172.21.15.71:0/1307790339 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:30:17.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.344+0000 7f200effd700 1 --2- 172.21.15.71:0/1307790339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2010105bf0 0x7f2010197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:17.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.344+0000 7f200f7fe700 1 -- 172.21.15.71:0/1307790339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2010105bf0 msgr2=0x7f2010197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:17.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.344+0000 7f200f7fe700 1 --2- 172.21.15.71:0/1307790339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2010105bf0 0x7f2010197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:17.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.344+0000 7f200f7fe700 1 -- 172.21.15.71:0/1307790339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20040095f0 con 0x7f20100ffb40 2024-04-23T02:30:17.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.344+0000 7f200f7fe700 1 --2- 172.21.15.71:0/1307790339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20100ffb40 0x7f2010197570 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f200000d8d0 tx=0x7f200000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:17.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.344+0000 7f200cff9700 1 -- 172.21.15.71:0/1307790339 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20000098a0 con 0x7f20100ffb40 2024-04-23T02:30:17.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.344+0000 7f200cff9700 1 -- 172.21.15.71:0/1307790339 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2000010460 con 0x7f20100ffb40 2024-04-23T02:30:17.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.344+0000 7f200cff9700 1 -- 172.21.15.71:0/1307790339 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2000009ea0 con 0x7f20100ffb40 2024-04-23T02:30:17.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.344+0000 7f2016019700 1 -- 172.21.15.71:0/1307790339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20101a05e0 con 0x7f20100ffb40 2024-04-23T02:30:17.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.344+0000 7f2016019700 1 -- 172.21.15.71:0/1307790339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20101a0ad0 con 0x7f20100ffb40 2024-04-23T02:30:17.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.345+0000 7f2016019700 1 -- 172.21.15.71:0/1307790339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20101080a0 con 0x7f20100ffb40 2024-04-23T02:30:17.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.345+0000 7f200cff9700 1 -- 172.21.15.71:0/1307790339 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f20000105d0 con 0x7f20100ffb40 2024-04-23T02:30:17.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.346+0000 7f200cff9700 1 --2- 172.21.15.71:0/1307790339 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1ff806cce0 0x7f1ff806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:17.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.346+0000 7f200cff9700 1 -- 172.21.15.71:0/1307790339 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f200008c3a0 con 0x7f20100ffb40 2024-04-23T02:30:17.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.346+0000 7f200effd700 1 --2- 172.21.15.71:0/1307790339 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1ff806cce0 0x7f1ff806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:17.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.346+0000 7f200effd700 1 --2- 172.21.15.71:0/1307790339 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1ff806cce0 0x7f1ff806f1a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f20100696e0 tx=0x7f200401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:17.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.350+0000 7f200cff9700 1 -- 172.21.15.71:0/1307790339 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2000056490 con 0x7f20100ffb40 2024-04-23T02:30:17.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.586+0000 7f2016019700 1 -- 172.21.15.71:0/1307790339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f201004f2e0 con 0x7f20100ffb40 2024-04-23T02:30:17.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.587+0000 7f200cff9700 1 -- 172.21.15.71:0/1307790339 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f2000010880 con 0x7f20100ffb40 2024-04-23T02:30:17.589 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:30:17.589 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:30:17.590 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:30:17.590 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:30:17.590 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:30:17.590 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:30:17.590 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:30:17.590 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:30:17.590 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:30:17.590 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:30:17.590 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:30:17.590 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:30:17.590 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:30:17.590 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:30:17.590 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:30:17.590 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:30:17.591 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:30:17.591 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:30:17.591 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:30:17.591 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:30:17.591 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:30:17.591 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:30:17.591 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:30:17.591 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:30:17.591 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:30:17.591 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:30:17.591 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:30:17.591 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:30:17.591 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:30:17.591 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:30:17.592 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:30:17.592 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:30:17.592 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:30:17.592 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:30:17.592 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:30:17.592 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:30:17.592 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:30:17.592 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:30:17.592 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:30:17.592 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:30:17.592 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:30:17.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.589+0000 7f2016019700 1 -- 172.21.15.71:0/1307790339 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1ff806cce0 msgr2=0x7f1ff806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:17.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.589+0000 7f2016019700 1 --2- 172.21.15.71:0/1307790339 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1ff806cce0 0x7f1ff806f1a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f20100696e0 tx=0x7f200401a040 comp rx=0 tx=0).stop 2024-04-23T02:30:17.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.589+0000 7f2016019700 1 -- 172.21.15.71:0/1307790339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20100ffb40 msgr2=0x7f2010197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:17.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.589+0000 7f2016019700 1 --2- 172.21.15.71:0/1307790339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20100ffb40 0x7f2010197570 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f200000d8d0 tx=0x7f200000dbe0 comp rx=0 tx=0).stop 2024-04-23T02:30:17.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.589+0000 7f2016019700 1 -- 172.21.15.71:0/1307790339 shutdown_connections 2024-04-23T02:30:17.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.589+0000 7f2016019700 1 --2- 172.21.15.71:0/1307790339 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1ff806cce0 0x7f1ff806f1a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:17.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.589+0000 7f2016019700 1 --2- 172.21.15.71:0/1307790339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20100ffb40 0x7f2010197570 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:17.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.589+0000 7f2016019700 1 --2- 172.21.15.71:0/1307790339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2010105bf0 0x7f2010197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:17.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.589+0000 7f2016019700 1 -- 172.21.15.71:0/1307790339 >> 172.21.15.71:0/1307790339 conn(0x7f20100fb610 msgr2=0x7f20100fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:17.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.589+0000 7f2016019700 1 -- 172.21.15.71:0/1307790339 shutdown_connections 2024-04-23T02:30:17.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.589+0000 7f2016019700 1 -- 172.21.15.71:0/1307790339 wait complete. 2024-04-23T02:30:17.594 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:30:17.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.692+0000 7f7e74d88700 1 -- 172.21.15.71:0/239917825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e700ff5a0 msgr2=0x7f7e700ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:17.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.692+0000 7f7e74d88700 1 --2- 172.21.15.71:0/239917825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e700ff5a0 0x7f7e700ff980 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009910 tx=0x7f7e64009c20 comp rx=0 tx=0).stop 2024-04-23T02:30:17.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.692+0000 7f7e74d88700 1 -- 172.21.15.71:0/239917825 shutdown_connections 2024-04-23T02:30:17.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.692+0000 7f7e74d88700 1 --2- 172.21.15.71:0/239917825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e700fff50 0x7f7e70105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:17.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.692+0000 7f7e74d88700 1 --2- 172.21.15.71:0/239917825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e700ff5a0 0x7f7e700ff980 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:17.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.692+0000 7f7e74d88700 1 -- 172.21.15.71:0/239917825 >> 172.21.15.71:0/239917825 conn(0x7f7e70074e30 msgr2=0x7f7e70075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:17.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.692+0000 7f7e74d88700 1 -- 172.21.15.71:0/239917825 shutdown_connections 2024-04-23T02:30:17.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.692+0000 7f7e74d88700 1 -- 172.21.15.71:0/239917825 wait complete. 2024-04-23T02:30:17.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.693+0000 7f7e74d88700 1 Processor -- start 2024-04-23T02:30:17.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.693+0000 7f7e74d88700 1 -- start start 2024-04-23T02:30:17.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.693+0000 7f7e74d88700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e700ff5a0 0x7f7e70100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:17.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.693+0000 7f7e74d88700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e700fff50 0x7f7e70100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:17.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.693+0000 7f7e74d88700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e70100fa0 con 0x7f7e700fff50 2024-04-23T02:30:17.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.693+0000 7f7e74d88700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e701010e0 con 0x7f7e700ff5a0 2024-04-23T02:30:17.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.693+0000 7f7e6dd9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e700fff50 0x7f7e70100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:17.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.694+0000 7f7e6dd9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e700fff50 0x7f7e70100a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58386/0 (socket says 172.21.15.71:58386) 2024-04-23T02:30:17.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.694+0000 7f7e6dd9b700 1 -- 172.21.15.71:0/1891894635 learned_addr learned my addr 172.21.15.71:0/1891894635 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:30:17.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.694+0000 7f7e6dd9b700 1 -- 172.21.15.71:0/1891894635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e700ff5a0 msgr2=0x7f7e70100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:17.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.694+0000 7f7e6e59c700 1 --2- 172.21.15.71:0/1891894635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e700ff5a0 0x7f7e70100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:17.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.694+0000 7f7e6dd9b700 1 --2- 172.21.15.71:0/1891894635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e700ff5a0 0x7f7e70100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:17.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.694+0000 7f7e6dd9b700 1 -- 172.21.15.71:0/1891894635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e640095f0 con 0x7f7e700fff50 2024-04-23T02:30:17.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.694+0000 7f7e6dd9b700 1 --2- 172.21.15.71:0/1891894635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e700fff50 0x7f7e70100a60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000b480 tx=0x7f7e6000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:17.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.694+0000 7f7e5f7fe700 1 -- 172.21.15.71:0/1891894635 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e600118c0 con 0x7f7e700fff50 2024-04-23T02:30:17.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.694+0000 7f7e74d88700 1 -- 172.21.15.71:0/1891894635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e701a2a60 con 0x7f7e700fff50 2024-04-23T02:30:17.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.694+0000 7f7e5f7fe700 1 -- 172.21.15.71:0/1891894635 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7e60010380 con 0x7f7e700fff50 2024-04-23T02:30:17.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.694+0000 7f7e5f7fe700 1 -- 172.21.15.71:0/1891894635 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e6000f510 con 0x7f7e700fff50 2024-04-23T02:30:17.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.694+0000 7f7e74d88700 1 -- 172.21.15.71:0/1891894635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e701a2e70 con 0x7f7e700fff50 2024-04-23T02:30:17.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.695+0000 7f7e74d88700 1 -- 172.21.15.71:0/1891894635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e70109270 con 0x7f7e700fff50 2024-04-23T02:30:17.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.696+0000 7f7e5f7fe700 1 -- 172.21.15.71:0/1891894635 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7e600104f0 con 0x7f7e700fff50 2024-04-23T02:30:17.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.696+0000 7f7e5f7fe700 1 --2- 172.21.15.71:0/1891894635 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7e5806cce0 0x7f7e5806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:17.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.696+0000 7f7e5f7fe700 1 -- 172.21.15.71:0/1891894635 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7e6008c210 con 0x7f7e700fff50 2024-04-23T02:30:17.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.696+0000 7f7e6e59c700 1 --2- 172.21.15.71:0/1891894635 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7e5806cce0 0x7f7e5806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:17.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.696+0000 7f7e6e59c700 1 --2- 172.21.15.71:0/1891894635 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7e5806cce0 0x7f7e5806f1a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400b5c0 tx=0x7f7e64005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:17.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.700+0000 7f7e5f7fe700 1 -- 172.21.15.71:0/1891894635 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7e60056280 con 0x7f7e700fff50 2024-04-23T02:30:17.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.913+0000 7f7e74d88700 1 -- 172.21.15.71:0/1891894635 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e7002cce0 con 0x7f7e5806cce0 2024-04-23T02:30:17.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.915+0000 7f7e5f7fe700 1 -- 172.21.15.71:0/1891894635 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7e7002cce0 con 0x7f7e5806cce0 2024-04-23T02:30:17.917 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:30:17.917 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:30:17.917 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:30:17.917 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:30:17.917 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:30:17.918 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:30:17.918 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:30:17.918 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:30:17.918 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:30:17.918 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:30:17.918 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:30:17.918 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:30:17.918 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:30:17.918 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:30:17.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.917+0000 7f7e74d88700 1 -- 172.21.15.71:0/1891894635 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7e5806cce0 msgr2=0x7f7e5806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:17.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.917+0000 7f7e74d88700 1 --2- 172.21.15.71:0/1891894635 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7e5806cce0 0x7f7e5806f1a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400b5c0 tx=0x7f7e64005fd0 comp rx=0 tx=0).stop 2024-04-23T02:30:17.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.917+0000 7f7e74d88700 1 -- 172.21.15.71:0/1891894635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e700fff50 msgr2=0x7f7e70100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:17.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.917+0000 7f7e74d88700 1 --2- 172.21.15.71:0/1891894635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e700fff50 0x7f7e70100a60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000b480 tx=0x7f7e6000b840 comp rx=0 tx=0).stop 2024-04-23T02:30:17.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.917+0000 7f7e74d88700 1 -- 172.21.15.71:0/1891894635 shutdown_connections 2024-04-23T02:30:17.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.917+0000 7f7e74d88700 1 --2- 172.21.15.71:0/1891894635 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7e5806cce0 0x7f7e5806f1a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:17.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.917+0000 7f7e74d88700 1 --2- 172.21.15.71:0/1891894635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e700ff5a0 0x7f7e70100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:17.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.917+0000 7f7e74d88700 1 --2- 172.21.15.71:0/1891894635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e700fff50 0x7f7e70100a60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:17.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.917+0000 7f7e74d88700 1 -- 172.21.15.71:0/1891894635 >> 172.21.15.71:0/1891894635 conn(0x7f7e70074e30 msgr2=0x7f7e700fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:17.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.917+0000 7f7e74d88700 1 -- 172.21.15.71:0/1891894635 shutdown_connections 2024-04-23T02:30:17.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:17.917+0000 7f7e74d88700 1 -- 172.21.15.71:0/1891894635 wait complete. 2024-04-23T02:30:18.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.019+0000 7f0c035d0700 1 -- 172.21.15.71:0/4029531144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc101850 msgr2=0x7f0bfc1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:18.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.019+0000 7f0c035d0700 1 --2- 172.21.15.71:0/4029531144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc101850 0x7f0bfc1058a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8009910 tx=0x7f0bf8009c20 comp rx=0 tx=0).stop 2024-04-23T02:30:18.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.019+0000 7f0c035d0700 1 -- 172.21.15.71:0/4029531144 shutdown_connections 2024-04-23T02:30:18.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.019+0000 7f0c035d0700 1 --2- 172.21.15.71:0/4029531144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc101850 0x7f0bfc1058a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:18.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.019+0000 7f0c035d0700 1 --2- 172.21.15.71:0/4029531144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bfc100ea0 0x7f0bfc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:18.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.019+0000 7f0c035d0700 1 -- 172.21.15.71:0/4029531144 >> 172.21.15.71:0/4029531144 conn(0x7f0bfc0fc750 msgr2=0x7f0bfc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:18.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.019+0000 7f0c035d0700 1 -- 172.21.15.71:0/4029531144 shutdown_connections 2024-04-23T02:30:18.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.019+0000 7f0c035d0700 1 -- 172.21.15.71:0/4029531144 wait complete. 2024-04-23T02:30:18.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.020+0000 7f0c035d0700 1 Processor -- start 2024-04-23T02:30:18.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.020+0000 7f0c035d0700 1 -- start start 2024-04-23T02:30:18.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.020+0000 7f0c035d0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bfc100ea0 0x7f0bfc19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:18.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.020+0000 7f0c035d0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc101850 0x7f0bfc19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:18.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.020+0000 7f0c035d0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc19d210 con 0x7f0bfc101850 2024-04-23T02:30:18.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.020+0000 7f0c035d0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc1966c0 con 0x7f0bfc100ea0 2024-04-23T02:30:18.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.020+0000 7f0c00b6b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc101850 0x7f0bfc19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:18.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.020+0000 7f0c00b6b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc101850 0x7f0bfc19cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58410/0 (socket says 172.21.15.71:58410) 2024-04-23T02:30:18.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.020+0000 7f0c00b6b700 1 -- 172.21.15.71:0/281110005 learned_addr learned my addr 172.21.15.71:0/281110005 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:30:18.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.021+0000 7f0c0136c700 1 --2- 172.21.15.71:0/281110005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bfc100ea0 0x7f0bfc19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:18.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.021+0000 7f0c00b6b700 1 -- 172.21.15.71:0/281110005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bfc100ea0 msgr2=0x7f0bfc19c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:18.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.021+0000 7f0c00b6b700 1 --2- 172.21.15.71:0/281110005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bfc100ea0 0x7f0bfc19c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:18.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.021+0000 7f0c00b6b700 1 -- 172.21.15.71:0/281110005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf80095f0 con 0x7f0bfc101850 2024-04-23T02:30:18.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.021+0000 7f0c00b6b700 1 --2- 172.21.15.71:0/281110005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc101850 0x7f0bfc19cb80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80092e0 tx=0x7f0bf800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:18.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.021+0000 7f0bf27fc700 1 -- 172.21.15.71:0/281110005 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bf801d070 con 0x7f0bfc101850 2024-04-23T02:30:18.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.021+0000 7f0bf27fc700 1 -- 172.21.15.71:0/281110005 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0bf8005940 con 0x7f0bfc101850 2024-04-23T02:30:18.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.021+0000 7f0c035d0700 1 -- 172.21.15.71:0/281110005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bfc196940 con 0x7f0bfc101850 2024-04-23T02:30:18.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.021+0000 7f0bf27fc700 1 -- 172.21.15.71:0/281110005 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bf800fe50 con 0x7f0bfc101850 2024-04-23T02:30:18.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.021+0000 7f0c035d0700 1 -- 172.21.15.71:0/281110005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bfc196e30 con 0x7f0bfc101850 2024-04-23T02:30:18.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.022+0000 7f0c035d0700 1 -- 172.21.15.71:0/281110005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bfc1092a0 con 0x7f0bfc101850 2024-04-23T02:30:18.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.022+0000 7f0bf27fc700 1 -- 172.21.15.71:0/281110005 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0bf8003a60 con 0x7f0bfc101850 2024-04-23T02:30:18.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.023+0000 7f0bf27fc700 1 --2- 172.21.15.71:0/281110005 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0be806cce0 0x7f0be806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:18.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.023+0000 7f0bf27fc700 1 -- 172.21.15.71:0/281110005 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0bf808d740 con 0x7f0bfc101850 2024-04-23T02:30:18.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.023+0000 7f0c0136c700 1 --2- 172.21.15.71:0/281110005 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0be806cce0 0x7f0be806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:18.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.023+0000 7f0c0136c700 1 --2- 172.21.15.71:0/281110005 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0be806cce0 0x7f0be806f1a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0bec005fd0 tx=0x7f0bec005f60 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:18.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.027+0000 7f0bf27fc700 1 -- 172.21.15.71:0/281110005 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0bf805b240 con 0x7f0bfc101850 2024-04-23T02:30:18.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.300+0000 7f0c035d0700 1 -- 172.21.15.71:0/281110005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0bfc04f2e0 con 0x7f0bfc101850 2024-04-23T02:30:18.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.300+0000 7f0bf27fc700 1 -- 172.21.15.71:0/281110005 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f0bf805add0 con 0x7f0bfc101850 2024-04-23T02:30:18.303 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:30:18.303 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:30:18.303 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:30:18.303 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:30:18.303 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:30:18.303 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:30:18.303 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:30:18.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.302+0000 7f0c035d0700 1 -- 172.21.15.71:0/281110005 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0be806cce0 msgr2=0x7f0be806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:18.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.302+0000 7f0c035d0700 1 --2- 172.21.15.71:0/281110005 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0be806cce0 0x7f0be806f1a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0bec005fd0 tx=0x7f0bec005f60 comp rx=0 tx=0).stop 2024-04-23T02:30:18.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.302+0000 7f0c035d0700 1 -- 172.21.15.71:0/281110005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc101850 msgr2=0x7f0bfc19cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:18.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.302+0000 7f0c035d0700 1 --2- 172.21.15.71:0/281110005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc101850 0x7f0bfc19cb80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80092e0 tx=0x7f0bf800be20 comp rx=0 tx=0).stop 2024-04-23T02:30:18.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.302+0000 7f0c035d0700 1 -- 172.21.15.71:0/281110005 shutdown_connections 2024-04-23T02:30:18.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.302+0000 7f0c035d0700 1 --2- 172.21.15.71:0/281110005 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0be806cce0 0x7f0be806f1a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:18.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.302+0000 7f0c035d0700 1 --2- 172.21.15.71:0/281110005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bfc100ea0 0x7f0bfc19c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:18.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.302+0000 7f0c035d0700 1 --2- 172.21.15.71:0/281110005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc101850 0x7f0bfc19cb80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:18.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.302+0000 7f0c035d0700 1 -- 172.21.15.71:0/281110005 >> 172.21.15.71:0/281110005 conn(0x7f0bfc0fc750 msgr2=0x7f0bfc0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:18.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.302+0000 7f0c035d0700 1 -- 172.21.15.71:0/281110005 shutdown_connections 2024-04-23T02:30:18.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:18.302+0000 7f0c035d0700 1 -- 172.21.15.71:0/281110005 wait complete. 2024-04-23T02:30:18.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:18 smithi192 ceph-mon[108617]: from='client.34766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:30:18.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:18 smithi192 ceph-mon[108617]: from='client.34770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:30:18.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:18 smithi192 ceph-mon[108617]: pgmap v509: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:18.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:18 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2437969671' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:30:18.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:18 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1307790339' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:30:18.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:18 smithi071 ceph-mon[144544]: from='client.34766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:30:18.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:18 smithi071 ceph-mon[144544]: from='client.34770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:30:18.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:18 smithi071 ceph-mon[144544]: pgmap v509: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:18.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:18 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2437969671' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:30:18.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:18 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1307790339' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:30:19.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:19 smithi192 ceph-mon[108617]: from='client.34782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:30:19.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:19 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/281110005' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:30:19.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:19 smithi071 ceph-mon[144544]: from='client.34782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:30:19.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:19 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/281110005' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:30:20.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:20 smithi192 ceph-mon[108617]: pgmap v510: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:20.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:30:20.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:20 smithi071 ceph-mon[144544]: pgmap v510: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:20.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:30:22.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:22 smithi071 ceph-mon[144544]: pgmap v511: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:22.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:22 smithi192 ceph-mon[108617]: pgmap v511: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:24.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:24 smithi192 ceph-mon[108617]: pgmap v512: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:24.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:24 smithi071 ceph-mon[144544]: pgmap v512: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:26.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:26 smithi192 ceph-mon[108617]: pgmap v513: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:26.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:26 smithi071 ceph-mon[144544]: pgmap v513: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:28.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:28 smithi192 ceph-mon[108617]: pgmap v514: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:28.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:28 smithi071 ceph-mon[144544]: pgmap v514: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:30.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:30 smithi192 ceph-mon[108617]: pgmap v515: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:30.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:30 smithi071 ceph-mon[144544]: pgmap v515: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:32.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:32 smithi192 ceph-mon[108617]: pgmap v516: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:32.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:32 smithi071 ceph-mon[144544]: pgmap v516: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:34.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:34 smithi192 ceph-mon[108617]: pgmap v517: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:34.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:34 smithi071 ceph-mon[144544]: pgmap v517: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:35.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:30:35.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:30:36.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:36 smithi192 ceph-mon[108617]: pgmap v518: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:36.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:36 smithi071 ceph-mon[144544]: pgmap v518: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:38.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:38 smithi192 ceph-mon[108617]: pgmap v519: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:38.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:38 smithi071 ceph-mon[144544]: pgmap v519: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:40.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:40 smithi192 ceph-mon[108617]: pgmap v520: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:40.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:40 smithi071 ceph-mon[144544]: pgmap v520: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:42.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:42 smithi192 ceph-mon[108617]: pgmap v521: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:42.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:42 smithi071 ceph-mon[144544]: pgmap v521: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:44.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:44 smithi071 ceph-mon[144544]: pgmap v522: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:44.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:44 smithi192 ceph-mon[108617]: pgmap v522: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:46.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:46 smithi192 ceph-mon[108617]: pgmap v523: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:46.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:46 smithi071 ceph-mon[144544]: pgmap v523: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:48.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:48 smithi192 ceph-mon[108617]: pgmap v524: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:48.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.419+0000 7f9a51282700 1 -- 172.21.15.71:0/1338497275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a4c102720 msgr2=0x7f9a4c10ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:48.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.419+0000 7f9a51282700 1 --2- 172.21.15.71:0/1338497275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a4c102720 0x7f9a4c10ac10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009910 tx=0x7f9a40009c20 comp rx=0 tx=0).stop 2024-04-23T02:30:48.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.419+0000 7f9a51282700 1 -- 172.21.15.71:0/1338497275 shutdown_connections 2024-04-23T02:30:48.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.419+0000 7f9a51282700 1 --2- 172.21.15.71:0/1338497275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a4c102720 0x7f9a4c10ac10 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:48.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.419+0000 7f9a51282700 1 --2- 172.21.15.71:0/1338497275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a4c101e00 0x7f9a4c1021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:48.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.419+0000 7f9a51282700 1 -- 172.21.15.71:0/1338497275 >> 172.21.15.71:0/1338497275 conn(0x7f9a4c0fb650 msgr2=0x7f9a4c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:48.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.419+0000 7f9a51282700 1 -- 172.21.15.71:0/1338497275 shutdown_connections 2024-04-23T02:30:48.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.419+0000 7f9a51282700 1 -- 172.21.15.71:0/1338497275 wait complete. 2024-04-23T02:30:48.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.419+0000 7f9a51282700 1 Processor -- start 2024-04-23T02:30:48.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.420+0000 7f9a51282700 1 -- start start 2024-04-23T02:30:48.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.420+0000 7f9a51282700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a4c101e00 0x7f9a4c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:48.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.420+0000 7f9a51282700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a4c102720 0x7f9a4c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:48.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.420+0000 7f9a51282700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c19d220 con 0x7f9a4c101e00 2024-04-23T02:30:48.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.420+0000 7f9a51282700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c1966d0 con 0x7f9a4c102720 2024-04-23T02:30:48.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.420+0000 7f9a4affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a4c101e00 0x7f9a4c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:48.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.420+0000 7f9a4affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a4c101e00 0x7f9a4c19c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48862/0 (socket says 172.21.15.71:48862) 2024-04-23T02:30:48.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.420+0000 7f9a4affd700 1 -- 172.21.15.71:0/2576802274 learned_addr learned my addr 172.21.15.71:0/2576802274 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:30:48.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.420+0000 7f9a4a7fc700 1 --2- 172.21.15.71:0/2576802274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a4c102720 0x7f9a4c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:48.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.420+0000 7f9a4affd700 1 -- 172.21.15.71:0/2576802274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a4c102720 msgr2=0x7f9a4c19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:48.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.420+0000 7f9a4affd700 1 --2- 172.21.15.71:0/2576802274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a4c102720 0x7f9a4c19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:48.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.420+0000 7f9a4affd700 1 -- 172.21.15.71:0/2576802274 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a400095f0 con 0x7f9a4c101e00 2024-04-23T02:30:48.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.420+0000 7f9a4affd700 1 --2- 172.21.15.71:0/2576802274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a4c101e00 0x7f9a4c19c650 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00d8d0 tx=0x7f9a3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:48.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.420+0000 7f9a33fff700 1 -- 172.21.15.71:0/2576802274 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a3c009940 con 0x7f9a4c101e00 2024-04-23T02:30:48.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.420+0000 7f9a33fff700 1 -- 172.21.15.71:0/2576802274 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9a3c010460 con 0x7f9a4c101e00 2024-04-23T02:30:48.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.420+0000 7f9a51282700 1 -- 172.21.15.71:0/2576802274 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4c1969b0 con 0x7f9a4c101e00 2024-04-23T02:30:48.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.421+0000 7f9a33fff700 1 -- 172.21.15.71:0/2576802274 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a3c00f5f0 con 0x7f9a4c101e00 2024-04-23T02:30:48.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.421+0000 7f9a51282700 1 -- 172.21.15.71:0/2576802274 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4c196ed0 con 0x7f9a4c101e00 2024-04-23T02:30:48.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.421+0000 7f9a51282700 1 -- 172.21.15.71:0/2576802274 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4c108310 con 0x7f9a4c101e00 2024-04-23T02:30:48.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.422+0000 7f9a33fff700 1 -- 172.21.15.71:0/2576802274 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9a3c0105d0 con 0x7f9a4c101e00 2024-04-23T02:30:48.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.422+0000 7f9a33fff700 1 --2- 172.21.15.71:0/2576802274 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9a3406cce0 0x7f9a3406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:48.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.423+0000 7f9a4a7fc700 1 --2- 172.21.15.71:0/2576802274 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9a3406cce0 0x7f9a3406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:48.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.423+0000 7f9a33fff700 1 -- 172.21.15.71:0/2576802274 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9a3c08c5e0 con 0x7f9a4c101e00 2024-04-23T02:30:48.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.423+0000 7f9a4a7fc700 1 --2- 172.21.15.71:0/2576802274 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9a3406cce0 0x7f9a3406f1a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9a40006010 tx=0x7f9a4001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:48.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.427+0000 7f9a33fff700 1 -- 172.21.15.71:0/2576802274 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9a3c05a0e0 con 0x7f9a4c101e00 2024-04-23T02:30:48.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:48 smithi071 ceph-mon[144544]: pgmap v524: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:48.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.640+0000 7f9a51282700 1 -- 172.21.15.71:0/2576802274 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a4c197cf0 con 0x7f9a3406cce0 2024-04-23T02:30:48.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.641+0000 7f9a33fff700 1 -- 172.21.15.71:0/2576802274 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9a4c197cf0 con 0x7f9a3406cce0 2024-04-23T02:30:48.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.643+0000 7f9a51282700 1 -- 172.21.15.71:0/2576802274 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9a3406cce0 msgr2=0x7f9a3406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:48.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.643+0000 7f9a51282700 1 --2- 172.21.15.71:0/2576802274 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9a3406cce0 0x7f9a3406f1a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9a40006010 tx=0x7f9a4001a040 comp rx=0 tx=0).stop 2024-04-23T02:30:48.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.643+0000 7f9a51282700 1 -- 172.21.15.71:0/2576802274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a4c101e00 msgr2=0x7f9a4c19c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:48.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.643+0000 7f9a51282700 1 --2- 172.21.15.71:0/2576802274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a4c101e00 0x7f9a4c19c650 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00d8d0 tx=0x7f9a3c00dc90 comp rx=0 tx=0).stop 2024-04-23T02:30:48.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.643+0000 7f9a51282700 1 -- 172.21.15.71:0/2576802274 shutdown_connections 2024-04-23T02:30:48.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.643+0000 7f9a51282700 1 --2- 172.21.15.71:0/2576802274 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9a3406cce0 0x7f9a3406f1a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:48.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.643+0000 7f9a51282700 1 --2- 172.21.15.71:0/2576802274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a4c101e00 0x7f9a4c19c650 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:48.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.643+0000 7f9a51282700 1 --2- 172.21.15.71:0/2576802274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a4c102720 0x7f9a4c19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:48.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.643+0000 7f9a51282700 1 -- 172.21.15.71:0/2576802274 >> 172.21.15.71:0/2576802274 conn(0x7f9a4c0fb650 msgr2=0x7f9a4c105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:48.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.643+0000 7f9a51282700 1 -- 172.21.15.71:0/2576802274 shutdown_connections 2024-04-23T02:30:48.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.643+0000 7f9a51282700 1 -- 172.21.15.71:0/2576802274 wait complete. 2024-04-23T02:30:48.656 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:30:48.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.747+0000 7f8ab6d65700 1 -- 172.21.15.71:0/3076590497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ab0105a20 msgr2=0x7f8ab0105e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:48.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.747+0000 7f8ab6d65700 1 --2- 172.21.15.71:0/3076590497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ab0105a20 0x7f8ab0105e00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009910 tx=0x7f8aa4009c20 comp rx=0 tx=0).stop 2024-04-23T02:30:48.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.747+0000 7f8ab6d65700 1 -- 172.21.15.71:0/3076590497 shutdown_connections 2024-04-23T02:30:48.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.747+0000 7f8ab6d65700 1 --2- 172.21.15.71:0/3076590497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab00ffb60 0x7f8ab00fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:48.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.747+0000 7f8ab6d65700 1 --2- 172.21.15.71:0/3076590497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ab0105a20 0x7f8ab0105e00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:48.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.747+0000 7f8ab6d65700 1 -- 172.21.15.71:0/3076590497 >> 172.21.15.71:0/3076590497 conn(0x7f8ab00754a0 msgr2=0x7f8ab00758b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:48.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.747+0000 7f8ab6d65700 1 -- 172.21.15.71:0/3076590497 shutdown_connections 2024-04-23T02:30:48.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.747+0000 7f8ab6d65700 1 -- 172.21.15.71:0/3076590497 wait complete. 2024-04-23T02:30:48.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.747+0000 7f8ab6d65700 1 Processor -- start 2024-04-23T02:30:48.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.748+0000 7f8ab6d65700 1 -- start start 2024-04-23T02:30:48.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.748+0000 7f8ab6d65700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab00ffb60 0x7f8ab01965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:48.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.748+0000 7f8ab6d65700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ab0105a20 0x7f8ab0196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:48.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.748+0000 7f8ab6d65700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab0197150 con 0x7f8ab0105a20 2024-04-23T02:30:48.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.748+0000 7f8ab6d65700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab0197290 con 0x7f8ab00ffb60 2024-04-23T02:30:48.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.748+0000 7f8aaffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ab0105a20 0x7f8ab0196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:48.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.748+0000 7f8aaffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ab0105a20 0x7f8ab0196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48882/0 (socket says 172.21.15.71:48882) 2024-04-23T02:30:48.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.748+0000 7f8aaffff700 1 -- 172.21.15.71:0/3036074966 learned_addr learned my addr 172.21.15.71:0/3036074966 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:30:48.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.748+0000 7f8ab4b01700 1 --2- 172.21.15.71:0/3036074966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab00ffb60 0x7f8ab01965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:48.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.748+0000 7f8aaffff700 1 -- 172.21.15.71:0/3036074966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab00ffb60 msgr2=0x7f8ab01965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:48.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.748+0000 7f8aaffff700 1 --2- 172.21.15.71:0/3036074966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab00ffb60 0x7f8ab01965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:48.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.748+0000 7f8aaffff700 1 -- 172.21.15.71:0/3036074966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa40095f0 con 0x7f8ab0105a20 2024-04-23T02:30:48.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.748+0000 7f8ab4b01700 1 --2- 172.21.15.71:0/3036074966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab00ffb60 0x7f8ab01965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:30:48.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.748+0000 7f8aaffff700 1 --2- 172.21.15.71:0/3036074966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ab0105a20 0x7f8ab0196b00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00c930 tx=0x7f8a9c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:48.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.748+0000 7f8aadffb700 1 -- 172.21.15.71:0/3036074966 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a9c007ad0 con 0x7f8ab0105a20 2024-04-23T02:30:48.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.748+0000 7f8ab6d65700 1 -- 172.21.15.71:0/3036074966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab019b0e0 con 0x7f8ab0105a20 2024-04-23T02:30:48.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.748+0000 7f8aadffb700 1 -- 172.21.15.71:0/3036074966 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8a9c007c30 con 0x7f8ab0105a20 2024-04-23T02:30:48.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.748+0000 7f8aadffb700 1 -- 172.21.15.71:0/3036074966 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a9c018630 con 0x7f8ab0105a20 2024-04-23T02:30:48.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.749+0000 7f8ab6d65700 1 -- 172.21.15.71:0/3036074966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab019b630 con 0x7f8ab0105a20 2024-04-23T02:30:48.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.749+0000 7f8ab6d65700 1 -- 172.21.15.71:0/3036074966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab004f2e0 con 0x7f8ab0105a20 2024-04-23T02:30:48.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.750+0000 7f8aadffb700 1 -- 172.21.15.71:0/3036074966 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8a9c00f450 con 0x7f8ab0105a20 2024-04-23T02:30:48.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.750+0000 7f8aadffb700 1 --2- 172.21.15.71:0/3036074966 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8aa006cce0 0x7f8aa006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:48.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.750+0000 7f8aadffb700 1 -- 172.21.15.71:0/3036074966 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8a9c08c870 con 0x7f8ab0105a20 2024-04-23T02:30:48.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.750+0000 7f8ab4b01700 1 --2- 172.21.15.71:0/3036074966 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8aa006cce0 0x7f8aa006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:48.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.751+0000 7f8ab4b01700 1 --2- 172.21.15.71:0/3036074966 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8aa006cce0 0x7f8aa006f1a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400b5c0 tx=0x7f8aa4005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:48.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.755+0000 7f8aadffb700 1 -- 172.21.15.71:0/3036074966 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8a9c0568f0 con 0x7f8ab0105a20 2024-04-23T02:30:48.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.967+0000 7f8ab6d65700 1 -- 172.21.15.71:0/3036074966 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ab019bab0 con 0x7f8aa006cce0 2024-04-23T02:30:48.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.968+0000 7f8aadffb700 1 -- 172.21.15.71:0/3036074966 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8ab019bab0 con 0x7f8aa006cce0 2024-04-23T02:30:48.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.970+0000 7f8ab6d65700 1 -- 172.21.15.71:0/3036074966 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8aa006cce0 msgr2=0x7f8aa006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:48.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.970+0000 7f8ab6d65700 1 --2- 172.21.15.71:0/3036074966 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8aa006cce0 0x7f8aa006f1a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400b5c0 tx=0x7f8aa4005fd0 comp rx=0 tx=0).stop 2024-04-23T02:30:48.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.970+0000 7f8ab6d65700 1 -- 172.21.15.71:0/3036074966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ab0105a20 msgr2=0x7f8ab0196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:48.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.970+0000 7f8ab6d65700 1 --2- 172.21.15.71:0/3036074966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ab0105a20 0x7f8ab0196b00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00c930 tx=0x7f8a9c00ccf0 comp rx=0 tx=0).stop 2024-04-23T02:30:48.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.970+0000 7f8ab6d65700 1 -- 172.21.15.71:0/3036074966 shutdown_connections 2024-04-23T02:30:48.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.970+0000 7f8ab6d65700 1 --2- 172.21.15.71:0/3036074966 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8aa006cce0 0x7f8aa006f1a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:48.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.970+0000 7f8ab6d65700 1 --2- 172.21.15.71:0/3036074966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab00ffb60 0x7f8ab01965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:48.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.970+0000 7f8ab6d65700 1 --2- 172.21.15.71:0/3036074966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ab0105a20 0x7f8ab0196b00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:48.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.970+0000 7f8ab6d65700 1 -- 172.21.15.71:0/3036074966 >> 172.21.15.71:0/3036074966 conn(0x7f8ab00754a0 msgr2=0x7f8ab00686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:48.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.970+0000 7f8ab6d65700 1 -- 172.21.15.71:0/3036074966 shutdown_connections 2024-04-23T02:30:48.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:48.970+0000 7f8ab6d65700 1 -- 172.21.15.71:0/3036074966 wait complete. 2024-04-23T02:30:49.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.073+0000 7fb3379a9700 1 -- 172.21.15.71:0/2513971557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb330100e90 msgr2=0x7fb330101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:49.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.073+0000 7fb3379a9700 1 --2- 172.21.15.71:0/2513971557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb330100e90 0x7fb330101270 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009910 tx=0x7fb32c009c20 comp rx=0 tx=0).stop 2024-04-23T02:30:49.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.073+0000 7fb3379a9700 1 -- 172.21.15.71:0/2513971557 shutdown_connections 2024-04-23T02:30:49.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.073+0000 7fb3379a9700 1 --2- 172.21.15.71:0/2513971557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb330101840 0x7fb330105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:49.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.073+0000 7fb3379a9700 1 --2- 172.21.15.71:0/2513971557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb330100e90 0x7fb330101270 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:49.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.073+0000 7fb3379a9700 1 -- 172.21.15.71:0/2513971557 >> 172.21.15.71:0/2513971557 conn(0x7fb3300fc740 msgr2=0x7fb3300feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:49.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.073+0000 7fb3379a9700 1 -- 172.21.15.71:0/2513971557 shutdown_connections 2024-04-23T02:30:49.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.073+0000 7fb3379a9700 1 -- 172.21.15.71:0/2513971557 wait complete. 2024-04-23T02:30:49.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.073+0000 7fb3379a9700 1 Processor -- start 2024-04-23T02:30:49.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.073+0000 7fb3379a9700 1 -- start start 2024-04-23T02:30:49.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.074+0000 7fb3379a9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb330100e90 0x7fb33019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:49.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.074+0000 7fb3379a9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb330101840 0x7fb33019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:49.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.074+0000 7fb3379a9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33019d190 con 0x7fb330100e90 2024-04-23T02:30:49.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.074+0000 7fb3379a9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb330196640 con 0x7fb330101840 2024-04-23T02:30:49.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.074+0000 7fb335745700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb330100e90 0x7fb33019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:49.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.074+0000 7fb335745700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb330100e90 0x7fb33019c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48908/0 (socket says 172.21.15.71:48908) 2024-04-23T02:30:49.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.074+0000 7fb335745700 1 -- 172.21.15.71:0/3492429131 learned_addr learned my addr 172.21.15.71:0/3492429131 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:30:49.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.074+0000 7fb334f44700 1 --2- 172.21.15.71:0/3492429131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb330101840 0x7fb33019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:49.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.074+0000 7fb335745700 1 -- 172.21.15.71:0/3492429131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb330101840 msgr2=0x7fb33019cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:49.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.074+0000 7fb335745700 1 --2- 172.21.15.71:0/3492429131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb330101840 0x7fb33019cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:49.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.074+0000 7fb335745700 1 -- 172.21.15.71:0/3492429131 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c0095f0 con 0x7fb330100e90 2024-04-23T02:30:49.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.074+0000 7fb335745700 1 --2- 172.21.15.71:0/3492429131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb330100e90 0x7fb33019c5c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00b5c0 tx=0x7fb32c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:49.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.074+0000 7fb3267fc700 1 -- 172.21.15.71:0/3492429131 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb32c01d070 con 0x7fb330100e90 2024-04-23T02:30:49.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.074+0000 7fb3267fc700 1 -- 172.21.15.71:0/3492429131 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb32c005e00 con 0x7fb330100e90 2024-04-23T02:30:49.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.074+0000 7fb3379a9700 1 -- 172.21.15.71:0/3492429131 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3301968c0 con 0x7fb330100e90 2024-04-23T02:30:49.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.074+0000 7fb3267fc700 1 -- 172.21.15.71:0/3492429131 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb32c00fe90 con 0x7fb330100e90 2024-04-23T02:30:49.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.074+0000 7fb3379a9700 1 -- 172.21.15.71:0/3492429131 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb330196db0 con 0x7fb330100e90 2024-04-23T02:30:49.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.076+0000 7fb3379a9700 1 -- 172.21.15.71:0/3492429131 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb330109230 con 0x7fb330100e90 2024-04-23T02:30:49.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.076+0000 7fb3267fc700 1 -- 172.21.15.71:0/3492429131 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb32c00f6d0 con 0x7fb330100e90 2024-04-23T02:30:49.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.076+0000 7fb3267fc700 1 --2- 172.21.15.71:0/3492429131 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb31c06cce0 0x7fb31c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:49.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.076+0000 7fb3267fc700 1 -- 172.21.15.71:0/3492429131 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb32c08d7e0 con 0x7fb330100e90 2024-04-23T02:30:49.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.076+0000 7fb334f44700 1 --2- 172.21.15.71:0/3492429131 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb31c06cce0 0x7fb31c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:49.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.077+0000 7fb334f44700 1 --2- 172.21.15.71:0/3492429131 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb31c06cce0 0x7fb31c06f1a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb3300fd000 tx=0x7fb32000b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:49.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.081+0000 7fb3267fc700 1 -- 172.21.15.71:0/3492429131 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb32c05b2e0 con 0x7fb330100e90 2024-04-23T02:30:49.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.280+0000 7fb3379a9700 1 -- 172.21.15.71:0/3492429131 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb330196fd0 con 0x7fb31c06cce0 2024-04-23T02:30:49.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.290+0000 7fb3267fc700 1 -- 172.21.15.71:0/3492429131 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb330196fd0 con 0x7fb31c06cce0 2024-04-23T02:30:49.293 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:30:49.293 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (14m) 8m ago 25m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:30:49.293 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (26m) 8m ago 26m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:30:49.293 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (24m) 7m ago 24m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:30:49.293 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (12m) 8m ago 26m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:30:49.293 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (12m) 7m ago 24m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:30:49.293 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (14m) 8m ago 25m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:30:49.293 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (8m) 8m ago 19m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:30:49.293 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (8m) 8m ago 19m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:30:49.294 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (19m) 7m ago 19m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:30:49.294 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (8m) 7m ago 19m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:30:49.294 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (16m) 8m ago 27m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:30:49.294 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (15m) 7m ago 24m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:30:49.294 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (12m) 8m ago 27m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:30:49.294 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (12m) 7m ago 24m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:30:49.294 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (15m) 8m ago 25m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:30:49.294 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (15m) 7m ago 24m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:30:49.294 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (11m) 8m ago 23m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:30:49.294 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (11m) 8m ago 23m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:30:49.294 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (10m) 8m ago 22m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:30:49.294 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (10m) 7m ago 22m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:30:49.294 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (9m) 7m ago 21m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:30:49.294 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (8m) 7m ago 21m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:30:49.295 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (14m) 8m ago 25m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:30:49.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.292+0000 7fb3379a9700 1 -- 172.21.15.71:0/3492429131 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb31c06cce0 msgr2=0x7fb31c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:49.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.292+0000 7fb3379a9700 1 --2- 172.21.15.71:0/3492429131 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb31c06cce0 0x7fb31c06f1a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb3300fd000 tx=0x7fb32000b410 comp rx=0 tx=0).stop 2024-04-23T02:30:49.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.292+0000 7fb3379a9700 1 -- 172.21.15.71:0/3492429131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb330100e90 msgr2=0x7fb33019c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:49.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.292+0000 7fb3379a9700 1 --2- 172.21.15.71:0/3492429131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb330100e90 0x7fb33019c5c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00b5c0 tx=0x7fb32c00bd70 comp rx=0 tx=0).stop 2024-04-23T02:30:49.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.292+0000 7fb3379a9700 1 -- 172.21.15.71:0/3492429131 shutdown_connections 2024-04-23T02:30:49.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.292+0000 7fb3379a9700 1 --2- 172.21.15.71:0/3492429131 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb31c06cce0 0x7fb31c06f1a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:49.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.292+0000 7fb3379a9700 1 --2- 172.21.15.71:0/3492429131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb330100e90 0x7fb33019c5c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:49.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.292+0000 7fb3379a9700 1 --2- 172.21.15.71:0/3492429131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb330101840 0x7fb33019cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:49.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.292+0000 7fb3379a9700 1 -- 172.21.15.71:0/3492429131 >> 172.21.15.71:0/3492429131 conn(0x7fb3300fc740 msgr2=0x7fb3300fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:49.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.293+0000 7fb3379a9700 1 -- 172.21.15.71:0/3492429131 shutdown_connections 2024-04-23T02:30:49.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.293+0000 7fb3379a9700 1 -- 172.21.15.71:0/3492429131 wait complete. 2024-04-23T02:30:49.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.394+0000 7fc7c0356700 1 -- 172.21.15.71:0/3250336638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7b8101e00 msgr2=0x7fc7b81021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:49.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.394+0000 7fc7c0356700 1 --2- 172.21.15.71:0/3250336638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7b8101e00 0x7fc7b81021e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009910 tx=0x7fc7a8009c20 comp rx=0 tx=0).stop 2024-04-23T02:30:49.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.394+0000 7fc7c0356700 1 -- 172.21.15.71:0/3250336638 shutdown_connections 2024-04-23T02:30:49.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.394+0000 7fc7c0356700 1 --2- 172.21.15.71:0/3250336638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7b8102720 0x7fc7b810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:49.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.394+0000 7fc7c0356700 1 --2- 172.21.15.71:0/3250336638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7b8101e00 0x7fc7b81021e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:49.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.394+0000 7fc7c0356700 1 -- 172.21.15.71:0/3250336638 >> 172.21.15.71:0/3250336638 conn(0x7fc7b80fb650 msgr2=0x7fc7b80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:49.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.394+0000 7fc7c0356700 1 -- 172.21.15.71:0/3250336638 shutdown_connections 2024-04-23T02:30:49.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.394+0000 7fc7c0356700 1 -- 172.21.15.71:0/3250336638 wait complete. 2024-04-23T02:30:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.394+0000 7fc7c0356700 1 Processor -- start 2024-04-23T02:30:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.394+0000 7fc7c0356700 1 -- start start 2024-04-23T02:30:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.395+0000 7fc7c0356700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7b8101e00 0x7fc7b819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.395+0000 7fc7c0356700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7b8102720 0x7fc7b819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.395+0000 7fc7c0356700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b819d200 con 0x7fc7b8102720 2024-04-23T02:30:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.395+0000 7fc7c0356700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b81966b0 con 0x7fc7b8101e00 2024-04-23T02:30:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.395+0000 7fc7bd8f1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7b8102720 0x7fc7b819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.395+0000 7fc7bd8f1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7b8102720 0x7fc7b819cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48924/0 (socket says 172.21.15.71:48924) 2024-04-23T02:30:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.395+0000 7fc7bd8f1700 1 -- 172.21.15.71:0/1092265858 learned_addr learned my addr 172.21.15.71:0/1092265858 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:30:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.395+0000 7fc7be0f2700 1 --2- 172.21.15.71:0/1092265858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7b8101e00 0x7fc7b819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.395+0000 7fc7bd8f1700 1 -- 172.21.15.71:0/1092265858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7b8101e00 msgr2=0x7fc7b819c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.395+0000 7fc7bd8f1700 1 --2- 172.21.15.71:0/1092265858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7b8101e00 0x7fc7b819c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.395+0000 7fc7bd8f1700 1 -- 172.21.15.71:0/1092265858 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a80095f0 con 0x7fc7b8102720 2024-04-23T02:30:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.395+0000 7fc7be0f2700 1 --2- 172.21.15.71:0/1092265858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7b8101e00 0x7fc7b819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:30:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.395+0000 7fc7bd8f1700 1 --2- 172.21.15.71:0/1092265858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7b8102720 0x7fc7b819cb70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400d8d0 tx=0x7fc7b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.395+0000 7fc7af7fe700 1 -- 172.21.15.71:0/1092265858 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7b400b920 con 0x7fc7b8102720 2024-04-23T02:30:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.395+0000 7fc7af7fe700 1 -- 172.21.15.71:0/1092265858 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc7b4007620 con 0x7fc7b8102720 2024-04-23T02:30:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.395+0000 7fc7c0356700 1 -- 172.21.15.71:0/1092265858 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b8196990 con 0x7fc7b8102720 2024-04-23T02:30:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.395+0000 7fc7af7fe700 1 -- 172.21.15.71:0/1092265858 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7b400f3c0 con 0x7fc7b8102720 2024-04-23T02:30:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.395+0000 7fc7c0356700 1 -- 172.21.15.71:0/1092265858 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b8196ee0 con 0x7fc7b8102720 2024-04-23T02:30:49.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.396+0000 7fc7c0356700 1 -- 172.21.15.71:0/1092265858 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b8108310 con 0x7fc7b8102720 2024-04-23T02:30:49.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.397+0000 7fc7af7fe700 1 -- 172.21.15.71:0/1092265858 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc7b4010460 con 0x7fc7b8102720 2024-04-23T02:30:49.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.397+0000 7fc7af7fe700 1 --2- 172.21.15.71:0/1092265858 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc7a406cce0 0x7fc7a406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:49.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.397+0000 7fc7be0f2700 1 --2- 172.21.15.71:0/1092265858 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc7a406cce0 0x7fc7a406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:49.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.397+0000 7fc7af7fe700 1 -- 172.21.15.71:0/1092265858 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc7b408c400 con 0x7fc7b8102720 2024-04-23T02:30:49.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.398+0000 7fc7be0f2700 1 --2- 172.21.15.71:0/1092265858 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc7a406cce0 0x7fc7a406f1a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8006010 tx=0x7fc7a800b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:49.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.402+0000 7fc7af7fe700 1 -- 172.21.15.71:0/1092265858 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc7b4059f80 con 0x7fc7b8102720 2024-04-23T02:30:49.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.676+0000 7fc7c0356700 1 -- 172.21.15.71:0/1092265858 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc7b804f2e0 con 0x7fc7b8102720 2024-04-23T02:30:49.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.676+0000 7fc7af7fe700 1 -- 172.21.15.71:0/1092265858 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7fc7b4059b10 con 0x7fc7b8102720 2024-04-23T02:30:49.679 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:30:49.679 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:30:49.679 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:30:49.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:30:49.679 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:30:49.679 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:30:49.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:30:49.679 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:30:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:30:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:30:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:30:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:30:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:30:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:30:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:30:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:30:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:30:49.680 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:30:49.680 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:30:49.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.678+0000 7fc7c0356700 1 -- 172.21.15.71:0/1092265858 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc7a406cce0 msgr2=0x7fc7a406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:49.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.678+0000 7fc7c0356700 1 --2- 172.21.15.71:0/1092265858 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc7a406cce0 0x7fc7a406f1a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8006010 tx=0x7fc7a800b540 comp rx=0 tx=0).stop 2024-04-23T02:30:49.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.678+0000 7fc7c0356700 1 -- 172.21.15.71:0/1092265858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7b8102720 msgr2=0x7fc7b819cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:49.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.678+0000 7fc7c0356700 1 --2- 172.21.15.71:0/1092265858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7b8102720 0x7fc7b819cb70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400d8d0 tx=0x7fc7b400dc90 comp rx=0 tx=0).stop 2024-04-23T02:30:49.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.678+0000 7fc7c0356700 1 -- 172.21.15.71:0/1092265858 shutdown_connections 2024-04-23T02:30:49.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.678+0000 7fc7c0356700 1 --2- 172.21.15.71:0/1092265858 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc7a406cce0 0x7fc7a406f1a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:49.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.678+0000 7fc7c0356700 1 --2- 172.21.15.71:0/1092265858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7b8101e00 0x7fc7b819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:49.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.678+0000 7fc7c0356700 1 --2- 172.21.15.71:0/1092265858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7b8102720 0x7fc7b819cb70 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:49.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.678+0000 7fc7c0356700 1 -- 172.21.15.71:0/1092265858 >> 172.21.15.71:0/1092265858 conn(0x7fc7b80fb650 msgr2=0x7fc7b8105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:49.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.679+0000 7fc7c0356700 1 -- 172.21.15.71:0/1092265858 shutdown_connections 2024-04-23T02:30:49.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.679+0000 7fc7c0356700 1 -- 172.21.15.71:0/1092265858 wait complete. 2024-04-23T02:30:49.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.782+0000 7fe964f9c700 1 -- 172.21.15.71:0/537515714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe960101e60 msgr2=0x7fe960102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:49.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.782+0000 7fe964f9c700 1 --2- 172.21.15.71:0/537515714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe960101e60 0x7fe960102240 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe954009940 tx=0x7fe954009c50 comp rx=0 tx=0).stop 2024-04-23T02:30:49.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.782+0000 7fe964f9c700 1 -- 172.21.15.71:0/537515714 shutdown_connections 2024-04-23T02:30:49.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.782+0000 7fe964f9c700 1 --2- 172.21.15.71:0/537515714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe960102780 0x7fe96010ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:49.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.782+0000 7fe964f9c700 1 --2- 172.21.15.71:0/537515714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe960101e60 0x7fe960102240 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:49.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.782+0000 7fe964f9c700 1 -- 172.21.15.71:0/537515714 >> 172.21.15.71:0/537515714 conn(0x7fe9600fb650 msgr2=0x7fe9600fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:49.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.782+0000 7fe964f9c700 1 -- 172.21.15.71:0/537515714 shutdown_connections 2024-04-23T02:30:49.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.782+0000 7fe964f9c700 1 -- 172.21.15.71:0/537515714 wait complete. 2024-04-23T02:30:49.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.783+0000 7fe964f9c700 1 Processor -- start 2024-04-23T02:30:49.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.783+0000 7fe964f9c700 1 -- start start 2024-04-23T02:30:49.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.783+0000 7fe964f9c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe960101e60 0x7fe96019c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:49.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.783+0000 7fe964f9c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe960102780 0x7fe96019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:49.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.783+0000 7fe964f9c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96019d1d0 con 0x7fe960101e60 2024-04-23T02:30:49.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.783+0000 7fe964f9c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe960196680 con 0x7fe960102780 2024-04-23T02:30:49.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.783+0000 7fe95e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe960101e60 0x7fe96019c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:49.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.783+0000 7fe95e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe960101e60 0x7fe96019c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48940/0 (socket says 172.21.15.71:48940) 2024-04-23T02:30:49.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.783+0000 7fe95e59c700 1 -- 172.21.15.71:0/942732755 learned_addr learned my addr 172.21.15.71:0/942732755 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:30:49.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.783+0000 7fe95dd9b700 1 --2- 172.21.15.71:0/942732755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe960102780 0x7fe96019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:49.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.783+0000 7fe95e59c700 1 -- 172.21.15.71:0/942732755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe960102780 msgr2=0x7fe96019cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:49.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.783+0000 7fe95e59c700 1 --2- 172.21.15.71:0/942732755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe960102780 0x7fe96019cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:49.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.783+0000 7fe95e59c700 1 -- 172.21.15.71:0/942732755 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9540095f0 con 0x7fe960101e60 2024-04-23T02:30:49.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.783+0000 7fe95e59c700 1 --2- 172.21.15.71:0/942732755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe960101e60 0x7fe96019c600 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe95400b580 tx=0x7fe9540055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:49.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.783+0000 7fe94f7fe700 1 -- 172.21.15.71:0/942732755 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe95401d070 con 0x7fe960101e60 2024-04-23T02:30:49.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.784+0000 7fe94f7fe700 1 -- 172.21.15.71:0/942732755 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe95400bea0 con 0x7fe960101e60 2024-04-23T02:30:49.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.784+0000 7fe964f9c700 1 -- 172.21.15.71:0/942732755 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe960196900 con 0x7fe960101e60 2024-04-23T02:30:49.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.784+0000 7fe94f7fe700 1 -- 172.21.15.71:0/942732755 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe954021410 con 0x7fe960101e60 2024-04-23T02:30:49.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.784+0000 7fe964f9c700 1 -- 172.21.15.71:0/942732755 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe960196df0 con 0x7fe960101e60 2024-04-23T02:30:49.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.785+0000 7fe964f9c700 1 -- 172.21.15.71:0/942732755 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe960108370 con 0x7fe960101e60 2024-04-23T02:30:49.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.785+0000 7fe94f7fe700 1 -- 172.21.15.71:0/942732755 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe95400f6f0 con 0x7fe960101e60 2024-04-23T02:30:49.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.786+0000 7fe94f7fe700 1 --2- 172.21.15.71:0/942732755 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe94806cce0 0x7fe94806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:49.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.786+0000 7fe94f7fe700 1 -- 172.21.15.71:0/942732755 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe95408dcd0 con 0x7fe960101e60 2024-04-23T02:30:49.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.786+0000 7fe95dd9b700 1 --2- 172.21.15.71:0/942732755 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe94806cce0 0x7fe94806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:49.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.786+0000 7fe95dd9b700 1 --2- 172.21.15.71:0/942732755 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe94806cce0 0x7fe94806f1a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe9600fce10 tx=0x7fe95000b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:49.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:49.790+0000 7fe94f7fe700 1 -- 172.21.15.71:0/942732755 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe95405b870 con 0x7fe960101e60 2024-04-23T02:30:50.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.028+0000 7fe964f9c700 1 -- 172.21.15.71:0/942732755 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe960197ca0 con 0x7fe960101e60 2024-04-23T02:30:50.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.029+0000 7fe94f7fe700 1 -- 172.21.15.71:0/942732755 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7fe95405b400 con 0x7fe960101e60 2024-04-23T02:30:50.031 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:30:50.031 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:30:50.031 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:30:50.032 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:30:50.033 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:30:50.034 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:30:50.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.031+0000 7fe964f9c700 1 -- 172.21.15.71:0/942732755 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe94806cce0 msgr2=0x7fe94806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:50.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.031+0000 7fe964f9c700 1 --2- 172.21.15.71:0/942732755 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe94806cce0 0x7fe94806f1a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe9600fce10 tx=0x7fe95000b410 comp rx=0 tx=0).stop 2024-04-23T02:30:50.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.031+0000 7fe964f9c700 1 -- 172.21.15.71:0/942732755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe960101e60 msgr2=0x7fe96019c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:50.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.031+0000 7fe964f9c700 1 --2- 172.21.15.71:0/942732755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe960101e60 0x7fe96019c600 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe95400b580 tx=0x7fe9540055f0 comp rx=0 tx=0).stop 2024-04-23T02:30:50.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.031+0000 7fe964f9c700 1 -- 172.21.15.71:0/942732755 shutdown_connections 2024-04-23T02:30:50.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.031+0000 7fe964f9c700 1 --2- 172.21.15.71:0/942732755 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe94806cce0 0x7fe94806f1a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:50.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.031+0000 7fe964f9c700 1 --2- 172.21.15.71:0/942732755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe960101e60 0x7fe96019c600 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:50.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.031+0000 7fe964f9c700 1 --2- 172.21.15.71:0/942732755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe960102780 0x7fe96019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:50.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.031+0000 7fe964f9c700 1 -- 172.21.15.71:0/942732755 >> 172.21.15.71:0/942732755 conn(0x7fe9600fb650 msgr2=0x7fe9601054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:50.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.031+0000 7fe964f9c700 1 -- 172.21.15.71:0/942732755 shutdown_connections 2024-04-23T02:30:50.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.031+0000 7fe964f9c700 1 -- 172.21.15.71:0/942732755 wait complete. 2024-04-23T02:30:50.036 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:30:50.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.144+0000 7fb6fd0d8700 1 -- 172.21.15.71:0/440075531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6f80fff50 msgr2=0x7fb6f8105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:50.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.144+0000 7fb6fd0d8700 1 --2- 172.21.15.71:0/440075531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6f80fff50 0x7fb6f8105840 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009910 tx=0x7fb6e8009c20 comp rx=0 tx=0).stop 2024-04-23T02:30:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.144+0000 7fb6fd0d8700 1 -- 172.21.15.71:0/440075531 shutdown_connections 2024-04-23T02:30:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.144+0000 7fb6fd0d8700 1 --2- 172.21.15.71:0/440075531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6f80fff50 0x7fb6f8105840 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.144+0000 7fb6fd0d8700 1 --2- 172.21.15.71:0/440075531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6f80ff5a0 0x7fb6f80ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.144+0000 7fb6fd0d8700 1 -- 172.21.15.71:0/440075531 >> 172.21.15.71:0/440075531 conn(0x7fb6f8074e30 msgr2=0x7fb6f8075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.144+0000 7fb6fd0d8700 1 -- 172.21.15.71:0/440075531 shutdown_connections 2024-04-23T02:30:50.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.144+0000 7fb6fd0d8700 1 -- 172.21.15.71:0/440075531 wait complete. 2024-04-23T02:30:50.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.145+0000 7fb6fd0d8700 1 Processor -- start 2024-04-23T02:30:50.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.145+0000 7fb6fd0d8700 1 -- start start 2024-04-23T02:30:50.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.145+0000 7fb6fd0d8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6f80ff5a0 0x7fb6f819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:50.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.145+0000 7fb6fd0d8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6f80fff50 0x7fb6f819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:50.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.145+0000 7fb6fd0d8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f819d150 con 0x7fb6f80ff5a0 2024-04-23T02:30:50.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.145+0000 7fb6fd0d8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f8196650 con 0x7fb6f80fff50 2024-04-23T02:30:50.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.145+0000 7fb6f6d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6f80ff5a0 0x7fb6f819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:50.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.145+0000 7fb6f6d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6f80ff5a0 0x7fb6f819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48952/0 (socket says 172.21.15.71:48952) 2024-04-23T02:30:50.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.145+0000 7fb6f6d9d700 1 -- 172.21.15.71:0/2943762421 learned_addr learned my addr 172.21.15.71:0/2943762421 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:30:50.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.145+0000 7fb6f659c700 1 --2- 172.21.15.71:0/2943762421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6f80fff50 0x7fb6f819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:50.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.145+0000 7fb6f6d9d700 1 -- 172.21.15.71:0/2943762421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6f80fff50 msgr2=0x7fb6f819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:50.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.145+0000 7fb6f6d9d700 1 --2- 172.21.15.71:0/2943762421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6f80fff50 0x7fb6f819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:50.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.145+0000 7fb6f6d9d700 1 -- 172.21.15.71:0/2943762421 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e80095f0 con 0x7fb6f80ff5a0 2024-04-23T02:30:50.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.145+0000 7fb6f659c700 1 --2- 172.21.15.71:0/2943762421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6f80fff50 0x7fb6f819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:30:50.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.146+0000 7fb6f6d9d700 1 --2- 172.21.15.71:0/2943762421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6f80ff5a0 0x7fb6f819c580 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00e940 tx=0x7fb6ec00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:50.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.146+0000 7fb6dffff700 1 -- 172.21.15.71:0/2943762421 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6ec0097b0 con 0x7fb6f80ff5a0 2024-04-23T02:30:50.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.146+0000 7fb6fd0d8700 1 -- 172.21.15.71:0/2943762421 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f8196930 con 0x7fb6f80ff5a0 2024-04-23T02:30:50.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.146+0000 7fb6dffff700 1 -- 172.21.15.71:0/2943762421 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb6ec004830 con 0x7fb6f80ff5a0 2024-04-23T02:30:50.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.146+0000 7fb6dffff700 1 -- 172.21.15.71:0/2943762421 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6ec0055b0 con 0x7fb6f80ff5a0 2024-04-23T02:30:50.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.146+0000 7fb6fd0d8700 1 -- 172.21.15.71:0/2943762421 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f8196e80 con 0x7fb6f80ff5a0 2024-04-23T02:30:50.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.147+0000 7fb6fd0d8700 1 -- 172.21.15.71:0/2943762421 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f8109200 con 0x7fb6f80ff5a0 2024-04-23T02:30:50.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.147+0000 7fb6dffff700 1 -- 172.21.15.71:0/2943762421 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb6ec0049a0 con 0x7fb6f80ff5a0 2024-04-23T02:30:50.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.148+0000 7fb6dffff700 1 --2- 172.21.15.71:0/2943762421 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb6e006cce0 0x7fb6e006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:50.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.148+0000 7fb6dffff700 1 -- 172.21.15.71:0/2943762421 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb6ec014070 con 0x7fb6f80ff5a0 2024-04-23T02:30:50.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.148+0000 7fb6f659c700 1 --2- 172.21.15.71:0/2943762421 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb6e006cce0 0x7fb6e006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:50.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.148+0000 7fb6f659c700 1 --2- 172.21.15.71:0/2943762421 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb6e006cce0 0x7fb6e006f1a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009fd0 tx=0x7fb6e8005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.152+0000 7fb6dffff700 1 -- 172.21.15.71:0/2943762421 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb6ec05a480 con 0x7fb6f80ff5a0 2024-04-23T02:30:50.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.362+0000 7fb6fd0d8700 1 -- 172.21.15.71:0/2943762421 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6f81971b0 con 0x7fb6e006cce0 2024-04-23T02:30:50.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.363+0000 7fb6dffff700 1 -- 172.21.15.71:0/2943762421 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb6f81971b0 con 0x7fb6e006cce0 2024-04-23T02:30:50.365 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:30:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:30:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:30:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:30:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:30:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:30:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:30:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:30:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:30:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:30:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:30:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:30:50.366 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:30:50.366 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:30:50.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.365+0000 7fb6fd0d8700 1 -- 172.21.15.71:0/2943762421 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb6e006cce0 msgr2=0x7fb6e006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:50.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.365+0000 7fb6fd0d8700 1 --2- 172.21.15.71:0/2943762421 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb6e006cce0 0x7fb6e006f1a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009fd0 tx=0x7fb6e8005940 comp rx=0 tx=0).stop 2024-04-23T02:30:50.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.365+0000 7fb6fd0d8700 1 -- 172.21.15.71:0/2943762421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6f80ff5a0 msgr2=0x7fb6f819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:50.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.365+0000 7fb6fd0d8700 1 --2- 172.21.15.71:0/2943762421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6f80ff5a0 0x7fb6f819c580 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00e940 tx=0x7fb6ec00ed00 comp rx=0 tx=0).stop 2024-04-23T02:30:50.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.365+0000 7fb6fd0d8700 1 -- 172.21.15.71:0/2943762421 shutdown_connections 2024-04-23T02:30:50.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.365+0000 7fb6fd0d8700 1 --2- 172.21.15.71:0/2943762421 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb6e006cce0 0x7fb6e006f1a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:50.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.365+0000 7fb6fd0d8700 1 --2- 172.21.15.71:0/2943762421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6f80ff5a0 0x7fb6f819c580 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:50.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.365+0000 7fb6fd0d8700 1 --2- 172.21.15.71:0/2943762421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6f80fff50 0x7fb6f819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:50.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.365+0000 7fb6fd0d8700 1 -- 172.21.15.71:0/2943762421 >> 172.21.15.71:0/2943762421 conn(0x7fb6f8074e30 msgr2=0x7fb6f80fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:50.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.365+0000 7fb6fd0d8700 1 -- 172.21.15.71:0/2943762421 shutdown_connections 2024-04-23T02:30:50.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.365+0000 7fb6fd0d8700 1 -- 172.21.15.71:0/2943762421 wait complete. 2024-04-23T02:30:50.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:50 smithi192 ceph-mon[108617]: from='client.34790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:30:50.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:50 smithi192 ceph-mon[108617]: pgmap v525: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:50.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:50 smithi192 ceph-mon[108617]: from='client.34794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:30:50.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:50 smithi192 ceph-mon[108617]: from='client.34798 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:30:50.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:50 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1092265858' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:30:50.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:30:50.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:50 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/942732755' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:30:50.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.468+0000 7f2a8eb6f700 1 -- 172.21.15.71:0/1550997853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a880ffc20 msgr2=0x7f2a88100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:50.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.468+0000 7f2a8eb6f700 1 --2- 172.21.15.71:0/1550997853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a880ffc20 0x7f2a88100080 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009910 tx=0x7f2a74009c20 comp rx=0 tx=0).stop 2024-04-23T02:30:50.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.468+0000 7f2a8eb6f700 1 -- 172.21.15.71:0/1550997853 shutdown_connections 2024-04-23T02:30:50.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.468+0000 7f2a8eb6f700 1 --2- 172.21.15.71:0/1550997853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a880ffc20 0x7f2a88100080 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:50.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.468+0000 7f2a8eb6f700 1 --2- 172.21.15.71:0/1550997853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a88105bd0 0x7f2a88105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:50.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.468+0000 7f2a8eb6f700 1 -- 172.21.15.71:0/1550997853 >> 172.21.15.71:0/1550997853 conn(0x7f2a88075670 msgr2=0x7f2a88075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:50.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.468+0000 7f2a8eb6f700 1 -- 172.21.15.71:0/1550997853 shutdown_connections 2024-04-23T02:30:50.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.468+0000 7f2a8eb6f700 1 -- 172.21.15.71:0/1550997853 wait complete. 2024-04-23T02:30:50.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.468+0000 7f2a8eb6f700 1 Processor -- start 2024-04-23T02:30:50.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.469+0000 7f2a8eb6f700 1 -- start start 2024-04-23T02:30:50.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.469+0000 7f2a8eb6f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a880ffc20 0x7f2a88197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:50.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.469+0000 7f2a8eb6f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a88105bd0 0x7f2a88197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:50.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.469+0000 7f2a8eb6f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a881982c0 con 0x7f2a88105bd0 2024-04-23T02:30:50.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.469+0000 7f2a8eb6f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a88198400 con 0x7f2a880ffc20 2024-04-23T02:30:50.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.469+0000 7f2a87fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a88105bd0 0x7f2a88197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:50.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.469+0000 7f2a87fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a88105bd0 0x7f2a88197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48960/0 (socket says 172.21.15.71:48960) 2024-04-23T02:30:50.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.469+0000 7f2a87fff700 1 -- 172.21.15.71:0/323480898 learned_addr learned my addr 172.21.15.71:0/323480898 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:30:50.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.469+0000 7f2a8c90b700 1 --2- 172.21.15.71:0/323480898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a880ffc20 0x7f2a88197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:50.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.469+0000 7f2a87fff700 1 -- 172.21.15.71:0/323480898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a880ffc20 msgr2=0x7f2a88197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:50.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.469+0000 7f2a87fff700 1 --2- 172.21.15.71:0/323480898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a880ffc20 0x7f2a88197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:50.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.469+0000 7f2a87fff700 1 -- 172.21.15.71:0/323480898 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a740095f0 con 0x7f2a88105bd0 2024-04-23T02:30:50.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.469+0000 7f2a87fff700 1 --2- 172.21.15.71:0/323480898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a88105bd0 0x7f2a88197ca0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2a74006010 tx=0x7f2a7400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:50.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.469+0000 7f2a85ffb700 1 -- 172.21.15.71:0/323480898 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a7401d070 con 0x7f2a88105bd0 2024-04-23T02:30:50.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.469+0000 7f2a85ffb700 1 -- 172.21.15.71:0/323480898 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2a7400f460 con 0x7f2a88105bd0 2024-04-23T02:30:50.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.469+0000 7f2a8eb6f700 1 -- 172.21.15.71:0/323480898 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a881a07d0 con 0x7f2a88105bd0 2024-04-23T02:30:50.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.470+0000 7f2a85ffb700 1 -- 172.21.15.71:0/323480898 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a74021620 con 0x7f2a88105bd0 2024-04-23T02:30:50.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.470+0000 7f2a8eb6f700 1 -- 172.21.15.71:0/323480898 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a881a0be0 con 0x7f2a88105bd0 2024-04-23T02:30:50.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.471+0000 7f2a8eb6f700 1 -- 172.21.15.71:0/323480898 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a881080f0 con 0x7f2a88105bd0 2024-04-23T02:30:50.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.471+0000 7f2a85ffb700 1 -- 172.21.15.71:0/323480898 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2a7400f5d0 con 0x7f2a88105bd0 2024-04-23T02:30:50.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.471+0000 7f2a85ffb700 1 --2- 172.21.15.71:0/323480898 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2a7806cce0 0x7f2a7806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:30:50.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.472+0000 7f2a85ffb700 1 -- 172.21.15.71:0/323480898 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2a7408d730 con 0x7f2a88105bd0 2024-04-23T02:30:50.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.472+0000 7f2a8c90b700 1 --2- 172.21.15.71:0/323480898 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2a7806cce0 0x7f2a7806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:30:50.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.472+0000 7f2a8c90b700 1 --2- 172.21.15.71:0/323480898 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2a7806cce0 0x7f2a7806f1a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009a10 tx=0x7f2a7c009190 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:30:50.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.476+0000 7f2a85ffb700 1 -- 172.21.15.71:0/323480898 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2a740577a0 con 0x7f2a88105bd0 2024-04-23T02:30:50.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:50 smithi071 ceph-mon[144544]: from='client.34790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:30:50.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:50 smithi071 ceph-mon[144544]: pgmap v525: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:50.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:50 smithi071 ceph-mon[144544]: from='client.34794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:30:50.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:50 smithi071 ceph-mon[144544]: from='client.34798 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:30:50.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:50 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1092265858' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:30:50.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:30:50.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:50 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/942732755' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:30:50.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.750+0000 7f2a8eb6f700 1 -- 172.21.15.71:0/323480898 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2a8804f2e0 con 0x7f2a88105bd0 2024-04-23T02:30:50.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.750+0000 7f2a85ffb700 1 -- 172.21.15.71:0/323480898 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f2a7405adc0 con 0x7f2a88105bd0 2024-04-23T02:30:50.753 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:30:50.753 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:30:50.753 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:30:50.753 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:30:50.753 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:30:50.753 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:30:50.753 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:30:50.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.752+0000 7f2a8eb6f700 1 -- 172.21.15.71:0/323480898 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2a7806cce0 msgr2=0x7f2a7806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:50.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.752+0000 7f2a8eb6f700 1 --2- 172.21.15.71:0/323480898 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2a7806cce0 0x7f2a7806f1a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009a10 tx=0x7f2a7c009190 comp rx=0 tx=0).stop 2024-04-23T02:30:50.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.752+0000 7f2a8eb6f700 1 -- 172.21.15.71:0/323480898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a88105bd0 msgr2=0x7f2a88197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:30:50.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.752+0000 7f2a8eb6f700 1 --2- 172.21.15.71:0/323480898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a88105bd0 0x7f2a88197ca0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2a74006010 tx=0x7f2a7400bd70 comp rx=0 tx=0).stop 2024-04-23T02:30:50.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.752+0000 7f2a8eb6f700 1 -- 172.21.15.71:0/323480898 shutdown_connections 2024-04-23T02:30:50.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.752+0000 7f2a8eb6f700 1 --2- 172.21.15.71:0/323480898 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2a7806cce0 0x7f2a7806f1a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:50.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.752+0000 7f2a8eb6f700 1 --2- 172.21.15.71:0/323480898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a880ffc20 0x7f2a88197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:50.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.752+0000 7f2a8eb6f700 1 --2- 172.21.15.71:0/323480898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a88105bd0 0x7f2a88197ca0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:30:50.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.752+0000 7f2a8eb6f700 1 -- 172.21.15.71:0/323480898 >> 172.21.15.71:0/323480898 conn(0x7f2a88075670 msgr2=0x7f2a880fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:30:50.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.752+0000 7f2a8eb6f700 1 -- 172.21.15.71:0/323480898 shutdown_connections 2024-04-23T02:30:50.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:30:50.752+0000 7f2a8eb6f700 1 -- 172.21.15.71:0/323480898 wait complete. 2024-04-23T02:30:51.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:51 smithi192 ceph-mon[108617]: from='client.34810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:30:51.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:51 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/323480898' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:30:51.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:51 smithi071 ceph-mon[144544]: from='client.34810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:30:51.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:51 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/323480898' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:30:52.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:52 smithi192 ceph-mon[108617]: pgmap v526: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:52.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:52 smithi071 ceph-mon[144544]: pgmap v526: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:54.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:54 smithi192 ceph-mon[108617]: pgmap v527: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:54.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:54 smithi071 ceph-mon[144544]: pgmap v527: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:56.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:56 smithi192 ceph-mon[108617]: pgmap v528: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:56.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:56 smithi071 ceph-mon[144544]: pgmap v528: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:58.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:30:58 smithi192 ceph-mon[108617]: pgmap v529: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:30:58.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:30:58 smithi071 ceph-mon[144544]: pgmap v529: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:00.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:00 smithi192 ceph-mon[108617]: pgmap v530: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:00.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:00 smithi071 ceph-mon[144544]: pgmap v530: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:02.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:02 smithi192 ceph-mon[108617]: pgmap v531: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:02.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:02 smithi071 ceph-mon[144544]: pgmap v531: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:04.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:04 smithi192 ceph-mon[108617]: pgmap v532: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:04.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:04 smithi071 ceph-mon[144544]: pgmap v532: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:05.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:31:05.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:31:06.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:06 smithi192 ceph-mon[108617]: pgmap v533: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:06.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:06 smithi071 ceph-mon[144544]: pgmap v533: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:08.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:08 smithi192 ceph-mon[108617]: pgmap v534: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:08.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:08 smithi071 ceph-mon[144544]: pgmap v534: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:10.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:10 smithi192 ceph-mon[108617]: pgmap v535: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:10.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:10 smithi071 ceph-mon[144544]: pgmap v535: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:12.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:12 smithi192 ceph-mon[108617]: pgmap v536: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:12.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:12 smithi071 ceph-mon[144544]: pgmap v536: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:14.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:14 smithi192 ceph-mon[108617]: pgmap v537: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:14.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:14 smithi071 ceph-mon[144544]: pgmap v537: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:16.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:16 smithi192 ceph-mon[108617]: pgmap v538: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:16.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:16 smithi071 ceph-mon[144544]: pgmap v538: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:18.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:18 smithi071 ceph-mon[144544]: pgmap v539: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:18.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:18 smithi192 ceph-mon[108617]: pgmap v539: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:20.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:20 smithi071 ceph-mon[144544]: pgmap v540: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:20.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:31:20.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.867+0000 7f26e5e69700 1 -- 172.21.15.71:0/1653770673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e0100e90 msgr2=0x7f26e0101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:20.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.867+0000 7f26e5e69700 1 --2- 172.21.15.71:0/1653770673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e0100e90 0x7f26e0101270 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009910 tx=0x7f26d4009c20 comp rx=0 tx=0).stop 2024-04-23T02:31:20.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.867+0000 7f26e5e69700 1 -- 172.21.15.71:0/1653770673 shutdown_connections 2024-04-23T02:31:20.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.868+0000 7f26e5e69700 1 --2- 172.21.15.71:0/1653770673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26e0101840 0x7f26e0105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:20.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.868+0000 7f26e5e69700 1 --2- 172.21.15.71:0/1653770673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e0100e90 0x7f26e0101270 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:20.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.868+0000 7f26e5e69700 1 -- 172.21.15.71:0/1653770673 >> 172.21.15.71:0/1653770673 conn(0x7f26e00fc740 msgr2=0x7f26e00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:20.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.868+0000 7f26e5e69700 1 -- 172.21.15.71:0/1653770673 shutdown_connections 2024-04-23T02:31:20.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.868+0000 7f26e5e69700 1 -- 172.21.15.71:0/1653770673 wait complete. 2024-04-23T02:31:20.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.868+0000 7f26e5e69700 1 Processor -- start 2024-04-23T02:31:20.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.868+0000 7f26e5e69700 1 -- start start 2024-04-23T02:31:20.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.869+0000 7f26e5e69700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e0100e90 0x7f26e019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:20.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.869+0000 7f26e5e69700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26e0101840 0x7f26e019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:20.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.869+0000 7f26e5e69700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e019d100 con 0x7f26e0100e90 2024-04-23T02:31:20.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.869+0000 7f26e5e69700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e0196640 con 0x7f26e0101840 2024-04-23T02:31:20.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.869+0000 7f26df7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e0100e90 0x7f26e019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:20.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.869+0000 7f26df7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e0100e90 0x7f26e019c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42716/0 (socket says 172.21.15.71:42716) 2024-04-23T02:31:20.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.869+0000 7f26df7fe700 1 -- 172.21.15.71:0/4074213269 learned_addr learned my addr 172.21.15.71:0/4074213269 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:31:20.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.869+0000 7f26deffd700 1 --2- 172.21.15.71:0/4074213269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26e0101840 0x7f26e019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:20.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.869+0000 7f26df7fe700 1 -- 172.21.15.71:0/4074213269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26e0101840 msgr2=0x7f26e019cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:20.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.869+0000 7f26df7fe700 1 --2- 172.21.15.71:0/4074213269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26e0101840 0x7f26e019cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:20.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.869+0000 7f26df7fe700 1 -- 172.21.15.71:0/4074213269 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d40095f0 con 0x7f26e0100e90 2024-04-23T02:31:20.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.869+0000 7f26df7fe700 1 --2- 172.21.15.71:0/4074213269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e0100e90 0x7f26e019c5c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f26d400b5c0 tx=0x7f26d400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:20.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.869+0000 7f26dcff9700 1 -- 172.21.15.71:0/4074213269 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26d401d070 con 0x7f26e0100e90 2024-04-23T02:31:20.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.869+0000 7f26e5e69700 1 -- 172.21.15.71:0/4074213269 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e01968c0 con 0x7f26e0100e90 2024-04-23T02:31:20.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.869+0000 7f26dcff9700 1 -- 172.21.15.71:0/4074213269 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f26d400f460 con 0x7f26e0100e90 2024-04-23T02:31:20.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.869+0000 7f26dcff9700 1 -- 172.21.15.71:0/4074213269 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26d4021640 con 0x7f26e0100e90 2024-04-23T02:31:20.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.869+0000 7f26e5e69700 1 -- 172.21.15.71:0/4074213269 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e0196db0 con 0x7f26e0100e90 2024-04-23T02:31:20.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.871+0000 7f26e5e69700 1 -- 172.21.15.71:0/4074213269 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e0109230 con 0x7f26e0100e90 2024-04-23T02:31:20.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.871+0000 7f26dcff9700 1 -- 172.21.15.71:0/4074213269 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f26d400fa20 con 0x7f26e0100e90 2024-04-23T02:31:20.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.871+0000 7f26dcff9700 1 --2- 172.21.15.71:0/4074213269 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f26c806cce0 0x7f26c806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:20.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.872+0000 7f26dcff9700 1 -- 172.21.15.71:0/4074213269 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f26d408d860 con 0x7f26e0100e90 2024-04-23T02:31:20.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.872+0000 7f26deffd700 1 --2- 172.21.15.71:0/4074213269 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f26c806cce0 0x7f26c806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:20.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.872+0000 7f26deffd700 1 --2- 172.21.15.71:0/4074213269 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f26c806cce0 0x7f26c806f1a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f26e00fd000 tx=0x7f26d000a380 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:20.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:20 smithi192 ceph-mon[108617]: pgmap v540: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:20.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:31:20.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:20.876+0000 7f26dcff9700 1 -- 172.21.15.71:0/4074213269 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f26d405b2c0 con 0x7f26e0100e90 2024-04-23T02:31:21.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.086+0000 7f26e5e69700 1 -- 172.21.15.71:0/4074213269 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f26e0197c70 con 0x7f26c806cce0 2024-04-23T02:31:21.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.087+0000 7f26dcff9700 1 -- 172.21.15.71:0/4074213269 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f26d000f990 con 0x7f26c806cce0 2024-04-23T02:31:21.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.089+0000 7f26e5e69700 1 -- 172.21.15.71:0/4074213269 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f26c806cce0 msgr2=0x7f26c806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:21.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.089+0000 7f26e5e69700 1 --2- 172.21.15.71:0/4074213269 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f26c806cce0 0x7f26c806f1a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f26e00fd000 tx=0x7f26d000a380 comp rx=0 tx=0).stop 2024-04-23T02:31:21.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.089+0000 7f26e5e69700 1 -- 172.21.15.71:0/4074213269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e0100e90 msgr2=0x7f26e019c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:21.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.089+0000 7f26e5e69700 1 --2- 172.21.15.71:0/4074213269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e0100e90 0x7f26e019c5c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f26d400b5c0 tx=0x7f26d400bd70 comp rx=0 tx=0).stop 2024-04-23T02:31:21.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.089+0000 7f26e5e69700 1 -- 172.21.15.71:0/4074213269 shutdown_connections 2024-04-23T02:31:21.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.089+0000 7f26e5e69700 1 --2- 172.21.15.71:0/4074213269 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f26c806cce0 0x7f26c806f1a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:21.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.089+0000 7f26e5e69700 1 --2- 172.21.15.71:0/4074213269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e0100e90 0x7f26e019c5c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:21.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.089+0000 7f26e5e69700 1 --2- 172.21.15.71:0/4074213269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26e0101840 0x7f26e019cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:21.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.089+0000 7f26e5e69700 1 -- 172.21.15.71:0/4074213269 >> 172.21.15.71:0/4074213269 conn(0x7f26e00fc740 msgr2=0x7f26e00fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:21.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.089+0000 7f26e5e69700 1 -- 172.21.15.71:0/4074213269 shutdown_connections 2024-04-23T02:31:21.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.090+0000 7f26e5e69700 1 -- 172.21.15.71:0/4074213269 wait complete. 2024-04-23T02:31:21.102 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:31:21.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.193+0000 7f7759c20700 1 -- 172.21.15.71:0/3350399273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7754105bd0 msgr2=0x7f7754105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:21.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.193+0000 7f7759c20700 1 --2- 172.21.15.71:0/3350399273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7754105bd0 0x7f7754105fb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7744009910 tx=0x7f7744009c20 comp rx=0 tx=0).stop 2024-04-23T02:31:21.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.193+0000 7f7759c20700 1 -- 172.21.15.71:0/3350399273 shutdown_connections 2024-04-23T02:31:21.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.193+0000 7f7759c20700 1 --2- 172.21.15.71:0/3350399273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77540ffc20 0x7f7754100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:21.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.193+0000 7f7759c20700 1 --2- 172.21.15.71:0/3350399273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7754105bd0 0x7f7754105fb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:21.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.193+0000 7f7759c20700 1 -- 172.21.15.71:0/3350399273 >> 172.21.15.71:0/3350399273 conn(0x7f7754075670 msgr2=0x7f7754075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:21.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.193+0000 7f7759c20700 1 -- 172.21.15.71:0/3350399273 shutdown_connections 2024-04-23T02:31:21.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.193+0000 7f7759c20700 1 -- 172.21.15.71:0/3350399273 wait complete. 2024-04-23T02:31:21.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.194+0000 7f7759c20700 1 Processor -- start 2024-04-23T02:31:21.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.194+0000 7f7759c20700 1 -- start start 2024-04-23T02:31:21.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.194+0000 7f7759c20700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77540ffc20 0x7f77541975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:21.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.194+0000 7f7759c20700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7754105bd0 0x7f7754197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:21.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.194+0000 7f7759c20700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7754198140 con 0x7f7754105bd0 2024-04-23T02:31:21.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.194+0000 7f7759c20700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77541982b0 con 0x7f77540ffc20 2024-04-23T02:31:21.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.195+0000 7f7752ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7754105bd0 0x7f7754197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:21.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.195+0000 7f7752ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7754105bd0 0x7f7754197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42728/0 (socket says 172.21.15.71:42728) 2024-04-23T02:31:21.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.195+0000 7f7752ffd700 1 -- 172.21.15.71:0/1815722217 learned_addr learned my addr 172.21.15.71:0/1815722217 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:31:21.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.195+0000 7f77537fe700 1 --2- 172.21.15.71:0/1815722217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77540ffc20 0x7f77541975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:21.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.195+0000 7f7752ffd700 1 -- 172.21.15.71:0/1815722217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77540ffc20 msgr2=0x7f77541975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:21.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.195+0000 7f7752ffd700 1 --2- 172.21.15.71:0/1815722217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77540ffc20 0x7f77541975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:21.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.195+0000 7f7752ffd700 1 -- 172.21.15.71:0/1815722217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77440095f0 con 0x7f7754105bd0 2024-04-23T02:31:21.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.195+0000 7f7752ffd700 1 --2- 172.21.15.71:0/1815722217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7754105bd0 0x7f7754197b20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f774800d900 tx=0x7f774800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:21.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.195+0000 7f7750ff9700 1 -- 172.21.15.71:0/1815722217 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77480041f0 con 0x7f7754105bd0 2024-04-23T02:31:21.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.195+0000 7f7750ff9700 1 -- 172.21.15.71:0/1815722217 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7748004cd0 con 0x7f7754105bd0 2024-04-23T02:31:21.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.195+0000 7f7759c20700 1 -- 172.21.15.71:0/1815722217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77541a0650 con 0x7f7754105bd0 2024-04-23T02:31:21.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.195+0000 7f7750ff9700 1 -- 172.21.15.71:0/1815722217 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7748005660 con 0x7f7754105bd0 2024-04-23T02:31:21.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.195+0000 7f7759c20700 1 -- 172.21.15.71:0/1815722217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77541a0b40 con 0x7f7754105bd0 2024-04-23T02:31:21.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.196+0000 7f7759c20700 1 -- 172.21.15.71:0/1815722217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77541080d0 con 0x7f7754105bd0 2024-04-23T02:31:21.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.197+0000 7f7750ff9700 1 -- 172.21.15.71:0/1815722217 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7748004e40 con 0x7f7754105bd0 2024-04-23T02:31:21.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.197+0000 7f7750ff9700 1 --2- 172.21.15.71:0/1815722217 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f773c06cce0 0x7f773c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:21.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.198+0000 7f7750ff9700 1 -- 172.21.15.71:0/1815722217 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f774808c920 con 0x7f7754105bd0 2024-04-23T02:31:21.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.198+0000 7f77537fe700 1 --2- 172.21.15.71:0/1815722217 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f773c06cce0 0x7f773c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:21.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.198+0000 7f77537fe700 1 --2- 172.21.15.71:0/1815722217 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f773c06cce0 0x7f773c06f1a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7744006010 tx=0x7f774401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:21.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.202+0000 7f7750ff9700 1 -- 172.21.15.71:0/1815722217 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7748056970 con 0x7f7754105bd0 2024-04-23T02:31:21.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.413+0000 7f7759c20700 1 -- 172.21.15.71:0/1815722217 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f775402cce0 con 0x7f773c06cce0 2024-04-23T02:31:21.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.414+0000 7f7750ff9700 1 -- 172.21.15.71:0/1815722217 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f775402cce0 con 0x7f773c06cce0 2024-04-23T02:31:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.416+0000 7f7759c20700 1 -- 172.21.15.71:0/1815722217 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f773c06cce0 msgr2=0x7f773c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.416+0000 7f7759c20700 1 --2- 172.21.15.71:0/1815722217 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f773c06cce0 0x7f773c06f1a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7744006010 tx=0x7f774401a040 comp rx=0 tx=0).stop 2024-04-23T02:31:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.416+0000 7f7759c20700 1 -- 172.21.15.71:0/1815722217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7754105bd0 msgr2=0x7f7754197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.416+0000 7f7759c20700 1 --2- 172.21.15.71:0/1815722217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7754105bd0 0x7f7754197b20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f774800d900 tx=0x7f774800dc10 comp rx=0 tx=0).stop 2024-04-23T02:31:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.416+0000 7f7759c20700 1 -- 172.21.15.71:0/1815722217 shutdown_connections 2024-04-23T02:31:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.416+0000 7f7759c20700 1 --2- 172.21.15.71:0/1815722217 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f773c06cce0 0x7f773c06f1a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.416+0000 7f7759c20700 1 --2- 172.21.15.71:0/1815722217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77540ffc20 0x7f77541975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.416+0000 7f7759c20700 1 --2- 172.21.15.71:0/1815722217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7754105bd0 0x7f7754197b20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.416+0000 7f7759c20700 1 -- 172.21.15.71:0/1815722217 >> 172.21.15.71:0/1815722217 conn(0x7f7754075670 msgr2=0x7f77540fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:21.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.417+0000 7f7759c20700 1 -- 172.21.15.71:0/1815722217 shutdown_connections 2024-04-23T02:31:21.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.417+0000 7f7759c20700 1 -- 172.21.15.71:0/1815722217 wait complete. 2024-04-23T02:31:21.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.519+0000 7f12137f1700 1 -- 172.21.15.71:0/2426024998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f120c100750 msgr2=0x7f120c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:21.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.519+0000 7f12137f1700 1 --2- 172.21.15.71:0/2426024998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f120c100750 0x7f120c1047a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1208009910 tx=0x7f1208009c20 comp rx=0 tx=0).stop 2024-04-23T02:31:21.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.519+0000 7f12137f1700 1 -- 172.21.15.71:0/2426024998 shutdown_connections 2024-04-23T02:31:21.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.519+0000 7f12137f1700 1 --2- 172.21.15.71:0/2426024998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f120c100750 0x7f120c1047a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:21.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.519+0000 7f12137f1700 1 --2- 172.21.15.71:0/2426024998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f120c0ffda0 0x7f120c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:21.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.519+0000 7f12137f1700 1 -- 172.21.15.71:0/2426024998 >> 172.21.15.71:0/2426024998 conn(0x7f120c0fb650 msgr2=0x7f120c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:21.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.519+0000 7f12137f1700 1 -- 172.21.15.71:0/2426024998 shutdown_connections 2024-04-23T02:31:21.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.519+0000 7f12137f1700 1 -- 172.21.15.71:0/2426024998 wait complete. 2024-04-23T02:31:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.520+0000 7f12137f1700 1 Processor -- start 2024-04-23T02:31:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.520+0000 7f12137f1700 1 -- start start 2024-04-23T02:31:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.520+0000 7f12137f1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f120c0ffda0 0x7f120c1954f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.520+0000 7f12137f1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f120c100750 0x7f120c195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.520+0000 7f12137f1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c195f70 con 0x7f120c100750 2024-04-23T02:31:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.520+0000 7f12137f1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c1960e0 con 0x7f120c0ffda0 2024-04-23T02:31:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.520+0000 7f1210d8c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f120c100750 0x7f120c195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.520+0000 7f1210d8c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f120c100750 0x7f120c195a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42742/0 (socket says 172.21.15.71:42742) 2024-04-23T02:31:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.520+0000 7f1210d8c700 1 -- 172.21.15.71:0/465754302 learned_addr learned my addr 172.21.15.71:0/465754302 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:31:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.520+0000 7f121158d700 1 --2- 172.21.15.71:0/465754302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f120c0ffda0 0x7f120c1954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.520+0000 7f1210d8c700 1 -- 172.21.15.71:0/465754302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f120c0ffda0 msgr2=0x7f120c1954f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.520+0000 7f1210d8c700 1 --2- 172.21.15.71:0/465754302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f120c0ffda0 0x7f120c1954f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.520+0000 7f1210d8c700 1 -- 172.21.15.71:0/465754302 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12080095f0 con 0x7f120c100750 2024-04-23T02:31:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.521+0000 7f1210d8c700 1 --2- 172.21.15.71:0/465754302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f120c100750 0x7f120c195a30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f12080092e0 tx=0x7f120800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.521+0000 7f11fe7fc700 1 -- 172.21.15.71:0/465754302 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f120801d070 con 0x7f120c100750 2024-04-23T02:31:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.521+0000 7f12137f1700 1 -- 172.21.15.71:0/465754302 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120c196360 con 0x7f120c100750 2024-04-23T02:31:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.521+0000 7f11fe7fc700 1 -- 172.21.15.71:0/465754302 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1208005dc0 con 0x7f120c100750 2024-04-23T02:31:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.521+0000 7f11fe7fc700 1 -- 172.21.15.71:0/465754302 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f120800fea0 con 0x7f120c100750 2024-04-23T02:31:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.521+0000 7f12137f1700 1 -- 172.21.15.71:0/465754302 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120c1a2de0 con 0x7f120c100750 2024-04-23T02:31:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.522+0000 7f12137f1700 1 -- 172.21.15.71:0/465754302 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120c1081a0 con 0x7f120c100750 2024-04-23T02:31:21.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.522+0000 7f11fe7fc700 1 -- 172.21.15.71:0/465754302 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1208003a60 con 0x7f120c100750 2024-04-23T02:31:21.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.522+0000 7f11fe7fc700 1 --2- 172.21.15.71:0/465754302 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f11f806cce0 0x7f11f806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:21.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.523+0000 7f11fe7fc700 1 -- 172.21.15.71:0/465754302 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f120808d5b0 con 0x7f120c100750 2024-04-23T02:31:21.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.523+0000 7f121158d700 1 --2- 172.21.15.71:0/465754302 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f11f806cce0 0x7f11f806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:21.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.523+0000 7f121158d700 1 --2- 172.21.15.71:0/465754302 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f11f806cce0 0x7f11f806f1a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1200005950 tx=0x7f120000b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:21.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.527+0000 7f11fe7fc700 1 -- 172.21.15.71:0/465754302 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f120805b0b0 con 0x7f120c100750 2024-04-23T02:31:21.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.731+0000 7f12137f1700 1 -- 172.21.15.71:0/465754302 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f120c196360 con 0x7f11f806cce0 2024-04-23T02:31:21.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.742+0000 7f11fe7fc700 1 -- 172.21.15.71:0/465754302 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f120c196360 con 0x7f11f806cce0 2024-04-23T02:31:21.745 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:31:21.745 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (15m) 8m ago 26m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:31:21.745 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (26m) 8m ago 26m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:31:21.745 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (25m) 8m ago 25m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:31:21.745 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (12m) 8m ago 26m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:31:21.745 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (12m) 8m ago 25m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:31:21.745 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (14m) 8m ago 26m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:31:21.745 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (9m) 8m ago 20m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:31:21.745 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (8m) 8m ago 20m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:31:21.745 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (20m) 8m ago 20m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:31:21.745 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (8m) 8m ago 20m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:31:21.745 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (16m) 8m ago 27m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:31:21.745 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (16m) 8m ago 25m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:31:21.745 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (13m) 8m ago 28m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:31:21.746 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (13m) 8m ago 25m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:31:21.746 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (15m) 8m ago 26m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:31:21.746 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (15m) 8m ago 25m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:31:21.746 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (12m) 8m ago 24m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:31:21.746 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (11m) 8m ago 23m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:31:21.746 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (11m) 8m ago 23m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:31:21.746 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (10m) 8m ago 22m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:31:21.746 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (10m) 8m ago 22m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:31:21.746 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (9m) 8m ago 21m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:31:21.746 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (15m) 8m ago 25m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:31:21.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.744+0000 7f12137f1700 1 -- 172.21.15.71:0/465754302 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f11f806cce0 msgr2=0x7f11f806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:21.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.744+0000 7f12137f1700 1 --2- 172.21.15.71:0/465754302 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f11f806cce0 0x7f11f806f1a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1200005950 tx=0x7f120000b410 comp rx=0 tx=0).stop 2024-04-23T02:31:21.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.744+0000 7f12137f1700 1 -- 172.21.15.71:0/465754302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f120c100750 msgr2=0x7f120c195a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:21.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.744+0000 7f12137f1700 1 --2- 172.21.15.71:0/465754302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f120c100750 0x7f120c195a30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f12080092e0 tx=0x7f120800bd70 comp rx=0 tx=0).stop 2024-04-23T02:31:21.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.744+0000 7f12137f1700 1 -- 172.21.15.71:0/465754302 shutdown_connections 2024-04-23T02:31:21.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.744+0000 7f12137f1700 1 --2- 172.21.15.71:0/465754302 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f11f806cce0 0x7f11f806f1a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:21.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.744+0000 7f12137f1700 1 --2- 172.21.15.71:0/465754302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f120c0ffda0 0x7f120c1954f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:21.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.744+0000 7f12137f1700 1 --2- 172.21.15.71:0/465754302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f120c100750 0x7f120c195a30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:21.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.744+0000 7f12137f1700 1 -- 172.21.15.71:0/465754302 >> 172.21.15.71:0/465754302 conn(0x7f120c0fb650 msgr2=0x7f120c0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:21.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.744+0000 7f12137f1700 1 -- 172.21.15.71:0/465754302 shutdown_connections 2024-04-23T02:31:21.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.744+0000 7f12137f1700 1 -- 172.21.15.71:0/465754302 wait complete. 2024-04-23T02:31:21.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.846+0000 7f92fdaa7700 1 -- 172.21.15.71:0/4014560574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f80ff9f0 msgr2=0x7f92f80ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:21.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.846+0000 7f92fdaa7700 1 --2- 172.21.15.71:0/4014560574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f80ff9f0 0x7f92f80ffe70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009910 tx=0x7f92e0009c20 comp rx=0 tx=0).stop 2024-04-23T02:31:21.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.846+0000 7f92fdaa7700 1 -- 172.21.15.71:0/4014560574 shutdown_connections 2024-04-23T02:31:21.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.846+0000 7f92fdaa7700 1 --2- 172.21.15.71:0/4014560574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f80ff9f0 0x7f92f80ffe70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:21.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.846+0000 7f92fdaa7700 1 --2- 172.21.15.71:0/4014560574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92f81059f0 0x7f92f8105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:21.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.846+0000 7f92fdaa7700 1 -- 172.21.15.71:0/4014560574 >> 172.21.15.71:0/4014560574 conn(0x7f92f80fb4e0 msgr2=0x7f92f80fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:21.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.846+0000 7f92fdaa7700 1 -- 172.21.15.71:0/4014560574 shutdown_connections 2024-04-23T02:31:21.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.846+0000 7f92fdaa7700 1 -- 172.21.15.71:0/4014560574 wait complete. 2024-04-23T02:31:21.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.847+0000 7f92fdaa7700 1 Processor -- start 2024-04-23T02:31:21.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.847+0000 7f92fdaa7700 1 -- start start 2024-04-23T02:31:21.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.847+0000 7f92fdaa7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92f80ff9f0 0x7f92f80fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:21.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.847+0000 7f92fdaa7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f81059f0 0x7f92f80fc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:21.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.847+0000 7f92fdaa7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f80fc8d0 con 0x7f92f81059f0 2024-04-23T02:31:21.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.847+0000 7f92fdaa7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f80fca10 con 0x7f92f80ff9f0 2024-04-23T02:31:21.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.847+0000 7f92f6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f81059f0 0x7f92f80fc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.847+0000 7f92f6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f81059f0 0x7f92f80fc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42764/0 (socket says 172.21.15.71:42764) 2024-04-23T02:31:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.847+0000 7f92f6ffd700 1 -- 172.21.15.71:0/2027020465 learned_addr learned my addr 172.21.15.71:0/2027020465 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:31:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.847+0000 7f92f77fe700 1 --2- 172.21.15.71:0/2027020465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92f80ff9f0 0x7f92f80fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.847+0000 7f92f6ffd700 1 -- 172.21.15.71:0/2027020465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92f80ff9f0 msgr2=0x7f92f80fbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.847+0000 7f92f6ffd700 1 --2- 172.21.15.71:0/2027020465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92f80ff9f0 0x7f92f80fbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.847+0000 7f92f6ffd700 1 -- 172.21.15.71:0/2027020465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e00095f0 con 0x7f92f81059f0 2024-04-23T02:31:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.847+0000 7f92f6ffd700 1 --2- 172.21.15.71:0/2027020465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f81059f0 0x7f92f80fc390 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f92e0000c00 tx=0x7f92e000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.848+0000 7f92f4ff9700 1 -- 172.21.15.71:0/2027020465 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92e001d070 con 0x7f92f81059f0 2024-04-23T02:31:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.848+0000 7f92fdaa7700 1 -- 172.21.15.71:0/2027020465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f80fcc90 con 0x7f92f81059f0 2024-04-23T02:31:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.848+0000 7f92f4ff9700 1 -- 172.21.15.71:0/2027020465 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f92e0005dc0 con 0x7f92f81059f0 2024-04-23T02:31:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.848+0000 7f92f4ff9700 1 -- 172.21.15.71:0/2027020465 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92e0003940 con 0x7f92f81059f0 2024-04-23T02:31:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.848+0000 7f92fdaa7700 1 -- 172.21.15.71:0/2027020465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f81a2a90 con 0x7f92f81059f0 2024-04-23T02:31:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.849+0000 7f92fdaa7700 1 -- 172.21.15.71:0/2027020465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f8108f90 con 0x7f92f81059f0 2024-04-23T02:31:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.849+0000 7f92f4ff9700 1 -- 172.21.15.71:0/2027020465 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f92e0004ae0 con 0x7f92f81059f0 2024-04-23T02:31:21.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.850+0000 7f92f4ff9700 1 --2- 172.21.15.71:0/2027020465 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f92e406cce0 0x7f92e406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:21.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.850+0000 7f92f77fe700 1 --2- 172.21.15.71:0/2027020465 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f92e406cce0 0x7f92e406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:21.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.850+0000 7f92f4ff9700 1 -- 172.21.15.71:0/2027020465 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f92e008db70 con 0x7f92f81059f0 2024-04-23T02:31:21.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.850+0000 7f92f77fe700 1 --2- 172.21.15.71:0/2027020465 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f92e406cce0 0x7f92e406f1a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009c60 tx=0x7f92e8009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:21.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:21.854+0000 7f92f4ff9700 1 -- 172.21.15.71:0/2027020465 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f92e0057be0 con 0x7f92f81059f0 2024-04-23T02:31:22.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.126+0000 7f92fdaa7700 1 -- 172.21.15.71:0/2027020465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f92f804f2e0 con 0x7f92f81059f0 2024-04-23T02:31:22.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.126+0000 7f92f4ff9700 1 -- 172.21.15.71:0/2027020465 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f92e0003b30 con 0x7f92f81059f0 2024-04-23T02:31:22.129 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:31:22.129 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:31:22.129 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:31:22.129 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:31:22.129 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:31:22.129 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:31:22.129 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:31:22.129 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:31:22.129 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:31:22.129 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:31:22.129 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:31:22.130 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:31:22.130 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:31:22.130 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:31:22.130 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:31:22.130 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:31:22.130 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:31:22.130 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:31:22.130 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:31:22.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.128+0000 7f92fdaa7700 1 -- 172.21.15.71:0/2027020465 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f92e406cce0 msgr2=0x7f92e406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:22.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.128+0000 7f92fdaa7700 1 --2- 172.21.15.71:0/2027020465 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f92e406cce0 0x7f92e406f1a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009c60 tx=0x7f92e8009260 comp rx=0 tx=0).stop 2024-04-23T02:31:22.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.128+0000 7f92fdaa7700 1 -- 172.21.15.71:0/2027020465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f81059f0 msgr2=0x7f92f80fc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:22.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.128+0000 7f92fdaa7700 1 --2- 172.21.15.71:0/2027020465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f81059f0 0x7f92f80fc390 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f92e0000c00 tx=0x7f92e000bde0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.128+0000 7f92fdaa7700 1 -- 172.21.15.71:0/2027020465 shutdown_connections 2024-04-23T02:31:22.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.128+0000 7f92fdaa7700 1 --2- 172.21.15.71:0/2027020465 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f92e406cce0 0x7f92e406f1a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.128+0000 7f92fdaa7700 1 --2- 172.21.15.71:0/2027020465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92f80ff9f0 0x7f92f80fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.128+0000 7f92fdaa7700 1 --2- 172.21.15.71:0/2027020465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92f81059f0 0x7f92f80fc390 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.128+0000 7f92fdaa7700 1 -- 172.21.15.71:0/2027020465 >> 172.21.15.71:0/2027020465 conn(0x7f92f80fb4e0 msgr2=0x7f92f80690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:22.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.128+0000 7f92fdaa7700 1 -- 172.21.15.71:0/2027020465 shutdown_connections 2024-04-23T02:31:22.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.128+0000 7f92fdaa7700 1 -- 172.21.15.71:0/2027020465 wait complete. 2024-04-23T02:31:22.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.230+0000 7f3d6ef12700 1 -- 172.21.15.71:0/3429875998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d680ffda0 msgr2=0x7f3d68100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:22.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.230+0000 7f3d6ef12700 1 --2- 172.21.15.71:0/3429875998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d680ffda0 0x7f3d68100180 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009910 tx=0x7f3d5c009c20 comp rx=0 tx=0).stop 2024-04-23T02:31:22.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.230+0000 7f3d6ef12700 1 -- 172.21.15.71:0/3429875998 shutdown_connections 2024-04-23T02:31:22.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.230+0000 7f3d6ef12700 1 --2- 172.21.15.71:0/3429875998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d68100750 0x7f3d681047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.230+0000 7f3d6ef12700 1 --2- 172.21.15.71:0/3429875998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d680ffda0 0x7f3d68100180 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.230+0000 7f3d6ef12700 1 -- 172.21.15.71:0/3429875998 >> 172.21.15.71:0/3429875998 conn(0x7f3d680fb650 msgr2=0x7f3d680fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:22.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.230+0000 7f3d6ef12700 1 -- 172.21.15.71:0/3429875998 shutdown_connections 2024-04-23T02:31:22.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.230+0000 7f3d6ef12700 1 -- 172.21.15.71:0/3429875998 wait complete. 2024-04-23T02:31:22.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.230+0000 7f3d6ef12700 1 Processor -- start 2024-04-23T02:31:22.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.230+0000 7f3d6ef12700 1 -- start start 2024-04-23T02:31:22.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.230+0000 7f3d6ef12700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d680ffda0 0x7f3d68197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:22.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.230+0000 7f3d6ef12700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d68100750 0x7f3d68197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:22.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.230+0000 7f3d6ef12700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d681981a0 con 0x7f3d68100750 2024-04-23T02:31:22.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.230+0000 7f3d6ef12700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d681982e0 con 0x7f3d680ffda0 2024-04-23T02:31:22.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.231+0000 7f3d67fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d68100750 0x7f3d68197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:22.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.231+0000 7f3d67fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d68100750 0x7f3d68197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42782/0 (socket says 172.21.15.71:42782) 2024-04-23T02:31:22.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.231+0000 7f3d67fff700 1 -- 172.21.15.71:0/1121985327 learned_addr learned my addr 172.21.15.71:0/1121985327 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:31:22.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.231+0000 7f3d67fff700 1 -- 172.21.15.71:0/1121985327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d680ffda0 msgr2=0x7f3d68197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:22.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.231+0000 7f3d6ccae700 1 --2- 172.21.15.71:0/1121985327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d680ffda0 0x7f3d68197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:22.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.231+0000 7f3d67fff700 1 --2- 172.21.15.71:0/1121985327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d680ffda0 0x7f3d68197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.231+0000 7f3d67fff700 1 -- 172.21.15.71:0/1121985327 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c0095f0 con 0x7f3d68100750 2024-04-23T02:31:22.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.231+0000 7f3d67fff700 1 --2- 172.21.15.71:0/1121985327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d68100750 0x7f3d68197b80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400c8f0 tx=0x7f3d5400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:22.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.231+0000 7f3d65ffb700 1 -- 172.21.15.71:0/1121985327 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d54007a50 con 0x7f3d68100750 2024-04-23T02:31:22.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.231+0000 7f3d65ffb700 1 -- 172.21.15.71:0/1121985327 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3d54007bb0 con 0x7f3d68100750 2024-04-23T02:31:22.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.231+0000 7f3d6ef12700 1 -- 172.21.15.71:0/1121985327 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d681a06b0 con 0x7f3d68100750 2024-04-23T02:31:22.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.231+0000 7f3d65ffb700 1 -- 172.21.15.71:0/1121985327 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d54018630 con 0x7f3d68100750 2024-04-23T02:31:22.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.231+0000 7f3d6ef12700 1 -- 172.21.15.71:0/1121985327 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d681a0b70 con 0x7f3d68100750 2024-04-23T02:31:22.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.232+0000 7f3d6ef12700 1 -- 172.21.15.71:0/1121985327 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d68108140 con 0x7f3d68100750 2024-04-23T02:31:22.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.233+0000 7f3d65ffb700 1 -- 172.21.15.71:0/1121985327 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3d54007d20 con 0x7f3d68100750 2024-04-23T02:31:22.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.233+0000 7f3d65ffb700 1 --2- 172.21.15.71:0/1121985327 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3d5806cce0 0x7f3d5806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:22.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.233+0000 7f3d65ffb700 1 -- 172.21.15.71:0/1121985327 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3d5408c590 con 0x7f3d68100750 2024-04-23T02:31:22.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.233+0000 7f3d6ccae700 1 --2- 172.21.15.71:0/1121985327 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3d5806cce0 0x7f3d5806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:22.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.234+0000 7f3d6ccae700 1 --2- 172.21.15.71:0/1121985327 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3d5806cce0 0x7f3d5806f1a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00b5c0 tx=0x7f3d5c005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:22.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.238+0000 7f3d65ffb700 1 -- 172.21.15.71:0/1121985327 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3d54056600 con 0x7f3d68100750 2024-04-23T02:31:22.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.481+0000 7f3d6ef12700 1 -- 172.21.15.71:0/1121985327 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3d6804f2e0 con 0x7f3d68100750 2024-04-23T02:31:22.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.481+0000 7f3d65ffb700 1 -- 172.21.15.71:0/1121985327 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f3d54059c20 con 0x7f3d68100750 2024-04-23T02:31:22.484 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:31:22.484 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:31:22.484 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:31:22.484 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:31:22.484 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:31:22.484 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:31:22.484 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:31:22.484 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:31:22.484 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:31:22.484 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:31:22.484 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:31:22.484 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:31:22.485 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:31:22.485 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:31:22.485 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:31:22.485 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:31:22.485 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:31:22.485 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:31:22.485 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:31:22.485 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:31:22.485 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:31:22.485 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:31:22.486 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:31:22.486 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:31:22.486 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:31:22.486 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:31:22.486 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:31:22.486 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:31:22.486 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:31:22.486 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:31:22.486 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:31:22.486 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:31:22.486 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:31:22.486 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:31:22.486 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:31:22.486 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:31:22.487 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:31:22.487 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:31:22.487 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:31:22.487 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:31:22.487 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:31:22.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.483+0000 7f3d6ef12700 1 -- 172.21.15.71:0/1121985327 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3d5806cce0 msgr2=0x7f3d5806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:22.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.483+0000 7f3d6ef12700 1 --2- 172.21.15.71:0/1121985327 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3d5806cce0 0x7f3d5806f1a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00b5c0 tx=0x7f3d5c005fb0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.483+0000 7f3d6ef12700 1 -- 172.21.15.71:0/1121985327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d68100750 msgr2=0x7f3d68197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:22.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.483+0000 7f3d6ef12700 1 --2- 172.21.15.71:0/1121985327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d68100750 0x7f3d68197b80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400c8f0 tx=0x7f3d5400ccb0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.483+0000 7f3d6ef12700 1 -- 172.21.15.71:0/1121985327 shutdown_connections 2024-04-23T02:31:22.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.483+0000 7f3d6ef12700 1 --2- 172.21.15.71:0/1121985327 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3d5806cce0 0x7f3d5806f1a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.483+0000 7f3d6ef12700 1 --2- 172.21.15.71:0/1121985327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d680ffda0 0x7f3d68197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.483+0000 7f3d6ef12700 1 --2- 172.21.15.71:0/1121985327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d68100750 0x7f3d68197b80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.483+0000 7f3d6ef12700 1 -- 172.21.15.71:0/1121985327 >> 172.21.15.71:0/1121985327 conn(0x7f3d680fb650 msgr2=0x7f3d680fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:22.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.483+0000 7f3d6ef12700 1 -- 172.21.15.71:0/1121985327 shutdown_connections 2024-04-23T02:31:22.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.483+0000 7f3d6ef12700 1 -- 172.21.15.71:0/1121985327 wait complete. 2024-04-23T02:31:22.489 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:31:22.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.587+0000 7f0a3b62e700 1 -- 172.21.15.71:0/460751565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a34068580 msgr2=0x7f0a340689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:22.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.587+0000 7f0a3b62e700 1 --2- 172.21.15.71:0/460751565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a34068580 0x7f0a340689e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009910 tx=0x7f0a24009c20 comp rx=0 tx=0).stop 2024-04-23T02:31:22.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.588+0000 7f0a3b62e700 1 -- 172.21.15.71:0/460751565 shutdown_connections 2024-04-23T02:31:22.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.588+0000 7f0a3b62e700 1 --2- 172.21.15.71:0/460751565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a34068580 0x7f0a340689e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.588+0000 7f0a3b62e700 1 --2- 172.21.15.71:0/460751565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a34106cd0 0x7f0a341070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.588+0000 7f0a3b62e700 1 -- 172.21.15.71:0/460751565 >> 172.21.15.71:0/460751565 conn(0x7f0a34075670 msgr2=0x7f0a34075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:22.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.588+0000 7f0a3b62e700 1 -- 172.21.15.71:0/460751565 shutdown_connections 2024-04-23T02:31:22.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.588+0000 7f0a3b62e700 1 -- 172.21.15.71:0/460751565 wait complete. 2024-04-23T02:31:22.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.588+0000 7f0a3b62e700 1 Processor -- start 2024-04-23T02:31:22.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.589+0000 7f0a3b62e700 1 -- start start 2024-04-23T02:31:22.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.589+0000 7f0a3b62e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a34068580 0x7f0a3419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:22.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.589+0000 7f0a3b62e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a34106cd0 0x7f0a3419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:22.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.589+0000 7f0a3b62e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3419d110 con 0x7f0a34068580 2024-04-23T02:31:22.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.589+0000 7f0a3b62e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a341965c0 con 0x7f0a34106cd0 2024-04-23T02:31:22.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.589+0000 7f0a393ca700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a34068580 0x7f0a3419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:22.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.589+0000 7f0a393ca700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a34068580 0x7f0a3419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42806/0 (socket says 172.21.15.71:42806) 2024-04-23T02:31:22.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.589+0000 7f0a393ca700 1 -- 172.21.15.71:0/2459333253 learned_addr learned my addr 172.21.15.71:0/2459333253 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:31:22.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.589+0000 7f0a38bc9700 1 --2- 172.21.15.71:0/2459333253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a34106cd0 0x7f0a3419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:22.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.589+0000 7f0a393ca700 1 -- 172.21.15.71:0/2459333253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a34106cd0 msgr2=0x7f0a3419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:22.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.589+0000 7f0a393ca700 1 --2- 172.21.15.71:0/2459333253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a34106cd0 0x7f0a3419ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.589+0000 7f0a393ca700 1 -- 172.21.15.71:0/2459333253 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a240095f0 con 0x7f0a34068580 2024-04-23T02:31:22.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.589+0000 7f0a38bc9700 1 --2- 172.21.15.71:0/2459333253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a34106cd0 0x7f0a3419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:31:22.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.589+0000 7f0a393ca700 1 --2- 172.21.15.71:0/2459333253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a34068580 0x7f0a3419c540 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000d8d0 tx=0x7f0a3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:22.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.589+0000 7f0a2a7fc700 1 -- 172.21.15.71:0/2459333253 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a30009940 con 0x7f0a34068580 2024-04-23T02:31:22.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.589+0000 7f0a2a7fc700 1 -- 172.21.15.71:0/2459333253 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0a30010460 con 0x7f0a34068580 2024-04-23T02:31:22.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.590+0000 7f0a3b62e700 1 -- 172.21.15.71:0/2459333253 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a341968a0 con 0x7f0a34068580 2024-04-23T02:31:22.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.590+0000 7f0a2a7fc700 1 -- 172.21.15.71:0/2459333253 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a3000f5f0 con 0x7f0a34068580 2024-04-23T02:31:22.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.590+0000 7f0a3b62e700 1 -- 172.21.15.71:0/2459333253 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a34196df0 con 0x7f0a34068580 2024-04-23T02:31:22.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.591+0000 7f0a3b62e700 1 -- 172.21.15.71:0/2459333253 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a341091d0 con 0x7f0a34068580 2024-04-23T02:31:22.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.591+0000 7f0a2a7fc700 1 -- 172.21.15.71:0/2459333253 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0a300105d0 con 0x7f0a34068580 2024-04-23T02:31:22.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.591+0000 7f0a2a7fc700 1 --2- 172.21.15.71:0/2459333253 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0a2006cce0 0x7f0a2006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:22.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.591+0000 7f0a2a7fc700 1 -- 172.21.15.71:0/2459333253 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0a3008c4f0 con 0x7f0a34068580 2024-04-23T02:31:22.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.591+0000 7f0a38bc9700 1 --2- 172.21.15.71:0/2459333253 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0a2006cce0 0x7f0a2006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:22.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.592+0000 7f0a38bc9700 1 --2- 172.21.15.71:0/2459333253 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0a2006cce0 0x7f0a2006f1a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0a24005b40 tx=0x7f0a2401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:22.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.596+0000 7f0a2a7fc700 1 -- 172.21.15.71:0/2459333253 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0a3005a070 con 0x7f0a34068580 2024-04-23T02:31:22.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:22 smithi071 ceph-mon[144544]: pgmap v541: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:22.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:22 smithi071 ceph-mon[144544]: from='client.34818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:31:22.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:22 smithi071 ceph-mon[144544]: from='client.34822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:31:22.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:22 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2027020465' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:31:22.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.807+0000 7f0a3b62e700 1 -- 172.21.15.71:0/2459333253 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a34197060 con 0x7f0a2006cce0 2024-04-23T02:31:22.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.808+0000 7f0a2a7fc700 1 -- 172.21.15.71:0/2459333253 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0a34197060 con 0x7f0a2006cce0 2024-04-23T02:31:22.810 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:31:22.810 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:31:22.810 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:31:22.811 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:31:22.811 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:31:22.811 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:31:22.811 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:31:22.811 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:31:22.811 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:31:22.811 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:31:22.811 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:31:22.811 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:31:22.811 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:31:22.811 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:31:22.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.810+0000 7f0a3b62e700 1 -- 172.21.15.71:0/2459333253 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0a2006cce0 msgr2=0x7f0a2006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:22.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.810+0000 7f0a3b62e700 1 --2- 172.21.15.71:0/2459333253 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0a2006cce0 0x7f0a2006f1a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0a24005b40 tx=0x7f0a2401a040 comp rx=0 tx=0).stop 2024-04-23T02:31:22.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.810+0000 7f0a3b62e700 1 -- 172.21.15.71:0/2459333253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a34068580 msgr2=0x7f0a3419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:22.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.810+0000 7f0a3b62e700 1 --2- 172.21.15.71:0/2459333253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a34068580 0x7f0a3419c540 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000d8d0 tx=0x7f0a3000dc90 comp rx=0 tx=0).stop 2024-04-23T02:31:22.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.810+0000 7f0a3b62e700 1 -- 172.21.15.71:0/2459333253 shutdown_connections 2024-04-23T02:31:22.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.810+0000 7f0a3b62e700 1 --2- 172.21.15.71:0/2459333253 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0a2006cce0 0x7f0a2006f1a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.810+0000 7f0a3b62e700 1 --2- 172.21.15.71:0/2459333253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a34068580 0x7f0a3419c540 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.810+0000 7f0a3b62e700 1 --2- 172.21.15.71:0/2459333253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a34106cd0 0x7f0a3419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.810+0000 7f0a3b62e700 1 -- 172.21.15.71:0/2459333253 >> 172.21.15.71:0/2459333253 conn(0x7f0a34075670 msgr2=0x7f0a340fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:22.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.810+0000 7f0a3b62e700 1 -- 172.21.15.71:0/2459333253 shutdown_connections 2024-04-23T02:31:22.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.810+0000 7f0a3b62e700 1 -- 172.21.15.71:0/2459333253 wait complete. 2024-04-23T02:31:22.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:22 smithi192 ceph-mon[108617]: pgmap v541: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:22.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:22 smithi192 ceph-mon[108617]: from='client.34818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:31:22.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:22 smithi192 ceph-mon[108617]: from='client.34822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:31:22.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:22 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2027020465' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:31:22.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.917+0000 7f82b4677700 1 -- 172.21.15.71:0/3306452780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82ac100750 msgr2=0x7f82ac1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:22.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.917+0000 7f82b4677700 1 --2- 172.21.15.71:0/3306452780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82ac100750 0x7f82ac1047a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009910 tx=0x7f82a8009c20 comp rx=0 tx=0).stop 2024-04-23T02:31:22.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.917+0000 7f82b4677700 1 -- 172.21.15.71:0/3306452780 shutdown_connections 2024-04-23T02:31:22.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.917+0000 7f82b4677700 1 --2- 172.21.15.71:0/3306452780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82ac100750 0x7f82ac1047a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.917+0000 7f82b4677700 1 --2- 172.21.15.71:0/3306452780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ac0ffda0 0x7f82ac100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.917+0000 7f82b4677700 1 -- 172.21.15.71:0/3306452780 >> 172.21.15.71:0/3306452780 conn(0x7f82ac0fb650 msgr2=0x7f82ac0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:22.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.917+0000 7f82b4677700 1 -- 172.21.15.71:0/3306452780 shutdown_connections 2024-04-23T02:31:22.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.917+0000 7f82b4677700 1 -- 172.21.15.71:0/3306452780 wait complete. 2024-04-23T02:31:22.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.917+0000 7f82b4677700 1 Processor -- start 2024-04-23T02:31:22.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.917+0000 7f82b4677700 1 -- start start 2024-04-23T02:31:22.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.918+0000 7f82b4677700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ac0ffda0 0x7f82ac19c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:22.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.918+0000 7f82b4677700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82ac100750 0x7f82ac19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:22.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.918+0000 7f82b4677700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac19d1a0 con 0x7f82ac100750 2024-04-23T02:31:22.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.918+0000 7f82b4677700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac196650 con 0x7f82ac0ffda0 2024-04-23T02:31:22.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.918+0000 7f82b1c12700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82ac100750 0x7f82ac19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:22.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.918+0000 7f82b1c12700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82ac100750 0x7f82ac19cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42816/0 (socket says 172.21.15.71:42816) 2024-04-23T02:31:22.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.918+0000 7f82b1c12700 1 -- 172.21.15.71:0/2383107141 learned_addr learned my addr 172.21.15.71:0/2383107141 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:31:22.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.918+0000 7f82b2413700 1 --2- 172.21.15.71:0/2383107141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ac0ffda0 0x7f82ac19c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:22.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.918+0000 7f82b1c12700 1 -- 172.21.15.71:0/2383107141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ac0ffda0 msgr2=0x7f82ac19c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:22.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.918+0000 7f82b1c12700 1 --2- 172.21.15.71:0/2383107141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ac0ffda0 0x7f82ac19c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:22.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.918+0000 7f82b1c12700 1 -- 172.21.15.71:0/2383107141 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a80095f0 con 0x7f82ac100750 2024-04-23T02:31:22.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.918+0000 7f82b2413700 1 --2- 172.21.15.71:0/2383107141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ac0ffda0 0x7f82ac19c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:31:22.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.918+0000 7f82b1c12700 1 --2- 172.21.15.71:0/2383107141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82ac100750 0x7f82ac19cb10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f82a80092e0 tx=0x7f82a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:22.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.918+0000 7f82a37fe700 1 -- 172.21.15.71:0/2383107141 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82a801d070 con 0x7f82ac100750 2024-04-23T02:31:22.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.918+0000 7f82b4677700 1 -- 172.21.15.71:0/2383107141 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ac1968d0 con 0x7f82ac100750 2024-04-23T02:31:22.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.918+0000 7f82a37fe700 1 -- 172.21.15.71:0/2383107141 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f82a8005940 con 0x7f82ac100750 2024-04-23T02:31:22.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.918+0000 7f82a37fe700 1 -- 172.21.15.71:0/2383107141 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82a800fe50 con 0x7f82ac100750 2024-04-23T02:31:22.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.918+0000 7f82b4677700 1 -- 172.21.15.71:0/2383107141 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82ac196dc0 con 0x7f82ac100750 2024-04-23T02:31:22.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.919+0000 7f82b4677700 1 -- 172.21.15.71:0/2383107141 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82ac04f2e0 con 0x7f82ac100750 2024-04-23T02:31:22.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.920+0000 7f82a37fe700 1 -- 172.21.15.71:0/2383107141 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f82a8003a60 con 0x7f82ac100750 2024-04-23T02:31:22.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.920+0000 7f82a37fe700 1 --2- 172.21.15.71:0/2383107141 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f829806cce0 0x7f829806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:22.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.920+0000 7f82a37fe700 1 -- 172.21.15.71:0/2383107141 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f82a808d7e0 con 0x7f82ac100750 2024-04-23T02:31:22.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.920+0000 7f82b2413700 1 --2- 172.21.15.71:0/2383107141 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f829806cce0 0x7f829806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:22.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.921+0000 7f82b2413700 1 --2- 172.21.15.71:0/2383107141 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f829806cce0 0x7f829806f1a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f829c005fd0 tx=0x7f829c005ee0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:22.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:22.925+0000 7f82a37fe700 1 -- 172.21.15.71:0/2383107141 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f82a805b2e0 con 0x7f82ac100750 2024-04-23T02:31:23.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:23.197+0000 7f82b4677700 1 -- 172.21.15.71:0/2383107141 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f82ac196fe0 con 0x7f82ac100750 2024-04-23T02:31:23.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:23.197+0000 7f82a37fe700 1 -- 172.21.15.71:0/2383107141 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f82a8003d10 con 0x7f82ac100750 2024-04-23T02:31:23.201 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:31:23.201 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:31:23.201 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:31:23.201 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:31:23.201 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:31:23.202 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:31:23.202 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:31:23.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:23.199+0000 7f82b4677700 1 -- 172.21.15.71:0/2383107141 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f829806cce0 msgr2=0x7f829806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:23.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:23.199+0000 7f82b4677700 1 --2- 172.21.15.71:0/2383107141 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f829806cce0 0x7f829806f1a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f829c005fd0 tx=0x7f829c005ee0 comp rx=0 tx=0).stop 2024-04-23T02:31:23.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:23.199+0000 7f82b4677700 1 -- 172.21.15.71:0/2383107141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82ac100750 msgr2=0x7f82ac19cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:23.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:23.199+0000 7f82b4677700 1 --2- 172.21.15.71:0/2383107141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82ac100750 0x7f82ac19cb10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f82a80092e0 tx=0x7f82a800be20 comp rx=0 tx=0).stop 2024-04-23T02:31:23.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:23.199+0000 7f82b4677700 1 -- 172.21.15.71:0/2383107141 shutdown_connections 2024-04-23T02:31:23.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:23.199+0000 7f82b4677700 1 --2- 172.21.15.71:0/2383107141 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f829806cce0 0x7f829806f1a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:23.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:23.199+0000 7f82b4677700 1 --2- 172.21.15.71:0/2383107141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ac0ffda0 0x7f82ac19c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:23.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:23.199+0000 7f82b4677700 1 --2- 172.21.15.71:0/2383107141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82ac100750 0x7f82ac19cb10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:23.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:23.199+0000 7f82b4677700 1 -- 172.21.15.71:0/2383107141 >> 172.21.15.71:0/2383107141 conn(0x7f82ac0fb650 msgr2=0x7f82ac0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:23.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:23.199+0000 7f82b4677700 1 -- 172.21.15.71:0/2383107141 shutdown_connections 2024-04-23T02:31:23.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:23.200+0000 7f82b4677700 1 -- 172.21.15.71:0/2383107141 wait complete. 2024-04-23T02:31:23.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:23 smithi071 ceph-mon[144544]: from='client.34826 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:31:23.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:23 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1121985327' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:31:23.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:23 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2383107141' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:31:23.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:23 smithi192 ceph-mon[108617]: from='client.34826 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:31:23.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:23 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1121985327' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:31:23.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:23 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2383107141' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:31:24.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:24 smithi071 ceph-mon[144544]: from='client.34838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:31:24.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:24 smithi071 ceph-mon[144544]: pgmap v542: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:24.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:24 smithi192 ceph-mon[108617]: from='client.34838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:31:24.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:24 smithi192 ceph-mon[108617]: pgmap v542: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:26.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:26 smithi071 ceph-mon[144544]: pgmap v543: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:26.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:26 smithi192 ceph-mon[108617]: pgmap v543: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:28.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:28 smithi071 ceph-mon[144544]: pgmap v544: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:28.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:28 smithi192 ceph-mon[108617]: pgmap v544: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:30.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:30 smithi071 ceph-mon[144544]: pgmap v545: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:30.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:30 smithi192 ceph-mon[108617]: pgmap v545: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:32.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:32 smithi071 ceph-mon[144544]: pgmap v546: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:32.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:32 smithi192 ceph-mon[108617]: pgmap v546: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:34.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:34 smithi071 ceph-mon[144544]: pgmap v547: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:34.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:34 smithi192 ceph-mon[108617]: pgmap v547: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:35.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:31:35.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:31:36.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:36 smithi071 ceph-mon[144544]: pgmap v548: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:36.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:36 smithi192 ceph-mon[108617]: pgmap v548: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:38.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:38 smithi071 ceph-mon[144544]: pgmap v549: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:38.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:38 smithi192 ceph-mon[108617]: pgmap v549: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:40.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:40 smithi071 ceph-mon[144544]: pgmap v550: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:40.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:40 smithi192 ceph-mon[108617]: pgmap v550: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:42.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:42 smithi071 ceph-mon[144544]: pgmap v551: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:42.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:42 smithi192 ceph-mon[108617]: pgmap v551: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:44.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:44 smithi071 ceph-mon[144544]: pgmap v552: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:44.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:44 smithi192 ceph-mon[108617]: pgmap v552: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:46.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:46 smithi071 ceph-mon[144544]: pgmap v553: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:46.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:46 smithi192 ceph-mon[108617]: pgmap v553: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:48.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:48 smithi071 ceph-mon[144544]: pgmap v554: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:48.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:48 smithi192 ceph-mon[108617]: pgmap v554: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:50.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:50 smithi071 ceph-mon[144544]: pgmap v555: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:50.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:31:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:50 smithi192 ceph-mon[108617]: pgmap v555: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:31:52.729 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:52 smithi071 ceph-mon[144544]: pgmap v556: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:52.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:52 smithi192 ceph-mon[108617]: pgmap v556: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:53.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.315+0000 7fd6f1b1f700 1 -- 172.21.15.71:0/1018917127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6ec101840 msgr2=0x7fd6ec105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:53.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.315+0000 7fd6f1b1f700 1 --2- 172.21.15.71:0/1018917127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6ec101840 0x7fd6ec105890 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009910 tx=0x7fd6dc009c20 comp rx=0 tx=0).stop 2024-04-23T02:31:53.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.315+0000 7fd6f1b1f700 1 -- 172.21.15.71:0/1018917127 shutdown_connections 2024-04-23T02:31:53.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.315+0000 7fd6f1b1f700 1 --2- 172.21.15.71:0/1018917127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6ec101840 0x7fd6ec105890 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:53.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.315+0000 7fd6f1b1f700 1 --2- 172.21.15.71:0/1018917127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6ec100e90 0x7fd6ec101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:53.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.315+0000 7fd6f1b1f700 1 -- 172.21.15.71:0/1018917127 >> 172.21.15.71:0/1018917127 conn(0x7fd6ec0fc740 msgr2=0x7fd6ec0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:53.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.315+0000 7fd6f1b1f700 1 -- 172.21.15.71:0/1018917127 shutdown_connections 2024-04-23T02:31:53.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.315+0000 7fd6f1b1f700 1 -- 172.21.15.71:0/1018917127 wait complete. 2024-04-23T02:31:53.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.315+0000 7fd6f1b1f700 1 Processor -- start 2024-04-23T02:31:53.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.316+0000 7fd6f1b1f700 1 -- start start 2024-04-23T02:31:53.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.316+0000 7fd6f1b1f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6ec100e90 0x7fd6ec073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:53.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.316+0000 7fd6f1b1f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6ec101840 0x7fd6ec0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:53.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.316+0000 7fd6f1b1f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec073b10 con 0x7fd6ec100e90 2024-04-23T02:31:53.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.316+0000 7fd6f1b1f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec073c50 con 0x7fd6ec101840 2024-04-23T02:31:53.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.316+0000 7fd6eb7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6ec100e90 0x7fd6ec073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:53.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.316+0000 7fd6eb7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6ec100e90 0x7fd6ec073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50192/0 (socket says 172.21.15.71:50192) 2024-04-23T02:31:53.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.316+0000 7fd6eb7fe700 1 -- 172.21.15.71:0/3180035380 learned_addr learned my addr 172.21.15.71:0/3180035380 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:31:53.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.316+0000 7fd6eaffd700 1 --2- 172.21.15.71:0/3180035380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6ec101840 0x7fd6ec0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:53.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.316+0000 7fd6eb7fe700 1 -- 172.21.15.71:0/3180035380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6ec101840 msgr2=0x7fd6ec0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:53.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.316+0000 7fd6eb7fe700 1 --2- 172.21.15.71:0/3180035380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6ec101840 0x7fd6ec0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:53.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.316+0000 7fd6eb7fe700 1 -- 172.21.15.71:0/3180035380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc0095f0 con 0x7fd6ec100e90 2024-04-23T02:31:53.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.316+0000 7fd6eaffd700 1 --2- 172.21.15.71:0/3180035380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6ec101840 0x7fd6ec0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:31:53.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.316+0000 7fd6eb7fe700 1 --2- 172.21.15.71:0/3180035380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6ec100e90 0x7fd6ec073090 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000d8d0 tx=0x7fd6e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:53.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.317+0000 7fd6e8ff9700 1 -- 172.21.15.71:0/3180035380 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6e0009940 con 0x7fd6ec100e90 2024-04-23T02:31:53.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.317+0000 7fd6e8ff9700 1 -- 172.21.15.71:0/3180035380 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd6e0010460 con 0x7fd6ec100e90 2024-04-23T02:31:53.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.317+0000 7fd6f1b1f700 1 -- 172.21.15.71:0/3180035380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ec1a2a30 con 0x7fd6ec100e90 2024-04-23T02:31:53.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.317+0000 7fd6e8ff9700 1 -- 172.21.15.71:0/3180035380 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6e000f5f0 con 0x7fd6ec100e90 2024-04-23T02:31:53.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.317+0000 7fd6f1b1f700 1 -- 172.21.15.71:0/3180035380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ec1a2e40 con 0x7fd6ec100e90 2024-04-23T02:31:53.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.317+0000 7fd6f1b1f700 1 -- 172.21.15.71:0/3180035380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ec109290 con 0x7fd6ec100e90 2024-04-23T02:31:53.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.318+0000 7fd6e8ff9700 1 -- 172.21.15.71:0/3180035380 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd6e00105d0 con 0x7fd6ec100e90 2024-04-23T02:31:53.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.318+0000 7fd6e8ff9700 1 --2- 172.21.15.71:0/3180035380 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6d406cce0 0x7fd6d406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:53.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.318+0000 7fd6e8ff9700 1 -- 172.21.15.71:0/3180035380 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd6e008c5e0 con 0x7fd6ec100e90 2024-04-23T02:31:53.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.319+0000 7fd6eaffd700 1 --2- 172.21.15.71:0/3180035380 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6d406cce0 0x7fd6d406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:53.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.319+0000 7fd6eaffd700 1 --2- 172.21.15.71:0/3180035380 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6d406cce0 0x7fd6d406f1a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec074880 tx=0x7fd6dc01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:53.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.323+0000 7fd6e8ff9700 1 -- 172.21.15.71:0/3180035380 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd6e0056650 con 0x7fd6ec100e90 2024-04-23T02:31:53.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.534+0000 7fd6f1b1f700 1 -- 172.21.15.71:0/3180035380 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6ec0611d0 con 0x7fd6d406cce0 2024-04-23T02:31:53.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.535+0000 7fd6e8ff9700 1 -- 172.21.15.71:0/3180035380 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd6ec0611d0 con 0x7fd6d406cce0 2024-04-23T02:31:53.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.538+0000 7fd6f1b1f700 1 -- 172.21.15.71:0/3180035380 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6d406cce0 msgr2=0x7fd6d406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:53.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.538+0000 7fd6f1b1f700 1 --2- 172.21.15.71:0/3180035380 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6d406cce0 0x7fd6d406f1a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec074880 tx=0x7fd6dc01a040 comp rx=0 tx=0).stop 2024-04-23T02:31:53.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.538+0000 7fd6f1b1f700 1 -- 172.21.15.71:0/3180035380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6ec100e90 msgr2=0x7fd6ec073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:53.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.538+0000 7fd6f1b1f700 1 --2- 172.21.15.71:0/3180035380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6ec100e90 0x7fd6ec073090 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000d8d0 tx=0x7fd6e000dc90 comp rx=0 tx=0).stop 2024-04-23T02:31:53.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.538+0000 7fd6f1b1f700 1 -- 172.21.15.71:0/3180035380 shutdown_connections 2024-04-23T02:31:53.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.538+0000 7fd6f1b1f700 1 --2- 172.21.15.71:0/3180035380 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6d406cce0 0x7fd6d406f1a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:53.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.538+0000 7fd6f1b1f700 1 --2- 172.21.15.71:0/3180035380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6ec100e90 0x7fd6ec073090 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:53.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.538+0000 7fd6f1b1f700 1 --2- 172.21.15.71:0/3180035380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6ec101840 0x7fd6ec0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:53.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.538+0000 7fd6f1b1f700 1 -- 172.21.15.71:0/3180035380 >> 172.21.15.71:0/3180035380 conn(0x7fd6ec0fc740 msgr2=0x7fd6ec0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:53.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.538+0000 7fd6f1b1f700 1 -- 172.21.15.71:0/3180035380 shutdown_connections 2024-04-23T02:31:53.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.538+0000 7fd6f1b1f700 1 -- 172.21.15.71:0/3180035380 wait complete. 2024-04-23T02:31:53.550 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:31:53.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.641+0000 7f46bbce2700 1 -- 172.21.15.71:0/2337676711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40fff50 msgr2=0x7f46b4105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.641+0000 7f46bbce2700 1 --2- 172.21.15.71:0/2337676711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40fff50 0x7f46b4105840 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009910 tx=0x7f46a4009c20 comp rx=0 tx=0).stop 2024-04-23T02:31:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.641+0000 7f46bbce2700 1 -- 172.21.15.71:0/2337676711 shutdown_connections 2024-04-23T02:31:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.641+0000 7f46bbce2700 1 --2- 172.21.15.71:0/2337676711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40fff50 0x7f46b4105840 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.641+0000 7f46bbce2700 1 --2- 172.21.15.71:0/2337676711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b40ff5a0 0x7f46b40ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.641+0000 7f46bbce2700 1 -- 172.21.15.71:0/2337676711 >> 172.21.15.71:0/2337676711 conn(0x7f46b4074e30 msgr2=0x7f46b4075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.641+0000 7f46bbce2700 1 -- 172.21.15.71:0/2337676711 shutdown_connections 2024-04-23T02:31:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.641+0000 7f46bbce2700 1 -- 172.21.15.71:0/2337676711 wait complete. 2024-04-23T02:31:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.642+0000 7f46bbce2700 1 Processor -- start 2024-04-23T02:31:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.642+0000 7f46bbce2700 1 -- start start 2024-04-23T02:31:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.642+0000 7f46bbce2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40ff5a0 0x7f46b419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.642+0000 7f46bbce2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b40fff50 0x7f46b419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.642+0000 7f46bbce2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b419d260 con 0x7f46b40ff5a0 2024-04-23T02:31:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.642+0000 7f46bbce2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b4196710 con 0x7f46b40fff50 2024-04-23T02:31:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.642+0000 7f46b9a7e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40ff5a0 0x7f46b419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.642+0000 7f46b9a7e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40ff5a0 0x7f46b419c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50218/0 (socket says 172.21.15.71:50218) 2024-04-23T02:31:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.642+0000 7f46b9a7e700 1 -- 172.21.15.71:0/1818762644 learned_addr learned my addr 172.21.15.71:0/1818762644 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:31:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.642+0000 7f46b927d700 1 --2- 172.21.15.71:0/1818762644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b40fff50 0x7f46b419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.642+0000 7f46b9a7e700 1 -- 172.21.15.71:0/1818762644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b40fff50 msgr2=0x7f46b419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.642+0000 7f46b9a7e700 1 --2- 172.21.15.71:0/1818762644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b40fff50 0x7f46b419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.642+0000 7f46b9a7e700 1 -- 172.21.15.71:0/1818762644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a40095f0 con 0x7f46b40ff5a0 2024-04-23T02:31:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.642+0000 7f46b9a7e700 1 --2- 172.21.15.71:0/1818762644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40ff5a0 0x7f46b419c690 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f46b000c8a0 tx=0x7f46b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.642+0000 7f46aaffd700 1 -- 172.21.15.71:0/1818762644 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46b000cea0 con 0x7f46b40ff5a0 2024-04-23T02:31:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.643+0000 7f46aaffd700 1 -- 172.21.15.71:0/1818762644 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f46b0004830 con 0x7f46b40ff5a0 2024-04-23T02:31:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.643+0000 7f46bbce2700 1 -- 172.21.15.71:0/1818762644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b41969f0 con 0x7f46b40ff5a0 2024-04-23T02:31:53.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.643+0000 7f46aaffd700 1 -- 172.21.15.71:0/1818762644 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46b0005500 con 0x7f46b40ff5a0 2024-04-23T02:31:53.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.643+0000 7f46bbce2700 1 -- 172.21.15.71:0/1818762644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b4196f40 con 0x7f46b40ff5a0 2024-04-23T02:31:53.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.644+0000 7f46bbce2700 1 -- 172.21.15.71:0/1818762644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b4109200 con 0x7f46b40ff5a0 2024-04-23T02:31:53.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.644+0000 7f46aaffd700 1 -- 172.21.15.71:0/1818762644 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f46b00049a0 con 0x7f46b40ff5a0 2024-04-23T02:31:53.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.644+0000 7f46aaffd700 1 --2- 172.21.15.71:0/1818762644 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f46a006cce0 0x7f46a006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:53.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.645+0000 7f46aaffd700 1 -- 172.21.15.71:0/1818762644 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f46b008c1d0 con 0x7f46b40ff5a0 2024-04-23T02:31:53.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.645+0000 7f46b927d700 1 --2- 172.21.15.71:0/1818762644 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f46a006cce0 0x7f46a006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:53.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.645+0000 7f46b927d700 1 --2- 172.21.15.71:0/1818762644 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f46a006cce0 0x7f46a006f1a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f46b4197b80 tx=0x7f46a401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:53.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.649+0000 7f46aaffd700 1 -- 172.21.15.71:0/1818762644 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f46b0059d50 con 0x7f46b40ff5a0 2024-04-23T02:31:53.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.860+0000 7f46bbce2700 1 -- 172.21.15.71:0/1818762644 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f46b4197d10 con 0x7f46a006cce0 2024-04-23T02:31:53.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.861+0000 7f46aaffd700 1 -- 172.21.15.71:0/1818762644 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f46b4197d10 con 0x7f46a006cce0 2024-04-23T02:31:53.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.863+0000 7f46bbce2700 1 -- 172.21.15.71:0/1818762644 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f46a006cce0 msgr2=0x7f46a006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:53.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.863+0000 7f46bbce2700 1 --2- 172.21.15.71:0/1818762644 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f46a006cce0 0x7f46a006f1a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f46b4197b80 tx=0x7f46a401a040 comp rx=0 tx=0).stop 2024-04-23T02:31:53.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.863+0000 7f46bbce2700 1 -- 172.21.15.71:0/1818762644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40ff5a0 msgr2=0x7f46b419c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:53.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.863+0000 7f46bbce2700 1 --2- 172.21.15.71:0/1818762644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40ff5a0 0x7f46b419c690 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f46b000c8a0 tx=0x7f46b000cc60 comp rx=0 tx=0).stop 2024-04-23T02:31:53.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.863+0000 7f46bbce2700 1 -- 172.21.15.71:0/1818762644 shutdown_connections 2024-04-23T02:31:53.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.863+0000 7f46bbce2700 1 --2- 172.21.15.71:0/1818762644 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f46a006cce0 0x7f46a006f1a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:53.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.863+0000 7f46bbce2700 1 --2- 172.21.15.71:0/1818762644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40ff5a0 0x7f46b419c690 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:53.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.863+0000 7f46bbce2700 1 --2- 172.21.15.71:0/1818762644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b40fff50 0x7f46b419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:53.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.863+0000 7f46bbce2700 1 -- 172.21.15.71:0/1818762644 >> 172.21.15.71:0/1818762644 conn(0x7f46b4074e30 msgr2=0x7f46b40fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:53.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.863+0000 7f46bbce2700 1 -- 172.21.15.71:0/1818762644 shutdown_connections 2024-04-23T02:31:53.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.863+0000 7f46bbce2700 1 -- 172.21.15.71:0/1818762644 wait complete. 2024-04-23T02:31:53.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.966+0000 7f05418bc700 1 -- 172.21.15.71:0/1284102965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f053c0ffb40 msgr2=0x7f053c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:53.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.966+0000 7f05418bc700 1 --2- 172.21.15.71:0/1284102965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f053c0ffb40 0x7f053c0fffc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f052c009910 tx=0x7f052c009c20 comp rx=0 tx=0).stop 2024-04-23T02:31:53.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.966+0000 7f05418bc700 1 -- 172.21.15.71:0/1284102965 shutdown_connections 2024-04-23T02:31:53.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.966+0000 7f05418bc700 1 --2- 172.21.15.71:0/1284102965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f053c0ffb40 0x7f053c0fffc0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:53.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.966+0000 7f05418bc700 1 --2- 172.21.15.71:0/1284102965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f053c105bf0 0x7f053c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:53.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.966+0000 7f05418bc700 1 -- 172.21.15.71:0/1284102965 >> 172.21.15.71:0/1284102965 conn(0x7f053c0fb610 msgr2=0x7f053c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:53.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.966+0000 7f05418bc700 1 -- 172.21.15.71:0/1284102965 shutdown_connections 2024-04-23T02:31:53.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.966+0000 7f05418bc700 1 -- 172.21.15.71:0/1284102965 wait complete. 2024-04-23T02:31:53.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.966+0000 7f05418bc700 1 Processor -- start 2024-04-23T02:31:53.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.967+0000 7f05418bc700 1 -- start start 2024-04-23T02:31:53.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.967+0000 7f05418bc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f053c0ffb40 0x7f053c195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:53.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.967+0000 7f05418bc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f053c105bf0 0x7f053c195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:53.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.967+0000 7f05418bc700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c196240 con 0x7f053c0ffb40 2024-04-23T02:31:53.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.967+0000 7f05418bc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c199fd0 con 0x7f053c105bf0 2024-04-23T02:31:53.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.967+0000 7f053affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f053c0ffb40 0x7f053c195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:53.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.967+0000 7f053affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f053c0ffb40 0x7f053c195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50242/0 (socket says 172.21.15.71:50242) 2024-04-23T02:31:53.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.967+0000 7f053affd700 1 -- 172.21.15.71:0/2242765896 learned_addr learned my addr 172.21.15.71:0/2242765896 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:31:53.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.967+0000 7f053a7fc700 1 --2- 172.21.15.71:0/2242765896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f053c105bf0 0x7f053c195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:53.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.967+0000 7f053affd700 1 -- 172.21.15.71:0/2242765896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f053c105bf0 msgr2=0x7f053c195b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:53.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.967+0000 7f053affd700 1 --2- 172.21.15.71:0/2242765896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f053c105bf0 0x7f053c195b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:53.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.967+0000 7f053affd700 1 -- 172.21.15.71:0/2242765896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c0095f0 con 0x7f053c0ffb40 2024-04-23T02:31:53.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.968+0000 7f053affd700 1 --2- 172.21.15.71:0/2242765896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f053c0ffb40 0x7f053c195620 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f053000e920 tx=0x7f053000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:53.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.968+0000 7f05408ba700 1 -- 172.21.15.71:0/2242765896 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0530009790 con 0x7f053c0ffb40 2024-04-23T02:31:53.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.968+0000 7f05418bc700 1 -- 172.21.15.71:0/2242765896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053c19a2b0 con 0x7f053c0ffb40 2024-04-23T02:31:53.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.968+0000 7f05408ba700 1 -- 172.21.15.71:0/2242765896 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f053000cba0 con 0x7f053c0ffb40 2024-04-23T02:31:53.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.968+0000 7f05408ba700 1 -- 172.21.15.71:0/2242765896 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0530018650 con 0x7f053c0ffb40 2024-04-23T02:31:53.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.968+0000 7f05418bc700 1 -- 172.21.15.71:0/2242765896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053c19a800 con 0x7f053c0ffb40 2024-04-23T02:31:53.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.969+0000 7f05418bc700 1 -- 172.21.15.71:0/2242765896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053c1080a0 con 0x7f053c0ffb40 2024-04-23T02:31:53.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.969+0000 7f05408ba700 1 -- 172.21.15.71:0/2242765896 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f053000cd10 con 0x7f053c0ffb40 2024-04-23T02:31:53.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.970+0000 7f05408ba700 1 --2- 172.21.15.71:0/2242765896 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f052406cce0 0x7f052406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:53.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.970+0000 7f05408ba700 1 -- 172.21.15.71:0/2242765896 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0530014070 con 0x7f053c0ffb40 2024-04-23T02:31:53.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.970+0000 7f053a7fc700 1 --2- 172.21.15.71:0/2242765896 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f052406cce0 0x7f052406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:53.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.970+0000 7f053a7fc700 1 --2- 172.21.15.71:0/2242765896 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f052406cce0 0x7f052406f1a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f052c009fd0 tx=0x7f052c005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:53.974+0000 7f05408ba700 1 -- 172.21.15.71:0/2242765896 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0530052c50 con 0x7f053c0ffb40 2024-04-23T02:31:54.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.176+0000 7f05418bc700 1 -- 172.21.15.71:0/2242765896 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f053c19aa20 con 0x7f052406cce0 2024-04-23T02:31:54.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.187+0000 7f05408ba700 1 -- 172.21.15.71:0/2242765896 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f053c19aa20 con 0x7f052406cce0 2024-04-23T02:31:54.189 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:31:54.189 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (15m) 9m ago 26m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:31:54.189 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (27m) 9m ago 27m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:31:54.189 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (26m) 9m ago 26m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:31:54.189 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (13m) 9m ago 27m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:31:54.189 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (13m) 9m ago 25m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:31:54.190 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (15m) 9m ago 26m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:31:54.190 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (9m) 9m ago 20m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:31:54.190 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (9m) 9m ago 20m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:31:54.190 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (20m) 9m ago 20m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:31:54.190 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (9m) 9m ago 20m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:31:54.190 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (17m) 9m ago 28m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:31:54.190 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (16m) 9m ago 25m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:31:54.190 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (14m) 9m ago 28m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:31:54.190 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (13m) 9m ago 25m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:31:54.190 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (16m) 9m ago 27m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:31:54.190 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (16m) 9m ago 25m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:31:54.190 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (13m) 9m ago 24m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:31:54.190 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (12m) 9m ago 24m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:31:54.190 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (11m) 9m ago 23m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:31:54.191 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (11m) 9m ago 23m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:31:54.191 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (10m) 9m ago 22m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:31:54.191 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (10m) 9m ago 22m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:31:54.191 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (15m) 9m ago 26m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:31:54.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.189+0000 7f05418bc700 1 -- 172.21.15.71:0/2242765896 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f052406cce0 msgr2=0x7f052406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:54.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.189+0000 7f05418bc700 1 --2- 172.21.15.71:0/2242765896 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f052406cce0 0x7f052406f1a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f052c009fd0 tx=0x7f052c005940 comp rx=0 tx=0).stop 2024-04-23T02:31:54.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.189+0000 7f05418bc700 1 -- 172.21.15.71:0/2242765896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f053c0ffb40 msgr2=0x7f053c195620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:54.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.189+0000 7f05418bc700 1 --2- 172.21.15.71:0/2242765896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f053c0ffb40 0x7f053c195620 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f053000e920 tx=0x7f053000ece0 comp rx=0 tx=0).stop 2024-04-23T02:31:54.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.189+0000 7f05418bc700 1 -- 172.21.15.71:0/2242765896 shutdown_connections 2024-04-23T02:31:54.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.189+0000 7f05418bc700 1 --2- 172.21.15.71:0/2242765896 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f052406cce0 0x7f052406f1a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:54.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.189+0000 7f05418bc700 1 --2- 172.21.15.71:0/2242765896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f053c0ffb40 0x7f053c195620 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:54.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.189+0000 7f05418bc700 1 --2- 172.21.15.71:0/2242765896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f053c105bf0 0x7f053c195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:54.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.189+0000 7f05418bc700 1 -- 172.21.15.71:0/2242765896 >> 172.21.15.71:0/2242765896 conn(0x7f053c0fb610 msgr2=0x7f053c0fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:54.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.189+0000 7f05418bc700 1 -- 172.21.15.71:0/2242765896 shutdown_connections 2024-04-23T02:31:54.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.189+0000 7f05418bc700 1 -- 172.21.15.71:0/2242765896 wait complete. 2024-04-23T02:31:54.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.290+0000 7f5615c3b700 1 -- 172.21.15.71:0/2858864185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56100ffda0 msgr2=0x7f5610100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:54.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.290+0000 7f5615c3b700 1 --2- 172.21.15.71:0/2858864185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56100ffda0 0x7f5610100180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5600009910 tx=0x7f5600009c20 comp rx=0 tx=0).stop 2024-04-23T02:31:54.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.290+0000 7f5615c3b700 1 -- 172.21.15.71:0/2858864185 shutdown_connections 2024-04-23T02:31:54.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.290+0000 7f5615c3b700 1 --2- 172.21.15.71:0/2858864185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5610100750 0x7f56101047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:54.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.290+0000 7f5615c3b700 1 --2- 172.21.15.71:0/2858864185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56100ffda0 0x7f5610100180 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:54.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.290+0000 7f5615c3b700 1 -- 172.21.15.71:0/2858864185 >> 172.21.15.71:0/2858864185 conn(0x7f56100fb650 msgr2=0x7f56100fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:54.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.290+0000 7f5615c3b700 1 -- 172.21.15.71:0/2858864185 shutdown_connections 2024-04-23T02:31:54.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.290+0000 7f5615c3b700 1 -- 172.21.15.71:0/2858864185 wait complete. 2024-04-23T02:31:54.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.291+0000 7f5615c3b700 1 Processor -- start 2024-04-23T02:31:54.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.291+0000 7f5615c3b700 1 -- start start 2024-04-23T02:31:54.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.291+0000 7f5615c3b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56100ffda0 0x7f561019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:54.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.291+0000 7f5615c3b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5610100750 0x7f561019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:54.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.291+0000 7f5615c3b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5610196670 con 0x7f5610100750 2024-04-23T02:31:54.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.291+0000 7f5615c3b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56101967e0 con 0x7f56100ffda0 2024-04-23T02:31:54.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.292+0000 7f560effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5610100750 0x7f561019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:54.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.292+0000 7f560effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5610100750 0x7f561019cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50250/0 (socket says 172.21.15.71:50250) 2024-04-23T02:31:54.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.292+0000 7f560effd700 1 -- 172.21.15.71:0/2378457197 learned_addr learned my addr 172.21.15.71:0/2378457197 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:31:54.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.292+0000 7f560f7fe700 1 --2- 172.21.15.71:0/2378457197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56100ffda0 0x7f561019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:54.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.292+0000 7f560effd700 1 -- 172.21.15.71:0/2378457197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56100ffda0 msgr2=0x7f561019c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:54.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.292+0000 7f560effd700 1 --2- 172.21.15.71:0/2378457197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56100ffda0 0x7f561019c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:54.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.292+0000 7f560effd700 1 -- 172.21.15.71:0/2378457197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56000095f0 con 0x7f5610100750 2024-04-23T02:31:54.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.292+0000 7f560effd700 1 --2- 172.21.15.71:0/2378457197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5610100750 0x7f561019cbc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f560400c8f0 tx=0x7f560400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:54.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.292+0000 7f560cff9700 1 -- 172.21.15.71:0/2378457197 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5604007a50 con 0x7f5610100750 2024-04-23T02:31:54.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.292+0000 7f560cff9700 1 -- 172.21.15.71:0/2378457197 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5604007bb0 con 0x7f5610100750 2024-04-23T02:31:54.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.292+0000 7f5615c3b700 1 -- 172.21.15.71:0/2378457197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5610196ac0 con 0x7f5610100750 2024-04-23T02:31:54.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.292+0000 7f560cff9700 1 -- 172.21.15.71:0/2378457197 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5604018630 con 0x7f5610100750 2024-04-23T02:31:54.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.292+0000 7f5615c3b700 1 -- 172.21.15.71:0/2378457197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5610197010 con 0x7f5610100750 2024-04-23T02:31:54.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.293+0000 7f5615c3b700 1 -- 172.21.15.71:0/2378457197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f561004f2e0 con 0x7f5610100750 2024-04-23T02:31:54.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.293+0000 7f560cff9700 1 -- 172.21.15.71:0/2378457197 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f560400f450 con 0x7f5610100750 2024-04-23T02:31:54.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.294+0000 7f560cff9700 1 --2- 172.21.15.71:0/2378457197 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f55f806cce0 0x7f55f806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:54.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.294+0000 7f560cff9700 1 -- 172.21.15.71:0/2378457197 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f560408c4f0 con 0x7f5610100750 2024-04-23T02:31:54.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.294+0000 7f560f7fe700 1 --2- 172.21.15.71:0/2378457197 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f55f806cce0 0x7f55f806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:54.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.294+0000 7f560f7fe700 1 --2- 172.21.15.71:0/2378457197 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f55f806cce0 0x7f55f806f1a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5600006010 tx=0x7f560001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:54.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.298+0000 7f560cff9700 1 -- 172.21.15.71:0/2378457197 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f56040565e0 con 0x7f5610100750 2024-04-23T02:31:54.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.574+0000 7f5615c3b700 1 -- 172.21.15.71:0/2378457197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f561010cc20 con 0x7f5610100750 2024-04-23T02:31:54.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.574+0000 7f560cff9700 1 -- 172.21.15.71:0/2378457197 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f5604059c00 con 0x7f5610100750 2024-04-23T02:31:54.577 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:31:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:31:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:31:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:31:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:31:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:31:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:31:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:31:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:31:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:31:54.578 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:31:54.578 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:31:54.578 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:31:54.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:31:54.578 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:31:54.578 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:31:54.578 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:31:54.578 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:31:54.578 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:31:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.576+0000 7f5615c3b700 1 -- 172.21.15.71:0/2378457197 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f55f806cce0 msgr2=0x7f55f806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.576+0000 7f5615c3b700 1 --2- 172.21.15.71:0/2378457197 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f55f806cce0 0x7f55f806f1a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5600006010 tx=0x7f560001a040 comp rx=0 tx=0).stop 2024-04-23T02:31:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.576+0000 7f5615c3b700 1 -- 172.21.15.71:0/2378457197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5610100750 msgr2=0x7f561019cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.576+0000 7f5615c3b700 1 --2- 172.21.15.71:0/2378457197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5610100750 0x7f561019cbc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f560400c8f0 tx=0x7f560400ccb0 comp rx=0 tx=0).stop 2024-04-23T02:31:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.576+0000 7f5615c3b700 1 -- 172.21.15.71:0/2378457197 shutdown_connections 2024-04-23T02:31:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.576+0000 7f5615c3b700 1 --2- 172.21.15.71:0/2378457197 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f55f806cce0 0x7f55f806f1a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.576+0000 7f5615c3b700 1 --2- 172.21.15.71:0/2378457197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56100ffda0 0x7f561019c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.576+0000 7f5615c3b700 1 --2- 172.21.15.71:0/2378457197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5610100750 0x7f561019cbc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.576+0000 7f5615c3b700 1 -- 172.21.15.71:0/2378457197 >> 172.21.15.71:0/2378457197 conn(0x7f56100fb650 msgr2=0x7f5610104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.577+0000 7f5615c3b700 1 -- 172.21.15.71:0/2378457197 shutdown_connections 2024-04-23T02:31:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.577+0000 7f5615c3b700 1 -- 172.21.15.71:0/2378457197 wait complete. 2024-04-23T02:31:54.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.678+0000 7f9e80e44700 1 -- 172.21.15.71:0/1620606530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e7c101850 msgr2=0x7f9e7c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:54.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.678+0000 7f9e80e44700 1 --2- 172.21.15.71:0/1620606530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e7c101850 0x7f9e7c1058a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009910 tx=0x7f9e6c009c20 comp rx=0 tx=0).stop 2024-04-23T02:31:54.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.678+0000 7f9e80e44700 1 -- 172.21.15.71:0/1620606530 shutdown_connections 2024-04-23T02:31:54.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.678+0000 7f9e80e44700 1 --2- 172.21.15.71:0/1620606530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e7c101850 0x7f9e7c1058a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:54.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.678+0000 7f9e80e44700 1 --2- 172.21.15.71:0/1620606530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c100ea0 0x7f9e7c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:54.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.678+0000 7f9e80e44700 1 -- 172.21.15.71:0/1620606530 >> 172.21.15.71:0/1620606530 conn(0x7f9e7c0fc750 msgr2=0x7f9e7c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:54.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.678+0000 7f9e80e44700 1 -- 172.21.15.71:0/1620606530 shutdown_connections 2024-04-23T02:31:54.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.678+0000 7f9e80e44700 1 -- 172.21.15.71:0/1620606530 wait complete. 2024-04-23T02:31:54.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.679+0000 7f9e80e44700 1 Processor -- start 2024-04-23T02:31:54.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.679+0000 7f9e80e44700 1 -- start start 2024-04-23T02:31:54.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.679+0000 7f9e80e44700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c100ea0 0x7f9e7c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:54.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.679+0000 7f9e80e44700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e7c101850 0x7f9e7c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.679+0000 7f9e80e44700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c19d1f0 con 0x7f9e7c101850 2024-04-23T02:31:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.679+0000 7f9e80e44700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c1966a0 con 0x7f9e7c100ea0 2024-04-23T02:31:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.680+0000 7f9e79d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e7c101850 0x7f9e7c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.680+0000 7f9e79d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e7c101850 0x7f9e7c19cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50274/0 (socket says 172.21.15.71:50274) 2024-04-23T02:31:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.680+0000 7f9e79d9b700 1 -- 172.21.15.71:0/832052211 learned_addr learned my addr 172.21.15.71:0/832052211 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:31:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.680+0000 7f9e7a59c700 1 --2- 172.21.15.71:0/832052211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c100ea0 0x7f9e7c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.680+0000 7f9e79d9b700 1 -- 172.21.15.71:0/832052211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c100ea0 msgr2=0x7f9e7c19c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.680+0000 7f9e79d9b700 1 --2- 172.21.15.71:0/832052211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c100ea0 0x7f9e7c19c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.680+0000 7f9e79d9b700 1 -- 172.21.15.71:0/832052211 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c0095f0 con 0x7f9e7c101850 2024-04-23T02:31:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.680+0000 7f9e79d9b700 1 --2- 172.21.15.71:0/832052211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e7c101850 0x7f9e7c19cb60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0092e0 tx=0x7f9e6c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.680+0000 7f9e6b7fe700 1 -- 172.21.15.71:0/832052211 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e6c01d070 con 0x7f9e7c101850 2024-04-23T02:31:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.680+0000 7f9e6b7fe700 1 -- 172.21.15.71:0/832052211 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9e6c005dc0 con 0x7f9e7c101850 2024-04-23T02:31:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.680+0000 7f9e6b7fe700 1 -- 172.21.15.71:0/832052211 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e6c00fea0 con 0x7f9e7c101850 2024-04-23T02:31:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.680+0000 7f9e80e44700 1 -- 172.21.15.71:0/832052211 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7c196920 con 0x7f9e7c101850 2024-04-23T02:31:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.680+0000 7f9e80e44700 1 -- 172.21.15.71:0/832052211 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7c196dc0 con 0x7f9e7c101850 2024-04-23T02:31:54.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.681+0000 7f9e80e44700 1 -- 172.21.15.71:0/832052211 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7c1092a0 con 0x7f9e7c101850 2024-04-23T02:31:54.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.682+0000 7f9e6b7fe700 1 -- 172.21.15.71:0/832052211 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9e6c003a60 con 0x7f9e7c101850 2024-04-23T02:31:54.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.682+0000 7f9e6b7fe700 1 --2- 172.21.15.71:0/832052211 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9e6406cce0 0x7f9e6406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:54.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.682+0000 7f9e6b7fe700 1 -- 172.21.15.71:0/832052211 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9e6c08d740 con 0x7f9e7c101850 2024-04-23T02:31:54.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.682+0000 7f9e7a59c700 1 --2- 172.21.15.71:0/832052211 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9e6406cce0 0x7f9e6406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:54.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.683+0000 7f9e7a59c700 1 --2- 172.21.15.71:0/832052211 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9e6406cce0 0x7f9e6406f1a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9e70005950 tx=0x7f9e7000b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:54.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.687+0000 7f9e6b7fe700 1 -- 172.21.15.71:0/832052211 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9e6c05b240 con 0x7f9e7c101850 2024-04-23T02:31:54.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:54 smithi071 ceph-mon[144544]: pgmap v557: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:54.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:54 smithi192 ceph-mon[108617]: pgmap v557: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:54.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.930+0000 7f9e80e44700 1 -- 172.21.15.71:0/832052211 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9e7c04f2e0 con 0x7f9e7c101850 2024-04-23T02:31:54.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.931+0000 7f9e6b7fe700 1 -- 172.21.15.71:0/832052211 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f9e6c003d10 con 0x7f9e7c101850 2024-04-23T02:31:54.933 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:31:54.933 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:31:54.933 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:31:54.933 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:31:54.933 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:31:54.933 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:31:54.933 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:31:54.934 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:31:54.934 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:31:54.934 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:31:54.934 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:31:54.934 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:31:54.934 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:31:54.934 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:31:54.934 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:31:54.934 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:31:54.934 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:31:54.934 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:31:54.934 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:31:54.934 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:31:54.934 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:31:54.934 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:31:54.935 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:31:54.935 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:31:54.935 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:31:54.935 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:31:54.935 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:31:54.935 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:31:54.935 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:31:54.935 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:31:54.935 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:31:54.935 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:31:54.935 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:31:54.935 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:31:54.935 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:31:54.935 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:31:54.935 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:31:54.936 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:31:54.936 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:31:54.936 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:31:54.936 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:31:54.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.932+0000 7f9e80e44700 1 -- 172.21.15.71:0/832052211 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9e6406cce0 msgr2=0x7f9e6406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:54.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.932+0000 7f9e80e44700 1 --2- 172.21.15.71:0/832052211 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9e6406cce0 0x7f9e6406f1a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9e70005950 tx=0x7f9e7000b410 comp rx=0 tx=0).stop 2024-04-23T02:31:54.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.932+0000 7f9e80e44700 1 -- 172.21.15.71:0/832052211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e7c101850 msgr2=0x7f9e7c19cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:54.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.932+0000 7f9e80e44700 1 --2- 172.21.15.71:0/832052211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e7c101850 0x7f9e7c19cb60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0092e0 tx=0x7f9e6c00bd70 comp rx=0 tx=0).stop 2024-04-23T02:31:54.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.933+0000 7f9e80e44700 1 -- 172.21.15.71:0/832052211 shutdown_connections 2024-04-23T02:31:54.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.933+0000 7f9e80e44700 1 --2- 172.21.15.71:0/832052211 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9e6406cce0 0x7f9e6406f1a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:54.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.933+0000 7f9e80e44700 1 --2- 172.21.15.71:0/832052211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c100ea0 0x7f9e7c19c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:54.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.933+0000 7f9e80e44700 1 --2- 172.21.15.71:0/832052211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e7c101850 0x7f9e7c19cb60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:54.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.933+0000 7f9e80e44700 1 -- 172.21.15.71:0/832052211 >> 172.21.15.71:0/832052211 conn(0x7f9e7c0fc750 msgr2=0x7f9e7c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:54.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.933+0000 7f9e80e44700 1 -- 172.21.15.71:0/832052211 shutdown_connections 2024-04-23T02:31:54.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:54.933+0000 7f9e80e44700 1 -- 172.21.15.71:0/832052211 wait complete. 2024-04-23T02:31:54.937 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:31:55.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.035+0000 7ff016b4a700 1 -- 172.21.15.71:0/2084178974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff010100750 msgr2=0x7ff0101047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:55.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.035+0000 7ff016b4a700 1 --2- 172.21.15.71:0/2084178974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff010100750 0x7ff0101047a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7feffc009910 tx=0x7feffc009c20 comp rx=0 tx=0).stop 2024-04-23T02:31:55.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.036+0000 7ff016b4a700 1 -- 172.21.15.71:0/2084178974 shutdown_connections 2024-04-23T02:31:55.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.036+0000 7ff016b4a700 1 --2- 172.21.15.71:0/2084178974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff010100750 0x7ff0101047a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:55.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.036+0000 7ff016b4a700 1 --2- 172.21.15.71:0/2084178974 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0100ffda0 0x7ff010100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:55.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.036+0000 7ff016b4a700 1 -- 172.21.15.71:0/2084178974 >> 172.21.15.71:0/2084178974 conn(0x7ff0100fb650 msgr2=0x7ff0100fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:55.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.036+0000 7ff016b4a700 1 -- 172.21.15.71:0/2084178974 shutdown_connections 2024-04-23T02:31:55.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.036+0000 7ff016b4a700 1 -- 172.21.15.71:0/2084178974 wait complete. 2024-04-23T02:31:55.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.036+0000 7ff016b4a700 1 Processor -- start 2024-04-23T02:31:55.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.036+0000 7ff016b4a700 1 -- start start 2024-04-23T02:31:55.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.036+0000 7ff016b4a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0100ffda0 0x7ff01019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:55.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.036+0000 7ff016b4a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff010100750 0x7ff01019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:55.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.036+0000 7ff016b4a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01019d190 con 0x7ff010100750 2024-04-23T02:31:55.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.036+0000 7ff016b4a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff010196640 con 0x7ff0100ffda0 2024-04-23T02:31:55.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.037+0000 7ff00ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff010100750 0x7ff01019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:55.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.037+0000 7ff00ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff010100750 0x7ff01019cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50286/0 (socket says 172.21.15.71:50286) 2024-04-23T02:31:55.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.037+0000 7ff00ffff700 1 -- 172.21.15.71:0/2263245564 learned_addr learned my addr 172.21.15.71:0/2263245564 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:31:55.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.037+0000 7ff0148e6700 1 --2- 172.21.15.71:0/2263245564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0100ffda0 0x7ff01019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:55.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.037+0000 7ff00ffff700 1 -- 172.21.15.71:0/2263245564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0100ffda0 msgr2=0x7ff01019c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:55.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.037+0000 7ff00ffff700 1 --2- 172.21.15.71:0/2263245564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0100ffda0 0x7ff01019c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:55.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.037+0000 7ff00ffff700 1 -- 172.21.15.71:0/2263245564 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc0095f0 con 0x7ff010100750 2024-04-23T02:31:55.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.037+0000 7ff0148e6700 1 --2- 172.21.15.71:0/2263245564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0100ffda0 0x7ff01019c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:31:55.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.037+0000 7ff00ffff700 1 --2- 172.21.15.71:0/2263245564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff010100750 0x7ff01019cb00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7feffc0092e0 tx=0x7feffc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:55.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.037+0000 7ff00dffb700 1 -- 172.21.15.71:0/2263245564 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feffc01d070 con 0x7ff010100750 2024-04-23T02:31:55.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.037+0000 7ff00dffb700 1 -- 172.21.15.71:0/2263245564 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feffc00f460 con 0x7ff010100750 2024-04-23T02:31:55.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.037+0000 7ff016b4a700 1 -- 172.21.15.71:0/2263245564 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0101968c0 con 0x7ff010100750 2024-04-23T02:31:55.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.037+0000 7ff00dffb700 1 -- 172.21.15.71:0/2263245564 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feffc021600 con 0x7ff010100750 2024-04-23T02:31:55.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.037+0000 7ff016b4a700 1 -- 172.21.15.71:0/2263245564 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff010196db0 con 0x7ff010100750 2024-04-23T02:31:55.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.038+0000 7ff016b4a700 1 -- 172.21.15.71:0/2263245564 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff01004f2e0 con 0x7ff010100750 2024-04-23T02:31:55.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.039+0000 7ff00dffb700 1 -- 172.21.15.71:0/2263245564 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7feffc00f5d0 con 0x7ff010100750 2024-04-23T02:31:55.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.039+0000 7ff00dffb700 1 --2- 172.21.15.71:0/2263245564 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff00006cce0 0x7ff00006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:55.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.039+0000 7ff00dffb700 1 -- 172.21.15.71:0/2263245564 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7feffc08d7c0 con 0x7ff010100750 2024-04-23T02:31:55.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.039+0000 7ff0148e6700 1 --2- 172.21.15.71:0/2263245564 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff00006cce0 0x7ff00006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:55.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.040+0000 7ff0148e6700 1 --2- 172.21.15.71:0/2263245564 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff00006cce0 0x7ff00006f1a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff004005950 tx=0x7ff004005f30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:55.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.044+0000 7ff00dffb700 1 -- 172.21.15.71:0/2263245564 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7feffc05b2c0 con 0x7ff010100750 2024-04-23T02:31:55.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.251+0000 7ff016b4a700 1 -- 172.21.15.71:0/2263245564 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff010197080 con 0x7ff00006cce0 2024-04-23T02:31:55.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.253+0000 7ff00dffb700 1 -- 172.21.15.71:0/2263245564 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff010197080 con 0x7ff00006cce0 2024-04-23T02:31:55.255 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:31:55.255 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:31:55.255 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:31:55.255 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:31:55.255 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:31:55.255 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:31:55.255 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:31:55.256 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:31:55.256 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:31:55.256 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:31:55.256 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:31:55.256 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:31:55.256 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:31:55.256 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:31:55.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.255+0000 7ff016b4a700 1 -- 172.21.15.71:0/2263245564 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff00006cce0 msgr2=0x7ff00006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:55.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.255+0000 7ff016b4a700 1 --2- 172.21.15.71:0/2263245564 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff00006cce0 0x7ff00006f1a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff004005950 tx=0x7ff004005f30 comp rx=0 tx=0).stop 2024-04-23T02:31:55.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.255+0000 7ff016b4a700 1 -- 172.21.15.71:0/2263245564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff010100750 msgr2=0x7ff01019cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:55.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.255+0000 7ff016b4a700 1 --2- 172.21.15.71:0/2263245564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff010100750 0x7ff01019cb00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7feffc0092e0 tx=0x7feffc00be20 comp rx=0 tx=0).stop 2024-04-23T02:31:55.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.255+0000 7ff016b4a700 1 -- 172.21.15.71:0/2263245564 shutdown_connections 2024-04-23T02:31:55.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.255+0000 7ff016b4a700 1 --2- 172.21.15.71:0/2263245564 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff00006cce0 0x7ff00006f1a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:55.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.255+0000 7ff016b4a700 1 --2- 172.21.15.71:0/2263245564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0100ffda0 0x7ff01019c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:55.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.255+0000 7ff016b4a700 1 --2- 172.21.15.71:0/2263245564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff010100750 0x7ff01019cb00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:55.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.255+0000 7ff016b4a700 1 -- 172.21.15.71:0/2263245564 >> 172.21.15.71:0/2263245564 conn(0x7ff0100fb650 msgr2=0x7ff0100fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:55.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.255+0000 7ff016b4a700 1 -- 172.21.15.71:0/2263245564 shutdown_connections 2024-04-23T02:31:55.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.255+0000 7ff016b4a700 1 -- 172.21.15.71:0/2263245564 wait complete. 2024-04-23T02:31:55.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.357+0000 7f7cad29c700 1 -- 172.21.15.71:0/796834896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca80ffda0 msgr2=0x7f7ca8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:55.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.357+0000 7f7cad29c700 1 --2- 172.21.15.71:0/796834896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca80ffda0 0x7f7ca8100180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009910 tx=0x7f7c90009c20 comp rx=0 tx=0).stop 2024-04-23T02:31:55.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.357+0000 7f7cad29c700 1 -- 172.21.15.71:0/796834896 shutdown_connections 2024-04-23T02:31:55.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.357+0000 7f7cad29c700 1 --2- 172.21.15.71:0/796834896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ca8100750 0x7f7ca81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:55.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.357+0000 7f7cad29c700 1 --2- 172.21.15.71:0/796834896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca80ffda0 0x7f7ca8100180 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:55.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.357+0000 7f7cad29c700 1 -- 172.21.15.71:0/796834896 >> 172.21.15.71:0/796834896 conn(0x7f7ca80fb650 msgr2=0x7f7ca80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:55.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.357+0000 7f7cad29c700 1 -- 172.21.15.71:0/796834896 shutdown_connections 2024-04-23T02:31:55.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.357+0000 7f7cad29c700 1 -- 172.21.15.71:0/796834896 wait complete. 2024-04-23T02:31:55.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.358+0000 7f7cad29c700 1 Processor -- start 2024-04-23T02:31:55.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.358+0000 7f7cad29c700 1 -- start start 2024-04-23T02:31:55.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.358+0000 7f7cad29c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca80ffda0 0x7f7ca819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:55.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.358+0000 7f7cad29c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ca8100750 0x7f7ca819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:55.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.358+0000 7f7cad29c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca8196670 con 0x7f7ca80ffda0 2024-04-23T02:31:55.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.358+0000 7f7cad29c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca81967e0 con 0x7f7ca8100750 2024-04-23T02:31:55.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.358+0000 7f7ca6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca80ffda0 0x7f7ca819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:55.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.358+0000 7f7ca6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca80ffda0 0x7f7ca819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46444/0 (socket says 172.21.15.71:46444) 2024-04-23T02:31:55.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.358+0000 7f7ca6ffd700 1 -- 172.21.15.71:0/3377391504 learned_addr learned my addr 172.21.15.71:0/3377391504 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:31:55.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.358+0000 7f7ca67fc700 1 --2- 172.21.15.71:0/3377391504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ca8100750 0x7f7ca819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:55.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.358+0000 7f7ca6ffd700 1 -- 172.21.15.71:0/3377391504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ca8100750 msgr2=0x7f7ca819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:55.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.358+0000 7f7ca6ffd700 1 --2- 172.21.15.71:0/3377391504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ca8100750 0x7f7ca819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:55.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.359+0000 7f7ca6ffd700 1 -- 172.21.15.71:0/3377391504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c900095f0 con 0x7f7ca80ffda0 2024-04-23T02:31:55.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.359+0000 7f7ca6ffd700 1 --2- 172.21.15.71:0/3377391504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca80ffda0 0x7f7ca819c680 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7c90006010 tx=0x7f7c9000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:55.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.359+0000 7f7c9ffff700 1 -- 172.21.15.71:0/3377391504 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c9001d070 con 0x7f7ca80ffda0 2024-04-23T02:31:55.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.359+0000 7f7c9ffff700 1 -- 172.21.15.71:0/3377391504 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7c9000f460 con 0x7f7ca80ffda0 2024-04-23T02:31:55.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.359+0000 7f7cad29c700 1 -- 172.21.15.71:0/3377391504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca8196a60 con 0x7f7ca80ffda0 2024-04-23T02:31:55.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.359+0000 7f7c9ffff700 1 -- 172.21.15.71:0/3377391504 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c90021620 con 0x7f7ca80ffda0 2024-04-23T02:31:55.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.359+0000 7f7cad29c700 1 -- 172.21.15.71:0/3377391504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca8196f50 con 0x7f7ca80ffda0 2024-04-23T02:31:55.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.360+0000 7f7cad29c700 1 -- 172.21.15.71:0/3377391504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca804f2e0 con 0x7f7ca80ffda0 2024-04-23T02:31:55.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.360+0000 7f7c9ffff700 1 -- 172.21.15.71:0/3377391504 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7c9002b440 con 0x7f7ca80ffda0 2024-04-23T02:31:55.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.361+0000 7f7c9ffff700 1 --2- 172.21.15.71:0/3377391504 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7c9406cce0 0x7f7c9406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:31:55.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.361+0000 7f7ca67fc700 1 --2- 172.21.15.71:0/3377391504 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7c9406cce0 0x7f7c9406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:31:55.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.361+0000 7f7c9ffff700 1 -- 172.21.15.71:0/3377391504 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7c9008d6d0 con 0x7f7ca80ffda0 2024-04-23T02:31:55.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.361+0000 7f7ca67fc700 1 --2- 172.21.15.71:0/3377391504 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7c9406cce0 0x7f7c9406f1a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7c98005950 tx=0x7f7c980058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:31:55.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.365+0000 7f7c9ffff700 1 -- 172.21.15.71:0/3377391504 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7c900578a0 con 0x7f7ca80ffda0 2024-04-23T02:31:55.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.640+0000 7f7cad29c700 1 -- 172.21.15.71:0/3377391504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7ca81971c0 con 0x7f7ca80ffda0 2024-04-23T02:31:55.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.641+0000 7f7c9ffff700 1 -- 172.21.15.71:0/3377391504 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f7c90057020 con 0x7f7ca80ffda0 2024-04-23T02:31:55.643 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:31:55.643 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:31:55.643 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:31:55.644 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:31:55.644 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:31:55.644 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:31:55.644 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:31:55.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.643+0000 7f7cad29c700 1 -- 172.21.15.71:0/3377391504 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7c9406cce0 msgr2=0x7f7c9406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:55.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.643+0000 7f7cad29c700 1 --2- 172.21.15.71:0/3377391504 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7c9406cce0 0x7f7c9406f1a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7c98005950 tx=0x7f7c980058e0 comp rx=0 tx=0).stop 2024-04-23T02:31:55.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.643+0000 7f7cad29c700 1 -- 172.21.15.71:0/3377391504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca80ffda0 msgr2=0x7f7ca819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:31:55.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.643+0000 7f7cad29c700 1 --2- 172.21.15.71:0/3377391504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca80ffda0 0x7f7ca819c680 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7c90006010 tx=0x7f7c9000be20 comp rx=0 tx=0).stop 2024-04-23T02:31:55.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.643+0000 7f7cad29c700 1 -- 172.21.15.71:0/3377391504 shutdown_connections 2024-04-23T02:31:55.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.643+0000 7f7cad29c700 1 --2- 172.21.15.71:0/3377391504 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7c9406cce0 0x7f7c9406f1a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:55.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.643+0000 7f7cad29c700 1 --2- 172.21.15.71:0/3377391504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca80ffda0 0x7f7ca819c680 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:55.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.643+0000 7f7cad29c700 1 --2- 172.21.15.71:0/3377391504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ca8100750 0x7f7ca819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:31:55.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.643+0000 7f7cad29c700 1 -- 172.21.15.71:0/3377391504 >> 172.21.15.71:0/3377391504 conn(0x7f7ca80fb650 msgr2=0x7f7ca8104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:31:55.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.643+0000 7f7cad29c700 1 -- 172.21.15.71:0/3377391504 shutdown_connections 2024-04-23T02:31:55.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:31:55.643+0000 7f7cad29c700 1 -- 172.21.15.71:0/3377391504 wait complete. 2024-04-23T02:31:55.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:55 smithi071 ceph-mon[144544]: from='client.34846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:31:55.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:55 smithi071 ceph-mon[144544]: from='client.34850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:31:55.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:55 smithi071 ceph-mon[144544]: from='client.34854 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:31:55.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:55 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2378457197' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:31:55.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:55 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/832052211' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:31:55.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:55 smithi192 ceph-mon[108617]: from='client.34846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:31:55.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:55 smithi192 ceph-mon[108617]: from='client.34850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:31:55.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:55 smithi192 ceph-mon[108617]: from='client.34854 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:31:55.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:55 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2378457197' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:31:55.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:55 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/832052211' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:31:56.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:56 smithi071 ceph-mon[144544]: pgmap v558: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:56.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:56 smithi071 ceph-mon[144544]: from='client.34866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:31:56.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:56 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3377391504' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:31:56.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:56 smithi192 ceph-mon[108617]: pgmap v558: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:56.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:56 smithi192 ceph-mon[108617]: from='client.34866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:31:56.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:56 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3377391504' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:31:58.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:31:58 smithi071 ceph-mon[144544]: pgmap v559: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:31:58.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:31:58 smithi192 ceph-mon[108617]: pgmap v559: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:00.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:00 smithi071 ceph-mon[144544]: pgmap v560: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:00.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:00 smithi192 ceph-mon[108617]: pgmap v560: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:02.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:02 smithi071 ceph-mon[144544]: pgmap v561: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:02.796 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:02 smithi192 ceph-mon[108617]: pgmap v561: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:04.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:04 smithi071 ceph-mon[144544]: pgmap v562: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:04.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:04 smithi192 ceph-mon[108617]: pgmap v562: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:05.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:32:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:32:06.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:06 smithi071 ceph-mon[144544]: pgmap v563: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:06.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:06 smithi192 ceph-mon[108617]: pgmap v563: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:07.842 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:07 smithi192 ceph-mon[108617]: pgmap v564: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:07.869 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:07 smithi071 ceph-mon[144544]: pgmap v564: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:10.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:09 smithi071 ceph-mon[144544]: pgmap v565: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:10.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:09 smithi192 ceph-mon[108617]: pgmap v565: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:12.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:11 smithi071 ceph-mon[144544]: pgmap v566: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:12.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:11 smithi192 ceph-mon[108617]: pgmap v566: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:14.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:13 smithi071 ceph-mon[144544]: pgmap v567: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:14.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:13 smithi192 ceph-mon[108617]: pgmap v567: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:16.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:15 smithi071 ceph-mon[144544]: pgmap v568: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:16.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:15 smithi192 ceph-mon[108617]: pgmap v568: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:18.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:17 smithi071 ceph-mon[144544]: pgmap v569: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:18.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:17 smithi192 ceph-mon[108617]: pgmap v569: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:20.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:19 smithi071 ceph-mon[144544]: pgmap v570: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:20.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:32:20.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:19 smithi192 ceph-mon[108617]: pgmap v570: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:20.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:32:22.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:21 smithi071 ceph-mon[144544]: pgmap v571: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:22.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:21 smithi192 ceph-mon[108617]: pgmap v571: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:24.239 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:23 smithi192 ceph-mon[108617]: pgmap v572: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:24.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:23 smithi071 ceph-mon[144544]: pgmap v572: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:25.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.758+0000 7ffb57b53700 1 -- 172.21.15.71:0/4118368930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb50068ea0 msgr2=0x7ffb5010d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:25.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.758+0000 7ffb57b53700 1 --2- 172.21.15.71:0/4118368930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb50068ea0 0x7ffb5010d290 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009910 tx=0x7ffb40009c20 comp rx=0 tx=0).stop 2024-04-23T02:32:25.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.758+0000 7ffb57b53700 1 -- 172.21.15.71:0/4118368930 shutdown_connections 2024-04-23T02:32:25.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.758+0000 7ffb57b53700 1 --2- 172.21.15.71:0/4118368930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb50068ea0 0x7ffb5010d290 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:25.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.758+0000 7ffb57b53700 1 --2- 172.21.15.71:0/4118368930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb50068580 0x7ffb50068960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:25.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.758+0000 7ffb57b53700 1 -- 172.21.15.71:0/4118368930 >> 172.21.15.71:0/4118368930 conn(0x7ffb50075670 msgr2=0x7ffb50075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:25.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.758+0000 7ffb57b53700 1 -- 172.21.15.71:0/4118368930 shutdown_connections 2024-04-23T02:32:25.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.758+0000 7ffb57b53700 1 -- 172.21.15.71:0/4118368930 wait complete. 2024-04-23T02:32:25.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.758+0000 7ffb57b53700 1 Processor -- start 2024-04-23T02:32:25.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.758+0000 7ffb57b53700 1 -- start start 2024-04-23T02:32:25.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.759+0000 7ffb57b53700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb50068580 0x7ffb5019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:25.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.759+0000 7ffb57b53700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb50068ea0 0x7ffb5019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:25.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.759+0000 7ffb57b53700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5019f3d0 con 0x7ffb50068580 2024-04-23T02:32:25.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.759+0000 7ffb57b53700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb50198880 con 0x7ffb50068ea0 2024-04-23T02:32:25.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.759+0000 7ffb558ef700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb50068580 0x7ffb5019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:25.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.759+0000 7ffb558ef700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb50068580 0x7ffb5019e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49560/0 (socket says 172.21.15.71:49560) 2024-04-23T02:32:25.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.759+0000 7ffb558ef700 1 -- 172.21.15.71:0/186840693 learned_addr learned my addr 172.21.15.71:0/186840693 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:32:25.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.759+0000 7ffb550ee700 1 --2- 172.21.15.71:0/186840693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb50068ea0 0x7ffb5019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:25.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.759+0000 7ffb558ef700 1 -- 172.21.15.71:0/186840693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb50068ea0 msgr2=0x7ffb5019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:25.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.759+0000 7ffb558ef700 1 --2- 172.21.15.71:0/186840693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb50068ea0 0x7ffb5019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:25.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.759+0000 7ffb558ef700 1 -- 172.21.15.71:0/186840693 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb400095f0 con 0x7ffb50068580 2024-04-23T02:32:25.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.759+0000 7ffb550ee700 1 --2- 172.21.15.71:0/186840693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb50068ea0 0x7ffb5019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:32:25.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.759+0000 7ffb558ef700 1 --2- 172.21.15.71:0/186840693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb50068580 0x7ffb5019e800 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00c8a0 tx=0x7ffb4c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:25.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.759+0000 7ffb46ffd700 1 -- 172.21.15.71:0/186840693 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb4c00cea0 con 0x7ffb50068580 2024-04-23T02:32:25.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.759+0000 7ffb46ffd700 1 -- 172.21.15.71:0/186840693 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffb4c004830 con 0x7ffb50068580 2024-04-23T02:32:25.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.759+0000 7ffb57b53700 1 -- 172.21.15.71:0/186840693 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb50198b60 con 0x7ffb50068580 2024-04-23T02:32:25.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.759+0000 7ffb46ffd700 1 -- 172.21.15.71:0/186840693 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb4c005500 con 0x7ffb50068580 2024-04-23T02:32:25.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.759+0000 7ffb57b53700 1 -- 172.21.15.71:0/186840693 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb501990b0 con 0x7ffb50068580 2024-04-23T02:32:25.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.760+0000 7ffb57b53700 1 -- 172.21.15.71:0/186840693 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5010aa00 con 0x7ffb50068580 2024-04-23T02:32:25.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.761+0000 7ffb46ffd700 1 -- 172.21.15.71:0/186840693 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ffb4c0049a0 con 0x7ffb50068580 2024-04-23T02:32:25.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.761+0000 7ffb46ffd700 1 --2- 172.21.15.71:0/186840693 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ffb3c06cce0 0x7ffb3c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:25.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.761+0000 7ffb46ffd700 1 -- 172.21.15.71:0/186840693 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ffb4c08c280 con 0x7ffb50068580 2024-04-23T02:32:25.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.761+0000 7ffb550ee700 1 --2- 172.21.15.71:0/186840693 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ffb3c06cce0 0x7ffb3c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:25.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.762+0000 7ffb550ee700 1 --2- 172.21.15.71:0/186840693 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ffb3c06cce0 0x7ffb3c06f1a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffb400059d0 tx=0x7ffb4001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:25.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.766+0000 7ffb46ffd700 1 -- 172.21.15.71:0/186840693 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ffb4c059d80 con 0x7ffb50068580 2024-04-23T02:32:25.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.977+0000 7ffb57b53700 1 -- 172.21.15.71:0/186840693 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb50199330 con 0x7ffb3c06cce0 2024-04-23T02:32:25.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.979+0000 7ffb46ffd700 1 -- 172.21.15.71:0/186840693 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ffb50199330 con 0x7ffb3c06cce0 2024-04-23T02:32:25.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.981+0000 7ffb57b53700 1 -- 172.21.15.71:0/186840693 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ffb3c06cce0 msgr2=0x7ffb3c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:25.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.981+0000 7ffb57b53700 1 --2- 172.21.15.71:0/186840693 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ffb3c06cce0 0x7ffb3c06f1a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffb400059d0 tx=0x7ffb4001a040 comp rx=0 tx=0).stop 2024-04-23T02:32:25.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.981+0000 7ffb57b53700 1 -- 172.21.15.71:0/186840693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb50068580 msgr2=0x7ffb5019e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:25.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.981+0000 7ffb57b53700 1 --2- 172.21.15.71:0/186840693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb50068580 0x7ffb5019e800 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00c8a0 tx=0x7ffb4c00cc60 comp rx=0 tx=0).stop 2024-04-23T02:32:25.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.981+0000 7ffb57b53700 1 -- 172.21.15.71:0/186840693 shutdown_connections 2024-04-23T02:32:25.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.981+0000 7ffb57b53700 1 --2- 172.21.15.71:0/186840693 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ffb3c06cce0 0x7ffb3c06f1a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:25.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.981+0000 7ffb57b53700 1 --2- 172.21.15.71:0/186840693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb50068580 0x7ffb5019e800 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:25.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.981+0000 7ffb57b53700 1 --2- 172.21.15.71:0/186840693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb50068ea0 0x7ffb5019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:25.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.981+0000 7ffb57b53700 1 -- 172.21.15.71:0/186840693 >> 172.21.15.71:0/186840693 conn(0x7ffb50075670 msgr2=0x7ffb500fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:25.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.981+0000 7ffb57b53700 1 -- 172.21.15.71:0/186840693 shutdown_connections 2024-04-23T02:32:25.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:25.981+0000 7ffb57b53700 1 -- 172.21.15.71:0/186840693 wait complete. 2024-04-23T02:32:25.996 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:32:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.088+0000 7fc85bfbe700 1 -- 172.21.15.71:0/2366845175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc854105bd0 msgr2=0x7fc854105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.088+0000 7fc85bfbe700 1 --2- 172.21.15.71:0/2366845175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc854105bd0 0x7fc854105fb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc850009910 tx=0x7fc850009c20 comp rx=0 tx=0).stop 2024-04-23T02:32:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.088+0000 7fc85bfbe700 1 -- 172.21.15.71:0/2366845175 shutdown_connections 2024-04-23T02:32:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.088+0000 7fc85bfbe700 1 --2- 172.21.15.71:0/2366845175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8540ffc20 0x7fc854100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.088+0000 7fc85bfbe700 1 --2- 172.21.15.71:0/2366845175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc854105bd0 0x7fc854105fb0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.088+0000 7fc85bfbe700 1 -- 172.21.15.71:0/2366845175 >> 172.21.15.71:0/2366845175 conn(0x7fc854075670 msgr2=0x7fc854075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.088+0000 7fc85bfbe700 1 -- 172.21.15.71:0/2366845175 shutdown_connections 2024-04-23T02:32:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.088+0000 7fc85bfbe700 1 -- 172.21.15.71:0/2366845175 wait complete. 2024-04-23T02:32:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.088+0000 7fc85bfbe700 1 Processor -- start 2024-04-23T02:32:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.088+0000 7fc85bfbe700 1 -- start start 2024-04-23T02:32:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.088+0000 7fc85bfbe700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8540ffc20 0x7fc854101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:26.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.088+0000 7fc85bfbe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc854105bd0 0x7fc854101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:26.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.088+0000 7fc85bfbe700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8541022c0 con 0x7fc854105bd0 2024-04-23T02:32:26.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.088+0000 7fc85bfbe700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc854102400 con 0x7fc8540ffc20 2024-04-23T02:32:26.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.089+0000 7fc859559700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc854105bd0 0x7fc854101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:26.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.089+0000 7fc859559700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc854105bd0 0x7fc854101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49570/0 (socket says 172.21.15.71:49570) 2024-04-23T02:32:26.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.089+0000 7fc859559700 1 -- 172.21.15.71:0/3142112808 learned_addr learned my addr 172.21.15.71:0/3142112808 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:32:26.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.089+0000 7fc859d5a700 1 --2- 172.21.15.71:0/3142112808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8540ffc20 0x7fc854101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:26.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.089+0000 7fc859559700 1 -- 172.21.15.71:0/3142112808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8540ffc20 msgr2=0x7fc854101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:26.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.089+0000 7fc859559700 1 --2- 172.21.15.71:0/3142112808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8540ffc20 0x7fc854101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:26.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.089+0000 7fc859559700 1 -- 172.21.15.71:0/3142112808 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8500095f0 con 0x7fc854105bd0 2024-04-23T02:32:26.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.089+0000 7fc859559700 1 --2- 172.21.15.71:0/3142112808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc854105bd0 0x7fc854101d80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc84400c8a0 tx=0x7fc84400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:26.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.089+0000 7fc84affd700 1 -- 172.21.15.71:0/3142112808 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc84400cea0 con 0x7fc854105bd0 2024-04-23T02:32:26.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.089+0000 7fc84affd700 1 -- 172.21.15.71:0/3142112808 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc844004830 con 0x7fc854105bd0 2024-04-23T02:32:26.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.089+0000 7fc85bfbe700 1 -- 172.21.15.71:0/3142112808 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8541a2990 con 0x7fc854105bd0 2024-04-23T02:32:26.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.089+0000 7fc84affd700 1 -- 172.21.15.71:0/3142112808 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc844005500 con 0x7fc854105bd0 2024-04-23T02:32:26.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.089+0000 7fc85bfbe700 1 -- 172.21.15.71:0/3142112808 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8541a2cf0 con 0x7fc854105bd0 2024-04-23T02:32:26.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.090+0000 7fc85bfbe700 1 -- 172.21.15.71:0/3142112808 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85404f2e0 con 0x7fc854105bd0 2024-04-23T02:32:26.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.091+0000 7fc84affd700 1 -- 172.21.15.71:0/3142112808 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc8440049a0 con 0x7fc854105bd0 2024-04-23T02:32:26.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.091+0000 7fc84affd700 1 --2- 172.21.15.71:0/3142112808 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc84006cce0 0x7fc84006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:26.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.091+0000 7fc84affd700 1 -- 172.21.15.71:0/3142112808 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc84408c1b0 con 0x7fc854105bd0 2024-04-23T02:32:26.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.091+0000 7fc859d5a700 1 --2- 172.21.15.71:0/3142112808 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc84006cce0 0x7fc84006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:26.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.091+0000 7fc859d5a700 1 --2- 172.21.15.71:0/3142112808 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc84006cce0 0x7fc84006f1a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc85000b5c0 tx=0x7fc850005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:26.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.096+0000 7fc84affd700 1 -- 172.21.15.71:0/3142112808 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc8440562a0 con 0x7fc854105bd0 2024-04-23T02:32:26.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:25 smithi071 ceph-mon[144544]: pgmap v573: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:26.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.305+0000 7fc85bfbe700 1 -- 172.21.15.71:0/3142112808 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc854068db0 con 0x7fc84006cce0 2024-04-23T02:32:26.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.306+0000 7fc84affd700 1 -- 172.21.15.71:0/3142112808 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc854068db0 con 0x7fc84006cce0 2024-04-23T02:32:26.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.308+0000 7fc85bfbe700 1 -- 172.21.15.71:0/3142112808 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc84006cce0 msgr2=0x7fc84006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:26.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.308+0000 7fc85bfbe700 1 --2- 172.21.15.71:0/3142112808 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc84006cce0 0x7fc84006f1a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc85000b5c0 tx=0x7fc850005fd0 comp rx=0 tx=0).stop 2024-04-23T02:32:26.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.308+0000 7fc85bfbe700 1 -- 172.21.15.71:0/3142112808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc854105bd0 msgr2=0x7fc854101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:26.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.308+0000 7fc85bfbe700 1 --2- 172.21.15.71:0/3142112808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc854105bd0 0x7fc854101d80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc84400c8a0 tx=0x7fc84400cc60 comp rx=0 tx=0).stop 2024-04-23T02:32:26.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.308+0000 7fc85bfbe700 1 -- 172.21.15.71:0/3142112808 shutdown_connections 2024-04-23T02:32:26.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.308+0000 7fc85bfbe700 1 --2- 172.21.15.71:0/3142112808 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc84006cce0 0x7fc84006f1a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:26.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.308+0000 7fc85bfbe700 1 --2- 172.21.15.71:0/3142112808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8540ffc20 0x7fc854101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:26.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.308+0000 7fc85bfbe700 1 --2- 172.21.15.71:0/3142112808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc854105bd0 0x7fc854101d80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:26.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.308+0000 7fc85bfbe700 1 -- 172.21.15.71:0/3142112808 >> 172.21.15.71:0/3142112808 conn(0x7fc854075670 msgr2=0x7fc8540686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:26.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.308+0000 7fc85bfbe700 1 -- 172.21.15.71:0/3142112808 shutdown_connections 2024-04-23T02:32:26.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.308+0000 7fc85bfbe700 1 -- 172.21.15.71:0/3142112808 wait complete. 2024-04-23T02:32:26.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:25 smithi192 ceph-mon[108617]: pgmap v573: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:26.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.409+0000 7f0bb6753700 1 -- 172.21.15.71:0/4160615876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bb0105bd0 msgr2=0x7f0bb0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:26.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.409+0000 7f0bb6753700 1 --2- 172.21.15.71:0/4160615876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bb0105bd0 0x7f0bb0105fb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009910 tx=0x7f0ba4009c20 comp rx=0 tx=0).stop 2024-04-23T02:32:26.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.410+0000 7f0bb6753700 1 -- 172.21.15.71:0/4160615876 shutdown_connections 2024-04-23T02:32:26.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.410+0000 7f0bb6753700 1 --2- 172.21.15.71:0/4160615876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bb00ffc20 0x7f0bb0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:26.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.410+0000 7f0bb6753700 1 --2- 172.21.15.71:0/4160615876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bb0105bd0 0x7f0bb0105fb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:26.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.410+0000 7f0bb6753700 1 -- 172.21.15.71:0/4160615876 >> 172.21.15.71:0/4160615876 conn(0x7f0bb0075670 msgr2=0x7f0bb0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:26.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.410+0000 7f0bb6753700 1 -- 172.21.15.71:0/4160615876 shutdown_connections 2024-04-23T02:32:26.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.410+0000 7f0bb6753700 1 -- 172.21.15.71:0/4160615876 wait complete. 2024-04-23T02:32:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.410+0000 7f0bb6753700 1 Processor -- start 2024-04-23T02:32:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.410+0000 7f0bb6753700 1 -- start start 2024-04-23T02:32:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.410+0000 7f0bb6753700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bb00ffc20 0x7f0bb0101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.411+0000 7f0bb6753700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bb0105bd0 0x7f0bb0101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.411+0000 7f0bb6753700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb01022c0 con 0x7f0bb0105bd0 2024-04-23T02:32:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.411+0000 7f0bb6753700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb0102400 con 0x7f0bb00ffc20 2024-04-23T02:32:26.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.411+0000 7f0baf7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bb0105bd0 0x7f0bb0101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:26.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.411+0000 7f0baf7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bb0105bd0 0x7f0bb0101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49580/0 (socket says 172.21.15.71:49580) 2024-04-23T02:32:26.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.411+0000 7f0baf7fe700 1 -- 172.21.15.71:0/1009119408 learned_addr learned my addr 172.21.15.71:0/1009119408 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:32:26.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.411+0000 7f0baffff700 1 --2- 172.21.15.71:0/1009119408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bb00ffc20 0x7f0bb0101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:26.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.411+0000 7f0baf7fe700 1 -- 172.21.15.71:0/1009119408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bb00ffc20 msgr2=0x7f0bb0101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:26.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.411+0000 7f0baf7fe700 1 --2- 172.21.15.71:0/1009119408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bb00ffc20 0x7f0bb0101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:26.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.411+0000 7f0baf7fe700 1 -- 172.21.15.71:0/1009119408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba40095f0 con 0x7f0bb0105bd0 2024-04-23T02:32:26.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.411+0000 7f0baf7fe700 1 --2- 172.21.15.71:0/1009119408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bb0105bd0 0x7f0bb0101d80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000d8d0 tx=0x7f0ba000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:26.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.411+0000 7f0bad7fa700 1 -- 172.21.15.71:0/1009119408 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ba0009940 con 0x7f0bb0105bd0 2024-04-23T02:32:26.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.411+0000 7f0bb6753700 1 -- 172.21.15.71:0/1009119408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb01a2990 con 0x7f0bb0105bd0 2024-04-23T02:32:26.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.411+0000 7f0bad7fa700 1 -- 172.21.15.71:0/1009119408 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0ba0010460 con 0x7f0bb0105bd0 2024-04-23T02:32:26.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.411+0000 7f0bad7fa700 1 -- 172.21.15.71:0/1009119408 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ba000f5f0 con 0x7f0bb0105bd0 2024-04-23T02:32:26.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.411+0000 7f0bb6753700 1 -- 172.21.15.71:0/1009119408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb01a2da0 con 0x7f0bb0105bd0 2024-04-23T02:32:26.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.412+0000 7f0bb6753700 1 -- 172.21.15.71:0/1009119408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb004f2e0 con 0x7f0bb0105bd0 2024-04-23T02:32:26.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.413+0000 7f0bad7fa700 1 -- 172.21.15.71:0/1009119408 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0ba00105d0 con 0x7f0bb0105bd0 2024-04-23T02:32:26.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.413+0000 7f0bad7fa700 1 --2- 172.21.15.71:0/1009119408 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0b9806cce0 0x7f0b9806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:26.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.413+0000 7f0bad7fa700 1 -- 172.21.15.71:0/1009119408 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0ba008c5e0 con 0x7f0bb0105bd0 2024-04-23T02:32:26.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.413+0000 7f0baffff700 1 --2- 172.21.15.71:0/1009119408 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0b9806cce0 0x7f0b9806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:26.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.413+0000 7f0baffff700 1 --2- 172.21.15.71:0/1009119408 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0b9806cce0 0x7f0b9806f1a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0ba400b5c0 tx=0x7f0ba4005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:26.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.417+0000 7f0bad7fa700 1 -- 172.21.15.71:0/1009119408 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0ba0056650 con 0x7f0bb0105bd0 2024-04-23T02:32:26.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.622+0000 7f0bb6753700 1 -- 172.21.15.71:0/1009119408 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0bb0068db0 con 0x7f0b9806cce0 2024-04-23T02:32:26.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.632+0000 7f0bad7fa700 1 -- 172.21.15.71:0/1009119408 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0bb0068db0 con 0x7f0b9806cce0 2024-04-23T02:32:26.636 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:32:26.636 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (16m) 9m ago 27m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:32:26.636 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (27m) 9m ago 27m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:32:26.636 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (26m) 9m ago 26m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:32:26.636 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (14m) 9m ago 27m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:32:26.636 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (13m) 9m ago 26m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:32:26.636 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (15m) 9m ago 27m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:32:26.636 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (10m) 9m ago 21m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:32:26.636 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (9m) 9m ago 21m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:32:26.636 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (21m) 9m ago 21m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:32:26.636 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (9m) 9m ago 21m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:32:26.636 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (18m) 9m ago 29m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:32:26.636 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (17m) 9m ago 26m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:32:26.636 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (14m) 9m ago 29m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:32:26.636 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (14m) 9m ago 26m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:32:26.637 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (17m) 9m ago 27m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:32:26.637 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (16m) 9m ago 26m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:32:26.637 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (13m) 9m ago 25m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:32:26.637 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (12m) 9m ago 24m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:32:26.637 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (12m) 9m ago 24m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:32:26.637 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (11m) 9m ago 23m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:32:26.638 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (11m) 9m ago 23m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:32:26.638 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (10m) 9m ago 23m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:32:26.638 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (16m) 9m ago 26m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:32:26.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.634+0000 7f0bb6753700 1 -- 172.21.15.71:0/1009119408 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0b9806cce0 msgr2=0x7f0b9806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:26.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.634+0000 7f0bb6753700 1 --2- 172.21.15.71:0/1009119408 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0b9806cce0 0x7f0b9806f1a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0ba400b5c0 tx=0x7f0ba4005fd0 comp rx=0 tx=0).stop 2024-04-23T02:32:26.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.634+0000 7f0bb6753700 1 -- 172.21.15.71:0/1009119408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bb0105bd0 msgr2=0x7f0bb0101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:26.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.634+0000 7f0bb6753700 1 --2- 172.21.15.71:0/1009119408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bb0105bd0 0x7f0bb0101d80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000d8d0 tx=0x7f0ba000dc90 comp rx=0 tx=0).stop 2024-04-23T02:32:26.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.634+0000 7f0bb6753700 1 -- 172.21.15.71:0/1009119408 shutdown_connections 2024-04-23T02:32:26.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.634+0000 7f0bb6753700 1 --2- 172.21.15.71:0/1009119408 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0b9806cce0 0x7f0b9806f1a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:26.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.634+0000 7f0bb6753700 1 --2- 172.21.15.71:0/1009119408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bb00ffc20 0x7f0bb0101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:26.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.634+0000 7f0bb6753700 1 --2- 172.21.15.71:0/1009119408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bb0105bd0 0x7f0bb0101d80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:26.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.634+0000 7f0bb6753700 1 -- 172.21.15.71:0/1009119408 >> 172.21.15.71:0/1009119408 conn(0x7f0bb0075670 msgr2=0x7f0bb00686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:26.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.634+0000 7f0bb6753700 1 -- 172.21.15.71:0/1009119408 shutdown_connections 2024-04-23T02:32:26.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.634+0000 7f0bb6753700 1 -- 172.21.15.71:0/1009119408 wait complete. 2024-04-23T02:32:26.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.735+0000 7f4bc492a700 1 -- 172.21.15.71:0/658248292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bc0100e90 msgr2=0x7f4bc0101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:26.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.735+0000 7f4bc492a700 1 --2- 172.21.15.71:0/658248292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bc0100e90 0x7f4bc0101270 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009910 tx=0x7f4bb0009c20 comp rx=0 tx=0).stop 2024-04-23T02:32:26.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.735+0000 7f4bc492a700 1 -- 172.21.15.71:0/658248292 shutdown_connections 2024-04-23T02:32:26.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.735+0000 7f4bc492a700 1 --2- 172.21.15.71:0/658248292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bc0101840 0x7f4bc0105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:26.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.735+0000 7f4bc492a700 1 --2- 172.21.15.71:0/658248292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bc0100e90 0x7f4bc0101270 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:26.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.735+0000 7f4bc492a700 1 -- 172.21.15.71:0/658248292 >> 172.21.15.71:0/658248292 conn(0x7f4bc00fc740 msgr2=0x7f4bc00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:26.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.736+0000 7f4bc492a700 1 -- 172.21.15.71:0/658248292 shutdown_connections 2024-04-23T02:32:26.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.736+0000 7f4bc492a700 1 -- 172.21.15.71:0/658248292 wait complete. 2024-04-23T02:32:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.736+0000 7f4bc492a700 1 Processor -- start 2024-04-23T02:32:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.737+0000 7f4bc492a700 1 -- start start 2024-04-23T02:32:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.737+0000 7f4bc492a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bc0100e90 0x7f4bc019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.737+0000 7f4bc492a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bc0101840 0x7f4bc019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.737+0000 7f4bc492a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc019d100 con 0x7f4bc0101840 2024-04-23T02:32:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.737+0000 7f4bc492a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc0196640 con 0x7f4bc0100e90 2024-04-23T02:32:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.737+0000 7f4bbdd9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bc0101840 0x7f4bc019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.737+0000 7f4bbdd9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bc0101840 0x7f4bc019cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49602/0 (socket says 172.21.15.71:49602) 2024-04-23T02:32:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.737+0000 7f4bbdd9b700 1 -- 172.21.15.71:0/2110898486 learned_addr learned my addr 172.21.15.71:0/2110898486 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:32:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.737+0000 7f4bbe59c700 1 --2- 172.21.15.71:0/2110898486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bc0100e90 0x7f4bc019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.737+0000 7f4bbdd9b700 1 -- 172.21.15.71:0/2110898486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bc0100e90 msgr2=0x7f4bc019c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.737+0000 7f4bbdd9b700 1 --2- 172.21.15.71:0/2110898486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bc0100e90 0x7f4bc019c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.737+0000 7f4bbdd9b700 1 -- 172.21.15.71:0/2110898486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb00095f0 con 0x7f4bc0101840 2024-04-23T02:32:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.737+0000 7f4bbdd9b700 1 --2- 172.21.15.71:0/2110898486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bc0101840 0x7f4bc019cb00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400c960 tx=0x7f4bb400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.737+0000 7f4baf7fe700 1 -- 172.21.15.71:0/2110898486 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bb4007a70 con 0x7f4bc0101840 2024-04-23T02:32:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.737+0000 7f4bc492a700 1 -- 172.21.15.71:0/2110898486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc0196920 con 0x7f4bc0101840 2024-04-23T02:32:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.738+0000 7f4baf7fe700 1 -- 172.21.15.71:0/2110898486 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4bb4007bd0 con 0x7f4bc0101840 2024-04-23T02:32:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.738+0000 7f4baf7fe700 1 -- 172.21.15.71:0/2110898486 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bb4018630 con 0x7f4bc0101840 2024-04-23T02:32:26.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.738+0000 7f4bc492a700 1 -- 172.21.15.71:0/2110898486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc0196e70 con 0x7f4bc0101840 2024-04-23T02:32:26.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.738+0000 7f4bc492a700 1 -- 172.21.15.71:0/2110898486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc0109230 con 0x7f4bc0101840 2024-04-23T02:32:26.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.739+0000 7f4baf7fe700 1 -- 172.21.15.71:0/2110898486 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4bb400f9f0 con 0x7f4bc0101840 2024-04-23T02:32:26.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.739+0000 7f4baf7fe700 1 --2- 172.21.15.71:0/2110898486 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4ba806cce0 0x7f4ba806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:26.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.739+0000 7f4baf7fe700 1 -- 172.21.15.71:0/2110898486 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4bb408c5e0 con 0x7f4bc0101840 2024-04-23T02:32:26.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.739+0000 7f4bbe59c700 1 --2- 172.21.15.71:0/2110898486 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4ba806cce0 0x7f4ba806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:26.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.740+0000 7f4bbe59c700 1 --2- 172.21.15.71:0/2110898486 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4ba806cce0 0x7f4ba806f1a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000b5c0 tx=0x7f4bb0005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:26.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:26.744+0000 7f4baf7fe700 1 -- 172.21.15.71:0/2110898486 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f4bb4059e50 con 0x7f4bc0101840 2024-04-23T02:32:27.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.018+0000 7f4bc492a700 1 -- 172.21.15.71:0/2110898486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4bc004f2e0 con 0x7f4bc0101840 2024-04-23T02:32:27.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.018+0000 7f4baf7fe700 1 -- 172.21.15.71:0/2110898486 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f4bb4059c70 con 0x7f4bc0101840 2024-04-23T02:32:27.022 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:32:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:32:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:32:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:32:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:32:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:32:27.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:32:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:32:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:32:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:32:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:32:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:32:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:32:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:32:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:32:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:32:27.023 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:32:27.024 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:32:27.024 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:32:27.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.020+0000 7f4bc492a700 1 -- 172.21.15.71:0/2110898486 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4ba806cce0 msgr2=0x7f4ba806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:27.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.020+0000 7f4bc492a700 1 --2- 172.21.15.71:0/2110898486 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4ba806cce0 0x7f4ba806f1a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000b5c0 tx=0x7f4bb0005fb0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.020+0000 7f4bc492a700 1 -- 172.21.15.71:0/2110898486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bc0101840 msgr2=0x7f4bc019cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.020+0000 7f4bc492a700 1 --2- 172.21.15.71:0/2110898486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bc0101840 0x7f4bc019cb00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400c960 tx=0x7f4bb400cd20 comp rx=0 tx=0).stop 2024-04-23T02:32:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.020+0000 7f4bc492a700 1 -- 172.21.15.71:0/2110898486 shutdown_connections 2024-04-23T02:32:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.020+0000 7f4bc492a700 1 --2- 172.21.15.71:0/2110898486 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4ba806cce0 0x7f4ba806f1a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.020+0000 7f4bc492a700 1 --2- 172.21.15.71:0/2110898486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bc0100e90 0x7f4bc019c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.020+0000 7f4bc492a700 1 --2- 172.21.15.71:0/2110898486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4bc0101840 0x7f4bc019cb00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.020+0000 7f4bc492a700 1 -- 172.21.15.71:0/2110898486 >> 172.21.15.71:0/2110898486 conn(0x7f4bc00fc740 msgr2=0x7f4bc00fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.020+0000 7f4bc492a700 1 -- 172.21.15.71:0/2110898486 shutdown_connections 2024-04-23T02:32:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.020+0000 7f4bc492a700 1 -- 172.21.15.71:0/2110898486 wait complete. 2024-04-23T02:32:27.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.122+0000 7efe039fb700 1 -- 172.21.15.71:0/2939994319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdfc101840 msgr2=0x7efdfc105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:27.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.122+0000 7efe039fb700 1 --2- 172.21.15.71:0/2939994319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdfc101840 0x7efdfc105890 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009910 tx=0x7efdf8009c20 comp rx=0 tx=0).stop 2024-04-23T02:32:27.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.122+0000 7efe039fb700 1 -- 172.21.15.71:0/2939994319 shutdown_connections 2024-04-23T02:32:27.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.122+0000 7efe039fb700 1 --2- 172.21.15.71:0/2939994319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdfc101840 0x7efdfc105890 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.122+0000 7efe039fb700 1 --2- 172.21.15.71:0/2939994319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdfc100e90 0x7efdfc101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.122+0000 7efe039fb700 1 -- 172.21.15.71:0/2939994319 >> 172.21.15.71:0/2939994319 conn(0x7efdfc0fc740 msgr2=0x7efdfc0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:27.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.122+0000 7efe039fb700 1 -- 172.21.15.71:0/2939994319 shutdown_connections 2024-04-23T02:32:27.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.122+0000 7efe039fb700 1 -- 172.21.15.71:0/2939994319 wait complete. 2024-04-23T02:32:27.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.123+0000 7efe039fb700 1 Processor -- start 2024-04-23T02:32:27.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.123+0000 7efe039fb700 1 -- start start 2024-04-23T02:32:27.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.123+0000 7efe039fb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdfc100e90 0x7efdfc19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:27.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.123+0000 7efe039fb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdfc101840 0x7efdfc19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:27.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.123+0000 7efe039fb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc19d200 con 0x7efdfc101840 2024-04-23T02:32:27.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.123+0000 7efe039fb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc1966b0 con 0x7efdfc100e90 2024-04-23T02:32:27.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.123+0000 7efe00f96700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdfc101840 0x7efdfc19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:27.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.123+0000 7efe00f96700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdfc101840 0x7efdfc19cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49618/0 (socket says 172.21.15.71:49618) 2024-04-23T02:32:27.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.123+0000 7efe00f96700 1 -- 172.21.15.71:0/795266974 learned_addr learned my addr 172.21.15.71:0/795266974 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:32:27.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.123+0000 7efe01797700 1 --2- 172.21.15.71:0/795266974 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdfc100e90 0x7efdfc19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:27.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.123+0000 7efe00f96700 1 -- 172.21.15.71:0/795266974 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdfc100e90 msgr2=0x7efdfc19c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:27.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.123+0000 7efe00f96700 1 --2- 172.21.15.71:0/795266974 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdfc100e90 0x7efdfc19c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.123+0000 7efe00f96700 1 -- 172.21.15.71:0/795266974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf80095f0 con 0x7efdfc101840 2024-04-23T02:32:27.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.124+0000 7efe00f96700 1 --2- 172.21.15.71:0/795266974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdfc101840 0x7efdfc19cb70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efdf80092e0 tx=0x7efdf800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:27.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.124+0000 7efdf27fc700 1 -- 172.21.15.71:0/795266974 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdf801d070 con 0x7efdfc101840 2024-04-23T02:32:27.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.124+0000 7efdf27fc700 1 -- 172.21.15.71:0/795266974 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efdf800f460 con 0x7efdfc101840 2024-04-23T02:32:27.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.124+0000 7efdf27fc700 1 -- 172.21.15.71:0/795266974 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdf8021600 con 0x7efdfc101840 2024-04-23T02:32:27.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.124+0000 7efe039fb700 1 -- 172.21.15.71:0/795266974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdfc196930 con 0x7efdfc101840 2024-04-23T02:32:27.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.124+0000 7efe039fb700 1 -- 172.21.15.71:0/795266974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc196e20 con 0x7efdfc101840 2024-04-23T02:32:27.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.125+0000 7efe039fb700 1 -- 172.21.15.71:0/795266974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdfc109290 con 0x7efdfc101840 2024-04-23T02:32:27.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.125+0000 7efdf27fc700 1 -- 172.21.15.71:0/795266974 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7efdf800f5d0 con 0x7efdfc101840 2024-04-23T02:32:27.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.125+0000 7efdf27fc700 1 --2- 172.21.15.71:0/795266974 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efde806cce0 0x7efde806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:27.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.126+0000 7efdf27fc700 1 -- 172.21.15.71:0/795266974 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7efdf808d840 con 0x7efdfc101840 2024-04-23T02:32:27.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.126+0000 7efe01797700 1 --2- 172.21.15.71:0/795266974 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efde806cce0 0x7efde806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:27.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.126+0000 7efe01797700 1 --2- 172.21.15.71:0/795266974 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efde806cce0 0x7efde806f1a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efdec005950 tx=0x7efdec005cf0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:27.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.130+0000 7efdf27fc700 1 -- 172.21.15.71:0/795266974 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7efdf805b340 con 0x7efdfc101840 2024-04-23T02:32:27.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:26 smithi071 ceph-mon[144544]: from='client.34874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:32:27.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:26 smithi071 ceph-mon[144544]: from='client.34878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:32:27.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.364+0000 7efe039fb700 1 -- 172.21.15.71:0/795266974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efdfc04f2e0 con 0x7efdfc101840 2024-04-23T02:32:27.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.364+0000 7efdf27fc700 1 -- 172.21.15.71:0/795266974 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7efdf805aed0 con 0x7efdfc101840 2024-04-23T02:32:27.368 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:32:27.368 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:32:27.368 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:32:27.368 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:32:27.368 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:32:27.368 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:32:27.368 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:32:27.368 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:32:27.368 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:32:27.368 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:32:27.368 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:32:27.368 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:32:27.369 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:32:27.369 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:32:27.369 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:32:27.369 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:32:27.369 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:32:27.369 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:32:27.369 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:32:27.369 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:32:27.369 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:32:27.369 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:32:27.369 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:32:27.369 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:32:27.369 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:32:27.369 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:32:27.370 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:32:27.370 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:32:27.370 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:32:27.370 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:32:27.370 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:32:27.370 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:32:27.370 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:32:27.370 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:32:27.370 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:32:27.370 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:32:27.370 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:32:27.370 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:32:27.370 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:32:27.371 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:32:27.371 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:32:27.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.366+0000 7efe039fb700 1 -- 172.21.15.71:0/795266974 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efde806cce0 msgr2=0x7efde806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:27.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.366+0000 7efe039fb700 1 --2- 172.21.15.71:0/795266974 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efde806cce0 0x7efde806f1a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efdec005950 tx=0x7efdec005cf0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.366+0000 7efe039fb700 1 -- 172.21.15.71:0/795266974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdfc101840 msgr2=0x7efdfc19cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:27.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.366+0000 7efe039fb700 1 --2- 172.21.15.71:0/795266974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdfc101840 0x7efdfc19cb70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efdf80092e0 tx=0x7efdf800be20 comp rx=0 tx=0).stop 2024-04-23T02:32:27.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.367+0000 7efe039fb700 1 -- 172.21.15.71:0/795266974 shutdown_connections 2024-04-23T02:32:27.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.367+0000 7efe039fb700 1 --2- 172.21.15.71:0/795266974 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efde806cce0 0x7efde806f1a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.367+0000 7efe039fb700 1 --2- 172.21.15.71:0/795266974 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdfc100e90 0x7efdfc19c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.367+0000 7efe039fb700 1 --2- 172.21.15.71:0/795266974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdfc101840 0x7efdfc19cb70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.367+0000 7efe039fb700 1 -- 172.21.15.71:0/795266974 >> 172.21.15.71:0/795266974 conn(0x7efdfc0fc740 msgr2=0x7efdfc0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:27.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.367+0000 7efe039fb700 1 -- 172.21.15.71:0/795266974 shutdown_connections 2024-04-23T02:32:27.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.367+0000 7efe039fb700 1 -- 172.21.15.71:0/795266974 wait complete. 2024-04-23T02:32:27.372 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:32:27.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:26 smithi192 ceph-mon[108617]: from='client.34874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:32:27.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:26 smithi192 ceph-mon[108617]: from='client.34878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:32:27.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.470+0000 7f963e5c0700 1 -- 172.21.15.71:0/1125977470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9638100750 msgr2=0x7f96381047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:27.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.470+0000 7f963e5c0700 1 --2- 172.21.15.71:0/1125977470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9638100750 0x7f96381047a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9628009910 tx=0x7f9628009c20 comp rx=0 tx=0).stop 2024-04-23T02:32:27.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.470+0000 7f963e5c0700 1 -- 172.21.15.71:0/1125977470 shutdown_connections 2024-04-23T02:32:27.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.470+0000 7f963e5c0700 1 --2- 172.21.15.71:0/1125977470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9638100750 0x7f96381047a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.470+0000 7f963e5c0700 1 --2- 172.21.15.71:0/1125977470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96380ffda0 0x7f9638100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.470+0000 7f963e5c0700 1 -- 172.21.15.71:0/1125977470 >> 172.21.15.71:0/1125977470 conn(0x7f96380fb650 msgr2=0x7f96380fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:27.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.470+0000 7f963e5c0700 1 -- 172.21.15.71:0/1125977470 shutdown_connections 2024-04-23T02:32:27.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.470+0000 7f963e5c0700 1 -- 172.21.15.71:0/1125977470 wait complete. 2024-04-23T02:32:27.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.470+0000 7f963e5c0700 1 Processor -- start 2024-04-23T02:32:27.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.470+0000 7f963e5c0700 1 -- start start 2024-04-23T02:32:27.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.471+0000 7f963e5c0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96380ffda0 0x7f96381976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:27.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.471+0000 7f963e5c0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9638100750 0x7f9638197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:27.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.471+0000 7f963e5c0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9638198220 con 0x7f96380ffda0 2024-04-23T02:32:27.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.471+0000 7f963e5c0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9638198360 con 0x7f9638100750 2024-04-23T02:32:27.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.471+0000 7f9637fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96380ffda0 0x7f96381976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:27.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.471+0000 7f9637fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96380ffda0 0x7f96381976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49648/0 (socket says 172.21.15.71:49648) 2024-04-23T02:32:27.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.471+0000 7f9637fff700 1 -- 172.21.15.71:0/1486144194 learned_addr learned my addr 172.21.15.71:0/1486144194 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:32:27.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.471+0000 7f96377fe700 1 --2- 172.21.15.71:0/1486144194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9638100750 0x7f9638197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:27.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.471+0000 7f9637fff700 1 -- 172.21.15.71:0/1486144194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9638100750 msgr2=0x7f9638197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:27.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.471+0000 7f9637fff700 1 --2- 172.21.15.71:0/1486144194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9638100750 0x7f9638197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.471+0000 7f9637fff700 1 -- 172.21.15.71:0/1486144194 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96280095f0 con 0x7f96380ffda0 2024-04-23T02:32:27.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.471+0000 7f9637fff700 1 --2- 172.21.15.71:0/1486144194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96380ffda0 0x7f96381976c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f962000d8d0 tx=0x7f962000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:27.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.471+0000 7f96357fa700 1 -- 172.21.15.71:0/1486144194 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96200098a0 con 0x7f96380ffda0 2024-04-23T02:32:27.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.471+0000 7f96357fa700 1 -- 172.21.15.71:0/1486144194 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9620010460 con 0x7f96380ffda0 2024-04-23T02:32:27.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.471+0000 7f963e5c0700 1 -- 172.21.15.71:0/1486144194 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96381a0730 con 0x7f96380ffda0 2024-04-23T02:32:27.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.471+0000 7f96357fa700 1 -- 172.21.15.71:0/1486144194 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9620009ea0 con 0x7f96380ffda0 2024-04-23T02:32:27.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.472+0000 7f963e5c0700 1 -- 172.21.15.71:0/1486144194 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96381a0bf0 con 0x7f96380ffda0 2024-04-23T02:32:27.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.473+0000 7f963e5c0700 1 -- 172.21.15.71:0/1486144194 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96381081a0 con 0x7f96380ffda0 2024-04-23T02:32:27.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.473+0000 7f96357fa700 1 -- 172.21.15.71:0/1486144194 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f96200105d0 con 0x7f96380ffda0 2024-04-23T02:32:27.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.473+0000 7f96357fa700 1 --2- 172.21.15.71:0/1486144194 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f962406cce0 0x7f962406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:27.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.474+0000 7f96357fa700 1 -- 172.21.15.71:0/1486144194 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f962008c480 con 0x7f96380ffda0 2024-04-23T02:32:27.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.474+0000 7f96377fe700 1 --2- 172.21.15.71:0/1486144194 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f962406cce0 0x7f962406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:27.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.474+0000 7f96377fe700 1 --2- 172.21.15.71:0/1486144194 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f962406cce0 0x7f962406f1a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9628006010 tx=0x7f962801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:27.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.478+0000 7f96357fa700 1 -- 172.21.15.71:0/1486144194 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f96200564f0 con 0x7f96380ffda0 2024-04-23T02:32:27.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.688+0000 7f963e5c0700 1 -- 172.21.15.71:0/1486144194 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f96380611d0 con 0x7f962406cce0 2024-04-23T02:32:27.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.689+0000 7f96357fa700 1 -- 172.21.15.71:0/1486144194 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f96380611d0 con 0x7f962406cce0 2024-04-23T02:32:27.692 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:32:27.693 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:32:27.693 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:32:27.693 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:32:27.693 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:32:27.693 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:32:27.693 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:32:27.693 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:32:27.693 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:32:27.693 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:32:27.693 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:32:27.693 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:32:27.693 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:32:27.694 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:32:27.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.691+0000 7f963e5c0700 1 -- 172.21.15.71:0/1486144194 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f962406cce0 msgr2=0x7f962406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:27.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.691+0000 7f963e5c0700 1 --2- 172.21.15.71:0/1486144194 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f962406cce0 0x7f962406f1a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9628006010 tx=0x7f962801a040 comp rx=0 tx=0).stop 2024-04-23T02:32:27.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.691+0000 7f963e5c0700 1 -- 172.21.15.71:0/1486144194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96380ffda0 msgr2=0x7f96381976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:27.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.691+0000 7f963e5c0700 1 --2- 172.21.15.71:0/1486144194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96380ffda0 0x7f96381976c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f962000d8d0 tx=0x7f962000dbe0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.691+0000 7f963e5c0700 1 -- 172.21.15.71:0/1486144194 shutdown_connections 2024-04-23T02:32:27.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.691+0000 7f963e5c0700 1 --2- 172.21.15.71:0/1486144194 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f962406cce0 0x7f962406f1a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.691+0000 7f963e5c0700 1 --2- 172.21.15.71:0/1486144194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96380ffda0 0x7f96381976c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.691+0000 7f963e5c0700 1 --2- 172.21.15.71:0/1486144194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9638100750 0x7f9638197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.691+0000 7f963e5c0700 1 -- 172.21.15.71:0/1486144194 >> 172.21.15.71:0/1486144194 conn(0x7f96380fb650 msgr2=0x7f96380fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:27.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.691+0000 7f963e5c0700 1 -- 172.21.15.71:0/1486144194 shutdown_connections 2024-04-23T02:32:27.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.691+0000 7f963e5c0700 1 -- 172.21.15.71:0/1486144194 wait complete. 2024-04-23T02:32:27.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.793+0000 7f145a88d700 1 -- 172.21.15.71:0/123863510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14540ffc20 msgr2=0x7f1454100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:27.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.793+0000 7f145a88d700 1 --2- 172.21.15.71:0/123863510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14540ffc20 0x7f1454100080 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f143c009910 tx=0x7f143c009c20 comp rx=0 tx=0).stop 2024-04-23T02:32:27.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.793+0000 7f145a88d700 1 -- 172.21.15.71:0/123863510 shutdown_connections 2024-04-23T02:32:27.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.793+0000 7f145a88d700 1 --2- 172.21.15.71:0/123863510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14540ffc20 0x7f1454100080 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.793+0000 7f145a88d700 1 --2- 172.21.15.71:0/123863510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1454105bd0 0x7f1454105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.793+0000 7f145a88d700 1 -- 172.21.15.71:0/123863510 >> 172.21.15.71:0/123863510 conn(0x7f1454075670 msgr2=0x7f1454075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:27.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.793+0000 7f145a88d700 1 -- 172.21.15.71:0/123863510 shutdown_connections 2024-04-23T02:32:27.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.793+0000 7f145a88d700 1 -- 172.21.15.71:0/123863510 wait complete. 2024-04-23T02:32:27.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.793+0000 7f145a88d700 1 Processor -- start 2024-04-23T02:32:27.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.794+0000 7f145a88d700 1 -- start start 2024-04-23T02:32:27.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.794+0000 7f145a88d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14540ffc20 0x7f145419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:27.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.794+0000 7f145a88d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1454105bd0 0x7f145419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:27.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.794+0000 7f145a88d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14541987e0 con 0x7f1454105bd0 2024-04-23T02:32:27.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.794+0000 7f145a88d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1454198950 con 0x7f14540ffc20 2024-04-23T02:32:27.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.794+0000 7f14537fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1454105bd0 0x7f145419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:27.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.794+0000 7f14537fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1454105bd0 0x7f145419ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49658/0 (socket says 172.21.15.71:49658) 2024-04-23T02:32:27.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.794+0000 7f14537fe700 1 -- 172.21.15.71:0/805388865 learned_addr learned my addr 172.21.15.71:0/805388865 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:32:27.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.794+0000 7f1453fff700 1 --2- 172.21.15.71:0/805388865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14540ffc20 0x7f145419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:27.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.794+0000 7f14537fe700 1 -- 172.21.15.71:0/805388865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14540ffc20 msgr2=0x7f145419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:27.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.794+0000 7f14537fe700 1 --2- 172.21.15.71:0/805388865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14540ffc20 0x7f145419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:27.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.794+0000 7f14537fe700 1 -- 172.21.15.71:0/805388865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143c0095f0 con 0x7f1454105bd0 2024-04-23T02:32:27.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.794+0000 7f1453fff700 1 --2- 172.21.15.71:0/805388865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14540ffc20 0x7f145419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:32:27.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.794+0000 7f14537fe700 1 --2- 172.21.15.71:0/805388865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1454105bd0 0x7f145419ed30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f143c006010 tx=0x7f143c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:27.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.794+0000 7f14517fa700 1 -- 172.21.15.71:0/805388865 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f143c01d070 con 0x7f1454105bd0 2024-04-23T02:32:27.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.795+0000 7f14517fa700 1 -- 172.21.15.71:0/805388865 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f143c00f460 con 0x7f1454105bd0 2024-04-23T02:32:27.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.795+0000 7f145a88d700 1 -- 172.21.15.71:0/805388865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1454198bd0 con 0x7f1454105bd0 2024-04-23T02:32:27.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.795+0000 7f14517fa700 1 -- 172.21.15.71:0/805388865 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f143c021620 con 0x7f1454105bd0 2024-04-23T02:32:27.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.795+0000 7f145a88d700 1 -- 172.21.15.71:0/805388865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14541990c0 con 0x7f1454105bd0 2024-04-23T02:32:27.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.796+0000 7f145a88d700 1 -- 172.21.15.71:0/805388865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f145404f2e0 con 0x7f1454105bd0 2024-04-23T02:32:27.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.796+0000 7f14517fa700 1 -- 172.21.15.71:0/805388865 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f143c00fa40 con 0x7f1454105bd0 2024-04-23T02:32:27.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.796+0000 7f14517fa700 1 --2- 172.21.15.71:0/805388865 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f144006cce0 0x7f144006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:27.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.796+0000 7f14517fa700 1 -- 172.21.15.71:0/805388865 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f143c08d6c0 con 0x7f1454105bd0 2024-04-23T02:32:27.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.796+0000 7f1453fff700 1 --2- 172.21.15.71:0/805388865 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f144006cce0 0x7f144006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:27.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.797+0000 7f1453fff700 1 --2- 172.21.15.71:0/805388865 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f144006cce0 0x7f144006f1a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1444005fd0 tx=0x7f1444005ee0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:27.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:27.801+0000 7f14517fa700 1 -- 172.21.15.71:0/805388865 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f143c0577b0 con 0x7f1454105bd0 2024-04-23T02:32:28.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:28.085+0000 7f145a88d700 1 -- 172.21.15.71:0/805388865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1454199330 con 0x7f1454105bd0 2024-04-23T02:32:28.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:28.085+0000 7f14517fa700 1 -- 172.21.15.71:0/805388865 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f143c05add0 con 0x7f1454105bd0 2024-04-23T02:32:28.088 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:32:28.089 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:32:28.089 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:32:28.089 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:32:28.089 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:32:28.089 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:32:28.089 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:32:28.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:28.087+0000 7f145a88d700 1 -- 172.21.15.71:0/805388865 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f144006cce0 msgr2=0x7f144006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:28.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:28.087+0000 7f145a88d700 1 --2- 172.21.15.71:0/805388865 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f144006cce0 0x7f144006f1a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1444005fd0 tx=0x7f1444005ee0 comp rx=0 tx=0).stop 2024-04-23T02:32:28.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:28.087+0000 7f145a88d700 1 -- 172.21.15.71:0/805388865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1454105bd0 msgr2=0x7f145419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:28.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:28.087+0000 7f145a88d700 1 --2- 172.21.15.71:0/805388865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1454105bd0 0x7f145419ed30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f143c006010 tx=0x7f143c00be20 comp rx=0 tx=0).stop 2024-04-23T02:32:28.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:28.087+0000 7f145a88d700 1 -- 172.21.15.71:0/805388865 shutdown_connections 2024-04-23T02:32:28.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:28.087+0000 7f145a88d700 1 --2- 172.21.15.71:0/805388865 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f144006cce0 0x7f144006f1a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:28.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:28.087+0000 7f145a88d700 1 --2- 172.21.15.71:0/805388865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14540ffc20 0x7f145419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:28.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:28.087+0000 7f145a88d700 1 --2- 172.21.15.71:0/805388865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1454105bd0 0x7f145419ed30 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:28.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:28.087+0000 7f145a88d700 1 -- 172.21.15.71:0/805388865 >> 172.21.15.71:0/805388865 conn(0x7f1454075670 msgr2=0x7f14540687a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:28.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:28.087+0000 7f145a88d700 1 -- 172.21.15.71:0/805388865 shutdown_connections 2024-04-23T02:32:28.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:28.087+0000 7f145a88d700 1 -- 172.21.15.71:0/805388865 wait complete. 2024-04-23T02:32:28.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:27 smithi071 ceph-mon[144544]: from='client.34882 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:32:28.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:27 smithi071 ceph-mon[144544]: pgmap v574: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:28.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:27 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2110898486' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:32:28.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:27 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/795266974' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:32:28.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:27 smithi192 ceph-mon[108617]: from='client.34882 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:32:28.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:27 smithi192 ceph-mon[108617]: pgmap v574: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:28.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:27 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2110898486' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:32:28.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:27 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/795266974' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:32:29.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:28 smithi192 ceph-mon[108617]: from='client.34894 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:32:29.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:28 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/805388865' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:32:29.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:28 smithi071 ceph-mon[144544]: from='client.34894 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:32:29.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:28 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/805388865' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:32:30.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:29 smithi192 ceph-mon[108617]: pgmap v575: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:30.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:29 smithi071 ceph-mon[144544]: pgmap v575: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:32.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:32 smithi192 ceph-mon[108617]: pgmap v576: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:32.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:32 smithi071 ceph-mon[144544]: pgmap v576: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:34.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:34 smithi192 ceph-mon[108617]: pgmap v577: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:34.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:34 smithi071 ceph-mon[144544]: pgmap v577: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:35.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:32:35.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:32:36.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:36 smithi192 ceph-mon[108617]: pgmap v578: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:36.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:36 smithi071 ceph-mon[144544]: pgmap v578: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:38.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:38 smithi192 ceph-mon[108617]: pgmap v579: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:38.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:38 smithi071 ceph-mon[144544]: pgmap v579: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:40.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:40 smithi192 ceph-mon[108617]: pgmap v580: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:40.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:40 smithi071 ceph-mon[144544]: pgmap v580: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:42.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:42 smithi192 ceph-mon[108617]: pgmap v581: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:42.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:42 smithi071 ceph-mon[144544]: pgmap v581: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:44.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:44 smithi192 ceph-mon[108617]: pgmap v582: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:44.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:44 smithi071 ceph-mon[144544]: pgmap v582: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:46.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:46 smithi192 ceph-mon[108617]: pgmap v583: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:46.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:46 smithi071 ceph-mon[144544]: pgmap v583: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:48.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:48 smithi192 ceph-mon[108617]: pgmap v584: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:48.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:48 smithi071 ceph-mon[144544]: pgmap v584: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:50.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:50 smithi192 ceph-mon[108617]: pgmap v585: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:50.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:32:50.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:50 smithi071 ceph-mon[144544]: pgmap v585: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:50.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:32:52.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:52 smithi192 ceph-mon[108617]: pgmap v586: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:52.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:52 smithi071 ceph-mon[144544]: pgmap v586: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:54.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:54 smithi192 ceph-mon[108617]: pgmap v587: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:54.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:54 smithi071 ceph-mon[144544]: pgmap v587: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:56.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:56 smithi192 ceph-mon[108617]: pgmap v588: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:56.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:56 smithi071 ceph-mon[144544]: pgmap v588: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:58.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.206+0000 7fe7e79fb700 1 -- 172.21.15.71:0/1472156545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7e0105350 msgr2=0x7fe7e0105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:58.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.206+0000 7fe7e79fb700 1 --2- 172.21.15.71:0/1472156545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7e0105350 0x7fe7e0105730 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009910 tx=0x7fe7d0009c20 comp rx=0 tx=0).stop 2024-04-23T02:32:58.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.206+0000 7fe7e79fb700 1 -- 172.21.15.71:0/1472156545 shutdown_connections 2024-04-23T02:32:58.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.206+0000 7fe7e79fb700 1 --2- 172.21.15.71:0/1472156545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7e00ff340 0x7fe7e00ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:58.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.206+0000 7fe7e79fb700 1 --2- 172.21.15.71:0/1472156545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7e0105350 0x7fe7e0105730 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:58.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.206+0000 7fe7e79fb700 1 -- 172.21.15.71:0/1472156545 >> 172.21.15.71:0/1472156545 conn(0x7fe7e0074e30 msgr2=0x7fe7e0075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:58.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.206+0000 7fe7e79fb700 1 -- 172.21.15.71:0/1472156545 shutdown_connections 2024-04-23T02:32:58.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.206+0000 7fe7e79fb700 1 -- 172.21.15.71:0/1472156545 wait complete. 2024-04-23T02:32:58.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.206+0000 7fe7e79fb700 1 Processor -- start 2024-04-23T02:32:58.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.207+0000 7fe7e79fb700 1 -- start start 2024-04-23T02:32:58.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.207+0000 7fe7e79fb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7e00ff340 0x7fe7e0100fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:58.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.207+0000 7fe7e79fb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7e0105350 0x7fe7e0101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:58.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.207+0000 7fe7e79fb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0101a40 con 0x7fe7e0105350 2024-04-23T02:32:58.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.207+0000 7fe7e79fb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0101b80 con 0x7fe7e00ff340 2024-04-23T02:32:58.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.207+0000 7fe7e4f96700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7e0105350 0x7fe7e0101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:58.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.207+0000 7fe7e4f96700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7e0105350 0x7fe7e0101500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37170/0 (socket says 172.21.15.71:37170) 2024-04-23T02:32:58.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.207+0000 7fe7e4f96700 1 -- 172.21.15.71:0/369736780 learned_addr learned my addr 172.21.15.71:0/369736780 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:32:58.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.207+0000 7fe7e5797700 1 --2- 172.21.15.71:0/369736780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7e00ff340 0x7fe7e0100fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:58.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.207+0000 7fe7e4f96700 1 -- 172.21.15.71:0/369736780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7e00ff340 msgr2=0x7fe7e0100fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:58.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.207+0000 7fe7e4f96700 1 --2- 172.21.15.71:0/369736780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7e00ff340 0x7fe7e0100fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:58.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.207+0000 7fe7e4f96700 1 -- 172.21.15.71:0/369736780 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d00095f0 con 0x7fe7e0105350 2024-04-23T02:32:58.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.207+0000 7fe7e4f96700 1 --2- 172.21.15.71:0/369736780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7e0105350 0x7fe7e0101500 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00c8a0 tx=0x7fe7dc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:58.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.208+0000 7fe7d67fc700 1 -- 172.21.15.71:0/369736780 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7dc0078e0 con 0x7fe7e0105350 2024-04-23T02:32:58.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.208+0000 7fe7d67fc700 1 -- 172.21.15.71:0/369736780 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe7dc00f450 con 0x7fe7e0105350 2024-04-23T02:32:58.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.208+0000 7fe7d67fc700 1 -- 172.21.15.71:0/369736780 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7dc00e5e0 con 0x7fe7e0105350 2024-04-23T02:32:58.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.208+0000 7fe7e79fb700 1 -- 172.21.15.71:0/369736780 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e01a2950 con 0x7fe7e0105350 2024-04-23T02:32:58.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.208+0000 7fe7e79fb700 1 -- 172.21.15.71:0/369736780 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e01a2d00 con 0x7fe7e0105350 2024-04-23T02:32:58.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.209+0000 7fe7e79fb700 1 -- 172.21.15.71:0/369736780 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e0109880 con 0x7fe7e0105350 2024-04-23T02:32:58.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.209+0000 7fe7d67fc700 1 -- 172.21.15.71:0/369736780 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe7dc00f5c0 con 0x7fe7e0105350 2024-04-23T02:32:58.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.209+0000 7fe7d67fc700 1 --2- 172.21.15.71:0/369736780 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe7cc06cce0 0x7fe7cc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:58.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.210+0000 7fe7d67fc700 1 -- 172.21.15.71:0/369736780 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe7dc08c1d0 con 0x7fe7e0105350 2024-04-23T02:32:58.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.210+0000 7fe7e5797700 1 --2- 172.21.15.71:0/369736780 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe7cc06cce0 0x7fe7cc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:58.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.210+0000 7fe7e5797700 1 --2- 172.21.15.71:0/369736780 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe7cc06cce0 0x7fe7cc06f1a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0006010 tx=0x7fe7d001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:58.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.214+0000 7fe7d67fc700 1 -- 172.21.15.71:0/369736780 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe7dc056240 con 0x7fe7e0105350 2024-04-23T02:32:58.302 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:58 smithi192 ceph-mon[108617]: pgmap v589: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:58.340 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:58 smithi071 ceph-mon[144544]: pgmap v589: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:32:58.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.433+0000 7fe7e79fb700 1 -- 172.21.15.71:0/369736780 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7e002ccc0 con 0x7fe7cc06cce0 2024-04-23T02:32:58.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.434+0000 7fe7d67fc700 1 -- 172.21.15.71:0/369736780 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe7e002ccc0 con 0x7fe7cc06cce0 2024-04-23T02:32:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.436+0000 7fe7e79fb700 1 -- 172.21.15.71:0/369736780 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe7cc06cce0 msgr2=0x7fe7cc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.436+0000 7fe7e79fb700 1 --2- 172.21.15.71:0/369736780 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe7cc06cce0 0x7fe7cc06f1a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0006010 tx=0x7fe7d001a040 comp rx=0 tx=0).stop 2024-04-23T02:32:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.436+0000 7fe7e79fb700 1 -- 172.21.15.71:0/369736780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7e0105350 msgr2=0x7fe7e0101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.436+0000 7fe7e79fb700 1 --2- 172.21.15.71:0/369736780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7e0105350 0x7fe7e0101500 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00c8a0 tx=0x7fe7dc00cbb0 comp rx=0 tx=0).stop 2024-04-23T02:32:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.437+0000 7fe7e79fb700 1 -- 172.21.15.71:0/369736780 shutdown_connections 2024-04-23T02:32:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.437+0000 7fe7e79fb700 1 --2- 172.21.15.71:0/369736780 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe7cc06cce0 0x7fe7cc06f1a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.437+0000 7fe7e79fb700 1 --2- 172.21.15.71:0/369736780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7e00ff340 0x7fe7e0100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.437+0000 7fe7e79fb700 1 --2- 172.21.15.71:0/369736780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7e0105350 0x7fe7e0101500 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:58.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.437+0000 7fe7e79fb700 1 -- 172.21.15.71:0/369736780 >> 172.21.15.71:0/369736780 conn(0x7fe7e0074e30 msgr2=0x7fe7e0103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:58.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.437+0000 7fe7e79fb700 1 -- 172.21.15.71:0/369736780 shutdown_connections 2024-04-23T02:32:58.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.437+0000 7fe7e79fb700 1 -- 172.21.15.71:0/369736780 wait complete. 2024-04-23T02:32:58.449 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:32:58.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.541+0000 7f1a45ede700 1 -- 172.21.15.71:0/537469297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a400ffc20 msgr2=0x7f1a40100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:58.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.541+0000 7f1a45ede700 1 --2- 172.21.15.71:0/537469297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a400ffc20 0x7f1a40100080 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009910 tx=0x7f1a30009c20 comp rx=0 tx=0).stop 2024-04-23T02:32:58.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.541+0000 7f1a45ede700 1 -- 172.21.15.71:0/537469297 shutdown_connections 2024-04-23T02:32:58.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.541+0000 7f1a45ede700 1 --2- 172.21.15.71:0/537469297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a400ffc20 0x7f1a40100080 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:58.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.541+0000 7f1a45ede700 1 --2- 172.21.15.71:0/537469297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a40105bd0 0x7f1a40105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:58.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.541+0000 7f1a45ede700 1 -- 172.21.15.71:0/537469297 >> 172.21.15.71:0/537469297 conn(0x7f1a40075670 msgr2=0x7f1a40075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:58.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.541+0000 7f1a45ede700 1 -- 172.21.15.71:0/537469297 shutdown_connections 2024-04-23T02:32:58.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.541+0000 7f1a45ede700 1 -- 172.21.15.71:0/537469297 wait complete. 2024-04-23T02:32:58.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.542+0000 7f1a45ede700 1 Processor -- start 2024-04-23T02:32:58.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.542+0000 7f1a45ede700 1 -- start start 2024-04-23T02:32:58.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.542+0000 7f1a45ede700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a400ffc20 0x7f1a401954b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:58.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.542+0000 7f1a45ede700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a40105bd0 0x7f1a401959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:58.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.542+0000 7f1a45ede700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a40195f30 con 0x7f1a40105bd0 2024-04-23T02:32:58.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.542+0000 7f1a45ede700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a401960a0 con 0x7f1a400ffc20 2024-04-23T02:32:58.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.542+0000 7f1a3effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a40105bd0 0x7f1a401959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:58.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.542+0000 7f1a3effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a40105bd0 0x7f1a401959f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37174/0 (socket says 172.21.15.71:37174) 2024-04-23T02:32:58.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.542+0000 7f1a3effd700 1 -- 172.21.15.71:0/1184544240 learned_addr learned my addr 172.21.15.71:0/1184544240 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:32:58.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.542+0000 7f1a3f7fe700 1 --2- 172.21.15.71:0/1184544240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a400ffc20 0x7f1a401954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:58.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.543+0000 7f1a3effd700 1 -- 172.21.15.71:0/1184544240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a400ffc20 msgr2=0x7f1a401954b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:58.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.543+0000 7f1a3effd700 1 --2- 172.21.15.71:0/1184544240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a400ffc20 0x7f1a401954b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:58.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.543+0000 7f1a3effd700 1 -- 172.21.15.71:0/1184544240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a300095f0 con 0x7f1a40105bd0 2024-04-23T02:32:58.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.543+0000 7f1a3effd700 1 --2- 172.21.15.71:0/1184544240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a40105bd0 0x7f1a401959f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1a30006010 tx=0x7f1a3000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:58.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.543+0000 7f1a3cff9700 1 -- 172.21.15.71:0/1184544240 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a3001d070 con 0x7f1a40105bd0 2024-04-23T02:32:58.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.543+0000 7f1a45ede700 1 -- 172.21.15.71:0/1184544240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a40196320 con 0x7f1a40105bd0 2024-04-23T02:32:58.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.543+0000 7f1a3cff9700 1 -- 172.21.15.71:0/1184544240 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1a30005710 con 0x7f1a40105bd0 2024-04-23T02:32:58.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.543+0000 7f1a3cff9700 1 -- 172.21.15.71:0/1184544240 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a3000fe20 con 0x7f1a40105bd0 2024-04-23T02:32:58.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.543+0000 7f1a45ede700 1 -- 172.21.15.71:0/1184544240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a401a2da0 con 0x7f1a40105bd0 2024-04-23T02:32:58.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.544+0000 7f1a45ede700 1 -- 172.21.15.71:0/1184544240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a401080d0 con 0x7f1a40105bd0 2024-04-23T02:32:58.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.544+0000 7f1a3cff9700 1 -- 172.21.15.71:0/1184544240 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1a30003a60 con 0x7f1a40105bd0 2024-04-23T02:32:58.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.545+0000 7f1a3cff9700 1 --2- 172.21.15.71:0/1184544240 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1a2806cce0 0x7f1a2806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:58.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.545+0000 7f1a3cff9700 1 -- 172.21.15.71:0/1184544240 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1a3008d600 con 0x7f1a40105bd0 2024-04-23T02:32:58.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.545+0000 7f1a3f7fe700 1 --2- 172.21.15.71:0/1184544240 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1a2806cce0 0x7f1a2806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:58.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.545+0000 7f1a3f7fe700 1 --2- 172.21.15.71:0/1184544240 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1a2806cce0 0x7f1a2806f1a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1a34005950 tx=0x7f1a3400b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:58.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.549+0000 7f1a3cff9700 1 -- 172.21.15.71:0/1184544240 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1a3005b100 con 0x7f1a40105bd0 2024-04-23T02:32:58.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.758+0000 7f1a45ede700 1 -- 172.21.15.71:0/1184544240 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a40196320 con 0x7f1a2806cce0 2024-04-23T02:32:58.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.759+0000 7f1a3cff9700 1 -- 172.21.15.71:0/1184544240 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1a40196320 con 0x7f1a2806cce0 2024-04-23T02:32:58.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.761+0000 7f1a45ede700 1 -- 172.21.15.71:0/1184544240 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1a2806cce0 msgr2=0x7f1a2806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:58.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.761+0000 7f1a45ede700 1 --2- 172.21.15.71:0/1184544240 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1a2806cce0 0x7f1a2806f1a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1a34005950 tx=0x7f1a3400b410 comp rx=0 tx=0).stop 2024-04-23T02:32:58.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.761+0000 7f1a45ede700 1 -- 172.21.15.71:0/1184544240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a40105bd0 msgr2=0x7f1a401959f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:58.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.761+0000 7f1a45ede700 1 --2- 172.21.15.71:0/1184544240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a40105bd0 0x7f1a401959f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1a30006010 tx=0x7f1a3000bd70 comp rx=0 tx=0).stop 2024-04-23T02:32:58.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.761+0000 7f1a45ede700 1 -- 172.21.15.71:0/1184544240 shutdown_connections 2024-04-23T02:32:58.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.761+0000 7f1a45ede700 1 --2- 172.21.15.71:0/1184544240 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1a2806cce0 0x7f1a2806f1a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:58.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.761+0000 7f1a45ede700 1 --2- 172.21.15.71:0/1184544240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a400ffc20 0x7f1a401954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:58.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.761+0000 7f1a45ede700 1 --2- 172.21.15.71:0/1184544240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a40105bd0 0x7f1a401959f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:58.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.761+0000 7f1a45ede700 1 -- 172.21.15.71:0/1184544240 >> 172.21.15.71:0/1184544240 conn(0x7f1a40075670 msgr2=0x7f1a400fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:58.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.762+0000 7f1a45ede700 1 -- 172.21.15.71:0/1184544240 shutdown_connections 2024-04-23T02:32:58.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.762+0000 7f1a45ede700 1 -- 172.21.15.71:0/1184544240 wait complete. 2024-04-23T02:32:58.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.864+0000 7fc6edca8700 1 -- 172.21.15.71:0/3440565180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6e80ff5a0 msgr2=0x7fc6e80ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:58.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.864+0000 7fc6edca8700 1 --2- 172.21.15.71:0/3440565180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6e80ff5a0 0x7fc6e80ff980 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009910 tx=0x7fc6d8009c20 comp rx=0 tx=0).stop 2024-04-23T02:32:58.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.864+0000 7fc6edca8700 1 -- 172.21.15.71:0/3440565180 shutdown_connections 2024-04-23T02:32:58.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.864+0000 7fc6edca8700 1 --2- 172.21.15.71:0/3440565180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6e80fff50 0x7fc6e8105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:58.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.864+0000 7fc6edca8700 1 --2- 172.21.15.71:0/3440565180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6e80ff5a0 0x7fc6e80ff980 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:58.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.864+0000 7fc6edca8700 1 -- 172.21.15.71:0/3440565180 >> 172.21.15.71:0/3440565180 conn(0x7fc6e8074e30 msgr2=0x7fc6e8075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:58.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.864+0000 7fc6edca8700 1 -- 172.21.15.71:0/3440565180 shutdown_connections 2024-04-23T02:32:58.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.864+0000 7fc6edca8700 1 -- 172.21.15.71:0/3440565180 wait complete. 2024-04-23T02:32:58.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.865+0000 7fc6edca8700 1 Processor -- start 2024-04-23T02:32:58.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.865+0000 7fc6edca8700 1 -- start start 2024-04-23T02:32:58.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.865+0000 7fc6edca8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6e80ff5a0 0x7fc6e8100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:58.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.865+0000 7fc6edca8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6e80fff50 0x7fc6e8100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:58.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.865+0000 7fc6edca8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e8100fa0 con 0x7fc6e80fff50 2024-04-23T02:32:58.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.865+0000 7fc6edca8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e81010e0 con 0x7fc6e80ff5a0 2024-04-23T02:32:58.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.865+0000 7fc6e6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6e80fff50 0x7fc6e8100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:58.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.865+0000 7fc6e6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6e80fff50 0x7fc6e8100a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37186/0 (socket says 172.21.15.71:37186) 2024-04-23T02:32:58.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.865+0000 7fc6e6ffd700 1 -- 172.21.15.71:0/1789456371 learned_addr learned my addr 172.21.15.71:0/1789456371 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:32:58.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.865+0000 7fc6e77fe700 1 --2- 172.21.15.71:0/1789456371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6e80ff5a0 0x7fc6e8100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:58.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.866+0000 7fc6e6ffd700 1 -- 172.21.15.71:0/1789456371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6e80ff5a0 msgr2=0x7fc6e8100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:58.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.866+0000 7fc6e6ffd700 1 --2- 172.21.15.71:0/1789456371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6e80ff5a0 0x7fc6e8100520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:58.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.866+0000 7fc6e6ffd700 1 -- 172.21.15.71:0/1789456371 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d80095f0 con 0x7fc6e80fff50 2024-04-23T02:32:58.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.866+0000 7fc6e6ffd700 1 --2- 172.21.15.71:0/1789456371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6e80fff50 0x7fc6e8100a60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000d8d0 tx=0x7fc6d000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:58.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.866+0000 7fc6e4ff9700 1 -- 172.21.15.71:0/1789456371 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6d0004a00 con 0x7fc6e80fff50 2024-04-23T02:32:58.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.866+0000 7fc6e4ff9700 1 -- 172.21.15.71:0/1789456371 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc6d0005020 con 0x7fc6e80fff50 2024-04-23T02:32:58.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.866+0000 7fc6edca8700 1 -- 172.21.15.71:0/1789456371 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e81a29a0 con 0x7fc6e80fff50 2024-04-23T02:32:58.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.866+0000 7fc6e4ff9700 1 -- 172.21.15.71:0/1789456371 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6d000fb80 con 0x7fc6e80fff50 2024-04-23T02:32:58.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.866+0000 7fc6edca8700 1 -- 172.21.15.71:0/1789456371 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e81a2db0 con 0x7fc6e80fff50 2024-04-23T02:32:58.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.867+0000 7fc6edca8700 1 -- 172.21.15.71:0/1789456371 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e8109270 con 0x7fc6e80fff50 2024-04-23T02:32:58.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.867+0000 7fc6e4ff9700 1 -- 172.21.15.71:0/1789456371 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc6d000fce0 con 0x7fc6e80fff50 2024-04-23T02:32:58.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.868+0000 7fc6e4ff9700 1 --2- 172.21.15.71:0/1789456371 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc6d406cce0 0x7fc6d406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:58.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.868+0000 7fc6e4ff9700 1 -- 172.21.15.71:0/1789456371 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc6d008bdb0 con 0x7fc6e80fff50 2024-04-23T02:32:58.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.868+0000 7fc6e77fe700 1 --2- 172.21.15.71:0/1789456371 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc6d406cce0 0x7fc6d406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:58.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.868+0000 7fc6e77fe700 1 --2- 172.21.15.71:0/1789456371 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc6d406cce0 0x7fc6d406f1a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8006010 tx=0x7fc6d801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:58.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:58.872+0000 7fc6e4ff9700 1 -- 172.21.15.71:0/1789456371 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc6d0055ea0 con 0x7fc6e80fff50 2024-04-23T02:32:59.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.079+0000 7fc6edca8700 1 -- 172.21.15.71:0/1789456371 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6e80611d0 con 0x7fc6d406cce0 2024-04-23T02:32:59.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.090+0000 7fc6e4ff9700 1 -- 172.21.15.71:0/1789456371 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc6e80611d0 con 0x7fc6d406cce0 2024-04-23T02:32:59.092 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:32:59.092 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (16m) 10m ago 27m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:32:59.093 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (28m) 10m ago 28m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:32:59.093 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (27m) 10m ago 27m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:32:59.093 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (14m) 10m ago 28m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:32:59.093 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (14m) 10m ago 27m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:32:59.093 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (16m) 10m ago 27m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:32:59.093 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (10m) 10m ago 22m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:32:59.093 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (10m) 10m ago 21m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:32:59.093 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (21m) 10m ago 21m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:32:59.093 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (10m) 10m ago 22m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:32:59.093 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (18m) 10m ago 29m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:32:59.093 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (17m) 10m ago 26m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:32:59.093 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (15m) 10m ago 29m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:32:59.093 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (14m) 10m ago 26m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:32:59.093 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (17m) 10m ago 28m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:32:59.093 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (17m) 10m ago 26m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:32:59.094 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (14m) 10m ago 26m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:32:59.094 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (13m) 10m ago 25m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:32:59.094 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (12m) 10m ago 24m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:32:59.094 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (12m) 10m ago 24m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:32:59.094 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (11m) 10m ago 24m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:32:59.094 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (11m) 10m ago 23m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:32:59.094 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (16m) 10m ago 27m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:32:59.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.092+0000 7fc6edca8700 1 -- 172.21.15.71:0/1789456371 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc6d406cce0 msgr2=0x7fc6d406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:59.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.092+0000 7fc6edca8700 1 --2- 172.21.15.71:0/1789456371 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc6d406cce0 0x7fc6d406f1a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8006010 tx=0x7fc6d801a040 comp rx=0 tx=0).stop 2024-04-23T02:32:59.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.092+0000 7fc6edca8700 1 -- 172.21.15.71:0/1789456371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6e80fff50 msgr2=0x7fc6e8100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:59.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.092+0000 7fc6edca8700 1 --2- 172.21.15.71:0/1789456371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6e80fff50 0x7fc6e8100a60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000d8d0 tx=0x7fc6d000dbe0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.092+0000 7fc6edca8700 1 -- 172.21.15.71:0/1789456371 shutdown_connections 2024-04-23T02:32:59.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.092+0000 7fc6edca8700 1 --2- 172.21.15.71:0/1789456371 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc6d406cce0 0x7fc6d406f1a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.092+0000 7fc6edca8700 1 --2- 172.21.15.71:0/1789456371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6e80ff5a0 0x7fc6e8100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.092+0000 7fc6edca8700 1 --2- 172.21.15.71:0/1789456371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6e80fff50 0x7fc6e8100a60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.092+0000 7fc6edca8700 1 -- 172.21.15.71:0/1789456371 >> 172.21.15.71:0/1789456371 conn(0x7fc6e8074e30 msgr2=0x7fc6e80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:59.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.092+0000 7fc6edca8700 1 -- 172.21.15.71:0/1789456371 shutdown_connections 2024-04-23T02:32:59.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.092+0000 7fc6edca8700 1 -- 172.21.15.71:0/1789456371 wait complete. 2024-04-23T02:32:59.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.194+0000 7f0a657d5700 1 -- 172.21.15.71:0/2396517659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a600ffc20 msgr2=0x7f0a60100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:59.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.194+0000 7f0a657d5700 1 --2- 172.21.15.71:0/2396517659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a600ffc20 0x7f0a60100080 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009910 tx=0x7f0a50009c20 comp rx=0 tx=0).stop 2024-04-23T02:32:59.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.194+0000 7f0a657d5700 1 -- 172.21.15.71:0/2396517659 shutdown_connections 2024-04-23T02:32:59.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.194+0000 7f0a657d5700 1 --2- 172.21.15.71:0/2396517659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a600ffc20 0x7f0a60100080 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.194+0000 7f0a657d5700 1 --2- 172.21.15.71:0/2396517659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a60105bd0 0x7f0a60105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.194+0000 7f0a657d5700 1 -- 172.21.15.71:0/2396517659 >> 172.21.15.71:0/2396517659 conn(0x7f0a60075670 msgr2=0x7f0a60075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:59.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.194+0000 7f0a657d5700 1 -- 172.21.15.71:0/2396517659 shutdown_connections 2024-04-23T02:32:59.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.194+0000 7f0a657d5700 1 -- 172.21.15.71:0/2396517659 wait complete. 2024-04-23T02:32:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.194+0000 7f0a657d5700 1 Processor -- start 2024-04-23T02:32:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.194+0000 7f0a657d5700 1 -- start start 2024-04-23T02:32:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.195+0000 7f0a657d5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a600ffc20 0x7f0a6019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.195+0000 7f0a657d5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a60105bd0 0x7f0a6019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.195+0000 7f0a657d5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6019d270 con 0x7f0a600ffc20 2024-04-23T02:32:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.195+0000 7f0a657d5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a60196720 con 0x7f0a60105bd0 2024-04-23T02:32:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.195+0000 7f0a5effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a600ffc20 0x7f0a6019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.195+0000 7f0a5effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a600ffc20 0x7f0a6019c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37200/0 (socket says 172.21.15.71:37200) 2024-04-23T02:32:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.195+0000 7f0a5effd700 1 -- 172.21.15.71:0/1305596629 learned_addr learned my addr 172.21.15.71:0/1305596629 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:32:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.195+0000 7f0a5e7fc700 1 --2- 172.21.15.71:0/1305596629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a60105bd0 0x7f0a6019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.195+0000 7f0a5effd700 1 -- 172.21.15.71:0/1305596629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a60105bd0 msgr2=0x7f0a6019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.195+0000 7f0a5effd700 1 --2- 172.21.15.71:0/1305596629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a60105bd0 0x7f0a6019cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.195+0000 7f0a5effd700 1 -- 172.21.15.71:0/1305596629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a500095f0 con 0x7f0a600ffc20 2024-04-23T02:32:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.195+0000 7f0a5effd700 1 --2- 172.21.15.71:0/1305596629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a600ffc20 0x7f0a6019c6a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400b4b0 tx=0x7f0a5400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.195+0000 7f0a47fff700 1 -- 172.21.15.71:0/1305596629 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a54011800 con 0x7f0a600ffc20 2024-04-23T02:32:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.195+0000 7f0a657d5700 1 -- 172.21.15.71:0/1305596629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a60196a00 con 0x7f0a600ffc20 2024-04-23T02:32:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.195+0000 7f0a47fff700 1 -- 172.21.15.71:0/1305596629 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0a54011e00 con 0x7f0a600ffc20 2024-04-23T02:32:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.195+0000 7f0a47fff700 1 -- 172.21.15.71:0/1305596629 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a54010e50 con 0x7f0a600ffc20 2024-04-23T02:32:59.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.195+0000 7f0a657d5700 1 -- 172.21.15.71:0/1305596629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a60196f50 con 0x7f0a600ffc20 2024-04-23T02:32:59.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.196+0000 7f0a657d5700 1 -- 172.21.15.71:0/1305596629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6004f2e0 con 0x7f0a600ffc20 2024-04-23T02:32:59.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.197+0000 7f0a47fff700 1 -- 172.21.15.71:0/1305596629 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0a54010380 con 0x7f0a600ffc20 2024-04-23T02:32:59.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.197+0000 7f0a47fff700 1 --2- 172.21.15.71:0/1305596629 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0a4806cce0 0x7f0a4806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:59.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.197+0000 7f0a47fff700 1 -- 172.21.15.71:0/1305596629 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0a5408c170 con 0x7f0a600ffc20 2024-04-23T02:32:59.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.197+0000 7f0a5e7fc700 1 --2- 172.21.15.71:0/1305596629 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0a4806cce0 0x7f0a4806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:59.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.198+0000 7f0a5e7fc700 1 --2- 172.21.15.71:0/1305596629 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0a4806cce0 0x7f0a4806f1a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0a50000c00 tx=0x7f0a50005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:59.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.202+0000 7f0a47fff700 1 -- 172.21.15.71:0/1305596629 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0a54059c70 con 0x7f0a600ffc20 2024-04-23T02:32:59.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:32:59 smithi192 ceph-mon[108617]: from='client.34902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:32:59.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.480+0000 7f0a657d5700 1 -- 172.21.15.71:0/1305596629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0a60197c30 con 0x7f0a600ffc20 2024-04-23T02:32:59.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.481+0000 7f0a47fff700 1 -- 172.21.15.71:0/1305596629 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f0a54059800 con 0x7f0a600ffc20 2024-04-23T02:32:59.483 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:32:59.483 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:32:59.483 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:32:59.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:32:59.483 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:32:59.483 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:32:59.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:32:59.483 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:32:59.484 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:32:59.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:32:59.484 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:32:59.484 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:32:59.484 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:32:59.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:32:59.484 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:32:59.484 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:32:59.484 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:32:59.484 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:32:59.484 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:32:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.483+0000 7f0a657d5700 1 -- 172.21.15.71:0/1305596629 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0a4806cce0 msgr2=0x7f0a4806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.483+0000 7f0a657d5700 1 --2- 172.21.15.71:0/1305596629 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0a4806cce0 0x7f0a4806f1a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0a50000c00 tx=0x7f0a50005fb0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.483+0000 7f0a657d5700 1 -- 172.21.15.71:0/1305596629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a600ffc20 msgr2=0x7f0a6019c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.483+0000 7f0a657d5700 1 --2- 172.21.15.71:0/1305596629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a600ffc20 0x7f0a6019c6a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400b4b0 tx=0x7f0a5400b870 comp rx=0 tx=0).stop 2024-04-23T02:32:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.483+0000 7f0a657d5700 1 -- 172.21.15.71:0/1305596629 shutdown_connections 2024-04-23T02:32:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.483+0000 7f0a657d5700 1 --2- 172.21.15.71:0/1305596629 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0a4806cce0 0x7f0a4806f1a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.483+0000 7f0a657d5700 1 --2- 172.21.15.71:0/1305596629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a600ffc20 0x7f0a6019c6a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.483+0000 7f0a657d5700 1 --2- 172.21.15.71:0/1305596629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a60105bd0 0x7f0a6019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.483+0000 7f0a657d5700 1 -- 172.21.15.71:0/1305596629 >> 172.21.15.71:0/1305596629 conn(0x7f0a60075670 msgr2=0x7f0a600686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.483+0000 7f0a657d5700 1 -- 172.21.15.71:0/1305596629 shutdown_connections 2024-04-23T02:32:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.483+0000 7f0a657d5700 1 -- 172.21.15.71:0/1305596629 wait complete. 2024-04-23T02:32:59.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:32:59 smithi071 ceph-mon[144544]: from='client.34902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:32:59.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.585+0000 7fdc73185700 1 -- 172.21.15.71:0/2943848866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c0ffc20 msgr2=0x7fdc6c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:59.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.585+0000 7fdc73185700 1 --2- 172.21.15.71:0/2943848866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c0ffc20 0x7fdc6c100080 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009910 tx=0x7fdc58009c20 comp rx=0 tx=0).stop 2024-04-23T02:32:59.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.586+0000 7fdc73185700 1 -- 172.21.15.71:0/2943848866 shutdown_connections 2024-04-23T02:32:59.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.586+0000 7fdc73185700 1 --2- 172.21.15.71:0/2943848866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c0ffc20 0x7fdc6c100080 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.586+0000 7fdc73185700 1 --2- 172.21.15.71:0/2943848866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc6c105bd0 0x7fdc6c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.586+0000 7fdc73185700 1 -- 172.21.15.71:0/2943848866 >> 172.21.15.71:0/2943848866 conn(0x7fdc6c075670 msgr2=0x7fdc6c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:59.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.586+0000 7fdc73185700 1 -- 172.21.15.71:0/2943848866 shutdown_connections 2024-04-23T02:32:59.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.586+0000 7fdc73185700 1 -- 172.21.15.71:0/2943848866 wait complete. 2024-04-23T02:32:59.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.586+0000 7fdc73185700 1 Processor -- start 2024-04-23T02:32:59.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.586+0000 7fdc73185700 1 -- start start 2024-04-23T02:32:59.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.586+0000 7fdc73185700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc6c0ffc20 0x7fdc6c19c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:59.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.586+0000 7fdc73185700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c105bd0 0x7fdc6c19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:59.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.586+0000 7fdc73185700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c19d1c0 con 0x7fdc6c105bd0 2024-04-23T02:32:59.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.586+0000 7fdc73185700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c196670 con 0x7fdc6c0ffc20 2024-04-23T02:32:59.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.587+0000 7fdc6bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c105bd0 0x7fdc6c19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:59.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.587+0000 7fdc6bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c105bd0 0x7fdc6c19cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37216/0 (socket says 172.21.15.71:37216) 2024-04-23T02:32:59.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.587+0000 7fdc6bfff700 1 -- 172.21.15.71:0/3383907363 learned_addr learned my addr 172.21.15.71:0/3383907363 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:32:59.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.587+0000 7fdc70f21700 1 --2- 172.21.15.71:0/3383907363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc6c0ffc20 0x7fdc6c19c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:59.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.587+0000 7fdc6bfff700 1 -- 172.21.15.71:0/3383907363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc6c0ffc20 msgr2=0x7fdc6c19c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:59.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.587+0000 7fdc6bfff700 1 --2- 172.21.15.71:0/3383907363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc6c0ffc20 0x7fdc6c19c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.587+0000 7fdc6bfff700 1 -- 172.21.15.71:0/3383907363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc580095f0 con 0x7fdc6c105bd0 2024-04-23T02:32:59.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.587+0000 7fdc6bfff700 1 --2- 172.21.15.71:0/3383907363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c105bd0 0x7fdc6c19cb30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdc580092e0 tx=0x7fdc5800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:59.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.587+0000 7fdc69ffb700 1 -- 172.21.15.71:0/3383907363 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc5801d070 con 0x7fdc6c105bd0 2024-04-23T02:32:59.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.587+0000 7fdc69ffb700 1 -- 172.21.15.71:0/3383907363 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdc58005e00 con 0x7fdc6c105bd0 2024-04-23T02:32:59.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.587+0000 7fdc73185700 1 -- 172.21.15.71:0/3383907363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6c1968f0 con 0x7fdc6c105bd0 2024-04-23T02:32:59.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.587+0000 7fdc69ffb700 1 -- 172.21.15.71:0/3383907363 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc5800fe90 con 0x7fdc6c105bd0 2024-04-23T02:32:59.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.587+0000 7fdc73185700 1 -- 172.21.15.71:0/3383907363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6c196de0 con 0x7fdc6c105bd0 2024-04-23T02:32:59.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.588+0000 7fdc73185700 1 -- 172.21.15.71:0/3383907363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc6c04f2e0 con 0x7fdc6c105bd0 2024-04-23T02:32:59.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.589+0000 7fdc69ffb700 1 -- 172.21.15.71:0/3383907363 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fdc5800f6d0 con 0x7fdc6c105bd0 2024-04-23T02:32:59.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.589+0000 7fdc69ffb700 1 --2- 172.21.15.71:0/3383907363 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdc5c06cce0 0x7fdc5c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:59.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.589+0000 7fdc70f21700 1 --2- 172.21.15.71:0/3383907363 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdc5c06cce0 0x7fdc5c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:59.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.589+0000 7fdc69ffb700 1 -- 172.21.15.71:0/3383907363 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdc5808d7e0 con 0x7fdc6c105bd0 2024-04-23T02:32:59.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.590+0000 7fdc70f21700 1 --2- 172.21.15.71:0/3383907363 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdc5c06cce0 0x7fdc5c06f1a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdc60005950 tx=0x7fdc6000b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:59.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.594+0000 7fdc69ffb700 1 -- 172.21.15.71:0/3383907363 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fdc5805b2e0 con 0x7fdc6c105bd0 2024-04-23T02:32:59.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.829+0000 7fdc73185700 1 -- 172.21.15.71:0/3383907363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdc6c197b80 con 0x7fdc6c105bd0 2024-04-23T02:32:59.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.829+0000 7fdc69ffb700 1 -- 172.21.15.71:0/3383907363 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7fdc5805ae70 con 0x7fdc6c105bd0 2024-04-23T02:32:59.832 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:32:59.832 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:32:59.832 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:32:59.832 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:32:59.832 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:32:59.832 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:32:59.832 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:32:59.832 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:32:59.832 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:32:59.833 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:32:59.833 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:32:59.833 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:32:59.833 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:32:59.833 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:32:59.833 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:32:59.833 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:32:59.833 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:32:59.833 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:32:59.833 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:32:59.833 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:32:59.833 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:32:59.833 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:32:59.834 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:32:59.834 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:32:59.834 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:32:59.834 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:32:59.834 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:32:59.834 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:32:59.834 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:32:59.834 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:32:59.834 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:32:59.834 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:32:59.834 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:32:59.834 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:32:59.835 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:32:59.835 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:32:59.835 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:32:59.835 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:32:59.835 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:32:59.835 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:32:59.835 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:32:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.831+0000 7fdc73185700 1 -- 172.21.15.71:0/3383907363 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdc5c06cce0 msgr2=0x7fdc5c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.831+0000 7fdc73185700 1 --2- 172.21.15.71:0/3383907363 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdc5c06cce0 0x7fdc5c06f1a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdc60005950 tx=0x7fdc6000b410 comp rx=0 tx=0).stop 2024-04-23T02:32:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.831+0000 7fdc73185700 1 -- 172.21.15.71:0/3383907363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c105bd0 msgr2=0x7fdc6c19cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.831+0000 7fdc73185700 1 --2- 172.21.15.71:0/3383907363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c105bd0 0x7fdc6c19cb30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdc580092e0 tx=0x7fdc5800bd70 comp rx=0 tx=0).stop 2024-04-23T02:32:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.831+0000 7fdc73185700 1 -- 172.21.15.71:0/3383907363 shutdown_connections 2024-04-23T02:32:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.831+0000 7fdc73185700 1 --2- 172.21.15.71:0/3383907363 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdc5c06cce0 0x7fdc5c06f1a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.831+0000 7fdc73185700 1 --2- 172.21.15.71:0/3383907363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc6c0ffc20 0x7fdc6c19c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.831+0000 7fdc73185700 1 --2- 172.21.15.71:0/3383907363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c105bd0 0x7fdc6c19cb30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.831+0000 7fdc73185700 1 -- 172.21.15.71:0/3383907363 >> 172.21.15.71:0/3383907363 conn(0x7fdc6c075670 msgr2=0x7fdc6c103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.832+0000 7fdc73185700 1 -- 172.21.15.71:0/3383907363 shutdown_connections 2024-04-23T02:32:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.832+0000 7fdc73185700 1 -- 172.21.15.71:0/3383907363 wait complete. 2024-04-23T02:32:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:32:59.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.935+0000 7f6d2b7df700 1 -- 172.21.15.71:0/572220824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d24100750 msgr2=0x7f6d241047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:59.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.935+0000 7f6d2b7df700 1 --2- 172.21.15.71:0/572220824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d24100750 0x7f6d241047a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009910 tx=0x7f6d14009c20 comp rx=0 tx=0).stop 2024-04-23T02:32:59.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.935+0000 7f6d2b7df700 1 -- 172.21.15.71:0/572220824 shutdown_connections 2024-04-23T02:32:59.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.935+0000 7f6d2b7df700 1 --2- 172.21.15.71:0/572220824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d24100750 0x7f6d241047a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.935+0000 7f6d2b7df700 1 --2- 172.21.15.71:0/572220824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d240ffda0 0x7f6d24100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.935+0000 7f6d2b7df700 1 -- 172.21.15.71:0/572220824 >> 172.21.15.71:0/572220824 conn(0x7f6d240fb650 msgr2=0x7f6d240fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:32:59.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.936+0000 7f6d2b7df700 1 -- 172.21.15.71:0/572220824 shutdown_connections 2024-04-23T02:32:59.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.936+0000 7f6d2b7df700 1 -- 172.21.15.71:0/572220824 wait complete. 2024-04-23T02:32:59.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.936+0000 7f6d2b7df700 1 Processor -- start 2024-04-23T02:32:59.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.936+0000 7f6d2b7df700 1 -- start start 2024-04-23T02:32:59.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.936+0000 7f6d2b7df700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d240ffda0 0x7f6d24197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:59.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.936+0000 7f6d2b7df700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d24100750 0x7f6d24197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:59.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.936+0000 7f6d2b7df700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d24198260 con 0x7f6d240ffda0 2024-04-23T02:32:59.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.936+0000 7f6d2b7df700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d241983a0 con 0x7f6d24100750 2024-04-23T02:32:59.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.936+0000 7f6d2957b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d240ffda0 0x7f6d24197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:59.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.936+0000 7f6d2957b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d240ffda0 0x7f6d24197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37244/0 (socket says 172.21.15.71:37244) 2024-04-23T02:32:59.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.937+0000 7f6d2957b700 1 -- 172.21.15.71:0/3278478728 learned_addr learned my addr 172.21.15.71:0/3278478728 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:32:59.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.937+0000 7f6d28d7a700 1 --2- 172.21.15.71:0/3278478728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d24100750 0x7f6d24197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:59.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.937+0000 7f6d2957b700 1 -- 172.21.15.71:0/3278478728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d24100750 msgr2=0x7f6d24197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:32:59.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.937+0000 7f6d2957b700 1 --2- 172.21.15.71:0/3278478728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d24100750 0x7f6d24197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:32:59.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.937+0000 7f6d2957b700 1 -- 172.21.15.71:0/3278478728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d140095f0 con 0x7f6d240ffda0 2024-04-23T02:32:59.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.937+0000 7f6d28d7a700 1 --2- 172.21.15.71:0/3278478728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d24100750 0x7f6d24197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:32:59.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.937+0000 7f6d2957b700 1 --2- 172.21.15.71:0/3278478728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d240ffda0 0x7f6d24197700 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6d2000d8d0 tx=0x7f6d2000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:59.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.937+0000 7f6d1a7fc700 1 -- 172.21.15.71:0/3278478728 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d20009940 con 0x7f6d240ffda0 2024-04-23T02:32:59.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.937+0000 7f6d2b7df700 1 -- 172.21.15.71:0/3278478728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d241a0770 con 0x7f6d240ffda0 2024-04-23T02:32:59.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.937+0000 7f6d1a7fc700 1 -- 172.21.15.71:0/3278478728 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6d20010460 con 0x7f6d240ffda0 2024-04-23T02:32:59.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.937+0000 7f6d2b7df700 1 -- 172.21.15.71:0/3278478728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d241a0c30 con 0x7f6d240ffda0 2024-04-23T02:32:59.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.937+0000 7f6d1a7fc700 1 -- 172.21.15.71:0/3278478728 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d2000f5f0 con 0x7f6d240ffda0 2024-04-23T02:32:59.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.938+0000 7f6d2b7df700 1 -- 172.21.15.71:0/3278478728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d08005320 con 0x7f6d240ffda0 2024-04-23T02:32:59.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.939+0000 7f6d1a7fc700 1 -- 172.21.15.71:0/3278478728 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6d20009aa0 con 0x7f6d240ffda0 2024-04-23T02:32:59.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.939+0000 7f6d1a7fc700 1 --2- 172.21.15.71:0/3278478728 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6d1006cce0 0x7f6d1006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:32:59.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.939+0000 7f6d1a7fc700 1 -- 172.21.15.71:0/3278478728 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6d2008c530 con 0x7f6d240ffda0 2024-04-23T02:32:59.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.939+0000 7f6d28d7a700 1 --2- 172.21.15.71:0/3278478728 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6d1006cce0 0x7f6d1006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:32:59.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.939+0000 7f6d28d7a700 1 --2- 172.21.15.71:0/3278478728 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6d1006cce0 0x7f6d1006f1a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6d14006010 tx=0x7f6d1401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:32:59.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:32:59.944+0000 7f6d1a7fc700 1 -- 172.21.15.71:0/3278478728 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6d2005a030 con 0x7f6d240ffda0 2024-04-23T02:33:00.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.156+0000 7f6d2b7df700 1 -- 172.21.15.71:0/3278478728 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d08000bf0 con 0x7f6d1006cce0 2024-04-23T02:33:00.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.158+0000 7f6d1a7fc700 1 -- 172.21.15.71:0/3278478728 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6d08000bf0 con 0x7f6d1006cce0 2024-04-23T02:33:00.160 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:33:00.160 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:33:00.160 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:33:00.160 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:33:00.160 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:33:00.160 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:33:00.160 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:33:00.160 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:33:00.160 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:33:00.160 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:33:00.161 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:33:00.161 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:33:00.161 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:33:00.161 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:33:00.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.159+0000 7f6d2b7df700 1 -- 172.21.15.71:0/3278478728 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6d1006cce0 msgr2=0x7f6d1006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:00.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.159+0000 7f6d2b7df700 1 --2- 172.21.15.71:0/3278478728 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6d1006cce0 0x7f6d1006f1a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6d14006010 tx=0x7f6d1401a040 comp rx=0 tx=0).stop 2024-04-23T02:33:00.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.159+0000 7f6d2b7df700 1 -- 172.21.15.71:0/3278478728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d240ffda0 msgr2=0x7f6d24197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:00.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.159+0000 7f6d2b7df700 1 --2- 172.21.15.71:0/3278478728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d240ffda0 0x7f6d24197700 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6d2000d8d0 tx=0x7f6d2000dc90 comp rx=0 tx=0).stop 2024-04-23T02:33:00.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.160+0000 7f6d2b7df700 1 -- 172.21.15.71:0/3278478728 shutdown_connections 2024-04-23T02:33:00.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.160+0000 7f6d2b7df700 1 --2- 172.21.15.71:0/3278478728 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6d1006cce0 0x7f6d1006f1a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:00.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.160+0000 7f6d2b7df700 1 --2- 172.21.15.71:0/3278478728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d240ffda0 0x7f6d24197700 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:00.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.160+0000 7f6d2b7df700 1 --2- 172.21.15.71:0/3278478728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d24100750 0x7f6d24197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:00.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.160+0000 7f6d2b7df700 1 -- 172.21.15.71:0/3278478728 >> 172.21.15.71:0/3278478728 conn(0x7f6d240fb650 msgr2=0x7f6d240fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:33:00.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.160+0000 7f6d2b7df700 1 -- 172.21.15.71:0/3278478728 shutdown_connections 2024-04-23T02:33:00.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.160+0000 7f6d2b7df700 1 -- 172.21.15.71:0/3278478728 wait complete. 2024-04-23T02:33:00.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.263+0000 7f34c01a0700 1 -- 172.21.15.71:0/1353010520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b80ffc20 msgr2=0x7f34b8100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:00.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.263+0000 7f34c01a0700 1 --2- 172.21.15.71:0/1353010520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b80ffc20 0x7f34b8100080 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009910 tx=0x7f34b4009c20 comp rx=0 tx=0).stop 2024-04-23T02:33:00.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.263+0000 7f34c01a0700 1 -- 172.21.15.71:0/1353010520 shutdown_connections 2024-04-23T02:33:00.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.263+0000 7f34c01a0700 1 --2- 172.21.15.71:0/1353010520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b80ffc20 0x7f34b8100080 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:00.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.263+0000 7f34c01a0700 1 --2- 172.21.15.71:0/1353010520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34b8105bd0 0x7f34b8105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:00.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.263+0000 7f34c01a0700 1 -- 172.21.15.71:0/1353010520 >> 172.21.15.71:0/1353010520 conn(0x7f34b8075670 msgr2=0x7f34b8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:33:00.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.263+0000 7f34c01a0700 1 -- 172.21.15.71:0/1353010520 shutdown_connections 2024-04-23T02:33:00.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.263+0000 7f34c01a0700 1 -- 172.21.15.71:0/1353010520 wait complete. 2024-04-23T02:33:00.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.264+0000 7f34c01a0700 1 Processor -- start 2024-04-23T02:33:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.264+0000 7f34c01a0700 1 -- start start 2024-04-23T02:33:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.264+0000 7f34c01a0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34b80ffc20 0x7f34b81976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.264+0000 7f34c01a0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b8105bd0 0x7f34b8197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.264+0000 7f34c01a0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b8198200 con 0x7f34b8105bd0 2024-04-23T02:33:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.264+0000 7f34c01a0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b8198370 con 0x7f34b80ffc20 2024-04-23T02:33:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.264+0000 7f34bd73b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b8105bd0 0x7f34b8197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.264+0000 7f34bd73b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b8105bd0 0x7f34b8197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37250/0 (socket says 172.21.15.71:37250) 2024-04-23T02:33:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.264+0000 7f34bd73b700 1 -- 172.21.15.71:0/1574962600 learned_addr learned my addr 172.21.15.71:0/1574962600 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:33:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.264+0000 7f34bdf3c700 1 --2- 172.21.15.71:0/1574962600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34b80ffc20 0x7f34b81976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.264+0000 7f34bd73b700 1 -- 172.21.15.71:0/1574962600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34b80ffc20 msgr2=0x7f34b81976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.264+0000 7f34bd73b700 1 --2- 172.21.15.71:0/1574962600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34b80ffc20 0x7f34b81976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.264+0000 7f34bd73b700 1 -- 172.21.15.71:0/1574962600 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b40095f0 con 0x7f34b8105bd0 2024-04-23T02:33:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.264+0000 7f34bdf3c700 1 --2- 172.21.15.71:0/1574962600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34b80ffc20 0x7f34b81976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:33:00.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.264+0000 7f34bd73b700 1 --2- 172.21.15.71:0/1574962600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b8105bd0 0x7f34b8197be0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f34b4000c00 tx=0x7f34b400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:33:00.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.265+0000 7f34aaffd700 1 -- 172.21.15.71:0/1574962600 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34b401d070 con 0x7f34b8105bd0 2024-04-23T02:33:00.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.265+0000 7f34aaffd700 1 -- 172.21.15.71:0/1574962600 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f34b40044c0 con 0x7f34b8105bd0 2024-04-23T02:33:00.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.265+0000 7f34c01a0700 1 -- 172.21.15.71:0/1574962600 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b81a0710 con 0x7f34b8105bd0 2024-04-23T02:33:00.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.265+0000 7f34aaffd700 1 -- 172.21.15.71:0/1574962600 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34b40039d0 con 0x7f34b8105bd0 2024-04-23T02:33:00.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.265+0000 7f34c01a0700 1 -- 172.21.15.71:0/1574962600 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b81a0b20 con 0x7f34b8105bd0 2024-04-23T02:33:00.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.266+0000 7f34c01a0700 1 -- 172.21.15.71:0/1574962600 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b81080d0 con 0x7f34b8105bd0 2024-04-23T02:33:00.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.266+0000 7f34aaffd700 1 -- 172.21.15.71:0/1574962600 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f34b40048e0 con 0x7f34b8105bd0 2024-04-23T02:33:00.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.266+0000 7f34aaffd700 1 --2- 172.21.15.71:0/1574962600 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f34a406cce0 0x7f34a406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:00.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.266+0000 7f34aaffd700 1 -- 172.21.15.71:0/1574962600 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f34b408dc60 con 0x7f34b8105bd0 2024-04-23T02:33:00.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.266+0000 7f34bdf3c700 1 --2- 172.21.15.71:0/1574962600 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f34a406cce0 0x7f34a406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:00.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.267+0000 7f34bdf3c700 1 --2- 172.21.15.71:0/1574962600 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f34a406cce0 0x7f34a406f1a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009be0 tx=0x7f34ac009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:33:00.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.271+0000 7f34aaffd700 1 -- 172.21.15.71:0/1574962600 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f34b4057cd0 con 0x7f34b8105bd0 2024-04-23T02:33:00.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:00 smithi192 ceph-mon[108617]: from='client.34906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:33:00.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:00 smithi192 ceph-mon[108617]: pgmap v590: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:00.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:00 smithi192 ceph-mon[108617]: from='client.34910 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:33:00.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:00 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1305596629' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:33:00.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:00 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3383907363' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:33:00.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:00 smithi071 ceph-mon[144544]: from='client.34906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:33:00.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:00 smithi071 ceph-mon[144544]: pgmap v590: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:00.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:00 smithi071 ceph-mon[144544]: from='client.34910 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:33:00.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:00 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1305596629' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:33:00.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:00 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3383907363' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:33:00.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.542+0000 7f34c01a0700 1 -- 172.21.15.71:0/1574962600 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f34b8068c30 con 0x7f34b8105bd0 2024-04-23T02:33:00.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.543+0000 7f34aaffd700 1 -- 172.21.15.71:0/1574962600 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f34b40261c0 con 0x7f34b8105bd0 2024-04-23T02:33:00.545 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:33:00.545 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:33:00.545 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:33:00.545 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:33:00.546 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:33:00.546 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:33:00.546 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:33:00.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.545+0000 7f34c01a0700 1 -- 172.21.15.71:0/1574962600 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f34a406cce0 msgr2=0x7f34a406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:00.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.545+0000 7f34c01a0700 1 --2- 172.21.15.71:0/1574962600 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f34a406cce0 0x7f34a406f1a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009be0 tx=0x7f34ac009260 comp rx=0 tx=0).stop 2024-04-23T02:33:00.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.545+0000 7f34c01a0700 1 -- 172.21.15.71:0/1574962600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b8105bd0 msgr2=0x7f34b8197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:00.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.545+0000 7f34c01a0700 1 --2- 172.21.15.71:0/1574962600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b8105bd0 0x7f34b8197be0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f34b4000c00 tx=0x7f34b400be90 comp rx=0 tx=0).stop 2024-04-23T02:33:00.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.545+0000 7f34c01a0700 1 -- 172.21.15.71:0/1574962600 shutdown_connections 2024-04-23T02:33:00.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.545+0000 7f34c01a0700 1 --2- 172.21.15.71:0/1574962600 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f34a406cce0 0x7f34a406f1a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:00.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.545+0000 7f34c01a0700 1 --2- 172.21.15.71:0/1574962600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34b80ffc20 0x7f34b81976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:00.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.545+0000 7f34c01a0700 1 --2- 172.21.15.71:0/1574962600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b8105bd0 0x7f34b8197be0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:00.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.545+0000 7f34c01a0700 1 -- 172.21.15.71:0/1574962600 >> 172.21.15.71:0/1574962600 conn(0x7f34b8075670 msgr2=0x7f34b80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:33:00.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.545+0000 7f34c01a0700 1 -- 172.21.15.71:0/1574962600 shutdown_connections 2024-04-23T02:33:00.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:00.545+0000 7f34c01a0700 1 -- 172.21.15.71:0/1574962600 wait complete. 2024-04-23T02:33:01.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:01 smithi192 ceph-mon[108617]: from='client.34922 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:33:01.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:01 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1574962600' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:33:01.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:01 smithi071 ceph-mon[144544]: from='client.34922 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:33:01.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:01 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1574962600' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:33:02.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:02 smithi192 ceph-mon[108617]: pgmap v591: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:02.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:02 smithi071 ceph-mon[144544]: pgmap v591: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:04.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:04 smithi192 ceph-mon[108617]: pgmap v592: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:04.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:04 smithi071 ceph-mon[144544]: pgmap v592: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:05.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:33:05.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:33:06.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:06 smithi192 ceph-mon[108617]: pgmap v593: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:06.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:06 smithi071 ceph-mon[144544]: pgmap v593: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:08.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:08 smithi192 ceph-mon[108617]: pgmap v594: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:08.435 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:08 smithi071 ceph-mon[144544]: pgmap v594: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:10.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:10 smithi192 ceph-mon[108617]: pgmap v595: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:10.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:10 smithi071 ceph-mon[144544]: pgmap v595: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:12.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:12 smithi192 ceph-mon[108617]: pgmap v596: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:12.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:12 smithi071 ceph-mon[144544]: pgmap v596: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:14.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:14 smithi192 ceph-mon[108617]: pgmap v597: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:14.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:14 smithi071 ceph-mon[144544]: pgmap v597: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:16.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:16 smithi192 ceph-mon[108617]: pgmap v598: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:16.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:16 smithi071 ceph-mon[144544]: pgmap v598: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:18.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:18 smithi192 ceph-mon[108617]: pgmap v599: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:18.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:18 smithi071 ceph-mon[144544]: pgmap v599: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:20.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:20 smithi192 ceph-mon[108617]: pgmap v600: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:20.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:33:20.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:20 smithi071 ceph-mon[144544]: pgmap v600: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:20.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:33:22.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:22 smithi192 ceph-mon[108617]: pgmap v601: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:22.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:22 smithi071 ceph-mon[144544]: pgmap v601: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:24.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:24 smithi192 ceph-mon[108617]: pgmap v602: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:24.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:24 smithi071 ceph-mon[144544]: pgmap v602: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:26.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:26 smithi192 ceph-mon[108617]: pgmap v603: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:26.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:26 smithi071 ceph-mon[144544]: pgmap v603: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:28.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:28 smithi192 ceph-mon[108617]: pgmap v604: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:28.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:28 smithi071 ceph-mon[144544]: pgmap v604: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:30.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:30 smithi192 ceph-mon[108617]: pgmap v605: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:30.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:30 smithi071 ceph-mon[144544]: pgmap v605: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:30.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.661+0000 7f1762da9700 1 -- 172.21.15.71:0/1388030451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f175c101de0 msgr2=0x7f175c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:30.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.661+0000 7f1762da9700 1 --2- 172.21.15.71:0/1388030451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f175c101de0 0x7f175c1021c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1750009910 tx=0x7f1750009c20 comp rx=0 tx=0).stop 2024-04-23T02:33:30.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.661+0000 7f1762da9700 1 -- 172.21.15.71:0/1388030451 shutdown_connections 2024-04-23T02:33:30.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.661+0000 7f1762da9700 1 --2- 172.21.15.71:0/1388030451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f175c102700 0x7f175c10abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:30.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.661+0000 7f1762da9700 1 --2- 172.21.15.71:0/1388030451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f175c101de0 0x7f175c1021c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:30.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.661+0000 7f1762da9700 1 -- 172.21.15.71:0/1388030451 >> 172.21.15.71:0/1388030451 conn(0x7f175c0fb610 msgr2=0x7f175c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:33:30.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.661+0000 7f1762da9700 1 -- 172.21.15.71:0/1388030451 shutdown_connections 2024-04-23T02:33:30.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.661+0000 7f1762da9700 1 -- 172.21.15.71:0/1388030451 wait complete. 2024-04-23T02:33:30.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.662+0000 7f1762da9700 1 Processor -- start 2024-04-23T02:33:30.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.662+0000 7f1762da9700 1 -- start start 2024-04-23T02:33:30.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.662+0000 7f1762da9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f175c101de0 0x7f175c19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:30.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.662+0000 7f1762da9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f175c102700 0x7f175c19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:30.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.662+0000 7f1762da9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c19d190 con 0x7f175c102700 2024-04-23T02:33:30.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.662+0000 7f1762da9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c196640 con 0x7f175c101de0 2024-04-23T02:33:30.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.662+0000 7f175bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f175c102700 0x7f175c19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:30.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.662+0000 7f175bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f175c102700 0x7f175c19cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60148/0 (socket says 172.21.15.71:60148) 2024-04-23T02:33:30.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.662+0000 7f175bfff700 1 -- 172.21.15.71:0/3362739857 learned_addr learned my addr 172.21.15.71:0/3362739857 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:33:30.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.662+0000 7f1760b45700 1 --2- 172.21.15.71:0/3362739857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f175c101de0 0x7f175c19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:30.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.662+0000 7f175bfff700 1 -- 172.21.15.71:0/3362739857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f175c101de0 msgr2=0x7f175c19c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:30.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.662+0000 7f175bfff700 1 --2- 172.21.15.71:0/3362739857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f175c101de0 0x7f175c19c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:30.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.662+0000 7f175bfff700 1 -- 172.21.15.71:0/3362739857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17500095f0 con 0x7f175c102700 2024-04-23T02:33:30.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.662+0000 7f175bfff700 1 --2- 172.21.15.71:0/3362739857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f175c102700 0x7f175c19cb00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f174800d6d0 tx=0x7f174800d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:33:30.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.663+0000 7f1759ffb700 1 -- 172.21.15.71:0/3362739857 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f174800f860 con 0x7f175c102700 2024-04-23T02:33:30.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.663+0000 7f1759ffb700 1 -- 172.21.15.71:0/3362739857 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f174800fe60 con 0x7f175c102700 2024-04-23T02:33:30.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.663+0000 7f1759ffb700 1 -- 172.21.15.71:0/3362739857 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f174800e3b0 con 0x7f175c102700 2024-04-23T02:33:30.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.663+0000 7f1762da9700 1 -- 172.21.15.71:0/3362739857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175c196920 con 0x7f175c102700 2024-04-23T02:33:30.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.663+0000 7f1762da9700 1 -- 172.21.15.71:0/3362739857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175c196e70 con 0x7f175c102700 2024-04-23T02:33:30.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.664+0000 7f1762da9700 1 -- 172.21.15.71:0/3362739857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175c1082f0 con 0x7f175c102700 2024-04-23T02:33:30.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.665+0000 7f1759ffb700 1 -- 172.21.15.71:0/3362739857 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1748010460 con 0x7f175c102700 2024-04-23T02:33:30.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.665+0000 7f1759ffb700 1 --2- 172.21.15.71:0/3362739857 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f174c06cce0 0x7f174c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:30.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.665+0000 7f1759ffb700 1 -- 172.21.15.71:0/3362739857 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f174808c790 con 0x7f175c102700 2024-04-23T02:33:30.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.665+0000 7f1760b45700 1 --2- 172.21.15.71:0/3362739857 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f174c06cce0 0x7f174c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:30.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.666+0000 7f1760b45700 1 --2- 172.21.15.71:0/3362739857 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f174c06cce0 0x7f174c06f1a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1750006010 tx=0x7f175001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:33:30.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.669+0000 7f1759ffb700 1 -- 172.21.15.71:0/3362739857 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f174805a2f0 con 0x7f175c102700 2024-04-23T02:33:30.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.880+0000 7f1762da9700 1 -- 172.21.15.71:0/3362739857 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f175c197140 con 0x7f174c06cce0 2024-04-23T02:33:30.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.881+0000 7f1759ffb700 1 -- 172.21.15.71:0/3362739857 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f175c197140 con 0x7f174c06cce0 2024-04-23T02:33:30.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.883+0000 7f1762da9700 1 -- 172.21.15.71:0/3362739857 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f174c06cce0 msgr2=0x7f174c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:30.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.883+0000 7f1762da9700 1 --2- 172.21.15.71:0/3362739857 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f174c06cce0 0x7f174c06f1a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1750006010 tx=0x7f175001a040 comp rx=0 tx=0).stop 2024-04-23T02:33:30.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.883+0000 7f1762da9700 1 -- 172.21.15.71:0/3362739857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f175c102700 msgr2=0x7f175c19cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:30.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.883+0000 7f1762da9700 1 --2- 172.21.15.71:0/3362739857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f175c102700 0x7f175c19cb00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f174800d6d0 tx=0x7f174800d9e0 comp rx=0 tx=0).stop 2024-04-23T02:33:30.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.883+0000 7f1762da9700 1 -- 172.21.15.71:0/3362739857 shutdown_connections 2024-04-23T02:33:30.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.883+0000 7f1762da9700 1 --2- 172.21.15.71:0/3362739857 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f174c06cce0 0x7f174c06f1a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:30.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.883+0000 7f1762da9700 1 --2- 172.21.15.71:0/3362739857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f175c101de0 0x7f175c19c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:30.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.883+0000 7f1762da9700 1 --2- 172.21.15.71:0/3362739857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f175c102700 0x7f175c19cb00 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:30.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.883+0000 7f1762da9700 1 -- 172.21.15.71:0/3362739857 >> 172.21.15.71:0/3362739857 conn(0x7f175c0fb610 msgr2=0x7f175c105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:33:30.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.883+0000 7f1762da9700 1 -- 172.21.15.71:0/3362739857 shutdown_connections 2024-04-23T02:33:30.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.883+0000 7f1762da9700 1 -- 172.21.15.71:0/3362739857 wait complete. 2024-04-23T02:33:30.897 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:33:30.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.988+0000 7f522853f700 1 -- 172.21.15.71:0/4088162730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52200ffc20 msgr2=0x7f5220100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:30.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.988+0000 7f522853f700 1 --2- 172.21.15.71:0/4088162730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52200ffc20 0x7f5220100080 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f521c009910 tx=0x7f521c009c20 comp rx=0 tx=0).stop 2024-04-23T02:33:30.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.988+0000 7f522853f700 1 -- 172.21.15.71:0/4088162730 shutdown_connections 2024-04-23T02:33:30.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.988+0000 7f522853f700 1 --2- 172.21.15.71:0/4088162730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52200ffc20 0x7f5220100080 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:30.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.988+0000 7f522853f700 1 --2- 172.21.15.71:0/4088162730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5220105bd0 0x7f5220105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:30.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.988+0000 7f522853f700 1 -- 172.21.15.71:0/4088162730 >> 172.21.15.71:0/4088162730 conn(0x7f5220075670 msgr2=0x7f5220075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:33:30.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.988+0000 7f522853f700 1 -- 172.21.15.71:0/4088162730 shutdown_connections 2024-04-23T02:33:30.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.988+0000 7f522853f700 1 -- 172.21.15.71:0/4088162730 wait complete. 2024-04-23T02:33:30.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.989+0000 7f522853f700 1 Processor -- start 2024-04-23T02:33:30.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.989+0000 7f522853f700 1 -- start start 2024-04-23T02:33:30.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.989+0000 7f522853f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52200ffc20 0x7f522019b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:30.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.989+0000 7f522853f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5220105bd0 0x7f522019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:30.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.989+0000 7f522853f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522019c0f0 con 0x7f52200ffc20 2024-04-23T02:33:30.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.989+0000 7f522853f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52201955a0 con 0x7f5220105bd0 2024-04-23T02:33:30.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.989+0000 7f52262db700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52200ffc20 0x7f522019b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:30.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.989+0000 7f52262db700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52200ffc20 0x7f522019b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60172/0 (socket says 172.21.15.71:60172) 2024-04-23T02:33:30.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.989+0000 7f52262db700 1 -- 172.21.15.71:0/1117895526 learned_addr learned my addr 172.21.15.71:0/1117895526 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:33:30.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.989+0000 7f5225ada700 1 --2- 172.21.15.71:0/1117895526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5220105bd0 0x7f522019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:30.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.989+0000 7f52262db700 1 -- 172.21.15.71:0/1117895526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5220105bd0 msgr2=0x7f522019ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:30.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.989+0000 7f52262db700 1 --2- 172.21.15.71:0/1117895526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5220105bd0 0x7f522019ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.989+0000 7f52262db700 1 -- 172.21.15.71:0/1117895526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c0095f0 con 0x7f52200ffc20 2024-04-23T02:33:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.989+0000 7f5225ada700 1 --2- 172.21.15.71:0/1117895526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5220105bd0 0x7f522019ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:33:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.990+0000 7f52262db700 1 --2- 172.21.15.71:0/1117895526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52200ffc20 0x7f522019b520 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5210009970 tx=0x7f5210009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:33:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.990+0000 7f52177fe700 1 -- 172.21.15.71:0/1117895526 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5210012070 con 0x7f52200ffc20 2024-04-23T02:33:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.990+0000 7f52177fe700 1 -- 172.21.15.71:0/1117895526 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5210007ad0 con 0x7f52200ffc20 2024-04-23T02:33:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.990+0000 7f52177fe700 1 -- 172.21.15.71:0/1117895526 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5210017c00 con 0x7f52200ffc20 2024-04-23T02:33:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.990+0000 7f522853f700 1 -- 172.21.15.71:0/1117895526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5220195820 con 0x7f52200ffc20 2024-04-23T02:33:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.990+0000 7f522853f700 1 -- 172.21.15.71:0/1117895526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5220195d70 con 0x7f52200ffc20 2024-04-23T02:33:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.991+0000 7f522853f700 1 -- 172.21.15.71:0/1117895526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522004f2e0 con 0x7f52200ffc20 2024-04-23T02:33:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.991+0000 7f52177fe700 1 -- 172.21.15.71:0/1117895526 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5210007c40 con 0x7f52200ffc20 2024-04-23T02:33:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.991+0000 7f52177fe700 1 --2- 172.21.15.71:0/1117895526 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f520c06cce0 0x7f520c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.992+0000 7f52177fe700 1 -- 172.21.15.71:0/1117895526 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f521008dbc0 con 0x7f52200ffc20 2024-04-23T02:33:30.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.992+0000 7f5225ada700 1 --2- 172.21.15.71:0/1117895526 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f520c06cce0 0x7f520c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:30.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.992+0000 7f5225ada700 1 --2- 172.21.15.71:0/1117895526 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f520c06cce0 0x7f520c06f1a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f52200fe2d0 tx=0x7f521c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:33:30.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:30.996+0000 7f52177fe700 1 -- 172.21.15.71:0/1117895526 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f521005b770 con 0x7f52200ffc20 2024-04-23T02:33:31.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.206+0000 7f522853f700 1 -- 172.21.15.71:0/1117895526 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5220196080 con 0x7f520c06cce0 2024-04-23T02:33:31.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.207+0000 7f52177fe700 1 -- 172.21.15.71:0/1117895526 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5220196080 con 0x7f520c06cce0 2024-04-23T02:33:31.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.210+0000 7f522853f700 1 -- 172.21.15.71:0/1117895526 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f520c06cce0 msgr2=0x7f520c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:31.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.210+0000 7f522853f700 1 --2- 172.21.15.71:0/1117895526 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f520c06cce0 0x7f520c06f1a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f52200fe2d0 tx=0x7f521c01a040 comp rx=0 tx=0).stop 2024-04-23T02:33:31.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.210+0000 7f522853f700 1 -- 172.21.15.71:0/1117895526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52200ffc20 msgr2=0x7f522019b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:31.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.210+0000 7f522853f700 1 --2- 172.21.15.71:0/1117895526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52200ffc20 0x7f522019b520 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5210009970 tx=0x7f5210009d30 comp rx=0 tx=0).stop 2024-04-23T02:33:31.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.210+0000 7f522853f700 1 -- 172.21.15.71:0/1117895526 shutdown_connections 2024-04-23T02:33:31.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.210+0000 7f522853f700 1 --2- 172.21.15.71:0/1117895526 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f520c06cce0 0x7f520c06f1a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:31.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.210+0000 7f522853f700 1 --2- 172.21.15.71:0/1117895526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52200ffc20 0x7f522019b520 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:31.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.210+0000 7f522853f700 1 --2- 172.21.15.71:0/1117895526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5220105bd0 0x7f522019ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:31.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.210+0000 7f522853f700 1 -- 172.21.15.71:0/1117895526 >> 172.21.15.71:0/1117895526 conn(0x7f5220075670 msgr2=0x7f52200fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:33:31.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.210+0000 7f522853f700 1 -- 172.21.15.71:0/1117895526 shutdown_connections 2024-04-23T02:33:31.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.210+0000 7f522853f700 1 -- 172.21.15.71:0/1117895526 wait complete. 2024-04-23T02:33:31.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.312+0000 7f541ad0c700 1 -- 172.21.15.71:0/4101630222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54141006b0 msgr2=0x7f5414100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:31.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.312+0000 7f541ad0c700 1 --2- 172.21.15.71:0/4101630222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54141006b0 0x7f5414100a90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5408009910 tx=0x7f5408009c20 comp rx=0 tx=0).stop 2024-04-23T02:33:31.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.312+0000 7f541ad0c700 1 -- 172.21.15.71:0/4101630222 shutdown_connections 2024-04-23T02:33:31.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.312+0000 7f541ad0c700 1 --2- 172.21.15.71:0/4101630222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5414101060 0x7f54141047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:31.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.312+0000 7f541ad0c700 1 --2- 172.21.15.71:0/4101630222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54141006b0 0x7f5414100a90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:31.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.312+0000 7f541ad0c700 1 -- 172.21.15.71:0/4101630222 >> 172.21.15.71:0/4101630222 conn(0x7f5414074e30 msgr2=0x7f5414075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:33:31.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.312+0000 7f541ad0c700 1 -- 172.21.15.71:0/4101630222 shutdown_connections 2024-04-23T02:33:31.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.312+0000 7f541ad0c700 1 -- 172.21.15.71:0/4101630222 wait complete. 2024-04-23T02:33:31.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.312+0000 7f541ad0c700 1 Processor -- start 2024-04-23T02:33:31.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.312+0000 7f541ad0c700 1 -- start start 2024-04-23T02:33:31.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.313+0000 7f541ad0c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54141006b0 0x7f54141956a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:31.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.313+0000 7f541ad0c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5414101060 0x7f5414195be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:31.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.313+0000 7f541ad0c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5414196270 con 0x7f5414101060 2024-04-23T02:33:31.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.313+0000 7f541ad0c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5414198fa0 con 0x7f54141006b0 2024-04-23T02:33:31.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.313+0000 7f5413fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5414101060 0x7f5414195be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:31.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.313+0000 7f5413fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5414101060 0x7f5414195be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60182/0 (socket says 172.21.15.71:60182) 2024-04-23T02:33:31.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.313+0000 7f5413fff700 1 -- 172.21.15.71:0/3289583494 learned_addr learned my addr 172.21.15.71:0/3289583494 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:33:31.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.313+0000 7f5418aa8700 1 --2- 172.21.15.71:0/3289583494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54141006b0 0x7f54141956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:31.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.313+0000 7f5413fff700 1 -- 172.21.15.71:0/3289583494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54141006b0 msgr2=0x7f54141956a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:31.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.313+0000 7f5413fff700 1 --2- 172.21.15.71:0/3289583494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54141006b0 0x7f54141956a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:31.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.313+0000 7f5413fff700 1 -- 172.21.15.71:0/3289583494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54080095f0 con 0x7f5414101060 2024-04-23T02:33:31.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.313+0000 7f5413fff700 1 --2- 172.21.15.71:0/3289583494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5414101060 0x7f5414195be0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f54000078e0 tx=0x7f5400007bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:33:31.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.313+0000 7f5411ffb700 1 -- 172.21.15.71:0/3289583494 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5400012070 con 0x7f5414101060 2024-04-23T02:33:31.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.313+0000 7f5411ffb700 1 -- 172.21.15.71:0/3289583494 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f540000d7f0 con 0x7f5414101060 2024-04-23T02:33:31.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.313+0000 7f541ad0c700 1 -- 172.21.15.71:0/3289583494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5414199220 con 0x7f5414101060 2024-04-23T02:33:31.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.313+0000 7f5411ffb700 1 -- 172.21.15.71:0/3289583494 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54000097f0 con 0x7f5414101060 2024-04-23T02:33:31.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.314+0000 7f541ad0c700 1 -- 172.21.15.71:0/3289583494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5414199770 con 0x7f5414101060 2024-04-23T02:33:31.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.315+0000 7f541ad0c700 1 -- 172.21.15.71:0/3289583494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541404f2e0 con 0x7f5414101060 2024-04-23T02:33:31.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.315+0000 7f5411ffb700 1 -- 172.21.15.71:0/3289583494 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5400016070 con 0x7f5414101060 2024-04-23T02:33:31.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.315+0000 7f5411ffb700 1 --2- 172.21.15.71:0/3289583494 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f540406cce0 0x7f540406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:31.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.316+0000 7f5411ffb700 1 -- 172.21.15.71:0/3289583494 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f540008d670 con 0x7f5414101060 2024-04-23T02:33:31.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.316+0000 7f5418aa8700 1 --2- 172.21.15.71:0/3289583494 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f540406cce0 0x7f540406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:31.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.316+0000 7f5418aa8700 1 --2- 172.21.15.71:0/3289583494 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f540406cce0 0x7f540406f1a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5408006010 tx=0x7f540801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:33:31.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.320+0000 7f5411ffb700 1 -- 172.21.15.71:0/3289583494 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f54000576e0 con 0x7f5414101060 2024-04-23T02:33:31.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.519+0000 7f541ad0c700 1 -- 172.21.15.71:0/3289583494 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5414199a00 con 0x7f540406cce0 2024-04-23T02:33:31.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.530+0000 7f5411ffb700 1 -- 172.21.15.71:0/3289583494 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5414199a00 con 0x7f540406cce0 2024-04-23T02:33:31.532 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:33:31.532 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (17m) 10m ago 28m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:33:31.532 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (28m) 10m ago 28m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:33:31.533 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (27m) 10m ago 27m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:33:31.533 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (15m) 10m ago 28m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:33:31.533 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (15m) 10m ago 27m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:33:31.533 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (16m) 10m ago 28m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:33:31.533 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (11m) 10m ago 22m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:33:31.533 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (11m) 10m ago 22m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:33:31.533 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (22m) 10m ago 22m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:33:31.533 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (10m) 10m ago 22m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:33:31.533 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (19m) 10m ago 30m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:33:31.533 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (18m) 10m ago 27m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:33:31.533 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (15m) 10m ago 30m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:33:31.533 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (15m) 10m ago 27m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:33:31.533 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (18m) 10m ago 28m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:33:31.533 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (18m) 10m ago 27m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:33:31.533 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (14m) 10m ago 26m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:33:31.534 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (14m) 10m ago 26m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:33:31.534 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (13m) 10m ago 25m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:33:31.534 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (12m) 10m ago 25m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:33:31.534 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (12m) 10m ago 24m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:33:31.534 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (11m) 10m ago 24m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:33:31.534 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (17m) 10m ago 27m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:33:31.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.532+0000 7f541ad0c700 1 -- 172.21.15.71:0/3289583494 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f540406cce0 msgr2=0x7f540406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:31.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.532+0000 7f541ad0c700 1 --2- 172.21.15.71:0/3289583494 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f540406cce0 0x7f540406f1a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5408006010 tx=0x7f540801a040 comp rx=0 tx=0).stop 2024-04-23T02:33:31.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.532+0000 7f541ad0c700 1 -- 172.21.15.71:0/3289583494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5414101060 msgr2=0x7f5414195be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:31.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.532+0000 7f541ad0c700 1 --2- 172.21.15.71:0/3289583494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5414101060 0x7f5414195be0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f54000078e0 tx=0x7f5400007bf0 comp rx=0 tx=0).stop 2024-04-23T02:33:31.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.532+0000 7f541ad0c700 1 -- 172.21.15.71:0/3289583494 shutdown_connections 2024-04-23T02:33:31.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.532+0000 7f541ad0c700 1 --2- 172.21.15.71:0/3289583494 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f540406cce0 0x7f540406f1a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:31.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.532+0000 7f541ad0c700 1 --2- 172.21.15.71:0/3289583494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54141006b0 0x7f54141956a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:31.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.532+0000 7f541ad0c700 1 --2- 172.21.15.71:0/3289583494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5414101060 0x7f5414195be0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:31.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.532+0000 7f541ad0c700 1 -- 172.21.15.71:0/3289583494 >> 172.21.15.71:0/3289583494 conn(0x7f5414074e30 msgr2=0x7f54140fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:33:31.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.532+0000 7f541ad0c700 1 -- 172.21.15.71:0/3289583494 shutdown_connections 2024-04-23T02:33:31.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.532+0000 7f541ad0c700 1 -- 172.21.15.71:0/3289583494 wait complete. 2024-04-23T02:33:31.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.633+0000 7fd0fed30700 1 -- 172.21.15.71:0/451148714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f8102780 msgr2=0x7fd0f810ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:31.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.633+0000 7fd0fed30700 1 --2- 172.21.15.71:0/451148714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f8102780 0x7fd0f810ac70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009940 tx=0x7fd0e4009c50 comp rx=0 tx=0).stop 2024-04-23T02:33:31.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.633+0000 7fd0fed30700 1 -- 172.21.15.71:0/451148714 shutdown_connections 2024-04-23T02:33:31.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.633+0000 7fd0fed30700 1 --2- 172.21.15.71:0/451148714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f8102780 0x7fd0f810ac70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:31.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.633+0000 7fd0fed30700 1 --2- 172.21.15.71:0/451148714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f8101e60 0x7fd0f8102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:31.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.633+0000 7fd0fed30700 1 -- 172.21.15.71:0/451148714 >> 172.21.15.71:0/451148714 conn(0x7fd0f80fb610 msgr2=0x7fd0f80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:33:31.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.633+0000 7fd0fed30700 1 -- 172.21.15.71:0/451148714 shutdown_connections 2024-04-23T02:33:31.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.633+0000 7fd0fed30700 1 -- 172.21.15.71:0/451148714 wait complete. 2024-04-23T02:33:31.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.633+0000 7fd0fed30700 1 Processor -- start 2024-04-23T02:33:31.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.633+0000 7fd0fed30700 1 -- start start 2024-04-23T02:33:31.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.634+0000 7fd0fed30700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f8101e60 0x7fd0f819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:31.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.634+0000 7fd0fed30700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f8102780 0x7fd0f819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:31.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.634+0000 7fd0fed30700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f819d190 con 0x7fd0f8101e60 2024-04-23T02:33:31.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.634+0000 7fd0fed30700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f8196640 con 0x7fd0f8102780 2024-04-23T02:33:31.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.634+0000 7fd0fcacc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f8101e60 0x7fd0f819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:31.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.634+0000 7fd0fcacc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f8101e60 0x7fd0f819c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60212/0 (socket says 172.21.15.71:60212) 2024-04-23T02:33:31.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.634+0000 7fd0fcacc700 1 -- 172.21.15.71:0/3911363132 learned_addr learned my addr 172.21.15.71:0/3911363132 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:33:31.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.634+0000 7fd0f7fff700 1 --2- 172.21.15.71:0/3911363132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f8102780 0x7fd0f819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:31.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.634+0000 7fd0fcacc700 1 -- 172.21.15.71:0/3911363132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f8102780 msgr2=0x7fd0f819cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:31.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.634+0000 7fd0fcacc700 1 --2- 172.21.15.71:0/3911363132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f8102780 0x7fd0f819cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:31.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.634+0000 7fd0fcacc700 1 -- 172.21.15.71:0/3911363132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e40095f0 con 0x7fd0f8101e60 2024-04-23T02:33:31.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.634+0000 7fd0fcacc700 1 --2- 172.21.15.71:0/3911363132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f8101e60 0x7fd0f819c5c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00c8a0 tx=0x7fd0ec00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:33:31.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.634+0000 7fd0f5ffb700 1 -- 172.21.15.71:0/3911363132 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0ec0078e0 con 0x7fd0f8101e60 2024-04-23T02:33:31.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.634+0000 7fd0f5ffb700 1 -- 172.21.15.71:0/3911363132 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd0ec00f450 con 0x7fd0f8101e60 2024-04-23T02:33:31.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.634+0000 7fd0fed30700 1 -- 172.21.15.71:0/3911363132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f8196920 con 0x7fd0f8101e60 2024-04-23T02:33:31.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.634+0000 7fd0f5ffb700 1 -- 172.21.15.71:0/3911363132 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0ec00e5e0 con 0x7fd0f8101e60 2024-04-23T02:33:31.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.635+0000 7fd0fed30700 1 -- 172.21.15.71:0/3911363132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f8196e40 con 0x7fd0f8101e60 2024-04-23T02:33:31.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.636+0000 7fd0fed30700 1 -- 172.21.15.71:0/3911363132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f8108370 con 0x7fd0f8101e60 2024-04-23T02:33:31.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.636+0000 7fd0f5ffb700 1 -- 172.21.15.71:0/3911363132 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd0ec00f5c0 con 0x7fd0f8101e60 2024-04-23T02:33:31.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.636+0000 7fd0f5ffb700 1 --2- 172.21.15.71:0/3911363132 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd0e806cce0 0x7fd0e806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:31.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.636+0000 7fd0f5ffb700 1 -- 172.21.15.71:0/3911363132 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd0ec08c480 con 0x7fd0f8101e60 2024-04-23T02:33:31.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.636+0000 7fd0f7fff700 1 --2- 172.21.15.71:0/3911363132 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd0e806cce0 0x7fd0e806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:31.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.637+0000 7fd0f7fff700 1 --2- 172.21.15.71:0/3911363132 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd0e806cce0 0x7fd0e806f1a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4006010 tx=0x7fd0e4005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:33:31.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.641+0000 7fd0f5ffb700 1 -- 172.21.15.71:0/3911363132 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd0ec05a000 con 0x7fd0f8101e60 2024-04-23T02:33:31.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.914+0000 7fd0fed30700 1 -- 172.21.15.71:0/3911363132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd0f8197110 con 0x7fd0f8101e60 2024-04-23T02:33:31.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.914+0000 7fd0f5ffb700 1 -- 172.21.15.71:0/3911363132 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7fd0ec059b90 con 0x7fd0f8101e60 2024-04-23T02:33:31.917 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:33:31.917 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:33:31.917 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:33:31.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:33:31.917 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:33:31.917 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:33:31.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:33:31.917 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:33:31.917 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:33:31.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:33:31.918 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:33:31.918 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:33:31.918 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:33:31.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:33:31.918 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:33:31.918 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:33:31.918 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:33:31.918 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:33:31.918 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:33:31.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.916+0000 7fd0fed30700 1 -- 172.21.15.71:0/3911363132 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd0e806cce0 msgr2=0x7fd0e806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:31.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.916+0000 7fd0fed30700 1 --2- 172.21.15.71:0/3911363132 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd0e806cce0 0x7fd0e806f1a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4006010 tx=0x7fd0e4005960 comp rx=0 tx=0).stop 2024-04-23T02:33:31.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.916+0000 7fd0fed30700 1 -- 172.21.15.71:0/3911363132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f8101e60 msgr2=0x7fd0f819c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:31.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.916+0000 7fd0fed30700 1 --2- 172.21.15.71:0/3911363132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f8101e60 0x7fd0f819c5c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00c8a0 tx=0x7fd0ec00cbb0 comp rx=0 tx=0).stop 2024-04-23T02:33:31.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.917+0000 7fd0fed30700 1 -- 172.21.15.71:0/3911363132 shutdown_connections 2024-04-23T02:33:31.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.917+0000 7fd0fed30700 1 --2- 172.21.15.71:0/3911363132 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd0e806cce0 0x7fd0e806f1a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:31.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.917+0000 7fd0fed30700 1 --2- 172.21.15.71:0/3911363132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f8101e60 0x7fd0f819c5c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:31.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.917+0000 7fd0fed30700 1 --2- 172.21.15.71:0/3911363132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f8102780 0x7fd0f819cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:31.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.917+0000 7fd0fed30700 1 -- 172.21.15.71:0/3911363132 >> 172.21.15.71:0/3911363132 conn(0x7fd0f80fb610 msgr2=0x7fd0f81054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:33:31.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.917+0000 7fd0fed30700 1 -- 172.21.15.71:0/3911363132 shutdown_connections 2024-04-23T02:33:31.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:31.917+0000 7fd0fed30700 1 -- 172.21.15.71:0/3911363132 wait complete. 2024-04-23T02:33:32.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.018+0000 7f93e744e700 1 -- 172.21.15.71:0/2820521650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93e0068580 msgr2=0x7f93e00689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:32.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.018+0000 7f93e744e700 1 --2- 172.21.15.71:0/2820521650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93e0068580 0x7f93e00689e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009910 tx=0x7f93d0009c20 comp rx=0 tx=0).stop 2024-04-23T02:33:32.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.018+0000 7f93e744e700 1 -- 172.21.15.71:0/2820521650 shutdown_connections 2024-04-23T02:33:32.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.018+0000 7f93e744e700 1 --2- 172.21.15.71:0/2820521650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93e0068580 0x7f93e00689e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:32.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.018+0000 7f93e744e700 1 --2- 172.21.15.71:0/2820521650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93e0106cd0 0x7f93e01070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:32.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.019+0000 7f93e744e700 1 -- 172.21.15.71:0/2820521650 >> 172.21.15.71:0/2820521650 conn(0x7f93e0075670 msgr2=0x7f93e0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:33:32.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.019+0000 7f93e744e700 1 -- 172.21.15.71:0/2820521650 shutdown_connections 2024-04-23T02:33:32.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.019+0000 7f93e744e700 1 -- 172.21.15.71:0/2820521650 wait complete. 2024-04-23T02:33:32.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.019+0000 7f93e744e700 1 Processor -- start 2024-04-23T02:33:32.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.019+0000 7f93e744e700 1 -- start start 2024-04-23T02:33:32.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.019+0000 7f93e744e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93e0068580 0x7f93e019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:32.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.019+0000 7f93e744e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93e0106cd0 0x7f93e019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:32.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.019+0000 7f93e744e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e019d110 con 0x7f93e0068580 2024-04-23T02:33:32.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.019+0000 7f93e744e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e01965c0 con 0x7f93e0106cd0 2024-04-23T02:33:32.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.020+0000 7f93e51ea700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93e0068580 0x7f93e019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:32.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.020+0000 7f93e51ea700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93e0068580 0x7f93e019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60234/0 (socket says 172.21.15.71:60234) 2024-04-23T02:33:32.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.020+0000 7f93e51ea700 1 -- 172.21.15.71:0/3950297837 learned_addr learned my addr 172.21.15.71:0/3950297837 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:33:32.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.020+0000 7f93e49e9700 1 --2- 172.21.15.71:0/3950297837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93e0106cd0 0x7f93e019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:32.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.020+0000 7f93e51ea700 1 -- 172.21.15.71:0/3950297837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93e0106cd0 msgr2=0x7f93e019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:32.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.020+0000 7f93e51ea700 1 --2- 172.21.15.71:0/3950297837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93e0106cd0 0x7f93e019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:32.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.020+0000 7f93e51ea700 1 -- 172.21.15.71:0/3950297837 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d00095f0 con 0x7f93e0068580 2024-04-23T02:33:32.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.020+0000 7f93e51ea700 1 --2- 172.21.15.71:0/3950297837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93e0068580 0x7f93e019c540 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00d900 tx=0x7f93dc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:33:32.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.020+0000 7f93d67fc700 1 -- 172.21.15.71:0/3950297837 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93dc0041f0 con 0x7f93e0068580 2024-04-23T02:33:32.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.020+0000 7f93d67fc700 1 -- 172.21.15.71:0/3950297837 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f93dc004cd0 con 0x7f93e0068580 2024-04-23T02:33:32.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.020+0000 7f93e744e700 1 -- 172.21.15.71:0/3950297837 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e01968a0 con 0x7f93e0068580 2024-04-23T02:33:32.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.020+0000 7f93d67fc700 1 -- 172.21.15.71:0/3950297837 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93dc005660 con 0x7f93e0068580 2024-04-23T02:33:32.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.020+0000 7f93e744e700 1 -- 172.21.15.71:0/3950297837 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e0196df0 con 0x7f93e0068580 2024-04-23T02:33:32.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.021+0000 7f93e744e700 1 -- 172.21.15.71:0/3950297837 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e01091d0 con 0x7f93e0068580 2024-04-23T02:33:32.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.021+0000 7f93d67fc700 1 -- 172.21.15.71:0/3950297837 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f93dc004e40 con 0x7f93e0068580 2024-04-23T02:33:32.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.022+0000 7f93d67fc700 1 --2- 172.21.15.71:0/3950297837 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f93cc06cce0 0x7f93cc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:32.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.022+0000 7f93d67fc700 1 -- 172.21.15.71:0/3950297837 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f93dc08c420 con 0x7f93e0068580 2024-04-23T02:33:32.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.022+0000 7f93e49e9700 1 --2- 172.21.15.71:0/3950297837 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f93cc06cce0 0x7f93cc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:32.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.022+0000 7f93e49e9700 1 --2- 172.21.15.71:0/3950297837 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f93cc06cce0 0x7f93cc06f1a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f93e00fe260 tx=0x7f93d001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:33:32.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.026+0000 7f93d67fc700 1 -- 172.21.15.71:0/3950297837 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f93dc059ec0 con 0x7f93e0068580 2024-04-23T02:33:32.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.266+0000 7f93e744e700 1 -- 172.21.15.71:0/3950297837 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f93e004f2e0 con 0x7f93e0068580 2024-04-23T02:33:32.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.267+0000 7f93d67fc700 1 -- 172.21.15.71:0/3950297837 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f93dc059a50 con 0x7f93e0068580 2024-04-23T02:33:32.269 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:33:32.269 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:33:32.270 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:33:32.270 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:33:32.270 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:33:32.270 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:33:32.270 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:33:32.270 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:33:32.270 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:33:32.270 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:33:32.270 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:33:32.270 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:33:32.270 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:33:32.270 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:33:32.270 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:33:32.270 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:33:32.270 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:33:32.271 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:33:32.271 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:33:32.271 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:33:32.271 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:33:32.271 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:33:32.271 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:33:32.271 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:33:32.271 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:33:32.271 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:33:32.271 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:33:32.271 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:33:32.271 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:33:32.271 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:33:32.271 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:33:32.272 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:33:32.272 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:33:32.272 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:33:32.272 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:33:32.272 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:33:32.272 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:33:32.272 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:33:32.272 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:33:32.272 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:33:32.272 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:33:32.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.269+0000 7f93e744e700 1 -- 172.21.15.71:0/3950297837 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f93cc06cce0 msgr2=0x7f93cc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:32.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.269+0000 7f93e744e700 1 --2- 172.21.15.71:0/3950297837 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f93cc06cce0 0x7f93cc06f1a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f93e00fe260 tx=0x7f93d001a040 comp rx=0 tx=0).stop 2024-04-23T02:33:32.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.269+0000 7f93e744e700 1 -- 172.21.15.71:0/3950297837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93e0068580 msgr2=0x7f93e019c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:32.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.269+0000 7f93e744e700 1 --2- 172.21.15.71:0/3950297837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93e0068580 0x7f93e019c540 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00d900 tx=0x7f93dc00dc10 comp rx=0 tx=0).stop 2024-04-23T02:33:32.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.269+0000 7f93e744e700 1 -- 172.21.15.71:0/3950297837 shutdown_connections 2024-04-23T02:33:32.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.269+0000 7f93e744e700 1 --2- 172.21.15.71:0/3950297837 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f93cc06cce0 0x7f93cc06f1a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:32.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.269+0000 7f93e744e700 1 --2- 172.21.15.71:0/3950297837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93e0068580 0x7f93e019c540 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:32.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.269+0000 7f93e744e700 1 --2- 172.21.15.71:0/3950297837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93e0106cd0 0x7f93e019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:32.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.269+0000 7f93e744e700 1 -- 172.21.15.71:0/3950297837 >> 172.21.15.71:0/3950297837 conn(0x7f93e0075670 msgr2=0x7f93e00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:33:32.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.269+0000 7f93e744e700 1 -- 172.21.15.71:0/3950297837 shutdown_connections 2024-04-23T02:33:32.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.269+0000 7f93e744e700 1 -- 172.21.15.71:0/3950297837 wait complete. 2024-04-23T02:33:32.274 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:33:32.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:32 smithi192 ceph-mon[108617]: from='client.34930 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:33:32.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:32 smithi192 ceph-mon[108617]: pgmap v606: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:32.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:32 smithi192 ceph-mon[108617]: from='client.34934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:33:32.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:32 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3911363132' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:33:32.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.373+0000 7fbd5c25f700 1 -- 172.21.15.71:0/879750541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd54100ea0 msgr2=0x7fbd54101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:32.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.373+0000 7fbd5c25f700 1 --2- 172.21.15.71:0/879750541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd54100ea0 0x7fbd54101280 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009910 tx=0x7fbd44009c20 comp rx=0 tx=0).stop 2024-04-23T02:33:32.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.373+0000 7fbd5c25f700 1 -- 172.21.15.71:0/879750541 shutdown_connections 2024-04-23T02:33:32.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.373+0000 7fbd5c25f700 1 --2- 172.21.15.71:0/879750541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd54101850 0x7fbd541058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:32.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.373+0000 7fbd5c25f700 1 --2- 172.21.15.71:0/879750541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd54100ea0 0x7fbd54101280 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:32.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.373+0000 7fbd5c25f700 1 -- 172.21.15.71:0/879750541 >> 172.21.15.71:0/879750541 conn(0x7fbd540fc750 msgr2=0x7fbd540feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:33:32.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.373+0000 7fbd5c25f700 1 -- 172.21.15.71:0/879750541 shutdown_connections 2024-04-23T02:33:32.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.373+0000 7fbd5c25f700 1 -- 172.21.15.71:0/879750541 wait complete. 2024-04-23T02:33:32.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.373+0000 7fbd5c25f700 1 Processor -- start 2024-04-23T02:33:32.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.373+0000 7fbd5c25f700 1 -- start start 2024-04-23T02:33:32.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.374+0000 7fbd5c25f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd54100ea0 0x7fbd5419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:32.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.374+0000 7fbd5c25f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd54101850 0x7fbd5419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:32.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.374+0000 7fbd5c25f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5419d230 con 0x7fbd54101850 2024-04-23T02:33:32.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.374+0000 7fbd5c25f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd541966e0 con 0x7fbd54100ea0 2024-04-23T02:33:32.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.374+0000 7fbd597fa700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd54101850 0x7fbd5419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:32.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.374+0000 7fbd597fa700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd54101850 0x7fbd5419cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60262/0 (socket says 172.21.15.71:60262) 2024-04-23T02:33:32.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.374+0000 7fbd597fa700 1 -- 172.21.15.71:0/3161845718 learned_addr learned my addr 172.21.15.71:0/3161845718 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:33:32.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.374+0000 7fbd59ffb700 1 --2- 172.21.15.71:0/3161845718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd54100ea0 0x7fbd5419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:32.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.374+0000 7fbd597fa700 1 -- 172.21.15.71:0/3161845718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd54100ea0 msgr2=0x7fbd5419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:32.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.374+0000 7fbd597fa700 1 --2- 172.21.15.71:0/3161845718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd54100ea0 0x7fbd5419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:32.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.374+0000 7fbd597fa700 1 -- 172.21.15.71:0/3161845718 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd440095f0 con 0x7fbd54101850 2024-04-23T02:33:32.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.374+0000 7fbd597fa700 1 --2- 172.21.15.71:0/3161845718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd54101850 0x7fbd5419cba0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000b480 tx=0x7fbd5000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:33:32.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.374+0000 7fbd4affd700 1 -- 172.21.15.71:0/3161845718 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd500118c0 con 0x7fbd54101850 2024-04-23T02:33:32.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.374+0000 7fbd4affd700 1 -- 172.21.15.71:0/3161845718 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbd50010380 con 0x7fbd54101850 2024-04-23T02:33:32.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.374+0000 7fbd5c25f700 1 -- 172.21.15.71:0/3161845718 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd541969c0 con 0x7fbd54101850 2024-04-23T02:33:32.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.374+0000 7fbd4affd700 1 -- 172.21.15.71:0/3161845718 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd5000f510 con 0x7fbd54101850 2024-04-23T02:33:32.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.374+0000 7fbd5c25f700 1 -- 172.21.15.71:0/3161845718 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd54196f10 con 0x7fbd54101850 2024-04-23T02:33:32.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.375+0000 7fbd5c25f700 1 -- 172.21.15.71:0/3161845718 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd54109240 con 0x7fbd54101850 2024-04-23T02:33:32.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.376+0000 7fbd4affd700 1 -- 172.21.15.71:0/3161845718 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbd500104f0 con 0x7fbd54101850 2024-04-23T02:33:32.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.376+0000 7fbd4affd700 1 --2- 172.21.15.71:0/3161845718 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbd4006cce0 0x7fbd4006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:32.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.376+0000 7fbd4affd700 1 -- 172.21.15.71:0/3161845718 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbd5008c210 con 0x7fbd54101850 2024-04-23T02:33:32.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.376+0000 7fbd59ffb700 1 --2- 172.21.15.71:0/3161845718 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbd4006cce0 0x7fbd4006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:32.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.377+0000 7fbd59ffb700 1 --2- 172.21.15.71:0/3161845718 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbd4006cce0 0x7fbd4006f1a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbd44006010 tx=0x7fbd4401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:33:32.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.381+0000 7fbd4affd700 1 -- 172.21.15.71:0/3161845718 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbd50059d10 con 0x7fbd54101850 2024-04-23T02:33:32.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:32 smithi071 ceph-mon[144544]: from='client.34930 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:33:32.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:32 smithi071 ceph-mon[144544]: pgmap v606: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:32.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:32 smithi071 ceph-mon[144544]: from='client.34934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:33:32.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:32 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3911363132' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:33:32.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.591+0000 7fbd5c25f700 1 -- 172.21.15.71:0/3161845718 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd54197190 con 0x7fbd4006cce0 2024-04-23T02:33:32.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.593+0000 7fbd4affd700 1 -- 172.21.15.71:0/3161845718 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbd54197190 con 0x7fbd4006cce0 2024-04-23T02:33:32.595 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:33:32.595 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:33:32.595 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:33:32.595 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:33:32.595 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:33:32.595 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:33:32.596 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:33:32.596 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:33:32.596 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:33:32.596 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:33:32.596 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:33:32.596 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:33:32.596 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:33:32.596 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:33:32.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.595+0000 7fbd5c25f700 1 -- 172.21.15.71:0/3161845718 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbd4006cce0 msgr2=0x7fbd4006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:32.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.595+0000 7fbd5c25f700 1 --2- 172.21.15.71:0/3161845718 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbd4006cce0 0x7fbd4006f1a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbd44006010 tx=0x7fbd4401a040 comp rx=0 tx=0).stop 2024-04-23T02:33:32.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.595+0000 7fbd5c25f700 1 -- 172.21.15.71:0/3161845718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd54101850 msgr2=0x7fbd5419cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:32.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.595+0000 7fbd5c25f700 1 --2- 172.21.15.71:0/3161845718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd54101850 0x7fbd5419cba0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000b480 tx=0x7fbd5000b840 comp rx=0 tx=0).stop 2024-04-23T02:33:32.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.595+0000 7fbd5c25f700 1 -- 172.21.15.71:0/3161845718 shutdown_connections 2024-04-23T02:33:32.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.595+0000 7fbd5c25f700 1 --2- 172.21.15.71:0/3161845718 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbd4006cce0 0x7fbd4006f1a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:32.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.595+0000 7fbd5c25f700 1 --2- 172.21.15.71:0/3161845718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd54100ea0 0x7fbd5419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:32.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.595+0000 7fbd5c25f700 1 --2- 172.21.15.71:0/3161845718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd54101850 0x7fbd5419cba0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:32.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.595+0000 7fbd5c25f700 1 -- 172.21.15.71:0/3161845718 >> 172.21.15.71:0/3161845718 conn(0x7fbd540fc750 msgr2=0x7fbd540fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:33:32.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.595+0000 7fbd5c25f700 1 -- 172.21.15.71:0/3161845718 shutdown_connections 2024-04-23T02:33:32.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.595+0000 7fbd5c25f700 1 -- 172.21.15.71:0/3161845718 wait complete. 2024-04-23T02:33:32.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.698+0000 7f30a9560700 1 -- 172.21.15.71:0/900663225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a40ffc20 msgr2=0x7f30a4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:32.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.698+0000 7f30a9560700 1 --2- 172.21.15.71:0/900663225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a40ffc20 0x7f30a4100080 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3094009910 tx=0x7f3094009c20 comp rx=0 tx=0).stop 2024-04-23T02:33:32.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.698+0000 7f30a9560700 1 -- 172.21.15.71:0/900663225 shutdown_connections 2024-04-23T02:33:32.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.698+0000 7f30a9560700 1 --2- 172.21.15.71:0/900663225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a40ffc20 0x7f30a4100080 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:32.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.698+0000 7f30a9560700 1 --2- 172.21.15.71:0/900663225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30a4105bd0 0x7f30a4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:32.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.698+0000 7f30a9560700 1 -- 172.21.15.71:0/900663225 >> 172.21.15.71:0/900663225 conn(0x7f30a4075670 msgr2=0x7f30a4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:33:32.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.698+0000 7f30a9560700 1 -- 172.21.15.71:0/900663225 shutdown_connections 2024-04-23T02:33:32.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.698+0000 7f30a9560700 1 -- 172.21.15.71:0/900663225 wait complete. 2024-04-23T02:33:32.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.698+0000 7f30a9560700 1 Processor -- start 2024-04-23T02:33:32.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.699+0000 7f30a9560700 1 -- start start 2024-04-23T02:33:32.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.699+0000 7f30a9560700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a40ffc20 0x7f30a419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:32.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.699+0000 7f30a9560700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30a4105bd0 0x7f30a419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:32.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.699+0000 7f30a9560700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a419c0f0 con 0x7f30a40ffc20 2024-04-23T02:33:32.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.699+0000 7f30a9560700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a41955a0 con 0x7f30a4105bd0 2024-04-23T02:33:32.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.699+0000 7f30a2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a40ffc20 0x7f30a419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:32.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.699+0000 7f30a2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a40ffc20 0x7f30a419b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60288/0 (socket says 172.21.15.71:60288) 2024-04-23T02:33:32.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.699+0000 7f30a2ffd700 1 -- 172.21.15.71:0/1582412408 learned_addr learned my addr 172.21.15.71:0/1582412408 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:33:32.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.699+0000 7f30a27fc700 1 --2- 172.21.15.71:0/1582412408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30a4105bd0 0x7f30a419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:32.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.699+0000 7f30a2ffd700 1 -- 172.21.15.71:0/1582412408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30a4105bd0 msgr2=0x7f30a419ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:32.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.699+0000 7f30a2ffd700 1 --2- 172.21.15.71:0/1582412408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30a4105bd0 0x7f30a419ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:32.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.699+0000 7f30a2ffd700 1 -- 172.21.15.71:0/1582412408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30940095f0 con 0x7f30a40ffc20 2024-04-23T02:33:32.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.699+0000 7f30a2ffd700 1 --2- 172.21.15.71:0/1582412408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a40ffc20 0x7f30a419b520 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3098009970 tx=0x7f3098009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:33:32.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.699+0000 7f308bfff700 1 -- 172.21.15.71:0/1582412408 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3098012070 con 0x7f30a40ffc20 2024-04-23T02:33:32.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.700+0000 7f30a9560700 1 -- 172.21.15.71:0/1582412408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a4195820 con 0x7f30a40ffc20 2024-04-23T02:33:32.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.700+0000 7f308bfff700 1 -- 172.21.15.71:0/1582412408 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3098007ad0 con 0x7f30a40ffc20 2024-04-23T02:33:32.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.700+0000 7f308bfff700 1 -- 172.21.15.71:0/1582412408 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3098017c00 con 0x7f30a40ffc20 2024-04-23T02:33:32.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.700+0000 7f30a9560700 1 -- 172.21.15.71:0/1582412408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a4195d70 con 0x7f30a40ffc20 2024-04-23T02:33:32.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.701+0000 7f30a9560700 1 -- 172.21.15.71:0/1582412408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a404f2e0 con 0x7f30a40ffc20 2024-04-23T02:33:32.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.701+0000 7f308bfff700 1 -- 172.21.15.71:0/1582412408 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3098007c40 con 0x7f30a40ffc20 2024-04-23T02:33:32.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.701+0000 7f308bfff700 1 --2- 172.21.15.71:0/1582412408 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f308c06cce0 0x7f308c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:33:32.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.702+0000 7f308bfff700 1 -- 172.21.15.71:0/1582412408 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f309808dbc0 con 0x7f30a40ffc20 2024-04-23T02:33:32.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.702+0000 7f30a27fc700 1 --2- 172.21.15.71:0/1582412408 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f308c06cce0 0x7f308c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:33:32.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.702+0000 7f30a27fc700 1 --2- 172.21.15.71:0/1582412408 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f308c06cce0 0x7f308c06f1a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f30a40fe2d0 tx=0x7f309401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:33:32.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.706+0000 7f308bfff700 1 -- 172.21.15.71:0/1582412408 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f309805b770 con 0x7f30a40ffc20 2024-04-23T02:33:32.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.982+0000 7f30a9560700 1 -- 172.21.15.71:0/1582412408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f30a4196080 con 0x7f30a40ffc20 2024-04-23T02:33:32.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.983+0000 7f308bfff700 1 -- 172.21.15.71:0/1582412408 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f309805b300 con 0x7f30a40ffc20 2024-04-23T02:33:32.985 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:33:32.985 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:33:32.985 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:33:32.985 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:33:32.985 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:33:32.986 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:33:32.986 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:33:32.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.985+0000 7f30a9560700 1 -- 172.21.15.71:0/1582412408 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f308c06cce0 msgr2=0x7f308c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:32.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.985+0000 7f30a9560700 1 --2- 172.21.15.71:0/1582412408 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f308c06cce0 0x7f308c06f1a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f30a40fe2d0 tx=0x7f309401a040 comp rx=0 tx=0).stop 2024-04-23T02:33:32.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.985+0000 7f30a9560700 1 -- 172.21.15.71:0/1582412408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a40ffc20 msgr2=0x7f30a419b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:33:32.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.985+0000 7f30a9560700 1 --2- 172.21.15.71:0/1582412408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a40ffc20 0x7f30a419b520 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3098009970 tx=0x7f3098009d30 comp rx=0 tx=0).stop 2024-04-23T02:33:32.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.985+0000 7f30a9560700 1 -- 172.21.15.71:0/1582412408 shutdown_connections 2024-04-23T02:33:32.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.985+0000 7f30a9560700 1 --2- 172.21.15.71:0/1582412408 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f308c06cce0 0x7f308c06f1a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:32.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.985+0000 7f30a9560700 1 --2- 172.21.15.71:0/1582412408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a40ffc20 0x7f30a419b520 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:32.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.985+0000 7f30a9560700 1 --2- 172.21.15.71:0/1582412408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30a4105bd0 0x7f30a419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:33:32.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.985+0000 7f30a9560700 1 -- 172.21.15.71:0/1582412408 >> 172.21.15.71:0/1582412408 conn(0x7f30a4075670 msgr2=0x7f30a40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:33:32.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.985+0000 7f30a9560700 1 -- 172.21.15.71:0/1582412408 shutdown_connections 2024-04-23T02:33:32.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:33:32.985+0000 7f30a9560700 1 -- 172.21.15.71:0/1582412408 wait complete. 2024-04-23T02:33:33.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:33 smithi192 ceph-mon[108617]: from='client.34938 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:33:33.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:33 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3950297837' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:33:33.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:33 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1582412408' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:33:33.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:33 smithi071 ceph-mon[144544]: from='client.34938 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:33:33.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:33 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3950297837' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:33:33.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:33 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1582412408' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:33:34.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:34 smithi192 ceph-mon[108617]: from='client.34950 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:33:34.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:34 smithi192 ceph-mon[108617]: pgmap v607: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:34.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:34 smithi071 ceph-mon[144544]: from='client.34950 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:33:34.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:34 smithi071 ceph-mon[144544]: pgmap v607: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:35.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:33:35.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:33:36.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:36 smithi071 ceph-mon[144544]: pgmap v608: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:36.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:36 smithi192 ceph-mon[108617]: pgmap v608: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:38.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:38 smithi071 ceph-mon[144544]: pgmap v609: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:38.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:38 smithi192 ceph-mon[108617]: pgmap v609: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:40.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:40 smithi071 ceph-mon[144544]: pgmap v610: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:40.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:40 smithi192 ceph-mon[108617]: pgmap v610: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:42.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:42 smithi071 ceph-mon[144544]: pgmap v611: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:42.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:42 smithi192 ceph-mon[108617]: pgmap v611: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:44.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:44 smithi071 ceph-mon[144544]: pgmap v612: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:44.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:44 smithi192 ceph-mon[108617]: pgmap v612: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:46.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:46 smithi071 ceph-mon[144544]: pgmap v613: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:46.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:46 smithi192 ceph-mon[108617]: pgmap v613: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:48.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:48 smithi071 ceph-mon[144544]: pgmap v614: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:48.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:48 smithi192 ceph-mon[108617]: pgmap v614: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:50.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:50 smithi071 ceph-mon[144544]: pgmap v615: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:50.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:33:50.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:50 smithi192 ceph-mon[108617]: pgmap v615: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:50.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:33:52.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:52 smithi071 ceph-mon[144544]: pgmap v616: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:52.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:52 smithi192 ceph-mon[108617]: pgmap v616: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:54.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:54 smithi071 ceph-mon[144544]: pgmap v617: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:54.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:54 smithi192 ceph-mon[108617]: pgmap v617: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:56.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:56 smithi071 ceph-mon[144544]: pgmap v618: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:56.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:56 smithi192 ceph-mon[108617]: pgmap v618: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:58.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:33:58 smithi071 ceph-mon[144544]: pgmap v619: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:33:58.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:33:58 smithi192 ceph-mon[108617]: pgmap v619: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:00.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:00 smithi071 ceph-mon[144544]: pgmap v620: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:00.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:00 smithi192 ceph-mon[108617]: pgmap v620: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:02.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:02 smithi071 ceph-mon[144544]: pgmap v621: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:02.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:02 smithi192 ceph-mon[108617]: pgmap v621: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:03.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.100+0000 7f618cc3f700 1 -- 172.21.15.71:0/1913777715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61880fff50 msgr2=0x7f6188105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:03.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.100+0000 7f618cc3f700 1 --2- 172.21.15.71:0/1913777715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61880fff50 0x7f6188105840 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6178009910 tx=0x7f6178009c20 comp rx=0 tx=0).stop 2024-04-23T02:34:03.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.100+0000 7f618cc3f700 1 -- 172.21.15.71:0/1913777715 shutdown_connections 2024-04-23T02:34:03.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.100+0000 7f618cc3f700 1 --2- 172.21.15.71:0/1913777715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61880fff50 0x7f6188105840 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:03.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.100+0000 7f618cc3f700 1 --2- 172.21.15.71:0/1913777715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61880ff5a0 0x7f61880ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:03.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.100+0000 7f618cc3f700 1 -- 172.21.15.71:0/1913777715 >> 172.21.15.71:0/1913777715 conn(0x7f6188074e30 msgr2=0x7f6188075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:03.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.100+0000 7f618cc3f700 1 -- 172.21.15.71:0/1913777715 shutdown_connections 2024-04-23T02:34:03.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.100+0000 7f618cc3f700 1 -- 172.21.15.71:0/1913777715 wait complete. 2024-04-23T02:34:03.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.101+0000 7f618cc3f700 1 Processor -- start 2024-04-23T02:34:03.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.101+0000 7f618cc3f700 1 -- start start 2024-04-23T02:34:03.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.101+0000 7f618cc3f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61880ff5a0 0x7f6188100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:03.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.102+0000 7f618cc3f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61880fff50 0x7f6188100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:03.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.102+0000 7f618cc3f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6188100fa0 con 0x7f61880fff50 2024-04-23T02:34:03.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.102+0000 7f618cc3f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61881010e0 con 0x7f61880ff5a0 2024-04-23T02:34:03.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.102+0000 7f6185d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61880fff50 0x7f6188100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:03.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.102+0000 7f6185d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61880fff50 0x7f6188100a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44234/0 (socket says 172.21.15.71:44234) 2024-04-23T02:34:03.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.102+0000 7f6185d9b700 1 -- 172.21.15.71:0/1240574549 learned_addr learned my addr 172.21.15.71:0/1240574549 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:34:03.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.102+0000 7f618659c700 1 --2- 172.21.15.71:0/1240574549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61880ff5a0 0x7f6188100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:03.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.102+0000 7f6185d9b700 1 -- 172.21.15.71:0/1240574549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61880ff5a0 msgr2=0x7f6188100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:03.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.102+0000 7f6185d9b700 1 --2- 172.21.15.71:0/1240574549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61880ff5a0 0x7f6188100520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:03.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.102+0000 7f6185d9b700 1 -- 172.21.15.71:0/1240574549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61780095f0 con 0x7f61880fff50 2024-04-23T02:34:03.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.102+0000 7f6185d9b700 1 --2- 172.21.15.71:0/1240574549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61880fff50 0x7f6188100a60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6178005f50 tx=0x7f617800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:03.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.102+0000 7f61777fe700 1 -- 172.21.15.71:0/1240574549 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f617801d070 con 0x7f61880fff50 2024-04-23T02:34:03.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.102+0000 7f61777fe700 1 -- 172.21.15.71:0/1240574549 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f617800f460 con 0x7f61880fff50 2024-04-23T02:34:03.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.102+0000 7f618cc3f700 1 -- 172.21.15.71:0/1240574549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6188101360 con 0x7f61880fff50 2024-04-23T02:34:03.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.102+0000 7f61777fe700 1 -- 172.21.15.71:0/1240574549 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6178021640 con 0x7f61880fff50 2024-04-23T02:34:03.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.102+0000 7f618cc3f700 1 -- 172.21.15.71:0/1240574549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61881a2dc0 con 0x7f61880fff50 2024-04-23T02:34:03.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.103+0000 7f618cc3f700 1 -- 172.21.15.71:0/1240574549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6188109200 con 0x7f61880fff50 2024-04-23T02:34:03.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.104+0000 7f61777fe700 1 -- 172.21.15.71:0/1240574549 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f617800f9e0 con 0x7f61880fff50 2024-04-23T02:34:03.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.104+0000 7f61777fe700 1 --2- 172.21.15.71:0/1240574549 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f617006cce0 0x7f617006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:03.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.104+0000 7f61777fe700 1 -- 172.21.15.71:0/1240574549 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f617808d7e0 con 0x7f61880fff50 2024-04-23T02:34:03.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.104+0000 7f618659c700 1 --2- 172.21.15.71:0/1240574549 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f617006cce0 0x7f617006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:03.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.105+0000 7f618659c700 1 --2- 172.21.15.71:0/1240574549 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f617006cce0 0x7f617006f1a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f617c005950 tx=0x7f617c00a300 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:03.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.109+0000 7f61777fe700 1 -- 172.21.15.71:0/1240574549 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6178057850 con 0x7f61880fff50 2024-04-23T02:34:03.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.320+0000 7f618cc3f700 1 -- 172.21.15.71:0/1240574549 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6188101ac0 con 0x7f617006cce0 2024-04-23T02:34:03.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.321+0000 7f61777fe700 1 -- 172.21.15.71:0/1240574549 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6188101ac0 con 0x7f617006cce0 2024-04-23T02:34:03.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.323+0000 7f618cc3f700 1 -- 172.21.15.71:0/1240574549 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f617006cce0 msgr2=0x7f617006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:03.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.323+0000 7f618cc3f700 1 --2- 172.21.15.71:0/1240574549 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f617006cce0 0x7f617006f1a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f617c005950 tx=0x7f617c00a300 comp rx=0 tx=0).stop 2024-04-23T02:34:03.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.323+0000 7f618cc3f700 1 -- 172.21.15.71:0/1240574549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61880fff50 msgr2=0x7f6188100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:03.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.323+0000 7f618cc3f700 1 --2- 172.21.15.71:0/1240574549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61880fff50 0x7f6188100a60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6178005f50 tx=0x7f617800bd70 comp rx=0 tx=0).stop 2024-04-23T02:34:03.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.323+0000 7f618cc3f700 1 -- 172.21.15.71:0/1240574549 shutdown_connections 2024-04-23T02:34:03.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.323+0000 7f618cc3f700 1 --2- 172.21.15.71:0/1240574549 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f617006cce0 0x7f617006f1a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:03.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.323+0000 7f618cc3f700 1 --2- 172.21.15.71:0/1240574549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61880ff5a0 0x7f6188100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:03.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.323+0000 7f618cc3f700 1 --2- 172.21.15.71:0/1240574549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61880fff50 0x7f6188100a60 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:03.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.323+0000 7f618cc3f700 1 -- 172.21.15.71:0/1240574549 >> 172.21.15.71:0/1240574549 conn(0x7f6188074e30 msgr2=0x7f61880fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:03.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.323+0000 7f618cc3f700 1 -- 172.21.15.71:0/1240574549 shutdown_connections 2024-04-23T02:34:03.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.323+0000 7f618cc3f700 1 -- 172.21.15.71:0/1240574549 wait complete. 2024-04-23T02:34:03.335 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:34:03.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.426+0000 7f4e7cd28700 1 -- 172.21.15.71:0/1074035815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e780ffb40 msgr2=0x7f4e780fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:03.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.426+0000 7f4e7cd28700 1 --2- 172.21.15.71:0/1074035815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e780ffb40 0x7f4e780fffc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009910 tx=0x7f4e68009c20 comp rx=0 tx=0).stop 2024-04-23T02:34:03.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.426+0000 7f4e7cd28700 1 -- 172.21.15.71:0/1074035815 shutdown_connections 2024-04-23T02:34:03.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.426+0000 7f4e7cd28700 1 --2- 172.21.15.71:0/1074035815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e780ffb40 0x7f4e780fffc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:03.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.426+0000 7f4e7cd28700 1 --2- 172.21.15.71:0/1074035815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e78105bf0 0x7f4e78105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:03.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.426+0000 7f4e7cd28700 1 -- 172.21.15.71:0/1074035815 >> 172.21.15.71:0/1074035815 conn(0x7f4e780fb610 msgr2=0x7f4e780fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:03.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.426+0000 7f4e7cd28700 1 -- 172.21.15.71:0/1074035815 shutdown_connections 2024-04-23T02:34:03.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.426+0000 7f4e7cd28700 1 -- 172.21.15.71:0/1074035815 wait complete. 2024-04-23T02:34:03.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.427+0000 7f4e7cd28700 1 Processor -- start 2024-04-23T02:34:03.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.427+0000 7f4e7cd28700 1 -- start start 2024-04-23T02:34:03.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.427+0000 7f4e7cd28700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e780ffb40 0x7f4e781976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:03.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.427+0000 7f4e7cd28700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e78105bf0 0x7f4e78197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:03.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.427+0000 7f4e7cd28700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e78198200 con 0x7f4e780ffb40 2024-04-23T02:34:03.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.427+0000 7f4e7cd28700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e78198370 con 0x7f4e78105bf0 2024-04-23T02:34:03.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.427+0000 7f4e7659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e780ffb40 0x7f4e781976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:03.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.427+0000 7f4e7659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e780ffb40 0x7f4e781976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44246/0 (socket says 172.21.15.71:44246) 2024-04-23T02:34:03.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.427+0000 7f4e7659c700 1 -- 172.21.15.71:0/1664867065 learned_addr learned my addr 172.21.15.71:0/1664867065 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:34:03.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.427+0000 7f4e75d9b700 1 --2- 172.21.15.71:0/1664867065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e78105bf0 0x7f4e78197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:03.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.427+0000 7f4e7659c700 1 -- 172.21.15.71:0/1664867065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e78105bf0 msgr2=0x7f4e78197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:03.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.427+0000 7f4e7659c700 1 --2- 172.21.15.71:0/1664867065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e78105bf0 0x7f4e78197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:03.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.427+0000 7f4e7659c700 1 -- 172.21.15.71:0/1664867065 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e680095f0 con 0x7f4e780ffb40 2024-04-23T02:34:03.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.427+0000 7f4e75d9b700 1 --2- 172.21.15.71:0/1664867065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e78105bf0 0x7f4e78197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:34:03.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.427+0000 7f4e7659c700 1 --2- 172.21.15.71:0/1664867065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e780ffb40 0x7f4e781976a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00b4b0 tx=0x7f4e6c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:03.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.428+0000 7f4e677fe700 1 -- 172.21.15.71:0/1664867065 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e6c011800 con 0x7f4e780ffb40 2024-04-23T02:34:03.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.428+0000 7f4e677fe700 1 -- 172.21.15.71:0/1664867065 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4e6c011e00 con 0x7f4e780ffb40 2024-04-23T02:34:03.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.428+0000 7f4e7cd28700 1 -- 172.21.15.71:0/1664867065 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e781a0710 con 0x7f4e780ffb40 2024-04-23T02:34:03.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.428+0000 7f4e677fe700 1 -- 172.21.15.71:0/1664867065 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e6c010e50 con 0x7f4e780ffb40 2024-04-23T02:34:03.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.428+0000 7f4e7cd28700 1 -- 172.21.15.71:0/1664867065 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e781a0c00 con 0x7f4e780ffb40 2024-04-23T02:34:03.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.428+0000 7f4e7cd28700 1 -- 172.21.15.71:0/1664867065 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e78108020 con 0x7f4e780ffb40 2024-04-23T02:34:03.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.429+0000 7f4e677fe700 1 -- 172.21.15.71:0/1664867065 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4e6c010380 con 0x7f4e780ffb40 2024-04-23T02:34:03.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.429+0000 7f4e677fe700 1 --2- 172.21.15.71:0/1664867065 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4e6006cce0 0x7f4e6006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:03.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.429+0000 7f4e677fe700 1 -- 172.21.15.71:0/1664867065 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4e6c08c0f0 con 0x7f4e780ffb40 2024-04-23T02:34:03.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.429+0000 7f4e75d9b700 1 --2- 172.21.15.71:0/1664867065 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4e6006cce0 0x7f4e6006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:03.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.430+0000 7f4e75d9b700 1 --2- 172.21.15.71:0/1664867065 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4e6006cce0 0x7f4e6006f1a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4e68006010 tx=0x7f4e6800b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:03.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.434+0000 7f4e677fe700 1 -- 172.21.15.71:0/1664867065 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f4e6c0561e0 con 0x7f4e780ffb40 2024-04-23T02:34:03.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.644+0000 7f4e7cd28700 1 -- 172.21.15.71:0/1664867065 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e780611d0 con 0x7f4e6006cce0 2024-04-23T02:34:03.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.646+0000 7f4e677fe700 1 -- 172.21.15.71:0/1664867065 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4e780611d0 con 0x7f4e6006cce0 2024-04-23T02:34:03.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.648+0000 7f4e7cd28700 1 -- 172.21.15.71:0/1664867065 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4e6006cce0 msgr2=0x7f4e6006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:03.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.648+0000 7f4e7cd28700 1 --2- 172.21.15.71:0/1664867065 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4e6006cce0 0x7f4e6006f1a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4e68006010 tx=0x7f4e6800b540 comp rx=0 tx=0).stop 2024-04-23T02:34:03.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.648+0000 7f4e7cd28700 1 -- 172.21.15.71:0/1664867065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e780ffb40 msgr2=0x7f4e781976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:03.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.648+0000 7f4e7cd28700 1 --2- 172.21.15.71:0/1664867065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e780ffb40 0x7f4e781976a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00b4b0 tx=0x7f4e6c00b870 comp rx=0 tx=0).stop 2024-04-23T02:34:03.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.648+0000 7f4e7cd28700 1 -- 172.21.15.71:0/1664867065 shutdown_connections 2024-04-23T02:34:03.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.648+0000 7f4e7cd28700 1 --2- 172.21.15.71:0/1664867065 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4e6006cce0 0x7f4e6006f1a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:03.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.648+0000 7f4e7cd28700 1 --2- 172.21.15.71:0/1664867065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e780ffb40 0x7f4e781976a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:03.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.648+0000 7f4e7cd28700 1 --2- 172.21.15.71:0/1664867065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e78105bf0 0x7f4e78197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:03.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.648+0000 7f4e7cd28700 1 -- 172.21.15.71:0/1664867065 >> 172.21.15.71:0/1664867065 conn(0x7f4e780fb610 msgr2=0x7f4e780fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:03.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.648+0000 7f4e7cd28700 1 -- 172.21.15.71:0/1664867065 shutdown_connections 2024-04-23T02:34:03.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.648+0000 7f4e7cd28700 1 -- 172.21.15.71:0/1664867065 wait complete. 2024-04-23T02:34:03.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.750+0000 7fe9dcaba700 1 -- 172.21.15.71:0/1573445476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9d8100e90 msgr2=0x7fe9d8101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:03.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.750+0000 7fe9dcaba700 1 --2- 172.21.15.71:0/1573445476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9d8100e90 0x7fe9d8101270 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009910 tx=0x7fe9c8009c20 comp rx=0 tx=0).stop 2024-04-23T02:34:03.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.751+0000 7fe9dcaba700 1 -- 172.21.15.71:0/1573445476 shutdown_connections 2024-04-23T02:34:03.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.751+0000 7fe9dcaba700 1 --2- 172.21.15.71:0/1573445476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9d8101840 0x7fe9d8105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:03.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.751+0000 7fe9dcaba700 1 --2- 172.21.15.71:0/1573445476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9d8100e90 0x7fe9d8101270 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:03.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.751+0000 7fe9dcaba700 1 -- 172.21.15.71:0/1573445476 >> 172.21.15.71:0/1573445476 conn(0x7fe9d80fc740 msgr2=0x7fe9d80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:03.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.751+0000 7fe9dcaba700 1 -- 172.21.15.71:0/1573445476 shutdown_connections 2024-04-23T02:34:03.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.751+0000 7fe9dcaba700 1 -- 172.21.15.71:0/1573445476 wait complete. 2024-04-23T02:34:03.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.751+0000 7fe9dcaba700 1 Processor -- start 2024-04-23T02:34:03.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.751+0000 7fe9dcaba700 1 -- start start 2024-04-23T02:34:03.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.752+0000 7fe9dcaba700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9d8100e90 0x7fe9d819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:03.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.752+0000 7fe9dcaba700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9d8101840 0x7fe9d819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:03.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.752+0000 7fe9dcaba700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d819d190 con 0x7fe9d8101840 2024-04-23T02:34:03.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.752+0000 7fe9dcaba700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d8196640 con 0x7fe9d8100e90 2024-04-23T02:34:03.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.752+0000 7fe9d5d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9d8101840 0x7fe9d819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:03.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.752+0000 7fe9d5d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9d8101840 0x7fe9d819cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44274/0 (socket says 172.21.15.71:44274) 2024-04-23T02:34:03.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.752+0000 7fe9d5d9b700 1 -- 172.21.15.71:0/2165639098 learned_addr learned my addr 172.21.15.71:0/2165639098 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:34:03.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.752+0000 7fe9d659c700 1 --2- 172.21.15.71:0/2165639098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9d8100e90 0x7fe9d819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:03.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.752+0000 7fe9d5d9b700 1 -- 172.21.15.71:0/2165639098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9d8100e90 msgr2=0x7fe9d819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:03.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.752+0000 7fe9d5d9b700 1 --2- 172.21.15.71:0/2165639098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9d8100e90 0x7fe9d819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:03.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.752+0000 7fe9d5d9b700 1 -- 172.21.15.71:0/2165639098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c80095f0 con 0x7fe9d8101840 2024-04-23T02:34:03.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.752+0000 7fe9d5d9b700 1 --2- 172.21.15.71:0/2165639098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9d8101840 0x7fe9d819cb00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00c8f0 tx=0x7fe9cc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:03.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.752+0000 7fe9c77fe700 1 -- 172.21.15.71:0/2165639098 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9cc0078a0 con 0x7fe9d8101840 2024-04-23T02:34:03.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.752+0000 7fe9c77fe700 1 -- 172.21.15.71:0/2165639098 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe9cc00f450 con 0x7fe9d8101840 2024-04-23T02:34:03.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.752+0000 7fe9dcaba700 1 -- 172.21.15.71:0/2165639098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d8196920 con 0x7fe9d8101840 2024-04-23T02:34:03.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.752+0000 7fe9c77fe700 1 -- 172.21.15.71:0/2165639098 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9cc007ea0 con 0x7fe9d8101840 2024-04-23T02:34:03.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.752+0000 7fe9dcaba700 1 -- 172.21.15.71:0/2165639098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d8196e70 con 0x7fe9d8101840 2024-04-23T02:34:03.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.753+0000 7fe9dcaba700 1 -- 172.21.15.71:0/2165639098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d8109230 con 0x7fe9d8101840 2024-04-23T02:34:03.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.754+0000 7fe9c77fe700 1 -- 172.21.15.71:0/2165639098 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe9cc00f5c0 con 0x7fe9d8101840 2024-04-23T02:34:03.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.755+0000 7fe9c77fe700 1 --2- 172.21.15.71:0/2165639098 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe9c006cce0 0x7fe9c006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:03.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.755+0000 7fe9c77fe700 1 -- 172.21.15.71:0/2165639098 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe9cc08c7f0 con 0x7fe9d8101840 2024-04-23T02:34:03.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.755+0000 7fe9d659c700 1 --2- 172.21.15.71:0/2165639098 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe9c006cce0 0x7fe9c006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:03.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.755+0000 7fe9d659c700 1 --2- 172.21.15.71:0/2165639098 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe9c006cce0 0x7fe9c006f1a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8006010 tx=0x7fe9c801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:03.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.758+0000 7fe9c77fe700 1 -- 172.21.15.71:0/2165639098 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe9cc05a410 con 0x7fe9d8101840 2024-04-23T02:34:03.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.970+0000 7fe9dcaba700 1 -- 172.21.15.71:0/2165639098 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9d8197140 con 0x7fe9c006cce0 2024-04-23T02:34:03.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.982+0000 7fe9c77fe700 1 -- 172.21.15.71:0/2165639098 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe9d8197140 con 0x7fe9c006cce0 2024-04-23T02:34:03.984 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:34:03.984 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (17m) 11m ago 29m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:34:03.984 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (29m) 11m ago 29m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:34:03.984 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (28m) 11m ago 28m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:34:03.984 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (15m) 11m ago 29m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:34:03.985 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (15m) 11m ago 28m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:34:03.985 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (17m) 11m ago 28m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:34:03.985 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (11m) 11m ago 23m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:34:03.985 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (11m) 11m ago 23m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:34:03.985 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (23m) 11m ago 22m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:34:03.985 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (11m) 11m ago 23m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:34:03.985 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (19m) 11m ago 30m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:34:03.985 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (18m) 11m ago 27m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:34:03.985 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (16m) 11m ago 30m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:34:03.985 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (15m) 11m ago 27m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:34:03.985 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (18m) 11m ago 29m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:34:03.985 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (18m) 11m ago 28m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:34:03.985 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (15m) 11m ago 27m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:34:03.985 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (14m) 11m ago 26m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:34:03.985 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (13m) 11m ago 26m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:34:03.986 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (13m) 11m ago 25m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:34:03.986 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (12m) 11m ago 25m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:34:03.986 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (12m) 11m ago 24m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:34:03.986 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (18m) 11m ago 28m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:34:03.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.984+0000 7fe9dcaba700 1 -- 172.21.15.71:0/2165639098 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe9c006cce0 msgr2=0x7fe9c006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:03.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.984+0000 7fe9dcaba700 1 --2- 172.21.15.71:0/2165639098 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe9c006cce0 0x7fe9c006f1a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8006010 tx=0x7fe9c801a040 comp rx=0 tx=0).stop 2024-04-23T02:34:03.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.984+0000 7fe9dcaba700 1 -- 172.21.15.71:0/2165639098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9d8101840 msgr2=0x7fe9d819cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:03.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.984+0000 7fe9dcaba700 1 --2- 172.21.15.71:0/2165639098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9d8101840 0x7fe9d819cb00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00c8f0 tx=0x7fe9cc00cc00 comp rx=0 tx=0).stop 2024-04-23T02:34:03.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.984+0000 7fe9dcaba700 1 -- 172.21.15.71:0/2165639098 shutdown_connections 2024-04-23T02:34:03.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.984+0000 7fe9dcaba700 1 --2- 172.21.15.71:0/2165639098 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe9c006cce0 0x7fe9c006f1a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:03.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.984+0000 7fe9dcaba700 1 --2- 172.21.15.71:0/2165639098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9d8100e90 0x7fe9d819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:03.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.984+0000 7fe9dcaba700 1 --2- 172.21.15.71:0/2165639098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9d8101840 0x7fe9d819cb00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:03.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.984+0000 7fe9dcaba700 1 -- 172.21.15.71:0/2165639098 >> 172.21.15.71:0/2165639098 conn(0x7fe9d80fc740 msgr2=0x7fe9d80fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:03.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.984+0000 7fe9dcaba700 1 -- 172.21.15.71:0/2165639098 shutdown_connections 2024-04-23T02:34:03.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:03.984+0000 7fe9dcaba700 1 -- 172.21.15.71:0/2165639098 wait complete. 2024-04-23T02:34:04.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.085+0000 7f2924f44700 1 -- 172.21.15.71:0/1351294509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2920068580 msgr2=0x7f29200689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:04.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.086+0000 7f2924f44700 1 --2- 172.21.15.71:0/1351294509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2920068580 0x7f29200689e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2910009910 tx=0x7f2910009c20 comp rx=0 tx=0).stop 2024-04-23T02:34:04.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.086+0000 7f2924f44700 1 -- 172.21.15.71:0/1351294509 shutdown_connections 2024-04-23T02:34:04.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.086+0000 7f2924f44700 1 --2- 172.21.15.71:0/1351294509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2920068580 0x7f29200689e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:04.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.086+0000 7f2924f44700 1 --2- 172.21.15.71:0/1351294509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2920106cd0 0x7f29201070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:04.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.086+0000 7f2924f44700 1 -- 172.21.15.71:0/1351294509 >> 172.21.15.71:0/1351294509 conn(0x7f2920075670 msgr2=0x7f2920075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:04.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.086+0000 7f2924f44700 1 -- 172.21.15.71:0/1351294509 shutdown_connections 2024-04-23T02:34:04.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.086+0000 7f2924f44700 1 -- 172.21.15.71:0/1351294509 wait complete. 2024-04-23T02:34:04.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.086+0000 7f2924f44700 1 Processor -- start 2024-04-23T02:34:04.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.086+0000 7f2924f44700 1 -- start start 2024-04-23T02:34:04.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.087+0000 7f2924f44700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2920068580 0x7f292019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:04.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.087+0000 7f2924f44700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2920106cd0 0x7f292019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:04.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.087+0000 7f2924f44700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292019d290 con 0x7f2920106cd0 2024-04-23T02:34:04.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.087+0000 7f2924f44700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2920196740 con 0x7f2920068580 2024-04-23T02:34:04.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.087+0000 7f291dd9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2920106cd0 0x7f292019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:04.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.087+0000 7f291dd9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2920106cd0 0x7f292019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44292/0 (socket says 172.21.15.71:44292) 2024-04-23T02:34:04.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.087+0000 7f291dd9b700 1 -- 172.21.15.71:0/1786398828 learned_addr learned my addr 172.21.15.71:0/1786398828 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:34:04.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.087+0000 7f291e59c700 1 --2- 172.21.15.71:0/1786398828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2920068580 0x7f292019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:04.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.087+0000 7f291dd9b700 1 -- 172.21.15.71:0/1786398828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2920068580 msgr2=0x7f292019c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:04.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.087+0000 7f291dd9b700 1 --2- 172.21.15.71:0/1786398828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2920068580 0x7f292019c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:04.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.087+0000 7f291dd9b700 1 -- 172.21.15.71:0/1786398828 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29100095f0 con 0x7f2920106cd0 2024-04-23T02:34:04.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.087+0000 7f291dd9b700 1 --2- 172.21.15.71:0/1786398828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2920106cd0 0x7f292019cc00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2910006010 tx=0x7f291000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:04.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.087+0000 7f290f7fe700 1 -- 172.21.15.71:0/1786398828 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f291001d070 con 0x7f2920106cd0 2024-04-23T02:34:04.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.087+0000 7f290f7fe700 1 -- 172.21.15.71:0/1786398828 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f291000f460 con 0x7f2920106cd0 2024-04-23T02:34:04.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.087+0000 7f2924f44700 1 -- 172.21.15.71:0/1786398828 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29201969c0 con 0x7f2920106cd0 2024-04-23T02:34:04.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.087+0000 7f290f7fe700 1 -- 172.21.15.71:0/1786398828 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2910021640 con 0x7f2920106cd0 2024-04-23T02:34:04.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.087+0000 7f2924f44700 1 -- 172.21.15.71:0/1786398828 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2920196eb0 con 0x7f2920106cd0 2024-04-23T02:34:04.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.088+0000 7f2924f44700 1 -- 172.21.15.71:0/1786398828 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29201091f0 con 0x7f2920106cd0 2024-04-23T02:34:04.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.089+0000 7f290f7fe700 1 -- 172.21.15.71:0/1786398828 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f291000fa60 con 0x7f2920106cd0 2024-04-23T02:34:04.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.089+0000 7f290f7fe700 1 --2- 172.21.15.71:0/1786398828 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f290806cce0 0x7f290806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:04.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.089+0000 7f290f7fe700 1 -- 172.21.15.71:0/1786398828 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f291008d880 con 0x7f2920106cd0 2024-04-23T02:34:04.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.089+0000 7f291e59c700 1 --2- 172.21.15.71:0/1786398828 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f290806cce0 0x7f290806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:04.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.090+0000 7f291e59c700 1 --2- 172.21.15.71:0/1786398828 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f290806cce0 0x7f290806f1a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2914006fd0 tx=0x7f2914008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:04.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.094+0000 7f290f7fe700 1 -- 172.21.15.71:0/1786398828 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f291005b380 con 0x7f2920106cd0 2024-04-23T02:34:04.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:04 smithi071 ceph-mon[144544]: pgmap v622: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:04.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:04 smithi071 ceph-mon[144544]: from='client.34958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:34:04.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.369+0000 7f2924f44700 1 -- 172.21.15.71:0/1786398828 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f292004f2e0 con 0x7f2920106cd0 2024-04-23T02:34:04.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.369+0000 7f290f7fe700 1 -- 172.21.15.71:0/1786398828 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f291005af10 con 0x7f2920106cd0 2024-04-23T02:34:04.372 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:34:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:34:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:34:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:34:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:34:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:34:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:34:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:34:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:34:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:34:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:34:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:34:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:34:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:34:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:34:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:34:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:34:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:34:04.373 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:34:04.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.371+0000 7f2924f44700 1 -- 172.21.15.71:0/1786398828 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f290806cce0 msgr2=0x7f290806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:04.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.371+0000 7f2924f44700 1 --2- 172.21.15.71:0/1786398828 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f290806cce0 0x7f290806f1a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2914006fd0 tx=0x7f2914008040 comp rx=0 tx=0).stop 2024-04-23T02:34:04.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.371+0000 7f2924f44700 1 -- 172.21.15.71:0/1786398828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2920106cd0 msgr2=0x7f292019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:04.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.371+0000 7f2924f44700 1 --2- 172.21.15.71:0/1786398828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2920106cd0 0x7f292019cc00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2910006010 tx=0x7f291000be20 comp rx=0 tx=0).stop 2024-04-23T02:34:04.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.371+0000 7f2924f44700 1 -- 172.21.15.71:0/1786398828 shutdown_connections 2024-04-23T02:34:04.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.371+0000 7f2924f44700 1 --2- 172.21.15.71:0/1786398828 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f290806cce0 0x7f290806f1a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:04.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.371+0000 7f2924f44700 1 --2- 172.21.15.71:0/1786398828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2920068580 0x7f292019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:04.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.371+0000 7f2924f44700 1 --2- 172.21.15.71:0/1786398828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2920106cd0 0x7f292019cc00 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:04.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.371+0000 7f2924f44700 1 -- 172.21.15.71:0/1786398828 >> 172.21.15.71:0/1786398828 conn(0x7f2920075670 msgr2=0x7f29200fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:04.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.371+0000 7f2924f44700 1 -- 172.21.15.71:0/1786398828 shutdown_connections 2024-04-23T02:34:04.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.371+0000 7f2924f44700 1 -- 172.21.15.71:0/1786398828 wait complete. 2024-04-23T02:34:04.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.476+0000 7f684bbeb700 1 -- 172.21.15.71:0/2903310821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68440ff5a0 msgr2=0x7f68440ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:04.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.476+0000 7f684bbeb700 1 --2- 172.21.15.71:0/2903310821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68440ff5a0 0x7f68440ff980 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6840009910 tx=0x7f6840009c20 comp rx=0 tx=0).stop 2024-04-23T02:34:04.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.476+0000 7f684bbeb700 1 -- 172.21.15.71:0/2903310821 shutdown_connections 2024-04-23T02:34:04.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.476+0000 7f684bbeb700 1 --2- 172.21.15.71:0/2903310821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68440fff50 0x7f6844105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:04.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.476+0000 7f684bbeb700 1 --2- 172.21.15.71:0/2903310821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68440ff5a0 0x7f68440ff980 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:04.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.476+0000 7f684bbeb700 1 -- 172.21.15.71:0/2903310821 >> 172.21.15.71:0/2903310821 conn(0x7f6844074e30 msgr2=0x7f6844075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:04.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.476+0000 7f684bbeb700 1 -- 172.21.15.71:0/2903310821 shutdown_connections 2024-04-23T02:34:04.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.476+0000 7f684bbeb700 1 -- 172.21.15.71:0/2903310821 wait complete. 2024-04-23T02:34:04.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.476+0000 7f684bbeb700 1 Processor -- start 2024-04-23T02:34:04.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.476+0000 7f684bbeb700 1 -- start start 2024-04-23T02:34:04.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.476+0000 7f684bbeb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68440ff5a0 0x7f6844100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:04.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.476+0000 7f684bbeb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68440fff50 0x7f6844100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:04.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.476+0000 7f684bbeb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6844100fa0 con 0x7f68440fff50 2024-04-23T02:34:04.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.476+0000 7f684bbeb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68441010e0 con 0x7f68440ff5a0 2024-04-23T02:34:04.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.477+0000 7f6849186700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68440fff50 0x7f6844100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:04.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.477+0000 7f6849186700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68440fff50 0x7f6844100a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44306/0 (socket says 172.21.15.71:44306) 2024-04-23T02:34:04.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.477+0000 7f6849186700 1 -- 172.21.15.71:0/1622546163 learned_addr learned my addr 172.21.15.71:0/1622546163 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:34:04.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.477+0000 7f6849987700 1 --2- 172.21.15.71:0/1622546163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68440ff5a0 0x7f6844100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:04.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.477+0000 7f6849186700 1 -- 172.21.15.71:0/1622546163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68440ff5a0 msgr2=0x7f6844100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:04.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.477+0000 7f6849186700 1 --2- 172.21.15.71:0/1622546163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68440ff5a0 0x7f6844100520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:04.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.477+0000 7f6849186700 1 -- 172.21.15.71:0/1622546163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68400095f0 con 0x7f68440fff50 2024-04-23T02:34:04.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.477+0000 7f6849186700 1 --2- 172.21.15.71:0/1622546163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68440fff50 0x7f6844100a60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f683400c8f0 tx=0x7f683400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:04.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.477+0000 7f683affd700 1 -- 172.21.15.71:0/1622546163 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68340078a0 con 0x7f68440fff50 2024-04-23T02:34:04.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.477+0000 7f683affd700 1 -- 172.21.15.71:0/1622546163 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f683400f450 con 0x7f68440fff50 2024-04-23T02:34:04.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.477+0000 7f684bbeb700 1 -- 172.21.15.71:0/1622546163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68441a29a0 con 0x7f68440fff50 2024-04-23T02:34:04.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.477+0000 7f683affd700 1 -- 172.21.15.71:0/1622546163 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6834007ea0 con 0x7f68440fff50 2024-04-23T02:34:04.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.477+0000 7f684bbeb700 1 -- 172.21.15.71:0/1622546163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68441a2db0 con 0x7f68440fff50 2024-04-23T02:34:04.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.478+0000 7f684bbeb700 1 -- 172.21.15.71:0/1622546163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6844109270 con 0x7f68440fff50 2024-04-23T02:34:04.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.479+0000 7f683affd700 1 -- 172.21.15.71:0/1622546163 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f683400f5c0 con 0x7f68440fff50 2024-04-23T02:34:04.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.479+0000 7f683affd700 1 --2- 172.21.15.71:0/1622546163 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f683006cce0 0x7f683006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:04.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.479+0000 7f6849987700 1 --2- 172.21.15.71:0/1622546163 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f683006cce0 0x7f683006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:04.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.479+0000 7f683affd700 1 -- 172.21.15.71:0/1622546163 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f683408c4d0 con 0x7f68440fff50 2024-04-23T02:34:04.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.480+0000 7f6849987700 1 --2- 172.21.15.71:0/1622546163 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f683006cce0 0x7f683006f1a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6840006010 tx=0x7f684001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:04.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.484+0000 7f683affd700 1 -- 172.21.15.71:0/1622546163 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6834056540 con 0x7f68440fff50 2024-04-23T02:34:04.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:04 smithi192 ceph-mon[108617]: pgmap v622: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:04.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:04 smithi192 ceph-mon[108617]: from='client.34958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:34:04.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.718+0000 7f684bbeb700 1 -- 172.21.15.71:0/1622546163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f684404f2e0 con 0x7f68440fff50 2024-04-23T02:34:04.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.718+0000 7f683affd700 1 -- 172.21.15.71:0/1622546163 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f6834059b60 con 0x7f68440fff50 2024-04-23T02:34:04.721 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:34:04.721 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:34:04.721 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:34:04.721 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:34:04.721 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:34:04.721 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:34:04.721 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:34:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:34:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:34:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:34:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:34:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:34:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:34:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:34:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:34:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:34:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:34:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:34:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:34:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:34:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:34:04.723 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:34:04.723 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:34:04.723 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:34:04.723 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:34:04.723 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:34:04.723 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:34:04.723 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:34:04.723 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:34:04.723 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:34:04.723 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:34:04.723 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:34:04.723 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:34:04.723 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:34:04.723 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:34:04.723 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:34:04.724 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:34:04.724 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:34:04.724 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:34:04.724 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:34:04.724 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:34:04.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.720+0000 7f684bbeb700 1 -- 172.21.15.71:0/1622546163 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f683006cce0 msgr2=0x7f683006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:04.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.720+0000 7f684bbeb700 1 --2- 172.21.15.71:0/1622546163 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f683006cce0 0x7f683006f1a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6840006010 tx=0x7f684001a040 comp rx=0 tx=0).stop 2024-04-23T02:34:04.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.720+0000 7f684bbeb700 1 -- 172.21.15.71:0/1622546163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68440fff50 msgr2=0x7f6844100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:04.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.720+0000 7f684bbeb700 1 --2- 172.21.15.71:0/1622546163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68440fff50 0x7f6844100a60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f683400c8f0 tx=0x7f683400cc00 comp rx=0 tx=0).stop 2024-04-23T02:34:04.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.721+0000 7f684bbeb700 1 -- 172.21.15.71:0/1622546163 shutdown_connections 2024-04-23T02:34:04.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.721+0000 7f684bbeb700 1 --2- 172.21.15.71:0/1622546163 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f683006cce0 0x7f683006f1a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:04.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.721+0000 7f684bbeb700 1 --2- 172.21.15.71:0/1622546163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68440ff5a0 0x7f6844100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:04.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.721+0000 7f684bbeb700 1 --2- 172.21.15.71:0/1622546163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68440fff50 0x7f6844100a60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:04.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.721+0000 7f684bbeb700 1 -- 172.21.15.71:0/1622546163 >> 172.21.15.71:0/1622546163 conn(0x7f6844074e30 msgr2=0x7f68440fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:04.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.721+0000 7f684bbeb700 1 -- 172.21.15.71:0/1622546163 shutdown_connections 2024-04-23T02:34:04.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.721+0000 7f684bbeb700 1 -- 172.21.15.71:0/1622546163 wait complete. 2024-04-23T02:34:04.725 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:34:04.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.826+0000 7f7ecfe7f700 1 -- 172.21.15.71:0/434025030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ec80ffda0 msgr2=0x7f7ec8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:04.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.826+0000 7f7ecfe7f700 1 --2- 172.21.15.71:0/434025030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ec80ffda0 0x7f7ec8100180 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009910 tx=0x7f7eb8009c20 comp rx=0 tx=0).stop 2024-04-23T02:34:04.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.826+0000 7f7ecfe7f700 1 -- 172.21.15.71:0/434025030 shutdown_connections 2024-04-23T02:34:04.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.826+0000 7f7ecfe7f700 1 --2- 172.21.15.71:0/434025030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ec8100750 0x7f7ec81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:04.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.826+0000 7f7ecfe7f700 1 --2- 172.21.15.71:0/434025030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ec80ffda0 0x7f7ec8100180 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:04.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.826+0000 7f7ecfe7f700 1 -- 172.21.15.71:0/434025030 >> 172.21.15.71:0/434025030 conn(0x7f7ec80fb650 msgr2=0x7f7ec80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:04.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.826+0000 7f7ecfe7f700 1 -- 172.21.15.71:0/434025030 shutdown_connections 2024-04-23T02:34:04.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.826+0000 7f7ecfe7f700 1 -- 172.21.15.71:0/434025030 wait complete. 2024-04-23T02:34:04.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.827+0000 7f7ecfe7f700 1 Processor -- start 2024-04-23T02:34:04.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.827+0000 7f7ecfe7f700 1 -- start start 2024-04-23T02:34:04.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.827+0000 7f7ecfe7f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ec80ffda0 0x7f7ec8197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:04.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.827+0000 7f7ecfe7f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ec8100750 0x7f7ec8197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:04.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.827+0000 7f7ecfe7f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec81981a0 con 0x7f7ec8100750 2024-04-23T02:34:04.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.827+0000 7f7ecfe7f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec81982e0 con 0x7f7ec80ffda0 2024-04-23T02:34:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.827+0000 7f7ecd41a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ec8100750 0x7f7ec8197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.827+0000 7f7ecd41a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ec8100750 0x7f7ec8197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44326/0 (socket says 172.21.15.71:44326) 2024-04-23T02:34:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.827+0000 7f7ecd41a700 1 -- 172.21.15.71:0/3889070939 learned_addr learned my addr 172.21.15.71:0/3889070939 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:34:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.827+0000 7f7ecdc1b700 1 --2- 172.21.15.71:0/3889070939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ec80ffda0 0x7f7ec8197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.828+0000 7f7ecd41a700 1 -- 172.21.15.71:0/3889070939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ec80ffda0 msgr2=0x7f7ec8197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.828+0000 7f7ecd41a700 1 --2- 172.21.15.71:0/3889070939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ec80ffda0 0x7f7ec8197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.828+0000 7f7ecd41a700 1 -- 172.21.15.71:0/3889070939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb80095f0 con 0x7f7ec8100750 2024-04-23T02:34:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.828+0000 7f7ecd41a700 1 --2- 172.21.15.71:0/3889070939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ec8100750 0x7f7ec8197b80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400c8f0 tx=0x7f7ec400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.828+0000 7f7ebeffd700 1 -- 172.21.15.71:0/3889070939 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ec4007a50 con 0x7f7ec8100750 2024-04-23T02:34:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.828+0000 7f7ebeffd700 1 -- 172.21.15.71:0/3889070939 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7ec4007bb0 con 0x7f7ec8100750 2024-04-23T02:34:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.828+0000 7f7ebeffd700 1 -- 172.21.15.71:0/3889070939 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ec4018630 con 0x7f7ec8100750 2024-04-23T02:34:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.828+0000 7f7ecfe7f700 1 -- 172.21.15.71:0/3889070939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec81a06b0 con 0x7f7ec8100750 2024-04-23T02:34:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.828+0000 7f7ecfe7f700 1 -- 172.21.15.71:0/3889070939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec81a0b70 con 0x7f7ec8100750 2024-04-23T02:34:04.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.829+0000 7f7ecfe7f700 1 -- 172.21.15.71:0/3889070939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec8108140 con 0x7f7ec8100750 2024-04-23T02:34:04.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.829+0000 7f7ebeffd700 1 -- 172.21.15.71:0/3889070939 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7ec4007d20 con 0x7f7ec8100750 2024-04-23T02:34:04.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.830+0000 7f7ebeffd700 1 --2- 172.21.15.71:0/3889070939 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7eb406cce0 0x7f7eb406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:04.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.830+0000 7f7ebeffd700 1 -- 172.21.15.71:0/3889070939 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7ec408c4b0 con 0x7f7ec8100750 2024-04-23T02:34:04.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.830+0000 7f7ecdc1b700 1 --2- 172.21.15.71:0/3889070939 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7eb406cce0 0x7f7eb406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:04.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.830+0000 7f7ecdc1b700 1 --2- 172.21.15.71:0/3889070939 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7eb406cce0 0x7f7eb406f1a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800b5c0 tx=0x7f7eb8005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:04.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:04.834+0000 7f7ebeffd700 1 -- 172.21.15.71:0/3889070939 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7ec40565a0 con 0x7f7ec8100750 2024-04-23T02:34:05.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.044+0000 7f7ecfe7f700 1 -- 172.21.15.71:0/3889070939 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ec80611d0 con 0x7f7eb406cce0 2024-04-23T02:34:05.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.045+0000 7f7ebeffd700 1 -- 172.21.15.71:0/3889070939 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7ec80611d0 con 0x7f7eb406cce0 2024-04-23T02:34:05.048 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:34:05.048 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:34:05.048 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:34:05.048 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:34:05.048 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:34:05.048 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:34:05.048 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:34:05.048 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:34:05.048 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:34:05.048 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:34:05.049 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:34:05.049 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:34:05.049 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:34:05.049 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:34:05.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.047+0000 7f7ecfe7f700 1 -- 172.21.15.71:0/3889070939 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7eb406cce0 msgr2=0x7f7eb406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:05.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.047+0000 7f7ecfe7f700 1 --2- 172.21.15.71:0/3889070939 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7eb406cce0 0x7f7eb406f1a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800b5c0 tx=0x7f7eb8005fd0 comp rx=0 tx=0).stop 2024-04-23T02:34:05.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.047+0000 7f7ecfe7f700 1 -- 172.21.15.71:0/3889070939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ec8100750 msgr2=0x7f7ec8197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:05.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.047+0000 7f7ecfe7f700 1 --2- 172.21.15.71:0/3889070939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ec8100750 0x7f7ec8197b80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400c8f0 tx=0x7f7ec400ccb0 comp rx=0 tx=0).stop 2024-04-23T02:34:05.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.048+0000 7f7ecfe7f700 1 -- 172.21.15.71:0/3889070939 shutdown_connections 2024-04-23T02:34:05.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.048+0000 7f7ecfe7f700 1 --2- 172.21.15.71:0/3889070939 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7eb406cce0 0x7f7eb406f1a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:05.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.048+0000 7f7ecfe7f700 1 --2- 172.21.15.71:0/3889070939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ec80ffda0 0x7f7ec8197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:05.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.048+0000 7f7ecfe7f700 1 --2- 172.21.15.71:0/3889070939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ec8100750 0x7f7ec8197b80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:05.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.048+0000 7f7ecfe7f700 1 -- 172.21.15.71:0/3889070939 >> 172.21.15.71:0/3889070939 conn(0x7f7ec80fb650 msgr2=0x7f7ec80fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:05.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.048+0000 7f7ecfe7f700 1 -- 172.21.15.71:0/3889070939 shutdown_connections 2024-04-23T02:34:05.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.048+0000 7f7ecfe7f700 1 -- 172.21.15.71:0/3889070939 wait complete. 2024-04-23T02:34:05.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.150+0000 7fc32b858700 1 -- 172.21.15.71:0/2046927718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc324100e90 msgr2=0x7fc324101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:05.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.150+0000 7fc32b858700 1 --2- 172.21.15.71:0/2046927718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc324100e90 0x7fc324101270 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc320009910 tx=0x7fc320009c20 comp rx=0 tx=0).stop 2024-04-23T02:34:05.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.150+0000 7fc32b858700 1 -- 172.21.15.71:0/2046927718 shutdown_connections 2024-04-23T02:34:05.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.151+0000 7fc32b858700 1 --2- 172.21.15.71:0/2046927718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc324101840 0x7fc324105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:05.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.151+0000 7fc32b858700 1 --2- 172.21.15.71:0/2046927718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc324100e90 0x7fc324101270 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:05.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.151+0000 7fc32b858700 1 -- 172.21.15.71:0/2046927718 >> 172.21.15.71:0/2046927718 conn(0x7fc3240fc740 msgr2=0x7fc3240feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:05.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.151+0000 7fc32b858700 1 -- 172.21.15.71:0/2046927718 shutdown_connections 2024-04-23T02:34:05.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.151+0000 7fc32b858700 1 -- 172.21.15.71:0/2046927718 wait complete. 2024-04-23T02:34:05.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.151+0000 7fc32b858700 1 Processor -- start 2024-04-23T02:34:05.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.151+0000 7fc32b858700 1 -- start start 2024-04-23T02:34:05.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.151+0000 7fc32b858700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc324100e90 0x7fc32419c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:05.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.151+0000 7fc32b858700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc324101840 0x7fc32419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:05.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.151+0000 7fc32b858700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32419d1b0 con 0x7fc324100e90 2024-04-23T02:34:05.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.151+0000 7fc32b858700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc324196660 con 0x7fc324101840 2024-04-23T02:34:05.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.152+0000 7fc3295f4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc324100e90 0x7fc32419c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:05.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.152+0000 7fc3295f4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc324100e90 0x7fc32419c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44342/0 (socket says 172.21.15.71:44342) 2024-04-23T02:34:05.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.152+0000 7fc3295f4700 1 -- 172.21.15.71:0/672765601 learned_addr learned my addr 172.21.15.71:0/672765601 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:34:05.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.152+0000 7fc328df3700 1 --2- 172.21.15.71:0/672765601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc324101840 0x7fc32419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:05.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.152+0000 7fc3295f4700 1 -- 172.21.15.71:0/672765601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc324101840 msgr2=0x7fc32419cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:05.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.152+0000 7fc3295f4700 1 --2- 172.21.15.71:0/672765601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc324101840 0x7fc32419cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:05.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.152+0000 7fc3295f4700 1 -- 172.21.15.71:0/672765601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3200095f0 con 0x7fc324100e90 2024-04-23T02:34:05.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.152+0000 7fc328df3700 1 --2- 172.21.15.71:0/672765601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc324101840 0x7fc32419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:34:05.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.152+0000 7fc3295f4700 1 --2- 172.21.15.71:0/672765601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc324100e90 0x7fc32419c5e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc320006010 tx=0x7fc32000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:05.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.152+0000 7fc31a7fc700 1 -- 172.21.15.71:0/672765601 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc32001d070 con 0x7fc324100e90 2024-04-23T02:34:05.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.152+0000 7fc31a7fc700 1 -- 172.21.15.71:0/672765601 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc32000f460 con 0x7fc324100e90 2024-04-23T02:34:05.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.152+0000 7fc31a7fc700 1 -- 172.21.15.71:0/672765601 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc320021640 con 0x7fc324100e90 2024-04-23T02:34:05.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.152+0000 7fc32b858700 1 -- 172.21.15.71:0/672765601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3241968e0 con 0x7fc324100e90 2024-04-23T02:34:05.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.152+0000 7fc32b858700 1 -- 172.21.15.71:0/672765601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc324196dd0 con 0x7fc324100e90 2024-04-23T02:34:05.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.153+0000 7fc32b858700 1 -- 172.21.15.71:0/672765601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc324109230 con 0x7fc324100e90 2024-04-23T02:34:05.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.154+0000 7fc31a7fc700 1 -- 172.21.15.71:0/672765601 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc32000fa60 con 0x7fc324100e90 2024-04-23T02:34:05.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.154+0000 7fc31a7fc700 1 --2- 172.21.15.71:0/672765601 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc31006cce0 0x7fc31006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:05.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.154+0000 7fc31a7fc700 1 -- 172.21.15.71:0/672765601 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc32008d880 con 0x7fc324100e90 2024-04-23T02:34:05.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.154+0000 7fc328df3700 1 --2- 172.21.15.71:0/672765601 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc31006cce0 0x7fc31006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:05.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.154+0000 7fc328df3700 1 --2- 172.21.15.71:0/672765601 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc31006cce0 0x7fc31006f1a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc314005fd0 tx=0x7fc314005dc0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:05.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.158+0000 7fc31a7fc700 1 -- 172.21.15.71:0/672765601 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc32005b380 con 0x7fc324100e90 2024-04-23T02:34:05.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.435+0000 7fc32b858700 1 -- 172.21.15.71:0/672765601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc324197c40 con 0x7fc324100e90 2024-04-23T02:34:05.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.435+0000 7fc31a7fc700 1 -- 172.21.15.71:0/672765601 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fc32005af10 con 0x7fc324100e90 2024-04-23T02:34:05.438 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:34:05.438 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:34:05.438 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:34:05.438 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:34:05.438 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:34:05.438 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:34:05.438 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:34:05.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.438+0000 7fc32b858700 1 -- 172.21.15.71:0/672765601 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc31006cce0 msgr2=0x7fc31006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:05.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.438+0000 7fc32b858700 1 --2- 172.21.15.71:0/672765601 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc31006cce0 0x7fc31006f1a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc314005fd0 tx=0x7fc314005dc0 comp rx=0 tx=0).stop 2024-04-23T02:34:05.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.438+0000 7fc32b858700 1 -- 172.21.15.71:0/672765601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc324100e90 msgr2=0x7fc32419c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:05.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.438+0000 7fc32b858700 1 --2- 172.21.15.71:0/672765601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc324100e90 0x7fc32419c5e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc320006010 tx=0x7fc32000be20 comp rx=0 tx=0).stop 2024-04-23T02:34:05.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.438+0000 7fc32b858700 1 -- 172.21.15.71:0/672765601 shutdown_connections 2024-04-23T02:34:05.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.438+0000 7fc32b858700 1 --2- 172.21.15.71:0/672765601 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc31006cce0 0x7fc31006f1a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:05.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.438+0000 7fc32b858700 1 --2- 172.21.15.71:0/672765601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc324100e90 0x7fc32419c5e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:05.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.438+0000 7fc32b858700 1 --2- 172.21.15.71:0/672765601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc324101840 0x7fc32419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:05.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.438+0000 7fc32b858700 1 -- 172.21.15.71:0/672765601 >> 172.21.15.71:0/672765601 conn(0x7fc3240fc740 msgr2=0x7fc3240fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:05.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.438+0000 7fc32b858700 1 -- 172.21.15.71:0/672765601 shutdown_connections 2024-04-23T02:34:05.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:05.438+0000 7fc32b858700 1 -- 172.21.15.71:0/672765601 wait complete. 2024-04-23T02:34:05.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:05 smithi071 ceph-mon[144544]: from='client.34962 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:34:05.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:05 smithi071 ceph-mon[144544]: from='client.34966 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:34:05.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:05 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1786398828' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:34:05.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:05 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1622546163' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:34:05.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:34:05.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:05 smithi192 ceph-mon[108617]: from='client.34962 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:34:05.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:05 smithi192 ceph-mon[108617]: from='client.34966 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:34:05.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:05 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1786398828' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:34:05.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:05 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1622546163' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:34:05.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:34:06.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:06 smithi071 ceph-mon[144544]: pgmap v623: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:06.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:06 smithi071 ceph-mon[144544]: from='client.34978 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:34:06.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:06 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/672765601' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:34:06.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:06 smithi192 ceph-mon[108617]: pgmap v623: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:06.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:06 smithi192 ceph-mon[108617]: from='client.34978 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:34:06.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:06 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/672765601' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:34:08.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:08 smithi071 ceph-mon[144544]: pgmap v624: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:08.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:08 smithi192 ceph-mon[108617]: pgmap v624: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:10.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:10 smithi071 ceph-mon[144544]: pgmap v625: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:10.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:10 smithi192 ceph-mon[108617]: pgmap v625: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:12.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:12 smithi071 ceph-mon[144544]: pgmap v626: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:12.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:12 smithi192 ceph-mon[108617]: pgmap v626: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:14.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:14 smithi192 ceph-mon[108617]: pgmap v627: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:14.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:14 smithi071 ceph-mon[144544]: pgmap v627: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:16.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:16 smithi071 ceph-mon[144544]: pgmap v628: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:16.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:16 smithi192 ceph-mon[108617]: pgmap v628: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:18.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:18 smithi071 ceph-mon[144544]: pgmap v629: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:18.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:18 smithi192 ceph-mon[108617]: pgmap v629: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:20.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:20 smithi071 ceph-mon[144544]: pgmap v630: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:20.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:34:20.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:20 smithi192 ceph-mon[108617]: pgmap v630: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:20.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:34:22.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:22 smithi071 ceph-mon[144544]: pgmap v631: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:22.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:22 smithi192 ceph-mon[108617]: pgmap v631: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:24.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:24 smithi192 ceph-mon[108617]: pgmap v632: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:24.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:24 smithi071 ceph-mon[144544]: pgmap v632: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:26.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:26 smithi071 ceph-mon[144544]: pgmap v633: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:26.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:26 smithi192 ceph-mon[108617]: pgmap v633: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:28.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:28 smithi071 ceph-mon[144544]: pgmap v634: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:28.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:28 smithi192 ceph-mon[108617]: pgmap v634: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:30.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:30 smithi071 ceph-mon[144544]: pgmap v635: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:30.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:30 smithi192 ceph-mon[108617]: pgmap v635: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:32.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:32 smithi071 ceph-mon[144544]: pgmap v636: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:32.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:32 smithi192 ceph-mon[108617]: pgmap v636: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:34.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:34 smithi071 ceph-mon[144544]: pgmap v637: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:34.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:34 smithi192 ceph-mon[108617]: pgmap v637: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:35.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:34:35.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.554+0000 7f94aa875700 1 -- 172.21.15.71:0/3056587784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a4105bd0 msgr2=0x7f94a4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:35.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.554+0000 7f94aa875700 1 --2- 172.21.15.71:0/3056587784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a4105bd0 0x7f94a4105fb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9498009910 tx=0x7f9498009c20 comp rx=0 tx=0).stop 2024-04-23T02:34:35.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.554+0000 7f94aa875700 1 -- 172.21.15.71:0/3056587784 shutdown_connections 2024-04-23T02:34:35.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.554+0000 7f94aa875700 1 --2- 172.21.15.71:0/3056587784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94a40ffc20 0x7f94a4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:35.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.554+0000 7f94aa875700 1 --2- 172.21.15.71:0/3056587784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a4105bd0 0x7f94a4105fb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:35.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.554+0000 7f94aa875700 1 -- 172.21.15.71:0/3056587784 >> 172.21.15.71:0/3056587784 conn(0x7f94a4075670 msgr2=0x7f94a4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:35.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.554+0000 7f94aa875700 1 -- 172.21.15.71:0/3056587784 shutdown_connections 2024-04-23T02:34:35.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.554+0000 7f94aa875700 1 -- 172.21.15.71:0/3056587784 wait complete. 2024-04-23T02:34:35.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.555+0000 7f94aa875700 1 Processor -- start 2024-04-23T02:34:35.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.555+0000 7f94aa875700 1 -- start start 2024-04-23T02:34:35.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.555+0000 7f94aa875700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94a40ffc20 0x7f94a4101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:35.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.555+0000 7f94aa875700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a4105bd0 0x7f94a4101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:35.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.555+0000 7f94aa875700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a41022c0 con 0x7f94a4105bd0 2024-04-23T02:34:35.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.555+0000 7f94aa875700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a4102400 con 0x7f94a40ffc20 2024-04-23T02:34:35.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.555+0000 7f94a37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a4105bd0 0x7f94a4101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:35.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.555+0000 7f94a37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a4105bd0 0x7f94a4101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60986/0 (socket says 172.21.15.71:60986) 2024-04-23T02:34:35.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.555+0000 7f94a37fe700 1 -- 172.21.15.71:0/2174633849 learned_addr learned my addr 172.21.15.71:0/2174633849 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:34:35.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.555+0000 7f94a3fff700 1 --2- 172.21.15.71:0/2174633849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94a40ffc20 0x7f94a4101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:35.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.556+0000 7f94a37fe700 1 -- 172.21.15.71:0/2174633849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94a40ffc20 msgr2=0x7f94a4101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:35.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.556+0000 7f94a37fe700 1 --2- 172.21.15.71:0/2174633849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94a40ffc20 0x7f94a4101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:35.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.556+0000 7f94a37fe700 1 -- 172.21.15.71:0/2174633849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94980095f0 con 0x7f94a4105bd0 2024-04-23T02:34:35.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.556+0000 7f94a37fe700 1 --2- 172.21.15.71:0/2174633849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a4105bd0 0x7f94a4101d80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f949400d8d0 tx=0x7f949400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:35.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.556+0000 7f94a17fa700 1 -- 172.21.15.71:0/2174633849 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94940098a0 con 0x7f94a4105bd0 2024-04-23T02:34:35.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.556+0000 7f94a17fa700 1 -- 172.21.15.71:0/2174633849 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9494010460 con 0x7f94a4105bd0 2024-04-23T02:34:35.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.556+0000 7f94aa875700 1 -- 172.21.15.71:0/2174633849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a41a2980 con 0x7f94a4105bd0 2024-04-23T02:34:35.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.556+0000 7f94a17fa700 1 -- 172.21.15.71:0/2174633849 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9494009ea0 con 0x7f94a4105bd0 2024-04-23T02:34:35.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.556+0000 7f94aa875700 1 -- 172.21.15.71:0/2174633849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a41a2d90 con 0x7f94a4105bd0 2024-04-23T02:34:35.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.557+0000 7f94aa875700 1 -- 172.21.15.71:0/2174633849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a404f2e0 con 0x7f94a4105bd0 2024-04-23T02:34:35.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.558+0000 7f94a17fa700 1 -- 172.21.15.71:0/2174633849 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f94940105d0 con 0x7f94a4105bd0 2024-04-23T02:34:35.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.558+0000 7f94a17fa700 1 --2- 172.21.15.71:0/2174633849 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f948c06cce0 0x7f948c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:35.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.558+0000 7f94a17fa700 1 -- 172.21.15.71:0/2174633849 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f949408c3a0 con 0x7f94a4105bd0 2024-04-23T02:34:35.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.558+0000 7f94a3fff700 1 --2- 172.21.15.71:0/2174633849 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f948c06cce0 0x7f948c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:35.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.558+0000 7f94a3fff700 1 --2- 172.21.15.71:0/2174633849 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f948c06cce0 0x7f948c06f1a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9498006010 tx=0x7f949801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:35.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.563+0000 7f94a17fa700 1 -- 172.21.15.71:0/2174633849 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9494056490 con 0x7f94a4105bd0 2024-04-23T02:34:35.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:34:35.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.774+0000 7f94aa875700 1 -- 172.21.15.71:0/2174633849 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f94a4068d90 con 0x7f948c06cce0 2024-04-23T02:34:35.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.776+0000 7f94a17fa700 1 -- 172.21.15.71:0/2174633849 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f94a4068d90 con 0x7f948c06cce0 2024-04-23T02:34:35.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.778+0000 7f94aa875700 1 -- 172.21.15.71:0/2174633849 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f948c06cce0 msgr2=0x7f948c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:35.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.778+0000 7f94aa875700 1 --2- 172.21.15.71:0/2174633849 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f948c06cce0 0x7f948c06f1a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9498006010 tx=0x7f949801a040 comp rx=0 tx=0).stop 2024-04-23T02:34:35.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.778+0000 7f94aa875700 1 -- 172.21.15.71:0/2174633849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a4105bd0 msgr2=0x7f94a4101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:35.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.778+0000 7f94aa875700 1 --2- 172.21.15.71:0/2174633849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a4105bd0 0x7f94a4101d80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f949400d8d0 tx=0x7f949400dbe0 comp rx=0 tx=0).stop 2024-04-23T02:34:35.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.778+0000 7f94aa875700 1 -- 172.21.15.71:0/2174633849 shutdown_connections 2024-04-23T02:34:35.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.778+0000 7f94aa875700 1 --2- 172.21.15.71:0/2174633849 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f948c06cce0 0x7f948c06f1a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:35.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.778+0000 7f94aa875700 1 --2- 172.21.15.71:0/2174633849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94a40ffc20 0x7f94a4101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:35.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.778+0000 7f94aa875700 1 --2- 172.21.15.71:0/2174633849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94a4105bd0 0x7f94a4101d80 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:35.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.778+0000 7f94aa875700 1 -- 172.21.15.71:0/2174633849 >> 172.21.15.71:0/2174633849 conn(0x7f94a4075670 msgr2=0x7f94a4068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:35.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.778+0000 7f94aa875700 1 -- 172.21.15.71:0/2174633849 shutdown_connections 2024-04-23T02:34:35.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.778+0000 7f94aa875700 1 -- 172.21.15.71:0/2174633849 wait complete. 2024-04-23T02:34:35.790 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:34:35.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.881+0000 7f9df6fb5700 1 -- 172.21.15.71:0/564519453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9df0100ea0 msgr2=0x7f9df0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:35.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.881+0000 7f9df6fb5700 1 --2- 172.21.15.71:0/564519453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9df0100ea0 0x7f9df0101280 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009910 tx=0x7f9de0009c20 comp rx=0 tx=0).stop 2024-04-23T02:34:35.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.881+0000 7f9df6fb5700 1 -- 172.21.15.71:0/564519453 shutdown_connections 2024-04-23T02:34:35.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.881+0000 7f9df6fb5700 1 --2- 172.21.15.71:0/564519453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9df0101850 0x7f9df01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:35.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.881+0000 7f9df6fb5700 1 --2- 172.21.15.71:0/564519453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9df0100ea0 0x7f9df0101280 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:35.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.881+0000 7f9df6fb5700 1 -- 172.21.15.71:0/564519453 >> 172.21.15.71:0/564519453 conn(0x7f9df00fc750 msgr2=0x7f9df00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:35.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.881+0000 7f9df6fb5700 1 -- 172.21.15.71:0/564519453 shutdown_connections 2024-04-23T02:34:35.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.881+0000 7f9df6fb5700 1 -- 172.21.15.71:0/564519453 wait complete. 2024-04-23T02:34:35.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.882+0000 7f9df6fb5700 1 Processor -- start 2024-04-23T02:34:35.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.882+0000 7f9df6fb5700 1 -- start start 2024-04-23T02:34:35.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.882+0000 7f9df6fb5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9df0100ea0 0x7f9df0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:35.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.882+0000 7f9df6fb5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9df0101850 0x7f9df00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:35.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.882+0000 7f9df6fb5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df0073b10 con 0x7f9df0100ea0 2024-04-23T02:34:35.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.882+0000 7f9df6fb5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df0073c50 con 0x7f9df0101850 2024-04-23T02:34:35.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.882+0000 7f9df4d51700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9df0100ea0 0x7f9df0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:35.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.882+0000 7f9df4d51700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9df0100ea0 0x7f9df0073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60996/0 (socket says 172.21.15.71:60996) 2024-04-23T02:34:35.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.882+0000 7f9df4d51700 1 -- 172.21.15.71:0/888679411 learned_addr learned my addr 172.21.15.71:0/888679411 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:34:35.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.883+0000 7f9df4d51700 1 -- 172.21.15.71:0/888679411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9df0101850 msgr2=0x7f9df00735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:35.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.883+0000 7f9df4d51700 1 --2- 172.21.15.71:0/888679411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9df0101850 0x7f9df00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:35.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.883+0000 7f9df4d51700 1 -- 172.21.15.71:0/888679411 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de00095f0 con 0x7f9df0100ea0 2024-04-23T02:34:35.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.883+0000 7f9df4d51700 1 --2- 172.21.15.71:0/888679411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9df0100ea0 0x7f9df0073090 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9de0006010 tx=0x7f9de000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:35.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.883+0000 7f9dedffb700 1 -- 172.21.15.71:0/888679411 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9de001d070 con 0x7f9df0100ea0 2024-04-23T02:34:35.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.883+0000 7f9df6fb5700 1 -- 172.21.15.71:0/888679411 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df0073ed0 con 0x7f9df0100ea0 2024-04-23T02:34:35.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.883+0000 7f9dedffb700 1 -- 172.21.15.71:0/888679411 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9de000f460 con 0x7f9df0100ea0 2024-04-23T02:34:35.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.883+0000 7f9dedffb700 1 -- 172.21.15.71:0/888679411 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9de0021620 con 0x7f9df0100ea0 2024-04-23T02:34:35.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.883+0000 7f9df6fb5700 1 -- 172.21.15.71:0/888679411 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df01a2c90 con 0x7f9df0100ea0 2024-04-23T02:34:35.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.884+0000 7f9df6fb5700 1 -- 172.21.15.71:0/888679411 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df0109240 con 0x7f9df0100ea0 2024-04-23T02:34:35.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.884+0000 7f9dedffb700 1 -- 172.21.15.71:0/888679411 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9de000fa40 con 0x7f9df0100ea0 2024-04-23T02:34:35.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.885+0000 7f9dedffb700 1 --2- 172.21.15.71:0/888679411 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9dd806cce0 0x7f9dd806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:35.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.885+0000 7f9dedffb700 1 -- 172.21.15.71:0/888679411 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9de008d790 con 0x7f9df0100ea0 2024-04-23T02:34:35.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.885+0000 7f9deffff700 1 --2- 172.21.15.71:0/888679411 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9dd806cce0 0x7f9dd806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:35.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.885+0000 7f9deffff700 1 --2- 172.21.15.71:0/888679411 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9dd806cce0 0x7f9dd806f1a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9de4005950 tx=0x7f9de40058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:35.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:35.890+0000 7f9dedffb700 1 -- 172.21.15.71:0/888679411 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9de0057800 con 0x7f9df0100ea0 2024-04-23T02:34:36.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.102+0000 7f9df6fb5700 1 -- 172.21.15.71:0/888679411 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9df00611d0 con 0x7f9dd806cce0 2024-04-23T02:34:36.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.103+0000 7f9dedffb700 1 -- 172.21.15.71:0/888679411 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9df00611d0 con 0x7f9dd806cce0 2024-04-23T02:34:36.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.105+0000 7f9df6fb5700 1 -- 172.21.15.71:0/888679411 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9dd806cce0 msgr2=0x7f9dd806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:36.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.105+0000 7f9df6fb5700 1 --2- 172.21.15.71:0/888679411 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9dd806cce0 0x7f9dd806f1a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9de4005950 tx=0x7f9de40058e0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.105+0000 7f9df6fb5700 1 -- 172.21.15.71:0/888679411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9df0100ea0 msgr2=0x7f9df0073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:36.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.105+0000 7f9df6fb5700 1 --2- 172.21.15.71:0/888679411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9df0100ea0 0x7f9df0073090 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9de0006010 tx=0x7f9de000be20 comp rx=0 tx=0).stop 2024-04-23T02:34:36.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.105+0000 7f9df6fb5700 1 -- 172.21.15.71:0/888679411 shutdown_connections 2024-04-23T02:34:36.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.105+0000 7f9df6fb5700 1 --2- 172.21.15.71:0/888679411 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9dd806cce0 0x7f9dd806f1a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.105+0000 7f9df6fb5700 1 --2- 172.21.15.71:0/888679411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9df0100ea0 0x7f9df0073090 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.105+0000 7f9df6fb5700 1 --2- 172.21.15.71:0/888679411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9df0101850 0x7f9df00735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.105+0000 7f9df6fb5700 1 -- 172.21.15.71:0/888679411 >> 172.21.15.71:0/888679411 conn(0x7f9df00fc750 msgr2=0x7f9df00fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:36.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.105+0000 7f9df6fb5700 1 -- 172.21.15.71:0/888679411 shutdown_connections 2024-04-23T02:34:36.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.105+0000 7f9df6fb5700 1 -- 172.21.15.71:0/888679411 wait complete. 2024-04-23T02:34:36.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.207+0000 7f118bd24700 1 -- 172.21.15.71:0/1271344763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1184068580 msgr2=0x7f11840689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:36.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.207+0000 7f118bd24700 1 --2- 172.21.15.71:0/1271344763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1184068580 0x7f11840689e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1180009910 tx=0x7f1180009c20 comp rx=0 tx=0).stop 2024-04-23T02:34:36.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.207+0000 7f118bd24700 1 -- 172.21.15.71:0/1271344763 shutdown_connections 2024-04-23T02:34:36.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.207+0000 7f118bd24700 1 --2- 172.21.15.71:0/1271344763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1184068580 0x7f11840689e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.207+0000 7f118bd24700 1 --2- 172.21.15.71:0/1271344763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1184106cd0 0x7f11841070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.207+0000 7f118bd24700 1 -- 172.21.15.71:0/1271344763 >> 172.21.15.71:0/1271344763 conn(0x7f1184075670 msgr2=0x7f1184075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:36.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.207+0000 7f118bd24700 1 -- 172.21.15.71:0/1271344763 shutdown_connections 2024-04-23T02:34:36.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.207+0000 7f118bd24700 1 -- 172.21.15.71:0/1271344763 wait complete. 2024-04-23T02:34:36.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.208+0000 7f118bd24700 1 Processor -- start 2024-04-23T02:34:36.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.208+0000 7f118bd24700 1 -- start start 2024-04-23T02:34:36.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.208+0000 7f118bd24700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1184068580 0x7f118419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:36.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.208+0000 7f118bd24700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1184106cd0 0x7f118419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:36.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.208+0000 7f118bd24700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118419d110 con 0x7f1184068580 2024-04-23T02:34:36.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.208+0000 7f118bd24700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11841965c0 con 0x7f1184106cd0 2024-04-23T02:34:36.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.208+0000 7f1189ac0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1184068580 0x7f118419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:36.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.208+0000 7f1189ac0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1184068580 0x7f118419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32780/0 (socket says 172.21.15.71:32780) 2024-04-23T02:34:36.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.208+0000 7f1189ac0700 1 -- 172.21.15.71:0/2004683127 learned_addr learned my addr 172.21.15.71:0/2004683127 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:34:36.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.208+0000 7f11892bf700 1 --2- 172.21.15.71:0/2004683127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1184106cd0 0x7f118419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:36.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.208+0000 7f1189ac0700 1 -- 172.21.15.71:0/2004683127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1184106cd0 msgr2=0x7f118419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:36.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.208+0000 7f1189ac0700 1 --2- 172.21.15.71:0/2004683127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1184106cd0 0x7f118419ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.208+0000 7f1189ac0700 1 -- 172.21.15.71:0/2004683127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11800095f0 con 0x7f1184068580 2024-04-23T02:34:36.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.208+0000 7f11892bf700 1 --2- 172.21.15.71:0/2004683127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1184106cd0 0x7f118419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:34:36.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.209+0000 7f1189ac0700 1 --2- 172.21.15.71:0/2004683127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1184068580 0x7f118419c540 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f117400c8a0 tx=0x7f117400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:36.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.209+0000 7f117affd700 1 -- 172.21.15.71:0/2004683127 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f117400cea0 con 0x7f1184068580 2024-04-23T02:34:36.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.209+0000 7f117affd700 1 -- 172.21.15.71:0/2004683127 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1174004830 con 0x7f1184068580 2024-04-23T02:34:36.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.209+0000 7f117affd700 1 -- 172.21.15.71:0/2004683127 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1174005500 con 0x7f1184068580 2024-04-23T02:34:36.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.209+0000 7f118bd24700 1 -- 172.21.15.71:0/2004683127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11841968a0 con 0x7f1184068580 2024-04-23T02:34:36.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.209+0000 7f118bd24700 1 -- 172.21.15.71:0/2004683127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1184196df0 con 0x7f1184068580 2024-04-23T02:34:36.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.210+0000 7f118bd24700 1 -- 172.21.15.71:0/2004683127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11841091d0 con 0x7f1184068580 2024-04-23T02:34:36.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.210+0000 7f117affd700 1 -- 172.21.15.71:0/2004683127 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f11740049a0 con 0x7f1184068580 2024-04-23T02:34:36.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.210+0000 7f117affd700 1 --2- 172.21.15.71:0/2004683127 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f117006cce0 0x7f117006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:36.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.211+0000 7f117affd700 1 -- 172.21.15.71:0/2004683127 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f117408c1d0 con 0x7f1184068580 2024-04-23T02:34:36.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.211+0000 7f11892bf700 1 --2- 172.21.15.71:0/2004683127 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f117006cce0 0x7f117006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:36.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.211+0000 7f11892bf700 1 --2- 172.21.15.71:0/2004683127 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f117006cce0 0x7f117006f1a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f11840fe260 tx=0x7f118001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:36.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.215+0000 7f117affd700 1 -- 172.21.15.71:0/2004683127 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1174059d50 con 0x7f1184068580 2024-04-23T02:34:36.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.417+0000 7f118bd24700 1 -- 172.21.15.71:0/2004683127 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f11841970c0 con 0x7f117006cce0 2024-04-23T02:34:36.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.428+0000 7f117affd700 1 -- 172.21.15.71:0/2004683127 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f11841970c0 con 0x7f117006cce0 2024-04-23T02:34:36.430 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:34:36.431 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (18m) 12m ago 29m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:34:36.431 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (29m) 12m ago 29m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:34:36.431 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (28m) 11m ago 28m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:34:36.431 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (16m) 12m ago 29m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:34:36.431 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (16m) 11m ago 28m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:34:36.431 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (18m) 12m ago 29m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:34:36.431 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (12m) 12m ago 23m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:34:36.431 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (12m) 12m ago 23m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:34:36.431 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (23m) 11m ago 23m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:34:36.431 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (11m) 11m ago 23m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:34:36.431 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (20m) 12m ago 31m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:34:36.431 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (19m) 11m ago 28m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:34:36.432 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (16m) 12m ago 31m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:34:36.432 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (16m) 11m ago 28m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:34:36.432 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (19m) 12m ago 29m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:34:36.432 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (19m) 11m ago 28m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:34:36.432 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (15m) 12m ago 27m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:34:36.432 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (15m) 12m ago 27m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:34:36.432 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (14m) 12m ago 26m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:34:36.432 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (13m) 11m ago 26m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:34:36.432 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (13m) 11m ago 25m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:34:36.432 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (12m) 11m ago 25m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:34:36.432 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (18m) 12m ago 29m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:34:36.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.430+0000 7f118bd24700 1 -- 172.21.15.71:0/2004683127 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f117006cce0 msgr2=0x7f117006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:36.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.430+0000 7f118bd24700 1 --2- 172.21.15.71:0/2004683127 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f117006cce0 0x7f117006f1a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f11840fe260 tx=0x7f118001a040 comp rx=0 tx=0).stop 2024-04-23T02:34:36.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.430+0000 7f118bd24700 1 -- 172.21.15.71:0/2004683127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1184068580 msgr2=0x7f118419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:36.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.430+0000 7f118bd24700 1 --2- 172.21.15.71:0/2004683127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1184068580 0x7f118419c540 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f117400c8a0 tx=0x7f117400cc60 comp rx=0 tx=0).stop 2024-04-23T02:34:36.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.430+0000 7f118bd24700 1 -- 172.21.15.71:0/2004683127 shutdown_connections 2024-04-23T02:34:36.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.430+0000 7f118bd24700 1 --2- 172.21.15.71:0/2004683127 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f117006cce0 0x7f117006f1a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.430+0000 7f118bd24700 1 --2- 172.21.15.71:0/2004683127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1184068580 0x7f118419c540 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.430+0000 7f118bd24700 1 --2- 172.21.15.71:0/2004683127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1184106cd0 0x7f118419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.430+0000 7f118bd24700 1 -- 172.21.15.71:0/2004683127 >> 172.21.15.71:0/2004683127 conn(0x7f1184075670 msgr2=0x7f11840fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:36.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.430+0000 7f118bd24700 1 -- 172.21.15.71:0/2004683127 shutdown_connections 2024-04-23T02:34:36.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.430+0000 7f118bd24700 1 -- 172.21.15.71:0/2004683127 wait complete. 2024-04-23T02:34:36.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:36 smithi071 ceph-mon[144544]: pgmap v638: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:36.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.531+0000 7f8603b04700 1 -- 172.21.15.71:0/3638783277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85fc0ffc20 msgr2=0x7f85fc100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:36.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.531+0000 7f8603b04700 1 --2- 172.21.15.71:0/3638783277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85fc0ffc20 0x7f85fc100080 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009910 tx=0x7f85f8009c20 comp rx=0 tx=0).stop 2024-04-23T02:34:36.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.531+0000 7f8603b04700 1 -- 172.21.15.71:0/3638783277 shutdown_connections 2024-04-23T02:34:36.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.531+0000 7f8603b04700 1 --2- 172.21.15.71:0/3638783277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85fc0ffc20 0x7f85fc100080 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.531+0000 7f8603b04700 1 --2- 172.21.15.71:0/3638783277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85fc105bd0 0x7f85fc105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.531+0000 7f8603b04700 1 -- 172.21.15.71:0/3638783277 >> 172.21.15.71:0/3638783277 conn(0x7f85fc075670 msgr2=0x7f85fc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:36.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.531+0000 7f8603b04700 1 -- 172.21.15.71:0/3638783277 shutdown_connections 2024-04-23T02:34:36.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.531+0000 7f8603b04700 1 -- 172.21.15.71:0/3638783277 wait complete. 2024-04-23T02:34:36.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.532+0000 7f8603b04700 1 Processor -- start 2024-04-23T02:34:36.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.532+0000 7f8603b04700 1 -- start start 2024-04-23T02:34:36.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.532+0000 7f8603b04700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85fc0ffc20 0x7f85fc19b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:36.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.532+0000 7f8603b04700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85fc105bd0 0x7f85fc19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:36.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.532+0000 7f8603b04700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc19c0f0 con 0x7f85fc0ffc20 2024-04-23T02:34:36.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.532+0000 7f8603b04700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc1955a0 con 0x7f85fc105bd0 2024-04-23T02:34:36.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.532+0000 7f86018a0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85fc0ffc20 0x7f85fc19b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:36.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.532+0000 7f86018a0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85fc0ffc20 0x7f85fc19b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32808/0 (socket says 172.21.15.71:32808) 2024-04-23T02:34:36.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.532+0000 7f86018a0700 1 -- 172.21.15.71:0/2847131005 learned_addr learned my addr 172.21.15.71:0/2847131005 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:34:36.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.532+0000 7f860109f700 1 --2- 172.21.15.71:0/2847131005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85fc105bd0 0x7f85fc19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:36.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.532+0000 7f86018a0700 1 -- 172.21.15.71:0/2847131005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85fc105bd0 msgr2=0x7f85fc19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:36.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.532+0000 7f86018a0700 1 --2- 172.21.15.71:0/2847131005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85fc105bd0 0x7f85fc19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.532+0000 7f86018a0700 1 -- 172.21.15.71:0/2847131005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f80095f0 con 0x7f85fc0ffc20 2024-04-23T02:34:36.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.532+0000 7f86018a0700 1 --2- 172.21.15.71:0/2847131005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85fc0ffc20 0x7f85fc19b520 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f85f000d8d0 tx=0x7f85f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:36.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.532+0000 7f85eeffd700 1 -- 172.21.15.71:0/2847131005 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85f0009940 con 0x7f85fc0ffc20 2024-04-23T02:34:36.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.532+0000 7f85eeffd700 1 -- 172.21.15.71:0/2847131005 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f85f0010460 con 0x7f85fc0ffc20 2024-04-23T02:34:36.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.533+0000 7f85eeffd700 1 -- 172.21.15.71:0/2847131005 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85f000f5f0 con 0x7f85fc0ffc20 2024-04-23T02:34:36.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.533+0000 7f8603b04700 1 -- 172.21.15.71:0/2847131005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85fc195880 con 0x7f85fc0ffc20 2024-04-23T02:34:36.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.533+0000 7f8603b04700 1 -- 172.21.15.71:0/2847131005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85fc195dd0 con 0x7f85fc0ffc20 2024-04-23T02:34:36.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.533+0000 7f8603b04700 1 -- 172.21.15.71:0/2847131005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85fc1080d0 con 0x7f85fc0ffc20 2024-04-23T02:34:36.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.534+0000 7f85eeffd700 1 -- 172.21.15.71:0/2847131005 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f85f00105d0 con 0x7f85fc0ffc20 2024-04-23T02:34:36.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.535+0000 7f85eeffd700 1 --2- 172.21.15.71:0/2847131005 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f85e806cce0 0x7f85e806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:36.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.535+0000 7f85eeffd700 1 -- 172.21.15.71:0/2847131005 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f85f008c560 con 0x7f85fc0ffc20 2024-04-23T02:34:36.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.535+0000 7f860109f700 1 --2- 172.21.15.71:0/2847131005 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f85e806cce0 0x7f85e806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:36.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.535+0000 7f860109f700 1 --2- 172.21.15.71:0/2847131005 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f85e806cce0 0x7f85e806f1a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0fe2d0 tx=0x7f85f801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:36.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.539+0000 7f85eeffd700 1 -- 172.21.15.71:0/2847131005 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f85f005a170 con 0x7f85fc0ffc20 2024-04-23T02:34:36.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:36 smithi192 ceph-mon[108617]: pgmap v638: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:36.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.815+0000 7f8603b04700 1 -- 172.21.15.71:0/2847131005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f85fc04f2e0 con 0x7f85fc0ffc20 2024-04-23T02:34:36.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.815+0000 7f85eeffd700 1 -- 172.21.15.71:0/2847131005 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f85f0010880 con 0x7f85fc0ffc20 2024-04-23T02:34:36.818 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:34:36.818 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:34:36.818 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:34:36.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:34:36.818 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:34:36.818 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:34:36.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:34:36.818 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:34:36.818 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:34:36.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:34:36.818 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:34:36.819 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:34:36.819 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:34:36.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:34:36.819 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:34:36.819 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:34:36.819 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:34:36.819 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:34:36.819 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:34:36.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.817+0000 7f8603b04700 1 -- 172.21.15.71:0/2847131005 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f85e806cce0 msgr2=0x7f85e806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:36.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.817+0000 7f8603b04700 1 --2- 172.21.15.71:0/2847131005 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f85e806cce0 0x7f85e806f1a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0fe2d0 tx=0x7f85f801a040 comp rx=0 tx=0).stop 2024-04-23T02:34:36.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.817+0000 7f8603b04700 1 -- 172.21.15.71:0/2847131005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85fc0ffc20 msgr2=0x7f85fc19b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:36.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.817+0000 7f8603b04700 1 --2- 172.21.15.71:0/2847131005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85fc0ffc20 0x7f85fc19b520 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f85f000d8d0 tx=0x7f85f000dc90 comp rx=0 tx=0).stop 2024-04-23T02:34:36.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.817+0000 7f8603b04700 1 -- 172.21.15.71:0/2847131005 shutdown_connections 2024-04-23T02:34:36.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.817+0000 7f8603b04700 1 --2- 172.21.15.71:0/2847131005 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f85e806cce0 0x7f85e806f1a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.817+0000 7f8603b04700 1 --2- 172.21.15.71:0/2847131005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85fc0ffc20 0x7f85fc19b520 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.817+0000 7f8603b04700 1 --2- 172.21.15.71:0/2847131005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85fc105bd0 0x7f85fc19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.817+0000 7f8603b04700 1 -- 172.21.15.71:0/2847131005 >> 172.21.15.71:0/2847131005 conn(0x7f85fc075670 msgr2=0x7f85fc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:36.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.817+0000 7f8603b04700 1 -- 172.21.15.71:0/2847131005 shutdown_connections 2024-04-23T02:34:36.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.817+0000 7f8603b04700 1 -- 172.21.15.71:0/2847131005 wait complete. 2024-04-23T02:34:36.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.922+0000 7fce4aa5d700 1 -- 172.21.15.71:0/1530776007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44101850 msgr2=0x7fce441058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:36.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.922+0000 7fce4aa5d700 1 --2- 172.21.15.71:0/1530776007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44101850 0x7fce441058a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fce38009910 tx=0x7fce38009c20 comp rx=0 tx=0).stop 2024-04-23T02:34:36.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.922+0000 7fce4aa5d700 1 -- 172.21.15.71:0/1530776007 shutdown_connections 2024-04-23T02:34:36.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.922+0000 7fce4aa5d700 1 --2- 172.21.15.71:0/1530776007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44101850 0x7fce441058a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.922+0000 7fce4aa5d700 1 --2- 172.21.15.71:0/1530776007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce44100ea0 0x7fce44101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.922+0000 7fce4aa5d700 1 -- 172.21.15.71:0/1530776007 >> 172.21.15.71:0/1530776007 conn(0x7fce440fc750 msgr2=0x7fce440feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:36.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.922+0000 7fce4aa5d700 1 -- 172.21.15.71:0/1530776007 shutdown_connections 2024-04-23T02:34:36.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.922+0000 7fce4aa5d700 1 -- 172.21.15.71:0/1530776007 wait complete. 2024-04-23T02:34:36.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.923+0000 7fce4aa5d700 1 Processor -- start 2024-04-23T02:34:36.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.923+0000 7fce4aa5d700 1 -- start start 2024-04-23T02:34:36.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.923+0000 7fce4aa5d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44100ea0 0x7fce4419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:36.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.923+0000 7fce4aa5d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce44101850 0x7fce4419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:36.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.923+0000 7fce4aa5d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4419d160 con 0x7fce44100ea0 2024-04-23T02:34:36.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.923+0000 7fce4aa5d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce441966a0 con 0x7fce44101850 2024-04-23T02:34:36.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.923+0000 7fce43fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44100ea0 0x7fce4419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:36.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.924+0000 7fce43fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44100ea0 0x7fce4419c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32828/0 (socket says 172.21.15.71:32828) 2024-04-23T02:34:36.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.924+0000 7fce43fff700 1 -- 172.21.15.71:0/2528046831 learned_addr learned my addr 172.21.15.71:0/2528046831 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:34:36.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.924+0000 7fce437fe700 1 --2- 172.21.15.71:0/2528046831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce44101850 0x7fce4419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:36.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.924+0000 7fce43fff700 1 -- 172.21.15.71:0/2528046831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce44101850 msgr2=0x7fce4419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:36.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.924+0000 7fce43fff700 1 --2- 172.21.15.71:0/2528046831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce44101850 0x7fce4419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:36.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.924+0000 7fce43fff700 1 -- 172.21.15.71:0/2528046831 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce380095f0 con 0x7fce44100ea0 2024-04-23T02:34:36.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.924+0000 7fce43fff700 1 --2- 172.21.15.71:0/2528046831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44100ea0 0x7fce4419c620 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fce3400c930 tx=0x7fce3400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:36.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.924+0000 7fce417fa700 1 -- 172.21.15.71:0/2528046831 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce34007ad0 con 0x7fce44100ea0 2024-04-23T02:34:36.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.924+0000 7fce417fa700 1 -- 172.21.15.71:0/2528046831 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fce3400ce80 con 0x7fce44100ea0 2024-04-23T02:34:36.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.924+0000 7fce4aa5d700 1 -- 172.21.15.71:0/2528046831 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce44196980 con 0x7fce44100ea0 2024-04-23T02:34:36.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.924+0000 7fce417fa700 1 -- 172.21.15.71:0/2528046831 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce34018630 con 0x7fce44100ea0 2024-04-23T02:34:36.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.924+0000 7fce4aa5d700 1 -- 172.21.15.71:0/2528046831 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce44196ed0 con 0x7fce44100ea0 2024-04-23T02:34:36.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.925+0000 7fce4aa5d700 1 -- 172.21.15.71:0/2528046831 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce441092a0 con 0x7fce44100ea0 2024-04-23T02:34:36.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.926+0000 7fce417fa700 1 -- 172.21.15.71:0/2528046831 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fce34007c30 con 0x7fce44100ea0 2024-04-23T02:34:36.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.926+0000 7fce417fa700 1 --2- 172.21.15.71:0/2528046831 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fce2c06cce0 0x7fce2c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:36.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.926+0000 7fce417fa700 1 -- 172.21.15.71:0/2528046831 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fce3408c5a0 con 0x7fce44100ea0 2024-04-23T02:34:36.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.926+0000 7fce437fe700 1 --2- 172.21.15.71:0/2528046831 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fce2c06cce0 0x7fce2c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:36.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.926+0000 7fce437fe700 1 --2- 172.21.15.71:0/2528046831 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fce2c06cce0 0x7fce2c06f1a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fce440fcfc0 tx=0x7fce3801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:36.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:36.930+0000 7fce417fa700 1 -- 172.21.15.71:0/2528046831 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fce3405a0a0 con 0x7fce44100ea0 2024-04-23T02:34:37.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.162+0000 7fce4aa5d700 1 -- 172.21.15.71:0/2528046831 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fce44197ca0 con 0x7fce44100ea0 2024-04-23T02:34:37.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.163+0000 7fce417fa700 1 -- 172.21.15.71:0/2528046831 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7fce34059c30 con 0x7fce44100ea0 2024-04-23T02:34:37.165 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:34:37.166 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:34:37.166 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:34:37.166 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:34:37.166 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:34:37.166 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:34:37.166 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:34:37.166 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:34:37.166 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:34:37.166 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:34:37.166 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:34:37.166 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:34:37.166 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:34:37.166 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:34:37.167 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:34:37.167 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:34:37.167 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:34:37.167 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:34:37.167 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:34:37.167 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:34:37.167 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:34:37.167 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:34:37.167 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:34:37.167 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:34:37.167 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:34:37.167 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:34:37.167 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:34:37.168 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:34:37.168 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:34:37.168 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:34:37.168 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:34:37.168 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:34:37.168 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:34:37.168 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:34:37.168 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:34:37.168 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:34:37.168 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:34:37.168 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:34:37.168 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:34:37.168 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:34:37.169 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:34:37.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.165+0000 7fce4aa5d700 1 -- 172.21.15.71:0/2528046831 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fce2c06cce0 msgr2=0x7fce2c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:37.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.165+0000 7fce4aa5d700 1 --2- 172.21.15.71:0/2528046831 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fce2c06cce0 0x7fce2c06f1a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fce440fcfc0 tx=0x7fce3801a040 comp rx=0 tx=0).stop 2024-04-23T02:34:37.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.165+0000 7fce4aa5d700 1 -- 172.21.15.71:0/2528046831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44100ea0 msgr2=0x7fce4419c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:37.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.165+0000 7fce4aa5d700 1 --2- 172.21.15.71:0/2528046831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44100ea0 0x7fce4419c620 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fce3400c930 tx=0x7fce3400cc40 comp rx=0 tx=0).stop 2024-04-23T02:34:37.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.165+0000 7fce4aa5d700 1 -- 172.21.15.71:0/2528046831 shutdown_connections 2024-04-23T02:34:37.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.165+0000 7fce4aa5d700 1 --2- 172.21.15.71:0/2528046831 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fce2c06cce0 0x7fce2c06f1a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:37.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.165+0000 7fce4aa5d700 1 --2- 172.21.15.71:0/2528046831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce44100ea0 0x7fce4419c620 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:37.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.165+0000 7fce4aa5d700 1 --2- 172.21.15.71:0/2528046831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce44101850 0x7fce4419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:37.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.165+0000 7fce4aa5d700 1 -- 172.21.15.71:0/2528046831 >> 172.21.15.71:0/2528046831 conn(0x7fce440fc750 msgr2=0x7fce440fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:37.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.165+0000 7fce4aa5d700 1 -- 172.21.15.71:0/2528046831 shutdown_connections 2024-04-23T02:34:37.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.165+0000 7fce4aa5d700 1 -- 172.21.15.71:0/2528046831 wait complete. 2024-04-23T02:34:37.170 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:34:37.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.270+0000 7f2bb20c9700 1 -- 172.21.15.71:0/253364774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bac102700 msgr2=0x7f2bac10abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:37.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.270+0000 7f2bb20c9700 1 --2- 172.21.15.71:0/253364774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bac102700 0x7f2bac10abf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c009910 tx=0x7f2b9c009c20 comp rx=0 tx=0).stop 2024-04-23T02:34:37.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.270+0000 7f2bb20c9700 1 -- 172.21.15.71:0/253364774 shutdown_connections 2024-04-23T02:34:37.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.270+0000 7f2bb20c9700 1 --2- 172.21.15.71:0/253364774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bac102700 0x7f2bac10abf0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:37.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.270+0000 7f2bb20c9700 1 --2- 172.21.15.71:0/253364774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2bac101de0 0x7f2bac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:37.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.270+0000 7f2bb20c9700 1 -- 172.21.15.71:0/253364774 >> 172.21.15.71:0/253364774 conn(0x7f2bac0fb610 msgr2=0x7f2bac0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:37.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.270+0000 7f2bb20c9700 1 -- 172.21.15.71:0/253364774 shutdown_connections 2024-04-23T02:34:37.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.270+0000 7f2bb20c9700 1 -- 172.21.15.71:0/253364774 wait complete. 2024-04-23T02:34:37.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.270+0000 7f2bb20c9700 1 Processor -- start 2024-04-23T02:34:37.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.270+0000 7f2bb20c9700 1 -- start start 2024-04-23T02:34:37.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.271+0000 7f2bb20c9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bac101de0 0x7f2bac0ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:37.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.271+0000 7f2bb20c9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2bac102700 0x7f2bac0fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:37.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.271+0000 7f2bb20c9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac1004c0 con 0x7f2bac101de0 2024-04-23T02:34:37.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.271+0000 7f2bb20c9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac100600 con 0x7f2bac102700 2024-04-23T02:34:37.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.271+0000 7f2bab7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bac101de0 0x7f2bac0ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:37.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.271+0000 7f2bab7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bac101de0 0x7f2bac0ffa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32846/0 (socket says 172.21.15.71:32846) 2024-04-23T02:34:37.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.271+0000 7f2bab7fe700 1 -- 172.21.15.71:0/3937745948 learned_addr learned my addr 172.21.15.71:0/3937745948 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:34:37.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.271+0000 7f2baaffd700 1 --2- 172.21.15.71:0/3937745948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2bac102700 0x7f2bac0fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:37.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.271+0000 7f2bab7fe700 1 -- 172.21.15.71:0/3937745948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2bac102700 msgr2=0x7f2bac0fff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:37.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.271+0000 7f2bab7fe700 1 --2- 172.21.15.71:0/3937745948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2bac102700 0x7f2bac0fff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:37.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.271+0000 7f2bab7fe700 1 -- 172.21.15.71:0/3937745948 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9c0095f0 con 0x7f2bac101de0 2024-04-23T02:34:37.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.271+0000 7f2baaffd700 1 --2- 172.21.15.71:0/3937745948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2bac102700 0x7f2bac0fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:34:37.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.271+0000 7f2bab7fe700 1 --2- 172.21.15.71:0/3937745948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bac101de0 0x7f2bac0ffa40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000c8a0 tx=0x7f2ba000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:37.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.271+0000 7f2ba8ff9700 1 -- 172.21.15.71:0/3937745948 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ba000cea0 con 0x7f2bac101de0 2024-04-23T02:34:37.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.271+0000 7f2ba8ff9700 1 -- 172.21.15.71:0/3937745948 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2ba0004830 con 0x7f2bac101de0 2024-04-23T02:34:37.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.271+0000 7f2bb20c9700 1 -- 172.21.15.71:0/3937745948 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bac1a29c0 con 0x7f2bac101de0 2024-04-23T02:34:37.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.271+0000 7f2ba8ff9700 1 -- 172.21.15.71:0/3937745948 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ba0005500 con 0x7f2bac101de0 2024-04-23T02:34:37.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.272+0000 7f2bb20c9700 1 -- 172.21.15.71:0/3937745948 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bac1a2dd0 con 0x7f2bac101de0 2024-04-23T02:34:37.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.273+0000 7f2bb20c9700 1 -- 172.21.15.71:0/3937745948 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bac1082f0 con 0x7f2bac101de0 2024-04-23T02:34:37.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.273+0000 7f2ba8ff9700 1 -- 172.21.15.71:0/3937745948 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2ba00049a0 con 0x7f2bac101de0 2024-04-23T02:34:37.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.273+0000 7f2ba8ff9700 1 --2- 172.21.15.71:0/3937745948 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2b9406cce0 0x7f2b9406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:37.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.274+0000 7f2ba8ff9700 1 -- 172.21.15.71:0/3937745948 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2ba008c2b0 con 0x7f2bac101de0 2024-04-23T02:34:37.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.274+0000 7f2baaffd700 1 --2- 172.21.15.71:0/3937745948 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2b9406cce0 0x7f2b9406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:37.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.274+0000 7f2baaffd700 1 --2- 172.21.15.71:0/3937745948 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2b9406cce0 0x7f2b9406f1a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2bac101230 tx=0x7f2b9c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:37.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.278+0000 7f2ba8ff9700 1 -- 172.21.15.71:0/3937745948 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2ba0056320 con 0x7f2bac101de0 2024-04-23T02:34:37.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.487+0000 7f2bb20c9700 1 -- 172.21.15.71:0/3937745948 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bac0611d0 con 0x7f2b9406cce0 2024-04-23T02:34:37.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.488+0000 7f2ba8ff9700 1 -- 172.21.15.71:0/3937745948 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2bac0611d0 con 0x7f2b9406cce0 2024-04-23T02:34:37.491 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:34:37.491 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:34:37.491 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:34:37.491 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:34:37.491 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:34:37.491 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:34:37.491 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:34:37.491 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:34:37.491 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:34:37.491 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:34:37.491 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:34:37.491 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:34:37.491 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:34:37.492 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:34:37.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:37 smithi071 ceph-mon[144544]: from='client.34986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:34:37.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:37 smithi071 ceph-mon[144544]: from='client.34990 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:34:37.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:37 smithi071 ceph-mon[144544]: from='client.34994 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:34:37.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:37 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2847131005' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:34:37.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:37 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2528046831' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:34:37.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.491+0000 7f2bb20c9700 1 -- 172.21.15.71:0/3937745948 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2b9406cce0 msgr2=0x7f2b9406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:37.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.491+0000 7f2bb20c9700 1 --2- 172.21.15.71:0/3937745948 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2b9406cce0 0x7f2b9406f1a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2bac101230 tx=0x7f2b9c01a040 comp rx=0 tx=0).stop 2024-04-23T02:34:37.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.491+0000 7f2bb20c9700 1 -- 172.21.15.71:0/3937745948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bac101de0 msgr2=0x7f2bac0ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:37.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.491+0000 7f2bb20c9700 1 --2- 172.21.15.71:0/3937745948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bac101de0 0x7f2bac0ffa40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000c8a0 tx=0x7f2ba000cc60 comp rx=0 tx=0).stop 2024-04-23T02:34:37.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.491+0000 7f2bb20c9700 1 -- 172.21.15.71:0/3937745948 shutdown_connections 2024-04-23T02:34:37.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.491+0000 7f2bb20c9700 1 --2- 172.21.15.71:0/3937745948 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2b9406cce0 0x7f2b9406f1a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:37.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.491+0000 7f2bb20c9700 1 --2- 172.21.15.71:0/3937745948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bac101de0 0x7f2bac0ffa40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:37.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.491+0000 7f2bb20c9700 1 --2- 172.21.15.71:0/3937745948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2bac102700 0x7f2bac0fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:37.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.491+0000 7f2bb20c9700 1 -- 172.21.15.71:0/3937745948 >> 172.21.15.71:0/3937745948 conn(0x7f2bac0fb610 msgr2=0x7f2bac105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:37.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.491+0000 7f2bb20c9700 1 -- 172.21.15.71:0/3937745948 shutdown_connections 2024-04-23T02:34:37.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.491+0000 7f2bb20c9700 1 -- 172.21.15.71:0/3937745948 wait complete. 2024-04-23T02:34:37.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.593+0000 7f8d54058700 1 -- 172.21.15.71:0/3842657319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d4c105bd0 msgr2=0x7f8d4c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:37.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.593+0000 7f8d54058700 1 --2- 172.21.15.71:0/3842657319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d4c105bd0 0x7f8d4c105fb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009910 tx=0x7f8d48009c20 comp rx=0 tx=0).stop 2024-04-23T02:34:37.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.593+0000 7f8d54058700 1 -- 172.21.15.71:0/3842657319 shutdown_connections 2024-04-23T02:34:37.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.593+0000 7f8d54058700 1 --2- 172.21.15.71:0/3842657319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d4c0ffc20 0x7f8d4c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:37.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.593+0000 7f8d54058700 1 --2- 172.21.15.71:0/3842657319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d4c105bd0 0x7f8d4c105fb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:37.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.593+0000 7f8d54058700 1 -- 172.21.15.71:0/3842657319 >> 172.21.15.71:0/3842657319 conn(0x7f8d4c075670 msgr2=0x7f8d4c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:37.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.593+0000 7f8d54058700 1 -- 172.21.15.71:0/3842657319 shutdown_connections 2024-04-23T02:34:37.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.593+0000 7f8d54058700 1 -- 172.21.15.71:0/3842657319 wait complete. 2024-04-23T02:34:37.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.594+0000 7f8d54058700 1 Processor -- start 2024-04-23T02:34:37.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.594+0000 7f8d54058700 1 -- start start 2024-04-23T02:34:37.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.594+0000 7f8d54058700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d4c0ffc20 0x7f8d4c195410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:37.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.594+0000 7f8d54058700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d4c105bd0 0x7f8d4c195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:37.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.594+0000 7f8d54058700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c195e90 con 0x7f8d4c105bd0 2024-04-23T02:34:37.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.594+0000 7f8d54058700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c196000 con 0x7f8d4c0ffc20 2024-04-23T02:34:37.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.594+0000 7f8d515f3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d4c105bd0 0x7f8d4c195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:37.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.594+0000 7f8d515f3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d4c105bd0 0x7f8d4c195950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32862/0 (socket says 172.21.15.71:32862) 2024-04-23T02:34:37.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.594+0000 7f8d515f3700 1 -- 172.21.15.71:0/1113282210 learned_addr learned my addr 172.21.15.71:0/1113282210 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:34:37.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.595+0000 7f8d51df4700 1 --2- 172.21.15.71:0/1113282210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d4c0ffc20 0x7f8d4c195410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:37.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.595+0000 7f8d515f3700 1 -- 172.21.15.71:0/1113282210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d4c0ffc20 msgr2=0x7f8d4c195410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:37.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.595+0000 7f8d515f3700 1 --2- 172.21.15.71:0/1113282210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d4c0ffc20 0x7f8d4c195410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:37.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.595+0000 7f8d515f3700 1 -- 172.21.15.71:0/1113282210 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d480095f0 con 0x7f8d4c105bd0 2024-04-23T02:34:37.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.595+0000 7f8d51df4700 1 --2- 172.21.15.71:0/1113282210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d4c0ffc20 0x7f8d4c195410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:34:37.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.595+0000 7f8d515f3700 1 --2- 172.21.15.71:0/1113282210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d4c105bd0 0x7f8d4c195950 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000c8f0 tx=0x7f8d4000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:37.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.595+0000 7f8d3effd700 1 -- 172.21.15.71:0/1113282210 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d40007a70 con 0x7f8d4c105bd0 2024-04-23T02:34:37.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.595+0000 7f8d54058700 1 -- 172.21.15.71:0/1113282210 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4c1a2a80 con 0x7f8d4c105bd0 2024-04-23T02:34:37.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.595+0000 7f8d3effd700 1 -- 172.21.15.71:0/1113282210 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8d40004cd0 con 0x7f8d4c105bd0 2024-04-23T02:34:37.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.595+0000 7f8d3effd700 1 -- 172.21.15.71:0/1113282210 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d40005580 con 0x7f8d4c105bd0 2024-04-23T02:34:37.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.595+0000 7f8d54058700 1 -- 172.21.15.71:0/1113282210 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4c1a2de0 con 0x7f8d4c105bd0 2024-04-23T02:34:37.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.596+0000 7f8d54058700 1 -- 172.21.15.71:0/1113282210 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d4c1080d0 con 0x7f8d4c105bd0 2024-04-23T02:34:37.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.597+0000 7f8d3effd700 1 -- 172.21.15.71:0/1113282210 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8d40004750 con 0x7f8d4c105bd0 2024-04-23T02:34:37.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.597+0000 7f8d3effd700 1 --2- 172.21.15.71:0/1113282210 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8d3806cce0 0x7f8d3806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:34:37.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.597+0000 7f8d3effd700 1 -- 172.21.15.71:0/1113282210 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8d40013070 con 0x7f8d4c105bd0 2024-04-23T02:34:37.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.597+0000 7f8d51df4700 1 --2- 172.21.15.71:0/1113282210 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8d3806cce0 0x7f8d3806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:34:37.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.597+0000 7f8d51df4700 1 --2- 172.21.15.71:0/1113282210 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8d3806cce0 0x7f8d3806f1a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800b5c0 tx=0x7f8d48005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:34:37.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.602+0000 7f8d3effd700 1 -- 172.21.15.71:0/1113282210 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8d40059f30 con 0x7f8d4c105bd0 2024-04-23T02:34:37.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:37 smithi192 ceph-mon[108617]: from='client.34986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:34:37.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:37 smithi192 ceph-mon[108617]: from='client.34990 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:34:37.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:37 smithi192 ceph-mon[108617]: from='client.34994 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:34:37.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:37 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2847131005' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:34:37.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:37 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2528046831' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:34:37.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.873+0000 7f8d54058700 1 -- 172.21.15.71:0/1113282210 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8d4c04f2e0 con 0x7f8d4c105bd0 2024-04-23T02:34:37.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.873+0000 7f8d3effd700 1 -- 172.21.15.71:0/1113282210 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f8d40059ac0 con 0x7f8d4c105bd0 2024-04-23T02:34:37.876 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:34:37.876 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:34:37.876 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:34:37.876 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:34:37.876 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:34:37.876 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:34:37.876 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:34:37.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.875+0000 7f8d54058700 1 -- 172.21.15.71:0/1113282210 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8d3806cce0 msgr2=0x7f8d3806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:37.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.875+0000 7f8d54058700 1 --2- 172.21.15.71:0/1113282210 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8d3806cce0 0x7f8d3806f1a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800b5c0 tx=0x7f8d48005960 comp rx=0 tx=0).stop 2024-04-23T02:34:37.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.875+0000 7f8d54058700 1 -- 172.21.15.71:0/1113282210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d4c105bd0 msgr2=0x7f8d4c195950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:34:37.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.875+0000 7f8d54058700 1 --2- 172.21.15.71:0/1113282210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d4c105bd0 0x7f8d4c195950 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000c8f0 tx=0x7f8d4000ccb0 comp rx=0 tx=0).stop 2024-04-23T02:34:37.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.875+0000 7f8d54058700 1 -- 172.21.15.71:0/1113282210 shutdown_connections 2024-04-23T02:34:37.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.875+0000 7f8d54058700 1 --2- 172.21.15.71:0/1113282210 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8d3806cce0 0x7f8d3806f1a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:37.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.875+0000 7f8d54058700 1 --2- 172.21.15.71:0/1113282210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d4c0ffc20 0x7f8d4c195410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:37.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.875+0000 7f8d54058700 1 --2- 172.21.15.71:0/1113282210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d4c105bd0 0x7f8d4c195950 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:34:37.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.875+0000 7f8d54058700 1 -- 172.21.15.71:0/1113282210 >> 172.21.15.71:0/1113282210 conn(0x7f8d4c075670 msgr2=0x7f8d4c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:34:37.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.875+0000 7f8d54058700 1 -- 172.21.15.71:0/1113282210 shutdown_connections 2024-04-23T02:34:37.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:34:37.875+0000 7f8d54058700 1 -- 172.21.15.71:0/1113282210 wait complete. 2024-04-23T02:34:38.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:38 smithi071 ceph-mon[144544]: pgmap v639: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:38.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:38 smithi071 ceph-mon[144544]: from='client.35006 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:34:38.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:38 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1113282210' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:34:38.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:38 smithi192 ceph-mon[108617]: pgmap v639: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:38.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:38 smithi192 ceph-mon[108617]: from='client.35006 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:34:38.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:38 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1113282210' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:34:40.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:40 smithi071 ceph-mon[144544]: pgmap v640: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:40.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:40 smithi192 ceph-mon[108617]: pgmap v640: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:42.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:42 smithi192 ceph-mon[108617]: pgmap v641: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:42.633 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:42 smithi071 ceph-mon[144544]: pgmap v641: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:44.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:44 smithi192 ceph-mon[108617]: pgmap v642: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:44.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:44 smithi071 ceph-mon[144544]: pgmap v642: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:46.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:46 smithi192 ceph-mon[108617]: pgmap v643: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:46.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:46 smithi071 ceph-mon[144544]: pgmap v643: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:48.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:48 smithi192 ceph-mon[108617]: pgmap v644: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:48.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:48 smithi071 ceph-mon[144544]: pgmap v644: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:50.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:50 smithi192 ceph-mon[108617]: pgmap v645: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:50.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:34:50.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:50 smithi071 ceph-mon[144544]: pgmap v645: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:50.746 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:34:52.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:52 smithi192 ceph-mon[108617]: pgmap v646: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:52.633 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:52 smithi071 ceph-mon[144544]: pgmap v646: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:54.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:54 smithi192 ceph-mon[108617]: pgmap v647: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:54.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:54 smithi071 ceph-mon[144544]: pgmap v647: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:56.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:56 smithi192 ceph-mon[108617]: pgmap v648: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:56.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:56 smithi071 ceph-mon[144544]: pgmap v648: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:58.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:34:58 smithi192 ceph-mon[108617]: pgmap v649: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:34:58.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:34:58 smithi071 ceph-mon[144544]: pgmap v649: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:00.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:00 smithi192 ceph-mon[108617]: pgmap v650: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:00.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:00 smithi071 ceph-mon[144544]: pgmap v650: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:02.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:02 smithi192 ceph-mon[108617]: pgmap v651: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:02.632 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:02 smithi071 ceph-mon[144544]: pgmap v651: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:04.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:04 smithi192 ceph-mon[108617]: pgmap v652: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:04.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:04 smithi071 ceph-mon[144544]: pgmap v652: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:05.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:35:05.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:35:06.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:06 smithi192 ceph-mon[108617]: pgmap v653: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:06.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:06 smithi071 ceph-mon[144544]: pgmap v653: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:07.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.990+0000 7fb1b8b13700 1 -- 172.21.15.71:0/4228934898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b4100ea0 msgr2=0x7fb1b4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:07.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.990+0000 7fb1b8b13700 1 --2- 172.21.15.71:0/4228934898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b4100ea0 0x7fb1b4101280 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009910 tx=0x7fb1a8009c20 comp rx=0 tx=0).stop 2024-04-23T02:35:07.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.990+0000 7fb1b8b13700 1 -- 172.21.15.71:0/4228934898 shutdown_connections 2024-04-23T02:35:07.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.990+0000 7fb1b8b13700 1 --2- 172.21.15.71:0/4228934898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1b4101850 0x7fb1b41058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:07.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.990+0000 7fb1b8b13700 1 --2- 172.21.15.71:0/4228934898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b4100ea0 0x7fb1b4101280 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:07.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.990+0000 7fb1b8b13700 1 -- 172.21.15.71:0/4228934898 >> 172.21.15.71:0/4228934898 conn(0x7fb1b40fc750 msgr2=0x7fb1b40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:07.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.990+0000 7fb1b8b13700 1 -- 172.21.15.71:0/4228934898 shutdown_connections 2024-04-23T02:35:07.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.990+0000 7fb1b8b13700 1 -- 172.21.15.71:0/4228934898 wait complete. 2024-04-23T02:35:07.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.991+0000 7fb1b8b13700 1 Processor -- start 2024-04-23T02:35:07.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.991+0000 7fb1b8b13700 1 -- start start 2024-04-23T02:35:07.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.991+0000 7fb1b8b13700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1b4100ea0 0x7fb1b419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:07.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.991+0000 7fb1b8b13700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b4101850 0x7fb1b419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:07.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.991+0000 7fb1b8b13700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b419d0a0 con 0x7fb1b4101850 2024-04-23T02:35:07.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.991+0000 7fb1b8b13700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b41965e0 con 0x7fb1b4100ea0 2024-04-23T02:35:07.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.991+0000 7fb1b1d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b4101850 0x7fb1b419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:07.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.991+0000 7fb1b1d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b4101850 0x7fb1b419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56214/0 (socket says 172.21.15.71:56214) 2024-04-23T02:35:07.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.991+0000 7fb1b1d9b700 1 -- 172.21.15.71:0/1837972687 learned_addr learned my addr 172.21.15.71:0/1837972687 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:35:07.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.992+0000 7fb1b259c700 1 --2- 172.21.15.71:0/1837972687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1b4100ea0 0x7fb1b419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:07.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.992+0000 7fb1b1d9b700 1 -- 172.21.15.71:0/1837972687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1b4100ea0 msgr2=0x7fb1b419c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:07.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.992+0000 7fb1b1d9b700 1 --2- 172.21.15.71:0/1837972687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1b4100ea0 0x7fb1b419c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:07.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.992+0000 7fb1b1d9b700 1 -- 172.21.15.71:0/1837972687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a80095f0 con 0x7fb1b4101850 2024-04-23T02:35:07.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.992+0000 7fb1b1d9b700 1 --2- 172.21.15.71:0/1837972687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b4101850 0x7fb1b419caa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400c960 tx=0x7fb1a400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:07.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.992+0000 7fb1a37fe700 1 -- 172.21.15.71:0/1837972687 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1a4007a70 con 0x7fb1b4101850 2024-04-23T02:35:07.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.992+0000 7fb1b8b13700 1 -- 172.21.15.71:0/1837972687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b41968c0 con 0x7fb1b4101850 2024-04-23T02:35:07.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.992+0000 7fb1a37fe700 1 -- 172.21.15.71:0/1837972687 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb1a4007bd0 con 0x7fb1b4101850 2024-04-23T02:35:07.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.992+0000 7fb1a37fe700 1 -- 172.21.15.71:0/1837972687 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1a4018630 con 0x7fb1b4101850 2024-04-23T02:35:07.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.992+0000 7fb1b8b13700 1 -- 172.21.15.71:0/1837972687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b4196e10 con 0x7fb1b4101850 2024-04-23T02:35:07.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.993+0000 7fb1b8b13700 1 -- 172.21.15.71:0/1837972687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b4109240 con 0x7fb1b4101850 2024-04-23T02:35:07.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.994+0000 7fb1a37fe700 1 -- 172.21.15.71:0/1837972687 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb1a400f9f0 con 0x7fb1b4101850 2024-04-23T02:35:07.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.994+0000 7fb1a37fe700 1 --2- 172.21.15.71:0/1837972687 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb19c06cce0 0x7fb19c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:07.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.994+0000 7fb1a37fe700 1 -- 172.21.15.71:0/1837972687 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb1a408c810 con 0x7fb1b4101850 2024-04-23T02:35:07.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.994+0000 7fb1b259c700 1 --2- 172.21.15.71:0/1837972687 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb19c06cce0 0x7fb19c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:07.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.995+0000 7fb1b259c700 1 --2- 172.21.15.71:0/1837972687 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb19c06cce0 0x7fb19c06f1a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8006010 tx=0x7fb1a801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:08.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:07.998+0000 7fb1a37fe700 1 -- 172.21.15.71:0/1837972687 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb1a405a0e0 con 0x7fb1b4101850 2024-04-23T02:35:08.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.211+0000 7fb1b8b13700 1 -- 172.21.15.71:0/1837972687 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1b4197090 con 0x7fb19c06cce0 2024-04-23T02:35:08.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.213+0000 7fb1a37fe700 1 -- 172.21.15.71:0/1837972687 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb1b4197090 con 0x7fb19c06cce0 2024-04-23T02:35:08.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.215+0000 7fb1b8b13700 1 -- 172.21.15.71:0/1837972687 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb19c06cce0 msgr2=0x7fb19c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:08.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.215+0000 7fb1b8b13700 1 --2- 172.21.15.71:0/1837972687 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb19c06cce0 0x7fb19c06f1a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8006010 tx=0x7fb1a801a040 comp rx=0 tx=0).stop 2024-04-23T02:35:08.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.215+0000 7fb1b8b13700 1 -- 172.21.15.71:0/1837972687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b4101850 msgr2=0x7fb1b419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:08.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.215+0000 7fb1b8b13700 1 --2- 172.21.15.71:0/1837972687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b4101850 0x7fb1b419caa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400c960 tx=0x7fb1a400cd20 comp rx=0 tx=0).stop 2024-04-23T02:35:08.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.215+0000 7fb1b8b13700 1 -- 172.21.15.71:0/1837972687 shutdown_connections 2024-04-23T02:35:08.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.215+0000 7fb1b8b13700 1 --2- 172.21.15.71:0/1837972687 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb19c06cce0 0x7fb19c06f1a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.215+0000 7fb1b8b13700 1 --2- 172.21.15.71:0/1837972687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1b4100ea0 0x7fb1b419c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.215+0000 7fb1b8b13700 1 --2- 172.21.15.71:0/1837972687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b4101850 0x7fb1b419caa0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.215+0000 7fb1b8b13700 1 -- 172.21.15.71:0/1837972687 >> 172.21.15.71:0/1837972687 conn(0x7fb1b40fc750 msgr2=0x7fb1b40fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:08.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.215+0000 7fb1b8b13700 1 -- 172.21.15.71:0/1837972687 shutdown_connections 2024-04-23T02:35:08.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.215+0000 7fb1b8b13700 1 -- 172.21.15.71:0/1837972687 wait complete. 2024-04-23T02:35:08.227 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:35:08.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.319+0000 7f5a4b31a700 1 -- 172.21.15.71:0/1187481658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a440ffb40 msgr2=0x7f5a440fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:08.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.319+0000 7f5a4b31a700 1 --2- 172.21.15.71:0/1187481658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a440ffb40 0x7f5a440fffc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009910 tx=0x7f5a40009c20 comp rx=0 tx=0).stop 2024-04-23T02:35:08.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.319+0000 7f5a4b31a700 1 -- 172.21.15.71:0/1187481658 shutdown_connections 2024-04-23T02:35:08.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.319+0000 7f5a4b31a700 1 --2- 172.21.15.71:0/1187481658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a440ffb40 0x7f5a440fffc0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.319+0000 7f5a4b31a700 1 --2- 172.21.15.71:0/1187481658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a44105bf0 0x7f5a44105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.319+0000 7f5a4b31a700 1 -- 172.21.15.71:0/1187481658 >> 172.21.15.71:0/1187481658 conn(0x7f5a440fb610 msgr2=0x7f5a440fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:08.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.319+0000 7f5a4b31a700 1 -- 172.21.15.71:0/1187481658 shutdown_connections 2024-04-23T02:35:08.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.319+0000 7f5a4b31a700 1 -- 172.21.15.71:0/1187481658 wait complete. 2024-04-23T02:35:08.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.320+0000 7f5a4b31a700 1 Processor -- start 2024-04-23T02:35:08.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.320+0000 7f5a4b31a700 1 -- start start 2024-04-23T02:35:08.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.320+0000 7f5a4b31a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a440ffb40 0x7f5a44197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:08.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.320+0000 7f5a4b31a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a44105bf0 0x7f5a44197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:08.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.320+0000 7f5a4b31a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a441980f0 con 0x7f5a440ffb40 2024-04-23T02:35:08.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.320+0000 7f5a4b31a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a44198230 con 0x7f5a44105bf0 2024-04-23T02:35:08.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.320+0000 7f5a490b6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a440ffb40 0x7f5a44197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:08.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.320+0000 7f5a490b6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a440ffb40 0x7f5a44197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56236/0 (socket says 172.21.15.71:56236) 2024-04-23T02:35:08.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.320+0000 7f5a490b6700 1 -- 172.21.15.71:0/2887824051 learned_addr learned my addr 172.21.15.71:0/2887824051 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:35:08.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.320+0000 7f5a488b5700 1 --2- 172.21.15.71:0/2887824051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a44105bf0 0x7f5a44197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:08.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.321+0000 7f5a490b6700 1 -- 172.21.15.71:0/2887824051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a44105bf0 msgr2=0x7f5a44197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:08.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.321+0000 7f5a490b6700 1 --2- 172.21.15.71:0/2887824051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a44105bf0 0x7f5a44197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.321+0000 7f5a490b6700 1 -- 172.21.15.71:0/2887824051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a400095f0 con 0x7f5a440ffb40 2024-04-23T02:35:08.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.321+0000 7f5a490b6700 1 --2- 172.21.15.71:0/2887824051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a440ffb40 0x7f5a44197590 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400c8a0 tx=0x7f5a3400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:08.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.321+0000 7f5a3a7fc700 1 -- 172.21.15.71:0/2887824051 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a3400cea0 con 0x7f5a440ffb40 2024-04-23T02:35:08.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.321+0000 7f5a3a7fc700 1 -- 172.21.15.71:0/2887824051 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5a34004830 con 0x7f5a440ffb40 2024-04-23T02:35:08.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.321+0000 7f5a3a7fc700 1 -- 172.21.15.71:0/2887824051 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a34005500 con 0x7f5a440ffb40 2024-04-23T02:35:08.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.321+0000 7f5a4b31a700 1 -- 172.21.15.71:0/2887824051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a441a0600 con 0x7f5a440ffb40 2024-04-23T02:35:08.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.321+0000 7f5a4b31a700 1 -- 172.21.15.71:0/2887824051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a441a0ac0 con 0x7f5a440ffb40 2024-04-23T02:35:08.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.322+0000 7f5a4b31a700 1 -- 172.21.15.71:0/2887824051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a441080a0 con 0x7f5a440ffb40 2024-04-23T02:35:08.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.322+0000 7f5a3a7fc700 1 -- 172.21.15.71:0/2887824051 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5a340049a0 con 0x7f5a440ffb40 2024-04-23T02:35:08.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.323+0000 7f5a3a7fc700 1 --2- 172.21.15.71:0/2887824051 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5a3006cce0 0x7f5a3006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:08.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.323+0000 7f5a3a7fc700 1 -- 172.21.15.71:0/2887824051 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5a3408c1d0 con 0x7f5a440ffb40 2024-04-23T02:35:08.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.323+0000 7f5a488b5700 1 --2- 172.21.15.71:0/2887824051 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5a3006cce0 0x7f5a3006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:08.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.323+0000 7f5a488b5700 1 --2- 172.21.15.71:0/2887824051 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5a3006cce0 0x7f5a3006f1a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5a440696c0 tx=0x7f5a4001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:08.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.327+0000 7f5a3a7fc700 1 -- 172.21.15.71:0/2887824051 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5a340562c0 con 0x7f5a440ffb40 2024-04-23T02:35:08.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.537+0000 7f5a4b31a700 1 -- 172.21.15.71:0/2887824051 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a440611d0 con 0x7f5a3006cce0 2024-04-23T02:35:08.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.538+0000 7f5a3a7fc700 1 -- 172.21.15.71:0/2887824051 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5a440611d0 con 0x7f5a3006cce0 2024-04-23T02:35:08.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.540+0000 7f5a4b31a700 1 -- 172.21.15.71:0/2887824051 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5a3006cce0 msgr2=0x7f5a3006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:08.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.540+0000 7f5a4b31a700 1 --2- 172.21.15.71:0/2887824051 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5a3006cce0 0x7f5a3006f1a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5a440696c0 tx=0x7f5a4001a040 comp rx=0 tx=0).stop 2024-04-23T02:35:08.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.540+0000 7f5a4b31a700 1 -- 172.21.15.71:0/2887824051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a440ffb40 msgr2=0x7f5a44197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:08.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.540+0000 7f5a4b31a700 1 --2- 172.21.15.71:0/2887824051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a440ffb40 0x7f5a44197590 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400c8a0 tx=0x7f5a3400cc60 comp rx=0 tx=0).stop 2024-04-23T02:35:08.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.540+0000 7f5a4b31a700 1 -- 172.21.15.71:0/2887824051 shutdown_connections 2024-04-23T02:35:08.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.540+0000 7f5a4b31a700 1 --2- 172.21.15.71:0/2887824051 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5a3006cce0 0x7f5a3006f1a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.540+0000 7f5a4b31a700 1 --2- 172.21.15.71:0/2887824051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a440ffb40 0x7f5a44197590 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.540+0000 7f5a4b31a700 1 --2- 172.21.15.71:0/2887824051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a44105bf0 0x7f5a44197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.540+0000 7f5a4b31a700 1 -- 172.21.15.71:0/2887824051 >> 172.21.15.71:0/2887824051 conn(0x7f5a440fb610 msgr2=0x7f5a440fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:08.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.540+0000 7f5a4b31a700 1 -- 172.21.15.71:0/2887824051 shutdown_connections 2024-04-23T02:35:08.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.540+0000 7f5a4b31a700 1 -- 172.21.15.71:0/2887824051 wait complete. 2024-04-23T02:35:08.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:08 smithi192 ceph-mon[108617]: pgmap v654: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:08.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.642+0000 7f0d1a3ac700 1 -- 172.21.15.71:0/1837317493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d14106cd0 msgr2=0x7f0d141070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:08.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.642+0000 7f0d1a3ac700 1 --2- 172.21.15.71:0/1837317493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d14106cd0 0x7f0d141070b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009910 tx=0x7f0d08009c20 comp rx=0 tx=0).stop 2024-04-23T02:35:08.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.642+0000 7f0d1a3ac700 1 -- 172.21.15.71:0/1837317493 shutdown_connections 2024-04-23T02:35:08.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.642+0000 7f0d1a3ac700 1 --2- 172.21.15.71:0/1837317493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d14068580 0x7f0d140689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.642+0000 7f0d1a3ac700 1 --2- 172.21.15.71:0/1837317493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d14106cd0 0x7f0d141070b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.642+0000 7f0d1a3ac700 1 -- 172.21.15.71:0/1837317493 >> 172.21.15.71:0/1837317493 conn(0x7f0d14075670 msgr2=0x7f0d14075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:08.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.642+0000 7f0d1a3ac700 1 -- 172.21.15.71:0/1837317493 shutdown_connections 2024-04-23T02:35:08.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.642+0000 7f0d1a3ac700 1 -- 172.21.15.71:0/1837317493 wait complete. 2024-04-23T02:35:08.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.643+0000 7f0d1a3ac700 1 Processor -- start 2024-04-23T02:35:08.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.643+0000 7f0d1a3ac700 1 -- start start 2024-04-23T02:35:08.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.643+0000 7f0d1a3ac700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d14068580 0x7f0d140ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:08.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.643+0000 7f0d1a3ac700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d14106cd0 0x7f0d14100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:08.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.643+0000 7f0d1a3ac700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d14100770 con 0x7f0d14068580 2024-04-23T02:35:08.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.643+0000 7f0d1a3ac700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d141008b0 con 0x7f0d14106cd0 2024-04-23T02:35:08.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.643+0000 7f0d13fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d14068580 0x7f0d140ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:08.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.643+0000 7f0d13fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d14068580 0x7f0d140ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56262/0 (socket says 172.21.15.71:56262) 2024-04-23T02:35:08.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.643+0000 7f0d13fff700 1 -- 172.21.15.71:0/3686762324 learned_addr learned my addr 172.21.15.71:0/3686762324 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:35:08.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.643+0000 7f0d137fe700 1 --2- 172.21.15.71:0/3686762324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d14106cd0 0x7f0d14100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:08.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.643+0000 7f0d13fff700 1 -- 172.21.15.71:0/3686762324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d14106cd0 msgr2=0x7f0d14100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:08.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.643+0000 7f0d13fff700 1 --2- 172.21.15.71:0/3686762324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d14106cd0 0x7f0d14100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.643+0000 7f0d13fff700 1 -- 172.21.15.71:0/3686762324 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d080095f0 con 0x7f0d14068580 2024-04-23T02:35:08.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.643+0000 7f0d13fff700 1 --2- 172.21.15.71:0/3686762324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d14068580 0x7f0d140ffcf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0d080092e0 tx=0x7f0d0800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:08.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.644+0000 7f0d117fa700 1 -- 172.21.15.71:0/3686762324 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d0801d070 con 0x7f0d14068580 2024-04-23T02:35:08.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.644+0000 7f0d1a3ac700 1 -- 172.21.15.71:0/3686762324 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d14100b30 con 0x7f0d14068580 2024-04-23T02:35:08.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.644+0000 7f0d117fa700 1 -- 172.21.15.71:0/3686762324 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0d08005e00 con 0x7f0d14068580 2024-04-23T02:35:08.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.644+0000 7f0d117fa700 1 -- 172.21.15.71:0/3686762324 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d0800fe60 con 0x7f0d14068580 2024-04-23T02:35:08.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.644+0000 7f0d1a3ac700 1 -- 172.21.15.71:0/3686762324 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d141a2c90 con 0x7f0d14068580 2024-04-23T02:35:08.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.645+0000 7f0d1a3ac700 1 -- 172.21.15.71:0/3686762324 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d141091f0 con 0x7f0d14068580 2024-04-23T02:35:08.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.645+0000 7f0d117fa700 1 -- 172.21.15.71:0/3686762324 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0d0800f6d0 con 0x7f0d14068580 2024-04-23T02:35:08.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.646+0000 7f0d117fa700 1 --2- 172.21.15.71:0/3686762324 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0cfc06cce0 0x7f0cfc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:08.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.646+0000 7f0d117fa700 1 -- 172.21.15.71:0/3686762324 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0d0808d8b0 con 0x7f0d14068580 2024-04-23T02:35:08.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.646+0000 7f0d137fe700 1 --2- 172.21.15.71:0/3686762324 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0cfc06cce0 0x7f0cfc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:08.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.646+0000 7f0d137fe700 1 --2- 172.21.15.71:0/3686762324 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0cfc06cce0 0x7f0cfc06f1a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0d04005950 tx=0x7f0d040058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:08.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.650+0000 7f0d117fa700 1 -- 172.21.15.71:0/3686762324 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0d08057920 con 0x7f0d14068580 2024-04-23T02:35:08.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:08 smithi071 ceph-mon[144544]: pgmap v654: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:08.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.854+0000 7f0d1a3ac700 1 -- 172.21.15.71:0/3686762324 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d140611d0 con 0x7f0cfc06cce0 2024-04-23T02:35:08.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.865+0000 7f0d117fa700 1 -- 172.21.15.71:0/3686762324 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0d140611d0 con 0x7f0cfc06cce0 2024-04-23T02:35:08.867 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:35:08.867 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (18m) 12m ago 30m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:35:08.867 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (30m) 12m ago 30m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:35:08.867 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (29m) 12m ago 29m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:35:08.868 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (16m) 12m ago 30m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:35:08.868 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (16m) 12m ago 29m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:35:08.868 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (18m) 12m ago 29m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:35:08.868 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (12m) 12m ago 24m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:35:08.868 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (12m) 12m ago 24m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:35:08.868 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (24m) 12m ago 24m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:35:08.868 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (12m) 12m ago 24m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:35:08.868 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (20m) 12m ago 31m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:35:08.868 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (20m) 12m ago 29m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:35:08.868 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (17m) 12m ago 31m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:35:08.868 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (16m) 12m ago 28m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:35:08.868 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (19m) 12m ago 30m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:35:08.868 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (19m) 12m ago 29m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:35:08.868 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (16m) 12m ago 28m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:35:08.868 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (15m) 12m ago 27m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:35:08.869 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (15m) 12m ago 27m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:35:08.869 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (14m) 12m ago 26m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:35:08.869 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (13m) 12m ago 26m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:35:08.869 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (13m) 12m ago 25m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:35:08.869 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (19m) 12m ago 29m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:35:08.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.867+0000 7f0d1a3ac700 1 -- 172.21.15.71:0/3686762324 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0cfc06cce0 msgr2=0x7f0cfc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:08.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.867+0000 7f0d1a3ac700 1 --2- 172.21.15.71:0/3686762324 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0cfc06cce0 0x7f0cfc06f1a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0d04005950 tx=0x7f0d040058e0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.867+0000 7f0d1a3ac700 1 -- 172.21.15.71:0/3686762324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d14068580 msgr2=0x7f0d140ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:08.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.867+0000 7f0d1a3ac700 1 --2- 172.21.15.71:0/3686762324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d14068580 0x7f0d140ffcf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0d080092e0 tx=0x7f0d0800be20 comp rx=0 tx=0).stop 2024-04-23T02:35:08.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.867+0000 7f0d1a3ac700 1 -- 172.21.15.71:0/3686762324 shutdown_connections 2024-04-23T02:35:08.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.867+0000 7f0d1a3ac700 1 --2- 172.21.15.71:0/3686762324 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0cfc06cce0 0x7f0cfc06f1a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.867+0000 7f0d1a3ac700 1 --2- 172.21.15.71:0/3686762324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d14068580 0x7f0d140ffcf0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.867+0000 7f0d1a3ac700 1 --2- 172.21.15.71:0/3686762324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d14106cd0 0x7f0d14100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.867+0000 7f0d1a3ac700 1 -- 172.21.15.71:0/3686762324 >> 172.21.15.71:0/3686762324 conn(0x7f0d14075670 msgr2=0x7f0d140fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:08.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.867+0000 7f0d1a3ac700 1 -- 172.21.15.71:0/3686762324 shutdown_connections 2024-04-23T02:35:08.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.867+0000 7f0d1a3ac700 1 -- 172.21.15.71:0/3686762324 wait complete. 2024-04-23T02:35:08.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.968+0000 7f44138b1700 1 -- 172.21.15.71:0/4054919593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f440c105bf0 msgr2=0x7f440c105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:08.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.968+0000 7f44138b1700 1 --2- 172.21.15.71:0/4054919593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f440c105bf0 0x7f440c105fd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009910 tx=0x7f43fc009c20 comp rx=0 tx=0).stop 2024-04-23T02:35:08.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.968+0000 7f44138b1700 1 -- 172.21.15.71:0/4054919593 shutdown_connections 2024-04-23T02:35:08.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.968+0000 7f44138b1700 1 --2- 172.21.15.71:0/4054919593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f440c0ffb40 0x7f440c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.968+0000 7f44138b1700 1 --2- 172.21.15.71:0/4054919593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f440c105bf0 0x7f440c105fd0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.968+0000 7f44138b1700 1 -- 172.21.15.71:0/4054919593 >> 172.21.15.71:0/4054919593 conn(0x7f440c0fb610 msgr2=0x7f440c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:08.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.968+0000 7f44138b1700 1 -- 172.21.15.71:0/4054919593 shutdown_connections 2024-04-23T02:35:08.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.968+0000 7f44138b1700 1 -- 172.21.15.71:0/4054919593 wait complete. 2024-04-23T02:35:08.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.969+0000 7f44138b1700 1 Processor -- start 2024-04-23T02:35:08.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.969+0000 7f44138b1700 1 -- start start 2024-04-23T02:35:08.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.969+0000 7f44138b1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f440c0ffb40 0x7f440c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:08.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.969+0000 7f44138b1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f440c105bf0 0x7f440c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:08.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.969+0000 7f44138b1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c198080 con 0x7f440c105bf0 2024-04-23T02:35:08.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.969+0000 7f44138b1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c1981c0 con 0x7f440c0ffb40 2024-04-23T02:35:08.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.969+0000 7f4410e4c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f440c105bf0 0x7f440c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:08.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.969+0000 7f4410e4c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f440c105bf0 0x7f440c197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56284/0 (socket says 172.21.15.71:56284) 2024-04-23T02:35:08.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.969+0000 7f4410e4c700 1 -- 172.21.15.71:0/3122511760 learned_addr learned my addr 172.21.15.71:0/3122511760 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:35:08.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.969+0000 7f441164d700 1 --2- 172.21.15.71:0/3122511760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f440c0ffb40 0x7f440c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:08.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.969+0000 7f4410e4c700 1 -- 172.21.15.71:0/3122511760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f440c0ffb40 msgr2=0x7f440c1975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:08.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.969+0000 7f4410e4c700 1 --2- 172.21.15.71:0/3122511760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f440c0ffb40 0x7f440c1975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:08.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.969+0000 7f4410e4c700 1 -- 172.21.15.71:0/3122511760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc0095f0 con 0x7f440c105bf0 2024-04-23T02:35:08.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.969+0000 7f441164d700 1 --2- 172.21.15.71:0/3122511760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f440c0ffb40 0x7f440c1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:35:08.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.970+0000 7f4410e4c700 1 --2- 172.21.15.71:0/3122511760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f440c105bf0 0x7f440c197af0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f440800c930 tx=0x7f440800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:08.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.970+0000 7f44027fc700 1 -- 172.21.15.71:0/3122511760 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4408007ad0 con 0x7f440c105bf0 2024-04-23T02:35:08.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.970+0000 7f44027fc700 1 -- 172.21.15.71:0/3122511760 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4408007c30 con 0x7f440c105bf0 2024-04-23T02:35:08.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.970+0000 7f44138b1700 1 -- 172.21.15.71:0/3122511760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440c1a0620 con 0x7f440c105bf0 2024-04-23T02:35:08.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.970+0000 7f44027fc700 1 -- 172.21.15.71:0/3122511760 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4408018630 con 0x7f440c105bf0 2024-04-23T02:35:08.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.970+0000 7f44138b1700 1 -- 172.21.15.71:0/3122511760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440c1a0a90 con 0x7f440c105bf0 2024-04-23T02:35:08.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.971+0000 7f44138b1700 1 -- 172.21.15.71:0/3122511760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f440c108020 con 0x7f440c105bf0 2024-04-23T02:35:08.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.971+0000 7f44027fc700 1 -- 172.21.15.71:0/3122511760 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f440800f450 con 0x7f440c105bf0 2024-04-23T02:35:08.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.971+0000 7f44027fc700 1 --2- 172.21.15.71:0/3122511760 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f43f806cce0 0x7f43f806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:08.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.972+0000 7f44027fc700 1 -- 172.21.15.71:0/3122511760 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f440808c780 con 0x7f440c105bf0 2024-04-23T02:35:08.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.972+0000 7f441164d700 1 --2- 172.21.15.71:0/3122511760 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f43f806cce0 0x7f43f806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:08.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.972+0000 7f441164d700 1 --2- 172.21.15.71:0/3122511760 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f43f806cce0 0x7f43f806f1a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f43fc00b5c0 tx=0x7f43fc005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:08.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:08.976+0000 7f44027fc700 1 -- 172.21.15.71:0/3122511760 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f4408056920 con 0x7f440c105bf0 2024-04-23T02:35:09.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.249+0000 7f44138b1700 1 -- 172.21.15.71:0/3122511760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f440c04f2e0 con 0x7f440c105bf0 2024-04-23T02:35:09.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.250+0000 7f44027fc700 1 -- 172.21.15.71:0/3122511760 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f440808ae60 con 0x7f440c105bf0 2024-04-23T02:35:09.252 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:35:09.252 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:35:09.252 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:35:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:35:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:35:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:35:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:35:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:35:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:35:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:35:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:35:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:35:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:35:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:35:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:35:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:35:09.253 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:35:09.254 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:35:09.254 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:35:09.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.252+0000 7f44138b1700 1 -- 172.21.15.71:0/3122511760 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f43f806cce0 msgr2=0x7f43f806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:09.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.252+0000 7f44138b1700 1 --2- 172.21.15.71:0/3122511760 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f43f806cce0 0x7f43f806f1a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f43fc00b5c0 tx=0x7f43fc005960 comp rx=0 tx=0).stop 2024-04-23T02:35:09.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.252+0000 7f44138b1700 1 -- 172.21.15.71:0/3122511760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f440c105bf0 msgr2=0x7f440c197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:09.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.252+0000 7f44138b1700 1 --2- 172.21.15.71:0/3122511760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f440c105bf0 0x7f440c197af0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f440800c930 tx=0x7f440800ccf0 comp rx=0 tx=0).stop 2024-04-23T02:35:09.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.252+0000 7f44138b1700 1 -- 172.21.15.71:0/3122511760 shutdown_connections 2024-04-23T02:35:09.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.252+0000 7f44138b1700 1 --2- 172.21.15.71:0/3122511760 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f43f806cce0 0x7f43f806f1a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:09.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.252+0000 7f44138b1700 1 --2- 172.21.15.71:0/3122511760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f440c0ffb40 0x7f440c1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:09.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.252+0000 7f44138b1700 1 --2- 172.21.15.71:0/3122511760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f440c105bf0 0x7f440c197af0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:09.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.252+0000 7f44138b1700 1 -- 172.21.15.71:0/3122511760 >> 172.21.15.71:0/3122511760 conn(0x7f440c0fb610 msgr2=0x7f440c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:09.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.252+0000 7f44138b1700 1 -- 172.21.15.71:0/3122511760 shutdown_connections 2024-04-23T02:35:09.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.252+0000 7f44138b1700 1 -- 172.21.15.71:0/3122511760 wait complete. 2024-04-23T02:35:09.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.356+0000 7f1a40631700 1 -- 172.21.15.71:0/1303635506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a380ffda0 msgr2=0x7f1a38100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:09.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.356+0000 7f1a40631700 1 --2- 172.21.15.71:0/1303635506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a380ffda0 0x7f1a38100180 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009910 tx=0x7f1a28009c20 comp rx=0 tx=0).stop 2024-04-23T02:35:09.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.356+0000 7f1a40631700 1 -- 172.21.15.71:0/1303635506 shutdown_connections 2024-04-23T02:35:09.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.356+0000 7f1a40631700 1 --2- 172.21.15.71:0/1303635506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a38100750 0x7f1a381047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:09.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.356+0000 7f1a40631700 1 --2- 172.21.15.71:0/1303635506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a380ffda0 0x7f1a38100180 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:09.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.356+0000 7f1a40631700 1 -- 172.21.15.71:0/1303635506 >> 172.21.15.71:0/1303635506 conn(0x7f1a380fb650 msgr2=0x7f1a380fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:09.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.356+0000 7f1a40631700 1 -- 172.21.15.71:0/1303635506 shutdown_connections 2024-04-23T02:35:09.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.356+0000 7f1a40631700 1 -- 172.21.15.71:0/1303635506 wait complete. 2024-04-23T02:35:09.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.356+0000 7f1a40631700 1 Processor -- start 2024-04-23T02:35:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.356+0000 7f1a40631700 1 -- start start 2024-04-23T02:35:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.357+0000 7f1a40631700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a380ffda0 0x7f1a381956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.357+0000 7f1a40631700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a38100750 0x7f1a38195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.357+0000 7f1a40631700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a38196290 con 0x7f1a380ffda0 2024-04-23T02:35:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.357+0000 7f1a40631700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a38198fc0 con 0x7f1a38100750 2024-04-23T02:35:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.357+0000 7f1a3e3cd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a380ffda0 0x7f1a381956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.357+0000 7f1a3e3cd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a380ffda0 0x7f1a381956c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56300/0 (socket says 172.21.15.71:56300) 2024-04-23T02:35:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.357+0000 7f1a3e3cd700 1 -- 172.21.15.71:0/2056302918 learned_addr learned my addr 172.21.15.71:0/2056302918 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:35:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.357+0000 7f1a3dbcc700 1 --2- 172.21.15.71:0/2056302918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a38100750 0x7f1a38195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.357+0000 7f1a3e3cd700 1 -- 172.21.15.71:0/2056302918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a38100750 msgr2=0x7f1a38195c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.357+0000 7f1a3e3cd700 1 --2- 172.21.15.71:0/2056302918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a38100750 0x7f1a38195c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.357+0000 7f1a3e3cd700 1 -- 172.21.15.71:0/2056302918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a280095f0 con 0x7f1a380ffda0 2024-04-23T02:35:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.357+0000 7f1a3e3cd700 1 --2- 172.21.15.71:0/2056302918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a380ffda0 0x7f1a381956c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1a28006010 tx=0x7f1a2800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.357+0000 7f1a2f7fe700 1 -- 172.21.15.71:0/2056302918 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a2801d070 con 0x7f1a380ffda0 2024-04-23T02:35:09.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.357+0000 7f1a2f7fe700 1 -- 172.21.15.71:0/2056302918 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1a2800f460 con 0x7f1a380ffda0 2024-04-23T02:35:09.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.357+0000 7f1a40631700 1 -- 172.21.15.71:0/2056302918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a38199240 con 0x7f1a380ffda0 2024-04-23T02:35:09.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.357+0000 7f1a2f7fe700 1 -- 172.21.15.71:0/2056302918 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a28021640 con 0x7f1a380ffda0 2024-04-23T02:35:09.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.357+0000 7f1a40631700 1 -- 172.21.15.71:0/2056302918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a38199730 con 0x7f1a380ffda0 2024-04-23T02:35:09.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.358+0000 7f1a40631700 1 -- 172.21.15.71:0/2056302918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a38108140 con 0x7f1a380ffda0 2024-04-23T02:35:09.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.359+0000 7f1a2f7fe700 1 -- 172.21.15.71:0/2056302918 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1a2800f9e0 con 0x7f1a380ffda0 2024-04-23T02:35:09.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.359+0000 7f1a2f7fe700 1 --2- 172.21.15.71:0/2056302918 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1a2406cce0 0x7f1a2406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:09.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.359+0000 7f1a2f7fe700 1 -- 172.21.15.71:0/2056302918 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1a2808d900 con 0x7f1a380ffda0 2024-04-23T02:35:09.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.359+0000 7f1a3dbcc700 1 --2- 172.21.15.71:0/2056302918 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1a2406cce0 0x7f1a2406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:09.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.360+0000 7f1a3dbcc700 1 --2- 172.21.15.71:0/2056302918 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1a2406cce0 0x7f1a2406f1a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1a34005fd0 tx=0x7f1a3400a560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:09.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.364+0000 7f1a2f7fe700 1 -- 172.21.15.71:0/2056302918 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1a28057970 con 0x7f1a380ffda0 2024-04-23T02:35:09.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.607+0000 7f1a40631700 1 -- 172.21.15.71:0/2056302918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1a38196a20 con 0x7f1a380ffda0 2024-04-23T02:35:09.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.607+0000 7f1a2f7fe700 1 -- 172.21.15.71:0/2056302918 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f1a2805af90 con 0x7f1a380ffda0 2024-04-23T02:35:09.610 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:35:09.610 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:35:09.610 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:35:09.610 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:35:09.610 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:35:09.610 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:35:09.611 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:35:09.611 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:35:09.611 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:35:09.611 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:35:09.611 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:35:09.611 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:35:09.611 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:35:09.611 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:35:09.611 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:35:09.611 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:35:09.611 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:35:09.611 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:35:09.611 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:35:09.611 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:35:09.611 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:35:09.612 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:35:09.612 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:35:09.612 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:35:09.612 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:35:09.612 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:35:09.612 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:35:09.612 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:35:09.612 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:35:09.612 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:35:09.612 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:35:09.612 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:35:09.612 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:35:09.612 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:35:09.612 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:35:09.613 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:35:09.613 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:35:09.613 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:35:09.613 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:35:09.613 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:35:09.613 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:35:09.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.609+0000 7f1a40631700 1 -- 172.21.15.71:0/2056302918 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1a2406cce0 msgr2=0x7f1a2406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:09.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.610+0000 7f1a40631700 1 --2- 172.21.15.71:0/2056302918 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1a2406cce0 0x7f1a2406f1a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1a34005fd0 tx=0x7f1a3400a560 comp rx=0 tx=0).stop 2024-04-23T02:35:09.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.610+0000 7f1a40631700 1 -- 172.21.15.71:0/2056302918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a380ffda0 msgr2=0x7f1a381956c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:09.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.610+0000 7f1a40631700 1 --2- 172.21.15.71:0/2056302918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a380ffda0 0x7f1a381956c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1a28006010 tx=0x7f1a2800be20 comp rx=0 tx=0).stop 2024-04-23T02:35:09.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.610+0000 7f1a40631700 1 -- 172.21.15.71:0/2056302918 shutdown_connections 2024-04-23T02:35:09.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.610+0000 7f1a40631700 1 --2- 172.21.15.71:0/2056302918 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1a2406cce0 0x7f1a2406f1a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:09.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.610+0000 7f1a40631700 1 --2- 172.21.15.71:0/2056302918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a380ffda0 0x7f1a381956c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:09.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.610+0000 7f1a40631700 1 --2- 172.21.15.71:0/2056302918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a38100750 0x7f1a38195c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:09.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.610+0000 7f1a40631700 1 -- 172.21.15.71:0/2056302918 >> 172.21.15.71:0/2056302918 conn(0x7f1a380fb650 msgr2=0x7f1a380fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:09.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.610+0000 7f1a40631700 1 -- 172.21.15.71:0/2056302918 shutdown_connections 2024-04-23T02:35:09.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.610+0000 7f1a40631700 1 -- 172.21.15.71:0/2056302918 wait complete. 2024-04-23T02:35:09.615 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:35:09.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:09 smithi192 ceph-mon[108617]: from='client.35014 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:35:09.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:09 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3122511760' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:35:09.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.713+0000 7fbf3e654700 1 -- 172.21.15.71:0/1035736103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf38106cd0 msgr2=0x7fbf381070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:09.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.713+0000 7fbf3e654700 1 --2- 172.21.15.71:0/1035736103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf38106cd0 0x7fbf381070b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009910 tx=0x7fbf2c009c20 comp rx=0 tx=0).stop 2024-04-23T02:35:09.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.713+0000 7fbf3e654700 1 -- 172.21.15.71:0/1035736103 shutdown_connections 2024-04-23T02:35:09.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.713+0000 7fbf3e654700 1 --2- 172.21.15.71:0/1035736103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf38068580 0x7fbf380689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:09.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.713+0000 7fbf3e654700 1 --2- 172.21.15.71:0/1035736103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf38106cd0 0x7fbf381070b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:09.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.713+0000 7fbf3e654700 1 -- 172.21.15.71:0/1035736103 >> 172.21.15.71:0/1035736103 conn(0x7fbf38075670 msgr2=0x7fbf38075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:09.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.713+0000 7fbf3e654700 1 -- 172.21.15.71:0/1035736103 shutdown_connections 2024-04-23T02:35:09.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.713+0000 7fbf3e654700 1 -- 172.21.15.71:0/1035736103 wait complete. 2024-04-23T02:35:09.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.714+0000 7fbf3e654700 1 Processor -- start 2024-04-23T02:35:09.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.714+0000 7fbf3e654700 1 -- start start 2024-04-23T02:35:09.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.714+0000 7fbf3e654700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf38068580 0x7fbf3819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:09.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.714+0000 7fbf3e654700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf38106cd0 0x7fbf3819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:09.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.714+0000 7fbf3e654700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3819d250 con 0x7fbf38068580 2024-04-23T02:35:09.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.714+0000 7fbf3e654700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf38196700 con 0x7fbf38106cd0 2024-04-23T02:35:09.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.714+0000 7fbf37fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf38068580 0x7fbf3819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:09.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.714+0000 7fbf37fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf38068580 0x7fbf3819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56316/0 (socket says 172.21.15.71:56316) 2024-04-23T02:35:09.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.714+0000 7fbf37fff700 1 -- 172.21.15.71:0/2754601314 learned_addr learned my addr 172.21.15.71:0/2754601314 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:35:09.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.715+0000 7fbf377fe700 1 --2- 172.21.15.71:0/2754601314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf38106cd0 0x7fbf3819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:09.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.715+0000 7fbf37fff700 1 -- 172.21.15.71:0/2754601314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf38106cd0 msgr2=0x7fbf3819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:09.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.715+0000 7fbf37fff700 1 --2- 172.21.15.71:0/2754601314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf38106cd0 0x7fbf3819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:09.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.715+0000 7fbf37fff700 1 -- 172.21.15.71:0/2754601314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c0095f0 con 0x7fbf38068580 2024-04-23T02:35:09.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.715+0000 7fbf377fe700 1 --2- 172.21.15.71:0/2754601314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf38106cd0 0x7fbf3819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:35:09.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.715+0000 7fbf37fff700 1 --2- 172.21.15.71:0/2754601314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf38068580 0x7fbf3819c680 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c006010 tx=0x7fbf2c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:09.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.715+0000 7fbf357fa700 1 -- 172.21.15.71:0/2754601314 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf2c01d070 con 0x7fbf38068580 2024-04-23T02:35:09.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.715+0000 7fbf3e654700 1 -- 172.21.15.71:0/2754601314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf38196980 con 0x7fbf38068580 2024-04-23T02:35:09.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.715+0000 7fbf357fa700 1 -- 172.21.15.71:0/2754601314 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbf2c00f460 con 0x7fbf38068580 2024-04-23T02:35:09.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.715+0000 7fbf357fa700 1 -- 172.21.15.71:0/2754601314 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf2c021640 con 0x7fbf38068580 2024-04-23T02:35:09.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.715+0000 7fbf3e654700 1 -- 172.21.15.71:0/2754601314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf38196e70 con 0x7fbf38068580 2024-04-23T02:35:09.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.716+0000 7fbf3e654700 1 -- 172.21.15.71:0/2754601314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf381091f0 con 0x7fbf38068580 2024-04-23T02:35:09.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.716+0000 7fbf357fa700 1 -- 172.21.15.71:0/2754601314 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbf2c00fa60 con 0x7fbf38068580 2024-04-23T02:35:09.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.717+0000 7fbf357fa700 1 --2- 172.21.15.71:0/2754601314 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbf2006cce0 0x7fbf2006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:09.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.717+0000 7fbf357fa700 1 -- 172.21.15.71:0/2754601314 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbf2c08d880 con 0x7fbf38068580 2024-04-23T02:35:09.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.717+0000 7fbf377fe700 1 --2- 172.21.15.71:0/2754601314 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbf2006cce0 0x7fbf2006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:09.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.717+0000 7fbf377fe700 1 --2- 172.21.15.71:0/2754601314 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbf2006cce0 0x7fbf2006f1a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbf38197b70 tx=0x7fbf28008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:09.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.721+0000 7fbf357fa700 1 -- 172.21.15.71:0/2754601314 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbf2c05b380 con 0x7fbf38068580 2024-04-23T02:35:09.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:09 smithi071 ceph-mon[144544]: from='client.35014 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:35:09.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:09 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3122511760' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:35:09.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.933+0000 7fbf3e654700 1 -- 172.21.15.71:0/2754601314 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf38197090 con 0x7fbf2006cce0 2024-04-23T02:35:09.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.934+0000 7fbf357fa700 1 -- 172.21.15.71:0/2754601314 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbf38197090 con 0x7fbf2006cce0 2024-04-23T02:35:09.937 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:35:09.937 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:35:09.937 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:35:09.937 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:35:09.937 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:35:09.937 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:35:09.937 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:35:09.937 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:35:09.937 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:35:09.938 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:35:09.938 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:35:09.938 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:35:09.938 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:35:09.938 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:35:09.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.937+0000 7fbf3e654700 1 -- 172.21.15.71:0/2754601314 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbf2006cce0 msgr2=0x7fbf2006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:09.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.937+0000 7fbf3e654700 1 --2- 172.21.15.71:0/2754601314 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbf2006cce0 0x7fbf2006f1a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbf38197b70 tx=0x7fbf28008040 comp rx=0 tx=0).stop 2024-04-23T02:35:09.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.937+0000 7fbf3e654700 1 -- 172.21.15.71:0/2754601314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf38068580 msgr2=0x7fbf3819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:09.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.937+0000 7fbf3e654700 1 --2- 172.21.15.71:0/2754601314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf38068580 0x7fbf3819c680 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c006010 tx=0x7fbf2c00be20 comp rx=0 tx=0).stop 2024-04-23T02:35:09.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.937+0000 7fbf3e654700 1 -- 172.21.15.71:0/2754601314 shutdown_connections 2024-04-23T02:35:09.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.937+0000 7fbf3e654700 1 --2- 172.21.15.71:0/2754601314 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbf2006cce0 0x7fbf2006f1a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:09.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.937+0000 7fbf3e654700 1 --2- 172.21.15.71:0/2754601314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf38068580 0x7fbf3819c680 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:09.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.937+0000 7fbf3e654700 1 --2- 172.21.15.71:0/2754601314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf38106cd0 0x7fbf3819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:09.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.937+0000 7fbf3e654700 1 -- 172.21.15.71:0/2754601314 >> 172.21.15.71:0/2754601314 conn(0x7fbf38075670 msgr2=0x7fbf380fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:09.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.937+0000 7fbf3e654700 1 -- 172.21.15.71:0/2754601314 shutdown_connections 2024-04-23T02:35:09.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:09.937+0000 7fbf3e654700 1 -- 172.21.15.71:0/2754601314 wait complete. 2024-04-23T02:35:10.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.039+0000 7fa2c6471700 1 -- 172.21.15.71:0/2597228547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2c0105bd0 msgr2=0x7fa2c0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:10.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.039+0000 7fa2c6471700 1 --2- 172.21.15.71:0/2597228547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2c0105bd0 0x7fa2c0105fb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009910 tx=0x7fa2b0009c20 comp rx=0 tx=0).stop 2024-04-23T02:35:10.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.039+0000 7fa2c6471700 1 -- 172.21.15.71:0/2597228547 shutdown_connections 2024-04-23T02:35:10.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.039+0000 7fa2c6471700 1 --2- 172.21.15.71:0/2597228547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2c00ffc20 0x7fa2c0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:10.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.039+0000 7fa2c6471700 1 --2- 172.21.15.71:0/2597228547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2c0105bd0 0x7fa2c0105fb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:10.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.039+0000 7fa2c6471700 1 -- 172.21.15.71:0/2597228547 >> 172.21.15.71:0/2597228547 conn(0x7fa2c0075670 msgr2=0x7fa2c0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:10.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.039+0000 7fa2c6471700 1 -- 172.21.15.71:0/2597228547 shutdown_connections 2024-04-23T02:35:10.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.039+0000 7fa2c6471700 1 -- 172.21.15.71:0/2597228547 wait complete. 2024-04-23T02:35:10.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.040+0000 7fa2c6471700 1 Processor -- start 2024-04-23T02:35:10.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.040+0000 7fa2c6471700 1 -- start start 2024-04-23T02:35:10.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.040+0000 7fa2c6471700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2c00ffc20 0x7fa2c01975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:10.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.040+0000 7fa2c6471700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2c0105bd0 0x7fa2c0197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:10.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.040+0000 7fa2c6471700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0198140 con 0x7fa2c00ffc20 2024-04-23T02:35:10.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.040+0000 7fa2c6471700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c01982b0 con 0x7fa2c0105bd0 2024-04-23T02:35:10.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.040+0000 7fa2bffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2c00ffc20 0x7fa2c01975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:10.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.041+0000 7fa2bffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2c00ffc20 0x7fa2c01975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56320/0 (socket says 172.21.15.71:56320) 2024-04-23T02:35:10.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.041+0000 7fa2bffff700 1 -- 172.21.15.71:0/1676832004 learned_addr learned my addr 172.21.15.71:0/1676832004 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:35:10.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.041+0000 7fa2bf7fe700 1 --2- 172.21.15.71:0/1676832004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2c0105bd0 0x7fa2c0197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:10.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.041+0000 7fa2bffff700 1 -- 172.21.15.71:0/1676832004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2c0105bd0 msgr2=0x7fa2c0197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:10.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.041+0000 7fa2bffff700 1 --2- 172.21.15.71:0/1676832004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2c0105bd0 0x7fa2c0197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:10.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.041+0000 7fa2bffff700 1 -- 172.21.15.71:0/1676832004 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b00095f0 con 0x7fa2c00ffc20 2024-04-23T02:35:10.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.041+0000 7fa2bf7fe700 1 --2- 172.21.15.71:0/1676832004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2c0105bd0 0x7fa2c0197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:35:10.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.041+0000 7fa2bffff700 1 --2- 172.21.15.71:0/1676832004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2c00ffc20 0x7fa2c01975e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0006010 tx=0x7fa2b000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:10.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.041+0000 7fa2bd7fa700 1 -- 172.21.15.71:0/1676832004 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2b001d070 con 0x7fa2c00ffc20 2024-04-23T02:35:10.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.041+0000 7fa2bd7fa700 1 -- 172.21.15.71:0/1676832004 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa2b000f460 con 0x7fa2c00ffc20 2024-04-23T02:35:10.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.041+0000 7fa2c6471700 1 -- 172.21.15.71:0/1676832004 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c01a0650 con 0x7fa2c00ffc20 2024-04-23T02:35:10.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.041+0000 7fa2bd7fa700 1 -- 172.21.15.71:0/1676832004 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2b0021640 con 0x7fa2c00ffc20 2024-04-23T02:35:10.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.041+0000 7fa2c6471700 1 -- 172.21.15.71:0/1676832004 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c01a0ac0 con 0x7fa2c00ffc20 2024-04-23T02:35:10.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.042+0000 7fa2c6471700 1 -- 172.21.15.71:0/1676832004 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c01080f0 con 0x7fa2c00ffc20 2024-04-23T02:35:10.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.043+0000 7fa2bd7fa700 1 -- 172.21.15.71:0/1676832004 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa2b0003a60 con 0x7fa2c00ffc20 2024-04-23T02:35:10.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.043+0000 7fa2bd7fa700 1 --2- 172.21.15.71:0/1676832004 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa2a806cce0 0x7fa2a806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:10.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.043+0000 7fa2bd7fa700 1 -- 172.21.15.71:0/1676832004 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa2b008d790 con 0x7fa2c00ffc20 2024-04-23T02:35:10.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.043+0000 7fa2bf7fe700 1 --2- 172.21.15.71:0/1676832004 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa2a806cce0 0x7fa2a806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:10.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.043+0000 7fa2bf7fe700 1 --2- 172.21.15.71:0/1676832004 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa2a806cce0 0x7fa2a806f1a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4005fd0 tx=0x7fa2b4005e80 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:10.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.047+0000 7fa2bd7fa700 1 -- 172.21.15.71:0/1676832004 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa2b0057880 con 0x7fa2c00ffc20 2024-04-23T02:35:10.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.321+0000 7fa2c6471700 1 -- 172.21.15.71:0/1676832004 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa2c0107ad0 con 0x7fa2c00ffc20 2024-04-23T02:35:10.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.321+0000 7fa2bd7fa700 1 -- 172.21.15.71:0/1676832004 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fa2b005aea0 con 0x7fa2c00ffc20 2024-04-23T02:35:10.324 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:35:10.324 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:35:10.324 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:35:10.324 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:35:10.324 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:35:10.324 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:35:10.324 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:35:10.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.323+0000 7fa2c6471700 1 -- 172.21.15.71:0/1676832004 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa2a806cce0 msgr2=0x7fa2a806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:10.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.323+0000 7fa2c6471700 1 --2- 172.21.15.71:0/1676832004 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa2a806cce0 0x7fa2a806f1a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4005fd0 tx=0x7fa2b4005e80 comp rx=0 tx=0).stop 2024-04-23T02:35:10.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.323+0000 7fa2c6471700 1 -- 172.21.15.71:0/1676832004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2c00ffc20 msgr2=0x7fa2c01975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:10.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.323+0000 7fa2c6471700 1 --2- 172.21.15.71:0/1676832004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2c00ffc20 0x7fa2c01975e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0006010 tx=0x7fa2b000be20 comp rx=0 tx=0).stop 2024-04-23T02:35:10.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.323+0000 7fa2c6471700 1 -- 172.21.15.71:0/1676832004 shutdown_connections 2024-04-23T02:35:10.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.323+0000 7fa2c6471700 1 --2- 172.21.15.71:0/1676832004 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa2a806cce0 0x7fa2a806f1a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:10.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.323+0000 7fa2c6471700 1 --2- 172.21.15.71:0/1676832004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2c00ffc20 0x7fa2c01975e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:10.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.323+0000 7fa2c6471700 1 --2- 172.21.15.71:0/1676832004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2c0105bd0 0x7fa2c0197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:10.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.323+0000 7fa2c6471700 1 -- 172.21.15.71:0/1676832004 >> 172.21.15.71:0/1676832004 conn(0x7fa2c0075670 msgr2=0x7fa2c00fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:10.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.323+0000 7fa2c6471700 1 -- 172.21.15.71:0/1676832004 shutdown_connections 2024-04-23T02:35:10.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:10.323+0000 7fa2c6471700 1 -- 172.21.15.71:0/1676832004 wait complete. 2024-04-23T02:35:10.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:10 smithi192 ceph-mon[108617]: from='client.35018 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:35:10.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:10 smithi192 ceph-mon[108617]: from='client.35022 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:35:10.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:10 smithi192 ceph-mon[108617]: pgmap v655: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:10.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:10 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2056302918' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:35:10.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:10 smithi071 ceph-mon[144544]: from='client.35018 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:35:10.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:10 smithi071 ceph-mon[144544]: from='client.35022 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:35:10.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:10 smithi071 ceph-mon[144544]: pgmap v655: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:10.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:10 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2056302918' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:35:11.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:11 smithi192 ceph-mon[108617]: from='client.35034 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:35:11.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:11 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1676832004' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:35:11.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:11 smithi071 ceph-mon[144544]: from='client.35034 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:35:11.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:11 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1676832004' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:35:12.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:12 smithi192 ceph-mon[108617]: pgmap v656: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:12.632 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:12 smithi071 ceph-mon[144544]: pgmap v656: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:14.594 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:14 smithi071 ceph-mon[144544]: pgmap v657: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:14.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:14 smithi192 ceph-mon[108617]: pgmap v657: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:16.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:16 smithi192 ceph-mon[108617]: pgmap v658: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:16.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:16 smithi071 ceph-mon[144544]: pgmap v658: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:18.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:18 smithi071 ceph-mon[144544]: pgmap v659: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:18.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:18 smithi192 ceph-mon[108617]: pgmap v659: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:20.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:20 smithi071 ceph-mon[144544]: pgmap v660: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:20.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:35:20.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:20 smithi192 ceph-mon[108617]: pgmap v660: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:20.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:35:22.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:22 smithi071 ceph-mon[144544]: pgmap v661: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:22.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:22 smithi192 ceph-mon[108617]: pgmap v661: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:24.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:24 smithi071 ceph-mon[144544]: pgmap v662: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:24.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:24 smithi192 ceph-mon[108617]: pgmap v662: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:26.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:26 smithi071 ceph-mon[144544]: pgmap v663: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:26.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:26 smithi192 ceph-mon[108617]: pgmap v663: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:28.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:28 smithi071 ceph-mon[144544]: pgmap v664: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:28.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:28 smithi192 ceph-mon[108617]: pgmap v664: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:30.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:30 smithi071 ceph-mon[144544]: pgmap v665: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:30.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:30 smithi192 ceph-mon[108617]: pgmap v665: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:32.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:32 smithi071 ceph-mon[144544]: pgmap v666: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:32.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:32 smithi192 ceph-mon[108617]: pgmap v666: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:34.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:34 smithi071 ceph-mon[144544]: pgmap v667: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:34.742 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:34 smithi192 ceph-mon[108617]: pgmap v667: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:35.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:35:35.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:35:36.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:36 smithi071 ceph-mon[144544]: pgmap v668: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:36.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:36 smithi192 ceph-mon[108617]: pgmap v668: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:38.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:38 smithi071 ceph-mon[144544]: pgmap v669: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:38.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:38 smithi192 ceph-mon[108617]: pgmap v669: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:40.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.439+0000 7f8b8ce1a700 1 -- 172.21.15.71:0/3982667052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b880ff5a0 msgr2=0x7f8b880ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:40.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.439+0000 7f8b8ce1a700 1 --2- 172.21.15.71:0/3982667052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b880ff5a0 0x7f8b880ff980 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009910 tx=0x7f8b7c009c20 comp rx=0 tx=0).stop 2024-04-23T02:35:40.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.439+0000 7f8b8ce1a700 1 -- 172.21.15.71:0/3982667052 shutdown_connections 2024-04-23T02:35:40.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.439+0000 7f8b8ce1a700 1 --2- 172.21.15.71:0/3982667052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b880fff50 0x7f8b88105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:40.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.439+0000 7f8b8ce1a700 1 --2- 172.21.15.71:0/3982667052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b880ff5a0 0x7f8b880ff980 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:40.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.439+0000 7f8b8ce1a700 1 -- 172.21.15.71:0/3982667052 >> 172.21.15.71:0/3982667052 conn(0x7f8b88074e30 msgr2=0x7f8b88075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:40.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.439+0000 7f8b8ce1a700 1 -- 172.21.15.71:0/3982667052 shutdown_connections 2024-04-23T02:35:40.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.439+0000 7f8b8ce1a700 1 -- 172.21.15.71:0/3982667052 wait complete. 2024-04-23T02:35:40.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.440+0000 7f8b8ce1a700 1 Processor -- start 2024-04-23T02:35:40.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.440+0000 7f8b8ce1a700 1 -- start start 2024-04-23T02:35:40.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.440+0000 7f8b8ce1a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b880ff5a0 0x7f8b8819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:40.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.440+0000 7f8b8ce1a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b880fff50 0x7f8b8819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:40.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.440+0000 7f8b8ce1a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8819d170 con 0x7f8b880ff5a0 2024-04-23T02:35:40.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.440+0000 7f8b8ce1a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b88196620 con 0x7f8b880fff50 2024-04-23T02:35:40.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.441+0000 7f8b8659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b880ff5a0 0x7f8b8819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:40.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.441+0000 7f8b8659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b880ff5a0 0x7f8b8819c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60584/0 (socket says 172.21.15.71:60584) 2024-04-23T02:35:40.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.441+0000 7f8b8659c700 1 -- 172.21.15.71:0/2098436360 learned_addr learned my addr 172.21.15.71:0/2098436360 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:35:40.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.441+0000 7f8b85d9b700 1 --2- 172.21.15.71:0/2098436360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b880fff50 0x7f8b8819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:40.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.441+0000 7f8b8659c700 1 -- 172.21.15.71:0/2098436360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b880fff50 msgr2=0x7f8b8819cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:40.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.441+0000 7f8b8659c700 1 --2- 172.21.15.71:0/2098436360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b880fff50 0x7f8b8819cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:40.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.441+0000 7f8b8659c700 1 -- 172.21.15.71:0/2098436360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c0095f0 con 0x7f8b880ff5a0 2024-04-23T02:35:40.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.441+0000 7f8b85d9b700 1 --2- 172.21.15.71:0/2098436360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b880fff50 0x7f8b8819cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:35:40.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.441+0000 7f8b8659c700 1 --2- 172.21.15.71:0/2098436360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b880ff5a0 0x7f8b8819c5a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c006010 tx=0x7f8b7c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:40.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.441+0000 7f8b777fe700 1 -- 172.21.15.71:0/2098436360 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b7c01d070 con 0x7f8b880ff5a0 2024-04-23T02:35:40.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.441+0000 7f8b777fe700 1 -- 172.21.15.71:0/2098436360 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8b7c00f460 con 0x7f8b880ff5a0 2024-04-23T02:35:40.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.441+0000 7f8b8ce1a700 1 -- 172.21.15.71:0/2098436360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b881968a0 con 0x7f8b880ff5a0 2024-04-23T02:35:40.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.441+0000 7f8b777fe700 1 -- 172.21.15.71:0/2098436360 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b7c021620 con 0x7f8b880ff5a0 2024-04-23T02:35:40.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.441+0000 7f8b8ce1a700 1 -- 172.21.15.71:0/2098436360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b88196d90 con 0x7f8b880ff5a0 2024-04-23T02:35:40.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.442+0000 7f8b8ce1a700 1 -- 172.21.15.71:0/2098436360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b88109270 con 0x7f8b880ff5a0 2024-04-23T02:35:40.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.443+0000 7f8b777fe700 1 -- 172.21.15.71:0/2098436360 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8b7c00fa40 con 0x7f8b880ff5a0 2024-04-23T02:35:40.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.443+0000 7f8b777fe700 1 --2- 172.21.15.71:0/2098436360 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8b7006cce0 0x7f8b7006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:40.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.443+0000 7f8b777fe700 1 -- 172.21.15.71:0/2098436360 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8b7c08d790 con 0x7f8b880ff5a0 2024-04-23T02:35:40.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.443+0000 7f8b85d9b700 1 --2- 172.21.15.71:0/2098436360 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8b7006cce0 0x7f8b7006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:40.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.444+0000 7f8b85d9b700 1 --2- 172.21.15.71:0/2098436360 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8b7006cce0 0x7f8b7006f1a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8b78005950 tx=0x7f8b780058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:40.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.448+0000 7f8b777fe700 1 -- 172.21.15.71:0/2098436360 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8b7c05b290 con 0x7f8b880ff5a0 2024-04-23T02:35:40.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.657+0000 7f8b8ce1a700 1 -- 172.21.15.71:0/2098436360 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b88197c10 con 0x7f8b7006cce0 2024-04-23T02:35:40.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.658+0000 7f8b777fe700 1 -- 172.21.15.71:0/2098436360 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8b88197c10 con 0x7f8b7006cce0 2024-04-23T02:35:40.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.660+0000 7f8b8ce1a700 1 -- 172.21.15.71:0/2098436360 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8b7006cce0 msgr2=0x7f8b7006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:40.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.660+0000 7f8b8ce1a700 1 --2- 172.21.15.71:0/2098436360 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8b7006cce0 0x7f8b7006f1a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8b78005950 tx=0x7f8b780058e0 comp rx=0 tx=0).stop 2024-04-23T02:35:40.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.660+0000 7f8b8ce1a700 1 -- 172.21.15.71:0/2098436360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b880ff5a0 msgr2=0x7f8b8819c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:40.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.660+0000 7f8b8ce1a700 1 --2- 172.21.15.71:0/2098436360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b880ff5a0 0x7f8b8819c5a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c006010 tx=0x7f8b7c00be20 comp rx=0 tx=0).stop 2024-04-23T02:35:40.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.660+0000 7f8b8ce1a700 1 -- 172.21.15.71:0/2098436360 shutdown_connections 2024-04-23T02:35:40.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.660+0000 7f8b8ce1a700 1 --2- 172.21.15.71:0/2098436360 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8b7006cce0 0x7f8b7006f1a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:40.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.660+0000 7f8b8ce1a700 1 --2- 172.21.15.71:0/2098436360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b880ff5a0 0x7f8b8819c5a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:40.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.660+0000 7f8b8ce1a700 1 --2- 172.21.15.71:0/2098436360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b880fff50 0x7f8b8819cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:40.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.660+0000 7f8b8ce1a700 1 -- 172.21.15.71:0/2098436360 >> 172.21.15.71:0/2098436360 conn(0x7f8b88074e30 msgr2=0x7f8b880fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:40.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.660+0000 7f8b8ce1a700 1 -- 172.21.15.71:0/2098436360 shutdown_connections 2024-04-23T02:35:40.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.660+0000 7f8b8ce1a700 1 -- 172.21.15.71:0/2098436360 wait complete. 2024-04-23T02:35:40.673 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:35:40.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:40 smithi071 ceph-mon[144544]: pgmap v670: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:40.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.764+0000 7fe340518700 1 -- 172.21.15.71:0/1736484555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3380ffda0 msgr2=0x7fe338100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:40.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.764+0000 7fe340518700 1 --2- 172.21.15.71:0/1736484555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3380ffda0 0x7fe338100180 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009910 tx=0x7fe32c009c20 comp rx=0 tx=0).stop 2024-04-23T02:35:40.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.764+0000 7fe340518700 1 -- 172.21.15.71:0/1736484555 shutdown_connections 2024-04-23T02:35:40.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.764+0000 7fe340518700 1 --2- 172.21.15.71:0/1736484555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe338100750 0x7fe3381047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:40.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.764+0000 7fe340518700 1 --2- 172.21.15.71:0/1736484555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3380ffda0 0x7fe338100180 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:40.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.764+0000 7fe340518700 1 -- 172.21.15.71:0/1736484555 >> 172.21.15.71:0/1736484555 conn(0x7fe3380fb650 msgr2=0x7fe3380fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:40.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.764+0000 7fe340518700 1 -- 172.21.15.71:0/1736484555 shutdown_connections 2024-04-23T02:35:40.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.764+0000 7fe340518700 1 -- 172.21.15.71:0/1736484555 wait complete. 2024-04-23T02:35:40.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.765+0000 7fe340518700 1 Processor -- start 2024-04-23T02:35:40.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.765+0000 7fe340518700 1 -- start start 2024-04-23T02:35:40.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.765+0000 7fe340518700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3380ffda0 0x7fe338197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:40.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.765+0000 7fe340518700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe338100750 0x7fe338197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:40.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.765+0000 7fe340518700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe338198190 con 0x7fe338100750 2024-04-23T02:35:40.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.765+0000 7fe340518700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe338198300 con 0x7fe3380ffda0 2024-04-23T02:35:40.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.765+0000 7fe33dab3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe338100750 0x7fe338197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:40.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.765+0000 7fe33dab3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe338100750 0x7fe338197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60588/0 (socket says 172.21.15.71:60588) 2024-04-23T02:35:40.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.765+0000 7fe33dab3700 1 -- 172.21.15.71:0/3969790272 learned_addr learned my addr 172.21.15.71:0/3969790272 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:35:40.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.766+0000 7fe33e2b4700 1 --2- 172.21.15.71:0/3969790272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3380ffda0 0x7fe338197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:40.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.766+0000 7fe33dab3700 1 -- 172.21.15.71:0/3969790272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3380ffda0 msgr2=0x7fe338197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:40.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.766+0000 7fe33dab3700 1 --2- 172.21.15.71:0/3969790272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3380ffda0 0x7fe338197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:40.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.766+0000 7fe33dab3700 1 -- 172.21.15.71:0/3969790272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c0095f0 con 0x7fe338100750 2024-04-23T02:35:40.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.766+0000 7fe33e2b4700 1 --2- 172.21.15.71:0/3969790272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3380ffda0 0x7fe338197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:35:40.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.766+0000 7fe33dab3700 1 --2- 172.21.15.71:0/3969790272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe338100750 0x7fe338197b70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe33400d8d0 tx=0x7fe33400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:40.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.766+0000 7fe32b7fe700 1 -- 172.21.15.71:0/3969790272 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe334004a00 con 0x7fe338100750 2024-04-23T02:35:40.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.766+0000 7fe340518700 1 -- 172.21.15.71:0/3969790272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3381a06a0 con 0x7fe338100750 2024-04-23T02:35:40.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.766+0000 7fe32b7fe700 1 -- 172.21.15.71:0/3969790272 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe334005020 con 0x7fe338100750 2024-04-23T02:35:40.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.766+0000 7fe32b7fe700 1 -- 172.21.15.71:0/3969790272 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe334009bd0 con 0x7fe338100750 2024-04-23T02:35:40.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.766+0000 7fe340518700 1 -- 172.21.15.71:0/3969790272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3381a0b90 con 0x7fe338100750 2024-04-23T02:35:40.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.767+0000 7fe340518700 1 -- 172.21.15.71:0/3969790272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe338108140 con 0x7fe338100750 2024-04-23T02:35:40.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.767+0000 7fe32b7fe700 1 -- 172.21.15.71:0/3969790272 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe334005190 con 0x7fe338100750 2024-04-23T02:35:40.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.768+0000 7fe32b7fe700 1 --2- 172.21.15.71:0/3969790272 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe32406cce0 0x7fe32406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:40.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.768+0000 7fe32b7fe700 1 -- 172.21.15.71:0/3969790272 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe33408bef0 con 0x7fe338100750 2024-04-23T02:35:40.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.768+0000 7fe33e2b4700 1 --2- 172.21.15.71:0/3969790272 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe32406cce0 0x7fe32406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:40.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.768+0000 7fe33e2b4700 1 --2- 172.21.15.71:0/3969790272 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe32406cce0 0x7fe32406f1a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00b5c0 tx=0x7fe32c005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.773+0000 7fe32b7fe700 1 -- 172.21.15.71:0/3969790272 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe334055f60 con 0x7fe338100750 2024-04-23T02:35:40.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:40 smithi192 ceph-mon[108617]: pgmap v670: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:40.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.980+0000 7fe340518700 1 -- 172.21.15.71:0/3969790272 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe33802ccc0 con 0x7fe32406cce0 2024-04-23T02:35:40.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.981+0000 7fe32b7fe700 1 -- 172.21.15.71:0/3969790272 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe33802ccc0 con 0x7fe32406cce0 2024-04-23T02:35:40.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.983+0000 7fe340518700 1 -- 172.21.15.71:0/3969790272 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe32406cce0 msgr2=0x7fe32406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:40.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.983+0000 7fe340518700 1 --2- 172.21.15.71:0/3969790272 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe32406cce0 0x7fe32406f1a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00b5c0 tx=0x7fe32c005960 comp rx=0 tx=0).stop 2024-04-23T02:35:40.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.983+0000 7fe340518700 1 -- 172.21.15.71:0/3969790272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe338100750 msgr2=0x7fe338197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:40.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.983+0000 7fe340518700 1 --2- 172.21.15.71:0/3969790272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe338100750 0x7fe338197b70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe33400d8d0 tx=0x7fe33400dc90 comp rx=0 tx=0).stop 2024-04-23T02:35:40.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.983+0000 7fe340518700 1 -- 172.21.15.71:0/3969790272 shutdown_connections 2024-04-23T02:35:40.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.983+0000 7fe340518700 1 --2- 172.21.15.71:0/3969790272 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe32406cce0 0x7fe32406f1a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:40.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.983+0000 7fe340518700 1 --2- 172.21.15.71:0/3969790272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3380ffda0 0x7fe338197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:40.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.983+0000 7fe340518700 1 --2- 172.21.15.71:0/3969790272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe338100750 0x7fe338197b70 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:40.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.983+0000 7fe340518700 1 -- 172.21.15.71:0/3969790272 >> 172.21.15.71:0/3969790272 conn(0x7fe3380fb650 msgr2=0x7fe3380fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:40.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.983+0000 7fe340518700 1 -- 172.21.15.71:0/3969790272 shutdown_connections 2024-04-23T02:35:40.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:40.983+0000 7fe340518700 1 -- 172.21.15.71:0/3969790272 wait complete. 2024-04-23T02:35:41.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.084+0000 7fbbfded9700 1 -- 172.21.15.71:0/1489168014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbf80ffda0 msgr2=0x7fbbf8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:41.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.084+0000 7fbbfded9700 1 --2- 172.21.15.71:0/1489168014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbf80ffda0 0x7fbbf8100180 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009910 tx=0x7fbbe8009c20 comp rx=0 tx=0).stop 2024-04-23T02:35:41.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.084+0000 7fbbfded9700 1 -- 172.21.15.71:0/1489168014 shutdown_connections 2024-04-23T02:35:41.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.084+0000 7fbbfded9700 1 --2- 172.21.15.71:0/1489168014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf8100750 0x7fbbf81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:41.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.084+0000 7fbbfded9700 1 --2- 172.21.15.71:0/1489168014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbf80ffda0 0x7fbbf8100180 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:41.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.084+0000 7fbbfded9700 1 -- 172.21.15.71:0/1489168014 >> 172.21.15.71:0/1489168014 conn(0x7fbbf80fb650 msgr2=0x7fbbf80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:41.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.084+0000 7fbbfded9700 1 -- 172.21.15.71:0/1489168014 shutdown_connections 2024-04-23T02:35:41.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.085+0000 7fbbfded9700 1 -- 172.21.15.71:0/1489168014 wait complete. 2024-04-23T02:35:41.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.085+0000 7fbbfded9700 1 Processor -- start 2024-04-23T02:35:41.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.085+0000 7fbbfded9700 1 -- start start 2024-04-23T02:35:41.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.085+0000 7fbbfded9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf80ffda0 0x7fbbf819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:41.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.085+0000 7fbbfded9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbf8100750 0x7fbbf819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:41.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.085+0000 7fbbfded9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf819f370 con 0x7fbbf8100750 2024-04-23T02:35:41.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.085+0000 7fbbfded9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf8198820 con 0x7fbbf80ffda0 2024-04-23T02:35:41.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.086+0000 7fbbf6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbf8100750 0x7fbbf819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:41.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.086+0000 7fbbf6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbf8100750 0x7fbbf819ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60596/0 (socket says 172.21.15.71:60596) 2024-04-23T02:35:41.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.086+0000 7fbbf6ffd700 1 -- 172.21.15.71:0/3670799485 learned_addr learned my addr 172.21.15.71:0/3670799485 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:35:41.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.086+0000 7fbbf77fe700 1 --2- 172.21.15.71:0/3670799485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf80ffda0 0x7fbbf819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:41.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.086+0000 7fbbf6ffd700 1 -- 172.21.15.71:0/3670799485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf80ffda0 msgr2=0x7fbbf819e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:41.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.086+0000 7fbbf6ffd700 1 --2- 172.21.15.71:0/3670799485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf80ffda0 0x7fbbf819e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:41.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.086+0000 7fbbf6ffd700 1 -- 172.21.15.71:0/3670799485 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe80095f0 con 0x7fbbf8100750 2024-04-23T02:35:41.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.086+0000 7fbbf77fe700 1 --2- 172.21.15.71:0/3670799485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf80ffda0 0x7fbbf819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:35:41.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.086+0000 7fbbf6ffd700 1 --2- 172.21.15.71:0/3670799485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbf8100750 0x7fbbf819ece0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00e8f0 tx=0x7fbbec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:41.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.086+0000 7fbbf4ff9700 1 -- 172.21.15.71:0/3670799485 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbec009790 con 0x7fbbf8100750 2024-04-23T02:35:41.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.086+0000 7fbbfded9700 1 -- 172.21.15.71:0/3670799485 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf8198b00 con 0x7fbbf8100750 2024-04-23T02:35:41.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.086+0000 7fbbf4ff9700 1 -- 172.21.15.71:0/3670799485 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbbec004cd0 con 0x7fbbf8100750 2024-04-23T02:35:41.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.086+0000 7fbbf4ff9700 1 -- 172.21.15.71:0/3670799485 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbec005580 con 0x7fbbf8100750 2024-04-23T02:35:41.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.086+0000 7fbbfded9700 1 -- 172.21.15.71:0/3670799485 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf8199050 con 0x7fbbf8100750 2024-04-23T02:35:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.087+0000 7fbbfded9700 1 -- 172.21.15.71:0/3670799485 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf804f2e0 con 0x7fbbf8100750 2024-04-23T02:35:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.088+0000 7fbbf4ff9700 1 -- 172.21.15.71:0/3670799485 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbbec004750 con 0x7fbbf8100750 2024-04-23T02:35:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.088+0000 7fbbf4ff9700 1 --2- 172.21.15.71:0/3670799485 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbbe006cce0 0x7fbbe006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.088+0000 7fbbf4ff9700 1 -- 172.21.15.71:0/3670799485 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbbec014070 con 0x7fbbf8100750 2024-04-23T02:35:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.088+0000 7fbbf77fe700 1 --2- 172.21.15.71:0/3670799485 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbbe006cce0 0x7fbbe006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:41.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.089+0000 7fbbf77fe700 1 --2- 172.21.15.71:0/3670799485 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbbe006cce0 0x7fbbe006f1a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800b6e0 tx=0x7fbbe8009f70 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:41.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.093+0000 7fbbf4ff9700 1 -- 172.21.15.71:0/3670799485 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbbec05a450 con 0x7fbbf8100750 2024-04-23T02:35:41.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.296+0000 7fbbfded9700 1 -- 172.21.15.71:0/3670799485 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbf8199340 con 0x7fbbe006cce0 2024-04-23T02:35:41.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.307+0000 7fbbf4ff9700 1 -- 172.21.15.71:0/3670799485 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbbf8199340 con 0x7fbbe006cce0 2024-04-23T02:35:41.309 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:35:41.309 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (19m) 13m ago 30m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:35:41.309 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (30m) 13m ago 30m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:35:41.309 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (29m) 12m ago 29m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:35:41.309 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (17m) 13m ago 30m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:35:41.309 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (17m) 12m ago 29m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:35:41.310 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (19m) 13m ago 30m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:35:41.310 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (13m) 13m ago 24m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:35:41.310 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (13m) 13m ago 24m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:35:41.310 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (24m) 12m ago 24m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:35:41.310 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (12m) 12m ago 24m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:35:41.310 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (21m) 13m ago 32m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:35:41.310 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (20m) 12m ago 29m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:35:41.310 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (17m) 13m ago 32m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:35:41.310 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (17m) 12m ago 29m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:35:41.310 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (20m) 13m ago 30m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:35:41.310 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (20m) 12m ago 29m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:35:41.310 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (16m) 13m ago 28m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:35:41.311 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (16m) 13m ago 28m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:35:41.311 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (15m) 13m ago 27m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:35:41.311 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (15m) 12m ago 27m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:35:41.311 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (14m) 12m ago 26m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:35:41.311 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (13m) 12m ago 26m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:35:41.311 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (19m) 13m ago 30m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:35:41.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.309+0000 7fbbfded9700 1 -- 172.21.15.71:0/3670799485 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbbe006cce0 msgr2=0x7fbbe006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:41.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.309+0000 7fbbfded9700 1 --2- 172.21.15.71:0/3670799485 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbbe006cce0 0x7fbbe006f1a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800b6e0 tx=0x7fbbe8009f70 comp rx=0 tx=0).stop 2024-04-23T02:35:41.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.309+0000 7fbbfded9700 1 -- 172.21.15.71:0/3670799485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbf8100750 msgr2=0x7fbbf819ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:41.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.309+0000 7fbbfded9700 1 --2- 172.21.15.71:0/3670799485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbf8100750 0x7fbbf819ece0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00e8f0 tx=0x7fbbec00ecb0 comp rx=0 tx=0).stop 2024-04-23T02:35:41.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.309+0000 7fbbfded9700 1 -- 172.21.15.71:0/3670799485 shutdown_connections 2024-04-23T02:35:41.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.309+0000 7fbbfded9700 1 --2- 172.21.15.71:0/3670799485 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbbe006cce0 0x7fbbe006f1a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:41.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.309+0000 7fbbfded9700 1 --2- 172.21.15.71:0/3670799485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbf80ffda0 0x7fbbf819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:41.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.309+0000 7fbbfded9700 1 --2- 172.21.15.71:0/3670799485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbf8100750 0x7fbbf819ece0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:41.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.309+0000 7fbbfded9700 1 -- 172.21.15.71:0/3670799485 >> 172.21.15.71:0/3670799485 conn(0x7fbbf80fb650 msgr2=0x7fbbf8103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:41.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.309+0000 7fbbfded9700 1 -- 172.21.15.71:0/3670799485 shutdown_connections 2024-04-23T02:35:41.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.309+0000 7fbbfded9700 1 -- 172.21.15.71:0/3670799485 wait complete. 2024-04-23T02:35:41.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.410+0000 7fb39cce1700 1 -- 172.21.15.71:0/2791348192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3980ffb40 msgr2=0x7fb3980fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:41.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.410+0000 7fb39cce1700 1 --2- 172.21.15.71:0/2791348192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3980ffb40 0x7fb3980fffc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb388009910 tx=0x7fb388009c20 comp rx=0 tx=0).stop 2024-04-23T02:35:41.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.410+0000 7fb39cce1700 1 -- 172.21.15.71:0/2791348192 shutdown_connections 2024-04-23T02:35:41.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.410+0000 7fb39cce1700 1 --2- 172.21.15.71:0/2791348192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3980ffb40 0x7fb3980fffc0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:41.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.410+0000 7fb39cce1700 1 --2- 172.21.15.71:0/2791348192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb398105bf0 0x7fb398105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:41.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.410+0000 7fb39cce1700 1 -- 172.21.15.71:0/2791348192 >> 172.21.15.71:0/2791348192 conn(0x7fb3980fb5f0 msgr2=0x7fb3980fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:41.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.410+0000 7fb39cce1700 1 -- 172.21.15.71:0/2791348192 shutdown_connections 2024-04-23T02:35:41.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.410+0000 7fb39cce1700 1 -- 172.21.15.71:0/2791348192 wait complete. 2024-04-23T02:35:41.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.411+0000 7fb39cce1700 1 Processor -- start 2024-04-23T02:35:41.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.411+0000 7fb39cce1700 1 -- start start 2024-04-23T02:35:41.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.411+0000 7fb39cce1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3980ffb40 0x7fb398197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:41.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.411+0000 7fb39cce1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb398105bf0 0x7fb398197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:41.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.411+0000 7fb39cce1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3981980f0 con 0x7fb398105bf0 2024-04-23T02:35:41.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.411+0000 7fb39cce1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb398198260 con 0x7fb3980ffb40 2024-04-23T02:35:41.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.411+0000 7fb395d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb398105bf0 0x7fb398197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:41.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.412+0000 7fb395d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb398105bf0 0x7fb398197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60610/0 (socket says 172.21.15.71:60610) 2024-04-23T02:35:41.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.412+0000 7fb395d9b700 1 -- 172.21.15.71:0/1375685844 learned_addr learned my addr 172.21.15.71:0/1375685844 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:35:41.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.412+0000 7fb39659c700 1 --2- 172.21.15.71:0/1375685844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3980ffb40 0x7fb398197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:41.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.412+0000 7fb395d9b700 1 -- 172.21.15.71:0/1375685844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3980ffb40 msgr2=0x7fb398197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:41.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.412+0000 7fb395d9b700 1 --2- 172.21.15.71:0/1375685844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3980ffb40 0x7fb398197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:41.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.412+0000 7fb395d9b700 1 -- 172.21.15.71:0/1375685844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3880095f0 con 0x7fb398105bf0 2024-04-23T02:35:41.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.412+0000 7fb39659c700 1 --2- 172.21.15.71:0/1375685844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3980ffb40 0x7fb398197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:35:41.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.412+0000 7fb395d9b700 1 --2- 172.21.15.71:0/1375685844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb398105bf0 0x7fb398197ad0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb388006010 tx=0x7fb38800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:41.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.412+0000 7fb3877fe700 1 -- 172.21.15.71:0/1375685844 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb38801d070 con 0x7fb398105bf0 2024-04-23T02:35:41.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.412+0000 7fb3877fe700 1 -- 172.21.15.71:0/1375685844 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb38800f460 con 0x7fb398105bf0 2024-04-23T02:35:41.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.412+0000 7fb39cce1700 1 -- 172.21.15.71:0/1375685844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3981a0600 con 0x7fb398105bf0 2024-04-23T02:35:41.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.412+0000 7fb3877fe700 1 -- 172.21.15.71:0/1375685844 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb388021640 con 0x7fb398105bf0 2024-04-23T02:35:41.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.412+0000 7fb39cce1700 1 -- 172.21.15.71:0/1375685844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3981a0a70 con 0x7fb398105bf0 2024-04-23T02:35:41.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.413+0000 7fb39cce1700 1 -- 172.21.15.71:0/1375685844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3981080a0 con 0x7fb398105bf0 2024-04-23T02:35:41.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.413+0000 7fb3877fe700 1 -- 172.21.15.71:0/1375685844 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb38800fa60 con 0x7fb398105bf0 2024-04-23T02:35:41.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.414+0000 7fb3877fe700 1 --2- 172.21.15.71:0/1375685844 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb38006cce0 0x7fb38006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:41.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.414+0000 7fb3877fe700 1 -- 172.21.15.71:0/1375685844 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb38808d860 con 0x7fb398105bf0 2024-04-23T02:35:41.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.414+0000 7fb39659c700 1 --2- 172.21.15.71:0/1375685844 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb38006cce0 0x7fb38006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:41.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.414+0000 7fb39659c700 1 --2- 172.21.15.71:0/1375685844 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb38006cce0 0x7fb38006f1a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb38c005fd0 tx=0x7fb38c005dc0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:41.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.418+0000 7fb3877fe700 1 -- 172.21.15.71:0/1375685844 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb3880578d0 con 0x7fb398105bf0 2024-04-23T02:35:41.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.691+0000 7fb39cce1700 1 -- 172.21.15.71:0/1375685844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb39802cc70 con 0x7fb398105bf0 2024-04-23T02:35:41.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.691+0000 7fb3877fe700 1 -- 172.21.15.71:0/1375685844 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7fb38805aef0 con 0x7fb398105bf0 2024-04-23T02:35:41.694 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:35:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:35:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:35:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:35:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:35:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:35:41.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:35:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:35:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:35:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:35:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:35:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:35:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:35:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:35:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:35:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:35:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:35:41.695 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:35:41.695 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:35:41.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.693+0000 7fb39cce1700 1 -- 172.21.15.71:0/1375685844 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb38006cce0 msgr2=0x7fb38006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:41.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.693+0000 7fb39cce1700 1 --2- 172.21.15.71:0/1375685844 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb38006cce0 0x7fb38006f1a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb38c005fd0 tx=0x7fb38c005dc0 comp rx=0 tx=0).stop 2024-04-23T02:35:41.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.693+0000 7fb39cce1700 1 -- 172.21.15.71:0/1375685844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb398105bf0 msgr2=0x7fb398197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:41.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.693+0000 7fb39cce1700 1 --2- 172.21.15.71:0/1375685844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb398105bf0 0x7fb398197ad0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb388006010 tx=0x7fb38800be20 comp rx=0 tx=0).stop 2024-04-23T02:35:41.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.693+0000 7fb39cce1700 1 -- 172.21.15.71:0/1375685844 shutdown_connections 2024-04-23T02:35:41.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.693+0000 7fb39cce1700 1 --2- 172.21.15.71:0/1375685844 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb38006cce0 0x7fb38006f1a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:41.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.693+0000 7fb39cce1700 1 --2- 172.21.15.71:0/1375685844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3980ffb40 0x7fb398197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:41.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.693+0000 7fb39cce1700 1 --2- 172.21.15.71:0/1375685844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb398105bf0 0x7fb398197ad0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:41.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.693+0000 7fb39cce1700 1 -- 172.21.15.71:0/1375685844 >> 172.21.15.71:0/1375685844 conn(0x7fb3980fb5f0 msgr2=0x7fb3980fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:41.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.693+0000 7fb39cce1700 1 -- 172.21.15.71:0/1375685844 shutdown_connections 2024-04-23T02:35:41.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.693+0000 7fb39cce1700 1 -- 172.21.15.71:0/1375685844 wait complete. 2024-04-23T02:35:41.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.796+0000 7fd7d94bb700 1 -- 172.21.15.71:0/3799162987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7d41006b0 msgr2=0x7fd7d4100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:41.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.796+0000 7fd7d94bb700 1 --2- 172.21.15.71:0/3799162987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7d41006b0 0x7fd7d4100a90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009910 tx=0x7fd7c8009c20 comp rx=0 tx=0).stop 2024-04-23T02:35:41.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.796+0000 7fd7d94bb700 1 -- 172.21.15.71:0/3799162987 shutdown_connections 2024-04-23T02:35:41.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.796+0000 7fd7d94bb700 1 --2- 172.21.15.71:0/3799162987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7d4101060 0x7fd7d41047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:41.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.796+0000 7fd7d94bb700 1 --2- 172.21.15.71:0/3799162987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7d41006b0 0x7fd7d4100a90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:41.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.796+0000 7fd7d94bb700 1 -- 172.21.15.71:0/3799162987 >> 172.21.15.71:0/3799162987 conn(0x7fd7d4074e30 msgr2=0x7fd7d4075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:41.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.796+0000 7fd7d94bb700 1 -- 172.21.15.71:0/3799162987 shutdown_connections 2024-04-23T02:35:41.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.796+0000 7fd7d94bb700 1 -- 172.21.15.71:0/3799162987 wait complete. 2024-04-23T02:35:41.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.796+0000 7fd7d94bb700 1 Processor -- start 2024-04-23T02:35:41.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7d94bb700 1 -- start start 2024-04-23T02:35:41.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7d94bb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7d41006b0 0x7fd7d4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:41.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7d94bb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7d4101060 0x7fd7d4198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:41.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7d94bb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d4199530 con 0x7fd7d41006b0 2024-04-23T02:35:41.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7d94bb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d419d340 con 0x7fd7d4101060 2024-04-23T02:35:41.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7d2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7d41006b0 0x7fd7d4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:41.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7d2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7d41006b0 0x7fd7d4198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60632/0 (socket says 172.21.15.71:60632) 2024-04-23T02:35:41.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7d2ffd700 1 -- 172.21.15.71:0/3332725601 learned_addr learned my addr 172.21.15.71:0/3332725601 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:35:41.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7d27fc700 1 --2- 172.21.15.71:0/3332725601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7d4101060 0x7fd7d4198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:41.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7d2ffd700 1 -- 172.21.15.71:0/3332725601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7d4101060 msgr2=0x7fd7d4198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:41.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7d2ffd700 1 --2- 172.21.15.71:0/3332725601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7d4101060 0x7fd7d4198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:41.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7d2ffd700 1 -- 172.21.15.71:0/3332725601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c80095f0 con 0x7fd7d41006b0 2024-04-23T02:35:41.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7d27fc700 1 --2- 172.21.15.71:0/3332725601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7d4101060 0x7fd7d4198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:35:41.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7d2ffd700 1 --2- 172.21.15.71:0/3332725601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7d41006b0 0x7fd7d4198960 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8006010 tx=0x7fd7c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:41.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7bbfff700 1 -- 172.21.15.71:0/3332725601 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7c801d070 con 0x7fd7d41006b0 2024-04-23T02:35:41.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7bbfff700 1 -- 172.21.15.71:0/3332725601 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd7c800f460 con 0x7fd7d41006b0 2024-04-23T02:35:41.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7d94bb700 1 -- 172.21.15.71:0/3332725601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d419d5c0 con 0x7fd7d41006b0 2024-04-23T02:35:41.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7bbfff700 1 -- 172.21.15.71:0/3332725601 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7c8021640 con 0x7fd7d41006b0 2024-04-23T02:35:41.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.797+0000 7fd7d94bb700 1 -- 172.21.15.71:0/3332725601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d419dab0 con 0x7fd7d41006b0 2024-04-23T02:35:41.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.798+0000 7fd7d94bb700 1 -- 172.21.15.71:0/3332725601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d404f2e0 con 0x7fd7d41006b0 2024-04-23T02:35:41.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.799+0000 7fd7bbfff700 1 -- 172.21.15.71:0/3332725601 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd7c800fa20 con 0x7fd7d41006b0 2024-04-23T02:35:41.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.799+0000 7fd7bbfff700 1 --2- 172.21.15.71:0/3332725601 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd7bc06cce0 0x7fd7bc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:41.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.799+0000 7fd7bbfff700 1 -- 172.21.15.71:0/3332725601 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd7c808d830 con 0x7fd7d41006b0 2024-04-23T02:35:41.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.799+0000 7fd7d27fc700 1 --2- 172.21.15.71:0/3332725601 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd7bc06cce0 0x7fd7bc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:41.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.800+0000 7fd7d27fc700 1 --2- 172.21.15.71:0/3332725601 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd7bc06cce0 0x7fd7bc06f1a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4005950 tx=0x7fd7c40058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:41.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:41.804+0000 7fd7bbfff700 1 -- 172.21.15.71:0/3332725601 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd7c805b330 con 0x7fd7d41006b0 2024-04-23T02:35:42.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.039+0000 7fd7d94bb700 1 -- 172.21.15.71:0/3332725601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd7d419dcd0 con 0x7fd7d41006b0 2024-04-23T02:35:42.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.039+0000 7fd7bbfff700 1 -- 172.21.15.71:0/3332725601 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7fd7c8057030 con 0x7fd7d41006b0 2024-04-23T02:35:42.042 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:35:42.042 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:35:42.042 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:35:42.042 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:35:42.042 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:35:42.042 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:35:42.042 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:35:42.042 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:35:42.042 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:35:42.042 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:35:42.042 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:35:42.042 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:35:42.043 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:35:42.043 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:35:42.043 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:35:42.043 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:35:42.043 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:35:42.043 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:35:42.043 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:35:42.043 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:35:42.043 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:35:42.043 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:35:42.043 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:35:42.043 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:35:42.043 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:35:42.043 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:35:42.043 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:35:42.043 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:35:42.044 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:35:42.044 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:35:42.044 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:35:42.044 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:35:42.044 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:35:42.044 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:35:42.044 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:35:42.044 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:35:42.044 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:35:42.044 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:35:42.044 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:35:42.044 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:35:42.044 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:35:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.041+0000 7fd7d94bb700 1 -- 172.21.15.71:0/3332725601 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd7bc06cce0 msgr2=0x7fd7bc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.041+0000 7fd7d94bb700 1 --2- 172.21.15.71:0/3332725601 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd7bc06cce0 0x7fd7bc06f1a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4005950 tx=0x7fd7c40058e0 comp rx=0 tx=0).stop 2024-04-23T02:35:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.041+0000 7fd7d94bb700 1 -- 172.21.15.71:0/3332725601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7d41006b0 msgr2=0x7fd7d4198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.041+0000 7fd7d94bb700 1 --2- 172.21.15.71:0/3332725601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7d41006b0 0x7fd7d4198960 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8006010 tx=0x7fd7c800be20 comp rx=0 tx=0).stop 2024-04-23T02:35:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.041+0000 7fd7d94bb700 1 -- 172.21.15.71:0/3332725601 shutdown_connections 2024-04-23T02:35:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.041+0000 7fd7d94bb700 1 --2- 172.21.15.71:0/3332725601 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd7bc06cce0 0x7fd7bc06f1a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.041+0000 7fd7d94bb700 1 --2- 172.21.15.71:0/3332725601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7d41006b0 0x7fd7d4198960 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:42.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.041+0000 7fd7d94bb700 1 --2- 172.21.15.71:0/3332725601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7d4101060 0x7fd7d4198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:42.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.041+0000 7fd7d94bb700 1 -- 172.21.15.71:0/3332725601 >> 172.21.15.71:0/3332725601 conn(0x7fd7d4074e30 msgr2=0x7fd7d40fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:42.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.041+0000 7fd7d94bb700 1 -- 172.21.15.71:0/3332725601 shutdown_connections 2024-04-23T02:35:42.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.041+0000 7fd7d94bb700 1 -- 172.21.15.71:0/3332725601 wait complete. 2024-04-23T02:35:42.046 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:35:42.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.144+0000 7f82d1431700 1 -- 172.21.15.71:0/2653765880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82cc105350 msgr2=0x7f82cc105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:42.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.144+0000 7f82d1431700 1 --2- 172.21.15.71:0/2653765880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82cc105350 0x7f82cc105730 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009910 tx=0x7f82bc009c20 comp rx=0 tx=0).stop 2024-04-23T02:35:42.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.144+0000 7f82d1431700 1 -- 172.21.15.71:0/2653765880 shutdown_connections 2024-04-23T02:35:42.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.144+0000 7f82d1431700 1 --2- 172.21.15.71:0/2653765880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82cc0ff340 0x7f82cc0ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:42.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.144+0000 7f82d1431700 1 --2- 172.21.15.71:0/2653765880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82cc105350 0x7f82cc105730 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:42.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.144+0000 7f82d1431700 1 -- 172.21.15.71:0/2653765880 >> 172.21.15.71:0/2653765880 conn(0x7f82cc074e30 msgr2=0x7f82cc075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:42.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.144+0000 7f82d1431700 1 -- 172.21.15.71:0/2653765880 shutdown_connections 2024-04-23T02:35:42.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.144+0000 7f82d1431700 1 -- 172.21.15.71:0/2653765880 wait complete. 2024-04-23T02:35:42.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.145+0000 7f82d1431700 1 Processor -- start 2024-04-23T02:35:42.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.145+0000 7f82d1431700 1 -- start start 2024-04-23T02:35:42.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.145+0000 7f82d1431700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82cc0ff340 0x7f82cc195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:42.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.145+0000 7f82d1431700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82cc105350 0x7f82cc195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:42.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.145+0000 7f82d1431700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc196280 con 0x7f82cc105350 2024-04-23T02:35:42.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.145+0000 7f82d1431700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc19a010 con 0x7f82cc0ff340 2024-04-23T02:35:42.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.145+0000 7f82ca7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82cc105350 0x7f82cc195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:42.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.145+0000 7f82ca7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82cc105350 0x7f82cc195ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60646/0 (socket says 172.21.15.71:60646) 2024-04-23T02:35:42.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.145+0000 7f82ca7fc700 1 -- 172.21.15.71:0/1644617209 learned_addr learned my addr 172.21.15.71:0/1644617209 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:35:42.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.145+0000 7f82caffd700 1 --2- 172.21.15.71:0/1644617209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82cc0ff340 0x7f82cc195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:42.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.145+0000 7f82ca7fc700 1 -- 172.21.15.71:0/1644617209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82cc0ff340 msgr2=0x7f82cc195660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:42.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.145+0000 7f82ca7fc700 1 --2- 172.21.15.71:0/1644617209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82cc0ff340 0x7f82cc195660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:42.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.145+0000 7f82ca7fc700 1 -- 172.21.15.71:0/1644617209 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82bc0095f0 con 0x7f82cc105350 2024-04-23T02:35:42.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.145+0000 7f82ca7fc700 1 --2- 172.21.15.71:0/1644617209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82cc105350 0x7f82cc195ba0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f82c000c8a0 tx=0x7f82c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:42.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.146+0000 7f82b3fff700 1 -- 172.21.15.71:0/1644617209 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82c000cea0 con 0x7f82cc105350 2024-04-23T02:35:42.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.146+0000 7f82b3fff700 1 -- 172.21.15.71:0/1644617209 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f82c0004830 con 0x7f82cc105350 2024-04-23T02:35:42.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.146+0000 7f82d1431700 1 -- 172.21.15.71:0/1644617209 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82cc19a2f0 con 0x7f82cc105350 2024-04-23T02:35:42.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.146+0000 7f82b3fff700 1 -- 172.21.15.71:0/1644617209 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82c0005500 con 0x7f82cc105350 2024-04-23T02:35:42.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.146+0000 7f82d1431700 1 -- 172.21.15.71:0/1644617209 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82cc19a840 con 0x7f82cc105350 2024-04-23T02:35:42.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.147+0000 7f82d1431700 1 -- 172.21.15.71:0/1644617209 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82cc04f2e0 con 0x7f82cc105350 2024-04-23T02:35:42.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.147+0000 7f82b3fff700 1 -- 172.21.15.71:0/1644617209 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f82c00049a0 con 0x7f82cc105350 2024-04-23T02:35:42.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.147+0000 7f82b3fff700 1 --2- 172.21.15.71:0/1644617209 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f82b406cce0 0x7f82b406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:42.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.148+0000 7f82b3fff700 1 -- 172.21.15.71:0/1644617209 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f82c008c250 con 0x7f82cc105350 2024-04-23T02:35:42.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.148+0000 7f82caffd700 1 --2- 172.21.15.71:0/1644617209 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f82b406cce0 0x7f82b406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:42.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.148+0000 7f82caffd700 1 --2- 172.21.15.71:0/1644617209 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f82b406cce0 0x7f82b406f1a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00b5c0 tx=0x7f82bc005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:42.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.152+0000 7f82b3fff700 1 -- 172.21.15.71:0/1644617209 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f82c00562c0 con 0x7f82cc105350 2024-04-23T02:35:42.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.361+0000 7f82d1431700 1 -- 172.21.15.71:0/1644617209 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f82cc19abd0 con 0x7f82b406cce0 2024-04-23T02:35:42.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.362+0000 7f82b3fff700 1 -- 172.21.15.71:0/1644617209 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f82cc19abd0 con 0x7f82b406cce0 2024-04-23T02:35:42.365 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:35:42.365 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:35:42.365 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:35:42.365 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:35:42.365 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:35:42.365 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:35:42.365 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:35:42.365 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:35:42.365 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:35:42.365 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:35:42.365 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:35:42.365 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:35:42.365 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:35:42.366 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:35:42.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.364+0000 7f82d1431700 1 -- 172.21.15.71:0/1644617209 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f82b406cce0 msgr2=0x7f82b406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:42.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.364+0000 7f82d1431700 1 --2- 172.21.15.71:0/1644617209 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f82b406cce0 0x7f82b406f1a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00b5c0 tx=0x7f82bc005fd0 comp rx=0 tx=0).stop 2024-04-23T02:35:42.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.364+0000 7f82d1431700 1 -- 172.21.15.71:0/1644617209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82cc105350 msgr2=0x7f82cc195ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:42.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.364+0000 7f82d1431700 1 --2- 172.21.15.71:0/1644617209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82cc105350 0x7f82cc195ba0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f82c000c8a0 tx=0x7f82c000cc60 comp rx=0 tx=0).stop 2024-04-23T02:35:42.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.364+0000 7f82d1431700 1 -- 172.21.15.71:0/1644617209 shutdown_connections 2024-04-23T02:35:42.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.364+0000 7f82d1431700 1 --2- 172.21.15.71:0/1644617209 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f82b406cce0 0x7f82b406f1a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:42.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.364+0000 7f82d1431700 1 --2- 172.21.15.71:0/1644617209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82cc0ff340 0x7f82cc195660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:42.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.364+0000 7f82d1431700 1 --2- 172.21.15.71:0/1644617209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82cc105350 0x7f82cc195ba0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:42.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.364+0000 7f82d1431700 1 -- 172.21.15.71:0/1644617209 >> 172.21.15.71:0/1644617209 conn(0x7f82cc074e30 msgr2=0x7f82cc1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:42.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.364+0000 7f82d1431700 1 -- 172.21.15.71:0/1644617209 shutdown_connections 2024-04-23T02:35:42.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.364+0000 7f82d1431700 1 -- 172.21.15.71:0/1644617209 wait complete. 2024-04-23T02:35:42.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.473+0000 7f6da0948700 1 -- 172.21.15.71:0/3279060342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d9c0ffb40 msgr2=0x7f6d9c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:42.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.473+0000 7f6da0948700 1 --2- 172.21.15.71:0/3279060342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d9c0ffb40 0x7f6d9c0fffc0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009910 tx=0x7f6d90009c20 comp rx=0 tx=0).stop 2024-04-23T02:35:42.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.473+0000 7f6da0948700 1 -- 172.21.15.71:0/3279060342 shutdown_connections 2024-04-23T02:35:42.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.473+0000 7f6da0948700 1 --2- 172.21.15.71:0/3279060342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d9c0ffb40 0x7f6d9c0fffc0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:42.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.473+0000 7f6da0948700 1 --2- 172.21.15.71:0/3279060342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d9c105bf0 0x7f6d9c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:42.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.473+0000 7f6da0948700 1 -- 172.21.15.71:0/3279060342 >> 172.21.15.71:0/3279060342 conn(0x7f6d9c0fb610 msgr2=0x7f6d9c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:42.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.473+0000 7f6da0948700 1 -- 172.21.15.71:0/3279060342 shutdown_connections 2024-04-23T02:35:42.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.473+0000 7f6da0948700 1 -- 172.21.15.71:0/3279060342 wait complete. 2024-04-23T02:35:42.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.474+0000 7f6da0948700 1 Processor -- start 2024-04-23T02:35:42.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.474+0000 7f6da0948700 1 -- start start 2024-04-23T02:35:42.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.474+0000 7f6da0948700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d9c0ffb40 0x7f6d9c197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:42.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.474+0000 7f6da0948700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d9c105bf0 0x7f6d9c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:42.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.474+0000 7f6da0948700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c1980d0 con 0x7f6d9c105bf0 2024-04-23T02:35:42.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.474+0000 7f6da0948700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c198240 con 0x7f6d9c0ffb40 2024-04-23T02:35:42.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.474+0000 7f6d99d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d9c105bf0 0x7f6d9c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:42.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.474+0000 7f6d99d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d9c105bf0 0x7f6d9c197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60664/0 (socket says 172.21.15.71:60664) 2024-04-23T02:35:42.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.474+0000 7f6d99d9b700 1 -- 172.21.15.71:0/4084959086 learned_addr learned my addr 172.21.15.71:0/4084959086 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:35:42.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.475+0000 7f6d9a59c700 1 --2- 172.21.15.71:0/4084959086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d9c0ffb40 0x7f6d9c197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:42.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.475+0000 7f6d99d9b700 1 -- 172.21.15.71:0/4084959086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d9c0ffb40 msgr2=0x7f6d9c197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.475+0000 7f6d99d9b700 1 --2- 172.21.15.71:0/4084959086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d9c0ffb40 0x7f6d9c197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.475+0000 7f6d99d9b700 1 -- 172.21.15.71:0/4084959086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d900095f0 con 0x7f6d9c105bf0 2024-04-23T02:35:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.475+0000 7f6d99d9b700 1 --2- 172.21.15.71:0/4084959086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d9c105bf0 0x7f6d9c197ab0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000b5c0 tx=0x7f6d9000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.475+0000 7f6d8b7fe700 1 -- 172.21.15.71:0/4084959086 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d9001d070 con 0x7f6d9c105bf0 2024-04-23T02:35:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.475+0000 7f6d8b7fe700 1 -- 172.21.15.71:0/4084959086 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6d90005e00 con 0x7f6d9c105bf0 2024-04-23T02:35:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.475+0000 7f6da0948700 1 -- 172.21.15.71:0/4084959086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9c1a05e0 con 0x7f6d9c105bf0 2024-04-23T02:35:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.475+0000 7f6d8b7fe700 1 -- 172.21.15.71:0/4084959086 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d9000fe60 con 0x7f6d9c105bf0 2024-04-23T02:35:42.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.475+0000 7f6da0948700 1 -- 172.21.15.71:0/4084959086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9c1a0a50 con 0x7f6d9c105bf0 2024-04-23T02:35:42.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.476+0000 7f6da0948700 1 -- 172.21.15.71:0/4084959086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d9c1080a0 con 0x7f6d9c105bf0 2024-04-23T02:35:42.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.476+0000 7f6d8b7fe700 1 -- 172.21.15.71:0/4084959086 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6d90003a60 con 0x7f6d9c105bf0 2024-04-23T02:35:42.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.477+0000 7f6d8b7fe700 1 --2- 172.21.15.71:0/4084959086 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6d8406cce0 0x7f6d8406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:35:42.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.477+0000 7f6d9a59c700 1 --2- 172.21.15.71:0/4084959086 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6d8406cce0 0x7f6d8406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:35:42.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.477+0000 7f6d8b7fe700 1 -- 172.21.15.71:0/4084959086 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6d9008d7c0 con 0x7f6d9c105bf0 2024-04-23T02:35:42.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.477+0000 7f6d9a59c700 1 --2- 172.21.15.71:0/4084959086 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6d8406cce0 0x7f6d8406f1a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c005fd0 tx=0x7f6d8c005f60 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:35:42.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.482+0000 7f6d8b7fe700 1 -- 172.21.15.71:0/4084959086 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6d90057830 con 0x7f6d9c105bf0 2024-04-23T02:35:42.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:42 smithi071 ceph-mon[144544]: from='client.35042 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:35:42.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:42 smithi071 ceph-mon[144544]: pgmap v671: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:42.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:42 smithi071 ceph-mon[144544]: from='client.35046 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:35:42.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:42 smithi071 ceph-mon[144544]: from='client.35050 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:35:42.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:42 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1375685844' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:35:42.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:42 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3332725601' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:35:42.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.754+0000 7f6da0948700 1 -- 172.21.15.71:0/4084959086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6d9c04f2e0 con 0x7f6d9c105bf0 2024-04-23T02:35:42.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.754+0000 7f6d8b7fe700 1 -- 172.21.15.71:0/4084959086 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f6d9005ae50 con 0x7f6d9c105bf0 2024-04-23T02:35:42.758 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:35:42.758 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:35:42.758 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:35:42.758 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:35:42.758 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:35:42.759 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:35:42.759 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:35:42.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.756+0000 7f6da0948700 1 -- 172.21.15.71:0/4084959086 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6d8406cce0 msgr2=0x7f6d8406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:42.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.756+0000 7f6da0948700 1 --2- 172.21.15.71:0/4084959086 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6d8406cce0 0x7f6d8406f1a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c005fd0 tx=0x7f6d8c005f60 comp rx=0 tx=0).stop 2024-04-23T02:35:42.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.756+0000 7f6da0948700 1 -- 172.21.15.71:0/4084959086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d9c105bf0 msgr2=0x7f6d9c197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:35:42.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.756+0000 7f6da0948700 1 --2- 172.21.15.71:0/4084959086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d9c105bf0 0x7f6d9c197ab0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000b5c0 tx=0x7f6d9000be20 comp rx=0 tx=0).stop 2024-04-23T02:35:42.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.756+0000 7f6da0948700 1 -- 172.21.15.71:0/4084959086 shutdown_connections 2024-04-23T02:35:42.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.756+0000 7f6da0948700 1 --2- 172.21.15.71:0/4084959086 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6d8406cce0 0x7f6d8406f1a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:42.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.756+0000 7f6da0948700 1 --2- 172.21.15.71:0/4084959086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d9c0ffb40 0x7f6d9c197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:42.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.756+0000 7f6da0948700 1 --2- 172.21.15.71:0/4084959086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d9c105bf0 0x7f6d9c197ab0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:35:42.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.756+0000 7f6da0948700 1 -- 172.21.15.71:0/4084959086 >> 172.21.15.71:0/4084959086 conn(0x7f6d9c0fb610 msgr2=0x7f6d9c0fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:35:42.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.757+0000 7f6da0948700 1 -- 172.21.15.71:0/4084959086 shutdown_connections 2024-04-23T02:35:42.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:35:42.757+0000 7f6da0948700 1 -- 172.21.15.71:0/4084959086 wait complete. 2024-04-23T02:35:42.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:42 smithi192 ceph-mon[108617]: from='client.35042 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:35:42.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:42 smithi192 ceph-mon[108617]: pgmap v671: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:42.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:42 smithi192 ceph-mon[108617]: from='client.35046 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:35:42.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:42 smithi192 ceph-mon[108617]: from='client.35050 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:35:42.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:42 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1375685844' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:35:42.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:42 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3332725601' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:35:43.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:43 smithi071 ceph-mon[144544]: from='client.35062 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:35:43.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:43 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/4084959086' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:35:43.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:43 smithi192 ceph-mon[108617]: from='client.35062 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:35:43.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:43 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/4084959086' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:35:44.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:44 smithi071 ceph-mon[144544]: pgmap v672: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:44.836 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:44 smithi192 ceph-mon[108617]: pgmap v672: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:46.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:46 smithi071 ceph-mon[144544]: pgmap v673: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:46.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:46 smithi192 ceph-mon[108617]: pgmap v673: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:48.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:48 smithi071 ceph-mon[144544]: pgmap v674: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:48.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:48 smithi192 ceph-mon[108617]: pgmap v674: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:50.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:50 smithi071 ceph-mon[144544]: pgmap v675: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:50.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:35:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:50 smithi192 ceph-mon[108617]: pgmap v675: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:35:52.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:52 smithi071 ceph-mon[144544]: pgmap v676: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:52.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:52 smithi192 ceph-mon[108617]: pgmap v676: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:54.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:54 smithi192 ceph-mon[108617]: pgmap v677: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:54.967 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:54 smithi071 ceph-mon[144544]: pgmap v677: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:56.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:56 smithi192 ceph-mon[108617]: pgmap v678: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:56.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:56 smithi071 ceph-mon[144544]: pgmap v678: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:58.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:35:58 smithi192 ceph-mon[108617]: pgmap v679: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:35:58.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:35:58 smithi071 ceph-mon[144544]: pgmap v679: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:00.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:00 smithi192 ceph-mon[108617]: pgmap v680: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:00.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:00 smithi071 ceph-mon[144544]: pgmap v680: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:02.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:02 smithi192 ceph-mon[108617]: pgmap v681: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:02.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:02 smithi071 ceph-mon[144544]: pgmap v681: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:04.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:04 smithi192 ceph-mon[108617]: pgmap v682: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:04.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:04 smithi071 ceph-mon[144544]: pgmap v682: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:36:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:36:06.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:06 smithi192 ceph-mon[108617]: pgmap v683: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:06.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:06 smithi071 ceph-mon[144544]: pgmap v683: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:08.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:08 smithi192 ceph-mon[108617]: pgmap v684: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:08.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:08 smithi071 ceph-mon[144544]: pgmap v684: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:10.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:10 smithi192 ceph-mon[108617]: pgmap v685: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:10.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:10 smithi071 ceph-mon[144544]: pgmap v685: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:11.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:11 smithi192 ceph-mon[108617]: pgmap v686: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:11.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:11 smithi071 ceph-mon[144544]: pgmap v686: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:12.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.873+0000 7fd5e8af5700 1 -- 172.21.15.71:0/717861661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5e4106cd0 msgr2=0x7fd5e41070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:12.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.873+0000 7fd5e8af5700 1 --2- 172.21.15.71:0/717861661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5e4106cd0 0x7fd5e41070b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009910 tx=0x7fd5d4009c20 comp rx=0 tx=0).stop 2024-04-23T02:36:12.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.873+0000 7fd5e8af5700 1 -- 172.21.15.71:0/717861661 shutdown_connections 2024-04-23T02:36:12.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.873+0000 7fd5e8af5700 1 --2- 172.21.15.71:0/717861661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5e4068580 0x7fd5e40689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:12.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.873+0000 7fd5e8af5700 1 --2- 172.21.15.71:0/717861661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5e4106cd0 0x7fd5e41070b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:12.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.873+0000 7fd5e8af5700 1 -- 172.21.15.71:0/717861661 >> 172.21.15.71:0/717861661 conn(0x7fd5e4075670 msgr2=0x7fd5e4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:12.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.873+0000 7fd5e8af5700 1 -- 172.21.15.71:0/717861661 shutdown_connections 2024-04-23T02:36:12.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.873+0000 7fd5e8af5700 1 -- 172.21.15.71:0/717861661 wait complete. 2024-04-23T02:36:12.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.874+0000 7fd5e8af5700 1 Processor -- start 2024-04-23T02:36:12.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.874+0000 7fd5e8af5700 1 -- start start 2024-04-23T02:36:12.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.874+0000 7fd5e8af5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5e4068580 0x7fd5e40ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:12.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.874+0000 7fd5e8af5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5e4106cd0 0x7fd5e4100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:12.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.874+0000 7fd5e8af5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e4100770 con 0x7fd5e4068580 2024-04-23T02:36:12.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.874+0000 7fd5e8af5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e41008b0 con 0x7fd5e4106cd0 2024-04-23T02:36:12.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.874+0000 7fd5e259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5e4068580 0x7fd5e40ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:12.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.874+0000 7fd5e259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5e4068580 0x7fd5e40ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44890/0 (socket says 172.21.15.71:44890) 2024-04-23T02:36:12.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.874+0000 7fd5e259c700 1 -- 172.21.15.71:0/4022967257 learned_addr learned my addr 172.21.15.71:0/4022967257 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:36:12.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.874+0000 7fd5e1d9b700 1 --2- 172.21.15.71:0/4022967257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5e4106cd0 0x7fd5e4100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:12.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.875+0000 7fd5e259c700 1 -- 172.21.15.71:0/4022967257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5e4106cd0 msgr2=0x7fd5e4100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:12.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.875+0000 7fd5e259c700 1 --2- 172.21.15.71:0/4022967257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5e4106cd0 0x7fd5e4100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:12.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.875+0000 7fd5e259c700 1 -- 172.21.15.71:0/4022967257 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d40095f0 con 0x7fd5e4068580 2024-04-23T02:36:12.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.875+0000 7fd5e259c700 1 --2- 172.21.15.71:0/4022967257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5e4068580 0x7fd5e40ffcf0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4006010 tx=0x7fd5d400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:12.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.875+0000 7fd5d37fe700 1 -- 172.21.15.71:0/4022967257 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5d401d070 con 0x7fd5e4068580 2024-04-23T02:36:12.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.875+0000 7fd5e8af5700 1 -- 172.21.15.71:0/4022967257 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e4100b30 con 0x7fd5e4068580 2024-04-23T02:36:12.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.875+0000 7fd5d37fe700 1 -- 172.21.15.71:0/4022967257 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd5d4005710 con 0x7fd5e4068580 2024-04-23T02:36:12.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.875+0000 7fd5d37fe700 1 -- 172.21.15.71:0/4022967257 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5d400fe20 con 0x7fd5e4068580 2024-04-23T02:36:12.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.875+0000 7fd5e8af5700 1 -- 172.21.15.71:0/4022967257 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e41a2c90 con 0x7fd5e4068580 2024-04-23T02:36:12.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.876+0000 7fd5e8af5700 1 -- 172.21.15.71:0/4022967257 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e41091f0 con 0x7fd5e4068580 2024-04-23T02:36:12.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.877+0000 7fd5d37fe700 1 -- 172.21.15.71:0/4022967257 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd5d4003a60 con 0x7fd5e4068580 2024-04-23T02:36:12.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.877+0000 7fd5d37fe700 1 --2- 172.21.15.71:0/4022967257 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd5cc06cce0 0x7fd5cc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:12.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.877+0000 7fd5d37fe700 1 -- 172.21.15.71:0/4022967257 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd5d408d790 con 0x7fd5e4068580 2024-04-23T02:36:12.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.877+0000 7fd5e1d9b700 1 --2- 172.21.15.71:0/4022967257 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd5cc06cce0 0x7fd5cc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:12.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.878+0000 7fd5e1d9b700 1 --2- 172.21.15.71:0/4022967257 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd5cc06cce0 0x7fd5cc06f1a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4101490 tx=0x7fd5d800b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:12.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:12.882+0000 7fd5d37fe700 1 -- 172.21.15.71:0/4022967257 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd5d4057800 con 0x7fd5e4068580 2024-04-23T02:36:13.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.092+0000 7fd5e8af5700 1 -- 172.21.15.71:0/4022967257 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5e40611d0 con 0x7fd5cc06cce0 2024-04-23T02:36:13.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.094+0000 7fd5d37fe700 1 -- 172.21.15.71:0/4022967257 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd5e40611d0 con 0x7fd5cc06cce0 2024-04-23T02:36:13.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.096+0000 7fd5e8af5700 1 -- 172.21.15.71:0/4022967257 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd5cc06cce0 msgr2=0x7fd5cc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:13.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.096+0000 7fd5e8af5700 1 --2- 172.21.15.71:0/4022967257 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd5cc06cce0 0x7fd5cc06f1a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4101490 tx=0x7fd5d800b410 comp rx=0 tx=0).stop 2024-04-23T02:36:13.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.096+0000 7fd5e8af5700 1 -- 172.21.15.71:0/4022967257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5e4068580 msgr2=0x7fd5e40ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:13.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.096+0000 7fd5e8af5700 1 --2- 172.21.15.71:0/4022967257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5e4068580 0x7fd5e40ffcf0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4006010 tx=0x7fd5d400bd70 comp rx=0 tx=0).stop 2024-04-23T02:36:13.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.096+0000 7fd5e8af5700 1 -- 172.21.15.71:0/4022967257 shutdown_connections 2024-04-23T02:36:13.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.096+0000 7fd5e8af5700 1 --2- 172.21.15.71:0/4022967257 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd5cc06cce0 0x7fd5cc06f1a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.096+0000 7fd5e8af5700 1 --2- 172.21.15.71:0/4022967257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5e4068580 0x7fd5e40ffcf0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.096+0000 7fd5e8af5700 1 --2- 172.21.15.71:0/4022967257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5e4106cd0 0x7fd5e4100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.096+0000 7fd5e8af5700 1 -- 172.21.15.71:0/4022967257 >> 172.21.15.71:0/4022967257 conn(0x7fd5e4075670 msgr2=0x7fd5e40fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:13.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.096+0000 7fd5e8af5700 1 -- 172.21.15.71:0/4022967257 shutdown_connections 2024-04-23T02:36:13.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.096+0000 7fd5e8af5700 1 -- 172.21.15.71:0/4022967257 wait complete. 2024-04-23T02:36:13.110 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:36:13.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.200+0000 7fe515615700 1 -- 172.21.15.71:0/886126813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5100ffda0 msgr2=0x7fe510100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:13.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.200+0000 7fe515615700 1 --2- 172.21.15.71:0/886126813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5100ffda0 0x7fe510100180 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe500009910 tx=0x7fe500009c20 comp rx=0 tx=0).stop 2024-04-23T02:36:13.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.200+0000 7fe515615700 1 -- 172.21.15.71:0/886126813 shutdown_connections 2024-04-23T02:36:13.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.200+0000 7fe515615700 1 --2- 172.21.15.71:0/886126813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe510100750 0x7fe5101047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.200+0000 7fe515615700 1 --2- 172.21.15.71:0/886126813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5100ffda0 0x7fe510100180 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.200+0000 7fe515615700 1 -- 172.21.15.71:0/886126813 >> 172.21.15.71:0/886126813 conn(0x7fe5100fb650 msgr2=0x7fe5100fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:13.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.200+0000 7fe515615700 1 -- 172.21.15.71:0/886126813 shutdown_connections 2024-04-23T02:36:13.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.200+0000 7fe515615700 1 -- 172.21.15.71:0/886126813 wait complete. 2024-04-23T02:36:13.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.201+0000 7fe515615700 1 Processor -- start 2024-04-23T02:36:13.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.201+0000 7fe515615700 1 -- start start 2024-04-23T02:36:13.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.201+0000 7fe515615700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5100ffda0 0x7fe510197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:13.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.201+0000 7fe515615700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe510100750 0x7fe510197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:13.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.201+0000 7fe515615700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5101982a0 con 0x7fe510100750 2024-04-23T02:36:13.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.201+0000 7fe515615700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5101983e0 con 0x7fe5100ffda0 2024-04-23T02:36:13.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.201+0000 7fe50e7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe510100750 0x7fe510197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:13.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.201+0000 7fe50e7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe510100750 0x7fe510197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44914/0 (socket says 172.21.15.71:44914) 2024-04-23T02:36:13.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.201+0000 7fe50e7fc700 1 -- 172.21.15.71:0/996536461 learned_addr learned my addr 172.21.15.71:0/996536461 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:36:13.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.202+0000 7fe50effd700 1 --2- 172.21.15.71:0/996536461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5100ffda0 0x7fe510197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:13.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.202+0000 7fe50e7fc700 1 -- 172.21.15.71:0/996536461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5100ffda0 msgr2=0x7fe510197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:13.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.202+0000 7fe50e7fc700 1 --2- 172.21.15.71:0/996536461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5100ffda0 0x7fe510197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.202+0000 7fe50e7fc700 1 -- 172.21.15.71:0/996536461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5000095f0 con 0x7fe510100750 2024-04-23T02:36:13.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.202+0000 7fe50e7fc700 1 --2- 172.21.15.71:0/996536461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe510100750 0x7fe510197c80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe50400b480 tx=0x7fe50400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:13.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.202+0000 7fe4f7fff700 1 -- 172.21.15.71:0/996536461 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5040118c0 con 0x7fe510100750 2024-04-23T02:36:13.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.202+0000 7fe4f7fff700 1 -- 172.21.15.71:0/996536461 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe504010380 con 0x7fe510100750 2024-04-23T02:36:13.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.202+0000 7fe515615700 1 -- 172.21.15.71:0/996536461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5101a07b0 con 0x7fe510100750 2024-04-23T02:36:13.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.202+0000 7fe4f7fff700 1 -- 172.21.15.71:0/996536461 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe50400f510 con 0x7fe510100750 2024-04-23T02:36:13.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.202+0000 7fe515615700 1 -- 172.21.15.71:0/996536461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5101a0c70 con 0x7fe510100750 2024-04-23T02:36:13.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.203+0000 7fe515615700 1 -- 172.21.15.71:0/996536461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe510108140 con 0x7fe510100750 2024-04-23T02:36:13.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.203+0000 7fe4f7fff700 1 -- 172.21.15.71:0/996536461 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe5040104f0 con 0x7fe510100750 2024-04-23T02:36:13.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.204+0000 7fe4f7fff700 1 --2- 172.21.15.71:0/996536461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe4f806cce0 0x7fe4f806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:13.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.204+0000 7fe4f7fff700 1 -- 172.21.15.71:0/996536461 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe50408c210 con 0x7fe510100750 2024-04-23T02:36:13.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.204+0000 7fe50effd700 1 --2- 172.21.15.71:0/996536461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe4f806cce0 0x7fe4f806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:13.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.204+0000 7fe50effd700 1 --2- 172.21.15.71:0/996536461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe4f806cce0 0x7fe4f806f1a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe50000b5c0 tx=0x7fe500005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:13.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.208+0000 7fe4f7fff700 1 -- 172.21.15.71:0/996536461 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe504056280 con 0x7fe510100750 2024-04-23T02:36:13.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.417+0000 7fe515615700 1 -- 172.21.15.71:0/996536461 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5100611d0 con 0x7fe4f806cce0 2024-04-23T02:36:13.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.419+0000 7fe4f7fff700 1 -- 172.21.15.71:0/996536461 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe5100611d0 con 0x7fe4f806cce0 2024-04-23T02:36:13.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.421+0000 7fe515615700 1 -- 172.21.15.71:0/996536461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe4f806cce0 msgr2=0x7fe4f806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:13.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.421+0000 7fe515615700 1 --2- 172.21.15.71:0/996536461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe4f806cce0 0x7fe4f806f1a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe50000b5c0 tx=0x7fe500005fb0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.421+0000 7fe515615700 1 -- 172.21.15.71:0/996536461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe510100750 msgr2=0x7fe510197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:13.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.421+0000 7fe515615700 1 --2- 172.21.15.71:0/996536461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe510100750 0x7fe510197c80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe50400b480 tx=0x7fe50400b840 comp rx=0 tx=0).stop 2024-04-23T02:36:13.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.421+0000 7fe515615700 1 -- 172.21.15.71:0/996536461 shutdown_connections 2024-04-23T02:36:13.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.421+0000 7fe515615700 1 --2- 172.21.15.71:0/996536461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe4f806cce0 0x7fe4f806f1a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.421+0000 7fe515615700 1 --2- 172.21.15.71:0/996536461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5100ffda0 0x7fe510197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.421+0000 7fe515615700 1 --2- 172.21.15.71:0/996536461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe510100750 0x7fe510197c80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.421+0000 7fe515615700 1 -- 172.21.15.71:0/996536461 >> 172.21.15.71:0/996536461 conn(0x7fe5100fb650 msgr2=0x7fe5100fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:13.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.421+0000 7fe515615700 1 -- 172.21.15.71:0/996536461 shutdown_connections 2024-04-23T02:36:13.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.421+0000 7fe515615700 1 -- 172.21.15.71:0/996536461 wait complete. 2024-04-23T02:36:13.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.524+0000 7fa72428f700 1 -- 172.21.15.71:0/2488063730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa71c100ea0 msgr2=0x7fa71c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:13.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.524+0000 7fa72428f700 1 --2- 172.21.15.71:0/2488063730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa71c100ea0 0x7fa71c101280 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009910 tx=0x7fa70c009c20 comp rx=0 tx=0).stop 2024-04-23T02:36:13.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.524+0000 7fa72428f700 1 -- 172.21.15.71:0/2488063730 shutdown_connections 2024-04-23T02:36:13.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.524+0000 7fa72428f700 1 --2- 172.21.15.71:0/2488063730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa71c101850 0x7fa71c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.524+0000 7fa72428f700 1 --2- 172.21.15.71:0/2488063730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa71c100ea0 0x7fa71c101280 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.524+0000 7fa72428f700 1 -- 172.21.15.71:0/2488063730 >> 172.21.15.71:0/2488063730 conn(0x7fa71c0fc750 msgr2=0x7fa71c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:13.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.524+0000 7fa72428f700 1 -- 172.21.15.71:0/2488063730 shutdown_connections 2024-04-23T02:36:13.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.524+0000 7fa72428f700 1 -- 172.21.15.71:0/2488063730 wait complete. 2024-04-23T02:36:13.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.524+0000 7fa72428f700 1 Processor -- start 2024-04-23T02:36:13.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.524+0000 7fa72428f700 1 -- start start 2024-04-23T02:36:13.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.525+0000 7fa72428f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa71c100ea0 0x7fa71c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:13.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.525+0000 7fa72428f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa71c101850 0x7fa71c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:13.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.525+0000 7fa72428f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c19d0a0 con 0x7fa71c101850 2024-04-23T02:36:13.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.525+0000 7fa72428f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c1965e0 con 0x7fa71c100ea0 2024-04-23T02:36:13.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.525+0000 7fa72182a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa71c101850 0x7fa71c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:13.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.525+0000 7fa72182a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa71c101850 0x7fa71c19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44934/0 (socket says 172.21.15.71:44934) 2024-04-23T02:36:13.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.525+0000 7fa72182a700 1 -- 172.21.15.71:0/1797906636 learned_addr learned my addr 172.21.15.71:0/1797906636 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:36:13.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.525+0000 7fa72202b700 1 --2- 172.21.15.71:0/1797906636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa71c100ea0 0x7fa71c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:13.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.525+0000 7fa72182a700 1 -- 172.21.15.71:0/1797906636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa71c100ea0 msgr2=0x7fa71c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:13.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.525+0000 7fa72182a700 1 --2- 172.21.15.71:0/1797906636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa71c100ea0 0x7fa71c19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.525+0000 7fa72182a700 1 -- 172.21.15.71:0/1797906636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c0095f0 con 0x7fa71c101850 2024-04-23T02:36:13.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.525+0000 7fa72182a700 1 --2- 172.21.15.71:0/1797906636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa71c101850 0x7fa71c19caa0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa71800c960 tx=0x7fa71800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:13.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.525+0000 7fa7137fe700 1 -- 172.21.15.71:0/1797906636 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa718007a70 con 0x7fa71c101850 2024-04-23T02:36:13.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.525+0000 7fa7137fe700 1 -- 172.21.15.71:0/1797906636 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa718007bd0 con 0x7fa71c101850 2024-04-23T02:36:13.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.525+0000 7fa7137fe700 1 -- 172.21.15.71:0/1797906636 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa718018630 con 0x7fa71c101850 2024-04-23T02:36:13.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.525+0000 7fa72428f700 1 -- 172.21.15.71:0/1797906636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa71c1968c0 con 0x7fa71c101850 2024-04-23T02:36:13.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.525+0000 7fa72428f700 1 -- 172.21.15.71:0/1797906636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa71c196e10 con 0x7fa71c101850 2024-04-23T02:36:13.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.526+0000 7fa72428f700 1 -- 172.21.15.71:0/1797906636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa71c109240 con 0x7fa71c101850 2024-04-23T02:36:13.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.527+0000 7fa7137fe700 1 -- 172.21.15.71:0/1797906636 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa71800f450 con 0x7fa71c101850 2024-04-23T02:36:13.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.527+0000 7fa7137fe700 1 --2- 172.21.15.71:0/1797906636 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa70806cce0 0x7fa70806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:13.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.528+0000 7fa7137fe700 1 -- 172.21.15.71:0/1797906636 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa71808c6c0 con 0x7fa71c101850 2024-04-23T02:36:13.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.528+0000 7fa72202b700 1 --2- 172.21.15.71:0/1797906636 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa70806cce0 0x7fa70806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:13.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.528+0000 7fa72202b700 1 --2- 172.21.15.71:0/1797906636 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa70806cce0 0x7fa70806f1a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa70c006010 tx=0x7fa70c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:13.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.532+0000 7fa7137fe700 1 -- 172.21.15.71:0/1797906636 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa71805a390 con 0x7fa71c101850 2024-04-23T02:36:13.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.734+0000 7fa72428f700 1 -- 172.21.15.71:0/1797906636 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa71c1970e0 con 0x7fa70806cce0 2024-04-23T02:36:13.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.745+0000 7fa7137fe700 1 -- 172.21.15.71:0/1797906636 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa71c1970e0 con 0x7fa70806cce0 2024-04-23T02:36:13.748 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:36:13.748 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (19m) 13m ago 31m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:36:13.748 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (31m) 13m ago 31m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:36:13.748 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (30m) 13m ago 30m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:36:13.748 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (17m) 13m ago 31m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:36:13.748 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (17m) 13m ago 30m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:36:13.749 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (19m) 13m ago 30m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:36:13.749 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (14m) 13m ago 25m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:36:13.749 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (13m) 13m ago 25m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:36:13.749 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (25m) 13m ago 25m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:36:13.749 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (13m) 13m ago 25m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:36:13.749 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (21m) 13m ago 32m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:36:13.749 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (21m) 13m ago 30m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:36:13.749 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (18m) 13m ago 32m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:36:13.749 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (18m) 13m ago 30m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:36:13.749 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (20m) 13m ago 31m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:36:13.749 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (20m) 13m ago 30m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:36:13.749 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (17m) 13m ago 29m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:36:13.750 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (16m) 13m ago 28m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:36:13.750 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (16m) 13m ago 28m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:36:13.750 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (15m) 13m ago 27m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:36:13.750 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (14m) 13m ago 27m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:36:13.750 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (14m) 13m ago 26m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:36:13.750 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (20m) 13m ago 30m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:36:13.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.747+0000 7fa72428f700 1 -- 172.21.15.71:0/1797906636 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa70806cce0 msgr2=0x7fa70806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:13.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.747+0000 7fa72428f700 1 --2- 172.21.15.71:0/1797906636 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa70806cce0 0x7fa70806f1a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa70c006010 tx=0x7fa70c01a040 comp rx=0 tx=0).stop 2024-04-23T02:36:13.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.747+0000 7fa72428f700 1 -- 172.21.15.71:0/1797906636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa71c101850 msgr2=0x7fa71c19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:13.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.747+0000 7fa72428f700 1 --2- 172.21.15.71:0/1797906636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa71c101850 0x7fa71c19caa0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa71800c960 tx=0x7fa71800cc70 comp rx=0 tx=0).stop 2024-04-23T02:36:13.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.747+0000 7fa72428f700 1 -- 172.21.15.71:0/1797906636 shutdown_connections 2024-04-23T02:36:13.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.747+0000 7fa72428f700 1 --2- 172.21.15.71:0/1797906636 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa70806cce0 0x7fa70806f1a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.747+0000 7fa72428f700 1 --2- 172.21.15.71:0/1797906636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa71c100ea0 0x7fa71c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.747+0000 7fa72428f700 1 --2- 172.21.15.71:0/1797906636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa71c101850 0x7fa71c19caa0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.747+0000 7fa72428f700 1 -- 172.21.15.71:0/1797906636 >> 172.21.15.71:0/1797906636 conn(0x7fa71c0fc750 msgr2=0x7fa71c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:13.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.747+0000 7fa72428f700 1 -- 172.21.15.71:0/1797906636 shutdown_connections 2024-04-23T02:36:13.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.747+0000 7fa72428f700 1 -- 172.21.15.71:0/1797906636 wait complete. 2024-04-23T02:36:13.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.848+0000 7f4d5f226700 1 -- 172.21.15.71:0/2175937976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d58105350 msgr2=0x7f4d58105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:13.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.848+0000 7f4d5f226700 1 --2- 172.21.15.71:0/2175937976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d58105350 0x7f4d58105730 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009910 tx=0x7f4d4c009c20 comp rx=0 tx=0).stop 2024-04-23T02:36:13.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.848+0000 7f4d5f226700 1 -- 172.21.15.71:0/2175937976 shutdown_connections 2024-04-23T02:36:13.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.848+0000 7f4d5f226700 1 --2- 172.21.15.71:0/2175937976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d580ff340 0x7f4d580ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.848+0000 7f4d5f226700 1 --2- 172.21.15.71:0/2175937976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d58105350 0x7f4d58105730 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.848+0000 7f4d5f226700 1 -- 172.21.15.71:0/2175937976 >> 172.21.15.71:0/2175937976 conn(0x7f4d58074e30 msgr2=0x7f4d58075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:13.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.848+0000 7f4d5f226700 1 -- 172.21.15.71:0/2175937976 shutdown_connections 2024-04-23T02:36:13.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.848+0000 7f4d5f226700 1 -- 172.21.15.71:0/2175937976 wait complete. 2024-04-23T02:36:13.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.848+0000 7f4d5f226700 1 Processor -- start 2024-04-23T02:36:13.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.848+0000 7f4d5f226700 1 -- start start 2024-04-23T02:36:13.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.848+0000 7f4d5f226700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d580ff340 0x7f4d58197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:13.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.848+0000 7f4d5f226700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d58105350 0x7f4d58197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:13.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.848+0000 7f4d5f226700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d58198180 con 0x7f4d58105350 2024-04-23T02:36:13.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.849+0000 7f4d5f226700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d581982c0 con 0x7f4d580ff340 2024-04-23T02:36:13.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.849+0000 7f4d57fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d58105350 0x7f4d58197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:13.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.849+0000 7f4d57fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d58105350 0x7f4d58197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44948/0 (socket says 172.21.15.71:44948) 2024-04-23T02:36:13.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.849+0000 7f4d57fff700 1 -- 172.21.15.71:0/210042478 learned_addr learned my addr 172.21.15.71:0/210042478 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:36:13.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.849+0000 7f4d5cfc2700 1 --2- 172.21.15.71:0/210042478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d580ff340 0x7f4d58197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:13.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.849+0000 7f4d57fff700 1 -- 172.21.15.71:0/210042478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d580ff340 msgr2=0x7f4d58197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:13.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.849+0000 7f4d57fff700 1 --2- 172.21.15.71:0/210042478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d580ff340 0x7f4d58197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:13.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.849+0000 7f4d57fff700 1 -- 172.21.15.71:0/210042478 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4c0095f0 con 0x7f4d58105350 2024-04-23T02:36:13.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.849+0000 7f4d57fff700 1 --2- 172.21.15.71:0/210042478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d58105350 0x7f4d58197b60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400e920 tx=0x7f4d4400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:13.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.849+0000 7f4d55ffb700 1 -- 172.21.15.71:0/210042478 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d44009790 con 0x7f4d58105350 2024-04-23T02:36:13.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.849+0000 7f4d55ffb700 1 -- 172.21.15.71:0/210042478 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4d4400cba0 con 0x7f4d58105350 2024-04-23T02:36:13.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.849+0000 7f4d5f226700 1 -- 172.21.15.71:0/210042478 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d581a0690 con 0x7f4d58105350 2024-04-23T02:36:13.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.849+0000 7f4d55ffb700 1 -- 172.21.15.71:0/210042478 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d44018650 con 0x7f4d58105350 2024-04-23T02:36:13.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.849+0000 7f4d5f226700 1 -- 172.21.15.71:0/210042478 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d581a0b50 con 0x7f4d58105350 2024-04-23T02:36:13.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.850+0000 7f4d5f226700 1 -- 172.21.15.71:0/210042478 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d5804f2e0 con 0x7f4d58105350 2024-04-23T02:36:13.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.851+0000 7f4d55ffb700 1 -- 172.21.15.71:0/210042478 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4d4400cd10 con 0x7f4d58105350 2024-04-23T02:36:13.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.851+0000 7f4d55ffb700 1 --2- 172.21.15.71:0/210042478 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4d4806cce0 0x7f4d4806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:13.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.852+0000 7f4d55ffb700 1 -- 172.21.15.71:0/210042478 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4d44014070 con 0x7f4d58105350 2024-04-23T02:36:13.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.852+0000 7f4d5cfc2700 1 --2- 172.21.15.71:0/210042478 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4d4806cce0 0x7f4d4806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:13.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.852+0000 7f4d5cfc2700 1 --2- 172.21.15.71:0/210042478 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4d4806cce0 0x7f4d4806f1a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c000c00 tx=0x7f4d4c009f70 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:13.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:13.855+0000 7f4d55ffb700 1 -- 172.21.15.71:0/210042478 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f4d44056be0 con 0x7f4d58105350 2024-04-23T02:36:14.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.130+0000 7f4d5f226700 1 -- 172.21.15.71:0/210042478 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4d580fe6b0 con 0x7f4d58105350 2024-04-23T02:36:14.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.130+0000 7f4d55ffb700 1 -- 172.21.15.71:0/210042478 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f4d4405a200 con 0x7f4d58105350 2024-04-23T02:36:14.134 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:36:14.134 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:36:14.135 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:36:14.135 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:36:14.135 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:36:14.135 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:36:14.135 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:36:14.135 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:36:14.136 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:36:14.136 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:36:14.136 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:36:14.136 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:36:14.136 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:36:14.136 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:36:14.136 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:36:14.137 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:36:14.137 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:36:14.137 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:36:14.137 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:36:14.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.132+0000 7f4d5f226700 1 -- 172.21.15.71:0/210042478 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4d4806cce0 msgr2=0x7f4d4806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:14.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.133+0000 7f4d5f226700 1 --2- 172.21.15.71:0/210042478 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4d4806cce0 0x7f4d4806f1a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c000c00 tx=0x7f4d4c009f70 comp rx=0 tx=0).stop 2024-04-23T02:36:14.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.133+0000 7f4d5f226700 1 -- 172.21.15.71:0/210042478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d58105350 msgr2=0x7f4d58197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:14.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.133+0000 7f4d5f226700 1 --2- 172.21.15.71:0/210042478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d58105350 0x7f4d58197b60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400e920 tx=0x7f4d4400ece0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.133+0000 7f4d5f226700 1 -- 172.21.15.71:0/210042478 shutdown_connections 2024-04-23T02:36:14.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.133+0000 7f4d5f226700 1 --2- 172.21.15.71:0/210042478 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4d4806cce0 0x7f4d4806f1a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.133+0000 7f4d5f226700 1 --2- 172.21.15.71:0/210042478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d580ff340 0x7f4d58197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.133+0000 7f4d5f226700 1 --2- 172.21.15.71:0/210042478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d58105350 0x7f4d58197b60 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.133+0000 7f4d5f226700 1 -- 172.21.15.71:0/210042478 >> 172.21.15.71:0/210042478 conn(0x7f4d58074e30 msgr2=0x7f4d58103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:14.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.133+0000 7f4d5f226700 1 -- 172.21.15.71:0/210042478 shutdown_connections 2024-04-23T02:36:14.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.133+0000 7f4d5f226700 1 -- 172.21.15.71:0/210042478 wait complete. 2024-04-23T02:36:14.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.235+0000 7ffbe4224700 1 -- 172.21.15.71:0/3960675779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbdc105bd0 msgr2=0x7ffbdc105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:14.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.235+0000 7ffbe4224700 1 --2- 172.21.15.71:0/3960675779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbdc105bd0 0x7ffbdc105fb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009910 tx=0x7ffbd0009c20 comp rx=0 tx=0).stop 2024-04-23T02:36:14.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.235+0000 7ffbe4224700 1 -- 172.21.15.71:0/3960675779 shutdown_connections 2024-04-23T02:36:14.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.235+0000 7ffbe4224700 1 --2- 172.21.15.71:0/3960675779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbdc0ffc20 0x7ffbdc100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.235+0000 7ffbe4224700 1 --2- 172.21.15.71:0/3960675779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbdc105bd0 0x7ffbdc105fb0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.235+0000 7ffbe4224700 1 -- 172.21.15.71:0/3960675779 >> 172.21.15.71:0/3960675779 conn(0x7ffbdc075670 msgr2=0x7ffbdc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:14.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.235+0000 7ffbe4224700 1 -- 172.21.15.71:0/3960675779 shutdown_connections 2024-04-23T02:36:14.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.235+0000 7ffbe4224700 1 -- 172.21.15.71:0/3960675779 wait complete. 2024-04-23T02:36:14.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.236+0000 7ffbe4224700 1 Processor -- start 2024-04-23T02:36:14.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.236+0000 7ffbe4224700 1 -- start start 2024-04-23T02:36:14.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.236+0000 7ffbe4224700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbdc0ffc20 0x7ffbdc195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:14.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.236+0000 7ffbe4224700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbdc105bd0 0x7ffbdc195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:14.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.236+0000 7ffbe4224700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc1961d0 con 0x7ffbdc0ffc20 2024-04-23T02:36:14.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.236+0000 7ffbe4224700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc196340 con 0x7ffbdc105bd0 2024-04-23T02:36:14.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.236+0000 7ffbe1fc0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbdc0ffc20 0x7ffbdc195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:14.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.236+0000 7ffbe1fc0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbdc0ffc20 0x7ffbdc195690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44960/0 (socket says 172.21.15.71:44960) 2024-04-23T02:36:14.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.236+0000 7ffbe1fc0700 1 -- 172.21.15.71:0/1792167993 learned_addr learned my addr 172.21.15.71:0/1792167993 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:36:14.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.236+0000 7ffbe17bf700 1 --2- 172.21.15.71:0/1792167993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbdc105bd0 0x7ffbdc195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:14.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.237+0000 7ffbe1fc0700 1 -- 172.21.15.71:0/1792167993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbdc105bd0 msgr2=0x7ffbdc195bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:14.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.237+0000 7ffbe1fc0700 1 --2- 172.21.15.71:0/1792167993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbdc105bd0 0x7ffbdc195bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.237+0000 7ffbe1fc0700 1 -- 172.21.15.71:0/1792167993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd00095f0 con 0x7ffbdc0ffc20 2024-04-23T02:36:14.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.237+0000 7ffbe1fc0700 1 --2- 172.21.15.71:0/1792167993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbdc0ffc20 0x7ffbdc195690 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0006010 tx=0x7ffbd000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:14.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.237+0000 7ffbceffd700 1 -- 172.21.15.71:0/1792167993 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbd001d070 con 0x7ffbdc0ffc20 2024-04-23T02:36:14.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.237+0000 7ffbceffd700 1 -- 172.21.15.71:0/1792167993 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffbd000f460 con 0x7ffbdc0ffc20 2024-04-23T02:36:14.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.237+0000 7ffbe4224700 1 -- 172.21.15.71:0/1792167993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbdc199080 con 0x7ffbdc0ffc20 2024-04-23T02:36:14.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.237+0000 7ffbceffd700 1 -- 172.21.15.71:0/1792167993 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbd0021640 con 0x7ffbdc0ffc20 2024-04-23T02:36:14.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.237+0000 7ffbe4224700 1 -- 172.21.15.71:0/1792167993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbdc199570 con 0x7ffbdc0ffc20 2024-04-23T02:36:14.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.238+0000 7ffbe4224700 1 -- 172.21.15.71:0/1792167993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbdc1080f0 con 0x7ffbdc0ffc20 2024-04-23T02:36:14.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.238+0000 7ffbceffd700 1 -- 172.21.15.71:0/1792167993 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ffbd000fa20 con 0x7ffbdc0ffc20 2024-04-23T02:36:14.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.239+0000 7ffbceffd700 1 --2- 172.21.15.71:0/1792167993 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ffbc806cce0 0x7ffbc806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:14.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.239+0000 7ffbe17bf700 1 --2- 172.21.15.71:0/1792167993 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ffbc806cce0 0x7ffbc806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:14.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.239+0000 7ffbceffd700 1 -- 172.21.15.71:0/1792167993 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ffbd008d7b0 con 0x7ffbdc0ffc20 2024-04-23T02:36:14.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.239+0000 7ffbe17bf700 1 --2- 172.21.15.71:0/1792167993 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ffbc806cce0 0x7ffbc806f1a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8005ea0 tx=0x7ffbd8005e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:14.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.243+0000 7ffbceffd700 1 -- 172.21.15.71:0/1792167993 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ffbd00578a0 con 0x7ffbdc0ffc20 2024-04-23T02:36:14.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:14 smithi192 ceph-mon[108617]: pgmap v687: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:14.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:14 smithi192 ceph-mon[108617]: from='client.35070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:36:14.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:14 smithi192 ceph-mon[108617]: from='client.35074 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:36:14.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.479+0000 7ffbe4224700 1 -- 172.21.15.71:0/1792167993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ffbdc107ad0 con 0x7ffbdc0ffc20 2024-04-23T02:36:14.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.479+0000 7ffbceffd700 1 -- 172.21.15.71:0/1792167993 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7ffbd0057020 con 0x7ffbdc0ffc20 2024-04-23T02:36:14.483 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:36:14.483 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:36:14.483 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:36:14.483 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:36:14.483 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:36:14.483 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:36:14.483 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:36:14.483 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:36:14.484 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:36:14.484 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:36:14.484 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:36:14.484 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:36:14.484 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:36:14.484 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:36:14.484 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:36:14.484 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:36:14.484 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:36:14.484 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:36:14.484 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:36:14.484 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:36:14.484 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:36:14.485 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:36:14.485 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:36:14.485 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:36:14.485 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:36:14.485 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:36:14.485 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:36:14.485 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:36:14.485 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:36:14.485 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:36:14.485 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:36:14.485 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:36:14.485 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:36:14.485 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:36:14.485 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:36:14.486 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:36:14.486 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:36:14.486 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:36:14.486 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:36:14.486 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:36:14.486 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:36:14.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.481+0000 7ffbe4224700 1 -- 172.21.15.71:0/1792167993 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ffbc806cce0 msgr2=0x7ffbc806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:14.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.481+0000 7ffbe4224700 1 --2- 172.21.15.71:0/1792167993 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ffbc806cce0 0x7ffbc806f1a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8005ea0 tx=0x7ffbd8005e30 comp rx=0 tx=0).stop 2024-04-23T02:36:14.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.481+0000 7ffbe4224700 1 -- 172.21.15.71:0/1792167993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbdc0ffc20 msgr2=0x7ffbdc195690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:14.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.481+0000 7ffbe4224700 1 --2- 172.21.15.71:0/1792167993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbdc0ffc20 0x7ffbdc195690 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0006010 tx=0x7ffbd000be20 comp rx=0 tx=0).stop 2024-04-23T02:36:14.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.482+0000 7ffbe4224700 1 -- 172.21.15.71:0/1792167993 shutdown_connections 2024-04-23T02:36:14.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.482+0000 7ffbe4224700 1 --2- 172.21.15.71:0/1792167993 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ffbc806cce0 0x7ffbc806f1a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.482+0000 7ffbe4224700 1 --2- 172.21.15.71:0/1792167993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbdc0ffc20 0x7ffbdc195690 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.482+0000 7ffbe4224700 1 --2- 172.21.15.71:0/1792167993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbdc105bd0 0x7ffbdc195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.482+0000 7ffbe4224700 1 -- 172.21.15.71:0/1792167993 >> 172.21.15.71:0/1792167993 conn(0x7ffbdc075670 msgr2=0x7ffbdc0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:14.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.482+0000 7ffbe4224700 1 -- 172.21.15.71:0/1792167993 shutdown_connections 2024-04-23T02:36:14.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.482+0000 7ffbe4224700 1 -- 172.21.15.71:0/1792167993 wait complete. 2024-04-23T02:36:14.487 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:36:14.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:14 smithi071 ceph-mon[144544]: pgmap v687: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:14.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:14 smithi071 ceph-mon[144544]: from='client.35070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:36:14.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:14 smithi071 ceph-mon[144544]: from='client.35074 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:36:14.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.585+0000 7f7ac4214700 1 -- 172.21.15.71:0/3701207934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7abc068580 msgr2=0x7f7abc0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:14.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.585+0000 7f7ac4214700 1 --2- 172.21.15.71:0/3701207934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7abc068580 0x7f7abc0689e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009910 tx=0x7f7ab8009c20 comp rx=0 tx=0).stop 2024-04-23T02:36:14.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.585+0000 7f7ac4214700 1 -- 172.21.15.71:0/3701207934 shutdown_connections 2024-04-23T02:36:14.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.585+0000 7f7ac4214700 1 --2- 172.21.15.71:0/3701207934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7abc068580 0x7f7abc0689e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.585+0000 7f7ac4214700 1 --2- 172.21.15.71:0/3701207934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7abc106cd0 0x7f7abc1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.585+0000 7f7ac4214700 1 -- 172.21.15.71:0/3701207934 >> 172.21.15.71:0/3701207934 conn(0x7f7abc075670 msgr2=0x7f7abc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:14.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.585+0000 7f7ac4214700 1 -- 172.21.15.71:0/3701207934 shutdown_connections 2024-04-23T02:36:14.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.585+0000 7f7ac4214700 1 -- 172.21.15.71:0/3701207934 wait complete. 2024-04-23T02:36:14.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.585+0000 7f7ac4214700 1 Processor -- start 2024-04-23T02:36:14.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.585+0000 7f7ac4214700 1 -- start start 2024-04-23T02:36:14.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.586+0000 7f7ac4214700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7abc068580 0x7f7abc0ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:14.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.586+0000 7f7ac4214700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7abc106cd0 0x7f7abc100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:14.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.586+0000 7f7ac4214700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc100770 con 0x7f7abc068580 2024-04-23T02:36:14.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.586+0000 7f7ac4214700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc1008b0 con 0x7f7abc106cd0 2024-04-23T02:36:14.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.586+0000 7f7ac1fb0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7abc068580 0x7f7abc0ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:14.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.586+0000 7f7ac1fb0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7abc068580 0x7f7abc0ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44980/0 (socket says 172.21.15.71:44980) 2024-04-23T02:36:14.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.586+0000 7f7ac1fb0700 1 -- 172.21.15.71:0/2607189696 learned_addr learned my addr 172.21.15.71:0/2607189696 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:36:14.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.586+0000 7f7ac17af700 1 --2- 172.21.15.71:0/2607189696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7abc106cd0 0x7f7abc100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:14.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.586+0000 7f7ac1fb0700 1 -- 172.21.15.71:0/2607189696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7abc106cd0 msgr2=0x7f7abc100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:14.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.586+0000 7f7ac1fb0700 1 --2- 172.21.15.71:0/2607189696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7abc106cd0 0x7f7abc100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.586+0000 7f7ac1fb0700 1 -- 172.21.15.71:0/2607189696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab80095f0 con 0x7f7abc068580 2024-04-23T02:36:14.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.586+0000 7f7ac17af700 1 --2- 172.21.15.71:0/2607189696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7abc106cd0 0x7f7abc100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:36:14.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.586+0000 7f7ac1fb0700 1 --2- 172.21.15.71:0/2607189696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7abc068580 0x7f7abc0ffcf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00c8a0 tx=0x7f7aac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:14.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.586+0000 7f7ab2ffd700 1 -- 172.21.15.71:0/2607189696 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7aac00cea0 con 0x7f7abc068580 2024-04-23T02:36:14.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.586+0000 7f7ab2ffd700 1 -- 172.21.15.71:0/2607189696 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7aac004830 con 0x7f7abc068580 2024-04-23T02:36:14.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.586+0000 7f7ab2ffd700 1 -- 172.21.15.71:0/2607189696 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7aac005500 con 0x7f7abc068580 2024-04-23T02:36:14.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.586+0000 7f7ac4214700 1 -- 172.21.15.71:0/2607189696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7abc1a2940 con 0x7f7abc068580 2024-04-23T02:36:14.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.586+0000 7f7ac4214700 1 -- 172.21.15.71:0/2607189696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7abc1a2d50 con 0x7f7abc068580 2024-04-23T02:36:14.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.587+0000 7f7ac4214700 1 -- 172.21.15.71:0/2607189696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7abc1091d0 con 0x7f7abc068580 2024-04-23T02:36:14.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.588+0000 7f7ab2ffd700 1 -- 172.21.15.71:0/2607189696 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7aac0049a0 con 0x7f7abc068580 2024-04-23T02:36:14.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.588+0000 7f7ab2ffd700 1 --2- 172.21.15.71:0/2607189696 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7aa806cce0 0x7f7aa806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:14.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.588+0000 7f7ab2ffd700 1 -- 172.21.15.71:0/2607189696 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7aac08c1d0 con 0x7f7abc068580 2024-04-23T02:36:14.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.588+0000 7f7ac17af700 1 --2- 172.21.15.71:0/2607189696 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7aa806cce0 0x7f7aa806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:14.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.589+0000 7f7ac17af700 1 --2- 172.21.15.71:0/2607189696 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7aa806cce0 0x7f7aa806f1a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800b5c0 tx=0x7f7ab8005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:14.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.593+0000 7f7ab2ffd700 1 -- 172.21.15.71:0/2607189696 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7aac0562c0 con 0x7f7abc068580 2024-04-23T02:36:14.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.803+0000 7f7ac4214700 1 -- 172.21.15.71:0/2607189696 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7abc0611d0 con 0x7f7aa806cce0 2024-04-23T02:36:14.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.804+0000 7f7ab2ffd700 1 -- 172.21.15.71:0/2607189696 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7abc0611d0 con 0x7f7aa806cce0 2024-04-23T02:36:14.808 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:36:14.808 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:36:14.808 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:36:14.808 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:36:14.808 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:36:14.808 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:36:14.808 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:36:14.808 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:36:14.808 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:36:14.808 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:36:14.808 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:36:14.809 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:36:14.809 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:36:14.809 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:36:14.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.806+0000 7f7ac4214700 1 -- 172.21.15.71:0/2607189696 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7aa806cce0 msgr2=0x7f7aa806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:14.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.806+0000 7f7ac4214700 1 --2- 172.21.15.71:0/2607189696 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7aa806cce0 0x7f7aa806f1a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800b5c0 tx=0x7f7ab8005960 comp rx=0 tx=0).stop 2024-04-23T02:36:14.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.806+0000 7f7ac4214700 1 -- 172.21.15.71:0/2607189696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7abc068580 msgr2=0x7f7abc0ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:14.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.806+0000 7f7ac4214700 1 --2- 172.21.15.71:0/2607189696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7abc068580 0x7f7abc0ffcf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00c8a0 tx=0x7f7aac00cc60 comp rx=0 tx=0).stop 2024-04-23T02:36:14.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.806+0000 7f7ac4214700 1 -- 172.21.15.71:0/2607189696 shutdown_connections 2024-04-23T02:36:14.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.806+0000 7f7ac4214700 1 --2- 172.21.15.71:0/2607189696 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7aa806cce0 0x7f7aa806f1a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.806+0000 7f7ac4214700 1 --2- 172.21.15.71:0/2607189696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7abc068580 0x7f7abc0ffcf0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.806+0000 7f7ac4214700 1 --2- 172.21.15.71:0/2607189696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7abc106cd0 0x7f7abc100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.806+0000 7f7ac4214700 1 -- 172.21.15.71:0/2607189696 >> 172.21.15.71:0/2607189696 conn(0x7f7abc075670 msgr2=0x7f7abc0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:14.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.807+0000 7f7ac4214700 1 -- 172.21.15.71:0/2607189696 shutdown_connections 2024-04-23T02:36:14.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.807+0000 7f7ac4214700 1 -- 172.21.15.71:0/2607189696 wait complete. 2024-04-23T02:36:14.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.909+0000 7f5a80871700 1 -- 172.21.15.71:0/2198665299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7c100ea0 msgr2=0x7f5a7c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:14.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.909+0000 7f5a80871700 1 --2- 172.21.15.71:0/2198665299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7c100ea0 0x7f5a7c101280 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009910 tx=0x7f5a70009c20 comp rx=0 tx=0).stop 2024-04-23T02:36:14.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.909+0000 7f5a80871700 1 -- 172.21.15.71:0/2198665299 shutdown_connections 2024-04-23T02:36:14.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.909+0000 7f5a80871700 1 --2- 172.21.15.71:0/2198665299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a7c101850 0x7f5a7c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.909+0000 7f5a80871700 1 --2- 172.21.15.71:0/2198665299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7c100ea0 0x7f5a7c101280 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.909+0000 7f5a80871700 1 -- 172.21.15.71:0/2198665299 >> 172.21.15.71:0/2198665299 conn(0x7f5a7c0fc750 msgr2=0x7f5a7c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:14.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.909+0000 7f5a80871700 1 -- 172.21.15.71:0/2198665299 shutdown_connections 2024-04-23T02:36:14.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.909+0000 7f5a80871700 1 -- 172.21.15.71:0/2198665299 wait complete. 2024-04-23T02:36:14.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.910+0000 7f5a80871700 1 Processor -- start 2024-04-23T02:36:14.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.910+0000 7f5a80871700 1 -- start start 2024-04-23T02:36:14.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.910+0000 7f5a80871700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a7c100ea0 0x7f5a7c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:14.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.910+0000 7f5a80871700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7c101850 0x7f5a7c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:14.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.910+0000 7f5a80871700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c19d130 con 0x7f5a7c101850 2024-04-23T02:36:14.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.910+0000 7f5a80871700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c1965e0 con 0x7f5a7c100ea0 2024-04-23T02:36:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.910+0000 7f5a79d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7c101850 0x7f5a7c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.910+0000 7f5a79d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7c101850 0x7f5a7c19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44996/0 (socket says 172.21.15.71:44996) 2024-04-23T02:36:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.910+0000 7f5a79d9b700 1 -- 172.21.15.71:0/1897279697 learned_addr learned my addr 172.21.15.71:0/1897279697 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:36:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.910+0000 7f5a7a59c700 1 --2- 172.21.15.71:0/1897279697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a7c100ea0 0x7f5a7c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.910+0000 7f5a79d9b700 1 -- 172.21.15.71:0/1897279697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a7c100ea0 msgr2=0x7f5a7c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.910+0000 7f5a79d9b700 1 --2- 172.21.15.71:0/1897279697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a7c100ea0 0x7f5a7c19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.910+0000 7f5a79d9b700 1 -- 172.21.15.71:0/1897279697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a700095f0 con 0x7f5a7c101850 2024-04-23T02:36:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.911+0000 7f5a7a59c700 1 --2- 172.21.15.71:0/1897279697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a7c100ea0 0x7f5a7c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:36:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.911+0000 7f5a79d9b700 1 --2- 172.21.15.71:0/1897279697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7c101850 0x7f5a7c19caa0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00d900 tx=0x7f5a6c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.911+0000 7f5a6b7fe700 1 -- 172.21.15.71:0/1897279697 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a6c009900 con 0x7f5a7c101850 2024-04-23T02:36:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.911+0000 7f5a80871700 1 -- 172.21.15.71:0/1897279697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7c1968c0 con 0x7f5a7c101850 2024-04-23T02:36:14.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.911+0000 7f5a6b7fe700 1 -- 172.21.15.71:0/1897279697 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5a6c010460 con 0x7f5a7c101850 2024-04-23T02:36:14.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.911+0000 7f5a6b7fe700 1 -- 172.21.15.71:0/1897279697 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a6c00f5f0 con 0x7f5a7c101850 2024-04-23T02:36:14.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.911+0000 7f5a80871700 1 -- 172.21.15.71:0/1897279697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a7c196e10 con 0x7f5a7c101850 2024-04-23T02:36:14.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.912+0000 7f5a80871700 1 -- 172.21.15.71:0/1897279697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a7c109240 con 0x7f5a7c101850 2024-04-23T02:36:14.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.912+0000 7f5a6b7fe700 1 -- 172.21.15.71:0/1897279697 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5a6c00fac0 con 0x7f5a7c101850 2024-04-23T02:36:14.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.912+0000 7f5a6b7fe700 1 --2- 172.21.15.71:0/1897279697 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5a6406cce0 0x7f5a6406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:14.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.913+0000 7f5a7a59c700 1 --2- 172.21.15.71:0/1897279697 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5a6406cce0 0x7f5a6406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:14.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.913+0000 7f5a6b7fe700 1 -- 172.21.15.71:0/1897279697 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5a6c08c3f0 con 0x7f5a7c101850 2024-04-23T02:36:14.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.913+0000 7f5a7a59c700 1 --2- 172.21.15.71:0/1897279697 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5a6406cce0 0x7f5a6406f1a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000b5c0 tx=0x7f5a7001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:14.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:14.917+0000 7f5a6b7fe700 1 -- 172.21.15.71:0/1897279697 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5a6c059f60 con 0x7f5a7c101850 2024-04-23T02:36:15.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:15.194+0000 7f5a80871700 1 -- 172.21.15.71:0/1897279697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5a7c04f2e0 con 0x7f5a7c101850 2024-04-23T02:36:15.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:15.194+0000 7f5a6b7fe700 1 -- 172.21.15.71:0/1897279697 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f5a6c059af0 con 0x7f5a7c101850 2024-04-23T02:36:15.198 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:36:15.198 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:36:15.198 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:36:15.198 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:36:15.199 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:36:15.199 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:36:15.199 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:36:15.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:15.197+0000 7f5a80871700 1 -- 172.21.15.71:0/1897279697 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5a6406cce0 msgr2=0x7f5a6406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:15.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:15.197+0000 7f5a80871700 1 --2- 172.21.15.71:0/1897279697 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5a6406cce0 0x7f5a6406f1a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000b5c0 tx=0x7f5a7001a040 comp rx=0 tx=0).stop 2024-04-23T02:36:15.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:15.197+0000 7f5a80871700 1 -- 172.21.15.71:0/1897279697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7c101850 msgr2=0x7f5a7c19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:15.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:15.197+0000 7f5a80871700 1 --2- 172.21.15.71:0/1897279697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7c101850 0x7f5a7c19caa0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00d900 tx=0x7f5a6c00dcc0 comp rx=0 tx=0).stop 2024-04-23T02:36:15.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:15.197+0000 7f5a80871700 1 -- 172.21.15.71:0/1897279697 shutdown_connections 2024-04-23T02:36:15.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:15.197+0000 7f5a80871700 1 --2- 172.21.15.71:0/1897279697 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5a6406cce0 0x7f5a6406f1a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:15.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:15.197+0000 7f5a80871700 1 --2- 172.21.15.71:0/1897279697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a7c100ea0 0x7f5a7c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:15.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:15.197+0000 7f5a80871700 1 --2- 172.21.15.71:0/1897279697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a7c101850 0x7f5a7c19caa0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:15.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:15.197+0000 7f5a80871700 1 -- 172.21.15.71:0/1897279697 >> 172.21.15.71:0/1897279697 conn(0x7f5a7c0fc750 msgr2=0x7f5a7c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:15.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:15.197+0000 7f5a80871700 1 -- 172.21.15.71:0/1897279697 shutdown_connections 2024-04-23T02:36:15.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:15.197+0000 7f5a80871700 1 -- 172.21.15.71:0/1897279697 wait complete. 2024-04-23T02:36:15.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:15 smithi192 ceph-mon[108617]: from='client.35078 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:36:15.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:15 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/210042478' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:36:15.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:15 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1792167993' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:36:15.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:15 smithi071 ceph-mon[144544]: from='client.35078 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:36:15.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:15 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/210042478' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:36:15.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:15 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1792167993' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:36:16.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:16 smithi192 ceph-mon[108617]: from='client.35090 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:36:16.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:16 smithi192 ceph-mon[108617]: pgmap v688: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:16.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:16 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1897279697' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:36:16.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:16 smithi071 ceph-mon[144544]: from='client.35090 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:36:16.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:16 smithi071 ceph-mon[144544]: pgmap v688: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:16.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:16 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1897279697' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:36:18.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:18 smithi071 ceph-mon[144544]: pgmap v689: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:18.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:18 smithi192 ceph-mon[108617]: pgmap v689: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:20.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:20 smithi071 ceph-mon[144544]: pgmap v690: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:20.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:36:20.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:20 smithi192 ceph-mon[108617]: pgmap v690: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:20.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:36:22.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:22 smithi071 ceph-mon[144544]: pgmap v691: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:22.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:22 smithi192 ceph-mon[108617]: pgmap v691: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:24.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:24 smithi071 ceph-mon[144544]: pgmap v692: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:24.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:24 smithi192 ceph-mon[108617]: pgmap v692: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:26.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:26 smithi071 ceph-mon[144544]: pgmap v693: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:26.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:26 smithi192 ceph-mon[108617]: pgmap v693: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:28.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:28 smithi071 ceph-mon[144544]: pgmap v694: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:28.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:28 smithi192 ceph-mon[108617]: pgmap v694: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:30.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:30 smithi192 ceph-mon[108617]: pgmap v695: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:30.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:30 smithi071 ceph-mon[144544]: pgmap v695: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:32.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:32 smithi071 ceph-mon[144544]: pgmap v696: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:32.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:32 smithi192 ceph-mon[108617]: pgmap v696: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:34.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:34 smithi071 ceph-mon[144544]: pgmap v697: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:34.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:34 smithi192 ceph-mon[108617]: pgmap v697: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:35.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:36:35.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:36:36.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:36 smithi071 ceph-mon[144544]: pgmap v698: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:36.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:36 smithi192 ceph-mon[108617]: pgmap v698: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:38.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:38 smithi071 ceph-mon[144544]: pgmap v699: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:38.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:38 smithi192 ceph-mon[108617]: pgmap v699: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:40.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:40 smithi192 ceph-mon[108617]: pgmap v700: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:40.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:40 smithi071 ceph-mon[144544]: pgmap v700: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:42.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:42 smithi071 ceph-mon[144544]: pgmap v701: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:42.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:42 smithi192 ceph-mon[108617]: pgmap v701: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:44.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:44 smithi071 ceph-mon[144544]: pgmap v702: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:44.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:44 smithi192 ceph-mon[108617]: pgmap v702: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:45.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.313+0000 7f1601192700 1 -- 172.21.15.71:0/241116892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15fc102780 msgr2=0x7f15fc10ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:45.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.313+0000 7f1601192700 1 --2- 172.21.15.71:0/241116892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15fc102780 0x7f15fc10ac70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009940 tx=0x7f15ec009c50 comp rx=0 tx=0).stop 2024-04-23T02:36:45.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.313+0000 7f1601192700 1 -- 172.21.15.71:0/241116892 shutdown_connections 2024-04-23T02:36:45.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.313+0000 7f1601192700 1 --2- 172.21.15.71:0/241116892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15fc102780 0x7f15fc10ac70 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:45.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.313+0000 7f1601192700 1 --2- 172.21.15.71:0/241116892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15fc101e60 0x7f15fc102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:45.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.313+0000 7f1601192700 1 -- 172.21.15.71:0/241116892 >> 172.21.15.71:0/241116892 conn(0x7f15fc0fb610 msgr2=0x7f15fc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:45.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.313+0000 7f1601192700 1 -- 172.21.15.71:0/241116892 shutdown_connections 2024-04-23T02:36:45.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.313+0000 7f1601192700 1 -- 172.21.15.71:0/241116892 wait complete. 2024-04-23T02:36:45.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.314+0000 7f1601192700 1 Processor -- start 2024-04-23T02:36:45.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.314+0000 7f1601192700 1 -- start start 2024-04-23T02:36:45.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.314+0000 7f1601192700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15fc101e60 0x7f15fc19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:45.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.314+0000 7f1601192700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15fc102780 0x7f15fc19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:45.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.314+0000 7f1601192700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc19d100 con 0x7f15fc101e60 2024-04-23T02:36:45.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.314+0000 7f1601192700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc196640 con 0x7f15fc102780 2024-04-23T02:36:45.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.314+0000 7f15fad9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15fc101e60 0x7f15fc19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:45.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.314+0000 7f15fad9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15fc101e60 0x7f15fc19c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47230/0 (socket says 172.21.15.71:47230) 2024-04-23T02:36:45.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.314+0000 7f15fad9d700 1 -- 172.21.15.71:0/189619424 learned_addr learned my addr 172.21.15.71:0/189619424 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:36:45.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.315+0000 7f15fa59c700 1 --2- 172.21.15.71:0/189619424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15fc102780 0x7f15fc19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:45.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.315+0000 7f15fad9d700 1 -- 172.21.15.71:0/189619424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15fc102780 msgr2=0x7f15fc19cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:45.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.315+0000 7f15fad9d700 1 --2- 172.21.15.71:0/189619424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15fc102780 0x7f15fc19cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:45.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.315+0000 7f15fad9d700 1 -- 172.21.15.71:0/189619424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec0095f0 con 0x7f15fc101e60 2024-04-23T02:36:45.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.315+0000 7f15fad9d700 1 --2- 172.21.15.71:0/189619424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15fc101e60 0x7f15fc19c5c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f15f000c930 tx=0x7f15f000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:45.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.315+0000 7f15e3fff700 1 -- 172.21.15.71:0/189619424 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15f0007ad0 con 0x7f15fc101e60 2024-04-23T02:36:45.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.315+0000 7f15e3fff700 1 -- 172.21.15.71:0/189619424 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f15f0007c30 con 0x7f15fc101e60 2024-04-23T02:36:45.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.315+0000 7f15e3fff700 1 -- 172.21.15.71:0/189619424 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15f0018630 con 0x7f15fc101e60 2024-04-23T02:36:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.315+0000 7f1601192700 1 -- 172.21.15.71:0/189619424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15fc196920 con 0x7f15fc101e60 2024-04-23T02:36:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.315+0000 7f1601192700 1 -- 172.21.15.71:0/189619424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15fc196e40 con 0x7f15fc101e60 2024-04-23T02:36:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.316+0000 7f1601192700 1 -- 172.21.15.71:0/189619424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15fc108370 con 0x7f15fc101e60 2024-04-23T02:36:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.317+0000 7f15e3fff700 1 -- 172.21.15.71:0/189619424 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f15f000f450 con 0x7f15fc101e60 2024-04-23T02:36:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.317+0000 7f15e3fff700 1 --2- 172.21.15.71:0/189619424 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f15e406cce0 0x7f15e406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.317+0000 7f15e3fff700 1 -- 172.21.15.71:0/189619424 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f15f008c780 con 0x7f15fc101e60 2024-04-23T02:36:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.317+0000 7f15fa59c700 1 --2- 172.21.15.71:0/189619424 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f15e406cce0 0x7f15e406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.318+0000 7f15fa59c700 1 --2- 172.21.15.71:0/189619424 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f15e406cce0 0x7f15e406f1a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00b7d0 tx=0x7f15ec005ea0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:45.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.322+0000 7f15e3fff700 1 -- 172.21.15.71:0/189619424 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f15f005a3b0 con 0x7f15fc101e60 2024-04-23T02:36:45.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.546+0000 7f1601192700 1 -- 172.21.15.71:0/189619424 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f15fc197c30 con 0x7f15e406cce0 2024-04-23T02:36:45.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.547+0000 7f15e3fff700 1 -- 172.21.15.71:0/189619424 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f15fc197c30 con 0x7f15e406cce0 2024-04-23T02:36:45.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.550+0000 7f1601192700 1 -- 172.21.15.71:0/189619424 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f15e406cce0 msgr2=0x7f15e406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:45.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.550+0000 7f1601192700 1 --2- 172.21.15.71:0/189619424 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f15e406cce0 0x7f15e406f1a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00b7d0 tx=0x7f15ec005ea0 comp rx=0 tx=0).stop 2024-04-23T02:36:45.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.550+0000 7f1601192700 1 -- 172.21.15.71:0/189619424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15fc101e60 msgr2=0x7f15fc19c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:45.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.550+0000 7f1601192700 1 --2- 172.21.15.71:0/189619424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15fc101e60 0x7f15fc19c5c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f15f000c930 tx=0x7f15f000ccf0 comp rx=0 tx=0).stop 2024-04-23T02:36:45.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.550+0000 7f1601192700 1 -- 172.21.15.71:0/189619424 shutdown_connections 2024-04-23T02:36:45.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.550+0000 7f1601192700 1 --2- 172.21.15.71:0/189619424 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f15e406cce0 0x7f15e406f1a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:45.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.550+0000 7f1601192700 1 --2- 172.21.15.71:0/189619424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15fc101e60 0x7f15fc19c5c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:45.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.550+0000 7f1601192700 1 --2- 172.21.15.71:0/189619424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15fc102780 0x7f15fc19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:45.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.550+0000 7f1601192700 1 -- 172.21.15.71:0/189619424 >> 172.21.15.71:0/189619424 conn(0x7f15fc0fb610 msgr2=0x7f15fc1054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:45.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.550+0000 7f1601192700 1 -- 172.21.15.71:0/189619424 shutdown_connections 2024-04-23T02:36:45.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.550+0000 7f1601192700 1 -- 172.21.15.71:0/189619424 wait complete. 2024-04-23T02:36:45.563 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:36:45.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.654+0000 7f49a44ce700 1 -- 172.21.15.71:0/3479320080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f499c105bd0 msgr2=0x7f499c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:45.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.654+0000 7f49a44ce700 1 --2- 172.21.15.71:0/3479320080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f499c105bd0 0x7f499c105fb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f498c009910 tx=0x7f498c009c20 comp rx=0 tx=0).stop 2024-04-23T02:36:45.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.654+0000 7f49a44ce700 1 -- 172.21.15.71:0/3479320080 shutdown_connections 2024-04-23T02:36:45.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.654+0000 7f49a44ce700 1 --2- 172.21.15.71:0/3479320080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f499c0ffc20 0x7f499c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:45.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.654+0000 7f49a44ce700 1 --2- 172.21.15.71:0/3479320080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f499c105bd0 0x7f499c105fb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:45.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.654+0000 7f49a44ce700 1 -- 172.21.15.71:0/3479320080 >> 172.21.15.71:0/3479320080 conn(0x7f499c075670 msgr2=0x7f499c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:45.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.654+0000 7f49a44ce700 1 -- 172.21.15.71:0/3479320080 shutdown_connections 2024-04-23T02:36:45.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.654+0000 7f49a44ce700 1 -- 172.21.15.71:0/3479320080 wait complete. 2024-04-23T02:36:45.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.655+0000 7f49a44ce700 1 Processor -- start 2024-04-23T02:36:45.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.655+0000 7f49a44ce700 1 -- start start 2024-04-23T02:36:45.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.655+0000 7f49a44ce700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f499c0ffc20 0x7f499c19c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:45.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.655+0000 7f49a44ce700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f499c105bd0 0x7f499c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:45.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.655+0000 7f49a44ce700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c19d160 con 0x7f499c0ffc20 2024-04-23T02:36:45.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.655+0000 7f49a44ce700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c196610 con 0x7f499c105bd0 2024-04-23T02:36:45.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.655+0000 7f49a226a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f499c0ffc20 0x7f499c19c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:45.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.656+0000 7f49a226a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f499c0ffc20 0x7f499c19c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47262/0 (socket says 172.21.15.71:47262) 2024-04-23T02:36:45.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.656+0000 7f49a226a700 1 -- 172.21.15.71:0/2931136827 learned_addr learned my addr 172.21.15.71:0/2931136827 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:36:45.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.656+0000 7f49a226a700 1 -- 172.21.15.71:0/2931136827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f499c105bd0 msgr2=0x7f499c19cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:45.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.656+0000 7f49a226a700 1 --2- 172.21.15.71:0/2931136827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f499c105bd0 0x7f499c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:45.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.656+0000 7f49a226a700 1 -- 172.21.15.71:0/2931136827 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c0095f0 con 0x7f499c0ffc20 2024-04-23T02:36:45.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.656+0000 7f49a226a700 1 --2- 172.21.15.71:0/2931136827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f499c0ffc20 0x7f499c19c590 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f498c006010 tx=0x7f498c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:45.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.656+0000 7f49937fe700 1 -- 172.21.15.71:0/2931136827 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f498c01d070 con 0x7f499c0ffc20 2024-04-23T02:36:45.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.656+0000 7f49937fe700 1 -- 172.21.15.71:0/2931136827 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f498c00f460 con 0x7f499c0ffc20 2024-04-23T02:36:45.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.656+0000 7f49a44ce700 1 -- 172.21.15.71:0/2931136827 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499c196890 con 0x7f499c0ffc20 2024-04-23T02:36:45.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.656+0000 7f49937fe700 1 -- 172.21.15.71:0/2931136827 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f498c021620 con 0x7f499c0ffc20 2024-04-23T02:36:45.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.656+0000 7f49a44ce700 1 -- 172.21.15.71:0/2931136827 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c196d80 con 0x7f499c0ffc20 2024-04-23T02:36:45.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.657+0000 7f49a44ce700 1 -- 172.21.15.71:0/2931136827 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499c04f2e0 con 0x7f499c0ffc20 2024-04-23T02:36:45.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.658+0000 7f49937fe700 1 -- 172.21.15.71:0/2931136827 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f498c00fa40 con 0x7f499c0ffc20 2024-04-23T02:36:45.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.658+0000 7f49937fe700 1 --2- 172.21.15.71:0/2931136827 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f498806cce0 0x7f498806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:45.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.658+0000 7f49937fe700 1 -- 172.21.15.71:0/2931136827 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f498c08d790 con 0x7f499c0ffc20 2024-04-23T02:36:45.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.658+0000 7f49a1a69700 1 --2- 172.21.15.71:0/2931136827 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f498806cce0 0x7f498806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:45.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.658+0000 7f49a1a69700 1 --2- 172.21.15.71:0/2931136827 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f498806cce0 0x7f498806f1a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4998005950 tx=0x7f49980058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:45.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.662+0000 7f49937fe700 1 -- 172.21.15.71:0/2931136827 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f498c05b290 con 0x7f499c0ffc20 2024-04-23T02:36:45.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.874+0000 7f49a44ce700 1 -- 172.21.15.71:0/2931136827 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f499c197140 con 0x7f498806cce0 2024-04-23T02:36:45.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.875+0000 7f49937fe700 1 -- 172.21.15.71:0/2931136827 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f499c197140 con 0x7f498806cce0 2024-04-23T02:36:45.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.877+0000 7f49a44ce700 1 -- 172.21.15.71:0/2931136827 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f498806cce0 msgr2=0x7f498806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:45.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.877+0000 7f49a44ce700 1 --2- 172.21.15.71:0/2931136827 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f498806cce0 0x7f498806f1a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4998005950 tx=0x7f49980058e0 comp rx=0 tx=0).stop 2024-04-23T02:36:45.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.877+0000 7f49a44ce700 1 -- 172.21.15.71:0/2931136827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f499c0ffc20 msgr2=0x7f499c19c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:45.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.877+0000 7f49a44ce700 1 --2- 172.21.15.71:0/2931136827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f499c0ffc20 0x7f499c19c590 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f498c006010 tx=0x7f498c00be20 comp rx=0 tx=0).stop 2024-04-23T02:36:45.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.877+0000 7f49a44ce700 1 -- 172.21.15.71:0/2931136827 shutdown_connections 2024-04-23T02:36:45.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.877+0000 7f49a44ce700 1 --2- 172.21.15.71:0/2931136827 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f498806cce0 0x7f498806f1a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:45.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.877+0000 7f49a44ce700 1 --2- 172.21.15.71:0/2931136827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f499c0ffc20 0x7f499c19c590 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:45.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.877+0000 7f49a44ce700 1 --2- 172.21.15.71:0/2931136827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f499c105bd0 0x7f499c19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:45.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.877+0000 7f49a44ce700 1 -- 172.21.15.71:0/2931136827 >> 172.21.15.71:0/2931136827 conn(0x7f499c075670 msgr2=0x7f499c0686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:45.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.878+0000 7f49a44ce700 1 -- 172.21.15.71:0/2931136827 shutdown_connections 2024-04-23T02:36:45.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.878+0000 7f49a44ce700 1 -- 172.21.15.71:0/2931136827 wait complete. 2024-04-23T02:36:45.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.980+0000 7fb036be3700 1 -- 172.21.15.71:0/2917376749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb030105bd0 msgr2=0x7fb030105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:45.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.980+0000 7fb036be3700 1 --2- 172.21.15.71:0/2917376749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb030105bd0 0x7fb030105fb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb024009910 tx=0x7fb024009c20 comp rx=0 tx=0).stop 2024-04-23T02:36:45.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.980+0000 7fb036be3700 1 -- 172.21.15.71:0/2917376749 shutdown_connections 2024-04-23T02:36:45.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.980+0000 7fb036be3700 1 --2- 172.21.15.71:0/2917376749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0300ffc20 0x7fb030100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:45.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.980+0000 7fb036be3700 1 --2- 172.21.15.71:0/2917376749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb030105bd0 0x7fb030105fb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:45.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.980+0000 7fb036be3700 1 -- 172.21.15.71:0/2917376749 >> 172.21.15.71:0/2917376749 conn(0x7fb030075670 msgr2=0x7fb030075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:45.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.980+0000 7fb036be3700 1 -- 172.21.15.71:0/2917376749 shutdown_connections 2024-04-23T02:36:45.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.980+0000 7fb036be3700 1 -- 172.21.15.71:0/2917376749 wait complete. 2024-04-23T02:36:45.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.981+0000 7fb036be3700 1 Processor -- start 2024-04-23T02:36:45.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.981+0000 7fb036be3700 1 -- start start 2024-04-23T02:36:45.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.981+0000 7fb036be3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0300ffc20 0x7fb03019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:45.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.981+0000 7fb036be3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb030105bd0 0x7fb03019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:45.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.981+0000 7fb036be3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0301988b0 con 0x7fb030105bd0 2024-04-23T02:36:45.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.981+0000 7fb036be3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb030198a20 con 0x7fb0300ffc20 2024-04-23T02:36:45.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.981+0000 7fb02ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb030105bd0 0x7fb03019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:45.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.981+0000 7fb02ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb030105bd0 0x7fb03019ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47272/0 (socket says 172.21.15.71:47272) 2024-04-23T02:36:45.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.981+0000 7fb02ffff700 1 -- 172.21.15.71:0/4132094544 learned_addr learned my addr 172.21.15.71:0/4132094544 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:36:45.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.981+0000 7fb03497f700 1 --2- 172.21.15.71:0/4132094544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0300ffc20 0x7fb03019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:45.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.981+0000 7fb02ffff700 1 -- 172.21.15.71:0/4132094544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0300ffc20 msgr2=0x7fb03019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:45.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.981+0000 7fb02ffff700 1 --2- 172.21.15.71:0/4132094544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0300ffc20 0x7fb03019e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:45.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.981+0000 7fb02ffff700 1 -- 172.21.15.71:0/4132094544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0240095f0 con 0x7fb030105bd0 2024-04-23T02:36:45.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.981+0000 7fb02ffff700 1 --2- 172.21.15.71:0/4132094544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb030105bd0 0x7fb03019ee00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00b4b0 tx=0x7fb01c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:45.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.982+0000 7fb02dffb700 1 -- 172.21.15.71:0/4132094544 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb01c011800 con 0x7fb030105bd0 2024-04-23T02:36:45.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.982+0000 7fb02dffb700 1 -- 172.21.15.71:0/4132094544 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb01c011e00 con 0x7fb030105bd0 2024-04-23T02:36:45.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.982+0000 7fb036be3700 1 -- 172.21.15.71:0/4132094544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb030198d00 con 0x7fb030105bd0 2024-04-23T02:36:45.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.982+0000 7fb02dffb700 1 -- 172.21.15.71:0/4132094544 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb01c010e60 con 0x7fb030105bd0 2024-04-23T02:36:45.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.982+0000 7fb036be3700 1 -- 172.21.15.71:0/4132094544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb030199250 con 0x7fb030105bd0 2024-04-23T02:36:45.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.983+0000 7fb036be3700 1 -- 172.21.15.71:0/4132094544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03004f2e0 con 0x7fb030105bd0 2024-04-23T02:36:45.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.983+0000 7fb02dffb700 1 -- 172.21.15.71:0/4132094544 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb01c011960 con 0x7fb030105bd0 2024-04-23T02:36:45.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.983+0000 7fb02dffb700 1 --2- 172.21.15.71:0/4132094544 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb02006cce0 0x7fb02006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:45.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.984+0000 7fb02dffb700 1 -- 172.21.15.71:0/4132094544 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb01c08c0e0 con 0x7fb030105bd0 2024-04-23T02:36:45.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.984+0000 7fb03497f700 1 --2- 172.21.15.71:0/4132094544 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb02006cce0 0x7fb02006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:45.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.984+0000 7fb03497f700 1 --2- 172.21.15.71:0/4132094544 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb02006cce0 0x7fb02006f1a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb024006010 tx=0x7fb02401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:45.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:45.988+0000 7fb02dffb700 1 -- 172.21.15.71:0/4132094544 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb01c056130 con 0x7fb030105bd0 2024-04-23T02:36:46.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.188+0000 7fb036be3700 1 -- 172.21.15.71:0/4132094544 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb030199540 con 0x7fb02006cce0 2024-04-23T02:36:46.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.199+0000 7fb02dffb700 1 -- 172.21.15.71:0/4132094544 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb030199540 con 0x7fb02006cce0 2024-04-23T02:36:46.201 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:36:46.202 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (20m) 14m ago 31m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:36:46.202 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (32m) 14m ago 32m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:36:46.202 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (30m) 13m ago 30m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:36:46.202 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (18m) 14m ago 31m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:36:46.202 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (18m) 13m ago 30m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:36:46.202 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (20m) 14m ago 31m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:36:46.202 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (14m) 14m ago 25m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:36:46.202 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (14m) 14m ago 25m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:36:46.202 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (25m) 13m ago 25m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:36:46.202 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (13m) 13m ago 25m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:36:46.202 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (22m) 14m ago 33m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:36:46.203 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (21m) 13m ago 30m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:36:46.203 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (18m) 14m ago 33m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:36:46.203 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (18m) 13m ago 30m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:36:46.203 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (21m) 14m ago 31m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:36:46.203 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (21m) 13m ago 30m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:36:46.203 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (17m) 14m ago 29m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:36:46.203 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (17m) 14m ago 29m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:36:46.203 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (16m) 14m ago 28m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:36:46.203 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (16m) 13m ago 28m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:36:46.203 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (15m) 13m ago 27m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:36:46.203 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (14m) 13m ago 27m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:36:46.203 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (20m) 14m ago 31m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:36:46.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.201+0000 7fb036be3700 1 -- 172.21.15.71:0/4132094544 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb02006cce0 msgr2=0x7fb02006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:46.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.201+0000 7fb036be3700 1 --2- 172.21.15.71:0/4132094544 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb02006cce0 0x7fb02006f1a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb024006010 tx=0x7fb02401a040 comp rx=0 tx=0).stop 2024-04-23T02:36:46.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.201+0000 7fb036be3700 1 -- 172.21.15.71:0/4132094544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb030105bd0 msgr2=0x7fb03019ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:46.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.201+0000 7fb036be3700 1 --2- 172.21.15.71:0/4132094544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb030105bd0 0x7fb03019ee00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00b4b0 tx=0x7fb01c00b7c0 comp rx=0 tx=0).stop 2024-04-23T02:36:46.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.201+0000 7fb036be3700 1 -- 172.21.15.71:0/4132094544 shutdown_connections 2024-04-23T02:36:46.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.201+0000 7fb036be3700 1 --2- 172.21.15.71:0/4132094544 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb02006cce0 0x7fb02006f1a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:46.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.201+0000 7fb036be3700 1 --2- 172.21.15.71:0/4132094544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0300ffc20 0x7fb03019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:46.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.201+0000 7fb036be3700 1 --2- 172.21.15.71:0/4132094544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb030105bd0 0x7fb03019ee00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:46.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.201+0000 7fb036be3700 1 -- 172.21.15.71:0/4132094544 >> 172.21.15.71:0/4132094544 conn(0x7fb030075670 msgr2=0x7fb030068780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:46.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.201+0000 7fb036be3700 1 -- 172.21.15.71:0/4132094544 shutdown_connections 2024-04-23T02:36:46.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.201+0000 7fb036be3700 1 -- 172.21.15.71:0/4132094544 wait complete. 2024-04-23T02:36:46.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.302+0000 7f09947cf700 1 -- 172.21.15.71:0/10119540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f098c105350 msgr2=0x7f098c105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:46.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.302+0000 7f09947cf700 1 --2- 172.21.15.71:0/10119540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f098c105350 0x7f098c105730 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0988009910 tx=0x7f0988009c20 comp rx=0 tx=0).stop 2024-04-23T02:36:46.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.302+0000 7f09947cf700 1 -- 172.21.15.71:0/10119540 shutdown_connections 2024-04-23T02:36:46.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.302+0000 7f09947cf700 1 --2- 172.21.15.71:0/10119540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f098c0ff340 0x7f098c0ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:46.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.302+0000 7f09947cf700 1 --2- 172.21.15.71:0/10119540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f098c105350 0x7f098c105730 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:46.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.302+0000 7f09947cf700 1 -- 172.21.15.71:0/10119540 >> 172.21.15.71:0/10119540 conn(0x7f098c074e30 msgr2=0x7f098c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:46.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.302+0000 7f09947cf700 1 -- 172.21.15.71:0/10119540 shutdown_connections 2024-04-23T02:36:46.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.302+0000 7f09947cf700 1 -- 172.21.15.71:0/10119540 wait complete. 2024-04-23T02:36:46.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.303+0000 7f09947cf700 1 Processor -- start 2024-04-23T02:36:46.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.303+0000 7f09947cf700 1 -- start start 2024-04-23T02:36:46.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.303+0000 7f09947cf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f098c0ff340 0x7f098c1956e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:46.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.303+0000 7f09947cf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f098c105350 0x7f098c195c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:46.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.303+0000 7f09947cf700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c1962b0 con 0x7f098c105350 2024-04-23T02:36:46.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.303+0000 7f09947cf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c198f90 con 0x7f098c0ff340 2024-04-23T02:36:46.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.303+0000 7f0991d6a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f098c105350 0x7f098c195c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:46.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.303+0000 7f0991d6a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f098c105350 0x7f098c195c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47288/0 (socket says 172.21.15.71:47288) 2024-04-23T02:36:46.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.303+0000 7f0991d6a700 1 -- 172.21.15.71:0/3278263086 learned_addr learned my addr 172.21.15.71:0/3278263086 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:36:46.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.303+0000 7f099256b700 1 --2- 172.21.15.71:0/3278263086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f098c0ff340 0x7f098c1956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:46.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.303+0000 7f0991d6a700 1 -- 172.21.15.71:0/3278263086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f098c0ff340 msgr2=0x7f098c1956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:46.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.303+0000 7f0991d6a700 1 --2- 172.21.15.71:0/3278263086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f098c0ff340 0x7f098c1956e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:46.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.303+0000 7f0991d6a700 1 -- 172.21.15.71:0/3278263086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09880095f0 con 0x7f098c105350 2024-04-23T02:36:46.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.303+0000 7f099256b700 1 --2- 172.21.15.71:0/3278263086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f098c0ff340 0x7f098c1956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:36:46.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.304+0000 7f0991d6a700 1 --2- 172.21.15.71:0/3278263086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f098c105350 0x7f098c195c20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f097c00d8d0 tx=0x7f097c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:46.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.304+0000 7f09837fe700 1 -- 172.21.15.71:0/3278263086 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f097c009940 con 0x7f098c105350 2024-04-23T02:36:46.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.304+0000 7f09947cf700 1 -- 172.21.15.71:0/3278263086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f098c199270 con 0x7f098c105350 2024-04-23T02:36:46.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.304+0000 7f09837fe700 1 -- 172.21.15.71:0/3278263086 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f097c010460 con 0x7f098c105350 2024-04-23T02:36:46.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.304+0000 7f09837fe700 1 -- 172.21.15.71:0/3278263086 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f097c00f5f0 con 0x7f098c105350 2024-04-23T02:36:46.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.304+0000 7f09947cf700 1 -- 172.21.15.71:0/3278263086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098c1997c0 con 0x7f098c105350 2024-04-23T02:36:46.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.305+0000 7f09947cf700 1 -- 172.21.15.71:0/3278263086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098c107850 con 0x7f098c105350 2024-04-23T02:36:46.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.305+0000 7f09837fe700 1 -- 172.21.15.71:0/3278263086 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f097c0105d0 con 0x7f098c105350 2024-04-23T02:36:46.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.306+0000 7f09837fe700 1 --2- 172.21.15.71:0/3278263086 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f097806cce0 0x7f097806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:46.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.306+0000 7f09837fe700 1 -- 172.21.15.71:0/3278263086 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f097c08c590 con 0x7f098c105350 2024-04-23T02:36:46.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.306+0000 7f099256b700 1 --2- 172.21.15.71:0/3278263086 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f097806cce0 0x7f097806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:46.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.306+0000 7f099256b700 1 --2- 172.21.15.71:0/3278263086 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f097806cce0 0x7f097806f1a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0988006010 tx=0x7f098801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:46.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.310+0000 7f09837fe700 1 -- 172.21.15.71:0/3278263086 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f097c056730 con 0x7f098c105350 2024-04-23T02:36:46.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.585+0000 7f09947cf700 1 -- 172.21.15.71:0/3278263086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f098c04f2e0 con 0x7f098c105350 2024-04-23T02:36:46.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.586+0000 7f09837fe700 1 -- 172.21.15.71:0/3278263086 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f097c010880 con 0x7f098c105350 2024-04-23T02:36:46.588 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:36:46.589 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:36:46.589 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:36:46.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:36:46.589 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:36:46.589 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:36:46.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:36:46.589 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:36:46.589 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:36:46.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:36:46.589 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:36:46.589 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:36:46.589 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:36:46.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:36:46.590 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:36:46.590 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:36:46.590 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:36:46.590 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:36:46.590 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:36:46.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.588+0000 7f09947cf700 1 -- 172.21.15.71:0/3278263086 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f097806cce0 msgr2=0x7f097806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:46.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.588+0000 7f09947cf700 1 --2- 172.21.15.71:0/3278263086 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f097806cce0 0x7f097806f1a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0988006010 tx=0x7f098801a040 comp rx=0 tx=0).stop 2024-04-23T02:36:46.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.588+0000 7f09947cf700 1 -- 172.21.15.71:0/3278263086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f098c105350 msgr2=0x7f098c195c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:46.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.588+0000 7f09947cf700 1 --2- 172.21.15.71:0/3278263086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f098c105350 0x7f098c195c20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f097c00d8d0 tx=0x7f097c00dc90 comp rx=0 tx=0).stop 2024-04-23T02:36:46.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.588+0000 7f09947cf700 1 -- 172.21.15.71:0/3278263086 shutdown_connections 2024-04-23T02:36:46.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.588+0000 7f09947cf700 1 --2- 172.21.15.71:0/3278263086 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f097806cce0 0x7f097806f1a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:46.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.588+0000 7f09947cf700 1 --2- 172.21.15.71:0/3278263086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f098c0ff340 0x7f098c1956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:46.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.588+0000 7f09947cf700 1 --2- 172.21.15.71:0/3278263086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f098c105350 0x7f098c195c20 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:46.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.588+0000 7f09947cf700 1 -- 172.21.15.71:0/3278263086 >> 172.21.15.71:0/3278263086 conn(0x7f098c074e30 msgr2=0x7f098c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:46.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.588+0000 7f09947cf700 1 -- 172.21.15.71:0/3278263086 shutdown_connections 2024-04-23T02:36:46.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.588+0000 7f09947cf700 1 -- 172.21.15.71:0/3278263086 wait complete. 2024-04-23T02:36:46.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.689+0000 7f30033a6700 1 -- 172.21.15.71:0/2068024993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ffc100750 msgr2=0x7f2ffc1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:46.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.689+0000 7f30033a6700 1 --2- 172.21.15.71:0/2068024993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ffc100750 0x7f2ffc1047a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009910 tx=0x7f2fec009c20 comp rx=0 tx=0).stop 2024-04-23T02:36:46.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.689+0000 7f30033a6700 1 -- 172.21.15.71:0/2068024993 shutdown_connections 2024-04-23T02:36:46.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.690+0000 7f30033a6700 1 --2- 172.21.15.71:0/2068024993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ffc100750 0x7f2ffc1047a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:46.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.690+0000 7f30033a6700 1 --2- 172.21.15.71:0/2068024993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ffc0ffda0 0x7f2ffc100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:46.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.690+0000 7f30033a6700 1 -- 172.21.15.71:0/2068024993 >> 172.21.15.71:0/2068024993 conn(0x7f2ffc0fb650 msgr2=0x7f2ffc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:46.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.690+0000 7f30033a6700 1 -- 172.21.15.71:0/2068024993 shutdown_connections 2024-04-23T02:36:46.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.690+0000 7f30033a6700 1 -- 172.21.15.71:0/2068024993 wait complete. 2024-04-23T02:36:46.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.690+0000 7f30033a6700 1 Processor -- start 2024-04-23T02:36:46.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.690+0000 7f30033a6700 1 -- start start 2024-04-23T02:36:46.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.690+0000 7f30033a6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ffc0ffda0 0x7f2ffc19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:46.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.690+0000 7f30033a6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ffc100750 0x7f2ffc19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:46.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.690+0000 7f30033a6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc19d150 con 0x7f2ffc0ffda0 2024-04-23T02:36:46.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.690+0000 7f30033a6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc196600 con 0x7f2ffc100750 2024-04-23T02:36:46.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.691+0000 7f3001142700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ffc0ffda0 0x7f2ffc19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:46.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.691+0000 7f3001142700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ffc0ffda0 0x7f2ffc19c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47304/0 (socket says 172.21.15.71:47304) 2024-04-23T02:36:46.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.691+0000 7f3001142700 1 -- 172.21.15.71:0/3981239434 learned_addr learned my addr 172.21.15.71:0/3981239434 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:36:46.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.691+0000 7f3000941700 1 --2- 172.21.15.71:0/3981239434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ffc100750 0x7f2ffc19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:46.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.691+0000 7f3001142700 1 -- 172.21.15.71:0/3981239434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ffc100750 msgr2=0x7f2ffc19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:46.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.691+0000 7f3001142700 1 --2- 172.21.15.71:0/3981239434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ffc100750 0x7f2ffc19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:46.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.691+0000 7f3001142700 1 -- 172.21.15.71:0/3981239434 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fec0095f0 con 0x7f2ffc0ffda0 2024-04-23T02:36:46.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.691+0000 7f3000941700 1 --2- 172.21.15.71:0/3981239434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ffc100750 0x7f2ffc19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:36:46.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.691+0000 7f3001142700 1 --2- 172.21.15.71:0/3981239434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ffc0ffda0 0x7f2ffc19c580 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800d8d0 tx=0x7f2ff800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:46.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.691+0000 7f2ff27fc700 1 -- 172.21.15.71:0/3981239434 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ff8009940 con 0x7f2ffc0ffda0 2024-04-23T02:36:46.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.691+0000 7f2ff27fc700 1 -- 172.21.15.71:0/3981239434 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2ff8010460 con 0x7f2ffc0ffda0 2024-04-23T02:36:46.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.691+0000 7f30033a6700 1 -- 172.21.15.71:0/3981239434 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ffc1968e0 con 0x7f2ffc0ffda0 2024-04-23T02:36:46.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.691+0000 7f2ff27fc700 1 -- 172.21.15.71:0/3981239434 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ff800f5f0 con 0x7f2ffc0ffda0 2024-04-23T02:36:46.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.691+0000 7f30033a6700 1 -- 172.21.15.71:0/3981239434 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ffc196e30 con 0x7f2ffc0ffda0 2024-04-23T02:36:46.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.692+0000 7f30033a6700 1 -- 172.21.15.71:0/3981239434 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ffc1091b0 con 0x7f2ffc0ffda0 2024-04-23T02:36:46.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.692+0000 7f2ff27fc700 1 -- 172.21.15.71:0/3981239434 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2ff80105d0 con 0x7f2ffc0ffda0 2024-04-23T02:36:46.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.693+0000 7f2ff27fc700 1 --2- 172.21.15.71:0/3981239434 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2fe806cce0 0x7f2fe806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:46.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.693+0000 7f2ff27fc700 1 -- 172.21.15.71:0/3981239434 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2ff808c4f0 con 0x7f2ffc0ffda0 2024-04-23T02:36:46.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.693+0000 7f3000941700 1 --2- 172.21.15.71:0/3981239434 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2fe806cce0 0x7f2fe806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:46.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.693+0000 7f3000941700 1 --2- 172.21.15.71:0/3981239434 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2fe806cce0 0x7f2fe806f1a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0fbdc0 tx=0x7f2fec01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:46.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.697+0000 7f2ff27fc700 1 -- 172.21.15.71:0/3981239434 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2ff805a070 con 0x7f2ffc0ffda0 2024-04-23T02:36:46.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:46 smithi192 ceph-mon[108617]: pgmap v703: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:46.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.933+0000 7f30033a6700 1 -- 172.21.15.71:0/3981239434 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2ffc04f2e0 con 0x7f2ffc0ffda0 2024-04-23T02:36:46.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.933+0000 7f2ff27fc700 1 -- 172.21.15.71:0/3981239434 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f2ff8016740 con 0x7f2ffc0ffda0 2024-04-23T02:36:46.937 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:36:46.937 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:36:46.937 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:36:46.937 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:36:46.937 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:36:46.937 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:36:46.937 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:36:46.937 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:36:46.937 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:36:46.938 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:36:46.938 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:36:46.938 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:36:46.938 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:36:46.938 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:36:46.938 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:36:46.938 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:36:46.938 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:36:46.938 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:36:46.938 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:36:46.938 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:36:46.938 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:36:46.938 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:36:46.938 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:36:46.939 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:36:46.939 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:36:46.939 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:36:46.939 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:36:46.939 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:36:46.939 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:36:46.939 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:36:46.939 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:36:46.939 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:36:46.939 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:36:46.939 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:36:46.939 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:36:46.939 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:36:46.939 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:36:46.939 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:36:46.940 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:36:46.940 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:36:46.940 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:36:46.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.935+0000 7f30033a6700 1 -- 172.21.15.71:0/3981239434 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2fe806cce0 msgr2=0x7f2fe806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:46.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.935+0000 7f30033a6700 1 --2- 172.21.15.71:0/3981239434 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2fe806cce0 0x7f2fe806f1a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0fbdc0 tx=0x7f2fec01a040 comp rx=0 tx=0).stop 2024-04-23T02:36:46.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.935+0000 7f30033a6700 1 -- 172.21.15.71:0/3981239434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ffc0ffda0 msgr2=0x7f2ffc19c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:46.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.935+0000 7f30033a6700 1 --2- 172.21.15.71:0/3981239434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ffc0ffda0 0x7f2ffc19c580 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800d8d0 tx=0x7f2ff800dc90 comp rx=0 tx=0).stop 2024-04-23T02:36:46.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.935+0000 7f30033a6700 1 -- 172.21.15.71:0/3981239434 shutdown_connections 2024-04-23T02:36:46.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.935+0000 7f30033a6700 1 --2- 172.21.15.71:0/3981239434 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2fe806cce0 0x7f2fe806f1a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:46.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.936+0000 7f30033a6700 1 --2- 172.21.15.71:0/3981239434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ffc0ffda0 0x7f2ffc19c580 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:46.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.936+0000 7f30033a6700 1 --2- 172.21.15.71:0/3981239434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ffc100750 0x7f2ffc19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:46.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.936+0000 7f30033a6700 1 -- 172.21.15.71:0/3981239434 >> 172.21.15.71:0/3981239434 conn(0x7f2ffc0fb650 msgr2=0x7f2ffc069190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:46.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.936+0000 7f30033a6700 1 -- 172.21.15.71:0/3981239434 shutdown_connections 2024-04-23T02:36:46.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:46.936+0000 7f30033a6700 1 -- 172.21.15.71:0/3981239434 wait complete. 2024-04-23T02:36:46.941 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:36:46.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:46 smithi071 ceph-mon[144544]: pgmap v703: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:47.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.040+0000 7f9ee1d50700 1 -- 172.21.15.71:0/2837372528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9edc101e00 msgr2=0x7f9edc1021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:47.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.040+0000 7f9ee1d50700 1 --2- 172.21.15.71:0/2837372528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9edc101e00 0x7f9edc1021e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009910 tx=0x7f9ec4009c20 comp rx=0 tx=0).stop 2024-04-23T02:36:47.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.040+0000 7f9ee1d50700 1 -- 172.21.15.71:0/2837372528 shutdown_connections 2024-04-23T02:36:47.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.040+0000 7f9ee1d50700 1 --2- 172.21.15.71:0/2837372528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9edc102720 0x7f9edc10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:47.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.040+0000 7f9ee1d50700 1 --2- 172.21.15.71:0/2837372528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9edc101e00 0x7f9edc1021e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:47.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.040+0000 7f9ee1d50700 1 -- 172.21.15.71:0/2837372528 >> 172.21.15.71:0/2837372528 conn(0x7f9edc0fb650 msgr2=0x7f9edc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:47.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.040+0000 7f9ee1d50700 1 -- 172.21.15.71:0/2837372528 shutdown_connections 2024-04-23T02:36:47.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.040+0000 7f9ee1d50700 1 -- 172.21.15.71:0/2837372528 wait complete. 2024-04-23T02:36:47.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.041+0000 7f9ee1d50700 1 Processor -- start 2024-04-23T02:36:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.041+0000 7f9ee1d50700 1 -- start start 2024-04-23T02:36:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.041+0000 7f9ee1d50700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9edc101e00 0x7f9edc19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.041+0000 7f9ee1d50700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9edc102720 0x7f9edc19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.041+0000 7f9ee1d50700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc19d200 con 0x7f9edc102720 2024-04-23T02:36:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.041+0000 7f9ee1d50700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc1966b0 con 0x7f9edc101e00 2024-04-23T02:36:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.041+0000 7f9edaffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9edc102720 0x7f9edc19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.041+0000 7f9edaffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9edc102720 0x7f9edc19cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47324/0 (socket says 172.21.15.71:47324) 2024-04-23T02:36:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.041+0000 7f9edaffd700 1 -- 172.21.15.71:0/1121785551 learned_addr learned my addr 172.21.15.71:0/1121785551 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:36:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.041+0000 7f9edb7fe700 1 --2- 172.21.15.71:0/1121785551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9edc101e00 0x7f9edc19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.042+0000 7f9edaffd700 1 -- 172.21.15.71:0/1121785551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9edc101e00 msgr2=0x7f9edc19c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.042+0000 7f9edaffd700 1 --2- 172.21.15.71:0/1121785551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9edc101e00 0x7f9edc19c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.042+0000 7f9edaffd700 1 -- 172.21.15.71:0/1121785551 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec40095f0 con 0x7f9edc102720 2024-04-23T02:36:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.042+0000 7f9edb7fe700 1 --2- 172.21.15.71:0/1121785551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9edc101e00 0x7f9edc19c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:36:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.042+0000 7f9edaffd700 1 --2- 172.21.15.71:0/1121785551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9edc102720 0x7f9edc19cb70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00c8f0 tx=0x7f9ecc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.042+0000 7f9ed8ff9700 1 -- 172.21.15.71:0/1121785551 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ecc007ad0 con 0x7f9edc102720 2024-04-23T02:36:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.042+0000 7f9ed8ff9700 1 -- 172.21.15.71:0/1121785551 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9ecc00ce90 con 0x7f9edc102720 2024-04-23T02:36:47.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.042+0000 7f9ee1d50700 1 -- 172.21.15.71:0/1121785551 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9edc196990 con 0x7f9edc102720 2024-04-23T02:36:47.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.042+0000 7f9ed8ff9700 1 -- 172.21.15.71:0/1121785551 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ecc018630 con 0x7f9edc102720 2024-04-23T02:36:47.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.042+0000 7f9ee1d50700 1 -- 172.21.15.71:0/1121785551 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9edc196ee0 con 0x7f9edc102720 2024-04-23T02:36:47.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.043+0000 7f9ee1d50700 1 -- 172.21.15.71:0/1121785551 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9edc108310 con 0x7f9edc102720 2024-04-23T02:36:47.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.043+0000 7f9ed8ff9700 1 -- 172.21.15.71:0/1121785551 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9ecc007c30 con 0x7f9edc102720 2024-04-23T02:36:47.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.044+0000 7f9ed8ff9700 1 --2- 172.21.15.71:0/1121785551 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9ec806cce0 0x7f9ec806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:47.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.044+0000 7f9ed8ff9700 1 -- 172.21.15.71:0/1121785551 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9ecc08c560 con 0x7f9edc102720 2024-04-23T02:36:47.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.044+0000 7f9edb7fe700 1 --2- 172.21.15.71:0/1121785551 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9ec806cce0 0x7f9ec806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:47.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.044+0000 7f9edb7fe700 1 --2- 172.21.15.71:0/1121785551 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9ec806cce0 0x7f9ec806f1a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400b5c0 tx=0x7f9ec4005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:47.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.048+0000 7f9ed8ff9700 1 -- 172.21.15.71:0/1121785551 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9ecc05a060 con 0x7f9edc102720 2024-04-23T02:36:47.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.257+0000 7f9ee1d50700 1 -- 172.21.15.71:0/1121785551 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9edc197160 con 0x7f9ec806cce0 2024-04-23T02:36:47.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.259+0000 7f9ed8ff9700 1 -- 172.21.15.71:0/1121785551 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9edc197160 con 0x7f9ec806cce0 2024-04-23T02:36:47.261 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:36:47.261 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:36:47.262 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:36:47.262 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:36:47.262 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:36:47.262 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:36:47.262 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:36:47.262 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:36:47.262 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:36:47.262 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:36:47.262 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:36:47.262 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:36:47.262 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:36:47.262 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:36:47.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.261+0000 7f9ee1d50700 1 -- 172.21.15.71:0/1121785551 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9ec806cce0 msgr2=0x7f9ec806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:47.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.261+0000 7f9ee1d50700 1 --2- 172.21.15.71:0/1121785551 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9ec806cce0 0x7f9ec806f1a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400b5c0 tx=0x7f9ec4005960 comp rx=0 tx=0).stop 2024-04-23T02:36:47.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.261+0000 7f9ee1d50700 1 -- 172.21.15.71:0/1121785551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9edc102720 msgr2=0x7f9edc19cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:47.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.261+0000 7f9ee1d50700 1 --2- 172.21.15.71:0/1121785551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9edc102720 0x7f9edc19cb70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00c8f0 tx=0x7f9ecc00ccb0 comp rx=0 tx=0).stop 2024-04-23T02:36:47.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.261+0000 7f9ee1d50700 1 -- 172.21.15.71:0/1121785551 shutdown_connections 2024-04-23T02:36:47.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.261+0000 7f9ee1d50700 1 --2- 172.21.15.71:0/1121785551 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9ec806cce0 0x7f9ec806f1a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:47.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.261+0000 7f9ee1d50700 1 --2- 172.21.15.71:0/1121785551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9edc101e00 0x7f9edc19c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:47.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.261+0000 7f9ee1d50700 1 --2- 172.21.15.71:0/1121785551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9edc102720 0x7f9edc19cb70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:47.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.261+0000 7f9ee1d50700 1 -- 172.21.15.71:0/1121785551 >> 172.21.15.71:0/1121785551 conn(0x7f9edc0fb650 msgr2=0x7f9edc105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:47.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.261+0000 7f9ee1d50700 1 -- 172.21.15.71:0/1121785551 shutdown_connections 2024-04-23T02:36:47.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.261+0000 7f9ee1d50700 1 -- 172.21.15.71:0/1121785551 wait complete. 2024-04-23T02:36:47.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.363+0000 7fcf01af2700 1 -- 172.21.15.71:0/3109070673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcefc0ffc20 msgr2=0x7fcefc100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:47.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.363+0000 7fcf01af2700 1 --2- 172.21.15.71:0/3109070673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcefc0ffc20 0x7fcefc100080 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009910 tx=0x7fcef0009c20 comp rx=0 tx=0).stop 2024-04-23T02:36:47.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.363+0000 7fcf01af2700 1 -- 172.21.15.71:0/3109070673 shutdown_connections 2024-04-23T02:36:47.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.363+0000 7fcf01af2700 1 --2- 172.21.15.71:0/3109070673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcefc0ffc20 0x7fcefc100080 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:47.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.363+0000 7fcf01af2700 1 --2- 172.21.15.71:0/3109070673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcefc105bd0 0x7fcefc105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:47.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.363+0000 7fcf01af2700 1 -- 172.21.15.71:0/3109070673 >> 172.21.15.71:0/3109070673 conn(0x7fcefc075670 msgr2=0x7fcefc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:47.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.363+0000 7fcf01af2700 1 -- 172.21.15.71:0/3109070673 shutdown_connections 2024-04-23T02:36:47.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.364+0000 7fcf01af2700 1 -- 172.21.15.71:0/3109070673 wait complete. 2024-04-23T02:36:47.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.364+0000 7fcf01af2700 1 Processor -- start 2024-04-23T02:36:47.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.364+0000 7fcf01af2700 1 -- start start 2024-04-23T02:36:47.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.364+0000 7fcf01af2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcefc0ffc20 0x7fcefc19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:47.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.364+0000 7fcf01af2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcefc105bd0 0x7fcefc19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:47.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.364+0000 7fcf01af2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc19d150 con 0x7fcefc0ffc20 2024-04-23T02:36:47.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.364+0000 7fcf01af2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc196600 con 0x7fcefc105bd0 2024-04-23T02:36:47.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.365+0000 7fcefb7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcefc0ffc20 0x7fcefc19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:47.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.365+0000 7fcefb7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcefc0ffc20 0x7fcefc19c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47350/0 (socket says 172.21.15.71:47350) 2024-04-23T02:36:47.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.365+0000 7fcefb7fe700 1 -- 172.21.15.71:0/2861474345 learned_addr learned my addr 172.21.15.71:0/2861474345 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:36:47.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.365+0000 7fcefaffd700 1 --2- 172.21.15.71:0/2861474345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcefc105bd0 0x7fcefc19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:47.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.365+0000 7fcefb7fe700 1 -- 172.21.15.71:0/2861474345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcefc105bd0 msgr2=0x7fcefc19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:47.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.365+0000 7fcefb7fe700 1 --2- 172.21.15.71:0/2861474345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcefc105bd0 0x7fcefc19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:47.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.365+0000 7fcefb7fe700 1 -- 172.21.15.71:0/2861474345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef00095f0 con 0x7fcefc0ffc20 2024-04-23T02:36:47.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.365+0000 7fcefb7fe700 1 --2- 172.21.15.71:0/2861474345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcefc0ffc20 0x7fcefc19c580 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fceec00d8d0 tx=0x7fceec00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:47.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.365+0000 7fcef8ff9700 1 -- 172.21.15.71:0/2861474345 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fceec0098a0 con 0x7fcefc0ffc20 2024-04-23T02:36:47.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.365+0000 7fcf01af2700 1 -- 172.21.15.71:0/2861474345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcefc1968e0 con 0x7fcefc0ffc20 2024-04-23T02:36:47.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.365+0000 7fcef8ff9700 1 -- 172.21.15.71:0/2861474345 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fceec010460 con 0x7fcefc0ffc20 2024-04-23T02:36:47.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.365+0000 7fcef8ff9700 1 -- 172.21.15.71:0/2861474345 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fceec009ea0 con 0x7fcefc0ffc20 2024-04-23T02:36:47.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.365+0000 7fcf01af2700 1 -- 172.21.15.71:0/2861474345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcefc196e30 con 0x7fcefc0ffc20 2024-04-23T02:36:47.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.366+0000 7fcf01af2700 1 -- 172.21.15.71:0/2861474345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcefc04f2e0 con 0x7fcefc0ffc20 2024-04-23T02:36:47.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.366+0000 7fcef8ff9700 1 -- 172.21.15.71:0/2861474345 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fceec0105d0 con 0x7fcefc0ffc20 2024-04-23T02:36:47.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.367+0000 7fcef8ff9700 1 --2- 172.21.15.71:0/2861474345 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcee406cce0 0x7fcee406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:36:47.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.367+0000 7fcef8ff9700 1 -- 172.21.15.71:0/2861474345 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fceec08c480 con 0x7fcefc0ffc20 2024-04-23T02:36:47.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.367+0000 7fcefaffd700 1 --2- 172.21.15.71:0/2861474345 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcee406cce0 0x7fcee406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:36:47.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.367+0000 7fcefaffd700 1 --2- 172.21.15.71:0/2861474345 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcee406cce0 0x7fcee406f1a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcefc075de0 tx=0x7fcef001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:36:47.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.371+0000 7fcef8ff9700 1 -- 172.21.15.71:0/2861474345 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fceec059f80 con 0x7fcefc0ffc20 2024-04-23T02:36:47.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.642+0000 7fcf01af2700 1 -- 172.21.15.71:0/2861474345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcefc1970f0 con 0x7fcefc0ffc20 2024-04-23T02:36:47.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.642+0000 7fcef8ff9700 1 -- 172.21.15.71:0/2861474345 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fceec010880 con 0x7fcefc0ffc20 2024-04-23T02:36:47.645 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:36:47.645 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:36:47.645 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:36:47.645 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:36:47.645 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:36:47.645 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:36:47.646 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:36:47.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.644+0000 7fcf01af2700 1 -- 172.21.15.71:0/2861474345 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcee406cce0 msgr2=0x7fcee406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:47.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.645+0000 7fcf01af2700 1 --2- 172.21.15.71:0/2861474345 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcee406cce0 0x7fcee406f1a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcefc075de0 tx=0x7fcef001a040 comp rx=0 tx=0).stop 2024-04-23T02:36:47.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.645+0000 7fcf01af2700 1 -- 172.21.15.71:0/2861474345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcefc0ffc20 msgr2=0x7fcefc19c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:36:47.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.645+0000 7fcf01af2700 1 --2- 172.21.15.71:0/2861474345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcefc0ffc20 0x7fcefc19c580 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fceec00d8d0 tx=0x7fceec00dbe0 comp rx=0 tx=0).stop 2024-04-23T02:36:47.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.645+0000 7fcf01af2700 1 -- 172.21.15.71:0/2861474345 shutdown_connections 2024-04-23T02:36:47.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.645+0000 7fcf01af2700 1 --2- 172.21.15.71:0/2861474345 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcee406cce0 0x7fcee406f1a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:47.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.645+0000 7fcf01af2700 1 --2- 172.21.15.71:0/2861474345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcefc0ffc20 0x7fcefc19c580 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:47.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.645+0000 7fcf01af2700 1 --2- 172.21.15.71:0/2861474345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcefc105bd0 0x7fcefc19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:36:47.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.645+0000 7fcf01af2700 1 -- 172.21.15.71:0/2861474345 >> 172.21.15.71:0/2861474345 conn(0x7fcefc075670 msgr2=0x7fcefc068720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:36:47.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.645+0000 7fcf01af2700 1 -- 172.21.15.71:0/2861474345 shutdown_connections 2024-04-23T02:36:47.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:36:47.645+0000 7fcf01af2700 1 -- 172.21.15.71:0/2861474345 wait complete. 2024-04-23T02:36:47.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:47 smithi192 ceph-mon[108617]: from='client.35098 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:36:47.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:47 smithi192 ceph-mon[108617]: from='client.35102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:36:47.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:47 smithi192 ceph-mon[108617]: from='client.35106 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:36:47.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:47 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3278263086' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:36:47.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:47 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3981239434' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:36:47.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:47 smithi071 ceph-mon[144544]: from='client.35098 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:36:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:47 smithi071 ceph-mon[144544]: from='client.35102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:36:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:47 smithi071 ceph-mon[144544]: from='client.35106 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:36:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:47 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3278263086' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:36:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:47 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3981239434' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:36:48.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:48 smithi192 ceph-mon[108617]: pgmap v704: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:48.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:48 smithi192 ceph-mon[108617]: from='client.35118 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:36:48.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:48 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2861474345' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:36:48.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:48 smithi071 ceph-mon[144544]: pgmap v704: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:48.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:48 smithi071 ceph-mon[144544]: from='client.35118 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:36:48.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:48 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2861474345' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:36:50.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:50 smithi071 ceph-mon[144544]: pgmap v705: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:50.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:36:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:50 smithi192 ceph-mon[108617]: pgmap v705: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:36:52.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:52 smithi192 ceph-mon[108617]: pgmap v706: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:52.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:52 smithi071 ceph-mon[144544]: pgmap v706: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:54.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:54 smithi192 ceph-mon[108617]: pgmap v707: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:54.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:54 smithi071 ceph-mon[144544]: pgmap v707: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:56.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:56 smithi192 ceph-mon[108617]: pgmap v708: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:56.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:56 smithi071 ceph-mon[144544]: pgmap v708: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:58.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:36:58 smithi192 ceph-mon[108617]: pgmap v709: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:36:58.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:36:58 smithi071 ceph-mon[144544]: pgmap v709: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:00.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:00 smithi192 ceph-mon[108617]: pgmap v710: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:00.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:00 smithi071 ceph-mon[144544]: pgmap v710: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:02.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:02 smithi192 ceph-mon[108617]: pgmap v711: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:02.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:02 smithi071 ceph-mon[144544]: pgmap v711: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:04.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:04 smithi192 ceph-mon[108617]: pgmap v712: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:04.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:04 smithi071 ceph-mon[144544]: pgmap v712: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:37:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:37:06.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:06 smithi192 ceph-mon[108617]: pgmap v713: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:06.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:06 smithi071 ceph-mon[144544]: pgmap v713: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:07.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:07 smithi192 ceph-mon[108617]: pgmap v714: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:07.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:07 smithi071 ceph-mon[144544]: pgmap v714: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:10.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:10 smithi192 ceph-mon[108617]: pgmap v715: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:10.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:10 smithi071 ceph-mon[144544]: pgmap v715: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:12.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:12 smithi192 ceph-mon[108617]: pgmap v716: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:12.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:12 smithi071 ceph-mon[144544]: pgmap v716: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:14.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:14 smithi192 ceph-mon[108617]: pgmap v717: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:14.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:14 smithi071 ceph-mon[144544]: pgmap v717: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:16.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:16 smithi192 ceph-mon[108617]: pgmap v718: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:16.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:16 smithi071 ceph-mon[144544]: pgmap v718: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:17.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.761+0000 7fdf981be700 1 -- 172.21.15.71:0/3305098743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90106cd0 msgr2=0x7fdf901070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:17.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.761+0000 7fdf981be700 1 --2- 172.21.15.71:0/3305098743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90106cd0 0x7fdf901070b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009910 tx=0x7fdf80009c20 comp rx=0 tx=0).stop 2024-04-23T02:37:17.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.761+0000 7fdf981be700 1 -- 172.21.15.71:0/3305098743 shutdown_connections 2024-04-23T02:37:17.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.761+0000 7fdf981be700 1 --2- 172.21.15.71:0/3305098743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf90068580 0x7fdf900689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:17.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.761+0000 7fdf981be700 1 --2- 172.21.15.71:0/3305098743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90106cd0 0x7fdf901070b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:17.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.761+0000 7fdf981be700 1 -- 172.21.15.71:0/3305098743 >> 172.21.15.71:0/3305098743 conn(0x7fdf90075670 msgr2=0x7fdf90075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:17.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.761+0000 7fdf981be700 1 -- 172.21.15.71:0/3305098743 shutdown_connections 2024-04-23T02:37:17.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.761+0000 7fdf981be700 1 -- 172.21.15.71:0/3305098743 wait complete. 2024-04-23T02:37:17.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.762+0000 7fdf981be700 1 Processor -- start 2024-04-23T02:37:17.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.762+0000 7fdf981be700 1 -- start start 2024-04-23T02:37:17.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.762+0000 7fdf981be700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90068580 0x7fdf9019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:17.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.762+0000 7fdf981be700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf90106cd0 0x7fdf9019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:17.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.762+0000 7fdf981be700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9019d130 con 0x7fdf90068580 2024-04-23T02:37:17.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.762+0000 7fdf981be700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf90196630 con 0x7fdf90106cd0 2024-04-23T02:37:17.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.762+0000 7fdf95f5a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90068580 0x7fdf9019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:17.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.762+0000 7fdf95f5a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90068580 0x7fdf9019c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52670/0 (socket says 172.21.15.71:52670) 2024-04-23T02:37:17.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.762+0000 7fdf95f5a700 1 -- 172.21.15.71:0/695995927 learned_addr learned my addr 172.21.15.71:0/695995927 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:37:17.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.762+0000 7fdf95759700 1 --2- 172.21.15.71:0/695995927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf90106cd0 0x7fdf9019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:17.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.762+0000 7fdf95f5a700 1 -- 172.21.15.71:0/695995927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf90106cd0 msgr2=0x7fdf9019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:17.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.762+0000 7fdf95f5a700 1 --2- 172.21.15.71:0/695995927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf90106cd0 0x7fdf9019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:17.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.762+0000 7fdf95f5a700 1 -- 172.21.15.71:0/695995927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf800095f0 con 0x7fdf90068580 2024-04-23T02:37:17.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.763+0000 7fdf95f5a700 1 --2- 172.21.15.71:0/695995927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90068580 0x7fdf9019c560 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdf80006010 tx=0x7fdf8000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:17.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.763+0000 7fdf86ffd700 1 -- 172.21.15.71:0/695995927 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf8001d070 con 0x7fdf90068580 2024-04-23T02:37:17.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.763+0000 7fdf981be700 1 -- 172.21.15.71:0/695995927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf901968b0 con 0x7fdf90068580 2024-04-23T02:37:17.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.763+0000 7fdf86ffd700 1 -- 172.21.15.71:0/695995927 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdf8000f460 con 0x7fdf90068580 2024-04-23T02:37:17.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.763+0000 7fdf86ffd700 1 -- 172.21.15.71:0/695995927 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf80021620 con 0x7fdf90068580 2024-04-23T02:37:17.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.763+0000 7fdf981be700 1 -- 172.21.15.71:0/695995927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf90196da0 con 0x7fdf90068580 2024-04-23T02:37:17.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.764+0000 7fdf981be700 1 -- 172.21.15.71:0/695995927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf901091f0 con 0x7fdf90068580 2024-04-23T02:37:17.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.764+0000 7fdf86ffd700 1 -- 172.21.15.71:0/695995927 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fdf8000fa40 con 0x7fdf90068580 2024-04-23T02:37:17.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.765+0000 7fdf86ffd700 1 --2- 172.21.15.71:0/695995927 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdf7c06cce0 0x7fdf7c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:17.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.765+0000 7fdf86ffd700 1 -- 172.21.15.71:0/695995927 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdf8008d830 con 0x7fdf90068580 2024-04-23T02:37:17.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.765+0000 7fdf95759700 1 --2- 172.21.15.71:0/695995927 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdf7c06cce0 0x7fdf7c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:17.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.765+0000 7fdf95759700 1 --2- 172.21.15.71:0/695995927 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdf7c06cce0 0x7fdf7c06f1a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c005950 tx=0x7fdf8c0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:17.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.769+0000 7fdf86ffd700 1 -- 172.21.15.71:0/695995927 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fdf8005b290 con 0x7fdf90068580 2024-04-23T02:37:17.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.981+0000 7fdf981be700 1 -- 172.21.15.71:0/695995927 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf90196fc0 con 0x7fdf7c06cce0 2024-04-23T02:37:17.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.982+0000 7fdf86ffd700 1 -- 172.21.15.71:0/695995927 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdf90196fc0 con 0x7fdf7c06cce0 2024-04-23T02:37:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.984+0000 7fdf981be700 1 -- 172.21.15.71:0/695995927 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdf7c06cce0 msgr2=0x7fdf7c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.984+0000 7fdf981be700 1 --2- 172.21.15.71:0/695995927 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdf7c06cce0 0x7fdf7c06f1a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c005950 tx=0x7fdf8c0058e0 comp rx=0 tx=0).stop 2024-04-23T02:37:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.984+0000 7fdf981be700 1 -- 172.21.15.71:0/695995927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90068580 msgr2=0x7fdf9019c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.984+0000 7fdf981be700 1 --2- 172.21.15.71:0/695995927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90068580 0x7fdf9019c560 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdf80006010 tx=0x7fdf8000be20 comp rx=0 tx=0).stop 2024-04-23T02:37:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.984+0000 7fdf981be700 1 -- 172.21.15.71:0/695995927 shutdown_connections 2024-04-23T02:37:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.984+0000 7fdf981be700 1 --2- 172.21.15.71:0/695995927 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdf7c06cce0 0x7fdf7c06f1a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.984+0000 7fdf981be700 1 --2- 172.21.15.71:0/695995927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90068580 0x7fdf9019c560 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.984+0000 7fdf981be700 1 --2- 172.21.15.71:0/695995927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf90106cd0 0x7fdf9019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.984+0000 7fdf981be700 1 -- 172.21.15.71:0/695995927 >> 172.21.15.71:0/695995927 conn(0x7fdf90075670 msgr2=0x7fdf900fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.984+0000 7fdf981be700 1 -- 172.21.15.71:0/695995927 shutdown_connections 2024-04-23T02:37:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:17.984+0000 7fdf981be700 1 -- 172.21.15.71:0/695995927 wait complete. 2024-04-23T02:37:17.997 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:37:18.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.088+0000 7f85e2777700 1 -- 172.21.15.71:0/2666624907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85dc0fff50 msgr2=0x7f85dc105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:18.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.088+0000 7f85e2777700 1 --2- 172.21.15.71:0/2666624907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85dc0fff50 0x7f85dc105840 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009910 tx=0x7f85d0009c20 comp rx=0 tx=0).stop 2024-04-23T02:37:18.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.088+0000 7f85e2777700 1 -- 172.21.15.71:0/2666624907 shutdown_connections 2024-04-23T02:37:18.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.088+0000 7f85e2777700 1 --2- 172.21.15.71:0/2666624907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85dc0fff50 0x7f85dc105840 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:18.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.088+0000 7f85e2777700 1 --2- 172.21.15.71:0/2666624907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85dc0ff5a0 0x7f85dc0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:18.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.088+0000 7f85e2777700 1 -- 172.21.15.71:0/2666624907 >> 172.21.15.71:0/2666624907 conn(0x7f85dc074e30 msgr2=0x7f85dc075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:18.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.089+0000 7f85e2777700 1 -- 172.21.15.71:0/2666624907 shutdown_connections 2024-04-23T02:37:18.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.089+0000 7f85e2777700 1 -- 172.21.15.71:0/2666624907 wait complete. 2024-04-23T02:37:18.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.089+0000 7f85e2777700 1 Processor -- start 2024-04-23T02:37:18.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.089+0000 7f85e2777700 1 -- start start 2024-04-23T02:37:18.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.089+0000 7f85e2777700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85dc0ff5a0 0x7f85dc100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:18.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.089+0000 7f85e2777700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85dc0fff50 0x7f85dc100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:18.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.089+0000 7f85e2777700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc100fa0 con 0x7f85dc0ff5a0 2024-04-23T02:37:18.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.089+0000 7f85e2777700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc1010e0 con 0x7f85dc0fff50 2024-04-23T02:37:18.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.090+0000 7f85dbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85dc0ff5a0 0x7f85dc100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:18.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.090+0000 7f85dbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85dc0ff5a0 0x7f85dc100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52676/0 (socket says 172.21.15.71:52676) 2024-04-23T02:37:18.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.090+0000 7f85dbfff700 1 -- 172.21.15.71:0/9908978 learned_addr learned my addr 172.21.15.71:0/9908978 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:37:18.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.090+0000 7f85db7fe700 1 --2- 172.21.15.71:0/9908978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85dc0fff50 0x7f85dc100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:18.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.090+0000 7f85dbfff700 1 -- 172.21.15.71:0/9908978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85dc0fff50 msgr2=0x7f85dc100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:18.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.090+0000 7f85dbfff700 1 --2- 172.21.15.71:0/9908978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85dc0fff50 0x7f85dc100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:18.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.090+0000 7f85dbfff700 1 -- 172.21.15.71:0/9908978 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d00095f0 con 0x7f85dc0ff5a0 2024-04-23T02:37:18.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.090+0000 7f85dbfff700 1 --2- 172.21.15.71:0/9908978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85dc0ff5a0 0x7f85dc100520 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00d8d0 tx=0x7f85cc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:18.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.090+0000 7f85d97fa700 1 -- 172.21.15.71:0/9908978 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85cc0098a0 con 0x7f85dc0ff5a0 2024-04-23T02:37:18.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.090+0000 7f85d97fa700 1 -- 172.21.15.71:0/9908978 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f85cc010460 con 0x7f85dc0ff5a0 2024-04-23T02:37:18.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.090+0000 7f85e2777700 1 -- 172.21.15.71:0/9908978 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85dc1a2a90 con 0x7f85dc0ff5a0 2024-04-23T02:37:18.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.090+0000 7f85d97fa700 1 -- 172.21.15.71:0/9908978 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85cc009ea0 con 0x7f85dc0ff5a0 2024-04-23T02:37:18.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.090+0000 7f85e2777700 1 -- 172.21.15.71:0/9908978 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85dc1a2ea0 con 0x7f85dc0ff5a0 2024-04-23T02:37:18.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.091+0000 7f85e2777700 1 -- 172.21.15.71:0/9908978 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85dc109200 con 0x7f85dc0ff5a0 2024-04-23T02:37:18.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.092+0000 7f85d97fa700 1 -- 172.21.15.71:0/9908978 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f85cc0105d0 con 0x7f85dc0ff5a0 2024-04-23T02:37:18.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.092+0000 7f85d97fa700 1 --2- 172.21.15.71:0/9908978 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f85c406cce0 0x7f85c406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:18.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.092+0000 7f85d97fa700 1 -- 172.21.15.71:0/9908978 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f85cc08c480 con 0x7f85dc0ff5a0 2024-04-23T02:37:18.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.092+0000 7f85db7fe700 1 --2- 172.21.15.71:0/9908978 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f85c406cce0 0x7f85c406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:18.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.093+0000 7f85db7fe700 1 --2- 172.21.15.71:0/9908978 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f85c406cce0 0x7f85c406f1a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f85dc101d10 tx=0x7f85d001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:18.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.097+0000 7f85d97fa700 1 -- 172.21.15.71:0/9908978 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f85cc0564f0 con 0x7f85dc0ff5a0 2024-04-23T02:37:18.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.304+0000 7f85e2777700 1 -- 172.21.15.71:0/9908978 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f85dc0611d0 con 0x7f85c406cce0 2024-04-23T02:37:18.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.305+0000 7f85d97fa700 1 -- 172.21.15.71:0/9908978 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f85dc0611d0 con 0x7f85c406cce0 2024-04-23T02:37:18.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.307+0000 7f85e2777700 1 -- 172.21.15.71:0/9908978 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f85c406cce0 msgr2=0x7f85c406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:18.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.307+0000 7f85e2777700 1 --2- 172.21.15.71:0/9908978 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f85c406cce0 0x7f85c406f1a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f85dc101d10 tx=0x7f85d001a040 comp rx=0 tx=0).stop 2024-04-23T02:37:18.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.307+0000 7f85e2777700 1 -- 172.21.15.71:0/9908978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85dc0ff5a0 msgr2=0x7f85dc100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:18.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.307+0000 7f85e2777700 1 --2- 172.21.15.71:0/9908978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85dc0ff5a0 0x7f85dc100520 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00d8d0 tx=0x7f85cc00dbe0 comp rx=0 tx=0).stop 2024-04-23T02:37:18.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.307+0000 7f85e2777700 1 -- 172.21.15.71:0/9908978 shutdown_connections 2024-04-23T02:37:18.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.307+0000 7f85e2777700 1 --2- 172.21.15.71:0/9908978 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f85c406cce0 0x7f85c406f1a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:18.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.307+0000 7f85e2777700 1 --2- 172.21.15.71:0/9908978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85dc0ff5a0 0x7f85dc100520 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:18.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.307+0000 7f85e2777700 1 --2- 172.21.15.71:0/9908978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85dc0fff50 0x7f85dc100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:18.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.307+0000 7f85e2777700 1 -- 172.21.15.71:0/9908978 >> 172.21.15.71:0/9908978 conn(0x7f85dc074e30 msgr2=0x7f85dc0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:18.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.307+0000 7f85e2777700 1 -- 172.21.15.71:0/9908978 shutdown_connections 2024-04-23T02:37:18.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.307+0000 7f85e2777700 1 -- 172.21.15.71:0/9908978 wait complete. 2024-04-23T02:37:18.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:18 smithi192 ceph-mon[108617]: pgmap v719: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:18.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.411+0000 7fc047529700 1 -- 172.21.15.71:0/610259961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc040101840 msgr2=0x7fc040105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:18.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.411+0000 7fc047529700 1 --2- 172.21.15.71:0/610259961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc040101840 0x7fc040105890 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009910 tx=0x7fc03c009c20 comp rx=0 tx=0).stop 2024-04-23T02:37:18.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.411+0000 7fc047529700 1 -- 172.21.15.71:0/610259961 shutdown_connections 2024-04-23T02:37:18.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.411+0000 7fc047529700 1 --2- 172.21.15.71:0/610259961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc040101840 0x7fc040105890 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:18.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.411+0000 7fc047529700 1 --2- 172.21.15.71:0/610259961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc040100e90 0x7fc040101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:18.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.411+0000 7fc047529700 1 -- 172.21.15.71:0/610259961 >> 172.21.15.71:0/610259961 conn(0x7fc0400fc740 msgr2=0x7fc0400feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:18.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.411+0000 7fc047529700 1 -- 172.21.15.71:0/610259961 shutdown_connections 2024-04-23T02:37:18.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.411+0000 7fc047529700 1 -- 172.21.15.71:0/610259961 wait complete. 2024-04-23T02:37:18.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.411+0000 7fc047529700 1 Processor -- start 2024-04-23T02:37:18.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.411+0000 7fc047529700 1 -- start start 2024-04-23T02:37:18.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.412+0000 7fc047529700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc040100e90 0x7fc04019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:18.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.412+0000 7fc047529700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc040101840 0x7fc04019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:18.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.412+0000 7fc047529700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04019d200 con 0x7fc040101840 2024-04-23T02:37:18.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.412+0000 7fc047529700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0401966b0 con 0x7fc040100e90 2024-04-23T02:37:18.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.412+0000 7fc0452c5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc040100e90 0x7fc04019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:18.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.412+0000 7fc044ac4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc040101840 0x7fc04019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:18.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.412+0000 7fc0452c5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc040100e90 0x7fc04019c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.71:60258/0 (socket says 172.21.15.71:60258) 2024-04-23T02:37:18.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.412+0000 7fc0452c5700 1 -- 172.21.15.71:0/2453446097 learned_addr learned my addr 172.21.15.71:0/2453446097 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:37:18.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.412+0000 7fc044ac4700 1 -- 172.21.15.71:0/2453446097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc040100e90 msgr2=0x7fc04019c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:18.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.412+0000 7fc044ac4700 1 --2- 172.21.15.71:0/2453446097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc040100e90 0x7fc04019c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:18.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.412+0000 7fc044ac4700 1 -- 172.21.15.71:0/2453446097 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c0095f0 con 0x7fc040101840 2024-04-23T02:37:18.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.412+0000 7fc044ac4700 1 --2- 172.21.15.71:0/2453446097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc040101840 0x7fc04019cb70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc040067ee0 tx=0x7fc03c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:18.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.413+0000 7fc0367fc700 1 -- 172.21.15.71:0/2453446097 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc03c01d070 con 0x7fc040101840 2024-04-23T02:37:18.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.413+0000 7fc0367fc700 1 -- 172.21.15.71:0/2453446097 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc03c005dc0 con 0x7fc040101840 2024-04-23T02:37:18.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.413+0000 7fc047529700 1 -- 172.21.15.71:0/2453446097 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc040196930 con 0x7fc040101840 2024-04-23T02:37:18.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.413+0000 7fc0367fc700 1 -- 172.21.15.71:0/2453446097 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc03c00fe60 con 0x7fc040101840 2024-04-23T02:37:18.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.413+0000 7fc047529700 1 -- 172.21.15.71:0/2453446097 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc040196e20 con 0x7fc040101840 2024-04-23T02:37:18.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.414+0000 7fc047529700 1 -- 172.21.15.71:0/2453446097 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc040109290 con 0x7fc040101840 2024-04-23T02:37:18.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.414+0000 7fc0367fc700 1 -- 172.21.15.71:0/2453446097 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc03c00f6d0 con 0x7fc040101840 2024-04-23T02:37:18.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.415+0000 7fc0367fc700 1 --2- 172.21.15.71:0/2453446097 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc02c06cce0 0x7fc02c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:18.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.415+0000 7fc0367fc700 1 -- 172.21.15.71:0/2453446097 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc03c08d5d0 con 0x7fc040101840 2024-04-23T02:37:18.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.415+0000 7fc0452c5700 1 --2- 172.21.15.71:0/2453446097 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc02c06cce0 0x7fc02c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:18.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.415+0000 7fc0452c5700 1 --2- 172.21.15.71:0/2453446097 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc02c06cce0 0x7fc02c06f1a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc030005950 tx=0x7fc03000b500 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:18.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.419+0000 7fc0367fc700 1 -- 172.21.15.71:0/2453446097 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc03c05b150 con 0x7fc040101840 2024-04-23T02:37:18.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:18 smithi071 ceph-mon[144544]: pgmap v719: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:18.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.621+0000 7fc047529700 1 -- 172.21.15.71:0/2453446097 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc040197040 con 0x7fc02c06cce0 2024-04-23T02:37:18.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.632+0000 7fc0367fc700 1 -- 172.21.15.71:0/2453446097 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc040197040 con 0x7fc02c06cce0 2024-04-23T02:37:18.635 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:37:18.635 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (21m) 14m ago 32m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:37:18.635 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (32m) 14m ago 32m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:37:18.635 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (31m) 14m ago 31m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:37:18.635 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (18m) 14m ago 32m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:37:18.636 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (18m) 14m ago 31m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:37:18.636 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (20m) 14m ago 32m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:37:18.636 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (15m) 14m ago 26m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:37:18.636 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (14m) 14m ago 26m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:37:18.636 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (26m) 14m ago 26m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:37:18.636 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (14m) 14m ago 26m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:37:18.637 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (22m) 14m ago 33m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:37:18.637 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (22m) 14m ago 31m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:37:18.637 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (19m) 14m ago 34m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:37:18.637 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (19m) 14m ago 31m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:37:18.637 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (21m) 14m ago 32m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:37:18.637 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (21m) 14m ago 31m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:37:18.637 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (18m) 14m ago 30m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:37:18.637 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (17m) 14m ago 29m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:37:18.637 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (17m) 14m ago 29m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:37:18.637 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (16m) 14m ago 28m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:37:18.637 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (16m) 14m ago 28m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:37:18.637 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (15m) 14m ago 27m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:37:18.637 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (21m) 14m ago 31m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:37:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.634+0000 7fc047529700 1 -- 172.21.15.71:0/2453446097 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc02c06cce0 msgr2=0x7fc02c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.634+0000 7fc047529700 1 --2- 172.21.15.71:0/2453446097 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc02c06cce0 0x7fc02c06f1a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc030005950 tx=0x7fc03000b500 comp rx=0 tx=0).stop 2024-04-23T02:37:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.634+0000 7fc047529700 1 -- 172.21.15.71:0/2453446097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc040101840 msgr2=0x7fc04019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.634+0000 7fc047529700 1 --2- 172.21.15.71:0/2453446097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc040101840 0x7fc04019cb70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc040067ee0 tx=0x7fc03c00bd70 comp rx=0 tx=0).stop 2024-04-23T02:37:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.634+0000 7fc047529700 1 -- 172.21.15.71:0/2453446097 shutdown_connections 2024-04-23T02:37:18.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.634+0000 7fc047529700 1 --2- 172.21.15.71:0/2453446097 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc02c06cce0 0x7fc02c06f1a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:18.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.634+0000 7fc047529700 1 --2- 172.21.15.71:0/2453446097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc040100e90 0x7fc04019c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:18.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.634+0000 7fc047529700 1 --2- 172.21.15.71:0/2453446097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc040101840 0x7fc04019cb70 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:18.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.634+0000 7fc047529700 1 -- 172.21.15.71:0/2453446097 >> 172.21.15.71:0/2453446097 conn(0x7fc0400fc740 msgr2=0x7fc0400fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:18.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.634+0000 7fc047529700 1 -- 172.21.15.71:0/2453446097 shutdown_connections 2024-04-23T02:37:18.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.634+0000 7fc047529700 1 -- 172.21.15.71:0/2453446097 wait complete. 2024-04-23T02:37:18.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.736+0000 7f809f708700 1 -- 172.21.15.71:0/3651350885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8098101060 msgr2=0x7f80981047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:18.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.736+0000 7f809f708700 1 --2- 172.21.15.71:0/3651350885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8098101060 0x7f80981047b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8094009910 tx=0x7f8094009c20 comp rx=0 tx=0).stop 2024-04-23T02:37:18.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.736+0000 7f809f708700 1 -- 172.21.15.71:0/3651350885 shutdown_connections 2024-04-23T02:37:18.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.736+0000 7f809f708700 1 --2- 172.21.15.71:0/3651350885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8098101060 0x7f80981047b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:18.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.736+0000 7f809f708700 1 --2- 172.21.15.71:0/3651350885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80981006b0 0x7f8098100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:18.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.736+0000 7f809f708700 1 -- 172.21.15.71:0/3651350885 >> 172.21.15.71:0/3651350885 conn(0x7f8098074e30 msgr2=0x7f8098075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:18.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.736+0000 7f809f708700 1 -- 172.21.15.71:0/3651350885 shutdown_connections 2024-04-23T02:37:18.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.736+0000 7f809f708700 1 -- 172.21.15.71:0/3651350885 wait complete. 2024-04-23T02:37:18.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.737+0000 7f809f708700 1 Processor -- start 2024-04-23T02:37:18.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.737+0000 7f809f708700 1 -- start start 2024-04-23T02:37:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.737+0000 7f809f708700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80981006b0 0x7f809819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.737+0000 7f809f708700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8098101060 0x7f809819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.737+0000 7f809f708700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809819c0c0 con 0x7f80981006b0 2024-04-23T02:37:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.737+0000 7f809f708700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8098195570 con 0x7f8098101060 2024-04-23T02:37:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.737+0000 7f809d4a4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80981006b0 0x7f809819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.737+0000 7f809d4a4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80981006b0 0x7f809819b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52732/0 (socket says 172.21.15.71:52732) 2024-04-23T02:37:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.737+0000 7f809d4a4700 1 -- 172.21.15.71:0/2950455251 learned_addr learned my addr 172.21.15.71:0/2950455251 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:37:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.737+0000 7f809cca3700 1 --2- 172.21.15.71:0/2950455251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8098101060 0x7f809819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.738+0000 7f809d4a4700 1 -- 172.21.15.71:0/2950455251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8098101060 msgr2=0x7f809819ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.738+0000 7f809d4a4700 1 --2- 172.21.15.71:0/2950455251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8098101060 0x7f809819ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.738+0000 7f809d4a4700 1 -- 172.21.15.71:0/2950455251 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80940095f0 con 0x7f80981006b0 2024-04-23T02:37:18.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.738+0000 7f809d4a4700 1 --2- 172.21.15.71:0/2950455251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80981006b0 0x7f809819b4f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f808800d8d0 tx=0x7f808800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:18.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.738+0000 7f808e7fc700 1 -- 172.21.15.71:0/2950455251 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80880098a0 con 0x7f80981006b0 2024-04-23T02:37:18.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.738+0000 7f808e7fc700 1 -- 172.21.15.71:0/2950455251 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8088010460 con 0x7f80981006b0 2024-04-23T02:37:18.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.738+0000 7f809f708700 1 -- 172.21.15.71:0/2950455251 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8098195850 con 0x7f80981006b0 2024-04-23T02:37:18.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.738+0000 7f808e7fc700 1 -- 172.21.15.71:0/2950455251 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8088009ea0 con 0x7f80981006b0 2024-04-23T02:37:18.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.738+0000 7f809f708700 1 -- 172.21.15.71:0/2950455251 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8098195d70 con 0x7f80981006b0 2024-04-23T02:37:18.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.739+0000 7f809f708700 1 -- 172.21.15.71:0/2950455251 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8098108090 con 0x7f80981006b0 2024-04-23T02:37:18.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.739+0000 7f808e7fc700 1 -- 172.21.15.71:0/2950455251 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f80880105d0 con 0x7f80981006b0 2024-04-23T02:37:18.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.740+0000 7f808e7fc700 1 --2- 172.21.15.71:0/2950455251 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f808406cce0 0x7f808406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:18.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.740+0000 7f808e7fc700 1 -- 172.21.15.71:0/2950455251 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f808808c480 con 0x7f80981006b0 2024-04-23T02:37:18.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.740+0000 7f809cca3700 1 --2- 172.21.15.71:0/2950455251 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f808406cce0 0x7f808406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:18.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.740+0000 7f809cca3700 1 --2- 172.21.15.71:0/2950455251 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f808406cce0 0x7f808406f1a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8094006010 tx=0x7f809401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:18.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:18.744+0000 7f808e7fc700 1 -- 172.21.15.71:0/2950455251 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8088059f80 con 0x7f80981006b0 2024-04-23T02:37:19.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.023+0000 7f809f708700 1 -- 172.21.15.71:0/2950455251 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f809804f2e0 con 0x7f80981006b0 2024-04-23T02:37:19.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.024+0000 7f808e7fc700 1 -- 172.21.15.71:0/2950455251 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f8088010880 con 0x7f80981006b0 2024-04-23T02:37:19.026 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:37:19.026 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:37:19.027 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:37:19.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:37:19.027 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:37:19.027 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:37:19.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:37:19.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:37:19.027 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:37:19.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:37:19.027 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:37:19.027 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:37:19.027 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:37:19.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:37:19.027 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:37:19.027 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:37:19.028 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:37:19.028 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:37:19.028 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:37:19.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.026+0000 7f809f708700 1 -- 172.21.15.71:0/2950455251 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f808406cce0 msgr2=0x7f808406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:19.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.026+0000 7f809f708700 1 --2- 172.21.15.71:0/2950455251 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f808406cce0 0x7f808406f1a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8094006010 tx=0x7f809401a040 comp rx=0 tx=0).stop 2024-04-23T02:37:19.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.026+0000 7f809f708700 1 -- 172.21.15.71:0/2950455251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80981006b0 msgr2=0x7f809819b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:19.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.026+0000 7f809f708700 1 --2- 172.21.15.71:0/2950455251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80981006b0 0x7f809819b4f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f808800d8d0 tx=0x7f808800dbe0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.026+0000 7f809f708700 1 -- 172.21.15.71:0/2950455251 shutdown_connections 2024-04-23T02:37:19.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.026+0000 7f809f708700 1 --2- 172.21.15.71:0/2950455251 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f808406cce0 0x7f808406f1a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.026+0000 7f809f708700 1 --2- 172.21.15.71:0/2950455251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80981006b0 0x7f809819b4f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.026+0000 7f809f708700 1 --2- 172.21.15.71:0/2950455251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8098101060 0x7f809819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.026+0000 7f809f708700 1 -- 172.21.15.71:0/2950455251 >> 172.21.15.71:0/2950455251 conn(0x7f8098074e30 msgr2=0x7f80980fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:19.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.026+0000 7f809f708700 1 -- 172.21.15.71:0/2950455251 shutdown_connections 2024-04-23T02:37:19.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.026+0000 7f809f708700 1 -- 172.21.15.71:0/2950455251 wait complete. 2024-04-23T02:37:19.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.129+0000 7f44ffcd7700 1 -- 172.21.15.71:0/3915830057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f80ffda0 msgr2=0x7f44f8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:19.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.129+0000 7f44ffcd7700 1 --2- 172.21.15.71:0/3915830057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f80ffda0 0x7f44f8100180 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009910 tx=0x7f44ec009c20 comp rx=0 tx=0).stop 2024-04-23T02:37:19.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.129+0000 7f44ffcd7700 1 -- 172.21.15.71:0/3915830057 shutdown_connections 2024-04-23T02:37:19.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.129+0000 7f44ffcd7700 1 --2- 172.21.15.71:0/3915830057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f8100750 0x7f44f81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.129+0000 7f44ffcd7700 1 --2- 172.21.15.71:0/3915830057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f80ffda0 0x7f44f8100180 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.129+0000 7f44ffcd7700 1 -- 172.21.15.71:0/3915830057 >> 172.21.15.71:0/3915830057 conn(0x7f44f80fb650 msgr2=0x7f44f80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:19.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.129+0000 7f44ffcd7700 1 -- 172.21.15.71:0/3915830057 shutdown_connections 2024-04-23T02:37:19.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.129+0000 7f44ffcd7700 1 -- 172.21.15.71:0/3915830057 wait complete. 2024-04-23T02:37:19.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.130+0000 7f44ffcd7700 1 Processor -- start 2024-04-23T02:37:19.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.130+0000 7f44ffcd7700 1 -- start start 2024-04-23T02:37:19.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.130+0000 7f44ffcd7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f80ffda0 0x7f44f8195430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:19.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.130+0000 7f44ffcd7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f8100750 0x7f44f8195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:19.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.130+0000 7f44ffcd7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f8195eb0 con 0x7f44f80ffda0 2024-04-23T02:37:19.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.130+0000 7f44ffcd7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f8196020 con 0x7f44f8100750 2024-04-23T02:37:19.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.130+0000 7f44fda73700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f80ffda0 0x7f44f8195430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:19.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.130+0000 7f44fda73700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f80ffda0 0x7f44f8195430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52754/0 (socket says 172.21.15.71:52754) 2024-04-23T02:37:19.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.130+0000 7f44fda73700 1 -- 172.21.15.71:0/1746369053 learned_addr learned my addr 172.21.15.71:0/1746369053 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:37:19.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.130+0000 7f44fd272700 1 --2- 172.21.15.71:0/1746369053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f8100750 0x7f44f8195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:19.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.130+0000 7f44fda73700 1 -- 172.21.15.71:0/1746369053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f8100750 msgr2=0x7f44f8195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:19.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.130+0000 7f44fda73700 1 --2- 172.21.15.71:0/1746369053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f8100750 0x7f44f8195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.130+0000 7f44fda73700 1 -- 172.21.15.71:0/1746369053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec0095f0 con 0x7f44f80ffda0 2024-04-23T02:37:19.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.130+0000 7f44fda73700 1 --2- 172.21.15.71:0/1746369053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f80ffda0 0x7f44f8195430 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f44ec006010 tx=0x7f44ec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:19.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.131+0000 7f44eaffd700 1 -- 172.21.15.71:0/1746369053 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44ec01d070 con 0x7f44f80ffda0 2024-04-23T02:37:19.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.131+0000 7f44eaffd700 1 -- 172.21.15.71:0/1746369053 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f44ec00f460 con 0x7f44f80ffda0 2024-04-23T02:37:19.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.131+0000 7f44ffcd7700 1 -- 172.21.15.71:0/1746369053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f81962a0 con 0x7f44f80ffda0 2024-04-23T02:37:19.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.131+0000 7f44eaffd700 1 -- 172.21.15.71:0/1746369053 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44ec021620 con 0x7f44f80ffda0 2024-04-23T02:37:19.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.131+0000 7f44ffcd7700 1 -- 172.21.15.71:0/1746369053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f81a2d20 con 0x7f44f80ffda0 2024-04-23T02:37:19.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.132+0000 7f44ffcd7700 1 -- 172.21.15.71:0/1746369053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f8108140 con 0x7f44f80ffda0 2024-04-23T02:37:19.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.132+0000 7f44eaffd700 1 -- 172.21.15.71:0/1746369053 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f44ec00fa40 con 0x7f44f80ffda0 2024-04-23T02:37:19.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.133+0000 7f44eaffd700 1 --2- 172.21.15.71:0/1746369053 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f44e406cce0 0x7f44e406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:19.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.133+0000 7f44fd272700 1 --2- 172.21.15.71:0/1746369053 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f44e406cce0 0x7f44e406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:19.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.133+0000 7f44eaffd700 1 -- 172.21.15.71:0/1746369053 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f44ec08d8c0 con 0x7f44f80ffda0 2024-04-23T02:37:19.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.133+0000 7f44fd272700 1 --2- 172.21.15.71:0/1746369053 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f44e406cce0 0x7f44e406f1a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f44f4005950 tx=0x7f44f40058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:19.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.137+0000 7f44eaffd700 1 -- 172.21.15.71:0/1746369053 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f44ec05b4c0 con 0x7f44f80ffda0 2024-04-23T02:37:19.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.370+0000 7f44ffcd7700 1 -- 172.21.15.71:0/1746369053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f44f81962a0 con 0x7f44f80ffda0 2024-04-23T02:37:19.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.370+0000 7f44eaffd700 1 -- 172.21.15.71:0/1746369053 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f44f81962a0 con 0x7f44f80ffda0 2024-04-23T02:37:19.373 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:37:19.373 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:37:19.373 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:37:19.373 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:37:19.373 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:37:19.373 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:37:19.373 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:37:19.373 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:37:19.373 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:37:19.373 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:37:19.373 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:37:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:37:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:37:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:37:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:37:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:37:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:37:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:37:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:37:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:37:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:37:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:37:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:37:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:37:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:37:19.374 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:37:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:37:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:37:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:37:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:37:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:37:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:37:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:37:19.375 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:37:19.375 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:37:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:37:19.375 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:37:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:37:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:37:19.375 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:37:19.376 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:37:19.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:19 smithi192 ceph-mon[108617]: from='client.35126 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:37:19.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:19 smithi192 ceph-mon[108617]: from='client.35130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:37:19.377 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:19 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2950455251' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:37:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.372+0000 7f44ffcd7700 1 -- 172.21.15.71:0/1746369053 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f44e406cce0 msgr2=0x7f44e406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.372+0000 7f44ffcd7700 1 --2- 172.21.15.71:0/1746369053 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f44e406cce0 0x7f44e406f1a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f44f4005950 tx=0x7f44f40058e0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.372+0000 7f44ffcd7700 1 -- 172.21.15.71:0/1746369053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f80ffda0 msgr2=0x7f44f8195430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.372+0000 7f44ffcd7700 1 --2- 172.21.15.71:0/1746369053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f80ffda0 0x7f44f8195430 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f44ec006010 tx=0x7f44ec00be20 comp rx=0 tx=0).stop 2024-04-23T02:37:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.372+0000 7f44ffcd7700 1 -- 172.21.15.71:0/1746369053 shutdown_connections 2024-04-23T02:37:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.372+0000 7f44ffcd7700 1 --2- 172.21.15.71:0/1746369053 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f44e406cce0 0x7f44e406f1a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.372+0000 7f44ffcd7700 1 --2- 172.21.15.71:0/1746369053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f80ffda0 0x7f44f8195430 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.372+0000 7f44ffcd7700 1 --2- 172.21.15.71:0/1746369053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f8100750 0x7f44f8195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.372+0000 7f44ffcd7700 1 -- 172.21.15.71:0/1746369053 >> 172.21.15.71:0/1746369053 conn(0x7f44f80fb650 msgr2=0x7f44f80fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:19.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.372+0000 7f44ffcd7700 1 -- 172.21.15.71:0/1746369053 shutdown_connections 2024-04-23T02:37:19.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.372+0000 7f44ffcd7700 1 -- 172.21.15.71:0/1746369053 wait complete. 2024-04-23T02:37:19.378 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:37:19.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.475+0000 7ff72f8ff700 1 -- 172.21.15.71:0/3627817207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7280ffc20 msgr2=0x7ff728100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:19.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.475+0000 7ff72f8ff700 1 --2- 172.21.15.71:0/3627817207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7280ffc20 0x7ff728100080 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff718009910 tx=0x7ff718009c20 comp rx=0 tx=0).stop 2024-04-23T02:37:19.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.476+0000 7ff72f8ff700 1 -- 172.21.15.71:0/3627817207 shutdown_connections 2024-04-23T02:37:19.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.476+0000 7ff72f8ff700 1 --2- 172.21.15.71:0/3627817207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7280ffc20 0x7ff728100080 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.476+0000 7ff72f8ff700 1 --2- 172.21.15.71:0/3627817207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff728105bd0 0x7ff728105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.476+0000 7ff72f8ff700 1 -- 172.21.15.71:0/3627817207 >> 172.21.15.71:0/3627817207 conn(0x7ff728075670 msgr2=0x7ff728075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:19.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.476+0000 7ff72f8ff700 1 -- 172.21.15.71:0/3627817207 shutdown_connections 2024-04-23T02:37:19.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.476+0000 7ff72f8ff700 1 -- 172.21.15.71:0/3627817207 wait complete. 2024-04-23T02:37:19.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.476+0000 7ff72f8ff700 1 Processor -- start 2024-04-23T02:37:19.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.476+0000 7ff72f8ff700 1 -- start start 2024-04-23T02:37:19.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.476+0000 7ff72f8ff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7280ffc20 0x7ff7281976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:19.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.476+0000 7ff72f8ff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff728105bd0 0x7ff728197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:19.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.476+0000 7ff72f8ff700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff728198220 con 0x7ff7280ffc20 2024-04-23T02:37:19.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.476+0000 7ff72f8ff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff728198360 con 0x7ff728105bd0 2024-04-23T02:37:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.477+0000 7ff72d69b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7280ffc20 0x7ff7281976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.477+0000 7ff72d69b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7280ffc20 0x7ff7281976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52766/0 (socket says 172.21.15.71:52766) 2024-04-23T02:37:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.477+0000 7ff72d69b700 1 -- 172.21.15.71:0/1031545898 learned_addr learned my addr 172.21.15.71:0/1031545898 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:37:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.477+0000 7ff72ce9a700 1 --2- 172.21.15.71:0/1031545898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff728105bd0 0x7ff728197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.477+0000 7ff72d69b700 1 -- 172.21.15.71:0/1031545898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff728105bd0 msgr2=0x7ff728197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.477+0000 7ff72d69b700 1 --2- 172.21.15.71:0/1031545898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff728105bd0 0x7ff728197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.477+0000 7ff72d69b700 1 -- 172.21.15.71:0/1031545898 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7180095f0 con 0x7ff7280ffc20 2024-04-23T02:37:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.477+0000 7ff72ce9a700 1 --2- 172.21.15.71:0/1031545898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff728105bd0 0x7ff728197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:37:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.477+0000 7ff72d69b700 1 --2- 172.21.15.71:0/1031545898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7280ffc20 0x7ff7281976c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff72400c8a0 tx=0x7ff72400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.477+0000 7ff71e7fc700 1 -- 172.21.15.71:0/1031545898 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff72400cea0 con 0x7ff7280ffc20 2024-04-23T02:37:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.477+0000 7ff71e7fc700 1 -- 172.21.15.71:0/1031545898 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff724004830 con 0x7ff7280ffc20 2024-04-23T02:37:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.477+0000 7ff71e7fc700 1 -- 172.21.15.71:0/1031545898 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff724005500 con 0x7ff7280ffc20 2024-04-23T02:37:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.477+0000 7ff72f8ff700 1 -- 172.21.15.71:0/1031545898 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7281a0730 con 0x7ff7280ffc20 2024-04-23T02:37:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.477+0000 7ff72f8ff700 1 -- 172.21.15.71:0/1031545898 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7281a0bf0 con 0x7ff7280ffc20 2024-04-23T02:37:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.478+0000 7ff72f8ff700 1 -- 172.21.15.71:0/1031545898 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7281080d0 con 0x7ff7280ffc20 2024-04-23T02:37:19.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.479+0000 7ff71e7fc700 1 -- 172.21.15.71:0/1031545898 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff7240049a0 con 0x7ff7280ffc20 2024-04-23T02:37:19.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.479+0000 7ff71e7fc700 1 --2- 172.21.15.71:0/1031545898 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff71406cce0 0x7ff71406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:19.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.479+0000 7ff71e7fc700 1 -- 172.21.15.71:0/1031545898 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff72408c1d0 con 0x7ff7280ffc20 2024-04-23T02:37:19.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.479+0000 7ff72ce9a700 1 --2- 172.21.15.71:0/1031545898 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff71406cce0 0x7ff71406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:19.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.479+0000 7ff72ce9a700 1 --2- 172.21.15.71:0/1031545898 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff71406cce0 0x7ff71406f1a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff718006010 tx=0x7ff718005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:19.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.483+0000 7ff71e7fc700 1 -- 172.21.15.71:0/1031545898 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff7240562c0 con 0x7ff7280ffc20 2024-04-23T02:37:19.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:19 smithi071 ceph-mon[144544]: from='client.35126 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:37:19.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:19 smithi071 ceph-mon[144544]: from='client.35130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:37:19.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:19 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2950455251' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:37:19.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.695+0000 7ff72f8ff700 1 -- 172.21.15.71:0/1031545898 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7280611d0 con 0x7ff71406cce0 2024-04-23T02:37:19.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.696+0000 7ff71e7fc700 1 -- 172.21.15.71:0/1031545898 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff7280611d0 con 0x7ff71406cce0 2024-04-23T02:37:19.698 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:37:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:37:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:37:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:37:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:37:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:37:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:37:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:37:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:37:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:37:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:37:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:37:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:37:19.699 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:37:19.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.698+0000 7ff72f8ff700 1 -- 172.21.15.71:0/1031545898 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff71406cce0 msgr2=0x7ff71406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.698+0000 7ff72f8ff700 1 --2- 172.21.15.71:0/1031545898 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff71406cce0 0x7ff71406f1a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff718006010 tx=0x7ff718005960 comp rx=0 tx=0).stop 2024-04-23T02:37:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.698+0000 7ff72f8ff700 1 -- 172.21.15.71:0/1031545898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7280ffc20 msgr2=0x7ff7281976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.698+0000 7ff72f8ff700 1 --2- 172.21.15.71:0/1031545898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7280ffc20 0x7ff7281976c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff72400c8a0 tx=0x7ff72400cc60 comp rx=0 tx=0).stop 2024-04-23T02:37:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.698+0000 7ff72f8ff700 1 -- 172.21.15.71:0/1031545898 shutdown_connections 2024-04-23T02:37:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.698+0000 7ff72f8ff700 1 --2- 172.21.15.71:0/1031545898 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff71406cce0 0x7ff71406f1a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.698+0000 7ff72f8ff700 1 --2- 172.21.15.71:0/1031545898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7280ffc20 0x7ff7281976c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.698+0000 7ff72f8ff700 1 --2- 172.21.15.71:0/1031545898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff728105bd0 0x7ff728197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.698+0000 7ff72f8ff700 1 -- 172.21.15.71:0/1031545898 >> 172.21.15.71:0/1031545898 conn(0x7ff728075670 msgr2=0x7ff7280fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.698+0000 7ff72f8ff700 1 -- 172.21.15.71:0/1031545898 shutdown_connections 2024-04-23T02:37:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.698+0000 7ff72f8ff700 1 -- 172.21.15.71:0/1031545898 wait complete. 2024-04-23T02:37:19.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.801+0000 7fb4f0fac700 1 -- 172.21.15.71:0/3486898346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4ec100750 msgr2=0x7fb4ec1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:19.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.801+0000 7fb4f0fac700 1 --2- 172.21.15.71:0/3486898346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4ec100750 0x7fb4ec1047a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0009910 tx=0x7fb4e0009c20 comp rx=0 tx=0).stop 2024-04-23T02:37:19.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.801+0000 7fb4f0fac700 1 -- 172.21.15.71:0/3486898346 shutdown_connections 2024-04-23T02:37:19.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.801+0000 7fb4f0fac700 1 --2- 172.21.15.71:0/3486898346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4ec100750 0x7fb4ec1047a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.801+0000 7fb4f0fac700 1 --2- 172.21.15.71:0/3486898346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4ec0ffda0 0x7fb4ec100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.801+0000 7fb4f0fac700 1 -- 172.21.15.71:0/3486898346 >> 172.21.15.71:0/3486898346 conn(0x7fb4ec0fb650 msgr2=0x7fb4ec0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:19.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.801+0000 7fb4f0fac700 1 -- 172.21.15.71:0/3486898346 shutdown_connections 2024-04-23T02:37:19.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.801+0000 7fb4f0fac700 1 -- 172.21.15.71:0/3486898346 wait complete. 2024-04-23T02:37:19.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.802+0000 7fb4f0fac700 1 Processor -- start 2024-04-23T02:37:19.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.802+0000 7fb4f0fac700 1 -- start start 2024-04-23T02:37:19.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.802+0000 7fb4f0fac700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4ec0ffda0 0x7fb4ec1977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:19.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.802+0000 7fb4f0fac700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4ec100750 0x7fb4ec197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:19.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.802+0000 7fb4f0fac700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec198330 con 0x7fb4ec100750 2024-04-23T02:37:19.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.802+0000 7fb4f0fac700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec198470 con 0x7fb4ec0ffda0 2024-04-23T02:37:19.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.803+0000 7fb4e9d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4ec100750 0x7fb4ec197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:19.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.803+0000 7fb4e9d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4ec100750 0x7fb4ec197d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52798/0 (socket says 172.21.15.71:52798) 2024-04-23T02:37:19.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.803+0000 7fb4e9d9b700 1 -- 172.21.15.71:0/4154143108 learned_addr learned my addr 172.21.15.71:0/4154143108 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:37:19.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.803+0000 7fb4ea59c700 1 --2- 172.21.15.71:0/4154143108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4ec0ffda0 0x7fb4ec1977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:19.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.803+0000 7fb4e9d9b700 1 -- 172.21.15.71:0/4154143108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4ec0ffda0 msgr2=0x7fb4ec1977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:19.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.803+0000 7fb4e9d9b700 1 --2- 172.21.15.71:0/4154143108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4ec0ffda0 0x7fb4ec1977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:19.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.803+0000 7fb4e9d9b700 1 -- 172.21.15.71:0/4154143108 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e00095f0 con 0x7fb4ec100750 2024-04-23T02:37:19.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.803+0000 7fb4e9d9b700 1 --2- 172.21.15.71:0/4154143108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4ec100750 0x7fb4ec197d10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00092e0 tx=0x7fb4e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:19.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.803+0000 7fb4db7fe700 1 -- 172.21.15.71:0/4154143108 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4e001d070 con 0x7fb4ec100750 2024-04-23T02:37:19.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.803+0000 7fb4db7fe700 1 -- 172.21.15.71:0/4154143108 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb4e000f460 con 0x7fb4ec100750 2024-04-23T02:37:19.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.803+0000 7fb4db7fe700 1 -- 172.21.15.71:0/4154143108 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4e0021600 con 0x7fb4ec100750 2024-04-23T02:37:19.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.803+0000 7fb4f0fac700 1 -- 172.21.15.71:0/4154143108 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4ec1a0840 con 0x7fb4ec100750 2024-04-23T02:37:19.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.803+0000 7fb4f0fac700 1 -- 172.21.15.71:0/4154143108 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4ec1a0c50 con 0x7fb4ec100750 2024-04-23T02:37:19.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.804+0000 7fb4f0fac700 1 -- 172.21.15.71:0/4154143108 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4ec1080f0 con 0x7fb4ec100750 2024-04-23T02:37:19.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.805+0000 7fb4db7fe700 1 -- 172.21.15.71:0/4154143108 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb4e000f5d0 con 0x7fb4ec100750 2024-04-23T02:37:19.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.805+0000 7fb4db7fe700 1 --2- 172.21.15.71:0/4154143108 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb4d406cce0 0x7fb4d406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:19.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.805+0000 7fb4db7fe700 1 -- 172.21.15.71:0/4154143108 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb4e008d840 con 0x7fb4ec100750 2024-04-23T02:37:19.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.805+0000 7fb4ea59c700 1 --2- 172.21.15.71:0/4154143108 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb4d406cce0 0x7fb4d406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:19.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.805+0000 7fb4ea59c700 1 --2- 172.21.15.71:0/4154143108 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb4d406cce0 0x7fb4d406f1a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc006fd0 tx=0x7fb4dc008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:19.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:19.810+0000 7fb4db7fe700 1 -- 172.21.15.71:0/4154143108 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb4e00578b0 con 0x7fb4ec100750 2024-04-23T02:37:20.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:20.080+0000 7fb4f0fac700 1 -- 172.21.15.71:0/4154143108 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb4ec04f2e0 con 0x7fb4ec100750 2024-04-23T02:37:20.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:20.081+0000 7fb4db7fe700 1 -- 172.21.15.71:0/4154143108 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fb4e005aed0 con 0x7fb4ec100750 2024-04-23T02:37:20.083 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:37:20.083 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:37:20.083 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:37:20.083 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:37:20.084 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:37:20.084 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:37:20.084 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:37:20.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:20.083+0000 7fb4f0fac700 1 -- 172.21.15.71:0/4154143108 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb4d406cce0 msgr2=0x7fb4d406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:20.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:20.083+0000 7fb4f0fac700 1 --2- 172.21.15.71:0/4154143108 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb4d406cce0 0x7fb4d406f1a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc006fd0 tx=0x7fb4dc008040 comp rx=0 tx=0).stop 2024-04-23T02:37:20.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:20.083+0000 7fb4f0fac700 1 -- 172.21.15.71:0/4154143108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4ec100750 msgr2=0x7fb4ec197d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:20.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:20.083+0000 7fb4f0fac700 1 --2- 172.21.15.71:0/4154143108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4ec100750 0x7fb4ec197d10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00092e0 tx=0x7fb4e000be20 comp rx=0 tx=0).stop 2024-04-23T02:37:20.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:20.083+0000 7fb4f0fac700 1 -- 172.21.15.71:0/4154143108 shutdown_connections 2024-04-23T02:37:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:20.083+0000 7fb4f0fac700 1 --2- 172.21.15.71:0/4154143108 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb4d406cce0 0x7fb4d406f1a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:20.083+0000 7fb4f0fac700 1 --2- 172.21.15.71:0/4154143108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4ec0ffda0 0x7fb4ec1977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:20.083+0000 7fb4f0fac700 1 --2- 172.21.15.71:0/4154143108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4ec100750 0x7fb4ec197d10 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:20.083+0000 7fb4f0fac700 1 -- 172.21.15.71:0/4154143108 >> 172.21.15.71:0/4154143108 conn(0x7fb4ec0fb650 msgr2=0x7fb4ec0fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:20.083+0000 7fb4f0fac700 1 -- 172.21.15.71:0/4154143108 shutdown_connections 2024-04-23T02:37:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:20.083+0000 7fb4f0fac700 1 -- 172.21.15.71:0/4154143108 wait complete. 2024-04-23T02:37:20.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:20 smithi192 ceph-mon[108617]: from='client.35134 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:37:20.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:20 smithi192 ceph-mon[108617]: pgmap v720: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:20.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:20 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1746369053' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:37:20.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:37:20.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:20 smithi071 ceph-mon[144544]: from='client.35134 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:37:20.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:20 smithi071 ceph-mon[144544]: pgmap v720: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:20.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:20 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1746369053' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:37:20.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:37:21.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:21 smithi192 ceph-mon[108617]: from='client.35146 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:37:21.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:21 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/4154143108' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:37:21.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:21 smithi071 ceph-mon[144544]: from='client.35146 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:37:21.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:21 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/4154143108' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:37:22.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:22 smithi192 ceph-mon[108617]: pgmap v721: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:22.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:22 smithi071 ceph-mon[144544]: pgmap v721: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:24.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:24 smithi192 ceph-mon[108617]: pgmap v722: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:24.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:24 smithi071 ceph-mon[144544]: pgmap v722: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:26.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:26 smithi192 ceph-mon[108617]: pgmap v723: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:26.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:26 smithi071 ceph-mon[144544]: pgmap v723: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:28.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:28 smithi192 ceph-mon[108617]: pgmap v724: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:28.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:28 smithi071 ceph-mon[144544]: pgmap v724: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:30.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:30 smithi192 ceph-mon[108617]: pgmap v725: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:30.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:30 smithi071 ceph-mon[144544]: pgmap v725: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:32.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:32 smithi192 ceph-mon[108617]: pgmap v726: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:32.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:32 smithi071 ceph-mon[144544]: pgmap v726: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:34.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:34 smithi192 ceph-mon[108617]: pgmap v727: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:34.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:34 smithi071 ceph-mon[144544]: pgmap v727: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:35.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:37:35.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:37:36.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:36 smithi192 ceph-mon[108617]: pgmap v728: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:36.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:36 smithi071 ceph-mon[144544]: pgmap v728: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:38.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:38 smithi192 ceph-mon[108617]: pgmap v729: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:38.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:38 smithi071 ceph-mon[144544]: pgmap v729: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:40.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:40 smithi192 ceph-mon[108617]: pgmap v730: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:40.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:40 smithi071 ceph-mon[144544]: pgmap v730: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:42.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:42 smithi192 ceph-mon[108617]: pgmap v731: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:42.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:42 smithi071 ceph-mon[144544]: pgmap v731: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:44.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:44 smithi192 ceph-mon[108617]: pgmap v732: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:44.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:44 smithi071 ceph-mon[144544]: pgmap v732: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:46.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:46 smithi071 ceph-mon[144544]: pgmap v733: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:46.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:46 smithi192 ceph-mon[108617]: pgmap v733: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:48.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:48 smithi192 ceph-mon[108617]: pgmap v734: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:48.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:48 smithi071 ceph-mon[144544]: pgmap v734: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:50.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.199+0000 7f714dd5e700 1 -- 172.21.15.71:0/359860658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71480ffc20 msgr2=0x7f7148100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:50.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.199+0000 7f714dd5e700 1 --2- 172.21.15.71:0/359860658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71480ffc20 0x7f7148100080 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f713c009910 tx=0x7f713c009c20 comp rx=0 tx=0).stop 2024-04-23T02:37:50.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.199+0000 7f714dd5e700 1 -- 172.21.15.71:0/359860658 shutdown_connections 2024-04-23T02:37:50.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.199+0000 7f714dd5e700 1 --2- 172.21.15.71:0/359860658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71480ffc20 0x7f7148100080 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:50.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.199+0000 7f714dd5e700 1 --2- 172.21.15.71:0/359860658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7148105bd0 0x7f7148105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:50.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.199+0000 7f714dd5e700 1 -- 172.21.15.71:0/359860658 >> 172.21.15.71:0/359860658 conn(0x7f7148075670 msgr2=0x7f7148075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:50.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.199+0000 7f714dd5e700 1 -- 172.21.15.71:0/359860658 shutdown_connections 2024-04-23T02:37:50.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.199+0000 7f714dd5e700 1 -- 172.21.15.71:0/359860658 wait complete. 2024-04-23T02:37:50.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.200+0000 7f714dd5e700 1 Processor -- start 2024-04-23T02:37:50.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.200+0000 7f714dd5e700 1 -- start start 2024-04-23T02:37:50.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.200+0000 7f714dd5e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71480ffc20 0x7f714819c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:50.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.201+0000 7f714dd5e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7148105bd0 0x7f714819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:50.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.201+0000 7f714dd5e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714819d1c0 con 0x7f7148105bd0 2024-04-23T02:37:50.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.201+0000 7f714dd5e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7148196670 con 0x7f71480ffc20 2024-04-23T02:37:50.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.201+0000 7f7146ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7148105bd0 0x7f714819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:50.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.201+0000 7f7146ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7148105bd0 0x7f714819cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35990/0 (socket says 172.21.15.71:35990) 2024-04-23T02:37:50.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.201+0000 7f7146ffd700 1 -- 172.21.15.71:0/1980400867 learned_addr learned my addr 172.21.15.71:0/1980400867 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:37:50.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.201+0000 7f71477fe700 1 --2- 172.21.15.71:0/1980400867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71480ffc20 0x7f714819c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:50.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.201+0000 7f7146ffd700 1 -- 172.21.15.71:0/1980400867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71480ffc20 msgr2=0x7f714819c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:50.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.201+0000 7f7146ffd700 1 --2- 172.21.15.71:0/1980400867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71480ffc20 0x7f714819c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:50.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.201+0000 7f7146ffd700 1 -- 172.21.15.71:0/1980400867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713c0095f0 con 0x7f7148105bd0 2024-04-23T02:37:50.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.201+0000 7f7146ffd700 1 --2- 172.21.15.71:0/1980400867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7148105bd0 0x7f714819cb30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f713c006010 tx=0x7f713c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:50.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.201+0000 7f7144ff9700 1 -- 172.21.15.71:0/1980400867 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f713c01d070 con 0x7f7148105bd0 2024-04-23T02:37:50.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.201+0000 7f7144ff9700 1 -- 172.21.15.71:0/1980400867 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f713c00f460 con 0x7f7148105bd0 2024-04-23T02:37:50.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.201+0000 7f714dd5e700 1 -- 172.21.15.71:0/1980400867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71481968f0 con 0x7f7148105bd0 2024-04-23T02:37:50.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.201+0000 7f7144ff9700 1 -- 172.21.15.71:0/1980400867 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f713c021640 con 0x7f7148105bd0 2024-04-23T02:37:50.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.201+0000 7f714dd5e700 1 -- 172.21.15.71:0/1980400867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7148196de0 con 0x7f7148105bd0 2024-04-23T02:37:50.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.203+0000 7f714dd5e700 1 -- 172.21.15.71:0/1980400867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f714804f2e0 con 0x7f7148105bd0 2024-04-23T02:37:50.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.203+0000 7f7144ff9700 1 -- 172.21.15.71:0/1980400867 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f713c003a60 con 0x7f7148105bd0 2024-04-23T02:37:50.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.203+0000 7f7144ff9700 1 --2- 172.21.15.71:0/1980400867 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f713006cce0 0x7f713006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:50.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.203+0000 7f7144ff9700 1 -- 172.21.15.71:0/1980400867 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f713c08d830 con 0x7f7148105bd0 2024-04-23T02:37:50.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.203+0000 7f71477fe700 1 --2- 172.21.15.71:0/1980400867 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f713006cce0 0x7f713006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:50.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.204+0000 7f71477fe700 1 --2- 172.21.15.71:0/1980400867 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f713006cce0 0x7f713006f1a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7138005fd0 tx=0x7f713800a560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:50.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.208+0000 7f7144ff9700 1 -- 172.21.15.71:0/1980400867 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f713c05b330 con 0x7f7148105bd0 2024-04-23T02:37:50.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:50 smithi192 ceph-mon[108617]: pgmap v735: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:50.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:37:50.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.422+0000 7f714dd5e700 1 -- 172.21.15.71:0/1980400867 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7148197070 con 0x7f713006cce0 2024-04-23T02:37:50.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.424+0000 7f7144ff9700 1 -- 172.21.15.71:0/1980400867 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7148197070 con 0x7f713006cce0 2024-04-23T02:37:50.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.426+0000 7f714dd5e700 1 -- 172.21.15.71:0/1980400867 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f713006cce0 msgr2=0x7f713006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:50.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.426+0000 7f714dd5e700 1 --2- 172.21.15.71:0/1980400867 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f713006cce0 0x7f713006f1a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7138005fd0 tx=0x7f713800a560 comp rx=0 tx=0).stop 2024-04-23T02:37:50.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.426+0000 7f714dd5e700 1 -- 172.21.15.71:0/1980400867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7148105bd0 msgr2=0x7f714819cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:50.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.426+0000 7f714dd5e700 1 --2- 172.21.15.71:0/1980400867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7148105bd0 0x7f714819cb30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f713c006010 tx=0x7f713c00be20 comp rx=0 tx=0).stop 2024-04-23T02:37:50.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.426+0000 7f714dd5e700 1 -- 172.21.15.71:0/1980400867 shutdown_connections 2024-04-23T02:37:50.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.426+0000 7f714dd5e700 1 --2- 172.21.15.71:0/1980400867 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f713006cce0 0x7f713006f1a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:50.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.426+0000 7f714dd5e700 1 --2- 172.21.15.71:0/1980400867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71480ffc20 0x7f714819c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:50.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.426+0000 7f714dd5e700 1 --2- 172.21.15.71:0/1980400867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7148105bd0 0x7f714819cb30 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:50.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.426+0000 7f714dd5e700 1 -- 172.21.15.71:0/1980400867 >> 172.21.15.71:0/1980400867 conn(0x7f7148075670 msgr2=0x7f7148103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:50.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.426+0000 7f714dd5e700 1 -- 172.21.15.71:0/1980400867 shutdown_connections 2024-04-23T02:37:50.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.426+0000 7f714dd5e700 1 -- 172.21.15.71:0/1980400867 wait complete. 2024-04-23T02:37:50.439 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:37:50.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:50 smithi071 ceph-mon[144544]: pgmap v735: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:50.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:37:50.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.532+0000 7f2008037700 1 -- 172.21.15.71:0/3957836966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2000100e90 msgr2=0x7f2000101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:50.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.532+0000 7f2008037700 1 --2- 172.21.15.71:0/3957836966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2000100e90 0x7f2000101270 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009910 tx=0x7f1ff0009c20 comp rx=0 tx=0).stop 2024-04-23T02:37:50.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.532+0000 7f2008037700 1 -- 172.21.15.71:0/3957836966 shutdown_connections 2024-04-23T02:37:50.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.532+0000 7f2008037700 1 --2- 172.21.15.71:0/3957836966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2000101840 0x7f2000105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:50.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.532+0000 7f2008037700 1 --2- 172.21.15.71:0/3957836966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2000100e90 0x7f2000101270 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:50.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.532+0000 7f2008037700 1 -- 172.21.15.71:0/3957836966 >> 172.21.15.71:0/3957836966 conn(0x7f20000fc740 msgr2=0x7f20000feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.532+0000 7f2008037700 1 -- 172.21.15.71:0/3957836966 shutdown_connections 2024-04-23T02:37:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.532+0000 7f2008037700 1 -- 172.21.15.71:0/3957836966 wait complete. 2024-04-23T02:37:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.532+0000 7f2008037700 1 Processor -- start 2024-04-23T02:37:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.533+0000 7f2008037700 1 -- start start 2024-04-23T02:37:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.533+0000 7f2008037700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2000100e90 0x7f200019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.533+0000 7f2008037700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2000101840 0x7f200019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.533+0000 7f2008037700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200019d190 con 0x7f2000101840 2024-04-23T02:37:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.533+0000 7f2008037700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2000196640 con 0x7f2000100e90 2024-04-23T02:37:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.533+0000 7f20055d2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2000101840 0x7f200019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.533+0000 7f20055d2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2000101840 0x7f200019cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36004/0 (socket says 172.21.15.71:36004) 2024-04-23T02:37:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.533+0000 7f20055d2700 1 -- 172.21.15.71:0/4010257101 learned_addr learned my addr 172.21.15.71:0/4010257101 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:37:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.533+0000 7f2005dd3700 1 --2- 172.21.15.71:0/4010257101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2000100e90 0x7f200019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.533+0000 7f20055d2700 1 -- 172.21.15.71:0/4010257101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2000100e90 msgr2=0x7f200019c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.533+0000 7f20055d2700 1 --2- 172.21.15.71:0/4010257101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2000100e90 0x7f200019c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.533+0000 7f20055d2700 1 -- 172.21.15.71:0/4010257101 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff00095f0 con 0x7f2000101840 2024-04-23T02:37:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.533+0000 7f2005dd3700 1 --2- 172.21.15.71:0/4010257101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2000100e90 0x7f200019c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:37:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.533+0000 7f20055d2700 1 --2- 172.21.15.71:0/4010257101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2000101840 0x7f200019cb00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00d900 tx=0x7f1ffc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.533+0000 7f1ff6ffd700 1 -- 172.21.15.71:0/4010257101 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ffc009900 con 0x7f2000101840 2024-04-23T02:37:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.534+0000 7f1ff6ffd700 1 -- 172.21.15.71:0/4010257101 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1ffc010460 con 0x7f2000101840 2024-04-23T02:37:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.534+0000 7f1ff6ffd700 1 -- 172.21.15.71:0/4010257101 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ffc00f5f0 con 0x7f2000101840 2024-04-23T02:37:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.534+0000 7f2008037700 1 -- 172.21.15.71:0/4010257101 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2000196920 con 0x7f2000101840 2024-04-23T02:37:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.534+0000 7f2008037700 1 -- 172.21.15.71:0/4010257101 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2000196e70 con 0x7f2000101840 2024-04-23T02:37:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.534+0000 7f2008037700 1 -- 172.21.15.71:0/4010257101 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2000109230 con 0x7f2000101840 2024-04-23T02:37:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.535+0000 7f1ff6ffd700 1 -- 172.21.15.71:0/4010257101 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1ffc0105d0 con 0x7f2000101840 2024-04-23T02:37:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.535+0000 7f1ff6ffd700 1 --2- 172.21.15.71:0/4010257101 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1fec06cce0 0x7f1fec06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.535+0000 7f1ff6ffd700 1 -- 172.21.15.71:0/4010257101 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1ffc08c310 con 0x7f2000101840 2024-04-23T02:37:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.536+0000 7f2005dd3700 1 --2- 172.21.15.71:0/4010257101 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1fec06cce0 0x7f1fec06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.536+0000 7f2005dd3700 1 --2- 172.21.15.71:0/4010257101 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1fec06cce0 0x7f1fec06f1a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000b5c0 tx=0x7f1ff001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:50.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.540+0000 7f1ff6ffd700 1 -- 172.21.15.71:0/4010257101 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1ffc059e90 con 0x7f2000101840 2024-04-23T02:37:50.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.753+0000 7f2008037700 1 -- 172.21.15.71:0/4010257101 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f20001970e0 con 0x7f1fec06cce0 2024-04-23T02:37:50.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.755+0000 7f1ff6ffd700 1 -- 172.21.15.71:0/4010257101 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f20001970e0 con 0x7f1fec06cce0 2024-04-23T02:37:50.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.756+0000 7f2008037700 1 -- 172.21.15.71:0/4010257101 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1fec06cce0 msgr2=0x7f1fec06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:50.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.756+0000 7f2008037700 1 --2- 172.21.15.71:0/4010257101 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1fec06cce0 0x7f1fec06f1a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000b5c0 tx=0x7f1ff001a040 comp rx=0 tx=0).stop 2024-04-23T02:37:50.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.757+0000 7f2008037700 1 -- 172.21.15.71:0/4010257101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2000101840 msgr2=0x7f200019cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:50.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.757+0000 7f2008037700 1 --2- 172.21.15.71:0/4010257101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2000101840 0x7f200019cb00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00d900 tx=0x7f1ffc00dcc0 comp rx=0 tx=0).stop 2024-04-23T02:37:50.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.757+0000 7f2008037700 1 -- 172.21.15.71:0/4010257101 shutdown_connections 2024-04-23T02:37:50.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.757+0000 7f2008037700 1 --2- 172.21.15.71:0/4010257101 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1fec06cce0 0x7f1fec06f1a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:50.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.757+0000 7f2008037700 1 --2- 172.21.15.71:0/4010257101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2000100e90 0x7f200019c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:50.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.757+0000 7f2008037700 1 --2- 172.21.15.71:0/4010257101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2000101840 0x7f200019cb00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:50.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.757+0000 7f2008037700 1 -- 172.21.15.71:0/4010257101 >> 172.21.15.71:0/4010257101 conn(0x7f20000fc740 msgr2=0x7f20000fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:50.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.757+0000 7f2008037700 1 -- 172.21.15.71:0/4010257101 shutdown_connections 2024-04-23T02:37:50.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.757+0000 7f2008037700 1 -- 172.21.15.71:0/4010257101 wait complete. 2024-04-23T02:37:50.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.859+0000 7f5698881700 1 -- 172.21.15.71:0/2708736241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56940ffc20 msgr2=0x7f5694100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:50.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.859+0000 7f5698881700 1 --2- 172.21.15.71:0/2708736241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56940ffc20 0x7f5694100080 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5688009910 tx=0x7f5688009c20 comp rx=0 tx=0).stop 2024-04-23T02:37:50.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.860+0000 7f5698881700 1 -- 172.21.15.71:0/2708736241 shutdown_connections 2024-04-23T02:37:50.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.860+0000 7f5698881700 1 --2- 172.21.15.71:0/2708736241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56940ffc20 0x7f5694100080 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:50.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.860+0000 7f5698881700 1 --2- 172.21.15.71:0/2708736241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5694105bd0 0x7f5694105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:50.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.860+0000 7f5698881700 1 -- 172.21.15.71:0/2708736241 >> 172.21.15.71:0/2708736241 conn(0x7f5694075670 msgr2=0x7f5694075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:50.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.860+0000 7f5698881700 1 -- 172.21.15.71:0/2708736241 shutdown_connections 2024-04-23T02:37:50.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.860+0000 7f5698881700 1 -- 172.21.15.71:0/2708736241 wait complete. 2024-04-23T02:37:50.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.861+0000 7f5698881700 1 Processor -- start 2024-04-23T02:37:50.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.861+0000 7f5698881700 1 -- start start 2024-04-23T02:37:50.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.861+0000 7f5698881700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56940ffc20 0x7f569419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:50.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.861+0000 7f5698881700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5694105bd0 0x7f569419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:50.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.861+0000 7f5698881700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569419c0f0 con 0x7f5694105bd0 2024-04-23T02:37:50.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.861+0000 7f5698881700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56941955a0 con 0x7f56940ffc20 2024-04-23T02:37:50.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.861+0000 7f5691d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5694105bd0 0x7f569419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:50.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.861+0000 7f5691d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5694105bd0 0x7f569419ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36020/0 (socket says 172.21.15.71:36020) 2024-04-23T02:37:50.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.861+0000 7f5691d9b700 1 -- 172.21.15.71:0/1836854882 learned_addr learned my addr 172.21.15.71:0/1836854882 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:37:50.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.861+0000 7f569259c700 1 --2- 172.21.15.71:0/1836854882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56940ffc20 0x7f569419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:50.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.861+0000 7f5691d9b700 1 -- 172.21.15.71:0/1836854882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56940ffc20 msgr2=0x7f569419b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:50.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.861+0000 7f5691d9b700 1 --2- 172.21.15.71:0/1836854882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56940ffc20 0x7f569419b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:50.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.861+0000 7f5691d9b700 1 -- 172.21.15.71:0/1836854882 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56880095f0 con 0x7f5694105bd0 2024-04-23T02:37:50.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.862+0000 7f5691d9b700 1 --2- 172.21.15.71:0/1836854882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5694105bd0 0x7f569419ba60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5688006010 tx=0x7f568800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:50.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.862+0000 7f56837fe700 1 -- 172.21.15.71:0/1836854882 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f568801f070 con 0x7f5694105bd0 2024-04-23T02:37:50.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.862+0000 7f5698881700 1 -- 172.21.15.71:0/1836854882 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5694195820 con 0x7f5694105bd0 2024-04-23T02:37:50.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.862+0000 7f56837fe700 1 -- 172.21.15.71:0/1836854882 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5688005710 con 0x7f5694105bd0 2024-04-23T02:37:50.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.862+0000 7f56837fe700 1 -- 172.21.15.71:0/1836854882 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f568800f460 con 0x7f5694105bd0 2024-04-23T02:37:50.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.862+0000 7f5698881700 1 -- 172.21.15.71:0/1836854882 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5694195d10 con 0x7f5694105bd0 2024-04-23T02:37:50.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.863+0000 7f5698881700 1 -- 172.21.15.71:0/1836854882 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f569404f2e0 con 0x7f5694105bd0 2024-04-23T02:37:50.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.863+0000 7f56837fe700 1 -- 172.21.15.71:0/1836854882 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f568800f5e0 con 0x7f5694105bd0 2024-04-23T02:37:50.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.864+0000 7f56837fe700 1 --2- 172.21.15.71:0/1836854882 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f567c06cce0 0x7f567c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:50.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.864+0000 7f56837fe700 1 -- 172.21.15.71:0/1836854882 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f568808f920 con 0x7f5694105bd0 2024-04-23T02:37:50.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.864+0000 7f569259c700 1 --2- 172.21.15.71:0/1836854882 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f567c06cce0 0x7f567c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:50.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.864+0000 7f569259c700 1 --2- 172.21.15.71:0/1836854882 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f567c06cce0 0x7f567c06f1a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5684005950 tx=0x7f568400b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:50.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:50.868+0000 7f56837fe700 1 -- 172.21.15.71:0/1836854882 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f568805d420 con 0x7f5694105bd0 2024-04-23T02:37:51.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.073+0000 7f5698881700 1 -- 172.21.15.71:0/1836854882 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5694195f90 con 0x7f567c06cce0 2024-04-23T02:37:51.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.084+0000 7f56837fe700 1 -- 172.21.15.71:0/1836854882 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5694195f90 con 0x7f567c06cce0 2024-04-23T02:37:51.086 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:37:51.086 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (21m) 15m ago 32m 26.5M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:37:51.086 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (33m) 15m ago 33m 8442k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:37:51.087 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (31m) 14m ago 31m 8400k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:37:51.087 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (19m) 15m ago 33m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:37:51.087 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (19m) 14m ago 31m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:37:51.087 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (21m) 15m ago 32m 85.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:37:51.087 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (15m) 15m ago 26m 13.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:37:51.087 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (15m) 15m ago 26m 11.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:37:51.087 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (26m) 14m ago 26m 12.0M - 18.2.1 5be31c24972a 663023fcb14d 2024-04-23T02:37:51.087 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (15m) 14m ago 26m 12.9M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:37:51.087 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (23m) 15m ago 34m 511M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:37:51.087 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (22m) 14m ago 31m 454M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:37:51.087 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (19m) 15m ago 34m 53.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:37:51.087 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (19m) 14m ago 31m 44.6M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:37:51.087 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (22m) 15m ago 32m 19.2M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:37:51.087 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (22m) 14m ago 31m 18.4M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:37:51.088 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (18m) 15m ago 30m 90.4M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:37:51.088 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (18m) 15m ago 30m 89.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:37:51.088 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (17m) 15m ago 29m 81.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:37:51.088 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (17m) 14m ago 29m 103M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:37:51.088 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (16m) 14m ago 28m 91.0M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:37:51.088 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (16m) 14m ago 28m 81.3M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:37:51.088 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (21m) 15m ago 32m 65.7M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:37:51.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.086+0000 7f5698881700 1 -- 172.21.15.71:0/1836854882 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f567c06cce0 msgr2=0x7f567c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:51.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.086+0000 7f5698881700 1 --2- 172.21.15.71:0/1836854882 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f567c06cce0 0x7f567c06f1a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5684005950 tx=0x7f568400b410 comp rx=0 tx=0).stop 2024-04-23T02:37:51.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.086+0000 7f5698881700 1 -- 172.21.15.71:0/1836854882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5694105bd0 msgr2=0x7f569419ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:51.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.086+0000 7f5698881700 1 --2- 172.21.15.71:0/1836854882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5694105bd0 0x7f569419ba60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5688006010 tx=0x7f568800bd70 comp rx=0 tx=0).stop 2024-04-23T02:37:51.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.086+0000 7f5698881700 1 -- 172.21.15.71:0/1836854882 shutdown_connections 2024-04-23T02:37:51.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.086+0000 7f5698881700 1 --2- 172.21.15.71:0/1836854882 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f567c06cce0 0x7f567c06f1a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.086+0000 7f5698881700 1 --2- 172.21.15.71:0/1836854882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56940ffc20 0x7f569419b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.086+0000 7f5698881700 1 --2- 172.21.15.71:0/1836854882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5694105bd0 0x7f569419ba60 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.086+0000 7f5698881700 1 -- 172.21.15.71:0/1836854882 >> 172.21.15.71:0/1836854882 conn(0x7f5694075670 msgr2=0x7f56940fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:51.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.086+0000 7f5698881700 1 -- 172.21.15.71:0/1836854882 shutdown_connections 2024-04-23T02:37:51.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.086+0000 7f5698881700 1 -- 172.21.15.71:0/1836854882 wait complete. 2024-04-23T02:37:51.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.189+0000 7fbda5141700 1 -- 172.21.15.71:0/1286002164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbda0101840 msgr2=0x7fbda0105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:51.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.189+0000 7fbda5141700 1 --2- 172.21.15.71:0/1286002164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbda0101840 0x7fbda0105890 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009910 tx=0x7fbd94009c20 comp rx=0 tx=0).stop 2024-04-23T02:37:51.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.189+0000 7fbda5141700 1 -- 172.21.15.71:0/1286002164 shutdown_connections 2024-04-23T02:37:51.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.189+0000 7fbda5141700 1 --2- 172.21.15.71:0/1286002164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbda0101840 0x7fbda0105890 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.189+0000 7fbda5141700 1 --2- 172.21.15.71:0/1286002164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda0100e90 0x7fbda0101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.189+0000 7fbda5141700 1 -- 172.21.15.71:0/1286002164 >> 172.21.15.71:0/1286002164 conn(0x7fbda00fc740 msgr2=0x7fbda00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:51.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.189+0000 7fbda5141700 1 -- 172.21.15.71:0/1286002164 shutdown_connections 2024-04-23T02:37:51.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.189+0000 7fbda5141700 1 -- 172.21.15.71:0/1286002164 wait complete. 2024-04-23T02:37:51.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.190+0000 7fbda5141700 1 Processor -- start 2024-04-23T02:37:51.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.190+0000 7fbda5141700 1 -- start start 2024-04-23T02:37:51.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.190+0000 7fbda5141700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbda0100e90 0x7fbda019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:51.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.190+0000 7fbda5141700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda0101840 0x7fbda019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:51.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.190+0000 7fbda5141700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda019d200 con 0x7fbda0100e90 2024-04-23T02:37:51.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.190+0000 7fbda5141700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda01966b0 con 0x7fbda0101840 2024-04-23T02:37:51.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.190+0000 7fbd9ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbda0100e90 0x7fbda019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:51.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.190+0000 7fbd9ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbda0100e90 0x7fbda019c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36026/0 (socket says 172.21.15.71:36026) 2024-04-23T02:37:51.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.190+0000 7fbd9ed9d700 1 -- 172.21.15.71:0/1920398956 learned_addr learned my addr 172.21.15.71:0/1920398956 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:37:51.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.191+0000 7fbd9e59c700 1 --2- 172.21.15.71:0/1920398956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda0101840 0x7fbda019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:51.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.191+0000 7fbd9ed9d700 1 -- 172.21.15.71:0/1920398956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda0101840 msgr2=0x7fbda019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:51.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.191+0000 7fbd9ed9d700 1 --2- 172.21.15.71:0/1920398956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda0101840 0x7fbda019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.191+0000 7fbd9ed9d700 1 -- 172.21.15.71:0/1920398956 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd940095f0 con 0x7fbda0100e90 2024-04-23T02:37:51.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.191+0000 7fbd9ed9d700 1 --2- 172.21.15.71:0/1920398956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbda0100e90 0x7fbda019c630 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000c8a0 tx=0x7fbd9000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:51.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.191+0000 7fbd87fff700 1 -- 172.21.15.71:0/1920398956 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd900078e0 con 0x7fbda0100e90 2024-04-23T02:37:51.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.191+0000 7fbd87fff700 1 -- 172.21.15.71:0/1920398956 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbd9000f450 con 0x7fbda0100e90 2024-04-23T02:37:51.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.191+0000 7fbda5141700 1 -- 172.21.15.71:0/1920398956 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda0196990 con 0x7fbda0100e90 2024-04-23T02:37:51.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.191+0000 7fbd87fff700 1 -- 172.21.15.71:0/1920398956 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd9000e5e0 con 0x7fbda0100e90 2024-04-23T02:37:51.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.191+0000 7fbda5141700 1 -- 172.21.15.71:0/1920398956 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda0196eb0 con 0x7fbda0100e90 2024-04-23T02:37:51.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.192+0000 7fbda5141700 1 -- 172.21.15.71:0/1920398956 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda0109290 con 0x7fbda0100e90 2024-04-23T02:37:51.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.193+0000 7fbd87fff700 1 -- 172.21.15.71:0/1920398956 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbd9000f5c0 con 0x7fbda0100e90 2024-04-23T02:37:51.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.194+0000 7fbd87fff700 1 --2- 172.21.15.71:0/1920398956 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbd8806cce0 0x7fbd8806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:51.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.194+0000 7fbd9e59c700 1 --2- 172.21.15.71:0/1920398956 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbd8806cce0 0x7fbd8806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:51.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.194+0000 7fbd87fff700 1 -- 172.21.15.71:0/1920398956 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbd9008c670 con 0x7fbda0100e90 2024-04-23T02:37:51.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.194+0000 7fbd9e59c700 1 --2- 172.21.15.71:0/1920398956 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbd8806cce0 0x7fbd8806f1a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbd94006010 tx=0x7fbd9401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:51.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.197+0000 7fbd87fff700 1 -- 172.21.15.71:0/1920398956 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbd9005a1d0 con 0x7fbda0100e90 2024-04-23T02:37:51.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:51 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:37:51.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:51 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:37:51.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:51 smithi192 ceph-mon[108617]: pgmap v736: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:51.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:51 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:37:51.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:51 smithi192 ceph-mon[108617]: from='client.35154 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:37:51.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:51 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:37:51.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:51 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:37:51.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:51 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:37:51.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:51 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:37:51.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:51 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:37:51.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.471+0000 7fbda5141700 1 -- 172.21.15.71:0/1920398956 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbda004f2e0 con 0x7fbda0100e90 2024-04-23T02:37:51.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.472+0000 7fbd87fff700 1 -- 172.21.15.71:0/1920398956 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7fbd90059d60 con 0x7fbda0100e90 2024-04-23T02:37:51.474 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:37:51.474 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:37:51.474 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:37:51.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:37:51.475 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:37:51.475 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:37:51.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:37:51.475 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:37:51.475 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:37:51.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:37:51.475 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:37:51.475 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:37:51.475 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 3 2024-04-23T02:37:51.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:37:51.475 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:37:51.475 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-23T02:37:51.475 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 13 2024-04-23T02:37:51.475 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:37:51.476 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:37:51.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.474+0000 7fbda5141700 1 -- 172.21.15.71:0/1920398956 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbd8806cce0 msgr2=0x7fbd8806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:51.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.474+0000 7fbda5141700 1 --2- 172.21.15.71:0/1920398956 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbd8806cce0 0x7fbd8806f1a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbd94006010 tx=0x7fbd9401a040 comp rx=0 tx=0).stop 2024-04-23T02:37:51.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.474+0000 7fbda5141700 1 -- 172.21.15.71:0/1920398956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbda0100e90 msgr2=0x7fbda019c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:51.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.474+0000 7fbda5141700 1 --2- 172.21.15.71:0/1920398956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbda0100e90 0x7fbda019c630 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000c8a0 tx=0x7fbd9000cbb0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.474+0000 7fbda5141700 1 -- 172.21.15.71:0/1920398956 shutdown_connections 2024-04-23T02:37:51.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.474+0000 7fbda5141700 1 --2- 172.21.15.71:0/1920398956 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbd8806cce0 0x7fbd8806f1a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.474+0000 7fbda5141700 1 --2- 172.21.15.71:0/1920398956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbda0100e90 0x7fbda019c630 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.474+0000 7fbda5141700 1 --2- 172.21.15.71:0/1920398956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda0101840 0x7fbda019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.474+0000 7fbda5141700 1 -- 172.21.15.71:0/1920398956 >> 172.21.15.71:0/1920398956 conn(0x7fbda00fc740 msgr2=0x7fbda00fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:51.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.474+0000 7fbda5141700 1 -- 172.21.15.71:0/1920398956 shutdown_connections 2024-04-23T02:37:51.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.474+0000 7fbda5141700 1 -- 172.21.15.71:0/1920398956 wait complete. 2024-04-23T02:37:51.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:51 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:37:51.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:51 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:37:51.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:51 smithi071 ceph-mon[144544]: pgmap v736: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:51.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:51 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:37:51.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:51 smithi071 ceph-mon[144544]: from='client.35154 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:37:51.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:51 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:37:51.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:51 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:37:51.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:51 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:37:51.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:51 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:37:51.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:51 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:37:51.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.576+0000 7fc431b09700 1 -- 172.21.15.71:0/3282486736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc42c101060 msgr2=0x7fc42c1047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:51.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.576+0000 7fc431b09700 1 --2- 172.21.15.71:0/3282486736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc42c101060 0x7fc42c1047b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc420009910 tx=0x7fc420009c20 comp rx=0 tx=0).stop 2024-04-23T02:37:51.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.576+0000 7fc431b09700 1 -- 172.21.15.71:0/3282486736 shutdown_connections 2024-04-23T02:37:51.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.576+0000 7fc431b09700 1 --2- 172.21.15.71:0/3282486736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc42c101060 0x7fc42c1047b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.576+0000 7fc431b09700 1 --2- 172.21.15.71:0/3282486736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc42c1006b0 0x7fc42c100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.576+0000 7fc431b09700 1 -- 172.21.15.71:0/3282486736 >> 172.21.15.71:0/3282486736 conn(0x7fc42c074e30 msgr2=0x7fc42c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:51.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.577+0000 7fc431b09700 1 -- 172.21.15.71:0/3282486736 shutdown_connections 2024-04-23T02:37:51.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.577+0000 7fc431b09700 1 -- 172.21.15.71:0/3282486736 wait complete. 2024-04-23T02:37:51.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.577+0000 7fc431b09700 1 Processor -- start 2024-04-23T02:37:51.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.577+0000 7fc431b09700 1 -- start start 2024-04-23T02:37:51.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.577+0000 7fc431b09700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc42c1006b0 0x7fc42c101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:51.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.577+0000 7fc431b09700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc42c101060 0x7fc42c101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:51.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.577+0000 7fc431b09700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c1022d0 con 0x7fc42c1006b0 2024-04-23T02:37:51.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.577+0000 7fc431b09700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c102410 con 0x7fc42c101060 2024-04-23T02:37:51.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.578+0000 7fc42b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc42c1006b0 0x7fc42c101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:51.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.578+0000 7fc42b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc42c1006b0 0x7fc42c101850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36042/0 (socket says 172.21.15.71:36042) 2024-04-23T02:37:51.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.578+0000 7fc42b7fe700 1 -- 172.21.15.71:0/2625451328 learned_addr learned my addr 172.21.15.71:0/2625451328 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:37:51.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.578+0000 7fc42affd700 1 --2- 172.21.15.71:0/2625451328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc42c101060 0x7fc42c101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:51.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.578+0000 7fc42b7fe700 1 -- 172.21.15.71:0/2625451328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc42c101060 msgr2=0x7fc42c101d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:51.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.578+0000 7fc42b7fe700 1 --2- 172.21.15.71:0/2625451328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc42c101060 0x7fc42c101d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.578+0000 7fc42b7fe700 1 -- 172.21.15.71:0/2625451328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4200095f0 con 0x7fc42c1006b0 2024-04-23T02:37:51.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.578+0000 7fc42b7fe700 1 --2- 172.21.15.71:0/2625451328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc42c1006b0 0x7fc42c101850 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00e920 tx=0x7fc41c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:51.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.578+0000 7fc428ff9700 1 -- 172.21.15.71:0/2625451328 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc41c00c9d0 con 0x7fc42c1006b0 2024-04-23T02:37:51.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.578+0000 7fc428ff9700 1 -- 172.21.15.71:0/2625451328 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc41c010430 con 0x7fc42c1006b0 2024-04-23T02:37:51.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.578+0000 7fc428ff9700 1 -- 172.21.15.71:0/2625451328 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc41c018650 con 0x7fc42c1006b0 2024-04-23T02:37:51.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.578+0000 7fc431b09700 1 -- 172.21.15.71:0/2625451328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc42c1a2980 con 0x7fc42c1006b0 2024-04-23T02:37:51.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.578+0000 7fc431b09700 1 -- 172.21.15.71:0/2625451328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc42c1a2d30 con 0x7fc42c1006b0 2024-04-23T02:37:51.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.579+0000 7fc431b09700 1 -- 172.21.15.71:0/2625451328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc42c108090 con 0x7fc42c1006b0 2024-04-23T02:37:51.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.580+0000 7fc428ff9700 1 -- 172.21.15.71:0/2625451328 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc41c0105a0 con 0x7fc42c1006b0 2024-04-23T02:37:51.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.580+0000 7fc428ff9700 1 --2- 172.21.15.71:0/2625451328 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc41406cce0 0x7fc41406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:51.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.580+0000 7fc428ff9700 1 -- 172.21.15.71:0/2625451328 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc41c014070 con 0x7fc42c1006b0 2024-04-23T02:37:51.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.580+0000 7fc42affd700 1 --2- 172.21.15.71:0/2625451328 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc41406cce0 0x7fc41406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:51.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.580+0000 7fc42affd700 1 --2- 172.21.15.71:0/2625451328 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc41406cce0 0x7fc41406f1a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc420009fd0 tx=0x7fc420005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:51.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.584+0000 7fc428ff9700 1 -- 172.21.15.71:0/2625451328 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc41c052c50 con 0x7fc42c1006b0 2024-04-23T02:37:51.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.817+0000 7fc431b09700 1 -- 172.21.15.71:0/2625451328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc42c04f2e0 con 0x7fc42c1006b0 2024-04-23T02:37:51.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.818+0000 7fc428ff9700 1 -- 172.21.15.71:0/2625451328 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1836 (secure 0 0 0) 0x7fc41c056910 con 0x7fc42c1006b0 2024-04-23T02:37:51.820 INFO:teuthology.orchestra.run.smithi071.stdout:e34 2024-04-23T02:37:51.820 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:37:51.820 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:37:51.820 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:37:51.821 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:37:51.821 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:37:51.821 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:37:51.821 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:37:51.821 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:37:51.821 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:37:51.821 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:37:51.821 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:37:51.821 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:37:51.821 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:37:51.821 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:37:51.821 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:37:51.821 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:37:51.821 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:37:51.822 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:37:51.822 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:37:51.822 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:37:51.822 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:37:51.822 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:37:51.822 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:37:51.822 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:37:51.822 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:37:51.822 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:37:51.822 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:37:51.822 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:37:51.822 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:37:51.822 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:37:51.822 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:37:51.823 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:37:51.823 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:37:51.823 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:37:51.823 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:37:51.823 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:37:51.823 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:37:51.823 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:37:51.823 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44263} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/418886657,v1:172.21.15.192:6827/418886657] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:37:51.823 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:37:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.820+0000 7fc431b09700 1 -- 172.21.15.71:0/2625451328 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc41406cce0 msgr2=0x7fc41406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.820+0000 7fc431b09700 1 --2- 172.21.15.71:0/2625451328 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc41406cce0 0x7fc41406f1a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc420009fd0 tx=0x7fc420005940 comp rx=0 tx=0).stop 2024-04-23T02:37:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.820+0000 7fc431b09700 1 -- 172.21.15.71:0/2625451328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc42c1006b0 msgr2=0x7fc42c101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.820+0000 7fc431b09700 1 --2- 172.21.15.71:0/2625451328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc42c1006b0 0x7fc42c101850 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00e920 tx=0x7fc41c00ec30 comp rx=0 tx=0).stop 2024-04-23T02:37:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.820+0000 7fc431b09700 1 -- 172.21.15.71:0/2625451328 shutdown_connections 2024-04-23T02:37:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.820+0000 7fc431b09700 1 --2- 172.21.15.71:0/2625451328 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc41406cce0 0x7fc41406f1a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.820+0000 7fc431b09700 1 --2- 172.21.15.71:0/2625451328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc42c1006b0 0x7fc42c101850 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.820+0000 7fc431b09700 1 --2- 172.21.15.71:0/2625451328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc42c101060 0x7fc42c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.820+0000 7fc431b09700 1 -- 172.21.15.71:0/2625451328 >> 172.21.15.71:0/2625451328 conn(0x7fc42c074e30 msgr2=0x7fc42c103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.820+0000 7fc431b09700 1 -- 172.21.15.71:0/2625451328 shutdown_connections 2024-04-23T02:37:51.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.820+0000 7fc431b09700 1 -- 172.21.15.71:0/2625451328 wait complete. 2024-04-23T02:37:51.825 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:37:51.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.925+0000 7f05a626e700 1 -- 172.21.15.71:0/2284062137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05a0106cd0 msgr2=0x7f05a01070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:51.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.925+0000 7f05a626e700 1 --2- 172.21.15.71:0/2284062137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05a0106cd0 0x7f05a01070b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0594009910 tx=0x7f0594009c20 comp rx=0 tx=0).stop 2024-04-23T02:37:51.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.925+0000 7f05a626e700 1 -- 172.21.15.71:0/2284062137 shutdown_connections 2024-04-23T02:37:51.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.925+0000 7f05a626e700 1 --2- 172.21.15.71:0/2284062137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a0068580 0x7f05a00689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.925+0000 7f05a626e700 1 --2- 172.21.15.71:0/2284062137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05a0106cd0 0x7f05a01070b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.925+0000 7f05a626e700 1 -- 172.21.15.71:0/2284062137 >> 172.21.15.71:0/2284062137 conn(0x7f05a0075670 msgr2=0x7f05a0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:51.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.925+0000 7f05a626e700 1 -- 172.21.15.71:0/2284062137 shutdown_connections 2024-04-23T02:37:51.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.925+0000 7f05a626e700 1 -- 172.21.15.71:0/2284062137 wait complete. 2024-04-23T02:37:51.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.925+0000 7f05a626e700 1 Processor -- start 2024-04-23T02:37:51.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.925+0000 7f05a626e700 1 -- start start 2024-04-23T02:37:51.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.926+0000 7f05a626e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05a0068580 0x7f05a019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:51.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.926+0000 7f05a626e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a0106cd0 0x7f05a019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:51.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.926+0000 7f05a626e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a019d130 con 0x7f05a0068580 2024-04-23T02:37:51.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.926+0000 7f05a626e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a0196630 con 0x7f05a0106cd0 2024-04-23T02:37:51.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.926+0000 7f059ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05a0068580 0x7f05a019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:51.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.926+0000 7f059ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05a0068580 0x7f05a019c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36056/0 (socket says 172.21.15.71:36056) 2024-04-23T02:37:51.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.926+0000 7f059ffff700 1 -- 172.21.15.71:0/779293072 learned_addr learned my addr 172.21.15.71:0/779293072 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:37:51.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.926+0000 7f059f7fe700 1 --2- 172.21.15.71:0/779293072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a0106cd0 0x7f05a019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.926+0000 7f059ffff700 1 -- 172.21.15.71:0/779293072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a0106cd0 msgr2=0x7f05a019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.926+0000 7f059ffff700 1 --2- 172.21.15.71:0/779293072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a0106cd0 0x7f05a019caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.926+0000 7f059ffff700 1 -- 172.21.15.71:0/779293072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05940095f0 con 0x7f05a0068580 2024-04-23T02:37:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.926+0000 7f059ffff700 1 --2- 172.21.15.71:0/779293072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05a0068580 0x7f05a019c560 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0594000c00 tx=0x7f059400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.926+0000 7f059d7fa700 1 -- 172.21.15.71:0/779293072 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f059401d070 con 0x7f05a0068580 2024-04-23T02:37:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.926+0000 7f05a626e700 1 -- 172.21.15.71:0/779293072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a01968b0 con 0x7f05a0068580 2024-04-23T02:37:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.926+0000 7f059d7fa700 1 -- 172.21.15.71:0/779293072 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f05940044c0 con 0x7f05a0068580 2024-04-23T02:37:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.926+0000 7f059d7fa700 1 -- 172.21.15.71:0/779293072 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05940039d0 con 0x7f05a0068580 2024-04-23T02:37:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.926+0000 7f05a626e700 1 -- 172.21.15.71:0/779293072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a0196d40 con 0x7f05a0068580 2024-04-23T02:37:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.927+0000 7f05a626e700 1 -- 172.21.15.71:0/779293072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a01091f0 con 0x7f05a0068580 2024-04-23T02:37:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.928+0000 7f059d7fa700 1 -- 172.21.15.71:0/779293072 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f05940048e0 con 0x7f05a0068580 2024-04-23T02:37:51.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.928+0000 7f059d7fa700 1 --2- 172.21.15.71:0/779293072 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f058806cce0 0x7f058806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:51.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.928+0000 7f059f7fe700 1 --2- 172.21.15.71:0/779293072 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f058806cce0 0x7f058806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:51.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.928+0000 7f059d7fa700 1 -- 172.21.15.71:0/779293072 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f059408dc60 con 0x7f05a0068580 2024-04-23T02:37:51.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.929+0000 7f059f7fe700 1 --2- 172.21.15.71:0/779293072 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f058806cce0 0x7f058806f1a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f05a00fe390 tx=0x7f0590009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:51.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:51.933+0000 7f059d7fa700 1 -- 172.21.15.71:0/779293072 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f059405b760 con 0x7f05a0068580 2024-04-23T02:37:52.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.144+0000 7f05a626e700 1 -- 172.21.15.71:0/779293072 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f05a0196fc0 con 0x7f058806cce0 2024-04-23T02:37:52.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.145+0000 7f059d7fa700 1 -- 172.21.15.71:0/779293072 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f05a0196fc0 con 0x7f058806cce0 2024-04-23T02:37:52.148 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:37:52.150 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:37:52.150 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:37:52.150 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:37:52.150 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:37:52.150 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-04-23T02:37:52.150 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:37:52.150 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:37:52.150 INFO:teuthology.orchestra.run.smithi071.stdout: "osd" 2024-04-23T02:37:52.150 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:37:52.150 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "15/23 daemons upgraded", 2024-04-23T02:37:52.151 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:37:52.151 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:37:52.151 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:37:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.147+0000 7f05a626e700 1 -- 172.21.15.71:0/779293072 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f058806cce0 msgr2=0x7f058806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.147+0000 7f05a626e700 1 --2- 172.21.15.71:0/779293072 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f058806cce0 0x7f058806f1a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f05a00fe390 tx=0x7f0590009260 comp rx=0 tx=0).stop 2024-04-23T02:37:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.147+0000 7f05a626e700 1 -- 172.21.15.71:0/779293072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05a0068580 msgr2=0x7f05a019c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.147+0000 7f05a626e700 1 --2- 172.21.15.71:0/779293072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05a0068580 0x7f05a019c560 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0594000c00 tx=0x7f059400be90 comp rx=0 tx=0).stop 2024-04-23T02:37:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.147+0000 7f05a626e700 1 -- 172.21.15.71:0/779293072 shutdown_connections 2024-04-23T02:37:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.147+0000 7f05a626e700 1 --2- 172.21.15.71:0/779293072 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f058806cce0 0x7f058806f1a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.147+0000 7f05a626e700 1 --2- 172.21.15.71:0/779293072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05a0068580 0x7f05a019c560 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.147+0000 7f05a626e700 1 --2- 172.21.15.71:0/779293072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a0106cd0 0x7f05a019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.147+0000 7f05a626e700 1 -- 172.21.15.71:0/779293072 >> 172.21.15.71:0/779293072 conn(0x7f05a0075670 msgr2=0x7f05a00fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.147+0000 7f05a626e700 1 -- 172.21.15.71:0/779293072 shutdown_connections 2024-04-23T02:37:52.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.147+0000 7f05a626e700 1 -- 172.21.15.71:0/779293072 wait complete. 2024-04-23T02:37:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.250+0000 7f8921869700 1 -- 172.21.15.71:0/4130177788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891c105bf0 msgr2=0x7f891c105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.250+0000 7f8921869700 1 --2- 172.21.15.71:0/4130177788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891c105bf0 0x7f891c105fd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8910009910 tx=0x7f8910009c20 comp rx=0 tx=0).stop 2024-04-23T02:37:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.250+0000 7f8921869700 1 -- 172.21.15.71:0/4130177788 shutdown_connections 2024-04-23T02:37:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.250+0000 7f8921869700 1 --2- 172.21.15.71:0/4130177788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f891c0ffb40 0x7f891c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.250+0000 7f8921869700 1 --2- 172.21.15.71:0/4130177788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891c105bf0 0x7f891c105fd0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.250+0000 7f8921869700 1 -- 172.21.15.71:0/4130177788 >> 172.21.15.71:0/4130177788 conn(0x7f891c0fb650 msgr2=0x7f891c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:52.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.250+0000 7f8921869700 1 -- 172.21.15.71:0/4130177788 shutdown_connections 2024-04-23T02:37:52.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.250+0000 7f8921869700 1 -- 172.21.15.71:0/4130177788 wait complete. 2024-04-23T02:37:52.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.251+0000 7f8921869700 1 Processor -- start 2024-04-23T02:37:52.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.251+0000 7f8921869700 1 -- start start 2024-04-23T02:37:52.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.251+0000 7f8921869700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f891c0ffb40 0x7f891c19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:52.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.251+0000 7f8921869700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891c105bf0 0x7f891c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:52.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.251+0000 7f8921869700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c19d0c0 con 0x7f891c105bf0 2024-04-23T02:37:52.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.251+0000 7f8921869700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c1965c0 con 0x7f891c0ffb40 2024-04-23T02:37:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.251+0000 7f891a7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891c105bf0 0x7f891c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.251+0000 7f891a7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891c105bf0 0x7f891c19ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36076/0 (socket says 172.21.15.71:36076) 2024-04-23T02:37:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.251+0000 7f891a7fc700 1 -- 172.21.15.71:0/289533379 learned_addr learned my addr 172.21.15.71:0/289533379 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:37:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.251+0000 7f891affd700 1 --2- 172.21.15.71:0/289533379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f891c0ffb40 0x7f891c19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.252+0000 7f891a7fc700 1 -- 172.21.15.71:0/289533379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f891c0ffb40 msgr2=0x7f891c19c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.252+0000 7f891a7fc700 1 --2- 172.21.15.71:0/289533379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f891c0ffb40 0x7f891c19c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.252+0000 7f891a7fc700 1 -- 172.21.15.71:0/289533379 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89100095f0 con 0x7f891c105bf0 2024-04-23T02:37:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.252+0000 7f891a7fc700 1 --2- 172.21.15.71:0/289533379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891c105bf0 0x7f891c19ca30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f890c00d8d0 tx=0x7f890c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.252+0000 7f8920867700 1 -- 172.21.15.71:0/289533379 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f890c0098a0 con 0x7f891c105bf0 2024-04-23T02:37:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.252+0000 7f8920867700 1 -- 172.21.15.71:0/289533379 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f890c010460 con 0x7f891c105bf0 2024-04-23T02:37:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.252+0000 7f8921869700 1 -- 172.21.15.71:0/289533379 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891c1968a0 con 0x7f891c105bf0 2024-04-23T02:37:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.252+0000 7f8920867700 1 -- 172.21.15.71:0/289533379 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f890c009ea0 con 0x7f891c105bf0 2024-04-23T02:37:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.252+0000 7f8921869700 1 -- 172.21.15.71:0/289533379 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891c196df0 con 0x7f891c105bf0 2024-04-23T02:37:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.253+0000 7f8921869700 1 -- 172.21.15.71:0/289533379 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891c04f2e0 con 0x7f891c105bf0 2024-04-23T02:37:52.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.253+0000 7f8920867700 1 -- 172.21.15.71:0/289533379 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f890c0105d0 con 0x7f891c105bf0 2024-04-23T02:37:52.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.254+0000 7f8920867700 1 --2- 172.21.15.71:0/289533379 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f890406cce0 0x7f890406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:37:52.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.254+0000 7f8920867700 1 -- 172.21.15.71:0/289533379 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f890c08c3a0 con 0x7f891c105bf0 2024-04-23T02:37:52.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.254+0000 7f891affd700 1 --2- 172.21.15.71:0/289533379 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f890406cce0 0x7f890406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:37:52.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.254+0000 7f891affd700 1 --2- 172.21.15.71:0/289533379 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f890406cce0 0x7f890406f1a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8910006010 tx=0x7f891001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:37:52.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.258+0000 7f8920867700 1 -- 172.21.15.71:0/289533379 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f890c059f20 con 0x7f891c105bf0 2024-04-23T02:37:52.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:52 smithi192 ceph-mon[108617]: from='client.35158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:37:52.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:52 smithi192 ceph-mon[108617]: from='client.35162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:37:52.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:52 smithi192 ceph-mon[108617]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-04-23T02:37:52.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:52 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1920398956' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:37:52.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:52 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2625451328' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:37:52.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:52 smithi071 ceph-mon[144544]: from='client.35158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:37:52.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:52 smithi071 ceph-mon[144544]: from='client.35162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:37:52.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:52 smithi071 ceph-mon[144544]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-04-23T02:37:52.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:52 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1920398956' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:37:52.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:52 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2625451328' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:37:52.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.532+0000 7f8921869700 1 -- 172.21.15.71:0/289533379 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f891c1971d0 con 0x7f891c105bf0 2024-04-23T02:37:52.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.532+0000 7f8920867700 1 -- 172.21.15.71:0/289533379 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7f890c010880 con 0x7f891c105bf0 2024-04-23T02:37:52.535 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:37:52.535 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:37:52.535 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:37:52.535 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:37:52.535 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:37:52.535 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:37:52.536 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:37:52.536 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:37:52.536 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:37:52.536 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:37:52.536 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:37:52.536 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:37:52.536 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:37:52.536 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:37:52.536 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:37:52.536 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:37:52.536 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:37:52.536 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:37:52.536 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:37:52.536 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:37:52.536 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:37:52.537 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:37:52.537 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:37:52.537 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:37:52.537 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:37:52.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.534+0000 7f8921869700 1 -- 172.21.15.71:0/289533379 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f890406cce0 msgr2=0x7f890406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:52.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.534+0000 7f8921869700 1 --2- 172.21.15.71:0/289533379 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f890406cce0 0x7f890406f1a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8910006010 tx=0x7f891001a040 comp rx=0 tx=0).stop 2024-04-23T02:37:52.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.534+0000 7f8921869700 1 -- 172.21.15.71:0/289533379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891c105bf0 msgr2=0x7f891c19ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:37:52.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.534+0000 7f8921869700 1 --2- 172.21.15.71:0/289533379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891c105bf0 0x7f891c19ca30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f890c00d8d0 tx=0x7f890c00dbe0 comp rx=0 tx=0).stop 2024-04-23T02:37:52.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.535+0000 7f8921869700 1 -- 172.21.15.71:0/289533379 shutdown_connections 2024-04-23T02:37:52.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.535+0000 7f8921869700 1 --2- 172.21.15.71:0/289533379 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f890406cce0 0x7f890406f1a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:52.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.535+0000 7f8921869700 1 --2- 172.21.15.71:0/289533379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f891c0ffb40 0x7f891c19c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:52.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.535+0000 7f8921869700 1 --2- 172.21.15.71:0/289533379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891c105bf0 0x7f891c19ca30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:37:52.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.535+0000 7f8921869700 1 -- 172.21.15.71:0/289533379 >> 172.21.15.71:0/289533379 conn(0x7f891c0fb650 msgr2=0x7f891c069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:37:52.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.535+0000 7f8921869700 1 -- 172.21.15.71:0/289533379 shutdown_connections 2024-04-23T02:37:52.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:37:52.535+0000 7f8921869700 1 -- 172.21.15.71:0/289533379 wait complete. 2024-04-23T02:37:53.458 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:53 smithi192 ceph-mon[108617]: Upgrade: Updating mds.cephfs.smithi192.azdyze 2024-04-23T02:37:53.459 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:53 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:37:53.459 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:53 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.azdyze", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T02:37:53.459 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:53 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:37:53.459 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:53 smithi192 ceph-mon[108617]: Deploying daemon mds.cephfs.smithi192.azdyze on smithi192 2024-04-23T02:37:53.459 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:53 smithi192 ceph-mon[108617]: from='client.35174 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:37:53.459 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:53 smithi192 ceph-mon[108617]: pgmap v737: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:53.459 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:53 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/289533379' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:37:53.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:53 smithi071 ceph-mon[144544]: Upgrade: Updating mds.cephfs.smithi192.azdyze 2024-04-23T02:37:53.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:53 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:37:53.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:53 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi192.azdyze", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T02:37:53.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:53 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:37:53.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:53 smithi071 ceph-mon[144544]: Deploying daemon mds.cephfs.smithi192.azdyze on smithi192 2024-04-23T02:37:53.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:53 smithi071 ceph-mon[144544]: from='client.35174 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:37:53.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:53 smithi071 ceph-mon[144544]: pgmap v737: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:53.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:53 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/289533379' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:37:55.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:54 smithi071 ceph-mon[144544]: osdmap e84: 6 total, 6 up, 6 in 2024-04-23T02:37:55.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:54 smithi071 ceph-mon[144544]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-23T02:37:55.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:54 smithi071 ceph-mon[144544]: pgmap v739: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:55.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:54 smithi192 ceph-mon[108617]: osdmap e84: 6 total, 6 up, 6 in 2024-04-23T02:37:55.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:54 smithi192 ceph-mon[108617]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-23T02:37:55.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:54 smithi192 ceph-mon[108617]: pgmap v739: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:56.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:56 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.azdyze"}]: dispatch 2024-04-23T02:37:56.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:56 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.azdyze"}]: dispatch 2024-04-23T02:37:57.375 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:57 smithi192 ceph-mon[108617]: pgmap v740: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:57.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:57 smithi071 ceph-mon[144544]: pgmap v740: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:59.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:58 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:37:59.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:58 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:37:59.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:58 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:37:59.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:58 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:37:59.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:58 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:37:59.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:58 smithi192 ceph-mon[108617]: mds.? [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] up:boot 2024-04-23T02:37:59.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:58 smithi192 ceph-mon[108617]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-23T02:37:59.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:58 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.azdyze"}]: dispatch 2024-04-23T02:37:59.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:58 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:37:59.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:58 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:37:59.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:37:58 smithi192 ceph-mon[108617]: pgmap v741: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:37:59.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:58 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:37:59.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:58 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:37:59.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:58 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:37:59.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:58 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:37:59.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:58 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:37:59.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:58 smithi071 ceph-mon[144544]: mds.? [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] up:boot 2024-04-23T02:37:59.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:58 smithi071 ceph-mon[144544]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-23T02:37:59.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:58 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi192.azdyze"}]: dispatch 2024-04-23T02:37:59.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:58 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:37:59.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:58 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:37:59.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:37:58 smithi071 ceph-mon[144544]: pgmap v741: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:01.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:01 smithi071 ceph-mon[144544]: pgmap v742: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:01.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:01 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:38:01.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:01 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:38:01.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:01 smithi192 ceph-mon[108617]: pgmap v742: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:01.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:01 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:38:01.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:01 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:38:02.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:02 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:38:02.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:02 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:38:02.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:02 smithi071 ceph-mon[144544]: pgmap v743: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:03.068 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:02 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:38:03.068 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:02 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:38:03.068 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:02 smithi192 ceph-mon[108617]: pgmap v743: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:04.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:04 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:38:04.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:04 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:38:04.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:04 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:38:04.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:04 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:38:05.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:05 smithi071 ceph-mon[144544]: pgmap v744: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:05.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:38:05.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:38:05.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:05 smithi192 ceph-mon[108617]: pgmap v744: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:05.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:38:05.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:38:07.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:07 smithi071 ceph-mon[144544]: pgmap v745: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:07.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:07 smithi192 ceph-mon[108617]: pgmap v745: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:09.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:09 smithi071 ceph-mon[144544]: pgmap v746: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:09.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:09 smithi192 ceph-mon[108617]: pgmap v746: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:11.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:11 smithi071 ceph-mon[144544]: pgmap v747: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:11.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:11 smithi192 ceph-mon[108617]: pgmap v747: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:13.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:13 smithi071 ceph-mon[144544]: pgmap v748: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:13.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:13 smithi192 ceph-mon[108617]: pgmap v748: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:15.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:15 smithi071 ceph-mon[144544]: pgmap v749: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:15.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:15 smithi192 ceph-mon[108617]: pgmap v749: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:17.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:17 smithi071 ceph-mon[144544]: pgmap v750: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:17.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:17 smithi192 ceph-mon[108617]: pgmap v750: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:19.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:19 smithi071 ceph-mon[144544]: pgmap v751: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:19.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:19 smithi192 ceph-mon[108617]: pgmap v751: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:20.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:38:20.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:38:21.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:21 smithi192 ceph-mon[108617]: pgmap v752: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:21.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:21 smithi071 ceph-mon[144544]: pgmap v752: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:22.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.652+0000 7fd6d8cf5700 1 -- 172.21.15.71:0/3774303410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d40ffc20 msgr2=0x7fd6d4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:22.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.652+0000 7fd6d8cf5700 1 --2- 172.21.15.71:0/3774303410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d40ffc20 0x7fd6d4100080 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009910 tx=0x7fd6c8009c20 comp rx=0 tx=0).stop 2024-04-23T02:38:22.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.652+0000 7fd6d8cf5700 1 -- 172.21.15.71:0/3774303410 shutdown_connections 2024-04-23T02:38:22.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.652+0000 7fd6d8cf5700 1 --2- 172.21.15.71:0/3774303410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d40ffc20 0x7fd6d4100080 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:22.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.652+0000 7fd6d8cf5700 1 --2- 172.21.15.71:0/3774303410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6d4105bd0 0x7fd6d4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:22.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.652+0000 7fd6d8cf5700 1 -- 172.21.15.71:0/3774303410 >> 172.21.15.71:0/3774303410 conn(0x7fd6d4075670 msgr2=0x7fd6d4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:22.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.652+0000 7fd6d8cf5700 1 -- 172.21.15.71:0/3774303410 shutdown_connections 2024-04-23T02:38:22.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.652+0000 7fd6d8cf5700 1 -- 172.21.15.71:0/3774303410 wait complete. 2024-04-23T02:38:22.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.653+0000 7fd6d8cf5700 1 Processor -- start 2024-04-23T02:38:22.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.653+0000 7fd6d8cf5700 1 -- start start 2024-04-23T02:38:22.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.653+0000 7fd6d8cf5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6d40ffc20 0x7fd6d41954b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:22.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.653+0000 7fd6d8cf5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d4105bd0 0x7fd6d41959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:22.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.653+0000 7fd6d8cf5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d4195f30 con 0x7fd6d4105bd0 2024-04-23T02:38:22.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.653+0000 7fd6d8cf5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d41960a0 con 0x7fd6d40ffc20 2024-04-23T02:38:22.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.653+0000 7fd6d1d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d4105bd0 0x7fd6d41959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:22.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.653+0000 7fd6d1d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d4105bd0 0x7fd6d41959f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44184/0 (socket says 172.21.15.71:44184) 2024-04-23T02:38:22.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.653+0000 7fd6d1d9b700 1 -- 172.21.15.71:0/3953050884 learned_addr learned my addr 172.21.15.71:0/3953050884 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:38:22.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.653+0000 7fd6d259c700 1 --2- 172.21.15.71:0/3953050884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6d40ffc20 0x7fd6d41954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:22.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.654+0000 7fd6d1d9b700 1 -- 172.21.15.71:0/3953050884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6d40ffc20 msgr2=0x7fd6d41954b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:22.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.654+0000 7fd6d1d9b700 1 --2- 172.21.15.71:0/3953050884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6d40ffc20 0x7fd6d41954b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:22.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.654+0000 7fd6d1d9b700 1 -- 172.21.15.71:0/3953050884 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c80095f0 con 0x7fd6d4105bd0 2024-04-23T02:38:22.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.654+0000 7fd6d259c700 1 --2- 172.21.15.71:0/3953050884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6d40ffc20 0x7fd6d41954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:38:22.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.654+0000 7fd6d1d9b700 1 --2- 172.21.15.71:0/3953050884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d4105bd0 0x7fd6d41959f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8006010 tx=0x7fd6c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:22.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.654+0000 7fd6c37fe700 1 -- 172.21.15.71:0/3953050884 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6c801d070 con 0x7fd6d4105bd0 2024-04-23T02:38:22.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.654+0000 7fd6c37fe700 1 -- 172.21.15.71:0/3953050884 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd6c800f460 con 0x7fd6d4105bd0 2024-04-23T02:38:22.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.654+0000 7fd6d8cf5700 1 -- 172.21.15.71:0/3953050884 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d4196320 con 0x7fd6d4105bd0 2024-04-23T02:38:22.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.654+0000 7fd6c37fe700 1 -- 172.21.15.71:0/3953050884 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6c8021620 con 0x7fd6d4105bd0 2024-04-23T02:38:22.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.654+0000 7fd6d8cf5700 1 -- 172.21.15.71:0/3953050884 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d41a2da0 con 0x7fd6d4105bd0 2024-04-23T02:38:22.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.655+0000 7fd6d8cf5700 1 -- 172.21.15.71:0/3953050884 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d41080d0 con 0x7fd6d4105bd0 2024-04-23T02:38:22.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.656+0000 7fd6c37fe700 1 -- 172.21.15.71:0/3953050884 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd6c800fa40 con 0x7fd6d4105bd0 2024-04-23T02:38:22.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.656+0000 7fd6c37fe700 1 --2- 172.21.15.71:0/3953050884 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6bc06cce0 0x7fd6bc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:22.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.657+0000 7fd6c37fe700 1 -- 172.21.15.71:0/3953050884 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd6c808db60 con 0x7fd6d4105bd0 2024-04-23T02:38:22.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.657+0000 7fd6d259c700 1 --2- 172.21.15.71:0/3953050884 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6bc06cce0 0x7fd6bc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:22.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.657+0000 7fd6d259c700 1 --2- 172.21.15.71:0/3953050884 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6bc06cce0 0x7fd6bc06f1a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4005d90 tx=0x7fd6c4005d00 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:22.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.661+0000 7fd6c37fe700 1 -- 172.21.15.71:0/3953050884 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd6c805b7e0 con 0x7fd6d4105bd0 2024-04-23T02:38:22.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.875+0000 7fd6d8cf5700 1 -- 172.21.15.71:0/3953050884 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6d4196320 con 0x7fd6bc06cce0 2024-04-23T02:38:22.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.876+0000 7fd6c37fe700 1 -- 172.21.15.71:0/3953050884 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd6d4196320 con 0x7fd6bc06cce0 2024-04-23T02:38:22.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.878+0000 7fd6d8cf5700 1 -- 172.21.15.71:0/3953050884 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6bc06cce0 msgr2=0x7fd6bc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:22.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.878+0000 7fd6d8cf5700 1 --2- 172.21.15.71:0/3953050884 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6bc06cce0 0x7fd6bc06f1a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4005d90 tx=0x7fd6c4005d00 comp rx=0 tx=0).stop 2024-04-23T02:38:22.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.878+0000 7fd6d8cf5700 1 -- 172.21.15.71:0/3953050884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d4105bd0 msgr2=0x7fd6d41959f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:22.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.878+0000 7fd6d8cf5700 1 --2- 172.21.15.71:0/3953050884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d4105bd0 0x7fd6d41959f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8006010 tx=0x7fd6c800be20 comp rx=0 tx=0).stop 2024-04-23T02:38:22.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.878+0000 7fd6d8cf5700 1 -- 172.21.15.71:0/3953050884 shutdown_connections 2024-04-23T02:38:22.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.878+0000 7fd6d8cf5700 1 --2- 172.21.15.71:0/3953050884 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6bc06cce0 0x7fd6bc06f1a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:22.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.878+0000 7fd6d8cf5700 1 --2- 172.21.15.71:0/3953050884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6d40ffc20 0x7fd6d41954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:22.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.878+0000 7fd6d8cf5700 1 --2- 172.21.15.71:0/3953050884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6d4105bd0 0x7fd6d41959f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:22.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.878+0000 7fd6d8cf5700 1 -- 172.21.15.71:0/3953050884 >> 172.21.15.71:0/3953050884 conn(0x7fd6d4075670 msgr2=0x7fd6d40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:22.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.878+0000 7fd6d8cf5700 1 -- 172.21.15.71:0/3953050884 shutdown_connections 2024-04-23T02:38:22.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.878+0000 7fd6d8cf5700 1 -- 172.21.15.71:0/3953050884 wait complete. 2024-04-23T02:38:22.891 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:38:22.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.982+0000 7fc017444700 1 -- 172.21.15.71:0/259095946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc010068580 msgr2=0x7fc0100689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:22.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.982+0000 7fc017444700 1 --2- 172.21.15.71:0/259095946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc010068580 0x7fc0100689e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc000009910 tx=0x7fc000009c20 comp rx=0 tx=0).stop 2024-04-23T02:38:22.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.982+0000 7fc017444700 1 -- 172.21.15.71:0/259095946 shutdown_connections 2024-04-23T02:38:22.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.982+0000 7fc017444700 1 --2- 172.21.15.71:0/259095946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc010068580 0x7fc0100689e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:22.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.982+0000 7fc017444700 1 --2- 172.21.15.71:0/259095946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc010106cd0 0x7fc0101070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:22.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.982+0000 7fc017444700 1 -- 172.21.15.71:0/259095946 >> 172.21.15.71:0/259095946 conn(0x7fc010075670 msgr2=0x7fc010075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:22.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.982+0000 7fc017444700 1 -- 172.21.15.71:0/259095946 shutdown_connections 2024-04-23T02:38:22.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.982+0000 7fc017444700 1 -- 172.21.15.71:0/259095946 wait complete. 2024-04-23T02:38:22.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.983+0000 7fc017444700 1 Processor -- start 2024-04-23T02:38:22.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.983+0000 7fc017444700 1 -- start start 2024-04-23T02:38:22.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.983+0000 7fc017444700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc010068580 0x7fc0100ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:22.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.983+0000 7fc017444700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc010106cd0 0x7fc010100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:22.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.983+0000 7fc017444700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc010100770 con 0x7fc010068580 2024-04-23T02:38:22.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.983+0000 7fc017444700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0101008b0 con 0x7fc010106cd0 2024-04-23T02:38:22.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.983+0000 7fc0151e0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc010068580 0x7fc0100ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:22.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.983+0000 7fc0151e0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc010068580 0x7fc0100ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44200/0 (socket says 172.21.15.71:44200) 2024-04-23T02:38:22.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.983+0000 7fc0151e0700 1 -- 172.21.15.71:0/2626640937 learned_addr learned my addr 172.21.15.71:0/2626640937 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:38:22.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.983+0000 7fc0149df700 1 --2- 172.21.15.71:0/2626640937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc010106cd0 0x7fc010100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:22.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.983+0000 7fc0151e0700 1 -- 172.21.15.71:0/2626640937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc010106cd0 msgr2=0x7fc010100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:22.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.983+0000 7fc0151e0700 1 --2- 172.21.15.71:0/2626640937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc010106cd0 0x7fc010100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:22.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.984+0000 7fc0151e0700 1 -- 172.21.15.71:0/2626640937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0000095f0 con 0x7fc010068580 2024-04-23T02:38:22.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.984+0000 7fc0149df700 1 --2- 172.21.15.71:0/2626640937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc010106cd0 0x7fc010100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:38:22.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.984+0000 7fc0151e0700 1 --2- 172.21.15.71:0/2626640937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc010068580 0x7fc0100ffcf0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00c8a0 tx=0x7fc00c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:22.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.984+0000 7fc0067fc700 1 -- 172.21.15.71:0/2626640937 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc00c00cea0 con 0x7fc010068580 2024-04-23T02:38:22.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.984+0000 7fc0067fc700 1 -- 172.21.15.71:0/2626640937 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc00c004830 con 0x7fc010068580 2024-04-23T02:38:22.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.984+0000 7fc017444700 1 -- 172.21.15.71:0/2626640937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0101a2940 con 0x7fc010068580 2024-04-23T02:38:22.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.984+0000 7fc0067fc700 1 -- 172.21.15.71:0/2626640937 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc00c005500 con 0x7fc010068580 2024-04-23T02:38:22.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.984+0000 7fc017444700 1 -- 172.21.15.71:0/2626640937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0101a2d50 con 0x7fc010068580 2024-04-23T02:38:22.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.985+0000 7fc017444700 1 -- 172.21.15.71:0/2626640937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0101091d0 con 0x7fc010068580 2024-04-23T02:38:22.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.985+0000 7fc0067fc700 1 -- 172.21.15.71:0/2626640937 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc00c0049a0 con 0x7fc010068580 2024-04-23T02:38:22.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.986+0000 7fc0067fc700 1 --2- 172.21.15.71:0/2626640937 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbffc06cce0 0x7fbffc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:22.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.986+0000 7fc0149df700 1 --2- 172.21.15.71:0/2626640937 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbffc06cce0 0x7fbffc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:22.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.986+0000 7fc0067fc700 1 -- 172.21.15.71:0/2626640937 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc00c08c1d0 con 0x7fc010068580 2024-04-23T02:38:22.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.986+0000 7fc0149df700 1 --2- 172.21.15.71:0/2626640937 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbffc06cce0 0x7fbffc06f1a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc0101014e0 tx=0x7fc00001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:22.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:22.990+0000 7fc0067fc700 1 -- 172.21.15.71:0/2626640937 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc00c0562c0 con 0x7fc010068580 2024-04-23T02:38:23.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.199+0000 7fc017444700 1 -- 172.21.15.71:0/2626640937 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0100611d0 con 0x7fbffc06cce0 2024-04-23T02:38:23.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.200+0000 7fc0067fc700 1 -- 172.21.15.71:0/2626640937 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc0100611d0 con 0x7fbffc06cce0 2024-04-23T02:38:23.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.202+0000 7fc017444700 1 -- 172.21.15.71:0/2626640937 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbffc06cce0 msgr2=0x7fbffc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:23.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.202+0000 7fc017444700 1 --2- 172.21.15.71:0/2626640937 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbffc06cce0 0x7fbffc06f1a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc0101014e0 tx=0x7fc00001a040 comp rx=0 tx=0).stop 2024-04-23T02:38:23.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.202+0000 7fc017444700 1 -- 172.21.15.71:0/2626640937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc010068580 msgr2=0x7fc0100ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:23.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.202+0000 7fc017444700 1 --2- 172.21.15.71:0/2626640937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc010068580 0x7fc0100ffcf0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00c8a0 tx=0x7fc00c00cc60 comp rx=0 tx=0).stop 2024-04-23T02:38:23.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.202+0000 7fc017444700 1 -- 172.21.15.71:0/2626640937 shutdown_connections 2024-04-23T02:38:23.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.202+0000 7fc017444700 1 --2- 172.21.15.71:0/2626640937 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbffc06cce0 0x7fbffc06f1a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:23.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.202+0000 7fc017444700 1 --2- 172.21.15.71:0/2626640937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc010068580 0x7fc0100ffcf0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:23.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.202+0000 7fc017444700 1 --2- 172.21.15.71:0/2626640937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc010106cd0 0x7fc010100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:23.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.202+0000 7fc017444700 1 -- 172.21.15.71:0/2626640937 >> 172.21.15.71:0/2626640937 conn(0x7fc010075670 msgr2=0x7fc0100fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:23.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.202+0000 7fc017444700 1 -- 172.21.15.71:0/2626640937 shutdown_connections 2024-04-23T02:38:23.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.202+0000 7fc017444700 1 -- 172.21.15.71:0/2626640937 wait complete. 2024-04-23T02:38:23.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.305+0000 7f70f5458700 1 -- 172.21.15.71:0/2155490420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f0101840 msgr2=0x7f70f0105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:23.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.305+0000 7f70f5458700 1 --2- 172.21.15.71:0/2155490420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f0101840 0x7f70f0105890 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f70e0009910 tx=0x7f70e0009c20 comp rx=0 tx=0).stop 2024-04-23T02:38:23.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.305+0000 7f70f5458700 1 -- 172.21.15.71:0/2155490420 shutdown_connections 2024-04-23T02:38:23.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.305+0000 7f70f5458700 1 --2- 172.21.15.71:0/2155490420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f0101840 0x7f70f0105890 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:23.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.305+0000 7f70f5458700 1 --2- 172.21.15.71:0/2155490420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70f0100e90 0x7f70f0101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:23.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.305+0000 7f70f5458700 1 -- 172.21.15.71:0/2155490420 >> 172.21.15.71:0/2155490420 conn(0x7f70f00fc740 msgr2=0x7f70f00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:23.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.305+0000 7f70f5458700 1 -- 172.21.15.71:0/2155490420 shutdown_connections 2024-04-23T02:38:23.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.305+0000 7f70f5458700 1 -- 172.21.15.71:0/2155490420 wait complete. 2024-04-23T02:38:23.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.305+0000 7f70f5458700 1 Processor -- start 2024-04-23T02:38:23.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.305+0000 7f70f5458700 1 -- start start 2024-04-23T02:38:23.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.306+0000 7f70f5458700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f0100e90 0x7f70f019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:23.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.306+0000 7f70f5458700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70f0101840 0x7f70f019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:23.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.306+0000 7f70f5458700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f019d220 con 0x7f70f0100e90 2024-04-23T02:38:23.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.306+0000 7f70f5458700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f01966d0 con 0x7f70f0101840 2024-04-23T02:38:23.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.306+0000 7f70eeffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f0100e90 0x7f70f019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:23.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.306+0000 7f70eeffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f0100e90 0x7f70f019c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44230/0 (socket says 172.21.15.71:44230) 2024-04-23T02:38:23.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.306+0000 7f70eeffd700 1 -- 172.21.15.71:0/2245132298 learned_addr learned my addr 172.21.15.71:0/2245132298 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:38:23.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.306+0000 7f70ee7fc700 1 --2- 172.21.15.71:0/2245132298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70f0101840 0x7f70f019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:23.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.306+0000 7f70eeffd700 1 -- 172.21.15.71:0/2245132298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70f0101840 msgr2=0x7f70f019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:23.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.306+0000 7f70eeffd700 1 --2- 172.21.15.71:0/2245132298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70f0101840 0x7f70f019cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:23.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.306+0000 7f70eeffd700 1 -- 172.21.15.71:0/2245132298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e00095f0 con 0x7f70f0100e90 2024-04-23T02:38:23.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.306+0000 7f70eeffd700 1 --2- 172.21.15.71:0/2245132298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f0100e90 0x7f70f019c650 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f70e400d900 tx=0x7f70e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:23.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.306+0000 7f70d7fff700 1 -- 172.21.15.71:0/2245132298 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70e4004a00 con 0x7f70f0100e90 2024-04-23T02:38:23.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.306+0000 7f70f5458700 1 -- 172.21.15.71:0/2245132298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f01969b0 con 0x7f70f0100e90 2024-04-23T02:38:23.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.306+0000 7f70d7fff700 1 -- 172.21.15.71:0/2245132298 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f70e4005020 con 0x7f70f0100e90 2024-04-23T02:38:23.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.306+0000 7f70d7fff700 1 -- 172.21.15.71:0/2245132298 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70e4009c00 con 0x7f70f0100e90 2024-04-23T02:38:23.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.306+0000 7f70f5458700 1 -- 172.21.15.71:0/2245132298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f0196f00 con 0x7f70f0100e90 2024-04-23T02:38:23.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.307+0000 7f70f5458700 1 -- 172.21.15.71:0/2245132298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f0109290 con 0x7f70f0100e90 2024-04-23T02:38:23.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.308+0000 7f70d7fff700 1 -- 172.21.15.71:0/2245132298 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f70e4005190 con 0x7f70f0100e90 2024-04-23T02:38:23.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.308+0000 7f70d7fff700 1 --2- 172.21.15.71:0/2245132298 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f70d806cce0 0x7f70d806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:23.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.308+0000 7f70ee7fc700 1 --2- 172.21.15.71:0/2245132298 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f70d806cce0 0x7f70d806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:23.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.308+0000 7f70d7fff700 1 -- 172.21.15.71:0/2245132298 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f70e408c770 con 0x7f70f0100e90 2024-04-23T02:38:23.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.309+0000 7f70ee7fc700 1 --2- 172.21.15.71:0/2245132298 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f70d806cce0 0x7f70d806f1a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f70e000b5c0 tx=0x7f70e0005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:23.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.313+0000 7f70d7fff700 1 -- 172.21.15.71:0/2245132298 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f70e405a270 con 0x7f70f0100e90 2024-04-23T02:38:23.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.513+0000 7f70f5458700 1 -- 172.21.15.71:0/2245132298 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f70f0197cf0 con 0x7f70d806cce0 2024-04-23T02:38:23.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.524+0000 7f70d7fff700 1 -- 172.21.15.71:0/2245132298 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f70f0197cf0 con 0x7f70d806cce0 2024-04-23T02:38:23.526 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:38:23.526 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (22m) 20s ago 33m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:38:23.526 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (33m) 20s ago 33m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:38:23.526 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (32m) 22s ago 32m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:38:23.526 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (19m) 20s ago 33m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:38:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (19m) 22s ago 32m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:38:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (21m) 20s ago 33m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:38:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (16m) 20s ago 27m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:38:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (15m) 20s ago 27m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:38:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (26s) 22s ago 27m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:38:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (15m) 22s ago 27m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:38:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (24m) 20s ago 35m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:38:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (23m) 22s ago 32m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:38:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (20m) 20s ago 35m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:38:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (20m) 22s ago 32m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:38:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (23m) 20s ago 33m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:38:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (22m) 22s ago 32m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:38:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (19m) 20s ago 31m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:38:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (18m) 20s ago 30m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:38:23.528 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (18m) 20s ago 30m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:38:23.528 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (17m) 22s ago 29m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:38:23.528 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (17m) 22s ago 29m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:38:23.528 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (16m) 22s ago 29m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:38:23.528 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (22m) 20s ago 32m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:38:23.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.526+0000 7f70f5458700 1 -- 172.21.15.71:0/2245132298 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f70d806cce0 msgr2=0x7f70d806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:23.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.526+0000 7f70f5458700 1 --2- 172.21.15.71:0/2245132298 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f70d806cce0 0x7f70d806f1a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f70e000b5c0 tx=0x7f70e0005fd0 comp rx=0 tx=0).stop 2024-04-23T02:38:23.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.526+0000 7f70f5458700 1 -- 172.21.15.71:0/2245132298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f0100e90 msgr2=0x7f70f019c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:23.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.526+0000 7f70f5458700 1 --2- 172.21.15.71:0/2245132298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f0100e90 0x7f70f019c650 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f70e400d900 tx=0x7f70e400dcc0 comp rx=0 tx=0).stop 2024-04-23T02:38:23.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.526+0000 7f70f5458700 1 -- 172.21.15.71:0/2245132298 shutdown_connections 2024-04-23T02:38:23.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.526+0000 7f70f5458700 1 --2- 172.21.15.71:0/2245132298 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f70d806cce0 0x7f70d806f1a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:23.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.526+0000 7f70f5458700 1 --2- 172.21.15.71:0/2245132298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70f0100e90 0x7f70f019c650 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:23.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.526+0000 7f70f5458700 1 --2- 172.21.15.71:0/2245132298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70f0101840 0x7f70f019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:23.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.526+0000 7f70f5458700 1 -- 172.21.15.71:0/2245132298 >> 172.21.15.71:0/2245132298 conn(0x7f70f00fc740 msgr2=0x7f70f00fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:23.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.526+0000 7f70f5458700 1 -- 172.21.15.71:0/2245132298 shutdown_connections 2024-04-23T02:38:23.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.526+0000 7f70f5458700 1 -- 172.21.15.71:0/2245132298 wait complete. 2024-04-23T02:38:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.627+0000 7fd99f484700 1 -- 172.21.15.71:0/58663661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9980ffc20 msgr2=0x7fd998100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.627+0000 7fd99f484700 1 --2- 172.21.15.71:0/58663661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9980ffc20 0x7fd998100080 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd994009910 tx=0x7fd994009c20 comp rx=0 tx=0).stop 2024-04-23T02:38:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.627+0000 7fd99f484700 1 -- 172.21.15.71:0/58663661 shutdown_connections 2024-04-23T02:38:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.627+0000 7fd99f484700 1 --2- 172.21.15.71:0/58663661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9980ffc20 0x7fd998100080 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.627+0000 7fd99f484700 1 --2- 172.21.15.71:0/58663661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd998105bd0 0x7fd998105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.627+0000 7fd99f484700 1 -- 172.21.15.71:0/58663661 >> 172.21.15.71:0/58663661 conn(0x7fd998075670 msgr2=0x7fd998075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:23.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.628+0000 7fd99f484700 1 -- 172.21.15.71:0/58663661 shutdown_connections 2024-04-23T02:38:23.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.628+0000 7fd99f484700 1 -- 172.21.15.71:0/58663661 wait complete. 2024-04-23T02:38:23.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.628+0000 7fd99f484700 1 Processor -- start 2024-04-23T02:38:23.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.628+0000 7fd99f484700 1 -- start start 2024-04-23T02:38:23.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.628+0000 7fd99f484700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9980ffc20 0x7fd998197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:23.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.628+0000 7fd99f484700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd998105bd0 0x7fd998197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:23.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.628+0000 7fd99f484700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9981982c0 con 0x7fd9980ffc20 2024-04-23T02:38:23.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.628+0000 7fd99f484700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd998198400 con 0x7fd998105bd0 2024-04-23T02:38:23.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.628+0000 7fd99d220700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9980ffc20 0x7fd998197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:23.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.628+0000 7fd99d220700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9980ffc20 0x7fd998197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44252/0 (socket says 172.21.15.71:44252) 2024-04-23T02:38:23.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.628+0000 7fd99d220700 1 -- 172.21.15.71:0/4208712673 learned_addr learned my addr 172.21.15.71:0/4208712673 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:38:23.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.629+0000 7fd99d220700 1 -- 172.21.15.71:0/4208712673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd998105bd0 msgr2=0x7fd998197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:23.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.629+0000 7fd99d220700 1 --2- 172.21.15.71:0/4208712673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd998105bd0 0x7fd998197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:23.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.629+0000 7fd99d220700 1 -- 172.21.15.71:0/4208712673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9940095f0 con 0x7fd9980ffc20 2024-04-23T02:38:23.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.629+0000 7fd99d220700 1 --2- 172.21.15.71:0/4208712673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9980ffc20 0x7fd998197760 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd98800b4b0 tx=0x7fd98800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:23.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.629+0000 7fd98e7fc700 1 -- 172.21.15.71:0/4208712673 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd988011800 con 0x7fd9980ffc20 2024-04-23T02:38:23.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.629+0000 7fd98e7fc700 1 -- 172.21.15.71:0/4208712673 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd988011e00 con 0x7fd9980ffc20 2024-04-23T02:38:23.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.629+0000 7fd99f484700 1 -- 172.21.15.71:0/4208712673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9981a07d0 con 0x7fd9980ffc20 2024-04-23T02:38:23.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.629+0000 7fd98e7fc700 1 -- 172.21.15.71:0/4208712673 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd988010e50 con 0x7fd9980ffc20 2024-04-23T02:38:23.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.629+0000 7fd99f484700 1 -- 172.21.15.71:0/4208712673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9981a0c90 con 0x7fd9980ffc20 2024-04-23T02:38:23.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.630+0000 7fd99f484700 1 -- 172.21.15.71:0/4208712673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9981080d0 con 0x7fd9980ffc20 2024-04-23T02:38:23.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.630+0000 7fd98e7fc700 1 -- 172.21.15.71:0/4208712673 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd988011960 con 0x7fd9980ffc20 2024-04-23T02:38:23.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.631+0000 7fd98e7fc700 1 --2- 172.21.15.71:0/4208712673 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd98406cce0 0x7fd98406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:23.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.631+0000 7fd98e7fc700 1 -- 172.21.15.71:0/4208712673 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd98808c0d0 con 0x7fd9980ffc20 2024-04-23T02:38:23.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.631+0000 7fd99ca1f700 1 --2- 172.21.15.71:0/4208712673 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd98406cce0 0x7fd98406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:23.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.631+0000 7fd99ca1f700 1 --2- 172.21.15.71:0/4208712673 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd98406cce0 0x7fd98406f1a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd99400b5c0 tx=0x7fd994005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:23.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.635+0000 7fd98e7fc700 1 -- 172.21.15.71:0/4208712673 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd9880561c0 con 0x7fd9980ffc20 2024-04-23T02:38:23.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:23 smithi192 ceph-mon[108617]: pgmap v753: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:23.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.907+0000 7fd99f484700 1 -- 172.21.15.71:0/4208712673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd99804f2e0 con 0x7fd9980ffc20 2024-04-23T02:38:23.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.908+0000 7fd98e7fc700 1 -- 172.21.15.71:0/4208712673 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fd9880597e0 con 0x7fd9980ffc20 2024-04-23T02:38:23.910 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:38:23.911 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:38:23.911 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:38:23.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:38:23.911 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:38:23.911 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:38:23.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:38:23.911 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:38:23.911 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:38:23.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:38:23.911 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:38:23.911 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:38:23.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:38:23.911 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:38:23.912 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:38:23.912 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:38:23.912 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:38:23.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.910+0000 7fd99f484700 1 -- 172.21.15.71:0/4208712673 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd98406cce0 msgr2=0x7fd98406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:23.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.910+0000 7fd99f484700 1 --2- 172.21.15.71:0/4208712673 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd98406cce0 0x7fd98406f1a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd99400b5c0 tx=0x7fd994005960 comp rx=0 tx=0).stop 2024-04-23T02:38:23.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.910+0000 7fd99f484700 1 -- 172.21.15.71:0/4208712673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9980ffc20 msgr2=0x7fd998197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:23.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.910+0000 7fd99f484700 1 --2- 172.21.15.71:0/4208712673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9980ffc20 0x7fd998197760 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd98800b4b0 tx=0x7fd98800b870 comp rx=0 tx=0).stop 2024-04-23T02:38:23.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.910+0000 7fd99f484700 1 -- 172.21.15.71:0/4208712673 shutdown_connections 2024-04-23T02:38:23.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.910+0000 7fd99f484700 1 --2- 172.21.15.71:0/4208712673 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd98406cce0 0x7fd98406f1a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:23.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.910+0000 7fd99f484700 1 --2- 172.21.15.71:0/4208712673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9980ffc20 0x7fd998197760 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:23.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.910+0000 7fd99f484700 1 --2- 172.21.15.71:0/4208712673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd998105bd0 0x7fd998197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:23.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.910+0000 7fd99f484700 1 -- 172.21.15.71:0/4208712673 >> 172.21.15.71:0/4208712673 conn(0x7fd998075670 msgr2=0x7fd9980fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:23.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.910+0000 7fd99f484700 1 -- 172.21.15.71:0/4208712673 shutdown_connections 2024-04-23T02:38:23.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:23.910+0000 7fd99f484700 1 -- 172.21.15.71:0/4208712673 wait complete. 2024-04-23T02:38:23.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:23 smithi071 ceph-mon[144544]: pgmap v753: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:24.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.012+0000 7fee4037f700 1 -- 172.21.15.71:0/3274362504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee38101e80 msgr2=0x7fee38102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:24.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.012+0000 7fee4037f700 1 --2- 172.21.15.71:0/3274362504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee38101e80 0x7fee38102260 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fee34009910 tx=0x7fee34009c20 comp rx=0 tx=0).stop 2024-04-23T02:38:24.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.013+0000 7fee4037f700 1 -- 172.21.15.71:0/3274362504 shutdown_connections 2024-04-23T02:38:24.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.013+0000 7fee4037f700 1 --2- 172.21.15.71:0/3274362504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee381027a0 0x7fee3810ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.013+0000 7fee4037f700 1 --2- 172.21.15.71:0/3274362504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee38101e80 0x7fee38102260 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.013+0000 7fee4037f700 1 -- 172.21.15.71:0/3274362504 >> 172.21.15.71:0/3274362504 conn(0x7fee380fb610 msgr2=0x7fee380fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:24.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.013+0000 7fee4037f700 1 -- 172.21.15.71:0/3274362504 shutdown_connections 2024-04-23T02:38:24.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.013+0000 7fee4037f700 1 -- 172.21.15.71:0/3274362504 wait complete. 2024-04-23T02:38:24.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.013+0000 7fee4037f700 1 Processor -- start 2024-04-23T02:38:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.013+0000 7fee4037f700 1 -- start start 2024-04-23T02:38:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.013+0000 7fee4037f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee38101e80 0x7fee381989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.013+0000 7fee4037f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee381027a0 0x7fee38198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.013+0000 7fee4037f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee38199570 con 0x7fee38101e80 2024-04-23T02:38:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.013+0000 7fee4037f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3819d380 con 0x7fee381027a0 2024-04-23T02:38:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.014+0000 7fee3e11b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee38101e80 0x7fee381989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.014+0000 7fee3e11b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee38101e80 0x7fee381989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44264/0 (socket says 172.21.15.71:44264) 2024-04-23T02:38:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.014+0000 7fee3e11b700 1 -- 172.21.15.71:0/3611279967 learned_addr learned my addr 172.21.15.71:0/3611279967 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:38:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.014+0000 7fee3d91a700 1 --2- 172.21.15.71:0/3611279967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee381027a0 0x7fee38198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.014+0000 7fee3e11b700 1 -- 172.21.15.71:0/3611279967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee381027a0 msgr2=0x7fee38198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.014+0000 7fee3e11b700 1 --2- 172.21.15.71:0/3611279967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee381027a0 0x7fee38198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.014+0000 7fee3e11b700 1 -- 172.21.15.71:0/3611279967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee340095f0 con 0x7fee38101e80 2024-04-23T02:38:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.014+0000 7fee3e11b700 1 --2- 172.21.15.71:0/3611279967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee38101e80 0x7fee381989a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fee34006010 tx=0x7fee3400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:24.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.014+0000 7fee2f7fe700 1 -- 172.21.15.71:0/3611279967 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee3401d070 con 0x7fee38101e80 2024-04-23T02:38:24.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.014+0000 7fee2f7fe700 1 -- 172.21.15.71:0/3611279967 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fee3400f460 con 0x7fee38101e80 2024-04-23T02:38:24.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.014+0000 7fee4037f700 1 -- 172.21.15.71:0/3611279967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3819d600 con 0x7fee38101e80 2024-04-23T02:38:24.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.014+0000 7fee2f7fe700 1 -- 172.21.15.71:0/3611279967 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee34021640 con 0x7fee38101e80 2024-04-23T02:38:24.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.014+0000 7fee4037f700 1 -- 172.21.15.71:0/3611279967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3819daf0 con 0x7fee38101e80 2024-04-23T02:38:24.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.015+0000 7fee4037f700 1 -- 172.21.15.71:0/3611279967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3810a7f0 con 0x7fee38101e80 2024-04-23T02:38:24.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.016+0000 7fee2f7fe700 1 -- 172.21.15.71:0/3611279967 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fee3400f5d0 con 0x7fee38101e80 2024-04-23T02:38:24.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.016+0000 7fee2f7fe700 1 --2- 172.21.15.71:0/3611279967 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee2406cce0 0x7fee2406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:24.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.016+0000 7fee2f7fe700 1 -- 172.21.15.71:0/3611279967 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fee3408d7e0 con 0x7fee38101e80 2024-04-23T02:38:24.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.016+0000 7fee3d91a700 1 --2- 172.21.15.71:0/3611279967 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee2406cce0 0x7fee2406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:24.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.016+0000 7fee3d91a700 1 --2- 172.21.15.71:0/3611279967 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee2406cce0 0x7fee2406f1a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fee28005950 tx=0x7fee280058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:24.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.020+0000 7fee2f7fe700 1 -- 172.21.15.71:0/3611279967 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fee3405b2e0 con 0x7fee38101e80 2024-04-23T02:38:24.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.255+0000 7fee4037f700 1 -- 172.21.15.71:0/3611279967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fee38199da0 con 0x7fee38101e80 2024-04-23T02:38:24.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.255+0000 7fee2f7fe700 1 -- 172.21.15.71:0/3611279967 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7fee3400f880 con 0x7fee38101e80 2024-04-23T02:38:24.258 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:38:24.258 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:38:24.258 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:38:24.258 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:38:24.259 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:38:24.259 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:38:24.259 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:38:24.259 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:38:24.259 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:38:24.259 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:38:24.259 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:38:24.259 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:38:24.259 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:38:24.259 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:38:24.259 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:38:24.259 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:38:24.259 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:38:24.259 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:38:24.260 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:38:24.260 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:38:24.260 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:38:24.260 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:38:24.260 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:38:24.260 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:38:24.260 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:38:24.260 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:38:24.260 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:38:24.260 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:38:24.260 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:38:24.260 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:38:24.260 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:38:24.260 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:38:24.261 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:38:24.261 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:38:24.261 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:38:24.261 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:38:24.261 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:38:24.261 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:38:24.261 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:38:24.261 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:38:24.261 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:38:24.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.258+0000 7fee4037f700 1 -- 172.21.15.71:0/3611279967 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee2406cce0 msgr2=0x7fee2406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:24.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.258+0000 7fee4037f700 1 --2- 172.21.15.71:0/3611279967 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee2406cce0 0x7fee2406f1a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fee28005950 tx=0x7fee280058e0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.258+0000 7fee4037f700 1 -- 172.21.15.71:0/3611279967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee38101e80 msgr2=0x7fee381989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:24.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.258+0000 7fee4037f700 1 --2- 172.21.15.71:0/3611279967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee38101e80 0x7fee381989a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fee34006010 tx=0x7fee3400be20 comp rx=0 tx=0).stop 2024-04-23T02:38:24.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.258+0000 7fee4037f700 1 -- 172.21.15.71:0/3611279967 shutdown_connections 2024-04-23T02:38:24.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.258+0000 7fee4037f700 1 --2- 172.21.15.71:0/3611279967 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee2406cce0 0x7fee2406f1a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.258+0000 7fee4037f700 1 --2- 172.21.15.71:0/3611279967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee38101e80 0x7fee381989a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.258+0000 7fee4037f700 1 --2- 172.21.15.71:0/3611279967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee381027a0 0x7fee38198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.258+0000 7fee4037f700 1 -- 172.21.15.71:0/3611279967 >> 172.21.15.71:0/3611279967 conn(0x7fee380fb610 msgr2=0x7fee381002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:24.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.258+0000 7fee4037f700 1 -- 172.21.15.71:0/3611279967 shutdown_connections 2024-04-23T02:38:24.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.258+0000 7fee4037f700 1 -- 172.21.15.71:0/3611279967 wait complete. 2024-04-23T02:38:24.263 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:38:24.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.361+0000 7f1733d20700 1 -- 172.21.15.71:0/727736217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f172c105bd0 msgr2=0x7f172c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:24.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.361+0000 7f1733d20700 1 --2- 172.21.15.71:0/727736217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f172c105bd0 0x7f172c105fb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1728009910 tx=0x7f1728009c20 comp rx=0 tx=0).stop 2024-04-23T02:38:24.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.361+0000 7f1733d20700 1 -- 172.21.15.71:0/727736217 shutdown_connections 2024-04-23T02:38:24.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.361+0000 7f1733d20700 1 --2- 172.21.15.71:0/727736217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f172c0ffc20 0x7f172c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.361+0000 7f1733d20700 1 --2- 172.21.15.71:0/727736217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f172c105bd0 0x7f172c105fb0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.361+0000 7f1733d20700 1 -- 172.21.15.71:0/727736217 >> 172.21.15.71:0/727736217 conn(0x7f172c075670 msgr2=0x7f172c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:24.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.362+0000 7f1733d20700 1 -- 172.21.15.71:0/727736217 shutdown_connections 2024-04-23T02:38:24.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.362+0000 7f1733d20700 1 -- 172.21.15.71:0/727736217 wait complete. 2024-04-23T02:38:24.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.362+0000 7f1733d20700 1 Processor -- start 2024-04-23T02:38:24.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.362+0000 7f1733d20700 1 -- start start 2024-04-23T02:38:24.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.362+0000 7f1733d20700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f172c0ffc20 0x7f172c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:24.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.362+0000 7f1733d20700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f172c105bd0 0x7f172c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:24.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.362+0000 7f1733d20700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c198140 con 0x7f172c0ffc20 2024-04-23T02:38:24.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.362+0000 7f1733d20700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c1982b0 con 0x7f172c105bd0 2024-04-23T02:38:24.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.362+0000 7f1731abc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f172c0ffc20 0x7f172c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:24.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.363+0000 7f1731abc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f172c0ffc20 0x7f172c1975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44286/0 (socket says 172.21.15.71:44286) 2024-04-23T02:38:24.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.363+0000 7f1731abc700 1 -- 172.21.15.71:0/827210239 learned_addr learned my addr 172.21.15.71:0/827210239 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:38:24.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.363+0000 7f17312bb700 1 --2- 172.21.15.71:0/827210239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f172c105bd0 0x7f172c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:24.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.363+0000 7f1731abc700 1 -- 172.21.15.71:0/827210239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f172c105bd0 msgr2=0x7f172c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:24.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.363+0000 7f1731abc700 1 --2- 172.21.15.71:0/827210239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f172c105bd0 0x7f172c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.363+0000 7f1731abc700 1 -- 172.21.15.71:0/827210239 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17280095f0 con 0x7f172c0ffc20 2024-04-23T02:38:24.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.363+0000 7f1731abc700 1 --2- 172.21.15.71:0/827210239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f172c0ffc20 0x7f172c1975e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1728006010 tx=0x7f172800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:24.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.363+0000 7f1722ffd700 1 -- 172.21.15.71:0/827210239 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f172801d070 con 0x7f172c0ffc20 2024-04-23T02:38:24.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.363+0000 7f1722ffd700 1 -- 172.21.15.71:0/827210239 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1728005710 con 0x7f172c0ffc20 2024-04-23T02:38:24.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.363+0000 7f1722ffd700 1 -- 172.21.15.71:0/827210239 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f172800fe20 con 0x7f172c0ffc20 2024-04-23T02:38:24.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.363+0000 7f1733d20700 1 -- 172.21.15.71:0/827210239 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172c1a0650 con 0x7f172c0ffc20 2024-04-23T02:38:24.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.363+0000 7f1733d20700 1 -- 172.21.15.71:0/827210239 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172c1a0a60 con 0x7f172c0ffc20 2024-04-23T02:38:24.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.364+0000 7f1733d20700 1 -- 172.21.15.71:0/827210239 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172c1080f0 con 0x7f172c0ffc20 2024-04-23T02:38:24.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.365+0000 7f1722ffd700 1 -- 172.21.15.71:0/827210239 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1728003a60 con 0x7f172c0ffc20 2024-04-23T02:38:24.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.365+0000 7f1722ffd700 1 --2- 172.21.15.71:0/827210239 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f171806cce0 0x7f171806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:24.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.365+0000 7f1722ffd700 1 -- 172.21.15.71:0/827210239 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f172808d6f0 con 0x7f172c0ffc20 2024-04-23T02:38:24.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.365+0000 7f17312bb700 1 --2- 172.21.15.71:0/827210239 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f171806cce0 0x7f171806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:24.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.365+0000 7f17312bb700 1 --2- 172.21.15.71:0/827210239 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f171806cce0 0x7f171806f1a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f172c068e50 tx=0x7f171c00b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:24.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.370+0000 7f1722ffd700 1 -- 172.21.15.71:0/827210239 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1728057760 con 0x7f172c0ffc20 2024-04-23T02:38:24.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.581+0000 7f1733d20700 1 -- 172.21.15.71:0/827210239 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f172c0611d0 con 0x7f171806cce0 2024-04-23T02:38:24.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.583+0000 7f1722ffd700 1 -- 172.21.15.71:0/827210239 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f172c0611d0 con 0x7f171806cce0 2024-04-23T02:38:24.585 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:38:24.585 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:38:24.585 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:38:24.585 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:38:24.586 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:38:24.586 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:38:24.586 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:38:24.586 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:38:24.586 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:38:24.586 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:38:24.586 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:38:24.586 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:38:24.586 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:38:24.586 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:38:24.586 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:38:24.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.585+0000 7f1733d20700 1 -- 172.21.15.71:0/827210239 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f171806cce0 msgr2=0x7f171806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:24.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.585+0000 7f1733d20700 1 --2- 172.21.15.71:0/827210239 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f171806cce0 0x7f171806f1a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f172c068e50 tx=0x7f171c00b410 comp rx=0 tx=0).stop 2024-04-23T02:38:24.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.585+0000 7f1733d20700 1 -- 172.21.15.71:0/827210239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f172c0ffc20 msgr2=0x7f172c1975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:24.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.585+0000 7f1733d20700 1 --2- 172.21.15.71:0/827210239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f172c0ffc20 0x7f172c1975e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1728006010 tx=0x7f172800bd70 comp rx=0 tx=0).stop 2024-04-23T02:38:24.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.585+0000 7f1733d20700 1 -- 172.21.15.71:0/827210239 shutdown_connections 2024-04-23T02:38:24.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.585+0000 7f1733d20700 1 --2- 172.21.15.71:0/827210239 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f171806cce0 0x7f171806f1a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.585+0000 7f1733d20700 1 --2- 172.21.15.71:0/827210239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f172c0ffc20 0x7f172c1975e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.585+0000 7f1733d20700 1 --2- 172.21.15.71:0/827210239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f172c105bd0 0x7f172c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.585+0000 7f1733d20700 1 -- 172.21.15.71:0/827210239 >> 172.21.15.71:0/827210239 conn(0x7f172c075670 msgr2=0x7f172c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:24.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.585+0000 7f1733d20700 1 -- 172.21.15.71:0/827210239 shutdown_connections 2024-04-23T02:38:24.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.585+0000 7f1733d20700 1 -- 172.21.15.71:0/827210239 wait complete. 2024-04-23T02:38:24.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.686+0000 7f862c88f700 1 -- 172.21.15.71:0/2239973732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8628105bd0 msgr2=0x7f8628105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:24.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.686+0000 7f862c88f700 1 --2- 172.21.15.71:0/2239973732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8628105bd0 0x7f8628105fb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8618009910 tx=0x7f8618009c20 comp rx=0 tx=0).stop 2024-04-23T02:38:24.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.686+0000 7f862c88f700 1 -- 172.21.15.71:0/2239973732 shutdown_connections 2024-04-23T02:38:24.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.686+0000 7f862c88f700 1 --2- 172.21.15.71:0/2239973732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86280ffc20 0x7f8628100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.686+0000 7f862c88f700 1 --2- 172.21.15.71:0/2239973732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8628105bd0 0x7f8628105fb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.686+0000 7f862c88f700 1 -- 172.21.15.71:0/2239973732 >> 172.21.15.71:0/2239973732 conn(0x7f8628075670 msgr2=0x7f8628075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:24.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.686+0000 7f862c88f700 1 -- 172.21.15.71:0/2239973732 shutdown_connections 2024-04-23T02:38:24.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.687+0000 7f862c88f700 1 -- 172.21.15.71:0/2239973732 wait complete. 2024-04-23T02:38:24.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.687+0000 7f862c88f700 1 Processor -- start 2024-04-23T02:38:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.687+0000 7f862c88f700 1 -- start start 2024-04-23T02:38:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.687+0000 7f862c88f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86280ffc20 0x7f862819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.687+0000 7f862c88f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8628105bd0 0x7f862819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.687+0000 7f862c88f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8628196610 con 0x7f86280ffc20 2024-04-23T02:38:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.687+0000 7f862c88f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8628196780 con 0x7f8628105bd0 2024-04-23T02:38:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.688+0000 7f862659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86280ffc20 0x7f862819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.688+0000 7f862659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86280ffc20 0x7f862819c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44302/0 (socket says 172.21.15.71:44302) 2024-04-23T02:38:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.688+0000 7f862659c700 1 -- 172.21.15.71:0/305353228 learned_addr learned my addr 172.21.15.71:0/305353228 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:38:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.688+0000 7f8625d9b700 1 --2- 172.21.15.71:0/305353228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8628105bd0 0x7f862819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.688+0000 7f862659c700 1 -- 172.21.15.71:0/305353228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8628105bd0 msgr2=0x7f862819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.688+0000 7f862659c700 1 --2- 172.21.15.71:0/305353228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8628105bd0 0x7f862819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.688+0000 7f862659c700 1 -- 172.21.15.71:0/305353228 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86180095f0 con 0x7f86280ffc20 2024-04-23T02:38:24.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.688+0000 7f8625d9b700 1 --2- 172.21.15.71:0/305353228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8628105bd0 0x7f862819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:38:24.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.688+0000 7f862659c700 1 --2- 172.21.15.71:0/305353228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86280ffc20 0x7f862819c620 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8618006010 tx=0x7f861800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:24.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.688+0000 7f86177fe700 1 -- 172.21.15.71:0/305353228 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f861801d070 con 0x7f86280ffc20 2024-04-23T02:38:24.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.688+0000 7f862c88f700 1 -- 172.21.15.71:0/305353228 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8628196a00 con 0x7f86280ffc20 2024-04-23T02:38:24.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.688+0000 7f86177fe700 1 -- 172.21.15.71:0/305353228 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f861800f460 con 0x7f86280ffc20 2024-04-23T02:38:24.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.688+0000 7f86177fe700 1 -- 172.21.15.71:0/305353228 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8618021640 con 0x7f86280ffc20 2024-04-23T02:38:24.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.688+0000 7f862c88f700 1 -- 172.21.15.71:0/305353228 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8628196ef0 con 0x7f86280ffc20 2024-04-23T02:38:24.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.689+0000 7f862c88f700 1 -- 172.21.15.71:0/305353228 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f862804f2e0 con 0x7f86280ffc20 2024-04-23T02:38:24.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.690+0000 7f86177fe700 1 -- 172.21.15.71:0/305353228 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f861800fa60 con 0x7f86280ffc20 2024-04-23T02:38:24.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.690+0000 7f86177fe700 1 --2- 172.21.15.71:0/305353228 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f861006cce0 0x7f861006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:24.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.690+0000 7f8625d9b700 1 --2- 172.21.15.71:0/305353228 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f861006cce0 0x7f861006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:24.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.690+0000 7f86177fe700 1 -- 172.21.15.71:0/305353228 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f861808d880 con 0x7f86280ffc20 2024-04-23T02:38:24.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.691+0000 7f8625d9b700 1 --2- 172.21.15.71:0/305353228 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f861006cce0 0x7f861006f1a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f861c005fd0 tx=0x7f861c005dc0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:24.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.695+0000 7f86177fe700 1 -- 172.21.15.71:0/305353228 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f86180578f0 con 0x7f86280ffc20 2024-04-23T02:38:24.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:24 smithi192 ceph-mon[108617]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:38:24.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:24 smithi192 ceph-mon[108617]: from='client.35188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:38:24.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:24 smithi192 ceph-mon[108617]: from='client.35192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:38:24.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:24 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/4208712673' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:38:24.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:24 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3611279967' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:38:24.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.975+0000 7f862c88f700 1 -- 172.21.15.71:0/305353228 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8628197110 con 0x7f86280ffc20 2024-04-23T02:38:24.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.975+0000 7f86177fe700 1 -- 172.21.15.71:0/305353228 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7f861805af10 con 0x7f86280ffc20 2024-04-23T02:38:24.977 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:38:24.978 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:38:24.978 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:38:24.978 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:38:24.978 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:38:24.978 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:38:24.978 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:38:24.978 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:38:24.978 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:38:24.978 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:38:24.978 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:38:24.978 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:38:24.978 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:38:24.979 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:38:24.979 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:38:24.979 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:38:24.979 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:38:24.979 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:38:24.979 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:38:24.979 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:38:24.979 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:38:24.979 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:38:24.979 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:38:24.979 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:38:24.979 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:38:24.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.977+0000 7f862c88f700 1 -- 172.21.15.71:0/305353228 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f861006cce0 msgr2=0x7f861006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:24.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.977+0000 7f862c88f700 1 --2- 172.21.15.71:0/305353228 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f861006cce0 0x7f861006f1a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f861c005fd0 tx=0x7f861c005dc0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.977+0000 7f862c88f700 1 -- 172.21.15.71:0/305353228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86280ffc20 msgr2=0x7f862819c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:24.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.977+0000 7f862c88f700 1 --2- 172.21.15.71:0/305353228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86280ffc20 0x7f862819c620 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8618006010 tx=0x7f861800be20 comp rx=0 tx=0).stop 2024-04-23T02:38:24.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.977+0000 7f862c88f700 1 -- 172.21.15.71:0/305353228 shutdown_connections 2024-04-23T02:38:24.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.977+0000 7f862c88f700 1 --2- 172.21.15.71:0/305353228 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f861006cce0 0x7f861006f1a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.977+0000 7f862c88f700 1 --2- 172.21.15.71:0/305353228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86280ffc20 0x7f862819c620 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.977+0000 7f862c88f700 1 --2- 172.21.15.71:0/305353228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8628105bd0 0x7f862819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:24.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.977+0000 7f862c88f700 1 -- 172.21.15.71:0/305353228 >> 172.21.15.71:0/305353228 conn(0x7f8628075670 msgr2=0x7f8628068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:24.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.977+0000 7f862c88f700 1 -- 172.21.15.71:0/305353228 shutdown_connections 2024-04-23T02:38:24.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:24.977+0000 7f862c88f700 1 -- 172.21.15.71:0/305353228 wait complete. 2024-04-23T02:38:24.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:24 smithi071 ceph-mon[144544]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:38:24.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:24 smithi071 ceph-mon[144544]: from='client.35188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:38:24.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:24 smithi071 ceph-mon[144544]: from='client.35192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:38:24.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:24 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/4208712673' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:38:24.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:24 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3611279967' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:38:25.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:25 smithi192 ceph-mon[108617]: pgmap v754: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:25.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:25 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/305353228' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:38:25.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:25 smithi071 ceph-mon[144544]: pgmap v754: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:25.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:25 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/305353228' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:38:26.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:26 smithi192 ceph-mon[108617]: from='client.35204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:38:26.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:26 smithi071 ceph-mon[144544]: from='client.35204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:38:27.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:27 smithi192 ceph-mon[108617]: pgmap v755: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:27.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:27 smithi071 ceph-mon[144544]: pgmap v755: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:29.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:29 smithi192 ceph-mon[108617]: pgmap v756: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:29.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:29 smithi071 ceph-mon[144544]: pgmap v756: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:31.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:31 smithi192 ceph-mon[108617]: pgmap v757: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:31.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:31 smithi071 ceph-mon[144544]: pgmap v757: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:33.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:33 smithi192 ceph-mon[108617]: pgmap v758: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:33.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:33 smithi071 ceph-mon[144544]: pgmap v758: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:35.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:35 smithi192 ceph-mon[108617]: pgmap v759: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:35.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:38:35.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:35 smithi071 ceph-mon[144544]: pgmap v759: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:35.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:38:37.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:37 smithi192 ceph-mon[108617]: pgmap v760: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:37.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:37 smithi071 ceph-mon[144544]: pgmap v760: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:39.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:39 smithi192 ceph-mon[108617]: pgmap v761: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:39.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:39 smithi071 ceph-mon[144544]: pgmap v761: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:41.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:41 smithi192 ceph-mon[108617]: pgmap v762: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:41.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:41 smithi071 ceph-mon[144544]: pgmap v762: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:43.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:43 smithi192 ceph-mon[108617]: pgmap v763: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:43.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:43 smithi071 ceph-mon[144544]: pgmap v763: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:45.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:45 smithi192 ceph-mon[108617]: pgmap v764: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:45.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:45 smithi071 ceph-mon[144544]: pgmap v764: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:47.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:47 smithi192 ceph-mon[108617]: pgmap v765: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:47.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:47 smithi071 ceph-mon[144544]: pgmap v765: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:49.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:49 smithi192 ceph-mon[108617]: pgmap v766: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:49.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:49 smithi071 ceph-mon[144544]: pgmap v766: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:38:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:38:51.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:51 smithi192 ceph-mon[108617]: pgmap v767: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:51.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:51 smithi071 ceph-mon[144544]: pgmap v767: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:52.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:52 smithi192 ceph-mon[108617]: pgmap v768: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:52.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:52 smithi071 ceph-mon[144544]: pgmap v768: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:55.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.091+0000 7fbb7d008700 1 -- 172.21.15.71:0/2892675842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb780ff5a0 msgr2=0x7fbb780ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:55.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.091+0000 7fbb7d008700 1 --2- 172.21.15.71:0/2892675842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb780ff5a0 0x7fbb780ff980 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800b0a0 tx=0x7fbb6800b3b0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.091+0000 7fbb7d008700 1 -- 172.21.15.71:0/2892675842 shutdown_connections 2024-04-23T02:38:55.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.091+0000 7fbb7d008700 1 --2- 172.21.15.71:0/2892675842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb780fff50 0x7fbb78105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.091+0000 7fbb7d008700 1 --2- 172.21.15.71:0/2892675842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb780ff5a0 0x7fbb780ff980 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.091+0000 7fbb7d008700 1 -- 172.21.15.71:0/2892675842 >> 172.21.15.71:0/2892675842 conn(0x7fbb78074e30 msgr2=0x7fbb78075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:55.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.091+0000 7fbb7d008700 1 -- 172.21.15.71:0/2892675842 shutdown_connections 2024-04-23T02:38:55.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.091+0000 7fbb7d008700 1 -- 172.21.15.71:0/2892675842 wait complete. 2024-04-23T02:38:55.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.092+0000 7fbb7d008700 1 Processor -- start 2024-04-23T02:38:55.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.093+0000 7fbb7d008700 1 -- start start 2024-04-23T02:38:55.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.093+0000 7fbb7d008700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb780ff5a0 0x7fbb7819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:55.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.093+0000 7fbb7d008700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb780fff50 0x7fbb7819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:55.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.093+0000 7fbb7d008700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7819d170 con 0x7fbb780fff50 2024-04-23T02:38:55.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.093+0000 7fbb7d008700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb78196620 con 0x7fbb780ff5a0 2024-04-23T02:38:55.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.093+0000 7fbb7659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb780fff50 0x7fbb7819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:55.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.093+0000 7fbb7659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb780fff50 0x7fbb7819cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56300/0 (socket says 172.21.15.71:56300) 2024-04-23T02:38:55.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.093+0000 7fbb7659c700 1 -- 172.21.15.71:0/142951763 learned_addr learned my addr 172.21.15.71:0/142951763 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:38:55.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.093+0000 7fbb76d9d700 1 --2- 172.21.15.71:0/142951763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb780ff5a0 0x7fbb7819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:55.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.093+0000 7fbb7659c700 1 -- 172.21.15.71:0/142951763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb780ff5a0 msgr2=0x7fbb7819c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:55.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.093+0000 7fbb7659c700 1 --2- 172.21.15.71:0/142951763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb780ff5a0 0x7fbb7819c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.093+0000 7fbb7659c700 1 -- 172.21.15.71:0/142951763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb68009d00 con 0x7fbb780fff50 2024-04-23T02:38:55.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.094+0000 7fbb7659c700 1 --2- 172.21.15.71:0/142951763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb780fff50 0x7fbb7819cae0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000d8d0 tx=0x7fbb6000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:55.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.094+0000 7fbb6ffff700 1 -- 172.21.15.71:0/142951763 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb60004a00 con 0x7fbb780fff50 2024-04-23T02:38:55.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.094+0000 7fbb6ffff700 1 -- 172.21.15.71:0/142951763 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbb60005020 con 0x7fbb780fff50 2024-04-23T02:38:55.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.094+0000 7fbb6ffff700 1 -- 172.21.15.71:0/142951763 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb6000fb80 con 0x7fbb780fff50 2024-04-23T02:38:55.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.094+0000 7fbb7d008700 1 -- 172.21.15.71:0/142951763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb78196900 con 0x7fbb780fff50 2024-04-23T02:38:55.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.094+0000 7fbb7d008700 1 -- 172.21.15.71:0/142951763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb78196e00 con 0x7fbb780fff50 2024-04-23T02:38:55.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.095+0000 7fbb7d008700 1 -- 172.21.15.71:0/142951763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb78109270 con 0x7fbb780fff50 2024-04-23T02:38:55.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.095+0000 7fbb6ffff700 1 -- 172.21.15.71:0/142951763 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbb60005190 con 0x7fbb780fff50 2024-04-23T02:38:55.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.096+0000 7fbb6ffff700 1 --2- 172.21.15.71:0/142951763 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbb6406cce0 0x7fbb6406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:55.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.096+0000 7fbb6ffff700 1 -- 172.21.15.71:0/142951763 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbb6008bf10 con 0x7fbb780fff50 2024-04-23T02:38:55.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.096+0000 7fbb76d9d700 1 --2- 172.21.15.71:0/142951763 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbb6406cce0 0x7fbb6406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:55.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.096+0000 7fbb76d9d700 1 --2- 172.21.15.71:0/142951763 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbb6406cce0 0x7fbb6406f1a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbb68009fd0 tx=0x7fbb680060c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:55.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.101+0000 7fbb6ffff700 1 -- 172.21.15.71:0/142951763 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbb600599b0 con 0x7fbb780fff50 2024-04-23T02:38:55.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.312+0000 7fbb7d008700 1 -- 172.21.15.71:0/142951763 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb78197070 con 0x7fbb6406cce0 2024-04-23T02:38:55.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.313+0000 7fbb6ffff700 1 -- 172.21.15.71:0/142951763 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbb78197070 con 0x7fbb6406cce0 2024-04-23T02:38:55.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.315+0000 7fbb7d008700 1 -- 172.21.15.71:0/142951763 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbb6406cce0 msgr2=0x7fbb6406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:55.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.315+0000 7fbb7d008700 1 --2- 172.21.15.71:0/142951763 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbb6406cce0 0x7fbb6406f1a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbb68009fd0 tx=0x7fbb680060c0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.315+0000 7fbb7d008700 1 -- 172.21.15.71:0/142951763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb780fff50 msgr2=0x7fbb7819cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:55.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.315+0000 7fbb7d008700 1 --2- 172.21.15.71:0/142951763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb780fff50 0x7fbb7819cae0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000d8d0 tx=0x7fbb6000dbe0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.315+0000 7fbb7d008700 1 -- 172.21.15.71:0/142951763 shutdown_connections 2024-04-23T02:38:55.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.315+0000 7fbb7d008700 1 --2- 172.21.15.71:0/142951763 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbb6406cce0 0x7fbb6406f1a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.315+0000 7fbb7d008700 1 --2- 172.21.15.71:0/142951763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb780ff5a0 0x7fbb7819c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.315+0000 7fbb7d008700 1 --2- 172.21.15.71:0/142951763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb780fff50 0x7fbb7819cae0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.315+0000 7fbb7d008700 1 -- 172.21.15.71:0/142951763 >> 172.21.15.71:0/142951763 conn(0x7fbb78074e30 msgr2=0x7fbb780fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:55.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.315+0000 7fbb7d008700 1 -- 172.21.15.71:0/142951763 shutdown_connections 2024-04-23T02:38:55.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.315+0000 7fbb7d008700 1 -- 172.21.15.71:0/142951763 wait complete. 2024-04-23T02:38:55.329 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:38:55.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.419+0000 7f354e15b700 1 -- 172.21.15.71:0/2858123393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3548101de0 msgr2=0x7f35481021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:55.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.419+0000 7f354e15b700 1 --2- 172.21.15.71:0/2858123393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3548101de0 0x7f35481021c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3530009910 tx=0x7f3530009c20 comp rx=0 tx=0).stop 2024-04-23T02:38:55.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.419+0000 7f354e15b700 1 -- 172.21.15.71:0/2858123393 shutdown_connections 2024-04-23T02:38:55.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.419+0000 7f354e15b700 1 --2- 172.21.15.71:0/2858123393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3548102700 0x7f354810abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.419+0000 7f354e15b700 1 --2- 172.21.15.71:0/2858123393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3548101de0 0x7f35481021c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.419+0000 7f354e15b700 1 -- 172.21.15.71:0/2858123393 >> 172.21.15.71:0/2858123393 conn(0x7f35480fb610 msgr2=0x7f35480fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:55.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.419+0000 7f354e15b700 1 -- 172.21.15.71:0/2858123393 shutdown_connections 2024-04-23T02:38:55.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.419+0000 7f354e15b700 1 -- 172.21.15.71:0/2858123393 wait complete. 2024-04-23T02:38:55.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.419+0000 7f354e15b700 1 Processor -- start 2024-04-23T02:38:55.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.419+0000 7f354e15b700 1 -- start start 2024-04-23T02:38:55.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.420+0000 7f354e15b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3548101de0 0x7f35480ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:55.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.420+0000 7f354e15b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3548102700 0x7f35480fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:55.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.420+0000 7f354e15b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35481004c0 con 0x7f3548102700 2024-04-23T02:38:55.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.420+0000 7f354e15b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3548100600 con 0x7f3548101de0 2024-04-23T02:38:55.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.420+0000 7f3546ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3548102700 0x7f35480fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:55.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.420+0000 7f3546ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3548102700 0x7f35480fff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35176/0 (socket says 172.21.15.71:35176) 2024-04-23T02:38:55.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.420+0000 7f3546ffd700 1 -- 172.21.15.71:0/4034045433 learned_addr learned my addr 172.21.15.71:0/4034045433 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:38:55.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.420+0000 7f35477fe700 1 --2- 172.21.15.71:0/4034045433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3548101de0 0x7f35480ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:55.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.420+0000 7f3546ffd700 1 -- 172.21.15.71:0/4034045433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3548101de0 msgr2=0x7f35480ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:55.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.420+0000 7f3546ffd700 1 --2- 172.21.15.71:0/4034045433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3548101de0 0x7f35480ffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.420+0000 7f3546ffd700 1 -- 172.21.15.71:0/4034045433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35300095f0 con 0x7f3548102700 2024-04-23T02:38:55.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.420+0000 7f3546ffd700 1 --2- 172.21.15.71:0/4034045433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3548102700 0x7f35480fff80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f353800d6d0 tx=0x7f353800d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:55.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.420+0000 7f3544ff9700 1 -- 172.21.15.71:0/4034045433 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f353800f860 con 0x7f3548102700 2024-04-23T02:38:55.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.420+0000 7f3544ff9700 1 -- 172.21.15.71:0/4034045433 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f353800fe60 con 0x7f3548102700 2024-04-23T02:38:55.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.420+0000 7f354e15b700 1 -- 172.21.15.71:0/4034045433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35481a29c0 con 0x7f3548102700 2024-04-23T02:38:55.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.420+0000 7f3544ff9700 1 -- 172.21.15.71:0/4034045433 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f353800e3b0 con 0x7f3548102700 2024-04-23T02:38:55.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.420+0000 7f354e15b700 1 -- 172.21.15.71:0/4034045433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35481a2dd0 con 0x7f3548102700 2024-04-23T02:38:55.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.421+0000 7f354e15b700 1 -- 172.21.15.71:0/4034045433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35481082f0 con 0x7f3548102700 2024-04-23T02:38:55.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.422+0000 7f3544ff9700 1 -- 172.21.15.71:0/4034045433 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3538010460 con 0x7f3548102700 2024-04-23T02:38:55.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.422+0000 7f3544ff9700 1 --2- 172.21.15.71:0/4034045433 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f353406cce0 0x7f353406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:55.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.422+0000 7f3544ff9700 1 -- 172.21.15.71:0/4034045433 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f353808c280 con 0x7f3548102700 2024-04-23T02:38:55.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.422+0000 7f35477fe700 1 --2- 172.21.15.71:0/4034045433 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f353406cce0 0x7f353406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:55.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.423+0000 7f35477fe700 1 --2- 172.21.15.71:0/4034045433 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f353406cce0 0x7f353406f1a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3530006010 tx=0x7f353001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:55.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.427+0000 7f3544ff9700 1 -- 172.21.15.71:0/4034045433 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f35380562f0 con 0x7f3548102700 2024-04-23T02:38:55.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.636+0000 7f354e15b700 1 -- 172.21.15.71:0/4034045433 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35480611d0 con 0x7f353406cce0 2024-04-23T02:38:55.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.638+0000 7f3544ff9700 1 -- 172.21.15.71:0/4034045433 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f35480611d0 con 0x7f353406cce0 2024-04-23T02:38:55.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.640+0000 7f354e15b700 1 -- 172.21.15.71:0/4034045433 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f353406cce0 msgr2=0x7f353406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:55.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.640+0000 7f354e15b700 1 --2- 172.21.15.71:0/4034045433 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f353406cce0 0x7f353406f1a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3530006010 tx=0x7f353001a040 comp rx=0 tx=0).stop 2024-04-23T02:38:55.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.640+0000 7f354e15b700 1 -- 172.21.15.71:0/4034045433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3548102700 msgr2=0x7f35480fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:55.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.640+0000 7f354e15b700 1 --2- 172.21.15.71:0/4034045433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3548102700 0x7f35480fff80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f353800d6d0 tx=0x7f353800d9e0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.640+0000 7f354e15b700 1 -- 172.21.15.71:0/4034045433 shutdown_connections 2024-04-23T02:38:55.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.640+0000 7f354e15b700 1 --2- 172.21.15.71:0/4034045433 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f353406cce0 0x7f353406f1a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.640+0000 7f354e15b700 1 --2- 172.21.15.71:0/4034045433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3548101de0 0x7f35480ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.640+0000 7f354e15b700 1 --2- 172.21.15.71:0/4034045433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3548102700 0x7f35480fff80 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.640+0000 7f354e15b700 1 -- 172.21.15.71:0/4034045433 >> 172.21.15.71:0/4034045433 conn(0x7f35480fb610 msgr2=0x7f3548105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:55.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.640+0000 7f354e15b700 1 -- 172.21.15.71:0/4034045433 shutdown_connections 2024-04-23T02:38:55.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.640+0000 7f354e15b700 1 -- 172.21.15.71:0/4034045433 wait complete. 2024-04-23T02:38:55.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:55 smithi071 ceph-mon[144544]: pgmap v769: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:55.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.743+0000 7fe216d7a700 1 -- 172.21.15.71:0/537669876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2100ffb40 msgr2=0x7fe2100fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:55.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.743+0000 7fe216d7a700 1 --2- 172.21.15.71:0/537669876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2100ffb40 0x7fe2100fffc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009910 tx=0x7fe1fc009c20 comp rx=0 tx=0).stop 2024-04-23T02:38:55.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.743+0000 7fe216d7a700 1 -- 172.21.15.71:0/537669876 shutdown_connections 2024-04-23T02:38:55.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.743+0000 7fe216d7a700 1 --2- 172.21.15.71:0/537669876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2100ffb40 0x7fe2100fffc0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.743+0000 7fe216d7a700 1 --2- 172.21.15.71:0/537669876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe210105bf0 0x7fe210105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.743+0000 7fe216d7a700 1 -- 172.21.15.71:0/537669876 >> 172.21.15.71:0/537669876 conn(0x7fe2100fb610 msgr2=0x7fe2100fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:55.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.743+0000 7fe216d7a700 1 -- 172.21.15.71:0/537669876 shutdown_connections 2024-04-23T02:38:55.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.743+0000 7fe216d7a700 1 -- 172.21.15.71:0/537669876 wait complete. 2024-04-23T02:38:55.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.743+0000 7fe216d7a700 1 Processor -- start 2024-04-23T02:38:55.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.744+0000 7fe216d7a700 1 -- start start 2024-04-23T02:38:55.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.744+0000 7fe216d7a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2100ffb40 0x7fe210197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:55.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.744+0000 7fe216d7a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe210105bf0 0x7fe210197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:55.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.744+0000 7fe216d7a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2101980d0 con 0x7fe2100ffb40 2024-04-23T02:38:55.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.744+0000 7fe216d7a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe210198240 con 0x7fe210105bf0 2024-04-23T02:38:55.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.744+0000 7fe214b16700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2100ffb40 0x7fe210197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:55.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.744+0000 7fe214b16700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2100ffb40 0x7fe210197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35192/0 (socket says 172.21.15.71:35192) 2024-04-23T02:38:55.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.744+0000 7fe214b16700 1 -- 172.21.15.71:0/2026142986 learned_addr learned my addr 172.21.15.71:0/2026142986 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:38:55.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.744+0000 7fe20ffff700 1 --2- 172.21.15.71:0/2026142986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe210105bf0 0x7fe210197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:55.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.744+0000 7fe214b16700 1 -- 172.21.15.71:0/2026142986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe210105bf0 msgr2=0x7fe210197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:55.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.744+0000 7fe214b16700 1 --2- 172.21.15.71:0/2026142986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe210105bf0 0x7fe210197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.744+0000 7fe214b16700 1 -- 172.21.15.71:0/2026142986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc0095f0 con 0x7fe2100ffb40 2024-04-23T02:38:55.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.744+0000 7fe214b16700 1 --2- 172.21.15.71:0/2026142986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2100ffb40 0x7fe210197570 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe20400d8d0 tx=0x7fe20400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:55.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.744+0000 7fe20dffb700 1 -- 172.21.15.71:0/2026142986 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe204009940 con 0x7fe2100ffb40 2024-04-23T02:38:55.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.744+0000 7fe20dffb700 1 -- 172.21.15.71:0/2026142986 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe204010460 con 0x7fe2100ffb40 2024-04-23T02:38:55.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.744+0000 7fe216d7a700 1 -- 172.21.15.71:0/2026142986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2101a05e0 con 0x7fe2100ffb40 2024-04-23T02:38:55.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.745+0000 7fe20dffb700 1 -- 172.21.15.71:0/2026142986 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe20400f5f0 con 0x7fe2100ffb40 2024-04-23T02:38:55.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.745+0000 7fe216d7a700 1 -- 172.21.15.71:0/2026142986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2101a0ad0 con 0x7fe2100ffb40 2024-04-23T02:38:55.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.746+0000 7fe216d7a700 1 -- 172.21.15.71:0/2026142986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2101080a0 con 0x7fe2100ffb40 2024-04-23T02:38:55.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.746+0000 7fe20dffb700 1 -- 172.21.15.71:0/2026142986 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe2040105d0 con 0x7fe2100ffb40 2024-04-23T02:38:55.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.746+0000 7fe20dffb700 1 --2- 172.21.15.71:0/2026142986 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe20006cce0 0x7fe20006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:55.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.747+0000 7fe20dffb700 1 -- 172.21.15.71:0/2026142986 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe20408c5e0 con 0x7fe2100ffb40 2024-04-23T02:38:55.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.747+0000 7fe20ffff700 1 --2- 172.21.15.71:0/2026142986 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe20006cce0 0x7fe20006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:55.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.747+0000 7fe20ffff700 1 --2- 172.21.15.71:0/2026142986 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe20006cce0 0x7fe20006f1a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe2100696e0 tx=0x7fe1fc01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:55.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.751+0000 7fe20dffb700 1 -- 172.21.15.71:0/2026142986 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe204056650 con 0x7fe2100ffb40 2024-04-23T02:38:55.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:55 smithi192 ceph-mon[108617]: pgmap v769: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:55.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.955+0000 7fe216d7a700 1 -- 172.21.15.71:0/2026142986 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2100611d0 con 0x7fe20006cce0 2024-04-23T02:38:55.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.965+0000 7fe20dffb700 1 -- 172.21.15.71:0/2026142986 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe2100611d0 con 0x7fe20006cce0 2024-04-23T02:38:55.969 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:38:55.969 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (22m) 52s ago 33m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:38:55.969 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (34m) 52s ago 34m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:38:55.969 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (33m) 54s ago 33m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:38:55.969 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (20m) 52s ago 34m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:38:55.969 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (20m) 54s ago 32m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:38:55.969 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (22m) 52s ago 33m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:38:55.969 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (16m) 52s ago 28m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:38:55.969 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (16m) 52s ago 27m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:38:55.970 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (58s) 54s ago 27m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:38:55.970 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (16m) 54s ago 27m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:38:55.970 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (24m) 52s ago 35m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:38:55.970 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (23m) 54s ago 32m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:38:55.970 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (21m) 52s ago 35m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:38:55.970 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (20m) 54s ago 32m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:38:55.970 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (23m) 52s ago 34m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:38:55.970 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (23m) 54s ago 32m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:38:55.970 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (20m) 52s ago 31m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:38:55.970 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (19m) 52s ago 31m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:38:55.970 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (18m) 52s ago 30m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:38:55.970 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (18m) 54s ago 30m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:38:55.970 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (17m) 54s ago 30m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:38:55.970 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (17m) 54s ago 29m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:38:55.970 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (22m) 52s ago 33m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:38:55.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.967+0000 7fe216d7a700 1 -- 172.21.15.71:0/2026142986 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe20006cce0 msgr2=0x7fe20006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:55.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.967+0000 7fe216d7a700 1 --2- 172.21.15.71:0/2026142986 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe20006cce0 0x7fe20006f1a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe2100696e0 tx=0x7fe1fc01a040 comp rx=0 tx=0).stop 2024-04-23T02:38:55.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.967+0000 7fe216d7a700 1 -- 172.21.15.71:0/2026142986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2100ffb40 msgr2=0x7fe210197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:55.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.967+0000 7fe216d7a700 1 --2- 172.21.15.71:0/2026142986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2100ffb40 0x7fe210197570 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe20400d8d0 tx=0x7fe20400dc90 comp rx=0 tx=0).stop 2024-04-23T02:38:55.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.968+0000 7fe216d7a700 1 -- 172.21.15.71:0/2026142986 shutdown_connections 2024-04-23T02:38:55.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.968+0000 7fe216d7a700 1 --2- 172.21.15.71:0/2026142986 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe20006cce0 0x7fe20006f1a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.968+0000 7fe216d7a700 1 --2- 172.21.15.71:0/2026142986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2100ffb40 0x7fe210197570 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.968+0000 7fe216d7a700 1 --2- 172.21.15.71:0/2026142986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe210105bf0 0x7fe210197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:55.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.968+0000 7fe216d7a700 1 -- 172.21.15.71:0/2026142986 >> 172.21.15.71:0/2026142986 conn(0x7fe2100fb610 msgr2=0x7fe2100fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:55.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.968+0000 7fe216d7a700 1 -- 172.21.15.71:0/2026142986 shutdown_connections 2024-04-23T02:38:55.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:55.968+0000 7fe216d7a700 1 -- 172.21.15.71:0/2026142986 wait complete. 2024-04-23T02:38:56.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.069+0000 7fa2df047700 1 -- 172.21.15.71:0/4232301293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2d80ffda0 msgr2=0x7fa2d8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:56.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.069+0000 7fa2df047700 1 --2- 172.21.15.71:0/4232301293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2d80ffda0 0x7fa2d8100180 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009910 tx=0x7fa2cc009c20 comp rx=0 tx=0).stop 2024-04-23T02:38:56.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.069+0000 7fa2df047700 1 -- 172.21.15.71:0/4232301293 shutdown_connections 2024-04-23T02:38:56.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.069+0000 7fa2df047700 1 --2- 172.21.15.71:0/4232301293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2d8100750 0x7fa2d81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:56.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.069+0000 7fa2df047700 1 --2- 172.21.15.71:0/4232301293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2d80ffda0 0x7fa2d8100180 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:56.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.069+0000 7fa2df047700 1 -- 172.21.15.71:0/4232301293 >> 172.21.15.71:0/4232301293 conn(0x7fa2d80fb650 msgr2=0x7fa2d80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:56.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.069+0000 7fa2df047700 1 -- 172.21.15.71:0/4232301293 shutdown_connections 2024-04-23T02:38:56.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.069+0000 7fa2df047700 1 -- 172.21.15.71:0/4232301293 wait complete. 2024-04-23T02:38:56.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.070+0000 7fa2df047700 1 Processor -- start 2024-04-23T02:38:56.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.070+0000 7fa2df047700 1 -- start start 2024-04-23T02:38:56.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.070+0000 7fa2df047700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2d80ffda0 0x7fa2d819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:56.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.070+0000 7fa2df047700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2d8100750 0x7fa2d819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:56.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.070+0000 7fa2df047700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d819f370 con 0x7fa2d8100750 2024-04-23T02:38:56.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.070+0000 7fa2df047700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d8198820 con 0x7fa2d80ffda0 2024-04-23T02:38:56.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.070+0000 7fa2d7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2d8100750 0x7fa2d819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:56.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.070+0000 7fa2d7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2d8100750 0x7fa2d819ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35208/0 (socket says 172.21.15.71:35208) 2024-04-23T02:38:56.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.070+0000 7fa2d7fff700 1 -- 172.21.15.71:0/1695740918 learned_addr learned my addr 172.21.15.71:0/1695740918 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:38:56.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.070+0000 7fa2dcde3700 1 --2- 172.21.15.71:0/1695740918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2d80ffda0 0x7fa2d819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:56.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.070+0000 7fa2d7fff700 1 -- 172.21.15.71:0/1695740918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2d80ffda0 msgr2=0x7fa2d819e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:56.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.070+0000 7fa2d7fff700 1 --2- 172.21.15.71:0/1695740918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2d80ffda0 0x7fa2d819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:56.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.070+0000 7fa2d7fff700 1 -- 172.21.15.71:0/1695740918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc0095f0 con 0x7fa2d8100750 2024-04-23T02:38:56.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.070+0000 7fa2d7fff700 1 --2- 172.21.15.71:0/1695740918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2d8100750 0x7fa2d819ece0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400d900 tx=0x7fa2c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:56.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.070+0000 7fa2d5ffb700 1 -- 172.21.15.71:0/1695740918 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2c4009900 con 0x7fa2d8100750 2024-04-23T02:38:56.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.070+0000 7fa2d5ffb700 1 -- 172.21.15.71:0/1695740918 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa2c4010460 con 0x7fa2d8100750 2024-04-23T02:38:56.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.070+0000 7fa2d5ffb700 1 -- 172.21.15.71:0/1695740918 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2c400f5f0 con 0x7fa2d8100750 2024-04-23T02:38:56.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.070+0000 7fa2df047700 1 -- 172.21.15.71:0/1695740918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d8198b00 con 0x7fa2d8100750 2024-04-23T02:38:56.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.071+0000 7fa2df047700 1 -- 172.21.15.71:0/1695740918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d8199050 con 0x7fa2d8100750 2024-04-23T02:38:56.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.072+0000 7fa2df047700 1 -- 172.21.15.71:0/1695740918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d804f2e0 con 0x7fa2d8100750 2024-04-23T02:38:56.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.072+0000 7fa2d5ffb700 1 -- 172.21.15.71:0/1695740918 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa2c40105d0 con 0x7fa2d8100750 2024-04-23T02:38:56.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.072+0000 7fa2d5ffb700 1 --2- 172.21.15.71:0/1695740918 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa2c806cce0 0x7fa2c806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:56.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.072+0000 7fa2d5ffb700 1 -- 172.21.15.71:0/1695740918 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa2c408c310 con 0x7fa2d8100750 2024-04-23T02:38:56.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.072+0000 7fa2dcde3700 1 --2- 172.21.15.71:0/1695740918 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa2c806cce0 0x7fa2c806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:56.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.073+0000 7fa2dcde3700 1 --2- 172.21.15.71:0/1695740918 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa2c806cce0 0x7fa2c806f1a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00b5c0 tx=0x7fa2cc005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:56.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.077+0000 7fa2d5ffb700 1 -- 172.21.15.71:0/1695740918 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa2c4059e90 con 0x7fa2d8100750 2024-04-23T02:38:56.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.350+0000 7fa2df047700 1 -- 172.21.15.71:0/1695740918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa2d81992c0 con 0x7fa2d8100750 2024-04-23T02:38:56.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.350+0000 7fa2d5ffb700 1 -- 172.21.15.71:0/1695740918 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fa2c4059a20 con 0x7fa2d8100750 2024-04-23T02:38:56.354 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:38:56.354 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:38:56.354 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:38:56.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:38:56.355 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:38:56.355 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:38:56.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:38:56.355 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:38:56.355 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:38:56.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:38:56.355 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:38:56.355 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:38:56.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:38:56.355 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:38:56.355 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:38:56.356 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:38:56.356 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:38:56.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.352+0000 7fa2df047700 1 -- 172.21.15.71:0/1695740918 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa2c806cce0 msgr2=0x7fa2c806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:56.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.352+0000 7fa2df047700 1 --2- 172.21.15.71:0/1695740918 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa2c806cce0 0x7fa2c806f1a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00b5c0 tx=0x7fa2cc005fb0 comp rx=0 tx=0).stop 2024-04-23T02:38:56.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.352+0000 7fa2df047700 1 -- 172.21.15.71:0/1695740918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2d8100750 msgr2=0x7fa2d819ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:56.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.352+0000 7fa2df047700 1 --2- 172.21.15.71:0/1695740918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2d8100750 0x7fa2d819ece0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400d900 tx=0x7fa2c400dcc0 comp rx=0 tx=0).stop 2024-04-23T02:38:56.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.352+0000 7fa2df047700 1 -- 172.21.15.71:0/1695740918 shutdown_connections 2024-04-23T02:38:56.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.352+0000 7fa2df047700 1 --2- 172.21.15.71:0/1695740918 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa2c806cce0 0x7fa2c806f1a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:56.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.352+0000 7fa2df047700 1 --2- 172.21.15.71:0/1695740918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2d80ffda0 0x7fa2d819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:56.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.352+0000 7fa2df047700 1 --2- 172.21.15.71:0/1695740918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2d8100750 0x7fa2d819ece0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:56.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.352+0000 7fa2df047700 1 -- 172.21.15.71:0/1695740918 >> 172.21.15.71:0/1695740918 conn(0x7fa2d80fb650 msgr2=0x7fa2d8103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:56.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.352+0000 7fa2df047700 1 -- 172.21.15.71:0/1695740918 shutdown_connections 2024-04-23T02:38:56.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.353+0000 7fa2df047700 1 -- 172.21.15.71:0/1695740918 wait complete. 2024-04-23T02:38:56.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.456+0000 7fad9ba67700 1 -- 172.21.15.71:0/1669702883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad940fff50 msgr2=0x7fad94105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:56.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.456+0000 7fad9ba67700 1 --2- 172.21.15.71:0/1669702883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad940fff50 0x7fad94105840 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fad84009910 tx=0x7fad84009c20 comp rx=0 tx=0).stop 2024-04-23T02:38:56.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.456+0000 7fad9ba67700 1 -- 172.21.15.71:0/1669702883 shutdown_connections 2024-04-23T02:38:56.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.456+0000 7fad9ba67700 1 --2- 172.21.15.71:0/1669702883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad940fff50 0x7fad94105840 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:56.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.456+0000 7fad9ba67700 1 --2- 172.21.15.71:0/1669702883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad940ff5a0 0x7fad940ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:56.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.456+0000 7fad9ba67700 1 -- 172.21.15.71:0/1669702883 >> 172.21.15.71:0/1669702883 conn(0x7fad94074e30 msgr2=0x7fad94075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:56.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.456+0000 7fad9ba67700 1 -- 172.21.15.71:0/1669702883 shutdown_connections 2024-04-23T02:38:56.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.456+0000 7fad9ba67700 1 -- 172.21.15.71:0/1669702883 wait complete. 2024-04-23T02:38:56.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.457+0000 7fad9ba67700 1 Processor -- start 2024-04-23T02:38:56.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.457+0000 7fad9ba67700 1 -- start start 2024-04-23T02:38:56.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.457+0000 7fad9ba67700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad940ff5a0 0x7fad9419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:56.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.457+0000 7fad9ba67700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad940fff50 0x7fad9419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:56.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.457+0000 7fad9ba67700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9419d260 con 0x7fad940ff5a0 2024-04-23T02:38:56.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.457+0000 7fad9ba67700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad94196710 con 0x7fad940fff50 2024-04-23T02:38:56.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.457+0000 7fad99803700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad940ff5a0 0x7fad9419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:56.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.457+0000 7fad99803700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad940ff5a0 0x7fad9419c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35226/0 (socket says 172.21.15.71:35226) 2024-04-23T02:38:56.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.457+0000 7fad99803700 1 -- 172.21.15.71:0/2756372246 learned_addr learned my addr 172.21.15.71:0/2756372246 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:38:56.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.457+0000 7fad99803700 1 -- 172.21.15.71:0/2756372246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad940fff50 msgr2=0x7fad9419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:56.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.457+0000 7fad99803700 1 --2- 172.21.15.71:0/2756372246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad940fff50 0x7fad9419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:56.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.457+0000 7fad99803700 1 -- 172.21.15.71:0/2756372246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad840095f0 con 0x7fad940ff5a0 2024-04-23T02:38:56.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.457+0000 7fad99803700 1 --2- 172.21.15.71:0/2756372246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad940ff5a0 0x7fad9419c690 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fad9000d8d0 tx=0x7fad9000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:56.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.457+0000 7fad8affd700 1 -- 172.21.15.71:0/2756372246 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad90009940 con 0x7fad940ff5a0 2024-04-23T02:38:56.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.458+0000 7fad8affd700 1 -- 172.21.15.71:0/2756372246 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fad90010460 con 0x7fad940ff5a0 2024-04-23T02:38:56.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.458+0000 7fad9ba67700 1 -- 172.21.15.71:0/2756372246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad941969f0 con 0x7fad940ff5a0 2024-04-23T02:38:56.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.458+0000 7fad8affd700 1 -- 172.21.15.71:0/2756372246 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad9000f5f0 con 0x7fad940ff5a0 2024-04-23T02:38:56.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.458+0000 7fad9ba67700 1 -- 172.21.15.71:0/2756372246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad94196f40 con 0x7fad940ff5a0 2024-04-23T02:38:56.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.459+0000 7fad9ba67700 1 -- 172.21.15.71:0/2756372246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad94109200 con 0x7fad940ff5a0 2024-04-23T02:38:56.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.459+0000 7fad8affd700 1 -- 172.21.15.71:0/2756372246 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fad900105d0 con 0x7fad940ff5a0 2024-04-23T02:38:56.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.459+0000 7fad8affd700 1 --2- 172.21.15.71:0/2756372246 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fad8006cce0 0x7fad8006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:56.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.460+0000 7fad99002700 1 --2- 172.21.15.71:0/2756372246 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fad8006cce0 0x7fad8006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:56.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.460+0000 7fad8affd700 1 -- 172.21.15.71:0/2756372246 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fad9008c5e0 con 0x7fad940ff5a0 2024-04-23T02:38:56.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.460+0000 7fad99002700 1 --2- 172.21.15.71:0/2756372246 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fad8006cce0 0x7fad8006f1a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fad84006010 tx=0x7fad8400b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:56.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.464+0000 7fad8affd700 1 -- 172.21.15.71:0/2756372246 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fad9005a0e0 con 0x7fad940ff5a0 2024-04-23T02:38:56.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.709+0000 7fad9ba67700 1 -- 172.21.15.71:0/2756372246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fad94197c20 con 0x7fad940ff5a0 2024-04-23T02:38:56.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.709+0000 7fad8affd700 1 -- 172.21.15.71:0/2756372246 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7fad90010880 con 0x7fad940ff5a0 2024-04-23T02:38:56.713 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:38:56.713 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:38:56.713 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:38:56.713 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:38:56.713 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:38:56.713 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:38:56.713 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:38:56.713 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:38:56.713 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:38:56.713 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:38:56.713 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:38:56.714 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:38:56.714 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:38:56.714 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:38:56.714 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:38:56.714 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:38:56.714 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:38:56.714 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:38:56.714 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:38:56.714 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:38:56.714 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:38:56.714 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:38:56.714 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:38:56.715 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:38:56.715 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:38:56.715 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:38:56.715 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:38:56.715 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:38:56.715 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:38:56.715 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:38:56.715 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:38:56.715 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:38:56.715 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:38:56.715 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:38:56.715 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:38:56.716 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:38:56.716 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:38:56.716 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:38:56.716 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:38:56.716 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:38:56.716 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:38:56.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.711+0000 7fad9ba67700 1 -- 172.21.15.71:0/2756372246 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fad8006cce0 msgr2=0x7fad8006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:56.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.711+0000 7fad9ba67700 1 --2- 172.21.15.71:0/2756372246 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fad8006cce0 0x7fad8006f1a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fad84006010 tx=0x7fad8400b540 comp rx=0 tx=0).stop 2024-04-23T02:38:56.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.711+0000 7fad9ba67700 1 -- 172.21.15.71:0/2756372246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad940ff5a0 msgr2=0x7fad9419c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:56.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.711+0000 7fad9ba67700 1 --2- 172.21.15.71:0/2756372246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad940ff5a0 0x7fad9419c690 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fad9000d8d0 tx=0x7fad9000dc90 comp rx=0 tx=0).stop 2024-04-23T02:38:56.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.711+0000 7fad9ba67700 1 -- 172.21.15.71:0/2756372246 shutdown_connections 2024-04-23T02:38:56.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.711+0000 7fad9ba67700 1 --2- 172.21.15.71:0/2756372246 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fad8006cce0 0x7fad8006f1a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:56.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.711+0000 7fad9ba67700 1 --2- 172.21.15.71:0/2756372246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad940ff5a0 0x7fad9419c690 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:56.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.711+0000 7fad9ba67700 1 --2- 172.21.15.71:0/2756372246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad940fff50 0x7fad9419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:56.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.711+0000 7fad9ba67700 1 -- 172.21.15.71:0/2756372246 >> 172.21.15.71:0/2756372246 conn(0x7fad94074e30 msgr2=0x7fad940fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:56.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.711+0000 7fad9ba67700 1 -- 172.21.15.71:0/2756372246 shutdown_connections 2024-04-23T02:38:56.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.711+0000 7fad9ba67700 1 -- 172.21.15.71:0/2756372246 wait complete. 2024-04-23T02:38:56.717 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:38:56.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:56 smithi071 ceph-mon[144544]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:38:56.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:56 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1695740918' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:38:56.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.815+0000 7f5e85ef5700 1 -- 172.21.15.71:0/2612486899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e80068580 msgr2=0x7f5e800689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:56.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.815+0000 7f5e85ef5700 1 --2- 172.21.15.71:0/2612486899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e80068580 0x7f5e800689e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5e68009910 tx=0x7f5e68009c20 comp rx=0 tx=0).stop 2024-04-23T02:38:56.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.815+0000 7f5e85ef5700 1 -- 172.21.15.71:0/2612486899 shutdown_connections 2024-04-23T02:38:56.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.815+0000 7f5e85ef5700 1 --2- 172.21.15.71:0/2612486899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e80068580 0x7f5e800689e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:56.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.815+0000 7f5e85ef5700 1 --2- 172.21.15.71:0/2612486899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e80106cd0 0x7f5e801070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:56.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.815+0000 7f5e85ef5700 1 -- 172.21.15.71:0/2612486899 >> 172.21.15.71:0/2612486899 conn(0x7f5e80075670 msgr2=0x7f5e80075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:56.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.815+0000 7f5e85ef5700 1 -- 172.21.15.71:0/2612486899 shutdown_connections 2024-04-23T02:38:56.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.815+0000 7f5e85ef5700 1 -- 172.21.15.71:0/2612486899 wait complete. 2024-04-23T02:38:56.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.815+0000 7f5e85ef5700 1 Processor -- start 2024-04-23T02:38:56.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.815+0000 7f5e85ef5700 1 -- start start 2024-04-23T02:38:56.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.816+0000 7f5e85ef5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e80068580 0x7f5e800ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:56.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.816+0000 7f5e85ef5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e80106cd0 0x7f5e80100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:56.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.816+0000 7f5e85ef5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e80100770 con 0x7f5e80106cd0 2024-04-23T02:38:56.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.816+0000 7f5e85ef5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e801008b0 con 0x7f5e80068580 2024-04-23T02:38:56.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.816+0000 7f5e76ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e80106cd0 0x7f5e80100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:56.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.816+0000 7f5e76ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e80106cd0 0x7f5e80100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35252/0 (socket says 172.21.15.71:35252) 2024-04-23T02:38:56.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.816+0000 7f5e76ffd700 1 -- 172.21.15.71:0/1107170191 learned_addr learned my addr 172.21.15.71:0/1107170191 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:38:56.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.816+0000 7f5e76ffd700 1 -- 172.21.15.71:0/1107170191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e80068580 msgr2=0x7f5e800ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:56.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.816+0000 7f5e7f7fe700 1 --2- 172.21.15.71:0/1107170191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e80068580 0x7f5e800ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:56.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.816+0000 7f5e76ffd700 1 --2- 172.21.15.71:0/1107170191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e80068580 0x7f5e800ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:56.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.816+0000 7f5e76ffd700 1 -- 172.21.15.71:0/1107170191 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e680095f0 con 0x7f5e80106cd0 2024-04-23T02:38:56.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.816+0000 7f5e7f7fe700 1 --2- 172.21.15.71:0/1107170191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e80068580 0x7f5e800ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:38:56.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.816+0000 7f5e76ffd700 1 --2- 172.21.15.71:0/1107170191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e80106cd0 0x7f5e80100230 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5e68006010 tx=0x7f5e6800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:56.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.816+0000 7f5e7d7fa700 1 -- 172.21.15.71:0/1107170191 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e6801d070 con 0x7f5e80106cd0 2024-04-23T02:38:56.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.816+0000 7f5e7d7fa700 1 -- 172.21.15.71:0/1107170191 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5e6800f460 con 0x7f5e80106cd0 2024-04-23T02:38:56.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.816+0000 7f5e7d7fa700 1 -- 172.21.15.71:0/1107170191 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e68021640 con 0x7f5e80106cd0 2024-04-23T02:38:56.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.816+0000 7f5e85ef5700 1 -- 172.21.15.71:0/1107170191 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e80100b30 con 0x7f5e80106cd0 2024-04-23T02:38:56.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.816+0000 7f5e85ef5700 1 -- 172.21.15.71:0/1107170191 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e801a2df0 con 0x7f5e80106cd0 2024-04-23T02:38:56.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.817+0000 7f5e85ef5700 1 -- 172.21.15.71:0/1107170191 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e801091f0 con 0x7f5e80106cd0 2024-04-23T02:38:56.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.818+0000 7f5e7d7fa700 1 -- 172.21.15.71:0/1107170191 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5e6800fa20 con 0x7f5e80106cd0 2024-04-23T02:38:56.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.818+0000 7f5e7d7fa700 1 --2- 172.21.15.71:0/1107170191 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5e6c06cce0 0x7f5e6c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:56.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.818+0000 7f5e7f7fe700 1 --2- 172.21.15.71:0/1107170191 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5e6c06cce0 0x7f5e6c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:56.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.818+0000 7f5e7d7fa700 1 -- 172.21.15.71:0/1107170191 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5e6808d7b0 con 0x7f5e80106cd0 2024-04-23T02:38:56.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.819+0000 7f5e7f7fe700 1 --2- 172.21.15.71:0/1107170191 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5e6c06cce0 0x7f5e6c06f1a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5e70007900 tx=0x7f5e70008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:56.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:56.823+0000 7f5e7d7fa700 1 -- 172.21.15.71:0/1107170191 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5e680578a0 con 0x7f5e80106cd0 2024-04-23T02:38:56.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:56 smithi192 ceph-mon[108617]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:38:56.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:56 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1695740918' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:38:57.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.033+0000 7f5e85ef5700 1 -- 172.21.15.71:0/1107170191 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e800611d0 con 0x7f5e6c06cce0 2024-04-23T02:38:57.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.034+0000 7f5e7d7fa700 1 -- 172.21.15.71:0/1107170191 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5e800611d0 con 0x7f5e6c06cce0 2024-04-23T02:38:57.038 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:38:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:38:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:38:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:38:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:38:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:38:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:38:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:38:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:38:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:38:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:38:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:38:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:38:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:38:57.039 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:38:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.036+0000 7f5e85ef5700 1 -- 172.21.15.71:0/1107170191 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5e6c06cce0 msgr2=0x7f5e6c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.036+0000 7f5e85ef5700 1 --2- 172.21.15.71:0/1107170191 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5e6c06cce0 0x7f5e6c06f1a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5e70007900 tx=0x7f5e70008040 comp rx=0 tx=0).stop 2024-04-23T02:38:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.036+0000 7f5e85ef5700 1 -- 172.21.15.71:0/1107170191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e80106cd0 msgr2=0x7f5e80100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.036+0000 7f5e85ef5700 1 --2- 172.21.15.71:0/1107170191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e80106cd0 0x7f5e80100230 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5e68006010 tx=0x7f5e6800be20 comp rx=0 tx=0).stop 2024-04-23T02:38:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.036+0000 7f5e85ef5700 1 -- 172.21.15.71:0/1107170191 shutdown_connections 2024-04-23T02:38:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.036+0000 7f5e85ef5700 1 --2- 172.21.15.71:0/1107170191 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5e6c06cce0 0x7f5e6c06f1a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.036+0000 7f5e85ef5700 1 --2- 172.21.15.71:0/1107170191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e80068580 0x7f5e800ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:57.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.036+0000 7f5e85ef5700 1 --2- 172.21.15.71:0/1107170191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e80106cd0 0x7f5e80100230 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:57.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.036+0000 7f5e85ef5700 1 -- 172.21.15.71:0/1107170191 >> 172.21.15.71:0/1107170191 conn(0x7f5e80075670 msgr2=0x7f5e800fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:57.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.036+0000 7f5e85ef5700 1 -- 172.21.15.71:0/1107170191 shutdown_connections 2024-04-23T02:38:57.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.036+0000 7f5e85ef5700 1 -- 172.21.15.71:0/1107170191 wait complete. 2024-04-23T02:38:57.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.138+0000 7ff2847a7700 1 -- 172.21.15.71:0/2920153216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff27c100750 msgr2=0x7ff27c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:57.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.138+0000 7ff2847a7700 1 --2- 172.21.15.71:0/2920153216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff27c100750 0x7ff27c1047a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff278009910 tx=0x7ff278009c20 comp rx=0 tx=0).stop 2024-04-23T02:38:57.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.138+0000 7ff2847a7700 1 -- 172.21.15.71:0/2920153216 shutdown_connections 2024-04-23T02:38:57.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.138+0000 7ff2847a7700 1 --2- 172.21.15.71:0/2920153216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff27c100750 0x7ff27c1047a0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:57.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.138+0000 7ff2847a7700 1 --2- 172.21.15.71:0/2920153216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff27c0ffda0 0x7ff27c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:57.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.138+0000 7ff2847a7700 1 -- 172.21.15.71:0/2920153216 >> 172.21.15.71:0/2920153216 conn(0x7ff27c0fb650 msgr2=0x7ff27c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:57.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.138+0000 7ff2847a7700 1 -- 172.21.15.71:0/2920153216 shutdown_connections 2024-04-23T02:38:57.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.138+0000 7ff2847a7700 1 -- 172.21.15.71:0/2920153216 wait complete. 2024-04-23T02:38:57.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.138+0000 7ff2847a7700 1 Processor -- start 2024-04-23T02:38:57.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.138+0000 7ff2847a7700 1 -- start start 2024-04-23T02:38:57.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.138+0000 7ff2847a7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff27c0ffda0 0x7ff27c19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:57.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.138+0000 7ff2847a7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff27c100750 0x7ff27c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:57.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.139+0000 7ff2847a7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c19d280 con 0x7ff27c100750 2024-04-23T02:38:57.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.139+0000 7ff2847a7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c196730 con 0x7ff27c0ffda0 2024-04-23T02:38:57.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.139+0000 7ff281d42700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff27c100750 0x7ff27c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:57.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.139+0000 7ff281d42700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff27c100750 0x7ff27c19cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35262/0 (socket says 172.21.15.71:35262) 2024-04-23T02:38:57.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.139+0000 7ff281d42700 1 -- 172.21.15.71:0/236823979 learned_addr learned my addr 172.21.15.71:0/236823979 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:38:57.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.139+0000 7ff282543700 1 --2- 172.21.15.71:0/236823979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff27c0ffda0 0x7ff27c19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:57.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.139+0000 7ff281d42700 1 -- 172.21.15.71:0/236823979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff27c0ffda0 msgr2=0x7ff27c19c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:57.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.139+0000 7ff281d42700 1 --2- 172.21.15.71:0/236823979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff27c0ffda0 0x7ff27c19c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:57.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.139+0000 7ff281d42700 1 -- 172.21.15.71:0/236823979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2780095f0 con 0x7ff27c100750 2024-04-23T02:38:57.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.139+0000 7ff281d42700 1 --2- 172.21.15.71:0/236823979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff27c100750 0x7ff27c19cbf0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff2780092e0 tx=0x7ff27800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:57.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.139+0000 7ff2737fe700 1 -- 172.21.15.71:0/236823979 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff27801d070 con 0x7ff27c100750 2024-04-23T02:38:57.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.139+0000 7ff2737fe700 1 -- 172.21.15.71:0/236823979 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff27800f460 con 0x7ff27c100750 2024-04-23T02:38:57.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.139+0000 7ff2737fe700 1 -- 172.21.15.71:0/236823979 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff278021600 con 0x7ff27c100750 2024-04-23T02:38:57.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.139+0000 7ff2847a7700 1 -- 172.21.15.71:0/236823979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27c1969b0 con 0x7ff27c100750 2024-04-23T02:38:57.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.139+0000 7ff2847a7700 1 -- 172.21.15.71:0/236823979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27c196ea0 con 0x7ff27c100750 2024-04-23T02:38:57.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.140+0000 7ff2847a7700 1 -- 172.21.15.71:0/236823979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff27c04f2e0 con 0x7ff27c100750 2024-04-23T02:38:57.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.141+0000 7ff2737fe700 1 -- 172.21.15.71:0/236823979 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff27800f5d0 con 0x7ff27c100750 2024-04-23T02:38:57.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.142+0000 7ff2737fe700 1 --2- 172.21.15.71:0/236823979 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff26806cce0 0x7ff26806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:38:57.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.142+0000 7ff2737fe700 1 -- 172.21.15.71:0/236823979 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff27808dc60 con 0x7ff27c100750 2024-04-23T02:38:57.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.142+0000 7ff282543700 1 --2- 172.21.15.71:0/236823979 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff26806cce0 0x7ff26806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:38:57.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.142+0000 7ff282543700 1 --2- 172.21.15.71:0/236823979 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff26806cce0 0x7ff26806f1a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff26c006fd0 tx=0x7ff26c008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:38:57.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.146+0000 7ff2737fe700 1 -- 172.21.15.71:0/236823979 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff27805b740 con 0x7ff27c100750 2024-04-23T02:38:57.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.420+0000 7ff2847a7700 1 -- 172.21.15.71:0/236823979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff27c197c40 con 0x7ff27c100750 2024-04-23T02:38:57.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.420+0000 7ff2737fe700 1 -- 172.21.15.71:0/236823979 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7ff27805b2d0 con 0x7ff27c100750 2024-04-23T02:38:57.424 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:38:57.424 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:38:57.424 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:38:57.424 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:38:57.424 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:38:57.424 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:38:57.424 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:38:57.425 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:38:57.425 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:38:57.425 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:38:57.425 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:38:57.425 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:38:57.425 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:38:57.425 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:38:57.425 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:38:57.425 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:38:57.425 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:38:57.425 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:38:57.425 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:38:57.426 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:38:57.426 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:38:57.426 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:38:57.426 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:38:57.426 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:38:57.426 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:38:57.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.422+0000 7ff2847a7700 1 -- 172.21.15.71:0/236823979 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff26806cce0 msgr2=0x7ff26806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:57.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.422+0000 7ff2847a7700 1 --2- 172.21.15.71:0/236823979 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff26806cce0 0x7ff26806f1a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff26c006fd0 tx=0x7ff26c008040 comp rx=0 tx=0).stop 2024-04-23T02:38:57.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.422+0000 7ff2847a7700 1 -- 172.21.15.71:0/236823979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff27c100750 msgr2=0x7ff27c19cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:38:57.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.422+0000 7ff2847a7700 1 --2- 172.21.15.71:0/236823979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff27c100750 0x7ff27c19cbf0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff2780092e0 tx=0x7ff27800be20 comp rx=0 tx=0).stop 2024-04-23T02:38:57.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.422+0000 7ff2847a7700 1 -- 172.21.15.71:0/236823979 shutdown_connections 2024-04-23T02:38:57.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.422+0000 7ff2847a7700 1 --2- 172.21.15.71:0/236823979 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff26806cce0 0x7ff26806f1a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:57.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.422+0000 7ff2847a7700 1 --2- 172.21.15.71:0/236823979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff27c0ffda0 0x7ff27c19c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:57.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.422+0000 7ff2847a7700 1 --2- 172.21.15.71:0/236823979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff27c100750 0x7ff27c19cbf0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:38:57.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.422+0000 7ff2847a7700 1 -- 172.21.15.71:0/236823979 >> 172.21.15.71:0/236823979 conn(0x7ff27c0fb650 msgr2=0x7ff27c104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:38:57.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.422+0000 7ff2847a7700 1 -- 172.21.15.71:0/236823979 shutdown_connections 2024-04-23T02:38:57.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:38:57.422+0000 7ff2847a7700 1 -- 172.21.15.71:0/236823979 wait complete. 2024-04-23T02:38:57.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:57 smithi071 ceph-mon[144544]: from='client.35216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:38:57.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:57 smithi071 ceph-mon[144544]: from='client.35220 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:38:57.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:57 smithi071 ceph-mon[144544]: pgmap v770: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:57.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:57 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2756372246' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:38:57.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:57 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/236823979' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:38:57.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:57 smithi192 ceph-mon[108617]: from='client.35216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:38:57.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:57 smithi192 ceph-mon[108617]: from='client.35220 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:38:57.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:57 smithi192 ceph-mon[108617]: pgmap v770: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:57.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:57 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2756372246' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:38:57.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:57 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/236823979' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:38:58.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:58 smithi071 ceph-mon[144544]: from='client.35232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:38:58.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:58 smithi192 ceph-mon[108617]: from='client.35232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:38:59.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:38:59 smithi071 ceph-mon[144544]: pgmap v771: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:38:59.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:38:59 smithi192 ceph-mon[108617]: pgmap v771: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:01.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:01 smithi071 ceph-mon[144544]: pgmap v772: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:01.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:01 smithi192 ceph-mon[108617]: pgmap v772: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:03.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:03 smithi192 ceph-mon[108617]: pgmap v773: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:03.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:03 smithi071 ceph-mon[144544]: pgmap v773: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:05 smithi192 ceph-mon[108617]: pgmap v774: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:39:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:05 smithi071 ceph-mon[144544]: pgmap v774: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:39:07.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:07 smithi192 ceph-mon[108617]: pgmap v775: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:07.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:07 smithi071 ceph-mon[144544]: pgmap v775: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:09.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:09 smithi192 ceph-mon[108617]: pgmap v776: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:09.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:09 smithi071 ceph-mon[144544]: pgmap v776: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:11.777 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:11 smithi071 ceph-mon[144544]: pgmap v777: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:11.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:11 smithi192 ceph-mon[108617]: pgmap v777: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:13.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:13 smithi192 ceph-mon[108617]: pgmap v778: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:13.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:13 smithi071 ceph-mon[144544]: pgmap v778: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:15.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:15 smithi192 ceph-mon[108617]: pgmap v779: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:15.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:15 smithi071 ceph-mon[144544]: pgmap v779: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:17.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:17 smithi192 ceph-mon[108617]: pgmap v780: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:17.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:17 smithi071 ceph-mon[144544]: pgmap v780: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:19.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:19 smithi192 ceph-mon[108617]: pgmap v781: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:19.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:19 smithi071 ceph-mon[144544]: pgmap v781: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:20.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:39:20.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:39:21.817 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:21 smithi192 ceph-mon[108617]: pgmap v782: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:21.870 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:21 smithi071 ceph-mon[144544]: pgmap v782: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:23.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:23 smithi192 ceph-mon[108617]: pgmap v783: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:23.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:23 smithi071 ceph-mon[144544]: pgmap v783: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:25.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:25 smithi192 ceph-mon[108617]: pgmap v784: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:25.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:25 smithi071 ceph-mon[144544]: pgmap v784: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:27.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.539+0000 7f51d589c700 1 -- 172.21.15.71:0/1032319352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d00ffda0 msgr2=0x7f51d0100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:27.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.539+0000 7f51d589c700 1 --2- 172.21.15.71:0/1032319352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d00ffda0 0x7f51d0100180 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009910 tx=0x7f51c4009c20 comp rx=0 tx=0).stop 2024-04-23T02:39:27.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.539+0000 7f51d589c700 1 -- 172.21.15.71:0/1032319352 shutdown_connections 2024-04-23T02:39:27.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.539+0000 7f51d589c700 1 --2- 172.21.15.71:0/1032319352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51d0100750 0x7f51d01047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:27.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.539+0000 7f51d589c700 1 --2- 172.21.15.71:0/1032319352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d00ffda0 0x7f51d0100180 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:27.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.539+0000 7f51d589c700 1 -- 172.21.15.71:0/1032319352 >> 172.21.15.71:0/1032319352 conn(0x7f51d00fb650 msgr2=0x7f51d00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:39:27.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.539+0000 7f51d589c700 1 -- 172.21.15.71:0/1032319352 shutdown_connections 2024-04-23T02:39:27.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.539+0000 7f51d589c700 1 -- 172.21.15.71:0/1032319352 wait complete. 2024-04-23T02:39:27.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.540+0000 7f51d589c700 1 Processor -- start 2024-04-23T02:39:27.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.540+0000 7f51d589c700 1 -- start start 2024-04-23T02:39:27.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.540+0000 7f51d589c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51d00ffda0 0x7f51d01943c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:27.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.540+0000 7f51d589c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d0100750 0x7f51d0194900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:27.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.540+0000 7f51d589c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d01964e0 con 0x7f51d0100750 2024-04-23T02:39:27.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.540+0000 7f51d589c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d0194e40 con 0x7f51d00ffda0 2024-04-23T02:39:27.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.540+0000 7f51ce7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d0100750 0x7f51d0194900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:27.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.540+0000 7f51ce7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d0100750 0x7f51d0194900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56042/0 (socket says 172.21.15.71:56042) 2024-04-23T02:39:27.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.540+0000 7f51ce7fc700 1 -- 172.21.15.71:0/91259447 learned_addr learned my addr 172.21.15.71:0/91259447 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:39:27.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.541+0000 7f51ceffd700 1 --2- 172.21.15.71:0/91259447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51d00ffda0 0x7f51d01943c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:27.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.541+0000 7f51ce7fc700 1 -- 172.21.15.71:0/91259447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51d00ffda0 msgr2=0x7f51d01943c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:27.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.541+0000 7f51ce7fc700 1 --2- 172.21.15.71:0/91259447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51d00ffda0 0x7f51d01943c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:27.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.541+0000 7f51ce7fc700 1 -- 172.21.15.71:0/91259447 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c40095f0 con 0x7f51d0100750 2024-04-23T02:39:27.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.541+0000 7f51ce7fc700 1 --2- 172.21.15.71:0/91259447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d0100750 0x7f51d0194900 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f51c000e8f0 tx=0x7f51c000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:39:27.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.541+0000 7f51d489a700 1 -- 172.21.15.71:0/91259447 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51c0009790 con 0x7f51d0100750 2024-04-23T02:39:27.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.541+0000 7f51d489a700 1 -- 172.21.15.71:0/91259447 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f51c000c810 con 0x7f51d0100750 2024-04-23T02:39:27.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.541+0000 7f51d489a700 1 -- 172.21.15.71:0/91259447 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51c0005520 con 0x7f51d0100750 2024-04-23T02:39:27.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.541+0000 7f51d589c700 1 -- 172.21.15.71:0/91259447 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d0195120 con 0x7f51d0100750 2024-04-23T02:39:27.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.541+0000 7f51d589c700 1 -- 172.21.15.71:0/91259447 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d01a7190 con 0x7f51d0100750 2024-04-23T02:39:27.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.542+0000 7f51d589c700 1 -- 172.21.15.71:0/91259447 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d004f2e0 con 0x7f51d0100750 2024-04-23T02:39:27.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.543+0000 7f51d489a700 1 -- 172.21.15.71:0/91259447 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f51c000c980 con 0x7f51d0100750 2024-04-23T02:39:27.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.543+0000 7f51d489a700 1 --2- 172.21.15.71:0/91259447 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f51b806cce0 0x7f51b806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:27.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.543+0000 7f51d489a700 1 -- 172.21.15.71:0/91259447 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f51c0014070 con 0x7f51d0100750 2024-04-23T02:39:27.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.543+0000 7f51ceffd700 1 --2- 172.21.15.71:0/91259447 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f51b806cce0 0x7f51b806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:27.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.543+0000 7f51ceffd700 1 --2- 172.21.15.71:0/91259447 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f51b806cce0 0x7f51b806f1a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f51c400b6e0 tx=0x7f51c4005a90 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:39:27.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.548+0000 7f51d489a700 1 -- 172.21.15.71:0/91259447 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f51c00568a0 con 0x7f51d0100750 2024-04-23T02:39:27.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.761+0000 7f51d589c700 1 -- 172.21.15.71:0/91259447 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f51d0103780 con 0x7f51b806cce0 2024-04-23T02:39:27.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.762+0000 7f51d489a700 1 -- 172.21.15.71:0/91259447 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f51d0103780 con 0x7f51b806cce0 2024-04-23T02:39:27.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.764+0000 7f51d589c700 1 -- 172.21.15.71:0/91259447 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f51b806cce0 msgr2=0x7f51b806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:27.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.764+0000 7f51d589c700 1 --2- 172.21.15.71:0/91259447 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f51b806cce0 0x7f51b806f1a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f51c400b6e0 tx=0x7f51c4005a90 comp rx=0 tx=0).stop 2024-04-23T02:39:27.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.764+0000 7f51d589c700 1 -- 172.21.15.71:0/91259447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d0100750 msgr2=0x7f51d0194900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:27.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.764+0000 7f51d589c700 1 --2- 172.21.15.71:0/91259447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d0100750 0x7f51d0194900 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f51c000e8f0 tx=0x7f51c000ec00 comp rx=0 tx=0).stop 2024-04-23T02:39:27.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.764+0000 7f51d589c700 1 -- 172.21.15.71:0/91259447 shutdown_connections 2024-04-23T02:39:27.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.764+0000 7f51d589c700 1 --2- 172.21.15.71:0/91259447 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f51b806cce0 0x7f51b806f1a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:27.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.764+0000 7f51d589c700 1 --2- 172.21.15.71:0/91259447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51d00ffda0 0x7f51d01943c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:27.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.764+0000 7f51d589c700 1 --2- 172.21.15.71:0/91259447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d0100750 0x7f51d0194900 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:27.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.764+0000 7f51d589c700 1 -- 172.21.15.71:0/91259447 >> 172.21.15.71:0/91259447 conn(0x7f51d00fb650 msgr2=0x7f51d00fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:39:27.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.764+0000 7f51d589c700 1 -- 172.21.15.71:0/91259447 shutdown_connections 2024-04-23T02:39:27.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.764+0000 7f51d589c700 1 -- 172.21.15.71:0/91259447 wait complete. 2024-04-23T02:39:27.777 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:39:27.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.868+0000 7f543c5cf700 1 -- 172.21.15.71:0/642371049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54340ffda0 msgr2=0x7f5434100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:27.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.868+0000 7f543c5cf700 1 --2- 172.21.15.71:0/642371049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54340ffda0 0x7f5434100180 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f542400b0a0 tx=0x7f542400b3b0 comp rx=0 tx=0).stop 2024-04-23T02:39:27.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.868+0000 7f543c5cf700 1 -- 172.21.15.71:0/642371049 shutdown_connections 2024-04-23T02:39:27.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.868+0000 7f543c5cf700 1 --2- 172.21.15.71:0/642371049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5434100750 0x7f54341047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:27.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.868+0000 7f543c5cf700 1 --2- 172.21.15.71:0/642371049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54340ffda0 0x7f5434100180 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:27.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.868+0000 7f543c5cf700 1 -- 172.21.15.71:0/642371049 >> 172.21.15.71:0/642371049 conn(0x7f54340fb650 msgr2=0x7f54340fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:39:27.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.868+0000 7f543c5cf700 1 -- 172.21.15.71:0/642371049 shutdown_connections 2024-04-23T02:39:27.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.868+0000 7f543c5cf700 1 -- 172.21.15.71:0/642371049 wait complete. 2024-04-23T02:39:27.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.869+0000 7f543c5cf700 1 Processor -- start 2024-04-23T02:39:27.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.869+0000 7f543c5cf700 1 -- start start 2024-04-23T02:39:27.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.869+0000 7f543c5cf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54340ffda0 0x7f543419b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:27.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.869+0000 7f543c5cf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5434100750 0x7f543419ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:27.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.869+0000 7f543c5cf700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543419c110 con 0x7f54340ffda0 2024-04-23T02:39:27.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.869+0000 7f543c5cf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54341955c0 con 0x7f5434100750 2024-04-23T02:39:27.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.869+0000 7f543a36b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54340ffda0 0x7f543419b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:27.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.869+0000 7f543a36b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54340ffda0 0x7f543419b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56074/0 (socket says 172.21.15.71:56074) 2024-04-23T02:39:27.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.869+0000 7f543a36b700 1 -- 172.21.15.71:0/4063797673 learned_addr learned my addr 172.21.15.71:0/4063797673 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:39:27.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.869+0000 7f5439b6a700 1 --2- 172.21.15.71:0/4063797673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5434100750 0x7f543419ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:27.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.869+0000 7f543a36b700 1 -- 172.21.15.71:0/4063797673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5434100750 msgr2=0x7f543419ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:27.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.869+0000 7f543a36b700 1 --2- 172.21.15.71:0/4063797673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5434100750 0x7f543419ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:27.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.869+0000 7f543a36b700 1 -- 172.21.15.71:0/4063797673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5424009d00 con 0x7f54340ffda0 2024-04-23T02:39:27.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.870+0000 7f543a36b700 1 --2- 172.21.15.71:0/4063797673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54340ffda0 0x7f543419b540 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5424015040 tx=0x7f5424007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:39:27.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.870+0000 7f54337fe700 1 -- 172.21.15.71:0/4063797673 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f542400e040 con 0x7f54340ffda0 2024-04-23T02:39:27.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.870+0000 7f54337fe700 1 -- 172.21.15.71:0/4063797673 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f54240044e0 con 0x7f54340ffda0 2024-04-23T02:39:27.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.870+0000 7f54337fe700 1 -- 172.21.15.71:0/4063797673 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f542401d9e0 con 0x7f54340ffda0 2024-04-23T02:39:27.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.870+0000 7f543c5cf700 1 -- 172.21.15.71:0/4063797673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5434195840 con 0x7f54340ffda0 2024-04-23T02:39:27.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.870+0000 7f543c5cf700 1 -- 172.21.15.71:0/4063797673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5434195d30 con 0x7f54340ffda0 2024-04-23T02:39:27.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.871+0000 7f543c5cf700 1 -- 172.21.15.71:0/4063797673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5434108140 con 0x7f54340ffda0 2024-04-23T02:39:27.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:27 smithi192 ceph-mon[108617]: pgmap v785: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:27.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.872+0000 7f54337fe700 1 -- 172.21.15.71:0/4063797673 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f542401dbe0 con 0x7f54340ffda0 2024-04-23T02:39:27.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.872+0000 7f54337fe700 1 --2- 172.21.15.71:0/4063797673 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f541c06cce0 0x7f541c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:27.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.873+0000 7f54337fe700 1 -- 172.21.15.71:0/4063797673 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f542408e460 con 0x7f54340ffda0 2024-04-23T02:39:27.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.873+0000 7f5439b6a700 1 --2- 172.21.15.71:0/4063797673 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f541c06cce0 0x7f541c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:27.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.873+0000 7f5439b6a700 1 --2- 172.21.15.71:0/4063797673 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f541c06cce0 0x7f541c06f1a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5428005fd0 tx=0x7f542800a560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:39:27.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:27.876+0000 7f54337fe700 1 -- 172.21.15.71:0/4063797673 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f542405bf40 con 0x7f54340ffda0 2024-04-23T02:39:27.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:27 smithi071 ceph-mon[144544]: pgmap v785: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:28.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.089+0000 7f543c5cf700 1 -- 172.21.15.71:0/4063797673 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5434195f50 con 0x7f541c06cce0 2024-04-23T02:39:28.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.090+0000 7f54337fe700 1 -- 172.21.15.71:0/4063797673 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5434195f50 con 0x7f541c06cce0 2024-04-23T02:39:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.092+0000 7f543c5cf700 1 -- 172.21.15.71:0/4063797673 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f541c06cce0 msgr2=0x7f541c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.092+0000 7f543c5cf700 1 --2- 172.21.15.71:0/4063797673 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f541c06cce0 0x7f541c06f1a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5428005fd0 tx=0x7f542800a560 comp rx=0 tx=0).stop 2024-04-23T02:39:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.092+0000 7f543c5cf700 1 -- 172.21.15.71:0/4063797673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54340ffda0 msgr2=0x7f543419b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.092+0000 7f543c5cf700 1 --2- 172.21.15.71:0/4063797673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54340ffda0 0x7f543419b540 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5424015040 tx=0x7f5424007be0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.092+0000 7f543c5cf700 1 -- 172.21.15.71:0/4063797673 shutdown_connections 2024-04-23T02:39:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.092+0000 7f543c5cf700 1 --2- 172.21.15.71:0/4063797673 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f541c06cce0 0x7f541c06f1a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.092+0000 7f543c5cf700 1 --2- 172.21.15.71:0/4063797673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54340ffda0 0x7f543419b540 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.092+0000 7f543c5cf700 1 --2- 172.21.15.71:0/4063797673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5434100750 0x7f543419ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.092+0000 7f543c5cf700 1 -- 172.21.15.71:0/4063797673 >> 172.21.15.71:0/4063797673 conn(0x7f54340fb650 msgr2=0x7f54340fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:39:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.092+0000 7f543c5cf700 1 -- 172.21.15.71:0/4063797673 shutdown_connections 2024-04-23T02:39:28.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.093+0000 7f543c5cf700 1 -- 172.21.15.71:0/4063797673 wait complete. 2024-04-23T02:39:28.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.195+0000 7f1c94239700 1 -- 172.21.15.71:0/2646684965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c8c101840 msgr2=0x7f1c8c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:28.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.195+0000 7f1c94239700 1 --2- 172.21.15.71:0/2646684965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c8c101840 0x7f1c8c105890 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009910 tx=0x7f1c7c009c20 comp rx=0 tx=0).stop 2024-04-23T02:39:28.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.195+0000 7f1c94239700 1 -- 172.21.15.71:0/2646684965 shutdown_connections 2024-04-23T02:39:28.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.195+0000 7f1c94239700 1 --2- 172.21.15.71:0/2646684965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c8c101840 0x7f1c8c105890 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.195+0000 7f1c94239700 1 --2- 172.21.15.71:0/2646684965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c8c100e90 0x7f1c8c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.195+0000 7f1c94239700 1 -- 172.21.15.71:0/2646684965 >> 172.21.15.71:0/2646684965 conn(0x7f1c8c0fc740 msgr2=0x7f1c8c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:39:28.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.195+0000 7f1c94239700 1 -- 172.21.15.71:0/2646684965 shutdown_connections 2024-04-23T02:39:28.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.195+0000 7f1c94239700 1 -- 172.21.15.71:0/2646684965 wait complete. 2024-04-23T02:39:28.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.195+0000 7f1c94239700 1 Processor -- start 2024-04-23T02:39:28.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.195+0000 7f1c94239700 1 -- start start 2024-04-23T02:39:28.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.196+0000 7f1c94239700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c8c100e90 0x7f1c8c19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:28.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.196+0000 7f1c94239700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c8c101840 0x7f1c8c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:28.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.196+0000 7f1c94239700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c19d200 con 0x7f1c8c101840 2024-04-23T02:39:28.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.196+0000 7f1c94239700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8c1966b0 con 0x7f1c8c100e90 2024-04-23T02:39:28.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.196+0000 7f1c917d4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c8c101840 0x7f1c8c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:28.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.196+0000 7f1c917d4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c8c101840 0x7f1c8c19cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56090/0 (socket says 172.21.15.71:56090) 2024-04-23T02:39:28.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.196+0000 7f1c917d4700 1 -- 172.21.15.71:0/416490082 learned_addr learned my addr 172.21.15.71:0/416490082 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:39:28.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.196+0000 7f1c91fd5700 1 --2- 172.21.15.71:0/416490082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c8c100e90 0x7f1c8c19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:28.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.196+0000 7f1c917d4700 1 -- 172.21.15.71:0/416490082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c8c100e90 msgr2=0x7f1c8c19c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:28.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.196+0000 7f1c917d4700 1 --2- 172.21.15.71:0/416490082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c8c100e90 0x7f1c8c19c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.196+0000 7f1c917d4700 1 -- 172.21.15.71:0/416490082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c0095f0 con 0x7f1c8c101840 2024-04-23T02:39:28.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.196+0000 7f1c917d4700 1 --2- 172.21.15.71:0/416490082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c8c101840 0x7f1c8c19cb70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00bd70 tx=0x7f1c7c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:39:28.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.196+0000 7f1c82ffd700 1 -- 172.21.15.71:0/416490082 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c7c01d070 con 0x7f1c8c101840 2024-04-23T02:39:28.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.196+0000 7f1c82ffd700 1 -- 172.21.15.71:0/416490082 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1c7c00f460 con 0x7f1c8c101840 2024-04-23T02:39:28.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.196+0000 7f1c94239700 1 -- 172.21.15.71:0/416490082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8c196930 con 0x7f1c8c101840 2024-04-23T02:39:28.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.196+0000 7f1c82ffd700 1 -- 172.21.15.71:0/416490082 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c7c021640 con 0x7f1c8c101840 2024-04-23T02:39:28.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.196+0000 7f1c94239700 1 -- 172.21.15.71:0/416490082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8c196e20 con 0x7f1c8c101840 2024-04-23T02:39:28.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.197+0000 7f1c94239700 1 -- 172.21.15.71:0/416490082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c8c109290 con 0x7f1c8c101840 2024-04-23T02:39:28.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.198+0000 7f1c82ffd700 1 -- 172.21.15.71:0/416490082 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1c7c00f9e0 con 0x7f1c8c101840 2024-04-23T02:39:28.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.198+0000 7f1c82ffd700 1 --2- 172.21.15.71:0/416490082 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1c7806cce0 0x7f1c7806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:28.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.198+0000 7f1c82ffd700 1 -- 172.21.15.71:0/416490082 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1c7c08d820 con 0x7f1c8c101840 2024-04-23T02:39:28.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.198+0000 7f1c91fd5700 1 --2- 172.21.15.71:0/416490082 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1c7806cce0 0x7f1c7806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:28.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.199+0000 7f1c91fd5700 1 --2- 172.21.15.71:0/416490082 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1c7806cce0 0x7f1c7806f1a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009af0 tx=0x7f1c880094f0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:39:28.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.203+0000 7f1c82ffd700 1 -- 172.21.15.71:0/416490082 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1c7c05b2c0 con 0x7f1c8c101840 2024-04-23T02:39:28.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.402+0000 7f1c94239700 1 -- 172.21.15.71:0/416490082 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c8c197cb0 con 0x7f1c7806cce0 2024-04-23T02:39:28.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.413+0000 7f1c82ffd700 1 -- 172.21.15.71:0/416490082 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1c8c197cb0 con 0x7f1c7806cce0 2024-04-23T02:39:28.416 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:39:28.416 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (23m) 85s ago 34m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:39:28.416 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (34m) 85s ago 34m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:39:28.416 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (33m) 87s ago 33m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:39:28.416 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (21m) 85s ago 34m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:39:28.416 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (20m) 87s ago 33m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:39:28.416 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (22m) 85s ago 34m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:39:28.416 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (17m) 85s ago 28m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:39:28.416 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (16m) 85s ago 28m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:39:28.416 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (91s) 87s ago 28m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:39:28.416 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (16m) 87s ago 28m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:39:28.416 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (25m) 85s ago 36m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:39:28.416 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (24m) 87s ago 33m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:39:28.417 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (21m) 85s ago 36m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:39:28.417 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (21m) 87s ago 33m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:39:28.417 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (24m) 85s ago 34m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:39:28.417 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (23m) 87s ago 33m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:39:28.417 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (20m) 85s ago 32m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:39:28.417 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (20m) 85s ago 32m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:39:28.417 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (19m) 85s ago 31m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:39:28.417 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (18m) 87s ago 31m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:39:28.417 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (18m) 87s ago 30m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:39:28.417 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (17m) 87s ago 30m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:39:28.417 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (23m) 85s ago 33m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:39:28.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.415+0000 7f1c94239700 1 -- 172.21.15.71:0/416490082 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1c7806cce0 msgr2=0x7f1c7806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:28.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.415+0000 7f1c94239700 1 --2- 172.21.15.71:0/416490082 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1c7806cce0 0x7f1c7806f1a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009af0 tx=0x7f1c880094f0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.415+0000 7f1c94239700 1 -- 172.21.15.71:0/416490082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c8c101840 msgr2=0x7f1c8c19cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:28.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.415+0000 7f1c94239700 1 --2- 172.21.15.71:0/416490082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c8c101840 0x7f1c8c19cb70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00bd70 tx=0x7f1c7c00be50 comp rx=0 tx=0).stop 2024-04-23T02:39:28.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.415+0000 7f1c94239700 1 -- 172.21.15.71:0/416490082 shutdown_connections 2024-04-23T02:39:28.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.415+0000 7f1c94239700 1 --2- 172.21.15.71:0/416490082 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1c7806cce0 0x7f1c7806f1a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.415+0000 7f1c94239700 1 --2- 172.21.15.71:0/416490082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c8c100e90 0x7f1c8c19c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.415+0000 7f1c94239700 1 --2- 172.21.15.71:0/416490082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c8c101840 0x7f1c8c19cb70 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.415+0000 7f1c94239700 1 -- 172.21.15.71:0/416490082 >> 172.21.15.71:0/416490082 conn(0x7f1c8c0fc740 msgr2=0x7f1c8c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:39:28.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.415+0000 7f1c94239700 1 -- 172.21.15.71:0/416490082 shutdown_connections 2024-04-23T02:39:28.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.415+0000 7f1c94239700 1 -- 172.21.15.71:0/416490082 wait complete. 2024-04-23T02:39:28.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.517+0000 7f447126d700 1 -- 172.21.15.71:0/534976573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f446c105bd0 msgr2=0x7f446c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:28.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.517+0000 7f447126d700 1 --2- 172.21.15.71:0/534976573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f446c105bd0 0x7f446c105fb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4460009910 tx=0x7f4460009c20 comp rx=0 tx=0).stop 2024-04-23T02:39:28.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.517+0000 7f447126d700 1 -- 172.21.15.71:0/534976573 shutdown_connections 2024-04-23T02:39:28.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.517+0000 7f447126d700 1 --2- 172.21.15.71:0/534976573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f446c0ffc20 0x7f446c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.517+0000 7f447126d700 1 --2- 172.21.15.71:0/534976573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f446c105bd0 0x7f446c105fb0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.517+0000 7f447126d700 1 -- 172.21.15.71:0/534976573 >> 172.21.15.71:0/534976573 conn(0x7f446c075670 msgr2=0x7f446c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:39:28.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.517+0000 7f447126d700 1 -- 172.21.15.71:0/534976573 shutdown_connections 2024-04-23T02:39:28.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.517+0000 7f447126d700 1 -- 172.21.15.71:0/534976573 wait complete. 2024-04-23T02:39:28.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.517+0000 7f447126d700 1 Processor -- start 2024-04-23T02:39:28.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.517+0000 7f447126d700 1 -- start start 2024-04-23T02:39:28.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.518+0000 7f447126d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f446c0ffc20 0x7f446c197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:28.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.518+0000 7f447126d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f446c105bd0 0x7f446c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:28.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.518+0000 7f447126d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c1980d0 con 0x7f446c105bd0 2024-04-23T02:39:28.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.518+0000 7f447126d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c198210 con 0x7f446c0ffc20 2024-04-23T02:39:28.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.518+0000 7f446a7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f446c105bd0 0x7f446c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:28.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.518+0000 7f446a7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f446c105bd0 0x7f446c197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56116/0 (socket says 172.21.15.71:56116) 2024-04-23T02:39:28.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.518+0000 7f446a7fc700 1 -- 172.21.15.71:0/638337524 learned_addr learned my addr 172.21.15.71:0/638337524 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:39:28.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.518+0000 7f446affd700 1 --2- 172.21.15.71:0/638337524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f446c0ffc20 0x7f446c197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:28.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.518+0000 7f446a7fc700 1 -- 172.21.15.71:0/638337524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f446c0ffc20 msgr2=0x7f446c197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:28.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.518+0000 7f446a7fc700 1 --2- 172.21.15.71:0/638337524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f446c0ffc20 0x7f446c197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.518+0000 7f446a7fc700 1 -- 172.21.15.71:0/638337524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44600095f0 con 0x7f446c105bd0 2024-04-23T02:39:28.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.518+0000 7f446a7fc700 1 --2- 172.21.15.71:0/638337524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f446c105bd0 0x7f446c197b40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f445c00c930 tx=0x7f445c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:39:28.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.518+0000 7f4453fff700 1 -- 172.21.15.71:0/638337524 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f445c007ad0 con 0x7f446c105bd0 2024-04-23T02:39:28.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.518+0000 7f4453fff700 1 -- 172.21.15.71:0/638337524 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f445c00ce80 con 0x7f446c105bd0 2024-04-23T02:39:28.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.518+0000 7f447126d700 1 -- 172.21.15.71:0/638337524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446c1a0670 con 0x7f446c105bd0 2024-04-23T02:39:28.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.518+0000 7f4453fff700 1 -- 172.21.15.71:0/638337524 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f445c018630 con 0x7f446c105bd0 2024-04-23T02:39:28.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.518+0000 7f447126d700 1 -- 172.21.15.71:0/638337524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446c1a0ae0 con 0x7f446c105bd0 2024-04-23T02:39:28.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.519+0000 7f447126d700 1 -- 172.21.15.71:0/638337524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f446c1080d0 con 0x7f446c105bd0 2024-04-23T02:39:28.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.520+0000 7f4453fff700 1 -- 172.21.15.71:0/638337524 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f445c007c30 con 0x7f446c105bd0 2024-04-23T02:39:28.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.520+0000 7f4453fff700 1 --2- 172.21.15.71:0/638337524 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f445406cce0 0x7f445406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:28.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.520+0000 7f446affd700 1 --2- 172.21.15.71:0/638337524 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f445406cce0 0x7f445406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:28.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.520+0000 7f4453fff700 1 -- 172.21.15.71:0/638337524 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f445c08c550 con 0x7f446c105bd0 2024-04-23T02:39:28.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.521+0000 7f446affd700 1 --2- 172.21.15.71:0/638337524 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f445406cce0 0x7f445406f1a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4460006010 tx=0x7f446001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:39:28.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.525+0000 7f4453fff700 1 -- 172.21.15.71:0/638337524 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f445c0565a0 con 0x7f446c105bd0 2024-04-23T02:39:28.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.796+0000 7f447126d700 1 -- 172.21.15.71:0/638337524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f446c04f2e0 con 0x7f446c105bd0 2024-04-23T02:39:28.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.797+0000 7f4453fff700 1 -- 172.21.15.71:0/638337524 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f445c059bc0 con 0x7f446c105bd0 2024-04-23T02:39:28.800 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:39:28.800 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:39:28.800 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:39:28.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:39:28.800 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:39:28.800 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:39:28.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:39:28.800 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:39:28.800 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:39:28.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:39:28.800 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:39:28.800 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:39:28.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:39:28.801 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:39:28.801 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:39:28.801 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:39:28.801 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:39:28.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.799+0000 7f447126d700 1 -- 172.21.15.71:0/638337524 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f445406cce0 msgr2=0x7f445406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:28.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.799+0000 7f447126d700 1 --2- 172.21.15.71:0/638337524 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f445406cce0 0x7f445406f1a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4460006010 tx=0x7f446001a040 comp rx=0 tx=0).stop 2024-04-23T02:39:28.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.799+0000 7f447126d700 1 -- 172.21.15.71:0/638337524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f446c105bd0 msgr2=0x7f446c197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:28.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.799+0000 7f447126d700 1 --2- 172.21.15.71:0/638337524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f446c105bd0 0x7f446c197b40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f445c00c930 tx=0x7f445c00cc40 comp rx=0 tx=0).stop 2024-04-23T02:39:28.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.799+0000 7f447126d700 1 -- 172.21.15.71:0/638337524 shutdown_connections 2024-04-23T02:39:28.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.799+0000 7f447126d700 1 --2- 172.21.15.71:0/638337524 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f445406cce0 0x7f445406f1a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.799+0000 7f447126d700 1 --2- 172.21.15.71:0/638337524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f446c0ffc20 0x7f446c197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.799+0000 7f447126d700 1 --2- 172.21.15.71:0/638337524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f446c105bd0 0x7f446c197b40 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.799+0000 7f447126d700 1 -- 172.21.15.71:0/638337524 >> 172.21.15.71:0/638337524 conn(0x7f446c075670 msgr2=0x7f446c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:39:28.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.799+0000 7f447126d700 1 -- 172.21.15.71:0/638337524 shutdown_connections 2024-04-23T02:39:28.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.799+0000 7f447126d700 1 -- 172.21.15.71:0/638337524 wait complete. 2024-04-23T02:39:28.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.901+0000 7fbb2e902700 1 -- 172.21.15.71:0/180327476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb28068580 msgr2=0x7fbb280689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:28.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.901+0000 7fbb2e902700 1 --2- 172.21.15.71:0/180327476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb28068580 0x7fbb280689e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009910 tx=0x7fbb18009c20 comp rx=0 tx=0).stop 2024-04-23T02:39:28.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.901+0000 7fbb2e902700 1 -- 172.21.15.71:0/180327476 shutdown_connections 2024-04-23T02:39:28.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.901+0000 7fbb2e902700 1 --2- 172.21.15.71:0/180327476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb28068580 0x7fbb280689e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.902+0000 7fbb2e902700 1 --2- 172.21.15.71:0/180327476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb28106cd0 0x7fbb281070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.902+0000 7fbb2e902700 1 -- 172.21.15.71:0/180327476 >> 172.21.15.71:0/180327476 conn(0x7fbb28075670 msgr2=0x7fbb28075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:39:28.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.902+0000 7fbb2e902700 1 -- 172.21.15.71:0/180327476 shutdown_connections 2024-04-23T02:39:28.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.902+0000 7fbb2e902700 1 -- 172.21.15.71:0/180327476 wait complete. 2024-04-23T02:39:28.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.902+0000 7fbb2e902700 1 Processor -- start 2024-04-23T02:39:28.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.902+0000 7fbb2e902700 1 -- start start 2024-04-23T02:39:28.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.902+0000 7fbb2e902700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb28068580 0x7fbb280ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:28.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.902+0000 7fbb2e902700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb28106cd0 0x7fbb28100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:28.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.902+0000 7fbb2e902700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb28100770 con 0x7fbb28106cd0 2024-04-23T02:39:28.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.903+0000 7fbb2e902700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb281008b0 con 0x7fbb28068580 2024-04-23T02:39:28.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.903+0000 7fbb277fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb28106cd0 0x7fbb28100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:28.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.903+0000 7fbb277fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb28106cd0 0x7fbb28100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56132/0 (socket says 172.21.15.71:56132) 2024-04-23T02:39:28.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.903+0000 7fbb277fe700 1 -- 172.21.15.71:0/198398450 learned_addr learned my addr 172.21.15.71:0/198398450 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:39:28.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.903+0000 7fbb27fff700 1 --2- 172.21.15.71:0/198398450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb28068580 0x7fbb280ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:28.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.903+0000 7fbb277fe700 1 -- 172.21.15.71:0/198398450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb28068580 msgr2=0x7fbb280ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:28.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.903+0000 7fbb277fe700 1 --2- 172.21.15.71:0/198398450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb28068580 0x7fbb280ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:28.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.903+0000 7fbb277fe700 1 -- 172.21.15.71:0/198398450 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180095f0 con 0x7fbb28106cd0 2024-04-23T02:39:28.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.903+0000 7fbb277fe700 1 --2- 172.21.15.71:0/198398450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb28106cd0 0x7fbb28100230 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800b5c0 tx=0x7fbb1800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:39:28.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.903+0000 7fbb257fa700 1 -- 172.21.15.71:0/198398450 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb1801d070 con 0x7fbb28106cd0 2024-04-23T02:39:28.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.903+0000 7fbb2e902700 1 -- 172.21.15.71:0/198398450 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb28100b30 con 0x7fbb28106cd0 2024-04-23T02:39:28.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.903+0000 7fbb257fa700 1 -- 172.21.15.71:0/198398450 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbb1800f460 con 0x7fbb28106cd0 2024-04-23T02:39:28.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.903+0000 7fbb257fa700 1 -- 172.21.15.71:0/198398450 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb18021640 con 0x7fbb28106cd0 2024-04-23T02:39:28.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.903+0000 7fbb2e902700 1 -- 172.21.15.71:0/198398450 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb281a2df0 con 0x7fbb28106cd0 2024-04-23T02:39:28.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.904+0000 7fbb2e902700 1 -- 172.21.15.71:0/198398450 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb281091f0 con 0x7fbb28106cd0 2024-04-23T02:39:28.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.905+0000 7fbb257fa700 1 -- 172.21.15.71:0/198398450 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbb1800f9b0 con 0x7fbb28106cd0 2024-04-23T02:39:28.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.905+0000 7fbb257fa700 1 --2- 172.21.15.71:0/198398450 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbb1006cce0 0x7fbb1006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:28.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.905+0000 7fbb257fa700 1 -- 172.21.15.71:0/198398450 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbb1808d8c0 con 0x7fbb28106cd0 2024-04-23T02:39:28.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.905+0000 7fbb27fff700 1 --2- 172.21.15.71:0/198398450 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbb1006cce0 0x7fbb1006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:28.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.905+0000 7fbb27fff700 1 --2- 172.21.15.71:0/198398450 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbb1006cce0 0x7fbb1006f1a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c006fd0 tx=0x7fbb1c008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:39:28.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:28.909+0000 7fbb257fa700 1 -- 172.21.15.71:0/198398450 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbb18057930 con 0x7fbb28106cd0 2024-04-23T02:39:29.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.143+0000 7fbb2e902700 1 -- 172.21.15.71:0/198398450 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbb2804f2e0 con 0x7fbb28106cd0 2024-04-23T02:39:29.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.144+0000 7fbb257fa700 1 -- 172.21.15.71:0/198398450 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7fbb1805af50 con 0x7fbb28106cd0 2024-04-23T02:39:29.146 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:39:29.146 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:39:29.147 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:39:29.147 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:39:29.147 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:39:29.147 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:39:29.147 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:39:29.147 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:39:29.147 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:39:29.147 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:39:29.147 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:39:29.147 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:39:29.147 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:39:29.147 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:39:29.147 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:39:29.147 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:39:29.148 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:39:29.148 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:39:29.148 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:39:29.148 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:39:29.148 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:39:29.148 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:39:29.148 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:39:29.148 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:39:29.148 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:39:29.148 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:39:29.148 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:39:29.148 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:39:29.148 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:39:29.148 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:39:29.149 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:39:29.149 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:39:29.149 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:39:29.149 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:39:29.149 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:39:29.149 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:39:29.149 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:39:29.149 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:39:29.149 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:39:29.149 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:39:29.149 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:39:29.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.146+0000 7fbb2e902700 1 -- 172.21.15.71:0/198398450 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbb1006cce0 msgr2=0x7fbb1006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:29.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.146+0000 7fbb2e902700 1 --2- 172.21.15.71:0/198398450 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbb1006cce0 0x7fbb1006f1a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c006fd0 tx=0x7fbb1c008040 comp rx=0 tx=0).stop 2024-04-23T02:39:29.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.146+0000 7fbb2e902700 1 -- 172.21.15.71:0/198398450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb28106cd0 msgr2=0x7fbb28100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:29.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.146+0000 7fbb2e902700 1 --2- 172.21.15.71:0/198398450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb28106cd0 0x7fbb28100230 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800b5c0 tx=0x7fbb1800be20 comp rx=0 tx=0).stop 2024-04-23T02:39:29.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.146+0000 7fbb2e902700 1 -- 172.21.15.71:0/198398450 shutdown_connections 2024-04-23T02:39:29.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.146+0000 7fbb2e902700 1 --2- 172.21.15.71:0/198398450 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbb1006cce0 0x7fbb1006f1a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:29.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.146+0000 7fbb2e902700 1 --2- 172.21.15.71:0/198398450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb28068580 0x7fbb280ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:29.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.146+0000 7fbb2e902700 1 --2- 172.21.15.71:0/198398450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb28106cd0 0x7fbb28100230 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:29.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.146+0000 7fbb2e902700 1 -- 172.21.15.71:0/198398450 >> 172.21.15.71:0/198398450 conn(0x7fbb28075670 msgr2=0x7fbb280fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:39:29.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.146+0000 7fbb2e902700 1 -- 172.21.15.71:0/198398450 shutdown_connections 2024-04-23T02:39:29.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.146+0000 7fbb2e902700 1 -- 172.21.15.71:0/198398450 wait complete. 2024-04-23T02:39:29.151 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:39:29.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.248+0000 7fc969b4c700 1 -- 172.21.15.71:0/2316341759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc964102700 msgr2=0x7fc96410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.248+0000 7fc969b4c700 1 --2- 172.21.15.71:0/2316341759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc964102700 0x7fc96410abf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc958009910 tx=0x7fc958009c20 comp rx=0 tx=0).stop 2024-04-23T02:39:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.248+0000 7fc969b4c700 1 -- 172.21.15.71:0/2316341759 shutdown_connections 2024-04-23T02:39:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.248+0000 7fc969b4c700 1 --2- 172.21.15.71:0/2316341759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc964102700 0x7fc96410abf0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.248+0000 7fc969b4c700 1 --2- 172.21.15.71:0/2316341759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc964101de0 0x7fc9641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.248+0000 7fc969b4c700 1 -- 172.21.15.71:0/2316341759 >> 172.21.15.71:0/2316341759 conn(0x7fc9640fb610 msgr2=0x7fc9640fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:39:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.248+0000 7fc969b4c700 1 -- 172.21.15.71:0/2316341759 shutdown_connections 2024-04-23T02:39:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.248+0000 7fc969b4c700 1 -- 172.21.15.71:0/2316341759 wait complete. 2024-04-23T02:39:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.249+0000 7fc969b4c700 1 Processor -- start 2024-04-23T02:39:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.249+0000 7fc969b4c700 1 -- start start 2024-04-23T02:39:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.249+0000 7fc969b4c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc964101de0 0x7fc96419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.249+0000 7fc969b4c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc964102700 0x7fc96419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.249+0000 7fc969b4c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96419d190 con 0x7fc964101de0 2024-04-23T02:39:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.249+0000 7fc969b4c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc964196640 con 0x7fc964102700 2024-04-23T02:39:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.249+0000 7fc9637fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc964101de0 0x7fc96419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.249+0000 7fc9637fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc964101de0 0x7fc96419c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56146/0 (socket says 172.21.15.71:56146) 2024-04-23T02:39:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.249+0000 7fc9637fe700 1 -- 172.21.15.71:0/3746135762 learned_addr learned my addr 172.21.15.71:0/3746135762 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:39:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.250+0000 7fc962ffd700 1 --2- 172.21.15.71:0/3746135762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc964102700 0x7fc96419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.250+0000 7fc9637fe700 1 -- 172.21.15.71:0/3746135762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc964102700 msgr2=0x7fc96419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.250+0000 7fc9637fe700 1 --2- 172.21.15.71:0/3746135762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc964102700 0x7fc96419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.250+0000 7fc9637fe700 1 -- 172.21.15.71:0/3746135762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9580095f0 con 0x7fc964101de0 2024-04-23T02:39:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.250+0000 7fc9637fe700 1 --2- 172.21.15.71:0/3746135762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc964101de0 0x7fc96419c5c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc95400c8a0 tx=0x7fc95400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:39:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.250+0000 7fc960ff9700 1 -- 172.21.15.71:0/3746135762 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc95400cea0 con 0x7fc964101de0 2024-04-23T02:39:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.250+0000 7fc960ff9700 1 -- 172.21.15.71:0/3746135762 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc954004830 con 0x7fc964101de0 2024-04-23T02:39:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.250+0000 7fc969b4c700 1 -- 172.21.15.71:0/3746135762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc964196920 con 0x7fc964101de0 2024-04-23T02:39:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.250+0000 7fc960ff9700 1 -- 172.21.15.71:0/3746135762 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc954005500 con 0x7fc964101de0 2024-04-23T02:39:29.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.250+0000 7fc969b4c700 1 -- 172.21.15.71:0/3746135762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc964196e70 con 0x7fc964101de0 2024-04-23T02:39:29.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.251+0000 7fc969b4c700 1 -- 172.21.15.71:0/3746135762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9641082f0 con 0x7fc964101de0 2024-04-23T02:39:29.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.252+0000 7fc960ff9700 1 -- 172.21.15.71:0/3746135762 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc9540049a0 con 0x7fc964101de0 2024-04-23T02:39:29.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.252+0000 7fc960ff9700 1 --2- 172.21.15.71:0/3746135762 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc94c06cce0 0x7fc94c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:29.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.252+0000 7fc962ffd700 1 --2- 172.21.15.71:0/3746135762 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc94c06cce0 0x7fc94c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:29.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.252+0000 7fc960ff9700 1 -- 172.21.15.71:0/3746135762 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc95408c2b0 con 0x7fc964101de0 2024-04-23T02:39:29.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.252+0000 7fc962ffd700 1 --2- 172.21.15.71:0/3746135762 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc94c06cce0 0x7fc94c06f1a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc9640fcdf0 tx=0x7fc95801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:39:29.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.257+0000 7fc960ff9700 1 -- 172.21.15.71:0/3746135762 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc954059db0 con 0x7fc964101de0 2024-04-23T02:39:29.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.469+0000 7fc969b4c700 1 -- 172.21.15.71:0/3746135762 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9641970f0 con 0x7fc94c06cce0 2024-04-23T02:39:29.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.470+0000 7fc960ff9700 1 -- 172.21.15.71:0/3746135762 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc9641970f0 con 0x7fc94c06cce0 2024-04-23T02:39:29.473 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:39:29.473 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:39:29.473 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:39:29.473 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:39:29.473 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:39:29.473 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:39:29.473 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:39:29.473 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:39:29.474 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:39:29.474 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:39:29.474 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:39:29.474 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:39:29.474 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:39:29.474 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:39:29.474 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:39:29.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.472+0000 7fc969b4c700 1 -- 172.21.15.71:0/3746135762 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc94c06cce0 msgr2=0x7fc94c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:29.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.472+0000 7fc969b4c700 1 --2- 172.21.15.71:0/3746135762 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc94c06cce0 0x7fc94c06f1a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc9640fcdf0 tx=0x7fc95801a040 comp rx=0 tx=0).stop 2024-04-23T02:39:29.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.472+0000 7fc969b4c700 1 -- 172.21.15.71:0/3746135762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc964101de0 msgr2=0x7fc96419c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:29.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.472+0000 7fc969b4c700 1 --2- 172.21.15.71:0/3746135762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc964101de0 0x7fc96419c5c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc95400c8a0 tx=0x7fc95400cc60 comp rx=0 tx=0).stop 2024-04-23T02:39:29.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.472+0000 7fc969b4c700 1 -- 172.21.15.71:0/3746135762 shutdown_connections 2024-04-23T02:39:29.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.472+0000 7fc969b4c700 1 --2- 172.21.15.71:0/3746135762 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc94c06cce0 0x7fc94c06f1a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:29.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.472+0000 7fc969b4c700 1 --2- 172.21.15.71:0/3746135762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc964101de0 0x7fc96419c5c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:29.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.473+0000 7fc969b4c700 1 --2- 172.21.15.71:0/3746135762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc964102700 0x7fc96419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:29.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.473+0000 7fc969b4c700 1 -- 172.21.15.71:0/3746135762 >> 172.21.15.71:0/3746135762 conn(0x7fc9640fb610 msgr2=0x7fc964105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:39:29.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.473+0000 7fc969b4c700 1 -- 172.21.15.71:0/3746135762 shutdown_connections 2024-04-23T02:39:29.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.473+0000 7fc969b4c700 1 -- 172.21.15.71:0/3746135762 wait complete. 2024-04-23T02:39:29.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.576+0000 7f7cbc226700 1 -- 172.21.15.71:0/2565616140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cb4105bf0 msgr2=0x7f7cb4105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:29.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.576+0000 7f7cbc226700 1 --2- 172.21.15.71:0/2565616140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cb4105bf0 0x7f7cb4105fd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009910 tx=0x7f7ca4009c20 comp rx=0 tx=0).stop 2024-04-23T02:39:29.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.576+0000 7f7cbc226700 1 -- 172.21.15.71:0/2565616140 shutdown_connections 2024-04-23T02:39:29.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.576+0000 7f7cbc226700 1 --2- 172.21.15.71:0/2565616140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb40ffb40 0x7f7cb40fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:29.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.576+0000 7f7cbc226700 1 --2- 172.21.15.71:0/2565616140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cb4105bf0 0x7f7cb4105fd0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:29.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.576+0000 7f7cbc226700 1 -- 172.21.15.71:0/2565616140 >> 172.21.15.71:0/2565616140 conn(0x7f7cb40fb610 msgr2=0x7f7cb40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:39:29.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.576+0000 7f7cbc226700 1 -- 172.21.15.71:0/2565616140 shutdown_connections 2024-04-23T02:39:29.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.576+0000 7f7cbc226700 1 -- 172.21.15.71:0/2565616140 wait complete. 2024-04-23T02:39:29.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.577+0000 7f7cbc226700 1 Processor -- start 2024-04-23T02:39:29.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.577+0000 7f7cbc226700 1 -- start start 2024-04-23T02:39:29.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.578+0000 7f7cbc226700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb40ffb40 0x7f7cb4195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:29.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.578+0000 7f7cbc226700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cb4105bf0 0x7f7cb4195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:29.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.578+0000 7f7cbc226700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4196260 con 0x7f7cb4105bf0 2024-04-23T02:39:29.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.578+0000 7f7cbc226700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4199ff0 con 0x7f7cb40ffb40 2024-04-23T02:39:29.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.578+0000 7f7cb97c1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cb4105bf0 0x7f7cb4195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:29.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.578+0000 7f7cb97c1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cb4105bf0 0x7f7cb4195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56172/0 (socket says 172.21.15.71:56172) 2024-04-23T02:39:29.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.578+0000 7f7cb97c1700 1 -- 172.21.15.71:0/1250408507 learned_addr learned my addr 172.21.15.71:0/1250408507 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:39:29.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.578+0000 7f7cb9fc2700 1 --2- 172.21.15.71:0/1250408507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb40ffb40 0x7f7cb4195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:29.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.578+0000 7f7cb97c1700 1 -- 172.21.15.71:0/1250408507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb40ffb40 msgr2=0x7f7cb4195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:29.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.578+0000 7f7cb97c1700 1 --2- 172.21.15.71:0/1250408507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb40ffb40 0x7f7cb4195640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:29.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.578+0000 7f7cb97c1700 1 -- 172.21.15.71:0/1250408507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca40095f0 con 0x7f7cb4105bf0 2024-04-23T02:39:29.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.578+0000 7f7cb97c1700 1 --2- 172.21.15.71:0/1250408507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cb4105bf0 0x7f7cb4195b80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000eab0 tx=0x7f7cb000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:39:29.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.578+0000 7f7caaffd700 1 -- 172.21.15.71:0/1250408507 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cb0013070 con 0x7f7cb4105bf0 2024-04-23T02:39:29.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.578+0000 7f7caaffd700 1 -- 172.21.15.71:0/1250408507 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7cb0017410 con 0x7f7cb4105bf0 2024-04-23T02:39:29.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.578+0000 7f7cbc226700 1 -- 172.21.15.71:0/1250408507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb419a270 con 0x7f7cb4105bf0 2024-04-23T02:39:29.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.578+0000 7f7caaffd700 1 -- 172.21.15.71:0/1250408507 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cb00165e0 con 0x7f7cb4105bf0 2024-04-23T02:39:29.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.578+0000 7f7cbc226700 1 -- 172.21.15.71:0/1250408507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb419a7c0 con 0x7f7cb4105bf0 2024-04-23T02:39:29.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.579+0000 7f7cbc226700 1 -- 172.21.15.71:0/1250408507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb404f2e0 con 0x7f7cb4105bf0 2024-04-23T02:39:29.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.580+0000 7f7caaffd700 1 -- 172.21.15.71:0/1250408507 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7cb0017580 con 0x7f7cb4105bf0 2024-04-23T02:39:29.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.580+0000 7f7caaffd700 1 --2- 172.21.15.71:0/1250408507 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7ca006cce0 0x7f7ca006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:29.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.580+0000 7f7caaffd700 1 -- 172.21.15.71:0/1250408507 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7cb008dd40 con 0x7f7cb4105bf0 2024-04-23T02:39:29.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.580+0000 7f7cb9fc2700 1 --2- 172.21.15.71:0/1250408507 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7ca006cce0 0x7f7ca006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.581+0000 7f7cb9fc2700 1 --2- 172.21.15.71:0/1250408507 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7ca006cce0 0x7f7ca006f1a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009fd0 tx=0x7f7ca4005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:39:29.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.585+0000 7f7caaffd700 1 -- 172.21.15.71:0/1250408507 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7cb0057ee0 con 0x7f7cb4105bf0 2024-04-23T02:39:29.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.856+0000 7f7cbc226700 1 -- 172.21.15.71:0/1250408507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7cb41969a0 con 0x7f7cb4105bf0 2024-04-23T02:39:29.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.856+0000 7f7caaffd700 1 -- 172.21.15.71:0/1250408507 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7f7cb001c090 con 0x7f7cb4105bf0 2024-04-23T02:39:29.859 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:39:29.859 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:39:29.859 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:39:29.859 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:39:29.859 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:39:29.859 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:39:29.859 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:39:29.860 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:39:29.860 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:39:29.860 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:39:29.860 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:39:29.860 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:39:29.860 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:39:29.860 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:39:29.860 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:39:29.860 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:39:29.860 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:39:29.860 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:39:29.860 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:39:29.860 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:39:29.861 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:39:29.861 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:39:29.861 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:39:29.861 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:39:29.861 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:39:29.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.858+0000 7f7cbc226700 1 -- 172.21.15.71:0/1250408507 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7ca006cce0 msgr2=0x7f7ca006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:29.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.858+0000 7f7cbc226700 1 --2- 172.21.15.71:0/1250408507 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7ca006cce0 0x7f7ca006f1a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009fd0 tx=0x7f7ca4005940 comp rx=0 tx=0).stop 2024-04-23T02:39:29.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.858+0000 7f7cbc226700 1 -- 172.21.15.71:0/1250408507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cb4105bf0 msgr2=0x7f7cb4195b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:29.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.858+0000 7f7cbc226700 1 --2- 172.21.15.71:0/1250408507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cb4105bf0 0x7f7cb4195b80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000eab0 tx=0x7f7cb000edc0 comp rx=0 tx=0).stop 2024-04-23T02:39:29.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.858+0000 7f7cbc226700 1 -- 172.21.15.71:0/1250408507 shutdown_connections 2024-04-23T02:39:29.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.858+0000 7f7cbc226700 1 --2- 172.21.15.71:0/1250408507 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7ca006cce0 0x7f7ca006f1a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:29.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.858+0000 7f7cbc226700 1 --2- 172.21.15.71:0/1250408507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb40ffb40 0x7f7cb4195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:29.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.858+0000 7f7cbc226700 1 --2- 172.21.15.71:0/1250408507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7cb4105bf0 0x7f7cb4195b80 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:29.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.858+0000 7f7cbc226700 1 -- 172.21.15.71:0/1250408507 >> 172.21.15.71:0/1250408507 conn(0x7f7cb40fb610 msgr2=0x7f7cb40fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:39:29.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.858+0000 7f7cbc226700 1 -- 172.21.15.71:0/1250408507 shutdown_connections 2024-04-23T02:39:29.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:29.858+0000 7f7cbc226700 1 -- 172.21.15.71:0/1250408507 wait complete. 2024-04-23T02:39:29.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:29 smithi192 ceph-mon[108617]: from='client.35240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:39:29.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:29 smithi192 ceph-mon[108617]: from='client.35244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:39:29.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:29 smithi192 ceph-mon[108617]: from='client.35248 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:39:29.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:29 smithi192 ceph-mon[108617]: pgmap v786: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:29.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:29 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/638337524' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:39:29.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:29 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/198398450' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:39:29.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:29 smithi071 ceph-mon[144544]: from='client.35240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:39:29.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:29 smithi071 ceph-mon[144544]: from='client.35244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:39:29.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:29 smithi071 ceph-mon[144544]: from='client.35248 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:39:29.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:29 smithi071 ceph-mon[144544]: pgmap v786: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:29.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:29 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/638337524' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:39:29.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:29 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/198398450' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:39:30.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:30 smithi192 ceph-mon[108617]: from='client.35260 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:39:30.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:30 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1250408507' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:39:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:30 smithi071 ceph-mon[144544]: from='client.35260 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:39:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:30 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1250408507' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:39:31.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:31 smithi192 ceph-mon[108617]: pgmap v787: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:31.963 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:31 smithi071 ceph-mon[144544]: pgmap v787: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:33.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:33 smithi192 ceph-mon[108617]: pgmap v788: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:33.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:33 smithi071 ceph-mon[144544]: pgmap v788: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:35.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:35 smithi192 ceph-mon[108617]: pgmap v789: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:35.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:39:35.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:35 smithi071 ceph-mon[144544]: pgmap v789: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:39:36.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:36 smithi192 ceph-mon[108617]: pgmap v790: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:36.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:36 smithi071 ceph-mon[144544]: pgmap v790: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:39.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:39 smithi071 ceph-mon[144544]: pgmap v791: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:39.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:39 smithi192 ceph-mon[108617]: pgmap v791: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:41.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:41 smithi192 ceph-mon[108617]: pgmap v792: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:41.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:41 smithi071 ceph-mon[144544]: pgmap v792: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:43.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:43 smithi192 ceph-mon[108617]: pgmap v793: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:43.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:43 smithi071 ceph-mon[144544]: pgmap v793: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:45.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:45 smithi192 ceph-mon[108617]: pgmap v794: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:45.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:45 smithi071 ceph-mon[144544]: pgmap v794: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:47.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:47 smithi192 ceph-mon[108617]: pgmap v795: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:47.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:47 smithi071 ceph-mon[144544]: pgmap v795: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:49.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:49 smithi192 ceph-mon[108617]: pgmap v796: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:49.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:49 smithi071 ceph-mon[144544]: pgmap v796: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:39:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:39:51.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:51 smithi192 ceph-mon[108617]: pgmap v797: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:51.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:51 smithi071 ceph-mon[144544]: pgmap v797: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:53.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:53 smithi192 ceph-mon[108617]: pgmap v798: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:53.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:53 smithi071 ceph-mon[144544]: pgmap v798: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:55.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:55 smithi192 ceph-mon[108617]: pgmap v799: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:55.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:55 smithi071 ceph-mon[144544]: pgmap v799: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:57.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:57 smithi192 ceph-mon[108617]: pgmap v800: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:57.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:57 smithi071 ceph-mon[144544]: pgmap v800: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:59.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:39:59 smithi192 ceph-mon[108617]: pgmap v801: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:39:59.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.971+0000 7f8d77bda700 1 -- 172.21.15.71:0/2991255391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d70106cd0 msgr2=0x7f8d701070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:59.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.971+0000 7f8d77bda700 1 --2- 172.21.15.71:0/2991255391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d70106cd0 0x7f8d701070b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009910 tx=0x7f8d60009c20 comp rx=0 tx=0).stop 2024-04-23T02:39:59.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.971+0000 7f8d77bda700 1 -- 172.21.15.71:0/2991255391 shutdown_connections 2024-04-23T02:39:59.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.971+0000 7f8d77bda700 1 --2- 172.21.15.71:0/2991255391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d70068580 0x7f8d700689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:59.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.971+0000 7f8d77bda700 1 --2- 172.21.15.71:0/2991255391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d70106cd0 0x7f8d701070b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:59.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.971+0000 7f8d77bda700 1 -- 172.21.15.71:0/2991255391 >> 172.21.15.71:0/2991255391 conn(0x7f8d70075670 msgr2=0x7f8d70075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:39:59.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.971+0000 7f8d77bda700 1 -- 172.21.15.71:0/2991255391 shutdown_connections 2024-04-23T02:39:59.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.971+0000 7f8d77bda700 1 -- 172.21.15.71:0/2991255391 wait complete. 2024-04-23T02:39:59.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.972+0000 7f8d77bda700 1 Processor -- start 2024-04-23T02:39:59.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.972+0000 7f8d77bda700 1 -- start start 2024-04-23T02:39:59.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.972+0000 7f8d77bda700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d70068580 0x7f8d7019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:59.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.972+0000 7f8d77bda700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d70106cd0 0x7f8d7019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:59.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.972+0000 7f8d77bda700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7019d130 con 0x7f8d70106cd0 2024-04-23T02:39:59.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.972+0000 7f8d77bda700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d70196630 con 0x7f8d70068580 2024-04-23T02:39:59.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.972+0000 7f8d75175700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d70106cd0 0x7f8d7019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:59.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.972+0000 7f8d75175700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d70106cd0 0x7f8d7019caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48676/0 (socket says 172.21.15.71:48676) 2024-04-23T02:39:59.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.972+0000 7f8d75175700 1 -- 172.21.15.71:0/370497947 learned_addr learned my addr 172.21.15.71:0/370497947 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:39:59.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.972+0000 7f8d75976700 1 --2- 172.21.15.71:0/370497947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d70068580 0x7f8d7019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:59.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.972+0000 7f8d75175700 1 -- 172.21.15.71:0/370497947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d70068580 msgr2=0x7f8d7019c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:39:59.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.972+0000 7f8d75175700 1 --2- 172.21.15.71:0/370497947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d70068580 0x7f8d7019c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:39:59.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.972+0000 7f8d75175700 1 -- 172.21.15.71:0/370497947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d600095f0 con 0x7f8d70106cd0 2024-04-23T02:39:59.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.973+0000 7f8d75175700 1 --2- 172.21.15.71:0/370497947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d70106cd0 0x7f8d7019caa0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00d8d0 tx=0x7f8d6c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:39:59.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.973+0000 7f8d66ffd700 1 -- 172.21.15.71:0/370497947 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d6c0098a0 con 0x7f8d70106cd0 2024-04-23T02:39:59.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.973+0000 7f8d77bda700 1 -- 172.21.15.71:0/370497947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d70196910 con 0x7f8d70106cd0 2024-04-23T02:39:59.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.973+0000 7f8d66ffd700 1 -- 172.21.15.71:0/370497947 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8d6c010460 con 0x7f8d70106cd0 2024-04-23T02:39:59.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.973+0000 7f8d66ffd700 1 -- 172.21.15.71:0/370497947 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d6c009ea0 con 0x7f8d70106cd0 2024-04-23T02:39:59.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.973+0000 7f8d77bda700 1 -- 172.21.15.71:0/370497947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d70196e60 con 0x7f8d70106cd0 2024-04-23T02:39:59.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.974+0000 7f8d77bda700 1 -- 172.21.15.71:0/370497947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d701091d0 con 0x7f8d70106cd0 2024-04-23T02:39:59.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.974+0000 7f8d66ffd700 1 -- 172.21.15.71:0/370497947 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8d6c0105d0 con 0x7f8d70106cd0 2024-04-23T02:39:59.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.975+0000 7f8d66ffd700 1 --2- 172.21.15.71:0/370497947 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8d5c06cce0 0x7f8d5c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:39:59.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.975+0000 7f8d66ffd700 1 -- 172.21.15.71:0/370497947 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8d6c08c480 con 0x7f8d70106cd0 2024-04-23T02:39:59.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.975+0000 7f8d75976700 1 --2- 172.21.15.71:0/370497947 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8d5c06cce0 0x7f8d5c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:39:59.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.975+0000 7f8d75976700 1 --2- 172.21.15.71:0/370497947 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8d5c06cce0 0x7f8d5c06f1a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8d6000b5c0 tx=0x7f8d6001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:39:59.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:39:59.979+0000 7f8d66ffd700 1 -- 172.21.15.71:0/370497947 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8d6c059f80 con 0x7f8d70106cd0 2024-04-23T02:39:59.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:39:59 smithi071 ceph-mon[144544]: pgmap v801: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:00.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.191+0000 7f8d77bda700 1 -- 172.21.15.71:0/370497947 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d70197080 con 0x7f8d5c06cce0 2024-04-23T02:40:00.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.192+0000 7f8d66ffd700 1 -- 172.21.15.71:0/370497947 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8d70197080 con 0x7f8d5c06cce0 2024-04-23T02:40:00.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.194+0000 7f8d77bda700 1 -- 172.21.15.71:0/370497947 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8d5c06cce0 msgr2=0x7f8d5c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:00.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.194+0000 7f8d77bda700 1 --2- 172.21.15.71:0/370497947 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8d5c06cce0 0x7f8d5c06f1a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8d6000b5c0 tx=0x7f8d6001a040 comp rx=0 tx=0).stop 2024-04-23T02:40:00.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.194+0000 7f8d77bda700 1 -- 172.21.15.71:0/370497947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d70106cd0 msgr2=0x7f8d7019caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:00.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.194+0000 7f8d77bda700 1 --2- 172.21.15.71:0/370497947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d70106cd0 0x7f8d7019caa0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00d8d0 tx=0x7f8d6c00dbe0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.194+0000 7f8d77bda700 1 -- 172.21.15.71:0/370497947 shutdown_connections 2024-04-23T02:40:00.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.194+0000 7f8d77bda700 1 --2- 172.21.15.71:0/370497947 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8d5c06cce0 0x7f8d5c06f1a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.194+0000 7f8d77bda700 1 --2- 172.21.15.71:0/370497947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d70068580 0x7f8d7019c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.194+0000 7f8d77bda700 1 --2- 172.21.15.71:0/370497947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d70106cd0 0x7f8d7019caa0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.194+0000 7f8d77bda700 1 -- 172.21.15.71:0/370497947 >> 172.21.15.71:0/370497947 conn(0x7f8d70075670 msgr2=0x7f8d700fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:00.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.194+0000 7f8d77bda700 1 -- 172.21.15.71:0/370497947 shutdown_connections 2024-04-23T02:40:00.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.194+0000 7f8d77bda700 1 -- 172.21.15.71:0/370497947 wait complete. 2024-04-23T02:40:00.207 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:40:00.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.297+0000 7fd04f4b3700 1 -- 172.21.15.71:0/1105993949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0480ffc20 msgr2=0x7fd048100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:00.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.297+0000 7fd04f4b3700 1 --2- 172.21.15.71:0/1105993949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0480ffc20 0x7fd048100080 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd044009910 tx=0x7fd044009c20 comp rx=0 tx=0).stop 2024-04-23T02:40:00.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.297+0000 7fd04f4b3700 1 -- 172.21.15.71:0/1105993949 shutdown_connections 2024-04-23T02:40:00.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.297+0000 7fd04f4b3700 1 --2- 172.21.15.71:0/1105993949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0480ffc20 0x7fd048100080 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.297+0000 7fd04f4b3700 1 --2- 172.21.15.71:0/1105993949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd048105bd0 0x7fd048105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.297+0000 7fd04f4b3700 1 -- 172.21.15.71:0/1105993949 >> 172.21.15.71:0/1105993949 conn(0x7fd048075670 msgr2=0x7fd048075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:00.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.297+0000 7fd04f4b3700 1 -- 172.21.15.71:0/1105993949 shutdown_connections 2024-04-23T02:40:00.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.297+0000 7fd04f4b3700 1 -- 172.21.15.71:0/1105993949 wait complete. 2024-04-23T02:40:00.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.298+0000 7fd04f4b3700 1 Processor -- start 2024-04-23T02:40:00.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.298+0000 7fd04f4b3700 1 -- start start 2024-04-23T02:40:00.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.298+0000 7fd04f4b3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0480ffc20 0x7fd04819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:00.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.298+0000 7fd04f4b3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd048105bd0 0x7fd04819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:00.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.298+0000 7fd04f4b3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04819d150 con 0x7fd0480ffc20 2024-04-23T02:40:00.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.298+0000 7fd04f4b3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd048196600 con 0x7fd048105bd0 2024-04-23T02:40:00.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.298+0000 7fd04d24f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0480ffc20 0x7fd04819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:00.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.298+0000 7fd04d24f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0480ffc20 0x7fd04819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48706/0 (socket says 172.21.15.71:48706) 2024-04-23T02:40:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.298+0000 7fd04d24f700 1 -- 172.21.15.71:0/3963873247 learned_addr learned my addr 172.21.15.71:0/3963873247 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:40:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.298+0000 7fd04ca4e700 1 --2- 172.21.15.71:0/3963873247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd048105bd0 0x7fd04819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.298+0000 7fd04d24f700 1 -- 172.21.15.71:0/3963873247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd048105bd0 msgr2=0x7fd04819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.298+0000 7fd04d24f700 1 --2- 172.21.15.71:0/3963873247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd048105bd0 0x7fd04819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.298+0000 7fd04d24f700 1 -- 172.21.15.71:0/3963873247 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0440095f0 con 0x7fd0480ffc20 2024-04-23T02:40:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.298+0000 7fd04d24f700 1 --2- 172.21.15.71:0/3963873247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0480ffc20 0x7fd04819c580 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00d8d0 tx=0x7fd03c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.299+0000 7fd03a7fc700 1 -- 172.21.15.71:0/3963873247 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd03c0098a0 con 0x7fd0480ffc20 2024-04-23T02:40:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.299+0000 7fd03a7fc700 1 -- 172.21.15.71:0/3963873247 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd03c010460 con 0x7fd0480ffc20 2024-04-23T02:40:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.299+0000 7fd04f4b3700 1 -- 172.21.15.71:0/3963873247 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0481968e0 con 0x7fd0480ffc20 2024-04-23T02:40:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.299+0000 7fd03a7fc700 1 -- 172.21.15.71:0/3963873247 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd03c009ea0 con 0x7fd0480ffc20 2024-04-23T02:40:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.299+0000 7fd04f4b3700 1 -- 172.21.15.71:0/3963873247 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd048196e30 con 0x7fd0480ffc20 2024-04-23T02:40:00.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.300+0000 7fd04f4b3700 1 -- 172.21.15.71:0/3963873247 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04804f2e0 con 0x7fd0480ffc20 2024-04-23T02:40:00.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.300+0000 7fd03a7fc700 1 -- 172.21.15.71:0/3963873247 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd03c0105d0 con 0x7fd0480ffc20 2024-04-23T02:40:00.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.300+0000 7fd03a7fc700 1 --2- 172.21.15.71:0/3963873247 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd03406cce0 0x7fd03406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:00.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.301+0000 7fd03a7fc700 1 -- 172.21.15.71:0/3963873247 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd03c08c480 con 0x7fd0480ffc20 2024-04-23T02:40:00.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.301+0000 7fd04ca4e700 1 --2- 172.21.15.71:0/3963873247 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd03406cce0 0x7fd03406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:00.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.301+0000 7fd04ca4e700 1 --2- 172.21.15.71:0/3963873247 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd03406cce0 0x7fd03406f1a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd048075de0 tx=0x7fd04401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:00.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.305+0000 7fd03a7fc700 1 -- 172.21.15.71:0/3963873247 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd03c059f80 con 0x7fd0480ffc20 2024-04-23T02:40:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:40:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:40:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:40:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : /usr/bin/podman: timeout after 895 seconds 2024-04-23T02:40:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:40:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : Traceback (most recent call last): 2024-04-23T02:40:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:40:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : "__main__", mod_spec) 2024-04-23T02:40:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:40:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : exec(code, run_globals) 2024-04-23T02:40:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:40:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:40:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:40:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:40:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:40:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:40:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:40:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:40:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:40:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:40:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-04-23T02:40:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:40:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:40:00.494 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:40:00.495 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:39:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-04-23T02:40:00.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.515+0000 7fd04f4b3700 1 -- 172.21.15.71:0/3963873247 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0481970f0 con 0x7fd03406cce0 2024-04-23T02:40:00.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.516+0000 7fd03a7fc700 1 -- 172.21.15.71:0/3963873247 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd0481970f0 con 0x7fd03406cce0 2024-04-23T02:40:00.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.518+0000 7fd04f4b3700 1 -- 172.21.15.71:0/3963873247 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd03406cce0 msgr2=0x7fd03406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:00.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.518+0000 7fd04f4b3700 1 --2- 172.21.15.71:0/3963873247 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd03406cce0 0x7fd03406f1a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd048075de0 tx=0x7fd04401a040 comp rx=0 tx=0).stop 2024-04-23T02:40:00.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.518+0000 7fd04f4b3700 1 -- 172.21.15.71:0/3963873247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0480ffc20 msgr2=0x7fd04819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:00.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.518+0000 7fd04f4b3700 1 --2- 172.21.15.71:0/3963873247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0480ffc20 0x7fd04819c580 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00d8d0 tx=0x7fd03c00dbe0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.518+0000 7fd04f4b3700 1 -- 172.21.15.71:0/3963873247 shutdown_connections 2024-04-23T02:40:00.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.518+0000 7fd04f4b3700 1 --2- 172.21.15.71:0/3963873247 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd03406cce0 0x7fd03406f1a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.518+0000 7fd04f4b3700 1 --2- 172.21.15.71:0/3963873247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0480ffc20 0x7fd04819c580 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.518+0000 7fd04f4b3700 1 --2- 172.21.15.71:0/3963873247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd048105bd0 0x7fd04819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.518+0000 7fd04f4b3700 1 -- 172.21.15.71:0/3963873247 >> 172.21.15.71:0/3963873247 conn(0x7fd048075670 msgr2=0x7fd048068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:00.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.518+0000 7fd04f4b3700 1 -- 172.21.15.71:0/3963873247 shutdown_connections 2024-04-23T02:40:00.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.518+0000 7fd04f4b3700 1 -- 172.21.15.71:0/3963873247 wait complete. 2024-04-23T02:40:00.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.622+0000 7fae7de70700 1 -- 172.21.15.71:0/3077041245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae780ffda0 msgr2=0x7fae78100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:00.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.622+0000 7fae7de70700 1 --2- 172.21.15.71:0/3077041245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae780ffda0 0x7fae78100180 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fae68009910 tx=0x7fae68009c20 comp rx=0 tx=0).stop 2024-04-23T02:40:00.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.622+0000 7fae7de70700 1 -- 172.21.15.71:0/3077041245 shutdown_connections 2024-04-23T02:40:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.622+0000 7fae7de70700 1 --2- 172.21.15.71:0/3077041245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae78100750 0x7fae781047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.622+0000 7fae7de70700 1 --2- 172.21.15.71:0/3077041245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae780ffda0 0x7fae78100180 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.622+0000 7fae7de70700 1 -- 172.21.15.71:0/3077041245 >> 172.21.15.71:0/3077041245 conn(0x7fae780fb650 msgr2=0x7fae780fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.622+0000 7fae7de70700 1 -- 172.21.15.71:0/3077041245 shutdown_connections 2024-04-23T02:40:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.622+0000 7fae7de70700 1 -- 172.21.15.71:0/3077041245 wait complete. 2024-04-23T02:40:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.623+0000 7fae7de70700 1 Processor -- start 2024-04-23T02:40:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.623+0000 7fae7de70700 1 -- start start 2024-04-23T02:40:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.623+0000 7fae7de70700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae780ffda0 0x7fae7819c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.623+0000 7fae7de70700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae78100750 0x7fae7819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.623+0000 7fae7de70700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7819d140 con 0x7fae780ffda0 2024-04-23T02:40:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.623+0000 7fae7de70700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae78196660 con 0x7fae78100750 2024-04-23T02:40:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.623+0000 7fae777fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae780ffda0 0x7fae7819c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.623+0000 7fae777fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae780ffda0 0x7fae7819c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48720/0 (socket says 172.21.15.71:48720) 2024-04-23T02:40:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.623+0000 7fae777fe700 1 -- 172.21.15.71:0/99630191 learned_addr learned my addr 172.21.15.71:0/99630191 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:40:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.623+0000 7fae76ffd700 1 --2- 172.21.15.71:0/99630191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae78100750 0x7fae7819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.623+0000 7fae777fe700 1 -- 172.21.15.71:0/99630191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae78100750 msgr2=0x7fae7819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.623+0000 7fae777fe700 1 --2- 172.21.15.71:0/99630191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae78100750 0x7fae7819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.623+0000 7fae777fe700 1 -- 172.21.15.71:0/99630191 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae680095f0 con 0x7fae780ffda0 2024-04-23T02:40:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.623+0000 7fae76ffd700 1 --2- 172.21.15.71:0/99630191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae78100750 0x7fae7819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:40:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.624+0000 7fae777fe700 1 --2- 172.21.15.71:0/99630191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae780ffda0 0x7fae7819c5e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fae68006010 tx=0x7fae6800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.624+0000 7fae74ff9700 1 -- 172.21.15.71:0/99630191 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae6801d070 con 0x7fae780ffda0 2024-04-23T02:40:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.624+0000 7fae74ff9700 1 -- 172.21.15.71:0/99630191 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae6800f460 con 0x7fae780ffda0 2024-04-23T02:40:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.624+0000 7fae7de70700 1 -- 172.21.15.71:0/99630191 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae781968e0 con 0x7fae780ffda0 2024-04-23T02:40:00.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.624+0000 7fae74ff9700 1 -- 172.21.15.71:0/99630191 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae68021620 con 0x7fae780ffda0 2024-04-23T02:40:00.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.624+0000 7fae7de70700 1 -- 172.21.15.71:0/99630191 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae78196dd0 con 0x7fae780ffda0 2024-04-23T02:40:00.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.625+0000 7fae7de70700 1 -- 172.21.15.71:0/99630191 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae7804f2e0 con 0x7fae780ffda0 2024-04-23T02:40:00.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.625+0000 7fae74ff9700 1 -- 172.21.15.71:0/99630191 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fae6802b440 con 0x7fae780ffda0 2024-04-23T02:40:00.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.625+0000 7fae74ff9700 1 --2- 172.21.15.71:0/99630191 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fae6006cce0 0x7fae6006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:00.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.626+0000 7fae74ff9700 1 -- 172.21.15.71:0/99630191 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fae6808d7e0 con 0x7fae780ffda0 2024-04-23T02:40:00.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.626+0000 7fae76ffd700 1 --2- 172.21.15.71:0/99630191 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fae6006cce0 0x7fae6006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:00.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.626+0000 7fae76ffd700 1 --2- 172.21.15.71:0/99630191 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fae6006cce0 0x7fae6006f1a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fae6c005fd0 tx=0x7fae6c005ee0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:00.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.630+0000 7fae74ff9700 1 -- 172.21.15.71:0/99630191 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fae6805b350 con 0x7fae780ffda0 2024-04-23T02:40:00.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.830+0000 7fae7de70700 1 -- 172.21.15.71:0/99630191 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fae78197060 con 0x7fae6006cce0 2024-04-23T02:40:00.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.841+0000 7fae74ff9700 1 -- 172.21.15.71:0/99630191 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fae78197060 con 0x7fae6006cce0 2024-04-23T02:40:00.844 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:40:00.844 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (23m) 117s ago 34m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:40:00.844 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (35m) 117s ago 35m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:40:00.844 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (34m) 119s ago 34m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:40:00.844 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (21m) 117s ago 35m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:40:00.844 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (21m) 119s ago 34m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:40:00.844 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (23m) 117s ago 34m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:40:00.844 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (17m) 117s ago 29m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:40:00.844 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (17m) 117s ago 28m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:40:00.844 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (2m) 119s ago 28m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:40:00.844 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (17m) 119s ago 29m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:40:00.844 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (25m) 117s ago 36m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:40:00.845 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (24m) 119s ago 33m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:40:00.845 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (22m) 117s ago 36m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:40:00.845 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (21m) 119s ago 33m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:40:00.845 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (24m) 117s ago 35m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:40:00.845 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (24m) 119s ago 33m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:40:00.845 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (21m) 117s ago 33m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:40:00.845 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (20m) 117s ago 32m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:40:00.845 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (19m) 117s ago 32m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:40:00.845 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (19m) 119s ago 31m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:40:00.845 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (18m) 119s ago 31m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:40:00.845 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (18m) 119s ago 30m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:40:00.845 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (24m) 117s ago 34m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:40:00.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.843+0000 7fae7de70700 1 -- 172.21.15.71:0/99630191 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fae6006cce0 msgr2=0x7fae6006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:00.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.843+0000 7fae7de70700 1 --2- 172.21.15.71:0/99630191 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fae6006cce0 0x7fae6006f1a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fae6c005fd0 tx=0x7fae6c005ee0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.843+0000 7fae7de70700 1 -- 172.21.15.71:0/99630191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae780ffda0 msgr2=0x7fae7819c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:00.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.843+0000 7fae7de70700 1 --2- 172.21.15.71:0/99630191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae780ffda0 0x7fae7819c5e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fae68006010 tx=0x7fae6800be20 comp rx=0 tx=0).stop 2024-04-23T02:40:00.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.843+0000 7fae7de70700 1 -- 172.21.15.71:0/99630191 shutdown_connections 2024-04-23T02:40:00.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.843+0000 7fae7de70700 1 --2- 172.21.15.71:0/99630191 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fae6006cce0 0x7fae6006f1a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.843+0000 7fae7de70700 1 --2- 172.21.15.71:0/99630191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae780ffda0 0x7fae7819c5e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.843+0000 7fae7de70700 1 --2- 172.21.15.71:0/99630191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae78100750 0x7fae7819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.843+0000 7fae7de70700 1 -- 172.21.15.71:0/99630191 >> 172.21.15.71:0/99630191 conn(0x7fae780fb650 msgr2=0x7fae78103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:00.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.843+0000 7fae7de70700 1 -- 172.21.15.71:0/99630191 shutdown_connections 2024-04-23T02:40:00.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.843+0000 7fae7de70700 1 -- 172.21.15.71:0/99630191 wait complete. 2024-04-23T02:40:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:40:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:40:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:40:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: /usr/bin/podman: timeout after 895 seconds 2024-04-23T02:40:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:40:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: Traceback (most recent call last): 2024-04-23T02:40:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:40:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: "__main__", mod_spec) 2024-04-23T02:40:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:40:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: exec(code, run_globals) 2024-04-23T02:40:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:40:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:40:00.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:40:00.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:40:00.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:40:00.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:40:00.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:40:00.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:40:00.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:40:00.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:40:00.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: fs cephfs is degraded 2024-04-23T02:40:00.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:40:00.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:40:00.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:40:00.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:00 smithi192 ceph-mon[108617]: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:40:00.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.945+0000 7f5ee9c0d700 1 -- 172.21.15.71:0/511658831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ee4102720 msgr2=0x7f5ee410ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:00.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.945+0000 7f5ee9c0d700 1 --2- 172.21.15.71:0/511658831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ee4102720 0x7f5ee410ac10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009910 tx=0x7f5ed4009c20 comp rx=0 tx=0).stop 2024-04-23T02:40:00.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.945+0000 7f5ee9c0d700 1 -- 172.21.15.71:0/511658831 shutdown_connections 2024-04-23T02:40:00.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.945+0000 7f5ee9c0d700 1 --2- 172.21.15.71:0/511658831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ee4102720 0x7f5ee410ac10 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.945+0000 7f5ee9c0d700 1 --2- 172.21.15.71:0/511658831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ee4101e00 0x7f5ee41021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.945+0000 7f5ee9c0d700 1 -- 172.21.15.71:0/511658831 >> 172.21.15.71:0/511658831 conn(0x7f5ee40fb650 msgr2=0x7f5ee40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:00.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.945+0000 7f5ee9c0d700 1 -- 172.21.15.71:0/511658831 shutdown_connections 2024-04-23T02:40:00.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.945+0000 7f5ee9c0d700 1 -- 172.21.15.71:0/511658831 wait complete. 2024-04-23T02:40:00.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.946+0000 7f5ee9c0d700 1 Processor -- start 2024-04-23T02:40:00.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.946+0000 7f5ee9c0d700 1 -- start start 2024-04-23T02:40:00.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.946+0000 7f5ee9c0d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ee4101e00 0x7f5ee419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:00.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.947+0000 7f5ee9c0d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ee4102720 0x7f5ee419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:00.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.947+0000 7f5ee9c0d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee419d220 con 0x7f5ee4101e00 2024-04-23T02:40:00.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.947+0000 7f5ee9c0d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee41966d0 con 0x7f5ee4102720 2024-04-23T02:40:00.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.947+0000 7f5ee37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ee4101e00 0x7f5ee419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:00.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.947+0000 7f5ee37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ee4101e00 0x7f5ee419c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48750/0 (socket says 172.21.15.71:48750) 2024-04-23T02:40:00.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.947+0000 7f5ee37fe700 1 -- 172.21.15.71:0/3355339704 learned_addr learned my addr 172.21.15.71:0/3355339704 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:40:00.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.947+0000 7f5ee2ffd700 1 --2- 172.21.15.71:0/3355339704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ee4102720 0x7f5ee419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:00.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.947+0000 7f5ee37fe700 1 -- 172.21.15.71:0/3355339704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ee4102720 msgr2=0x7f5ee419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:00.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.947+0000 7f5ee37fe700 1 --2- 172.21.15.71:0/3355339704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ee4102720 0x7f5ee419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:00.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.947+0000 7f5ee37fe700 1 -- 172.21.15.71:0/3355339704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed40095f0 con 0x7f5ee4101e00 2024-04-23T02:40:00.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.947+0000 7f5ee37fe700 1 --2- 172.21.15.71:0/3355339704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ee4101e00 0x7f5ee419c650 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800d8d0 tx=0x7f5ed800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:00.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.947+0000 7f5ee0ff9700 1 -- 172.21.15.71:0/3355339704 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ed8009940 con 0x7f5ee4101e00 2024-04-23T02:40:00.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.947+0000 7f5ee0ff9700 1 -- 172.21.15.71:0/3355339704 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5ed8010460 con 0x7f5ee4101e00 2024-04-23T02:40:00.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.947+0000 7f5ee9c0d700 1 -- 172.21.15.71:0/3355339704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee41969b0 con 0x7f5ee4101e00 2024-04-23T02:40:00.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.947+0000 7f5ee0ff9700 1 -- 172.21.15.71:0/3355339704 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ed800f5f0 con 0x7f5ee4101e00 2024-04-23T02:40:00.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.947+0000 7f5ee9c0d700 1 -- 172.21.15.71:0/3355339704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee4196ed0 con 0x7f5ee4101e00 2024-04-23T02:40:00.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.948+0000 7f5ee9c0d700 1 -- 172.21.15.71:0/3355339704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee4108310 con 0x7f5ee4101e00 2024-04-23T02:40:00.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.949+0000 7f5ee0ff9700 1 -- 172.21.15.71:0/3355339704 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5ed80105d0 con 0x7f5ee4101e00 2024-04-23T02:40:00.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.949+0000 7f5ee0ff9700 1 --2- 172.21.15.71:0/3355339704 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5ecc06cce0 0x7f5ecc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:00.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.949+0000 7f5ee0ff9700 1 -- 172.21.15.71:0/3355339704 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5ed808c5e0 con 0x7f5ee4101e00 2024-04-23T02:40:00.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.949+0000 7f5ee2ffd700 1 --2- 172.21.15.71:0/3355339704 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5ecc06cce0 0x7f5ecc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:00.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.950+0000 7f5ee2ffd700 1 --2- 172.21.15.71:0/3355339704 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5ecc06cce0 0x7f5ecc06f1a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4006010 tx=0x7f5ed401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:00.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:00.954+0000 7f5ee0ff9700 1 -- 172.21.15.71:0/3355339704 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5ed805a0e0 con 0x7f5ee4101e00 2024-04-23T02:40:00.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:40:00.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:40:00.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:40:00.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: /usr/bin/podman: timeout after 895 seconds 2024-04-23T02:40:00.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:40:00.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: Traceback (most recent call last): 2024-04-23T02:40:00.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:40:00.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: "__main__", mod_spec) 2024-04-23T02:40:00.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:40:00.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: exec(code, run_globals) 2024-04-23T02:40:00.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:40:00.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:40:00.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:40:00.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:40:00.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:40:00.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:40:00.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:40:00.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:40:00.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:40:00.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:40:00.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: fs cephfs is degraded 2024-04-23T02:40:00.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:40:00.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:40:00.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:40:00.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:00 smithi071 ceph-mon[144544]: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:40:01.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.226+0000 7f5ee9c0d700 1 -- 172.21.15.71:0/3355339704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5ee4197c80 con 0x7f5ee4101e00 2024-04-23T02:40:01.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.226+0000 7f5ee0ff9700 1 -- 172.21.15.71:0/3355339704 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f5ed8010880 con 0x7f5ee4101e00 2024-04-23T02:40:01.229 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:40:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:40:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:40:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:40:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:40:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:40:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:40:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:40:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:40:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:40:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:40:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:40:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:40:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:40:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:40:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:40:01.230 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:40:01.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.229+0000 7f5ee9c0d700 1 -- 172.21.15.71:0/3355339704 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5ecc06cce0 msgr2=0x7f5ecc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:01.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.229+0000 7f5ee9c0d700 1 --2- 172.21.15.71:0/3355339704 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5ecc06cce0 0x7f5ecc06f1a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4006010 tx=0x7f5ed401a040 comp rx=0 tx=0).stop 2024-04-23T02:40:01.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.229+0000 7f5ee9c0d700 1 -- 172.21.15.71:0/3355339704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ee4101e00 msgr2=0x7f5ee419c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:01.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.229+0000 7f5ee9c0d700 1 --2- 172.21.15.71:0/3355339704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ee4101e00 0x7f5ee419c650 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800d8d0 tx=0x7f5ed800dc90 comp rx=0 tx=0).stop 2024-04-23T02:40:01.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.229+0000 7f5ee9c0d700 1 -- 172.21.15.71:0/3355339704 shutdown_connections 2024-04-23T02:40:01.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.229+0000 7f5ee9c0d700 1 --2- 172.21.15.71:0/3355339704 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5ecc06cce0 0x7f5ecc06f1a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:01.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.229+0000 7f5ee9c0d700 1 --2- 172.21.15.71:0/3355339704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ee4101e00 0x7f5ee419c650 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:01.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.229+0000 7f5ee9c0d700 1 --2- 172.21.15.71:0/3355339704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ee4102720 0x7f5ee419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:01.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.229+0000 7f5ee9c0d700 1 -- 172.21.15.71:0/3355339704 >> 172.21.15.71:0/3355339704 conn(0x7f5ee40fb650 msgr2=0x7f5ee4105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:01.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.229+0000 7f5ee9c0d700 1 -- 172.21.15.71:0/3355339704 shutdown_connections 2024-04-23T02:40:01.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.229+0000 7f5ee9c0d700 1 -- 172.21.15.71:0/3355339704 wait complete. 2024-04-23T02:40:01.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.331+0000 7f36fa46c700 1 -- 172.21.15.71:0/5800287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36f4101060 msgr2=0x7f36f41047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:01.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.331+0000 7f36fa46c700 1 --2- 172.21.15.71:0/5800287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36f4101060 0x7f36f41047b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009910 tx=0x7f36dc009c20 comp rx=0 tx=0).stop 2024-04-23T02:40:01.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.331+0000 7f36fa46c700 1 -- 172.21.15.71:0/5800287 shutdown_connections 2024-04-23T02:40:01.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.331+0000 7f36fa46c700 1 --2- 172.21.15.71:0/5800287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36f4101060 0x7f36f41047b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:01.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.331+0000 7f36fa46c700 1 --2- 172.21.15.71:0/5800287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36f41006b0 0x7f36f4100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:01.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.331+0000 7f36fa46c700 1 -- 172.21.15.71:0/5800287 >> 172.21.15.71:0/5800287 conn(0x7f36f4074e30 msgr2=0x7f36f4075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:01.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.331+0000 7f36fa46c700 1 -- 172.21.15.71:0/5800287 shutdown_connections 2024-04-23T02:40:01.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.331+0000 7f36fa46c700 1 -- 172.21.15.71:0/5800287 wait complete. 2024-04-23T02:40:01.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.331+0000 7f36fa46c700 1 Processor -- start 2024-04-23T02:40:01.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.331+0000 7f36fa46c700 1 -- start start 2024-04-23T02:40:01.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.332+0000 7f36fa46c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36f41006b0 0x7f36f41975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:01.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.332+0000 7f36fa46c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36f4101060 0x7f36f4197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:01.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.332+0000 7f36fa46c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f4198150 con 0x7f36f4101060 2024-04-23T02:40:01.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.332+0000 7f36fa46c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f41982c0 con 0x7f36f41006b0 2024-04-23T02:40:01.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.332+0000 7f36f37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36f4101060 0x7f36f4197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:01.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.332+0000 7f36f37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36f4101060 0x7f36f4197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48764/0 (socket says 172.21.15.71:48764) 2024-04-23T02:40:01.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.332+0000 7f36f37fe700 1 -- 172.21.15.71:0/2515574042 learned_addr learned my addr 172.21.15.71:0/2515574042 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:40:01.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.332+0000 7f36f3fff700 1 --2- 172.21.15.71:0/2515574042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36f41006b0 0x7f36f41975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:01.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.332+0000 7f36f37fe700 1 -- 172.21.15.71:0/2515574042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36f41006b0 msgr2=0x7f36f41975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:01.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.332+0000 7f36f37fe700 1 --2- 172.21.15.71:0/2515574042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36f41006b0 0x7f36f41975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:01.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.332+0000 7f36f37fe700 1 -- 172.21.15.71:0/2515574042 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36dc0095f0 con 0x7f36f4101060 2024-04-23T02:40:01.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.332+0000 7f36f3fff700 1 --2- 172.21.15.71:0/2515574042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36f41006b0 0x7f36f41975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:40:01.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.332+0000 7f36f37fe700 1 --2- 172.21.15.71:0/2515574042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36f4101060 0x7f36f4197b30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0092e0 tx=0x7f36dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:01.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.332+0000 7f36f17fa700 1 -- 172.21.15.71:0/2515574042 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36dc01d070 con 0x7f36f4101060 2024-04-23T02:40:01.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.332+0000 7f36fa46c700 1 -- 172.21.15.71:0/2515574042 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f41a0660 con 0x7f36f4101060 2024-04-23T02:40:01.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.332+0000 7f36f17fa700 1 -- 172.21.15.71:0/2515574042 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f36dc005940 con 0x7f36f4101060 2024-04-23T02:40:01.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.332+0000 7f36f17fa700 1 -- 172.21.15.71:0/2515574042 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36dc00fe50 con 0x7f36f4101060 2024-04-23T02:40:01.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.332+0000 7f36fa46c700 1 -- 172.21.15.71:0/2515574042 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f41a0ad0 con 0x7f36f4101060 2024-04-23T02:40:01.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.333+0000 7f36fa46c700 1 -- 172.21.15.71:0/2515574042 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f4108090 con 0x7f36f4101060 2024-04-23T02:40:01.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.334+0000 7f36f17fa700 1 -- 172.21.15.71:0/2515574042 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f36dc003a60 con 0x7f36f4101060 2024-04-23T02:40:01.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.334+0000 7f36f17fa700 1 --2- 172.21.15.71:0/2515574042 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f36e006cce0 0x7f36e006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:01.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.334+0000 7f36f17fa700 1 -- 172.21.15.71:0/2515574042 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f36dc08d6f0 con 0x7f36f4101060 2024-04-23T02:40:01.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.334+0000 7f36f3fff700 1 --2- 172.21.15.71:0/2515574042 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f36e006cce0 0x7f36e006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:01.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.335+0000 7f36f3fff700 1 --2- 172.21.15.71:0/2515574042 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f36e006cce0 0x7f36e006f1a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f36e4005950 tx=0x7f36e40058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:01.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.339+0000 7f36f17fa700 1 -- 172.21.15.71:0/2515574042 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f36dc057760 con 0x7f36f4101060 2024-04-23T02:40:01.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.575+0000 7f36fa46c700 1 -- 172.21.15.71:0/2515574042 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f36f402cc70 con 0x7f36f4101060 2024-04-23T02:40:01.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.575+0000 7f36f17fa700 1 -- 172.21.15.71:0/2515574042 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f36dc061070 con 0x7f36f4101060 2024-04-23T02:40:01.578 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:40:01.578 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:40:01.578 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:40:01.579 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:40:01.579 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:40:01.579 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:40:01.579 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:40:01.579 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:40:01.579 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:40:01.579 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:40:01.579 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:40:01.579 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:40:01.579 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:40:01.579 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:40:01.579 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:40:01.579 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:40:01.579 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:40:01.580 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:40:01.580 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:40:01.580 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:40:01.580 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:40:01.580 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:40:01.580 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:40:01.580 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:40:01.580 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:40:01.580 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:40:01.580 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:40:01.580 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:40:01.580 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:40:01.581 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:40:01.581 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:40:01.581 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:40:01.581 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:40:01.581 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:40:01.581 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:40:01.581 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:40:01.581 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:40:01.581 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:40:01.581 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:40:01.581 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:40:01.581 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:40:01.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.578+0000 7f36fa46c700 1 -- 172.21.15.71:0/2515574042 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f36e006cce0 msgr2=0x7f36e006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:01.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.578+0000 7f36fa46c700 1 --2- 172.21.15.71:0/2515574042 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f36e006cce0 0x7f36e006f1a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f36e4005950 tx=0x7f36e40058e0 comp rx=0 tx=0).stop 2024-04-23T02:40:01.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.578+0000 7f36fa46c700 1 -- 172.21.15.71:0/2515574042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36f4101060 msgr2=0x7f36f4197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:01.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.578+0000 7f36fa46c700 1 --2- 172.21.15.71:0/2515574042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36f4101060 0x7f36f4197b30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0092e0 tx=0x7f36dc00be20 comp rx=0 tx=0).stop 2024-04-23T02:40:01.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.578+0000 7f36fa46c700 1 -- 172.21.15.71:0/2515574042 shutdown_connections 2024-04-23T02:40:01.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.578+0000 7f36fa46c700 1 --2- 172.21.15.71:0/2515574042 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f36e006cce0 0x7f36e006f1a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:01.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.578+0000 7f36fa46c700 1 --2- 172.21.15.71:0/2515574042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36f41006b0 0x7f36f41975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:01.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.578+0000 7f36fa46c700 1 --2- 172.21.15.71:0/2515574042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36f4101060 0x7f36f4197b30 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:01.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.578+0000 7f36fa46c700 1 -- 172.21.15.71:0/2515574042 >> 172.21.15.71:0/2515574042 conn(0x7f36f4074e30 msgr2=0x7f36f40fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:01.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.578+0000 7f36fa46c700 1 -- 172.21.15.71:0/2515574042 shutdown_connections 2024-04-23T02:40:01.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.578+0000 7f36fa46c700 1 -- 172.21.15.71:0/2515574042 wait complete. 2024-04-23T02:40:01.583 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:40:01.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.682+0000 7f21ec0a7700 1 -- 172.21.15.71:0/844445092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21e4102700 msgr2=0x7f21e410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:01.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.682+0000 7f21ec0a7700 1 --2- 172.21.15.71:0/844445092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21e4102700 0x7f21e410abf0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009910 tx=0x7f21e0009c20 comp rx=0 tx=0).stop 2024-04-23T02:40:01.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.682+0000 7f21ec0a7700 1 -- 172.21.15.71:0/844445092 shutdown_connections 2024-04-23T02:40:01.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.682+0000 7f21ec0a7700 1 --2- 172.21.15.71:0/844445092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21e4102700 0x7f21e410abf0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:01.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.682+0000 7f21ec0a7700 1 --2- 172.21.15.71:0/844445092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21e4101de0 0x7f21e41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:01.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.682+0000 7f21ec0a7700 1 -- 172.21.15.71:0/844445092 >> 172.21.15.71:0/844445092 conn(0x7f21e40fb610 msgr2=0x7f21e40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:01.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.682+0000 7f21ec0a7700 1 -- 172.21.15.71:0/844445092 shutdown_connections 2024-04-23T02:40:01.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.682+0000 7f21ec0a7700 1 -- 172.21.15.71:0/844445092 wait complete. 2024-04-23T02:40:01.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.682+0000 7f21ec0a7700 1 Processor -- start 2024-04-23T02:40:01.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.682+0000 7f21ec0a7700 1 -- start start 2024-04-23T02:40:01.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.683+0000 7f21ec0a7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21e4101de0 0x7f21e40ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:01.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.683+0000 7f21ec0a7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21e4102700 0x7f21e40fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:01.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.683+0000 7f21ec0a7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e41004c0 con 0x7f21e4101de0 2024-04-23T02:40:01.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.683+0000 7f21ec0a7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e4100600 con 0x7f21e4102700 2024-04-23T02:40:01.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.683+0000 7f21e9e43700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21e4101de0 0x7f21e40ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:01.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.683+0000 7f21e9e43700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21e4101de0 0x7f21e40ffa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48786/0 (socket says 172.21.15.71:48786) 2024-04-23T02:40:01.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.683+0000 7f21e9e43700 1 -- 172.21.15.71:0/3328295811 learned_addr learned my addr 172.21.15.71:0/3328295811 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:40:01.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.683+0000 7f21e9642700 1 --2- 172.21.15.71:0/3328295811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21e4102700 0x7f21e40fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:01.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.683+0000 7f21e9e43700 1 -- 172.21.15.71:0/3328295811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21e4102700 msgr2=0x7f21e40fff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:01.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.683+0000 7f21e9e43700 1 --2- 172.21.15.71:0/3328295811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21e4102700 0x7f21e40fff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:01.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.683+0000 7f21e9e43700 1 -- 172.21.15.71:0/3328295811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e00095f0 con 0x7f21e4101de0 2024-04-23T02:40:01.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.683+0000 7f21e9e43700 1 --2- 172.21.15.71:0/3328295811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21e4101de0 0x7f21e40ffa40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f21d400c930 tx=0x7f21d400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:01.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.683+0000 7f21daffd700 1 -- 172.21.15.71:0/3328295811 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21d4007ad0 con 0x7f21e4101de0 2024-04-23T02:40:01.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.683+0000 7f21daffd700 1 -- 172.21.15.71:0/3328295811 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f21d4007c30 con 0x7f21e4101de0 2024-04-23T02:40:01.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.683+0000 7f21ec0a7700 1 -- 172.21.15.71:0/3328295811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e41a29c0 con 0x7f21e4101de0 2024-04-23T02:40:01.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.683+0000 7f21daffd700 1 -- 172.21.15.71:0/3328295811 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21d4018630 con 0x7f21e4101de0 2024-04-23T02:40:01.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.683+0000 7f21ec0a7700 1 -- 172.21.15.71:0/3328295811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e41a2d20 con 0x7f21e4101de0 2024-04-23T02:40:01.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.684+0000 7f21ec0a7700 1 -- 172.21.15.71:0/3328295811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e41082f0 con 0x7f21e4101de0 2024-04-23T02:40:01.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.685+0000 7f21daffd700 1 -- 172.21.15.71:0/3328295811 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f21d400f450 con 0x7f21e4101de0 2024-04-23T02:40:01.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.685+0000 7f21daffd700 1 --2- 172.21.15.71:0/3328295811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f21d006cce0 0x7f21d006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:01.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.685+0000 7f21daffd700 1 -- 172.21.15.71:0/3328295811 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f21d408c7d0 con 0x7f21e4101de0 2024-04-23T02:40:01.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.685+0000 7f21e9642700 1 --2- 172.21.15.71:0/3328295811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f21d006cce0 0x7f21d006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:01.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.686+0000 7f21e9642700 1 --2- 172.21.15.71:0/3328295811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f21d006cce0 0x7f21d006f1a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f21e4101230 tx=0x7f21e001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:01.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.690+0000 7f21daffd700 1 -- 172.21.15.71:0/3328295811 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f21d40568f0 con 0x7f21e4101de0 2024-04-23T02:40:01.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:01 smithi192 ceph-mon[108617]: from='client.35268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:40:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:01 smithi192 ceph-mon[108617]: pgmap v802: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:01 smithi192 ceph-mon[108617]: from='client.35272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:40:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:01 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3355339704' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:40:01.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.902+0000 7f21ec0a7700 1 -- 172.21.15.71:0/3328295811 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f21e40611d0 con 0x7f21d006cce0 2024-04-23T02:40:01.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.903+0000 7f21daffd700 1 -- 172.21.15.71:0/3328295811 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f21e40611d0 con 0x7f21d006cce0 2024-04-23T02:40:01.906 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:40:01.906 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:40:01.906 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:40:01.906 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:40:01.906 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:40:01.906 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:40:01.906 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:40:01.906 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:40:01.906 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:40:01.907 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:40:01.907 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:40:01.907 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:40:01.907 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:40:01.907 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:40:01.907 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:40:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.905+0000 7f21ec0a7700 1 -- 172.21.15.71:0/3328295811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f21d006cce0 msgr2=0x7f21d006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.905+0000 7f21ec0a7700 1 --2- 172.21.15.71:0/3328295811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f21d006cce0 0x7f21d006f1a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f21e4101230 tx=0x7f21e001a040 comp rx=0 tx=0).stop 2024-04-23T02:40:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.905+0000 7f21ec0a7700 1 -- 172.21.15.71:0/3328295811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21e4101de0 msgr2=0x7f21e40ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.905+0000 7f21ec0a7700 1 --2- 172.21.15.71:0/3328295811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21e4101de0 0x7f21e40ffa40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f21d400c930 tx=0x7f21d400ccf0 comp rx=0 tx=0).stop 2024-04-23T02:40:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.905+0000 7f21ec0a7700 1 -- 172.21.15.71:0/3328295811 shutdown_connections 2024-04-23T02:40:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.905+0000 7f21ec0a7700 1 --2- 172.21.15.71:0/3328295811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f21d006cce0 0x7f21d006f1a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.905+0000 7f21ec0a7700 1 --2- 172.21.15.71:0/3328295811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21e4101de0 0x7f21e40ffa40 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.905+0000 7f21ec0a7700 1 --2- 172.21.15.71:0/3328295811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21e4102700 0x7f21e40fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:01.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.905+0000 7f21ec0a7700 1 -- 172.21.15.71:0/3328295811 >> 172.21.15.71:0/3328295811 conn(0x7f21e40fb610 msgr2=0x7f21e4105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:01.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.906+0000 7f21ec0a7700 1 -- 172.21.15.71:0/3328295811 shutdown_connections 2024-04-23T02:40:01.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:01.906+0000 7f21ec0a7700 1 -- 172.21.15.71:0/3328295811 wait complete. 2024-04-23T02:40:01.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:01 smithi071 ceph-mon[144544]: from='client.35268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:40:01.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:01 smithi071 ceph-mon[144544]: pgmap v802: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:01.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:01 smithi071 ceph-mon[144544]: from='client.35272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:40:01.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:01 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3355339704' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:40:02.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.008+0000 7f091b608700 1 -- 172.21.15.71:0/428496831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09140ffb40 msgr2=0x7f09140fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:02.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.008+0000 7f091b608700 1 --2- 172.21.15.71:0/428496831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09140ffb40 0x7f09140fffc0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0910009910 tx=0x7f0910009c20 comp rx=0 tx=0).stop 2024-04-23T02:40:02.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.008+0000 7f091b608700 1 -- 172.21.15.71:0/428496831 shutdown_connections 2024-04-23T02:40:02.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.008+0000 7f091b608700 1 --2- 172.21.15.71:0/428496831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09140ffb40 0x7f09140fffc0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:02.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.008+0000 7f091b608700 1 --2- 172.21.15.71:0/428496831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0914105bf0 0x7f0914105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:02.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.008+0000 7f091b608700 1 -- 172.21.15.71:0/428496831 >> 172.21.15.71:0/428496831 conn(0x7f09140fb610 msgr2=0x7f09140fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:02.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.008+0000 7f091b608700 1 -- 172.21.15.71:0/428496831 shutdown_connections 2024-04-23T02:40:02.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.008+0000 7f091b608700 1 -- 172.21.15.71:0/428496831 wait complete. 2024-04-23T02:40:02.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.008+0000 7f091b608700 1 Processor -- start 2024-04-23T02:40:02.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.009+0000 7f091b608700 1 -- start start 2024-04-23T02:40:02.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.009+0000 7f091b608700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09140ffb40 0x7f091419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:02.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.009+0000 7f091b608700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0914105bf0 0x7f091419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:02.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.009+0000 7f091b608700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091419d0f0 con 0x7f09140ffb40 2024-04-23T02:40:02.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.009+0000 7f091b608700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09141965a0 con 0x7f0914105bf0 2024-04-23T02:40:02.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.009+0000 7f09193a4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09140ffb40 0x7f091419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:02.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.009+0000 7f09193a4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09140ffb40 0x7f091419c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48808/0 (socket says 172.21.15.71:48808) 2024-04-23T02:40:02.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.009+0000 7f09193a4700 1 -- 172.21.15.71:0/3298739031 learned_addr learned my addr 172.21.15.71:0/3298739031 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:40:02.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.009+0000 7f0918ba3700 1 --2- 172.21.15.71:0/3298739031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0914105bf0 0x7f091419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:02.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.009+0000 7f09193a4700 1 -- 172.21.15.71:0/3298739031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0914105bf0 msgr2=0x7f091419ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:02.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.009+0000 7f09193a4700 1 --2- 172.21.15.71:0/3298739031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0914105bf0 0x7f091419ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:02.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.009+0000 7f09193a4700 1 -- 172.21.15.71:0/3298739031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09100095f0 con 0x7f09140ffb40 2024-04-23T02:40:02.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.009+0000 7f09193a4700 1 --2- 172.21.15.71:0/3298739031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09140ffb40 0x7f091419c520 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f090400e940 tx=0x7f090400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:02.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.009+0000 7f090a7fc700 1 -- 172.21.15.71:0/3298739031 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f090400cba0 con 0x7f09140ffb40 2024-04-23T02:40:02.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.009+0000 7f090a7fc700 1 -- 172.21.15.71:0/3298739031 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f090400ee90 con 0x7f09140ffb40 2024-04-23T02:40:02.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.009+0000 7f090a7fc700 1 -- 172.21.15.71:0/3298739031 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09040055f0 con 0x7f09140ffb40 2024-04-23T02:40:02.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.009+0000 7f091b608700 1 -- 172.21.15.71:0/3298739031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0914196880 con 0x7f09140ffb40 2024-04-23T02:40:02.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.010+0000 7f091b608700 1 -- 172.21.15.71:0/3298739031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0914196dd0 con 0x7f09140ffb40 2024-04-23T02:40:02.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.010+0000 7f091b608700 1 -- 172.21.15.71:0/3298739031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091404f2e0 con 0x7f09140ffb40 2024-04-23T02:40:02.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.012+0000 7f090a7fc700 1 -- 172.21.15.71:0/3298739031 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0904004750 con 0x7f09140ffb40 2024-04-23T02:40:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.012+0000 7f090a7fc700 1 --2- 172.21.15.71:0/3298739031 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f090006cce0 0x7f090006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.012+0000 7f090a7fc700 1 -- 172.21.15.71:0/3298739031 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0904014070 con 0x7f09140ffb40 2024-04-23T02:40:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.012+0000 7f0918ba3700 1 --2- 172.21.15.71:0/3298739031 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f090006cce0 0x7f090006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.012+0000 7f0918ba3700 1 --2- 172.21.15.71:0/3298739031 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f090006cce0 0x7f090006f1a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f09140feb70 tx=0x7f0910005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.016+0000 7f090a7fc700 1 -- 172.21.15.71:0/3298739031 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f090405a580 con 0x7f09140ffb40 2024-04-23T02:40:02.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.288+0000 7f091b608700 1 -- 172.21.15.71:0/3298739031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0914197ab0 con 0x7f09140ffb40 2024-04-23T02:40:02.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.289+0000 7f090a7fc700 1 -- 172.21.15.71:0/3298739031 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7f090405a110 con 0x7f09140ffb40 2024-04-23T02:40:02.292 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:40:02.292 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:40:02.292 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:40:02.292 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:40:02.292 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:40:02.292 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:40:02.292 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:40:02.292 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:40:02.292 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:40:02.292 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:40:02.292 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:40:02.293 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:40:02.293 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:40:02.293 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:40:02.293 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:40:02.293 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:40:02.293 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:40:02.293 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:40:02.293 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:40:02.293 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:40:02.293 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:40:02.293 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:40:02.293 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:40:02.293 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:40:02.293 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:40:02.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.291+0000 7f091b608700 1 -- 172.21.15.71:0/3298739031 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f090006cce0 msgr2=0x7f090006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:02.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.291+0000 7f091b608700 1 --2- 172.21.15.71:0/3298739031 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f090006cce0 0x7f090006f1a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f09140feb70 tx=0x7f0910005940 comp rx=0 tx=0).stop 2024-04-23T02:40:02.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.291+0000 7f091b608700 1 -- 172.21.15.71:0/3298739031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09140ffb40 msgr2=0x7f091419c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:02.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.291+0000 7f091b608700 1 --2- 172.21.15.71:0/3298739031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09140ffb40 0x7f091419c520 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f090400e940 tx=0x7f090400ec50 comp rx=0 tx=0).stop 2024-04-23T02:40:02.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.291+0000 7f091b608700 1 -- 172.21.15.71:0/3298739031 shutdown_connections 2024-04-23T02:40:02.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.291+0000 7f091b608700 1 --2- 172.21.15.71:0/3298739031 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f090006cce0 0x7f090006f1a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:02.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.291+0000 7f091b608700 1 --2- 172.21.15.71:0/3298739031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09140ffb40 0x7f091419c520 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:02.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.291+0000 7f091b608700 1 --2- 172.21.15.71:0/3298739031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0914105bf0 0x7f091419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:02.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.291+0000 7f091b608700 1 -- 172.21.15.71:0/3298739031 >> 172.21.15.71:0/3298739031 conn(0x7f09140fb610 msgr2=0x7f0914068f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:02.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.291+0000 7f091b608700 1 -- 172.21.15.71:0/3298739031 shutdown_connections 2024-04-23T02:40:02.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:02.291+0000 7f091b608700 1 -- 172.21.15.71:0/3298739031 wait complete. 2024-04-23T02:40:02.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:02 smithi192 ceph-mon[108617]: from='client.35276 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:40:02.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:02 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2515574042' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:40:02.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:02 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3298739031' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:40:02.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:02 smithi071 ceph-mon[144544]: from='client.35276 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:40:02.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:02 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2515574042' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:40:02.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:02 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3298739031' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:40:03.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:03 smithi192 ceph-mon[108617]: from='client.35288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:40:03.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:03 smithi192 ceph-mon[108617]: pgmap v803: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:03.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:03 smithi071 ceph-mon[144544]: from='client.35288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:40:03.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:03 smithi071 ceph-mon[144544]: pgmap v803: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:05 smithi192 ceph-mon[108617]: pgmap v804: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:40:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:05 smithi071 ceph-mon[144544]: pgmap v804: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:05.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:40:07.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:07 smithi192 ceph-mon[108617]: pgmap v805: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:07.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:07 smithi071 ceph-mon[144544]: pgmap v805: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:09.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:09 smithi192 ceph-mon[108617]: pgmap v806: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:09.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:09 smithi071 ceph-mon[144544]: pgmap v806: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:11.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:11 smithi192 ceph-mon[108617]: pgmap v807: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:11.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:11 smithi071 ceph-mon[144544]: pgmap v807: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:13.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:13 smithi192 ceph-mon[108617]: pgmap v808: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:13.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:13 smithi071 ceph-mon[144544]: pgmap v808: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:15.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:15 smithi192 ceph-mon[108617]: pgmap v809: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:15.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:15 smithi071 ceph-mon[144544]: pgmap v809: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:17.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:17 smithi192 ceph-mon[108617]: pgmap v810: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:17.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:17 smithi071 ceph-mon[144544]: pgmap v810: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:18.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:18 smithi192 ceph-mon[108617]: pgmap v811: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:18.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:18 smithi071 ceph-mon[144544]: pgmap v811: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:20.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:40:20.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:40:21.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:20 smithi192 ceph-mon[108617]: pgmap v812: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:21.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:20 smithi071 ceph-mon[144544]: pgmap v812: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:23.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:23 smithi192 ceph-mon[108617]: pgmap v813: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:23.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:23 smithi071 ceph-mon[144544]: pgmap v813: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:25.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:25 smithi192 ceph-mon[108617]: pgmap v814: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:25.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:25 smithi071 ceph-mon[144544]: pgmap v814: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:27.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:27 smithi071 ceph-mon[144544]: pgmap v815: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:27.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:27 smithi192 ceph-mon[108617]: pgmap v815: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:29.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:29 smithi192 ceph-mon[108617]: pgmap v816: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:29.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:29 smithi071 ceph-mon[144544]: pgmap v816: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:31.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:31 smithi192 ceph-mon[108617]: pgmap v817: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:31.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:31 smithi071 ceph-mon[144544]: pgmap v817: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:32.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.408+0000 7fdce033a700 1 -- 172.21.15.71:0/3428524920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcd8102720 msgr2=0x7fdcd810ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:32.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.408+0000 7fdce033a700 1 --2- 172.21.15.71:0/3428524920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcd8102720 0x7fdcd810ac10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009910 tx=0x7fdcc8009c20 comp rx=0 tx=0).stop 2024-04-23T02:40:32.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.408+0000 7fdce033a700 1 -- 172.21.15.71:0/3428524920 shutdown_connections 2024-04-23T02:40:32.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.408+0000 7fdce033a700 1 --2- 172.21.15.71:0/3428524920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcd8102720 0x7fdcd810ac10 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:32.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.408+0000 7fdce033a700 1 --2- 172.21.15.71:0/3428524920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcd8101e00 0x7fdcd81021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:32.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.408+0000 7fdce033a700 1 -- 172.21.15.71:0/3428524920 >> 172.21.15.71:0/3428524920 conn(0x7fdcd80fb650 msgr2=0x7fdcd80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:32.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.408+0000 7fdce033a700 1 -- 172.21.15.71:0/3428524920 shutdown_connections 2024-04-23T02:40:32.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.408+0000 7fdce033a700 1 -- 172.21.15.71:0/3428524920 wait complete. 2024-04-23T02:40:32.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.409+0000 7fdce033a700 1 Processor -- start 2024-04-23T02:40:32.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.409+0000 7fdce033a700 1 -- start start 2024-04-23T02:40:32.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.409+0000 7fdce033a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcd8101e00 0x7fdcd819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:32.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.409+0000 7fdce033a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcd8102720 0x7fdcd819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:32.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.409+0000 7fdce033a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd819d190 con 0x7fdcd8101e00 2024-04-23T02:40:32.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.409+0000 7fdce033a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd8196640 con 0x7fdcd8102720 2024-04-23T02:40:32.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.409+0000 7fdcde0d6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcd8101e00 0x7fdcd819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:32.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.409+0000 7fdcde0d6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcd8101e00 0x7fdcd819c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54980/0 (socket says 172.21.15.71:54980) 2024-04-23T02:40:32.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.409+0000 7fdcde0d6700 1 -- 172.21.15.71:0/685240291 learned_addr learned my addr 172.21.15.71:0/685240291 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:40:32.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.409+0000 7fdcdd8d5700 1 --2- 172.21.15.71:0/685240291 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcd8102720 0x7fdcd819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:32.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.409+0000 7fdcde0d6700 1 -- 172.21.15.71:0/685240291 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcd8102720 msgr2=0x7fdcd819cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:32.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.409+0000 7fdcde0d6700 1 --2- 172.21.15.71:0/685240291 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcd8102720 0x7fdcd819cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:32.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.410+0000 7fdcde0d6700 1 -- 172.21.15.71:0/685240291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc80095f0 con 0x7fdcd8101e00 2024-04-23T02:40:32.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.410+0000 7fdcdd8d5700 1 --2- 172.21.15.71:0/685240291 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcd8102720 0x7fdcd819cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:40:32.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.410+0000 7fdcde0d6700 1 --2- 172.21.15.71:0/685240291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcd8101e00 0x7fdcd819c5c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400d8d0 tx=0x7fdcd400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:32.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.410+0000 7fdccf7fe700 1 -- 172.21.15.71:0/685240291 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcd4009940 con 0x7fdcd8101e00 2024-04-23T02:40:32.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.410+0000 7fdccf7fe700 1 -- 172.21.15.71:0/685240291 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdcd4010460 con 0x7fdcd8101e00 2024-04-23T02:40:32.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.410+0000 7fdce033a700 1 -- 172.21.15.71:0/685240291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd8196920 con 0x7fdcd8101e00 2024-04-23T02:40:32.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.410+0000 7fdccf7fe700 1 -- 172.21.15.71:0/685240291 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcd400f5f0 con 0x7fdcd8101e00 2024-04-23T02:40:32.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.410+0000 7fdce033a700 1 -- 172.21.15.71:0/685240291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd8196e70 con 0x7fdcd8101e00 2024-04-23T02:40:32.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.411+0000 7fdce033a700 1 -- 172.21.15.71:0/685240291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd8108310 con 0x7fdcd8101e00 2024-04-23T02:40:32.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.412+0000 7fdccf7fe700 1 -- 172.21.15.71:0/685240291 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fdcd40105d0 con 0x7fdcd8101e00 2024-04-23T02:40:32.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.412+0000 7fdccf7fe700 1 --2- 172.21.15.71:0/685240291 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdcc406cce0 0x7fdcc406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:32.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.412+0000 7fdccf7fe700 1 -- 172.21.15.71:0/685240291 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdcd408c5e0 con 0x7fdcd8101e00 2024-04-23T02:40:32.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.412+0000 7fdcdd8d5700 1 --2- 172.21.15.71:0/685240291 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdcc406cce0 0x7fdcc406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:32.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.413+0000 7fdcdd8d5700 1 --2- 172.21.15.71:0/685240291 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdcc406cce0 0x7fdcc406f1a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80059d0 tx=0x7fdcc8005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:32.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.417+0000 7fdccf7fe700 1 -- 172.21.15.71:0/685240291 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fdcd405a0e0 con 0x7fdcd8101e00 2024-04-23T02:40:32.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.635+0000 7fdce033a700 1 -- 172.21.15.71:0/685240291 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcd8197090 con 0x7fdcc406cce0 2024-04-23T02:40:32.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.636+0000 7fdccf7fe700 1 -- 172.21.15.71:0/685240291 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdcd8197090 con 0x7fdcc406cce0 2024-04-23T02:40:32.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.638+0000 7fdce033a700 1 -- 172.21.15.71:0/685240291 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdcc406cce0 msgr2=0x7fdcc406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:32.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.638+0000 7fdce033a700 1 --2- 172.21.15.71:0/685240291 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdcc406cce0 0x7fdcc406f1a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80059d0 tx=0x7fdcc8005960 comp rx=0 tx=0).stop 2024-04-23T02:40:32.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.638+0000 7fdce033a700 1 -- 172.21.15.71:0/685240291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcd8101e00 msgr2=0x7fdcd819c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:32.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.638+0000 7fdce033a700 1 --2- 172.21.15.71:0/685240291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcd8101e00 0x7fdcd819c5c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400d8d0 tx=0x7fdcd400dc90 comp rx=0 tx=0).stop 2024-04-23T02:40:32.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.638+0000 7fdce033a700 1 -- 172.21.15.71:0/685240291 shutdown_connections 2024-04-23T02:40:32.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.638+0000 7fdce033a700 1 --2- 172.21.15.71:0/685240291 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdcc406cce0 0x7fdcc406f1a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:32.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.638+0000 7fdce033a700 1 --2- 172.21.15.71:0/685240291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcd8101e00 0x7fdcd819c5c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:32.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.638+0000 7fdce033a700 1 --2- 172.21.15.71:0/685240291 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcd8102720 0x7fdcd819cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:32.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.638+0000 7fdce033a700 1 -- 172.21.15.71:0/685240291 >> 172.21.15.71:0/685240291 conn(0x7fdcd80fb650 msgr2=0x7fdcd8105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:32.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.638+0000 7fdce033a700 1 -- 172.21.15.71:0/685240291 shutdown_connections 2024-04-23T02:40:32.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.638+0000 7fdce033a700 1 -- 172.21.15.71:0/685240291 wait complete. 2024-04-23T02:40:32.651 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:40:32.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.742+0000 7f7f29143700 1 -- 172.21.15.71:0/1816313392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f24105bf0 msgr2=0x7f7f24105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:32.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.742+0000 7f7f29143700 1 --2- 172.21.15.71:0/1816313392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f24105bf0 0x7f7f24105fd0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7f14009910 tx=0x7f7f14009c20 comp rx=0 tx=0).stop 2024-04-23T02:40:32.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.742+0000 7f7f29143700 1 -- 172.21.15.71:0/1816313392 shutdown_connections 2024-04-23T02:40:32.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.742+0000 7f7f29143700 1 --2- 172.21.15.71:0/1816313392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f240ffb40 0x7f7f240fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:32.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.742+0000 7f7f29143700 1 --2- 172.21.15.71:0/1816313392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f24105bf0 0x7f7f24105fd0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:32.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.742+0000 7f7f29143700 1 -- 172.21.15.71:0/1816313392 >> 172.21.15.71:0/1816313392 conn(0x7f7f240fb610 msgr2=0x7f7f240fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:32.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.742+0000 7f7f29143700 1 -- 172.21.15.71:0/1816313392 shutdown_connections 2024-04-23T02:40:32.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.742+0000 7f7f29143700 1 -- 172.21.15.71:0/1816313392 wait complete. 2024-04-23T02:40:32.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.743+0000 7f7f29143700 1 Processor -- start 2024-04-23T02:40:32.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.743+0000 7f7f29143700 1 -- start start 2024-04-23T02:40:32.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.743+0000 7f7f29143700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f240ffb40 0x7f7f2419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:32.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.744+0000 7f7f29143700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f24105bf0 0x7f7f2419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:32.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.744+0000 7f7f29143700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2419d0a0 con 0x7f7f24105bf0 2024-04-23T02:40:32.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.744+0000 7f7f29143700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f241965e0 con 0x7f7f240ffb40 2024-04-23T02:40:32.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.744+0000 7f7f2259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f24105bf0 0x7f7f2419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:32.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.744+0000 7f7f2259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f24105bf0 0x7f7f2419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54996/0 (socket says 172.21.15.71:54996) 2024-04-23T02:40:32.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.744+0000 7f7f2259c700 1 -- 172.21.15.71:0/2653606365 learned_addr learned my addr 172.21.15.71:0/2653606365 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:40:32.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.744+0000 7f7f22d9d700 1 --2- 172.21.15.71:0/2653606365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f240ffb40 0x7f7f2419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:32.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.744+0000 7f7f2259c700 1 -- 172.21.15.71:0/2653606365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f240ffb40 msgr2=0x7f7f2419c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:32.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.744+0000 7f7f2259c700 1 --2- 172.21.15.71:0/2653606365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f240ffb40 0x7f7f2419c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:32.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.744+0000 7f7f2259c700 1 -- 172.21.15.71:0/2653606365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f140095f0 con 0x7f7f24105bf0 2024-04-23T02:40:32.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.744+0000 7f7f2259c700 1 --2- 172.21.15.71:0/2653606365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f24105bf0 0x7f7f2419caa0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800c930 tx=0x7f7f1800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:32.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.744+0000 7f7f0bfff700 1 -- 172.21.15.71:0/2653606365 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f18007ad0 con 0x7f7f24105bf0 2024-04-23T02:40:32.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.744+0000 7f7f0bfff700 1 -- 172.21.15.71:0/2653606365 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7f1800ce80 con 0x7f7f24105bf0 2024-04-23T02:40:32.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.744+0000 7f7f29143700 1 -- 172.21.15.71:0/2653606365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f241968c0 con 0x7f7f24105bf0 2024-04-23T02:40:32.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.744+0000 7f7f0bfff700 1 -- 172.21.15.71:0/2653606365 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f18018630 con 0x7f7f24105bf0 2024-04-23T02:40:32.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.744+0000 7f7f29143700 1 -- 172.21.15.71:0/2653606365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f24196e10 con 0x7f7f24105bf0 2024-04-23T02:40:32.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.745+0000 7f7f29143700 1 -- 172.21.15.71:0/2653606365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f2404f2e0 con 0x7f7f24105bf0 2024-04-23T02:40:32.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.746+0000 7f7f0bfff700 1 -- 172.21.15.71:0/2653606365 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7f18007c30 con 0x7f7f24105bf0 2024-04-23T02:40:32.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.746+0000 7f7f0bfff700 1 --2- 172.21.15.71:0/2653606365 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7f0c06cd30 0x7f7f0c06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:32.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.746+0000 7f7f0bfff700 1 -- 172.21.15.71:0/2653606365 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7f1808c560 con 0x7f7f24105bf0 2024-04-23T02:40:32.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.746+0000 7f7f22d9d700 1 --2- 172.21.15.71:0/2653606365 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7f0c06cd30 0x7f7f0c06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:32.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.747+0000 7f7f22d9d700 1 --2- 172.21.15.71:0/2653606365 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7f0c06cd30 0x7f7f0c06f1f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7f14006010 tx=0x7f7f1401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:32.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.751+0000 7f7f0bfff700 1 -- 172.21.15.71:0/2653606365 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7f1805a040 con 0x7f7f24105bf0 2024-04-23T02:40:32.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.965+0000 7f7f29143700 1 -- 172.21.15.71:0/2653606365 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f24197110 con 0x7f7f0c06cd30 2024-04-23T02:40:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.967+0000 7f7f0bfff700 1 -- 172.21.15.71:0/2653606365 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7f24197110 con 0x7f7f0c06cd30 2024-04-23T02:40:32.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.969+0000 7f7f29143700 1 -- 172.21.15.71:0/2653606365 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7f0c06cd30 msgr2=0x7f7f0c06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.969+0000 7f7f29143700 1 --2- 172.21.15.71:0/2653606365 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7f0c06cd30 0x7f7f0c06f1f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7f14006010 tx=0x7f7f1401a040 comp rx=0 tx=0).stop 2024-04-23T02:40:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.969+0000 7f7f29143700 1 -- 172.21.15.71:0/2653606365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f24105bf0 msgr2=0x7f7f2419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.969+0000 7f7f29143700 1 --2- 172.21.15.71:0/2653606365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f24105bf0 0x7f7f2419caa0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800c930 tx=0x7f7f1800cc40 comp rx=0 tx=0).stop 2024-04-23T02:40:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.969+0000 7f7f29143700 1 -- 172.21.15.71:0/2653606365 shutdown_connections 2024-04-23T02:40:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.969+0000 7f7f29143700 1 --2- 172.21.15.71:0/2653606365 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7f0c06cd30 0x7f7f0c06f1f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.969+0000 7f7f29143700 1 --2- 172.21.15.71:0/2653606365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f240ffb40 0x7f7f2419c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.969+0000 7f7f29143700 1 --2- 172.21.15.71:0/2653606365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f24105bf0 0x7f7f2419caa0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.969+0000 7f7f29143700 1 -- 172.21.15.71:0/2653606365 >> 172.21.15.71:0/2653606365 conn(0x7f7f240fb610 msgr2=0x7f7f24103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.969+0000 7f7f29143700 1 -- 172.21.15.71:0/2653606365 shutdown_connections 2024-04-23T02:40:32.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:32.969+0000 7f7f29143700 1 -- 172.21.15.71:0/2653606365 wait complete. 2024-04-23T02:40:33.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.071+0000 7f438b2fc700 1 -- 172.21.15.71:0/3214422789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4384101de0 msgr2=0x7f43841021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:33.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.071+0000 7f438b2fc700 1 --2- 172.21.15.71:0/3214422789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4384101de0 0x7f43841021c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4374009910 tx=0x7f4374009c20 comp rx=0 tx=0).stop 2024-04-23T02:40:33.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.071+0000 7f438b2fc700 1 -- 172.21.15.71:0/3214422789 shutdown_connections 2024-04-23T02:40:33.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.071+0000 7f438b2fc700 1 --2- 172.21.15.71:0/3214422789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4384102700 0x7f438410abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:33.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.071+0000 7f438b2fc700 1 --2- 172.21.15.71:0/3214422789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4384101de0 0x7f43841021c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:33.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.071+0000 7f438b2fc700 1 -- 172.21.15.71:0/3214422789 >> 172.21.15.71:0/3214422789 conn(0x7f43840fb610 msgr2=0x7f43840fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:33.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.071+0000 7f438b2fc700 1 -- 172.21.15.71:0/3214422789 shutdown_connections 2024-04-23T02:40:33.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.071+0000 7f438b2fc700 1 -- 172.21.15.71:0/3214422789 wait complete. 2024-04-23T02:40:33.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.071+0000 7f438b2fc700 1 Processor -- start 2024-04-23T02:40:33.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.071+0000 7f438b2fc700 1 -- start start 2024-04-23T02:40:33.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.072+0000 7f438b2fc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4384101de0 0x7f438419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:33.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.072+0000 7f438b2fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4384102700 0x7f438419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:33.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.072+0000 7f438b2fc700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438419d280 con 0x7f4384102700 2024-04-23T02:40:33.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.072+0000 7f438b2fc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4384196780 con 0x7f4384101de0 2024-04-23T02:40:33.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.072+0000 7f4388897700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4384102700 0x7f438419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:33.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.072+0000 7f4388897700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4384102700 0x7f438419cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55018/0 (socket says 172.21.15.71:55018) 2024-04-23T02:40:33.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.072+0000 7f4388897700 1 -- 172.21.15.71:0/3094265095 learned_addr learned my addr 172.21.15.71:0/3094265095 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:40:33.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.072+0000 7f4389098700 1 --2- 172.21.15.71:0/3094265095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4384101de0 0x7f438419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:33.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.072+0000 7f4388897700 1 -- 172.21.15.71:0/3094265095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4384101de0 msgr2=0x7f438419c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:33.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.072+0000 7f4388897700 1 --2- 172.21.15.71:0/3094265095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4384101de0 0x7f438419c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:33.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.072+0000 7f4388897700 1 -- 172.21.15.71:0/3094265095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43740095f0 con 0x7f4384102700 2024-04-23T02:40:33.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.072+0000 7f4388897700 1 --2- 172.21.15.71:0/3094265095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4384102700 0x7f438419cbf0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f438000b3d0 tx=0x7f438000b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:33.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.072+0000 7f437a7fc700 1 -- 172.21.15.71:0/3094265095 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4380011800 con 0x7f4384102700 2024-04-23T02:40:33.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.072+0000 7f438b2fc700 1 -- 172.21.15.71:0/3094265095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4384196a60 con 0x7f4384102700 2024-04-23T02:40:33.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.072+0000 7f437a7fc700 1 -- 172.21.15.71:0/3094265095 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f438000be90 con 0x7f4384102700 2024-04-23T02:40:33.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.072+0000 7f437a7fc700 1 -- 172.21.15.71:0/3094265095 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4380010da0 con 0x7f4384102700 2024-04-23T02:40:33.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.072+0000 7f438b2fc700 1 -- 172.21.15.71:0/3094265095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4384196fb0 con 0x7f4384102700 2024-04-23T02:40:33.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.073+0000 7f438b2fc700 1 -- 172.21.15.71:0/3094265095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43841082f0 con 0x7f4384102700 2024-04-23T02:40:33.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.074+0000 7f437a7fc700 1 -- 172.21.15.71:0/3094265095 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f438001a400 con 0x7f4384102700 2024-04-23T02:40:33.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.074+0000 7f437a7fc700 1 --2- 172.21.15.71:0/3094265095 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f437006cce0 0x7f437006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:33.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.074+0000 7f437a7fc700 1 -- 172.21.15.71:0/3094265095 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f438008bf80 con 0x7f4384102700 2024-04-23T02:40:33.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.074+0000 7f4389098700 1 --2- 172.21.15.71:0/3094265095 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f437006cce0 0x7f437006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:33.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.075+0000 7f4389098700 1 --2- 172.21.15.71:0/3094265095 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f437006cce0 0x7f437006f1a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4374006010 tx=0x7f437401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:33.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.079+0000 7f437a7fc700 1 -- 172.21.15.71:0/3094265095 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f4380059a80 con 0x7f4384102700 2024-04-23T02:40:33.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.279+0000 7f438b2fc700 1 -- 172.21.15.71:0/3094265095 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4384197230 con 0x7f437006cce0 2024-04-23T02:40:33.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.290+0000 7f437a7fc700 1 -- 172.21.15.71:0/3094265095 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4384197230 con 0x7f437006cce0 2024-04-23T02:40:33.292 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:40:33.292 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (24m) 2m ago 35m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:40:33.293 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (35m) 2m ago 35m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:40:33.293 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (34m) 2m ago 34m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:40:33.293 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (22m) 2m ago 35m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:40:33.293 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (22m) 2m ago 34m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:40:33.293 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (23m) 2m ago 35m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:40:33.293 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (18m) 2m ago 29m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:40:33.293 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (18m) 2m ago 29m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:40:33.293 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (2m) 2m ago 29m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:40:33.293 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (17m) 2m ago 29m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:40:33.293 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (26m) 2m ago 37m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:40:33.293 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (25m) 2m ago 34m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:40:33.293 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (22m) 2m ago 37m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:40:33.293 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (22m) 2m ago 34m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:40:33.294 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (25m) 2m ago 35m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:40:33.294 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (25m) 2m ago 34m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:40:33.294 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (21m) 2m ago 33m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:40:33.294 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (21m) 2m ago 33m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:40:33.294 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (20m) 2m ago 32m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:40:33.294 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (19m) 2m ago 32m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:40:33.294 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (19m) 2m ago 31m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:40:33.294 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (18m) 2m ago 31m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:40:33.294 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (24m) 2m ago 34m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:40:33.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.292+0000 7f438b2fc700 1 -- 172.21.15.71:0/3094265095 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f437006cce0 msgr2=0x7f437006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:33.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.292+0000 7f438b2fc700 1 --2- 172.21.15.71:0/3094265095 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f437006cce0 0x7f437006f1a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4374006010 tx=0x7f437401a040 comp rx=0 tx=0).stop 2024-04-23T02:40:33.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.292+0000 7f438b2fc700 1 -- 172.21.15.71:0/3094265095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4384102700 msgr2=0x7f438419cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:33.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.292+0000 7f438b2fc700 1 --2- 172.21.15.71:0/3094265095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4384102700 0x7f438419cbf0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f438000b3d0 tx=0x7f438000b6e0 comp rx=0 tx=0).stop 2024-04-23T02:40:33.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.292+0000 7f438b2fc700 1 -- 172.21.15.71:0/3094265095 shutdown_connections 2024-04-23T02:40:33.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.292+0000 7f438b2fc700 1 --2- 172.21.15.71:0/3094265095 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f437006cce0 0x7f437006f1a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:33.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.292+0000 7f438b2fc700 1 --2- 172.21.15.71:0/3094265095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4384101de0 0x7f438419c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:33.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.292+0000 7f438b2fc700 1 --2- 172.21.15.71:0/3094265095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4384102700 0x7f438419cbf0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:33.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.292+0000 7f438b2fc700 1 -- 172.21.15.71:0/3094265095 >> 172.21.15.71:0/3094265095 conn(0x7f43840fb610 msgr2=0x7f4384105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:33.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.292+0000 7f438b2fc700 1 -- 172.21.15.71:0/3094265095 shutdown_connections 2024-04-23T02:40:33.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.292+0000 7f438b2fc700 1 -- 172.21.15.71:0/3094265095 wait complete. 2024-04-23T02:40:33.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.393+0000 7f2e4f5b0700 1 -- 172.21.15.71:0/3044804636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e48101e00 msgr2=0x7f2e481021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:33.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.393+0000 7f2e4f5b0700 1 --2- 172.21.15.71:0/3044804636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e48101e00 0x7f2e481021e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009910 tx=0x7f2e38009c20 comp rx=0 tx=0).stop 2024-04-23T02:40:33.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.393+0000 7f2e4f5b0700 1 -- 172.21.15.71:0/3044804636 shutdown_connections 2024-04-23T02:40:33.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.393+0000 7f2e4f5b0700 1 --2- 172.21.15.71:0/3044804636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e48102720 0x7f2e4810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:33.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.393+0000 7f2e4f5b0700 1 --2- 172.21.15.71:0/3044804636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e48101e00 0x7f2e481021e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:33.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.393+0000 7f2e4f5b0700 1 -- 172.21.15.71:0/3044804636 >> 172.21.15.71:0/3044804636 conn(0x7f2e480fb650 msgr2=0x7f2e480fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:33.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.393+0000 7f2e4f5b0700 1 -- 172.21.15.71:0/3044804636 shutdown_connections 2024-04-23T02:40:33.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.393+0000 7f2e4f5b0700 1 -- 172.21.15.71:0/3044804636 wait complete. 2024-04-23T02:40:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.394+0000 7f2e4f5b0700 1 Processor -- start 2024-04-23T02:40:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.394+0000 7f2e4f5b0700 1 -- start start 2024-04-23T02:40:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.394+0000 7f2e4f5b0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e48101e00 0x7f2e4819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.394+0000 7f2e4f5b0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e48102720 0x7f2e4819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.394+0000 7f2e4f5b0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4819d200 con 0x7f2e48101e00 2024-04-23T02:40:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.394+0000 7f2e4f5b0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e481966b0 con 0x7f2e48102720 2024-04-23T02:40:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.394+0000 7f2e4d34c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e48101e00 0x7f2e4819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.394+0000 7f2e4d34c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e48101e00 0x7f2e4819c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55036/0 (socket says 172.21.15.71:55036) 2024-04-23T02:40:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.394+0000 7f2e4d34c700 1 -- 172.21.15.71:0/1442316676 learned_addr learned my addr 172.21.15.71:0/1442316676 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:40:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.394+0000 7f2e4cb4b700 1 --2- 172.21.15.71:0/1442316676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e48102720 0x7f2e4819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.394+0000 7f2e4d34c700 1 -- 172.21.15.71:0/1442316676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e48102720 msgr2=0x7f2e4819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.394+0000 7f2e4d34c700 1 --2- 172.21.15.71:0/1442316676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e48102720 0x7f2e4819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.394+0000 7f2e4d34c700 1 -- 172.21.15.71:0/1442316676 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e380095f0 con 0x7f2e48101e00 2024-04-23T02:40:33.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.394+0000 7f2e4d34c700 1 --2- 172.21.15.71:0/1442316676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e48101e00 0x7f2e4819c630 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2e38006010 tx=0x7f2e3800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:33.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.394+0000 7f2e3e7fc700 1 -- 172.21.15.71:0/1442316676 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e3801d070 con 0x7f2e48101e00 2024-04-23T02:40:33.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.395+0000 7f2e4f5b0700 1 -- 172.21.15.71:0/1442316676 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e48196930 con 0x7f2e48101e00 2024-04-23T02:40:33.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.395+0000 7f2e3e7fc700 1 -- 172.21.15.71:0/1442316676 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2e3800f460 con 0x7f2e48101e00 2024-04-23T02:40:33.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.395+0000 7f2e3e7fc700 1 -- 172.21.15.71:0/1442316676 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e38021640 con 0x7f2e48101e00 2024-04-23T02:40:33.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.395+0000 7f2e4f5b0700 1 -- 172.21.15.71:0/1442316676 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e48196e20 con 0x7f2e48101e00 2024-04-23T02:40:33.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.395+0000 7f2e4f5b0700 1 -- 172.21.15.71:0/1442316676 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e48108310 con 0x7f2e48101e00 2024-04-23T02:40:33.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.396+0000 7f2e3e7fc700 1 -- 172.21.15.71:0/1442316676 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2e3800fa60 con 0x7f2e48101e00 2024-04-23T02:40:33.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.396+0000 7f2e3e7fc700 1 --2- 172.21.15.71:0/1442316676 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2e3406cce0 0x7f2e3406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:33.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.396+0000 7f2e3e7fc700 1 -- 172.21.15.71:0/1442316676 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2e3808d8c0 con 0x7f2e48101e00 2024-04-23T02:40:33.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.396+0000 7f2e4cb4b700 1 --2- 172.21.15.71:0/1442316676 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2e3406cce0 0x7f2e3406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:33.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.397+0000 7f2e4cb4b700 1 --2- 172.21.15.71:0/1442316676 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2e3406cce0 0x7f2e3406f1a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2e44005fd0 tx=0x7f2e44005dd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:33.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.401+0000 7f2e3e7fc700 1 -- 172.21.15.71:0/1442316676 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2e3805b3c0 con 0x7f2e48101e00 2024-04-23T02:40:33.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.672+0000 7f2e4f5b0700 1 -- 172.21.15.71:0/1442316676 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2e48107cf0 con 0x7f2e48101e00 2024-04-23T02:40:33.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.673+0000 7f2e3e7fc700 1 -- 172.21.15.71:0/1442316676 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f2e3805af50 con 0x7f2e48101e00 2024-04-23T02:40:33.675 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:40:33.675 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:40:33.675 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:40:33.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:40:33.675 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:40:33.675 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:40:33.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:40:33.676 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:40:33.676 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:40:33.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:40:33.676 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:40:33.676 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:40:33.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:40:33.676 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:40:33.676 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:40:33.676 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:40:33.676 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:40:33.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.675+0000 7f2e4f5b0700 1 -- 172.21.15.71:0/1442316676 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2e3406cce0 msgr2=0x7f2e3406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:33.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.675+0000 7f2e4f5b0700 1 --2- 172.21.15.71:0/1442316676 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2e3406cce0 0x7f2e3406f1a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2e44005fd0 tx=0x7f2e44005dd0 comp rx=0 tx=0).stop 2024-04-23T02:40:33.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.675+0000 7f2e4f5b0700 1 -- 172.21.15.71:0/1442316676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e48101e00 msgr2=0x7f2e4819c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:33.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.675+0000 7f2e4f5b0700 1 --2- 172.21.15.71:0/1442316676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e48101e00 0x7f2e4819c630 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2e38006010 tx=0x7f2e3800be20 comp rx=0 tx=0).stop 2024-04-23T02:40:33.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.675+0000 7f2e4f5b0700 1 -- 172.21.15.71:0/1442316676 shutdown_connections 2024-04-23T02:40:33.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.675+0000 7f2e4f5b0700 1 --2- 172.21.15.71:0/1442316676 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2e3406cce0 0x7f2e3406f1a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:33.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.675+0000 7f2e4f5b0700 1 --2- 172.21.15.71:0/1442316676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e48101e00 0x7f2e4819c630 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:33.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.675+0000 7f2e4f5b0700 1 --2- 172.21.15.71:0/1442316676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e48102720 0x7f2e4819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:33.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.675+0000 7f2e4f5b0700 1 -- 172.21.15.71:0/1442316676 >> 172.21.15.71:0/1442316676 conn(0x7f2e480fb650 msgr2=0x7f2e48105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:33.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.675+0000 7f2e4f5b0700 1 -- 172.21.15.71:0/1442316676 shutdown_connections 2024-04-23T02:40:33.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.675+0000 7f2e4f5b0700 1 -- 172.21.15.71:0/1442316676 wait complete. 2024-04-23T02:40:33.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.777+0000 7f217406b700 1 -- 172.21.15.71:0/2916351476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f216c100e90 msgr2=0x7f216c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:33.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.777+0000 7f217406b700 1 --2- 172.21.15.71:0/2916351476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f216c100e90 0x7f216c101270 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f215c009910 tx=0x7f215c009c20 comp rx=0 tx=0).stop 2024-04-23T02:40:33.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.777+0000 7f217406b700 1 -- 172.21.15.71:0/2916351476 shutdown_connections 2024-04-23T02:40:33.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.777+0000 7f217406b700 1 --2- 172.21.15.71:0/2916351476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f216c101840 0x7f216c105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:33.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.777+0000 7f217406b700 1 --2- 172.21.15.71:0/2916351476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f216c100e90 0x7f216c101270 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:33.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.777+0000 7f217406b700 1 -- 172.21.15.71:0/2916351476 >> 172.21.15.71:0/2916351476 conn(0x7f216c0fc740 msgr2=0x7f216c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:33.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.777+0000 7f217406b700 1 -- 172.21.15.71:0/2916351476 shutdown_connections 2024-04-23T02:40:33.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.777+0000 7f217406b700 1 -- 172.21.15.71:0/2916351476 wait complete. 2024-04-23T02:40:33.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.778+0000 7f217406b700 1 Processor -- start 2024-04-23T02:40:33.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.778+0000 7f217406b700 1 -- start start 2024-04-23T02:40:33.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.778+0000 7f217406b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f216c100e90 0x7f216c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:33.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.778+0000 7f217406b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f216c101840 0x7f216c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:33.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.778+0000 7f217406b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c073b10 con 0x7f216c100e90 2024-04-23T02:40:33.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.778+0000 7f217406b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c073c50 con 0x7f216c101840 2024-04-23T02:40:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.778+0000 7f2171e07700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f216c100e90 0x7f216c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.778+0000 7f2171e07700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f216c100e90 0x7f216c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55052/0 (socket says 172.21.15.71:55052) 2024-04-23T02:40:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.778+0000 7f2171e07700 1 -- 172.21.15.71:0/693338461 learned_addr learned my addr 172.21.15.71:0/693338461 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:40:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.778+0000 7f2171606700 1 --2- 172.21.15.71:0/693338461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f216c101840 0x7f216c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.779+0000 7f2171e07700 1 -- 172.21.15.71:0/693338461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f216c101840 msgr2=0x7f216c0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.779+0000 7f2171e07700 1 --2- 172.21.15.71:0/693338461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f216c101840 0x7f216c0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.779+0000 7f2171e07700 1 -- 172.21.15.71:0/693338461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c0095f0 con 0x7f216c100e90 2024-04-23T02:40:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.779+0000 7f2171606700 1 --2- 172.21.15.71:0/693338461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f216c101840 0x7f216c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:40:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.779+0000 7f2171e07700 1 --2- 172.21.15.71:0/693338461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f216c100e90 0x7f216c073090 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f215c006010 tx=0x7f215c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.779+0000 7f2162ffd700 1 -- 172.21.15.71:0/693338461 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f215c01d070 con 0x7f216c100e90 2024-04-23T02:40:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.779+0000 7f2162ffd700 1 -- 172.21.15.71:0/693338461 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f215c00f460 con 0x7f216c100e90 2024-04-23T02:40:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.779+0000 7f217406b700 1 -- 172.21.15.71:0/693338461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216c073ed0 con 0x7f216c100e90 2024-04-23T02:40:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.779+0000 7f2162ffd700 1 -- 172.21.15.71:0/693338461 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f215c021640 con 0x7f216c100e90 2024-04-23T02:40:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.779+0000 7f217406b700 1 -- 172.21.15.71:0/693338461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216c1a2ca0 con 0x7f216c100e90 2024-04-23T02:40:33.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.780+0000 7f217406b700 1 -- 172.21.15.71:0/693338461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216c109230 con 0x7f216c100e90 2024-04-23T02:40:33.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.780+0000 7f2162ffd700 1 -- 172.21.15.71:0/693338461 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f215c00fa20 con 0x7f216c100e90 2024-04-23T02:40:33.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.781+0000 7f2162ffd700 1 --2- 172.21.15.71:0/693338461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f215806cce0 0x7f215806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:33.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.781+0000 7f2162ffd700 1 -- 172.21.15.71:0/693338461 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f215c08d8c0 con 0x7f216c100e90 2024-04-23T02:40:33.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.781+0000 7f2171606700 1 --2- 172.21.15.71:0/693338461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f215806cce0 0x7f215806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:33.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.781+0000 7f2171606700 1 --2- 172.21.15.71:0/693338461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f215806cce0 0x7f215806f1a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2168005fd0 tx=0x7f2168005e20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:33.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:33.785+0000 7f2162ffd700 1 -- 172.21.15.71:0/693338461 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f215c057930 con 0x7f216c100e90 2024-04-23T02:40:33.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:33 smithi192 ceph-mon[108617]: pgmap v818: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:33.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:33 smithi071 ceph-mon[144544]: pgmap v818: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:34.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.023+0000 7f217406b700 1 -- 172.21.15.71:0/693338461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f216c108c10 con 0x7f216c100e90 2024-04-23T02:40:34.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.023+0000 7f2162ffd700 1 -- 172.21.15.71:0/693338461 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f215c05af50 con 0x7f216c100e90 2024-04-23T02:40:34.026 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:40:34.026 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:40:34.026 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:40:34.026 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:40:34.026 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:40:34.027 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:40:34.027 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:40:34.027 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:40:34.027 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:40:34.027 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:40:34.027 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:40:34.027 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:40:34.027 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:40:34.027 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:40:34.027 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:40:34.027 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:40:34.027 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:40:34.027 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:40:34.027 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:40:34.029 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:40:34.029 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:40:34.029 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:40:34.029 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:40:34.029 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:40:34.029 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:40:34.029 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:40:34.029 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:40:34.029 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:40:34.030 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:40:34.030 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:40:34.030 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:40:34.030 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:40:34.030 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:40:34.030 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:40:34.030 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:40:34.030 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:40:34.030 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:40:34.030 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:40:34.030 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:40:34.030 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:40:34.030 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:40:34.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.025+0000 7f217406b700 1 -- 172.21.15.71:0/693338461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f215806cce0 msgr2=0x7f215806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:34.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.025+0000 7f217406b700 1 --2- 172.21.15.71:0/693338461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f215806cce0 0x7f215806f1a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2168005fd0 tx=0x7f2168005e20 comp rx=0 tx=0).stop 2024-04-23T02:40:34.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.025+0000 7f217406b700 1 -- 172.21.15.71:0/693338461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f216c100e90 msgr2=0x7f216c073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:34.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.025+0000 7f217406b700 1 --2- 172.21.15.71:0/693338461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f216c100e90 0x7f216c073090 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f215c006010 tx=0x7f215c00be20 comp rx=0 tx=0).stop 2024-04-23T02:40:34.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.025+0000 7f217406b700 1 -- 172.21.15.71:0/693338461 shutdown_connections 2024-04-23T02:40:34.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.025+0000 7f217406b700 1 --2- 172.21.15.71:0/693338461 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f215806cce0 0x7f215806f1a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:34.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.026+0000 7f217406b700 1 --2- 172.21.15.71:0/693338461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f216c100e90 0x7f216c073090 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:34.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.026+0000 7f217406b700 1 --2- 172.21.15.71:0/693338461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f216c101840 0x7f216c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:34.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.026+0000 7f217406b700 1 -- 172.21.15.71:0/693338461 >> 172.21.15.71:0/693338461 conn(0x7f216c0fc740 msgr2=0x7f216c0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:34.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.026+0000 7f217406b700 1 -- 172.21.15.71:0/693338461 shutdown_connections 2024-04-23T02:40:34.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.026+0000 7f217406b700 1 -- 172.21.15.71:0/693338461 wait complete. 2024-04-23T02:40:34.032 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:40:34.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.128+0000 7f979bf10700 1 -- 172.21.15.71:0/3203262530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97940ffda0 msgr2=0x7f9794100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:34.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.128+0000 7f979bf10700 1 --2- 172.21.15.71:0/3203262530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97940ffda0 0x7f9794100180 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9790009910 tx=0x7f9790009c20 comp rx=0 tx=0).stop 2024-04-23T02:40:34.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.128+0000 7f979bf10700 1 -- 172.21.15.71:0/3203262530 shutdown_connections 2024-04-23T02:40:34.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.128+0000 7f979bf10700 1 --2- 172.21.15.71:0/3203262530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9794100750 0x7f97941047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:34.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.128+0000 7f979bf10700 1 --2- 172.21.15.71:0/3203262530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97940ffda0 0x7f9794100180 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:34.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.128+0000 7f979bf10700 1 -- 172.21.15.71:0/3203262530 >> 172.21.15.71:0/3203262530 conn(0x7f97940fb650 msgr2=0x7f97940fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:34.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.128+0000 7f979bf10700 1 -- 172.21.15.71:0/3203262530 shutdown_connections 2024-04-23T02:40:34.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.128+0000 7f979bf10700 1 -- 172.21.15.71:0/3203262530 wait complete. 2024-04-23T02:40:34.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.129+0000 7f979bf10700 1 Processor -- start 2024-04-23T02:40:34.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.129+0000 7f979bf10700 1 -- start start 2024-04-23T02:40:34.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.130+0000 7f979bf10700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97940ffda0 0x7f97941956d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:34.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.130+0000 7f979bf10700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9794100750 0x7f9794195c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:34.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.130+0000 7f979bf10700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97941962a0 con 0x7f97940ffda0 2024-04-23T02:40:34.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.130+0000 7f979bf10700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9794198fd0 con 0x7f9794100750 2024-04-23T02:40:34.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.130+0000 7f9799cac700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97940ffda0 0x7f97941956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:34.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.130+0000 7f9799cac700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97940ffda0 0x7f97941956d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55068/0 (socket says 172.21.15.71:55068) 2024-04-23T02:40:34.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.130+0000 7f9799cac700 1 -- 172.21.15.71:0/3697639264 learned_addr learned my addr 172.21.15.71:0/3697639264 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:40:34.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.130+0000 7f97994ab700 1 --2- 172.21.15.71:0/3697639264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9794100750 0x7f9794195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:34.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.130+0000 7f9799cac700 1 -- 172.21.15.71:0/3697639264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9794100750 msgr2=0x7f9794195c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:34.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.130+0000 7f9799cac700 1 --2- 172.21.15.71:0/3697639264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9794100750 0x7f9794195c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:34.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.130+0000 7f9799cac700 1 -- 172.21.15.71:0/3697639264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97900095f0 con 0x7f97940ffda0 2024-04-23T02:40:34.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.130+0000 7f97994ab700 1 --2- 172.21.15.71:0/3697639264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9794100750 0x7f9794195c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:40:34.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.130+0000 7f9799cac700 1 --2- 172.21.15.71:0/3697639264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97940ffda0 0x7f97941956d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9790006010 tx=0x7f979000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:34.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.130+0000 7f978affd700 1 -- 172.21.15.71:0/3697639264 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f979001d070 con 0x7f97940ffda0 2024-04-23T02:40:34.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.130+0000 7f978affd700 1 -- 172.21.15.71:0/3697639264 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f979000f460 con 0x7f97940ffda0 2024-04-23T02:40:34.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.130+0000 7f979bf10700 1 -- 172.21.15.71:0/3697639264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9794199250 con 0x7f97940ffda0 2024-04-23T02:40:34.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.130+0000 7f978affd700 1 -- 172.21.15.71:0/3697639264 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9790021640 con 0x7f97940ffda0 2024-04-23T02:40:34.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.130+0000 7f979bf10700 1 -- 172.21.15.71:0/3697639264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9794199740 con 0x7f97940ffda0 2024-04-23T02:40:34.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.131+0000 7f979bf10700 1 -- 172.21.15.71:0/3697639264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9794108140 con 0x7f97940ffda0 2024-04-23T02:40:34.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.132+0000 7f978affd700 1 -- 172.21.15.71:0/3697639264 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f979000fa60 con 0x7f97940ffda0 2024-04-23T02:40:34.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.132+0000 7f978affd700 1 --2- 172.21.15.71:0/3697639264 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f978006cce0 0x7f978006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:34.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.132+0000 7f978affd700 1 -- 172.21.15.71:0/3697639264 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f979008d880 con 0x7f97940ffda0 2024-04-23T02:40:34.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.132+0000 7f97994ab700 1 --2- 172.21.15.71:0/3697639264 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f978006cce0 0x7f978006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:34.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.132+0000 7f97994ab700 1 --2- 172.21.15.71:0/3697639264 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f978006cce0 0x7f978006f1a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9794196cf0 tx=0x7f9784005f50 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:34.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.136+0000 7f978affd700 1 -- 172.21.15.71:0/3697639264 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f97900578f0 con 0x7f97940ffda0 2024-04-23T02:40:34.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.347+0000 7f979bf10700 1 -- 172.21.15.71:0/3697639264 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9794196b70 con 0x7f978006cce0 2024-04-23T02:40:34.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.349+0000 7f978affd700 1 -- 172.21.15.71:0/3697639264 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f978400a820 con 0x7f978006cce0 2024-04-23T02:40:34.351 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:40:34.351 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:40:34.351 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:40:34.351 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:40:34.351 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:40:34.351 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:40:34.351 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:40:34.352 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:40:34.352 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:40:34.352 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:40:34.352 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:40:34.352 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:40:34.352 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:40:34.352 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:40:34.352 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:40:34.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.350+0000 7f979bf10700 1 -- 172.21.15.71:0/3697639264 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f978006cce0 msgr2=0x7f978006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:34.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.350+0000 7f979bf10700 1 --2- 172.21.15.71:0/3697639264 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f978006cce0 0x7f978006f1a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9794196cf0 tx=0x7f9784005f50 comp rx=0 tx=0).stop 2024-04-23T02:40:34.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.351+0000 7f979bf10700 1 -- 172.21.15.71:0/3697639264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97940ffda0 msgr2=0x7f97941956d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:34.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.351+0000 7f979bf10700 1 --2- 172.21.15.71:0/3697639264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97940ffda0 0x7f97941956d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9790006010 tx=0x7f979000be20 comp rx=0 tx=0).stop 2024-04-23T02:40:34.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.351+0000 7f979bf10700 1 -- 172.21.15.71:0/3697639264 shutdown_connections 2024-04-23T02:40:34.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.351+0000 7f979bf10700 1 --2- 172.21.15.71:0/3697639264 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f978006cce0 0x7f978006f1a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:34.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.351+0000 7f979bf10700 1 --2- 172.21.15.71:0/3697639264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97940ffda0 0x7f97941956d0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:34.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.351+0000 7f979bf10700 1 --2- 172.21.15.71:0/3697639264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9794100750 0x7f9794195c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:34.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.351+0000 7f979bf10700 1 -- 172.21.15.71:0/3697639264 >> 172.21.15.71:0/3697639264 conn(0x7f97940fb650 msgr2=0x7f97940fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:34.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.351+0000 7f979bf10700 1 -- 172.21.15.71:0/3697639264 shutdown_connections 2024-04-23T02:40:34.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.351+0000 7f979bf10700 1 -- 172.21.15.71:0/3697639264 wait complete. 2024-04-23T02:40:34.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.453+0000 7f087f9e7700 1 -- 172.21.15.71:0/2011447230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0878102700 msgr2=0x7f087810abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:34.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.453+0000 7f087f9e7700 1 --2- 172.21.15.71:0/2011447230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0878102700 0x7f087810abf0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0874009910 tx=0x7f0874009c20 comp rx=0 tx=0).stop 2024-04-23T02:40:34.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.453+0000 7f087f9e7700 1 -- 172.21.15.71:0/2011447230 shutdown_connections 2024-04-23T02:40:34.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.453+0000 7f087f9e7700 1 --2- 172.21.15.71:0/2011447230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0878102700 0x7f087810abf0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:34.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.453+0000 7f087f9e7700 1 --2- 172.21.15.71:0/2011447230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0878101de0 0x7f08781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:34.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.453+0000 7f087f9e7700 1 -- 172.21.15.71:0/2011447230 >> 172.21.15.71:0/2011447230 conn(0x7f08780fb610 msgr2=0x7f08780fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:34.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.453+0000 7f087f9e7700 1 -- 172.21.15.71:0/2011447230 shutdown_connections 2024-04-23T02:40:34.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.453+0000 7f087f9e7700 1 -- 172.21.15.71:0/2011447230 wait complete. 2024-04-23T02:40:34.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.454+0000 7f087f9e7700 1 Processor -- start 2024-04-23T02:40:34.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.454+0000 7f087f9e7700 1 -- start start 2024-04-23T02:40:34.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.454+0000 7f087f9e7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0878101de0 0x7f087819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:34.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.454+0000 7f087f9e7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0878102700 0x7f087819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:34.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.454+0000 7f087f9e7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087819d190 con 0x7f0878102700 2024-04-23T02:40:34.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.454+0000 7f087f9e7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0878196640 con 0x7f0878101de0 2024-04-23T02:40:34.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.454+0000 7f087cf82700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0878102700 0x7f087819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:34.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.454+0000 7f087cf82700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0878102700 0x7f087819cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55088/0 (socket says 172.21.15.71:55088) 2024-04-23T02:40:34.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.454+0000 7f087cf82700 1 -- 172.21.15.71:0/3626613505 learned_addr learned my addr 172.21.15.71:0/3626613505 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:40:34.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.454+0000 7f087d783700 1 --2- 172.21.15.71:0/3626613505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0878101de0 0x7f087819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:34.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.454+0000 7f087cf82700 1 -- 172.21.15.71:0/3626613505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0878101de0 msgr2=0x7f087819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:34.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.454+0000 7f087cf82700 1 --2- 172.21.15.71:0/3626613505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0878101de0 0x7f087819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:34.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.454+0000 7f087cf82700 1 -- 172.21.15.71:0/3626613505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08740095f0 con 0x7f0878102700 2024-04-23T02:40:34.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.454+0000 7f087cf82700 1 --2- 172.21.15.71:0/3626613505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0878102700 0x7f087819cb00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f08740092e0 tx=0x7f087400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:34.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.455+0000 7f086e7fc700 1 -- 172.21.15.71:0/3626613505 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f087401d070 con 0x7f0878102700 2024-04-23T02:40:34.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.455+0000 7f086e7fc700 1 -- 172.21.15.71:0/3626613505 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0874003d30 con 0x7f0878102700 2024-04-23T02:40:34.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.455+0000 7f087f9e7700 1 -- 172.21.15.71:0/3626613505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08781968c0 con 0x7f0878102700 2024-04-23T02:40:34.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.455+0000 7f086e7fc700 1 -- 172.21.15.71:0/3626613505 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f087400fdb0 con 0x7f0878102700 2024-04-23T02:40:34.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.455+0000 7f087f9e7700 1 -- 172.21.15.71:0/3626613505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0878196db0 con 0x7f0878102700 2024-04-23T02:40:34.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.456+0000 7f087f9e7700 1 -- 172.21.15.71:0/3626613505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08781082f0 con 0x7f0878102700 2024-04-23T02:40:34.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.456+0000 7f086e7fc700 1 -- 172.21.15.71:0/3626613505 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f087400f5f0 con 0x7f0878102700 2024-04-23T02:40:34.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.456+0000 7f086e7fc700 1 --2- 172.21.15.71:0/3626613505 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f086406cce0 0x7f086406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:40:34.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.456+0000 7f086e7fc700 1 -- 172.21.15.71:0/3626613505 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f087408d6b0 con 0x7f0878102700 2024-04-23T02:40:34.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.457+0000 7f087d783700 1 --2- 172.21.15.71:0/3626613505 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f086406cce0 0x7f086406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:40:34.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.457+0000 7f087d783700 1 --2- 172.21.15.71:0/3626613505 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f086406cce0 0x7f086406f1a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0868005950 tx=0x7f086800b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:40:34.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.461+0000 7f086e7fc700 1 -- 172.21.15.71:0/3626613505 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f087405b1b0 con 0x7f0878102700 2024-04-23T02:40:34.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.735+0000 7f087f9e7700 1 -- 172.21.15.71:0/3626613505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f087804f2e0 con 0x7f0878102700 2024-04-23T02:40:34.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.736+0000 7f086e7fc700 1 -- 172.21.15.71:0/3626613505 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7f087405ad40 con 0x7f0878102700 2024-04-23T02:40:34.739 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:40:34.739 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:40:34.739 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:40:34.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:40:34.739 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:40:34.739 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:40:34.739 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:40:34.739 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:40:34.739 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:40:34.739 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:40:34.739 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:40:34.739 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:40:34.739 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:40:34.740 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:40:34.740 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:40:34.740 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:40:34.740 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:40:34.740 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:40:34.740 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:40:34.740 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:40:34.740 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:40:34.740 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:40:34.740 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:40:34.740 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:40:34.740 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:40:34.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.738+0000 7f087f9e7700 1 -- 172.21.15.71:0/3626613505 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f086406cce0 msgr2=0x7f086406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:34.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.738+0000 7f087f9e7700 1 --2- 172.21.15.71:0/3626613505 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f086406cce0 0x7f086406f1a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0868005950 tx=0x7f086800b410 comp rx=0 tx=0).stop 2024-04-23T02:40:34.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.738+0000 7f087f9e7700 1 -- 172.21.15.71:0/3626613505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0878102700 msgr2=0x7f087819cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:40:34.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.738+0000 7f087f9e7700 1 --2- 172.21.15.71:0/3626613505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0878102700 0x7f087819cb00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f08740092e0 tx=0x7f087400bd70 comp rx=0 tx=0).stop 2024-04-23T02:40:34.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.738+0000 7f087f9e7700 1 -- 172.21.15.71:0/3626613505 shutdown_connections 2024-04-23T02:40:34.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.738+0000 7f087f9e7700 1 --2- 172.21.15.71:0/3626613505 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f086406cce0 0x7f086406f1a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:34.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.738+0000 7f087f9e7700 1 --2- 172.21.15.71:0/3626613505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0878101de0 0x7f087819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:34.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.738+0000 7f087f9e7700 1 --2- 172.21.15.71:0/3626613505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0878102700 0x7f087819cb00 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:40:34.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.738+0000 7f087f9e7700 1 -- 172.21.15.71:0/3626613505 >> 172.21.15.71:0/3626613505 conn(0x7f08780fb610 msgr2=0x7f0878105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:40:34.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.738+0000 7f087f9e7700 1 -- 172.21.15.71:0/3626613505 shutdown_connections 2024-04-23T02:40:34.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:40:34.738+0000 7f087f9e7700 1 -- 172.21.15.71:0/3626613505 wait complete. 2024-04-23T02:40:34.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:34 smithi192 ceph-mon[108617]: from='client.35296 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:40:34.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:34 smithi192 ceph-mon[108617]: from='client.35300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:40:34.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:34 smithi192 ceph-mon[108617]: from='client.35304 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:40:34.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:34 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1442316676' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:40:34.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:34 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/693338461' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:40:34.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:34 smithi071 ceph-mon[144544]: from='client.35296 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:40:34.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:34 smithi071 ceph-mon[144544]: from='client.35300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:40:34.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:34 smithi071 ceph-mon[144544]: from='client.35304 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:40:34.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:34 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1442316676' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:40:34.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:34 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/693338461' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:40:35.781 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:35 smithi192 ceph-mon[108617]: from='client.35316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:40:35.781 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:35 smithi192 ceph-mon[108617]: pgmap v819: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:35.781 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:35 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3626613505' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:40:35.782 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:40:35.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:35 smithi071 ceph-mon[144544]: from='client.35316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:40:35.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:35 smithi071 ceph-mon[144544]: pgmap v819: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:35 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3626613505' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:40:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:40:37.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:37 smithi192 ceph-mon[108617]: pgmap v820: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:37.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:37 smithi071 ceph-mon[144544]: pgmap v820: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:39.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:39 smithi192 ceph-mon[108617]: pgmap v821: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:39.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:39 smithi071 ceph-mon[144544]: pgmap v821: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:41.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:41 smithi192 ceph-mon[108617]: pgmap v822: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:41.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:41 smithi071 ceph-mon[144544]: pgmap v822: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:43.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:43 smithi192 ceph-mon[108617]: pgmap v823: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:43.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:43 smithi071 ceph-mon[144544]: pgmap v823: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:45.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:45 smithi192 ceph-mon[108617]: pgmap v824: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:45.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:45 smithi071 ceph-mon[144544]: pgmap v824: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:47.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:47 smithi192 ceph-mon[108617]: pgmap v825: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:47.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:47 smithi071 ceph-mon[144544]: pgmap v825: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:49.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:49 smithi192 ceph-mon[108617]: pgmap v826: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:49.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:49 smithi071 ceph-mon[144544]: pgmap v826: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:40:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:40:51.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:51 smithi192 ceph-mon[108617]: pgmap v827: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:51.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:51 smithi071 ceph-mon[144544]: pgmap v827: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:53.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:53 smithi192 ceph-mon[108617]: pgmap v828: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:53.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:53 smithi071 ceph-mon[144544]: pgmap v828: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:55.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:55 smithi192 ceph-mon[108617]: pgmap v829: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:55.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:55 smithi071 ceph-mon[144544]: pgmap v829: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:57.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:57 smithi192 ceph-mon[108617]: pgmap v830: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:57.997 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:57 smithi071 ceph-mon[144544]: pgmap v830: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:59.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:40:59 smithi192 ceph-mon[108617]: pgmap v831: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:40:59.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:40:59 smithi071 ceph-mon[144544]: pgmap v831: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:01.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:01 smithi192 ceph-mon[108617]: pgmap v832: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:01.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:01 smithi071 ceph-mon[144544]: pgmap v832: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:02.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:02 smithi192 ceph-mon[108617]: pgmap v833: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:02.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:02 smithi071 ceph-mon[144544]: pgmap v833: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:04.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.853+0000 7fe796b55700 1 -- 172.21.15.71:0/2917600981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7900ffc20 msgr2=0x7fe790100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:04.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.853+0000 7fe796b55700 1 --2- 172.21.15.71:0/2917600981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7900ffc20 0x7fe790100080 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe77c009910 tx=0x7fe77c009c20 comp rx=0 tx=0).stop 2024-04-23T02:41:04.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.853+0000 7fe796b55700 1 -- 172.21.15.71:0/2917600981 shutdown_connections 2024-04-23T02:41:04.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.853+0000 7fe796b55700 1 --2- 172.21.15.71:0/2917600981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7900ffc20 0x7fe790100080 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:04.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.853+0000 7fe796b55700 1 --2- 172.21.15.71:0/2917600981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe790105bd0 0x7fe790105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:04.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.853+0000 7fe796b55700 1 -- 172.21.15.71:0/2917600981 >> 172.21.15.71:0/2917600981 conn(0x7fe790075670 msgr2=0x7fe790075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:04.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.854+0000 7fe796b55700 1 -- 172.21.15.71:0/2917600981 shutdown_connections 2024-04-23T02:41:04.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.854+0000 7fe796b55700 1 -- 172.21.15.71:0/2917600981 wait complete. 2024-04-23T02:41:04.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.854+0000 7fe796b55700 1 Processor -- start 2024-04-23T02:41:04.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.854+0000 7fe796b55700 1 -- start start 2024-04-23T02:41:04.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.854+0000 7fe796b55700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7900ffc20 0x7fe79019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:04.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.854+0000 7fe796b55700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe790105bd0 0x7fe79019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:04.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.854+0000 7fe796b55700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79019d0c0 con 0x7fe7900ffc20 2024-04-23T02:41:04.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.854+0000 7fe796b55700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe790196600 con 0x7fe790105bd0 2024-04-23T02:41:04.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.854+0000 7fe7948f1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7900ffc20 0x7fe79019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:04.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.854+0000 7fe7948f1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7900ffc20 0x7fe79019c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54188/0 (socket says 172.21.15.71:54188) 2024-04-23T02:41:04.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.855+0000 7fe7948f1700 1 -- 172.21.15.71:0/3809291052 learned_addr learned my addr 172.21.15.71:0/3809291052 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:41:04.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.855+0000 7fe78ffff700 1 --2- 172.21.15.71:0/3809291052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe790105bd0 0x7fe79019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:04.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.855+0000 7fe7948f1700 1 -- 172.21.15.71:0/3809291052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe790105bd0 msgr2=0x7fe79019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:04.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.855+0000 7fe7948f1700 1 --2- 172.21.15.71:0/3809291052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe790105bd0 0x7fe79019cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:04.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.855+0000 7fe7948f1700 1 -- 172.21.15.71:0/3809291052 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c0095f0 con 0x7fe7900ffc20 2024-04-23T02:41:04.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.855+0000 7fe7948f1700 1 --2- 172.21.15.71:0/3809291052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7900ffc20 0x7fe79019c580 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe78400c930 tx=0x7fe78400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:04.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.855+0000 7fe78dffb700 1 -- 172.21.15.71:0/3809291052 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe784007ad0 con 0x7fe7900ffc20 2024-04-23T02:41:04.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.855+0000 7fe78dffb700 1 -- 172.21.15.71:0/3809291052 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe784007c30 con 0x7fe7900ffc20 2024-04-23T02:41:04.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.855+0000 7fe796b55700 1 -- 172.21.15.71:0/3809291052 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7901968e0 con 0x7fe7900ffc20 2024-04-23T02:41:04.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.855+0000 7fe78dffb700 1 -- 172.21.15.71:0/3809291052 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe784018630 con 0x7fe7900ffc20 2024-04-23T02:41:04.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.855+0000 7fe796b55700 1 -- 172.21.15.71:0/3809291052 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe790196e30 con 0x7fe7900ffc20 2024-04-23T02:41:04.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.857+0000 7fe796b55700 1 -- 172.21.15.71:0/3809291052 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79004f2e0 con 0x7fe7900ffc20 2024-04-23T02:41:04.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.857+0000 7fe78dffb700 1 -- 172.21.15.71:0/3809291052 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe78400f450 con 0x7fe7900ffc20 2024-04-23T02:41:04.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.857+0000 7fe78dffb700 1 --2- 172.21.15.71:0/3809291052 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe78006cce0 0x7fe78006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:04.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.857+0000 7fe78dffb700 1 -- 172.21.15.71:0/3809291052 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe78408c870 con 0x7fe7900ffc20 2024-04-23T02:41:04.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.857+0000 7fe78ffff700 1 --2- 172.21.15.71:0/3809291052 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe78006cce0 0x7fe78006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:04.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.858+0000 7fe78ffff700 1 --2- 172.21.15.71:0/3809291052 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe78006cce0 0x7fe78006f1a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe790075de0 tx=0x7fe77c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:04.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:04.862+0000 7fe78dffb700 1 -- 172.21.15.71:0/3809291052 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe78405a380 con 0x7fe7900ffc20 2024-04-23T02:41:05.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.079+0000 7fe796b55700 1 -- 172.21.15.71:0/3809291052 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7901970e0 con 0x7fe78006cce0 2024-04-23T02:41:05.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.080+0000 7fe78dffb700 1 -- 172.21.15.71:0/3809291052 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe7901970e0 con 0x7fe78006cce0 2024-04-23T02:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.082+0000 7fe796b55700 1 -- 172.21.15.71:0/3809291052 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe78006cce0 msgr2=0x7fe78006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.082+0000 7fe796b55700 1 --2- 172.21.15.71:0/3809291052 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe78006cce0 0x7fe78006f1a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe790075de0 tx=0x7fe77c01a040 comp rx=0 tx=0).stop 2024-04-23T02:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.082+0000 7fe796b55700 1 -- 172.21.15.71:0/3809291052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7900ffc20 msgr2=0x7fe79019c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.082+0000 7fe796b55700 1 --2- 172.21.15.71:0/3809291052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7900ffc20 0x7fe79019c580 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe78400c930 tx=0x7fe78400ccf0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.082+0000 7fe796b55700 1 -- 172.21.15.71:0/3809291052 shutdown_connections 2024-04-23T02:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.082+0000 7fe796b55700 1 --2- 172.21.15.71:0/3809291052 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe78006cce0 0x7fe78006f1a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.082+0000 7fe796b55700 1 --2- 172.21.15.71:0/3809291052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7900ffc20 0x7fe79019c580 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.082+0000 7fe796b55700 1 --2- 172.21.15.71:0/3809291052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe790105bd0 0x7fe79019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.082+0000 7fe796b55700 1 -- 172.21.15.71:0/3809291052 >> 172.21.15.71:0/3809291052 conn(0x7fe790075670 msgr2=0x7fe790068720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.082+0000 7fe796b55700 1 -- 172.21.15.71:0/3809291052 shutdown_connections 2024-04-23T02:41:05.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.082+0000 7fe796b55700 1 -- 172.21.15.71:0/3809291052 wait complete. 2024-04-23T02:41:05.095 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:41:05.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.185+0000 7f479adae700 1 -- 172.21.15.71:0/939150465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4794101850 msgr2=0x7f47941058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:05.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.185+0000 7f479adae700 1 --2- 172.21.15.71:0/939150465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4794101850 0x7f47941058a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4780009910 tx=0x7f4780009c20 comp rx=0 tx=0).stop 2024-04-23T02:41:05.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.185+0000 7f479adae700 1 -- 172.21.15.71:0/939150465 shutdown_connections 2024-04-23T02:41:05.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.185+0000 7f479adae700 1 --2- 172.21.15.71:0/939150465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4794101850 0x7f47941058a0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.185+0000 7f479adae700 1 --2- 172.21.15.71:0/939150465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4794100ea0 0x7f4794101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.185+0000 7f479adae700 1 -- 172.21.15.71:0/939150465 >> 172.21.15.71:0/939150465 conn(0x7f47940fc750 msgr2=0x7f47940feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:05.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.185+0000 7f479adae700 1 -- 172.21.15.71:0/939150465 shutdown_connections 2024-04-23T02:41:05.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.185+0000 7f479adae700 1 -- 172.21.15.71:0/939150465 wait complete. 2024-04-23T02:41:05.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.186+0000 7f479adae700 1 Processor -- start 2024-04-23T02:41:05.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.186+0000 7f479adae700 1 -- start start 2024-04-23T02:41:05.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.186+0000 7f479adae700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4794100ea0 0x7f479419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:05.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.186+0000 7f479adae700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4794101850 0x7f479419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:05.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.186+0000 7f479adae700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479419d1f0 con 0x7f4794100ea0 2024-04-23T02:41:05.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.186+0000 7f479adae700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47941966a0 con 0x7f4794101850 2024-04-23T02:41:05.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.186+0000 7f4798b4a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4794100ea0 0x7f479419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:05.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.186+0000 7f4798b4a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4794100ea0 0x7f479419c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54210/0 (socket says 172.21.15.71:54210) 2024-04-23T02:41:05.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.186+0000 7f4798b4a700 1 -- 172.21.15.71:0/2924573045 learned_addr learned my addr 172.21.15.71:0/2924573045 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:41:05.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.186+0000 7f4793fff700 1 --2- 172.21.15.71:0/2924573045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4794101850 0x7f479419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:05.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.186+0000 7f4798b4a700 1 -- 172.21.15.71:0/2924573045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4794101850 msgr2=0x7f479419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:05.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.186+0000 7f4798b4a700 1 --2- 172.21.15.71:0/2924573045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4794101850 0x7f479419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.186+0000 7f4798b4a700 1 -- 172.21.15.71:0/2924573045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47800095f0 con 0x7f4794100ea0 2024-04-23T02:41:05.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.187+0000 7f4798b4a700 1 --2- 172.21.15.71:0/2924573045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4794100ea0 0x7f479419c620 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f478800e920 tx=0x7f478800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:05.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.187+0000 7f4791ffb700 1 -- 172.21.15.71:0/2924573045 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f478800c9d0 con 0x7f4794100ea0 2024-04-23T02:41:05.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.187+0000 7f4791ffb700 1 -- 172.21.15.71:0/2924573045 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4788010430 con 0x7f4794100ea0 2024-04-23T02:41:05.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.187+0000 7f479adae700 1 -- 172.21.15.71:0/2924573045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4794196980 con 0x7f4794100ea0 2024-04-23T02:41:05.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.187+0000 7f4791ffb700 1 -- 172.21.15.71:0/2924573045 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4788018650 con 0x7f4794100ea0 2024-04-23T02:41:05.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.187+0000 7f479adae700 1 -- 172.21.15.71:0/2924573045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4794196ea0 con 0x7f4794100ea0 2024-04-23T02:41:05.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.188+0000 7f479adae700 1 -- 172.21.15.71:0/2924573045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47941092a0 con 0x7f4794100ea0 2024-04-23T02:41:05.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.188+0000 7f4791ffb700 1 -- 172.21.15.71:0/2924573045 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f47880105a0 con 0x7f4794100ea0 2024-04-23T02:41:05.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.189+0000 7f4791ffb700 1 --2- 172.21.15.71:0/2924573045 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f478406cce0 0x7f478406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:05.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.189+0000 7f4791ffb700 1 -- 172.21.15.71:0/2924573045 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4788014070 con 0x7f4794100ea0 2024-04-23T02:41:05.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.189+0000 7f4793fff700 1 --2- 172.21.15.71:0/2924573045 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f478406cce0 0x7f478406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:05.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.189+0000 7f4793fff700 1 --2- 172.21.15.71:0/2924573045 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f478406cce0 0x7f478406f1a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4780009fd0 tx=0x7f4780005a90 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:05.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.193+0000 7f4791ffb700 1 -- 172.21.15.71:0/2924573045 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f4788052c50 con 0x7f4794100ea0 2024-04-23T02:41:05.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.402+0000 7f479adae700 1 -- 172.21.15.71:0/2924573045 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47941970c0 con 0x7f478406cce0 2024-04-23T02:41:05.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.403+0000 7f4791ffb700 1 -- 172.21.15.71:0/2924573045 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f47941970c0 con 0x7f478406cce0 2024-04-23T02:41:05.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.406+0000 7f479adae700 1 -- 172.21.15.71:0/2924573045 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f478406cce0 msgr2=0x7f478406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:05.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.406+0000 7f479adae700 1 --2- 172.21.15.71:0/2924573045 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f478406cce0 0x7f478406f1a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4780009fd0 tx=0x7f4780005a90 comp rx=0 tx=0).stop 2024-04-23T02:41:05.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.406+0000 7f479adae700 1 -- 172.21.15.71:0/2924573045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4794100ea0 msgr2=0x7f479419c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:05.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.406+0000 7f479adae700 1 --2- 172.21.15.71:0/2924573045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4794100ea0 0x7f479419c620 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f478800e920 tx=0x7f478800ec30 comp rx=0 tx=0).stop 2024-04-23T02:41:05.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.406+0000 7f479adae700 1 -- 172.21.15.71:0/2924573045 shutdown_connections 2024-04-23T02:41:05.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.406+0000 7f479adae700 1 --2- 172.21.15.71:0/2924573045 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f478406cce0 0x7f478406f1a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.406+0000 7f479adae700 1 --2- 172.21.15.71:0/2924573045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4794100ea0 0x7f479419c620 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.406+0000 7f479adae700 1 --2- 172.21.15.71:0/2924573045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4794101850 0x7f479419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.406+0000 7f479adae700 1 -- 172.21.15.71:0/2924573045 >> 172.21.15.71:0/2924573045 conn(0x7f47940fc750 msgr2=0x7f47940fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:05.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.406+0000 7f479adae700 1 -- 172.21.15.71:0/2924573045 shutdown_connections 2024-04-23T02:41:05.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.406+0000 7f479adae700 1 -- 172.21.15.71:0/2924573045 wait complete. 2024-04-23T02:41:05.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.508+0000 7f7844697700 1 -- 172.21.15.71:0/2144889501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f783c100ea0 msgr2=0x7f783c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:05.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.508+0000 7f7844697700 1 --2- 172.21.15.71:0/2144889501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f783c100ea0 0x7f783c101280 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7830009910 tx=0x7f7830009c20 comp rx=0 tx=0).stop 2024-04-23T02:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.508+0000 7f7844697700 1 -- 172.21.15.71:0/2144889501 shutdown_connections 2024-04-23T02:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.508+0000 7f7844697700 1 --2- 172.21.15.71:0/2144889501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f783c101850 0x7f783c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.508+0000 7f7844697700 1 --2- 172.21.15.71:0/2144889501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f783c100ea0 0x7f783c101280 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.508+0000 7f7844697700 1 -- 172.21.15.71:0/2144889501 >> 172.21.15.71:0/2144889501 conn(0x7f783c0fc750 msgr2=0x7f783c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.508+0000 7f7844697700 1 -- 172.21.15.71:0/2144889501 shutdown_connections 2024-04-23T02:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.508+0000 7f7844697700 1 -- 172.21.15.71:0/2144889501 wait complete. 2024-04-23T02:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.509+0000 7f7844697700 1 Processor -- start 2024-04-23T02:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.509+0000 7f7844697700 1 -- start start 2024-04-23T02:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.509+0000 7f7844697700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f783c100ea0 0x7f783c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.509+0000 7f7844697700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f783c101850 0x7f783c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:05.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.509+0000 7f7844697700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c073b10 con 0x7f783c100ea0 2024-04-23T02:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.509+0000 7f7844697700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c073c50 con 0x7f783c101850 2024-04-23T02:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.509+0000 7f7842433700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f783c100ea0 0x7f783c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.509+0000 7f7842433700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f783c100ea0 0x7f783c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57122/0 (socket says 172.21.15.71:57122) 2024-04-23T02:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.509+0000 7f7842433700 1 -- 172.21.15.71:0/585759484 learned_addr learned my addr 172.21.15.71:0/585759484 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.509+0000 7f7841c32700 1 --2- 172.21.15.71:0/585759484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f783c101850 0x7f783c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.509+0000 7f7842433700 1 -- 172.21.15.71:0/585759484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f783c101850 msgr2=0x7f783c0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.509+0000 7f7842433700 1 --2- 172.21.15.71:0/585759484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f783c101850 0x7f783c0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.509+0000 7f7842433700 1 -- 172.21.15.71:0/585759484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78300095f0 con 0x7f783c100ea0 2024-04-23T02:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.509+0000 7f7842433700 1 --2- 172.21.15.71:0/585759484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f783c100ea0 0x7f783c073090 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7830006010 tx=0x7f783000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.509+0000 7f782f7fe700 1 -- 172.21.15.71:0/585759484 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f783001d070 con 0x7f783c100ea0 2024-04-23T02:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.509+0000 7f782f7fe700 1 -- 172.21.15.71:0/585759484 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f783000f460 con 0x7f783c100ea0 2024-04-23T02:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.509+0000 7f7844697700 1 -- 172.21.15.71:0/585759484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783c073ed0 con 0x7f783c100ea0 2024-04-23T02:41:05.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.509+0000 7f782f7fe700 1 -- 172.21.15.71:0/585759484 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7830021620 con 0x7f783c100ea0 2024-04-23T02:41:05.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.510+0000 7f7844697700 1 -- 172.21.15.71:0/585759484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783c1a2c90 con 0x7f783c100ea0 2024-04-23T02:41:05.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.511+0000 7f7844697700 1 -- 172.21.15.71:0/585759484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f783c109240 con 0x7f783c100ea0 2024-04-23T02:41:05.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.511+0000 7f782f7fe700 1 -- 172.21.15.71:0/585759484 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f783000fa40 con 0x7f783c100ea0 2024-04-23T02:41:05.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.511+0000 7f782f7fe700 1 --2- 172.21.15.71:0/585759484 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f782806cce0 0x7f782806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:05.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.512+0000 7f782f7fe700 1 -- 172.21.15.71:0/585759484 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f783008d6c0 con 0x7f783c100ea0 2024-04-23T02:41:05.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.512+0000 7f7841c32700 1 --2- 172.21.15.71:0/585759484 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f782806cce0 0x7f782806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:05.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.512+0000 7f7841c32700 1 --2- 172.21.15.71:0/585759484 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f782806cce0 0x7f782806f1a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7838005950 tx=0x7f78380058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:05.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.520+0000 7f782f7fe700 1 -- 172.21.15.71:0/585759484 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f78300577b0 con 0x7f783c100ea0 2024-04-23T02:41:05.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.725+0000 7f7844697700 1 -- 172.21.15.71:0/585759484 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f783c0611d0 con 0x7f782806cce0 2024-04-23T02:41:05.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.736+0000 7f782f7fe700 1 -- 172.21.15.71:0/585759484 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f783c0611d0 con 0x7f782806cce0 2024-04-23T02:41:05.739 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:41:05.739 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (24m) 3m ago 36m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:41:05.739 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (36m) 3m ago 36m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:41:05.739 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (35m) 3m ago 35m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:41:05.739 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (22m) 3m ago 36m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:41:05.739 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (22m) 3m ago 35m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:41:05.739 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (24m) 3m ago 35m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:41:05.739 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (18m) 3m ago 30m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:41:05.739 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (18m) 3m ago 30m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:41:05.739 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (3m) 3m ago 30m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:41:05.739 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (18m) 3m ago 30m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:41:05.739 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (26m) 3m ago 37m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:41:05.740 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (25m) 3m ago 35m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:41:05.740 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (23m) 3m ago 37m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:41:05.740 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (22m) 3m ago 34m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:41:05.740 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (25m) 3m ago 36m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:41:05.740 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (25m) 3m ago 35m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:41:05.740 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (22m) 3m ago 34m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:41:05.740 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (21m) 3m ago 33m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:41:05.740 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (21m) 3m ago 33m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:41:05.740 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (20m) 3m ago 32m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:41:05.740 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (19m) 3m ago 32m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:41:05.740 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (19m) 3m ago 31m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:41:05.740 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (25m) 3m ago 35m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:41:05.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.738+0000 7f7844697700 1 -- 172.21.15.71:0/585759484 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f782806cce0 msgr2=0x7f782806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:05.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.738+0000 7f7844697700 1 --2- 172.21.15.71:0/585759484 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f782806cce0 0x7f782806f1a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7838005950 tx=0x7f78380058e0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.738+0000 7f7844697700 1 -- 172.21.15.71:0/585759484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f783c100ea0 msgr2=0x7f783c073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:05.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.738+0000 7f7844697700 1 --2- 172.21.15.71:0/585759484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f783c100ea0 0x7f783c073090 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7830006010 tx=0x7f783000be20 comp rx=0 tx=0).stop 2024-04-23T02:41:05.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.738+0000 7f7844697700 1 -- 172.21.15.71:0/585759484 shutdown_connections 2024-04-23T02:41:05.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.738+0000 7f7844697700 1 --2- 172.21.15.71:0/585759484 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f782806cce0 0x7f782806f1a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.738+0000 7f7844697700 1 --2- 172.21.15.71:0/585759484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f783c100ea0 0x7f783c073090 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.738+0000 7f7844697700 1 --2- 172.21.15.71:0/585759484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f783c101850 0x7f783c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.738+0000 7f7844697700 1 -- 172.21.15.71:0/585759484 >> 172.21.15.71:0/585759484 conn(0x7f783c0fc750 msgr2=0x7f783c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:05.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.738+0000 7f7844697700 1 -- 172.21.15.71:0/585759484 shutdown_connections 2024-04-23T02:41:05.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.738+0000 7f7844697700 1 -- 172.21.15.71:0/585759484 wait complete. 2024-04-23T02:41:05.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.839+0000 7f576e7a2700 1 -- 172.21.15.71:0/827203424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5768100ea0 msgr2=0x7f5768101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:05.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.839+0000 7f576e7a2700 1 --2- 172.21.15.71:0/827203424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5768100ea0 0x7f5768101280 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5758009910 tx=0x7f5758009c20 comp rx=0 tx=0).stop 2024-04-23T02:41:05.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.839+0000 7f576e7a2700 1 -- 172.21.15.71:0/827203424 shutdown_connections 2024-04-23T02:41:05.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.839+0000 7f576e7a2700 1 --2- 172.21.15.71:0/827203424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5768101850 0x7f57681058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.839+0000 7f576e7a2700 1 --2- 172.21.15.71:0/827203424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5768100ea0 0x7f5768101280 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.839+0000 7f576e7a2700 1 -- 172.21.15.71:0/827203424 >> 172.21.15.71:0/827203424 conn(0x7f57680fc750 msgr2=0x7f57680feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:05.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.839+0000 7f576e7a2700 1 -- 172.21.15.71:0/827203424 shutdown_connections 2024-04-23T02:41:05.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.839+0000 7f576e7a2700 1 -- 172.21.15.71:0/827203424 wait complete. 2024-04-23T02:41:05.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.840+0000 7f576e7a2700 1 Processor -- start 2024-04-23T02:41:05.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.840+0000 7f576e7a2700 1 -- start start 2024-04-23T02:41:05.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.840+0000 7f576e7a2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5768100ea0 0x7f5768073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:05.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.840+0000 7f576e7a2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5768101850 0x7f57680735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:05.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.840+0000 7f576e7a2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5768073b10 con 0x7f5768101850 2024-04-23T02:41:05.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.840+0000 7f576e7a2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5768073c50 con 0x7f5768100ea0 2024-04-23T02:41:05.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.840+0000 7f57677fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5768101850 0x7f57680735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:05.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.840+0000 7f57677fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5768101850 0x7f57680735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57142/0 (socket says 172.21.15.71:57142) 2024-04-23T02:41:05.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.840+0000 7f57677fe700 1 -- 172.21.15.71:0/2268587111 learned_addr learned my addr 172.21.15.71:0/2268587111 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:41:05.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.840+0000 7f5767fff700 1 --2- 172.21.15.71:0/2268587111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5768100ea0 0x7f5768073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:05.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.841+0000 7f57677fe700 1 -- 172.21.15.71:0/2268587111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5768100ea0 msgr2=0x7f5768073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:05.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.841+0000 7f57677fe700 1 --2- 172.21.15.71:0/2268587111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5768100ea0 0x7f5768073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:05.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.841+0000 7f57677fe700 1 -- 172.21.15.71:0/2268587111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57580095f0 con 0x7f5768101850 2024-04-23T02:41:05.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.841+0000 7f57677fe700 1 --2- 172.21.15.71:0/2268587111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5768101850 0x7f57680735d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f575c00d8d0 tx=0x7f575c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:05.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.841+0000 7f57657fa700 1 -- 172.21.15.71:0/2268587111 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f575c0041f0 con 0x7f5768101850 2024-04-23T02:41:05.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.841+0000 7f576e7a2700 1 -- 172.21.15.71:0/2268587111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57681a2960 con 0x7f5768101850 2024-04-23T02:41:05.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.841+0000 7f57657fa700 1 -- 172.21.15.71:0/2268587111 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f575c007620 con 0x7f5768101850 2024-04-23T02:41:05.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.841+0000 7f57657fa700 1 -- 172.21.15.71:0/2268587111 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f575c005610 con 0x7f5768101850 2024-04-23T02:41:05.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.841+0000 7f576e7a2700 1 -- 172.21.15.71:0/2268587111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57681a2d70 con 0x7f5768101850 2024-04-23T02:41:05.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.842+0000 7f576e7a2700 1 -- 172.21.15.71:0/2268587111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5768109240 con 0x7f5768101850 2024-04-23T02:41:05.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.843+0000 7f57657fa700 1 -- 172.21.15.71:0/2268587111 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f575c004750 con 0x7f5768101850 2024-04-23T02:41:05.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.843+0000 7f57657fa700 1 --2- 172.21.15.71:0/2268587111 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f575006cce0 0x7f575006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:05.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.843+0000 7f57657fa700 1 -- 172.21.15.71:0/2268587111 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f575c08c8e0 con 0x7f5768101850 2024-04-23T02:41:05.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.843+0000 7f5767fff700 1 --2- 172.21.15.71:0/2268587111 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f575006cce0 0x7f575006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:05.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.844+0000 7f5767fff700 1 --2- 172.21.15.71:0/2268587111 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f575006cce0 0x7f575006f1a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5758006010 tx=0x7f575801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:05.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:05.847+0000 7f57657fa700 1 -- 172.21.15.71:0/2268587111 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f575c0569d0 con 0x7f5768101850 2024-04-23T02:41:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:05 smithi192 ceph-mon[108617]: pgmap v834: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:41:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:05 smithi071 ceph-mon[144544]: pgmap v834: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:41:06.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.122+0000 7f576e7a2700 1 -- 172.21.15.71:0/2268587111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f576804f2e0 con 0x7f5768101850 2024-04-23T02:41:06.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.123+0000 7f57657fa700 1 -- 172.21.15.71:0/2268587111 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f575c016290 con 0x7f5768101850 2024-04-23T02:41:06.125 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:41:06.125 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:41:06.125 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:41:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:41:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:41:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:41:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:41:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:41:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:41:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:41:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:41:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:41:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:41:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:41:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:41:06.126 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:41:06.126 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:41:06.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.125+0000 7f576e7a2700 1 -- 172.21.15.71:0/2268587111 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f575006cce0 msgr2=0x7f575006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:06.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.125+0000 7f576e7a2700 1 --2- 172.21.15.71:0/2268587111 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f575006cce0 0x7f575006f1a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5758006010 tx=0x7f575801a040 comp rx=0 tx=0).stop 2024-04-23T02:41:06.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.125+0000 7f576e7a2700 1 -- 172.21.15.71:0/2268587111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5768101850 msgr2=0x7f57680735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:06.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.125+0000 7f576e7a2700 1 --2- 172.21.15.71:0/2268587111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5768101850 0x7f57680735d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f575c00d8d0 tx=0x7f575c00dbe0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.125+0000 7f576e7a2700 1 -- 172.21.15.71:0/2268587111 shutdown_connections 2024-04-23T02:41:06.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.125+0000 7f576e7a2700 1 --2- 172.21.15.71:0/2268587111 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f575006cce0 0x7f575006f1a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.125+0000 7f576e7a2700 1 --2- 172.21.15.71:0/2268587111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5768100ea0 0x7f5768073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.125+0000 7f576e7a2700 1 --2- 172.21.15.71:0/2268587111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5768101850 0x7f57680735d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.125+0000 7f576e7a2700 1 -- 172.21.15.71:0/2268587111 >> 172.21.15.71:0/2268587111 conn(0x7f57680fc750 msgr2=0x7f57680fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:06.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.125+0000 7f576e7a2700 1 -- 172.21.15.71:0/2268587111 shutdown_connections 2024-04-23T02:41:06.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.125+0000 7f576e7a2700 1 -- 172.21.15.71:0/2268587111 wait complete. 2024-04-23T02:41:06.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.227+0000 7f0657216700 1 -- 172.21.15.71:0/1909426855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0650105bf0 msgr2=0x7f0650105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:06.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.227+0000 7f0657216700 1 --2- 172.21.15.71:0/1909426855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0650105bf0 0x7f0650105fd0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0644009910 tx=0x7f0644009c20 comp rx=0 tx=0).stop 2024-04-23T02:41:06.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.227+0000 7f0657216700 1 -- 172.21.15.71:0/1909426855 shutdown_connections 2024-04-23T02:41:06.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.227+0000 7f0657216700 1 --2- 172.21.15.71:0/1909426855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06500ffb40 0x7f06500fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.227+0000 7f0657216700 1 --2- 172.21.15.71:0/1909426855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0650105bf0 0x7f0650105fd0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.227+0000 7f0657216700 1 -- 172.21.15.71:0/1909426855 >> 172.21.15.71:0/1909426855 conn(0x7f06500fb610 msgr2=0x7f06500fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.227+0000 7f0657216700 1 -- 172.21.15.71:0/1909426855 shutdown_connections 2024-04-23T02:41:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.227+0000 7f0657216700 1 -- 172.21.15.71:0/1909426855 wait complete. 2024-04-23T02:41:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.228+0000 7f0657216700 1 Processor -- start 2024-04-23T02:41:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.228+0000 7f0657216700 1 -- start start 2024-04-23T02:41:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.228+0000 7f0657216700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06500ffb40 0x7f065019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.228+0000 7f0657216700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0650105bf0 0x7f065019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.228+0000 7f0657216700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065019f2c0 con 0x7f06500ffb40 2024-04-23T02:41:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.228+0000 7f0657216700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0650198770 con 0x7f0650105bf0 2024-04-23T02:41:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.228+0000 7f0654fb2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06500ffb40 0x7f065019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.228+0000 7f0654fb2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06500ffb40 0x7f065019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57150/0 (socket says 172.21.15.71:57150) 2024-04-23T02:41:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.228+0000 7f0654fb2700 1 -- 172.21.15.71:0/3010019347 learned_addr learned my addr 172.21.15.71:0/3010019347 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:41:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.228+0000 7f064ffff700 1 --2- 172.21.15.71:0/3010019347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0650105bf0 0x7f065019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.228+0000 7f0654fb2700 1 -- 172.21.15.71:0/3010019347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0650105bf0 msgr2=0x7f065019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.228+0000 7f0654fb2700 1 --2- 172.21.15.71:0/3010019347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0650105bf0 0x7f065019ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.228+0000 7f0654fb2700 1 -- 172.21.15.71:0/3010019347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06440095f0 con 0x7f06500ffb40 2024-04-23T02:41:06.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.228+0000 7f0654fb2700 1 --2- 172.21.15.71:0/3010019347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06500ffb40 0x7f065019e6f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f06440092e0 tx=0x7f064400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:06.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.228+0000 7f064dffb700 1 -- 172.21.15.71:0/3010019347 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f064401d070 con 0x7f06500ffb40 2024-04-23T02:41:06.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.229+0000 7f0657216700 1 -- 172.21.15.71:0/3010019347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06501989f0 con 0x7f06500ffb40 2024-04-23T02:41:06.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.229+0000 7f064dffb700 1 -- 172.21.15.71:0/3010019347 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0644005e00 con 0x7f06500ffb40 2024-04-23T02:41:06.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.229+0000 7f064dffb700 1 -- 172.21.15.71:0/3010019347 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f064400fe60 con 0x7f06500ffb40 2024-04-23T02:41:06.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.229+0000 7f0657216700 1 -- 172.21.15.71:0/3010019347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0650198ee0 con 0x7f06500ffb40 2024-04-23T02:41:06.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.230+0000 7f0657216700 1 -- 172.21.15.71:0/3010019347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065004f2e0 con 0x7f06500ffb40 2024-04-23T02:41:06.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.230+0000 7f064dffb700 1 -- 172.21.15.71:0/3010019347 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f064400f6d0 con 0x7f06500ffb40 2024-04-23T02:41:06.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.231+0000 7f064dffb700 1 --2- 172.21.15.71:0/3010019347 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f064006cce0 0x7f064006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:06.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.231+0000 7f064dffb700 1 -- 172.21.15.71:0/3010019347 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f064408d8b0 con 0x7f06500ffb40 2024-04-23T02:41:06.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.231+0000 7f064ffff700 1 --2- 172.21.15.71:0/3010019347 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f064006cce0 0x7f064006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:06.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.231+0000 7f064ffff700 1 --2- 172.21.15.71:0/3010019347 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f064006cce0 0x7f064006f1a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f063c005950 tx=0x7f063c0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:06.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.235+0000 7f064dffb700 1 -- 172.21.15.71:0/3010019347 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f064405b3b0 con 0x7f06500ffb40 2024-04-23T02:41:06.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.469+0000 7f0657216700 1 -- 172.21.15.71:0/3010019347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0650199c80 con 0x7f06500ffb40 2024-04-23T02:41:06.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.470+0000 7f064dffb700 1 -- 172.21.15.71:0/3010019347 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f064405af40 con 0x7f06500ffb40 2024-04-23T02:41:06.473 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:41:06.473 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:41:06.473 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:41:06.473 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:41:06.473 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:41:06.473 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:41:06.473 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:41:06.473 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:41:06.473 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:41:06.473 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:41:06.473 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:41:06.473 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:41:06.473 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:41:06.474 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:41:06.474 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:41:06.474 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:41:06.474 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:41:06.474 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:41:06.474 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:41:06.474 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:41:06.474 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:41:06.474 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:41:06.474 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:41:06.474 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:41:06.474 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:41:06.474 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:41:06.475 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:41:06.475 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:41:06.475 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:41:06.475 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:41:06.475 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:41:06.475 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:41:06.475 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:41:06.475 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:41:06.475 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:41:06.475 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:41:06.475 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:41:06.475 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:41:06.475 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:41:06.475 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:41:06.476 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:41:06.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.472+0000 7f0657216700 1 -- 172.21.15.71:0/3010019347 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f064006cce0 msgr2=0x7f064006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:06.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.472+0000 7f0657216700 1 --2- 172.21.15.71:0/3010019347 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f064006cce0 0x7f064006f1a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f063c005950 tx=0x7f063c0058e0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.472+0000 7f0657216700 1 -- 172.21.15.71:0/3010019347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06500ffb40 msgr2=0x7f065019e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:06.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.472+0000 7f0657216700 1 --2- 172.21.15.71:0/3010019347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06500ffb40 0x7f065019e6f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f06440092e0 tx=0x7f064400be20 comp rx=0 tx=0).stop 2024-04-23T02:41:06.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.472+0000 7f0657216700 1 -- 172.21.15.71:0/3010019347 shutdown_connections 2024-04-23T02:41:06.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.472+0000 7f0657216700 1 --2- 172.21.15.71:0/3010019347 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f064006cce0 0x7f064006f1a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.472+0000 7f0657216700 1 --2- 172.21.15.71:0/3010019347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06500ffb40 0x7f065019e6f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.472+0000 7f0657216700 1 --2- 172.21.15.71:0/3010019347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0650105bf0 0x7f065019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.472+0000 7f0657216700 1 -- 172.21.15.71:0/3010019347 >> 172.21.15.71:0/3010019347 conn(0x7f06500fb610 msgr2=0x7f0650068f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:06.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.472+0000 7f0657216700 1 -- 172.21.15.71:0/3010019347 shutdown_connections 2024-04-23T02:41:06.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.472+0000 7f0657216700 1 -- 172.21.15.71:0/3010019347 wait complete. 2024-04-23T02:41:06.477 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:41:06.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.578+0000 7f84cd32e700 1 -- 172.21.15.71:0/351597708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c80ffb40 msgr2=0x7f84c80fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:06.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.578+0000 7f84cd32e700 1 --2- 172.21.15.71:0/351597708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c80ffb40 0x7f84c80fffc0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009910 tx=0x7f84bc009c20 comp rx=0 tx=0).stop 2024-04-23T02:41:06.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.578+0000 7f84cd32e700 1 -- 172.21.15.71:0/351597708 shutdown_connections 2024-04-23T02:41:06.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.578+0000 7f84cd32e700 1 --2- 172.21.15.71:0/351597708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c80ffb40 0x7f84c80fffc0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.578+0000 7f84cd32e700 1 --2- 172.21.15.71:0/351597708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84c8105bf0 0x7f84c8105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.578+0000 7f84cd32e700 1 -- 172.21.15.71:0/351597708 >> 172.21.15.71:0/351597708 conn(0x7f84c80fb650 msgr2=0x7f84c80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:06.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.578+0000 7f84cd32e700 1 -- 172.21.15.71:0/351597708 shutdown_connections 2024-04-23T02:41:06.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.578+0000 7f84cd32e700 1 -- 172.21.15.71:0/351597708 wait complete. 2024-04-23T02:41:06.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.579+0000 7f84cd32e700 1 Processor -- start 2024-04-23T02:41:06.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.579+0000 7f84cd32e700 1 -- start start 2024-04-23T02:41:06.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.579+0000 7f84cd32e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c80ffb40 0x7f84c8196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:06.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.579+0000 7f84cd32e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84c8105bf0 0x7f84c8196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:06.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.579+0000 7f84cd32e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c8197240 con 0x7f84c80ffb40 2024-04-23T02:41:06.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.579+0000 7f84cd32e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c81973b0 con 0x7f84c8105bf0 2024-04-23T02:41:06.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.579+0000 7f84c6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c80ffb40 0x7f84c8196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:06.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.579+0000 7f84c6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c80ffb40 0x7f84c8196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57176/0 (socket says 172.21.15.71:57176) 2024-04-23T02:41:06.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.579+0000 7f84c6ffd700 1 -- 172.21.15.71:0/4086110090 learned_addr learned my addr 172.21.15.71:0/4086110090 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:41:06.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.580+0000 7f84c67fc700 1 --2- 172.21.15.71:0/4086110090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84c8105bf0 0x7f84c8196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:06.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.580+0000 7f84c6ffd700 1 -- 172.21.15.71:0/4086110090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84c8105bf0 msgr2=0x7f84c8196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:06.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.580+0000 7f84c6ffd700 1 --2- 172.21.15.71:0/4086110090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84c8105bf0 0x7f84c8196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.580+0000 7f84c6ffd700 1 -- 172.21.15.71:0/4086110090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84bc0095f0 con 0x7f84c80ffb40 2024-04-23T02:41:06.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.580+0000 7f84c6ffd700 1 --2- 172.21.15.71:0/4086110090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c80ffb40 0x7f84c8196700 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f84b800c930 tx=0x7f84b800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:06.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.580+0000 7f84affff700 1 -- 172.21.15.71:0/4086110090 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84b8007ad0 con 0x7f84c80ffb40 2024-04-23T02:41:06.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.580+0000 7f84cd32e700 1 -- 172.21.15.71:0/4086110090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c819a150 con 0x7f84c80ffb40 2024-04-23T02:41:06.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.580+0000 7f84affff700 1 -- 172.21.15.71:0/4086110090 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f84b8007c30 con 0x7f84c80ffb40 2024-04-23T02:41:06.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.580+0000 7f84affff700 1 -- 172.21.15.71:0/4086110090 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84b8018630 con 0x7f84c80ffb40 2024-04-23T02:41:06.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.580+0000 7f84cd32e700 1 -- 172.21.15.71:0/4086110090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c819a6a0 con 0x7f84c80ffb40 2024-04-23T02:41:06.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.581+0000 7f84cd32e700 1 -- 172.21.15.71:0/4086110090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c804f2e0 con 0x7f84c80ffb40 2024-04-23T02:41:06.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.581+0000 7f84affff700 1 -- 172.21.15.71:0/4086110090 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f84b800f450 con 0x7f84c80ffb40 2024-04-23T02:41:06.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.582+0000 7f84affff700 1 --2- 172.21.15.71:0/4086110090 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f84b006cce0 0x7f84b006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:06.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.582+0000 7f84affff700 1 -- 172.21.15.71:0/4086110090 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f84b808c7d0 con 0x7f84c80ffb40 2024-04-23T02:41:06.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.582+0000 7f84c67fc700 1 --2- 172.21.15.71:0/4086110090 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f84b006cce0 0x7f84b006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:06.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.582+0000 7f84c67fc700 1 --2- 172.21.15.71:0/4086110090 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f84b006cce0 0x7f84b006f1a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f84c8197d20 tx=0x7f84bc01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:06.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.586+0000 7f84affff700 1 -- 172.21.15.71:0/4086110090 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f84b80568f0 con 0x7f84c80ffb40 2024-04-23T02:41:06.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.796+0000 7f84cd32e700 1 -- 172.21.15.71:0/4086110090 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f84c819ab80 con 0x7f84b006cce0 2024-04-23T02:41:06.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.797+0000 7f84affff700 1 -- 172.21.15.71:0/4086110090 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f84c819ab80 con 0x7f84b006cce0 2024-04-23T02:41:06.800 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:41:06.800 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:41:06.800 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:41:06.800 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:41:06.800 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:41:06.800 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:41:06.800 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:41:06.800 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:41:06.800 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:41:06.800 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:41:06.800 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:41:06.800 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:41:06.800 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:41:06.800 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:41:06.801 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:41:06.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.799+0000 7f84cd32e700 1 -- 172.21.15.71:0/4086110090 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f84b006cce0 msgr2=0x7f84b006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:06.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.799+0000 7f84cd32e700 1 --2- 172.21.15.71:0/4086110090 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f84b006cce0 0x7f84b006f1a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f84c8197d20 tx=0x7f84bc01a040 comp rx=0 tx=0).stop 2024-04-23T02:41:06.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.799+0000 7f84cd32e700 1 -- 172.21.15.71:0/4086110090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c80ffb40 msgr2=0x7f84c8196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:06.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.799+0000 7f84cd32e700 1 --2- 172.21.15.71:0/4086110090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c80ffb40 0x7f84c8196700 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f84b800c930 tx=0x7f84b800ccf0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.799+0000 7f84cd32e700 1 -- 172.21.15.71:0/4086110090 shutdown_connections 2024-04-23T02:41:06.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.799+0000 7f84cd32e700 1 --2- 172.21.15.71:0/4086110090 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f84b006cce0 0x7f84b006f1a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.799+0000 7f84cd32e700 1 --2- 172.21.15.71:0/4086110090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c80ffb40 0x7f84c8196700 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.799+0000 7f84cd32e700 1 --2- 172.21.15.71:0/4086110090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84c8105bf0 0x7f84c8196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.799+0000 7f84cd32e700 1 -- 172.21.15.71:0/4086110090 >> 172.21.15.71:0/4086110090 conn(0x7f84c80fb650 msgr2=0x7f84c8068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:06.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.799+0000 7f84cd32e700 1 -- 172.21.15.71:0/4086110090 shutdown_connections 2024-04-23T02:41:06.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.799+0000 7f84cd32e700 1 -- 172.21.15.71:0/4086110090 wait complete. 2024-04-23T02:41:06.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:06 smithi192 ceph-mon[108617]: from='client.35324 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:41:06.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:06 smithi192 ceph-mon[108617]: from='client.35328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:41:06.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:06 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2268587111' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:41:06.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:06 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3010019347' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:41:06.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.903+0000 7fd50bdba700 1 -- 172.21.15.71:0/3546933857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5040ffb40 msgr2=0x7fd5040fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:06.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.903+0000 7fd50bdba700 1 --2- 172.21.15.71:0/3546933857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5040ffb40 0x7fd5040fffc0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd500009910 tx=0x7fd500009c20 comp rx=0 tx=0).stop 2024-04-23T02:41:06.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.903+0000 7fd50bdba700 1 -- 172.21.15.71:0/3546933857 shutdown_connections 2024-04-23T02:41:06.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.903+0000 7fd50bdba700 1 --2- 172.21.15.71:0/3546933857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5040ffb40 0x7fd5040fffc0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.903+0000 7fd50bdba700 1 --2- 172.21.15.71:0/3546933857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd504105bf0 0x7fd504105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.903+0000 7fd50bdba700 1 -- 172.21.15.71:0/3546933857 >> 172.21.15.71:0/3546933857 conn(0x7fd5040fb610 msgr2=0x7fd5040fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:06.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.903+0000 7fd50bdba700 1 -- 172.21.15.71:0/3546933857 shutdown_connections 2024-04-23T02:41:06.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.903+0000 7fd50bdba700 1 -- 172.21.15.71:0/3546933857 wait complete. 2024-04-23T02:41:06.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.904+0000 7fd50bdba700 1 Processor -- start 2024-04-23T02:41:06.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.904+0000 7fd50bdba700 1 -- start start 2024-04-23T02:41:06.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.904+0000 7fd50bdba700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5040ffb40 0x7fd504197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:06.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.904+0000 7fd50bdba700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd504105bf0 0x7fd504197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:06.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.904+0000 7fd50bdba700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5041980d0 con 0x7fd5040ffb40 2024-04-23T02:41:06.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.904+0000 7fd50bdba700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd504198240 con 0x7fd504105bf0 2024-04-23T02:41:06.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.904+0000 7fd509b56700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5040ffb40 0x7fd504197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:06.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.904+0000 7fd509b56700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5040ffb40 0x7fd504197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57198/0 (socket says 172.21.15.71:57198) 2024-04-23T02:41:06.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.904+0000 7fd509b56700 1 -- 172.21.15.71:0/3722172767 learned_addr learned my addr 172.21.15.71:0/3722172767 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:41:06.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.905+0000 7fd509355700 1 --2- 172.21.15.71:0/3722172767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd504105bf0 0x7fd504197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:06.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.905+0000 7fd509b56700 1 -- 172.21.15.71:0/3722172767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd504105bf0 msgr2=0x7fd504197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:06.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.905+0000 7fd509b56700 1 --2- 172.21.15.71:0/3722172767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd504105bf0 0x7fd504197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:06.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.905+0000 7fd509b56700 1 -- 172.21.15.71:0/3722172767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5000095f0 con 0x7fd5040ffb40 2024-04-23T02:41:06.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.905+0000 7fd509b56700 1 --2- 172.21.15.71:0/3722172767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5040ffb40 0x7fd504197570 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400d900 tx=0x7fd4f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:06.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.905+0000 7fd4faffd700 1 -- 172.21.15.71:0/3722172767 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4f40041f0 con 0x7fd5040ffb40 2024-04-23T02:41:06.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.905+0000 7fd4faffd700 1 -- 172.21.15.71:0/3722172767 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd4f4004cd0 con 0x7fd5040ffb40 2024-04-23T02:41:06.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.905+0000 7fd50bdba700 1 -- 172.21.15.71:0/3722172767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5041a05e0 con 0x7fd5040ffb40 2024-04-23T02:41:06.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.905+0000 7fd4faffd700 1 -- 172.21.15.71:0/3722172767 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4f40056d0 con 0x7fd5040ffb40 2024-04-23T02:41:06.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.905+0000 7fd50bdba700 1 -- 172.21.15.71:0/3722172767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5041a0ad0 con 0x7fd5040ffb40 2024-04-23T02:41:06.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.907+0000 7fd50bdba700 1 -- 172.21.15.71:0/3722172767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5041080a0 con 0x7fd5040ffb40 2024-04-23T02:41:06.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.907+0000 7fd4faffd700 1 -- 172.21.15.71:0/3722172767 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd4f4004e40 con 0x7fd5040ffb40 2024-04-23T02:41:06.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.907+0000 7fd4faffd700 1 --2- 172.21.15.71:0/3722172767 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd4f006cce0 0x7fd4f006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:06.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.907+0000 7fd4faffd700 1 -- 172.21.15.71:0/3722172767 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd4f408c6b0 con 0x7fd5040ffb40 2024-04-23T02:41:06.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.907+0000 7fd509355700 1 --2- 172.21.15.71:0/3722172767 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd4f006cce0 0x7fd4f006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:06.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.908+0000 7fd509355700 1 --2- 172.21.15.71:0/3722172767 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd4f006cce0 0x7fd4f006f1a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd5040696e0 tx=0x7fd50001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:06.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:06.912+0000 7fd4faffd700 1 -- 172.21.15.71:0/3722172767 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd4f4056700 con 0x7fd5040ffb40 2024-04-23T02:41:06.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:06 smithi071 ceph-mon[144544]: from='client.35324 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:41:06.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:06 smithi071 ceph-mon[144544]: from='client.35328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:41:06.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:06 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2268587111' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:41:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:06 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3010019347' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:41:07.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:07.185+0000 7fd50bdba700 1 -- 172.21.15.71:0/3722172767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd50404f2e0 con 0x7fd5040ffb40 2024-04-23T02:41:07.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:07.185+0000 7fd4faffd700 1 -- 172.21.15.71:0/3722172767 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7fd4f4059d20 con 0x7fd5040ffb40 2024-04-23T02:41:07.188 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:41:07.188 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:41:07.188 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:41:07.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:41:07.188 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:41:07.188 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:41:07.188 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:41:07.188 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:41:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:41:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:41:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:41:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:41:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:41:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:41:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:41:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:41:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:41:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:41:07.189 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:41:07.189 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:41:07.189 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:41:07.189 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:41:07.190 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:41:07.190 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:41:07.190 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:41:07.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:07.187+0000 7fd50bdba700 1 -- 172.21.15.71:0/3722172767 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd4f006cce0 msgr2=0x7fd4f006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:07.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:07.187+0000 7fd50bdba700 1 --2- 172.21.15.71:0/3722172767 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd4f006cce0 0x7fd4f006f1a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd5040696e0 tx=0x7fd50001a040 comp rx=0 tx=0).stop 2024-04-23T02:41:07.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:07.187+0000 7fd50bdba700 1 -- 172.21.15.71:0/3722172767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5040ffb40 msgr2=0x7fd504197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:07.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:07.187+0000 7fd50bdba700 1 --2- 172.21.15.71:0/3722172767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5040ffb40 0x7fd504197570 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400d900 tx=0x7fd4f400dcc0 comp rx=0 tx=0).stop 2024-04-23T02:41:07.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:07.188+0000 7fd50bdba700 1 -- 172.21.15.71:0/3722172767 shutdown_connections 2024-04-23T02:41:07.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:07.188+0000 7fd50bdba700 1 --2- 172.21.15.71:0/3722172767 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd4f006cce0 0x7fd4f006f1a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:07.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:07.188+0000 7fd50bdba700 1 --2- 172.21.15.71:0/3722172767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5040ffb40 0x7fd504197570 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:07.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:07.188+0000 7fd50bdba700 1 --2- 172.21.15.71:0/3722172767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd504105bf0 0x7fd504197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:07.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:07.188+0000 7fd50bdba700 1 -- 172.21.15.71:0/3722172767 >> 172.21.15.71:0/3722172767 conn(0x7fd5040fb610 msgr2=0x7fd5040fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:07.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:07.188+0000 7fd50bdba700 1 -- 172.21.15.71:0/3722172767 shutdown_connections 2024-04-23T02:41:07.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:07.188+0000 7fd50bdba700 1 -- 172.21.15.71:0/3722172767 wait complete. 2024-04-23T02:41:07.795 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:07 smithi192 ceph-mon[108617]: from='client.35332 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:41:07.795 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:07 smithi192 ceph-mon[108617]: pgmap v835: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:07.795 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:07 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3722172767' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:41:07.842 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:07 smithi071 ceph-mon[144544]: from='client.35332 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:41:07.842 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:07 smithi071 ceph-mon[144544]: pgmap v835: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:07.843 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:07 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3722172767' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:41:08.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:08 smithi192 ceph-mon[108617]: from='client.35344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:41:08.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:08 smithi071 ceph-mon[144544]: from='client.35344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:41:09.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:09 smithi192 ceph-mon[108617]: pgmap v836: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:09.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:09 smithi071 ceph-mon[144544]: pgmap v836: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:11.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:11 smithi192 ceph-mon[108617]: pgmap v837: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:11.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:11 smithi071 ceph-mon[144544]: pgmap v837: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:13.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:13 smithi192 ceph-mon[108617]: pgmap v838: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:13.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:13 smithi071 ceph-mon[144544]: pgmap v838: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:15.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:15 smithi192 ceph-mon[108617]: pgmap v839: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:15.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:15 smithi071 ceph-mon[144544]: pgmap v839: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:17.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:17 smithi192 ceph-mon[108617]: pgmap v840: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:17.932 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:17 smithi071 ceph-mon[144544]: pgmap v840: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:19.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:19 smithi192 ceph-mon[108617]: pgmap v841: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:19.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:19 smithi071 ceph-mon[144544]: pgmap v841: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:20.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:41:20.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:41:21.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:21 smithi192 ceph-mon[108617]: pgmap v842: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:21.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:21 smithi071 ceph-mon[144544]: pgmap v842: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:23.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:23 smithi192 ceph-mon[108617]: pgmap v843: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:23.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:23 smithi071 ceph-mon[144544]: pgmap v843: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:25.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:25 smithi192 ceph-mon[108617]: pgmap v844: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:25.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:25 smithi071 ceph-mon[144544]: pgmap v844: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:27.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:27 smithi192 ceph-mon[108617]: pgmap v845: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:27.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:27 smithi071 ceph-mon[144544]: pgmap v845: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:29.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:29 smithi192 ceph-mon[108617]: pgmap v846: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:29.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:29 smithi071 ceph-mon[144544]: pgmap v846: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:31.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:31 smithi192 ceph-mon[108617]: pgmap v847: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:31.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:31 smithi071 ceph-mon[144544]: pgmap v847: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:33.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:33 smithi192 ceph-mon[108617]: pgmap v848: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:33.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:33 smithi071 ceph-mon[144544]: pgmap v848: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:34.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:34 smithi071 ceph-mon[144544]: pgmap v849: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:35.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:34 smithi192 ceph-mon[108617]: pgmap v849: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:35.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:41:36.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:41:36.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:36 smithi071 ceph-mon[144544]: pgmap v850: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:37.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:36 smithi192 ceph-mon[108617]: pgmap v850: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:37.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.303+0000 7ff9d05eb700 1 -- 172.21.15.71:0/1434986804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9c8105bf0 msgr2=0x7ff9c8105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:37.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.303+0000 7ff9d05eb700 1 --2- 172.21.15.71:0/1434986804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9c8105bf0 0x7ff9c8105fd0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009910 tx=0x7ff9bc009c20 comp rx=0 tx=0).stop 2024-04-23T02:41:37.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.303+0000 7ff9d05eb700 1 -- 172.21.15.71:0/1434986804 shutdown_connections 2024-04-23T02:41:37.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.303+0000 7ff9d05eb700 1 --2- 172.21.15.71:0/1434986804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9c80ffb40 0x7ff9c80fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:37.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.303+0000 7ff9d05eb700 1 --2- 172.21.15.71:0/1434986804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9c8105bf0 0x7ff9c8105fd0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:37.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.303+0000 7ff9d05eb700 1 -- 172.21.15.71:0/1434986804 >> 172.21.15.71:0/1434986804 conn(0x7ff9c80fb610 msgr2=0x7ff9c80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.303+0000 7ff9d05eb700 1 -- 172.21.15.71:0/1434986804 shutdown_connections 2024-04-23T02:41:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.303+0000 7ff9d05eb700 1 -- 172.21.15.71:0/1434986804 wait complete. 2024-04-23T02:41:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.304+0000 7ff9d05eb700 1 Processor -- start 2024-04-23T02:41:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.304+0000 7ff9d05eb700 1 -- start start 2024-04-23T02:41:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.304+0000 7ff9d05eb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9c80ffb40 0x7ff9c819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.304+0000 7ff9d05eb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9c8105bf0 0x7ff9c819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.304+0000 7ff9d05eb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c819d0c0 con 0x7ff9c80ffb40 2024-04-23T02:41:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.304+0000 7ff9d05eb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c81965c0 con 0x7ff9c8105bf0 2024-04-23T02:41:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.304+0000 7ff9ce387700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9c80ffb40 0x7ff9c819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.304+0000 7ff9ce387700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9c80ffb40 0x7ff9c819c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39146/0 (socket says 172.21.15.71:39146) 2024-04-23T02:41:37.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.304+0000 7ff9ce387700 1 -- 172.21.15.71:0/2346778176 learned_addr learned my addr 172.21.15.71:0/2346778176 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:41:37.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.304+0000 7ff9cdb86700 1 --2- 172.21.15.71:0/2346778176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9c8105bf0 0x7ff9c819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:37.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.305+0000 7ff9ce387700 1 -- 172.21.15.71:0/2346778176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9c8105bf0 msgr2=0x7ff9c819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:37.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.305+0000 7ff9ce387700 1 --2- 172.21.15.71:0/2346778176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9c8105bf0 0x7ff9c819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:37.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.305+0000 7ff9ce387700 1 -- 172.21.15.71:0/2346778176 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9bc0095f0 con 0x7ff9c80ffb40 2024-04-23T02:41:37.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.305+0000 7ff9ce387700 1 --2- 172.21.15.71:0/2346778176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9c80ffb40 0x7ff9c819c4f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc006010 tx=0x7ff9bc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:37.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.305+0000 7ff9bb7fe700 1 -- 172.21.15.71:0/2346778176 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9bc01d070 con 0x7ff9c80ffb40 2024-04-23T02:41:37.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.305+0000 7ff9bb7fe700 1 -- 172.21.15.71:0/2346778176 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff9bc00f460 con 0x7ff9c80ffb40 2024-04-23T02:41:37.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.305+0000 7ff9d05eb700 1 -- 172.21.15.71:0/2346778176 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c8196840 con 0x7ff9c80ffb40 2024-04-23T02:41:37.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.305+0000 7ff9bb7fe700 1 -- 172.21.15.71:0/2346778176 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9bc021640 con 0x7ff9c80ffb40 2024-04-23T02:41:37.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.305+0000 7ff9d05eb700 1 -- 172.21.15.71:0/2346778176 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c8196d30 con 0x7ff9c80ffb40 2024-04-23T02:41:37.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.306+0000 7ff9d05eb700 1 -- 172.21.15.71:0/2346778176 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c804f2e0 con 0x7ff9c80ffb40 2024-04-23T02:41:37.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.306+0000 7ff9bb7fe700 1 -- 172.21.15.71:0/2346778176 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff9bc00fa60 con 0x7ff9c80ffb40 2024-04-23T02:41:37.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.307+0000 7ff9bb7fe700 1 --2- 172.21.15.71:0/2346778176 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff9b406cce0 0x7ff9b406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:37.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.307+0000 7ff9bb7fe700 1 -- 172.21.15.71:0/2346778176 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff9bc08d7e0 con 0x7ff9c80ffb40 2024-04-23T02:41:37.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.307+0000 7ff9cdb86700 1 --2- 172.21.15.71:0/2346778176 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff9b406cce0 0x7ff9b406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:37.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.307+0000 7ff9cdb86700 1 --2- 172.21.15.71:0/2346778176 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff9b406cce0 0x7ff9b406f1a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4005fd0 tx=0x7ff9c400a560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:37.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.311+0000 7ff9bb7fe700 1 -- 172.21.15.71:0/2346778176 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff9bc05b360 con 0x7ff9c80ffb40 2024-04-23T02:41:37.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.522+0000 7ff9d05eb700 1 -- 172.21.15.71:0/2346778176 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9c8197180 con 0x7ff9b406cce0 2024-04-23T02:41:37.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.524+0000 7ff9bb7fe700 1 -- 172.21.15.71:0/2346778176 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff9c8197180 con 0x7ff9b406cce0 2024-04-23T02:41:37.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.526+0000 7ff9d05eb700 1 -- 172.21.15.71:0/2346778176 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff9b406cce0 msgr2=0x7ff9b406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:37.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.526+0000 7ff9d05eb700 1 --2- 172.21.15.71:0/2346778176 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff9b406cce0 0x7ff9b406f1a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4005fd0 tx=0x7ff9c400a560 comp rx=0 tx=0).stop 2024-04-23T02:41:37.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.526+0000 7ff9d05eb700 1 -- 172.21.15.71:0/2346778176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9c80ffb40 msgr2=0x7ff9c819c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:37.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.526+0000 7ff9d05eb700 1 --2- 172.21.15.71:0/2346778176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9c80ffb40 0x7ff9c819c4f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc006010 tx=0x7ff9bc00be20 comp rx=0 tx=0).stop 2024-04-23T02:41:37.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.526+0000 7ff9d05eb700 1 -- 172.21.15.71:0/2346778176 shutdown_connections 2024-04-23T02:41:37.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.526+0000 7ff9d05eb700 1 --2- 172.21.15.71:0/2346778176 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff9b406cce0 0x7ff9b406f1a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:37.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.526+0000 7ff9d05eb700 1 --2- 172.21.15.71:0/2346778176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9c80ffb40 0x7ff9c819c4f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:37.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.526+0000 7ff9d05eb700 1 --2- 172.21.15.71:0/2346778176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9c8105bf0 0x7ff9c819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:37.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.526+0000 7ff9d05eb700 1 -- 172.21.15.71:0/2346778176 >> 172.21.15.71:0/2346778176 conn(0x7ff9c80fb610 msgr2=0x7ff9c8068f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:37.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.526+0000 7ff9d05eb700 1 -- 172.21.15.71:0/2346778176 shutdown_connections 2024-04-23T02:41:37.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.526+0000 7ff9d05eb700 1 -- 172.21.15.71:0/2346778176 wait complete. 2024-04-23T02:41:37.538 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:41:37.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.629+0000 7f4cbd1d7700 1 -- 172.21.15.71:0/246334700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cb80ff5a0 msgr2=0x7f4cb80ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:37.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.629+0000 7f4cbd1d7700 1 --2- 172.21.15.71:0/246334700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cb80ff5a0 0x7f4cb80ff980 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4cac009910 tx=0x7f4cac009c20 comp rx=0 tx=0).stop 2024-04-23T02:41:37.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.629+0000 7f4cbd1d7700 1 -- 172.21.15.71:0/246334700 shutdown_connections 2024-04-23T02:41:37.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.629+0000 7f4cbd1d7700 1 --2- 172.21.15.71:0/246334700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cb80fff50 0x7f4cb8105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:37.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.629+0000 7f4cbd1d7700 1 --2- 172.21.15.71:0/246334700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cb80ff5a0 0x7f4cb80ff980 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:37.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.629+0000 7f4cbd1d7700 1 -- 172.21.15.71:0/246334700 >> 172.21.15.71:0/246334700 conn(0x7f4cb8074e30 msgr2=0x7f4cb8075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:37.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.629+0000 7f4cbd1d7700 1 -- 172.21.15.71:0/246334700 shutdown_connections 2024-04-23T02:41:37.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.629+0000 7f4cbd1d7700 1 -- 172.21.15.71:0/246334700 wait complete. 2024-04-23T02:41:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.630+0000 7f4cbd1d7700 1 Processor -- start 2024-04-23T02:41:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.630+0000 7f4cbd1d7700 1 -- start start 2024-04-23T02:41:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.630+0000 7f4cbd1d7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cb80ff5a0 0x7f4cb819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.630+0000 7f4cbd1d7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cb80fff50 0x7f4cb819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.630+0000 7f4cbd1d7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb819d170 con 0x7f4cb80fff50 2024-04-23T02:41:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.630+0000 7f4cbd1d7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb8196620 con 0x7f4cb80ff5a0 2024-04-23T02:41:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.630+0000 7f4cb659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cb80fff50 0x7f4cb819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.630+0000 7f4cb659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cb80fff50 0x7f4cb819cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39168/0 (socket says 172.21.15.71:39168) 2024-04-23T02:41:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.630+0000 7f4cb659c700 1 -- 172.21.15.71:0/3828624774 learned_addr learned my addr 172.21.15.71:0/3828624774 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:41:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.631+0000 7f4cb6d9d700 1 --2- 172.21.15.71:0/3828624774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cb80ff5a0 0x7f4cb819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.631+0000 7f4cb659c700 1 -- 172.21.15.71:0/3828624774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cb80ff5a0 msgr2=0x7f4cb819c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:37.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.631+0000 7f4cb659c700 1 --2- 172.21.15.71:0/3828624774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cb80ff5a0 0x7f4cb819c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:37.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.631+0000 7f4cb659c700 1 -- 172.21.15.71:0/3828624774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cac0095f0 con 0x7f4cb80fff50 2024-04-23T02:41:37.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.631+0000 7f4cb659c700 1 --2- 172.21.15.71:0/3828624774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cb80fff50 0x7f4cb819cae0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800d900 tx=0x7f4ca800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:37.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.631+0000 7f4c9ffff700 1 -- 172.21.15.71:0/3828624774 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ca8009900 con 0x7f4cb80fff50 2024-04-23T02:41:37.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.631+0000 7f4c9ffff700 1 -- 172.21.15.71:0/3828624774 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4ca8010460 con 0x7f4cb80fff50 2024-04-23T02:41:37.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.631+0000 7f4cbd1d7700 1 -- 172.21.15.71:0/3828624774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb8196900 con 0x7f4cb80fff50 2024-04-23T02:41:37.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.631+0000 7f4c9ffff700 1 -- 172.21.15.71:0/3828624774 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ca800f5f0 con 0x7f4cb80fff50 2024-04-23T02:41:37.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.631+0000 7f4cbd1d7700 1 -- 172.21.15.71:0/3828624774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb8196e50 con 0x7f4cb80fff50 2024-04-23T02:41:37.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.632+0000 7f4cbd1d7700 1 -- 172.21.15.71:0/3828624774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb8109270 con 0x7f4cb80fff50 2024-04-23T02:41:37.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.633+0000 7f4c9ffff700 1 -- 172.21.15.71:0/3828624774 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4ca80105d0 con 0x7f4cb80fff50 2024-04-23T02:41:37.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.633+0000 7f4c9ffff700 1 --2- 172.21.15.71:0/3828624774 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4ca006cce0 0x7f4ca006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:37.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.633+0000 7f4c9ffff700 1 -- 172.21.15.71:0/3828624774 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4ca808c0e0 con 0x7f4cb80fff50 2024-04-23T02:41:37.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.633+0000 7f4cb6d9d700 1 --2- 172.21.15.71:0/3828624774 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4ca006cce0 0x7f4ca006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:37.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.634+0000 7f4cb6d9d700 1 --2- 172.21.15.71:0/3828624774 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4ca006cce0 0x7f4ca006f1a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00b5c0 tx=0x7f4cac01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:37.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.638+0000 7f4c9ffff700 1 -- 172.21.15.71:0/3828624774 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f4ca8059be0 con 0x7f4cb80fff50 2024-04-23T02:41:37.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.848+0000 7f4cbd1d7700 1 -- 172.21.15.71:0/3828624774 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cb8197070 con 0x7f4ca006cce0 2024-04-23T02:41:37.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.849+0000 7f4c9ffff700 1 -- 172.21.15.71:0/3828624774 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4cb8197070 con 0x7f4ca006cce0 2024-04-23T02:41:37.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.851+0000 7f4cbd1d7700 1 -- 172.21.15.71:0/3828624774 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4ca006cce0 msgr2=0x7f4ca006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:37.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.851+0000 7f4cbd1d7700 1 --2- 172.21.15.71:0/3828624774 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4ca006cce0 0x7f4ca006f1a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00b5c0 tx=0x7f4cac01a040 comp rx=0 tx=0).stop 2024-04-23T02:41:37.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.851+0000 7f4cbd1d7700 1 -- 172.21.15.71:0/3828624774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cb80fff50 msgr2=0x7f4cb819cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:37.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.851+0000 7f4cbd1d7700 1 --2- 172.21.15.71:0/3828624774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cb80fff50 0x7f4cb819cae0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800d900 tx=0x7f4ca800dc10 comp rx=0 tx=0).stop 2024-04-23T02:41:37.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.851+0000 7f4cbd1d7700 1 -- 172.21.15.71:0/3828624774 shutdown_connections 2024-04-23T02:41:37.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.851+0000 7f4cbd1d7700 1 --2- 172.21.15.71:0/3828624774 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4ca006cce0 0x7f4ca006f1a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:37.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.851+0000 7f4cbd1d7700 1 --2- 172.21.15.71:0/3828624774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4cb80ff5a0 0x7f4cb819c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:37.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.851+0000 7f4cbd1d7700 1 --2- 172.21.15.71:0/3828624774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cb80fff50 0x7f4cb819cae0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:37.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.851+0000 7f4cbd1d7700 1 -- 172.21.15.71:0/3828624774 >> 172.21.15.71:0/3828624774 conn(0x7f4cb8074e30 msgr2=0x7f4cb80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:37.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.851+0000 7f4cbd1d7700 1 -- 172.21.15.71:0/3828624774 shutdown_connections 2024-04-23T02:41:37.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.851+0000 7f4cbd1d7700 1 -- 172.21.15.71:0/3828624774 wait complete. 2024-04-23T02:41:37.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.954+0000 7f2923e99700 1 -- 172.21.15.71:0/1565848361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291c101840 msgr2=0x7f291c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:37.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.954+0000 7f2923e99700 1 --2- 172.21.15.71:0/1565848361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291c101840 0x7f291c105890 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2918009910 tx=0x7f2918009c20 comp rx=0 tx=0).stop 2024-04-23T02:41:37.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.954+0000 7f2923e99700 1 -- 172.21.15.71:0/1565848361 shutdown_connections 2024-04-23T02:41:37.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.954+0000 7f2923e99700 1 --2- 172.21.15.71:0/1565848361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291c101840 0x7f291c105890 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:37.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.954+0000 7f2923e99700 1 --2- 172.21.15.71:0/1565848361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f291c100e90 0x7f291c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:37.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.954+0000 7f2923e99700 1 -- 172.21.15.71:0/1565848361 >> 172.21.15.71:0/1565848361 conn(0x7f291c0fc740 msgr2=0x7f291c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:37.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.954+0000 7f2923e99700 1 -- 172.21.15.71:0/1565848361 shutdown_connections 2024-04-23T02:41:37.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.954+0000 7f2923e99700 1 -- 172.21.15.71:0/1565848361 wait complete. 2024-04-23T02:41:37.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.955+0000 7f2923e99700 1 Processor -- start 2024-04-23T02:41:37.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.955+0000 7f2923e99700 1 -- start start 2024-04-23T02:41:37.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.955+0000 7f2923e99700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291c100e90 0x7f291c19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:37.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.955+0000 7f2923e99700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f291c101840 0x7f291c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:37.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.955+0000 7f2923e99700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c19d170 con 0x7f291c100e90 2024-04-23T02:41:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.955+0000 7f2923e99700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c1966b0 con 0x7f291c101840 2024-04-23T02:41:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.955+0000 7f2921c35700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291c100e90 0x7f291c19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.955+0000 7f2921c35700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291c100e90 0x7f291c19c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39194/0 (socket says 172.21.15.71:39194) 2024-04-23T02:41:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.955+0000 7f2921c35700 1 -- 172.21.15.71:0/1644793120 learned_addr learned my addr 172.21.15.71:0/1644793120 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:41:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.956+0000 7f2921434700 1 --2- 172.21.15.71:0/1644793120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f291c101840 0x7f291c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.956+0000 7f2921c35700 1 -- 172.21.15.71:0/1644793120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f291c101840 msgr2=0x7f291c19cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.956+0000 7f2921c35700 1 --2- 172.21.15.71:0/1644793120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f291c101840 0x7f291c19cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.956+0000 7f2921c35700 1 -- 172.21.15.71:0/1644793120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29180095f0 con 0x7f291c100e90 2024-04-23T02:41:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.956+0000 7f2921c35700 1 --2- 172.21.15.71:0/1644793120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291c100e90 0x7f291c19c630 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f290c00c930 tx=0x7f290c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.956+0000 7f2912ffd700 1 -- 172.21.15.71:0/1644793120 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f290c007ad0 con 0x7f291c100e90 2024-04-23T02:41:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.956+0000 7f2912ffd700 1 -- 172.21.15.71:0/1644793120 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f290c007c30 con 0x7f291c100e90 2024-04-23T02:41:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.956+0000 7f2923e99700 1 -- 172.21.15.71:0/1644793120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291c196990 con 0x7f291c100e90 2024-04-23T02:41:37.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.956+0000 7f2912ffd700 1 -- 172.21.15.71:0/1644793120 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f290c018630 con 0x7f291c100e90 2024-04-23T02:41:37.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.956+0000 7f2923e99700 1 -- 172.21.15.71:0/1644793120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291c196ee0 con 0x7f291c100e90 2024-04-23T02:41:37.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.957+0000 7f2923e99700 1 -- 172.21.15.71:0/1644793120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f291c109290 con 0x7f291c100e90 2024-04-23T02:41:37.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.957+0000 7f2912ffd700 1 -- 172.21.15.71:0/1644793120 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f290c00f450 con 0x7f291c100e90 2024-04-23T02:41:37.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.958+0000 7f2912ffd700 1 --2- 172.21.15.71:0/1644793120 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f290806cce0 0x7f290806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:37.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.958+0000 7f2912ffd700 1 -- 172.21.15.71:0/1644793120 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f290c08c780 con 0x7f291c100e90 2024-04-23T02:41:37.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.958+0000 7f2921434700 1 --2- 172.21.15.71:0/1644793120 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f290806cce0 0x7f290806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:37.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.958+0000 7f2921434700 1 --2- 172.21.15.71:0/1644793120 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f290806cce0 0x7f290806f1a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f291800b5c0 tx=0x7f2918005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:37.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:37.962+0000 7f2912ffd700 1 -- 172.21.15.71:0/1644793120 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f290c05a3b0 con 0x7f291c100e90 2024-04-23T02:41:37.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:37 smithi071 ceph-mon[144544]: from='client.35352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:41:38.074 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:37 smithi192 ceph-mon[108617]: from='client.35352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:41:38.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.189+0000 7f2923e99700 1 -- 172.21.15.71:0/1644793120 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f291c197c60 con 0x7f290806cce0 2024-04-23T02:41:38.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.200+0000 7f2912ffd700 1 -- 172.21.15.71:0/1644793120 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f291c197c60 con 0x7f290806cce0 2024-04-23T02:41:38.204 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:41:38.205 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (25m) 3m ago 36m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:41:38.205 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (36m) 3m ago 36m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:41:38.205 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (35m) 3m ago 35m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:41:38.205 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (23m) 3m ago 36m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:41:38.205 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (23m) 3m ago 35m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:41:38.205 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (25m) 3m ago 36m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:41:38.205 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (19m) 3m ago 30m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:41:38.205 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (19m) 3m ago 30m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:41:38.205 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (3m) 3m ago 30m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:41:38.205 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (18m) 3m ago 30m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:41:38.205 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (27m) 3m ago 38m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:41:38.205 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (26m) 3m ago 35m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:41:38.206 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (23m) 3m ago 38m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:41:38.206 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (23m) 3m ago 35m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:41:38.206 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (26m) 3m ago 36m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:41:38.206 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (26m) 3m ago 35m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:41:38.206 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (22m) 3m ago 34m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:41:38.206 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (22m) 3m ago 34m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:41:38.206 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (21m) 3m ago 33m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:41:38.206 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (20m) 3m ago 33m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:41:38.206 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (20m) 3m ago 32m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:41:38.206 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (19m) 3m ago 32m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:41:38.206 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (25m) 3m ago 36m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:41:38.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.202+0000 7f2923e99700 1 -- 172.21.15.71:0/1644793120 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f290806cce0 msgr2=0x7f290806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:38.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.202+0000 7f2923e99700 1 --2- 172.21.15.71:0/1644793120 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f290806cce0 0x7f290806f1a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f291800b5c0 tx=0x7f2918005fb0 comp rx=0 tx=0).stop 2024-04-23T02:41:38.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.202+0000 7f2923e99700 1 -- 172.21.15.71:0/1644793120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291c100e90 msgr2=0x7f291c19c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:38.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.202+0000 7f2923e99700 1 --2- 172.21.15.71:0/1644793120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291c100e90 0x7f291c19c630 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f290c00c930 tx=0x7f290c00ccf0 comp rx=0 tx=0).stop 2024-04-23T02:41:38.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.202+0000 7f2923e99700 1 -- 172.21.15.71:0/1644793120 shutdown_connections 2024-04-23T02:41:38.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.202+0000 7f2923e99700 1 --2- 172.21.15.71:0/1644793120 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f290806cce0 0x7f290806f1a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:38.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.202+0000 7f2923e99700 1 --2- 172.21.15.71:0/1644793120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291c100e90 0x7f291c19c630 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:38.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.202+0000 7f2923e99700 1 --2- 172.21.15.71:0/1644793120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f291c101840 0x7f291c19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:38.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.202+0000 7f2923e99700 1 -- 172.21.15.71:0/1644793120 >> 172.21.15.71:0/1644793120 conn(0x7f291c0fc740 msgr2=0x7f291c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:38.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.202+0000 7f2923e99700 1 -- 172.21.15.71:0/1644793120 shutdown_connections 2024-04-23T02:41:38.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.202+0000 7f2923e99700 1 -- 172.21.15.71:0/1644793120 wait complete. 2024-04-23T02:41:38.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.303+0000 7f1586ad8700 1 -- 172.21.15.71:0/2741509443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15800ff5a0 msgr2=0x7f15800ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:38.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.303+0000 7f1586ad8700 1 --2- 172.21.15.71:0/2741509443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15800ff5a0 0x7f15800ff980 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1574009910 tx=0x7f1574009c20 comp rx=0 tx=0).stop 2024-04-23T02:41:38.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.303+0000 7f1586ad8700 1 -- 172.21.15.71:0/2741509443 shutdown_connections 2024-04-23T02:41:38.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.303+0000 7f1586ad8700 1 --2- 172.21.15.71:0/2741509443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15800fff50 0x7f1580105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:38.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.303+0000 7f1586ad8700 1 --2- 172.21.15.71:0/2741509443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15800ff5a0 0x7f15800ff980 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:38.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.303+0000 7f1586ad8700 1 -- 172.21.15.71:0/2741509443 >> 172.21.15.71:0/2741509443 conn(0x7f1580074e30 msgr2=0x7f1580075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:38.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.303+0000 7f1586ad8700 1 -- 172.21.15.71:0/2741509443 shutdown_connections 2024-04-23T02:41:38.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.303+0000 7f1586ad8700 1 -- 172.21.15.71:0/2741509443 wait complete. 2024-04-23T02:41:38.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.304+0000 7f1586ad8700 1 Processor -- start 2024-04-23T02:41:38.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.304+0000 7f1586ad8700 1 -- start start 2024-04-23T02:41:38.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.304+0000 7f1586ad8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15800ff5a0 0x7f1580100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:38.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.304+0000 7f1586ad8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15800fff50 0x7f1580100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:38.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.304+0000 7f1586ad8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1580100fa0 con 0x7f15800ff5a0 2024-04-23T02:41:38.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.304+0000 7f1586ad8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15801010e0 con 0x7f15800fff50 2024-04-23T02:41:38.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.304+0000 7f1584874700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15800ff5a0 0x7f1580100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:38.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.304+0000 7f1584874700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15800ff5a0 0x7f1580100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39204/0 (socket says 172.21.15.71:39204) 2024-04-23T02:41:38.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.304+0000 7f1584874700 1 -- 172.21.15.71:0/3622553475 learned_addr learned my addr 172.21.15.71:0/3622553475 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:41:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.304+0000 7f157ffff700 1 --2- 172.21.15.71:0/3622553475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15800fff50 0x7f1580100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.304+0000 7f1584874700 1 -- 172.21.15.71:0/3622553475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15800fff50 msgr2=0x7f1580100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.304+0000 7f1584874700 1 --2- 172.21.15.71:0/3622553475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15800fff50 0x7f1580100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.304+0000 7f1584874700 1 -- 172.21.15.71:0/3622553475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15740095f0 con 0x7f15800ff5a0 2024-04-23T02:41:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.304+0000 7f1584874700 1 --2- 172.21.15.71:0/3622553475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15800ff5a0 0x7f1580100520 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1574006010 tx=0x7f157400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.305+0000 7f157dffb700 1 -- 172.21.15.71:0/3622553475 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f157401d070 con 0x7f15800ff5a0 2024-04-23T02:41:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.305+0000 7f157dffb700 1 -- 172.21.15.71:0/3622553475 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f157400f460 con 0x7f15800ff5a0 2024-04-23T02:41:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.305+0000 7f1586ad8700 1 -- 172.21.15.71:0/3622553475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1580101360 con 0x7f15800ff5a0 2024-04-23T02:41:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.305+0000 7f157dffb700 1 -- 172.21.15.71:0/3622553475 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1574021640 con 0x7f15800ff5a0 2024-04-23T02:41:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.305+0000 7f1586ad8700 1 -- 172.21.15.71:0/3622553475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15801a2c80 con 0x7f15800ff5a0 2024-04-23T02:41:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.306+0000 7f1586ad8700 1 -- 172.21.15.71:0/3622553475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1580109270 con 0x7f15800ff5a0 2024-04-23T02:41:38.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.306+0000 7f157dffb700 1 -- 172.21.15.71:0/3622553475 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f157402b440 con 0x7f15800ff5a0 2024-04-23T02:41:38.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.306+0000 7f157dffb700 1 --2- 172.21.15.71:0/3622553475 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f157006cce0 0x7f157006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:38.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.307+0000 7f157dffb700 1 -- 172.21.15.71:0/3622553475 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f157408d8c0 con 0x7f15800ff5a0 2024-04-23T02:41:38.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.307+0000 7f157ffff700 1 --2- 172.21.15.71:0/3622553475 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f157006cce0 0x7f157006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:38.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.307+0000 7f157ffff700 1 --2- 172.21.15.71:0/3622553475 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f157006cce0 0x7f157006f1a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f156c005ea0 tx=0x7f156c005e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:38.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.311+0000 7f157dffb700 1 -- 172.21.15.71:0/3622553475 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f15740579a0 con 0x7f15800ff5a0 2024-04-23T02:41:38.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.581+0000 7f1586ad8700 1 -- 172.21.15.71:0/3622553475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1580108c50 con 0x7f15800ff5a0 2024-04-23T02:41:38.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.582+0000 7f157dffb700 1 -- 172.21.15.71:0/3622553475 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f157405afc0 con 0x7f15800ff5a0 2024-04-23T02:41:38.584 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:41:38.584 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:41:38.585 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:41:38.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:41:38.585 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:41:38.585 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:41:38.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:41:38.585 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:41:38.585 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:41:38.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:41:38.585 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:41:38.585 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:41:38.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:41:38.585 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:41:38.585 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:41:38.585 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:41:38.585 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:41:38.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.584+0000 7f1586ad8700 1 -- 172.21.15.71:0/3622553475 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f157006cce0 msgr2=0x7f157006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:38.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.584+0000 7f1586ad8700 1 --2- 172.21.15.71:0/3622553475 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f157006cce0 0x7f157006f1a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f156c005ea0 tx=0x7f156c005e30 comp rx=0 tx=0).stop 2024-04-23T02:41:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.584+0000 7f1586ad8700 1 -- 172.21.15.71:0/3622553475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15800ff5a0 msgr2=0x7f1580100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.584+0000 7f1586ad8700 1 --2- 172.21.15.71:0/3622553475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15800ff5a0 0x7f1580100520 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1574006010 tx=0x7f157400be20 comp rx=0 tx=0).stop 2024-04-23T02:41:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.584+0000 7f1586ad8700 1 -- 172.21.15.71:0/3622553475 shutdown_connections 2024-04-23T02:41:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.584+0000 7f1586ad8700 1 --2- 172.21.15.71:0/3622553475 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f157006cce0 0x7f157006f1a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.584+0000 7f1586ad8700 1 --2- 172.21.15.71:0/3622553475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15800ff5a0 0x7f1580100520 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.584+0000 7f1586ad8700 1 --2- 172.21.15.71:0/3622553475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15800fff50 0x7f1580100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.584+0000 7f1586ad8700 1 -- 172.21.15.71:0/3622553475 >> 172.21.15.71:0/3622553475 conn(0x7f1580074e30 msgr2=0x7f15800fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.584+0000 7f1586ad8700 1 -- 172.21.15.71:0/3622553475 shutdown_connections 2024-04-23T02:41:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.584+0000 7f1586ad8700 1 -- 172.21.15.71:0/3622553475 wait complete. 2024-04-23T02:41:38.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.688+0000 7fdb60e5c700 1 -- 172.21.15.71:0/2093374556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb5c1027c0 msgr2=0x7fdb5c10acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:38.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.688+0000 7fdb60e5c700 1 --2- 172.21.15.71:0/2093374556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb5c1027c0 0x7fdb5c10acb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009910 tx=0x7fdb50009c20 comp rx=0 tx=0).stop 2024-04-23T02:41:38.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.688+0000 7fdb60e5c700 1 -- 172.21.15.71:0/2093374556 shutdown_connections 2024-04-23T02:41:38.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.688+0000 7fdb60e5c700 1 --2- 172.21.15.71:0/2093374556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb5c1027c0 0x7fdb5c10acb0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:38.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.688+0000 7fdb60e5c700 1 --2- 172.21.15.71:0/2093374556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb5c101ea0 0x7fdb5c102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:38.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.689+0000 7fdb60e5c700 1 -- 172.21.15.71:0/2093374556 >> 172.21.15.71:0/2093374556 conn(0x7fdb5c0fb650 msgr2=0x7fdb5c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:38.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.689+0000 7fdb60e5c700 1 -- 172.21.15.71:0/2093374556 shutdown_connections 2024-04-23T02:41:38.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.689+0000 7fdb60e5c700 1 -- 172.21.15.71:0/2093374556 wait complete. 2024-04-23T02:41:38.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.689+0000 7fdb60e5c700 1 Processor -- start 2024-04-23T02:41:38.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.689+0000 7fdb60e5c700 1 -- start start 2024-04-23T02:41:38.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.689+0000 7fdb60e5c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb5c101ea0 0x7fdb5c19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:38.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.689+0000 7fdb60e5c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb5c1027c0 0x7fdb5c19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:38.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.689+0000 7fdb60e5c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c19d190 con 0x7fdb5c101ea0 2024-04-23T02:41:38.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.690+0000 7fdb60e5c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c196640 con 0x7fdb5c1027c0 2024-04-23T02:41:38.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.690+0000 7fdb5a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb5c101ea0 0x7fdb5c19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:38.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.690+0000 7fdb5a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb5c101ea0 0x7fdb5c19c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39224/0 (socket says 172.21.15.71:39224) 2024-04-23T02:41:38.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.690+0000 7fdb5a59c700 1 -- 172.21.15.71:0/655436515 learned_addr learned my addr 172.21.15.71:0/655436515 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:41:38.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.690+0000 7fdb5a59c700 1 -- 172.21.15.71:0/655436515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb5c1027c0 msgr2=0x7fdb5c19cb00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-23T02:41:38.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.690+0000 7fdb5a59c700 1 --2- 172.21.15.71:0/655436515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb5c1027c0 0x7fdb5c19cb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:38.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.690+0000 7fdb5a59c700 1 -- 172.21.15.71:0/655436515 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb500095f0 con 0x7fdb5c101ea0 2024-04-23T02:41:38.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.690+0000 7fdb5a59c700 1 --2- 172.21.15.71:0/655436515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb5c101ea0 0x7fdb5c19c5c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00d8d0 tx=0x7fdb4c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:38.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.690+0000 7fdb4b7fe700 1 -- 172.21.15.71:0/655436515 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb4c009940 con 0x7fdb5c101ea0 2024-04-23T02:41:38.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.690+0000 7fdb60e5c700 1 -- 172.21.15.71:0/655436515 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5c196920 con 0x7fdb5c101ea0 2024-04-23T02:41:38.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.690+0000 7fdb4b7fe700 1 -- 172.21.15.71:0/655436515 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdb4c010460 con 0x7fdb5c101ea0 2024-04-23T02:41:38.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.690+0000 7fdb4b7fe700 1 -- 172.21.15.71:0/655436515 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb4c00f5f0 con 0x7fdb5c101ea0 2024-04-23T02:41:38.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.690+0000 7fdb60e5c700 1 -- 172.21.15.71:0/655436515 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5c196e70 con 0x7fdb5c101ea0 2024-04-23T02:41:38.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.691+0000 7fdb60e5c700 1 -- 172.21.15.71:0/655436515 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5c1083b0 con 0x7fdb5c101ea0 2024-04-23T02:41:38.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.692+0000 7fdb4b7fe700 1 -- 172.21.15.71:0/655436515 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fdb4c0105d0 con 0x7fdb5c101ea0 2024-04-23T02:41:38.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.692+0000 7fdb4b7fe700 1 --2- 172.21.15.71:0/655436515 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdb4406cce0 0x7fdb4406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:38.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.692+0000 7fdb4b7fe700 1 -- 172.21.15.71:0/655436515 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdb4c08c5e0 con 0x7fdb5c101ea0 2024-04-23T02:41:38.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.692+0000 7fdb59d9b700 1 --2- 172.21.15.71:0/655436515 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdb4406cce0 0x7fdb4406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:38.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.693+0000 7fdb59d9b700 1 --2- 172.21.15.71:0/655436515 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdb4406cce0 0x7fdb4406f1a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdb50000c00 tx=0x7fdb50005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:38.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.697+0000 7fdb4b7fe700 1 -- 172.21.15.71:0/655436515 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fdb4c05a0e0 con 0x7fdb5c101ea0 2024-04-23T02:41:38.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.931+0000 7fdb60e5c700 1 -- 172.21.15.71:0/655436515 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdb5c197090 con 0x7fdb5c101ea0 2024-04-23T02:41:38.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.932+0000 7fdb4b7fe700 1 -- 172.21.15.71:0/655436515 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7fdb4c010880 con 0x7fdb5c101ea0 2024-04-23T02:41:38.934 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:41:38.935 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:41:38.935 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:41:38.935 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:41:38.935 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:41:38.935 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:41:38.935 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:41:38.935 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:41:38.935 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:41:38.935 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:41:38.935 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:41:38.935 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:41:38.935 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:41:38.935 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:41:38.936 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:41:38.936 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:41:38.936 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:41:38.936 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:41:38.936 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:41:38.936 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:41:38.936 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:41:38.936 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:41:38.936 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:41:38.936 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:41:38.936 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:41:38.936 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:41:38.936 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:41:38.937 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:41:38.937 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:41:38.937 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:41:38.937 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:41:38.937 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:41:38.937 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:41:38.937 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:41:38.937 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:41:38.937 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:41:38.937 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:41:38.937 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:41:38.937 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:41:38.937 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:41:38.937 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:41:38.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.934+0000 7fdb60e5c700 1 -- 172.21.15.71:0/655436515 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdb4406cce0 msgr2=0x7fdb4406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:38.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.934+0000 7fdb60e5c700 1 --2- 172.21.15.71:0/655436515 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdb4406cce0 0x7fdb4406f1a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdb50000c00 tx=0x7fdb50005fb0 comp rx=0 tx=0).stop 2024-04-23T02:41:38.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.934+0000 7fdb60e5c700 1 -- 172.21.15.71:0/655436515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb5c101ea0 msgr2=0x7fdb5c19c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:38.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.934+0000 7fdb60e5c700 1 --2- 172.21.15.71:0/655436515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb5c101ea0 0x7fdb5c19c5c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00d8d0 tx=0x7fdb4c00dc90 comp rx=0 tx=0).stop 2024-04-23T02:41:38.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.934+0000 7fdb60e5c700 1 -- 172.21.15.71:0/655436515 shutdown_connections 2024-04-23T02:41:38.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.934+0000 7fdb60e5c700 1 --2- 172.21.15.71:0/655436515 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdb4406cce0 0x7fdb4406f1a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:38.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.934+0000 7fdb60e5c700 1 --2- 172.21.15.71:0/655436515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb5c101ea0 0x7fdb5c19c5c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:38.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.934+0000 7fdb60e5c700 1 --2- 172.21.15.71:0/655436515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb5c1027c0 0x7fdb5c19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:38.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.934+0000 7fdb60e5c700 1 -- 172.21.15.71:0/655436515 >> 172.21.15.71:0/655436515 conn(0x7fdb5c0fb650 msgr2=0x7fdb5c1054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:38.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.934+0000 7fdb60e5c700 1 -- 172.21.15.71:0/655436515 shutdown_connections 2024-04-23T02:41:38.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:38.934+0000 7fdb60e5c700 1 -- 172.21.15.71:0/655436515 wait complete. 2024-04-23T02:41:38.939 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:41:38.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:38 smithi071 ceph-mon[144544]: from='client.35356 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:41:38.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:38 smithi071 ceph-mon[144544]: from='client.35360 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:41:38.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:38 smithi071 ceph-mon[144544]: pgmap v851: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:38.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:38 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3622553475' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:41:39.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.037+0000 7f243d548700 1 -- 172.21.15.71:0/724648253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2438100750 msgr2=0x7f24381047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:39.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.037+0000 7f243d548700 1 --2- 172.21.15.71:0/724648253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2438100750 0x7f24381047a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f242c009910 tx=0x7f242c009c20 comp rx=0 tx=0).stop 2024-04-23T02:41:39.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.037+0000 7f243d548700 1 -- 172.21.15.71:0/724648253 shutdown_connections 2024-04-23T02:41:39.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.037+0000 7f243d548700 1 --2- 172.21.15.71:0/724648253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2438100750 0x7f24381047a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:39.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.037+0000 7f243d548700 1 --2- 172.21.15.71:0/724648253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24380ffda0 0x7f2438100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:39.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.037+0000 7f243d548700 1 -- 172.21.15.71:0/724648253 >> 172.21.15.71:0/724648253 conn(0x7f24380fb650 msgr2=0x7f24380fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:39.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.038+0000 7f243d548700 1 -- 172.21.15.71:0/724648253 shutdown_connections 2024-04-23T02:41:39.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.038+0000 7f243d548700 1 -- 172.21.15.71:0/724648253 wait complete. 2024-04-23T02:41:39.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.038+0000 7f243d548700 1 Processor -- start 2024-04-23T02:41:39.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.038+0000 7f243d548700 1 -- start start 2024-04-23T02:41:39.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.039+0000 7f243d548700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24380ffda0 0x7f243819c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:39.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.039+0000 7f243d548700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2438100750 0x7f243819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:39.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.039+0000 7f243d548700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243819d1a0 con 0x7f2438100750 2024-04-23T02:41:39.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.039+0000 7f243d548700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2438196650 con 0x7f24380ffda0 2024-04-23T02:41:39.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.039+0000 7f24367fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2438100750 0x7f243819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:39.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.039+0000 7f24367fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2438100750 0x7f243819cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39234/0 (socket says 172.21.15.71:39234) 2024-04-23T02:41:39.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.039+0000 7f24367fc700 1 -- 172.21.15.71:0/2523126455 learned_addr learned my addr 172.21.15.71:0/2523126455 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:41:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.039+0000 7f2436ffd700 1 --2- 172.21.15.71:0/2523126455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24380ffda0 0x7f243819c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.039+0000 7f24367fc700 1 -- 172.21.15.71:0/2523126455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24380ffda0 msgr2=0x7f243819c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.039+0000 7f24367fc700 1 --2- 172.21.15.71:0/2523126455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24380ffda0 0x7f243819c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.039+0000 7f24367fc700 1 -- 172.21.15.71:0/2523126455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c0095f0 con 0x7f2438100750 2024-04-23T02:41:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.039+0000 7f24367fc700 1 --2- 172.21.15.71:0/2523126455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2438100750 0x7f243819cb10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f242c000c00 tx=0x7f242c00ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.039+0000 7f241ffff700 1 -- 172.21.15.71:0/2523126455 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f242c01d070 con 0x7f2438100750 2024-04-23T02:41:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.039+0000 7f241ffff700 1 -- 172.21.15.71:0/2523126455 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f242c00bed0 con 0x7f2438100750 2024-04-23T02:41:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.039+0000 7f243d548700 1 -- 172.21.15.71:0/2523126455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24381968d0 con 0x7f2438100750 2024-04-23T02:41:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.039+0000 7f241ffff700 1 -- 172.21.15.71:0/2523126455 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f242c00fd90 con 0x7f2438100750 2024-04-23T02:41:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.039+0000 7f243d548700 1 -- 172.21.15.71:0/2523126455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2438196d60 con 0x7f2438100750 2024-04-23T02:41:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.040+0000 7f243d548700 1 -- 172.21.15.71:0/2523126455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f243804f2e0 con 0x7f2438100750 2024-04-23T02:41:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.041+0000 7f241ffff700 1 -- 172.21.15.71:0/2523126455 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f242c021410 con 0x7f2438100750 2024-04-23T02:41:39.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.041+0000 7f241ffff700 1 --2- 172.21.15.71:0/2523126455 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f242006cce0 0x7f242006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:39.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.041+0000 7f241ffff700 1 -- 172.21.15.71:0/2523126455 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f242c08d780 con 0x7f2438100750 2024-04-23T02:41:39.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.041+0000 7f2436ffd700 1 --2- 172.21.15.71:0/2523126455 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f242006cce0 0x7f242006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:39.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.042+0000 7f2436ffd700 1 --2- 172.21.15.71:0/2523126455 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f242006cce0 0x7f242006f1a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2428009c60 tx=0x7f2428009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:39.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.046+0000 7f241ffff700 1 -- 172.21.15.71:0/2523126455 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f242c05b280 con 0x7f2438100750 2024-04-23T02:41:39.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:38 smithi192 ceph-mon[108617]: from='client.35356 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:41:39.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:38 smithi192 ceph-mon[108617]: from='client.35360 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:41:39.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:38 smithi192 ceph-mon[108617]: pgmap v851: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:39.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:38 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3622553475' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:41:39.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.254+0000 7f243d548700 1 -- 172.21.15.71:0/2523126455 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f24381970c0 con 0x7f242006cce0 2024-04-23T02:41:39.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.255+0000 7f241ffff700 1 -- 172.21.15.71:0/2523126455 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f24381970c0 con 0x7f242006cce0 2024-04-23T02:41:39.258 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:41:39.258 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:41:39.258 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:41:39.258 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:41:39.258 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:41:39.258 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:41:39.258 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:41:39.258 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:41:39.258 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:41:39.259 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:41:39.259 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:41:39.259 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:41:39.259 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:41:39.259 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:41:39.259 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:41:39.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.257+0000 7f243d548700 1 -- 172.21.15.71:0/2523126455 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f242006cce0 msgr2=0x7f242006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:39.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.257+0000 7f243d548700 1 --2- 172.21.15.71:0/2523126455 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f242006cce0 0x7f242006f1a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2428009c60 tx=0x7f2428009260 comp rx=0 tx=0).stop 2024-04-23T02:41:39.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.257+0000 7f243d548700 1 -- 172.21.15.71:0/2523126455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2438100750 msgr2=0x7f243819cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:39.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.257+0000 7f243d548700 1 --2- 172.21.15.71:0/2523126455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2438100750 0x7f243819cb10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f242c000c00 tx=0x7f242c00ba70 comp rx=0 tx=0).stop 2024-04-23T02:41:39.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.257+0000 7f243d548700 1 -- 172.21.15.71:0/2523126455 shutdown_connections 2024-04-23T02:41:39.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.257+0000 7f243d548700 1 --2- 172.21.15.71:0/2523126455 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f242006cce0 0x7f242006f1a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:39.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.257+0000 7f243d548700 1 --2- 172.21.15.71:0/2523126455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24380ffda0 0x7f243819c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:39.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.257+0000 7f243d548700 1 --2- 172.21.15.71:0/2523126455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2438100750 0x7f243819cb10 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:39.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.257+0000 7f243d548700 1 -- 172.21.15.71:0/2523126455 >> 172.21.15.71:0/2523126455 conn(0x7f24380fb650 msgr2=0x7f2438068f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:39.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.258+0000 7f243d548700 1 -- 172.21.15.71:0/2523126455 shutdown_connections 2024-04-23T02:41:39.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.258+0000 7f243d548700 1 -- 172.21.15.71:0/2523126455 wait complete. 2024-04-23T02:41:39.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.359+0000 7f8517a0a700 1 -- 172.21.15.71:0/356978373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8510100750 msgr2=0x7f85101047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:39.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.359+0000 7f8517a0a700 1 --2- 172.21.15.71:0/356978373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8510100750 0x7f85101047a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8500009910 tx=0x7f8500009c20 comp rx=0 tx=0).stop 2024-04-23T02:41:39.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.359+0000 7f8517a0a700 1 -- 172.21.15.71:0/356978373 shutdown_connections 2024-04-23T02:41:39.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.359+0000 7f8517a0a700 1 --2- 172.21.15.71:0/356978373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8510100750 0x7f85101047a0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:39.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.359+0000 7f8517a0a700 1 --2- 172.21.15.71:0/356978373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85100ffda0 0x7f8510100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:39.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.359+0000 7f8517a0a700 1 -- 172.21.15.71:0/356978373 >> 172.21.15.71:0/356978373 conn(0x7f85100fb650 msgr2=0x7f85100fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:39.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.359+0000 7f8517a0a700 1 -- 172.21.15.71:0/356978373 shutdown_connections 2024-04-23T02:41:39.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.359+0000 7f8517a0a700 1 -- 172.21.15.71:0/356978373 wait complete. 2024-04-23T02:41:39.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.360+0000 7f8517a0a700 1 Processor -- start 2024-04-23T02:41:39.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.360+0000 7f8517a0a700 1 -- start start 2024-04-23T02:41:39.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.360+0000 7f8517a0a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85100ffda0 0x7f8510101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:39.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.360+0000 7f8517a0a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8510100750 0x7f8510101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:39.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.360+0000 7f8517a0a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85101022c0 con 0x7f85100ffda0 2024-04-23T02:41:39.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.360+0000 7f8517a0a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8510102400 con 0x7f8510100750 2024-04-23T02:41:39.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.360+0000 7f85157a6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85100ffda0 0x7f8510101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:39.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.360+0000 7f85157a6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85100ffda0 0x7f8510101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39254/0 (socket says 172.21.15.71:39254) 2024-04-23T02:41:39.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.360+0000 7f85157a6700 1 -- 172.21.15.71:0/77552306 learned_addr learned my addr 172.21.15.71:0/77552306 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:41:39.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.360+0000 7f8514fa5700 1 --2- 172.21.15.71:0/77552306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8510100750 0x7f8510101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:39.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.361+0000 7f85157a6700 1 -- 172.21.15.71:0/77552306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8510100750 msgr2=0x7f8510101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:39.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.361+0000 7f85157a6700 1 --2- 172.21.15.71:0/77552306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8510100750 0x7f8510101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:39.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.361+0000 7f85157a6700 1 -- 172.21.15.71:0/77552306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85000095f0 con 0x7f85100ffda0 2024-04-23T02:41:39.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.361+0000 7f8514fa5700 1 --2- 172.21.15.71:0/77552306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8510100750 0x7f8510101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:41:39.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.361+0000 7f85157a6700 1 --2- 172.21.15.71:0/77552306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85100ffda0 0x7f8510101840 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f850c00d8d0 tx=0x7f850c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:39.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.361+0000 7f85067fc700 1 -- 172.21.15.71:0/77552306 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f850c009940 con 0x7f85100ffda0 2024-04-23T02:41:39.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.361+0000 7f8517a0a700 1 -- 172.21.15.71:0/77552306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85101a29b0 con 0x7f85100ffda0 2024-04-23T02:41:39.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.361+0000 7f85067fc700 1 -- 172.21.15.71:0/77552306 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f850c010460 con 0x7f85100ffda0 2024-04-23T02:41:39.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.361+0000 7f85067fc700 1 -- 172.21.15.71:0/77552306 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f850c00f5f0 con 0x7f85100ffda0 2024-04-23T02:41:39.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.361+0000 7f8517a0a700 1 -- 172.21.15.71:0/77552306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85101a2dc0 con 0x7f85100ffda0 2024-04-23T02:41:39.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.362+0000 7f8517a0a700 1 -- 172.21.15.71:0/77552306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851004f2e0 con 0x7f85100ffda0 2024-04-23T02:41:39.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.362+0000 7f85067fc700 1 -- 172.21.15.71:0/77552306 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f850c0105d0 con 0x7f85100ffda0 2024-04-23T02:41:39.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.363+0000 7f85067fc700 1 --2- 172.21.15.71:0/77552306 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f84fc06cce0 0x7f84fc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:41:39.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.363+0000 7f85067fc700 1 -- 172.21.15.71:0/77552306 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f850c08c5e0 con 0x7f85100ffda0 2024-04-23T02:41:39.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.363+0000 7f8514fa5700 1 --2- 172.21.15.71:0/77552306 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f84fc06cce0 0x7f84fc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:41:39.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.363+0000 7f8514fa5700 1 --2- 172.21.15.71:0/77552306 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f84fc06cce0 0x7f84fc06f1a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8510103030 tx=0x7f850001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:41:39.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.367+0000 7f85067fc700 1 -- 172.21.15.71:0/77552306 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f850c056650 con 0x7f85100ffda0 2024-04-23T02:41:39.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.637+0000 7f8517a0a700 1 -- 172.21.15.71:0/77552306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f85101a3070 con 0x7f85100ffda0 2024-04-23T02:41:39.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.638+0000 7f85067fc700 1 -- 172.21.15.71:0/77552306 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7f850c010880 con 0x7f85100ffda0 2024-04-23T02:41:39.641 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:41:39.641 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:41:39.641 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:41:39.641 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:41:39.641 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:41:39.641 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:41:39.641 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:41:39.641 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:41:39.641 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:41:39.641 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:41:39.641 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:41:39.641 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:41:39.642 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:41:39.642 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:41:39.642 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:41:39.642 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:41:39.642 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:41:39.642 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:41:39.642 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:41:39.642 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:41:39.642 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:41:39.642 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:41:39.642 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:41:39.642 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:41:39.642 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:41:39.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.640+0000 7f8517a0a700 1 -- 172.21.15.71:0/77552306 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f84fc06cce0 msgr2=0x7f84fc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:39.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.640+0000 7f8517a0a700 1 --2- 172.21.15.71:0/77552306 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f84fc06cce0 0x7f84fc06f1a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8510103030 tx=0x7f850001a040 comp rx=0 tx=0).stop 2024-04-23T02:41:39.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.640+0000 7f8517a0a700 1 -- 172.21.15.71:0/77552306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85100ffda0 msgr2=0x7f8510101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:41:39.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.640+0000 7f8517a0a700 1 --2- 172.21.15.71:0/77552306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85100ffda0 0x7f8510101840 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f850c00d8d0 tx=0x7f850c00dc90 comp rx=0 tx=0).stop 2024-04-23T02:41:39.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.640+0000 7f8517a0a700 1 -- 172.21.15.71:0/77552306 shutdown_connections 2024-04-23T02:41:39.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.640+0000 7f8517a0a700 1 --2- 172.21.15.71:0/77552306 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f84fc06cce0 0x7f84fc06f1a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:39.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.640+0000 7f8517a0a700 1 --2- 172.21.15.71:0/77552306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85100ffda0 0x7f8510101840 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:39.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.640+0000 7f8517a0a700 1 --2- 172.21.15.71:0/77552306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8510100750 0x7f8510101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:41:39.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.640+0000 7f8517a0a700 1 -- 172.21.15.71:0/77552306 >> 172.21.15.71:0/77552306 conn(0x7f85100fb650 msgr2=0x7f8510068f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:41:39.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.640+0000 7f8517a0a700 1 -- 172.21.15.71:0/77552306 shutdown_connections 2024-04-23T02:41:39.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:41:39.640+0000 7f8517a0a700 1 -- 172.21.15.71:0/77552306 wait complete. 2024-04-23T02:41:39.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:39 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/655436515' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:41:39.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:39 smithi071 ceph-mon[144544]: from='client.35372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:41:40.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:39 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/655436515' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:41:40.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:39 smithi192 ceph-mon[108617]: from='client.35372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:41:40.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:40 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/77552306' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:41:40.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:40 smithi071 ceph-mon[144544]: pgmap v852: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:41.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:40 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/77552306' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:41:41.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:40 smithi192 ceph-mon[108617]: pgmap v852: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:43.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:43 smithi192 ceph-mon[108617]: pgmap v853: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:43.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:43 smithi071 ceph-mon[144544]: pgmap v853: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:45.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:45 smithi192 ceph-mon[108617]: pgmap v854: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:45.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:45 smithi071 ceph-mon[144544]: pgmap v854: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:47.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:47 smithi192 ceph-mon[108617]: pgmap v855: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:47.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:47 smithi071 ceph-mon[144544]: pgmap v855: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:49.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:49 smithi192 ceph-mon[108617]: pgmap v856: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:49.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:49 smithi071 ceph-mon[144544]: pgmap v856: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:41:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:41:51.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:51 smithi192 ceph-mon[108617]: pgmap v857: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:51.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:51 smithi071 ceph-mon[144544]: pgmap v857: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:53.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:53 smithi192 ceph-mon[108617]: pgmap v858: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:53.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:53 smithi071 ceph-mon[144544]: pgmap v858: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:55.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:55 smithi192 ceph-mon[108617]: pgmap v859: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:55.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:55 smithi071 ceph-mon[144544]: pgmap v859: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:57.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:57 smithi192 ceph-mon[108617]: pgmap v860: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:57.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:57 smithi071 ceph-mon[144544]: pgmap v860: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:59.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:41:59 smithi192 ceph-mon[108617]: pgmap v861: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:41:59.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:41:59 smithi071 ceph-mon[144544]: pgmap v861: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:01.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:01 smithi192 ceph-mon[108617]: pgmap v862: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:01.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:01 smithi071 ceph-mon[144544]: pgmap v862: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:03.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:03 smithi071 ceph-mon[144544]: pgmap v863: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:03.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:03 smithi192 ceph-mon[108617]: pgmap v863: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:05 smithi192 ceph-mon[108617]: pgmap v864: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:42:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:05 smithi071 ceph-mon[144544]: pgmap v864: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:42:07.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:07 smithi192 ceph-mon[108617]: pgmap v865: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:07.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:07 smithi071 ceph-mon[144544]: pgmap v865: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:09.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.757+0000 7fbe87606700 1 -- 172.21.15.71:0/3409117496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe800ffc20 msgr2=0x7fbe80100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:09.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.757+0000 7fbe87606700 1 --2- 172.21.15.71:0/3409117496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe800ffc20 0x7fbe80100080 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c009910 tx=0x7fbe7c009c20 comp rx=0 tx=0).stop 2024-04-23T02:42:09.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.757+0000 7fbe87606700 1 -- 172.21.15.71:0/3409117496 shutdown_connections 2024-04-23T02:42:09.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.757+0000 7fbe87606700 1 --2- 172.21.15.71:0/3409117496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe800ffc20 0x7fbe80100080 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:09.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.757+0000 7fbe87606700 1 --2- 172.21.15.71:0/3409117496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe80105bd0 0x7fbe80105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:09.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.757+0000 7fbe87606700 1 -- 172.21.15.71:0/3409117496 >> 172.21.15.71:0/3409117496 conn(0x7fbe80075670 msgr2=0x7fbe80075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:09.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.757+0000 7fbe87606700 1 -- 172.21.15.71:0/3409117496 shutdown_connections 2024-04-23T02:42:09.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.757+0000 7fbe87606700 1 -- 172.21.15.71:0/3409117496 wait complete. 2024-04-23T02:42:09.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.758+0000 7fbe87606700 1 Processor -- start 2024-04-23T02:42:09.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.758+0000 7fbe87606700 1 -- start start 2024-04-23T02:42:09.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.758+0000 7fbe87606700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe800ffc20 0x7fbe8019c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:09.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.758+0000 7fbe87606700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe80105bd0 0x7fbe8019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:09.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.758+0000 7fbe87606700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8019d1c0 con 0x7fbe800ffc20 2024-04-23T02:42:09.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.758+0000 7fbe87606700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe80196670 con 0x7fbe80105bd0 2024-04-23T02:42:09.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.758+0000 7fbe853a2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe800ffc20 0x7fbe8019c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:09.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.758+0000 7fbe853a2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe800ffc20 0x7fbe8019c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50912/0 (socket says 172.21.15.71:50912) 2024-04-23T02:42:09.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.758+0000 7fbe853a2700 1 -- 172.21.15.71:0/4173384527 learned_addr learned my addr 172.21.15.71:0/4173384527 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:42:09.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.758+0000 7fbe84ba1700 1 --2- 172.21.15.71:0/4173384527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe80105bd0 0x7fbe8019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:09.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.758+0000 7fbe853a2700 1 -- 172.21.15.71:0/4173384527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe80105bd0 msgr2=0x7fbe8019cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:09.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.758+0000 7fbe853a2700 1 --2- 172.21.15.71:0/4173384527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe80105bd0 0x7fbe8019cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:09.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.758+0000 7fbe853a2700 1 -- 172.21.15.71:0/4173384527 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7c0095f0 con 0x7fbe800ffc20 2024-04-23T02:42:09.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.758+0000 7fbe84ba1700 1 --2- 172.21.15.71:0/4173384527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe80105bd0 0x7fbe8019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:42:09.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.758+0000 7fbe853a2700 1 --2- 172.21.15.71:0/4173384527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe800ffc20 0x7fbe8019c5f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000d8d0 tx=0x7fbe7000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:09.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.759+0000 7fbe767fc700 1 -- 172.21.15.71:0/4173384527 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe70009940 con 0x7fbe800ffc20 2024-04-23T02:42:09.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.759+0000 7fbe767fc700 1 -- 172.21.15.71:0/4173384527 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe70010460 con 0x7fbe800ffc20 2024-04-23T02:42:09.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.759+0000 7fbe767fc700 1 -- 172.21.15.71:0/4173384527 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe7000f5f0 con 0x7fbe800ffc20 2024-04-23T02:42:09.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.759+0000 7fbe87606700 1 -- 172.21.15.71:0/4173384527 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe80196950 con 0x7fbe800ffc20 2024-04-23T02:42:09.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.759+0000 7fbe87606700 1 -- 172.21.15.71:0/4173384527 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe80196ea0 con 0x7fbe800ffc20 2024-04-23T02:42:09.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.760+0000 7fbe87606700 1 -- 172.21.15.71:0/4173384527 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8004f2e0 con 0x7fbe800ffc20 2024-04-23T02:42:09.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.760+0000 7fbe767fc700 1 -- 172.21.15.71:0/4173384527 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbe700105d0 con 0x7fbe800ffc20 2024-04-23T02:42:09.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.761+0000 7fbe767fc700 1 --2- 172.21.15.71:0/4173384527 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbe6c06cce0 0x7fbe6c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:09.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.761+0000 7fbe84ba1700 1 --2- 172.21.15.71:0/4173384527 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbe6c06cce0 0x7fbe6c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:09.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.761+0000 7fbe767fc700 1 -- 172.21.15.71:0/4173384527 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbe7008c5e0 con 0x7fbe800ffc20 2024-04-23T02:42:09.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.761+0000 7fbe84ba1700 1 --2- 172.21.15.71:0/4173384527 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbe6c06cce0 0x7fbe6c06f1a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00b5c0 tx=0x7fbe7c005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:09.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.765+0000 7fbe767fc700 1 -- 172.21.15.71:0/4173384527 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbe7005a0e0 con 0x7fbe800ffc20 2024-04-23T02:42:09.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:09 smithi192 ceph-mon[108617]: pgmap v866: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:09.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.979+0000 7fbe87606700 1 -- 172.21.15.71:0/4173384527 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe80197130 con 0x7fbe6c06cce0 2024-04-23T02:42:09.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.980+0000 7fbe767fc700 1 -- 172.21.15.71:0/4173384527 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbe80197130 con 0x7fbe6c06cce0 2024-04-23T02:42:09.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.982+0000 7fbe87606700 1 -- 172.21.15.71:0/4173384527 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbe6c06cce0 msgr2=0x7fbe6c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:09.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.982+0000 7fbe87606700 1 --2- 172.21.15.71:0/4173384527 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbe6c06cce0 0x7fbe6c06f1a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00b5c0 tx=0x7fbe7c005960 comp rx=0 tx=0).stop 2024-04-23T02:42:09.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.982+0000 7fbe87606700 1 -- 172.21.15.71:0/4173384527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe800ffc20 msgr2=0x7fbe8019c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:09.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.982+0000 7fbe87606700 1 --2- 172.21.15.71:0/4173384527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe800ffc20 0x7fbe8019c5f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000d8d0 tx=0x7fbe7000dc90 comp rx=0 tx=0).stop 2024-04-23T02:42:09.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.982+0000 7fbe87606700 1 -- 172.21.15.71:0/4173384527 shutdown_connections 2024-04-23T02:42:09.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.982+0000 7fbe87606700 1 --2- 172.21.15.71:0/4173384527 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbe6c06cce0 0x7fbe6c06f1a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:09.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.982+0000 7fbe87606700 1 --2- 172.21.15.71:0/4173384527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe800ffc20 0x7fbe8019c5f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:09.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.982+0000 7fbe87606700 1 --2- 172.21.15.71:0/4173384527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe80105bd0 0x7fbe8019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:09.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.982+0000 7fbe87606700 1 -- 172.21.15.71:0/4173384527 >> 172.21.15.71:0/4173384527 conn(0x7fbe80075670 msgr2=0x7fbe80103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:09.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.982+0000 7fbe87606700 1 -- 172.21.15.71:0/4173384527 shutdown_connections 2024-04-23T02:42:09.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:09.982+0000 7fbe87606700 1 -- 172.21.15.71:0/4173384527 wait complete. 2024-04-23T02:42:09.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:09 smithi071 ceph-mon[144544]: pgmap v866: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:09.995 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:42:10.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.085+0000 7fd58e440700 1 -- 172.21.15.71:0/3186333884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd588101850 msgr2=0x7fd5881058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:10.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.085+0000 7fd58e440700 1 --2- 172.21.15.71:0/3186333884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd588101850 0x7fd5881058a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009910 tx=0x7fd57c009c20 comp rx=0 tx=0).stop 2024-04-23T02:42:10.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.086+0000 7fd58e440700 1 -- 172.21.15.71:0/3186333884 shutdown_connections 2024-04-23T02:42:10.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.086+0000 7fd58e440700 1 --2- 172.21.15.71:0/3186333884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd588101850 0x7fd5881058a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:10.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.086+0000 7fd58e440700 1 --2- 172.21.15.71:0/3186333884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd588100ea0 0x7fd588101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:10.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.086+0000 7fd58e440700 1 -- 172.21.15.71:0/3186333884 >> 172.21.15.71:0/3186333884 conn(0x7fd5880fc750 msgr2=0x7fd5880feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:10.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.086+0000 7fd58e440700 1 -- 172.21.15.71:0/3186333884 shutdown_connections 2024-04-23T02:42:10.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.086+0000 7fd58e440700 1 -- 172.21.15.71:0/3186333884 wait complete. 2024-04-23T02:42:10.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.086+0000 7fd58e440700 1 Processor -- start 2024-04-23T02:42:10.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.086+0000 7fd58e440700 1 -- start start 2024-04-23T02:42:10.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.087+0000 7fd58e440700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd588100ea0 0x7fd588073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:10.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.087+0000 7fd58e440700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd588101850 0x7fd5880735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:10.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.087+0000 7fd58e440700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd588073b10 con 0x7fd588100ea0 2024-04-23T02:42:10.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.087+0000 7fd58e440700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd588073c50 con 0x7fd588101850 2024-04-23T02:42:10.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.087+0000 7fd587fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd588100ea0 0x7fd588073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:10.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.087+0000 7fd587fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd588100ea0 0x7fd588073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50936/0 (socket says 172.21.15.71:50936) 2024-04-23T02:42:10.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.087+0000 7fd587fff700 1 -- 172.21.15.71:0/3694527249 learned_addr learned my addr 172.21.15.71:0/3694527249 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:42:10.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.087+0000 7fd5877fe700 1 --2- 172.21.15.71:0/3694527249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd588101850 0x7fd5880735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:10.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.087+0000 7fd587fff700 1 -- 172.21.15.71:0/3694527249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd588101850 msgr2=0x7fd5880735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:10.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.087+0000 7fd587fff700 1 --2- 172.21.15.71:0/3694527249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd588101850 0x7fd5880735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:10.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.087+0000 7fd587fff700 1 -- 172.21.15.71:0/3694527249 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c0095f0 con 0x7fd588100ea0 2024-04-23T02:42:10.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.087+0000 7fd587fff700 1 --2- 172.21.15.71:0/3694527249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd588100ea0 0x7fd588073090 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd57800c8a0 tx=0x7fd57800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:10.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.087+0000 7fd5857fa700 1 -- 172.21.15.71:0/3694527249 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5780078e0 con 0x7fd588100ea0 2024-04-23T02:42:10.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.087+0000 7fd5857fa700 1 -- 172.21.15.71:0/3694527249 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd57800f450 con 0x7fd588100ea0 2024-04-23T02:42:10.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.087+0000 7fd58e440700 1 -- 172.21.15.71:0/3694527249 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5881a2a40 con 0x7fd588100ea0 2024-04-23T02:42:10.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.087+0000 7fd5857fa700 1 -- 172.21.15.71:0/3694527249 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd57800e5e0 con 0x7fd588100ea0 2024-04-23T02:42:10.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.087+0000 7fd58e440700 1 -- 172.21.15.71:0/3694527249 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5881a2e50 con 0x7fd588100ea0 2024-04-23T02:42:10.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.088+0000 7fd58e440700 1 -- 172.21.15.71:0/3694527249 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5881092a0 con 0x7fd588100ea0 2024-04-23T02:42:10.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.089+0000 7fd5857fa700 1 -- 172.21.15.71:0/3694527249 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd578007a40 con 0x7fd588100ea0 2024-04-23T02:42:10.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.089+0000 7fd5857fa700 1 --2- 172.21.15.71:0/3694527249 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd57006cce0 0x7fd57006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:10.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.089+0000 7fd5857fa700 1 -- 172.21.15.71:0/3694527249 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd57808c4b0 con 0x7fd588100ea0 2024-04-23T02:42:10.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.089+0000 7fd5877fe700 1 --2- 172.21.15.71:0/3694527249 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd57006cce0 0x7fd57006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:10.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.090+0000 7fd5877fe700 1 --2- 172.21.15.71:0/3694527249 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd57006cce0 0x7fd57006f1a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd588074880 tx=0x7fd57c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:10.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.094+0000 7fd5857fa700 1 -- 172.21.15.71:0/3694527249 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd5780565a0 con 0x7fd588100ea0 2024-04-23T02:42:10.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.302+0000 7fd58e440700 1 -- 172.21.15.71:0/3694527249 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5880611d0 con 0x7fd57006cce0 2024-04-23T02:42:10.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.303+0000 7fd5857fa700 1 -- 172.21.15.71:0/3694527249 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd5880611d0 con 0x7fd57006cce0 2024-04-23T02:42:10.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.305+0000 7fd58e440700 1 -- 172.21.15.71:0/3694527249 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd57006cce0 msgr2=0x7fd57006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:10.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.305+0000 7fd58e440700 1 --2- 172.21.15.71:0/3694527249 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd57006cce0 0x7fd57006f1a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd588074880 tx=0x7fd57c01a040 comp rx=0 tx=0).stop 2024-04-23T02:42:10.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.305+0000 7fd58e440700 1 -- 172.21.15.71:0/3694527249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd588100ea0 msgr2=0x7fd588073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:10.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.305+0000 7fd58e440700 1 --2- 172.21.15.71:0/3694527249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd588100ea0 0x7fd588073090 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd57800c8a0 tx=0x7fd57800cbb0 comp rx=0 tx=0).stop 2024-04-23T02:42:10.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.306+0000 7fd58e440700 1 -- 172.21.15.71:0/3694527249 shutdown_connections 2024-04-23T02:42:10.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.306+0000 7fd58e440700 1 --2- 172.21.15.71:0/3694527249 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd57006cce0 0x7fd57006f1a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:10.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.306+0000 7fd58e440700 1 --2- 172.21.15.71:0/3694527249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd588100ea0 0x7fd588073090 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:10.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.306+0000 7fd58e440700 1 --2- 172.21.15.71:0/3694527249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd588101850 0x7fd5880735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:10.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.306+0000 7fd58e440700 1 -- 172.21.15.71:0/3694527249 >> 172.21.15.71:0/3694527249 conn(0x7fd5880fc750 msgr2=0x7fd5880fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:10.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.306+0000 7fd58e440700 1 -- 172.21.15.71:0/3694527249 shutdown_connections 2024-04-23T02:42:10.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.306+0000 7fd58e440700 1 -- 172.21.15.71:0/3694527249 wait complete. 2024-04-23T02:42:10.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.408+0000 7fb766a74700 1 -- 172.21.15.71:0/4176111827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb760105bd0 msgr2=0x7fb760105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:10.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.408+0000 7fb766a74700 1 --2- 172.21.15.71:0/4176111827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb760105bd0 0x7fb760105fb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb754009910 tx=0x7fb754009c20 comp rx=0 tx=0).stop 2024-04-23T02:42:10.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.408+0000 7fb766a74700 1 -- 172.21.15.71:0/4176111827 shutdown_connections 2024-04-23T02:42:10.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.408+0000 7fb766a74700 1 --2- 172.21.15.71:0/4176111827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7600ffc20 0x7fb760100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:10.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.408+0000 7fb766a74700 1 --2- 172.21.15.71:0/4176111827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb760105bd0 0x7fb760105fb0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:10.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.408+0000 7fb766a74700 1 -- 172.21.15.71:0/4176111827 >> 172.21.15.71:0/4176111827 conn(0x7fb760075670 msgr2=0x7fb760075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:10.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.408+0000 7fb766a74700 1 -- 172.21.15.71:0/4176111827 shutdown_connections 2024-04-23T02:42:10.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.408+0000 7fb766a74700 1 -- 172.21.15.71:0/4176111827 wait complete. 2024-04-23T02:42:10.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.409+0000 7fb766a74700 1 Processor -- start 2024-04-23T02:42:10.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.409+0000 7fb766a74700 1 -- start start 2024-04-23T02:42:10.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.409+0000 7fb766a74700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7600ffc20 0x7fb76019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:10.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.409+0000 7fb766a74700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb760105bd0 0x7fb76019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:10.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.409+0000 7fb766a74700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76019d080 con 0x7fb760105bd0 2024-04-23T02:42:10.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.409+0000 7fb766a74700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb760196610 con 0x7fb7600ffc20 2024-04-23T02:42:10.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.409+0000 7fb75ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb760105bd0 0x7fb76019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:10.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.409+0000 7fb75ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb760105bd0 0x7fb76019ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50942/0 (socket says 172.21.15.71:50942) 2024-04-23T02:42:10.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.409+0000 7fb75ffff700 1 -- 172.21.15.71:0/3163541292 learned_addr learned my addr 172.21.15.71:0/3163541292 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:42:10.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.409+0000 7fb764810700 1 --2- 172.21.15.71:0/3163541292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7600ffc20 0x7fb76019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:10.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.409+0000 7fb75ffff700 1 -- 172.21.15.71:0/3163541292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7600ffc20 msgr2=0x7fb76019c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:10.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.409+0000 7fb75ffff700 1 --2- 172.21.15.71:0/3163541292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7600ffc20 0x7fb76019c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:10.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.409+0000 7fb75ffff700 1 -- 172.21.15.71:0/3163541292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7540095f0 con 0x7fb760105bd0 2024-04-23T02:42:10.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.409+0000 7fb75ffff700 1 --2- 172.21.15.71:0/3163541292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb760105bd0 0x7fb76019ca60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00d8d0 tx=0x7fb74c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:10.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.410+0000 7fb75dffb700 1 -- 172.21.15.71:0/3163541292 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb74c009940 con 0x7fb760105bd0 2024-04-23T02:42:10.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.410+0000 7fb766a74700 1 -- 172.21.15.71:0/3163541292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7601968f0 con 0x7fb760105bd0 2024-04-23T02:42:10.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.410+0000 7fb75dffb700 1 -- 172.21.15.71:0/3163541292 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb74c010460 con 0x7fb760105bd0 2024-04-23T02:42:10.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.410+0000 7fb75dffb700 1 -- 172.21.15.71:0/3163541292 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb74c00f5f0 con 0x7fb760105bd0 2024-04-23T02:42:10.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.410+0000 7fb766a74700 1 -- 172.21.15.71:0/3163541292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb760196e40 con 0x7fb760105bd0 2024-04-23T02:42:10.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.411+0000 7fb766a74700 1 -- 172.21.15.71:0/3163541292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb76004f2e0 con 0x7fb760105bd0 2024-04-23T02:42:10.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.411+0000 7fb75dffb700 1 -- 172.21.15.71:0/3163541292 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb74c0105d0 con 0x7fb760105bd0 2024-04-23T02:42:10.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.411+0000 7fb75dffb700 1 --2- 172.21.15.71:0/3163541292 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb75006cce0 0x7fb75006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:10.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.412+0000 7fb75dffb700 1 -- 172.21.15.71:0/3163541292 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb74c08c5e0 con 0x7fb760105bd0 2024-04-23T02:42:10.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.412+0000 7fb764810700 1 --2- 172.21.15.71:0/3163541292 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb75006cce0 0x7fb75006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:10.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.412+0000 7fb764810700 1 --2- 172.21.15.71:0/3163541292 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb75006cce0 0x7fb75006f1a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb754006010 tx=0x7fb75401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:10.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.416+0000 7fb75dffb700 1 -- 172.21.15.71:0/3163541292 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb74c05a0e0 con 0x7fb760105bd0 2024-04-23T02:42:10.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.618+0000 7fb766a74700 1 -- 172.21.15.71:0/3163541292 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb760197180 con 0x7fb75006cce0 2024-04-23T02:42:10.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.629+0000 7fb75dffb700 1 -- 172.21.15.71:0/3163541292 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb760197180 con 0x7fb75006cce0 2024-04-23T02:42:10.671 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:42:10.671 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (25m) 4m ago 37m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:42:10.672 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (37m) 4m ago 37m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:42:10.672 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (36m) 4m ago 36m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:42:10.672 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (23m) 4m ago 37m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:42:10.672 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (23m) 4m ago 36m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:42:10.672 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (25m) 4m ago 36m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:42:10.672 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (20m) 4m ago 31m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:42:10.672 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (19m) 4m ago 31m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:42:10.672 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (4m) 4m ago 31m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:42:10.672 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (19m) 4m ago 31m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:42:10.672 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (27m) 4m ago 38m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:42:10.672 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (27m) 4m ago 36m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:42:10.672 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (24m) 4m ago 38m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:42:10.672 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (24m) 4m ago 36m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:42:10.672 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (26m) 4m ago 37m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:42:10.673 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (26m) 4m ago 36m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:42:10.673 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (23m) 4m ago 35m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:42:10.673 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (22m) 4m ago 34m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:42:10.673 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (22m) 4m ago 34m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:42:10.673 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (21m) 4m ago 33m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:42:10.673 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (20m) 4m ago 33m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:42:10.673 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (20m) 4m ago 32m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:42:10.673 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (26m) 4m ago 36m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:42:10.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.631+0000 7fb766a74700 1 -- 172.21.15.71:0/3163541292 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb75006cce0 msgr2=0x7fb75006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:10.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.631+0000 7fb766a74700 1 --2- 172.21.15.71:0/3163541292 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb75006cce0 0x7fb75006f1a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb754006010 tx=0x7fb75401a040 comp rx=0 tx=0).stop 2024-04-23T02:42:10.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.631+0000 7fb766a74700 1 -- 172.21.15.71:0/3163541292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb760105bd0 msgr2=0x7fb76019ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:10.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.631+0000 7fb766a74700 1 --2- 172.21.15.71:0/3163541292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb760105bd0 0x7fb76019ca60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00d8d0 tx=0x7fb74c00dc90 comp rx=0 tx=0).stop 2024-04-23T02:42:10.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.631+0000 7fb766a74700 1 -- 172.21.15.71:0/3163541292 shutdown_connections 2024-04-23T02:42:10.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.631+0000 7fb766a74700 1 --2- 172.21.15.71:0/3163541292 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb75006cce0 0x7fb75006f1a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:10.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.631+0000 7fb766a74700 1 --2- 172.21.15.71:0/3163541292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7600ffc20 0x7fb76019c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:10.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.631+0000 7fb766a74700 1 --2- 172.21.15.71:0/3163541292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb760105bd0 0x7fb76019ca60 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:10.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.631+0000 7fb766a74700 1 -- 172.21.15.71:0/3163541292 >> 172.21.15.71:0/3163541292 conn(0x7fb760075670 msgr2=0x7fb7600686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:10.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.631+0000 7fb766a74700 1 -- 172.21.15.71:0/3163541292 shutdown_connections 2024-04-23T02:42:10.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.631+0000 7fb766a74700 1 -- 172.21.15.71:0/3163541292 wait complete. 2024-04-23T02:42:10.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.731+0000 7f037aac0700 1 -- 172.21.15.71:0/451329893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0374101840 msgr2=0x7f0374105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:10.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.731+0000 7f037aac0700 1 --2- 172.21.15.71:0/451329893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0374101840 0x7f0374105890 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0360009910 tx=0x7f0360009c20 comp rx=0 tx=0).stop 2024-04-23T02:42:10.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.732+0000 7f037aac0700 1 -- 172.21.15.71:0/451329893 shutdown_connections 2024-04-23T02:42:10.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.732+0000 7f037aac0700 1 --2- 172.21.15.71:0/451329893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0374101840 0x7f0374105890 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:10.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.732+0000 7f037aac0700 1 --2- 172.21.15.71:0/451329893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0374100e90 0x7f0374101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:10.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.732+0000 7f037aac0700 1 -- 172.21.15.71:0/451329893 >> 172.21.15.71:0/451329893 conn(0x7f03740fc740 msgr2=0x7f03740feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:10.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.732+0000 7f037aac0700 1 -- 172.21.15.71:0/451329893 shutdown_connections 2024-04-23T02:42:10.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.732+0000 7f037aac0700 1 -- 172.21.15.71:0/451329893 wait complete. 2024-04-23T02:42:10.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.732+0000 7f037aac0700 1 Processor -- start 2024-04-23T02:42:10.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.732+0000 7f037aac0700 1 -- start start 2024-04-23T02:42:10.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.732+0000 7f037aac0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0374100e90 0x7f037419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:10.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.732+0000 7f037aac0700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0374101840 0x7f037419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:10.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.732+0000 7f037aac0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037419d200 con 0x7f0374100e90 2024-04-23T02:42:10.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.732+0000 7f037aac0700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03741966b0 con 0x7f0374101840 2024-04-23T02:42:10.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.732+0000 7f037885c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0374100e90 0x7f037419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:10.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.733+0000 7f037885c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0374100e90 0x7f037419c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50960/0 (socket says 172.21.15.71:50960) 2024-04-23T02:42:10.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.733+0000 7f037885c700 1 -- 172.21.15.71:0/3662112952 learned_addr learned my addr 172.21.15.71:0/3662112952 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:42:10.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.733+0000 7f0373fff700 1 --2- 172.21.15.71:0/3662112952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0374101840 0x7f037419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:10.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.733+0000 7f037885c700 1 -- 172.21.15.71:0/3662112952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0374101840 msgr2=0x7f037419cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:10.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.733+0000 7f037885c700 1 --2- 172.21.15.71:0/3662112952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0374101840 0x7f037419cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:10.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.733+0000 7f037885c700 1 -- 172.21.15.71:0/3662112952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03600095f0 con 0x7f0374100e90 2024-04-23T02:42:10.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.733+0000 7f037885c700 1 --2- 172.21.15.71:0/3662112952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0374100e90 0x7f037419c630 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f036800d8d0 tx=0x7f036800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:10.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.733+0000 7f0371ffb700 1 -- 172.21.15.71:0/3662112952 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03680098a0 con 0x7f0374100e90 2024-04-23T02:42:10.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.733+0000 7f0371ffb700 1 -- 172.21.15.71:0/3662112952 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0368010460 con 0x7f0374100e90 2024-04-23T02:42:10.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.733+0000 7f0371ffb700 1 -- 172.21.15.71:0/3662112952 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0368009ea0 con 0x7f0374100e90 2024-04-23T02:42:10.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.733+0000 7f037aac0700 1 -- 172.21.15.71:0/3662112952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0374196990 con 0x7f0374100e90 2024-04-23T02:42:10.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.733+0000 7f037aac0700 1 -- 172.21.15.71:0/3662112952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0374196ee0 con 0x7f0374100e90 2024-04-23T02:42:10.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.734+0000 7f037aac0700 1 -- 172.21.15.71:0/3662112952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0374109290 con 0x7f0374100e90 2024-04-23T02:42:10.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.735+0000 7f0371ffb700 1 -- 172.21.15.71:0/3662112952 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f03680105d0 con 0x7f0374100e90 2024-04-23T02:42:10.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.735+0000 7f0371ffb700 1 --2- 172.21.15.71:0/3662112952 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f036406cce0 0x7f036406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:10.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.735+0000 7f0371ffb700 1 -- 172.21.15.71:0/3662112952 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f036808c3a0 con 0x7f0374100e90 2024-04-23T02:42:10.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.735+0000 7f0373fff700 1 --2- 172.21.15.71:0/3662112952 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f036406cce0 0x7f036406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:10.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.736+0000 7f0373fff700 1 --2- 172.21.15.71:0/3662112952 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f036406cce0 0x7f036406f1a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f03740fcfb0 tx=0x7f036001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:10.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:10.740+0000 7f0371ffb700 1 -- 172.21.15.71:0/3662112952 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0368059f20 con 0x7f0374100e90 2024-04-23T02:42:11.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.014+0000 7f037aac0700 1 -- 172.21.15.71:0/3662112952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0374197150 con 0x7f0374100e90 2024-04-23T02:42:11.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.014+0000 7f0371ffb700 1 -- 172.21.15.71:0/3662112952 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f0368010880 con 0x7f0374100e90 2024-04-23T02:42:11.017 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:42:11.017 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:42:11.017 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:42:11.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:42:11.017 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:42:11.017 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:42:11.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:42:11.018 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:42:11.018 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:42:11.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:42:11.018 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:42:11.018 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:42:11.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:42:11.018 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:42:11.018 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:42:11.018 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:42:11.018 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:42:11.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.016+0000 7f037aac0700 1 -- 172.21.15.71:0/3662112952 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f036406cce0 msgr2=0x7f036406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:11.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.016+0000 7f037aac0700 1 --2- 172.21.15.71:0/3662112952 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f036406cce0 0x7f036406f1a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f03740fcfb0 tx=0x7f036001a040 comp rx=0 tx=0).stop 2024-04-23T02:42:11.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.016+0000 7f037aac0700 1 -- 172.21.15.71:0/3662112952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0374100e90 msgr2=0x7f037419c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:11.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.016+0000 7f037aac0700 1 --2- 172.21.15.71:0/3662112952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0374100e90 0x7f037419c630 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f036800d8d0 tx=0x7f036800dbe0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.016+0000 7f037aac0700 1 -- 172.21.15.71:0/3662112952 shutdown_connections 2024-04-23T02:42:11.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.016+0000 7f037aac0700 1 --2- 172.21.15.71:0/3662112952 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f036406cce0 0x7f036406f1a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.016+0000 7f037aac0700 1 --2- 172.21.15.71:0/3662112952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0374100e90 0x7f037419c630 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.016+0000 7f037aac0700 1 --2- 172.21.15.71:0/3662112952 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0374101840 0x7f037419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.016+0000 7f037aac0700 1 -- 172.21.15.71:0/3662112952 >> 172.21.15.71:0/3662112952 conn(0x7f03740fc740 msgr2=0x7f03740fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:11.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.017+0000 7f037aac0700 1 -- 172.21.15.71:0/3662112952 shutdown_connections 2024-04-23T02:42:11.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.017+0000 7f037aac0700 1 -- 172.21.15.71:0/3662112952 wait complete. 2024-04-23T02:42:11.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.118+0000 7fa8ac0dd700 1 -- 172.21.15.71:0/3765234866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8a40ffc20 msgr2=0x7fa8a4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:11.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.118+0000 7fa8ac0dd700 1 --2- 172.21.15.71:0/3765234866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8a40ffc20 0x7fa8a4100080 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa894009910 tx=0x7fa894009c20 comp rx=0 tx=0).stop 2024-04-23T02:42:11.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.118+0000 7fa8ac0dd700 1 -- 172.21.15.71:0/3765234866 shutdown_connections 2024-04-23T02:42:11.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.119+0000 7fa8ac0dd700 1 --2- 172.21.15.71:0/3765234866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8a40ffc20 0x7fa8a4100080 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.119+0000 7fa8ac0dd700 1 --2- 172.21.15.71:0/3765234866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a4105bd0 0x7fa8a4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.119+0000 7fa8ac0dd700 1 -- 172.21.15.71:0/3765234866 >> 172.21.15.71:0/3765234866 conn(0x7fa8a4075670 msgr2=0x7fa8a4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:11.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.119+0000 7fa8ac0dd700 1 -- 172.21.15.71:0/3765234866 shutdown_connections 2024-04-23T02:42:11.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.119+0000 7fa8ac0dd700 1 -- 172.21.15.71:0/3765234866 wait complete. 2024-04-23T02:42:11.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.119+0000 7fa8ac0dd700 1 Processor -- start 2024-04-23T02:42:11.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.119+0000 7fa8ac0dd700 1 -- start start 2024-04-23T02:42:11.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.120+0000 7fa8ac0dd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a40ffc20 0x7fa8a419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:11.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.120+0000 7fa8ac0dd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8a4105bd0 0x7fa8a419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:11.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.120+0000 7fa8ac0dd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a419c060 con 0x7fa8a4105bd0 2024-04-23T02:42:11.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.120+0000 7fa8ac0dd700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a41955a0 con 0x7fa8a40ffc20 2024-04-23T02:42:11.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.120+0000 7fa8a9678700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8a4105bd0 0x7fa8a419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:11.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.120+0000 7fa8a9678700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8a4105bd0 0x7fa8a419ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50982/0 (socket says 172.21.15.71:50982) 2024-04-23T02:42:11.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.120+0000 7fa8a9678700 1 -- 172.21.15.71:0/1711730863 learned_addr learned my addr 172.21.15.71:0/1711730863 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:42:11.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.120+0000 7fa8a9e79700 1 --2- 172.21.15.71:0/1711730863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a40ffc20 0x7fa8a419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:11.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.120+0000 7fa8a9678700 1 -- 172.21.15.71:0/1711730863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a40ffc20 msgr2=0x7fa8a419b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:11.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.120+0000 7fa8a9678700 1 --2- 172.21.15.71:0/1711730863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a40ffc20 0x7fa8a419b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.120+0000 7fa8a9678700 1 -- 172.21.15.71:0/1711730863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8940095f0 con 0x7fa8a4105bd0 2024-04-23T02:42:11.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.120+0000 7fa8a9678700 1 --2- 172.21.15.71:0/1711730863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8a4105bd0 0x7fa8a419ba60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa894006010 tx=0x7fa89400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:11.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.120+0000 7fa89affd700 1 -- 172.21.15.71:0/1711730863 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa89401d070 con 0x7fa8a4105bd0 2024-04-23T02:42:11.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.120+0000 7fa89affd700 1 -- 172.21.15.71:0/1711730863 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa894005940 con 0x7fa8a4105bd0 2024-04-23T02:42:11.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.120+0000 7fa8ac0dd700 1 -- 172.21.15.71:0/1711730863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a4195820 con 0x7fa8a4105bd0 2024-04-23T02:42:11.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.121+0000 7fa89affd700 1 -- 172.21.15.71:0/1711730863 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa894021410 con 0x7fa8a4105bd0 2024-04-23T02:42:11.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.121+0000 7fa8ac0dd700 1 -- 172.21.15.71:0/1711730863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a4195d10 con 0x7fa8a4105bd0 2024-04-23T02:42:11.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.121+0000 7fa8ac0dd700 1 -- 172.21.15.71:0/1711730863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a41080d0 con 0x7fa8a4105bd0 2024-04-23T02:42:11.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.122+0000 7fa89affd700 1 -- 172.21.15.71:0/1711730863 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa894003a60 con 0x7fa8a4105bd0 2024-04-23T02:42:11.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.122+0000 7fa89affd700 1 --2- 172.21.15.71:0/1711730863 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa89006cce0 0x7fa89006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:11.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.122+0000 7fa89affd700 1 -- 172.21.15.71:0/1711730863 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa89408d740 con 0x7fa8a4105bd0 2024-04-23T02:42:11.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.122+0000 7fa8a9e79700 1 --2- 172.21.15.71:0/1711730863 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa89006cce0 0x7fa89006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:11.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.123+0000 7fa8a9e79700 1 --2- 172.21.15.71:0/1711730863 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa89006cce0 0x7fa89006f1a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000ac50 tx=0x7fa8a000a380 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:11.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.127+0000 7fa89affd700 1 -- 172.21.15.71:0/1711730863 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa89405b240 con 0x7fa8a4105bd0 2024-04-23T02:42:11.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.361+0000 7fa8ac0dd700 1 -- 172.21.15.71:0/1711730863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa8a4196b40 con 0x7fa8a4105bd0 2024-04-23T02:42:11.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.361+0000 7fa89affd700 1 -- 172.21.15.71:0/1711730863 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7fa894003d10 con 0x7fa8a4105bd0 2024-04-23T02:42:11.364 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:42:11.364 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:42:11.364 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:42:11.364 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:42:11.364 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:42:11.364 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:42:11.364 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:42:11.365 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:42:11.365 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:42:11.365 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:42:11.365 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:42:11.365 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:42:11.365 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:42:11.365 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:42:11.365 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:42:11.365 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:42:11.365 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:42:11.365 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:42:11.365 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:42:11.365 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:42:11.365 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:42:11.366 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:42:11.366 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:42:11.366 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:42:11.366 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:42:11.366 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:42:11.366 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:42:11.366 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:42:11.366 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:42:11.366 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:42:11.366 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:42:11.366 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:42:11.366 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:42:11.366 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:42:11.367 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:42:11.367 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:42:11.367 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:42:11.367 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:42:11.367 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:42:11.367 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:42:11.367 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:42:11.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.363+0000 7fa8ac0dd700 1 -- 172.21.15.71:0/1711730863 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa89006cce0 msgr2=0x7fa89006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:11.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.363+0000 7fa8ac0dd700 1 --2- 172.21.15.71:0/1711730863 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa89006cce0 0x7fa89006f1a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000ac50 tx=0x7fa8a000a380 comp rx=0 tx=0).stop 2024-04-23T02:42:11.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.363+0000 7fa8ac0dd700 1 -- 172.21.15.71:0/1711730863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8a4105bd0 msgr2=0x7fa8a419ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:11.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.363+0000 7fa8ac0dd700 1 --2- 172.21.15.71:0/1711730863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8a4105bd0 0x7fa8a419ba60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa894006010 tx=0x7fa89400bd70 comp rx=0 tx=0).stop 2024-04-23T02:42:11.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.363+0000 7fa8ac0dd700 1 -- 172.21.15.71:0/1711730863 shutdown_connections 2024-04-23T02:42:11.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.363+0000 7fa8ac0dd700 1 --2- 172.21.15.71:0/1711730863 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa89006cce0 0x7fa89006f1a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.363+0000 7fa8ac0dd700 1 --2- 172.21.15.71:0/1711730863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8a40ffc20 0x7fa8a419b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.363+0000 7fa8ac0dd700 1 --2- 172.21.15.71:0/1711730863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8a4105bd0 0x7fa8a419ba60 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.363+0000 7fa8ac0dd700 1 -- 172.21.15.71:0/1711730863 >> 172.21.15.71:0/1711730863 conn(0x7fa8a4075670 msgr2=0x7fa8a40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:11.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.363+0000 7fa8ac0dd700 1 -- 172.21.15.71:0/1711730863 shutdown_connections 2024-04-23T02:42:11.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.363+0000 7fa8ac0dd700 1 -- 172.21.15.71:0/1711730863 wait complete. 2024-04-23T02:42:11.369 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:42:11.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.467+0000 7fb28afd3700 1 -- 172.21.15.71:0/4126851819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2840ffc20 msgr2=0x7fb284100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:11.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.467+0000 7fb28afd3700 1 --2- 172.21.15.71:0/4126851819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2840ffc20 0x7fb284100080 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb270009910 tx=0x7fb270009c20 comp rx=0 tx=0).stop 2024-04-23T02:42:11.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.467+0000 7fb28afd3700 1 -- 172.21.15.71:0/4126851819 shutdown_connections 2024-04-23T02:42:11.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.467+0000 7fb28afd3700 1 --2- 172.21.15.71:0/4126851819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2840ffc20 0x7fb284100080 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.467+0000 7fb28afd3700 1 --2- 172.21.15.71:0/4126851819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb284105bd0 0x7fb284105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.467+0000 7fb28afd3700 1 -- 172.21.15.71:0/4126851819 >> 172.21.15.71:0/4126851819 conn(0x7fb284075670 msgr2=0x7fb284075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:11.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.467+0000 7fb28afd3700 1 -- 172.21.15.71:0/4126851819 shutdown_connections 2024-04-23T02:42:11.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.467+0000 7fb28afd3700 1 -- 172.21.15.71:0/4126851819 wait complete. 2024-04-23T02:42:11.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.467+0000 7fb28afd3700 1 Processor -- start 2024-04-23T02:42:11.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.467+0000 7fb28afd3700 1 -- start start 2024-04-23T02:42:11.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.467+0000 7fb28afd3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2840ffc20 0x7fb28419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:11.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.467+0000 7fb28afd3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb284105bd0 0x7fb28419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:11.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.467+0000 7fb28afd3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28419d150 con 0x7fb284105bd0 2024-04-23T02:42:11.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.467+0000 7fb28afd3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb284196600 con 0x7fb2840ffc20 2024-04-23T02:42:11.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.468+0000 7fb283fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb284105bd0 0x7fb28419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:11.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.468+0000 7fb283fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb284105bd0 0x7fb28419cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50988/0 (socket says 172.21.15.71:50988) 2024-04-23T02:42:11.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.468+0000 7fb283fff700 1 -- 172.21.15.71:0/471831563 learned_addr learned my addr 172.21.15.71:0/471831563 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:42:11.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.468+0000 7fb288d6f700 1 --2- 172.21.15.71:0/471831563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2840ffc20 0x7fb28419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:11.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.468+0000 7fb283fff700 1 -- 172.21.15.71:0/471831563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2840ffc20 msgr2=0x7fb28419c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:11.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.468+0000 7fb283fff700 1 --2- 172.21.15.71:0/471831563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2840ffc20 0x7fb28419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.468+0000 7fb283fff700 1 -- 172.21.15.71:0/471831563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2700095f0 con 0x7fb284105bd0 2024-04-23T02:42:11.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.468+0000 7fb283fff700 1 --2- 172.21.15.71:0/471831563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb284105bd0 0x7fb28419cac0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb270000c00 tx=0x7fb27000be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:11.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.468+0000 7fb281ffb700 1 -- 172.21.15.71:0/471831563 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb27001d070 con 0x7fb284105bd0 2024-04-23T02:42:11.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.468+0000 7fb281ffb700 1 -- 172.21.15.71:0/471831563 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb270005dc0 con 0x7fb284105bd0 2024-04-23T02:42:11.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.468+0000 7fb28afd3700 1 -- 172.21.15.71:0/471831563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb284196880 con 0x7fb284105bd0 2024-04-23T02:42:11.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.468+0000 7fb281ffb700 1 -- 172.21.15.71:0/471831563 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2700039b0 con 0x7fb284105bd0 2024-04-23T02:42:11.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.468+0000 7fb28afd3700 1 -- 172.21.15.71:0/471831563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb284196d10 con 0x7fb284105bd0 2024-04-23T02:42:11.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.470+0000 7fb28afd3700 1 -- 172.21.15.71:0/471831563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb28404f2e0 con 0x7fb284105bd0 2024-04-23T02:42:11.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.470+0000 7fb281ffb700 1 -- 172.21.15.71:0/471831563 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb2700048e0 con 0x7fb284105bd0 2024-04-23T02:42:11.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.470+0000 7fb281ffb700 1 --2- 172.21.15.71:0/471831563 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb27406cce0 0x7fb27406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:11.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.470+0000 7fb281ffb700 1 -- 172.21.15.71:0/471831563 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb27008dc50 con 0x7fb284105bd0 2024-04-23T02:42:11.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.470+0000 7fb288d6f700 1 --2- 172.21.15.71:0/471831563 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb27406cce0 0x7fb27406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:11.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.471+0000 7fb288d6f700 1 --2- 172.21.15.71:0/471831563 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb27406cce0 0x7fb27406f1a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb278009b30 tx=0x7fb278009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:11.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.475+0000 7fb281ffb700 1 -- 172.21.15.71:0/471831563 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb27005b750 con 0x7fb284105bd0 2024-04-23T02:42:11.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.685+0000 7fb28afd3700 1 -- 172.21.15.71:0/471831563 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb284196fc0 con 0x7fb27406cce0 2024-04-23T02:42:11.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.686+0000 7fb281ffb700 1 -- 172.21.15.71:0/471831563 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb284196fc0 con 0x7fb27406cce0 2024-04-23T02:42:11.689 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:42:11.689 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:42:11.689 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:42:11.689 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:42:11.689 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:42:11.689 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:42:11.689 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:42:11.689 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:42:11.689 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:42:11.689 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:42:11.689 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:42:11.689 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:42:11.689 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:42:11.690 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:42:11.690 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:42:11.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.688+0000 7fb28afd3700 1 -- 172.21.15.71:0/471831563 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb27406cce0 msgr2=0x7fb27406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:11.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.688+0000 7fb28afd3700 1 --2- 172.21.15.71:0/471831563 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb27406cce0 0x7fb27406f1a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb278009b30 tx=0x7fb278009260 comp rx=0 tx=0).stop 2024-04-23T02:42:11.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.688+0000 7fb28afd3700 1 -- 172.21.15.71:0/471831563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb284105bd0 msgr2=0x7fb28419cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:11.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.688+0000 7fb28afd3700 1 --2- 172.21.15.71:0/471831563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb284105bd0 0x7fb28419cac0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb270000c00 tx=0x7fb27000be90 comp rx=0 tx=0).stop 2024-04-23T02:42:11.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.688+0000 7fb28afd3700 1 -- 172.21.15.71:0/471831563 shutdown_connections 2024-04-23T02:42:11.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.688+0000 7fb28afd3700 1 --2- 172.21.15.71:0/471831563 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb27406cce0 0x7fb27406f1a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.688+0000 7fb28afd3700 1 --2- 172.21.15.71:0/471831563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2840ffc20 0x7fb28419c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.688+0000 7fb28afd3700 1 --2- 172.21.15.71:0/471831563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb284105bd0 0x7fb28419cac0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.688+0000 7fb28afd3700 1 -- 172.21.15.71:0/471831563 >> 172.21.15.71:0/471831563 conn(0x7fb284075670 msgr2=0x7fb2840686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:11.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.688+0000 7fb28afd3700 1 -- 172.21.15.71:0/471831563 shutdown_connections 2024-04-23T02:42:11.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.688+0000 7fb28afd3700 1 -- 172.21.15.71:0/471831563 wait complete. 2024-04-23T02:42:11.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.791+0000 7fa0a7e96700 1 -- 172.21.15.71:0/3594622157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0a0105bd0 msgr2=0x7fa0a0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:11.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.791+0000 7fa0a7e96700 1 --2- 172.21.15.71:0/3594622157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0a0105bd0 0x7fa0a0105fb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa090009910 tx=0x7fa090009c20 comp rx=0 tx=0).stop 2024-04-23T02:42:11.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.791+0000 7fa0a7e96700 1 -- 172.21.15.71:0/3594622157 shutdown_connections 2024-04-23T02:42:11.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.791+0000 7fa0a7e96700 1 --2- 172.21.15.71:0/3594622157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0a00ffc20 0x7fa0a0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.791+0000 7fa0a7e96700 1 --2- 172.21.15.71:0/3594622157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0a0105bd0 0x7fa0a0105fb0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.791+0000 7fa0a7e96700 1 -- 172.21.15.71:0/3594622157 >> 172.21.15.71:0/3594622157 conn(0x7fa0a0075670 msgr2=0x7fa0a0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:11.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.791+0000 7fa0a7e96700 1 -- 172.21.15.71:0/3594622157 shutdown_connections 2024-04-23T02:42:11.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.791+0000 7fa0a7e96700 1 -- 172.21.15.71:0/3594622157 wait complete. 2024-04-23T02:42:11.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.792+0000 7fa0a7e96700 1 Processor -- start 2024-04-23T02:42:11.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.792+0000 7fa0a7e96700 1 -- start start 2024-04-23T02:42:11.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.792+0000 7fa0a7e96700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0a00ffc20 0x7fa0a0101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:11.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.792+0000 7fa0a7e96700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0a0105bd0 0x7fa0a0102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:11.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.792+0000 7fa0a7e96700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a01027c0 con 0x7fa0a0105bd0 2024-04-23T02:42:11.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.792+0000 7fa0a7e96700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a0102900 con 0x7fa0a00ffc20 2024-04-23T02:42:11.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.792+0000 7fa0a5431700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0a0105bd0 0x7fa0a0102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:11.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.792+0000 7fa0a5431700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0a0105bd0 0x7fa0a0102280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50994/0 (socket says 172.21.15.71:50994) 2024-04-23T02:42:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.792+0000 7fa0a5431700 1 -- 172.21.15.71:0/1236431713 learned_addr learned my addr 172.21.15.71:0/1236431713 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:42:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.792+0000 7fa0a5c32700 1 --2- 172.21.15.71:0/1236431713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0a00ffc20 0x7fa0a0101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.792+0000 7fa0a5431700 1 -- 172.21.15.71:0/1236431713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0a00ffc20 msgr2=0x7fa0a0101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.792+0000 7fa0a5431700 1 --2- 172.21.15.71:0/1236431713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0a00ffc20 0x7fa0a0101d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.792+0000 7fa0a5431700 1 -- 172.21.15.71:0/1236431713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0900095f0 con 0x7fa0a0105bd0 2024-04-23T02:42:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.792+0000 7fa0a5431700 1 --2- 172.21.15.71:0/1236431713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0a0105bd0 0x7fa0a0102280 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa0940060b0 tx=0x7fa09400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.792+0000 7fa09effd700 1 -- 172.21.15.71:0/1236431713 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa09400dda0 con 0x7fa0a0105bd0 2024-04-23T02:42:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.793+0000 7fa09effd700 1 -- 172.21.15.71:0/1236431713 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa09400f040 con 0x7fa0a0105bd0 2024-04-23T02:42:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.793+0000 7fa0a7e96700 1 -- 172.21.15.71:0/1236431713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a01a2970 con 0x7fa0a0105bd0 2024-04-23T02:42:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.793+0000 7fa09effd700 1 -- 172.21.15.71:0/1236431713 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0940145d0 con 0x7fa0a0105bd0 2024-04-23T02:42:11.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.793+0000 7fa0a7e96700 1 -- 172.21.15.71:0/1236431713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a01a2d80 con 0x7fa0a0105bd0 2024-04-23T02:42:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.793+0000 7fa0a7e96700 1 -- 172.21.15.71:0/1236431713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a004f2e0 con 0x7fa0a0105bd0 2024-04-23T02:42:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.794+0000 7fa09effd700 1 -- 172.21.15.71:0/1236431713 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa094015400 con 0x7fa0a0105bd0 2024-04-23T02:42:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.794+0000 7fa09effd700 1 --2- 172.21.15.71:0/1236431713 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa08806cce0 0x7fa08806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.794+0000 7fa0a5c32700 1 --2- 172.21.15.71:0/1236431713 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa08806cce0 0x7fa08806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.794+0000 7fa09effd700 1 -- 172.21.15.71:0/1236431713 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa09408bfe0 con 0x7fa0a0105bd0 2024-04-23T02:42:11.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.795+0000 7fa0a5c32700 1 --2- 172.21.15.71:0/1236431713 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa08806cce0 0x7fa08806f1a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa09000b5c0 tx=0x7fa09001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:11.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:11.799+0000 7fa09effd700 1 -- 172.21.15.71:0/1236431713 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa094056100 con 0x7fa0a0105bd0 2024-04-23T02:42:11.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:11 smithi192 ceph-mon[108617]: from='client.35380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:42:11.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:11 smithi192 ceph-mon[108617]: from='client.35384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:42:11.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:11 smithi192 ceph-mon[108617]: pgmap v867: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:11.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:11 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3662112952' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:42:11.877 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:11 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1711730863' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:42:11.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:11 smithi071 ceph-mon[144544]: from='client.35380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:42:11.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:11 smithi071 ceph-mon[144544]: from='client.35384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:42:11.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:11 smithi071 ceph-mon[144544]: pgmap v867: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:11.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:11 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3662112952' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:42:11.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:11 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1711730863' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:42:12.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:12.068+0000 7fa0a7e96700 1 -- 172.21.15.71:0/1236431713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa0a01a2fd0 con 0x7fa0a0105bd0 2024-04-23T02:42:12.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:12.068+0000 7fa09effd700 1 -- 172.21.15.71:0/1236431713 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7fa094059720 con 0x7fa0a0105bd0 2024-04-23T02:42:12.071 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:42:12.071 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:42:12.071 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:42:12.071 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:42:12.071 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:42:12.072 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:42:12.072 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:42:12.072 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:42:12.072 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:42:12.072 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:42:12.072 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:42:12.072 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:42:12.072 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:42:12.072 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:42:12.072 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:42:12.072 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:42:12.072 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:42:12.073 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:42:12.073 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:42:12.073 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:42:12.073 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:42:12.073 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:42:12.073 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:42:12.073 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:42:12.073 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:42:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:12.070+0000 7fa0a7e96700 1 -- 172.21.15.71:0/1236431713 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa08806cce0 msgr2=0x7fa08806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:12.070+0000 7fa0a7e96700 1 --2- 172.21.15.71:0/1236431713 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa08806cce0 0x7fa08806f1a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa09000b5c0 tx=0x7fa09001a040 comp rx=0 tx=0).stop 2024-04-23T02:42:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:12.070+0000 7fa0a7e96700 1 -- 172.21.15.71:0/1236431713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0a0105bd0 msgr2=0x7fa0a0102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:12.070+0000 7fa0a7e96700 1 --2- 172.21.15.71:0/1236431713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0a0105bd0 0x7fa0a0102280 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa0940060b0 tx=0x7fa09400d3b0 comp rx=0 tx=0).stop 2024-04-23T02:42:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:12.070+0000 7fa0a7e96700 1 -- 172.21.15.71:0/1236431713 shutdown_connections 2024-04-23T02:42:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:12.070+0000 7fa0a7e96700 1 --2- 172.21.15.71:0/1236431713 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa08806cce0 0x7fa08806f1a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:12.070+0000 7fa0a7e96700 1 --2- 172.21.15.71:0/1236431713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0a00ffc20 0x7fa0a0101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:12.070+0000 7fa0a7e96700 1 --2- 172.21.15.71:0/1236431713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0a0105bd0 0x7fa0a0102280 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:12.070+0000 7fa0a7e96700 1 -- 172.21.15.71:0/1236431713 >> 172.21.15.71:0/1236431713 conn(0x7fa0a0075670 msgr2=0x7fa0a0103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:12.070+0000 7fa0a7e96700 1 -- 172.21.15.71:0/1236431713 shutdown_connections 2024-04-23T02:42:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:12.070+0000 7fa0a7e96700 1 -- 172.21.15.71:0/1236431713 wait complete. 2024-04-23T02:42:12.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:12 smithi192 ceph-mon[108617]: from='client.35388 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:42:12.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:12 smithi192 ceph-mon[108617]: from='client.35400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:42:12.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:12 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1236431713' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:42:12.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:12 smithi192 ceph-mon[108617]: pgmap v868: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:12.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:12 smithi071 ceph-mon[144544]: from='client.35388 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:42:12.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:12 smithi071 ceph-mon[144544]: from='client.35400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:42:12.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:12 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1236431713' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:42:12.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:12 smithi071 ceph-mon[144544]: pgmap v868: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:15.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:15 smithi192 ceph-mon[108617]: pgmap v869: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:15.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:15 smithi071 ceph-mon[144544]: pgmap v869: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:17.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:17 smithi192 ceph-mon[108617]: pgmap v870: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:17.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:17 smithi071 ceph-mon[144544]: pgmap v870: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:19.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:19 smithi192 ceph-mon[108617]: pgmap v871: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:19.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:19 smithi071 ceph-mon[144544]: pgmap v871: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:20.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:42:20.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:42:21.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:21 smithi192 ceph-mon[108617]: pgmap v872: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:21.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:21 smithi071 ceph-mon[144544]: pgmap v872: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:23.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:23 smithi192 ceph-mon[108617]: pgmap v873: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:23.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:23 smithi071 ceph-mon[144544]: pgmap v873: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:25.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:25 smithi192 ceph-mon[108617]: pgmap v874: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:25.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:25 smithi071 ceph-mon[144544]: pgmap v874: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:27.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:27 smithi192 ceph-mon[108617]: pgmap v875: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:27.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:27 smithi071 ceph-mon[144544]: pgmap v875: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:29.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:29 smithi192 ceph-mon[108617]: pgmap v876: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:29.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:29 smithi071 ceph-mon[144544]: pgmap v876: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:31.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:31 smithi192 ceph-mon[108617]: pgmap v877: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:31.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:31 smithi071 ceph-mon[144544]: pgmap v877: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:33.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:33 smithi192 ceph-mon[108617]: pgmap v878: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:33.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:33 smithi071 ceph-mon[144544]: pgmap v878: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:35.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:35 smithi192 ceph-mon[108617]: pgmap v879: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:35.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:42:35.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:35 smithi071 ceph-mon[144544]: pgmap v879: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:35.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:42:37.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:37 smithi192 ceph-mon[108617]: pgmap v880: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:37.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:37 smithi071 ceph-mon[144544]: pgmap v880: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:39.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:39 smithi192 ceph-mon[108617]: pgmap v881: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:39.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:39 smithi071 ceph-mon[144544]: pgmap v881: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:41.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:41 smithi192 ceph-mon[108617]: pgmap v882: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:41.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:41 smithi071 ceph-mon[144544]: pgmap v882: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:42.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.184+0000 7f23119f8700 1 -- 172.21.15.71:0/720046629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c0ff5a0 msgr2=0x7f230c0ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:42.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.184+0000 7f23119f8700 1 --2- 172.21.15.71:0/720046629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c0ff5a0 0x7f230c0ff980 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009910 tx=0x7f22fc009c20 comp rx=0 tx=0).stop 2024-04-23T02:42:42.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.185+0000 7f23119f8700 1 -- 172.21.15.71:0/720046629 shutdown_connections 2024-04-23T02:42:42.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.185+0000 7f23119f8700 1 --2- 172.21.15.71:0/720046629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f230c0fff50 0x7f230c105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:42.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.185+0000 7f23119f8700 1 --2- 172.21.15.71:0/720046629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c0ff5a0 0x7f230c0ff980 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:42.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.185+0000 7f23119f8700 1 -- 172.21.15.71:0/720046629 >> 172.21.15.71:0/720046629 conn(0x7f230c074e30 msgr2=0x7f230c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:42.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.185+0000 7f23119f8700 1 -- 172.21.15.71:0/720046629 shutdown_connections 2024-04-23T02:42:42.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.185+0000 7f23119f8700 1 -- 172.21.15.71:0/720046629 wait complete. 2024-04-23T02:42:42.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.185+0000 7f23119f8700 1 Processor -- start 2024-04-23T02:42:42.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.186+0000 7f23119f8700 1 -- start start 2024-04-23T02:42:42.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.186+0000 7f23119f8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f230c0ff5a0 0x7f230c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:42.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.186+0000 7f23119f8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c0fff50 0x7f230c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:42.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.186+0000 7f23119f8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c19d0a0 con 0x7f230c0fff50 2024-04-23T02:42:42.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.186+0000 7f23119f8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c1965e0 con 0x7f230c0ff5a0 2024-04-23T02:42:42.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.186+0000 7f230a7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c0fff50 0x7f230c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:42.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.186+0000 7f230a7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c0fff50 0x7f230c19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38888/0 (socket says 172.21.15.71:38888) 2024-04-23T02:42:42.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.186+0000 7f230a7fc700 1 -- 172.21.15.71:0/2792898383 learned_addr learned my addr 172.21.15.71:0/2792898383 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:42:42.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.186+0000 7f230affd700 1 --2- 172.21.15.71:0/2792898383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f230c0ff5a0 0x7f230c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:42.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.186+0000 7f230a7fc700 1 -- 172.21.15.71:0/2792898383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f230c0ff5a0 msgr2=0x7f230c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:42.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.186+0000 7f230a7fc700 1 --2- 172.21.15.71:0/2792898383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f230c0ff5a0 0x7f230c19c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:42.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.186+0000 7f230a7fc700 1 -- 172.21.15.71:0/2792898383 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22fc0095f0 con 0x7f230c0fff50 2024-04-23T02:42:42.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.187+0000 7f230a7fc700 1 --2- 172.21.15.71:0/2792898383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c0fff50 0x7f230c19caa0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f230000c960 tx=0x7f230000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:42.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.187+0000 7f23109f6700 1 -- 172.21.15.71:0/2792898383 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2300007a70 con 0x7f230c0fff50 2024-04-23T02:42:42.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.187+0000 7f23109f6700 1 -- 172.21.15.71:0/2792898383 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2300007bd0 con 0x7f230c0fff50 2024-04-23T02:42:42.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.187+0000 7f23119f8700 1 -- 172.21.15.71:0/2792898383 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230c1968c0 con 0x7f230c0fff50 2024-04-23T02:42:42.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.187+0000 7f23109f6700 1 -- 172.21.15.71:0/2792898383 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2300018630 con 0x7f230c0fff50 2024-04-23T02:42:42.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.187+0000 7f23119f8700 1 -- 172.21.15.71:0/2792898383 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230c196e10 con 0x7f230c0fff50 2024-04-23T02:42:42.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.188+0000 7f23119f8700 1 -- 172.21.15.71:0/2792898383 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230c109270 con 0x7f230c0fff50 2024-04-23T02:42:42.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.188+0000 7f23109f6700 1 -- 172.21.15.71:0/2792898383 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f230000f9f0 con 0x7f230c0fff50 2024-04-23T02:42:42.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.189+0000 7f23109f6700 1 --2- 172.21.15.71:0/2792898383 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f22f406cce0 0x7f22f406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:42.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.189+0000 7f23109f6700 1 -- 172.21.15.71:0/2792898383 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f230008c5e0 con 0x7f230c0fff50 2024-04-23T02:42:42.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.189+0000 7f230affd700 1 --2- 172.21.15.71:0/2792898383 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f22f406cce0 0x7f22f406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:42.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.189+0000 7f230affd700 1 --2- 172.21.15.71:0/2792898383 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f22f406cce0 0x7f22f406f1a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00b5c0 tx=0x7f22fc005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:42.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.193+0000 7f23109f6700 1 -- 172.21.15.71:0/2792898383 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2300059e50 con 0x7f230c0fff50 2024-04-23T02:42:42.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.406+0000 7f23119f8700 1 -- 172.21.15.71:0/2792898383 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f230c197090 con 0x7f22f406cce0 2024-04-23T02:42:42.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.407+0000 7f23109f6700 1 -- 172.21.15.71:0/2792898383 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f230c197090 con 0x7f22f406cce0 2024-04-23T02:42:42.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.409+0000 7f23119f8700 1 -- 172.21.15.71:0/2792898383 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f22f406cce0 msgr2=0x7f22f406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:42.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.409+0000 7f23119f8700 1 --2- 172.21.15.71:0/2792898383 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f22f406cce0 0x7f22f406f1a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00b5c0 tx=0x7f22fc005fb0 comp rx=0 tx=0).stop 2024-04-23T02:42:42.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.409+0000 7f23119f8700 1 -- 172.21.15.71:0/2792898383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c0fff50 msgr2=0x7f230c19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:42.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.409+0000 7f23119f8700 1 --2- 172.21.15.71:0/2792898383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c0fff50 0x7f230c19caa0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f230000c960 tx=0x7f230000cd20 comp rx=0 tx=0).stop 2024-04-23T02:42:42.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.409+0000 7f23119f8700 1 -- 172.21.15.71:0/2792898383 shutdown_connections 2024-04-23T02:42:42.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.409+0000 7f23119f8700 1 --2- 172.21.15.71:0/2792898383 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f22f406cce0 0x7f22f406f1a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:42.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.409+0000 7f23119f8700 1 --2- 172.21.15.71:0/2792898383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f230c0ff5a0 0x7f230c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:42.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.409+0000 7f23119f8700 1 --2- 172.21.15.71:0/2792898383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c0fff50 0x7f230c19caa0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:42.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.409+0000 7f23119f8700 1 -- 172.21.15.71:0/2792898383 >> 172.21.15.71:0/2792898383 conn(0x7f230c074e30 msgr2=0x7f230c0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:42.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.409+0000 7f23119f8700 1 -- 172.21.15.71:0/2792898383 shutdown_connections 2024-04-23T02:42:42.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.409+0000 7f23119f8700 1 -- 172.21.15.71:0/2792898383 wait complete. 2024-04-23T02:42:42.423 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:42:42.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.512+0000 7fbbda8e9700 1 -- 172.21.15.71:0/2731035712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbd4101830 msgr2=0x7fbbd4105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:42.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.512+0000 7fbbda8e9700 1 --2- 172.21.15.71:0/2731035712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbd4101830 0x7fbbd4105880 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009910 tx=0x7fbbbc009c20 comp rx=0 tx=0).stop 2024-04-23T02:42:42.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.512+0000 7fbbda8e9700 1 -- 172.21.15.71:0/2731035712 shutdown_connections 2024-04-23T02:42:42.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.512+0000 7fbbda8e9700 1 --2- 172.21.15.71:0/2731035712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbd4101830 0x7fbbd4105880 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:42.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.512+0000 7fbbda8e9700 1 --2- 172.21.15.71:0/2731035712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbd4100e80 0x7fbbd4101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:42.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.512+0000 7fbbda8e9700 1 -- 172.21.15.71:0/2731035712 >> 172.21.15.71:0/2731035712 conn(0x7fbbd40fc730 msgr2=0x7fbbd40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:42.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.512+0000 7fbbda8e9700 1 -- 172.21.15.71:0/2731035712 shutdown_connections 2024-04-23T02:42:42.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.512+0000 7fbbda8e9700 1 -- 172.21.15.71:0/2731035712 wait complete. 2024-04-23T02:42:42.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.513+0000 7fbbda8e9700 1 Processor -- start 2024-04-23T02:42:42.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.513+0000 7fbbda8e9700 1 -- start start 2024-04-23T02:42:42.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.513+0000 7fbbda8e9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbd4100e80 0x7fbbd419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:42.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.513+0000 7fbbda8e9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbd4101830 0x7fbbd419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:42.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.513+0000 7fbbda8e9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd419d260 con 0x7fbbd4100e80 2024-04-23T02:42:42.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.513+0000 7fbbda8e9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd4196710 con 0x7fbbd4101830 2024-04-23T02:42:42.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.514+0000 7fbbd3fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbd4100e80 0x7fbbd419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:42.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.514+0000 7fbbd3fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbd4100e80 0x7fbbd419c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38896/0 (socket says 172.21.15.71:38896) 2024-04-23T02:42:42.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.514+0000 7fbbd3fff700 1 -- 172.21.15.71:0/3823043661 learned_addr learned my addr 172.21.15.71:0/3823043661 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:42:42.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.514+0000 7fbbd37fe700 1 --2- 172.21.15.71:0/3823043661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbd4101830 0x7fbbd419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:42.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.514+0000 7fbbd3fff700 1 -- 172.21.15.71:0/3823043661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbd4101830 msgr2=0x7fbbd419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:42.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.514+0000 7fbbd3fff700 1 --2- 172.21.15.71:0/3823043661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbd4101830 0x7fbbd419cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:42.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.514+0000 7fbbd3fff700 1 -- 172.21.15.71:0/3823043661 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbbc0095f0 con 0x7fbbd4100e80 2024-04-23T02:42:42.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.514+0000 7fbbd3fff700 1 --2- 172.21.15.71:0/3823043661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbd4100e80 0x7fbbd419c690 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400b4b0 tx=0x7fbbc400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:42.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.514+0000 7fbbd17fa700 1 -- 172.21.15.71:0/3823043661 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbc4011800 con 0x7fbbd4100e80 2024-04-23T02:42:42.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.514+0000 7fbbd17fa700 1 -- 172.21.15.71:0/3823043661 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbbc4011e00 con 0x7fbbd4100e80 2024-04-23T02:42:42.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.514+0000 7fbbda8e9700 1 -- 172.21.15.71:0/3823043661 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd41969f0 con 0x7fbbd4100e80 2024-04-23T02:42:42.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.514+0000 7fbbd17fa700 1 -- 172.21.15.71:0/3823043661 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbc4010e50 con 0x7fbbd4100e80 2024-04-23T02:42:42.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.514+0000 7fbbda8e9700 1 -- 172.21.15.71:0/3823043661 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd4196f40 con 0x7fbbd4100e80 2024-04-23T02:42:42.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.515+0000 7fbbda8e9700 1 -- 172.21.15.71:0/3823043661 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd41091b0 con 0x7fbbd4100e80 2024-04-23T02:42:42.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.516+0000 7fbbd17fa700 1 -- 172.21.15.71:0/3823043661 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbbc4010380 con 0x7fbbd4100e80 2024-04-23T02:42:42.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.516+0000 7fbbd17fa700 1 --2- 172.21.15.71:0/3823043661 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbbc006cce0 0x7fbbc006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:42.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.516+0000 7fbbd17fa700 1 -- 172.21.15.71:0/3823043661 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbbc408c190 con 0x7fbbd4100e80 2024-04-23T02:42:42.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.516+0000 7fbbd37fe700 1 --2- 172.21.15.71:0/3823043661 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbbc006cce0 0x7fbbc006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:42.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.516+0000 7fbbd37fe700 1 --2- 172.21.15.71:0/3823043661 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbbc006cce0 0x7fbbc006f1a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00b5c0 tx=0x7fbbbc005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:42.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.521+0000 7fbbd17fa700 1 -- 172.21.15.71:0/3823043661 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbbc4059c90 con 0x7fbbd4100e80 2024-04-23T02:42:42.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.733+0000 7fbbda8e9700 1 -- 172.21.15.71:0/3823043661 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbd41971c0 con 0x7fbbc006cce0 2024-04-23T02:42:42.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.734+0000 7fbbd17fa700 1 -- 172.21.15.71:0/3823043661 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbbd41971c0 con 0x7fbbc006cce0 2024-04-23T02:42:42.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.736+0000 7fbbda8e9700 1 -- 172.21.15.71:0/3823043661 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbbc006cce0 msgr2=0x7fbbc006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:42.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.736+0000 7fbbda8e9700 1 --2- 172.21.15.71:0/3823043661 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbbc006cce0 0x7fbbc006f1a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00b5c0 tx=0x7fbbbc005fd0 comp rx=0 tx=0).stop 2024-04-23T02:42:42.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.736+0000 7fbbda8e9700 1 -- 172.21.15.71:0/3823043661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbd4100e80 msgr2=0x7fbbd419c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:42.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.736+0000 7fbbda8e9700 1 --2- 172.21.15.71:0/3823043661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbd4100e80 0x7fbbd419c690 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400b4b0 tx=0x7fbbc400b870 comp rx=0 tx=0).stop 2024-04-23T02:42:42.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.736+0000 7fbbda8e9700 1 -- 172.21.15.71:0/3823043661 shutdown_connections 2024-04-23T02:42:42.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.736+0000 7fbbda8e9700 1 --2- 172.21.15.71:0/3823043661 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbbc006cce0 0x7fbbc006f1a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:42.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.736+0000 7fbbda8e9700 1 --2- 172.21.15.71:0/3823043661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbd4100e80 0x7fbbd419c690 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:42.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.736+0000 7fbbda8e9700 1 --2- 172.21.15.71:0/3823043661 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbd4101830 0x7fbbd419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:42.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.736+0000 7fbbda8e9700 1 -- 172.21.15.71:0/3823043661 >> 172.21.15.71:0/3823043661 conn(0x7fbbd40fc730 msgr2=0x7fbbd40fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:42.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.736+0000 7fbbda8e9700 1 -- 172.21.15.71:0/3823043661 shutdown_connections 2024-04-23T02:42:42.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.736+0000 7fbbda8e9700 1 -- 172.21.15.71:0/3823043661 wait complete. 2024-04-23T02:42:42.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.838+0000 7fee7e5ef700 1 -- 172.21.15.71:0/47148488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78101060 msgr2=0x7fee781047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:42.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.838+0000 7fee7e5ef700 1 --2- 172.21.15.71:0/47148488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78101060 0x7fee781047b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fee60009910 tx=0x7fee60009c20 comp rx=0 tx=0).stop 2024-04-23T02:42:42.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.838+0000 7fee7e5ef700 1 -- 172.21.15.71:0/47148488 shutdown_connections 2024-04-23T02:42:42.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.838+0000 7fee7e5ef700 1 --2- 172.21.15.71:0/47148488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78101060 0x7fee781047b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:42.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.838+0000 7fee7e5ef700 1 --2- 172.21.15.71:0/47148488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee781006b0 0x7fee78100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:42.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.838+0000 7fee7e5ef700 1 -- 172.21.15.71:0/47148488 >> 172.21.15.71:0/47148488 conn(0x7fee78074e30 msgr2=0x7fee78075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:42.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.838+0000 7fee7e5ef700 1 -- 172.21.15.71:0/47148488 shutdown_connections 2024-04-23T02:42:42.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.838+0000 7fee7e5ef700 1 -- 172.21.15.71:0/47148488 wait complete. 2024-04-23T02:42:42.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.839+0000 7fee7e5ef700 1 Processor -- start 2024-04-23T02:42:42.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.839+0000 7fee7e5ef700 1 -- start start 2024-04-23T02:42:42.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.839+0000 7fee7e5ef700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee781006b0 0x7fee7819b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:42.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.839+0000 7fee7e5ef700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78101060 0x7fee7819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:42.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.839+0000 7fee7e5ef700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7819c1e0 con 0x7fee78101060 2024-04-23T02:42:42.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.839+0000 7fee7e5ef700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee78195690 con 0x7fee781006b0 2024-04-23T02:42:42.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.839+0000 7fee777fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78101060 0x7fee7819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:42.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.839+0000 7fee777fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78101060 0x7fee7819bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38918/0 (socket says 172.21.15.71:38918) 2024-04-23T02:42:42.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.839+0000 7fee777fe700 1 -- 172.21.15.71:0/1381646896 learned_addr learned my addr 172.21.15.71:0/1381646896 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:42:42.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.839+0000 7fee77fff700 1 --2- 172.21.15.71:0/1381646896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee781006b0 0x7fee7819b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:42.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.839+0000 7fee777fe700 1 -- 172.21.15.71:0/1381646896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee781006b0 msgr2=0x7fee7819b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:42.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.839+0000 7fee777fe700 1 --2- 172.21.15.71:0/1381646896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee781006b0 0x7fee7819b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:42.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.839+0000 7fee777fe700 1 -- 172.21.15.71:0/1381646896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee600095f0 con 0x7fee78101060 2024-04-23T02:42:42.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.839+0000 7fee777fe700 1 --2- 172.21.15.71:0/1381646896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78101060 0x7fee7819bb50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fee600092e0 tx=0x7fee6000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:42.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.840+0000 7fee757fa700 1 -- 172.21.15.71:0/1381646896 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee6001d070 con 0x7fee78101060 2024-04-23T02:42:42.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.840+0000 7fee757fa700 1 -- 172.21.15.71:0/1381646896 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fee6000f460 con 0x7fee78101060 2024-04-23T02:42:42.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.840+0000 7fee7e5ef700 1 -- 172.21.15.71:0/1381646896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee78195910 con 0x7fee78101060 2024-04-23T02:42:42.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.840+0000 7fee757fa700 1 -- 172.21.15.71:0/1381646896 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee60021620 con 0x7fee78101060 2024-04-23T02:42:42.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.840+0000 7fee7e5ef700 1 -- 172.21.15.71:0/1381646896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee78195e00 con 0x7fee78101060 2024-04-23T02:42:42.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.841+0000 7fee7e5ef700 1 -- 172.21.15.71:0/1381646896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee78108140 con 0x7fee78101060 2024-04-23T02:42:42.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.841+0000 7fee757fa700 1 -- 172.21.15.71:0/1381646896 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fee6000f5d0 con 0x7fee78101060 2024-04-23T02:42:42.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.842+0000 7fee757fa700 1 --2- 172.21.15.71:0/1381646896 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee6406cce0 0x7fee6406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:42.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.842+0000 7fee757fa700 1 -- 172.21.15.71:0/1381646896 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fee6008dba0 con 0x7fee78101060 2024-04-23T02:42:42.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.842+0000 7fee77fff700 1 --2- 172.21.15.71:0/1381646896 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee6406cce0 0x7fee6406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:42.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.842+0000 7fee77fff700 1 --2- 172.21.15.71:0/1381646896 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee6406cce0 0x7fee6406f1a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fee68009a10 tx=0x7fee68009190 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:42.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:42.846+0000 7fee757fa700 1 -- 172.21.15.71:0/1381646896 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fee6005b720 con 0x7fee78101060 2024-04-23T02:42:43.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.051+0000 7fee7e5ef700 1 -- 172.21.15.71:0/1381646896 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fee78196020 con 0x7fee6406cce0 2024-04-23T02:42:43.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.062+0000 7fee757fa700 1 -- 172.21.15.71:0/1381646896 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fee78196020 con 0x7fee6406cce0 2024-04-23T02:42:43.065 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:42:43.066 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (26m) 4m ago 37m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:42:43.066 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (38m) 4m ago 38m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:42:43.066 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (36m) 4m ago 36m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:42:43.066 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (24m) 4m ago 37m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:42:43.066 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (24m) 4m ago 36m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:42:43.066 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (26m) 4m ago 37m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:42:43.066 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (20m) 4m ago 31m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:42:43.066 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (20m) 4m ago 31m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:42:43.066 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (4m) 4m ago 31m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:42:43.066 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (19m) 4m ago 31m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:42:43.066 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (28m) 4m ago 39m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:42:43.066 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (27m) 4m ago 36m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:42:43.066 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (24m) 4m ago 39m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:42:43.067 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (24m) 4m ago 36m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:42:43.067 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (27m) 4m ago 37m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:42:43.067 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (27m) 4m ago 36m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:42:43.067 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (23m) 4m ago 35m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:42:43.067 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (23m) 4m ago 35m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:42:43.067 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (22m) 4m ago 34m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:42:43.067 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (22m) 4m ago 34m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:42:43.067 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (21m) 4m ago 33m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:42:43.067 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (20m) 4m ago 33m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:42:43.067 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (26m) 4m ago 37m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:42:43.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.064+0000 7fee7e5ef700 1 -- 172.21.15.71:0/1381646896 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee6406cce0 msgr2=0x7fee6406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:43.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.064+0000 7fee7e5ef700 1 --2- 172.21.15.71:0/1381646896 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee6406cce0 0x7fee6406f1a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fee68009a10 tx=0x7fee68009190 comp rx=0 tx=0).stop 2024-04-23T02:42:43.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.064+0000 7fee7e5ef700 1 -- 172.21.15.71:0/1381646896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78101060 msgr2=0x7fee7819bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:43.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.064+0000 7fee7e5ef700 1 --2- 172.21.15.71:0/1381646896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78101060 0x7fee7819bb50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fee600092e0 tx=0x7fee6000bd70 comp rx=0 tx=0).stop 2024-04-23T02:42:43.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.064+0000 7fee7e5ef700 1 -- 172.21.15.71:0/1381646896 shutdown_connections 2024-04-23T02:42:43.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.064+0000 7fee7e5ef700 1 --2- 172.21.15.71:0/1381646896 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fee6406cce0 0x7fee6406f1a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.064+0000 7fee7e5ef700 1 --2- 172.21.15.71:0/1381646896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee781006b0 0x7fee7819b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.064+0000 7fee7e5ef700 1 --2- 172.21.15.71:0/1381646896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee78101060 0x7fee7819bb50 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.064+0000 7fee7e5ef700 1 -- 172.21.15.71:0/1381646896 >> 172.21.15.71:0/1381646896 conn(0x7fee78074e30 msgr2=0x7fee780fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:43.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.064+0000 7fee7e5ef700 1 -- 172.21.15.71:0/1381646896 shutdown_connections 2024-04-23T02:42:43.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.064+0000 7fee7e5ef700 1 -- 172.21.15.71:0/1381646896 wait complete. 2024-04-23T02:42:43.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.166+0000 7f79f47b9700 1 -- 172.21.15.71:0/1241968970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79ec0ffda0 msgr2=0x7f79ec100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.166+0000 7f79f47b9700 1 --2- 172.21.15.71:0/1241968970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79ec0ffda0 0x7f79ec100180 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009910 tx=0x7f79dc009c20 comp rx=0 tx=0).stop 2024-04-23T02:42:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.166+0000 7f79f47b9700 1 -- 172.21.15.71:0/1241968970 shutdown_connections 2024-04-23T02:42:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.166+0000 7f79f47b9700 1 --2- 172.21.15.71:0/1241968970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79ec100750 0x7f79ec1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.166+0000 7f79f47b9700 1 --2- 172.21.15.71:0/1241968970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79ec0ffda0 0x7f79ec100180 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.166+0000 7f79f47b9700 1 -- 172.21.15.71:0/1241968970 >> 172.21.15.71:0/1241968970 conn(0x7f79ec0fb650 msgr2=0x7f79ec0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.166+0000 7f79f47b9700 1 -- 172.21.15.71:0/1241968970 shutdown_connections 2024-04-23T02:42:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.166+0000 7f79f47b9700 1 -- 172.21.15.71:0/1241968970 wait complete. 2024-04-23T02:42:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.167+0000 7f79f47b9700 1 Processor -- start 2024-04-23T02:42:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.167+0000 7f79f47b9700 1 -- start start 2024-04-23T02:42:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.167+0000 7f79f47b9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79ec0ffda0 0x7f79ec1956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.167+0000 7f79f47b9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79ec100750 0x7f79ec195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:43.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.167+0000 7f79f47b9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec196290 con 0x7f79ec100750 2024-04-23T02:42:43.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.167+0000 7f79f47b9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec198fc0 con 0x7f79ec0ffda0 2024-04-23T02:42:43.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.167+0000 7f79f1d54700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79ec100750 0x7f79ec195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:43.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.167+0000 7f79f1d54700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79ec100750 0x7f79ec195c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38942/0 (socket says 172.21.15.71:38942) 2024-04-23T02:42:43.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.167+0000 7f79f1d54700 1 -- 172.21.15.71:0/2050969178 learned_addr learned my addr 172.21.15.71:0/2050969178 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:42:43.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.167+0000 7f79f2555700 1 --2- 172.21.15.71:0/2050969178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79ec0ffda0 0x7f79ec1956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:43.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.168+0000 7f79f1d54700 1 -- 172.21.15.71:0/2050969178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79ec0ffda0 msgr2=0x7f79ec1956c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:43.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.168+0000 7f79f1d54700 1 --2- 172.21.15.71:0/2050969178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79ec0ffda0 0x7f79ec1956c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.168+0000 7f79f1d54700 1 -- 172.21.15.71:0/2050969178 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc0095f0 con 0x7f79ec100750 2024-04-23T02:42:43.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.168+0000 7f79f1d54700 1 --2- 172.21.15.71:0/2050969178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79ec100750 0x7f79ec195c00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f79e800c8f0 tx=0x7f79e800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:43.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.168+0000 7f79e37fe700 1 -- 172.21.15.71:0/2050969178 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79e80078a0 con 0x7f79ec100750 2024-04-23T02:42:43.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.168+0000 7f79f47b9700 1 -- 172.21.15.71:0/2050969178 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ec1992a0 con 0x7f79ec100750 2024-04-23T02:42:43.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.168+0000 7f79e37fe700 1 -- 172.21.15.71:0/2050969178 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f79e800f450 con 0x7f79ec100750 2024-04-23T02:42:43.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.168+0000 7f79e37fe700 1 -- 172.21.15.71:0/2050969178 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79e8007ea0 con 0x7f79ec100750 2024-04-23T02:42:43.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.168+0000 7f79f47b9700 1 -- 172.21.15.71:0/2050969178 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ec1997f0 con 0x7f79ec100750 2024-04-23T02:42:43.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.169+0000 7f79f47b9700 1 -- 172.21.15.71:0/2050969178 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ec108140 con 0x7f79ec100750 2024-04-23T02:42:43.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.169+0000 7f79e37fe700 1 -- 172.21.15.71:0/2050969178 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f79e800f5c0 con 0x7f79ec100750 2024-04-23T02:42:43.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.170+0000 7f79e37fe700 1 --2- 172.21.15.71:0/2050969178 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f79d806cce0 0x7f79d806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:43.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.170+0000 7f79e37fe700 1 -- 172.21.15.71:0/2050969178 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f79e808c4d0 con 0x7f79ec100750 2024-04-23T02:42:43.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.170+0000 7f79f2555700 1 --2- 172.21.15.71:0/2050969178 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f79d806cce0 0x7f79d806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:43.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.170+0000 7f79f2555700 1 --2- 172.21.15.71:0/2050969178 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f79d806cce0 0x7f79d806f1a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f79dc006010 tx=0x7f79dc01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:43.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.174+0000 7f79e37fe700 1 -- 172.21.15.71:0/2050969178 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f79e8056540 con 0x7f79ec100750 2024-04-23T02:42:43.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.443+0000 7f79f47b9700 1 -- 172.21.15.71:0/2050969178 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f79ec04f2e0 con 0x7f79ec100750 2024-04-23T02:42:43.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.443+0000 7f79e37fe700 1 -- 172.21.15.71:0/2050969178 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f79e8059b60 con 0x7f79ec100750 2024-04-23T02:42:43.447 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:42:43.447 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:42:43.448 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:42:43.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:42:43.448 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:42:43.448 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:42:43.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:42:43.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:42:43.448 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:42:43.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:42:43.448 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:42:43.448 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:42:43.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:42:43.448 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:42:43.448 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:42:43.448 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:42:43.448 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:42:43.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.445+0000 7f79f47b9700 1 -- 172.21.15.71:0/2050969178 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f79d806cce0 msgr2=0x7f79d806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:43.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.445+0000 7f79f47b9700 1 --2- 172.21.15.71:0/2050969178 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f79d806cce0 0x7f79d806f1a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f79dc006010 tx=0x7f79dc01a040 comp rx=0 tx=0).stop 2024-04-23T02:42:43.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.445+0000 7f79f47b9700 1 -- 172.21.15.71:0/2050969178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79ec100750 msgr2=0x7f79ec195c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:43.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.445+0000 7f79f47b9700 1 --2- 172.21.15.71:0/2050969178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79ec100750 0x7f79ec195c00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f79e800c8f0 tx=0x7f79e800cc00 comp rx=0 tx=0).stop 2024-04-23T02:42:43.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.445+0000 7f79f47b9700 1 -- 172.21.15.71:0/2050969178 shutdown_connections 2024-04-23T02:42:43.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.445+0000 7f79f47b9700 1 --2- 172.21.15.71:0/2050969178 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f79d806cce0 0x7f79d806f1a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.445+0000 7f79f47b9700 1 --2- 172.21.15.71:0/2050969178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79ec0ffda0 0x7f79ec1956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.445+0000 7f79f47b9700 1 --2- 172.21.15.71:0/2050969178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79ec100750 0x7f79ec195c00 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.445+0000 7f79f47b9700 1 -- 172.21.15.71:0/2050969178 >> 172.21.15.71:0/2050969178 conn(0x7f79ec0fb650 msgr2=0x7f79ec0fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:43.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.446+0000 7f79f47b9700 1 -- 172.21.15.71:0/2050969178 shutdown_connections 2024-04-23T02:42:43.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.446+0000 7f79f47b9700 1 -- 172.21.15.71:0/2050969178 wait complete. 2024-04-23T02:42:43.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.549+0000 7f362d5d6700 1 -- 172.21.15.71:0/4281486157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3628100e90 msgr2=0x7f3628101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:43.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.549+0000 7f362d5d6700 1 --2- 172.21.15.71:0/4281486157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3628100e90 0x7f3628101270 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f361c009910 tx=0x7f361c009c20 comp rx=0 tx=0).stop 2024-04-23T02:42:43.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.549+0000 7f362d5d6700 1 -- 172.21.15.71:0/4281486157 shutdown_connections 2024-04-23T02:42:43.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.549+0000 7f362d5d6700 1 --2- 172.21.15.71:0/4281486157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3628101840 0x7f3628105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.549+0000 7f362d5d6700 1 --2- 172.21.15.71:0/4281486157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3628100e90 0x7f3628101270 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.549+0000 7f362d5d6700 1 -- 172.21.15.71:0/4281486157 >> 172.21.15.71:0/4281486157 conn(0x7f36280fc740 msgr2=0x7f36280feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:43.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.549+0000 7f362d5d6700 1 -- 172.21.15.71:0/4281486157 shutdown_connections 2024-04-23T02:42:43.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.549+0000 7f362d5d6700 1 -- 172.21.15.71:0/4281486157 wait complete. 2024-04-23T02:42:43.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.549+0000 7f362d5d6700 1 Processor -- start 2024-04-23T02:42:43.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.549+0000 7f362d5d6700 1 -- start start 2024-04-23T02:42:43.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.550+0000 7f362d5d6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3628100e90 0x7f362819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:43.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.550+0000 7f362d5d6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3628101840 0x7f362819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:43.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.550+0000 7f362d5d6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362819d190 con 0x7f3628101840 2024-04-23T02:42:43.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.550+0000 7f362d5d6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3628196640 con 0x7f3628100e90 2024-04-23T02:42:43.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.550+0000 7f36267fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3628101840 0x7f362819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:43.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.550+0000 7f36267fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3628101840 0x7f362819cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38962/0 (socket says 172.21.15.71:38962) 2024-04-23T02:42:43.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.550+0000 7f36267fc700 1 -- 172.21.15.71:0/1765019125 learned_addr learned my addr 172.21.15.71:0/1765019125 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:42:43.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.550+0000 7f3626ffd700 1 --2- 172.21.15.71:0/1765019125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3628100e90 0x7f362819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:43.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.550+0000 7f36267fc700 1 -- 172.21.15.71:0/1765019125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3628100e90 msgr2=0x7f362819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:43.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.550+0000 7f36267fc700 1 --2- 172.21.15.71:0/1765019125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3628100e90 0x7f362819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.550+0000 7f36267fc700 1 -- 172.21.15.71:0/1765019125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c0095f0 con 0x7f3628101840 2024-04-23T02:42:43.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.550+0000 7f3626ffd700 1 --2- 172.21.15.71:0/1765019125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3628100e90 0x7f362819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:42:43.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.550+0000 7f36267fc700 1 --2- 172.21.15.71:0/1765019125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3628101840 0x7f362819cb00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f361800d900 tx=0x7f361800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:43.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.550+0000 7f360ffff700 1 -- 172.21.15.71:0/1765019125 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3618009900 con 0x7f3628101840 2024-04-23T02:42:43.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.550+0000 7f360ffff700 1 -- 172.21.15.71:0/1765019125 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3618010460 con 0x7f3628101840 2024-04-23T02:42:43.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.550+0000 7f360ffff700 1 -- 172.21.15.71:0/1765019125 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f361800f5f0 con 0x7f3628101840 2024-04-23T02:42:43.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.550+0000 7f362d5d6700 1 -- 172.21.15.71:0/1765019125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3628196920 con 0x7f3628101840 2024-04-23T02:42:43.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.550+0000 7f362d5d6700 1 -- 172.21.15.71:0/1765019125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3628196e70 con 0x7f3628101840 2024-04-23T02:42:43.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.551+0000 7f362d5d6700 1 -- 172.21.15.71:0/1765019125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3628109230 con 0x7f3628101840 2024-04-23T02:42:43.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.552+0000 7f360ffff700 1 -- 172.21.15.71:0/1765019125 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f36180105d0 con 0x7f3628101840 2024-04-23T02:42:43.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.552+0000 7f360ffff700 1 --2- 172.21.15.71:0/1765019125 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f361006cce0 0x7f361006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:43.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.552+0000 7f360ffff700 1 -- 172.21.15.71:0/1765019125 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f361808c3b0 con 0x7f3628101840 2024-04-23T02:42:43.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.552+0000 7f3626ffd700 1 --2- 172.21.15.71:0/1765019125 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f361006cce0 0x7f361006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:43.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.553+0000 7f3626ffd700 1 --2- 172.21.15.71:0/1765019125 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f361006cce0 0x7f361006f1a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f361c00b5c0 tx=0x7f361c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:43.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.557+0000 7f360ffff700 1 -- 172.21.15.71:0/1765019125 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3618059e90 con 0x7f3628101840 2024-04-23T02:42:43.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.795+0000 7f362d5d6700 1 -- 172.21.15.71:0/1765019125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f362804f2e0 con 0x7f3628101840 2024-04-23T02:42:43.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.795+0000 7f360ffff700 1 -- 172.21.15.71:0/1765019125 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f3618017740 con 0x7f3628101840 2024-04-23T02:42:43.799 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:42:43.799 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:42:43.799 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:42:43.799 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:42:43.799 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:42:43.799 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:42:43.799 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:42:43.800 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:42:43.800 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:42:43.800 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:42:43.800 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:42:43.800 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:42:43.800 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:42:43.800 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:42:43.800 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:42:43.800 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:42:43.800 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:42:43.800 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:42:43.800 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:42:43.801 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:42:43.801 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:42:43.801 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:42:43.801 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:42:43.801 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:42:43.801 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:42:43.801 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:42:43.801 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:42:43.801 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:42:43.801 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:42:43.801 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:42:43.801 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:42:43.802 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:42:43.802 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:42:43.802 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:42:43.802 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:42:43.802 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:42:43.802 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:42:43.802 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:42:43.802 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:42:43.802 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:42:43.802 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:42:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.797+0000 7f362d5d6700 1 -- 172.21.15.71:0/1765019125 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f361006cce0 msgr2=0x7f361006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.797+0000 7f362d5d6700 1 --2- 172.21.15.71:0/1765019125 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f361006cce0 0x7f361006f1a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f361c00b5c0 tx=0x7f361c01a040 comp rx=0 tx=0).stop 2024-04-23T02:42:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.797+0000 7f362d5d6700 1 -- 172.21.15.71:0/1765019125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3628101840 msgr2=0x7f362819cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.797+0000 7f362d5d6700 1 --2- 172.21.15.71:0/1765019125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3628101840 0x7f362819cb00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f361800d900 tx=0x7f361800dcc0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.797+0000 7f362d5d6700 1 -- 172.21.15.71:0/1765019125 shutdown_connections 2024-04-23T02:42:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.797+0000 7f362d5d6700 1 --2- 172.21.15.71:0/1765019125 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f361006cce0 0x7f361006f1a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.797+0000 7f362d5d6700 1 --2- 172.21.15.71:0/1765019125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3628100e90 0x7f362819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.797+0000 7f362d5d6700 1 --2- 172.21.15.71:0/1765019125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3628101840 0x7f362819cb00 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.797+0000 7f362d5d6700 1 -- 172.21.15.71:0/1765019125 >> 172.21.15.71:0/1765019125 conn(0x7f36280fc740 msgr2=0x7f36280fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:43.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.797+0000 7f362d5d6700 1 -- 172.21.15.71:0/1765019125 shutdown_connections 2024-04-23T02:42:43.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.797+0000 7f362d5d6700 1 -- 172.21.15.71:0/1765019125 wait complete. 2024-04-23T02:42:43.804 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:42:43.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:43 smithi192 ceph-mon[108617]: from='client.35408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:42:43.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:43 smithi192 ceph-mon[108617]: pgmap v883: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:43.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:43 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2050969178' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:42:43.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.900+0000 7fb2d2314700 1 -- 172.21.15.71:0/1228407694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc101850 msgr2=0x7fb2cc1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:43.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.900+0000 7fb2d2314700 1 --2- 172.21.15.71:0/1228407694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc101850 0x7fb2cc1058a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009910 tx=0x7fb2c0009c20 comp rx=0 tx=0).stop 2024-04-23T02:42:43.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.900+0000 7fb2d2314700 1 -- 172.21.15.71:0/1228407694 shutdown_connections 2024-04-23T02:42:43.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.900+0000 7fb2d2314700 1 --2- 172.21.15.71:0/1228407694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc101850 0x7fb2cc1058a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.900+0000 7fb2d2314700 1 --2- 172.21.15.71:0/1228407694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2cc100ea0 0x7fb2cc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.900+0000 7fb2d2314700 1 -- 172.21.15.71:0/1228407694 >> 172.21.15.71:0/1228407694 conn(0x7fb2cc0fc750 msgr2=0x7fb2cc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:43.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.900+0000 7fb2d2314700 1 -- 172.21.15.71:0/1228407694 shutdown_connections 2024-04-23T02:42:43.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.900+0000 7fb2d2314700 1 -- 172.21.15.71:0/1228407694 wait complete. 2024-04-23T02:42:43.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.901+0000 7fb2d2314700 1 Processor -- start 2024-04-23T02:42:43.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.901+0000 7fb2d2314700 1 -- start start 2024-04-23T02:42:43.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.901+0000 7fb2d2314700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2cc100ea0 0x7fb2cc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:43.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.901+0000 7fb2d2314700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc101850 0x7fb2cc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:43.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.901+0000 7fb2d2314700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc073b10 con 0x7fb2cc101850 2024-04-23T02:42:43.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.901+0000 7fb2d2314700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc073c50 con 0x7fb2cc100ea0 2024-04-23T02:42:43.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.901+0000 7fb2cb7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc101850 0x7fb2cc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:43.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.901+0000 7fb2cb7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc101850 0x7fb2cc0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38980/0 (socket says 172.21.15.71:38980) 2024-04-23T02:42:43.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.901+0000 7fb2cb7fe700 1 -- 172.21.15.71:0/1312713571 learned_addr learned my addr 172.21.15.71:0/1312713571 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:42:43.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.901+0000 7fb2cb7fe700 1 -- 172.21.15.71:0/1312713571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2cc100ea0 msgr2=0x7fb2cc073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:43.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.901+0000 7fb2cb7fe700 1 --2- 172.21.15.71:0/1312713571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2cc100ea0 0x7fb2cc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:43.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.901+0000 7fb2cb7fe700 1 -- 172.21.15.71:0/1312713571 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c00095f0 con 0x7fb2cc101850 2024-04-23T02:42:43.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.901+0000 7fb2cb7fe700 1 --2- 172.21.15.71:0/1312713571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc101850 0x7fb2cc0735d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00092e0 tx=0x7fb2c000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:43.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.902+0000 7fb2c97fa700 1 -- 172.21.15.71:0/1312713571 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2c001d070 con 0x7fb2cc101850 2024-04-23T02:42:43.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.902+0000 7fb2c97fa700 1 -- 172.21.15.71:0/1312713571 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb2c0005940 con 0x7fb2cc101850 2024-04-23T02:42:43.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.902+0000 7fb2c97fa700 1 -- 172.21.15.71:0/1312713571 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2c000fe50 con 0x7fb2cc101850 2024-04-23T02:42:43.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.902+0000 7fb2d2314700 1 -- 172.21.15.71:0/1312713571 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2cc073ed0 con 0x7fb2cc101850 2024-04-23T02:42:43.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.902+0000 7fb2d2314700 1 -- 172.21.15.71:0/1312713571 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2cc1a2d50 con 0x7fb2cc101850 2024-04-23T02:42:43.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.903+0000 7fb2d2314700 1 -- 172.21.15.71:0/1312713571 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2cc1092a0 con 0x7fb2cc101850 2024-04-23T02:42:43.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.903+0000 7fb2c97fa700 1 -- 172.21.15.71:0/1312713571 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb2c0003a60 con 0x7fb2cc101850 2024-04-23T02:42:43.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.903+0000 7fb2c97fa700 1 --2- 172.21.15.71:0/1312713571 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb2b406cce0 0x7fb2b406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:43.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.904+0000 7fb2c97fa700 1 -- 172.21.15.71:0/1312713571 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb2c008d720 con 0x7fb2cc101850 2024-04-23T02:42:43.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.904+0000 7fb2cbfff700 1 --2- 172.21.15.71:0/1312713571 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb2b406cce0 0x7fb2b406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:43.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.904+0000 7fb2cbfff700 1 --2- 172.21.15.71:0/1312713571 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb2b406cce0 0x7fb2b406f1a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc005950 tx=0x7fb2bc0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:43.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:43.908+0000 7fb2c97fa700 1 -- 172.21.15.71:0/1312713571 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb2c0057790 con 0x7fb2cc101850 2024-04-23T02:42:43.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:43 smithi071 ceph-mon[144544]: from='client.35408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:42:43.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:43 smithi071 ceph-mon[144544]: pgmap v883: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:43.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:43 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2050969178' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:42:44.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.118+0000 7fb2d2314700 1 -- 172.21.15.71:0/1312713571 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2cc0611d0 con 0x7fb2b406cce0 2024-04-23T02:42:44.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.120+0000 7fb2c97fa700 1 -- 172.21.15.71:0/1312713571 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb2cc0611d0 con 0x7fb2b406cce0 2024-04-23T02:42:44.123 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:42:44.123 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:42:44.124 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:42:44.124 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:42:44.124 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:42:44.124 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:42:44.124 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:42:44.124 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:42:44.124 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:42:44.124 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:42:44.124 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:42:44.124 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:42:44.124 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:42:44.124 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:42:44.125 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:42:44.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.122+0000 7fb2d2314700 1 -- 172.21.15.71:0/1312713571 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb2b406cce0 msgr2=0x7fb2b406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:44.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.122+0000 7fb2d2314700 1 --2- 172.21.15.71:0/1312713571 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb2b406cce0 0x7fb2b406f1a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc005950 tx=0x7fb2bc0058e0 comp rx=0 tx=0).stop 2024-04-23T02:42:44.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.122+0000 7fb2d2314700 1 -- 172.21.15.71:0/1312713571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc101850 msgr2=0x7fb2cc0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:44.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.122+0000 7fb2d2314700 1 --2- 172.21.15.71:0/1312713571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc101850 0x7fb2cc0735d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00092e0 tx=0x7fb2c000be20 comp rx=0 tx=0).stop 2024-04-23T02:42:44.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.122+0000 7fb2d2314700 1 -- 172.21.15.71:0/1312713571 shutdown_connections 2024-04-23T02:42:44.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.122+0000 7fb2d2314700 1 --2- 172.21.15.71:0/1312713571 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb2b406cce0 0x7fb2b406f1a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:44.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.122+0000 7fb2d2314700 1 --2- 172.21.15.71:0/1312713571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2cc100ea0 0x7fb2cc073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:44.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.122+0000 7fb2d2314700 1 --2- 172.21.15.71:0/1312713571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2cc101850 0x7fb2cc0735d0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:44.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.122+0000 7fb2d2314700 1 -- 172.21.15.71:0/1312713571 >> 172.21.15.71:0/1312713571 conn(0x7fb2cc0fc750 msgr2=0x7fb2cc0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:44.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.122+0000 7fb2d2314700 1 -- 172.21.15.71:0/1312713571 shutdown_connections 2024-04-23T02:42:44.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.122+0000 7fb2d2314700 1 -- 172.21.15.71:0/1312713571 wait complete. 2024-04-23T02:42:44.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.224+0000 7fe8eeb6a700 1 -- 172.21.15.71:0/2733660914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8e8101840 msgr2=0x7fe8e8105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.224+0000 7fe8eeb6a700 1 --2- 172.21.15.71:0/2733660914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8e8101840 0x7fe8e8105890 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009910 tx=0x7fe8d4009c20 comp rx=0 tx=0).stop 2024-04-23T02:42:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.224+0000 7fe8eeb6a700 1 -- 172.21.15.71:0/2733660914 shutdown_connections 2024-04-23T02:42:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.224+0000 7fe8eeb6a700 1 --2- 172.21.15.71:0/2733660914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8e8101840 0x7fe8e8105890 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.224+0000 7fe8eeb6a700 1 --2- 172.21.15.71:0/2733660914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8e8100e90 0x7fe8e8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.224+0000 7fe8eeb6a700 1 -- 172.21.15.71:0/2733660914 >> 172.21.15.71:0/2733660914 conn(0x7fe8e80fc740 msgr2=0x7fe8e80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.224+0000 7fe8eeb6a700 1 -- 172.21.15.71:0/2733660914 shutdown_connections 2024-04-23T02:42:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.224+0000 7fe8eeb6a700 1 -- 172.21.15.71:0/2733660914 wait complete. 2024-04-23T02:42:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.225+0000 7fe8eeb6a700 1 Processor -- start 2024-04-23T02:42:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.225+0000 7fe8eeb6a700 1 -- start start 2024-04-23T02:42:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.225+0000 7fe8eeb6a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8e8100e90 0x7fe8e8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.225+0000 7fe8eeb6a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8e8101840 0x7fe8e80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.225+0000 7fe8eeb6a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e8073b10 con 0x7fe8e8101840 2024-04-23T02:42:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.225+0000 7fe8eeb6a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e8073c50 con 0x7fe8e8100e90 2024-04-23T02:42:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.225+0000 7fe8e7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8e8101840 0x7fe8e80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.225+0000 7fe8e7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8e8101840 0x7fe8e80735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39000/0 (socket says 172.21.15.71:39000) 2024-04-23T02:42:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.225+0000 7fe8e7fff700 1 -- 172.21.15.71:0/2445738633 learned_addr learned my addr 172.21.15.71:0/2445738633 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:42:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.225+0000 7fe8ec906700 1 --2- 172.21.15.71:0/2445738633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8e8100e90 0x7fe8e8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.225+0000 7fe8e7fff700 1 -- 172.21.15.71:0/2445738633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8e8100e90 msgr2=0x7fe8e8073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.225+0000 7fe8e7fff700 1 --2- 172.21.15.71:0/2445738633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8e8100e90 0x7fe8e8073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.225+0000 7fe8e7fff700 1 -- 172.21.15.71:0/2445738633 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d40095f0 con 0x7fe8e8101840 2024-04-23T02:42:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.226+0000 7fe8e7fff700 1 --2- 172.21.15.71:0/2445738633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8e8101840 0x7fe8e80735d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40092e0 tx=0x7fe8d400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.226+0000 7fe8e5ffb700 1 -- 172.21.15.71:0/2445738633 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8d401d070 con 0x7fe8e8101840 2024-04-23T02:42:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.226+0000 7fe8e5ffb700 1 -- 172.21.15.71:0/2445738633 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe8d4005940 con 0x7fe8e8101840 2024-04-23T02:42:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.226+0000 7fe8eeb6a700 1 -- 172.21.15.71:0/2445738633 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e8073ed0 con 0x7fe8e8101840 2024-04-23T02:42:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.226+0000 7fe8e5ffb700 1 -- 172.21.15.71:0/2445738633 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8d400fe50 con 0x7fe8e8101840 2024-04-23T02:42:44.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.226+0000 7fe8eeb6a700 1 -- 172.21.15.71:0/2445738633 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e81a2d60 con 0x7fe8e8101840 2024-04-23T02:42:44.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.227+0000 7fe8eeb6a700 1 -- 172.21.15.71:0/2445738633 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e8109290 con 0x7fe8e8101840 2024-04-23T02:42:44.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.227+0000 7fe8e5ffb700 1 -- 172.21.15.71:0/2445738633 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe8d4003a60 con 0x7fe8e8101840 2024-04-23T02:42:44.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.228+0000 7fe8e5ffb700 1 --2- 172.21.15.71:0/2445738633 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe8d806cce0 0x7fe8d806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:42:44.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.228+0000 7fe8e5ffb700 1 -- 172.21.15.71:0/2445738633 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe8d408d720 con 0x7fe8e8101840 2024-04-23T02:42:44.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.228+0000 7fe8ec906700 1 --2- 172.21.15.71:0/2445738633 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe8d806cce0 0x7fe8d806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:42:44.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.228+0000 7fe8ec906700 1 --2- 172.21.15.71:0/2445738633 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe8d806cce0 0x7fe8d806f1a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc005950 tx=0x7fe8dc0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:42:44.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.232+0000 7fe8e5ffb700 1 -- 172.21.15.71:0/2445738633 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe8d4057790 con 0x7fe8e8101840 2024-04-23T02:42:44.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.501+0000 7fe8eeb6a700 1 -- 172.21.15.71:0/2445738633 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe8e80745c0 con 0x7fe8e8101840 2024-04-23T02:42:44.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.501+0000 7fe8e5ffb700 1 -- 172.21.15.71:0/2445738633 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7fe8d405adb0 con 0x7fe8e8101840 2024-04-23T02:42:44.505 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:42:44.505 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:42:44.505 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:42:44.505 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:42:44.505 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:42:44.505 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:42:44.505 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:42:44.505 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:42:44.505 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:42:44.505 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:42:44.505 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:42:44.506 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:42:44.506 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:42:44.506 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:42:44.506 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:42:44.506 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:42:44.506 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:42:44.506 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:42:44.506 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:42:44.506 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:42:44.506 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:42:44.506 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:42:44.506 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:42:44.506 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:42:44.506 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:42:44.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.503+0000 7fe8eeb6a700 1 -- 172.21.15.71:0/2445738633 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe8d806cce0 msgr2=0x7fe8d806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:44.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.503+0000 7fe8eeb6a700 1 --2- 172.21.15.71:0/2445738633 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe8d806cce0 0x7fe8d806f1a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc005950 tx=0x7fe8dc0058e0 comp rx=0 tx=0).stop 2024-04-23T02:42:44.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.503+0000 7fe8eeb6a700 1 -- 172.21.15.71:0/2445738633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8e8101840 msgr2=0x7fe8e80735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:42:44.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.503+0000 7fe8eeb6a700 1 --2- 172.21.15.71:0/2445738633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8e8101840 0x7fe8e80735d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40092e0 tx=0x7fe8d400be20 comp rx=0 tx=0).stop 2024-04-23T02:42:44.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.503+0000 7fe8eeb6a700 1 -- 172.21.15.71:0/2445738633 shutdown_connections 2024-04-23T02:42:44.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.503+0000 7fe8eeb6a700 1 --2- 172.21.15.71:0/2445738633 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe8d806cce0 0x7fe8d806f1a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:44.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.503+0000 7fe8eeb6a700 1 --2- 172.21.15.71:0/2445738633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8e8100e90 0x7fe8e8073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:44.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.503+0000 7fe8eeb6a700 1 --2- 172.21.15.71:0/2445738633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8e8101840 0x7fe8e80735d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:42:44.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.503+0000 7fe8eeb6a700 1 -- 172.21.15.71:0/2445738633 >> 172.21.15.71:0/2445738633 conn(0x7fe8e80fc740 msgr2=0x7fe8e80fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:42:44.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.503+0000 7fe8eeb6a700 1 -- 172.21.15.71:0/2445738633 shutdown_connections 2024-04-23T02:42:44.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:42:44.503+0000 7fe8eeb6a700 1 -- 172.21.15.71:0/2445738633 wait complete. 2024-04-23T02:42:44.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:44 smithi192 ceph-mon[108617]: from='client.35412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:42:44.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:44 smithi192 ceph-mon[108617]: from='client.35416 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:42:44.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:44 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1765019125' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:42:44.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:44 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2445738633' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:42:44.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:44 smithi071 ceph-mon[144544]: from='client.35412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:42:44.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:44 smithi071 ceph-mon[144544]: from='client.35416 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:42:44.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:44 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1765019125' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:42:44.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:44 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2445738633' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:42:45.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:45 smithi192 ceph-mon[108617]: from='client.35428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:42:45.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:45 smithi192 ceph-mon[108617]: pgmap v884: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:45.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:45 smithi071 ceph-mon[144544]: from='client.35428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:42:45.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:45 smithi071 ceph-mon[144544]: pgmap v884: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:46.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:46 smithi192 ceph-mon[108617]: pgmap v885: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:46.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:46 smithi071 ceph-mon[144544]: pgmap v885: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:49.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:49 smithi192 ceph-mon[108617]: pgmap v886: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:49.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:49 smithi071 ceph-mon[144544]: pgmap v886: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:42:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:42:51.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:51 smithi192 ceph-mon[108617]: pgmap v887: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:51.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:51 smithi071 ceph-mon[144544]: pgmap v887: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:53.828 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:53 smithi071 ceph-mon[144544]: pgmap v888: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:53.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:53 smithi192 ceph-mon[108617]: pgmap v888: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:55.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:55 smithi192 ceph-mon[108617]: pgmap v889: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:55.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:55 smithi071 ceph-mon[144544]: pgmap v889: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:57.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:57 smithi192 ceph-mon[108617]: pgmap v890: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:57.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:57 smithi071 ceph-mon[144544]: pgmap v890: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:59.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:42:59 smithi192 ceph-mon[108617]: pgmap v891: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:42:59.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:42:59 smithi071 ceph-mon[144544]: pgmap v891: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:01.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:01 smithi192 ceph-mon[108617]: pgmap v892: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:01.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:01 smithi071 ceph-mon[144544]: pgmap v892: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:03.861 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:03 smithi192 ceph-mon[108617]: pgmap v893: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:03.922 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:03 smithi071 ceph-mon[144544]: pgmap v893: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:05 smithi192 ceph-mon[108617]: pgmap v894: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:43:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:05 smithi071 ceph-mon[144544]: pgmap v894: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:43:07.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:07 smithi192 ceph-mon[108617]: pgmap v895: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:07.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:07 smithi071 ceph-mon[144544]: pgmap v895: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:09.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:09 smithi192 ceph-mon[108617]: pgmap v896: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:09.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:09 smithi071 ceph-mon[144544]: pgmap v896: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:11.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:11 smithi192 ceph-mon[108617]: pgmap v897: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:11.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:11 smithi071 ceph-mon[144544]: pgmap v897: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:13.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:13 smithi192 ceph-mon[108617]: pgmap v898: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:13.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:13 smithi071 ceph-mon[144544]: pgmap v898: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:14.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.620+0000 7f3295e7b700 1 -- 172.21.15.71:0/2388862259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3290100750 msgr2=0x7f32901047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:14.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.620+0000 7f3295e7b700 1 --2- 172.21.15.71:0/2388862259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3290100750 0x7f32901047a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3280009910 tx=0x7f3280009c20 comp rx=0 tx=0).stop 2024-04-23T02:43:14.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.620+0000 7f3295e7b700 1 -- 172.21.15.71:0/2388862259 shutdown_connections 2024-04-23T02:43:14.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.620+0000 7f3295e7b700 1 --2- 172.21.15.71:0/2388862259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3290100750 0x7f32901047a0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:14.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.620+0000 7f3295e7b700 1 --2- 172.21.15.71:0/2388862259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32900ffda0 0x7f3290100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:14.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.620+0000 7f3295e7b700 1 -- 172.21.15.71:0/2388862259 >> 172.21.15.71:0/2388862259 conn(0x7f32900fb650 msgr2=0x7f32900fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:14.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.620+0000 7f3295e7b700 1 -- 172.21.15.71:0/2388862259 shutdown_connections 2024-04-23T02:43:14.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.620+0000 7f3295e7b700 1 -- 172.21.15.71:0/2388862259 wait complete. 2024-04-23T02:43:14.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.621+0000 7f3295e7b700 1 Processor -- start 2024-04-23T02:43:14.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.621+0000 7f3295e7b700 1 -- start start 2024-04-23T02:43:14.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.621+0000 7f3295e7b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32900ffda0 0x7f3290101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:14.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.621+0000 7f3295e7b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3290100750 0x7f3290101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:14.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.621+0000 7f3295e7b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32901022c0 con 0x7f3290100750 2024-04-23T02:43:14.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.621+0000 7f3295e7b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3290102400 con 0x7f32900ffda0 2024-04-23T02:43:14.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.621+0000 7f328effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3290100750 0x7f3290101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:14.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.621+0000 7f328effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3290100750 0x7f3290101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51444/0 (socket says 172.21.15.71:51444) 2024-04-23T02:43:14.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.621+0000 7f328effd700 1 -- 172.21.15.71:0/2963611774 learned_addr learned my addr 172.21.15.71:0/2963611774 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:43:14.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.622+0000 7f328f7fe700 1 --2- 172.21.15.71:0/2963611774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32900ffda0 0x7f3290101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:14.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.622+0000 7f328effd700 1 -- 172.21.15.71:0/2963611774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32900ffda0 msgr2=0x7f3290101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:14.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.622+0000 7f328effd700 1 --2- 172.21.15.71:0/2963611774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32900ffda0 0x7f3290101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:14.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.622+0000 7f328effd700 1 -- 172.21.15.71:0/2963611774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32800095f0 con 0x7f3290100750 2024-04-23T02:43:14.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.622+0000 7f328effd700 1 --2- 172.21.15.71:0/2963611774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3290100750 0x7f3290101d80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3280005f50 tx=0x7f328000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:14.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.622+0000 7f328cff9700 1 -- 172.21.15.71:0/2963611774 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f328001d070 con 0x7f3290100750 2024-04-23T02:43:14.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.622+0000 7f328cff9700 1 -- 172.21.15.71:0/2963611774 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f328000f460 con 0x7f3290100750 2024-04-23T02:43:14.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.622+0000 7f3295e7b700 1 -- 172.21.15.71:0/2963611774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3290102680 con 0x7f3290100750 2024-04-23T02:43:14.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.622+0000 7f328cff9700 1 -- 172.21.15.71:0/2963611774 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3280021640 con 0x7f3290100750 2024-04-23T02:43:14.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.622+0000 7f3295e7b700 1 -- 172.21.15.71:0/2963611774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32901a2cb0 con 0x7f3290100750 2024-04-23T02:43:14.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.623+0000 7f3295e7b700 1 -- 172.21.15.71:0/2963611774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32901091b0 con 0x7f3290100750 2024-04-23T02:43:14.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.623+0000 7f328cff9700 1 -- 172.21.15.71:0/2963611774 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f328000f9e0 con 0x7f3290100750 2024-04-23T02:43:14.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.624+0000 7f328cff9700 1 --2- 172.21.15.71:0/2963611774 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f327806cce0 0x7f327806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:14.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.624+0000 7f328cff9700 1 -- 172.21.15.71:0/2963611774 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f328008d880 con 0x7f3290100750 2024-04-23T02:43:14.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.624+0000 7f328f7fe700 1 --2- 172.21.15.71:0/2963611774 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f327806cce0 0x7f327806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:14.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.624+0000 7f328f7fe700 1 --2- 172.21.15.71:0/2963611774 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f327806cce0 0x7f327806f1a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3284005fd0 tx=0x7f328400a560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:14.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.628+0000 7f328cff9700 1 -- 172.21.15.71:0/2963611774 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f32800578f0 con 0x7f3290100750 2024-04-23T02:43:14.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.838+0000 7f3295e7b700 1 -- 172.21.15.71:0/2963611774 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f32900611d0 con 0x7f327806cce0 2024-04-23T02:43:14.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.839+0000 7f328cff9700 1 -- 172.21.15.71:0/2963611774 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f32900611d0 con 0x7f327806cce0 2024-04-23T02:43:14.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.841+0000 7f3295e7b700 1 -- 172.21.15.71:0/2963611774 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f327806cce0 msgr2=0x7f327806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:14.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.841+0000 7f3295e7b700 1 --2- 172.21.15.71:0/2963611774 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f327806cce0 0x7f327806f1a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3284005fd0 tx=0x7f328400a560 comp rx=0 tx=0).stop 2024-04-23T02:43:14.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.841+0000 7f3295e7b700 1 -- 172.21.15.71:0/2963611774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3290100750 msgr2=0x7f3290101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:14.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.841+0000 7f3295e7b700 1 --2- 172.21.15.71:0/2963611774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3290100750 0x7f3290101d80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3280005f50 tx=0x7f328000be20 comp rx=0 tx=0).stop 2024-04-23T02:43:14.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.841+0000 7f3295e7b700 1 -- 172.21.15.71:0/2963611774 shutdown_connections 2024-04-23T02:43:14.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.841+0000 7f3295e7b700 1 --2- 172.21.15.71:0/2963611774 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f327806cce0 0x7f327806f1a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:14.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.841+0000 7f3295e7b700 1 --2- 172.21.15.71:0/2963611774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32900ffda0 0x7f3290101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:14.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.841+0000 7f3295e7b700 1 --2- 172.21.15.71:0/2963611774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3290100750 0x7f3290101d80 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:14.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.841+0000 7f3295e7b700 1 -- 172.21.15.71:0/2963611774 >> 172.21.15.71:0/2963611774 conn(0x7f32900fb650 msgr2=0x7f3290069190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:14.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.841+0000 7f3295e7b700 1 -- 172.21.15.71:0/2963611774 shutdown_connections 2024-04-23T02:43:14.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.841+0000 7f3295e7b700 1 -- 172.21.15.71:0/2963611774 wait complete. 2024-04-23T02:43:14.854 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:43:14.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.947+0000 7f46ba7e9700 1 -- 172.21.15.71:0/3390052006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40fff50 msgr2=0x7f46b4105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:14.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.947+0000 7f46ba7e9700 1 --2- 172.21.15.71:0/3390052006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40fff50 0x7f46b4105840 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009910 tx=0x7f46a8009c20 comp rx=0 tx=0).stop 2024-04-23T02:43:14.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.947+0000 7f46ba7e9700 1 -- 172.21.15.71:0/3390052006 shutdown_connections 2024-04-23T02:43:14.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.947+0000 7f46ba7e9700 1 --2- 172.21.15.71:0/3390052006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40fff50 0x7f46b4105840 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:14.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.947+0000 7f46ba7e9700 1 --2- 172.21.15.71:0/3390052006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b40ff5a0 0x7f46b40ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:14.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.947+0000 7f46ba7e9700 1 -- 172.21.15.71:0/3390052006 >> 172.21.15.71:0/3390052006 conn(0x7f46b4074e30 msgr2=0x7f46b4075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:14.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.947+0000 7f46ba7e9700 1 -- 172.21.15.71:0/3390052006 shutdown_connections 2024-04-23T02:43:14.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.947+0000 7f46ba7e9700 1 -- 172.21.15.71:0/3390052006 wait complete. 2024-04-23T02:43:14.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.948+0000 7f46ba7e9700 1 Processor -- start 2024-04-23T02:43:14.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.948+0000 7f46ba7e9700 1 -- start start 2024-04-23T02:43:14.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.948+0000 7f46ba7e9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b40ff5a0 0x7f46b419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:14.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.948+0000 7f46ba7e9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40fff50 0x7f46b419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:14.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.948+0000 7f46ba7e9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b419d260 con 0x7f46b40fff50 2024-04-23T02:43:14.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.948+0000 7f46ba7e9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b4196710 con 0x7f46b40ff5a0 2024-04-23T02:43:14.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.948+0000 7f46b37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40fff50 0x7f46b419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:14.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.948+0000 7f46b37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40fff50 0x7f46b419cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51466/0 (socket says 172.21.15.71:51466) 2024-04-23T02:43:14.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.948+0000 7f46b37fe700 1 -- 172.21.15.71:0/3279198237 learned_addr learned my addr 172.21.15.71:0/3279198237 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:43:14.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.948+0000 7f46b37fe700 1 -- 172.21.15.71:0/3279198237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b40ff5a0 msgr2=0x7f46b419c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:14.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.948+0000 7f46b3fff700 1 --2- 172.21.15.71:0/3279198237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b40ff5a0 0x7f46b419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:14.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.949+0000 7f46b37fe700 1 --2- 172.21.15.71:0/3279198237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b40ff5a0 0x7f46b419c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:14.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.949+0000 7f46b37fe700 1 -- 172.21.15.71:0/3279198237 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a80095f0 con 0x7f46b40fff50 2024-04-23T02:43:14.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.949+0000 7f46b37fe700 1 --2- 172.21.15.71:0/3279198237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40fff50 0x7f46b419cbd0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f46a80092e0 tx=0x7f46a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:14.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.949+0000 7f46b17fa700 1 -- 172.21.15.71:0/3279198237 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46a801d070 con 0x7f46b40fff50 2024-04-23T02:43:14.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.949+0000 7f46b17fa700 1 -- 172.21.15.71:0/3279198237 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f46a8005940 con 0x7f46b40fff50 2024-04-23T02:43:14.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.949+0000 7f46ba7e9700 1 -- 172.21.15.71:0/3279198237 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b4196990 con 0x7f46b40fff50 2024-04-23T02:43:14.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.949+0000 7f46b17fa700 1 -- 172.21.15.71:0/3279198237 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46a800fe50 con 0x7f46b40fff50 2024-04-23T02:43:14.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.949+0000 7f46ba7e9700 1 -- 172.21.15.71:0/3279198237 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b4196e80 con 0x7f46b40fff50 2024-04-23T02:43:14.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.950+0000 7f46ba7e9700 1 -- 172.21.15.71:0/3279198237 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b4109200 con 0x7f46b40fff50 2024-04-23T02:43:14.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.950+0000 7f46b17fa700 1 -- 172.21.15.71:0/3279198237 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f46a8003a60 con 0x7f46b40fff50 2024-04-23T02:43:14.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.951+0000 7f46b17fa700 1 --2- 172.21.15.71:0/3279198237 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f469c06cce0 0x7f469c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:14.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.951+0000 7f46b17fa700 1 -- 172.21.15.71:0/3279198237 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f46a808d740 con 0x7f46b40fff50 2024-04-23T02:43:14.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.951+0000 7f46b3fff700 1 --2- 172.21.15.71:0/3279198237 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f469c06cce0 0x7f469c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:14.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.951+0000 7f46b3fff700 1 --2- 172.21.15.71:0/3279198237 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f469c06cce0 0x7f469c06f1a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f46a4005950 tx=0x7f46a40058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:14.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:14.955+0000 7f46b17fa700 1 -- 172.21.15.71:0/3279198237 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f46a805b240 con 0x7f46b40fff50 2024-04-23T02:43:15.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.165+0000 7f46ba7e9700 1 -- 172.21.15.71:0/3279198237 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f46b4197ce0 con 0x7f469c06cce0 2024-04-23T02:43:15.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.166+0000 7f46b17fa700 1 -- 172.21.15.71:0/3279198237 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f46b4197ce0 con 0x7f469c06cce0 2024-04-23T02:43:15.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.168+0000 7f46ba7e9700 1 -- 172.21.15.71:0/3279198237 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f469c06cce0 msgr2=0x7f469c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:15.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.168+0000 7f46ba7e9700 1 --2- 172.21.15.71:0/3279198237 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f469c06cce0 0x7f469c06f1a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f46a4005950 tx=0x7f46a40058e0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.168+0000 7f46ba7e9700 1 -- 172.21.15.71:0/3279198237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40fff50 msgr2=0x7f46b419cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:15.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.168+0000 7f46ba7e9700 1 --2- 172.21.15.71:0/3279198237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40fff50 0x7f46b419cbd0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f46a80092e0 tx=0x7f46a800be20 comp rx=0 tx=0).stop 2024-04-23T02:43:15.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.168+0000 7f46ba7e9700 1 -- 172.21.15.71:0/3279198237 shutdown_connections 2024-04-23T02:43:15.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.168+0000 7f46ba7e9700 1 --2- 172.21.15.71:0/3279198237 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f469c06cce0 0x7f469c06f1a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.168+0000 7f46ba7e9700 1 --2- 172.21.15.71:0/3279198237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b40ff5a0 0x7f46b419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.168+0000 7f46ba7e9700 1 --2- 172.21.15.71:0/3279198237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b40fff50 0x7f46b419cbd0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.168+0000 7f46ba7e9700 1 -- 172.21.15.71:0/3279198237 >> 172.21.15.71:0/3279198237 conn(0x7f46b4074e30 msgr2=0x7f46b40fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:15.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.169+0000 7f46ba7e9700 1 -- 172.21.15.71:0/3279198237 shutdown_connections 2024-04-23T02:43:15.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.169+0000 7f46ba7e9700 1 -- 172.21.15.71:0/3279198237 wait complete. 2024-04-23T02:43:15.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.271+0000 7f440e1ac700 1 -- 172.21.15.71:0/3363243195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4408101840 msgr2=0x7f4408105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:15.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.271+0000 7f440e1ac700 1 --2- 172.21.15.71:0/3363243195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4408101840 0x7f4408105890 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009910 tx=0x7f43f8009c20 comp rx=0 tx=0).stop 2024-04-23T02:43:15.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.271+0000 7f440e1ac700 1 -- 172.21.15.71:0/3363243195 shutdown_connections 2024-04-23T02:43:15.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.271+0000 7f440e1ac700 1 --2- 172.21.15.71:0/3363243195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4408101840 0x7f4408105890 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.271+0000 7f440e1ac700 1 --2- 172.21.15.71:0/3363243195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4408100e90 0x7f4408101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.271+0000 7f440e1ac700 1 -- 172.21.15.71:0/3363243195 >> 172.21.15.71:0/3363243195 conn(0x7f44080fc740 msgr2=0x7f44080feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:15.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.271+0000 7f440e1ac700 1 -- 172.21.15.71:0/3363243195 shutdown_connections 2024-04-23T02:43:15.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.271+0000 7f440e1ac700 1 -- 172.21.15.71:0/3363243195 wait complete. 2024-04-23T02:43:15.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.272+0000 7f440e1ac700 1 Processor -- start 2024-04-23T02:43:15.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.272+0000 7f440e1ac700 1 -- start start 2024-04-23T02:43:15.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.272+0000 7f440e1ac700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4408100e90 0x7f440819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:15.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.272+0000 7f440e1ac700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4408101840 0x7f440819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:15.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.272+0000 7f440e1ac700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440819d200 con 0x7f4408100e90 2024-04-23T02:43:15.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.272+0000 7f440e1ac700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44081966b0 con 0x7f4408101840 2024-04-23T02:43:15.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.272+0000 7f44077fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4408100e90 0x7f440819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:15.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.272+0000 7f44077fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4408100e90 0x7f440819c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51480/0 (socket says 172.21.15.71:51480) 2024-04-23T02:43:15.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.272+0000 7f44077fe700 1 -- 172.21.15.71:0/3530781229 learned_addr learned my addr 172.21.15.71:0/3530781229 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:43:15.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.272+0000 7f4406ffd700 1 --2- 172.21.15.71:0/3530781229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4408101840 0x7f440819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:15.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.273+0000 7f44077fe700 1 -- 172.21.15.71:0/3530781229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4408101840 msgr2=0x7f440819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:15.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.273+0000 7f44077fe700 1 --2- 172.21.15.71:0/3530781229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4408101840 0x7f440819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.273+0000 7f44077fe700 1 -- 172.21.15.71:0/3530781229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f80095f0 con 0x7f4408100e90 2024-04-23T02:43:15.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.273+0000 7f44077fe700 1 --2- 172.21.15.71:0/3530781229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4408100e90 0x7f440819c630 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f43fc00d900 tx=0x7f43fc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:15.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.273+0000 7f4404ff9700 1 -- 172.21.15.71:0/3530781229 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43fc0041f0 con 0x7f4408100e90 2024-04-23T02:43:15.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.273+0000 7f440e1ac700 1 -- 172.21.15.71:0/3530781229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4408196990 con 0x7f4408100e90 2024-04-23T02:43:15.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.273+0000 7f4404ff9700 1 -- 172.21.15.71:0/3530781229 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f43fc004cd0 con 0x7f4408100e90 2024-04-23T02:43:15.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.273+0000 7f4404ff9700 1 -- 172.21.15.71:0/3530781229 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43fc005660 con 0x7f4408100e90 2024-04-23T02:43:15.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.273+0000 7f440e1ac700 1 -- 172.21.15.71:0/3530781229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4408196ee0 con 0x7f4408100e90 2024-04-23T02:43:15.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.274+0000 7f440e1ac700 1 -- 172.21.15.71:0/3530781229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4408109290 con 0x7f4408100e90 2024-04-23T02:43:15.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.274+0000 7f4404ff9700 1 -- 172.21.15.71:0/3530781229 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f43fc004830 con 0x7f4408100e90 2024-04-23T02:43:15.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.275+0000 7f4404ff9700 1 --2- 172.21.15.71:0/3530781229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f43f006cce0 0x7f43f006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:15.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.275+0000 7f4404ff9700 1 -- 172.21.15.71:0/3530781229 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f43fc08c390 con 0x7f4408100e90 2024-04-23T02:43:15.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.275+0000 7f4406ffd700 1 --2- 172.21.15.71:0/3530781229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f43f006cce0 0x7f43f006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:15.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.275+0000 7f4406ffd700 1 --2- 172.21.15.71:0/3530781229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f43f006cce0 0x7f43f006f1a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f44080fcfb0 tx=0x7f43f801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:15.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.279+0000 7f4404ff9700 1 -- 172.21.15.71:0/3530781229 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f43fc059e90 con 0x7f4408100e90 2024-04-23T02:43:15.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.479+0000 7f440e1ac700 1 -- 172.21.15.71:0/3530781229 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4408197100 con 0x7f43f006cce0 2024-04-23T02:43:15.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.489+0000 7f4404ff9700 1 -- 172.21.15.71:0/3530781229 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4408197100 con 0x7f43f006cce0 2024-04-23T02:43:15.492 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:43:15.492 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (26m) 5m ago 38m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:43:15.493 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (38m) 5m ago 38m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:43:15.493 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (37m) 5m ago 37m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:43:15.493 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (24m) 5m ago 38m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:43:15.493 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (24m) 5m ago 37m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:43:15.493 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (26m) 5m ago 38m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:43:15.493 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (21m) 5m ago 32m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:43:15.493 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (20m) 5m ago 32m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:43:15.493 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (5m) 5m ago 32m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:43:15.493 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (20m) 5m ago 32m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:43:15.493 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (28m) 5m ago 39m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:43:15.493 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (28m) 5m ago 37m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:43:15.493 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (25m) 5m ago 40m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:43:15.493 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (25m) 5m ago 37m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:43:15.493 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (27m) 5m ago 38m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:43:15.493 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (27m) 5m ago 37m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:43:15.494 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (24m) 5m ago 36m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:43:15.494 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (23m) 5m ago 35m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:43:15.494 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (23m) 5m ago 35m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:43:15.494 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (22m) 5m ago 34m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:43:15.494 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (22m) 5m ago 34m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:43:15.494 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (21m) 5m ago 33m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:43:15.494 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (27m) 5m ago 37m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:43:15.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.492+0000 7f440e1ac700 1 -- 172.21.15.71:0/3530781229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f43f006cce0 msgr2=0x7f43f006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:15.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.492+0000 7f440e1ac700 1 --2- 172.21.15.71:0/3530781229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f43f006cce0 0x7f43f006f1a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f44080fcfb0 tx=0x7f43f801a040 comp rx=0 tx=0).stop 2024-04-23T02:43:15.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.492+0000 7f440e1ac700 1 -- 172.21.15.71:0/3530781229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4408100e90 msgr2=0x7f440819c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:15.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.492+0000 7f440e1ac700 1 --2- 172.21.15.71:0/3530781229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4408100e90 0x7f440819c630 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f43fc00d900 tx=0x7f43fc00dc10 comp rx=0 tx=0).stop 2024-04-23T02:43:15.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.492+0000 7f440e1ac700 1 -- 172.21.15.71:0/3530781229 shutdown_connections 2024-04-23T02:43:15.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.492+0000 7f440e1ac700 1 --2- 172.21.15.71:0/3530781229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f43f006cce0 0x7f43f006f1a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.492+0000 7f440e1ac700 1 --2- 172.21.15.71:0/3530781229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4408100e90 0x7f440819c630 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.492+0000 7f440e1ac700 1 --2- 172.21.15.71:0/3530781229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4408101840 0x7f440819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.492+0000 7f440e1ac700 1 -- 172.21.15.71:0/3530781229 >> 172.21.15.71:0/3530781229 conn(0x7f44080fc740 msgr2=0x7f44080fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:15.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.492+0000 7f440e1ac700 1 -- 172.21.15.71:0/3530781229 shutdown_connections 2024-04-23T02:43:15.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.492+0000 7f440e1ac700 1 -- 172.21.15.71:0/3530781229 wait complete. 2024-04-23T02:43:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.593+0000 7fa7985ec700 1 -- 172.21.15.71:0/3721524340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7900ffc20 msgr2=0x7fa790100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.593+0000 7fa7985ec700 1 --2- 172.21.15.71:0/3721524340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7900ffc20 0x7fa790100080 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa780009910 tx=0x7fa780009c20 comp rx=0 tx=0).stop 2024-04-23T02:43:15.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.593+0000 7fa7985ec700 1 -- 172.21.15.71:0/3721524340 shutdown_connections 2024-04-23T02:43:15.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.593+0000 7fa7985ec700 1 --2- 172.21.15.71:0/3721524340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7900ffc20 0x7fa790100080 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.593+0000 7fa7985ec700 1 --2- 172.21.15.71:0/3721524340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa790105bd0 0x7fa790105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.593+0000 7fa7985ec700 1 -- 172.21.15.71:0/3721524340 >> 172.21.15.71:0/3721524340 conn(0x7fa790075670 msgr2=0x7fa790075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:15.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.593+0000 7fa7985ec700 1 -- 172.21.15.71:0/3721524340 shutdown_connections 2024-04-23T02:43:15.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.593+0000 7fa7985ec700 1 -- 172.21.15.71:0/3721524340 wait complete. 2024-04-23T02:43:15.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.594+0000 7fa7985ec700 1 Processor -- start 2024-04-23T02:43:15.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.594+0000 7fa7985ec700 1 -- start start 2024-04-23T02:43:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.594+0000 7fa7985ec700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7900ffc20 0x7fa790101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.594+0000 7fa7985ec700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa790105bd0 0x7fa790101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.594+0000 7fa7985ec700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7901022c0 con 0x7fa790105bd0 2024-04-23T02:43:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.594+0000 7fa7985ec700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa790102400 con 0x7fa7900ffc20 2024-04-23T02:43:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.594+0000 7fa795b87700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa790105bd0 0x7fa790101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.594+0000 7fa795b87700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa790105bd0 0x7fa790101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37992/0 (socket says 172.21.15.71:37992) 2024-04-23T02:43:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.594+0000 7fa795b87700 1 -- 172.21.15.71:0/3111116374 learned_addr learned my addr 172.21.15.71:0/3111116374 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:43:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.595+0000 7fa796388700 1 --2- 172.21.15.71:0/3111116374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7900ffc20 0x7fa790101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.595+0000 7fa795b87700 1 -- 172.21.15.71:0/3111116374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7900ffc20 msgr2=0x7fa790101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.595+0000 7fa795b87700 1 --2- 172.21.15.71:0/3111116374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7900ffc20 0x7fa790101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.595+0000 7fa795b87700 1 -- 172.21.15.71:0/3111116374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7800095f0 con 0x7fa790105bd0 2024-04-23T02:43:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.595+0000 7fa796388700 1 --2- 172.21.15.71:0/3111116374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7900ffc20 0x7fa790101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:43:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.595+0000 7fa795b87700 1 --2- 172.21.15.71:0/3111116374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa790105bd0 0x7fa790101d80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa78000b5c0 tx=0x7fa78000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.595+0000 7fa7877fe700 1 -- 172.21.15.71:0/3111116374 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa78001d070 con 0x7fa790105bd0 2024-04-23T02:43:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.595+0000 7fa7877fe700 1 -- 172.21.15.71:0/3111116374 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa78000f460 con 0x7fa790105bd0 2024-04-23T02:43:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.595+0000 7fa7877fe700 1 -- 172.21.15.71:0/3111116374 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa780021640 con 0x7fa790105bd0 2024-04-23T02:43:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.595+0000 7fa7985ec700 1 -- 172.21.15.71:0/3111116374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa790102680 con 0x7fa790105bd0 2024-04-23T02:43:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.595+0000 7fa7985ec700 1 -- 172.21.15.71:0/3111116374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7901a2c60 con 0x7fa790105bd0 2024-04-23T02:43:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.596+0000 7fa7985ec700 1 -- 172.21.15.71:0/3111116374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa79004f2e0 con 0x7fa790105bd0 2024-04-23T02:43:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.597+0000 7fa7877fe700 1 -- 172.21.15.71:0/3111116374 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa78000f9b0 con 0x7fa790105bd0 2024-04-23T02:43:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.597+0000 7fa7877fe700 1 --2- 172.21.15.71:0/3111116374 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa77c06cce0 0x7fa77c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.597+0000 7fa7877fe700 1 -- 172.21.15.71:0/3111116374 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa78008d7e0 con 0x7fa790105bd0 2024-04-23T02:43:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.597+0000 7fa796388700 1 --2- 172.21.15.71:0/3111116374 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa77c06cce0 0x7fa77c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.597+0000 7fa796388700 1 --2- 172.21.15.71:0/3111116374 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa77c06cce0 0x7fa77c06f1a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa78c005fd0 tx=0x7fa78c005e20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:15.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.601+0000 7fa7877fe700 1 -- 172.21.15.71:0/3111116374 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa780057850 con 0x7fa790105bd0 2024-04-23T02:43:15.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:15 smithi192 ceph-mon[108617]: pgmap v899: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:15.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.877+0000 7fa7985ec700 1 -- 172.21.15.71:0/3111116374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa7901a2ea0 con 0x7fa790105bd0 2024-04-23T02:43:15.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.878+0000 7fa7877fe700 1 -- 172.21.15.71:0/3111116374 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fa78005ae70 con 0x7fa790105bd0 2024-04-23T02:43:15.880 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:43:15.880 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:43:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:43:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:43:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:43:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:43:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:43:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:43:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:43:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:43:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:43:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:43:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:43:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:43:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:43:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:43:15.882 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:43:15.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.880+0000 7fa7985ec700 1 -- 172.21.15.71:0/3111116374 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa77c06cce0 msgr2=0x7fa77c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:15.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.880+0000 7fa7985ec700 1 --2- 172.21.15.71:0/3111116374 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa77c06cce0 0x7fa77c06f1a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa78c005fd0 tx=0x7fa78c005e20 comp rx=0 tx=0).stop 2024-04-23T02:43:15.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.880+0000 7fa7985ec700 1 -- 172.21.15.71:0/3111116374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa790105bd0 msgr2=0x7fa790101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:15.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.880+0000 7fa7985ec700 1 --2- 172.21.15.71:0/3111116374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa790105bd0 0x7fa790101d80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa78000b5c0 tx=0x7fa78000be20 comp rx=0 tx=0).stop 2024-04-23T02:43:15.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.880+0000 7fa7985ec700 1 -- 172.21.15.71:0/3111116374 shutdown_connections 2024-04-23T02:43:15.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.880+0000 7fa7985ec700 1 --2- 172.21.15.71:0/3111116374 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa77c06cce0 0x7fa77c06f1a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.880+0000 7fa7985ec700 1 --2- 172.21.15.71:0/3111116374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7900ffc20 0x7fa790101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.880+0000 7fa7985ec700 1 --2- 172.21.15.71:0/3111116374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa790105bd0 0x7fa790101d80 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.880+0000 7fa7985ec700 1 -- 172.21.15.71:0/3111116374 >> 172.21.15.71:0/3111116374 conn(0x7fa790075670 msgr2=0x7fa790068720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:15.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.880+0000 7fa7985ec700 1 -- 172.21.15.71:0/3111116374 shutdown_connections 2024-04-23T02:43:15.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.880+0000 7fa7985ec700 1 -- 172.21.15.71:0/3111116374 wait complete. 2024-04-23T02:43:15.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.983+0000 7f647402d700 1 -- 172.21.15.71:0/3228748889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f646c100ea0 msgr2=0x7f646c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:15.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.983+0000 7f647402d700 1 --2- 172.21.15.71:0/3228748889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f646c100ea0 0x7f646c101280 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f645c009910 tx=0x7f645c009c20 comp rx=0 tx=0).stop 2024-04-23T02:43:15.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.983+0000 7f647402d700 1 -- 172.21.15.71:0/3228748889 shutdown_connections 2024-04-23T02:43:15.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.983+0000 7f647402d700 1 --2- 172.21.15.71:0/3228748889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f646c101850 0x7f646c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.983+0000 7f647402d700 1 --2- 172.21.15.71:0/3228748889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f646c100ea0 0x7f646c101280 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.983+0000 7f647402d700 1 -- 172.21.15.71:0/3228748889 >> 172.21.15.71:0/3228748889 conn(0x7f646c0fc750 msgr2=0x7f646c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:15.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.983+0000 7f647402d700 1 -- 172.21.15.71:0/3228748889 shutdown_connections 2024-04-23T02:43:15.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.983+0000 7f647402d700 1 -- 172.21.15.71:0/3228748889 wait complete. 2024-04-23T02:43:15.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.984+0000 7f647402d700 1 Processor -- start 2024-04-23T02:43:15.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.984+0000 7f647402d700 1 -- start start 2024-04-23T02:43:15.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.984+0000 7f647402d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f646c100ea0 0x7f646c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:15.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.984+0000 7f647402d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f646c101850 0x7f646c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:15.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.984+0000 7f647402d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c19d130 con 0x7f646c101850 2024-04-23T02:43:15.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.984+0000 7f647402d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c1965e0 con 0x7f646c100ea0 2024-04-23T02:43:15.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.984+0000 7f64715c8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f646c101850 0x7f646c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:15.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.984+0000 7f64715c8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f646c101850 0x7f646c19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38006/0 (socket says 172.21.15.71:38006) 2024-04-23T02:43:15.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.984+0000 7f64715c8700 1 -- 172.21.15.71:0/2632000508 learned_addr learned my addr 172.21.15.71:0/2632000508 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:43:15.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.984+0000 7f6471dc9700 1 --2- 172.21.15.71:0/2632000508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f646c100ea0 0x7f646c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:15.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.984+0000 7f64715c8700 1 -- 172.21.15.71:0/2632000508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f646c100ea0 msgr2=0x7f646c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:15.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.984+0000 7f64715c8700 1 --2- 172.21.15.71:0/2632000508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f646c100ea0 0x7f646c19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:15.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.984+0000 7f64715c8700 1 -- 172.21.15.71:0/2632000508 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c0095f0 con 0x7f646c101850 2024-04-23T02:43:15.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.985+0000 7f64715c8700 1 --2- 172.21.15.71:0/2632000508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f646c101850 0x7f646c19caa0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f646800d900 tx=0x7f646800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:15.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.985+0000 7f6462ffd700 1 -- 172.21.15.71:0/2632000508 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6468009900 con 0x7f646c101850 2024-04-23T02:43:15.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.985+0000 7f6462ffd700 1 -- 172.21.15.71:0/2632000508 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6468010460 con 0x7f646c101850 2024-04-23T02:43:15.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.985+0000 7f6462ffd700 1 -- 172.21.15.71:0/2632000508 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f646800f5f0 con 0x7f646c101850 2024-04-23T02:43:15.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.985+0000 7f647402d700 1 -- 172.21.15.71:0/2632000508 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646c1968c0 con 0x7f646c101850 2024-04-23T02:43:15.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.985+0000 7f647402d700 1 -- 172.21.15.71:0/2632000508 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646c196e10 con 0x7f646c101850 2024-04-23T02:43:15.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.986+0000 7f647402d700 1 -- 172.21.15.71:0/2632000508 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646c109240 con 0x7f646c101850 2024-04-23T02:43:15.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.986+0000 7f6462ffd700 1 -- 172.21.15.71:0/2632000508 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f64680105d0 con 0x7f646c101850 2024-04-23T02:43:15.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.987+0000 7f6462ffd700 1 --2- 172.21.15.71:0/2632000508 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f645806cce0 0x7f645806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:15.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.987+0000 7f6462ffd700 1 -- 172.21.15.71:0/2632000508 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f646808c010 con 0x7f646c101850 2024-04-23T02:43:15.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.987+0000 7f6471dc9700 1 --2- 172.21.15.71:0/2632000508 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f645806cce0 0x7f645806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:15.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.987+0000 7f6471dc9700 1 --2- 172.21.15.71:0/2632000508 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f645806cce0 0x7f645806f1a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f645c006010 tx=0x7f645c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:15.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:15 smithi071 ceph-mon[144544]: pgmap v899: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:15.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:15.991+0000 7f6462ffd700 1 -- 172.21.15.71:0/2632000508 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6468059b90 con 0x7f646c101850 2024-04-23T02:43:16.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.225+0000 7f647402d700 1 -- 172.21.15.71:0/2632000508 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f646c04f2e0 con 0x7f646c101850 2024-04-23T02:43:16.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.226+0000 7f6462ffd700 1 -- 172.21.15.71:0/2632000508 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f6468020740 con 0x7f646c101850 2024-04-23T02:43:16.229 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:43:16.229 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:43:16.229 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:43:16.229 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:43:16.229 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:43:16.229 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:43:16.229 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:43:16.229 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:43:16.229 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:43:16.229 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:43:16.229 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:43:16.230 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:43:16.230 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:43:16.230 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:43:16.230 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:43:16.230 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:43:16.230 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:43:16.230 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:43:16.230 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:43:16.230 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:43:16.230 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:43:16.230 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:43:16.230 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:43:16.230 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:43:16.231 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:43:16.231 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:43:16.231 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:43:16.231 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:43:16.231 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:43:16.231 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:43:16.231 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:43:16.231 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:43:16.231 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:43:16.231 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:43:16.231 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:43:16.231 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:43:16.231 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:43:16.231 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:43:16.232 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:43:16.232 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:43:16.232 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:43:16.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.228+0000 7f647402d700 1 -- 172.21.15.71:0/2632000508 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f645806cce0 msgr2=0x7f645806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:16.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.228+0000 7f647402d700 1 --2- 172.21.15.71:0/2632000508 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f645806cce0 0x7f645806f1a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f645c006010 tx=0x7f645c01a040 comp rx=0 tx=0).stop 2024-04-23T02:43:16.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.228+0000 7f647402d700 1 -- 172.21.15.71:0/2632000508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f646c101850 msgr2=0x7f646c19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:16.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.228+0000 7f647402d700 1 --2- 172.21.15.71:0/2632000508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f646c101850 0x7f646c19caa0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f646800d900 tx=0x7f646800dc10 comp rx=0 tx=0).stop 2024-04-23T02:43:16.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.228+0000 7f647402d700 1 -- 172.21.15.71:0/2632000508 shutdown_connections 2024-04-23T02:43:16.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.228+0000 7f647402d700 1 --2- 172.21.15.71:0/2632000508 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f645806cce0 0x7f645806f1a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:16.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.228+0000 7f647402d700 1 --2- 172.21.15.71:0/2632000508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f646c100ea0 0x7f646c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:16.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.228+0000 7f647402d700 1 --2- 172.21.15.71:0/2632000508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f646c101850 0x7f646c19caa0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:16.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.228+0000 7f647402d700 1 -- 172.21.15.71:0/2632000508 >> 172.21.15.71:0/2632000508 conn(0x7f646c0fc750 msgr2=0x7f646c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:16.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.228+0000 7f647402d700 1 -- 172.21.15.71:0/2632000508 shutdown_connections 2024-04-23T02:43:16.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.228+0000 7f647402d700 1 -- 172.21.15.71:0/2632000508 wait complete. 2024-04-23T02:43:16.233 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:43:16.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.331+0000 7f864b832700 1 -- 172.21.15.71:0/111907645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8644102780 msgr2=0x7f864410ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:16.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.331+0000 7f864b832700 1 --2- 172.21.15.71:0/111907645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8644102780 0x7f864410ac70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8640009940 tx=0x7f8640009c50 comp rx=0 tx=0).stop 2024-04-23T02:43:16.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.331+0000 7f864b832700 1 -- 172.21.15.71:0/111907645 shutdown_connections 2024-04-23T02:43:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.331+0000 7f864b832700 1 --2- 172.21.15.71:0/111907645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8644102780 0x7f864410ac70 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.331+0000 7f864b832700 1 --2- 172.21.15.71:0/111907645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8644101e60 0x7f8644102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.331+0000 7f864b832700 1 -- 172.21.15.71:0/111907645 >> 172.21.15.71:0/111907645 conn(0x7f86440fb650 msgr2=0x7f86440fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.331+0000 7f864b832700 1 -- 172.21.15.71:0/111907645 shutdown_connections 2024-04-23T02:43:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.331+0000 7f864b832700 1 -- 172.21.15.71:0/111907645 wait complete. 2024-04-23T02:43:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.332+0000 7f864b832700 1 Processor -- start 2024-04-23T02:43:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.332+0000 7f864b832700 1 -- start start 2024-04-23T02:43:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.332+0000 7f864b832700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8644101e60 0x7f864419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.332+0000 7f864b832700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8644102780 0x7f864419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.332+0000 7f864b832700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864419d210 con 0x7f8644101e60 2024-04-23T02:43:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.332+0000 7f864b832700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86441966c0 con 0x7f8644102780 2024-04-23T02:43:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.332+0000 7f86495ce700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8644101e60 0x7f864419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.332+0000 7f86495ce700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8644101e60 0x7f864419c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38026/0 (socket says 172.21.15.71:38026) 2024-04-23T02:43:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.332+0000 7f86495ce700 1 -- 172.21.15.71:0/3484679780 learned_addr learned my addr 172.21.15.71:0/3484679780 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:43:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.332+0000 7f8648dcd700 1 --2- 172.21.15.71:0/3484679780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8644102780 0x7f864419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.332+0000 7f86495ce700 1 -- 172.21.15.71:0/3484679780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8644102780 msgr2=0x7f864419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.332+0000 7f86495ce700 1 --2- 172.21.15.71:0/3484679780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8644102780 0x7f864419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.332+0000 7f86495ce700 1 -- 172.21.15.71:0/3484679780 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86400095f0 con 0x7f8644101e60 2024-04-23T02:43:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.332+0000 7f8648dcd700 1 --2- 172.21.15.71:0/3484679780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8644102780 0x7f864419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:43:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.332+0000 7f86495ce700 1 --2- 172.21.15.71:0/3484679780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8644101e60 0x7f864419c640 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f863400d8d0 tx=0x7f863400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.332+0000 7f863a7fc700 1 -- 172.21.15.71:0/3484679780 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8634009940 con 0x7f8644101e60 2024-04-23T02:43:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.333+0000 7f864b832700 1 -- 172.21.15.71:0/3484679780 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86441969a0 con 0x7f8644101e60 2024-04-23T02:43:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.333+0000 7f863a7fc700 1 -- 172.21.15.71:0/3484679780 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8634010460 con 0x7f8644101e60 2024-04-23T02:43:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.333+0000 7f863a7fc700 1 -- 172.21.15.71:0/3484679780 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f863400f5f0 con 0x7f8644101e60 2024-04-23T02:43:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.333+0000 7f864b832700 1 -- 172.21.15.71:0/3484679780 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8644196ec0 con 0x7f8644101e60 2024-04-23T02:43:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.333+0000 7f864b832700 1 -- 172.21.15.71:0/3484679780 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8644108370 con 0x7f8644101e60 2024-04-23T02:43:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.334+0000 7f863a7fc700 1 -- 172.21.15.71:0/3484679780 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f86340105d0 con 0x7f8644101e60 2024-04-23T02:43:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.334+0000 7f863a7fc700 1 --2- 172.21.15.71:0/3484679780 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f863006cce0 0x7f863006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.334+0000 7f863a7fc700 1 -- 172.21.15.71:0/3484679780 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f863408c5e0 con 0x7f8644101e60 2024-04-23T02:43:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.334+0000 7f8648dcd700 1 --2- 172.21.15.71:0/3484679780 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f863006cce0 0x7f863006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.335+0000 7f8648dcd700 1 --2- 172.21.15.71:0/3484679780 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f863006cce0 0x7f863006f1a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8640009910 tx=0x7f8640005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:16.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.339+0000 7f863a7fc700 1 -- 172.21.15.71:0/3484679780 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f863405a0e0 con 0x7f8644101e60 2024-04-23T02:43:16.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.547+0000 7f864b832700 1 -- 172.21.15.71:0/3484679780 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8644197ce0 con 0x7f863006cce0 2024-04-23T02:43:16.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.548+0000 7f863a7fc700 1 -- 172.21.15.71:0/3484679780 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8644197ce0 con 0x7f863006cce0 2024-04-23T02:43:16.553 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:43:16.553 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:43:16.553 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:43:16.554 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:43:16.554 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:43:16.554 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:43:16.554 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:43:16.554 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:43:16.554 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:43:16.554 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:43:16.554 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:43:16.554 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:43:16.554 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:43:16.554 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:43:16.554 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:43:16.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.550+0000 7f864b832700 1 -- 172.21.15.71:0/3484679780 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f863006cce0 msgr2=0x7f863006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:16.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.550+0000 7f864b832700 1 --2- 172.21.15.71:0/3484679780 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f863006cce0 0x7f863006f1a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8640009910 tx=0x7f8640005960 comp rx=0 tx=0).stop 2024-04-23T02:43:16.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.550+0000 7f864b832700 1 -- 172.21.15.71:0/3484679780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8644101e60 msgr2=0x7f864419c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:16.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.550+0000 7f864b832700 1 --2- 172.21.15.71:0/3484679780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8644101e60 0x7f864419c640 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f863400d8d0 tx=0x7f863400dc90 comp rx=0 tx=0).stop 2024-04-23T02:43:16.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.550+0000 7f864b832700 1 -- 172.21.15.71:0/3484679780 shutdown_connections 2024-04-23T02:43:16.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.550+0000 7f864b832700 1 --2- 172.21.15.71:0/3484679780 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f863006cce0 0x7f863006f1a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:16.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.550+0000 7f864b832700 1 --2- 172.21.15.71:0/3484679780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8644101e60 0x7f864419c640 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:16.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.550+0000 7f864b832700 1 --2- 172.21.15.71:0/3484679780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8644102780 0x7f864419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:16.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.550+0000 7f864b832700 1 -- 172.21.15.71:0/3484679780 >> 172.21.15.71:0/3484679780 conn(0x7f86440fb650 msgr2=0x7f86441054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:16.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.550+0000 7f864b832700 1 -- 172.21.15.71:0/3484679780 shutdown_connections 2024-04-23T02:43:16.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.550+0000 7f864b832700 1 -- 172.21.15.71:0/3484679780 wait complete. 2024-04-23T02:43:16.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.656+0000 7f34589a3700 1 -- 172.21.15.71:0/3021476888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3454068580 msgr2=0x7f34540689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:16.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.656+0000 7f34589a3700 1 --2- 172.21.15.71:0/3021476888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3454068580 0x7f34540689e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3444009910 tx=0x7f3444009c20 comp rx=0 tx=0).stop 2024-04-23T02:43:16.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.656+0000 7f34589a3700 1 -- 172.21.15.71:0/3021476888 shutdown_connections 2024-04-23T02:43:16.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.656+0000 7f34589a3700 1 --2- 172.21.15.71:0/3021476888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3454068580 0x7f34540689e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:16.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.656+0000 7f34589a3700 1 --2- 172.21.15.71:0/3021476888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3454106cd0 0x7f34541070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:16.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.656+0000 7f34589a3700 1 -- 172.21.15.71:0/3021476888 >> 172.21.15.71:0/3021476888 conn(0x7f3454075670 msgr2=0x7f3454075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:16.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.656+0000 7f34589a3700 1 -- 172.21.15.71:0/3021476888 shutdown_connections 2024-04-23T02:43:16.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.656+0000 7f34589a3700 1 -- 172.21.15.71:0/3021476888 wait complete. 2024-04-23T02:43:16.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.657+0000 7f34589a3700 1 Processor -- start 2024-04-23T02:43:16.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.657+0000 7f34589a3700 1 -- start start 2024-04-23T02:43:16.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.657+0000 7f34589a3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3454068580 0x7f345419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:16.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.657+0000 7f34589a3700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3454106cd0 0x7f345419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:16.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.657+0000 7f34589a3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345419d110 con 0x7f3454068580 2024-04-23T02:43:16.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.657+0000 7f34589a3700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34541965c0 con 0x7f3454106cd0 2024-04-23T02:43:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.657+0000 7f345259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3454068580 0x7f345419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.657+0000 7f345259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3454068580 0x7f345419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38044/0 (socket says 172.21.15.71:38044) 2024-04-23T02:43:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.657+0000 7f345259c700 1 -- 172.21.15.71:0/4136432585 learned_addr learned my addr 172.21.15.71:0/4136432585 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:43:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.657+0000 7f345259c700 1 -- 172.21.15.71:0/4136432585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3454106cd0 msgr2=0x7f345419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.657+0000 7f345259c700 1 --2- 172.21.15.71:0/4136432585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3454106cd0 0x7f345419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.657+0000 7f345259c700 1 -- 172.21.15.71:0/4136432585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34440095f0 con 0x7f3454068580 2024-04-23T02:43:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.657+0000 7f345259c700 1 --2- 172.21.15.71:0/4136432585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3454068580 0x7f345419c540 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f344800d8d0 tx=0x7f344800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.657+0000 7f34437fe700 1 -- 172.21.15.71:0/4136432585 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3448009940 con 0x7f3454068580 2024-04-23T02:43:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.657+0000 7f34589a3700 1 -- 172.21.15.71:0/4136432585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34541968a0 con 0x7f3454068580 2024-04-23T02:43:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.657+0000 7f34437fe700 1 -- 172.21.15.71:0/4136432585 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3448010460 con 0x7f3454068580 2024-04-23T02:43:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.657+0000 7f34437fe700 1 -- 172.21.15.71:0/4136432585 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f344800f5f0 con 0x7f3454068580 2024-04-23T02:43:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.658+0000 7f34589a3700 1 -- 172.21.15.71:0/4136432585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3454196df0 con 0x7f3454068580 2024-04-23T02:43:16.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.658+0000 7f34589a3700 1 -- 172.21.15.71:0/4136432585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34541091d0 con 0x7f3454068580 2024-04-23T02:43:16.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.659+0000 7f34437fe700 1 -- 172.21.15.71:0/4136432585 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f34480105d0 con 0x7f3454068580 2024-04-23T02:43:16.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.659+0000 7f34437fe700 1 --2- 172.21.15.71:0/4136432585 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f343c06cce0 0x7f343c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:16.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.659+0000 7f34437fe700 1 -- 172.21.15.71:0/4136432585 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f344808c680 con 0x7f3454068580 2024-04-23T02:43:16.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.659+0000 7f3451d9b700 1 --2- 172.21.15.71:0/4136432585 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f343c06cce0 0x7f343c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:16.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.660+0000 7f3451d9b700 1 --2- 172.21.15.71:0/4136432585 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f343c06cce0 0x7f343c06f1a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f344400b5c0 tx=0x7f3444005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:16.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.664+0000 7f34437fe700 1 -- 172.21.15.71:0/4136432585 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f344805a0e0 con 0x7f3454068580 2024-04-23T02:43:16.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:16 smithi192 ceph-mon[108617]: from='client.35436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:43:16.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:16 smithi192 ceph-mon[108617]: from='client.35440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:43:16.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:16 smithi192 ceph-mon[108617]: from='client.35444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:43:16.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:16 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3111116374' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:43:16.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:16 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2632000508' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:43:16.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.937+0000 7f34589a3700 1 -- 172.21.15.71:0/4136432585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3454197010 con 0x7f3454068580 2024-04-23T02:43:16.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.937+0000 7f34437fe700 1 -- 172.21.15.71:0/4136432585 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7f3448010880 con 0x7f3454068580 2024-04-23T02:43:16.940 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:43:16.940 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:43:16.940 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:43:16.940 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:43:16.940 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:43:16.940 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:43:16.940 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:43:16.940 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:43:16.940 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:43:16.940 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:43:16.941 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:43:16.941 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:43:16.941 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:43:16.941 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:43:16.941 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:43:16.941 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:43:16.941 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:43:16.941 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:43:16.941 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:43:16.941 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:43:16.941 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:43:16.941 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:43:16.941 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:43:16.942 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:43:16.942 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:43:16.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.939+0000 7f34589a3700 1 -- 172.21.15.71:0/4136432585 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f343c06cce0 msgr2=0x7f343c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:16.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.939+0000 7f34589a3700 1 --2- 172.21.15.71:0/4136432585 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f343c06cce0 0x7f343c06f1a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f344400b5c0 tx=0x7f3444005960 comp rx=0 tx=0).stop 2024-04-23T02:43:16.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.939+0000 7f34589a3700 1 -- 172.21.15.71:0/4136432585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3454068580 msgr2=0x7f345419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:16.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.939+0000 7f34589a3700 1 --2- 172.21.15.71:0/4136432585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3454068580 0x7f345419c540 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f344800d8d0 tx=0x7f344800dc90 comp rx=0 tx=0).stop 2024-04-23T02:43:16.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.939+0000 7f34589a3700 1 -- 172.21.15.71:0/4136432585 shutdown_connections 2024-04-23T02:43:16.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.939+0000 7f34589a3700 1 --2- 172.21.15.71:0/4136432585 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f343c06cce0 0x7f343c06f1a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:16.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.939+0000 7f34589a3700 1 --2- 172.21.15.71:0/4136432585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3454068580 0x7f345419c540 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:16.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.939+0000 7f34589a3700 1 --2- 172.21.15.71:0/4136432585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3454106cd0 0x7f345419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:16.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.939+0000 7f34589a3700 1 -- 172.21.15.71:0/4136432585 >> 172.21.15.71:0/4136432585 conn(0x7f3454075670 msgr2=0x7f34540fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:16.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.939+0000 7f34589a3700 1 -- 172.21.15.71:0/4136432585 shutdown_connections 2024-04-23T02:43:16.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:16.939+0000 7f34589a3700 1 -- 172.21.15.71:0/4136432585 wait complete. 2024-04-23T02:43:16.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:16 smithi071 ceph-mon[144544]: from='client.35436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:43:16.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:16 smithi071 ceph-mon[144544]: from='client.35440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:43:16.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:16 smithi071 ceph-mon[144544]: from='client.35444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:43:16.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:16 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3111116374' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:43:16.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:16 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2632000508' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:43:17.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:17 smithi192 ceph-mon[108617]: pgmap v900: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:17.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:17 smithi192 ceph-mon[108617]: from='client.35456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:43:17.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:17 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/4136432585' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:43:17.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:17 smithi071 ceph-mon[144544]: pgmap v900: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:17.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:17 smithi071 ceph-mon[144544]: from='client.35456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:43:17.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:17 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/4136432585' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:43:18.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:18 smithi192 ceph-mon[108617]: pgmap v901: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:18.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:18 smithi071 ceph-mon[144544]: pgmap v901: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:20.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:43:20.246 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:43:21.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:20 smithi192 ceph-mon[108617]: pgmap v902: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:21.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:20 smithi071 ceph-mon[144544]: pgmap v902: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:23.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:23 smithi192 ceph-mon[108617]: pgmap v903: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:23.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:23 smithi071 ceph-mon[144544]: pgmap v903: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:25.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:25 smithi192 ceph-mon[108617]: pgmap v904: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:25.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:25 smithi071 ceph-mon[144544]: pgmap v904: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:27.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:27 smithi192 ceph-mon[108617]: pgmap v905: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:27.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:27 smithi071 ceph-mon[144544]: pgmap v905: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:29.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:29 smithi192 ceph-mon[108617]: pgmap v906: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:29.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:29 smithi071 ceph-mon[144544]: pgmap v906: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:31.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:31 smithi192 ceph-mon[108617]: pgmap v907: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:31.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:31 smithi071 ceph-mon[144544]: pgmap v907: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:33.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:33 smithi192 ceph-mon[108617]: pgmap v908: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:33.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:33 smithi071 ceph-mon[144544]: pgmap v908: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:35.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:35 smithi192 ceph-mon[108617]: pgmap v909: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:35.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:43:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:35 smithi071 ceph-mon[144544]: pgmap v909: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:43:37.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:37 smithi192 ceph-mon[108617]: pgmap v910: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:37.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:37 smithi071 ceph-mon[144544]: pgmap v910: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:39.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:39 smithi192 ceph-mon[108617]: pgmap v911: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:39.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:39 smithi071 ceph-mon[144544]: pgmap v911: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:41.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:41 smithi192 ceph-mon[108617]: pgmap v912: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:41.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:41 smithi071 ceph-mon[144544]: pgmap v912: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:43.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:43 smithi192 ceph-mon[108617]: pgmap v913: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:43.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:43 smithi071 ceph-mon[144544]: pgmap v913: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:45.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:45 smithi192 ceph-mon[108617]: pgmap v914: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:45.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:45 smithi071 ceph-mon[144544]: pgmap v914: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:47.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.055+0000 7f85b0d39700 1 -- 172.21.15.71:0/268364431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85ac101830 msgr2=0x7f85ac105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:47.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.055+0000 7f85b0d39700 1 --2- 172.21.15.71:0/268364431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85ac101830 0x7f85ac105880 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f859c009910 tx=0x7f859c009c20 comp rx=0 tx=0).stop 2024-04-23T02:43:47.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.055+0000 7f85b0d39700 1 -- 172.21.15.71:0/268364431 shutdown_connections 2024-04-23T02:43:47.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.055+0000 7f85b0d39700 1 --2- 172.21.15.71:0/268364431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85ac101830 0x7f85ac105880 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.055+0000 7f85b0d39700 1 --2- 172.21.15.71:0/268364431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85ac100e80 0x7f85ac101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.055+0000 7f85b0d39700 1 -- 172.21.15.71:0/268364431 >> 172.21.15.71:0/268364431 conn(0x7f85ac0fc730 msgr2=0x7f85ac0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:47.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.055+0000 7f85b0d39700 1 -- 172.21.15.71:0/268364431 shutdown_connections 2024-04-23T02:43:47.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.055+0000 7f85b0d39700 1 -- 172.21.15.71:0/268364431 wait complete. 2024-04-23T02:43:47.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.056+0000 7f85b0d39700 1 Processor -- start 2024-04-23T02:43:47.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.056+0000 7f85b0d39700 1 -- start start 2024-04-23T02:43:47.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.056+0000 7f85b0d39700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85ac100e80 0x7f85ac073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:47.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.056+0000 7f85b0d39700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85ac101830 0x7f85ac0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:47.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.056+0000 7f85b0d39700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac073b10 con 0x7f85ac100e80 2024-04-23T02:43:47.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.056+0000 7f85b0d39700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac073c50 con 0x7f85ac101830 2024-04-23T02:43:47.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.056+0000 7f85aa59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85ac100e80 0x7f85ac073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:47.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.056+0000 7f85aa59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85ac100e80 0x7f85ac073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51446/0 (socket says 172.21.15.71:51446) 2024-04-23T02:43:47.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.056+0000 7f85aa59c700 1 -- 172.21.15.71:0/3330733972 learned_addr learned my addr 172.21.15.71:0/3330733972 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:43:47.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.056+0000 7f85a9d9b700 1 --2- 172.21.15.71:0/3330733972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85ac101830 0x7f85ac0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:47.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.057+0000 7f85aa59c700 1 -- 172.21.15.71:0/3330733972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85ac101830 msgr2=0x7f85ac0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:47.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.057+0000 7f85aa59c700 1 --2- 172.21.15.71:0/3330733972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85ac101830 0x7f85ac0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.057+0000 7f85aa59c700 1 -- 172.21.15.71:0/3330733972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859c0095f0 con 0x7f85ac100e80 2024-04-23T02:43:47.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.057+0000 7f85aa59c700 1 --2- 172.21.15.71:0/3330733972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85ac100e80 0x7f85ac073090 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f85a000d8d0 tx=0x7f85a000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:47.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.057+0000 7f859b7fe700 1 -- 172.21.15.71:0/3330733972 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85a00098a0 con 0x7f85ac100e80 2024-04-23T02:43:47.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.057+0000 7f859b7fe700 1 -- 172.21.15.71:0/3330733972 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f85a0010460 con 0x7f85ac100e80 2024-04-23T02:43:47.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.057+0000 7f85b0d39700 1 -- 172.21.15.71:0/3330733972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ac1a2a00 con 0x7f85ac100e80 2024-04-23T02:43:47.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.057+0000 7f859b7fe700 1 -- 172.21.15.71:0/3330733972 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85a0009ea0 con 0x7f85ac100e80 2024-04-23T02:43:47.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.057+0000 7f85b0d39700 1 -- 172.21.15.71:0/3330733972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ac1a2e10 con 0x7f85ac100e80 2024-04-23T02:43:47.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.058+0000 7f85b0d39700 1 -- 172.21.15.71:0/3330733972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85ac109280 con 0x7f85ac100e80 2024-04-23T02:43:47.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.058+0000 7f859b7fe700 1 -- 172.21.15.71:0/3330733972 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f85a00105d0 con 0x7f85ac100e80 2024-04-23T02:43:47.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.059+0000 7f859b7fe700 1 --2- 172.21.15.71:0/3330733972 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f859406cce0 0x7f859406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:47.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.059+0000 7f859b7fe700 1 -- 172.21.15.71:0/3330733972 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f85a008c480 con 0x7f85ac100e80 2024-04-23T02:43:47.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.059+0000 7f85a9d9b700 1 --2- 172.21.15.71:0/3330733972 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f859406cce0 0x7f859406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:47.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.059+0000 7f85a9d9b700 1 --2- 172.21.15.71:0/3330733972 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f859406cce0 0x7f859406f1a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f85ac074880 tx=0x7f859c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:47.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.063+0000 7f859b7fe700 1 -- 172.21.15.71:0/3330733972 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f85a00564f0 con 0x7f85ac100e80 2024-04-23T02:43:47.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.272+0000 7f85b0d39700 1 -- 172.21.15.71:0/3330733972 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f85ac0611d0 con 0x7f859406cce0 2024-04-23T02:43:47.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.273+0000 7f859b7fe700 1 -- 172.21.15.71:0/3330733972 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f85ac0611d0 con 0x7f859406cce0 2024-04-23T02:43:47.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.275+0000 7f85b0d39700 1 -- 172.21.15.71:0/3330733972 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f859406cce0 msgr2=0x7f859406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:47.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.275+0000 7f85b0d39700 1 --2- 172.21.15.71:0/3330733972 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f859406cce0 0x7f859406f1a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f85ac074880 tx=0x7f859c01a040 comp rx=0 tx=0).stop 2024-04-23T02:43:47.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.275+0000 7f85b0d39700 1 -- 172.21.15.71:0/3330733972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85ac100e80 msgr2=0x7f85ac073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:47.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.275+0000 7f85b0d39700 1 --2- 172.21.15.71:0/3330733972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85ac100e80 0x7f85ac073090 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f85a000d8d0 tx=0x7f85a000dbe0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.276+0000 7f85b0d39700 1 -- 172.21.15.71:0/3330733972 shutdown_connections 2024-04-23T02:43:47.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.276+0000 7f85b0d39700 1 --2- 172.21.15.71:0/3330733972 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f859406cce0 0x7f859406f1a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.276+0000 7f85b0d39700 1 --2- 172.21.15.71:0/3330733972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85ac100e80 0x7f85ac073090 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.276+0000 7f85b0d39700 1 --2- 172.21.15.71:0/3330733972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85ac101830 0x7f85ac0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.276+0000 7f85b0d39700 1 -- 172.21.15.71:0/3330733972 >> 172.21.15.71:0/3330733972 conn(0x7f85ac0fc730 msgr2=0x7f85ac0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:47.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.276+0000 7f85b0d39700 1 -- 172.21.15.71:0/3330733972 shutdown_connections 2024-04-23T02:43:47.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.276+0000 7f85b0d39700 1 -- 172.21.15.71:0/3330733972 wait complete. 2024-04-23T02:43:47.288 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:43:47.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.379+0000 7f0c7ca41700 1 -- 172.21.15.71:0/4201835701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c78101de0 msgr2=0x7f0c781021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:47.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.379+0000 7f0c7ca41700 1 --2- 172.21.15.71:0/4201835701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c78101de0 0x7f0c781021c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009910 tx=0x7f0c6c009c20 comp rx=0 tx=0).stop 2024-04-23T02:43:47.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.379+0000 7f0c7ca41700 1 -- 172.21.15.71:0/4201835701 shutdown_connections 2024-04-23T02:43:47.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.379+0000 7f0c7ca41700 1 --2- 172.21.15.71:0/4201835701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c78102700 0x7f0c7810abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.379+0000 7f0c7ca41700 1 --2- 172.21.15.71:0/4201835701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c78101de0 0x7f0c781021c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.379+0000 7f0c7ca41700 1 -- 172.21.15.71:0/4201835701 >> 172.21.15.71:0/4201835701 conn(0x7f0c780fb610 msgr2=0x7f0c780fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.379+0000 7f0c7ca41700 1 -- 172.21.15.71:0/4201835701 shutdown_connections 2024-04-23T02:43:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.379+0000 7f0c7ca41700 1 -- 172.21.15.71:0/4201835701 wait complete. 2024-04-23T02:43:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.379+0000 7f0c7ca41700 1 Processor -- start 2024-04-23T02:43:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.380+0000 7f0c7ca41700 1 -- start start 2024-04-23T02:43:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.380+0000 7f0c7ca41700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c78101de0 0x7f0c7819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.380+0000 7f0c7ca41700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c78102700 0x7f0c7819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.380+0000 7f0c7ca41700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7819d190 con 0x7f0c78102700 2024-04-23T02:43:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.380+0000 7f0c7ca41700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c78196640 con 0x7f0c78101de0 2024-04-23T02:43:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.380+0000 7f0c75d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c78102700 0x7f0c7819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.380+0000 7f0c75d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c78102700 0x7f0c7819cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51468/0 (socket says 172.21.15.71:51468) 2024-04-23T02:43:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.380+0000 7f0c75d9b700 1 -- 172.21.15.71:0/2953780101 learned_addr learned my addr 172.21.15.71:0/2953780101 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:43:47.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.380+0000 7f0c7659c700 1 --2- 172.21.15.71:0/2953780101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c78101de0 0x7f0c7819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:47.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.380+0000 7f0c75d9b700 1 -- 172.21.15.71:0/2953780101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c78101de0 msgr2=0x7f0c7819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:47.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.380+0000 7f0c75d9b700 1 --2- 172.21.15.71:0/2953780101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c78101de0 0x7f0c7819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.380+0000 7f0c75d9b700 1 -- 172.21.15.71:0/2953780101 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c0095f0 con 0x7f0c78102700 2024-04-23T02:43:47.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.380+0000 7f0c75d9b700 1 --2- 172.21.15.71:0/2953780101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c78102700 0x7f0c7819cb00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800d6d0 tx=0x7f0c6800d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:47.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.380+0000 7f0c677fe700 1 -- 172.21.15.71:0/2953780101 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c6800f860 con 0x7f0c78102700 2024-04-23T02:43:47.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.380+0000 7f0c7ca41700 1 -- 172.21.15.71:0/2953780101 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c78196920 con 0x7f0c78102700 2024-04-23T02:43:47.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.380+0000 7f0c677fe700 1 -- 172.21.15.71:0/2953780101 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c6800fe60 con 0x7f0c78102700 2024-04-23T02:43:47.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.380+0000 7f0c677fe700 1 -- 172.21.15.71:0/2953780101 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c6800e3b0 con 0x7f0c78102700 2024-04-23T02:43:47.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.381+0000 7f0c7ca41700 1 -- 172.21.15.71:0/2953780101 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c78196e70 con 0x7f0c78102700 2024-04-23T02:43:47.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.382+0000 7f0c7ca41700 1 -- 172.21.15.71:0/2953780101 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c781082f0 con 0x7f0c78102700 2024-04-23T02:43:47.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.382+0000 7f0c677fe700 1 -- 172.21.15.71:0/2953780101 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0c68010460 con 0x7f0c78102700 2024-04-23T02:43:47.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.382+0000 7f0c677fe700 1 --2- 172.21.15.71:0/2953780101 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0c6006cce0 0x7f0c6006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:47.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.382+0000 7f0c7659c700 1 --2- 172.21.15.71:0/2953780101 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0c6006cce0 0x7f0c6006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:47.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.382+0000 7f0c677fe700 1 -- 172.21.15.71:0/2953780101 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0c6808c280 con 0x7f0c78102700 2024-04-23T02:43:47.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.383+0000 7f0c7659c700 1 --2- 172.21.15.71:0/2953780101 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0c6006cce0 0x7f0c6006f1a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c006010 tx=0x7f0c6c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:47.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.387+0000 7f0c677fe700 1 -- 172.21.15.71:0/2953780101 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0c68059d80 con 0x7f0c78102700 2024-04-23T02:43:47.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.595+0000 7f0c7ca41700 1 -- 172.21.15.71:0/2953780101 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c781970f0 con 0x7f0c6006cce0 2024-04-23T02:43:47.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.596+0000 7f0c677fe700 1 -- 172.21.15.71:0/2953780101 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0c781970f0 con 0x7f0c6006cce0 2024-04-23T02:43:47.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.598+0000 7f0c7ca41700 1 -- 172.21.15.71:0/2953780101 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0c6006cce0 msgr2=0x7f0c6006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:47.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.598+0000 7f0c7ca41700 1 --2- 172.21.15.71:0/2953780101 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0c6006cce0 0x7f0c6006f1a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c006010 tx=0x7f0c6c01a040 comp rx=0 tx=0).stop 2024-04-23T02:43:47.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.598+0000 7f0c7ca41700 1 -- 172.21.15.71:0/2953780101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c78102700 msgr2=0x7f0c7819cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:47.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.598+0000 7f0c7ca41700 1 --2- 172.21.15.71:0/2953780101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c78102700 0x7f0c7819cb00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800d6d0 tx=0x7f0c6800d9e0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.598+0000 7f0c7ca41700 1 -- 172.21.15.71:0/2953780101 shutdown_connections 2024-04-23T02:43:47.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.598+0000 7f0c7ca41700 1 --2- 172.21.15.71:0/2953780101 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0c6006cce0 0x7f0c6006f1a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.598+0000 7f0c7ca41700 1 --2- 172.21.15.71:0/2953780101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c78101de0 0x7f0c7819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.598+0000 7f0c7ca41700 1 --2- 172.21.15.71:0/2953780101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c78102700 0x7f0c7819cb00 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.598+0000 7f0c7ca41700 1 -- 172.21.15.71:0/2953780101 >> 172.21.15.71:0/2953780101 conn(0x7f0c780fb610 msgr2=0x7f0c78105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:47.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.598+0000 7f0c7ca41700 1 -- 172.21.15.71:0/2953780101 shutdown_connections 2024-04-23T02:43:47.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.598+0000 7f0c7ca41700 1 -- 172.21.15.71:0/2953780101 wait complete. 2024-04-23T02:43:47.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.700+0000 7f238e6c5700 1 -- 172.21.15.71:0/244528435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2388068580 msgr2=0x7f23880689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:47.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.700+0000 7f238e6c5700 1 --2- 172.21.15.71:0/244528435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2388068580 0x7f23880689e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2370009910 tx=0x7f2370009c20 comp rx=0 tx=0).stop 2024-04-23T02:43:47.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.701+0000 7f238e6c5700 1 -- 172.21.15.71:0/244528435 shutdown_connections 2024-04-23T02:43:47.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.701+0000 7f238e6c5700 1 --2- 172.21.15.71:0/244528435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2388068580 0x7f23880689e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.701+0000 7f238e6c5700 1 --2- 172.21.15.71:0/244528435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2388106cd0 0x7f23881070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.701+0000 7f238e6c5700 1 -- 172.21.15.71:0/244528435 >> 172.21.15.71:0/244528435 conn(0x7f2388075670 msgr2=0x7f2388075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:47.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.701+0000 7f238e6c5700 1 -- 172.21.15.71:0/244528435 shutdown_connections 2024-04-23T02:43:47.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.701+0000 7f238e6c5700 1 -- 172.21.15.71:0/244528435 wait complete. 2024-04-23T02:43:47.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.701+0000 7f238e6c5700 1 Processor -- start 2024-04-23T02:43:47.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.701+0000 7f238e6c5700 1 -- start start 2024-04-23T02:43:47.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.701+0000 7f238e6c5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2388068580 0x7f23880ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:47.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.702+0000 7f238e6c5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2388106cd0 0x7f2388100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:47.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.702+0000 7f238e6c5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2388100770 con 0x7f2388068580 2024-04-23T02:43:47.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.702+0000 7f238e6c5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23881008b0 con 0x7f2388106cd0 2024-04-23T02:43:47.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.702+0000 7f2387fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2388068580 0x7f23880ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:47.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.702+0000 7f2387fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2388068580 0x7f23880ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51494/0 (socket says 172.21.15.71:51494) 2024-04-23T02:43:47.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.702+0000 7f2387fff700 1 -- 172.21.15.71:0/2868066919 learned_addr learned my addr 172.21.15.71:0/2868066919 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:43:47.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.702+0000 7f23877fe700 1 --2- 172.21.15.71:0/2868066919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2388106cd0 0x7f2388100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:47.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.702+0000 7f2387fff700 1 -- 172.21.15.71:0/2868066919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2388106cd0 msgr2=0x7f2388100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:47.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.702+0000 7f2387fff700 1 --2- 172.21.15.71:0/2868066919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2388106cd0 0x7f2388100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.702+0000 7f2387fff700 1 -- 172.21.15.71:0/2868066919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23700095f0 con 0x7f2388068580 2024-04-23T02:43:47.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.702+0000 7f23877fe700 1 --2- 172.21.15.71:0/2868066919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2388106cd0 0x7f2388100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:43:47.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.702+0000 7f2387fff700 1 --2- 172.21.15.71:0/2868066919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2388068580 0x7f23880ffcf0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f237800c930 tx=0x7f237800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:47.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.702+0000 7f23857fa700 1 -- 172.21.15.71:0/2868066919 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2378007ad0 con 0x7f2388068580 2024-04-23T02:43:47.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.702+0000 7f238e6c5700 1 -- 172.21.15.71:0/2868066919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23881a2940 con 0x7f2388068580 2024-04-23T02:43:47.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.702+0000 7f23857fa700 1 -- 172.21.15.71:0/2868066919 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2378007c30 con 0x7f2388068580 2024-04-23T02:43:47.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.702+0000 7f23857fa700 1 -- 172.21.15.71:0/2868066919 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2378018630 con 0x7f2388068580 2024-04-23T02:43:47.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.702+0000 7f238e6c5700 1 -- 172.21.15.71:0/2868066919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23881a2ca0 con 0x7f2388068580 2024-04-23T02:43:47.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.703+0000 7f238e6c5700 1 -- 172.21.15.71:0/2868066919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23881091d0 con 0x7f2388068580 2024-04-23T02:43:47.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.704+0000 7f23857fa700 1 -- 172.21.15.71:0/2868066919 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f237800f450 con 0x7f2388068580 2024-04-23T02:43:47.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.704+0000 7f23857fa700 1 --2- 172.21.15.71:0/2868066919 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f237406cce0 0x7f237406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:47.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.704+0000 7f23857fa700 1 -- 172.21.15.71:0/2868066919 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f237808c7d0 con 0x7f2388068580 2024-04-23T02:43:47.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.704+0000 7f23877fe700 1 --2- 172.21.15.71:0/2868066919 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f237406cce0 0x7f237406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:47.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.704+0000 7f23877fe700 1 --2- 172.21.15.71:0/2868066919 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f237406cce0 0x7f237406f1a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f23881014e0 tx=0x7f237001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:47.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.709+0000 7f23857fa700 1 -- 172.21.15.71:0/2868066919 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f23780568f0 con 0x7f2388068580 2024-04-23T02:43:47.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:47 smithi192 ceph-mon[108617]: pgmap v915: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:47.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.913+0000 7f238e6c5700 1 -- 172.21.15.71:0/2868066919 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f23880611d0 con 0x7f237406cce0 2024-04-23T02:43:47.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.923+0000 7f23857fa700 1 -- 172.21.15.71:0/2868066919 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f23880611d0 con 0x7f237406cce0 2024-04-23T02:43:47.926 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:43:47.926 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (27m) 5m ago 38m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:43:47.926 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (39m) 5m ago 39m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:43:47.926 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (37m) 5m ago 37m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:43:47.926 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (25m) 5m ago 39m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:43:47.926 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (25m) 5m ago 37m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:43:47.926 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (27m) 5m ago 38m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:43:47.926 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (21m) 5m ago 32m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:43:47.926 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (21m) 5m ago 32m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:43:47.926 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (5m) 5m ago 32m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:43:47.926 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (20m) 5m ago 32m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:43:47.927 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (29m) 5m ago 40m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:43:47.927 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (28m) 5m ago 37m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:43:47.927 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (25m) 5m ago 40m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:43:47.927 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (25m) 5m ago 37m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:43:47.927 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (28m) 5m ago 38m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:43:47.927 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (28m) 5m ago 37m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:43:47.927 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (24m) 5m ago 36m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:43:47.927 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (24m) 5m ago 36m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:43:47.927 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (23m) 5m ago 35m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:43:47.927 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (23m) 5m ago 35m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:43:47.927 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (22m) 5m ago 34m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:43:47.927 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (21m) 5m ago 34m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:43:47.927 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (27m) 5m ago 38m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:43:47.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.925+0000 7f238e6c5700 1 -- 172.21.15.71:0/2868066919 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f237406cce0 msgr2=0x7f237406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:47.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.925+0000 7f238e6c5700 1 --2- 172.21.15.71:0/2868066919 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f237406cce0 0x7f237406f1a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f23881014e0 tx=0x7f237001a040 comp rx=0 tx=0).stop 2024-04-23T02:43:47.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.925+0000 7f238e6c5700 1 -- 172.21.15.71:0/2868066919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2388068580 msgr2=0x7f23880ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:47.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.925+0000 7f238e6c5700 1 --2- 172.21.15.71:0/2868066919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2388068580 0x7f23880ffcf0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f237800c930 tx=0x7f237800ccf0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.925+0000 7f238e6c5700 1 -- 172.21.15.71:0/2868066919 shutdown_connections 2024-04-23T02:43:47.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.925+0000 7f238e6c5700 1 --2- 172.21.15.71:0/2868066919 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f237406cce0 0x7f237406f1a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.925+0000 7f238e6c5700 1 --2- 172.21.15.71:0/2868066919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2388068580 0x7f23880ffcf0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.925+0000 7f238e6c5700 1 --2- 172.21.15.71:0/2868066919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2388106cd0 0x7f2388100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:47.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.925+0000 7f238e6c5700 1 -- 172.21.15.71:0/2868066919 >> 172.21.15.71:0/2868066919 conn(0x7f2388075670 msgr2=0x7f23880fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:47.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.925+0000 7f238e6c5700 1 -- 172.21.15.71:0/2868066919 shutdown_connections 2024-04-23T02:43:47.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:47.925+0000 7f238e6c5700 1 -- 172.21.15.71:0/2868066919 wait complete. 2024-04-23T02:43:47.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:47 smithi071 ceph-mon[144544]: pgmap v915: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.027+0000 7f7c26f18700 1 -- 172.21.15.71:0/1540789982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c20102720 msgr2=0x7f7c2010ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.027+0000 7f7c26f18700 1 --2- 172.21.15.71:0/1540789982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c20102720 0x7f7c2010ac10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009910 tx=0x7f7c0c009c20 comp rx=0 tx=0).stop 2024-04-23T02:43:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.027+0000 7f7c26f18700 1 -- 172.21.15.71:0/1540789982 shutdown_connections 2024-04-23T02:43:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.027+0000 7f7c26f18700 1 --2- 172.21.15.71:0/1540789982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c20102720 0x7f7c2010ac10 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.027+0000 7f7c26f18700 1 --2- 172.21.15.71:0/1540789982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c20101e00 0x7f7c201021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.027+0000 7f7c26f18700 1 -- 172.21.15.71:0/1540789982 >> 172.21.15.71:0/1540789982 conn(0x7f7c200fb650 msgr2=0x7f7c200fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.027+0000 7f7c26f18700 1 -- 172.21.15.71:0/1540789982 shutdown_connections 2024-04-23T02:43:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.027+0000 7f7c26f18700 1 -- 172.21.15.71:0/1540789982 wait complete. 2024-04-23T02:43:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.028+0000 7f7c26f18700 1 Processor -- start 2024-04-23T02:43:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.028+0000 7f7c26f18700 1 -- start start 2024-04-23T02:43:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.028+0000 7f7c26f18700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c20101e00 0x7f7c2019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.028+0000 7f7c26f18700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c20102720 0x7f7c2019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.028+0000 7f7c26f18700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2019d1e0 con 0x7f7c20101e00 2024-04-23T02:43:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.028+0000 7f7c26f18700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c20196690 con 0x7f7c20102720 2024-04-23T02:43:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.028+0000 7f7c24cb4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c20101e00 0x7f7c2019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.028+0000 7f7c24cb4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c20101e00 0x7f7c2019c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51502/0 (socket says 172.21.15.71:51502) 2024-04-23T02:43:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.028+0000 7f7c24cb4700 1 -- 172.21.15.71:0/537638673 learned_addr learned my addr 172.21.15.71:0/537638673 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:43:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.028+0000 7f7c1ffff700 1 --2- 172.21.15.71:0/537638673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c20102720 0x7f7c2019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:48.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.028+0000 7f7c24cb4700 1 -- 172.21.15.71:0/537638673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c20102720 msgr2=0x7f7c2019cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:48.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.028+0000 7f7c24cb4700 1 --2- 172.21.15.71:0/537638673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c20102720 0x7f7c2019cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.028+0000 7f7c24cb4700 1 -- 172.21.15.71:0/537638673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0c0095f0 con 0x7f7c20101e00 2024-04-23T02:43:48.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.028+0000 7f7c1ffff700 1 --2- 172.21.15.71:0/537638673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c20102720 0x7f7c2019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:43:48.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.029+0000 7f7c24cb4700 1 --2- 172.21.15.71:0/537638673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c20101e00 0x7f7c2019c610 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400d900 tx=0x7f7c1400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:48.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.029+0000 7f7c1dffb700 1 -- 172.21.15.71:0/537638673 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c14004a00 con 0x7f7c20101e00 2024-04-23T02:43:48.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.029+0000 7f7c1dffb700 1 -- 172.21.15.71:0/537638673 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7c14005020 con 0x7f7c20101e00 2024-04-23T02:43:48.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.029+0000 7f7c26f18700 1 -- 172.21.15.71:0/537638673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c20196970 con 0x7f7c20101e00 2024-04-23T02:43:48.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.029+0000 7f7c1dffb700 1 -- 172.21.15.71:0/537638673 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c14009c00 con 0x7f7c20101e00 2024-04-23T02:43:48.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.029+0000 7f7c26f18700 1 -- 172.21.15.71:0/537638673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c20196e90 con 0x7f7c20101e00 2024-04-23T02:43:48.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.030+0000 7f7c26f18700 1 -- 172.21.15.71:0/537638673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c20108310 con 0x7f7c20101e00 2024-04-23T02:43:48.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.030+0000 7f7c1dffb700 1 -- 172.21.15.71:0/537638673 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7c14005190 con 0x7f7c20101e00 2024-04-23T02:43:48.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.031+0000 7f7c1dffb700 1 --2- 172.21.15.71:0/537638673 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7c1006cce0 0x7f7c1006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:48.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.031+0000 7f7c1dffb700 1 -- 172.21.15.71:0/537638673 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7c1408c770 con 0x7f7c20101e00 2024-04-23T02:43:48.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.031+0000 7f7c1ffff700 1 --2- 172.21.15.71:0/537638673 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7c1006cce0 0x7f7c1006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:48.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.031+0000 7f7c1ffff700 1 --2- 172.21.15.71:0/537638673 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7c1006cce0 0x7f7c1006f1a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c006010 tx=0x7f7c0c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:48.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.035+0000 7f7c1dffb700 1 -- 172.21.15.71:0/537638673 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7c1405a270 con 0x7f7c20101e00 2024-04-23T02:43:48.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.305+0000 7f7c26f18700 1 -- 172.21.15.71:0/537638673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7c20197c80 con 0x7f7c20101e00 2024-04-23T02:43:48.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.305+0000 7f7c1dffb700 1 -- 172.21.15.71:0/537638673 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f7c14005440 con 0x7f7c20101e00 2024-04-23T02:43:48.308 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:43:48.308 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:43:48.308 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:43:48.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:43:48.308 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:43:48.308 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:43:48.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:43:48.309 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:43:48.309 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:43:48.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:43:48.309 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:43:48.309 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:43:48.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:43:48.309 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:43:48.309 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:43:48.309 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:43:48.309 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:43:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.307+0000 7f7c26f18700 1 -- 172.21.15.71:0/537638673 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7c1006cce0 msgr2=0x7f7c1006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.307+0000 7f7c26f18700 1 --2- 172.21.15.71:0/537638673 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7c1006cce0 0x7f7c1006f1a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c006010 tx=0x7f7c0c01a040 comp rx=0 tx=0).stop 2024-04-23T02:43:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.307+0000 7f7c26f18700 1 -- 172.21.15.71:0/537638673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c20101e00 msgr2=0x7f7c2019c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.307+0000 7f7c26f18700 1 --2- 172.21.15.71:0/537638673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c20101e00 0x7f7c2019c610 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400d900 tx=0x7f7c1400dcc0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.307+0000 7f7c26f18700 1 -- 172.21.15.71:0/537638673 shutdown_connections 2024-04-23T02:43:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.307+0000 7f7c26f18700 1 --2- 172.21.15.71:0/537638673 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7c1006cce0 0x7f7c1006f1a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.307+0000 7f7c26f18700 1 --2- 172.21.15.71:0/537638673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c20101e00 0x7f7c2019c610 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.307+0000 7f7c26f18700 1 --2- 172.21.15.71:0/537638673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c20102720 0x7f7c2019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.307+0000 7f7c26f18700 1 -- 172.21.15.71:0/537638673 >> 172.21.15.71:0/537638673 conn(0x7f7c200fb650 msgr2=0x7f7c20105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.307+0000 7f7c26f18700 1 -- 172.21.15.71:0/537638673 shutdown_connections 2024-04-23T02:43:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.307+0000 7f7c26f18700 1 -- 172.21.15.71:0/537638673 wait complete. 2024-04-23T02:43:48.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.409+0000 7f279d819700 1 -- 172.21.15.71:0/17363036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2798100ea0 msgr2=0x7f2798101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:48.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.409+0000 7f279d819700 1 --2- 172.21.15.71:0/17363036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2798100ea0 0x7f2798101280 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f278c009910 tx=0x7f278c009c20 comp rx=0 tx=0).stop 2024-04-23T02:43:48.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.409+0000 7f279d819700 1 -- 172.21.15.71:0/17363036 shutdown_connections 2024-04-23T02:43:48.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.409+0000 7f279d819700 1 --2- 172.21.15.71:0/17363036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2798101850 0x7f27981058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.409+0000 7f279d819700 1 --2- 172.21.15.71:0/17363036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2798100ea0 0x7f2798101280 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.409+0000 7f279d819700 1 -- 172.21.15.71:0/17363036 >> 172.21.15.71:0/17363036 conn(0x7f27980fc750 msgr2=0x7f27980feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.409+0000 7f279d819700 1 -- 172.21.15.71:0/17363036 shutdown_connections 2024-04-23T02:43:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.409+0000 7f279d819700 1 -- 172.21.15.71:0/17363036 wait complete. 2024-04-23T02:43:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.410+0000 7f279d819700 1 Processor -- start 2024-04-23T02:43:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.410+0000 7f279d819700 1 -- start start 2024-04-23T02:43:48.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.410+0000 7f279d819700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2798100ea0 0x7f279819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:48.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.410+0000 7f279d819700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2798101850 0x7f279819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:48.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.410+0000 7f279d819700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279819d130 con 0x7f2798100ea0 2024-04-23T02:43:48.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.410+0000 7f279d819700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27981965e0 con 0x7f2798101850 2024-04-23T02:43:48.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.410+0000 7f2796ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2798100ea0 0x7f279819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:48.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.410+0000 7f2796ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2798100ea0 0x7f279819c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51518/0 (socket says 172.21.15.71:51518) 2024-04-23T02:43:48.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.410+0000 7f2796ffd700 1 -- 172.21.15.71:0/3458771656 learned_addr learned my addr 172.21.15.71:0/3458771656 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:43:48.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.410+0000 7f27967fc700 1 --2- 172.21.15.71:0/3458771656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2798101850 0x7f279819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:48.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.410+0000 7f2796ffd700 1 -- 172.21.15.71:0/3458771656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2798101850 msgr2=0x7f279819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:48.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.410+0000 7f2796ffd700 1 --2- 172.21.15.71:0/3458771656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2798101850 0x7f279819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.411+0000 7f2796ffd700 1 -- 172.21.15.71:0/3458771656 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c0095f0 con 0x7f2798100ea0 2024-04-23T02:43:48.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.411+0000 7f2796ffd700 1 --2- 172.21.15.71:0/3458771656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2798100ea0 0x7f279819c560 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f278c006010 tx=0x7f278c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:48.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.411+0000 7f279c817700 1 -- 172.21.15.71:0/3458771656 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f278c01d070 con 0x7f2798100ea0 2024-04-23T02:43:48.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.411+0000 7f279c817700 1 -- 172.21.15.71:0/3458771656 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f278c005710 con 0x7f2798100ea0 2024-04-23T02:43:48.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.411+0000 7f279d819700 1 -- 172.21.15.71:0/3458771656 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2798196860 con 0x7f2798100ea0 2024-04-23T02:43:48.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.411+0000 7f279c817700 1 -- 172.21.15.71:0/3458771656 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f278c00fe20 con 0x7f2798100ea0 2024-04-23T02:43:48.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.411+0000 7f279d819700 1 -- 172.21.15.71:0/3458771656 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2798196d50 con 0x7f2798100ea0 2024-04-23T02:43:48.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.412+0000 7f279d819700 1 -- 172.21.15.71:0/3458771656 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2798109240 con 0x7f2798100ea0 2024-04-23T02:43:48.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.412+0000 7f279c817700 1 -- 172.21.15.71:0/3458771656 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f278c003a60 con 0x7f2798100ea0 2024-04-23T02:43:48.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.413+0000 7f279c817700 1 --2- 172.21.15.71:0/3458771656 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f278006cce0 0x7f278006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:48.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.413+0000 7f279c817700 1 -- 172.21.15.71:0/3458771656 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f278c08d750 con 0x7f2798100ea0 2024-04-23T02:43:48.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.413+0000 7f27967fc700 1 --2- 172.21.15.71:0/3458771656 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f278006cce0 0x7f278006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:48.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.413+0000 7f27967fc700 1 --2- 172.21.15.71:0/3458771656 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f278006cce0 0x7f278006f1a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2798105b20 tx=0x7f278800b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:48.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.417+0000 7f279c817700 1 -- 172.21.15.71:0/3458771656 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f278c05b250 con 0x7f2798100ea0 2024-04-23T02:43:48.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.651+0000 7f279d819700 1 -- 172.21.15.71:0/3458771656 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2798108c20 con 0x7f2798100ea0 2024-04-23T02:43:48.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.651+0000 7f279c817700 1 -- 172.21.15.71:0/3458771656 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f278c05ade0 con 0x7f2798100ea0 2024-04-23T02:43:48.654 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:43:48.654 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:43:48.654 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:43:48.654 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:43:48.654 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:43:48.654 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:43:48.654 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:43:48.654 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:43:48.654 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:43:48.654 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:43:48.655 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:43:48.655 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:43:48.655 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:43:48.655 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:43:48.655 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:43:48.655 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:43:48.655 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:43:48.655 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:43:48.655 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:43:48.655 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:43:48.655 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:43:48.655 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:43:48.655 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:43:48.655 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:43:48.656 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:43:48.656 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:43:48.656 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:43:48.656 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:43:48.656 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:43:48.656 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:43:48.656 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:43:48.656 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:43:48.656 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:43:48.656 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:43:48.656 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:43:48.656 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:43:48.656 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:43:48.656 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:43:48.656 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:43:48.657 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:43:48.657 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:43:48.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.653+0000 7f279d819700 1 -- 172.21.15.71:0/3458771656 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f278006cce0 msgr2=0x7f278006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:48.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.653+0000 7f279d819700 1 --2- 172.21.15.71:0/3458771656 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f278006cce0 0x7f278006f1a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2798105b20 tx=0x7f278800b410 comp rx=0 tx=0).stop 2024-04-23T02:43:48.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.653+0000 7f279d819700 1 -- 172.21.15.71:0/3458771656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2798100ea0 msgr2=0x7f279819c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:48.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.653+0000 7f279d819700 1 --2- 172.21.15.71:0/3458771656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2798100ea0 0x7f279819c560 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f278c006010 tx=0x7f278c00bd70 comp rx=0 tx=0).stop 2024-04-23T02:43:48.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.653+0000 7f279d819700 1 -- 172.21.15.71:0/3458771656 shutdown_connections 2024-04-23T02:43:48.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.653+0000 7f279d819700 1 --2- 172.21.15.71:0/3458771656 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f278006cce0 0x7f278006f1a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.653+0000 7f279d819700 1 --2- 172.21.15.71:0/3458771656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2798100ea0 0x7f279819c560 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.653+0000 7f279d819700 1 --2- 172.21.15.71:0/3458771656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2798101850 0x7f279819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.653+0000 7f279d819700 1 -- 172.21.15.71:0/3458771656 >> 172.21.15.71:0/3458771656 conn(0x7f27980fc750 msgr2=0x7f27980fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:48.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.653+0000 7f279d819700 1 -- 172.21.15.71:0/3458771656 shutdown_connections 2024-04-23T02:43:48.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.653+0000 7f279d819700 1 -- 172.21.15.71:0/3458771656 wait complete. 2024-04-23T02:43:48.658 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:43:48.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.756+0000 7f090dd06700 1 -- 172.21.15.71:0/2428978224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0908068580 msgr2=0x7f09080689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:48.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.756+0000 7f090dd06700 1 --2- 172.21.15.71:0/2428978224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0908068580 0x7f09080689e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009910 tx=0x7f08f0009c20 comp rx=0 tx=0).stop 2024-04-23T02:43:48.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.756+0000 7f090dd06700 1 -- 172.21.15.71:0/2428978224 shutdown_connections 2024-04-23T02:43:48.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.756+0000 7f090dd06700 1 --2- 172.21.15.71:0/2428978224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0908068580 0x7f09080689e0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.756+0000 7f090dd06700 1 --2- 172.21.15.71:0/2428978224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0908106cd0 0x7f09081070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.756+0000 7f090dd06700 1 -- 172.21.15.71:0/2428978224 >> 172.21.15.71:0/2428978224 conn(0x7f0908075670 msgr2=0x7f0908075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:48.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.756+0000 7f090dd06700 1 -- 172.21.15.71:0/2428978224 shutdown_connections 2024-04-23T02:43:48.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.756+0000 7f090dd06700 1 -- 172.21.15.71:0/2428978224 wait complete. 2024-04-23T02:43:48.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.757+0000 7f090dd06700 1 Processor -- start 2024-04-23T02:43:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.757+0000 7f090dd06700 1 -- start start 2024-04-23T02:43:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.757+0000 7f090dd06700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0908068580 0x7f09080ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.757+0000 7f090dd06700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0908106cd0 0x7f0908100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.757+0000 7f090dd06700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0908100770 con 0x7f0908106cd0 2024-04-23T02:43:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.757+0000 7f090dd06700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09081008b0 con 0x7f0908068580 2024-04-23T02:43:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.757+0000 7f0906ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0908106cd0 0x7f0908100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.757+0000 7f0906ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0908106cd0 0x7f0908100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51522/0 (socket says 172.21.15.71:51522) 2024-04-23T02:43:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.757+0000 7f0906ffd700 1 -- 172.21.15.71:0/430120073 learned_addr learned my addr 172.21.15.71:0/430120073 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:43:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.757+0000 7f09077fe700 1 --2- 172.21.15.71:0/430120073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0908068580 0x7f09080ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.757+0000 7f0906ffd700 1 -- 172.21.15.71:0/430120073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0908068580 msgr2=0x7f09080ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.757+0000 7f0906ffd700 1 --2- 172.21.15.71:0/430120073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0908068580 0x7f09080ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.757+0000 7f0906ffd700 1 -- 172.21.15.71:0/430120073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f00095f0 con 0x7f0908106cd0 2024-04-23T02:43:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.758+0000 7f0906ffd700 1 --2- 172.21.15.71:0/430120073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0908106cd0 0x7f0908100230 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f08f0006010 tx=0x7f08f000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.758+0000 7f0904ff9700 1 -- 172.21.15.71:0/430120073 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08f001d070 con 0x7f0908106cd0 2024-04-23T02:43:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.758+0000 7f0904ff9700 1 -- 172.21.15.71:0/430120073 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f08f000f460 con 0x7f0908106cd0 2024-04-23T02:43:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.758+0000 7f090dd06700 1 -- 172.21.15.71:0/430120073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0908100b30 con 0x7f0908106cd0 2024-04-23T02:43:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.758+0000 7f0904ff9700 1 -- 172.21.15.71:0/430120073 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08f0021640 con 0x7f0908106cd0 2024-04-23T02:43:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.758+0000 7f090dd06700 1 -- 172.21.15.71:0/430120073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09081a2c70 con 0x7f0908106cd0 2024-04-23T02:43:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.759+0000 7f090dd06700 1 -- 172.21.15.71:0/430120073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09081091d0 con 0x7f0908106cd0 2024-04-23T02:43:48.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.759+0000 7f0904ff9700 1 -- 172.21.15.71:0/430120073 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f08f000fa20 con 0x7f0908106cd0 2024-04-23T02:43:48.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.759+0000 7f0904ff9700 1 --2- 172.21.15.71:0/430120073 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f08f406cce0 0x7f08f406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:48.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.760+0000 7f0904ff9700 1 -- 172.21.15.71:0/430120073 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f08f008d740 con 0x7f0908106cd0 2024-04-23T02:43:48.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.760+0000 7f09077fe700 1 --2- 172.21.15.71:0/430120073 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f08f406cce0 0x7f08f406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:48.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.760+0000 7f09077fe700 1 --2- 172.21.15.71:0/430120073 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f08f406cce0 0x7f08f406f1a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f08f8005950 tx=0x7f08f80058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:48.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.764+0000 7f0904ff9700 1 -- 172.21.15.71:0/430120073 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f08f0057830 con 0x7f0908106cd0 2024-04-23T02:43:48.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:48 smithi192 ceph-mon[108617]: from='client.35464 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:43:48.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:48 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/537638673' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:43:48.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.974+0000 7f090dd06700 1 -- 172.21.15.71:0/430120073 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f09080611d0 con 0x7f08f406cce0 2024-04-23T02:43:48.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.975+0000 7f0904ff9700 1 -- 172.21.15.71:0/430120073 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f09080611d0 con 0x7f08f406cce0 2024-04-23T02:43:48.978 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:43:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:43:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:43:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:43:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:43:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:43:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:43:48.978 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:43:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:43:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:43:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:43:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:43:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:43:48.979 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:43:48.979 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:43:48.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.977+0000 7f090dd06700 1 -- 172.21.15.71:0/430120073 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f08f406cce0 msgr2=0x7f08f406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:48.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.977+0000 7f090dd06700 1 --2- 172.21.15.71:0/430120073 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f08f406cce0 0x7f08f406f1a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f08f8005950 tx=0x7f08f80058e0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.977+0000 7f090dd06700 1 -- 172.21.15.71:0/430120073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0908106cd0 msgr2=0x7f0908100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:48.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.977+0000 7f090dd06700 1 --2- 172.21.15.71:0/430120073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0908106cd0 0x7f0908100230 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f08f0006010 tx=0x7f08f000be20 comp rx=0 tx=0).stop 2024-04-23T02:43:48.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.977+0000 7f090dd06700 1 -- 172.21.15.71:0/430120073 shutdown_connections 2024-04-23T02:43:48.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.977+0000 7f090dd06700 1 --2- 172.21.15.71:0/430120073 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f08f406cce0 0x7f08f406f1a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.977+0000 7f090dd06700 1 --2- 172.21.15.71:0/430120073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0908068580 0x7f09080ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.977+0000 7f090dd06700 1 --2- 172.21.15.71:0/430120073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0908106cd0 0x7f0908100230 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:48.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.977+0000 7f090dd06700 1 -- 172.21.15.71:0/430120073 >> 172.21.15.71:0/430120073 conn(0x7f0908075670 msgr2=0x7f09080fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:48.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.977+0000 7f090dd06700 1 -- 172.21.15.71:0/430120073 shutdown_connections 2024-04-23T02:43:48.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:48.977+0000 7f090dd06700 1 -- 172.21.15.71:0/430120073 wait complete. 2024-04-23T02:43:48.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:48 smithi071 ceph-mon[144544]: from='client.35464 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:43:48.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:48 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/537638673' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:43:49.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.079+0000 7f3a64473700 1 -- 172.21.15.71:0/1030666259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a5c068580 msgr2=0x7f3a5c0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:49.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.079+0000 7f3a64473700 1 --2- 172.21.15.71:0/1030666259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a5c068580 0x7f3a5c0689e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009910 tx=0x7f3a4c009c20 comp rx=0 tx=0).stop 2024-04-23T02:43:49.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.079+0000 7f3a64473700 1 -- 172.21.15.71:0/1030666259 shutdown_connections 2024-04-23T02:43:49.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.079+0000 7f3a64473700 1 --2- 172.21.15.71:0/1030666259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a5c068580 0x7f3a5c0689e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:49.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.079+0000 7f3a64473700 1 --2- 172.21.15.71:0/1030666259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a5c106cd0 0x7f3a5c1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:49.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.079+0000 7f3a64473700 1 -- 172.21.15.71:0/1030666259 >> 172.21.15.71:0/1030666259 conn(0x7f3a5c075670 msgr2=0x7f3a5c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:49.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.079+0000 7f3a64473700 1 -- 172.21.15.71:0/1030666259 shutdown_connections 2024-04-23T02:43:49.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.080+0000 7f3a64473700 1 -- 172.21.15.71:0/1030666259 wait complete. 2024-04-23T02:43:49.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.080+0000 7f3a64473700 1 Processor -- start 2024-04-23T02:43:49.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.080+0000 7f3a64473700 1 -- start start 2024-04-23T02:43:49.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.080+0000 7f3a64473700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a5c068580 0x7f3a5c0ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:49.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.080+0000 7f3a64473700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a5c106cd0 0x7f3a5c100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:49.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.080+0000 7f3a64473700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c100770 con 0x7f3a5c106cd0 2024-04-23T02:43:49.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.080+0000 7f3a64473700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c1008b0 con 0x7f3a5c068580 2024-04-23T02:43:49.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.080+0000 7f3a61a0e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a5c106cd0 0x7f3a5c100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:49.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.080+0000 7f3a61a0e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a5c106cd0 0x7f3a5c100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51534/0 (socket says 172.21.15.71:51534) 2024-04-23T02:43:49.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.080+0000 7f3a61a0e700 1 -- 172.21.15.71:0/1947331020 learned_addr learned my addr 172.21.15.71:0/1947331020 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:43:49.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.081+0000 7f3a6220f700 1 --2- 172.21.15.71:0/1947331020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a5c068580 0x7f3a5c0ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:49.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.081+0000 7f3a61a0e700 1 -- 172.21.15.71:0/1947331020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a5c068580 msgr2=0x7f3a5c0ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:49.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.081+0000 7f3a61a0e700 1 --2- 172.21.15.71:0/1947331020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a5c068580 0x7f3a5c0ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:49.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.081+0000 7f3a61a0e700 1 -- 172.21.15.71:0/1947331020 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c0095f0 con 0x7f3a5c106cd0 2024-04-23T02:43:49.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.081+0000 7f3a6220f700 1 --2- 172.21.15.71:0/1947331020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a5c068580 0x7f3a5c0ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:43:49.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.081+0000 7f3a61a0e700 1 --2- 172.21.15.71:0/1947331020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a5c106cd0 0x7f3a5c100230 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c006010 tx=0x7f3a4c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:49.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.081+0000 7f3a537fe700 1 -- 172.21.15.71:0/1947331020 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a4c01d070 con 0x7f3a5c106cd0 2024-04-23T02:43:49.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.081+0000 7f3a64473700 1 -- 172.21.15.71:0/1947331020 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5c100b30 con 0x7f3a5c106cd0 2024-04-23T02:43:49.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.081+0000 7f3a537fe700 1 -- 172.21.15.71:0/1947331020 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3a4c00f460 con 0x7f3a5c106cd0 2024-04-23T02:43:49.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.081+0000 7f3a537fe700 1 -- 172.21.15.71:0/1947331020 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a4c021640 con 0x7f3a5c106cd0 2024-04-23T02:43:49.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.081+0000 7f3a64473700 1 -- 172.21.15.71:0/1947331020 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5c1a2df0 con 0x7f3a5c106cd0 2024-04-23T02:43:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.082+0000 7f3a64473700 1 -- 172.21.15.71:0/1947331020 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5c1091f0 con 0x7f3a5c106cd0 2024-04-23T02:43:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.082+0000 7f3a537fe700 1 -- 172.21.15.71:0/1947331020 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3a4c00fa60 con 0x7f3a5c106cd0 2024-04-23T02:43:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.083+0000 7f3a537fe700 1 --2- 172.21.15.71:0/1947331020 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3a4806cce0 0x7f3a4806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:43:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.083+0000 7f3a537fe700 1 -- 172.21.15.71:0/1947331020 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3a4c08d860 con 0x7f3a5c106cd0 2024-04-23T02:43:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.083+0000 7f3a6220f700 1 --2- 172.21.15.71:0/1947331020 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3a4806cce0 0x7f3a4806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:43:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.083+0000 7f3a6220f700 1 --2- 172.21.15.71:0/1947331020 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3a4806cce0 0x7f3a4806f1a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3a580099d0 tx=0x7f3a58008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:43:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.087+0000 7f3a537fe700 1 -- 172.21.15.71:0/1947331020 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3a4c0578d0 con 0x7f3a5c106cd0 2024-04-23T02:43:49.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.369+0000 7f3a64473700 1 -- 172.21.15.71:0/1947331020 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3a5c04f2e0 con 0x7f3a5c106cd0 2024-04-23T02:43:49.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.370+0000 7f3a537fe700 1 -- 172.21.15.71:0/1947331020 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7f3a4c05aef0 con 0x7f3a5c106cd0 2024-04-23T02:43:49.373 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:43:49.373 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:43:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:43:49.373 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:43:49.373 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:43:49.373 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:43:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:43:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:43:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:43:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:43:49.373 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:43:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:43:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:43:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:43:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:43:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:43:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:43:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:43:49.374 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:43:49.374 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:43:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:43:49.374 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:43:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:43:49.374 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:43:49.374 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:43:49.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.372+0000 7f3a64473700 1 -- 172.21.15.71:0/1947331020 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3a4806cce0 msgr2=0x7f3a4806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:49.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.372+0000 7f3a64473700 1 --2- 172.21.15.71:0/1947331020 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3a4806cce0 0x7f3a4806f1a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3a580099d0 tx=0x7f3a58008040 comp rx=0 tx=0).stop 2024-04-23T02:43:49.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.372+0000 7f3a64473700 1 -- 172.21.15.71:0/1947331020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a5c106cd0 msgr2=0x7f3a5c100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:43:49.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.372+0000 7f3a64473700 1 --2- 172.21.15.71:0/1947331020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a5c106cd0 0x7f3a5c100230 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c006010 tx=0x7f3a4c00be20 comp rx=0 tx=0).stop 2024-04-23T02:43:49.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.372+0000 7f3a64473700 1 -- 172.21.15.71:0/1947331020 shutdown_connections 2024-04-23T02:43:49.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.372+0000 7f3a64473700 1 --2- 172.21.15.71:0/1947331020 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3a4806cce0 0x7f3a4806f1a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:49.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.372+0000 7f3a64473700 1 --2- 172.21.15.71:0/1947331020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a5c068580 0x7f3a5c0ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:49.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.372+0000 7f3a64473700 1 --2- 172.21.15.71:0/1947331020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a5c106cd0 0x7f3a5c100230 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:43:49.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.372+0000 7f3a64473700 1 -- 172.21.15.71:0/1947331020 >> 172.21.15.71:0/1947331020 conn(0x7f3a5c075670 msgr2=0x7f3a5c0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:43:49.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.372+0000 7f3a64473700 1 -- 172.21.15.71:0/1947331020 shutdown_connections 2024-04-23T02:43:49.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:43:49.372+0000 7f3a64473700 1 -- 172.21.15.71:0/1947331020 wait complete. 2024-04-23T02:43:49.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:49 smithi192 ceph-mon[108617]: from='client.35468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:43:49.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:49 smithi192 ceph-mon[108617]: from='client.35472 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:43:49.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:49 smithi192 ceph-mon[108617]: pgmap v916: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:49.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:49 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3458771656' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:43:49.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:49 smithi192 ceph-mon[108617]: from='client.35484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:43:49.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:49 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1947331020' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:43:49.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:49 smithi071 ceph-mon[144544]: from='client.35468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:43:49.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:49 smithi071 ceph-mon[144544]: from='client.35472 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:43:49.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:49 smithi071 ceph-mon[144544]: pgmap v916: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:49.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:49 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3458771656' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:43:49.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:49 smithi071 ceph-mon[144544]: from='client.35484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:43:49.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:49 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1947331020' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:43:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:43:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:50 smithi192 ceph-mon[108617]: pgmap v917: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:43:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:50 smithi071 ceph-mon[144544]: pgmap v917: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:53.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:53 smithi192 ceph-mon[108617]: pgmap v918: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:53.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:53 smithi071 ceph-mon[144544]: pgmap v918: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:55.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:55 smithi192 ceph-mon[108617]: pgmap v919: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:55.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:55 smithi071 ceph-mon[144544]: pgmap v919: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:57.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:57 smithi192 ceph-mon[108617]: pgmap v920: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:57.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:57 smithi071 ceph-mon[144544]: pgmap v920: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:59.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:43:59 smithi192 ceph-mon[108617]: pgmap v921: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:43:59.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:43:59 smithi071 ceph-mon[144544]: pgmap v921: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:01.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:01 smithi192 ceph-mon[108617]: pgmap v922: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:01.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:01 smithi071 ceph-mon[144544]: pgmap v922: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:03.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:03 smithi192 ceph-mon[108617]: pgmap v923: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:03.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:03 smithi071 ceph-mon[144544]: pgmap v923: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:05 smithi192 ceph-mon[108617]: pgmap v924: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:44:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:05 smithi071 ceph-mon[144544]: pgmap v924: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:05.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:44:07.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:07 smithi192 ceph-mon[108617]: pgmap v925: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:07.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:07 smithi071 ceph-mon[144544]: pgmap v925: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:09.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:09 smithi192 ceph-mon[108617]: pgmap v926: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:09.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:09 smithi071 ceph-mon[144544]: pgmap v926: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:11.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:11 smithi192 ceph-mon[108617]: pgmap v927: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:11.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:11 smithi071 ceph-mon[144544]: pgmap v927: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:13.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:13 smithi192 ceph-mon[108617]: pgmap v928: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:13.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:13 smithi071 ceph-mon[144544]: pgmap v928: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:15.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:15 smithi192 ceph-mon[108617]: pgmap v929: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:15.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:15 smithi071 ceph-mon[144544]: pgmap v929: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:16.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:16 smithi071 ceph-mon[144544]: pgmap v930: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:17.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:16 smithi192 ceph-mon[108617]: pgmap v930: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:19.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.488+0000 7f0ee9abc700 1 -- 172.21.15.71:0/2668220118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ee40ffc20 msgr2=0x7f0ee4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:19.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.488+0000 7f0ee9abc700 1 --2- 172.21.15.71:0/2668220118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ee40ffc20 0x7f0ee4100080 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4009910 tx=0x7f0ed4009c20 comp rx=0 tx=0).stop 2024-04-23T02:44:19.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.488+0000 7f0ee9abc700 1 -- 172.21.15.71:0/2668220118 shutdown_connections 2024-04-23T02:44:19.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.488+0000 7f0ee9abc700 1 --2- 172.21.15.71:0/2668220118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ee40ffc20 0x7f0ee4100080 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:19.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.488+0000 7f0ee9abc700 1 --2- 172.21.15.71:0/2668220118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ee4105bd0 0x7f0ee4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:19.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.488+0000 7f0ee9abc700 1 -- 172.21.15.71:0/2668220118 >> 172.21.15.71:0/2668220118 conn(0x7f0ee4075670 msgr2=0x7f0ee4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:19.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.488+0000 7f0ee9abc700 1 -- 172.21.15.71:0/2668220118 shutdown_connections 2024-04-23T02:44:19.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.488+0000 7f0ee9abc700 1 -- 172.21.15.71:0/2668220118 wait complete. 2024-04-23T02:44:19.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.489+0000 7f0ee9abc700 1 Processor -- start 2024-04-23T02:44:19.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.489+0000 7f0ee9abc700 1 -- start start 2024-04-23T02:44:19.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.489+0000 7f0ee9abc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ee4105bd0 0x7f0ee419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:19.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.489+0000 7f0ee9abc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ee419cd20 0x7f0ee4196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:19.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.489+0000 7f0ee9abc700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee419d2f0 con 0x7f0ee4105bd0 2024-04-23T02:44:19.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.489+0000 7f0ee9abc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee4196da0 con 0x7f0ee419cd20 2024-04-23T02:44:19.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.489+0000 7f0ee37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ee4105bd0 0x7f0ee419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:19.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.489+0000 7f0ee37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ee4105bd0 0x7f0ee419c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44992/0 (socket says 172.21.15.71:44992) 2024-04-23T02:44:19.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.489+0000 7f0ee37fe700 1 -- 172.21.15.71:0/1464248016 learned_addr learned my addr 172.21.15.71:0/1464248016 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:44:19.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.489+0000 7f0ee2ffd700 1 --2- 172.21.15.71:0/1464248016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ee419cd20 0x7f0ee4196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:19.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.490+0000 7f0ee37fe700 1 -- 172.21.15.71:0/1464248016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ee419cd20 msgr2=0x7f0ee4196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:19.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.490+0000 7f0ee37fe700 1 --2- 172.21.15.71:0/1464248016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ee419cd20 0x7f0ee4196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:19.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.490+0000 7f0ee37fe700 1 -- 172.21.15.71:0/1464248016 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed40095f0 con 0x7f0ee4105bd0 2024-04-23T02:44:19.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.490+0000 7f0ee37fe700 1 --2- 172.21.15.71:0/1464248016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ee4105bd0 0x7f0ee419c7e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800d8d0 tx=0x7f0ed800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:19.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.490+0000 7f0ee0ff9700 1 -- 172.21.15.71:0/1464248016 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ed8009940 con 0x7f0ee4105bd0 2024-04-23T02:44:19.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.490+0000 7f0ee0ff9700 1 -- 172.21.15.71:0/1464248016 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0ed8010460 con 0x7f0ee4105bd0 2024-04-23T02:44:19.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.490+0000 7f0ee9abc700 1 -- 172.21.15.71:0/1464248016 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee4197080 con 0x7f0ee4105bd0 2024-04-23T02:44:19.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.490+0000 7f0ee0ff9700 1 -- 172.21.15.71:0/1464248016 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ed800f5f0 con 0x7f0ee4105bd0 2024-04-23T02:44:19.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.490+0000 7f0ee9abc700 1 -- 172.21.15.71:0/1464248016 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee41975d0 con 0x7f0ee4105bd0 2024-04-23T02:44:19.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.491+0000 7f0ee9abc700 1 -- 172.21.15.71:0/1464248016 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee404f2e0 con 0x7f0ee4105bd0 2024-04-23T02:44:19.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.492+0000 7f0ee0ff9700 1 -- 172.21.15.71:0/1464248016 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0ed80105d0 con 0x7f0ee4105bd0 2024-04-23T02:44:19.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.492+0000 7f0ee0ff9700 1 --2- 172.21.15.71:0/1464248016 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ecc06cce0 0x7f0ecc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:19.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.492+0000 7f0ee0ff9700 1 -- 172.21.15.71:0/1464248016 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0ed808c4a0 con 0x7f0ee4105bd0 2024-04-23T02:44:19.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.492+0000 7f0ee2ffd700 1 --2- 172.21.15.71:0/1464248016 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ecc06cce0 0x7f0ecc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:19.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.493+0000 7f0ee2ffd700 1 --2- 172.21.15.71:0/1464248016 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ecc06cce0 0x7f0ecc06f1a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4009fd0 tx=0x7f0ed401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:19.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.496+0000 7f0ee0ff9700 1 -- 172.21.15.71:0/1464248016 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0ed805a170 con 0x7f0ee4105bd0 2024-04-23T02:44:19.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.711+0000 7f0ee9abc700 1 -- 172.21.15.71:0/1464248016 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ee4104990 con 0x7f0ecc06cce0 2024-04-23T02:44:19.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.712+0000 7f0ee0ff9700 1 -- 172.21.15.71:0/1464248016 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0ee4104990 con 0x7f0ecc06cce0 2024-04-23T02:44:19.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.714+0000 7f0ee9abc700 1 -- 172.21.15.71:0/1464248016 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ecc06cce0 msgr2=0x7f0ecc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:19.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.714+0000 7f0ee9abc700 1 --2- 172.21.15.71:0/1464248016 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ecc06cce0 0x7f0ecc06f1a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4009fd0 tx=0x7f0ed401a040 comp rx=0 tx=0).stop 2024-04-23T02:44:19.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.714+0000 7f0ee9abc700 1 -- 172.21.15.71:0/1464248016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ee4105bd0 msgr2=0x7f0ee419c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:19.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.714+0000 7f0ee9abc700 1 --2- 172.21.15.71:0/1464248016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ee4105bd0 0x7f0ee419c7e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800d8d0 tx=0x7f0ed800dc90 comp rx=0 tx=0).stop 2024-04-23T02:44:19.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.714+0000 7f0ee9abc700 1 -- 172.21.15.71:0/1464248016 shutdown_connections 2024-04-23T02:44:19.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.714+0000 7f0ee9abc700 1 --2- 172.21.15.71:0/1464248016 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ecc06cce0 0x7f0ecc06f1a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:19.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.714+0000 7f0ee9abc700 1 --2- 172.21.15.71:0/1464248016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ee4105bd0 0x7f0ee419c7e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:19.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.714+0000 7f0ee9abc700 1 --2- 172.21.15.71:0/1464248016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ee419cd20 0x7f0ee4196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:19.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.714+0000 7f0ee9abc700 1 -- 172.21.15.71:0/1464248016 >> 172.21.15.71:0/1464248016 conn(0x7f0ee4075670 msgr2=0x7f0ee4068880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:19.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.714+0000 7f0ee9abc700 1 -- 172.21.15.71:0/1464248016 shutdown_connections 2024-04-23T02:44:19.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.714+0000 7f0ee9abc700 1 -- 172.21.15.71:0/1464248016 wait complete. 2024-04-23T02:44:19.727 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:44:19.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.819+0000 7f6f3453f700 1 -- 172.21.15.71:0/4210729902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f2c101850 msgr2=0x7f6f2c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:19.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.819+0000 7f6f3453f700 1 --2- 172.21.15.71:0/4210729902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f2c101850 0x7f6f2c1058a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009910 tx=0x7f6f20009c20 comp rx=0 tx=0).stop 2024-04-23T02:44:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.819+0000 7f6f3453f700 1 -- 172.21.15.71:0/4210729902 shutdown_connections 2024-04-23T02:44:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.819+0000 7f6f3453f700 1 --2- 172.21.15.71:0/4210729902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f2c101850 0x7f6f2c1058a0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.819+0000 7f6f3453f700 1 --2- 172.21.15.71:0/4210729902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f2c100ea0 0x7f6f2c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.819+0000 7f6f3453f700 1 -- 172.21.15.71:0/4210729902 >> 172.21.15.71:0/4210729902 conn(0x7f6f2c0fc750 msgr2=0x7f6f2c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.819+0000 7f6f3453f700 1 -- 172.21.15.71:0/4210729902 shutdown_connections 2024-04-23T02:44:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.819+0000 7f6f3453f700 1 -- 172.21.15.71:0/4210729902 wait complete. 2024-04-23T02:44:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.820+0000 7f6f3453f700 1 Processor -- start 2024-04-23T02:44:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.820+0000 7f6f3453f700 1 -- start start 2024-04-23T02:44:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.820+0000 7f6f3453f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f2c100ea0 0x7f6f2c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.820+0000 7f6f3453f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f2c101850 0x7f6f2c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:19.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.820+0000 7f6f3453f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c073b10 con 0x7f6f2c101850 2024-04-23T02:44:19.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.820+0000 7f6f3453f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c073c50 con 0x7f6f2c100ea0 2024-04-23T02:44:19.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.820+0000 7f6f31ada700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f2c101850 0x7f6f2c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:19.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.820+0000 7f6f31ada700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f2c101850 0x7f6f2c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45004/0 (socket says 172.21.15.71:45004) 2024-04-23T02:44:19.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.820+0000 7f6f31ada700 1 -- 172.21.15.71:0/402192627 learned_addr learned my addr 172.21.15.71:0/402192627 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:44:19.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.820+0000 7f6f31ada700 1 -- 172.21.15.71:0/402192627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f2c100ea0 msgr2=0x7f6f2c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:19.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.820+0000 7f6f322db700 1 --2- 172.21.15.71:0/402192627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f2c100ea0 0x7f6f2c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:19.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.820+0000 7f6f31ada700 1 --2- 172.21.15.71:0/402192627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f2c100ea0 0x7f6f2c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:19.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.820+0000 7f6f31ada700 1 -- 172.21.15.71:0/402192627 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f200095f0 con 0x7f6f2c101850 2024-04-23T02:44:19.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.820+0000 7f6f322db700 1 --2- 172.21.15.71:0/402192627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f2c100ea0 0x7f6f2c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:44:19.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.820+0000 7f6f31ada700 1 --2- 172.21.15.71:0/402192627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f2c101850 0x7f6f2c0735d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6f200092e0 tx=0x7f6f2000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:19.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.821+0000 7f6f2b7fe700 1 -- 172.21.15.71:0/402192627 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f2001d070 con 0x7f6f2c101850 2024-04-23T02:44:19.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.821+0000 7f6f2b7fe700 1 -- 172.21.15.71:0/402192627 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6f2000f460 con 0x7f6f2c101850 2024-04-23T02:44:19.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.821+0000 7f6f3453f700 1 -- 172.21.15.71:0/402192627 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2c073ed0 con 0x7f6f2c101850 2024-04-23T02:44:19.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.821+0000 7f6f2b7fe700 1 -- 172.21.15.71:0/402192627 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f20021600 con 0x7f6f2c101850 2024-04-23T02:44:19.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.821+0000 7f6f3453f700 1 -- 172.21.15.71:0/402192627 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2c1a2e10 con 0x7f6f2c101850 2024-04-23T02:44:19.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.822+0000 7f6f3453f700 1 -- 172.21.15.71:0/402192627 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f2c1091f0 con 0x7f6f2c101850 2024-04-23T02:44:19.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.822+0000 7f6f2b7fe700 1 -- 172.21.15.71:0/402192627 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6f2000f5d0 con 0x7f6f2c101850 2024-04-23T02:44:19.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.822+0000 7f6f2b7fe700 1 --2- 172.21.15.71:0/402192627 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6f1406cce0 0x7f6f1406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:19.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.823+0000 7f6f2b7fe700 1 -- 172.21.15.71:0/402192627 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6f2008d860 con 0x7f6f2c101850 2024-04-23T02:44:19.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.823+0000 7f6f322db700 1 --2- 172.21.15.71:0/402192627 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6f1406cce0 0x7f6f1406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:19.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.823+0000 7f6f322db700 1 --2- 172.21.15.71:0/402192627 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6f1406cce0 0x7f6f1406f1a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c007eb0 tx=0x7f6f1c0073c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:19.827+0000 7f6f2b7fe700 1 -- 172.21.15.71:0/402192627 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6f200578d0 con 0x7f6f2c101850 2024-04-23T02:44:19.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:19 smithi192 ceph-mon[108617]: pgmap v931: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:19.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:19 smithi071 ceph-mon[144544]: pgmap v931: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:20.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.040+0000 7f6f3453f700 1 -- 172.21.15.71:0/402192627 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f2c0611d0 con 0x7f6f1406cce0 2024-04-23T02:44:20.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.041+0000 7f6f2b7fe700 1 -- 172.21.15.71:0/402192627 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6f2c0611d0 con 0x7f6f1406cce0 2024-04-23T02:44:20.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.043+0000 7f6f3453f700 1 -- 172.21.15.71:0/402192627 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6f1406cce0 msgr2=0x7f6f1406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:20.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.043+0000 7f6f3453f700 1 --2- 172.21.15.71:0/402192627 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6f1406cce0 0x7f6f1406f1a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c007eb0 tx=0x7f6f1c0073c0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.043+0000 7f6f3453f700 1 -- 172.21.15.71:0/402192627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f2c101850 msgr2=0x7f6f2c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:20.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.043+0000 7f6f3453f700 1 --2- 172.21.15.71:0/402192627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f2c101850 0x7f6f2c0735d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6f200092e0 tx=0x7f6f2000be20 comp rx=0 tx=0).stop 2024-04-23T02:44:20.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.043+0000 7f6f3453f700 1 -- 172.21.15.71:0/402192627 shutdown_connections 2024-04-23T02:44:20.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.043+0000 7f6f3453f700 1 --2- 172.21.15.71:0/402192627 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6f1406cce0 0x7f6f1406f1a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.043+0000 7f6f3453f700 1 --2- 172.21.15.71:0/402192627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f2c100ea0 0x7f6f2c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.043+0000 7f6f3453f700 1 --2- 172.21.15.71:0/402192627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f2c101850 0x7f6f2c0735d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.043+0000 7f6f3453f700 1 -- 172.21.15.71:0/402192627 >> 172.21.15.71:0/402192627 conn(0x7f6f2c0fc750 msgr2=0x7f6f2c0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:20.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.043+0000 7f6f3453f700 1 -- 172.21.15.71:0/402192627 shutdown_connections 2024-04-23T02:44:20.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.043+0000 7f6f3453f700 1 -- 172.21.15.71:0/402192627 wait complete. 2024-04-23T02:44:20.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.147+0000 7f0ca439e700 1 -- 172.21.15.71:0/2059545319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c9c0ffc20 msgr2=0x7f0c9c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:20.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.147+0000 7f0ca439e700 1 --2- 172.21.15.71:0/2059545319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c9c0ffc20 0x7f0c9c100080 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009910 tx=0x7f0c98009c20 comp rx=0 tx=0).stop 2024-04-23T02:44:20.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.147+0000 7f0ca439e700 1 -- 172.21.15.71:0/2059545319 shutdown_connections 2024-04-23T02:44:20.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.147+0000 7f0ca439e700 1 --2- 172.21.15.71:0/2059545319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c9c0ffc20 0x7f0c9c100080 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.147+0000 7f0ca439e700 1 --2- 172.21.15.71:0/2059545319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c9c105bd0 0x7f0c9c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.147+0000 7f0ca439e700 1 -- 172.21.15.71:0/2059545319 >> 172.21.15.71:0/2059545319 conn(0x7f0c9c075670 msgr2=0x7f0c9c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:20.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.147+0000 7f0ca439e700 1 -- 172.21.15.71:0/2059545319 shutdown_connections 2024-04-23T02:44:20.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.147+0000 7f0ca439e700 1 -- 172.21.15.71:0/2059545319 wait complete. 2024-04-23T02:44:20.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.148+0000 7f0ca439e700 1 Processor -- start 2024-04-23T02:44:20.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.148+0000 7f0ca439e700 1 -- start start 2024-04-23T02:44:20.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.148+0000 7f0ca439e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c9c0ffc20 0x7f0c9c1976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:20.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.148+0000 7f0ca439e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c9c105bd0 0x7f0c9c197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:20.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.148+0000 7f0ca439e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c198220 con 0x7f0c9c0ffc20 2024-04-23T02:44:20.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.148+0000 7f0ca439e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c198360 con 0x7f0c9c105bd0 2024-04-23T02:44:20.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.148+0000 7f0ca213a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c9c0ffc20 0x7f0c9c1976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:20.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.148+0000 7f0ca213a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c9c0ffc20 0x7f0c9c1976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45030/0 (socket says 172.21.15.71:45030) 2024-04-23T02:44:20.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.148+0000 7f0ca213a700 1 -- 172.21.15.71:0/1679792358 learned_addr learned my addr 172.21.15.71:0/1679792358 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:44:20.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.148+0000 7f0ca1939700 1 --2- 172.21.15.71:0/1679792358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c9c105bd0 0x7f0c9c197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:20.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.148+0000 7f0ca213a700 1 -- 172.21.15.71:0/1679792358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c9c105bd0 msgr2=0x7f0c9c197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:20.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.148+0000 7f0ca213a700 1 --2- 172.21.15.71:0/1679792358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c9c105bd0 0x7f0c9c197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.148+0000 7f0ca213a700 1 -- 172.21.15.71:0/1679792358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c980095f0 con 0x7f0c9c0ffc20 2024-04-23T02:44:20.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.148+0000 7f0ca213a700 1 --2- 172.21.15.71:0/1679792358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c9c0ffc20 0x7f0c9c1976c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000c8a0 tx=0x7f0c9000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:20.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.149+0000 7f0c8f7fe700 1 -- 172.21.15.71:0/1679792358 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c9000cea0 con 0x7f0c9c0ffc20 2024-04-23T02:44:20.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.149+0000 7f0ca439e700 1 -- 172.21.15.71:0/1679792358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9c1a0730 con 0x7f0c9c0ffc20 2024-04-23T02:44:20.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.149+0000 7f0c8f7fe700 1 -- 172.21.15.71:0/1679792358 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c90004830 con 0x7f0c9c0ffc20 2024-04-23T02:44:20.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.149+0000 7f0c8f7fe700 1 -- 172.21.15.71:0/1679792358 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c90005500 con 0x7f0c9c0ffc20 2024-04-23T02:44:20.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.149+0000 7f0ca439e700 1 -- 172.21.15.71:0/1679792358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9c1a0bf0 con 0x7f0c9c0ffc20 2024-04-23T02:44:20.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.150+0000 7f0ca439e700 1 -- 172.21.15.71:0/1679792358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9c1080d0 con 0x7f0c9c0ffc20 2024-04-23T02:44:20.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.150+0000 7f0c8f7fe700 1 -- 172.21.15.71:0/1679792358 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0c900049a0 con 0x7f0c9c0ffc20 2024-04-23T02:44:20.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.150+0000 7f0c8f7fe700 1 --2- 172.21.15.71:0/1679792358 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0c8806cce0 0x7f0c8806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:20.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.151+0000 7f0c8f7fe700 1 -- 172.21.15.71:0/1679792358 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0c9008c2b0 con 0x7f0c9c0ffc20 2024-04-23T02:44:20.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.151+0000 7f0ca1939700 1 --2- 172.21.15.71:0/1679792358 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0c8806cce0 0x7f0c8806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:20.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.151+0000 7f0ca1939700 1 --2- 172.21.15.71:0/1679792358 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0c8806cce0 0x7f0c8806f1a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c068e30 tx=0x7f0c9801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:20.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.155+0000 7f0c8f7fe700 1 -- 172.21.15.71:0/1679792358 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0c90056320 con 0x7f0c9c0ffc20 2024-04-23T02:44:20.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.355+0000 7f0ca439e700 1 -- 172.21.15.71:0/1679792358 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c9c0611d0 con 0x7f0c8806cce0 2024-04-23T02:44:20.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.365+0000 7f0c8f7fe700 1 -- 172.21.15.71:0/1679792358 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0c9c0611d0 con 0x7f0c8806cce0 2024-04-23T02:44:20.368 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:44:20.368 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (28m) 6m ago 39m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:44:20.368 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (39m) 6m ago 39m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:44:20.368 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (38m) 6m ago 38m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:44:20.368 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (25m) 6m ago 39m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:44:20.368 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (25m) 6m ago 38m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:44:20.368 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (27m) 6m ago 39m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:44:20.368 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (22m) 6m ago 33m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:44:20.369 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (21m) 6m ago 33m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:44:20.369 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (6m) 6m ago 33m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:44:20.369 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (21m) 6m ago 33m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:44:20.369 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (29m) 6m ago 40m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:44:20.369 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (29m) 6m ago 38m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:44:20.369 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (26m) 6m ago 41m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:44:20.369 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (26m) 6m ago 38m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:44:20.369 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (28m) 6m ago 39m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:44:20.369 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (28m) 6m ago 38m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:44:20.369 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (25m) 6m ago 37m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:44:20.369 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (24m) 6m ago 36m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:44:20.369 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (24m) 6m ago 36m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:44:20.369 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (23m) 6m ago 35m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:44:20.369 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (23m) 6m ago 35m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:44:20.369 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (22m) 6m ago 34m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:44:20.370 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (28m) 6m ago 38m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:44:20.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.367+0000 7f0ca439e700 1 -- 172.21.15.71:0/1679792358 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0c8806cce0 msgr2=0x7f0c8806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:20.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.367+0000 7f0ca439e700 1 --2- 172.21.15.71:0/1679792358 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0c8806cce0 0x7f0c8806f1a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c068e30 tx=0x7f0c9801a040 comp rx=0 tx=0).stop 2024-04-23T02:44:20.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.367+0000 7f0ca439e700 1 -- 172.21.15.71:0/1679792358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c9c0ffc20 msgr2=0x7f0c9c1976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:20.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.367+0000 7f0ca439e700 1 --2- 172.21.15.71:0/1679792358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c9c0ffc20 0x7f0c9c1976c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000c8a0 tx=0x7f0c9000cc60 comp rx=0 tx=0).stop 2024-04-23T02:44:20.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.367+0000 7f0ca439e700 1 -- 172.21.15.71:0/1679792358 shutdown_connections 2024-04-23T02:44:20.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.367+0000 7f0ca439e700 1 --2- 172.21.15.71:0/1679792358 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0c8806cce0 0x7f0c8806f1a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.367+0000 7f0ca439e700 1 --2- 172.21.15.71:0/1679792358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c9c0ffc20 0x7f0c9c1976c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.367+0000 7f0ca439e700 1 --2- 172.21.15.71:0/1679792358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c9c105bd0 0x7f0c9c197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.367+0000 7f0ca439e700 1 -- 172.21.15.71:0/1679792358 >> 172.21.15.71:0/1679792358 conn(0x7f0c9c075670 msgr2=0x7f0c9c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:20.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.367+0000 7f0ca439e700 1 -- 172.21.15.71:0/1679792358 shutdown_connections 2024-04-23T02:44:20.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.368+0000 7f0ca439e700 1 -- 172.21.15.71:0/1679792358 wait complete. 2024-04-23T02:44:20.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.468+0000 7faaac995700 1 -- 172.21.15.71:0/129149586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaa8106cd0 msgr2=0x7faaa81070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.468+0000 7faaac995700 1 --2- 172.21.15.71:0/129149586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaa8106cd0 0x7faaa81070b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009910 tx=0x7faa9c009c20 comp rx=0 tx=0).stop 2024-04-23T02:44:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.468+0000 7faaac995700 1 -- 172.21.15.71:0/129149586 shutdown_connections 2024-04-23T02:44:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.468+0000 7faaac995700 1 --2- 172.21.15.71:0/129149586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faaa8068580 0x7faaa80689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.468+0000 7faaac995700 1 --2- 172.21.15.71:0/129149586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaa8106cd0 0x7faaa81070b0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.468+0000 7faaac995700 1 -- 172.21.15.71:0/129149586 >> 172.21.15.71:0/129149586 conn(0x7faaa8075670 msgr2=0x7faaa8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.468+0000 7faaac995700 1 -- 172.21.15.71:0/129149586 shutdown_connections 2024-04-23T02:44:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.468+0000 7faaac995700 1 -- 172.21.15.71:0/129149586 wait complete. 2024-04-23T02:44:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.469+0000 7faaac995700 1 Processor -- start 2024-04-23T02:44:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.469+0000 7faaac995700 1 -- start start 2024-04-23T02:44:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.469+0000 7faaac995700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faaa8068580 0x7faaa80ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.469+0000 7faaac995700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaa8106cd0 0x7faaa8100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.469+0000 7faaac995700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa8100770 con 0x7faaa8106cd0 2024-04-23T02:44:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.469+0000 7faaac995700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa81008b0 con 0x7faaa8068580 2024-04-23T02:44:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.469+0000 7faaa5d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaa8106cd0 0x7faaa8100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.469+0000 7faaa5d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaa8106cd0 0x7faaa8100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45048/0 (socket says 172.21.15.71:45048) 2024-04-23T02:44:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.469+0000 7faaa5d9b700 1 -- 172.21.15.71:0/2114910472 learned_addr learned my addr 172.21.15.71:0/2114910472 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:44:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.469+0000 7faaa659c700 1 --2- 172.21.15.71:0/2114910472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faaa8068580 0x7faaa80ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.469+0000 7faaa5d9b700 1 -- 172.21.15.71:0/2114910472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faaa8068580 msgr2=0x7faaa80ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.469+0000 7faaa5d9b700 1 --2- 172.21.15.71:0/2114910472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faaa8068580 0x7faaa80ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.469+0000 7faaa5d9b700 1 -- 172.21.15.71:0/2114910472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9c0095f0 con 0x7faaa8106cd0 2024-04-23T02:44:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.469+0000 7faaa5d9b700 1 --2- 172.21.15.71:0/2114910472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaa8106cd0 0x7faaa8100230 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7faa9800d8d0 tx=0x7faa9800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.469+0000 7faa977fe700 1 -- 172.21.15.71:0/2114910472 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa98009940 con 0x7faaa8106cd0 2024-04-23T02:44:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.469+0000 7faa977fe700 1 -- 172.21.15.71:0/2114910472 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faa98010460 con 0x7faaa8106cd0 2024-04-23T02:44:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.470+0000 7faaac995700 1 -- 172.21.15.71:0/2114910472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa81a29b0 con 0x7faaa8106cd0 2024-04-23T02:44:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.470+0000 7faa977fe700 1 -- 172.21.15.71:0/2114910472 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa9800f5f0 con 0x7faaa8106cd0 2024-04-23T02:44:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.470+0000 7faaac995700 1 -- 172.21.15.71:0/2114910472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa81a2d60 con 0x7faaa8106cd0 2024-04-23T02:44:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.470+0000 7faaac995700 1 -- 172.21.15.71:0/2114910472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa81091d0 con 0x7faaa8106cd0 2024-04-23T02:44:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.471+0000 7faa977fe700 1 -- 172.21.15.71:0/2114910472 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7faa980105d0 con 0x7faaa8106cd0 2024-04-23T02:44:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.471+0000 7faa977fe700 1 --2- 172.21.15.71:0/2114910472 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7faa9006cce0 0x7faa9006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.471+0000 7faa977fe700 1 -- 172.21.15.71:0/2114910472 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7faa9808c630 con 0x7faaa8106cd0 2024-04-23T02:44:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.471+0000 7faaa659c700 1 --2- 172.21.15.71:0/2114910472 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7faa9006cce0 0x7faa9006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:20.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.472+0000 7faaa659c700 1 --2- 172.21.15.71:0/2114910472 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7faa9006cce0 0x7faa9006f1a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faa9c00b5c0 tx=0x7faa9c005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:20.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.476+0000 7faa977fe700 1 -- 172.21.15.71:0/2114910472 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7faa98056600 con 0x7faaa8106cd0 2024-04-23T02:44:20.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.746+0000 7faaac995700 1 -- 172.21.15.71:0/2114910472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faaa8100b60 con 0x7faaa8106cd0 2024-04-23T02:44:20.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.747+0000 7faa977fe700 1 -- 172.21.15.71:0/2114910472 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7faaa8100b60 con 0x7faaa8106cd0 2024-04-23T02:44:20.750 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:44:20.750 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:44:20.750 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:44:20.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:44:20.751 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:44:20.751 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:44:20.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:44:20.751 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:44:20.751 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:44:20.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:44:20.751 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:44:20.751 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:44:20.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:44:20.751 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:44:20.751 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:44:20.751 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:44:20.751 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:44:20.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.749+0000 7faaac995700 1 -- 172.21.15.71:0/2114910472 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7faa9006cce0 msgr2=0x7faa9006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:20.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.749+0000 7faaac995700 1 --2- 172.21.15.71:0/2114910472 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7faa9006cce0 0x7faa9006f1a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faa9c00b5c0 tx=0x7faa9c005fd0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.749+0000 7faaac995700 1 -- 172.21.15.71:0/2114910472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaa8106cd0 msgr2=0x7faaa8100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:20.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.749+0000 7faaac995700 1 --2- 172.21.15.71:0/2114910472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaa8106cd0 0x7faaa8100230 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7faa9800d8d0 tx=0x7faa9800dc90 comp rx=0 tx=0).stop 2024-04-23T02:44:20.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.749+0000 7faaac995700 1 -- 172.21.15.71:0/2114910472 shutdown_connections 2024-04-23T02:44:20.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.749+0000 7faaac995700 1 --2- 172.21.15.71:0/2114910472 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7faa9006cce0 0x7faa9006f1a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.749+0000 7faaac995700 1 --2- 172.21.15.71:0/2114910472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faaa8068580 0x7faaa80ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.749+0000 7faaac995700 1 --2- 172.21.15.71:0/2114910472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faaa8106cd0 0x7faaa8100230 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.749+0000 7faaac995700 1 -- 172.21.15.71:0/2114910472 >> 172.21.15.71:0/2114910472 conn(0x7faaa8075670 msgr2=0x7faaa80fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:20.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.749+0000 7faaac995700 1 -- 172.21.15.71:0/2114910472 shutdown_connections 2024-04-23T02:44:20.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.749+0000 7faaac995700 1 -- 172.21.15.71:0/2114910472 wait complete. 2024-04-23T02:44:20.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.850+0000 7f0d32456700 1 -- 172.21.15.71:0/857958448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c105bf0 msgr2=0x7f0d2c105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:20.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.850+0000 7f0d32456700 1 --2- 172.21.15.71:0/857958448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c105bf0 0x7f0d2c105fd0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009910 tx=0x7f0d1c009c20 comp rx=0 tx=0).stop 2024-04-23T02:44:20.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.850+0000 7f0d32456700 1 -- 172.21.15.71:0/857958448 shutdown_connections 2024-04-23T02:44:20.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.850+0000 7f0d32456700 1 --2- 172.21.15.71:0/857958448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d2c0ffb40 0x7f0d2c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.850+0000 7f0d32456700 1 --2- 172.21.15.71:0/857958448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c105bf0 0x7f0d2c105fd0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.850+0000 7f0d32456700 1 -- 172.21.15.71:0/857958448 >> 172.21.15.71:0/857958448 conn(0x7f0d2c0fb610 msgr2=0x7f0d2c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:20.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.851+0000 7f0d32456700 1 -- 172.21.15.71:0/857958448 shutdown_connections 2024-04-23T02:44:20.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.851+0000 7f0d32456700 1 -- 172.21.15.71:0/857958448 wait complete. 2024-04-23T02:44:20.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.851+0000 7f0d32456700 1 Processor -- start 2024-04-23T02:44:20.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.851+0000 7f0d32456700 1 -- start start 2024-04-23T02:44:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.851+0000 7f0d32456700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c0ffb40 0x7f0d2c195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.851+0000 7f0d32456700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d2c105bf0 0x7f0d2c195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.852+0000 7f0d32456700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c196260 con 0x7f0d2c0ffb40 2024-04-23T02:44:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.852+0000 7f0d32456700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c199ff0 con 0x7f0d2c105bf0 2024-04-23T02:44:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.852+0000 7f0d2bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c0ffb40 0x7f0d2c195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.852+0000 7f0d2bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c0ffb40 0x7f0d2c195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45062/0 (socket says 172.21.15.71:45062) 2024-04-23T02:44:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.852+0000 7f0d2bfff700 1 -- 172.21.15.71:0/42477268 learned_addr learned my addr 172.21.15.71:0/42477268 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:44:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.852+0000 7f0d2b7fe700 1 --2- 172.21.15.71:0/42477268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d2c105bf0 0x7f0d2c195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.852+0000 7f0d2bfff700 1 -- 172.21.15.71:0/42477268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d2c105bf0 msgr2=0x7f0d2c195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.852+0000 7f0d2bfff700 1 --2- 172.21.15.71:0/42477268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d2c105bf0 0x7f0d2c195b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.852+0000 7f0d2bfff700 1 -- 172.21.15.71:0/42477268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c0095f0 con 0x7f0d2c0ffb40 2024-04-23T02:44:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.852+0000 7f0d2bfff700 1 --2- 172.21.15.71:0/42477268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c0ffb40 0x7f0d2c195640 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c006010 tx=0x7f0d1c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.852+0000 7f0d297fa700 1 -- 172.21.15.71:0/42477268 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d1c01d070 con 0x7f0d2c0ffb40 2024-04-23T02:44:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.852+0000 7f0d32456700 1 -- 172.21.15.71:0/42477268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d2c19a270 con 0x7f0d2c0ffb40 2024-04-23T02:44:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.852+0000 7f0d297fa700 1 -- 172.21.15.71:0/42477268 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0d1c00f460 con 0x7f0d2c0ffb40 2024-04-23T02:44:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.852+0000 7f0d297fa700 1 -- 172.21.15.71:0/42477268 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d1c021620 con 0x7f0d2c0ffb40 2024-04-23T02:44:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.852+0000 7f0d32456700 1 -- 172.21.15.71:0/42477268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d2c19a760 con 0x7f0d2c0ffb40 2024-04-23T02:44:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.853+0000 7f0d32456700 1 -- 172.21.15.71:0/42477268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2c108020 con 0x7f0d2c0ffb40 2024-04-23T02:44:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.854+0000 7f0d297fa700 1 -- 172.21.15.71:0/42477268 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0d1c02b440 con 0x7f0d2c0ffb40 2024-04-23T02:44:20.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.854+0000 7f0d297fa700 1 --2- 172.21.15.71:0/42477268 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0d1406cce0 0x7f0d1406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:20.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.854+0000 7f0d297fa700 1 -- 172.21.15.71:0/42477268 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0d1c08d7f0 con 0x7f0d2c0ffb40 2024-04-23T02:44:20.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.854+0000 7f0d2b7fe700 1 --2- 172.21.15.71:0/42477268 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0d1406cce0 0x7f0d1406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:20.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.854+0000 7f0d2b7fe700 1 --2- 172.21.15.71:0/42477268 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0d1406cce0 0x7f0d1406f1a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0d20005950 tx=0x7f0d200058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:20.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:20.858+0000 7f0d297fa700 1 -- 172.21.15.71:0/42477268 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0d1c0578d0 con 0x7f0d2c0ffb40 2024-04-23T02:44:20.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:44:20.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:44:21.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.094+0000 7f0d32456700 1 -- 172.21.15.71:0/42477268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0d2c02cc70 con 0x7f0d2c0ffb40 2024-04-23T02:44:21.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.094+0000 7f0d297fa700 1 -- 172.21.15.71:0/42477268 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f0d1c05aef0 con 0x7f0d2c0ffb40 2024-04-23T02:44:21.097 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:44:21.097 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:44:21.097 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:44:21.097 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:44:21.097 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:44:21.097 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:44:21.097 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:44:21.097 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:44:21.097 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:44:21.097 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:44:21.098 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:44:21.098 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:44:21.098 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:44:21.098 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:44:21.098 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:44:21.098 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:44:21.098 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:44:21.098 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:44:21.098 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:44:21.098 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:44:21.098 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:44:21.098 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:44:21.098 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:44:21.099 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:44:21.099 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:44:21.099 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:44:21.099 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:44:21.099 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:44:21.099 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:44:21.099 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:44:21.099 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:44:21.099 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:44:21.099 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:44:21.099 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:44:21.099 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:44:21.099 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:44:21.099 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:44:21.099 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:44:21.100 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:44:21.100 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:44:21.100 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:44:21.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.096+0000 7f0d32456700 1 -- 172.21.15.71:0/42477268 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0d1406cce0 msgr2=0x7f0d1406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:21.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.096+0000 7f0d32456700 1 --2- 172.21.15.71:0/42477268 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0d1406cce0 0x7f0d1406f1a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0d20005950 tx=0x7f0d200058e0 comp rx=0 tx=0).stop 2024-04-23T02:44:21.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.096+0000 7f0d32456700 1 -- 172.21.15.71:0/42477268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c0ffb40 msgr2=0x7f0d2c195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:21.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.096+0000 7f0d32456700 1 --2- 172.21.15.71:0/42477268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c0ffb40 0x7f0d2c195640 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c006010 tx=0x7f0d1c00be20 comp rx=0 tx=0).stop 2024-04-23T02:44:21.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.096+0000 7f0d32456700 1 -- 172.21.15.71:0/42477268 shutdown_connections 2024-04-23T02:44:21.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.096+0000 7f0d32456700 1 --2- 172.21.15.71:0/42477268 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0d1406cce0 0x7f0d1406f1a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:21.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.096+0000 7f0d32456700 1 --2- 172.21.15.71:0/42477268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c0ffb40 0x7f0d2c195640 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:21.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.096+0000 7f0d32456700 1 --2- 172.21.15.71:0/42477268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d2c105bf0 0x7f0d2c195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:21.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.096+0000 7f0d32456700 1 -- 172.21.15.71:0/42477268 >> 172.21.15.71:0/42477268 conn(0x7f0d2c0fb610 msgr2=0x7f0d2c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:21.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.096+0000 7f0d32456700 1 -- 172.21.15.71:0/42477268 shutdown_connections 2024-04-23T02:44:21.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.096+0000 7f0d32456700 1 -- 172.21.15.71:0/42477268 wait complete. 2024-04-23T02:44:21.101 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:44:21.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.199+0000 7f67a46a2700 1 -- 172.21.15.71:0/3356705546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f679c1027c0 msgr2=0x7f679c10acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:21.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.199+0000 7f67a46a2700 1 --2- 172.21.15.71:0/3356705546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f679c1027c0 0x7f679c10acb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6798009910 tx=0x7f6798009c20 comp rx=0 tx=0).stop 2024-04-23T02:44:21.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.199+0000 7f67a46a2700 1 -- 172.21.15.71:0/3356705546 shutdown_connections 2024-04-23T02:44:21.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.199+0000 7f67a46a2700 1 --2- 172.21.15.71:0/3356705546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f679c1027c0 0x7f679c10acb0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:21.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.199+0000 7f67a46a2700 1 --2- 172.21.15.71:0/3356705546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f679c101ea0 0x7f679c102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:21.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.199+0000 7f67a46a2700 1 -- 172.21.15.71:0/3356705546 >> 172.21.15.71:0/3356705546 conn(0x7f679c0fb650 msgr2=0x7f679c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:21.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.199+0000 7f67a46a2700 1 -- 172.21.15.71:0/3356705546 shutdown_connections 2024-04-23T02:44:21.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.199+0000 7f67a46a2700 1 -- 172.21.15.71:0/3356705546 wait complete. 2024-04-23T02:44:21.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.200+0000 7f67a46a2700 1 Processor -- start 2024-04-23T02:44:21.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.200+0000 7f67a46a2700 1 -- start start 2024-04-23T02:44:21.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.200+0000 7f67a46a2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f679c101ea0 0x7f679c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:21.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.200+0000 7f67a46a2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f679c1027c0 0x7f679c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:21.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.200+0000 7f67a46a2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c19f4a0 con 0x7f679c1027c0 2024-04-23T02:44:21.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.200+0000 7f67a46a2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c198950 con 0x7f679c101ea0 2024-04-23T02:44:21.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.200+0000 7f67a1c3d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f679c1027c0 0x7f679c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.200+0000 7f67a1c3d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f679c1027c0 0x7f679c19ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45082/0 (socket says 172.21.15.71:45082) 2024-04-23T02:44:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.200+0000 7f67a1c3d700 1 -- 172.21.15.71:0/3241626090 learned_addr learned my addr 172.21.15.71:0/3241626090 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:44:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.200+0000 7f67a1c3d700 1 -- 172.21.15.71:0/3241626090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f679c101ea0 msgr2=0x7f679c19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.200+0000 7f67a1c3d700 1 --2- 172.21.15.71:0/3241626090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f679c101ea0 0x7f679c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.200+0000 7f67a1c3d700 1 -- 172.21.15.71:0/3241626090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67980095f0 con 0x7f679c1027c0 2024-04-23T02:44:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.201+0000 7f67a1c3d700 1 --2- 172.21.15.71:0/3241626090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f679c1027c0 0x7f679c19ee10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f67980092e0 tx=0x7f679800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.201+0000 7f67937fe700 1 -- 172.21.15.71:0/3241626090 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f679801d070 con 0x7f679c1027c0 2024-04-23T02:44:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.201+0000 7f67a46a2700 1 -- 172.21.15.71:0/3241626090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679c198bd0 con 0x7f679c1027c0 2024-04-23T02:44:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.201+0000 7f67937fe700 1 -- 172.21.15.71:0/3241626090 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f679800f460 con 0x7f679c1027c0 2024-04-23T02:44:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.201+0000 7f67937fe700 1 -- 172.21.15.71:0/3241626090 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6798021600 con 0x7f679c1027c0 2024-04-23T02:44:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.201+0000 7f67a46a2700 1 -- 172.21.15.71:0/3241626090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679c1990c0 con 0x7f679c1027c0 2024-04-23T02:44:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.202+0000 7f67a46a2700 1 -- 172.21.15.71:0/3241626090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f679c10a7a0 con 0x7f679c1027c0 2024-04-23T02:44:21.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.202+0000 7f67937fe700 1 -- 172.21.15.71:0/3241626090 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f679800f5d0 con 0x7f679c1027c0 2024-04-23T02:44:21.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.203+0000 7f67937fe700 1 --2- 172.21.15.71:0/3241626090 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f678806cce0 0x7f678806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:21.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.203+0000 7f67937fe700 1 -- 172.21.15.71:0/3241626090 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f679808d8e0 con 0x7f679c1027c0 2024-04-23T02:44:21.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.203+0000 7f67a243e700 1 --2- 172.21.15.71:0/3241626090 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f678806cce0 0x7f678806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:21.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.203+0000 7f67a243e700 1 --2- 172.21.15.71:0/3241626090 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f678806cce0 0x7f678806f1a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f678c0097b0 tx=0x7f678c006d20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:21.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.207+0000 7f67937fe700 1 -- 172.21.15.71:0/3241626090 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f679805b340 con 0x7f679c1027c0 2024-04-23T02:44:21.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.416+0000 7f67a46a2700 1 -- 172.21.15.71:0/3241626090 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f679c199ed0 con 0x7f678806cce0 2024-04-23T02:44:21.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.417+0000 7f67937fe700 1 -- 172.21.15.71:0/3241626090 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f679c199ed0 con 0x7f678806cce0 2024-04-23T02:44:21.420 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:44:21.420 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:44:21.420 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:44:21.420 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:44:21.420 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:44:21.420 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:44:21.420 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:44:21.420 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:44:21.420 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:44:21.421 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:44:21.421 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:44:21.421 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:44:21.421 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:44:21.421 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:44:21.421 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:44:21.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.419+0000 7f67a46a2700 1 -- 172.21.15.71:0/3241626090 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f678806cce0 msgr2=0x7f678806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:21.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.419+0000 7f67a46a2700 1 --2- 172.21.15.71:0/3241626090 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f678806cce0 0x7f678806f1a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f678c0097b0 tx=0x7f678c006d20 comp rx=0 tx=0).stop 2024-04-23T02:44:21.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.419+0000 7f67a46a2700 1 -- 172.21.15.71:0/3241626090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f679c1027c0 msgr2=0x7f679c19ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:21.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.419+0000 7f67a46a2700 1 --2- 172.21.15.71:0/3241626090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f679c1027c0 0x7f679c19ee10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f67980092e0 tx=0x7f679800be20 comp rx=0 tx=0).stop 2024-04-23T02:44:21.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.419+0000 7f67a46a2700 1 -- 172.21.15.71:0/3241626090 shutdown_connections 2024-04-23T02:44:21.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.419+0000 7f67a46a2700 1 --2- 172.21.15.71:0/3241626090 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f678806cce0 0x7f678806f1a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:21.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.419+0000 7f67a46a2700 1 --2- 172.21.15.71:0/3241626090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f679c101ea0 0x7f679c19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:21.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.419+0000 7f67a46a2700 1 --2- 172.21.15.71:0/3241626090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f679c1027c0 0x7f679c19ee10 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:21.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.419+0000 7f67a46a2700 1 -- 172.21.15.71:0/3241626090 >> 172.21.15.71:0/3241626090 conn(0x7f679c0fb650 msgr2=0x7f679c1002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:21.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.419+0000 7f67a46a2700 1 -- 172.21.15.71:0/3241626090 shutdown_connections 2024-04-23T02:44:21.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.419+0000 7f67a46a2700 1 -- 172.21.15.71:0/3241626090 wait complete. 2024-04-23T02:44:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.521+0000 7f60b67f9700 1 -- 172.21.15.71:0/349712937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60b00ffb40 msgr2=0x7f60b00fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.521+0000 7f60b67f9700 1 --2- 172.21.15.71:0/349712937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60b00ffb40 0x7f60b00fffc0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009910 tx=0x7f60a4009c20 comp rx=0 tx=0).stop 2024-04-23T02:44:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.521+0000 7f60b67f9700 1 -- 172.21.15.71:0/349712937 shutdown_connections 2024-04-23T02:44:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.521+0000 7f60b67f9700 1 --2- 172.21.15.71:0/349712937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60b00ffb40 0x7f60b00fffc0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.521+0000 7f60b67f9700 1 --2- 172.21.15.71:0/349712937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60b0105bf0 0x7f60b0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.521+0000 7f60b67f9700 1 -- 172.21.15.71:0/349712937 >> 172.21.15.71:0/349712937 conn(0x7f60b00fb610 msgr2=0x7f60b00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.521+0000 7f60b67f9700 1 -- 172.21.15.71:0/349712937 shutdown_connections 2024-04-23T02:44:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.521+0000 7f60b67f9700 1 -- 172.21.15.71:0/349712937 wait complete. 2024-04-23T02:44:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.521+0000 7f60b67f9700 1 Processor -- start 2024-04-23T02:44:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.521+0000 7f60b67f9700 1 -- start start 2024-04-23T02:44:21.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.522+0000 7f60b67f9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60b00ffb40 0x7f60b019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:21.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.522+0000 7f60b67f9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60b0105bf0 0x7f60b019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:21.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.522+0000 7f60b67f9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b019d0c0 con 0x7f60b00ffb40 2024-04-23T02:44:21.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.522+0000 7f60b67f9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b0196570 con 0x7f60b0105bf0 2024-04-23T02:44:21.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.522+0000 7f60affff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60b00ffb40 0x7f60b019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:21.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.522+0000 7f60affff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60b00ffb40 0x7f60b019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45108/0 (socket says 172.21.15.71:45108) 2024-04-23T02:44:21.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.522+0000 7f60affff700 1 -- 172.21.15.71:0/4137010667 learned_addr learned my addr 172.21.15.71:0/4137010667 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:44:21.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.522+0000 7f60af7fe700 1 --2- 172.21.15.71:0/4137010667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60b0105bf0 0x7f60b019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:21.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.522+0000 7f60affff700 1 -- 172.21.15.71:0/4137010667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60b0105bf0 msgr2=0x7f60b019ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:21.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.522+0000 7f60affff700 1 --2- 172.21.15.71:0/4137010667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60b0105bf0 0x7f60b019ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:21.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.522+0000 7f60affff700 1 -- 172.21.15.71:0/4137010667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a40095f0 con 0x7f60b00ffb40 2024-04-23T02:44:21.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.522+0000 7f60affff700 1 --2- 172.21.15.71:0/4137010667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60b00ffb40 0x7f60b019c4f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f60a000d8d0 tx=0x7f60a000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:21.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.522+0000 7f60ad7fa700 1 -- 172.21.15.71:0/4137010667 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60a00098a0 con 0x7f60b00ffb40 2024-04-23T02:44:21.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.523+0000 7f60ad7fa700 1 -- 172.21.15.71:0/4137010667 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f60a0010460 con 0x7f60b00ffb40 2024-04-23T02:44:21.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.523+0000 7f60b67f9700 1 -- 172.21.15.71:0/4137010667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b0196850 con 0x7f60b00ffb40 2024-04-23T02:44:21.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.523+0000 7f60ad7fa700 1 -- 172.21.15.71:0/4137010667 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60a0009ea0 con 0x7f60b00ffb40 2024-04-23T02:44:21.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.523+0000 7f60b67f9700 1 -- 172.21.15.71:0/4137010667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b0196da0 con 0x7f60b00ffb40 2024-04-23T02:44:21.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.523+0000 7f60b67f9700 1 -- 172.21.15.71:0/4137010667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b004f2e0 con 0x7f60b00ffb40 2024-04-23T02:44:21.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.524+0000 7f60ad7fa700 1 -- 172.21.15.71:0/4137010667 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f60a00105d0 con 0x7f60b00ffb40 2024-04-23T02:44:21.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.524+0000 7f60ad7fa700 1 --2- 172.21.15.71:0/4137010667 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f609806cce0 0x7f609806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:21.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.524+0000 7f60ad7fa700 1 -- 172.21.15.71:0/4137010667 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f60a008c480 con 0x7f60b00ffb40 2024-04-23T02:44:21.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.524+0000 7f60af7fe700 1 --2- 172.21.15.71:0/4137010667 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f609806cce0 0x7f609806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:21.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.525+0000 7f60af7fe700 1 --2- 172.21.15.71:0/4137010667 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f609806cce0 0x7f609806f1a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f60b00feb60 tx=0x7f60a401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:21.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.529+0000 7f60ad7fa700 1 -- 172.21.15.71:0/4137010667 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f60a0059f80 con 0x7f60b00ffb40 2024-04-23T02:44:21.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.803+0000 7f60b67f9700 1 -- 172.21.15.71:0/4137010667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f60b01970e0 con 0x7f60b00ffb40 2024-04-23T02:44:21.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.804+0000 7f60ad7fa700 1 -- 172.21.15.71:0/4137010667 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7f60a0010880 con 0x7f60b00ffb40 2024-04-23T02:44:21.807 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:44:21.807 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:44:21.807 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:44:21.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:44:21.807 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:44:21.807 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:44:21.807 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:44:21.807 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:44:21.807 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:44:21.808 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:44:21.808 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:44:21.808 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:44:21.808 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:44:21.808 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:44:21.808 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:44:21.808 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:44:21.808 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:44:21.808 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:44:21.808 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:44:21.808 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:44:21.808 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:44:21.808 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:44:21.808 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:44:21.809 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:44:21.809 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:44:21.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.806+0000 7f60b67f9700 1 -- 172.21.15.71:0/4137010667 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f609806cce0 msgr2=0x7f609806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:21.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.806+0000 7f60b67f9700 1 --2- 172.21.15.71:0/4137010667 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f609806cce0 0x7f609806f1a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f60b00feb60 tx=0x7f60a401a040 comp rx=0 tx=0).stop 2024-04-23T02:44:21.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.806+0000 7f60b67f9700 1 -- 172.21.15.71:0/4137010667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60b00ffb40 msgr2=0x7f60b019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:21.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.806+0000 7f60b67f9700 1 --2- 172.21.15.71:0/4137010667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60b00ffb40 0x7f60b019c4f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f60a000d8d0 tx=0x7f60a000dbe0 comp rx=0 tx=0).stop 2024-04-23T02:44:21.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.806+0000 7f60b67f9700 1 -- 172.21.15.71:0/4137010667 shutdown_connections 2024-04-23T02:44:21.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.806+0000 7f60b67f9700 1 --2- 172.21.15.71:0/4137010667 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f609806cce0 0x7f609806f1a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:21.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.806+0000 7f60b67f9700 1 --2- 172.21.15.71:0/4137010667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60b00ffb40 0x7f60b019c4f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:21.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.806+0000 7f60b67f9700 1 --2- 172.21.15.71:0/4137010667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60b0105bf0 0x7f60b019ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:21.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.806+0000 7f60b67f9700 1 -- 172.21.15.71:0/4137010667 >> 172.21.15.71:0/4137010667 conn(0x7f60b00fb610 msgr2=0x7f60b0068f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:21.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.806+0000 7f60b67f9700 1 -- 172.21.15.71:0/4137010667 shutdown_connections 2024-04-23T02:44:21.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:21.806+0000 7f60b67f9700 1 -- 172.21.15.71:0/4137010667 wait complete. 2024-04-23T02:44:21.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:21 smithi192 ceph-mon[108617]: from='client.35492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:44:21.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:21 smithi192 ceph-mon[108617]: from='client.35496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:44:21.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:21 smithi192 ceph-mon[108617]: from='client.35500 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:44:21.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:21 smithi192 ceph-mon[108617]: pgmap v932: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:21.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:21 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2114910472' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:44:21.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:21 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/42477268' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:44:21.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:21 smithi071 ceph-mon[144544]: from='client.35492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:44:21.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:21 smithi071 ceph-mon[144544]: from='client.35496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:44:21.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:21 smithi071 ceph-mon[144544]: from='client.35500 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:44:21.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:21 smithi071 ceph-mon[144544]: pgmap v932: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:21.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:21 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2114910472' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:44:21.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:21 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/42477268' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:44:22.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:22 smithi192 ceph-mon[108617]: from='client.35512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:44:22.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:22 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/4137010667' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:44:22.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:22 smithi071 ceph-mon[144544]: from='client.35512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:44:22.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:22 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/4137010667' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:44:23.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:23 smithi192 ceph-mon[108617]: pgmap v933: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:23.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:23 smithi071 ceph-mon[144544]: pgmap v933: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:25.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:25 smithi192 ceph-mon[108617]: pgmap v934: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:25.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:25 smithi071 ceph-mon[144544]: pgmap v934: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:27.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:27 smithi192 ceph-mon[108617]: pgmap v935: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:27.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:27 smithi071 ceph-mon[144544]: pgmap v935: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:29.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:29 smithi192 ceph-mon[108617]: pgmap v936: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:29.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:29 smithi071 ceph-mon[144544]: pgmap v936: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:31.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:31 smithi192 ceph-mon[108617]: pgmap v937: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:31.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:31 smithi071 ceph-mon[144544]: pgmap v937: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:33.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:33 smithi192 ceph-mon[108617]: pgmap v938: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:33.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:33 smithi071 ceph-mon[144544]: pgmap v938: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:35.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:35 smithi192 ceph-mon[108617]: pgmap v939: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:35.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:44:35.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:35 smithi071 ceph-mon[144544]: pgmap v939: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:44:36.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:36 smithi071 ceph-mon[144544]: pgmap v940: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:37.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:36 smithi192 ceph-mon[108617]: pgmap v940: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:39.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:39 smithi192 ceph-mon[108617]: pgmap v941: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:39.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:39 smithi071 ceph-mon[144544]: pgmap v941: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:41.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:41 smithi192 ceph-mon[108617]: pgmap v942: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:41.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:41 smithi071 ceph-mon[144544]: pgmap v942: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:43.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:43 smithi192 ceph-mon[108617]: pgmap v943: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:43.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:43 smithi071 ceph-mon[144544]: pgmap v943: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:45.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:45 smithi192 ceph-mon[108617]: pgmap v944: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:45.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:45 smithi071 ceph-mon[144544]: pgmap v944: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:47.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:47 smithi192 ceph-mon[108617]: pgmap v945: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:47 smithi071 ceph-mon[144544]: pgmap v945: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:49.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:49 smithi192 ceph-mon[108617]: pgmap v946: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:49.897 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:49 smithi071 ceph-mon[144544]: pgmap v946: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:44:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:44:51.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:51 smithi192 ceph-mon[108617]: pgmap v947: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:51.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.927+0000 7fa03f3f5700 1 -- 172.21.15.71:0/2849668497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa038068ac0 msgr2=0x7fa038068f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.927+0000 7fa03f3f5700 1 --2- 172.21.15.71:0/2849668497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa038068ac0 0x7fa038068f20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa028009910 tx=0x7fa028009c20 comp rx=0 tx=0).stop 2024-04-23T02:44:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.928+0000 7fa03f3f5700 1 -- 172.21.15.71:0/2849668497 shutdown_connections 2024-04-23T02:44:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.928+0000 7fa03f3f5700 1 --2- 172.21.15.71:0/2849668497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa038068ac0 0x7fa038068f20 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.928+0000 7fa03f3f5700 1 --2- 172.21.15.71:0/2849668497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa038105b80 0x7fa038068580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.928+0000 7fa03f3f5700 1 -- 172.21.15.71:0/2849668497 >> 172.21.15.71:0/2849668497 conn(0x7fa038075670 msgr2=0x7fa038075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.928+0000 7fa03f3f5700 1 -- 172.21.15.71:0/2849668497 shutdown_connections 2024-04-23T02:44:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.928+0000 7fa03f3f5700 1 -- 172.21.15.71:0/2849668497 wait complete. 2024-04-23T02:44:51.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.928+0000 7fa03f3f5700 1 Processor -- start 2024-04-23T02:44:51.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.928+0000 7fa03f3f5700 1 -- start start 2024-04-23T02:44:51.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.928+0000 7fa03f3f5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa038068ac0 0x7fa03819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:51.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.928+0000 7fa03f3f5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa038105b80 0x7fa03819d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:51.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.928+0000 7fa03f3f5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03819d9c0 con 0x7fa038105b80 2024-04-23T02:44:51.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.928+0000 7fa03f3f5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0381a1750 con 0x7fa038068ac0 2024-04-23T02:44:51.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.929+0000 7fa03c990700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa038105b80 0x7fa03819d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:51.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.929+0000 7fa03c990700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa038105b80 0x7fa03819d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52430/0 (socket says 172.21.15.71:52430) 2024-04-23T02:44:51.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.929+0000 7fa03c990700 1 -- 172.21.15.71:0/1674776339 learned_addr learned my addr 172.21.15.71:0/1674776339 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:44:51.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.929+0000 7fa03d191700 1 --2- 172.21.15.71:0/1674776339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa038068ac0 0x7fa03819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:51.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.929+0000 7fa03c990700 1 -- 172.21.15.71:0/1674776339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa038068ac0 msgr2=0x7fa03819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:51.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.929+0000 7fa03c990700 1 --2- 172.21.15.71:0/1674776339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa038068ac0 0x7fa03819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:51.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.929+0000 7fa03c990700 1 -- 172.21.15.71:0/1674776339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0280095f0 con 0x7fa038105b80 2024-04-23T02:44:51.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.929+0000 7fa03c990700 1 --2- 172.21.15.71:0/1674776339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa038105b80 0x7fa03819d2e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa028006010 tx=0x7fa02800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:51.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.929+0000 7fa02e7fc700 1 -- 172.21.15.71:0/1674776339 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa02801d070 con 0x7fa038105b80 2024-04-23T02:44:51.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.929+0000 7fa02e7fc700 1 -- 172.21.15.71:0/1674776339 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa028005710 con 0x7fa038105b80 2024-04-23T02:44:51.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.929+0000 7fa03f3f5700 1 -- 172.21.15.71:0/1674776339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0381a19d0 con 0x7fa038105b80 2024-04-23T02:44:51.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.929+0000 7fa02e7fc700 1 -- 172.21.15.71:0/1674776339 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa02800fe20 con 0x7fa038105b80 2024-04-23T02:44:51.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.929+0000 7fa03f3f5700 1 -- 172.21.15.71:0/1674776339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0381a1ec0 con 0x7fa038105b80 2024-04-23T02:44:51.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.930+0000 7fa03f3f5700 1 -- 172.21.15.71:0/1674776339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0381101f0 con 0x7fa038105b80 2024-04-23T02:44:51.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.931+0000 7fa02e7fc700 1 -- 172.21.15.71:0/1674776339 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa028003a60 con 0x7fa038105b80 2024-04-23T02:44:51.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.931+0000 7fa02e7fc700 1 --2- 172.21.15.71:0/1674776339 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa02406cce0 0x7fa02406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:51.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.931+0000 7fa02e7fc700 1 -- 172.21.15.71:0/1674776339 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa02808d750 con 0x7fa038105b80 2024-04-23T02:44:51.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.931+0000 7fa03d191700 1 --2- 172.21.15.71:0/1674776339 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa02406cce0 0x7fa02406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:51.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.932+0000 7fa03d191700 1 --2- 172.21.15.71:0/1674776339 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa02406cce0 0x7fa02406f1a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa034005950 tx=0x7fa03400b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:51.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:51.936+0000 7fa02e7fc700 1 -- 172.21.15.71:0/1674776339 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa0280577c0 con 0x7fa038105b80 2024-04-23T02:44:51.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:51 smithi071 ceph-mon[144544]: pgmap v947: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:52.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.148+0000 7fa03f3f5700 1 -- 172.21.15.71:0/1674776339 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa03819e1e0 con 0x7fa02406cce0 2024-04-23T02:44:52.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.149+0000 7fa02e7fc700 1 -- 172.21.15.71:0/1674776339 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa03819e1e0 con 0x7fa02406cce0 2024-04-23T02:44:52.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.151+0000 7fa03f3f5700 1 -- 172.21.15.71:0/1674776339 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa02406cce0 msgr2=0x7fa02406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:52.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.151+0000 7fa03f3f5700 1 --2- 172.21.15.71:0/1674776339 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa02406cce0 0x7fa02406f1a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa034005950 tx=0x7fa03400b410 comp rx=0 tx=0).stop 2024-04-23T02:44:52.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.151+0000 7fa03f3f5700 1 -- 172.21.15.71:0/1674776339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa038105b80 msgr2=0x7fa03819d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:52.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.151+0000 7fa03f3f5700 1 --2- 172.21.15.71:0/1674776339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa038105b80 0x7fa03819d2e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa028006010 tx=0x7fa02800bd70 comp rx=0 tx=0).stop 2024-04-23T02:44:52.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.151+0000 7fa03f3f5700 1 -- 172.21.15.71:0/1674776339 shutdown_connections 2024-04-23T02:44:52.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.151+0000 7fa03f3f5700 1 --2- 172.21.15.71:0/1674776339 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa02406cce0 0x7fa02406f1a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:52.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.151+0000 7fa03f3f5700 1 --2- 172.21.15.71:0/1674776339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa038068ac0 0x7fa03819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:52.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.151+0000 7fa03f3f5700 1 --2- 172.21.15.71:0/1674776339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa038105b80 0x7fa03819d2e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:52.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.151+0000 7fa03f3f5700 1 -- 172.21.15.71:0/1674776339 >> 172.21.15.71:0/1674776339 conn(0x7fa038075670 msgr2=0x7fa038101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:52.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.151+0000 7fa03f3f5700 1 -- 172.21.15.71:0/1674776339 shutdown_connections 2024-04-23T02:44:52.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.151+0000 7fa03f3f5700 1 -- 172.21.15.71:0/1674776339 wait complete. 2024-04-23T02:44:52.164 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:44:52.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.255+0000 7f3d2390a700 1 -- 172.21.15.71:0/87202020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d1c101e80 msgr2=0x7f3d1c102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:52.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.255+0000 7f3d2390a700 1 --2- 172.21.15.71:0/87202020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d1c101e80 0x7f3d1c102260 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009910 tx=0x7f3d18009c20 comp rx=0 tx=0).stop 2024-04-23T02:44:52.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.255+0000 7f3d2390a700 1 -- 172.21.15.71:0/87202020 shutdown_connections 2024-04-23T02:44:52.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.255+0000 7f3d2390a700 1 --2- 172.21.15.71:0/87202020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d1c1027a0 0x7f3d1c10ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:52.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.255+0000 7f3d2390a700 1 --2- 172.21.15.71:0/87202020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d1c101e80 0x7f3d1c102260 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:52.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.255+0000 7f3d2390a700 1 -- 172.21.15.71:0/87202020 >> 172.21.15.71:0/87202020 conn(0x7f3d1c0fb610 msgr2=0x7f3d1c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:52.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.255+0000 7f3d2390a700 1 -- 172.21.15.71:0/87202020 shutdown_connections 2024-04-23T02:44:52.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.255+0000 7f3d2390a700 1 -- 172.21.15.71:0/87202020 wait complete. 2024-04-23T02:44:52.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.255+0000 7f3d2390a700 1 Processor -- start 2024-04-23T02:44:52.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.255+0000 7f3d2390a700 1 -- start start 2024-04-23T02:44:52.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.256+0000 7f3d2390a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d1c101e80 0x7f3d1c19c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:52.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.256+0000 7f3d2390a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d1c1027a0 0x7f3d1c19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:52.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.256+0000 7f3d2390a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c19d0b0 con 0x7f3d1c101e80 2024-04-23T02:44:52.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.256+0000 7f3d2390a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c196640 con 0x7f3d1c1027a0 2024-04-23T02:44:52.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.256+0000 7f3d216a6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d1c101e80 0x7f3d1c19c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:52.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.256+0000 7f3d216a6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d1c101e80 0x7f3d1c19c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52452/0 (socket says 172.21.15.71:52452) 2024-04-23T02:44:52.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.256+0000 7f3d216a6700 1 -- 172.21.15.71:0/4095137696 learned_addr learned my addr 172.21.15.71:0/4095137696 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:44:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.256+0000 7f3d20ea5700 1 --2- 172.21.15.71:0/4095137696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d1c1027a0 0x7f3d1c19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.256+0000 7f3d216a6700 1 -- 172.21.15.71:0/4095137696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d1c1027a0 msgr2=0x7f3d1c19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.256+0000 7f3d216a6700 1 --2- 172.21.15.71:0/4095137696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d1c1027a0 0x7f3d1c19cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.256+0000 7f3d216a6700 1 -- 172.21.15.71:0/4095137696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d180095f0 con 0x7f3d1c101e80 2024-04-23T02:44:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.256+0000 7f3d216a6700 1 --2- 172.21.15.71:0/4095137696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d1c101e80 0x7f3d1c19c570 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3d18006010 tx=0x7f3d1800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.256+0000 7f3d127fc700 1 -- 172.21.15.71:0/4095137696 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d1801d070 con 0x7f3d1c101e80 2024-04-23T02:44:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.256+0000 7f3d127fc700 1 -- 172.21.15.71:0/4095137696 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3d1800f460 con 0x7f3d1c101e80 2024-04-23T02:44:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.257+0000 7f3d2390a700 1 -- 172.21.15.71:0/4095137696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1c1968c0 con 0x7f3d1c101e80 2024-04-23T02:44:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.257+0000 7f3d127fc700 1 -- 172.21.15.71:0/4095137696 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d18021620 con 0x7f3d1c101e80 2024-04-23T02:44:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.257+0000 7f3d2390a700 1 -- 172.21.15.71:0/4095137696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1c196db0 con 0x7f3d1c101e80 2024-04-23T02:44:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.257+0000 7f3d2390a700 1 -- 172.21.15.71:0/4095137696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1c108390 con 0x7f3d1c101e80 2024-04-23T02:44:52.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.258+0000 7f3d127fc700 1 -- 172.21.15.71:0/4095137696 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3d1800f5d0 con 0x7f3d1c101e80 2024-04-23T02:44:52.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.258+0000 7f3d127fc700 1 --2- 172.21.15.71:0/4095137696 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3d0806cce0 0x7f3d0806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:52.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.258+0000 7f3d127fc700 1 -- 172.21.15.71:0/4095137696 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3d1808d8d0 con 0x7f3d1c101e80 2024-04-23T02:44:52.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.258+0000 7f3d20ea5700 1 --2- 172.21.15.71:0/4095137696 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3d0806cce0 0x7f3d0806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:52.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.259+0000 7f3d20ea5700 1 --2- 172.21.15.71:0/4095137696 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3d0806cce0 0x7f3d0806f1a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0fcdf0 tx=0x7f3d0c005d50 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:52.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.263+0000 7f3d127fc700 1 -- 172.21.15.71:0/4095137696 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3d1805b3d0 con 0x7f3d1c101e80 2024-04-23T02:44:52.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.473+0000 7f3d2390a700 1 -- 172.21.15.71:0/4095137696 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d1c197be0 con 0x7f3d0806cce0 2024-04-23T02:44:52.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.475+0000 7f3d127fc700 1 -- 172.21.15.71:0/4095137696 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3d1c197be0 con 0x7f3d0806cce0 2024-04-23T02:44:52.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.477+0000 7f3d2390a700 1 -- 172.21.15.71:0/4095137696 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3d0806cce0 msgr2=0x7f3d0806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:52.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.477+0000 7f3d2390a700 1 --2- 172.21.15.71:0/4095137696 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3d0806cce0 0x7f3d0806f1a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0fcdf0 tx=0x7f3d0c005d50 comp rx=0 tx=0).stop 2024-04-23T02:44:52.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.477+0000 7f3d2390a700 1 -- 172.21.15.71:0/4095137696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d1c101e80 msgr2=0x7f3d1c19c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:52.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.477+0000 7f3d2390a700 1 --2- 172.21.15.71:0/4095137696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d1c101e80 0x7f3d1c19c570 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3d18006010 tx=0x7f3d1800be20 comp rx=0 tx=0).stop 2024-04-23T02:44:52.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.477+0000 7f3d2390a700 1 -- 172.21.15.71:0/4095137696 shutdown_connections 2024-04-23T02:44:52.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.477+0000 7f3d2390a700 1 --2- 172.21.15.71:0/4095137696 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3d0806cce0 0x7f3d0806f1a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:52.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.477+0000 7f3d2390a700 1 --2- 172.21.15.71:0/4095137696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d1c101e80 0x7f3d1c19c570 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:52.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.477+0000 7f3d2390a700 1 --2- 172.21.15.71:0/4095137696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d1c1027a0 0x7f3d1c19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:52.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.477+0000 7f3d2390a700 1 -- 172.21.15.71:0/4095137696 >> 172.21.15.71:0/4095137696 conn(0x7f3d1c0fb610 msgr2=0x7f3d1c1054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:52.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.477+0000 7f3d2390a700 1 -- 172.21.15.71:0/4095137696 shutdown_connections 2024-04-23T02:44:52.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.477+0000 7f3d2390a700 1 -- 172.21.15.71:0/4095137696 wait complete. 2024-04-23T02:44:52.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.580+0000 7f51c2055700 1 -- 172.21.15.71:0/1956202538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51bc105bd0 msgr2=0x7f51bc105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:52.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.580+0000 7f51c2055700 1 --2- 172.21.15.71:0/1956202538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51bc105bd0 0x7f51bc105fb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009910 tx=0x7f51a4009c20 comp rx=0 tx=0).stop 2024-04-23T02:44:52.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.580+0000 7f51c2055700 1 -- 172.21.15.71:0/1956202538 shutdown_connections 2024-04-23T02:44:52.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.580+0000 7f51c2055700 1 --2- 172.21.15.71:0/1956202538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51bc0ffc20 0x7f51bc100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:52.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.580+0000 7f51c2055700 1 --2- 172.21.15.71:0/1956202538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51bc105bd0 0x7f51bc105fb0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:52.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.580+0000 7f51c2055700 1 -- 172.21.15.71:0/1956202538 >> 172.21.15.71:0/1956202538 conn(0x7f51bc075670 msgr2=0x7f51bc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:52.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.580+0000 7f51c2055700 1 -- 172.21.15.71:0/1956202538 shutdown_connections 2024-04-23T02:44:52.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.580+0000 7f51c2055700 1 -- 172.21.15.71:0/1956202538 wait complete. 2024-04-23T02:44:52.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.581+0000 7f51c2055700 1 Processor -- start 2024-04-23T02:44:52.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.581+0000 7f51c2055700 1 -- start start 2024-04-23T02:44:52.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.581+0000 7f51c2055700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51bc0ffc20 0x7f51bc195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:52.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.581+0000 7f51c2055700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51bc105bd0 0x7f51bc195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:52.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.581+0000 7f51c2055700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc196260 con 0x7f51bc105bd0 2024-04-23T02:44:52.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.581+0000 7f51c2055700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc198f90 con 0x7f51bc0ffc20 2024-04-23T02:44:52.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.581+0000 7f51baffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51bc105bd0 0x7f51bc195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:52.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.581+0000 7f51baffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51bc105bd0 0x7f51bc195bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52466/0 (socket says 172.21.15.71:52466) 2024-04-23T02:44:52.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.581+0000 7f51baffd700 1 -- 172.21.15.71:0/3061790316 learned_addr learned my addr 172.21.15.71:0/3061790316 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:44:52.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.581+0000 7f51bb7fe700 1 --2- 172.21.15.71:0/3061790316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51bc0ffc20 0x7f51bc195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:52.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.581+0000 7f51baffd700 1 -- 172.21.15.71:0/3061790316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51bc0ffc20 msgr2=0x7f51bc195690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:52.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.581+0000 7f51baffd700 1 --2- 172.21.15.71:0/3061790316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51bc0ffc20 0x7f51bc195690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:52.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.581+0000 7f51baffd700 1 -- 172.21.15.71:0/3061790316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a40095f0 con 0x7f51bc105bd0 2024-04-23T02:44:52.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.582+0000 7f51baffd700 1 --2- 172.21.15.71:0/3061790316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51bc105bd0 0x7f51bc195bd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009970 tx=0x7f51ac009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:52.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.582+0000 7f51b8ff9700 1 -- 172.21.15.71:0/3061790316 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51ac012070 con 0x7f51bc105bd0 2024-04-23T02:44:52.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.582+0000 7f51b8ff9700 1 -- 172.21.15.71:0/3061790316 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f51ac007ad0 con 0x7f51bc105bd0 2024-04-23T02:44:52.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.582+0000 7f51c2055700 1 -- 172.21.15.71:0/3061790316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51bc199210 con 0x7f51bc105bd0 2024-04-23T02:44:52.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.582+0000 7f51b8ff9700 1 -- 172.21.15.71:0/3061790316 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51ac017c00 con 0x7f51bc105bd0 2024-04-23T02:44:52.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.582+0000 7f51c2055700 1 -- 172.21.15.71:0/3061790316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51bc199760 con 0x7f51bc105bd0 2024-04-23T02:44:52.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.583+0000 7f51c2055700 1 -- 172.21.15.71:0/3061790316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51bc04f2e0 con 0x7f51bc105bd0 2024-04-23T02:44:52.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.583+0000 7f51b8ff9700 1 -- 172.21.15.71:0/3061790316 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f51ac007c40 con 0x7f51bc105bd0 2024-04-23T02:44:52.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.584+0000 7f51b8ff9700 1 --2- 172.21.15.71:0/3061790316 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f51a806cce0 0x7f51a806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:52.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.584+0000 7f51b8ff9700 1 -- 172.21.15.71:0/3061790316 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f51ac08dbc0 con 0x7f51bc105bd0 2024-04-23T02:44:52.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.584+0000 7f51bb7fe700 1 --2- 172.21.15.71:0/3061790316 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f51a806cce0 0x7f51a806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:52.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.584+0000 7f51bb7fe700 1 --2- 172.21.15.71:0/3061790316 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f51a806cce0 0x7f51a806f1a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f51a4006010 tx=0x7f51a401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:52.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.588+0000 7f51b8ff9700 1 -- 172.21.15.71:0/3061790316 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f51ac057ce0 con 0x7f51bc105bd0 2024-04-23T02:44:52.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.792+0000 7f51c2055700 1 -- 172.21.15.71:0/3061790316 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f51bc199a50 con 0x7f51a806cce0 2024-04-23T02:44:52.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.802+0000 7f51b8ff9700 1 -- 172.21.15.71:0/3061790316 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f51bc199a50 con 0x7f51a806cce0 2024-04-23T02:44:52.805 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:44:52.805 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (28m) 6m ago 39m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:44:52.805 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (40m) 6m ago 40m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:44:52.805 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (38m) 6m ago 38m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:44:52.805 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (26m) 6m ago 40m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:44:52.806 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (26m) 6m ago 38m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:44:52.806 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (28m) 6m ago 39m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:44:52.806 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (22m) 6m ago 33m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:44:52.806 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (22m) 6m ago 33m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:44:52.806 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (6m) 6m ago 33m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:44:52.806 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (22m) 6m ago 33m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:44:52.806 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (30m) 6m ago 41m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:44:52.806 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (29m) 6m ago 38m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:44:52.806 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (27m) 6m ago 41m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:44:52.806 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (26m) 6m ago 38m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:44:52.806 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (29m) 6m ago 40m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:44:52.806 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (29m) 6m ago 38m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:44:52.806 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (26m) 6m ago 37m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:44:52.806 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (25m) 6m ago 37m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:44:52.807 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (24m) 6m ago 36m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:44:52.807 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (24m) 6m ago 36m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:44:52.807 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (23m) 6m ago 35m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:44:52.807 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (23m) 6m ago 35m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:44:52.807 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (28m) 6m ago 39m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:44:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.804+0000 7f51c2055700 1 -- 172.21.15.71:0/3061790316 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f51a806cce0 msgr2=0x7f51a806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.804+0000 7f51c2055700 1 --2- 172.21.15.71:0/3061790316 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f51a806cce0 0x7f51a806f1a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f51a4006010 tx=0x7f51a401a040 comp rx=0 tx=0).stop 2024-04-23T02:44:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.804+0000 7f51c2055700 1 -- 172.21.15.71:0/3061790316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51bc105bd0 msgr2=0x7f51bc195bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.804+0000 7f51c2055700 1 --2- 172.21.15.71:0/3061790316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51bc105bd0 0x7f51bc195bd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009970 tx=0x7f51ac009d30 comp rx=0 tx=0).stop 2024-04-23T02:44:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.804+0000 7f51c2055700 1 -- 172.21.15.71:0/3061790316 shutdown_connections 2024-04-23T02:44:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.804+0000 7f51c2055700 1 --2- 172.21.15.71:0/3061790316 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f51a806cce0 0x7f51a806f1a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.805+0000 7f51c2055700 1 --2- 172.21.15.71:0/3061790316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51bc0ffc20 0x7f51bc195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.805+0000 7f51c2055700 1 --2- 172.21.15.71:0/3061790316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51bc105bd0 0x7f51bc195bd0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.805+0000 7f51c2055700 1 -- 172.21.15.71:0/3061790316 >> 172.21.15.71:0/3061790316 conn(0x7f51bc075670 msgr2=0x7f51bc0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.805+0000 7f51c2055700 1 -- 172.21.15.71:0/3061790316 shutdown_connections 2024-04-23T02:44:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.805+0000 7f51c2055700 1 -- 172.21.15.71:0/3061790316 wait complete. 2024-04-23T02:44:52.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.908+0000 7fbe26e3d700 1 -- 172.21.15.71:0/3425516186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe20101840 msgr2=0x7fbe20105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:52.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.908+0000 7fbe26e3d700 1 --2- 172.21.15.71:0/3425516186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe20101840 0x7fbe20105890 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009910 tx=0x7fbe0c009c20 comp rx=0 tx=0).stop 2024-04-23T02:44:52.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.908+0000 7fbe26e3d700 1 -- 172.21.15.71:0/3425516186 shutdown_connections 2024-04-23T02:44:52.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.908+0000 7fbe26e3d700 1 --2- 172.21.15.71:0/3425516186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe20101840 0x7fbe20105890 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:52.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.908+0000 7fbe26e3d700 1 --2- 172.21.15.71:0/3425516186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe20100e90 0x7fbe20101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:52.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.908+0000 7fbe26e3d700 1 -- 172.21.15.71:0/3425516186 >> 172.21.15.71:0/3425516186 conn(0x7fbe200fc740 msgr2=0x7fbe200feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:52.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.908+0000 7fbe26e3d700 1 -- 172.21.15.71:0/3425516186 shutdown_connections 2024-04-23T02:44:52.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.908+0000 7fbe26e3d700 1 -- 172.21.15.71:0/3425516186 wait complete. 2024-04-23T02:44:52.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.909+0000 7fbe26e3d700 1 Processor -- start 2024-04-23T02:44:52.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.909+0000 7fbe26e3d700 1 -- start start 2024-04-23T02:44:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.910+0000 7fbe26e3d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe20100e90 0x7fbe2019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.910+0000 7fbe26e3d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe20101840 0x7fbe2019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.910+0000 7fbe26e3d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2019d200 con 0x7fbe20101840 2024-04-23T02:44:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.910+0000 7fbe26e3d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe201966b0 con 0x7fbe20100e90 2024-04-23T02:44:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.910+0000 7fbe1ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe20101840 0x7fbe2019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.910+0000 7fbe1ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe20101840 0x7fbe2019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52498/0 (socket says 172.21.15.71:52498) 2024-04-23T02:44:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.910+0000 7fbe1ffff700 1 -- 172.21.15.71:0/681066312 learned_addr learned my addr 172.21.15.71:0/681066312 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:44:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.910+0000 7fbe24bd9700 1 --2- 172.21.15.71:0/681066312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe20100e90 0x7fbe2019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.910+0000 7fbe1ffff700 1 -- 172.21.15.71:0/681066312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe20100e90 msgr2=0x7fbe2019c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.910+0000 7fbe1ffff700 1 --2- 172.21.15.71:0/681066312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe20100e90 0x7fbe2019c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.910+0000 7fbe1ffff700 1 -- 172.21.15.71:0/681066312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0c0095f0 con 0x7fbe20101840 2024-04-23T02:44:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.910+0000 7fbe1ffff700 1 --2- 172.21.15.71:0/681066312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe20101840 0x7fbe2019cb70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00bda0 tx=0x7fbe0c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.910+0000 7fbe1dffb700 1 -- 172.21.15.71:0/681066312 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe0c01d070 con 0x7fbe20101840 2024-04-23T02:44:52.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.910+0000 7fbe1dffb700 1 -- 172.21.15.71:0/681066312 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe0c005710 con 0x7fbe20101840 2024-04-23T02:44:52.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.910+0000 7fbe26e3d700 1 -- 172.21.15.71:0/681066312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe20196930 con 0x7fbe20101840 2024-04-23T02:44:52.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.910+0000 7fbe1dffb700 1 -- 172.21.15.71:0/681066312 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe0c021410 con 0x7fbe20101840 2024-04-23T02:44:52.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.910+0000 7fbe26e3d700 1 -- 172.21.15.71:0/681066312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe20196e20 con 0x7fbe20101840 2024-04-23T02:44:52.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.911+0000 7fbe26e3d700 1 -- 172.21.15.71:0/681066312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe20109290 con 0x7fbe20101840 2024-04-23T02:44:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.912+0000 7fbe1dffb700 1 -- 172.21.15.71:0/681066312 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbe0c00f460 con 0x7fbe20101840 2024-04-23T02:44:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.912+0000 7fbe1dffb700 1 --2- 172.21.15.71:0/681066312 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbe1006cce0 0x7fbe1006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.913+0000 7fbe24bd9700 1 --2- 172.21.15.71:0/681066312 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbe1006cce0 0x7fbe1006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.913+0000 7fbe1dffb700 1 -- 172.21.15.71:0/681066312 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbe0c08d7d0 con 0x7fbe20101840 2024-04-23T02:44:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.913+0000 7fbe24bd9700 1 --2- 172.21.15.71:0/681066312 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbe1006cce0 0x7fbe1006f1a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbe14005950 tx=0x7fbe140058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:52.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:52.917+0000 7fbe1dffb700 1 -- 172.21.15.71:0/681066312 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbe0c05b3f0 con 0x7fbe20101840 2024-04-23T02:44:53.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.188+0000 7fbe26e3d700 1 -- 172.21.15.71:0/681066312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbe2004f2e0 con 0x7fbe20101840 2024-04-23T02:44:53.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.188+0000 7fbe1dffb700 1 -- 172.21.15.71:0/681066312 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fbe0c05af80 con 0x7fbe20101840 2024-04-23T02:44:53.191 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:44:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:44:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:44:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:44:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:44:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:44:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:44:53.191 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:44:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:44:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:44:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:44:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:44:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:44:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:44:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:44:53.192 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:44:53.192 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:44:53.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.191+0000 7fbe26e3d700 1 -- 172.21.15.71:0/681066312 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbe1006cce0 msgr2=0x7fbe1006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:53.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.191+0000 7fbe26e3d700 1 --2- 172.21.15.71:0/681066312 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbe1006cce0 0x7fbe1006f1a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbe14005950 tx=0x7fbe140058e0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.191+0000 7fbe26e3d700 1 -- 172.21.15.71:0/681066312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe20101840 msgr2=0x7fbe2019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:53.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.191+0000 7fbe26e3d700 1 --2- 172.21.15.71:0/681066312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe20101840 0x7fbe2019cb70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00bda0 tx=0x7fbe0c00be80 comp rx=0 tx=0).stop 2024-04-23T02:44:53.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.191+0000 7fbe26e3d700 1 -- 172.21.15.71:0/681066312 shutdown_connections 2024-04-23T02:44:53.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.191+0000 7fbe26e3d700 1 --2- 172.21.15.71:0/681066312 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbe1006cce0 0x7fbe1006f1a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.191+0000 7fbe26e3d700 1 --2- 172.21.15.71:0/681066312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe20100e90 0x7fbe2019c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.191+0000 7fbe26e3d700 1 --2- 172.21.15.71:0/681066312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe20101840 0x7fbe2019cb70 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.191+0000 7fbe26e3d700 1 -- 172.21.15.71:0/681066312 >> 172.21.15.71:0/681066312 conn(0x7fbe200fc740 msgr2=0x7fbe200fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:53.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.191+0000 7fbe26e3d700 1 -- 172.21.15.71:0/681066312 shutdown_connections 2024-04-23T02:44:53.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.191+0000 7fbe26e3d700 1 -- 172.21.15.71:0/681066312 wait complete. 2024-04-23T02:44:53.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.293+0000 7fe5a0773700 1 -- 172.21.15.71:0/3171370147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5980ffc20 msgr2=0x7fe598100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:53.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.293+0000 7fe5a0773700 1 --2- 172.21.15.71:0/3171370147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5980ffc20 0x7fe598100080 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe588009910 tx=0x7fe588009c20 comp rx=0 tx=0).stop 2024-04-23T02:44:53.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.293+0000 7fe5a0773700 1 -- 172.21.15.71:0/3171370147 shutdown_connections 2024-04-23T02:44:53.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.293+0000 7fe5a0773700 1 --2- 172.21.15.71:0/3171370147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5980ffc20 0x7fe598100080 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.293+0000 7fe5a0773700 1 --2- 172.21.15.71:0/3171370147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe598105bd0 0x7fe598105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.293+0000 7fe5a0773700 1 -- 172.21.15.71:0/3171370147 >> 172.21.15.71:0/3171370147 conn(0x7fe598075670 msgr2=0x7fe598075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:53.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.293+0000 7fe5a0773700 1 -- 172.21.15.71:0/3171370147 shutdown_connections 2024-04-23T02:44:53.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.294+0000 7fe5a0773700 1 -- 172.21.15.71:0/3171370147 wait complete. 2024-04-23T02:44:53.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.294+0000 7fe5a0773700 1 Processor -- start 2024-04-23T02:44:53.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.294+0000 7fe5a0773700 1 -- start start 2024-04-23T02:44:53.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.294+0000 7fe5a0773700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5980ffc20 0x7fe598197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:53.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.294+0000 7fe5a0773700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe598105bd0 0x7fe598197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:53.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.294+0000 7fe5a0773700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe598198320 con 0x7fe5980ffc20 2024-04-23T02:44:53.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.294+0000 7fe5a0773700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe598198460 con 0x7fe598105bd0 2024-04-23T02:44:53.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.294+0000 7fe59e50f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5980ffc20 0x7fe598197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:53.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.294+0000 7fe59e50f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5980ffc20 0x7fe598197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52508/0 (socket says 172.21.15.71:52508) 2024-04-23T02:44:53.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.294+0000 7fe59e50f700 1 -- 172.21.15.71:0/408049719 learned_addr learned my addr 172.21.15.71:0/408049719 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:44:53.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.295+0000 7fe59e50f700 1 -- 172.21.15.71:0/408049719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe598105bd0 msgr2=0x7fe598197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:53.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.295+0000 7fe59dd0e700 1 --2- 172.21.15.71:0/408049719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe598105bd0 0x7fe598197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:53.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.295+0000 7fe59e50f700 1 --2- 172.21.15.71:0/408049719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe598105bd0 0x7fe598197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.295+0000 7fe59e50f700 1 -- 172.21.15.71:0/408049719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5880095f0 con 0x7fe5980ffc20 2024-04-23T02:44:53.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.295+0000 7fe59e50f700 1 --2- 172.21.15.71:0/408049719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5980ffc20 0x7fe598197700 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe59400b4b0 tx=0x7fe59400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:53.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.295+0000 7fe58f7fe700 1 -- 172.21.15.71:0/408049719 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe594011800 con 0x7fe5980ffc20 2024-04-23T02:44:53.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.295+0000 7fe58f7fe700 1 -- 172.21.15.71:0/408049719 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe594011e00 con 0x7fe5980ffc20 2024-04-23T02:44:53.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.295+0000 7fe5a0773700 1 -- 172.21.15.71:0/408049719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5981a0770 con 0x7fe5980ffc20 2024-04-23T02:44:53.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.295+0000 7fe58f7fe700 1 -- 172.21.15.71:0/408049719 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe594010e50 con 0x7fe5980ffc20 2024-04-23T02:44:53.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.295+0000 7fe5a0773700 1 -- 172.21.15.71:0/408049719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5981a0c90 con 0x7fe5980ffc20 2024-04-23T02:44:53.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.296+0000 7fe5a0773700 1 -- 172.21.15.71:0/408049719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5981080d0 con 0x7fe5980ffc20 2024-04-23T02:44:53.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.297+0000 7fe58f7fe700 1 -- 172.21.15.71:0/408049719 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe594010380 con 0x7fe5980ffc20 2024-04-23T02:44:53.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.297+0000 7fe58f7fe700 1 --2- 172.21.15.71:0/408049719 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe58406cce0 0x7fe58406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:53.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.297+0000 7fe59dd0e700 1 --2- 172.21.15.71:0/408049719 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe58406cce0 0x7fe58406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:53.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.297+0000 7fe58f7fe700 1 -- 172.21.15.71:0/408049719 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe59408c0f0 con 0x7fe5980ffc20 2024-04-23T02:44:53.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.297+0000 7fe59dd0e700 1 --2- 172.21.15.71:0/408049719 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe58406cce0 0x7fe58406f1a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe58800b5c0 tx=0x7fe588005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:53.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.302+0000 7fe58f7fe700 1 -- 172.21.15.71:0/408049719 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe5940561e0 con 0x7fe5980ffc20 2024-04-23T02:44:53.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.539+0000 7fe5a0773700 1 -- 172.21.15.71:0/408049719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe598107ab0 con 0x7fe5980ffc20 2024-04-23T02:44:53.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.540+0000 7fe58f7fe700 1 -- 172.21.15.71:0/408049719 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7fe594059800 con 0x7fe5980ffc20 2024-04-23T02:44:53.542 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:44:53.543 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:44:53.543 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:44:53.543 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:44:53.543 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:44:53.543 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:44:53.543 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:44:53.543 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:44:53.543 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:44:53.543 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:44:53.543 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:44:53.543 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:44:53.543 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:44:53.543 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:44:53.543 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:44:53.543 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:44:53.544 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:44:53.544 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:44:53.544 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:44:53.544 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:44:53.544 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:44:53.544 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:44:53.544 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:44:53.544 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:44:53.544 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:44:53.544 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:44:53.544 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:44:53.544 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:44:53.544 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:44:53.544 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:44:53.545 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:44:53.545 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:44:53.545 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:44:53.545 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:44:53.545 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:44:53.545 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:44:53.545 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:44:53.545 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:44:53.545 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:44:53.545 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:44:53.545 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:44:53.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.542+0000 7fe5a0773700 1 -- 172.21.15.71:0/408049719 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe58406cce0 msgr2=0x7fe58406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:53.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.542+0000 7fe5a0773700 1 --2- 172.21.15.71:0/408049719 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe58406cce0 0x7fe58406f1a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe58800b5c0 tx=0x7fe588005fd0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.542+0000 7fe5a0773700 1 -- 172.21.15.71:0/408049719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5980ffc20 msgr2=0x7fe598197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:53.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.542+0000 7fe5a0773700 1 --2- 172.21.15.71:0/408049719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5980ffc20 0x7fe598197700 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe59400b4b0 tx=0x7fe59400b870 comp rx=0 tx=0).stop 2024-04-23T02:44:53.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.542+0000 7fe5a0773700 1 -- 172.21.15.71:0/408049719 shutdown_connections 2024-04-23T02:44:53.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.542+0000 7fe5a0773700 1 --2- 172.21.15.71:0/408049719 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe58406cce0 0x7fe58406f1a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.542+0000 7fe5a0773700 1 --2- 172.21.15.71:0/408049719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5980ffc20 0x7fe598197700 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.542+0000 7fe5a0773700 1 --2- 172.21.15.71:0/408049719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe598105bd0 0x7fe598197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.542+0000 7fe5a0773700 1 -- 172.21.15.71:0/408049719 >> 172.21.15.71:0/408049719 conn(0x7fe598075670 msgr2=0x7fe5980fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:53.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.542+0000 7fe5a0773700 1 -- 172.21.15.71:0/408049719 shutdown_connections 2024-04-23T02:44:53.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.542+0000 7fe5a0773700 1 -- 172.21.15.71:0/408049719 wait complete. 2024-04-23T02:44:53.547 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:44:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.646+0000 7fc8638b1700 1 -- 172.21.15.71:0/604433712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c0ff5a0 msgr2=0x7fc85c0ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.646+0000 7fc8638b1700 1 --2- 172.21.15.71:0/604433712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c0ff5a0 0x7fc85c0ff980 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc84c009910 tx=0x7fc84c009c20 comp rx=0 tx=0).stop 2024-04-23T02:44:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.646+0000 7fc8638b1700 1 -- 172.21.15.71:0/604433712 shutdown_connections 2024-04-23T02:44:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.646+0000 7fc8638b1700 1 --2- 172.21.15.71:0/604433712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc85c0fff50 0x7fc85c105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.646+0000 7fc8638b1700 1 --2- 172.21.15.71:0/604433712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c0ff5a0 0x7fc85c0ff980 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.646+0000 7fc8638b1700 1 -- 172.21.15.71:0/604433712 >> 172.21.15.71:0/604433712 conn(0x7fc85c074e30 msgr2=0x7fc85c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.646+0000 7fc8638b1700 1 -- 172.21.15.71:0/604433712 shutdown_connections 2024-04-23T02:44:53.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.646+0000 7fc8638b1700 1 -- 172.21.15.71:0/604433712 wait complete. 2024-04-23T02:44:53.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.647+0000 7fc8638b1700 1 Processor -- start 2024-04-23T02:44:53.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.647+0000 7fc8638b1700 1 -- start start 2024-04-23T02:44:53.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.647+0000 7fc8638b1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc85c0ff5a0 0x7fc85c100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:53.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.647+0000 7fc8638b1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c0fff50 0x7fc85c100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:53.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.647+0000 7fc8638b1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c100fa0 con 0x7fc85c0fff50 2024-04-23T02:44:53.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.647+0000 7fc8638b1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c1010e0 con 0x7fc85c0ff5a0 2024-04-23T02:44:53.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.647+0000 7fc860e4c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c0fff50 0x7fc85c100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:53.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.647+0000 7fc860e4c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c0fff50 0x7fc85c100a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52524/0 (socket says 172.21.15.71:52524) 2024-04-23T02:44:53.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.647+0000 7fc860e4c700 1 -- 172.21.15.71:0/1982714342 learned_addr learned my addr 172.21.15.71:0/1982714342 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:44:53.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.647+0000 7fc86164d700 1 --2- 172.21.15.71:0/1982714342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc85c0ff5a0 0x7fc85c100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:53.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.647+0000 7fc860e4c700 1 -- 172.21.15.71:0/1982714342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc85c0ff5a0 msgr2=0x7fc85c100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:53.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.647+0000 7fc860e4c700 1 --2- 172.21.15.71:0/1982714342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc85c0ff5a0 0x7fc85c100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.647+0000 7fc860e4c700 1 -- 172.21.15.71:0/1982714342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c0095f0 con 0x7fc85c0fff50 2024-04-23T02:44:53.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.647+0000 7fc860e4c700 1 --2- 172.21.15.71:0/1982714342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c0fff50 0x7fc85c100a60 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc85800c960 tx=0x7fc85800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:53.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.648+0000 7fc8527fc700 1 -- 172.21.15.71:0/1982714342 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc858007a70 con 0x7fc85c0fff50 2024-04-23T02:44:53.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.648+0000 7fc8527fc700 1 -- 172.21.15.71:0/1982714342 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc858007bd0 con 0x7fc85c0fff50 2024-04-23T02:44:53.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.648+0000 7fc8527fc700 1 -- 172.21.15.71:0/1982714342 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc858018630 con 0x7fc85c0fff50 2024-04-23T02:44:53.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.648+0000 7fc8638b1700 1 -- 172.21.15.71:0/1982714342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c1a2970 con 0x7fc85c0fff50 2024-04-23T02:44:53.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.648+0000 7fc8638b1700 1 -- 172.21.15.71:0/1982714342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c1a2cd0 con 0x7fc85c0fff50 2024-04-23T02:44:53.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.649+0000 7fc8638b1700 1 -- 172.21.15.71:0/1982714342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85c109270 con 0x7fc85c0fff50 2024-04-23T02:44:53.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.649+0000 7fc8527fc700 1 -- 172.21.15.71:0/1982714342 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc85800f9f0 con 0x7fc85c0fff50 2024-04-23T02:44:53.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.649+0000 7fc8527fc700 1 --2- 172.21.15.71:0/1982714342 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc84806cce0 0x7fc84806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:53.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.649+0000 7fc8527fc700 1 -- 172.21.15.71:0/1982714342 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc85808c560 con 0x7fc85c0fff50 2024-04-23T02:44:53.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.650+0000 7fc86164d700 1 --2- 172.21.15.71:0/1982714342 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc84806cce0 0x7fc84806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:53.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.650+0000 7fc86164d700 1 --2- 172.21.15.71:0/1982714342 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc84806cce0 0x7fc84806f1a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00b5c0 tx=0x7fc84c005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:53.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.654+0000 7fc8527fc700 1 -- 172.21.15.71:0/1982714342 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc85805a0e0 con 0x7fc85c0fff50 2024-04-23T02:44:53.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.865+0000 7fc8638b1700 1 -- 172.21.15.71:0/1982714342 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc85c02ccc0 con 0x7fc84806cce0 2024-04-23T02:44:53.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.866+0000 7fc8527fc700 1 -- 172.21.15.71:0/1982714342 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc85c02ccc0 con 0x7fc84806cce0 2024-04-23T02:44:53.868 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:44:53.869 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:44:53.869 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:44:53.869 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:44:53.869 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:44:53.869 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:44:53.869 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:44:53.869 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:44:53.869 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:44:53.869 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:44:53.869 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:44:53.869 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:44:53.869 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:44:53.869 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:44:53.869 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:44:53.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.868+0000 7fc8638b1700 1 -- 172.21.15.71:0/1982714342 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc84806cce0 msgr2=0x7fc84806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.868+0000 7fc8638b1700 1 --2- 172.21.15.71:0/1982714342 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc84806cce0 0x7fc84806f1a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00b5c0 tx=0x7fc84c005fd0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.868+0000 7fc8638b1700 1 -- 172.21.15.71:0/1982714342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c0fff50 msgr2=0x7fc85c100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.868+0000 7fc8638b1700 1 --2- 172.21.15.71:0/1982714342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c0fff50 0x7fc85c100a60 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc85800c960 tx=0x7fc85800cd20 comp rx=0 tx=0).stop 2024-04-23T02:44:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.868+0000 7fc8638b1700 1 -- 172.21.15.71:0/1982714342 shutdown_connections 2024-04-23T02:44:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.868+0000 7fc8638b1700 1 --2- 172.21.15.71:0/1982714342 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc84806cce0 0x7fc84806f1a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.868+0000 7fc8638b1700 1 --2- 172.21.15.71:0/1982714342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc85c0ff5a0 0x7fc85c100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.868+0000 7fc8638b1700 1 --2- 172.21.15.71:0/1982714342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c0fff50 0x7fc85c100a60 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.868+0000 7fc8638b1700 1 -- 172.21.15.71:0/1982714342 >> 172.21.15.71:0/1982714342 conn(0x7fc85c074e30 msgr2=0x7fc85c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.868+0000 7fc8638b1700 1 -- 172.21.15.71:0/1982714342 shutdown_connections 2024-04-23T02:44:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.868+0000 7fc8638b1700 1 -- 172.21.15.71:0/1982714342 wait complete. 2024-04-23T02:44:53.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:53 smithi192 ceph-mon[108617]: from='client.35520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:44:53.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:53 smithi192 ceph-mon[108617]: from='client.35524 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:44:53.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:53 smithi192 ceph-mon[108617]: pgmap v948: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:53.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:53 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/681066312' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:44:53.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:53 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/408049719' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:44:53.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.970+0000 7fd1650ef700 1 -- 172.21.15.71:0/2734210607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd160100ea0 msgr2=0x7fd160101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:53.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.970+0000 7fd1650ef700 1 --2- 172.21.15.71:0/2734210607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd160100ea0 0x7fd160101280 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd154009910 tx=0x7fd154009c20 comp rx=0 tx=0).stop 2024-04-23T02:44:53.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.970+0000 7fd1650ef700 1 -- 172.21.15.71:0/2734210607 shutdown_connections 2024-04-23T02:44:53.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.970+0000 7fd1650ef700 1 --2- 172.21.15.71:0/2734210607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd160101850 0x7fd1601058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.970+0000 7fd1650ef700 1 --2- 172.21.15.71:0/2734210607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd160100ea0 0x7fd160101280 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.970+0000 7fd1650ef700 1 -- 172.21.15.71:0/2734210607 >> 172.21.15.71:0/2734210607 conn(0x7fd1600fc750 msgr2=0x7fd1600feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:53.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.970+0000 7fd1650ef700 1 -- 172.21.15.71:0/2734210607 shutdown_connections 2024-04-23T02:44:53.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.970+0000 7fd1650ef700 1 -- 172.21.15.71:0/2734210607 wait complete. 2024-04-23T02:44:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.971+0000 7fd1650ef700 1 Processor -- start 2024-04-23T02:44:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.971+0000 7fd1650ef700 1 -- start start 2024-04-23T02:44:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.971+0000 7fd1650ef700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd160100ea0 0x7fd160073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.971+0000 7fd1650ef700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd160101850 0x7fd1600735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.971+0000 7fd1650ef700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd160073b10 con 0x7fd160101850 2024-04-23T02:44:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.971+0000 7fd1650ef700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd160073c50 con 0x7fd160100ea0 2024-04-23T02:44:53.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.971+0000 7fd15e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd160101850 0x7fd1600735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.971+0000 7fd15e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd160101850 0x7fd1600735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52544/0 (socket says 172.21.15.71:52544) 2024-04-23T02:44:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.971+0000 7fd15e59c700 1 -- 172.21.15.71:0/3401718270 learned_addr learned my addr 172.21.15.71:0/3401718270 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:44:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.972+0000 7fd15e59c700 1 -- 172.21.15.71:0/3401718270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd160100ea0 msgr2=0x7fd160073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.972+0000 7fd15ed9d700 1 --2- 172.21.15.71:0/3401718270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd160100ea0 0x7fd160073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.972+0000 7fd15e59c700 1 --2- 172.21.15.71:0/3401718270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd160100ea0 0x7fd160073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.972+0000 7fd15e59c700 1 -- 172.21.15.71:0/3401718270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1540095f0 con 0x7fd160101850 2024-04-23T02:44:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.972+0000 7fd15e59c700 1 --2- 172.21.15.71:0/3401718270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd160101850 0x7fd1600735d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd15000d900 tx=0x7fd15000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.972+0000 7fd147fff700 1 -- 172.21.15.71:0/3401718270 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd150009900 con 0x7fd160101850 2024-04-23T02:44:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.972+0000 7fd1650ef700 1 -- 172.21.15.71:0/3401718270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1601a2960 con 0x7fd160101850 2024-04-23T02:44:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.972+0000 7fd147fff700 1 -- 172.21.15.71:0/3401718270 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd150010460 con 0x7fd160101850 2024-04-23T02:44:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.972+0000 7fd147fff700 1 -- 172.21.15.71:0/3401718270 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd15000f5f0 con 0x7fd160101850 2024-04-23T02:44:53.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.972+0000 7fd1650ef700 1 -- 172.21.15.71:0/3401718270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1601a2d70 con 0x7fd160101850 2024-04-23T02:44:53.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.973+0000 7fd1650ef700 1 -- 172.21.15.71:0/3401718270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd160109240 con 0x7fd160101850 2024-04-23T02:44:53.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.973+0000 7fd147fff700 1 -- 172.21.15.71:0/3401718270 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd15000fac0 con 0x7fd160101850 2024-04-23T02:44:53.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.974+0000 7fd147fff700 1 --2- 172.21.15.71:0/3401718270 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd14806cce0 0x7fd14806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:44:53.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.974+0000 7fd15ed9d700 1 --2- 172.21.15.71:0/3401718270 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd14806cce0 0x7fd14806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:44:53.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.974+0000 7fd147fff700 1 -- 172.21.15.71:0/3401718270 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd15008c3c0 con 0x7fd160101850 2024-04-23T02:44:53.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.974+0000 7fd15ed9d700 1 --2- 172.21.15.71:0/3401718270 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd14806cce0 0x7fd14806f1a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd15400b5c0 tx=0x7fd154005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:44:53.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:53.979+0000 7fd147fff700 1 -- 172.21.15.71:0/3401718270 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd150056430 con 0x7fd160101850 2024-04-23T02:44:53.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:53 smithi071 ceph-mon[144544]: from='client.35520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:44:53.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:53 smithi071 ceph-mon[144544]: from='client.35524 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:44:53.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:53 smithi071 ceph-mon[144544]: pgmap v948: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:53.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:53 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/681066312' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:44:53.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:53 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/408049719' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:44:54.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:54.249+0000 7fd1650ef700 1 -- 172.21.15.71:0/3401718270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd16004f2e0 con 0x7fd160101850 2024-04-23T02:44:54.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:54.250+0000 7fd147fff700 1 -- 172.21.15.71:0/3401718270 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7fd150059a50 con 0x7fd160101850 2024-04-23T02:44:54.253 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:44:54.253 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:44:54.253 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:44:54.253 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:44:54.253 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:44:54.253 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:44:54.253 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:44:54.253 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:44:54.253 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:44:54.253 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:44:54.253 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:44:54.253 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:44:54.253 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:44:54.254 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:44:54.254 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:44:54.254 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:44:54.254 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:44:54.254 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:44:54.254 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:44:54.254 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:44:54.254 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:44:54.254 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:44:54.254 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:44:54.254 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:44:54.254 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:44:54.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:54.252+0000 7fd1650ef700 1 -- 172.21.15.71:0/3401718270 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd14806cce0 msgr2=0x7fd14806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:54.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:54.252+0000 7fd1650ef700 1 --2- 172.21.15.71:0/3401718270 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd14806cce0 0x7fd14806f1a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd15400b5c0 tx=0x7fd154005fd0 comp rx=0 tx=0).stop 2024-04-23T02:44:54.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:54.252+0000 7fd1650ef700 1 -- 172.21.15.71:0/3401718270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd160101850 msgr2=0x7fd1600735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:44:54.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:54.252+0000 7fd1650ef700 1 --2- 172.21.15.71:0/3401718270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd160101850 0x7fd1600735d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd15000d900 tx=0x7fd15000dcc0 comp rx=0 tx=0).stop 2024-04-23T02:44:54.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:54.252+0000 7fd1650ef700 1 -- 172.21.15.71:0/3401718270 shutdown_connections 2024-04-23T02:44:54.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:54.252+0000 7fd1650ef700 1 --2- 172.21.15.71:0/3401718270 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd14806cce0 0x7fd14806f1a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:54.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:54.252+0000 7fd1650ef700 1 --2- 172.21.15.71:0/3401718270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd160100ea0 0x7fd160073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:54.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:54.252+0000 7fd1650ef700 1 --2- 172.21.15.71:0/3401718270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd160101850 0x7fd1600735d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:44:54.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:54.252+0000 7fd1650ef700 1 -- 172.21.15.71:0/3401718270 >> 172.21.15.71:0/3401718270 conn(0x7fd1600fc750 msgr2=0x7fd1600fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:44:54.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:54.252+0000 7fd1650ef700 1 -- 172.21.15.71:0/3401718270 shutdown_connections 2024-04-23T02:44:54.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:44:54.252+0000 7fd1650ef700 1 -- 172.21.15.71:0/3401718270 wait complete. 2024-04-23T02:44:54.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:54 smithi192 ceph-mon[108617]: from='client.35528 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:44:54.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:54 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3401718270' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:44:54.944 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:54 smithi071 ceph-mon[144544]: from='client.35528 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:44:54.945 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:54 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3401718270' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:44:55.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:55 smithi192 ceph-mon[108617]: from='client.35540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:44:55.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:55 smithi192 ceph-mon[108617]: pgmap v949: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:55.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:55 smithi071 ceph-mon[144544]: from='client.35540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:44:55.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:55 smithi071 ceph-mon[144544]: pgmap v949: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:56.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:56 smithi192 ceph-mon[108617]: pgmap v950: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:56.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:56 smithi071 ceph-mon[144544]: pgmap v950: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:59.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:44:59 smithi192 ceph-mon[108617]: pgmap v951: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:44:59.989 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:44:59 smithi071 ceph-mon[144544]: pgmap v951: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:01.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:01 smithi192 ceph-mon[108617]: pgmap v952: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:01.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:01 smithi071 ceph-mon[144544]: pgmap v952: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:03.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:03 smithi192 ceph-mon[108617]: pgmap v953: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:03.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:03 smithi071 ceph-mon[144544]: pgmap v953: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:05 smithi192 ceph-mon[108617]: pgmap v954: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:45:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:05 smithi071 ceph-mon[144544]: pgmap v954: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:05.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:45:07.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:07 smithi192 ceph-mon[108617]: pgmap v955: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:07.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:07 smithi071 ceph-mon[144544]: pgmap v955: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:09.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:09 smithi192 ceph-mon[108617]: pgmap v956: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:09.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:09 smithi071 ceph-mon[144544]: pgmap v956: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:11.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:11 smithi192 ceph-mon[108617]: pgmap v957: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:11.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:11 smithi071 ceph-mon[144544]: pgmap v957: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:13.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:13 smithi192 ceph-mon[108617]: pgmap v958: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:13.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:13 smithi071 ceph-mon[144544]: pgmap v958: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:15.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:15 smithi192 ceph-mon[108617]: pgmap v959: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:15.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:15 smithi071 ceph-mon[144544]: pgmap v959: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:17.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:17 smithi192 ceph-mon[108617]: pgmap v960: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:17.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:17 smithi071 ceph-mon[144544]: pgmap v960: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:18.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:18 smithi071 ceph-mon[144544]: pgmap v961: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:19.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:18 smithi192 ceph-mon[108617]: pgmap v961: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:20.118 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:45:20.176 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:45:21.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:20 smithi192 ceph-mon[108617]: pgmap v962: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:21.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:20 smithi071 ceph-mon[144544]: pgmap v962: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:23.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:23 smithi192 ceph-mon[108617]: pgmap v963: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:23.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:23 smithi071 ceph-mon[144544]: pgmap v963: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:24.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.368+0000 7f7fee1f5700 1 -- 172.21.15.71:0/2811218829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe8105bd0 msgr2=0x7f7fe8105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:24.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.368+0000 7f7fee1f5700 1 --2- 172.21.15.71:0/2811218829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe8105bd0 0x7f7fe8105fb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8009910 tx=0x7f7fd8009c20 comp rx=0 tx=0).stop 2024-04-23T02:45:24.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.368+0000 7f7fee1f5700 1 -- 172.21.15.71:0/2811218829 shutdown_connections 2024-04-23T02:45:24.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.368+0000 7f7fee1f5700 1 --2- 172.21.15.71:0/2811218829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe80ffc20 0x7f7fe8100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:24.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.368+0000 7f7fee1f5700 1 --2- 172.21.15.71:0/2811218829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe8105bd0 0x7f7fe8105fb0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:24.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.368+0000 7f7fee1f5700 1 -- 172.21.15.71:0/2811218829 >> 172.21.15.71:0/2811218829 conn(0x7f7fe8075670 msgr2=0x7f7fe8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:24.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.368+0000 7f7fee1f5700 1 -- 172.21.15.71:0/2811218829 shutdown_connections 2024-04-23T02:45:24.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.368+0000 7f7fee1f5700 1 -- 172.21.15.71:0/2811218829 wait complete. 2024-04-23T02:45:24.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.369+0000 7f7fee1f5700 1 Processor -- start 2024-04-23T02:45:24.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.369+0000 7f7fee1f5700 1 -- start start 2024-04-23T02:45:24.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.369+0000 7f7fee1f5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe80ffc20 0x7f7fe8195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:24.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.369+0000 7f7fee1f5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe8105bd0 0x7f7fe8195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:24.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.369+0000 7f7fee1f5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe8196260 con 0x7f7fe80ffc20 2024-04-23T02:45:24.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.369+0000 7f7fee1f5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe8198f90 con 0x7f7fe8105bd0 2024-04-23T02:45:24.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.369+0000 7f7fe77fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe80ffc20 0x7f7fe8195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:24.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.369+0000 7f7fe77fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe80ffc20 0x7f7fe8195690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56700/0 (socket says 172.21.15.71:56700) 2024-04-23T02:45:24.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.369+0000 7f7fe77fe700 1 -- 172.21.15.71:0/4203922299 learned_addr learned my addr 172.21.15.71:0/4203922299 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:45:24.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.369+0000 7f7fe77fe700 1 -- 172.21.15.71:0/4203922299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe8105bd0 msgr2=0x7f7fe8195bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:24.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.369+0000 7f7fe77fe700 1 --2- 172.21.15.71:0/4203922299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe8105bd0 0x7f7fe8195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:24.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.369+0000 7f7fe77fe700 1 -- 172.21.15.71:0/4203922299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd80095f0 con 0x7f7fe80ffc20 2024-04-23T02:45:24.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.370+0000 7f7fe77fe700 1 --2- 172.21.15.71:0/4203922299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe80ffc20 0x7f7fe8195690 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8006010 tx=0x7f7fd800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:24.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.370+0000 7f7fe4ff9700 1 -- 172.21.15.71:0/4203922299 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fd801d070 con 0x7f7fe80ffc20 2024-04-23T02:45:24.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.370+0000 7f7fe4ff9700 1 -- 172.21.15.71:0/4203922299 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7fd800f460 con 0x7f7fe80ffc20 2024-04-23T02:45:24.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.370+0000 7f7fee1f5700 1 -- 172.21.15.71:0/4203922299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe8199210 con 0x7f7fe80ffc20 2024-04-23T02:45:24.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.370+0000 7f7fe4ff9700 1 -- 172.21.15.71:0/4203922299 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fd8021640 con 0x7f7fe80ffc20 2024-04-23T02:45:24.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.370+0000 7f7fee1f5700 1 -- 172.21.15.71:0/4203922299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe8199700 con 0x7f7fe80ffc20 2024-04-23T02:45:24.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.371+0000 7f7fee1f5700 1 -- 172.21.15.71:0/4203922299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe81080f0 con 0x7f7fe80ffc20 2024-04-23T02:45:24.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.371+0000 7f7fe4ff9700 1 -- 172.21.15.71:0/4203922299 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7fd800fa20 con 0x7f7fe80ffc20 2024-04-23T02:45:24.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.372+0000 7f7fe4ff9700 1 --2- 172.21.15.71:0/4203922299 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7fd006cce0 0x7f7fd006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:24.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.372+0000 7f7fe4ff9700 1 -- 172.21.15.71:0/4203922299 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7fd808d830 con 0x7f7fe80ffc20 2024-04-23T02:45:24.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.372+0000 7f7fe6ffd700 1 --2- 172.21.15.71:0/4203922299 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7fd006cce0 0x7f7fd006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:24.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.372+0000 7f7fe6ffd700 1 --2- 172.21.15.71:0/4203922299 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7fd006cce0 0x7f7fd006f1a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc005950 tx=0x7f7fdc0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:24.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.376+0000 7f7fe4ff9700 1 -- 172.21.15.71:0/4203922299 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7fd80578a0 con 0x7f7fe80ffc20 2024-04-23T02:45:24.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.587+0000 7f7fee1f5700 1 -- 172.21.15.71:0/4203922299 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fe8196ad0 con 0x7f7fd006cce0 2024-04-23T02:45:24.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.589+0000 7f7fe4ff9700 1 -- 172.21.15.71:0/4203922299 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7fe8196ad0 con 0x7f7fd006cce0 2024-04-23T02:45:24.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.591+0000 7f7fee1f5700 1 -- 172.21.15.71:0/4203922299 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7fd006cce0 msgr2=0x7f7fd006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:24.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.591+0000 7f7fee1f5700 1 --2- 172.21.15.71:0/4203922299 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7fd006cce0 0x7f7fd006f1a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc005950 tx=0x7f7fdc0058e0 comp rx=0 tx=0).stop 2024-04-23T02:45:24.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.591+0000 7f7fee1f5700 1 -- 172.21.15.71:0/4203922299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe80ffc20 msgr2=0x7f7fe8195690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:24.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.591+0000 7f7fee1f5700 1 --2- 172.21.15.71:0/4203922299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe80ffc20 0x7f7fe8195690 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8006010 tx=0x7f7fd800be20 comp rx=0 tx=0).stop 2024-04-23T02:45:24.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.591+0000 7f7fee1f5700 1 -- 172.21.15.71:0/4203922299 shutdown_connections 2024-04-23T02:45:24.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.591+0000 7f7fee1f5700 1 --2- 172.21.15.71:0/4203922299 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7fd006cce0 0x7f7fd006f1a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:24.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.591+0000 7f7fee1f5700 1 --2- 172.21.15.71:0/4203922299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe80ffc20 0x7f7fe8195690 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:24.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.591+0000 7f7fee1f5700 1 --2- 172.21.15.71:0/4203922299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe8105bd0 0x7f7fe8195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:24.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.591+0000 7f7fee1f5700 1 -- 172.21.15.71:0/4203922299 >> 172.21.15.71:0/4203922299 conn(0x7f7fe8075670 msgr2=0x7f7fe80fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:24.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.591+0000 7f7fee1f5700 1 -- 172.21.15.71:0/4203922299 shutdown_connections 2024-04-23T02:45:24.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.591+0000 7f7fee1f5700 1 -- 172.21.15.71:0/4203922299 wait complete. 2024-04-23T02:45:24.604 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:45:24.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.694+0000 7fe69d711700 1 -- 172.21.15.71:0/3645908767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe698100ea0 msgr2=0x7fe698101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:24.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.694+0000 7fe69d711700 1 --2- 172.21.15.71:0/3645908767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe698100ea0 0x7fe698101280 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe68c009910 tx=0x7fe68c009c20 comp rx=0 tx=0).stop 2024-04-23T02:45:24.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.694+0000 7fe69d711700 1 -- 172.21.15.71:0/3645908767 shutdown_connections 2024-04-23T02:45:24.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.694+0000 7fe69d711700 1 --2- 172.21.15.71:0/3645908767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe698101850 0x7fe6981058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:24.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.694+0000 7fe69d711700 1 --2- 172.21.15.71:0/3645908767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe698100ea0 0x7fe698101280 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:24.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.694+0000 7fe69d711700 1 -- 172.21.15.71:0/3645908767 >> 172.21.15.71:0/3645908767 conn(0x7fe6980fc750 msgr2=0x7fe6980feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:24.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.694+0000 7fe69d711700 1 -- 172.21.15.71:0/3645908767 shutdown_connections 2024-04-23T02:45:24.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.694+0000 7fe69d711700 1 -- 172.21.15.71:0/3645908767 wait complete. 2024-04-23T02:45:24.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.694+0000 7fe69d711700 1 Processor -- start 2024-04-23T02:45:24.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.695+0000 7fe69d711700 1 -- start start 2024-04-23T02:45:24.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.695+0000 7fe69d711700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe698100ea0 0x7fe69819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:24.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.695+0000 7fe69d711700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe698101850 0x7fe69819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:24.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.695+0000 7fe69d711700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69819d230 con 0x7fe698100ea0 2024-04-23T02:45:24.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.695+0000 7fe69d711700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6981966e0 con 0x7fe698101850 2024-04-23T02:45:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.695+0000 7fe696ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe698100ea0 0x7fe69819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.695+0000 7fe696ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe698100ea0 0x7fe69819c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56710/0 (socket says 172.21.15.71:56710) 2024-04-23T02:45:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.695+0000 7fe696ffd700 1 -- 172.21.15.71:0/1203256654 learned_addr learned my addr 172.21.15.71:0/1203256654 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:45:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.695+0000 7fe6967fc700 1 --2- 172.21.15.71:0/1203256654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe698101850 0x7fe69819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.695+0000 7fe696ffd700 1 -- 172.21.15.71:0/1203256654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe698101850 msgr2=0x7fe69819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.695+0000 7fe696ffd700 1 --2- 172.21.15.71:0/1203256654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe698101850 0x7fe69819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.695+0000 7fe696ffd700 1 -- 172.21.15.71:0/1203256654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c0095f0 con 0x7fe698100ea0 2024-04-23T02:45:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.695+0000 7fe696ffd700 1 --2- 172.21.15.71:0/1203256654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe698100ea0 0x7fe69819c660 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe68c006010 tx=0x7fe68c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.695+0000 7fe67ffff700 1 -- 172.21.15.71:0/1203256654 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe68c01d070 con 0x7fe698100ea0 2024-04-23T02:45:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.696+0000 7fe67ffff700 1 -- 172.21.15.71:0/1203256654 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe68c00f460 con 0x7fe698100ea0 2024-04-23T02:45:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.696+0000 7fe69d711700 1 -- 172.21.15.71:0/1203256654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe698196960 con 0x7fe698100ea0 2024-04-23T02:45:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.696+0000 7fe67ffff700 1 -- 172.21.15.71:0/1203256654 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe68c021640 con 0x7fe698100ea0 2024-04-23T02:45:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.696+0000 7fe69d711700 1 -- 172.21.15.71:0/1203256654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe698196e50 con 0x7fe698100ea0 2024-04-23T02:45:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.697+0000 7fe69d711700 1 -- 172.21.15.71:0/1203256654 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe698109240 con 0x7fe698100ea0 2024-04-23T02:45:24.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.697+0000 7fe67ffff700 1 -- 172.21.15.71:0/1203256654 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe68c00fa60 con 0x7fe698100ea0 2024-04-23T02:45:24.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.697+0000 7fe67ffff700 1 --2- 172.21.15.71:0/1203256654 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe68006cce0 0x7fe68006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:24.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.698+0000 7fe67ffff700 1 -- 172.21.15.71:0/1203256654 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe68c08d880 con 0x7fe698100ea0 2024-04-23T02:45:24.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.698+0000 7fe6967fc700 1 --2- 172.21.15.71:0/1203256654 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe68006cce0 0x7fe68006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:24.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.698+0000 7fe6967fc700 1 --2- 172.21.15.71:0/1203256654 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe68006cce0 0x7fe68006f1a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe688006fd0 tx=0x7fe688008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:24.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.702+0000 7fe67ffff700 1 -- 172.21.15.71:0/1203256654 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe68c05b380 con 0x7fe698100ea0 2024-04-23T02:45:24.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.910+0000 7fe69d711700 1 -- 172.21.15.71:0/1203256654 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe698197d00 con 0x7fe68006cce0 2024-04-23T02:45:24.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.912+0000 7fe67ffff700 1 -- 172.21.15.71:0/1203256654 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe698197d00 con 0x7fe68006cce0 2024-04-23T02:45:24.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.914+0000 7fe69d711700 1 -- 172.21.15.71:0/1203256654 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe68006cce0 msgr2=0x7fe68006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:24.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.914+0000 7fe69d711700 1 --2- 172.21.15.71:0/1203256654 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe68006cce0 0x7fe68006f1a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe688006fd0 tx=0x7fe688008040 comp rx=0 tx=0).stop 2024-04-23T02:45:24.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.914+0000 7fe69d711700 1 -- 172.21.15.71:0/1203256654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe698100ea0 msgr2=0x7fe69819c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:24.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.914+0000 7fe69d711700 1 --2- 172.21.15.71:0/1203256654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe698100ea0 0x7fe69819c660 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe68c006010 tx=0x7fe68c00be20 comp rx=0 tx=0).stop 2024-04-23T02:45:24.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.914+0000 7fe69d711700 1 -- 172.21.15.71:0/1203256654 shutdown_connections 2024-04-23T02:45:24.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.914+0000 7fe69d711700 1 --2- 172.21.15.71:0/1203256654 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe68006cce0 0x7fe68006f1a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:24.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.914+0000 7fe69d711700 1 --2- 172.21.15.71:0/1203256654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe698100ea0 0x7fe69819c660 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:24.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.914+0000 7fe69d711700 1 --2- 172.21.15.71:0/1203256654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe698101850 0x7fe69819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:24.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.914+0000 7fe69d711700 1 -- 172.21.15.71:0/1203256654 >> 172.21.15.71:0/1203256654 conn(0x7fe6980fc750 msgr2=0x7fe6980fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:24.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.914+0000 7fe69d711700 1 -- 172.21.15.71:0/1203256654 shutdown_connections 2024-04-23T02:45:24.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:24.914+0000 7fe69d711700 1 -- 172.21.15.71:0/1203256654 wait complete. 2024-04-23T02:45:25.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.016+0000 7f2942278700 1 -- 172.21.15.71:0/3443455191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f293c101840 msgr2=0x7f293c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:25.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.016+0000 7f2942278700 1 --2- 172.21.15.71:0/3443455191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f293c101840 0x7f293c105890 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f2924009910 tx=0x7f2924009c20 comp rx=0 tx=0).stop 2024-04-23T02:45:25.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.016+0000 7f2942278700 1 -- 172.21.15.71:0/3443455191 shutdown_connections 2024-04-23T02:45:25.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.016+0000 7f2942278700 1 --2- 172.21.15.71:0/3443455191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f293c101840 0x7f293c105890 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.016+0000 7f2942278700 1 --2- 172.21.15.71:0/3443455191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f293c100e90 0x7f293c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.016+0000 7f2942278700 1 -- 172.21.15.71:0/3443455191 >> 172.21.15.71:0/3443455191 conn(0x7f293c0fc740 msgr2=0x7f293c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:25.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.016+0000 7f2942278700 1 -- 172.21.15.71:0/3443455191 shutdown_connections 2024-04-23T02:45:25.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.016+0000 7f2942278700 1 -- 172.21.15.71:0/3443455191 wait complete. 2024-04-23T02:45:25.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.016+0000 7f2942278700 1 Processor -- start 2024-04-23T02:45:25.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.016+0000 7f2942278700 1 -- start start 2024-04-23T02:45:25.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.017+0000 7f2942278700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f293c100e90 0x7f293c19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:25.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.017+0000 7f2942278700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f293c101840 0x7f293c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:25.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.017+0000 7f2942278700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c19d200 con 0x7f293c100e90 2024-04-23T02:45:25.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.017+0000 7f2942278700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c1966b0 con 0x7f293c101840 2024-04-23T02:45:25.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.017+0000 7f293bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f293c100e90 0x7f293c19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:25.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.017+0000 7f293bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f293c100e90 0x7f293c19c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56728/0 (socket says 172.21.15.71:56728) 2024-04-23T02:45:25.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.017+0000 7f293bfff700 1 -- 172.21.15.71:0/3009105417 learned_addr learned my addr 172.21.15.71:0/3009105417 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:45:25.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.017+0000 7f293b7fe700 1 --2- 172.21.15.71:0/3009105417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f293c101840 0x7f293c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:25.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.017+0000 7f293bfff700 1 -- 172.21.15.71:0/3009105417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f293c101840 msgr2=0x7f293c19cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:25.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.017+0000 7f293bfff700 1 --2- 172.21.15.71:0/3009105417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f293c101840 0x7f293c19cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.017+0000 7f293bfff700 1 -- 172.21.15.71:0/3009105417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29240095f0 con 0x7f293c100e90 2024-04-23T02:45:25.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.017+0000 7f293bfff700 1 --2- 172.21.15.71:0/3009105417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f293c100e90 0x7f293c19c630 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f292c00d900 tx=0x7f292c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:25.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.017+0000 7f29397fa700 1 -- 172.21.15.71:0/3009105417 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f292c0041f0 con 0x7f293c100e90 2024-04-23T02:45:25.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.017+0000 7f29397fa700 1 -- 172.21.15.71:0/3009105417 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f292c004cd0 con 0x7f293c100e90 2024-04-23T02:45:25.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.017+0000 7f2942278700 1 -- 172.21.15.71:0/3009105417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293c196990 con 0x7f293c100e90 2024-04-23T02:45:25.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.017+0000 7f29397fa700 1 -- 172.21.15.71:0/3009105417 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f292c005660 con 0x7f293c100e90 2024-04-23T02:45:25.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.018+0000 7f2942278700 1 -- 172.21.15.71:0/3009105417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f293c196ee0 con 0x7f293c100e90 2024-04-23T02:45:25.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.019+0000 7f2942278700 1 -- 172.21.15.71:0/3009105417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f293c109290 con 0x7f293c100e90 2024-04-23T02:45:25.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.019+0000 7f29397fa700 1 -- 172.21.15.71:0/3009105417 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f292c004e40 con 0x7f293c100e90 2024-04-23T02:45:25.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.019+0000 7f29397fa700 1 --2- 172.21.15.71:0/3009105417 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f292806cce0 0x7f292806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:25.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.020+0000 7f293b7fe700 1 --2- 172.21.15.71:0/3009105417 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f292806cce0 0x7f292806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:25.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.020+0000 7f29397fa700 1 -- 172.21.15.71:0/3009105417 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f292c08c3c0 con 0x7f293c100e90 2024-04-23T02:45:25.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.020+0000 7f293b7fe700 1 --2- 172.21.15.71:0/3009105417 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f292806cce0 0x7f292806f1a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f293c0fcfb0 tx=0x7f292401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:25.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.024+0000 7f29397fa700 1 -- 172.21.15.71:0/3009105417 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f292c059ec0 con 0x7f293c100e90 2024-04-23T02:45:25.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.229+0000 7f2942278700 1 -- 172.21.15.71:0/3009105417 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f293c197100 con 0x7f292806cce0 2024-04-23T02:45:25.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.244+0000 7f29397fa700 1 -- 172.21.15.71:0/3009105417 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f293c197100 con 0x7f292806cce0 2024-04-23T02:45:25.248 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:45:25.248 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (29m) 7m ago 40m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:45:25.248 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (40m) 7m ago 40m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:45:25.248 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (39m) 7m ago 39m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:45:25.248 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (27m) 7m ago 40m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:45:25.248 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (26m) 7m ago 39m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:45:25.248 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (28m) 7m ago 40m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:45:25.248 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (23m) 7m ago 34m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:45:25.248 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (22m) 7m ago 34m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:45:25.249 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (7m) 7m ago 34m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:45:25.249 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (22m) 7m ago 34m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:45:25.249 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (31m) 7m ago 42m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:45:25.249 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (30m) 7m ago 39m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:45:25.249 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (27m) 7m ago 42m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:45:25.249 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (27m) 7m ago 39m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:45:25.249 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (30m) 7m ago 40m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:45:25.249 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (29m) 7m ago 39m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:45:25.249 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (26m) 7m ago 38m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:45:25.249 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (25m) 7m ago 37m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:45:25.249 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (25m) 7m ago 37m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:45:25.249 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (24m) 7m ago 36m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:45:25.249 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (24m) 7m ago 36m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:45:25.249 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (23m) 7m ago 36m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:45:25.249 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (29m) 7m ago 39m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:45:25.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.247+0000 7f2942278700 1 -- 172.21.15.71:0/3009105417 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f292806cce0 msgr2=0x7f292806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:25.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.247+0000 7f2942278700 1 --2- 172.21.15.71:0/3009105417 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f292806cce0 0x7f292806f1a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f293c0fcfb0 tx=0x7f292401a040 comp rx=0 tx=0).stop 2024-04-23T02:45:25.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.247+0000 7f2942278700 1 -- 172.21.15.71:0/3009105417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f293c100e90 msgr2=0x7f293c19c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:25.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.247+0000 7f2942278700 1 --2- 172.21.15.71:0/3009105417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f293c100e90 0x7f293c19c630 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f292c00d900 tx=0x7f292c00dc10 comp rx=0 tx=0).stop 2024-04-23T02:45:25.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.247+0000 7f2942278700 1 -- 172.21.15.71:0/3009105417 shutdown_connections 2024-04-23T02:45:25.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.247+0000 7f2942278700 1 --2- 172.21.15.71:0/3009105417 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f292806cce0 0x7f292806f1a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.247+0000 7f2942278700 1 --2- 172.21.15.71:0/3009105417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f293c100e90 0x7f293c19c630 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.247+0000 7f2942278700 1 --2- 172.21.15.71:0/3009105417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f293c101840 0x7f293c19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.247+0000 7f2942278700 1 -- 172.21.15.71:0/3009105417 >> 172.21.15.71:0/3009105417 conn(0x7f293c0fc740 msgr2=0x7f293c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:25.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.247+0000 7f2942278700 1 -- 172.21.15.71:0/3009105417 shutdown_connections 2024-04-23T02:45:25.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.247+0000 7f2942278700 1 -- 172.21.15.71:0/3009105417 wait complete. 2024-04-23T02:45:25.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.350+0000 7f4653eea700 1 -- 172.21.15.71:0/3502867174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f464c0ff5a0 msgr2=0x7f464c0ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:25.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.350+0000 7f4653eea700 1 --2- 172.21.15.71:0/3502867174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f464c0ff5a0 0x7f464c0ff980 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4640009910 tx=0x7f4640009c20 comp rx=0 tx=0).stop 2024-04-23T02:45:25.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.350+0000 7f4653eea700 1 -- 172.21.15.71:0/3502867174 shutdown_connections 2024-04-23T02:45:25.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.350+0000 7f4653eea700 1 --2- 172.21.15.71:0/3502867174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f464c0fff50 0x7f464c105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.350+0000 7f4653eea700 1 --2- 172.21.15.71:0/3502867174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f464c0ff5a0 0x7f464c0ff980 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.350+0000 7f4653eea700 1 -- 172.21.15.71:0/3502867174 >> 172.21.15.71:0/3502867174 conn(0x7f464c074e30 msgr2=0x7f464c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:25.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.350+0000 7f4653eea700 1 -- 172.21.15.71:0/3502867174 shutdown_connections 2024-04-23T02:45:25.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.350+0000 7f4653eea700 1 -- 172.21.15.71:0/3502867174 wait complete. 2024-04-23T02:45:25.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.350+0000 7f4653eea700 1 Processor -- start 2024-04-23T02:45:25.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f4653eea700 1 -- start start 2024-04-23T02:45:25.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f4653eea700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f464c0ff5a0 0x7f464c100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:25.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f4653eea700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f464c0fff50 0x7f464c100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:25.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f4653eea700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c100fa0 con 0x7f464c0fff50 2024-04-23T02:45:25.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f4653eea700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c1010e0 con 0x7f464c0ff5a0 2024-04-23T02:45:25.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f4651485700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f464c0fff50 0x7f464c100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:25.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f4651485700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f464c0fff50 0x7f464c100a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45232/0 (socket says 172.21.15.71:45232) 2024-04-23T02:45:25.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f4651485700 1 -- 172.21.15.71:0/751317665 learned_addr learned my addr 172.21.15.71:0/751317665 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:45:25.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f4651c86700 1 --2- 172.21.15.71:0/751317665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f464c0ff5a0 0x7f464c100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:25.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f4651485700 1 -- 172.21.15.71:0/751317665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f464c0ff5a0 msgr2=0x7f464c100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:25.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f4651485700 1 --2- 172.21.15.71:0/751317665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f464c0ff5a0 0x7f464c100520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f4651485700 1 -- 172.21.15.71:0/751317665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46400095f0 con 0x7f464c0fff50 2024-04-23T02:45:25.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f4651c86700 1 --2- 172.21.15.71:0/751317665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f464c0ff5a0 0x7f464c100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:45:25.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f4651485700 1 --2- 172.21.15.71:0/751317665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f464c0fff50 0x7f464c100a60 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f464800c960 tx=0x7f464800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:25.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f463effd700 1 -- 172.21.15.71:0/751317665 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4648007a70 con 0x7f464c0fff50 2024-04-23T02:45:25.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f463effd700 1 -- 172.21.15.71:0/751317665 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4648007bd0 con 0x7f464c0fff50 2024-04-23T02:45:25.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f4653eea700 1 -- 172.21.15.71:0/751317665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464c1a29a0 con 0x7f464c0fff50 2024-04-23T02:45:25.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f463effd700 1 -- 172.21.15.71:0/751317665 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4648018630 con 0x7f464c0fff50 2024-04-23T02:45:25.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.351+0000 7f4653eea700 1 -- 172.21.15.71:0/751317665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464c1a2d00 con 0x7f464c0fff50 2024-04-23T02:45:25.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.352+0000 7f4653eea700 1 -- 172.21.15.71:0/751317665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f464c109270 con 0x7f464c0fff50 2024-04-23T02:45:25.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.353+0000 7f463effd700 1 -- 172.21.15.71:0/751317665 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f464800f9f0 con 0x7f464c0fff50 2024-04-23T02:45:25.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.353+0000 7f463effd700 1 --2- 172.21.15.71:0/751317665 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f463806cce0 0x7f463806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:25.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.353+0000 7f463effd700 1 -- 172.21.15.71:0/751317665 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f464808c560 con 0x7f464c0fff50 2024-04-23T02:45:25.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.353+0000 7f4651c86700 1 --2- 172.21.15.71:0/751317665 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f463806cce0 0x7f463806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:25.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.354+0000 7f4651c86700 1 --2- 172.21.15.71:0/751317665 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f463806cce0 0x7f463806f1a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4640006010 tx=0x7f464001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:25.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.358+0000 7f463effd700 1 -- 172.21.15.71:0/751317665 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f464805a0e0 con 0x7f464c0fff50 2024-04-23T02:45:25.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.634+0000 7f4653eea700 1 -- 172.21.15.71:0/751317665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f464c04f2e0 con 0x7f464c0fff50 2024-04-23T02:45:25.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.634+0000 7f463effd700 1 -- 172.21.15.71:0/751317665 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f4648059c70 con 0x7f464c0fff50 2024-04-23T02:45:25.638 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:45:25.638 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:45:25.638 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:45:25.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:45:25.638 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:45:25.638 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:45:25.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:45:25.638 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:45:25.639 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:45:25.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:45:25.639 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:45:25.639 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:45:25.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:45:25.639 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:45:25.639 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:45:25.639 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:45:25.639 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:45:25.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.636+0000 7f4653eea700 1 -- 172.21.15.71:0/751317665 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f463806cce0 msgr2=0x7f463806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:25.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.636+0000 7f4653eea700 1 --2- 172.21.15.71:0/751317665 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f463806cce0 0x7f463806f1a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4640006010 tx=0x7f464001a040 comp rx=0 tx=0).stop 2024-04-23T02:45:25.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.636+0000 7f4653eea700 1 -- 172.21.15.71:0/751317665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f464c0fff50 msgr2=0x7f464c100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:25.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.636+0000 7f4653eea700 1 --2- 172.21.15.71:0/751317665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f464c0fff50 0x7f464c100a60 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f464800c960 tx=0x7f464800cd20 comp rx=0 tx=0).stop 2024-04-23T02:45:25.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.636+0000 7f4653eea700 1 -- 172.21.15.71:0/751317665 shutdown_connections 2024-04-23T02:45:25.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.636+0000 7f4653eea700 1 --2- 172.21.15.71:0/751317665 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f463806cce0 0x7f463806f1a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.636+0000 7f4653eea700 1 --2- 172.21.15.71:0/751317665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f464c0ff5a0 0x7f464c100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.636+0000 7f4653eea700 1 --2- 172.21.15.71:0/751317665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f464c0fff50 0x7f464c100a60 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.636+0000 7f4653eea700 1 -- 172.21.15.71:0/751317665 >> 172.21.15.71:0/751317665 conn(0x7f464c074e30 msgr2=0x7f464c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:25.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.636+0000 7f4653eea700 1 -- 172.21.15.71:0/751317665 shutdown_connections 2024-04-23T02:45:25.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.636+0000 7f4653eea700 1 -- 172.21.15.71:0/751317665 wait complete. 2024-04-23T02:45:25.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.738+0000 7f7c3a25e700 1 -- 172.21.15.71:0/1170335554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c34101850 msgr2=0x7f7c341058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:25.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.738+0000 7f7c3a25e700 1 --2- 172.21.15.71:0/1170335554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c34101850 0x7f7c341058a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009910 tx=0x7f7c28009c20 comp rx=0 tx=0).stop 2024-04-23T02:45:25.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.738+0000 7f7c3a25e700 1 -- 172.21.15.71:0/1170335554 shutdown_connections 2024-04-23T02:45:25.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.738+0000 7f7c3a25e700 1 --2- 172.21.15.71:0/1170335554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c34101850 0x7f7c341058a0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.738+0000 7f7c3a25e700 1 --2- 172.21.15.71:0/1170335554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c34100ea0 0x7f7c34101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.738+0000 7f7c3a25e700 1 -- 172.21.15.71:0/1170335554 >> 172.21.15.71:0/1170335554 conn(0x7f7c340fc750 msgr2=0x7f7c340feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:25.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.738+0000 7f7c3a25e700 1 -- 172.21.15.71:0/1170335554 shutdown_connections 2024-04-23T02:45:25.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.738+0000 7f7c3a25e700 1 -- 172.21.15.71:0/1170335554 wait complete. 2024-04-23T02:45:25.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.739+0000 7f7c3a25e700 1 Processor -- start 2024-04-23T02:45:25.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.739+0000 7f7c3a25e700 1 -- start start 2024-04-23T02:45:25.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.739+0000 7f7c3a25e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c34100ea0 0x7f7c3419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:25.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.739+0000 7f7c3a25e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c34101850 0x7f7c3419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:25.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.739+0000 7f7c3a25e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3419d160 con 0x7f7c34100ea0 2024-04-23T02:45:25.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.739+0000 7f7c3a25e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c341966a0 con 0x7f7c34101850 2024-04-23T02:45:25.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.739+0000 7f7c337fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c34100ea0 0x7f7c3419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:25.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.739+0000 7f7c337fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c34100ea0 0x7f7c3419c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45250/0 (socket says 172.21.15.71:45250) 2024-04-23T02:45:25.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.739+0000 7f7c337fe700 1 -- 172.21.15.71:0/2195571983 learned_addr learned my addr 172.21.15.71:0/2195571983 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:45:25.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.740+0000 7f7c337fe700 1 -- 172.21.15.71:0/2195571983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c34101850 msgr2=0x7f7c3419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:25.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.740+0000 7f7c337fe700 1 --2- 172.21.15.71:0/2195571983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c34101850 0x7f7c3419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.740+0000 7f7c337fe700 1 -- 172.21.15.71:0/2195571983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c280095f0 con 0x7f7c34100ea0 2024-04-23T02:45:25.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.740+0000 7f7c337fe700 1 --2- 172.21.15.71:0/2195571983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c34100ea0 0x7f7c3419c620 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400c930 tx=0x7f7c2400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:25.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.740+0000 7f7c30ff9700 1 -- 172.21.15.71:0/2195571983 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c24007ad0 con 0x7f7c34100ea0 2024-04-23T02:45:25.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.740+0000 7f7c30ff9700 1 -- 172.21.15.71:0/2195571983 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7c24007c30 con 0x7f7c34100ea0 2024-04-23T02:45:25.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.740+0000 7f7c3a25e700 1 -- 172.21.15.71:0/2195571983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c34196980 con 0x7f7c34100ea0 2024-04-23T02:45:25.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.740+0000 7f7c30ff9700 1 -- 172.21.15.71:0/2195571983 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c24018630 con 0x7f7c34100ea0 2024-04-23T02:45:25.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.740+0000 7f7c3a25e700 1 -- 172.21.15.71:0/2195571983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c34196ed0 con 0x7f7c34100ea0 2024-04-23T02:45:25.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.741+0000 7f7c3a25e700 1 -- 172.21.15.71:0/2195571983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c341092a0 con 0x7f7c34100ea0 2024-04-23T02:45:25.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.742+0000 7f7c30ff9700 1 -- 172.21.15.71:0/2195571983 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7c2400f450 con 0x7f7c34100ea0 2024-04-23T02:45:25.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.743+0000 7f7c30ff9700 1 --2- 172.21.15.71:0/2195571983 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7c1c06cce0 0x7f7c1c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:25.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.743+0000 7f7c30ff9700 1 -- 172.21.15.71:0/2195571983 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7c2408c920 con 0x7f7c34100ea0 2024-04-23T02:45:25.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.743+0000 7f7c32ffd700 1 --2- 172.21.15.71:0/2195571983 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7c1c06cce0 0x7f7c1c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:25.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.743+0000 7f7c32ffd700 1 --2- 172.21.15.71:0/2195571983 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7c1c06cce0 0x7f7c1c06f1a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800b5c0 tx=0x7f7c28005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:25.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.747+0000 7f7c30ff9700 1 -- 172.21.15.71:0/2195571983 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7c2405a480 con 0x7f7c34100ea0 2024-04-23T02:45:25.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:25 smithi192 ceph-mon[108617]: pgmap v964: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:25.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:25 smithi192 ceph-mon[108617]: from='client.35548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:45:25.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.982+0000 7f7c3a25e700 1 -- 172.21.15.71:0/2195571983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7c34197cb0 con 0x7f7c34100ea0 2024-04-23T02:45:25.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.982+0000 7f7c30ff9700 1 -- 172.21.15.71:0/2195571983 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f7c34197cb0 con 0x7f7c34100ea0 2024-04-23T02:45:25.986 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:45:25.986 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:45:25.987 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:45:25.987 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:45:25.987 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:45:25.987 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:45:25.987 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:45:25.987 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:45:25.987 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:45:25.987 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:45:25.987 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:45:25.987 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:45:25.987 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:45:25.987 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:45:25.987 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:45:25.987 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:45:25.987 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:45:25.987 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:45:25.987 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:45:25.988 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:45:25.988 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:45:25.988 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:45:25.988 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:45:25.988 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:45:25.988 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:45:25.988 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:45:25.988 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:45:25.988 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:45:25.988 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:45:25.988 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:45:25.988 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:45:25.988 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:45:25.988 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:45:25.988 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:45:25.989 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:45:25.989 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:45:25.989 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:45:25.989 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:45:25.989 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:45:25.989 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:45:25.989 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:45:25.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.984+0000 7f7c3a25e700 1 -- 172.21.15.71:0/2195571983 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7c1c06cce0 msgr2=0x7f7c1c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:25.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.984+0000 7f7c3a25e700 1 --2- 172.21.15.71:0/2195571983 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7c1c06cce0 0x7f7c1c06f1a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800b5c0 tx=0x7f7c28005960 comp rx=0 tx=0).stop 2024-04-23T02:45:25.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.984+0000 7f7c3a25e700 1 -- 172.21.15.71:0/2195571983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c34100ea0 msgr2=0x7f7c3419c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:25.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.984+0000 7f7c3a25e700 1 --2- 172.21.15.71:0/2195571983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c34100ea0 0x7f7c3419c620 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400c930 tx=0x7f7c2400ccf0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.984+0000 7f7c3a25e700 1 -- 172.21.15.71:0/2195571983 shutdown_connections 2024-04-23T02:45:25.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.984+0000 7f7c3a25e700 1 --2- 172.21.15.71:0/2195571983 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7c1c06cce0 0x7f7c1c06f1a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.984+0000 7f7c3a25e700 1 --2- 172.21.15.71:0/2195571983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c34100ea0 0x7f7c3419c620 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.984+0000 7f7c3a25e700 1 --2- 172.21.15.71:0/2195571983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c34101850 0x7f7c3419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:25.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.984+0000 7f7c3a25e700 1 -- 172.21.15.71:0/2195571983 >> 172.21.15.71:0/2195571983 conn(0x7f7c340fc750 msgr2=0x7f7c340fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:25.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.985+0000 7f7c3a25e700 1 -- 172.21.15.71:0/2195571983 shutdown_connections 2024-04-23T02:45:25.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:25.985+0000 7f7c3a25e700 1 -- 172.21.15.71:0/2195571983 wait complete. 2024-04-23T02:45:25.990 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:45:25.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:25 smithi071 ceph-mon[144544]: pgmap v964: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:25.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:25 smithi071 ceph-mon[144544]: from='client.35548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:45:26.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.087+0000 7f3f01354700 1 -- 172.21.15.71:0/716772960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3efc101850 msgr2=0x7f3efc1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:26.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.087+0000 7f3f01354700 1 --2- 172.21.15.71:0/716772960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3efc101850 0x7f3efc1058a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009910 tx=0x7f3ef0009c20 comp rx=0 tx=0).stop 2024-04-23T02:45:26.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.088+0000 7f3f01354700 1 -- 172.21.15.71:0/716772960 shutdown_connections 2024-04-23T02:45:26.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.088+0000 7f3f01354700 1 --2- 172.21.15.71:0/716772960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3efc101850 0x7f3efc1058a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:26.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.088+0000 7f3f01354700 1 --2- 172.21.15.71:0/716772960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3efc100ea0 0x7f3efc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:26.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.088+0000 7f3f01354700 1 -- 172.21.15.71:0/716772960 >> 172.21.15.71:0/716772960 conn(0x7f3efc0fc750 msgr2=0x7f3efc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:26.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.088+0000 7f3f01354700 1 -- 172.21.15.71:0/716772960 shutdown_connections 2024-04-23T02:45:26.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.088+0000 7f3f01354700 1 -- 172.21.15.71:0/716772960 wait complete. 2024-04-23T02:45:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.088+0000 7f3f01354700 1 Processor -- start 2024-04-23T02:45:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.088+0000 7f3f01354700 1 -- start start 2024-04-23T02:45:26.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.089+0000 7f3f01354700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3efc100ea0 0x7f3efc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:26.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.089+0000 7f3f01354700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3efc101850 0x7f3efc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:26.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.089+0000 7f3f01354700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc073b10 con 0x7f3efc100ea0 2024-04-23T02:45:26.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.089+0000 7f3f01354700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc073c50 con 0x7f3efc101850 2024-04-23T02:45:26.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.089+0000 7f3efaffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3efc100ea0 0x7f3efc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:26.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.089+0000 7f3efaffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3efc100ea0 0x7f3efc073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45268/0 (socket says 172.21.15.71:45268) 2024-04-23T02:45:26.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.089+0000 7f3efaffd700 1 -- 172.21.15.71:0/3443406404 learned_addr learned my addr 172.21.15.71:0/3443406404 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:45:26.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.089+0000 7f3efa7fc700 1 --2- 172.21.15.71:0/3443406404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3efc101850 0x7f3efc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:26.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.089+0000 7f3efaffd700 1 -- 172.21.15.71:0/3443406404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3efc101850 msgr2=0x7f3efc0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:26.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.089+0000 7f3efaffd700 1 --2- 172.21.15.71:0/3443406404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3efc101850 0x7f3efc0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:26.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.089+0000 7f3efaffd700 1 -- 172.21.15.71:0/3443406404 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef00095f0 con 0x7f3efc100ea0 2024-04-23T02:45:26.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.089+0000 7f3efaffd700 1 --2- 172.21.15.71:0/3443406404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3efc100ea0 0x7f3efc073090 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00d8d0 tx=0x7f3eec00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:26.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.089+0000 7f3ee3fff700 1 -- 172.21.15.71:0/3443406404 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3eec0098a0 con 0x7f3efc100ea0 2024-04-23T02:45:26.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.089+0000 7f3f01354700 1 -- 172.21.15.71:0/3443406404 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3efc1a2a40 con 0x7f3efc100ea0 2024-04-23T02:45:26.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.089+0000 7f3f01354700 1 -- 172.21.15.71:0/3443406404 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3efc1a2df0 con 0x7f3efc100ea0 2024-04-23T02:45:26.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.089+0000 7f3ee3fff700 1 -- 172.21.15.71:0/3443406404 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3eec010460 con 0x7f3efc100ea0 2024-04-23T02:45:26.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.089+0000 7f3ee3fff700 1 -- 172.21.15.71:0/3443406404 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3eec009ea0 con 0x7f3efc100ea0 2024-04-23T02:45:26.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.090+0000 7f3f01354700 1 -- 172.21.15.71:0/3443406404 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee8005320 con 0x7f3efc100ea0 2024-04-23T02:45:26.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.091+0000 7f3ee3fff700 1 -- 172.21.15.71:0/3443406404 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3eec00f5f0 con 0x7f3efc100ea0 2024-04-23T02:45:26.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.091+0000 7f3ee3fff700 1 --2- 172.21.15.71:0/3443406404 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3ee406cce0 0x7f3ee406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:26.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.091+0000 7f3ee3fff700 1 -- 172.21.15.71:0/3443406404 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3eec08c3c0 con 0x7f3efc100ea0 2024-04-23T02:45:26.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.091+0000 7f3efa7fc700 1 --2- 172.21.15.71:0/3443406404 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3ee406cce0 0x7f3ee406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:26.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.092+0000 7f3efa7fc700 1 --2- 172.21.15.71:0/3443406404 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3ee406cce0 0x7f3ee406f1a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0006010 tx=0x7f3ef001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:26.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.096+0000 7f3ee3fff700 1 -- 172.21.15.71:0/3443406404 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3eec056430 con 0x7f3efc100ea0 2024-04-23T02:45:26.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.306+0000 7f3f01354700 1 -- 172.21.15.71:0/3443406404 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ee8000bf0 con 0x7f3ee406cce0 2024-04-23T02:45:26.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.308+0000 7f3ee3fff700 1 -- 172.21.15.71:0/3443406404 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3ee8000bf0 con 0x7f3ee406cce0 2024-04-23T02:45:26.311 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:45:26.311 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:45:26.311 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:45:26.311 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:45:26.312 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:45:26.312 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:45:26.312 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:45:26.312 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:45:26.312 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:45:26.312 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:45:26.312 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:45:26.312 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:45:26.312 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:45:26.312 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:45:26.312 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:45:26.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.309+0000 7f3f01354700 1 -- 172.21.15.71:0/3443406404 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3ee406cce0 msgr2=0x7f3ee406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:26.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.310+0000 7f3f01354700 1 --2- 172.21.15.71:0/3443406404 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3ee406cce0 0x7f3ee406f1a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0006010 tx=0x7f3ef001a040 comp rx=0 tx=0).stop 2024-04-23T02:45:26.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.310+0000 7f3f01354700 1 -- 172.21.15.71:0/3443406404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3efc100ea0 msgr2=0x7f3efc073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:26.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.310+0000 7f3f01354700 1 --2- 172.21.15.71:0/3443406404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3efc100ea0 0x7f3efc073090 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00d8d0 tx=0x7f3eec00dbe0 comp rx=0 tx=0).stop 2024-04-23T02:45:26.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.310+0000 7f3f01354700 1 -- 172.21.15.71:0/3443406404 shutdown_connections 2024-04-23T02:45:26.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.310+0000 7f3f01354700 1 --2- 172.21.15.71:0/3443406404 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3ee406cce0 0x7f3ee406f1a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:26.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.310+0000 7f3f01354700 1 --2- 172.21.15.71:0/3443406404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3efc100ea0 0x7f3efc073090 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:26.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.310+0000 7f3f01354700 1 --2- 172.21.15.71:0/3443406404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3efc101850 0x7f3efc0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:26.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.310+0000 7f3f01354700 1 -- 172.21.15.71:0/3443406404 >> 172.21.15.71:0/3443406404 conn(0x7f3efc0fc750 msgr2=0x7f3efc0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:26.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.310+0000 7f3f01354700 1 -- 172.21.15.71:0/3443406404 shutdown_connections 2024-04-23T02:45:26.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.310+0000 7f3f01354700 1 -- 172.21.15.71:0/3443406404 wait complete. 2024-04-23T02:45:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.412+0000 7f181aa8c700 1 -- 172.21.15.71:0/254948771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1814100750 msgr2=0x7f18141047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.412+0000 7f181aa8c700 1 --2- 172.21.15.71:0/254948771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1814100750 0x7f18141047a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009910 tx=0x7f17fc009c20 comp rx=0 tx=0).stop 2024-04-23T02:45:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.412+0000 7f181aa8c700 1 -- 172.21.15.71:0/254948771 shutdown_connections 2024-04-23T02:45:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.412+0000 7f181aa8c700 1 --2- 172.21.15.71:0/254948771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1814100750 0x7f18141047a0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.412+0000 7f181aa8c700 1 --2- 172.21.15.71:0/254948771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18140ffda0 0x7f1814100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.412+0000 7f181aa8c700 1 -- 172.21.15.71:0/254948771 >> 172.21.15.71:0/254948771 conn(0x7f18140fb650 msgr2=0x7f18140fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.412+0000 7f181aa8c700 1 -- 172.21.15.71:0/254948771 shutdown_connections 2024-04-23T02:45:26.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.412+0000 7f181aa8c700 1 -- 172.21.15.71:0/254948771 wait complete. 2024-04-23T02:45:26.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.413+0000 7f181aa8c700 1 Processor -- start 2024-04-23T02:45:26.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.413+0000 7f181aa8c700 1 -- start start 2024-04-23T02:45:26.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.413+0000 7f181aa8c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18140ffda0 0x7f181419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.413+0000 7f181aa8c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1814100750 0x7f181419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.413+0000 7f181aa8c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181419c0f0 con 0x7f1814100750 2024-04-23T02:45:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.413+0000 7f181aa8c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18141955a0 con 0x7f18140ffda0 2024-04-23T02:45:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.413+0000 7f1813fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1814100750 0x7f181419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.413+0000 7f1813fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1814100750 0x7f181419ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45288/0 (socket says 172.21.15.71:45288) 2024-04-23T02:45:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.413+0000 7f1813fff700 1 -- 172.21.15.71:0/1500295484 learned_addr learned my addr 172.21.15.71:0/1500295484 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:45:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.413+0000 7f1818828700 1 --2- 172.21.15.71:0/1500295484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18140ffda0 0x7f181419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.413+0000 7f1813fff700 1 -- 172.21.15.71:0/1500295484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18140ffda0 msgr2=0x7f181419b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.413+0000 7f1813fff700 1 --2- 172.21.15.71:0/1500295484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18140ffda0 0x7f181419b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.413+0000 7f1813fff700 1 -- 172.21.15.71:0/1500295484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc0095f0 con 0x7f1814100750 2024-04-23T02:45:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.414+0000 7f1813fff700 1 --2- 172.21.15.71:0/1500295484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1814100750 0x7f181419ba60 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0092e0 tx=0x7f17fc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.414+0000 7f1811ffb700 1 -- 172.21.15.71:0/1500295484 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17fc01d070 con 0x7f1814100750 2024-04-23T02:45:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.414+0000 7f1811ffb700 1 -- 172.21.15.71:0/1500295484 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f17fc005dc0 con 0x7f1814100750 2024-04-23T02:45:26.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.414+0000 7f181aa8c700 1 -- 172.21.15.71:0/1500295484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1814195820 con 0x7f1814100750 2024-04-23T02:45:26.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.414+0000 7f1811ffb700 1 -- 172.21.15.71:0/1500295484 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17fc00fea0 con 0x7f1814100750 2024-04-23T02:45:26.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.414+0000 7f181aa8c700 1 -- 172.21.15.71:0/1500295484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1814195d10 con 0x7f1814100750 2024-04-23T02:45:26.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.415+0000 7f181aa8c700 1 -- 172.21.15.71:0/1500295484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18141081a0 con 0x7f1814100750 2024-04-23T02:45:26.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.415+0000 7f1811ffb700 1 -- 172.21.15.71:0/1500295484 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f17fc003a60 con 0x7f1814100750 2024-04-23T02:45:26.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.416+0000 7f1811ffb700 1 --2- 172.21.15.71:0/1500295484 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f180006cce0 0x7f180006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:26.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.416+0000 7f1811ffb700 1 -- 172.21.15.71:0/1500295484 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f17fc08d6c0 con 0x7f1814100750 2024-04-23T02:45:26.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.416+0000 7f1818828700 1 --2- 172.21.15.71:0/1500295484 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f180006cce0 0x7f180006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:26.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.416+0000 7f1818828700 1 --2- 172.21.15.71:0/1500295484 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f180006cce0 0x7f180006f1a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f180c009550 tx=0x7f180c0091b0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:26.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.420+0000 7f1811ffb700 1 -- 172.21.15.71:0/1500295484 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f17fc05b240 con 0x7f1814100750 2024-04-23T02:45:26.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.692+0000 7f181aa8c700 1 -- 172.21.15.71:0/1500295484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1814196b90 con 0x7f1814100750 2024-04-23T02:45:26.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.692+0000 7f1811ffb700 1 -- 172.21.15.71:0/1500295484 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7f1814196b90 con 0x7f1814100750 2024-04-23T02:45:26.696 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:45:26.696 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:45:26.696 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:45:26.696 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:45:26.696 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:45:26.696 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:45:26.696 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:45:26.696 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:45:26.696 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:45:26.696 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:45:26.696 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:45:26.697 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:45:26.697 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:45:26.697 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:45:26.697 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:45:26.697 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:45:26.697 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:45:26.697 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:45:26.697 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:45:26.697 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:45:26.697 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:45:26.697 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:45:26.697 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:45:26.697 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:45:26.698 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:45:26.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.694+0000 7f181aa8c700 1 -- 172.21.15.71:0/1500295484 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f180006cce0 msgr2=0x7f180006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:26.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.694+0000 7f181aa8c700 1 --2- 172.21.15.71:0/1500295484 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f180006cce0 0x7f180006f1a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f180c009550 tx=0x7f180c0091b0 comp rx=0 tx=0).stop 2024-04-23T02:45:26.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.694+0000 7f181aa8c700 1 -- 172.21.15.71:0/1500295484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1814100750 msgr2=0x7f181419ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:26.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.694+0000 7f181aa8c700 1 --2- 172.21.15.71:0/1500295484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1814100750 0x7f181419ba60 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0092e0 tx=0x7f17fc00bd70 comp rx=0 tx=0).stop 2024-04-23T02:45:26.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.694+0000 7f181aa8c700 1 -- 172.21.15.71:0/1500295484 shutdown_connections 2024-04-23T02:45:26.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.694+0000 7f181aa8c700 1 --2- 172.21.15.71:0/1500295484 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f180006cce0 0x7f180006f1a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:26.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.694+0000 7f181aa8c700 1 --2- 172.21.15.71:0/1500295484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18140ffda0 0x7f181419b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:26.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.694+0000 7f181aa8c700 1 --2- 172.21.15.71:0/1500295484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1814100750 0x7f181419ba60 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:26.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.694+0000 7f181aa8c700 1 -- 172.21.15.71:0/1500295484 >> 172.21.15.71:0/1500295484 conn(0x7f18140fb650 msgr2=0x7f18140fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:26.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.694+0000 7f181aa8c700 1 -- 172.21.15.71:0/1500295484 shutdown_connections 2024-04-23T02:45:26.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:26.694+0000 7f181aa8c700 1 -- 172.21.15.71:0/1500295484 wait complete. 2024-04-23T02:45:26.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:26 smithi192 ceph-mon[108617]: from='client.35552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:45:26.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:26 smithi192 ceph-mon[108617]: from='client.35556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:45:26.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:26 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/751317665' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:45:26.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:26 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2195571983' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:45:26.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:26 smithi071 ceph-mon[144544]: from='client.35552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:45:26.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:26 smithi071 ceph-mon[144544]: from='client.35556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:45:26.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:26 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/751317665' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:45:26.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:26 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2195571983' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:45:27.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:27 smithi192 ceph-mon[108617]: from='client.35568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:45:27.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:27 smithi192 ceph-mon[108617]: pgmap v965: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:27.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:27 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1500295484' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:45:27.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:27 smithi071 ceph-mon[144544]: from='client.35568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:45:27.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:27 smithi071 ceph-mon[144544]: pgmap v965: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:27.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:27 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1500295484' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:45:29.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:29 smithi192 ceph-mon[108617]: pgmap v966: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:29.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:29 smithi071 ceph-mon[144544]: pgmap v966: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:31.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:31 smithi192 ceph-mon[108617]: pgmap v967: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:31.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:31 smithi071 ceph-mon[144544]: pgmap v967: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:33.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:33 smithi192 ceph-mon[108617]: pgmap v968: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:33.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:33 smithi071 ceph-mon[144544]: pgmap v968: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:35.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:35 smithi192 ceph-mon[108617]: pgmap v969: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:35.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:45:35.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:35 smithi071 ceph-mon[144544]: pgmap v969: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:45:36.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:36 smithi192 ceph-mon[108617]: pgmap v970: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:36.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:36 smithi071 ceph-mon[144544]: pgmap v970: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:39.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:39 smithi192 ceph-mon[108617]: pgmap v971: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:39.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:39 smithi071 ceph-mon[144544]: pgmap v971: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:41.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:41 smithi192 ceph-mon[108617]: pgmap v972: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:41.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:41 smithi071 ceph-mon[144544]: pgmap v972: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:43.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:43 smithi192 ceph-mon[108617]: pgmap v973: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:43.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:43 smithi071 ceph-mon[144544]: pgmap v973: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:45.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:45 smithi192 ceph-mon[108617]: pgmap v974: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:45.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:45 smithi071 ceph-mon[144544]: pgmap v974: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:47.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:47 smithi192 ceph-mon[108617]: pgmap v975: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:47.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:47 smithi071 ceph-mon[144544]: pgmap v975: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:49.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:49 smithi192 ceph-mon[108617]: pgmap v976: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:49.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:49 smithi071 ceph-mon[144544]: pgmap v976: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:50.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:45:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:45:51.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:51 smithi192 ceph-mon[108617]: pgmap v977: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:51.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:51 smithi071 ceph-mon[144544]: pgmap v977: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:52.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:52 smithi192 ceph-mon[108617]: pgmap v978: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:52.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:52 smithi071 ceph-mon[144544]: pgmap v978: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:55.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:55 smithi192 ceph-mon[108617]: pgmap v979: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:55.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:55 smithi071 ceph-mon[144544]: pgmap v979: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:56.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.809+0000 7f33a75c5700 1 -- 172.21.15.71:0/1093375079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33a0100ea0 msgr2=0x7f33a0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:56.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.809+0000 7f33a75c5700 1 --2- 172.21.15.71:0/1093375079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33a0100ea0 0x7f33a0101280 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3394009910 tx=0x7f3394009c20 comp rx=0 tx=0).stop 2024-04-23T02:45:56.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.809+0000 7f33a75c5700 1 -- 172.21.15.71:0/1093375079 shutdown_connections 2024-04-23T02:45:56.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.809+0000 7f33a75c5700 1 --2- 172.21.15.71:0/1093375079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33a0101850 0x7f33a01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:56.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.809+0000 7f33a75c5700 1 --2- 172.21.15.71:0/1093375079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33a0100ea0 0x7f33a0101280 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:56.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.809+0000 7f33a75c5700 1 -- 172.21.15.71:0/1093375079 >> 172.21.15.71:0/1093375079 conn(0x7f33a00fc750 msgr2=0x7f33a00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:56.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.809+0000 7f33a75c5700 1 -- 172.21.15.71:0/1093375079 shutdown_connections 2024-04-23T02:45:56.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.809+0000 7f33a75c5700 1 -- 172.21.15.71:0/1093375079 wait complete. 2024-04-23T02:45:56.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.810+0000 7f33a75c5700 1 Processor -- start 2024-04-23T02:45:56.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.810+0000 7f33a75c5700 1 -- start start 2024-04-23T02:45:56.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.810+0000 7f33a75c5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33a0100ea0 0x7f33a019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:56.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.810+0000 7f33a75c5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33a0101850 0x7f33a019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:56.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.810+0000 7f33a75c5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a019f360 con 0x7f33a0100ea0 2024-04-23T02:45:56.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.810+0000 7f33a75c5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a0198810 con 0x7f33a0101850 2024-04-23T02:45:56.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.810+0000 7f33a5361700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33a0100ea0 0x7f33a019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:56.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.810+0000 7f33a5361700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33a0100ea0 0x7f33a019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51230/0 (socket says 172.21.15.71:51230) 2024-04-23T02:45:56.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.810+0000 7f33a5361700 1 -- 172.21.15.71:0/1902396104 learned_addr learned my addr 172.21.15.71:0/1902396104 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:45:56.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.811+0000 7f33a4b60700 1 --2- 172.21.15.71:0/1902396104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33a0101850 0x7f33a019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:56.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.811+0000 7f33a5361700 1 -- 172.21.15.71:0/1902396104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33a0101850 msgr2=0x7f33a019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:56.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.811+0000 7f33a5361700 1 --2- 172.21.15.71:0/1902396104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33a0101850 0x7f33a019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:56.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.811+0000 7f33a5361700 1 -- 172.21.15.71:0/1902396104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33940095f0 con 0x7f33a0100ea0 2024-04-23T02:45:56.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.811+0000 7f33a5361700 1 --2- 172.21.15.71:0/1902396104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33a0100ea0 0x7f33a019e790 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f33940092e0 tx=0x7f339400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:56.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.811+0000 7f33927fc700 1 -- 172.21.15.71:0/1902396104 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f339401d070 con 0x7f33a0100ea0 2024-04-23T02:45:56.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.811+0000 7f33927fc700 1 -- 172.21.15.71:0/1902396104 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3394005e00 con 0x7f33a0100ea0 2024-04-23T02:45:56.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.811+0000 7f33a75c5700 1 -- 172.21.15.71:0/1902396104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a0198a90 con 0x7f33a0100ea0 2024-04-23T02:45:56.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.811+0000 7f33927fc700 1 -- 172.21.15.71:0/1902396104 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f339400fe60 con 0x7f33a0100ea0 2024-04-23T02:45:56.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.811+0000 7f33a75c5700 1 -- 172.21.15.71:0/1902396104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a0198f80 con 0x7f33a0100ea0 2024-04-23T02:45:56.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.812+0000 7f33a75c5700 1 -- 172.21.15.71:0/1902396104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a0105020 con 0x7f33a0100ea0 2024-04-23T02:45:56.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.812+0000 7f33927fc700 1 -- 172.21.15.71:0/1902396104 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f339400f6d0 con 0x7f33a0100ea0 2024-04-23T02:45:56.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.813+0000 7f33927fc700 1 --2- 172.21.15.71:0/1902396104 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f338c06cce0 0x7f338c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:56.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.813+0000 7f33927fc700 1 -- 172.21.15.71:0/1902396104 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f339408d8b0 con 0x7f33a0100ea0 2024-04-23T02:45:56.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.813+0000 7f33a4b60700 1 --2- 172.21.15.71:0/1902396104 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f338c06cce0 0x7f338c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:56.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.813+0000 7f33a4b60700 1 --2- 172.21.15.71:0/1902396104 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f338c06cce0 0x7f338c06f1a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f339c005950 tx=0x7f339c0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:56.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:56.817+0000 7f33927fc700 1 -- 172.21.15.71:0/1902396104 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f339405b3b0 con 0x7f33a0100ea0 2024-04-23T02:45:57.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.027+0000 7f33a75c5700 1 -- 172.21.15.71:0/1902396104 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f33a01991a0 con 0x7f338c06cce0 2024-04-23T02:45:57.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.028+0000 7f33927fc700 1 -- 172.21.15.71:0/1902396104 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f33a01991a0 con 0x7f338c06cce0 2024-04-23T02:45:57.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.030+0000 7f33a75c5700 1 -- 172.21.15.71:0/1902396104 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f338c06cce0 msgr2=0x7f338c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:57.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.030+0000 7f33a75c5700 1 --2- 172.21.15.71:0/1902396104 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f338c06cce0 0x7f338c06f1a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f339c005950 tx=0x7f339c0058e0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.030+0000 7f33a75c5700 1 -- 172.21.15.71:0/1902396104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33a0100ea0 msgr2=0x7f33a019e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:57.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.030+0000 7f33a75c5700 1 --2- 172.21.15.71:0/1902396104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33a0100ea0 0x7f33a019e790 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f33940092e0 tx=0x7f339400be20 comp rx=0 tx=0).stop 2024-04-23T02:45:57.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.030+0000 7f33a75c5700 1 -- 172.21.15.71:0/1902396104 shutdown_connections 2024-04-23T02:45:57.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.030+0000 7f33a75c5700 1 --2- 172.21.15.71:0/1902396104 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f338c06cce0 0x7f338c06f1a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.030+0000 7f33a75c5700 1 --2- 172.21.15.71:0/1902396104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33a0100ea0 0x7f33a019e790 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.030+0000 7f33a75c5700 1 --2- 172.21.15.71:0/1902396104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33a0101850 0x7f33a019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.030+0000 7f33a75c5700 1 -- 172.21.15.71:0/1902396104 >> 172.21.15.71:0/1902396104 conn(0x7f33a00fc750 msgr2=0x7f33a00fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:57.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.030+0000 7f33a75c5700 1 -- 172.21.15.71:0/1902396104 shutdown_connections 2024-04-23T02:45:57.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.031+0000 7f33a75c5700 1 -- 172.21.15.71:0/1902396104 wait complete. 2024-04-23T02:45:57.043 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:45:57.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.133+0000 7f0532452700 1 -- 172.21.15.71:0/3902403987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f052c106cd0 msgr2=0x7f052c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:57.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.133+0000 7f0532452700 1 --2- 172.21.15.71:0/3902403987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f052c106cd0 0x7f052c1070b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0520009910 tx=0x7f0520009c20 comp rx=0 tx=0).stop 2024-04-23T02:45:57.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.134+0000 7f0532452700 1 -- 172.21.15.71:0/3902403987 shutdown_connections 2024-04-23T02:45:57.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.134+0000 7f0532452700 1 --2- 172.21.15.71:0/3902403987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f052c068580 0x7f052c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.134+0000 7f0532452700 1 --2- 172.21.15.71:0/3902403987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f052c106cd0 0x7f052c1070b0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.134+0000 7f0532452700 1 -- 172.21.15.71:0/3902403987 >> 172.21.15.71:0/3902403987 conn(0x7f052c075670 msgr2=0x7f052c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:57.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.134+0000 7f0532452700 1 -- 172.21.15.71:0/3902403987 shutdown_connections 2024-04-23T02:45:57.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.134+0000 7f0532452700 1 -- 172.21.15.71:0/3902403987 wait complete. 2024-04-23T02:45:57.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.134+0000 7f0532452700 1 Processor -- start 2024-04-23T02:45:57.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.134+0000 7f0532452700 1 -- start start 2024-04-23T02:45:57.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.134+0000 7f0532452700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f052c068580 0x7f052c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:57.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.134+0000 7f0532452700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f052c106cd0 0x7f052c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:57.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.134+0000 7f0532452700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c19d130 con 0x7f052c068580 2024-04-23T02:45:57.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.134+0000 7f0532452700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c196630 con 0x7f052c106cd0 2024-04-23T02:45:57.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.135+0000 7f052bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f052c068580 0x7f052c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:57.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.135+0000 7f052bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f052c068580 0x7f052c19c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51246/0 (socket says 172.21.15.71:51246) 2024-04-23T02:45:57.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.135+0000 7f052bfff700 1 -- 172.21.15.71:0/3480338100 learned_addr learned my addr 172.21.15.71:0/3480338100 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:45:57.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.135+0000 7f052b7fe700 1 --2- 172.21.15.71:0/3480338100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f052c106cd0 0x7f052c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:57.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.135+0000 7f052bfff700 1 -- 172.21.15.71:0/3480338100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f052c106cd0 msgr2=0x7f052c19caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:57.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.135+0000 7f052bfff700 1 --2- 172.21.15.71:0/3480338100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f052c106cd0 0x7f052c19caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.135+0000 7f052bfff700 1 -- 172.21.15.71:0/3480338100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05200095f0 con 0x7f052c068580 2024-04-23T02:45:57.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.135+0000 7f052bfff700 1 --2- 172.21.15.71:0/3480338100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f052c068580 0x7f052c19c560 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0520000c00 tx=0x7f052000be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:57.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.135+0000 7f05297fa700 1 -- 172.21.15.71:0/3480338100 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f052001d070 con 0x7f052c068580 2024-04-23T02:45:57.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.135+0000 7f0532452700 1 -- 172.21.15.71:0/3480338100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052c1968b0 con 0x7f052c068580 2024-04-23T02:45:57.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.135+0000 7f05297fa700 1 -- 172.21.15.71:0/3480338100 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0520005dc0 con 0x7f052c068580 2024-04-23T02:45:57.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.135+0000 7f05297fa700 1 -- 172.21.15.71:0/3480338100 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05200039b0 con 0x7f052c068580 2024-04-23T02:45:57.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.135+0000 7f0532452700 1 -- 172.21.15.71:0/3480338100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c196d40 con 0x7f052c068580 2024-04-23T02:45:57.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.136+0000 7f0532452700 1 -- 172.21.15.71:0/3480338100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052c1091f0 con 0x7f052c068580 2024-04-23T02:45:57.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.137+0000 7f05297fa700 1 -- 172.21.15.71:0/3480338100 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f05200048e0 con 0x7f052c068580 2024-04-23T02:45:57.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.137+0000 7f05297fa700 1 --2- 172.21.15.71:0/3480338100 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f051406cce0 0x7f051406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:57.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.137+0000 7f05297fa700 1 -- 172.21.15.71:0/3480338100 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f052008dc50 con 0x7f052c068580 2024-04-23T02:45:57.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.137+0000 7f052b7fe700 1 --2- 172.21.15.71:0/3480338100 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f051406cce0 0x7f051406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:57.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.137+0000 7f052b7fe700 1 --2- 172.21.15.71:0/3480338100 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f051406cce0 0x7f051406f1a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f052c0fe390 tx=0x7f051c009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:57.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.141+0000 7f05297fa700 1 -- 172.21.15.71:0/3480338100 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f052005b750 con 0x7f052c068580 2024-04-23T02:45:57.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.356+0000 7f0532452700 1 -- 172.21.15.71:0/3480338100 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f052c196fc0 con 0x7f051406cce0 2024-04-23T02:45:57.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.357+0000 7f05297fa700 1 -- 172.21.15.71:0/3480338100 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f052c196fc0 con 0x7f051406cce0 2024-04-23T02:45:57.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.359+0000 7f0532452700 1 -- 172.21.15.71:0/3480338100 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f051406cce0 msgr2=0x7f051406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:57.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.359+0000 7f0532452700 1 --2- 172.21.15.71:0/3480338100 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f051406cce0 0x7f051406f1a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f052c0fe390 tx=0x7f051c009260 comp rx=0 tx=0).stop 2024-04-23T02:45:57.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.359+0000 7f0532452700 1 -- 172.21.15.71:0/3480338100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f052c068580 msgr2=0x7f052c19c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:57.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.359+0000 7f0532452700 1 --2- 172.21.15.71:0/3480338100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f052c068580 0x7f052c19c560 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0520000c00 tx=0x7f052000be90 comp rx=0 tx=0).stop 2024-04-23T02:45:57.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.359+0000 7f0532452700 1 -- 172.21.15.71:0/3480338100 shutdown_connections 2024-04-23T02:45:57.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.359+0000 7f0532452700 1 --2- 172.21.15.71:0/3480338100 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f051406cce0 0x7f051406f1a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.359+0000 7f0532452700 1 --2- 172.21.15.71:0/3480338100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f052c068580 0x7f052c19c560 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.359+0000 7f0532452700 1 --2- 172.21.15.71:0/3480338100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f052c106cd0 0x7f052c19caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.359+0000 7f0532452700 1 -- 172.21.15.71:0/3480338100 >> 172.21.15.71:0/3480338100 conn(0x7f052c075670 msgr2=0x7f052c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:57.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.359+0000 7f0532452700 1 -- 172.21.15.71:0/3480338100 shutdown_connections 2024-04-23T02:45:57.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.359+0000 7f0532452700 1 -- 172.21.15.71:0/3480338100 wait complete. 2024-04-23T02:45:57.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.461+0000 7f0b4bf8e700 1 -- 172.21.15.71:0/4017137563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b44105bd0 msgr2=0x7f0b44105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:57.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.461+0000 7f0b4bf8e700 1 --2- 172.21.15.71:0/4017137563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b44105bd0 0x7f0b44105fb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009910 tx=0x7f0b38009c20 comp rx=0 tx=0).stop 2024-04-23T02:45:57.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.461+0000 7f0b4bf8e700 1 -- 172.21.15.71:0/4017137563 shutdown_connections 2024-04-23T02:45:57.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.461+0000 7f0b4bf8e700 1 --2- 172.21.15.71:0/4017137563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b440ffc20 0x7f0b44100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.461+0000 7f0b4bf8e700 1 --2- 172.21.15.71:0/4017137563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b44105bd0 0x7f0b44105fb0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.461+0000 7f0b4bf8e700 1 -- 172.21.15.71:0/4017137563 >> 172.21.15.71:0/4017137563 conn(0x7f0b44075670 msgr2=0x7f0b44075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:57.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.461+0000 7f0b4bf8e700 1 -- 172.21.15.71:0/4017137563 shutdown_connections 2024-04-23T02:45:57.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.461+0000 7f0b4bf8e700 1 -- 172.21.15.71:0/4017137563 wait complete. 2024-04-23T02:45:57.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.462+0000 7f0b4bf8e700 1 Processor -- start 2024-04-23T02:45:57.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.462+0000 7f0b4bf8e700 1 -- start start 2024-04-23T02:45:57.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.462+0000 7f0b4bf8e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b440ffc20 0x7f0b4419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:57.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.462+0000 7f0b4bf8e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b44105bd0 0x7f0b4419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:57.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.462+0000 7f0b4bf8e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4419d160 con 0x7f0b44105bd0 2024-04-23T02:45:57.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.462+0000 7f0b4bf8e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b44196610 con 0x7f0b440ffc20 2024-04-23T02:45:57.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.462+0000 7f0b49529700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b44105bd0 0x7f0b4419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:57.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.462+0000 7f0b49529700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b44105bd0 0x7f0b4419cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51268/0 (socket says 172.21.15.71:51268) 2024-04-23T02:45:57.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.462+0000 7f0b49529700 1 -- 172.21.15.71:0/2846639090 learned_addr learned my addr 172.21.15.71:0/2846639090 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:45:57.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.462+0000 7f0b49d2a700 1 --2- 172.21.15.71:0/2846639090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b440ffc20 0x7f0b4419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:57.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.462+0000 7f0b49529700 1 -- 172.21.15.71:0/2846639090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b440ffc20 msgr2=0x7f0b4419c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:57.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.462+0000 7f0b49529700 1 --2- 172.21.15.71:0/2846639090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b440ffc20 0x7f0b4419c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.462+0000 7f0b49529700 1 -- 172.21.15.71:0/2846639090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b380095f0 con 0x7f0b44105bd0 2024-04-23T02:45:57.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.462+0000 7f0b49529700 1 --2- 172.21.15.71:0/2846639090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b44105bd0 0x7f0b4419cad0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000c8a0 tx=0x7f0b4000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:57.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.463+0000 7f0b36ffd700 1 -- 172.21.15.71:0/2846639090 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b4000cea0 con 0x7f0b44105bd0 2024-04-23T02:45:57.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.463+0000 7f0b36ffd700 1 -- 172.21.15.71:0/2846639090 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0b40004830 con 0x7f0b44105bd0 2024-04-23T02:45:57.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.463+0000 7f0b36ffd700 1 -- 172.21.15.71:0/2846639090 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b40005500 con 0x7f0b44105bd0 2024-04-23T02:45:57.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.463+0000 7f0b4bf8e700 1 -- 172.21.15.71:0/2846639090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b441968f0 con 0x7f0b44105bd0 2024-04-23T02:45:57.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.463+0000 7f0b4bf8e700 1 -- 172.21.15.71:0/2846639090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b44196e40 con 0x7f0b44105bd0 2024-04-23T02:45:57.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.463+0000 7f0b4bf8e700 1 -- 172.21.15.71:0/2846639090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4404f2e0 con 0x7f0b44105bd0 2024-04-23T02:45:57.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.464+0000 7f0b36ffd700 1 -- 172.21.15.71:0/2846639090 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0b400049a0 con 0x7f0b44105bd0 2024-04-23T02:45:57.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.464+0000 7f0b36ffd700 1 --2- 172.21.15.71:0/2846639090 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0b3006cce0 0x7f0b3006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:57.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.465+0000 7f0b49d2a700 1 --2- 172.21.15.71:0/2846639090 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0b3006cce0 0x7f0b3006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:57.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.465+0000 7f0b36ffd700 1 -- 172.21.15.71:0/2846639090 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0b4008c1d0 con 0x7f0b44105bd0 2024-04-23T02:45:57.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.465+0000 7f0b49d2a700 1 --2- 172.21.15.71:0/2846639090 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0b3006cce0 0x7f0b3006f1a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0b38006010 tx=0x7f0b3801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:57.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.469+0000 7f0b36ffd700 1 -- 172.21.15.71:0/2846639090 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0b40059d50 con 0x7f0b44105bd0 2024-04-23T02:45:57.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.674+0000 7f0b4bf8e700 1 -- 172.21.15.71:0/2846639090 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b44197170 con 0x7f0b3006cce0 2024-04-23T02:45:57.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.684+0000 7f0b36ffd700 1 -- 172.21.15.71:0/2846639090 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0b44197170 con 0x7f0b3006cce0 2024-04-23T02:45:57.687 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:45:57.687 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (29m) 7m ago 40m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:45:57.687 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (41m) 7m ago 41m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:45:57.687 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (40m) 7m ago 40m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:45:57.687 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (27m) 7m ago 41m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:45:57.687 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (27m) 7m ago 40m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:45:57.687 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (29m) 7m ago 40m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:45:57.687 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (23m) 7m ago 35m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:45:57.687 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (23m) 7m ago 34m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:45:57.687 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (8m) 7m ago 34m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:45:57.688 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (23m) 7m ago 34m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:45:57.688 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (31m) 7m ago 42m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:45:57.688 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (30m) 7m ago 39m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:45:57.688 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (28m) 7m ago 42m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:45:57.688 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (27m) 7m ago 39m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:45:57.688 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (30m) 7m ago 41m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:45:57.688 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (30m) 7m ago 39m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:45:57.688 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (27m) 7m ago 39m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:45:57.688 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (26m) 7m ago 38m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:45:57.688 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (25m) 7m ago 37m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:45:57.688 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (25m) 7m ago 37m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:45:57.688 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (24m) 7m ago 37m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:45:57.688 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (24m) 7m ago 36m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:45:57.688 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (29m) 7m ago 40m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:45:57.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.686+0000 7f0b4bf8e700 1 -- 172.21.15.71:0/2846639090 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0b3006cce0 msgr2=0x7f0b3006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:57.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.686+0000 7f0b4bf8e700 1 --2- 172.21.15.71:0/2846639090 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0b3006cce0 0x7f0b3006f1a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0b38006010 tx=0x7f0b3801a040 comp rx=0 tx=0).stop 2024-04-23T02:45:57.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.686+0000 7f0b4bf8e700 1 -- 172.21.15.71:0/2846639090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b44105bd0 msgr2=0x7f0b4419cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:57.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.686+0000 7f0b4bf8e700 1 --2- 172.21.15.71:0/2846639090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b44105bd0 0x7f0b4419cad0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000c8a0 tx=0x7f0b4000cc60 comp rx=0 tx=0).stop 2024-04-23T02:45:57.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.686+0000 7f0b4bf8e700 1 -- 172.21.15.71:0/2846639090 shutdown_connections 2024-04-23T02:45:57.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.686+0000 7f0b4bf8e700 1 --2- 172.21.15.71:0/2846639090 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0b3006cce0 0x7f0b3006f1a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.686+0000 7f0b4bf8e700 1 --2- 172.21.15.71:0/2846639090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b440ffc20 0x7f0b4419c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.686+0000 7f0b4bf8e700 1 --2- 172.21.15.71:0/2846639090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b44105bd0 0x7f0b4419cad0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.686+0000 7f0b4bf8e700 1 -- 172.21.15.71:0/2846639090 >> 172.21.15.71:0/2846639090 conn(0x7f0b44075670 msgr2=0x7f0b44068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:57.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.686+0000 7f0b4bf8e700 1 -- 172.21.15.71:0/2846639090 shutdown_connections 2024-04-23T02:45:57.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.686+0000 7f0b4bf8e700 1 -- 172.21.15.71:0/2846639090 wait complete. 2024-04-23T02:45:57.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.788+0000 7f5eab2dc700 1 -- 172.21.15.71:0/4288249015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea40ffc20 msgr2=0x7f5ea4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:57.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.788+0000 7f5eab2dc700 1 --2- 172.21.15.71:0/4288249015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea40ffc20 0x7f5ea4100080 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009910 tx=0x7f5ea0009c20 comp rx=0 tx=0).stop 2024-04-23T02:45:57.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.788+0000 7f5eab2dc700 1 -- 172.21.15.71:0/4288249015 shutdown_connections 2024-04-23T02:45:57.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.788+0000 7f5eab2dc700 1 --2- 172.21.15.71:0/4288249015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea40ffc20 0x7f5ea4100080 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.788+0000 7f5eab2dc700 1 --2- 172.21.15.71:0/4288249015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ea4105bd0 0x7f5ea4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.788+0000 7f5eab2dc700 1 -- 172.21.15.71:0/4288249015 >> 172.21.15.71:0/4288249015 conn(0x7f5ea4075670 msgr2=0x7f5ea4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:57.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.788+0000 7f5eab2dc700 1 -- 172.21.15.71:0/4288249015 shutdown_connections 2024-04-23T02:45:57.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.788+0000 7f5eab2dc700 1 -- 172.21.15.71:0/4288249015 wait complete. 2024-04-23T02:45:57.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.789+0000 7f5eab2dc700 1 Processor -- start 2024-04-23T02:45:57.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.789+0000 7f5eab2dc700 1 -- start start 2024-04-23T02:45:57.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.789+0000 7f5eab2dc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea40ffc20 0x7f5ea419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:57.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.789+0000 7f5eab2dc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ea4105bd0 0x7f5ea419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:57.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.789+0000 7f5eab2dc700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea419c0f0 con 0x7f5ea40ffc20 2024-04-23T02:45:57.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.789+0000 7f5eab2dc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea41955a0 con 0x7f5ea4105bd0 2024-04-23T02:45:57.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.789+0000 7f5ea9078700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea40ffc20 0x7f5ea419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:57.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.789+0000 7f5ea9078700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea40ffc20 0x7f5ea419b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51278/0 (socket says 172.21.15.71:51278) 2024-04-23T02:45:57.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.789+0000 7f5ea9078700 1 -- 172.21.15.71:0/2882721103 learned_addr learned my addr 172.21.15.71:0/2882721103 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:45:57.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.789+0000 7f5ea8877700 1 --2- 172.21.15.71:0/2882721103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ea4105bd0 0x7f5ea419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:57.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.789+0000 7f5ea9078700 1 -- 172.21.15.71:0/2882721103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ea4105bd0 msgr2=0x7f5ea419ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:57.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.789+0000 7f5ea9078700 1 --2- 172.21.15.71:0/2882721103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ea4105bd0 0x7f5ea419ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:57.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.789+0000 7f5ea9078700 1 -- 172.21.15.71:0/2882721103 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea00095f0 con 0x7f5ea40ffc20 2024-04-23T02:45:57.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.789+0000 7f5ea9078700 1 --2- 172.21.15.71:0/2882721103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea40ffc20 0x7f5ea419b520 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009970 tx=0x7f5e94009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:57.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.790+0000 7f5e9a7fc700 1 -- 172.21.15.71:0/2882721103 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e94012070 con 0x7f5ea40ffc20 2024-04-23T02:45:57.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.790+0000 7f5e9a7fc700 1 -- 172.21.15.71:0/2882721103 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5e94007ad0 con 0x7f5ea40ffc20 2024-04-23T02:45:57.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.790+0000 7f5eab2dc700 1 -- 172.21.15.71:0/2882721103 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea4195820 con 0x7f5ea40ffc20 2024-04-23T02:45:57.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.790+0000 7f5e9a7fc700 1 -- 172.21.15.71:0/2882721103 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e94017c00 con 0x7f5ea40ffc20 2024-04-23T02:45:57.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.790+0000 7f5eab2dc700 1 -- 172.21.15.71:0/2882721103 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea4195d70 con 0x7f5ea40ffc20 2024-04-23T02:45:57.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.790+0000 7f5eab2dc700 1 -- 172.21.15.71:0/2882721103 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea404f2e0 con 0x7f5ea40ffc20 2024-04-23T02:45:57.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.791+0000 7f5e9a7fc700 1 -- 172.21.15.71:0/2882721103 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5e94007c40 con 0x7f5ea40ffc20 2024-04-23T02:45:57.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.791+0000 7f5e9a7fc700 1 --2- 172.21.15.71:0/2882721103 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5e9006cce0 0x7f5e9006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:57.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.791+0000 7f5e9a7fc700 1 -- 172.21.15.71:0/2882721103 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5e9408da80 con 0x7f5ea40ffc20 2024-04-23T02:45:57.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.791+0000 7f5ea8877700 1 --2- 172.21.15.71:0/2882721103 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5e9006cce0 0x7f5e9006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:57.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.792+0000 7f5ea8877700 1 --2- 172.21.15.71:0/2882721103 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5e9006cce0 0x7f5e9006f1a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40fe2d0 tx=0x7f5ea001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:57.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:57.796+0000 7f5e9a7fc700 1 -- 172.21.15.71:0/2882721103 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5e9405b6b0 con 0x7f5ea40ffc20 2024-04-23T02:45:57.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:57 smithi192 ceph-mon[108617]: pgmap v980: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:57.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:57 smithi071 ceph-mon[144544]: pgmap v980: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:58.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.069+0000 7f5eab2dc700 1 -- 172.21.15.71:0/2882721103 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5ea41960d0 con 0x7f5ea40ffc20 2024-04-23T02:45:58.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.069+0000 7f5e9a7fc700 1 -- 172.21.15.71:0/2882721103 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f5e9405b240 con 0x7f5ea40ffc20 2024-04-23T02:45:58.072 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:45:58.072 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:45:58.072 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:45:58.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:45:58.073 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:45:58.073 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:45:58.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:45:58.073 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:45:58.073 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:45:58.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:45:58.073 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:45:58.073 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:45:58.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:45:58.074 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:45:58.074 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:45:58.074 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:45:58.074 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:45:58.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.071+0000 7f5eab2dc700 1 -- 172.21.15.71:0/2882721103 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5e9006cce0 msgr2=0x7f5e9006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:58.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.071+0000 7f5eab2dc700 1 --2- 172.21.15.71:0/2882721103 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5e9006cce0 0x7f5e9006f1a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40fe2d0 tx=0x7f5ea001a040 comp rx=0 tx=0).stop 2024-04-23T02:45:58.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.071+0000 7f5eab2dc700 1 -- 172.21.15.71:0/2882721103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea40ffc20 msgr2=0x7f5ea419b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:58.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.071+0000 7f5eab2dc700 1 --2- 172.21.15.71:0/2882721103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea40ffc20 0x7f5ea419b520 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009970 tx=0x7f5e94009d30 comp rx=0 tx=0).stop 2024-04-23T02:45:58.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.071+0000 7f5eab2dc700 1 -- 172.21.15.71:0/2882721103 shutdown_connections 2024-04-23T02:45:58.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.071+0000 7f5eab2dc700 1 --2- 172.21.15.71:0/2882721103 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5e9006cce0 0x7f5e9006f1a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.071+0000 7f5eab2dc700 1 --2- 172.21.15.71:0/2882721103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea40ffc20 0x7f5ea419b520 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.071+0000 7f5eab2dc700 1 --2- 172.21.15.71:0/2882721103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ea4105bd0 0x7f5ea419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.072+0000 7f5eab2dc700 1 -- 172.21.15.71:0/2882721103 >> 172.21.15.71:0/2882721103 conn(0x7f5ea4075670 msgr2=0x7f5ea40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:58.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.072+0000 7f5eab2dc700 1 -- 172.21.15.71:0/2882721103 shutdown_connections 2024-04-23T02:45:58.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.072+0000 7f5eab2dc700 1 -- 172.21.15.71:0/2882721103 wait complete. 2024-04-23T02:45:58.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.175+0000 7fc1e4fce700 1 -- 172.21.15.71:0/3168122117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e00ffc20 msgr2=0x7fc1e0100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:58.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.175+0000 7fc1e4fce700 1 --2- 172.21.15.71:0/3168122117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e00ffc20 0x7fc1e0100080 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009910 tx=0x7fc1c8009c20 comp rx=0 tx=0).stop 2024-04-23T02:45:58.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.175+0000 7fc1e4fce700 1 -- 172.21.15.71:0/3168122117 shutdown_connections 2024-04-23T02:45:58.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.175+0000 7fc1e4fce700 1 --2- 172.21.15.71:0/3168122117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e00ffc20 0x7fc1e0100080 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.175+0000 7fc1e4fce700 1 --2- 172.21.15.71:0/3168122117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1e0105bd0 0x7fc1e0105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.175+0000 7fc1e4fce700 1 -- 172.21.15.71:0/3168122117 >> 172.21.15.71:0/3168122117 conn(0x7fc1e0075670 msgr2=0x7fc1e0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:58.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.175+0000 7fc1e4fce700 1 -- 172.21.15.71:0/3168122117 shutdown_connections 2024-04-23T02:45:58.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.175+0000 7fc1e4fce700 1 -- 172.21.15.71:0/3168122117 wait complete. 2024-04-23T02:45:58.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.176+0000 7fc1e4fce700 1 Processor -- start 2024-04-23T02:45:58.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.176+0000 7fc1e4fce700 1 -- start start 2024-04-23T02:45:58.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.176+0000 7fc1e4fce700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e00ffc20 0x7fc1e019b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:58.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.176+0000 7fc1e4fce700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1e0105bd0 0x7fc1e019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:58.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.176+0000 7fc1e4fce700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e019c0f0 con 0x7fc1e00ffc20 2024-04-23T02:45:58.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.176+0000 7fc1e4fce700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e01955a0 con 0x7fc1e0105bd0 2024-04-23T02:45:58.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.176+0000 7fc1de59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e00ffc20 0x7fc1e019b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:58.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.176+0000 7fc1de59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e00ffc20 0x7fc1e019b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51298/0 (socket says 172.21.15.71:51298) 2024-04-23T02:45:58.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.176+0000 7fc1de59c700 1 -- 172.21.15.71:0/4132087497 learned_addr learned my addr 172.21.15.71:0/4132087497 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:45:58.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.176+0000 7fc1de59c700 1 -- 172.21.15.71:0/4132087497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1e0105bd0 msgr2=0x7fc1e019ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:58.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.176+0000 7fc1de59c700 1 --2- 172.21.15.71:0/4132087497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1e0105bd0 0x7fc1e019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.176+0000 7fc1de59c700 1 -- 172.21.15.71:0/4132087497 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c80095f0 con 0x7fc1e00ffc20 2024-04-23T02:45:58.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.177+0000 7fc1de59c700 1 --2- 172.21.15.71:0/4132087497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e00ffc20 0x7fc1e019b520 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc1d80060b0 tx=0x7fc1d800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:58.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.177+0000 7fc1d77fe700 1 -- 172.21.15.71:0/4132087497 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1d800dda0 con 0x7fc1e00ffc20 2024-04-23T02:45:58.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.177+0000 7fc1d77fe700 1 -- 172.21.15.71:0/4132087497 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc1d800f040 con 0x7fc1e00ffc20 2024-04-23T02:45:58.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.177+0000 7fc1d77fe700 1 -- 172.21.15.71:0/4132087497 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1d80146f0 con 0x7fc1e00ffc20 2024-04-23T02:45:58.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.177+0000 7fc1e4fce700 1 -- 172.21.15.71:0/4132087497 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e0195880 con 0x7fc1e00ffc20 2024-04-23T02:45:58.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.177+0000 7fc1e4fce700 1 -- 172.21.15.71:0/4132087497 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e0195dd0 con 0x7fc1e00ffc20 2024-04-23T02:45:58.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.178+0000 7fc1e4fce700 1 -- 172.21.15.71:0/4132087497 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e01080d0 con 0x7fc1e00ffc20 2024-04-23T02:45:58.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.178+0000 7fc1d77fe700 1 -- 172.21.15.71:0/4132087497 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc1d8015400 con 0x7fc1e00ffc20 2024-04-23T02:45:58.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.178+0000 7fc1d77fe700 1 --2- 172.21.15.71:0/4132087497 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc1cc06cce0 0x7fc1cc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:58.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.179+0000 7fc1d77fe700 1 -- 172.21.15.71:0/4132087497 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc1d808bf90 con 0x7fc1e00ffc20 2024-04-23T02:45:58.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.179+0000 7fc1ddd9b700 1 --2- 172.21.15.71:0/4132087497 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc1cc06cce0 0x7fc1cc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:58.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.179+0000 7fc1ddd9b700 1 --2- 172.21.15.71:0/4132087497 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc1cc06cce0 0x7fc1cc06f1a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8006010 tx=0x7fc1c800b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:58.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.183+0000 7fc1d77fe700 1 -- 172.21.15.71:0/4132087497 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc1d8059bc0 con 0x7fc1e00ffc20 2024-04-23T02:45:58.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.415+0000 7fc1e4fce700 1 -- 172.21.15.71:0/4132087497 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc1e01960a0 con 0x7fc1e00ffc20 2024-04-23T02:45:58.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.416+0000 7fc1d77fe700 1 -- 172.21.15.71:0/4132087497 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7fc1d8059750 con 0x7fc1e00ffc20 2024-04-23T02:45:58.418 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:45:58.419 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:45:58.419 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:45:58.419 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:45:58.419 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:45:58.419 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:45:58.419 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:45:58.419 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:45:58.419 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:45:58.419 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:45:58.419 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:45:58.419 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:45:58.419 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:45:58.419 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:45:58.419 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:45:58.419 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:45:58.419 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:45:58.420 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:45:58.420 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:45:58.420 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:45:58.420 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:45:58.420 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:45:58.420 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:45:58.420 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:45:58.420 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:45:58.420 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:45:58.420 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:45:58.420 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:45:58.421 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:45:58.421 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:45:58.421 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:45:58.421 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:45:58.421 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:45:58.421 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:45:58.421 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:45:58.421 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:45:58.421 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:45:58.421 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:45:58.421 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:45:58.421 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:45:58.421 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:45:58.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.417+0000 7fc1e4fce700 1 -- 172.21.15.71:0/4132087497 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc1cc06cce0 msgr2=0x7fc1cc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:58.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.417+0000 7fc1e4fce700 1 --2- 172.21.15.71:0/4132087497 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc1cc06cce0 0x7fc1cc06f1a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8006010 tx=0x7fc1c800b540 comp rx=0 tx=0).stop 2024-04-23T02:45:58.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.417+0000 7fc1e4fce700 1 -- 172.21.15.71:0/4132087497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e00ffc20 msgr2=0x7fc1e019b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:58.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.417+0000 7fc1e4fce700 1 --2- 172.21.15.71:0/4132087497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e00ffc20 0x7fc1e019b520 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc1d80060b0 tx=0x7fc1d800d3b0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.418+0000 7fc1e4fce700 1 -- 172.21.15.71:0/4132087497 shutdown_connections 2024-04-23T02:45:58.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.418+0000 7fc1e4fce700 1 --2- 172.21.15.71:0/4132087497 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc1cc06cce0 0x7fc1cc06f1a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.418+0000 7fc1e4fce700 1 --2- 172.21.15.71:0/4132087497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1e00ffc20 0x7fc1e019b520 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.418+0000 7fc1e4fce700 1 --2- 172.21.15.71:0/4132087497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1e0105bd0 0x7fc1e019ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.418+0000 7fc1e4fce700 1 -- 172.21.15.71:0/4132087497 >> 172.21.15.71:0/4132087497 conn(0x7fc1e0075670 msgr2=0x7fc1e00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:58.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.418+0000 7fc1e4fce700 1 -- 172.21.15.71:0/4132087497 shutdown_connections 2024-04-23T02:45:58.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.418+0000 7fc1e4fce700 1 -- 172.21.15.71:0/4132087497 wait complete. 2024-04-23T02:45:58.423 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:45:58.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.522+0000 7fddfe0e7700 1 -- 172.21.15.71:0/4169111897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf8105bd0 msgr2=0x7fddf8105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:58.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.522+0000 7fddfe0e7700 1 --2- 172.21.15.71:0/4169111897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf8105bd0 0x7fddf8105fb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fdde800b0a0 tx=0x7fdde800b3b0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.522+0000 7fddfe0e7700 1 -- 172.21.15.71:0/4169111897 shutdown_connections 2024-04-23T02:45:58.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.522+0000 7fddfe0e7700 1 --2- 172.21.15.71:0/4169111897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddf80ffc20 0x7fddf8100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.522+0000 7fddfe0e7700 1 --2- 172.21.15.71:0/4169111897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf8105bd0 0x7fddf8105fb0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.522+0000 7fddfe0e7700 1 -- 172.21.15.71:0/4169111897 >> 172.21.15.71:0/4169111897 conn(0x7fddf8075670 msgr2=0x7fddf8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:58.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.522+0000 7fddfe0e7700 1 -- 172.21.15.71:0/4169111897 shutdown_connections 2024-04-23T02:45:58.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.522+0000 7fddfe0e7700 1 -- 172.21.15.71:0/4169111897 wait complete. 2024-04-23T02:45:58.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.523+0000 7fddfe0e7700 1 Processor -- start 2024-04-23T02:45:58.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.523+0000 7fddfe0e7700 1 -- start start 2024-04-23T02:45:58.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.523+0000 7fddfe0e7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddf80ffc20 0x7fddf8197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:58.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.523+0000 7fddfe0e7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf8105bd0 0x7fddf8197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:58.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.523+0000 7fddfe0e7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf8198160 con 0x7fddf8105bd0 2024-04-23T02:45:58.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.523+0000 7fddfe0e7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf81982a0 con 0x7fddf80ffc20 2024-04-23T02:45:58.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.523+0000 7fddf6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf8105bd0 0x7fddf8197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:58.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.523+0000 7fddf6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf8105bd0 0x7fddf8197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51314/0 (socket says 172.21.15.71:51314) 2024-04-23T02:45:58.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.523+0000 7fddf6ffd700 1 -- 172.21.15.71:0/1907336217 learned_addr learned my addr 172.21.15.71:0/1907336217 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:45:58.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.523+0000 7fddf77fe700 1 --2- 172.21.15.71:0/1907336217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddf80ffc20 0x7fddf8197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:58.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.523+0000 7fddf6ffd700 1 -- 172.21.15.71:0/1907336217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddf80ffc20 msgr2=0x7fddf8197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:58.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.523+0000 7fddf6ffd700 1 --2- 172.21.15.71:0/1907336217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddf80ffc20 0x7fddf8197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.523+0000 7fddf6ffd700 1 -- 172.21.15.71:0/1907336217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde8009d00 con 0x7fddf8105bd0 2024-04-23T02:45:58.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.523+0000 7fddf77fe700 1 --2- 172.21.15.71:0/1907336217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddf80ffc20 0x7fddf8197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:45:58.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.524+0000 7fddf6ffd700 1 --2- 172.21.15.71:0/1907336217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf8105bd0 0x7fddf8197b40 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fdde000c8a0 tx=0x7fdde000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:58.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.524+0000 7fddf4ff9700 1 -- 172.21.15.71:0/1907336217 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdde000cea0 con 0x7fddf8105bd0 2024-04-23T02:45:58.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.524+0000 7fddfe0e7700 1 -- 172.21.15.71:0/1907336217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf81a0670 con 0x7fddf8105bd0 2024-04-23T02:45:58.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.524+0000 7fddf4ff9700 1 -- 172.21.15.71:0/1907336217 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdde0004830 con 0x7fddf8105bd0 2024-04-23T02:45:58.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.524+0000 7fddf4ff9700 1 -- 172.21.15.71:0/1907336217 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdde0005500 con 0x7fddf8105bd0 2024-04-23T02:45:58.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.524+0000 7fddfe0e7700 1 -- 172.21.15.71:0/1907336217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf81a0b30 con 0x7fddf8105bd0 2024-04-23T02:45:58.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.525+0000 7fddfe0e7700 1 -- 172.21.15.71:0/1907336217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf81080d0 con 0x7fddf8105bd0 2024-04-23T02:45:58.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.525+0000 7fddf4ff9700 1 -- 172.21.15.71:0/1907336217 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fdde00049a0 con 0x7fddf8105bd0 2024-04-23T02:45:58.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.525+0000 7fddf4ff9700 1 --2- 172.21.15.71:0/1907336217 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdde406cce0 0x7fdde406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:58.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.526+0000 7fddf4ff9700 1 -- 172.21.15.71:0/1907336217 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdde008c2b0 con 0x7fddf8105bd0 2024-04-23T02:45:58.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.526+0000 7fddf77fe700 1 --2- 172.21.15.71:0/1907336217 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdde406cce0 0x7fdde406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:58.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.526+0000 7fddf77fe700 1 --2- 172.21.15.71:0/1907336217 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdde406cce0 0x7fdde406f1a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdde8009480 tx=0x7fdde8006000 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:58.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.530+0000 7fddf4ff9700 1 -- 172.21.15.71:0/1907336217 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fdde0056320 con 0x7fddf8105bd0 2024-04-23T02:45:58.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.744+0000 7fddfe0e7700 1 -- 172.21.15.71:0/1907336217 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fddf80611d0 con 0x7fdde406cce0 2024-04-23T02:45:58.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.745+0000 7fddf4ff9700 1 -- 172.21.15.71:0/1907336217 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fddf80611d0 con 0x7fdde406cce0 2024-04-23T02:45:58.748 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:45:58.748 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:45:58.748 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:45:58.748 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:45:58.749 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:45:58.749 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:45:58.749 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:45:58.749 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:45:58.749 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:45:58.749 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:45:58.749 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:45:58.749 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:45:58.749 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:45:58.749 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:45:58.749 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:45:58.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.747+0000 7fddfe0e7700 1 -- 172.21.15.71:0/1907336217 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdde406cce0 msgr2=0x7fdde406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:58.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.747+0000 7fddfe0e7700 1 --2- 172.21.15.71:0/1907336217 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdde406cce0 0x7fdde406f1a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdde8009480 tx=0x7fdde8006000 comp rx=0 tx=0).stop 2024-04-23T02:45:58.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.747+0000 7fddfe0e7700 1 -- 172.21.15.71:0/1907336217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf8105bd0 msgr2=0x7fddf8197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:58.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.747+0000 7fddfe0e7700 1 --2- 172.21.15.71:0/1907336217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf8105bd0 0x7fddf8197b40 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fdde000c8a0 tx=0x7fdde000cc60 comp rx=0 tx=0).stop 2024-04-23T02:45:58.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.748+0000 7fddfe0e7700 1 -- 172.21.15.71:0/1907336217 shutdown_connections 2024-04-23T02:45:58.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.748+0000 7fddfe0e7700 1 --2- 172.21.15.71:0/1907336217 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fdde406cce0 0x7fdde406f1a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.748+0000 7fddfe0e7700 1 --2- 172.21.15.71:0/1907336217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddf80ffc20 0x7fddf8197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.748+0000 7fddfe0e7700 1 --2- 172.21.15.71:0/1907336217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fddf8105bd0 0x7fddf8197b40 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.748+0000 7fddfe0e7700 1 -- 172.21.15.71:0/1907336217 >> 172.21.15.71:0/1907336217 conn(0x7fddf8075670 msgr2=0x7fddf80fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:58.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.748+0000 7fddfe0e7700 1 -- 172.21.15.71:0/1907336217 shutdown_connections 2024-04-23T02:45:58.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.748+0000 7fddfe0e7700 1 -- 172.21.15.71:0/1907336217 wait complete. 2024-04-23T02:45:58.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.850+0000 7f24d71a1700 1 -- 172.21.15.71:0/3680817924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d0105bd0 msgr2=0x7f24d0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:58.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.850+0000 7f24d71a1700 1 --2- 172.21.15.71:0/3680817924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d0105bd0 0x7f24d0105fb0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f24c4009910 tx=0x7f24c4009c20 comp rx=0 tx=0).stop 2024-04-23T02:45:58.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.850+0000 7f24d71a1700 1 -- 172.21.15.71:0/3680817924 shutdown_connections 2024-04-23T02:45:58.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.850+0000 7f24d71a1700 1 --2- 172.21.15.71:0/3680817924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24d00ffc20 0x7f24d0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.850+0000 7f24d71a1700 1 --2- 172.21.15.71:0/3680817924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d0105bd0 0x7f24d0105fb0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.851+0000 7f24d71a1700 1 -- 172.21.15.71:0/3680817924 >> 172.21.15.71:0/3680817924 conn(0x7f24d0075670 msgr2=0x7f24d0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:58.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.851+0000 7f24d71a1700 1 -- 172.21.15.71:0/3680817924 shutdown_connections 2024-04-23T02:45:58.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.851+0000 7f24d71a1700 1 -- 172.21.15.71:0/3680817924 wait complete. 2024-04-23T02:45:58.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.851+0000 7f24d71a1700 1 Processor -- start 2024-04-23T02:45:58.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.851+0000 7f24d71a1700 1 -- start start 2024-04-23T02:45:58.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.851+0000 7f24d71a1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d00ffc20 0x7f24d019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:58.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.851+0000 7f24d71a1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24d0105bd0 0x7f24d019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:58.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.851+0000 7f24d71a1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d019d160 con 0x7f24d00ffc20 2024-04-23T02:45:58.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.851+0000 7f24d71a1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d0196610 con 0x7f24d0105bd0 2024-04-23T02:45:58.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.851+0000 7f24d4f3d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d00ffc20 0x7f24d019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:58.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.851+0000 7f24d4f3d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d00ffc20 0x7f24d019c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51322/0 (socket says 172.21.15.71:51322) 2024-04-23T02:45:58.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.851+0000 7f24d4f3d700 1 -- 172.21.15.71:0/2396669220 learned_addr learned my addr 172.21.15.71:0/2396669220 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:45:58.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.852+0000 7f24cffff700 1 --2- 172.21.15.71:0/2396669220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24d0105bd0 0x7f24d019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:58.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.852+0000 7f24d4f3d700 1 -- 172.21.15.71:0/2396669220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24d0105bd0 msgr2=0x7f24d019cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:58.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.852+0000 7f24d4f3d700 1 --2- 172.21.15.71:0/2396669220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24d0105bd0 0x7f24d019cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:58.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.852+0000 7f24d4f3d700 1 -- 172.21.15.71:0/2396669220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c40095f0 con 0x7f24d00ffc20 2024-04-23T02:45:58.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.852+0000 7f24cffff700 1 --2- 172.21.15.71:0/2396669220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24d0105bd0 0x7f24d019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:45:58.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.852+0000 7f24d4f3d700 1 --2- 172.21.15.71:0/2396669220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d00ffc20 0x7f24d019c590 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f24c40092e0 tx=0x7f24c400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:58.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.852+0000 7f24cdffb700 1 -- 172.21.15.71:0/2396669220 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24c401d070 con 0x7f24d00ffc20 2024-04-23T02:45:58.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.852+0000 7f24cdffb700 1 -- 172.21.15.71:0/2396669220 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f24c4005e00 con 0x7f24d00ffc20 2024-04-23T02:45:58.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.852+0000 7f24d71a1700 1 -- 172.21.15.71:0/2396669220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d0196890 con 0x7f24d00ffc20 2024-04-23T02:45:58.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.852+0000 7f24cdffb700 1 -- 172.21.15.71:0/2396669220 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24c400fe60 con 0x7f24d00ffc20 2024-04-23T02:45:58.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.852+0000 7f24d71a1700 1 -- 172.21.15.71:0/2396669220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d0196d80 con 0x7f24d00ffc20 2024-04-23T02:45:58.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.853+0000 7f24d71a1700 1 -- 172.21.15.71:0/2396669220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d004f2e0 con 0x7f24d00ffc20 2024-04-23T02:45:58.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.854+0000 7f24cdffb700 1 -- 172.21.15.71:0/2396669220 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f24c400f6d0 con 0x7f24d00ffc20 2024-04-23T02:45:58.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.854+0000 7f24cdffb700 1 --2- 172.21.15.71:0/2396669220 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f24c006cce0 0x7f24c006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:45:58.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.854+0000 7f24cdffb700 1 -- 172.21.15.71:0/2396669220 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f24c408d7e0 con 0x7f24d00ffc20 2024-04-23T02:45:58.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.854+0000 7f24cffff700 1 --2- 172.21.15.71:0/2396669220 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f24c006cce0 0x7f24c006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:45:58.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.855+0000 7f24cffff700 1 --2- 172.21.15.71:0/2396669220 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f24c006cce0 0x7f24c006f1a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f24bc005fd0 tx=0x7f24bc005ee0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:45:58.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:58.859+0000 7f24cdffb700 1 -- 172.21.15.71:0/2396669220 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f24c405b2e0 con 0x7f24d00ffc20 2024-04-23T02:45:58.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:58 smithi192 ceph-mon[108617]: from='client.35576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:45:58.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:58 smithi192 ceph-mon[108617]: from='client.35580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:45:58.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:58 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2882721103' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:45:58.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:58 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/4132087497' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:45:58.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:58 smithi071 ceph-mon[144544]: from='client.35576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:45:58.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:58 smithi071 ceph-mon[144544]: from='client.35580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:45:58.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:58 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2882721103' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:45:58.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:58 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/4132087497' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:45:59.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:59.133+0000 7f24d71a1700 1 -- 172.21.15.71:0/2396669220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f24d0197080 con 0x7f24d00ffc20 2024-04-23T02:45:59.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:59.133+0000 7f24cdffb700 1 -- 172.21.15.71:0/2396669220 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7f24c405ae70 con 0x7f24d00ffc20 2024-04-23T02:45:59.136 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:45:59.136 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:45:59.137 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:45:59.137 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:45:59.137 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:45:59.137 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:45:59.137 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:45:59.137 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:45:59.137 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:45:59.137 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:45:59.137 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:45:59.137 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:45:59.137 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:45:59.137 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:45:59.137 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:45:59.137 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:45:59.137 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:45:59.138 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:45:59.138 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:45:59.138 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:45:59.138 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:45:59.138 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:45:59.138 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:45:59.138 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:45:59.138 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:45:59.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:59.135+0000 7f24d71a1700 1 -- 172.21.15.71:0/2396669220 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f24c006cce0 msgr2=0x7f24c006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:59.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:59.136+0000 7f24d71a1700 1 --2- 172.21.15.71:0/2396669220 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f24c006cce0 0x7f24c006f1a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f24bc005fd0 tx=0x7f24bc005ee0 comp rx=0 tx=0).stop 2024-04-23T02:45:59.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:59.136+0000 7f24d71a1700 1 -- 172.21.15.71:0/2396669220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d00ffc20 msgr2=0x7f24d019c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:45:59.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:59.136+0000 7f24d71a1700 1 --2- 172.21.15.71:0/2396669220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d00ffc20 0x7f24d019c590 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f24c40092e0 tx=0x7f24c400be20 comp rx=0 tx=0).stop 2024-04-23T02:45:59.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:59.136+0000 7f24d71a1700 1 -- 172.21.15.71:0/2396669220 shutdown_connections 2024-04-23T02:45:59.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:59.136+0000 7f24d71a1700 1 --2- 172.21.15.71:0/2396669220 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f24c006cce0 0x7f24c006f1a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:59.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:59.136+0000 7f24d71a1700 1 --2- 172.21.15.71:0/2396669220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24d00ffc20 0x7f24d019c590 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:59.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:59.136+0000 7f24d71a1700 1 --2- 172.21.15.71:0/2396669220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24d0105bd0 0x7f24d019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:45:59.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:59.136+0000 7f24d71a1700 1 -- 172.21.15.71:0/2396669220 >> 172.21.15.71:0/2396669220 conn(0x7f24d0075670 msgr2=0x7f24d0068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:45:59.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:59.136+0000 7f24d71a1700 1 -- 172.21.15.71:0/2396669220 shutdown_connections 2024-04-23T02:45:59.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:45:59.136+0000 7f24d71a1700 1 -- 172.21.15.71:0/2396669220 wait complete. 2024-04-23T02:45:59.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:59 smithi192 ceph-mon[108617]: from='client.35584 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:45:59.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:59 smithi192 ceph-mon[108617]: pgmap v981: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:59.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:45:59 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2396669220' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:45:59.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:59 smithi071 ceph-mon[144544]: from='client.35584 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:45:59.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:59 smithi071 ceph-mon[144544]: pgmap v981: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:45:59.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:45:59 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2396669220' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:46:00.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:00 smithi192 ceph-mon[108617]: from='client.35596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:46:00.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:00 smithi071 ceph-mon[144544]: from='client.35596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:46:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:01 smithi192 ceph-mon[108617]: pgmap v982: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:01.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:01 smithi071 ceph-mon[144544]: pgmap v982: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:02.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:02 smithi192 ceph-mon[108617]: pgmap v983: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:02.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:02 smithi071 ceph-mon[144544]: pgmap v983: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:05 smithi192 ceph-mon[108617]: pgmap v984: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:46:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:05 smithi071 ceph-mon[144544]: pgmap v984: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:05.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:46:07.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:07 smithi192 ceph-mon[108617]: pgmap v985: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:07.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:07 smithi071 ceph-mon[144544]: pgmap v985: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:09.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:09 smithi192 ceph-mon[108617]: pgmap v986: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:09.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:09 smithi071 ceph-mon[144544]: pgmap v986: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:11.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:11 smithi192 ceph-mon[108617]: pgmap v987: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:11.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:11 smithi071 ceph-mon[144544]: pgmap v987: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:13.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:13 smithi192 ceph-mon[108617]: pgmap v988: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:13.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:13 smithi071 ceph-mon[144544]: pgmap v988: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:15.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:15 smithi192 ceph-mon[108617]: pgmap v989: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:15.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:15 smithi071 ceph-mon[144544]: pgmap v989: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:16.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:16 smithi071 ceph-mon[144544]: pgmap v990: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:17.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:16 smithi192 ceph-mon[108617]: pgmap v990: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:19.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:19 smithi192 ceph-mon[108617]: pgmap v991: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:19.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:19 smithi071 ceph-mon[144544]: pgmap v991: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:20.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:46:20.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:46:21.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:21 smithi192 ceph-mon[108617]: pgmap v992: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:21.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:21 smithi071 ceph-mon[144544]: pgmap v992: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:23.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:23 smithi192 ceph-mon[108617]: pgmap v993: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:23.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:23 smithi071 ceph-mon[144544]: pgmap v993: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:25.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:25 smithi192 ceph-mon[108617]: pgmap v994: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:25.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:25 smithi071 ceph-mon[144544]: pgmap v994: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:26.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:26 smithi071 ceph-mon[144544]: pgmap v995: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:27.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:26 smithi192 ceph-mon[108617]: pgmap v995: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:29.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.252+0000 7fcf65378700 1 -- 172.21.15.71:0/3129997639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf60100750 msgr2=0x7fcf601047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.252+0000 7fcf65378700 1 --2- 172.21.15.71:0/3129997639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf60100750 0x7fcf601047a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009910 tx=0x7fcf50009c20 comp rx=0 tx=0).stop 2024-04-23T02:46:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.252+0000 7fcf65378700 1 -- 172.21.15.71:0/3129997639 shutdown_connections 2024-04-23T02:46:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.252+0000 7fcf65378700 1 --2- 172.21.15.71:0/3129997639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf60100750 0x7fcf601047a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.252+0000 7fcf65378700 1 --2- 172.21.15.71:0/3129997639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf600ffda0 0x7fcf60100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.252+0000 7fcf65378700 1 -- 172.21.15.71:0/3129997639 >> 172.21.15.71:0/3129997639 conn(0x7fcf600fb650 msgr2=0x7fcf600fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:46:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.252+0000 7fcf65378700 1 -- 172.21.15.71:0/3129997639 shutdown_connections 2024-04-23T02:46:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.252+0000 7fcf65378700 1 -- 172.21.15.71:0/3129997639 wait complete. 2024-04-23T02:46:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.252+0000 7fcf65378700 1 Processor -- start 2024-04-23T02:46:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.253+0000 7fcf65378700 1 -- start start 2024-04-23T02:46:29.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.253+0000 7fcf65378700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf600ffda0 0x7fcf601977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.253+0000 7fcf65378700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf60100750 0x7fcf60197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.253+0000 7fcf65378700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf60198330 con 0x7fcf60100750 2024-04-23T02:46:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.253+0000 7fcf65378700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf60198470 con 0x7fcf600ffda0 2024-04-23T02:46:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.253+0000 7fcf5e7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf60100750 0x7fcf60197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.253+0000 7fcf5e7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf60100750 0x7fcf60197d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58284/0 (socket says 172.21.15.71:58284) 2024-04-23T02:46:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.253+0000 7fcf5e7fc700 1 -- 172.21.15.71:0/326865193 learned_addr learned my addr 172.21.15.71:0/326865193 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:46:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.253+0000 7fcf5effd700 1 --2- 172.21.15.71:0/326865193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf600ffda0 0x7fcf601977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.253+0000 7fcf5e7fc700 1 -- 172.21.15.71:0/326865193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf600ffda0 msgr2=0x7fcf601977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.253+0000 7fcf5e7fc700 1 --2- 172.21.15.71:0/326865193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf600ffda0 0x7fcf601977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.253+0000 7fcf5e7fc700 1 -- 172.21.15.71:0/326865193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf500095f0 con 0x7fcf60100750 2024-04-23T02:46:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.253+0000 7fcf5e7fc700 1 --2- 172.21.15.71:0/326865193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf60100750 0x7fcf60197d10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000bda0 tx=0x7fcf5000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:46:29.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.254+0000 7fcf47fff700 1 -- 172.21.15.71:0/326865193 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf5001d070 con 0x7fcf60100750 2024-04-23T02:46:29.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.254+0000 7fcf65378700 1 -- 172.21.15.71:0/326865193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf601a0840 con 0x7fcf60100750 2024-04-23T02:46:29.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.254+0000 7fcf47fff700 1 -- 172.21.15.71:0/326865193 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcf5000f460 con 0x7fcf60100750 2024-04-23T02:46:29.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.254+0000 7fcf47fff700 1 -- 172.21.15.71:0/326865193 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf50021640 con 0x7fcf60100750 2024-04-23T02:46:29.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.254+0000 7fcf65378700 1 -- 172.21.15.71:0/326865193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf601a0c50 con 0x7fcf60100750 2024-04-23T02:46:29.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.255+0000 7fcf65378700 1 -- 172.21.15.71:0/326865193 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf601080f0 con 0x7fcf60100750 2024-04-23T02:46:29.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.255+0000 7fcf47fff700 1 -- 172.21.15.71:0/326865193 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fcf5000f9b0 con 0x7fcf60100750 2024-04-23T02:46:29.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.256+0000 7fcf47fff700 1 --2- 172.21.15.71:0/326865193 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcf4806cce0 0x7fcf4806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:29.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.256+0000 7fcf47fff700 1 -- 172.21.15.71:0/326865193 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcf5008d880 con 0x7fcf60100750 2024-04-23T02:46:29.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.256+0000 7fcf5effd700 1 --2- 172.21.15.71:0/326865193 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcf4806cce0 0x7fcf4806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:29.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.256+0000 7fcf5effd700 1 --2- 172.21.15.71:0/326865193 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcf4806cce0 0x7fcf4806f1a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcf54006fd0 tx=0x7fcf54008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:46:29.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.260+0000 7fcf47fff700 1 -- 172.21.15.71:0/326865193 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fcf500578f0 con 0x7fcf60100750 2024-04-23T02:46:29.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.469+0000 7fcf65378700 1 -- 172.21.15.71:0/326865193 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf600611d0 con 0x7fcf4806cce0 2024-04-23T02:46:29.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.471+0000 7fcf47fff700 1 -- 172.21.15.71:0/326865193 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcf600611d0 con 0x7fcf4806cce0 2024-04-23T02:46:29.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.473+0000 7fcf65378700 1 -- 172.21.15.71:0/326865193 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcf4806cce0 msgr2=0x7fcf4806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:29.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.473+0000 7fcf65378700 1 --2- 172.21.15.71:0/326865193 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcf4806cce0 0x7fcf4806f1a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcf54006fd0 tx=0x7fcf54008040 comp rx=0 tx=0).stop 2024-04-23T02:46:29.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.473+0000 7fcf65378700 1 -- 172.21.15.71:0/326865193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf60100750 msgr2=0x7fcf60197d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:29.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.473+0000 7fcf65378700 1 --2- 172.21.15.71:0/326865193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf60100750 0x7fcf60197d10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000bda0 tx=0x7fcf5000be80 comp rx=0 tx=0).stop 2024-04-23T02:46:29.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.473+0000 7fcf65378700 1 -- 172.21.15.71:0/326865193 shutdown_connections 2024-04-23T02:46:29.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.473+0000 7fcf65378700 1 --2- 172.21.15.71:0/326865193 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcf4806cce0 0x7fcf4806f1a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:29.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.473+0000 7fcf65378700 1 --2- 172.21.15.71:0/326865193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf600ffda0 0x7fcf601977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:29.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.473+0000 7fcf65378700 1 --2- 172.21.15.71:0/326865193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf60100750 0x7fcf60197d10 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:29.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.473+0000 7fcf65378700 1 -- 172.21.15.71:0/326865193 >> 172.21.15.71:0/326865193 conn(0x7fcf600fb650 msgr2=0x7fcf600fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:46:29.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.473+0000 7fcf65378700 1 -- 172.21.15.71:0/326865193 shutdown_connections 2024-04-23T02:46:29.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.473+0000 7fcf65378700 1 -- 172.21.15.71:0/326865193 wait complete. 2024-04-23T02:46:29.487 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:46:29.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.577+0000 7fefa7a88700 1 -- 172.21.15.71:0/731463724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fefa0105bf0 msgr2=0x7fefa0105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:29.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.577+0000 7fefa7a88700 1 --2- 172.21.15.71:0/731463724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fefa0105bf0 0x7fefa0105fd0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fef90009910 tx=0x7fef90009c20 comp rx=0 tx=0).stop 2024-04-23T02:46:29.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.577+0000 7fefa7a88700 1 -- 172.21.15.71:0/731463724 shutdown_connections 2024-04-23T02:46:29.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.577+0000 7fefa7a88700 1 --2- 172.21.15.71:0/731463724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefa00ffb40 0x7fefa00fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:29.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.577+0000 7fefa7a88700 1 --2- 172.21.15.71:0/731463724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fefa0105bf0 0x7fefa0105fd0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:29.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.577+0000 7fefa7a88700 1 -- 172.21.15.71:0/731463724 >> 172.21.15.71:0/731463724 conn(0x7fefa00fb610 msgr2=0x7fefa00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:46:29.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.577+0000 7fefa7a88700 1 -- 172.21.15.71:0/731463724 shutdown_connections 2024-04-23T02:46:29.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.577+0000 7fefa7a88700 1 -- 172.21.15.71:0/731463724 wait complete. 2024-04-23T02:46:29.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.577+0000 7fefa7a88700 1 Processor -- start 2024-04-23T02:46:29.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.577+0000 7fefa7a88700 1 -- start start 2024-04-23T02:46:29.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.578+0000 7fefa7a88700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fefa00ffb40 0x7fefa0196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:29.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.578+0000 7fefa7a88700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefa0105bf0 0x7fefa0196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:29.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.578+0000 7fefa7a88700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa01972d0 con 0x7fefa00ffb40 2024-04-23T02:46:29.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.578+0000 7fefa7a88700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa019a000 con 0x7fefa0105bf0 2024-04-23T02:46:29.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.578+0000 7fefa5824700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fefa00ffb40 0x7fefa0196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.578+0000 7fefa5824700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fefa00ffb40 0x7fefa0196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58308/0 (socket says 172.21.15.71:58308) 2024-04-23T02:46:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.578+0000 7fefa5824700 1 -- 172.21.15.71:0/1245933274 learned_addr learned my addr 172.21.15.71:0/1245933274 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:46:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.578+0000 7fefa5023700 1 --2- 172.21.15.71:0/1245933274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefa0105bf0 0x7fefa0196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.578+0000 7fefa5824700 1 -- 172.21.15.71:0/1245933274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefa0105bf0 msgr2=0x7fefa0196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.578+0000 7fefa5824700 1 --2- 172.21.15.71:0/1245933274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefa0105bf0 0x7fefa0196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.578+0000 7fefa5824700 1 -- 172.21.15.71:0/1245933274 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef900095f0 con 0x7fefa00ffb40 2024-04-23T02:46:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.578+0000 7fefa5023700 1 --2- 172.21.15.71:0/1245933274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefa0105bf0 0x7fefa0196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:46:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.578+0000 7fefa5824700 1 --2- 172.21.15.71:0/1245933274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fefa00ffb40 0x7fefa0196700 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fef90006010 tx=0x7fef9000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:46:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.578+0000 7fef96ffd700 1 -- 172.21.15.71:0/1245933274 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef9001d070 con 0x7fefa00ffb40 2024-04-23T02:46:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.578+0000 7fefa7a88700 1 -- 172.21.15.71:0/1245933274 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa019a280 con 0x7fefa00ffb40 2024-04-23T02:46:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.578+0000 7fef96ffd700 1 -- 172.21.15.71:0/1245933274 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fef9000f460 con 0x7fefa00ffb40 2024-04-23T02:46:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.578+0000 7fef96ffd700 1 -- 172.21.15.71:0/1245933274 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef90021620 con 0x7fefa00ffb40 2024-04-23T02:46:29.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.578+0000 7fefa7a88700 1 -- 172.21.15.71:0/1245933274 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa019a770 con 0x7fefa00ffb40 2024-04-23T02:46:29.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.579+0000 7fefa7a88700 1 -- 172.21.15.71:0/1245933274 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa00679c0 con 0x7fefa00ffb40 2024-04-23T02:46:29.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.580+0000 7fef96ffd700 1 -- 172.21.15.71:0/1245933274 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fef9000fa40 con 0x7fefa00ffb40 2024-04-23T02:46:29.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.580+0000 7fef96ffd700 1 --2- 172.21.15.71:0/1245933274 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fef8c06cce0 0x7fef8c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:29.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.580+0000 7fef96ffd700 1 -- 172.21.15.71:0/1245933274 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fef9008d830 con 0x7fefa00ffb40 2024-04-23T02:46:29.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.580+0000 7fefa5023700 1 --2- 172.21.15.71:0/1245933274 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fef8c06cce0 0x7fef8c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:29.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.581+0000 7fefa5023700 1 --2- 172.21.15.71:0/1245933274 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fef8c06cce0 0x7fef8c06f1a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fef9c005fd0 tx=0x7fef9c005ee0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:46:29.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.585+0000 7fef96ffd700 1 -- 172.21.15.71:0/1245933274 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fef90057800 con 0x7fefa00ffb40 2024-04-23T02:46:29.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.794+0000 7fefa7a88700 1 -- 172.21.15.71:0/1245933274 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fefa019aaa0 con 0x7fef8c06cce0 2024-04-23T02:46:29.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.795+0000 7fef96ffd700 1 -- 172.21.15.71:0/1245933274 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fefa019aaa0 con 0x7fef8c06cce0 2024-04-23T02:46:29.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.797+0000 7fefa7a88700 1 -- 172.21.15.71:0/1245933274 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fef8c06cce0 msgr2=0x7fef8c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:29.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.797+0000 7fefa7a88700 1 --2- 172.21.15.71:0/1245933274 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fef8c06cce0 0x7fef8c06f1a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fef9c005fd0 tx=0x7fef9c005ee0 comp rx=0 tx=0).stop 2024-04-23T02:46:29.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.797+0000 7fefa7a88700 1 -- 172.21.15.71:0/1245933274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fefa00ffb40 msgr2=0x7fefa0196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:29.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.797+0000 7fefa7a88700 1 --2- 172.21.15.71:0/1245933274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fefa00ffb40 0x7fefa0196700 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fef90006010 tx=0x7fef9000be20 comp rx=0 tx=0).stop 2024-04-23T02:46:29.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.797+0000 7fefa7a88700 1 -- 172.21.15.71:0/1245933274 shutdown_connections 2024-04-23T02:46:29.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.797+0000 7fefa7a88700 1 --2- 172.21.15.71:0/1245933274 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fef8c06cce0 0x7fef8c06f1a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:29.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.797+0000 7fefa7a88700 1 --2- 172.21.15.71:0/1245933274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fefa00ffb40 0x7fefa0196700 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:29.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.797+0000 7fefa7a88700 1 --2- 172.21.15.71:0/1245933274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefa0105bf0 0x7fefa0196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:29.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.797+0000 7fefa7a88700 1 -- 172.21.15.71:0/1245933274 >> 172.21.15.71:0/1245933274 conn(0x7fefa00fb610 msgr2=0x7fefa0068f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:46:29.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.797+0000 7fefa7a88700 1 -- 172.21.15.71:0/1245933274 shutdown_connections 2024-04-23T02:46:29.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.797+0000 7fefa7a88700 1 -- 172.21.15.71:0/1245933274 wait complete. 2024-04-23T02:46:29.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:29 smithi192 ceph-mon[108617]: pgmap v996: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:29.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.900+0000 7f2fbbae9700 1 -- 172.21.15.71:0/1221037773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fb4100e90 msgr2=0x7f2fb4101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:29.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.900+0000 7f2fbbae9700 1 --2- 172.21.15.71:0/1221037773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fb4100e90 0x7f2fb4101270 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0009910 tx=0x7f2fb0009c20 comp rx=0 tx=0).stop 2024-04-23T02:46:29.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.901+0000 7f2fbbae9700 1 -- 172.21.15.71:0/1221037773 shutdown_connections 2024-04-23T02:46:29.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.901+0000 7f2fbbae9700 1 --2- 172.21.15.71:0/1221037773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fb4101840 0x7f2fb4105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:29.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.901+0000 7f2fbbae9700 1 --2- 172.21.15.71:0/1221037773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fb4100e90 0x7f2fb4101270 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:29.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.901+0000 7f2fbbae9700 1 -- 172.21.15.71:0/1221037773 >> 172.21.15.71:0/1221037773 conn(0x7f2fb40fc740 msgr2=0x7f2fb40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:46:29.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.901+0000 7f2fbbae9700 1 -- 172.21.15.71:0/1221037773 shutdown_connections 2024-04-23T02:46:29.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.901+0000 7f2fbbae9700 1 -- 172.21.15.71:0/1221037773 wait complete. 2024-04-23T02:46:29.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.901+0000 7f2fbbae9700 1 Processor -- start 2024-04-23T02:46:29.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.901+0000 7f2fbbae9700 1 -- start start 2024-04-23T02:46:29.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.902+0000 7f2fbbae9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fb4100e90 0x7f2fb419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:29.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.902+0000 7f2fbbae9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fb4101840 0x7f2fb419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:29.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.902+0000 7f2fbbae9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb419d190 con 0x7f2fb4101840 2024-04-23T02:46:29.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.902+0000 7f2fbbae9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb4196640 con 0x7f2fb4100e90 2024-04-23T02:46:29.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.902+0000 7f2fb9084700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fb4101840 0x7f2fb419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:29.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.902+0000 7f2fb9084700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fb4101840 0x7f2fb419cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58318/0 (socket says 172.21.15.71:58318) 2024-04-23T02:46:29.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.902+0000 7f2fb9084700 1 -- 172.21.15.71:0/2366407229 learned_addr learned my addr 172.21.15.71:0/2366407229 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:46:29.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.902+0000 7f2fb9885700 1 --2- 172.21.15.71:0/2366407229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fb4100e90 0x7f2fb419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:29.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.902+0000 7f2fb9084700 1 -- 172.21.15.71:0/2366407229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fb4100e90 msgr2=0x7f2fb419c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:29.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.902+0000 7f2fb9084700 1 --2- 172.21.15.71:0/2366407229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fb4100e90 0x7f2fb419c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:29.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.902+0000 7f2fb9084700 1 -- 172.21.15.71:0/2366407229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb00095f0 con 0x7f2fb4101840 2024-04-23T02:46:29.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.902+0000 7f2fb9885700 1 --2- 172.21.15.71:0/2366407229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fb4100e90 0x7f2fb419c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:46:29.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.902+0000 7f2fb9084700 1 --2- 172.21.15.71:0/2366407229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fb4101840 0x7f2fb419cb00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400d900 tx=0x7f2fa400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:46:29.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.902+0000 7f2faaffd700 1 -- 172.21.15.71:0/2366407229 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fa4009900 con 0x7f2fb4101840 2024-04-23T02:46:29.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.902+0000 7f2fbbae9700 1 -- 172.21.15.71:0/2366407229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb4196920 con 0x7f2fb4101840 2024-04-23T02:46:29.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.902+0000 7f2faaffd700 1 -- 172.21.15.71:0/2366407229 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2fa4010460 con 0x7f2fb4101840 2024-04-23T02:46:29.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.902+0000 7f2faaffd700 1 -- 172.21.15.71:0/2366407229 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fa400f5f0 con 0x7f2fb4101840 2024-04-23T02:46:29.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.903+0000 7f2fbbae9700 1 -- 172.21.15.71:0/2366407229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb4196e70 con 0x7f2fb4101840 2024-04-23T02:46:29.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.903+0000 7f2fbbae9700 1 -- 172.21.15.71:0/2366407229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb4109230 con 0x7f2fb4101840 2024-04-23T02:46:29.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.904+0000 7f2faaffd700 1 -- 172.21.15.71:0/2366407229 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2fa40105d0 con 0x7f2fb4101840 2024-04-23T02:46:29.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.904+0000 7f2faaffd700 1 --2- 172.21.15.71:0/2366407229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2fa006cce0 0x7f2fa006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:29.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.905+0000 7f2fb9885700 1 --2- 172.21.15.71:0/2366407229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2fa006cce0 0x7f2fa006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:29.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.905+0000 7f2faaffd700 1 -- 172.21.15.71:0/2366407229 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2fa408c3c0 con 0x7f2fb4101840 2024-04-23T02:46:29.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.905+0000 7f2fb9885700 1 --2- 172.21.15.71:0/2366407229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2fa006cce0 0x7f2fa006f1a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000b5c0 tx=0x7f2fb001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:46:29.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:29.909+0000 7f2faaffd700 1 -- 172.21.15.71:0/2366407229 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2fa4059ec0 con 0x7f2fb4101840 2024-04-23T02:46:29.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:29 smithi071 ceph-mon[144544]: pgmap v996: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:30.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.112+0000 7f2fbbae9700 1 -- 172.21.15.71:0/2366407229 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fb4197090 con 0x7f2fa006cce0 2024-04-23T02:46:30.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.123+0000 7f2faaffd700 1 -- 172.21.15.71:0/2366407229 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2fb4197090 con 0x7f2fa006cce0 2024-04-23T02:46:30.127 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:46:30.127 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (30m) 8m ago 41m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:46:30.127 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (41m) 8m ago 41m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:46:30.127 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (40m) 8m ago 40m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:46:30.127 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (28m) 8m ago 41m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:46:30.127 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (28m) 8m ago 40m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:46:30.127 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (29m) 8m ago 41m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:46:30.127 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (24m) 8m ago 35m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:46:30.127 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (24m) 8m ago 35m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:46:30.127 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (8m) 8m ago 35m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:46:30.127 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (23m) 8m ago 35m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:46:30.127 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (32m) 8m ago 43m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:46:30.127 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (31m) 8m ago 40m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:46:30.128 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (28m) 8m ago 43m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:46:30.128 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (28m) 8m ago 40m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:46:30.128 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (31m) 8m ago 41m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:46:30.128 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (31m) 8m ago 40m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:46:30.128 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (27m) 8m ago 39m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:46:30.128 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (27m) 8m ago 39m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:46:30.128 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (26m) 8m ago 38m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:46:30.128 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (25m) 8m ago 38m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:46:30.128 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (25m) 8m ago 37m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:46:30.128 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (24m) 8m ago 37m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:46:30.128 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (30m) 8m ago 40m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:46:30.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.125+0000 7f2fbbae9700 1 -- 172.21.15.71:0/2366407229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2fa006cce0 msgr2=0x7f2fa006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:30.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.125+0000 7f2fbbae9700 1 --2- 172.21.15.71:0/2366407229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2fa006cce0 0x7f2fa006f1a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000b5c0 tx=0x7f2fb001a040 comp rx=0 tx=0).stop 2024-04-23T02:46:30.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.125+0000 7f2fbbae9700 1 -- 172.21.15.71:0/2366407229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fb4101840 msgr2=0x7f2fb419cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:30.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.125+0000 7f2fbbae9700 1 --2- 172.21.15.71:0/2366407229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fb4101840 0x7f2fb419cb00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400d900 tx=0x7f2fa400dcc0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.125+0000 7f2fbbae9700 1 -- 172.21.15.71:0/2366407229 shutdown_connections 2024-04-23T02:46:30.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.125+0000 7f2fbbae9700 1 --2- 172.21.15.71:0/2366407229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f2fa006cce0 0x7f2fa006f1a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.125+0000 7f2fbbae9700 1 --2- 172.21.15.71:0/2366407229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fb4100e90 0x7f2fb419c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.125+0000 7f2fbbae9700 1 --2- 172.21.15.71:0/2366407229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fb4101840 0x7f2fb419cb00 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.125+0000 7f2fbbae9700 1 -- 172.21.15.71:0/2366407229 >> 172.21.15.71:0/2366407229 conn(0x7f2fb40fc740 msgr2=0x7f2fb40fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:46:30.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.125+0000 7f2fbbae9700 1 -- 172.21.15.71:0/2366407229 shutdown_connections 2024-04-23T02:46:30.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.125+0000 7f2fbbae9700 1 -- 172.21.15.71:0/2366407229 wait complete. 2024-04-23T02:46:30.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.226+0000 7f34b6af1700 1 -- 172.21.15.71:0/2118373551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b00ffb40 msgr2=0x7f34b00fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:30.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.226+0000 7f34b6af1700 1 --2- 172.21.15.71:0/2118373551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b00ffb40 0x7f34b00fffc0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009910 tx=0x7f34a4009c20 comp rx=0 tx=0).stop 2024-04-23T02:46:30.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.226+0000 7f34b6af1700 1 -- 172.21.15.71:0/2118373551 shutdown_connections 2024-04-23T02:46:30.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.226+0000 7f34b6af1700 1 --2- 172.21.15.71:0/2118373551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b00ffb40 0x7f34b00fffc0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.226+0000 7f34b6af1700 1 --2- 172.21.15.71:0/2118373551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34b0105bf0 0x7f34b0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.226+0000 7f34b6af1700 1 -- 172.21.15.71:0/2118373551 >> 172.21.15.71:0/2118373551 conn(0x7f34b00fb610 msgr2=0x7f34b00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:46:30.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.226+0000 7f34b6af1700 1 -- 172.21.15.71:0/2118373551 shutdown_connections 2024-04-23T02:46:30.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.226+0000 7f34b6af1700 1 -- 172.21.15.71:0/2118373551 wait complete. 2024-04-23T02:46:30.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.227+0000 7f34b6af1700 1 Processor -- start 2024-04-23T02:46:30.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.227+0000 7f34b6af1700 1 -- start start 2024-04-23T02:46:30.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.227+0000 7f34b6af1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34b00ffb40 0x7f34b019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:30.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.227+0000 7f34b6af1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b0105bf0 0x7f34b019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:30.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.227+0000 7f34b6af1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b019d0d0 con 0x7f34b0105bf0 2024-04-23T02:46:30.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.227+0000 7f34b6af1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b0196580 con 0x7f34b00ffb40 2024-04-23T02:46:30.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.227+0000 7f34affff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b0105bf0 0x7f34b019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:30.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.227+0000 7f34affff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b0105bf0 0x7f34b019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58340/0 (socket says 172.21.15.71:58340) 2024-04-23T02:46:30.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.227+0000 7f34affff700 1 -- 172.21.15.71:0/2574214032 learned_addr learned my addr 172.21.15.71:0/2574214032 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:46:30.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.227+0000 7f34b488d700 1 --2- 172.21.15.71:0/2574214032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34b00ffb40 0x7f34b019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:30.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.227+0000 7f34affff700 1 -- 172.21.15.71:0/2574214032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34b00ffb40 msgr2=0x7f34b019c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:30.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.227+0000 7f34affff700 1 --2- 172.21.15.71:0/2574214032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34b00ffb40 0x7f34b019c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.227+0000 7f34affff700 1 -- 172.21.15.71:0/2574214032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a40095f0 con 0x7f34b0105bf0 2024-04-23T02:46:30.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.228+0000 7f34affff700 1 --2- 172.21.15.71:0/2574214032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b0105bf0 0x7f34b019ca40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f34a4000c00 tx=0x7f34a400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:46:30.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.228+0000 7f34adffb700 1 -- 172.21.15.71:0/2574214032 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34a401d070 con 0x7f34b0105bf0 2024-04-23T02:46:30.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.228+0000 7f34adffb700 1 -- 172.21.15.71:0/2574214032 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f34a40044c0 con 0x7f34b0105bf0 2024-04-23T02:46:30.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.228+0000 7f34b6af1700 1 -- 172.21.15.71:0/2574214032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b0196800 con 0x7f34b0105bf0 2024-04-23T02:46:30.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.228+0000 7f34adffb700 1 -- 172.21.15.71:0/2574214032 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34a40039d0 con 0x7f34b0105bf0 2024-04-23T02:46:30.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.228+0000 7f34b6af1700 1 -- 172.21.15.71:0/2574214032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b0196c90 con 0x7f34b0105bf0 2024-04-23T02:46:30.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.229+0000 7f34b6af1700 1 -- 172.21.15.71:0/2574214032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b004f2e0 con 0x7f34b0105bf0 2024-04-23T02:46:30.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.229+0000 7f34adffb700 1 -- 172.21.15.71:0/2574214032 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f34a40048e0 con 0x7f34b0105bf0 2024-04-23T02:46:30.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.230+0000 7f34adffb700 1 --2- 172.21.15.71:0/2574214032 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f349806cce0 0x7f349806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:30.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.230+0000 7f34adffb700 1 -- 172.21.15.71:0/2574214032 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f34a408dc60 con 0x7f34b0105bf0 2024-04-23T02:46:30.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.230+0000 7f34b488d700 1 --2- 172.21.15.71:0/2574214032 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f349806cce0 0x7f349806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:30.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.230+0000 7f34b488d700 1 --2- 172.21.15.71:0/2574214032 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f349806cce0 0x7f349806f1a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009380 tx=0x7f34a000b040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:46:30.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.235+0000 7f34adffb700 1 -- 172.21.15.71:0/2574214032 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f34a405b760 con 0x7f34b0105bf0 2024-04-23T02:46:30.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.507+0000 7f34b6af1700 1 -- 172.21.15.71:0/2574214032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f34b0197a90 con 0x7f34b0105bf0 2024-04-23T02:46:30.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.508+0000 7f34adffb700 1 -- 172.21.15.71:0/2574214032 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f34a405b2f0 con 0x7f34b0105bf0 2024-04-23T02:46:30.512 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:46:30.512 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:46:30.512 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:46:30.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:46:30.512 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:46:30.512 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:46:30.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:46:30.512 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:46:30.512 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:46:30.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:46:30.512 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:46:30.513 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:46:30.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:46:30.513 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:46:30.513 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:46:30.513 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:46:30.513 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:46:30.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.511+0000 7f34b6af1700 1 -- 172.21.15.71:0/2574214032 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f349806cce0 msgr2=0x7f349806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:30.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.511+0000 7f34b6af1700 1 --2- 172.21.15.71:0/2574214032 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f349806cce0 0x7f349806f1a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009380 tx=0x7f34a000b040 comp rx=0 tx=0).stop 2024-04-23T02:46:30.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.511+0000 7f34b6af1700 1 -- 172.21.15.71:0/2574214032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b0105bf0 msgr2=0x7f34b019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:30.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.511+0000 7f34b6af1700 1 --2- 172.21.15.71:0/2574214032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b0105bf0 0x7f34b019ca40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f34a4000c00 tx=0x7f34a400be90 comp rx=0 tx=0).stop 2024-04-23T02:46:30.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.511+0000 7f34b6af1700 1 -- 172.21.15.71:0/2574214032 shutdown_connections 2024-04-23T02:46:30.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.511+0000 7f34b6af1700 1 --2- 172.21.15.71:0/2574214032 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f349806cce0 0x7f349806f1a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.511+0000 7f34b6af1700 1 --2- 172.21.15.71:0/2574214032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34b00ffb40 0x7f34b019c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.511+0000 7f34b6af1700 1 --2- 172.21.15.71:0/2574214032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34b0105bf0 0x7f34b019ca40 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.511+0000 7f34b6af1700 1 -- 172.21.15.71:0/2574214032 >> 172.21.15.71:0/2574214032 conn(0x7f34b00fb610 msgr2=0x7f34b0103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:46:30.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.511+0000 7f34b6af1700 1 -- 172.21.15.71:0/2574214032 shutdown_connections 2024-04-23T02:46:30.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.511+0000 7f34b6af1700 1 -- 172.21.15.71:0/2574214032 wait complete. 2024-04-23T02:46:30.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.612+0000 7f5311f4f700 1 -- 172.21.15.71:0/3973584080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f530c0ffb40 msgr2=0x7f530c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:30.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.612+0000 7f5311f4f700 1 --2- 172.21.15.71:0/3973584080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f530c0ffb40 0x7f530c0fffc0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5300009910 tx=0x7f5300009c20 comp rx=0 tx=0).stop 2024-04-23T02:46:30.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.612+0000 7f5311f4f700 1 -- 172.21.15.71:0/3973584080 shutdown_connections 2024-04-23T02:46:30.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.612+0000 7f5311f4f700 1 --2- 172.21.15.71:0/3973584080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f530c0ffb40 0x7f530c0fffc0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.612+0000 7f5311f4f700 1 --2- 172.21.15.71:0/3973584080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c105bf0 0x7f530c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.612+0000 7f5311f4f700 1 -- 172.21.15.71:0/3973584080 >> 172.21.15.71:0/3973584080 conn(0x7f530c0fb610 msgr2=0x7f530c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:46:30.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.612+0000 7f5311f4f700 1 -- 172.21.15.71:0/3973584080 shutdown_connections 2024-04-23T02:46:30.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.612+0000 7f5311f4f700 1 -- 172.21.15.71:0/3973584080 wait complete. 2024-04-23T02:46:30.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.613+0000 7f5311f4f700 1 Processor -- start 2024-04-23T02:46:30.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.613+0000 7f5311f4f700 1 -- start start 2024-04-23T02:46:30.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.613+0000 7f5311f4f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f530c0ffb40 0x7f530c195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:30.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.613+0000 7f5311f4f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c105bf0 0x7f530c195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:30.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.613+0000 7f5311f4f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c196240 con 0x7f530c0ffb40 2024-04-23T02:46:30.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.613+0000 7f5311f4f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c199fd0 con 0x7f530c105bf0 2024-04-23T02:46:30.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.614+0000 7f530b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f530c0ffb40 0x7f530c195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:30.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.614+0000 7f530b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f530c0ffb40 0x7f530c195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58364/0 (socket says 172.21.15.71:58364) 2024-04-23T02:46:30.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.614+0000 7f530b7fe700 1 -- 172.21.15.71:0/3574353641 learned_addr learned my addr 172.21.15.71:0/3574353641 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:46:30.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.614+0000 7f530affd700 1 --2- 172.21.15.71:0/3574353641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c105bf0 0x7f530c195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:30.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.614+0000 7f530b7fe700 1 -- 172.21.15.71:0/3574353641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c105bf0 msgr2=0x7f530c195b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:30.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.614+0000 7f530b7fe700 1 --2- 172.21.15.71:0/3574353641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c105bf0 0x7f530c195b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.614+0000 7f530b7fe700 1 -- 172.21.15.71:0/3574353641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53000095f0 con 0x7f530c0ffb40 2024-04-23T02:46:30.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.614+0000 7f530affd700 1 --2- 172.21.15.71:0/3574353641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c105bf0 0x7f530c195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:46:30.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.614+0000 7f530b7fe700 1 --2- 172.21.15.71:0/3574353641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f530c0ffb40 0x7f530c195620 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f52fc00d8d0 tx=0x7f52fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:46:30.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.614+0000 7f5308ff9700 1 -- 172.21.15.71:0/3574353641 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52fc009940 con 0x7f530c0ffb40 2024-04-23T02:46:30.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.614+0000 7f5308ff9700 1 -- 172.21.15.71:0/3574353641 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f52fc010460 con 0x7f530c0ffb40 2024-04-23T02:46:30.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.614+0000 7f5308ff9700 1 -- 172.21.15.71:0/3574353641 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52fc00f5f0 con 0x7f530c0ffb40 2024-04-23T02:46:30.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.614+0000 7f5311f4f700 1 -- 172.21.15.71:0/3574353641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f530c19a2b0 con 0x7f530c0ffb40 2024-04-23T02:46:30.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.614+0000 7f5311f4f700 1 -- 172.21.15.71:0/3574353641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f530c19a800 con 0x7f530c0ffb40 2024-04-23T02:46:30.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.615+0000 7f5311f4f700 1 -- 172.21.15.71:0/3574353641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530c1080a0 con 0x7f530c0ffb40 2024-04-23T02:46:30.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.616+0000 7f5308ff9700 1 -- 172.21.15.71:0/3574353641 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f52fc0105d0 con 0x7f530c0ffb40 2024-04-23T02:46:30.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.616+0000 7f5308ff9700 1 --2- 172.21.15.71:0/3574353641 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f52f406cce0 0x7f52f406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:30.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.616+0000 7f5308ff9700 1 -- 172.21.15.71:0/3574353641 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f52fc08c5e0 con 0x7f530c0ffb40 2024-04-23T02:46:30.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.616+0000 7f530affd700 1 --2- 172.21.15.71:0/3574353641 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f52f406cce0 0x7f52f406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:30.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.616+0000 7f530affd700 1 --2- 172.21.15.71:0/3574353641 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f52f406cce0 0x7f52f406f1a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5300005b40 tx=0x7f530001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:46:30.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.620+0000 7f5308ff9700 1 -- 172.21.15.71:0/3574353641 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f52fc056650 con 0x7f530c0ffb40 2024-04-23T02:46:30.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.858+0000 7f5311f4f700 1 -- 172.21.15.71:0/3574353641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f530c04f2e0 con 0x7f530c0ffb40 2024-04-23T02:46:30.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.858+0000 7f5308ff9700 1 -- 172.21.15.71:0/3574353641 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f52fc010880 con 0x7f530c0ffb40 2024-04-23T02:46:30.862 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:46:30.863 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:46:30.863 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:46:30.863 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:46:30.863 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:46:30.863 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:46:30.863 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:46:30.863 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:46:30.863 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:46:30.863 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:46:30.863 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:46:30.863 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:46:30.863 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:46:30.864 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:46:30.864 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:46:30.864 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:46:30.864 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:46:30.864 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:46:30.864 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:46:30.864 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:46:30.864 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:46:30.864 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:46:30.864 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:46:30.864 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:46:30.865 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:46:30.865 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:46:30.865 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:46:30.865 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:46:30.865 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:46:30.865 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:46:30.865 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:46:30.865 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:46:30.865 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:46:30.865 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:46:30.865 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:46:30.865 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:46:30.865 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:46:30.866 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:46:30.866 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:46:30.866 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:46:30.866 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:46:30.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.860+0000 7f5311f4f700 1 -- 172.21.15.71:0/3574353641 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f52f406cce0 msgr2=0x7f52f406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:30.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.860+0000 7f5311f4f700 1 --2- 172.21.15.71:0/3574353641 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f52f406cce0 0x7f52f406f1a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5300005b40 tx=0x7f530001a040 comp rx=0 tx=0).stop 2024-04-23T02:46:30.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.860+0000 7f5311f4f700 1 -- 172.21.15.71:0/3574353641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f530c0ffb40 msgr2=0x7f530c195620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:30.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.860+0000 7f5311f4f700 1 --2- 172.21.15.71:0/3574353641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f530c0ffb40 0x7f530c195620 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f52fc00d8d0 tx=0x7f52fc00dc90 comp rx=0 tx=0).stop 2024-04-23T02:46:30.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.861+0000 7f5311f4f700 1 -- 172.21.15.71:0/3574353641 shutdown_connections 2024-04-23T02:46:30.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.861+0000 7f5311f4f700 1 --2- 172.21.15.71:0/3574353641 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f52f406cce0 0x7f52f406f1a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.861+0000 7f5311f4f700 1 --2- 172.21.15.71:0/3574353641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f530c0ffb40 0x7f530c195620 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.861+0000 7f5311f4f700 1 --2- 172.21.15.71:0/3574353641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c105bf0 0x7f530c195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.861+0000 7f5311f4f700 1 -- 172.21.15.71:0/3574353641 >> 172.21.15.71:0/3574353641 conn(0x7f530c0fb610 msgr2=0x7f530c0fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:46:30.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.861+0000 7f5311f4f700 1 -- 172.21.15.71:0/3574353641 shutdown_connections 2024-04-23T02:46:30.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.861+0000 7f5311f4f700 1 -- 172.21.15.71:0/3574353641 wait complete. 2024-04-23T02:46:30.867 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:46:30.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:30 smithi192 ceph-mon[108617]: from='client.35604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:46:30.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:30 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2574214032' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:46:30.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.964+0000 7f819c91a700 1 -- 172.21.15.71:0/2878479149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81980ffc20 msgr2=0x7f8198100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:30.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.964+0000 7f819c91a700 1 --2- 172.21.15.71:0/2878479149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81980ffc20 0x7f8198100080 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8180009910 tx=0x7f8180009c20 comp rx=0 tx=0).stop 2024-04-23T02:46:30.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.964+0000 7f819c91a700 1 -- 172.21.15.71:0/2878479149 shutdown_connections 2024-04-23T02:46:30.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.964+0000 7f819c91a700 1 --2- 172.21.15.71:0/2878479149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81980ffc20 0x7f8198100080 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.964+0000 7f819c91a700 1 --2- 172.21.15.71:0/2878479149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8198105bd0 0x7f8198105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.964+0000 7f819c91a700 1 -- 172.21.15.71:0/2878479149 >> 172.21.15.71:0/2878479149 conn(0x7f8198075670 msgr2=0x7f8198075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:46:30.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.964+0000 7f819c91a700 1 -- 172.21.15.71:0/2878479149 shutdown_connections 2024-04-23T02:46:30.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.964+0000 7f819c91a700 1 -- 172.21.15.71:0/2878479149 wait complete. 2024-04-23T02:46:30.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.965+0000 7f819c91a700 1 Processor -- start 2024-04-23T02:46:30.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.965+0000 7f819c91a700 1 -- start start 2024-04-23T02:46:30.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.965+0000 7f819c91a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81980ffc20 0x7f8198197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:30.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.965+0000 7f819c91a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8198105bd0 0x7f8198197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:30.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.965+0000 7f819c91a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81981982a0 con 0x7f81980ffc20 2024-04-23T02:46:30.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.965+0000 7f819c91a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8198198410 con 0x7f8198105bd0 2024-04-23T02:46:30.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.965+0000 7f819659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81980ffc20 0x7f8198197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:30.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.965+0000 7f819659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81980ffc20 0x7f8198197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58386/0 (socket says 172.21.15.71:58386) 2024-04-23T02:46:30.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.965+0000 7f819659c700 1 -- 172.21.15.71:0/1508651285 learned_addr learned my addr 172.21.15.71:0/1508651285 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:46:30.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.966+0000 7f8195d9b700 1 --2- 172.21.15.71:0/1508651285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8198105bd0 0x7f8198197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:30.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.966+0000 7f819659c700 1 -- 172.21.15.71:0/1508651285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8198105bd0 msgr2=0x7f8198197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:30.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.966+0000 7f819659c700 1 --2- 172.21.15.71:0/1508651285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8198105bd0 0x7f8198197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:30.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.966+0000 7f819659c700 1 -- 172.21.15.71:0/1508651285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81800095f0 con 0x7f81980ffc20 2024-04-23T02:46:30.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.966+0000 7f819659c700 1 --2- 172.21.15.71:0/1508651285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81980ffc20 0x7f8198197740 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f818800ba50 tx=0x7f818800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:46:30.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.966+0000 7f818f7fe700 1 -- 172.21.15.71:0/1508651285 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f818800ca00 con 0x7f81980ffc20 2024-04-23T02:46:30.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.966+0000 7f818f7fe700 1 -- 172.21.15.71:0/1508651285 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8188014440 con 0x7f81980ffc20 2024-04-23T02:46:30.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.966+0000 7f819c91a700 1 -- 172.21.15.71:0/1508651285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81981a07b0 con 0x7f81980ffc20 2024-04-23T02:46:30.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.966+0000 7f818f7fe700 1 -- 172.21.15.71:0/1508651285 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81880075a0 con 0x7f81980ffc20 2024-04-23T02:46:30.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.966+0000 7f819c91a700 1 -- 172.21.15.71:0/1508651285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81981a0ca0 con 0x7f81980ffc20 2024-04-23T02:46:30.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.967+0000 7f819c91a700 1 -- 172.21.15.71:0/1508651285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81981080f0 con 0x7f81980ffc20 2024-04-23T02:46:30.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.967+0000 7f818f7fe700 1 -- 172.21.15.71:0/1508651285 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f81880145b0 con 0x7f81980ffc20 2024-04-23T02:46:30.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.968+0000 7f818f7fe700 1 --2- 172.21.15.71:0/1508651285 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f818406cce0 0x7f818406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:30.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.968+0000 7f818f7fe700 1 -- 172.21.15.71:0/1508651285 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8188006e40 con 0x7f81980ffc20 2024-04-23T02:46:30.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.968+0000 7f8195d9b700 1 --2- 172.21.15.71:0/1508651285 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f818406cce0 0x7f818406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:30.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.968+0000 7f8195d9b700 1 --2- 172.21.15.71:0/1508651285 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f818406cce0 0x7f818406f1a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8180006010 tx=0x7f818000b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:46:30.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:30.972+0000 7f818f7fe700 1 -- 172.21.15.71:0/1508651285 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8188092080 con 0x7f81980ffc20 2024-04-23T02:46:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:30 smithi071 ceph-mon[144544]: from='client.35604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:46:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:30 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2574214032' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:46:31.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.183+0000 7f819c91a700 1 -- 172.21.15.71:0/1508651285 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f819802cce0 con 0x7f818406cce0 2024-04-23T02:46:31.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.185+0000 7f818f7fe700 1 -- 172.21.15.71:0/1508651285 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f819802cce0 con 0x7f818406cce0 2024-04-23T02:46:31.187 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:46:31.188 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:46:31.188 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:46:31.188 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:46:31.188 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:46:31.188 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:46:31.188 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:46:31.188 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:46:31.188 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:46:31.188 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:46:31.188 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:46:31.188 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:46:31.188 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:46:31.188 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:46:31.188 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:46:31.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.187+0000 7f819c91a700 1 -- 172.21.15.71:0/1508651285 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f818406cce0 msgr2=0x7f818406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:31.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.187+0000 7f819c91a700 1 --2- 172.21.15.71:0/1508651285 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f818406cce0 0x7f818406f1a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8180006010 tx=0x7f818000b540 comp rx=0 tx=0).stop 2024-04-23T02:46:31.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.187+0000 7f819c91a700 1 -- 172.21.15.71:0/1508651285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81980ffc20 msgr2=0x7f8198197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:31.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.187+0000 7f819c91a700 1 --2- 172.21.15.71:0/1508651285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81980ffc20 0x7f8198197740 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f818800ba50 tx=0x7f818800be10 comp rx=0 tx=0).stop 2024-04-23T02:46:31.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.187+0000 7f819c91a700 1 -- 172.21.15.71:0/1508651285 shutdown_connections 2024-04-23T02:46:31.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.187+0000 7f819c91a700 1 --2- 172.21.15.71:0/1508651285 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f818406cce0 0x7f818406f1a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:31.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.187+0000 7f819c91a700 1 --2- 172.21.15.71:0/1508651285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81980ffc20 0x7f8198197740 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:31.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.187+0000 7f819c91a700 1 --2- 172.21.15.71:0/1508651285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8198105bd0 0x7f8198197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:31.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.187+0000 7f819c91a700 1 -- 172.21.15.71:0/1508651285 >> 172.21.15.71:0/1508651285 conn(0x7f8198075670 msgr2=0x7f81980fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:46:31.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.187+0000 7f819c91a700 1 -- 172.21.15.71:0/1508651285 shutdown_connections 2024-04-23T02:46:31.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.187+0000 7f819c91a700 1 -- 172.21.15.71:0/1508651285 wait complete. 2024-04-23T02:46:31.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.288+0000 7fcc1938d700 1 -- 172.21.15.71:0/2263184338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc140ffda0 msgr2=0x7fcc14100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:31.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.288+0000 7fcc1938d700 1 --2- 172.21.15.71:0/2263184338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc140ffda0 0x7fcc14100180 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009910 tx=0x7fcc08009c20 comp rx=0 tx=0).stop 2024-04-23T02:46:31.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.288+0000 7fcc1938d700 1 -- 172.21.15.71:0/2263184338 shutdown_connections 2024-04-23T02:46:31.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.288+0000 7fcc1938d700 1 --2- 172.21.15.71:0/2263184338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc14100750 0x7fcc141047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:31.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.288+0000 7fcc1938d700 1 --2- 172.21.15.71:0/2263184338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc140ffda0 0x7fcc14100180 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:31.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.288+0000 7fcc1938d700 1 -- 172.21.15.71:0/2263184338 >> 172.21.15.71:0/2263184338 conn(0x7fcc140fb650 msgr2=0x7fcc140fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:46:31.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.288+0000 7fcc1938d700 1 -- 172.21.15.71:0/2263184338 shutdown_connections 2024-04-23T02:46:31.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.288+0000 7fcc1938d700 1 -- 172.21.15.71:0/2263184338 wait complete. 2024-04-23T02:46:31.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.289+0000 7fcc1938d700 1 Processor -- start 2024-04-23T02:46:31.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.289+0000 7fcc1938d700 1 -- start start 2024-04-23T02:46:31.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.289+0000 7fcc1938d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc140ffda0 0x7fcc141956d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:31.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.290+0000 7fcc1938d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc14100750 0x7fcc14195c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:31.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.290+0000 7fcc1938d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc141962a0 con 0x7fcc14100750 2024-04-23T02:46:31.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.290+0000 7fcc1938d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc14198fd0 con 0x7fcc140ffda0 2024-04-23T02:46:31.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.290+0000 7fcc127fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc14100750 0x7fcc14195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:31.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.290+0000 7fcc127fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc14100750 0x7fcc14195c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58410/0 (socket says 172.21.15.71:58410) 2024-04-23T02:46:31.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.290+0000 7fcc127fc700 1 -- 172.21.15.71:0/3365457236 learned_addr learned my addr 172.21.15.71:0/3365457236 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:46:31.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.290+0000 7fcc127fc700 1 -- 172.21.15.71:0/3365457236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc140ffda0 msgr2=0x7fcc141956d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:31.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.290+0000 7fcc12ffd700 1 --2- 172.21.15.71:0/3365457236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc140ffda0 0x7fcc141956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:31.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.290+0000 7fcc127fc700 1 --2- 172.21.15.71:0/3365457236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc140ffda0 0x7fcc141956d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:31.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.290+0000 7fcc127fc700 1 -- 172.21.15.71:0/3365457236 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc080095f0 con 0x7fcc14100750 2024-04-23T02:46:31.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.290+0000 7fcc127fc700 1 --2- 172.21.15.71:0/3365457236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc14100750 0x7fcc14195c10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400d8d0 tx=0x7fcc0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:46:31.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.290+0000 7fcbfbfff700 1 -- 172.21.15.71:0/3365457236 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc040041f0 con 0x7fcc14100750 2024-04-23T02:46:31.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.290+0000 7fcc1938d700 1 -- 172.21.15.71:0/3365457236 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc141992b0 con 0x7fcc14100750 2024-04-23T02:46:31.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.290+0000 7fcbfbfff700 1 -- 172.21.15.71:0/3365457236 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc04007620 con 0x7fcc14100750 2024-04-23T02:46:31.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.290+0000 7fcbfbfff700 1 -- 172.21.15.71:0/3365457236 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc04003b50 con 0x7fcc14100750 2024-04-23T02:46:31.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.290+0000 7fcc1938d700 1 -- 172.21.15.71:0/3365457236 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc14199800 con 0x7fcc14100750 2024-04-23T02:46:31.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.291+0000 7fcc1938d700 1 -- 172.21.15.71:0/3365457236 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc14108140 con 0x7fcc14100750 2024-04-23T02:46:31.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.292+0000 7fcbfbfff700 1 -- 172.21.15.71:0/3365457236 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fcc04004350 con 0x7fcc14100750 2024-04-23T02:46:31.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.292+0000 7fcbfbfff700 1 --2- 172.21.15.71:0/3365457236 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcbfc06cce0 0x7fcbfc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:46:31.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.292+0000 7fcbfbfff700 1 -- 172.21.15.71:0/3365457236 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcc0408c170 con 0x7fcc14100750 2024-04-23T02:46:31.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.292+0000 7fcc12ffd700 1 --2- 172.21.15.71:0/3365457236 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcbfc06cce0 0x7fcbfc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:46:31.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.293+0000 7fcc12ffd700 1 --2- 172.21.15.71:0/3365457236 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcbfc06cce0 0x7fcbfc06f1a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800b5c0 tx=0x7fcc08005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:46:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.297+0000 7fcbfbfff700 1 -- 172.21.15.71:0/3365457236 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fcc040561e0 con 0x7fcc14100750 2024-04-23T02:46:31.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.568+0000 7fcc1938d700 1 -- 172.21.15.71:0/3365457236 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcc14196ad0 con 0x7fcc14100750 2024-04-23T02:46:31.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.569+0000 7fcbfbfff700 1 -- 172.21.15.71:0/3365457236 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7fcc04059800 con 0x7fcc14100750 2024-04-23T02:46:31.572 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:46:31.572 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:46:31.572 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:46:31.572 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:46:31.572 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:46:31.572 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:46:31.572 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:46:31.572 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:46:31.572 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:46:31.572 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:46:31.572 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:46:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:46:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:46:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:46:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:46:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:46:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:46:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:46:31.573 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:46:31.573 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:46:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:46:31.573 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:46:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:46:31.573 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:46:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:46:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.571+0000 7fcc1938d700 1 -- 172.21.15.71:0/3365457236 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcbfc06cce0 msgr2=0x7fcbfc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.571+0000 7fcc1938d700 1 --2- 172.21.15.71:0/3365457236 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcbfc06cce0 0x7fcbfc06f1a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800b5c0 tx=0x7fcc08005fb0 comp rx=0 tx=0).stop 2024-04-23T02:46:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.571+0000 7fcc1938d700 1 -- 172.21.15.71:0/3365457236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc14100750 msgr2=0x7fcc14195c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:46:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.571+0000 7fcc1938d700 1 --2- 172.21.15.71:0/3365457236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc14100750 0x7fcc14195c10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400d8d0 tx=0x7fcc0400dc90 comp rx=0 tx=0).stop 2024-04-23T02:46:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.571+0000 7fcc1938d700 1 -- 172.21.15.71:0/3365457236 shutdown_connections 2024-04-23T02:46:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.571+0000 7fcc1938d700 1 --2- 172.21.15.71:0/3365457236 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcbfc06cce0 0x7fcbfc06f1a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:31.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.571+0000 7fcc1938d700 1 --2- 172.21.15.71:0/3365457236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc140ffda0 0x7fcc141956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:31.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.571+0000 7fcc1938d700 1 --2- 172.21.15.71:0/3365457236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc14100750 0x7fcc14195c10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:46:31.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.571+0000 7fcc1938d700 1 -- 172.21.15.71:0/3365457236 >> 172.21.15.71:0/3365457236 conn(0x7fcc140fb650 msgr2=0x7fcc140fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:46:31.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.571+0000 7fcc1938d700 1 -- 172.21.15.71:0/3365457236 shutdown_connections 2024-04-23T02:46:31.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:46:31.571+0000 7fcc1938d700 1 -- 172.21.15.71:0/3365457236 wait complete. 2024-04-23T02:46:31.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:31 smithi192 ceph-mon[108617]: from='client.35608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:46:31.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:31 smithi192 ceph-mon[108617]: from='client.35612 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:46:31.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:31 smithi192 ceph-mon[108617]: pgmap v997: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:31.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:31 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3574353641' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:46:31.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:31 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3365457236' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:46:31.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:31 smithi071 ceph-mon[144544]: from='client.35608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:46:31.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:31 smithi071 ceph-mon[144544]: from='client.35612 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:46:31.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:31 smithi071 ceph-mon[144544]: pgmap v997: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:31.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:31 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3574353641' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:46:31.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:31 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3365457236' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:46:32.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:32 smithi192 ceph-mon[108617]: from='client.35624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:46:32.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:32 smithi071 ceph-mon[144544]: from='client.35624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:46:33.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:33 smithi192 ceph-mon[108617]: pgmap v998: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:33.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:33 smithi071 ceph-mon[144544]: pgmap v998: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:34.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:34 smithi192 ceph-mon[108617]: pgmap v999: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:34.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:34 smithi071 ceph-mon[144544]: pgmap v999: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:35.876 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:46:36.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:46:36.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:36 smithi071 ceph-mon[144544]: pgmap v1000: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:37.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:36 smithi192 ceph-mon[108617]: pgmap v1000: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:39.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:39 smithi192 ceph-mon[108617]: pgmap v1001: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:39.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:39 smithi071 ceph-mon[144544]: pgmap v1001: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:41.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:41 smithi192 ceph-mon[108617]: pgmap v1002: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:41.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:41 smithi071 ceph-mon[144544]: pgmap v1002: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:43.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:43 smithi192 ceph-mon[108617]: pgmap v1003: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:43.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:43 smithi071 ceph-mon[144544]: pgmap v1003: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:45.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:45 smithi192 ceph-mon[108617]: pgmap v1004: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:45.970 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:45 smithi071 ceph-mon[144544]: pgmap v1004: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:46.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:46 smithi071 ceph-mon[144544]: pgmap v1005: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:47.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:46 smithi192 ceph-mon[108617]: pgmap v1005: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:49.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:49 smithi192 ceph-mon[108617]: pgmap v1006: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:49.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:49 smithi071 ceph-mon[144544]: pgmap v1006: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:46:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:46:51.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:51 smithi192 ceph-mon[108617]: pgmap v1007: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:51.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:51 smithi071 ceph-mon[144544]: pgmap v1007: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:52.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:52 smithi071 ceph-mon[144544]: pgmap v1008: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:53.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:52 smithi192 ceph-mon[108617]: pgmap v1008: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:55.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:55 smithi192 ceph-mon[108617]: pgmap v1009: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:55.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:55 smithi071 ceph-mon[144544]: pgmap v1009: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:57.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:57 smithi192 ceph-mon[108617]: pgmap v1010: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:57.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:57 smithi071 ceph-mon[144544]: pgmap v1010: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:59.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:46:59 smithi192 ceph-mon[108617]: pgmap v1011: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:46:59.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:46:59 smithi071 ceph-mon[144544]: pgmap v1011: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:00.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:00 smithi071 ceph-mon[144544]: pgmap v1012: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:01.046 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:00 smithi192 ceph-mon[108617]: pgmap v1012: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:01.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.687+0000 7f398d386700 1 -- 172.21.15.71:0/2797479718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3988100ea0 msgr2=0x7f3988101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:01.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.687+0000 7f398d386700 1 --2- 172.21.15.71:0/2797479718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3988100ea0 0x7f3988101280 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f397c009910 tx=0x7f397c009c20 comp rx=0 tx=0).stop 2024-04-23T02:47:01.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.687+0000 7f398d386700 1 -- 172.21.15.71:0/2797479718 shutdown_connections 2024-04-23T02:47:01.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.687+0000 7f398d386700 1 --2- 172.21.15.71:0/2797479718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3988101850 0x7f39881058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:01.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.687+0000 7f398d386700 1 --2- 172.21.15.71:0/2797479718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3988100ea0 0x7f3988101280 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:01.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.687+0000 7f398d386700 1 -- 172.21.15.71:0/2797479718 >> 172.21.15.71:0/2797479718 conn(0x7f39880fc750 msgr2=0x7f39880feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:01.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.687+0000 7f398d386700 1 -- 172.21.15.71:0/2797479718 shutdown_connections 2024-04-23T02:47:01.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.687+0000 7f398d386700 1 -- 172.21.15.71:0/2797479718 wait complete. 2024-04-23T02:47:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.688+0000 7f398d386700 1 Processor -- start 2024-04-23T02:47:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.688+0000 7f398d386700 1 -- start start 2024-04-23T02:47:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.688+0000 7f398d386700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3988100ea0 0x7f398819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.688+0000 7f398d386700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3988101850 0x7f398819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.688+0000 7f398d386700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398819d130 con 0x7f3988101850 2024-04-23T02:47:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.688+0000 7f398d386700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39881965e0 con 0x7f3988100ea0 2024-04-23T02:47:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.688+0000 7f39867fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3988101850 0x7f398819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:01.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.688+0000 7f39867fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3988101850 0x7f398819caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52122/0 (socket says 172.21.15.71:52122) 2024-04-23T02:47:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.688+0000 7f39867fc700 1 -- 172.21.15.71:0/1355146811 learned_addr learned my addr 172.21.15.71:0/1355146811 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:47:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.689+0000 7f3986ffd700 1 --2- 172.21.15.71:0/1355146811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3988100ea0 0x7f398819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.689+0000 7f39867fc700 1 -- 172.21.15.71:0/1355146811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3988100ea0 msgr2=0x7f398819c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.689+0000 7f39867fc700 1 --2- 172.21.15.71:0/1355146811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3988100ea0 0x7f398819c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.689+0000 7f39867fc700 1 -- 172.21.15.71:0/1355146811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c0095f0 con 0x7f3988101850 2024-04-23T02:47:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.689+0000 7f3986ffd700 1 --2- 172.21.15.71:0/1355146811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3988100ea0 0x7f398819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:47:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.689+0000 7f39867fc700 1 --2- 172.21.15.71:0/1355146811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3988101850 0x7f398819caa0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f397800d900 tx=0x7f397800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.689+0000 7f396ffff700 1 -- 172.21.15.71:0/1355146811 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3978009900 con 0x7f3988101850 2024-04-23T02:47:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.689+0000 7f396ffff700 1 -- 172.21.15.71:0/1355146811 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3978010460 con 0x7f3988101850 2024-04-23T02:47:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.689+0000 7f398d386700 1 -- 172.21.15.71:0/1355146811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39881968c0 con 0x7f3988101850 2024-04-23T02:47:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.689+0000 7f396ffff700 1 -- 172.21.15.71:0/1355146811 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f397800f5f0 con 0x7f3988101850 2024-04-23T02:47:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.689+0000 7f398d386700 1 -- 172.21.15.71:0/1355146811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3988196e10 con 0x7f3988101850 2024-04-23T02:47:01.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.690+0000 7f398d386700 1 -- 172.21.15.71:0/1355146811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3988109240 con 0x7f3988101850 2024-04-23T02:47:01.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.691+0000 7f396ffff700 1 -- 172.21.15.71:0/1355146811 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f39780105d0 con 0x7f3988101850 2024-04-23T02:47:01.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.691+0000 7f396ffff700 1 --2- 172.21.15.71:0/1355146811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f397006cce0 0x7f397006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:01.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.691+0000 7f396ffff700 1 -- 172.21.15.71:0/1355146811 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f397808c310 con 0x7f3988101850 2024-04-23T02:47:01.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.691+0000 7f3986ffd700 1 --2- 172.21.15.71:0/1355146811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f397006cce0 0x7f397006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:01.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.692+0000 7f3986ffd700 1 --2- 172.21.15.71:0/1355146811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f397006cce0 0x7f397006f1a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f397c006010 tx=0x7f397c00b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:01.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.695+0000 7f396ffff700 1 -- 172.21.15.71:0/1355146811 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3978059f30 con 0x7f3988101850 2024-04-23T02:47:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.905+0000 7f398d386700 1 -- 172.21.15.71:0/1355146811 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3988197080 con 0x7f397006cce0 2024-04-23T02:47:01.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.906+0000 7f396ffff700 1 -- 172.21.15.71:0/1355146811 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3988197080 con 0x7f397006cce0 2024-04-23T02:47:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.908+0000 7f398d386700 1 -- 172.21.15.71:0/1355146811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f397006cce0 msgr2=0x7f397006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.908+0000 7f398d386700 1 --2- 172.21.15.71:0/1355146811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f397006cce0 0x7f397006f1a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f397c006010 tx=0x7f397c00b540 comp rx=0 tx=0).stop 2024-04-23T02:47:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.908+0000 7f398d386700 1 -- 172.21.15.71:0/1355146811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3988101850 msgr2=0x7f398819caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.908+0000 7f398d386700 1 --2- 172.21.15.71:0/1355146811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3988101850 0x7f398819caa0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f397800d900 tx=0x7f397800dcc0 comp rx=0 tx=0).stop 2024-04-23T02:47:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.909+0000 7f398d386700 1 -- 172.21.15.71:0/1355146811 shutdown_connections 2024-04-23T02:47:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.909+0000 7f398d386700 1 --2- 172.21.15.71:0/1355146811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f397006cce0 0x7f397006f1a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.909+0000 7f398d386700 1 --2- 172.21.15.71:0/1355146811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3988100ea0 0x7f398819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.909+0000 7f398d386700 1 --2- 172.21.15.71:0/1355146811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3988101850 0x7f398819caa0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.909+0000 7f398d386700 1 -- 172.21.15.71:0/1355146811 >> 172.21.15.71:0/1355146811 conn(0x7f39880fc750 msgr2=0x7f39880fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:01.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.909+0000 7f398d386700 1 -- 172.21.15.71:0/1355146811 shutdown_connections 2024-04-23T02:47:01.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:01.909+0000 7f398d386700 1 -- 172.21.15.71:0/1355146811 wait complete. 2024-04-23T02:47:01.922 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:47:02.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.012+0000 7f8d577c6700 1 -- 172.21.15.71:0/2656427555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d500ffc20 msgr2=0x7f8d50100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:02.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.012+0000 7f8d577c6700 1 --2- 172.21.15.71:0/2656427555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d500ffc20 0x7f8d50100080 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009910 tx=0x7f8d4c009c20 comp rx=0 tx=0).stop 2024-04-23T02:47:02.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.012+0000 7f8d577c6700 1 -- 172.21.15.71:0/2656427555 shutdown_connections 2024-04-23T02:47:02.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.012+0000 7f8d577c6700 1 --2- 172.21.15.71:0/2656427555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d500ffc20 0x7f8d50100080 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.012+0000 7f8d577c6700 1 --2- 172.21.15.71:0/2656427555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d50105bd0 0x7f8d50105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.012+0000 7f8d577c6700 1 -- 172.21.15.71:0/2656427555 >> 172.21.15.71:0/2656427555 conn(0x7f8d50075670 msgr2=0x7f8d50075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:02.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.012+0000 7f8d577c6700 1 -- 172.21.15.71:0/2656427555 shutdown_connections 2024-04-23T02:47:02.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.012+0000 7f8d577c6700 1 -- 172.21.15.71:0/2656427555 wait complete. 2024-04-23T02:47:02.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.013+0000 7f8d577c6700 1 Processor -- start 2024-04-23T02:47:02.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.013+0000 7f8d577c6700 1 -- start start 2024-04-23T02:47:02.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.013+0000 7f8d577c6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d500ffc20 0x7f8d5019c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.013+0000 7f8d577c6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d50105bd0 0x7f8d5019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.013+0000 7f8d577c6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5019d1c0 con 0x7f8d50105bd0 2024-04-23T02:47:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.013+0000 7f8d577c6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d50196670 con 0x7f8d500ffc20 2024-04-23T02:47:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.013+0000 7f8d54d61700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d50105bd0 0x7f8d5019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.013+0000 7f8d54d61700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d50105bd0 0x7f8d5019cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52138/0 (socket says 172.21.15.71:52138) 2024-04-23T02:47:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.013+0000 7f8d54d61700 1 -- 172.21.15.71:0/2683193885 learned_addr learned my addr 172.21.15.71:0/2683193885 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:47:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.013+0000 7f8d55562700 1 --2- 172.21.15.71:0/2683193885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d500ffc20 0x7f8d5019c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.013+0000 7f8d54d61700 1 -- 172.21.15.71:0/2683193885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d500ffc20 msgr2=0x7f8d5019c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.013+0000 7f8d54d61700 1 --2- 172.21.15.71:0/2683193885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d500ffc20 0x7f8d5019c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.013+0000 7f8d54d61700 1 -- 172.21.15.71:0/2683193885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c0095f0 con 0x7f8d50105bd0 2024-04-23T02:47:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.014+0000 7f8d54d61700 1 --2- 172.21.15.71:0/2683193885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d50105bd0 0x7f8d5019cb30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c006010 tx=0x7f8d4c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.014+0000 7f8d467fc700 1 -- 172.21.15.71:0/2683193885 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d4c01d070 con 0x7f8d50105bd0 2024-04-23T02:47:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.014+0000 7f8d467fc700 1 -- 172.21.15.71:0/2683193885 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8d4c005710 con 0x7f8d50105bd0 2024-04-23T02:47:02.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.014+0000 7f8d577c6700 1 -- 172.21.15.71:0/2683193885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d501968f0 con 0x7f8d50105bd0 2024-04-23T02:47:02.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.014+0000 7f8d467fc700 1 -- 172.21.15.71:0/2683193885 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d4c00fe20 con 0x7f8d50105bd0 2024-04-23T02:47:02.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.014+0000 7f8d577c6700 1 -- 172.21.15.71:0/2683193885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d50196de0 con 0x7f8d50105bd0 2024-04-23T02:47:02.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.015+0000 7f8d577c6700 1 -- 172.21.15.71:0/2683193885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d5004f2e0 con 0x7f8d50105bd0 2024-04-23T02:47:02.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.015+0000 7f8d467fc700 1 -- 172.21.15.71:0/2683193885 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8d4c003a60 con 0x7f8d50105bd0 2024-04-23T02:47:02.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.015+0000 7f8d467fc700 1 --2- 172.21.15.71:0/2683193885 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8d3c06cce0 0x7f8d3c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:02.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.016+0000 7f8d467fc700 1 -- 172.21.15.71:0/2683193885 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8d4c08d6f0 con 0x7f8d50105bd0 2024-04-23T02:47:02.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.016+0000 7f8d55562700 1 --2- 172.21.15.71:0/2683193885 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8d3c06cce0 0x7f8d3c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:02.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.016+0000 7f8d55562700 1 --2- 172.21.15.71:0/2683193885 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8d3c06cce0 0x7f8d3c06f1a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8d40005950 tx=0x7f8d4000b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:02.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.020+0000 7f8d467fc700 1 -- 172.21.15.71:0/2683193885 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8d4c05b1f0 con 0x7f8d50105bd0 2024-04-23T02:47:02.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.229+0000 7f8d577c6700 1 -- 172.21.15.71:0/2683193885 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d50197000 con 0x7f8d3c06cce0 2024-04-23T02:47:02.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.231+0000 7f8d467fc700 1 -- 172.21.15.71:0/2683193885 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8d50197000 con 0x7f8d3c06cce0 2024-04-23T02:47:02.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.233+0000 7f8d577c6700 1 -- 172.21.15.71:0/2683193885 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8d3c06cce0 msgr2=0x7f8d3c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:02.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.233+0000 7f8d577c6700 1 --2- 172.21.15.71:0/2683193885 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8d3c06cce0 0x7f8d3c06f1a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8d40005950 tx=0x7f8d4000b410 comp rx=0 tx=0).stop 2024-04-23T02:47:02.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.233+0000 7f8d577c6700 1 -- 172.21.15.71:0/2683193885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d50105bd0 msgr2=0x7f8d5019cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:02.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.233+0000 7f8d577c6700 1 --2- 172.21.15.71:0/2683193885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d50105bd0 0x7f8d5019cb30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c006010 tx=0x7f8d4c00bd70 comp rx=0 tx=0).stop 2024-04-23T02:47:02.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.233+0000 7f8d577c6700 1 -- 172.21.15.71:0/2683193885 shutdown_connections 2024-04-23T02:47:02.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.233+0000 7f8d577c6700 1 --2- 172.21.15.71:0/2683193885 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8d3c06cce0 0x7f8d3c06f1a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.233+0000 7f8d577c6700 1 --2- 172.21.15.71:0/2683193885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d500ffc20 0x7f8d5019c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.233+0000 7f8d577c6700 1 --2- 172.21.15.71:0/2683193885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d50105bd0 0x7f8d5019cb30 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.233+0000 7f8d577c6700 1 -- 172.21.15.71:0/2683193885 >> 172.21.15.71:0/2683193885 conn(0x7f8d50075670 msgr2=0x7f8d50103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:02.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.233+0000 7f8d577c6700 1 -- 172.21.15.71:0/2683193885 shutdown_connections 2024-04-23T02:47:02.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.233+0000 7f8d577c6700 1 -- 172.21.15.71:0/2683193885 wait complete. 2024-04-23T02:47:02.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.335+0000 7ff70dcf4700 1 -- 172.21.15.71:0/2977164037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff708105bd0 msgr2=0x7ff708105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:02.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.335+0000 7ff70dcf4700 1 --2- 172.21.15.71:0/2977164037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff708105bd0 0x7ff708105fb0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009910 tx=0x7ff6f0009c20 comp rx=0 tx=0).stop 2024-04-23T02:47:02.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.335+0000 7ff70dcf4700 1 -- 172.21.15.71:0/2977164037 shutdown_connections 2024-04-23T02:47:02.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.335+0000 7ff70dcf4700 1 --2- 172.21.15.71:0/2977164037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7080ffc20 0x7ff708100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.335+0000 7ff70dcf4700 1 --2- 172.21.15.71:0/2977164037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff708105bd0 0x7ff708105fb0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.335+0000 7ff70dcf4700 1 -- 172.21.15.71:0/2977164037 >> 172.21.15.71:0/2977164037 conn(0x7ff708075670 msgr2=0x7ff708075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:02.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.335+0000 7ff70dcf4700 1 -- 172.21.15.71:0/2977164037 shutdown_connections 2024-04-23T02:47:02.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.335+0000 7ff70dcf4700 1 -- 172.21.15.71:0/2977164037 wait complete. 2024-04-23T02:47:02.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.336+0000 7ff70dcf4700 1 Processor -- start 2024-04-23T02:47:02.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.336+0000 7ff70dcf4700 1 -- start start 2024-04-23T02:47:02.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.336+0000 7ff70dcf4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7080ffc20 0x7ff70819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:02.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.336+0000 7ff70dcf4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff708105bd0 0x7ff70819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:02.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.336+0000 7ff70dcf4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff708196610 con 0x7ff7080ffc20 2024-04-23T02:47:02.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.336+0000 7ff70dcf4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff708196780 con 0x7ff708105bd0 2024-04-23T02:47:02.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.336+0000 7ff7077fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7080ffc20 0x7ff70819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:02.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.336+0000 7ff7077fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7080ffc20 0x7ff70819c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52156/0 (socket says 172.21.15.71:52156) 2024-04-23T02:47:02.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.336+0000 7ff7077fe700 1 -- 172.21.15.71:0/296427612 learned_addr learned my addr 172.21.15.71:0/296427612 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:47:02.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.336+0000 7ff706ffd700 1 --2- 172.21.15.71:0/296427612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff708105bd0 0x7ff70819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:02.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.336+0000 7ff7077fe700 1 -- 172.21.15.71:0/296427612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff708105bd0 msgr2=0x7ff70819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:02.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.336+0000 7ff7077fe700 1 --2- 172.21.15.71:0/296427612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff708105bd0 0x7ff70819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.336+0000 7ff7077fe700 1 -- 172.21.15.71:0/296427612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f00095f0 con 0x7ff7080ffc20 2024-04-23T02:47:02.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.337+0000 7ff7077fe700 1 --2- 172.21.15.71:0/296427612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7080ffc20 0x7ff70819c620 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0006010 tx=0x7ff6f000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:02.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.337+0000 7ff704ff9700 1 -- 172.21.15.71:0/296427612 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6f001d070 con 0x7ff7080ffc20 2024-04-23T02:47:02.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.337+0000 7ff704ff9700 1 -- 172.21.15.71:0/296427612 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff6f000f460 con 0x7ff7080ffc20 2024-04-23T02:47:02.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.337+0000 7ff70dcf4700 1 -- 172.21.15.71:0/296427612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff708196a00 con 0x7ff7080ffc20 2024-04-23T02:47:02.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.337+0000 7ff704ff9700 1 -- 172.21.15.71:0/296427612 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6f0021620 con 0x7ff7080ffc20 2024-04-23T02:47:02.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.337+0000 7ff70dcf4700 1 -- 172.21.15.71:0/296427612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff708196ef0 con 0x7ff7080ffc20 2024-04-23T02:47:02.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.338+0000 7ff70dcf4700 1 -- 172.21.15.71:0/296427612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff70804f2e0 con 0x7ff7080ffc20 2024-04-23T02:47:02.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.339+0000 7ff704ff9700 1 -- 172.21.15.71:0/296427612 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff6f000f5d0 con 0x7ff7080ffc20 2024-04-23T02:47:02.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.339+0000 7ff704ff9700 1 --2- 172.21.15.71:0/296427612 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff6f406cce0 0x7ff6f406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:02.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.339+0000 7ff704ff9700 1 -- 172.21.15.71:0/296427612 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff6f008dcf0 con 0x7ff7080ffc20 2024-04-23T02:47:02.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.339+0000 7ff706ffd700 1 --2- 172.21.15.71:0/296427612 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff6f406cce0 0x7ff6f406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:02.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.339+0000 7ff706ffd700 1 --2- 172.21.15.71:0/296427612 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff6f406cce0 0x7ff6f406f1a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff708197eb0 tx=0x7ff6f800a300 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:02.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.343+0000 7ff704ff9700 1 -- 172.21.15.71:0/296427612 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff6f0057cc0 con 0x7ff7080ffc20 2024-04-23T02:47:02.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.545+0000 7ff70dcf4700 1 -- 172.21.15.71:0/296427612 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7081971b0 con 0x7ff6f406cce0 2024-04-23T02:47:02.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.555+0000 7ff704ff9700 1 -- 172.21.15.71:0/296427612 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff7081971b0 con 0x7ff6f406cce0 2024-04-23T02:47:02.558 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:47:02.558 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (30m) 8m ago 42m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:47:02.558 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (42m) 8m ago 42m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:47:02.558 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (41m) 9m ago 41m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:47:02.559 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (28m) 8m ago 42m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:47:02.559 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (28m) 9m ago 41m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:47:02.559 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (30m) 8m ago 41m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:47:02.559 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (24m) 8m ago 36m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:47:02.559 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (24m) 8m ago 36m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:47:02.559 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (9m) 9m ago 35m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:47:02.559 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (24m) 9m ago 36m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:47:02.559 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (32m) 8m ago 43m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:47:02.559 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (31m) 9m ago 40m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:47:02.559 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (29m) 8m ago 43m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:47:02.559 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (28m) 9m ago 40m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:47:02.559 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (31m) 8m ago 42m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:47:02.559 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (31m) 9m ago 41m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:47:02.560 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (28m) 8m ago 40m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:47:02.560 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (27m) 8m ago 39m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:47:02.560 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (26m) 8m ago 39m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:47:02.560 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (26m) 9m ago 38m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:47:02.560 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (25m) 9m ago 38m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:47:02.560 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (25m) 9m ago 37m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:47:02.560 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (31m) 8m ago 41m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:47:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.558+0000 7ff70dcf4700 1 -- 172.21.15.71:0/296427612 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff6f406cce0 msgr2=0x7ff6f406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.558+0000 7ff70dcf4700 1 --2- 172.21.15.71:0/296427612 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff6f406cce0 0x7ff6f406f1a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff708197eb0 tx=0x7ff6f800a300 comp rx=0 tx=0).stop 2024-04-23T02:47:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.558+0000 7ff70dcf4700 1 -- 172.21.15.71:0/296427612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7080ffc20 msgr2=0x7ff70819c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.558+0000 7ff70dcf4700 1 --2- 172.21.15.71:0/296427612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7080ffc20 0x7ff70819c620 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0006010 tx=0x7ff6f000bd70 comp rx=0 tx=0).stop 2024-04-23T02:47:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.558+0000 7ff70dcf4700 1 -- 172.21.15.71:0/296427612 shutdown_connections 2024-04-23T02:47:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.558+0000 7ff70dcf4700 1 --2- 172.21.15.71:0/296427612 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff6f406cce0 0x7ff6f406f1a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.558+0000 7ff70dcf4700 1 --2- 172.21.15.71:0/296427612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7080ffc20 0x7ff70819c620 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.558+0000 7ff70dcf4700 1 --2- 172.21.15.71:0/296427612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff708105bd0 0x7ff70819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.558+0000 7ff70dcf4700 1 -- 172.21.15.71:0/296427612 >> 172.21.15.71:0/296427612 conn(0x7ff708075670 msgr2=0x7ff708068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.558+0000 7ff70dcf4700 1 -- 172.21.15.71:0/296427612 shutdown_connections 2024-04-23T02:47:02.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.558+0000 7ff70dcf4700 1 -- 172.21.15.71:0/296427612 wait complete. 2024-04-23T02:47:02.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.661+0000 7fa17be74700 1 -- 172.21.15.71:0/274537779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1740ffc20 msgr2=0x7fa174100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:02.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.661+0000 7fa17be74700 1 --2- 172.21.15.71:0/274537779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1740ffc20 0x7fa174100080 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa164009910 tx=0x7fa164009c20 comp rx=0 tx=0).stop 2024-04-23T02:47:02.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.661+0000 7fa17be74700 1 -- 172.21.15.71:0/274537779 shutdown_connections 2024-04-23T02:47:02.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.661+0000 7fa17be74700 1 --2- 172.21.15.71:0/274537779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1740ffc20 0x7fa174100080 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.661+0000 7fa17be74700 1 --2- 172.21.15.71:0/274537779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa174105bd0 0x7fa174105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.661+0000 7fa17be74700 1 -- 172.21.15.71:0/274537779 >> 172.21.15.71:0/274537779 conn(0x7fa174075670 msgr2=0x7fa174075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:02.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.661+0000 7fa17be74700 1 -- 172.21.15.71:0/274537779 shutdown_connections 2024-04-23T02:47:02.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.661+0000 7fa17be74700 1 -- 172.21.15.71:0/274537779 wait complete. 2024-04-23T02:47:02.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.662+0000 7fa17be74700 1 Processor -- start 2024-04-23T02:47:02.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.662+0000 7fa17be74700 1 -- start start 2024-04-23T02:47:02.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.662+0000 7fa17be74700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1740ffc20 0x7fa174101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:02.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.662+0000 7fa17be74700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa174105bd0 0x7fa174101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:02.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.662+0000 7fa17be74700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1741022c0 con 0x7fa174105bd0 2024-04-23T02:47:02.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.662+0000 7fa17be74700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa174102400 con 0x7fa1740ffc20 2024-04-23T02:47:02.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.662+0000 7fa17940f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa174105bd0 0x7fa174101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:02.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.662+0000 7fa17940f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa174105bd0 0x7fa174101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52168/0 (socket says 172.21.15.71:52168) 2024-04-23T02:47:02.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.662+0000 7fa17940f700 1 -- 172.21.15.71:0/334053811 learned_addr learned my addr 172.21.15.71:0/334053811 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:47:02.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.662+0000 7fa179c10700 1 --2- 172.21.15.71:0/334053811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1740ffc20 0x7fa174101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:02.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.662+0000 7fa17940f700 1 -- 172.21.15.71:0/334053811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1740ffc20 msgr2=0x7fa174101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:02.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.662+0000 7fa17940f700 1 --2- 172.21.15.71:0/334053811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1740ffc20 0x7fa174101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.662+0000 7fa17940f700 1 -- 172.21.15.71:0/334053811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1640095f0 con 0x7fa174105bd0 2024-04-23T02:47:02.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.662+0000 7fa17940f700 1 --2- 172.21.15.71:0/334053811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa174105bd0 0x7fa174101d80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa164006010 tx=0x7fa16400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:02.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.663+0000 7fa16affd700 1 -- 172.21.15.71:0/334053811 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa16401d070 con 0x7fa174105bd0 2024-04-23T02:47:02.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.663+0000 7fa16affd700 1 -- 172.21.15.71:0/334053811 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa16400f460 con 0x7fa174105bd0 2024-04-23T02:47:02.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.663+0000 7fa17be74700 1 -- 172.21.15.71:0/334053811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa174102680 con 0x7fa174105bd0 2024-04-23T02:47:02.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.663+0000 7fa16affd700 1 -- 172.21.15.71:0/334053811 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa164021620 con 0x7fa174105bd0 2024-04-23T02:47:02.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.663+0000 7fa17be74700 1 -- 172.21.15.71:0/334053811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1741a2cb0 con 0x7fa174105bd0 2024-04-23T02:47:02.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.664+0000 7fa17be74700 1 -- 172.21.15.71:0/334053811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17404f2e0 con 0x7fa174105bd0 2024-04-23T02:47:02.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.664+0000 7fa16affd700 1 -- 172.21.15.71:0/334053811 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa16400fa40 con 0x7fa174105bd0 2024-04-23T02:47:02.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.664+0000 7fa16affd700 1 --2- 172.21.15.71:0/334053811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa16006cce0 0x7fa16006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:02.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.664+0000 7fa16affd700 1 -- 172.21.15.71:0/334053811 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa16408d6c0 con 0x7fa174105bd0 2024-04-23T02:47:02.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.664+0000 7fa179c10700 1 --2- 172.21.15.71:0/334053811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa16006cce0 0x7fa16006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:02.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.665+0000 7fa179c10700 1 --2- 172.21.15.71:0/334053811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa16006cce0 0x7fa16006f1a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa170005950 tx=0x7fa1700058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:02.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.669+0000 7fa16affd700 1 -- 172.21.15.71:0/334053811 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa1640577b0 con 0x7fa174105bd0 2024-04-23T02:47:02.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.946+0000 7fa17be74700 1 -- 172.21.15.71:0/334053811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa1741a2f20 con 0x7fa174105bd0 2024-04-23T02:47:02.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.946+0000 7fa16affd700 1 -- 172.21.15.71:0/334053811 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fa16405add0 con 0x7fa174105bd0 2024-04-23T02:47:02.949 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:47:02.950 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:47:02.950 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:47:02.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:47:02.950 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:47:02.950 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:47:02.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:47:02.950 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:47:02.950 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:47:02.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:47:02.950 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:47:02.950 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:47:02.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:47:02.950 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:47:02.950 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:47:02.951 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:47:02.951 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:47:02.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.948+0000 7fa17be74700 1 -- 172.21.15.71:0/334053811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa16006cce0 msgr2=0x7fa16006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:02.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.948+0000 7fa17be74700 1 --2- 172.21.15.71:0/334053811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa16006cce0 0x7fa16006f1a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa170005950 tx=0x7fa1700058e0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.948+0000 7fa17be74700 1 -- 172.21.15.71:0/334053811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa174105bd0 msgr2=0x7fa174101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:02.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.948+0000 7fa17be74700 1 --2- 172.21.15.71:0/334053811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa174105bd0 0x7fa174101d80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa164006010 tx=0x7fa16400be20 comp rx=0 tx=0).stop 2024-04-23T02:47:02.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.949+0000 7fa17be74700 1 -- 172.21.15.71:0/334053811 shutdown_connections 2024-04-23T02:47:02.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.949+0000 7fa17be74700 1 --2- 172.21.15.71:0/334053811 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa16006cce0 0x7fa16006f1a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.949+0000 7fa17be74700 1 --2- 172.21.15.71:0/334053811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1740ffc20 0x7fa174101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.949+0000 7fa17be74700 1 --2- 172.21.15.71:0/334053811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa174105bd0 0x7fa174101d80 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:02.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.949+0000 7fa17be74700 1 -- 172.21.15.71:0/334053811 >> 172.21.15.71:0/334053811 conn(0x7fa174075670 msgr2=0x7fa174068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:02.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.949+0000 7fa17be74700 1 -- 172.21.15.71:0/334053811 shutdown_connections 2024-04-23T02:47:02.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:02.949+0000 7fa17be74700 1 -- 172.21.15.71:0/334053811 wait complete. 2024-04-23T02:47:03.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.051+0000 7f9b836f8700 1 -- 172.21.15.71:0/3786321622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b7c0ffb40 msgr2=0x7f9b7c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:03.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.051+0000 7f9b836f8700 1 --2- 172.21.15.71:0/3786321622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b7c0ffb40 0x7f9b7c0fffc0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009910 tx=0x7f9b70009c20 comp rx=0 tx=0).stop 2024-04-23T02:47:03.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.051+0000 7f9b836f8700 1 -- 172.21.15.71:0/3786321622 shutdown_connections 2024-04-23T02:47:03.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.051+0000 7f9b836f8700 1 --2- 172.21.15.71:0/3786321622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b7c0ffb40 0x7f9b7c0fffc0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:03.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.051+0000 7f9b836f8700 1 --2- 172.21.15.71:0/3786321622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b7c105bf0 0x7f9b7c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:03.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.051+0000 7f9b836f8700 1 -- 172.21.15.71:0/3786321622 >> 172.21.15.71:0/3786321622 conn(0x7f9b7c0fb610 msgr2=0x7f9b7c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:03.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.051+0000 7f9b836f8700 1 -- 172.21.15.71:0/3786321622 shutdown_connections 2024-04-23T02:47:03.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.051+0000 7f9b836f8700 1 -- 172.21.15.71:0/3786321622 wait complete. 2024-04-23T02:47:03.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.052+0000 7f9b836f8700 1 Processor -- start 2024-04-23T02:47:03.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.052+0000 7f9b836f8700 1 -- start start 2024-04-23T02:47:03.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.052+0000 7f9b836f8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b7c0ffb40 0x7f9b7c1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:03.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.052+0000 7f9b836f8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b7c105bf0 0x7f9b7c101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:03.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.052+0000 7f9b836f8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c102230 con 0x7f9b7c0ffb40 2024-04-23T02:47:03.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.052+0000 7f9b836f8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c102370 con 0x7f9b7c105bf0 2024-04-23T02:47:03.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.052+0000 7f9b81494700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b7c0ffb40 0x7f9b7c1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:03.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.052+0000 7f9b81494700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b7c0ffb40 0x7f9b7c1017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52184/0 (socket says 172.21.15.71:52184) 2024-04-23T02:47:03.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.052+0000 7f9b81494700 1 -- 172.21.15.71:0/2224032113 learned_addr learned my addr 172.21.15.71:0/2224032113 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:47:03.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.052+0000 7f9b80c93700 1 --2- 172.21.15.71:0/2224032113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b7c105bf0 0x7f9b7c101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:03.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.052+0000 7f9b81494700 1 -- 172.21.15.71:0/2224032113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b7c105bf0 msgr2=0x7f9b7c101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:03.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.052+0000 7f9b81494700 1 --2- 172.21.15.71:0/2224032113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b7c105bf0 0x7f9b7c101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:03.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.052+0000 7f9b81494700 1 -- 172.21.15.71:0/2224032113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b700095f0 con 0x7f9b7c0ffb40 2024-04-23T02:47:03.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.052+0000 7f9b81494700 1 --2- 172.21.15.71:0/2224032113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b7c0ffb40 0x7f9b7c1017b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800d900 tx=0x7f9b7800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:03.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.052+0000 7f9b6e7fc700 1 -- 172.21.15.71:0/2224032113 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b78004a00 con 0x7f9b7c0ffb40 2024-04-23T02:47:03.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.053+0000 7f9b6e7fc700 1 -- 172.21.15.71:0/2224032113 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9b78005020 con 0x7f9b7c0ffb40 2024-04-23T02:47:03.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.053+0000 7f9b836f8700 1 -- 172.21.15.71:0/2224032113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b7c1a2900 con 0x7f9b7c0ffb40 2024-04-23T02:47:03.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.053+0000 7f9b6e7fc700 1 -- 172.21.15.71:0/2224032113 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b78009c00 con 0x7f9b7c0ffb40 2024-04-23T02:47:03.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.053+0000 7f9b836f8700 1 -- 172.21.15.71:0/2224032113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7c1a2d10 con 0x7f9b7c0ffb40 2024-04-23T02:47:03.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.054+0000 7f9b836f8700 1 -- 172.21.15.71:0/2224032113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b7c04f2e0 con 0x7f9b7c0ffb40 2024-04-23T02:47:03.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.054+0000 7f9b6e7fc700 1 -- 172.21.15.71:0/2224032113 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9b78005190 con 0x7f9b7c0ffb40 2024-04-23T02:47:03.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.054+0000 7f9b6e7fc700 1 --2- 172.21.15.71:0/2224032113 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9b6806cce0 0x7f9b6806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:03.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.054+0000 7f9b6e7fc700 1 -- 172.21.15.71:0/2224032113 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9b7808c690 con 0x7f9b7c0ffb40 2024-04-23T02:47:03.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.054+0000 7f9b80c93700 1 --2- 172.21.15.71:0/2224032113 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9b6806cce0 0x7f9b6806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:03.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.055+0000 7f9b80c93700 1 --2- 172.21.15.71:0/2224032113 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9b6806cce0 0x7f9b6806f1a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c102fa0 tx=0x7f9b7001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:03.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.059+0000 7f9b6e7fc700 1 -- 172.21.15.71:0/2224032113 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9b78056780 con 0x7f9b7c0ffb40 2024-04-23T02:47:03.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.292+0000 7f9b836f8700 1 -- 172.21.15.71:0/2224032113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9b7c1a2f50 con 0x7f9b7c0ffb40 2024-04-23T02:47:03.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.293+0000 7f9b6e7fc700 1 -- 172.21.15.71:0/2224032113 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f9b78017740 con 0x7f9b7c0ffb40 2024-04-23T02:47:03.296 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:47:03.296 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:47:03.296 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:47:03.296 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:47:03.296 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:47:03.296 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:47:03.296 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:47:03.296 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:47:03.296 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:47:03.296 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:47:03.296 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:47:03.296 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:47:03.296 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:47:03.297 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:47:03.297 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:47:03.297 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:47:03.297 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:47:03.297 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:47:03.297 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:47:03.297 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:47:03.297 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:47:03.297 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:47:03.297 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:47:03.297 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:47:03.297 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:47:03.297 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:47:03.297 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:47:03.298 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:47:03.298 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:47:03.298 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:47:03.298 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:47:03.298 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:47:03.298 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:47:03.298 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:47:03.298 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:47:03.298 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:47:03.298 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:47:03.298 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:47:03.298 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:47:03.298 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:47:03.298 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:47:03.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.295+0000 7f9b836f8700 1 -- 172.21.15.71:0/2224032113 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9b6806cce0 msgr2=0x7f9b6806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:03.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.295+0000 7f9b836f8700 1 --2- 172.21.15.71:0/2224032113 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9b6806cce0 0x7f9b6806f1a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c102fa0 tx=0x7f9b7001a040 comp rx=0 tx=0).stop 2024-04-23T02:47:03.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.295+0000 7f9b836f8700 1 -- 172.21.15.71:0/2224032113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b7c0ffb40 msgr2=0x7f9b7c1017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:03.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.295+0000 7f9b836f8700 1 --2- 172.21.15.71:0/2224032113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b7c0ffb40 0x7f9b7c1017b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800d900 tx=0x7f9b7800dcc0 comp rx=0 tx=0).stop 2024-04-23T02:47:03.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.295+0000 7f9b836f8700 1 -- 172.21.15.71:0/2224032113 shutdown_connections 2024-04-23T02:47:03.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.295+0000 7f9b836f8700 1 --2- 172.21.15.71:0/2224032113 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9b6806cce0 0x7f9b6806f1a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:03.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.295+0000 7f9b836f8700 1 --2- 172.21.15.71:0/2224032113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b7c0ffb40 0x7f9b7c1017b0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:03.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.295+0000 7f9b836f8700 1 --2- 172.21.15.71:0/2224032113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b7c105bf0 0x7f9b7c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:03.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.295+0000 7f9b836f8700 1 -- 172.21.15.71:0/2224032113 >> 172.21.15.71:0/2224032113 conn(0x7f9b7c0fb610 msgr2=0x7f9b7c068f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:03.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.295+0000 7f9b836f8700 1 -- 172.21.15.71:0/2224032113 shutdown_connections 2024-04-23T02:47:03.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.295+0000 7f9b836f8700 1 -- 172.21.15.71:0/2224032113 wait complete. 2024-04-23T02:47:03.300 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:47:03.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.398+0000 7f6ba5f7d700 1 -- 172.21.15.71:0/3716314579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba0101850 msgr2=0x7f6ba01058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:03.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.398+0000 7f6ba5f7d700 1 --2- 172.21.15.71:0/3716314579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba0101850 0x7f6ba01058a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009910 tx=0x7f6b90009c20 comp rx=0 tx=0).stop 2024-04-23T02:47:03.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.398+0000 7f6ba5f7d700 1 -- 172.21.15.71:0/3716314579 shutdown_connections 2024-04-23T02:47:03.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.398+0000 7f6ba5f7d700 1 --2- 172.21.15.71:0/3716314579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba0101850 0x7f6ba01058a0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:03.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.398+0000 7f6ba5f7d700 1 --2- 172.21.15.71:0/3716314579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba0100ea0 0x7f6ba0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:03.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.398+0000 7f6ba5f7d700 1 -- 172.21.15.71:0/3716314579 >> 172.21.15.71:0/3716314579 conn(0x7f6ba00fc750 msgr2=0x7f6ba00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:03.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.398+0000 7f6ba5f7d700 1 -- 172.21.15.71:0/3716314579 shutdown_connections 2024-04-23T02:47:03.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.398+0000 7f6ba5f7d700 1 -- 172.21.15.71:0/3716314579 wait complete. 2024-04-23T02:47:03.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.399+0000 7f6ba5f7d700 1 Processor -- start 2024-04-23T02:47:03.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.399+0000 7f6ba5f7d700 1 -- start start 2024-04-23T02:47:03.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.399+0000 7f6ba5f7d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba0100ea0 0x7f6ba019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:03.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.399+0000 7f6ba5f7d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba0101850 0x7f6ba019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:03.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.399+0000 7f6ba5f7d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba019d210 con 0x7f6ba0101850 2024-04-23T02:47:03.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.399+0000 7f6ba5f7d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba01966c0 con 0x7f6ba0100ea0 2024-04-23T02:47:03.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.400+0000 7f6b9effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba0101850 0x7f6ba019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.400+0000 7f6b9effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba0101850 0x7f6ba019cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52202/0 (socket says 172.21.15.71:52202) 2024-04-23T02:47:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.400+0000 7f6b9effd700 1 -- 172.21.15.71:0/1763026040 learned_addr learned my addr 172.21.15.71:0/1763026040 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:47:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.400+0000 7f6b9f7fe700 1 --2- 172.21.15.71:0/1763026040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba0100ea0 0x7f6ba019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.400+0000 7f6b9effd700 1 -- 172.21.15.71:0/1763026040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba0100ea0 msgr2=0x7f6ba019c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.400+0000 7f6b9effd700 1 --2- 172.21.15.71:0/1763026040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba0100ea0 0x7f6ba019c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.400+0000 7f6b9effd700 1 -- 172.21.15.71:0/1763026040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b900095f0 con 0x7f6ba0101850 2024-04-23T02:47:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.400+0000 7f6b9f7fe700 1 --2- 172.21.15.71:0/1763026040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba0100ea0 0x7f6ba019c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:47:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.400+0000 7f6b9effd700 1 --2- 172.21.15.71:0/1763026040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba0101850 0x7f6ba019cb80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f6b900092e0 tx=0x7f6b9000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.400+0000 7f6b9cff9700 1 -- 172.21.15.71:0/1763026040 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b9001d070 con 0x7f6ba0101850 2024-04-23T02:47:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.400+0000 7f6b9cff9700 1 -- 172.21.15.71:0/1763026040 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6b9000f460 con 0x7f6ba0101850 2024-04-23T02:47:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.400+0000 7f6ba5f7d700 1 -- 172.21.15.71:0/1763026040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba0196940 con 0x7f6ba0101850 2024-04-23T02:47:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.400+0000 7f6b9cff9700 1 -- 172.21.15.71:0/1763026040 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b90021600 con 0x7f6ba0101850 2024-04-23T02:47:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.400+0000 7f6ba5f7d700 1 -- 172.21.15.71:0/1763026040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba0196e30 con 0x7f6ba0101850 2024-04-23T02:47:03.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.401+0000 7f6ba5f7d700 1 -- 172.21.15.71:0/1763026040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba01092a0 con 0x7f6ba0101850 2024-04-23T02:47:03.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.402+0000 7f6b9cff9700 1 -- 172.21.15.71:0/1763026040 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6b9000f5d0 con 0x7f6ba0101850 2024-04-23T02:47:03.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.402+0000 7f6b9cff9700 1 --2- 172.21.15.71:0/1763026040 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6b8c06cce0 0x7f6b8c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:03.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.402+0000 7f6b9cff9700 1 -- 172.21.15.71:0/1763026040 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6b9008d7b0 con 0x7f6ba0101850 2024-04-23T02:47:03.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.402+0000 7f6b9f7fe700 1 --2- 172.21.15.71:0/1763026040 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6b8c06cce0 0x7f6b8c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:03.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.402+0000 7f6b9f7fe700 1 --2- 172.21.15.71:0/1763026040 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6b8c06cce0 0x7f6b8c06f1a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6b88005fd0 tx=0x7f6b88005dc0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:03.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.407+0000 7f6b9cff9700 1 -- 172.21.15.71:0/1763026040 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6b9005b2b0 con 0x7f6ba0101850 2024-04-23T02:47:03.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.618+0000 7f6ba5f7d700 1 -- 172.21.15.71:0/1763026040 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ba01970a0 con 0x7f6b8c06cce0 2024-04-23T02:47:03.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.619+0000 7f6b9cff9700 1 -- 172.21.15.71:0/1763026040 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6ba01970a0 con 0x7f6b8c06cce0 2024-04-23T02:47:03.621 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:47:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:47:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:47:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:47:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:47:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:47:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:47:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:47:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:47:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:47:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:47:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:47:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:47:03.622 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:47:03.622 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:47:03.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.621+0000 7f6ba5f7d700 1 -- 172.21.15.71:0/1763026040 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6b8c06cce0 msgr2=0x7f6b8c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:03.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.621+0000 7f6ba5f7d700 1 --2- 172.21.15.71:0/1763026040 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6b8c06cce0 0x7f6b8c06f1a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6b88005fd0 tx=0x7f6b88005dc0 comp rx=0 tx=0).stop 2024-04-23T02:47:03.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.621+0000 7f6ba5f7d700 1 -- 172.21.15.71:0/1763026040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba0101850 msgr2=0x7f6ba019cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:03.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.621+0000 7f6ba5f7d700 1 --2- 172.21.15.71:0/1763026040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba0101850 0x7f6ba019cb80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f6b900092e0 tx=0x7f6b9000be20 comp rx=0 tx=0).stop 2024-04-23T02:47:03.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.621+0000 7f6ba5f7d700 1 -- 172.21.15.71:0/1763026040 shutdown_connections 2024-04-23T02:47:03.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.621+0000 7f6ba5f7d700 1 --2- 172.21.15.71:0/1763026040 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6b8c06cce0 0x7f6b8c06f1a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:03.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.621+0000 7f6ba5f7d700 1 --2- 172.21.15.71:0/1763026040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba0100ea0 0x7f6ba019c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:03.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.621+0000 7f6ba5f7d700 1 --2- 172.21.15.71:0/1763026040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba0101850 0x7f6ba019cb80 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:03.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.621+0000 7f6ba5f7d700 1 -- 172.21.15.71:0/1763026040 >> 172.21.15.71:0/1763026040 conn(0x7f6ba00fc750 msgr2=0x7f6ba00fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:03.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.621+0000 7f6ba5f7d700 1 -- 172.21.15.71:0/1763026040 shutdown_connections 2024-04-23T02:47:03.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.621+0000 7f6ba5f7d700 1 -- 172.21.15.71:0/1763026040 wait complete. 2024-04-23T02:47:03.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.723+0000 7fa30f573700 1 -- 172.21.15.71:0/3025655602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa308105bd0 msgr2=0x7fa308105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:03.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.723+0000 7fa30f573700 1 --2- 172.21.15.71:0/3025655602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa308105bd0 0x7fa308105fb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009910 tx=0x7fa2f8009c20 comp rx=0 tx=0).stop 2024-04-23T02:47:03.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.723+0000 7fa30f573700 1 -- 172.21.15.71:0/3025655602 shutdown_connections 2024-04-23T02:47:03.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.723+0000 7fa30f573700 1 --2- 172.21.15.71:0/3025655602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3080ffc20 0x7fa308100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:03.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.723+0000 7fa30f573700 1 --2- 172.21.15.71:0/3025655602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa308105bd0 0x7fa308105fb0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:03.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.723+0000 7fa30f573700 1 -- 172.21.15.71:0/3025655602 >> 172.21.15.71:0/3025655602 conn(0x7fa308075670 msgr2=0x7fa308075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:03.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.723+0000 7fa30f573700 1 -- 172.21.15.71:0/3025655602 shutdown_connections 2024-04-23T02:47:03.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.723+0000 7fa30f573700 1 -- 172.21.15.71:0/3025655602 wait complete. 2024-04-23T02:47:03.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.724+0000 7fa30f573700 1 Processor -- start 2024-04-23T02:47:03.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.724+0000 7fa30f573700 1 -- start start 2024-04-23T02:47:03.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.724+0000 7fa30f573700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3080ffc20 0x7fa30819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:03.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.724+0000 7fa30f573700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa308105bd0 0x7fa30819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:03.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.724+0000 7fa30f573700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30819d150 con 0x7fa3080ffc20 2024-04-23T02:47:03.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.724+0000 7fa30f573700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa308196600 con 0x7fa308105bd0 2024-04-23T02:47:03.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.724+0000 7fa30d30f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3080ffc20 0x7fa30819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:03.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.724+0000 7fa30d30f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3080ffc20 0x7fa30819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52230/0 (socket says 172.21.15.71:52230) 2024-04-23T02:47:03.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.724+0000 7fa30d30f700 1 -- 172.21.15.71:0/3626581644 learned_addr learned my addr 172.21.15.71:0/3626581644 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:47:03.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.725+0000 7fa30d30f700 1 -- 172.21.15.71:0/3626581644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa308105bd0 msgr2=0x7fa30819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:03.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.725+0000 7fa30d30f700 1 --2- 172.21.15.71:0/3626581644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa308105bd0 0x7fa30819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:03.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.725+0000 7fa30d30f700 1 -- 172.21.15.71:0/3626581644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f80095f0 con 0x7fa3080ffc20 2024-04-23T02:47:03.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.725+0000 7fa30d30f700 1 --2- 172.21.15.71:0/3626581644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3080ffc20 0x7fa30819c580 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8006010 tx=0x7fa2f800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:03.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.725+0000 7fa2fe7fc700 1 -- 172.21.15.71:0/3626581644 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2f801d070 con 0x7fa3080ffc20 2024-04-23T02:47:03.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.725+0000 7fa2fe7fc700 1 -- 172.21.15.71:0/3626581644 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa2f800f460 con 0x7fa3080ffc20 2024-04-23T02:47:03.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.725+0000 7fa30f573700 1 -- 172.21.15.71:0/3626581644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa308196880 con 0x7fa3080ffc20 2024-04-23T02:47:03.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.725+0000 7fa2fe7fc700 1 -- 172.21.15.71:0/3626581644 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2f8021620 con 0x7fa3080ffc20 2024-04-23T02:47:03.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.725+0000 7fa30f573700 1 -- 172.21.15.71:0/3626581644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa308196d70 con 0x7fa3080ffc20 2024-04-23T02:47:03.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.726+0000 7fa30f573700 1 -- 172.21.15.71:0/3626581644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30804f2e0 con 0x7fa3080ffc20 2024-04-23T02:47:03.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.726+0000 7fa2fe7fc700 1 -- 172.21.15.71:0/3626581644 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa2f800fa40 con 0x7fa3080ffc20 2024-04-23T02:47:03.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.727+0000 7fa2fe7fc700 1 --2- 172.21.15.71:0/3626581644 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa2f406cce0 0x7fa2f406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:03.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.727+0000 7fa2fe7fc700 1 -- 172.21.15.71:0/3626581644 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa2f808d790 con 0x7fa3080ffc20 2024-04-23T02:47:03.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.727+0000 7fa30cb0e700 1 --2- 172.21.15.71:0/3626581644 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa2f406cce0 0x7fa2f406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:03.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.727+0000 7fa30cb0e700 1 --2- 172.21.15.71:0/3626581644 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa2f406cce0 0x7fa2f406f1a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa304005950 tx=0x7fa3040058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:03.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:03.731+0000 7fa2fe7fc700 1 -- 172.21.15.71:0/3626581644 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa2f805b290 con 0x7fa3080ffc20 2024-04-23T02:47:03.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:03 smithi192 ceph-mon[108617]: from='client.35632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:47:03.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:03 smithi192 ceph-mon[108617]: from='client.35636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:47:03.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:03 smithi192 ceph-mon[108617]: from='client.35640 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:47:03.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:03 smithi192 ceph-mon[108617]: pgmap v1013: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:03.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:03 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/334053811' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:47:03.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:03 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2224032113' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:47:03.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:03 smithi071 ceph-mon[144544]: from='client.35632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:47:03.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:03 smithi071 ceph-mon[144544]: from='client.35636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:47:03.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:03 smithi071 ceph-mon[144544]: from='client.35640 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:47:03.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:03 smithi071 ceph-mon[144544]: pgmap v1013: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:03.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:03 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/334053811' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:47:03.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:03 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2224032113' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:47:04.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:04.002+0000 7fa30f573700 1 -- 172.21.15.71:0/3626581644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa308197060 con 0x7fa3080ffc20 2024-04-23T02:47:04.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:04.002+0000 7fa2fe7fc700 1 -- 172.21.15.71:0/3626581644 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7fa2f805ae20 con 0x7fa3080ffc20 2024-04-23T02:47:04.005 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:47:04.005 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:47:04.005 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:47:04.005 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:47:04.006 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:47:04.006 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:47:04.006 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:47:04.006 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:47:04.006 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:47:04.006 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:47:04.006 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:47:04.006 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:47:04.006 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:47:04.006 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:47:04.006 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:47:04.006 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:47:04.007 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:47:04.007 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:47:04.007 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:47:04.007 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:47:04.007 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:47:04.007 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:47:04.007 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:47:04.007 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:47:04.007 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:47:04.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:04.004+0000 7fa30f573700 1 -- 172.21.15.71:0/3626581644 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa2f406cce0 msgr2=0x7fa2f406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:04.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:04.004+0000 7fa30f573700 1 --2- 172.21.15.71:0/3626581644 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa2f406cce0 0x7fa2f406f1a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa304005950 tx=0x7fa3040058e0 comp rx=0 tx=0).stop 2024-04-23T02:47:04.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:04.004+0000 7fa30f573700 1 -- 172.21.15.71:0/3626581644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3080ffc20 msgr2=0x7fa30819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:04.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:04.004+0000 7fa30f573700 1 --2- 172.21.15.71:0/3626581644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3080ffc20 0x7fa30819c580 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8006010 tx=0x7fa2f800be20 comp rx=0 tx=0).stop 2024-04-23T02:47:04.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:04.005+0000 7fa30f573700 1 -- 172.21.15.71:0/3626581644 shutdown_connections 2024-04-23T02:47:04.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:04.005+0000 7fa30f573700 1 --2- 172.21.15.71:0/3626581644 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa2f406cce0 0x7fa2f406f1a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:04.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:04.005+0000 7fa30f573700 1 --2- 172.21.15.71:0/3626581644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3080ffc20 0x7fa30819c580 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:04.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:04.005+0000 7fa30f573700 1 --2- 172.21.15.71:0/3626581644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa308105bd0 0x7fa30819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:04.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:04.005+0000 7fa30f573700 1 -- 172.21.15.71:0/3626581644 >> 172.21.15.71:0/3626581644 conn(0x7fa308075670 msgr2=0x7fa308068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:04.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:04.005+0000 7fa30f573700 1 -- 172.21.15.71:0/3626581644 shutdown_connections 2024-04-23T02:47:04.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:04.005+0000 7fa30f573700 1 -- 172.21.15.71:0/3626581644 wait complete. 2024-04-23T02:47:04.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:04 smithi192 ceph-mon[108617]: from='client.35652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:47:04.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:04 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3626581644' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:47:04.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:04 smithi071 ceph-mon[144544]: from='client.35652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:47:04.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:04 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3626581644' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:47:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:05 smithi192 ceph-mon[108617]: pgmap v1014: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:47:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:05 smithi071 ceph-mon[144544]: pgmap v1014: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:47:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:06 smithi071 ceph-mon[144544]: pgmap v1015: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:07.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:06 smithi192 ceph-mon[108617]: pgmap v1015: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:09.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:09 smithi192 ceph-mon[108617]: pgmap v1016: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:09.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:09 smithi071 ceph-mon[144544]: pgmap v1016: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:11.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:11 smithi192 ceph-mon[108617]: pgmap v1017: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:11.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:11 smithi071 ceph-mon[144544]: pgmap v1017: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:13.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:13 smithi192 ceph-mon[108617]: pgmap v1018: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:13.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:13 smithi071 ceph-mon[144544]: pgmap v1018: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:14.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:14 smithi071 ceph-mon[144544]: pgmap v1019: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:15.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:14 smithi192 ceph-mon[108617]: pgmap v1019: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:17.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:17 smithi192 ceph-mon[108617]: pgmap v1020: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:17.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:17 smithi071 ceph-mon[144544]: pgmap v1020: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:19.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:19 smithi192 ceph-mon[108617]: pgmap v1021: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:19.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:19 smithi071 ceph-mon[144544]: pgmap v1021: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:20.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:47:21.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:47:21.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:21 smithi071 ceph-mon[144544]: pgmap v1022: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:22.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:21 smithi192 ceph-mon[108617]: pgmap v1022: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:22.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:22 smithi071 ceph-mon[144544]: pgmap v1023: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:23.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:22 smithi192 ceph-mon[108617]: pgmap v1023: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:25.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:25 smithi192 ceph-mon[108617]: pgmap v1024: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:25.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:25 smithi071 ceph-mon[144544]: pgmap v1024: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:27.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:27 smithi071 ceph-mon[144544]: pgmap v1025: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:28.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:27 smithi192 ceph-mon[108617]: pgmap v1025: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:28.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:28 smithi071 ceph-mon[144544]: pgmap v1026: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:29.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:28 smithi192 ceph-mon[108617]: pgmap v1026: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:31.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:31 smithi071 ceph-mon[144544]: pgmap v1027: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:32.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:31 smithi192 ceph-mon[108617]: pgmap v1027: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:32.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:32 smithi071 ceph-mon[144544]: pgmap v1028: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:33.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:32 smithi192 ceph-mon[108617]: pgmap v1028: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:34.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.120+0000 7f5c20e9f700 1 -- 172.21.15.71:0/3710447120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c1c101850 msgr2=0x7f5c1c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:34.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.120+0000 7f5c20e9f700 1 --2- 172.21.15.71:0/3710447120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c1c101850 0x7f5c1c1058a0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009910 tx=0x7f5c0c009c20 comp rx=0 tx=0).stop 2024-04-23T02:47:34.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.120+0000 7f5c20e9f700 1 -- 172.21.15.71:0/3710447120 shutdown_connections 2024-04-23T02:47:34.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.120+0000 7f5c20e9f700 1 --2- 172.21.15.71:0/3710447120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c1c101850 0x7f5c1c1058a0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:34.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.120+0000 7f5c20e9f700 1 --2- 172.21.15.71:0/3710447120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c1c100ea0 0x7f5c1c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:34.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.120+0000 7f5c20e9f700 1 -- 172.21.15.71:0/3710447120 >> 172.21.15.71:0/3710447120 conn(0x7f5c1c0fc750 msgr2=0x7f5c1c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:34.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.120+0000 7f5c20e9f700 1 -- 172.21.15.71:0/3710447120 shutdown_connections 2024-04-23T02:47:34.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.120+0000 7f5c20e9f700 1 -- 172.21.15.71:0/3710447120 wait complete. 2024-04-23T02:47:34.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.121+0000 7f5c20e9f700 1 Processor -- start 2024-04-23T02:47:34.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.121+0000 7f5c20e9f700 1 -- start start 2024-04-23T02:47:34.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.121+0000 7f5c20e9f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c1c100ea0 0x7f5c1c19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:34.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.121+0000 7f5c20e9f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c1c101850 0x7f5c1c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:34.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.121+0000 7f5c20e9f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c19d210 con 0x7f5c1c101850 2024-04-23T02:47:34.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.121+0000 7f5c20e9f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c1966c0 con 0x7f5c1c100ea0 2024-04-23T02:47:34.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.122+0000 7f5c19d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c1c101850 0x7f5c1c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:34.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.122+0000 7f5c19d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c1c101850 0x7f5c1c19cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57210/0 (socket says 172.21.15.71:57210) 2024-04-23T02:47:34.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.122+0000 7f5c19d9b700 1 -- 172.21.15.71:0/2708306060 learned_addr learned my addr 172.21.15.71:0/2708306060 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:47:34.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.122+0000 7f5c1a59c700 1 --2- 172.21.15.71:0/2708306060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c1c100ea0 0x7f5c1c19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:34.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.122+0000 7f5c19d9b700 1 -- 172.21.15.71:0/2708306060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c1c100ea0 msgr2=0x7f5c1c19c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:34.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.122+0000 7f5c19d9b700 1 --2- 172.21.15.71:0/2708306060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c1c100ea0 0x7f5c1c19c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:34.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.122+0000 7f5c19d9b700 1 -- 172.21.15.71:0/2708306060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c0095f0 con 0x7f5c1c101850 2024-04-23T02:47:34.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.122+0000 7f5c1a59c700 1 --2- 172.21.15.71:0/2708306060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c1c100ea0 0x7f5c1c19c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:47:34.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.122+0000 7f5c19d9b700 1 --2- 172.21.15.71:0/2708306060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c1c101850 0x7f5c1c19cb80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c000c00 tx=0x7f5c0c00bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:34.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.122+0000 7f5c137fe700 1 -- 172.21.15.71:0/2708306060 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c0c01d070 con 0x7f5c1c101850 2024-04-23T02:47:34.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.122+0000 7f5c20e9f700 1 -- 172.21.15.71:0/2708306060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1c196940 con 0x7f5c1c101850 2024-04-23T02:47:34.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.122+0000 7f5c137fe700 1 -- 172.21.15.71:0/2708306060 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5c0c003cf0 con 0x7f5c1c101850 2024-04-23T02:47:34.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.122+0000 7f5c137fe700 1 -- 172.21.15.71:0/2708306060 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c0c021410 con 0x7f5c1c101850 2024-04-23T02:47:34.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.122+0000 7f5c20e9f700 1 -- 172.21.15.71:0/2708306060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1c196dd0 con 0x7f5c1c101850 2024-04-23T02:47:34.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.123+0000 7f5c20e9f700 1 -- 172.21.15.71:0/2708306060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1c1092a0 con 0x7f5c1c101850 2024-04-23T02:47:34.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.124+0000 7f5c137fe700 1 -- 172.21.15.71:0/2708306060 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5c0c003850 con 0x7f5c1c101850 2024-04-23T02:47:34.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.124+0000 7f5c137fe700 1 --2- 172.21.15.71:0/2708306060 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5c0806cce0 0x7f5c0806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:34.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.124+0000 7f5c137fe700 1 -- 172.21.15.71:0/2708306060 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5c0c08d810 con 0x7f5c1c101850 2024-04-23T02:47:34.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.124+0000 7f5c1a59c700 1 --2- 172.21.15.71:0/2708306060 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5c0806cce0 0x7f5c0806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:34.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.125+0000 7f5c1a59c700 1 --2- 172.21.15.71:0/2708306060 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5c0806cce0 0x7f5c0806f1a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009cb0 tx=0x7f5c04009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:34.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.129+0000 7f5c137fe700 1 -- 172.21.15.71:0/2708306060 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5c0c05b310 con 0x7f5c1c101850 2024-04-23T02:47:34.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.338+0000 7f5c20e9f700 1 -- 172.21.15.71:0/2708306060 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c1c197050 con 0x7f5c0806cce0 2024-04-23T02:47:34.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.339+0000 7f5c137fe700 1 -- 172.21.15.71:0/2708306060 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5c1c197050 con 0x7f5c0806cce0 2024-04-23T02:47:34.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.341+0000 7f5c20e9f700 1 -- 172.21.15.71:0/2708306060 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5c0806cce0 msgr2=0x7f5c0806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:34.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.341+0000 7f5c20e9f700 1 --2- 172.21.15.71:0/2708306060 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5c0806cce0 0x7f5c0806f1a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009cb0 tx=0x7f5c04009260 comp rx=0 tx=0).stop 2024-04-23T02:47:34.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.341+0000 7f5c20e9f700 1 -- 172.21.15.71:0/2708306060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c1c101850 msgr2=0x7f5c1c19cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:34.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.341+0000 7f5c20e9f700 1 --2- 172.21.15.71:0/2708306060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c1c101850 0x7f5c1c19cb80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c000c00 tx=0x7f5c0c00bb20 comp rx=0 tx=0).stop 2024-04-23T02:47:34.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.341+0000 7f5c20e9f700 1 -- 172.21.15.71:0/2708306060 shutdown_connections 2024-04-23T02:47:34.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.341+0000 7f5c20e9f700 1 --2- 172.21.15.71:0/2708306060 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5c0806cce0 0x7f5c0806f1a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:34.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.341+0000 7f5c20e9f700 1 --2- 172.21.15.71:0/2708306060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c1c100ea0 0x7f5c1c19c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:34.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.341+0000 7f5c20e9f700 1 --2- 172.21.15.71:0/2708306060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c1c101850 0x7f5c1c19cb80 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:34.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.341+0000 7f5c20e9f700 1 -- 172.21.15.71:0/2708306060 >> 172.21.15.71:0/2708306060 conn(0x7f5c1c0fc750 msgr2=0x7f5c1c0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:34.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.341+0000 7f5c20e9f700 1 -- 172.21.15.71:0/2708306060 shutdown_connections 2024-04-23T02:47:34.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.341+0000 7f5c20e9f700 1 -- 172.21.15.71:0/2708306060 wait complete. 2024-04-23T02:47:34.354 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:47:34.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.445+0000 7f9e33fe9700 1 -- 172.21.15.71:0/2332958330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e2c100750 msgr2=0x7f9e2c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:34.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.445+0000 7f9e33fe9700 1 --2- 172.21.15.71:0/2332958330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e2c100750 0x7f9e2c1047a0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009910 tx=0x7f9e28009c20 comp rx=0 tx=0).stop 2024-04-23T02:47:34.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.445+0000 7f9e33fe9700 1 -- 172.21.15.71:0/2332958330 shutdown_connections 2024-04-23T02:47:34.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.445+0000 7f9e33fe9700 1 --2- 172.21.15.71:0/2332958330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e2c100750 0x7f9e2c1047a0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:34.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.445+0000 7f9e33fe9700 1 --2- 172.21.15.71:0/2332958330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e2c0ffda0 0x7f9e2c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:34.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.445+0000 7f9e33fe9700 1 -- 172.21.15.71:0/2332958330 >> 172.21.15.71:0/2332958330 conn(0x7f9e2c0fb650 msgr2=0x7f9e2c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:34.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.445+0000 7f9e33fe9700 1 -- 172.21.15.71:0/2332958330 shutdown_connections 2024-04-23T02:47:34.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.445+0000 7f9e33fe9700 1 -- 172.21.15.71:0/2332958330 wait complete. 2024-04-23T02:47:34.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.446+0000 7f9e33fe9700 1 Processor -- start 2024-04-23T02:47:34.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.446+0000 7f9e33fe9700 1 -- start start 2024-04-23T02:47:34.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.446+0000 7f9e33fe9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e2c0ffda0 0x7f9e2c19b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:34.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.446+0000 7f9e33fe9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e2c100750 0x7f9e2c19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:34.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.446+0000 7f9e33fe9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c19c0f0 con 0x7f9e2c0ffda0 2024-04-23T02:47:34.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.446+0000 7f9e33fe9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c1955a0 con 0x7f9e2c100750 2024-04-23T02:47:34.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.446+0000 7f9e31d85700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e2c0ffda0 0x7f9e2c19b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:34.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.446+0000 7f9e31d85700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e2c0ffda0 0x7f9e2c19b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57232/0 (socket says 172.21.15.71:57232) 2024-04-23T02:47:34.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.446+0000 7f9e31d85700 1 -- 172.21.15.71:0/109915593 learned_addr learned my addr 172.21.15.71:0/109915593 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:47:34.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.446+0000 7f9e31584700 1 --2- 172.21.15.71:0/109915593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e2c100750 0x7f9e2c19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:34.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.446+0000 7f9e31d85700 1 -- 172.21.15.71:0/109915593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e2c100750 msgr2=0x7f9e2c19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:34.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.446+0000 7f9e31d85700 1 --2- 172.21.15.71:0/109915593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e2c100750 0x7f9e2c19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:34.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.446+0000 7f9e31d85700 1 -- 172.21.15.71:0/109915593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e280095f0 con 0x7f9e2c0ffda0 2024-04-23T02:47:34.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.446+0000 7f9e31d85700 1 --2- 172.21.15.71:0/109915593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e2c0ffda0 0x7f9e2c19b520 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000d8d0 tx=0x7f9e2000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:34.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.447+0000 7f9e1effd700 1 -- 172.21.15.71:0/109915593 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e20009940 con 0x7f9e2c0ffda0 2024-04-23T02:47:34.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.447+0000 7f9e33fe9700 1 -- 172.21.15.71:0/109915593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2c195880 con 0x7f9e2c0ffda0 2024-04-23T02:47:34.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.447+0000 7f9e1effd700 1 -- 172.21.15.71:0/109915593 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9e20010460 con 0x7f9e2c0ffda0 2024-04-23T02:47:34.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.447+0000 7f9e1effd700 1 -- 172.21.15.71:0/109915593 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e2000f5f0 con 0x7f9e2c0ffda0 2024-04-23T02:47:34.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.447+0000 7f9e33fe9700 1 -- 172.21.15.71:0/109915593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2c195dd0 con 0x7f9e2c0ffda0 2024-04-23T02:47:34.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.448+0000 7f9e33fe9700 1 -- 172.21.15.71:0/109915593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e2c1081a0 con 0x7f9e2c0ffda0 2024-04-23T02:47:34.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.448+0000 7f9e1effd700 1 -- 172.21.15.71:0/109915593 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9e200105d0 con 0x7f9e2c0ffda0 2024-04-23T02:47:34.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.448+0000 7f9e1effd700 1 --2- 172.21.15.71:0/109915593 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9e1806cce0 0x7f9e1806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:34.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.449+0000 7f9e1effd700 1 -- 172.21.15.71:0/109915593 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9e2008c5e0 con 0x7f9e2c0ffda0 2024-04-23T02:47:34.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.449+0000 7f9e31584700 1 --2- 172.21.15.71:0/109915593 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9e1806cce0 0x7f9e1806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:34.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.449+0000 7f9e31584700 1 --2- 172.21.15.71:0/109915593 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9e1806cce0 0x7f9e1806f1a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c0fbf10 tx=0x7f9e2801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:34.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.453+0000 7f9e1effd700 1 -- 172.21.15.71:0/109915593 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9e2005a0e0 con 0x7f9e2c0ffda0 2024-04-23T02:47:34.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.663+0000 7f9e33fe9700 1 -- 172.21.15.71:0/109915593 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e2c196bc0 con 0x7f9e1806cce0 2024-04-23T02:47:34.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.664+0000 7f9e1effd700 1 -- 172.21.15.71:0/109915593 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9e2c196bc0 con 0x7f9e1806cce0 2024-04-23T02:47:34.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.666+0000 7f9e33fe9700 1 -- 172.21.15.71:0/109915593 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9e1806cce0 msgr2=0x7f9e1806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:34.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.666+0000 7f9e33fe9700 1 --2- 172.21.15.71:0/109915593 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9e1806cce0 0x7f9e1806f1a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c0fbf10 tx=0x7f9e2801a040 comp rx=0 tx=0).stop 2024-04-23T02:47:34.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.666+0000 7f9e33fe9700 1 -- 172.21.15.71:0/109915593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e2c0ffda0 msgr2=0x7f9e2c19b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:34.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.666+0000 7f9e33fe9700 1 --2- 172.21.15.71:0/109915593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e2c0ffda0 0x7f9e2c19b520 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000d8d0 tx=0x7f9e2000dc90 comp rx=0 tx=0).stop 2024-04-23T02:47:34.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.666+0000 7f9e33fe9700 1 -- 172.21.15.71:0/109915593 shutdown_connections 2024-04-23T02:47:34.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.666+0000 7f9e33fe9700 1 --2- 172.21.15.71:0/109915593 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9e1806cce0 0x7f9e1806f1a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:34.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.666+0000 7f9e33fe9700 1 --2- 172.21.15.71:0/109915593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e2c0ffda0 0x7f9e2c19b520 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:34.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.666+0000 7f9e33fe9700 1 --2- 172.21.15.71:0/109915593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e2c100750 0x7f9e2c19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:34.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.666+0000 7f9e33fe9700 1 -- 172.21.15.71:0/109915593 >> 172.21.15.71:0/109915593 conn(0x7f9e2c0fb650 msgr2=0x7f9e2c0fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:34.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.666+0000 7f9e33fe9700 1 -- 172.21.15.71:0/109915593 shutdown_connections 2024-04-23T02:47:34.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.666+0000 7f9e33fe9700 1 -- 172.21.15.71:0/109915593 wait complete. 2024-04-23T02:47:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.769+0000 7f68001a8700 1 -- 172.21.15.71:0/2376009681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67f8100750 msgr2=0x7f67f81047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.769+0000 7f68001a8700 1 --2- 172.21.15.71:0/2376009681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67f8100750 0x7f67f81047a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009910 tx=0x7f67f4009c20 comp rx=0 tx=0).stop 2024-04-23T02:47:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.769+0000 7f68001a8700 1 -- 172.21.15.71:0/2376009681 shutdown_connections 2024-04-23T02:47:34.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.769+0000 7f68001a8700 1 --2- 172.21.15.71:0/2376009681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67f8100750 0x7f67f81047a0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:34.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.769+0000 7f68001a8700 1 --2- 172.21.15.71:0/2376009681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67f80ffda0 0x7f67f8100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:34.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.769+0000 7f68001a8700 1 -- 172.21.15.71:0/2376009681 >> 172.21.15.71:0/2376009681 conn(0x7f67f80fb650 msgr2=0x7f67f80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:34.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.769+0000 7f68001a8700 1 -- 172.21.15.71:0/2376009681 shutdown_connections 2024-04-23T02:47:34.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.769+0000 7f68001a8700 1 -- 172.21.15.71:0/2376009681 wait complete. 2024-04-23T02:47:34.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.770+0000 7f68001a8700 1 Processor -- start 2024-04-23T02:47:34.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.770+0000 7f68001a8700 1 -- start start 2024-04-23T02:47:34.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.770+0000 7f68001a8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67f80ffda0 0x7f67f819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:34.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.770+0000 7f68001a8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67f8100750 0x7f67f819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:34.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.770+0000 7f68001a8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f819d240 con 0x7f67f8100750 2024-04-23T02:47:34.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.770+0000 7f68001a8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f81966f0 con 0x7f67f80ffda0 2024-04-23T02:47:34.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.770+0000 7f67fd743700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67f8100750 0x7f67f819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:34.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.770+0000 7f67fd743700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67f8100750 0x7f67f819cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57260/0 (socket says 172.21.15.71:57260) 2024-04-23T02:47:34.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.770+0000 7f67fd743700 1 -- 172.21.15.71:0/4283091501 learned_addr learned my addr 172.21.15.71:0/4283091501 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:47:34.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.770+0000 7f67fdf44700 1 --2- 172.21.15.71:0/4283091501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67f80ffda0 0x7f67f819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:34.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.771+0000 7f67fd743700 1 -- 172.21.15.71:0/4283091501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67f80ffda0 msgr2=0x7f67f819c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:34.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.771+0000 7f67fd743700 1 --2- 172.21.15.71:0/4283091501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67f80ffda0 0x7f67f819c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:34.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.771+0000 7f67fd743700 1 -- 172.21.15.71:0/4283091501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f40095f0 con 0x7f67f8100750 2024-04-23T02:47:34.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.771+0000 7f67fd743700 1 --2- 172.21.15.71:0/4283091501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67f8100750 0x7f67f819cbb0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f67f40092e0 tx=0x7f67f400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:34.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.771+0000 7f67eeffd700 1 -- 172.21.15.71:0/4283091501 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67f401d070 con 0x7f67f8100750 2024-04-23T02:47:34.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.771+0000 7f67eeffd700 1 -- 172.21.15.71:0/4283091501 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f67f400f460 con 0x7f67f8100750 2024-04-23T02:47:34.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.771+0000 7f68001a8700 1 -- 172.21.15.71:0/4283091501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f8196970 con 0x7f67f8100750 2024-04-23T02:47:34.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.771+0000 7f67eeffd700 1 -- 172.21.15.71:0/4283091501 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67f4021620 con 0x7f67f8100750 2024-04-23T02:47:34.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.771+0000 7f68001a8700 1 -- 172.21.15.71:0/4283091501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f8196e60 con 0x7f67f8100750 2024-04-23T02:47:34.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.772+0000 7f68001a8700 1 -- 172.21.15.71:0/4283091501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f81091b0 con 0x7f67f8100750 2024-04-23T02:47:34.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.772+0000 7f67eeffd700 1 -- 172.21.15.71:0/4283091501 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f67f400f5d0 con 0x7f67f8100750 2024-04-23T02:47:34.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.773+0000 7f67eeffd700 1 --2- 172.21.15.71:0/4283091501 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f67e406cce0 0x7f67e406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:34.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.773+0000 7f67eeffd700 1 -- 172.21.15.71:0/4283091501 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f67f408d780 con 0x7f67f8100750 2024-04-23T02:47:34.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.773+0000 7f67fdf44700 1 --2- 172.21.15.71:0/4283091501 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f67e406cce0 0x7f67e406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:34.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.773+0000 7f67fdf44700 1 --2- 172.21.15.71:0/4283091501 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f67e406cce0 0x7f67e406f1a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f67e8005950 tx=0x7f67e800a300 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:34.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.777+0000 7f67eeffd700 1 -- 172.21.15.71:0/4283091501 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f67f405b300 con 0x7f67f8100750 2024-04-23T02:47:34.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.979+0000 7f68001a8700 1 -- 172.21.15.71:0/4283091501 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f67f81970d0 con 0x7f67e406cce0 2024-04-23T02:47:34.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.990+0000 7f67eeffd700 1 -- 172.21.15.71:0/4283091501 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f67f81970d0 con 0x7f67e406cce0 2024-04-23T02:47:34.993 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:47:34.993 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (31m) 9m ago 42m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:47:34.993 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (42m) 9m ago 42m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:47:34.993 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (41m) 9m ago 41m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:47:34.993 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (29m) 9m ago 42m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:47:34.993 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (29m) 9m ago 41m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:47:34.993 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (30m) 9m ago 42m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:47:34.993 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (25m) 9m ago 36m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:47:34.993 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (25m) 9m ago 36m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:47:34.993 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (9m) 9m ago 36m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:47:34.993 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (24m) 9m ago 36m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:47:34.993 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (33m) 9m ago 44m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:47:34.993 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (32m) 9m ago 41m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:47:34.994 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (29m) 9m ago 44m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:47:34.994 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (29m) 9m ago 41m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:47:34.994 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (32m) 9m ago 42m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:47:34.994 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (32m) 9m ago 41m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:47:34.994 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (28m) 9m ago 40m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:47:34.994 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (28m) 9m ago 40m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:47:34.994 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (27m) 9m ago 39m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:47:34.994 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (26m) 9m ago 39m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:47:34.994 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (26m) 9m ago 38m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:47:34.994 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (25m) 9m ago 38m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:47:34.994 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (31m) 9m ago 42m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:47:34.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.992+0000 7f68001a8700 1 -- 172.21.15.71:0/4283091501 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f67e406cce0 msgr2=0x7f67e406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:34.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.992+0000 7f68001a8700 1 --2- 172.21.15.71:0/4283091501 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f67e406cce0 0x7f67e406f1a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f67e8005950 tx=0x7f67e800a300 comp rx=0 tx=0).stop 2024-04-23T02:47:34.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.992+0000 7f68001a8700 1 -- 172.21.15.71:0/4283091501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67f8100750 msgr2=0x7f67f819cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:34.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.992+0000 7f68001a8700 1 --2- 172.21.15.71:0/4283091501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67f8100750 0x7f67f819cbb0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f67f40092e0 tx=0x7f67f400bd70 comp rx=0 tx=0).stop 2024-04-23T02:47:34.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.992+0000 7f68001a8700 1 -- 172.21.15.71:0/4283091501 shutdown_connections 2024-04-23T02:47:34.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.992+0000 7f68001a8700 1 --2- 172.21.15.71:0/4283091501 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f67e406cce0 0x7f67e406f1a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:34.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.992+0000 7f68001a8700 1 --2- 172.21.15.71:0/4283091501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67f80ffda0 0x7f67f819c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:34.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.992+0000 7f68001a8700 1 --2- 172.21.15.71:0/4283091501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67f8100750 0x7f67f819cbb0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:34.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.992+0000 7f68001a8700 1 -- 172.21.15.71:0/4283091501 >> 172.21.15.71:0/4283091501 conn(0x7f67f80fb650 msgr2=0x7f67f8069210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:34.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.992+0000 7f68001a8700 1 -- 172.21.15.71:0/4283091501 shutdown_connections 2024-04-23T02:47:34.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:34.992+0000 7f68001a8700 1 -- 172.21.15.71:0/4283091501 wait complete. 2024-04-23T02:47:35.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.093+0000 7fec3d29b700 1 -- 172.21.15.71:0/1207521964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec38100310 msgr2=0x7fec3810d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:35.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.093+0000 7fec3d29b700 1 --2- 172.21.15.71:0/1207521964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec38100310 0x7fec3810d060 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fec20009910 tx=0x7fec20009c20 comp rx=0 tx=0).stop 2024-04-23T02:47:35.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.093+0000 7fec3d29b700 1 -- 172.21.15.71:0/1207521964 shutdown_connections 2024-04-23T02:47:35.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.093+0000 7fec3d29b700 1 --2- 172.21.15.71:0/1207521964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec38100310 0x7fec3810d060 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.093+0000 7fec3d29b700 1 --2- 172.21.15.71:0/1207521964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec380ff9f0 0x7fec380ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.093+0000 7fec3d29b700 1 -- 172.21.15.71:0/1207521964 >> 172.21.15.71:0/1207521964 conn(0x7fec380fb650 msgr2=0x7fec380fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:35.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.093+0000 7fec3d29b700 1 -- 172.21.15.71:0/1207521964 shutdown_connections 2024-04-23T02:47:35.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.093+0000 7fec3d29b700 1 -- 172.21.15.71:0/1207521964 wait complete. 2024-04-23T02:47:35.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.094+0000 7fec3d29b700 1 Processor -- start 2024-04-23T02:47:35.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.094+0000 7fec3d29b700 1 -- start start 2024-04-23T02:47:35.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.094+0000 7fec3d29b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec380ff9f0 0x7fec38198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:35.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.094+0000 7fec3d29b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec38100310 0x7fec38198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:35.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.094+0000 7fec3d29b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec38199560 con 0x7fec38100310 2024-04-23T02:47:35.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.094+0000 7fec3d29b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3819d370 con 0x7fec380ff9f0 2024-04-23T02:47:35.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.094+0000 7fec2e7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec38100310 0x7fec38198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:35.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.094+0000 7fec2e7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec38100310 0x7fec38198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57276/0 (socket says 172.21.15.71:57276) 2024-04-23T02:47:35.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.094+0000 7fec2e7fc700 1 -- 172.21.15.71:0/1954626726 learned_addr learned my addr 172.21.15.71:0/1954626726 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:47:35.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.094+0000 7fec36ffd700 1 --2- 172.21.15.71:0/1954626726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec380ff9f0 0x7fec38198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:35.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.094+0000 7fec2e7fc700 1 -- 172.21.15.71:0/1954626726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec380ff9f0 msgr2=0x7fec38198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:35.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.094+0000 7fec2e7fc700 1 --2- 172.21.15.71:0/1954626726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec380ff9f0 0x7fec38198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.095+0000 7fec2e7fc700 1 -- 172.21.15.71:0/1954626726 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec200095f0 con 0x7fec38100310 2024-04-23T02:47:35.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.095+0000 7fec2e7fc700 1 --2- 172.21.15.71:0/1954626726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec38100310 0x7fec38198ed0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fec20005620 tx=0x7fec20005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:35.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.095+0000 7fec34ff9700 1 -- 172.21.15.71:0/1954626726 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec2001d070 con 0x7fec38100310 2024-04-23T02:47:35.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.095+0000 7fec3d29b700 1 -- 172.21.15.71:0/1954626726 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3819d5f0 con 0x7fec38100310 2024-04-23T02:47:35.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.095+0000 7fec34ff9700 1 -- 172.21.15.71:0/1954626726 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fec2000f460 con 0x7fec38100310 2024-04-23T02:47:35.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.095+0000 7fec34ff9700 1 -- 172.21.15.71:0/1954626726 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec20021640 con 0x7fec38100310 2024-04-23T02:47:35.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.095+0000 7fec3d29b700 1 -- 172.21.15.71:0/1954626726 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3819db40 con 0x7fec38100310 2024-04-23T02:47:35.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.096+0000 7fec3d29b700 1 -- 172.21.15.71:0/1954626726 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec3810a7d0 con 0x7fec38100310 2024-04-23T02:47:35.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.096+0000 7fec34ff9700 1 -- 172.21.15.71:0/1954626726 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fec20003a60 con 0x7fec38100310 2024-04-23T02:47:35.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.097+0000 7fec34ff9700 1 --2- 172.21.15.71:0/1954626726 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fec2406cce0 0x7fec2406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:35.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.097+0000 7fec34ff9700 1 -- 172.21.15.71:0/1954626726 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fec2008d800 con 0x7fec38100310 2024-04-23T02:47:35.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.097+0000 7fec36ffd700 1 --2- 172.21.15.71:0/1954626726 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fec2406cce0 0x7fec2406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:35.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.097+0000 7fec36ffd700 1 --2- 172.21.15.71:0/1954626726 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fec2406cce0 0x7fec2406f1a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fec28009af0 tx=0x7fec280094f0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:35.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.101+0000 7fec34ff9700 1 -- 172.21.15.71:0/1954626726 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fec2005b300 con 0x7fec38100310 2024-04-23T02:47:35.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.373+0000 7fec3d29b700 1 -- 172.21.15.71:0/1954626726 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fec3804f2e0 con 0x7fec38100310 2024-04-23T02:47:35.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.374+0000 7fec34ff9700 1 -- 172.21.15.71:0/1954626726 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fec2005ae90 con 0x7fec38100310 2024-04-23T02:47:35.377 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:47:35.377 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:47:35.377 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:47:35.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:47:35.378 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:47:35.378 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:47:35.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:47:35.378 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:47:35.378 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:47:35.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:47:35.378 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:47:35.378 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:47:35.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:47:35.378 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:47:35.378 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:47:35.378 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:47:35.378 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:47:35.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.376+0000 7fec3d29b700 1 -- 172.21.15.71:0/1954626726 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fec2406cce0 msgr2=0x7fec2406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:35.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.376+0000 7fec3d29b700 1 --2- 172.21.15.71:0/1954626726 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fec2406cce0 0x7fec2406f1a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fec28009af0 tx=0x7fec280094f0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.376+0000 7fec3d29b700 1 -- 172.21.15.71:0/1954626726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec38100310 msgr2=0x7fec38198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:35.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.376+0000 7fec3d29b700 1 --2- 172.21.15.71:0/1954626726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec38100310 0x7fec38198ed0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fec20005620 tx=0x7fec20005700 comp rx=0 tx=0).stop 2024-04-23T02:47:35.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.376+0000 7fec3d29b700 1 -- 172.21.15.71:0/1954626726 shutdown_connections 2024-04-23T02:47:35.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.376+0000 7fec3d29b700 1 --2- 172.21.15.71:0/1954626726 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fec2406cce0 0x7fec2406f1a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.376+0000 7fec3d29b700 1 --2- 172.21.15.71:0/1954626726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec380ff9f0 0x7fec38198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.376+0000 7fec3d29b700 1 --2- 172.21.15.71:0/1954626726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec38100310 0x7fec38198ed0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.376+0000 7fec3d29b700 1 -- 172.21.15.71:0/1954626726 >> 172.21.15.71:0/1954626726 conn(0x7fec380fb650 msgr2=0x7fec380fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:35.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.376+0000 7fec3d29b700 1 -- 172.21.15.71:0/1954626726 shutdown_connections 2024-04-23T02:47:35.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.376+0000 7fec3d29b700 1 -- 172.21.15.71:0/1954626726 wait complete. 2024-04-23T02:47:35.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.478+0000 7fa1c8ea1700 1 -- 172.21.15.71:0/970922215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4100ea0 msgr2=0x7fa1c4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:35.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.478+0000 7fa1c8ea1700 1 --2- 172.21.15.71:0/970922215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4100ea0 0x7fa1c4101280 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400b0a0 tx=0x7fa1b400b3b0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.478+0000 7fa1c8ea1700 1 -- 172.21.15.71:0/970922215 shutdown_connections 2024-04-23T02:47:35.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.478+0000 7fa1c8ea1700 1 --2- 172.21.15.71:0/970922215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c4101850 0x7fa1c41058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.478+0000 7fa1c8ea1700 1 --2- 172.21.15.71:0/970922215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4100ea0 0x7fa1c4101280 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.478+0000 7fa1c8ea1700 1 -- 172.21.15.71:0/970922215 >> 172.21.15.71:0/970922215 conn(0x7fa1c40fc750 msgr2=0x7fa1c40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:35.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.478+0000 7fa1c8ea1700 1 -- 172.21.15.71:0/970922215 shutdown_connections 2024-04-23T02:47:35.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.478+0000 7fa1c8ea1700 1 -- 172.21.15.71:0/970922215 wait complete. 2024-04-23T02:47:35.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.479+0000 7fa1c8ea1700 1 Processor -- start 2024-04-23T02:47:35.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.479+0000 7fa1c8ea1700 1 -- start start 2024-04-23T02:47:35.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.479+0000 7fa1c8ea1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4100ea0 0x7fa1c419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:35.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.479+0000 7fa1c8ea1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c4101850 0x7fa1c419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:35.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.479+0000 7fa1c8ea1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c419d130 con 0x7fa1c4100ea0 2024-04-23T02:47:35.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.479+0000 7fa1c8ea1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c41965e0 con 0x7fa1c4101850 2024-04-23T02:47:35.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.479+0000 7fa1c259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4100ea0 0x7fa1c419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:35.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.479+0000 7fa1c259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4100ea0 0x7fa1c419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46684/0 (socket says 172.21.15.71:46684) 2024-04-23T02:47:35.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.479+0000 7fa1c259c700 1 -- 172.21.15.71:0/3297111325 learned_addr learned my addr 172.21.15.71:0/3297111325 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:47:35.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.479+0000 7fa1c1d9b700 1 --2- 172.21.15.71:0/3297111325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c4101850 0x7fa1c419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:35.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.479+0000 7fa1c259c700 1 -- 172.21.15.71:0/3297111325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c4101850 msgr2=0x7fa1c419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:35.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.479+0000 7fa1c259c700 1 --2- 172.21.15.71:0/3297111325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c4101850 0x7fa1c419caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.479+0000 7fa1c259c700 1 -- 172.21.15.71:0/3297111325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b4009d00 con 0x7fa1c4100ea0 2024-04-23T02:47:35.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.479+0000 7fa1c1d9b700 1 --2- 172.21.15.71:0/3297111325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c4101850 0x7fa1c419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:47:35.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.479+0000 7fa1c259c700 1 --2- 172.21.15.71:0/3297111325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4100ea0 0x7fa1c419c560 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400d010 tx=0x7fa1b4007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:35.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.480+0000 7fa1bb7fe700 1 -- 172.21.15.71:0/3297111325 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1b400e040 con 0x7fa1c4100ea0 2024-04-23T02:47:35.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.480+0000 7fa1bb7fe700 1 -- 172.21.15.71:0/3297111325 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa1b4003e60 con 0x7fa1c4100ea0 2024-04-23T02:47:35.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.480+0000 7fa1c8ea1700 1 -- 172.21.15.71:0/3297111325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c4196860 con 0x7fa1c4100ea0 2024-04-23T02:47:35.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.480+0000 7fa1bb7fe700 1 -- 172.21.15.71:0/3297111325 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1b401d7c0 con 0x7fa1c4100ea0 2024-04-23T02:47:35.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.480+0000 7fa1c8ea1700 1 -- 172.21.15.71:0/3297111325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c4196d50 con 0x7fa1c4100ea0 2024-04-23T02:47:35.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.481+0000 7fa1c8ea1700 1 -- 172.21.15.71:0/3297111325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c4109240 con 0x7fa1c4100ea0 2024-04-23T02:47:35.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.481+0000 7fa1bb7fe700 1 -- 172.21.15.71:0/3297111325 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa1b40046e0 con 0x7fa1c4100ea0 2024-04-23T02:47:35.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.481+0000 7fa1bb7fe700 1 --2- 172.21.15.71:0/3297111325 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa1b006cce0 0x7fa1b006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:35.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.481+0000 7fa1bb7fe700 1 -- 172.21.15.71:0/3297111325 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa1b408df80 con 0x7fa1c4100ea0 2024-04-23T02:47:35.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.481+0000 7fa1c1d9b700 1 --2- 172.21.15.71:0/3297111325 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa1b006cce0 0x7fa1b006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:35.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.482+0000 7fa1c1d9b700 1 --2- 172.21.15.71:0/3297111325 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa1b006cce0 0x7fa1b006f1a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac005950 tx=0x7fa1ac0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:35.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.486+0000 7fa1bb7fe700 1 -- 172.21.15.71:0/3297111325 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa1b405ba80 con 0x7fa1c4100ea0 2024-04-23T02:47:35.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.722+0000 7fa1c8ea1700 1 -- 172.21.15.71:0/3297111325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa1c404f2e0 con 0x7fa1c4100ea0 2024-04-23T02:47:35.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.723+0000 7fa1bb7fe700 1 -- 172.21.15.71:0/3297111325 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7fa1b405b610 con 0x7fa1c4100ea0 2024-04-23T02:47:35.726 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:47:35.726 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:47:35.726 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:47:35.726 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:47:35.726 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:47:35.726 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:47:35.726 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:47:35.726 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:47:35.727 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:47:35.727 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:47:35.727 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:47:35.727 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:47:35.727 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:47:35.727 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:47:35.727 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:47:35.727 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:47:35.727 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:47:35.727 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:47:35.727 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:47:35.727 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:47:35.727 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:47:35.727 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:47:35.728 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:47:35.728 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:47:35.728 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:47:35.728 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:47:35.728 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:47:35.728 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:47:35.728 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:47:35.728 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:47:35.728 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:47:35.728 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:47:35.728 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:47:35.729 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:47:35.729 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:47:35.729 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:47:35.729 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:47:35.729 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:47:35.729 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:47:35.729 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:47:35.729 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:47:35.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.725+0000 7fa1c8ea1700 1 -- 172.21.15.71:0/3297111325 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa1b006cce0 msgr2=0x7fa1b006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:35.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.725+0000 7fa1c8ea1700 1 --2- 172.21.15.71:0/3297111325 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa1b006cce0 0x7fa1b006f1a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac005950 tx=0x7fa1ac0058e0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.725+0000 7fa1c8ea1700 1 -- 172.21.15.71:0/3297111325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4100ea0 msgr2=0x7fa1c419c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:35.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.725+0000 7fa1c8ea1700 1 --2- 172.21.15.71:0/3297111325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4100ea0 0x7fa1c419c560 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400d010 tx=0x7fa1b4007be0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.725+0000 7fa1c8ea1700 1 -- 172.21.15.71:0/3297111325 shutdown_connections 2024-04-23T02:47:35.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.725+0000 7fa1c8ea1700 1 --2- 172.21.15.71:0/3297111325 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa1b006cce0 0x7fa1b006f1a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.725+0000 7fa1c8ea1700 1 --2- 172.21.15.71:0/3297111325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4100ea0 0x7fa1c419c560 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.725+0000 7fa1c8ea1700 1 --2- 172.21.15.71:0/3297111325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1c4101850 0x7fa1c419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.725+0000 7fa1c8ea1700 1 -- 172.21.15.71:0/3297111325 >> 172.21.15.71:0/3297111325 conn(0x7fa1c40fc750 msgr2=0x7fa1c40fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:35.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.725+0000 7fa1c8ea1700 1 -- 172.21.15.71:0/3297111325 shutdown_connections 2024-04-23T02:47:35.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.725+0000 7fa1c8ea1700 1 -- 172.21.15.71:0/3297111325 wait complete. 2024-04-23T02:47:35.731 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:47:35.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.828+0000 7f814b005700 1 -- 172.21.15.71:0/704138353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81440ffc20 msgr2=0x7f8144100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:35.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.828+0000 7f814b005700 1 --2- 172.21.15.71:0/704138353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81440ffc20 0x7f8144100080 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8130009910 tx=0x7f8130009c20 comp rx=0 tx=0).stop 2024-04-23T02:47:35.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.828+0000 7f814b005700 1 -- 172.21.15.71:0/704138353 shutdown_connections 2024-04-23T02:47:35.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.828+0000 7f814b005700 1 --2- 172.21.15.71:0/704138353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81440ffc20 0x7f8144100080 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.828+0000 7f814b005700 1 --2- 172.21.15.71:0/704138353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8144105bd0 0x7f8144105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.828+0000 7f814b005700 1 -- 172.21.15.71:0/704138353 >> 172.21.15.71:0/704138353 conn(0x7f8144075670 msgr2=0x7f8144075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:35.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.828+0000 7f814b005700 1 -- 172.21.15.71:0/704138353 shutdown_connections 2024-04-23T02:47:35.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.828+0000 7f814b005700 1 -- 172.21.15.71:0/704138353 wait complete. 2024-04-23T02:47:35.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.829+0000 7f814b005700 1 Processor -- start 2024-04-23T02:47:35.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.829+0000 7f814b005700 1 -- start start 2024-04-23T02:47:35.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.829+0000 7f814b005700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81440ffc20 0x7f8144197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:35.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.829+0000 7f814b005700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8144105bd0 0x7f8144197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:35.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.829+0000 7f814b005700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8144198320 con 0x7f8144105bd0 2024-04-23T02:47:35.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.829+0000 7f814b005700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8144198460 con 0x7f81440ffc20 2024-04-23T02:47:35.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.829+0000 7f8143fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8144105bd0 0x7f8144197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:35.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.829+0000 7f8143fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8144105bd0 0x7f8144197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46692/0 (socket says 172.21.15.71:46692) 2024-04-23T02:47:35.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.829+0000 7f8143fff700 1 -- 172.21.15.71:0/1623696166 learned_addr learned my addr 172.21.15.71:0/1623696166 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:47:35.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.829+0000 7f8148da1700 1 --2- 172.21.15.71:0/1623696166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81440ffc20 0x7f8144197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:35.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.829+0000 7f8143fff700 1 -- 172.21.15.71:0/1623696166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81440ffc20 msgr2=0x7f8144197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:35.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.829+0000 7f8143fff700 1 --2- 172.21.15.71:0/1623696166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81440ffc20 0x7f8144197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:35.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.829+0000 7f8143fff700 1 -- 172.21.15.71:0/1623696166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81300095f0 con 0x7f8144105bd0 2024-04-23T02:47:35.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.829+0000 7f8143fff700 1 --2- 172.21.15.71:0/1623696166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8144105bd0 0x7f8144197c40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8130006010 tx=0x7f813000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:35.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.829+0000 7f8141ffb700 1 -- 172.21.15.71:0/1623696166 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f813001d070 con 0x7f8144105bd0 2024-04-23T02:47:35.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.830+0000 7f814b005700 1 -- 172.21.15.71:0/1623696166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81441a0770 con 0x7f8144105bd0 2024-04-23T02:47:35.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.830+0000 7f8141ffb700 1 -- 172.21.15.71:0/1623696166 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f813000f460 con 0x7f8144105bd0 2024-04-23T02:47:35.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.830+0000 7f8141ffb700 1 -- 172.21.15.71:0/1623696166 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8130021640 con 0x7f8144105bd0 2024-04-23T02:47:35.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.830+0000 7f814b005700 1 -- 172.21.15.71:0/1623696166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81441a0be0 con 0x7f8144105bd0 2024-04-23T02:47:35.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.831+0000 7f814b005700 1 -- 172.21.15.71:0/1623696166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81441080f0 con 0x7f8144105bd0 2024-04-23T02:47:35.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.832+0000 7f8141ffb700 1 -- 172.21.15.71:0/1623696166 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f813000fa20 con 0x7f8144105bd0 2024-04-23T02:47:35.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.832+0000 7f8141ffb700 1 --2- 172.21.15.71:0/1623696166 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f813406cce0 0x7f813406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:35.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.832+0000 7f8141ffb700 1 -- 172.21.15.71:0/1623696166 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f813008dd60 con 0x7f8144105bd0 2024-04-23T02:47:35.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.832+0000 7f8148da1700 1 --2- 172.21.15.71:0/1623696166 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f813406cce0 0x7f813406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:35.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.832+0000 7f8148da1700 1 --2- 172.21.15.71:0/1623696166 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f813406cce0 0x7f813406f1a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8138006fd0 tx=0x7f8138008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:35.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:35.836+0000 7f8141ffb700 1 -- 172.21.15.71:0/1623696166 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8130057ed0 con 0x7f8144105bd0 2024-04-23T02:47:35.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:35 smithi192 ceph-mon[108617]: from='client.35660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:47:35.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:35 smithi192 ceph-mon[108617]: pgmap v1029: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:35.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:47:35.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:35 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1954626726' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:47:35.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:35 smithi071 ceph-mon[144544]: from='client.35660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:47:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:35 smithi071 ceph-mon[144544]: pgmap v1029: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:47:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:35 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1954626726' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:47:36.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.050+0000 7f814b005700 1 -- 172.21.15.71:0/1623696166 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f814402cce0 con 0x7f813406cce0 2024-04-23T02:47:36.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.052+0000 7f8141ffb700 1 -- 172.21.15.71:0/1623696166 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f814402cce0 con 0x7f813406cce0 2024-04-23T02:47:36.055 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:47:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:47:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:47:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:47:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:47:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:47:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:47:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:47:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:47:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:47:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:47:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:47:36.055 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:47:36.056 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:47:36.056 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:47:36.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.054+0000 7f814b005700 1 -- 172.21.15.71:0/1623696166 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f813406cce0 msgr2=0x7f813406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:36.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.054+0000 7f814b005700 1 --2- 172.21.15.71:0/1623696166 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f813406cce0 0x7f813406f1a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8138006fd0 tx=0x7f8138008040 comp rx=0 tx=0).stop 2024-04-23T02:47:36.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.054+0000 7f814b005700 1 -- 172.21.15.71:0/1623696166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8144105bd0 msgr2=0x7f8144197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:36.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.054+0000 7f814b005700 1 --2- 172.21.15.71:0/1623696166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8144105bd0 0x7f8144197c40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8130006010 tx=0x7f813000be20 comp rx=0 tx=0).stop 2024-04-23T02:47:36.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.054+0000 7f814b005700 1 -- 172.21.15.71:0/1623696166 shutdown_connections 2024-04-23T02:47:36.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.054+0000 7f814b005700 1 --2- 172.21.15.71:0/1623696166 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f813406cce0 0x7f813406f1a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:36.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.054+0000 7f814b005700 1 --2- 172.21.15.71:0/1623696166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81440ffc20 0x7f8144197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:36.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.054+0000 7f814b005700 1 --2- 172.21.15.71:0/1623696166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8144105bd0 0x7f8144197c40 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:36.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.054+0000 7f814b005700 1 -- 172.21.15.71:0/1623696166 >> 172.21.15.71:0/1623696166 conn(0x7f8144075670 msgr2=0x7f81440fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:36.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.054+0000 7f814b005700 1 -- 172.21.15.71:0/1623696166 shutdown_connections 2024-04-23T02:47:36.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.054+0000 7f814b005700 1 -- 172.21.15.71:0/1623696166 wait complete. 2024-04-23T02:47:36.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.156+0000 7f6826b3c700 1 -- 172.21.15.71:0/2987547370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68200ff5a0 msgr2=0x7f68200ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:36.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.156+0000 7f6826b3c700 1 --2- 172.21.15.71:0/2987547370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68200ff5a0 0x7f68200ff980 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f6814009910 tx=0x7f6814009c20 comp rx=0 tx=0).stop 2024-04-23T02:47:36.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.156+0000 7f6826b3c700 1 -- 172.21.15.71:0/2987547370 shutdown_connections 2024-04-23T02:47:36.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.156+0000 7f6826b3c700 1 --2- 172.21.15.71:0/2987547370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68200fff50 0x7f6820105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:36.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.156+0000 7f6826b3c700 1 --2- 172.21.15.71:0/2987547370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68200ff5a0 0x7f68200ff980 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:36.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.156+0000 7f6826b3c700 1 -- 172.21.15.71:0/2987547370 >> 172.21.15.71:0/2987547370 conn(0x7f6820074e30 msgr2=0x7f6820075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:36.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.156+0000 7f6826b3c700 1 -- 172.21.15.71:0/2987547370 shutdown_connections 2024-04-23T02:47:36.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.156+0000 7f6826b3c700 1 -- 172.21.15.71:0/2987547370 wait complete. 2024-04-23T02:47:36.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.157+0000 7f6826b3c700 1 Processor -- start 2024-04-23T02:47:36.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.157+0000 7f6826b3c700 1 -- start start 2024-04-23T02:47:36.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.157+0000 7f6826b3c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68200ff5a0 0x7f682019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:36.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.157+0000 7f6826b3c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68200fff50 0x7f682019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:36.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.157+0000 7f6826b3c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682019d170 con 0x7f68200ff5a0 2024-04-23T02:47:36.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.157+0000 7f6826b3c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6820196620 con 0x7f68200fff50 2024-04-23T02:47:36.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.157+0000 7f68248d8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68200ff5a0 0x7f682019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:36.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.157+0000 7f68248d8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68200ff5a0 0x7f682019c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46718/0 (socket says 172.21.15.71:46718) 2024-04-23T02:47:36.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.157+0000 7f68248d8700 1 -- 172.21.15.71:0/650931370 learned_addr learned my addr 172.21.15.71:0/650931370 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:47:36.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.157+0000 7f681ffff700 1 --2- 172.21.15.71:0/650931370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68200fff50 0x7f682019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:36.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.158+0000 7f68248d8700 1 -- 172.21.15.71:0/650931370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68200fff50 msgr2=0x7f682019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:36.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.158+0000 7f68248d8700 1 --2- 172.21.15.71:0/650931370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68200fff50 0x7f682019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:36.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.158+0000 7f68248d8700 1 -- 172.21.15.71:0/650931370 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68140095f0 con 0x7f68200ff5a0 2024-04-23T02:47:36.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.158+0000 7f68248d8700 1 --2- 172.21.15.71:0/650931370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68200ff5a0 0x7f682019c5a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6814000c00 tx=0x7f681400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:36.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.158+0000 7f681dffb700 1 -- 172.21.15.71:0/650931370 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f681401d070 con 0x7f68200ff5a0 2024-04-23T02:47:36.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.158+0000 7f681dffb700 1 -- 172.21.15.71:0/650931370 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6814005dc0 con 0x7f68200ff5a0 2024-04-23T02:47:36.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.158+0000 7f681dffb700 1 -- 172.21.15.71:0/650931370 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6814003940 con 0x7f68200ff5a0 2024-04-23T02:47:36.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.158+0000 7f6826b3c700 1 -- 172.21.15.71:0/650931370 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68201968a0 con 0x7f68200ff5a0 2024-04-23T02:47:36.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.158+0000 7f6826b3c700 1 -- 172.21.15.71:0/650931370 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6820196ce0 con 0x7f68200ff5a0 2024-04-23T02:47:36.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.159+0000 7f6826b3c700 1 -- 172.21.15.71:0/650931370 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6820109270 con 0x7f68200ff5a0 2024-04-23T02:47:36.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.160+0000 7f681dffb700 1 -- 172.21.15.71:0/650931370 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6814004ae0 con 0x7f68200ff5a0 2024-04-23T02:47:36.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.160+0000 7f681dffb700 1 --2- 172.21.15.71:0/650931370 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f681006cce0 0x7f681006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:47:36.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.160+0000 7f681dffb700 1 -- 172.21.15.71:0/650931370 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f681408db70 con 0x7f68200ff5a0 2024-04-23T02:47:36.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.160+0000 7f681ffff700 1 --2- 172.21.15.71:0/650931370 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f681006cce0 0x7f681006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:47:36.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.160+0000 7f681ffff700 1 --2- 172.21.15.71:0/650931370 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f681006cce0 0x7f681006f1a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6820107770 tx=0x7f680c009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:47:36.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.164+0000 7f681dffb700 1 -- 172.21.15.71:0/650931370 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f681405b670 con 0x7f68200ff5a0 2024-04-23T02:47:36.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.439+0000 7f6826b3c700 1 -- 172.21.15.71:0/650931370 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6820197c10 con 0x7f68200ff5a0 2024-04-23T02:47:36.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.439+0000 7f681dffb700 1 -- 172.21.15.71:0/650931370 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7f6814003b30 con 0x7f68200ff5a0 2024-04-23T02:47:36.442 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:47:36.442 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:47:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:47:36.442 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:47:36.442 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:47:36.442 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:47:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:47:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:47:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:47:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:47:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:47:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:47:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:47:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:47:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:47:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:47:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:47:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:47:36.443 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:47:36.443 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:47:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:47:36.444 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:47:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:47:36.444 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:47:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:47:36.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.442+0000 7f6826b3c700 1 -- 172.21.15.71:0/650931370 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f681006cce0 msgr2=0x7f681006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:36.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.442+0000 7f6826b3c700 1 --2- 172.21.15.71:0/650931370 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f681006cce0 0x7f681006f1a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6820107770 tx=0x7f680c009260 comp rx=0 tx=0).stop 2024-04-23T02:47:36.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.442+0000 7f6826b3c700 1 -- 172.21.15.71:0/650931370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68200ff5a0 msgr2=0x7f682019c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:47:36.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.442+0000 7f6826b3c700 1 --2- 172.21.15.71:0/650931370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68200ff5a0 0x7f682019c5a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6814000c00 tx=0x7f681400bde0 comp rx=0 tx=0).stop 2024-04-23T02:47:36.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.442+0000 7f6826b3c700 1 -- 172.21.15.71:0/650931370 shutdown_connections 2024-04-23T02:47:36.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.442+0000 7f6826b3c700 1 --2- 172.21.15.71:0/650931370 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f681006cce0 0x7f681006f1a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:36.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.442+0000 7f6826b3c700 1 --2- 172.21.15.71:0/650931370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68200ff5a0 0x7f682019c5a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:36.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.442+0000 7f6826b3c700 1 --2- 172.21.15.71:0/650931370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68200fff50 0x7f682019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:47:36.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.442+0000 7f6826b3c700 1 -- 172.21.15.71:0/650931370 >> 172.21.15.71:0/650931370 conn(0x7f6820074e30 msgr2=0x7f68200fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:47:36.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.442+0000 7f6826b3c700 1 -- 172.21.15.71:0/650931370 shutdown_connections 2024-04-23T02:47:36.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:47:36.442+0000 7f6826b3c700 1 -- 172.21.15.71:0/650931370 wait complete. 2024-04-23T02:47:36.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:36 smithi071 ceph-mon[144544]: from='client.35664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:47:36.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:36 smithi071 ceph-mon[144544]: from='client.35668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:47:36.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:36 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3297111325' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:47:36.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:36 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/650931370' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:47:37.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:36 smithi192 ceph-mon[108617]: from='client.35664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:47:37.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:36 smithi192 ceph-mon[108617]: from='client.35668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:47:37.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:36 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3297111325' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:47:37.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:36 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/650931370' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:47:37.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:37 smithi071 ceph-mon[144544]: from='client.35680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:47:37.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:37 smithi071 ceph-mon[144544]: pgmap v1030: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:38.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:37 smithi192 ceph-mon[108617]: from='client.35680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:47:38.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:37 smithi192 ceph-mon[108617]: pgmap v1030: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:38.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:38 smithi071 ceph-mon[144544]: pgmap v1031: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:39.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:38 smithi192 ceph-mon[108617]: pgmap v1031: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:41.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:41 smithi071 ceph-mon[144544]: pgmap v1032: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:41.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:41 smithi192 ceph-mon[108617]: pgmap v1032: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:42.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:42 smithi071 ceph-mon[144544]: pgmap v1033: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:43.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:42 smithi192 ceph-mon[108617]: pgmap v1033: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:45.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:45 smithi071 ceph-mon[144544]: pgmap v1034: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:46.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:45 smithi192 ceph-mon[108617]: pgmap v1034: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:46.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:46 smithi192 ceph-mon[108617]: pgmap v1035: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:46.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:46 smithi071 ceph-mon[144544]: pgmap v1035: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:48.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:48 smithi071 ceph-mon[144544]: pgmap v1036: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:49.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:48 smithi192 ceph-mon[108617]: pgmap v1036: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:50.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:47:50.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:47:51.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:50 smithi192 ceph-mon[108617]: pgmap v1037: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:51.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:50 smithi071 ceph-mon[144544]: pgmap v1037: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:52.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:52 smithi071 ceph-mon[144544]: pgmap v1038: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:53.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:52 smithi192 ceph-mon[108617]: pgmap v1038: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:54.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:54 smithi071 ceph-mon[144544]: pgmap v1039: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:55.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:54 smithi192 ceph-mon[108617]: pgmap v1039: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:56.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:56 smithi192 ceph-mon[108617]: pgmap v1040: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:56.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:56 smithi071 ceph-mon[144544]: pgmap v1040: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:58.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:47:58 smithi071 ceph-mon[144544]: pgmap v1041: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:47:59.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:47:58 smithi192 ceph-mon[108617]: pgmap v1041: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:00.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:00 smithi071 ceph-mon[144544]: pgmap v1042: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:01.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:00 smithi192 ceph-mon[108617]: pgmap v1042: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:02.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:02 smithi071 ceph-mon[144544]: pgmap v1043: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:03.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:02 smithi192 ceph-mon[108617]: pgmap v1043: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:04.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:04 smithi071 ceph-mon[144544]: pgmap v1044: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:05.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:04 smithi192 ceph-mon[108617]: pgmap v1044: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:05.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:48:06.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:48:06.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.559+0000 7f4702ba9700 1 -- 172.21.15.71:0/3377782103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46fc0ff340 msgr2=0x7f46fc0ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:06.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.559+0000 7f4702ba9700 1 --2- 172.21.15.71:0/3377782103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46fc0ff340 0x7f46fc0ff7c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009910 tx=0x7f46e8009c20 comp rx=0 tx=0).stop 2024-04-23T02:48:06.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.559+0000 7f4702ba9700 1 -- 172.21.15.71:0/3377782103 shutdown_connections 2024-04-23T02:48:06.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.559+0000 7f4702ba9700 1 --2- 172.21.15.71:0/3377782103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46fc0ff340 0x7f46fc0ff7c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:06.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.559+0000 7f4702ba9700 1 --2- 172.21.15.71:0/3377782103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46fc101bb0 0x7f46fc101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:06.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.559+0000 7f4702ba9700 1 -- 172.21.15.71:0/3377782103 >> 172.21.15.71:0/3377782103 conn(0x7f46fc074e30 msgr2=0x7f46fc075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:06.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.559+0000 7f4702ba9700 1 -- 172.21.15.71:0/3377782103 shutdown_connections 2024-04-23T02:48:06.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.559+0000 7f4702ba9700 1 -- 172.21.15.71:0/3377782103 wait complete. 2024-04-23T02:48:06.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.559+0000 7f4702ba9700 1 Processor -- start 2024-04-23T02:48:06.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.559+0000 7f4702ba9700 1 -- start start 2024-04-23T02:48:06.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.560+0000 7f4702ba9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46fc0ff340 0x7f46fc19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:06.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.560+0000 7f4702ba9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46fc101bb0 0x7f46fc19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:06.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.560+0000 7f4702ba9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc19f300 con 0x7f46fc0ff340 2024-04-23T02:48:06.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.560+0000 7f4702ba9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc1987c0 con 0x7f46fc101bb0 2024-04-23T02:48:06.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.560+0000 7f4700945700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46fc0ff340 0x7f46fc19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:06.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.560+0000 7f4700945700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46fc0ff340 0x7f46fc19e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35076/0 (socket says 172.21.15.71:35076) 2024-04-23T02:48:06.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.560+0000 7f4700945700 1 -- 172.21.15.71:0/4145230504 learned_addr learned my addr 172.21.15.71:0/4145230504 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:48:06.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.560+0000 7f46fbfff700 1 --2- 172.21.15.71:0/4145230504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46fc101bb0 0x7f46fc19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:06.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.560+0000 7f4700945700 1 -- 172.21.15.71:0/4145230504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46fc101bb0 msgr2=0x7f46fc19ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:06.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.560+0000 7f4700945700 1 --2- 172.21.15.71:0/4145230504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46fc101bb0 0x7f46fc19ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:06.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.560+0000 7f4700945700 1 -- 172.21.15.71:0/4145230504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e80095f0 con 0x7f46fc0ff340 2024-04-23T02:48:06.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.560+0000 7f4700945700 1 --2- 172.21.15.71:0/4145230504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46fc0ff340 0x7f46fc19e6e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f46f000d8d0 tx=0x7f46f000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:06.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.560+0000 7f46f9ffb700 1 -- 172.21.15.71:0/4145230504 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46f00098a0 con 0x7f46fc0ff340 2024-04-23T02:48:06.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.560+0000 7f46f9ffb700 1 -- 172.21.15.71:0/4145230504 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f46f0010460 con 0x7f46fc0ff340 2024-04-23T02:48:06.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.560+0000 7f4702ba9700 1 -- 172.21.15.71:0/4145230504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46fc198aa0 con 0x7f46fc0ff340 2024-04-23T02:48:06.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.560+0000 7f46f9ffb700 1 -- 172.21.15.71:0/4145230504 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46f0009ea0 con 0x7f46fc0ff340 2024-04-23T02:48:06.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.561+0000 7f4702ba9700 1 -- 172.21.15.71:0/4145230504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46fc198ff0 con 0x7f46fc0ff340 2024-04-23T02:48:06.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.562+0000 7f4702ba9700 1 -- 172.21.15.71:0/4145230504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46fc10b230 con 0x7f46fc0ff340 2024-04-23T02:48:06.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.562+0000 7f46f9ffb700 1 -- 172.21.15.71:0/4145230504 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f46f00105d0 con 0x7f46fc0ff340 2024-04-23T02:48:06.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.563+0000 7f46f9ffb700 1 --2- 172.21.15.71:0/4145230504 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f46ec06cce0 0x7f46ec06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:06.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.563+0000 7f46fbfff700 1 --2- 172.21.15.71:0/4145230504 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f46ec06cce0 0x7f46ec06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:06.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.563+0000 7f46f9ffb700 1 -- 172.21.15.71:0/4145230504 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f46f008c3a0 con 0x7f46fc0ff340 2024-04-23T02:48:06.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.563+0000 7f46fbfff700 1 --2- 172.21.15.71:0/4145230504 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f46ec06cce0 0x7f46ec06f1a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f46fc199fb0 tx=0x7f46e801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:06.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.567+0000 7f46f9ffb700 1 -- 172.21.15.71:0/4145230504 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f46f0059f20 con 0x7f46fc0ff340 2024-04-23T02:48:06.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.783+0000 7f4702ba9700 1 -- 172.21.15.71:0/4145230504 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f46fc199260 con 0x7f46ec06cce0 2024-04-23T02:48:06.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.785+0000 7f46f9ffb700 1 -- 172.21.15.71:0/4145230504 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f46fc199260 con 0x7f46ec06cce0 2024-04-23T02:48:06.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.787+0000 7f4702ba9700 1 -- 172.21.15.71:0/4145230504 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f46ec06cce0 msgr2=0x7f46ec06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:06.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.787+0000 7f4702ba9700 1 --2- 172.21.15.71:0/4145230504 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f46ec06cce0 0x7f46ec06f1a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f46fc199fb0 tx=0x7f46e801a040 comp rx=0 tx=0).stop 2024-04-23T02:48:06.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.787+0000 7f4702ba9700 1 -- 172.21.15.71:0/4145230504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46fc0ff340 msgr2=0x7f46fc19e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:06.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.787+0000 7f4702ba9700 1 --2- 172.21.15.71:0/4145230504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46fc0ff340 0x7f46fc19e6e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f46f000d8d0 tx=0x7f46f000dbe0 comp rx=0 tx=0).stop 2024-04-23T02:48:06.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.787+0000 7f4702ba9700 1 -- 172.21.15.71:0/4145230504 shutdown_connections 2024-04-23T02:48:06.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.787+0000 7f4702ba9700 1 --2- 172.21.15.71:0/4145230504 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f46ec06cce0 0x7f46ec06f1a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:06.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.787+0000 7f4702ba9700 1 --2- 172.21.15.71:0/4145230504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46fc0ff340 0x7f46fc19e6e0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:06.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.787+0000 7f4702ba9700 1 --2- 172.21.15.71:0/4145230504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46fc101bb0 0x7f46fc19ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:06.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.787+0000 7f4702ba9700 1 -- 172.21.15.71:0/4145230504 >> 172.21.15.71:0/4145230504 conn(0x7f46fc074e30 msgr2=0x7f46fc0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:06.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.787+0000 7f4702ba9700 1 -- 172.21.15.71:0/4145230504 shutdown_connections 2024-04-23T02:48:06.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.787+0000 7f4702ba9700 1 -- 172.21.15.71:0/4145230504 wait complete. 2024-04-23T02:48:06.800 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:48:06.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.896+0000 7efea4245700 1 -- 172.21.15.71:0/3407469392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe9c105bd0 msgr2=0x7efe9c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:06.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.896+0000 7efea4245700 1 --2- 172.21.15.71:0/3407469392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe9c105bd0 0x7efe9c105fb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009910 tx=0x7efe8c009c20 comp rx=0 tx=0).stop 2024-04-23T02:48:06.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.896+0000 7efea4245700 1 -- 172.21.15.71:0/3407469392 shutdown_connections 2024-04-23T02:48:06.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.896+0000 7efea4245700 1 --2- 172.21.15.71:0/3407469392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe9c0ffc20 0x7efe9c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:06.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.896+0000 7efea4245700 1 --2- 172.21.15.71:0/3407469392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe9c105bd0 0x7efe9c105fb0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:06.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.896+0000 7efea4245700 1 -- 172.21.15.71:0/3407469392 >> 172.21.15.71:0/3407469392 conn(0x7efe9c075670 msgr2=0x7efe9c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:06.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.896+0000 7efea4245700 1 -- 172.21.15.71:0/3407469392 shutdown_connections 2024-04-23T02:48:06.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.896+0000 7efea4245700 1 -- 172.21.15.71:0/3407469392 wait complete. 2024-04-23T02:48:06.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.897+0000 7efea4245700 1 Processor -- start 2024-04-23T02:48:06.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.897+0000 7efea4245700 1 -- start start 2024-04-23T02:48:06.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.897+0000 7efea4245700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe9c0ffc20 0x7efe9c19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:06.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.897+0000 7efea4245700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe9c105bd0 0x7efe9c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:06.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.897+0000 7efea4245700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c19d210 con 0x7efe9c0ffc20 2024-04-23T02:48:06.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.897+0000 7efea4245700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c1966c0 con 0x7efe9c105bd0 2024-04-23T02:48:06.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.897+0000 7efea1fe1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe9c0ffc20 0x7efe9c19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:06.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.897+0000 7efea1fe1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe9c0ffc20 0x7efe9c19c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35080/0 (socket says 172.21.15.71:35080) 2024-04-23T02:48:06.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.897+0000 7efea1fe1700 1 -- 172.21.15.71:0/3173538768 learned_addr learned my addr 172.21.15.71:0/3173538768 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:48:06.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.897+0000 7efea1fe1700 1 -- 172.21.15.71:0/3173538768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe9c105bd0 msgr2=0x7efe9c19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:06.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.897+0000 7efea1fe1700 1 --2- 172.21.15.71:0/3173538768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe9c105bd0 0x7efe9c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:06.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.897+0000 7efea1fe1700 1 -- 172.21.15.71:0/3173538768 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c0095f0 con 0x7efe9c0ffc20 2024-04-23T02:48:06.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.897+0000 7efea1fe1700 1 --2- 172.21.15.71:0/3173538768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe9c0ffc20 0x7efe9c19c640 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7efe8c006010 tx=0x7efe8c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:06.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.897+0000 7efe92ffd700 1 -- 172.21.15.71:0/3173538768 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe8c01d070 con 0x7efe9c0ffc20 2024-04-23T02:48:06.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.897+0000 7efea4245700 1 -- 172.21.15.71:0/3173538768 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9c196940 con 0x7efe9c0ffc20 2024-04-23T02:48:06.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.898+0000 7efe92ffd700 1 -- 172.21.15.71:0/3173538768 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efe8c00f460 con 0x7efe9c0ffc20 2024-04-23T02:48:06.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.898+0000 7efe92ffd700 1 -- 172.21.15.71:0/3173538768 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe8c021640 con 0x7efe9c0ffc20 2024-04-23T02:48:06.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.898+0000 7efea4245700 1 -- 172.21.15.71:0/3173538768 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9c196e30 con 0x7efe9c0ffc20 2024-04-23T02:48:06.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.899+0000 7efea4245700 1 -- 172.21.15.71:0/3173538768 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe9c1080f0 con 0x7efe9c0ffc20 2024-04-23T02:48:06.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.899+0000 7efe92ffd700 1 -- 172.21.15.71:0/3173538768 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7efe8c00f9e0 con 0x7efe9c0ffc20 2024-04-23T02:48:06.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.899+0000 7efe92ffd700 1 --2- 172.21.15.71:0/3173538768 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efe8806cce0 0x7efe8806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:06.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.900+0000 7efea17e0700 1 --2- 172.21.15.71:0/3173538768 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efe8806cce0 0x7efe8806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:06.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.900+0000 7efe92ffd700 1 -- 172.21.15.71:0/3173538768 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7efe8c08d900 con 0x7efe9c0ffc20 2024-04-23T02:48:06.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.900+0000 7efea17e0700 1 --2- 172.21.15.71:0/3173538768 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efe8806cce0 0x7efe8806f1a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efe9c197b30 tx=0x7efe98006d20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:06.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:06.904+0000 7efe92ffd700 1 -- 172.21.15.71:0/3173538768 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7efe8c05b400 con 0x7efe9c0ffc20 2024-04-23T02:48:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:06 smithi071 ceph-mon[144544]: pgmap v1045: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:07.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.118+0000 7efea4245700 1 -- 172.21.15.71:0/3173538768 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efe9c197050 con 0x7efe8806cce0 2024-04-23T02:48:07.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.119+0000 7efe92ffd700 1 -- 172.21.15.71:0/3173538768 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7efe9c197050 con 0x7efe8806cce0 2024-04-23T02:48:07.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.121+0000 7efea4245700 1 -- 172.21.15.71:0/3173538768 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efe8806cce0 msgr2=0x7efe8806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:07.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.121+0000 7efea4245700 1 --2- 172.21.15.71:0/3173538768 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efe8806cce0 0x7efe8806f1a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efe9c197b30 tx=0x7efe98006d20 comp rx=0 tx=0).stop 2024-04-23T02:48:07.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.121+0000 7efea4245700 1 -- 172.21.15.71:0/3173538768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe9c0ffc20 msgr2=0x7efe9c19c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:07.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.121+0000 7efea4245700 1 --2- 172.21.15.71:0/3173538768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe9c0ffc20 0x7efe9c19c640 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7efe8c006010 tx=0x7efe8c00be20 comp rx=0 tx=0).stop 2024-04-23T02:48:07.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.121+0000 7efea4245700 1 -- 172.21.15.71:0/3173538768 shutdown_connections 2024-04-23T02:48:07.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.121+0000 7efea4245700 1 --2- 172.21.15.71:0/3173538768 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efe8806cce0 0x7efe8806f1a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.121+0000 7efea4245700 1 --2- 172.21.15.71:0/3173538768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe9c0ffc20 0x7efe9c19c640 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.121+0000 7efea4245700 1 --2- 172.21.15.71:0/3173538768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe9c105bd0 0x7efe9c19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.121+0000 7efea4245700 1 -- 172.21.15.71:0/3173538768 >> 172.21.15.71:0/3173538768 conn(0x7efe9c075670 msgr2=0x7efe9c0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:07.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.121+0000 7efea4245700 1 -- 172.21.15.71:0/3173538768 shutdown_connections 2024-04-23T02:48:07.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.122+0000 7efea4245700 1 -- 172.21.15.71:0/3173538768 wait complete. 2024-04-23T02:48:07.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:06 smithi192 ceph-mon[108617]: pgmap v1045: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:07.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.224+0000 7fcc5dc18700 1 -- 172.21.15.71:0/3474026198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc58068580 msgr2=0x7fcc580689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:07.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.224+0000 7fcc5dc18700 1 --2- 172.21.15.71:0/3474026198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc58068580 0x7fcc580689e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009910 tx=0x7fcc40009c20 comp rx=0 tx=0).stop 2024-04-23T02:48:07.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.224+0000 7fcc5dc18700 1 -- 172.21.15.71:0/3474026198 shutdown_connections 2024-04-23T02:48:07.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.224+0000 7fcc5dc18700 1 --2- 172.21.15.71:0/3474026198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc58068580 0x7fcc580689e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.224+0000 7fcc5dc18700 1 --2- 172.21.15.71:0/3474026198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc58106cd0 0x7fcc581070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.224+0000 7fcc5dc18700 1 -- 172.21.15.71:0/3474026198 >> 172.21.15.71:0/3474026198 conn(0x7fcc58075670 msgr2=0x7fcc58075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:07.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.224+0000 7fcc5dc18700 1 -- 172.21.15.71:0/3474026198 shutdown_connections 2024-04-23T02:48:07.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.224+0000 7fcc5dc18700 1 -- 172.21.15.71:0/3474026198 wait complete. 2024-04-23T02:48:07.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.225+0000 7fcc5dc18700 1 Processor -- start 2024-04-23T02:48:07.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.225+0000 7fcc5dc18700 1 -- start start 2024-04-23T02:48:07.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.225+0000 7fcc5dc18700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc58068580 0x7fcc580ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:07.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.225+0000 7fcc5dc18700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc58106cd0 0x7fcc58100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:07.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.225+0000 7fcc5dc18700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc58100770 con 0x7fcc58106cd0 2024-04-23T02:48:07.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.225+0000 7fcc5dc18700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc581008b0 con 0x7fcc58068580 2024-04-23T02:48:07.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.225+0000 7fcc56ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc58106cd0 0x7fcc58100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:07.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.225+0000 7fcc56ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc58106cd0 0x7fcc58100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35094/0 (socket says 172.21.15.71:35094) 2024-04-23T02:48:07.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.225+0000 7fcc56ffd700 1 -- 172.21.15.71:0/3318626231 learned_addr learned my addr 172.21.15.71:0/3318626231 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:48:07.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.225+0000 7fcc577fe700 1 --2- 172.21.15.71:0/3318626231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc58068580 0x7fcc580ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:07.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.225+0000 7fcc56ffd700 1 -- 172.21.15.71:0/3318626231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc58068580 msgr2=0x7fcc580ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:07.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.225+0000 7fcc56ffd700 1 --2- 172.21.15.71:0/3318626231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc58068580 0x7fcc580ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.225+0000 7fcc56ffd700 1 -- 172.21.15.71:0/3318626231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc400095f0 con 0x7fcc58106cd0 2024-04-23T02:48:07.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.225+0000 7fcc577fe700 1 --2- 172.21.15.71:0/3318626231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc58068580 0x7fcc580ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:48:07.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.226+0000 7fcc56ffd700 1 --2- 172.21.15.71:0/3318626231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc58106cd0 0x7fcc58100230 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fcc40006010 tx=0x7fcc4000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:07.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.226+0000 7fcc54ff9700 1 -- 172.21.15.71:0/3318626231 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc4001d070 con 0x7fcc58106cd0 2024-04-23T02:48:07.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.226+0000 7fcc54ff9700 1 -- 172.21.15.71:0/3318626231 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc4000f460 con 0x7fcc58106cd0 2024-04-23T02:48:07.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.226+0000 7fcc5dc18700 1 -- 172.21.15.71:0/3318626231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc58100b30 con 0x7fcc58106cd0 2024-04-23T02:48:07.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.226+0000 7fcc54ff9700 1 -- 172.21.15.71:0/3318626231 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc40021620 con 0x7fcc58106cd0 2024-04-23T02:48:07.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.226+0000 7fcc5dc18700 1 -- 172.21.15.71:0/3318626231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc581a2c90 con 0x7fcc58106cd0 2024-04-23T02:48:07.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.227+0000 7fcc5dc18700 1 -- 172.21.15.71:0/3318626231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc581091d0 con 0x7fcc58106cd0 2024-04-23T02:48:07.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.227+0000 7fcc54ff9700 1 -- 172.21.15.71:0/3318626231 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fcc4000fa40 con 0x7fcc58106cd0 2024-04-23T02:48:07.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.227+0000 7fcc54ff9700 1 --2- 172.21.15.71:0/3318626231 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcc4406cce0 0x7fcc4406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:07.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.228+0000 7fcc54ff9700 1 -- 172.21.15.71:0/3318626231 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcc4008d6c0 con 0x7fcc58106cd0 2024-04-23T02:48:07.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.228+0000 7fcc577fe700 1 --2- 172.21.15.71:0/3318626231 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcc4406cce0 0x7fcc4406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:07.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.228+0000 7fcc577fe700 1 --2- 172.21.15.71:0/3318626231 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcc4406cce0 0x7fcc4406f1a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800bcf0 tx=0x7fcc4800b2c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:07.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.232+0000 7fcc54ff9700 1 -- 172.21.15.71:0/3318626231 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fcc400577b0 con 0x7fcc58106cd0 2024-04-23T02:48:07.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.432+0000 7fcc5dc18700 1 -- 172.21.15.71:0/3318626231 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc580611d0 con 0x7fcc4406cce0 2024-04-23T02:48:07.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.443+0000 7fcc54ff9700 1 -- 172.21.15.71:0/3318626231 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fcc580611d0 con 0x7fcc4406cce0 2024-04-23T02:48:07.446 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:48:07.446 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (31m) 10m ago 43m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:48:07.446 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (43m) 10m ago 43m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:48:07.446 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (42m) 10m ago 42m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:48:07.447 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (29m) 10m ago 43m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:48:07.447 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (29m) 10m ago 42m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:48:07.447 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (31m) 10m ago 42m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:48:07.447 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (25m) 10m ago 37m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:48:07.447 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (25m) 10m ago 37m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:48:07.447 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (10m) 10m ago 37m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:48:07.447 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (25m) 10m ago 37m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:48:07.447 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (33m) 10m ago 44m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:48:07.447 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (33m) 10m ago 42m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:48:07.447 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (30m) 10m ago 44m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:48:07.447 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (29m) 10m ago 41m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:48:07.447 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (32m) 10m ago 43m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:48:07.447 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (32m) 10m ago 42m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:48:07.447 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (29m) 10m ago 41m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:48:07.448 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (28m) 10m ago 40m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:48:07.448 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (28m) 10m ago 40m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:48:07.448 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (27m) 10m ago 39m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:48:07.448 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (26m) 10m ago 39m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:48:07.448 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (26m) 10m ago 38m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:48:07.448 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (32m) 10m ago 42m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:48:07.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.445+0000 7fcc5dc18700 1 -- 172.21.15.71:0/3318626231 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcc4406cce0 msgr2=0x7fcc4406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:07.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.445+0000 7fcc5dc18700 1 --2- 172.21.15.71:0/3318626231 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcc4406cce0 0x7fcc4406f1a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800bcf0 tx=0x7fcc4800b2c0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.445+0000 7fcc5dc18700 1 -- 172.21.15.71:0/3318626231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc58106cd0 msgr2=0x7fcc58100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:07.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.445+0000 7fcc5dc18700 1 --2- 172.21.15.71:0/3318626231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc58106cd0 0x7fcc58100230 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fcc40006010 tx=0x7fcc4000be20 comp rx=0 tx=0).stop 2024-04-23T02:48:07.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.445+0000 7fcc5dc18700 1 -- 172.21.15.71:0/3318626231 shutdown_connections 2024-04-23T02:48:07.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.445+0000 7fcc5dc18700 1 --2- 172.21.15.71:0/3318626231 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcc4406cce0 0x7fcc4406f1a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.445+0000 7fcc5dc18700 1 --2- 172.21.15.71:0/3318626231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc58068580 0x7fcc580ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.445+0000 7fcc5dc18700 1 --2- 172.21.15.71:0/3318626231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc58106cd0 0x7fcc58100230 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.445+0000 7fcc5dc18700 1 -- 172.21.15.71:0/3318626231 >> 172.21.15.71:0/3318626231 conn(0x7fcc58075670 msgr2=0x7fcc580fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:07.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.446+0000 7fcc5dc18700 1 -- 172.21.15.71:0/3318626231 shutdown_connections 2024-04-23T02:48:07.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.446+0000 7fcc5dc18700 1 -- 172.21.15.71:0/3318626231 wait complete. 2024-04-23T02:48:07.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.547+0000 7fa08db2a700 1 -- 172.21.15.71:0/2177821879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0880fff50 msgr2=0x7fa088105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:07.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.547+0000 7fa08db2a700 1 --2- 172.21.15.71:0/2177821879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0880fff50 0x7fa088105840 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa078009910 tx=0x7fa078009c20 comp rx=0 tx=0).stop 2024-04-23T02:48:07.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.547+0000 7fa08db2a700 1 -- 172.21.15.71:0/2177821879 shutdown_connections 2024-04-23T02:48:07.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.547+0000 7fa08db2a700 1 --2- 172.21.15.71:0/2177821879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0880fff50 0x7fa088105840 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.547+0000 7fa08db2a700 1 --2- 172.21.15.71:0/2177821879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0880ff5a0 0x7fa0880ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.547+0000 7fa08db2a700 1 -- 172.21.15.71:0/2177821879 >> 172.21.15.71:0/2177821879 conn(0x7fa088074e30 msgr2=0x7fa088075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:07.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.547+0000 7fa08db2a700 1 -- 172.21.15.71:0/2177821879 shutdown_connections 2024-04-23T02:48:07.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.547+0000 7fa08db2a700 1 -- 172.21.15.71:0/2177821879 wait complete. 2024-04-23T02:48:07.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.547+0000 7fa08db2a700 1 Processor -- start 2024-04-23T02:48:07.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.547+0000 7fa08db2a700 1 -- start start 2024-04-23T02:48:07.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.548+0000 7fa08db2a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0880ff5a0 0x7fa08819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:07.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.548+0000 7fa08db2a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0880fff50 0x7fa08819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:07.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.548+0000 7fa08db2a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08819d260 con 0x7fa0880fff50 2024-04-23T02:48:07.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.548+0000 7fa08db2a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa088196710 con 0x7fa0880ff5a0 2024-04-23T02:48:07.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.548+0000 7fa086ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0880fff50 0x7fa08819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:07.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.548+0000 7fa086ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0880fff50 0x7fa08819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35114/0 (socket says 172.21.15.71:35114) 2024-04-23T02:48:07.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.548+0000 7fa086ffd700 1 -- 172.21.15.71:0/2167099733 learned_addr learned my addr 172.21.15.71:0/2167099733 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:48:07.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.548+0000 7fa0877fe700 1 --2- 172.21.15.71:0/2167099733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0880ff5a0 0x7fa08819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:07.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.548+0000 7fa086ffd700 1 -- 172.21.15.71:0/2167099733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0880ff5a0 msgr2=0x7fa08819c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:07.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.548+0000 7fa086ffd700 1 --2- 172.21.15.71:0/2167099733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0880ff5a0 0x7fa08819c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.548+0000 7fa086ffd700 1 -- 172.21.15.71:0/2167099733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0780095f0 con 0x7fa0880fff50 2024-04-23T02:48:07.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.548+0000 7fa086ffd700 1 --2- 172.21.15.71:0/2167099733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0880fff50 0x7fa08819cbd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa0780092e0 tx=0x7fa07800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:07.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.548+0000 7fa084ff9700 1 -- 172.21.15.71:0/2167099733 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa07801d070 con 0x7fa0880fff50 2024-04-23T02:48:07.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.548+0000 7fa084ff9700 1 -- 172.21.15.71:0/2167099733 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa078005dc0 con 0x7fa0880fff50 2024-04-23T02:48:07.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.548+0000 7fa084ff9700 1 -- 172.21.15.71:0/2167099733 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa07800fea0 con 0x7fa0880fff50 2024-04-23T02:48:07.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.548+0000 7fa08db2a700 1 -- 172.21.15.71:0/2167099733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa088196990 con 0x7fa0880fff50 2024-04-23T02:48:07.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.548+0000 7fa08db2a700 1 -- 172.21.15.71:0/2167099733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa088196e80 con 0x7fa0880fff50 2024-04-23T02:48:07.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.549+0000 7fa08db2a700 1 -- 172.21.15.71:0/2167099733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa088109200 con 0x7fa0880fff50 2024-04-23T02:48:07.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.550+0000 7fa084ff9700 1 -- 172.21.15.71:0/2167099733 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa078003a60 con 0x7fa0880fff50 2024-04-23T02:48:07.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.550+0000 7fa084ff9700 1 --2- 172.21.15.71:0/2167099733 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa07006cce0 0x7fa07006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:07.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.550+0000 7fa084ff9700 1 -- 172.21.15.71:0/2167099733 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa07808d740 con 0x7fa0880fff50 2024-04-23T02:48:07.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.550+0000 7fa0877fe700 1 --2- 172.21.15.71:0/2167099733 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa07006cce0 0x7fa07006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:07.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.551+0000 7fa0877fe700 1 --2- 172.21.15.71:0/2167099733 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa07006cce0 0x7fa07006f1a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa07c005950 tx=0x7fa07c00b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:07.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.555+0000 7fa084ff9700 1 -- 172.21.15.71:0/2167099733 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa07805b240 con 0x7fa0880fff50 2024-04-23T02:48:07.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.830+0000 7fa08db2a700 1 -- 172.21.15.71:0/2167099733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa08804f2e0 con 0x7fa0880fff50 2024-04-23T02:48:07.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.831+0000 7fa084ff9700 1 -- 172.21.15.71:0/2167099733 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fa078003d10 con 0x7fa0880fff50 2024-04-23T02:48:07.833 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:48:07.834 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:48:07.834 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:48:07.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:48:07.834 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:48:07.834 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:48:07.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:48:07.834 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:48:07.834 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:48:07.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:48:07.834 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:48:07.834 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:48:07.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:48:07.834 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:48:07.835 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:48:07.835 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:48:07.835 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:48:07.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.833+0000 7fa08db2a700 1 -- 172.21.15.71:0/2167099733 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa07006cce0 msgr2=0x7fa07006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:07.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.833+0000 7fa08db2a700 1 --2- 172.21.15.71:0/2167099733 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa07006cce0 0x7fa07006f1a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa07c005950 tx=0x7fa07c00b410 comp rx=0 tx=0).stop 2024-04-23T02:48:07.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.833+0000 7fa08db2a700 1 -- 172.21.15.71:0/2167099733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0880fff50 msgr2=0x7fa08819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:07.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.833+0000 7fa08db2a700 1 --2- 172.21.15.71:0/2167099733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0880fff50 0x7fa08819cbd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa0780092e0 tx=0x7fa07800bd70 comp rx=0 tx=0).stop 2024-04-23T02:48:07.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.833+0000 7fa08db2a700 1 -- 172.21.15.71:0/2167099733 shutdown_connections 2024-04-23T02:48:07.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.833+0000 7fa08db2a700 1 --2- 172.21.15.71:0/2167099733 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa07006cce0 0x7fa07006f1a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.833+0000 7fa08db2a700 1 --2- 172.21.15.71:0/2167099733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0880ff5a0 0x7fa08819c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.833+0000 7fa08db2a700 1 --2- 172.21.15.71:0/2167099733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0880fff50 0x7fa08819cbd0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.833+0000 7fa08db2a700 1 -- 172.21.15.71:0/2167099733 >> 172.21.15.71:0/2167099733 conn(0x7fa088074e30 msgr2=0x7fa0880fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:07.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.833+0000 7fa08db2a700 1 -- 172.21.15.71:0/2167099733 shutdown_connections 2024-04-23T02:48:07.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.833+0000 7fa08db2a700 1 -- 172.21.15.71:0/2167099733 wait complete. 2024-04-23T02:48:07.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.936+0000 7f610812a700 1 -- 172.21.15.71:0/2874265000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6100101840 msgr2=0x7f6100105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:07.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.936+0000 7f610812a700 1 --2- 172.21.15.71:0/2874265000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6100101840 0x7f6100105890 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f60fc009910 tx=0x7f60fc009c20 comp rx=0 tx=0).stop 2024-04-23T02:48:07.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.936+0000 7f610812a700 1 -- 172.21.15.71:0/2874265000 shutdown_connections 2024-04-23T02:48:07.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.936+0000 7f610812a700 1 --2- 172.21.15.71:0/2874265000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6100101840 0x7f6100105890 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.936+0000 7f610812a700 1 --2- 172.21.15.71:0/2874265000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6100100e90 0x7f6100101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.936+0000 7f610812a700 1 -- 172.21.15.71:0/2874265000 >> 172.21.15.71:0/2874265000 conn(0x7f61000fc740 msgr2=0x7f61000feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:07.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.936+0000 7f610812a700 1 -- 172.21.15.71:0/2874265000 shutdown_connections 2024-04-23T02:48:07.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.936+0000 7f610812a700 1 -- 172.21.15.71:0/2874265000 wait complete. 2024-04-23T02:48:07.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.936+0000 7f610812a700 1 Processor -- start 2024-04-23T02:48:07.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.936+0000 7f610812a700 1 -- start start 2024-04-23T02:48:07.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.936+0000 7f610812a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6100100e90 0x7f610019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:07.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.936+0000 7f610812a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6100101840 0x7f610019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:07.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.937+0000 7f610812a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610019d240 con 0x7f6100100e90 2024-04-23T02:48:07.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.937+0000 7f610812a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61001966f0 con 0x7f6100101840 2024-04-23T02:48:07.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.937+0000 7f6105ec6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6100100e90 0x7f610019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:07.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.937+0000 7f6105ec6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6100100e90 0x7f610019c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35136/0 (socket says 172.21.15.71:35136) 2024-04-23T02:48:07.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.937+0000 7f6105ec6700 1 -- 172.21.15.71:0/2286699318 learned_addr learned my addr 172.21.15.71:0/2286699318 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:48:07.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.937+0000 7f61056c5700 1 --2- 172.21.15.71:0/2286699318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6100101840 0x7f610019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:07.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.937+0000 7f6105ec6700 1 -- 172.21.15.71:0/2286699318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6100101840 msgr2=0x7f610019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:07.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.937+0000 7f6105ec6700 1 --2- 172.21.15.71:0/2286699318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6100101840 0x7f610019cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:07.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.937+0000 7f6105ec6700 1 -- 172.21.15.71:0/2286699318 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60fc0095f0 con 0x7f6100100e90 2024-04-23T02:48:07.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.937+0000 7f6105ec6700 1 --2- 172.21.15.71:0/2286699318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6100100e90 0x7f610019c670 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f60f000b4b0 tx=0x7f60f000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:07.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.937+0000 7f60f6ffd700 1 -- 172.21.15.71:0/2286699318 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60f0011800 con 0x7f6100100e90 2024-04-23T02:48:07.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.937+0000 7f60f6ffd700 1 -- 172.21.15.71:0/2286699318 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f60f0011e00 con 0x7f6100100e90 2024-04-23T02:48:07.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.937+0000 7f60f6ffd700 1 -- 172.21.15.71:0/2286699318 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60f0010e50 con 0x7f6100100e90 2024-04-23T02:48:07.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.937+0000 7f610812a700 1 -- 172.21.15.71:0/2286699318 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61001969d0 con 0x7f6100100e90 2024-04-23T02:48:07.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.937+0000 7f610812a700 1 -- 172.21.15.71:0/2286699318 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6100196f20 con 0x7f6100100e90 2024-04-23T02:48:07.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.938+0000 7f610812a700 1 -- 172.21.15.71:0/2286699318 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61001091c0 con 0x7f6100100e90 2024-04-23T02:48:07.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.939+0000 7f60f6ffd700 1 -- 172.21.15.71:0/2286699318 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f60f0010380 con 0x7f6100100e90 2024-04-23T02:48:07.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.939+0000 7f60f6ffd700 1 --2- 172.21.15.71:0/2286699318 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f60ec06cce0 0x7f60ec06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:07.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.939+0000 7f60f6ffd700 1 -- 172.21.15.71:0/2286699318 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f60f008c130 con 0x7f6100100e90 2024-04-23T02:48:07.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.939+0000 7f61056c5700 1 --2- 172.21.15.71:0/2286699318 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f60ec06cce0 0x7f60ec06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:07.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.940+0000 7f61056c5700 1 --2- 172.21.15.71:0/2286699318 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f60ec06cce0 0x7f60ec06f1a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00b5c0 tx=0x7f60fc005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:07.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:07.944+0000 7f60f6ffd700 1 -- 172.21.15.71:0/2286699318 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f60f0059cb0 con 0x7f6100100e90 2024-04-23T02:48:07.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:07 smithi071 ceph-mon[144544]: from='client.35688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:48:07.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:07 smithi071 ceph-mon[144544]: from='client.35692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:48:07.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:07 smithi071 ceph-mon[144544]: from='client.35696 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:48:08.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:07 smithi192 ceph-mon[108617]: from='client.35688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:48:08.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:07 smithi192 ceph-mon[108617]: from='client.35692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:48:08.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:07 smithi192 ceph-mon[108617]: from='client.35696 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:48:08.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.178+0000 7f610812a700 1 -- 172.21.15.71:0/2286699318 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6100197d00 con 0x7f6100100e90 2024-04-23T02:48:08.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.178+0000 7f60f6ffd700 1 -- 172.21.15.71:0/2286699318 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f6100197d00 con 0x7f6100100e90 2024-04-23T02:48:08.181 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:48:08.181 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:48:08.181 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:48:08.181 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:48:08.181 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:48:08.181 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:48:08.181 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:48:08.182 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:48:08.182 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:48:08.182 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:48:08.182 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:48:08.182 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:48:08.182 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:48:08.182 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:48:08.182 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:48:08.182 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:48:08.182 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:48:08.182 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:48:08.182 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:48:08.182 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:48:08.182 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:48:08.182 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:48:08.183 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:48:08.183 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:48:08.183 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:48:08.183 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:48:08.183 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:48:08.183 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:48:08.183 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:48:08.183 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:48:08.183 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:48:08.183 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:48:08.183 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:48:08.183 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:48:08.183 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:48:08.183 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:48:08.183 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:48:08.184 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:48:08.184 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:48:08.184 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:48:08.184 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:48:08.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.180+0000 7f610812a700 1 -- 172.21.15.71:0/2286699318 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f60ec06cce0 msgr2=0x7f60ec06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:08.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.180+0000 7f610812a700 1 --2- 172.21.15.71:0/2286699318 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f60ec06cce0 0x7f60ec06f1a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00b5c0 tx=0x7f60fc005fd0 comp rx=0 tx=0).stop 2024-04-23T02:48:08.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.180+0000 7f610812a700 1 -- 172.21.15.71:0/2286699318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6100100e90 msgr2=0x7f610019c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:08.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.180+0000 7f610812a700 1 --2- 172.21.15.71:0/2286699318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6100100e90 0x7f610019c670 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f60f000b4b0 tx=0x7f60f000b870 comp rx=0 tx=0).stop 2024-04-23T02:48:08.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.180+0000 7f610812a700 1 -- 172.21.15.71:0/2286699318 shutdown_connections 2024-04-23T02:48:08.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.180+0000 7f610812a700 1 --2- 172.21.15.71:0/2286699318 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f60ec06cce0 0x7f60ec06f1a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:08.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.180+0000 7f610812a700 1 --2- 172.21.15.71:0/2286699318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6100100e90 0x7f610019c670 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:08.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.180+0000 7f610812a700 1 --2- 172.21.15.71:0/2286699318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6100101840 0x7f610019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:08.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.180+0000 7f610812a700 1 -- 172.21.15.71:0/2286699318 >> 172.21.15.71:0/2286699318 conn(0x7f61000fc740 msgr2=0x7f61000fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:08.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.180+0000 7f610812a700 1 -- 172.21.15.71:0/2286699318 shutdown_connections 2024-04-23T02:48:08.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.180+0000 7f610812a700 1 -- 172.21.15.71:0/2286699318 wait complete. 2024-04-23T02:48:08.185 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:48:08.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.283+0000 7f23aab80700 1 -- 172.21.15.71:0/1960486082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23a4101e00 msgr2=0x7f23a41021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:08.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.283+0000 7f23aab80700 1 --2- 172.21.15.71:0/1960486082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23a4101e00 0x7f23a41021e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f2398009910 tx=0x7f2398009c20 comp rx=0 tx=0).stop 2024-04-23T02:48:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.283+0000 7f23aab80700 1 -- 172.21.15.71:0/1960486082 shutdown_connections 2024-04-23T02:48:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.283+0000 7f23aab80700 1 --2- 172.21.15.71:0/1960486082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23a4102720 0x7f23a410ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.283+0000 7f23aab80700 1 --2- 172.21.15.71:0/1960486082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23a4101e00 0x7f23a41021e0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.283+0000 7f23aab80700 1 -- 172.21.15.71:0/1960486082 >> 172.21.15.71:0/1960486082 conn(0x7f23a40fb650 msgr2=0x7f23a40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.283+0000 7f23aab80700 1 -- 172.21.15.71:0/1960486082 shutdown_connections 2024-04-23T02:48:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.283+0000 7f23aab80700 1 -- 172.21.15.71:0/1960486082 wait complete. 2024-04-23T02:48:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.284+0000 7f23aab80700 1 Processor -- start 2024-04-23T02:48:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.284+0000 7f23aab80700 1 -- start start 2024-04-23T02:48:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.284+0000 7f23aab80700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23a4102720 0x7f23a419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.284+0000 7f23aab80700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23a419cd30 0x7f23a4196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.284+0000 7f23aab80700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a419d300 con 0x7f23a419cd30 2024-04-23T02:48:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.284+0000 7f23aab80700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a4196db0 con 0x7f23a4102720 2024-04-23T02:48:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.284+0000 7f23a3fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23a419cd30 0x7f23a4196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.284+0000 7f23a3fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23a419cd30 0x7f23a4196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35142/0 (socket says 172.21.15.71:35142) 2024-04-23T02:48:08.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.284+0000 7f23a3fff700 1 -- 172.21.15.71:0/2297837192 learned_addr learned my addr 172.21.15.71:0/2297837192 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:48:08.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.284+0000 7f23a891c700 1 --2- 172.21.15.71:0/2297837192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23a4102720 0x7f23a419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:08.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.284+0000 7f23a3fff700 1 -- 172.21.15.71:0/2297837192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23a4102720 msgr2=0x7f23a419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:08.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.284+0000 7f23a3fff700 1 --2- 172.21.15.71:0/2297837192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23a4102720 0x7f23a419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:08.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.284+0000 7f23a3fff700 1 -- 172.21.15.71:0/2297837192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23980095f0 con 0x7f23a419cd30 2024-04-23T02:48:08.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.284+0000 7f23a3fff700 1 --2- 172.21.15.71:0/2297837192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23a419cd30 0x7f23a4196870 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f239000d8d0 tx=0x7f239000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:08.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.285+0000 7f23a1ffb700 1 -- 172.21.15.71:0/2297837192 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23900041f0 con 0x7f23a419cd30 2024-04-23T02:48:08.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.285+0000 7f23aab80700 1 -- 172.21.15.71:0/2297837192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a4197090 con 0x7f23a419cd30 2024-04-23T02:48:08.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.285+0000 7f23a1ffb700 1 -- 172.21.15.71:0/2297837192 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2390004350 con 0x7f23a419cd30 2024-04-23T02:48:08.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.285+0000 7f23a1ffb700 1 -- 172.21.15.71:0/2297837192 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2390003c10 con 0x7f23a419cd30 2024-04-23T02:48:08.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.285+0000 7f23aab80700 1 -- 172.21.15.71:0/2297837192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a41975e0 con 0x7f23a419cd30 2024-04-23T02:48:08.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.286+0000 7f23aab80700 1 -- 172.21.15.71:0/2297837192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a4108310 con 0x7f23a419cd30 2024-04-23T02:48:08.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.286+0000 7f23a1ffb700 1 -- 172.21.15.71:0/2297837192 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2390003d70 con 0x7f23a419cd30 2024-04-23T02:48:08.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.286+0000 7f23a1ffb700 1 --2- 172.21.15.71:0/2297837192 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f239406cce0 0x7f239406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:08.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.286+0000 7f23a1ffb700 1 -- 172.21.15.71:0/2297837192 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f239008c170 con 0x7f23a419cd30 2024-04-23T02:48:08.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.287+0000 7f23a891c700 1 --2- 172.21.15.71:0/2297837192 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f239406cce0 0x7f239406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:08.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.287+0000 7f23a891c700 1 --2- 172.21.15.71:0/2297837192 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f239406cce0 0x7f239406f1a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2398009fd0 tx=0x7f239801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:08.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.291+0000 7f23a1ffb700 1 -- 172.21.15.71:0/2297837192 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2390059c70 con 0x7f23a419cd30 2024-04-23T02:48:08.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.503+0000 7f23aab80700 1 -- 172.21.15.71:0/2297837192 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f23a4197d80 con 0x7f239406cce0 2024-04-23T02:48:08.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.504+0000 7f23a1ffb700 1 -- 172.21.15.71:0/2297837192 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f23a4197d80 con 0x7f239406cce0 2024-04-23T02:48:08.507 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:48:08.507 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:48:08.507 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:48:08.507 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:48:08.507 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:48:08.507 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:48:08.507 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:48:08.507 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:48:08.507 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:48:08.507 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:48:08.507 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:48:08.507 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:48:08.508 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:48:08.508 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:48:08.508 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:48:08.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.506+0000 7f23aab80700 1 -- 172.21.15.71:0/2297837192 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f239406cce0 msgr2=0x7f239406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:08.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.506+0000 7f23aab80700 1 --2- 172.21.15.71:0/2297837192 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f239406cce0 0x7f239406f1a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2398009fd0 tx=0x7f239801a040 comp rx=0 tx=0).stop 2024-04-23T02:48:08.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.506+0000 7f23aab80700 1 -- 172.21.15.71:0/2297837192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23a419cd30 msgr2=0x7f23a4196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:08.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.506+0000 7f23aab80700 1 --2- 172.21.15.71:0/2297837192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23a419cd30 0x7f23a4196870 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f239000d8d0 tx=0x7f239000dbe0 comp rx=0 tx=0).stop 2024-04-23T02:48:08.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.506+0000 7f23aab80700 1 -- 172.21.15.71:0/2297837192 shutdown_connections 2024-04-23T02:48:08.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.506+0000 7f23aab80700 1 --2- 172.21.15.71:0/2297837192 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f239406cce0 0x7f239406f1a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:08.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.506+0000 7f23aab80700 1 --2- 172.21.15.71:0/2297837192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23a4102720 0x7f23a419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:08.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.506+0000 7f23aab80700 1 --2- 172.21.15.71:0/2297837192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23a419cd30 0x7f23a4196870 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:08.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.506+0000 7f23aab80700 1 -- 172.21.15.71:0/2297837192 >> 172.21.15.71:0/2297837192 conn(0x7f23a40fb650 msgr2=0x7f23a4105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:08.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.506+0000 7f23aab80700 1 -- 172.21.15.71:0/2297837192 shutdown_connections 2024-04-23T02:48:08.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.506+0000 7f23aab80700 1 -- 172.21.15.71:0/2297837192 wait complete. 2024-04-23T02:48:08.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.608+0000 7fe1f6615700 1 -- 172.21.15.71:0/3426980465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1f0105bd0 msgr2=0x7fe1f0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:08.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.608+0000 7fe1f6615700 1 --2- 172.21.15.71:0/3426980465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1f0105bd0 0x7fe1f0105fb0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009910 tx=0x7fe1e0009c20 comp rx=0 tx=0).stop 2024-04-23T02:48:08.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.608+0000 7fe1f6615700 1 -- 172.21.15.71:0/3426980465 shutdown_connections 2024-04-23T02:48:08.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.608+0000 7fe1f6615700 1 --2- 172.21.15.71:0/3426980465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1f00ffc20 0x7fe1f0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:08.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.608+0000 7fe1f6615700 1 --2- 172.21.15.71:0/3426980465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1f0105bd0 0x7fe1f0105fb0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:08.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.608+0000 7fe1f6615700 1 -- 172.21.15.71:0/3426980465 >> 172.21.15.71:0/3426980465 conn(0x7fe1f0075670 msgr2=0x7fe1f0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:08.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.608+0000 7fe1f6615700 1 -- 172.21.15.71:0/3426980465 shutdown_connections 2024-04-23T02:48:08.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.608+0000 7fe1f6615700 1 -- 172.21.15.71:0/3426980465 wait complete. 2024-04-23T02:48:08.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.609+0000 7fe1f6615700 1 Processor -- start 2024-04-23T02:48:08.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.609+0000 7fe1f6615700 1 -- start start 2024-04-23T02:48:08.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.609+0000 7fe1f6615700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1f00ffc20 0x7fe1f0195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:08.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.609+0000 7fe1f6615700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1f0105bd0 0x7fe1f0195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:08.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.609+0000 7fe1f6615700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f0196260 con 0x7fe1f0105bd0 2024-04-23T02:48:08.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.609+0000 7fe1f6615700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f0198f90 con 0x7fe1f00ffc20 2024-04-23T02:48:08.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.609+0000 7fe1ef7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1f0105bd0 0x7fe1f0195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:08.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.609+0000 7fe1ef7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1f0105bd0 0x7fe1f0195bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35166/0 (socket says 172.21.15.71:35166) 2024-04-23T02:48:08.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.609+0000 7fe1ef7fe700 1 -- 172.21.15.71:0/741195229 learned_addr learned my addr 172.21.15.71:0/741195229 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:48:08.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.609+0000 7fe1effff700 1 --2- 172.21.15.71:0/741195229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1f00ffc20 0x7fe1f0195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:08.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.610+0000 7fe1ef7fe700 1 -- 172.21.15.71:0/741195229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1f00ffc20 msgr2=0x7fe1f0195690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:08.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.610+0000 7fe1ef7fe700 1 --2- 172.21.15.71:0/741195229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1f00ffc20 0x7fe1f0195690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:08.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.610+0000 7fe1ef7fe700 1 -- 172.21.15.71:0/741195229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e00095f0 con 0x7fe1f0105bd0 2024-04-23T02:48:08.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.610+0000 7fe1ef7fe700 1 --2- 172.21.15.71:0/741195229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1f0105bd0 0x7fe1f0195bd0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009970 tx=0x7fe1d8009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:08.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.610+0000 7fe1ed7fa700 1 -- 172.21.15.71:0/741195229 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1d8012070 con 0x7fe1f0105bd0 2024-04-23T02:48:08.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.610+0000 7fe1ed7fa700 1 -- 172.21.15.71:0/741195229 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe1d8007ad0 con 0x7fe1f0105bd0 2024-04-23T02:48:08.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.610+0000 7fe1f6615700 1 -- 172.21.15.71:0/741195229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f0199210 con 0x7fe1f0105bd0 2024-04-23T02:48:08.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.610+0000 7fe1ed7fa700 1 -- 172.21.15.71:0/741195229 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1d8017bb0 con 0x7fe1f0105bd0 2024-04-23T02:48:08.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.610+0000 7fe1f6615700 1 -- 172.21.15.71:0/741195229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f0199760 con 0x7fe1f0105bd0 2024-04-23T02:48:08.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.611+0000 7fe1f6615700 1 -- 172.21.15.71:0/741195229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f004f2e0 con 0x7fe1f0105bd0 2024-04-23T02:48:08.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.611+0000 7fe1ed7fa700 1 -- 172.21.15.71:0/741195229 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe1d8007c40 con 0x7fe1f0105bd0 2024-04-23T02:48:08.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.612+0000 7fe1ed7fa700 1 --2- 172.21.15.71:0/741195229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe1dc06cce0 0x7fe1dc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:08.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.612+0000 7fe1ed7fa700 1 -- 172.21.15.71:0/741195229 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe1d808dae0 con 0x7fe1f0105bd0 2024-04-23T02:48:08.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.612+0000 7fe1effff700 1 --2- 172.21.15.71:0/741195229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe1dc06cce0 0x7fe1dc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:08.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.612+0000 7fe1effff700 1 --2- 172.21.15.71:0/741195229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe1dc06cce0 0x7fe1dc06f1a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0006010 tx=0x7fe1e001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:08.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.616+0000 7fe1ed7fa700 1 -- 172.21.15.71:0/741195229 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe1d8057c00 con 0x7fe1f0105bd0 2024-04-23T02:48:08.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.889+0000 7fe1f6615700 1 -- 172.21.15.71:0/741195229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe1f01999e0 con 0x7fe1f0105bd0 2024-04-23T02:48:08.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.890+0000 7fe1ed7fa700 1 -- 172.21.15.71:0/741195229 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7fe1d805b220 con 0x7fe1f0105bd0 2024-04-23T02:48:08.893 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:48:08.893 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:48:08.893 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:48:08.893 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:48:08.893 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:48:08.893 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:48:08.893 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:48:08.893 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:48:08.893 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:48:08.893 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:48:08.893 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:48:08.893 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:48:08.894 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:48:08.894 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:48:08.894 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:48:08.894 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:48:08.894 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:48:08.894 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:48:08.894 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:48:08.894 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:48:08.894 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:48:08.894 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:48:08.894 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:48:08.894 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:48:08.894 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:48:08.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.892+0000 7fe1f6615700 1 -- 172.21.15.71:0/741195229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe1dc06cce0 msgr2=0x7fe1dc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:08.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.892+0000 7fe1f6615700 1 --2- 172.21.15.71:0/741195229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe1dc06cce0 0x7fe1dc06f1a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0006010 tx=0x7fe1e001a040 comp rx=0 tx=0).stop 2024-04-23T02:48:08.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.892+0000 7fe1f6615700 1 -- 172.21.15.71:0/741195229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1f0105bd0 msgr2=0x7fe1f0195bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:08.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.892+0000 7fe1f6615700 1 --2- 172.21.15.71:0/741195229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1f0105bd0 0x7fe1f0195bd0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009970 tx=0x7fe1d8009c80 comp rx=0 tx=0).stop 2024-04-23T02:48:08.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.892+0000 7fe1f6615700 1 -- 172.21.15.71:0/741195229 shutdown_connections 2024-04-23T02:48:08.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.892+0000 7fe1f6615700 1 --2- 172.21.15.71:0/741195229 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe1dc06cce0 0x7fe1dc06f1a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:08.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.892+0000 7fe1f6615700 1 --2- 172.21.15.71:0/741195229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1f00ffc20 0x7fe1f0195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:08.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.892+0000 7fe1f6615700 1 --2- 172.21.15.71:0/741195229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1f0105bd0 0x7fe1f0195bd0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:08.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.892+0000 7fe1f6615700 1 -- 172.21.15.71:0/741195229 >> 172.21.15.71:0/741195229 conn(0x7fe1f0075670 msgr2=0x7fe1f00fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:08.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.892+0000 7fe1f6615700 1 -- 172.21.15.71:0/741195229 shutdown_connections 2024-04-23T02:48:08.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:08.892+0000 7fe1f6615700 1 -- 172.21.15.71:0/741195229 wait complete. 2024-04-23T02:48:08.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:08 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2167099733' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:48:08.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:08 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2286699318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:48:08.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:08 smithi071 ceph-mon[144544]: from='client.35708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:48:08.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:08 smithi071 ceph-mon[144544]: pgmap v1046: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:09.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:08 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2167099733' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:48:09.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:08 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2286699318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:48:09.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:08 smithi192 ceph-mon[108617]: from='client.35708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:48:09.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:08 smithi192 ceph-mon[108617]: pgmap v1046: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:09.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:09 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/741195229' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:48:10.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:09 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/741195229' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:48:10.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:10 smithi071 ceph-mon[144544]: pgmap v1047: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:11.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:10 smithi192 ceph-mon[108617]: pgmap v1047: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:12.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:12 smithi071 ceph-mon[144544]: pgmap v1048: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:13.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:12 smithi192 ceph-mon[108617]: pgmap v1048: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:14.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:14 smithi071 ceph-mon[144544]: pgmap v1049: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:15.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:14 smithi192 ceph-mon[108617]: pgmap v1049: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:16.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:16 smithi071 ceph-mon[144544]: pgmap v1050: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:17.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:16 smithi192 ceph-mon[108617]: pgmap v1050: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:18.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:18 smithi071 ceph-mon[144544]: pgmap v1051: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:19.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:18 smithi192 ceph-mon[108617]: pgmap v1051: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:20.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:48:20.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:48:21.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:20 smithi192 ceph-mon[108617]: pgmap v1052: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:21.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:20 smithi071 ceph-mon[144544]: pgmap v1052: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:22.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:22 smithi071 ceph-mon[144544]: pgmap v1053: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:23.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:22 smithi192 ceph-mon[108617]: pgmap v1053: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:24.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:24 smithi071 ceph-mon[144544]: pgmap v1054: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:25.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:24 smithi192 ceph-mon[108617]: pgmap v1054: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:26.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:26 smithi071 ceph-mon[144544]: pgmap v1055: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:27.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:26 smithi192 ceph-mon[108617]: pgmap v1055: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:28.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:28 smithi071 ceph-mon[144544]: pgmap v1056: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:29.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:28 smithi192 ceph-mon[108617]: pgmap v1056: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:30.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:30 smithi071 ceph-mon[144544]: pgmap v1057: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:31.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:30 smithi192 ceph-mon[108617]: pgmap v1057: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:32.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:32 smithi071 ceph-mon[144544]: pgmap v1058: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:33.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:32 smithi192 ceph-mon[108617]: pgmap v1058: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:34.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:34 smithi071 ceph-mon[144544]: pgmap v1059: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:35.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:34 smithi192 ceph-mon[108617]: pgmap v1059: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:48:36.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:48:36.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:36 smithi071 ceph-mon[144544]: pgmap v1060: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:37.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:36 smithi192 ceph-mon[108617]: pgmap v1060: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:38.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:38 smithi071 ceph-mon[144544]: pgmap v1061: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:39.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.008+0000 7fae1f30f700 1 -- 172.21.15.71:0/3279643331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae18100ea0 msgr2=0x7fae18101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:39.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.008+0000 7fae1f30f700 1 --2- 172.21.15.71:0/3279643331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae18100ea0 0x7fae18101280 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fae14009910 tx=0x7fae14009c20 comp rx=0 tx=0).stop 2024-04-23T02:48:39.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.008+0000 7fae1f30f700 1 -- 172.21.15.71:0/3279643331 shutdown_connections 2024-04-23T02:48:39.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.008+0000 7fae1f30f700 1 --2- 172.21.15.71:0/3279643331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae18101850 0x7fae181058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.008+0000 7fae1f30f700 1 --2- 172.21.15.71:0/3279643331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae18100ea0 0x7fae18101280 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.008+0000 7fae1f30f700 1 -- 172.21.15.71:0/3279643331 >> 172.21.15.71:0/3279643331 conn(0x7fae180fc750 msgr2=0x7fae180feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:39.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.009+0000 7fae1f30f700 1 -- 172.21.15.71:0/3279643331 shutdown_connections 2024-04-23T02:48:39.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.009+0000 7fae1f30f700 1 -- 172.21.15.71:0/3279643331 wait complete. 2024-04-23T02:48:39.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.009+0000 7fae1f30f700 1 Processor -- start 2024-04-23T02:48:39.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.009+0000 7fae1f30f700 1 -- start start 2024-04-23T02:48:39.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.009+0000 7fae1f30f700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae18100ea0 0x7fae18073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:39.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.009+0000 7fae1f30f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae18101850 0x7fae180735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:39.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.009+0000 7fae1f30f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae18073b10 con 0x7fae18101850 2024-04-23T02:48:39.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.009+0000 7fae1f30f700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae18073c50 con 0x7fae18100ea0 2024-04-23T02:48:39.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.010+0000 7fae1c8aa700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae18101850 0x7fae180735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:39.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.010+0000 7fae1c8aa700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae18101850 0x7fae180735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39938/0 (socket says 172.21.15.71:39938) 2024-04-23T02:48:39.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.010+0000 7fae1c8aa700 1 -- 172.21.15.71:0/1774770919 learned_addr learned my addr 172.21.15.71:0/1774770919 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:48:39.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.010+0000 7fae1d0ab700 1 --2- 172.21.15.71:0/1774770919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae18100ea0 0x7fae18073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:39.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.010+0000 7fae1c8aa700 1 -- 172.21.15.71:0/1774770919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae18100ea0 msgr2=0x7fae18073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:39.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.010+0000 7fae1c8aa700 1 --2- 172.21.15.71:0/1774770919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae18100ea0 0x7fae18073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.010+0000 7fae1c8aa700 1 -- 172.21.15.71:0/1774770919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae140095f0 con 0x7fae18101850 2024-04-23T02:48:39.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.010+0000 7fae1c8aa700 1 --2- 172.21.15.71:0/1774770919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae18101850 0x7fae180735d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fae0800d8d0 tx=0x7fae0800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:39.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.010+0000 7fae0e7fc700 1 -- 172.21.15.71:0/1774770919 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae080041f0 con 0x7fae18101850 2024-04-23T02:48:39.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.010+0000 7fae0e7fc700 1 -- 172.21.15.71:0/1774770919 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae08007620 con 0x7fae18101850 2024-04-23T02:48:39.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.010+0000 7fae1f30f700 1 -- 172.21.15.71:0/1774770919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae181a2960 con 0x7fae18101850 2024-04-23T02:48:39.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.010+0000 7fae0e7fc700 1 -- 172.21.15.71:0/1774770919 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae08005610 con 0x7fae18101850 2024-04-23T02:48:39.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.010+0000 7fae1f30f700 1 -- 172.21.15.71:0/1774770919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae181a2d70 con 0x7fae18101850 2024-04-23T02:48:39.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.011+0000 7fae1f30f700 1 -- 172.21.15.71:0/1774770919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae18109240 con 0x7fae18101850 2024-04-23T02:48:39.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.012+0000 7fae0e7fc700 1 -- 172.21.15.71:0/1774770919 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fae08004750 con 0x7fae18101850 2024-04-23T02:48:39.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.012+0000 7fae0e7fc700 1 --2- 172.21.15.71:0/1774770919 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fae0406cce0 0x7fae0406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:39.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.012+0000 7fae0e7fc700 1 -- 172.21.15.71:0/1774770919 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fae0808c430 con 0x7fae18101850 2024-04-23T02:48:39.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.012+0000 7fae1d0ab700 1 --2- 172.21.15.71:0/1774770919 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fae0406cce0 0x7fae0406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:39.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.013+0000 7fae1d0ab700 1 --2- 172.21.15.71:0/1774770919 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fae0406cce0 0x7fae0406f1a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fae14006010 tx=0x7fae1401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:39.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.017+0000 7fae0e7fc700 1 -- 172.21.15.71:0/1774770919 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fae080564a0 con 0x7fae18101850 2024-04-23T02:48:39.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:38 smithi192 ceph-mon[108617]: pgmap v1061: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:39.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.228+0000 7fae1f30f700 1 -- 172.21.15.71:0/1774770919 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fae180611d0 con 0x7fae0406cce0 2024-04-23T02:48:39.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.229+0000 7fae0e7fc700 1 -- 172.21.15.71:0/1774770919 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fae180611d0 con 0x7fae0406cce0 2024-04-23T02:48:39.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.231+0000 7fae1f30f700 1 -- 172.21.15.71:0/1774770919 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fae0406cce0 msgr2=0x7fae0406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:39.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.231+0000 7fae1f30f700 1 --2- 172.21.15.71:0/1774770919 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fae0406cce0 0x7fae0406f1a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fae14006010 tx=0x7fae1401a040 comp rx=0 tx=0).stop 2024-04-23T02:48:39.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.231+0000 7fae1f30f700 1 -- 172.21.15.71:0/1774770919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae18101850 msgr2=0x7fae180735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:39.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.231+0000 7fae1f30f700 1 --2- 172.21.15.71:0/1774770919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae18101850 0x7fae180735d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fae0800d8d0 tx=0x7fae0800dbe0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.231+0000 7fae1f30f700 1 -- 172.21.15.71:0/1774770919 shutdown_connections 2024-04-23T02:48:39.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.231+0000 7fae1f30f700 1 --2- 172.21.15.71:0/1774770919 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fae0406cce0 0x7fae0406f1a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.231+0000 7fae1f30f700 1 --2- 172.21.15.71:0/1774770919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae18100ea0 0x7fae18073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.231+0000 7fae1f30f700 1 --2- 172.21.15.71:0/1774770919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae18101850 0x7fae180735d0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.231+0000 7fae1f30f700 1 -- 172.21.15.71:0/1774770919 >> 172.21.15.71:0/1774770919 conn(0x7fae180fc750 msgr2=0x7fae180fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:39.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.231+0000 7fae1f30f700 1 -- 172.21.15.71:0/1774770919 shutdown_connections 2024-04-23T02:48:39.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.231+0000 7fae1f30f700 1 -- 172.21.15.71:0/1774770919 wait complete. 2024-04-23T02:48:39.244 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:48:39.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.334+0000 7ff0fafb8700 1 -- 172.21.15.71:0/782071422 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f4100ea0 msgr2=0x7ff0f4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:39.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.334+0000 7ff0fafb8700 1 --2- 172.21.15.71:0/782071422 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f4100ea0 0x7ff0f4101280 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009910 tx=0x7ff0e8009c20 comp rx=0 tx=0).stop 2024-04-23T02:48:39.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.334+0000 7ff0fafb8700 1 -- 172.21.15.71:0/782071422 shutdown_connections 2024-04-23T02:48:39.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.334+0000 7ff0fafb8700 1 --2- 172.21.15.71:0/782071422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0f4101850 0x7ff0f41058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.334+0000 7ff0fafb8700 1 --2- 172.21.15.71:0/782071422 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f4100ea0 0x7ff0f4101280 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.334+0000 7ff0fafb8700 1 -- 172.21.15.71:0/782071422 >> 172.21.15.71:0/782071422 conn(0x7ff0f40fc750 msgr2=0x7ff0f40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:39.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.335+0000 7ff0fafb8700 1 -- 172.21.15.71:0/782071422 shutdown_connections 2024-04-23T02:48:39.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.335+0000 7ff0fafb8700 1 -- 172.21.15.71:0/782071422 wait complete. 2024-04-23T02:48:39.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.335+0000 7ff0fafb8700 1 Processor -- start 2024-04-23T02:48:39.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.335+0000 7ff0fafb8700 1 -- start start 2024-04-23T02:48:39.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.336+0000 7ff0fafb8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0f4100ea0 0x7ff0f419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:39.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.336+0000 7ff0fafb8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f4101850 0x7ff0f419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:39.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.336+0000 7ff0fafb8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f419d0a0 con 0x7ff0f4101850 2024-04-23T02:48:39.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.336+0000 7ff0fafb8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f41965e0 con 0x7ff0f4100ea0 2024-04-23T02:48:39.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.336+0000 7ff0f3fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f4101850 0x7ff0f419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:39.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.336+0000 7ff0f3fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f4101850 0x7ff0f419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39954/0 (socket says 172.21.15.71:39954) 2024-04-23T02:48:39.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.336+0000 7ff0f3fff700 1 -- 172.21.15.71:0/475374039 learned_addr learned my addr 172.21.15.71:0/475374039 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:48:39.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.336+0000 7ff0f8d54700 1 --2- 172.21.15.71:0/475374039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0f4100ea0 0x7ff0f419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:39.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.336+0000 7ff0f3fff700 1 -- 172.21.15.71:0/475374039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0f4100ea0 msgr2=0x7ff0f419c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:39.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.336+0000 7ff0f3fff700 1 --2- 172.21.15.71:0/475374039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0f4100ea0 0x7ff0f419c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.336+0000 7ff0f3fff700 1 -- 172.21.15.71:0/475374039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e80095f0 con 0x7ff0f4101850 2024-04-23T02:48:39.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.336+0000 7ff0f3fff700 1 --2- 172.21.15.71:0/475374039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f4101850 0x7ff0f419caa0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000c960 tx=0x7ff0e000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:39.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.336+0000 7ff0f1ffb700 1 -- 172.21.15.71:0/475374039 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0e0007a70 con 0x7ff0f4101850 2024-04-23T02:48:39.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.336+0000 7ff0f1ffb700 1 -- 172.21.15.71:0/475374039 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff0e0007bd0 con 0x7ff0f4101850 2024-04-23T02:48:39.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.336+0000 7ff0fafb8700 1 -- 172.21.15.71:0/475374039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f41968c0 con 0x7ff0f4101850 2024-04-23T02:48:39.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.336+0000 7ff0f1ffb700 1 -- 172.21.15.71:0/475374039 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0e0018630 con 0x7ff0f4101850 2024-04-23T02:48:39.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.337+0000 7ff0fafb8700 1 -- 172.21.15.71:0/475374039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f4196e10 con 0x7ff0f4101850 2024-04-23T02:48:39.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.337+0000 7ff0fafb8700 1 -- 172.21.15.71:0/475374039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f4109240 con 0x7ff0f4101850 2024-04-23T02:48:39.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.338+0000 7ff0f1ffb700 1 -- 172.21.15.71:0/475374039 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff0e000f450 con 0x7ff0f4101850 2024-04-23T02:48:39.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.338+0000 7ff0f1ffb700 1 --2- 172.21.15.71:0/475374039 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff0e406cce0 0x7ff0e406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:39.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.338+0000 7ff0f1ffb700 1 -- 172.21.15.71:0/475374039 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff0e008c4c0 con 0x7ff0f4101850 2024-04-23T02:48:39.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.338+0000 7ff0f8d54700 1 --2- 172.21.15.71:0/475374039 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff0e406cce0 0x7ff0e406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:39.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.339+0000 7ff0f8d54700 1 --2- 172.21.15.71:0/475374039 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff0e406cce0 0x7ff0e406f1a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8006010 tx=0x7ff0e801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:39.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.343+0000 7ff0f1ffb700 1 -- 172.21.15.71:0/475374039 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff0e005a040 con 0x7ff0f4101850 2024-04-23T02:48:39.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.551+0000 7ff0fafb8700 1 -- 172.21.15.71:0/475374039 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0f4197be0 con 0x7ff0e406cce0 2024-04-23T02:48:39.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.553+0000 7ff0f1ffb700 1 -- 172.21.15.71:0/475374039 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff0f4197be0 con 0x7ff0e406cce0 2024-04-23T02:48:39.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.555+0000 7ff0fafb8700 1 -- 172.21.15.71:0/475374039 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff0e406cce0 msgr2=0x7ff0e406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:39.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.555+0000 7ff0fafb8700 1 --2- 172.21.15.71:0/475374039 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff0e406cce0 0x7ff0e406f1a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8006010 tx=0x7ff0e801a040 comp rx=0 tx=0).stop 2024-04-23T02:48:39.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.555+0000 7ff0fafb8700 1 -- 172.21.15.71:0/475374039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f4101850 msgr2=0x7ff0f419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:39.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.555+0000 7ff0fafb8700 1 --2- 172.21.15.71:0/475374039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f4101850 0x7ff0f419caa0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000c960 tx=0x7ff0e000cc70 comp rx=0 tx=0).stop 2024-04-23T02:48:39.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.555+0000 7ff0fafb8700 1 -- 172.21.15.71:0/475374039 shutdown_connections 2024-04-23T02:48:39.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.555+0000 7ff0fafb8700 1 --2- 172.21.15.71:0/475374039 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff0e406cce0 0x7ff0e406f1a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.555+0000 7ff0fafb8700 1 --2- 172.21.15.71:0/475374039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0f4100ea0 0x7ff0f419c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.555+0000 7ff0fafb8700 1 --2- 172.21.15.71:0/475374039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f4101850 0x7ff0f419caa0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.555+0000 7ff0fafb8700 1 -- 172.21.15.71:0/475374039 >> 172.21.15.71:0/475374039 conn(0x7ff0f40fc750 msgr2=0x7ff0f40fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:39.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.555+0000 7ff0fafb8700 1 -- 172.21.15.71:0/475374039 shutdown_connections 2024-04-23T02:48:39.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.555+0000 7ff0fafb8700 1 -- 172.21.15.71:0/475374039 wait complete. 2024-04-23T02:48:39.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.657+0000 7f500fbcd700 1 -- 172.21.15.71:0/2366273038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5008105bd0 msgr2=0x7f5008105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:39.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.657+0000 7f500fbcd700 1 --2- 172.21.15.71:0/2366273038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5008105bd0 0x7f5008105fb0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc009910 tx=0x7f4ffc009c20 comp rx=0 tx=0).stop 2024-04-23T02:48:39.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.657+0000 7f500fbcd700 1 -- 172.21.15.71:0/2366273038 shutdown_connections 2024-04-23T02:48:39.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.657+0000 7f500fbcd700 1 --2- 172.21.15.71:0/2366273038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50080ffc20 0x7f5008100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.657+0000 7f500fbcd700 1 --2- 172.21.15.71:0/2366273038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5008105bd0 0x7f5008105fb0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.657+0000 7f500fbcd700 1 -- 172.21.15.71:0/2366273038 >> 172.21.15.71:0/2366273038 conn(0x7f5008075670 msgr2=0x7f5008075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:39.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.657+0000 7f500fbcd700 1 -- 172.21.15.71:0/2366273038 shutdown_connections 2024-04-23T02:48:39.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.657+0000 7f500fbcd700 1 -- 172.21.15.71:0/2366273038 wait complete. 2024-04-23T02:48:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.658+0000 7f500fbcd700 1 Processor -- start 2024-04-23T02:48:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.658+0000 7f500fbcd700 1 -- start start 2024-04-23T02:48:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.658+0000 7f500fbcd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50080ffc20 0x7f5008197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.658+0000 7f500fbcd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5008105bd0 0x7f5008197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.658+0000 7f500fbcd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50081980d0 con 0x7f50080ffc20 2024-04-23T02:48:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.658+0000 7f500fbcd700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5008198210 con 0x7f5008105bd0 2024-04-23T02:48:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.658+0000 7f500d969700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50080ffc20 0x7f5008197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.658+0000 7f500d969700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50080ffc20 0x7f5008197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39972/0 (socket says 172.21.15.71:39972) 2024-04-23T02:48:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.658+0000 7f500d969700 1 -- 172.21.15.71:0/1601546438 learned_addr learned my addr 172.21.15.71:0/1601546438 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:48:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.659+0000 7f500d969700 1 -- 172.21.15.71:0/1601546438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5008105bd0 msgr2=0x7f5008197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.659+0000 7f500d969700 1 --2- 172.21.15.71:0/1601546438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5008105bd0 0x7f5008197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.659+0000 7f500d969700 1 -- 172.21.15.71:0/1601546438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc0095f0 con 0x7f50080ffc20 2024-04-23T02:48:39.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.659+0000 7f500d969700 1 --2- 172.21.15.71:0/1601546438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50080ffc20 0x7f5008197600 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc006010 tx=0x7f4ffc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:39.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.659+0000 7f4ffaffd700 1 -- 172.21.15.71:0/1601546438 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ffc01d070 con 0x7f50080ffc20 2024-04-23T02:48:39.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.659+0000 7f4ffaffd700 1 -- 172.21.15.71:0/1601546438 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4ffc00f460 con 0x7f50080ffc20 2024-04-23T02:48:39.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.659+0000 7f500fbcd700 1 -- 172.21.15.71:0/1601546438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50081a0670 con 0x7f50080ffc20 2024-04-23T02:48:39.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.659+0000 7f4ffaffd700 1 -- 172.21.15.71:0/1601546438 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ffc021640 con 0x7f50080ffc20 2024-04-23T02:48:39.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.659+0000 7f500fbcd700 1 -- 172.21.15.71:0/1601546438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50081a0a20 con 0x7f50080ffc20 2024-04-23T02:48:39.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.660+0000 7f500fbcd700 1 -- 172.21.15.71:0/1601546438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50081080f0 con 0x7f50080ffc20 2024-04-23T02:48:39.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.660+0000 7f4ffaffd700 1 -- 172.21.15.71:0/1601546438 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4ffc00fa20 con 0x7f50080ffc20 2024-04-23T02:48:39.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.661+0000 7f4ffaffd700 1 --2- 172.21.15.71:0/1601546438 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4ff406cce0 0x7f4ff406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:39.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.661+0000 7f4ffaffd700 1 -- 172.21.15.71:0/1601546438 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4ffc08d8c0 con 0x7f50080ffc20 2024-04-23T02:48:39.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.661+0000 7f500d168700 1 --2- 172.21.15.71:0/1601546438 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4ff406cce0 0x7f4ff406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:39.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.661+0000 7f500d168700 1 --2- 172.21.15.71:0/1601546438 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4ff406cce0 0x7f4ff406f1a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5004005fd0 tx=0x7f5004005d00 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:39.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.665+0000 7f4ffaffd700 1 -- 172.21.15.71:0/1601546438 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f4ffc057930 con 0x7f50080ffc20 2024-04-23T02:48:39.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.866+0000 7f500fbcd700 1 -- 172.21.15.71:0/1601546438 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f50080611d0 con 0x7f4ff406cce0 2024-04-23T02:48:39.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.877+0000 7f4ffaffd700 1 -- 172.21.15.71:0/1601546438 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f50080611d0 con 0x7f4ff406cce0 2024-04-23T02:48:39.880 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:48:39.880 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (32m) 10m ago 43m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:48:39.880 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (43m) 10m ago 43m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:48:39.880 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (42m) 10m ago 42m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:48:39.880 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (30m) 10m ago 43m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:48:39.880 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (30m) 10m ago 42m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:48:39.880 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (32m) 10m ago 43m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:48:39.881 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (26m) 10m ago 37m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:48:39.881 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (26m) 10m ago 37m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:48:39.881 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (10m) 10m ago 37m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:48:39.881 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (25m) 10m ago 37m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:48:39.881 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (34m) 10m ago 45m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:48:39.881 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (33m) 10m ago 42m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:48:39.881 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (30m) 10m ago 45m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:48:39.881 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (30m) 10m ago 42m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:48:39.881 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (33m) 10m ago 43m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:48:39.881 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (33m) 10m ago 42m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:48:39.881 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (29m) 10m ago 41m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:48:39.881 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (29m) 10m ago 41m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:48:39.881 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (28m) 10m ago 40m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:48:39.881 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (27m) 10m ago 40m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:48:39.881 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (27m) 10m ago 39m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:48:39.882 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (26m) 10m ago 39m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:48:39.882 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (32m) 10m ago 43m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:48:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.879+0000 7f500fbcd700 1 -- 172.21.15.71:0/1601546438 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4ff406cce0 msgr2=0x7f4ff406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.879+0000 7f500fbcd700 1 --2- 172.21.15.71:0/1601546438 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4ff406cce0 0x7f4ff406f1a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5004005fd0 tx=0x7f5004005d00 comp rx=0 tx=0).stop 2024-04-23T02:48:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.879+0000 7f500fbcd700 1 -- 172.21.15.71:0/1601546438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50080ffc20 msgr2=0x7f5008197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.879+0000 7f500fbcd700 1 --2- 172.21.15.71:0/1601546438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50080ffc20 0x7f5008197600 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc006010 tx=0x7f4ffc00be20 comp rx=0 tx=0).stop 2024-04-23T02:48:39.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.879+0000 7f500fbcd700 1 -- 172.21.15.71:0/1601546438 shutdown_connections 2024-04-23T02:48:39.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.879+0000 7f500fbcd700 1 --2- 172.21.15.71:0/1601546438 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4ff406cce0 0x7f4ff406f1a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.879+0000 7f500fbcd700 1 --2- 172.21.15.71:0/1601546438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50080ffc20 0x7f5008197600 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.879+0000 7f500fbcd700 1 --2- 172.21.15.71:0/1601546438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5008105bd0 0x7f5008197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.879+0000 7f500fbcd700 1 -- 172.21.15.71:0/1601546438 >> 172.21.15.71:0/1601546438 conn(0x7f5008075670 msgr2=0x7f50080fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:39.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.879+0000 7f500fbcd700 1 -- 172.21.15.71:0/1601546438 shutdown_connections 2024-04-23T02:48:39.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.879+0000 7f500fbcd700 1 -- 172.21.15.71:0/1601546438 wait complete. 2024-04-23T02:48:39.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.981+0000 7f715eaed700 1 -- 172.21.15.71:0/2513174340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7158068580 msgr2=0x7f7158068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:39.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.981+0000 7f715eaed700 1 --2- 172.21.15.71:0/2513174340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7158068580 0x7f7158068960 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f714c009910 tx=0x7f714c009c20 comp rx=0 tx=0).stop 2024-04-23T02:48:39.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.981+0000 7f715eaed700 1 -- 172.21.15.71:0/2513174340 shutdown_connections 2024-04-23T02:48:39.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.981+0000 7f715eaed700 1 --2- 172.21.15.71:0/2513174340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7158068ea0 0x7f715810d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.981+0000 7f715eaed700 1 --2- 172.21.15.71:0/2513174340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7158068580 0x7f7158068960 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.981+0000 7f715eaed700 1 -- 172.21.15.71:0/2513174340 >> 172.21.15.71:0/2513174340 conn(0x7f7158075670 msgr2=0x7f7158075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:39.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.981+0000 7f715eaed700 1 -- 172.21.15.71:0/2513174340 shutdown_connections 2024-04-23T02:48:39.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.981+0000 7f715eaed700 1 -- 172.21.15.71:0/2513174340 wait complete. 2024-04-23T02:48:39.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.981+0000 7f715eaed700 1 Processor -- start 2024-04-23T02:48:39.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.981+0000 7f715eaed700 1 -- start start 2024-04-23T02:48:39.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.982+0000 7f715eaed700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7158068580 0x7f7158198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:39.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.982+0000 7f715eaed700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7158068ea0 0x7f7158198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:39.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.982+0000 7f715eaed700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7158199550 con 0x7f7158068580 2024-04-23T02:48:39.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.982+0000 7f715eaed700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715819d360 con 0x7f7158068ea0 2024-04-23T02:48:39.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.982+0000 7f715c889700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7158068580 0x7f7158198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:39.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.982+0000 7f715c889700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7158068580 0x7f7158198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39992/0 (socket says 172.21.15.71:39992) 2024-04-23T02:48:39.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.982+0000 7f715c889700 1 -- 172.21.15.71:0/803927802 learned_addr learned my addr 172.21.15.71:0/803927802 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:48:39.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.982+0000 7f7157fff700 1 --2- 172.21.15.71:0/803927802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7158068ea0 0x7f7158198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:39.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.982+0000 7f715c889700 1 -- 172.21.15.71:0/803927802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7158068ea0 msgr2=0x7f7158198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:39.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.982+0000 7f715c889700 1 --2- 172.21.15.71:0/803927802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7158068ea0 0x7f7158198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:39.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.982+0000 7f715c889700 1 -- 172.21.15.71:0/803927802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714c0095f0 con 0x7f7158068580 2024-04-23T02:48:39.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.982+0000 7f715c889700 1 --2- 172.21.15.71:0/803927802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7158068580 0x7f7158198980 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f714c006010 tx=0x7f714c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:39.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.982+0000 7f7155ffb700 1 -- 172.21.15.71:0/803927802 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f714c01d070 con 0x7f7158068580 2024-04-23T02:48:39.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.982+0000 7f7155ffb700 1 -- 172.21.15.71:0/803927802 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f714c00f460 con 0x7f7158068580 2024-04-23T02:48:39.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.982+0000 7f715eaed700 1 -- 172.21.15.71:0/803927802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715819d5e0 con 0x7f7158068580 2024-04-23T02:48:39.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.982+0000 7f7155ffb700 1 -- 172.21.15.71:0/803927802 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f714c021640 con 0x7f7158068580 2024-04-23T02:48:39.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.982+0000 7f715eaed700 1 -- 172.21.15.71:0/803927802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715819dad0 con 0x7f7158068580 2024-04-23T02:48:39.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.984+0000 7f715eaed700 1 -- 172.21.15.71:0/803927802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715810aa00 con 0x7f7158068580 2024-04-23T02:48:39.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.984+0000 7f7155ffb700 1 -- 172.21.15.71:0/803927802 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f714c003a60 con 0x7f7158068580 2024-04-23T02:48:39.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.984+0000 7f7155ffb700 1 --2- 172.21.15.71:0/803927802 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f714806cce0 0x7f714806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:39.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.984+0000 7f7155ffb700 1 -- 172.21.15.71:0/803927802 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f714c08d8a0 con 0x7f7158068580 2024-04-23T02:48:39.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.984+0000 7f7157fff700 1 --2- 172.21.15.71:0/803927802 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f714806cce0 0x7f714806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:39.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.985+0000 7f7157fff700 1 --2- 172.21.15.71:0/803927802 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f714806cce0 0x7f714806f1a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7144005fd0 tx=0x7f714400a560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:39.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:39.989+0000 7f7155ffb700 1 -- 172.21.15.71:0/803927802 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f714c05b3a0 con 0x7f7158068580 2024-04-23T02:48:39.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:39 smithi071 ceph-mon[144544]: from='client.35716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:48:39.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:39 smithi071 ceph-mon[144544]: from='client.35720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:48:40.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:39 smithi192 ceph-mon[108617]: from='client.35716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:48:40.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:39 smithi192 ceph-mon[108617]: from='client.35720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:48:40.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.261+0000 7f715eaed700 1 -- 172.21.15.71:0/803927802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f715804f2e0 con 0x7f7158068580 2024-04-23T02:48:40.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.262+0000 7f7155ffb700 1 -- 172.21.15.71:0/803927802 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f714c05af30 con 0x7f7158068580 2024-04-23T02:48:40.265 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:48:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:48:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:48:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:48:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:48:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:48:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:48:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:48:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:48:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:48:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:48:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:48:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:48:40.265 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:48:40.266 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:48:40.266 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:48:40.266 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:48:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.264+0000 7f715eaed700 1 -- 172.21.15.71:0/803927802 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f714806cce0 msgr2=0x7f714806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.264+0000 7f715eaed700 1 --2- 172.21.15.71:0/803927802 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f714806cce0 0x7f714806f1a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7144005fd0 tx=0x7f714400a560 comp rx=0 tx=0).stop 2024-04-23T02:48:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.264+0000 7f715eaed700 1 -- 172.21.15.71:0/803927802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7158068580 msgr2=0x7f7158198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.264+0000 7f715eaed700 1 --2- 172.21.15.71:0/803927802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7158068580 0x7f7158198980 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f714c006010 tx=0x7f714c00be20 comp rx=0 tx=0).stop 2024-04-23T02:48:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.264+0000 7f715eaed700 1 -- 172.21.15.71:0/803927802 shutdown_connections 2024-04-23T02:48:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.264+0000 7f715eaed700 1 --2- 172.21.15.71:0/803927802 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f714806cce0 0x7f714806f1a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.264+0000 7f715eaed700 1 --2- 172.21.15.71:0/803927802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7158068580 0x7f7158198980 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.264+0000 7f715eaed700 1 --2- 172.21.15.71:0/803927802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7158068ea0 0x7f7158198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.264+0000 7f715eaed700 1 -- 172.21.15.71:0/803927802 >> 172.21.15.71:0/803927802 conn(0x7f7158075670 msgr2=0x7f71580fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.264+0000 7f715eaed700 1 -- 172.21.15.71:0/803927802 shutdown_connections 2024-04-23T02:48:40.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.264+0000 7f715eaed700 1 -- 172.21.15.71:0/803927802 wait complete. 2024-04-23T02:48:40.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.366+0000 7f961bf5e700 1 -- 172.21.15.71:0/1713962928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9614106bf0 msgr2=0x7f9614106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:40.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.366+0000 7f961bf5e700 1 --2- 172.21.15.71:0/1713962928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9614106bf0 0x7f9614106fd0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9610009910 tx=0x7f9610009c20 comp rx=0 tx=0).stop 2024-04-23T02:48:40.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.366+0000 7f961bf5e700 1 -- 172.21.15.71:0/1713962928 shutdown_connections 2024-04-23T02:48:40.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.366+0000 7f961bf5e700 1 --2- 172.21.15.71:0/1713962928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9614100b40 0x7f9614100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:40.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.366+0000 7f961bf5e700 1 --2- 172.21.15.71:0/1713962928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9614106bf0 0x7f9614106fd0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:40.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.366+0000 7f961bf5e700 1 -- 172.21.15.71:0/1713962928 >> 172.21.15.71:0/1713962928 conn(0x7f96140fc610 msgr2=0x7f96140fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:40.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.366+0000 7f961bf5e700 1 -- 172.21.15.71:0/1713962928 shutdown_connections 2024-04-23T02:48:40.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.366+0000 7f961bf5e700 1 -- 172.21.15.71:0/1713962928 wait complete. 2024-04-23T02:48:40.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.367+0000 7f961bf5e700 1 Processor -- start 2024-04-23T02:48:40.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.367+0000 7f961bf5e700 1 -- start start 2024-04-23T02:48:40.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.367+0000 7f961bf5e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9614100b40 0x7f961419c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:40.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.367+0000 7f961bf5e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9614106bf0 0x7f961419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:40.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.367+0000 7f961bf5e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961419d000 con 0x7f9614106bf0 2024-04-23T02:48:40.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.367+0000 7f961bf5e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96141964c0 con 0x7f9614100b40 2024-04-23T02:48:40.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.367+0000 7f96194f9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9614106bf0 0x7f961419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:40.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.367+0000 7f96194f9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9614106bf0 0x7f961419c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40012/0 (socket says 172.21.15.71:40012) 2024-04-23T02:48:40.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.367+0000 7f96194f9700 1 -- 172.21.15.71:0/2106314845 learned_addr learned my addr 172.21.15.71:0/2106314845 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:48:40.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.368+0000 7f9619cfa700 1 --2- 172.21.15.71:0/2106314845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9614100b40 0x7f961419c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:40.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.368+0000 7f96194f9700 1 -- 172.21.15.71:0/2106314845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9614100b40 msgr2=0x7f961419c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:40.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.368+0000 7f96194f9700 1 --2- 172.21.15.71:0/2106314845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9614100b40 0x7f961419c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:40.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.368+0000 7f96194f9700 1 -- 172.21.15.71:0/2106314845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96100095f0 con 0x7f9614106bf0 2024-04-23T02:48:40.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.368+0000 7f9619cfa700 1 --2- 172.21.15.71:0/2106314845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9614100b40 0x7f961419c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:48:40.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.368+0000 7f96194f9700 1 --2- 172.21.15.71:0/2106314845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9614106bf0 0x7f961419c920 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f960400b4b0 tx=0x7f960400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:40.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.368+0000 7f960affd700 1 -- 172.21.15.71:0/2106314845 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9604011800 con 0x7f9614106bf0 2024-04-23T02:48:40.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.368+0000 7f960affd700 1 -- 172.21.15.71:0/2106314845 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9604011e00 con 0x7f9614106bf0 2024-04-23T02:48:40.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.368+0000 7f961bf5e700 1 -- 172.21.15.71:0/2106314845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96141967a0 con 0x7f9614106bf0 2024-04-23T02:48:40.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.368+0000 7f960affd700 1 -- 172.21.15.71:0/2106314845 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9604010e50 con 0x7f9614106bf0 2024-04-23T02:48:40.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.368+0000 7f961bf5e700 1 -- 172.21.15.71:0/2106314845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9614196d70 con 0x7f9614106bf0 2024-04-23T02:48:40.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.369+0000 7f961bf5e700 1 -- 172.21.15.71:0/2106314845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9614109020 con 0x7f9614106bf0 2024-04-23T02:48:40.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.369+0000 7f960affd700 1 -- 172.21.15.71:0/2106314845 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9604010380 con 0x7f9614106bf0 2024-04-23T02:48:40.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.370+0000 7f960affd700 1 --2- 172.21.15.71:0/2106314845 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f960006cce0 0x7f960006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:40.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.370+0000 7f960affd700 1 -- 172.21.15.71:0/2106314845 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f960408c0f0 con 0x7f9614106bf0 2024-04-23T02:48:40.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.370+0000 7f9619cfa700 1 --2- 172.21.15.71:0/2106314845 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f960006cce0 0x7f960006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:40.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.370+0000 7f9619cfa700 1 --2- 172.21.15.71:0/2106314845 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f960006cce0 0x7f960006f1a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f961000b5c0 tx=0x7f9610005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:40.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.374+0000 7f960affd700 1 -- 172.21.15.71:0/2106314845 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9604059c70 con 0x7f9614106bf0 2024-04-23T02:48:40.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.610+0000 7f961bf5e700 1 -- 172.21.15.71:0/2106314845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9614196ff0 con 0x7f9614106bf0 2024-04-23T02:48:40.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.611+0000 7f960affd700 1 -- 172.21.15.71:0/2106314845 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f9604059800 con 0x7f9614106bf0 2024-04-23T02:48:40.614 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:48:40.614 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:48:40.614 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:48:40.614 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:48:40.614 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:48:40.614 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:48:40.614 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:48:40.614 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:48:40.614 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:48:40.614 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:48:40.614 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:48:40.614 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:48:40.614 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:48:40.615 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:48:40.615 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:48:40.615 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:48:40.615 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:48:40.615 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:48:40.615 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:48:40.615 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:48:40.615 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:48:40.615 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:48:40.615 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:48:40.615 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:48:40.615 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:48:40.615 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:48:40.615 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:48:40.616 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:48:40.616 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:48:40.616 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:48:40.616 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:48:40.616 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:48:40.616 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:48:40.616 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:48:40.616 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:48:40.616 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:48:40.616 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:48:40.616 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:48:40.616 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:48:40.616 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:48:40.616 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:48:40.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.613+0000 7f961bf5e700 1 -- 172.21.15.71:0/2106314845 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f960006cce0 msgr2=0x7f960006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:40.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.613+0000 7f961bf5e700 1 --2- 172.21.15.71:0/2106314845 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f960006cce0 0x7f960006f1a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f961000b5c0 tx=0x7f9610005960 comp rx=0 tx=0).stop 2024-04-23T02:48:40.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.613+0000 7f961bf5e700 1 -- 172.21.15.71:0/2106314845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9614106bf0 msgr2=0x7f961419c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:40.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.613+0000 7f961bf5e700 1 --2- 172.21.15.71:0/2106314845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9614106bf0 0x7f961419c920 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f960400b4b0 tx=0x7f960400b870 comp rx=0 tx=0).stop 2024-04-23T02:48:40.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.613+0000 7f961bf5e700 1 -- 172.21.15.71:0/2106314845 shutdown_connections 2024-04-23T02:48:40.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.613+0000 7f961bf5e700 1 --2- 172.21.15.71:0/2106314845 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f960006cce0 0x7f960006f1a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:40.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.613+0000 7f961bf5e700 1 --2- 172.21.15.71:0/2106314845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9614100b40 0x7f961419c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:40.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.613+0000 7f961bf5e700 1 --2- 172.21.15.71:0/2106314845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9614106bf0 0x7f961419c920 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:40.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.613+0000 7f961bf5e700 1 -- 172.21.15.71:0/2106314845 >> 172.21.15.71:0/2106314845 conn(0x7f96140fc610 msgr2=0x7f96140fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:40.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.613+0000 7f961bf5e700 1 -- 172.21.15.71:0/2106314845 shutdown_connections 2024-04-23T02:48:40.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.613+0000 7f961bf5e700 1 -- 172.21.15.71:0/2106314845 wait complete. 2024-04-23T02:48:40.618 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:48:40.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.717+0000 7fad4c334700 1 -- 172.21.15.71:0/925712529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad44101850 msgr2=0x7fad441058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:40.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.717+0000 7fad4c334700 1 --2- 172.21.15.71:0/925712529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad44101850 0x7fad441058a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fad34009910 tx=0x7fad34009c20 comp rx=0 tx=0).stop 2024-04-23T02:48:40.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.717+0000 7fad4c334700 1 -- 172.21.15.71:0/925712529 shutdown_connections 2024-04-23T02:48:40.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.717+0000 7fad4c334700 1 --2- 172.21.15.71:0/925712529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad44101850 0x7fad441058a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:40.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.717+0000 7fad4c334700 1 --2- 172.21.15.71:0/925712529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad44100ea0 0x7fad44101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:40.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.717+0000 7fad4c334700 1 -- 172.21.15.71:0/925712529 >> 172.21.15.71:0/925712529 conn(0x7fad440fc750 msgr2=0x7fad440feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:40.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.717+0000 7fad4c334700 1 -- 172.21.15.71:0/925712529 shutdown_connections 2024-04-23T02:48:40.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.717+0000 7fad4c334700 1 -- 172.21.15.71:0/925712529 wait complete. 2024-04-23T02:48:40.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.718+0000 7fad4c334700 1 Processor -- start 2024-04-23T02:48:40.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.718+0000 7fad4c334700 1 -- start start 2024-04-23T02:48:40.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.718+0000 7fad4c334700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad44100ea0 0x7fad44073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:40.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.718+0000 7fad4c334700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad44101850 0x7fad440735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:40.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.718+0000 7fad4c334700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad44073b10 con 0x7fad44100ea0 2024-04-23T02:48:40.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.718+0000 7fad4c334700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad44073c50 con 0x7fad44101850 2024-04-23T02:48:40.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.718+0000 7fad4a0d0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad44100ea0 0x7fad44073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:40.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.718+0000 7fad4a0d0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad44100ea0 0x7fad44073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40032/0 (socket says 172.21.15.71:40032) 2024-04-23T02:48:40.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.718+0000 7fad4a0d0700 1 -- 172.21.15.71:0/2579969914 learned_addr learned my addr 172.21.15.71:0/2579969914 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:48:40.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.718+0000 7fad498cf700 1 --2- 172.21.15.71:0/2579969914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad44101850 0x7fad440735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:40.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.719+0000 7fad4a0d0700 1 -- 172.21.15.71:0/2579969914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad44101850 msgr2=0x7fad440735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:40.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.719+0000 7fad4a0d0700 1 --2- 172.21.15.71:0/2579969914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad44101850 0x7fad440735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:40.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.719+0000 7fad4a0d0700 1 -- 172.21.15.71:0/2579969914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad340095f0 con 0x7fad44100ea0 2024-04-23T02:48:40.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.719+0000 7fad4a0d0700 1 --2- 172.21.15.71:0/2579969914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad44100ea0 0x7fad44073090 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fad4000c8a0 tx=0x7fad4000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:40.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.719+0000 7fad3b7fe700 1 -- 172.21.15.71:0/2579969914 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad400078e0 con 0x7fad44100ea0 2024-04-23T02:48:40.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.719+0000 7fad3b7fe700 1 -- 172.21.15.71:0/2579969914 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fad4000f450 con 0x7fad44100ea0 2024-04-23T02:48:40.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.719+0000 7fad4c334700 1 -- 172.21.15.71:0/2579969914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad441a2a20 con 0x7fad44100ea0 2024-04-23T02:48:40.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.719+0000 7fad3b7fe700 1 -- 172.21.15.71:0/2579969914 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad4000e5e0 con 0x7fad44100ea0 2024-04-23T02:48:40.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.719+0000 7fad4c334700 1 -- 172.21.15.71:0/2579969914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad441a2dd0 con 0x7fad44100ea0 2024-04-23T02:48:40.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.720+0000 7fad4c334700 1 -- 172.21.15.71:0/2579969914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad441092a0 con 0x7fad44100ea0 2024-04-23T02:48:40.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.720+0000 7fad3b7fe700 1 -- 172.21.15.71:0/2579969914 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fad4000f5c0 con 0x7fad44100ea0 2024-04-23T02:48:40.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.721+0000 7fad3b7fe700 1 --2- 172.21.15.71:0/2579969914 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fad3006cce0 0x7fad3006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:40.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.721+0000 7fad3b7fe700 1 -- 172.21.15.71:0/2579969914 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fad4008c480 con 0x7fad44100ea0 2024-04-23T02:48:40.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.721+0000 7fad498cf700 1 --2- 172.21.15.71:0/2579969914 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fad3006cce0 0x7fad3006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:40.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.721+0000 7fad498cf700 1 --2- 172.21.15.71:0/2579969914 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fad3006cce0 0x7fad3006f1a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fad34006010 tx=0x7fad3401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:40.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.725+0000 7fad3b7fe700 1 -- 172.21.15.71:0/2579969914 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fad40056570 con 0x7fad44100ea0 2024-04-23T02:48:40.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.938+0000 7fad4c334700 1 -- 172.21.15.71:0/2579969914 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fad440611d0 con 0x7fad3006cce0 2024-04-23T02:48:40.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.939+0000 7fad3b7fe700 1 -- 172.21.15.71:0/2579969914 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fad440611d0 con 0x7fad3006cce0 2024-04-23T02:48:40.942 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:48:40.943 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:48:40.943 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:48:40.943 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:48:40.943 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:48:40.943 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:48:40.943 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:48:40.943 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:48:40.943 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:48:40.943 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:48:40.943 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:48:40.943 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:48:40.943 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:48:40.943 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:48:40.943 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:48:40.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.941+0000 7fad4c334700 1 -- 172.21.15.71:0/2579969914 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fad3006cce0 msgr2=0x7fad3006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:40.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.941+0000 7fad4c334700 1 --2- 172.21.15.71:0/2579969914 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fad3006cce0 0x7fad3006f1a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fad34006010 tx=0x7fad3401a040 comp rx=0 tx=0).stop 2024-04-23T02:48:40.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.941+0000 7fad4c334700 1 -- 172.21.15.71:0/2579969914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad44100ea0 msgr2=0x7fad44073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:40.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.941+0000 7fad4c334700 1 --2- 172.21.15.71:0/2579969914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad44100ea0 0x7fad44073090 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fad4000c8a0 tx=0x7fad4000cbb0 comp rx=0 tx=0).stop 2024-04-23T02:48:40.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.941+0000 7fad4c334700 1 -- 172.21.15.71:0/2579969914 shutdown_connections 2024-04-23T02:48:40.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.941+0000 7fad4c334700 1 --2- 172.21.15.71:0/2579969914 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fad3006cce0 0x7fad3006f1a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:40.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.942+0000 7fad4c334700 1 --2- 172.21.15.71:0/2579969914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad44100ea0 0x7fad44073090 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:40.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.942+0000 7fad4c334700 1 --2- 172.21.15.71:0/2579969914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad44101850 0x7fad440735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:40.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.942+0000 7fad4c334700 1 -- 172.21.15.71:0/2579969914 >> 172.21.15.71:0/2579969914 conn(0x7fad440fc750 msgr2=0x7fad440fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:40.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.942+0000 7fad4c334700 1 -- 172.21.15.71:0/2579969914 shutdown_connections 2024-04-23T02:48:40.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:40.942+0000 7fad4c334700 1 -- 172.21.15.71:0/2579969914 wait complete. 2024-04-23T02:48:40.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:40 smithi071 ceph-mon[144544]: from='client.35724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:48:40.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:40 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/803927802' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:48:40.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:40 smithi071 ceph-mon[144544]: pgmap v1062: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:40.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:40 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2106314845' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:48:41.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.045+0000 7fe7a8d6e700 1 -- 172.21.15.71:0/716642598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7a4101830 msgr2=0x7fe7a4105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:41.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.045+0000 7fe7a8d6e700 1 --2- 172.21.15.71:0/716642598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7a4101830 0x7fe7a4105880 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe794009910 tx=0x7fe794009c20 comp rx=0 tx=0).stop 2024-04-23T02:48:41.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.045+0000 7fe7a8d6e700 1 -- 172.21.15.71:0/716642598 shutdown_connections 2024-04-23T02:48:41.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.045+0000 7fe7a8d6e700 1 --2- 172.21.15.71:0/716642598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7a4101830 0x7fe7a4105880 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:41.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.045+0000 7fe7a8d6e700 1 --2- 172.21.15.71:0/716642598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a4100e80 0x7fe7a4101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:41.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.045+0000 7fe7a8d6e700 1 -- 172.21.15.71:0/716642598 >> 172.21.15.71:0/716642598 conn(0x7fe7a40fc710 msgr2=0x7fe7a40feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:41.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.045+0000 7fe7a8d6e700 1 -- 172.21.15.71:0/716642598 shutdown_connections 2024-04-23T02:48:41.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.045+0000 7fe7a8d6e700 1 -- 172.21.15.71:0/716642598 wait complete. 2024-04-23T02:48:41.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.045+0000 7fe7a8d6e700 1 Processor -- start 2024-04-23T02:48:41.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.045+0000 7fe7a8d6e700 1 -- start start 2024-04-23T02:48:41.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.046+0000 7fe7a8d6e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a4100e80 0x7fe7a4073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:41.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.046+0000 7fe7a8d6e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7a4101830 0x7fe7a40735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:41.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.046+0000 7fe7a8d6e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a4073b10 con 0x7fe7a4101830 2024-04-23T02:48:41.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.046+0000 7fe7a8d6e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a4073c50 con 0x7fe7a4100e80 2024-04-23T02:48:41.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.046+0000 7fe7a1d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7a4101830 0x7fe7a40735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:41.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.046+0000 7fe7a1d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7a4101830 0x7fe7a40735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40060/0 (socket says 172.21.15.71:40060) 2024-04-23T02:48:41.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.046+0000 7fe7a1d9b700 1 -- 172.21.15.71:0/177822969 learned_addr learned my addr 172.21.15.71:0/177822969 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:48:41.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.046+0000 7fe7a259c700 1 --2- 172.21.15.71:0/177822969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a4100e80 0x7fe7a4073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:41.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.046+0000 7fe7a1d9b700 1 -- 172.21.15.71:0/177822969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a4100e80 msgr2=0x7fe7a4073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:41.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.046+0000 7fe7a1d9b700 1 --2- 172.21.15.71:0/177822969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a4100e80 0x7fe7a4073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:41.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.046+0000 7fe7a1d9b700 1 -- 172.21.15.71:0/177822969 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7940095f0 con 0x7fe7a4101830 2024-04-23T02:48:41.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.046+0000 7fe7a259c700 1 --2- 172.21.15.71:0/177822969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a4100e80 0x7fe7a4073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:48:41.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.046+0000 7fe7a1d9b700 1 --2- 172.21.15.71:0/177822969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7a4101830 0x7fe7a40735d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe7940092e0 tx=0x7fe79400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:41.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.046+0000 7fe7937fe700 1 -- 172.21.15.71:0/177822969 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe79401d070 con 0x7fe7a4101830 2024-04-23T02:48:41.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.046+0000 7fe7937fe700 1 -- 172.21.15.71:0/177822969 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe79400f460 con 0x7fe7a4101830 2024-04-23T02:48:41.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.046+0000 7fe7a8d6e700 1 -- 172.21.15.71:0/177822969 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a4073ed0 con 0x7fe7a4101830 2024-04-23T02:48:41.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.046+0000 7fe7937fe700 1 -- 172.21.15.71:0/177822969 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe794021600 con 0x7fe7a4101830 2024-04-23T02:48:41.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.046+0000 7fe7a8d6e700 1 -- 172.21.15.71:0/177822969 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a41a2e10 con 0x7fe7a4101830 2024-04-23T02:48:41.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.047+0000 7fe7a8d6e700 1 -- 172.21.15.71:0/177822969 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a41091d0 con 0x7fe7a4101830 2024-04-23T02:48:41.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.048+0000 7fe7937fe700 1 -- 172.21.15.71:0/177822969 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe79400f5d0 con 0x7fe7a4101830 2024-04-23T02:48:41.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.048+0000 7fe7937fe700 1 --2- 172.21.15.71:0/177822969 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe78c06cce0 0x7fe78c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:48:41.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.048+0000 7fe7937fe700 1 -- 172.21.15.71:0/177822969 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe79408d860 con 0x7fe7a4101830 2024-04-23T02:48:41.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.048+0000 7fe7a259c700 1 --2- 172.21.15.71:0/177822969 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe78c06cce0 0x7fe78c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:48:41.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.049+0000 7fe7a259c700 1 --2- 172.21.15.71:0/177822969 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe78c06cce0 0x7fe78c06f1a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe7980099d0 tx=0x7fe798008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:48:41.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.053+0000 7fe7937fe700 1 -- 172.21.15.71:0/177822969 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe7940578d0 con 0x7fe7a4101830 2024-04-23T02:48:41.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:40 smithi192 ceph-mon[108617]: from='client.35724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:48:41.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:40 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/803927802' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:48:41.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:40 smithi192 ceph-mon[108617]: pgmap v1062: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:41.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:40 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2106314845' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:48:41.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.322+0000 7fe7a8d6e700 1 -- 172.21.15.71:0/177822969 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe7a4074660 con 0x7fe7a4101830 2024-04-23T02:48:41.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.322+0000 7fe7937fe700 1 -- 172.21.15.71:0/177822969 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7fe79405aef0 con 0x7fe7a4101830 2024-04-23T02:48:41.325 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:48:41.325 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:48:41.325 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:48:41.325 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:48:41.325 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:48:41.325 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:48:41.325 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:48:41.326 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:48:41.326 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:48:41.326 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:48:41.326 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:48:41.326 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:48:41.326 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:48:41.326 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:48:41.326 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:48:41.326 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:48:41.326 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:48:41.326 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:48:41.326 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:48:41.326 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:48:41.327 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:48:41.327 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:48:41.327 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:48:41.327 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:48:41.327 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:48:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.324+0000 7fe7a8d6e700 1 -- 172.21.15.71:0/177822969 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe78c06cce0 msgr2=0x7fe78c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:41.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.324+0000 7fe7a8d6e700 1 --2- 172.21.15.71:0/177822969 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe78c06cce0 0x7fe78c06f1a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe7980099d0 tx=0x7fe798008040 comp rx=0 tx=0).stop 2024-04-23T02:48:41.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.324+0000 7fe7a8d6e700 1 -- 172.21.15.71:0/177822969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7a4101830 msgr2=0x7fe7a40735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:48:41.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.324+0000 7fe7a8d6e700 1 --2- 172.21.15.71:0/177822969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7a4101830 0x7fe7a40735d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe7940092e0 tx=0x7fe79400be20 comp rx=0 tx=0).stop 2024-04-23T02:48:41.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.324+0000 7fe7a8d6e700 1 -- 172.21.15.71:0/177822969 shutdown_connections 2024-04-23T02:48:41.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.324+0000 7fe7a8d6e700 1 --2- 172.21.15.71:0/177822969 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe78c06cce0 0x7fe78c06f1a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:41.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.324+0000 7fe7a8d6e700 1 --2- 172.21.15.71:0/177822969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a4100e80 0x7fe7a4073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:41.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.324+0000 7fe7a8d6e700 1 --2- 172.21.15.71:0/177822969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7a4101830 0x7fe7a40735d0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:48:41.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.324+0000 7fe7a8d6e700 1 -- 172.21.15.71:0/177822969 >> 172.21.15.71:0/177822969 conn(0x7fe7a40fc710 msgr2=0x7fe7a40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:48:41.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.324+0000 7fe7a8d6e700 1 -- 172.21.15.71:0/177822969 shutdown_connections 2024-04-23T02:48:41.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:48:41.324+0000 7fe7a8d6e700 1 -- 172.21.15.71:0/177822969 wait complete. 2024-04-23T02:48:41.970 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:41 smithi192 ceph-mon[108617]: from='client.35736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:48:41.970 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:41 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/177822969' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:48:41.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:41 smithi071 ceph-mon[144544]: from='client.35736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:48:41.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:41 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/177822969' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:48:42.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:42 smithi071 ceph-mon[144544]: pgmap v1063: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:43.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:42 smithi192 ceph-mon[108617]: pgmap v1063: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:44.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:44 smithi071 ceph-mon[144544]: pgmap v1064: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:45.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:44 smithi192 ceph-mon[108617]: pgmap v1064: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:46.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:46 smithi071 ceph-mon[144544]: pgmap v1065: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:47.016 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:46 smithi192 ceph-mon[108617]: pgmap v1065: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:48.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:48 smithi071 ceph-mon[144544]: pgmap v1066: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:49.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:48 smithi192 ceph-mon[108617]: pgmap v1066: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:50.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:48:50.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:48:51.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:50 smithi192 ceph-mon[108617]: pgmap v1067: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:51.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:50 smithi071 ceph-mon[144544]: pgmap v1067: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:52.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:52 smithi071 ceph-mon[144544]: pgmap v1068: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:53.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:52 smithi192 ceph-mon[108617]: pgmap v1068: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:54.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:54 smithi071 ceph-mon[144544]: pgmap v1069: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:55.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:54 smithi192 ceph-mon[108617]: pgmap v1069: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:56.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:56 smithi071 ceph-mon[144544]: pgmap v1070: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:57.109 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:56 smithi192 ceph-mon[108617]: pgmap v1070: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:58.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:48:58 smithi071 ceph-mon[144544]: pgmap v1071: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:48:59.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:48:58 smithi192 ceph-mon[108617]: pgmap v1071: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:00.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:00 smithi071 ceph-mon[144544]: pgmap v1072: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:01.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:00 smithi192 ceph-mon[108617]: pgmap v1072: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:02.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:02 smithi071 ceph-mon[144544]: pgmap v1073: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:03.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:02 smithi192 ceph-mon[108617]: pgmap v1073: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:04.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:04 smithi071 ceph-mon[144544]: pgmap v1074: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:05.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:04 smithi192 ceph-mon[108617]: pgmap v1074: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:05.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:49:06.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:49:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:06 smithi071 ceph-mon[144544]: pgmap v1075: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:07.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:06 smithi192 ceph-mon[108617]: pgmap v1075: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:08.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:08 smithi071 ceph-mon[144544]: pgmap v1076: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:09.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:08 smithi192 ceph-mon[108617]: pgmap v1076: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:10.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:10 smithi071 ceph-mon[144544]: pgmap v1077: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:11.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:10 smithi192 ceph-mon[108617]: pgmap v1077: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:11.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.438+0000 7f5162b4b700 1 -- 172.21.15.71:0/3638604616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f515c0ffb40 msgr2=0x7f515c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:11.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.438+0000 7f5162b4b700 1 --2- 172.21.15.71:0/3638604616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f515c0ffb40 0x7f515c0fffc0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f5148009910 tx=0x7f5148009c20 comp rx=0 tx=0).stop 2024-04-23T02:49:11.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.438+0000 7f5162b4b700 1 -- 172.21.15.71:0/3638604616 shutdown_connections 2024-04-23T02:49:11.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.438+0000 7f5162b4b700 1 --2- 172.21.15.71:0/3638604616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f515c0ffb40 0x7f515c0fffc0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:11.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.438+0000 7f5162b4b700 1 --2- 172.21.15.71:0/3638604616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f515c105bf0 0x7f515c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:11.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.438+0000 7f5162b4b700 1 -- 172.21.15.71:0/3638604616 >> 172.21.15.71:0/3638604616 conn(0x7f515c0fb610 msgr2=0x7f515c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:11.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.438+0000 7f5162b4b700 1 -- 172.21.15.71:0/3638604616 shutdown_connections 2024-04-23T02:49:11.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.438+0000 7f5162b4b700 1 -- 172.21.15.71:0/3638604616 wait complete. 2024-04-23T02:49:11.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.439+0000 7f5162b4b700 1 Processor -- start 2024-04-23T02:49:11.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.439+0000 7f5162b4b700 1 -- start start 2024-04-23T02:49:11.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.439+0000 7f5162b4b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f515c0ffb40 0x7f515c195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:11.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.439+0000 7f5162b4b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f515c105bf0 0x7f515c195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:11.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.439+0000 7f5162b4b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515c196240 con 0x7f515c0ffb40 2024-04-23T02:49:11.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.439+0000 7f5162b4b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515c199fd0 con 0x7f515c105bf0 2024-04-23T02:49:11.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.439+0000 7f51608e7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f515c0ffb40 0x7f515c195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:11.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.439+0000 7f51608e7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f515c0ffb40 0x7f515c195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49766/0 (socket says 172.21.15.71:49766) 2024-04-23T02:49:11.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.439+0000 7f51608e7700 1 -- 172.21.15.71:0/4083132238 learned_addr learned my addr 172.21.15.71:0/4083132238 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:49:11.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.440+0000 7f515bfff700 1 --2- 172.21.15.71:0/4083132238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f515c105bf0 0x7f515c195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:11.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.440+0000 7f51608e7700 1 -- 172.21.15.71:0/4083132238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f515c105bf0 msgr2=0x7f515c195b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:11.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.440+0000 7f51608e7700 1 --2- 172.21.15.71:0/4083132238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f515c105bf0 0x7f515c195b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:11.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.440+0000 7f51608e7700 1 -- 172.21.15.71:0/4083132238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51480095f0 con 0x7f515c0ffb40 2024-04-23T02:49:11.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.440+0000 7f515bfff700 1 --2- 172.21.15.71:0/4083132238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f515c105bf0 0x7f515c195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:49:11.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.440+0000 7f51608e7700 1 --2- 172.21.15.71:0/4083132238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f515c0ffb40 0x7f515c195620 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f5150007910 tx=0x7f5150007cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:11.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.440+0000 7f5159ffb700 1 -- 172.21.15.71:0/4083132238 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5150012070 con 0x7f515c0ffb40 2024-04-23T02:49:11.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.440+0000 7f5159ffb700 1 -- 172.21.15.71:0/4083132238 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f515000d810 con 0x7f515c0ffb40 2024-04-23T02:49:11.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.440+0000 7f5162b4b700 1 -- 172.21.15.71:0/4083132238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515c19a250 con 0x7f515c0ffb40 2024-04-23T02:49:11.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.440+0000 7f5159ffb700 1 -- 172.21.15.71:0/4083132238 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51500053d0 con 0x7f515c0ffb40 2024-04-23T02:49:11.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.440+0000 7f5162b4b700 1 -- 172.21.15.71:0/4083132238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515c19a7a0 con 0x7f515c0ffb40 2024-04-23T02:49:11.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.441+0000 7f5162b4b700 1 -- 172.21.15.71:0/4083132238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f515c04f2e0 con 0x7f515c0ffb40 2024-04-23T02:49:11.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.442+0000 7f5159ffb700 1 -- 172.21.15.71:0/4083132238 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5150016070 con 0x7f515c0ffb40 2024-04-23T02:49:11.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.442+0000 7f5159ffb700 1 --2- 172.21.15.71:0/4083132238 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f514c06cce0 0x7f514c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:11.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.442+0000 7f5159ffb700 1 -- 172.21.15.71:0/4083132238 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f515008d760 con 0x7f515c0ffb40 2024-04-23T02:49:11.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.442+0000 7f515bfff700 1 --2- 172.21.15.71:0/4083132238 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f514c06cce0 0x7f514c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:11.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.443+0000 7f515bfff700 1 --2- 172.21.15.71:0/4083132238 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f514c06cce0 0x7f514c06f1a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f514800b5c0 tx=0x7f5148005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:11.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.447+0000 7f5159ffb700 1 -- 172.21.15.71:0/4083132238 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5150057850 con 0x7f515c0ffb40 2024-04-23T02:49:11.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.658+0000 7f5162b4b700 1 -- 172.21.15.71:0/4083132238 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f515c19acb0 con 0x7f514c06cce0 2024-04-23T02:49:11.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.659+0000 7f5159ffb700 1 -- 172.21.15.71:0/4083132238 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f515c19acb0 con 0x7f514c06cce0 2024-04-23T02:49:11.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.661+0000 7f5162b4b700 1 -- 172.21.15.71:0/4083132238 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f514c06cce0 msgr2=0x7f514c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:11.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.661+0000 7f5162b4b700 1 --2- 172.21.15.71:0/4083132238 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f514c06cce0 0x7f514c06f1a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f514800b5c0 tx=0x7f5148005fd0 comp rx=0 tx=0).stop 2024-04-23T02:49:11.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.661+0000 7f5162b4b700 1 -- 172.21.15.71:0/4083132238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f515c0ffb40 msgr2=0x7f515c195620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:11.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.661+0000 7f5162b4b700 1 --2- 172.21.15.71:0/4083132238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f515c0ffb40 0x7f515c195620 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f5150007910 tx=0x7f5150007cd0 comp rx=0 tx=0).stop 2024-04-23T02:49:11.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.661+0000 7f5162b4b700 1 -- 172.21.15.71:0/4083132238 shutdown_connections 2024-04-23T02:49:11.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.661+0000 7f5162b4b700 1 --2- 172.21.15.71:0/4083132238 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f514c06cce0 0x7f514c06f1a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:11.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.661+0000 7f5162b4b700 1 --2- 172.21.15.71:0/4083132238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f515c0ffb40 0x7f515c195620 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:11.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.661+0000 7f5162b4b700 1 --2- 172.21.15.71:0/4083132238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f515c105bf0 0x7f515c195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:11.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.661+0000 7f5162b4b700 1 -- 172.21.15.71:0/4083132238 >> 172.21.15.71:0/4083132238 conn(0x7f515c0fb610 msgr2=0x7f515c0fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:11.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.662+0000 7f5162b4b700 1 -- 172.21.15.71:0/4083132238 shutdown_connections 2024-04-23T02:49:11.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.662+0000 7f5162b4b700 1 -- 172.21.15.71:0/4083132238 wait complete. 2024-04-23T02:49:11.676 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:49:11.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.765+0000 7fbdf6cdd700 1 -- 172.21.15.71:0/2587056644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf00ffda0 msgr2=0x7fbdf0100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:11.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.765+0000 7fbdf6cdd700 1 --2- 172.21.15.71:0/2587056644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf00ffda0 0x7fbdf0100180 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009910 tx=0x7fbde0009c20 comp rx=0 tx=0).stop 2024-04-23T02:49:11.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.766+0000 7fbdf6cdd700 1 -- 172.21.15.71:0/2587056644 shutdown_connections 2024-04-23T02:49:11.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.766+0000 7fbdf6cdd700 1 --2- 172.21.15.71:0/2587056644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdf0100750 0x7fbdf01047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:11.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.766+0000 7fbdf6cdd700 1 --2- 172.21.15.71:0/2587056644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf00ffda0 0x7fbdf0100180 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:11.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.766+0000 7fbdf6cdd700 1 -- 172.21.15.71:0/2587056644 >> 172.21.15.71:0/2587056644 conn(0x7fbdf00fb650 msgr2=0x7fbdf00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:11.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.766+0000 7fbdf6cdd700 1 -- 172.21.15.71:0/2587056644 shutdown_connections 2024-04-23T02:49:11.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.766+0000 7fbdf6cdd700 1 -- 172.21.15.71:0/2587056644 wait complete. 2024-04-23T02:49:11.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.766+0000 7fbdf6cdd700 1 Processor -- start 2024-04-23T02:49:11.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.766+0000 7fbdf6cdd700 1 -- start start 2024-04-23T02:49:11.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.766+0000 7fbdf6cdd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf00ffda0 0x7fbdf019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:11.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.766+0000 7fbdf6cdd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdf0100750 0x7fbdf019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:11.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.766+0000 7fbdf6cdd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf019d150 con 0x7fbdf00ffda0 2024-04-23T02:49:11.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.766+0000 7fbdf6cdd700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf0196600 con 0x7fbdf0100750 2024-04-23T02:49:11.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.766+0000 7fbdf4a79700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf00ffda0 0x7fbdf019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:11.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.767+0000 7fbdf4a79700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf00ffda0 0x7fbdf019c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49770/0 (socket says 172.21.15.71:49770) 2024-04-23T02:49:11.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.767+0000 7fbdf4a79700 1 -- 172.21.15.71:0/1486792151 learned_addr learned my addr 172.21.15.71:0/1486792151 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:49:11.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.767+0000 7fbdeffff700 1 --2- 172.21.15.71:0/1486792151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdf0100750 0x7fbdf019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:11.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.767+0000 7fbdf4a79700 1 -- 172.21.15.71:0/1486792151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdf0100750 msgr2=0x7fbdf019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:11.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.767+0000 7fbdf4a79700 1 --2- 172.21.15.71:0/1486792151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdf0100750 0x7fbdf019cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:11.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.767+0000 7fbdf4a79700 1 -- 172.21.15.71:0/1486792151 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde00095f0 con 0x7fbdf00ffda0 2024-04-23T02:49:11.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.767+0000 7fbdeffff700 1 --2- 172.21.15.71:0/1486792151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdf0100750 0x7fbdf019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:49:11.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.767+0000 7fbdf4a79700 1 --2- 172.21.15.71:0/1486792151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf00ffda0 0x7fbdf019c580 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fbde0006010 tx=0x7fbde000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:11.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.767+0000 7fbdedffb700 1 -- 172.21.15.71:0/1486792151 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbde001d070 con 0x7fbdf00ffda0 2024-04-23T02:49:11.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.767+0000 7fbdf6cdd700 1 -- 172.21.15.71:0/1486792151 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf0196880 con 0x7fbdf00ffda0 2024-04-23T02:49:11.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.767+0000 7fbdedffb700 1 -- 172.21.15.71:0/1486792151 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbde000f460 con 0x7fbdf00ffda0 2024-04-23T02:49:11.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.767+0000 7fbdedffb700 1 -- 172.21.15.71:0/1486792151 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbde0021640 con 0x7fbdf00ffda0 2024-04-23T02:49:11.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.767+0000 7fbdf6cdd700 1 -- 172.21.15.71:0/1486792151 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf0196d70 con 0x7fbdf00ffda0 2024-04-23T02:49:11.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.768+0000 7fbdf6cdd700 1 -- 172.21.15.71:0/1486792151 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf004f2e0 con 0x7fbdf00ffda0 2024-04-23T02:49:11.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.769+0000 7fbdedffb700 1 -- 172.21.15.71:0/1486792151 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbde000fa20 con 0x7fbdf00ffda0 2024-04-23T02:49:11.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.769+0000 7fbdedffb700 1 --2- 172.21.15.71:0/1486792151 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbdd806cce0 0x7fbdd806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:11.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.769+0000 7fbdedffb700 1 -- 172.21.15.71:0/1486792151 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbde008d830 con 0x7fbdf00ffda0 2024-04-23T02:49:11.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.769+0000 7fbdeffff700 1 --2- 172.21.15.71:0/1486792151 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbdd806cce0 0x7fbdd806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:11.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.770+0000 7fbdeffff700 1 --2- 172.21.15.71:0/1486792151 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbdd806cce0 0x7fbdd806f1a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbdf00fe090 tx=0x7fbde400b5f0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:11.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.774+0000 7fbdedffb700 1 -- 172.21.15.71:0/1486792151 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbde005b330 con 0x7fbdf00ffda0 2024-04-23T02:49:11.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.986+0000 7fbdf6cdd700 1 -- 172.21.15.71:0/1486792151 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdf0196fc0 con 0x7fbdd806cce0 2024-04-23T02:49:11.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.987+0000 7fbdedffb700 1 -- 172.21.15.71:0/1486792151 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbdf0196fc0 con 0x7fbdd806cce0 2024-04-23T02:49:11.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.989+0000 7fbdf6cdd700 1 -- 172.21.15.71:0/1486792151 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbdd806cce0 msgr2=0x7fbdd806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:11.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.989+0000 7fbdf6cdd700 1 --2- 172.21.15.71:0/1486792151 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbdd806cce0 0x7fbdd806f1a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbdf00fe090 tx=0x7fbde400b5f0 comp rx=0 tx=0).stop 2024-04-23T02:49:11.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.989+0000 7fbdf6cdd700 1 -- 172.21.15.71:0/1486792151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf00ffda0 msgr2=0x7fbdf019c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:11.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.989+0000 7fbdf6cdd700 1 --2- 172.21.15.71:0/1486792151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf00ffda0 0x7fbdf019c580 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fbde0006010 tx=0x7fbde000be20 comp rx=0 tx=0).stop 2024-04-23T02:49:11.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.989+0000 7fbdf6cdd700 1 -- 172.21.15.71:0/1486792151 shutdown_connections 2024-04-23T02:49:11.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.989+0000 7fbdf6cdd700 1 --2- 172.21.15.71:0/1486792151 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbdd806cce0 0x7fbdd806f1a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:11.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.989+0000 7fbdf6cdd700 1 --2- 172.21.15.71:0/1486792151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdf00ffda0 0x7fbdf019c580 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:11.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.989+0000 7fbdf6cdd700 1 --2- 172.21.15.71:0/1486792151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdf0100750 0x7fbdf019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:11.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.989+0000 7fbdf6cdd700 1 -- 172.21.15.71:0/1486792151 >> 172.21.15.71:0/1486792151 conn(0x7fbdf00fb650 msgr2=0x7fbdf0103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:11.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.989+0000 7fbdf6cdd700 1 -- 172.21.15.71:0/1486792151 shutdown_connections 2024-04-23T02:49:11.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:11.989+0000 7fbdf6cdd700 1 -- 172.21.15.71:0/1486792151 wait complete. 2024-04-23T02:49:12.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.091+0000 7fbaf72c7700 1 -- 172.21.15.71:0/1788390628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf00ffb40 msgr2=0x7fbaf00fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.091+0000 7fbaf72c7700 1 --2- 172.21.15.71:0/1788390628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf00ffb40 0x7fbaf00fffc0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009910 tx=0x7fbae0009c20 comp rx=0 tx=0).stop 2024-04-23T02:49:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.091+0000 7fbaf72c7700 1 -- 172.21.15.71:0/1788390628 shutdown_connections 2024-04-23T02:49:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.091+0000 7fbaf72c7700 1 --2- 172.21.15.71:0/1788390628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf00ffb40 0x7fbaf00fffc0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.091+0000 7fbaf72c7700 1 --2- 172.21.15.71:0/1788390628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaf0105bf0 0x7fbaf0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.092+0000 7fbaf72c7700 1 -- 172.21.15.71:0/1788390628 >> 172.21.15.71:0/1788390628 conn(0x7fbaf00fb610 msgr2=0x7fbaf00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.092+0000 7fbaf72c7700 1 -- 172.21.15.71:0/1788390628 shutdown_connections 2024-04-23T02:49:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.092+0000 7fbaf72c7700 1 -- 172.21.15.71:0/1788390628 wait complete. 2024-04-23T02:49:12.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.092+0000 7fbaf72c7700 1 Processor -- start 2024-04-23T02:49:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.092+0000 7fbaf72c7700 1 -- start start 2024-04-23T02:49:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.092+0000 7fbaf72c7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf00ffb40 0x7fbaf019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.092+0000 7fbaf72c7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaf0105bf0 0x7fbaf019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.092+0000 7fbaf72c7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf019d0d0 con 0x7fbaf00ffb40 2024-04-23T02:49:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.092+0000 7fbaf72c7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf0196580 con 0x7fbaf0105bf0 2024-04-23T02:49:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.093+0000 7fbaf5063700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf00ffb40 0x7fbaf019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.093+0000 7fbaf5063700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf00ffb40 0x7fbaf019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49782/0 (socket says 172.21.15.71:49782) 2024-04-23T02:49:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.093+0000 7fbaf5063700 1 -- 172.21.15.71:0/792119246 learned_addr learned my addr 172.21.15.71:0/792119246 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:49:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.093+0000 7fbaf4862700 1 --2- 172.21.15.71:0/792119246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaf0105bf0 0x7fbaf019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.093+0000 7fbaf5063700 1 -- 172.21.15.71:0/792119246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaf0105bf0 msgr2=0x7fbaf019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.093+0000 7fbaf5063700 1 --2- 172.21.15.71:0/792119246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaf0105bf0 0x7fbaf019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.093+0000 7fbaf5063700 1 -- 172.21.15.71:0/792119246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae00095f0 con 0x7fbaf00ffb40 2024-04-23T02:49:12.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.093+0000 7fbaf5063700 1 --2- 172.21.15.71:0/792119246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf00ffb40 0x7fbaf019c500 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00c8a0 tx=0x7fbaec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:12.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.093+0000 7fbae67fc700 1 -- 172.21.15.71:0/792119246 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaec00cea0 con 0x7fbaf00ffb40 2024-04-23T02:49:12.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.093+0000 7fbae67fc700 1 -- 172.21.15.71:0/792119246 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbaec004830 con 0x7fbaf00ffb40 2024-04-23T02:49:12.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.093+0000 7fbaf72c7700 1 -- 172.21.15.71:0/792119246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf0196860 con 0x7fbaf00ffb40 2024-04-23T02:49:12.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.093+0000 7fbae67fc700 1 -- 172.21.15.71:0/792119246 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaec005500 con 0x7fbaf00ffb40 2024-04-23T02:49:12.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.093+0000 7fbaf72c7700 1 -- 172.21.15.71:0/792119246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf0196db0 con 0x7fbaf00ffb40 2024-04-23T02:49:12.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.094+0000 7fbaf72c7700 1 -- 172.21.15.71:0/792119246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf004f2e0 con 0x7fbaf00ffb40 2024-04-23T02:49:12.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.095+0000 7fbae67fc700 1 -- 172.21.15.71:0/792119246 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbaec0049a0 con 0x7fbaf00ffb40 2024-04-23T02:49:12.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.095+0000 7fbae67fc700 1 --2- 172.21.15.71:0/792119246 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbadc06cce0 0x7fbadc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:12.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.095+0000 7fbae67fc700 1 -- 172.21.15.71:0/792119246 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbaec08c200 con 0x7fbaf00ffb40 2024-04-23T02:49:12.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.095+0000 7fbaf4862700 1 --2- 172.21.15.71:0/792119246 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbadc06cce0 0x7fbadc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:12.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.095+0000 7fbaf4862700 1 --2- 172.21.15.71:0/792119246 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbadc06cce0 0x7fbadc06f1a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbaf00fcc10 tx=0x7fbae001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:12.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.100+0000 7fbae67fc700 1 -- 172.21.15.71:0/792119246 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbaec059d80 con 0x7fbaf00ffb40 2024-04-23T02:49:12.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.300+0000 7fbaf72c7700 1 -- 172.21.15.71:0/792119246 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbaf01971a0 con 0x7fbadc06cce0 2024-04-23T02:49:12.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.311+0000 7fbae67fc700 1 -- 172.21.15.71:0/792119246 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbaf01971a0 con 0x7fbadc06cce0 2024-04-23T02:49:12.315 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:49:12.315 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (32m) 11m ago 44m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:49:12.315 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (44m) 11m ago 44m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:49:12.315 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (43m) 11m ago 43m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:49:12.315 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (30m) 11m ago 44m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:49:12.315 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (30m) 11m ago 43m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:49:12.315 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (32m) 11m ago 43m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:49:12.315 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (27m) 11m ago 38m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:49:12.315 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (26m) 11m ago 38m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:49:12.315 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (11m) 11m ago 38m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:49:12.315 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (26m) 11m ago 38m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:49:12.315 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (34m) 11m ago 45m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:49:12.315 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (34m) 11m ago 43m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:49:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (31m) 11m ago 45m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:49:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (31m) 11m ago 43m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:49:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (33m) 11m ago 44m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:49:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (33m) 11m ago 43m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:49:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (30m) 11m ago 42m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:49:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (29m) 11m ago 41m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:49:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (29m) 11m ago 41m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:49:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (28m) 11m ago 40m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:49:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (27m) 11m ago 40m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:49:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (27m) 11m ago 39m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:49:12.316 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (33m) 11m ago 43m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:49:12.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.313+0000 7fbaf72c7700 1 -- 172.21.15.71:0/792119246 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbadc06cce0 msgr2=0x7fbadc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:12.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.313+0000 7fbaf72c7700 1 --2- 172.21.15.71:0/792119246 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbadc06cce0 0x7fbadc06f1a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbaf00fcc10 tx=0x7fbae001a040 comp rx=0 tx=0).stop 2024-04-23T02:49:12.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.313+0000 7fbaf72c7700 1 -- 172.21.15.71:0/792119246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf00ffb40 msgr2=0x7fbaf019c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:12.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.313+0000 7fbaf72c7700 1 --2- 172.21.15.71:0/792119246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf00ffb40 0x7fbaf019c500 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00c8a0 tx=0x7fbaec00cc60 comp rx=0 tx=0).stop 2024-04-23T02:49:12.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.313+0000 7fbaf72c7700 1 -- 172.21.15.71:0/792119246 shutdown_connections 2024-04-23T02:49:12.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.313+0000 7fbaf72c7700 1 --2- 172.21.15.71:0/792119246 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbadc06cce0 0x7fbadc06f1a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:12.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.313+0000 7fbaf72c7700 1 --2- 172.21.15.71:0/792119246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf00ffb40 0x7fbaf019c500 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:12.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.313+0000 7fbaf72c7700 1 --2- 172.21.15.71:0/792119246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaf0105bf0 0x7fbaf019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:12.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.313+0000 7fbaf72c7700 1 -- 172.21.15.71:0/792119246 >> 172.21.15.71:0/792119246 conn(0x7fbaf00fb610 msgr2=0x7fbaf0103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:12.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.313+0000 7fbaf72c7700 1 -- 172.21.15.71:0/792119246 shutdown_connections 2024-04-23T02:49:12.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.313+0000 7fbaf72c7700 1 -- 172.21.15.71:0/792119246 wait complete. 2024-04-23T02:49:12.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.418+0000 7fc216f38700 1 -- 172.21.15.71:0/3187898845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc210102700 msgr2=0x7fc21010abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.418+0000 7fc216f38700 1 --2- 172.21.15.71:0/3187898845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc210102700 0x7fc21010abf0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009910 tx=0x7fc1fc009c20 comp rx=0 tx=0).stop 2024-04-23T02:49:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.418+0000 7fc216f38700 1 -- 172.21.15.71:0/3187898845 shutdown_connections 2024-04-23T02:49:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.418+0000 7fc216f38700 1 --2- 172.21.15.71:0/3187898845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc210102700 0x7fc21010abf0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.418+0000 7fc216f38700 1 --2- 172.21.15.71:0/3187898845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc210101de0 0x7fc2101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.418+0000 7fc216f38700 1 -- 172.21.15.71:0/3187898845 >> 172.21.15.71:0/3187898845 conn(0x7fc2100fb610 msgr2=0x7fc2100fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.418+0000 7fc216f38700 1 -- 172.21.15.71:0/3187898845 shutdown_connections 2024-04-23T02:49:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.418+0000 7fc216f38700 1 -- 172.21.15.71:0/3187898845 wait complete. 2024-04-23T02:49:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.418+0000 7fc216f38700 1 Processor -- start 2024-04-23T02:49:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.419+0000 7fc216f38700 1 -- start start 2024-04-23T02:49:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.419+0000 7fc216f38700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc210101de0 0x7fc21019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.419+0000 7fc216f38700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc210102700 0x7fc21019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.419+0000 7fc216f38700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21019d290 con 0x7fc210102700 2024-04-23T02:49:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.419+0000 7fc216f38700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc210196740 con 0x7fc210101de0 2024-04-23T02:49:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.419+0000 7fc20ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc210102700 0x7fc21019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:12.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.419+0000 7fc20ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc210102700 0x7fc21019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49802/0 (socket says 172.21.15.71:49802) 2024-04-23T02:49:12.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.419+0000 7fc20ffff700 1 -- 172.21.15.71:0/1743142836 learned_addr learned my addr 172.21.15.71:0/1743142836 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:49:12.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.419+0000 7fc214cd4700 1 --2- 172.21.15.71:0/1743142836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc210101de0 0x7fc21019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:12.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.419+0000 7fc20ffff700 1 -- 172.21.15.71:0/1743142836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc210101de0 msgr2=0x7fc21019c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:12.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.419+0000 7fc20ffff700 1 --2- 172.21.15.71:0/1743142836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc210101de0 0x7fc21019c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:12.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.419+0000 7fc20ffff700 1 -- 172.21.15.71:0/1743142836 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc0095f0 con 0x7fc210102700 2024-04-23T02:49:12.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.419+0000 7fc20ffff700 1 --2- 172.21.15.71:0/1743142836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc210102700 0x7fc21019cc00 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0092e0 tx=0x7fc1fc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:12.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.419+0000 7fc20dffb700 1 -- 172.21.15.71:0/1743142836 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1fc01d070 con 0x7fc210102700 2024-04-23T02:49:12.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.419+0000 7fc20dffb700 1 -- 172.21.15.71:0/1743142836 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc1fc003d30 con 0x7fc210102700 2024-04-23T02:49:12.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.419+0000 7fc216f38700 1 -- 172.21.15.71:0/1743142836 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2101969c0 con 0x7fc210102700 2024-04-23T02:49:12.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.420+0000 7fc20dffb700 1 -- 172.21.15.71:0/1743142836 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1fc021410 con 0x7fc210102700 2024-04-23T02:49:12.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.420+0000 7fc216f38700 1 -- 172.21.15.71:0/1743142836 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc210196eb0 con 0x7fc210102700 2024-04-23T02:49:12.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.421+0000 7fc216f38700 1 -- 172.21.15.71:0/1743142836 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2101082f0 con 0x7fc210102700 2024-04-23T02:49:12.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.421+0000 7fc20dffb700 1 -- 172.21.15.71:0/1743142836 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc1fc003890 con 0x7fc210102700 2024-04-23T02:49:12.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.421+0000 7fc20dffb700 1 --2- 172.21.15.71:0/1743142836 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc20006cce0 0x7fc20006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:12.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.422+0000 7fc20dffb700 1 -- 172.21.15.71:0/1743142836 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc1fc08d820 con 0x7fc210102700 2024-04-23T02:49:12.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.422+0000 7fc214cd4700 1 --2- 172.21.15.71:0/1743142836 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc20006cce0 0x7fc20006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:12.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.422+0000 7fc214cd4700 1 --2- 172.21.15.71:0/1743142836 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc20006cce0 0x7fc20006f1a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc204009a10 tx=0x7fc204009190 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:12.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.426+0000 7fc20dffb700 1 -- 172.21.15.71:0/1743142836 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc1fc05b2c0 con 0x7fc210102700 2024-04-23T02:49:12.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.711+0000 7fc216f38700 1 -- 172.21.15.71:0/1743142836 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc210197c80 con 0x7fc210102700 2024-04-23T02:49:12.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.712+0000 7fc20dffb700 1 -- 172.21.15.71:0/1743142836 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fc1fc05ae50 con 0x7fc210102700 2024-04-23T02:49:12.716 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:49:12.716 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:49:12.716 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:49:12.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:49:12.716 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:49:12.716 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:49:12.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:49:12.716 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:49:12.716 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:49:12.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:49:12.716 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:49:12.717 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:49:12.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:49:12.717 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:49:12.717 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:49:12.717 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:49:12.717 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:49:12.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.714+0000 7fc216f38700 1 -- 172.21.15.71:0/1743142836 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc20006cce0 msgr2=0x7fc20006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:12.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.714+0000 7fc216f38700 1 --2- 172.21.15.71:0/1743142836 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc20006cce0 0x7fc20006f1a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc204009a10 tx=0x7fc204009190 comp rx=0 tx=0).stop 2024-04-23T02:49:12.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.714+0000 7fc216f38700 1 -- 172.21.15.71:0/1743142836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc210102700 msgr2=0x7fc21019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:12.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.714+0000 7fc216f38700 1 --2- 172.21.15.71:0/1743142836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc210102700 0x7fc21019cc00 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0092e0 tx=0x7fc1fc00bd70 comp rx=0 tx=0).stop 2024-04-23T02:49:12.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.714+0000 7fc216f38700 1 -- 172.21.15.71:0/1743142836 shutdown_connections 2024-04-23T02:49:12.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.714+0000 7fc216f38700 1 --2- 172.21.15.71:0/1743142836 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc20006cce0 0x7fc20006f1a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:12.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.714+0000 7fc216f38700 1 --2- 172.21.15.71:0/1743142836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc210101de0 0x7fc21019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:12.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.714+0000 7fc216f38700 1 --2- 172.21.15.71:0/1743142836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc210102700 0x7fc21019cc00 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:12.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.714+0000 7fc216f38700 1 -- 172.21.15.71:0/1743142836 >> 172.21.15.71:0/1743142836 conn(0x7fc2100fb610 msgr2=0x7fc210105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:12.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.714+0000 7fc216f38700 1 -- 172.21.15.71:0/1743142836 shutdown_connections 2024-04-23T02:49:12.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.714+0000 7fc216f38700 1 -- 172.21.15.71:0/1743142836 wait complete. 2024-04-23T02:49:12.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.815+0000 7f40e74e1700 1 -- 172.21.15.71:0/3792808415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40e0101ea0 msgr2=0x7f40e0102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:12.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.815+0000 7f40e74e1700 1 --2- 172.21.15.71:0/3792808415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40e0101ea0 0x7f40e0102280 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009910 tx=0x7f40dc009c20 comp rx=0 tx=0).stop 2024-04-23T02:49:12.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.815+0000 7f40e74e1700 1 -- 172.21.15.71:0/3792808415 shutdown_connections 2024-04-23T02:49:12.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.815+0000 7f40e74e1700 1 --2- 172.21.15.71:0/3792808415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40e01027c0 0x7f40e010acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:12.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.815+0000 7f40e74e1700 1 --2- 172.21.15.71:0/3792808415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40e0101ea0 0x7f40e0102280 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:12.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.815+0000 7f40e74e1700 1 -- 172.21.15.71:0/3792808415 >> 172.21.15.71:0/3792808415 conn(0x7f40e00fb650 msgr2=0x7f40e00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:12.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.816+0000 7f40e74e1700 1 -- 172.21.15.71:0/3792808415 shutdown_connections 2024-04-23T02:49:12.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.816+0000 7f40e74e1700 1 -- 172.21.15.71:0/3792808415 wait complete. 2024-04-23T02:49:12.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.816+0000 7f40e74e1700 1 Processor -- start 2024-04-23T02:49:12.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.816+0000 7f40e74e1700 1 -- start start 2024-04-23T02:49:12.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.816+0000 7f40e74e1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40e0101ea0 0x7f40e00ffb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:12.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.816+0000 7f40e74e1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40e01027c0 0x7f40e0100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:12.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.816+0000 7f40e74e1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e0100580 con 0x7f40e01027c0 2024-04-23T02:49:12.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.816+0000 7f40e74e1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e01006c0 con 0x7f40e0101ea0 2024-04-23T02:49:12.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.816+0000 7f40e4a7c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40e01027c0 0x7f40e0100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:12.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.816+0000 7f40e4a7c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40e01027c0 0x7f40e0100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49826/0 (socket says 172.21.15.71:49826) 2024-04-23T02:49:12.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.816+0000 7f40e4a7c700 1 -- 172.21.15.71:0/2471476749 learned_addr learned my addr 172.21.15.71:0/2471476749 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:49:12.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.816+0000 7f40e527d700 1 --2- 172.21.15.71:0/2471476749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40e0101ea0 0x7f40e00ffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:12.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.817+0000 7f40e4a7c700 1 -- 172.21.15.71:0/2471476749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40e0101ea0 msgr2=0x7f40e00ffb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:12.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.817+0000 7f40e4a7c700 1 --2- 172.21.15.71:0/2471476749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40e0101ea0 0x7f40e00ffb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:12.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.817+0000 7f40e4a7c700 1 -- 172.21.15.71:0/2471476749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40dc0095f0 con 0x7f40e01027c0 2024-04-23T02:49:12.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.817+0000 7f40e4a7c700 1 --2- 172.21.15.71:0/2471476749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40e01027c0 0x7f40e0100040 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f40d000e8f0 tx=0x7f40d000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:12.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.817+0000 7f40d67fc700 1 -- 172.21.15.71:0/2471476749 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40d0009790 con 0x7f40e01027c0 2024-04-23T02:49:12.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.817+0000 7f40d67fc700 1 -- 172.21.15.71:0/2471476749 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f40d000c810 con 0x7f40e01027c0 2024-04-23T02:49:12.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.817+0000 7f40e74e1700 1 -- 172.21.15.71:0/2471476749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e01a2970 con 0x7f40e01027c0 2024-04-23T02:49:12.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.817+0000 7f40d67fc700 1 -- 172.21.15.71:0/2471476749 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40d0005520 con 0x7f40e01027c0 2024-04-23T02:49:12.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.817+0000 7f40e74e1700 1 -- 172.21.15.71:0/2471476749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e01a2d80 con 0x7f40e01027c0 2024-04-23T02:49:12.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.818+0000 7f40e74e1700 1 -- 172.21.15.71:0/2471476749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e01083b0 con 0x7f40e01027c0 2024-04-23T02:49:12.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.818+0000 7f40d67fc700 1 -- 172.21.15.71:0/2471476749 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f40d000c980 con 0x7f40e01027c0 2024-04-23T02:49:12.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.819+0000 7f40d67fc700 1 --2- 172.21.15.71:0/2471476749 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f40cc06cce0 0x7f40cc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:12.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.819+0000 7f40e527d700 1 --2- 172.21.15.71:0/2471476749 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f40cc06cce0 0x7f40cc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:12.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.819+0000 7f40d67fc700 1 -- 172.21.15.71:0/2471476749 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f40d0014070 con 0x7f40e01027c0 2024-04-23T02:49:12.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.819+0000 7f40e527d700 1 --2- 172.21.15.71:0/2471476749 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f40cc06cce0 0x7f40cc06f1a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f40dc000c00 tx=0x7f40dc009f70 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:12.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:12.824+0000 7f40d67fc700 1 -- 172.21.15.71:0/2471476749 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f40d0056920 con 0x7f40e01027c0 2024-04-23T02:49:12.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:12 smithi071 ceph-mon[144544]: from='client.35744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:49:12.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:12 smithi071 ceph-mon[144544]: from='client.35748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:49:12.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:12 smithi071 ceph-mon[144544]: from='client.35752 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:49:12.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:12 smithi071 ceph-mon[144544]: pgmap v1078: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:13.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.063+0000 7f40e74e1700 1 -- 172.21.15.71:0/2471476749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f40e004f2e0 con 0x7f40e01027c0 2024-04-23T02:49:13.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.063+0000 7f40d67fc700 1 -- 172.21.15.71:0/2471476749 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f40d000cc30 con 0x7f40e01027c0 2024-04-23T02:49:13.067 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:49:13.067 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:49:13.067 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:49:13.067 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:49:13.067 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:49:13.068 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:49:13.068 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:49:13.068 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:49:13.068 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:49:13.068 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:49:13.068 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:49:13.068 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:49:13.068 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:49:13.068 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:49:13.068 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:49:13.068 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:49:13.068 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:49:13.068 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:49:13.068 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:49:13.068 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:49:13.069 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:49:13.069 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:49:13.069 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:49:13.069 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:49:13.069 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:49:13.069 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:49:13.069 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:49:13.069 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:49:13.069 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:49:13.069 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:49:13.069 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:49:13.069 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:49:13.069 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:49:13.069 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:49:13.069 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:49:13.070 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:49:13.070 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:49:13.070 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:49:13.070 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:49:13.070 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:49:13.070 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:49:13.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.065+0000 7f40e74e1700 1 -- 172.21.15.71:0/2471476749 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f40cc06cce0 msgr2=0x7f40cc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:13.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.065+0000 7f40e74e1700 1 --2- 172.21.15.71:0/2471476749 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f40cc06cce0 0x7f40cc06f1a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f40dc000c00 tx=0x7f40dc009f70 comp rx=0 tx=0).stop 2024-04-23T02:49:13.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.065+0000 7f40e74e1700 1 -- 172.21.15.71:0/2471476749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40e01027c0 msgr2=0x7f40e0100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:13.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.065+0000 7f40e74e1700 1 --2- 172.21.15.71:0/2471476749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40e01027c0 0x7f40e0100040 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f40d000e8f0 tx=0x7f40d000ec00 comp rx=0 tx=0).stop 2024-04-23T02:49:13.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.065+0000 7f40e74e1700 1 -- 172.21.15.71:0/2471476749 shutdown_connections 2024-04-23T02:49:13.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.065+0000 7f40e74e1700 1 --2- 172.21.15.71:0/2471476749 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f40cc06cce0 0x7f40cc06f1a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:13.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.065+0000 7f40e74e1700 1 --2- 172.21.15.71:0/2471476749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40e0101ea0 0x7f40e00ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:13.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.065+0000 7f40e74e1700 1 --2- 172.21.15.71:0/2471476749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40e01027c0 0x7f40e0100040 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:13.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.065+0000 7f40e74e1700 1 -- 172.21.15.71:0/2471476749 >> 172.21.15.71:0/2471476749 conn(0x7f40e00fb650 msgr2=0x7f40e01054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:13.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.065+0000 7f40e74e1700 1 -- 172.21.15.71:0/2471476749 shutdown_connections 2024-04-23T02:49:13.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.065+0000 7f40e74e1700 1 -- 172.21.15.71:0/2471476749 wait complete. 2024-04-23T02:49:13.072 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:49:13.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:12 smithi192 ceph-mon[108617]: from='client.35744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:49:13.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:12 smithi192 ceph-mon[108617]: from='client.35748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:49:13.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:12 smithi192 ceph-mon[108617]: from='client.35752 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:49:13.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:12 smithi192 ceph-mon[108617]: pgmap v1078: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:13.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.168+0000 7f425bac5700 1 -- 172.21.15.71:0/4203902818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4254105bd0 msgr2=0x7f4254105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:13.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.169+0000 7f425bac5700 1 --2- 172.21.15.71:0/4203902818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4254105bd0 0x7f4254105fb0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f4244009910 tx=0x7f4244009c20 comp rx=0 tx=0).stop 2024-04-23T02:49:13.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.169+0000 7f425bac5700 1 -- 172.21.15.71:0/4203902818 shutdown_connections 2024-04-23T02:49:13.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.169+0000 7f425bac5700 1 --2- 172.21.15.71:0/4203902818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42540ffc20 0x7f4254100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:13.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.169+0000 7f425bac5700 1 --2- 172.21.15.71:0/4203902818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4254105bd0 0x7f4254105fb0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:13.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.169+0000 7f425bac5700 1 -- 172.21.15.71:0/4203902818 >> 172.21.15.71:0/4203902818 conn(0x7f4254075670 msgr2=0x7f4254075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:13.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.169+0000 7f425bac5700 1 -- 172.21.15.71:0/4203902818 shutdown_connections 2024-04-23T02:49:13.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.169+0000 7f425bac5700 1 -- 172.21.15.71:0/4203902818 wait complete. 2024-04-23T02:49:13.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.169+0000 7f425bac5700 1 Processor -- start 2024-04-23T02:49:13.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.169+0000 7f425bac5700 1 -- start start 2024-04-23T02:49:13.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.169+0000 7f425bac5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42540ffc20 0x7f4254197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:13.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.169+0000 7f425bac5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4254105bd0 0x7f4254197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:13.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.170+0000 7f425bac5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4254198160 con 0x7f42540ffc20 2024-04-23T02:49:13.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.170+0000 7f425bac5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42541982a0 con 0x7f4254105bd0 2024-04-23T02:49:13.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.170+0000 7f4259861700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42540ffc20 0x7f4254197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:13.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.170+0000 7f4259861700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42540ffc20 0x7f4254197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49842/0 (socket says 172.21.15.71:49842) 2024-04-23T02:49:13.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.170+0000 7f4259861700 1 -- 172.21.15.71:0/643063524 learned_addr learned my addr 172.21.15.71:0/643063524 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:49:13.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.170+0000 7f4259060700 1 --2- 172.21.15.71:0/643063524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4254105bd0 0x7f4254197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:13.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.170+0000 7f4259861700 1 -- 172.21.15.71:0/643063524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4254105bd0 msgr2=0x7f4254197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:13.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.170+0000 7f4259861700 1 --2- 172.21.15.71:0/643063524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4254105bd0 0x7f4254197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:13.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.170+0000 7f4259861700 1 -- 172.21.15.71:0/643063524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42440095f0 con 0x7f42540ffc20 2024-04-23T02:49:13.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.170+0000 7f4259060700 1 --2- 172.21.15.71:0/643063524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4254105bd0 0x7f4254197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:49:13.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.170+0000 7f4259861700 1 --2- 172.21.15.71:0/643063524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42540ffc20 0x7f4254197600 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4244006010 tx=0x7f424400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:13.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.170+0000 7f424affd700 1 -- 172.21.15.71:0/643063524 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f424401d070 con 0x7f42540ffc20 2024-04-23T02:49:13.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.170+0000 7f425bac5700 1 -- 172.21.15.71:0/643063524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42541a0670 con 0x7f42540ffc20 2024-04-23T02:49:13.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.170+0000 7f424affd700 1 -- 172.21.15.71:0/643063524 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f424400f460 con 0x7f42540ffc20 2024-04-23T02:49:13.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.170+0000 7f424affd700 1 -- 172.21.15.71:0/643063524 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4244021640 con 0x7f42540ffc20 2024-04-23T02:49:13.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.170+0000 7f425bac5700 1 -- 172.21.15.71:0/643063524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42541a0a80 con 0x7f42540ffc20 2024-04-23T02:49:13.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.171+0000 7f425bac5700 1 -- 172.21.15.71:0/643063524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42541080f0 con 0x7f42540ffc20 2024-04-23T02:49:13.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.172+0000 7f424affd700 1 -- 172.21.15.71:0/643063524 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f424400fa20 con 0x7f42540ffc20 2024-04-23T02:49:13.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.172+0000 7f424affd700 1 --2- 172.21.15.71:0/643063524 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f424006cce0 0x7f424006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:13.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.172+0000 7f424affd700 1 -- 172.21.15.71:0/643063524 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f424408d830 con 0x7f42540ffc20 2024-04-23T02:49:13.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.172+0000 7f4259060700 1 --2- 172.21.15.71:0/643063524 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f424006cce0 0x7f424006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:13.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.173+0000 7f4259060700 1 --2- 172.21.15.71:0/643063524 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f424006cce0 0x7f424006f1a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4250005950 tx=0x7f42500058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:13.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.177+0000 7f424affd700 1 -- 172.21.15.71:0/643063524 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f42440578a0 con 0x7f42540ffc20 2024-04-23T02:49:13.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.385+0000 7f425bac5700 1 -- 172.21.15.71:0/643063524 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f42540611d0 con 0x7f424006cce0 2024-04-23T02:49:13.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.386+0000 7f424affd700 1 -- 172.21.15.71:0/643063524 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f42540611d0 con 0x7f424006cce0 2024-04-23T02:49:13.390 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:49:13.390 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:49:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:49:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:49:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:49:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:49:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:49:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:49:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:49:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:49:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:49:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:49:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:49:13.391 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:49:13.391 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:49:13.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.388+0000 7f425bac5700 1 -- 172.21.15.71:0/643063524 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f424006cce0 msgr2=0x7f424006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:13.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.388+0000 7f425bac5700 1 --2- 172.21.15.71:0/643063524 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f424006cce0 0x7f424006f1a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4250005950 tx=0x7f42500058e0 comp rx=0 tx=0).stop 2024-04-23T02:49:13.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.388+0000 7f425bac5700 1 -- 172.21.15.71:0/643063524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42540ffc20 msgr2=0x7f4254197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:13.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.388+0000 7f425bac5700 1 --2- 172.21.15.71:0/643063524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42540ffc20 0x7f4254197600 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4244006010 tx=0x7f424400be20 comp rx=0 tx=0).stop 2024-04-23T02:49:13.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.388+0000 7f425bac5700 1 -- 172.21.15.71:0/643063524 shutdown_connections 2024-04-23T02:49:13.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.388+0000 7f425bac5700 1 --2- 172.21.15.71:0/643063524 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f424006cce0 0x7f424006f1a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:13.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.388+0000 7f425bac5700 1 --2- 172.21.15.71:0/643063524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42540ffc20 0x7f4254197600 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:13.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.388+0000 7f425bac5700 1 --2- 172.21.15.71:0/643063524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4254105bd0 0x7f4254197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:13.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.388+0000 7f425bac5700 1 -- 172.21.15.71:0/643063524 >> 172.21.15.71:0/643063524 conn(0x7f4254075670 msgr2=0x7f42540fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:13.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.389+0000 7f425bac5700 1 -- 172.21.15.71:0/643063524 shutdown_connections 2024-04-23T02:49:13.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.389+0000 7f425bac5700 1 -- 172.21.15.71:0/643063524 wait complete. 2024-04-23T02:49:13.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.492+0000 7fd6c0dab700 1 -- 172.21.15.71:0/2234423062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6bc0ffda0 msgr2=0x7fd6bc100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:13.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.492+0000 7fd6c0dab700 1 --2- 172.21.15.71:0/2234423062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6bc0ffda0 0x7fd6bc100180 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009910 tx=0x7fd6b0009c20 comp rx=0 tx=0).stop 2024-04-23T02:49:13.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.492+0000 7fd6c0dab700 1 -- 172.21.15.71:0/2234423062 shutdown_connections 2024-04-23T02:49:13.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.492+0000 7fd6c0dab700 1 --2- 172.21.15.71:0/2234423062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6bc100750 0x7fd6bc1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:13.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.492+0000 7fd6c0dab700 1 --2- 172.21.15.71:0/2234423062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6bc0ffda0 0x7fd6bc100180 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:13.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.492+0000 7fd6c0dab700 1 -- 172.21.15.71:0/2234423062 >> 172.21.15.71:0/2234423062 conn(0x7fd6bc0fb650 msgr2=0x7fd6bc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:13.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.492+0000 7fd6c0dab700 1 -- 172.21.15.71:0/2234423062 shutdown_connections 2024-04-23T02:49:13.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.492+0000 7fd6c0dab700 1 -- 172.21.15.71:0/2234423062 wait complete. 2024-04-23T02:49:13.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.493+0000 7fd6c0dab700 1 Processor -- start 2024-04-23T02:49:13.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.493+0000 7fd6c0dab700 1 -- start start 2024-04-23T02:49:13.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.493+0000 7fd6c0dab700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6bc0ffda0 0x7fd6bc197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:13.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.493+0000 7fd6c0dab700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6bc100750 0x7fd6bc197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:13.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.493+0000 7fd6c0dab700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc198190 con 0x7fd6bc100750 2024-04-23T02:49:13.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.493+0000 7fd6c0dab700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc1982d0 con 0x7fd6bc0ffda0 2024-04-23T02:49:13.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.493+0000 7fd6b9d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6bc100750 0x7fd6bc197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:13.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.493+0000 7fd6b9d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6bc100750 0x7fd6bc197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49860/0 (socket says 172.21.15.71:49860) 2024-04-23T02:49:13.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.493+0000 7fd6b9d9b700 1 -- 172.21.15.71:0/271591159 learned_addr learned my addr 172.21.15.71:0/271591159 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:49:13.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.493+0000 7fd6ba59c700 1 --2- 172.21.15.71:0/271591159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6bc0ffda0 0x7fd6bc197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:13.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.493+0000 7fd6b9d9b700 1 -- 172.21.15.71:0/271591159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6bc0ffda0 msgr2=0x7fd6bc197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:13.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.493+0000 7fd6b9d9b700 1 --2- 172.21.15.71:0/271591159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6bc0ffda0 0x7fd6bc197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:13.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.493+0000 7fd6b9d9b700 1 -- 172.21.15.71:0/271591159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b00095f0 con 0x7fd6bc100750 2024-04-23T02:49:13.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.493+0000 7fd6b9d9b700 1 --2- 172.21.15.71:0/271591159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6bc100750 0x7fd6bc197b70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00e8f0 tx=0x7fd6ac00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:13.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.494+0000 7fd6ab7fe700 1 -- 172.21.15.71:0/271591159 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6ac009790 con 0x7fd6bc100750 2024-04-23T02:49:13.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.494+0000 7fd6c0dab700 1 -- 172.21.15.71:0/271591159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6bc1a06a0 con 0x7fd6bc100750 2024-04-23T02:49:13.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.494+0000 7fd6ab7fe700 1 -- 172.21.15.71:0/271591159 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd6ac00c810 con 0x7fd6bc100750 2024-04-23T02:49:13.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.494+0000 7fd6ab7fe700 1 -- 172.21.15.71:0/271591159 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6ac005520 con 0x7fd6bc100750 2024-04-23T02:49:13.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.494+0000 7fd6c0dab700 1 -- 172.21.15.71:0/271591159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6bc1a0b60 con 0x7fd6bc100750 2024-04-23T02:49:13.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.495+0000 7fd6c0dab700 1 -- 172.21.15.71:0/271591159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6bc108140 con 0x7fd6bc100750 2024-04-23T02:49:13.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.495+0000 7fd6ab7fe700 1 -- 172.21.15.71:0/271591159 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd6ac00c980 con 0x7fd6bc100750 2024-04-23T02:49:13.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.495+0000 7fd6ab7fe700 1 --2- 172.21.15.71:0/271591159 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6a406cce0 0x7fd6a406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:13.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.495+0000 7fd6ab7fe700 1 -- 172.21.15.71:0/271591159 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd6ac014070 con 0x7fd6bc100750 2024-04-23T02:49:13.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.495+0000 7fd6ba59c700 1 --2- 172.21.15.71:0/271591159 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6a406cce0 0x7fd6a406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:13.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.496+0000 7fd6ba59c700 1 --2- 172.21.15.71:0/271591159 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6a406cce0 0x7fd6a406f1a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000b6e0 tx=0x7fd6b0005a90 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:13.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.500+0000 7fd6ab7fe700 1 -- 172.21.15.71:0/271591159 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd6ac056920 con 0x7fd6bc100750 2024-04-23T02:49:13.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.775+0000 7fd6c0dab700 1 -- 172.21.15.71:0/271591159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd6bc04f2e0 con 0x7fd6bc100750 2024-04-23T02:49:13.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.775+0000 7fd6ab7fe700 1 -- 172.21.15.71:0/271591159 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7fd6ac00cc30 con 0x7fd6bc100750 2024-04-23T02:49:13.779 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:49:13.780 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:49:13.780 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:49:13.780 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:49:13.780 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:49:13.780 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:49:13.780 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:49:13.780 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:49:13.780 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:49:13.780 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:49:13.780 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:49:13.780 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:49:13.780 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:49:13.780 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:49:13.781 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:49:13.781 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:49:13.781 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:49:13.781 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:49:13.781 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:49:13.781 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:49:13.781 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:49:13.781 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:49:13.781 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:49:13.781 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:49:13.781 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:49:13.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.777+0000 7fd6c0dab700 1 -- 172.21.15.71:0/271591159 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6a406cce0 msgr2=0x7fd6a406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:13.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.777+0000 7fd6c0dab700 1 --2- 172.21.15.71:0/271591159 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6a406cce0 0x7fd6a406f1a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000b6e0 tx=0x7fd6b0005a90 comp rx=0 tx=0).stop 2024-04-23T02:49:13.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.777+0000 7fd6c0dab700 1 -- 172.21.15.71:0/271591159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6bc100750 msgr2=0x7fd6bc197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:13.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.777+0000 7fd6c0dab700 1 --2- 172.21.15.71:0/271591159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6bc100750 0x7fd6bc197b70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00e8f0 tx=0x7fd6ac00ec00 comp rx=0 tx=0).stop 2024-04-23T02:49:13.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.777+0000 7fd6c0dab700 1 -- 172.21.15.71:0/271591159 shutdown_connections 2024-04-23T02:49:13.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.777+0000 7fd6c0dab700 1 --2- 172.21.15.71:0/271591159 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd6a406cce0 0x7fd6a406f1a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:13.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.777+0000 7fd6c0dab700 1 --2- 172.21.15.71:0/271591159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6bc0ffda0 0x7fd6bc197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:13.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.777+0000 7fd6c0dab700 1 --2- 172.21.15.71:0/271591159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6bc100750 0x7fd6bc197b70 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:13.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.777+0000 7fd6c0dab700 1 -- 172.21.15.71:0/271591159 >> 172.21.15.71:0/271591159 conn(0x7fd6bc0fb650 msgr2=0x7fd6bc0fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:13.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.777+0000 7fd6c0dab700 1 -- 172.21.15.71:0/271591159 shutdown_connections 2024-04-23T02:49:13.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:13.777+0000 7fd6c0dab700 1 -- 172.21.15.71:0/271591159 wait complete. 2024-04-23T02:49:13.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:13 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1743142836' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:49:13.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:13 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2471476749' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:49:13.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:13 smithi071 ceph-mon[144544]: from='client.35764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:49:14.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:13 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1743142836' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:49:14.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:13 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2471476749' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:49:14.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:13 smithi192 ceph-mon[108617]: from='client.35764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:49:14.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:14 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/271591159' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:49:14.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:14 smithi071 ceph-mon[144544]: pgmap v1079: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:15.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:14 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/271591159' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:49:15.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:14 smithi192 ceph-mon[108617]: pgmap v1079: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:16.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:16 smithi071 ceph-mon[144544]: pgmap v1080: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:17.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:16 smithi192 ceph-mon[108617]: pgmap v1080: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:18.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:18 smithi071 ceph-mon[144544]: pgmap v1081: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:19.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:18 smithi192 ceph-mon[108617]: pgmap v1081: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:20.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:49:20.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:49:21.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:20 smithi192 ceph-mon[108617]: pgmap v1082: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:21.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:20 smithi071 ceph-mon[144544]: pgmap v1082: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:22.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:22 smithi071 ceph-mon[144544]: pgmap v1083: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:23.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:22 smithi192 ceph-mon[108617]: pgmap v1083: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:24.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:24 smithi071 ceph-mon[144544]: pgmap v1084: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:25.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:24 smithi192 ceph-mon[108617]: pgmap v1084: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:26.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:26 smithi071 ceph-mon[144544]: pgmap v1085: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:27.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:26 smithi192 ceph-mon[108617]: pgmap v1085: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:28.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:28 smithi071 ceph-mon[144544]: pgmap v1086: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:29.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:28 smithi192 ceph-mon[108617]: pgmap v1086: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:30.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:30 smithi071 ceph-mon[144544]: pgmap v1087: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:31.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:30 smithi192 ceph-mon[108617]: pgmap v1087: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:32.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:32 smithi071 ceph-mon[144544]: pgmap v1088: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:33.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:32 smithi192 ceph-mon[108617]: pgmap v1088: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:34.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:34 smithi071 ceph-mon[144544]: pgmap v1089: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:35.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:34 smithi192 ceph-mon[108617]: pgmap v1089: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:35.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:49:36.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:49:36.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:36 smithi071 ceph-mon[144544]: pgmap v1090: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:37.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:36 smithi192 ceph-mon[108617]: pgmap v1090: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:38.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:38 smithi071 ceph-mon[144544]: pgmap v1091: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:39.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:38 smithi192 ceph-mon[108617]: pgmap v1091: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:40.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:40 smithi071 ceph-mon[144544]: pgmap v1092: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:41.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:40 smithi192 ceph-mon[108617]: pgmap v1092: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:42.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:42 smithi192 ceph-mon[108617]: pgmap v1093: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:42.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:42 smithi071 ceph-mon[144544]: pgmap v1093: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:43.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.895+0000 7efffdd49700 1 -- 172.21.15.71:0/3532834396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efff8100e90 msgr2=0x7efff8101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:43.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.895+0000 7efffdd49700 1 --2- 172.21.15.71:0/3532834396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efff8100e90 0x7efff8101270 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7effe8009910 tx=0x7effe8009c20 comp rx=0 tx=0).stop 2024-04-23T02:49:43.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.895+0000 7efffdd49700 1 -- 172.21.15.71:0/3532834396 shutdown_connections 2024-04-23T02:49:43.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.895+0000 7efffdd49700 1 --2- 172.21.15.71:0/3532834396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efff8101840 0x7efff8105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:43.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.895+0000 7efffdd49700 1 --2- 172.21.15.71:0/3532834396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efff8100e90 0x7efff8101270 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:43.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.895+0000 7efffdd49700 1 -- 172.21.15.71:0/3532834396 >> 172.21.15.71:0/3532834396 conn(0x7efff80fc740 msgr2=0x7efff80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:43.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.895+0000 7efffdd49700 1 -- 172.21.15.71:0/3532834396 shutdown_connections 2024-04-23T02:49:43.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.895+0000 7efffdd49700 1 -- 172.21.15.71:0/3532834396 wait complete. 2024-04-23T02:49:43.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.896+0000 7efffdd49700 1 Processor -- start 2024-04-23T02:49:43.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.896+0000 7efffdd49700 1 -- start start 2024-04-23T02:49:43.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.896+0000 7efffdd49700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efff8100e90 0x7efff819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:43.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.896+0000 7efffdd49700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efff8101840 0x7efff819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:43.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.896+0000 7efffdd49700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff819d190 con 0x7efff8101840 2024-04-23T02:49:43.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.896+0000 7efffdd49700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff8196640 con 0x7efff8100e90 2024-04-23T02:49:43.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.896+0000 7efff6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efff8101840 0x7efff819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:43.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.896+0000 7efff6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efff8101840 0x7efff819cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52250/0 (socket says 172.21.15.71:52250) 2024-04-23T02:49:43.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.896+0000 7efff6ffd700 1 -- 172.21.15.71:0/3162816280 learned_addr learned my addr 172.21.15.71:0/3162816280 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:49:43.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.897+0000 7efff77fe700 1 --2- 172.21.15.71:0/3162816280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efff8100e90 0x7efff819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:43.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.897+0000 7efff6ffd700 1 -- 172.21.15.71:0/3162816280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efff8100e90 msgr2=0x7efff819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:43.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.897+0000 7efff6ffd700 1 --2- 172.21.15.71:0/3162816280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efff8100e90 0x7efff819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:43.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.897+0000 7efff6ffd700 1 -- 172.21.15.71:0/3162816280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe80095f0 con 0x7efff8101840 2024-04-23T02:49:43.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.897+0000 7efff6ffd700 1 --2- 172.21.15.71:0/3162816280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efff8101840 0x7efff819cb00 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7effe000d900 tx=0x7effe000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:43.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.897+0000 7efff4ff9700 1 -- 172.21.15.71:0/3162816280 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effe0009900 con 0x7efff8101840 2024-04-23T02:49:43.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.897+0000 7efff4ff9700 1 -- 172.21.15.71:0/3162816280 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7effe0010460 con 0x7efff8101840 2024-04-23T02:49:43.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.897+0000 7efffdd49700 1 -- 172.21.15.71:0/3162816280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff8196920 con 0x7efff8101840 2024-04-23T02:49:43.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.897+0000 7efff4ff9700 1 -- 172.21.15.71:0/3162816280 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effe000f5f0 con 0x7efff8101840 2024-04-23T02:49:43.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.897+0000 7efffdd49700 1 -- 172.21.15.71:0/3162816280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff8196e70 con 0x7efff8101840 2024-04-23T02:49:43.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.898+0000 7efffdd49700 1 -- 172.21.15.71:0/3162816280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff8109230 con 0x7efff8101840 2024-04-23T02:49:43.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.898+0000 7efff4ff9700 1 -- 172.21.15.71:0/3162816280 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7effe00105d0 con 0x7efff8101840 2024-04-23T02:49:43.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.899+0000 7efff4ff9700 1 --2- 172.21.15.71:0/3162816280 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7effe406cce0 0x7effe406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:43.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.899+0000 7efff4ff9700 1 -- 172.21.15.71:0/3162816280 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7effe008c310 con 0x7efff8101840 2024-04-23T02:49:43.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.899+0000 7efff77fe700 1 --2- 172.21.15.71:0/3162816280 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7effe406cce0 0x7effe406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:43.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.900+0000 7efff77fe700 1 --2- 172.21.15.71:0/3162816280 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7effe406cce0 0x7effe406f1a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7effe8006010 tx=0x7effe801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:43.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:43.904+0000 7efff4ff9700 1 -- 172.21.15.71:0/3162816280 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7effe0059e90 con 0x7efff8101840 2024-04-23T02:49:44.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.115+0000 7efffdd49700 1 -- 172.21.15.71:0/3162816280 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efff81970e0 con 0x7effe406cce0 2024-04-23T02:49:44.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.116+0000 7efff4ff9700 1 -- 172.21.15.71:0/3162816280 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7efff81970e0 con 0x7effe406cce0 2024-04-23T02:49:44.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.118+0000 7efffdd49700 1 -- 172.21.15.71:0/3162816280 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7effe406cce0 msgr2=0x7effe406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:44.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.118+0000 7efffdd49700 1 --2- 172.21.15.71:0/3162816280 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7effe406cce0 0x7effe406f1a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7effe8006010 tx=0x7effe801a040 comp rx=0 tx=0).stop 2024-04-23T02:49:44.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.118+0000 7efffdd49700 1 -- 172.21.15.71:0/3162816280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efff8101840 msgr2=0x7efff819cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:44.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.118+0000 7efffdd49700 1 --2- 172.21.15.71:0/3162816280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efff8101840 0x7efff819cb00 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7effe000d900 tx=0x7effe000dcc0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.118+0000 7efffdd49700 1 -- 172.21.15.71:0/3162816280 shutdown_connections 2024-04-23T02:49:44.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.118+0000 7efffdd49700 1 --2- 172.21.15.71:0/3162816280 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7effe406cce0 0x7effe406f1a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.118+0000 7efffdd49700 1 --2- 172.21.15.71:0/3162816280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efff8100e90 0x7efff819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.118+0000 7efffdd49700 1 --2- 172.21.15.71:0/3162816280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efff8101840 0x7efff819cb00 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.118+0000 7efffdd49700 1 -- 172.21.15.71:0/3162816280 >> 172.21.15.71:0/3162816280 conn(0x7efff80fc740 msgr2=0x7efff80fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:44.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.118+0000 7efffdd49700 1 -- 172.21.15.71:0/3162816280 shutdown_connections 2024-04-23T02:49:44.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.119+0000 7efffdd49700 1 -- 172.21.15.71:0/3162816280 wait complete. 2024-04-23T02:49:44.132 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:49:44.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.222+0000 7f6880bcc700 1 -- 172.21.15.71:0/188153012 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f687c0ffb40 msgr2=0x7f687c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:44.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.222+0000 7f6880bcc700 1 --2- 172.21.15.71:0/188153012 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f687c0ffb40 0x7f687c0fffc0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f686c009910 tx=0x7f686c009c20 comp rx=0 tx=0).stop 2024-04-23T02:49:44.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.222+0000 7f6880bcc700 1 -- 172.21.15.71:0/188153012 shutdown_connections 2024-04-23T02:49:44.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.222+0000 7f6880bcc700 1 --2- 172.21.15.71:0/188153012 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f687c0ffb40 0x7f687c0fffc0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.222+0000 7f6880bcc700 1 --2- 172.21.15.71:0/188153012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f687c105bf0 0x7f687c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.222+0000 7f6880bcc700 1 -- 172.21.15.71:0/188153012 >> 172.21.15.71:0/188153012 conn(0x7f687c0fb610 msgr2=0x7f687c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:44.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.222+0000 7f6880bcc700 1 -- 172.21.15.71:0/188153012 shutdown_connections 2024-04-23T02:49:44.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.222+0000 7f6880bcc700 1 -- 172.21.15.71:0/188153012 wait complete. 2024-04-23T02:49:44.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.223+0000 7f6880bcc700 1 Processor -- start 2024-04-23T02:49:44.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.223+0000 7f6880bcc700 1 -- start start 2024-04-23T02:49:44.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.223+0000 7f6880bcc700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f687c0ffb40 0x7f687c195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:44.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.223+0000 7f6880bcc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f687c105bf0 0x7f687c195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:44.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.223+0000 7f6880bcc700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c1961b0 con 0x7f687c105bf0 2024-04-23T02:49:44.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.223+0000 7f6880bcc700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c1962f0 con 0x7f687c0ffb40 2024-04-23T02:49:44.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.223+0000 7f6879d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f687c105bf0 0x7f687c195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:44.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.224+0000 7f6879d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f687c105bf0 0x7f687c195b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52262/0 (socket says 172.21.15.71:52262) 2024-04-23T02:49:44.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.224+0000 7f6879d9b700 1 -- 172.21.15.71:0/3743096879 learned_addr learned my addr 172.21.15.71:0/3743096879 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:49:44.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.224+0000 7f687a59c700 1 --2- 172.21.15.71:0/3743096879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f687c0ffb40 0x7f687c195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:44.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.224+0000 7f6879d9b700 1 -- 172.21.15.71:0/3743096879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f687c0ffb40 msgr2=0x7f687c195620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:44.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.224+0000 7f6879d9b700 1 --2- 172.21.15.71:0/3743096879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f687c0ffb40 0x7f687c195620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.224+0000 7f6879d9b700 1 -- 172.21.15.71:0/3743096879 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c0095f0 con 0x7f687c105bf0 2024-04-23T02:49:44.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.224+0000 7f6879d9b700 1 --2- 172.21.15.71:0/3743096879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f687c105bf0 0x7f687c195b60 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f686c006010 tx=0x7f686c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:44.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.224+0000 7f68737fe700 1 -- 172.21.15.71:0/3743096879 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f686c01d070 con 0x7f687c105bf0 2024-04-23T02:49:44.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.224+0000 7f68737fe700 1 -- 172.21.15.71:0/3743096879 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f686c005940 con 0x7f687c105bf0 2024-04-23T02:49:44.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.224+0000 7f6880bcc700 1 -- 172.21.15.71:0/3743096879 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687c19a0e0 con 0x7f687c105bf0 2024-04-23T02:49:44.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.224+0000 7f68737fe700 1 -- 172.21.15.71:0/3743096879 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f686c021410 con 0x7f687c105bf0 2024-04-23T02:49:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.224+0000 7f6880bcc700 1 -- 172.21.15.71:0/3743096879 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687c19a5d0 con 0x7f687c105bf0 2024-04-23T02:49:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.225+0000 7f6880bcc700 1 -- 172.21.15.71:0/3743096879 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f687c04f2e0 con 0x7f687c105bf0 2024-04-23T02:49:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.226+0000 7f68737fe700 1 -- 172.21.15.71:0/3743096879 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f686c003a60 con 0x7f687c105bf0 2024-04-23T02:49:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.226+0000 7f68737fe700 1 --2- 172.21.15.71:0/3743096879 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f686806cce0 0x7f686806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.226+0000 7f68737fe700 1 -- 172.21.15.71:0/3743096879 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f686c08dc70 con 0x7f687c105bf0 2024-04-23T02:49:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.226+0000 7f687a59c700 1 --2- 172.21.15.71:0/3743096879 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f686806cce0 0x7f686806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.227+0000 7f687a59c700 1 --2- 172.21.15.71:0/3743096879 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f686806cce0 0x7f686806f1a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f686400ac50 tx=0x7f686400a380 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:44.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.230+0000 7f68737fe700 1 -- 172.21.15.71:0/3743096879 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f686c057c40 con 0x7f687c105bf0 2024-04-23T02:49:44.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.439+0000 7f6880bcc700 1 -- 172.21.15.71:0/3743096879 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f687c107c70 con 0x7f686806cce0 2024-04-23T02:49:44.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.441+0000 7f68737fe700 1 -- 172.21.15.71:0/3743096879 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f687c107c70 con 0x7f686806cce0 2024-04-23T02:49:44.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.443+0000 7f6880bcc700 1 -- 172.21.15.71:0/3743096879 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f686806cce0 msgr2=0x7f686806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:44.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.443+0000 7f6880bcc700 1 --2- 172.21.15.71:0/3743096879 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f686806cce0 0x7f686806f1a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f686400ac50 tx=0x7f686400a380 comp rx=0 tx=0).stop 2024-04-23T02:49:44.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.443+0000 7f6880bcc700 1 -- 172.21.15.71:0/3743096879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f687c105bf0 msgr2=0x7f687c195b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:44.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.443+0000 7f6880bcc700 1 --2- 172.21.15.71:0/3743096879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f687c105bf0 0x7f687c195b60 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f686c006010 tx=0x7f686c00bd70 comp rx=0 tx=0).stop 2024-04-23T02:49:44.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.443+0000 7f6880bcc700 1 -- 172.21.15.71:0/3743096879 shutdown_connections 2024-04-23T02:49:44.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.443+0000 7f6880bcc700 1 --2- 172.21.15.71:0/3743096879 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f686806cce0 0x7f686806f1a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.443+0000 7f6880bcc700 1 --2- 172.21.15.71:0/3743096879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f687c0ffb40 0x7f687c195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.443+0000 7f6880bcc700 1 --2- 172.21.15.71:0/3743096879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f687c105bf0 0x7f687c195b60 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.443+0000 7f6880bcc700 1 -- 172.21.15.71:0/3743096879 >> 172.21.15.71:0/3743096879 conn(0x7f687c0fb610 msgr2=0x7f687c0fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:44.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.443+0000 7f6880bcc700 1 -- 172.21.15.71:0/3743096879 shutdown_connections 2024-04-23T02:49:44.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.443+0000 7f6880bcc700 1 -- 172.21.15.71:0/3743096879 wait complete. 2024-04-23T02:49:44.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.545+0000 7f1455052700 1 -- 172.21.15.71:0/1669085681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14500ffda0 msgr2=0x7f1450100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:44.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.545+0000 7f1455052700 1 --2- 172.21.15.71:0/1669085681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14500ffda0 0x7f1450100180 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1444009910 tx=0x7f1444009c20 comp rx=0 tx=0).stop 2024-04-23T02:49:44.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.545+0000 7f1455052700 1 -- 172.21.15.71:0/1669085681 shutdown_connections 2024-04-23T02:49:44.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.545+0000 7f1455052700 1 --2- 172.21.15.71:0/1669085681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1450100750 0x7f14501047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.545+0000 7f1455052700 1 --2- 172.21.15.71:0/1669085681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14500ffda0 0x7f1450100180 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.545+0000 7f1455052700 1 -- 172.21.15.71:0/1669085681 >> 172.21.15.71:0/1669085681 conn(0x7f14500fb650 msgr2=0x7f14500fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:44.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.545+0000 7f1455052700 1 -- 172.21.15.71:0/1669085681 shutdown_connections 2024-04-23T02:49:44.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.545+0000 7f1455052700 1 -- 172.21.15.71:0/1669085681 wait complete. 2024-04-23T02:49:44.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.545+0000 7f1455052700 1 Processor -- start 2024-04-23T02:49:44.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.546+0000 7f1455052700 1 -- start start 2024-04-23T02:49:44.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.546+0000 7f1455052700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14500ffda0 0x7f145019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:44.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.546+0000 7f1455052700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1450100750 0x7f145019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:44.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.546+0000 7f1455052700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1450196670 con 0x7f1450100750 2024-04-23T02:49:44.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.546+0000 7f1455052700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14501967e0 con 0x7f14500ffda0 2024-04-23T02:49:44.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.546+0000 7f144e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1450100750 0x7f145019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:44.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.546+0000 7f144e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1450100750 0x7f145019cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52284/0 (socket says 172.21.15.71:52284) 2024-04-23T02:49:44.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.546+0000 7f144e59c700 1 -- 172.21.15.71:0/969509219 learned_addr learned my addr 172.21.15.71:0/969509219 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:49:44.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.546+0000 7f144ed9d700 1 --2- 172.21.15.71:0/969509219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14500ffda0 0x7f145019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:44.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.546+0000 7f144e59c700 1 -- 172.21.15.71:0/969509219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14500ffda0 msgr2=0x7f145019c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:44.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.546+0000 7f144e59c700 1 --2- 172.21.15.71:0/969509219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14500ffda0 0x7f145019c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.546+0000 7f144e59c700 1 -- 172.21.15.71:0/969509219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14440095f0 con 0x7f1450100750 2024-04-23T02:49:44.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.546+0000 7f144e59c700 1 --2- 172.21.15.71:0/969509219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1450100750 0x7f145019cbc0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f144000e8f0 tx=0x7f144000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:44.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.546+0000 7f1437fff700 1 -- 172.21.15.71:0/969509219 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1440009790 con 0x7f1450100750 2024-04-23T02:49:44.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.546+0000 7f1455052700 1 -- 172.21.15.71:0/969509219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1450196ac0 con 0x7f1450100750 2024-04-23T02:49:44.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.547+0000 7f1437fff700 1 -- 172.21.15.71:0/969509219 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1440004cd0 con 0x7f1450100750 2024-04-23T02:49:44.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.547+0000 7f1437fff700 1 -- 172.21.15.71:0/969509219 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1440005580 con 0x7f1450100750 2024-04-23T02:49:44.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.547+0000 7f1455052700 1 -- 172.21.15.71:0/969509219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1450197010 con 0x7f1450100750 2024-04-23T02:49:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.547+0000 7f1455052700 1 -- 172.21.15.71:0/969509219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f145004f2e0 con 0x7f1450100750 2024-04-23T02:49:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.548+0000 7f1437fff700 1 -- 172.21.15.71:0/969509219 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1440004750 con 0x7f1450100750 2024-04-23T02:49:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.548+0000 7f1437fff700 1 --2- 172.21.15.71:0/969509219 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f143806cce0 0x7f143806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.549+0000 7f144ed9d700 1 --2- 172.21.15.71:0/969509219 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f143806cce0 0x7f143806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.549+0000 7f1437fff700 1 -- 172.21.15.71:0/969509219 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1440014070 con 0x7f1450100750 2024-04-23T02:49:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.549+0000 7f144ed9d700 1 --2- 172.21.15.71:0/969509219 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f143806cce0 0x7f143806f1a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1444009fd0 tx=0x7f1444005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:44.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.553+0000 7f1437fff700 1 -- 172.21.15.71:0/969509219 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f14400569d0 con 0x7f1450100750 2024-04-23T02:49:44.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.756+0000 7f1455052700 1 -- 172.21.15.71:0/969509219 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1450197290 con 0x7f143806cce0 2024-04-23T02:49:44.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.766+0000 7f1437fff700 1 -- 172.21.15.71:0/969509219 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1450197290 con 0x7f143806cce0 2024-04-23T02:49:44.769 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:49:44.769 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (33m) 11m ago 44m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:49:44.769 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (45m) 11m ago 45m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:49:44.770 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (43m) 11m ago 43m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:49:44.770 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (31m) 11m ago 44m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:49:44.770 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (31m) 11m ago 43m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:49:44.770 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (33m) 11m ago 44m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:49:44.770 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (27m) 11m ago 38m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:49:44.770 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (27m) 11m ago 38m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:49:44.770 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (11m) 11m ago 38m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:49:44.770 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (26m) 11m ago 38m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:49:44.770 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (35m) 11m ago 46m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:49:44.770 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (34m) 11m ago 43m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:49:44.770 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (31m) 11m ago 46m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:49:44.770 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (31m) 11m ago 43m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:49:44.770 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (34m) 11m ago 44m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:49:44.770 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (34m) 11m ago 43m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:49:44.770 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (30m) 11m ago 42m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:49:44.771 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (30m) 11m ago 42m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:49:44.771 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (29m) 11m ago 41m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:49:44.771 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (29m) 11m ago 41m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:49:44.771 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (28m) 11m ago 40m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:49:44.771 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (27m) 11m ago 40m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:49:44.771 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (33m) 11m ago 44m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:49:44.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.768+0000 7f1455052700 1 -- 172.21.15.71:0/969509219 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f143806cce0 msgr2=0x7f143806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:44.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.768+0000 7f1455052700 1 --2- 172.21.15.71:0/969509219 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f143806cce0 0x7f143806f1a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1444009fd0 tx=0x7f1444005940 comp rx=0 tx=0).stop 2024-04-23T02:49:44.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.768+0000 7f1455052700 1 -- 172.21.15.71:0/969509219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1450100750 msgr2=0x7f145019cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:44.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.768+0000 7f1455052700 1 --2- 172.21.15.71:0/969509219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1450100750 0x7f145019cbc0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f144000e8f0 tx=0x7f144000ecb0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.768+0000 7f1455052700 1 -- 172.21.15.71:0/969509219 shutdown_connections 2024-04-23T02:49:44.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.768+0000 7f1455052700 1 --2- 172.21.15.71:0/969509219 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f143806cce0 0x7f143806f1a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.768+0000 7f1455052700 1 --2- 172.21.15.71:0/969509219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14500ffda0 0x7f145019c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.768+0000 7f1455052700 1 --2- 172.21.15.71:0/969509219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1450100750 0x7f145019cbc0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.768+0000 7f1455052700 1 -- 172.21.15.71:0/969509219 >> 172.21.15.71:0/969509219 conn(0x7f14500fb650 msgr2=0x7f1450104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:44.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.768+0000 7f1455052700 1 -- 172.21.15.71:0/969509219 shutdown_connections 2024-04-23T02:49:44.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.768+0000 7f1455052700 1 -- 172.21.15.71:0/969509219 wait complete. 2024-04-23T02:49:44.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.870+0000 7f763c978700 1 -- 172.21.15.71:0/3525643644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7638100e90 msgr2=0x7f7638101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:44.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.870+0000 7f763c978700 1 --2- 172.21.15.71:0/3525643644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7638100e90 0x7f7638101270 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f762c009910 tx=0x7f762c009c20 comp rx=0 tx=0).stop 2024-04-23T02:49:44.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.870+0000 7f763c978700 1 -- 172.21.15.71:0/3525643644 shutdown_connections 2024-04-23T02:49:44.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.870+0000 7f763c978700 1 --2- 172.21.15.71:0/3525643644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7638101840 0x7f7638105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.870+0000 7f763c978700 1 --2- 172.21.15.71:0/3525643644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7638100e90 0x7f7638101270 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.870+0000 7f763c978700 1 -- 172.21.15.71:0/3525643644 >> 172.21.15.71:0/3525643644 conn(0x7f76380fc740 msgr2=0x7f76380feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:44.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.870+0000 7f763c978700 1 -- 172.21.15.71:0/3525643644 shutdown_connections 2024-04-23T02:49:44.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.870+0000 7f763c978700 1 -- 172.21.15.71:0/3525643644 wait complete. 2024-04-23T02:49:44.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.871+0000 7f763c978700 1 Processor -- start 2024-04-23T02:49:44.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.871+0000 7f763c978700 1 -- start start 2024-04-23T02:49:44.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.871+0000 7f763c978700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7638100e90 0x7f763819c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:44.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.871+0000 7f763c978700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7638101840 0x7f763819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:44.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.871+0000 7f763c978700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763819d1b0 con 0x7f7638100e90 2024-04-23T02:49:44.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.871+0000 7f763c978700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7638196660 con 0x7f7638101840 2024-04-23T02:49:44.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.871+0000 7f763659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7638100e90 0x7f763819c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:44.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.871+0000 7f763659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7638100e90 0x7f763819c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52298/0 (socket says 172.21.15.71:52298) 2024-04-23T02:49:44.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.871+0000 7f763659c700 1 -- 172.21.15.71:0/2797473394 learned_addr learned my addr 172.21.15.71:0/2797473394 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:49:44.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.872+0000 7f763659c700 1 -- 172.21.15.71:0/2797473394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7638101840 msgr2=0x7f763819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:44.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.872+0000 7f763659c700 1 --2- 172.21.15.71:0/2797473394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7638101840 0x7f763819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:44.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.872+0000 7f763659c700 1 -- 172.21.15.71:0/2797473394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762c0095f0 con 0x7f7638100e90 2024-04-23T02:49:44.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.872+0000 7f763659c700 1 --2- 172.21.15.71:0/2797473394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7638100e90 0x7f763819c5e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f762c006010 tx=0x7f762c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:44.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.872+0000 7f76277fe700 1 -- 172.21.15.71:0/2797473394 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f762c01d070 con 0x7f7638100e90 2024-04-23T02:49:44.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.872+0000 7f76277fe700 1 -- 172.21.15.71:0/2797473394 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f762c00f460 con 0x7f7638100e90 2024-04-23T02:49:44.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.872+0000 7f763c978700 1 -- 172.21.15.71:0/2797473394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76381968e0 con 0x7f7638100e90 2024-04-23T02:49:44.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.872+0000 7f76277fe700 1 -- 172.21.15.71:0/2797473394 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f762c021620 con 0x7f7638100e90 2024-04-23T02:49:44.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.872+0000 7f763c978700 1 -- 172.21.15.71:0/2797473394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7638196dd0 con 0x7f7638100e90 2024-04-23T02:49:44.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.873+0000 7f763c978700 1 -- 172.21.15.71:0/2797473394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7638109230 con 0x7f7638100e90 2024-04-23T02:49:44.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.873+0000 7f76277fe700 1 -- 172.21.15.71:0/2797473394 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f762c00fa40 con 0x7f7638100e90 2024-04-23T02:49:44.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.874+0000 7f76277fe700 1 --2- 172.21.15.71:0/2797473394 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f762006cce0 0x7f762006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:44.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.874+0000 7f76277fe700 1 -- 172.21.15.71:0/2797473394 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f762c08d790 con 0x7f7638100e90 2024-04-23T02:49:44.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.874+0000 7f7635d9b700 1 --2- 172.21.15.71:0/2797473394 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f762006cce0 0x7f762006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:44.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.874+0000 7f7635d9b700 1 --2- 172.21.15.71:0/2797473394 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f762006cce0 0x7f762006f1a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7628005950 tx=0x7f76280058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:44.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:44.878+0000 7f76277fe700 1 -- 172.21.15.71:0/2797473394 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f762c05b290 con 0x7f7638100e90 2024-04-23T02:49:44.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:44 smithi071 ceph-mon[144544]: from='client.35772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:49:44.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:44 smithi071 ceph-mon[144544]: from='client.35776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:49:44.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:44 smithi071 ceph-mon[144544]: pgmap v1094: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:45.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:44 smithi192 ceph-mon[108617]: from='client.35772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:49:45.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:44 smithi192 ceph-mon[108617]: from='client.35776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:49:45.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:44 smithi192 ceph-mon[108617]: pgmap v1094: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:45.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.151+0000 7f763c978700 1 -- 172.21.15.71:0/2797473394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f763804f2e0 con 0x7f7638100e90 2024-04-23T02:49:45.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.151+0000 7f76277fe700 1 -- 172.21.15.71:0/2797473394 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f762c05ae20 con 0x7f7638100e90 2024-04-23T02:49:45.154 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:49:45.155 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:49:45.155 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:49:45.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:49:45.155 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:49:45.155 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:49:45.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:49:45.155 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:49:45.155 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:49:45.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:49:45.155 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:49:45.155 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:49:45.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:49:45.155 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:49:45.155 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:49:45.156 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:49:45.156 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:49:45.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.153+0000 7f763c978700 1 -- 172.21.15.71:0/2797473394 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f762006cce0 msgr2=0x7f762006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:45.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.153+0000 7f763c978700 1 --2- 172.21.15.71:0/2797473394 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f762006cce0 0x7f762006f1a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7628005950 tx=0x7f76280058e0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.153+0000 7f763c978700 1 -- 172.21.15.71:0/2797473394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7638100e90 msgr2=0x7f763819c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:45.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.153+0000 7f763c978700 1 --2- 172.21.15.71:0/2797473394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7638100e90 0x7f763819c5e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f762c006010 tx=0x7f762c00be20 comp rx=0 tx=0).stop 2024-04-23T02:49:45.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.153+0000 7f763c978700 1 -- 172.21.15.71:0/2797473394 shutdown_connections 2024-04-23T02:49:45.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.153+0000 7f763c978700 1 --2- 172.21.15.71:0/2797473394 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f762006cce0 0x7f762006f1a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.153+0000 7f763c978700 1 --2- 172.21.15.71:0/2797473394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7638100e90 0x7f763819c5e0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.153+0000 7f763c978700 1 --2- 172.21.15.71:0/2797473394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7638101840 0x7f763819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.153+0000 7f763c978700 1 -- 172.21.15.71:0/2797473394 >> 172.21.15.71:0/2797473394 conn(0x7f76380fc740 msgr2=0x7f76380fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:45.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.154+0000 7f763c978700 1 -- 172.21.15.71:0/2797473394 shutdown_connections 2024-04-23T02:49:45.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.154+0000 7f763c978700 1 -- 172.21.15.71:0/2797473394 wait complete. 2024-04-23T02:49:45.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.256+0000 7f0b119a7700 1 -- 172.21.15.71:0/717752303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b0c105bf0 msgr2=0x7f0b0c105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:45.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.256+0000 7f0b119a7700 1 --2- 172.21.15.71:0/717752303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b0c105bf0 0x7f0b0c105fd0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009910 tx=0x7f0b00009c20 comp rx=0 tx=0).stop 2024-04-23T02:49:45.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.256+0000 7f0b119a7700 1 -- 172.21.15.71:0/717752303 shutdown_connections 2024-04-23T02:49:45.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.256+0000 7f0b119a7700 1 --2- 172.21.15.71:0/717752303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b0c0ffb40 0x7f0b0c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.256+0000 7f0b119a7700 1 --2- 172.21.15.71:0/717752303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b0c105bf0 0x7f0b0c105fd0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.256+0000 7f0b119a7700 1 -- 172.21.15.71:0/717752303 >> 172.21.15.71:0/717752303 conn(0x7f0b0c0fb610 msgr2=0x7f0b0c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:45.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.256+0000 7f0b119a7700 1 -- 172.21.15.71:0/717752303 shutdown_connections 2024-04-23T02:49:45.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.256+0000 7f0b119a7700 1 -- 172.21.15.71:0/717752303 wait complete. 2024-04-23T02:49:45.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.256+0000 7f0b119a7700 1 Processor -- start 2024-04-23T02:49:45.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.257+0000 7f0b119a7700 1 -- start start 2024-04-23T02:49:45.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.257+0000 7f0b119a7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b0c0ffb40 0x7f0b0c197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:45.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.257+0000 7f0b119a7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b0c105bf0 0x7f0b0c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:45.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.257+0000 7f0b119a7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c1980f0 con 0x7f0b0c105bf0 2024-04-23T02:49:45.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.257+0000 7f0b119a7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c198260 con 0x7f0b0c0ffb40 2024-04-23T02:49:45.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.257+0000 7f0b0a7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b0c105bf0 0x7f0b0c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:45.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.257+0000 7f0b0a7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b0c105bf0 0x7f0b0c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52310/0 (socket says 172.21.15.71:52310) 2024-04-23T02:49:45.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.257+0000 7f0b0a7fc700 1 -- 172.21.15.71:0/2977519372 learned_addr learned my addr 172.21.15.71:0/2977519372 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:49:45.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.257+0000 7f0b0affd700 1 --2- 172.21.15.71:0/2977519372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b0c0ffb40 0x7f0b0c197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:45.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.257+0000 7f0b0a7fc700 1 -- 172.21.15.71:0/2977519372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b0c0ffb40 msgr2=0x7f0b0c197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:45.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.257+0000 7f0b0a7fc700 1 --2- 172.21.15.71:0/2977519372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b0c0ffb40 0x7f0b0c197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.257+0000 7f0b0a7fc700 1 -- 172.21.15.71:0/2977519372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b000095f0 con 0x7f0b0c105bf0 2024-04-23T02:49:45.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.257+0000 7f0b0affd700 1 --2- 172.21.15.71:0/2977519372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b0c0ffb40 0x7f0b0c197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:49:45.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.257+0000 7f0b0a7fc700 1 --2- 172.21.15.71:0/2977519372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b0c105bf0 0x7f0b0c197ad0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00d8d0 tx=0x7f0afc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:45.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.257+0000 7f0b109a5700 1 -- 172.21.15.71:0/2977519372 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0afc009940 con 0x7f0b0c105bf0 2024-04-23T02:49:45.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.257+0000 7f0b109a5700 1 -- 172.21.15.71:0/2977519372 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0afc010460 con 0x7f0b0c105bf0 2024-04-23T02:49:45.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.257+0000 7f0b119a7700 1 -- 172.21.15.71:0/2977519372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0c1a0600 con 0x7f0b0c105bf0 2024-04-23T02:49:45.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.258+0000 7f0b109a5700 1 -- 172.21.15.71:0/2977519372 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0afc00f5f0 con 0x7f0b0c105bf0 2024-04-23T02:49:45.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.258+0000 7f0b119a7700 1 -- 172.21.15.71:0/2977519372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0c1a0af0 con 0x7f0b0c105bf0 2024-04-23T02:49:45.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.258+0000 7f0b119a7700 1 -- 172.21.15.71:0/2977519372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b0c108020 con 0x7f0b0c105bf0 2024-04-23T02:49:45.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.259+0000 7f0b109a5700 1 -- 172.21.15.71:0/2977519372 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0afc0105d0 con 0x7f0b0c105bf0 2024-04-23T02:49:45.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.259+0000 7f0b109a5700 1 --2- 172.21.15.71:0/2977519372 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0af406cce0 0x7f0af406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:45.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.259+0000 7f0b109a5700 1 -- 172.21.15.71:0/2977519372 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0afc08c4f0 con 0x7f0b0c105bf0 2024-04-23T02:49:45.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.259+0000 7f0b0affd700 1 --2- 172.21.15.71:0/2977519372 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0af406cce0 0x7f0af406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:45.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.260+0000 7f0b0affd700 1 --2- 172.21.15.71:0/2977519372 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0af406cce0 0x7f0af406f1a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000b5c0 tx=0x7f0b00005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:45.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.264+0000 7f0b109a5700 1 -- 172.21.15.71:0/2977519372 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0afc0565e0 con 0x7f0b0c105bf0 2024-04-23T02:49:45.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.495+0000 7f0b119a7700 1 -- 172.21.15.71:0/2977519372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0b0c04f2e0 con 0x7f0b0c105bf0 2024-04-23T02:49:45.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.495+0000 7f0b109a5700 1 -- 172.21.15.71:0/2977519372 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f0afc016740 con 0x7f0b0c105bf0 2024-04-23T02:49:45.498 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:49:45.498 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:49:45.498 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:49:45.498 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:49:45.498 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:49:45.498 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:49:45.499 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:49:45.499 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:49:45.499 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:49:45.499 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:49:45.499 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:49:45.499 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:49:45.499 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:49:45.499 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:49:45.499 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:49:45.499 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:49:45.499 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:49:45.499 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:49:45.499 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:49:45.499 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:49:45.500 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:49:45.500 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:49:45.500 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:49:45.500 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:49:45.500 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:49:45.500 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:49:45.500 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:49:45.500 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:49:45.500 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:49:45.500 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:49:45.500 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:49:45.500 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:49:45.500 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:49:45.500 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:49:45.500 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:49:45.501 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:49:45.501 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:49:45.501 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:49:45.501 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:49:45.501 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:49:45.501 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:49:45.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.497+0000 7f0b119a7700 1 -- 172.21.15.71:0/2977519372 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0af406cce0 msgr2=0x7f0af406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:45.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.497+0000 7f0b119a7700 1 --2- 172.21.15.71:0/2977519372 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0af406cce0 0x7f0af406f1a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000b5c0 tx=0x7f0b00005960 comp rx=0 tx=0).stop 2024-04-23T02:49:45.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.497+0000 7f0b119a7700 1 -- 172.21.15.71:0/2977519372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b0c105bf0 msgr2=0x7f0b0c197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:45.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.497+0000 7f0b119a7700 1 --2- 172.21.15.71:0/2977519372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b0c105bf0 0x7f0b0c197ad0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00d8d0 tx=0x7f0afc00dc90 comp rx=0 tx=0).stop 2024-04-23T02:49:45.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.497+0000 7f0b119a7700 1 -- 172.21.15.71:0/2977519372 shutdown_connections 2024-04-23T02:49:45.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.497+0000 7f0b119a7700 1 --2- 172.21.15.71:0/2977519372 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0af406cce0 0x7f0af406f1a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.497+0000 7f0b119a7700 1 --2- 172.21.15.71:0/2977519372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b0c0ffb40 0x7f0b0c197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.497+0000 7f0b119a7700 1 --2- 172.21.15.71:0/2977519372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b0c105bf0 0x7f0b0c197ad0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.497+0000 7f0b119a7700 1 -- 172.21.15.71:0/2977519372 >> 172.21.15.71:0/2977519372 conn(0x7f0b0c0fb610 msgr2=0x7f0b0c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:45.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.497+0000 7f0b119a7700 1 -- 172.21.15.71:0/2977519372 shutdown_connections 2024-04-23T02:49:45.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.497+0000 7f0b119a7700 1 -- 172.21.15.71:0/2977519372 wait complete. 2024-04-23T02:49:45.502 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:49:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.600+0000 7f8473e71700 1 -- 172.21.15.71:0/3438460071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f846c105350 msgr2=0x7f846c105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.600+0000 7f8473e71700 1 --2- 172.21.15.71:0/3438460071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f846c105350 0x7f846c105730 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f845c009910 tx=0x7f845c009c20 comp rx=0 tx=0).stop 2024-04-23T02:49:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.600+0000 7f8473e71700 1 -- 172.21.15.71:0/3438460071 shutdown_connections 2024-04-23T02:49:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.600+0000 7f8473e71700 1 --2- 172.21.15.71:0/3438460071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f846c0ff340 0x7f846c0ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.600+0000 7f8473e71700 1 --2- 172.21.15.71:0/3438460071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f846c105350 0x7f846c105730 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.600+0000 7f8473e71700 1 -- 172.21.15.71:0/3438460071 >> 172.21.15.71:0/3438460071 conn(0x7f846c074e30 msgr2=0x7f846c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.600+0000 7f8473e71700 1 -- 172.21.15.71:0/3438460071 shutdown_connections 2024-04-23T02:49:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.600+0000 7f8473e71700 1 -- 172.21.15.71:0/3438460071 wait complete. 2024-04-23T02:49:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.601+0000 7f8473e71700 1 Processor -- start 2024-04-23T02:49:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.601+0000 7f8473e71700 1 -- start start 2024-04-23T02:49:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.601+0000 7f8473e71700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f846c0ff340 0x7f846c194e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.601+0000 7f8473e71700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f846c105350 0x7f846c195390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.601+0000 7f8473e71700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c195a70 con 0x7f846c0ff340 2024-04-23T02:49:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.601+0000 7f8473e71700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c1997b0 con 0x7f846c105350 2024-04-23T02:49:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.601+0000 7f8471c0d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f846c0ff340 0x7f846c194e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.601+0000 7f8471c0d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f846c0ff340 0x7f846c194e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39664/0 (socket says 172.21.15.71:39664) 2024-04-23T02:49:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.601+0000 7f8471c0d700 1 -- 172.21.15.71:0/2964428365 learned_addr learned my addr 172.21.15.71:0/2964428365 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:49:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.601+0000 7f847140c700 1 --2- 172.21.15.71:0/2964428365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f846c105350 0x7f846c195390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.601+0000 7f8471c0d700 1 -- 172.21.15.71:0/2964428365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f846c105350 msgr2=0x7f846c195390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.601+0000 7f8471c0d700 1 --2- 172.21.15.71:0/2964428365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f846c105350 0x7f846c195390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.601+0000 7f8471c0d700 1 -- 172.21.15.71:0/2964428365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c0095f0 con 0x7f846c0ff340 2024-04-23T02:49:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.602+0000 7f8471c0d700 1 --2- 172.21.15.71:0/2964428365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f846c0ff340 0x7f846c194e50 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f845c006010 tx=0x7f845c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.602+0000 7f8462ffd700 1 -- 172.21.15.71:0/2964428365 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f845c003a60 con 0x7f846c0ff340 2024-04-23T02:49:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.602+0000 7f8473e71700 1 -- 172.21.15.71:0/2964428365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846c068580 con 0x7f846c0ff340 2024-04-23T02:49:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.602+0000 7f8462ffd700 1 -- 172.21.15.71:0/2964428365 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f845c003bc0 con 0x7f846c0ff340 2024-04-23T02:49:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.602+0000 7f8462ffd700 1 -- 172.21.15.71:0/2964428365 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f845c020650 con 0x7f846c0ff340 2024-04-23T02:49:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.602+0000 7f8473e71700 1 -- 172.21.15.71:0/2964428365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846c068a70 con 0x7f846c0ff340 2024-04-23T02:49:45.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.603+0000 7f8473e71700 1 -- 172.21.15.71:0/2964428365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f846c04f2e0 con 0x7f846c0ff340 2024-04-23T02:49:45.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.603+0000 7f8462ffd700 1 -- 172.21.15.71:0/2964428365 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f845c00f460 con 0x7f846c0ff340 2024-04-23T02:49:45.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.604+0000 7f8462ffd700 1 --2- 172.21.15.71:0/2964428365 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f845806eef0 0x7f84580713b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:45.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.604+0000 7f847140c700 1 --2- 172.21.15.71:0/2964428365 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f845806eef0 0x7f84580713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:45.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.604+0000 7f8462ffd700 1 -- 172.21.15.71:0/2964428365 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f845c08dfb0 con 0x7f846c0ff340 2024-04-23T02:49:45.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.604+0000 7f847140c700 1 --2- 172.21.15.71:0/2964428365 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f845806eef0 0x7f84580713b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8468005950 tx=0x7f84680058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:45.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.608+0000 7f8462ffd700 1 -- 172.21.15.71:0/2964428365 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f845c058140 con 0x7f846c0ff340 2024-04-23T02:49:45.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.819+0000 7f8473e71700 1 -- 172.21.15.71:0/2964428365 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f846c19a1b0 con 0x7f845806eef0 2024-04-23T02:49:45.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.820+0000 7f8462ffd700 1 -- 172.21.15.71:0/2964428365 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f846c19a1b0 con 0x7f845806eef0 2024-04-23T02:49:45.823 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:49:45.823 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:49:45.823 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:49:45.823 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:49:45.823 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:49:45.824 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:49:45.824 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:49:45.824 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:49:45.824 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:49:45.824 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:49:45.824 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:49:45.824 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:49:45.824 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:49:45.824 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:49:45.824 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:49:45.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.822+0000 7f8473e71700 1 -- 172.21.15.71:0/2964428365 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f845806eef0 msgr2=0x7f84580713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:45.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.822+0000 7f8473e71700 1 --2- 172.21.15.71:0/2964428365 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f845806eef0 0x7f84580713b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8468005950 tx=0x7f84680058e0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.822+0000 7f8473e71700 1 -- 172.21.15.71:0/2964428365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f846c0ff340 msgr2=0x7f846c194e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:45.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.822+0000 7f8473e71700 1 --2- 172.21.15.71:0/2964428365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f846c0ff340 0x7f846c194e50 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f845c006010 tx=0x7f845c0057c0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.822+0000 7f8473e71700 1 -- 172.21.15.71:0/2964428365 shutdown_connections 2024-04-23T02:49:45.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.822+0000 7f8473e71700 1 --2- 172.21.15.71:0/2964428365 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f845806eef0 0x7f84580713b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.822+0000 7f8473e71700 1 --2- 172.21.15.71:0/2964428365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f846c0ff340 0x7f846c194e50 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.822+0000 7f8473e71700 1 --2- 172.21.15.71:0/2964428365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f846c105350 0x7f846c195390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.822+0000 7f8473e71700 1 -- 172.21.15.71:0/2964428365 >> 172.21.15.71:0/2964428365 conn(0x7f846c074e30 msgr2=0x7f846c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:45.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.822+0000 7f8473e71700 1 -- 172.21.15.71:0/2964428365 shutdown_connections 2024-04-23T02:49:45.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.822+0000 7f8473e71700 1 -- 172.21.15.71:0/2964428365 wait complete. 2024-04-23T02:49:45.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.926+0000 7f0190f27700 1 -- 172.21.15.71:0/2529628377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c0ffb40 msgr2=0x7f018c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:45.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.926+0000 7f0190f27700 1 --2- 172.21.15.71:0/2529628377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c0ffb40 0x7f018c0fffc0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f017c009910 tx=0x7f017c009c20 comp rx=0 tx=0).stop 2024-04-23T02:49:45.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.926+0000 7f0190f27700 1 -- 172.21.15.71:0/2529628377 shutdown_connections 2024-04-23T02:49:45.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.926+0000 7f0190f27700 1 --2- 172.21.15.71:0/2529628377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c0ffb40 0x7f018c0fffc0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.926+0000 7f0190f27700 1 --2- 172.21.15.71:0/2529628377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f018c105bf0 0x7f018c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.926+0000 7f0190f27700 1 -- 172.21.15.71:0/2529628377 >> 172.21.15.71:0/2529628377 conn(0x7f018c0fb610 msgr2=0x7f018c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:45.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.926+0000 7f0190f27700 1 -- 172.21.15.71:0/2529628377 shutdown_connections 2024-04-23T02:49:45.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.926+0000 7f0190f27700 1 -- 172.21.15.71:0/2529628377 wait complete. 2024-04-23T02:49:45.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.927+0000 7f0190f27700 1 Processor -- start 2024-04-23T02:49:45.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.927+0000 7f0190f27700 1 -- start start 2024-04-23T02:49:45.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.928+0000 7f0190f27700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c0ffb40 0x7f018c197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:45.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.928+0000 7f0190f27700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f018c105bf0 0x7f018c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:45.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.928+0000 7f0190f27700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018c1980d0 con 0x7f018c0ffb40 2024-04-23T02:49:45.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.928+0000 7f0190f27700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018c198240 con 0x7f018c105bf0 2024-04-23T02:49:45.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.928+0000 7f018a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c0ffb40 0x7f018c197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:45.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.928+0000 7f018a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c0ffb40 0x7f018c197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39692/0 (socket says 172.21.15.71:39692) 2024-04-23T02:49:45.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.928+0000 7f018a59c700 1 -- 172.21.15.71:0/219495912 learned_addr learned my addr 172.21.15.71:0/219495912 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:49:45.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.928+0000 7f0189d9b700 1 --2- 172.21.15.71:0/219495912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f018c105bf0 0x7f018c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:45.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.928+0000 7f018a59c700 1 -- 172.21.15.71:0/219495912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f018c105bf0 msgr2=0x7f018c197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:45.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.928+0000 7f018a59c700 1 --2- 172.21.15.71:0/219495912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f018c105bf0 0x7f018c197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:45.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.928+0000 7f018a59c700 1 -- 172.21.15.71:0/219495912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017c0095f0 con 0x7f018c0ffb40 2024-04-23T02:49:45.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.928+0000 7f018a59c700 1 --2- 172.21.15.71:0/219495912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c0ffb40 0x7f018c197570 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f018000d8d0 tx=0x7f018000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:45.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.928+0000 7f017b7fe700 1 -- 172.21.15.71:0/219495912 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01800098a0 con 0x7f018c0ffb40 2024-04-23T02:49:45.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.928+0000 7f0190f27700 1 -- 172.21.15.71:0/219495912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f018c1a05e0 con 0x7f018c0ffb40 2024-04-23T02:49:45.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.928+0000 7f017b7fe700 1 -- 172.21.15.71:0/219495912 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0180010460 con 0x7f018c0ffb40 2024-04-23T02:49:45.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.928+0000 7f017b7fe700 1 -- 172.21.15.71:0/219495912 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0180009ea0 con 0x7f018c0ffb40 2024-04-23T02:49:45.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.928+0000 7f0190f27700 1 -- 172.21.15.71:0/219495912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f018c1a0ad0 con 0x7f018c0ffb40 2024-04-23T02:49:45.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.929+0000 7f0190f27700 1 -- 172.21.15.71:0/219495912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f018c1080a0 con 0x7f018c0ffb40 2024-04-23T02:49:45.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.930+0000 7f017b7fe700 1 -- 172.21.15.71:0/219495912 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f01800105d0 con 0x7f018c0ffb40 2024-04-23T02:49:45.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.930+0000 7f017b7fe700 1 --2- 172.21.15.71:0/219495912 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f017406cce0 0x7f017406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:49:45.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.930+0000 7f017b7fe700 1 -- 172.21.15.71:0/219495912 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f018008c480 con 0x7f018c0ffb40 2024-04-23T02:49:45.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.930+0000 7f0189d9b700 1 --2- 172.21.15.71:0/219495912 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f017406cce0 0x7f017406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:49:45.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.931+0000 7f0189d9b700 1 --2- 172.21.15.71:0/219495912 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f017406cce0 0x7f017406f1a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f018c0696e0 tx=0x7f017c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:49:45.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:45.935+0000 7f017b7fe700 1 -- 172.21.15.71:0/219495912 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f01800564f0 con 0x7f018c0ffb40 2024-04-23T02:49:45.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:45 smithi071 ceph-mon[144544]: from='client.35780 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:49:45.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:45 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2797473394' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:49:45.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:45 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2977519372' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:49:46.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:45 smithi192 ceph-mon[108617]: from='client.35780 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:49:46.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:45 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2797473394' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:49:46.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:45 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2977519372' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:49:46.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:46.205+0000 7f0190f27700 1 -- 172.21.15.71:0/219495912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f018c04f2e0 con 0x7f018c0ffb40 2024-04-23T02:49:46.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:46.205+0000 7f017b7fe700 1 -- 172.21.15.71:0/219495912 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7f0180010880 con 0x7f018c0ffb40 2024-04-23T02:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:49:46.208 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:49:46.209 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:49:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:49:46.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:46.207+0000 7f0190f27700 1 -- 172.21.15.71:0/219495912 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f017406cce0 msgr2=0x7f017406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:46.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:46.207+0000 7f0190f27700 1 --2- 172.21.15.71:0/219495912 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f017406cce0 0x7f017406f1a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f018c0696e0 tx=0x7f017c01a040 comp rx=0 tx=0).stop 2024-04-23T02:49:46.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:46.207+0000 7f0190f27700 1 -- 172.21.15.71:0/219495912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c0ffb40 msgr2=0x7f018c197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:49:46.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:46.207+0000 7f0190f27700 1 --2- 172.21.15.71:0/219495912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c0ffb40 0x7f018c197570 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f018000d8d0 tx=0x7f018000dbe0 comp rx=0 tx=0).stop 2024-04-23T02:49:46.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:46.207+0000 7f0190f27700 1 -- 172.21.15.71:0/219495912 shutdown_connections 2024-04-23T02:49:46.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:46.207+0000 7f0190f27700 1 --2- 172.21.15.71:0/219495912 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f017406cce0 0x7f017406f1a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:46.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:46.207+0000 7f0190f27700 1 --2- 172.21.15.71:0/219495912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f018c0ffb40 0x7f018c197570 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:46.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:46.207+0000 7f0190f27700 1 --2- 172.21.15.71:0/219495912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f018c105bf0 0x7f018c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:49:46.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:46.207+0000 7f0190f27700 1 -- 172.21.15.71:0/219495912 >> 172.21.15.71:0/219495912 conn(0x7f018c0fb610 msgr2=0x7f018c0fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:49:46.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:46.207+0000 7f0190f27700 1 -- 172.21.15.71:0/219495912 shutdown_connections 2024-04-23T02:49:46.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:49:46.207+0000 7f0190f27700 1 -- 172.21.15.71:0/219495912 wait complete. 2024-04-23T02:49:46.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:46 smithi071 ceph-mon[144544]: from='client.35792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:49:46.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:46 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/219495912' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:49:46.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:46 smithi071 ceph-mon[144544]: pgmap v1095: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:47.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:46 smithi192 ceph-mon[108617]: from='client.35792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:49:47.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:46 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/219495912' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:49:47.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:46 smithi192 ceph-mon[108617]: pgmap v1095: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:48.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:48 smithi071 ceph-mon[144544]: pgmap v1096: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:49.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:48 smithi192 ceph-mon[108617]: pgmap v1096: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:50.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:49:50.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:49:51.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:50 smithi192 ceph-mon[108617]: pgmap v1097: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:51.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:50 smithi071 ceph-mon[144544]: pgmap v1097: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:52.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:52 smithi192 ceph-mon[108617]: pgmap v1098: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:52.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:52 smithi071 ceph-mon[144544]: pgmap v1098: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:54.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:54 smithi071 ceph-mon[144544]: pgmap v1099: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:55.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:54 smithi192 ceph-mon[108617]: pgmap v1099: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:56.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:56 smithi071 ceph-mon[144544]: pgmap v1100: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:57.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:56 smithi192 ceph-mon[108617]: pgmap v1100: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:58.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:49:58 smithi071 ceph-mon[144544]: pgmap v1101: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:49:59.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:49:58 smithi192 ceph-mon[108617]: pgmap v1101: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:00.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:00 smithi192 ceph-mon[108617]: overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:50:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:00 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T02:49:59.999+0000 7fc17a16e700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:50:00.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:00 smithi071 ceph-mon[144544]: overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:50:01.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:01 smithi192 ceph-mon[108617]: pgmap v1102: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:01.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:01 smithi071 ceph-mon[144544]: pgmap v1102: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:02.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:02 smithi071 ceph-mon[144544]: pgmap v1103: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:03.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:02 smithi192 ceph-mon[108617]: pgmap v1103: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:04.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:04 smithi071 ceph-mon[144544]: pgmap v1104: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:05.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:04 smithi192 ceph-mon[108617]: pgmap v1104: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:05.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:50:06.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:50:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:06 smithi071 ceph-mon[144544]: pgmap v1105: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:07.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:06 smithi192 ceph-mon[108617]: pgmap v1105: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:08.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:08 smithi071 ceph-mon[144544]: pgmap v1106: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:09.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:08 smithi192 ceph-mon[108617]: pgmap v1106: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:10.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:10 smithi071 ceph-mon[144544]: pgmap v1107: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:11.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:10 smithi192 ceph-mon[108617]: pgmap v1107: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:12.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:12 smithi071 ceph-mon[144544]: pgmap v1108: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:13.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:12 smithi192 ceph-mon[108617]: pgmap v1108: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:14.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:14 smithi071 ceph-mon[144544]: pgmap v1109: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:15.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:14 smithi192 ceph-mon[108617]: pgmap v1109: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:16.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.322+0000 7f0ae103d700 1 -- 172.21.15.71:0/3408471734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 msgr2=0x7f0adc1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:16.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.322+0000 7f0ae103d700 1 --2- 172.21.15.71:0/3408471734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 0x7f0adc1070b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009910 tx=0x7f0ad0009c20 comp rx=0 tx=0).stop 2024-04-23T02:50:16.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.322+0000 7f0ae103d700 1 -- 172.21.15.71:0/3408471734 shutdown_connections 2024-04-23T02:50:16.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.322+0000 7f0ae103d700 1 --2- 172.21.15.71:0/3408471734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0adc068580 0x7f0adc0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:16.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.322+0000 7f0ae103d700 1 --2- 172.21.15.71:0/3408471734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 0x7f0adc1070b0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:16.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.322+0000 7f0ae103d700 1 -- 172.21.15.71:0/3408471734 >> 172.21.15.71:0/3408471734 conn(0x7f0adc075670 msgr2=0x7f0adc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:16.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.322+0000 7f0ae103d700 1 -- 172.21.15.71:0/3408471734 shutdown_connections 2024-04-23T02:50:16.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.322+0000 7f0ae103d700 1 -- 172.21.15.71:0/3408471734 wait complete. 2024-04-23T02:50:16.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.323+0000 7f0ae103d700 1 Processor -- start 2024-04-23T02:50:16.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.323+0000 7f0ae103d700 1 -- start start 2024-04-23T02:50:16.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.323+0000 7f0ae103d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0adc068580 0x7f0adc19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:16.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.323+0000 7f0ae103d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 0x7f0adc19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:16.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.323+0000 7f0ae103d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc19d130 con 0x7f0adc106cd0 2024-04-23T02:50:16.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.323+0000 7f0ae103d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc196630 con 0x7f0adc068580 2024-04-23T02:50:16.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.323+0000 7f0ada59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 0x7f0adc19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:16.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.323+0000 7f0ada59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 0x7f0adc19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46196/0 (socket says 172.21.15.71:46196) 2024-04-23T02:50:16.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.323+0000 7f0ada59c700 1 -- 172.21.15.71:0/1443213025 learned_addr learned my addr 172.21.15.71:0/1443213025 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:50:16.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.323+0000 7f0adad9d700 1 --2- 172.21.15.71:0/1443213025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0adc068580 0x7f0adc19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:16.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.324+0000 7f0ada59c700 1 -- 172.21.15.71:0/1443213025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0adc068580 msgr2=0x7f0adc19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:16.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.324+0000 7f0ada59c700 1 --2- 172.21.15.71:0/1443213025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0adc068580 0x7f0adc19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:16.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.324+0000 7f0ada59c700 1 -- 172.21.15.71:0/1443213025 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad00095f0 con 0x7f0adc106cd0 2024-04-23T02:50:16.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.324+0000 7f0ada59c700 1 --2- 172.21.15.71:0/1443213025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 0x7f0adc19caa0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00d8d0 tx=0x7f0acc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:16.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.324+0000 7f0ac3fff700 1 -- 172.21.15.71:0/1443213025 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0acc0098a0 con 0x7f0adc106cd0 2024-04-23T02:50:16.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.324+0000 7f0ae103d700 1 -- 172.21.15.71:0/1443213025 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0adc196910 con 0x7f0adc106cd0 2024-04-23T02:50:16.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.324+0000 7f0ac3fff700 1 -- 172.21.15.71:0/1443213025 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0acc010460 con 0x7f0adc106cd0 2024-04-23T02:50:16.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.324+0000 7f0ac3fff700 1 -- 172.21.15.71:0/1443213025 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0acc009ea0 con 0x7f0adc106cd0 2024-04-23T02:50:16.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.324+0000 7f0ae103d700 1 -- 172.21.15.71:0/1443213025 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0adc196e60 con 0x7f0adc106cd0 2024-04-23T02:50:16.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.325+0000 7f0ae103d700 1 -- 172.21.15.71:0/1443213025 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0adc1091d0 con 0x7f0adc106cd0 2024-04-23T02:50:16.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.325+0000 7f0ac3fff700 1 -- 172.21.15.71:0/1443213025 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0acc0105d0 con 0x7f0adc106cd0 2024-04-23T02:50:16.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.326+0000 7f0ac3fff700 1 --2- 172.21.15.71:0/1443213025 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ac406cce0 0x7f0ac406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:16.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.326+0000 7f0ac3fff700 1 -- 172.21.15.71:0/1443213025 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0acc08c480 con 0x7f0adc106cd0 2024-04-23T02:50:16.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.326+0000 7f0adad9d700 1 --2- 172.21.15.71:0/1443213025 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ac406cce0 0x7f0ac406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:16.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.326+0000 7f0adad9d700 1 --2- 172.21.15.71:0/1443213025 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ac406cce0 0x7f0ac406f1a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0006010 tx=0x7f0ad001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:16.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.330+0000 7f0ac3fff700 1 -- 172.21.15.71:0/1443213025 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0acc059f80 con 0x7f0adc106cd0 2024-04-23T02:50:16.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.541+0000 7f0ae103d700 1 -- 172.21.15.71:0/1443213025 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0adc197080 con 0x7f0ac406cce0 2024-04-23T02:50:16.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.542+0000 7f0ac3fff700 1 -- 172.21.15.71:0/1443213025 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0adc197080 con 0x7f0ac406cce0 2024-04-23T02:50:16.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.544+0000 7f0ae103d700 1 -- 172.21.15.71:0/1443213025 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ac406cce0 msgr2=0x7f0ac406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:16.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.544+0000 7f0ae103d700 1 --2- 172.21.15.71:0/1443213025 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ac406cce0 0x7f0ac406f1a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0006010 tx=0x7f0ad001a040 comp rx=0 tx=0).stop 2024-04-23T02:50:16.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.544+0000 7f0ae103d700 1 -- 172.21.15.71:0/1443213025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 msgr2=0x7f0adc19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:16.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.544+0000 7f0ae103d700 1 --2- 172.21.15.71:0/1443213025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 0x7f0adc19caa0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00d8d0 tx=0x7f0acc00dbe0 comp rx=0 tx=0).stop 2024-04-23T02:50:16.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.545+0000 7f0ae103d700 1 -- 172.21.15.71:0/1443213025 shutdown_connections 2024-04-23T02:50:16.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.545+0000 7f0ae103d700 1 --2- 172.21.15.71:0/1443213025 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0ac406cce0 0x7f0ac406f1a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:16.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.545+0000 7f0ae103d700 1 --2- 172.21.15.71:0/1443213025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0adc068580 0x7f0adc19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:16.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.545+0000 7f0ae103d700 1 --2- 172.21.15.71:0/1443213025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0adc106cd0 0x7f0adc19caa0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:16.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.545+0000 7f0ae103d700 1 -- 172.21.15.71:0/1443213025 >> 172.21.15.71:0/1443213025 conn(0x7f0adc075670 msgr2=0x7f0adc0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:16.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.545+0000 7f0ae103d700 1 -- 172.21.15.71:0/1443213025 shutdown_connections 2024-04-23T02:50:16.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.545+0000 7f0ae103d700 1 -- 172.21.15.71:0/1443213025 wait complete. 2024-04-23T02:50:16.558 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:50:16.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.648+0000 7f17f5237700 1 -- 172.21.15.71:0/3233950933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f00ffda0 msgr2=0x7f17f0100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:16.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.648+0000 7f17f5237700 1 --2- 172.21.15.71:0/3233950933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f00ffda0 0x7f17f0100180 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009910 tx=0x7f17e0009c20 comp rx=0 tx=0).stop 2024-04-23T02:50:16.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.648+0000 7f17f5237700 1 -- 172.21.15.71:0/3233950933 shutdown_connections 2024-04-23T02:50:16.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.648+0000 7f17f5237700 1 --2- 172.21.15.71:0/3233950933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17f0100750 0x7f17f01047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:16.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.648+0000 7f17f5237700 1 --2- 172.21.15.71:0/3233950933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f00ffda0 0x7f17f0100180 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:16.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.648+0000 7f17f5237700 1 -- 172.21.15.71:0/3233950933 >> 172.21.15.71:0/3233950933 conn(0x7f17f00fb650 msgr2=0x7f17f00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:16.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.648+0000 7f17f5237700 1 -- 172.21.15.71:0/3233950933 shutdown_connections 2024-04-23T02:50:16.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.648+0000 7f17f5237700 1 -- 172.21.15.71:0/3233950933 wait complete. 2024-04-23T02:50:16.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.649+0000 7f17f5237700 1 Processor -- start 2024-04-23T02:50:16.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.649+0000 7f17f5237700 1 -- start start 2024-04-23T02:50:16.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.649+0000 7f17f5237700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f00ffda0 0x7f17f019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:16.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.649+0000 7f17f5237700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17f0100750 0x7f17f019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:16.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.649+0000 7f17f5237700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f019d160 con 0x7f17f00ffda0 2024-04-23T02:50:16.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.649+0000 7f17f5237700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f0196610 con 0x7f17f0100750 2024-04-23T02:50:16.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.650+0000 7f17e6d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f00ffda0 0x7f17f019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:16.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.650+0000 7f17e6d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f00ffda0 0x7f17f019c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46216/0 (socket says 172.21.15.71:46216) 2024-04-23T02:50:16.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.650+0000 7f17e6d9d700 1 -- 172.21.15.71:0/865771054 learned_addr learned my addr 172.21.15.71:0/865771054 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:50:16.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.650+0000 7f17eed9d700 1 --2- 172.21.15.71:0/865771054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17f0100750 0x7f17f019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:16.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.650+0000 7f17e6d9d700 1 -- 172.21.15.71:0/865771054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17f0100750 msgr2=0x7f17f019cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:16.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.650+0000 7f17e6d9d700 1 --2- 172.21.15.71:0/865771054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17f0100750 0x7f17f019cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:16.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.650+0000 7f17e6d9d700 1 -- 172.21.15.71:0/865771054 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e00095f0 con 0x7f17f00ffda0 2024-04-23T02:50:16.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.650+0000 7f17e6d9d700 1 --2- 172.21.15.71:0/865771054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f00ffda0 0x7f17f019c590 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f17e00092e0 tx=0x7f17e000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:16.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.650+0000 7f17ecd99700 1 -- 172.21.15.71:0/865771054 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17e001d070 con 0x7f17f00ffda0 2024-04-23T02:50:16.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.650+0000 7f17ecd99700 1 -- 172.21.15.71:0/865771054 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f17e0005e00 con 0x7f17f00ffda0 2024-04-23T02:50:16.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.650+0000 7f17f5237700 1 -- 172.21.15.71:0/865771054 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f0196890 con 0x7f17f00ffda0 2024-04-23T02:50:16.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.650+0000 7f17ecd99700 1 -- 172.21.15.71:0/865771054 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17e000fe90 con 0x7f17f00ffda0 2024-04-23T02:50:16.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.650+0000 7f17f5237700 1 -- 172.21.15.71:0/865771054 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f0196d80 con 0x7f17f00ffda0 2024-04-23T02:50:16.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.651+0000 7f17f5237700 1 -- 172.21.15.71:0/865771054 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f004f2e0 con 0x7f17f00ffda0 2024-04-23T02:50:16.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.652+0000 7f17ecd99700 1 -- 172.21.15.71:0/865771054 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f17e000f6d0 con 0x7f17f00ffda0 2024-04-23T02:50:16.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.652+0000 7f17ecd99700 1 --2- 172.21.15.71:0/865771054 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f17dc06cce0 0x7f17dc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:16.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.652+0000 7f17ecd99700 1 -- 172.21.15.71:0/865771054 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f17e008d7e0 con 0x7f17f00ffda0 2024-04-23T02:50:16.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.652+0000 7f17eed9d700 1 --2- 172.21.15.71:0/865771054 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f17dc06cce0 0x7f17dc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:16.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.653+0000 7f17eed9d700 1 --2- 172.21.15.71:0/865771054 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f17dc06cce0 0x7f17dc06f1a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f17f00fbce0 tx=0x7f17d800b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:16.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.657+0000 7f17ecd99700 1 -- 172.21.15.71:0/865771054 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f17e005b2e0 con 0x7f17f00ffda0 2024-04-23T02:50:16.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.867+0000 7f17f5237700 1 -- 172.21.15.71:0/865771054 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f17f0197030 con 0x7f17dc06cce0 2024-04-23T02:50:16.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.868+0000 7f17ecd99700 1 -- 172.21.15.71:0/865771054 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f17f0197030 con 0x7f17dc06cce0 2024-04-23T02:50:16.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.870+0000 7f17f5237700 1 -- 172.21.15.71:0/865771054 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f17dc06cce0 msgr2=0x7f17dc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:16.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.870+0000 7f17f5237700 1 --2- 172.21.15.71:0/865771054 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f17dc06cce0 0x7f17dc06f1a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f17f00fbce0 tx=0x7f17d800b410 comp rx=0 tx=0).stop 2024-04-23T02:50:16.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.870+0000 7f17f5237700 1 -- 172.21.15.71:0/865771054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f00ffda0 msgr2=0x7f17f019c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:16.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.870+0000 7f17f5237700 1 --2- 172.21.15.71:0/865771054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f00ffda0 0x7f17f019c590 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f17e00092e0 tx=0x7f17e000bd70 comp rx=0 tx=0).stop 2024-04-23T02:50:16.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.871+0000 7f17f5237700 1 -- 172.21.15.71:0/865771054 shutdown_connections 2024-04-23T02:50:16.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.871+0000 7f17f5237700 1 --2- 172.21.15.71:0/865771054 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f17dc06cce0 0x7f17dc06f1a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:16.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.871+0000 7f17f5237700 1 --2- 172.21.15.71:0/865771054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17f00ffda0 0x7f17f019c590 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:16.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.871+0000 7f17f5237700 1 --2- 172.21.15.71:0/865771054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17f0100750 0x7f17f019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:16.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.871+0000 7f17f5237700 1 -- 172.21.15.71:0/865771054 >> 172.21.15.71:0/865771054 conn(0x7f17f00fb650 msgr2=0x7f17f0103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:16.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.871+0000 7f17f5237700 1 -- 172.21.15.71:0/865771054 shutdown_connections 2024-04-23T02:50:16.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.871+0000 7f17f5237700 1 -- 172.21.15.71:0/865771054 wait complete. 2024-04-23T02:50:16.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.974+0000 7eff2ec97700 1 -- 172.21.15.71:0/1021601906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff28101850 msgr2=0x7eff281058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:16.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.974+0000 7eff2ec97700 1 --2- 172.21.15.71:0/1021601906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff28101850 0x7eff281058a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7eff14009910 tx=0x7eff14009c20 comp rx=0 tx=0).stop 2024-04-23T02:50:16.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.974+0000 7eff2ec97700 1 -- 172.21.15.71:0/1021601906 shutdown_connections 2024-04-23T02:50:16.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.974+0000 7eff2ec97700 1 --2- 172.21.15.71:0/1021601906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff28101850 0x7eff281058a0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:16.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.974+0000 7eff2ec97700 1 --2- 172.21.15.71:0/1021601906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff28100ea0 0x7eff28101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:16.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.974+0000 7eff2ec97700 1 -- 172.21.15.71:0/1021601906 >> 172.21.15.71:0/1021601906 conn(0x7eff280fc750 msgr2=0x7eff280feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:16.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.974+0000 7eff2ec97700 1 -- 172.21.15.71:0/1021601906 shutdown_connections 2024-04-23T02:50:16.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.974+0000 7eff2ec97700 1 -- 172.21.15.71:0/1021601906 wait complete. 2024-04-23T02:50:16.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.974+0000 7eff2ec97700 1 Processor -- start 2024-04-23T02:50:16.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.974+0000 7eff2ec97700 1 -- start start 2024-04-23T02:50:16.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.975+0000 7eff2ec97700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff28100ea0 0x7eff2819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:16.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.975+0000 7eff2ec97700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff28101850 0x7eff2819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:16.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.975+0000 7eff2ec97700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2819d210 con 0x7eff28101850 2024-04-23T02:50:16.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.975+0000 7eff2ec97700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff281966c0 con 0x7eff28100ea0 2024-04-23T02:50:16.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.975+0000 7eff27fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff28101850 0x7eff2819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:16.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.975+0000 7eff27fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff28101850 0x7eff2819cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46222/0 (socket says 172.21.15.71:46222) 2024-04-23T02:50:16.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.975+0000 7eff27fff700 1 -- 172.21.15.71:0/1631476704 learned_addr learned my addr 172.21.15.71:0/1631476704 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:50:16.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.975+0000 7eff2ca33700 1 --2- 172.21.15.71:0/1631476704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff28100ea0 0x7eff2819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:16.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.975+0000 7eff27fff700 1 -- 172.21.15.71:0/1631476704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff28100ea0 msgr2=0x7eff2819c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:16.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.975+0000 7eff27fff700 1 --2- 172.21.15.71:0/1631476704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff28100ea0 0x7eff2819c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:16.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.975+0000 7eff27fff700 1 -- 172.21.15.71:0/1631476704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff140095f0 con 0x7eff28101850 2024-04-23T02:50:16.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.975+0000 7eff27fff700 1 --2- 172.21.15.71:0/1631476704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff28101850 0x7eff2819cb80 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7eff1400bd70 tx=0x7eff1400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:16.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.975+0000 7eff25ffb700 1 -- 172.21.15.71:0/1631476704 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff1401d070 con 0x7eff28101850 2024-04-23T02:50:16.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.975+0000 7eff25ffb700 1 -- 172.21.15.71:0/1631476704 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7eff1400f460 con 0x7eff28101850 2024-04-23T02:50:16.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.975+0000 7eff2ec97700 1 -- 172.21.15.71:0/1631476704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff28196940 con 0x7eff28101850 2024-04-23T02:50:16.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.975+0000 7eff25ffb700 1 -- 172.21.15.71:0/1631476704 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff14021640 con 0x7eff28101850 2024-04-23T02:50:16.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.976+0000 7eff2ec97700 1 -- 172.21.15.71:0/1631476704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff28196e30 con 0x7eff28101850 2024-04-23T02:50:16.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.976+0000 7eff2ec97700 1 -- 172.21.15.71:0/1631476704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff281092a0 con 0x7eff28101850 2024-04-23T02:50:16.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.977+0000 7eff25ffb700 1 -- 172.21.15.71:0/1631476704 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7eff1400f9e0 con 0x7eff28101850 2024-04-23T02:50:16.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.977+0000 7eff25ffb700 1 --2- 172.21.15.71:0/1631476704 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7eff1806cce0 0x7eff1806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:16.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.977+0000 7eff25ffb700 1 -- 172.21.15.71:0/1631476704 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7eff1408d620 con 0x7eff28101850 2024-04-23T02:50:16.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.977+0000 7eff2ca33700 1 --2- 172.21.15.71:0/1631476704 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7eff1806cce0 0x7eff1806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:16.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.978+0000 7eff2ca33700 1 --2- 172.21.15.71:0/1631476704 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7eff1806cce0 0x7eff1806f1a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7eff1c007be0 tx=0x7eff1c0073d0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:16.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:16.982+0000 7eff25ffb700 1 -- 172.21.15.71:0/1631476704 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7eff1405b1a0 con 0x7eff28101850 2024-04-23T02:50:16.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:16 smithi071 ceph-mon[144544]: from='client.35800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:50:17.022 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:16 smithi071 ceph-mon[144544]: pgmap v1110: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:17.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:16 smithi192 ceph-mon[108617]: from='client.35800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:50:17.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:16 smithi192 ceph-mon[108617]: pgmap v1110: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:17.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.181+0000 7eff2ec97700 1 -- 172.21.15.71:0/1631476704 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7eff281970a0 con 0x7eff1806cce0 2024-04-23T02:50:17.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.192+0000 7eff25ffb700 1 -- 172.21.15.71:0/1631476704 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7eff281970a0 con 0x7eff1806cce0 2024-04-23T02:50:17.195 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:50:17.195 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (33m) 12m ago 45m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:50:17.196 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (45m) 12m ago 45m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:50:17.196 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (44m) 12m ago 44m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:50:17.196 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (31m) 12m ago 45m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:50:17.196 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (31m) 12m ago 44m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:50:17.196 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (33m) 12m ago 45m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:50:17.196 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (28m) 12m ago 39m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:50:17.197 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (27m) 12m ago 39m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:50:17.197 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (12m) 12m ago 39m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:50:17.197 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (27m) 12m ago 39m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:50:17.197 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (35m) 12m ago 46m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:50:17.197 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (35m) 12m ago 44m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:50:17.197 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (32m) 12m ago 47m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:50:17.197 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (32m) 12m ago 44m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:50:17.198 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (34m) 12m ago 45m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:50:17.198 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (34m) 12m ago 44m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:50:17.198 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (31m) 12m ago 43m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:50:17.198 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (30m) 12m ago 42m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:50:17.198 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (30m) 12m ago 42m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:50:17.198 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (29m) 12m ago 41m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:50:17.198 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (29m) 12m ago 41m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:50:17.199 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (28m) 12m ago 40m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:50:17.199 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (34m) 12m ago 44m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:50:17.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.194+0000 7eff2ec97700 1 -- 172.21.15.71:0/1631476704 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7eff1806cce0 msgr2=0x7eff1806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:17.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.194+0000 7eff2ec97700 1 --2- 172.21.15.71:0/1631476704 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7eff1806cce0 0x7eff1806f1a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7eff1c007be0 tx=0x7eff1c0073d0 comp rx=0 tx=0).stop 2024-04-23T02:50:17.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.194+0000 7eff2ec97700 1 -- 172.21.15.71:0/1631476704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff28101850 msgr2=0x7eff2819cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:17.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.194+0000 7eff2ec97700 1 --2- 172.21.15.71:0/1631476704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff28101850 0x7eff2819cb80 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7eff1400bd70 tx=0x7eff1400bda0 comp rx=0 tx=0).stop 2024-04-23T02:50:17.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.194+0000 7eff2ec97700 1 -- 172.21.15.71:0/1631476704 shutdown_connections 2024-04-23T02:50:17.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.194+0000 7eff2ec97700 1 --2- 172.21.15.71:0/1631476704 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7eff1806cce0 0x7eff1806f1a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:17.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.194+0000 7eff2ec97700 1 --2- 172.21.15.71:0/1631476704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff28100ea0 0x7eff2819c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:17.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.194+0000 7eff2ec97700 1 --2- 172.21.15.71:0/1631476704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff28101850 0x7eff2819cb80 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:17.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.194+0000 7eff2ec97700 1 -- 172.21.15.71:0/1631476704 >> 172.21.15.71:0/1631476704 conn(0x7eff280fc750 msgr2=0x7eff280fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:17.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.194+0000 7eff2ec97700 1 -- 172.21.15.71:0/1631476704 shutdown_connections 2024-04-23T02:50:17.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.194+0000 7eff2ec97700 1 -- 172.21.15.71:0/1631476704 wait complete. 2024-04-23T02:50:17.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.296+0000 7fc3c3df6700 1 -- 172.21.15.71:0/3594507195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3bc0ffc20 msgr2=0x7fc3bc100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:17.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.296+0000 7fc3c3df6700 1 --2- 172.21.15.71:0/3594507195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3bc0ffc20 0x7fc3bc100080 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8009910 tx=0x7fc3b8009c20 comp rx=0 tx=0).stop 2024-04-23T02:50:17.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.296+0000 7fc3c3df6700 1 -- 172.21.15.71:0/3594507195 shutdown_connections 2024-04-23T02:50:17.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.296+0000 7fc3c3df6700 1 --2- 172.21.15.71:0/3594507195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3bc0ffc20 0x7fc3bc100080 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:17.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.296+0000 7fc3c3df6700 1 --2- 172.21.15.71:0/3594507195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3bc105bd0 0x7fc3bc105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:17.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.296+0000 7fc3c3df6700 1 -- 172.21.15.71:0/3594507195 >> 172.21.15.71:0/3594507195 conn(0x7fc3bc075670 msgr2=0x7fc3bc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:17.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.296+0000 7fc3c3df6700 1 -- 172.21.15.71:0/3594507195 shutdown_connections 2024-04-23T02:50:17.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.296+0000 7fc3c3df6700 1 -- 172.21.15.71:0/3594507195 wait complete. 2024-04-23T02:50:17.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.296+0000 7fc3c3df6700 1 Processor -- start 2024-04-23T02:50:17.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.297+0000 7fc3c3df6700 1 -- start start 2024-04-23T02:50:17.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.297+0000 7fc3c3df6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3bc0ffc20 0x7fc3bc194650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:17.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.297+0000 7fc3c3df6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3bc105bd0 0x7fc3bc194b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:17.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.297+0000 7fc3c3df6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc1951e0 con 0x7fc3bc0ffc20 2024-04-23T02:50:17.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.297+0000 7fc3c3df6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc195320 con 0x7fc3bc105bd0 2024-04-23T02:50:17.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.297+0000 7fc3c1b92700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3bc0ffc20 0x7fc3bc194650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:17.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.297+0000 7fc3c1b92700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3bc0ffc20 0x7fc3bc194650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46234/0 (socket says 172.21.15.71:46234) 2024-04-23T02:50:17.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.297+0000 7fc3c1b92700 1 -- 172.21.15.71:0/3504404735 learned_addr learned my addr 172.21.15.71:0/3504404735 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:50:17.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.297+0000 7fc3c1391700 1 --2- 172.21.15.71:0/3504404735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3bc105bd0 0x7fc3bc194b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:17.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.297+0000 7fc3c1b92700 1 -- 172.21.15.71:0/3504404735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3bc105bd0 msgr2=0x7fc3bc194b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:17.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.297+0000 7fc3c1b92700 1 --2- 172.21.15.71:0/3504404735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3bc105bd0 0x7fc3bc194b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:17.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.297+0000 7fc3c1b92700 1 -- 172.21.15.71:0/3504404735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b80095f0 con 0x7fc3bc0ffc20 2024-04-23T02:50:17.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.297+0000 7fc3c1b92700 1 --2- 172.21.15.71:0/3504404735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3bc0ffc20 0x7fc3bc194650 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00c930 tx=0x7fc3ac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:17.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.297+0000 7fc3b2ffd700 1 -- 172.21.15.71:0/3504404735 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3ac007ad0 con 0x7fc3bc0ffc20 2024-04-23T02:50:17.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.297+0000 7fc3b2ffd700 1 -- 172.21.15.71:0/3504404735 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc3ac00ce80 con 0x7fc3bc0ffc20 2024-04-23T02:50:17.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.298+0000 7fc3c3df6700 1 -- 172.21.15.71:0/3504404735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3bc199170 con 0x7fc3bc0ffc20 2024-04-23T02:50:17.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.298+0000 7fc3b2ffd700 1 -- 172.21.15.71:0/3504404735 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3ac018630 con 0x7fc3bc0ffc20 2024-04-23T02:50:17.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.298+0000 7fc3c3df6700 1 -- 172.21.15.71:0/3504404735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3bc1996c0 con 0x7fc3bc0ffc20 2024-04-23T02:50:17.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.298+0000 7fc3c3df6700 1 -- 172.21.15.71:0/3504404735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3bc1080d0 con 0x7fc3bc0ffc20 2024-04-23T02:50:17.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.299+0000 7fc3b2ffd700 1 -- 172.21.15.71:0/3504404735 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc3ac007c30 con 0x7fc3bc0ffc20 2024-04-23T02:50:17.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.299+0000 7fc3b2ffd700 1 --2- 172.21.15.71:0/3504404735 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc3a806eef0 0x7fc3a80713b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:17.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.300+0000 7fc3c1391700 1 --2- 172.21.15.71:0/3504404735 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc3a806eef0 0x7fc3a80713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:17.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.300+0000 7fc3b2ffd700 1 -- 172.21.15.71:0/3504404735 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc3ac08c5a0 con 0x7fc3bc0ffc20 2024-04-23T02:50:17.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.300+0000 7fc3c1391700 1 --2- 172.21.15.71:0/3504404735 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc3a806eef0 0x7fc3a80713b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc195c70 tx=0x7fc3b801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:17.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.304+0000 7fc3b2ffd700 1 -- 172.21.15.71:0/3504404735 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc3ac056610 con 0x7fc3bc0ffc20 2024-04-23T02:50:17.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.575+0000 7fc3c3df6700 1 -- 172.21.15.71:0/3504404735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc3bc107ab0 con 0x7fc3bc0ffc20 2024-04-23T02:50:17.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.575+0000 7fc3b2ffd700 1 -- 172.21.15.71:0/3504404735 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fc3ac059c30 con 0x7fc3bc0ffc20 2024-04-23T02:50:17.579 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:50:17.579 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:50:17.579 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:50:17.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:50:17.579 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:50:17.579 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:50:17.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:50:17.579 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:50:17.579 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:50:17.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:50:17.579 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:50:17.579 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:50:17.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:50:17.579 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:50:17.579 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:50:17.579 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:50:17.580 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:50:17.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.577+0000 7fc3c3df6700 1 -- 172.21.15.71:0/3504404735 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc3a806eef0 msgr2=0x7fc3a80713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:17.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.577+0000 7fc3c3df6700 1 --2- 172.21.15.71:0/3504404735 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc3a806eef0 0x7fc3a80713b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc195c70 tx=0x7fc3b801a040 comp rx=0 tx=0).stop 2024-04-23T02:50:17.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.577+0000 7fc3c3df6700 1 -- 172.21.15.71:0/3504404735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3bc0ffc20 msgr2=0x7fc3bc194650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:17.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.577+0000 7fc3c3df6700 1 --2- 172.21.15.71:0/3504404735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3bc0ffc20 0x7fc3bc194650 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00c930 tx=0x7fc3ac00cc40 comp rx=0 tx=0).stop 2024-04-23T02:50:17.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.577+0000 7fc3c3df6700 1 -- 172.21.15.71:0/3504404735 shutdown_connections 2024-04-23T02:50:17.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.577+0000 7fc3c3df6700 1 --2- 172.21.15.71:0/3504404735 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc3a806eef0 0x7fc3a80713b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:17.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.577+0000 7fc3c3df6700 1 --2- 172.21.15.71:0/3504404735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3bc0ffc20 0x7fc3bc194650 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:17.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.577+0000 7fc3c3df6700 1 --2- 172.21.15.71:0/3504404735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3bc105bd0 0x7fc3bc194b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:17.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.578+0000 7fc3c3df6700 1 -- 172.21.15.71:0/3504404735 >> 172.21.15.71:0/3504404735 conn(0x7fc3bc075670 msgr2=0x7fc3bc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:17.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.578+0000 7fc3c3df6700 1 -- 172.21.15.71:0/3504404735 shutdown_connections 2024-04-23T02:50:17.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.578+0000 7fc3c3df6700 1 -- 172.21.15.71:0/3504404735 wait complete. 2024-04-23T02:50:17.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.680+0000 7f9a3af32700 1 -- 172.21.15.71:0/1844122597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a34105bd0 msgr2=0x7f9a34105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:17.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.680+0000 7f9a3af32700 1 --2- 172.21.15.71:0/1844122597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a34105bd0 0x7f9a34105fb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009910 tx=0x7f9a28009c20 comp rx=0 tx=0).stop 2024-04-23T02:50:17.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.680+0000 7f9a3af32700 1 -- 172.21.15.71:0/1844122597 shutdown_connections 2024-04-23T02:50:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.680+0000 7f9a3af32700 1 --2- 172.21.15.71:0/1844122597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a340ffc20 0x7f9a34100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.680+0000 7f9a3af32700 1 --2- 172.21.15.71:0/1844122597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a34105bd0 0x7f9a34105fb0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.680+0000 7f9a3af32700 1 -- 172.21.15.71:0/1844122597 >> 172.21.15.71:0/1844122597 conn(0x7f9a34075670 msgr2=0x7f9a34075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.680+0000 7f9a3af32700 1 -- 172.21.15.71:0/1844122597 shutdown_connections 2024-04-23T02:50:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.680+0000 7f9a3af32700 1 -- 172.21.15.71:0/1844122597 wait complete. 2024-04-23T02:50:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.680+0000 7f9a3af32700 1 Processor -- start 2024-04-23T02:50:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.680+0000 7f9a3af32700 1 -- start start 2024-04-23T02:50:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.681+0000 7f9a3af32700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a340ffc20 0x7f9a3419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.681+0000 7f9a3af32700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a34105bd0 0x7f9a3419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.681+0000 7f9a3af32700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3419d160 con 0x7f9a34105bd0 2024-04-23T02:50:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.681+0000 7f9a3af32700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a34196610 con 0x7f9a340ffc20 2024-04-23T02:50:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.681+0000 7f9a33fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a34105bd0 0x7f9a3419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:17.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.681+0000 7f9a33fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a34105bd0 0x7f9a3419cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46248/0 (socket says 172.21.15.71:46248) 2024-04-23T02:50:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.681+0000 7f9a33fff700 1 -- 172.21.15.71:0/1587519950 learned_addr learned my addr 172.21.15.71:0/1587519950 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:50:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.681+0000 7f9a38cce700 1 --2- 172.21.15.71:0/1587519950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a340ffc20 0x7f9a3419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.681+0000 7f9a33fff700 1 -- 172.21.15.71:0/1587519950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a340ffc20 msgr2=0x7f9a3419c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.681+0000 7f9a33fff700 1 --2- 172.21.15.71:0/1587519950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a340ffc20 0x7f9a3419c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.681+0000 7f9a33fff700 1 -- 172.21.15.71:0/1587519950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a280095f0 con 0x7f9a34105bd0 2024-04-23T02:50:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.681+0000 7f9a33fff700 1 --2- 172.21.15.71:0/1587519950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a34105bd0 0x7f9a3419cad0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f9a2000c8a0 tx=0x7f9a2000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.681+0000 7f9a31ffb700 1 -- 172.21.15.71:0/1587519950 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a200078e0 con 0x7f9a34105bd0 2024-04-23T02:50:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.681+0000 7f9a31ffb700 1 -- 172.21.15.71:0/1587519950 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9a2000f450 con 0x7f9a34105bd0 2024-04-23T02:50:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.681+0000 7f9a3af32700 1 -- 172.21.15.71:0/1587519950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a341968f0 con 0x7f9a34105bd0 2024-04-23T02:50:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.681+0000 7f9a31ffb700 1 -- 172.21.15.71:0/1587519950 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a2000e5e0 con 0x7f9a34105bd0 2024-04-23T02:50:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.682+0000 7f9a3af32700 1 -- 172.21.15.71:0/1587519950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a34196e40 con 0x7f9a34105bd0 2024-04-23T02:50:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.682+0000 7f9a3af32700 1 -- 172.21.15.71:0/1587519950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3404f2e0 con 0x7f9a34105bd0 2024-04-23T02:50:17.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.683+0000 7f9a31ffb700 1 -- 172.21.15.71:0/1587519950 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9a2000f5c0 con 0x7f9a34105bd0 2024-04-23T02:50:17.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.683+0000 7f9a31ffb700 1 --2- 172.21.15.71:0/1587519950 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9a2406cce0 0x7f9a2406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:17.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.683+0000 7f9a31ffb700 1 -- 172.21.15.71:0/1587519950 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9a2008c480 con 0x7f9a34105bd0 2024-04-23T02:50:17.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.683+0000 7f9a38cce700 1 --2- 172.21.15.71:0/1587519950 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9a2406cce0 0x7f9a2406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:17.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.684+0000 7f9a38cce700 1 --2- 172.21.15.71:0/1587519950 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9a2406cce0 0x7f9a2406f1a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9a28005f50 tx=0x7f9a2801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:17.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.688+0000 7f9a31ffb700 1 -- 172.21.15.71:0/1587519950 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9a2005a000 con 0x7f9a34105bd0 2024-04-23T02:50:17.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.922+0000 7f9a3af32700 1 -- 172.21.15.71:0/1587519950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9a34197b20 con 0x7f9a34105bd0 2024-04-23T02:50:17.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.922+0000 7f9a31ffb700 1 -- 172.21.15.71:0/1587519950 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f9a20059b90 con 0x7f9a34105bd0 2024-04-23T02:50:17.925 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:50:17.925 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:50:17.925 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:50:17.925 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:50:17.926 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:50:17.926 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:50:17.926 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:50:17.926 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:50:17.926 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:50:17.926 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:50:17.926 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:50:17.926 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:50:17.926 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:50:17.926 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:50:17.926 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:50:17.926 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:50:17.927 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:50:17.927 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:50:17.927 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:50:17.927 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:50:17.927 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:50:17.927 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:50:17.927 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:50:17.927 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:50:17.927 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:50:17.927 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:50:17.927 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:50:17.927 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:50:17.927 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:50:17.928 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:50:17.928 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:50:17.928 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:50:17.928 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:50:17.928 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:50:17.928 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:50:17.928 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:50:17.928 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:50:17.928 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:50:17.928 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:50:17.928 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:50:17.928 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:50:17.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.924+0000 7f9a3af32700 1 -- 172.21.15.71:0/1587519950 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9a2406cce0 msgr2=0x7f9a2406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:17.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.924+0000 7f9a3af32700 1 --2- 172.21.15.71:0/1587519950 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9a2406cce0 0x7f9a2406f1a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9a28005f50 tx=0x7f9a2801a040 comp rx=0 tx=0).stop 2024-04-23T02:50:17.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.924+0000 7f9a3af32700 1 -- 172.21.15.71:0/1587519950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a34105bd0 msgr2=0x7f9a3419cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:17.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.924+0000 7f9a3af32700 1 --2- 172.21.15.71:0/1587519950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a34105bd0 0x7f9a3419cad0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f9a2000c8a0 tx=0x7f9a2000cbb0 comp rx=0 tx=0).stop 2024-04-23T02:50:17.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.924+0000 7f9a3af32700 1 -- 172.21.15.71:0/1587519950 shutdown_connections 2024-04-23T02:50:17.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.924+0000 7f9a3af32700 1 --2- 172.21.15.71:0/1587519950 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9a2406cce0 0x7f9a2406f1a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:17.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.924+0000 7f9a3af32700 1 --2- 172.21.15.71:0/1587519950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a340ffc20 0x7f9a3419c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:17.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.924+0000 7f9a3af32700 1 --2- 172.21.15.71:0/1587519950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a34105bd0 0x7f9a3419cad0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:17.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.924+0000 7f9a3af32700 1 -- 172.21.15.71:0/1587519950 >> 172.21.15.71:0/1587519950 conn(0x7f9a34075670 msgr2=0x7f9a34068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:17.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.924+0000 7f9a3af32700 1 -- 172.21.15.71:0/1587519950 shutdown_connections 2024-04-23T02:50:17.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:17.924+0000 7f9a3af32700 1 -- 172.21.15.71:0/1587519950 wait complete. 2024-04-23T02:50:17.930 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:50:17.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:17 smithi071 ceph-mon[144544]: from='client.35804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:50:17.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:17 smithi071 ceph-mon[144544]: from='client.35808 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:50:17.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:17 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3504404735' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:50:18.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.031+0000 7f98c9e31700 1 -- 172.21.15.71:0/4184787759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c4100750 msgr2=0x7f98c41047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:18.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.031+0000 7f98c9e31700 1 --2- 172.21.15.71:0/4184787759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c4100750 0x7f98c41047a0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009910 tx=0x7f98b4009c20 comp rx=0 tx=0).stop 2024-04-23T02:50:18.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.031+0000 7f98c9e31700 1 -- 172.21.15.71:0/4184787759 shutdown_connections 2024-04-23T02:50:18.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.031+0000 7f98c9e31700 1 --2- 172.21.15.71:0/4184787759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c4100750 0x7f98c41047a0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:18.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.031+0000 7f98c9e31700 1 --2- 172.21.15.71:0/4184787759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98c40ffda0 0x7f98c4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:18.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.031+0000 7f98c9e31700 1 -- 172.21.15.71:0/4184787759 >> 172.21.15.71:0/4184787759 conn(0x7f98c40fb650 msgr2=0x7f98c40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:18.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.031+0000 7f98c9e31700 1 -- 172.21.15.71:0/4184787759 shutdown_connections 2024-04-23T02:50:18.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.031+0000 7f98c9e31700 1 -- 172.21.15.71:0/4184787759 wait complete. 2024-04-23T02:50:18.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.031+0000 7f98c9e31700 1 Processor -- start 2024-04-23T02:50:18.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.031+0000 7f98c9e31700 1 -- start start 2024-04-23T02:50:18.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.032+0000 7f98c9e31700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98c40ffda0 0x7f98c419c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:18.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.032+0000 7f98c9e31700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c4100750 0x7f98c419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:18.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.032+0000 7f98c9e31700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c419d180 con 0x7f98c4100750 2024-04-23T02:50:18.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.032+0000 7f98c9e31700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c4196630 con 0x7f98c40ffda0 2024-04-23T02:50:18.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.032+0000 7f98c2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c4100750 0x7f98c419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:18.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.032+0000 7f98c2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c4100750 0x7f98c419caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46272/0 (socket says 172.21.15.71:46272) 2024-04-23T02:50:18.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.032+0000 7f98c2ffd700 1 -- 172.21.15.71:0/4106382770 learned_addr learned my addr 172.21.15.71:0/4106382770 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:50:18.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.032+0000 7f98c37fe700 1 --2- 172.21.15.71:0/4106382770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98c40ffda0 0x7f98c419c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:18.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.032+0000 7f98c2ffd700 1 -- 172.21.15.71:0/4106382770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98c40ffda0 msgr2=0x7f98c419c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:18.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.032+0000 7f98c2ffd700 1 --2- 172.21.15.71:0/4106382770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98c40ffda0 0x7f98c419c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:18.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.032+0000 7f98c2ffd700 1 -- 172.21.15.71:0/4106382770 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b40095f0 con 0x7f98c4100750 2024-04-23T02:50:18.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.032+0000 7f98c2ffd700 1 --2- 172.21.15.71:0/4106382770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c4100750 0x7f98c419caf0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f98b40092e0 tx=0x7f98b400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:18.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.032+0000 7f98c0ff9700 1 -- 172.21.15.71:0/4106382770 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98b401d070 con 0x7f98c4100750 2024-04-23T02:50:18.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.032+0000 7f98c0ff9700 1 -- 172.21.15.71:0/4106382770 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f98b4005dc0 con 0x7f98c4100750 2024-04-23T02:50:18.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.032+0000 7f98c9e31700 1 -- 172.21.15.71:0/4106382770 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c41968b0 con 0x7f98c4100750 2024-04-23T02:50:18.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.032+0000 7f98c0ff9700 1 -- 172.21.15.71:0/4106382770 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98b400fea0 con 0x7f98c4100750 2024-04-23T02:50:18.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.032+0000 7f98c9e31700 1 -- 172.21.15.71:0/4106382770 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c4196da0 con 0x7f98c4100750 2024-04-23T02:50:18.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.033+0000 7f98c9e31700 1 -- 172.21.15.71:0/4106382770 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c404f2e0 con 0x7f98c4100750 2024-04-23T02:50:18.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.034+0000 7f98c0ff9700 1 -- 172.21.15.71:0/4106382770 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f98b4003a60 con 0x7f98c4100750 2024-04-23T02:50:18.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.034+0000 7f98c0ff9700 1 --2- 172.21.15.71:0/4106382770 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98ac06cce0 0x7f98ac06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:18.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.034+0000 7f98c0ff9700 1 -- 172.21.15.71:0/4106382770 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f98b408d740 con 0x7f98c4100750 2024-04-23T02:50:18.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.034+0000 7f98c37fe700 1 --2- 172.21.15.71:0/4106382770 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98ac06cce0 0x7f98ac06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:18.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.035+0000 7f98c37fe700 1 --2- 172.21.15.71:0/4106382770 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98ac06cce0 0x7f98ac06f1a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f98b8005950 tx=0x7f98b800b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:18.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.039+0000 7f98c0ff9700 1 -- 172.21.15.71:0/4106382770 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f98b405b240 con 0x7f98c4100750 2024-04-23T02:50:18.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:17 smithi192 ceph-mon[108617]: from='client.35804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:50:18.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:17 smithi192 ceph-mon[108617]: from='client.35808 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:50:18.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:17 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3504404735' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:50:18.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.248+0000 7f98c9e31700 1 -- 172.21.15.71:0/4106382770 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f98c4197020 con 0x7f98ac06cce0 2024-04-23T02:50:18.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.249+0000 7f98c0ff9700 1 -- 172.21.15.71:0/4106382770 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f98c4197020 con 0x7f98ac06cce0 2024-04-23T02:50:18.252 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:50:18.252 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:50:18.252 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:50:18.252 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:50:18.252 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:50:18.252 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:50:18.253 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:50:18.253 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:50:18.253 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:50:18.253 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:50:18.253 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:50:18.253 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:50:18.253 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:50:18.253 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:50:18.253 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:50:18.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.251+0000 7f98c9e31700 1 -- 172.21.15.71:0/4106382770 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98ac06cce0 msgr2=0x7f98ac06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:18.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.251+0000 7f98c9e31700 1 --2- 172.21.15.71:0/4106382770 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98ac06cce0 0x7f98ac06f1a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f98b8005950 tx=0x7f98b800b410 comp rx=0 tx=0).stop 2024-04-23T02:50:18.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.251+0000 7f98c9e31700 1 -- 172.21.15.71:0/4106382770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c4100750 msgr2=0x7f98c419caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:18.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.251+0000 7f98c9e31700 1 --2- 172.21.15.71:0/4106382770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c4100750 0x7f98c419caf0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f98b40092e0 tx=0x7f98b400bd70 comp rx=0 tx=0).stop 2024-04-23T02:50:18.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.251+0000 7f98c9e31700 1 -- 172.21.15.71:0/4106382770 shutdown_connections 2024-04-23T02:50:18.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.251+0000 7f98c9e31700 1 --2- 172.21.15.71:0/4106382770 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f98ac06cce0 0x7f98ac06f1a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:18.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.251+0000 7f98c9e31700 1 --2- 172.21.15.71:0/4106382770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98c40ffda0 0x7f98c419c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:18.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.251+0000 7f98c9e31700 1 --2- 172.21.15.71:0/4106382770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c4100750 0x7f98c419caf0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:18.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.251+0000 7f98c9e31700 1 -- 172.21.15.71:0/4106382770 >> 172.21.15.71:0/4106382770 conn(0x7f98c40fb650 msgr2=0x7f98c4103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:18.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.251+0000 7f98c9e31700 1 -- 172.21.15.71:0/4106382770 shutdown_connections 2024-04-23T02:50:18.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.251+0000 7f98c9e31700 1 -- 172.21.15.71:0/4106382770 wait complete. 2024-04-23T02:50:18.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.354+0000 7f5b17ebf700 1 -- 172.21.15.71:0/2160991482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b100fff50 msgr2=0x7f5b10105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:18.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.354+0000 7f5b17ebf700 1 --2- 172.21.15.71:0/2160991482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b100fff50 0x7f5b10105840 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009910 tx=0x7f5b0c009c20 comp rx=0 tx=0).stop 2024-04-23T02:50:18.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.354+0000 7f5b17ebf700 1 -- 172.21.15.71:0/2160991482 shutdown_connections 2024-04-23T02:50:18.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.354+0000 7f5b17ebf700 1 --2- 172.21.15.71:0/2160991482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b100fff50 0x7f5b10105840 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:18.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.354+0000 7f5b17ebf700 1 --2- 172.21.15.71:0/2160991482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b100ff5a0 0x7f5b100ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:18.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.354+0000 7f5b17ebf700 1 -- 172.21.15.71:0/2160991482 >> 172.21.15.71:0/2160991482 conn(0x7f5b10074e30 msgr2=0x7f5b10075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:18.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.354+0000 7f5b17ebf700 1 -- 172.21.15.71:0/2160991482 shutdown_connections 2024-04-23T02:50:18.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.354+0000 7f5b17ebf700 1 -- 172.21.15.71:0/2160991482 wait complete. 2024-04-23T02:50:18.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.355+0000 7f5b17ebf700 1 Processor -- start 2024-04-23T02:50:18.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.355+0000 7f5b17ebf700 1 -- start start 2024-04-23T02:50:18.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.355+0000 7f5b17ebf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b100ff5a0 0x7f5b1019c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:18.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.355+0000 7f5b17ebf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b100fff50 0x7f5b1019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:18.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.355+0000 7f5b17ebf700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1019d180 con 0x7f5b100fff50 2024-04-23T02:50:18.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.355+0000 7f5b17ebf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b10196630 con 0x7f5b100ff5a0 2024-04-23T02:50:18.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.355+0000 7f5b1545a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b100fff50 0x7f5b1019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:18.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.355+0000 7f5b1545a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b100fff50 0x7f5b1019caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46290/0 (socket says 172.21.15.71:46290) 2024-04-23T02:50:18.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.355+0000 7f5b1545a700 1 -- 172.21.15.71:0/1086888092 learned_addr learned my addr 172.21.15.71:0/1086888092 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:50:18.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.355+0000 7f5b15c5b700 1 --2- 172.21.15.71:0/1086888092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b100ff5a0 0x7f5b1019c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:18.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.355+0000 7f5b1545a700 1 -- 172.21.15.71:0/1086888092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b100ff5a0 msgr2=0x7f5b1019c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:18.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.355+0000 7f5b1545a700 1 --2- 172.21.15.71:0/1086888092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b100ff5a0 0x7f5b1019c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:18.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.355+0000 7f5b1545a700 1 -- 172.21.15.71:0/1086888092 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c0095f0 con 0x7f5b100fff50 2024-04-23T02:50:18.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.356+0000 7f5b1545a700 1 --2- 172.21.15.71:0/1086888092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b100fff50 0x7f5b1019caf0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00bd70 tx=0x7f5b0c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:18.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.356+0000 7f5b06ffd700 1 -- 172.21.15.71:0/1086888092 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b0c01d070 con 0x7f5b100fff50 2024-04-23T02:50:18.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.356+0000 7f5b06ffd700 1 -- 172.21.15.71:0/1086888092 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5b0c00f460 con 0x7f5b100fff50 2024-04-23T02:50:18.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.356+0000 7f5b17ebf700 1 -- 172.21.15.71:0/1086888092 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b101968b0 con 0x7f5b100fff50 2024-04-23T02:50:18.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.356+0000 7f5b06ffd700 1 -- 172.21.15.71:0/1086888092 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b0c021640 con 0x7f5b100fff50 2024-04-23T02:50:18.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.356+0000 7f5b17ebf700 1 -- 172.21.15.71:0/1086888092 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b10196da0 con 0x7f5b100fff50 2024-04-23T02:50:18.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.357+0000 7f5b17ebf700 1 -- 172.21.15.71:0/1086888092 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b10109200 con 0x7f5b100fff50 2024-04-23T02:50:18.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.357+0000 7f5b06ffd700 1 -- 172.21.15.71:0/1086888092 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5b0c00f9e0 con 0x7f5b100fff50 2024-04-23T02:50:18.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.358+0000 7f5b06ffd700 1 --2- 172.21.15.71:0/1086888092 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5afc06cce0 0x7f5afc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:18.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.358+0000 7f5b06ffd700 1 -- 172.21.15.71:0/1086888092 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5b0c08d6c0 con 0x7f5b100fff50 2024-04-23T02:50:18.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.358+0000 7f5b15c5b700 1 --2- 172.21.15.71:0/1086888092 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5afc06cce0 0x7f5afc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:18.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.358+0000 7f5b15c5b700 1 --2- 172.21.15.71:0/1086888092 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5afc06cce0 0x7f5afc06f1a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5b00007be0 tx=0x7f5b000073d0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:18.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.362+0000 7f5b06ffd700 1 -- 172.21.15.71:0/1086888092 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5b0c05b1a0 con 0x7f5b100fff50 2024-04-23T02:50:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.635+0000 7f5b17ebf700 1 -- 172.21.15.71:0/1086888092 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5b10108be0 con 0x7f5b100fff50 2024-04-23T02:50:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.635+0000 7f5b06ffd700 1 -- 172.21.15.71:0/1086888092 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7f5b0c05ad30 con 0x7f5b100fff50 2024-04-23T02:50:18.638 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:50:18.638 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:50:18.639 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:50:18.639 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:50:18.639 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:50:18.639 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:50:18.639 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:50:18.639 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:50:18.639 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:50:18.639 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:50:18.639 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:50:18.639 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:50:18.639 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:50:18.639 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:50:18.639 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:50:18.639 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:50:18.639 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:50:18.640 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:50:18.640 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:50:18.640 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:50:18.640 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:50:18.640 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:50:18.640 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:50:18.640 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:50:18.640 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:50:18.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.637+0000 7f5b17ebf700 1 -- 172.21.15.71:0/1086888092 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5afc06cce0 msgr2=0x7f5afc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:18.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.637+0000 7f5b17ebf700 1 --2- 172.21.15.71:0/1086888092 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5afc06cce0 0x7f5afc06f1a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5b00007be0 tx=0x7f5b000073d0 comp rx=0 tx=0).stop 2024-04-23T02:50:18.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.637+0000 7f5b17ebf700 1 -- 172.21.15.71:0/1086888092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b100fff50 msgr2=0x7f5b1019caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:18.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.637+0000 7f5b17ebf700 1 --2- 172.21.15.71:0/1086888092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b100fff50 0x7f5b1019caf0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00bd70 tx=0x7f5b0c00bda0 comp rx=0 tx=0).stop 2024-04-23T02:50:18.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.637+0000 7f5b17ebf700 1 -- 172.21.15.71:0/1086888092 shutdown_connections 2024-04-23T02:50:18.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.637+0000 7f5b17ebf700 1 --2- 172.21.15.71:0/1086888092 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5afc06cce0 0x7f5afc06f1a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:18.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.637+0000 7f5b17ebf700 1 --2- 172.21.15.71:0/1086888092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b100ff5a0 0x7f5b1019c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:18.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.637+0000 7f5b17ebf700 1 --2- 172.21.15.71:0/1086888092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b100fff50 0x7f5b1019caf0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:18.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.637+0000 7f5b17ebf700 1 -- 172.21.15.71:0/1086888092 >> 172.21.15.71:0/1086888092 conn(0x7f5b10074e30 msgr2=0x7f5b100fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:18.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.637+0000 7f5b17ebf700 1 -- 172.21.15.71:0/1086888092 shutdown_connections 2024-04-23T02:50:18.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:18.637+0000 7f5b17ebf700 1 -- 172.21.15.71:0/1086888092 wait complete. 2024-04-23T02:50:18.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:18 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1587519950' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:50:18.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:18 smithi071 ceph-mon[144544]: from='client.35820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:50:18.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:18 smithi071 ceph-mon[144544]: pgmap v1111: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:18.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:18 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1086888092' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:50:19.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:18 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1587519950' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:50:19.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:18 smithi192 ceph-mon[108617]: from='client.35820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:50:19.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:18 smithi192 ceph-mon[108617]: pgmap v1111: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:19.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:18 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1086888092' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:50:20.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:50:20.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:50:21.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:20 smithi192 ceph-mon[108617]: pgmap v1112: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:21.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:20 smithi071 ceph-mon[144544]: pgmap v1112: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:22.963 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:22 smithi071 ceph-mon[144544]: pgmap v1113: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:23.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:22 smithi192 ceph-mon[108617]: pgmap v1113: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:24.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:24 smithi071 ceph-mon[144544]: pgmap v1114: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:25.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:24 smithi192 ceph-mon[108617]: pgmap v1114: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:26.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:26 smithi071 ceph-mon[144544]: pgmap v1115: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:27.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:26 smithi192 ceph-mon[108617]: pgmap v1115: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:28.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:28 smithi071 ceph-mon[144544]: pgmap v1116: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:29.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:28 smithi192 ceph-mon[108617]: pgmap v1116: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:30.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:30 smithi071 ceph-mon[144544]: pgmap v1117: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:31.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:30 smithi192 ceph-mon[108617]: pgmap v1117: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:32.985 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:32 smithi192 ceph-mon[108617]: pgmap v1118: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:32.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:32 smithi071 ceph-mon[144544]: pgmap v1118: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:34.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:34 smithi071 ceph-mon[144544]: pgmap v1119: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:35.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:34 smithi192 ceph-mon[108617]: pgmap v1119: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:50:36.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:50:36.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:36 smithi071 ceph-mon[144544]: pgmap v1120: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:37.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:36 smithi192 ceph-mon[108617]: pgmap v1120: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:38.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:38 smithi071 ceph-mon[144544]: pgmap v1121: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:39.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:38 smithi192 ceph-mon[108617]: pgmap v1121: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:40.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:40 smithi071 ceph-mon[144544]: pgmap v1122: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:41.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:40 smithi192 ceph-mon[108617]: pgmap v1122: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:42.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:42 smithi071 ceph-mon[144544]: pgmap v1123: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:43.077 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:42 smithi192 ceph-mon[108617]: pgmap v1123: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:44.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:44 smithi071 ceph-mon[144544]: pgmap v1124: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:45.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:44 smithi192 ceph-mon[108617]: pgmap v1124: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:46.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:46 smithi071 ceph-mon[144544]: pgmap v1125: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:47.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:46 smithi192 ceph-mon[108617]: pgmap v1125: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:48.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.755+0000 7fea36184700 1 -- 172.21.15.71:0/3487356899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea300ffc20 msgr2=0x7fea30100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:48.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.755+0000 7fea36184700 1 --2- 172.21.15.71:0/3487356899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea300ffc20 0x7fea30100080 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fea20009910 tx=0x7fea20009c20 comp rx=0 tx=0).stop 2024-04-23T02:50:48.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.755+0000 7fea36184700 1 -- 172.21.15.71:0/3487356899 shutdown_connections 2024-04-23T02:50:48.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.755+0000 7fea36184700 1 --2- 172.21.15.71:0/3487356899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea300ffc20 0x7fea30100080 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:48.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.755+0000 7fea36184700 1 --2- 172.21.15.71:0/3487356899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea30105bd0 0x7fea30105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:48.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.755+0000 7fea36184700 1 -- 172.21.15.71:0/3487356899 >> 172.21.15.71:0/3487356899 conn(0x7fea30075670 msgr2=0x7fea30075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:48.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.755+0000 7fea36184700 1 -- 172.21.15.71:0/3487356899 shutdown_connections 2024-04-23T02:50:48.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.755+0000 7fea36184700 1 -- 172.21.15.71:0/3487356899 wait complete. 2024-04-23T02:50:48.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.756+0000 7fea36184700 1 Processor -- start 2024-04-23T02:50:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.756+0000 7fea36184700 1 -- start start 2024-04-23T02:50:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.756+0000 7fea36184700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea300ffc20 0x7fea30196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.756+0000 7fea36184700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea30105bd0 0x7fea30196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.756+0000 7fea36184700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea30197520 con 0x7fea30105bd0 2024-04-23T02:50:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.756+0000 7fea36184700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3019b330 con 0x7fea300ffc20 2024-04-23T02:50:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.756+0000 7fea2effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea30105bd0 0x7fea30196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.756+0000 7fea2effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea30105bd0 0x7fea30196e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39858/0 (socket says 172.21.15.71:39858) 2024-04-23T02:50:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.756+0000 7fea2effd700 1 -- 172.21.15.71:0/4270764502 learned_addr learned my addr 172.21.15.71:0/4270764502 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:50:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.757+0000 7fea2effd700 1 -- 172.21.15.71:0/4270764502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea300ffc20 msgr2=0x7fea30196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.757+0000 7fea2f7fe700 1 --2- 172.21.15.71:0/4270764502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea300ffc20 0x7fea30196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.757+0000 7fea2effd700 1 --2- 172.21.15.71:0/4270764502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea300ffc20 0x7fea30196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.757+0000 7fea2effd700 1 -- 172.21.15.71:0/4270764502 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea200095f0 con 0x7fea30105bd0 2024-04-23T02:50:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.757+0000 7fea2effd700 1 --2- 172.21.15.71:0/4270764502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea30105bd0 0x7fea30196e90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fea2000b5c0 tx=0x7fea2000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.757+0000 7fea2cff9700 1 -- 172.21.15.71:0/4270764502 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea2001d070 con 0x7fea30105bd0 2024-04-23T02:50:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.757+0000 7fea36184700 1 -- 172.21.15.71:0/4270764502 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3019b5b0 con 0x7fea30105bd0 2024-04-23T02:50:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.757+0000 7fea2cff9700 1 -- 172.21.15.71:0/4270764502 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fea2000f460 con 0x7fea30105bd0 2024-04-23T02:50:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.757+0000 7fea2cff9700 1 -- 172.21.15.71:0/4270764502 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea20021640 con 0x7fea30105bd0 2024-04-23T02:50:48.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.757+0000 7fea36184700 1 -- 172.21.15.71:0/4270764502 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3019baa0 con 0x7fea30105bd0 2024-04-23T02:50:48.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.759+0000 7fea2cff9700 1 -- 172.21.15.71:0/4270764502 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fea2000f9b0 con 0x7fea30105bd0 2024-04-23T02:50:48.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.759+0000 7fea36184700 1 -- 172.21.15.71:0/4270764502 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3004f2e0 con 0x7fea30105bd0 2024-04-23T02:50:48.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.759+0000 7fea2cff9700 1 --2- 172.21.15.71:0/4270764502 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fea1c06cce0 0x7fea1c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:48.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.759+0000 7fea2cff9700 1 -- 172.21.15.71:0/4270764502 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fea2008d970 con 0x7fea30105bd0 2024-04-23T02:50:48.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.759+0000 7fea2f7fe700 1 --2- 172.21.15.71:0/4270764502 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fea1c06cce0 0x7fea1c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:48.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.760+0000 7fea2f7fe700 1 --2- 172.21.15.71:0/4270764502 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fea1c06cce0 0x7fea1c06f1a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fea18006fd0 tx=0x7fea18008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:48.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.764+0000 7fea2cff9700 1 -- 172.21.15.71:0/4270764502 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fea2005b3d0 con 0x7fea30105bd0 2024-04-23T02:50:48.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.974+0000 7fea36184700 1 -- 172.21.15.71:0/4270764502 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fea3019bcc0 con 0x7fea1c06cce0 2024-04-23T02:50:48.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.975+0000 7fea2cff9700 1 -- 172.21.15.71:0/4270764502 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fea3019bcc0 con 0x7fea1c06cce0 2024-04-23T02:50:48.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.977+0000 7fea36184700 1 -- 172.21.15.71:0/4270764502 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fea1c06cce0 msgr2=0x7fea1c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:48.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.977+0000 7fea36184700 1 --2- 172.21.15.71:0/4270764502 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fea1c06cce0 0x7fea1c06f1a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fea18006fd0 tx=0x7fea18008040 comp rx=0 tx=0).stop 2024-04-23T02:50:48.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.977+0000 7fea36184700 1 -- 172.21.15.71:0/4270764502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea30105bd0 msgr2=0x7fea30196e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:48.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.977+0000 7fea36184700 1 --2- 172.21.15.71:0/4270764502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea30105bd0 0x7fea30196e90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fea2000b5c0 tx=0x7fea2000be20 comp rx=0 tx=0).stop 2024-04-23T02:50:48.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.978+0000 7fea36184700 1 -- 172.21.15.71:0/4270764502 shutdown_connections 2024-04-23T02:50:48.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.978+0000 7fea36184700 1 --2- 172.21.15.71:0/4270764502 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fea1c06cce0 0x7fea1c06f1a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:48.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.978+0000 7fea36184700 1 --2- 172.21.15.71:0/4270764502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea300ffc20 0x7fea30196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:48.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.978+0000 7fea36184700 1 --2- 172.21.15.71:0/4270764502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fea30105bd0 0x7fea30196e90 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:48.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.978+0000 7fea36184700 1 -- 172.21.15.71:0/4270764502 >> 172.21.15.71:0/4270764502 conn(0x7fea30075670 msgr2=0x7fea30068740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:48.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.978+0000 7fea36184700 1 -- 172.21.15.71:0/4270764502 shutdown_connections 2024-04-23T02:50:48.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:48.978+0000 7fea36184700 1 -- 172.21.15.71:0/4270764502 wait complete. 2024-04-23T02:50:48.991 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:50:48.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:48 smithi071 ceph-mon[144544]: pgmap v1126: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.082+0000 7f664289b700 1 -- 172.21.15.71:0/2717087497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c0ffb40 msgr2=0x7f663c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.082+0000 7f664289b700 1 --2- 172.21.15.71:0/2717087497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c0ffb40 0x7f663c0fffc0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f6624009910 tx=0x7f6624009c20 comp rx=0 tx=0).stop 2024-04-23T02:50:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.082+0000 7f664289b700 1 -- 172.21.15.71:0/2717087497 shutdown_connections 2024-04-23T02:50:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.082+0000 7f664289b700 1 --2- 172.21.15.71:0/2717087497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c0ffb40 0x7f663c0fffc0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.082+0000 7f664289b700 1 --2- 172.21.15.71:0/2717087497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f663c105bf0 0x7f663c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.082+0000 7f664289b700 1 -- 172.21.15.71:0/2717087497 >> 172.21.15.71:0/2717087497 conn(0x7f663c0fb610 msgr2=0x7f663c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.082+0000 7f664289b700 1 -- 172.21.15.71:0/2717087497 shutdown_connections 2024-04-23T02:50:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.082+0000 7f664289b700 1 -- 172.21.15.71:0/2717087497 wait complete. 2024-04-23T02:50:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.083+0000 7f664289b700 1 Processor -- start 2024-04-23T02:50:49.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.083+0000 7f664289b700 1 -- start start 2024-04-23T02:50:49.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.083+0000 7f664289b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f663c0ffb40 0x7f663c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:49.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.083+0000 7f664289b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c105bf0 0x7f663c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:49.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.083+0000 7f664289b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c19d0d0 con 0x7f663c105bf0 2024-04-23T02:50:49.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.083+0000 7f664289b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c196580 con 0x7f663c0ffb40 2024-04-23T02:50:49.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.083+0000 7f663b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c105bf0 0x7f663c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:49.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.083+0000 7f663b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c105bf0 0x7f663c19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39878/0 (socket says 172.21.15.71:39878) 2024-04-23T02:50:49.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.083+0000 7f663b7fe700 1 -- 172.21.15.71:0/1198164455 learned_addr learned my addr 172.21.15.71:0/1198164455 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:50:49.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.083+0000 7f663bfff700 1 --2- 172.21.15.71:0/1198164455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f663c0ffb40 0x7f663c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:49.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.083+0000 7f663b7fe700 1 -- 172.21.15.71:0/1198164455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f663c0ffb40 msgr2=0x7f663c19c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:49.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.083+0000 7f663b7fe700 1 --2- 172.21.15.71:0/1198164455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f663c0ffb40 0x7f663c19c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:49.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.083+0000 7f663b7fe700 1 -- 172.21.15.71:0/1198164455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66240095f0 con 0x7f663c105bf0 2024-04-23T02:50:49.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.084+0000 7f663bfff700 1 --2- 172.21.15.71:0/1198164455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f663c0ffb40 0x7f663c19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:50:49.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.084+0000 7f663b7fe700 1 --2- 172.21.15.71:0/1198164455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c105bf0 0x7f663c19ca40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6624006010 tx=0x7f662400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:49.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.084+0000 7f66397fa700 1 -- 172.21.15.71:0/1198164455 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f662401d070 con 0x7f663c105bf0 2024-04-23T02:50:49.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.084+0000 7f66397fa700 1 -- 172.21.15.71:0/1198164455 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f662400f460 con 0x7f663c105bf0 2024-04-23T02:50:49.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.084+0000 7f664289b700 1 -- 172.21.15.71:0/1198164455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663c196800 con 0x7f663c105bf0 2024-04-23T02:50:49.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.084+0000 7f66397fa700 1 -- 172.21.15.71:0/1198164455 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6624021640 con 0x7f663c105bf0 2024-04-23T02:50:49.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.084+0000 7f664289b700 1 -- 172.21.15.71:0/1198164455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663c196cf0 con 0x7f663c105bf0 2024-04-23T02:50:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.085+0000 7f664289b700 1 -- 172.21.15.71:0/1198164455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663c04f2e0 con 0x7f663c105bf0 2024-04-23T02:50:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.085+0000 7f66397fa700 1 -- 172.21.15.71:0/1198164455 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f662400fa60 con 0x7f663c105bf0 2024-04-23T02:50:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.085+0000 7f66397fa700 1 --2- 172.21.15.71:0/1198164455 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f662806cce0 0x7f662806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.086+0000 7f66397fa700 1 -- 172.21.15.71:0/1198164455 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f662408d880 con 0x7f663c105bf0 2024-04-23T02:50:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.086+0000 7f663bfff700 1 --2- 172.21.15.71:0/1198164455 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f662806cce0 0x7f662806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.086+0000 7f663bfff700 1 --2- 172.21.15.71:0/1198164455 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f662806cce0 0x7f662806f1a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f662c005950 tx=0x7f662c005f50 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:49.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.090+0000 7f66397fa700 1 -- 172.21.15.71:0/1198164455 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f662405b380 con 0x7f663c105bf0 2024-04-23T02:50:49.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:48 smithi192 ceph-mon[108617]: pgmap v1126: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:49.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.300+0000 7f664289b700 1 -- 172.21.15.71:0/1198164455 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f663c1970b0 con 0x7f662806cce0 2024-04-23T02:50:49.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.301+0000 7f66397fa700 1 -- 172.21.15.71:0/1198164455 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f663c1970b0 con 0x7f662806cce0 2024-04-23T02:50:49.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.303+0000 7f664289b700 1 -- 172.21.15.71:0/1198164455 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f662806cce0 msgr2=0x7f662806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:49.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.303+0000 7f664289b700 1 --2- 172.21.15.71:0/1198164455 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f662806cce0 0x7f662806f1a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f662c005950 tx=0x7f662c005f50 comp rx=0 tx=0).stop 2024-04-23T02:50:49.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.303+0000 7f664289b700 1 -- 172.21.15.71:0/1198164455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c105bf0 msgr2=0x7f663c19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:49.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.303+0000 7f664289b700 1 --2- 172.21.15.71:0/1198164455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c105bf0 0x7f663c19ca40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6624006010 tx=0x7f662400be20 comp rx=0 tx=0).stop 2024-04-23T02:50:49.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.303+0000 7f664289b700 1 -- 172.21.15.71:0/1198164455 shutdown_connections 2024-04-23T02:50:49.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.303+0000 7f664289b700 1 --2- 172.21.15.71:0/1198164455 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f662806cce0 0x7f662806f1a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:49.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.303+0000 7f664289b700 1 --2- 172.21.15.71:0/1198164455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f663c0ffb40 0x7f663c19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:49.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.303+0000 7f664289b700 1 --2- 172.21.15.71:0/1198164455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f663c105bf0 0x7f663c19ca40 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:49.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.303+0000 7f664289b700 1 -- 172.21.15.71:0/1198164455 >> 172.21.15.71:0/1198164455 conn(0x7f663c0fb610 msgr2=0x7f663c103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:49.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.304+0000 7f664289b700 1 -- 172.21.15.71:0/1198164455 shutdown_connections 2024-04-23T02:50:49.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.304+0000 7f664289b700 1 -- 172.21.15.71:0/1198164455 wait complete. 2024-04-23T02:50:49.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.406+0000 7f2123e91700 1 -- 172.21.15.71:0/372411404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f211c100750 msgr2=0x7f211c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:49.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.406+0000 7f2123e91700 1 --2- 172.21.15.71:0/372411404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f211c100750 0x7f211c1047a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f210c009910 tx=0x7f210c009c20 comp rx=0 tx=0).stop 2024-04-23T02:50:49.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.406+0000 7f2123e91700 1 -- 172.21.15.71:0/372411404 shutdown_connections 2024-04-23T02:50:49.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.406+0000 7f2123e91700 1 --2- 172.21.15.71:0/372411404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f211c100750 0x7f211c1047a0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:49.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.406+0000 7f2123e91700 1 --2- 172.21.15.71:0/372411404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f211c0ffda0 0x7f211c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:49.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.406+0000 7f2123e91700 1 -- 172.21.15.71:0/372411404 >> 172.21.15.71:0/372411404 conn(0x7f211c0fb650 msgr2=0x7f211c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:49.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.407+0000 7f2123e91700 1 -- 172.21.15.71:0/372411404 shutdown_connections 2024-04-23T02:50:49.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.407+0000 7f2123e91700 1 -- 172.21.15.71:0/372411404 wait complete. 2024-04-23T02:50:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.407+0000 7f2123e91700 1 Processor -- start 2024-04-23T02:50:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.407+0000 7f2123e91700 1 -- start start 2024-04-23T02:50:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.407+0000 7f2123e91700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f211c0ffda0 0x7f211c101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.407+0000 7f2123e91700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f211c100750 0x7f211c101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.407+0000 7f2123e91700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c1022c0 con 0x7f211c0ffda0 2024-04-23T02:50:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.407+0000 7f2123e91700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c102400 con 0x7f211c100750 2024-04-23T02:50:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.407+0000 7f2121c2d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f211c0ffda0 0x7f211c101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.408+0000 7f2121c2d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f211c0ffda0 0x7f211c101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39888/0 (socket says 172.21.15.71:39888) 2024-04-23T02:50:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.408+0000 7f2121c2d700 1 -- 172.21.15.71:0/232093137 learned_addr learned my addr 172.21.15.71:0/232093137 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:50:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.408+0000 7f2121c2d700 1 -- 172.21.15.71:0/232093137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f211c100750 msgr2=0x7f211c101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:49.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.408+0000 7f2121c2d700 1 --2- 172.21.15.71:0/232093137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f211c100750 0x7f211c101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:49.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.408+0000 7f2121c2d700 1 -- 172.21.15.71:0/232093137 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c0095f0 con 0x7f211c0ffda0 2024-04-23T02:50:49.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.408+0000 7f2121c2d700 1 --2- 172.21.15.71:0/232093137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f211c0ffda0 0x7f211c101840 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f211800d8d0 tx=0x7f211800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:49.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.408+0000 7f2112ffd700 1 -- 172.21.15.71:0/232093137 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2118009940 con 0x7f211c0ffda0 2024-04-23T02:50:49.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.408+0000 7f2112ffd700 1 -- 172.21.15.71:0/232093137 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2118010460 con 0x7f211c0ffda0 2024-04-23T02:50:49.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.408+0000 7f2112ffd700 1 -- 172.21.15.71:0/232093137 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f211800f5f0 con 0x7f211c0ffda0 2024-04-23T02:50:49.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.408+0000 7f2123e91700 1 -- 172.21.15.71:0/232093137 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211c1a29d0 con 0x7f211c0ffda0 2024-04-23T02:50:49.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.408+0000 7f2123e91700 1 -- 172.21.15.71:0/232093137 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c1a2de0 con 0x7f211c0ffda0 2024-04-23T02:50:49.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.409+0000 7f2123e91700 1 -- 172.21.15.71:0/232093137 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f211c04f2e0 con 0x7f211c0ffda0 2024-04-23T02:50:49.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.409+0000 7f2112ffd700 1 -- 172.21.15.71:0/232093137 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f21180105d0 con 0x7f211c0ffda0 2024-04-23T02:50:49.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.410+0000 7f2112ffd700 1 --2- 172.21.15.71:0/232093137 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f210806cce0 0x7f210806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:49.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.410+0000 7f2112ffd700 1 -- 172.21.15.71:0/232093137 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f211808c4f0 con 0x7f211c0ffda0 2024-04-23T02:50:49.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.410+0000 7f212142c700 1 --2- 172.21.15.71:0/232093137 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f210806cce0 0x7f210806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:49.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.410+0000 7f212142c700 1 --2- 172.21.15.71:0/232093137 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f210806cce0 0x7f210806f1a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f210c00b5c0 tx=0x7f210c005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:49.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.414+0000 7f2112ffd700 1 -- 172.21.15.71:0/232093137 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f21180565e0 con 0x7f211c0ffda0 2024-04-23T02:50:49.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.616+0000 7f2123e91700 1 -- 172.21.15.71:0/232093137 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f211c103960 con 0x7f210806cce0 2024-04-23T02:50:49.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.626+0000 7f2112ffd700 1 -- 172.21.15.71:0/232093137 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f211c103960 con 0x7f210806cce0 2024-04-23T02:50:49.629 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:50:49.629 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (34m) 12m ago 45m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:50:49.629 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (46m) 12m ago 46m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:50:49.629 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (44m) 12m ago 44m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:50:49.629 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (32m) 12m ago 46m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:50:49.630 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (32m) 12m ago 44m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:50:49.630 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (34m) 12m ago 45m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:50:49.630 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (28m) 12m ago 39m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:50:49.630 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (28m) 12m ago 39m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:50:49.630 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (12m) 12m ago 39m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:50:49.630 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (28m) 12m ago 39m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:50:49.630 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (36m) 12m ago 47m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:50:49.630 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (35m) 12m ago 44m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:50:49.630 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (32m) 12m ago 47m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:50:49.630 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (32m) 12m ago 44m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:50:49.630 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (35m) 12m ago 45m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:50:49.630 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (35m) 12m ago 44m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:50:49.630 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (31m) 12m ago 43m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:50:49.630 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (31m) 12m ago 43m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:50:49.631 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (30m) 12m ago 42m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:50:49.631 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (30m) 12m ago 42m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:50:49.631 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (29m) 12m ago 41m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:50:49.631 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (28m) 12m ago 41m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:50:49.631 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (34m) 12m ago 45m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:50:49.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.628+0000 7f2123e91700 1 -- 172.21.15.71:0/232093137 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f210806cce0 msgr2=0x7f210806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:49.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.628+0000 7f2123e91700 1 --2- 172.21.15.71:0/232093137 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f210806cce0 0x7f210806f1a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f210c00b5c0 tx=0x7f210c005960 comp rx=0 tx=0).stop 2024-04-23T02:50:49.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.628+0000 7f2123e91700 1 -- 172.21.15.71:0/232093137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f211c0ffda0 msgr2=0x7f211c101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:49.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.628+0000 7f2123e91700 1 --2- 172.21.15.71:0/232093137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f211c0ffda0 0x7f211c101840 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f211800d8d0 tx=0x7f211800dc90 comp rx=0 tx=0).stop 2024-04-23T02:50:49.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.628+0000 7f2123e91700 1 -- 172.21.15.71:0/232093137 shutdown_connections 2024-04-23T02:50:49.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.628+0000 7f2123e91700 1 --2- 172.21.15.71:0/232093137 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f210806cce0 0x7f210806f1a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:49.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.628+0000 7f2123e91700 1 --2- 172.21.15.71:0/232093137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f211c0ffda0 0x7f211c101840 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:49.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.628+0000 7f2123e91700 1 --2- 172.21.15.71:0/232093137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f211c100750 0x7f211c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:49.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.628+0000 7f2123e91700 1 -- 172.21.15.71:0/232093137 >> 172.21.15.71:0/232093137 conn(0x7f211c0fb650 msgr2=0x7f211c0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:49.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.628+0000 7f2123e91700 1 -- 172.21.15.71:0/232093137 shutdown_connections 2024-04-23T02:50:49.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.628+0000 7f2123e91700 1 -- 172.21.15.71:0/232093137 wait complete. 2024-04-23T02:50:49.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.732+0000 7f7518841700 1 -- 172.21.15.71:0/4047690886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7514100e80 msgr2=0x7f7514101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:49.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.732+0000 7f7518841700 1 --2- 172.21.15.71:0/4047690886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7514100e80 0x7f7514101260 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7504009910 tx=0x7f7504009c20 comp rx=0 tx=0).stop 2024-04-23T02:50:49.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.732+0000 7f7518841700 1 -- 172.21.15.71:0/4047690886 shutdown_connections 2024-04-23T02:50:49.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.732+0000 7f7518841700 1 --2- 172.21.15.71:0/4047690886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7514101830 0x7f7514105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:49.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.732+0000 7f7518841700 1 --2- 172.21.15.71:0/4047690886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7514100e80 0x7f7514101260 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:49.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.732+0000 7f7518841700 1 -- 172.21.15.71:0/4047690886 >> 172.21.15.71:0/4047690886 conn(0x7f75140fc730 msgr2=0x7f75140feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:49.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.732+0000 7f7518841700 1 -- 172.21.15.71:0/4047690886 shutdown_connections 2024-04-23T02:50:49.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.732+0000 7f7518841700 1 -- 172.21.15.71:0/4047690886 wait complete. 2024-04-23T02:50:49.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.732+0000 7f7518841700 1 Processor -- start 2024-04-23T02:50:49.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.733+0000 7f7518841700 1 -- start start 2024-04-23T02:50:49.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.733+0000 7f7518841700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7514100e80 0x7f751419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:49.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.733+0000 7f7518841700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7514101830 0x7f751419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:49.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.733+0000 7f7518841700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751419f3c0 con 0x7f7514100e80 2024-04-23T02:50:49.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.733+0000 7f7518841700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7514198870 con 0x7f7514101830 2024-04-23T02:50:49.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.733+0000 7f751259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7514100e80 0x7f751419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:49.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.733+0000 7f751259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7514100e80 0x7f751419e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39906/0 (socket says 172.21.15.71:39906) 2024-04-23T02:50:49.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.733+0000 7f751259c700 1 -- 172.21.15.71:0/3806827632 learned_addr learned my addr 172.21.15.71:0/3806827632 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:50:49.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.733+0000 7f7511d9b700 1 --2- 172.21.15.71:0/3806827632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7514101830 0x7f751419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:49.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.733+0000 7f751259c700 1 -- 172.21.15.71:0/3806827632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7514101830 msgr2=0x7f751419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:49.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.733+0000 7f751259c700 1 --2- 172.21.15.71:0/3806827632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7514101830 0x7f751419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:49.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.733+0000 7f751259c700 1 -- 172.21.15.71:0/3806827632 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75040095f0 con 0x7f7514100e80 2024-04-23T02:50:49.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.733+0000 7f751259c700 1 --2- 172.21.15.71:0/3806827632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7514100e80 0x7f751419e7f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f7504006010 tx=0x7f750400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:49.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.733+0000 7f75037fe700 1 -- 172.21.15.71:0/3806827632 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f750401d070 con 0x7f7514100e80 2024-04-23T02:50:49.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.733+0000 7f7518841700 1 -- 172.21.15.71:0/3806827632 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7514198af0 con 0x7f7514100e80 2024-04-23T02:50:49.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.733+0000 7f75037fe700 1 -- 172.21.15.71:0/3806827632 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f750400f460 con 0x7f7514100e80 2024-04-23T02:50:49.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.733+0000 7f75037fe700 1 -- 172.21.15.71:0/3806827632 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7504021640 con 0x7f7514100e80 2024-04-23T02:50:49.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.734+0000 7f7518841700 1 -- 172.21.15.71:0/3806827632 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7514198fe0 con 0x7f7514100e80 2024-04-23T02:50:49.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.734+0000 7f7518841700 1 -- 172.21.15.71:0/3806827632 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7514105000 con 0x7f7514100e80 2024-04-23T02:50:49.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.735+0000 7f75037fe700 1 -- 172.21.15.71:0/3806827632 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f750400f9e0 con 0x7f7514100e80 2024-04-23T02:50:49.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.735+0000 7f75037fe700 1 --2- 172.21.15.71:0/3806827632 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f74fc06cce0 0x7f74fc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:49.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.735+0000 7f75037fe700 1 -- 172.21.15.71:0/3806827632 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f750408d900 con 0x7f7514100e80 2024-04-23T02:50:49.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.735+0000 7f7511d9b700 1 --2- 172.21.15.71:0/3806827632 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f74fc06cce0 0x7f74fc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:49.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.736+0000 7f7511d9b700 1 --2- 172.21.15.71:0/3806827632 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f74fc06cce0 0x7f74fc06f1a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7508005fd0 tx=0x7f750800a560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:49.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:49.740+0000 7f75037fe700 1 -- 172.21.15.71:0/3806827632 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f750405b400 con 0x7f7514100e80 2024-04-23T02:50:49.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:49 smithi071 ceph-mon[144544]: from='client.35828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:50:49.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:49 smithi071 ceph-mon[144544]: from='client.35832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:50:49.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:49 smithi071 ceph-mon[144544]: from='client.35836 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:50:50.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.017+0000 7f7518841700 1 -- 172.21.15.71:0/3806827632 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f751404f2e0 con 0x7f7514100e80 2024-04-23T02:50:50.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.018+0000 7f75037fe700 1 -- 172.21.15.71:0/3806827632 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f750405af90 con 0x7f7514100e80 2024-04-23T02:50:50.021 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:50:50.021 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:50:50.021 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:50:50.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:50:50.021 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:50:50.021 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:50:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:50:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:50:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:50:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:50:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:50:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:50:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:50:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:50:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:50:50.022 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:50:50.022 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:50:50.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.020+0000 7f7518841700 1 -- 172.21.15.71:0/3806827632 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f74fc06cce0 msgr2=0x7f74fc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:50.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.020+0000 7f7518841700 1 --2- 172.21.15.71:0/3806827632 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f74fc06cce0 0x7f74fc06f1a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7508005fd0 tx=0x7f750800a560 comp rx=0 tx=0).stop 2024-04-23T02:50:50.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.020+0000 7f7518841700 1 -- 172.21.15.71:0/3806827632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7514100e80 msgr2=0x7f751419e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:50.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.020+0000 7f7518841700 1 --2- 172.21.15.71:0/3806827632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7514100e80 0x7f751419e7f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f7504006010 tx=0x7f750400be20 comp rx=0 tx=0).stop 2024-04-23T02:50:50.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.020+0000 7f7518841700 1 -- 172.21.15.71:0/3806827632 shutdown_connections 2024-04-23T02:50:50.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.020+0000 7f7518841700 1 --2- 172.21.15.71:0/3806827632 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f74fc06cce0 0x7f74fc06f1a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.020+0000 7f7518841700 1 --2- 172.21.15.71:0/3806827632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7514100e80 0x7f751419e7f0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.020+0000 7f7518841700 1 --2- 172.21.15.71:0/3806827632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7514101830 0x7f751419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.020+0000 7f7518841700 1 -- 172.21.15.71:0/3806827632 >> 172.21.15.71:0/3806827632 conn(0x7f75140fc730 msgr2=0x7f75140fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:50.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.020+0000 7f7518841700 1 -- 172.21.15.71:0/3806827632 shutdown_connections 2024-04-23T02:50:50.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.020+0000 7f7518841700 1 -- 172.21.15.71:0/3806827632 wait complete. 2024-04-23T02:50:50.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:49 smithi192 ceph-mon[108617]: from='client.35828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:50:50.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:49 smithi192 ceph-mon[108617]: from='client.35832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:50:50.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:49 smithi192 ceph-mon[108617]: from='client.35836 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:50:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.123+0000 7f32a1a86700 1 -- 172.21.15.71:0/1670703150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0ff5a0 msgr2=0x7f329c0ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.123+0000 7f32a1a86700 1 --2- 172.21.15.71:0/1670703150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0ff5a0 0x7f329c0ff980 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3290009910 tx=0x7f3290009c20 comp rx=0 tx=0).stop 2024-04-23T02:50:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.123+0000 7f32a1a86700 1 -- 172.21.15.71:0/1670703150 shutdown_connections 2024-04-23T02:50:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.123+0000 7f32a1a86700 1 --2- 172.21.15.71:0/1670703150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f329c0fff50 0x7f329c105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.123+0000 7f32a1a86700 1 --2- 172.21.15.71:0/1670703150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0ff5a0 0x7f329c0ff980 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.123+0000 7f32a1a86700 1 -- 172.21.15.71:0/1670703150 >> 172.21.15.71:0/1670703150 conn(0x7f329c074e30 msgr2=0x7f329c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.123+0000 7f32a1a86700 1 -- 172.21.15.71:0/1670703150 shutdown_connections 2024-04-23T02:50:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.123+0000 7f32a1a86700 1 -- 172.21.15.71:0/1670703150 wait complete. 2024-04-23T02:50:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.124+0000 7f32a1a86700 1 Processor -- start 2024-04-23T02:50:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.124+0000 7f32a1a86700 1 -- start start 2024-04-23T02:50:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.124+0000 7f32a1a86700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0ff5a0 0x7f329c100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.124+0000 7f32a1a86700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f329c0fff50 0x7f329c100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.124+0000 7f32a1a86700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c100fa0 con 0x7f329c0ff5a0 2024-04-23T02:50:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.124+0000 7f32a1a86700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c1010e0 con 0x7f329c0fff50 2024-04-23T02:50:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.124+0000 7f329b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0ff5a0 0x7f329c100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.124+0000 7f329b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0ff5a0 0x7f329c100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39918/0 (socket says 172.21.15.71:39918) 2024-04-23T02:50:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.124+0000 7f329b7fe700 1 -- 172.21.15.71:0/1414697111 learned_addr learned my addr 172.21.15.71:0/1414697111 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:50:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.124+0000 7f329b7fe700 1 -- 172.21.15.71:0/1414697111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f329c0fff50 msgr2=0x7f329c100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.124+0000 7f329b7fe700 1 --2- 172.21.15.71:0/1414697111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f329c0fff50 0x7f329c100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.124+0000 7f329b7fe700 1 -- 172.21.15.71:0/1414697111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32900095f0 con 0x7f329c0ff5a0 2024-04-23T02:50:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.125+0000 7f329b7fe700 1 --2- 172.21.15.71:0/1414697111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0ff5a0 0x7f329c100520 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3290006010 tx=0x7f329000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.125+0000 7f3298ff9700 1 -- 172.21.15.71:0/1414697111 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f329001d070 con 0x7f329c0ff5a0 2024-04-23T02:50:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.125+0000 7f32a1a86700 1 -- 172.21.15.71:0/1414697111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329c101360 con 0x7f329c0ff5a0 2024-04-23T02:50:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.125+0000 7f3298ff9700 1 -- 172.21.15.71:0/1414697111 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f329000f460 con 0x7f329c0ff5a0 2024-04-23T02:50:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.125+0000 7f3298ff9700 1 -- 172.21.15.71:0/1414697111 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3290021640 con 0x7f329c0ff5a0 2024-04-23T02:50:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.125+0000 7f32a1a86700 1 -- 172.21.15.71:0/1414697111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329c1a2cd0 con 0x7f329c0ff5a0 2024-04-23T02:50:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.126+0000 7f32a1a86700 1 -- 172.21.15.71:0/1414697111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329c109270 con 0x7f329c0ff5a0 2024-04-23T02:50:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.126+0000 7f3298ff9700 1 -- 172.21.15.71:0/1414697111 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f329000f9e0 con 0x7f329c0ff5a0 2024-04-23T02:50:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.127+0000 7f3298ff9700 1 --2- 172.21.15.71:0/1414697111 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f328406cce0 0x7f328406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.127+0000 7f3298ff9700 1 -- 172.21.15.71:0/1414697111 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f329008d900 con 0x7f329c0ff5a0 2024-04-23T02:50:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.127+0000 7f329affd700 1 --2- 172.21.15.71:0/1414697111 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f328406cce0 0x7f328406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.127+0000 7f329affd700 1 --2- 172.21.15.71:0/1414697111 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f328406cce0 0x7f328406f1a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f329c101d10 tx=0x7f328c005c80 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:50.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.131+0000 7f3298ff9700 1 -- 172.21.15.71:0/1414697111 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3290057970 con 0x7f329c0ff5a0 2024-04-23T02:50:50.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.366+0000 7f32a1a86700 1 -- 172.21.15.71:0/1414697111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f329c108c50 con 0x7f329c0ff5a0 2024-04-23T02:50:50.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.366+0000 7f3298ff9700 1 -- 172.21.15.71:0/1414697111 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f329005af90 con 0x7f329c0ff5a0 2024-04-23T02:50:50.370 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:50:50.370 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:50:50.370 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:50:50.370 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:50:50.370 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:50:50.370 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:50:50.370 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:50:50.370 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:50:50.370 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:50:50.370 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:50:50.370 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:50:50.370 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:50:50.370 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:50:50.371 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:50:50.371 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:50:50.371 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:50:50.371 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:50:50.371 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:50:50.371 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:50:50.371 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:50:50.371 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:50:50.371 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:50:50.371 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:50:50.371 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:50:50.371 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:50:50.371 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:50:50.371 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:50:50.371 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:50:50.372 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:50:50.372 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:50:50.372 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:50:50.372 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:50:50.372 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:50:50.372 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:50:50.372 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:50:50.372 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:50:50.372 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:50:50.372 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:50:50.372 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:50:50.372 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:50:50.372 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:50:50.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.368+0000 7f32a1a86700 1 -- 172.21.15.71:0/1414697111 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f328406cce0 msgr2=0x7f328406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:50.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.368+0000 7f32a1a86700 1 --2- 172.21.15.71:0/1414697111 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f328406cce0 0x7f328406f1a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f329c101d10 tx=0x7f328c005c80 comp rx=0 tx=0).stop 2024-04-23T02:50:50.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.368+0000 7f32a1a86700 1 -- 172.21.15.71:0/1414697111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0ff5a0 msgr2=0x7f329c100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:50.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.368+0000 7f32a1a86700 1 --2- 172.21.15.71:0/1414697111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0ff5a0 0x7f329c100520 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3290006010 tx=0x7f329000be20 comp rx=0 tx=0).stop 2024-04-23T02:50:50.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.368+0000 7f32a1a86700 1 -- 172.21.15.71:0/1414697111 shutdown_connections 2024-04-23T02:50:50.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.368+0000 7f32a1a86700 1 --2- 172.21.15.71:0/1414697111 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f328406cce0 0x7f328406f1a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.368+0000 7f32a1a86700 1 --2- 172.21.15.71:0/1414697111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0ff5a0 0x7f329c100520 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.368+0000 7f32a1a86700 1 --2- 172.21.15.71:0/1414697111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f329c0fff50 0x7f329c100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.368+0000 7f32a1a86700 1 -- 172.21.15.71:0/1414697111 >> 172.21.15.71:0/1414697111 conn(0x7f329c074e30 msgr2=0x7f329c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:50.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.369+0000 7f32a1a86700 1 -- 172.21.15.71:0/1414697111 shutdown_connections 2024-04-23T02:50:50.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.369+0000 7f32a1a86700 1 -- 172.21.15.71:0/1414697111 wait complete. 2024-04-23T02:50:50.374 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:50:50.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.472+0000 7fd21edb9700 1 -- 172.21.15.71:0/3843468177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2180ffb40 msgr2=0x7fd2180fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:50.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.472+0000 7fd21edb9700 1 --2- 172.21.15.71:0/3843468177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2180ffb40 0x7fd2180fffc0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd204009910 tx=0x7fd204009c20 comp rx=0 tx=0).stop 2024-04-23T02:50:50.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.472+0000 7fd21edb9700 1 -- 172.21.15.71:0/3843468177 shutdown_connections 2024-04-23T02:50:50.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.472+0000 7fd21edb9700 1 --2- 172.21.15.71:0/3843468177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2180ffb40 0x7fd2180fffc0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.472+0000 7fd21edb9700 1 --2- 172.21.15.71:0/3843468177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd218105bf0 0x7fd218105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.472+0000 7fd21edb9700 1 -- 172.21.15.71:0/3843468177 >> 172.21.15.71:0/3843468177 conn(0x7fd2180fb610 msgr2=0x7fd2180fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:50.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.472+0000 7fd21edb9700 1 -- 172.21.15.71:0/3843468177 shutdown_connections 2024-04-23T02:50:50.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.472+0000 7fd21edb9700 1 -- 172.21.15.71:0/3843468177 wait complete. 2024-04-23T02:50:50.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.472+0000 7fd21edb9700 1 Processor -- start 2024-04-23T02:50:50.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.472+0000 7fd21edb9700 1 -- start start 2024-04-23T02:50:50.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.473+0000 7fd21edb9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2180ffb40 0x7fd21819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:50.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.473+0000 7fd21edb9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd218105bf0 0x7fd21819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:50.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.473+0000 7fd21edb9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21819d0c0 con 0x7fd218105bf0 2024-04-23T02:50:50.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.473+0000 7fd21edb9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd218196570 con 0x7fd2180ffb40 2024-04-23T02:50:50.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.473+0000 7fd217fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd218105bf0 0x7fd21819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:50.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.473+0000 7fd217fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd218105bf0 0x7fd21819ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39942/0 (socket says 172.21.15.71:39942) 2024-04-23T02:50:50.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.473+0000 7fd217fff700 1 -- 172.21.15.71:0/1920628134 learned_addr learned my addr 172.21.15.71:0/1920628134 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:50:50.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.473+0000 7fd21cb55700 1 --2- 172.21.15.71:0/1920628134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2180ffb40 0x7fd21819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:50.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.473+0000 7fd217fff700 1 -- 172.21.15.71:0/1920628134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2180ffb40 msgr2=0x7fd21819c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:50.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.473+0000 7fd217fff700 1 --2- 172.21.15.71:0/1920628134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2180ffb40 0x7fd21819c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.473+0000 7fd217fff700 1 -- 172.21.15.71:0/1920628134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2040095f0 con 0x7fd218105bf0 2024-04-23T02:50:50.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.473+0000 7fd21cb55700 1 --2- 172.21.15.71:0/1920628134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2180ffb40 0x7fd21819c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:50:50.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.473+0000 7fd217fff700 1 --2- 172.21.15.71:0/1920628134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd218105bf0 0x7fd21819ca30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fd20400b5c0 tx=0x7fd20400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:50.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.473+0000 7fd215ffb700 1 -- 172.21.15.71:0/1920628134 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd20401d070 con 0x7fd218105bf0 2024-04-23T02:50:50.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.473+0000 7fd21edb9700 1 -- 172.21.15.71:0/1920628134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2181967f0 con 0x7fd218105bf0 2024-04-23T02:50:50.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.473+0000 7fd215ffb700 1 -- 172.21.15.71:0/1920628134 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd204005e00 con 0x7fd218105bf0 2024-04-23T02:50:50.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.473+0000 7fd215ffb700 1 -- 172.21.15.71:0/1920628134 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd20400fe60 con 0x7fd218105bf0 2024-04-23T02:50:50.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.474+0000 7fd21edb9700 1 -- 172.21.15.71:0/1920628134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd218196ce0 con 0x7fd218105bf0 2024-04-23T02:50:50.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.474+0000 7fd21edb9700 1 -- 172.21.15.71:0/1920628134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21804f2e0 con 0x7fd218105bf0 2024-04-23T02:50:50.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.475+0000 7fd215ffb700 1 -- 172.21.15.71:0/1920628134 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd20400f6d0 con 0x7fd218105bf0 2024-04-23T02:50:50.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.475+0000 7fd215ffb700 1 --2- 172.21.15.71:0/1920628134 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd20806cce0 0x7fd20806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:50.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.475+0000 7fd215ffb700 1 -- 172.21.15.71:0/1920628134 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd20408d740 con 0x7fd218105bf0 2024-04-23T02:50:50.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.475+0000 7fd21cb55700 1 --2- 172.21.15.71:0/1920628134 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd20806cce0 0x7fd20806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:50.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.476+0000 7fd21cb55700 1 --2- 172.21.15.71:0/1920628134 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd20806cce0 0x7fd20806f1a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd20c005950 tx=0x7fd20c0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:50.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.480+0000 7fd215ffb700 1 -- 172.21.15.71:0/1920628134 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd20405b2f0 con 0x7fd218105bf0 2024-04-23T02:50:50.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.690+0000 7fd21edb9700 1 -- 172.21.15.71:0/1920628134 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd218196fc0 con 0x7fd20806cce0 2024-04-23T02:50:50.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.691+0000 7fd215ffb700 1 -- 172.21.15.71:0/1920628134 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd218196fc0 con 0x7fd20806cce0 2024-04-23T02:50:50.694 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:50:50.694 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:50:50.694 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:50:50.694 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:50:50.694 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:50:50.695 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:50:50.695 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:50:50.695 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:50:50.695 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:50:50.695 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:50:50.695 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:50:50.695 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:50:50.695 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:50:50.695 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:50:50.695 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:50:50.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.694+0000 7fd21edb9700 1 -- 172.21.15.71:0/1920628134 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd20806cce0 msgr2=0x7fd20806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:50.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.694+0000 7fd21edb9700 1 --2- 172.21.15.71:0/1920628134 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd20806cce0 0x7fd20806f1a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd20c005950 tx=0x7fd20c0058e0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.694+0000 7fd21edb9700 1 -- 172.21.15.71:0/1920628134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd218105bf0 msgr2=0x7fd21819ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:50.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.694+0000 7fd21edb9700 1 --2- 172.21.15.71:0/1920628134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd218105bf0 0x7fd21819ca30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fd20400b5c0 tx=0x7fd20400be20 comp rx=0 tx=0).stop 2024-04-23T02:50:50.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.694+0000 7fd21edb9700 1 -- 172.21.15.71:0/1920628134 shutdown_connections 2024-04-23T02:50:50.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.694+0000 7fd21edb9700 1 --2- 172.21.15.71:0/1920628134 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd20806cce0 0x7fd20806f1a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.694+0000 7fd21edb9700 1 --2- 172.21.15.71:0/1920628134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2180ffb40 0x7fd21819c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.694+0000 7fd21edb9700 1 --2- 172.21.15.71:0/1920628134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd218105bf0 0x7fd21819ca30 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.694+0000 7fd21edb9700 1 -- 172.21.15.71:0/1920628134 >> 172.21.15.71:0/1920628134 conn(0x7fd2180fb610 msgr2=0x7fd218068f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:50.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.694+0000 7fd21edb9700 1 -- 172.21.15.71:0/1920628134 shutdown_connections 2024-04-23T02:50:50.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.694+0000 7fd21edb9700 1 -- 172.21.15.71:0/1920628134 wait complete. 2024-04-23T02:50:50.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.799+0000 7fab4b281700 1 -- 172.21.15.71:0/541434806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab440ffda0 msgr2=0x7fab44100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:50.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.799+0000 7fab4b281700 1 --2- 172.21.15.71:0/541434806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab440ffda0 0x7fab44100180 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fab40009910 tx=0x7fab40009c20 comp rx=0 tx=0).stop 2024-04-23T02:50:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.799+0000 7fab4b281700 1 -- 172.21.15.71:0/541434806 shutdown_connections 2024-04-23T02:50:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.799+0000 7fab4b281700 1 --2- 172.21.15.71:0/541434806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab44100750 0x7fab441047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.799+0000 7fab4b281700 1 --2- 172.21.15.71:0/541434806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab440ffda0 0x7fab44100180 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.799+0000 7fab4b281700 1 -- 172.21.15.71:0/541434806 >> 172.21.15.71:0/541434806 conn(0x7fab440fb650 msgr2=0x7fab440fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.799+0000 7fab4b281700 1 -- 172.21.15.71:0/541434806 shutdown_connections 2024-04-23T02:50:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.799+0000 7fab4b281700 1 -- 172.21.15.71:0/541434806 wait complete. 2024-04-23T02:50:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.800+0000 7fab4b281700 1 Processor -- start 2024-04-23T02:50:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.800+0000 7fab4b281700 1 -- start start 2024-04-23T02:50:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.800+0000 7fab4b281700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab440ffda0 0x7fab44197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.800+0000 7fab4b281700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab44100750 0x7fab44197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.800+0000 7fab4b281700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab44198180 con 0x7fab44100750 2024-04-23T02:50:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.800+0000 7fab4b281700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab441982f0 con 0x7fab440ffda0 2024-04-23T02:50:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.800+0000 7fab4881c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab44100750 0x7fab44197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.800+0000 7fab4881c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab44100750 0x7fab44197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39970/0 (socket says 172.21.15.71:39970) 2024-04-23T02:50:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.800+0000 7fab4881c700 1 -- 172.21.15.71:0/2699835128 learned_addr learned my addr 172.21.15.71:0/2699835128 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:50:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.801+0000 7fab4901d700 1 --2- 172.21.15.71:0/2699835128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab440ffda0 0x7fab44197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.801+0000 7fab4881c700 1 -- 172.21.15.71:0/2699835128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab440ffda0 msgr2=0x7fab44197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.801+0000 7fab4881c700 1 --2- 172.21.15.71:0/2699835128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab440ffda0 0x7fab44197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.801+0000 7fab4881c700 1 -- 172.21.15.71:0/2699835128 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab400095f0 con 0x7fab44100750 2024-04-23T02:50:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.801+0000 7fab4881c700 1 --2- 172.21.15.71:0/2699835128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab44100750 0x7fab44197b60 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fab3400c8f0 tx=0x7fab3400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.801+0000 7fab3a7fc700 1 -- 172.21.15.71:0/2699835128 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab34007a50 con 0x7fab44100750 2024-04-23T02:50:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.801+0000 7fab3a7fc700 1 -- 172.21.15.71:0/2699835128 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fab34007bb0 con 0x7fab44100750 2024-04-23T02:50:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.801+0000 7fab4b281700 1 -- 172.21.15.71:0/2699835128 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab441a0690 con 0x7fab44100750 2024-04-23T02:50:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.801+0000 7fab3a7fc700 1 -- 172.21.15.71:0/2699835128 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab34018630 con 0x7fab44100750 2024-04-23T02:50:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.801+0000 7fab4b281700 1 -- 172.21.15.71:0/2699835128 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab441a0b80 con 0x7fab44100750 2024-04-23T02:50:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.802+0000 7fab4b281700 1 -- 172.21.15.71:0/2699835128 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab44108140 con 0x7fab44100750 2024-04-23T02:50:50.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.802+0000 7fab3a7fc700 1 -- 172.21.15.71:0/2699835128 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fab34007d20 con 0x7fab44100750 2024-04-23T02:50:50.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.803+0000 7fab3a7fc700 1 --2- 172.21.15.71:0/2699835128 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fab3006cce0 0x7fab3006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:50:50.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.803+0000 7fab3a7fc700 1 -- 172.21.15.71:0/2699835128 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fab3408c590 con 0x7fab44100750 2024-04-23T02:50:50.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.803+0000 7fab4901d700 1 --2- 172.21.15.71:0/2699835128 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fab3006cce0 0x7fab3006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:50:50.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.803+0000 7fab4901d700 1 --2- 172.21.15.71:0/2699835128 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fab3006cce0 0x7fab3006f1a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fab4000b5c0 tx=0x7fab40005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:50:50.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:50.807+0000 7fab3a7fc700 1 -- 172.21.15.71:0/2699835128 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fab34056600 con 0x7fab44100750 2024-04-23T02:50:50.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:50:50.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:50 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3806827632' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:50:50.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:50 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1414697111' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:50:50.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:50 smithi071 ceph-mon[144544]: pgmap v1127: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:51.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:51.079+0000 7fab4b281700 1 -- 172.21.15.71:0/2699835128 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fab4404f2e0 con 0x7fab44100750 2024-04-23T02:50:51.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:51.080+0000 7fab3a7fc700 1 -- 172.21.15.71:0/2699835128 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7fab34059c20 con 0x7fab44100750 2024-04-23T02:50:51.083 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:50:51.083 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:50:51.083 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:50:51.083 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:50:51.083 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:50:51.083 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:50:51.083 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:50:51.083 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:50:51.084 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:50:51.084 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:50:51.084 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:50:51.084 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:50:51.084 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:50:51.084 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:50:51.084 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:50:51.084 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:50:51.084 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:50:51.084 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:50:51.084 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:50:51.084 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:50:51.084 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:50:51.084 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:50:51.085 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:50:51.085 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:50:51.085 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:50:51.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:51.082+0000 7fab4b281700 1 -- 172.21.15.71:0/2699835128 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fab3006cce0 msgr2=0x7fab3006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:51.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:51.082+0000 7fab4b281700 1 --2- 172.21.15.71:0/2699835128 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fab3006cce0 0x7fab3006f1a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fab4000b5c0 tx=0x7fab40005fd0 comp rx=0 tx=0).stop 2024-04-23T02:50:51.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:51.082+0000 7fab4b281700 1 -- 172.21.15.71:0/2699835128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab44100750 msgr2=0x7fab44197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:50:51.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:51.082+0000 7fab4b281700 1 --2- 172.21.15.71:0/2699835128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab44100750 0x7fab44197b60 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fab3400c8f0 tx=0x7fab3400ccb0 comp rx=0 tx=0).stop 2024-04-23T02:50:51.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:51.082+0000 7fab4b281700 1 -- 172.21.15.71:0/2699835128 shutdown_connections 2024-04-23T02:50:51.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:51.082+0000 7fab4b281700 1 --2- 172.21.15.71:0/2699835128 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fab3006cce0 0x7fab3006f1a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:51.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:51.082+0000 7fab4b281700 1 --2- 172.21.15.71:0/2699835128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab440ffda0 0x7fab44197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:51.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:51.082+0000 7fab4b281700 1 --2- 172.21.15.71:0/2699835128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab44100750 0x7fab44197b60 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:50:51.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:51.082+0000 7fab4b281700 1 -- 172.21.15.71:0/2699835128 >> 172.21.15.71:0/2699835128 conn(0x7fab440fb650 msgr2=0x7fab440fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:50:51.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:51.082+0000 7fab4b281700 1 -- 172.21.15.71:0/2699835128 shutdown_connections 2024-04-23T02:50:51.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:50:51.082+0000 7fab4b281700 1 -- 172.21.15.71:0/2699835128 wait complete. 2024-04-23T02:50:51.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:50:51.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:50 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3806827632' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:50:51.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:50 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1414697111' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:50:51.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:50 smithi192 ceph-mon[108617]: pgmap v1127: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:51.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:51 smithi071 ceph-mon[144544]: from='client.35848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:50:51.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:51 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2699835128' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:50:52.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:51 smithi192 ceph-mon[108617]: from='client.35848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:50:52.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:51 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2699835128' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:50:52.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:52 smithi071 ceph-mon[144544]: pgmap v1128: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:53.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:52 smithi192 ceph-mon[108617]: pgmap v1128: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:54.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:54 smithi071 ceph-mon[144544]: pgmap v1129: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:55.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:54 smithi192 ceph-mon[108617]: pgmap v1129: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:56.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:56 smithi071 ceph-mon[144544]: pgmap v1130: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:57.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:56 smithi192 ceph-mon[108617]: pgmap v1130: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:58.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:50:58 smithi071 ceph-mon[144544]: pgmap v1131: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:50:59.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:50:58 smithi192 ceph-mon[108617]: pgmap v1131: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:00.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:00 smithi071 ceph-mon[144544]: pgmap v1132: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:01.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:00 smithi192 ceph-mon[108617]: pgmap v1132: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:02.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:02 smithi071 ceph-mon[144544]: pgmap v1133: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:03.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:02 smithi192 ceph-mon[108617]: pgmap v1133: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:04.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:04 smithi071 ceph-mon[144544]: pgmap v1134: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:05.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:04 smithi192 ceph-mon[108617]: pgmap v1134: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:05.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:51:06.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:51:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:06 smithi071 ceph-mon[144544]: pgmap v1135: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:07.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:06 smithi192 ceph-mon[108617]: pgmap v1135: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:08.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:08 smithi071 ceph-mon[144544]: pgmap v1136: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:09.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:08 smithi192 ceph-mon[108617]: pgmap v1136: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:10.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:10 smithi071 ceph-mon[144544]: pgmap v1137: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:11.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:10 smithi192 ceph-mon[108617]: pgmap v1137: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:12.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:12 smithi071 ceph-mon[144544]: pgmap v1138: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:13.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:12 smithi192 ceph-mon[108617]: pgmap v1138: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:14.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:14 smithi071 ceph-mon[144544]: pgmap v1139: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:15.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:14 smithi192 ceph-mon[108617]: pgmap v1139: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:16.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:16 smithi071 ceph-mon[144544]: pgmap v1140: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:17.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:16 smithi192 ceph-mon[108617]: pgmap v1140: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:18.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:18 smithi071 ceph-mon[144544]: pgmap v1141: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:19.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:18 smithi192 ceph-mon[108617]: pgmap v1141: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:20.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:51:20.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:51:21.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:20 smithi192 ceph-mon[108617]: pgmap v1142: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:21.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.198+0000 7f6101a62700 1 -- 172.21.15.71:0/258174758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc101e00 msgr2=0x7f60fc1021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:21.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.198+0000 7f6101a62700 1 --2- 172.21.15.71:0/258174758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc101e00 0x7f60fc1021e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00b0a0 tx=0x7f60ec00b3b0 comp rx=0 tx=0).stop 2024-04-23T02:51:21.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.198+0000 7f6101a62700 1 -- 172.21.15.71:0/258174758 shutdown_connections 2024-04-23T02:51:21.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.198+0000 7f6101a62700 1 --2- 172.21.15.71:0/258174758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60fc102720 0x7f60fc10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:21.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.198+0000 7f6101a62700 1 --2- 172.21.15.71:0/258174758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc101e00 0x7f60fc1021e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:21.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.198+0000 7f6101a62700 1 -- 172.21.15.71:0/258174758 >> 172.21.15.71:0/258174758 conn(0x7f60fc0fb650 msgr2=0x7f60fc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:21.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.198+0000 7f6101a62700 1 -- 172.21.15.71:0/258174758 shutdown_connections 2024-04-23T02:51:21.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.198+0000 7f6101a62700 1 -- 172.21.15.71:0/258174758 wait complete. 2024-04-23T02:51:21.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.199+0000 7f6101a62700 1 Processor -- start 2024-04-23T02:51:21.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.199+0000 7f6101a62700 1 -- start start 2024-04-23T02:51:21.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.199+0000 7f6101a62700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60fc101e00 0x7f60fc0ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:21.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.199+0000 7f6101a62700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc102720 0x7f60fc0fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:21.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.199+0000 7f6101a62700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc1004e0 con 0x7f60fc102720 2024-04-23T02:51:21.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.199+0000 7f6101a62700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc100620 con 0x7f60fc101e00 2024-04-23T02:51:21.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.199+0000 7f60faffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc102720 0x7f60fc0fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:21.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.199+0000 7f60faffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc102720 0x7f60fc0fffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57782/0 (socket says 172.21.15.71:57782) 2024-04-23T02:51:21.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.199+0000 7f60faffd700 1 -- 172.21.15.71:0/2959974517 learned_addr learned my addr 172.21.15.71:0/2959974517 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:51:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.200+0000 7f60faffd700 1 -- 172.21.15.71:0/2959974517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60fc101e00 msgr2=0x7f60fc0ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.200+0000 7f60faffd700 1 --2- 172.21.15.71:0/2959974517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60fc101e00 0x7f60fc0ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.200+0000 7f60faffd700 1 -- 172.21.15.71:0/2959974517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec009d00 con 0x7f60fc102720 2024-04-23T02:51:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.200+0000 7f60faffd700 1 --2- 172.21.15.71:0/2959974517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc102720 0x7f60fc0fffa0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f60f000c8f0 tx=0x7f60f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.200+0000 7f60f8ff9700 1 -- 172.21.15.71:0/2959974517 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60f0007ad0 con 0x7f60fc102720 2024-04-23T02:51:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.200+0000 7f6101a62700 1 -- 172.21.15.71:0/2959974517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60fc1a2a90 con 0x7f60fc102720 2024-04-23T02:51:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.200+0000 7f60f8ff9700 1 -- 172.21.15.71:0/2959974517 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f60f000ce90 con 0x7f60fc102720 2024-04-23T02:51:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.200+0000 7f60f8ff9700 1 -- 172.21.15.71:0/2959974517 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60f0018630 con 0x7f60fc102720 2024-04-23T02:51:21.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.200+0000 7f6101a62700 1 -- 172.21.15.71:0/2959974517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60fc1a2ea0 con 0x7f60fc102720 2024-04-23T02:51:21.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.201+0000 7f6101a62700 1 -- 172.21.15.71:0/2959974517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60fc108310 con 0x7f60fc102720 2024-04-23T02:51:21.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.202+0000 7f60f8ff9700 1 -- 172.21.15.71:0/2959974517 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f60f0007c30 con 0x7f60fc102720 2024-04-23T02:51:21.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.202+0000 7f60f8ff9700 1 --2- 172.21.15.71:0/2959974517 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f60e406cce0 0x7f60e406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:21.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.202+0000 7f60f8ff9700 1 -- 172.21.15.71:0/2959974517 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f60f008c560 con 0x7f60fc102720 2024-04-23T02:51:21.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.202+0000 7f60fb7fe700 1 --2- 172.21.15.71:0/2959974517 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f60e406cce0 0x7f60e406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:21.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.203+0000 7f60fb7fe700 1 --2- 172.21.15.71:0/2959974517 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f60e406cce0 0x7f60e406f1a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009480 tx=0x7f60ec006000 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:21.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.207+0000 7f60f8ff9700 1 -- 172.21.15.71:0/2959974517 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f60f00565d0 con 0x7f60fc102720 2024-04-23T02:51:21.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:20 smithi071 ceph-mon[144544]: pgmap v1142: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:21.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.418+0000 7f6101a62700 1 -- 172.21.15.71:0/2959974517 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f60fc0611d0 con 0x7f60e406cce0 2024-04-23T02:51:21.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.419+0000 7f60f8ff9700 1 -- 172.21.15.71:0/2959974517 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f60fc0611d0 con 0x7f60e406cce0 2024-04-23T02:51:21.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.421+0000 7f6101a62700 1 -- 172.21.15.71:0/2959974517 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f60e406cce0 msgr2=0x7f60e406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:21.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.421+0000 7f6101a62700 1 --2- 172.21.15.71:0/2959974517 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f60e406cce0 0x7f60e406f1a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009480 tx=0x7f60ec006000 comp rx=0 tx=0).stop 2024-04-23T02:51:21.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.421+0000 7f6101a62700 1 -- 172.21.15.71:0/2959974517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc102720 msgr2=0x7f60fc0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:21.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.421+0000 7f6101a62700 1 --2- 172.21.15.71:0/2959974517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc102720 0x7f60fc0fffa0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f60f000c8f0 tx=0x7f60f000ccb0 comp rx=0 tx=0).stop 2024-04-23T02:51:21.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.421+0000 7f6101a62700 1 -- 172.21.15.71:0/2959974517 shutdown_connections 2024-04-23T02:51:21.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.422+0000 7f6101a62700 1 --2- 172.21.15.71:0/2959974517 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f60e406cce0 0x7f60e406f1a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:21.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.422+0000 7f6101a62700 1 --2- 172.21.15.71:0/2959974517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60fc101e00 0x7f60fc0ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:21.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.422+0000 7f6101a62700 1 --2- 172.21.15.71:0/2959974517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60fc102720 0x7f60fc0fffa0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:21.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.422+0000 7f6101a62700 1 -- 172.21.15.71:0/2959974517 >> 172.21.15.71:0/2959974517 conn(0x7f60fc0fb650 msgr2=0x7f60fc105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:21.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.422+0000 7f6101a62700 1 -- 172.21.15.71:0/2959974517 shutdown_connections 2024-04-23T02:51:21.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.422+0000 7f6101a62700 1 -- 172.21.15.71:0/2959974517 wait complete. 2024-04-23T02:51:21.435 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:51:21.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.526+0000 7f5446142700 1 -- 172.21.15.71:0/1215833294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54400ffda0 msgr2=0x7f5440100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:21.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.526+0000 7f5446142700 1 --2- 172.21.15.71:0/1215833294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54400ffda0 0x7f5440100180 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f5434009910 tx=0x7f5434009c20 comp rx=0 tx=0).stop 2024-04-23T02:51:21.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.526+0000 7f5446142700 1 -- 172.21.15.71:0/1215833294 shutdown_connections 2024-04-23T02:51:21.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.526+0000 7f5446142700 1 --2- 172.21.15.71:0/1215833294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5440100750 0x7f54401047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:21.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.526+0000 7f5446142700 1 --2- 172.21.15.71:0/1215833294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54400ffda0 0x7f5440100180 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:21.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.526+0000 7f5446142700 1 -- 172.21.15.71:0/1215833294 >> 172.21.15.71:0/1215833294 conn(0x7f54400fb650 msgr2=0x7f54400fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:21.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.526+0000 7f5446142700 1 -- 172.21.15.71:0/1215833294 shutdown_connections 2024-04-23T02:51:21.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.526+0000 7f5446142700 1 -- 172.21.15.71:0/1215833294 wait complete. 2024-04-23T02:51:21.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.526+0000 7f5446142700 1 Processor -- start 2024-04-23T02:51:21.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.527+0000 7f5446142700 1 -- start start 2024-04-23T02:51:21.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.527+0000 7f5446142700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54400ffda0 0x7f54401956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:21.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.527+0000 7f5446142700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5440100750 0x7f5440195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:21.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.527+0000 7f5446142700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5440196290 con 0x7f5440100750 2024-04-23T02:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.527+0000 7f5446142700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5440198fc0 con 0x7f54400ffda0 2024-04-23T02:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.527+0000 7f543effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5440100750 0x7f5440195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.527+0000 7f543effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5440100750 0x7f5440195c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57788/0 (socket says 172.21.15.71:57788) 2024-04-23T02:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.527+0000 7f543effd700 1 -- 172.21.15.71:0/2202135764 learned_addr learned my addr 172.21.15.71:0/2202135764 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.527+0000 7f543f7fe700 1 --2- 172.21.15.71:0/2202135764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54400ffda0 0x7f54401956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.527+0000 7f543effd700 1 -- 172.21.15.71:0/2202135764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54400ffda0 msgr2=0x7f54401956c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.527+0000 7f543effd700 1 --2- 172.21.15.71:0/2202135764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54400ffda0 0x7f54401956c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.527+0000 7f543effd700 1 -- 172.21.15.71:0/2202135764 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54340095f0 con 0x7f5440100750 2024-04-23T02:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.527+0000 7f543effd700 1 --2- 172.21.15.71:0/2202135764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5440100750 0x7f5440195c00 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f543000d900 tx=0x7f543000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.527+0000 7f543cff9700 1 -- 172.21.15.71:0/2202135764 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5430009900 con 0x7f5440100750 2024-04-23T02:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.527+0000 7f5446142700 1 -- 172.21.15.71:0/2202135764 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54401992a0 con 0x7f5440100750 2024-04-23T02:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.527+0000 7f543cff9700 1 -- 172.21.15.71:0/2202135764 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5430010460 con 0x7f5440100750 2024-04-23T02:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.527+0000 7f543cff9700 1 -- 172.21.15.71:0/2202135764 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f543000f5f0 con 0x7f5440100750 2024-04-23T02:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.528+0000 7f5446142700 1 -- 172.21.15.71:0/2202135764 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54401997f0 con 0x7f5440100750 2024-04-23T02:51:21.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.528+0000 7f5446142700 1 -- 172.21.15.71:0/2202135764 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5440108140 con 0x7f5440100750 2024-04-23T02:51:21.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.529+0000 7f543cff9700 1 -- 172.21.15.71:0/2202135764 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f54300105d0 con 0x7f5440100750 2024-04-23T02:51:21.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.529+0000 7f543cff9700 1 --2- 172.21.15.71:0/2202135764 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f542806cce0 0x7f542806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:21.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.529+0000 7f543cff9700 1 -- 172.21.15.71:0/2202135764 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f543008c460 con 0x7f5440100750 2024-04-23T02:51:21.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.529+0000 7f543f7fe700 1 --2- 172.21.15.71:0/2202135764 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f542806cce0 0x7f542806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:21.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.530+0000 7f543f7fe700 1 --2- 172.21.15.71:0/2202135764 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f542806cce0 0x7f542806f1a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5434006010 tx=0x7f543401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:21.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.534+0000 7f543cff9700 1 -- 172.21.15.71:0/2202135764 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5430056430 con 0x7f5440100750 2024-04-23T02:51:21.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.743+0000 7f5446142700 1 -- 172.21.15.71:0/2202135764 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5440196b00 con 0x7f542806cce0 2024-04-23T02:51:21.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.744+0000 7f543cff9700 1 -- 172.21.15.71:0/2202135764 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5440196b00 con 0x7f542806cce0 2024-04-23T02:51:21.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.746+0000 7f5446142700 1 -- 172.21.15.71:0/2202135764 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f542806cce0 msgr2=0x7f542806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:21.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.746+0000 7f5446142700 1 --2- 172.21.15.71:0/2202135764 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f542806cce0 0x7f542806f1a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5434006010 tx=0x7f543401a040 comp rx=0 tx=0).stop 2024-04-23T02:51:21.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.746+0000 7f5446142700 1 -- 172.21.15.71:0/2202135764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5440100750 msgr2=0x7f5440195c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:21.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.746+0000 7f5446142700 1 --2- 172.21.15.71:0/2202135764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5440100750 0x7f5440195c00 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f543000d900 tx=0x7f543000dcc0 comp rx=0 tx=0).stop 2024-04-23T02:51:21.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.746+0000 7f5446142700 1 -- 172.21.15.71:0/2202135764 shutdown_connections 2024-04-23T02:51:21.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.746+0000 7f5446142700 1 --2- 172.21.15.71:0/2202135764 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f542806cce0 0x7f542806f1a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:21.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.746+0000 7f5446142700 1 --2- 172.21.15.71:0/2202135764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54400ffda0 0x7f54401956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:21.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.746+0000 7f5446142700 1 --2- 172.21.15.71:0/2202135764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5440100750 0x7f5440195c00 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:21.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.746+0000 7f5446142700 1 -- 172.21.15.71:0/2202135764 >> 172.21.15.71:0/2202135764 conn(0x7f54400fb650 msgr2=0x7f54400fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:21.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.746+0000 7f5446142700 1 -- 172.21.15.71:0/2202135764 shutdown_connections 2024-04-23T02:51:21.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.746+0000 7f5446142700 1 -- 172.21.15.71:0/2202135764 wait complete. 2024-04-23T02:51:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.850+0000 7fc6551af700 1 -- 172.21.15.71:0/2277696548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc650105bd0 msgr2=0x7fc650105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.850+0000 7fc6551af700 1 --2- 172.21.15.71:0/2277696548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc650105bd0 0x7fc650105fb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc644009910 tx=0x7fc644009c20 comp rx=0 tx=0).stop 2024-04-23T02:51:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.850+0000 7fc6551af700 1 -- 172.21.15.71:0/2277696548 shutdown_connections 2024-04-23T02:51:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.850+0000 7fc6551af700 1 --2- 172.21.15.71:0/2277696548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6500ffc20 0x7fc650100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.850+0000 7fc6551af700 1 --2- 172.21.15.71:0/2277696548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc650105bd0 0x7fc650105fb0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.850+0000 7fc6551af700 1 -- 172.21.15.71:0/2277696548 >> 172.21.15.71:0/2277696548 conn(0x7fc650075670 msgr2=0x7fc650075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.850+0000 7fc6551af700 1 -- 172.21.15.71:0/2277696548 shutdown_connections 2024-04-23T02:51:21.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.850+0000 7fc6551af700 1 -- 172.21.15.71:0/2277696548 wait complete. 2024-04-23T02:51:21.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.851+0000 7fc6551af700 1 Processor -- start 2024-04-23T02:51:21.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.851+0000 7fc6551af700 1 -- start start 2024-04-23T02:51:21.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.851+0000 7fc6551af700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6500ffc20 0x7fc6501975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:21.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.851+0000 7fc6551af700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc650105bd0 0x7fc650197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:21.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.851+0000 7fc6551af700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6501980b0 con 0x7fc6500ffc20 2024-04-23T02:51:21.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.851+0000 7fc6551af700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc650198220 con 0x7fc650105bd0 2024-04-23T02:51:21.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.851+0000 7fc64ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6500ffc20 0x7fc6501975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:21.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.851+0000 7fc64ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6500ffc20 0x7fc6501975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57794/0 (socket says 172.21.15.71:57794) 2024-04-23T02:51:21.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.851+0000 7fc64ed9d700 1 -- 172.21.15.71:0/3131034906 learned_addr learned my addr 172.21.15.71:0/3131034906 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:51:21.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.851+0000 7fc64e59c700 1 --2- 172.21.15.71:0/3131034906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc650105bd0 0x7fc650197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:21.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.851+0000 7fc64ed9d700 1 -- 172.21.15.71:0/3131034906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc650105bd0 msgr2=0x7fc650197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:21.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.851+0000 7fc64ed9d700 1 --2- 172.21.15.71:0/3131034906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc650105bd0 0x7fc650197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:21.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.851+0000 7fc64ed9d700 1 -- 172.21.15.71:0/3131034906 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6440095f0 con 0x7fc6500ffc20 2024-04-23T02:51:21.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.852+0000 7fc64ed9d700 1 --2- 172.21.15.71:0/3131034906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6500ffc20 0x7fc6501975e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc644006010 tx=0x7fc64400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:21.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.852+0000 7fc637fff700 1 -- 172.21.15.71:0/3131034906 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc64401d070 con 0x7fc6500ffc20 2024-04-23T02:51:21.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.852+0000 7fc637fff700 1 -- 172.21.15.71:0/3131034906 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc64400f460 con 0x7fc6500ffc20 2024-04-23T02:51:21.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.852+0000 7fc6551af700 1 -- 172.21.15.71:0/3131034906 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6501a0650 con 0x7fc6500ffc20 2024-04-23T02:51:21.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.852+0000 7fc637fff700 1 -- 172.21.15.71:0/3131034906 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc644021640 con 0x7fc6500ffc20 2024-04-23T02:51:21.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.852+0000 7fc6551af700 1 -- 172.21.15.71:0/3131034906 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6501a0a00 con 0x7fc6500ffc20 2024-04-23T02:51:21.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.853+0000 7fc6551af700 1 -- 172.21.15.71:0/3131034906 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6501080f0 con 0x7fc6500ffc20 2024-04-23T02:51:21.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.853+0000 7fc637fff700 1 -- 172.21.15.71:0/3131034906 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc64400fa20 con 0x7fc6500ffc20 2024-04-23T02:51:21.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.854+0000 7fc637fff700 1 --2- 172.21.15.71:0/3131034906 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc63806cce0 0x7fc63806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:21.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.854+0000 7fc637fff700 1 -- 172.21.15.71:0/3131034906 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc64408d8c0 con 0x7fc6500ffc20 2024-04-23T02:51:21.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.854+0000 7fc64e59c700 1 --2- 172.21.15.71:0/3131034906 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc63806cce0 0x7fc63806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:21.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.854+0000 7fc64e59c700 1 --2- 172.21.15.71:0/3131034906 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc63806cce0 0x7fc63806f1a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc650068e50 tx=0x7fc640005e10 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:21.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:21.858+0000 7fc637fff700 1 -- 172.21.15.71:0/3131034906 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc644057930 con 0x7fc6500ffc20 2024-04-23T02:51:22.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.064+0000 7fc6551af700 1 -- 172.21.15.71:0/3131034906 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6500611d0 con 0x7fc63806cce0 2024-04-23T02:51:22.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.074+0000 7fc637fff700 1 -- 172.21.15.71:0/3131034906 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc6500611d0 con 0x7fc63806cce0 2024-04-23T02:51:22.077 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:51:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (35m) 13m ago 46m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:51:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (46m) 13m ago 46m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:51:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (45m) 13m ago 45m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:51:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (32m) 13m ago 46m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:51:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (32m) 13m ago 45m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:51:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (34m) 13m ago 46m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:51:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (29m) 13m ago 40m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:51:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (28m) 13m ago 40m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:51:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (13m) 13m ago 40m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:51:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (28m) 13m ago 40m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:51:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (37m) 13m ago 47m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:51:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (36m) 13m ago 45m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:51:22.078 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (33m) 13m ago 48m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:51:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (33m) 13m ago 45m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:51:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (36m) 13m ago 46m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:51:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (35m) 13m ago 45m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:51:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (32m) 13m ago 44m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:51:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (31m) 13m ago 43m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:51:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (31m) 13m ago 43m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:51:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (30m) 13m ago 42m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:51:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (30m) 13m ago 42m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:51:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (29m) 13m ago 41m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:51:22.079 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (35m) 13m ago 45m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:51:22.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.076+0000 7fc6551af700 1 -- 172.21.15.71:0/3131034906 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc63806cce0 msgr2=0x7fc63806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:22.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.076+0000 7fc6551af700 1 --2- 172.21.15.71:0/3131034906 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc63806cce0 0x7fc63806f1a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc650068e50 tx=0x7fc640005e10 comp rx=0 tx=0).stop 2024-04-23T02:51:22.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.076+0000 7fc6551af700 1 -- 172.21.15.71:0/3131034906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6500ffc20 msgr2=0x7fc6501975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:22.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.076+0000 7fc6551af700 1 --2- 172.21.15.71:0/3131034906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6500ffc20 0x7fc6501975e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc644006010 tx=0x7fc64400be20 comp rx=0 tx=0).stop 2024-04-23T02:51:22.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.076+0000 7fc6551af700 1 -- 172.21.15.71:0/3131034906 shutdown_connections 2024-04-23T02:51:22.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.076+0000 7fc6551af700 1 --2- 172.21.15.71:0/3131034906 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc63806cce0 0x7fc63806f1a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.076+0000 7fc6551af700 1 --2- 172.21.15.71:0/3131034906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6500ffc20 0x7fc6501975e0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.076+0000 7fc6551af700 1 --2- 172.21.15.71:0/3131034906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc650105bd0 0x7fc650197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.076+0000 7fc6551af700 1 -- 172.21.15.71:0/3131034906 >> 172.21.15.71:0/3131034906 conn(0x7fc650075670 msgr2=0x7fc6500fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:22.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.077+0000 7fc6551af700 1 -- 172.21.15.71:0/3131034906 shutdown_connections 2024-04-23T02:51:22.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.077+0000 7fc6551af700 1 -- 172.21.15.71:0/3131034906 wait complete. 2024-04-23T02:51:22.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:21 smithi192 ceph-mon[108617]: from='client.35856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:51:22.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.178+0000 7f3cf2531700 1 -- 172.21.15.71:0/2381217187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cec101850 msgr2=0x7f3cec1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:22.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.178+0000 7f3cf2531700 1 --2- 172.21.15.71:0/2381217187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cec101850 0x7f3cec1058a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009910 tx=0x7f3ce0009c20 comp rx=0 tx=0).stop 2024-04-23T02:51:22.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.178+0000 7f3cf2531700 1 -- 172.21.15.71:0/2381217187 shutdown_connections 2024-04-23T02:51:22.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.178+0000 7f3cf2531700 1 --2- 172.21.15.71:0/2381217187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cec101850 0x7f3cec1058a0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.178+0000 7f3cf2531700 1 --2- 172.21.15.71:0/2381217187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cec100ea0 0x7f3cec101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.178+0000 7f3cf2531700 1 -- 172.21.15.71:0/2381217187 >> 172.21.15.71:0/2381217187 conn(0x7f3cec0fc750 msgr2=0x7f3cec0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:22.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.178+0000 7f3cf2531700 1 -- 172.21.15.71:0/2381217187 shutdown_connections 2024-04-23T02:51:22.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.178+0000 7f3cf2531700 1 -- 172.21.15.71:0/2381217187 wait complete. 2024-04-23T02:51:22.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.178+0000 7f3cf2531700 1 Processor -- start 2024-04-23T02:51:22.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.178+0000 7f3cf2531700 1 -- start start 2024-04-23T02:51:22.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.179+0000 7f3cf2531700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cec100ea0 0x7f3cec19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:22.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.179+0000 7f3cf2531700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cec101850 0x7f3cec19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:22.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.179+0000 7f3cf2531700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec19d180 con 0x7f3cec100ea0 2024-04-23T02:51:22.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.179+0000 7f3cf2531700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec1966c0 con 0x7f3cec101850 2024-04-23T02:51:22.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.179+0000 7f3cebfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cec100ea0 0x7f3cec19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:22.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.179+0000 7f3cebfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cec100ea0 0x7f3cec19c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57798/0 (socket says 172.21.15.71:57798) 2024-04-23T02:51:22.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.179+0000 7f3cebfff700 1 -- 172.21.15.71:0/1143899526 learned_addr learned my addr 172.21.15.71:0/1143899526 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:51:22.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.179+0000 7f3ceb7fe700 1 --2- 172.21.15.71:0/1143899526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cec101850 0x7f3cec19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:22.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.179+0000 7f3cebfff700 1 -- 172.21.15.71:0/1143899526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cec101850 msgr2=0x7f3cec19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:22.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.179+0000 7f3cebfff700 1 --2- 172.21.15.71:0/1143899526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cec101850 0x7f3cec19cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.179+0000 7f3cebfff700 1 -- 172.21.15.71:0/1143899526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce00095f0 con 0x7f3cec100ea0 2024-04-23T02:51:22.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.179+0000 7f3cebfff700 1 --2- 172.21.15.71:0/1143899526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cec100ea0 0x7f3cec19c640 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00c930 tx=0x7f3cdc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:22.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.179+0000 7f3ce97fa700 1 -- 172.21.15.71:0/1143899526 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cdc007ad0 con 0x7f3cec100ea0 2024-04-23T02:51:22.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.180+0000 7f3ce97fa700 1 -- 172.21.15.71:0/1143899526 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3cdc007c30 con 0x7f3cec100ea0 2024-04-23T02:51:22.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.180+0000 7f3cf2531700 1 -- 172.21.15.71:0/1143899526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cec1969a0 con 0x7f3cec100ea0 2024-04-23T02:51:22.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.180+0000 7f3ce97fa700 1 -- 172.21.15.71:0/1143899526 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cdc018630 con 0x7f3cec100ea0 2024-04-23T02:51:22.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.180+0000 7f3cf2531700 1 -- 172.21.15.71:0/1143899526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cec196ef0 con 0x7f3cec100ea0 2024-04-23T02:51:22.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.181+0000 7f3cf2531700 1 -- 172.21.15.71:0/1143899526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cec1092a0 con 0x7f3cec100ea0 2024-04-23T02:51:22.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.181+0000 7f3ce97fa700 1 -- 172.21.15.71:0/1143899526 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3cdc00f450 con 0x7f3cec100ea0 2024-04-23T02:51:22.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.181+0000 7f3ce97fa700 1 --2- 172.21.15.71:0/1143899526 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3cd406cce0 0x7f3cd406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:22.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.181+0000 7f3ce97fa700 1 -- 172.21.15.71:0/1143899526 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3cdc08c780 con 0x7f3cec100ea0 2024-04-23T02:51:22.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.181+0000 7f3ceb7fe700 1 --2- 172.21.15.71:0/1143899526 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3cd406cce0 0x7f3cd406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:22.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.182+0000 7f3ceb7fe700 1 --2- 172.21.15.71:0/1143899526 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3cd406cce0 0x7f3cd406f1a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3cec197b30 tx=0x7f3ce001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:22.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.186+0000 7f3ce97fa700 1 -- 172.21.15.71:0/1143899526 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3cdc05a3b0 con 0x7f3cec100ea0 2024-04-23T02:51:22.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:21 smithi071 ceph-mon[144544]: from='client.35856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:51:22.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.461+0000 7f3cf2531700 1 -- 172.21.15.71:0/1143899526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3cec04f2e0 con 0x7f3cec100ea0 2024-04-23T02:51:22.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.462+0000 7f3ce97fa700 1 -- 172.21.15.71:0/1143899526 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f3cdc08ae60 con 0x7f3cec100ea0 2024-04-23T02:51:22.465 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:51:22.465 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:51:22.465 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:51:22.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:51:22.465 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:51:22.465 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:51:22.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:51:22.465 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:51:22.466 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:51:22.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:51:22.466 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:51:22.466 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:51:22.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:51:22.466 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:51:22.466 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:51:22.466 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:51:22.466 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:51:22.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.464+0000 7f3cf2531700 1 -- 172.21.15.71:0/1143899526 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3cd406cce0 msgr2=0x7f3cd406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:22.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.464+0000 7f3cf2531700 1 --2- 172.21.15.71:0/1143899526 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3cd406cce0 0x7f3cd406f1a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3cec197b30 tx=0x7f3ce001a040 comp rx=0 tx=0).stop 2024-04-23T02:51:22.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.464+0000 7f3cf2531700 1 -- 172.21.15.71:0/1143899526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cec100ea0 msgr2=0x7f3cec19c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:22.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.464+0000 7f3cf2531700 1 --2- 172.21.15.71:0/1143899526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cec100ea0 0x7f3cec19c640 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00c930 tx=0x7f3cdc00ccf0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.464+0000 7f3cf2531700 1 -- 172.21.15.71:0/1143899526 shutdown_connections 2024-04-23T02:51:22.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.464+0000 7f3cf2531700 1 --2- 172.21.15.71:0/1143899526 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3cd406cce0 0x7f3cd406f1a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.464+0000 7f3cf2531700 1 --2- 172.21.15.71:0/1143899526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cec100ea0 0x7f3cec19c640 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.464+0000 7f3cf2531700 1 --2- 172.21.15.71:0/1143899526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cec101850 0x7f3cec19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.464+0000 7f3cf2531700 1 -- 172.21.15.71:0/1143899526 >> 172.21.15.71:0/1143899526 conn(0x7f3cec0fc750 msgr2=0x7f3cec0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:22.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.464+0000 7f3cf2531700 1 -- 172.21.15.71:0/1143899526 shutdown_connections 2024-04-23T02:51:22.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.464+0000 7f3cf2531700 1 -- 172.21.15.71:0/1143899526 wait complete. 2024-04-23T02:51:22.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.567+0000 7f99f6529700 1 -- 172.21.15.71:0/3516123890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f0100ea0 msgr2=0x7f99f0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:22.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.567+0000 7f99f6529700 1 --2- 172.21.15.71:0/3516123890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f0100ea0 0x7f99f0101280 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009910 tx=0x7f99e0009c20 comp rx=0 tx=0).stop 2024-04-23T02:51:22.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.567+0000 7f99f6529700 1 -- 172.21.15.71:0/3516123890 shutdown_connections 2024-04-23T02:51:22.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.567+0000 7f99f6529700 1 --2- 172.21.15.71:0/3516123890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99f0101850 0x7f99f01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.567+0000 7f99f6529700 1 --2- 172.21.15.71:0/3516123890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f0100ea0 0x7f99f0101280 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.567+0000 7f99f6529700 1 -- 172.21.15.71:0/3516123890 >> 172.21.15.71:0/3516123890 conn(0x7f99f00fc750 msgr2=0x7f99f00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:22.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.567+0000 7f99f6529700 1 -- 172.21.15.71:0/3516123890 shutdown_connections 2024-04-23T02:51:22.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.568+0000 7f99f6529700 1 -- 172.21.15.71:0/3516123890 wait complete. 2024-04-23T02:51:22.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.568+0000 7f99f6529700 1 Processor -- start 2024-04-23T02:51:22.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.568+0000 7f99f6529700 1 -- start start 2024-04-23T02:51:22.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.568+0000 7f99f6529700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f0100ea0 0x7f99f019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:22.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.568+0000 7f99f6529700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99f0101850 0x7f99f019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:22.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.568+0000 7f99f6529700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f019d130 con 0x7f99f0100ea0 2024-04-23T02:51:22.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.568+0000 7f99f6529700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f01965e0 con 0x7f99f0101850 2024-04-23T02:51:22.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.569+0000 7f99effff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f0100ea0 0x7f99f019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:22.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.569+0000 7f99effff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f0100ea0 0x7f99f019c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57820/0 (socket says 172.21.15.71:57820) 2024-04-23T02:51:22.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.569+0000 7f99effff700 1 -- 172.21.15.71:0/3155609176 learned_addr learned my addr 172.21.15.71:0/3155609176 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:51:22.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.569+0000 7f99ef7fe700 1 --2- 172.21.15.71:0/3155609176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99f0101850 0x7f99f019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:22.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.569+0000 7f99effff700 1 -- 172.21.15.71:0/3155609176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99f0101850 msgr2=0x7f99f019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:22.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.569+0000 7f99effff700 1 --2- 172.21.15.71:0/3155609176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99f0101850 0x7f99f019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.569+0000 7f99effff700 1 -- 172.21.15.71:0/3155609176 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e00095f0 con 0x7f99f0100ea0 2024-04-23T02:51:22.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.569+0000 7f99effff700 1 --2- 172.21.15.71:0/3155609176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f0100ea0 0x7f99f019c560 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f99e0006010 tx=0x7f99e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:22.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.569+0000 7f99ed7fa700 1 -- 172.21.15.71:0/3155609176 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99e001d070 con 0x7f99f0100ea0 2024-04-23T02:51:22.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.569+0000 7f99ed7fa700 1 -- 172.21.15.71:0/3155609176 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f99e000f460 con 0x7f99f0100ea0 2024-04-23T02:51:22.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.569+0000 7f99f6529700 1 -- 172.21.15.71:0/3155609176 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f0196860 con 0x7f99f0100ea0 2024-04-23T02:51:22.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.569+0000 7f99ed7fa700 1 -- 172.21.15.71:0/3155609176 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99e0021620 con 0x7f99f0100ea0 2024-04-23T02:51:22.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.569+0000 7f99f6529700 1 -- 172.21.15.71:0/3155609176 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f0196d50 con 0x7f99f0100ea0 2024-04-23T02:51:22.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.570+0000 7f99f6529700 1 -- 172.21.15.71:0/3155609176 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f0109240 con 0x7f99f0100ea0 2024-04-23T02:51:22.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.571+0000 7f99ed7fa700 1 -- 172.21.15.71:0/3155609176 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f99e000fa40 con 0x7f99f0100ea0 2024-04-23T02:51:22.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.571+0000 7f99ed7fa700 1 --2- 172.21.15.71:0/3155609176 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f99d806cce0 0x7f99d806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:22.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.571+0000 7f99ed7fa700 1 -- 172.21.15.71:0/3155609176 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f99e008d790 con 0x7f99f0100ea0 2024-04-23T02:51:22.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.571+0000 7f99ef7fe700 1 --2- 172.21.15.71:0/3155609176 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f99d806cce0 0x7f99d806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:22.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.571+0000 7f99ef7fe700 1 --2- 172.21.15.71:0/3155609176 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f99d806cce0 0x7f99d806f1a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f99e4005950 tx=0x7f99e40058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:22.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.576+0000 7f99ed7fa700 1 -- 172.21.15.71:0/3155609176 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f99e005b290 con 0x7f99f0100ea0 2024-04-23T02:51:22.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.810+0000 7f99f6529700 1 -- 172.21.15.71:0/3155609176 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f99f004f2e0 con 0x7f99f0100ea0 2024-04-23T02:51:22.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.811+0000 7f99ed7fa700 1 -- 172.21.15.71:0/3155609176 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f99e005ae20 con 0x7f99f0100ea0 2024-04-23T02:51:22.814 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:51:22.814 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:51:22.814 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:51:22.814 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:51:22.814 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:51:22.814 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:51:22.814 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:51:22.814 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:51:22.814 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:51:22.815 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:51:22.815 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:51:22.815 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:51:22.815 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:51:22.815 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:51:22.815 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:51:22.815 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:51:22.815 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:51:22.815 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:51:22.815 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:51:22.815 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:51:22.815 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:51:22.815 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:51:22.815 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:51:22.816 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:51:22.816 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:51:22.816 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:51:22.816 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:51:22.816 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:51:22.816 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:51:22.816 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:51:22.816 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:51:22.816 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:51:22.816 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:51:22.816 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:51:22.816 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:51:22.816 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:51:22.816 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:51:22.816 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:51:22.817 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:51:22.817 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:51:22.817 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:51:22.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.813+0000 7f99f6529700 1 -- 172.21.15.71:0/3155609176 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f99d806cce0 msgr2=0x7f99d806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:22.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.813+0000 7f99f6529700 1 --2- 172.21.15.71:0/3155609176 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f99d806cce0 0x7f99d806f1a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f99e4005950 tx=0x7f99e40058e0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.813+0000 7f99f6529700 1 -- 172.21.15.71:0/3155609176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f0100ea0 msgr2=0x7f99f019c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:22.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.813+0000 7f99f6529700 1 --2- 172.21.15.71:0/3155609176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f0100ea0 0x7f99f019c560 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f99e0006010 tx=0x7f99e000be20 comp rx=0 tx=0).stop 2024-04-23T02:51:22.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.813+0000 7f99f6529700 1 -- 172.21.15.71:0/3155609176 shutdown_connections 2024-04-23T02:51:22.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.813+0000 7f99f6529700 1 --2- 172.21.15.71:0/3155609176 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f99d806cce0 0x7f99d806f1a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.813+0000 7f99f6529700 1 --2- 172.21.15.71:0/3155609176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f0100ea0 0x7f99f019c560 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.813+0000 7f99f6529700 1 --2- 172.21.15.71:0/3155609176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99f0101850 0x7f99f019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.813+0000 7f99f6529700 1 -- 172.21.15.71:0/3155609176 >> 172.21.15.71:0/3155609176 conn(0x7f99f00fc750 msgr2=0x7f99f00fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:22.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.813+0000 7f99f6529700 1 -- 172.21.15.71:0/3155609176 shutdown_connections 2024-04-23T02:51:22.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.813+0000 7f99f6529700 1 -- 172.21.15.71:0/3155609176 wait complete. 2024-04-23T02:51:22.819 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:51:22.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.918+0000 7fa3639e5700 1 -- 172.21.15.71:0/4254051198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c068ea0 msgr2=0x7fa35c10d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:22.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.918+0000 7fa3639e5700 1 --2- 172.21.15.71:0/4254051198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c068ea0 0x7fa35c10d290 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa358009910 tx=0x7fa358009c20 comp rx=0 tx=0).stop 2024-04-23T02:51:22.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.918+0000 7fa3639e5700 1 -- 172.21.15.71:0/4254051198 shutdown_connections 2024-04-23T02:51:22.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.918+0000 7fa3639e5700 1 --2- 172.21.15.71:0/4254051198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c068ea0 0x7fa35c10d290 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.918+0000 7fa3639e5700 1 --2- 172.21.15.71:0/4254051198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c068580 0x7fa35c068960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.918+0000 7fa3639e5700 1 -- 172.21.15.71:0/4254051198 >> 172.21.15.71:0/4254051198 conn(0x7fa35c075670 msgr2=0x7fa35c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:22.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.918+0000 7fa3639e5700 1 -- 172.21.15.71:0/4254051198 shutdown_connections 2024-04-23T02:51:22.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.918+0000 7fa3639e5700 1 -- 172.21.15.71:0/4254051198 wait complete. 2024-04-23T02:51:22.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.919+0000 7fa3639e5700 1 Processor -- start 2024-04-23T02:51:22.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.919+0000 7fa3639e5700 1 -- start start 2024-04-23T02:51:22.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.919+0000 7fa3639e5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c068580 0x7fa35c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:22.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.919+0000 7fa3639e5700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c068ea0 0x7fa35c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:22.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.919+0000 7fa3639e5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c19f450 con 0x7fa35c068580 2024-04-23T02:51:22.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.919+0000 7fa3639e5700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c198900 con 0x7fa35c068ea0 2024-04-23T02:51:22.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.919+0000 7fa361781700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c068580 0x7fa35c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:22.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.919+0000 7fa361781700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c068580 0x7fa35c19e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57844/0 (socket says 172.21.15.71:57844) 2024-04-23T02:51:22.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.919+0000 7fa361781700 1 -- 172.21.15.71:0/3934493116 learned_addr learned my addr 172.21.15.71:0/3934493116 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:51:22.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.919+0000 7fa360f80700 1 --2- 172.21.15.71:0/3934493116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c068ea0 0x7fa35c19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:22.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.919+0000 7fa361781700 1 -- 172.21.15.71:0/3934493116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c068ea0 msgr2=0x7fa35c19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:22.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.919+0000 7fa361781700 1 --2- 172.21.15.71:0/3934493116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c068ea0 0x7fa35c19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:22.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.919+0000 7fa361781700 1 -- 172.21.15.71:0/3934493116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3580095f0 con 0x7fa35c068580 2024-04-23T02:51:22.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.919+0000 7fa360f80700 1 --2- 172.21.15.71:0/3934493116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c068ea0 0x7fa35c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:51:22.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.919+0000 7fa361781700 1 --2- 172.21.15.71:0/3934493116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c068580 0x7fa35c19e880 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00b4b0 tx=0x7fa34c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:22.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.919+0000 7fa3527fc700 1 -- 172.21.15.71:0/3934493116 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa34c011800 con 0x7fa35c068580 2024-04-23T02:51:22.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.920+0000 7fa3527fc700 1 -- 172.21.15.71:0/3934493116 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa34c011e00 con 0x7fa35c068580 2024-04-23T02:51:22.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.920+0000 7fa3527fc700 1 -- 172.21.15.71:0/3934493116 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa34c010e50 con 0x7fa35c068580 2024-04-23T02:51:22.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.920+0000 7fa3639e5700 1 -- 172.21.15.71:0/3934493116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35c198be0 con 0x7fa35c068580 2024-04-23T02:51:22.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.920+0000 7fa3639e5700 1 -- 172.21.15.71:0/3934493116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35c1990e0 con 0x7fa35c068580 2024-04-23T02:51:22.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.920+0000 7fa3639e5700 1 -- 172.21.15.71:0/3934493116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35c10aa00 con 0x7fa35c068580 2024-04-23T02:51:22.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.921+0000 7fa3527fc700 1 -- 172.21.15.71:0/3934493116 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa34c010380 con 0x7fa35c068580 2024-04-23T02:51:22.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.921+0000 7fa3527fc700 1 --2- 172.21.15.71:0/3934493116 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa34806cce0 0x7fa34806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:22.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.921+0000 7fa3527fc700 1 -- 172.21.15.71:0/3934493116 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa34c08c1b0 con 0x7fa35c068580 2024-04-23T02:51:22.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.921+0000 7fa360f80700 1 --2- 172.21.15.71:0/3934493116 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa34806cce0 0x7fa34806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:22.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.922+0000 7fa360f80700 1 --2- 172.21.15.71:0/3934493116 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa34806cce0 0x7fa34806f1a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa3580059d0 tx=0x7fa35801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:22.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:22.926+0000 7fa3527fc700 1 -- 172.21.15.71:0/3934493116 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa34c059cb0 con 0x7fa35c068580 2024-04-23T02:51:22.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:22 smithi071 ceph-mon[144544]: from='client.35860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:51:22.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:22 smithi071 ceph-mon[144544]: from='client.35864 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:51:22.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:22 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1143899526' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:51:22.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:22 smithi071 ceph-mon[144544]: pgmap v1143: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:22.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:22 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3155609176' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:51:23.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:22 smithi192 ceph-mon[108617]: from='client.35860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:51:23.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:22 smithi192 ceph-mon[108617]: from='client.35864 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:51:23.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:22 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1143899526' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:51:23.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:22 smithi192 ceph-mon[108617]: pgmap v1143: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:23.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:22 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3155609176' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:51:23.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.138+0000 7fa3639e5700 1 -- 172.21.15.71:0/3934493116 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa35c1993b0 con 0x7fa34806cce0 2024-04-23T02:51:23.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.139+0000 7fa3527fc700 1 -- 172.21.15.71:0/3934493116 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa35c1993b0 con 0x7fa34806cce0 2024-04-23T02:51:23.142 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:51:23.142 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:51:23.142 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:51:23.142 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:51:23.142 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:51:23.142 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:51:23.142 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:51:23.142 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:51:23.142 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:51:23.142 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:51:23.143 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:51:23.143 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:51:23.143 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:51:23.143 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:51:23.143 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:51:23.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.141+0000 7fa3639e5700 1 -- 172.21.15.71:0/3934493116 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa34806cce0 msgr2=0x7fa34806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:23.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.141+0000 7fa3639e5700 1 --2- 172.21.15.71:0/3934493116 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa34806cce0 0x7fa34806f1a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa3580059d0 tx=0x7fa35801a040 comp rx=0 tx=0).stop 2024-04-23T02:51:23.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.141+0000 7fa3639e5700 1 -- 172.21.15.71:0/3934493116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c068580 msgr2=0x7fa35c19e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:23.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.141+0000 7fa3639e5700 1 --2- 172.21.15.71:0/3934493116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c068580 0x7fa35c19e880 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00b4b0 tx=0x7fa34c00b870 comp rx=0 tx=0).stop 2024-04-23T02:51:23.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.141+0000 7fa3639e5700 1 -- 172.21.15.71:0/3934493116 shutdown_connections 2024-04-23T02:51:23.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.141+0000 7fa3639e5700 1 --2- 172.21.15.71:0/3934493116 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa34806cce0 0x7fa34806f1a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:23.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.141+0000 7fa3639e5700 1 --2- 172.21.15.71:0/3934493116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa35c068580 0x7fa35c19e880 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:23.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.141+0000 7fa3639e5700 1 --2- 172.21.15.71:0/3934493116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35c068ea0 0x7fa35c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:23.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.141+0000 7fa3639e5700 1 -- 172.21.15.71:0/3934493116 >> 172.21.15.71:0/3934493116 conn(0x7fa35c075670 msgr2=0x7fa35c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:23.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.141+0000 7fa3639e5700 1 -- 172.21.15.71:0/3934493116 shutdown_connections 2024-04-23T02:51:23.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.141+0000 7fa3639e5700 1 -- 172.21.15.71:0/3934493116 wait complete. 2024-04-23T02:51:23.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.244+0000 7fcfd8d4d700 1 -- 172.21.15.71:0/3014990836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4105bd0 msgr2=0x7fcfd4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:23.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.244+0000 7fcfd8d4d700 1 --2- 172.21.15.71:0/3014990836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4105bd0 0x7fcfd4105fb0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009910 tx=0x7fcfc8009c20 comp rx=0 tx=0).stop 2024-04-23T02:51:23.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.244+0000 7fcfd8d4d700 1 -- 172.21.15.71:0/3014990836 shutdown_connections 2024-04-23T02:51:23.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.244+0000 7fcfd8d4d700 1 --2- 172.21.15.71:0/3014990836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd40ffc20 0x7fcfd4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:23.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.244+0000 7fcfd8d4d700 1 --2- 172.21.15.71:0/3014990836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4105bd0 0x7fcfd4105fb0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:23.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.244+0000 7fcfd8d4d700 1 -- 172.21.15.71:0/3014990836 >> 172.21.15.71:0/3014990836 conn(0x7fcfd4075670 msgr2=0x7fcfd4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:23.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.244+0000 7fcfd8d4d700 1 -- 172.21.15.71:0/3014990836 shutdown_connections 2024-04-23T02:51:23.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.244+0000 7fcfd8d4d700 1 -- 172.21.15.71:0/3014990836 wait complete. 2024-04-23T02:51:23.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.244+0000 7fcfd8d4d700 1 Processor -- start 2024-04-23T02:51:23.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.245+0000 7fcfd8d4d700 1 -- start start 2024-04-23T02:51:23.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.245+0000 7fcfd8d4d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd40ffc20 0x7fcfd4101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:23.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.245+0000 7fcfd8d4d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd4105bd0 0x7fcfd4101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:23.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.245+0000 7fcfd8d4d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd41022c0 con 0x7fcfd40ffc20 2024-04-23T02:51:23.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.245+0000 7fcfd8d4d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd4102430 con 0x7fcfd4105bd0 2024-04-23T02:51:23.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.245+0000 7fcfd259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd40ffc20 0x7fcfd4101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:23.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.245+0000 7fcfd259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd40ffc20 0x7fcfd4101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57870/0 (socket says 172.21.15.71:57870) 2024-04-23T02:51:23.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.245+0000 7fcfd259c700 1 -- 172.21.15.71:0/1587407124 learned_addr learned my addr 172.21.15.71:0/1587407124 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:51:23.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.245+0000 7fcfd1d9b700 1 --2- 172.21.15.71:0/1587407124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd4105bd0 0x7fcfd4101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:23.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.245+0000 7fcfd259c700 1 -- 172.21.15.71:0/1587407124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd4105bd0 msgr2=0x7fcfd4101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:23.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.245+0000 7fcfd259c700 1 --2- 172.21.15.71:0/1587407124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd4105bd0 0x7fcfd4101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:23.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.245+0000 7fcfd259c700 1 -- 172.21.15.71:0/1587407124 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc80095f0 con 0x7fcfd40ffc20 2024-04-23T02:51:23.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.245+0000 7fcfd1d9b700 1 --2- 172.21.15.71:0/1587407124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd4105bd0 0x7fcfd4101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:51:23.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.245+0000 7fcfd259c700 1 --2- 172.21.15.71:0/1587407124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd40ffc20 0x7fcfd4101840 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8000c00 tx=0x7fcfc800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:23.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.245+0000 7fcfc37fe700 1 -- 172.21.15.71:0/1587407124 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc801d070 con 0x7fcfd40ffc20 2024-04-23T02:51:23.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.245+0000 7fcfc37fe700 1 -- 172.21.15.71:0/1587407124 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcfc8005dc0 con 0x7fcfd40ffc20 2024-04-23T02:51:23.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.246+0000 7fcfd8d4d700 1 -- 172.21.15.71:0/1587407124 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd41026b0 con 0x7fcfd40ffc20 2024-04-23T02:51:23.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.246+0000 7fcfc37fe700 1 -- 172.21.15.71:0/1587407124 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfc80039b0 con 0x7fcfd40ffc20 2024-04-23T02:51:23.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.246+0000 7fcfd8d4d700 1 -- 172.21.15.71:0/1587407124 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd41a2c50 con 0x7fcfd40ffc20 2024-04-23T02:51:23.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.246+0000 7fcfd8d4d700 1 -- 172.21.15.71:0/1587407124 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd404f2e0 con 0x7fcfd40ffc20 2024-04-23T02:51:23.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.247+0000 7fcfc37fe700 1 -- 172.21.15.71:0/1587407124 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fcfc8003ba0 con 0x7fcfd40ffc20 2024-04-23T02:51:23.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.247+0000 7fcfc37fe700 1 --2- 172.21.15.71:0/1587407124 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcfbc06cce0 0x7fcfbc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:23.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.247+0000 7fcfc37fe700 1 -- 172.21.15.71:0/1587407124 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcfc808d9e0 con 0x7fcfd40ffc20 2024-04-23T02:51:23.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.247+0000 7fcfd1d9b700 1 --2- 172.21.15.71:0/1587407124 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcfbc06cce0 0x7fcfbc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:23.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.248+0000 7fcfd1d9b700 1 --2- 172.21.15.71:0/1587407124 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcfbc06cce0 0x7fcfbc06f1a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4102fc0 tx=0x7fcfc4009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:23.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.252+0000 7fcfc37fe700 1 -- 172.21.15.71:0/1587407124 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fcfc80579f0 con 0x7fcfd40ffc20 2024-04-23T02:51:23.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.522+0000 7fcfd8d4d700 1 -- 172.21.15.71:0/1587407124 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcfd4102650 con 0x7fcfd40ffc20 2024-04-23T02:51:23.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.522+0000 7fcfc37fe700 1 -- 172.21.15.71:0/1587407124 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7fcfc805b010 con 0x7fcfd40ffc20 2024-04-23T02:51:23.525 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:51:23.526 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:51:23.526 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:51:23.526 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:51:23.526 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:51:23.526 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:51:23.526 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:51:23.526 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:51:23.526 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:51:23.526 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:51:23.526 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:51:23.526 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:51:23.526 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:51:23.527 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:51:23.527 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:51:23.527 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:51:23.527 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:51:23.527 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:51:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:51:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:51:23.527 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:51:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:51:23.527 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:51:23.527 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:51:23.527 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:51:23.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.524+0000 7fcfd8d4d700 1 -- 172.21.15.71:0/1587407124 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcfbc06cce0 msgr2=0x7fcfbc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:23.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.524+0000 7fcfd8d4d700 1 --2- 172.21.15.71:0/1587407124 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcfbc06cce0 0x7fcfbc06f1a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4102fc0 tx=0x7fcfc4009260 comp rx=0 tx=0).stop 2024-04-23T02:51:23.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.524+0000 7fcfd8d4d700 1 -- 172.21.15.71:0/1587407124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd40ffc20 msgr2=0x7fcfd4101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:23.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.524+0000 7fcfd8d4d700 1 --2- 172.21.15.71:0/1587407124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd40ffc20 0x7fcfd4101840 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8000c00 tx=0x7fcfc800be90 comp rx=0 tx=0).stop 2024-04-23T02:51:23.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.524+0000 7fcfd8d4d700 1 -- 172.21.15.71:0/1587407124 shutdown_connections 2024-04-23T02:51:23.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.524+0000 7fcfd8d4d700 1 --2- 172.21.15.71:0/1587407124 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcfbc06cce0 0x7fcfbc06f1a0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:23.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.524+0000 7fcfd8d4d700 1 --2- 172.21.15.71:0/1587407124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd40ffc20 0x7fcfd4101840 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:23.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.524+0000 7fcfd8d4d700 1 --2- 172.21.15.71:0/1587407124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfd4105bd0 0x7fcfd4101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:23.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.524+0000 7fcfd8d4d700 1 -- 172.21.15.71:0/1587407124 >> 172.21.15.71:0/1587407124 conn(0x7fcfd4075670 msgr2=0x7fcfd4068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:23.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.524+0000 7fcfd8d4d700 1 -- 172.21.15.71:0/1587407124 shutdown_connections 2024-04-23T02:51:23.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:23.524+0000 7fcfd8d4d700 1 -- 172.21.15.71:0/1587407124 wait complete. 2024-04-23T02:51:24.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:23 smithi192 ceph-mon[108617]: from='client.35876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:51:24.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:23 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1587407124' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:51:24.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:23 smithi071 ceph-mon[144544]: from='client.35876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:51:24.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:23 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1587407124' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:51:25.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:24 smithi192 ceph-mon[108617]: pgmap v1144: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:25.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:24 smithi071 ceph-mon[144544]: pgmap v1144: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:26.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:26 smithi071 ceph-mon[144544]: pgmap v1145: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:27.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:26 smithi192 ceph-mon[108617]: pgmap v1145: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:28.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:28 smithi192 ceph-mon[108617]: pgmap v1146: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:28.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:28 smithi071 ceph-mon[144544]: pgmap v1146: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:30.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:30 smithi071 ceph-mon[144544]: pgmap v1147: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:31.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:30 smithi192 ceph-mon[108617]: pgmap v1147: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:32.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:32 smithi071 ceph-mon[144544]: pgmap v1148: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:33.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:32 smithi192 ceph-mon[108617]: pgmap v1148: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:34.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:34 smithi071 ceph-mon[144544]: pgmap v1149: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:35.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:34 smithi192 ceph-mon[108617]: pgmap v1149: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:51:36.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:51:36.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:36 smithi071 ceph-mon[144544]: pgmap v1150: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:37.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:36 smithi192 ceph-mon[108617]: pgmap v1150: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:38.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:38 smithi192 ceph-mon[108617]: pgmap v1151: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:38.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:38 smithi071 ceph-mon[144544]: pgmap v1151: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:40.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:40 smithi071 ceph-mon[144544]: pgmap v1152: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:41.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:40 smithi192 ceph-mon[108617]: pgmap v1152: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:42.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:42 smithi071 ceph-mon[144544]: pgmap v1153: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:43.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:42 smithi192 ceph-mon[108617]: pgmap v1153: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:44.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:44 smithi071 ceph-mon[144544]: pgmap v1154: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:45.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:44 smithi192 ceph-mon[108617]: pgmap v1154: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:46.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:46 smithi071 ceph-mon[144544]: pgmap v1155: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:47.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:46 smithi192 ceph-mon[108617]: pgmap v1155: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:48.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:48 smithi071 ceph-mon[144544]: pgmap v1156: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:49.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:48 smithi192 ceph-mon[108617]: pgmap v1156: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:50.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:51:50.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:51:51.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:50 smithi192 ceph-mon[108617]: pgmap v1157: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:51.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:50 smithi071 ceph-mon[144544]: pgmap v1157: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:52.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:52 smithi071 ceph-mon[144544]: pgmap v1158: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:53.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:52 smithi192 ceph-mon[108617]: pgmap v1158: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.641+0000 7f708cecb700 1 -- 172.21.15.71:0/4193789979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7088100e90 msgr2=0x7f7088101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.641+0000 7f708cecb700 1 --2- 172.21.15.71:0/4193789979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7088100e90 0x7f7088101270 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f7070009910 tx=0x7f7070009c20 comp rx=0 tx=0).stop 2024-04-23T02:51:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.641+0000 7f708cecb700 1 -- 172.21.15.71:0/4193789979 shutdown_connections 2024-04-23T02:51:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.641+0000 7f708cecb700 1 --2- 172.21.15.71:0/4193789979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7088101840 0x7f7088105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.641+0000 7f708cecb700 1 --2- 172.21.15.71:0/4193789979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7088100e90 0x7f7088101270 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.641+0000 7f708cecb700 1 -- 172.21.15.71:0/4193789979 >> 172.21.15.71:0/4193789979 conn(0x7f70880fc740 msgr2=0x7f70880feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.641+0000 7f708cecb700 1 -- 172.21.15.71:0/4193789979 shutdown_connections 2024-04-23T02:51:53.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.641+0000 7f708cecb700 1 -- 172.21.15.71:0/4193789979 wait complete. 2024-04-23T02:51:53.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.642+0000 7f708cecb700 1 Processor -- start 2024-04-23T02:51:53.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.642+0000 7f708cecb700 1 -- start start 2024-04-23T02:51:53.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.642+0000 7f708cecb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7088100e90 0x7f7088073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:53.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.642+0000 7f708cecb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7088101840 0x7f70880735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:53.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.642+0000 7f708cecb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7088073b10 con 0x7f7088101840 2024-04-23T02:51:53.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.642+0000 7f708cecb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7088073c50 con 0x7f7088100e90 2024-04-23T02:51:53.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.642+0000 7f7085d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7088101840 0x7f70880735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:53.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.642+0000 7f7085d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7088101840 0x7f70880735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42680/0 (socket says 172.21.15.71:42680) 2024-04-23T02:51:53.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.642+0000 7f7085d9b700 1 -- 172.21.15.71:0/2220027083 learned_addr learned my addr 172.21.15.71:0/2220027083 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:51:53.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.642+0000 7f708659c700 1 --2- 172.21.15.71:0/2220027083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7088100e90 0x7f7088073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:53.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.642+0000 7f7085d9b700 1 -- 172.21.15.71:0/2220027083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7088100e90 msgr2=0x7f7088073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:53.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.642+0000 7f7085d9b700 1 --2- 172.21.15.71:0/2220027083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7088100e90 0x7f7088073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:53.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.642+0000 7f7085d9b700 1 -- 172.21.15.71:0/2220027083 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70700095f0 con 0x7f7088101840 2024-04-23T02:51:53.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.643+0000 7f7085d9b700 1 --2- 172.21.15.71:0/2220027083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7088101840 0x7f70880735d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f707800e8f0 tx=0x7f707800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:53.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.643+0000 7f707f7fe700 1 -- 172.21.15.71:0/2220027083 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7078009790 con 0x7f7088101840 2024-04-23T02:51:53.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.643+0000 7f707f7fe700 1 -- 172.21.15.71:0/2220027083 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7078004cd0 con 0x7f7088101840 2024-04-23T02:51:53.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.643+0000 7f708cecb700 1 -- 172.21.15.71:0/2220027083 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70881a29c0 con 0x7f7088101840 2024-04-23T02:51:53.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.643+0000 7f707f7fe700 1 -- 172.21.15.71:0/2220027083 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7078005580 con 0x7f7088101840 2024-04-23T02:51:53.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.643+0000 7f708cecb700 1 -- 172.21.15.71:0/2220027083 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70881a2dd0 con 0x7f7088101840 2024-04-23T02:51:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.644+0000 7f707f7fe700 1 -- 172.21.15.71:0/2220027083 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7078004750 con 0x7f7088101840 2024-04-23T02:51:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.644+0000 7f708cecb700 1 -- 172.21.15.71:0/2220027083 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7088109230 con 0x7f7088101840 2024-04-23T02:51:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.645+0000 7f707f7fe700 1 --2- 172.21.15.71:0/2220027083 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f707406cce0 0x7f707406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.645+0000 7f707f7fe700 1 -- 172.21.15.71:0/2220027083 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7078014070 con 0x7f7088101840 2024-04-23T02:51:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.645+0000 7f708659c700 1 --2- 172.21.15.71:0/2220027083 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f707406cce0 0x7f707406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.645+0000 7f708659c700 1 --2- 172.21.15.71:0/2220027083 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f707406cce0 0x7f707406f1a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7070006010 tx=0x7f707000b5a0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:53.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.649+0000 7f707f7fe700 1 -- 172.21.15.71:0/2220027083 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f70780569c0 con 0x7f7088101840 2024-04-23T02:51:53.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.864+0000 7f708cecb700 1 -- 172.21.15.71:0/2220027083 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f70880611d0 con 0x7f707406cce0 2024-04-23T02:51:53.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.866+0000 7f707f7fe700 1 -- 172.21.15.71:0/2220027083 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f70880611d0 con 0x7f707406cce0 2024-04-23T02:51:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.868+0000 7f708cecb700 1 -- 172.21.15.71:0/2220027083 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f707406cce0 msgr2=0x7f707406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.868+0000 7f708cecb700 1 --2- 172.21.15.71:0/2220027083 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f707406cce0 0x7f707406f1a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7070006010 tx=0x7f707000b5a0 comp rx=0 tx=0).stop 2024-04-23T02:51:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.868+0000 7f708cecb700 1 -- 172.21.15.71:0/2220027083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7088101840 msgr2=0x7f70880735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.868+0000 7f708cecb700 1 --2- 172.21.15.71:0/2220027083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7088101840 0x7f70880735d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f707800e8f0 tx=0x7f707800ecb0 comp rx=0 tx=0).stop 2024-04-23T02:51:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.868+0000 7f708cecb700 1 -- 172.21.15.71:0/2220027083 shutdown_connections 2024-04-23T02:51:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.868+0000 7f708cecb700 1 --2- 172.21.15.71:0/2220027083 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f707406cce0 0x7f707406f1a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.868+0000 7f708cecb700 1 --2- 172.21.15.71:0/2220027083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7088100e90 0x7f7088073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.868+0000 7f708cecb700 1 --2- 172.21.15.71:0/2220027083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7088101840 0x7f70880735d0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.868+0000 7f708cecb700 1 -- 172.21.15.71:0/2220027083 >> 172.21.15.71:0/2220027083 conn(0x7f70880fc740 msgr2=0x7f70880fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.868+0000 7f708cecb700 1 -- 172.21.15.71:0/2220027083 shutdown_connections 2024-04-23T02:51:53.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.868+0000 7f708cecb700 1 -- 172.21.15.71:0/2220027083 wait complete. 2024-04-23T02:51:53.881 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:51:53.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.972+0000 7f0242ece700 1 -- 172.21.15.71:0/1614844304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f023c106cd0 msgr2=0x7f023c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:53.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.972+0000 7f0242ece700 1 --2- 172.21.15.71:0/1614844304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f023c106cd0 0x7f023c1070b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0230009910 tx=0x7f0230009c20 comp rx=0 tx=0).stop 2024-04-23T02:51:53.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.972+0000 7f0242ece700 1 -- 172.21.15.71:0/1614844304 shutdown_connections 2024-04-23T02:51:53.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.972+0000 7f0242ece700 1 --2- 172.21.15.71:0/1614844304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f023c068580 0x7f023c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:53.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.972+0000 7f0242ece700 1 --2- 172.21.15.71:0/1614844304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f023c106cd0 0x7f023c1070b0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:53.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.972+0000 7f0242ece700 1 -- 172.21.15.71:0/1614844304 >> 172.21.15.71:0/1614844304 conn(0x7f023c075670 msgr2=0x7f023c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:53.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.972+0000 7f0242ece700 1 -- 172.21.15.71:0/1614844304 shutdown_connections 2024-04-23T02:51:53.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.972+0000 7f0242ece700 1 -- 172.21.15.71:0/1614844304 wait complete. 2024-04-23T02:51:53.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.972+0000 7f0242ece700 1 Processor -- start 2024-04-23T02:51:53.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.972+0000 7f0242ece700 1 -- start start 2024-04-23T02:51:53.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.972+0000 7f0242ece700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f023c068580 0x7f023c0ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:53.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.972+0000 7f0242ece700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f023c106cd0 0x7f023c100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:53.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.972+0000 7f0242ece700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023c100770 con 0x7f023c106cd0 2024-04-23T02:51:53.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.972+0000 7f0242ece700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023c1008b0 con 0x7f023c068580 2024-04-23T02:51:53.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.973+0000 7f023bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f023c106cd0 0x7f023c100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:53.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.973+0000 7f023bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f023c106cd0 0x7f023c100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42702/0 (socket says 172.21.15.71:42702) 2024-04-23T02:51:53.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.973+0000 7f023bfff700 1 -- 172.21.15.71:0/611290565 learned_addr learned my addr 172.21.15.71:0/611290565 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:51:53.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.973+0000 7f0240c6a700 1 --2- 172.21.15.71:0/611290565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f023c068580 0x7f023c0ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:53.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.973+0000 7f023bfff700 1 -- 172.21.15.71:0/611290565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f023c068580 msgr2=0x7f023c0ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:53.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.973+0000 7f023bfff700 1 --2- 172.21.15.71:0/611290565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f023c068580 0x7f023c0ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:53.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.973+0000 7f023bfff700 1 -- 172.21.15.71:0/611290565 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02300095f0 con 0x7f023c106cd0 2024-04-23T02:51:53.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.973+0000 7f0240c6a700 1 --2- 172.21.15.71:0/611290565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f023c068580 0x7f023c0ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:51:53.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.973+0000 7f023bfff700 1 --2- 172.21.15.71:0/611290565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f023c106cd0 0x7f023c100230 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f022800b4b0 tx=0x7f022800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:53.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.973+0000 7f0239ffb700 1 -- 172.21.15.71:0/611290565 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0228011800 con 0x7f023c106cd0 2024-04-23T02:51:53.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.973+0000 7f0239ffb700 1 -- 172.21.15.71:0/611290565 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0228011e00 con 0x7f023c106cd0 2024-04-23T02:51:53.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.973+0000 7f0242ece700 1 -- 172.21.15.71:0/611290565 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f023c1a2a80 con 0x7f023c106cd0 2024-04-23T02:51:53.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.973+0000 7f0239ffb700 1 -- 172.21.15.71:0/611290565 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0228010e50 con 0x7f023c106cd0 2024-04-23T02:51:53.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.973+0000 7f0242ece700 1 -- 172.21.15.71:0/611290565 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f023c1a2e90 con 0x7f023c106cd0 2024-04-23T02:51:53.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.974+0000 7f0242ece700 1 -- 172.21.15.71:0/611290565 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f023c1091d0 con 0x7f023c106cd0 2024-04-23T02:51:53.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.975+0000 7f0239ffb700 1 -- 172.21.15.71:0/611290565 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0228010380 con 0x7f023c106cd0 2024-04-23T02:51:53.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.975+0000 7f0239ffb700 1 --2- 172.21.15.71:0/611290565 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f022c06cce0 0x7f022c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:53.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.975+0000 7f0239ffb700 1 -- 172.21.15.71:0/611290565 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f022808c0f0 con 0x7f023c106cd0 2024-04-23T02:51:53.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.975+0000 7f0240c6a700 1 --2- 172.21.15.71:0/611290565 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f022c06cce0 0x7f022c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:53.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.976+0000 7f0240c6a700 1 --2- 172.21.15.71:0/611290565 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f022c06cce0 0x7f022c06f1a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0230006010 tx=0x7f023001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:53.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:53.980+0000 7f0239ffb700 1 -- 172.21.15.71:0/611290565 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f02280561e0 con 0x7f023c106cd0 2024-04-23T02:51:54.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.190+0000 7f0242ece700 1 -- 172.21.15.71:0/611290565 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f023c0611d0 con 0x7f022c06cce0 2024-04-23T02:51:54.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.192+0000 7f0239ffb700 1 -- 172.21.15.71:0/611290565 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f023c0611d0 con 0x7f022c06cce0 2024-04-23T02:51:54.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.194+0000 7f0242ece700 1 -- 172.21.15.71:0/611290565 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f022c06cce0 msgr2=0x7f022c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:54.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.194+0000 7f0242ece700 1 --2- 172.21.15.71:0/611290565 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f022c06cce0 0x7f022c06f1a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0230006010 tx=0x7f023001a040 comp rx=0 tx=0).stop 2024-04-23T02:51:54.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.194+0000 7f0242ece700 1 -- 172.21.15.71:0/611290565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f023c106cd0 msgr2=0x7f023c100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:54.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.194+0000 7f0242ece700 1 --2- 172.21.15.71:0/611290565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f023c106cd0 0x7f023c100230 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f022800b4b0 tx=0x7f022800b870 comp rx=0 tx=0).stop 2024-04-23T02:51:54.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.194+0000 7f0242ece700 1 -- 172.21.15.71:0/611290565 shutdown_connections 2024-04-23T02:51:54.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.194+0000 7f0242ece700 1 --2- 172.21.15.71:0/611290565 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f022c06cce0 0x7f022c06f1a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:54.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.194+0000 7f0242ece700 1 --2- 172.21.15.71:0/611290565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f023c068580 0x7f023c0ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:54.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.194+0000 7f0242ece700 1 --2- 172.21.15.71:0/611290565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f023c106cd0 0x7f023c100230 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:54.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.194+0000 7f0242ece700 1 -- 172.21.15.71:0/611290565 >> 172.21.15.71:0/611290565 conn(0x7f023c075670 msgr2=0x7f023c0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:54.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.194+0000 7f0242ece700 1 -- 172.21.15.71:0/611290565 shutdown_connections 2024-04-23T02:51:54.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.194+0000 7f0242ece700 1 -- 172.21.15.71:0/611290565 wait complete. 2024-04-23T02:51:54.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.298+0000 7f0c53ae8700 1 -- 172.21.15.71:0/1216693576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c4c105bd0 msgr2=0x7f0c4c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:54.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.298+0000 7f0c53ae8700 1 --2- 172.21.15.71:0/1216693576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c4c105bd0 0x7f0c4c105fb0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009910 tx=0x7f0c3c009c20 comp rx=0 tx=0).stop 2024-04-23T02:51:54.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.298+0000 7f0c53ae8700 1 -- 172.21.15.71:0/1216693576 shutdown_connections 2024-04-23T02:51:54.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.298+0000 7f0c53ae8700 1 --2- 172.21.15.71:0/1216693576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c4c0ffc20 0x7f0c4c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:54.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.298+0000 7f0c53ae8700 1 --2- 172.21.15.71:0/1216693576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c4c105bd0 0x7f0c4c105fb0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:54.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.298+0000 7f0c53ae8700 1 -- 172.21.15.71:0/1216693576 >> 172.21.15.71:0/1216693576 conn(0x7f0c4c075670 msgr2=0x7f0c4c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:54.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.298+0000 7f0c53ae8700 1 -- 172.21.15.71:0/1216693576 shutdown_connections 2024-04-23T02:51:54.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.298+0000 7f0c53ae8700 1 -- 172.21.15.71:0/1216693576 wait complete. 2024-04-23T02:51:54.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.298+0000 7f0c53ae8700 1 Processor -- start 2024-04-23T02:51:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.299+0000 7f0c53ae8700 1 -- start start 2024-04-23T02:51:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.299+0000 7f0c53ae8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c4c0ffc20 0x7f0c4c197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.299+0000 7f0c53ae8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c4c105bd0 0x7f0c4c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.299+0000 7f0c53ae8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c198160 con 0x7f0c4c105bd0 2024-04-23T02:51:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.299+0000 7f0c53ae8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c1982a0 con 0x7f0c4c0ffc20 2024-04-23T02:51:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.299+0000 7f0c51083700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c4c105bd0 0x7f0c4c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.299+0000 7f0c51083700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c4c105bd0 0x7f0c4c197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42724/0 (socket says 172.21.15.71:42724) 2024-04-23T02:51:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.299+0000 7f0c51083700 1 -- 172.21.15.71:0/3759174046 learned_addr learned my addr 172.21.15.71:0/3759174046 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:51:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.299+0000 7f0c51884700 1 --2- 172.21.15.71:0/3759174046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c4c0ffc20 0x7f0c4c197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.299+0000 7f0c51083700 1 -- 172.21.15.71:0/3759174046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c4c0ffc20 msgr2=0x7f0c4c197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.299+0000 7f0c51083700 1 --2- 172.21.15.71:0/3759174046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c4c0ffc20 0x7f0c4c197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.299+0000 7f0c51083700 1 -- 172.21.15.71:0/3759174046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c0095f0 con 0x7f0c4c105bd0 2024-04-23T02:51:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.299+0000 7f0c51884700 1 --2- 172.21.15.71:0/3759174046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c4c0ffc20 0x7f0c4c197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:51:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.299+0000 7f0c51083700 1 --2- 172.21.15.71:0/3759174046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c4c105bd0 0x7f0c4c197b40 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800d900 tx=0x7f0c4800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:54.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.299+0000 7f0c42ffd700 1 -- 172.21.15.71:0/3759174046 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c480041f0 con 0x7f0c4c105bd0 2024-04-23T02:51:54.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.299+0000 7f0c42ffd700 1 -- 172.21.15.71:0/3759174046 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c48004cd0 con 0x7f0c4c105bd0 2024-04-23T02:51:54.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.299+0000 7f0c53ae8700 1 -- 172.21.15.71:0/3759174046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4c1a0670 con 0x7f0c4c105bd0 2024-04-23T02:51:54.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.299+0000 7f0c42ffd700 1 -- 172.21.15.71:0/3759174046 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c480056d0 con 0x7f0c4c105bd0 2024-04-23T02:51:54.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.300+0000 7f0c53ae8700 1 -- 172.21.15.71:0/3759174046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4c1a0b30 con 0x7f0c4c105bd0 2024-04-23T02:51:54.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.301+0000 7f0c42ffd700 1 -- 172.21.15.71:0/3759174046 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0c48004e40 con 0x7f0c4c105bd0 2024-04-23T02:51:54.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.301+0000 7f0c53ae8700 1 -- 172.21.15.71:0/3759174046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4c1080d0 con 0x7f0c4c105bd0 2024-04-23T02:51:54.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.301+0000 7f0c42ffd700 1 --2- 172.21.15.71:0/3759174046 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0c3806cce0 0x7f0c3806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:54.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.301+0000 7f0c42ffd700 1 -- 172.21.15.71:0/3759174046 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0c4808c5d0 con 0x7f0c4c105bd0 2024-04-23T02:51:54.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.301+0000 7f0c51884700 1 --2- 172.21.15.71:0/3759174046 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0c3806cce0 0x7f0c3806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:54.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.302+0000 7f0c51884700 1 --2- 172.21.15.71:0/3759174046 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0c3806cce0 0x7f0c3806f1a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c006010 tx=0x7f0c3c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:54.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.306+0000 7f0c42ffd700 1 -- 172.21.15.71:0/3759174046 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0c480566c0 con 0x7f0c4c105bd0 2024-04-23T02:51:54.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.509+0000 7f0c53ae8700 1 -- 172.21.15.71:0/3759174046 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c4c02cce0 con 0x7f0c3806cce0 2024-04-23T02:51:54.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.520+0000 7f0c42ffd700 1 -- 172.21.15.71:0/3759174046 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0c4c02cce0 con 0x7f0c3806cce0 2024-04-23T02:51:54.523 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:51:54.523 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (35m) 13m ago 46m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:51:54.523 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (47m) 13m ago 47m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:51:54.523 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (46m) 13m ago 46m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:51:54.523 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (33m) 13m ago 47m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:51:54.523 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (33m) 13m ago 45m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:51:54.523 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (35m) 13m ago 46m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:51:54.524 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (29m) 13m ago 40m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:51:54.524 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (29m) 13m ago 40m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:51:54.524 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (13m) 13m ago 40m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:51:54.524 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (29m) 13m ago 40m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:51:54.524 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (37m) 13m ago 48m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:51:54.524 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (36m) 13m ago 45m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:51:54.524 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (34m) 13m ago 48m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:51:54.524 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (33m) 13m ago 45m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:51:54.524 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (36m) 13m ago 47m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:51:54.524 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (36m) 13m ago 45m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:51:54.524 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (33m) 13m ago 44m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:51:54.524 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (32m) 13m ago 44m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:51:54.524 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (31m) 13m ago 43m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:51:54.524 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (31m) 13m ago 43m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:51:54.524 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (30m) 13m ago 42m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:51:54.525 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (30m) 13m ago 42m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:51:54.525 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (35m) 13m ago 46m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:51:54.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.522+0000 7f0c53ae8700 1 -- 172.21.15.71:0/3759174046 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0c3806cce0 msgr2=0x7f0c3806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:54.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.522+0000 7f0c53ae8700 1 --2- 172.21.15.71:0/3759174046 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0c3806cce0 0x7f0c3806f1a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c006010 tx=0x7f0c3c01a040 comp rx=0 tx=0).stop 2024-04-23T02:51:54.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.522+0000 7f0c53ae8700 1 -- 172.21.15.71:0/3759174046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c4c105bd0 msgr2=0x7f0c4c197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:54.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.522+0000 7f0c53ae8700 1 --2- 172.21.15.71:0/3759174046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c4c105bd0 0x7f0c4c197b40 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800d900 tx=0x7f0c4800dcc0 comp rx=0 tx=0).stop 2024-04-23T02:51:54.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.522+0000 7f0c53ae8700 1 -- 172.21.15.71:0/3759174046 shutdown_connections 2024-04-23T02:51:54.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.522+0000 7f0c53ae8700 1 --2- 172.21.15.71:0/3759174046 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0c3806cce0 0x7f0c3806f1a0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:54.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.522+0000 7f0c53ae8700 1 --2- 172.21.15.71:0/3759174046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c4c0ffc20 0x7f0c4c197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:54.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.522+0000 7f0c53ae8700 1 --2- 172.21.15.71:0/3759174046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c4c105bd0 0x7f0c4c197b40 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:54.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.522+0000 7f0c53ae8700 1 -- 172.21.15.71:0/3759174046 >> 172.21.15.71:0/3759174046 conn(0x7f0c4c075670 msgr2=0x7f0c4c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:54.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.523+0000 7f0c53ae8700 1 -- 172.21.15.71:0/3759174046 shutdown_connections 2024-04-23T02:51:54.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.523+0000 7f0c53ae8700 1 -- 172.21.15.71:0/3759174046 wait complete. 2024-04-23T02:51:54.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.624+0000 7f9c280d2700 1 -- 172.21.15.71:0/3386258241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c20100750 msgr2=0x7f9c201047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:54.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.624+0000 7f9c280d2700 1 --2- 172.21.15.71:0/3386258241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c20100750 0x7f9c201047a0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009910 tx=0x7f9c1c009c20 comp rx=0 tx=0).stop 2024-04-23T02:51:54.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.624+0000 7f9c280d2700 1 -- 172.21.15.71:0/3386258241 shutdown_connections 2024-04-23T02:51:54.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.624+0000 7f9c280d2700 1 --2- 172.21.15.71:0/3386258241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c20100750 0x7f9c201047a0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:54.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.624+0000 7f9c280d2700 1 --2- 172.21.15.71:0/3386258241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c200ffda0 0x7f9c20100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:54.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.624+0000 7f9c280d2700 1 -- 172.21.15.71:0/3386258241 >> 172.21.15.71:0/3386258241 conn(0x7f9c200fb650 msgr2=0x7f9c200fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:54.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.624+0000 7f9c280d2700 1 -- 172.21.15.71:0/3386258241 shutdown_connections 2024-04-23T02:51:54.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.624+0000 7f9c280d2700 1 -- 172.21.15.71:0/3386258241 wait complete. 2024-04-23T02:51:54.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.624+0000 7f9c280d2700 1 Processor -- start 2024-04-23T02:51:54.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.624+0000 7f9c280d2700 1 -- start start 2024-04-23T02:51:54.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.625+0000 7f9c280d2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c200ffda0 0x7f9c201976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:54.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.625+0000 7f9c280d2700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c20100750 0x7f9c20197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:54.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.625+0000 7f9c280d2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c201982c0 con 0x7f9c200ffda0 2024-04-23T02:51:54.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.625+0000 7f9c280d2700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c20198400 con 0x7f9c20100750 2024-04-23T02:51:54.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.625+0000 7f9c25e6e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c200ffda0 0x7f9c201976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:54.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.625+0000 7f9c25e6e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c200ffda0 0x7f9c201976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42750/0 (socket says 172.21.15.71:42750) 2024-04-23T02:51:54.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.625+0000 7f9c25e6e700 1 -- 172.21.15.71:0/3645322876 learned_addr learned my addr 172.21.15.71:0/3645322876 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:51:54.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.625+0000 7f9c25e6e700 1 -- 172.21.15.71:0/3645322876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c20100750 msgr2=0x7f9c20197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:54.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.625+0000 7f9c2566d700 1 --2- 172.21.15.71:0/3645322876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c20100750 0x7f9c20197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:54.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.625+0000 7f9c25e6e700 1 --2- 172.21.15.71:0/3645322876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c20100750 0x7f9c20197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:54.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.625+0000 7f9c25e6e700 1 -- 172.21.15.71:0/3645322876 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c0095f0 con 0x7f9c200ffda0 2024-04-23T02:51:54.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.625+0000 7f9c25e6e700 1 --2- 172.21.15.71:0/3645322876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c200ffda0 0x7f9c201976a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000d8d0 tx=0x7f9c1000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:54.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.625+0000 7f9c16ffd700 1 -- 172.21.15.71:0/3645322876 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c10009940 con 0x7f9c200ffda0 2024-04-23T02:51:54.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.625+0000 7f9c16ffd700 1 -- 172.21.15.71:0/3645322876 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9c10010460 con 0x7f9c200ffda0 2024-04-23T02:51:54.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.625+0000 7f9c16ffd700 1 -- 172.21.15.71:0/3645322876 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c1000f5f0 con 0x7f9c200ffda0 2024-04-23T02:51:54.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.625+0000 7f9c280d2700 1 -- 172.21.15.71:0/3645322876 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c201a0710 con 0x7f9c200ffda0 2024-04-23T02:51:54.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.625+0000 7f9c280d2700 1 -- 172.21.15.71:0/3645322876 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c201a0c30 con 0x7f9c200ffda0 2024-04-23T02:51:54.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.626+0000 7f9c280d2700 1 -- 172.21.15.71:0/3645322876 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c201081a0 con 0x7f9c200ffda0 2024-04-23T02:51:54.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.627+0000 7f9c16ffd700 1 -- 172.21.15.71:0/3645322876 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9c100105d0 con 0x7f9c200ffda0 2024-04-23T02:51:54.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.627+0000 7f9c16ffd700 1 --2- 172.21.15.71:0/3645322876 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9c0c06cce0 0x7f9c0c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:54.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.627+0000 7f9c16ffd700 1 -- 172.21.15.71:0/3645322876 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9c1008c5e0 con 0x7f9c200ffda0 2024-04-23T02:51:54.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.627+0000 7f9c2566d700 1 --2- 172.21.15.71:0/3645322876 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9c0c06cce0 0x7f9c0c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:54.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.628+0000 7f9c2566d700 1 --2- 172.21.15.71:0/3645322876 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9c0c06cce0 0x7f9c0c06f1a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00b5c0 tx=0x7f9c1c005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:54.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.632+0000 7f9c16ffd700 1 -- 172.21.15.71:0/3645322876 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9c10056650 con 0x7f9c200ffda0 2024-04-23T02:51:54.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.907+0000 7f9c280d2700 1 -- 172.21.15.71:0/3645322876 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9c20107b80 con 0x7f9c200ffda0 2024-04-23T02:51:54.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.908+0000 7f9c16ffd700 1 -- 172.21.15.71:0/3645322876 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f9c10010880 con 0x7f9c200ffda0 2024-04-23T02:51:54.911 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:51:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:51:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:51:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:51:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:51:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:51:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:51:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:51:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:51:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:51:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:51:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:51:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:51:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:51:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:51:54.912 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:51:54.912 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:51:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.910+0000 7f9c280d2700 1 -- 172.21.15.71:0/3645322876 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9c0c06cce0 msgr2=0x7f9c0c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.910+0000 7f9c280d2700 1 --2- 172.21.15.71:0/3645322876 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9c0c06cce0 0x7f9c0c06f1a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00b5c0 tx=0x7f9c1c005fd0 comp rx=0 tx=0).stop 2024-04-23T02:51:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.910+0000 7f9c280d2700 1 -- 172.21.15.71:0/3645322876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c200ffda0 msgr2=0x7f9c201976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.910+0000 7f9c280d2700 1 --2- 172.21.15.71:0/3645322876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c200ffda0 0x7f9c201976a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000d8d0 tx=0x7f9c1000dc90 comp rx=0 tx=0).stop 2024-04-23T02:51:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.910+0000 7f9c280d2700 1 -- 172.21.15.71:0/3645322876 shutdown_connections 2024-04-23T02:51:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.910+0000 7f9c280d2700 1 --2- 172.21.15.71:0/3645322876 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9c0c06cce0 0x7f9c0c06f1a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.910+0000 7f9c280d2700 1 --2- 172.21.15.71:0/3645322876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c200ffda0 0x7f9c201976a0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.910+0000 7f9c280d2700 1 --2- 172.21.15.71:0/3645322876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c20100750 0x7f9c20197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.910+0000 7f9c280d2700 1 -- 172.21.15.71:0/3645322876 >> 172.21.15.71:0/3645322876 conn(0x7f9c200fb650 msgr2=0x7f9c200fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:54.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.910+0000 7f9c280d2700 1 -- 172.21.15.71:0/3645322876 shutdown_connections 2024-04-23T02:51:54.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:54.910+0000 7f9c280d2700 1 -- 172.21.15.71:0/3645322876 wait complete. 2024-04-23T02:51:54.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:54 smithi071 ceph-mon[144544]: from='client.35884 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:51:54.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:54 smithi071 ceph-mon[144544]: from='client.35888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:51:54.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:54 smithi071 ceph-mon[144544]: from='client.35892 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:51:54.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:54 smithi071 ceph-mon[144544]: pgmap v1159: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:55.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.014+0000 7f31a526e700 1 -- 172.21.15.71:0/499426564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31a0068580 msgr2=0x7f31a00689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:55.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.014+0000 7f31a526e700 1 --2- 172.21.15.71:0/499426564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31a0068580 0x7f31a00689e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f3190009910 tx=0x7f3190009c20 comp rx=0 tx=0).stop 2024-04-23T02:51:55.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.014+0000 7f31a526e700 1 -- 172.21.15.71:0/499426564 shutdown_connections 2024-04-23T02:51:55.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.014+0000 7f31a526e700 1 --2- 172.21.15.71:0/499426564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31a0068580 0x7f31a00689e0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.014+0000 7f31a526e700 1 --2- 172.21.15.71:0/499426564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31a0106cd0 0x7f31a01070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.014+0000 7f31a526e700 1 -- 172.21.15.71:0/499426564 >> 172.21.15.71:0/499426564 conn(0x7f31a0075670 msgr2=0x7f31a0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:55.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.014+0000 7f31a526e700 1 -- 172.21.15.71:0/499426564 shutdown_connections 2024-04-23T02:51:55.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.014+0000 7f31a526e700 1 -- 172.21.15.71:0/499426564 wait complete. 2024-04-23T02:51:55.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.014+0000 7f31a526e700 1 Processor -- start 2024-04-23T02:51:55.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.015+0000 7f31a526e700 1 -- start start 2024-04-23T02:51:55.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.015+0000 7f31a526e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31a0068580 0x7f31a00ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:55.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.015+0000 7f31a526e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31a0106cd0 0x7f31a0100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:55.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.015+0000 7f31a526e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a0100770 con 0x7f31a0106cd0 2024-04-23T02:51:55.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.015+0000 7f31a526e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a01008b0 con 0x7f31a0068580 2024-04-23T02:51:55.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.015+0000 7f319e7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31a0106cd0 0x7f31a0100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:55.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.015+0000 7f319e7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31a0106cd0 0x7f31a0100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42766/0 (socket says 172.21.15.71:42766) 2024-04-23T02:51:55.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.015+0000 7f319e7fc700 1 -- 172.21.15.71:0/532415237 learned_addr learned my addr 172.21.15.71:0/532415237 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:51:55.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.015+0000 7f319effd700 1 --2- 172.21.15.71:0/532415237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31a0068580 0x7f31a00ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:55.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.015+0000 7f319e7fc700 1 -- 172.21.15.71:0/532415237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31a0068580 msgr2=0x7f31a00ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:55.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.015+0000 7f319e7fc700 1 --2- 172.21.15.71:0/532415237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31a0068580 0x7f31a00ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.015+0000 7f319e7fc700 1 -- 172.21.15.71:0/532415237 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31900095f0 con 0x7f31a0106cd0 2024-04-23T02:51:55.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.015+0000 7f319e7fc700 1 --2- 172.21.15.71:0/532415237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31a0106cd0 0x7f31a0100230 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3190006010 tx=0x7f319000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:55.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.016+0000 7f3187fff700 1 -- 172.21.15.71:0/532415237 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f319001d070 con 0x7f31a0106cd0 2024-04-23T02:51:55.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.016+0000 7f3187fff700 1 -- 172.21.15.71:0/532415237 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3190005710 con 0x7f31a0106cd0 2024-04-23T02:51:55.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.016+0000 7f31a526e700 1 -- 172.21.15.71:0/532415237 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a0100b30 con 0x7f31a0106cd0 2024-04-23T02:51:55.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.016+0000 7f3187fff700 1 -- 172.21.15.71:0/532415237 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f319000fe20 con 0x7f31a0106cd0 2024-04-23T02:51:55.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.016+0000 7f31a526e700 1 -- 172.21.15.71:0/532415237 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a01a2c70 con 0x7f31a0106cd0 2024-04-23T02:51:55.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.017+0000 7f31a526e700 1 -- 172.21.15.71:0/532415237 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a01091d0 con 0x7f31a0106cd0 2024-04-23T02:51:55.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.017+0000 7f3187fff700 1 -- 172.21.15.71:0/532415237 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3190003a60 con 0x7f31a0106cd0 2024-04-23T02:51:55.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.017+0000 7f3187fff700 1 --2- 172.21.15.71:0/532415237 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f318806cce0 0x7f318806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:55.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.017+0000 7f3187fff700 1 -- 172.21.15.71:0/532415237 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f319008d790 con 0x7f31a0106cd0 2024-04-23T02:51:55.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.017+0000 7f319effd700 1 --2- 172.21.15.71:0/532415237 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f318806cce0 0x7f318806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:55.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.018+0000 7f319effd700 1 --2- 172.21.15.71:0/532415237 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f318806cce0 0x7f318806f1a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3194005950 tx=0x7f319400b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:55.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.022+0000 7f3187fff700 1 -- 172.21.15.71:0/532415237 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3190057800 con 0x7f31a0106cd0 2024-04-23T02:51:55.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:54 smithi192 ceph-mon[108617]: from='client.35884 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:51:55.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:54 smithi192 ceph-mon[108617]: from='client.35888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:51:55.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:54 smithi192 ceph-mon[108617]: from='client.35892 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:51:55.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:54 smithi192 ceph-mon[108617]: pgmap v1159: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:55.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.257+0000 7f31a526e700 1 -- 172.21.15.71:0/532415237 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f31a004f2e0 con 0x7f31a0106cd0 2024-04-23T02:51:55.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.257+0000 7f3187fff700 1 -- 172.21.15.71:0/532415237 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7f3190003d10 con 0x7f31a0106cd0 2024-04-23T02:51:55.260 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:51:55.260 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:51:55.260 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:51:55.260 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:51:55.260 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:51:55.260 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:51:55.260 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:51:55.260 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:51:55.261 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:51:55.261 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:51:55.261 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:51:55.261 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:51:55.261 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:51:55.261 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:51:55.261 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:51:55.261 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:51:55.261 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:51:55.261 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:51:55.261 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:51:55.261 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:51:55.261 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:51:55.261 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:51:55.261 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:51:55.262 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:51:55.262 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:51:55.262 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:51:55.262 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:51:55.262 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:51:55.262 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:51:55.262 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:51:55.262 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:51:55.262 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:51:55.262 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:51:55.262 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:51:55.262 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:51:55.262 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:51:55.263 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:51:55.263 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:51:55.263 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:51:55.263 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:51:55.263 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:51:55.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.259+0000 7f31a526e700 1 -- 172.21.15.71:0/532415237 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f318806cce0 msgr2=0x7f318806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:55.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.259+0000 7f31a526e700 1 --2- 172.21.15.71:0/532415237 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f318806cce0 0x7f318806f1a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3194005950 tx=0x7f319400b410 comp rx=0 tx=0).stop 2024-04-23T02:51:55.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.259+0000 7f31a526e700 1 -- 172.21.15.71:0/532415237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31a0106cd0 msgr2=0x7f31a0100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:55.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.259+0000 7f31a526e700 1 --2- 172.21.15.71:0/532415237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31a0106cd0 0x7f31a0100230 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3190006010 tx=0x7f319000bd70 comp rx=0 tx=0).stop 2024-04-23T02:51:55.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.259+0000 7f31a526e700 1 -- 172.21.15.71:0/532415237 shutdown_connections 2024-04-23T02:51:55.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.259+0000 7f31a526e700 1 --2- 172.21.15.71:0/532415237 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f318806cce0 0x7f318806f1a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.259+0000 7f31a526e700 1 --2- 172.21.15.71:0/532415237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31a0068580 0x7f31a00ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.259+0000 7f31a526e700 1 --2- 172.21.15.71:0/532415237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31a0106cd0 0x7f31a0100230 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.259+0000 7f31a526e700 1 -- 172.21.15.71:0/532415237 >> 172.21.15.71:0/532415237 conn(0x7f31a0075670 msgr2=0x7f31a00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:55.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.259+0000 7f31a526e700 1 -- 172.21.15.71:0/532415237 shutdown_connections 2024-04-23T02:51:55.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.259+0000 7f31a526e700 1 -- 172.21.15.71:0/532415237 wait complete. 2024-04-23T02:51:55.264 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:51:55.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.362+0000 7ff779d96700 1 -- 172.21.15.71:0/599365119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff774100750 msgr2=0x7ff7741047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:55.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.362+0000 7ff779d96700 1 --2- 172.21.15.71:0/599365119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff774100750 0x7ff7741047a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff768009910 tx=0x7ff768009c20 comp rx=0 tx=0).stop 2024-04-23T02:51:55.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.362+0000 7ff779d96700 1 -- 172.21.15.71:0/599365119 shutdown_connections 2024-04-23T02:51:55.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.362+0000 7ff779d96700 1 --2- 172.21.15.71:0/599365119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff774100750 0x7ff7741047a0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.362+0000 7ff779d96700 1 --2- 172.21.15.71:0/599365119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7740ffda0 0x7ff774100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.362+0000 7ff779d96700 1 -- 172.21.15.71:0/599365119 >> 172.21.15.71:0/599365119 conn(0x7ff7740fb650 msgr2=0x7ff7740fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:55.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.362+0000 7ff779d96700 1 -- 172.21.15.71:0/599365119 shutdown_connections 2024-04-23T02:51:55.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.362+0000 7ff779d96700 1 -- 172.21.15.71:0/599365119 wait complete. 2024-04-23T02:51:55.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.363+0000 7ff779d96700 1 Processor -- start 2024-04-23T02:51:55.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.363+0000 7ff779d96700 1 -- start start 2024-04-23T02:51:55.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.363+0000 7ff779d96700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7740ffda0 0x7ff774197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:55.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.363+0000 7ff779d96700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff774100750 0x7ff774197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:55.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.363+0000 7ff779d96700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff774198260 con 0x7ff7740ffda0 2024-04-23T02:51:55.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.363+0000 7ff779d96700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7741983a0 con 0x7ff774100750 2024-04-23T02:51:55.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.364+0000 7ff7737fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7740ffda0 0x7ff774197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:55.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.364+0000 7ff7737fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7740ffda0 0x7ff774197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40406/0 (socket says 172.21.15.71:40406) 2024-04-23T02:51:55.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.364+0000 7ff7737fe700 1 -- 172.21.15.71:0/2359771245 learned_addr learned my addr 172.21.15.71:0/2359771245 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:51:55.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.364+0000 7ff772ffd700 1 --2- 172.21.15.71:0/2359771245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff774100750 0x7ff774197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:55.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.364+0000 7ff7737fe700 1 -- 172.21.15.71:0/2359771245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff774100750 msgr2=0x7ff774197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:55.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.364+0000 7ff7737fe700 1 --2- 172.21.15.71:0/2359771245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff774100750 0x7ff774197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.364+0000 7ff7737fe700 1 -- 172.21.15.71:0/2359771245 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7680095f0 con 0x7ff7740ffda0 2024-04-23T02:51:55.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.364+0000 7ff772ffd700 1 --2- 172.21.15.71:0/2359771245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff774100750 0x7ff774197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:51:55.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.364+0000 7ff7737fe700 1 --2- 172.21.15.71:0/2359771245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7740ffda0 0x7ff774197700 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff76400d8d0 tx=0x7ff76400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:55.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.364+0000 7ff770ff9700 1 -- 172.21.15.71:0/2359771245 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff764009940 con 0x7ff7740ffda0 2024-04-23T02:51:55.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.364+0000 7ff770ff9700 1 -- 172.21.15.71:0/2359771245 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff764010460 con 0x7ff7740ffda0 2024-04-23T02:51:55.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.364+0000 7ff779d96700 1 -- 172.21.15.71:0/2359771245 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7741a0770 con 0x7ff7740ffda0 2024-04-23T02:51:55.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.364+0000 7ff770ff9700 1 -- 172.21.15.71:0/2359771245 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff76400f5f0 con 0x7ff7740ffda0 2024-04-23T02:51:55.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.364+0000 7ff779d96700 1 -- 172.21.15.71:0/2359771245 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7741a0c30 con 0x7ff7740ffda0 2024-04-23T02:51:55.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.365+0000 7ff779d96700 1 -- 172.21.15.71:0/2359771245 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7741081a0 con 0x7ff7740ffda0 2024-04-23T02:51:55.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.366+0000 7ff770ff9700 1 -- 172.21.15.71:0/2359771245 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff7640105d0 con 0x7ff7740ffda0 2024-04-23T02:51:55.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.366+0000 7ff770ff9700 1 --2- 172.21.15.71:0/2359771245 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff75c06cce0 0x7ff75c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:55.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.366+0000 7ff770ff9700 1 -- 172.21.15.71:0/2359771245 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff76408c5e0 con 0x7ff7740ffda0 2024-04-23T02:51:55.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.366+0000 7ff772ffd700 1 --2- 172.21.15.71:0/2359771245 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff75c06cce0 0x7ff75c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:55.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.366+0000 7ff772ffd700 1 --2- 172.21.15.71:0/2359771245 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff75c06cce0 0x7ff75c06f1a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff768006010 tx=0x7ff76801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:55.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.370+0000 7ff770ff9700 1 -- 172.21.15.71:0/2359771245 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff764056650 con 0x7ff7740ffda0 2024-04-23T02:51:55.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.578+0000 7ff779d96700 1 -- 172.21.15.71:0/2359771245 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7740611d0 con 0x7ff75c06cce0 2024-04-23T02:51:55.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.580+0000 7ff770ff9700 1 -- 172.21.15.71:0/2359771245 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff7740611d0 con 0x7ff75c06cce0 2024-04-23T02:51:55.583 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:51:55.583 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:51:55.583 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:51:55.583 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:51:55.583 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:51:55.583 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:51:55.583 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:51:55.583 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:51:55.583 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:51:55.583 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:51:55.583 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:51:55.583 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:51:55.583 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:51:55.583 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:51:55.584 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:51:55.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.582+0000 7ff779d96700 1 -- 172.21.15.71:0/2359771245 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff75c06cce0 msgr2=0x7ff75c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:55.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.582+0000 7ff779d96700 1 --2- 172.21.15.71:0/2359771245 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff75c06cce0 0x7ff75c06f1a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff768006010 tx=0x7ff76801a040 comp rx=0 tx=0).stop 2024-04-23T02:51:55.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.582+0000 7ff779d96700 1 -- 172.21.15.71:0/2359771245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7740ffda0 msgr2=0x7ff774197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:55.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.582+0000 7ff779d96700 1 --2- 172.21.15.71:0/2359771245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7740ffda0 0x7ff774197700 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff76400d8d0 tx=0x7ff76400dc90 comp rx=0 tx=0).stop 2024-04-23T02:51:55.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.582+0000 7ff779d96700 1 -- 172.21.15.71:0/2359771245 shutdown_connections 2024-04-23T02:51:55.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.582+0000 7ff779d96700 1 --2- 172.21.15.71:0/2359771245 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff75c06cce0 0x7ff75c06f1a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.582+0000 7ff779d96700 1 --2- 172.21.15.71:0/2359771245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7740ffda0 0x7ff774197700 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.582+0000 7ff779d96700 1 --2- 172.21.15.71:0/2359771245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff774100750 0x7ff774197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.582+0000 7ff779d96700 1 -- 172.21.15.71:0/2359771245 >> 172.21.15.71:0/2359771245 conn(0x7ff7740fb650 msgr2=0x7ff7740fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:55.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.582+0000 7ff779d96700 1 -- 172.21.15.71:0/2359771245 shutdown_connections 2024-04-23T02:51:55.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.582+0000 7ff779d96700 1 -- 172.21.15.71:0/2359771245 wait complete. 2024-04-23T02:51:55.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.685+0000 7fb55b103700 1 -- 172.21.15.71:0/4041847545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb554101850 msgr2=0x7fb5541058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:55.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.685+0000 7fb55b103700 1 --2- 172.21.15.71:0/4041847545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb554101850 0x7fb5541058a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb540009910 tx=0x7fb540009c20 comp rx=0 tx=0).stop 2024-04-23T02:51:55.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.685+0000 7fb55b103700 1 -- 172.21.15.71:0/4041847545 shutdown_connections 2024-04-23T02:51:55.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.685+0000 7fb55b103700 1 --2- 172.21.15.71:0/4041847545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb554101850 0x7fb5541058a0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.685+0000 7fb55b103700 1 --2- 172.21.15.71:0/4041847545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb554100ea0 0x7fb554101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.685+0000 7fb55b103700 1 -- 172.21.15.71:0/4041847545 >> 172.21.15.71:0/4041847545 conn(0x7fb5540fc750 msgr2=0x7fb5540feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:55.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.685+0000 7fb55b103700 1 -- 172.21.15.71:0/4041847545 shutdown_connections 2024-04-23T02:51:55.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.685+0000 7fb55b103700 1 -- 172.21.15.71:0/4041847545 wait complete. 2024-04-23T02:51:55.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.685+0000 7fb55b103700 1 Processor -- start 2024-04-23T02:51:55.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.685+0000 7fb55b103700 1 -- start start 2024-04-23T02:51:55.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.685+0000 7fb55b103700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb554100ea0 0x7fb554073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:55.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.685+0000 7fb55b103700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb554101850 0x7fb5540735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:55.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.686+0000 7fb55b103700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb554073b10 con 0x7fb554101850 2024-04-23T02:51:55.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.686+0000 7fb55b103700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb554073c50 con 0x7fb554100ea0 2024-04-23T02:51:55.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.686+0000 7fb553fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb554101850 0x7fb5540735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:55.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.686+0000 7fb553fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb554101850 0x7fb5540735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40422/0 (socket says 172.21.15.71:40422) 2024-04-23T02:51:55.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.686+0000 7fb553fff700 1 -- 172.21.15.71:0/2558233967 learned_addr learned my addr 172.21.15.71:0/2558233967 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:51:55.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.686+0000 7fb558e9f700 1 --2- 172.21.15.71:0/2558233967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb554100ea0 0x7fb554073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:55.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.686+0000 7fb553fff700 1 -- 172.21.15.71:0/2558233967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb554100ea0 msgr2=0x7fb554073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:55.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.686+0000 7fb553fff700 1 --2- 172.21.15.71:0/2558233967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb554100ea0 0x7fb554073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.686+0000 7fb553fff700 1 -- 172.21.15.71:0/2558233967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5400095f0 con 0x7fb554101850 2024-04-23T02:51:55.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.686+0000 7fb558e9f700 1 --2- 172.21.15.71:0/2558233967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb554100ea0 0x7fb554073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:51:55.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.686+0000 7fb553fff700 1 --2- 172.21.15.71:0/2558233967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb554101850 0x7fb5540735d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb5400092e0 tx=0x7fb54000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:55.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.686+0000 7fb551ffb700 1 -- 172.21.15.71:0/2558233967 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb54001d070 con 0x7fb554101850 2024-04-23T02:51:55.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.686+0000 7fb551ffb700 1 -- 172.21.15.71:0/2558233967 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb540005940 con 0x7fb554101850 2024-04-23T02:51:55.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.686+0000 7fb551ffb700 1 -- 172.21.15.71:0/2558233967 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb54000fe50 con 0x7fb554101850 2024-04-23T02:51:55.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.686+0000 7fb55b103700 1 -- 172.21.15.71:0/2558233967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb554073ed0 con 0x7fb554101850 2024-04-23T02:51:55.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.686+0000 7fb55b103700 1 -- 172.21.15.71:0/2558233967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5541a2d50 con 0x7fb554101850 2024-04-23T02:51:55.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.687+0000 7fb55b103700 1 -- 172.21.15.71:0/2558233967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5541092a0 con 0x7fb554101850 2024-04-23T02:51:55.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.688+0000 7fb551ffb700 1 -- 172.21.15.71:0/2558233967 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb540003a60 con 0x7fb554101850 2024-04-23T02:51:55.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.688+0000 7fb551ffb700 1 --2- 172.21.15.71:0/2558233967 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb54406cce0 0x7fb54406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:51:55.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.688+0000 7fb551ffb700 1 -- 172.21.15.71:0/2558233967 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb54008d790 con 0x7fb554101850 2024-04-23T02:51:55.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.688+0000 7fb558e9f700 1 --2- 172.21.15.71:0/2558233967 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb54406cce0 0x7fb54406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:51:55.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.689+0000 7fb558e9f700 1 --2- 172.21.15.71:0/2558233967 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb54406cce0 0x7fb54406f1a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb548005fd0 tx=0x7fb548005ee0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:51:55.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.693+0000 7fb551ffb700 1 -- 172.21.15.71:0/2558233967 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb540057800 con 0x7fb554101850 2024-04-23T02:51:55.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.967+0000 7fb55b103700 1 -- 172.21.15.71:0/2558233967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb554074660 con 0x7fb554101850 2024-04-23T02:51:55.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.967+0000 7fb551ffb700 1 -- 172.21.15.71:0/2558233967 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7fb54005ae20 con 0x7fb554101850 2024-04-23T02:51:55.971 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:51:55.971 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:51:55.971 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:51:55.971 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:51:55.971 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:51:55.971 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:51:55.972 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:51:55.972 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:51:55.972 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:51:55.972 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:51:55.972 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:51:55.972 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:51:55.972 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:51:55.972 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:51:55.972 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:51:55.972 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:51:55.972 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:51:55.972 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:51:55.972 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:51:55.972 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:51:55.973 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:51:55.973 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:51:55.973 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:51:55.973 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:51:55.973 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:51:55.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.969+0000 7fb55b103700 1 -- 172.21.15.71:0/2558233967 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb54406cce0 msgr2=0x7fb54406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:55.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.969+0000 7fb55b103700 1 --2- 172.21.15.71:0/2558233967 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb54406cce0 0x7fb54406f1a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb548005fd0 tx=0x7fb548005ee0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.969+0000 7fb55b103700 1 -- 172.21.15.71:0/2558233967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb554101850 msgr2=0x7fb5540735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:51:55.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.969+0000 7fb55b103700 1 --2- 172.21.15.71:0/2558233967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb554101850 0x7fb5540735d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb5400092e0 tx=0x7fb54000be20 comp rx=0 tx=0).stop 2024-04-23T02:51:55.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.970+0000 7fb55b103700 1 -- 172.21.15.71:0/2558233967 shutdown_connections 2024-04-23T02:51:55.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.970+0000 7fb55b103700 1 --2- 172.21.15.71:0/2558233967 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb54406cce0 0x7fb54406f1a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.970+0000 7fb55b103700 1 --2- 172.21.15.71:0/2558233967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb554100ea0 0x7fb554073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.970+0000 7fb55b103700 1 --2- 172.21.15.71:0/2558233967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb554101850 0x7fb5540735d0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:51:55.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.970+0000 7fb55b103700 1 -- 172.21.15.71:0/2558233967 >> 172.21.15.71:0/2558233967 conn(0x7fb5540fc750 msgr2=0x7fb5540fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:51:55.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.970+0000 7fb55b103700 1 -- 172.21.15.71:0/2558233967 shutdown_connections 2024-04-23T02:51:55.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:51:55.970+0000 7fb55b103700 1 -- 172.21.15.71:0/2558233967 wait complete. 2024-04-23T02:51:55.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:55 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3645322876' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:51:55.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:55 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/532415237' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:51:55.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:55 smithi071 ceph-mon[144544]: from='client.35904 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:51:56.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:55 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3645322876' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:51:56.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:55 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/532415237' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:51:56.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:55 smithi192 ceph-mon[108617]: from='client.35904 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:51:56.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:56 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2558233967' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:51:56.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:56 smithi071 ceph-mon[144544]: pgmap v1160: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:57.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:56 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2558233967' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:51:57.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:56 smithi192 ceph-mon[108617]: pgmap v1160: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:58.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:51:58 smithi071 ceph-mon[144544]: pgmap v1161: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:51:59.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:51:58 smithi192 ceph-mon[108617]: pgmap v1161: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:00.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:00 smithi071 ceph-mon[144544]: pgmap v1162: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:01.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:00 smithi192 ceph-mon[108617]: pgmap v1162: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:02.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:02 smithi071 ceph-mon[144544]: pgmap v1163: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:03.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:02 smithi192 ceph-mon[108617]: pgmap v1163: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:04.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:04 smithi071 ceph-mon[144544]: pgmap v1164: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:05.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:04 smithi192 ceph-mon[108617]: pgmap v1164: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:52:06.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:52:06.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:06 smithi071 ceph-mon[144544]: pgmap v1165: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:07.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:06 smithi192 ceph-mon[108617]: pgmap v1165: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:08.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:08 smithi071 ceph-mon[144544]: pgmap v1166: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:09.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:08 smithi192 ceph-mon[108617]: pgmap v1166: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:10.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:10 smithi071 ceph-mon[144544]: pgmap v1167: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:11.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:10 smithi192 ceph-mon[108617]: pgmap v1167: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:12.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:12 smithi071 ceph-mon[144544]: pgmap v1168: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:13.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:12 smithi192 ceph-mon[108617]: pgmap v1168: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:14.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:14 smithi071 ceph-mon[144544]: pgmap v1169: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:15.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:14 smithi192 ceph-mon[108617]: pgmap v1169: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:16.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:16 smithi071 ceph-mon[144544]: pgmap v1170: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:17.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:16 smithi192 ceph-mon[108617]: pgmap v1170: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:18.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:18 smithi071 ceph-mon[144544]: pgmap v1171: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:19.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:18 smithi192 ceph-mon[108617]: pgmap v1171: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:20.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:52:20.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:52:21.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:20 smithi192 ceph-mon[108617]: pgmap v1172: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:21.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:20 smithi071 ceph-mon[144544]: pgmap v1172: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:22.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:22 smithi071 ceph-mon[144544]: pgmap v1173: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:23.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:22 smithi192 ceph-mon[108617]: pgmap v1173: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:24.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:24 smithi071 ceph-mon[144544]: pgmap v1174: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:25.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:24 smithi192 ceph-mon[108617]: pgmap v1174: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:26.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.085+0000 7f4a202fd700 1 -- 172.21.15.71:0/644711900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a18105bd0 msgr2=0x7f4a18105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:26.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.085+0000 7f4a202fd700 1 --2- 172.21.15.71:0/644711900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a18105bd0 0x7f4a18105fb0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009910 tx=0x7f4a14009c20 comp rx=0 tx=0).stop 2024-04-23T02:52:26.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.085+0000 7f4a202fd700 1 -- 172.21.15.71:0/644711900 shutdown_connections 2024-04-23T02:52:26.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.085+0000 7f4a202fd700 1 --2- 172.21.15.71:0/644711900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a180ffc20 0x7f4a18100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.085+0000 7f4a202fd700 1 --2- 172.21.15.71:0/644711900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a18105bd0 0x7f4a18105fb0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.085+0000 7f4a202fd700 1 -- 172.21.15.71:0/644711900 >> 172.21.15.71:0/644711900 conn(0x7f4a18075670 msgr2=0x7f4a18075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:26.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.085+0000 7f4a202fd700 1 -- 172.21.15.71:0/644711900 shutdown_connections 2024-04-23T02:52:26.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.085+0000 7f4a202fd700 1 -- 172.21.15.71:0/644711900 wait complete. 2024-04-23T02:52:26.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.086+0000 7f4a202fd700 1 Processor -- start 2024-04-23T02:52:26.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.086+0000 7f4a202fd700 1 -- start start 2024-04-23T02:52:26.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.086+0000 7f4a202fd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a180ffc20 0x7f4a18195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:26.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.086+0000 7f4a202fd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a18105bd0 0x7f4a18195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:26.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.086+0000 7f4a202fd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a18196260 con 0x7f4a180ffc20 2024-04-23T02:52:26.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.086+0000 7f4a202fd700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a18198f90 con 0x7f4a18105bd0 2024-04-23T02:52:26.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.086+0000 7f4a1e099700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a180ffc20 0x7f4a18195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:26.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.086+0000 7f4a1e099700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a180ffc20 0x7f4a18195690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33256/0 (socket says 172.21.15.71:33256) 2024-04-23T02:52:26.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.086+0000 7f4a1e099700 1 -- 172.21.15.71:0/3917474518 learned_addr learned my addr 172.21.15.71:0/3917474518 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:52:26.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.086+0000 7f4a1d898700 1 --2- 172.21.15.71:0/3917474518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a18105bd0 0x7f4a18195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:26.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.086+0000 7f4a1e099700 1 -- 172.21.15.71:0/3917474518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a18105bd0 msgr2=0x7f4a18195bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:26.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.086+0000 7f4a1e099700 1 --2- 172.21.15.71:0/3917474518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a18105bd0 0x7f4a18195bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.086+0000 7f4a1e099700 1 -- 172.21.15.71:0/3917474518 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a140095f0 con 0x7f4a180ffc20 2024-04-23T02:52:26.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.086+0000 7f4a1d898700 1 --2- 172.21.15.71:0/3917474518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a18105bd0 0x7f4a18195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:52:26.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.087+0000 7f4a1e099700 1 --2- 172.21.15.71:0/3917474518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a180ffc20 0x7f4a18195690 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f4a14006010 tx=0x7f4a1400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:26.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.087+0000 7f4a0f7fe700 1 -- 172.21.15.71:0/3917474518 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a1401d070 con 0x7f4a180ffc20 2024-04-23T02:52:26.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.087+0000 7f4a0f7fe700 1 -- 172.21.15.71:0/3917474518 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4a1400f460 con 0x7f4a180ffc20 2024-04-23T02:52:26.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.087+0000 7f4a202fd700 1 -- 172.21.15.71:0/3917474518 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a18199210 con 0x7f4a180ffc20 2024-04-23T02:52:26.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.087+0000 7f4a0f7fe700 1 -- 172.21.15.71:0/3917474518 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a14021620 con 0x7f4a180ffc20 2024-04-23T02:52:26.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.087+0000 7f4a202fd700 1 -- 172.21.15.71:0/3917474518 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a18199700 con 0x7f4a180ffc20 2024-04-23T02:52:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.088+0000 7f4a202fd700 1 -- 172.21.15.71:0/3917474518 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a181080f0 con 0x7f4a180ffc20 2024-04-23T02:52:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.088+0000 7f4a0f7fe700 1 -- 172.21.15.71:0/3917474518 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4a1400fa40 con 0x7f4a180ffc20 2024-04-23T02:52:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.089+0000 7f4a0f7fe700 1 --2- 172.21.15.71:0/3917474518 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4a0406cce0 0x7f4a0406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.089+0000 7f4a0f7fe700 1 -- 172.21.15.71:0/3917474518 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4a1408d790 con 0x7f4a180ffc20 2024-04-23T02:52:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.089+0000 7f4a1d898700 1 --2- 172.21.15.71:0/3917474518 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4a0406cce0 0x7f4a0406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:26.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.089+0000 7f4a1d898700 1 --2- 172.21.15.71:0/3917474518 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4a0406cce0 0x7f4a0406f1a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4a08005950 tx=0x7f4a080058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:26.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.094+0000 7f4a0f7fe700 1 -- 172.21.15.71:0/3917474518 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f4a14057800 con 0x7f4a180ffc20 2024-04-23T02:52:26.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.302+0000 7f4a202fd700 1 -- 172.21.15.71:0/3917474518 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a18196b30 con 0x7f4a0406cce0 2024-04-23T02:52:26.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.304+0000 7f4a0f7fe700 1 -- 172.21.15.71:0/3917474518 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4a0800b950 con 0x7f4a0406cce0 2024-04-23T02:52:26.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.306+0000 7f4a202fd700 1 -- 172.21.15.71:0/3917474518 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4a0406cce0 msgr2=0x7f4a0406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:26.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.306+0000 7f4a202fd700 1 --2- 172.21.15.71:0/3917474518 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4a0406cce0 0x7f4a0406f1a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4a08005950 tx=0x7f4a080058e0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.306+0000 7f4a202fd700 1 -- 172.21.15.71:0/3917474518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a180ffc20 msgr2=0x7f4a18195690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:26.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.306+0000 7f4a202fd700 1 --2- 172.21.15.71:0/3917474518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a180ffc20 0x7f4a18195690 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f4a14006010 tx=0x7f4a1400be20 comp rx=0 tx=0).stop 2024-04-23T02:52:26.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.306+0000 7f4a202fd700 1 -- 172.21.15.71:0/3917474518 shutdown_connections 2024-04-23T02:52:26.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.306+0000 7f4a202fd700 1 --2- 172.21.15.71:0/3917474518 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4a0406cce0 0x7f4a0406f1a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.306+0000 7f4a202fd700 1 --2- 172.21.15.71:0/3917474518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a180ffc20 0x7f4a18195690 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.306+0000 7f4a202fd700 1 --2- 172.21.15.71:0/3917474518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a18105bd0 0x7f4a18195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.306+0000 7f4a202fd700 1 -- 172.21.15.71:0/3917474518 >> 172.21.15.71:0/3917474518 conn(0x7f4a18075670 msgr2=0x7f4a180fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:26.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.306+0000 7f4a202fd700 1 -- 172.21.15.71:0/3917474518 shutdown_connections 2024-04-23T02:52:26.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.306+0000 7f4a202fd700 1 -- 172.21.15.71:0/3917474518 wait complete. 2024-04-23T02:52:26.318 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:52:26.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.410+0000 7efe64888700 1 -- 172.21.15.71:0/3184720794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe600ffb40 msgr2=0x7efe600fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:26.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.410+0000 7efe64888700 1 --2- 172.21.15.71:0/3184720794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe600ffb40 0x7efe600fffc0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7efe50009910 tx=0x7efe50009c20 comp rx=0 tx=0).stop 2024-04-23T02:52:26.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.410+0000 7efe64888700 1 -- 172.21.15.71:0/3184720794 shutdown_connections 2024-04-23T02:52:26.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.410+0000 7efe64888700 1 --2- 172.21.15.71:0/3184720794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe600ffb40 0x7efe600fffc0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.410+0000 7efe64888700 1 --2- 172.21.15.71:0/3184720794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe60105bf0 0x7efe60105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.410+0000 7efe64888700 1 -- 172.21.15.71:0/3184720794 >> 172.21.15.71:0/3184720794 conn(0x7efe600fb610 msgr2=0x7efe600fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:26.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.410+0000 7efe64888700 1 -- 172.21.15.71:0/3184720794 shutdown_connections 2024-04-23T02:52:26.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.411+0000 7efe64888700 1 -- 172.21.15.71:0/3184720794 wait complete. 2024-04-23T02:52:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.411+0000 7efe64888700 1 Processor -- start 2024-04-23T02:52:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.411+0000 7efe64888700 1 -- start start 2024-04-23T02:52:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.411+0000 7efe64888700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe600ffb40 0x7efe6019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.411+0000 7efe64888700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe60105bf0 0x7efe6019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.411+0000 7efe64888700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6019f2c0 con 0x7efe600ffb40 2024-04-23T02:52:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.411+0000 7efe64888700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe60198770 con 0x7efe60105bf0 2024-04-23T02:52:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.412+0000 7efe5e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe600ffb40 0x7efe6019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.412+0000 7efe5e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe600ffb40 0x7efe6019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33280/0 (socket says 172.21.15.71:33280) 2024-04-23T02:52:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.412+0000 7efe5e59c700 1 -- 172.21.15.71:0/3195521069 learned_addr learned my addr 172.21.15.71:0/3195521069 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:52:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.412+0000 7efe5e59c700 1 -- 172.21.15.71:0/3195521069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe60105bf0 msgr2=0x7efe6019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.412+0000 7efe5dd9b700 1 --2- 172.21.15.71:0/3195521069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe60105bf0 0x7efe6019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.412+0000 7efe5e59c700 1 --2- 172.21.15.71:0/3195521069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe60105bf0 0x7efe6019ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.412+0000 7efe5e59c700 1 -- 172.21.15.71:0/3195521069 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe500095f0 con 0x7efe600ffb40 2024-04-23T02:52:26.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.412+0000 7efe5e59c700 1 --2- 172.21.15.71:0/3195521069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe600ffb40 0x7efe6019e6f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7efe5400d900 tx=0x7efe5400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:26.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.412+0000 7efe4f7fe700 1 -- 172.21.15.71:0/3195521069 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe54004a00 con 0x7efe600ffb40 2024-04-23T02:52:26.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.412+0000 7efe64888700 1 -- 172.21.15.71:0/3195521069 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe60198a50 con 0x7efe600ffb40 2024-04-23T02:52:26.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.412+0000 7efe4f7fe700 1 -- 172.21.15.71:0/3195521069 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efe54005020 con 0x7efe600ffb40 2024-04-23T02:52:26.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.412+0000 7efe4f7fe700 1 -- 172.21.15.71:0/3195521069 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe54009c00 con 0x7efe600ffb40 2024-04-23T02:52:26.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.412+0000 7efe64888700 1 -- 172.21.15.71:0/3195521069 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe60198fa0 con 0x7efe600ffb40 2024-04-23T02:52:26.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.413+0000 7efe4f7fe700 1 -- 172.21.15.71:0/3195521069 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7efe54005190 con 0x7efe600ffb40 2024-04-23T02:52:26.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.414+0000 7efe64888700 1 -- 172.21.15.71:0/3195521069 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6004f2e0 con 0x7efe600ffb40 2024-04-23T02:52:26.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.414+0000 7efe4f7fe700 1 --2- 172.21.15.71:0/3195521069 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efe4806cce0 0x7efe4806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:26.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.414+0000 7efe4f7fe700 1 -- 172.21.15.71:0/3195521069 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7efe5408c770 con 0x7efe600ffb40 2024-04-23T02:52:26.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.414+0000 7efe5dd9b700 1 --2- 172.21.15.71:0/3195521069 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efe4806cce0 0x7efe4806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:26.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.414+0000 7efe5dd9b700 1 --2- 172.21.15.71:0/3195521069 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efe4806cce0 0x7efe4806f1a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efe5000b5c0 tx=0x7efe50005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:26.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.419+0000 7efe4f7fe700 1 -- 172.21.15.71:0/3195521069 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7efe5405a270 con 0x7efe600ffb40 2024-04-23T02:52:26.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.629+0000 7efe64888700 1 -- 172.21.15.71:0/3195521069 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efe60199320 con 0x7efe4806cce0 2024-04-23T02:52:26.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.630+0000 7efe4f7fe700 1 -- 172.21.15.71:0/3195521069 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7efe60199320 con 0x7efe4806cce0 2024-04-23T02:52:26.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.632+0000 7efe64888700 1 -- 172.21.15.71:0/3195521069 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efe4806cce0 msgr2=0x7efe4806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:26.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.632+0000 7efe64888700 1 --2- 172.21.15.71:0/3195521069 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efe4806cce0 0x7efe4806f1a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efe5000b5c0 tx=0x7efe50005fd0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.632+0000 7efe64888700 1 -- 172.21.15.71:0/3195521069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe600ffb40 msgr2=0x7efe6019e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:26.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.632+0000 7efe64888700 1 --2- 172.21.15.71:0/3195521069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe600ffb40 0x7efe6019e6f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7efe5400d900 tx=0x7efe5400dcc0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.632+0000 7efe64888700 1 -- 172.21.15.71:0/3195521069 shutdown_connections 2024-04-23T02:52:26.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.632+0000 7efe64888700 1 --2- 172.21.15.71:0/3195521069 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7efe4806cce0 0x7efe4806f1a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.632+0000 7efe64888700 1 --2- 172.21.15.71:0/3195521069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe600ffb40 0x7efe6019e6f0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.632+0000 7efe64888700 1 --2- 172.21.15.71:0/3195521069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe60105bf0 0x7efe6019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.632+0000 7efe64888700 1 -- 172.21.15.71:0/3195521069 >> 172.21.15.71:0/3195521069 conn(0x7efe600fb610 msgr2=0x7efe60068f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:26.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.632+0000 7efe64888700 1 -- 172.21.15.71:0/3195521069 shutdown_connections 2024-04-23T02:52:26.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.632+0000 7efe64888700 1 -- 172.21.15.71:0/3195521069 wait complete. 2024-04-23T02:52:26.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.735+0000 7f72390c9700 1 -- 172.21.15.71:0/3308004684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7234105bf0 msgr2=0x7f7234105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:26.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.735+0000 7f72390c9700 1 --2- 172.21.15.71:0/3308004684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7234105bf0 0x7f7234105fd0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f7224009910 tx=0x7f7224009c20 comp rx=0 tx=0).stop 2024-04-23T02:52:26.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.735+0000 7f72390c9700 1 -- 172.21.15.71:0/3308004684 shutdown_connections 2024-04-23T02:52:26.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.735+0000 7f72390c9700 1 --2- 172.21.15.71:0/3308004684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72340ffb40 0x7f72340fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.735+0000 7f72390c9700 1 --2- 172.21.15.71:0/3308004684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7234105bf0 0x7f7234105fd0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.735+0000 7f72390c9700 1 -- 172.21.15.71:0/3308004684 >> 172.21.15.71:0/3308004684 conn(0x7f72340fb610 msgr2=0x7f72340fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:26.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.735+0000 7f72390c9700 1 -- 172.21.15.71:0/3308004684 shutdown_connections 2024-04-23T02:52:26.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.735+0000 7f72390c9700 1 -- 172.21.15.71:0/3308004684 wait complete. 2024-04-23T02:52:26.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.736+0000 7f72390c9700 1 Processor -- start 2024-04-23T02:52:26.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.736+0000 7f72390c9700 1 -- start start 2024-04-23T02:52:26.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.737+0000 7f72390c9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72340ffb40 0x7f72341017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:26.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.737+0000 7f72390c9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7234105bf0 0x7f7234101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:26.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.737+0000 7f72390c9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7234102230 con 0x7f7234105bf0 2024-04-23T02:52:26.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.737+0000 7f72390c9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7234102370 con 0x7f72340ffb40 2024-04-23T02:52:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.737+0000 7f723259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7234105bf0 0x7f7234101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.737+0000 7f723259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7234105bf0 0x7f7234101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33304/0 (socket says 172.21.15.71:33304) 2024-04-23T02:52:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.737+0000 7f723259c700 1 -- 172.21.15.71:0/2167316117 learned_addr learned my addr 172.21.15.71:0/2167316117 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:52:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.737+0000 7f7232d9d700 1 --2- 172.21.15.71:0/2167316117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72340ffb40 0x7f72341017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.737+0000 7f723259c700 1 -- 172.21.15.71:0/2167316117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72340ffb40 msgr2=0x7f72341017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.737+0000 7f723259c700 1 --2- 172.21.15.71:0/2167316117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72340ffb40 0x7f72341017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.737+0000 7f723259c700 1 -- 172.21.15.71:0/2167316117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72240095f0 con 0x7f7234105bf0 2024-04-23T02:52:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.737+0000 7f723259c700 1 --2- 172.21.15.71:0/2167316117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7234105bf0 0x7f7234101cf0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f722800d8d0 tx=0x7f722800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.737+0000 7f721bfff700 1 -- 172.21.15.71:0/2167316117 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72280098a0 con 0x7f7234105bf0 2024-04-23T02:52:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.737+0000 7f72390c9700 1 -- 172.21.15.71:0/2167316117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72341a28e0 con 0x7f7234105bf0 2024-04-23T02:52:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.737+0000 7f721bfff700 1 -- 172.21.15.71:0/2167316117 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7228010460 con 0x7f7234105bf0 2024-04-23T02:52:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.737+0000 7f721bfff700 1 -- 172.21.15.71:0/2167316117 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7228009ea0 con 0x7f7234105bf0 2024-04-23T02:52:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.737+0000 7f72390c9700 1 -- 172.21.15.71:0/2167316117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72341a2cf0 con 0x7f7234105bf0 2024-04-23T02:52:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.738+0000 7f72390c9700 1 -- 172.21.15.71:0/2167316117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f723404f2e0 con 0x7f7234105bf0 2024-04-23T02:52:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.739+0000 7f721bfff700 1 -- 172.21.15.71:0/2167316117 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7228009a00 con 0x7f7234105bf0 2024-04-23T02:52:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.739+0000 7f721bfff700 1 --2- 172.21.15.71:0/2167316117 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f721c06cce0 0x7f721c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.739+0000 7f721bfff700 1 -- 172.21.15.71:0/2167316117 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f722808c450 con 0x7f7234105bf0 2024-04-23T02:52:26.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.739+0000 7f7232d9d700 1 --2- 172.21.15.71:0/2167316117 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f721c06cce0 0x7f721c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:26.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.740+0000 7f7232d9d700 1 --2- 172.21.15.71:0/2167316117 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f721c06cce0 0x7f721c06f1a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7224006010 tx=0x7f722401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:26.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.744+0000 7f721bfff700 1 -- 172.21.15.71:0/2167316117 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f72280564c0 con 0x7f7234105bf0 2024-04-23T02:52:26.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.946+0000 7f72390c9700 1 -- 172.21.15.71:0/2167316117 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7234069600 con 0x7f721c06cce0 2024-04-23T02:52:26.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.957+0000 7f721bfff700 1 -- 172.21.15.71:0/2167316117 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7234069600 con 0x7f721c06cce0 2024-04-23T02:52:26.960 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:52:26.960 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (36m) 14m ago 47m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:52:26.960 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (47m) 14m ago 47m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:52:26.961 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (46m) 14m ago 46m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:52:26.961 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (34m) 14m ago 47m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:52:26.961 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (33m) 14m ago 46m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:52:26.961 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (35m) 14m ago 47m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:52:26.961 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (30m) 14m ago 41m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:52:26.961 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (29m) 14m ago 41m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:52:26.961 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (14m) 14m ago 41m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:52:26.961 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (29m) 14m ago 41m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:52:26.961 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (38m) 14m ago 49m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:52:26.961 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (37m) 14m ago 46m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:52:26.961 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (34m) 14m ago 49m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:52:26.961 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (34m) 14m ago 46m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:52:26.962 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (37m) 14m ago 47m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:52:26.962 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (36m) 14m ago 46m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:52:26.962 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (33m) 14m ago 45m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:52:26.962 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (32m) 14m ago 44m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:52:26.962 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (32m) 14m ago 44m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:52:26.962 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (31m) 14m ago 44m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:52:26.962 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (31m) 14m ago 43m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:52:26.962 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (30m) 14m ago 43m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:52:26.962 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (36m) 14m ago 46m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:52:26.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.959+0000 7f72390c9700 1 -- 172.21.15.71:0/2167316117 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f721c06cce0 msgr2=0x7f721c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:26.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.959+0000 7f72390c9700 1 --2- 172.21.15.71:0/2167316117 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f721c06cce0 0x7f721c06f1a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7224006010 tx=0x7f722401a040 comp rx=0 tx=0).stop 2024-04-23T02:52:26.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.959+0000 7f72390c9700 1 -- 172.21.15.71:0/2167316117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7234105bf0 msgr2=0x7f7234101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:26.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.959+0000 7f72390c9700 1 --2- 172.21.15.71:0/2167316117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7234105bf0 0x7f7234101cf0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f722800d8d0 tx=0x7f722800dbe0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.959+0000 7f72390c9700 1 -- 172.21.15.71:0/2167316117 shutdown_connections 2024-04-23T02:52:26.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.959+0000 7f72390c9700 1 --2- 172.21.15.71:0/2167316117 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f721c06cce0 0x7f721c06f1a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.959+0000 7f72390c9700 1 --2- 172.21.15.71:0/2167316117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72340ffb40 0x7f72341017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.959+0000 7f72390c9700 1 --2- 172.21.15.71:0/2167316117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7234105bf0 0x7f7234101cf0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:26.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.959+0000 7f72390c9700 1 -- 172.21.15.71:0/2167316117 >> 172.21.15.71:0/2167316117 conn(0x7f72340fb610 msgr2=0x7f7234068ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:26.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.959+0000 7f72390c9700 1 -- 172.21.15.71:0/2167316117 shutdown_connections 2024-04-23T02:52:26.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:26.959+0000 7f72390c9700 1 -- 172.21.15.71:0/2167316117 wait complete. 2024-04-23T02:52:26.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:26 smithi071 ceph-mon[144544]: from='client.35912 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:52:26.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:26 smithi071 ceph-mon[144544]: from='client.35916 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:52:26.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:26 smithi071 ceph-mon[144544]: pgmap v1175: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:27.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.061+0000 7f0031790700 1 -- 172.21.15.71:0/3860153550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c100ea0 msgr2=0x7f002c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:27.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.061+0000 7f0031790700 1 --2- 172.21.15.71:0/3860153550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c100ea0 0x7f002c101280 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0020009910 tx=0x7f0020009c20 comp rx=0 tx=0).stop 2024-04-23T02:52:27.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.061+0000 7f0031790700 1 -- 172.21.15.71:0/3860153550 shutdown_connections 2024-04-23T02:52:27.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.061+0000 7f0031790700 1 --2- 172.21.15.71:0/3860153550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f002c101850 0x7f002c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:27.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.061+0000 7f0031790700 1 --2- 172.21.15.71:0/3860153550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c100ea0 0x7f002c101280 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:27.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.061+0000 7f0031790700 1 -- 172.21.15.71:0/3860153550 >> 172.21.15.71:0/3860153550 conn(0x7f002c0fc750 msgr2=0x7f002c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:27.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.061+0000 7f0031790700 1 -- 172.21.15.71:0/3860153550 shutdown_connections 2024-04-23T02:52:27.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.061+0000 7f0031790700 1 -- 172.21.15.71:0/3860153550 wait complete. 2024-04-23T02:52:27.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.062+0000 7f0031790700 1 Processor -- start 2024-04-23T02:52:27.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.062+0000 7f0031790700 1 -- start start 2024-04-23T02:52:27.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.062+0000 7f0031790700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f002c100ea0 0x7f002c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:27.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.062+0000 7f0031790700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c101850 0x7f002c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:27.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.062+0000 7f0031790700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c19d130 con 0x7f002c101850 2024-04-23T02:52:27.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.062+0000 7f0031790700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c1965e0 con 0x7f002c100ea0 2024-04-23T02:52:27.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.062+0000 7f002a7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c101850 0x7f002c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:27.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.062+0000 7f002a7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c101850 0x7f002c19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33316/0 (socket says 172.21.15.71:33316) 2024-04-23T02:52:27.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.062+0000 7f002a7fc700 1 -- 172.21.15.71:0/1199139335 learned_addr learned my addr 172.21.15.71:0/1199139335 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:52:27.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.062+0000 7f002affd700 1 --2- 172.21.15.71:0/1199139335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f002c100ea0 0x7f002c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:27.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.062+0000 7f002a7fc700 1 -- 172.21.15.71:0/1199139335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f002c100ea0 msgr2=0x7f002c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:27.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.062+0000 7f002a7fc700 1 --2- 172.21.15.71:0/1199139335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f002c100ea0 0x7f002c19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:27.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.062+0000 7f002a7fc700 1 -- 172.21.15.71:0/1199139335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00200095f0 con 0x7f002c101850 2024-04-23T02:52:27.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.063+0000 7f002a7fc700 1 --2- 172.21.15.71:0/1199139335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c101850 0x7f002c19caa0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f001c00d900 tx=0x7f001c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:27.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.063+0000 7f0013fff700 1 -- 172.21.15.71:0/1199139335 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f001c009900 con 0x7f002c101850 2024-04-23T02:52:27.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.063+0000 7f0031790700 1 -- 172.21.15.71:0/1199139335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002c1968c0 con 0x7f002c101850 2024-04-23T02:52:27.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.063+0000 7f0013fff700 1 -- 172.21.15.71:0/1199139335 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f001c010460 con 0x7f002c101850 2024-04-23T02:52:27.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.063+0000 7f0013fff700 1 -- 172.21.15.71:0/1199139335 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f001c00f5f0 con 0x7f002c101850 2024-04-23T02:52:27.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.063+0000 7f0031790700 1 -- 172.21.15.71:0/1199139335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002c196e10 con 0x7f002c101850 2024-04-23T02:52:27.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.064+0000 7f0031790700 1 -- 172.21.15.71:0/1199139335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002c109240 con 0x7f002c101850 2024-04-23T02:52:27.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.064+0000 7f0013fff700 1 -- 172.21.15.71:0/1199139335 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f001c0105d0 con 0x7f002c101850 2024-04-23T02:52:27.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.065+0000 7f0013fff700 1 --2- 172.21.15.71:0/1199139335 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f001406cce0 0x7f001406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:27.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.065+0000 7f002affd700 1 --2- 172.21.15.71:0/1199139335 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f001406cce0 0x7f001406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:27.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.065+0000 7f0013fff700 1 -- 172.21.15.71:0/1199139335 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f001c08c3c0 con 0x7f002c101850 2024-04-23T02:52:27.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.065+0000 7f002affd700 1 --2- 172.21.15.71:0/1199139335 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f001406cce0 0x7f001406f1a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0020006010 tx=0x7f002001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:27.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.069+0000 7f0013fff700 1 -- 172.21.15.71:0/1199139335 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f001c059ec0 con 0x7f002c101850 2024-04-23T02:52:27.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:26 smithi192 ceph-mon[108617]: from='client.35912 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:52:27.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:26 smithi192 ceph-mon[108617]: from='client.35916 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:52:27.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:26 smithi192 ceph-mon[108617]: pgmap v1175: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:27.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.343+0000 7f0031790700 1 -- 172.21.15.71:0/1199139335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f002c197030 con 0x7f002c101850 2024-04-23T02:52:27.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.343+0000 7f0013fff700 1 -- 172.21.15.71:0/1199139335 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f001c059a50 con 0x7f002c101850 2024-04-23T02:52:27.346 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:52:27.347 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:52:27.347 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:52:27.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:52:27.347 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:52:27.347 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:52:27.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:52:27.347 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:52:27.347 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:52:27.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:52:27.347 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:52:27.347 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:52:27.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:52:27.347 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:52:27.347 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:52:27.348 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:52:27.348 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:52:27.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.345+0000 7f0031790700 1 -- 172.21.15.71:0/1199139335 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f001406cce0 msgr2=0x7f001406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:27.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.345+0000 7f0031790700 1 --2- 172.21.15.71:0/1199139335 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f001406cce0 0x7f001406f1a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0020006010 tx=0x7f002001a040 comp rx=0 tx=0).stop 2024-04-23T02:52:27.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.346+0000 7f0031790700 1 -- 172.21.15.71:0/1199139335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c101850 msgr2=0x7f002c19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:27.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.346+0000 7f0031790700 1 --2- 172.21.15.71:0/1199139335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c101850 0x7f002c19caa0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f001c00d900 tx=0x7f001c00dcc0 comp rx=0 tx=0).stop 2024-04-23T02:52:27.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.346+0000 7f0031790700 1 -- 172.21.15.71:0/1199139335 shutdown_connections 2024-04-23T02:52:27.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.346+0000 7f0031790700 1 --2- 172.21.15.71:0/1199139335 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f001406cce0 0x7f001406f1a0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:27.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.346+0000 7f0031790700 1 --2- 172.21.15.71:0/1199139335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f002c100ea0 0x7f002c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:27.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.346+0000 7f0031790700 1 --2- 172.21.15.71:0/1199139335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c101850 0x7f002c19caa0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:27.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.346+0000 7f0031790700 1 -- 172.21.15.71:0/1199139335 >> 172.21.15.71:0/1199139335 conn(0x7f002c0fc750 msgr2=0x7f002c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:27.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.346+0000 7f0031790700 1 -- 172.21.15.71:0/1199139335 shutdown_connections 2024-04-23T02:52:27.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.346+0000 7f0031790700 1 -- 172.21.15.71:0/1199139335 wait complete. 2024-04-23T02:52:27.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.448+0000 7fe5c5458700 1 -- 172.21.15.71:0/3417172293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c0100750 msgr2=0x7fe5c01047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:27.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.448+0000 7fe5c5458700 1 --2- 172.21.15.71:0/3417172293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c0100750 0x7fe5c01047a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009910 tx=0x7fe5b0009c20 comp rx=0 tx=0).stop 2024-04-23T02:52:27.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.448+0000 7fe5c5458700 1 -- 172.21.15.71:0/3417172293 shutdown_connections 2024-04-23T02:52:27.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.448+0000 7fe5c5458700 1 --2- 172.21.15.71:0/3417172293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c0100750 0x7fe5c01047a0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:27.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.448+0000 7fe5c5458700 1 --2- 172.21.15.71:0/3417172293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5c00ffda0 0x7fe5c0100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:27.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.448+0000 7fe5c5458700 1 -- 172.21.15.71:0/3417172293 >> 172.21.15.71:0/3417172293 conn(0x7fe5c00fb650 msgr2=0x7fe5c00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:27.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.448+0000 7fe5c5458700 1 -- 172.21.15.71:0/3417172293 shutdown_connections 2024-04-23T02:52:27.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.448+0000 7fe5c5458700 1 -- 172.21.15.71:0/3417172293 wait complete. 2024-04-23T02:52:27.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.449+0000 7fe5c5458700 1 Processor -- start 2024-04-23T02:52:27.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.449+0000 7fe5c5458700 1 -- start start 2024-04-23T02:52:27.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.449+0000 7fe5c5458700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5c00ffda0 0x7fe5c019b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:27.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.449+0000 7fe5c5458700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c0100750 0x7fe5c019bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:27.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.449+0000 7fe5c5458700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c019c230 con 0x7fe5c0100750 2024-04-23T02:52:27.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.449+0000 7fe5c5458700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c01956e0 con 0x7fe5c00ffda0 2024-04-23T02:52:27.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.449+0000 7fe5be7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c0100750 0x7fe5c019bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:27.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.449+0000 7fe5be7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c0100750 0x7fe5c019bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33338/0 (socket says 172.21.15.71:33338) 2024-04-23T02:52:27.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.449+0000 7fe5be7fc700 1 -- 172.21.15.71:0/2816444397 learned_addr learned my addr 172.21.15.71:0/2816444397 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:52:27.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.450+0000 7fe5beffd700 1 --2- 172.21.15.71:0/2816444397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5c00ffda0 0x7fe5c019b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:27.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.450+0000 7fe5be7fc700 1 -- 172.21.15.71:0/2816444397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5c00ffda0 msgr2=0x7fe5c019b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:27.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.450+0000 7fe5be7fc700 1 --2- 172.21.15.71:0/2816444397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5c00ffda0 0x7fe5c019b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:27.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.450+0000 7fe5be7fc700 1 -- 172.21.15.71:0/2816444397 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b00095f0 con 0x7fe5c0100750 2024-04-23T02:52:27.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.450+0000 7fe5beffd700 1 --2- 172.21.15.71:0/2816444397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5c00ffda0 0x7fe5c019b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:52:27.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.450+0000 7fe5be7fc700 1 --2- 172.21.15.71:0/2816444397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c0100750 0x7fe5c019bba0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00092e0 tx=0x7fe5b000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:27.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.450+0000 7fe5a7fff700 1 -- 172.21.15.71:0/2816444397 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5b001f070 con 0x7fe5c0100750 2024-04-23T02:52:27.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.450+0000 7fe5a7fff700 1 -- 172.21.15.71:0/2816444397 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe5b0019610 con 0x7fe5c0100750 2024-04-23T02:52:27.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.450+0000 7fe5c5458700 1 -- 172.21.15.71:0/2816444397 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c0195960 con 0x7fe5c0100750 2024-04-23T02:52:27.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.450+0000 7fe5a7fff700 1 -- 172.21.15.71:0/2816444397 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5b000f690 con 0x7fe5c0100750 2024-04-23T02:52:27.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.450+0000 7fe5c5458700 1 -- 172.21.15.71:0/2816444397 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c0195e50 con 0x7fe5c0100750 2024-04-23T02:52:27.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.451+0000 7fe5c5458700 1 -- 172.21.15.71:0/2816444397 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c004f2e0 con 0x7fe5c0100750 2024-04-23T02:52:27.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.452+0000 7fe5a7fff700 1 -- 172.21.15.71:0/2816444397 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe5b0019780 con 0x7fe5c0100750 2024-04-23T02:52:27.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.452+0000 7fe5a7fff700 1 --2- 172.21.15.71:0/2816444397 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe5a806cce0 0x7fe5a806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:27.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.452+0000 7fe5a7fff700 1 -- 172.21.15.71:0/2816444397 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe5b008fa70 con 0x7fe5c0100750 2024-04-23T02:52:27.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.452+0000 7fe5beffd700 1 --2- 172.21.15.71:0/2816444397 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe5a806cce0 0x7fe5a806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:27.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.453+0000 7fe5beffd700 1 --2- 172.21.15.71:0/2816444397 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe5a806cce0 0x7fe5a806f1a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe5b40099d0 tx=0x7fe5b4008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:27.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.457+0000 7fe5a7fff700 1 -- 172.21.15.71:0/2816444397 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe5b005d570 con 0x7fe5c0100750 2024-04-23T02:52:27.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.689+0000 7fe5c5458700 1 -- 172.21.15.71:0/2816444397 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe5c0196bf0 con 0x7fe5c0100750 2024-04-23T02:52:27.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.689+0000 7fe5a7fff700 1 -- 172.21.15.71:0/2816444397 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1838 (secure 0 0 0) 0x7fe5b005d100 con 0x7fe5c0100750 2024-04-23T02:52:27.692 INFO:teuthology.orchestra.run.smithi071.stdout:e36 2024-04-23T02:52:27.692 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:52:27.692 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:52:27.692 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:52:27.692 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:52:27.692 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:52:27.692 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:52:27.692 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 27 2024-04-23T02:52:27.693 INFO:teuthology.orchestra.run.smithi071.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-23T02:52:27.693 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:52:27.693 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:22:00.280923+0000 2024-04-23T02:52:27.693 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:52:27.693 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:52:27.693 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:52:27.693 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:52:27.693 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:52:27.693 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:52:27.693 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:52:27.693 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:52:27.693 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:52:27.693 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:52:27.693 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:52:27.693 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:52:27.694 INFO:teuthology.orchestra.run.smithi071.stdout:up {} 2024-04-23T02:52:27.694 INFO:teuthology.orchestra.run.smithi071.stdout:failed 0,1 2024-04-23T02:52:27.694 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:52:27.694 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:52:27.694 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:52:27.694 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:52:27.694 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:52:27.694 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:52:27.694 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:52:27.694 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:52:27.694 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:52:27.694 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:52:27.694 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:52:27.694 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:52:27.695 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{-1:34334} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:52:27.695 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:52:27.695 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:52:27.695 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:52:27.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.691+0000 7fe5c5458700 1 -- 172.21.15.71:0/2816444397 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe5a806cce0 msgr2=0x7fe5a806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:27.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.691+0000 7fe5c5458700 1 --2- 172.21.15.71:0/2816444397 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe5a806cce0 0x7fe5a806f1a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe5b40099d0 tx=0x7fe5b4008040 comp rx=0 tx=0).stop 2024-04-23T02:52:27.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.691+0000 7fe5c5458700 1 -- 172.21.15.71:0/2816444397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c0100750 msgr2=0x7fe5c019bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:27.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.691+0000 7fe5c5458700 1 --2- 172.21.15.71:0/2816444397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c0100750 0x7fe5c019bba0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00092e0 tx=0x7fe5b000be20 comp rx=0 tx=0).stop 2024-04-23T02:52:27.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.691+0000 7fe5c5458700 1 -- 172.21.15.71:0/2816444397 shutdown_connections 2024-04-23T02:52:27.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.691+0000 7fe5c5458700 1 --2- 172.21.15.71:0/2816444397 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fe5a806cce0 0x7fe5a806f1a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:27.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.691+0000 7fe5c5458700 1 --2- 172.21.15.71:0/2816444397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5c00ffda0 0x7fe5c019b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:27.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.691+0000 7fe5c5458700 1 --2- 172.21.15.71:0/2816444397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5c0100750 0x7fe5c019bba0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:27.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.691+0000 7fe5c5458700 1 -- 172.21.15.71:0/2816444397 >> 172.21.15.71:0/2816444397 conn(0x7fe5c00fb650 msgr2=0x7fe5c00fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:27.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.691+0000 7fe5c5458700 1 -- 172.21.15.71:0/2816444397 shutdown_connections 2024-04-23T02:52:27.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.691+0000 7fe5c5458700 1 -- 172.21.15.71:0/2816444397 wait complete. 2024-04-23T02:52:27.696 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:52:27.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.797+0000 7f9592bcd700 1 -- 172.21.15.71:0/373688138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f958c0ffc20 msgr2=0x7f958c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:27.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.797+0000 7f9592bcd700 1 --2- 172.21.15.71:0/373688138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f958c0ffc20 0x7f958c100080 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f9578009910 tx=0x7f9578009c20 comp rx=0 tx=0).stop 2024-04-23T02:52:27.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.797+0000 7f9592bcd700 1 -- 172.21.15.71:0/373688138 shutdown_connections 2024-04-23T02:52:27.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.797+0000 7f9592bcd700 1 --2- 172.21.15.71:0/373688138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f958c0ffc20 0x7f958c100080 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:27.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.797+0000 7f9592bcd700 1 --2- 172.21.15.71:0/373688138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f958c105bd0 0x7f958c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:27.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.797+0000 7f9592bcd700 1 -- 172.21.15.71:0/373688138 >> 172.21.15.71:0/373688138 conn(0x7f958c075670 msgr2=0x7f958c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:27.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.797+0000 7f9592bcd700 1 -- 172.21.15.71:0/373688138 shutdown_connections 2024-04-23T02:52:27.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.797+0000 7f9592bcd700 1 -- 172.21.15.71:0/373688138 wait complete. 2024-04-23T02:52:27.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.798+0000 7f9592bcd700 1 Processor -- start 2024-04-23T02:52:27.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.798+0000 7f9592bcd700 1 -- start start 2024-04-23T02:52:27.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.798+0000 7f9592bcd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f958c0ffc20 0x7f958c19c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:27.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.798+0000 7f9592bcd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f958c105bd0 0x7f958c19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:27.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.798+0000 7f9592bcd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c19d1c0 con 0x7f958c0ffc20 2024-04-23T02:52:27.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.798+0000 7f9592bcd700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c196670 con 0x7f958c105bd0 2024-04-23T02:52:27.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.798+0000 7f9590969700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f958c0ffc20 0x7f958c19c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:27.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.798+0000 7f9590969700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f958c0ffc20 0x7f958c19c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33356/0 (socket says 172.21.15.71:33356) 2024-04-23T02:52:27.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.798+0000 7f9590969700 1 -- 172.21.15.71:0/2391231941 learned_addr learned my addr 172.21.15.71:0/2391231941 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:52:27.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.799+0000 7f9590969700 1 -- 172.21.15.71:0/2391231941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f958c105bd0 msgr2=0x7f958c19cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:27.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.799+0000 7f958bfff700 1 --2- 172.21.15.71:0/2391231941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f958c105bd0 0x7f958c19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:27.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.799+0000 7f9590969700 1 --2- 172.21.15.71:0/2391231941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f958c105bd0 0x7f958c19cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:27.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.799+0000 7f9590969700 1 -- 172.21.15.71:0/2391231941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95780095f0 con 0x7f958c0ffc20 2024-04-23T02:52:27.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.799+0000 7f9590969700 1 --2- 172.21.15.71:0/2391231941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f958c0ffc20 0x7f958c19c5f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f958000c8a0 tx=0x7f958000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:27.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.799+0000 7f9589ffb700 1 -- 172.21.15.71:0/2391231941 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f958000cea0 con 0x7f958c0ffc20 2024-04-23T02:52:27.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.799+0000 7f9589ffb700 1 -- 172.21.15.71:0/2391231941 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9580004830 con 0x7f958c0ffc20 2024-04-23T02:52:27.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.799+0000 7f9592bcd700 1 -- 172.21.15.71:0/2391231941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958c196950 con 0x7f958c0ffc20 2024-04-23T02:52:27.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.799+0000 7f9589ffb700 1 -- 172.21.15.71:0/2391231941 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9580005500 con 0x7f958c0ffc20 2024-04-23T02:52:27.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.799+0000 7f9592bcd700 1 -- 172.21.15.71:0/2391231941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958c196ea0 con 0x7f958c0ffc20 2024-04-23T02:52:27.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.800+0000 7f9592bcd700 1 -- 172.21.15.71:0/2391231941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958c04f2e0 con 0x7f958c0ffc20 2024-04-23T02:52:27.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.800+0000 7f9589ffb700 1 -- 172.21.15.71:0/2391231941 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f95800049a0 con 0x7f958c0ffc20 2024-04-23T02:52:27.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.801+0000 7f9589ffb700 1 --2- 172.21.15.71:0/2391231941 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f957c06cce0 0x7f957c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:27.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.801+0000 7f9589ffb700 1 -- 172.21.15.71:0/2391231941 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f958008c250 con 0x7f958c0ffc20 2024-04-23T02:52:27.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.801+0000 7f958bfff700 1 --2- 172.21.15.71:0/2391231941 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f957c06cce0 0x7f957c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:27.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.801+0000 7f958bfff700 1 --2- 172.21.15.71:0/2391231941 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f957c06cce0 0x7f957c06f1a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f957800b5c0 tx=0x7f9578005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:27.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:27.805+0000 7f9589ffb700 1 -- 172.21.15.71:0/2391231941 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9580059d50 con 0x7f958c0ffc20 2024-04-23T02:52:27.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:27 smithi071 ceph-mon[144544]: from='client.35920 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:52:27.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:27 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1199139335' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:52:27.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:27 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2816444397' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:52:28.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.017+0000 7f9592bcd700 1 -- 172.21.15.71:0/2391231941 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f958c197190 con 0x7f957c06cce0 2024-04-23T02:52:28.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.018+0000 7f9589ffb700 1 -- 172.21.15.71:0/2391231941 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f958c197190 con 0x7f957c06cce0 2024-04-23T02:52:28.021 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:52:28.021 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:52:28.021 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:52:28.022 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:52:28.022 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:52:28.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:52:28.022 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:52:28.022 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:52:28.022 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:52:28.022 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:52:28.022 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:52:28.022 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:52:28.022 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading mds daemons", 2024-04-23T02:52:28.022 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:52:28.022 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:52:28.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.021+0000 7f9592bcd700 1 -- 172.21.15.71:0/2391231941 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f957c06cce0 msgr2=0x7f957c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:28.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.021+0000 7f9592bcd700 1 --2- 172.21.15.71:0/2391231941 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f957c06cce0 0x7f957c06f1a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f957800b5c0 tx=0x7f9578005fd0 comp rx=0 tx=0).stop 2024-04-23T02:52:28.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.021+0000 7f9592bcd700 1 -- 172.21.15.71:0/2391231941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f958c0ffc20 msgr2=0x7f958c19c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:28.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.021+0000 7f9592bcd700 1 --2- 172.21.15.71:0/2391231941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f958c0ffc20 0x7f958c19c5f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f958000c8a0 tx=0x7f958000cc60 comp rx=0 tx=0).stop 2024-04-23T02:52:28.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.021+0000 7f9592bcd700 1 -- 172.21.15.71:0/2391231941 shutdown_connections 2024-04-23T02:52:28.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.021+0000 7f9592bcd700 1 --2- 172.21.15.71:0/2391231941 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f957c06cce0 0x7f957c06f1a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:28.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.021+0000 7f9592bcd700 1 --2- 172.21.15.71:0/2391231941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f958c0ffc20 0x7f958c19c5f0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:28.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.021+0000 7f9592bcd700 1 --2- 172.21.15.71:0/2391231941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f958c105bd0 0x7f958c19cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:28.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.021+0000 7f9592bcd700 1 -- 172.21.15.71:0/2391231941 >> 172.21.15.71:0/2391231941 conn(0x7f958c075670 msgr2=0x7f958c103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:28.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.021+0000 7f9592bcd700 1 -- 172.21.15.71:0/2391231941 shutdown_connections 2024-04-23T02:52:28.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.021+0000 7f9592bcd700 1 -- 172.21.15.71:0/2391231941 wait complete. 2024-04-23T02:52:28.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:27 smithi192 ceph-mon[108617]: from='client.35920 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:52:28.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:27 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1199139335' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:52:28.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:27 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2816444397' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:52:28.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.123+0000 7f43c5bdf700 1 -- 172.21.15.71:0/3497946544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43c0100ea0 msgr2=0x7f43c0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:28.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.123+0000 7f43c5bdf700 1 --2- 172.21.15.71:0/3497946544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43c0100ea0 0x7f43c0101280 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009910 tx=0x7f43b0009c20 comp rx=0 tx=0).stop 2024-04-23T02:52:28.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.123+0000 7f43c5bdf700 1 -- 172.21.15.71:0/3497946544 shutdown_connections 2024-04-23T02:52:28.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.123+0000 7f43c5bdf700 1 --2- 172.21.15.71:0/3497946544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c0101850 0x7f43c01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:28.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.123+0000 7f43c5bdf700 1 --2- 172.21.15.71:0/3497946544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43c0100ea0 0x7f43c0101280 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:28.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.123+0000 7f43c5bdf700 1 -- 172.21.15.71:0/3497946544 >> 172.21.15.71:0/3497946544 conn(0x7f43c00fc750 msgr2=0x7f43c00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:28.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.123+0000 7f43c5bdf700 1 -- 172.21.15.71:0/3497946544 shutdown_connections 2024-04-23T02:52:28.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.123+0000 7f43c5bdf700 1 -- 172.21.15.71:0/3497946544 wait complete. 2024-04-23T02:52:28.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.124+0000 7f43c5bdf700 1 Processor -- start 2024-04-23T02:52:28.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.124+0000 7f43c5bdf700 1 -- start start 2024-04-23T02:52:28.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.124+0000 7f43c5bdf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43c0100ea0 0x7f43c0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:28.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.124+0000 7f43c5bdf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c0101850 0x7f43c00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:28.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.124+0000 7f43c5bdf700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c0073b10 con 0x7f43c0100ea0 2024-04-23T02:52:28.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.124+0000 7f43c5bdf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c0073c50 con 0x7f43c0101850 2024-04-23T02:52:28.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.124+0000 7f43bf7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43c0100ea0 0x7f43c0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:28.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.124+0000 7f43bf7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43c0100ea0 0x7f43c0073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33370/0 (socket says 172.21.15.71:33370) 2024-04-23T02:52:28.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.124+0000 7f43bf7fe700 1 -- 172.21.15.71:0/286274621 learned_addr learned my addr 172.21.15.71:0/286274621 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:52:28.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.124+0000 7f43bf7fe700 1 -- 172.21.15.71:0/286274621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c0101850 msgr2=0x7f43c00735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:28.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.124+0000 7f43bf7fe700 1 --2- 172.21.15.71:0/286274621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c0101850 0x7f43c00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:28.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.124+0000 7f43bf7fe700 1 -- 172.21.15.71:0/286274621 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b00095f0 con 0x7f43c0100ea0 2024-04-23T02:52:28.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.125+0000 7f43bf7fe700 1 --2- 172.21.15.71:0/286274621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43c0100ea0 0x7f43c0073090 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f43b0006010 tx=0x7f43b000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:28.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.125+0000 7f43bcff9700 1 -- 172.21.15.71:0/286274621 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43b001d070 con 0x7f43c0100ea0 2024-04-23T02:52:28.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.125+0000 7f43bcff9700 1 -- 172.21.15.71:0/286274621 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f43b000f460 con 0x7f43c0100ea0 2024-04-23T02:52:28.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.125+0000 7f43c5bdf700 1 -- 172.21.15.71:0/286274621 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c0073ed0 con 0x7f43c0100ea0 2024-04-23T02:52:28.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.125+0000 7f43bcff9700 1 -- 172.21.15.71:0/286274621 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43b0021640 con 0x7f43c0100ea0 2024-04-23T02:52:28.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.125+0000 7f43c5bdf700 1 -- 172.21.15.71:0/286274621 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c01a2c90 con 0x7f43c0100ea0 2024-04-23T02:52:28.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.126+0000 7f43c5bdf700 1 -- 172.21.15.71:0/286274621 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c0109240 con 0x7f43c0100ea0 2024-04-23T02:52:28.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.126+0000 7f43bcff9700 1 -- 172.21.15.71:0/286274621 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f43b000fa60 con 0x7f43c0100ea0 2024-04-23T02:52:28.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.127+0000 7f43bcff9700 1 --2- 172.21.15.71:0/286274621 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f43a806cce0 0x7f43a806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:28.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.127+0000 7f43bcff9700 1 -- 172.21.15.71:0/286274621 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f43b008d860 con 0x7f43c0100ea0 2024-04-23T02:52:28.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.127+0000 7f43beffd700 1 --2- 172.21.15.71:0/286274621 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f43a806cce0 0x7f43a806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:28.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.127+0000 7f43beffd700 1 --2- 172.21.15.71:0/286274621 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f43a806cce0 0x7f43a806f1a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f43c0074880 tx=0x7f43b4005c80 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:28.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.131+0000 7f43bcff9700 1 -- 172.21.15.71:0/286274621 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f43b00578d0 con 0x7f43c0100ea0 2024-04-23T02:52:28.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.399+0000 7f43c5bdf700 1 -- 172.21.15.71:0/286274621 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f43c004f2e0 con 0x7f43c0100ea0 2024-04-23T02:52:28.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.399+0000 7f43bcff9700 1 -- 172.21.15.71:0/286274621 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3773 (secure 0 0 0) 0x7f43b005aef0 con 0x7f43c0100ea0 2024-04-23T02:52:28.402 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-23T02:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:52:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:52:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:52:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:52:28.404 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpe_7hwgfc:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:52:28.404 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:52:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:52:28.404 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:52:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:52:28.404 INFO:teuthology.orchestra.run.smithi071.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T02:52:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-23T02:52:28.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.401+0000 7f43c5bdf700 1 -- 172.21.15.71:0/286274621 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f43a806cce0 msgr2=0x7f43a806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:28.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.401+0000 7f43c5bdf700 1 --2- 172.21.15.71:0/286274621 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f43a806cce0 0x7f43a806f1a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f43c0074880 tx=0x7f43b4005c80 comp rx=0 tx=0).stop 2024-04-23T02:52:28.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.401+0000 7f43c5bdf700 1 -- 172.21.15.71:0/286274621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43c0100ea0 msgr2=0x7f43c0073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:28.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.401+0000 7f43c5bdf700 1 --2- 172.21.15.71:0/286274621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43c0100ea0 0x7f43c0073090 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f43b0006010 tx=0x7f43b000be20 comp rx=0 tx=0).stop 2024-04-23T02:52:28.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.402+0000 7f43c5bdf700 1 -- 172.21.15.71:0/286274621 shutdown_connections 2024-04-23T02:52:28.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.402+0000 7f43c5bdf700 1 --2- 172.21.15.71:0/286274621 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f43a806cce0 0x7f43a806f1a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:28.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.402+0000 7f43c5bdf700 1 --2- 172.21.15.71:0/286274621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43c0100ea0 0x7f43c0073090 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:28.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.402+0000 7f43c5bdf700 1 --2- 172.21.15.71:0/286274621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43c0101850 0x7f43c00735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:28.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.402+0000 7f43c5bdf700 1 -- 172.21.15.71:0/286274621 >> 172.21.15.71:0/286274621 conn(0x7f43c00fc750 msgr2=0x7f43c00fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:28.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.402+0000 7f43c5bdf700 1 -- 172.21.15.71:0/286274621 shutdown_connections 2024-04-23T02:52:28.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:28.402+0000 7f43c5bdf700 1 -- 172.21.15.71:0/286274621 wait complete. 2024-04-23T02:52:28.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:28 smithi071 ceph-mon[144544]: from='client.35932 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:52:28.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:28 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/286274621' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:52:28.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:28 smithi071 ceph-mon[144544]: pgmap v1176: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:29.045 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:28 smithi192 ceph-mon[108617]: from='client.35932 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:52:29.045 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:28 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/286274621' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:52:29.045 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:28 smithi192 ceph-mon[108617]: pgmap v1176: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:30 smithi071 ceph-mon[144544]: pgmap v1177: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:31.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:30 smithi192 ceph-mon[108617]: pgmap v1177: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:32.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:32 smithi071 ceph-mon[144544]: pgmap v1178: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:33.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:32 smithi192 ceph-mon[108617]: pgmap v1178: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:34.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:34 smithi071 ceph-mon[144544]: pgmap v1179: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:35.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:34 smithi192 ceph-mon[108617]: pgmap v1179: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:35.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:52:36.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:52:36.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:36 smithi071 ceph-mon[144544]: pgmap v1180: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:37.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:36 smithi192 ceph-mon[108617]: pgmap v1180: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:38.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:38 smithi071 ceph-mon[144544]: pgmap v1181: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:39.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:38 smithi192 ceph-mon[108617]: pgmap v1181: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:40.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:40 smithi071 ceph-mon[144544]: pgmap v1182: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:41.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:40 smithi192 ceph-mon[108617]: pgmap v1182: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:42.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:42 smithi071 ceph-mon[144544]: pgmap v1183: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:43.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:42 smithi192 ceph-mon[108617]: pgmap v1183: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:44.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:44 smithi071 ceph-mon[144544]: pgmap v1184: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:45.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:44 smithi192 ceph-mon[108617]: pgmap v1184: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:46.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:46 smithi071 ceph-mon[144544]: pgmap v1185: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:47.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:46 smithi192 ceph-mon[108617]: pgmap v1185: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:48.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:48 smithi071 ceph-mon[144544]: pgmap v1186: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:49.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:48 smithi192 ceph-mon[108617]: pgmap v1186: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:50.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:49 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:52:50.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:49 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:52:51.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:50 smithi192 ceph-mon[108617]: pgmap v1187: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:51.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:50 smithi071 ceph-mon[144544]: pgmap v1187: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:52.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:52 smithi071 ceph-mon[144544]: pgmap v1188: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:53.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:52 smithi192 ceph-mon[108617]: pgmap v1188: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:54.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:54 smithi071 ceph-mon[144544]: pgmap v1189: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:55.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:54 smithi192 ceph-mon[108617]: pgmap v1189: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:56.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:56 smithi071 ceph-mon[144544]: pgmap v1190: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:57.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:56 smithi192 ceph-mon[108617]: pgmap v1190: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:58.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.520+0000 7f69ee13b700 1 -- 172.21.15.71:0/2943819794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69e8101840 msgr2=0x7f69e8105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:58.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.520+0000 7f69ee13b700 1 --2- 172.21.15.71:0/2943819794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69e8101840 0x7f69e8105890 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009910 tx=0x7f69d8009c20 comp rx=0 tx=0).stop 2024-04-23T02:52:58.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.520+0000 7f69ee13b700 1 -- 172.21.15.71:0/2943819794 shutdown_connections 2024-04-23T02:52:58.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.520+0000 7f69ee13b700 1 --2- 172.21.15.71:0/2943819794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69e8101840 0x7f69e8105890 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:58.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.520+0000 7f69ee13b700 1 --2- 172.21.15.71:0/2943819794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69e8100e90 0x7f69e8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:58.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.520+0000 7f69ee13b700 1 -- 172.21.15.71:0/2943819794 >> 172.21.15.71:0/2943819794 conn(0x7f69e80fc740 msgr2=0x7f69e80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:58.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.520+0000 7f69ee13b700 1 -- 172.21.15.71:0/2943819794 shutdown_connections 2024-04-23T02:52:58.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.520+0000 7f69ee13b700 1 -- 172.21.15.71:0/2943819794 wait complete. 2024-04-23T02:52:58.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.521+0000 7f69ee13b700 1 Processor -- start 2024-04-23T02:52:58.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.521+0000 7f69ee13b700 1 -- start start 2024-04-23T02:52:58.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.521+0000 7f69ee13b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69e8100e90 0x7f69e819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:58.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.521+0000 7f69ee13b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69e8101840 0x7f69e819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:58.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.521+0000 7f69ee13b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e819d240 con 0x7f69e8101840 2024-04-23T02:52:58.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.521+0000 7f69ee13b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e81966f0 con 0x7f69e8100e90 2024-04-23T02:52:58.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.521+0000 7f69e6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69e8101840 0x7f69e819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:58.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.521+0000 7f69e6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69e8101840 0x7f69e819cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57416/0 (socket says 172.21.15.71:57416) 2024-04-23T02:52:58.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.521+0000 7f69e6ffd700 1 -- 172.21.15.71:0/850090877 learned_addr learned my addr 172.21.15.71:0/850090877 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:52:58.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.521+0000 7f69e6ffd700 1 -- 172.21.15.71:0/850090877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69e8100e90 msgr2=0x7f69e819c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:58.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.521+0000 7f69e77fe700 1 --2- 172.21.15.71:0/850090877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69e8100e90 0x7f69e819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:58.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.522+0000 7f69e6ffd700 1 --2- 172.21.15.71:0/850090877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69e8100e90 0x7f69e819c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:58.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.522+0000 7f69e6ffd700 1 -- 172.21.15.71:0/850090877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d80095f0 con 0x7f69e8101840 2024-04-23T02:52:58.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.522+0000 7f69e6ffd700 1 --2- 172.21.15.71:0/850090877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69e8101840 0x7f69e819cbb0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f69d80092e0 tx=0x7f69d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:58.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.522+0000 7f69e4ff9700 1 -- 172.21.15.71:0/850090877 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69d801d070 con 0x7f69e8101840 2024-04-23T02:52:58.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.522+0000 7f69ee13b700 1 -- 172.21.15.71:0/850090877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e8196970 con 0x7f69e8101840 2024-04-23T02:52:58.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.522+0000 7f69e4ff9700 1 -- 172.21.15.71:0/850090877 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f69d800f460 con 0x7f69e8101840 2024-04-23T02:52:58.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.522+0000 7f69e4ff9700 1 -- 172.21.15.71:0/850090877 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69d8021600 con 0x7f69e8101840 2024-04-23T02:52:58.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.522+0000 7f69ee13b700 1 -- 172.21.15.71:0/850090877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e8196e60 con 0x7f69e8101840 2024-04-23T02:52:58.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.523+0000 7f69ee13b700 1 -- 172.21.15.71:0/850090877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e81091e0 con 0x7f69e8101840 2024-04-23T02:52:58.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.524+0000 7f69e4ff9700 1 -- 172.21.15.71:0/850090877 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f69d800f5d0 con 0x7f69e8101840 2024-04-23T02:52:58.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.524+0000 7f69e4ff9700 1 --2- 172.21.15.71:0/850090877 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f69d006cce0 0x7f69d006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:58.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.524+0000 7f69e4ff9700 1 -- 172.21.15.71:0/850090877 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f69d808dcd0 con 0x7f69e8101840 2024-04-23T02:52:58.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.524+0000 7f69e77fe700 1 --2- 172.21.15.71:0/850090877 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f69d006cce0 0x7f69d006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:58.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.525+0000 7f69e77fe700 1 --2- 172.21.15.71:0/850090877 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f69d006cce0 0x7f69d006f1a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f69dc006fd0 tx=0x7f69dc008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:58.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.528+0000 7f69e4ff9700 1 -- 172.21.15.71:0/850090877 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f69d805b8f0 con 0x7f69e8101840 2024-04-23T02:52:58.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:58 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:52:58.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:58 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:52:58.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:58 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:52:58.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.738+0000 7f69ee13b700 1 -- 172.21.15.71:0/850090877 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f69e8197080 con 0x7f69d006cce0 2024-04-23T02:52:58.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:58 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:52:58.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:58 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:52:58.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:58 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:52:58.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.739+0000 7f69e4ff9700 1 -- 172.21.15.71:0/850090877 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f69e8197080 con 0x7f69d006cce0 2024-04-23T02:52:58.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.741+0000 7f69ee13b700 1 -- 172.21.15.71:0/850090877 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f69d006cce0 msgr2=0x7f69d006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:58.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.741+0000 7f69ee13b700 1 --2- 172.21.15.71:0/850090877 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f69d006cce0 0x7f69d006f1a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f69dc006fd0 tx=0x7f69dc008040 comp rx=0 tx=0).stop 2024-04-23T02:52:58.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.741+0000 7f69ee13b700 1 -- 172.21.15.71:0/850090877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69e8101840 msgr2=0x7f69e819cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:58.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.741+0000 7f69ee13b700 1 --2- 172.21.15.71:0/850090877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69e8101840 0x7f69e819cbb0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f69d80092e0 tx=0x7f69d800be20 comp rx=0 tx=0).stop 2024-04-23T02:52:58.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.741+0000 7f69ee13b700 1 -- 172.21.15.71:0/850090877 shutdown_connections 2024-04-23T02:52:58.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.741+0000 7f69ee13b700 1 --2- 172.21.15.71:0/850090877 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f69d006cce0 0x7f69d006f1a0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:58.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.741+0000 7f69ee13b700 1 --2- 172.21.15.71:0/850090877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69e8100e90 0x7f69e819c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:58.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.741+0000 7f69ee13b700 1 --2- 172.21.15.71:0/850090877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69e8101840 0x7f69e819cbb0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:58.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.741+0000 7f69ee13b700 1 -- 172.21.15.71:0/850090877 >> 172.21.15.71:0/850090877 conn(0x7f69e80fc740 msgr2=0x7f69e80fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:58.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.741+0000 7f69ee13b700 1 -- 172.21.15.71:0/850090877 shutdown_connections 2024-04-23T02:52:58.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.741+0000 7f69ee13b700 1 -- 172.21.15.71:0/850090877 wait complete. 2024-04-23T02:52:58.755 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:52:58.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.845+0000 7fd38faf7700 1 -- 172.21.15.71:0/725367426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd388100750 msgr2=0x7fd3881047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:58.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.845+0000 7fd38faf7700 1 --2- 172.21.15.71:0/725367426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd388100750 0x7fd3881047a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd384009910 tx=0x7fd384009c20 comp rx=0 tx=0).stop 2024-04-23T02:52:58.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.845+0000 7fd38faf7700 1 -- 172.21.15.71:0/725367426 shutdown_connections 2024-04-23T02:52:58.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.845+0000 7fd38faf7700 1 --2- 172.21.15.71:0/725367426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd388100750 0x7fd3881047a0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:58.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.845+0000 7fd38faf7700 1 --2- 172.21.15.71:0/725367426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3880ffda0 0x7fd388100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:58.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.845+0000 7fd38faf7700 1 -- 172.21.15.71:0/725367426 >> 172.21.15.71:0/725367426 conn(0x7fd3880fb650 msgr2=0x7fd3880fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:58.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.845+0000 7fd38faf7700 1 -- 172.21.15.71:0/725367426 shutdown_connections 2024-04-23T02:52:58.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.845+0000 7fd38faf7700 1 -- 172.21.15.71:0/725367426 wait complete. 2024-04-23T02:52:58.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.845+0000 7fd38faf7700 1 Processor -- start 2024-04-23T02:52:58.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.846+0000 7fd38faf7700 1 -- start start 2024-04-23T02:52:58.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.846+0000 7fd38faf7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3880ffda0 0x7fd38819b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:58.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.846+0000 7fd38faf7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd388100750 0x7fd38819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:58.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.846+0000 7fd38faf7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38819c0f0 con 0x7fd3880ffda0 2024-04-23T02:52:58.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.846+0000 7fd38faf7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3881955a0 con 0x7fd388100750 2024-04-23T02:52:58.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.846+0000 7fd38d893700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3880ffda0 0x7fd38819b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:58.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.846+0000 7fd38d893700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3880ffda0 0x7fd38819b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57436/0 (socket says 172.21.15.71:57436) 2024-04-23T02:52:58.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.846+0000 7fd38d893700 1 -- 172.21.15.71:0/2407976171 learned_addr learned my addr 172.21.15.71:0/2407976171 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:52:58.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.846+0000 7fd38d893700 1 -- 172.21.15.71:0/2407976171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd388100750 msgr2=0x7fd38819ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:58.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.846+0000 7fd38d092700 1 --2- 172.21.15.71:0/2407976171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd388100750 0x7fd38819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:58.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.846+0000 7fd38d893700 1 --2- 172.21.15.71:0/2407976171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd388100750 0x7fd38819ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:58.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.846+0000 7fd38d893700 1 -- 172.21.15.71:0/2407976171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3840095f0 con 0x7fd3880ffda0 2024-04-23T02:52:58.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.846+0000 7fd38d893700 1 --2- 172.21.15.71:0/2407976171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3880ffda0 0x7fd38819b520 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd378007910 tx=0x7fd378007cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:58.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.846+0000 7fd37effd700 1 -- 172.21.15.71:0/2407976171 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd378012070 con 0x7fd3880ffda0 2024-04-23T02:52:58.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.846+0000 7fd37effd700 1 -- 172.21.15.71:0/2407976171 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd37800d810 con 0x7fd3880ffda0 2024-04-23T02:52:58.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.846+0000 7fd38faf7700 1 -- 172.21.15.71:0/2407976171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd388195820 con 0x7fd3880ffda0 2024-04-23T02:52:58.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.846+0000 7fd37effd700 1 -- 172.21.15.71:0/2407976171 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3780053d0 con 0x7fd3880ffda0 2024-04-23T02:52:58.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.847+0000 7fd38faf7700 1 -- 172.21.15.71:0/2407976171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd388195d70 con 0x7fd3880ffda0 2024-04-23T02:52:58.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.847+0000 7fd38faf7700 1 -- 172.21.15.71:0/2407976171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38804f2e0 con 0x7fd3880ffda0 2024-04-23T02:52:58.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.848+0000 7fd37effd700 1 -- 172.21.15.71:0/2407976171 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd378016070 con 0x7fd3880ffda0 2024-04-23T02:52:58.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.848+0000 7fd37effd700 1 --2- 172.21.15.71:0/2407976171 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd37406cce0 0x7fd37406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:58.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.848+0000 7fd38d092700 1 --2- 172.21.15.71:0/2407976171 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd37406cce0 0x7fd37406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:58.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.848+0000 7fd37effd700 1 -- 172.21.15.71:0/2407976171 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd37808d760 con 0x7fd3880ffda0 2024-04-23T02:52:58.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.849+0000 7fd38d092700 1 --2- 172.21.15.71:0/2407976171 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd37406cce0 0x7fd37406f1a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd38400b5c0 tx=0x7fd384005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:58.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:58.853+0000 7fd37effd700 1 -- 172.21.15.71:0/2407976171 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd37805b2e0 con 0x7fd3880ffda0 2024-04-23T02:52:59.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.065+0000 7fd38faf7700 1 -- 172.21.15.71:0/2407976171 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd388196060 con 0x7fd37406cce0 2024-04-23T02:52:59.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.067+0000 7fd37effd700 1 -- 172.21.15.71:0/2407976171 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd388196060 con 0x7fd37406cce0 2024-04-23T02:52:59.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.069+0000 7fd38faf7700 1 -- 172.21.15.71:0/2407976171 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd37406cce0 msgr2=0x7fd37406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:59.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.069+0000 7fd38faf7700 1 --2- 172.21.15.71:0/2407976171 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd37406cce0 0x7fd37406f1a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd38400b5c0 tx=0x7fd384005fb0 comp rx=0 tx=0).stop 2024-04-23T02:52:59.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.069+0000 7fd38faf7700 1 -- 172.21.15.71:0/2407976171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3880ffda0 msgr2=0x7fd38819b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:59.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.069+0000 7fd38faf7700 1 --2- 172.21.15.71:0/2407976171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3880ffda0 0x7fd38819b520 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd378007910 tx=0x7fd378007cd0 comp rx=0 tx=0).stop 2024-04-23T02:52:59.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.069+0000 7fd38faf7700 1 -- 172.21.15.71:0/2407976171 shutdown_connections 2024-04-23T02:52:59.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.069+0000 7fd38faf7700 1 --2- 172.21.15.71:0/2407976171 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd37406cce0 0x7fd37406f1a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:59.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.069+0000 7fd38faf7700 1 --2- 172.21.15.71:0/2407976171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3880ffda0 0x7fd38819b520 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:59.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.069+0000 7fd38faf7700 1 --2- 172.21.15.71:0/2407976171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd388100750 0x7fd38819ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:59.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.069+0000 7fd38faf7700 1 -- 172.21.15.71:0/2407976171 >> 172.21.15.71:0/2407976171 conn(0x7fd3880fb650 msgr2=0x7fd3880fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:59.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.069+0000 7fd38faf7700 1 -- 172.21.15.71:0/2407976171 shutdown_connections 2024-04-23T02:52:59.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.069+0000 7fd38faf7700 1 -- 172.21.15.71:0/2407976171 wait complete. 2024-04-23T02:52:59.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.171+0000 7f6f099f8700 1 -- 172.21.15.71:0/3741214690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f04101850 msgr2=0x7f6f041058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:59.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.171+0000 7f6f099f8700 1 --2- 172.21.15.71:0/3741214690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f04101850 0x7f6f041058a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009910 tx=0x7f6ef4009c20 comp rx=0 tx=0).stop 2024-04-23T02:52:59.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.171+0000 7f6f099f8700 1 -- 172.21.15.71:0/3741214690 shutdown_connections 2024-04-23T02:52:59.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.171+0000 7f6f099f8700 1 --2- 172.21.15.71:0/3741214690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f04101850 0x7f6f041058a0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:59.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.171+0000 7f6f099f8700 1 --2- 172.21.15.71:0/3741214690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f04100ea0 0x7f6f04101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:59.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.171+0000 7f6f099f8700 1 -- 172.21.15.71:0/3741214690 >> 172.21.15.71:0/3741214690 conn(0x7f6f040fc750 msgr2=0x7f6f040feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:59.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.171+0000 7f6f099f8700 1 -- 172.21.15.71:0/3741214690 shutdown_connections 2024-04-23T02:52:59.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.171+0000 7f6f099f8700 1 -- 172.21.15.71:0/3741214690 wait complete. 2024-04-23T02:52:59.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.172+0000 7f6f099f8700 1 Processor -- start 2024-04-23T02:52:59.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.172+0000 7f6f099f8700 1 -- start start 2024-04-23T02:52:59.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.172+0000 7f6f099f8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f04100ea0 0x7f6f04073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:59.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.172+0000 7f6f099f8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f04101850 0x7f6f040735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:59.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.172+0000 7f6f099f8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f04073b10 con 0x7f6f04100ea0 2024-04-23T02:52:59.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.172+0000 7f6f099f8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f04073c50 con 0x7f6f04101850 2024-04-23T02:52:59.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.172+0000 7f6f02ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f04100ea0 0x7f6f04073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:59.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.172+0000 7f6f02ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f04100ea0 0x7f6f04073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57448/0 (socket says 172.21.15.71:57448) 2024-04-23T02:52:59.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.172+0000 7f6f02ffd700 1 -- 172.21.15.71:0/4233603794 learned_addr learned my addr 172.21.15.71:0/4233603794 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:52:59.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.172+0000 7f6f027fc700 1 --2- 172.21.15.71:0/4233603794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f04101850 0x7f6f040735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:59.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.173+0000 7f6f02ffd700 1 -- 172.21.15.71:0/4233603794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f04101850 msgr2=0x7f6f040735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:59.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.173+0000 7f6f02ffd700 1 --2- 172.21.15.71:0/4233603794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f04101850 0x7f6f040735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:59.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.173+0000 7f6f02ffd700 1 -- 172.21.15.71:0/4233603794 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef40095f0 con 0x7f6f04100ea0 2024-04-23T02:52:59.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.173+0000 7f6f02ffd700 1 --2- 172.21.15.71:0/4233603794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f04100ea0 0x7f6f04073090 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800c930 tx=0x7f6ef800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:59.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.173+0000 7f6f089f6700 1 -- 172.21.15.71:0/4233603794 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ef8007ad0 con 0x7f6f04100ea0 2024-04-23T02:52:59.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.173+0000 7f6f089f6700 1 -- 172.21.15.71:0/4233603794 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6ef800ce80 con 0x7f6f04100ea0 2024-04-23T02:52:59.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.173+0000 7f6f089f6700 1 -- 172.21.15.71:0/4233603794 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ef8018630 con 0x7f6f04100ea0 2024-04-23T02:52:59.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.173+0000 7f6f099f8700 1 -- 172.21.15.71:0/4233603794 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f041a2a40 con 0x7f6f04100ea0 2024-04-23T02:52:59.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.173+0000 7f6f099f8700 1 -- 172.21.15.71:0/4233603794 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f041a2da0 con 0x7f6f04100ea0 2024-04-23T02:52:59.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.174+0000 7f6f099f8700 1 -- 172.21.15.71:0/4233603794 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f041092a0 con 0x7f6f04100ea0 2024-04-23T02:52:59.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.174+0000 7f6f089f6700 1 -- 172.21.15.71:0/4233603794 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6ef800f450 con 0x7f6f04100ea0 2024-04-23T02:52:59.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.175+0000 7f6f089f6700 1 --2- 172.21.15.71:0/4233603794 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6eec06cce0 0x7f6eec06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:59.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.175+0000 7f6f089f6700 1 -- 172.21.15.71:0/4233603794 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6ef808c500 con 0x7f6f04100ea0 2024-04-23T02:52:59.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.175+0000 7f6f027fc700 1 --2- 172.21.15.71:0/4233603794 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6eec06cce0 0x7f6eec06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:59.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.175+0000 7f6f027fc700 1 --2- 172.21.15.71:0/4233603794 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6eec06cce0 0x7f6eec06f1a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6f04074880 tx=0x7f6ef401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:59.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.179+0000 7f6f089f6700 1 -- 172.21.15.71:0/4233603794 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6ef80565f0 con 0x7f6f04100ea0 2024-04-23T02:52:59.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.381+0000 7f6f099f8700 1 -- 172.21.15.71:0/4233603794 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f040611d0 con 0x7f6eec06cce0 2024-04-23T02:52:59.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.392+0000 7f6f089f6700 1 -- 172.21.15.71:0/4233603794 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6f040611d0 con 0x7f6eec06cce0 2024-04-23T02:52:59.396 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:52:59.396 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (36m) 14m ago 47m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:52:59.396 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (48m) 14m ago 48m 9671k - 18.2.1 5be31c24972a 42182f8cf9d6 2024-04-23T02:52:59.396 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (47m) 14m ago 47m 9605k - 18.2.1 5be31c24972a a2af51d36f55 2024-04-23T02:52:59.396 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (34m) 14m ago 48m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:52:59.396 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (34m) 14m ago 47m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:52:59.396 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (36m) 14m ago 47m 88.1M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:52:59.396 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (30m) 14m ago 42m 20.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:52:59.396 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (30m) 14m ago 41m 20.2M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:52:59.396 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (15m) 14m ago 41m 12.4M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:52:59.396 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (30m) 14m ago 42m 21.7M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:52:59.396 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (38m) 14m ago 49m 514M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:52:59.396 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (37m) 14m ago 46m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:52:59.397 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (35m) 14m ago 49m 70.0M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:52:59.397 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (34m) 14m ago 46m 52.5M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:52:59.397 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (37m) 14m ago 48m 21.7M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:52:59.397 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (37m) 14m ago 46m 18.3M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:52:59.397 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (34m) 14m ago 46m 93.5M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:52:59.397 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (33m) 14m ago 45m 92.8M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:52:59.397 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (32m) 14m ago 45m 85.0M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:52:59.397 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (32m) 14m ago 44m 107M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:52:59.397 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (31m) 14m ago 44m 94.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:52:59.397 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (31m) 14m ago 43m 84.7M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:52:59.397 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (36m) 14m ago 47m 84.1M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:52:59.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.394+0000 7f6f099f8700 1 -- 172.21.15.71:0/4233603794 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6eec06cce0 msgr2=0x7f6eec06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:59.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.394+0000 7f6f099f8700 1 --2- 172.21.15.71:0/4233603794 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6eec06cce0 0x7f6eec06f1a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6f04074880 tx=0x7f6ef401a040 comp rx=0 tx=0).stop 2024-04-23T02:52:59.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.394+0000 7f6f099f8700 1 -- 172.21.15.71:0/4233603794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f04100ea0 msgr2=0x7f6f04073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:59.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.394+0000 7f6f099f8700 1 --2- 172.21.15.71:0/4233603794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f04100ea0 0x7f6f04073090 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800c930 tx=0x7f6ef800cc40 comp rx=0 tx=0).stop 2024-04-23T02:52:59.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.394+0000 7f6f099f8700 1 -- 172.21.15.71:0/4233603794 shutdown_connections 2024-04-23T02:52:59.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.394+0000 7f6f099f8700 1 --2- 172.21.15.71:0/4233603794 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6eec06cce0 0x7f6eec06f1a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:59.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.394+0000 7f6f099f8700 1 --2- 172.21.15.71:0/4233603794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f04100ea0 0x7f6f04073090 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:59.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.394+0000 7f6f099f8700 1 --2- 172.21.15.71:0/4233603794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f04101850 0x7f6f040735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:59.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.394+0000 7f6f099f8700 1 -- 172.21.15.71:0/4233603794 >> 172.21.15.71:0/4233603794 conn(0x7f6f040fc750 msgr2=0x7f6f040fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:59.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.394+0000 7f6f099f8700 1 -- 172.21.15.71:0/4233603794 shutdown_connections 2024-04-23T02:52:59.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.394+0000 7f6f099f8700 1 -- 172.21.15.71:0/4233603794 wait complete. 2024-04-23T02:52:59.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.525+0000 7ff2e53f7700 1 -- 172.21.15.71:0/784975797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e00ff340 msgr2=0x7ff2e00ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:59.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.525+0000 7ff2e53f7700 1 --2- 172.21.15.71:0/784975797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e00ff340 0x7ff2e00ff7a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009910 tx=0x7ff2d4009c20 comp rx=0 tx=0).stop 2024-04-23T02:52:59.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.525+0000 7ff2e53f7700 1 -- 172.21.15.71:0/784975797 shutdown_connections 2024-04-23T02:52:59.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.525+0000 7ff2e53f7700 1 --2- 172.21.15.71:0/784975797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e00ff340 0x7ff2e00ff7a0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:59.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.525+0000 7ff2e53f7700 1 --2- 172.21.15.71:0/784975797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2e0105350 0x7ff2e0105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:59.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.526+0000 7ff2e53f7700 1 -- 172.21.15.71:0/784975797 >> 172.21.15.71:0/784975797 conn(0x7ff2e0074e30 msgr2=0x7ff2e0075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:59.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.526+0000 7ff2e53f7700 1 -- 172.21.15.71:0/784975797 shutdown_connections 2024-04-23T02:52:59.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.526+0000 7ff2e53f7700 1 -- 172.21.15.71:0/784975797 wait complete. 2024-04-23T02:52:59.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.526+0000 7ff2e53f7700 1 Processor -- start 2024-04-23T02:52:59.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.527+0000 7ff2e53f7700 1 -- start start 2024-04-23T02:52:59.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.527+0000 7ff2e53f7700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2e00ff340 0x7ff2e0194590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:59.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.527+0000 7ff2e53f7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e0105350 0x7ff2e0194ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:59.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.527+0000 7ff2e53f7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e01951b0 con 0x7ff2e0105350 2024-04-23T02:52:59.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.527+0000 7ff2e53f7700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e0198f40 con 0x7ff2e00ff340 2024-04-23T02:52:59.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.527+0000 7ff2deffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2e00ff340 0x7ff2e0194590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:59.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.527+0000 7ff2deffd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2e00ff340 0x7ff2e0194590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.71:47374/0 (socket says 172.21.15.71:47374) 2024-04-23T02:52:59.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.527+0000 7ff2deffd700 1 -- 172.21.15.71:0/969448800 learned_addr learned my addr 172.21.15.71:0/969448800 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:52:59.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.527+0000 7ff2de7fc700 1 --2- 172.21.15.71:0/969448800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e0105350 0x7ff2e0194ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:59.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.528+0000 7ff2deffd700 1 -- 172.21.15.71:0/969448800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e0105350 msgr2=0x7ff2e0194ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:59.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.528+0000 7ff2deffd700 1 --2- 172.21.15.71:0/969448800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e0105350 0x7ff2e0194ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:59.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.528+0000 7ff2deffd700 1 -- 172.21.15.71:0/969448800 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d40095f0 con 0x7ff2e00ff340 2024-04-23T02:52:59.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.528+0000 7ff2deffd700 1 --2- 172.21.15.71:0/969448800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2e00ff340 0x7ff2e0194590 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000dd40 tx=0x7ff2d000a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:59.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.528+0000 7ff2c7fff700 1 -- 172.21.15.71:0/969448800 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2d000ad10 con 0x7ff2e00ff340 2024-04-23T02:52:59.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.528+0000 7ff2c7fff700 1 -- 172.21.15.71:0/969448800 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff2d000ae70 con 0x7ff2e00ff340 2024-04-23T02:52:59.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.528+0000 7ff2c7fff700 1 -- 172.21.15.71:0/969448800 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2d0016830 con 0x7ff2e00ff340 2024-04-23T02:52:59.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.528+0000 7ff2e53f7700 1 -- 172.21.15.71:0/969448800 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e0199220 con 0x7ff2e00ff340 2024-04-23T02:52:59.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.528+0000 7ff2e53f7700 1 -- 172.21.15.71:0/969448800 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e01996f0 con 0x7ff2e00ff340 2024-04-23T02:52:59.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.530+0000 7ff2e53f7700 1 -- 172.21.15.71:0/969448800 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e0107850 con 0x7ff2e00ff340 2024-04-23T02:52:59.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.530+0000 7ff2c7fff700 1 -- 172.21.15.71:0/969448800 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff2d000e450 con 0x7ff2e00ff340 2024-04-23T02:52:59.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.530+0000 7ff2c7fff700 1 --2- 172.21.15.71:0/969448800 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff2c8071100 0x7ff2c80735c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:52:59.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.530+0000 7ff2c7fff700 1 -- 172.21.15.71:0/969448800 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff2d008c4c0 con 0x7ff2e00ff340 2024-04-23T02:52:59.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.531+0000 7ff2de7fc700 1 --2- 172.21.15.71:0/969448800 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff2c8071100 0x7ff2c80735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:52:59.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.531+0000 7ff2de7fc700 1 --2- 172.21.15.71:0/969448800 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff2c8071100 0x7ff2c80735c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400b5c0 tx=0x7ff2d401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:52:59.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.535+0000 7ff2c7fff700 1 -- 172.21.15.71:0/969448800 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff2d00565e0 con 0x7ff2e00ff340 2024-04-23T02:52:59.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: pgmap v1191: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:59.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: pgmap v1192: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:59.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:52:59.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:52:59.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:52:59.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:52:59.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:52:59.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:52:59.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: Upgrade: Setting container_image for all mds 2024-04-23T02:52:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:52:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi071.qusrrb"}]: dispatch 2024-04-23T02:52:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi071.qusrrb"}]': finished 2024-04-23T02:52:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi071.vxejye"}]: dispatch 2024-04-23T02:52:59.627 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi071.vxejye"}]': finished 2024-04-23T02:52:59.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.azdyze"}]: dispatch 2024-04-23T02:52:59.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.azdyze"}]': finished 2024-04-23T02:52:59.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.gtmyst"}]: dispatch 2024-04-23T02:52:59.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.gtmyst"}]': finished 2024-04-23T02:52:59.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: Upgrade: Setting filesystem cephfs Joinable 2024-04-23T02:52:59.628 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:52:59 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-23T02:52:59.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: pgmap v1191: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:59.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: pgmap v1192: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:52:59.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:52:59.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:52:59.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:52:59.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:52:59.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:52:59.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:52:59.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: Upgrade: Setting container_image for all mds 2024-04-23T02:52:59.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:52:59.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi071.qusrrb"}]: dispatch 2024-04-23T02:52:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi071.qusrrb"}]': finished 2024-04-23T02:52:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi071.vxejye"}]: dispatch 2024-04-23T02:52:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi071.vxejye"}]': finished 2024-04-23T02:52:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.azdyze"}]: dispatch 2024-04-23T02:52:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.azdyze"}]': finished 2024-04-23T02:52:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.gtmyst"}]: dispatch 2024-04-23T02:52:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi192.gtmyst"}]': finished 2024-04-23T02:52:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: Upgrade: Setting filesystem cephfs Joinable 2024-04-23T02:52:59.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:52:59 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-23T02:52:59.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.978+0000 7ff2e53f7700 1 -- 172.21.15.71:0/969448800 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff2e01999c0 con 0x7ff2e00ff340 2024-04-23T02:52:59.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.978+0000 7ff2c7fff700 1 -- 172.21.15.71:0/969448800 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7ff2d0059c00 con 0x7ff2e00ff340 2024-04-23T02:52:59.982 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:52:59.982 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:52:59.983 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:52:59.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:52:59.983 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:52:59.983 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:52:59.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:52:59.983 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:52:59.983 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:52:59.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:52:59.983 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:52:59.983 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:52:59.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:52:59.983 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:52:59.983 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:52:59.983 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:52:59.983 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:52:59.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.983+0000 7ff2e53f7700 1 -- 172.21.15.71:0/969448800 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff2c8071100 msgr2=0x7ff2c80735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:59.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.983+0000 7ff2e53f7700 1 --2- 172.21.15.71:0/969448800 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff2c8071100 0x7ff2c80735c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400b5c0 tx=0x7ff2d401a040 comp rx=0 tx=0).stop 2024-04-23T02:52:59.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.983+0000 7ff2e53f7700 1 -- 172.21.15.71:0/969448800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2e00ff340 msgr2=0x7ff2e0194590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:52:59.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.983+0000 7ff2e53f7700 1 --2- 172.21.15.71:0/969448800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2e00ff340 0x7ff2e0194590 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000dd40 tx=0x7ff2d000a530 comp rx=0 tx=0).stop 2024-04-23T02:52:59.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.984+0000 7ff2e53f7700 1 -- 172.21.15.71:0/969448800 shutdown_connections 2024-04-23T02:52:59.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.984+0000 7ff2e53f7700 1 --2- 172.21.15.71:0/969448800 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff2c8071100 0x7ff2c80735c0 secure :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400b5c0 tx=0x7ff2d401a040 comp rx=0 tx=0).stop 2024-04-23T02:52:59.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.984+0000 7ff2e53f7700 1 --2- 172.21.15.71:0/969448800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2e00ff340 0x7ff2e0194590 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:59.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.984+0000 7ff2e53f7700 1 --2- 172.21.15.71:0/969448800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e0105350 0x7ff2e0194ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:52:59.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.984+0000 7ff2e53f7700 1 -- 172.21.15.71:0/969448800 >> 172.21.15.71:0/969448800 conn(0x7ff2e0074e30 msgr2=0x7ff2e00fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:52:59.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.984+0000 7ff2e53f7700 1 -- 172.21.15.71:0/969448800 shutdown_connections 2024-04-23T02:52:59.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:52:59.984+0000 7ff2e53f7700 1 -- 172.21.15.71:0/969448800 wait complete. 2024-04-23T02:53:00.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.106+0000 7f9bd875c700 1 -- 172.21.15.71:0/2265771633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bd0106cd0 msgr2=0x7f9bd01070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:00.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.106+0000 7f9bd875c700 1 --2- 172.21.15.71:0/2265771633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bd0106cd0 0x7f9bd01070b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc009910 tx=0x7f9bcc009c20 comp rx=0 tx=0).stop 2024-04-23T02:53:00.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.107+0000 7f9bd875c700 1 -- 172.21.15.71:0/2265771633 shutdown_connections 2024-04-23T02:53:00.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.107+0000 7f9bd875c700 1 --2- 172.21.15.71:0/2265771633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bd0068580 0x7f9bd00689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:00.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.107+0000 7f9bd875c700 1 --2- 172.21.15.71:0/2265771633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bd0106cd0 0x7f9bd01070b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:00.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.107+0000 7f9bd875c700 1 -- 172.21.15.71:0/2265771633 >> 172.21.15.71:0/2265771633 conn(0x7f9bd0075670 msgr2=0x7f9bd0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:53:00.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.107+0000 7f9bd875c700 1 -- 172.21.15.71:0/2265771633 shutdown_connections 2024-04-23T02:53:00.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.107+0000 7f9bd875c700 1 -- 172.21.15.71:0/2265771633 wait complete. 2024-04-23T02:53:00.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.107+0000 7f9bd875c700 1 Processor -- start 2024-04-23T02:53:00.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.108+0000 7f9bd875c700 1 -- start start 2024-04-23T02:53:00.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.108+0000 7f9bd875c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bd0068580 0x7f9bd0199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:00.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.108+0000 7f9bd875c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bd0106cd0 0x7f9bd0199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:00.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.108+0000 7f9bd875c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd019f090 con 0x7f9bd0068580 2024-04-23T02:53:00.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.108+0000 7f9bd875c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd019f200 con 0x7f9bd0106cd0 2024-04-23T02:53:00.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.108+0000 7f9bd64f8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bd0068580 0x7f9bd0199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:00.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.108+0000 7f9bd64f8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bd0068580 0x7f9bd0199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57476/0 (socket says 172.21.15.71:57476) 2024-04-23T02:53:00.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.108+0000 7f9bd64f8700 1 -- 172.21.15.71:0/3235217657 learned_addr learned my addr 172.21.15.71:0/3235217657 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:53:00.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.108+0000 7f9bd5cf7700 1 --2- 172.21.15.71:0/3235217657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bd0106cd0 0x7f9bd0199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:00.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.108+0000 7f9bd64f8700 1 -- 172.21.15.71:0/3235217657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bd0106cd0 msgr2=0x7f9bd0199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:00.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.108+0000 7f9bd64f8700 1 --2- 172.21.15.71:0/3235217657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bd0106cd0 0x7f9bd0199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:00.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.108+0000 7f9bd64f8700 1 -- 172.21.15.71:0/3235217657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc0009520 con 0x7f9bd0068580 2024-04-23T02:53:00.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.109+0000 7f9bd64f8700 1 --2- 172.21.15.71:0/3235217657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bd0068580 0x7f9bd0199300 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00bde0 tx=0x7f9bcc00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:53:00.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.109+0000 7f9bc77fe700 1 -- 172.21.15.71:0/3235217657 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bcc01d070 con 0x7f9bd0068580 2024-04-23T02:53:00.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.109+0000 7f9bc77fe700 1 -- 172.21.15.71:0/3235217657 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9bcc0044c0 con 0x7f9bd0068580 2024-04-23T02:53:00.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.109+0000 7f9bc77fe700 1 -- 172.21.15.71:0/3235217657 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bcc0039d0 con 0x7f9bd0068580 2024-04-23T02:53:00.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.109+0000 7f9bd875c700 1 -- 172.21.15.71:0/3235217657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bcc0095f0 con 0x7f9bd0068580 2024-04-23T02:53:00.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.109+0000 7f9bd875c700 1 -- 172.21.15.71:0/3235217657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd019f700 con 0x7f9bd0068580 2024-04-23T02:53:00.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.111+0000 7f9bd875c700 1 -- 172.21.15.71:0/3235217657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd004f2e0 con 0x7f9bd0068580 2024-04-23T02:53:00.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.112+0000 7f9bc77fe700 1 -- 172.21.15.71:0/3235217657 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9bcc004ba0 con 0x7f9bd0068580 2024-04-23T02:53:00.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.113+0000 7f9bc77fe700 1 --2- 172.21.15.71:0/3235217657 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9bbc06cd30 0x7f9bbc06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:00.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.113+0000 7f9bc77fe700 1 -- 172.21.15.71:0/3235217657 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9bcc08e280 con 0x7f9bd0068580 2024-04-23T02:53:00.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.115+0000 7f9bd5cf7700 1 --2- 172.21.15.71:0/3235217657 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9bbc06cd30 0x7f9bbc06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:00.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.115+0000 7f9bd5cf7700 1 --2- 172.21.15.71:0/3235217657 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9bbc06cd30 0x7f9bbc06f1f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0195ba0 tx=0x7f9bc0009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:53:00.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.117+0000 7f9bc77fe700 1 -- 172.21.15.71:0/3235217657 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9bcc058410 con 0x7f9bd0068580 2024-04-23T02:53:00.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.534+0000 7f9bd875c700 1 -- 172.21.15.71:0/3235217657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9bd019f9e0 con 0x7f9bd0068580 2024-04-23T02:53:00.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.535+0000 7f9bc77fe700 1 -- 172.21.15.71:0/3235217657 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 39 v39) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f9bcc05ba30 con 0x7f9bd0068580 2024-04-23T02:53:00.538 INFO:teuthology.orchestra.run.smithi071.stdout:e39 2024-04-23T02:53:00.538 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:53:00.538 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:53:00.538 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:53:00.539 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:53:00.539 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:53:00.539 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:53:00.539 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 39 2024-04-23T02:53:00.539 INFO:teuthology.orchestra.run.smithi071.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T02:53:00.539 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:53:00.539 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:53:00.460445+0000 2024-04-23T02:53:00.539 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:53:00.539 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:53:00.539 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:53:00.539 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:53:00.539 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:53:00.540 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:53:00.540 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:53:00.540 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:53:00.540 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:53:00.540 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:53:00.540 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:53:00.540 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:53:00.540 INFO:teuthology.orchestra.run.smithi071.stdout:up {0=34334,1=34362} 2024-04-23T02:53:00.540 INFO:teuthology.orchestra.run.smithi071.stdout:failed 2024-04-23T02:53:00.540 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:53:00.540 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:53:00.540 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:53:00.540 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:53:00.540 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:53:00.541 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:53:00.541 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:53:00.541 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:53:00.541 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{0:34334} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:53:00.541 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{1:34362} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:53:00.541 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:53:00.541 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:53:00.541 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-04-23T02:53:00.541 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:53:00.541 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:53:00.541 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{-1:44793} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:53:00.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.538+0000 7f9bd875c700 1 -- 172.21.15.71:0/3235217657 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9bbc06cd30 msgr2=0x7f9bbc06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:00.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.538+0000 7f9bd875c700 1 --2- 172.21.15.71:0/3235217657 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9bbc06cd30 0x7f9bbc06f1f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0195ba0 tx=0x7f9bc0009260 comp rx=0 tx=0).stop 2024-04-23T02:53:00.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.538+0000 7f9bd875c700 1 -- 172.21.15.71:0/3235217657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bd0068580 msgr2=0x7f9bd0199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:00.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.538+0000 7f9bd875c700 1 --2- 172.21.15.71:0/3235217657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bd0068580 0x7f9bd0199300 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00bde0 tx=0x7f9bcc00be10 comp rx=0 tx=0).stop 2024-04-23T02:53:00.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.538+0000 7f9bd875c700 1 -- 172.21.15.71:0/3235217657 shutdown_connections 2024-04-23T02:53:00.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.538+0000 7f9bd875c700 1 --2- 172.21.15.71:0/3235217657 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9bbc06cd30 0x7f9bbc06f1f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:00.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.538+0000 7f9bd875c700 1 --2- 172.21.15.71:0/3235217657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bd0068580 0x7f9bd0199300 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:00.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.538+0000 7f9bd875c700 1 --2- 172.21.15.71:0/3235217657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bd0106cd0 0x7f9bd0199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:00.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.538+0000 7f9bd875c700 1 -- 172.21.15.71:0/3235217657 >> 172.21.15.71:0/3235217657 conn(0x7f9bd0075670 msgr2=0x7f9bd00fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:53:00.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.538+0000 7f9bd875c700 1 -- 172.21.15.71:0/3235217657 shutdown_connections 2024-04-23T02:53:00.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.538+0000 7f9bd875c700 1 -- 172.21.15.71:0/3235217657 wait complete. 2024-04-23T02:53:00.554 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 39 2024-04-23T02:53:00.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.667+0000 7f1ac3f85700 1 -- 172.21.15.71:0/1147012712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc0ffb40 msgr2=0x7f1abc0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:00.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.667+0000 7f1ac3f85700 1 --2- 172.21.15.71:0/1147012712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc0ffb40 0x7f1abc0fffc0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009910 tx=0x7f1ab8009c20 comp rx=0 tx=0).stop 2024-04-23T02:53:00.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.669+0000 7f1ac3f85700 1 -- 172.21.15.71:0/1147012712 shutdown_connections 2024-04-23T02:53:00.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.669+0000 7f1ac3f85700 1 --2- 172.21.15.71:0/1147012712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc0ffb40 0x7f1abc0fffc0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:00.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.669+0000 7f1ac3f85700 1 --2- 172.21.15.71:0/1147012712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1abc105bf0 0x7f1abc105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:00.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.669+0000 7f1ac3f85700 1 -- 172.21.15.71:0/1147012712 >> 172.21.15.71:0/1147012712 conn(0x7f1abc0fb610 msgr2=0x7f1abc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:53:00.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.669+0000 7f1ac3f85700 1 -- 172.21.15.71:0/1147012712 shutdown_connections 2024-04-23T02:53:00.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.669+0000 7f1ac3f85700 1 -- 172.21.15.71:0/1147012712 wait complete. 2024-04-23T02:53:00.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.670+0000 7f1ac3f85700 1 Processor -- start 2024-04-23T02:53:00.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.670+0000 7f1ac3f85700 1 -- start start 2024-04-23T02:53:00.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.671+0000 7f1ac3f85700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1abc0ffb40 0x7f1abc19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:00.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.671+0000 7f1ac3f85700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc105bf0 0x7f1abc19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:00.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.671+0000 7f1ac3f85700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc19d1a0 con 0x7f1abc105bf0 2024-04-23T02:53:00.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.671+0000 7f1ac3f85700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc196600 con 0x7f1abc0ffb40 2024-04-23T02:53:00.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.671+0000 7f1ac1520700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc105bf0 0x7f1abc19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:00.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.671+0000 7f1ac1520700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc105bf0 0x7f1abc19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57484/0 (socket says 172.21.15.71:57484) 2024-04-23T02:53:00.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.671+0000 7f1ac1520700 1 -- 172.21.15.71:0/3510548911 learned_addr learned my addr 172.21.15.71:0/3510548911 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:53:00.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.671+0000 7f1ac1d21700 1 --2- 172.21.15.71:0/3510548911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1abc0ffb40 0x7f1abc19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:00.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.671+0000 7f1ac1520700 1 -- 172.21.15.71:0/3510548911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1abc0ffb40 msgr2=0x7f1abc19c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:00.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.671+0000 7f1ac1520700 1 --2- 172.21.15.71:0/3510548911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1abc0ffb40 0x7f1abc19c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:00.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.671+0000 7f1ac1520700 1 -- 172.21.15.71:0/3510548911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab80095f0 con 0x7f1abc105bf0 2024-04-23T02:53:00.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.672+0000 7f1ac1520700 1 --2- 172.21.15.71:0/3510548911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc105bf0 0x7f1abc19cb50 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8006010 tx=0x7f1ab800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:53:00.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.672+0000 7f1ab2ffd700 1 -- 172.21.15.71:0/3510548911 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ab801d070 con 0x7f1abc105bf0 2024-04-23T02:53:00.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.672+0000 7f1ac3f85700 1 -- 172.21.15.71:0/3510548911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1abc196880 con 0x7f1abc105bf0 2024-04-23T02:53:00.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.672+0000 7f1ab2ffd700 1 -- 172.21.15.71:0/3510548911 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1ab8005940 con 0x7f1abc105bf0 2024-04-23T02:53:00.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.672+0000 7f1ab2ffd700 1 -- 172.21.15.71:0/3510548911 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ab8021410 con 0x7f1abc105bf0 2024-04-23T02:53:00.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.672+0000 7f1ac3f85700 1 -- 172.21.15.71:0/3510548911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1abc196d70 con 0x7f1abc105bf0 2024-04-23T02:53:00.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.673+0000 7f1ac3f85700 1 -- 172.21.15.71:0/3510548911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1abc108020 con 0x7f1abc105bf0 2024-04-23T02:53:00.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.674+0000 7f1ab2ffd700 1 -- 172.21.15.71:0/3510548911 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1ab8003a60 con 0x7f1abc105bf0 2024-04-23T02:53:00.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.674+0000 7f1ab2ffd700 1 --2- 172.21.15.71:0/3510548911 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1aa806cd30 0x7f1aa806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:00.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.675+0000 7f1ab2ffd700 1 -- 172.21.15.71:0/3510548911 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1ab808dc70 con 0x7f1abc105bf0 2024-04-23T02:53:00.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.675+0000 7f1ac1d21700 1 --2- 172.21.15.71:0/3510548911 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1aa806cd30 0x7f1aa806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:00.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.675+0000 7f1ac1d21700 1 --2- 172.21.15.71:0/3510548911 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1aa806cd30 0x7f1aa806f1f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00ac50 tx=0x7f1aac00a380 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:53:00.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:00.683+0000 7f1ab2ffd700 1 -- 172.21.15.71:0/3510548911 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1ab8057c40 con 0x7f1abc105bf0 2024-04-23T02:53:00.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:00 smithi071 ceph-mon[144544]: from='client.35940 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:53:00.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:00 smithi071 ceph-mon[144544]: from='client.35944 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:53:00.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:00 smithi071 ceph-mon[144544]: from='client.35948 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:53:00.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:00 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-23T02:53:00.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:00 smithi071 ceph-mon[144544]: Standby daemon mds.cephfs.smithi071.qusrrb assigned to filesystem cephfs as rank 0 2024-04-23T02:53:00.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:00 smithi071 ceph-mon[144544]: Standby daemon mds.cephfs.smithi071.vxejye assigned to filesystem cephfs as rank 1 2024-04-23T02:53:00.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:00 smithi071 ceph-mon[144544]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T02:53:00.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:00 smithi071 ceph-mon[144544]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-23T02:53:00.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:00 smithi071 ceph-mon[144544]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-04-23T02:53:00.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:00 smithi071 ceph-mon[144544]: fsmap cephfs:2/2 {0=cephfs.smithi071.qusrrb=up:replay,1=cephfs.smithi071.vxejye=up:replay} 2 up:standby 2024-04-23T02:53:00.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:00 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-04-23T02:53:00.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:00 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/969448800' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:00.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:00 smithi192 ceph-mon[108617]: from='client.35940 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:53:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:00 smithi192 ceph-mon[108617]: from='client.35944 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:53:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:00 smithi192 ceph-mon[108617]: from='client.35948 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:53:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:00 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-23T02:53:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:00 smithi192 ceph-mon[108617]: Standby daemon mds.cephfs.smithi071.qusrrb assigned to filesystem cephfs as rank 0 2024-04-23T02:53:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:00 smithi192 ceph-mon[108617]: Standby daemon mds.cephfs.smithi071.vxejye assigned to filesystem cephfs as rank 1 2024-04-23T02:53:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:00 smithi192 ceph-mon[108617]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T02:53:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:00 smithi192 ceph-mon[108617]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-23T02:53:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:00 smithi192 ceph-mon[108617]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-04-23T02:53:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:00 smithi192 ceph-mon[108617]: fsmap cephfs:2/2 {0=cephfs.smithi071.qusrrb=up:replay,1=cephfs.smithi071.vxejye=up:replay} 2 up:standby 2024-04-23T02:53:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:00 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-04-23T02:53:00.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:00 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/969448800' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:01.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.024+0000 7f1ac3f85700 1 -- 172.21.15.71:0/3510548911 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1abc196f90 con 0x7f1aa806cd30 2024-04-23T02:53:01.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.025+0000 7f1ab2ffd700 1 -- 172.21.15.71:0/3510548911 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f1abc196f90 con 0x7f1aa806cd30 2024-04-23T02:53:01.029 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:53:01.029 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210", 2024-04-23T02:53:01.029 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": true, 2024-04-23T02:53:01.029 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-23T02:53:01.029 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [ 2024-04-23T02:53:01.029 INFO:teuthology.orchestra.run.smithi071.stdout: "osd", 2024-04-23T02:53:01.030 INFO:teuthology.orchestra.run.smithi071.stdout: "mds", 2024-04-23T02:53:01.030 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr", 2024-04-23T02:53:01.030 INFO:teuthology.orchestra.run.smithi071.stdout: "mon", 2024-04-23T02:53:01.030 INFO:teuthology.orchestra.run.smithi071.stdout: "crash" 2024-04-23T02:53:01.030 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-04-23T02:53:01.030 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": "16/23 daemons upgraded", 2024-04-23T02:53:01.030 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-04-23T02:53:01.030 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:53:01.030 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:53:01.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.029+0000 7f1ac3f85700 1 -- 172.21.15.71:0/3510548911 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1aa806cd30 msgr2=0x7f1aa806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:01.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.029+0000 7f1ac3f85700 1 --2- 172.21.15.71:0/3510548911 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1aa806cd30 0x7f1aa806f1f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00ac50 tx=0x7f1aac00a380 comp rx=0 tx=0).stop 2024-04-23T02:53:01.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.029+0000 7f1ac3f85700 1 -- 172.21.15.71:0/3510548911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc105bf0 msgr2=0x7f1abc19cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:01.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.029+0000 7f1ac3f85700 1 --2- 172.21.15.71:0/3510548911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc105bf0 0x7f1abc19cb50 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8006010 tx=0x7f1ab800bd70 comp rx=0 tx=0).stop 2024-04-23T02:53:01.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.030+0000 7f1ac3f85700 1 -- 172.21.15.71:0/3510548911 shutdown_connections 2024-04-23T02:53:01.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.030+0000 7f1ac3f85700 1 --2- 172.21.15.71:0/3510548911 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1aa806cd30 0x7f1aa806f1f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:01.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.030+0000 7f1ac3f85700 1 --2- 172.21.15.71:0/3510548911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1abc0ffb40 0x7f1abc19c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:01.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.030+0000 7f1ac3f85700 1 --2- 172.21.15.71:0/3510548911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc105bf0 0x7f1abc19cb50 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:01.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.030+0000 7f1ac3f85700 1 -- 172.21.15.71:0/3510548911 >> 172.21.15.71:0/3510548911 conn(0x7f1abc0fb610 msgr2=0x7f1abc0fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:53:01.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.030+0000 7f1ac3f85700 1 -- 172.21.15.71:0/3510548911 shutdown_connections 2024-04-23T02:53:01.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.030+0000 7f1ac3f85700 1 -- 172.21.15.71:0/3510548911 wait complete. 2024-04-23T02:53:01.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.188+0000 7f6d3f8ba700 1 -- 172.21.15.71:0/3395948805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d38068ac0 msgr2=0x7f6d38068f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:01.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.188+0000 7f6d3f8ba700 1 --2- 172.21.15.71:0/3395948805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d38068ac0 0x7f6d38068f20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009910 tx=0x7f6d28009c20 comp rx=0 tx=0).stop 2024-04-23T02:53:01.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.188+0000 7f6d3f8ba700 1 -- 172.21.15.71:0/3395948805 shutdown_connections 2024-04-23T02:53:01.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.188+0000 7f6d3f8ba700 1 --2- 172.21.15.71:0/3395948805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d38068ac0 0x7f6d38068f20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:01.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.188+0000 7f6d3f8ba700 1 --2- 172.21.15.71:0/3395948805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d38105b80 0x7f6d38068580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:01.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.188+0000 7f6d3f8ba700 1 -- 172.21.15.71:0/3395948805 >> 172.21.15.71:0/3395948805 conn(0x7f6d38075670 msgr2=0x7f6d38075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:53:01.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.189+0000 7f6d3f8ba700 1 -- 172.21.15.71:0/3395948805 shutdown_connections 2024-04-23T02:53:01.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.189+0000 7f6d3f8ba700 1 -- 172.21.15.71:0/3395948805 wait complete. 2024-04-23T02:53:01.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.190+0000 7f6d3f8ba700 1 Processor -- start 2024-04-23T02:53:01.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.191+0000 7f6d3f8ba700 1 -- start start 2024-04-23T02:53:01.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.191+0000 7f6d3f8ba700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d38068ac0 0x7f6d3819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:01.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.191+0000 7f6d3f8ba700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d38105b80 0x7f6d3819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:01.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.191+0000 7f6d3f8ba700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3819dad0 con 0x7f6d38068ac0 2024-04-23T02:53:01.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.191+0000 7f6d3f8ba700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d381a1810 con 0x7f6d38105b80 2024-04-23T02:53:01.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.191+0000 7f6d3ce55700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d38105b80 0x7f6d3819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:01.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.191+0000 7f6d3ce55700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d38105b80 0x7f6d3819d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.71:47426/0 (socket says 172.21.15.71:47426) 2024-04-23T02:53:01.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.191+0000 7f6d3ce55700 1 -- 172.21.15.71:0/915324390 learned_addr learned my addr 172.21.15.71:0/915324390 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:53:01.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.192+0000 7f6d3d656700 1 --2- 172.21.15.71:0/915324390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d38068ac0 0x7f6d3819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:01.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.192+0000 7f6d3ce55700 1 -- 172.21.15.71:0/915324390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d38068ac0 msgr2=0x7f6d3819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:01.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.192+0000 7f6d3ce55700 1 --2- 172.21.15.71:0/915324390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d38068ac0 0x7f6d3819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:01.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.192+0000 7f6d3ce55700 1 -- 172.21.15.71:0/915324390 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d280095f0 con 0x7f6d38105b80 2024-04-23T02:53:01.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.192+0000 7f6d3ce55700 1 --2- 172.21.15.71:0/915324390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d38105b80 0x7f6d3819d3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6d28006010 tx=0x7f6d2800be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:53:01.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.192+0000 7f6d2e7fc700 1 -- 172.21.15.71:0/915324390 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d2801d070 con 0x7f6d38105b80 2024-04-23T02:53:01.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.192+0000 7f6d3f8ba700 1 -- 172.21.15.71:0/915324390 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d381a1a90 con 0x7f6d38105b80 2024-04-23T02:53:01.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.192+0000 7f6d3f8ba700 1 -- 172.21.15.71:0/915324390 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d381a1f80 con 0x7f6d38105b80 2024-04-23T02:53:01.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.193+0000 7f6d2e7fc700 1 -- 172.21.15.71:0/915324390 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6d2800f460 con 0x7f6d38105b80 2024-04-23T02:53:01.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.193+0000 7f6d3f8ba700 1 -- 172.21.15.71:0/915324390 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d381101f0 con 0x7f6d38105b80 2024-04-23T02:53:01.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.193+0000 7f6d2e7fc700 1 -- 172.21.15.71:0/915324390 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d28021640 con 0x7f6d38105b80 2024-04-23T02:53:01.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.195+0000 7f6d2e7fc700 1 -- 172.21.15.71:0/915324390 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6d28003a60 con 0x7f6d38105b80 2024-04-23T02:53:01.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.195+0000 7f6d2e7fc700 1 --2- 172.21.15.71:0/915324390 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6d2406cd30 0x7f6d2406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:01.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.195+0000 7f6d2e7fc700 1 -- 172.21.15.71:0/915324390 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6d2808e1a0 con 0x7f6d38105b80 2024-04-23T02:53:01.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.199+0000 7f6d2e7fc700 1 -- 172.21.15.71:0/915324390 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6d28058390 con 0x7f6d38105b80 2024-04-23T02:53:01.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.201+0000 7f6d3d656700 1 --2- 172.21.15.71:0/915324390 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6d2406cd30 0x7f6d2406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:01.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.201+0000 7f6d3d656700 1 --2- 172.21.15.71:0/915324390 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6d2406cd30 0x7f6d2406f1f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6d34006fd0 tx=0x7f6d34008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:53:01.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:01 smithi071 ceph-mon[144544]: pgmap v1193: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:53:01.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:01 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-04-23T02:53:01.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:01 smithi071 ceph-mon[144544]: fsmap cephfs:2/2 {0=cephfs.smithi071.qusrrb=up:replay,1=cephfs.smithi071.vxejye=up:replay} 2 up:standby 2024-04-23T02:53:01.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:01 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:01.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:01 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:01.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:01 smithi071 ceph-mon[144544]: Upgrade: Setting container_image for all rgw 2024-04-23T02:53:01.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:01 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:01.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:01 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:01.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:01 smithi071 ceph-mon[144544]: Upgrade: Setting container_image for all rbd-mirror 2024-04-23T02:53:01.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:01 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:01.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:01 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:01.744 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:01 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3235217657' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:53:01.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.749+0000 7f6d3f8ba700 1 -- 172.21.15.71:0/915324390 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6d3804f2e0 con 0x7f6d38105b80 2024-04-23T02:53:01.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.750+0000 7f6d2e7fc700 1 -- 172.21.15.71:0/915324390 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3647 (secure 0 0 0) 0x7f6d28026e50 con 0x7f6d38105b80 2024-04-23T02:53:01.754 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_WARN failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-04-23T02:53:01.754 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-23T02:53:01.754 INFO:teuthology.orchestra.run.smithi071.stdout: host smithi192 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi192/config 2024-04-23T02:53:01.754 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-23T02:53:01.754 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmphxu_fjeo:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:53:01.754 INFO:teuthology.orchestra.run.smithi071.stdout:Traceback (most recent call last): 2024-04-23T02:53:01.754 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-23T02:53:01.754 INFO:teuthology.orchestra.run.smithi071.stdout: "__main__", mod_spec) 2024-04-23T02:53:01.754 INFO:teuthology.orchestra.run.smithi071.stdout: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-23T02:53:01.754 INFO:teuthology.orchestra.run.smithi071.stdout: exec(code, run_globals) 2024-04-23T02:53:01.754 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10889, in 2024-04-23T02:53:01.754 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 10877, in main 2024-04-23T02:53:01.755 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-23T02:53:01.755 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-23T02:53:01.755 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-23T02:53:01.755 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-23T02:53:01.755 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-23T02:53:01.755 INFO:teuthology.orchestra.run.smithi071.stdout: File "/tmp/tmpq4uopbc4.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-23T02:53:01.755 INFO:teuthology.orchestra.run.smithi071.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 -e NODE_NAME=smithi192 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmphxu_fjeo:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:417b28a7fda24d3105eeb9bae56a845b9ffa154499a85601880a5fd035847210 inventory --format=json-pretty --filter-for-batch 2024-04-23T02:53:01.755 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T02:53:01.755 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs is degraded 2024-04-23T02:53:01.755 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:53:01.755 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:53:01.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.753+0000 7f6d3f8ba700 1 -- 172.21.15.71:0/915324390 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6d2406cd30 msgr2=0x7f6d2406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:01.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.753+0000 7f6d3f8ba700 1 --2- 172.21.15.71:0/915324390 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6d2406cd30 0x7f6d2406f1f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6d34006fd0 tx=0x7f6d34008040 comp rx=0 tx=0).stop 2024-04-23T02:53:01.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.753+0000 7f6d3f8ba700 1 -- 172.21.15.71:0/915324390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d38105b80 msgr2=0x7f6d3819d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:01.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.753+0000 7f6d3f8ba700 1 --2- 172.21.15.71:0/915324390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d38105b80 0x7f6d3819d3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6d28006010 tx=0x7f6d2800be20 comp rx=0 tx=0).stop 2024-04-23T02:53:01.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.753+0000 7f6d3f8ba700 1 -- 172.21.15.71:0/915324390 shutdown_connections 2024-04-23T02:53:01.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.753+0000 7f6d3f8ba700 1 --2- 172.21.15.71:0/915324390 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f6d2406cd30 0x7f6d2406f1f0 secure :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6d34006fd0 tx=0x7f6d34008040 comp rx=0 tx=0).stop 2024-04-23T02:53:01.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.753+0000 7f6d3f8ba700 1 --2- 172.21.15.71:0/915324390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d38068ac0 0x7f6d3819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:01.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.753+0000 7f6d3f8ba700 1 --2- 172.21.15.71:0/915324390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d38105b80 0x7f6d3819d3f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:01.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.753+0000 7f6d3f8ba700 1 -- 172.21.15.71:0/915324390 >> 172.21.15.71:0/915324390 conn(0x7f6d38075670 msgr2=0x7f6d38101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:53:01.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.755+0000 7f6d3f8ba700 1 -- 172.21.15.71:0/915324390 shutdown_connections 2024-04-23T02:53:01.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:01.755+0000 7f6d3f8ba700 1 -- 172.21.15.71:0/915324390 wait complete. 2024-04-23T02:53:01.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:01 smithi192 ceph-mon[108617]: pgmap v1193: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-23T02:53:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:01 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-04-23T02:53:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:01 smithi192 ceph-mon[108617]: fsmap cephfs:2/2 {0=cephfs.smithi071.qusrrb=up:replay,1=cephfs.smithi071.vxejye=up:replay} 2 up:standby 2024-04-23T02:53:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:01 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:01 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:01 smithi192 ceph-mon[108617]: Upgrade: Setting container_image for all rgw 2024-04-23T02:53:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:01 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:01 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:01 smithi192 ceph-mon[108617]: Upgrade: Setting container_image for all rbd-mirror 2024-04-23T02:53:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:01 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:01 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:01.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:01 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3235217657' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:53:02.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:02 smithi071 ceph-mon[144544]: from='client.35960 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:53:02.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:02 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/915324390' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:53:02.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:02 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:02.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:02 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi071", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-23T02:53:02.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:02 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:53:02.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:02 smithi192 ceph-mon[108617]: from='client.35960 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:53:02.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:02 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/915324390' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:53:02.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:02 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:02.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:02 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi071", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-23T02:53:02.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:02 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:53:03.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:03 smithi071 ceph-mon[144544]: pgmap v1194: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 5.8 MiB/s rd, 2 op/s 2024-04-23T02:53:03.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:03 smithi071 ceph-mon[144544]: Upgrade: Updating ceph-exporter.smithi071 (1/2) 2024-04-23T02:53:03.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:03 smithi071 ceph-mon[144544]: Deploying daemon ceph-exporter.smithi071 on smithi071 2024-04-23T02:53:03.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:03 smithi192 ceph-mon[108617]: pgmap v1194: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 5.8 MiB/s rd, 2 op/s 2024-04-23T02:53:03.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:03 smithi192 ceph-mon[108617]: Upgrade: Updating ceph-exporter.smithi071 (1/2) 2024-04-23T02:53:03.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:03 smithi192 ceph-mon[108617]: Deploying daemon ceph-exporter.smithi071 on smithi071 2024-04-23T02:53:05.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:05 smithi071 ceph-mon[144544]: pgmap v1195: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 8 op/s 2024-04-23T02:53:05.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:05.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:53:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:05 smithi192 ceph-mon[108617]: pgmap v1195: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 8 op/s 2024-04-23T02:53:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:05.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:53:06.974 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:06 smithi192 ceph-mon[108617]: mds.? [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] up:resolve 2024-04-23T02:53:06.974 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:06 smithi192 ceph-mon[108617]: fsmap cephfs:2/2 {0=cephfs.smithi071.qusrrb=up:replay,1=cephfs.smithi071.vxejye=up:resolve} 2 up:standby 2024-04-23T02:53:06.974 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:06 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:06.974 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:06 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:06.974 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:06 smithi192 ceph-mon[108617]: pgmap v1196: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 9 op/s 2024-04-23T02:53:07.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:06 smithi071 ceph-mon[144544]: mds.? [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] up:resolve 2024-04-23T02:53:07.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:06 smithi071 ceph-mon[144544]: fsmap cephfs:2/2 {0=cephfs.smithi071.qusrrb=up:replay,1=cephfs.smithi071.vxejye=up:resolve} 2 up:standby 2024-04-23T02:53:07.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:06 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:07.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:06 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:07.243 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:06 smithi071 ceph-mon[144544]: pgmap v1196: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 9 op/s 2024-04-23T02:53:08.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:08 smithi071 ceph-mon[144544]: mds.? [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] up:resolve 2024-04-23T02:53:08.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:08 smithi071 ceph-mon[144544]: fsmap cephfs:2/2 {0=cephfs.smithi071.qusrrb=up:resolve,1=cephfs.smithi071.vxejye=up:resolve} 2 up:standby 2024-04-23T02:53:08.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:08 smithi071 ceph-mon[144544]: Upgrade: Updating ceph-exporter.smithi192 (2/2) 2024-04-23T02:53:08.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:08 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:08.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:08 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi192", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-23T02:53:08.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:08 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:53:08.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:08 smithi071 ceph-mon[144544]: Deploying daemon ceph-exporter.smithi192 on smithi192 2024-04-23T02:53:08.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:08 smithi192 ceph-mon[108617]: mds.? [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] up:resolve 2024-04-23T02:53:08.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:08 smithi192 ceph-mon[108617]: fsmap cephfs:2/2 {0=cephfs.smithi071.qusrrb=up:resolve,1=cephfs.smithi071.vxejye=up:resolve} 2 up:standby 2024-04-23T02:53:08.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:08 smithi192 ceph-mon[108617]: Upgrade: Updating ceph-exporter.smithi192 (2/2) 2024-04-23T02:53:08.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:08 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:08.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:08 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi192", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-23T02:53:08.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:08 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:53:08.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:08 smithi192 ceph-mon[108617]: Deploying daemon ceph-exporter.smithi192 on smithi192 2024-04-23T02:53:09.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:09 smithi192 ceph-mon[108617]: reconnect by client.14534 192.168.0.1:0/1982449790 after 0.000999982 2024-04-23T02:53:09.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:09 smithi192 ceph-mon[108617]: reconnect by client.24303 192.168.0.1:0/490649041 after 0.000999982 2024-04-23T02:53:09.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:09 smithi192 ceph-mon[108617]: reconnect by client.14534 192.168.0.1:0/1982449790 after 0.00199996 2024-04-23T02:53:09.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:09 smithi192 ceph-mon[108617]: reconnect by client.24303 192.168.0.1:0/490649041 after 0.00199996 2024-04-23T02:53:09.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:09 smithi192 ceph-mon[108617]: mds.? [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] up:reconnect 2024-04-23T02:53:09.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:09 smithi192 ceph-mon[108617]: mds.? [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] up:reconnect 2024-04-23T02:53:09.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:09 smithi192 ceph-mon[108617]: fsmap cephfs:2/2 {0=cephfs.smithi071.qusrrb=up:reconnect,1=cephfs.smithi071.vxejye=up:reconnect} 2 up:standby 2024-04-23T02:53:09.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:09 smithi192 ceph-mon[108617]: pgmap v1197: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 31 MiB/s rd, 10 op/s 2024-04-23T02:53:09.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:09 smithi071 ceph-mon[144544]: reconnect by client.14534 192.168.0.1:0/1982449790 after 0.000999982 2024-04-23T02:53:09.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:09 smithi071 ceph-mon[144544]: reconnect by client.24303 192.168.0.1:0/490649041 after 0.000999982 2024-04-23T02:53:09.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:09 smithi071 ceph-mon[144544]: reconnect by client.14534 192.168.0.1:0/1982449790 after 0.00199996 2024-04-23T02:53:09.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:09 smithi071 ceph-mon[144544]: reconnect by client.24303 192.168.0.1:0/490649041 after 0.00199996 2024-04-23T02:53:09.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:09 smithi071 ceph-mon[144544]: mds.? [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] up:reconnect 2024-04-23T02:53:09.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:09 smithi071 ceph-mon[144544]: mds.? [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] up:reconnect 2024-04-23T02:53:09.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:09 smithi071 ceph-mon[144544]: fsmap cephfs:2/2 {0=cephfs.smithi071.qusrrb=up:reconnect,1=cephfs.smithi071.vxejye=up:reconnect} 2 up:standby 2024-04-23T02:53:09.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:09 smithi071 ceph-mon[144544]: pgmap v1197: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 31 MiB/s rd, 10 op/s 2024-04-23T02:53:10.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:10 smithi192 ceph-mon[108617]: mds.? [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] up:rejoin 2024-04-23T02:53:10.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:10 smithi192 ceph-mon[108617]: mds.? [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] up:rejoin 2024-04-23T02:53:10.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:10 smithi192 ceph-mon[108617]: fsmap cephfs:2/2 {0=cephfs.smithi071.qusrrb=up:rejoin,1=cephfs.smithi071.vxejye=up:rejoin} 2 up:standby 2024-04-23T02:53:10.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:10 smithi192 ceph-mon[108617]: daemon mds.cephfs.smithi071.qusrrb is now active in filesystem cephfs as rank 0 2024-04-23T02:53:10.376 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:10 smithi192 ceph-mon[108617]: daemon mds.cephfs.smithi071.vxejye is now active in filesystem cephfs as rank 1 2024-04-23T02:53:10.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:10 smithi071 ceph-mon[144544]: mds.? [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] up:rejoin 2024-04-23T02:53:10.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:10 smithi071 ceph-mon[144544]: mds.? [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] up:rejoin 2024-04-23T02:53:10.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:10 smithi071 ceph-mon[144544]: fsmap cephfs:2/2 {0=cephfs.smithi071.qusrrb=up:rejoin,1=cephfs.smithi071.vxejye=up:rejoin} 2 up:standby 2024-04-23T02:53:10.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:10 smithi071 ceph-mon[144544]: daemon mds.cephfs.smithi071.qusrrb is now active in filesystem cephfs as rank 0 2024-04-23T02:53:10.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:10 smithi071 ceph-mon[144544]: daemon mds.cephfs.smithi071.vxejye is now active in filesystem cephfs as rank 1 2024-04-23T02:53:11.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:11 smithi071 ceph-mon[144544]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T02:53:11.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:11 smithi071 ceph-mon[144544]: mds.? [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] up:active 2024-04-23T02:53:11.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:11 smithi071 ceph-mon[144544]: mds.? [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] up:active 2024-04-23T02:53:11.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:11 smithi071 ceph-mon[144544]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi071.vxejye=up:active} 2 up:standby 2024-04-23T02:53:11.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:11 smithi071 ceph-mon[144544]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi071.vxejye=up:active} 2 up:standby-replay 2024-04-23T02:53:11.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:11 smithi071 ceph-mon[144544]: pgmap v1198: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 8 op/s 2024-04-23T02:53:11.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:11 smithi192 ceph-mon[108617]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T02:53:11.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:11 smithi192 ceph-mon[108617]: mds.? [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] up:active 2024-04-23T02:53:11.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:11 smithi192 ceph-mon[108617]: mds.? [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] up:active 2024-04-23T02:53:11.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:11 smithi192 ceph-mon[108617]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi071.vxejye=up:active} 2 up:standby 2024-04-23T02:53:11.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:11 smithi192 ceph-mon[108617]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi071.vxejye=up:active} 2 up:standby-replay 2024-04-23T02:53:11.626 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:11 smithi192 ceph-mon[108617]: pgmap v1198: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 8 op/s 2024-04-23T02:53:13.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:13 smithi071 ceph-mon[144544]: pgmap v1199: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 30 MiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-23T02:53:13.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:13 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:13.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:13 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:13.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:13 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:53:13.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:13 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:13.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:13 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:13.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:13 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:13.743 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:13 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:13.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:13 smithi192 ceph-mon[108617]: pgmap v1199: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 30 MiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-23T02:53:13.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:13 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:13.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:13 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:13.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:13 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:53:13.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:13 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:13.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:13 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:13.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:13 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:13.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:13 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:15.846 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:15 smithi071 ceph-mon[144544]: pgmap v1200: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 39 MiB/s rd, 1.7 KiB/s wr, 19 op/s 2024-04-23T02:53:15.846 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:15 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:15.846 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:15 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:15.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:15 smithi192 ceph-mon[108617]: pgmap v1200: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 39 MiB/s rd, 1.7 KiB/s wr, 19 op/s 2024-04-23T02:53:15.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:15 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:15.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:15 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:17.418 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:17 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:17.418 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:17 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:17.418 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:17 smithi192 ceph-mon[108617]: pgmap v1201: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 6.2 KiB/s wr, 17 op/s 2024-04-23T02:53:17.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:17 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:17.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:17 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:17.493 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:17 smithi071 ceph-mon[144544]: pgmap v1201: 65 pgs: 65 active+clean; 313 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 6.2 KiB/s wr, 17 op/s 2024-04-23T02:53:19.484 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:19 smithi192 ceph-mon[108617]: pgmap v1202: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 30 MiB/s rd, 14 KiB/s wr, 20 op/s 2024-04-23T02:53:19.484 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:19.484 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:19 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:19.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:19 smithi071 ceph-mon[144544]: pgmap v1202: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 30 MiB/s rd, 14 KiB/s wr, 20 op/s 2024-04-23T02:53:19.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:19.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:19 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:20.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:20.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:20.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:20.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:53:20.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:20.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:21.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:21.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:21.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:21.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:53:21.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:21.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:21.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:21 smithi071 ceph-mon[144544]: pgmap v1203: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 14 KiB/s wr, 19 op/s 2024-04-23T02:53:22.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:21 smithi192 ceph-mon[108617]: pgmap v1203: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 14 KiB/s wr, 19 op/s 2024-04-23T02:53:23.740 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:23 smithi071 ceph-mon[144544]: pgmap v1204: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 14 KiB/s wr, 20 op/s 2024-04-23T02:53:24.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:23 smithi192 ceph-mon[108617]: pgmap v1204: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 14 KiB/s wr, 20 op/s 2024-04-23T02:53:25.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:25 smithi071 ceph-mon[144544]: pgmap v1205: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-23T02:53:25.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:25.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:25.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:53:25.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:53:25.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:25.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:53:25.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:25.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:25.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:25.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:25.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:25.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:25 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-23T02:53:26.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:25 smithi192 ceph-mon[108617]: pgmap v1205: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-23T02:53:26.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:26.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:26.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:53:26.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:53:26.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:26.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:53:26.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:26.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:26.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:26.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:26.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:26.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:25 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-23T02:53:26.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: pgmap v1206: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 8.8 MiB/s rd, 14 KiB/s wr, 8 op/s 2024-04-23T02:53:26.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: Upgrade: Setting filesystem cephfs Joinable 2024-04-23T02:53:26.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-04-23T02:53:26.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-23T02:53:26.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi071.vxejye=up:active} 2 up:standby-replay 2024-04-23T02:53:26.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:26.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:26.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:26.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:26.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:26.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi071"}]: dispatch 2024-04-23T02:53:26.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi071"}]': finished 2024-04-23T02:53:26.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi192"}]: dispatch 2024-04-23T02:53:26.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi192"}]': finished 2024-04-23T02:53:26.994 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:26.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:26.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:26.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:26.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:26.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:26.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:26.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:26.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:26.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:26.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:26.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:26.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-23T02:53:26.995 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-23T02:53:26.996 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:53:26.996 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-23T02:53:26.996 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-23T02:53:26.996 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-23T02:53:26.996 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-23T02:53:26.996 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-23T02:53:26.996 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-23T02:53:26.996 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-23T02:53:26.996 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-23T02:53:26.996 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-23T02:53:26.996 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-23T02:53:26.996 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-23T02:53:26.996 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:53:26.997 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-23T02:53:26.997 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-23T02:53:26.997 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-23T02:53:26.997 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-23T02:53:26.997 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-23T02:53:26.997 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-23T02:53:26.997 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-23T02:53:26.997 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-23T02:53:26.997 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:53:26.997 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:53:26.997 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:53:26.997 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:53:26.998 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:53:26.998 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:53:26.998 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-04-23T02:53:26.998 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-04-23T02:53:26.998 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-23T02:53:26.998 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-23T02:53:26.998 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:53:26.998 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:53:26.998 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:53:26.998 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:26.998 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:53:26.998 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:53:26.998 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:53:26.999 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:26 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:27.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: pgmap v1206: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 8.8 MiB/s rd, 14 KiB/s wr, 8 op/s 2024-04-23T02:53:27.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: Upgrade: Setting filesystem cephfs Joinable 2024-04-23T02:53:27.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-04-23T02:53:27.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-23T02:53:27.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: fsmap cephfs:2 {0=cephfs.smithi071.qusrrb=up:active,1=cephfs.smithi071.vxejye=up:active} 2 up:standby-replay 2024-04-23T02:53:27.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:27.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:27.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:27.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:27.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:27.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi071"}]: dispatch 2024-04-23T02:53:27.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi071"}]': finished 2024-04-23T02:53:27.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi192"}]: dispatch 2024-04-23T02:53:27.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi192"}]': finished 2024-04-23T02:53:27.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:27.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:27.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:27.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:27.127 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:27.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:27.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:27.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:27.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:27.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:27.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:27.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:27.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-23T02:53:27.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-23T02:53:27.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:53:27.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-23T02:53:27.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-23T02:53:27.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-23T02:53:27.128 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-23T02:53:27.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-23T02:53:27.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-23T02:53:27.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-23T02:53:27.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-23T02:53:27.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-23T02:53:27.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-23T02:53:27.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-23T02:53:27.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:53:27.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-23T02:53:27.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-23T02:53:27.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-23T02:53:27.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-23T02:53:27.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-23T02:53:27.129 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-23T02:53:27.130 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-23T02:53:27.130 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-23T02:53:27.130 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:53:27.130 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:53:27.130 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:53:27.130 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:53:27.130 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:53:27.130 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-23T02:53:27.130 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-04-23T02:53:27.130 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-04-23T02:53:27.130 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-23T02:53:27.130 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-23T02:53:27.130 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:53:27.130 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:53:27.131 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:53:27.131 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:27.131 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:53:27.131 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:53:27.131 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:53:27.131 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:26 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:27.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:27 smithi192 ceph-mon[108617]: Upgrade: Setting container_image for all ceph-exporter 2024-04-23T02:53:27.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:27 smithi192 ceph-mon[108617]: Upgrade: Setting container_image for all iscsi 2024-04-23T02:53:27.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:27 smithi192 ceph-mon[108617]: Upgrade: Setting container_image for all nfs 2024-04-23T02:53:27.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:27 smithi192 ceph-mon[108617]: Upgrade: Setting container_image for all nvmeof 2024-04-23T02:53:27.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:27 smithi192 ceph-mon[108617]: Upgrade: Finalizing container_image settings 2024-04-23T02:53:27.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:27 smithi192 ceph-mon[108617]: Upgrade: Complete! 2024-04-23T02:53:27.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:27 smithi071 ceph-mon[144544]: Upgrade: Setting container_image for all ceph-exporter 2024-04-23T02:53:27.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:27 smithi071 ceph-mon[144544]: Upgrade: Setting container_image for all iscsi 2024-04-23T02:53:27.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:27 smithi071 ceph-mon[144544]: Upgrade: Setting container_image for all nfs 2024-04-23T02:53:27.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:27 smithi071 ceph-mon[144544]: Upgrade: Setting container_image for all nvmeof 2024-04-23T02:53:27.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:27 smithi071 ceph-mon[144544]: Upgrade: Finalizing container_image settings 2024-04-23T02:53:27.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:27 smithi071 ceph-mon[144544]: Upgrade: Complete! 2024-04-23T02:53:28.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:28 smithi071 ceph-mon[144544]: pgmap v1207: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6.6 MiB/s rd, 8.9 KiB/s wr, 7 op/s 2024-04-23T02:53:29.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:28 smithi192 ceph-mon[108617]: pgmap v1207: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6.6 MiB/s rd, 8.9 KiB/s wr, 7 op/s 2024-04-23T02:53:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:30 smithi071 ceph-mon[144544]: pgmap v1208: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 381 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-23T02:53:30.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:30 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:31.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:30 smithi192 ceph-mon[108617]: pgmap v1208: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 381 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-23T02:53:31.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:30 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:31.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.874+0000 7f3c877ac700 1 -- 172.21.15.71:0/1474113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c800fff50 msgr2=0x7f3c80105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:31.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.874+0000 7f3c877ac700 1 --2- 172.21.15.71:0/1474113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c800fff50 0x7f3c80105840 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009910 tx=0x7f3c7c009c20 comp rx=0 tx=0).stop 2024-04-23T02:53:31.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.874+0000 7f3c877ac700 1 -- 172.21.15.71:0/1474113 shutdown_connections 2024-04-23T02:53:31.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.874+0000 7f3c877ac700 1 --2- 172.21.15.71:0/1474113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c800fff50 0x7f3c80105840 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:31.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.874+0000 7f3c877ac700 1 --2- 172.21.15.71:0/1474113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c800ff5a0 0x7f3c800ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:31.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.874+0000 7f3c877ac700 1 -- 172.21.15.71:0/1474113 >> 172.21.15.71:0/1474113 conn(0x7f3c80074e30 msgr2=0x7f3c80075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:53:31.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.874+0000 7f3c877ac700 1 -- 172.21.15.71:0/1474113 shutdown_connections 2024-04-23T02:53:31.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.874+0000 7f3c877ac700 1 -- 172.21.15.71:0/1474113 wait complete. 2024-04-23T02:53:31.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.875+0000 7f3c877ac700 1 Processor -- start 2024-04-23T02:53:31.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.875+0000 7f3c877ac700 1 -- start start 2024-04-23T02:53:31.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.875+0000 7f3c877ac700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c800ff5a0 0x7f3c80100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:31.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.875+0000 7f3c877ac700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c800fff50 0x7f3c80100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:31.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.875+0000 7f3c877ac700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c80100fa0 con 0x7f3c800fff50 2024-04-23T02:53:31.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.875+0000 7f3c877ac700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c801010e0 con 0x7f3c800ff5a0 2024-04-23T02:53:31.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.875+0000 7f3c84d47700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c800fff50 0x7f3c80100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:31.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.875+0000 7f3c84d47700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c800fff50 0x7f3c80100a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46224/0 (socket says 172.21.15.71:46224) 2024-04-23T02:53:31.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.875+0000 7f3c84d47700 1 -- 172.21.15.71:0/1777835127 learned_addr learned my addr 172.21.15.71:0/1777835127 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:53:31.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.875+0000 7f3c85548700 1 --2- 172.21.15.71:0/1777835127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c800ff5a0 0x7f3c80100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:31.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.876+0000 7f3c84d47700 1 -- 172.21.15.71:0/1777835127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c800ff5a0 msgr2=0x7f3c80100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:31.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.876+0000 7f3c84d47700 1 --2- 172.21.15.71:0/1777835127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c800ff5a0 0x7f3c80100520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:31.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.876+0000 7f3c84d47700 1 -- 172.21.15.71:0/1777835127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c0095f0 con 0x7f3c800fff50 2024-04-23T02:53:31.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.876+0000 7f3c85548700 1 --2- 172.21.15.71:0/1777835127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c800ff5a0 0x7f3c80100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:53:31.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.876+0000 7f3c84d47700 1 --2- 172.21.15.71:0/1777835127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c800fff50 0x7f3c80100a60 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0092e0 tx=0x7f3c7c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:53:31.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.876+0000 7f3c767fc700 1 -- 172.21.15.71:0/1777835127 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c7c01d070 con 0x7f3c800fff50 2024-04-23T02:53:31.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.876+0000 7f3c767fc700 1 -- 172.21.15.71:0/1777835127 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3c7c00f460 con 0x7f3c800fff50 2024-04-23T02:53:31.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.876+0000 7f3c877ac700 1 -- 172.21.15.71:0/1777835127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c80101360 con 0x7f3c800fff50 2024-04-23T02:53:31.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.876+0000 7f3c767fc700 1 -- 172.21.15.71:0/1777835127 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c7c021600 con 0x7f3c800fff50 2024-04-23T02:53:31.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.876+0000 7f3c877ac700 1 -- 172.21.15.71:0/1777835127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c801a2dc0 con 0x7f3c800fff50 2024-04-23T02:53:31.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.877+0000 7f3c877ac700 1 -- 172.21.15.71:0/1777835127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c80109200 con 0x7f3c800fff50 2024-04-23T02:53:31.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.877+0000 7f3c767fc700 1 -- 172.21.15.71:0/1777835127 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3c7c00f5d0 con 0x7f3c800fff50 2024-04-23T02:53:31.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.878+0000 7f3c767fc700 1 --2- 172.21.15.71:0/1777835127 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3c6c06cce0 0x7f3c6c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:31.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.878+0000 7f3c85548700 1 --2- 172.21.15.71:0/1777835127 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3c6c06cce0 0x7f3c6c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:31.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.878+0000 7f3c767fc700 1 -- 172.21.15.71:0/1777835127 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3c7c08d860 con 0x7f3c800fff50 2024-04-23T02:53:31.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.878+0000 7f3c85548700 1 --2- 172.21.15.71:0/1777835127 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3c6c06cce0 0x7f3c6c06f1a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3c70005950 tx=0x7f3c70005f50 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:53:31.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:31.883+0000 7f3c767fc700 1 -- 172.21.15.71:0/1777835127 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3c7c0578d0 con 0x7f3c800fff50 2024-04-23T02:53:32.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:32.092+0000 7f3c877ac700 1 -- 172.21.15.71:0/1777835127 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c80101ac0 con 0x7f3c6c06cce0 2024-04-23T02:53:32.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:32.093+0000 7f3c767fc700 1 -- 172.21.15.71:0/1777835127 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f3c80101ac0 con 0x7f3c6c06cce0 2024-04-23T02:53:32.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:32.095+0000 7f3c877ac700 1 -- 172.21.15.71:0/1777835127 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3c6c06cce0 msgr2=0x7f3c6c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:32.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:32.095+0000 7f3c877ac700 1 --2- 172.21.15.71:0/1777835127 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3c6c06cce0 0x7f3c6c06f1a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3c70005950 tx=0x7f3c70005f50 comp rx=0 tx=0).stop 2024-04-23T02:53:32.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:32.095+0000 7f3c877ac700 1 -- 172.21.15.71:0/1777835127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c800fff50 msgr2=0x7f3c80100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:32.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:32.095+0000 7f3c877ac700 1 --2- 172.21.15.71:0/1777835127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c800fff50 0x7f3c80100a60 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0092e0 tx=0x7f3c7c00be20 comp rx=0 tx=0).stop 2024-04-23T02:53:32.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:32.096+0000 7f3c877ac700 1 -- 172.21.15.71:0/1777835127 shutdown_connections 2024-04-23T02:53:32.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:32.096+0000 7f3c877ac700 1 --2- 172.21.15.71:0/1777835127 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3c6c06cce0 0x7f3c6c06f1a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:32.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:32.096+0000 7f3c877ac700 1 --2- 172.21.15.71:0/1777835127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c800ff5a0 0x7f3c80100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:32.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:32.096+0000 7f3c877ac700 1 --2- 172.21.15.71:0/1777835127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c800fff50 0x7f3c80100a60 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:32.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:32.096+0000 7f3c877ac700 1 -- 172.21.15.71:0/1777835127 >> 172.21.15.71:0/1777835127 conn(0x7f3c80074e30 msgr2=0x7f3c800fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:53:32.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:32.096+0000 7f3c877ac700 1 -- 172.21.15.71:0/1777835127 shutdown_connections 2024-04-23T02:53:32.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:32.096+0000 7f3c877ac700 1 -- 172.21.15.71:0/1777835127 wait complete. 2024-04-23T02:53:32.741 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:32 smithi071 ceph-mon[144544]: pgmap v1209: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 4 op/s 2024-04-23T02:53:32.764 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph orch ps' 2024-04-23T02:53:33.037 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:53:33.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:32 smithi192 ceph-mon[108617]: pgmap v1209: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 4 op/s 2024-04-23T02:53:33.888 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:33 smithi071 ceph-mon[144544]: from='client.35970 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:53:34.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:33 smithi192 ceph-mon[108617]: from='client.35970 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:53:34.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:34 smithi071 ceph-mon[144544]: pgmap v1210: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-23T02:53:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.766+0000 7f2742054700 1 -- 172.21.15.71:0/2824405223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f273c068580 msgr2=0x7f273c0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.766+0000 7f2742054700 1 --2- 172.21.15.71:0/2824405223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f273c068580 0x7f273c0689e0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2730009910 tx=0x7f2730009c20 comp rx=0 tx=0).stop 2024-04-23T02:53:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.766+0000 7f2742054700 1 -- 172.21.15.71:0/2824405223 shutdown_connections 2024-04-23T02:53:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.766+0000 7f2742054700 1 --2- 172.21.15.71:0/2824405223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f273c068580 0x7f273c0689e0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.766+0000 7f2742054700 1 --2- 172.21.15.71:0/2824405223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f273c106cd0 0x7f273c1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.766+0000 7f2742054700 1 -- 172.21.15.71:0/2824405223 >> 172.21.15.71:0/2824405223 conn(0x7f273c075670 msgr2=0x7f273c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:53:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.766+0000 7f2742054700 1 -- 172.21.15.71:0/2824405223 shutdown_connections 2024-04-23T02:53:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.766+0000 7f2742054700 1 -- 172.21.15.71:0/2824405223 wait complete. 2024-04-23T02:53:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.766+0000 7f2742054700 1 Processor -- start 2024-04-23T02:53:34.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.767+0000 7f2742054700 1 -- start start 2024-04-23T02:53:34.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.767+0000 7f2742054700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f273c068580 0x7f273c19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:34.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.767+0000 7f2742054700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f273c106cd0 0x7f273c19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:34.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.767+0000 7f2742054700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c19d110 con 0x7f273c106cd0 2024-04-23T02:53:34.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.767+0000 7f2742054700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c1965c0 con 0x7f273c068580 2024-04-23T02:53:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.767+0000 7f273affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f273c106cd0 0x7f273c19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.767+0000 7f273affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f273c106cd0 0x7f273c19ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46238/0 (socket says 172.21.15.71:46238) 2024-04-23T02:53:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.767+0000 7f273affd700 1 -- 172.21.15.71:0/429861465 learned_addr learned my addr 172.21.15.71:0/429861465 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:53:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.767+0000 7f273b7fe700 1 --2- 172.21.15.71:0/429861465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f273c068580 0x7f273c19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.767+0000 7f273affd700 1 -- 172.21.15.71:0/429861465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f273c068580 msgr2=0x7f273c19c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.767+0000 7f273affd700 1 --2- 172.21.15.71:0/429861465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f273c068580 0x7f273c19c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.767+0000 7f273affd700 1 -- 172.21.15.71:0/429861465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27300095f0 con 0x7f273c106cd0 2024-04-23T02:53:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.767+0000 7f273affd700 1 --2- 172.21.15.71:0/429861465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f273c106cd0 0x7f273c19ca80 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f2730000c00 tx=0x7f273000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:53:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.767+0000 7f2738ff9700 1 -- 172.21.15.71:0/429861465 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f273001d070 con 0x7f273c106cd0 2024-04-23T02:53:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.768+0000 7f2738ff9700 1 -- 172.21.15.71:0/429861465 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2730005dc0 con 0x7f273c106cd0 2024-04-23T02:53:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.768+0000 7f2738ff9700 1 -- 172.21.15.71:0/429861465 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2730003940 con 0x7f273c106cd0 2024-04-23T02:53:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.768+0000 7f2742054700 1 -- 172.21.15.71:0/429861465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273c196840 con 0x7f273c106cd0 2024-04-23T02:53:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.768+0000 7f2742054700 1 -- 172.21.15.71:0/429861465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273c196cd0 con 0x7f273c106cd0 2024-04-23T02:53:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.769+0000 7f2742054700 1 -- 172.21.15.71:0/429861465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f273c1091d0 con 0x7f273c106cd0 2024-04-23T02:53:34.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.769+0000 7f2738ff9700 1 -- 172.21.15.71:0/429861465 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2730003aa0 con 0x7f273c106cd0 2024-04-23T02:53:34.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.769+0000 7f2738ff9700 1 --2- 172.21.15.71:0/429861465 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f272406cce0 0x7f272406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:34.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.769+0000 7f2738ff9700 1 -- 172.21.15.71:0/429861465 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f273008dac0 con 0x7f273c106cd0 2024-04-23T02:53:34.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.769+0000 7f273b7fe700 1 --2- 172.21.15.71:0/429861465 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f272406cce0 0x7f272406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:34.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.770+0000 7f273b7fe700 1 --2- 172.21.15.71:0/429861465 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f272406cce0 0x7f272406f1a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f272c009c60 tx=0x7f272c009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:53:34.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.774+0000 7f2738ff9700 1 -- 172.21.15.71:0/429861465 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f273005b5c0 con 0x7f273c106cd0 2024-04-23T02:53:34.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.984+0000 7f2742054700 1 -- 172.21.15.71:0/429861465 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f273c196fa0 con 0x7f272406cce0 2024-04-23T02:53:34.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.995+0000 7f2738ff9700 1 -- 172.21.15.71:0/429861465 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f273c196fa0 con 0x7f272406cce0 2024-04-23T02:53:34.998 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:53:34.998 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (37m) 15s ago 48m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:53:34.998 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (29s) 15s ago 48m 7335k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 53cec9e97349 2024-04-23T02:53:34.998 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (23s) 19s ago 47m 6887k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 78dbe1b6f2cd 2024-04-23T02:53:34.998 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (35m) 15s ago 48m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:53:34.998 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (35m) 19s ago 47m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:53:34.998 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (36m) 15s ago 48m 88.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:53:34.998 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (31m) 15s ago 42m 65.6M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:53:34.998 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (31m) 15s ago 42m 66.0M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:53:34.999 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (15m) 19s ago 42m 195M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:53:34.999 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (30m) 19s ago 42m 184M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:53:34.999 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (39m) 15s ago 50m 517M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:53:34.999 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (38m) 19s ago 47m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:53:34.999 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (35m) 15s ago 50m 85.3M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:53:34.999 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (35m) 19s ago 47m 60.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:53:34.999 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (38m) 15s ago 48m 22.0M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:53:34.999 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (38m) 19s ago 47m 18.5M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:53:34.999 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (34m) 15s ago 46m 101M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:53:34.999 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (34m) 15s ago 46m 103M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:53:34.999 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (33m) 15s ago 45m 91.9M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:53:34.999 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (32m) 19s ago 45m 120M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:53:34.999 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (32m) 19s ago 44m 101M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:53:34.999 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (31m) 19s ago 44m 93.4M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:53:35.000 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (37m) 15s ago 48m 93.3M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:53:35.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.997+0000 7f2742054700 1 -- 172.21.15.71:0/429861465 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f272406cce0 msgr2=0x7f272406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:35.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.997+0000 7f2742054700 1 --2- 172.21.15.71:0/429861465 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f272406cce0 0x7f272406f1a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f272c009c60 tx=0x7f272c009260 comp rx=0 tx=0).stop 2024-04-23T02:53:35.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.997+0000 7f2742054700 1 -- 172.21.15.71:0/429861465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f273c106cd0 msgr2=0x7f273c19ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:35.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.997+0000 7f2742054700 1 --2- 172.21.15.71:0/429861465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f273c106cd0 0x7f273c19ca80 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f2730000c00 tx=0x7f273000bde0 comp rx=0 tx=0).stop 2024-04-23T02:53:35.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.998+0000 7f2742054700 1 -- 172.21.15.71:0/429861465 shutdown_connections 2024-04-23T02:53:35.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.998+0000 7f2742054700 1 --2- 172.21.15.71:0/429861465 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f272406cce0 0x7f272406f1a0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:35.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.998+0000 7f2742054700 1 --2- 172.21.15.71:0/429861465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f273c068580 0x7f273c19c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:35.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.998+0000 7f2742054700 1 --2- 172.21.15.71:0/429861465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f273c106cd0 0x7f273c19ca80 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:35.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.998+0000 7f2742054700 1 -- 172.21.15.71:0/429861465 >> 172.21.15.71:0/429861465 conn(0x7f273c075670 msgr2=0x7f273c0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:53:35.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.998+0000 7f2742054700 1 -- 172.21.15.71:0/429861465 shutdown_connections 2024-04-23T02:53:35.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:34.998+0000 7f2742054700 1 -- 172.21.15.71:0/429861465 wait complete. 2024-04-23T02:53:35.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:34 smithi192 ceph-mon[108617]: pgmap v1210: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-23T02:53:35.648 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph orch upgrade status' 2024-04-23T02:53:35.878 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:53:35.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:53:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:35 smithi071 ceph-mon[144544]: from='client.35974 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:53:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:35 smithi071 ceph-mon[144544]: pgmap v1211: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-23T02:53:36.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:53:36.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:53:36.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:35 smithi192 ceph-mon[108617]: from='client.35974 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:53:36.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:35 smithi192 ceph-mon[108617]: pgmap v1211: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-23T02:53:37.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.627+0000 7ff6729cd700 1 -- 172.21.15.71:0/1620785277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff66c101850 msgr2=0x7ff66c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:37.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.627+0000 7ff6729cd700 1 --2- 172.21.15.71:0/1620785277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff66c101850 0x7ff66c1058a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009910 tx=0x7ff65c009c20 comp rx=0 tx=0).stop 2024-04-23T02:53:37.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.627+0000 7ff6729cd700 1 -- 172.21.15.71:0/1620785277 shutdown_connections 2024-04-23T02:53:37.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.627+0000 7ff6729cd700 1 --2- 172.21.15.71:0/1620785277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff66c101850 0x7ff66c1058a0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:37.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.627+0000 7ff6729cd700 1 --2- 172.21.15.71:0/1620785277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff66c100ea0 0x7ff66c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:37.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.627+0000 7ff6729cd700 1 -- 172.21.15.71:0/1620785277 >> 172.21.15.71:0/1620785277 conn(0x7ff66c0fc750 msgr2=0x7ff66c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:53:37.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.627+0000 7ff6729cd700 1 -- 172.21.15.71:0/1620785277 shutdown_connections 2024-04-23T02:53:37.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.627+0000 7ff6729cd700 1 -- 172.21.15.71:0/1620785277 wait complete. 2024-04-23T02:53:37.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.627+0000 7ff6729cd700 1 Processor -- start 2024-04-23T02:53:37.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.627+0000 7ff6729cd700 1 -- start start 2024-04-23T02:53:37.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.628+0000 7ff6729cd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff66c100ea0 0x7ff66c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:37.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.628+0000 7ff6729cd700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff66c101850 0x7ff66c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:37.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.628+0000 7ff6729cd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c19d1f0 con 0x7ff66c100ea0 2024-04-23T02:53:37.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.628+0000 7ff6729cd700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c1966a0 con 0x7ff66c101850 2024-04-23T02:53:37.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.628+0000 7ff66bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff66c100ea0 0x7ff66c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:37.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.628+0000 7ff66bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff66c100ea0 0x7ff66c19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60568/0 (socket says 172.21.15.71:60568) 2024-04-23T02:53:37.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.628+0000 7ff66bfff700 1 -- 172.21.15.71:0/220055545 learned_addr learned my addr 172.21.15.71:0/220055545 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:53:37.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.628+0000 7ff66b7fe700 1 --2- 172.21.15.71:0/220055545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff66c101850 0x7ff66c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.628+0000 7ff66bfff700 1 -- 172.21.15.71:0/220055545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff66c101850 msgr2=0x7ff66c19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.628+0000 7ff66bfff700 1 --2- 172.21.15.71:0/220055545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff66c101850 0x7ff66c19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.628+0000 7ff66bfff700 1 -- 172.21.15.71:0/220055545 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c0095f0 con 0x7ff66c100ea0 2024-04-23T02:53:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.628+0000 7ff66b7fe700 1 --2- 172.21.15.71:0/220055545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff66c101850 0x7ff66c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:53:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.628+0000 7ff66bfff700 1 --2- 172.21.15.71:0/220055545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff66c100ea0 0x7ff66c19c620 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7ff66000c8a0 tx=0x7ff66000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:53:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.629+0000 7ff6697fa700 1 -- 172.21.15.71:0/220055545 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff66000cea0 con 0x7ff66c100ea0 2024-04-23T02:53:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.629+0000 7ff6697fa700 1 -- 172.21.15.71:0/220055545 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff660004830 con 0x7ff66c100ea0 2024-04-23T02:53:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.629+0000 7ff6729cd700 1 -- 172.21.15.71:0/220055545 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66c196980 con 0x7ff66c100ea0 2024-04-23T02:53:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.629+0000 7ff6697fa700 1 -- 172.21.15.71:0/220055545 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff660005500 con 0x7ff66c100ea0 2024-04-23T02:53:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.629+0000 7ff6729cd700 1 -- 172.21.15.71:0/220055545 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66c196ed0 con 0x7ff66c100ea0 2024-04-23T02:53:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.630+0000 7ff6729cd700 1 -- 172.21.15.71:0/220055545 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff66c1092a0 con 0x7ff66c100ea0 2024-04-23T02:53:37.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.631+0000 7ff6697fa700 1 -- 172.21.15.71:0/220055545 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff6600049a0 con 0x7ff66c100ea0 2024-04-23T02:53:37.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.631+0000 7ff6697fa700 1 --2- 172.21.15.71:0/220055545 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff65406cce0 0x7ff65406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:37.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.631+0000 7ff6697fa700 1 -- 172.21.15.71:0/220055545 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff66008c640 con 0x7ff66c100ea0 2024-04-23T02:53:37.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.631+0000 7ff66b7fe700 1 --2- 172.21.15.71:0/220055545 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff65406cce0 0x7ff65406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:37.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.632+0000 7ff66b7fe700 1 --2- 172.21.15.71:0/220055545 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff65406cce0 0x7ff65406f1a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0fcfc0 tx=0x7ff65c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:53:37.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.635+0000 7ff6697fa700 1 -- 172.21.15.71:0/220055545 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff66005a260 con 0x7ff66c100ea0 2024-04-23T02:53:37.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.847+0000 7ff6729cd700 1 -- 172.21.15.71:0/220055545 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff66c197150 con 0x7ff65406cce0 2024-04-23T02:53:37.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.848+0000 7ff6697fa700 1 -- 172.21.15.71:0/220055545 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7ff66c197150 con 0x7ff65406cce0 2024-04-23T02:53:37.851 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:53:37.851 INFO:teuthology.orchestra.run.smithi071.stdout: "target_image": null, 2024-04-23T02:53:37.851 INFO:teuthology.orchestra.run.smithi071.stdout: "in_progress": false, 2024-04-23T02:53:37.851 INFO:teuthology.orchestra.run.smithi071.stdout: "which": "", 2024-04-23T02:53:37.851 INFO:teuthology.orchestra.run.smithi071.stdout: "services_complete": [], 2024-04-23T02:53:37.851 INFO:teuthology.orchestra.run.smithi071.stdout: "progress": null, 2024-04-23T02:53:37.851 INFO:teuthology.orchestra.run.smithi071.stdout: "message": "", 2024-04-23T02:53:37.851 INFO:teuthology.orchestra.run.smithi071.stdout: "is_paused": false 2024-04-23T02:53:37.851 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:53:37.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.850+0000 7ff6729cd700 1 -- 172.21.15.71:0/220055545 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff65406cce0 msgr2=0x7ff65406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:37.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.850+0000 7ff6729cd700 1 --2- 172.21.15.71:0/220055545 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff65406cce0 0x7ff65406f1a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0fcfc0 tx=0x7ff65c01a040 comp rx=0 tx=0).stop 2024-04-23T02:53:37.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.850+0000 7ff6729cd700 1 -- 172.21.15.71:0/220055545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff66c100ea0 msgr2=0x7ff66c19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:37.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.850+0000 7ff6729cd700 1 --2- 172.21.15.71:0/220055545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff66c100ea0 0x7ff66c19c620 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7ff66000c8a0 tx=0x7ff66000cc60 comp rx=0 tx=0).stop 2024-04-23T02:53:37.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.850+0000 7ff6729cd700 1 -- 172.21.15.71:0/220055545 shutdown_connections 2024-04-23T02:53:37.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.850+0000 7ff6729cd700 1 --2- 172.21.15.71:0/220055545 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff65406cce0 0x7ff65406f1a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:37.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.850+0000 7ff6729cd700 1 --2- 172.21.15.71:0/220055545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff66c100ea0 0x7ff66c19c620 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:37.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.850+0000 7ff6729cd700 1 --2- 172.21.15.71:0/220055545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff66c101850 0x7ff66c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:37.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.850+0000 7ff6729cd700 1 -- 172.21.15.71:0/220055545 >> 172.21.15.71:0/220055545 conn(0x7ff66c0fc750 msgr2=0x7ff66c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:53:37.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.851+0000 7ff6729cd700 1 -- 172.21.15.71:0/220055545 shutdown_connections 2024-04-23T02:53:37.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:37.851+0000 7ff6729cd700 1 -- 172.21.15.71:0/220055545 wait complete. 2024-04-23T02:53:38.453 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph health detail' 2024-04-23T02:53:38.680 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:53:38.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:38 smithi071 ceph-mon[144544]: pgmap v1212: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:53:38.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:38 smithi192 ceph-mon[108617]: pgmap v1212: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:53:39.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:39 smithi071 ceph-mon[144544]: from='client.35978 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:53:39.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:39 smithi192 ceph-mon[108617]: from='client.35978 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:53:40.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.373+0000 7fbb23205700 1 -- 172.21.15.71:0/408587180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb1c105bd0 msgr2=0x7fbb1c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:40.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.373+0000 7fbb23205700 1 --2- 172.21.15.71:0/408587180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb1c105bd0 0x7fbb1c105fb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009910 tx=0x7fbb10009c20 comp rx=0 tx=0).stop 2024-04-23T02:53:40.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.373+0000 7fbb23205700 1 -- 172.21.15.71:0/408587180 shutdown_connections 2024-04-23T02:53:40.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.373+0000 7fbb23205700 1 --2- 172.21.15.71:0/408587180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb1c0ffc20 0x7fbb1c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:40.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.373+0000 7fbb23205700 1 --2- 172.21.15.71:0/408587180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb1c105bd0 0x7fbb1c105fb0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:40.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.373+0000 7fbb23205700 1 -- 172.21.15.71:0/408587180 >> 172.21.15.71:0/408587180 conn(0x7fbb1c075670 msgr2=0x7fbb1c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:53:40.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.373+0000 7fbb23205700 1 -- 172.21.15.71:0/408587180 shutdown_connections 2024-04-23T02:53:40.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.373+0000 7fbb23205700 1 -- 172.21.15.71:0/408587180 wait complete. 2024-04-23T02:53:40.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.374+0000 7fbb23205700 1 Processor -- start 2024-04-23T02:53:40.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.374+0000 7fbb23205700 1 -- start start 2024-04-23T02:53:40.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.374+0000 7fbb23205700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb1c0ffc20 0x7fbb1c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:40.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.374+0000 7fbb23205700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb1c105bd0 0x7fbb1c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:40.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.374+0000 7fbb23205700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c198140 con 0x7fbb1c0ffc20 2024-04-23T02:53:40.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.374+0000 7fbb23205700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1982b0 con 0x7fbb1c105bd0 2024-04-23T02:53:40.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.374+0000 7fbb20fa1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb1c0ffc20 0x7fbb1c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:40.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.374+0000 7fbb20fa1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb1c0ffc20 0x7fbb1c1975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60586/0 (socket says 172.21.15.71:60586) 2024-04-23T02:53:40.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.374+0000 7fbb20fa1700 1 -- 172.21.15.71:0/2074020059 learned_addr learned my addr 172.21.15.71:0/2074020059 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:53:40.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.375+0000 7fbb1bfff700 1 --2- 172.21.15.71:0/2074020059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb1c105bd0 0x7fbb1c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:40.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.375+0000 7fbb20fa1700 1 -- 172.21.15.71:0/2074020059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb1c105bd0 msgr2=0x7fbb1c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:40.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.375+0000 7fbb20fa1700 1 --2- 172.21.15.71:0/2074020059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb1c105bd0 0x7fbb1c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:40.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.375+0000 7fbb20fa1700 1 -- 172.21.15.71:0/2074020059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb100095f0 con 0x7fbb1c0ffc20 2024-04-23T02:53:40.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.375+0000 7fbb1bfff700 1 --2- 172.21.15.71:0/2074020059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb1c105bd0 0x7fbb1c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:53:40.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.375+0000 7fbb20fa1700 1 --2- 172.21.15.71:0/2074020059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb1c0ffc20 0x7fbb1c1975e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fbb10006010 tx=0x7fbb1000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:53:40.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.375+0000 7fbb19ffb700 1 -- 172.21.15.71:0/2074020059 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb1001d070 con 0x7fbb1c0ffc20 2024-04-23T02:53:40.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.375+0000 7fbb19ffb700 1 -- 172.21.15.71:0/2074020059 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbb1000f460 con 0x7fbb1c0ffc20 2024-04-23T02:53:40.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.375+0000 7fbb23205700 1 -- 172.21.15.71:0/2074020059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1c1a0650 con 0x7fbb1c0ffc20 2024-04-23T02:53:40.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.375+0000 7fbb19ffb700 1 -- 172.21.15.71:0/2074020059 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb10021640 con 0x7fbb1c0ffc20 2024-04-23T02:53:40.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.375+0000 7fbb23205700 1 -- 172.21.15.71:0/2074020059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c1a0ac0 con 0x7fbb1c0ffc20 2024-04-23T02:53:40.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.376+0000 7fbb23205700 1 -- 172.21.15.71:0/2074020059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1c1080f0 con 0x7fbb1c0ffc20 2024-04-23T02:53:40.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.377+0000 7fbb19ffb700 1 -- 172.21.15.71:0/2074020059 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbb1002b440 con 0x7fbb1c0ffc20 2024-04-23T02:53:40.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.377+0000 7fbb19ffb700 1 --2- 172.21.15.71:0/2074020059 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbb0c06cce0 0x7fbb0c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:40.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.377+0000 7fbb19ffb700 1 -- 172.21.15.71:0/2074020059 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbb1008d7f0 con 0x7fbb1c0ffc20 2024-04-23T02:53:40.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.377+0000 7fbb1bfff700 1 --2- 172.21.15.71:0/2074020059 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbb0c06cce0 0x7fbb0c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:40.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.377+0000 7fbb1bfff700 1 --2- 172.21.15.71:0/2074020059 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbb0c06cce0 0x7fbb0c06f1a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbb08005fd0 tx=0x7fbb08005e80 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:53:40.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.381+0000 7fbb19ffb700 1 -- 172.21.15.71:0/2074020059 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbb10057a30 con 0x7fbb1c0ffc20 2024-04-23T02:53:40.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.656+0000 7fbb23205700 1 -- 172.21.15.71:0/2074020059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbb1c068c90 con 0x7fbb1c0ffc20 2024-04-23T02:53:40.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.656+0000 7fbb19ffb700 1 -- 172.21.15.71:0/2074020059 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fbb1005b050 con 0x7fbb1c0ffc20 2024-04-23T02:53:40.659 INFO:teuthology.orchestra.run.smithi071.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-23T02:53:40.659 INFO:teuthology.orchestra.run.smithi071.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T02:53:40.659 INFO:teuthology.orchestra.run.smithi071.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T02:53:40.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.658+0000 7fbb23205700 1 -- 172.21.15.71:0/2074020059 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbb0c06cce0 msgr2=0x7fbb0c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:40.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.658+0000 7fbb23205700 1 --2- 172.21.15.71:0/2074020059 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbb0c06cce0 0x7fbb0c06f1a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbb08005fd0 tx=0x7fbb08005e80 comp rx=0 tx=0).stop 2024-04-23T02:53:40.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.658+0000 7fbb23205700 1 -- 172.21.15.71:0/2074020059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb1c0ffc20 msgr2=0x7fbb1c1975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:40.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.658+0000 7fbb23205700 1 --2- 172.21.15.71:0/2074020059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb1c0ffc20 0x7fbb1c1975e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fbb10006010 tx=0x7fbb1000be20 comp rx=0 tx=0).stop 2024-04-23T02:53:40.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.658+0000 7fbb23205700 1 -- 172.21.15.71:0/2074020059 shutdown_connections 2024-04-23T02:53:40.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.658+0000 7fbb23205700 1 --2- 172.21.15.71:0/2074020059 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbb0c06cce0 0x7fbb0c06f1a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:40.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.658+0000 7fbb23205700 1 --2- 172.21.15.71:0/2074020059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb1c0ffc20 0x7fbb1c1975e0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:40.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.658+0000 7fbb23205700 1 --2- 172.21.15.71:0/2074020059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb1c105bd0 0x7fbb1c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:40.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.658+0000 7fbb23205700 1 -- 172.21.15.71:0/2074020059 >> 172.21.15.71:0/2074020059 conn(0x7fbb1c075670 msgr2=0x7fbb1c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:53:40.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.658+0000 7fbb23205700 1 -- 172.21.15.71:0/2074020059 shutdown_connections 2024-04-23T02:53:40.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:40.658+0000 7fbb23205700 1 -- 172.21.15.71:0/2074020059 wait complete. 2024-04-23T02:53:40.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:40 smithi071 ceph-mon[144544]: pgmap v1213: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:53:40.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:40 smithi192 ceph-mon[108617]: pgmap v1213: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:53:41.318 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph versions' 2024-04-23T02:53:41.552 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:53:41.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:41 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2074020059' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:53:41.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:41 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2074020059' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-23T02:53:42.632 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:42 smithi071 ceph-mon[144544]: pgmap v1214: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:53:42.755 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:42 smithi192 ceph-mon[108617]: pgmap v1214: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:53:43.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.177+0000 7f252961e700 1 -- 172.21.15.71:0/1732036522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25240fff50 msgr2=0x7f2524105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:43.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.177+0000 7f252961e700 1 --2- 172.21.15.71:0/1732036522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25240fff50 0x7f2524105840 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2518009910 tx=0x7f2518009c20 comp rx=0 tx=0).stop 2024-04-23T02:53:43.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.177+0000 7f252961e700 1 -- 172.21.15.71:0/1732036522 shutdown_connections 2024-04-23T02:53:43.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.177+0000 7f252961e700 1 --2- 172.21.15.71:0/1732036522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25240fff50 0x7f2524105840 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:43.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.177+0000 7f252961e700 1 --2- 172.21.15.71:0/1732036522 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25240ff5a0 0x7f25240ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:43.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.177+0000 7f252961e700 1 -- 172.21.15.71:0/1732036522 >> 172.21.15.71:0/1732036522 conn(0x7f2524074e30 msgr2=0x7f2524075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:53:43.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.177+0000 7f252961e700 1 -- 172.21.15.71:0/1732036522 shutdown_connections 2024-04-23T02:53:43.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.177+0000 7f252961e700 1 -- 172.21.15.71:0/1732036522 wait complete. 2024-04-23T02:53:43.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.178+0000 7f252961e700 1 Processor -- start 2024-04-23T02:53:43.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.178+0000 7f252961e700 1 -- start start 2024-04-23T02:53:43.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.178+0000 7f252961e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25240ff5a0 0x7f252419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:43.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.178+0000 7f252961e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25240fff50 0x7f252419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:43.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.178+0000 7f252961e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252419d160 con 0x7f25240ff5a0 2024-04-23T02:53:43.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.178+0000 7f252961e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2524196610 con 0x7f25240fff50 2024-04-23T02:53:43.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.179+0000 7f2522ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25240ff5a0 0x7f252419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:43.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.179+0000 7f2522ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25240ff5a0 0x7f252419c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60604/0 (socket says 172.21.15.71:60604) 2024-04-23T02:53:43.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.179+0000 7f2522ffd700 1 -- 172.21.15.71:0/3706004503 learned_addr learned my addr 172.21.15.71:0/3706004503 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:53:43.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.179+0000 7f25227fc700 1 --2- 172.21.15.71:0/3706004503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25240fff50 0x7f252419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:43.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.179+0000 7f2522ffd700 1 -- 172.21.15.71:0/3706004503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25240fff50 msgr2=0x7f252419cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:43.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.179+0000 7f2522ffd700 1 --2- 172.21.15.71:0/3706004503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25240fff50 0x7f252419cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:43.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.179+0000 7f2522ffd700 1 -- 172.21.15.71:0/3706004503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25180095f0 con 0x7f25240ff5a0 2024-04-23T02:53:43.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.179+0000 7f25227fc700 1 --2- 172.21.15.71:0/3706004503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25240fff50 0x7f252419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:53:43.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.179+0000 7f2522ffd700 1 --2- 172.21.15.71:0/3706004503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25240ff5a0 0x7f252419c590 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f251400d8d0 tx=0x7f251400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:53:43.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.179+0000 7f250bfff700 1 -- 172.21.15.71:0/3706004503 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2514009940 con 0x7f25240ff5a0 2024-04-23T02:53:43.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.179+0000 7f250bfff700 1 -- 172.21.15.71:0/3706004503 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2514010460 con 0x7f25240ff5a0 2024-04-23T02:53:43.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.179+0000 7f250bfff700 1 -- 172.21.15.71:0/3706004503 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f251400f5f0 con 0x7f25240ff5a0 2024-04-23T02:53:43.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.179+0000 7f252961e700 1 -- 172.21.15.71:0/3706004503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25241968f0 con 0x7f25240ff5a0 2024-04-23T02:53:43.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.179+0000 7f252961e700 1 -- 172.21.15.71:0/3706004503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2524196df0 con 0x7f25240ff5a0 2024-04-23T02:53:43.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.180+0000 7f252961e700 1 -- 172.21.15.71:0/3706004503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2524109200 con 0x7f25240ff5a0 2024-04-23T02:53:43.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.181+0000 7f250bfff700 1 -- 172.21.15.71:0/3706004503 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f25140105d0 con 0x7f25240ff5a0 2024-04-23T02:53:43.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.181+0000 7f250bfff700 1 --2- 172.21.15.71:0/3706004503 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f250c06cce0 0x7f250c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:53:43.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.181+0000 7f250bfff700 1 -- 172.21.15.71:0/3706004503 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f251408c5e0 con 0x7f25240ff5a0 2024-04-23T02:53:43.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.181+0000 7f25227fc700 1 --2- 172.21.15.71:0/3706004503 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f250c06cce0 0x7f250c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:53:43.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.181+0000 7f25227fc700 1 --2- 172.21.15.71:0/3706004503 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f250c06cce0 0x7f250c06f1a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f25180059d0 tx=0x7f2518005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:53:43.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.185+0000 7f250bfff700 1 -- 172.21.15.71:0/3706004503 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f251405a0e0 con 0x7f25240ff5a0 2024-04-23T02:53:43.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.458+0000 7f252961e700 1 -- 172.21.15.71:0/3706004503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2524197060 con 0x7f25240ff5a0 2024-04-23T02:53:43.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.458+0000 7f250bfff700 1 -- 172.21.15.71:0/3706004503 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f2514010880 con 0x7f25240ff5a0 2024-04-23T02:53:43.461 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:53:43.461 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:53:43.461 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:53:43.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:53:43.461 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:53:43.461 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:53:43.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:53:43.462 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:53:43.462 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:53:43.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:53:43.462 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:53:43.462 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:53:43.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:53:43.462 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:53:43.462 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:53:43.462 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:53:43.462 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:53:43.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.460+0000 7f252961e700 1 -- 172.21.15.71:0/3706004503 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f250c06cce0 msgr2=0x7f250c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:43.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.460+0000 7f252961e700 1 --2- 172.21.15.71:0/3706004503 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f250c06cce0 0x7f250c06f1a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f25180059d0 tx=0x7f2518005960 comp rx=0 tx=0).stop 2024-04-23T02:53:43.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.460+0000 7f252961e700 1 -- 172.21.15.71:0/3706004503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25240ff5a0 msgr2=0x7f252419c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:53:43.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.460+0000 7f252961e700 1 --2- 172.21.15.71:0/3706004503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25240ff5a0 0x7f252419c590 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f251400d8d0 tx=0x7f251400dc90 comp rx=0 tx=0).stop 2024-04-23T02:53:43.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.460+0000 7f252961e700 1 -- 172.21.15.71:0/3706004503 shutdown_connections 2024-04-23T02:53:43.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.460+0000 7f252961e700 1 --2- 172.21.15.71:0/3706004503 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f250c06cce0 0x7f250c06f1a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:43.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.460+0000 7f252961e700 1 --2- 172.21.15.71:0/3706004503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25240ff5a0 0x7f252419c590 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:43.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.460+0000 7f252961e700 1 --2- 172.21.15.71:0/3706004503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25240fff50 0x7f252419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:53:43.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.460+0000 7f252961e700 1 -- 172.21.15.71:0/3706004503 >> 172.21.15.71:0/3706004503 conn(0x7f2524074e30 msgr2=0x7f25240fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:53:43.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.460+0000 7f252961e700 1 -- 172.21.15.71:0/3706004503 shutdown_connections 2024-04-23T02:53:43.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:53:43.460+0000 7f252961e700 1 -- 172.21.15.71:0/3706004503 wait complete. 2024-04-23T02:53:43.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:43 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3706004503' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:43.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:43 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3706004503' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:53:44.205 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-04-23T02:53:44.433 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:53:44.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:44 smithi071 ceph-mon[144544]: pgmap v1215: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:53:44.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:44 smithi192 ceph-mon[108617]: pgmap v1215: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:53:46.207 INFO:teuthology.orchestra.run.smithi071.stdout:wait for servicemap items w/ changing names to refresh 2024-04-23T02:53:46.724 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:46 smithi071 ceph-mon[144544]: pgmap v1216: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:53:46.744 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'sleep 60' 2024-04-23T02:53:46.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:46 smithi192 ceph-mon[108617]: pgmap v1216: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:53:46.973 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:53:48.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:48 smithi071 ceph-mon[144544]: pgmap v1217: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:53:48.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:48 smithi192 ceph-mon[108617]: pgmap v1217: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:53:50.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:50 smithi071 ceph-mon[144544]: pgmap v1218: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:53:50.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:53:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:50 smithi192 ceph-mon[108617]: pgmap v1218: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:53:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:53:52.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:52 smithi071 ceph-mon[144544]: pgmap v1219: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:53:52.863 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:52 smithi192 ceph-mon[108617]: pgmap v1219: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:53:54.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:54 smithi192 ceph-mon[108617]: pgmap v1220: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:53:54.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:54 smithi071 ceph-mon[144544]: pgmap v1220: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:53:56.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:56 smithi192 ceph-mon[108617]: pgmap v1221: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:53:56.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:56 smithi071 ceph-mon[144544]: pgmap v1221: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:53:58.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:53:58 smithi192 ceph-mon[108617]: pgmap v1222: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:53:58.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:53:58 smithi071 ceph-mon[144544]: pgmap v1222: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:00.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:00 smithi192 ceph-mon[108617]: pgmap v1223: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:54:00.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:00 smithi071 ceph-mon[144544]: pgmap v1223: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:54:02.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:02 smithi192 ceph-mon[108617]: pgmap v1224: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:02.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:02 smithi071 ceph-mon[144544]: pgmap v1224: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:04.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:04 smithi192 ceph-mon[108617]: pgmap v1225: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:04.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:04 smithi071 ceph-mon[144544]: pgmap v1225: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:54:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:54:06.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:06 smithi192 ceph-mon[108617]: pgmap v1226: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:54:06.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:06 smithi071 ceph-mon[144544]: pgmap v1226: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:54:08.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:08 smithi192 ceph-mon[108617]: pgmap v1227: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:08.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:08 smithi071 ceph-mon[144544]: pgmap v1227: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:10.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:10 smithi192 ceph-mon[108617]: pgmap v1228: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:54:10.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:10 smithi071 ceph-mon[144544]: pgmap v1228: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:54:12.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:12 smithi192 ceph-mon[108617]: pgmap v1229: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:12.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:12 smithi071 ceph-mon[144544]: pgmap v1229: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:14.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:14 smithi192 ceph-mon[108617]: pgmap v1230: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:14.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:14 smithi071 ceph-mon[144544]: pgmap v1230: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:16.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:16 smithi192 ceph-mon[108617]: pgmap v1231: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:54:16.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:16 smithi071 ceph-mon[144544]: pgmap v1231: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:54:18.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:18 smithi192 ceph-mon[108617]: pgmap v1232: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:18.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:18 smithi071 ceph-mon[144544]: pgmap v1232: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:20.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:20 smithi192 ceph-mon[108617]: pgmap v1233: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:54:20.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:54:20.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:20 smithi071 ceph-mon[144544]: pgmap v1233: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:54:20.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:54:22.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:22 smithi192 ceph-mon[108617]: pgmap v1234: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:22.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:22 smithi071 ceph-mon[144544]: pgmap v1234: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:24.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:24 smithi192 ceph-mon[108617]: pgmap v1235: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:24.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:24 smithi071 ceph-mon[144544]: pgmap v1235: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:26.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:26 smithi192 ceph-mon[108617]: pgmap v1236: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:54:26.888 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:26 smithi071 ceph-mon[144544]: pgmap v1236: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:54:27.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:27 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:54:27.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:27 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:54:27.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:27 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:54:27.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:27 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:54:27.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:27 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:54:27.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:27 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:54:27.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:27 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:54:27.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:27 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:54:28.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:28 smithi192 ceph-mon[108617]: pgmap v1237: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:28.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:28 smithi071 ceph-mon[144544]: pgmap v1237: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:30.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:30 smithi192 ceph-mon[108617]: pgmap v1238: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:54:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:30 smithi071 ceph-mon[144544]: pgmap v1238: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:54:32.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:32 smithi192 ceph-mon[108617]: pgmap v1239: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:32.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:32 smithi071 ceph-mon[144544]: pgmap v1239: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:34.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:34 smithi192 ceph-mon[108617]: pgmap v1240: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:34.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:34 smithi071 ceph-mon[144544]: pgmap v1240: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:35.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:54:35.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:54:36.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:36 smithi192 ceph-mon[108617]: pgmap v1241: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:54:36.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:36 smithi071 ceph-mon[144544]: pgmap v1241: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:54:38.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:38 smithi192 ceph-mon[108617]: pgmap v1242: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:38.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:38 smithi071 ceph-mon[144544]: pgmap v1242: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:40.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:40 smithi192 ceph-mon[108617]: pgmap v1243: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:54:40.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:40 smithi071 ceph-mon[144544]: pgmap v1243: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:54:42.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:42 smithi192 ceph-mon[108617]: pgmap v1244: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:42.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:42 smithi071 ceph-mon[144544]: pgmap v1244: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:44.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:44 smithi192 ceph-mon[108617]: pgmap v1245: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:44.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:44 smithi071 ceph-mon[144544]: pgmap v1245: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:46.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:46 smithi192 ceph-mon[108617]: pgmap v1246: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:54:46.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:46 smithi071 ceph-mon[144544]: pgmap v1246: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:54:48.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:48 smithi192 ceph-mon[108617]: pgmap v1247: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:48.915 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:48 smithi071 ceph-mon[144544]: pgmap v1247: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:49.186 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph orch ps' 2024-04-23T02:54:49.418 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:54:50.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:50 smithi071 ceph-mon[144544]: pgmap v1248: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:54:50.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:54:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:50 smithi192 ceph-mon[108617]: pgmap v1248: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:54:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:54:51.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.421+0000 7f695a95e700 1 -- 172.21.15.71:0/3212981930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6954068580 msgr2=0x7f69540689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:54:51.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.421+0000 7f695a95e700 1 --2- 172.21.15.71:0/3212981930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6954068580 0x7f69540689e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f693c009910 tx=0x7f693c009c20 comp rx=0 tx=0).stop 2024-04-23T02:54:51.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.421+0000 7f695a95e700 1 -- 172.21.15.71:0/3212981930 shutdown_connections 2024-04-23T02:54:51.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.421+0000 7f695a95e700 1 --2- 172.21.15.71:0/3212981930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6954068580 0x7f69540689e0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:51.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.421+0000 7f695a95e700 1 --2- 172.21.15.71:0/3212981930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6954106cd0 0x7f69541070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:51.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.421+0000 7f695a95e700 1 -- 172.21.15.71:0/3212981930 >> 172.21.15.71:0/3212981930 conn(0x7f6954075670 msgr2=0x7f6954075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:54:51.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.421+0000 7f695a95e700 1 -- 172.21.15.71:0/3212981930 shutdown_connections 2024-04-23T02:54:51.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.421+0000 7f695a95e700 1 -- 172.21.15.71:0/3212981930 wait complete. 2024-04-23T02:54:51.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.422+0000 7f695a95e700 1 Processor -- start 2024-04-23T02:54:51.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.422+0000 7f695a95e700 1 -- start start 2024-04-23T02:54:51.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.422+0000 7f695a95e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6954068580 0x7f695419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:54:51.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.422+0000 7f695a95e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6954106cd0 0x7f695419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:54:51.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.422+0000 7f695a95e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695419d290 con 0x7f6954068580 2024-04-23T02:54:51.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.422+0000 7f695a95e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6954196740 con 0x7f6954106cd0 2024-04-23T02:54:51.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.422+0000 7f6953fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6954068580 0x7f695419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:54:51.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.422+0000 7f6953fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6954068580 0x7f695419c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34446/0 (socket says 172.21.15.71:34446) 2024-04-23T02:54:51.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.422+0000 7f6953fff700 1 -- 172.21.15.71:0/4060543187 learned_addr learned my addr 172.21.15.71:0/4060543187 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:54:51.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.422+0000 7f694bfff700 1 --2- 172.21.15.71:0/4060543187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6954106cd0 0x7f695419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:54:51.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.422+0000 7f6953fff700 1 -- 172.21.15.71:0/4060543187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6954106cd0 msgr2=0x7f695419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:54:51.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.422+0000 7f6953fff700 1 --2- 172.21.15.71:0/4060543187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6954106cd0 0x7f695419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:51.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.422+0000 7f6953fff700 1 -- 172.21.15.71:0/4060543187 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693c0095f0 con 0x7f6954068580 2024-04-23T02:54:51.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.422+0000 7f694bfff700 1 --2- 172.21.15.71:0/4060543187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6954106cd0 0x7f695419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:54:51.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.423+0000 7f6953fff700 1 --2- 172.21.15.71:0/4060543187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6954068580 0x7f695419c6c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f694400b4b0 tx=0x7f694400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:54:51.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.423+0000 7f6951ffb700 1 -- 172.21.15.71:0/4060543187 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6944011800 con 0x7f6954068580 2024-04-23T02:54:51.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.423+0000 7f6951ffb700 1 -- 172.21.15.71:0/4060543187 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6944011e00 con 0x7f6954068580 2024-04-23T02:54:51.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.423+0000 7f695a95e700 1 -- 172.21.15.71:0/4060543187 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6954196a20 con 0x7f6954068580 2024-04-23T02:54:51.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.423+0000 7f6951ffb700 1 -- 172.21.15.71:0/4060543187 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6944010e50 con 0x7f6954068580 2024-04-23T02:54:51.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.423+0000 7f695a95e700 1 -- 172.21.15.71:0/4060543187 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6954196f70 con 0x7f6954068580 2024-04-23T02:54:51.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.424+0000 7f695a95e700 1 -- 172.21.15.71:0/4060543187 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69541091d0 con 0x7f6954068580 2024-04-23T02:54:51.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.424+0000 7f6951ffb700 1 -- 172.21.15.71:0/4060543187 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6944010380 con 0x7f6954068580 2024-04-23T02:54:51.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.424+0000 7f6951ffb700 1 --2- 172.21.15.71:0/4060543187 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f694006cce0 0x7f694006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:54:51.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.425+0000 7f6951ffb700 1 -- 172.21.15.71:0/4060543187 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f694408c150 con 0x7f6954068580 2024-04-23T02:54:51.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.425+0000 7f694bfff700 1 --2- 172.21.15.71:0/4060543187 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f694006cce0 0x7f694006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:54:51.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.425+0000 7f694bfff700 1 --2- 172.21.15.71:0/4060543187 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f694006cce0 0x7f694006f1a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f693c006010 tx=0x7f693c00b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:54:51.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.429+0000 7f6951ffb700 1 -- 172.21.15.71:0/4060543187 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6944059c50 con 0x7f6954068580 2024-04-23T02:54:51.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.632+0000 7f695a95e700 1 -- 172.21.15.71:0/4060543187 --> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f69541971f0 con 0x7f694006cce0 2024-04-23T02:54:51.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.642+0000 7f6951ffb700 1 -- 172.21.15.71:0/4060543187 <== mgr.14660 v2:172.21.15.71:6800/3011237878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f69541971f0 con 0x7f694006cce0 2024-04-23T02:54:51.646 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T02:54:51.647 INFO:teuthology.orchestra.run.smithi071.stdout:alertmanager.smithi071 smithi071 *:9093,9094 running (38m) 92s ago 49m 26.4M - 0.25.0 c8568f914cd2 2cc485ac58af 2024-04-23T02:54:51.647 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi071 smithi071 running (106s) 92s ago 50m 7335k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 53cec9e97349 2024-04-23T02:54:51.647 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-exporter.smithi192 smithi192 running (99s) 96s ago 48m 6887k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 78dbe1b6f2cd 2024-04-23T02:54:51.647 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi071 smithi071 running (36m) 92s ago 50m 6647k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 b1b5a5c4d92e 2024-04-23T02:54:51.647 INFO:teuthology.orchestra.run.smithi071.stdout:crash.smithi192 smithi192 running (36m) 96s ago 48m 6656k - 18.2.2-1237-g646b8c7d 9900ed1a6b81 466ee26af472 2024-04-23T02:54:51.647 INFO:teuthology.orchestra.run.smithi071.stdout:grafana.smithi071 smithi071 *:3000 running (38m) 92s ago 49m 88.8M - 9.4.7 954c08fa6188 637b82232857 2024-04-23T02:54:51.648 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.qusrrb smithi071 running (32m) 92s ago 43m 65.6M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 85323247ba2e 2024-04-23T02:54:51.648 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi071.vxejye smithi071 running (32m) 92s ago 43m 66.0M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 0588feb8347e 2024-04-23T02:54:51.648 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.azdyze smithi192 running (16m) 96s ago 43m 195M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 30ad708231fd 2024-04-23T02:54:51.648 INFO:teuthology.orchestra.run.smithi071.stdout:mds.cephfs.smithi192.gtmyst smithi192 running (32m) 96s ago 43m 184M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 e73cd2742a32 2024-04-23T02:54:51.648 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi071.ukdjvn smithi071 *:8443,9283,8765 running (40m) 92s ago 51m 517M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 cc36414706a8 2024-04-23T02:54:51.649 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.smithi192.qgecnv smithi192 *:8443,9283,8765 running (39m) 96s ago 48m 460M - 18.2.2-1237-g646b8c7d 9900ed1a6b81 093e8d97851a 2024-04-23T02:54:51.649 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi071 smithi071 running (36m) 92s ago 51m 85.3M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16378807fa69 2024-04-23T02:54:51.649 INFO:teuthology.orchestra.run.smithi071.stdout:mon.smithi192 smithi192 running (36m) 96s ago 48m 60.2M 2048M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a25130f1b27d 2024-04-23T02:54:51.649 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi071 smithi071 *:9100 running (39m) 92s ago 49m 22.0M - 1.5.0 0da6a335fe13 5908466b38a2 2024-04-23T02:54:51.649 INFO:teuthology.orchestra.run.smithi071.stdout:node-exporter.smithi192 smithi192 *:9100 running (39m) 96s ago 48m 18.5M - 1.5.0 0da6a335fe13 63949b1d70eb 2024-04-23T02:54:51.649 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (36m) 92s ago 47m 101M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 6ada0e60b90b 2024-04-23T02:54:51.649 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi071 running (35m) 92s ago 47m 103M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 a626640cb50f 2024-04-23T02:54:51.650 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi071 running (34m) 92s ago 46m 91.9M 1227M 18.2.2-1237-g646b8c7d 9900ed1a6b81 7e44c427695b 2024-04-23T02:54:51.650 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi192 running (34m) 96s ago 46m 120M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 aa11be9f1148 2024-04-23T02:54:51.650 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi192 running (33m) 96s ago 45m 101M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 c4eff10fbc30 2024-04-23T02:54:51.650 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi192 running (33m) 96s ago 45m 93.4M 2251M 18.2.2-1237-g646b8c7d 9900ed1a6b81 16449d741485 2024-04-23T02:54:51.650 INFO:teuthology.orchestra.run.smithi071.stdout:prometheus.smithi071 smithi071 *:9095 running (38m) 92s ago 49m 93.3M - 2.43.0 a07b618ecd1d ad49eb456bac 2024-04-23T02:54:51.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.645+0000 7f695a95e700 1 -- 172.21.15.71:0/4060543187 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f694006cce0 msgr2=0x7f694006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:54:51.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.645+0000 7f695a95e700 1 --2- 172.21.15.71:0/4060543187 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f694006cce0 0x7f694006f1a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f693c006010 tx=0x7f693c00b540 comp rx=0 tx=0).stop 2024-04-23T02:54:51.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.645+0000 7f695a95e700 1 -- 172.21.15.71:0/4060543187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6954068580 msgr2=0x7f695419c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:54:51.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.645+0000 7f695a95e700 1 --2- 172.21.15.71:0/4060543187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6954068580 0x7f695419c6c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f694400b4b0 tx=0x7f694400b870 comp rx=0 tx=0).stop 2024-04-23T02:54:51.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.645+0000 7f695a95e700 1 -- 172.21.15.71:0/4060543187 shutdown_connections 2024-04-23T02:54:51.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.645+0000 7f695a95e700 1 --2- 172.21.15.71:0/4060543187 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f694006cce0 0x7f694006f1a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:51.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.645+0000 7f695a95e700 1 --2- 172.21.15.71:0/4060543187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6954068580 0x7f695419c6c0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:51.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.645+0000 7f695a95e700 1 --2- 172.21.15.71:0/4060543187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6954106cd0 0x7f695419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:51.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.645+0000 7f695a95e700 1 -- 172.21.15.71:0/4060543187 >> 172.21.15.71:0/4060543187 conn(0x7f6954075670 msgr2=0x7f69540fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:54:51.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.645+0000 7f695a95e700 1 -- 172.21.15.71:0/4060543187 shutdown_connections 2024-04-23T02:54:51.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:51.645+0000 7f695a95e700 1 -- 172.21.15.71:0/4060543187 wait complete. 2024-04-23T02:54:52.230 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph versions' 2024-04-23T02:54:52.458 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:54:52.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:52 smithi192 ceph-mon[108617]: pgmap v1249: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:52.876 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:52 smithi192 ceph-mon[108617]: from='client.35990 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:54:52.962 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:52 smithi071 ceph-mon[144544]: pgmap v1249: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:52.962 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:52 smithi071 ceph-mon[144544]: from='client.35990 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T02:54:54.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.172+0000 7ff796281700 1 -- 172.21.15.71:0/4142611413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff790105bf0 msgr2=0x7ff790105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:54:54.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.172+0000 7ff796281700 1 --2- 172.21.15.71:0/4142611413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff790105bf0 0x7ff790105fd0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff780009910 tx=0x7ff780009c20 comp rx=0 tx=0).stop 2024-04-23T02:54:54.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.172+0000 7ff796281700 1 -- 172.21.15.71:0/4142611413 shutdown_connections 2024-04-23T02:54:54.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.172+0000 7ff796281700 1 --2- 172.21.15.71:0/4142611413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7900ffb40 0x7ff7900fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:54.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.172+0000 7ff796281700 1 --2- 172.21.15.71:0/4142611413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff790105bf0 0x7ff790105fd0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:54.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.172+0000 7ff796281700 1 -- 172.21.15.71:0/4142611413 >> 172.21.15.71:0/4142611413 conn(0x7ff7900fb650 msgr2=0x7ff7900fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:54:54.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.172+0000 7ff796281700 1 -- 172.21.15.71:0/4142611413 shutdown_connections 2024-04-23T02:54:54.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.172+0000 7ff796281700 1 -- 172.21.15.71:0/4142611413 wait complete. 2024-04-23T02:54:54.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.173+0000 7ff796281700 1 Processor -- start 2024-04-23T02:54:54.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.173+0000 7ff796281700 1 -- start start 2024-04-23T02:54:54.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.173+0000 7ff796281700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7900ffb40 0x7ff7901976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:54:54.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.173+0000 7ff796281700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff790105bf0 0x7ff790197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:54:54.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.173+0000 7ff796281700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff790198230 con 0x7ff7900ffb40 2024-04-23T02:54:54.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.173+0000 7ff796281700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff790198370 con 0x7ff790105bf0 2024-04-23T02:54:54.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.173+0000 7ff78ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7900ffb40 0x7ff7901976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:54:54.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.173+0000 7ff78ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7900ffb40 0x7ff7901976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34472/0 (socket says 172.21.15.71:34472) 2024-04-23T02:54:54.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.173+0000 7ff78ffff700 1 -- 172.21.15.71:0/396477536 learned_addr learned my addr 172.21.15.71:0/396477536 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:54:54.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.173+0000 7ff78f7fe700 1 --2- 172.21.15.71:0/396477536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff790105bf0 0x7ff790197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:54:54.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.173+0000 7ff78ffff700 1 -- 172.21.15.71:0/396477536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff790105bf0 msgr2=0x7ff790197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:54:54.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.173+0000 7ff78ffff700 1 --2- 172.21.15.71:0/396477536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff790105bf0 0x7ff790197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:54.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.173+0000 7ff78ffff700 1 -- 172.21.15.71:0/396477536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7800095f0 con 0x7ff7900ffb40 2024-04-23T02:54:54.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.174+0000 7ff78f7fe700 1 --2- 172.21.15.71:0/396477536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff790105bf0 0x7ff790197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:54:54.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.174+0000 7ff78ffff700 1 --2- 172.21.15.71:0/396477536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7900ffb40 0x7ff7901976d0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff780006010 tx=0x7ff78000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:54:54.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.174+0000 7ff78d7fa700 1 -- 172.21.15.71:0/396477536 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff78001d070 con 0x7ff7900ffb40 2024-04-23T02:54:54.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.174+0000 7ff78d7fa700 1 -- 172.21.15.71:0/396477536 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff78000f460 con 0x7ff7900ffb40 2024-04-23T02:54:54.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.174+0000 7ff796281700 1 -- 172.21.15.71:0/396477536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7901a0740 con 0x7ff7900ffb40 2024-04-23T02:54:54.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.174+0000 7ff78d7fa700 1 -- 172.21.15.71:0/396477536 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff780021640 con 0x7ff7900ffb40 2024-04-23T02:54:54.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.174+0000 7ff796281700 1 -- 172.21.15.71:0/396477536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7901a0b50 con 0x7ff7900ffb40 2024-04-23T02:54:54.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.175+0000 7ff796281700 1 -- 172.21.15.71:0/396477536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff790108020 con 0x7ff7900ffb40 2024-04-23T02:54:54.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.175+0000 7ff78d7fa700 1 -- 172.21.15.71:0/396477536 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff78000fa60 con 0x7ff7900ffb40 2024-04-23T02:54:54.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.176+0000 7ff78d7fa700 1 --2- 172.21.15.71:0/396477536 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff77806cce0 0x7ff77806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:54:54.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.176+0000 7ff78d7fa700 1 -- 172.21.15.71:0/396477536 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff78008d8e0 con 0x7ff7900ffb40 2024-04-23T02:54:54.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.176+0000 7ff78f7fe700 1 --2- 172.21.15.71:0/396477536 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff77806cce0 0x7ff77806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:54:54.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.176+0000 7ff78f7fe700 1 --2- 172.21.15.71:0/396477536 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff77806cce0 0x7ff77806f1a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff7900696e0 tx=0x7ff784008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:54:54.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.180+0000 7ff78d7fa700 1 -- 172.21.15.71:0/396477536 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff7800578f0 con 0x7ff7900ffb40 2024-04-23T02:54:54.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.457+0000 7ff796281700 1 -- 172.21.15.71:0/396477536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff79004f2e0 con 0x7ff7900ffb40 2024-04-23T02:54:54.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.457+0000 7ff78d7fa700 1 -- 172.21.15.71:0/396477536 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7ff78005af10 con 0x7ff7900ffb40 2024-04-23T02:54:54.461 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-23T02:54:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": { 2024-04-23T02:54:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:54:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:54:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": { 2024-04-23T02:54:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 2 2024-04-23T02:54:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:54:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": { 2024-04-23T02:54:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 6 2024-04-23T02:54:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:54:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-23T02:54:54.461 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 4 2024-04-23T02:54:54.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-23T02:54:54.462 INFO:teuthology.orchestra.run.smithi071.stdout: "overall": { 2024-04-23T02:54:54.462 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)": 14 2024-04-23T02:54:54.462 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-23T02:54:54.462 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-23T02:54:54.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.460+0000 7ff796281700 1 -- 172.21.15.71:0/396477536 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff77806cce0 msgr2=0x7ff77806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:54:54.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.460+0000 7ff796281700 1 --2- 172.21.15.71:0/396477536 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff77806cce0 0x7ff77806f1a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff7900696e0 tx=0x7ff784008040 comp rx=0 tx=0).stop 2024-04-23T02:54:54.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.460+0000 7ff796281700 1 -- 172.21.15.71:0/396477536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7900ffb40 msgr2=0x7ff7901976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:54:54.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.460+0000 7ff796281700 1 --2- 172.21.15.71:0/396477536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7900ffb40 0x7ff7901976d0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff780006010 tx=0x7ff78000be20 comp rx=0 tx=0).stop 2024-04-23T02:54:54.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.460+0000 7ff796281700 1 -- 172.21.15.71:0/396477536 shutdown_connections 2024-04-23T02:54:54.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.460+0000 7ff796281700 1 --2- 172.21.15.71:0/396477536 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff77806cce0 0x7ff77806f1a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:54.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.460+0000 7ff796281700 1 --2- 172.21.15.71:0/396477536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7900ffb40 0x7ff7901976d0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:54.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.460+0000 7ff796281700 1 --2- 172.21.15.71:0/396477536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff790105bf0 0x7ff790197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:54.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.460+0000 7ff796281700 1 -- 172.21.15.71:0/396477536 >> 172.21.15.71:0/396477536 conn(0x7ff7900fb650 msgr2=0x7ff7900fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:54:54.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.460+0000 7ff796281700 1 -- 172.21.15.71:0/396477536 shutdown_connections 2024-04-23T02:54:54.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:54.460+0000 7ff796281700 1 -- 172.21.15.71:0/396477536 wait complete. 2024-04-23T02:54:54.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:54 smithi071 ceph-mon[144544]: pgmap v1250: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:54.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:54 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/396477536' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:54:54.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:54 smithi192 ceph-mon[108617]: pgmap v1250: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:54.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:54 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/396477536' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:54:55.107 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-04-23T02:54:55.336 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:54:56.793 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:56 smithi071 ceph-mon[144544]: pgmap v1251: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:54:56.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:56 smithi192 ceph-mon[108617]: pgmap v1251: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:54:57.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.035+0000 7f9d7af71700 1 -- 172.21.15.71:0/3804902389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d740ffc20 msgr2=0x7f9d74100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:54:57.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.035+0000 7f9d7af71700 1 --2- 172.21.15.71:0/3804902389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d740ffc20 0x7f9d74100080 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009910 tx=0x7f9d60009c20 comp rx=0 tx=0).stop 2024-04-23T02:54:57.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.035+0000 7f9d7af71700 1 -- 172.21.15.71:0/3804902389 shutdown_connections 2024-04-23T02:54:57.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.035+0000 7f9d7af71700 1 --2- 172.21.15.71:0/3804902389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d740ffc20 0x7f9d74100080 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:57.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.035+0000 7f9d7af71700 1 --2- 172.21.15.71:0/3804902389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d74105bd0 0x7f9d74105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:57.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.035+0000 7f9d7af71700 1 -- 172.21.15.71:0/3804902389 >> 172.21.15.71:0/3804902389 conn(0x7f9d74075670 msgr2=0x7f9d74075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:54:57.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.035+0000 7f9d7af71700 1 -- 172.21.15.71:0/3804902389 shutdown_connections 2024-04-23T02:54:57.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.035+0000 7f9d7af71700 1 -- 172.21.15.71:0/3804902389 wait complete. 2024-04-23T02:54:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.036+0000 7f9d7af71700 1 Processor -- start 2024-04-23T02:54:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.036+0000 7f9d7af71700 1 -- start start 2024-04-23T02:54:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.036+0000 7f9d7af71700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d740ffc20 0x7f9d741976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:54:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.036+0000 7f9d7af71700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d74105bd0 0x7f9d74197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:54:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.036+0000 7f9d7af71700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d74198220 con 0x7f9d74105bd0 2024-04-23T02:54:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.036+0000 7f9d7af71700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d74198360 con 0x7f9d740ffc20 2024-04-23T02:54:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.036+0000 7f9d73fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d74105bd0 0x7f9d74197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:54:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.036+0000 7f9d73fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d74105bd0 0x7f9d74197c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55692/0 (socket says 172.21.15.71:55692) 2024-04-23T02:54:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.036+0000 7f9d73fff700 1 -- 172.21.15.71:0/583598863 learned_addr learned my addr 172.21.15.71:0/583598863 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:54:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.037+0000 7f9d73fff700 1 -- 172.21.15.71:0/583598863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d740ffc20 msgr2=0x7f9d741976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:54:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.037+0000 7f9d78d0d700 1 --2- 172.21.15.71:0/583598863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d740ffc20 0x7f9d741976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:54:57.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.037+0000 7f9d73fff700 1 --2- 172.21.15.71:0/583598863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d740ffc20 0x7f9d741976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:57.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.037+0000 7f9d73fff700 1 -- 172.21.15.71:0/583598863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d600095f0 con 0x7f9d74105bd0 2024-04-23T02:54:57.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.037+0000 7f9d73fff700 1 --2- 172.21.15.71:0/583598863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d74105bd0 0x7f9d74197c00 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f9d60006010 tx=0x7f9d6000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:54:57.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.037+0000 7f9d71ffb700 1 -- 172.21.15.71:0/583598863 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d6001d070 con 0x7f9d74105bd0 2024-04-23T02:54:57.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.037+0000 7f9d71ffb700 1 -- 172.21.15.71:0/583598863 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9d6000f460 con 0x7f9d74105bd0 2024-04-23T02:54:57.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.037+0000 7f9d7af71700 1 -- 172.21.15.71:0/583598863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d741a0730 con 0x7f9d74105bd0 2024-04-23T02:54:57.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.037+0000 7f9d71ffb700 1 -- 172.21.15.71:0/583598863 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d60021640 con 0x7f9d74105bd0 2024-04-23T02:54:57.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.037+0000 7f9d7af71700 1 -- 172.21.15.71:0/583598863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d741a0b40 con 0x7f9d74105bd0 2024-04-23T02:54:57.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.038+0000 7f9d7af71700 1 -- 172.21.15.71:0/583598863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d741080d0 con 0x7f9d74105bd0 2024-04-23T02:54:57.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.039+0000 7f9d71ffb700 1 -- 172.21.15.71:0/583598863 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9d6000fa60 con 0x7f9d74105bd0 2024-04-23T02:54:57.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.039+0000 7f9d71ffb700 1 --2- 172.21.15.71:0/583598863 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9d6406cce0 0x7f9d6406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:54:57.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.039+0000 7f9d71ffb700 1 -- 172.21.15.71:0/583598863 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9d6008d920 con 0x7f9d74105bd0 2024-04-23T02:54:57.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.039+0000 7f9d78d0d700 1 --2- 172.21.15.71:0/583598863 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9d6406cce0 0x7f9d6406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:54:57.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.039+0000 7f9d78d0d700 1 --2- 172.21.15.71:0/583598863 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9d6406cce0 0x7f9d6406f1a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9d68005950 tx=0x7f9d68005cf0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:54:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.043+0000 7f9d71ffb700 1 -- 172.21.15.71:0/583598863 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9d600578f0 con 0x7f9d74105bd0 2024-04-23T02:54:57.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.321+0000 7f9d7af71700 1 -- 172.21.15.71:0/583598863 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9d7404f2e0 con 0x7f9d74105bd0 2024-04-23T02:54:57.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.322+0000 7f9d71ffb700 1 -- 172.21.15.71:0/583598863 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f9d6005af10 con 0x7f9d74105bd0 2024-04-23T02:54:57.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.324+0000 7f9d7af71700 1 -- 172.21.15.71:0/583598863 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9d6406cce0 msgr2=0x7f9d6406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:54:57.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.324+0000 7f9d7af71700 1 --2- 172.21.15.71:0/583598863 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9d6406cce0 0x7f9d6406f1a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9d68005950 tx=0x7f9d68005cf0 comp rx=0 tx=0).stop 2024-04-23T02:54:57.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.324+0000 7f9d7af71700 1 -- 172.21.15.71:0/583598863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d74105bd0 msgr2=0x7f9d74197c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:54:57.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.324+0000 7f9d7af71700 1 --2- 172.21.15.71:0/583598863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d74105bd0 0x7f9d74197c00 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f9d60006010 tx=0x7f9d6000be20 comp rx=0 tx=0).stop 2024-04-23T02:54:57.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.324+0000 7f9d7af71700 1 -- 172.21.15.71:0/583598863 shutdown_connections 2024-04-23T02:54:57.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.324+0000 7f9d7af71700 1 --2- 172.21.15.71:0/583598863 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f9d6406cce0 0x7f9d6406f1a0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:57.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.324+0000 7f9d7af71700 1 --2- 172.21.15.71:0/583598863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d740ffc20 0x7f9d741976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:57.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.324+0000 7f9d7af71700 1 --2- 172.21.15.71:0/583598863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d74105bd0 0x7f9d74197c00 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:57.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.324+0000 7f9d7af71700 1 -- 172.21.15.71:0/583598863 >> 172.21.15.71:0/583598863 conn(0x7f9d74075670 msgr2=0x7f9d740fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:54:57.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.324+0000 7f9d7af71700 1 -- 172.21.15.71:0/583598863 shutdown_connections 2024-04-23T02:54:57.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:57.324+0000 7f9d7af71700 1 -- 172.21.15.71:0/583598863 wait complete. 2024-04-23T02:54:57.336 INFO:teuthology.orchestra.run.smithi071.stdout:true 2024-04-23T02:54:57.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:57 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/583598863' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:54:57.950 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:57 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/583598863' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:54:57.970 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -e sha1=646b8c7d293d8ec31a40c69f7b3237b19c1be24d -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-04-23T02:54:58.197 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:54:58.740 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:54:58 smithi071 ceph-mon[144544]: pgmap v1252: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:58.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:54:58 smithi192 ceph-mon[108617]: pgmap v1252: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:54:59.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.956+0000 7f87bb16a700 1 -- 172.21.15.71:0/3269089655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87b40ffda0 msgr2=0x7f87b4100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:54:59.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.956+0000 7f87bb16a700 1 --2- 172.21.15.71:0/3269089655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87b40ffda0 0x7f87b4100180 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009910 tx=0x7f87a8009c20 comp rx=0 tx=0).stop 2024-04-23T02:54:59.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.956+0000 7f87bb16a700 1 -- 172.21.15.71:0/3269089655 shutdown_connections 2024-04-23T02:54:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.956+0000 7f87bb16a700 1 --2- 172.21.15.71:0/3269089655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87b4100750 0x7f87b41047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.956+0000 7f87bb16a700 1 --2- 172.21.15.71:0/3269089655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87b40ffda0 0x7f87b4100180 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.956+0000 7f87bb16a700 1 -- 172.21.15.71:0/3269089655 >> 172.21.15.71:0/3269089655 conn(0x7f87b40fb650 msgr2=0x7f87b40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:54:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.956+0000 7f87bb16a700 1 -- 172.21.15.71:0/3269089655 shutdown_connections 2024-04-23T02:54:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.956+0000 7f87bb16a700 1 -- 172.21.15.71:0/3269089655 wait complete. 2024-04-23T02:54:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.957+0000 7f87bb16a700 1 Processor -- start 2024-04-23T02:54:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.957+0000 7f87bb16a700 1 -- start start 2024-04-23T02:54:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.957+0000 7f87bb16a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87b40ffda0 0x7f87b4101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:54:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.957+0000 7f87bb16a700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87b4100750 0x7f87b4101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:54:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.957+0000 7f87bb16a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b41022c0 con 0x7f87b40ffda0 2024-04-23T02:54:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.957+0000 7f87bb16a700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4102400 con 0x7f87b4100750 2024-04-23T02:54:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.957+0000 7f87b8f06700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87b40ffda0 0x7f87b4101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:54:59.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.957+0000 7f87b8f06700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87b40ffda0 0x7f87b4101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55700/0 (socket says 172.21.15.71:55700) 2024-04-23T02:54:59.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.957+0000 7f87b8f06700 1 -- 172.21.15.71:0/12632448 learned_addr learned my addr 172.21.15.71:0/12632448 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:54:59.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.957+0000 7f87b3fff700 1 --2- 172.21.15.71:0/12632448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87b4100750 0x7f87b4101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:54:59.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.957+0000 7f87b8f06700 1 -- 172.21.15.71:0/12632448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87b4100750 msgr2=0x7f87b4101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:54:59.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.957+0000 7f87b8f06700 1 --2- 172.21.15.71:0/12632448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87b4100750 0x7f87b4101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:54:59.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.958+0000 7f87b8f06700 1 -- 172.21.15.71:0/12632448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a80095f0 con 0x7f87b40ffda0 2024-04-23T02:54:59.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.958+0000 7f87b8f06700 1 --2- 172.21.15.71:0/12632448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87b40ffda0 0x7f87b4101840 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f87a8006010 tx=0x7f87a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:54:59.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.958+0000 7f87b1ffb700 1 -- 172.21.15.71:0/12632448 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87a801d070 con 0x7f87b40ffda0 2024-04-23T02:54:59.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.958+0000 7f87b1ffb700 1 -- 172.21.15.71:0/12632448 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f87a800f460 con 0x7f87b40ffda0 2024-04-23T02:54:59.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.958+0000 7f87bb16a700 1 -- 172.21.15.71:0/12632448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b4102680 con 0x7f87b40ffda0 2024-04-23T02:54:59.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.958+0000 7f87b1ffb700 1 -- 172.21.15.71:0/12632448 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87a8021640 con 0x7f87b40ffda0 2024-04-23T02:54:59.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.958+0000 7f87bb16a700 1 -- 172.21.15.71:0/12632448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b41a2d70 con 0x7f87b40ffda0 2024-04-23T02:54:59.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.959+0000 7f87bb16a700 1 -- 172.21.15.71:0/12632448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b404f2e0 con 0x7f87b40ffda0 2024-04-23T02:54:59.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.959+0000 7f87b1ffb700 1 -- 172.21.15.71:0/12632448 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f87a800fa60 con 0x7f87b40ffda0 2024-04-23T02:54:59.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.960+0000 7f87b1ffb700 1 --2- 172.21.15.71:0/12632448 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f87a406cce0 0x7f87a406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:54:59.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.960+0000 7f87b1ffb700 1 -- 172.21.15.71:0/12632448 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f87a808d860 con 0x7f87b40ffda0 2024-04-23T02:54:59.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.960+0000 7f87b3fff700 1 --2- 172.21.15.71:0/12632448 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f87a406cce0 0x7f87a406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:54:59.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.960+0000 7f87b3fff700 1 --2- 172.21.15.71:0/12632448 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f87a406cce0 0x7f87a406f1a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f87b4103030 tx=0x7f87a0005eb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:54:59.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:54:59.964+0000 7f87b1ffb700 1 -- 172.21.15.71:0/12632448 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f87a80578d0 con 0x7f87b40ffda0 2024-04-23T02:55:00.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:00.238+0000 7f87bb16a700 1 -- 172.21.15.71:0/12632448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f87b41a2f90 con 0x7f87b40ffda0 2024-04-23T02:55:00.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:00.238+0000 7f87b1ffb700 1 -- 172.21.15.71:0/12632448 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f87a805aef0 con 0x7f87b40ffda0 2024-04-23T02:55:00.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:00.240+0000 7f87bb16a700 1 -- 172.21.15.71:0/12632448 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f87a406cce0 msgr2=0x7f87a406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:00.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:00.240+0000 7f87bb16a700 1 --2- 172.21.15.71:0/12632448 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f87a406cce0 0x7f87a406f1a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f87b4103030 tx=0x7f87a0005eb0 comp rx=0 tx=0).stop 2024-04-23T02:55:00.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:00.240+0000 7f87bb16a700 1 -- 172.21.15.71:0/12632448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87b40ffda0 msgr2=0x7f87b4101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:00.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:00.240+0000 7f87bb16a700 1 --2- 172.21.15.71:0/12632448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87b40ffda0 0x7f87b4101840 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f87a8006010 tx=0x7f87a800be20 comp rx=0 tx=0).stop 2024-04-23T02:55:00.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:00.240+0000 7f87bb16a700 1 -- 172.21.15.71:0/12632448 shutdown_connections 2024-04-23T02:55:00.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:00.240+0000 7f87bb16a700 1 --2- 172.21.15.71:0/12632448 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f87a406cce0 0x7f87a406f1a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:00.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:00.240+0000 7f87bb16a700 1 --2- 172.21.15.71:0/12632448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87b40ffda0 0x7f87b4101840 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:00.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:00.240+0000 7f87bb16a700 1 --2- 172.21.15.71:0/12632448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87b4100750 0x7f87b4101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:00.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:00.240+0000 7f87bb16a700 1 -- 172.21.15.71:0/12632448 >> 172.21.15.71:0/12632448 conn(0x7f87b40fb650 msgr2=0x7f87b4103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:00.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:00.241+0000 7f87bb16a700 1 -- 172.21.15.71:0/12632448 shutdown_connections 2024-04-23T02:55:00.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:00.241+0000 7f87bb16a700 1 -- 172.21.15.71:0/12632448 wait complete. 2024-04-23T02:55:00.253 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph version 18.2.2-1237-g646b8c7d (646b8c7d293d8ec31a40c69f7b3237b19c1be24d) reef (stable)" 2024-04-23T02:55:00.857 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:00 smithi071 ceph-mon[144544]: pgmap v1253: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:55:00.857 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:00 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/12632448' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:55:00.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:00 smithi192 ceph-mon[108617]: pgmap v1253: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:55:00.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:00 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/12632448' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-23T02:55:00.877 DEBUG:teuthology.parallel:result is None 2024-04-23T02:55:00.877 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T02:55:00.890 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi071.front.sepia.ceph.com 2024-04-23T02:55:00.890 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 91f5470c-0115-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-23T02:55:01.120 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/mon.smithi071/config 2024-04-23T02:55:02.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.787+0000 7fb04fcff700 1 -- 172.21.15.71:0/3610684007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0480ffda0 msgr2=0x7fb048100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:02.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.787+0000 7fb04fcff700 1 --2- 172.21.15.71:0/3610684007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0480ffda0 0x7fb048100180 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb038009910 tx=0x7fb038009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:02.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.787+0000 7fb04fcff700 1 -- 172.21.15.71:0/3610684007 shutdown_connections 2024-04-23T02:55:02.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.787+0000 7fb04fcff700 1 --2- 172.21.15.71:0/3610684007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb048100750 0x7fb0481047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:02.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.787+0000 7fb04fcff700 1 --2- 172.21.15.71:0/3610684007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0480ffda0 0x7fb048100180 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:02.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.787+0000 7fb04fcff700 1 -- 172.21.15.71:0/3610684007 >> 172.21.15.71:0/3610684007 conn(0x7fb0480fb650 msgr2=0x7fb0480fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:02.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.787+0000 7fb04fcff700 1 -- 172.21.15.71:0/3610684007 shutdown_connections 2024-04-23T02:55:02.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.787+0000 7fb04fcff700 1 -- 172.21.15.71:0/3610684007 wait complete. 2024-04-23T02:55:02.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.788+0000 7fb04fcff700 1 Processor -- start 2024-04-23T02:55:02.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.788+0000 7fb04fcff700 1 -- start start 2024-04-23T02:55:02.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.788+0000 7fb04fcff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0480ffda0 0x7fb04819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:02.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.788+0000 7fb04fcff700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb048100750 0x7fb04819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:02.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.788+0000 7fb04fcff700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04819d360 con 0x7fb0480ffda0 2024-04-23T02:55:02.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.788+0000 7fb04fcff700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb048196810 con 0x7fb048100750 2024-04-23T02:55:02.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.788+0000 7fb04da9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0480ffda0 0x7fb04819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:02.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.788+0000 7fb04da9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0480ffda0 0x7fb04819c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55716/0 (socket says 172.21.15.71:55716) 2024-04-23T02:55:02.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.788+0000 7fb04da9b700 1 -- 172.21.15.71:0/401305949 learned_addr learned my addr 172.21.15.71:0/401305949 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:02.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.788+0000 7fb04d29a700 1 --2- 172.21.15.71:0/401305949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb048100750 0x7fb04819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:02.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.788+0000 7fb04da9b700 1 -- 172.21.15.71:0/401305949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb048100750 msgr2=0x7fb04819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:02.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.788+0000 7fb04da9b700 1 --2- 172.21.15.71:0/401305949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb048100750 0x7fb04819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:02.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.788+0000 7fb04da9b700 1 -- 172.21.15.71:0/401305949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0380095f0 con 0x7fb0480ffda0 2024-04-23T02:55:02.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.788+0000 7fb04da9b700 1 --2- 172.21.15.71:0/401305949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0480ffda0 0x7fb04819c790 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb038006010 tx=0x7fb03800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:02.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.789+0000 7fb03effd700 1 -- 172.21.15.71:0/401305949 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb03801d070 con 0x7fb0480ffda0 2024-04-23T02:55:02.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.789+0000 7fb03effd700 1 -- 172.21.15.71:0/401305949 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb03800f460 con 0x7fb0480ffda0 2024-04-23T02:55:02.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.789+0000 7fb04fcff700 1 -- 172.21.15.71:0/401305949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb048196a90 con 0x7fb0480ffda0 2024-04-23T02:55:02.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.789+0000 7fb03effd700 1 -- 172.21.15.71:0/401305949 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb038005de0 con 0x7fb0480ffda0 2024-04-23T02:55:02.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.789+0000 7fb04fcff700 1 -- 172.21.15.71:0/401305949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb048196f80 con 0x7fb0480ffda0 2024-04-23T02:55:02.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.790+0000 7fb04fcff700 1 -- 172.21.15.71:0/401305949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04804f2e0 con 0x7fb0480ffda0 2024-04-23T02:55:02.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.790+0000 7fb03effd700 1 -- 172.21.15.71:0/401305949 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb03800f5d0 con 0x7fb0480ffda0 2024-04-23T02:55:02.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.790+0000 7fb03effd700 1 --2- 172.21.15.71:0/401305949 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb03406cce0 0x7fb03406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:02.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.791+0000 7fb03effd700 1 -- 172.21.15.71:0/401305949 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb03808d8d0 con 0x7fb0480ffda0 2024-04-23T02:55:02.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.791+0000 7fb04d29a700 1 --2- 172.21.15.71:0/401305949 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb03406cce0 0x7fb03406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:02.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.791+0000 7fb04d29a700 1 --2- 172.21.15.71:0/401305949 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb03406cce0 0x7fb03406f1a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb044006fd0 tx=0x7fb044008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:02.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:02.795+0000 7fb03effd700 1 -- 172.21.15.71:0/401305949 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb03805b3d0 con 0x7fb0480ffda0 2024-04-23T02:55:02.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:02 smithi192 ceph-mon[108617]: pgmap v1254: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:02.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:02 smithi071 ceph-mon[144544]: pgmap v1254: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:03.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.033+0000 7fb04fcff700 1 -- 172.21.15.71:0/401305949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb048197260 con 0x7fb0480ffda0 2024-04-23T02:55:03.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.033+0000 7fb03effd700 1 -- 172.21.15.71:0/401305949 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 46 v46) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fb03800f880 con 0x7fb0480ffda0 2024-04-23T02:55:03.036 INFO:teuthology.orchestra.run.smithi071.stdout:e46 2024-04-23T02:55:03.036 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T02:55:03.036 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:55:03.036 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-04-23T02:55:03.037 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:03.037 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-04-23T02:55:03.037 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-04-23T02:55:03.037 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 46 2024-04-23T02:55:03.037 INFO:teuthology.orchestra.run.smithi071.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T02:55:03.037 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-04-23T02:10:51.346345+0000 2024-04-23T02:55:03.037 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-04-23T02:53:26.403853+0000 2024-04-23T02:55:03.037 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-04-23T02:55:03.037 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-04-23T02:55:03.037 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 60 2024-04-23T02:55:03.037 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-04-23T02:55:03.037 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-04-23T02:55:03.037 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-04-23T02:55:03.038 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-04-23T02:55:03.038 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-04-23T02:55:03.038 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 80 2024-04-23T02:55:03.038 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-23T02:55:03.038 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 2 2024-04-23T02:55:03.038 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1 2024-04-23T02:55:03.039 INFO:teuthology.orchestra.run.smithi071.stdout:up {0=34334,1=34362} 2024-04-23T02:55:03.039 INFO:teuthology.orchestra.run.smithi071.stdout:failed 2024-04-23T02:55:03.039 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-04-23T02:55:03.039 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-04-23T02:55:03.039 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3] 2024-04-23T02:55:03.039 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-04-23T02:55:03.039 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data enabled 2024-04-23T02:55:03.040 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-04-23T02:55:03.040 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-04-23T02:55:03.040 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 1 2024-04-23T02:55:03.040 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.qusrrb{0:34334} state up:active seq 469 join_fscid=1 addr [v2:172.21.15.71:6826/2539819173,v1:172.21.15.71:6827/2539819173] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:55:03.040 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.gtmyst{0:44319} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.192:6824/3688927981,v1:172.21.15.192:6825/3688927981] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:55:03.040 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi071.vxejye{1:34362} state up:active seq 463 join_fscid=1 addr [v2:172.21.15.71:6828/143423621,v1:172.21.15.71:6829/143423621] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:55:03.040 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.cephfs.smithi192.azdyze{1:44793} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.192:6826/3768627645,v1:172.21.15.192:6827/3768627645] compat {c=[1],r=[1],i=[7ff]}] 2024-04-23T02:55:03.040 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:03.040 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:03.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.035+0000 7fb04fcff700 1 -- 172.21.15.71:0/401305949 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb03406cce0 msgr2=0x7fb03406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:03.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.035+0000 7fb04fcff700 1 --2- 172.21.15.71:0/401305949 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb03406cce0 0x7fb03406f1a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb044006fd0 tx=0x7fb044008040 comp rx=0 tx=0).stop 2024-04-23T02:55:03.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.035+0000 7fb04fcff700 1 -- 172.21.15.71:0/401305949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0480ffda0 msgr2=0x7fb04819c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:03.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.035+0000 7fb04fcff700 1 --2- 172.21.15.71:0/401305949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0480ffda0 0x7fb04819c790 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb038006010 tx=0x7fb03800be20 comp rx=0 tx=0).stop 2024-04-23T02:55:03.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.035+0000 7fb04fcff700 1 -- 172.21.15.71:0/401305949 shutdown_connections 2024-04-23T02:55:03.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.035+0000 7fb04fcff700 1 --2- 172.21.15.71:0/401305949 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fb03406cce0 0x7fb03406f1a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:03.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.035+0000 7fb04fcff700 1 --2- 172.21.15.71:0/401305949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0480ffda0 0x7fb04819c790 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:03.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.035+0000 7fb04fcff700 1 --2- 172.21.15.71:0/401305949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb048100750 0x7fb04819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:03.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.035+0000 7fb04fcff700 1 -- 172.21.15.71:0/401305949 >> 172.21.15.71:0/401305949 conn(0x7fb0480fb650 msgr2=0x7fb048104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:03.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.036+0000 7fb04fcff700 1 -- 172.21.15.71:0/401305949 shutdown_connections 2024-04-23T02:55:03.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.036+0000 7fb04fcff700 1 -- 172.21.15.71:0/401305949 wait complete. 2024-04-23T02:55:03.042 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 46 2024-04-23T02:55:03.655 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-04-23T02:55:03.679 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T02:55:03.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.803+0000 7f3ce0650700 1 -- 172.21.15.71:0/170040738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd8102570 msgr2=0x7f3cd810aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:03.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.803+0000 7f3ce0650700 1 --2- 172.21.15.71:0/170040738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd8102570 0x7f3cd810aa50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4009910 tx=0x7f3cd4009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:03.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.803+0000 7f3ce0650700 1 -- 172.21.15.71:0/170040738 shutdown_connections 2024-04-23T02:55:03.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.803+0000 7f3ce0650700 1 --2- 172.21.15.71:0/170040738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd8102570 0x7f3cd810aa50 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:03.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.803+0000 7f3ce0650700 1 --2- 172.21.15.71:0/170040738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cd8101c50 0x7f3cd8102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:03.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.803+0000 7f3ce0650700 1 -- 172.21.15.71:0/170040738 >> 172.21.15.71:0/170040738 conn(0x7f3cd80fb430 msgr2=0x7f3cd80fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:03.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.803+0000 7f3ce0650700 1 -- 172.21.15.71:0/170040738 shutdown_connections 2024-04-23T02:55:03.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.803+0000 7f3ce0650700 1 -- 172.21.15.71:0/170040738 wait complete. 2024-04-23T02:55:03.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.804+0000 7f3ce0650700 1 Processor -- start 2024-04-23T02:55:03.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.804+0000 7f3ce0650700 1 -- start start 2024-04-23T02:55:03.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.804+0000 7f3ce0650700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cd8101c50 0x7f3cd8196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:03.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.804+0000 7f3ce0650700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd8102570 0x7f3cd8196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:03.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.804+0000 7f3ce0650700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd8197220 con 0x7f3cd8102570 2024-04-23T02:55:03.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.804+0000 7f3ce0650700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd819afa0 con 0x7f3cd8101c50 2024-04-23T02:55:03.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.804+0000 7f3cddbeb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd8102570 0x7f3cd8196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:03.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.804+0000 7f3cddbeb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd8102570 0x7f3cd8196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55732/0 (socket says 172.21.15.71:55732) 2024-04-23T02:55:03.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.804+0000 7f3cddbeb700 1 -- 172.21.15.71:0/1717776555 learned_addr learned my addr 172.21.15.71:0/1717776555 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:03.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.804+0000 7f3cde3ec700 1 --2- 172.21.15.71:0/1717776555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cd8101c50 0x7f3cd8196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:03.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.804+0000 7f3cddbeb700 1 -- 172.21.15.71:0/1717776555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cd8101c50 msgr2=0x7f3cd8196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:03.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.804+0000 7f3cddbeb700 1 --2- 172.21.15.71:0/1717776555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cd8101c50 0x7f3cd8196600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:03.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.804+0000 7f3cddbeb700 1 -- 172.21.15.71:0/1717776555 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd40095f0 con 0x7f3cd8102570 2024-04-23T02:55:03.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.804+0000 7f3cddbeb700 1 --2- 172.21.15.71:0/1717776555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd8102570 0x7f3cd8196b40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40092e0 tx=0x7f3cd400bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:03.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.804+0000 7f3ccf7fe700 1 -- 172.21.15.71:0/1717776555 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cd401d070 con 0x7f3cd8102570 2024-04-23T02:55:03.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.804+0000 7f3ccf7fe700 1 -- 172.21.15.71:0/1717776555 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3cd400be10 con 0x7f3cd8102570 2024-04-23T02:55:03.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.804+0000 7f3ce0650700 1 -- 172.21.15.71:0/1717776555 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd819b220 con 0x7f3cd8102570 2024-04-23T02:55:03.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.805+0000 7f3ccf7fe700 1 -- 172.21.15.71:0/1717776555 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cd4021410 con 0x7f3cd8102570 2024-04-23T02:55:03.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.805+0000 7f3ce0650700 1 -- 172.21.15.71:0/1717776555 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd819b720 con 0x7f3cd8102570 2024-04-23T02:55:03.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.805+0000 7f3ce0650700 1 -- 172.21.15.71:0/1717776555 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd81080f0 con 0x7f3cd8102570 2024-04-23T02:55:03.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.806+0000 7f3ccf7fe700 1 -- 172.21.15.71:0/1717776555 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3cd400f460 con 0x7f3cd8102570 2024-04-23T02:55:03.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.806+0000 7f3ccf7fe700 1 --2- 172.21.15.71:0/1717776555 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3cc406cda0 0x7f3cc406f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:03.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.806+0000 7f3ccf7fe700 1 -- 172.21.15.71:0/1717776555 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3cd408d760 con 0x7f3cd8102570 2024-04-23T02:55:03.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.806+0000 7f3cde3ec700 1 --2- 172.21.15.71:0/1717776555 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3cc406cda0 0x7f3cc406f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:03.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.807+0000 7f3cde3ec700 1 --2- 172.21.15.71:0/1717776555 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3cc406cda0 0x7f3cc406f260 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8005950 tx=0x7f3cc80058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:03.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:03.811+0000 7f3ccf7fe700 1 -- 172.21.15.71:0/1717776555 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3cd4057850 con 0x7f3cd8102570 2024-04-23T02:55:03.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:03 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/401305949' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:55:03.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:03 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/401305949' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T02:55:04.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.007+0000 7f3ce0650700 1 -- 172.21.15.71:0/1717776555 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cd804ea90 con 0x7f3cd8102570 2024-04-23T02:55:04.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.008+0000 7f3ccf7fe700 1 -- 172.21.15.71:0/1717776555 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16305 (secure 0 0 0) 0x7f3cd405ae70 con 0x7f3cd8102570 2024-04-23T02:55:04.010 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:04.011 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":84,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","created":"2024-04-23T02:03:14.872097+0000","modified":"2024-04-23T02:37:53.913348+0000","last_up_change":"2024-04-23T02:21:57.350899+0000","last_in_change":"2024-04-23T02:09:09.560591+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T02:08:06.416728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-23T02:10:50.283526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-23T02:10:50.576356+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5f48bf1-6e98-4432-885a-f6ed0c42e5d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2865488005},{"type":"v1","addr":"172.21.15.71:6803","nonce":2865488005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2865488005},{"type":"v1","addr":"172.21.15.71:6805","nonce":2865488005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2865488005},{"type":"v1","addr":"172.21.15.71:6809","nonce":2865488005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2865488005},{"type":"v1","addr":"172.21.15.71:6807","nonce":2865488005}]},"public_addr":"172.21.15.71:6803/2865488005","cluster_addr":"172.21.15.71:6805/2865488005","heartbeat_back_addr":"172.21.15.71:6809/2865488005","heartbeat_front_addr":"172.21.15.71:6807/2865488005","state":["exists","up"]},{"osd":1,"uuid":"04304fd2-42f9-4525-88e8-c7e20c9c51e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2293979729},{"type":"v1","addr":"172.21.15.71:6811","nonce":2293979729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2293979729},{"type":"v1","addr":"172.21.15.71:6813","nonce":2293979729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2293979729},{"type":"v1","addr":"172.21.15.71:6817","nonce":2293979729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2293979729},{"type":"v1","addr":"172.21.15.71:6815","nonce":2293979729}]},"public_addr":"172.21.15.71:6811/2293979729","cluster_addr":"172.21.15.71:6813/2293979729","heartbeat_back_addr":"172.21.15.71:6817/2293979729","heartbeat_front_addr":"172.21.15.71:6815/2293979729","state":["exists","up"]},{"osd":2,"uuid":"6797c77c-4ccc-4489-9269-77105d7c45c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3388560056},{"type":"v1","addr":"172.21.15.71:6819","nonce":3388560056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3388560056},{"type":"v1","addr":"172.21.15.71:6821","nonce":3388560056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3388560056},{"type":"v1","addr":"172.21.15.71:6825","nonce":3388560056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3388560056},{"type":"v1","addr":"172.21.15.71:6823","nonce":3388560056}]},"public_addr":"172.21.15.71:6819/3388560056","cluster_addr":"172.21.15.71:6821/3388560056","heartbeat_back_addr":"172.21.15.71:6825/3388560056","heartbeat_front_addr":"172.21.15.71:6823/3388560056","state":["exists","up"]},{"osd":3,"uuid":"62033906-4d1c-431d-a87d-a12e0a134383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4106623627},{"type":"v1","addr":"172.21.15.192:6801","nonce":4106623627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4106623627},{"type":"v1","addr":"172.21.15.192:6803","nonce":4106623627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4106623627},{"type":"v1","addr":"172.21.15.192:6807","nonce":4106623627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4106623627},{"type":"v1","addr":"172.21.15.192:6805","nonce":4106623627}]},"public_addr":"172.21.15.192:6801/4106623627","cluster_addr":"172.21.15.192:6803/4106623627","heartbeat_back_addr":"172.21.15.192:6807/4106623627","heartbeat_front_addr":"172.21.15.192:6805/4106623627","state":["exists","up"]},{"osd":4,"uuid":"6a7317f5-dc05-415a-8409-81f80c196a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1707328698},{"type":"v1","addr":"172.21.15.192:6809","nonce":1707328698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1707328698},{"type":"v1","addr":"172.21.15.192:6811","nonce":1707328698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1707328698},{"type":"v1","addr":"172.21.15.192:6815","nonce":1707328698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1707328698},{"type":"v1","addr":"172.21.15.192:6813","nonce":1707328698}]},"public_addr":"172.21.15.192:6809/1707328698","cluster_addr":"172.21.15.192:6811/1707328698","heartbeat_back_addr":"172.21.15.192:6815/1707328698","heartbeat_front_addr":"172.21.15.192:6813/1707328698","state":["exists","up"]},{"osd":5,"uuid":"68b550d5-d74e-417e-9bcc-98740efaf0ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":868066904},{"type":"v1","addr":"172.21.15.192:6817","nonce":868066904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":868066904},{"type":"v1","addr":"172.21.15.192:6819","nonce":868066904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":868066904},{"type":"v1","addr":"172.21.15.192:6823","nonce":868066904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":868066904},{"type":"v1","addr":"172.21.15.192:6821","nonce":868066904}]},"public_addr":"172.21.15.192:6817/868066904","cluster_addr":"172.21.15.192:6819/868066904","heartbeat_back_addr":"172.21.15.192:6823/868066904","heartbeat_front_addr":"172.21.15.192:6821/868066904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:07:01.863954+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:07:32.930839+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:04.093747+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:31.574485+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:59.698428+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:09:28.076223+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6828/467545394":"2024-04-24T02:22:00.280478+0000","172.21.15.71:6801/3794652670":"2024-04-24T02:03:48.178880+0000","172.21.15.71:6801/4165305842":"2024-04-24T02:05:42.095805+0000","172.21.15.71:0/56041968":"2024-04-24T02:05:42.095805+0000","172.21.15.192:0/1753179829":"2024-04-24T02:14:34.629978+0000","172.21.15.71:6800/4165305842":"2024-04-24T02:05:42.095805+0000","172.21.15.192:0/2569976755":"2024-04-24T02:14:34.629978+0000","172.21.15.71:0/823266478":"2024-04-24T02:04:23.247815+0000","172.21.15.192:6828/966231696":"2024-04-24T02:14:34.629978+0000","172.21.15.192:6824/3780549896":"2024-04-24T02:22:00.280491+0000","172.21.15.192:0/1701960325":"2024-04-24T02:14:34.629978+0000","172.21.15.71:0/2225989508":"2024-04-24T02:03:48.178880+0000","172.21.15.71:6829/467545394":"2024-04-24T02:22:00.280478+0000","172.21.15.192:0/2687070241":"2024-04-24T02:14:34.629978+0000","172.21.15.71:0/17588705":"2024-04-24T02:13:11.575106+0000","172.21.15.71:0/3579793925":"2024-04-24T02:04:23.247815+0000","172.21.15.71:0/1995129374":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/294030681":"2024-04-24T02:04:23.247815+0000","172.21.15.71:0/936344738":"2024-04-24T02:05:42.095805+0000","172.21.15.192:6829/966231696":"2024-04-24T02:14:34.629978+0000","172.21.15.192:0/2727292170":"2024-04-24T02:14:34.629978+0000","172.21.15.71:0/1815873101":"2024-04-24T02:13:11.575106+0000","172.21.15.192:6825/3780549896":"2024-04-24T02:22:00.280491+0000","172.21.15.71:6801/3878148961":"2024-04-24T02:04:23.247815+0000","172.21.15.71:6827/3504670324":"2024-04-24T02:11:05.663451+0000","172.21.15.192:0/2877439214":"2024-04-24T02:14:34.629978+0000","172.21.15.71:6800/4249907286":"2024-04-24T02:13:11.575106+0000","172.21.15.71:6800/3878148961":"2024-04-24T02:04:23.247815+0000","172.21.15.71:6800/3794652670":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/4205332842":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/3618236204":"2024-04-24T02:13:11.575106+0000","172.21.15.71:0/3473332587":"2024-04-24T02:13:11.575106+0000","172.21.15.71:0/2206241071":"2024-04-24T02:05:42.095805+0000","172.21.15.71:6801/4249907286":"2024-04-24T02:13:11.575106+0000","172.21.15.71:6826/3504670324":"2024-04-24T02:11:05.663451+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T02:55:04.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.010+0000 7f3ce0650700 1 -- 172.21.15.71:0/1717776555 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3cc406cda0 msgr2=0x7f3cc406f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:04.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.010+0000 7f3ce0650700 1 --2- 172.21.15.71:0/1717776555 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3cc406cda0 0x7f3cc406f260 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8005950 tx=0x7f3cc80058e0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.010+0000 7f3ce0650700 1 -- 172.21.15.71:0/1717776555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd8102570 msgr2=0x7f3cd8196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:04.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.010+0000 7f3ce0650700 1 --2- 172.21.15.71:0/1717776555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd8102570 0x7f3cd8196b40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40092e0 tx=0x7f3cd400bbf0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.010+0000 7f3ce0650700 1 -- 172.21.15.71:0/1717776555 shutdown_connections 2024-04-23T02:55:04.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.010+0000 7f3ce0650700 1 --2- 172.21.15.71:0/1717776555 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3cc406cda0 0x7f3cc406f260 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.010+0000 7f3ce0650700 1 --2- 172.21.15.71:0/1717776555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cd8101c50 0x7f3cd8196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.010+0000 7f3ce0650700 1 --2- 172.21.15.71:0/1717776555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cd8102570 0x7f3cd8196b40 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.010+0000 7f3ce0650700 1 -- 172.21.15.71:0/1717776555 >> 172.21.15.71:0/1717776555 conn(0x7f3cd80fb430 msgr2=0x7f3cd8105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:04.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.010+0000 7f3ce0650700 1 -- 172.21.15.71:0/1717776555 shutdown_connections 2024-04-23T02:55:04.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.010+0000 7f3ce0650700 1 -- 172.21.15.71:0/1717776555 wait complete. 2024-04-23T02:55:04.025 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T02:08:06.416728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-23T02:10:50.283526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-23T02:10:50.576356+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T02:55:04.025 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T02:55:04.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.148+0000 7f916100c700 1 -- 172.21.15.71:0/2976502030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f915c0ffe10 msgr2=0x7f915c105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:04.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.148+0000 7f916100c700 1 --2- 172.21.15.71:0/2976502030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f915c0ffe10 0x7f915c105790 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9150009910 tx=0x7f9150009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:04.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.148+0000 7f916100c700 1 -- 172.21.15.71:0/2976502030 shutdown_connections 2024-04-23T02:55:04.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.148+0000 7f916100c700 1 --2- 172.21.15.71:0/2976502030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f915c0ffe10 0x7f915c105790 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.148+0000 7f916100c700 1 --2- 172.21.15.71:0/2976502030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f915c0ff460 0x7f915c0ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.148+0000 7f916100c700 1 -- 172.21.15.71:0/2976502030 >> 172.21.15.71:0/2976502030 conn(0x7f915c074b80 msgr2=0x7f915c074f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:04.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.148+0000 7f916100c700 1 -- 172.21.15.71:0/2976502030 shutdown_connections 2024-04-23T02:55:04.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.148+0000 7f916100c700 1 -- 172.21.15.71:0/2976502030 wait complete. 2024-04-23T02:55:04.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.149+0000 7f916100c700 1 Processor -- start 2024-04-23T02:55:04.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.149+0000 7f916100c700 1 -- start start 2024-04-23T02:55:04.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.149+0000 7f916100c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f915c0ff460 0x7f915c196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:04.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.149+0000 7f916100c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f915c196b80 0x7f915c19afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:04.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.149+0000 7f916100c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c1971a0 con 0x7f915c0ff460 2024-04-23T02:55:04.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.149+0000 7f916100c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c197310 con 0x7f915c196b80 2024-04-23T02:55:04.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.149+0000 7f915ad9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f915c0ff460 0x7f915c196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:04.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.149+0000 7f915ad9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f915c0ff460 0x7f915c196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55748/0 (socket says 172.21.15.71:55748) 2024-04-23T02:55:04.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.149+0000 7f915ad9d700 1 -- 172.21.15.71:0/2417002489 learned_addr learned my addr 172.21.15.71:0/2417002489 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:04.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.149+0000 7f915a59c700 1 --2- 172.21.15.71:0/2417002489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f915c196b80 0x7f915c19afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:04.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.150+0000 7f915ad9d700 1 -- 172.21.15.71:0/2417002489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f915c196b80 msgr2=0x7f915c19afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:04.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.150+0000 7f915ad9d700 1 --2- 172.21.15.71:0/2417002489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f915c196b80 0x7f915c19afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.150+0000 7f915ad9d700 1 -- 172.21.15.71:0/2417002489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91500095f0 con 0x7f915c0ff460 2024-04-23T02:55:04.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.150+0000 7f915ad9d700 1 --2- 172.21.15.71:0/2417002489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f915c0ff460 0x7f915c196640 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f914c00e940 tx=0x7f914c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:04.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.150+0000 7f9143fff700 1 -- 172.21.15.71:0/2417002489 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f914c00cba0 con 0x7f915c0ff460 2024-04-23T02:55:04.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.150+0000 7f916100c700 1 -- 172.21.15.71:0/2417002489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915c19b640 con 0x7f915c0ff460 2024-04-23T02:55:04.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.150+0000 7f9143fff700 1 -- 172.21.15.71:0/2417002489 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f914c00ee90 con 0x7f915c0ff460 2024-04-23T02:55:04.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.150+0000 7f9143fff700 1 -- 172.21.15.71:0/2417002489 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f914c0055f0 con 0x7f915c0ff460 2024-04-23T02:55:04.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.150+0000 7f916100c700 1 -- 172.21.15.71:0/2417002489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f915c19bba0 con 0x7f915c0ff460 2024-04-23T02:55:04.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.151+0000 7f916100c700 1 -- 172.21.15.71:0/2417002489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f915c19be20 con 0x7f915c0ff460 2024-04-23T02:55:04.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.151+0000 7f9143fff700 1 -- 172.21.15.71:0/2417002489 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f914c004750 con 0x7f915c0ff460 2024-04-23T02:55:04.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.152+0000 7f9143fff700 1 --2- 172.21.15.71:0/2417002489 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f914406cce0 0x7f914406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:04.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.152+0000 7f9143fff700 1 -- 172.21.15.71:0/2417002489 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f914c014070 con 0x7f915c0ff460 2024-04-23T02:55:04.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.152+0000 7f915a59c700 1 --2- 172.21.15.71:0/2417002489 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f914406cce0 0x7f914406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:04.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.152+0000 7f915a59c700 1 --2- 172.21.15.71:0/2417002489 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f914406cce0 0x7f914406f1a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f91500059d0 tx=0x7f9150005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:04.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.156+0000 7f9143fff700 1 -- 172.21.15.71:0/2417002489 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f914c0568d0 con 0x7f915c0ff460 2024-04-23T02:55:04.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.348+0000 7f916100c700 1 -- 172.21.15.71:0/2417002489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f915c197a50 con 0x7f915c0ff460 2024-04-23T02:55:04.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.348+0000 7f9143fff700 1 -- 172.21.15.71:0/2417002489 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f914c059ef0 con 0x7f915c0ff460 2024-04-23T02:55:04.351 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-23T02:55:04.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.351+0000 7f916100c700 1 -- 172.21.15.71:0/2417002489 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f914406cce0 msgr2=0x7f914406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:04.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.351+0000 7f916100c700 1 --2- 172.21.15.71:0/2417002489 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f914406cce0 0x7f914406f1a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f91500059d0 tx=0x7f9150005940 comp rx=0 tx=0).stop 2024-04-23T02:55:04.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.351+0000 7f916100c700 1 -- 172.21.15.71:0/2417002489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f915c0ff460 msgr2=0x7f915c196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:04.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.351+0000 7f916100c700 1 --2- 172.21.15.71:0/2417002489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f915c0ff460 0x7f915c196640 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f914c00e940 tx=0x7f914c00ec50 comp rx=0 tx=0).stop 2024-04-23T02:55:04.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.351+0000 7f916100c700 1 -- 172.21.15.71:0/2417002489 shutdown_connections 2024-04-23T02:55:04.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.351+0000 7f916100c700 1 --2- 172.21.15.71:0/2417002489 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f914406cce0 0x7f914406f1a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.351+0000 7f916100c700 1 --2- 172.21.15.71:0/2417002489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f915c0ff460 0x7f915c196640 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.351+0000 7f916100c700 1 --2- 172.21.15.71:0/2417002489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f915c196b80 0x7f915c19afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.351+0000 7f916100c700 1 -- 172.21.15.71:0/2417002489 >> 172.21.15.71:0/2417002489 conn(0x7f915c074b80 msgr2=0x7f915c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:04.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.351+0000 7f916100c700 1 -- 172.21.15.71:0/2417002489 shutdown_connections 2024-04-23T02:55:04.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.351+0000 7f916100c700 1 -- 172.21.15.71:0/2417002489 wait complete. 2024-04-23T02:55:04.366 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-23T02:55:04.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.487+0000 7f8c05bfb700 1 -- 172.21.15.71:0/3087293347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c00100f00 msgr2=0x7f8c00104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:04.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.487+0000 7f8c05bfb700 1 --2- 172.21.15.71:0/3087293347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c00100f00 0x7f8c00104600 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0009910 tx=0x7f8bf0009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:04.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.488+0000 7f8c05bfb700 1 -- 172.21.15.71:0/3087293347 shutdown_connections 2024-04-23T02:55:04.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.488+0000 7f8c05bfb700 1 --2- 172.21.15.71:0/3087293347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c00100f00 0x7f8c00104600 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.488+0000 7f8c05bfb700 1 --2- 172.21.15.71:0/3087293347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c00100550 0x7f8c00100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.488+0000 7f8c05bfb700 1 -- 172.21.15.71:0/3087293347 >> 172.21.15.71:0/3087293347 conn(0x7f8c00074c10 msgr2=0x7f8c00075020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:04.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.488+0000 7f8c05bfb700 1 -- 172.21.15.71:0/3087293347 shutdown_connections 2024-04-23T02:55:04.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.488+0000 7f8c05bfb700 1 -- 172.21.15.71:0/3087293347 wait complete. 2024-04-23T02:55:04.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.488+0000 7f8c05bfb700 1 Processor -- start 2024-04-23T02:55:04.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.489+0000 7f8c05bfb700 1 -- start start 2024-04-23T02:55:04.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.489+0000 7f8c05bfb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c00100550 0x7f8c00195530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:04.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.489+0000 7f8c05bfb700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c00195a70 0x7f8c00199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:04.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.489+0000 7f8c05bfb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c00196090 con 0x7f8c00100550 2024-04-23T02:55:04.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.489+0000 7f8c05bfb700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c00196200 con 0x7f8c00195a70 2024-04-23T02:55:04.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.489+0000 7f8bff7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c00100550 0x7f8c00195530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:04.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.489+0000 7f8bff7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c00100550 0x7f8c00195530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55772/0 (socket says 172.21.15.71:55772) 2024-04-23T02:55:04.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.489+0000 7f8bff7fe700 1 -- 172.21.15.71:0/3515185038 learned_addr learned my addr 172.21.15.71:0/3515185038 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:04.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.489+0000 7f8bfeffd700 1 --2- 172.21.15.71:0/3515185038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c00195a70 0x7f8c00199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:04.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.489+0000 7f8bff7fe700 1 -- 172.21.15.71:0/3515185038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c00195a70 msgr2=0x7f8c00199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:04.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.489+0000 7f8bff7fe700 1 --2- 172.21.15.71:0/3515185038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c00195a70 0x7f8c00199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.489+0000 7f8bff7fe700 1 -- 172.21.15.71:0/3515185038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf00095f0 con 0x7f8c00100550 2024-04-23T02:55:04.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.489+0000 7f8bff7fe700 1 --2- 172.21.15.71:0/3515185038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c00100550 0x7f8c00195530 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400d8d0 tx=0x7f8bf400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:04.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.489+0000 7f8bfcff9700 1 -- 172.21.15.71:0/3515185038 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bf4009940 con 0x7f8c00100550 2024-04-23T02:55:04.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.489+0000 7f8bfcff9700 1 -- 172.21.15.71:0/3515185038 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8bf4010460 con 0x7f8c00100550 2024-04-23T02:55:04.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.489+0000 7f8c05bfb700 1 -- 172.21.15.71:0/3515185038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0019a530 con 0x7f8c00100550 2024-04-23T02:55:04.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.489+0000 7f8bfcff9700 1 -- 172.21.15.71:0/3515185038 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bf400f5f0 con 0x7f8c00100550 2024-04-23T02:55:04.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.490+0000 7f8c05bfb700 1 -- 172.21.15.71:0/3515185038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0019aa90 con 0x7f8c00100550 2024-04-23T02:55:04.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.490+0000 7f8c05bfb700 1 -- 172.21.15.71:0/3515185038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c00108040 con 0x7f8c00100550 2024-04-23T02:55:04.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.491+0000 7f8bfcff9700 1 -- 172.21.15.71:0/3515185038 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8bf40105d0 con 0x7f8c00100550 2024-04-23T02:55:04.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.491+0000 7f8bfcff9700 1 --2- 172.21.15.71:0/3515185038 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8be806cce0 0x7f8be806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:04.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.491+0000 7f8bfeffd700 1 --2- 172.21.15.71:0/3515185038 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8be806cce0 0x7f8be806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:04.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.492+0000 7f8bfcff9700 1 -- 172.21.15.71:0/3515185038 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8bf408c5e0 con 0x7f8c00100550 2024-04-23T02:55:04.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.492+0000 7f8bfeffd700 1 --2- 172.21.15.71:0/3515185038 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8be806cce0 0x7f8be806f1a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0006010 tx=0x7f8bf001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:04.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.496+0000 7f8bfcff9700 1 -- 172.21.15.71:0/3515185038 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8bf4056650 con 0x7f8c00100550 2024-04-23T02:55:04.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.690+0000 7f8c05bfb700 1 -- 172.21.15.71:0/3515185038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f8c00196940 con 0x7f8c00100550 2024-04-23T02:55:04.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.690+0000 7f8bfcff9700 1 -- 172.21.15.71:0/3515185038 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f8bf4010880 con 0x7f8c00100550 2024-04-23T02:55:04.693 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-23T02:55:04.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.692+0000 7f8c05bfb700 1 -- 172.21.15.71:0/3515185038 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8be806cce0 msgr2=0x7f8be806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:04.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.692+0000 7f8c05bfb700 1 --2- 172.21.15.71:0/3515185038 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8be806cce0 0x7f8be806f1a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0006010 tx=0x7f8bf001a040 comp rx=0 tx=0).stop 2024-04-23T02:55:04.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.692+0000 7f8c05bfb700 1 -- 172.21.15.71:0/3515185038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c00100550 msgr2=0x7f8c00195530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:04.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.692+0000 7f8c05bfb700 1 --2- 172.21.15.71:0/3515185038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c00100550 0x7f8c00195530 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400d8d0 tx=0x7f8bf400dc90 comp rx=0 tx=0).stop 2024-04-23T02:55:04.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.692+0000 7f8c05bfb700 1 -- 172.21.15.71:0/3515185038 shutdown_connections 2024-04-23T02:55:04.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.692+0000 7f8c05bfb700 1 --2- 172.21.15.71:0/3515185038 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f8be806cce0 0x7f8be806f1a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.692+0000 7f8c05bfb700 1 --2- 172.21.15.71:0/3515185038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c00100550 0x7f8c00195530 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.692+0000 7f8c05bfb700 1 --2- 172.21.15.71:0/3515185038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c00195a70 0x7f8c00199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.692+0000 7f8c05bfb700 1 -- 172.21.15.71:0/3515185038 >> 172.21.15.71:0/3515185038 conn(0x7f8c00074c10 msgr2=0x7f8c000fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:04.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.692+0000 7f8c05bfb700 1 -- 172.21.15.71:0/3515185038 shutdown_connections 2024-04-23T02:55:04.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.692+0000 7f8c05bfb700 1 -- 172.21.15.71:0/3515185038 wait complete. 2024-04-23T02:55:04.707 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-23T02:55:04.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.830+0000 7f4d22cbf700 1 -- 172.21.15.71:0/2025422861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d1c0ff990 msgr2=0x7f4d1c0ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:04.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.830+0000 7f4d22cbf700 1 --2- 172.21.15.71:0/2025422861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d1c0ff990 0x7f4d1c0ffe10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009910 tx=0x7f4d08009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:04.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.830+0000 7f4d22cbf700 1 -- 172.21.15.71:0/2025422861 shutdown_connections 2024-04-23T02:55:04.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.830+0000 7f4d22cbf700 1 --2- 172.21.15.71:0/2025422861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d1c0ff990 0x7f4d1c0ffe10 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.830+0000 7f4d22cbf700 1 --2- 172.21.15.71:0/2025422861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d1c1059a0 0x7f4d1c105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.830+0000 7f4d22cbf700 1 -- 172.21.15.71:0/2025422861 >> 172.21.15.71:0/2025422861 conn(0x7f4d1c0fb460 msgr2=0x7f4d1c0fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:04.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.830+0000 7f4d22cbf700 1 -- 172.21.15.71:0/2025422861 shutdown_connections 2024-04-23T02:55:04.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.830+0000 7f4d22cbf700 1 -- 172.21.15.71:0/2025422861 wait complete. 2024-04-23T02:55:04.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.831+0000 7f4d22cbf700 1 Processor -- start 2024-04-23T02:55:04.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.831+0000 7f4d22cbf700 1 -- start start 2024-04-23T02:55:04.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.831+0000 7f4d22cbf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d1c0ff990 0x7f4d1c196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:04.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.831+0000 7f4d22cbf700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d1c1059a0 0x7f4d1c196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:04.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.831+0000 7f4d22cbf700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c197170 con 0x7f4d1c0ff990 2024-04-23T02:55:04.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.831+0000 7f4d22cbf700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c19aef0 con 0x7f4d1c1059a0 2024-04-23T02:55:04.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.831+0000 7f4d20a5b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d1c0ff990 0x7f4d1c196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:04.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.831+0000 7f4d20a5b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d1c0ff990 0x7f4d1c196550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55786/0 (socket says 172.21.15.71:55786) 2024-04-23T02:55:04.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.831+0000 7f4d20a5b700 1 -- 172.21.15.71:0/4200221092 learned_addr learned my addr 172.21.15.71:0/4200221092 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:04.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.831+0000 7f4d1bfff700 1 --2- 172.21.15.71:0/4200221092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d1c1059a0 0x7f4d1c196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:04.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.831+0000 7f4d20a5b700 1 -- 172.21.15.71:0/4200221092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d1c1059a0 msgr2=0x7f4d1c196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:04.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.831+0000 7f4d20a5b700 1 --2- 172.21.15.71:0/4200221092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d1c1059a0 0x7f4d1c196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:04.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.831+0000 7f4d20a5b700 1 -- 172.21.15.71:0/4200221092 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d080095f0 con 0x7f4d1c0ff990 2024-04-23T02:55:04.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.831+0000 7f4d20a5b700 1 --2- 172.21.15.71:0/4200221092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d1c0ff990 0x7f4d1c196550 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000d8d0 tx=0x7f4d1000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:04.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.832+0000 7f4d19ffb700 1 -- 172.21.15.71:0/4200221092 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d100098a0 con 0x7f4d1c0ff990 2024-04-23T02:55:04.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.832+0000 7f4d19ffb700 1 -- 172.21.15.71:0/4200221092 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4d10010460 con 0x7f4d1c0ff990 2024-04-23T02:55:04.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.832+0000 7f4d22cbf700 1 -- 172.21.15.71:0/4200221092 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c19b1d0 con 0x7f4d1c0ff990 2024-04-23T02:55:04.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.832+0000 7f4d19ffb700 1 -- 172.21.15.71:0/4200221092 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d10009ea0 con 0x7f4d1c0ff990 2024-04-23T02:55:04.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.832+0000 7f4d22cbf700 1 -- 172.21.15.71:0/4200221092 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c19b730 con 0x7f4d1c0ff990 2024-04-23T02:55:04.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.833+0000 7f4d22cbf700 1 -- 172.21.15.71:0/4200221092 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c108f30 con 0x7f4d1c0ff990 2024-04-23T02:55:04.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.833+0000 7f4d19ffb700 1 -- 172.21.15.71:0/4200221092 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4d100105d0 con 0x7f4d1c0ff990 2024-04-23T02:55:04.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.833+0000 7f4d19ffb700 1 --2- 172.21.15.71:0/4200221092 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4d0c06cce0 0x7f4d0c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:04.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.834+0000 7f4d19ffb700 1 -- 172.21.15.71:0/4200221092 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4d1008c480 con 0x7f4d1c0ff990 2024-04-23T02:55:04.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.834+0000 7f4d1bfff700 1 --2- 172.21.15.71:0/4200221092 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4d0c06cce0 0x7f4d0c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:04.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.834+0000 7f4d1bfff700 1 --2- 172.21.15.71:0/4200221092 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4d0c06cce0 0x7f4d0c06f1a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c197b60 tx=0x7f4d0801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:04.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:04.838+0000 7f4d19ffb700 1 -- 172.21.15.71:0/4200221092 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f4d100564f0 con 0x7f4d1c0ff990 2024-04-23T02:55:04.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:04 smithi192 ceph-mon[108617]: pgmap v1255: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:04.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:04 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1717776555' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T02:55:04.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:04 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2417002489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T02:55:04.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:04 smithi071 ceph-mon[144544]: pgmap v1255: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:04.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:04 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1717776555' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T02:55:04.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:04 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2417002489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T02:55:05.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.030+0000 7f4d22cbf700 1 -- 172.21.15.71:0/4200221092 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f4d1c04ea90 con 0x7f4d1c0ff990 2024-04-23T02:55:05.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.030+0000 7f4d19ffb700 1 -- 172.21.15.71:0/4200221092 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f4d10010880 con 0x7f4d1c0ff990 2024-04-23T02:55:05.033 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-23T02:55:05.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.033+0000 7f4d22cbf700 1 -- 172.21.15.71:0/4200221092 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4d0c06cce0 msgr2=0x7f4d0c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:05.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.033+0000 7f4d22cbf700 1 --2- 172.21.15.71:0/4200221092 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4d0c06cce0 0x7f4d0c06f1a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c197b60 tx=0x7f4d0801a040 comp rx=0 tx=0).stop 2024-04-23T02:55:05.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.033+0000 7f4d22cbf700 1 -- 172.21.15.71:0/4200221092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d1c0ff990 msgr2=0x7f4d1c196550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:05.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.033+0000 7f4d22cbf700 1 --2- 172.21.15.71:0/4200221092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d1c0ff990 0x7f4d1c196550 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000d8d0 tx=0x7f4d1000dbe0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.033+0000 7f4d22cbf700 1 -- 172.21.15.71:0/4200221092 shutdown_connections 2024-04-23T02:55:05.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.033+0000 7f4d22cbf700 1 --2- 172.21.15.71:0/4200221092 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4d0c06cce0 0x7f4d0c06f1a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.033+0000 7f4d22cbf700 1 --2- 172.21.15.71:0/4200221092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d1c0ff990 0x7f4d1c196550 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.033+0000 7f4d22cbf700 1 --2- 172.21.15.71:0/4200221092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d1c1059a0 0x7f4d1c196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.033+0000 7f4d22cbf700 1 -- 172.21.15.71:0/4200221092 >> 172.21.15.71:0/4200221092 conn(0x7f4d1c0fb460 msgr2=0x7f4d1c068be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:05.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.033+0000 7f4d22cbf700 1 -- 172.21.15.71:0/4200221092 shutdown_connections 2024-04-23T02:55:05.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.033+0000 7f4d22cbf700 1 -- 172.21.15.71:0/4200221092 wait complete. 2024-04-23T02:55:05.047 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T02:55:05.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.166+0000 7f09a8861700 1 -- 172.21.15.71:0/3761884758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09a40ffac0 msgr2=0x7f09a40fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:05.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.166+0000 7f09a8861700 1 --2- 172.21.15.71:0/3761884758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09a40ffac0 0x7f09a40fff20 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f0994009910 tx=0x7f0994009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:05.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.166+0000 7f09a8861700 1 -- 172.21.15.71:0/3761884758 shutdown_connections 2024-04-23T02:55:05.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.166+0000 7f09a8861700 1 --2- 172.21.15.71:0/3761884758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09a40ffac0 0x7f09a40fff20 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.166+0000 7f09a8861700 1 --2- 172.21.15.71:0/3761884758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a4105990 0x7f09a4105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.166+0000 7f09a8861700 1 -- 172.21.15.71:0/3761884758 >> 172.21.15.71:0/3761884758 conn(0x7f09a4075400 msgr2=0x7f09a4075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:05.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.166+0000 7f09a8861700 1 -- 172.21.15.71:0/3761884758 shutdown_connections 2024-04-23T02:55:05.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.166+0000 7f09a8861700 1 -- 172.21.15.71:0/3761884758 wait complete. 2024-04-23T02:55:05.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.166+0000 7f09a8861700 1 Processor -- start 2024-04-23T02:55:05.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.167+0000 7f09a8861700 1 -- start start 2024-04-23T02:55:05.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.167+0000 7f09a8861700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a40ffac0 0x7f09a41953c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:05.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.167+0000 7f09a8861700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09a4105990 0x7f09a4195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:05.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.167+0000 7f09a8861700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a4195fe0 con 0x7f09a4105990 2024-04-23T02:55:05.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.167+0000 7f09a8861700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a4199d60 con 0x7f09a40ffac0 2024-04-23T02:55:05.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.167+0000 7f09a1d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09a4105990 0x7f09a4195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:05.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.167+0000 7f09a1d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09a4105990 0x7f09a4195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55806/0 (socket says 172.21.15.71:55806) 2024-04-23T02:55:05.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.167+0000 7f09a1d9b700 1 -- 172.21.15.71:0/2925642895 learned_addr learned my addr 172.21.15.71:0/2925642895 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:05.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.167+0000 7f09a259c700 1 --2- 172.21.15.71:0/2925642895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a40ffac0 0x7f09a41953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:05.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.167+0000 7f09a1d9b700 1 -- 172.21.15.71:0/2925642895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a40ffac0 msgr2=0x7f09a41953c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:05.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.167+0000 7f09a1d9b700 1 --2- 172.21.15.71:0/2925642895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a40ffac0 0x7f09a41953c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.167+0000 7f09a1d9b700 1 -- 172.21.15.71:0/2925642895 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09940095f0 con 0x7f09a4105990 2024-04-23T02:55:05.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.167+0000 7f09a1d9b700 1 --2- 172.21.15.71:0/2925642895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09a4105990 0x7f09a4195900 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0994006010 tx=0x7f099400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:05.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.167+0000 7f09937fe700 1 -- 172.21.15.71:0/2925642895 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f099401d070 con 0x7f09a4105990 2024-04-23T02:55:05.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.167+0000 7f09a8861700 1 -- 172.21.15.71:0/2925642895 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a4199fe0 con 0x7f09a4105990 2024-04-23T02:55:05.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.167+0000 7f09937fe700 1 -- 172.21.15.71:0/2925642895 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0994005710 con 0x7f09a4105990 2024-04-23T02:55:05.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.167+0000 7f09937fe700 1 -- 172.21.15.71:0/2925642895 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f099400fe20 con 0x7f09a4105990 2024-04-23T02:55:05.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.168+0000 7f09a8861700 1 -- 172.21.15.71:0/2925642895 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a419a4e0 con 0x7f09a4105990 2024-04-23T02:55:05.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.168+0000 7f09a8861700 1 -- 172.21.15.71:0/2925642895 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a4107f10 con 0x7f09a4105990 2024-04-23T02:55:05.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.169+0000 7f09937fe700 1 -- 172.21.15.71:0/2925642895 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0994003a60 con 0x7f09a4105990 2024-04-23T02:55:05.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.169+0000 7f09937fe700 1 --2- 172.21.15.71:0/2925642895 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f098c06cda0 0x7f098c06f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:05.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.169+0000 7f09a259c700 1 --2- 172.21.15.71:0/2925642895 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f098c06cda0 0x7f098c06f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:05.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.170+0000 7f09937fe700 1 -- 172.21.15.71:0/2925642895 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f099408d750 con 0x7f09a4105990 2024-04-23T02:55:05.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.170+0000 7f09a259c700 1 --2- 172.21.15.71:0/2925642895 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f098c06cda0 0x7f098c06f260 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0998005950 tx=0x7f099800b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:05.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.174+0000 7f09937fe700 1 -- 172.21.15.71:0/2925642895 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f09940577c0 con 0x7f09a4105990 2024-04-23T02:55:05.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.410+0000 7f09a8861700 1 -- 172.21.15.71:0/2925642895 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09a404ea90 con 0x7f09a4105990 2024-04-23T02:55:05.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.410+0000 7f09937fe700 1 -- 172.21.15.71:0/2925642895 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4864 (secure 0 0 0) 0x7f099405ade0 con 0x7f09a4105990 2024-04-23T02:55:05.412 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:05.413 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":46,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:53:26.403853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34334,"mds_1":34362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34334":{"gid":34334,"name":"cephfs.smithi071.qusrrb","rank":0,"incarnation":38,"state":"up:active","state_seq":469,"addr":"172.21.15.71:6827/2539819173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2539819173},{"type":"v1","addr":"172.21.15.71:6827","nonce":2539819173}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34362":{"gid":34362,"name":"cephfs.smithi071.vxejye","rank":1,"incarnation":38,"state":"up:active","state_seq":463,"addr":"172.21.15.71:6829/143423621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":143423621},{"type":"v1","addr":"172.21.15.71:6829","nonce":143423621}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_44319":{"gid":44319,"name":"cephfs.smithi192.gtmyst","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6825/3688927981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3688927981},{"type":"v1","addr":"172.21.15.192:6825","nonce":3688927981}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_44793":{"gid":44793,"name":"cephfs.smithi192.azdyze","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6827/3768627645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3768627645},{"type":"v1","addr":"172.21.15.192:6827","nonce":3768627645}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:05.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.412+0000 7f09a8861700 1 -- 172.21.15.71:0/2925642895 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f098c06cda0 msgr2=0x7f098c06f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:05.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.412+0000 7f09a8861700 1 --2- 172.21.15.71:0/2925642895 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f098c06cda0 0x7f098c06f260 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0998005950 tx=0x7f099800b410 comp rx=0 tx=0).stop 2024-04-23T02:55:05.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.412+0000 7f09a8861700 1 -- 172.21.15.71:0/2925642895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09a4105990 msgr2=0x7f09a4195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:05.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.412+0000 7f09a8861700 1 --2- 172.21.15.71:0/2925642895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09a4105990 0x7f09a4195900 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0994006010 tx=0x7f099400bd70 comp rx=0 tx=0).stop 2024-04-23T02:55:05.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.412+0000 7f09a8861700 1 -- 172.21.15.71:0/2925642895 shutdown_connections 2024-04-23T02:55:05.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.412+0000 7f09a8861700 1 --2- 172.21.15.71:0/2925642895 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f098c06cda0 0x7f098c06f260 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.412+0000 7f09a8861700 1 --2- 172.21.15.71:0/2925642895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a40ffac0 0x7f09a41953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.412+0000 7f09a8861700 1 --2- 172.21.15.71:0/2925642895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09a4105990 0x7f09a4195900 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.412+0000 7f09a8861700 1 -- 172.21.15.71:0/2925642895 >> 172.21.15.71:0/2925642895 conn(0x7f09a4075400 msgr2=0x7f09a40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:05.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.412+0000 7f09a8861700 1 -- 172.21.15.71:0/2925642895 shutdown_connections 2024-04-23T02:55:05.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.412+0000 7f09a8861700 1 -- 172.21.15.71:0/2925642895 wait complete. 2024-04-23T02:55:05.416 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 46 2024-04-23T02:55:05.427 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 21, 'max_mds': 2, 'flags': 50} 2024-04-23T02:55:05.427 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-04-23T02:55:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.547+0000 7f926abe4700 1 -- 172.21.15.71:0/1405311891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92640ffac0 msgr2=0x7f92640fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.547+0000 7f926abe4700 1 --2- 172.21.15.71:0/1405311891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92640ffac0 0x7f92640fff20 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f9250009910 tx=0x7f9250009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.547+0000 7f926abe4700 1 -- 172.21.15.71:0/1405311891 shutdown_connections 2024-04-23T02:55:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.547+0000 7f926abe4700 1 --2- 172.21.15.71:0/1405311891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92640ffac0 0x7f92640fff20 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.547+0000 7f926abe4700 1 --2- 172.21.15.71:0/1405311891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9264105990 0x7f9264105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.547+0000 7f926abe4700 1 -- 172.21.15.71:0/1405311891 >> 172.21.15.71:0/1405311891 conn(0x7f9264075400 msgr2=0x7f9264075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:05.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.547+0000 7f926abe4700 1 -- 172.21.15.71:0/1405311891 shutdown_connections 2024-04-23T02:55:05.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.547+0000 7f926abe4700 1 -- 172.21.15.71:0/1405311891 wait complete. 2024-04-23T02:55:05.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.547+0000 7f926abe4700 1 Processor -- start 2024-04-23T02:55:05.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.547+0000 7f926abe4700 1 -- start start 2024-04-23T02:55:05.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.548+0000 7f926abe4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92640ffac0 0x7f9264197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:05.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.548+0000 7f926abe4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9264105990 0x7f9264197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:05.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.548+0000 7f926abe4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9264197e80 con 0x7f9264105990 2024-04-23T02:55:05.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.548+0000 7f926abe4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9264197fc0 con 0x7f92640ffac0 2024-04-23T02:55:05.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.548+0000 7f9263fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9264105990 0x7f9264197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:05.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.548+0000 7f9263fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9264105990 0x7f9264197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60432/0 (socket says 172.21.15.71:60432) 2024-04-23T02:55:05.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.548+0000 7f9263fff700 1 -- 172.21.15.71:0/2169177238 learned_addr learned my addr 172.21.15.71:0/2169177238 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:05.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.548+0000 7f9268980700 1 --2- 172.21.15.71:0/2169177238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92640ffac0 0x7f9264197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:05.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.548+0000 7f9263fff700 1 -- 172.21.15.71:0/2169177238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92640ffac0 msgr2=0x7f9264197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:05.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.548+0000 7f9263fff700 1 --2- 172.21.15.71:0/2169177238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92640ffac0 0x7f9264197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.548+0000 7f9263fff700 1 -- 172.21.15.71:0/2169177238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92500095f0 con 0x7f9264105990 2024-04-23T02:55:05.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.548+0000 7f9268980700 1 --2- 172.21.15.71:0/2169177238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92640ffac0 0x7f9264197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:55:05.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.548+0000 7f9263fff700 1 --2- 172.21.15.71:0/2169177238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9264105990 0x7f9264197860 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f9250006010 tx=0x7f925000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:05.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.548+0000 7f9261ffb700 1 -- 172.21.15.71:0/2169177238 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f925001d070 con 0x7f9264105990 2024-04-23T02:55:05.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.548+0000 7f9261ffb700 1 -- 172.21.15.71:0/2169177238 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f925000f460 con 0x7f9264105990 2024-04-23T02:55:05.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.548+0000 7f926abe4700 1 -- 172.21.15.71:0/2169177238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92641a0390 con 0x7f9264105990 2024-04-23T02:55:05.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.548+0000 7f9261ffb700 1 -- 172.21.15.71:0/2169177238 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9250021640 con 0x7f9264105990 2024-04-23T02:55:05.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.549+0000 7f926abe4700 1 -- 172.21.15.71:0/2169177238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92641a07b0 con 0x7f9264105990 2024-04-23T02:55:05.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.550+0000 7f926abe4700 1 -- 172.21.15.71:0/2169177238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9264107f10 con 0x7f9264105990 2024-04-23T02:55:05.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.550+0000 7f9261ffb700 1 -- 172.21.15.71:0/2169177238 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f925000fa60 con 0x7f9264105990 2024-04-23T02:55:05.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.550+0000 7f9261ffb700 1 --2- 172.21.15.71:0/2169177238 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f925406cda0 0x7f925406f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:05.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.551+0000 7f9261ffb700 1 -- 172.21.15.71:0/2169177238 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f925008d7e0 con 0x7f9264105990 2024-04-23T02:55:05.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.551+0000 7f9268980700 1 --2- 172.21.15.71:0/2169177238 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f925406cda0 0x7f925406f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:05.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.551+0000 7f9268980700 1 --2- 172.21.15.71:0/2169177238 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f925406cda0 0x7f925406f260 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9258005fd0 tx=0x7f9258005dc0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:05.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.555+0000 7f9261ffb700 1 -- 172.21.15.71:0/2169177238 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f92500578d0 con 0x7f9264105990 2024-04-23T02:55:05.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:05 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3515185038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-23T02:55:05.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:55:05.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:05 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/4200221092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-23T02:55:05.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:05 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2925642895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T02:55:05.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.795+0000 7f926abe4700 1 -- 172.21.15.71:0/2169177238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f926402cdd0 con 0x7f9264105990 2024-04-23T02:55:05.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.795+0000 7f9261ffb700 1 -- 172.21.15.71:0/2169177238 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v46) v1 ==== 107+0+4860 (secure 0 0 0) 0x7f925005aef0 con 0x7f9264105990 2024-04-23T02:55:05.798 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:05.798 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:13:02.397905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14478,"mds_1":24243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi071.vxejye","rank":0,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6829/467545394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":467545394},{"type":"v1","addr":"172.21.15.71:6829","nonce":467545394}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14490":{"gid":14490,"name":"cephfs.smithi071.qusrrb","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6827/3442634960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3442634960},{"type":"v1","addr":"172.21.15.71:6827","nonce":3442634960}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24243":{"gid":24243,"name":"cephfs.smithi192.gtmyst","rank":1,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.192:6825/3780549896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3780549896},{"type":"v1","addr":"172.21.15.192:6825","nonce":3780549896}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24255":{"gid":24255,"name":"cephfs.smithi192.azdyze","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.192:6827/2100078284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2100078284},{"type":"v1","addr":"172.21.15.192:6827","nonce":2100078284}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:05.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.798+0000 7f926abe4700 1 -- 172.21.15.71:0/2169177238 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f925406cda0 msgr2=0x7f925406f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:05.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.798+0000 7f926abe4700 1 --2- 172.21.15.71:0/2169177238 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f925406cda0 0x7f925406f260 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9258005fd0 tx=0x7f9258005dc0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.798+0000 7f926abe4700 1 -- 172.21.15.71:0/2169177238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9264105990 msgr2=0x7f9264197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:05.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.798+0000 7f926abe4700 1 --2- 172.21.15.71:0/2169177238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9264105990 0x7f9264197860 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f9250006010 tx=0x7f925000be20 comp rx=0 tx=0).stop 2024-04-23T02:55:05.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.798+0000 7f926abe4700 1 -- 172.21.15.71:0/2169177238 shutdown_connections 2024-04-23T02:55:05.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.798+0000 7f926abe4700 1 --2- 172.21.15.71:0/2169177238 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f925406cda0 0x7f925406f260 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.798+0000 7f926abe4700 1 --2- 172.21.15.71:0/2169177238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92640ffac0 0x7f9264197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.798+0000 7f926abe4700 1 --2- 172.21.15.71:0/2169177238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9264105990 0x7f9264197860 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.798+0000 7f926abe4700 1 -- 172.21.15.71:0/2169177238 >> 172.21.15.71:0/2169177238 conn(0x7f9264075400 msgr2=0x7f92640fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:05.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.798+0000 7f926abe4700 1 -- 172.21.15.71:0/2169177238 shutdown_connections 2024-04-23T02:55:05.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.798+0000 7f926abe4700 1 -- 172.21.15.71:0/2169177238 wait complete. 2024-04-23T02:55:05.801 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 22 2024-04-23T02:55:05.812 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-04-23T02:55:05.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.933+0000 7f248980b700 1 -- 172.21.15.71:0/1874214113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2484105990 msgr2=0x7f2484105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:05.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.933+0000 7f248980b700 1 --2- 172.21.15.71:0/1874214113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2484105990 0x7f2484105d70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2474009910 tx=0x7f2474009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:05.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.933+0000 7f248980b700 1 -- 172.21.15.71:0/1874214113 shutdown_connections 2024-04-23T02:55:05.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.933+0000 7f248980b700 1 --2- 172.21.15.71:0/1874214113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24840ff980 0x7f24840ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.933+0000 7f248980b700 1 --2- 172.21.15.71:0/1874214113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2484105990 0x7f2484105d70 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.933+0000 7f248980b700 1 -- 172.21.15.71:0/1874214113 >> 172.21.15.71:0/1874214113 conn(0x7f24840fb430 msgr2=0x7f24840fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:05.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.933+0000 7f248980b700 1 -- 172.21.15.71:0/1874214113 shutdown_connections 2024-04-23T02:55:05.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.933+0000 7f248980b700 1 -- 172.21.15.71:0/1874214113 wait complete. 2024-04-23T02:55:05.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.933+0000 7f248980b700 1 Processor -- start 2024-04-23T02:55:05.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.934+0000 7f248980b700 1 -- start start 2024-04-23T02:55:05.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.934+0000 7f248980b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24840ff980 0x7f2484195430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:05.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.934+0000 7f248980b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2484105990 0x7f2484195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:05.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.934+0000 7f248980b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2484196050 con 0x7f2484105990 2024-04-23T02:55:05.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.934+0000 7f248980b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2484199dd0 con 0x7f24840ff980 2024-04-23T02:55:05.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.934+0000 7f24827fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2484105990 0x7f2484195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:05.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.934+0000 7f24827fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2484105990 0x7f2484195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60444/0 (socket says 172.21.15.71:60444) 2024-04-23T02:55:05.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.934+0000 7f24827fc700 1 -- 172.21.15.71:0/3111762749 learned_addr learned my addr 172.21.15.71:0/3111762749 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:05.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.934+0000 7f2482ffd700 1 --2- 172.21.15.71:0/3111762749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24840ff980 0x7f2484195430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:05.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.934+0000 7f24827fc700 1 -- 172.21.15.71:0/3111762749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24840ff980 msgr2=0x7f2484195430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:05.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.934+0000 7f24827fc700 1 --2- 172.21.15.71:0/3111762749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24840ff980 0x7f2484195430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:05.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.934+0000 7f24827fc700 1 -- 172.21.15.71:0/3111762749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24740095f0 con 0x7f2484105990 2024-04-23T02:55:05.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.934+0000 7f24827fc700 1 --2- 172.21.15.71:0/3111762749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2484105990 0x7f2484195970 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f247800d8d0 tx=0x7f247800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:05.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.934+0000 7f2488809700 1 -- 172.21.15.71:0/3111762749 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2478009940 con 0x7f2484105990 2024-04-23T02:55:05.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.934+0000 7f2488809700 1 -- 172.21.15.71:0/3111762749 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2478010460 con 0x7f2484105990 2024-04-23T02:55:05.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.934+0000 7f248980b700 1 -- 172.21.15.71:0/3111762749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248419a0b0 con 0x7f2484105990 2024-04-23T02:55:05.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.934+0000 7f2488809700 1 -- 172.21.15.71:0/3111762749 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f247800f5f0 con 0x7f2484105990 2024-04-23T02:55:05.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.934+0000 7f248980b700 1 -- 172.21.15.71:0/3111762749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248419a610 con 0x7f2484105990 2024-04-23T02:55:05.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.935+0000 7f248980b700 1 -- 172.21.15.71:0/3111762749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f248404ea90 con 0x7f2484105990 2024-04-23T02:55:05.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.936+0000 7f2488809700 1 -- 172.21.15.71:0/3111762749 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f24780105d0 con 0x7f2484105990 2024-04-23T02:55:05.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.936+0000 7f2488809700 1 --2- 172.21.15.71:0/3111762749 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f246c06cda0 0x7f246c06f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:05.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.936+0000 7f2488809700 1 -- 172.21.15.71:0/3111762749 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f247808c5e0 con 0x7f2484105990 2024-04-23T02:55:05.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.936+0000 7f2482ffd700 1 --2- 172.21.15.71:0/3111762749 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f246c06cda0 0x7f246c06f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:05.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.937+0000 7f2482ffd700 1 --2- 172.21.15.71:0/3111762749 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f246c06cda0 0x7f246c06f260 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2474006010 tx=0x7f247401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:05.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:05.941+0000 7f2488809700 1 -- 172.21.15.71:0/3111762749 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2478056650 con 0x7f2484105990 2024-04-23T02:55:06.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:05 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3515185038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-23T02:55:06.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:55:06.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:05 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/4200221092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-23T02:55:06.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:05 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2925642895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T02:55:06.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.180+0000 7f248980b700 1 -- 172.21.15.71:0/3111762749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f248419a8d0 con 0x7f2484105990 2024-04-23T02:55:06.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.181+0000 7f2488809700 1 -- 172.21.15.71:0/3111762749 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v46) v1 ==== 107+0+4861 (secure 0 0 0) 0x7f2478010880 con 0x7f2484105990 2024-04-23T02:55:06.183 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:06.183 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:15:23.464717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14478,"mds_1":24243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi071.vxejye","rank":0,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6829/467545394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":467545394},{"type":"v1","addr":"172.21.15.71:6829","nonce":467545394}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14490":{"gid":14490,"name":"cephfs.smithi071.qusrrb","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6827/3442634960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3442634960},{"type":"v1","addr":"172.21.15.71:6827","nonce":3442634960}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24243":{"gid":24243,"name":"cephfs.smithi192.gtmyst","rank":1,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.192:6825/3780549896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3780549896},{"type":"v1","addr":"172.21.15.192:6825","nonce":3780549896}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24255":{"gid":24255,"name":"cephfs.smithi192.azdyze","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.192:6827/2100078284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2100078284},{"type":"v1","addr":"172.21.15.192:6827","nonce":2100078284}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:06.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.183+0000 7f248980b700 1 -- 172.21.15.71:0/3111762749 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f246c06cda0 msgr2=0x7f246c06f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:06.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.183+0000 7f248980b700 1 --2- 172.21.15.71:0/3111762749 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f246c06cda0 0x7f246c06f260 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2474006010 tx=0x7f247401a040 comp rx=0 tx=0).stop 2024-04-23T02:55:06.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.183+0000 7f248980b700 1 -- 172.21.15.71:0/3111762749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2484105990 msgr2=0x7f2484195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:06.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.183+0000 7f248980b700 1 --2- 172.21.15.71:0/3111762749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2484105990 0x7f2484195970 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f247800d8d0 tx=0x7f247800dc90 comp rx=0 tx=0).stop 2024-04-23T02:55:06.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.183+0000 7f248980b700 1 -- 172.21.15.71:0/3111762749 shutdown_connections 2024-04-23T02:55:06.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.183+0000 7f248980b700 1 --2- 172.21.15.71:0/3111762749 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f246c06cda0 0x7f246c06f260 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:06.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.183+0000 7f248980b700 1 --2- 172.21.15.71:0/3111762749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24840ff980 0x7f2484195430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:06.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.183+0000 7f248980b700 1 --2- 172.21.15.71:0/3111762749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2484105990 0x7f2484195970 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:06.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.184+0000 7f248980b700 1 -- 172.21.15.71:0/3111762749 >> 172.21.15.71:0/3111762749 conn(0x7f24840fb430 msgr2=0x7f24840fbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:06.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.184+0000 7f248980b700 1 -- 172.21.15.71:0/3111762749 shutdown_connections 2024-04-23T02:55:06.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.184+0000 7f248980b700 1 -- 172.21.15.71:0/3111762749 wait complete. 2024-04-23T02:55:06.187 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 23 2024-04-23T02:55:06.199 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-04-23T02:55:06.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.318+0000 7fcc5405c700 1 -- 172.21.15.71:0/2794557698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc4c100ac0 msgr2=0x7fcc4c100f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.318+0000 7fcc5405c700 1 --2- 172.21.15.71:0/2794557698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc4c100ac0 0x7fcc4c100f40 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009910 tx=0x7fcc48009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.318+0000 7fcc5405c700 1 -- 172.21.15.71:0/2794557698 shutdown_connections 2024-04-23T02:55:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.318+0000 7fcc5405c700 1 --2- 172.21.15.71:0/2794557698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc4c100ac0 0x7fcc4c100f40 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.318+0000 7fcc5405c700 1 --2- 172.21.15.71:0/2794557698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc4c106ad0 0x7fcc4c106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.318+0000 7fcc5405c700 1 -- 172.21.15.71:0/2794557698 >> 172.21.15.71:0/2794557698 conn(0x7fcc4c0fc5d0 msgr2=0x7fcc4c0fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.318+0000 7fcc5405c700 1 -- 172.21.15.71:0/2794557698 shutdown_connections 2024-04-23T02:55:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.318+0000 7fcc5405c700 1 -- 172.21.15.71:0/2794557698 wait complete. 2024-04-23T02:55:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.319+0000 7fcc5405c700 1 Processor -- start 2024-04-23T02:55:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.319+0000 7fcc5405c700 1 -- start start 2024-04-23T02:55:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.319+0000 7fcc5405c700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc4c100ac0 0x7fcc4c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.319+0000 7fcc5405c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc4c106ad0 0x7fcc4c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:06.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.319+0000 7fcc5405c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c073ba0 con 0x7fcc4c106ad0 2024-04-23T02:55:06.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.319+0000 7fcc5405c700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c073d10 con 0x7fcc4c100ac0 2024-04-23T02:55:06.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.319+0000 7fcc515f7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc4c106ad0 0x7fcc4c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:06.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.319+0000 7fcc515f7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc4c106ad0 0x7fcc4c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60460/0 (socket says 172.21.15.71:60460) 2024-04-23T02:55:06.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.319+0000 7fcc515f7700 1 -- 172.21.15.71:0/2745965261 learned_addr learned my addr 172.21.15.71:0/2745965261 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:06.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.319+0000 7fcc515f7700 1 -- 172.21.15.71:0/2745965261 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc4c100ac0 msgr2=0x7fcc4c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:06.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.319+0000 7fcc515f7700 1 --2- 172.21.15.71:0/2745965261 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc4c100ac0 0x7fcc4c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:06.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.319+0000 7fcc515f7700 1 -- 172.21.15.71:0/2745965261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc480095f0 con 0x7fcc4c106ad0 2024-04-23T02:55:06.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.320+0000 7fcc515f7700 1 --2- 172.21.15.71:0/2745965261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc4c106ad0 0x7fcc4c0735d0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fcc48006010 tx=0x7fcc4800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:06.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.320+0000 7fcc42ffd700 1 -- 172.21.15.71:0/2745965261 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc4801d070 con 0x7fcc4c106ad0 2024-04-23T02:55:06.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.320+0000 7fcc42ffd700 1 -- 172.21.15.71:0/2745965261 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc4800f460 con 0x7fcc4c106ad0 2024-04-23T02:55:06.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.320+0000 7fcc5405c700 1 -- 172.21.15.71:0/2745965261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc4c1a27d0 con 0x7fcc4c106ad0 2024-04-23T02:55:06.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.320+0000 7fcc42ffd700 1 -- 172.21.15.71:0/2745965261 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc48021640 con 0x7fcc4c106ad0 2024-04-23T02:55:06.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.320+0000 7fcc5405c700 1 -- 172.21.15.71:0/2745965261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4c1a2b90 con 0x7fcc4c106ad0 2024-04-23T02:55:06.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.321+0000 7fcc5405c700 1 -- 172.21.15.71:0/2745965261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4c1090e0 con 0x7fcc4c106ad0 2024-04-23T02:55:06.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.321+0000 7fcc42ffd700 1 -- 172.21.15.71:0/2745965261 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fcc4800f9e0 con 0x7fcc4c106ad0 2024-04-23T02:55:06.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.321+0000 7fcc42ffd700 1 --2- 172.21.15.71:0/2745965261 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcc3806cda0 0x7fcc3806f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:06.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.322+0000 7fcc42ffd700 1 -- 172.21.15.71:0/2745965261 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcc4808d900 con 0x7fcc4c106ad0 2024-04-23T02:55:06.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.322+0000 7fcc51df8700 1 --2- 172.21.15.71:0/2745965261 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcc3806cda0 0x7fcc3806f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:06.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.322+0000 7fcc51df8700 1 --2- 172.21.15.71:0/2745965261 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcc3806cda0 0x7fcc3806f260 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0097b0 tx=0x7fcc3c006d20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:06.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.326+0000 7fcc42ffd700 1 -- 172.21.15.71:0/2745965261 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fcc48057970 con 0x7fcc4c106ad0 2024-04-23T02:55:06.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.564+0000 7fcc5405c700 1 -- 172.21.15.71:0/2745965261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7fcc4c02cc70 con 0x7fcc4c106ad0 2024-04-23T02:55:06.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.564+0000 7fcc42ffd700 1 -- 172.21.15.71:0/2745965261 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v46) v1 ==== 107+0+4860 (secure 0 0 0) 0x7fcc4805af90 con 0x7fcc4c106ad0 2024-04-23T02:55:06.567 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:06.567 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:17:16.787570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14478,"mds_1":24243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi071.vxejye","rank":0,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6829/467545394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":467545394},{"type":"v1","addr":"172.21.15.71:6829","nonce":467545394}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14490":{"gid":14490,"name":"cephfs.smithi071.qusrrb","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6827/3442634960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3442634960},{"type":"v1","addr":"172.21.15.71:6827","nonce":3442634960}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24243":{"gid":24243,"name":"cephfs.smithi192.gtmyst","rank":1,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.192:6825/3780549896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3780549896},{"type":"v1","addr":"172.21.15.192:6825","nonce":3780549896}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24255":{"gid":24255,"name":"cephfs.smithi192.azdyze","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.192:6827/2100078284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2100078284},{"type":"v1","addr":"172.21.15.192:6827","nonce":2100078284}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:06.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.566+0000 7fcc5405c700 1 -- 172.21.15.71:0/2745965261 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcc3806cda0 msgr2=0x7fcc3806f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:06.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.566+0000 7fcc5405c700 1 --2- 172.21.15.71:0/2745965261 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcc3806cda0 0x7fcc3806f260 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0097b0 tx=0x7fcc3c006d20 comp rx=0 tx=0).stop 2024-04-23T02:55:06.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.566+0000 7fcc5405c700 1 -- 172.21.15.71:0/2745965261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc4c106ad0 msgr2=0x7fcc4c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:06.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.566+0000 7fcc5405c700 1 --2- 172.21.15.71:0/2745965261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc4c106ad0 0x7fcc4c0735d0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fcc48006010 tx=0x7fcc4800be20 comp rx=0 tx=0).stop 2024-04-23T02:55:06.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.566+0000 7fcc5405c700 1 -- 172.21.15.71:0/2745965261 shutdown_connections 2024-04-23T02:55:06.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.566+0000 7fcc5405c700 1 --2- 172.21.15.71:0/2745965261 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fcc3806cda0 0x7fcc3806f260 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:06.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.566+0000 7fcc5405c700 1 --2- 172.21.15.71:0/2745965261 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc4c100ac0 0x7fcc4c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:06.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.566+0000 7fcc5405c700 1 --2- 172.21.15.71:0/2745965261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc4c106ad0 0x7fcc4c0735d0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:06.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.566+0000 7fcc5405c700 1 -- 172.21.15.71:0/2745965261 >> 172.21.15.71:0/2745965261 conn(0x7fcc4c0fc5d0 msgr2=0x7fcc4c0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:06.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.566+0000 7fcc5405c700 1 -- 172.21.15.71:0/2745965261 shutdown_connections 2024-04-23T02:55:06.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.566+0000 7fcc5405c700 1 -- 172.21.15.71:0/2745965261 wait complete. 2024-04-23T02:55:06.570 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 24 2024-04-23T02:55:06.581 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-04-23T02:55:06.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.702+0000 7f150f48d700 1 -- 172.21.15.71:0/1990676054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15080ffdd0 msgr2=0x7f1508105750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:06.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.702+0000 7f150f48d700 1 --2- 172.21.15.71:0/1990676054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15080ffdd0 0x7f1508105750 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009910 tx=0x7f14f8009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:06.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.702+0000 7f150f48d700 1 -- 172.21.15.71:0/1990676054 shutdown_connections 2024-04-23T02:55:06.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.702+0000 7f150f48d700 1 --2- 172.21.15.71:0/1990676054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15080ffdd0 0x7f1508105750 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:06.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.702+0000 7f150f48d700 1 --2- 172.21.15.71:0/1990676054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15080ff420 0x7f15080ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:06.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.702+0000 7f150f48d700 1 -- 172.21.15.71:0/1990676054 >> 172.21.15.71:0/1990676054 conn(0x7f1508074c20 msgr2=0x7f1508075030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:06.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.703+0000 7f150f48d700 1 -- 172.21.15.71:0/1990676054 shutdown_connections 2024-04-23T02:55:06.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.703+0000 7f150f48d700 1 -- 172.21.15.71:0/1990676054 wait complete. 2024-04-23T02:55:06.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.703+0000 7f150f48d700 1 Processor -- start 2024-04-23T02:55:06.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.703+0000 7f150f48d700 1 -- start start 2024-04-23T02:55:06.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.703+0000 7f150f48d700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15080ff420 0x7f1508100390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:06.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.703+0000 7f150f48d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15081008d0 0x7f1508100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:06.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.703+0000 7f150f48d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15081a26d0 con 0x7f15081008d0 2024-04-23T02:55:06.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.703+0000 7f150f48d700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15081a2810 con 0x7f15080ff420 2024-04-23T02:55:06.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.703+0000 7f150ca28700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15081008d0 0x7f1508100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:06.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.703+0000 7f150ca28700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15081008d0 0x7f1508100d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60480/0 (socket says 172.21.15.71:60480) 2024-04-23T02:55:06.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.703+0000 7f150ca28700 1 -- 172.21.15.71:0/3034398791 learned_addr learned my addr 172.21.15.71:0/3034398791 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:06.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.704+0000 7f150d229700 1 --2- 172.21.15.71:0/3034398791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15080ff420 0x7f1508100390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:06.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.704+0000 7f150ca28700 1 -- 172.21.15.71:0/3034398791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15080ff420 msgr2=0x7f1508100390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:06.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.704+0000 7f150ca28700 1 --2- 172.21.15.71:0/3034398791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15080ff420 0x7f1508100390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:06.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.704+0000 7f150ca28700 1 -- 172.21.15.71:0/3034398791 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f80095f0 con 0x7f15081008d0 2024-04-23T02:55:06.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.704+0000 7f150ca28700 1 --2- 172.21.15.71:0/3034398791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15081008d0 0x7f1508100d50 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f14f800bda0 tx=0x7f14f800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:06.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.704+0000 7f14fe7fc700 1 -- 172.21.15.71:0/3034398791 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14f801d070 con 0x7f15081008d0 2024-04-23T02:55:06.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.704+0000 7f150f48d700 1 -- 172.21.15.71:0/3034398791 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15081a2950 con 0x7f15081008d0 2024-04-23T02:55:06.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.704+0000 7f14fe7fc700 1 -- 172.21.15.71:0/3034398791 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f14f800f460 con 0x7f15081008d0 2024-04-23T02:55:06.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.704+0000 7f14fe7fc700 1 -- 172.21.15.71:0/3034398791 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14f8017630 con 0x7f15081008d0 2024-04-23T02:55:06.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.704+0000 7f150f48d700 1 -- 172.21.15.71:0/3034398791 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15081a2e00 con 0x7f15081008d0 2024-04-23T02:55:06.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.705+0000 7f150f48d700 1 -- 172.21.15.71:0/3034398791 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1508109080 con 0x7f15081008d0 2024-04-23T02:55:06.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.705+0000 7f14fe7fc700 1 -- 172.21.15.71:0/3034398791 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f14f800fa20 con 0x7f15081008d0 2024-04-23T02:55:06.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.706+0000 7f14fe7fc700 1 --2- 172.21.15.71:0/3034398791 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f14f406cda0 0x7f14f406f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:06.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.706+0000 7f14fe7fc700 1 -- 172.21.15.71:0/3034398791 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f14f808d940 con 0x7f15081008d0 2024-04-23T02:55:06.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.706+0000 7f150d229700 1 --2- 172.21.15.71:0/3034398791 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f14f406cda0 0x7f14f406f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:06.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.706+0000 7f150d229700 1 --2- 172.21.15.71:0/3034398791 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f14f406cda0 0x7f14f406f260 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1504005950 tx=0x7f150400b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:06.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.710+0000 7f14fe7fc700 1 -- 172.21.15.71:0/3034398791 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f14f80579b0 con 0x7f15081008d0 2024-04-23T02:55:06.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.952+0000 7f150f48d700 1 -- 172.21.15.71:0/3034398791 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f150804ea90 con 0x7f15081008d0 2024-04-23T02:55:06.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.953+0000 7f14fe7fc700 1 -- 172.21.15.71:0/3034398791 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v46) v1 ==== 107+0+4859 (secure 0 0 0) 0x7f14f805afd0 con 0x7f15081008d0 2024-04-23T02:55:06.958 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:06.958 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:17:27.814232+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14478,"mds_1":24243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi071.vxejye","rank":0,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6829/467545394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":467545394},{"type":"v1","addr":"172.21.15.71:6829","nonce":467545394}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14490":{"gid":14490,"name":"cephfs.smithi071.qusrrb","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6827/3442634960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3442634960},{"type":"v1","addr":"172.21.15.71:6827","nonce":3442634960}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24243":{"gid":24243,"name":"cephfs.smithi192.gtmyst","rank":1,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.192:6825/3780549896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3780549896},{"type":"v1","addr":"172.21.15.192:6825","nonce":3780549896}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24255":{"gid":24255,"name":"cephfs.smithi192.azdyze","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.192:6827/2100078284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2100078284},{"type":"v1","addr":"172.21.15.192:6827","nonce":2100078284}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:06.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.955+0000 7f150f48d700 1 -- 172.21.15.71:0/3034398791 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f14f406cda0 msgr2=0x7f14f406f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:06.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.955+0000 7f150f48d700 1 --2- 172.21.15.71:0/3034398791 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f14f406cda0 0x7f14f406f260 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1504005950 tx=0x7f150400b410 comp rx=0 tx=0).stop 2024-04-23T02:55:06.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.955+0000 7f150f48d700 1 -- 172.21.15.71:0/3034398791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15081008d0 msgr2=0x7f1508100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:06.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.955+0000 7f150f48d700 1 --2- 172.21.15.71:0/3034398791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15081008d0 0x7f1508100d50 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f14f800bda0 tx=0x7f14f800bdd0 comp rx=0 tx=0).stop 2024-04-23T02:55:06.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.955+0000 7f150f48d700 1 -- 172.21.15.71:0/3034398791 shutdown_connections 2024-04-23T02:55:06.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.955+0000 7f150f48d700 1 --2- 172.21.15.71:0/3034398791 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f14f406cda0 0x7f14f406f260 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:06.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.955+0000 7f150f48d700 1 --2- 172.21.15.71:0/3034398791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15080ff420 0x7f1508100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:06.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.955+0000 7f150f48d700 1 --2- 172.21.15.71:0/3034398791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15081008d0 0x7f1508100d50 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:06.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.955+0000 7f150f48d700 1 -- 172.21.15.71:0/3034398791 >> 172.21.15.71:0/3034398791 conn(0x7f1508074c20 msgr2=0x7f15080fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:06.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.955+0000 7f150f48d700 1 -- 172.21.15.71:0/3034398791 shutdown_connections 2024-04-23T02:55:06.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:06.955+0000 7f150f48d700 1 -- 172.21.15.71:0/3034398791 wait complete. 2024-04-23T02:55:06.961 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 25 2024-04-23T02:55:06.970 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:06 smithi071 ceph-mon[144544]: pgmap v1256: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:55:06.970 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:06 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2169177238' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-23T02:55:06.971 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:06 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3111762749' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-23T02:55:06.971 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:06 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2745965261' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-23T02:55:06.972 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-04-23T02:55:07.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.092+0000 7f0267b2e700 1 -- 172.21.15.71:0/2067509994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0260100530 msgr2=0x7f0260100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:07.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.092+0000 7f0267b2e700 1 --2- 172.21.15.71:0/2067509994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0260100530 0x7f0260100910 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0250009910 tx=0x7f0250009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:07.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.092+0000 7f0267b2e700 1 -- 172.21.15.71:0/2067509994 shutdown_connections 2024-04-23T02:55:07.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.092+0000 7f0267b2e700 1 --2- 172.21.15.71:0/2067509994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0260100ee0 0x7f02601045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:07.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.092+0000 7f0267b2e700 1 --2- 172.21.15.71:0/2067509994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0260100530 0x7f0260100910 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:07.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.092+0000 7f0267b2e700 1 -- 172.21.15.71:0/2067509994 >> 172.21.15.71:0/2067509994 conn(0x7f0260074c20 msgr2=0x7f0260075030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:07.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.092+0000 7f0267b2e700 1 -- 172.21.15.71:0/2067509994 shutdown_connections 2024-04-23T02:55:07.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.092+0000 7f0267b2e700 1 -- 172.21.15.71:0/2067509994 wait complete. 2024-04-23T02:55:07.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.092+0000 7f0267b2e700 1 Processor -- start 2024-04-23T02:55:07.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.092+0000 7f0267b2e700 1 -- start start 2024-04-23T02:55:07.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.092+0000 7f0267b2e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0260100530 0x7f0260101680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:07.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.092+0000 7f0267b2e700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0260101bc0 0x7f0260102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:07.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.092+0000 7f0267b2e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02601a2700 con 0x7f0260100530 2024-04-23T02:55:07.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.092+0000 7f0267b2e700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02601a2840 con 0x7f0260101bc0 2024-04-23T02:55:07.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.093+0000 7f02658ca700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0260100530 0x7f0260101680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:07.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.093+0000 7f02658ca700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0260100530 0x7f0260101680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60500/0 (socket says 172.21.15.71:60500) 2024-04-23T02:55:07.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.093+0000 7f02658ca700 1 -- 172.21.15.71:0/4012753507 learned_addr learned my addr 172.21.15.71:0/4012753507 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:07.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.093+0000 7f02658ca700 1 -- 172.21.15.71:0/4012753507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0260101bc0 msgr2=0x7f0260102040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:07.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.093+0000 7f02658ca700 1 --2- 172.21.15.71:0/4012753507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0260101bc0 0x7f0260102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:07.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.093+0000 7f02658ca700 1 -- 172.21.15.71:0/4012753507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02500095f0 con 0x7f0260100530 2024-04-23T02:55:07.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.093+0000 7f02658ca700 1 --2- 172.21.15.71:0/4012753507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0260100530 0x7f0260101680 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f025000bd70 tx=0x7f025000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:07.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.093+0000 7f0256ffd700 1 -- 172.21.15.71:0/4012753507 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f025001d070 con 0x7f0260100530 2024-04-23T02:55:07.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.093+0000 7f0267b2e700 1 -- 172.21.15.71:0/4012753507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02601a2980 con 0x7f0260100530 2024-04-23T02:55:07.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.093+0000 7f0256ffd700 1 -- 172.21.15.71:0/4012753507 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f025000f460 con 0x7f0260100530 2024-04-23T02:55:07.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.093+0000 7f0256ffd700 1 -- 172.21.15.71:0/4012753507 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0250017630 con 0x7f0260100530 2024-04-23T02:55:07.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.093+0000 7f0267b2e700 1 -- 172.21.15.71:0/4012753507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02601a2e30 con 0x7f0260100530 2024-04-23T02:55:07.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.094+0000 7f0267b2e700 1 -- 172.21.15.71:0/4012753507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0260108f60 con 0x7f0260100530 2024-04-23T02:55:07.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.094+0000 7f0256ffd700 1 -- 172.21.15.71:0/4012753507 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f025000fa60 con 0x7f0260100530 2024-04-23T02:55:07.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.095+0000 7f0256ffd700 1 --2- 172.21.15.71:0/4012753507 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f024c06cda0 0x7f024c06f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:07.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.095+0000 7f0256ffd700 1 -- 172.21.15.71:0/4012753507 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f025008db00 con 0x7f0260100530 2024-04-23T02:55:07.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.095+0000 7f02650c9700 1 --2- 172.21.15.71:0/4012753507 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f024c06cda0 0x7f024c06f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:07.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.095+0000 7f02650c9700 1 --2- 172.21.15.71:0/4012753507 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f024c06cda0 0x7f024c06f260 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0260073000 tx=0x7f025c005c80 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:07.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.099+0000 7f0256ffd700 1 -- 172.21.15.71:0/4012753507 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0250057b70 con 0x7f0260100530 2024-04-23T02:55:07.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:06 smithi192 ceph-mon[108617]: pgmap v1256: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:55:07.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:06 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2169177238' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-23T02:55:07.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:06 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3111762749' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-23T02:55:07.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:06 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2745965261' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-23T02:55:07.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.342+0000 7f0267b2e700 1 -- 172.21.15.71:0/4012753507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f026004ea90 con 0x7f0260100530 2024-04-23T02:55:07.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.343+0000 7f0256ffd700 1 -- 172.21.15.71:0/4012753507 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v46) v1 ==== 107+0+3284 (secure 0 0 0) 0x7f025005b190 con 0x7f0260100530 2024-04-23T02:55:07.345 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:07.345 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:22:00.113733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14478,"mds_1":24243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi071.vxejye","rank":0,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.71:6829/467545394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":467545394},{"type":"v1","addr":"172.21.15.71:6829","nonce":467545394}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24243":{"gid":24243,"name":"cephfs.smithi192.gtmyst","rank":1,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.192:6825/3780549896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3780549896},{"type":"v1","addr":"172.21.15.192:6825","nonce":3780549896}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:07.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.345+0000 7f0267b2e700 1 -- 172.21.15.71:0/4012753507 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f024c06cda0 msgr2=0x7f024c06f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:07.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.345+0000 7f0267b2e700 1 --2- 172.21.15.71:0/4012753507 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f024c06cda0 0x7f024c06f260 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0260073000 tx=0x7f025c005c80 comp rx=0 tx=0).stop 2024-04-23T02:55:07.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.345+0000 7f0267b2e700 1 -- 172.21.15.71:0/4012753507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0260100530 msgr2=0x7f0260101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:07.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.345+0000 7f0267b2e700 1 --2- 172.21.15.71:0/4012753507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0260100530 0x7f0260101680 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f025000bd70 tx=0x7f025000be50 comp rx=0 tx=0).stop 2024-04-23T02:55:07.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.345+0000 7f0267b2e700 1 -- 172.21.15.71:0/4012753507 shutdown_connections 2024-04-23T02:55:07.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.345+0000 7f0267b2e700 1 --2- 172.21.15.71:0/4012753507 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f024c06cda0 0x7f024c06f260 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:07.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.345+0000 7f0267b2e700 1 --2- 172.21.15.71:0/4012753507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0260100530 0x7f0260101680 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:07.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.345+0000 7f0267b2e700 1 --2- 172.21.15.71:0/4012753507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0260101bc0 0x7f0260102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:07.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.345+0000 7f0267b2e700 1 -- 172.21.15.71:0/4012753507 >> 172.21.15.71:0/4012753507 conn(0x7f0260074c20 msgr2=0x7f0260103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:07.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.345+0000 7f0267b2e700 1 -- 172.21.15.71:0/4012753507 shutdown_connections 2024-04-23T02:55:07.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.345+0000 7f0267b2e700 1 -- 172.21.15.71:0/4012753507 wait complete. 2024-04-23T02:55:07.348 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 26 2024-04-23T02:55:07.360 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-04-23T02:55:07.360 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-04-23T02:55:07.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.481+0000 7f7bd1619700 1 -- 172.21.15.71:0/1828241082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bcc105990 msgr2=0x7f7bcc105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:07.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.481+0000 7f7bd1619700 1 --2- 172.21.15.71:0/1828241082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bcc105990 0x7f7bcc105d70 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400b0a0 tx=0x7f7bc400b3b0 comp rx=0 tx=0).stop 2024-04-23T02:55:07.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.481+0000 7f7bd1619700 1 -- 172.21.15.71:0/1828241082 shutdown_connections 2024-04-23T02:55:07.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.481+0000 7f7bd1619700 1 --2- 172.21.15.71:0/1828241082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bcc0ffac0 0x7f7bcc0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:07.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.481+0000 7f7bd1619700 1 --2- 172.21.15.71:0/1828241082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bcc105990 0x7f7bcc105d70 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:07.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.481+0000 7f7bd1619700 1 -- 172.21.15.71:0/1828241082 >> 172.21.15.71:0/1828241082 conn(0x7f7bcc075400 msgr2=0x7f7bcc075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:07.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.481+0000 7f7bd1619700 1 -- 172.21.15.71:0/1828241082 shutdown_connections 2024-04-23T02:55:07.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.481+0000 7f7bd1619700 1 -- 172.21.15.71:0/1828241082 wait complete. 2024-04-23T02:55:07.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.481+0000 7f7bd1619700 1 Processor -- start 2024-04-23T02:55:07.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.482+0000 7f7bd1619700 1 -- start start 2024-04-23T02:55:07.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.482+0000 7f7bd1619700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bcc0ffac0 0x7f7bcc195400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:07.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.482+0000 7f7bd1619700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bcc105990 0x7f7bcc195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:07.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.482+0000 7f7bd1619700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc196020 con 0x7f7bcc105990 2024-04-23T02:55:07.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.482+0000 7f7bd1619700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc199da0 con 0x7f7bcc0ffac0 2024-04-23T02:55:07.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.482+0000 7f7bca7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bcc105990 0x7f7bcc195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.482+0000 7f7bca7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bcc105990 0x7f7bcc195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60520/0 (socket says 172.21.15.71:60520) 2024-04-23T02:55:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.482+0000 7f7bca7fc700 1 -- 172.21.15.71:0/3179835581 learned_addr learned my addr 172.21.15.71:0/3179835581 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.482+0000 7f7bcaffd700 1 --2- 172.21.15.71:0/3179835581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bcc0ffac0 0x7f7bcc195400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.482+0000 7f7bca7fc700 1 -- 172.21.15.71:0/3179835581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bcc0ffac0 msgr2=0x7f7bcc195400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.482+0000 7f7bca7fc700 1 --2- 172.21.15.71:0/3179835581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bcc0ffac0 0x7f7bcc195400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.482+0000 7f7bca7fc700 1 -- 172.21.15.71:0/3179835581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc4009d00 con 0x7f7bcc105990 2024-04-23T02:55:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.482+0000 7f7bcaffd700 1 --2- 172.21.15.71:0/3179835581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bcc0ffac0 0x7f7bcc195400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:55:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.483+0000 7f7bca7fc700 1 --2- 172.21.15.71:0/3179835581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bcc105990 0x7f7bcc195940 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400d8d0 tx=0x7f7bb400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.483+0000 7f7bc3fff700 1 -- 172.21.15.71:0/3179835581 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bb4009940 con 0x7f7bcc105990 2024-04-23T02:55:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.483+0000 7f7bd1619700 1 -- 172.21.15.71:0/3179835581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bcc19a080 con 0x7f7bcc105990 2024-04-23T02:55:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.483+0000 7f7bc3fff700 1 -- 172.21.15.71:0/3179835581 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7bb4010460 con 0x7f7bcc105990 2024-04-23T02:55:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.483+0000 7f7bc3fff700 1 -- 172.21.15.71:0/3179835581 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bb400f5f0 con 0x7f7bcc105990 2024-04-23T02:55:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.483+0000 7f7bd1619700 1 -- 172.21.15.71:0/3179835581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc19a5e0 con 0x7f7bcc105990 2024-04-23T02:55:07.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.484+0000 7f7bd1619700 1 -- 172.21.15.71:0/3179835581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bcc107f30 con 0x7f7bcc105990 2024-04-23T02:55:07.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.484+0000 7f7bc3fff700 1 -- 172.21.15.71:0/3179835581 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7bb40105d0 con 0x7f7bcc105990 2024-04-23T02:55:07.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.484+0000 7f7bc3fff700 1 --2- 172.21.15.71:0/3179835581 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7bb806cda0 0x7f7bb806f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:07.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.485+0000 7f7bc3fff700 1 -- 172.21.15.71:0/3179835581 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7bb408c5e0 con 0x7f7bcc105990 2024-04-23T02:55:07.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.485+0000 7f7bcaffd700 1 --2- 172.21.15.71:0/3179835581 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7bb806cda0 0x7f7bb806f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:07.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.485+0000 7f7bcaffd700 1 --2- 172.21.15.71:0/3179835581 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7bb806cda0 0x7f7bb806f260 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4000f80 tx=0x7f7bc400b7f0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:07.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.489+0000 7f7bc3fff700 1 -- 172.21.15.71:0/3179835581 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7bb4056650 con 0x7f7bcc105990 2024-04-23T02:55:07.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.729+0000 7f7bd1619700 1 -- 172.21.15.71:0/3179835581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f7bcc19a8c0 con 0x7f7bcc105990 2024-04-23T02:55:07.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.730+0000 7f7bc3fff700 1 -- 172.21.15.71:0/3179835581 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v46) v1 ==== 107+0+3263 (secure 0 0 0) 0x7f7bb4020cd0 con 0x7f7bcc105990 2024-04-23T02:55:07.732 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:07.732 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34318,"name":"cephfs.smithi071.qusrrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6827/1456175422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1456175422},{"type":"v1","addr":"172.21.15.71:6827","nonce":1456175422}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44263,"name":"cephfs.smithi192.azdyze","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/418886657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":418886657},{"type":"v1","addr":"172.21.15.192:6827","nonce":418886657}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:22:00.280923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:07.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.732+0000 7f7bd1619700 1 -- 172.21.15.71:0/3179835581 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7bb806cda0 msgr2=0x7f7bb806f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:07.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.732+0000 7f7bd1619700 1 --2- 172.21.15.71:0/3179835581 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7bb806cda0 0x7f7bb806f260 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4000f80 tx=0x7f7bc400b7f0 comp rx=0 tx=0).stop 2024-04-23T02:55:07.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.732+0000 7f7bd1619700 1 -- 172.21.15.71:0/3179835581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bcc105990 msgr2=0x7f7bcc195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:07.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.732+0000 7f7bd1619700 1 --2- 172.21.15.71:0/3179835581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bcc105990 0x7f7bcc195940 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400d8d0 tx=0x7f7bb400dc90 comp rx=0 tx=0).stop 2024-04-23T02:55:07.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.732+0000 7f7bd1619700 1 -- 172.21.15.71:0/3179835581 shutdown_connections 2024-04-23T02:55:07.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.732+0000 7f7bd1619700 1 --2- 172.21.15.71:0/3179835581 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f7bb806cda0 0x7f7bb806f260 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:07.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.732+0000 7f7bd1619700 1 --2- 172.21.15.71:0/3179835581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bcc0ffac0 0x7f7bcc195400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:07.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.732+0000 7f7bd1619700 1 --2- 172.21.15.71:0/3179835581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bcc105990 0x7f7bcc195940 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:07.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.732+0000 7f7bd1619700 1 -- 172.21.15.71:0/3179835581 >> 172.21.15.71:0/3179835581 conn(0x7f7bcc075400 msgr2=0x7f7bcc0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:07.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.732+0000 7f7bd1619700 1 -- 172.21.15.71:0/3179835581 shutdown_connections 2024-04-23T02:55:07.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.733+0000 7f7bd1619700 1 -- 172.21.15.71:0/3179835581 wait complete. 2024-04-23T02:55:07.736 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 27 2024-04-23T02:55:07.747 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-23T02:55:07.747 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-04-23T02:55:07.747 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-04-23T02:55:07.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.866+0000 7f9959335700 1 -- 172.21.15.71:0/2613285488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99540ff230 msgr2=0x7f99540ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:07.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.866+0000 7f9959335700 1 --2- 172.21.15.71:0/2613285488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99540ff230 0x7f99540ff690 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f9944009910 tx=0x7f9944009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:07.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.866+0000 7f9959335700 1 -- 172.21.15.71:0/2613285488 shutdown_connections 2024-04-23T02:55:07.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.866+0000 7f9959335700 1 --2- 172.21.15.71:0/2613285488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99540ff230 0x7f99540ff690 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:07.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.866+0000 7f9959335700 1 --2- 172.21.15.71:0/2613285488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99541051f0 0x7f99541055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:07.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.866+0000 7f9959335700 1 -- 172.21.15.71:0/2613285488 >> 172.21.15.71:0/2613285488 conn(0x7f9954074b60 msgr2=0x7f9954074f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:07.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.866+0000 7f9959335700 1 -- 172.21.15.71:0/2613285488 shutdown_connections 2024-04-23T02:55:07.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.866+0000 7f9959335700 1 -- 172.21.15.71:0/2613285488 wait complete. 2024-04-23T02:55:07.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.867+0000 7f9959335700 1 Processor -- start 2024-04-23T02:55:07.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.867+0000 7f9959335700 1 -- start start 2024-04-23T02:55:07.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.867+0000 7f9959335700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99540ff230 0x7f9954195450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:07.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.867+0000 7f9959335700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99541051f0 0x7f9954195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:07.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.867+0000 7f9959335700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9954196070 con 0x7f99540ff230 2024-04-23T02:55:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.867+0000 7f9959335700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9954199df0 con 0x7f99541051f0 2024-04-23T02:55:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.867+0000 7f9952ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99540ff230 0x7f9954195450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.867+0000 7f9952ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99540ff230 0x7f9954195450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60532/0 (socket says 172.21.15.71:60532) 2024-04-23T02:55:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.867+0000 7f9952ffd700 1 -- 172.21.15.71:0/1843845328 learned_addr learned my addr 172.21.15.71:0/1843845328 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.867+0000 7f99527fc700 1 --2- 172.21.15.71:0/1843845328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99541051f0 0x7f9954195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.867+0000 7f9952ffd700 1 -- 172.21.15.71:0/1843845328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99541051f0 msgr2=0x7f9954195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.867+0000 7f9952ffd700 1 --2- 172.21.15.71:0/1843845328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99541051f0 0x7f9954195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.867+0000 7f9952ffd700 1 -- 172.21.15.71:0/1843845328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99440095f0 con 0x7f99540ff230 2024-04-23T02:55:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.867+0000 7f99527fc700 1 --2- 172.21.15.71:0/1843845328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99541051f0 0x7f9954195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:55:07.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.868+0000 7f9952ffd700 1 --2- 172.21.15.71:0/1843845328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99540ff230 0x7f9954195450 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f993c00d8d0 tx=0x7f993c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:07.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.868+0000 7f994bfff700 1 -- 172.21.15.71:0/1843845328 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f993c009940 con 0x7f99540ff230 2024-04-23T02:55:07.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.868+0000 7f994bfff700 1 -- 172.21.15.71:0/1843845328 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f993c010460 con 0x7f99540ff230 2024-04-23T02:55:07.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.868+0000 7f9959335700 1 -- 172.21.15.71:0/1843845328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995419a0d0 con 0x7f99540ff230 2024-04-23T02:55:07.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.868+0000 7f994bfff700 1 -- 172.21.15.71:0/1843845328 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f993c00f5f0 con 0x7f99540ff230 2024-04-23T02:55:07.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.868+0000 7f9959335700 1 -- 172.21.15.71:0/1843845328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995419a630 con 0x7f99540ff230 2024-04-23T02:55:07.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.869+0000 7f9959335700 1 -- 172.21.15.71:0/1843845328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f995404ea90 con 0x7f99540ff230 2024-04-23T02:55:07.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.869+0000 7f994bfff700 1 -- 172.21.15.71:0/1843845328 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f993c0105d0 con 0x7f99540ff230 2024-04-23T02:55:07.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.870+0000 7f994bfff700 1 --2- 172.21.15.71:0/1843845328 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f994006cda0 0x7f994006f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:07.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.870+0000 7f994bfff700 1 -- 172.21.15.71:0/1843845328 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f993c08c680 con 0x7f99540ff230 2024-04-23T02:55:07.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.870+0000 7f99527fc700 1 --2- 172.21.15.71:0/1843845328 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f994006cda0 0x7f994006f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:07.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.870+0000 7f99527fc700 1 --2- 172.21.15.71:0/1843845328 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f994006cda0 0x7f994006f260 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9944005b40 tx=0x7f994401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:07.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:07.874+0000 7f994bfff700 1 -- 172.21.15.71:0/1843845328 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f993c056650 con 0x7f99540ff230 2024-04-23T02:55:07.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:07 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3034398791' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-23T02:55:07.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:07 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/4012753507' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-23T02:55:08.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.113+0000 7f9959335700 1 -- 172.21.15.71:0/1843845328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f995419a950 con 0x7f99540ff230 2024-04-23T02:55:08.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.113+0000 7f994bfff700 1 -- 172.21.15.71:0/1843845328 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v46) v1 ==== 107+0+4824 (secure 0 0 0) 0x7f993c010880 con 0x7f99540ff230 2024-04-23T02:55:08.116 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:08.116 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34318,"name":"cephfs.smithi071.qusrrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6827/1456175422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1456175422},{"type":"v1","addr":"172.21.15.71:6827","nonce":1456175422}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44263,"name":"cephfs.smithi192.azdyze","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/418886657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":418886657},{"type":"v1","addr":"172.21.15.192:6827","nonce":418886657}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44271,"name":"cephfs.smithi192.gtmyst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/1096782156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1096782156},{"type":"v1","addr":"172.21.15.192:6825","nonce":1096782156}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":44273,"name":"cephfs.smithi071.vxejye","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6829/3220156840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3220156840},{"type":"v1","addr":"172.21.15.71:6829","nonce":3220156840}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:22:00.280923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:08.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.115+0000 7f9959335700 1 -- 172.21.15.71:0/1843845328 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f994006cda0 msgr2=0x7f994006f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:08.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.115+0000 7f9959335700 1 --2- 172.21.15.71:0/1843845328 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f994006cda0 0x7f994006f260 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9944005b40 tx=0x7f994401a040 comp rx=0 tx=0).stop 2024-04-23T02:55:08.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.115+0000 7f9959335700 1 -- 172.21.15.71:0/1843845328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99540ff230 msgr2=0x7f9954195450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:08.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.115+0000 7f9959335700 1 --2- 172.21.15.71:0/1843845328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99540ff230 0x7f9954195450 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f993c00d8d0 tx=0x7f993c00dc90 comp rx=0 tx=0).stop 2024-04-23T02:55:08.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.115+0000 7f9959335700 1 -- 172.21.15.71:0/1843845328 shutdown_connections 2024-04-23T02:55:08.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.115+0000 7f9959335700 1 --2- 172.21.15.71:0/1843845328 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f994006cda0 0x7f994006f260 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:08.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.115+0000 7f9959335700 1 --2- 172.21.15.71:0/1843845328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99540ff230 0x7f9954195450 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:08.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.115+0000 7f9959335700 1 --2- 172.21.15.71:0/1843845328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99541051f0 0x7f9954195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:08.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.115+0000 7f9959335700 1 -- 172.21.15.71:0/1843845328 >> 172.21.15.71:0/1843845328 conn(0x7f9954074b60 msgr2=0x7f99540fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:08.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.116+0000 7f9959335700 1 -- 172.21.15.71:0/1843845328 shutdown_connections 2024-04-23T02:55:08.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.116+0000 7f9959335700 1 -- 172.21.15.71:0/1843845328 wait complete. 2024-04-23T02:55:08.119 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 28 2024-04-23T02:55:08.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:07 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3034398791' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-23T02:55:08.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:07 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/4012753507' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-23T02:55:08.130 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-23T02:55:08.130 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-04-23T02:55:08.130 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-04-23T02:55:08.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.249+0000 7f25ccc85700 1 -- 172.21.15.71:0/1792541521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25c8105960 msgr2=0x7f25c8105d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:08.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.249+0000 7f25ccc85700 1 --2- 172.21.15.71:0/1792541521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25c8105960 0x7f25c8105d40 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009910 tx=0x7f25bc009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:08.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.249+0000 7f25ccc85700 1 -- 172.21.15.71:0/1792541521 shutdown_connections 2024-04-23T02:55:08.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.249+0000 7f25ccc85700 1 --2- 172.21.15.71:0/1792541521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25c80ff950 0x7f25c80ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:08.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.249+0000 7f25ccc85700 1 --2- 172.21.15.71:0/1792541521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25c8105960 0x7f25c8105d40 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:08.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.249+0000 7f25ccc85700 1 -- 172.21.15.71:0/1792541521 >> 172.21.15.71:0/1792541521 conn(0x7f25c80fb460 msgr2=0x7f25c80fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:08.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.249+0000 7f25ccc85700 1 -- 172.21.15.71:0/1792541521 shutdown_connections 2024-04-23T02:55:08.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.249+0000 7f25ccc85700 1 -- 172.21.15.71:0/1792541521 wait complete. 2024-04-23T02:55:08.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.250+0000 7f25ccc85700 1 Processor -- start 2024-04-23T02:55:08.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.250+0000 7f25ccc85700 1 -- start start 2024-04-23T02:55:08.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.250+0000 7f25ccc85700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25c80ff950 0x7f25c8195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:08.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.250+0000 7f25ccc85700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25c8105960 0x7f25c81959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:08.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.250+0000 7f25ccc85700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c8196090 con 0x7f25c80ff950 2024-04-23T02:55:08.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.250+0000 7f25ccc85700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c8199e10 con 0x7f25c8105960 2024-04-23T02:55:08.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.250+0000 7f25c659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25c80ff950 0x7f25c8195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:08.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.251+0000 7f25c659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25c80ff950 0x7f25c8195470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60552/0 (socket says 172.21.15.71:60552) 2024-04-23T02:55:08.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.251+0000 7f25c659c700 1 -- 172.21.15.71:0/2698021059 learned_addr learned my addr 172.21.15.71:0/2698021059 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.251+0000 7f25c5d9b700 1 --2- 172.21.15.71:0/2698021059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25c8105960 0x7f25c81959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.251+0000 7f25c659c700 1 -- 172.21.15.71:0/2698021059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25c8105960 msgr2=0x7f25c81959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.251+0000 7f25c659c700 1 --2- 172.21.15.71:0/2698021059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25c8105960 0x7f25c81959b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.251+0000 7f25c659c700 1 -- 172.21.15.71:0/2698021059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc0095f0 con 0x7f25c80ff950 2024-04-23T02:55:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.251+0000 7f25c659c700 1 --2- 172.21.15.71:0/2698021059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25c80ff950 0x7f25c8195470 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f25bc006010 tx=0x7f25bc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.251+0000 7f25b77fe700 1 -- 172.21.15.71:0/2698021059 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25bc01f070 con 0x7f25c80ff950 2024-04-23T02:55:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.251+0000 7f25b77fe700 1 -- 172.21.15.71:0/2698021059 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f25bc019610 con 0x7f25c80ff950 2024-04-23T02:55:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.251+0000 7f25ccc85700 1 -- 172.21.15.71:0/2698021059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c819a090 con 0x7f25c80ff950 2024-04-23T02:55:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.251+0000 7f25b77fe700 1 -- 172.21.15.71:0/2698021059 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25bc00f670 con 0x7f25c80ff950 2024-04-23T02:55:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.251+0000 7f25ccc85700 1 -- 172.21.15.71:0/2698021059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c819a590 con 0x7f25c80ff950 2024-04-23T02:55:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.252+0000 7f25ccc85700 1 -- 172.21.15.71:0/2698021059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c804ea90 con 0x7f25c80ff950 2024-04-23T02:55:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.253+0000 7f25b77fe700 1 -- 172.21.15.71:0/2698021059 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f25bc019bf0 con 0x7f25c80ff950 2024-04-23T02:55:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.254+0000 7f25b77fe700 1 --2- 172.21.15.71:0/2698021059 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f25b006cda0 0x7f25b006f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.254+0000 7f25c5d9b700 1 --2- 172.21.15.71:0/2698021059 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f25b006cda0 0x7f25b006f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.254+0000 7f25b77fe700 1 -- 172.21.15.71:0/2698021059 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f25bc08fc40 con 0x7f25c80ff950 2024-04-23T02:55:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.254+0000 7f25c5d9b700 1 --2- 172.21.15.71:0/2698021059 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f25b006cda0 0x7f25b006f260 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f25b8005950 tx=0x7f25b80058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:08.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.258+0000 7f25b77fe700 1 -- 172.21.15.71:0/2698021059 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f25bc059cd0 con 0x7f25c80ff950 2024-04-23T02:55:08.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.492+0000 7f25ccc85700 1 -- 172.21.15.71:0/2698021059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f25c819a8b0 con 0x7f25c80ff950 2024-04-23T02:55:08.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.493+0000 7f25b77fe700 1 -- 172.21.15.71:0/2698021059 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v46) v1 ==== 107+0+4045 (secure 0 0 0) 0x7f25bc028090 con 0x7f25c80ff950 2024-04-23T02:55:08.495 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:08.495 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44263,"name":"cephfs.smithi192.azdyze","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/418886657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":418886657},{"type":"v1","addr":"172.21.15.192:6827","nonce":418886657}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44271,"name":"cephfs.smithi192.gtmyst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/1096782156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1096782156},{"type":"v1","addr":"172.21.15.192:6825","nonce":1096782156}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":44273,"name":"cephfs.smithi071.vxejye","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6829/3220156840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3220156840},{"type":"v1","addr":"172.21.15.71:6829","nonce":3220156840}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:22:00.280923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:08.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.495+0000 7f25ccc85700 1 -- 172.21.15.71:0/2698021059 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f25b006cda0 msgr2=0x7f25b006f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:08.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.495+0000 7f25ccc85700 1 --2- 172.21.15.71:0/2698021059 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f25b006cda0 0x7f25b006f260 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f25b8005950 tx=0x7f25b80058e0 comp rx=0 tx=0).stop 2024-04-23T02:55:08.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.495+0000 7f25ccc85700 1 -- 172.21.15.71:0/2698021059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25c80ff950 msgr2=0x7f25c8195470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:08.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.495+0000 7f25ccc85700 1 --2- 172.21.15.71:0/2698021059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25c80ff950 0x7f25c8195470 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f25bc006010 tx=0x7f25bc00be20 comp rx=0 tx=0).stop 2024-04-23T02:55:08.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.495+0000 7f25ccc85700 1 -- 172.21.15.71:0/2698021059 shutdown_connections 2024-04-23T02:55:08.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.495+0000 7f25ccc85700 1 --2- 172.21.15.71:0/2698021059 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f25b006cda0 0x7f25b006f260 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:08.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.495+0000 7f25ccc85700 1 --2- 172.21.15.71:0/2698021059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25c80ff950 0x7f25c8195470 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:08.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.495+0000 7f25ccc85700 1 --2- 172.21.15.71:0/2698021059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25c8105960 0x7f25c81959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:08.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.495+0000 7f25ccc85700 1 -- 172.21.15.71:0/2698021059 >> 172.21.15.71:0/2698021059 conn(0x7f25c80fb460 msgr2=0x7f25c80fbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:08.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.495+0000 7f25ccc85700 1 -- 172.21.15.71:0/2698021059 shutdown_connections 2024-04-23T02:55:08.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.495+0000 7f25ccc85700 1 -- 172.21.15.71:0/2698021059 wait complete. 2024-04-23T02:55:08.499 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 29 2024-04-23T02:55:08.510 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-23T02:55:08.510 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-04-23T02:55:08.510 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-04-23T02:55:08.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.630+0000 7fbc5f166700 1 -- 172.21.15.71:0/264179773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc58106a80 msgr2=0x7fbc58106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:08.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.630+0000 7fbc5f166700 1 --2- 172.21.15.71:0/264179773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc58106a80 0x7fbc58106e60 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009910 tx=0x7fbc4c009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:08.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.630+0000 7fbc5f166700 1 -- 172.21.15.71:0/264179773 shutdown_connections 2024-04-23T02:55:08.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.630+0000 7fbc5f166700 1 --2- 172.21.15.71:0/264179773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc58067da0 0x7fbc58068220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:08.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.630+0000 7fbc5f166700 1 --2- 172.21.15.71:0/264179773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc58106a80 0x7fbc58106e60 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:08.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.630+0000 7fbc5f166700 1 -- 172.21.15.71:0/264179773 >> 172.21.15.71:0/264179773 conn(0x7fbc58075400 msgr2=0x7fbc58075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:08.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.630+0000 7fbc5f166700 1 -- 172.21.15.71:0/264179773 shutdown_connections 2024-04-23T02:55:08.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.630+0000 7fbc5f166700 1 -- 172.21.15.71:0/264179773 wait complete. 2024-04-23T02:55:08.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.630+0000 7fbc5f166700 1 Processor -- start 2024-04-23T02:55:08.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.630+0000 7fbc5f166700 1 -- start start 2024-04-23T02:55:08.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.631+0000 7fbc5f166700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc58067da0 0x7fbc581964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:08.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.631+0000 7fbc5f166700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc58106a80 0x7fbc581969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:08.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.631+0000 7fbc5f166700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc581970d0 con 0x7fbc58106a80 2024-04-23T02:55:08.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.631+0000 7fbc5f166700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5819ae50 con 0x7fbc58067da0 2024-04-23T02:55:08.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.631+0000 7fbc57fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc58106a80 0x7fbc581969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:08.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.631+0000 7fbc57fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc58106a80 0x7fbc581969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60562/0 (socket says 172.21.15.71:60562) 2024-04-23T02:55:08.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.631+0000 7fbc57fff700 1 -- 172.21.15.71:0/4260317865 learned_addr learned my addr 172.21.15.71:0/4260317865 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:08.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.631+0000 7fbc5cf02700 1 --2- 172.21.15.71:0/4260317865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc58067da0 0x7fbc581964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:08.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.631+0000 7fbc57fff700 1 -- 172.21.15.71:0/4260317865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc58067da0 msgr2=0x7fbc581964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:08.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.631+0000 7fbc57fff700 1 --2- 172.21.15.71:0/4260317865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc58067da0 0x7fbc581964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:08.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.631+0000 7fbc57fff700 1 -- 172.21.15.71:0/4260317865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c0095f0 con 0x7fbc58106a80 2024-04-23T02:55:08.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.631+0000 7fbc5cf02700 1 --2- 172.21.15.71:0/4260317865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc58067da0 0x7fbc581964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:55:08.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.631+0000 7fbc57fff700 1 --2- 172.21.15.71:0/4260317865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc58106a80 0x7fbc581969f0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400d8d0 tx=0x7fbc4400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:08.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.631+0000 7fbc55ffb700 1 -- 172.21.15.71:0/4260317865 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc44009940 con 0x7fbc58106a80 2024-04-23T02:55:08.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.631+0000 7fbc55ffb700 1 -- 172.21.15.71:0/4260317865 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbc44010460 con 0x7fbc58106a80 2024-04-23T02:55:08.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.631+0000 7fbc5f166700 1 -- 172.21.15.71:0/4260317865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5819b130 con 0x7fbc58106a80 2024-04-23T02:55:08.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.631+0000 7fbc55ffb700 1 -- 172.21.15.71:0/4260317865 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc4400f5f0 con 0x7fbc58106a80 2024-04-23T02:55:08.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.631+0000 7fbc5f166700 1 -- 172.21.15.71:0/4260317865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5819b690 con 0x7fbc58106a80 2024-04-23T02:55:08.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.632+0000 7fbc5f166700 1 -- 172.21.15.71:0/4260317865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5804ea90 con 0x7fbc58106a80 2024-04-23T02:55:08.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.633+0000 7fbc55ffb700 1 -- 172.21.15.71:0/4260317865 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbc440105d0 con 0x7fbc58106a80 2024-04-23T02:55:08.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.633+0000 7fbc55ffb700 1 --2- 172.21.15.71:0/4260317865 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbc4806cda0 0x7fbc4806f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:08.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.633+0000 7fbc55ffb700 1 -- 172.21.15.71:0/4260317865 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbc4408c5e0 con 0x7fbc58106a80 2024-04-23T02:55:08.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.633+0000 7fbc5cf02700 1 --2- 172.21.15.71:0/4260317865 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbc4806cda0 0x7fbc4806f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:08.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.633+0000 7fbc5cf02700 1 --2- 172.21.15.71:0/4260317865 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbc4806cda0 0x7fbc4806f260 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00b5c0 tx=0x7fbc4c005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:08.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.637+0000 7fbc55ffb700 1 -- 172.21.15.71:0/4260317865 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbc44056650 con 0x7fbc58106a80 2024-04-23T02:55:08.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.874+0000 7fbc5f166700 1 -- 172.21.15.71:0/4260317865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fbc5819b9b0 con 0x7fbc58106a80 2024-04-23T02:55:08.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.875+0000 7fbc55ffb700 1 -- 172.21.15.71:0/4260317865 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v46) v1 ==== 107+0+4824 (secure 0 0 0) 0x7fbc44010880 con 0x7fbc58106a80 2024-04-23T02:55:08.877 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:08.877 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34334,"name":"cephfs.smithi071.qusrrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6827/2539819173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2539819173},{"type":"v1","addr":"172.21.15.71:6827","nonce":2539819173}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":44263,"name":"cephfs.smithi192.azdyze","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/418886657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":418886657},{"type":"v1","addr":"172.21.15.192:6827","nonce":418886657}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44271,"name":"cephfs.smithi192.gtmyst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/1096782156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1096782156},{"type":"v1","addr":"172.21.15.192:6825","nonce":1096782156}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":44273,"name":"cephfs.smithi071.vxejye","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6829/3220156840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3220156840},{"type":"v1","addr":"172.21.15.71:6829","nonce":3220156840}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:22:00.280923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:08.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.877+0000 7fbc5f166700 1 -- 172.21.15.71:0/4260317865 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbc4806cda0 msgr2=0x7fbc4806f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:08.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.877+0000 7fbc5f166700 1 --2- 172.21.15.71:0/4260317865 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbc4806cda0 0x7fbc4806f260 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00b5c0 tx=0x7fbc4c005960 comp rx=0 tx=0).stop 2024-04-23T02:55:08.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.877+0000 7fbc5f166700 1 -- 172.21.15.71:0/4260317865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc58106a80 msgr2=0x7fbc581969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:08.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.877+0000 7fbc5f166700 1 --2- 172.21.15.71:0/4260317865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc58106a80 0x7fbc581969f0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400d8d0 tx=0x7fbc4400dc90 comp rx=0 tx=0).stop 2024-04-23T02:55:08.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.877+0000 7fbc5f166700 1 -- 172.21.15.71:0/4260317865 shutdown_connections 2024-04-23T02:55:08.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.877+0000 7fbc5f166700 1 --2- 172.21.15.71:0/4260317865 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fbc4806cda0 0x7fbc4806f260 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:08.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.877+0000 7fbc5f166700 1 --2- 172.21.15.71:0/4260317865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc58067da0 0x7fbc581964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:08.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.877+0000 7fbc5f166700 1 --2- 172.21.15.71:0/4260317865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc58106a80 0x7fbc581969f0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:08.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.877+0000 7fbc5f166700 1 -- 172.21.15.71:0/4260317865 >> 172.21.15.71:0/4260317865 conn(0x7fbc58075400 msgr2=0x7fbc580fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:08.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.877+0000 7fbc5f166700 1 -- 172.21.15.71:0/4260317865 shutdown_connections 2024-04-23T02:55:08.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:08.877+0000 7fbc5f166700 1 -- 172.21.15.71:0/4260317865 wait complete. 2024-04-23T02:55:08.881 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 30 2024-04-23T02:55:08.890 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:08 smithi071 ceph-mon[144544]: pgmap v1257: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:08.891 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:08 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3179835581' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-23T02:55:08.891 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:08 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1843845328' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-23T02:55:08.891 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:08 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2698021059' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-23T02:55:08.892 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-23T02:55:08.892 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-04-23T02:55:08.892 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-04-23T02:55:09.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.011+0000 7fa3ab649700 1 -- 172.21.15.71:0/4279336685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3a4067da0 msgr2=0x7fa3a4068220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:09.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.011+0000 7fa3ab649700 1 --2- 172.21.15.71:0/4279336685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3a4067da0 0x7fa3a4068220 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa394009910 tx=0x7fa394009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:09.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.011+0000 7fa3ab649700 1 -- 172.21.15.71:0/4279336685 shutdown_connections 2024-04-23T02:55:09.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.011+0000 7fa3ab649700 1 --2- 172.21.15.71:0/4279336685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3a4067da0 0x7fa3a4068220 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:09.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.011+0000 7fa3ab649700 1 --2- 172.21.15.71:0/4279336685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a4106a80 0x7fa3a4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:09.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.011+0000 7fa3ab649700 1 -- 172.21.15.71:0/4279336685 >> 172.21.15.71:0/4279336685 conn(0x7fa3a4075400 msgr2=0x7fa3a4075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:09.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.011+0000 7fa3ab649700 1 -- 172.21.15.71:0/4279336685 shutdown_connections 2024-04-23T02:55:09.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.011+0000 7fa3ab649700 1 -- 172.21.15.71:0/4279336685 wait complete. 2024-04-23T02:55:09.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.012+0000 7fa3ab649700 1 Processor -- start 2024-04-23T02:55:09.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.012+0000 7fa3ab649700 1 -- start start 2024-04-23T02:55:09.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.012+0000 7fa3ab649700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3a4067da0 0x7fa3a41a2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:09.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.012+0000 7fa3ab649700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a4106a80 0x7fa3a41a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:09.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.012+0000 7fa3ab649700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a41a34e0 con 0x7fa3a4067da0 2024-04-23T02:55:09.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.012+0000 7fa3ab649700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a419c980 con 0x7fa3a4106a80 2024-04-23T02:55:09.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.012+0000 7fa3a93e5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3a4067da0 0x7fa3a41a2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:09.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.012+0000 7fa3a93e5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3a4067da0 0x7fa3a41a2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60590/0 (socket says 172.21.15.71:60590) 2024-04-23T02:55:09.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.012+0000 7fa3a93e5700 1 -- 172.21.15.71:0/2923183005 learned_addr learned my addr 172.21.15.71:0/2923183005 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:09.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.012+0000 7fa3a8be4700 1 --2- 172.21.15.71:0/2923183005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a4106a80 0x7fa3a41a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:09.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.012+0000 7fa3a93e5700 1 -- 172.21.15.71:0/2923183005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a4106a80 msgr2=0x7fa3a41a2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:09.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.012+0000 7fa3a93e5700 1 --2- 172.21.15.71:0/2923183005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a4106a80 0x7fa3a41a2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:09.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.012+0000 7fa3a93e5700 1 -- 172.21.15.71:0/2923183005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3940095f0 con 0x7fa3a4067da0 2024-04-23T02:55:09.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.012+0000 7fa3a93e5700 1 --2- 172.21.15.71:0/2923183005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3a4067da0 0x7fa3a41a2910 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000d8d0 tx=0x7fa3a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:09.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.013+0000 7fa39a7fc700 1 -- 172.21.15.71:0/2923183005 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3a0009940 con 0x7fa3a4067da0 2024-04-23T02:55:09.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.013+0000 7fa39a7fc700 1 -- 172.21.15.71:0/2923183005 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa3a0010460 con 0x7fa3a4067da0 2024-04-23T02:55:09.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.013+0000 7fa39a7fc700 1 -- 172.21.15.71:0/2923183005 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3a000f5f0 con 0x7fa3a4067da0 2024-04-23T02:55:09.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.013+0000 7fa3ab649700 1 -- 172.21.15.71:0/2923183005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a419cc60 con 0x7fa3a4067da0 2024-04-23T02:55:09.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.013+0000 7fa3ab649700 1 -- 172.21.15.71:0/2923183005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a419d170 con 0x7fa3a4067da0 2024-04-23T02:55:09.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.013+0000 7fa3ab649700 1 -- 172.21.15.71:0/2923183005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3a4109000 con 0x7fa3a4067da0 2024-04-23T02:55:09.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.014+0000 7fa39a7fc700 1 -- 172.21.15.71:0/2923183005 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa3a00105d0 con 0x7fa3a4067da0 2024-04-23T02:55:09.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.014+0000 7fa39a7fc700 1 --2- 172.21.15.71:0/2923183005 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa39006cda0 0x7fa39006f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:09.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.014+0000 7fa39a7fc700 1 -- 172.21.15.71:0/2923183005 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa3a008c5e0 con 0x7fa3a4067da0 2024-04-23T02:55:09.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.014+0000 7fa3a8be4700 1 --2- 172.21.15.71:0/2923183005 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa39006cda0 0x7fa39006f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:09.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.015+0000 7fa3a8be4700 1 --2- 172.21.15.71:0/2923183005 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa39006cda0 0x7fa39006f260 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40759c0 tx=0x7fa39401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:09.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.019+0000 7fa39a7fc700 1 -- 172.21.15.71:0/2923183005 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa3a005a0e0 con 0x7fa3a4067da0 2024-04-23T02:55:09.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:08 smithi192 ceph-mon[108617]: pgmap v1257: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:09.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:08 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3179835581' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-23T02:55:09.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:08 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1843845328' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-23T02:55:09.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:08 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2698021059' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-23T02:55:09.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.259+0000 7fa3ab649700 1 -- 172.21.15.71:0/2923183005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7fa3a419d540 con 0x7fa3a4067da0 2024-04-23T02:55:09.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.260+0000 7fa39a7fc700 1 -- 172.21.15.71:0/2923183005 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v46) v1 ==== 107+0+4045 (secure 0 0 0) 0x7fa3a0010880 con 0x7fa3a4067da0 2024-04-23T02:55:09.262 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:09.262 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34334,"name":"cephfs.smithi071.qusrrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6827/2539819173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2539819173},{"type":"v1","addr":"172.21.15.71:6827","nonce":2539819173}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":44263,"name":"cephfs.smithi192.azdyze","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/418886657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":418886657},{"type":"v1","addr":"172.21.15.192:6827","nonce":418886657}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44271,"name":"cephfs.smithi192.gtmyst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/1096782156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1096782156},{"type":"v1","addr":"172.21.15.192:6825","nonce":1096782156}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:22:00.280923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:09.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.262+0000 7fa3ab649700 1 -- 172.21.15.71:0/2923183005 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa39006cda0 msgr2=0x7fa39006f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:09.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.262+0000 7fa3ab649700 1 --2- 172.21.15.71:0/2923183005 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa39006cda0 0x7fa39006f260 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40759c0 tx=0x7fa39401a040 comp rx=0 tx=0).stop 2024-04-23T02:55:09.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.262+0000 7fa3ab649700 1 -- 172.21.15.71:0/2923183005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3a4067da0 msgr2=0x7fa3a41a2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:09.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.262+0000 7fa3ab649700 1 --2- 172.21.15.71:0/2923183005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3a4067da0 0x7fa3a41a2910 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000d8d0 tx=0x7fa3a000dc90 comp rx=0 tx=0).stop 2024-04-23T02:55:09.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.262+0000 7fa3ab649700 1 -- 172.21.15.71:0/2923183005 shutdown_connections 2024-04-23T02:55:09.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.262+0000 7fa3ab649700 1 --2- 172.21.15.71:0/2923183005 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa39006cda0 0x7fa39006f260 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:09.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.262+0000 7fa3ab649700 1 --2- 172.21.15.71:0/2923183005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3a4067da0 0x7fa3a41a2910 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:09.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.262+0000 7fa3ab649700 1 --2- 172.21.15.71:0/2923183005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a4106a80 0x7fa3a41a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:09.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.262+0000 7fa3ab649700 1 -- 172.21.15.71:0/2923183005 >> 172.21.15.71:0/2923183005 conn(0x7fa3a4075400 msgr2=0x7fa3a40fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:09.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.262+0000 7fa3ab649700 1 -- 172.21.15.71:0/2923183005 shutdown_connections 2024-04-23T02:55:09.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.262+0000 7fa3ab649700 1 -- 172.21.15.71:0/2923183005 wait complete. 2024-04-23T02:55:09.266 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 31 2024-04-23T02:55:09.276 DEBUG:tasks.fs:max_mds not reduced in epoch 31 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-23T02:55:09.277 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-04-23T02:55:09.277 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-04-23T02:55:09.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.398+0000 7fd274131700 1 -- 172.21.15.71:0/1137055139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd26c106a80 msgr2=0x7fd26c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:09.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.398+0000 7fd274131700 1 --2- 172.21.15.71:0/1137055139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd26c106a80 0x7fd26c106e60 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009910 tx=0x7fd25c009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:09.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.398+0000 7fd274131700 1 -- 172.21.15.71:0/1137055139 shutdown_connections 2024-04-23T02:55:09.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.398+0000 7fd274131700 1 --2- 172.21.15.71:0/1137055139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd26c067da0 0x7fd26c068220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:09.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.398+0000 7fd274131700 1 --2- 172.21.15.71:0/1137055139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd26c106a80 0x7fd26c106e60 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:09.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.398+0000 7fd274131700 1 -- 172.21.15.71:0/1137055139 >> 172.21.15.71:0/1137055139 conn(0x7fd26c075400 msgr2=0x7fd26c075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:09.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.398+0000 7fd274131700 1 -- 172.21.15.71:0/1137055139 shutdown_connections 2024-04-23T02:55:09.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.398+0000 7fd274131700 1 -- 172.21.15.71:0/1137055139 wait complete. 2024-04-23T02:55:09.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.398+0000 7fd274131700 1 Processor -- start 2024-04-23T02:55:09.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.398+0000 7fd274131700 1 -- start start 2024-04-23T02:55:09.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.399+0000 7fd274131700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd26c067da0 0x7fd26c1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:09.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.399+0000 7fd274131700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd26c106a80 0x7fd26c1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:09.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.399+0000 7fd274131700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c1970d0 con 0x7fd26c106a80 2024-04-23T02:55:09.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.399+0000 7fd274131700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c19ae50 con 0x7fd26c067da0 2024-04-23T02:55:09.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.399+0000 7fd2716cc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd26c106a80 0x7fd26c1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:09.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.399+0000 7fd2716cc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd26c106a80 0x7fd26c1969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60604/0 (socket says 172.21.15.71:60604) 2024-04-23T02:55:09.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.399+0000 7fd2716cc700 1 -- 172.21.15.71:0/266247941 learned_addr learned my addr 172.21.15.71:0/266247941 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:09.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.399+0000 7fd271ecd700 1 --2- 172.21.15.71:0/266247941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd26c067da0 0x7fd26c1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:09.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.399+0000 7fd2716cc700 1 -- 172.21.15.71:0/266247941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd26c067da0 msgr2=0x7fd26c1964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:09.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.399+0000 7fd2716cc700 1 --2- 172.21.15.71:0/266247941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd26c067da0 0x7fd26c1964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:09.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.399+0000 7fd2716cc700 1 -- 172.21.15.71:0/266247941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c0095f0 con 0x7fd26c106a80 2024-04-23T02:55:09.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.399+0000 7fd271ecd700 1 --2- 172.21.15.71:0/266247941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd26c067da0 0x7fd26c1964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:55:09.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.399+0000 7fd2716cc700 1 --2- 172.21.15.71:0/266247941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd26c106a80 0x7fd26c1969f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd26800c8a0 tx=0x7fd26800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:09.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.399+0000 7fd262ffd700 1 -- 172.21.15.71:0/266247941 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd26800cea0 con 0x7fd26c106a80 2024-04-23T02:55:09.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.399+0000 7fd262ffd700 1 -- 172.21.15.71:0/266247941 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd268004830 con 0x7fd26c106a80 2024-04-23T02:55:09.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.399+0000 7fd274131700 1 -- 172.21.15.71:0/266247941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26c19b130 con 0x7fd26c106a80 2024-04-23T02:55:09.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.399+0000 7fd262ffd700 1 -- 172.21.15.71:0/266247941 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd268005500 con 0x7fd26c106a80 2024-04-23T02:55:09.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.399+0000 7fd274131700 1 -- 172.21.15.71:0/266247941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26c19b690 con 0x7fd26c106a80 2024-04-23T02:55:09.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.400+0000 7fd274131700 1 -- 172.21.15.71:0/266247941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26c04ea90 con 0x7fd26c106a80 2024-04-23T02:55:09.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.401+0000 7fd262ffd700 1 -- 172.21.15.71:0/266247941 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd2680049a0 con 0x7fd26c106a80 2024-04-23T02:55:09.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.401+0000 7fd262ffd700 1 --2- 172.21.15.71:0/266247941 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd25806cda0 0x7fd25806f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:09.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.401+0000 7fd262ffd700 1 -- 172.21.15.71:0/266247941 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd26808c1b0 con 0x7fd26c106a80 2024-04-23T02:55:09.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.401+0000 7fd271ecd700 1 --2- 172.21.15.71:0/266247941 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd25806cda0 0x7fd25806f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:09.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.402+0000 7fd271ecd700 1 --2- 172.21.15.71:0/266247941 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd25806cda0 0x7fd25806f260 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00b5c0 tx=0x7fd25c005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:09.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.406+0000 7fd262ffd700 1 -- 172.21.15.71:0/266247941 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd2680562a0 con 0x7fd26c106a80 2024-04-23T02:55:09.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.645+0000 7fd274131700 1 -- 172.21.15.71:0/266247941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7fd26c19ba60 con 0x7fd26c106a80 2024-04-23T02:55:09.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.646+0000 7fd262ffd700 1 -- 172.21.15.71:0/266247941 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v46) v1 ==== 107+0+4821 (secure 0 0 0) 0x7fd2680598c0 con 0x7fd26c106a80 2024-04-23T02:55:09.648 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:09.649 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34334,"name":"cephfs.smithi071.qusrrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6827/2539819173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2539819173},{"type":"v1","addr":"172.21.15.71:6827","nonce":2539819173}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":34362,"name":"cephfs.smithi071.vxejye","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6829/143423621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":143423621},{"type":"v1","addr":"172.21.15.71:6829","nonce":143423621}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":44263,"name":"cephfs.smithi192.azdyze","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/418886657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":418886657},{"type":"v1","addr":"172.21.15.192:6827","nonce":418886657}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44271,"name":"cephfs.smithi192.gtmyst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/1096782156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1096782156},{"type":"v1","addr":"172.21.15.192:6825","nonce":1096782156}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:22:00.280923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:09.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.648+0000 7fd274131700 1 -- 172.21.15.71:0/266247941 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd25806cda0 msgr2=0x7fd25806f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:09.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.648+0000 7fd274131700 1 --2- 172.21.15.71:0/266247941 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd25806cda0 0x7fd25806f260 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00b5c0 tx=0x7fd25c005960 comp rx=0 tx=0).stop 2024-04-23T02:55:09.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.648+0000 7fd274131700 1 -- 172.21.15.71:0/266247941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd26c106a80 msgr2=0x7fd26c1969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:09.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.648+0000 7fd274131700 1 --2- 172.21.15.71:0/266247941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd26c106a80 0x7fd26c1969f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd26800c8a0 tx=0x7fd26800cc60 comp rx=0 tx=0).stop 2024-04-23T02:55:09.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.648+0000 7fd274131700 1 -- 172.21.15.71:0/266247941 shutdown_connections 2024-04-23T02:55:09.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.648+0000 7fd274131700 1 --2- 172.21.15.71:0/266247941 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd25806cda0 0x7fd25806f260 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:09.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.648+0000 7fd274131700 1 --2- 172.21.15.71:0/266247941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd26c067da0 0x7fd26c1964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:09.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.648+0000 7fd274131700 1 --2- 172.21.15.71:0/266247941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd26c106a80 0x7fd26c1969f0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:09.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.648+0000 7fd274131700 1 -- 172.21.15.71:0/266247941 >> 172.21.15.71:0/266247941 conn(0x7fd26c075400 msgr2=0x7fd26c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:09.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.648+0000 7fd274131700 1 -- 172.21.15.71:0/266247941 shutdown_connections 2024-04-23T02:55:09.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.648+0000 7fd274131700 1 -- 172.21.15.71:0/266247941 wait complete. 2024-04-23T02:55:09.652 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 32 2024-04-23T02:55:09.663 DEBUG:tasks.fs:max_mds not reduced in epoch 32 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-23T02:55:09.663 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-04-23T02:55:09.663 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-04-23T02:55:09.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.783+0000 7f3111caa700 1 -- 172.21.15.71:0/251618343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f310c0ff4a0 msgr2=0x7f310c0ff880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:09.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.783+0000 7f3111caa700 1 --2- 172.21.15.71:0/251618343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f310c0ff4a0 0x7f310c0ff880 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009910 tx=0x7f30fc009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:09.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.783+0000 7f3111caa700 1 -- 172.21.15.71:0/251618343 shutdown_connections 2024-04-23T02:55:09.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.783+0000 7f3111caa700 1 --2- 172.21.15.71:0/251618343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f310c0ffe50 0x7f310c105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:09.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.783+0000 7f3111caa700 1 --2- 172.21.15.71:0/251618343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f310c0ff4a0 0x7f310c0ff880 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:09.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.783+0000 7f3111caa700 1 -- 172.21.15.71:0/251618343 >> 172.21.15.71:0/251618343 conn(0x7f310c074b60 msgr2=0x7f310c074f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:09.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.783+0000 7f3111caa700 1 -- 172.21.15.71:0/251618343 shutdown_connections 2024-04-23T02:55:09.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.783+0000 7f3111caa700 1 -- 172.21.15.71:0/251618343 wait complete. 2024-04-23T02:55:09.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.783+0000 7f3111caa700 1 Processor -- start 2024-04-23T02:55:09.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.784+0000 7f3111caa700 1 -- start start 2024-04-23T02:55:09.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.784+0000 7f3111caa700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f310c0ff4a0 0x7f310c196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:09.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.784+0000 7f3111caa700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f310c196a60 0x7f310c19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:09.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.784+0000 7f3111caa700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c197080 con 0x7f310c0ff4a0 2024-04-23T02:55:09.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.784+0000 7f3111caa700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c1971f0 con 0x7f310c196a60 2024-04-23T02:55:09.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.784+0000 7f310b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f310c0ff4a0 0x7f310c196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:09.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.784+0000 7f310b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f310c0ff4a0 0x7f310c196520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60618/0 (socket says 172.21.15.71:60618) 2024-04-23T02:55:09.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.784+0000 7f310b7fe700 1 -- 172.21.15.71:0/1334628325 learned_addr learned my addr 172.21.15.71:0/1334628325 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:09.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.784+0000 7f310affd700 1 --2- 172.21.15.71:0/1334628325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f310c196a60 0x7f310c19aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:09.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.784+0000 7f310b7fe700 1 -- 172.21.15.71:0/1334628325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f310c196a60 msgr2=0x7f310c19aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:09.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.784+0000 7f310b7fe700 1 --2- 172.21.15.71:0/1334628325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f310c196a60 0x7f310c19aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:09.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.784+0000 7f310b7fe700 1 -- 172.21.15.71:0/1334628325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc0095f0 con 0x7f310c0ff4a0 2024-04-23T02:55:09.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.784+0000 7f310b7fe700 1 --2- 172.21.15.71:0/1334628325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f310c0ff4a0 0x7f310c196520 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0092e0 tx=0x7f30fc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:09.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.784+0000 7f3108ff9700 1 -- 172.21.15.71:0/1334628325 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30fc01d070 con 0x7f310c0ff4a0 2024-04-23T02:55:09.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.785+0000 7f3111caa700 1 -- 172.21.15.71:0/1334628325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310c19b4c0 con 0x7f310c0ff4a0 2024-04-23T02:55:09.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.785+0000 7f3108ff9700 1 -- 172.21.15.71:0/1334628325 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f30fc005e00 con 0x7f310c0ff4a0 2024-04-23T02:55:09.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.785+0000 7f3108ff9700 1 -- 172.21.15.71:0/1334628325 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30fc00fe60 con 0x7f310c0ff4a0 2024-04-23T02:55:09.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.785+0000 7f3111caa700 1 -- 172.21.15.71:0/1334628325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310c19b9c0 con 0x7f310c0ff4a0 2024-04-23T02:55:09.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.785+0000 7f3111caa700 1 -- 172.21.15.71:0/1334628325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310c04ea90 con 0x7f310c0ff4a0 2024-04-23T02:55:09.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.786+0000 7f3108ff9700 1 -- 172.21.15.71:0/1334628325 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f30fc00f6d0 con 0x7f310c0ff4a0 2024-04-23T02:55:09.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.786+0000 7f3108ff9700 1 --2- 172.21.15.71:0/1334628325 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f30f406cda0 0x7f30f406f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:09.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.786+0000 7f3108ff9700 1 -- 172.21.15.71:0/1334628325 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f30fc08d8b0 con 0x7f310c0ff4a0 2024-04-23T02:55:09.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.786+0000 7f310affd700 1 --2- 172.21.15.71:0/1334628325 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f30f406cda0 0x7f30f406f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:09.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.787+0000 7f310affd700 1 --2- 172.21.15.71:0/1334628325 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f30f406cda0 0x7f30f406f260 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3100005950 tx=0x7f31000058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:09.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:09.791+0000 7f3108ff9700 1 -- 172.21.15.71:0/1334628325 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f30fc057920 con 0x7f310c0ff4a0 2024-04-23T02:55:09.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:09 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/4260317865' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-23T02:55:09.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:09 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2923183005' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-23T02:55:10.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.032+0000 7f3111caa700 1 -- 172.21.15.71:0/1334628325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f310c0ffe50 con 0x7f310c0ff4a0 2024-04-23T02:55:10.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.033+0000 7f3108ff9700 1 -- 172.21.15.71:0/1334628325 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v46) v1 ==== 107+0+4039 (secure 0 0 0) 0x7f30fc05af40 con 0x7f310c0ff4a0 2024-04-23T02:55:10.036 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:10.036 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34334,"name":"cephfs.smithi071.qusrrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6827/2539819173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2539819173},{"type":"v1","addr":"172.21.15.71:6827","nonce":2539819173}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":34362,"name":"cephfs.smithi071.vxejye","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6829/143423621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":143423621},{"type":"v1","addr":"172.21.15.71:6829","nonce":143423621}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":44263,"name":"cephfs.smithi192.azdyze","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/418886657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":418886657},{"type":"v1","addr":"172.21.15.192:6827","nonce":418886657}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:22:00.280923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:10.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.035+0000 7f3111caa700 1 -- 172.21.15.71:0/1334628325 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f30f406cda0 msgr2=0x7f30f406f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:10.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.035+0000 7f3111caa700 1 --2- 172.21.15.71:0/1334628325 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f30f406cda0 0x7f30f406f260 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3100005950 tx=0x7f31000058e0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.035+0000 7f3111caa700 1 -- 172.21.15.71:0/1334628325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f310c0ff4a0 msgr2=0x7f310c196520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:10.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.035+0000 7f3111caa700 1 --2- 172.21.15.71:0/1334628325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f310c0ff4a0 0x7f310c196520 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0092e0 tx=0x7f30fc00be20 comp rx=0 tx=0).stop 2024-04-23T02:55:10.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.035+0000 7f3111caa700 1 -- 172.21.15.71:0/1334628325 shutdown_connections 2024-04-23T02:55:10.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.035+0000 7f3111caa700 1 --2- 172.21.15.71:0/1334628325 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f30f406cda0 0x7f30f406f260 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.035+0000 7f3111caa700 1 --2- 172.21.15.71:0/1334628325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f310c0ff4a0 0x7f310c196520 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.035+0000 7f3111caa700 1 --2- 172.21.15.71:0/1334628325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f310c196a60 0x7f310c19aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.035+0000 7f3111caa700 1 -- 172.21.15.71:0/1334628325 >> 172.21.15.71:0/1334628325 conn(0x7f310c074b60 msgr2=0x7f310c0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:10.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.035+0000 7f3111caa700 1 -- 172.21.15.71:0/1334628325 shutdown_connections 2024-04-23T02:55:10.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.035+0000 7f3111caa700 1 -- 172.21.15.71:0/1334628325 wait complete. 2024-04-23T02:55:10.039 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 33 2024-04-23T02:55:10.051 DEBUG:tasks.fs:max_mds not reduced in epoch 33 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-23T02:55:10.051 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-04-23T02:55:10.051 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-04-23T02:55:10.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:09 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/4260317865' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-23T02:55:10.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:09 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2923183005' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-23T02:55:10.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.171+0000 7f3615a51700 1 -- 172.21.15.71:0/2304251191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3610067da0 msgr2=0x7f3610068220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:10.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.171+0000 7f3615a51700 1 --2- 172.21.15.71:0/2304251191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3610067da0 0x7f3610068220 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f3600009910 tx=0x7f3600009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:10.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.171+0000 7f3615a51700 1 -- 172.21.15.71:0/2304251191 shutdown_connections 2024-04-23T02:55:10.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.171+0000 7f3615a51700 1 --2- 172.21.15.71:0/2304251191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3610067da0 0x7f3610068220 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.171+0000 7f3615a51700 1 --2- 172.21.15.71:0/2304251191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3610106a80 0x7f3610106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.171+0000 7f3615a51700 1 -- 172.21.15.71:0/2304251191 >> 172.21.15.71:0/2304251191 conn(0x7f3610075400 msgr2=0x7f3610075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:10.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.171+0000 7f3615a51700 1 -- 172.21.15.71:0/2304251191 shutdown_connections 2024-04-23T02:55:10.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.171+0000 7f3615a51700 1 -- 172.21.15.71:0/2304251191 wait complete. 2024-04-23T02:55:10.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.172+0000 7f3615a51700 1 Processor -- start 2024-04-23T02:55:10.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.172+0000 7f3615a51700 1 -- start start 2024-04-23T02:55:10.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.172+0000 7f3615a51700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3610067da0 0x7f3610196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:10.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.172+0000 7f3615a51700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3610106a80 0x7f3610196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:10.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.172+0000 7f3615a51700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3610197160 con 0x7f3610106a80 2024-04-23T02:55:10.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.172+0000 7f3615a51700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361019aee0 con 0x7f3610067da0 2024-04-23T02:55:10.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.172+0000 7f360e7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3610106a80 0x7f3610196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:10.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.172+0000 7f360e7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3610106a80 0x7f3610196a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60628/0 (socket says 172.21.15.71:60628) 2024-04-23T02:55:10.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.172+0000 7f360e7fc700 1 -- 172.21.15.71:0/2880338491 learned_addr learned my addr 172.21.15.71:0/2880338491 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:10.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.172+0000 7f360effd700 1 --2- 172.21.15.71:0/2880338491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3610067da0 0x7f3610196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:10.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.172+0000 7f360e7fc700 1 -- 172.21.15.71:0/2880338491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3610067da0 msgr2=0x7f3610196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:10.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.172+0000 7f360e7fc700 1 --2- 172.21.15.71:0/2880338491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3610067da0 0x7f3610196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.172+0000 7f360e7fc700 1 -- 172.21.15.71:0/2880338491 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36000095f0 con 0x7f3610106a80 2024-04-23T02:55:10.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.172+0000 7f360effd700 1 --2- 172.21.15.71:0/2880338491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3610067da0 0x7f3610196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:55:10.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.172+0000 7f360e7fc700 1 --2- 172.21.15.71:0/2880338491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3610106a80 0x7f3610196a80 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f3600006010 tx=0x7f360000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:10.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.172+0000 7f3614a4f700 1 -- 172.21.15.71:0/2880338491 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f360001d070 con 0x7f3610106a80 2024-04-23T02:55:10.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.173+0000 7f3614a4f700 1 -- 172.21.15.71:0/2880338491 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f360000f460 con 0x7f3610106a80 2024-04-23T02:55:10.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.173+0000 7f3615a51700 1 -- 172.21.15.71:0/2880338491 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361019b160 con 0x7f3610106a80 2024-04-23T02:55:10.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.173+0000 7f3614a4f700 1 -- 172.21.15.71:0/2880338491 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3600021640 con 0x7f3610106a80 2024-04-23T02:55:10.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.173+0000 7f3615a51700 1 -- 172.21.15.71:0/2880338491 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361019b660 con 0x7f3610106a80 2024-04-23T02:55:10.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.173+0000 7f3615a51700 1 -- 172.21.15.71:0/2880338491 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3610109000 con 0x7f3610106a80 2024-04-23T02:55:10.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.174+0000 7f3614a4f700 1 -- 172.21.15.71:0/2880338491 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f360000fa20 con 0x7f3610106a80 2024-04-23T02:55:10.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.174+0000 7f3614a4f700 1 --2- 172.21.15.71:0/2880338491 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f35f806cda0 0x7f35f806f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:10.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.174+0000 7f3614a4f700 1 -- 172.21.15.71:0/2880338491 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f360008d740 con 0x7f3610106a80 2024-04-23T02:55:10.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.174+0000 7f360effd700 1 --2- 172.21.15.71:0/2880338491 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f35f806cda0 0x7f35f806f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:10.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.175+0000 7f360effd700 1 --2- 172.21.15.71:0/2880338491 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f35f806cda0 0x7f35f806f260 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f360400bc10 tx=0x7f360400b5f0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:10.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.179+0000 7f3614a4f700 1 -- 172.21.15.71:0/2880338491 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3600057830 con 0x7f3610106a80 2024-04-23T02:55:10.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.418+0000 7f3615a51700 1 -- 172.21.15.71:0/2880338491 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f361004ea90 con 0x7f3610106a80 2024-04-23T02:55:10.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.419+0000 7f3614a4f700 1 -- 172.21.15.71:0/2880338491 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v46) v1 ==== 107+0+4821 (secure 0 0 0) 0x7f360005ae50 con 0x7f3610106a80 2024-04-23T02:55:10.421 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:10.421 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34334,"name":"cephfs.smithi071.qusrrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6827/2539819173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2539819173},{"type":"v1","addr":"172.21.15.71:6827","nonce":2539819173}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":34362,"name":"cephfs.smithi071.vxejye","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6829/143423621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":143423621},{"type":"v1","addr":"172.21.15.71:6829","nonce":143423621}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":44263,"name":"cephfs.smithi192.azdyze","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/418886657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":418886657},{"type":"v1","addr":"172.21.15.192:6827","nonce":418886657}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44319,"name":"cephfs.smithi192.gtmyst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3688927981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3688927981},{"type":"v1","addr":"172.21.15.192:6825","nonce":3688927981}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:22:00.280923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:10.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.421+0000 7f3615a51700 1 -- 172.21.15.71:0/2880338491 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f35f806cda0 msgr2=0x7f35f806f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:10.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.421+0000 7f3615a51700 1 --2- 172.21.15.71:0/2880338491 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f35f806cda0 0x7f35f806f260 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f360400bc10 tx=0x7f360400b5f0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.421+0000 7f3615a51700 1 -- 172.21.15.71:0/2880338491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3610106a80 msgr2=0x7f3610196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:10.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.421+0000 7f3615a51700 1 --2- 172.21.15.71:0/2880338491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3610106a80 0x7f3610196a80 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f3600006010 tx=0x7f360000be20 comp rx=0 tx=0).stop 2024-04-23T02:55:10.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.421+0000 7f3615a51700 1 -- 172.21.15.71:0/2880338491 shutdown_connections 2024-04-23T02:55:10.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.421+0000 7f3615a51700 1 --2- 172.21.15.71:0/2880338491 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f35f806cda0 0x7f35f806f260 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.421+0000 7f3615a51700 1 --2- 172.21.15.71:0/2880338491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3610067da0 0x7f3610196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.421+0000 7f3615a51700 1 --2- 172.21.15.71:0/2880338491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3610106a80 0x7f3610196a80 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.421+0000 7f3615a51700 1 -- 172.21.15.71:0/2880338491 >> 172.21.15.71:0/2880338491 conn(0x7f3610075400 msgr2=0x7f36100fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:10.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.421+0000 7f3615a51700 1 -- 172.21.15.71:0/2880338491 shutdown_connections 2024-04-23T02:55:10.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.421+0000 7f3615a51700 1 -- 172.21.15.71:0/2880338491 wait complete. 2024-04-23T02:55:10.425 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2024-04-23T02:55:10.435 DEBUG:tasks.fs:max_mds not reduced in epoch 34 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-23T02:55:10.436 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-04-23T02:55:10.436 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-04-23T02:55:10.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.556+0000 7f2268998700 1 -- 172.21.15.71:0/2343869315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2264105990 msgr2=0x7f2264105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:10.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.556+0000 7f2268998700 1 --2- 172.21.15.71:0/2343869315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2264105990 0x7f2264105d70 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2258009910 tx=0x7f2258009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:10.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.556+0000 7f2268998700 1 -- 172.21.15.71:0/2343869315 shutdown_connections 2024-04-23T02:55:10.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.556+0000 7f2268998700 1 --2- 172.21.15.71:0/2343869315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22640ffac0 0x7f22640fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.556+0000 7f2268998700 1 --2- 172.21.15.71:0/2343869315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2264105990 0x7f2264105d70 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.556+0000 7f2268998700 1 -- 172.21.15.71:0/2343869315 >> 172.21.15.71:0/2343869315 conn(0x7f2264075400 msgr2=0x7f2264075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:10.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.556+0000 7f2268998700 1 -- 172.21.15.71:0/2343869315 shutdown_connections 2024-04-23T02:55:10.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.556+0000 7f2268998700 1 -- 172.21.15.71:0/2343869315 wait complete. 2024-04-23T02:55:10.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.557+0000 7f2268998700 1 Processor -- start 2024-04-23T02:55:10.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.557+0000 7f2268998700 1 -- start start 2024-04-23T02:55:10.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.557+0000 7f2268998700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22640ffac0 0x7f22641964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:10.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.557+0000 7f2268998700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2264105990 0x7f2264196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:10.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.557+0000 7f2268998700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2264197110 con 0x7f22640ffac0 2024-04-23T02:55:10.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.557+0000 7f2268998700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226419ae90 con 0x7f2264105990 2024-04-23T02:55:10.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.557+0000 7f226259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22640ffac0 0x7f22641964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:10.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.557+0000 7f226259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22640ffac0 0x7f22641964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60634/0 (socket says 172.21.15.71:60634) 2024-04-23T02:55:10.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.557+0000 7f226259c700 1 -- 172.21.15.71:0/2597170451 learned_addr learned my addr 172.21.15.71:0/2597170451 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:10.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.557+0000 7f2261d9b700 1 --2- 172.21.15.71:0/2597170451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2264105990 0x7f2264196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:10.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.557+0000 7f226259c700 1 -- 172.21.15.71:0/2597170451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2264105990 msgr2=0x7f2264196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:10.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.557+0000 7f226259c700 1 --2- 172.21.15.71:0/2597170451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2264105990 0x7f2264196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.557+0000 7f226259c700 1 -- 172.21.15.71:0/2597170451 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22580095f0 con 0x7f22640ffac0 2024-04-23T02:55:10.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.557+0000 7f226259c700 1 --2- 172.21.15.71:0/2597170451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22640ffac0 0x7f22641964f0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f2258000c00 tx=0x7f225800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:10.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.558+0000 7f22537fe700 1 -- 172.21.15.71:0/2597170451 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f225801d070 con 0x7f22640ffac0 2024-04-23T02:55:10.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.558+0000 7f22537fe700 1 -- 172.21.15.71:0/2597170451 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f22580044c0 con 0x7f22640ffac0 2024-04-23T02:55:10.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.558+0000 7f2268998700 1 -- 172.21.15.71:0/2597170451 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f226419b110 con 0x7f22640ffac0 2024-04-23T02:55:10.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.558+0000 7f22537fe700 1 -- 172.21.15.71:0/2597170451 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22580039f0 con 0x7f22640ffac0 2024-04-23T02:55:10.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.558+0000 7f2268998700 1 -- 172.21.15.71:0/2597170451 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226419b5b0 con 0x7f22640ffac0 2024-04-23T02:55:10.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.559+0000 7f2268998700 1 -- 172.21.15.71:0/2597170451 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226404ea90 con 0x7f22640ffac0 2024-04-23T02:55:10.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.559+0000 7f22537fe700 1 -- 172.21.15.71:0/2597170451 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f22580048e0 con 0x7f22640ffac0 2024-04-23T02:55:10.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.559+0000 7f22537fe700 1 --2- 172.21.15.71:0/2597170451 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f224c06cda0 0x7f224c06f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:10.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.559+0000 7f22537fe700 1 -- 172.21.15.71:0/2597170451 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f225808db90 con 0x7f22640ffac0 2024-04-23T02:55:10.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.559+0000 7f2261d9b700 1 --2- 172.21.15.71:0/2597170451 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f224c06cda0 0x7f224c06f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:10.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.560+0000 7f2261d9b700 1 --2- 172.21.15.71:0/2597170451 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f224c06cda0 0x7f224c06f260 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2264197b00 tx=0x7f2254009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:10.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.564+0000 7f22537fe700 1 -- 172.21.15.71:0/2597170451 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2258057c80 con 0x7f22640ffac0 2024-04-23T02:55:10.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:10 smithi071 ceph-mon[144544]: pgmap v1258: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:55:10.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:10 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/266247941' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-23T02:55:10.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:10 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1334628325' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-23T02:55:10.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:10 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2880338491' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-23T02:55:10.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.803+0000 7f2268998700 1 -- 172.21.15.71:0/2597170451 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f226419b920 con 0x7f22640ffac0 2024-04-23T02:55:10.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.804+0000 7f22537fe700 1 -- 172.21.15.71:0/2597170451 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v46) v1 ==== 107+0+4042 (secure 0 0 0) 0x7f2258026020 con 0x7f22640ffac0 2024-04-23T02:55:10.807 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:10.807 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34334,"name":"cephfs.smithi071.qusrrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6827/2539819173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2539819173},{"type":"v1","addr":"172.21.15.71:6827","nonce":2539819173}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":34362,"name":"cephfs.smithi071.vxejye","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6829/143423621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":143423621},{"type":"v1","addr":"172.21.15.71:6829","nonce":143423621}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":44319,"name":"cephfs.smithi192.gtmyst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3688927981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3688927981},{"type":"v1","addr":"172.21.15.192:6825","nonce":3688927981}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:22:00.280923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:10.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.806+0000 7f2268998700 1 -- 172.21.15.71:0/2597170451 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f224c06cda0 msgr2=0x7f224c06f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:10.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.806+0000 7f2268998700 1 --2- 172.21.15.71:0/2597170451 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f224c06cda0 0x7f224c06f260 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2264197b00 tx=0x7f2254009260 comp rx=0 tx=0).stop 2024-04-23T02:55:10.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.806+0000 7f2268998700 1 -- 172.21.15.71:0/2597170451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22640ffac0 msgr2=0x7f22641964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:10.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.806+0000 7f2268998700 1 --2- 172.21.15.71:0/2597170451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22640ffac0 0x7f22641964f0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f2258000c00 tx=0x7f225800be90 comp rx=0 tx=0).stop 2024-04-23T02:55:10.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.806+0000 7f2268998700 1 -- 172.21.15.71:0/2597170451 shutdown_connections 2024-04-23T02:55:10.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.806+0000 7f2268998700 1 --2- 172.21.15.71:0/2597170451 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f224c06cda0 0x7f224c06f260 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.806+0000 7f2268998700 1 --2- 172.21.15.71:0/2597170451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22640ffac0 0x7f22641964f0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.806+0000 7f2268998700 1 --2- 172.21.15.71:0/2597170451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2264105990 0x7f2264196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.806+0000 7f2268998700 1 -- 172.21.15.71:0/2597170451 >> 172.21.15.71:0/2597170451 conn(0x7f2264075400 msgr2=0x7f2264067da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:10.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.806+0000 7f2268998700 1 -- 172.21.15.71:0/2597170451 shutdown_connections 2024-04-23T02:55:10.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.806+0000 7f2268998700 1 -- 172.21.15.71:0/2597170451 wait complete. 2024-04-23T02:55:10.811 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 35 2024-04-23T02:55:10.821 DEBUG:tasks.fs:max_mds not reduced in epoch 35 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-23T02:55:10.821 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 35 2024-04-23T02:55:10.821 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-04-23T02:55:10.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.940+0000 7f853c5b9700 1 -- 172.21.15.71:0/1599667697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85340ffe50 msgr2=0x7f8534105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:10.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.940+0000 7f853c5b9700 1 --2- 172.21.15.71:0/1599667697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85340ffe50 0x7f8534105770 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f8530009910 tx=0x7f8530009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:10.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.940+0000 7f853c5b9700 1 -- 172.21.15.71:0/1599667697 shutdown_connections 2024-04-23T02:55:10.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.940+0000 7f853c5b9700 1 --2- 172.21.15.71:0/1599667697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85340ffe50 0x7f8534105770 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.940+0000 7f853c5b9700 1 --2- 172.21.15.71:0/1599667697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85340ff4a0 0x7f85340ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.940+0000 7f853c5b9700 1 -- 172.21.15.71:0/1599667697 >> 172.21.15.71:0/1599667697 conn(0x7f8534074b60 msgr2=0x7f8534074f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:10.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.940+0000 7f853c5b9700 1 -- 172.21.15.71:0/1599667697 shutdown_connections 2024-04-23T02:55:10.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.940+0000 7f853c5b9700 1 -- 172.21.15.71:0/1599667697 wait complete. 2024-04-23T02:55:10.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.941+0000 7f853c5b9700 1 Processor -- start 2024-04-23T02:55:10.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.941+0000 7f853c5b9700 1 -- start start 2024-04-23T02:55:10.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.941+0000 7f853c5b9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85340ff4a0 0x7f8534196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:10.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.941+0000 7f853c5b9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8534196a80 0x7f853419aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:10.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.941+0000 7f853c5b9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85341970a0 con 0x7f85340ff4a0 2024-04-23T02:55:10.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.941+0000 7f853c5b9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8534197210 con 0x7f8534196a80 2024-04-23T02:55:10.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.942+0000 7f853a355700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85340ff4a0 0x7f8534196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:10.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.942+0000 7f853a355700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85340ff4a0 0x7f8534196540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60656/0 (socket says 172.21.15.71:60656) 2024-04-23T02:55:10.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.942+0000 7f853a355700 1 -- 172.21.15.71:0/2711395589 learned_addr learned my addr 172.21.15.71:0/2711395589 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:10.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.942+0000 7f8539b54700 1 --2- 172.21.15.71:0/2711395589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8534196a80 0x7f853419aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:10.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.942+0000 7f853a355700 1 -- 172.21.15.71:0/2711395589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8534196a80 msgr2=0x7f853419aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:10.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.942+0000 7f853a355700 1 --2- 172.21.15.71:0/2711395589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8534196a80 0x7f853419aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:10.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.942+0000 7f853a355700 1 -- 172.21.15.71:0/2711395589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85300095f0 con 0x7f85340ff4a0 2024-04-23T02:55:10.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.942+0000 7f853a355700 1 --2- 172.21.15.71:0/2711395589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85340ff4a0 0x7f8534196540 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f852400d900 tx=0x7f852400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:10.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.942+0000 7f852b7fe700 1 -- 172.21.15.71:0/2711395589 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85240041f0 con 0x7f85340ff4a0 2024-04-23T02:55:10.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.942+0000 7f852b7fe700 1 -- 172.21.15.71:0/2711395589 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8524004cd0 con 0x7f85340ff4a0 2024-04-23T02:55:10.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.942+0000 7f853c5b9700 1 -- 172.21.15.71:0/2711395589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853419b540 con 0x7f85340ff4a0 2024-04-23T02:55:10.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.942+0000 7f852b7fe700 1 -- 172.21.15.71:0/2711395589 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8524005660 con 0x7f85340ff4a0 2024-04-23T02:55:10.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.942+0000 7f853c5b9700 1 -- 172.21.15.71:0/2711395589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853419baa0 con 0x7f85340ff4a0 2024-04-23T02:55:10.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.943+0000 7f853c5b9700 1 -- 172.21.15.71:0/2711395589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8534109070 con 0x7f85340ff4a0 2024-04-23T02:55:10.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.944+0000 7f852b7fe700 1 -- 172.21.15.71:0/2711395589 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8524004e40 con 0x7f85340ff4a0 2024-04-23T02:55:10.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.944+0000 7f852b7fe700 1 --2- 172.21.15.71:0/2711395589 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f852006cda0 0x7f852006f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:10.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.944+0000 7f852b7fe700 1 -- 172.21.15.71:0/2711395589 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f852408c3c0 con 0x7f85340ff4a0 2024-04-23T02:55:10.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.944+0000 7f8539b54700 1 --2- 172.21.15.71:0/2711395589 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f852006cda0 0x7f852006f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:10.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.944+0000 7f8539b54700 1 --2- 172.21.15.71:0/2711395589 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f852006cda0 0x7f852006f260 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8530006010 tx=0x7f853001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:10.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:10.948+0000 7f852b7fe700 1 -- 172.21.15.71:0/2711395589 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8524056430 con 0x7f85340ff4a0 2024-04-23T02:55:11.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:10 smithi192 ceph-mon[108617]: pgmap v1258: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:55:11.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:10 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/266247941' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-23T02:55:11.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:10 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1334628325' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-23T02:55:11.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:10 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2880338491' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-23T02:55:11.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.185+0000 7f853c5b9700 1 -- 172.21.15.71:0/2711395589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f853404ea90 con 0x7f85340ff4a0 2024-04-23T02:55:11.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.186+0000 7f852b7fe700 1 -- 172.21.15.71:0/2711395589 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v46) v1 ==== 107+0+4824 (secure 0 0 0) 0x7f8524059a50 con 0x7f85340ff4a0 2024-04-23T02:55:11.188 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:11.189 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34334,"name":"cephfs.smithi071.qusrrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6827/2539819173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2539819173},{"type":"v1","addr":"172.21.15.71:6827","nonce":2539819173}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":34362,"name":"cephfs.smithi071.vxejye","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6829/143423621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":143423621},{"type":"v1","addr":"172.21.15.71:6829","nonce":143423621}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":44319,"name":"cephfs.smithi192.gtmyst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3688927981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3688927981},{"type":"v1","addr":"172.21.15.192:6825","nonce":3688927981}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":44793,"name":"cephfs.smithi192.azdyze","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/3768627645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3768627645},{"type":"v1","addr":"172.21.15.192:6827","nonce":3768627645}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:22:00.280923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:11.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.188+0000 7f853c5b9700 1 -- 172.21.15.71:0/2711395589 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f852006cda0 msgr2=0x7f852006f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:11.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.188+0000 7f853c5b9700 1 --2- 172.21.15.71:0/2711395589 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f852006cda0 0x7f852006f260 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8530006010 tx=0x7f853001a040 comp rx=0 tx=0).stop 2024-04-23T02:55:11.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.188+0000 7f853c5b9700 1 -- 172.21.15.71:0/2711395589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85340ff4a0 msgr2=0x7f8534196540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:11.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.188+0000 7f853c5b9700 1 --2- 172.21.15.71:0/2711395589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85340ff4a0 0x7f8534196540 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f852400d900 tx=0x7f852400dc10 comp rx=0 tx=0).stop 2024-04-23T02:55:11.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.188+0000 7f853c5b9700 1 -- 172.21.15.71:0/2711395589 shutdown_connections 2024-04-23T02:55:11.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.188+0000 7f853c5b9700 1 --2- 172.21.15.71:0/2711395589 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f852006cda0 0x7f852006f260 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:11.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.188+0000 7f853c5b9700 1 --2- 172.21.15.71:0/2711395589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85340ff4a0 0x7f8534196540 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:11.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.188+0000 7f853c5b9700 1 --2- 172.21.15.71:0/2711395589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8534196a80 0x7f853419aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:11.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.188+0000 7f853c5b9700 1 -- 172.21.15.71:0/2711395589 >> 172.21.15.71:0/2711395589 conn(0x7f8534074b60 msgr2=0x7f85340fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:11.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.188+0000 7f853c5b9700 1 -- 172.21.15.71:0/2711395589 shutdown_connections 2024-04-23T02:55:11.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.188+0000 7f853c5b9700 1 -- 172.21.15.71:0/2711395589 wait complete. 2024-04-23T02:55:11.193 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-04-23T02:55:11.203 DEBUG:tasks.fs:max_mds not reduced in epoch 36 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-23T02:55:11.203 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 36 2024-04-23T02:55:11.203 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-04-23T02:55:11.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.323+0000 7f1161f89700 1 -- 172.21.15.71:0/1116347601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f115c0ffac0 msgr2=0x7f115c0fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:11.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.323+0000 7f1161f89700 1 --2- 172.21.15.71:0/1116347601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f115c0ffac0 0x7f115c0fff20 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1150009910 tx=0x7f1150009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:11.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.324+0000 7f1161f89700 1 -- 172.21.15.71:0/1116347601 shutdown_connections 2024-04-23T02:55:11.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.324+0000 7f1161f89700 1 --2- 172.21.15.71:0/1116347601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f115c0ffac0 0x7f115c0fff20 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:11.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.324+0000 7f1161f89700 1 --2- 172.21.15.71:0/1116347601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f115c105990 0x7f115c105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:11.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.324+0000 7f1161f89700 1 -- 172.21.15.71:0/1116347601 >> 172.21.15.71:0/1116347601 conn(0x7f115c075400 msgr2=0x7f115c075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:11.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.324+0000 7f1161f89700 1 -- 172.21.15.71:0/1116347601 shutdown_connections 2024-04-23T02:55:11.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.324+0000 7f1161f89700 1 -- 172.21.15.71:0/1116347601 wait complete. 2024-04-23T02:55:11.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.324+0000 7f1161f89700 1 Processor -- start 2024-04-23T02:55:11.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.324+0000 7f1161f89700 1 -- start start 2024-04-23T02:55:11.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.325+0000 7f1161f89700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f115c0ffac0 0x7f115c197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:11.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.325+0000 7f1161f89700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f115c105990 0x7f115c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:11.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.325+0000 7f1161f89700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c197e60 con 0x7f115c105990 2024-04-23T02:55:11.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.325+0000 7f1161f89700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c197fd0 con 0x7f115c0ffac0 2024-04-23T02:55:11.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.325+0000 7f115affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f115c105990 0x7f115c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:11.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.325+0000 7f115affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f115c105990 0x7f115c197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60670/0 (socket says 172.21.15.71:60670) 2024-04-23T02:55:11.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.325+0000 7f115affd700 1 -- 172.21.15.71:0/4106822409 learned_addr learned my addr 172.21.15.71:0/4106822409 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:11.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.325+0000 7f115b7fe700 1 --2- 172.21.15.71:0/4106822409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f115c0ffac0 0x7f115c197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:11.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.325+0000 7f115affd700 1 -- 172.21.15.71:0/4106822409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f115c0ffac0 msgr2=0x7f115c197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:11.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.325+0000 7f115affd700 1 --2- 172.21.15.71:0/4106822409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f115c0ffac0 0x7f115c197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:11.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.325+0000 7f115affd700 1 -- 172.21.15.71:0/4106822409 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11500095f0 con 0x7f115c105990 2024-04-23T02:55:11.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.325+0000 7f115affd700 1 --2- 172.21.15.71:0/4106822409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f115c105990 0x7f115c197840 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f1150000c00 tx=0x7f115000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:11.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.325+0000 7f1158ff9700 1 -- 172.21.15.71:0/4106822409 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f115001d070 con 0x7f115c105990 2024-04-23T02:55:11.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.325+0000 7f1161f89700 1 -- 172.21.15.71:0/4106822409 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115c1a0370 con 0x7f115c105990 2024-04-23T02:55:11.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.325+0000 7f1158ff9700 1 -- 172.21.15.71:0/4106822409 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1150005710 con 0x7f115c105990 2024-04-23T02:55:11.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.325+0000 7f1158ff9700 1 -- 172.21.15.71:0/4106822409 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11500039d0 con 0x7f115c105990 2024-04-23T02:55:11.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.325+0000 7f1161f89700 1 -- 172.21.15.71:0/4106822409 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115c1a0790 con 0x7f115c105990 2024-04-23T02:55:11.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.326+0000 7f1161f89700 1 -- 172.21.15.71:0/4106822409 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115c107f10 con 0x7f115c105990 2024-04-23T02:55:11.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.327+0000 7f1158ff9700 1 -- 172.21.15.71:0/4106822409 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1150003c50 con 0x7f115c105990 2024-04-23T02:55:11.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.327+0000 7f1158ff9700 1 --2- 172.21.15.71:0/4106822409 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f114406cda0 0x7f114406f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:11.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.327+0000 7f1158ff9700 1 -- 172.21.15.71:0/4106822409 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f115008db70 con 0x7f115c105990 2024-04-23T02:55:11.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.327+0000 7f115b7fe700 1 --2- 172.21.15.71:0/4106822409 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f114406cda0 0x7f114406f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:11.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.327+0000 7f115b7fe700 1 --2- 172.21.15.71:0/4106822409 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f114406cda0 0x7f114406f260 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f114c009c60 tx=0x7f114c009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:11.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.331+0000 7f1158ff9700 1 -- 172.21.15.71:0/4106822409 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1150057be0 con 0x7f115c105990 2024-04-23T02:55:11.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.570+0000 7f1161f89700 1 -- 172.21.15.71:0/4106822409 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7f115c04ea90 con 0x7f115c105990 2024-04-23T02:55:11.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.571+0000 7f1158ff9700 1 -- 172.21.15.71:0/4106822409 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v46) v1 ==== 107+0+4823 (secure 0 0 0) 0x7f115005b200 con 0x7f115c105990 2024-04-23T02:55:11.574 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:11.574 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34334,"name":"cephfs.smithi071.qusrrb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6827/2539819173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2539819173},{"type":"v1","addr":"172.21.15.71:6827","nonce":2539819173}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":34362,"name":"cephfs.smithi071.vxejye","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6829/143423621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":143423621},{"type":"v1","addr":"172.21.15.71:6829","nonce":143423621}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":44319,"name":"cephfs.smithi192.gtmyst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3688927981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3688927981},{"type":"v1","addr":"172.21.15.192:6825","nonce":3688927981}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":44793,"name":"cephfs.smithi192.azdyze","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/3768627645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3768627645},{"type":"v1","addr":"172.21.15.192:6827","nonce":3768627645}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:52:59.452988+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:11.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.573+0000 7f1161f89700 1 -- 172.21.15.71:0/4106822409 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f114406cda0 msgr2=0x7f114406f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:11.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.573+0000 7f1161f89700 1 --2- 172.21.15.71:0/4106822409 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f114406cda0 0x7f114406f260 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f114c009c60 tx=0x7f114c009260 comp rx=0 tx=0).stop 2024-04-23T02:55:11.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.573+0000 7f1161f89700 1 -- 172.21.15.71:0/4106822409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f115c105990 msgr2=0x7f115c197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:11.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.573+0000 7f1161f89700 1 --2- 172.21.15.71:0/4106822409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f115c105990 0x7f115c197840 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f1150000c00 tx=0x7f115000bde0 comp rx=0 tx=0).stop 2024-04-23T02:55:11.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.573+0000 7f1161f89700 1 -- 172.21.15.71:0/4106822409 shutdown_connections 2024-04-23T02:55:11.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.573+0000 7f1161f89700 1 --2- 172.21.15.71:0/4106822409 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f114406cda0 0x7f114406f260 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:11.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.573+0000 7f1161f89700 1 --2- 172.21.15.71:0/4106822409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f115c0ffac0 0x7f115c197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:11.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.573+0000 7f1161f89700 1 --2- 172.21.15.71:0/4106822409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f115c105990 0x7f115c197840 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:11.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.573+0000 7f1161f89700 1 -- 172.21.15.71:0/4106822409 >> 172.21.15.71:0/4106822409 conn(0x7f115c075400 msgr2=0x7f115c0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:11.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.574+0000 7f1161f89700 1 -- 172.21.15.71:0/4106822409 shutdown_connections 2024-04-23T02:55:11.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.574+0000 7f1161f89700 1 -- 172.21.15.71:0/4106822409 wait complete. 2024-04-23T02:55:11.577 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 37 2024-04-23T02:55:11.588 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 37 2024-04-23T02:55:11.588 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-04-23T02:55:11.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.709+0000 7f3ec3df9700 1 -- 172.21.15.71:0/1996428551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc0ffac0 msgr2=0x7f3ebc0fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:11.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.709+0000 7f3ec3df9700 1 --2- 172.21.15.71:0/1996428551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc0ffac0 0x7f3ebc0fff20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0009910 tx=0x7f3eb0009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:11.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.709+0000 7f3ec3df9700 1 -- 172.21.15.71:0/1996428551 shutdown_connections 2024-04-23T02:55:11.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.709+0000 7f3ec3df9700 1 --2- 172.21.15.71:0/1996428551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc0ffac0 0x7f3ebc0fff20 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:11.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.709+0000 7f3ec3df9700 1 --2- 172.21.15.71:0/1996428551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ebc105990 0x7f3ebc105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:11.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.709+0000 7f3ec3df9700 1 -- 172.21.15.71:0/1996428551 >> 172.21.15.71:0/1996428551 conn(0x7f3ebc075400 msgr2=0x7f3ebc075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:11.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.709+0000 7f3ec3df9700 1 -- 172.21.15.71:0/1996428551 shutdown_connections 2024-04-23T02:55:11.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.709+0000 7f3ec3df9700 1 -- 172.21.15.71:0/1996428551 wait complete. 2024-04-23T02:55:11.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.709+0000 7f3ec3df9700 1 Processor -- start 2024-04-23T02:55:11.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.709+0000 7f3ec3df9700 1 -- start start 2024-04-23T02:55:11.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.710+0000 7f3ec3df9700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ebc0ffac0 0x7f3ebc1953c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:11.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.710+0000 7f3ec3df9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc105990 0x7f3ebc195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:11.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.710+0000 7f3ec3df9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc195fe0 con 0x7f3ebc105990 2024-04-23T02:55:11.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.710+0000 7f3ec3df9700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc199d60 con 0x7f3ebc0ffac0 2024-04-23T02:55:11.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.710+0000 7f3ec1394700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc105990 0x7f3ebc195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:11.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.710+0000 7f3ec1394700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc105990 0x7f3ebc195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60698/0 (socket says 172.21.15.71:60698) 2024-04-23T02:55:11.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.710+0000 7f3ec1394700 1 -- 172.21.15.71:0/13975143 learned_addr learned my addr 172.21.15.71:0/13975143 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:11.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.710+0000 7f3ec1b95700 1 --2- 172.21.15.71:0/13975143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ebc0ffac0 0x7f3ebc1953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:11.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.710+0000 7f3ec1394700 1 -- 172.21.15.71:0/13975143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ebc0ffac0 msgr2=0x7f3ebc1953c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:11.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.710+0000 7f3ec1394700 1 --2- 172.21.15.71:0/13975143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ebc0ffac0 0x7f3ebc1953c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:11.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.710+0000 7f3ec1394700 1 -- 172.21.15.71:0/13975143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb00095f0 con 0x7f3ebc105990 2024-04-23T02:55:11.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.710+0000 7f3ec1394700 1 --2- 172.21.15.71:0/13975143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc105990 0x7f3ebc195900 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0006010 tx=0x7f3eb000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:11.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.710+0000 7f3ebaffd700 1 -- 172.21.15.71:0/13975143 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3eb001f070 con 0x7f3ebc105990 2024-04-23T02:55:11.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.710+0000 7f3ebaffd700 1 -- 172.21.15.71:0/13975143 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3eb0005710 con 0x7f3ebc105990 2024-04-23T02:55:11.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.710+0000 7f3ebaffd700 1 -- 172.21.15.71:0/13975143 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3eb000f460 con 0x7f3ebc105990 2024-04-23T02:55:11.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.710+0000 7f3ec3df9700 1 -- 172.21.15.71:0/13975143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ebc199fe0 con 0x7f3ebc105990 2024-04-23T02:55:11.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.711+0000 7f3ec3df9700 1 -- 172.21.15.71:0/13975143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ebc19a4e0 con 0x7f3ebc105990 2024-04-23T02:55:11.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.711+0000 7f3ec3df9700 1 -- 172.21.15.71:0/13975143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ebc04ea90 con 0x7f3ebc105990 2024-04-23T02:55:11.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.712+0000 7f3ebaffd700 1 -- 172.21.15.71:0/13975143 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3eb000f5e0 con 0x7f3ebc105990 2024-04-23T02:55:11.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.712+0000 7f3ebaffd700 1 --2- 172.21.15.71:0/13975143 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3ea406cda0 0x7f3ea406f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:11.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.712+0000 7f3ebaffd700 1 -- 172.21.15.71:0/13975143 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3eb008f920 con 0x7f3ebc105990 2024-04-23T02:55:11.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.712+0000 7f3ec1b95700 1 --2- 172.21.15.71:0/13975143 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3ea406cda0 0x7f3ea406f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:11.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.713+0000 7f3ec1b95700 1 --2- 172.21.15.71:0/13975143 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3ea406cda0 0x7f3ea406f260 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4005950 tx=0x7f3eb400b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:11.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.717+0000 7f3ebaffd700 1 -- 172.21.15.71:0/13975143 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3eb0059990 con 0x7f3ebc105990 2024-04-23T02:55:11.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.951+0000 7f3ec3df9700 1 -- 172.21.15.71:0/13975143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f3ebc19a7a0 con 0x7f3ebc105990 2024-04-23T02:55:11.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.952+0000 7f3ebaffd700 1 -- 172.21.15.71:0/13975143 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v46) v1 ==== 107+0+4846 (secure 0 0 0) 0x7f3eb005cfb0 con 0x7f3ebc105990 2024-04-23T02:55:11.954 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:11.954 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44319,"name":"cephfs.smithi192.gtmyst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3688927981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3688927981},{"type":"v1","addr":"172.21.15.192:6825","nonce":3688927981}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":44793,"name":"cephfs.smithi192.azdyze","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/3768627645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3768627645},{"type":"v1","addr":"172.21.15.192:6827","nonce":3768627645}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:52:59.456667+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34334,"mds_1":34362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34334":{"gid":34334,"name":"cephfs.smithi071.qusrrb","rank":0,"incarnation":38,"state":"up:replay","state_seq":1,"addr":"172.21.15.71:6827/2539819173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2539819173},{"type":"v1","addr":"172.21.15.71:6827","nonce":2539819173}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34362":{"gid":34362,"name":"cephfs.smithi071.vxejye","rank":1,"incarnation":38,"state":"up:replay","state_seq":1,"addr":"172.21.15.71:6829/143423621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":143423621},{"type":"v1","addr":"172.21.15.71:6829","nonce":143423621}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:11.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.954+0000 7f3ec3df9700 1 -- 172.21.15.71:0/13975143 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3ea406cda0 msgr2=0x7f3ea406f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:11.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.954+0000 7f3ec3df9700 1 --2- 172.21.15.71:0/13975143 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3ea406cda0 0x7f3ea406f260 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4005950 tx=0x7f3eb400b410 comp rx=0 tx=0).stop 2024-04-23T02:55:11.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.954+0000 7f3ec3df9700 1 -- 172.21.15.71:0/13975143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc105990 msgr2=0x7f3ebc195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:11.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.954+0000 7f3ec3df9700 1 --2- 172.21.15.71:0/13975143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc105990 0x7f3ebc195900 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0006010 tx=0x7f3eb000bd70 comp rx=0 tx=0).stop 2024-04-23T02:55:11.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.954+0000 7f3ec3df9700 1 -- 172.21.15.71:0/13975143 shutdown_connections 2024-04-23T02:55:11.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.954+0000 7f3ec3df9700 1 --2- 172.21.15.71:0/13975143 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3ea406cda0 0x7f3ea406f260 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:11.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.954+0000 7f3ec3df9700 1 --2- 172.21.15.71:0/13975143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ebc0ffac0 0x7f3ebc1953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:11.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.954+0000 7f3ec3df9700 1 --2- 172.21.15.71:0/13975143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc105990 0x7f3ebc195900 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:11.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.954+0000 7f3ec3df9700 1 -- 172.21.15.71:0/13975143 >> 172.21.15.71:0/13975143 conn(0x7f3ebc075400 msgr2=0x7f3ebc0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:11.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.954+0000 7f3ec3df9700 1 -- 172.21.15.71:0/13975143 shutdown_connections 2024-04-23T02:55:11.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:11.954+0000 7f3ec3df9700 1 -- 172.21.15.71:0/13975143 wait complete. 2024-04-23T02:55:11.958 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 38 2024-04-23T02:55:11.967 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:11 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2597170451' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-23T02:55:11.967 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:11 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2711395589' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-04-23T02:55:11.967 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:11 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/4106822409' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-04-23T02:55:11.968 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 38 2024-04-23T02:55:11.969 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 39 2024-04-23T02:55:12.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.087+0000 7ff8492a8700 1 -- 172.21.15.71:0/813536054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8440ff990 msgr2=0x7ff8440ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:12.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.087+0000 7ff8492a8700 1 --2- 172.21.15.71:0/813536054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8440ff990 0x7ff8440ffe10 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ff838009910 tx=0x7ff838009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:12.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.088+0000 7ff8492a8700 1 -- 172.21.15.71:0/813536054 shutdown_connections 2024-04-23T02:55:12.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.088+0000 7ff8492a8700 1 --2- 172.21.15.71:0/813536054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8440ff990 0x7ff8440ffe10 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:12.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.088+0000 7ff8492a8700 1 --2- 172.21.15.71:0/813536054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8441059a0 0x7ff844105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:12.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.088+0000 7ff8492a8700 1 -- 172.21.15.71:0/813536054 >> 172.21.15.71:0/813536054 conn(0x7ff8440fb480 msgr2=0x7ff8440fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:12.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.088+0000 7ff8492a8700 1 -- 172.21.15.71:0/813536054 shutdown_connections 2024-04-23T02:55:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.088+0000 7ff8492a8700 1 -- 172.21.15.71:0/813536054 wait complete. 2024-04-23T02:55:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.088+0000 7ff8492a8700 1 Processor -- start 2024-04-23T02:55:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.089+0000 7ff8492a8700 1 -- start start 2024-04-23T02:55:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.089+0000 7ff8492a8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8440ff990 0x7ff8441973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.089+0000 7ff8492a8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8441059a0 0x7ff844197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.089+0000 7ff8492a8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff844197f20 con 0x7ff8440ff990 2024-04-23T02:55:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.089+0000 7ff8492a8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff844198090 con 0x7ff8441059a0 2024-04-23T02:55:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.089+0000 7ff842ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8440ff990 0x7ff8441973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.089+0000 7ff842ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8440ff990 0x7ff8441973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60728/0 (socket says 172.21.15.71:60728) 2024-04-23T02:55:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.089+0000 7ff842ffd700 1 -- 172.21.15.71:0/4055066332 learned_addr learned my addr 172.21.15.71:0/4055066332 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.089+0000 7ff8427fc700 1 --2- 172.21.15.71:0/4055066332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8441059a0 0x7ff844197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.089+0000 7ff842ffd700 1 -- 172.21.15.71:0/4055066332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8441059a0 msgr2=0x7ff844197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.089+0000 7ff842ffd700 1 --2- 172.21.15.71:0/4055066332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8441059a0 0x7ff844197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.089+0000 7ff842ffd700 1 -- 172.21.15.71:0/4055066332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8380095f0 con 0x7ff8440ff990 2024-04-23T02:55:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.089+0000 7ff8427fc700 1 --2- 172.21.15.71:0/4055066332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8441059a0 0x7ff844197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:55:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.089+0000 7ff842ffd700 1 --2- 172.21.15.71:0/4055066332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8440ff990 0x7ff8441973c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7ff83400d900 tx=0x7ff83400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.089+0000 7ff82bfff700 1 -- 172.21.15.71:0/4055066332 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff834004a00 con 0x7ff8440ff990 2024-04-23T02:55:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.090+0000 7ff82bfff700 1 -- 172.21.15.71:0/4055066332 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff834005020 con 0x7ff8440ff990 2024-04-23T02:55:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.090+0000 7ff8492a8700 1 -- 172.21.15.71:0/4055066332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8441a0430 con 0x7ff8440ff990 2024-04-23T02:55:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.090+0000 7ff82bfff700 1 -- 172.21.15.71:0/4055066332 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff834009c00 con 0x7ff8440ff990 2024-04-23T02:55:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.090+0000 7ff8492a8700 1 -- 172.21.15.71:0/4055066332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8441a0930 con 0x7ff8440ff990 2024-04-23T02:55:12.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.091+0000 7ff8492a8700 1 -- 172.21.15.71:0/4055066332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff844107f20 con 0x7ff8440ff990 2024-04-23T02:55:12.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.091+0000 7ff82bfff700 1 -- 172.21.15.71:0/4055066332 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff834005190 con 0x7ff8440ff990 2024-04-23T02:55:12.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.091+0000 7ff82bfff700 1 --2- 172.21.15.71:0/4055066332 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff82c06cda0 0x7ff82c06f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:12.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.092+0000 7ff82bfff700 1 -- 172.21.15.71:0/4055066332 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff83408c770 con 0x7ff8440ff990 2024-04-23T02:55:12.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.092+0000 7ff8427fc700 1 --2- 172.21.15.71:0/4055066332 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff82c06cda0 0x7ff82c06f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.092+0000 7ff8427fc700 1 --2- 172.21.15.71:0/4055066332 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff82c06cda0 0x7ff82c06f260 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff838006010 tx=0x7ff838005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.096+0000 7ff82bfff700 1 -- 172.21.15.71:0/4055066332 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff8340567e0 con 0x7ff8440ff990 2024-04-23T02:55:12.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:11 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2597170451' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-23T02:55:12.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:11 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2711395589' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-04-23T02:55:12.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:11 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/4106822409' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-04-23T02:55:12.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.335+0000 7ff8492a8700 1 -- 172.21.15.71:0/4055066332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7ff84402cc70 con 0x7ff8440ff990 2024-04-23T02:55:12.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.335+0000 7ff82bfff700 1 -- 172.21.15.71:0/4055066332 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v46) v1 ==== 107+0+4845 (secure 0 0 0) 0x7ff834005440 con 0x7ff8440ff990 2024-04-23T02:55:12.338 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:12.338 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44319,"name":"cephfs.smithi192.gtmyst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3688927981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3688927981},{"type":"v1","addr":"172.21.15.192:6825","nonce":3688927981}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":44793,"name":"cephfs.smithi192.azdyze","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/3768627645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3768627645},{"type":"v1","addr":"172.21.15.192:6827","nonce":3768627645}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:53:00.460445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34334,"mds_1":34362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34334":{"gid":34334,"name":"cephfs.smithi071.qusrrb","rank":0,"incarnation":38,"state":"up:replay","state_seq":1,"addr":"172.21.15.71:6827/2539819173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2539819173},{"type":"v1","addr":"172.21.15.71:6827","nonce":2539819173}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34362":{"gid":34362,"name":"cephfs.smithi071.vxejye","rank":1,"incarnation":38,"state":"up:replay","state_seq":1,"addr":"172.21.15.71:6829/143423621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":143423621},{"type":"v1","addr":"172.21.15.71:6829","nonce":143423621}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:12.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.337+0000 7ff8492a8700 1 -- 172.21.15.71:0/4055066332 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff82c06cda0 msgr2=0x7ff82c06f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:12.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.337+0000 7ff8492a8700 1 --2- 172.21.15.71:0/4055066332 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff82c06cda0 0x7ff82c06f260 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff838006010 tx=0x7ff838005960 comp rx=0 tx=0).stop 2024-04-23T02:55:12.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.337+0000 7ff8492a8700 1 -- 172.21.15.71:0/4055066332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8440ff990 msgr2=0x7ff8441973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:12.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.337+0000 7ff8492a8700 1 --2- 172.21.15.71:0/4055066332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8440ff990 0x7ff8441973c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7ff83400d900 tx=0x7ff83400dcc0 comp rx=0 tx=0).stop 2024-04-23T02:55:12.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.337+0000 7ff8492a8700 1 -- 172.21.15.71:0/4055066332 shutdown_connections 2024-04-23T02:55:12.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.337+0000 7ff8492a8700 1 --2- 172.21.15.71:0/4055066332 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7ff82c06cda0 0x7ff82c06f260 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:12.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.338+0000 7ff8492a8700 1 --2- 172.21.15.71:0/4055066332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8440ff990 0x7ff8441973c0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:12.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.338+0000 7ff8492a8700 1 --2- 172.21.15.71:0/4055066332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8441059a0 0x7ff844197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:12.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.338+0000 7ff8492a8700 1 -- 172.21.15.71:0/4055066332 >> 172.21.15.71:0/4055066332 conn(0x7ff8440fb480 msgr2=0x7ff8440fbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:12.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.338+0000 7ff8492a8700 1 -- 172.21.15.71:0/4055066332 shutdown_connections 2024-04-23T02:55:12.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.338+0000 7ff8492a8700 1 -- 172.21.15.71:0/4055066332 wait complete. 2024-04-23T02:55:12.341 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 39 2024-04-23T02:55:12.352 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 40 2024-04-23T02:55:12.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.478+0000 7f3c1b669700 1 -- 172.21.15.71:0/3670885736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c14105990 msgr2=0x7f3c14105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:12.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.478+0000 7f3c1b669700 1 --2- 172.21.15.71:0/3670885736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c14105990 0x7f3c14105d70 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f3c10009910 tx=0x7f3c10009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:12.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.479+0000 7f3c1b669700 1 -- 172.21.15.71:0/3670885736 shutdown_connections 2024-04-23T02:55:12.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.479+0000 7f3c1b669700 1 --2- 172.21.15.71:0/3670885736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c140ffac0 0x7f3c140fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:12.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.479+0000 7f3c1b669700 1 --2- 172.21.15.71:0/3670885736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c14105990 0x7f3c14105d70 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:12.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.479+0000 7f3c1b669700 1 -- 172.21.15.71:0/3670885736 >> 172.21.15.71:0/3670885736 conn(0x7f3c14075400 msgr2=0x7f3c14075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:12.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.479+0000 7f3c1b669700 1 -- 172.21.15.71:0/3670885736 shutdown_connections 2024-04-23T02:55:12.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.479+0000 7f3c1b669700 1 -- 172.21.15.71:0/3670885736 wait complete. 2024-04-23T02:55:12.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.479+0000 7f3c1b669700 1 Processor -- start 2024-04-23T02:55:12.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.479+0000 7f3c1b669700 1 -- start start 2024-04-23T02:55:12.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.479+0000 7f3c1b669700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c140ffac0 0x7f3c141954f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:12.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.479+0000 7f3c1b669700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c14105990 0x7f3c14195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:12.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.479+0000 7f3c1b669700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c14196080 con 0x7f3c14105990 2024-04-23T02:55:12.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.479+0000 7f3c1b669700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c141961c0 con 0x7f3c140ffac0 2024-04-23T02:55:12.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.480+0000 7f3c18c04700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c14105990 0x7f3c14195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:12.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.480+0000 7f3c18c04700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c14105990 0x7f3c14195a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60750/0 (socket says 172.21.15.71:60750) 2024-04-23T02:55:12.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.480+0000 7f3c18c04700 1 -- 172.21.15.71:0/2104046798 learned_addr learned my addr 172.21.15.71:0/2104046798 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:12.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.480+0000 7f3c19405700 1 --2- 172.21.15.71:0/2104046798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c140ffac0 0x7f3c141954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:12.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.480+0000 7f3c18c04700 1 -- 172.21.15.71:0/2104046798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c140ffac0 msgr2=0x7f3c141954f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:12.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.480+0000 7f3c18c04700 1 --2- 172.21.15.71:0/2104046798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c140ffac0 0x7f3c141954f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:12.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.480+0000 7f3c18c04700 1 -- 172.21.15.71:0/2104046798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c100095f0 con 0x7f3c14105990 2024-04-23T02:55:12.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.480+0000 7f3c19405700 1 --2- 172.21.15.71:0/2104046798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c140ffac0 0x7f3c141954f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:55:12.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.480+0000 7f3c18c04700 1 --2- 172.21.15.71:0/2104046798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c14105990 0x7f3c14195a30 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f3c04007b50 tx=0x7f3c04007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:12.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.480+0000 7f3c0a7fc700 1 -- 172.21.15.71:0/2104046798 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c04011070 con 0x7f3c14105990 2024-04-23T02:55:12.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.480+0000 7f3c0a7fc700 1 -- 172.21.15.71:0/2104046798 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3c0400c2e0 con 0x7f3c14105990 2024-04-23T02:55:12.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.480+0000 7f3c0a7fc700 1 -- 172.21.15.71:0/2104046798 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c0400ac60 con 0x7f3c14105990 2024-04-23T02:55:12.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.480+0000 7f3c1b669700 1 -- 172.21.15.71:0/2104046798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c1419a030 con 0x7f3c14105990 2024-04-23T02:55:12.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.480+0000 7f3c1b669700 1 -- 172.21.15.71:0/2104046798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c1419a590 con 0x7f3c14105990 2024-04-23T02:55:12.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.481+0000 7f3c1b669700 1 -- 172.21.15.71:0/2104046798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c1404ea90 con 0x7f3c14105990 2024-04-23T02:55:12.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.482+0000 7f3c0a7fc700 1 -- 172.21.15.71:0/2104046798 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3c0400e710 con 0x7f3c14105990 2024-04-23T02:55:12.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.482+0000 7f3c0a7fc700 1 --2- 172.21.15.71:0/2104046798 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3c0006cda0 0x7f3c0006f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:12.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.482+0000 7f3c0a7fc700 1 -- 172.21.15.71:0/2104046798 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3c0408d890 con 0x7f3c14105990 2024-04-23T02:55:12.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.482+0000 7f3c19405700 1 --2- 172.21.15.71:0/2104046798 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3c0006cda0 0x7f3c0006f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:12.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.482+0000 7f3c19405700 1 --2- 172.21.15.71:0/2104046798 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3c0006cda0 0x7f3c0006f260 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3c10006010 tx=0x7f3c1001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:12.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.486+0000 7f3c0a7fc700 1 -- 172.21.15.71:0/2104046798 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3c04057900 con 0x7f3c14105990 2024-04-23T02:55:12.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.726+0000 7f3c1b669700 1 -- 172.21.15.71:0/2104046798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 40, "format": "json"} v 0) v1 -- 0x7f3c1419a810 con 0x7f3c14105990 2024-04-23T02:55:12.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.727+0000 7f3c0a7fc700 1 -- 172.21.15.71:0/2104046798 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 40, "format": "json"}]=0 dumped fsmap epoch 40 v46) v1 ==== 107+0+4848 (secure 0 0 0) 0x7f3c0405af20 con 0x7f3c14105990 2024-04-23T02:55:12.729 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:12.729 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44319,"name":"cephfs.smithi192.gtmyst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3688927981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3688927981},{"type":"v1","addr":"172.21.15.192:6825","nonce":3688927981}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":44793,"name":"cephfs.smithi192.azdyze","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/3768627645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3768627645},{"type":"v1","addr":"172.21.15.192:6827","nonce":3768627645}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:53:05.764132+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34334,"mds_1":34362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34334":{"gid":34334,"name":"cephfs.smithi071.qusrrb","rank":0,"incarnation":38,"state":"up:replay","state_seq":1,"addr":"172.21.15.71:6827/2539819173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2539819173},{"type":"v1","addr":"172.21.15.71:6827","nonce":2539819173}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34362":{"gid":34362,"name":"cephfs.smithi071.vxejye","rank":1,"incarnation":38,"state":"up:resolve","state_seq":460,"addr":"172.21.15.71:6829/143423621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":143423621},{"type":"v1","addr":"172.21.15.71:6829","nonce":143423621}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:12.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.729+0000 7f3c1b669700 1 -- 172.21.15.71:0/2104046798 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3c0006cda0 msgr2=0x7f3c0006f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:12.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.729+0000 7f3c1b669700 1 --2- 172.21.15.71:0/2104046798 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3c0006cda0 0x7f3c0006f260 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3c10006010 tx=0x7f3c1001a040 comp rx=0 tx=0).stop 2024-04-23T02:55:12.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.729+0000 7f3c1b669700 1 -- 172.21.15.71:0/2104046798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c14105990 msgr2=0x7f3c14195a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:12.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.729+0000 7f3c1b669700 1 --2- 172.21.15.71:0/2104046798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c14105990 0x7f3c14195a30 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f3c04007b50 tx=0x7f3c04007f10 comp rx=0 tx=0).stop 2024-04-23T02:55:12.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.729+0000 7f3c1b669700 1 -- 172.21.15.71:0/2104046798 shutdown_connections 2024-04-23T02:55:12.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.729+0000 7f3c1b669700 1 --2- 172.21.15.71:0/2104046798 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f3c0006cda0 0x7f3c0006f260 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:12.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.729+0000 7f3c1b669700 1 --2- 172.21.15.71:0/2104046798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c140ffac0 0x7f3c141954f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:12.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.729+0000 7f3c1b669700 1 --2- 172.21.15.71:0/2104046798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c14105990 0x7f3c14195a30 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:12.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.729+0000 7f3c1b669700 1 -- 172.21.15.71:0/2104046798 >> 172.21.15.71:0/2104046798 conn(0x7f3c14075400 msgr2=0x7f3c140fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:12.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.729+0000 7f3c1b669700 1 -- 172.21.15.71:0/2104046798 shutdown_connections 2024-04-23T02:55:12.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.729+0000 7f3c1b669700 1 -- 172.21.15.71:0/2104046798 wait complete. 2024-04-23T02:55:12.732 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 40 2024-04-23T02:55:12.743 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 41 2024-04-23T02:55:12.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.865+0000 7f5d71d78700 1 -- 172.21.15.71:0/2710747440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d6c0ffac0 msgr2=0x7f5d6c0fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:12.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.865+0000 7f5d71d78700 1 --2- 172.21.15.71:0/2710747440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d6c0ffac0 0x7f5d6c0fff20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009910 tx=0x7f5d60009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:12.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.865+0000 7f5d71d78700 1 -- 172.21.15.71:0/2710747440 shutdown_connections 2024-04-23T02:55:12.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.865+0000 7f5d71d78700 1 --2- 172.21.15.71:0/2710747440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d6c0ffac0 0x7f5d6c0fff20 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:12.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.865+0000 7f5d71d78700 1 --2- 172.21.15.71:0/2710747440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d6c105990 0x7f5d6c105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:12.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.865+0000 7f5d71d78700 1 -- 172.21.15.71:0/2710747440 >> 172.21.15.71:0/2710747440 conn(0x7f5d6c075400 msgr2=0x7f5d6c075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:12.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.865+0000 7f5d71d78700 1 -- 172.21.15.71:0/2710747440 shutdown_connections 2024-04-23T02:55:12.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.865+0000 7f5d71d78700 1 -- 172.21.15.71:0/2710747440 wait complete. 2024-04-23T02:55:12.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.866+0000 7f5d71d78700 1 Processor -- start 2024-04-23T02:55:12.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.866+0000 7f5d71d78700 1 -- start start 2024-04-23T02:55:12.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.866+0000 7f5d71d78700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d6c0ffac0 0x7f5d6c1953c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:12.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.867+0000 7f5d71d78700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d6c105990 0x7f5d6c195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:12.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.867+0000 7f5d71d78700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c195fe0 con 0x7f5d6c0ffac0 2024-04-23T02:55:12.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.867+0000 7f5d71d78700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c199d60 con 0x7f5d6c105990 2024-04-23T02:55:12.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.867+0000 7f5d6b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d6c0ffac0 0x7f5d6c1953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:12.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.867+0000 7f5d6b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d6c0ffac0 0x7f5d6c1953c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60756/0 (socket says 172.21.15.71:60756) 2024-04-23T02:55:12.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.867+0000 7f5d6b7fe700 1 -- 172.21.15.71:0/1553269857 learned_addr learned my addr 172.21.15.71:0/1553269857 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:12.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.867+0000 7f5d6affd700 1 --2- 172.21.15.71:0/1553269857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d6c105990 0x7f5d6c195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:12.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.867+0000 7f5d6b7fe700 1 -- 172.21.15.71:0/1553269857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d6c105990 msgr2=0x7f5d6c195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:12.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.867+0000 7f5d6b7fe700 1 --2- 172.21.15.71:0/1553269857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d6c105990 0x7f5d6c195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:12.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.867+0000 7f5d6b7fe700 1 -- 172.21.15.71:0/1553269857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d600095f0 con 0x7f5d6c0ffac0 2024-04-23T02:55:12.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.867+0000 7f5d6b7fe700 1 --2- 172.21.15.71:0/1553269857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d6c0ffac0 0x7f5d6c1953c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c00d8d0 tx=0x7f5d5c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:12.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.867+0000 7f5d68ff9700 1 -- 172.21.15.71:0/1553269857 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d5c0098a0 con 0x7f5d6c0ffac0 2024-04-23T02:55:12.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.867+0000 7f5d68ff9700 1 -- 172.21.15.71:0/1553269857 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5d5c010460 con 0x7f5d6c0ffac0 2024-04-23T02:55:12.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.867+0000 7f5d71d78700 1 -- 172.21.15.71:0/1553269857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6c19a040 con 0x7f5d6c0ffac0 2024-04-23T02:55:12.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.867+0000 7f5d68ff9700 1 -- 172.21.15.71:0/1553269857 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d5c009ea0 con 0x7f5d6c0ffac0 2024-04-23T02:55:12.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.867+0000 7f5d71d78700 1 -- 172.21.15.71:0/1553269857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6c19a5a0 con 0x7f5d6c0ffac0 2024-04-23T02:55:12.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.868+0000 7f5d71d78700 1 -- 172.21.15.71:0/1553269857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d6c107f10 con 0x7f5d6c0ffac0 2024-04-23T02:55:12.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.869+0000 7f5d68ff9700 1 -- 172.21.15.71:0/1553269857 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5d5c00f6c0 con 0x7f5d6c0ffac0 2024-04-23T02:55:12.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.869+0000 7f5d68ff9700 1 --2- 172.21.15.71:0/1553269857 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5d5406cda0 0x7f5d5406f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:12.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.869+0000 7f5d68ff9700 1 -- 172.21.15.71:0/1553269857 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5d5c08c450 con 0x7f5d6c0ffac0 2024-04-23T02:55:12.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.869+0000 7f5d6affd700 1 --2- 172.21.15.71:0/1553269857 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5d5406cda0 0x7f5d5406f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:12.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.869+0000 7f5d6affd700 1 --2- 172.21.15.71:0/1553269857 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5d5406cda0 0x7f5d5406f260 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c1969d0 tx=0x7f5d6001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:12.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:12.874+0000 7f5d68ff9700 1 -- 172.21.15.71:0/1553269857 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5d5c0564c0 con 0x7f5d6c0ffac0 2024-04-23T02:55:12.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:12 smithi071 ceph-mon[144544]: pgmap v1259: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:12.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:12 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/13975143' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-04-23T02:55:12.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:12 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/4055066332' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-04-23T02:55:13.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.110+0000 7f5d71d78700 1 -- 172.21.15.71:0/1553269857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 41, "format": "json"} v 0) v1 -- 0x7f5d6c19a880 con 0x7f5d6c0ffac0 2024-04-23T02:55:13.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.111+0000 7f5d68ff9700 1 -- 172.21.15.71:0/1553269857 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 41, "format": "json"}]=0 dumped fsmap epoch 41 v46) v1 ==== 107+0+4851 (secure 0 0 0) 0x7f5d5c059ae0 con 0x7f5d6c0ffac0 2024-04-23T02:55:13.113 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:13.114 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44319,"name":"cephfs.smithi192.gtmyst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3688927981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3688927981},{"type":"v1","addr":"172.21.15.192:6825","nonce":3688927981}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":44793,"name":"cephfs.smithi192.azdyze","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/3768627645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3768627645},{"type":"v1","addr":"172.21.15.192:6827","nonce":3768627645}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:53:07.126590+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34334,"mds_1":34362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34334":{"gid":34334,"name":"cephfs.smithi071.qusrrb","rank":0,"incarnation":38,"state":"up:resolve","state_seq":466,"addr":"172.21.15.71:6827/2539819173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2539819173},{"type":"v1","addr":"172.21.15.71:6827","nonce":2539819173}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34362":{"gid":34362,"name":"cephfs.smithi071.vxejye","rank":1,"incarnation":38,"state":"up:resolve","state_seq":460,"addr":"172.21.15.71:6829/143423621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":143423621},{"type":"v1","addr":"172.21.15.71:6829","nonce":143423621}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:13.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.113+0000 7f5d71d78700 1 -- 172.21.15.71:0/1553269857 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5d5406cda0 msgr2=0x7f5d5406f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:13.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.113+0000 7f5d71d78700 1 --2- 172.21.15.71:0/1553269857 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5d5406cda0 0x7f5d5406f260 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c1969d0 tx=0x7f5d6001a040 comp rx=0 tx=0).stop 2024-04-23T02:55:13.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.113+0000 7f5d71d78700 1 -- 172.21.15.71:0/1553269857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d6c0ffac0 msgr2=0x7f5d6c1953c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:13.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.113+0000 7f5d71d78700 1 --2- 172.21.15.71:0/1553269857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d6c0ffac0 0x7f5d6c1953c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c00d8d0 tx=0x7f5d5c00dbe0 comp rx=0 tx=0).stop 2024-04-23T02:55:13.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.113+0000 7f5d71d78700 1 -- 172.21.15.71:0/1553269857 shutdown_connections 2024-04-23T02:55:13.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.113+0000 7f5d71d78700 1 --2- 172.21.15.71:0/1553269857 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f5d5406cda0 0x7f5d5406f260 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:13.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.113+0000 7f5d71d78700 1 --2- 172.21.15.71:0/1553269857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d6c0ffac0 0x7f5d6c1953c0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:13.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.113+0000 7f5d71d78700 1 --2- 172.21.15.71:0/1553269857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d6c105990 0x7f5d6c195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:13.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.113+0000 7f5d71d78700 1 -- 172.21.15.71:0/1553269857 >> 172.21.15.71:0/1553269857 conn(0x7f5d6c075400 msgr2=0x7f5d6c0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:13.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.113+0000 7f5d71d78700 1 -- 172.21.15.71:0/1553269857 shutdown_connections 2024-04-23T02:55:13.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.113+0000 7f5d71d78700 1 -- 172.21.15.71:0/1553269857 wait complete. 2024-04-23T02:55:13.117 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 41 2024-04-23T02:55:13.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:12 smithi192 ceph-mon[108617]: pgmap v1259: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:13.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:12 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/13975143' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-04-23T02:55:13.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:12 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/4055066332' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-04-23T02:55:13.128 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 42 2024-04-23T02:55:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.248+0000 7fa7b0772700 1 -- 172.21.15.71:0/2018398692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8105990 msgr2=0x7fa7a8105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.248+0000 7fa7b0772700 1 --2- 172.21.15.71:0/2018398692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8105990 0x7fa7a8105d70 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fa798009910 tx=0x7fa798009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.248+0000 7fa7b0772700 1 -- 172.21.15.71:0/2018398692 shutdown_connections 2024-04-23T02:55:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.248+0000 7fa7b0772700 1 --2- 172.21.15.71:0/2018398692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7a80ffac0 0x7fa7a80fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.248+0000 7fa7b0772700 1 --2- 172.21.15.71:0/2018398692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8105990 0x7fa7a8105d70 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.248+0000 7fa7b0772700 1 -- 172.21.15.71:0/2018398692 >> 172.21.15.71:0/2018398692 conn(0x7fa7a8075400 msgr2=0x7fa7a8075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.248+0000 7fa7b0772700 1 -- 172.21.15.71:0/2018398692 shutdown_connections 2024-04-23T02:55:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.248+0000 7fa7b0772700 1 -- 172.21.15.71:0/2018398692 wait complete. 2024-04-23T02:55:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.249+0000 7fa7b0772700 1 Processor -- start 2024-04-23T02:55:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.249+0000 7fa7b0772700 1 -- start start 2024-04-23T02:55:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.249+0000 7fa7b0772700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7a80ffac0 0x7fa7a8197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.249+0000 7fa7b0772700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8105990 0x7fa7a8197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.249+0000 7fa7b0772700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a8197f90 con 0x7fa7a8105990 2024-04-23T02:55:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.249+0000 7fa7b0772700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a8198100 con 0x7fa7a80ffac0 2024-04-23T02:55:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.249+0000 7fa7add0d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8105990 0x7fa7a8197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.249+0000 7fa7add0d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8105990 0x7fa7a8197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60774/0 (socket says 172.21.15.71:60774) 2024-04-23T02:55:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.249+0000 7fa7add0d700 1 -- 172.21.15.71:0/1497776182 learned_addr learned my addr 172.21.15.71:0/1497776182 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.249+0000 7fa7add0d700 1 -- 172.21.15.71:0/1497776182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7a80ffac0 msgr2=0x7fa7a8197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.249+0000 7fa7add0d700 1 --2- 172.21.15.71:0/1497776182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7a80ffac0 0x7fa7a8197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.250+0000 7fa7add0d700 1 -- 172.21.15.71:0/1497776182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7980095f0 con 0x7fa7a8105990 2024-04-23T02:55:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.250+0000 7fa7add0d700 1 --2- 172.21.15.71:0/1497776182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8105990 0x7fa7a8197970 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400c8a0 tx=0x7fa7a400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.250+0000 7fa79f7fe700 1 -- 172.21.15.71:0/1497776182 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7a400cea0 con 0x7fa7a8105990 2024-04-23T02:55:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.250+0000 7fa79f7fe700 1 -- 172.21.15.71:0/1497776182 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa7a4004830 con 0x7fa7a8105990 2024-04-23T02:55:13.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.250+0000 7fa7b0772700 1 -- 172.21.15.71:0/1497776182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a81a04a0 con 0x7fa7a8105990 2024-04-23T02:55:13.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.250+0000 7fa79f7fe700 1 -- 172.21.15.71:0/1497776182 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7a4005500 con 0x7fa7a8105990 2024-04-23T02:55:13.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.250+0000 7fa7b0772700 1 -- 172.21.15.71:0/1497776182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a81a09a0 con 0x7fa7a8105990 2024-04-23T02:55:13.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.251+0000 7fa7b0772700 1 -- 172.21.15.71:0/1497776182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a8107f30 con 0x7fa7a8105990 2024-04-23T02:55:13.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.251+0000 7fa79f7fe700 1 -- 172.21.15.71:0/1497776182 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa7a40049a0 con 0x7fa7a8105990 2024-04-23T02:55:13.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.252+0000 7fa79f7fe700 1 --2- 172.21.15.71:0/1497776182 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa79406cda0 0x7fa79406f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:13.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.252+0000 7fa79f7fe700 1 -- 172.21.15.71:0/1497776182 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa7a408c250 con 0x7fa7a8105990 2024-04-23T02:55:13.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.252+0000 7fa7ae50e700 1 --2- 172.21.15.71:0/1497776182 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa79406cda0 0x7fa79406f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:13.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.252+0000 7fa7ae50e700 1 --2- 172.21.15.71:0/1497776182 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa79406cda0 0x7fa79406f260 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa79800b5c0 tx=0x7fa798005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:13.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.256+0000 7fa79f7fe700 1 -- 172.21.15.71:0/1497776182 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa7a40562c0 con 0x7fa7a8105990 2024-04-23T02:55:13.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.494+0000 7fa7b0772700 1 -- 172.21.15.71:0/1497776182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 42, "format": "json"} v 0) v1 -- 0x7fa7a802cc70 con 0x7fa7a8105990 2024-04-23T02:55:13.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.495+0000 7fa79f7fe700 1 -- 172.21.15.71:0/1497776182 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 42, "format": "json"}]=0 dumped fsmap epoch 42 v46) v1 ==== 107+0+4855 (secure 0 0 0) 0x7fa7a40598e0 con 0x7fa7a8105990 2024-04-23T02:55:13.497 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:13.497 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44319,"name":"cephfs.smithi192.gtmyst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3688927981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3688927981},{"type":"v1","addr":"172.21.15.192:6825","nonce":3688927981}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":44793,"name":"cephfs.smithi192.azdyze","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/3768627645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3768627645},{"type":"v1","addr":"172.21.15.192:6827","nonce":3768627645}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:53:08.135331+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34334,"mds_1":34362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34334":{"gid":34334,"name":"cephfs.smithi071.qusrrb","rank":0,"incarnation":38,"state":"up:reconnect","state_seq":467,"addr":"172.21.15.71:6827/2539819173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2539819173},{"type":"v1","addr":"172.21.15.71:6827","nonce":2539819173}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34362":{"gid":34362,"name":"cephfs.smithi071.vxejye","rank":1,"incarnation":38,"state":"up:reconnect","state_seq":461,"addr":"172.21.15.71:6829/143423621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":143423621},{"type":"v1","addr":"172.21.15.71:6829","nonce":143423621}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:13.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.497+0000 7fa7b0772700 1 -- 172.21.15.71:0/1497776182 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa79406cda0 msgr2=0x7fa79406f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:13.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.497+0000 7fa7b0772700 1 --2- 172.21.15.71:0/1497776182 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa79406cda0 0x7fa79406f260 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa79800b5c0 tx=0x7fa798005960 comp rx=0 tx=0).stop 2024-04-23T02:55:13.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.497+0000 7fa7b0772700 1 -- 172.21.15.71:0/1497776182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8105990 msgr2=0x7fa7a8197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:13.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.497+0000 7fa7b0772700 1 --2- 172.21.15.71:0/1497776182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8105990 0x7fa7a8197970 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400c8a0 tx=0x7fa7a400cc60 comp rx=0 tx=0).stop 2024-04-23T02:55:13.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.497+0000 7fa7b0772700 1 -- 172.21.15.71:0/1497776182 shutdown_connections 2024-04-23T02:55:13.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.497+0000 7fa7b0772700 1 --2- 172.21.15.71:0/1497776182 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fa79406cda0 0x7fa79406f260 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:13.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.497+0000 7fa7b0772700 1 --2- 172.21.15.71:0/1497776182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7a80ffac0 0x7fa7a8197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:13.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.497+0000 7fa7b0772700 1 --2- 172.21.15.71:0/1497776182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a8105990 0x7fa7a8197970 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:13.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.497+0000 7fa7b0772700 1 -- 172.21.15.71:0/1497776182 >> 172.21.15.71:0/1497776182 conn(0x7fa7a8075400 msgr2=0x7fa7a80fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:13.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.497+0000 7fa7b0772700 1 -- 172.21.15.71:0/1497776182 shutdown_connections 2024-04-23T02:55:13.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.497+0000 7fa7b0772700 1 -- 172.21.15.71:0/1497776182 wait complete. 2024-04-23T02:55:13.501 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 42 2024-04-23T02:55:13.512 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 43 2024-04-23T02:55:13.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.631+0000 7f4af783b700 1 -- 172.21.15.71:0/218521178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4af00ff420 msgr2=0x7f4af00ff800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:13.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.631+0000 7f4af783b700 1 --2- 172.21.15.71:0/218521178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4af00ff420 0x7f4af00ff800 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009910 tx=0x7f4ae0009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:13.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.631+0000 7f4af783b700 1 -- 172.21.15.71:0/218521178 shutdown_connections 2024-04-23T02:55:13.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.631+0000 7f4af783b700 1 --2- 172.21.15.71:0/218521178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4af00ffdd0 0x7f4af0105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:13.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.631+0000 7f4af783b700 1 --2- 172.21.15.71:0/218521178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4af00ff420 0x7f4af00ff800 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:13.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.631+0000 7f4af783b700 1 -- 172.21.15.71:0/218521178 >> 172.21.15.71:0/218521178 conn(0x7f4af0074c20 msgr2=0x7f4af0075030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:13.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.631+0000 7f4af783b700 1 -- 172.21.15.71:0/218521178 shutdown_connections 2024-04-23T02:55:13.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.631+0000 7f4af783b700 1 -- 172.21.15.71:0/218521178 wait complete. 2024-04-23T02:55:13.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.632+0000 7f4af783b700 1 Processor -- start 2024-04-23T02:55:13.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.632+0000 7f4af783b700 1 -- start start 2024-04-23T02:55:13.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.632+0000 7f4af783b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4af00ff420 0x7f4af0196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:13.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.632+0000 7f4af783b700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4af0196a50 0x7f4af019aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:13.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.632+0000 7f4af783b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af0196fe0 con 0x7f4af00ff420 2024-04-23T02:55:13.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.632+0000 7f4af783b700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af0197150 con 0x7f4af0196a50 2024-04-23T02:55:13.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.632+0000 7f4af55d7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4af00ff420 0x7f4af0196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:13.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.632+0000 7f4af55d7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4af00ff420 0x7f4af0196510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60800/0 (socket says 172.21.15.71:60800) 2024-04-23T02:55:13.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.632+0000 7f4af55d7700 1 -- 172.21.15.71:0/711035273 learned_addr learned my addr 172.21.15.71:0/711035273 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:13.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.632+0000 7f4af4dd6700 1 --2- 172.21.15.71:0/711035273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4af0196a50 0x7f4af019aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:13.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.632+0000 7f4af55d7700 1 -- 172.21.15.71:0/711035273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4af0196a50 msgr2=0x7f4af019aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:13.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.632+0000 7f4af55d7700 1 --2- 172.21.15.71:0/711035273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4af0196a50 0x7f4af019aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:13.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.632+0000 7f4af55d7700 1 -- 172.21.15.71:0/711035273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae00095f0 con 0x7f4af00ff420 2024-04-23T02:55:13.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.632+0000 7f4af55d7700 1 --2- 172.21.15.71:0/711035273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4af00ff420 0x7f4af0196510 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0006010 tx=0x7f4ae000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:13.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.633+0000 7f4ae67fc700 1 -- 172.21.15.71:0/711035273 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ae001d070 con 0x7f4af00ff420 2024-04-23T02:55:13.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.633+0000 7f4ae67fc700 1 -- 172.21.15.71:0/711035273 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4ae000f460 con 0x7f4af00ff420 2024-04-23T02:55:13.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.633+0000 7f4af783b700 1 -- 172.21.15.71:0/711035273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af019b450 con 0x7f4af00ff420 2024-04-23T02:55:13.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.633+0000 7f4ae67fc700 1 -- 172.21.15.71:0/711035273 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ae0021640 con 0x7f4af00ff420 2024-04-23T02:55:13.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.633+0000 7f4af783b700 1 -- 172.21.15.71:0/711035273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af019b980 con 0x7f4af00ff420 2024-04-23T02:55:13.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.634+0000 7f4af783b700 1 -- 172.21.15.71:0/711035273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af01090a0 con 0x7f4af00ff420 2024-04-23T02:55:13.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.634+0000 7f4ae67fc700 1 -- 172.21.15.71:0/711035273 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4ae000fa20 con 0x7f4af00ff420 2024-04-23T02:55:13.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.635+0000 7f4ae67fc700 1 --2- 172.21.15.71:0/711035273 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4adc06cda0 0x7f4adc06f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:13.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.635+0000 7f4ae67fc700 1 -- 172.21.15.71:0/711035273 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4ae008d7b0 con 0x7f4af00ff420 2024-04-23T02:55:13.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.635+0000 7f4af4dd6700 1 --2- 172.21.15.71:0/711035273 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4adc06cda0 0x7f4adc06f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:13.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.635+0000 7f4af4dd6700 1 --2- 172.21.15.71:0/711035273 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4adc06cda0 0x7f4adc06f260 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4aec005ea0 tx=0x7f4aec005e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:13.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.639+0000 7f4ae67fc700 1 -- 172.21.15.71:0/711035273 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f4ae00578a0 con 0x7f4af00ff420 2024-04-23T02:55:13.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.879+0000 7f4af783b700 1 -- 172.21.15.71:0/711035273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 43, "format": "json"} v 0) v1 -- 0x7f4af0197860 con 0x7f4af00ff420 2024-04-23T02:55:13.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.880+0000 7f4ae67fc700 1 -- 172.21.15.71:0/711035273 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 43, "format": "json"}]=0 dumped fsmap epoch 43 v46) v1 ==== 107+0+4849 (secure 0 0 0) 0x7f4ae0057020 con 0x7f4af00ff420 2024-04-23T02:55:13.882 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:13.883 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44319,"name":"cephfs.smithi192.gtmyst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3688927981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3688927981},{"type":"v1","addr":"172.21.15.192:6825","nonce":3688927981}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":44793,"name":"cephfs.smithi192.azdyze","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/3768627645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3768627645},{"type":"v1","addr":"172.21.15.192:6827","nonce":3768627645}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:53:09.136729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34334,"mds_1":34362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34334":{"gid":34334,"name":"cephfs.smithi071.qusrrb","rank":0,"incarnation":38,"state":"up:rejoin","state_seq":468,"addr":"172.21.15.71:6827/2539819173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2539819173},{"type":"v1","addr":"172.21.15.71:6827","nonce":2539819173}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34362":{"gid":34362,"name":"cephfs.smithi071.vxejye","rank":1,"incarnation":38,"state":"up:rejoin","state_seq":462,"addr":"172.21.15.71:6829/143423621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":143423621},{"type":"v1","addr":"172.21.15.71:6829","nonce":143423621}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:13.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.882+0000 7f4af783b700 1 -- 172.21.15.71:0/711035273 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4adc06cda0 msgr2=0x7f4adc06f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:13.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.882+0000 7f4af783b700 1 --2- 172.21.15.71:0/711035273 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4adc06cda0 0x7f4adc06f260 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4aec005ea0 tx=0x7f4aec005e30 comp rx=0 tx=0).stop 2024-04-23T02:55:13.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.882+0000 7f4af783b700 1 -- 172.21.15.71:0/711035273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4af00ff420 msgr2=0x7f4af0196510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:13.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.882+0000 7f4af783b700 1 --2- 172.21.15.71:0/711035273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4af00ff420 0x7f4af0196510 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0006010 tx=0x7f4ae000be20 comp rx=0 tx=0).stop 2024-04-23T02:55:13.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.883+0000 7f4af783b700 1 -- 172.21.15.71:0/711035273 shutdown_connections 2024-04-23T02:55:13.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.883+0000 7f4af783b700 1 --2- 172.21.15.71:0/711035273 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f4adc06cda0 0x7f4adc06f260 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:13.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.883+0000 7f4af783b700 1 --2- 172.21.15.71:0/711035273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4af00ff420 0x7f4af0196510 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:13.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.883+0000 7f4af783b700 1 --2- 172.21.15.71:0/711035273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4af0196a50 0x7f4af019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:13.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.883+0000 7f4af783b700 1 -- 172.21.15.71:0/711035273 >> 172.21.15.71:0/711035273 conn(0x7f4af0074c20 msgr2=0x7f4af00feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:13.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.883+0000 7f4af783b700 1 -- 172.21.15.71:0/711035273 shutdown_connections 2024-04-23T02:55:13.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:13.883+0000 7f4af783b700 1 -- 172.21.15.71:0/711035273 wait complete. 2024-04-23T02:55:13.886 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 43 2024-04-23T02:55:13.896 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:13 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2104046798' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-04-23T02:55:13.896 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:13 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1553269857' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-04-23T02:55:13.896 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:13 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1497776182' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 42, "format": "json"}]: dispatch 2024-04-23T02:55:13.897 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 44 2024-04-23T02:55:14.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.016+0000 7f292ae37700 1 -- 172.21.15.71:0/3786053472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2924067da0 msgr2=0x7f2924068220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:14.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.016+0000 7f292ae37700 1 --2- 172.21.15.71:0/3786053472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2924067da0 0x7f2924068220 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f2910009910 tx=0x7f2910009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:14.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.016+0000 7f292ae37700 1 -- 172.21.15.71:0/3786053472 shutdown_connections 2024-04-23T02:55:14.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.016+0000 7f292ae37700 1 --2- 172.21.15.71:0/3786053472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2924067da0 0x7f2924068220 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:14.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.016+0000 7f292ae37700 1 --2- 172.21.15.71:0/3786053472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2924106a80 0x7f2924106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:14.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.016+0000 7f292ae37700 1 -- 172.21.15.71:0/3786053472 >> 172.21.15.71:0/3786053472 conn(0x7f2924075400 msgr2=0x7f2924075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:14.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.016+0000 7f292ae37700 1 -- 172.21.15.71:0/3786053472 shutdown_connections 2024-04-23T02:55:14.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.016+0000 7f292ae37700 1 -- 172.21.15.71:0/3786053472 wait complete. 2024-04-23T02:55:14.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.017+0000 7f292ae37700 1 Processor -- start 2024-04-23T02:55:14.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.017+0000 7f292ae37700 1 -- start start 2024-04-23T02:55:14.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.017+0000 7f292ae37700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2924067da0 0x7f2924196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:14.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.017+0000 7f292ae37700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2924106a80 0x7f2924196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:14.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.017+0000 7f292ae37700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2924197290 con 0x7f2924106a80 2024-04-23T02:55:14.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.017+0000 7f292ae37700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292419b010 con 0x7f2924067da0 2024-04-23T02:55:14.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.017+0000 7f2923fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2924106a80 0x7f2924196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:14.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.017+0000 7f2923fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2924106a80 0x7f2924196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60816/0 (socket says 172.21.15.71:60816) 2024-04-23T02:55:14.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.017+0000 7f2923fff700 1 -- 172.21.15.71:0/2725042842 learned_addr learned my addr 172.21.15.71:0/2725042842 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:14.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.017+0000 7f2928bd3700 1 --2- 172.21.15.71:0/2725042842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2924067da0 0x7f2924196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:14.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.017+0000 7f2923fff700 1 -- 172.21.15.71:0/2725042842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2924067da0 msgr2=0x7f2924196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:14.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.018+0000 7f2923fff700 1 --2- 172.21.15.71:0/2725042842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2924067da0 0x7f2924196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:14.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.018+0000 7f2923fff700 1 -- 172.21.15.71:0/2725042842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29100095f0 con 0x7f2924106a80 2024-04-23T02:55:14.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.018+0000 7f2923fff700 1 --2- 172.21.15.71:0/2725042842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2924106a80 0x7f2924196bb0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f2910006010 tx=0x7f291000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:14.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.018+0000 7f2921ffb700 1 -- 172.21.15.71:0/2725042842 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f291001d070 con 0x7f2924106a80 2024-04-23T02:55:14.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.018+0000 7f2921ffb700 1 -- 172.21.15.71:0/2725042842 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f291000f460 con 0x7f2924106a80 2024-04-23T02:55:14.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.018+0000 7f292ae37700 1 -- 172.21.15.71:0/2725042842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f292419b290 con 0x7f2924106a80 2024-04-23T02:55:14.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.018+0000 7f2921ffb700 1 -- 172.21.15.71:0/2725042842 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2910021640 con 0x7f2924106a80 2024-04-23T02:55:14.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.018+0000 7f292ae37700 1 -- 172.21.15.71:0/2725042842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292419b790 con 0x7f2924106a80 2024-04-23T02:55:14.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.019+0000 7f292ae37700 1 -- 172.21.15.71:0/2725042842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2924109090 con 0x7f2924106a80 2024-04-23T02:55:14.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.019+0000 7f2921ffb700 1 -- 172.21.15.71:0/2725042842 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f291000fa60 con 0x7f2924106a80 2024-04-23T02:55:14.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.020+0000 7f2921ffb700 1 --2- 172.21.15.71:0/2725042842 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f291406cda0 0x7f291406f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:14.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.020+0000 7f2921ffb700 1 -- 172.21.15.71:0/2725042842 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f291008d7e0 con 0x7f2924106a80 2024-04-23T02:55:14.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.020+0000 7f2928bd3700 1 --2- 172.21.15.71:0/2725042842 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f291406cda0 0x7f291406f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:14.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.020+0000 7f2928bd3700 1 --2- 172.21.15.71:0/2725042842 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f291406cda0 0x7f291406f260 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2918006fd0 tx=0x7f2918008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:14.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.024+0000 7f2921ffb700 1 -- 172.21.15.71:0/2725042842 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f29100578d0 con 0x7f2924106a80 2024-04-23T02:55:14.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:13 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2104046798' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-04-23T02:55:14.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:13 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1553269857' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-04-23T02:55:14.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:13 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1497776182' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 42, "format": "json"}]: dispatch 2024-04-23T02:55:14.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.262+0000 7f292ae37700 1 -- 172.21.15.71:0/2725042842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 44, "format": "json"} v 0) v1 -- 0x7f2924108a70 con 0x7f2924106a80 2024-04-23T02:55:14.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.263+0000 7f2921ffb700 1 -- 172.21.15.71:0/2725042842 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 44, "format": "json"}]=0 dumped fsmap epoch 44 v46) v1 ==== 107+0+4849 (secure 0 0 0) 0x7f291005aef0 con 0x7f2924106a80 2024-04-23T02:55:14.265 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:14.265 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44319,"name":"cephfs.smithi192.gtmyst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6825/3688927981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3688927981},{"type":"v1","addr":"172.21.15.192:6825","nonce":3688927981}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":44793,"name":"cephfs.smithi192.azdyze","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6827/3768627645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3768627645},{"type":"v1","addr":"172.21.15.192:6827","nonce":3768627645}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:53:10.141466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34334,"mds_1":34362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34334":{"gid":34334,"name":"cephfs.smithi071.qusrrb","rank":0,"incarnation":38,"state":"up:active","state_seq":469,"addr":"172.21.15.71:6827/2539819173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2539819173},{"type":"v1","addr":"172.21.15.71:6827","nonce":2539819173}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34362":{"gid":34362,"name":"cephfs.smithi071.vxejye","rank":1,"incarnation":38,"state":"up:active","state_seq":463,"addr":"172.21.15.71:6829/143423621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":143423621},{"type":"v1","addr":"172.21.15.71:6829","nonce":143423621}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:14.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.265+0000 7f292ae37700 1 -- 172.21.15.71:0/2725042842 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f291406cda0 msgr2=0x7f291406f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:14.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.265+0000 7f292ae37700 1 --2- 172.21.15.71:0/2725042842 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f291406cda0 0x7f291406f260 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2918006fd0 tx=0x7f2918008040 comp rx=0 tx=0).stop 2024-04-23T02:55:14.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.265+0000 7f292ae37700 1 -- 172.21.15.71:0/2725042842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2924106a80 msgr2=0x7f2924196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:14.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.265+0000 7f292ae37700 1 --2- 172.21.15.71:0/2725042842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2924106a80 0x7f2924196bb0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f2910006010 tx=0x7f291000be20 comp rx=0 tx=0).stop 2024-04-23T02:55:14.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.265+0000 7f292ae37700 1 -- 172.21.15.71:0/2725042842 shutdown_connections 2024-04-23T02:55:14.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.265+0000 7f292ae37700 1 --2- 172.21.15.71:0/2725042842 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f291406cda0 0x7f291406f260 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:14.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.265+0000 7f292ae37700 1 --2- 172.21.15.71:0/2725042842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2924067da0 0x7f2924196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:14.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.265+0000 7f292ae37700 1 --2- 172.21.15.71:0/2725042842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2924106a80 0x7f2924196bb0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:14.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.265+0000 7f292ae37700 1 -- 172.21.15.71:0/2725042842 >> 172.21.15.71:0/2725042842 conn(0x7f2924075400 msgr2=0x7f29240fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:14.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.265+0000 7f292ae37700 1 -- 172.21.15.71:0/2725042842 shutdown_connections 2024-04-23T02:55:14.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.265+0000 7f292ae37700 1 -- 172.21.15.71:0/2725042842 wait complete. 2024-04-23T02:55:14.268 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 44 2024-04-23T02:55:14.279 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 45 2024-04-23T02:55:14.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.399+0000 7f0e684f1700 1 -- 172.21.15.71:0/1473898553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e60105990 msgr2=0x7f0e60105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:14.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.399+0000 7f0e684f1700 1 --2- 172.21.15.71:0/1473898553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e60105990 0x7f0e60105d70 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009910 tx=0x7f0e50009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:14.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.399+0000 7f0e684f1700 1 -- 172.21.15.71:0/1473898553 shutdown_connections 2024-04-23T02:55:14.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.399+0000 7f0e684f1700 1 --2- 172.21.15.71:0/1473898553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e600ffac0 0x7f0e600fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:14.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.399+0000 7f0e684f1700 1 --2- 172.21.15.71:0/1473898553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e60105990 0x7f0e60105d70 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:14.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.399+0000 7f0e684f1700 1 -- 172.21.15.71:0/1473898553 >> 172.21.15.71:0/1473898553 conn(0x7f0e60075400 msgr2=0x7f0e60075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:14.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.399+0000 7f0e684f1700 1 -- 172.21.15.71:0/1473898553 shutdown_connections 2024-04-23T02:55:14.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.399+0000 7f0e684f1700 1 -- 172.21.15.71:0/1473898553 wait complete. 2024-04-23T02:55:14.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e684f1700 1 Processor -- start 2024-04-23T02:55:14.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e684f1700 1 -- start start 2024-04-23T02:55:14.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e684f1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e600ffac0 0x7f0e60101550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:14.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e684f1700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e60105990 0x7f0e60101a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:14.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e684f1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e60101fd0 con 0x7f0e600ffac0 2024-04-23T02:55:14.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e684f1700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e60102110 con 0x7f0e60105990 2024-04-23T02:55:14.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e6628d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e600ffac0 0x7f0e60101550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:14.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e6628d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e600ffac0 0x7f0e60101550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60832/0 (socket says 172.21.15.71:60832) 2024-04-23T02:55:14.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e6628d700 1 -- 172.21.15.71:0/1001382452 learned_addr learned my addr 172.21.15.71:0/1001382452 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:14.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e65a8c700 1 --2- 172.21.15.71:0/1001382452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e60105990 0x7f0e60101a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:14.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e6628d700 1 -- 172.21.15.71:0/1001382452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e60105990 msgr2=0x7f0e60101a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:14.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e6628d700 1 --2- 172.21.15.71:0/1001382452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e60105990 0x7f0e60101a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:14.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e6628d700 1 -- 172.21.15.71:0/1001382452 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e500095f0 con 0x7f0e600ffac0 2024-04-23T02:55:14.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e65a8c700 1 --2- 172.21.15.71:0/1001382452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e60105990 0x7f0e60101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:55:14.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e6628d700 1 --2- 172.21.15.71:0/1001382452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e600ffac0 0x7f0e60101550 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f0e50006010 tx=0x7f0e5000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:14.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e577fe700 1 -- 172.21.15.71:0/1001382452 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e5001d070 con 0x7f0e600ffac0 2024-04-23T02:55:14.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e577fe700 1 -- 172.21.15.71:0/1001382452 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0e5000f460 con 0x7f0e600ffac0 2024-04-23T02:55:14.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e684f1700 1 -- 172.21.15.71:0/1001382452 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e60102390 con 0x7f0e600ffac0 2024-04-23T02:55:14.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.400+0000 7f0e577fe700 1 -- 172.21.15.71:0/1001382452 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e50021620 con 0x7f0e600ffac0 2024-04-23T02:55:14.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.401+0000 7f0e684f1700 1 -- 172.21.15.71:0/1001382452 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e601a2980 con 0x7f0e600ffac0 2024-04-23T02:55:14.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.402+0000 7f0e684f1700 1 -- 172.21.15.71:0/1001382452 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6004ea90 con 0x7f0e600ffac0 2024-04-23T02:55:14.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.402+0000 7f0e577fe700 1 -- 172.21.15.71:0/1001382452 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0e5000fa40 con 0x7f0e600ffac0 2024-04-23T02:55:14.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.402+0000 7f0e577fe700 1 --2- 172.21.15.71:0/1001382452 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0e4c06cda0 0x7f0e4c06f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:14.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.402+0000 7f0e577fe700 1 -- 172.21.15.71:0/1001382452 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0e5008d6c0 con 0x7f0e600ffac0 2024-04-23T02:55:14.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.402+0000 7f0e65a8c700 1 --2- 172.21.15.71:0/1001382452 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0e4c06cda0 0x7f0e4c06f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:14.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.403+0000 7f0e65a8c700 1 --2- 172.21.15.71:0/1001382452 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0e4c06cda0 0x7f0e4c06f260 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c005950 tx=0x7f0e5c0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:14.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.407+0000 7f0e577fe700 1 -- 172.21.15.71:0/1001382452 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0e500577b0 con 0x7f0e600ffac0 2024-04-23T02:55:14.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.643+0000 7f0e684f1700 1 -- 172.21.15.71:0/1001382452 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 45, "format": "json"} v 0) v1 -- 0x7f0e601a2e10 con 0x7f0e600ffac0 2024-04-23T02:55:14.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.643+0000 7f0e577fe700 1 -- 172.21.15.71:0/1001382452 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 45, "format": "json"}]=0 dumped fsmap epoch 45 v46) v1 ==== 107+0+4864 (secure 0 0 0) 0x7f0e5005add0 con 0x7f0e600ffac0 2024-04-23T02:55:14.646 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:14.646 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T02:10:51.346345+0000","modified":"2024-04-23T02:53:10.147044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34334,"mds_1":34362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34334":{"gid":34334,"name":"cephfs.smithi071.qusrrb","rank":0,"incarnation":38,"state":"up:active","state_seq":469,"addr":"172.21.15.71:6827/2539819173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2539819173},{"type":"v1","addr":"172.21.15.71:6827","nonce":2539819173}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34362":{"gid":34362,"name":"cephfs.smithi071.vxejye","rank":1,"incarnation":38,"state":"up:active","state_seq":463,"addr":"172.21.15.71:6829/143423621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":143423621},{"type":"v1","addr":"172.21.15.71:6829","nonce":143423621}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_44319":{"gid":44319,"name":"cephfs.smithi192.gtmyst","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6825/3688927981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":3688927981},{"type":"v1","addr":"172.21.15.192:6825","nonce":3688927981}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_44793":{"gid":44793,"name":"cephfs.smithi192.azdyze","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6827/3768627645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":3768627645},{"type":"v1","addr":"172.21.15.192:6827","nonce":3768627645}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-23T02:55:14.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.645+0000 7f0e684f1700 1 -- 172.21.15.71:0/1001382452 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0e4c06cda0 msgr2=0x7f0e4c06f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:14.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.645+0000 7f0e684f1700 1 --2- 172.21.15.71:0/1001382452 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0e4c06cda0 0x7f0e4c06f260 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c005950 tx=0x7f0e5c0058e0 comp rx=0 tx=0).stop 2024-04-23T02:55:14.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.645+0000 7f0e684f1700 1 -- 172.21.15.71:0/1001382452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e600ffac0 msgr2=0x7f0e60101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:14.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.645+0000 7f0e684f1700 1 --2- 172.21.15.71:0/1001382452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e600ffac0 0x7f0e60101550 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f0e50006010 tx=0x7f0e5000be20 comp rx=0 tx=0).stop 2024-04-23T02:55:14.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.645+0000 7f0e684f1700 1 -- 172.21.15.71:0/1001382452 shutdown_connections 2024-04-23T02:55:14.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.645+0000 7f0e684f1700 1 --2- 172.21.15.71:0/1001382452 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f0e4c06cda0 0x7f0e4c06f260 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:14.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.645+0000 7f0e684f1700 1 --2- 172.21.15.71:0/1001382452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e600ffac0 0x7f0e60101550 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:14.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.645+0000 7f0e684f1700 1 --2- 172.21.15.71:0/1001382452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e60105990 0x7f0e60101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:14.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.645+0000 7f0e684f1700 1 -- 172.21.15.71:0/1001382452 >> 172.21.15.71:0/1001382452 conn(0x7f0e60075400 msgr2=0x7f0e60067dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:14.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.645+0000 7f0e684f1700 1 -- 172.21.15.71:0/1001382452 shutdown_connections 2024-04-23T02:55:14.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.645+0000 7f0e684f1700 1 -- 172.21.15.71:0/1001382452 wait complete. 2024-04-23T02:55:14.649 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 45 2024-04-23T02:55:14.660 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-23T02:55:14.672 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-23T02:55:14.672 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T02:55:14.672 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T02:55:14.692 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T02:55:14.692 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T02:55:14.752 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T02:55:14.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.911+0000 7fc9a7cac700 1 -- 172.21.15.71:0/3631257966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9a00ffdd0 msgr2=0x7fc9a0105750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:14.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.911+0000 7fc9a7cac700 1 --2- 172.21.15.71:0/3631257966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9a00ffdd0 0x7fc9a0105750 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009910 tx=0x7fc99c009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:14.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.911+0000 7fc9a7cac700 1 -- 172.21.15.71:0/3631257966 shutdown_connections 2024-04-23T02:55:14.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.911+0000 7fc9a7cac700 1 --2- 172.21.15.71:0/3631257966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9a00ffdd0 0x7fc9a0105750 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:14.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.911+0000 7fc9a7cac700 1 --2- 172.21.15.71:0/3631257966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9a00ff420 0x7fc9a00ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:14.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.911+0000 7fc9a7cac700 1 -- 172.21.15.71:0/3631257966 >> 172.21.15.71:0/3631257966 conn(0x7fc9a0074c20 msgr2=0x7fc9a0075030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:14.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.911+0000 7fc9a7cac700 1 -- 172.21.15.71:0/3631257966 shutdown_connections 2024-04-23T02:55:14.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.911+0000 7fc9a7cac700 1 -- 172.21.15.71:0/3631257966 wait complete. 2024-04-23T02:55:14.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.911+0000 7fc9a7cac700 1 Processor -- start 2024-04-23T02:55:14.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.911+0000 7fc9a7cac700 1 -- start start 2024-04-23T02:55:14.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.912+0000 7fc9a7cac700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9a00ff420 0x7fc9a0196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:14.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.912+0000 7fc9a7cac700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9a0196b90 0x7fc9a019aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:14.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.912+0000 7fc9a7cac700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a01971b0 con 0x7fc9a0196b90 2024-04-23T02:55:14.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.912+0000 7fc9a7cac700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a0197320 con 0x7fc9a00ff420 2024-04-23T02:55:14.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.912+0000 7fc9a5247700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9a0196b90 0x7fc9a019aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:14.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.912+0000 7fc9a5247700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9a0196b90 0x7fc9a019aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60856/0 (socket says 172.21.15.71:60856) 2024-04-23T02:55:14.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.912+0000 7fc9a5247700 1 -- 172.21.15.71:0/704092107 learned_addr learned my addr 172.21.15.71:0/704092107 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:14.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.912+0000 7fc9a5a48700 1 --2- 172.21.15.71:0/704092107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9a00ff420 0x7fc9a0196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.912+0000 7fc9a5247700 1 -- 172.21.15.71:0/704092107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9a00ff420 msgr2=0x7fc9a0196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.912+0000 7fc9a5247700 1 --2- 172.21.15.71:0/704092107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9a00ff420 0x7fc9a0196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.912+0000 7fc9a5247700 1 -- 172.21.15.71:0/704092107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c0095f0 con 0x7fc9a0196b90 2024-04-23T02:55:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.912+0000 7fc9a5247700 1 --2- 172.21.15.71:0/704092107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9a0196b90 0x7fc9a019aff0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00bd70 tx=0x7fc99c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.912+0000 7fc996ffd700 1 -- 172.21.15.71:0/704092107 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc99c01d070 con 0x7fc9a0196b90 2024-04-23T02:55:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.912+0000 7fc996ffd700 1 -- 172.21.15.71:0/704092107 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc99c00f460 con 0x7fc9a0196b90 2024-04-23T02:55:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.912+0000 7fc9a7cac700 1 -- 172.21.15.71:0/704092107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a019b5f0 con 0x7fc9a0196b90 2024-04-23T02:55:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.912+0000 7fc996ffd700 1 -- 172.21.15.71:0/704092107 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc99c021640 con 0x7fc9a0196b90 2024-04-23T02:55:14.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.912+0000 7fc9a7cac700 1 -- 172.21.15.71:0/704092107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a019baf0 con 0x7fc9a0196b90 2024-04-23T02:55:14.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.913+0000 7fc9a7cac700 1 -- 172.21.15.71:0/704092107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a0109080 con 0x7fc9a0196b90 2024-04-23T02:55:14.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.914+0000 7fc996ffd700 1 -- 172.21.15.71:0/704092107 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc99c00fa20 con 0x7fc9a0196b90 2024-04-23T02:55:14.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.914+0000 7fc996ffd700 1 --2- 172.21.15.71:0/704092107 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc98c06cda0 0x7fc98c06f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:14.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.914+0000 7fc9a5a48700 1 --2- 172.21.15.71:0/704092107 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc98c06cda0 0x7fc98c06f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:14.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.914+0000 7fc996ffd700 1 -- 172.21.15.71:0/704092107 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc99c08d6b0 con 0x7fc9a0196b90 2024-04-23T02:55:14.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.915+0000 7fc9a5a48700 1 --2- 172.21.15.71:0/704092107 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc98c06cda0 0x7fc98c06f260 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc990004630 tx=0x7fc99000b3c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:14.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:14.919+0000 7fc996ffd700 1 -- 172.21.15.71:0/704092107 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc99c057720 con 0x7fc9a0196b90 2024-04-23T02:55:14.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:14 smithi071 ceph-mon[144544]: pgmap v1260: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:14.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:14 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/711035273' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 43, "format": "json"}]: dispatch 2024-04-23T02:55:14.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:14 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2725042842' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 44, "format": "json"}]: dispatch 2024-04-23T02:55:14.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:14 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1001382452' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 45, "format": "json"}]: dispatch 2024-04-23T02:55:15.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.115+0000 7fc9a7cac700 1 -- 172.21.15.71:0/704092107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9a004ea90 con 0x7fc9a0196b90 2024-04-23T02:55:15.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.116+0000 7fc996ffd700 1 -- 172.21.15.71:0/704092107 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16305 (secure 0 0 0) 0x7fc99c00f5d0 con 0x7fc9a0196b90 2024-04-23T02:55:15.119 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:15.119 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":84,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","created":"2024-04-23T02:03:14.872097+0000","modified":"2024-04-23T02:37:53.913348+0000","last_up_change":"2024-04-23T02:21:57.350899+0000","last_in_change":"2024-04-23T02:09:09.560591+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T02:08:06.416728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-23T02:10:50.283526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-23T02:10:50.576356+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5f48bf1-6e98-4432-885a-f6ed0c42e5d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2865488005},{"type":"v1","addr":"172.21.15.71:6803","nonce":2865488005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2865488005},{"type":"v1","addr":"172.21.15.71:6805","nonce":2865488005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2865488005},{"type":"v1","addr":"172.21.15.71:6809","nonce":2865488005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2865488005},{"type":"v1","addr":"172.21.15.71:6807","nonce":2865488005}]},"public_addr":"172.21.15.71:6803/2865488005","cluster_addr":"172.21.15.71:6805/2865488005","heartbeat_back_addr":"172.21.15.71:6809/2865488005","heartbeat_front_addr":"172.21.15.71:6807/2865488005","state":["exists","up"]},{"osd":1,"uuid":"04304fd2-42f9-4525-88e8-c7e20c9c51e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2293979729},{"type":"v1","addr":"172.21.15.71:6811","nonce":2293979729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2293979729},{"type":"v1","addr":"172.21.15.71:6813","nonce":2293979729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2293979729},{"type":"v1","addr":"172.21.15.71:6817","nonce":2293979729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2293979729},{"type":"v1","addr":"172.21.15.71:6815","nonce":2293979729}]},"public_addr":"172.21.15.71:6811/2293979729","cluster_addr":"172.21.15.71:6813/2293979729","heartbeat_back_addr":"172.21.15.71:6817/2293979729","heartbeat_front_addr":"172.21.15.71:6815/2293979729","state":["exists","up"]},{"osd":2,"uuid":"6797c77c-4ccc-4489-9269-77105d7c45c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3388560056},{"type":"v1","addr":"172.21.15.71:6819","nonce":3388560056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3388560056},{"type":"v1","addr":"172.21.15.71:6821","nonce":3388560056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3388560056},{"type":"v1","addr":"172.21.15.71:6825","nonce":3388560056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3388560056},{"type":"v1","addr":"172.21.15.71:6823","nonce":3388560056}]},"public_addr":"172.21.15.71:6819/3388560056","cluster_addr":"172.21.15.71:6821/3388560056","heartbeat_back_addr":"172.21.15.71:6825/3388560056","heartbeat_front_addr":"172.21.15.71:6823/3388560056","state":["exists","up"]},{"osd":3,"uuid":"62033906-4d1c-431d-a87d-a12e0a134383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4106623627},{"type":"v1","addr":"172.21.15.192:6801","nonce":4106623627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4106623627},{"type":"v1","addr":"172.21.15.192:6803","nonce":4106623627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4106623627},{"type":"v1","addr":"172.21.15.192:6807","nonce":4106623627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4106623627},{"type":"v1","addr":"172.21.15.192:6805","nonce":4106623627}]},"public_addr":"172.21.15.192:6801/4106623627","cluster_addr":"172.21.15.192:6803/4106623627","heartbeat_back_addr":"172.21.15.192:6807/4106623627","heartbeat_front_addr":"172.21.15.192:6805/4106623627","state":["exists","up"]},{"osd":4,"uuid":"6a7317f5-dc05-415a-8409-81f80c196a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1707328698},{"type":"v1","addr":"172.21.15.192:6809","nonce":1707328698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1707328698},{"type":"v1","addr":"172.21.15.192:6811","nonce":1707328698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1707328698},{"type":"v1","addr":"172.21.15.192:6815","nonce":1707328698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1707328698},{"type":"v1","addr":"172.21.15.192:6813","nonce":1707328698}]},"public_addr":"172.21.15.192:6809/1707328698","cluster_addr":"172.21.15.192:6811/1707328698","heartbeat_back_addr":"172.21.15.192:6815/1707328698","heartbeat_front_addr":"172.21.15.192:6813/1707328698","state":["exists","up"]},{"osd":5,"uuid":"68b550d5-d74e-417e-9bcc-98740efaf0ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":868066904},{"type":"v1","addr":"172.21.15.192:6817","nonce":868066904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":868066904},{"type":"v1","addr":"172.21.15.192:6819","nonce":868066904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":868066904},{"type":"v1","addr":"172.21.15.192:6823","nonce":868066904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":868066904},{"type":"v1","addr":"172.21.15.192:6821","nonce":868066904}]},"public_addr":"172.21.15.192:6817/868066904","cluster_addr":"172.21.15.192:6819/868066904","heartbeat_back_addr":"172.21.15.192:6823/868066904","heartbeat_front_addr":"172.21.15.192:6821/868066904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:07:01.863954+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:07:32.930839+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:04.093747+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:31.574485+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:59.698428+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:09:28.076223+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6828/467545394":"2024-04-24T02:22:00.280478+0000","172.21.15.71:6801/3794652670":"2024-04-24T02:03:48.178880+0000","172.21.15.71:6801/4165305842":"2024-04-24T02:05:42.095805+0000","172.21.15.71:0/56041968":"2024-04-24T02:05:42.095805+0000","172.21.15.192:0/1753179829":"2024-04-24T02:14:34.629978+0000","172.21.15.71:6800/4165305842":"2024-04-24T02:05:42.095805+0000","172.21.15.192:0/2569976755":"2024-04-24T02:14:34.629978+0000","172.21.15.71:0/823266478":"2024-04-24T02:04:23.247815+0000","172.21.15.192:6828/966231696":"2024-04-24T02:14:34.629978+0000","172.21.15.192:6824/3780549896":"2024-04-24T02:22:00.280491+0000","172.21.15.192:0/1701960325":"2024-04-24T02:14:34.629978+0000","172.21.15.71:0/2225989508":"2024-04-24T02:03:48.178880+0000","172.21.15.71:6829/467545394":"2024-04-24T02:22:00.280478+0000","172.21.15.192:0/2687070241":"2024-04-24T02:14:34.629978+0000","172.21.15.71:0/17588705":"2024-04-24T02:13:11.575106+0000","172.21.15.71:0/3579793925":"2024-04-24T02:04:23.247815+0000","172.21.15.71:0/1995129374":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/294030681":"2024-04-24T02:04:23.247815+0000","172.21.15.71:0/936344738":"2024-04-24T02:05:42.095805+0000","172.21.15.192:6829/966231696":"2024-04-24T02:14:34.629978+0000","172.21.15.192:0/2727292170":"2024-04-24T02:14:34.629978+0000","172.21.15.71:0/1815873101":"2024-04-24T02:13:11.575106+0000","172.21.15.192:6825/3780549896":"2024-04-24T02:22:00.280491+0000","172.21.15.71:6801/3878148961":"2024-04-24T02:04:23.247815+0000","172.21.15.71:6827/3504670324":"2024-04-24T02:11:05.663451+0000","172.21.15.192:0/2877439214":"2024-04-24T02:14:34.629978+0000","172.21.15.71:6800/4249907286":"2024-04-24T02:13:11.575106+0000","172.21.15.71:6800/3878148961":"2024-04-24T02:04:23.247815+0000","172.21.15.71:6800/3794652670":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/4205332842":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/3618236204":"2024-04-24T02:13:11.575106+0000","172.21.15.71:0/3473332587":"2024-04-24T02:13:11.575106+0000","172.21.15.71:0/2206241071":"2024-04-24T02:05:42.095805+0000","172.21.15.71:6801/4249907286":"2024-04-24T02:13:11.575106+0000","172.21.15.71:6826/3504670324":"2024-04-24T02:11:05.663451+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T02:55:15.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.119+0000 7fc9a7cac700 1 -- 172.21.15.71:0/704092107 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc98c06cda0 msgr2=0x7fc98c06f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:15.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.119+0000 7fc9a7cac700 1 --2- 172.21.15.71:0/704092107 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc98c06cda0 0x7fc98c06f260 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc990004630 tx=0x7fc99000b3c0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.119+0000 7fc9a7cac700 1 -- 172.21.15.71:0/704092107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9a0196b90 msgr2=0x7fc9a019aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:15.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.119+0000 7fc9a7cac700 1 --2- 172.21.15.71:0/704092107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9a0196b90 0x7fc9a019aff0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00bd70 tx=0x7fc99c00bda0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.119+0000 7fc9a7cac700 1 -- 172.21.15.71:0/704092107 shutdown_connections 2024-04-23T02:55:15.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.119+0000 7fc9a7cac700 1 --2- 172.21.15.71:0/704092107 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fc98c06cda0 0x7fc98c06f260 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.119+0000 7fc9a7cac700 1 --2- 172.21.15.71:0/704092107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9a00ff420 0x7fc9a0196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.119+0000 7fc9a7cac700 1 --2- 172.21.15.71:0/704092107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9a0196b90 0x7fc9a019aff0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.119+0000 7fc9a7cac700 1 -- 172.21.15.71:0/704092107 >> 172.21.15.71:0/704092107 conn(0x7fc9a0074c20 msgr2=0x7fc9a00fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:15.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.119+0000 7fc9a7cac700 1 -- 172.21.15.71:0/704092107 shutdown_connections 2024-04-23T02:55:15.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.119+0000 7fc9a7cac700 1 -- 172.21.15.71:0/704092107 wait complete. 2024-04-23T02:55:15.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:14 smithi192 ceph-mon[108617]: pgmap v1260: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:15.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:14 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/711035273' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 43, "format": "json"}]: dispatch 2024-04-23T02:55:15.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:14 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2725042842' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 44, "format": "json"}]: dispatch 2024-04-23T02:55:15.126 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:14 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1001382452' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 45, "format": "json"}]: dispatch 2024-04-23T02:55:15.134 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T02:08:06.416728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-23T02:10:50.283526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-23T02:10:50.576356+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T02:55:15.134 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T02:55:15.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.256+0000 7f66cfb20700 1 -- 172.21.15.71:0/3530588030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66c80ff440 msgr2=0x7f66c80ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.256+0000 7f66cfb20700 1 --2- 172.21.15.71:0/3530588030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66c80ff440 0x7f66c80ff820 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009910 tx=0x7f66c4009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.257+0000 7f66cfb20700 1 -- 172.21.15.71:0/3530588030 shutdown_connections 2024-04-23T02:55:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.257+0000 7f66cfb20700 1 --2- 172.21.15.71:0/3530588030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66c80ffdf0 0x7f66c8105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.257+0000 7f66cfb20700 1 --2- 172.21.15.71:0/3530588030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66c80ff440 0x7f66c80ff820 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.257+0000 7f66cfb20700 1 -- 172.21.15.71:0/3530588030 >> 172.21.15.71:0/3530588030 conn(0x7f66c8074c10 msgr2=0x7f66c8075020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.257+0000 7f66cfb20700 1 -- 172.21.15.71:0/3530588030 shutdown_connections 2024-04-23T02:55:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.257+0000 7f66cfb20700 1 -- 172.21.15.71:0/3530588030 wait complete. 2024-04-23T02:55:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.257+0000 7f66cfb20700 1 Processor -- start 2024-04-23T02:55:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.257+0000 7f66cfb20700 1 -- start start 2024-04-23T02:55:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.257+0000 7f66cfb20700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66c80ff440 0x7f66c8196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.257+0000 7f66cfb20700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66c8196bc0 0x7f66c819b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.257+0000 7f66cfb20700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c81971e0 con 0x7f66c80ff440 2024-04-23T02:55:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.257+0000 7f66cfb20700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c8197350 con 0x7f66c8196bc0 2024-04-23T02:55:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.257+0000 7f66cd8bc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66c80ff440 0x7f66c8196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:15.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.257+0000 7f66cd8bc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66c80ff440 0x7f66c8196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60870/0 (socket says 172.21.15.71:60870) 2024-04-23T02:55:15.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.257+0000 7f66cd8bc700 1 -- 172.21.15.71:0/3509888277 learned_addr learned my addr 172.21.15.71:0/3509888277 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:15.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.258+0000 7f66cd0bb700 1 --2- 172.21.15.71:0/3509888277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66c8196bc0 0x7f66c819b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:15.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.258+0000 7f66cd8bc700 1 -- 172.21.15.71:0/3509888277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66c8196bc0 msgr2=0x7f66c819b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:15.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.258+0000 7f66cd8bc700 1 --2- 172.21.15.71:0/3509888277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66c8196bc0 0x7f66c819b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.258+0000 7f66cd8bc700 1 -- 172.21.15.71:0/3509888277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c40095f0 con 0x7f66c80ff440 2024-04-23T02:55:15.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.258+0000 7f66cd0bb700 1 --2- 172.21.15.71:0/3509888277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66c8196bc0 0x7f66c819b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:55:15.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.258+0000 7f66cd8bc700 1 --2- 172.21.15.71:0/3509888277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66c80ff440 0x7f66c8196680 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f66c4006010 tx=0x7f66c400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:15.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.258+0000 7f66beffd700 1 -- 172.21.15.71:0/3509888277 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66c401d070 con 0x7f66c80ff440 2024-04-23T02:55:15.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.258+0000 7f66beffd700 1 -- 172.21.15.71:0/3509888277 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f66c400f460 con 0x7f66c80ff440 2024-04-23T02:55:15.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.258+0000 7f66cfb20700 1 -- 172.21.15.71:0/3509888277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c819b620 con 0x7f66c80ff440 2024-04-23T02:55:15.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.258+0000 7f66beffd700 1 -- 172.21.15.71:0/3509888277 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66c4021640 con 0x7f66c80ff440 2024-04-23T02:55:15.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.258+0000 7f66cfb20700 1 -- 172.21.15.71:0/3509888277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c819bb20 con 0x7f66c80ff440 2024-04-23T02:55:15.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.259+0000 7f66cfb20700 1 -- 172.21.15.71:0/3509888277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c8109040 con 0x7f66c80ff440 2024-04-23T02:55:15.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.259+0000 7f66beffd700 1 -- 172.21.15.71:0/3509888277 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f66c400f9e0 con 0x7f66c80ff440 2024-04-23T02:55:15.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.260+0000 7f66beffd700 1 --2- 172.21.15.71:0/3509888277 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f66b406cce0 0x7f66b406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:15.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.260+0000 7f66cd0bb700 1 --2- 172.21.15.71:0/3509888277 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f66b406cce0 0x7f66b406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:15.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.260+0000 7f66beffd700 1 -- 172.21.15.71:0/3509888277 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f66c408d830 con 0x7f66c80ff440 2024-04-23T02:55:15.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.260+0000 7f66cd0bb700 1 --2- 172.21.15.71:0/3509888277 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f66b406cce0 0x7f66b406f1a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f66b8005950 tx=0x7f66b8005f50 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:15.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.265+0000 7f66beffd700 1 -- 172.21.15.71:0/3509888277 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f66c40578a0 con 0x7f66c80ff440 2024-04-23T02:55:15.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.455+0000 7f66cfb20700 1 -- 172.21.15.71:0/3509888277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66c819be00 con 0x7f66c80ff440 2024-04-23T02:55:15.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.455+0000 7f66beffd700 1 -- 172.21.15.71:0/3509888277 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f66c4057020 con 0x7f66c80ff440 2024-04-23T02:55:15.458 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-23T02:55:15.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.457+0000 7f66cfb20700 1 -- 172.21.15.71:0/3509888277 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f66b406cce0 msgr2=0x7f66b406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:15.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.457+0000 7f66cfb20700 1 --2- 172.21.15.71:0/3509888277 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f66b406cce0 0x7f66b406f1a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f66b8005950 tx=0x7f66b8005f50 comp rx=0 tx=0).stop 2024-04-23T02:55:15.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.457+0000 7f66cfb20700 1 -- 172.21.15.71:0/3509888277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66c80ff440 msgr2=0x7f66c8196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:15.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.457+0000 7f66cfb20700 1 --2- 172.21.15.71:0/3509888277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66c80ff440 0x7f66c8196680 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f66c4006010 tx=0x7f66c400be20 comp rx=0 tx=0).stop 2024-04-23T02:55:15.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.457+0000 7f66cfb20700 1 -- 172.21.15.71:0/3509888277 shutdown_connections 2024-04-23T02:55:15.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.457+0000 7f66cfb20700 1 --2- 172.21.15.71:0/3509888277 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f66b406cce0 0x7f66b406f1a0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.457+0000 7f66cfb20700 1 --2- 172.21.15.71:0/3509888277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66c80ff440 0x7f66c8196680 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.457+0000 7f66cfb20700 1 --2- 172.21.15.71:0/3509888277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66c8196bc0 0x7f66c819b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.457+0000 7f66cfb20700 1 -- 172.21.15.71:0/3509888277 >> 172.21.15.71:0/3509888277 conn(0x7f66c8074c10 msgr2=0x7f66c80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:15.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.457+0000 7f66cfb20700 1 -- 172.21.15.71:0/3509888277 shutdown_connections 2024-04-23T02:55:15.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.457+0000 7f66cfb20700 1 -- 172.21.15.71:0/3509888277 wait complete. 2024-04-23T02:55:15.472 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-23T02:55:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.594+0000 7f65ffef6700 1 -- 172.21.15.71:0/3240618545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65f80ffdf0 msgr2=0x7f65f8105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.595+0000 7f65ffef6700 1 --2- 172.21.15.71:0/3240618545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65f80ffdf0 0x7f65f8105710 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009910 tx=0x7f65e8009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.595+0000 7f65ffef6700 1 -- 172.21.15.71:0/3240618545 shutdown_connections 2024-04-23T02:55:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.595+0000 7f65ffef6700 1 --2- 172.21.15.71:0/3240618545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65f80ffdf0 0x7f65f8105710 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.595+0000 7f65ffef6700 1 --2- 172.21.15.71:0/3240618545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65f80ff440 0x7f65f80ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.595+0000 7f65ffef6700 1 -- 172.21.15.71:0/3240618545 >> 172.21.15.71:0/3240618545 conn(0x7f65f8074c10 msgr2=0x7f65f8075020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.595+0000 7f65ffef6700 1 -- 172.21.15.71:0/3240618545 shutdown_connections 2024-04-23T02:55:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.595+0000 7f65ffef6700 1 -- 172.21.15.71:0/3240618545 wait complete. 2024-04-23T02:55:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.595+0000 7f65ffef6700 1 Processor -- start 2024-04-23T02:55:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.595+0000 7f65ffef6700 1 -- start start 2024-04-23T02:55:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.595+0000 7f65ffef6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65f80ff440 0x7f65f819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.595+0000 7f65ffef6700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65f819eb30 0x7f65f8198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.595+0000 7f65ffef6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f819f100 con 0x7f65f80ff440 2024-04-23T02:55:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.595+0000 7f65ffef6700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f8198ba0 con 0x7f65f819eb30 2024-04-23T02:55:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.596+0000 7f65fdc92700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65f80ff440 0x7f65f819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.596+0000 7f65fdc92700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65f80ff440 0x7f65f819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60526/0 (socket says 172.21.15.71:60526) 2024-04-23T02:55:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.596+0000 7f65fdc92700 1 -- 172.21.15.71:0/1061428667 learned_addr learned my addr 172.21.15.71:0/1061428667 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.596+0000 7f65fdc92700 1 -- 172.21.15.71:0/1061428667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65f819eb30 msgr2=0x7f65f8198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.596+0000 7f65fdc92700 1 --2- 172.21.15.71:0/1061428667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65f819eb30 0x7f65f8198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.596+0000 7f65fdc92700 1 -- 172.21.15.71:0/1061428667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e80095f0 con 0x7f65f80ff440 2024-04-23T02:55:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.596+0000 7f65fdc92700 1 --2- 172.21.15.71:0/1061428667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65f80ff440 0x7f65f819e5f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f65f400c8a0 tx=0x7f65f400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.596+0000 7f65eeffd700 1 -- 172.21.15.71:0/1061428667 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65f400cea0 con 0x7f65f80ff440 2024-04-23T02:55:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.596+0000 7f65ffef6700 1 -- 172.21.15.71:0/1061428667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f8198e80 con 0x7f65f80ff440 2024-04-23T02:55:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.596+0000 7f65eeffd700 1 -- 172.21.15.71:0/1061428667 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f65f4004830 con 0x7f65f80ff440 2024-04-23T02:55:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.596+0000 7f65eeffd700 1 -- 172.21.15.71:0/1061428667 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65f4005500 con 0x7f65f80ff440 2024-04-23T02:55:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.596+0000 7f65ffef6700 1 -- 172.21.15.71:0/1061428667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f81993e0 con 0x7f65f80ff440 2024-04-23T02:55:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.597+0000 7f65ffef6700 1 -- 172.21.15.71:0/1061428667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f81055a0 con 0x7f65f80ff440 2024-04-23T02:55:15.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.598+0000 7f65eeffd700 1 -- 172.21.15.71:0/1061428667 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f65f40049a0 con 0x7f65f80ff440 2024-04-23T02:55:15.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.598+0000 7f65eeffd700 1 --2- 172.21.15.71:0/1061428667 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f65e406cce0 0x7f65e406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:15.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.598+0000 7f65eeffd700 1 -- 172.21.15.71:0/1061428667 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f65f408c2b0 con 0x7f65f80ff440 2024-04-23T02:55:15.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.598+0000 7f65fd491700 1 --2- 172.21.15.71:0/1061428667 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f65e406cce0 0x7f65e406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:15.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.598+0000 7f65fd491700 1 --2- 172.21.15.71:0/1061428667 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f65e406cce0 0x7f65e406f1a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f65e8006010 tx=0x7f65e800b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:15.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.603+0000 7f65eeffd700 1 -- 172.21.15.71:0/1061428667 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f65f4059db0 con 0x7f65f80ff440 2024-04-23T02:55:15.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:15 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/704092107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T02:55:15.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:15 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3509888277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T02:55:15.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.795+0000 7f65ffef6700 1 -- 172.21.15.71:0/1061428667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f65f804ea90 con 0x7f65f80ff440 2024-04-23T02:55:15.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.796+0000 7f65eeffd700 1 -- 172.21.15.71:0/1061428667 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f65f4059940 con 0x7f65f80ff440 2024-04-23T02:55:15.798 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-23T02:55:15.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.798+0000 7f65ffef6700 1 -- 172.21.15.71:0/1061428667 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f65e406cce0 msgr2=0x7f65e406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:15.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.798+0000 7f65ffef6700 1 --2- 172.21.15.71:0/1061428667 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f65e406cce0 0x7f65e406f1a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f65e8006010 tx=0x7f65e800b540 comp rx=0 tx=0).stop 2024-04-23T02:55:15.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.798+0000 7f65ffef6700 1 -- 172.21.15.71:0/1061428667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65f80ff440 msgr2=0x7f65f819e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:15.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.798+0000 7f65ffef6700 1 --2- 172.21.15.71:0/1061428667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65f80ff440 0x7f65f819e5f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f65f400c8a0 tx=0x7f65f400cc60 comp rx=0 tx=0).stop 2024-04-23T02:55:15.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.798+0000 7f65ffef6700 1 -- 172.21.15.71:0/1061428667 shutdown_connections 2024-04-23T02:55:15.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.798+0000 7f65ffef6700 1 --2- 172.21.15.71:0/1061428667 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f65e406cce0 0x7f65e406f1a0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.798+0000 7f65ffef6700 1 --2- 172.21.15.71:0/1061428667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65f80ff440 0x7f65f819e5f0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.798+0000 7f65ffef6700 1 --2- 172.21.15.71:0/1061428667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65f819eb30 0x7f65f8198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.798+0000 7f65ffef6700 1 -- 172.21.15.71:0/1061428667 >> 172.21.15.71:0/1061428667 conn(0x7f65f8074c10 msgr2=0x7f65f81026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:15.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.798+0000 7f65ffef6700 1 -- 172.21.15.71:0/1061428667 shutdown_connections 2024-04-23T02:55:15.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.798+0000 7f65ffef6700 1 -- 172.21.15.71:0/1061428667 wait complete. 2024-04-23T02:55:15.812 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-23T02:55:15.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.935+0000 7f7091171700 1 -- 172.21.15.71:0/3539803776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708c0ffae0 msgr2=0x7f708c0fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:15.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.935+0000 7f7091171700 1 --2- 172.21.15.71:0/3539803776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708c0ffae0 0x7f708c0fff60 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f707c009910 tx=0x7f707c009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:15.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.935+0000 7f7091171700 1 -- 172.21.15.71:0/3539803776 shutdown_connections 2024-04-23T02:55:15.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.935+0000 7f7091171700 1 --2- 172.21.15.71:0/3539803776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708c0ffae0 0x7f708c0fff60 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.935+0000 7f7091171700 1 --2- 172.21.15.71:0/3539803776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f708c105970 0x7f708c105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.935+0000 7f7091171700 1 -- 172.21.15.71:0/3539803776 >> 172.21.15.71:0/3539803776 conn(0x7f708c0753d0 msgr2=0x7f708c0757e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:15.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.935+0000 7f7091171700 1 -- 172.21.15.71:0/3539803776 shutdown_connections 2024-04-23T02:55:15.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.935+0000 7f7091171700 1 -- 172.21.15.71:0/3539803776 wait complete. 2024-04-23T02:55:15.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.936+0000 7f7091171700 1 Processor -- start 2024-04-23T02:55:15.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.936+0000 7f7091171700 1 -- start start 2024-04-23T02:55:15.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.936+0000 7f7091171700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708c0ffae0 0x7f708c197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:15.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.936+0000 7f7091171700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f708c105970 0x7f708c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:15.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.936+0000 7f7091171700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c1980a0 con 0x7f708c0ffae0 2024-04-23T02:55:15.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.936+0000 7f7091171700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c198210 con 0x7f708c105970 2024-04-23T02:55:15.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.936+0000 7f708ad9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708c0ffae0 0x7f708c197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:15.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.936+0000 7f708ad9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708c0ffae0 0x7f708c197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60540/0 (socket says 172.21.15.71:60540) 2024-04-23T02:55:15.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.936+0000 7f708ad9d700 1 -- 172.21.15.71:0/2492078719 learned_addr learned my addr 172.21.15.71:0/2492078719 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:15.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.936+0000 7f708a59c700 1 --2- 172.21.15.71:0/2492078719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f708c105970 0x7f708c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:15.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.937+0000 7f708ad9d700 1 -- 172.21.15.71:0/2492078719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f708c105970 msgr2=0x7f708c1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:15.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.937+0000 7f708ad9d700 1 --2- 172.21.15.71:0/2492078719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f708c105970 0x7f708c1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:15.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.937+0000 7f708ad9d700 1 -- 172.21.15.71:0/2492078719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c0095f0 con 0x7f708c0ffae0 2024-04-23T02:55:15.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.937+0000 7f708ad9d700 1 --2- 172.21.15.71:0/2492078719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708c0ffae0 0x7f708c197480 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f707400d8d0 tx=0x7f707400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:15.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.937+0000 7f7083fff700 1 -- 172.21.15.71:0/2492078719 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70740098a0 con 0x7f708c0ffae0 2024-04-23T02:55:15.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.937+0000 7f7083fff700 1 -- 172.21.15.71:0/2492078719 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7074010460 con 0x7f708c0ffae0 2024-04-23T02:55:15.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.937+0000 7f7091171700 1 -- 172.21.15.71:0/2492078719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708c1a04f0 con 0x7f708c0ffae0 2024-04-23T02:55:15.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.937+0000 7f7083fff700 1 -- 172.21.15.71:0/2492078719 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7074009ea0 con 0x7f708c0ffae0 2024-04-23T02:55:15.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.937+0000 7f7091171700 1 -- 172.21.15.71:0/2492078719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708c1a0a50 con 0x7f708c0ffae0 2024-04-23T02:55:15.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.938+0000 7f7091171700 1 -- 172.21.15.71:0/2492078719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708c107ef0 con 0x7f708c0ffae0 2024-04-23T02:55:15.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.938+0000 7f7083fff700 1 -- 172.21.15.71:0/2492078719 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f70740105d0 con 0x7f708c0ffae0 2024-04-23T02:55:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.939+0000 7f7083fff700 1 --2- 172.21.15.71:0/2492078719 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f707806cce0 0x7f707806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.939+0000 7f708a59c700 1 --2- 172.21.15.71:0/2492078719 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f707806cce0 0x7f707806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.939+0000 7f7083fff700 1 -- 172.21.15.71:0/2492078719 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f707408c480 con 0x7f708c0ffae0 2024-04-23T02:55:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.939+0000 7f708a59c700 1 --2- 172.21.15.71:0/2492078719 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f707806cce0 0x7f707806f1a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f708c068e80 tx=0x7f707c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:15.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:15.944+0000 7f7083fff700 1 -- 172.21.15.71:0/2492078719 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f70740564f0 con 0x7f708c0ffae0 2024-04-23T02:55:16.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:15 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/704092107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T02:55:16.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:15 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3509888277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T02:55:16.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.133+0000 7f7091171700 1 -- 172.21.15.71:0/2492078719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f708c04ea90 con 0x7f708c0ffae0 2024-04-23T02:55:16.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.134+0000 7f7083fff700 1 -- 172.21.15.71:0/2492078719 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f708c04ea90 con 0x7f708c0ffae0 2024-04-23T02:55:16.136 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-23T02:55:16.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.136+0000 7f7091171700 1 -- 172.21.15.71:0/2492078719 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f707806cce0 msgr2=0x7f707806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:16.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.136+0000 7f7091171700 1 --2- 172.21.15.71:0/2492078719 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f707806cce0 0x7f707806f1a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f708c068e80 tx=0x7f707c01a040 comp rx=0 tx=0).stop 2024-04-23T02:55:16.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.136+0000 7f7091171700 1 -- 172.21.15.71:0/2492078719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708c0ffae0 msgr2=0x7f708c197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:16.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.136+0000 7f7091171700 1 --2- 172.21.15.71:0/2492078719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708c0ffae0 0x7f708c197480 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f707400d8d0 tx=0x7f707400dbe0 comp rx=0 tx=0).stop 2024-04-23T02:55:16.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.136+0000 7f7091171700 1 -- 172.21.15.71:0/2492078719 shutdown_connections 2024-04-23T02:55:16.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.136+0000 7f7091171700 1 --2- 172.21.15.71:0/2492078719 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f707806cce0 0x7f707806f1a0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:16.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.136+0000 7f7091171700 1 --2- 172.21.15.71:0/2492078719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f708c0ffae0 0x7f708c197480 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:16.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.136+0000 7f7091171700 1 --2- 172.21.15.71:0/2492078719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f708c105970 0x7f708c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:16.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.136+0000 7f7091171700 1 -- 172.21.15.71:0/2492078719 >> 172.21.15.71:0/2492078719 conn(0x7f708c0753d0 msgr2=0x7f708c0fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:16.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.136+0000 7f7091171700 1 -- 172.21.15.71:0/2492078719 shutdown_connections 2024-04-23T02:55:16.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.136+0000 7f7091171700 1 -- 172.21.15.71:0/2492078719 wait complete. 2024-04-23T02:55:16.151 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T02:55:16.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.270+0000 7f07835b4700 1 -- 172.21.15.71:0/3135086326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f077c105960 msgr2=0x7f077c105d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:16.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.270+0000 7f07835b4700 1 --2- 172.21.15.71:0/3135086326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f077c105960 0x7f077c105d40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f0778009910 tx=0x7f0778009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:16.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.270+0000 7f07835b4700 1 -- 172.21.15.71:0/3135086326 shutdown_connections 2024-04-23T02:55:16.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.270+0000 7f07835b4700 1 --2- 172.21.15.71:0/3135086326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f077c0ff950 0x7f077c0ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:16.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.270+0000 7f07835b4700 1 --2- 172.21.15.71:0/3135086326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f077c105960 0x7f077c105d40 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:16.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.270+0000 7f07835b4700 1 -- 172.21.15.71:0/3135086326 >> 172.21.15.71:0/3135086326 conn(0x7f077c0fb440 msgr2=0x7f077c0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:16.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.270+0000 7f07835b4700 1 -- 172.21.15.71:0/3135086326 shutdown_connections 2024-04-23T02:55:16.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.270+0000 7f07835b4700 1 -- 172.21.15.71:0/3135086326 wait complete. 2024-04-23T02:55:16.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.271+0000 7f07835b4700 1 Processor -- start 2024-04-23T02:55:16.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.271+0000 7f07835b4700 1 -- start start 2024-04-23T02:55:16.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.271+0000 7f07835b4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f077c0ff950 0x7f077c0fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:16.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.271+0000 7f07835b4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f077c105960 0x7f077c0fc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:16.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.271+0000 7f07835b4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c0fc720 con 0x7f077c0ff950 2024-04-23T02:55:16.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.271+0000 7f07835b4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c0fc860 con 0x7f077c105960 2024-04-23T02:55:16.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.271+0000 7f0781350700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f077c0ff950 0x7f077c0fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:16.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.271+0000 7f0781350700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f077c0ff950 0x7f077c0fbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60558/0 (socket says 172.21.15.71:60558) 2024-04-23T02:55:16.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.271+0000 7f0781350700 1 -- 172.21.15.71:0/4054445832 learned_addr learned my addr 172.21.15.71:0/4054445832 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:16.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.271+0000 7f0780b4f700 1 --2- 172.21.15.71:0/4054445832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f077c105960 0x7f077c0fc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:16.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.271+0000 7f0781350700 1 -- 172.21.15.71:0/4054445832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f077c105960 msgr2=0x7f077c0fc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:16.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.271+0000 7f0781350700 1 --2- 172.21.15.71:0/4054445832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f077c105960 0x7f077c0fc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:16.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.271+0000 7f0781350700 1 -- 172.21.15.71:0/4054445832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07780095f0 con 0x7f077c0ff950 2024-04-23T02:55:16.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.272+0000 7f0781350700 1 --2- 172.21.15.71:0/4054445832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f077c0ff950 0x7f077c0fbca0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0778006010 tx=0x7f077800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:16.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.272+0000 7f076e7fc700 1 -- 172.21.15.71:0/4054445832 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f077801d070 con 0x7f077c0ff950 2024-04-23T02:55:16.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.272+0000 7f076e7fc700 1 -- 172.21.15.71:0/4054445832 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0778005dc0 con 0x7f077c0ff950 2024-04-23T02:55:16.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.272+0000 7f07835b4700 1 -- 172.21.15.71:0/4054445832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077c0fcae0 con 0x7f077c0ff950 2024-04-23T02:55:16.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.272+0000 7f076e7fc700 1 -- 172.21.15.71:0/4054445832 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0778021410 con 0x7f077c0ff950 2024-04-23T02:55:16.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.272+0000 7f07835b4700 1 -- 172.21.15.71:0/4054445832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077c1a29e0 con 0x7f077c0ff950 2024-04-23T02:55:16.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.273+0000 7f07835b4700 1 -- 172.21.15.71:0/4054445832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077c108f60 con 0x7f077c0ff950 2024-04-23T02:55:16.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.273+0000 7f076e7fc700 1 -- 172.21.15.71:0/4054445832 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0778003a60 con 0x7f077c0ff950 2024-04-23T02:55:16.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.273+0000 7f076e7fc700 1 --2- 172.21.15.71:0/4054445832 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f076806cda0 0x7f076806f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:16.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.274+0000 7f0780b4f700 1 --2- 172.21.15.71:0/4054445832 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f076806cda0 0x7f076806f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:16.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.274+0000 7f076e7fc700 1 -- 172.21.15.71:0/4054445832 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f077808d7e0 con 0x7f077c0ff950 2024-04-23T02:55:16.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.274+0000 7f0780b4f700 1 --2- 172.21.15.71:0/4054445832 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f076806cda0 0x7f076806f260 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f077c0fd490 tx=0x7f077000b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:16.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.278+0000 7f076e7fc700 1 -- 172.21.15.71:0/4054445832 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0778057850 con 0x7f077c0ff950 2024-04-23T02:55:16.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.486+0000 7f07835b4700 1 -- 172.21.15.71:0/4054445832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f077c04ea90 con 0x7f077c0ff950 2024-04-23T02:55:16.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.486+0000 7f076e7fc700 1 -- 172.21.15.71:0/4054445832 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 35 entries v84) v1 ==== 81+0+2077 (secure 0 0 0) 0x7f0778003d10 con 0x7f077c0ff950 2024-04-23T02:55:16.488 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6828/467545394 2024-04-24T02:22:00.280478+0000 2024-04-23T02:55:16.489 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/3794652670 2024-04-24T02:03:48.178880+0000 2024-04-23T02:55:16.489 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/4165305842 2024-04-24T02:05:42.095805+0000 2024-04-23T02:55:16.489 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/56041968 2024-04-24T02:05:42.095805+0000 2024-04-23T02:55:16.489 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:0/1753179829 2024-04-24T02:14:34.629978+0000 2024-04-23T02:55:16.489 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/4165305842 2024-04-24T02:05:42.095805+0000 2024-04-23T02:55:16.489 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:0/2569976755 2024-04-24T02:14:34.629978+0000 2024-04-23T02:55:16.489 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/823266478 2024-04-24T02:04:23.247815+0000 2024-04-23T02:55:16.489 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:6828/966231696 2024-04-24T02:14:34.629978+0000 2024-04-23T02:55:16.489 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:6824/3780549896 2024-04-24T02:22:00.280491+0000 2024-04-23T02:55:16.489 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:0/1701960325 2024-04-24T02:14:34.629978+0000 2024-04-23T02:55:16.489 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/2225989508 2024-04-24T02:03:48.178880+0000 2024-04-23T02:55:16.489 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6829/467545394 2024-04-24T02:22:00.280478+0000 2024-04-23T02:55:16.489 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:0/2687070241 2024-04-24T02:14:34.629978+0000 2024-04-23T02:55:16.490 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/17588705 2024-04-24T02:13:11.575106+0000 2024-04-23T02:55:16.490 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3579793925 2024-04-24T02:04:23.247815+0000 2024-04-23T02:55:16.490 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/1995129374 2024-04-24T02:03:48.178880+0000 2024-04-23T02:55:16.490 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/294030681 2024-04-24T02:04:23.247815+0000 2024-04-23T02:55:16.490 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/936344738 2024-04-24T02:05:42.095805+0000 2024-04-23T02:55:16.490 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:6829/966231696 2024-04-24T02:14:34.629978+0000 2024-04-23T02:55:16.490 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:0/2727292170 2024-04-24T02:14:34.629978+0000 2024-04-23T02:55:16.490 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/1815873101 2024-04-24T02:13:11.575106+0000 2024-04-23T02:55:16.490 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:6825/3780549896 2024-04-24T02:22:00.280491+0000 2024-04-23T02:55:16.490 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/3878148961 2024-04-24T02:04:23.247815+0000 2024-04-23T02:55:16.490 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6827/3504670324 2024-04-24T02:11:05.663451+0000 2024-04-23T02:55:16.490 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:0/2877439214 2024-04-24T02:14:34.629978+0000 2024-04-23T02:55:16.490 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/4249907286 2024-04-24T02:13:11.575106+0000 2024-04-23T02:55:16.491 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/3878148961 2024-04-24T02:04:23.247815+0000 2024-04-23T02:55:16.491 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/3794652670 2024-04-24T02:03:48.178880+0000 2024-04-23T02:55:16.491 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/4205332842 2024-04-24T02:03:48.178880+0000 2024-04-23T02:55:16.491 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3618236204 2024-04-24T02:13:11.575106+0000 2024-04-23T02:55:16.491 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3473332587 2024-04-24T02:13:11.575106+0000 2024-04-23T02:55:16.491 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/2206241071 2024-04-24T02:05:42.095805+0000 2024-04-23T02:55:16.491 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/4249907286 2024-04-24T02:13:11.575106+0000 2024-04-23T02:55:16.491 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6826/3504670324 2024-04-24T02:11:05.663451+0000 2024-04-23T02:55:16.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.488+0000 7f07835b4700 1 -- 172.21.15.71:0/4054445832 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f076806cda0 msgr2=0x7f076806f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:16.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.488+0000 7f07835b4700 1 --2- 172.21.15.71:0/4054445832 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f076806cda0 0x7f076806f260 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f077c0fd490 tx=0x7f077000b410 comp rx=0 tx=0).stop 2024-04-23T02:55:16.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.488+0000 7f07835b4700 1 -- 172.21.15.71:0/4054445832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f077c0ff950 msgr2=0x7f077c0fbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:16.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.488+0000 7f07835b4700 1 --2- 172.21.15.71:0/4054445832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f077c0ff950 0x7f077c0fbca0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0778006010 tx=0x7f077800bd70 comp rx=0 tx=0).stop 2024-04-23T02:55:16.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.488+0000 7f07835b4700 1 -- 172.21.15.71:0/4054445832 shutdown_connections 2024-04-23T02:55:16.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.488+0000 7f07835b4700 1 --2- 172.21.15.71:0/4054445832 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f076806cda0 0x7f076806f260 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:16.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.488+0000 7f07835b4700 1 --2- 172.21.15.71:0/4054445832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f077c0ff950 0x7f077c0fbca0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:16.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.488+0000 7f07835b4700 1 --2- 172.21.15.71:0/4054445832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f077c105960 0x7f077c0fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:16.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.488+0000 7f07835b4700 1 -- 172.21.15.71:0/4054445832 >> 172.21.15.71:0/4054445832 conn(0x7f077c0fb440 msgr2=0x7f077c068f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:16.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.488+0000 7f07835b4700 1 -- 172.21.15.71:0/4054445832 shutdown_connections 2024-04-23T02:55:16.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.488+0000 7f07835b4700 1 -- 172.21.15.71:0/4054445832 wait complete. 2024-04-23T02:55:16.493 INFO:teuthology.orchestra.run.smithi071.stderr:listed 35 entries 2024-04-23T02:55:16.503 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-23T02:55:16.503 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T02:55:16.522 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T02:55:16.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.681+0000 7fd12c736700 1 -- 172.21.15.71:0/1428240492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1240ff4a0 msgr2=0x7fd1240ff880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:16.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.682+0000 7fd12c736700 1 --2- 172.21.15.71:0/1428240492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1240ff4a0 0x7fd1240ff880 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd114009910 tx=0x7fd114009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:16.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.682+0000 7fd12c736700 1 -- 172.21.15.71:0/1428240492 shutdown_connections 2024-04-23T02:55:16.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.682+0000 7fd12c736700 1 --2- 172.21.15.71:0/1428240492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1240ffe50 0x7fd124105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:16.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.682+0000 7fd12c736700 1 --2- 172.21.15.71:0/1428240492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1240ff4a0 0x7fd1240ff880 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:16.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.682+0000 7fd12c736700 1 -- 172.21.15.71:0/1428240492 >> 172.21.15.71:0/1428240492 conn(0x7fd124074b60 msgr2=0x7fd124074f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:16.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.682+0000 7fd12c736700 1 -- 172.21.15.71:0/1428240492 shutdown_connections 2024-04-23T02:55:16.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.682+0000 7fd12c736700 1 -- 172.21.15.71:0/1428240492 wait complete. 2024-04-23T02:55:16.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.682+0000 7fd12c736700 1 Processor -- start 2024-04-23T02:55:16.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.682+0000 7fd12c736700 1 -- start start 2024-04-23T02:55:16.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.682+0000 7fd12c736700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1240ff4a0 0x7fd124196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:16.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.682+0000 7fd12c736700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd124196bb0 0x7fd12419b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:16.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.682+0000 7fd12c736700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1241971d0 con 0x7fd1240ff4a0 2024-04-23T02:55:16.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.682+0000 7fd12c736700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd124197340 con 0x7fd124196bb0 2024-04-23T02:55:16.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.683+0000 7fd12a4d2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1240ff4a0 0x7fd124196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:16.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.683+0000 7fd12a4d2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1240ff4a0 0x7fd124196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60576/0 (socket says 172.21.15.71:60576) 2024-04-23T02:55:16.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.683+0000 7fd12a4d2700 1 -- 172.21.15.71:0/1328094372 learned_addr learned my addr 172.21.15.71:0/1328094372 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:16.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.683+0000 7fd12a4d2700 1 -- 172.21.15.71:0/1328094372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd124196bb0 msgr2=0x7fd12419b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:16.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.683+0000 7fd129cd1700 1 --2- 172.21.15.71:0/1328094372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd124196bb0 0x7fd12419b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:16.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.683+0000 7fd12a4d2700 1 --2- 172.21.15.71:0/1328094372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd124196bb0 0x7fd12419b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:16.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.683+0000 7fd12a4d2700 1 -- 172.21.15.71:0/1328094372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1140095f0 con 0x7fd1240ff4a0 2024-04-23T02:55:16.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.683+0000 7fd129cd1700 1 --2- 172.21.15.71:0/1328094372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd124196bb0 0x7fd12419b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T02:55:16.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.683+0000 7fd12a4d2700 1 --2- 172.21.15.71:0/1328094372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1240ff4a0 0x7fd124196670 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd114006010 tx=0x7fd11400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:16.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.683+0000 7fd11b7fe700 1 -- 172.21.15.71:0/1328094372 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd11401d070 con 0x7fd1240ff4a0 2024-04-23T02:55:16.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.683+0000 7fd11b7fe700 1 -- 172.21.15.71:0/1328094372 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd11400f460 con 0x7fd1240ff4a0 2024-04-23T02:55:16.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.683+0000 7fd12c736700 1 -- 172.21.15.71:0/1328094372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12419b610 con 0x7fd1240ff4a0 2024-04-23T02:55:16.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.683+0000 7fd11b7fe700 1 -- 172.21.15.71:0/1328094372 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd114021640 con 0x7fd1240ff4a0 2024-04-23T02:55:16.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.683+0000 7fd12c736700 1 -- 172.21.15.71:0/1328094372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12419bb10 con 0x7fd1240ff4a0 2024-04-23T02:55:16.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.684+0000 7fd12c736700 1 -- 172.21.15.71:0/1328094372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd124109100 con 0x7fd1240ff4a0 2024-04-23T02:55:16.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.685+0000 7fd11b7fe700 1 -- 172.21.15.71:0/1328094372 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd11400fa60 con 0x7fd1240ff4a0 2024-04-23T02:55:16.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.685+0000 7fd11b7fe700 1 --2- 172.21.15.71:0/1328094372 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd11006cda0 0x7fd11006f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:16.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.685+0000 7fd11b7fe700 1 -- 172.21.15.71:0/1328094372 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd11408d880 con 0x7fd1240ff4a0 2024-04-23T02:55:16.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.685+0000 7fd129cd1700 1 --2- 172.21.15.71:0/1328094372 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd11006cda0 0x7fd11006f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:16.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.685+0000 7fd129cd1700 1 --2- 172.21.15.71:0/1328094372 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd11006cda0 0x7fd11006f260 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd12404ed30 tx=0x7fd120008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:16.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.690+0000 7fd11b7fe700 1 -- 172.21.15.71:0/1328094372 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd1140578f0 con 0x7fd1240ff4a0 2024-04-23T02:55:16.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.884+0000 7fd12c736700 1 -- 172.21.15.71:0/1328094372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd12404ea90 con 0x7fd1240ff4a0 2024-04-23T02:55:16.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.885+0000 7fd11b7fe700 1 -- 172.21.15.71:0/1328094372 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16305 (secure 0 0 0) 0x7fd11405af10 con 0x7fd1240ff4a0 2024-04-23T02:55:16.887 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-23T02:55:16.887 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":84,"fsid":"91f5470c-0115-11ef-bc93-c7b262605968","created":"2024-04-23T02:03:14.872097+0000","modified":"2024-04-23T02:37:53.913348+0000","last_up_change":"2024-04-23T02:21:57.350899+0000","last_in_change":"2024-04-23T02:09:09.560591+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T02:08:06.416728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-23T02:10:50.283526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-23T02:10:50.576356+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5f48bf1-6e98-4432-885a-f6ed0c42e5d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2865488005},{"type":"v1","addr":"172.21.15.71:6803","nonce":2865488005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2865488005},{"type":"v1","addr":"172.21.15.71:6805","nonce":2865488005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":2865488005},{"type":"v1","addr":"172.21.15.71:6809","nonce":2865488005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2865488005},{"type":"v1","addr":"172.21.15.71:6807","nonce":2865488005}]},"public_addr":"172.21.15.71:6803/2865488005","cluster_addr":"172.21.15.71:6805/2865488005","heartbeat_back_addr":"172.21.15.71:6809/2865488005","heartbeat_front_addr":"172.21.15.71:6807/2865488005","state":["exists","up"]},{"osd":1,"uuid":"04304fd2-42f9-4525-88e8-c7e20c9c51e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":2293979729},{"type":"v1","addr":"172.21.15.71:6811","nonce":2293979729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":2293979729},{"type":"v1","addr":"172.21.15.71:6813","nonce":2293979729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":2293979729},{"type":"v1","addr":"172.21.15.71:6817","nonce":2293979729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":2293979729},{"type":"v1","addr":"172.21.15.71:6815","nonce":2293979729}]},"public_addr":"172.21.15.71:6811/2293979729","cluster_addr":"172.21.15.71:6813/2293979729","heartbeat_back_addr":"172.21.15.71:6817/2293979729","heartbeat_front_addr":"172.21.15.71:6815/2293979729","state":["exists","up"]},{"osd":2,"uuid":"6797c77c-4ccc-4489-9269-77105d7c45c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3388560056},{"type":"v1","addr":"172.21.15.71:6819","nonce":3388560056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3388560056},{"type":"v1","addr":"172.21.15.71:6821","nonce":3388560056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3388560056},{"type":"v1","addr":"172.21.15.71:6825","nonce":3388560056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3388560056},{"type":"v1","addr":"172.21.15.71:6823","nonce":3388560056}]},"public_addr":"172.21.15.71:6819/3388560056","cluster_addr":"172.21.15.71:6821/3388560056","heartbeat_back_addr":"172.21.15.71:6825/3388560056","heartbeat_front_addr":"172.21.15.71:6823/3388560056","state":["exists","up"]},{"osd":3,"uuid":"62033906-4d1c-431d-a87d-a12e0a134383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":4106623627},{"type":"v1","addr":"172.21.15.192:6801","nonce":4106623627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":4106623627},{"type":"v1","addr":"172.21.15.192:6803","nonce":4106623627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":4106623627},{"type":"v1","addr":"172.21.15.192:6807","nonce":4106623627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":4106623627},{"type":"v1","addr":"172.21.15.192:6805","nonce":4106623627}]},"public_addr":"172.21.15.192:6801/4106623627","cluster_addr":"172.21.15.192:6803/4106623627","heartbeat_back_addr":"172.21.15.192:6807/4106623627","heartbeat_front_addr":"172.21.15.192:6805/4106623627","state":["exists","up"]},{"osd":4,"uuid":"6a7317f5-dc05-415a-8409-81f80c196a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1707328698},{"type":"v1","addr":"172.21.15.192:6809","nonce":1707328698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1707328698},{"type":"v1","addr":"172.21.15.192:6811","nonce":1707328698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1707328698},{"type":"v1","addr":"172.21.15.192:6815","nonce":1707328698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1707328698},{"type":"v1","addr":"172.21.15.192:6813","nonce":1707328698}]},"public_addr":"172.21.15.192:6809/1707328698","cluster_addr":"172.21.15.192:6811/1707328698","heartbeat_back_addr":"172.21.15.192:6815/1707328698","heartbeat_front_addr":"172.21.15.192:6813/1707328698","state":["exists","up"]},{"osd":5,"uuid":"68b550d5-d74e-417e-9bcc-98740efaf0ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":868066904},{"type":"v1","addr":"172.21.15.192:6817","nonce":868066904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":868066904},{"type":"v1","addr":"172.21.15.192:6819","nonce":868066904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":868066904},{"type":"v1","addr":"172.21.15.192:6823","nonce":868066904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":868066904},{"type":"v1","addr":"172.21.15.192:6821","nonce":868066904}]},"public_addr":"172.21.15.192:6817/868066904","cluster_addr":"172.21.15.192:6819/868066904","heartbeat_back_addr":"172.21.15.192:6823/868066904","heartbeat_front_addr":"172.21.15.192:6821/868066904","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:07:01.863954+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:07:32.930839+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:04.093747+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:31.574485+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:08:59.698428+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T02:09:28.076223+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6828/467545394":"2024-04-24T02:22:00.280478+0000","172.21.15.71:6801/3794652670":"2024-04-24T02:03:48.178880+0000","172.21.15.71:6801/4165305842":"2024-04-24T02:05:42.095805+0000","172.21.15.71:0/56041968":"2024-04-24T02:05:42.095805+0000","172.21.15.192:0/1753179829":"2024-04-24T02:14:34.629978+0000","172.21.15.71:6800/4165305842":"2024-04-24T02:05:42.095805+0000","172.21.15.192:0/2569976755":"2024-04-24T02:14:34.629978+0000","172.21.15.71:0/823266478":"2024-04-24T02:04:23.247815+0000","172.21.15.192:6828/966231696":"2024-04-24T02:14:34.629978+0000","172.21.15.192:6824/3780549896":"2024-04-24T02:22:00.280491+0000","172.21.15.192:0/1701960325":"2024-04-24T02:14:34.629978+0000","172.21.15.71:0/2225989508":"2024-04-24T02:03:48.178880+0000","172.21.15.71:6829/467545394":"2024-04-24T02:22:00.280478+0000","172.21.15.192:0/2687070241":"2024-04-24T02:14:34.629978+0000","172.21.15.71:0/17588705":"2024-04-24T02:13:11.575106+0000","172.21.15.71:0/3579793925":"2024-04-24T02:04:23.247815+0000","172.21.15.71:0/1995129374":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/294030681":"2024-04-24T02:04:23.247815+0000","172.21.15.71:0/936344738":"2024-04-24T02:05:42.095805+0000","172.21.15.192:6829/966231696":"2024-04-24T02:14:34.629978+0000","172.21.15.192:0/2727292170":"2024-04-24T02:14:34.629978+0000","172.21.15.71:0/1815873101":"2024-04-24T02:13:11.575106+0000","172.21.15.192:6825/3780549896":"2024-04-24T02:22:00.280491+0000","172.21.15.71:6801/3878148961":"2024-04-24T02:04:23.247815+0000","172.21.15.71:6827/3504670324":"2024-04-24T02:11:05.663451+0000","172.21.15.192:0/2877439214":"2024-04-24T02:14:34.629978+0000","172.21.15.71:6800/4249907286":"2024-04-24T02:13:11.575106+0000","172.21.15.71:6800/3878148961":"2024-04-24T02:04:23.247815+0000","172.21.15.71:6800/3794652670":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/4205332842":"2024-04-24T02:03:48.178880+0000","172.21.15.71:0/3618236204":"2024-04-24T02:13:11.575106+0000","172.21.15.71:0/3473332587":"2024-04-24T02:13:11.575106+0000","172.21.15.71:0/2206241071":"2024-04-24T02:05:42.095805+0000","172.21.15.71:6801/4249907286":"2024-04-24T02:13:11.575106+0000","172.21.15.71:6826/3504670324":"2024-04-24T02:11:05.663451+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T02:55:16.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.887+0000 7fd12c736700 1 -- 172.21.15.71:0/1328094372 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd11006cda0 msgr2=0x7fd11006f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:16.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.887+0000 7fd12c736700 1 --2- 172.21.15.71:0/1328094372 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd11006cda0 0x7fd11006f260 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd12404ed30 tx=0x7fd120008040 comp rx=0 tx=0).stop 2024-04-23T02:55:16.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.887+0000 7fd12c736700 1 -- 172.21.15.71:0/1328094372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1240ff4a0 msgr2=0x7fd124196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:16.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.887+0000 7fd12c736700 1 --2- 172.21.15.71:0/1328094372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1240ff4a0 0x7fd124196670 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd114006010 tx=0x7fd11400be20 comp rx=0 tx=0).stop 2024-04-23T02:55:16.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.887+0000 7fd12c736700 1 -- 172.21.15.71:0/1328094372 shutdown_connections 2024-04-23T02:55:16.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.887+0000 7fd12c736700 1 --2- 172.21.15.71:0/1328094372 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7fd11006cda0 0x7fd11006f260 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:16.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.887+0000 7fd12c736700 1 --2- 172.21.15.71:0/1328094372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1240ff4a0 0x7fd124196670 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:16.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.887+0000 7fd12c736700 1 --2- 172.21.15.71:0/1328094372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd124196bb0 0x7fd12419b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:16.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.887+0000 7fd12c736700 1 -- 172.21.15.71:0/1328094372 >> 172.21.15.71:0/1328094372 conn(0x7fd124074b60 msgr2=0x7fd1240feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:16.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.887+0000 7fd12c736700 1 -- 172.21.15.71:0/1328094372 shutdown_connections 2024-04-23T02:55:16.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:16.887+0000 7fd12c736700 1 -- 172.21.15.71:0/1328094372 wait complete. 2024-04-23T02:55:16.900 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:16 smithi071 ceph-mon[144544]: pgmap v1261: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:55:16.900 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:16 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1061428667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-23T02:55:16.900 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:16 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/2492078719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-23T02:55:16.900 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:16 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/4054445832' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T02:55:16.902 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T02:08:06.416728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-23T02:10:50.283526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-23T02:10:50.576356+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T02:55:16.902 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T02:55:17.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.024+0000 7f1dae6b4700 1 -- 172.21.15.71:0/39136045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1da80ff460 msgr2=0x7f1da80ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:17.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.024+0000 7f1dae6b4700 1 --2- 172.21.15.71:0/39136045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1da80ff460 0x7f1da80ff840 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009910 tx=0x7f1d98009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:17.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.024+0000 7f1dae6b4700 1 -- 172.21.15.71:0/39136045 shutdown_connections 2024-04-23T02:55:17.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.024+0000 7f1dae6b4700 1 --2- 172.21.15.71:0/39136045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da80ffe10 0x7f1da8105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:17.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.024+0000 7f1dae6b4700 1 --2- 172.21.15.71:0/39136045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1da80ff460 0x7f1da80ff840 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:17.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.024+0000 7f1dae6b4700 1 -- 172.21.15.71:0/39136045 >> 172.21.15.71:0/39136045 conn(0x7f1da8074b80 msgr2=0x7f1da8074f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:17.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.024+0000 7f1dae6b4700 1 -- 172.21.15.71:0/39136045 shutdown_connections 2024-04-23T02:55:17.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.024+0000 7f1dae6b4700 1 -- 172.21.15.71:0/39136045 wait complete. 2024-04-23T02:55:17.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.024+0000 7f1dae6b4700 1 Processor -- start 2024-04-23T02:55:17.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.024+0000 7f1dae6b4700 1 -- start start 2024-04-23T02:55:17.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.025+0000 7f1dae6b4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1da80ff460 0x7f1da8196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:17.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.025+0000 7f1dae6b4700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da8196b90 0x7f1da819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:17.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.025+0000 7f1dae6b4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da81971b0 con 0x7f1da80ff460 2024-04-23T02:55:17.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.025+0000 7f1dae6b4700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da8197320 con 0x7f1da8196b90 2024-04-23T02:55:17.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.025+0000 7f1da7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1da80ff460 0x7f1da8196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:17.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.025+0000 7f1da7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1da80ff460 0x7f1da8196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60596/0 (socket says 172.21.15.71:60596) 2024-04-23T02:55:17.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.025+0000 7f1da7fff700 1 -- 172.21.15.71:0/720719345 learned_addr learned my addr 172.21.15.71:0/720719345 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:17.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.025+0000 7f1da77fe700 1 --2- 172.21.15.71:0/720719345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da8196b90 0x7f1da819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:17.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.025+0000 7f1da7fff700 1 -- 172.21.15.71:0/720719345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da8196b90 msgr2=0x7f1da819afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:17.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.025+0000 7f1da7fff700 1 --2- 172.21.15.71:0/720719345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da8196b90 0x7f1da819afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:17.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.025+0000 7f1da7fff700 1 -- 172.21.15.71:0/720719345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d980095f0 con 0x7f1da80ff460 2024-04-23T02:55:17.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.025+0000 7f1da77fe700 1 --2- 172.21.15.71:0/720719345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da8196b90 0x7f1da819afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:55:17.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.025+0000 7f1da7fff700 1 --2- 172.21.15.71:0/720719345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1da80ff460 0x7f1da8196650 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f1d98000c00 tx=0x7f1d9800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:17.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.025+0000 7f1da57fa700 1 -- 172.21.15.71:0/720719345 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d9801d070 con 0x7f1da80ff460 2024-04-23T02:55:17.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.025+0000 7f1da57fa700 1 -- 172.21.15.71:0/720719345 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1d980044c0 con 0x7f1da80ff460 2024-04-23T02:55:17.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.025+0000 7f1da57fa700 1 -- 172.21.15.71:0/720719345 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d980039d0 con 0x7f1da80ff460 2024-04-23T02:55:17.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.025+0000 7f1dae6b4700 1 -- 172.21.15.71:0/720719345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da819b5a0 con 0x7f1da80ff460 2024-04-23T02:55:17.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.025+0000 7f1dae6b4700 1 -- 172.21.15.71:0/720719345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da819b9f0 con 0x7f1da80ff460 2024-04-23T02:55:17.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.026+0000 7f1dae6b4700 1 -- 172.21.15.71:0/720719345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da81090e0 con 0x7f1da80ff460 2024-04-23T02:55:17.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.027+0000 7f1da57fa700 1 -- 172.21.15.71:0/720719345 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1d980048e0 con 0x7f1da80ff460 2024-04-23T02:55:17.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.027+0000 7f1da57fa700 1 --2- 172.21.15.71:0/720719345 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1d9006cce0 0x7f1d9006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:17.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.027+0000 7f1da57fa700 1 -- 172.21.15.71:0/720719345 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1d9808dc60 con 0x7f1da80ff460 2024-04-23T02:55:17.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.027+0000 7f1da77fe700 1 --2- 172.21.15.71:0/720719345 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1d9006cce0 0x7f1d9006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:17.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.028+0000 7f1da77fe700 1 --2- 172.21.15.71:0/720719345 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1d9006cce0 0x7f1d9006f1a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1da8067200 tx=0x7f1d9c009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:17.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.032+0000 7f1da57fa700 1 -- 172.21.15.71:0/720719345 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1d98057cd0 con 0x7f1da80ff460 2024-04-23T02:55:17.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:16 smithi192 ceph-mon[108617]: pgmap v1261: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:55:17.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:16 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1061428667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-23T02:55:17.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:16 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/2492078719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-23T02:55:17.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:16 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/4054445832' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T02:55:17.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.224+0000 7f1dae6b4700 1 -- 172.21.15.71:0/720719345 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1da819bd80 con 0x7f1da80ff460 2024-04-23T02:55:17.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.224+0000 7f1da57fa700 1 -- 172.21.15.71:0/720719345 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d9805b2f0 con 0x7f1da80ff460 2024-04-23T02:55:17.227 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-04-23T02:55:17.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.226+0000 7f1dae6b4700 1 -- 172.21.15.71:0/720719345 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1d9006cce0 msgr2=0x7f1d9006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:17.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.226+0000 7f1dae6b4700 1 --2- 172.21.15.71:0/720719345 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1d9006cce0 0x7f1d9006f1a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1da8067200 tx=0x7f1d9c009260 comp rx=0 tx=0).stop 2024-04-23T02:55:17.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.226+0000 7f1dae6b4700 1 -- 172.21.15.71:0/720719345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1da80ff460 msgr2=0x7f1da8196650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:17.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.226+0000 7f1dae6b4700 1 --2- 172.21.15.71:0/720719345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1da80ff460 0x7f1da8196650 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f1d98000c00 tx=0x7f1d9800be90 comp rx=0 tx=0).stop 2024-04-23T02:55:17.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.226+0000 7f1dae6b4700 1 -- 172.21.15.71:0/720719345 shutdown_connections 2024-04-23T02:55:17.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.226+0000 7f1dae6b4700 1 --2- 172.21.15.71:0/720719345 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1d9006cce0 0x7f1d9006f1a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:17.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.226+0000 7f1dae6b4700 1 --2- 172.21.15.71:0/720719345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1da80ff460 0x7f1da8196650 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:17.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.226+0000 7f1dae6b4700 1 --2- 172.21.15.71:0/720719345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da8196b90 0x7f1da819afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:17.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.226+0000 7f1dae6b4700 1 -- 172.21.15.71:0/720719345 >> 172.21.15.71:0/720719345 conn(0x7f1da8074b80 msgr2=0x7f1da80fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:17.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.226+0000 7f1dae6b4700 1 -- 172.21.15.71:0/720719345 shutdown_connections 2024-04-23T02:55:17.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.226+0000 7f1dae6b4700 1 -- 172.21.15.71:0/720719345 wait complete. 2024-04-23T02:55:17.241 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-23T02:55:17.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.362+0000 7f1b8e591700 1 -- 172.21.15.71:0/2099037186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b88105970 msgr2=0x7f1b88105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:17.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.362+0000 7f1b8e591700 1 --2- 172.21.15.71:0/2099037186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b88105970 0x7f1b88105d50 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009910 tx=0x7f1b7c009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:17.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.363+0000 7f1b8e591700 1 -- 172.21.15.71:0/2099037186 shutdown_connections 2024-04-23T02:55:17.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.363+0000 7f1b8e591700 1 --2- 172.21.15.71:0/2099037186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b880ffae0 0x7f1b880fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:17.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.363+0000 7f1b8e591700 1 --2- 172.21.15.71:0/2099037186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b88105970 0x7f1b88105d50 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:17.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.363+0000 7f1b8e591700 1 -- 172.21.15.71:0/2099037186 >> 172.21.15.71:0/2099037186 conn(0x7f1b880753d0 msgr2=0x7f1b880757e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:17.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.363+0000 7f1b8e591700 1 -- 172.21.15.71:0/2099037186 shutdown_connections 2024-04-23T02:55:17.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.363+0000 7f1b8e591700 1 -- 172.21.15.71:0/2099037186 wait complete. 2024-04-23T02:55:17.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.363+0000 7f1b8e591700 1 Processor -- start 2024-04-23T02:55:17.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.363+0000 7f1b8e591700 1 -- start start 2024-04-23T02:55:17.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.363+0000 7f1b8e591700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b880ffae0 0x7f1b88197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:17.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.363+0000 7f1b8e591700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b88105970 0x7f1b88197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:17.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.363+0000 7f1b8e591700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b88197fb0 con 0x7f1b88105970 2024-04-23T02:55:17.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.363+0000 7f1b8e591700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b88198120 con 0x7f1b880ffae0 2024-04-23T02:55:17.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.364+0000 7f1b877fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b88105970 0x7f1b88197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:17.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.364+0000 7f1b877fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b88105970 0x7f1b88197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60620/0 (socket says 172.21.15.71:60620) 2024-04-23T02:55:17.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.364+0000 7f1b877fe700 1 -- 172.21.15.71:0/4209218107 learned_addr learned my addr 172.21.15.71:0/4209218107 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:17.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.364+0000 7f1b87fff700 1 --2- 172.21.15.71:0/4209218107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b880ffae0 0x7f1b88197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:17.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.364+0000 7f1b877fe700 1 -- 172.21.15.71:0/4209218107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b880ffae0 msgr2=0x7f1b88197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:17.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.364+0000 7f1b877fe700 1 --2- 172.21.15.71:0/4209218107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b880ffae0 0x7f1b88197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:17.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.364+0000 7f1b877fe700 1 -- 172.21.15.71:0/4209218107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7c0095f0 con 0x7f1b88105970 2024-04-23T02:55:17.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.364+0000 7f1b87fff700 1 --2- 172.21.15.71:0/4209218107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b880ffae0 0x7f1b88197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:55:17.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.364+0000 7f1b877fe700 1 --2- 172.21.15.71:0/4209218107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b88105970 0x7f1b88197990 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800d8d0 tx=0x7f1b7800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:17.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.364+0000 7f1b857fa700 1 -- 172.21.15.71:0/4209218107 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b78009940 con 0x7f1b88105970 2024-04-23T02:55:17.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.364+0000 7f1b857fa700 1 -- 172.21.15.71:0/4209218107 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1b78010460 con 0x7f1b88105970 2024-04-23T02:55:17.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.364+0000 7f1b8e591700 1 -- 172.21.15.71:0/4209218107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b881a04c0 con 0x7f1b88105970 2024-04-23T02:55:17.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.364+0000 7f1b857fa700 1 -- 172.21.15.71:0/4209218107 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b7800f5f0 con 0x7f1b88105970 2024-04-23T02:55:17.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.364+0000 7f1b8e591700 1 -- 172.21.15.71:0/4209218107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b881a09c0 con 0x7f1b88105970 2024-04-23T02:55:17.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.365+0000 7f1b8e591700 1 -- 172.21.15.71:0/4209218107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b88107f60 con 0x7f1b88105970 2024-04-23T02:55:17.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.366+0000 7f1b857fa700 1 -- 172.21.15.71:0/4209218107 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1b780105d0 con 0x7f1b88105970 2024-04-23T02:55:17.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.366+0000 7f1b857fa700 1 --2- 172.21.15.71:0/4209218107 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1b7006cce0 0x7f1b7006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:17.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.366+0000 7f1b857fa700 1 -- 172.21.15.71:0/4209218107 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1b7808c5e0 con 0x7f1b88105970 2024-04-23T02:55:17.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.366+0000 7f1b87fff700 1 --2- 172.21.15.71:0/4209218107 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1b7006cce0 0x7f1b7006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:17.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.367+0000 7f1b87fff700 1 --2- 172.21.15.71:0/4209218107 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1b7006cce0 0x7f1b7006f1a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c006010 tx=0x7f1b7c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:17.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.371+0000 7f1b857fa700 1 -- 172.21.15.71:0/4209218107 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1b78056650 con 0x7f1b88105970 2024-04-23T02:55:17.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.567+0000 7f1b8e591700 1 -- 172.21.15.71:0/4209218107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f1b8804ea90 con 0x7f1b88105970 2024-04-23T02:55:17.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.567+0000 7f1b857fa700 1 -- 172.21.15.71:0/4209218107 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f1b78010880 con 0x7f1b88105970 2024-04-23T02:55:17.569 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-23T02:55:17.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.569+0000 7f1b8e591700 1 -- 172.21.15.71:0/4209218107 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1b7006cce0 msgr2=0x7f1b7006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:17.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.569+0000 7f1b8e591700 1 --2- 172.21.15.71:0/4209218107 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1b7006cce0 0x7f1b7006f1a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c006010 tx=0x7f1b7c01a040 comp rx=0 tx=0).stop 2024-04-23T02:55:17.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.569+0000 7f1b8e591700 1 -- 172.21.15.71:0/4209218107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b88105970 msgr2=0x7f1b88197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:17.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.569+0000 7f1b8e591700 1 --2- 172.21.15.71:0/4209218107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b88105970 0x7f1b88197990 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800d8d0 tx=0x7f1b7800dc90 comp rx=0 tx=0).stop 2024-04-23T02:55:17.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.569+0000 7f1b8e591700 1 -- 172.21.15.71:0/4209218107 shutdown_connections 2024-04-23T02:55:17.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.569+0000 7f1b8e591700 1 --2- 172.21.15.71:0/4209218107 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f1b7006cce0 0x7f1b7006f1a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:17.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.569+0000 7f1b8e591700 1 --2- 172.21.15.71:0/4209218107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1b880ffae0 0x7f1b88197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:17.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.569+0000 7f1b8e591700 1 --2- 172.21.15.71:0/4209218107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b88105970 0x7f1b88197990 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:17.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.569+0000 7f1b8e591700 1 -- 172.21.15.71:0/4209218107 >> 172.21.15.71:0/4209218107 conn(0x7f1b880753d0 msgr2=0x7f1b880fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:17.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.569+0000 7f1b8e591700 1 -- 172.21.15.71:0/4209218107 shutdown_connections 2024-04-23T02:55:17.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.569+0000 7f1b8e591700 1 -- 172.21.15.71:0/4209218107 wait complete. 2024-04-23T02:55:17.584 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-23T02:55:17.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.709+0000 7f3079cab700 1 -- 172.21.15.71:0/115691153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3074100550 msgr2=0x7f3074100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:17.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.709+0000 7f3079cab700 1 --2- 172.21.15.71:0/115691153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3074100550 0x7f3074100930 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f3068009910 tx=0x7f3068009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:17.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.710+0000 7f3079cab700 1 -- 172.21.15.71:0/115691153 shutdown_connections 2024-04-23T02:55:17.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.710+0000 7f3079cab700 1 --2- 172.21.15.71:0/115691153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3074100f00 0x7f3074104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:17.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.710+0000 7f3079cab700 1 --2- 172.21.15.71:0/115691153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3074100550 0x7f3074100930 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:17.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.710+0000 7f3079cab700 1 -- 172.21.15.71:0/115691153 >> 172.21.15.71:0/115691153 conn(0x7f3074074c10 msgr2=0x7f3074075020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:17.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.710+0000 7f3079cab700 1 -- 172.21.15.71:0/115691153 shutdown_connections 2024-04-23T02:55:17.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.710+0000 7f3079cab700 1 -- 172.21.15.71:0/115691153 wait complete. 2024-04-23T02:55:17.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.710+0000 7f3079cab700 1 Processor -- start 2024-04-23T02:55:17.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.710+0000 7f3079cab700 1 -- start start 2024-04-23T02:55:17.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.710+0000 7f3079cab700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3074100550 0x7f30741974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:17.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.710+0000 7f3079cab700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3074197a10 0x7f3074197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:17.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.710+0000 7f3079cab700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30741a0540 con 0x7f3074197a10 2024-04-23T02:55:17.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.711+0000 7f3079cab700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30741a0680 con 0x7f3074100550 2024-04-23T02:55:17.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.711+0000 7f3072ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3074197a10 0x7f3074197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:17.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.711+0000 7f3072ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3074197a10 0x7f3074197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60642/0 (socket says 172.21.15.71:60642) 2024-04-23T02:55:17.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.711+0000 7f3072ffd700 1 -- 172.21.15.71:0/3063319374 learned_addr learned my addr 172.21.15.71:0/3063319374 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:17.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.711+0000 7f30737fe700 1 --2- 172.21.15.71:0/3063319374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3074100550 0x7f30741974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:17.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.711+0000 7f3072ffd700 1 -- 172.21.15.71:0/3063319374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3074100550 msgr2=0x7f30741974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:17.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.711+0000 7f3072ffd700 1 --2- 172.21.15.71:0/3063319374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3074100550 0x7f30741974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:17.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.711+0000 7f3072ffd700 1 -- 172.21.15.71:0/3063319374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30680095f0 con 0x7f3074197a10 2024-04-23T02:55:17.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.711+0000 7f3072ffd700 1 --2- 172.21.15.71:0/3063319374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3074197a10 0x7f3074197e90 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f306400c960 tx=0x7f306400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:17.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.711+0000 7f3070ff9700 1 -- 172.21.15.71:0/3063319374 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3064007a70 con 0x7f3074197a10 2024-04-23T02:55:17.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.711+0000 7f3070ff9700 1 -- 172.21.15.71:0/3063319374 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3064007bd0 con 0x7f3074197a10 2024-04-23T02:55:17.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.711+0000 7f3079cab700 1 -- 172.21.15.71:0/3063319374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30741a0880 con 0x7f3074197a10 2024-04-23T02:55:17.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.711+0000 7f3070ff9700 1 -- 172.21.15.71:0/3063319374 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3064018630 con 0x7f3074197a10 2024-04-23T02:55:17.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.711+0000 7f3079cab700 1 -- 172.21.15.71:0/3063319374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30741a0db0 con 0x7f3074197a10 2024-04-23T02:55:17.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.712+0000 7f3079cab700 1 -- 172.21.15.71:0/3063319374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3074107f70 con 0x7f3074197a10 2024-04-23T02:55:17.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.713+0000 7f3070ff9700 1 -- 172.21.15.71:0/3063319374 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f306400f450 con 0x7f3074197a10 2024-04-23T02:55:17.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.713+0000 7f3070ff9700 1 --2- 172.21.15.71:0/3063319374 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f305c06cce0 0x7f305c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:17.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.713+0000 7f3070ff9700 1 -- 172.21.15.71:0/3063319374 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f306408c560 con 0x7f3074197a10 2024-04-23T02:55:17.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.713+0000 7f30737fe700 1 --2- 172.21.15.71:0/3063319374 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f305c06cce0 0x7f305c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:17.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.714+0000 7f30737fe700 1 --2- 172.21.15.71:0/3063319374 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f305c06cce0 0x7f305c06f1a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3068006010 tx=0x7f306801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:17.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.718+0000 7f3070ff9700 1 -- 172.21.15.71:0/3063319374 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f30640565d0 con 0x7f3074197a10 2024-04-23T02:55:17.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.912+0000 7f3079cab700 1 -- 172.21.15.71:0/3063319374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f307404ea90 con 0x7f3074197a10 2024-04-23T02:55:17.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.913+0000 7f3070ff9700 1 -- 172.21.15.71:0/3063319374 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f3064059bf0 con 0x7f3074197a10 2024-04-23T02:55:17.915 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-04-23T02:55:17.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.915+0000 7f3079cab700 1 -- 172.21.15.71:0/3063319374 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f305c06cce0 msgr2=0x7f305c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:17.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.915+0000 7f3079cab700 1 --2- 172.21.15.71:0/3063319374 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f305c06cce0 0x7f305c06f1a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3068006010 tx=0x7f306801a040 comp rx=0 tx=0).stop 2024-04-23T02:55:17.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.915+0000 7f3079cab700 1 -- 172.21.15.71:0/3063319374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3074197a10 msgr2=0x7f3074197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:17.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.915+0000 7f3079cab700 1 --2- 172.21.15.71:0/3063319374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3074197a10 0x7f3074197e90 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f306400c960 tx=0x7f306400cc70 comp rx=0 tx=0).stop 2024-04-23T02:55:17.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.915+0000 7f3079cab700 1 -- 172.21.15.71:0/3063319374 shutdown_connections 2024-04-23T02:55:17.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.915+0000 7f3079cab700 1 --2- 172.21.15.71:0/3063319374 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f305c06cce0 0x7f305c06f1a0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:17.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.915+0000 7f3079cab700 1 --2- 172.21.15.71:0/3063319374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3074100550 0x7f30741974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:17.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.915+0000 7f3079cab700 1 --2- 172.21.15.71:0/3063319374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3074197a10 0x7f3074197e90 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:17.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.915+0000 7f3079cab700 1 -- 172.21.15.71:0/3063319374 >> 172.21.15.71:0/3063319374 conn(0x7f3074074c10 msgr2=0x7f30740fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:17.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.915+0000 7f3079cab700 1 -- 172.21.15.71:0/3063319374 shutdown_connections 2024-04-23T02:55:17.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:17.915+0000 7f3079cab700 1 -- 172.21.15.71:0/3063319374 wait complete. 2024-04-23T02:55:17.928 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:17 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1328094372' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T02:55:17.928 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:17 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/720719345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T02:55:17.928 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:17 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/4209218107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-23T02:55:17.929 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T02:55:18.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.050+0000 7f661edd8700 1 -- 172.21.15.71:0/4128664531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66180ffac0 msgr2=0x7f66180fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:18.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.050+0000 7f661edd8700 1 --2- 172.21.15.71:0/4128664531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66180ffac0 0x7f66180fff20 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f6604009910 tx=0x7f6604009c20 comp rx=0 tx=0).stop 2024-04-23T02:55:18.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.050+0000 7f661edd8700 1 -- 172.21.15.71:0/4128664531 shutdown_connections 2024-04-23T02:55:18.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.050+0000 7f661edd8700 1 --2- 172.21.15.71:0/4128664531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66180ffac0 0x7f66180fff20 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:18.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.050+0000 7f661edd8700 1 --2- 172.21.15.71:0/4128664531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6618105990 0x7f6618105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:18.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.050+0000 7f661edd8700 1 -- 172.21.15.71:0/4128664531 >> 172.21.15.71:0/4128664531 conn(0x7f6618075400 msgr2=0x7f6618075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:18.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.050+0000 7f661edd8700 1 -- 172.21.15.71:0/4128664531 shutdown_connections 2024-04-23T02:55:18.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.050+0000 7f661edd8700 1 -- 172.21.15.71:0/4128664531 wait complete. 2024-04-23T02:55:18.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.051+0000 7f661edd8700 1 Processor -- start 2024-04-23T02:55:18.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.051+0000 7f661edd8700 1 -- start start 2024-04-23T02:55:18.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.051+0000 7f661edd8700 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66180ffac0 0x7f66181953c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:18.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.051+0000 7f661edd8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6618105990 0x7f6618195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:18.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.051+0000 7f661edd8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6618195fe0 con 0x7f6618105990 2024-04-23T02:55:18.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.051+0000 7f661edd8700 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6618199d60 con 0x7f66180ffac0 2024-04-23T02:55:18.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.051+0000 7f6617fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6618105990 0x7f6618195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:18.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.051+0000 7f6617fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6618105990 0x7f6618195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60658/0 (socket says 172.21.15.71:60658) 2024-04-23T02:55:18.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.051+0000 7f6617fff700 1 -- 172.21.15.71:0/1964184729 learned_addr learned my addr 172.21.15.71:0/1964184729 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-23T02:55:18.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.051+0000 7f661cb74700 1 --2- 172.21.15.71:0/1964184729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66180ffac0 0x7f66181953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:18.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.051+0000 7f6617fff700 1 -- 172.21.15.71:0/1964184729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66180ffac0 msgr2=0x7f66181953c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:18.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.051+0000 7f6617fff700 1 --2- 172.21.15.71:0/1964184729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66180ffac0 0x7f66181953c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:18.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.051+0000 7f6617fff700 1 -- 172.21.15.71:0/1964184729 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66040095f0 con 0x7f6618105990 2024-04-23T02:55:18.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.051+0000 7f661cb74700 1 --2- 172.21.15.71:0/1964184729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66180ffac0 0x7f66181953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T02:55:18.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.051+0000 7f6617fff700 1 --2- 172.21.15.71:0/1964184729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6618105990 0x7f6618195900 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f66040092e0 tx=0x7f660400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:18.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.051+0000 7f6615ffb700 1 -- 172.21.15.71:0/1964184729 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f660401d070 con 0x7f6618105990 2024-04-23T02:55:18.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.052+0000 7f6615ffb700 1 -- 172.21.15.71:0/1964184729 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6604005e00 con 0x7f6618105990 2024-04-23T02:55:18.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.052+0000 7f661edd8700 1 -- 172.21.15.71:0/1964184729 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6618199fe0 con 0x7f6618105990 2024-04-23T02:55:18.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.052+0000 7f6615ffb700 1 -- 172.21.15.71:0/1964184729 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f660400fe60 con 0x7f6618105990 2024-04-23T02:55:18.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.052+0000 7f661edd8700 1 -- 172.21.15.71:0/1964184729 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661819a4e0 con 0x7f6618105990 2024-04-23T02:55:18.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.053+0000 7f661edd8700 1 -- 172.21.15.71:0/1964184729 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6618107f10 con 0x7f6618105990 2024-04-23T02:55:18.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.053+0000 7f6615ffb700 1 -- 172.21.15.71:0/1964184729 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 32) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f660400f6d0 con 0x7f6618105990 2024-04-23T02:55:18.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.054+0000 7f6615ffb700 1 --2- 172.21.15.71:0/1964184729 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f660806cda0 0x7f660806f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T02:55:18.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.054+0000 7f6615ffb700 1 -- 172.21.15.71:0/1964184729 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f660408d8b0 con 0x7f6618105990 2024-04-23T02:55:18.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.054+0000 7f661cb74700 1 --2- 172.21.15.71:0/1964184729 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f660806cda0 0x7f660806f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T02:55:18.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.054+0000 7f661cb74700 1 --2- 172.21.15.71:0/1964184729 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f660806cda0 0x7f660806f260 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f660c005950 tx=0x7f660c0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T02:55:18.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.058+0000 7f6615ffb700 1 -- 172.21.15.71:0/1964184729 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6604057920 con 0x7f6618105990 2024-04-23T02:55:18.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:17 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1328094372' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T02:55:18.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:17 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/720719345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T02:55:18.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:17 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/4209218107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-23T02:55:18.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.269+0000 7f661edd8700 1 -- 172.21.15.71:0/1964184729 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f661804ea90 con 0x7f6618105990 2024-04-23T02:55:18.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.269+0000 7f6615ffb700 1 -- 172.21.15.71:0/1964184729 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 35 entries v84) v1 ==== 81+0+2077 (secure 0 0 0) 0x7f660405af40 con 0x7f6618105990 2024-04-23T02:55:18.272 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6828/467545394 2024-04-24T02:22:00.280478+0000 2024-04-23T02:55:18.272 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/3794652670 2024-04-24T02:03:48.178880+0000 2024-04-23T02:55:18.272 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/4165305842 2024-04-24T02:05:42.095805+0000 2024-04-23T02:55:18.272 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/56041968 2024-04-24T02:05:42.095805+0000 2024-04-23T02:55:18.272 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:0/1753179829 2024-04-24T02:14:34.629978+0000 2024-04-23T02:55:18.272 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/4165305842 2024-04-24T02:05:42.095805+0000 2024-04-23T02:55:18.272 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:0/2569976755 2024-04-24T02:14:34.629978+0000 2024-04-23T02:55:18.272 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/823266478 2024-04-24T02:04:23.247815+0000 2024-04-23T02:55:18.272 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:6828/966231696 2024-04-24T02:14:34.629978+0000 2024-04-23T02:55:18.272 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:6824/3780549896 2024-04-24T02:22:00.280491+0000 2024-04-23T02:55:18.272 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:0/1701960325 2024-04-24T02:14:34.629978+0000 2024-04-23T02:55:18.273 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/2225989508 2024-04-24T02:03:48.178880+0000 2024-04-23T02:55:18.273 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6829/467545394 2024-04-24T02:22:00.280478+0000 2024-04-23T02:55:18.273 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:0/2687070241 2024-04-24T02:14:34.629978+0000 2024-04-23T02:55:18.273 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/17588705 2024-04-24T02:13:11.575106+0000 2024-04-23T02:55:18.273 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3579793925 2024-04-24T02:04:23.247815+0000 2024-04-23T02:55:18.273 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/1995129374 2024-04-24T02:03:48.178880+0000 2024-04-23T02:55:18.273 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/294030681 2024-04-24T02:04:23.247815+0000 2024-04-23T02:55:18.273 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/936344738 2024-04-24T02:05:42.095805+0000 2024-04-23T02:55:18.273 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:6829/966231696 2024-04-24T02:14:34.629978+0000 2024-04-23T02:55:18.273 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:0/2727292170 2024-04-24T02:14:34.629978+0000 2024-04-23T02:55:18.273 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/1815873101 2024-04-24T02:13:11.575106+0000 2024-04-23T02:55:18.273 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:6825/3780549896 2024-04-24T02:22:00.280491+0000 2024-04-23T02:55:18.273 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/3878148961 2024-04-24T02:04:23.247815+0000 2024-04-23T02:55:18.273 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6827/3504670324 2024-04-24T02:11:05.663451+0000 2024-04-23T02:55:18.274 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.192:0/2877439214 2024-04-24T02:14:34.629978+0000 2024-04-23T02:55:18.274 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/4249907286 2024-04-24T02:13:11.575106+0000 2024-04-23T02:55:18.274 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/3878148961 2024-04-24T02:04:23.247815+0000 2024-04-23T02:55:18.274 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/3794652670 2024-04-24T02:03:48.178880+0000 2024-04-23T02:55:18.274 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/4205332842 2024-04-24T02:03:48.178880+0000 2024-04-23T02:55:18.274 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3618236204 2024-04-24T02:13:11.575106+0000 2024-04-23T02:55:18.274 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3473332587 2024-04-24T02:13:11.575106+0000 2024-04-23T02:55:18.274 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/2206241071 2024-04-24T02:05:42.095805+0000 2024-04-23T02:55:18.274 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/4249907286 2024-04-24T02:13:11.575106+0000 2024-04-23T02:55:18.274 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6826/3504670324 2024-04-24T02:11:05.663451+0000 2024-04-23T02:55:18.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.271+0000 7f661edd8700 1 -- 172.21.15.71:0/1964184729 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f660806cda0 msgr2=0x7f660806f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:18.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.271+0000 7f661edd8700 1 --2- 172.21.15.71:0/1964184729 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f660806cda0 0x7f660806f260 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f660c005950 tx=0x7f660c0058e0 comp rx=0 tx=0).stop 2024-04-23T02:55:18.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.271+0000 7f661edd8700 1 -- 172.21.15.71:0/1964184729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6618105990 msgr2=0x7f6618195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T02:55:18.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.271+0000 7f661edd8700 1 --2- 172.21.15.71:0/1964184729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6618105990 0x7f6618195900 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f66040092e0 tx=0x7f660400be20 comp rx=0 tx=0).stop 2024-04-23T02:55:18.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.272+0000 7f661edd8700 1 -- 172.21.15.71:0/1964184729 shutdown_connections 2024-04-23T02:55:18.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.272+0000 7f661edd8700 1 --2- 172.21.15.71:0/1964184729 >> [v2:172.21.15.71:6800/3011237878,v1:172.21.15.71:6801/3011237878] conn(0x7f660806cda0 0x7f660806f260 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:18.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.272+0000 7f661edd8700 1 --2- 172.21.15.71:0/1964184729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66180ffac0 0x7f66181953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:18.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.272+0000 7f661edd8700 1 --2- 172.21.15.71:0/1964184729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6618105990 0x7f6618195900 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T02:55:18.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.272+0000 7f661edd8700 1 -- 172.21.15.71:0/1964184729 >> 172.21.15.71:0/1964184729 conn(0x7f6618075400 msgr2=0x7f66180fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T02:55:18.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.272+0000 7f661edd8700 1 -- 172.21.15.71:0/1964184729 shutdown_connections 2024-04-23T02:55:18.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-23T02:55:18.272+0000 7f661edd8700 1 -- 172.21.15.71:0/1964184729 wait complete. 2024-04-23T02:55:18.276 INFO:teuthology.orchestra.run.smithi071.stderr:listed 35 entries 2024-04-23T02:55:18.286 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi071.front.sepia.ceph.com... 2024-04-23T02:55:18.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T02:55:18.286 DEBUG:teuthology.orchestra.run.smithi071:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-23T02:55:18.318 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T02:55:18.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:18 smithi071 ceph-mon[144544]: pgmap v1262: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:18.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:18 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/3063319374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-23T02:55:18.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:18 smithi071 ceph-mon[144544]: from='client.? 172.21.15.71:0/1964184729' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T02:55:19.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:18 smithi192 ceph-mon[108617]: pgmap v1262: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:19.141 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:18 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/3063319374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-23T02:55:19.141 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:18 smithi192 ceph-mon[108617]: from='client.? 172.21.15.71:0/1964184729' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T02:55:20.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:20 smithi071 ceph-mon[144544]: pgmap v1263: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:55:20.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:55:21.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:20 smithi192 ceph-mon[108617]: pgmap v1263: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:55:21.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:55:22.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:22 smithi071 ceph-mon[144544]: pgmap v1264: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:23.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:22 smithi192 ceph-mon[108617]: pgmap v1264: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:24.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:24 smithi071 ceph-mon[144544]: pgmap v1265: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:25.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:24 smithi192 ceph-mon[108617]: pgmap v1265: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:26.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:26 smithi071 ceph-mon[144544]: pgmap v1266: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:55:27.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:26 smithi192 ceph-mon[108617]: pgmap v1266: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:55:27.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:27 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:55:28.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:27 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:55:28.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:28 smithi071 ceph-mon[144544]: pgmap v1267: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:28.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:28 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:55:28.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:28 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:55:28.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:28 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:55:29.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:28 smithi192 ceph-mon[108617]: pgmap v1267: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:29.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:28 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:55:29.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:28 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:55:29.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:28 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:55:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:30 smithi071 ceph-mon[144544]: pgmap v1268: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:55:31.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:30 smithi192 ceph-mon[108617]: pgmap v1268: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:55:32.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:32 smithi071 ceph-mon[144544]: pgmap v1269: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:33.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:32 smithi192 ceph-mon[108617]: pgmap v1269: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:34.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:33 smithi192 ceph-mon[108617]: pgmap v1270: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:34.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:33 smithi071 ceph-mon[144544]: pgmap v1270: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:35.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:34 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:55:35.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:34 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:55:36.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:35 smithi192 ceph-mon[108617]: pgmap v1271: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:55:36.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:35 smithi071 ceph-mon[144544]: pgmap v1271: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:55:38.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:38 smithi071 ceph-mon[144544]: pgmap v1272: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:38.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:38 smithi192 ceph-mon[108617]: pgmap v1272: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:40.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:40 smithi071 ceph-mon[144544]: pgmap v1273: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:55:40.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:40 smithi192 ceph-mon[108617]: pgmap v1273: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:55:42.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:42 smithi192 ceph-mon[108617]: pgmap v1274: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:42.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:42 smithi071 ceph-mon[144544]: pgmap v1274: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:44.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:44 smithi192 ceph-mon[108617]: pgmap v1275: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:44.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:44 smithi071 ceph-mon[144544]: pgmap v1275: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:46.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:46 smithi192 ceph-mon[108617]: pgmap v1276: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:55:46.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:46 smithi071 ceph-mon[144544]: pgmap v1276: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:55:48.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:48 smithi192 ceph-mon[108617]: pgmap v1277: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:48.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:48 smithi071 ceph-mon[144544]: pgmap v1277: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:50 smithi192 ceph-mon[108617]: pgmap v1278: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:55:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:55:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:50 smithi071 ceph-mon[144544]: pgmap v1278: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:55:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:55:52.827 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:52 smithi071 ceph-mon[144544]: pgmap v1279: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:52.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:52 smithi192 ceph-mon[108617]: pgmap v1279: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:54.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:54 smithi192 ceph-mon[108617]: pgmap v1280: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:54.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:54 smithi071 ceph-mon[144544]: pgmap v1280: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:56.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:56 smithi192 ceph-mon[108617]: pgmap v1281: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:55:56.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:56 smithi071 ceph-mon[144544]: pgmap v1281: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:55:58.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:55:58 smithi192 ceph-mon[108617]: pgmap v1282: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:55:58.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:55:58 smithi071 ceph-mon[144544]: pgmap v1282: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:00.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:00 smithi192 ceph-mon[108617]: pgmap v1283: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:56:00.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:00 smithi071 ceph-mon[144544]: pgmap v1283: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:56:02.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:02 smithi192 ceph-mon[108617]: pgmap v1284: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:02.940 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:02 smithi071 ceph-mon[144544]: pgmap v1284: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:04.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:04 smithi192 ceph-mon[108617]: pgmap v1285: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:04.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:04 smithi071 ceph-mon[144544]: pgmap v1285: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:56:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:56:06.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:06 smithi192 ceph-mon[108617]: pgmap v1286: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:56:06.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:06 smithi071 ceph-mon[144544]: pgmap v1286: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:56:08.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:08 smithi192 ceph-mon[108617]: pgmap v1287: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:08.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:08 smithi071 ceph-mon[144544]: pgmap v1287: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:10.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:10 smithi192 ceph-mon[108617]: pgmap v1288: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:56:10.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:10 smithi071 ceph-mon[144544]: pgmap v1288: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:56:12.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:12 smithi192 ceph-mon[108617]: pgmap v1289: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:12.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:12 smithi071 ceph-mon[144544]: pgmap v1289: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:14.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:14 smithi192 ceph-mon[108617]: pgmap v1290: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:14.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:14 smithi071 ceph-mon[144544]: pgmap v1290: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:16.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:16 smithi192 ceph-mon[108617]: pgmap v1291: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:56:16.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:16 smithi071 ceph-mon[144544]: pgmap v1291: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:56:18.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:18 smithi192 ceph-mon[108617]: pgmap v1292: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:18.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:18 smithi071 ceph-mon[144544]: pgmap v1292: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:20.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:20 smithi192 ceph-mon[108617]: pgmap v1293: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:56:20.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:56:20.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:20 smithi071 ceph-mon[144544]: pgmap v1293: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:56:20.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:56:22.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:22 smithi192 ceph-mon[108617]: pgmap v1294: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:22.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:22 smithi071 ceph-mon[144544]: pgmap v1294: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:24.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:24 smithi192 ceph-mon[108617]: pgmap v1295: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:24.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:24 smithi071 ceph-mon[144544]: pgmap v1295: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:26.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:26 smithi192 ceph-mon[108617]: pgmap v1296: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:56:26.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:26 smithi071 ceph-mon[144544]: pgmap v1296: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:56:28.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:28 smithi192 ceph-mon[108617]: pgmap v1297: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:28.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:28 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:56:28.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:28 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:56:28.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:28 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:56:28.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:28 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:56:28.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:28 smithi071 ceph-mon[144544]: pgmap v1297: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:28.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:28 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:56:28.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:28 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:56:28.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:28 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:56:28.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:28 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:56:30.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:30 smithi192 ceph-mon[108617]: pgmap v1298: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:56:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:30 smithi071 ceph-mon[144544]: pgmap v1298: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:56:32.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:32 smithi192 ceph-mon[108617]: pgmap v1299: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:32.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:32 smithi071 ceph-mon[144544]: pgmap v1299: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:34.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:34 smithi192 ceph-mon[108617]: pgmap v1300: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:34.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:34 smithi071 ceph-mon[144544]: pgmap v1300: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:35.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:56:35.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:56:36.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:36 smithi192 ceph-mon[108617]: pgmap v1301: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:56:36.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:36 smithi071 ceph-mon[144544]: pgmap v1301: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:56:38.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:38 smithi192 ceph-mon[108617]: pgmap v1302: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:38.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:38 smithi071 ceph-mon[144544]: pgmap v1302: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:40.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:40 smithi192 ceph-mon[108617]: pgmap v1303: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:56:40.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:40 smithi071 ceph-mon[144544]: pgmap v1303: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:56:42.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:42 smithi192 ceph-mon[108617]: pgmap v1304: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:42.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:42 smithi071 ceph-mon[144544]: pgmap v1304: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:44.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:44 smithi192 ceph-mon[108617]: pgmap v1305: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:44.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:44 smithi071 ceph-mon[144544]: pgmap v1305: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:46.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:46 smithi192 ceph-mon[108617]: pgmap v1306: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:56:46.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:46 smithi071 ceph-mon[144544]: pgmap v1306: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:56:48.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:48 smithi071 ceph-mon[144544]: pgmap v1307: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:48.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:48 smithi192 ceph-mon[108617]: pgmap v1307: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:50 smithi192 ceph-mon[108617]: pgmap v1308: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:56:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:56:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:50 smithi071 ceph-mon[144544]: pgmap v1308: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:56:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:56:52.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:52 smithi192 ceph-mon[108617]: pgmap v1309: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:52.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:52 smithi071 ceph-mon[144544]: pgmap v1309: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:54.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:54 smithi192 ceph-mon[108617]: pgmap v1310: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:54.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:54 smithi071 ceph-mon[144544]: pgmap v1310: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:56:56.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:56 smithi192 ceph-mon[108617]: pgmap v1311: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:56:56.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:56 smithi071 ceph-mon[144544]: pgmap v1311: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:56:58.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:56:58 smithi192 ceph-mon[108617]: pgmap v1312: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-04-23T02:56:58.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:56:58 smithi071 ceph-mon[144544]: pgmap v1312: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-04-23T02:57:00.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:00 smithi192 ceph-mon[108617]: pgmap v1313: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-23T02:57:00.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:00 smithi071 ceph-mon[144544]: pgmap v1313: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-23T02:57:02.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:02 smithi071 ceph-mon[144544]: pgmap v1314: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-04-23T02:57:03.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:02 smithi192 ceph-mon[108617]: pgmap v1314: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-04-23T02:57:04.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:04 smithi071 ceph-mon[144544]: pgmap v1315: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-04-23T02:57:05.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:04 smithi192 ceph-mon[108617]: pgmap v1315: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-04-23T02:57:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:57:06.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:57:06.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:06 smithi071 ceph-mon[144544]: pgmap v1316: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-04-23T02:57:07.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:06 smithi192 ceph-mon[108617]: pgmap v1316: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-04-23T02:57:08.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:08 smithi071 ceph-mon[144544]: pgmap v1317: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-04-23T02:57:09.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:08 smithi192 ceph-mon[108617]: pgmap v1317: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-04-23T02:57:10.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:10 smithi071 ceph-mon[144544]: pgmap v1318: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:57:11.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:10 smithi192 ceph-mon[108617]: pgmap v1318: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:57:12.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:12 smithi071 ceph-mon[144544]: pgmap v1319: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:13.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:12 smithi192 ceph-mon[108617]: pgmap v1319: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:14.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:14 smithi071 ceph-mon[144544]: pgmap v1320: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:15.009 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:14 smithi192 ceph-mon[108617]: pgmap v1320: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:16.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:16 smithi071 ceph-mon[144544]: pgmap v1321: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:57:17.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:16 smithi192 ceph-mon[108617]: pgmap v1321: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:57:18.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:18 smithi071 ceph-mon[144544]: pgmap v1322: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:19.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:18 smithi192 ceph-mon[108617]: pgmap v1322: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:20.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:20 smithi071 ceph-mon[144544]: pgmap v1323: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:57:20.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:57:21.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:20 smithi192 ceph-mon[108617]: pgmap v1323: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:57:21.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:57:22.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:22 smithi071 ceph-mon[144544]: pgmap v1324: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:23.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:22 smithi192 ceph-mon[108617]: pgmap v1324: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:24.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:24 smithi071 ceph-mon[144544]: pgmap v1325: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:25.117 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:24 smithi192 ceph-mon[108617]: pgmap v1325: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:26.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:26 smithi071 ceph-mon[144544]: pgmap v1326: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:57:27.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:26 smithi192 ceph-mon[108617]: pgmap v1326: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:57:28.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:28 smithi071 ceph-mon[144544]: pgmap v1327: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:28.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:28 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:57:29.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:28 smithi192 ceph-mon[108617]: pgmap v1327: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:29.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:28 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:57:29.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:29 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:57:29.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:29 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:57:29.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:29 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:57:30.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:29 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:57:30.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:29 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:57:30.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:29 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:57:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:30 smithi071 ceph-mon[144544]: pgmap v1328: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:57:31.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:30 smithi192 ceph-mon[108617]: pgmap v1328: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:57:32.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:32 smithi071 ceph-mon[144544]: pgmap v1329: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:33.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:32 smithi192 ceph-mon[108617]: pgmap v1329: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:34.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:34 smithi071 ceph-mon[144544]: pgmap v1330: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:35.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:34 smithi192 ceph-mon[108617]: pgmap v1330: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:35.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:57:36.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:57:36.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:36 smithi071 ceph-mon[144544]: pgmap v1331: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:57:37.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:36 smithi192 ceph-mon[108617]: pgmap v1331: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:57:38.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:38 smithi071 ceph-mon[144544]: pgmap v1332: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:39.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:38 smithi192 ceph-mon[108617]: pgmap v1332: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:40.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:40 smithi071 ceph-mon[144544]: pgmap v1333: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:57:41.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:40 smithi192 ceph-mon[108617]: pgmap v1333: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:57:42.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:42 smithi071 ceph-mon[144544]: pgmap v1334: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:43.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:42 smithi192 ceph-mon[108617]: pgmap v1334: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:44.051 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:43 smithi071 ceph-mon[144544]: pgmap v1335: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:44.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:43 smithi192 ceph-mon[108617]: pgmap v1335: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:46.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:46 smithi192 ceph-mon[108617]: pgmap v1336: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:57:46.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:46 smithi071 ceph-mon[144544]: pgmap v1336: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:57:48.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:48 smithi192 ceph-mon[108617]: pgmap v1337: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:48.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:48 smithi071 ceph-mon[144544]: pgmap v1337: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:50 smithi192 ceph-mon[108617]: pgmap v1338: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:57:50.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:57:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:50 smithi071 ceph-mon[144544]: pgmap v1338: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:57:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:57:52.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:52 smithi192 ceph-mon[108617]: pgmap v1339: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:52.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:52 smithi071 ceph-mon[144544]: pgmap v1339: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:54.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:54 smithi192 ceph-mon[108617]: pgmap v1340: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:54.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:54 smithi071 ceph-mon[144544]: pgmap v1340: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:56.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:56 smithi192 ceph-mon[108617]: pgmap v1341: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:57:56.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:56 smithi071 ceph-mon[144544]: pgmap v1341: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:57:58.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:57:58 smithi192 ceph-mon[108617]: pgmap v1342: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:57:58.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:57:58 smithi071 ceph-mon[144544]: pgmap v1342: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:00.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:00 smithi192 ceph-mon[108617]: pgmap v1343: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:58:00.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:00 smithi071 ceph-mon[144544]: pgmap v1343: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:58:02.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:02 smithi192 ceph-mon[108617]: pgmap v1344: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:02.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:02 smithi071 ceph-mon[144544]: pgmap v1344: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:04.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:04 smithi192 ceph-mon[108617]: pgmap v1345: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:04.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:04 smithi071 ceph-mon[144544]: pgmap v1345: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:58:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:58:06.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:06 smithi192 ceph-mon[108617]: pgmap v1346: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:58:06.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:06 smithi071 ceph-mon[144544]: pgmap v1346: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:58:08.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:08 smithi192 ceph-mon[108617]: pgmap v1347: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:08.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:08 smithi071 ceph-mon[144544]: pgmap v1347: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:10.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:10 smithi192 ceph-mon[108617]: pgmap v1348: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:58:10.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:10 smithi071 ceph-mon[144544]: pgmap v1348: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:58:12.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:12 smithi192 ceph-mon[108617]: pgmap v1349: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:12.993 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:12 smithi071 ceph-mon[144544]: pgmap v1349: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:14.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:14 smithi071 ceph-mon[144544]: pgmap v1350: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:14.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:14 smithi192 ceph-mon[108617]: pgmap v1350: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:16.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:16 smithi192 ceph-mon[108617]: pgmap v1351: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:58:16.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:16 smithi071 ceph-mon[144544]: pgmap v1351: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:58:18.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:18 smithi071 ceph-mon[144544]: pgmap v1352: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:19.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:18 smithi192 ceph-mon[108617]: pgmap v1352: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:20.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:20 smithi071 ceph-mon[144544]: pgmap v1353: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:58:20.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:58:21.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:20 smithi192 ceph-mon[108617]: pgmap v1353: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:58:21.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:58:22.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:22 smithi071 ceph-mon[144544]: pgmap v1354: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:23.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:22 smithi192 ceph-mon[108617]: pgmap v1354: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:24.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:24 smithi071 ceph-mon[144544]: pgmap v1355: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:25.124 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:24 smithi192 ceph-mon[108617]: pgmap v1355: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:26.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:26 smithi071 ceph-mon[144544]: pgmap v1356: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:58:27.124 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:26 smithi192 ceph-mon[108617]: pgmap v1356: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:58:28.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:28 smithi071 ceph-mon[144544]: pgmap v1357: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:29.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:28 smithi192 ceph-mon[108617]: pgmap v1357: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:29.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:29 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:58:30.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:29 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:58:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:30 smithi071 ceph-mon[144544]: pgmap v1358: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:58:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:30 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:58:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:30 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:58:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:30 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:58:31.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:30 smithi192 ceph-mon[108617]: pgmap v1358: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:58:31.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:30 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:58:31.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:30 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:58:31.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:30 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:58:32.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:32 smithi071 ceph-mon[144544]: pgmap v1359: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:33.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:32 smithi192 ceph-mon[108617]: pgmap v1359: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:34.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:34 smithi071 ceph-mon[144544]: pgmap v1360: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:35.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:34 smithi192 ceph-mon[108617]: pgmap v1360: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:35.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:58:36.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:58:36.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:36 smithi071 ceph-mon[144544]: pgmap v1361: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:58:37.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:36 smithi192 ceph-mon[108617]: pgmap v1361: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:58:38.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:38 smithi071 ceph-mon[144544]: pgmap v1362: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:39.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:38 smithi192 ceph-mon[108617]: pgmap v1362: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:40.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:40 smithi071 ceph-mon[144544]: pgmap v1363: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:58:41.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:40 smithi192 ceph-mon[108617]: pgmap v1363: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:58:42.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:42 smithi071 ceph-mon[144544]: pgmap v1364: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:43.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:42 smithi192 ceph-mon[108617]: pgmap v1364: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:44.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:44 smithi071 ceph-mon[144544]: pgmap v1365: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:45.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:44 smithi192 ceph-mon[108617]: pgmap v1365: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:46.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:46 smithi071 ceph-mon[144544]: pgmap v1366: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:58:47.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:46 smithi192 ceph-mon[108617]: pgmap v1366: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:58:48.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:48 smithi071 ceph-mon[144544]: pgmap v1367: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:49.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:48 smithi192 ceph-mon[108617]: pgmap v1367: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:50 smithi071 ceph-mon[144544]: pgmap v1368: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:58:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:58:51.025 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:50 smithi192 ceph-mon[108617]: pgmap v1368: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:58:51.025 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:58:52.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:52 smithi071 ceph-mon[144544]: pgmap v1369: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:53.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:52 smithi192 ceph-mon[108617]: pgmap v1369: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:54.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:54 smithi071 ceph-mon[144544]: pgmap v1370: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:55.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:54 smithi192 ceph-mon[108617]: pgmap v1370: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:56.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:56 smithi071 ceph-mon[144544]: pgmap v1371: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:58:57.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:56 smithi192 ceph-mon[108617]: pgmap v1371: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:58:58.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:58:58 smithi071 ceph-mon[144544]: pgmap v1372: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:58:59.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:58:58 smithi192 ceph-mon[108617]: pgmap v1372: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:00.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:00 smithi071 ceph-mon[144544]: pgmap v1373: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:59:01.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:00 smithi192 ceph-mon[108617]: pgmap v1373: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:59:02.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:01 smithi192 ceph-mon[108617]: pgmap v1374: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:02.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:01 smithi071 ceph-mon[144544]: pgmap v1374: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:04.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:04 smithi192 ceph-mon[108617]: pgmap v1375: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:04.941 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:04 smithi071 ceph-mon[144544]: pgmap v1375: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:05.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:59:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:59:06.874 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:06 smithi192 ceph-mon[108617]: pgmap v1376: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:59:06.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:06 smithi071 ceph-mon[144544]: pgmap v1376: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:59:08.874 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:08 smithi192 ceph-mon[108617]: pgmap v1377: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:08.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:08 smithi071 ceph-mon[144544]: pgmap v1377: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:10.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:10 smithi192 ceph-mon[108617]: pgmap v1378: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:59:10.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:10 smithi071 ceph-mon[144544]: pgmap v1378: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:59:12.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:12 smithi192 ceph-mon[108617]: pgmap v1379: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:12.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:12 smithi071 ceph-mon[144544]: pgmap v1379: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:14.878 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:14 smithi192 ceph-mon[108617]: pgmap v1380: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:14.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:14 smithi071 ceph-mon[144544]: pgmap v1380: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:16.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:16 smithi192 ceph-mon[108617]: pgmap v1381: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:59:16.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:16 smithi071 ceph-mon[144544]: pgmap v1381: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:59:18.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:18 smithi071 ceph-mon[144544]: pgmap v1382: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:19.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:18 smithi192 ceph-mon[108617]: pgmap v1382: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:20.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:20 smithi071 ceph-mon[144544]: pgmap v1383: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:59:20.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:20 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:59:21.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:20 smithi192 ceph-mon[108617]: pgmap v1383: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:59:21.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:20 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:59:22.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:22 smithi071 ceph-mon[144544]: pgmap v1384: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:23.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:22 smithi192 ceph-mon[108617]: pgmap v1384: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:24.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:24 smithi071 ceph-mon[144544]: pgmap v1385: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:25.124 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:24 smithi192 ceph-mon[108617]: pgmap v1385: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:26.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:26 smithi071 ceph-mon[144544]: pgmap v1386: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:59:27.124 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:26 smithi192 ceph-mon[108617]: pgmap v1386: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:59:28.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:28 smithi071 ceph-mon[144544]: pgmap v1387: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:29.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:28 smithi192 ceph-mon[108617]: pgmap v1387: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:30 smithi071 ceph-mon[144544]: pgmap v1388: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:59:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:30 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:59:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:30 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:59:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:30 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:59:30.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:30 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:59:31.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:30 smithi192 ceph-mon[108617]: pgmap v1388: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:59:31.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:30 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T02:59:31.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:30 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T02:59:31.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:30 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T02:59:31.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:30 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' 2024-04-23T02:59:32.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:32 smithi071 ceph-mon[144544]: pgmap v1389: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:33.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:32 smithi192 ceph-mon[108617]: pgmap v1389: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:34.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:34 smithi071 ceph-mon[144544]: pgmap v1390: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:35.124 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:34 smithi192 ceph-mon[108617]: pgmap v1390: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:35.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:35 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:59:36.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:35 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:59:36.874 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:36 smithi192 ceph-mon[108617]: pgmap v1391: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:59:36.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:36 smithi071 ceph-mon[144544]: pgmap v1391: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:59:38.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:38 smithi071 ceph-mon[144544]: pgmap v1392: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:39.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:38 smithi192 ceph-mon[108617]: pgmap v1392: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:40.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:40 smithi071 ceph-mon[144544]: pgmap v1393: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:59:41.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:40 smithi192 ceph-mon[108617]: pgmap v1393: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:59:42.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:42 smithi071 ceph-mon[144544]: pgmap v1394: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:43.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:42 smithi192 ceph-mon[108617]: pgmap v1394: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:44.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:44 smithi071 ceph-mon[144544]: pgmap v1395: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:45.124 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:44 smithi192 ceph-mon[108617]: pgmap v1395: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:46.874 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:46 smithi192 ceph-mon[108617]: pgmap v1396: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:59:46.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:46 smithi071 ceph-mon[144544]: pgmap v1396: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:59:48.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:48 smithi071 ceph-mon[144544]: pgmap v1397: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:49.124 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:48 smithi192 ceph-mon[108617]: pgmap v1397: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:50 smithi071 ceph-mon[144544]: pgmap v1398: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:59:50.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:50 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:59:51.124 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:50 smithi192 ceph-mon[108617]: pgmap v1398: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T02:59:51.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:50 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T02:59:52.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:52 smithi071 ceph-mon[144544]: pgmap v1399: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:53.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:52 smithi192 ceph-mon[108617]: pgmap v1399: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:54.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:54 smithi071 ceph-mon[144544]: pgmap v1400: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:55.124 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:54 smithi192 ceph-mon[108617]: pgmap v1400: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:56.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:56 smithi071 ceph-mon[144544]: pgmap v1401: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:59:57.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:56 smithi192 ceph-mon[108617]: pgmap v1401: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T02:59:58.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 02:59:57 smithi192 ceph-mon[108617]: pgmap v1402: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T02:59:58.242 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 02:59:57 smithi071 ceph-mon[144544]: pgmap v1402: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T03:00:00.874 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 03:00:00 smithi192 ceph-mon[108617]: pgmap v1403: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T03:00:00.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 03:00:00 smithi192 ceph-mon[108617]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-23T03:00:00.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 03:00:00 smithi192 ceph-mon[108617]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T03:00:00.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 03:00:00 smithi192 ceph-mon[108617]: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T03:00:00.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 03:00:00 smithi071 ceph-mon[144544]: pgmap v1403: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T03:00:00.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 03:00:00 smithi071 ceph-mon[144544]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-23T03:00:00.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 03:00:00 smithi071 ceph-mon[144544]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-23T03:00:00.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 03:00:00 smithi071 ceph-mon[144544]: fs cephfs has deprecated feature inline_data enabled. 2024-04-23T03:00:02.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 03:00:02 smithi192 ceph-mon[108617]: pgmap v1404: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T03:00:02.991 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 03:00:02 smithi071 ceph-mon[144544]: pgmap v1404: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T03:00:04.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 03:00:04 smithi192 ceph-mon[108617]: pgmap v1405: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T03:00:04.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 03:00:04 smithi071 ceph-mon[144544]: pgmap v1405: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T03:00:05.874 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 03:00:05 smithi192 ceph-mon[108617]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T03:00:05.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 03:00:05 smithi071 ceph-mon[144544]: from='mgr.14660 172.21.15.71:0/715754414' entity='mgr.smithi071.ukdjvn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T03:00:06.833 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 03:00:06 smithi192 ceph-mon[108617]: pgmap v1406: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T03:00:06.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 03:00:06 smithi071 ceph-mon[144544]: pgmap v1406: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T03:00:08.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 03:00:08 smithi192 ceph-mon[108617]: pgmap v1407: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T03:00:08.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 03:00:08 smithi071 ceph-mon[144544]: pgmap v1407: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T03:00:10.874 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 03:00:10 smithi192 ceph-mon[108617]: pgmap v1408: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T03:00:10.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 03:00:10 smithi071 ceph-mon[144544]: pgmap v1408: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-23T03:00:12.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 03:00:12 smithi192 ceph-mon[108617]: pgmap v1409: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T03:00:12.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 03:00:12 smithi071 ceph-mon[144544]: pgmap v1409: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T03:00:14.875 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 03:00:14 smithi192 ceph-mon[108617]: pgmap v1410: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T03:00:14.992 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 03:00:14 smithi071 ceph-mon[144544]: pgmap v1410: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-23T03:00:16.874 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 03:00:16 smithi192 ceph-mon[108617]: pgmap v1411: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T03:00:16.991 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 03:00:16 smithi071 ceph-mon[144544]: pgmap v1411: 65 pgs: 65 active+clean; 314 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-23T03:00:17.394 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-04-23T03:00:17.395 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_f26c9c3fee57ca330501910de5a07c8769ef5dfc/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_f26c9c3fee57ca330501910de5a07c8769ef5dfc/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-23T03:00:17.396 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-23T03:00:17.407 INFO:tasks.cephadm:Teardown begin 2024-04-23T03:00:17.408 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_f26c9c3fee57ca330501910de5a07c8769ef5dfc/qa/tasks/cephadm.py", line 1750, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_f26c9c3fee57ca330501910de5a07c8769ef5dfc/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_f26c9c3fee57ca330501910de5a07c8769ef5dfc/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-23T03:00:17.409 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T03:00:17.449 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T03:00:17.489 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-23T03:00:17.489 DEBUG:teuthology.orchestra.run.smithi071:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T03:00:17.510 DEBUG:teuthology.orchestra.run.smithi192:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T03:00:17.550 INFO:tasks.cephadm:Stopping all daemons... 2024-04-23T03:00:17.551 INFO:tasks.cephadm.mon.smithi071:Stopping mon.smithi071... 2024-04-23T03:00:17.551 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-91f5470c-0115-11ef-bc93-c7b262605968@mon.smithi071 2024-04-23T03:00:17.742 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 03:00:17 smithi071 systemd[1]: Stopping Ceph mon.smithi071 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T03:00:18.227 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 03:00:17 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T03:00:17.914+0000 7fc17d174700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi071 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T03:00:18.227 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 03:00:17 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071[144521]: 2024-04-23T03:00:17.914+0000 7fc17d174700 -1 mon.smithi071@0(leader) e2 *** Got Signal Terminated *** 2024-04-23T03:00:18.492 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:Apr 23 03:00:18 smithi071 bash[209329]: ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi071 2024-04-23T03:00:18.788 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f5470c-0115-11ef-bc93-c7b262605968@mon.smithi071.service' 2024-04-23T03:00:18.835 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T03:00:18.836 INFO:tasks.cephadm.mon.smithi071:Stopped mon.smithi071 2024-04-23T03:00:18.836 INFO:tasks.cephadm.mon.smithi192:Stopping mon.smithi192... 2024-04-23T03:00:18.836 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-91f5470c-0115-11ef-bc93-c7b262605968@mon.smithi192 2024-04-23T03:00:19.125 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 03:00:18 smithi192 systemd[1]: Stopping Ceph mon.smithi192 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T03:00:19.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 03:00:19 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi192[108594]: 2024-04-23T03:00:19.214+0000 7f5d44a43700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi192 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T03:00:19.625 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 03:00:19 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi192[108594]: 2024-04-23T03:00:19.214+0000 7f5d44a43700 -1 mon.smithi192@1(peon) e2 *** Got Signal Terminated *** 2024-04-23T03:00:20.124 INFO:journalctl@ceph.mon.smithi192.smithi192.stdout:Apr 23 03:00:19 smithi192 bash[156947]: ceph-91f5470c-0115-11ef-bc93-c7b262605968-mon-smithi192 2024-04-23T03:00:20.328 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f5470c-0115-11ef-bc93-c7b262605968@mon.smithi192.service' 2024-04-23T03:00:20.371 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T03:00:20.371 INFO:tasks.cephadm.mon.smithi192:Stopped mon.smithi192 2024-04-23T03:00:20.372 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-23T03:00:20.372 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.0 2024-04-23T03:00:20.696 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 03:00:20 smithi071 systemd[1]: Stopping Ceph osd.0 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T03:00:20.991 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 03:00:20 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-0[156416]: 2024-04-23T03:00:20.692+0000 7f73bf597700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T03:00:20.992 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 03:00:20 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-0[156416]: 2024-04-23T03:00:20.692+0000 7f73bf597700 -1 osd.0 84 *** Got signal Terminated *** 2024-04-23T03:00:20.992 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 03:00:20 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-0[156416]: 2024-04-23T03:00:20.692+0000 7f73bf597700 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T03:00:26.742 INFO:journalctl@ceph.osd.0.smithi071.stdout:Apr 23 03:00:26 smithi071 bash[209490]: ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-0 2024-04-23T03:00:28.289 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.0.service' 2024-04-23T03:00:28.331 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T03:00:28.331 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-23T03:00:28.332 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-23T03:00:28.332 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.1 2024-04-23T03:00:28.622 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 03:00:28 smithi071 systemd[1]: Stopping Ceph osd.1 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T03:00:28.992 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 03:00:28 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-1[163250]: 2024-04-23T03:00:28.618+0000 7f24fea88700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T03:00:28.992 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 03:00:28 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-1[163250]: 2024-04-23T03:00:28.618+0000 7f24fea88700 -1 osd.1 84 *** Got signal Terminated *** 2024-04-23T03:00:28.992 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 03:00:28 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-1[163250]: 2024-04-23T03:00:28.618+0000 7f24fea88700 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T03:00:34.742 INFO:journalctl@ceph.osd.1.smithi071.stdout:Apr 23 03:00:34 smithi071 bash[209766]: ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-1 2024-04-23T03:00:36.033 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.1.service' 2024-04-23T03:00:36.073 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T03:00:36.073 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-23T03:00:36.073 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-23T03:00:36.073 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.2 2024-04-23T03:00:36.242 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 03:00:36 smithi071 systemd[1]: Stopping Ceph osd.2 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T03:00:36.742 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 03:00:36 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-2[170169]: 2024-04-23T03:00:36.371+0000 7f1487f44700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T03:00:36.742 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 03:00:36 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-2[170169]: 2024-04-23T03:00:36.371+0000 7f1487f44700 -1 osd.2 84 *** Got signal Terminated *** 2024-04-23T03:00:36.742 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 03:00:36 smithi071 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-2[170169]: 2024-04-23T03:00:36.371+0000 7f1487f44700 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T03:00:42.491 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 23 03:00:42 smithi071 bash[210035]: ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-2 2024-04-23T03:00:44.027 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.2.service' 2024-04-23T03:00:44.072 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T03:00:44.073 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-23T03:00:44.073 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-23T03:00:44.073 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.3 2024-04-23T03:00:44.360 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 03:00:44 smithi192 systemd[1]: Stopping Ceph osd.3 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T03:00:44.624 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 03:00:44 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-3[119789]: 2024-04-23T03:00:44.358+0000 7f57849a2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T03:00:44.624 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 03:00:44 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-3[119789]: 2024-04-23T03:00:44.358+0000 7f57849a2700 -1 osd.3 84 *** Got signal Terminated *** 2024-04-23T03:00:44.625 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 03:00:44 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-3[119789]: 2024-04-23T03:00:44.358+0000 7f57849a2700 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T03:00:48.374 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:00:47 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:00:47.886+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.737040+0000 front 2024-04-23T03:00:22.736948+0000 (oldest deadline 2024-04-23T03:00:47.436948+0000) 2024-04-23T03:00:49.125 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 03:00:48 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-3[119789]: 2024-04-23T03:00:48.739+0000 7f577d791700 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.150670+0000 front 2024-04-23T03:00:22.150655+0000 (oldest deadline 2024-04-23T03:00:48.050662+0000) 2024-04-23T03:00:49.125 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:00:48 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:00:48.854+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.737040+0000 front 2024-04-23T03:00:22.736948+0000 (oldest deadline 2024-04-23T03:00:47.436948+0000) 2024-04-23T03:00:50.125 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 03:00:49 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[125971]: 2024-04-23T03:00:49.991+0000 7fa24c5e2700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:24.962015+0000 front 2024-04-23T03:00:24.962030+0000 (oldest deadline 2024-04-23T03:00:49.061929+0000) 2024-04-23T03:00:50.125 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:00:49 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:00:49.833+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.737040+0000 front 2024-04-23T03:00:22.736948+0000 (oldest deadline 2024-04-23T03:00:47.436948+0000) 2024-04-23T03:00:50.625 INFO:journalctl@ceph.osd.3.smithi192.stdout:Apr 23 03:00:50 smithi192 bash[157104]: ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-3 2024-04-23T03:00:51.074 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 03:00:50 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[125971]: 2024-04-23T03:00:50.983+0000 7fa24c5e2700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:24.962015+0000 front 2024-04-23T03:00:24.962030+0000 (oldest deadline 2024-04-23T03:00:49.061929+0000) 2024-04-23T03:00:51.074 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:00:50 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:00:50.819+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.737040+0000 front 2024-04-23T03:00:22.736948+0000 (oldest deadline 2024-04-23T03:00:47.436948+0000) 2024-04-23T03:00:52.031 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.3.service' 2024-04-23T03:00:52.071 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 03:00:51 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[125971]: 2024-04-23T03:00:51.935+0000 7fa24c5e2700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:24.962015+0000 front 2024-04-23T03:00:24.962030+0000 (oldest deadline 2024-04-23T03:00:49.061929+0000) 2024-04-23T03:00:52.071 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:00:51 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:00:51.815+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.737040+0000 front 2024-04-23T03:00:22.736948+0000 (oldest deadline 2024-04-23T03:00:47.436948+0000) 2024-04-23T03:00:52.072 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T03:00:52.073 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-23T03:00:52.073 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-23T03:00:52.073 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.4 2024-04-23T03:00:52.374 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 03:00:52 smithi192 systemd[1]: Stopping Ceph osd.4 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T03:00:52.867 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 03:00:52 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[125971]: 2024-04-23T03:00:52.395+0000 7fa2537f3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T03:00:52.867 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 03:00:52 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[125971]: 2024-04-23T03:00:52.395+0000 7fa2537f3700 -1 osd.4 84 *** Got signal Terminated *** 2024-04-23T03:00:52.867 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 03:00:52 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[125971]: 2024-04-23T03:00:52.395+0000 7fa2537f3700 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T03:00:53.125 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 03:00:52 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[125971]: 2024-04-23T03:00:52.950+0000 7fa24c5e2700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:24.962015+0000 front 2024-04-23T03:00:24.962030+0000 (oldest deadline 2024-04-23T03:00:49.061929+0000) 2024-04-23T03:00:53.125 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:00:52 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:00:52.865+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.737040+0000 front 2024-04-23T03:00:22.736948+0000 (oldest deadline 2024-04-23T03:00:47.436948+0000) 2024-04-23T03:00:54.125 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 03:00:53 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[125971]: 2024-04-23T03:00:53.979+0000 7fa24c5e2700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:24.962015+0000 front 2024-04-23T03:00:24.962030+0000 (oldest deadline 2024-04-23T03:00:49.061929+0000) 2024-04-23T03:00:54.125 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:00:53 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:00:53.850+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.737040+0000 front 2024-04-23T03:00:22.736948+0000 (oldest deadline 2024-04-23T03:00:47.436948+0000) 2024-04-23T03:00:55.125 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 03:00:54 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[125971]: 2024-04-23T03:00:54.989+0000 7fa24c5e2700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:24.962015+0000 front 2024-04-23T03:00:24.962030+0000 (oldest deadline 2024-04-23T03:00:49.061929+0000) 2024-04-23T03:00:55.125 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 03:00:54 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[125971]: 2024-04-23T03:00:54.989+0000 7fa24c5e2700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-04-23T03:00:29.562873+0000 front 2024-04-23T03:00:29.562799+0000 (oldest deadline 2024-04-23T03:00:54.262787+0000) 2024-04-23T03:00:55.125 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:00:54 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:00:54.804+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.737040+0000 front 2024-04-23T03:00:22.736948+0000 (oldest deadline 2024-04-23T03:00:47.436948+0000) 2024-04-23T03:00:56.124 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 03:00:56 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[125971]: 2024-04-23T03:00:56.027+0000 7fa24c5e2700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:24.962015+0000 front 2024-04-23T03:00:24.962030+0000 (oldest deadline 2024-04-23T03:00:49.061929+0000) 2024-04-23T03:00:56.125 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 03:00:56 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[125971]: 2024-04-23T03:00:56.027+0000 7fa24c5e2700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-04-23T03:00:29.562873+0000 front 2024-04-23T03:00:29.562799+0000 (oldest deadline 2024-04-23T03:00:54.262787+0000) 2024-04-23T03:00:56.125 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:00:55 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:00:55.835+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.737040+0000 front 2024-04-23T03:00:22.736948+0000 (oldest deadline 2024-04-23T03:00:47.436948+0000) 2024-04-23T03:00:56.125 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:00:55 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:00:55.835+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-04-23T03:00:29.737729+0000 front 2024-04-23T03:00:29.737696+0000 (oldest deadline 2024-04-23T03:00:55.637691+0000) 2024-04-23T03:00:57.124 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:00:56 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:00:56.828+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.737040+0000 front 2024-04-23T03:00:22.736948+0000 (oldest deadline 2024-04-23T03:00:47.436948+0000) 2024-04-23T03:00:57.125 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:00:56 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:00:56.828+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-04-23T03:00:29.737729+0000 front 2024-04-23T03:00:29.737696+0000 (oldest deadline 2024-04-23T03:00:55.637691+0000) 2024-04-23T03:00:57.125 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 03:00:57 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[125971]: 2024-04-23T03:00:57.048+0000 7fa24c5e2700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:24.962015+0000 front 2024-04-23T03:00:24.962030+0000 (oldest deadline 2024-04-23T03:00:49.061929+0000) 2024-04-23T03:00:57.125 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 03:00:57 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4[125971]: 2024-04-23T03:00:57.048+0000 7fa24c5e2700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-04-23T03:00:29.562873+0000 front 2024-04-23T03:00:29.562799+0000 (oldest deadline 2024-04-23T03:00:54.262787+0000) 2024-04-23T03:00:58.167 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:00:57 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:00:57.878+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.737040+0000 front 2024-04-23T03:00:22.736948+0000 (oldest deadline 2024-04-23T03:00:47.436948+0000) 2024-04-23T03:00:58.168 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:00:57 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:00:57.878+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-04-23T03:00:29.737729+0000 front 2024-04-23T03:00:29.737696+0000 (oldest deadline 2024-04-23T03:00:55.637691+0000) 2024-04-23T03:00:58.624 INFO:journalctl@ceph.osd.4.smithi192.stdout:Apr 23 03:00:58 smithi192 bash[157375]: ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-4 2024-04-23T03:00:59.375 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:00:58 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:00:58.885+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.737040+0000 front 2024-04-23T03:00:22.736948+0000 (oldest deadline 2024-04-23T03:00:47.436948+0000) 2024-04-23T03:00:59.375 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:00:58 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:00:58.885+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-04-23T03:00:29.737729+0000 front 2024-04-23T03:00:29.737696+0000 (oldest deadline 2024-04-23T03:00:55.637691+0000) 2024-04-23T03:01:00.002 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:00:59 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:00:59.924+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.737040+0000 front 2024-04-23T03:00:22.736948+0000 (oldest deadline 2024-04-23T03:00:47.436948+0000) 2024-04-23T03:01:00.002 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:00:59 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:00:59.924+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-04-23T03:00:29.737729+0000 front 2024-04-23T03:00:29.737696+0000 (oldest deadline 2024-04-23T03:00:55.637691+0000) 2024-04-23T03:01:00.009 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.4.service' 2024-04-23T03:01:00.049 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T03:01:00.050 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-23T03:01:00.050 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-23T03:01:00.050 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.5 2024-04-23T03:01:00.367 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:01:00 smithi192 systemd[1]: Stopping Ceph osd.5 for 91f5470c-0115-11ef-bc93-c7b262605968... 2024-04-23T03:01:00.368 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:01:00 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:01:00.365+0000 7f0fdaf38700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T03:01:00.624 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:01:00 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:01:00.365+0000 7f0fdaf38700 -1 osd.5 84 *** Got signal Terminated *** 2024-04-23T03:01:00.625 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:01:00 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:01:00.365+0000 7f0fdaf38700 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T03:01:01.374 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:01:00 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:01:00.915+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.737040+0000 front 2024-04-23T03:00:22.736948+0000 (oldest deadline 2024-04-23T03:00:47.436948+0000) 2024-04-23T03:01:01.375 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:01:00 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:01:00.915+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-04-23T03:00:29.737729+0000 front 2024-04-23T03:00:29.737696+0000 (oldest deadline 2024-04-23T03:00:55.637691+0000) 2024-04-23T03:01:02.124 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:01:01 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:01:01.887+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.737040+0000 front 2024-04-23T03:00:22.736948+0000 (oldest deadline 2024-04-23T03:00:47.436948+0000) 2024-04-23T03:01:02.125 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:01:01 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:01:01.887+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-04-23T03:00:29.737729+0000 front 2024-04-23T03:00:29.737696+0000 (oldest deadline 2024-04-23T03:00:55.637691+0000) 2024-04-23T03:01:03.124 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:01:02 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:01:02.858+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.737040+0000 front 2024-04-23T03:00:22.736948+0000 (oldest deadline 2024-04-23T03:00:47.436948+0000) 2024-04-23T03:01:03.125 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:01:02 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:01:02.858+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-04-23T03:00:29.737729+0000 front 2024-04-23T03:00:29.737696+0000 (oldest deadline 2024-04-23T03:00:55.637691+0000) 2024-04-23T03:01:04.124 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:01:03 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:01:03.845+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.737040+0000 front 2024-04-23T03:00:22.736948+0000 (oldest deadline 2024-04-23T03:00:47.436948+0000) 2024-04-23T03:01:04.125 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:01:03 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:01:03.845+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-04-23T03:00:29.737729+0000 front 2024-04-23T03:00:29.737696+0000 (oldest deadline 2024-04-23T03:00:55.637691+0000) 2024-04-23T03:01:05.124 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:01:04 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:01:04.836+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-04-23T03:00:22.737040+0000 front 2024-04-23T03:00:22.736948+0000 (oldest deadline 2024-04-23T03:00:47.436948+0000) 2024-04-23T03:01:05.125 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:01:04 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:01:04.836+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6814 osd.1 since back 2024-04-23T03:00:29.737729+0000 front 2024-04-23T03:00:29.737696+0000 (oldest deadline 2024-04-23T03:00:55.637691+0000) 2024-04-23T03:01:05.125 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:01:04 smithi192 ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5[132937]: 2024-04-23T03:01:04.836+0000 7f0fd7d2f700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.71:6822 osd.2 since back 2024-04-23T03:00:38.538694+0000 front 2024-04-23T03:00:38.538795+0000 (oldest deadline 2024-04-23T03:01:04.438521+0000) 2024-04-23T03:01:06.375 INFO:journalctl@ceph.osd.5.smithi192.stdout:Apr 23 03:01:06 smithi192 bash[157643]: ceph-91f5470c-0115-11ef-bc93-c7b262605968-osd-5 2024-04-23T03:01:07.829 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-91f5470c-0115-11ef-bc93-c7b262605968@osd.5.service' 2024-04-23T03:01:07.869 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T03:01:07.869 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-23T03:01:07.869 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 91f5470c-0115-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T03:01:08.049 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T03:01:11.568 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[100693]: fuse finished with error 0 and tester_r 0 2024-04-23T03:01:31.338 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 91f5470c-0115-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T03:01:31.519 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T03:01:42.992 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T03:01:43.029 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T03:01:43.061 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-23T03:01:43.062 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-22_22:08:02-orch-reef-distro-default-smithi/7669099/remote/smithi071/crash 2024-04-23T03:01:43.063 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash -- . 2024-04-23T03:01:43.099 INFO:teuthology.orchestra.run.smithi071.stderr:tar: /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T03:01:43.100 INFO:teuthology.orchestra.run.smithi071.stderr:tar: Error is not recoverable: exiting now 2024-04-23T03:01:43.101 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-22_22:08:02-orch-reef-distro-default-smithi/7669099/remote/smithi192/crash 2024-04-23T03:01:43.102 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash -- . 2024-04-23T03:01:43.128 INFO:teuthology.orchestra.run.smithi192.stderr:tar: /var/lib/ceph/91f5470c-0115-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T03:01:43.128 INFO:teuthology.orchestra.run.smithi192.stderr:tar: Error is not recoverable: exiting now 2024-04-23T03:01:43.129 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-23T03:01:43.129 DEBUG:teuthology.orchestra.run.smithi071:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-04-23T03:01:43.173 INFO:teuthology.orchestra.run.smithi071.stderr:grep: /var/log/ceph/91f5470c-0115-11ef-bc93-c7b262605968/ceph.log: No such file or directory 2024-04-23T03:01:43.175 INFO:tasks.cephadm:Compressing logs... 2024-04-23T03:01:43.175 DEBUG:teuthology.orchestra.run.smithi071:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T03:01:43.217 DEBUG:teuthology.orchestra.run.smithi192:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T03:01:43.244 INFO:teuthology.orchestra.run.smithi071.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-23T03:01:43.245 INFO:teuthology.orchestra.run.smithi192.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-23T03:03:15.927 INFO:teuthology.orchestra.run.smithi192.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-04-23T03:03:26.495 INFO:tasks.cephadm:Archiving logs... 2024-04-23T03:03:26.496 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-22_22:08:02-orch-reef-distro-default-smithi/7669099/remote/smithi071/log 2024-04-23T03:03:26.497 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T03:03:38.542 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-22_22:08:02-orch-reef-distro-default-smithi/7669099/remote/smithi192/log 2024-04-23T03:03:38.543 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T03:03:49.362 INFO:tasks.cephadm:Removing cluster... 2024-04-23T03:03:49.362 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 91f5470c-0115-11ef-bc93-c7b262605968 --force 2024-04-23T03:03:49.551 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T03:03:49.871 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 91f5470c-0115-11ef-bc93-c7b262605968 --force 2024-04-23T03:03:50.061 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: 91f5470c-0115-11ef-bc93-c7b262605968 2024-04-23T03:03:50.369 INFO:tasks.cephadm:Removing cephadm ... 2024-04-23T03:03:50.369 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T03:03:50.390 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T03:03:50.409 INFO:tasks.cephadm:Teardown complete 2024-04-23T03:03:50.409 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-23T03:03:50.420 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_f26c9c3fee57ca330501910de5a07c8769ef5dfc/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_f26c9c3fee57ca330501910de5a07c8769ef5dfc/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-23T03:03:50.421 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-23T03:03:50.421 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T03:03:50.432 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T03:03:50.528 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-23T03:03:50.540 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-23T03:03:50.541 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T03:03:50.543 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T03:03:50.558 INFO:teuthology.orchestra.run.smithi071.stderr:bash: ntpq: command not found 2024-04-23T03:03:50.559 INFO:teuthology.orchestra.run.smithi192.stderr:bash: ntpq: command not found 2024-04-23T03:03:50.563 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T03:03:50.563 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-04-23T03:03:50.563 INFO:teuthology.orchestra.run.smithi071.stdout:^- hv01.front.sepia.ceph.com 2 6 377 36 +951us[ +950us] +/- 575ms 2024-04-23T03:03:50.563 INFO:teuthology.orchestra.run.smithi071.stdout:^* hv02.front.sepia.ceph.com 3 6 377 28 -848us[ -850us] +/- 56ms 2024-04-23T03:03:50.563 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 35 +1007us[+1006us] +/- 65ms 2024-04-23T03:03:50.564 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T03:03:50.565 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T03:03:50.565 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-04-23T03:03:50.565 INFO:teuthology.orchestra.run.smithi192.stdout:^- hv01.front.sepia.ceph.com 2 7 377 100 +1004us[+1004us] +/- 574ms 2024-04-23T03:03:50.565 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv02.front.sepia.ceph.com 3 7 377 101 -848us[ -848us] +/- 55ms 2024-04-23T03:03:50.565 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 +1002us[+1002us] +/- 65ms 2024-04-23T03:03:50.565 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T03:03:50.565 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-23T03:03:50.576 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-23T03:03:50.577 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-23T03:03:50.588 DEBUG:teuthology.orchestra.run.smithi071:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T03:03:50.679 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T03:03:50.750 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-23T03:03:50.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T03:03:50.781 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-23T03:03:50.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T03:03:50.812 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-23T03:03:50.824 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-23T03:03:50.835 INFO:teuthology.task.internal:Duration was 4285.940527 seconds 2024-04-23T03:03:50.836 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-23T03:03:50.847 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-23T03:03:50.847 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T03:03:50.853 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T03:03:50.893 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T03:03:50.899 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T03:03:51.253 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-23T03:03:51.253 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2024-04-23T03:03:51.254 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T03:03:51.316 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2024-04-23T03:03:51.316 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T03:03:51.339 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-23T03:03:51.339 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T03:03:51.358 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T03:03:51.440 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-23T03:03:51.441 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T03:03:51.443 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T03:03:51.579 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-23T03:03:51.590 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-23T03:03:51.590 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T03:03:51.616 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T03:03:51.643 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-23T03:03:51.655 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T03:03:51.658 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T03:03:51.683 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2024-04-23T03:03:51.710 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2024-04-23T03:03:51.760 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T03:03:51.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T03:03:51.777 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T03:03:51.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T03:03:51.818 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-23T03:03:51.830 INFO:teuthology.task.internal:Transferring archived files... 2024-04-23T03:03:51.831 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-22_22:08:02-orch-reef-distro-default-smithi/7669099/remote/smithi071 2024-04-23T03:03:51.831 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T03:03:51.879 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-22_22:08:02-orch-reef-distro-default-smithi/7669099/remote/smithi192 2024-04-23T03:03:51.879 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T03:03:51.922 INFO:teuthology.task.internal:Removing archive directory... 2024-04-23T03:03:51.922 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T03:03:51.925 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T03:03:51.973 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-23T03:03:51.987 INFO:teuthology.task.internal:Not uploading archives. 2024-04-23T03:03:51.987 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-23T03:03:51.998 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-23T03:03:51.998 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T03:03:52.001 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T03:03:52.016 INFO:teuthology.orchestra.run.smithi071.stdout: 525634 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 23 03:03 /home/ubuntu/cephtest 2024-04-23T03:03:52.016 INFO:teuthology.orchestra.run.smithi071.stdout: 526220 4 d--------- 2 ubuntu ubuntu 4096 Apr 23 02:11 /home/ubuntu/cephtest/mnt.0 2024-04-23T03:03:52.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T03:03:52.017 INFO:teuthology.orchestra.run.smithi071.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-04-23T03:03:52.018 INFO:teuthology.orchestra.run.smithi071.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-23T03:03:52.018 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_f26c9c3fee57ca330501910de5a07c8769ef5dfc/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_f26c9c3fee57ca330501910de5a07c8769ef5dfc/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi071 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-23T03:03:52.018 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-23T03:03:52.031 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-23T03:03:52.044 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-23T03:03:52.044 INFO:teuthology.nuke:Checking targets against current locks 2024-04-23T03:03:52.068 DEBUG:teuthology.nuke:shortname: smithi071 2024-04-23T03:03:52.068 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T03:03:52.088 DEBUG:teuthology.nuke:shortname: smithi192 2024-04-23T03:03:52.088 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T03:03:52.093 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-22_22:08:02-orch-reef-distro-default-smithi/7669099', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 01:43:34.202514', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC/46O7EJx9emMkSwmYqpp1F2tvT/Tqwo/XILMv//tKBiIHFHhqysvkxSnNwn2LIN0NwWzhQD18T3OZ5K2G93DM='} 2024-04-23T03:03:52.108 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-22_22:08:02-orch-reef-distro-default-smithi/7669099', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 01:43:34.200782', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3ZX0Tutjw6eGl1DG6COb2Z2KaK2K8u3uGyFZnA0zyQWVay9mv/guBJ3hZASCoFU0u5Ucav7d8UZsBgU3rznNU='} 2024-04-23T03:03:52.140 INFO:teuthology.orchestra.console.smithi071:Power off 2024-04-23T03:03:52.141 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T03:03:52.154 INFO:teuthology.orchestra.console.smithi192:Power off 2024-04-23T03:03:52.154 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T03:03:52.163 DEBUG:teuthology.orchestra.console.smithi071:power off output: Chassis Power Control: Down/Off 2024-04-23T03:03:52.164 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T03:03:52.176 DEBUG:teuthology.orchestra.console.smithi192:power off output: Chassis Power Control: Down/Off 2024-04-23T03:03:52.176 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T03:03:52.188 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-04-23T03:03:52.199 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-04-23T03:03:56.189 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T03:03:56.200 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T03:03:56.299 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-04-23T03:03:56.308 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-04-23T03:04:00.302 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T03:04:00.311 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T03:04:00.413 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off 2024-04-23T03:04:00.420 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is off 2024-04-23T03:04:00.515 INFO:teuthology.orchestra.console.smithi071:Power off completed 2024-04-23T03:04:00.521 INFO:teuthology.orchestra.console.smithi192:Power off completed 2024-04-23T03:04:00.626 INFO:teuthology.run:Summary data: description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 4285.940527439117 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-04-23T03:04:00.626 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T03:04:00.723 INFO:teuthology.run:FAIL